- - Invited speakers - -
Model-based deep learning in signal processing and communications
Recent years have witnessed a dramatically growing interest in machine learning (ML) methods. These data-driven trainable structures have demonstrated an unprecedented empirical success in various applications, including computer vision and speech processing. The benefits of ML-driven techniques over traditional model-based approaches are twofold: First, ML methods are independent of the underlying stochastic model, and thus can operate efficiently in scenarios where this model is unknown, or its parameters cannot be accurately estimated; Second, when the underlying model is extremely complex, ML algorithms have demonstrated the ability to extract and disentangle the meaningful semantic information from the observed data. Nonetheless, not every problem can and should be solved using deep neural networks (DNNs). In fact, in scenarios for which model-based algorithms exist and are computationally feasible, these analytical methods are typically preferable over ML schemes due to their theoretical performance guarantees and possible proven optimality. Notable application areas where model-based schemes are typically preferable, and whose characteristics are fundamentally different from conventional deep learning applications, include signal processing and digital communications. In this talk, I will present methods for combining DNNs with traditional model-based algorithms. We will show how hybrid model-based/data-driven implementations arise from classical methods in signal processing, compressed sensing, control, and digital communications, and show how fundamental classic techniques can be implemented without knowledge of the underlying statistical model, while achieving improved robustness to uncertainty.
Title: Information Bottleneck through variational glasses
The Information Bottleneck (IB) principle has become an important element in information-theoretic analyses of deep models. Many state-of-the-art generative models of both Variational Autoencoder (VAE) and Generative Adversarial Networks (GAN) families use various bounds on mutual information terms to introduce certain regularisation constraints. Accordingly, the main difference between these models consists in added regularisation constraints and targeted objectives. However, it is not always obvious what the underlying assumptions behind these constraints are. In this talk, we will consider the IB framework for several applications covering supervised and semi-supervised classification, generative models based in VAE and GAN families and regression problems. We will will show how applying a variational decomposition to mutual information leads to a common structure and allows us to easily establish connections between these models and to analyze underlying assumptions. We will show the advantages of these models in semi-supervised classification and demonstrate some interesting connections to existing generative models such as VAE, β−VAE, AAE, InfoVAE and VAE/GAN. We show that many known methods can be considered as a product of variational decomposition of mutual information terms in the IB framework. Finally, we extend the same framework to anomaly detection problems and several classes of regression problems.
Slava served as Associate Editor for IEEE TIFS (2013-2015). He was an elected member of the IEEE Information Forensics and Security Technical Committee (2011-2013) where he was an area chair in information-theoretic security and an associated member since 2015. He served as a guest editor to several special issues dedicated to information theory and security. He is a member of Eurasip BForSec SAT. He has served as a consultant to private industry and co-founded three companies. He leads several Swiss projects in Big Data acquisition, processing and analysis related to solar imaging and collaborates with CERN on machine learning in physics. S. Voloshynovskiy co-organized several interdisciplinary events between AI and physics. He was a recipient of the Swiss National Science Foundation Professorship Grant in 2003.