Modal Seminar, 2017-2018 (7 sessions)

Date: June 12, 2018 at 11.00 (Plenary Room)

Affiliation: Department of Mathematics and Statistics, San José State University

Webpage: Link.

Title: The Multiple Scaled Distributions in model based clustering

Abstract: Cluster analysis aims to identify homogeneous groups of units, called clusters, within data. Model-based clustering methods consider the overall population as a mixture of groups and each component of this mixture is modeled through its conditional probability distribution. The choice of the conditional probability distribution affects the clustering results and the performance of the algorithm. Recently the generalized hyperbolic distribution (GHD) has been used because it has the advantage of being really flexible. Another challenging issue is to model data sets characterized by the presence of outliers, the p-variate contaminated Gaussian distribution (CGD) was proposed to face this issue.
Despite the advantages in the use of the GHD and CGD, both distributions are characterized by some univariate parameters, i.e. some parameters are constant in each dimension. This is limiting for real applications, for example, the proportion of outliers may be different in each dimension. To face this issue, we proposed the use of multiple scaled distributions. The GHD and CGD are Gaussian scale mixtures with univariate weights, we proposed to incorporate multi-dimensional weights via an eigendecomposition of the symmetric positive-definite scale matrix. The generalized EM-algorithm is used for parameters estimation.
In this talk I’ll illustrate the use of multiple scaled distributions to detect flexible clusters.

Slides: Link.

Date: April 17, 2018 at 11.00 (Plenary Room)

Affiliation: George Washington University and Jean d’Alembert Fellow, Université Paris-Saclay

Webpage: Link.

Title: Algorithms for Climate Informatics: Learning from spatiotemporal data with both spatial and temporal non-stationarity

Abstract: Climate Informatics is emerging as a compelling application of machine learning. This is due in part to the urgent nature of climate change, and its many remaining uncertainties (e.g. how will a changing climate affect severe storms and other extreme weather events?). Meanwhile, progress in climate informatics is made possible in part by the public availability of vast amounts of data, both simulated by large-scale physics-based models, and observed. Not only are time series at the crux of the study of climate science, but also, by definition, climate change implies non-stationarity. In addition, much of the relevant data is spatiotemporal, and also varies over location. In this talk, I will discuss our work on learning in the presence of spatial and temporal non-stationarity, and exploiting local dependencies in time and space. Along the way, I will highlight open problems in which machine learning, including deep learning methods, may prove fruitful.

Bio: Claire Monteleoni is a Jean d’Alembert Fellow at the University of Paris-Saclay, hosted by CNRS, and an Associate Professor of Computer Science at George Washington University. Previously, she was research faculty at the Center for Computational Learning Systems, at Columbia University. She did a postdoc in Computer Science and Engineering at the University of California, San Diego, and completed her PhD and Master’s in Computer Science at MIT. She holds a Bachelor’s in Earth and Planetary Sciences from Harvard. Her research focuses on machine learning algorithms and theory for learning from data streams, spatiotemporal data, raw (unlabeled) data, and private data, and applications with societal benefit. Her research on machine learning for the study of climate science received the Best Application Paper Award at NASA CIDU 2010, and helped launch the interdisciplinary field of Climate Informatics. In 2011, she co-founded the International Workshop on Climate Informatics, which turned 7 in 2017, and has attracted climate scientists and data scientists from over 19 countries and 30 U.S. states. She gave an invited tutorial on climate informatics at NIPS 2014. She recently served as a Senior Area Chair for NIPS 2017, and she is an Area Chair for ICML 2018.

Slides: Link.

Date: April 3, 2018 at 11.00 (Room A00)

Affiliation: Department of Mathematics & Statistics, McMaster University

Title: Matrix Variate Mixtures

Abstract: Over the years data has become increasingly higher dimensional, which has prompted an increased need for dimension reduction techniques. This is perhaps especially true for clustering (unsupervised classification) as well as semi-supervised and supervised classification. Although dimension reduction in the area of clustering for multivariate data has been quite thoroughly discussed in the literature, there is relatively little work in the area of three way, or matrix variate, data. This talk will give a background in clustering matrix variate data, specifically using mixtures of skewed matrix variate distributions, followed by a discussion of the mixture of matrix variate bilinear factor analyzers (MMVBFA) model used for dimension reduction in both rows and columns of higher dimensional matrix variate data. Simulated data as well as real image data will be used for illustration.

Date: March 6, 2018 at 11.00 (Plenary Room)

Affiliation: Laboratoire Paul Painlevé, University de Lille

Webpage: Link.

Title: Vitesses minimax locales pour l’estimation des mélanges finis

Abstract: Étant donnée une distribution de mélange à $m_0$ composantes, sous des conditions de régularité et d’identifiabilité, la vitesse minimax locale d’estimation d’une distribution de mélange à $m\ge m_0$ composantes est $n^{-1/\left(4\right(m-m_0)+2)}$, où $n$ est la taille d’échantillon. Ce résultat, qui corrige un résultat antérieur de J. Chen, a des conséquences pratiques que l’on dégagera. Certaines idées de preuve seront abordées et développées selon le temps disponible. Ce travail a été effectué en collaboration avec Jonas Kahn.

Slides: Link

Related paper: Link (to appear in Annals of Statistics)

Date: February 20, 2018 at 11.00 (room A00)

Affiliation: Laboratoire Hubert Curien, University Jean Monnet, St-Étienne.

Webpage: Link.

Title: When PAC-Bayesian Majority Vote Meets Transfer Learning

Abstract: Nowadays, a plenty of data are available and many applications need to make use of supervised machine learning methods able to take into account different information sources. One natural solution consists in “combining” these sources. Here we focus on a particular combination: the PAC-Bayesian weighted majority vote. PAC-Bayesian majority vote is an ensemble method where several models are assigned a specific weight. Such approaches are motivated by the idea that a careful combination can potentially compensate for the individual model’s errors and thus achieve better robustness and performance on unseen data. In statistical machine learning, the capacity of a model to generalize on a data distribution is measured through generalization bounds. In this talk, after recalling the usual PAC-Bayesian generalization bound (the PAC-Bayesian Theorem), we extend it to two transfer learning tasks: (i) Multiview learning where the objective is to take advantage of different descriptions of the data (i.e. different input spaces); (ii) Domain adaptation where the objective is to adapt a model from one source data distribution to a different, but related, target distribution.

Slides: Link

Date: January 23, 2018 at 11.00 (Plenary room)

Affiliation: Modal, Inria Lille – Nord Europe.

Webpage: Link.

Title: A Tractable Multi-Partitions Clustering

Abstract: In the framework of model-based clustering a model allowing for several latent class variables is proposed. This model assumes that the distribution of the observed data can be factorized into several independent blocks of variables, each one following its own mixture distribution. The considered mixture distribution is a latent class model (i.e. conditional independence assumption). The proposed model includes variables selection as a special case and is able to cope with the mixed-data setting. The simplicity of the model allows to estimate the repartition of the variables into blocks and the mixtures parameters simultaneously, thus avoiding the need to run EM algorithms for each possible repartition of variables into blocks. The considered model choice criteria used to determine the number of block, the number of cluster inside each block and the repartition of variables into block are the BIC and the MICL criteria for which an efficient optimisation is proposed. The performances of the model are studied on simulated and real data for which it is shown to give a rich interpretation of the dataset at hand, i.e. analysis of the repartition of the variables into blocks and analysis of the clusters produced by each block of variables.

Slides: Link

Date: Nov. 7, 2017 at 14.00

Affiliation: Modal, Inria Lille – Nord Europe.

Webpage: Link.

Title: A Representation Learning Approach for Domain Adaptation.

Abstract: We present a representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions. Our approach is directly inspired by the theory on domain adaptation suggesting that, for effective domain transfer to be achieved, predictions must be made based on features that cannot discriminate between the training (source) and test (target) domains. The approach implements this idea in the context of neural network architectures that are trained on labeled data from the source domain and unlabeled data from the target domain (no labeled target-domain data is necessary). As the training progresses, the approach promotes the emergence of features that are (i) discriminative for the main learning task on the source domain and (ii) indiscriminate with respect to the shift between the domains. We show that this adaptation behavior can be achieved in almost any feed-forward model by augmenting a gradient reversal layer. The resulting augmented architecture can be trained using standard backpropagation and stochastic gradient descent, and can thus be implemented with little effort using any of the deep learning packages. We demonstrate the success of our approach for two distinct classification problems (document sentiment analysis and image classification), where state-of-the-art domain adaptation performance on standard benchmarks is achieved. Related paper.

Slides: Link.