May 30

Pauline Wauquier thesis defense

Pauline defended her PhD thesis on Task driven representation learning on May 29th 2017.


Machine learning proposes numerous algorithms to solve the different tasks that can be extracted from real world prediction problems. To solve the different concerned tasks, most Machine learning algorithms somehow rely on relationships between instances. Pairwise instances relationships can be obtained by computing a distance between the vectorial representations of the instances. Considering the available vectorial representation of the data, none of the commonly used distances is ensured to be representative of the task that aims at being solved. In this work, we investigate the gain of tuning the vectorial representation of the data to the distance to more optimally solve the task. We more particularly focus on an existing graph-based algorithm for classification task. An algorithm to learn a mapping of the data in a representation space which allows an optimal graph-based classification is first introduced. By projecting the data in a representation space in which the predefined distance is representative of the task, we aim at outperforming the initial vectorial representation of the data when solving the task. A theoretical analysis of the introduced algorithm is performed to define the conditions ensuring an optimal classification. A set of empirical experiments allows us to evaluate the gain of the introduced approach and to temper the theoretical analysis.

May 29

Workshop on Decentralized Machine Learning, Optimization and Privacy

We are organizing a Workshop on Decentralized Machine Learning, Optimization and Privacy on September 11-12, 2017 at INRIA Lille.

Nov 30


New ANR Project in Magnet: Grasp (GRAph-based machine learning for linguistic Structure Prediction).

Sep 06


New ANR Project in Magnet: Pamela (Personalized and decentrAlized MachinE Learning under constrAints).

Jun 09

Argumentation Mining Workshop

About computational approaches of argumentation.

Jun 03

RSS Workshop

Workshop on Graph based learning of the RSS North-European Team : UCL-Aalto-Lille

June 16-17, 2016

Sep 22

Projet de Master : Propagation d’étiquettes structurées pour le traitement automatique des langues

Titre : Propagation d’étiquettes structurées pour le traitement automatique des langues
Equipe : MAGNET
Responsable HDR : Marc Tommasi
Encadrant : Pascal Denis

Problématique :

Le traitement automatique des langues (TAL) offre deux défis centraux aux algorithmes d’apprentissage automatique: d’une part, la très grande dimensionalité de leur espace des sorties (leur nombre est le plus souvent exponentiel en la taille de l’input), et d’autre part, le faible volume de données annotées disponibles (qui vient du coût important associé à la collecte d’annotations linguistiques). Ces deux problèmes ont mené au développement d’algorithmes d’apprentissage qui intègrent ces sorties complexes, ainsi qu’à des approches qui exploitent, en plus des données annotées, des données non étiquetées (plus largement disponibles). Parmi ces dernières approches, les approches par graphe (basées notamment sur la propagation d’étiquettes et la régularisation par variété) se sont montrées extrêmement prometteuses. Malheureusement, ces approches par graphe n’ont jusqu’à présent pas été généralisées au cas des sorties structurées. La question est en effet de déterminer comment on peut propager des étiquettes structurées (p.ex., des séquences d’étiquettes ou des arbres en dépendances) à l’intérieur d’un graphe.

Travail réaliser :

Il s’agira tout d’abord pour l’étudiant de se familiariser, par le biais de lectures et l’écriture d’une synthèse, avec la littérature sur la prédiction de structure, l’apprentissage semi-supervisé par graphe, ainsi qu’un ou plusieurs problèmes de TAL (tels que l’analyse en partie de discours, le parsing syntaxique, ou l’analyse discursive) et les approches état-de-l’art pour ces problèmes. Dans un deuxième temps, l’étudiant reproduira quelques algorithmes état-de-l’art et les éprouvera sur des benchmarks. Enfin, l’étudiant tentera de combiner approches de prédiction de structures et de propagation par graphe.

Bibliographie :

Zhu, X., & Ghahramani, Z. (2002). Learning from labeled and unlabeled data with label propagation (Technical Report CMU-CALD-02-107). Carnegie Mellon University

Belkin, M., Niyogi, P., & Sindhwani, V. (2006). Manifold regularization: a geometric framework for learning from Labeled and Unlabeled Examples. Journal of Machine Learning Research, 7, 2399–2434.

A. Subramanya, S. Petrov, and F. C. N. Pereira. “Efficient Graph-Based Semi-Supervised Learning of Structured Tagging Models”. In: EMNLP. 2010, pp. 167–176.

D. Das and S. Petrov. “Unsupervised Part-of-Speech Tagging with Bilingual Graph-Based Projections”. In: ACL. 2011, pp. 600–609.

Jul 03

A Markov random field approach to online graph label prediction

when: July 2, 2014

where: salle plénière

speaker: Mark Herbster

title: A Markov random field approach to online graph label prediction

abstract: In graph label prediction each vertex is labeled with a 0 or 1.  The aim is given a partial labeling of the graph to extend to a complete labeling.  A common approach to this prediction problem is to relax the labels to [0,1] and then minimize a quadratic*energy*  function with respect to a partial labeling and predict with minimizer.  This approach is at the core of the very successful techniques in semi-supervised learning of label propagation [ZLG03] and manifold regularization [BN04].  We instead use the unrelaxed labels with an energy function to define a probability distribution over labelings (a Markov Random Field) and then predict by marginalization.  The relative drawback of this approach is the computational complexity.  We mitigate the problem with an efficient deterministic approximation.  For our approximation we prove worst-case online mistake bounds and also show that sequentially our approximate prediction cannot differ from the true marginal prediction very often for “easy’’ problems.

Jul 03

Online Bayesian Moment Matching for Latent Dirichlet Allocation

when: may 20, 2014 from 11 am to noon

where: A00

speaker: Pascal Poupart

title: Online Bayesian Moment Matching for Latent Dirichlet Allocation

abstract: Latent Dirichlet Allocation (LDA) is one of the most important models for the unsupervised discovery of latent components such as topics, entities and relations based on unlabeled corpora. The estimation of LDA parameters is challenging since maximum likelihood training leads to a non-convex optimization problem and Bayesian learning yields a posterior that consists of a mixture of exponentially many Dirichlet components. As a result, practitioners typically resort to approximations based on Gibbs sampling, variational Bayes or expectation propagation. While these approximations often work well in practice, it is not clear how to detect convergence in Gibbs sampling and local optima may trap both variational Bayes and expectation propagation. Hence, there is a need for more robust techniques. In this talk, I will describe a Bayesian moment matching technique that performs exact inference with respect to a set of initial distributions (different from the prior). The approach starts with a partially defined prior where only the first, second and third order moments are specified. The remaining moments of the prior are selected gradually on a need basis as the data of the corpus is processed. This lazy approach allows us to select moments that ensure tractability of the computation. As a result, the algorithm scales linearly with the amount of data and performs exact Bayesian learning with respect to this incrementally specified prior. The approach will be demonstrated on a topic modeling task with social media data. In this talk, I will describe a Bayesian moment matching technique to estimate the parameters of the Latent Dirichlet Allocation (LDA) model. While Bayesian learning yields a posterior with exponentially large mixture of Dirichlets, the full posterior is rarely used in practice. A few moments (i.e, mean and variance) are often sufficient to estimate the parameters and some form of confidence. We propose a moment matching technique to compute the first and second order moments of the posterior in an online fashion. While moment matching leads to an approximation, we show that there exists a set of initial distributions (different from the prior) for which exact inference yields the same first and second order moments in the posterior. The approach compares favorably to existing parameter estimation techniques for LDA both in terms of accuracy and time on synthetic and real data from social media.

Jul 03

Spectral Bandits for Smooth Graph Functions

when: may 16, 2014 from 11 am

where: A00

Speaker: Tomáš Kocák

title: Spectral Bandits for Smooth Graph Functions

abstract: Smooth functions on graphs have wide applications in manifold and semi-supervised learning. In this paper, we study a bandit problem where the payoffs of arms are smooth on a graph. This framework is suitable for solving online learning problems that involve graphs, such as content-based recommendation. In this problem, each recommended item is a node and its expected rating is similar to its neighbors. The goal is to recommend items that have high expected ratings. We aim for the algorithms where the cumulative regret would not scale poorly with the number of nodes. In particular, we introduce the notion of an effective dimension, which is small in real-world graphs, and propose two algorithms for solving our problem that scale linearly in this dimension. Our experiments on real-world content recommendation problem show that a good estimator of user preferences for thousands of items can be learned from just tens nodes evaluations.

Older posts «