Amazing PhD students

I feel blessed with my PhD students.

Last week saw the PhD day at our lab, where all second year PhD students of the lab present their work at a small open workshop. This year, the formula was changed to cope with the number of people presenting (around 20). We had 3 minute thesis presentations for the general flavor of the work followed by poster sessions for the technical details.

Marie-Morgane won the best oral presentation award for her amazing 3 minute presentation (she was the clear winner here). Pierre Jacob won the best poster presentation for his excellent explanations. I am very happy that both of my second year students won, because they put so much effort in their work that it has to be rewarded. Especially in the current context where it is so difficult to publish at major conferences. Last year, Diogo Luvizon won the oral presentation award and three years ago, Jérôme Fellus also won the oral presentation award.

I feel very lucky to work with these talented people.

Optimizing deep learning using Gossip

We recently published a paper entitled Distributed optimization for deep learning with gossip exchange with M. Blot, N. Thome and M. Cord. This work is about distributed optimization for deep neural networks in an asynchronous and decentralized setup. We tackle the case where you have several computing resources (e.g., GPU) and you want to train a single deep learning model. We propose an optimization procedure based on gossip, where each computing optimizes a local model and sometimes exchange their weights with a random neighbor. There are several key aspect to this research.

First, we show that the gossiping strategy is in expectation equivalent to performing a stochastic gradient descent with mini-batches of a size equivalent to the aggregation of all nodes. This means that you can optimize big models which are notoriously hard to train without a bigger batch size (I'm looking at you resnet) on a collection of small GPU, rather than having to buy a larger and much more expensive one.

Second, we also show that the gossip mechanism perform some sort of stochastic exploration that in my opinion is similar to dropout, but on entire models. In short, it is a way to train an ensemble and getting the aggregate of this ensemble thanks to the consensus optimization.

There are many interesting work to be done on this topic, mostly theoretical work, that I am very looking forward to in the future.

ECCV 2018

I'm attending ECCV 2018 right now! My PhD student Marie-Morgane is presenting her accepted paper "Image Reassembly Combining Deep Learning and Shortest Path Problem" at the poster session of Tuesday morning. We propose a new image reassembly task from unordered fragments and have an associated dataset. Send an email if you want to test your skills on this challenging task.

2017-2018 in a nutshell

2017-2018 has been a very busy year and this website is completely outdated, so here is the wrap up of the most important things that happened:
  • I've been on leave at LIP6, Sorbonne Université for the full year
  • I defended my Habilitation in November 2017
  • Jérôme Fellus defended his PhD on decentralized machine learning using gossip protocols in October 2017, with high quality theoretical contributions.
  • We extended the approach to deep learning with colleagues from Sorbonne Université.
  • I started another project during my leave at Sorbonne Université on cross-modal retrieval using deep embeddings. This project in collaboration with Laure Soulier and Matthieu Cord, and part of the PhDs of Remi Cadene and Micael Carvalho, led to a publication at SIGIR 2018
  • I started a new project on using machine learning and machine vision techniques to reassemble historical artifacts fragments. In this project, I supervise a new PhD candidate, Marie-Morgane Paumard.
  • I'm organizing a special session on Image processing for Cultural Heritage at ICIP 2018 next October in Athens
  • Speaking of ICIP, I have two papers accepted, one by Pierre Jacob, who started his PhD on deep learning for image retrieval with Aymeric Histace and me in spring last year, and one by Marie-Morgane.
  • I'll be at CVPR next week presenting the latest work Diogo Luvizon on 2D/3D Pose Estimation and Action Recognition using Multitask Deep Learning. Diogo started a PhD in late 2015 on action recognition in videos, supervised by Hedi Tabia and me.

LibAGML first release

Since we had a few publications on the topic of distributed machine learning (in particular a Neurocomputing paper on distributed PCA: "Asynchronous Gossip Principal Components Analysis"), let's talk a bit more about it. My Ph.D. student Jérôme Fellus has rolled out the version first version of his libagml library. This is a distributed machine learning library in C++ that relies on Gossip protocols.

The main page is here:

The way it works is dead simple: you have a mother class that corresponds to a node, and all you have to do is derive it to make your specifi local computation and aggregation procedures. All the networking, instantiation, etc, is handle by the library. Nice, isn't it?

page 1 of 7next»