PhD: Deep learning architectures for onboard satellite image analysis

Context:

This project takes place in the general context of information extraction from massive satellite data using advanced machine learning tools. The continuous proliferation and improvement of satellite data sensors yields a huge volume of Earth’s images with high spatial, spectral and temporal resolution (up to 30cm/pixel, > 50 bands, several times per day, covering the full planet!). These data open the door to a large range of important applications, such as the monitoring of natural and anthropogenic disasters, the planning of urban environments and precision agriculture. It has been recently shown that the recent deep machine learning approaches succeed in generalizing to analyze the data in new unseen Earth areas. However, these models are typically composed of thousands or even millions of trainable weights, and are not easy to be directly apply for onboard satellite image processing.

Subject:

The goal of this PhD thesis is to devise efficient learning algorithms, which would allow to extract the useful information from satellite images onboard. The developed architectures mush be both computationally and memory efficient. We will focus on the semantic labeling task, aiming at detecting and delineating objects belonging to the classes of interest, directly from onboard satellite optical images. Examples of the targeted applications are boat segmentation from high-resolution images and lightning detection from medium-resolution images.

The PhD thesis aims at onboard real-time automated semantic labeling, which raise several problems:

Problem 1: Design of lightweight deep learning models for semantic labelling.

It has been recently shown that reasonably lightweight deep learning models achieve good classification performances. We aim to design neural network architectures, which would yield good labelling performances with the minimum number of trainable weights. Since the training stage takes a significant amount of time and memory resources, the training will be done on the ground. However, an interesting research question consists in investigating the possibility of the model finetuning during the lifetime of the satellite.

Problem 2: Choice of the input data.

Several kinds of sensors are implemented in space instruments, with different characteristics, such as spectral range and spatial resolution. We will study what is the most optimal way to input the data to the designed inference algorithms, in particular if we can directly process the sensor level images or an additional preprocessing is needed, and what are the resolutions and spectral ranges needed for semantic labelling, in particular, for labeling of small objects. We will also study the relevance of image compression strategies in this context.

Problem 3: Relevance of the prediction depending on the available data.

It is interesting to study how much data are required to produce classification maps and how much training is needed. Moreover, an estimate of the relevance of the prediction is an essential component for those who have to take detections based on the predicted maps.

A detailed experimentation validation will be conducted on large-scale satellite images acquired by the latest sensors.

Research team: TITANE, Inria Sophia-Antipolis Mediterranee

Supervisor: Yuliya Tarabalka (yuliya.tarabalka@inria.fr)

More details on this PhD offer isĀ here