Our overall objective is the computerized geometric modeling of complex scenes from physical measurements. On the geometric modeling and processing pipeline, this objective corresponds to steps required for conversion from physical to effective digital representations: analysis, reconstruction and approximation. Another longer term objective is the synthesis of complex scenes. This objective is related to analysis as we assume that the main sources of data are measurements, and synthesis is assumed to be carried out from samples.
The related scientific challenges include i) being resilient to defect-laden data due to the uncertainty in the measurement processes and imperfect algorithms along the pipeline, ii) being resilient to heterogeneous data, both in type and in scale, iii) dealing with massive data, and iv) recovering or preserving the structure of complex scenes. We define the quality of a computerized representation by its i) geometric accuracy, or faithfulness to the physical scene, ii) complexity, iii) structure accuracy and control, and iv) amenability to effective processing and high level scene understanding.
Last activity report : 2017
The evolution of acquisition technologies and methods has translated in recent years in an increasing overlap of algorithms and data between computer vision, image processing and computer graphics communities. Beyond the spectacular increase of resolution through technological advances of sensors and methods for mosaicing images, the frontier is getting thinner between laser scans and photos. For example, combining laser scanners with panoramic cameras leads to massive 3D point sets with color attributes. In addition, it is now possible to generate very dense point sets not just from laser scanners but also from photogrammetry techniques when matching a precise acquisition protocol. Depth cameras are getting increasingly common, and beyond retrieving depth information we can enrich the main acquisition systems with additional hardware to measure geometric information about the sensor for improving data registration: e.g., accelerometers or GPS for geographic location, and compasses or gyrometers for orientation. Finally, complex scenes can be observed at different scales from satellite, aerial or pedestrian levels.
This evolution allows the practitioners to measure entire urban scenes at resolutions that were until now possible only at the scale of individual shapes. The related scientific challenge is however more than just dealing with massive data sets coming from increase of resolution, as complex scenes are composed of multiple objects with structural relationships. The latter relate to i) the way the individual shapes are grouped to form objects, classes of objects or hierarchies of objects, ii) to geometry when dealing with similarity, regularity, parallelism, co-planarity or symmetry (to cite a few), and more generally iii) to domain-specific semantic considerations. Beyond reconstruction and approximation, consolidation and synthesis of complex scenes require rich structural relationships.
The numerous problems arising from this evolution stimulates the growing synergy between geometry processing and image/vision communities, such that the strengths of geometry and images are increasingly combined in the form of new methodological solutions. Photo-consistent reconstruction, where geometry and images are symmetrically combined, is one approach for improving accuracy. Finally, the process of measuring the geometry of sensors (through gyrometers and accelerometers) itself requires both geometry process and image analysis for improving accuracy and robustness. Modeling urban scenes from measurements illustrates this growing synergy, and has become a central concern for a variety of applications ranging from urban planning to simulation through rendering and special effects.