Post-Doctorant F/H Few-view efficient Gaussian avatar reconstruction

Le centre Inria de l'Université de Rennes est l'un des huit centres d’Inria et compte plus d'une trentaine d’équipes de recherche. Le centre Inria est un acteur majeur et reconnu dans le domaine des sciences numériques. Il est au cœur d'un riche écosystème de R&D et d’innovation : PME fortement innovantes, grands groupes industriels, pôles de compétitivité, acteurs de la recherche et de l’enseignement supérieur, laboratoires d'excellence, institut de recherche technologique

In close collaboration with the EMOVA.ME company (https://www.emova.me/), the VIRTUS Team from IRISA and INRIA Center at Rennes University, are seeking to improve the 3D reconstruction of avatars from a few monocular views. Traditionally, avatar reconstruction methods search to fit a template polygonal mesh (a 3D morphable model) from multiple views, and estimate lighting properties to extract material properties as 2D textures [6]. Yet these techniques present limitations (dealing with hair or beard appearance, lack of specularity, lack of precision around key features such as eyes or mouth). Recent hybrid techniques have been mixing Neural Radiance Fields estimations (NeRFs [4], Gaussian Splats[3]) with mesh-based reconstruction to significantly improve the level of realism, by overlaying NeRFs, 2D or 3D splats on the surface of template meshes [1,2,7]. Yet, such Neural Radiance Field techniques require a large collection of views to perform qualitative estimations. In situations with limited views as input, the technique needs to rely on strong priors, either by encoding avatars appearance in a latent space representation [5,7], trained on thousands of real or synthetic models, or by providing additional guidance to ensure the convergence of the Neural Field reconstruction.

The objective of this post-doc position is to contribute to the field of Gaussian Avatar reconstruction from few views. The post-doc will work closely with a research engineer hired on the project whose role is to prepare the datasets, help testing and assessing the quality of the results, optimizing the performances and integrating the results in the framework designed by the EMOVA.ME company.

The tasks of the post-doc will consist in:

• working on key qualitative limitations in the current avatar reconstruction approach, and prototyping improvements;
• improving the use of 2D/3D Gaussian Splatting techniques in the avatar reconstruction process and evaluating its qualitative impact, specifically on details such as eyes / ears / hair /beard, through the design of novel perceptual metrics and user studies;
• Improving separation between materiel estimation (albedo, roughness, specularity) and illumination, to enable a re-rendering of the avatar in different CG lighting conditions;
• introducing priors and additional guidance techniques to reduce the number of input views required and regularize the mesh and radiance field estimations;

The post-doc will contribute to advancing the state of the art. He will work with the team to

• interact with the Emova company to understand requirements and constraints
• design and implement new research prototypes
• publish the results in selective venues
• evaluate the quality of the generated results

• Deep Learning
• 3D reconstruction techniques
• Computer vision
• Gaussian Splatting / Gaussian Avatars

• Restauration subventionnée
• Transports publics remboursés partiellement
• Congés: 7 semaines de congés annuels + 10 jours de RTT (base temps plein) + possibilité d'autorisations d'absence exceptionnelle (ex : enfants malades, déménagement)
• Possibilité de télétravail (après 6 mois d'ancienneté) et aménagement du temps de travail
• Équipements professionnels à disposition (visioconférence, prêts de matériels informatiques, etc.)
• Prestations sociales, culturelles et sportives (Association de gestion des œuvres sociales d'Inria)
• Accès à la formation professionnelle
• Sécurité sociale