Internship M2: Detailed riggable humans from multi-view video

The Inria Grenoble research center groups together almost 600 people in 27 research teams and 8 research support departments.

Staff is present on three campuses in Grenoble, in close collaboration with other research and higher education institutions (University Grenoble Alpes, CNRS, CEA, INRAE, …), but also with key economic players in the area.

Inria Grenoble is active in the fields of high-performance computing, verification and embedded systems, modeling of the environment at multiple levels, and data science and artificial intelligence. The center is a top-level scientific institute with an extensive network of international collaborations in Europe and the rest of the world.

Within the framework of a partnership (you can choose between)

• BPI transfer project Banque de France 4 years

Context

Many works nowadays provide solutions for the avatarization process, i.e. obtaining a 3D, animatable model from one or several images. This is a very hard problem as shape fidelity, animatability, fast computation time and low number of required input cameras are all desirable, but hardly realizable simultaneously. For example, obtaining plausible models form a single camera video is nowadays feasable, but often at the cost of shape quality due to the use of low-parametric models such as SMPL. Using many videos for redundancy can allow to acquire more detail, but at the expense of computation speed. And all this detail needs to be animatable, which gets more complex with the scale of detail (i.e. millimeter shape with only a human kinematic rig), again putting a burden on the model and its computation time.

Mission

In recent years the Morpheo team has come up with very precise multi-view reconstruction approaches [1].
In this master proposal, we wish to examine the problem of animating this type of detailed model and rig it, by exploring the stream of recent methods.

During his internship, the master candidate is expected to tackle the following tasks

• establish a more complete bibliography of relevant methods based on the initial suggested references
• propose and discuss likely and realizable methodological and architecture innovations / reparametrizations that allow to rig and estimate a detailed animated model from images, grounded in this existing work
• propose and discuss dataset enhancements that would enrich the training toward better performance for these tasks
• identify existing datasets that are relevant for comparative evaluation of performance of his proposals. In-house datasets such as 4DHumanOutfit[2]

• Subsidized meals
• Partial reimbursement of public transport costs
• Professional equipment available (videoconferencing, loan of computer equipment, etc.)
• Social, cultural and sports events and activities

• Minimum legal gratification