Internship (M/F): visual processing on an exoskeleton for contextual action prediction

Context and funding:

The position is funded by the PEPR O2R, a national French program to advance research in robotics which reunites several French laboratories in robotics, AI, and Social and Human Sciences.

In this context, the HUCEBOT team is involved in the projects AS3 and PI3, with the challenge to design AI-powered controllers for exoskeletons for improved assistance.

About the team:

The candidate will join the Human Centered Robotics team (HUCEBOT) in the Inria Center of the University of Lorraine in Nancy, France.

The team HUCEBOT develops control, learning, and interaction skills of human-centered robots, such as humanoid, mobile manipulators and exoskeletons. The team develops learning and control algorithms for teleoperated / supervised / autonomous robots, involved in complex manipulation tasks in man-made environments. It also develops prediction and control techniques for wearable exoskeletons designed to assist humans at work. The team has excellent robotics facilities, including several humanoid robots (Talos, iCub, G1), manipulators, drones, passive and active exoskeletons, wearable sensors, force plates etc. Its laboratory has a 3D printing facility and a mechatronic workshop for prototyping and maintenance.

The team consists of many research scientists, postdocs, PhD and has the support of 1 software and 1 mechatronics engineer. The team is international - English and French speaking. French is not required, although free French classes are available in the institute for non-French speakers.

About the laboratory and Nancy:

The Inria Center of the University of Lorraine, is co-located with the Loria laboratory, in the Science and Technology Campus of the University of Lorraine (Nancy, France), next to the Botanical Gardens, at 20 minutes by public transportation or bike from the Nancy train station and City Center. Several student residences and facilities are at walking distance. Nancy is a University town, with a high quality of life and a vibrant student and expat community.

The internship is about equipping an exoskeleton with cameras to retrieve contextual and environmental information to inform the exoskeleton controller. The purpose is to better estimate the current human activity, predict the future activity and its onset, recognize potentially critical situations, and predict relevant information about the objects and humans surrounding the exoskeleton – from the objects’ mass to the intention of the humans approaching the robot. The intern will work on an active exoskeleton prototype. He/she will select adequate hardware (sensors and electronics) to enable image processing embedded on the exoskeleton (if possible), as well as visual processing models to retrieve contextual information. He/she will conduct experiments with the sensorized exoskeleton to retrive contextual information and inform the robot controller.

The cameras will be mounted on one of the team exoskeletons and tested in laboratory conditions. On the software side, the cameras will have to be visible on a ROS2node. The candidate will collaborate with the team to re-use existing visual language models (VLMs) when appropriate.

The intern will be in charge of sensor integration and conducting validating experiments.

The internship is for a 3 to 6 months period (6 months ideally).

• Review state of the art in image sensing devices that can be embedded on wearables
• Selecting, testing and mounting the cameras in one of the exoskeletons of the team
• Write software to read from the cameras and streaming them
• Integrate sensor readings with VLMs, visual models to extract contextual information, object mass estimation, human intentions
• Write hardware/software report
• Collaborate with the team to integrate the cameras and the software modules in the current experiments and developments

• Technical skills:
• Very good programming skills (python, C++).
• Excellent skills with image processing and AI models.
• Love working with real robots.
• Experience with foundation models for speech and image processing
• Soft skills:
• Excellent communication skills at work, and ability to report progress
• Proactivity.
• Not afraid of challenging projects.
• Rigour and intellectual honesty
• Curiosity and desire to learn
• Practical mindset and ability to develop robust and reliable solutions
• Autonomy and organizational skills
• Love working in a multi-cultural environment
• Team player

• Subsidized meals
• Partial reimbursement of public transport costs
• Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
• Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
• Professional equipment available (videoconferencing, loan of computer equipment, etc.)
• Social, cultural and sports events and activities
• Access to vocational training
• Social security coverage

€4.35/hour