Post-Doctoral Research Visit F/M PAC-Bayesian Analysis of Dynamical Systems
Contract type : Fixed-term contract
Level of qualifications required : PhD or equivalent
Fonction : Post-Doctoral Research Visit
Context
The PostDoc position will be in the framework of the ERC Starting Grant DYNASTY (Dynamics-Aware Theory of Deep Learning).
The position might include traveling to conferences for paper presentation. Travel expenses will be covered within the limits of the scale in force.
Assignment
Deep learning shows significant empirical success in a wide range of applications. However, there is a lack of understanding on the theoretical side: it is not clear in which situations deep learning models generalize well. The PAC-Bayesian theory offers promising perspectives for this type of models [DR17; Per+21]. However, these bounds do not consider the learning algorithm, which might be key to obtaining tight generalization bounds. In order to reduce the gap between theory and practice (and obtain tight bounds), the hyper-parameters of the learning algorithm could be integrated into the bounds. More generally, integrating the dynamics of the learning processes might tighten the generalization bounds. In this context, the objective of the postdoc is (i) to develop new PAC-Bayesian bounds that take the learning process (and the dynamics) into account and (ii) to derive new learning algorithms based on the minimization of these new bounds [see e.g., Fre98]. For instance, the works of London [Lon17] and Rivasplata et al. [Riv+20] could be leveraged to obtain tight generalization bounds for models obtained through stochastic gradient descent with hyper-parameters sampled from a probability distribution.
References
[DR17] Gintare Karolina Dziugaite and Daniel M. Roy. “Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data”. In: Conference on Uncertainty in Artificial Intelligence (UAI). 2017.
[Fre98] Yoav Freund. “Self Bounding Learning Algorithms”. In: Conference on Learning Theory (COLT). 1998.
[Lon17] Ben London. “A PAC-Bayesian Analysis of Randomized Learning with Application to Stochastic Gradient Descent”. In: Advances
in Neural Information Processing System (NIPS). 2017.
[Per+21] Maria Perez-Ortiz et al. “Tighter Risk Certificates for Neural Networks”. In: Journal of Machine Learning Research (2021).
[Riv+20] Omar Rivasplata et al. “PAC-Bayes Analysis Beyond the Usual Bounds”. In: Advances in Neural Information Processing System
(NeurIPS). 2020.
Main activities
Main activities:
- Conduct theoretical research
- Conduct experiments for empirical verification
- Write scientific articles
- Disseminate the scientific work in appropriate venues.
Skills
Technical skills and level required :
Languages : High-level of professional/academic English
Coding skills : Good level of coding in Python and related deep learning libraries
Benefits package
- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking and flexible organization of working hours (after 12 months of employment)
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage
General Information
- Theme/Domain :
Optimization, machine learning and statistical methods
Statistics (Big data) (BAP E) - Town/city : Paris
- Inria Center : Centre Inria de Paris
- Starting date : 2024-10-01
- Duration of contract : 2 years
- Deadline to apply : 2024-05-18
Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.
Instruction to apply
Defence Security :
This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.
Recruitment Policy :
As part of its diversity policy, all Inria positions are accessible to people with disabilities.
Contacts
- Inria Team : SIERRA
-
Recruiter :
Simsekli Umut / umut.simsekli@inria.fr
About Inria
Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, often at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact.