2021-03921 - FPGA programming and embedded Linux

Contract type : Fixed-term contract

Renewable contract : Oui

Level of qualifications required : Graduate degree or equivalent

Other valued qualifications : Thèse

Fonction : Tempary Research Position

Level of experience : From 3 to 5 years

About the research centre or Inria department

Grenoble Rhône-Alpes Research Center groups together a few less than 650 people in 37 research teams and 8 research support departments. 

 

Staff is localized on 5 campuses in Grenoble and Lyon, in close collaboration with labs, research and higher education institutions in Grenoble and Lyon, but also with the economic players in these areas. 

 

Present in the fields of software, high-performance computing, Internet of things, image and data, but also simulation in oceanography and biology, it participates at the best level of international scientific achievements and collaborations in both Europe and the rest of the world.

  

Context

The Emeraude team is a collaboration between Citi and researchers from the GRAME group. GRAME is a "Center National de Création Musicale" (CNCM) organized into three departments: music production, transmission / mediation and research in musical data processing. The four GRAME researchers have expertise in computer science (compilation), audio DSP, digital violin making and HCI (Human Computer Interaction) in general. GRAME is the inventor of Faust (https://faust.grame.fr), a specific language for audio signal processing. Faust is mainly developed at GRAME and by a worldwide community , it is based on a compiler `` translating '' DSP specifications written in Faust into a wide range of lower level languages ​​(e.g. C, C ++, Rust, Java, WASM, LLVM bitcode, etc.).

Emeraude is participating in an ANR (FAST) project aimed in particular at low latency audio synthesis on FPGAs. This is to design an FPGA-based platform for high-level programmable ultra-low latency digital audio signal processing (DSP) with Faust. This platform (https://faust.grame.fr/syfala/) is intended to be usable for various applications ranging from sound synthesis and processing to active sound control and of a room or of an acoustics articificial sound field.

 

Technical objectives

The proposed work within FAST offers several technical challenges. They consist in managing the compilation tool chain from audio processing written in Faust to the FPGA bitstream using the Faust compiler,  Socrate' FloPoCo tool (http: //flopoco.gforge.inria .Fr), the Xilinx tools (the VivadoHSL, vivado and Vitis toolchains) as well as the embedded Linux distribution (petalinux distribution offered by Xilinx) deployed on the ARM processor of Xilinx SoCs. It is also a question of consolidating the distribution of the software. In practice, the organization of the current git repository will need to be completely revised so that the different versions of the tool are branches rather than subdirectories. The engineer will have to take charge of the various components of the tool chain which are already operating and maintain the entire chain in a coherent state according to the improvements proposed at the different levels. The engineer will also participate in the evolution of this compilation chain by proposing new extensions. In addition to this, and when the chain is stabilized, the engineer will participate in chain applications. We plan to use this new FPGA build stream to:

  • design a programmable sound processing / synthesis module for musicians,
  • create a system to actively modify the acoustics of a concert hall to make it "modular",
  • actively modify the acoustic properties of acoustic musical instruments.

In practice ...

The work will take place as a collaboration between Grame and Citi, most of the time the engineer will be based at Citi, but a regular meeting (approximately every two weeks) will take place. The position is 24 months and was obtained as part of the creation of the Emerald team, so there will be plenty of opportunities for other goals.

All this work must come in support of the doctoral student recruited on the ANR FAST who will experiment with this compilation chain. The engineer will be associated with certain publications of the doctoral student.

Assignment

Emeraude has set up, thanks to the Syfala project (https://faust.grame.fr/syfala/) a first version of a complete automatic tool chain from Faust to an FPGA for stereo Faust programs, there is many more engineering steps that must be carried out before obtaining a truly usable tool, these are:

0) Set up a new git structure (probably on github) better suited to the different versions of the compilation chain (with or without memory, with hardware or software controllers, with or without linux, with or without HLS etc.)

1) Use the available RAM to implement delay lines on the Zybo board (rather than the RAM of the FPGA block), in particular using the onboard linux. This step requires linux kernel development skills, including kernel driver development.

2) Deploy the "fixed-point" version of Syfala, i.e. the version using optimized bit widths for variables, thanks in particular to the FloPoCo operators and to the bit width analysis being implemented in the Faust compiler. This new version will improve the complexity of the FPGA designs obtained.

3) Integrate and consolidate the human-machine interface (i.e. Faust's `` controllers / sliders '') into the design of the FPGA. Currently, there are two ways to interact with the sound coming out of the card:

  • In the hardware (using the buttons)
  • In the software (on Linux embedded on ARM)

4) Integrate multi-channel audio processing (additional GPIO and ADC used)
5) Deploy the compilation chain on other Xilinx FPGA families to deal with problems with more computing power.
6) Provide effective demonstrators:

  • Digital instruments
  • Active sound control for acoustics

Main activities

The proposed work plan for the future ADT engineer is as follows:

  1. First, become familiar with the technologies used (Faust, Xilinx VivadoHLS, FloPoCo, audio signal processing, Xilinx FPGA hardware-software programming environment using the on-chip ARM processor and petalinux): between 1 and 2 months.
  2. Consolidate and optimize the basic Faust compilation flow and propose a new structure in the form of a git project that will be open to external users (recurring task)
  3. Consolidate the current version which uses integrated memory to store audio samples, validate the performance obtained and the limitations in terms of hardware complexity, number of memory accesses and latency (2 months)
  4. Propose the integration of the Faust IP with the onboard petalinux, develop a kernel device driver to be able to use the external RAM from linux and from the IP.
  5. Helping to create two interfaces, one hardware (potentiometers and buttons) and one software on the ARM, these interfaces will be used for demonstrators.
  6. Porting the build workflow to a more powerful FPGA architecture for use in dynamic acoustics adaptation.
  7. Perform a solid demo that presents the possibility of the new build stream: 4 months.
  8. Study how FloPoCo can be used to optimize the computation in the Faust IP (time remaining)

Skills

  • A good command of English
  • A solid knowledge of embedded programming, compilation and linux administration (kernel development)
  • Proficiency in the C language as well as a solid foundation in computer architecture are required.
  • Basic training in electronics and signal processing with knowledge of FPGA programming will be appreciated
  • Basic training in audio signal processing will be appreciated

Benefits package

  • Subsidized meals
  • Partial reimbursement of public transport costs
  • Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
  • Possibility of teleworking and flexible organization of working hours
  • Professional equipment available (videoconferencing, loan of computer equipment, etc.)
  • Social, cultural and sports events and activities
  • Access to vocational training
  • Social security coverage

Remuneration

Gross monthly salary (before taxes) : from 2 936 euros