Doctoral Thesis: Embedded Generative Ai for Sensor Applications

This thesis explores the deployment of generative AI models, including transformers, convolutional networks, and other architectures, across various modalities—such as natural language, vision, and audio—on resource-constrained embedded systems like microcontrollers. It addresses challenges of limited memory, computational capacity, energy efficiency, and low-latency requirements. The research investigates state-of-the-art techniques such as model compression, architecture optimization, and cross-modal co-design, while also proposing custom microcontroller enhancements, such as specialized accelerators and approximate computing techniques, to support diverse AI workloads. Through prototype implementations and real-world validation, this work aims to enable energy-efficient, scalable generative AI across a range of modalities, supporting the next generation of IoT and edge computing systems.
The research is carried out in cooperation with the University of Otto-von-Guericke Magdeburg and under the supervision of Prof. Dr. Fabian Lurz.

The tasks within the thesis will consist of:

- Research on GenAI literature and selection of tools and pipelines for embedded deployment
- Application of selected methods on available sensor data (e.g. microphone +radar + camera, etc.)
- Consolidation of selection of (embodied multi-modal) LLM model and processing pipeline
- Optimization and deployment models to embedded compute platforms (Cortex A,M55/U55, M85/U65)
- Propose accelerator architecture for selected GenAI architecture(s)
- Implementation of a demonstrator / POC on FPGA platform

The learnings out of the thesis will lead to:

- **Optimized Deployment Strategies**:Frameworks for deploying generative AI models across multiple modalities (language, vision, and audio) on resource-constrained systems
- **Energy-Efficient AI Solutions**:Techniques to improve the energy efficiency and real-time performance of embedded AI workloads
- **Hardware-Software Co-Design**:Insights into extending microcontroller architectures with lightweight accelerators or memory optimizations for generative AI
- **Advanced Model Optimization**:Enhanced methods for model compression, quantization, and sparsity tailored for ultra-constrained devices

**Your Profile**:

- A doctoral student is a research enthusiast, _
- who enjoys working in an industrial environment in combination with an Infineon partner university. _
- who appreciates open communication and the contribution of an international environment. _
- Are eligible for full-time PhD studies and have a master’s degree in Electrical Engineering, Computer Science, or a closely related field
- Already have some experience in deploying ML models on embedded hardware, such as TensorFlow Lite for - Microcontrollers or Apache TVM
- Have strong proficiency in machine learning techniques and frameworks, including TensorFlow, PyTorch, and Keras
- Have in-depth understanding of microcontroller architectures and machine learning accelerators
- Have solid expertise in digital circuit design and familiarity with common EDA tools for FPGA development, such as Vivado
- Are you enthusiastic about scientific work
- Question the status quo and like to break new groundHave good oral and written communication skills in English

**Contact**:
Britta Johansson

**#WeAreIn for driving decarbonization and digitalization.**
As a global leader in semiconductor solutions in power systems and IoT, Infineon enables game-changing solutions for green and efficient energy, clean and safe mobility, as well as smart and secure IoT. Together, we drive innovation and customer success, while caring for our people and empowering them to reach ambitious goals. Be a part of making life easier, safer and greener.
**Are you in?**

**- Power & Sensor Systems (PSS)drives leading-edge power management, sensing, and data transfer capabilities -**

**We are on a journey to create the best Infineon for everyone.**
This means we embrace diversity and inclusion and welcome everyone for who they are. At Infineon, we offer a working environment characterized by trust, openness, respect and tolerance and are committed to give all applicants and employees equal opportunities. We base our recruiting decisions on the applicant´s experience and skills.
We look forward to receiving your resume, even if you do not entirely meet all the requirements of the job posting.
Please let your recruiter know if they need to pay special attention to something in order to enable your participation in the interview process.
Click here for more information about Diversity & Inclusion at Infineon.