Master Thesis »real-time Piano Multipitch
vor 2 Monaten
The Fraunhofer Institute for Digital Media Technology IDMT is part of the Fraunhofer-Gesellschaft. Headquartered in Ilmenau, Germany, the institute is internationally recognized for its expertise in applied electroacoustics and audio engineering, AI-based signal analysis and machine learning, and data privacy and security. At the headquarters, on the campus of “Technische Universität Ilmenau” researchers work on technologies for robust, trustworthy AI-based analysis and classification of audio and video data. These are used, among other things, to monitor industrial production processes, but also in traffic monitoring or in the media context, for example when it comes to automatic metadata extraction and audio manipulation detection. Another focus is the development of algorithms for the areas of virtual product development, intelligent actuator-sensor systems and audio for the automotive sector. There are currently around 70 employees working at Fraunhofer IDMT in Ilmenau.
**What you will do**
Automatic music transcription (AMT) aims to extract score-like representations from audio recordings [1, 2]. It remains one of the most challenging tasks in Music Information Retrieval (MIR). AMT algorithms can be categorized according to the instrument being transcribed and its role within a musical ensemble. A distinction can be made between the transcription of percussive instruments such as drums, and harmonic instruments that play either the melody part, the bass, or the polyphonic accompaniment. In the latter case, multipitch estimation (MPE) aims to detect all active pitches played by an instrument such as a piano or guitar at the frame-level.
In this thesis, a recently proposed algorithm [3] for real-time MPE of piano recordings will be studied which uses a convolutional neural network (CNN) architecture to predict both the active pitches played on a piano and their velocity (loudness) values. The author demonstrates that it outperforms other recently proposed MPE models based on the transformer architecture such as [7].
In particular, in this Master's Thesis, the following objectives should be accomplished:
(1) Conduct a state-of-the-art research on MPE algorithms with a special focus on deep learning based methods. The ability to be run in real-time should be of particular importance. Furthermore, the student should become familiar with the Maestro v3 [4], MAPS [5], and SMD-synth [6] datasets and their annotation schemes.
(2) Re-implement the MPE model proposed in [3] and re-evaluate the results.
(3) Optimize the model architecture for real-time use by further reducing its memory footprint and temporal latency.
(4) In a more extensive evaluation investigate the robustness of the MPE model towards other pianos using cross-dataset evaluation and using the MAPS and SMD-synth datasets. Optionally, the influence of reverberation and background noises in the test set recordings shall be simulated to study their effect on MPE performance.
The student should document their work in a written thesis.
References:
[1] G. E. Poliner, D. P. W. Ellis, A. F. Ehmann, E. Gomez, S. Streich and B. Ong, "Melody Transcription From Music Audio: Approaches and Evaluation," in IEEE Transactions on Audio, Speech, and Language Processing, vol. 15, no. 4, pp. 1247-1256, May 2007, doi: 10.1109/TASL.2006.889797.
[2] E. Benetos, S. Dixon, Z. Duan and S. Ewert, "Automatic Music Transcription: An Overview," in IEEE Signal Processing Magazine, vol. 36, no. 1, pp. 20-30, Jan. 2019, doi: 10.1109/MSP.2018.2869928.
[7] C. Hawthorne, I. Simon, R. Swavely, E. Manilow, and J. H. Engel, Sequence-to-sequence piano transcription with transformers, in ISMIR Proceedings, 2021, pp. 246-253
**What you bring to the table**
Very good skills in music signal processing, machine learning, and deep learning are required, as well as a passion for music.
**What you can expect**
- exciting market-related topics with complex issues to be solved - you can be actively involved in shaping the future
- challenges at a high level - on top we offer you excellent opportunities for professional and technical trainings
- space to also implement your own ideas, such as in our quarterly open-topic idea contest
- an excellent technical infrastructure
- renowned partners and customers who work closely with you to develop the technologies of tomorrow
- a very good work-life balance thanks to flexible working hours, a co-child office, the option of digital childcare in case of daycare shortages, and the possibility of mobile working, because family comes first - we know that
- an open-minded and interested team, a tolerant and familiar atmosphere as well as regular team events
- good transport connections and proximity to the state capital Erfurt
- attractive special offers as part of Fraunhofer corporate benefits with numerous enterprise partners
- new work and diversity are not just empty buzzwords, but an integral part of our corporate culture
With its foc
-
Master Thesis »deep Learning Based Detection and
vor 1 Monat
Ilmenau, Deutschland Digital Media Technology VollzeitThe Fraunhofer Institute for Digital Media Technology IDMT is part of the Fraunhofer-Gesellschaft. Headquartered in Ilmenau, Germany, the institute is internationally recognized for its expertise in applied electroacoustics and audio engineering, AI-based signal analysis and machine learning, and data privacy and security. At the headquarters, on the campus...
-
Master Thesis »deep Learning Based Detection and
vor 1 Woche
Ilmenau, Deutschland Digital Media Technology VollzeitThe Fraunhofer Institute for Digital Media Technology IDMT is part of the Fraunhofer-Gesellschaft. Headquartered in Ilmenau, Germany, the institute is internationally recognized for its expertise in applied electroacoustics and audio engineering, AI-based signal analysis and machine learning, and data privacy and security. At the headquarters, on the campus...
-
Ilmenau, Deutschland Fraunhofer-Gesellschaft VollzeitThe Fraunhofer Institute for Digital Media Technology IDMT is part of the Fraunhofer-Gesellschaft. Headquartered in Ilmenau, Germany, the institute is internationally recognized for its expertise in applied electroacoustics and audio engineering, AI-based signal analysis and machine learning, and data privacy and security. At the headquarters, on the campus...
-
Master Thesis »music Tag Embeddings Using
Vor 3 Tagen
Ilmenau, Deutschland Fraunhofer-Gesellschaft VollzeitThe Fraunhofer Institute for Digital Media Technology IDMT is part of the Fraunhofer-Gesellschaft. Headquartered in Ilmenau, Germany, the institute is internationally recognized for its expertise in applied electroacoustics and audio engineering, AI-based signal analysis and machine learning, and data privacy and security. At the headquarters, on the campus...
-
Ilmenau, Deutschland Digital Media Technology VollzeitThe Fraunhofer Institute for Digital Media Technology IDMT is part of the Fraunhofer-Gesellschaft. Headquartered in Ilmenau, Germany, the institute is internationally recognized for its expertise in applied electroacoustics and audio engineering, AI-based signal analysis and machine learning, and data privacy and security. At the headquarters, on the campus...
-
Ilmenau, Deutschland Fraunhofer-Gesellschaft VollzeitWhat you bring to the table For this thesis topic, a solid understanding of the fundamentals of audio signal processing, machine learning, and deep learning, along with experience in using of version control systems such as Git is highly desirable. What you can expect exciting market-related topics with complex issues to...
-
Ilmenau, Deutschland Fraunhofer-Gesellschaft VollzeitWhat you bring to the table The prerequisites for this master's thesis topic are excellent skills in audio signal processing and deep learning, practical experience using Python and deep learning libraries such as TensorFlow or PyTorch, as well as a general interest in bioacoustic research topics. What you can expect ...
-
Ilmenau, Deutschland Fraunhofer-Gesellschaft VollzeitWhat you bring to the table The prerequisites for this master's thesis topic are excellent skills in audio signal processing and deep learning, practical experience using Python and deep learning libraries such as TensorFlow or PyTorch, as well as a general interest in bioacoustic research topics. What you can expect ...
-
12 Full-time Doctorate Research Fellowships
vor 2 Monaten
Ilmenau, Deutschland Die Technische Universität Ilmenau als Arbeitgeber Vollzeit**volume**: Full time **restriction**: until 30.06.2028 **salary**: EG 13 TV-L **beginning**: 01.07.2024 In the “Ilmenau School of Green Electronics (ISGE)” funded by the Carl Zeiss Foundation at the Technische Universität Ilmenau (Germany) is a vacancy for 12 Full-time doctorate research fellowships (f/m/d) from July 1st 2024.. These PhD...
-
Implementation of Mixed-signal Ultra-low-power
Vor 5 Tagen
Ilmenau, Deutschland IMMS Institut für Mikroelektronik- und Mechatronik-Systeme gemeinnützige GmbH (IMMS GmbH) VollzeitDuring your studies, you can contribute to our ongoing research projects. Join us in pushing the limits of what is technically feasible and be part of breaking new ground together. We offer a variety of challenging and practice-oriented topics as internships, bachelor’s or master’s theses or as student assistants (paid HiWi jobs). You will analyse...
-
Scientists / “wissenschaftliche Mitarbeitende”
vor 1 Woche
Ilmenau, Deutschland Die Technische Universität Ilmenau als Arbeitgeber Vollzeit**volume**: Full time **restriction**: 3 years **salary**: EG 13 TV-L **beginning**: as soon as possible In the Distributed and Operating Systems Group at the Department of Computer Science and Automation, TU Ilmenau, there are vacancies for two Scientists/“Wissenschaftliche Mitarbeitende” (f/m/d) with a focus on distributed in-network...
-
Evaluation of Algorithms for Runtime Power Modeling
vor 2 Monaten
Ilmenau, Deutschland IMMS Institut für Mikroelektronik- und Mechatronik-Systeme gemeinnützige GmbH (IMMS GmbH) VollzeitDuring your studies, you can contribute to our ongoing research projects. Join us in pushing the limits of what is technically feasible and be part of breaking new ground together. We offer a variety of challenging and practice-oriented topics as internships, bachelor’s or master’s theses or as student assistants (paid HiWi jobs). You will analyse...
-
Technology Mapping of Digital Circuits
Vor 5 Tagen
Ilmenau, Deutschland IMMS Institut für Mikroelektronik- und Mechatronik-Systeme gemeinnützige GmbH (IMMS GmbH) VollzeitDuring your studies, you can contribute to our ongoing research projects. Join us in pushing the limits of what is technically feasible and be part of breaking new ground together. We offer a variety of challenging and practice-oriented topics as internships, bachelor’s or master’s theses or as student assistants (paid HiWi jobs). You will analyse...
-
Doctoral Student: Ai-based Management of 5g/6g
vor 2 Monaten
Ilmenau, Deutschland Aivader Vollzeit**About the job** ∙Contract Type: Part-time, 20 hours/week ∙Location: Ilmenau, Thuringia ∙Start: Immediately or later **Roles and responsibilities** ∙Working on ways to automate network management of 5G/6G systems ∙Research and development of new innovative solutions for resource and network management of 5G and 6G networks ∙Practical...
-
Implementation of a Latch-based Pullup-emulator
vor 2 Monaten
Ilmenau, Deutschland IMMS Institut für Mikroelektronik- und Mechatronik-Systeme gemeinnützige GmbH (IMMS GmbH) VollzeitDuring your studies, you can contribute to our ongoing research projects. Join us in pushing the limits of what is technically feasible and be part of breaking new ground together. We offer a variety of challenging and practice-oriented topics as internships, bachelor’s or master’s theses or as student assistants (paid HiWi jobs). You will analyse...
-
Research Assistant
vor 2 Monaten
Ilmenau, Deutschland Die Technische Universität Ilmenau als Arbeitgeber Vollzeit**volume**: Full time/Part time **restriction**: 3 years **salary**: EG 13 TV-L **beginning**: as soon as possible In the department Computer Science and Automation, in the Group of Computer Architecture and Embedded Systems, at the Ilmenau University of Technology (TU Ilmenau, Germany) is a vacancy for a Research assistant/ PhD Student (f/m/d) as soon...
-
Research Assistant for The Application of Large
vor 2 Wochen
Ilmenau, Deutschland Optronics, System Technologies and Image Exploitation VollzeitThe Fraunhofer-Gesellschaft At the Fraunhofer Institute for Optronics, System Technology and Image Exploitation IOSB, we turn the latest scientific findings into technical innovations and want to make our contribution to shaping the future. We are currently looking for student support on the topic “Application of Large Language Models in the energy...
-
Ilmenau, Deutschland Fraunhofer-Gesellschaft VollzeitWhat you bring to the table You are enrolled (university / university of applied sciences) and are studying computer science or a comparable subject You have already gained initial experience in the area of machine learning or deep learning (ideally with a focus on LLM). You have...
-
Ilmenau, Deutschland Fraunhofer-Gesellschaft VollzeitWhat you bring to the table You are enrolled (university / university of applied sciences) and are studying computer science or a comparable subject You have already gained initial experience in the area of machine learning or deep learning (ideally with a focus on LLM). You have...
-
Doctoral Research Position
vor 2 Wochen
Ilmenau, Deutschland Die Technische Universität Ilmenau als Arbeitgeber Vollzeit**volume**: Full time **restriction**: 31.10.2027 **salary**: EG 13 TV-L **beginning**: 01.11.2024 In the Institute of Process Measurement and Sensor Technology at the Technische Universität Ilmenau (Germany) is a vacancy for up to 2 Doctoral research positions (f/m/d) for advanced academic training to obtain research skills and academic competences...
