Riccardo Simionato, PhD

Modeling musical devices, whether instruments or effects, acoustic or electronic, is a dynamic area of research and development characterized by significant scientific relevance, a rich history, and far-reaching artistic implications. This multidisciplinary field encompasses a diverse range of perspectives, from musical acoustics to digital signal processing. Musical devices present nonlinear and time-variant characteristics that provide unique soundings and behaviors that are challenging to replicate in digital emulation. Thanks to the increasing computational power of general-purpose computers, deep learning has emerged as an approach for modeling musical devices, particularly analog audio effects. In this thesis a step forward in this direction is taken, broadening the understanding of the use of neural networks to model musical devices that process or generate sounds, with a focus on complex devices that pose challenges for current neural modeling methods. In this research, novel neural architectures are investigated, placing great emphasis on essential aspects of modeling networks such as low latency, computational efficiency, and interactivity, all of which are crucial to achieving ’playable’ emulations. Furthermore, the thesis develops architectures, subcomponents, and methods designed for wide applicability across different musical device categories with minimal need for customization. The primary focus lies on two case studies involving nonlinear, time-variant musical devices representative of the acoustic and analog realms: the compressor, as a processor, and the acoustic piano, as a generator. Besides proposing novel neural architecture for modeling musical devices, in this thesis, the understanding of learning strategies in neural models is advanced, and the application of neural techniques for parameter-sensitive musical behavior is refined. Additionally, it compiles a variety of datasets of the musical devices being studied.