Esteban Gutiérrez1 and Lonce Wyse1
1 Department of Information and Communications Technologies, Universitat Pompeu Fabra
This repository contains the experiments conducted for the thesis titled "Statistics-Driven Texture Sound Synthesis Using Differentiable Digital Signal Processing-Based Architectures" authored by Esteban Gutiérrez and advised by Lonce Wyse at the Universitat Pompeu Fabra. The experiments are based on the GitHub repositories resulting from this thesis: ddsp_textures and VAE_SubEnv.
The thesis explores adapting Differentiable Digital Signal Processing (DDSP) architectures, first introduced by Engel et a
7914
l. in [1], for synthesizing and controlling texture sounds, which are complex and noisy compared to traditional pitched instrument timbres. It introduces two innovative synthesizers: the
The experiments folder contains several Jupyter notebooks detailing the experiments conducted for this thesis. Below is a description of each subfolder:
This section introduces two differentiable signal processors explored in this thesis. Details and experiments related to these processors can be found in the corresponding Jupyter notebooks.
In this section, a new loss function is introduced and preliminary research is conducted to assess its effectiveness. The relevant notebooks provide insights into the performance and efficiency of this loss function.
This subfolder contains Jupyter notebooks on models based on Differentiable Digital Signal Processing (DDSP) using the previously defined signal processors and loss function. These notebooks cover the training, exploration, and evaluation of these DDSP-based models.
Here, you’ll find notebooks detailing the training and evaluation of new model variants derived from the ones introduced in the thesis. These models are examined for their performance and effectiveness in the experiments.
[1] J. Engel, L. Hantrakul, C. Gu, and A. Roberts, “Ddsp: Differentiable digital signal processing,” in International Conference on Learning Representations, 2020.
[2] N. Saint-Arnaud, “Classification of Sound Textures,” Master’s thesis, Massachusetts Institute of Technology, Cambridge, MA, 1995.
[3] J. H. McDermott and E. P. Simoncelli, “Sound texture perception via statistics of the auditory periphery: evidence from sound synthesis,” Neuron, vol. 71, pp. 926–940, 2011.\