A review of differentiable digital signal processing for music and speech synthesis
View/ Open
Volume
3
Pagination
1284100 - ?
Publisher
DOI
10.3389/frsip.2023.1284100
Journal
Frontiers in Signal Processing
ISSN
2673-8198
Metadata
Show full item recordAbstract
The term “differentiable digital signal processing” describes a family of techniques in which loss function gradients are backpropagated through digital signal processors, facilitating their integration into neural networks. This article surveys the literature on differentiable audio signal processing, focusing on its use in music and speech synthesis. We catalogue applications to tasks including music performance rendering, sound matching, and voice transformation, discussing the motivations for and implications of the use of this methodology. This is accompanied by an overview of digital signal processing operations that have been implemented differentiably, which is further supported by a web book containing practical advice on differentiable synthesiser programming (https://intro2ddsp.github.io/). Finally, we highlight open challenges, including optimisation pathologies, robustness to real-world conditions, and design trade-offs, and discuss directions for future research.
Authors
Hayes, B; Shier, J; Fazekas, G; McPherson, A; Saitis, CLicence information
The following license files are associated with this item: