- Differentiable Digital Signal Processing (DDSP) — Engel et al., ICLR 2020
- Technique using gradient descent to optimize digital signal parameters
Semantifx
Tuning audio effects with natural language
Motivation and Goal
- Digital music production reinvents itself every twenty years
- Generative music seems to promise the next shift, I'm skeptical
-
What matters in new music tech:
- Interpretability
- Dimension of the latent space
Background
- Contrastive Language-Audio Pretraining (CLAP) — Elizalde et al., ICASSP 2024
- Deep Learning model that allows text and audio to be directly compared
- Think: How close is this audio to this text?
- Text2FX — Chu et al., ICASSP 2025
- Algorithm that uses CLAP to bring audio closer to a text prompt
- Uses DDSP to achieve this
Can natural-language input for audio effects support musical creativity while keeping musicians in control of every change?
The words you and I might use every day to describe sound.
“sharp” · “loud” · “distorted”
Digital tools that alter aspects of sound.
Equalizer → frequency · Delay → time
Here: musicians using Digital Audio Workstations (DAWs).
The exact settings of the audio effects and their parameters.
What is Semantifx
- An audio plugin in your DAW
- Audio + prompt in, settings out
- Every change stays editable
Building Semantifx
Results
Remaining Work
- Listening study with musicians
- More effect types
- Letting it build the chain