- Differentiable Digital Signal Processing (DDSP) — Engel et al., ICLR 2020
- Gradient descent can tune audio parameters toward a goal
Semantifx
Tuning audio effects with natural language
Motivation and Goal
- Digital music production reinvents itself every twenty years
- Generative music seems to promise the next shift, I'm skeptical
-
The tech that lasts is interpretable and expands what's possible
- Interpretability — you can understand and edit what it makes
- Dimension of the latent space — it lets you make what you couldn't before
Background
- Contrastive Language-Audio Pretraining (CLAP) — Elizalde et al., ICASSP 2024
- CLAP puts text and audio in one space, so they compare directly
- Think: How close is this audio to this text?
- Text2FX — Chu et al., ICASSP 2025
- Text2FX moves audio toward a text prompt
- CLAP judges the distance; DDSP descends it
Can natural-language input for audio effects support musical creativity while keeping musicians in control of every change?
The words you and I might use every day to describe sound.
“sharp” · “loud” · “distorted”
Digital tools that alter aspects of sound.
Equalizer → frequency · Delay → time
Here: musicians using Digital Audio Workstations (DAWs).
The exact settings of the audio effects and their parameters.
What is Semantifx
- Semantifx runs as an ordinary plugin in your DAW
- It turns your audio and a prompt into effect settings
- Every change stays editable
Building Semantifx
Results
The whole loop runs in a real DAW today.
Remaining Work
- A listening study will test whether musicians keep the settings
- More effect types extend the same chain
- Eventually it should build the chain, not just tune it