Adversarial TTS

Adversarial TTS

Detector-Guided Vocoder Training
for Adversarial Text-to-Speech Synthesis

Jiayan Cui, Davide Salvi, Paolo Bestagini, Stefano Tubaro

Politecnico di Milano

Listen & compareQuick start
The same words, three signalsWoman 29
VBaseline
VRRawNet2
VRTRawNet2 + TSSD
“The wingtip”Waveforms from the original samples

Hear the difference.

Choose a voice and a prompt.
Switch vocoders at the same playback position.

Loading the audio collection…

6 voices × 3 prompts × 3 vocoders

Headphones recommended. Original recordings; no processing during playback.

From words
to waveforms.

The acoustic model predicts the speech. The vocoder shapes the signal.

  1. 01

    Text + reference

    English or Mandarin,
    with voice conditioning.

  2. 02

    AdaSpeech

    A predicted postnet mel,
    80 frequency bands.

  3. 03

    BigVGAN

    V, VR, or VRT.
    A waveform at 22.05 kHz.

V

Baseline

Speech synthesis without a detector-specific adversarial objective.

No detector target
VR

One detector

Adversarial fine-tuning against RawNet2 in a white-box setting.

RawNet2
VRT

Two detectors

Adversarial fine-tuning against RawNet2 and TSSD together.

RawNet2 + TSSD

AASIST is held out of vocoder fine-tuning to evaluate transfer. Listening examples illustrate the methods; detector performance is evaluated separately in the study.

Models and configuration.

Matching acoustic weights, three vocoders,
and explicit inference configuration.

Model checkpoints

Acoustic + V / VR / VRT

Download the AdaSpeech acoustic model, V / VR / VRT vocoders, and matching configurations from Hugging Face. The package includes pronunciation dictionaries and a ready-to-use synthesis bundle.

Download modelsModel guide
Acoustic model
AdaSpeech · reference-conditioned
Vocoder
BigVGAN · V, VR, VRT
Interface
22.05 kHz · 80-band mel · hop 256
Voice conditioning
512-D embedding + reference mel or audio

Make your first
waveform.

Install the project, provide matching model assets and reference conditioning, then select a vocoder.

Python 3.10+
Acoustic: CPU or CUDA
Vocoder: CPU, CUDA, or explicit Apple MPS

Setup guide
Text → mel → WAVWith your configured assets
# Install once
python -m pip install -e ".[acoustic,vocoder]" huggingface_hub

# Download checkpoints
hf download Cocii/adversarial-tts --local-dir checkpoints

# Select V, VR, or VRT
python inference.py \
  --config checkpoints/tts.yaml \
  --text "Hello, this is a speech synthesis example." \
  --reference-embedding /path/to/speaker.npy \
  --reference-mel /path/to/reference-mel.npy \
  --vocoder VRT --output-dir outputs/example

Requires an existing model bundle, speaker embedding, and reference mel. Models are not downloaded automatically.

Code, context, and credits.

Acknowledgements

We thank Microsoft’s AdaSpeech for the acoustic-model foundation and NVIDIA BigVGAN for waveform synthesis, along with the authors of HiFi-GAN, Snake/alias-free activations, Tacotron utilities, RawNet2, TSSD, and AASIST. Original attribution and applicable license notices are retained in the source repository.

About the recordings

The collection includes four training-set speakers and two unseen voices. Gorou and Yae Miko are character voices from Genshin Impact by miHoYo, as identified in the original collection. Speaker references contain different text from the synthesized prompts; excerpts are reconstructed from source filenames.

The recordings illustrate the methods; detector metrics are evaluated separately in the paper. No blanket license is asserted for the recordings or model weights.

Original audio collection

Listen to the original research recordings, then explore the models and synthesis guide.