Synthesizing speech

Overview

Coqui TTS provides three main methods for inference:

  1. 🐍Python API

  2. TTS command line interface (CLI)

  3. Local demo server

🐍 Python API

Multi-speaker and multi-lingual model

import torch
from TTS.api import TTS

# Get device
device = "cuda" if torch.cuda.is_available() else "cpu"

# List available 🐸TTS models
print(TTS().list_models())

# Initialize TTS
tts = TTS("tts_models/multilingual/multi-dataset/xtts_v2").to(device)

# List speakers
print(tts.speakers)

# Run TTS
# ❗ XTTS supports both, but many models allow only one of the `speaker` and
# `speaker_wav` arguments

# TTS with list of amplitude values as output, clone the voice from `speaker_wav`
wav = tts.tts(
  text="Hello world!",
  speaker_wav="my/cloning/audio.wav",
  language="en"
)

# TTS to a file, use a preset speaker
tts.tts_to_file(
  text="Hello world!",
  speaker="Craig Gutsy",
  language="en",
  file_path="output.wav"
)

From version 0.27.0 you can cache cloned voices with a custom speaker ID, so you only need to pass audio files in speaker_wav once.

Note

For more control or additional outputs, e.g. timestamps, use the lower-level Synthesizer API.

Single speaker model

# Initialize TTS with the target model name
tts = TTS("tts_models/de/thorsten/tacotron2-DDC").to(device)

# Run TTS
tts.tts_to_file(text="Ich bin eine Testnachricht.", file_path=OUTPUT_PATH)

Voice conversion (VC)

Converting the voice in source_wav to the voice of target_wav:

tts = TTS("voice_conversion_models/multilingual/vctk/freevc24").to("cuda")
tts.voice_conversion_to_file(
  source_wav="my/source.wav",
  target_wav="my/target.wav",
  file_path="output.wav"
)

Other available voice conversion models:

  • voice_conversion_models/multilingual/multi-dataset/knnvc

  • voice_conversion_models/multilingual/multi-dataset/openvoice_v1

  • voice_conversion_models/multilingual/multi-dataset/openvoice_v2

For more details, see this dedicated page.

Voice cloning by combining single speaker TTS model with the default VC model

This way, you can clone voices by using any model in 🐸TTS. The FreeVC model is used for voice conversion after synthesizing speech.


tts = TTS("tts_models/de/thorsten/tacotron2-DDC")
tts.tts_with_vc_to_file(
    "Wie sage ich auf Italienisch, dass ich dich liebe?",
    speaker_wav="target/speaker.wav",
    file_path="output.wav"
)

TTS using Fairseq models in ~1100 languages 🤯

For Fairseq models, use the following name format: tts_models/<lang-iso_code>/fairseq/vits. You can find the language ISO codes here and learn about the Fairseq models here.

# TTS with fairseq models
api = TTS("tts_models/deu/fairseq/vits")
api.tts_to_file(
    "Wie sage ich auf Italienisch, dass ich dich liebe?",
    file_path="output.wav"
)

Note: Some Fairseq models need the romanization library uroman to be installed. For this you can install coqui-tts with the languages extra.

Command-line interface tts

Synthesize speech on the command line.

You can either use your trained model or choose a model from the provided list.

  • List provided models:

    tts --list_models

  • Get model information. Use the names obtained from --list_models.

    tts --model_info_by_name "<model_type>/<language>/<dataset>/<model_name>"

    For example:

    tts --model_info_by_name tts_models/tr/common-voice/glow-tts
tts --model_info_by_name vocoder_models/en/ljspeech/hifigan_v2

Single speaker models

  • Run TTS with the default model (tts_models/en/ljspeech/tacotron2-DDC):

    tts --text "Text for TTS" --out_path output/path/speech.wav

  • Run TTS and pipe out the generated TTS wav file data:

    tts --text "Text for TTS" --pipe_out --out_path output/path/speech.wav | aplay

  • Run a TTS model with its default vocoder model:

    tts --text "Text for TTS" \
    --model_name "<model_type>/<language>/<dataset>/<model_name>" \
    --out_path output/path/speech.wav

    For example:

    tts --text "Text for TTS" \
    --model_name "tts_models/en/ljspeech/glow-tts" \
    --out_path output/path/speech.wav

  • Run with specific TTS and vocoder models from the list. Note that not every vocoder is compatible with every TTS model.

    tts --text "Text for TTS" \
    --model_name "<model_type>/<language>/<dataset>/<model_name>" \
    --vocoder_name "<model_type>/<language>/<dataset>/<model_name>" \
    --out_path output/path/speech.wav

    For example:

    tts --text "Text for TTS" \
    --model_name "tts_models/en/ljspeech/glow-tts" \
    --vocoder_name "vocoder_models/en/ljspeech/univnet" \
    --out_path output/path/speech.wav

  • Run your own TTS model (using Griffin-Lim Vocoder):

    tts --text "Text for TTS" \
    --model_path path/to/model.pth \
    --config_path path/to/config.json \
    --out_path output/path/speech.wav

  • Run your own TTS and Vocoder models:

    tts --text "Text for TTS" \
    --model_path path/to/model.pth \
    --config_path path/to/config.json \
    --out_path output/path/speech.wav \
    --vocoder_path path/to/vocoder.pth \
    --vocoder_config_path path/to/vocoder_config.json

Multi-speaker models

  • List the available speakers and choose a <speaker_id> among them:

    tts --model_name "<language>/<dataset>/<model_name>"  --list_speaker_idxs

  • Run the multi-speaker TTS model with the target speaker ID:

    tts --text "Text for TTS." --out_path output/path/speech.wav \
    --model_name "<language>/<dataset>/<model_name>"  --speaker_idx <speaker_id>

  • Run your own multi-speaker TTS model:

    tts --text "Text for TTS" --out_path output/path/speech.wav \
    --model_path path/to/model.pth --config_path path/to/config.json \
    --speakers_file_path path/to/speaker.json --speaker_idx <speaker_id>

Voice conversion models

tts --out_path output/path/speech.wav --model_name "<language>/<dataset>/<model_name>" \
    --source_wav <path/to/speaker/wav> --target_wav <path/to/reference/wav>