stephan/audio-engine-hub
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
audio-engine-hub/XTTS_IMPLEMENTATION_PLAN.md
stephan 91887ae296 feat: Add XTTS v2 support, refactor Docker/GPU infra, and improve Piper engine 
- Add XTTS v2 configuration to .env.example
- Refactor Dockerfile to multi-stage build with CUDA 12.1 support
- Update Makefile with Kokoro and XTTS test environment targets
- Refactor Piper engine (app/engines/piper.py) to use python module execution
- Add comprehensive documentation for Kokoro and XTTS plans
- Add helper scripts and patches for build process
2025-12-13 11:37:58 +01:00

1.8 KiB
Raw Permalink Blame History

Coqui XTTS v2 Implementation Plan

Objective

Implement Coqui XTTS v2 (tts_models/multilingual/multi-dataset/xtts_v2) as a new engine in AudioEngineHub.

1. Dependencies (requirements.txt)

  • Add TTS (Coqui TTS).
  • Note: This is a heavy library. We will add it to requirements.txt.
  • Potential Conflict: TTS often requires specific torch versions. We need to ensure it plays nicely with f5-tts (which also uses torch) and kokoro.

2. Configuration (app/config.py & .env)

  • XTTS_DEVICE: "cuda" or "cpu" (default: "cuda")
  • XTTS_ACCEPT_LICENSE: "true" (required to use the model)
  • XTTS_MODEL_VERSION: "v2.0.2" (or "main" for latest)

3. Engine Implementation (app/engines/xtts.py)

  • Class: XTTSEngine (inherits TTSEngineBase)
  • Init:
    • Load TTS API.
    • Download/Load model: tts_models/multilingual/multi-dataset/xtts_v2.
    • Handle license agreement.
  • Synthesize:
    • Inputs: text, speaker (voice cloning reference), language.
    • Voice Cloning: The speaker argument will be interpreted as a filename in app/asset/voices/ (or a default provided sample).
    • Language: XTTS supports 17 languages. We will map them (e.g., "en", "de", "fr").
  • List Models: Return ['xtts_v2'].
  • List Voices: Scan app/asset/voices/ for .wav files to use as reference speakers.

4. System Updates

  • app/main.py: Register xtts in ALL_ENGINES.
  • Dockerfile: Ensure system dependencies (already have ffmpeg and espeak-ng, which are good).

5. Directory Structure

  • app/asset/voices/: Directory to store reference audio files for cloning.

Action Plan

  1. Update requirements.txt.
  2. Create app/engines/xtts.py.
  3. Update app/config.py.
  4. Update app/main.py.