feat: Add XTTS v2 support, refactor Docker/GPU infra, and improve Piper engine

- Add XTTS v2 configuration to .env.example
- Refactor Dockerfile to multi-stage build with CUDA 12.1 support
- Update Makefile with Kokoro and XTTS test environment targets
- Refactor Piper engine (app/engines/piper.py) to use python module execution
- Add comprehensive documentation for Kokoro and XTTS plans
- Add helper scripts and patches for build process
This commit is contained in:
2025-12-13 11:37:58 +01:00
parent d6d1fe9d23
commit 91887ae296
19 changed files with 1625 additions and 46 deletions

View File

@ -0,0 +1,40 @@
# Coqui XTTS v2 Implementation Plan
## Objective
Implement Coqui XTTS v2 (`tts_models/multilingual/multi-dataset/xtts_v2`) as a new engine in AudioEngineHub.
## 1. Dependencies (`requirements.txt`)
- Add `TTS` (Coqui TTS).
- **Note:** This is a heavy library. We will add it to `requirements.txt`.
- **Potential Conflict:** `TTS` often requires specific `torch` versions. We need to ensure it plays nicely with `f5-tts` (which also uses torch) and `kokoro`.
## 2. Configuration (`app/config.py` & `.env`)
- `XTTS_DEVICE`: "cuda" or "cpu" (default: "cuda")
- `XTTS_ACCEPT_LICENSE`: "true" (required to use the model)
- `XTTS_MODEL_VERSION`: "v2.0.2" (or "main" for latest)
## 3. Engine Implementation (`app/engines/xtts.py`)
- **Class:** `XTTSEngine` (inherits `TTSEngineBase`)
- **Init:**
- Load `TTS` API.
- Download/Load model: `tts_models/multilingual/multi-dataset/xtts_v2`.
- Handle license agreement.
- **Synthesize:**
- Inputs: `text`, `speaker` (voice cloning reference), `language`.
- **Voice Cloning:** The `speaker` argument will be interpreted as a filename in `app/asset/voices/` (or a default provided sample).
- **Language:** XTTS supports 17 languages. We will map them (e.g., "en", "de", "fr").
- **List Models:** Return `['xtts_v2']`.
- **List Voices:** Scan `app/asset/voices/` for `.wav` files to use as reference speakers.
## 4. System Updates
- **`app/main.py`**: Register `xtts` in `ALL_ENGINES`.
- **`Dockerfile`**: Ensure system dependencies (already have `ffmpeg` and `espeak-ng`, which are good).
## 5. Directory Structure
- `app/asset/voices/`: Directory to store reference audio files for cloning.
## Action Plan
1. Update `requirements.txt`.
2. Create `app/engines/xtts.py`.
3. Update `app/config.py`.
4. Update `app/main.py`.