stephan
91887ae296
- Add XTTS v2 configuration to .env.example - Refactor Dockerfile to multi-stage build with CUDA 12.1 support - Update Makefile with Kokoro and XTTS test environment targets - Refactor Piper engine (app/engines/piper.py) to use python module execution - Add comprehensive documentation for Kokoro and XTTS plans - Add helper scripts and patches for build process
41 lines
1.8 KiB
Markdown
41 lines
1.8 KiB
Markdown
# Coqui XTTS v2 Implementation Plan
|
|
|
|
## Objective
|
|
Implement Coqui XTTS v2 (`tts_models/multilingual/multi-dataset/xtts_v2`) as a new engine in AudioEngineHub.
|
|
|
|
## 1. Dependencies (`requirements.txt`)
|
|
- Add `TTS` (Coqui TTS).
|
|
- **Note:** This is a heavy library. We will add it to `requirements.txt`.
|
|
- **Potential Conflict:** `TTS` often requires specific `torch` versions. We need to ensure it plays nicely with `f5-tts` (which also uses torch) and `kokoro`.
|
|
|
|
## 2. Configuration (`app/config.py` & `.env`)
|
|
- `XTTS_DEVICE`: "cuda" or "cpu" (default: "cuda")
|
|
- `XTTS_ACCEPT_LICENSE`: "true" (required to use the model)
|
|
- `XTTS_MODEL_VERSION`: "v2.0.2" (or "main" for latest)
|
|
|
|
## 3. Engine Implementation (`app/engines/xtts.py`)
|
|
- **Class:** `XTTSEngine` (inherits `TTSEngineBase`)
|
|
- **Init:**
|
|
- Load `TTS` API.
|
|
- Download/Load model: `tts_models/multilingual/multi-dataset/xtts_v2`.
|
|
- Handle license agreement.
|
|
- **Synthesize:**
|
|
- Inputs: `text`, `speaker` (voice cloning reference), `language`.
|
|
- **Voice Cloning:** The `speaker` argument will be interpreted as a filename in `app/asset/voices/` (or a default provided sample).
|
|
- **Language:** XTTS supports 17 languages. We will map them (e.g., "en", "de", "fr").
|
|
- **List Models:** Return `['xtts_v2']`.
|
|
- **List Voices:** Scan `app/asset/voices/` for `.wav` files to use as reference speakers.
|
|
|
|
## 4. System Updates
|
|
- **`app/main.py`**: Register `xtts` in `ALL_ENGINES`.
|
|
- **`Dockerfile`**: Ensure system dependencies (already have `ffmpeg` and `espeak-ng`, which are good).
|
|
|
|
## 5. Directory Structure
|
|
- `app/asset/voices/`: Directory to store reference audio files for cloning.
|
|
|
|
## Action Plan
|
|
1. Update `requirements.txt`.
|
|
2. Create `app/engines/xtts.py`.
|
|
3. Update `app/config.py`.
|
|
4. Update `app/main.py`.
|