feat: Add XTTS v2 support, refactor Docker/GPU infra, and improve Piper engine
- Add XTTS v2 configuration to .env.example - Refactor Dockerfile to multi-stage build with CUDA 12.1 support - Update Makefile with Kokoro and XTTS test environment targets - Refactor Piper engine (app/engines/piper.py) to use python module execution - Add comprehensive documentation for Kokoro and XTTS plans - Add helper scripts and patches for build process
This commit is contained in:
40
XTTS_IMPLEMENTATION_PLAN.md
Normal file
40
XTTS_IMPLEMENTATION_PLAN.md
Normal file
@ -0,0 +1,40 @@
|
||||
# Coqui XTTS v2 Implementation Plan
|
||||
|
||||
## Objective
|
||||
Implement Coqui XTTS v2 (`tts_models/multilingual/multi-dataset/xtts_v2`) as a new engine in AudioEngineHub.
|
||||
|
||||
## 1. Dependencies (`requirements.txt`)
|
||||
- Add `TTS` (Coqui TTS).
|
||||
- **Note:** This is a heavy library. We will add it to `requirements.txt`.
|
||||
- **Potential Conflict:** `TTS` often requires specific `torch` versions. We need to ensure it plays nicely with `f5-tts` (which also uses torch) and `kokoro`.
|
||||
|
||||
## 2. Configuration (`app/config.py` & `.env`)
|
||||
- `XTTS_DEVICE`: "cuda" or "cpu" (default: "cuda")
|
||||
- `XTTS_ACCEPT_LICENSE`: "true" (required to use the model)
|
||||
- `XTTS_MODEL_VERSION`: "v2.0.2" (or "main" for latest)
|
||||
|
||||
## 3. Engine Implementation (`app/engines/xtts.py`)
|
||||
- **Class:** `XTTSEngine` (inherits `TTSEngineBase`)
|
||||
- **Init:**
|
||||
- Load `TTS` API.
|
||||
- Download/Load model: `tts_models/multilingual/multi-dataset/xtts_v2`.
|
||||
- Handle license agreement.
|
||||
- **Synthesize:**
|
||||
- Inputs: `text`, `speaker` (voice cloning reference), `language`.
|
||||
- **Voice Cloning:** The `speaker` argument will be interpreted as a filename in `app/asset/voices/` (or a default provided sample).
|
||||
- **Language:** XTTS supports 17 languages. We will map them (e.g., "en", "de", "fr").
|
||||
- **List Models:** Return `['xtts_v2']`.
|
||||
- **List Voices:** Scan `app/asset/voices/` for `.wav` files to use as reference speakers.
|
||||
|
||||
## 4. System Updates
|
||||
- **`app/main.py`**: Register `xtts` in `ALL_ENGINES`.
|
||||
- **`Dockerfile`**: Ensure system dependencies (already have `ffmpeg` and `espeak-ng`, which are good).
|
||||
|
||||
## 5. Directory Structure
|
||||
- `app/asset/voices/`: Directory to store reference audio files for cloning.
|
||||
|
||||
## Action Plan
|
||||
1. Update `requirements.txt`.
|
||||
2. Create `app/engines/xtts.py`.
|
||||
3. Update `app/config.py`.
|
||||
4. Update `app/main.py`.
|
||||
Reference in New Issue
Block a user