docs: Update documentation for OpenAI API and XTTS support

- Updated README.md to include XTTS engine details and model setup instructions.
- Added section on OpenAI API compatibility in README.md.
- Updated API_DOCUMENTATION.md to include the new POST /v1/audio/speech endpoint.
This commit is contained in:
2025-12-09 14:37:40 +01:00
parent 7a2c6bb209
commit d6d1fe9d23
2 changed files with 326 additions and 0 deletions

View File

@ -11,6 +11,23 @@ AudioEngineHub is a local-first, modular, multi-engine Text-to-Speech (TTS) serv
- **Automatic Port Finding:** Automatically finds and uses a free port when building locally.
- **Container Registry Support:** Pre-configured to push to and pull from a container registry.
## Supported TTS Engines
- **Piper** - Fast, lightweight ONNX-based TTS with 100+ voices across multiple languages
- **Kokoro** - High-performance 82M parameter TTS with 54 voices across 8 languages (EN-US, EN-GB, JA, ZH, ES, FR, HI, IT, PT, KO). Delivers ~90× real-time performance on consumer GPUs
- **XTTS (Coqui)** - State-of-the-art voice cloning and multilingual TTS. Supports 17 languages and instant voice cloning with a 6-second audio reference.
- **StyleTTS** - Expressive style-based TTS (placeholder implementation)
- **ChatTTS** - Conversational TTS (placeholder implementation)
- **F5-TTS** - Advanced flow-based TTS (planned)
## OpenAI API Compatibility
AudioEngineHub provides an OpenAI-compatible endpoint at `/v1/audio/speech`. This allows you to use it as a drop-in replacement for OpenAI's TTS service in any application or library (like LangChain, AutoGen, or the official OpenAI Python client).
- **Endpoint:** `POST /v1/audio/speech`
- **Supported Models:** `tts-1`, `tts-1-hd` (mapped to active local engines), or specific engine names like `kokoro`, `xtts`.
- **Supported Voices:** Maps the OpenAI `voice` parameter to the local engine's speaker.
## Getting Started
This guide covers local development. For information on using the container registry, see the "Container Registry" section below.
@ -57,6 +74,46 @@ AudioEngineHub/
The `styletts` engine is currently a placeholder (dummy implementation) and does not require external model downloads at this time. Its `list_models()` method provides hardcoded model names.
#### Kokoro Models
The Kokoro engine automatically downloads models from Hugging Face on first use (lazy loading). No manual download is required.
**Model Details:**
- **Source:** [Kokoro-82M on Hugging Face](https://huggingface.co/hexgrad/Kokoro-82M)
- **Size:** ~200MB per language model
- **Cache Location:** Models are cached in `~/.cache/huggingface/` inside the container
- **First Synthesis:** May take 30-60 seconds due to model download and compilation
- **Languages:** 8 languages available (EN-US, EN-GB, FR, ES, JA, ZH, IT, PT, HI, KO)
- **Voices:** 54 high-quality voices across all languages
- **GPU Support:** Automatically uses CUDA if available, falls back to CPU
- **Performance:** ~90× real-time on RTX 3090 Ti, ~210× on RTX 4090
**Configuration:**
```bash
# In .env file
KOKORO_DEVICE=cuda # or "cpu" for CPU-only systems
KOKORO_TIMEOUT_SECONDS=30
ACTIVE_ENGINES='["piper", "kokoro"]' # Enable Kokoro
```
#### XTTS Models (Coqui)
The XTTS v2 model is downloaded automatically on first use.
**Important:** You **must** explicitly accept the Coqui Public Model License to use this engine.
**Configuration:**
1. **License:** Set `XTTS_ACCEPT_LICENSE=true` in your `.env` file.
2. **Voice Cloning:** Place your reference audio files (e.g., `my_voice.wav`) in `app/asset/voices/`. The filename (without extension) becomes the `speaker` ID.
3. **Hardware:** CUDA (NVIDIA GPU) is highly recommended for reasonable inference speeds.
```bash
# In .env file
XTTS_DEVICE=cuda # or "cpu" (slow!)
XTTS_ACCEPT_LICENSE=true
ACTIVE_ENGINES='["piper", "xtts"]'
```
### Local Development Setup
1. **Clone the repository:**