stephan/audio-engine-hub
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Update documentation for OpenAI API and XTTS support

Browse Source 
- Updated README.md to include XTTS engine details and model setup instructions.
- Added section on OpenAI API compatibility in README.md.
- Updated API_DOCUMENTATION.md to include the new POST /v1/audio/speech endpoint.
This commit is contained in:
Stephan Wilkens
2025-12-09 14:37:40 +01:00
parent 7a2c6bb209
commit d6d1fe9d23
2 changed files with 326 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

269
API_DOCUMENTATION.md Normal file
View File

@ -0,0 +1,269 @@
# AudioEngineHub API Documentation
This document provides detailed information on how to interact with the AudioEngineHub API. AudioEngineHub is a local-first, modular multi-engine Text-to-Speech (TTS) server designed for homelabs and automation.
The API is built using FastAPI, which automatically generates OpenAPI (Swagger) documentation. If the AudioEngineHub service is running, you can typically access the interactive API documentation at `/docs` (e.g., `http://localhost:8000/docs`) and the OpenAPI specification JSON at `/openapi.json` (e.g., `http://localhost:8000/openapi.json`).
## Endpoints
---
### `POST /v1/audio/speech` (OpenAI Compatible)
A drop-in replacement for the [OpenAI Text-to-Speech API](https://platform.openai.com/docs/api-reference/audio/createSpeech). Synthesizes audio and streams the binary response directly.
* **HTTP Method:** `POST`
* **Description:** Allows integration with existing tools and libraries designed for OpenAI's TTS.
* **Request Body:**
* `model`: (string, required) The ID of the model/engine.
* Standard OpenAI IDs: `tts-1`, `tts-1-hd` (mapped to the first active local engine, e.g., Kokoro or XTTS).
* Local Engine IDs: `kokoro`, `xtts`, `piper`, `kokoro:en-us`, `xtts:v2`.
* `input`: (string, required) The text to generate audio for.
* `voice`: (string, required) The voice to use (maps to local `speaker`).
* `response_format`: (string, optional) `mp3`, `opus`, `aac`, `flac`, `wav`, `pcm`. Defaults to `mp3`.
* `speed`: (number, optional) Speed of generated audio (0.25 to 4.0). *Currently ignored by most local engines.*
* **Response:** Binary audio stream (content-type corresponds to `response_format`).
---
### `POST /tts`
Synthesizes text to speech using a specified engine and model.
* **HTTP Method:** `POST`
* **Description:** The main endpoint to synthesize text to speech. It accepts a `TTSRequest` body and can return audio either as a downloadable file or as a base64 encoded string.
* **Query Parameters:**
* `as_base64`:
* **Type:** `boolean`
* **Description:** If `true`, the audio will be returned as a base64 encoded string within a JSON response. If `false` (default), a JSON response with an `audio_url` for direct download will be returned.
* **Required:** No (default: `false`)
* **Request Body (`TTSRequest`):**
* **Description:** Defines the parameters for the text-to-speech synthesis.
* **Fields:**
* `text`:
* **Type:** `string`
* **Description:** The text to be synthesized.
* **Required:** Yes
* `engine`:
* **Type:** `string`
* **Description:** The name of the TTS engine to use (e.g., "piper", "kokoro").
* **Required:** Yes
* `model`:
* **Type:** `string`
* **Description:** The specific model to use within the chosen engine.
* **Required:** No (default: `null`)
* `speaker`:
* **Type:** `string`
**Description:** The speaker/voice to use for synthesis, if supported by the model/engine.
* **Required:** No (default: `null`)
* `format`:
* **Type:** `string`
* **Description:** The desired output audio format (e.g., "ogg", "wav", "mp3").
* **Required:** No (default: "ogg")
* `chunking`:
* **Type:** `boolean`
* **Description:** If `true`, the input text will be chunked into smaller pieces for synthesis, then concatenated. This can help with very long texts or engines with text length limitations.
* **Required:** No (default: `false`)
* **Responses:**
* **`200 OK` (Audio URL):**
```json
{
"engine": "piper",
"model": "en_GB-vctk-medium",
"speaker": "p225",
"format": "ogg",
"audio_url": "/audio/tts_some_unique_id.ogg",
"cached": true,
"chunking": false,
"message": "Audio served from cache. Download from audio_url"
}
```
* **Description:** Returned when `as_base64` is `false`. Contains a URL to download the synthesized audio file. `cached` indicates if the audio was served from the cache.
* **`200 OK` (Base64 Audio):**
```json
{
"engine": "piper",
"model": "en_GB-vctk-medium",
"speaker": "p225",
"format": "ogg",
"audio_base64": "data:audio/ogg;base64,...",
"chunking": false,
"message": "Audio from synth, base64 included"
}
```
* **Description:** Returned when `as_base64` is `true`. Contains the base64 encoded audio data directly in the response.
* **`400 Bad Request`:**
* **Description:** Returned if the provided model or speaker is not found for the selected engine, or if other input validation fails.
* **Example Body:** `{"detail": "Model 'invalid_model' not found for engine 'piper'. Available models: [...]"}`
* **`404 Not Found`:**
* **Description:** Returned if the specified engine is not found.
* **Example Body:** `{"detail": "Engine 'nonexistent_engine' not found."}`
* **`503 Service Unavailable`:**
* **Description:** Returned if the specified engine is not available or its health check fails.
* **Example Body:** `{"detail": "Engine 'piper' is not available. Status: initializing"}`
* **`500 Internal Server Error`:**
* **Description:** Returned if an unexpected error occurs during synthesis.
* **Example Body:** `{"detail": "Error during synthesis: some error message"}`
---
### `GET /audio/{filename}`
Retrieves a synthesized audio file from the cache.
* **HTTP Method:** `GET`
* **Description:** This endpoint allows direct download of audio files that were previously synthesized and cached. The `audio_url` provided by the `/tts` endpoint will typically point to this endpoint.
* **Path Parameters:**
* `filename`:
* **Type:** `string`
* **Description:** The full filename of the audio file to retrieve (e.g., `tts_some_unique_id.ogg`).
* **Required:** Yes
* **Query Parameters:** None
* **Request Body:** None
* **Responses:**
* **`200 OK` (Audio File):**
* **Content-Type:** `audio/wav`, `audio/ogg`, or `audio/mpeg` (depending on file extension)
* **Description:** The raw audio file bytes.
* **`404 Not Found`:**
* **Description:** Returned if the specified audio file does not exist in the cache.
* **Example Body:** `{"detail": "Audio file not found"}`
---
### `GET /engines`
Lists the currently active TTS engines and their health status.
* **HTTP Method:** `GET`
* **Description:** Returns a dictionary where keys are the names of the active engines and values are their respective health statuses.
* **Query Parameters:** None
* **Request Body:** None
* **Responses:**
* **`200 OK`:**
```json
{
"piper": {
"status": "ok",
"detail": "Engine is ready."
},
"kokoro": {
"status": "initializing",
"detail": "Models are loading..."
},
"styletts": {
"status": "ok",
"detail": "Engine is ready."
}
}
```
* **Description:** A JSON object detailing the health status of each active engine.
---
### `GET /models`
Lists the available models for each active TTS engine.
* **HTTP Method:** `GET`
* **Description:** Returns a dictionary where keys are engine names and values are lists of models available for that engine.
* **Query Parameters:** None
* **Request Body:** None
* **Responses:**
* **`200 OK`:**
```json
{
"piper": [
"en_US-kristin-medium",
"de_DE-thorsten-high",
"en_GB-vctk-medium"
],
"kokoro": [
"en-US-Standard-A",
"en-GB-Standard-B",
"ja-JP-Standard-C"
],
"styletts": [
"default"
]
}
```
* **Description:** A JSON object detailing the models available for each active engine. If an engine encounters an error while listing models, an "error" field will be present for that engine.
---
### `GET /speakers`
Lists the available speakers (voices) for a given engine and optionally a specific model.
* **HTTP Method:** `GET`
* **Description:** Retrieves a list of available speakers for a specified TTS engine. If a model is also specified, it will return speakers specific to that model.
* **Query Parameters:**
* `engine`:
* **Type:** `string`
* **Description:** The name of the TTS engine (e.g., "piper", "kokoro").
* **Required:** Yes
* `model`:
* **Type:** `string`
* **Description:** The specific model to query speakers for.
* **Required:** No
* **Request Body:** None
* **Responses:**
* **`200 OK`:**
```json
{
"engine": "piper",
"model": "en_GB-vctk-medium",
"speakers": [
"p225",
"p226",
"p227"
]
}
```
* **Description:** A JSON object containing the engine, model (if provided), and a list of available speakers.
* **`404 Not Found`:**
* **Description:** Returned if the specified engine is not found.
* **Example Body:** `{"detail": "Engine 'nonexistent_engine' not found."}`
---
### `GET /version`
Retrieves the current API version.
* **HTTP Method:** `GET`
* **Description:** Returns the version string of the AudioEngineHub API.
* **Query Parameters:** None
* **Request Body:** None
* **Responses:**
* **`200 OK`:**
```json
{
"version": "0.3.0"
}
```
* **Description:** A JSON object containing the API version.
---
### `GET /health`
Checks the health status of the API and all loaded engines.
* **HTTP Method:** `GET`
* **Description:** Provides an overview of the system's health, including the status of the API itself and each active TTS engine.
* **Query Parameters:** None
* **Request Body:** None
* **Responses:**
* **`200 OK`:**
```json
{
"status": {
"piper": "ok",
"kokoro": "ok",
"styletts": "ok"
},
"detail": "API and engines loaded"
}
```
* **Description:** A JSON object indicating the overall status (`detail`) and the individual health status of each active engine.

57
README.md
View File

@ -11,6 +11,23 @@ AudioEngineHub is a local-first, modular, multi-engine Text-to-Speech (TTS) serv
- **Automatic Port Finding:** Automatically finds and uses a free port when building locally.
- **Container Registry Support:** Pre-configured to push to and pull from a container registry.
## Supported TTS Engines
- **Piper** - Fast, lightweight ONNX-based TTS with 100+ voices across multiple languages
- **Kokoro** - High-performance 82M parameter TTS with 54 voices across 8 languages (EN-US, EN-GB, JA, ZH, ES, FR, HI, IT, PT, KO). Delivers ~90× real-time performance on consumer GPUs
- **XTTS (Coqui)** - State-of-the-art voice cloning and multilingual TTS. Supports 17 languages and instant voice cloning with a 6-second audio reference.
- **StyleTTS** - Expressive style-based TTS (placeholder implementation)
- **ChatTTS** - Conversational TTS (placeholder implementation)
- **F5-TTS** - Advanced flow-based TTS (planned)
## OpenAI API Compatibility
AudioEngineHub provides an OpenAI-compatible endpoint at `/v1/audio/speech`. This allows you to use it as a drop-in replacement for OpenAI's TTS service in any application or library (like LangChain, AutoGen, or the official OpenAI Python client).
- **Endpoint:** `POST /v1/audio/speech`
- **Supported Models:** `tts-1`, `tts-1-hd` (mapped to active local engines), or specific engine names like `kokoro`, `xtts`.
- **Supported Voices:** Maps the OpenAI `voice` parameter to the local engine's speaker.
## Getting Started
This guide covers local development. For information on using the container registry, see the "Container Registry" section below.
@ -57,6 +74,46 @@ AudioEngineHub/
The `styletts` engine is currently a placeholder (dummy implementation) and does not require external model downloads at this time. Its `list_models()` method provides hardcoded model names.
#### Kokoro Models
The Kokoro engine automatically downloads models from Hugging Face on first use (lazy loading). No manual download is required.
**Model Details:**
- **Source:** [Kokoro-82M on Hugging Face](https://huggingface.co/hexgrad/Kokoro-82M)
- **Size:** ~200MB per language model
- **Cache Location:** Models are cached in `~/.cache/huggingface/` inside the container
- **First Synthesis:** May take 30-60 seconds due to model download and compilation
- **Languages:** 8 languages available (EN-US, EN-GB, FR, ES, JA, ZH, IT, PT, HI, KO)
- **Voices:** 54 high-quality voices across all languages
- **GPU Support:** Automatically uses CUDA if available, falls back to CPU
- **Performance:** ~90× real-time on RTX 3090 Ti, ~210× on RTX 4090
**Configuration:**
```bash
# In .env file
KOKORO_DEVICE=cuda # or "cpu" for CPU-only systems
KOKORO_TIMEOUT_SECONDS=30
ACTIVE_ENGINES='["piper", "kokoro"]' # Enable Kokoro
```
#### XTTS Models (Coqui)
The XTTS v2 model is downloaded automatically on first use.
**Important:** You **must** explicitly accept the Coqui Public Model License to use this engine.
**Configuration:**
1. **License:** Set `XTTS_ACCEPT_LICENSE=true` in your `.env` file.
2. **Voice Cloning:** Place your reference audio files (e.g., `my_voice.wav`) in `app/asset/voices/`. The filename (without extension) becomes the `speaker` ID.
3. **Hardware:** CUDA (NVIDIA GPU) is highly recommended for reasonable inference speeds.
```bash
# In .env file
XTTS_DEVICE=cuda # or "cpu" (slow!)
XTTS_ACCEPT_LICENSE=true
ACTIVE_ENGINES='["piper", "xtts"]'
```
### Local Development Setup
1. **Clone the repository:**