diff --git a/API_DOCUMENTATION.md b/API_DOCUMENTATION.md new file mode 100644 index 0000000..6f8c6b9 --- /dev/null +++ b/API_DOCUMENTATION.md @@ -0,0 +1,269 @@ +# AudioEngineHub API Documentation + +This document provides detailed information on how to interact with the AudioEngineHub API. AudioEngineHub is a local-first, modular multi-engine Text-to-Speech (TTS) server designed for homelabs and automation. + +The API is built using FastAPI, which automatically generates OpenAPI (Swagger) documentation. If the AudioEngineHub service is running, you can typically access the interactive API documentation at `/docs` (e.g., `http://localhost:8000/docs`) and the OpenAPI specification JSON at `/openapi.json` (e.g., `http://localhost:8000/openapi.json`). + +## Endpoints + +--- + +### `POST /v1/audio/speech` (OpenAI Compatible) + +A drop-in replacement for the [OpenAI Text-to-Speech API](https://platform.openai.com/docs/api-reference/audio/createSpeech). Synthesizes audio and streams the binary response directly. + +* **HTTP Method:** `POST` +* **Description:** Allows integration with existing tools and libraries designed for OpenAI's TTS. +* **Request Body:** + * `model`: (string, required) The ID of the model/engine. + * Standard OpenAI IDs: `tts-1`, `tts-1-hd` (mapped to the first active local engine, e.g., Kokoro or XTTS). + * Local Engine IDs: `kokoro`, `xtts`, `piper`, `kokoro:en-us`, `xtts:v2`. + * `input`: (string, required) The text to generate audio for. + * `voice`: (string, required) The voice to use (maps to local `speaker`). + * `response_format`: (string, optional) `mp3`, `opus`, `aac`, `flac`, `wav`, `pcm`. Defaults to `mp3`. + * `speed`: (number, optional) Speed of generated audio (0.25 to 4.0). *Currently ignored by most local engines.* +* **Response:** Binary audio stream (content-type corresponds to `response_format`). + +--- + +### `POST /tts` + +Synthesizes text to speech using a specified engine and model. + +* **HTTP Method:** `POST` +* **Description:** The main endpoint to synthesize text to speech. It accepts a `TTSRequest` body and can return audio either as a downloadable file or as a base64 encoded string. +* **Query Parameters:** + * `as_base64`: + * **Type:** `boolean` + * **Description:** If `true`, the audio will be returned as a base64 encoded string within a JSON response. If `false` (default), a JSON response with an `audio_url` for direct download will be returned. + * **Required:** No (default: `false`) +* **Request Body (`TTSRequest`):** + * **Description:** Defines the parameters for the text-to-speech synthesis. + * **Fields:** + * `text`: + * **Type:** `string` + * **Description:** The text to be synthesized. + * **Required:** Yes + * `engine`: + * **Type:** `string` + * **Description:** The name of the TTS engine to use (e.g., "piper", "kokoro"). + * **Required:** Yes + * `model`: + * **Type:** `string` + * **Description:** The specific model to use within the chosen engine. + * **Required:** No (default: `null`) + * `speaker`: + * **Type:** `string` + **Description:** The speaker/voice to use for synthesis, if supported by the model/engine. + * **Required:** No (default: `null`) + * `format`: + * **Type:** `string` + * **Description:** The desired output audio format (e.g., "ogg", "wav", "mp3"). + * **Required:** No (default: "ogg") + * `chunking`: + * **Type:** `boolean` + * **Description:** If `true`, the input text will be chunked into smaller pieces for synthesis, then concatenated. This can help with very long texts or engines with text length limitations. + * **Required:** No (default: `false`) +* **Responses:** + * **`200 OK` (Audio URL):** + ```json + { + "engine": "piper", + "model": "en_GB-vctk-medium", + "speaker": "p225", + "format": "ogg", + "audio_url": "/audio/tts_some_unique_id.ogg", + "cached": true, + "chunking": false, + "message": "Audio served from cache. Download from audio_url" + } + ``` + * **Description:** Returned when `as_base64` is `false`. Contains a URL to download the synthesized audio file. `cached` indicates if the audio was served from the cache. + * **`200 OK` (Base64 Audio):** + ```json + { + "engine": "piper", + "model": "en_GB-vctk-medium", + "speaker": "p225", + "format": "ogg", + "audio_base64": "data:audio/ogg;base64,...", + "chunking": false, + "message": "Audio from synth, base64 included" + } + ``` + * **Description:** Returned when `as_base64` is `true`. Contains the base64 encoded audio data directly in the response. + * **`400 Bad Request`:** + * **Description:** Returned if the provided model or speaker is not found for the selected engine, or if other input validation fails. + * **Example Body:** `{"detail": "Model 'invalid_model' not found for engine 'piper'. Available models: [...]"}` + * **`404 Not Found`:** + * **Description:** Returned if the specified engine is not found. + * **Example Body:** `{"detail": "Engine 'nonexistent_engine' not found."}` + * **`503 Service Unavailable`:** + * **Description:** Returned if the specified engine is not available or its health check fails. + * **Example Body:** `{"detail": "Engine 'piper' is not available. Status: initializing"}` + * **`500 Internal Server Error`:** + * **Description:** Returned if an unexpected error occurs during synthesis. + * **Example Body:** `{"detail": "Error during synthesis: some error message"}` + +--- + +### `GET /audio/{filename}` + +Retrieves a synthesized audio file from the cache. + +* **HTTP Method:** `GET` +* **Description:** This endpoint allows direct download of audio files that were previously synthesized and cached. The `audio_url` provided by the `/tts` endpoint will typically point to this endpoint. +* **Path Parameters:** + * `filename`: + * **Type:** `string` + * **Description:** The full filename of the audio file to retrieve (e.g., `tts_some_unique_id.ogg`). + * **Required:** Yes +* **Query Parameters:** None +* **Request Body:** None +* **Responses:** + * **`200 OK` (Audio File):** + * **Content-Type:** `audio/wav`, `audio/ogg`, or `audio/mpeg` (depending on file extension) + * **Description:** The raw audio file bytes. + * **`404 Not Found`:** + * **Description:** Returned if the specified audio file does not exist in the cache. + * **Example Body:** `{"detail": "Audio file not found"}` + +--- + +### `GET /engines` + +Lists the currently active TTS engines and their health status. + +* **HTTP Method:** `GET` +* **Description:** Returns a dictionary where keys are the names of the active engines and values are their respective health statuses. +* **Query Parameters:** None +* **Request Body:** None +* **Responses:** + * **`200 OK`:** + ```json + { + "piper": { + "status": "ok", + "detail": "Engine is ready." + }, + "kokoro": { + "status": "initializing", + "detail": "Models are loading..." + }, + "styletts": { + "status": "ok", + "detail": "Engine is ready." + } + } + ``` + * **Description:** A JSON object detailing the health status of each active engine. + +--- + +### `GET /models` + +Lists the available models for each active TTS engine. + +* **HTTP Method:** `GET` +* **Description:** Returns a dictionary where keys are engine names and values are lists of models available for that engine. +* **Query Parameters:** None +* **Request Body:** None +* **Responses:** + * **`200 OK`:** + ```json + { + "piper": [ + "en_US-kristin-medium", + "de_DE-thorsten-high", + "en_GB-vctk-medium" + ], + "kokoro": [ + "en-US-Standard-A", + "en-GB-Standard-B", + "ja-JP-Standard-C" + ], + "styletts": [ + "default" + ] + } + ``` + * **Description:** A JSON object detailing the models available for each active engine. If an engine encounters an error while listing models, an "error" field will be present for that engine. + +--- + +### `GET /speakers` + +Lists the available speakers (voices) for a given engine and optionally a specific model. + +* **HTTP Method:** `GET` +* **Description:** Retrieves a list of available speakers for a specified TTS engine. If a model is also specified, it will return speakers specific to that model. +* **Query Parameters:** + * `engine`: + * **Type:** `string` + * **Description:** The name of the TTS engine (e.g., "piper", "kokoro"). + * **Required:** Yes + * `model`: + * **Type:** `string` + * **Description:** The specific model to query speakers for. + * **Required:** No +* **Request Body:** None +* **Responses:** + * **`200 OK`:** + ```json + { + "engine": "piper", + "model": "en_GB-vctk-medium", + "speakers": [ + "p225", + "p226", + "p227" + ] + } + ``` + * **Description:** A JSON object containing the engine, model (if provided), and a list of available speakers. + * **`404 Not Found`:** + * **Description:** Returned if the specified engine is not found. + * **Example Body:** `{"detail": "Engine 'nonexistent_engine' not found."}` + +--- + +### `GET /version` + +Retrieves the current API version. + +* **HTTP Method:** `GET` +* **Description:** Returns the version string of the AudioEngineHub API. +* **Query Parameters:** None +* **Request Body:** None +* **Responses:** + * **`200 OK`:** + ```json + { + "version": "0.3.0" + } + ``` + * **Description:** A JSON object containing the API version. + +--- + +### `GET /health` + +Checks the health status of the API and all loaded engines. + +* **HTTP Method:** `GET` +* **Description:** Provides an overview of the system's health, including the status of the API itself and each active TTS engine. +* **Query Parameters:** None +* **Request Body:** None +* **Responses:** + * **`200 OK`:** + ```json + { + "status": { + "piper": "ok", + "kokoro": "ok", + "styletts": "ok" + }, + "detail": "API and engines loaded" + } + ``` + * **Description:** A JSON object indicating the overall status (`detail`) and the individual health status of each active engine. diff --git a/README.md b/README.md index e95185b..c5de603 100644 --- a/README.md +++ b/README.md @@ -11,6 +11,23 @@ AudioEngineHub is a local-first, modular, multi-engine Text-to-Speech (TTS) serv - **Automatic Port Finding:** Automatically finds and uses a free port when building locally. - **Container Registry Support:** Pre-configured to push to and pull from a container registry. +## Supported TTS Engines + +- **Piper** - Fast, lightweight ONNX-based TTS with 100+ voices across multiple languages +- **Kokoro** - High-performance 82M parameter TTS with 54 voices across 8 languages (EN-US, EN-GB, JA, ZH, ES, FR, HI, IT, PT, KO). Delivers ~90× real-time performance on consumer GPUs +- **XTTS (Coqui)** - State-of-the-art voice cloning and multilingual TTS. Supports 17 languages and instant voice cloning with a 6-second audio reference. +- **StyleTTS** - Expressive style-based TTS (placeholder implementation) +- **ChatTTS** - Conversational TTS (placeholder implementation) +- **F5-TTS** - Advanced flow-based TTS (planned) + +## OpenAI API Compatibility + +AudioEngineHub provides an OpenAI-compatible endpoint at `/v1/audio/speech`. This allows you to use it as a drop-in replacement for OpenAI's TTS service in any application or library (like LangChain, AutoGen, or the official OpenAI Python client). + +- **Endpoint:** `POST /v1/audio/speech` +- **Supported Models:** `tts-1`, `tts-1-hd` (mapped to active local engines), or specific engine names like `kokoro`, `xtts`. +- **Supported Voices:** Maps the OpenAI `voice` parameter to the local engine's speaker. + ## Getting Started This guide covers local development. For information on using the container registry, see the "Container Registry" section below. @@ -57,6 +74,46 @@ AudioEngineHub/ The `styletts` engine is currently a placeholder (dummy implementation) and does not require external model downloads at this time. Its `list_models()` method provides hardcoded model names. +#### Kokoro Models + +The Kokoro engine automatically downloads models from Hugging Face on first use (lazy loading). No manual download is required. + +**Model Details:** +- **Source:** [Kokoro-82M on Hugging Face](https://huggingface.co/hexgrad/Kokoro-82M) +- **Size:** ~200MB per language model +- **Cache Location:** Models are cached in `~/.cache/huggingface/` inside the container +- **First Synthesis:** May take 30-60 seconds due to model download and compilation +- **Languages:** 8 languages available (EN-US, EN-GB, FR, ES, JA, ZH, IT, PT, HI, KO) +- **Voices:** 54 high-quality voices across all languages +- **GPU Support:** Automatically uses CUDA if available, falls back to CPU +- **Performance:** ~90× real-time on RTX 3090 Ti, ~210× on RTX 4090 + +**Configuration:** +```bash +# In .env file +KOKORO_DEVICE=cuda # or "cpu" for CPU-only systems +KOKORO_TIMEOUT_SECONDS=30 +ACTIVE_ENGINES='["piper", "kokoro"]' # Enable Kokoro +``` + +#### XTTS Models (Coqui) + +The XTTS v2 model is downloaded automatically on first use. + +**Important:** You **must** explicitly accept the Coqui Public Model License to use this engine. + +**Configuration:** +1. **License:** Set `XTTS_ACCEPT_LICENSE=true` in your `.env` file. +2. **Voice Cloning:** Place your reference audio files (e.g., `my_voice.wav`) in `app/asset/voices/`. The filename (without extension) becomes the `speaker` ID. +3. **Hardware:** CUDA (NVIDIA GPU) is highly recommended for reasonable inference speeds. + +```bash +# In .env file +XTTS_DEVICE=cuda # or "cpu" (slow!) +XTTS_ACCEPT_LICENSE=true +ACTIVE_ENGINES='["piper", "xtts"]' +``` + ### Local Development Setup 1. **Clone the repository:**