AudioEngineHub

AudioEngineHub is a local-first, modular, multi-engine Text-to-Speech (TTS) server designed for homelabs and automation. It provides a single, unified API to interact with various TTS engines like Piper and StyleTTS.

Features

• Multi-Engine Support: Easily switch between different TTS engines.
• Configurable Engines: Activate or deactivate engines on the fly via a simple configuration file.
• Caching: Caches generated audio to save resources and provide faster responses for repeated requests.
• Dockerized: Runs in a containerized environment for easy setup and dependency management.
• Automatic Port Finding: Automatically finds and uses a free port when building locally.
• Container Registry Support: Pre-configured to push to and pull from a container registry.

Supported TTS Engines

• Piper - Fast, lightweight ONNX-based TTS with 100+ voices across multiple languages
• Kokoro - High-performance 82M parameter TTS with 54 voices across 8 languages (EN-US, EN-GB, JA, ZH, ES, FR, HI, IT, PT, KO). Delivers ~90× real-time performance on consumer GPUs
• XTTS (Coqui) - State-of-the-art voice cloning and multilingual TTS. Supports 17 languages and instant voice cloning with a 6-second audio reference.
• StyleTTS - Expressive style-based TTS (placeholder implementation)
• ChatTTS - Conversational TTS (placeholder implementation)
• F5-TTS - Advanced flow-based TTS (planned)

OpenAI API Compatibility

AudioEngineHub provides an OpenAI-compatible endpoint at /v1/audio/speech. This allows you to use it as a drop-in replacement for OpenAI's TTS service in any application or library (like LangChain, AutoGen, or the official OpenAI Python client).

• Endpoint: POST /v1/audio/speech
• Supported Models: tts-1, tts-1-hd (mapped to active local engines), or specific engine names like kokoro, xtts.
• Supported Voices: Maps the OpenAI voice parameter to the local engine's speaker.

Getting Started

This guide covers local development. For information on using the container registry, see the "Container Registry" section below.

Prerequisites

• Docker
• Docker Compose

Downloading Models (Crucial Step!)

The Docker image for AudioEngineHub does not include the large TTS model files to keep the image small and portable. You need to manually download the models for the engines you wish to use and place them in the correct local directory. The docker-compose.yml then makes these models available to the container via a volume mount.

Piper Models

• Source: https://huggingface.co/rhasspy/piper-voices/tree/main

Instructions:

1. Go to the link above and navigate to a voice you want to use (e.g., en/en_GB/vctk/medium/).
2. For each voice, you need to download two files:
   • The .onnx model file (e.g., en_GB-vctk-medium.onnx)
   • The corresponding .onnx.json configuration file (e.g., en_GB-vctk-medium.onnx.json)
3. Create a directory for the voice inside your local app/models/piper/ directory. The directory name must match the model's base name (e.g., en_GB-vctk-medium).
4. Place both downloaded files into that new directory.

Example: Setting up en_GB-vctk-medium:

Your local directory structure should look like this:

AudioEngineHub/
├── app/
│   ├── models/
│   │   ├── piper/
│   │   │   ├── en_GB-vctk-medium/  <-- This directory's name MUST match the model name
│   │   │   │   ├── en_GB-vctk-medium.onnx
│   │   │   │   └── en_GB-vctk-medium.onnx.json
│   │   └── styletts/ # Placeholder, no external models currently needed
│   └── ...
├── ...

StyleTTS Models

The styletts engine is currently a placeholder (dummy implementation) and does not require external model downloads at this time. Its list_models() method provides hardcoded model names.

Kokoro Models

The Kokoro engine automatically downloads models from Hugging Face on first use (lazy loading). No manual download is required.

Model Details:

• Source: Kokoro-82M on Hugging Face
• Size: ~200MB per language model
• Cache Location: Models are cached in ~/.cache/huggingface/ inside the container
• First Synthesis: May take 30-60 seconds due to model download and compilation
• Languages: 8 languages available (EN-US, EN-GB, FR, ES, JA, ZH, IT, PT, HI, KO)
• Voices: 54 high-quality voices across all languages
• GPU Support: Automatically uses CUDA if available, falls back to CPU
• Performance: ~90× real-time on RTX 3090 Ti, ~210× on RTX 4090

Configuration:

# In .env file
KOKORO_DEVICE=cuda  # or "cpu" for CPU-only systems
KOKORO_TIMEOUT_SECONDS=30
ACTIVE_ENGINES='["piper", "kokoro"]'  # Enable Kokoro

XTTS Models (Coqui)

The XTTS v2 model is downloaded automatically on first use.

Important: You must explicitly accept the Coqui Public Model License to use this engine.

Configuration:

1. License: Set XTTS_ACCEPT_LICENSE=true in your .env file.
2. Voice Cloning: Place your reference audio files (e.g., my_voice.wav) in app/asset/voices/. The filename (without extension) becomes the speaker ID.
3. Hardware: CUDA (NVIDIA GPU) is highly recommended for reasonable inference speeds.

# In .env file
XTTS_DEVICE=cuda      # or "cpu" (slow!)
XTTS_ACCEPT_LICENSE=true
ACTIVE_ENGINES='["piper", "xtts"]'

Local Development Setup

1. Clone the repository:

   git clone <repository_url>
   cd AudioEngineHub
   

2. Configure the environment: Create a .env file by copying the example file:

   cp .env.example .env
   

   Open the .env file and configure the ACTIVE_ENGINES list to include the engines you want to use. Make sure the model directories exist for activated engines (e.g., if you enable piper, ensure its models are downloaded). For example:

   ACTIVE_ENGINES='["piper", "styletts"]'
   

3. Build and start the container: Use the make dev-up command to build the Docker image from your local source and start the service.

   make dev-up
   

   This command will automatically find a free port, build the image, and run the application.

   Note: For the most reliable port detection, it is recommended to run the command with sudo:

   sudo make dev-up
   

Container Registry

The project is configured to work with the container registry at git.wlkns.org.

Pushing an Image

1. Log in to the Registry: You only need to do this once per machine.

   docker login git.wlkns.org
   

2. Push the Image: This command will build your image, tag it correctly, and push it to the registry.

   make push
   

Pulling and Running an Image

1. Pull the Image: To download the latest image from the registry:

   make pull
   

2. Run the Image: This command will start the application using the pre-built image from the registry (pulling it if necessary).

   make up
   

Usage

Endpoints

• POST /tts: The main endpoint to synthesize text to speech.
• GET /health: Check the health of the API and the status of the loaded engines.
• GET /engines: List the currently active engines.
• GET /models: List the available models for each active engine. _ GET /speakers: List the available speakers for a given engine and model.

Makefile Commands

The project includes a Makefile with several commands to simplify development and management:

• make dev-up: Build the image from local source and start the application. Recommended for development.
• make up: Start the application using the image from the container registry (pulls if not present).
• make down: Stop the application container(s).
• make logs: View the application logs.
• make health-check: Run a sanity check to ensure the deployed container is healthy and all engines are "ok".
• make pull: Pull the latest image from the container registry.
• make push: Build, tag, and push the image to the container registry.
• make test: Run the pytest test suite.
• make help: Display a list of all available commands.

Configuration

The application is configured through the .env file in the root of the project.

• ACTIVE_ENGINES: A comma-separated list of strings specifying which TTS engines to activate. Available engines are defined in app/main.py.
• HOST: The host address for the server (defaults to 0.0.0.0).
• PORT: The internal port for the server (defaults to 8000).
• IMAGE_NAME: The name of the Docker image to build (defaults to audioenginehub).