XTTS2 OpenAI-Compatible TTS Server

Project Overview

This project is a high-performance, modular, and scalable Text-to-Speech (TTS) platform. It provides an API fully compatible with the OpenAI Speech API, powered by the XTTS2 model for high-quality synthesis and zero-shot voice cloning.

Architecture

The system follows a distributed architecture:

Gateway (gateway/): A FastAPI service that handles HTTP requests, validates input, and manages the voice registry. It pushes synthesis jobs to a Redis queue. It features dynamic port selection (8000-8100).
Redis: Acts as the message broker (Queue) and cache between the Gateway and Workers.
Worker (worker/): A background service that pulls jobs from Redis, performs the actual TTS inference using XTTS2 (with GPU acceleration if available), and returns the audio data. These can be scaled horizontally.

Key Technologies

Language: Python 3.10+
Framework: FastAPI (Gateway)
ML Model: Coqui XTTS v2
Infrastructure: Docker, Docker Compose, Redis
Tooling: Makefile for orchestration

Building and Running

The project relies heavily on make for orchestration.

Docker (Recommended)

Build Images:
```
make build
```
Start Services:
```
make up
```
- This runs a port scanner to find a free port between 8000-8100.
- The chosen port is saved to gateway/port.txt.
Check Status:
```
make status
```
View Logs:
```
make logs
```
Stop Services:
```
make down
```

Scaling Workers

To handle higher load, you can spawn multiple worker containers:

make worker-scale N=3

Verification

Run the self-test suite to verify Redis connectivity, worker processing, and audio synthesis:

make selftest

Development Conventions

Project Structure

gateway/: Code for the API server.
- main.py: Entry point.
- api/: Endpoint definitions (openai_speech.py, voices.py).
- core/: Configuration and utilities.
worker/: Code for the inference engine.
- engine/: XTTS2 model loading and audio export logic.
- core/: Queue processing and GPU detection.
scripts/: Utility scripts (e.g., find_port.py, selftest.py).

Local Development (Non-Docker)

Create a virtual environment:

python3 -m venv .venv
source .venv/bin/activate

Install dependencies:

pip install -r gateway/requirements.gateway.txt
pip install -r worker/requirements.worker.txt

Run Redis locally (e.g., docker run -p 6379:6379 redis:7).
Start Gateway: python gateway/main.py
Start Worker: python worker/main.py

API Usage

The API mirrors OpenAI's structure.

Generate Audio:

POST /v1/audio/speech
Content-Type: application/json

{
  "model": "xtts-v2",
  "input": "Hello world",
  "voice": "auto",
  "format": "wav"
}

Register Voice:

POST /v1/voices/register
Content-Type: application/json

{
  "name": "my-voice",
  "samples": ["https://example.com/sample.wav"]
}

Logging & Debugging

Gateway Logs: gateway/logs/gateway.log
Port Info: gateway/port.txt contains the active port.
GPU: Workers will automatically detect and use CUDA if available. Check nvidia-smi to monitor usage.

3.4 KiB Raw Blame History