Fix XTTS loader compatibility and add default voice
This commit is contained in:
133
GEMINI.md
Normal file
133
GEMINI.md
Normal file
@ -0,0 +1,133 @@
|
||||
# XTTS2 OpenAI-Compatible TTS Server
|
||||
|
||||
## Project Overview
|
||||
|
||||
This project is a high-performance, modular, and scalable Text-to-Speech (TTS) platform. It provides an API fully compatible with the **OpenAI Speech API**, powered by the **XTTS2** model for high-quality synthesis and zero-shot voice cloning.
|
||||
|
||||
### Architecture
|
||||
|
||||
The system follows a distributed architecture:
|
||||
|
||||
* **Gateway (`gateway/`):** A FastAPI service that handles HTTP requests, validates input, and manages the voice registry. It pushes synthesis jobs to a Redis queue. It features dynamic port selection (8000-8100).
|
||||
* **Redis:** Acts as the message broker (Queue) and cache between the Gateway and Workers.
|
||||
* **Worker (`worker/`):** A background service that pulls jobs from Redis, performs the actual TTS inference using XTTS2 (with GPU acceleration if available), and returns the audio data. These can be scaled horizontally.
|
||||
|
||||
### Key Technologies
|
||||
|
||||
* **Language:** Python 3.10+
|
||||
* **Framework:** FastAPI (Gateway)
|
||||
* **ML Model:** Coqui XTTS v2
|
||||
* **Infrastructure:** Docker, Docker Compose, Redis
|
||||
* **Tooling:** Makefile for orchestration
|
||||
|
||||
## Building and Running
|
||||
|
||||
The project relies heavily on `make` for orchestration.
|
||||
|
||||
### Docker (Recommended)
|
||||
|
||||
1. **Build Images:**
|
||||
```bash
|
||||
make build
|
||||
```
|
||||
|
||||
2. **Start Services:**
|
||||
```bash
|
||||
make up
|
||||
```
|
||||
* This runs a port scanner to find a free port between 8000-8100.
|
||||
* The chosen port is saved to `gateway/port.txt`.
|
||||
|
||||
3. **Check Status:**
|
||||
```bash
|
||||
make status
|
||||
```
|
||||
|
||||
4. **View Logs:**
|
||||
```bash
|
||||
make logs
|
||||
```
|
||||
|
||||
5. **Stop Services:**
|
||||
```bash
|
||||
make down
|
||||
```
|
||||
|
||||
### Scaling Workers
|
||||
|
||||
To handle higher load, you can spawn multiple worker containers:
|
||||
|
||||
```bash
|
||||
make worker-scale N=3
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
Run the self-test suite to verify Redis connectivity, worker processing, and audio synthesis:
|
||||
|
||||
```bash
|
||||
make selftest
|
||||
```
|
||||
|
||||
## Development Conventions
|
||||
|
||||
### Project Structure
|
||||
|
||||
* `gateway/`: Code for the API server.
|
||||
* `main.py`: Entry point.
|
||||
* `api/`: Endpoint definitions (`openai_speech.py`, `voices.py`).
|
||||
* `core/`: Configuration and utilities.
|
||||
* `worker/`: Code for the inference engine.
|
||||
* `engine/`: XTTS2 model loading and audio export logic.
|
||||
* `core/`: Queue processing and GPU detection.
|
||||
* `scripts/`: Utility scripts (e.g., `find_port.py`, `selftest.py`).
|
||||
|
||||
### Local Development (Non-Docker)
|
||||
|
||||
1. Create a virtual environment:
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
```
|
||||
2. Install dependencies:
|
||||
```bash
|
||||
pip install -r gateway/requirements.gateway.txt
|
||||
pip install -r worker/requirements.worker.txt
|
||||
```
|
||||
3. Run Redis locally (e.g., `docker run -p 6379:6379 redis:7`).
|
||||
4. Start Gateway: `python gateway/main.py`
|
||||
5. Start Worker: `python worker/main.py`
|
||||
|
||||
### API Usage
|
||||
|
||||
The API mirrors OpenAI's structure.
|
||||
|
||||
**Generate Audio:**
|
||||
```http
|
||||
POST /v1/audio/speech
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"model": "xtts-v2",
|
||||
"input": "Hello world",
|
||||
"voice": "auto",
|
||||
"format": "wav"
|
||||
}
|
||||
```
|
||||
|
||||
**Register Voice:**
|
||||
```http
|
||||
POST /v1/voices/register
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"name": "my-voice",
|
||||
"samples": ["https://example.com/sample.wav"]
|
||||
}
|
||||
```
|
||||
|
||||
### Logging & Debugging
|
||||
|
||||
* **Gateway Logs:** `gateway/logs/gateway.log`
|
||||
* **Port Info:** `gateway/port.txt` contains the active port.
|
||||
* **GPU:** Workers will automatically detect and use CUDA if available. Check `nvidia-smi` to monitor usage.
|
||||
Reference in New Issue
Block a user