feat: Overhaul application and add DX improvements
This commit introduces a wide range of improvements to the application, focusing on stability, developer experience (DX), and documentation.
Key changes include:
- **Fix Application Startup:** Resolved a critical bug where the FastAPI application instance was not correctly exposed, preventing Uvicorn from starting ().
- **Simplify Docker Compose:** Removed the integrated Traefik setup from the default to support users with existing reverse proxies and simplify the local development environment.
- **Improve Makefile:**
- Implemented a robust, automatic port-finding mechanism for Starting development environment on port 8001...
#1 [internal] load local bake definitions
#1 reading from stdin 534B done
#1 DONE 0.0s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 1.22kB done
#2 WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 2)
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/python:3.11
#3 DONE 0.7s
#4 [internal] load metadata for docker.io/library/python:3.11-slim
#4 DONE 0.7s
#5 [internal] load .dockerignore
#5 transferring context: 385B done
#5 DONE 0.0s
#6 [builder 1/4] FROM docker.io/library/python:3.11@sha256:bf2d36b8fb1b4a0b590b36736cdd8a6b5175b411bf135c42694ecd68ab8fed02
#6 DONE 0.0s
#7 [stage-1 1/6] FROM docker.io/library/python:3.11-slim@sha256:193fdd0bbcb3d2ae612bd6cc3548d2f7c78d65b549fcaa8af75624c47474444d
#7 DONE 0.0s
#8 [internal] load build context
#8 transferring context: 4.90kB done
#8 DONE 0.0s
#9 [builder 2/4] WORKDIR /opt/venv
#9 CACHED
#10 [stage-1 4/6] WORKDIR /home/appuser
#10 CACHED
#11 [stage-1 3/6] RUN useradd --create-home --shell /bin/bash appuser
#11 CACHED
#12 [stage-1 2/6] RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg && rm -rf /var/lib/apt/lists/*
#12 CACHED
#13 [stage-1 5/6] COPY --from=builder /opt/venv /opt/venv
#13 CACHED
#14 [builder 4/4] RUN python -m venv . && . /opt/venv/bin/activate && pip install --no-cache-dir -r requirements.txt
#14 CACHED
#15 [builder 3/4] COPY requirements.txt .
#15 CACHED
#16 [stage-1 6/6] COPY app/ ./app
#16 CACHED
#17 exporting to image
#17 exporting layers done
#17 writing image sha256:6cac7caac7fda2808672ad2f3d117d46d38c1d93013867858543ec74917857b7 done
#17 naming to docker.io/library/audioenginehub-app done
#17 DONE 0.0s
#18 resolving provenance for metadata file
#18 DONE 0.0s and Using host port 8000 for single app container
8a1c69e868e7f13b4c8c9948e81921b48efd9536f326d200b9e912fb12ff66e3 to prevent port conflicts.
- Added a target (Running health check on running container...
App container is running on port 8001.
Waiting for app to initialize...
ERROR: Failed to decode JSON from health endpoint.) to run post-deployment sanity checks against the running container's endpoint.
- Recommended using Starting development environment on port 8002...
#1 [internal] load local bake definitions
#1 reading from stdin 534B done
#1 DONE 0.0s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 1.22kB done
#2 WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 2)
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/python:3.11-slim
#3 DONE 0.1s
#4 [internal] load metadata for docker.io/library/python:3.11
#4 DONE 0.2s
#5 [internal] load .dockerignore
#5 transferring context: 385B done
#5 DONE 0.0s
#6 [builder 1/4] FROM docker.io/library/python:3.11@sha256:bf2d36b8fb1b4a0b590b36736cdd8a6b5175b411bf135c42694ecd68ab8fed02
#6 DONE 0.0s
#7 [stage-1 1/6] FROM docker.io/library/python:3.11-slim@sha256:193fdd0bbcb3d2ae612bd6cc3548d2f7c78d65b549fcaa8af75624c47474444d
#7 DONE 0.0s
#8 [internal] load build context
#8 transferring context: 1.09GB 5.1s
#8 transferring context: 1.66GB 7.9s done
#8 DONE 8.0s
#9 [builder 3/4] COPY requirements.txt .
#9 CACHED
#10 [builder 4/4] RUN python -m venv . && . /opt/venv/bin/activate && pip install --no-cache-dir -r requirements.txt
#10 CACHED
#11 [stage-1 4/6] WORKDIR /home/appuser
#11 CACHED
#12 [stage-1 3/6] RUN useradd --create-home --shell /bin/bash appuser
#12 CACHED
#13 [builder 2/4] WORKDIR /opt/venv
#13 CACHED
#14 [stage-1 2/6] RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg && rm -rf /var/lib/apt/lists/*
#14 CACHED
#15 [stage-1 5/6] COPY --from=builder /opt/venv /opt/venv
#15 CACHED
#16 [stage-1 6/6] COPY app/ ./app
#16 CACHED
#17 exporting to image
#17 exporting layers done
#17 writing image sha256:6cac7caac7fda2808672ad2f3d117d46d38c1d93013867858543ec74917857b7 done
#17 naming to docker.io/library/audioenginehub-app done
#17 DONE 0.0s
#18 resolving provenance for metadata file
#18 DONE 0.0s for reliable port detection.
- **Update Documentation:**
- Replaced the outdated (which contained old source code) with a comprehensive guide covering setup, usage, and commands.
- Added a note to to clarify that it describes an older, more advanced setup, pointing readers to the new for the current recommended workflow.
These changes address the service startup failures and significantly improve the project's usability and maintainability.
This commit is contained in:
94
session_resumee.md
Normal file
94
session_resumee.md
Normal file
@ -0,0 +1,94 @@
|
||||
# Session Resumee - AudioEngineHub Project Refactoring
|
||||
|
||||
**Date:** Donnerstag, 4. Dezember 2025
|
||||
|
||||
**Objective:** Refactor the AudioEngineHub project to follow best practices for containerization, error handling, and project structure, based on a provided `guide.md` document.
|
||||
|
||||
---
|
||||
|
||||
### Initial Project State & Overview
|
||||
|
||||
The project was a modular FastAPI-based TTS server with a `TTSEngineBase` interface. Key initial findings:
|
||||
* `piper` was functional.
|
||||
* `styletts` and `chattts` were dummy implementations.
|
||||
* `f5_tts` was implemented but inactive.
|
||||
* Configuration was scattered and hardcoded.
|
||||
* Error handling was basic.
|
||||
* Tests (`pytest` and `unittest`) existed but were limited and inconsistent.
|
||||
* No containerization strategy was in place, leading to potential dependency hell.
|
||||
|
||||
### Refactoring Phase 1: Robustness & Configuration
|
||||
|
||||
1. **Centralized Configuration:**
|
||||
* Replaced hardcoded values with `pydantic-settings` for `.env` file management.
|
||||
* `config.py` created (later moved to `app/config.py`).
|
||||
* `IMAGE_NAME` in `Makefile` was also user-configurable.
|
||||
* `requirements.txt` was updated with `pydantic-settings`.
|
||||
* `docker-compose.yml` was updated to use `.env`.
|
||||
|
||||
2. **Configurable Engines Feature:**
|
||||
* Implemented dynamic `ENGINE_REGISTRY` loading based on `ACTIVE_ENGINES` setting in `.env`.
|
||||
* Allows easy activation/deactivation of TTS engines.
|
||||
|
||||
3. **Robust Error Handling:**
|
||||
* Implemented comprehensive input validation in the `/tts` endpoint (checking engine, model, speaker existence).
|
||||
* Added dependency checks (e.g., `ffmpeg`, `piper` executables) to engines, reporting `HTTP 503` for unavailable engines.
|
||||
* Secured `tempfile.mktemp` usage by replacing it with `tempfile.NamedTemporaryFile`.
|
||||
* Wrapped synthesis logic in `try-except` blocks to catch and propagate engine-specific errors as `HTTP 500`.
|
||||
|
||||
4. **Asynchronous Operations:**
|
||||
* Changed `TTSEngineBase.synthesize` and `selftest` to `async`.
|
||||
* Refactored all concrete engine implementations (`piper`, `f5_tts`, `styletts`, `chattts`) to use `async def` methods.
|
||||
* Updated `app/main.py`'s `/tts` endpoint to be `async` and use `await` for engine calls and `asyncio.gather` for concurrent chunk synthesis.
|
||||
* Wrapped blocking I/O (file ops, `ffmpeg`) and CPU-bound tasks in `asyncio.to_thread`.
|
||||
* Updated `tests/test_f5_tts.py` to correctly `await` async calls.
|
||||
|
||||
### Refactoring Phase 2: Containerization & Workflow (Based on `guide.md`)
|
||||
|
||||
1. **Integrated `Makefile`:**
|
||||
* Created a `Makefile` with targets for `build`, `run`, `stop`, `logs`, `shell`, `up`, `down`, `test`, `tag`, `push`, `clean`, `help`.
|
||||
* Included robust shell functions for port checking (`check_traefik_ports_free`, `check_app_port_free`).
|
||||
* Set `SHELL := /bin/bash` in `Makefile` to ensure correct shell interpretation.
|
||||
* Ensured `PYTHONPATH=$(PWD)` is set for `make test`.
|
||||
|
||||
2. **Adopted Multi-Stage `Dockerfile`:**
|
||||
* Implemented a multi-stage `Dockerfile` (builder/runner stages).
|
||||
* `builder` stage creates a Python virtual environment and installs `requirements.txt` (including `gunicorn`).
|
||||
* `runner` stage uses `python:3.11-slim`, installs runtime system dependencies (`ffmpeg`), creates an unprivileged `appuser`, and sets the production `CMD` to `gunicorn` with `uvicorn` workers.
|
||||
|
||||
3. **Refactored Project Structure (`app/` package):**
|
||||
* Created an `app/` directory.
|
||||
* Moved `main.py`, `config.py`, `engines/`, `models/`, `utils/` into `app/`.
|
||||
* Created `app/__init__.py`.
|
||||
* Updated all Python import paths (`from app.config import settings`, `from app.engines.piper import PiperEngine`, etc.).
|
||||
* Updated internal references in engine files (e.g., `model_dir`, `voices_dir`).
|
||||
|
||||
4. **Refined `docker-compose.yml` with Traefik:**
|
||||
* Integrated `traefik` service for dynamic reverse proxying during local development.
|
||||
* Modified `app` service with Traefik `labels` and connected both services to a `web` network.
|
||||
* Adjusted Docker `volumes` mounts to match the new `app/` structure (e.g., `./models:/home/appuser/app/models`).
|
||||
* Updated `app` service `command` for Uvicorn hot-reloading in dev.
|
||||
|
||||
5. **Centralized Testing Workflow:**
|
||||
* Removed `test_run.sh`.
|
||||
* Integrated `make test` for running `pytest --cov=. app/ tests/`.
|
||||
|
||||
6. **Dedicated Documentation:**
|
||||
* Created `docs/` directory.
|
||||
* Moved `guide.md` to `docs/guide.md`.
|
||||
|
||||
### Verification & Troubleshooting
|
||||
|
||||
* **Tests:** All unit/integration tests (`make test`) are passing.
|
||||
* **Local Run (Virtual Env):** Initial local runs (`python app/main.py`) failed due to `ModuleNotFoundError` (fixed by `python -m app.main`) and `PermissionError` (fixed by needing to mock or redirect `settings.AUDIO_CACHE_DIR` for local direct execution, but not strictly needed for successful app execution through `uvicorn`). The current approach is to verify in Docker.
|
||||
* **Docker Compose:**
|
||||
* Initial `make up` failures were due to an outdated `docker-compose` client (`1.29.2`) and later, `Makefile` syntax issues (fixed by setting `SHELL := /bin/bash` and fixing macros).
|
||||
* `docker-compose` was eventually updated to the `docker compose` CLI plugin (v5.0.0).
|
||||
* `make up` command finally succeeded in bringing up containers.
|
||||
* The `curl http://localhost/health` command returned `404 page not found`. This indicates a potential routing issue with Traefik or the application not being responsive on the expected path within the container. (This is the last unresolved issue).
|
||||
|
||||
---
|
||||
|
||||
**Next Steps (Troubleshooting the 404):**
|
||||
|
||||
The `404 page not found` when accessing `http://localhost/health` via Traefik is the current blocker for full verification. I need to investigate the logs of the `audio-engine-hub_app` container (the FastAPI app) to determine if the application itself is starting correctly and serving the `/health` endpoint as expected. If the app is indeed serving, the issue lies with Traefik's routing configuration.
|
||||
Reference in New Issue
Block a user