diff --git a/README.md b/README.md index b682d42..5aecc46 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ weiter (Streaming, Tool Calls, JSON), schaltet zwischen drei festen llama.cpp-Profilen um, orchestriert lokale Bildgenerierung mit FLUX.2 [klein] 4B Base (GPU-Hotswap: Qwen stoppen → FLUX laden → Bild → FLUX entladen → Qwen wiederherstellen) und stellt lokale deutsche -Sprachausgabe bereit (Kokoro-82M, CPU-only, OpenAI-kompatibel). +Sprachausgabe bereit (XTTS-v2, CPU-only, OpenAI-kompatibel). ## Zielsystem @@ -18,7 +18,7 @@ Sprachausgabe bereit (Kokoro-82M, CPU-only, OpenAI-kompatibel). | Profil-Skript | `/usr/local/bin/llama-profile {fast\|medium\|long}` | | Router-Port | **8081** | | Router-Service | `mike-ai-profile-router.service` | -| TTS-Worker | `http://127.0.0.1:8082` (Service `mike-ai-kokoro.service`) | +| TTS-Worker | `http://127.0.0.1:8085` (Service `mike-ai-xtts.service`) | | STT-Worker | `http://127.0.0.1:8084` (Service `mike-ai-whisper.service`) | ## Profile / virtuelle Modelle @@ -40,7 +40,7 @@ Sprachausgabe bereit (Kokoro-82M, CPU-only, OpenAI-kompatibel). | `POST /v1/images/generations` | Bildgenerierung (FLUX.2 [klein] 4B Base, OpenAI-kompatibel) | | `GET /images` | Liste der gespeicherten Bilder (max. 200) | | `GET /images/` | PNG-Download (nur `images/`-Verzeichnis, validiert) | -| `POST /v1/audio/speech` | Deutsche Sprachausgabe (Kokoro-82M, OpenAI-kompatibel) | +| `POST /v1/audio/speech` | Sprachausgabe (XTTS-v2, OpenAI-kompatibel) | | `POST /v1/audio/transcriptions` | Deutsche Spracherkennung (whisper.cpp, OpenAI-kompatibel) | | `GET /v1/audio/models` | Verfügbare Audio-Modelle (STT + TTS) | | `GET /v1/audio/voices` | Verfügbare TTS-Stimmen | @@ -174,26 +174,21 @@ VRAM-Check). Das Venv liegt unter `/opt/mike-ai/ai-profile-router/venv/` und wird von `install.sh` automatisch angelegt/aktualisiert. -## Sprachausgabe (Kokoro-82M, deutsch, CPU-only) +## Sprachausgabe (XTTS-v2, CPU-only) -Der Router stellt lokale deutsche Sprachausgabe bereit. Die Synthese läuft -in einem **separaten, langlebigen Worker** (`mike-ai-kokoro.service`), der -die Kokoro-Modelle einmalig beim Start lädt und dauerhaft im RAM hält +Der Router stellt lokale Sprachausgabe bereit. Die Synthese läuft +in einem **separaten, langlebigen Worker** (`mike-ai-xtts.service`), der +das XTTS-v2-Modell einmalig beim Start lädt und dauerhaft im RAM hält (niedrige Warm-Start-Latenz). Der Worker ist CPU-only und blockiert weder Qwen/llama.cpp noch FLUX/GPU – er teilt sich nur den Prozessor. -- **Modell:** Kokoro-82M (hexgrad) mit zwei deutschen Feintunings - (`kikiri-tts/kikiri-german-martin`, `kikiri-tts/kikiri-german-victoria`, - beide Apache 2.0, je ~327 MB). -- **G2P:** Offizieller Kikiri-Deutsche-Pfad via `misaki.de.DEG2P` - (semidark/misaki-Fork 0.9.4): Text-Normalisierung (Zahlen, Daten, - Uhrzeiten, Währung, Abkürzungen) + `espeak-ng`-Phonemisierung + - Aussprache-Overrides für Marken-/Tech-Begriffe. Kein spacy nötig. -- **Pipeline:** semidark/kokoro-Fork (0.9.4) mit `lang_code='d'` und - verbessertem Chunking (Split an Satzgrenzen, 400 Zeichen). -- **Venv:** eigenes Venv unter `/opt/mike-ai/kokoro/venv/` mit CPU-only - `torch` (keine CUDA-Abhängigkeit, kein Konflikt mit dem Bild-Venv). -- **Modelle:** `/opt/mike-ai/models/kokoro/` (persistent). +- **Modell:** Coqui XTTS-v2 (`tts_models/multilingual/multi-dataset/xtts_v2`, + ~1.9 GB, CPML-Lizenz). +- **Stimme:** `claribel` (Claribel Dervla) – natürliche weibliche Stimme, + unterstützt Deutsch und Englisch. +- **Venv:** eigenes Venv unter `/opt/mike-ai/xtts/venv/` mit Python 3.11 + (Coqui TTS unterstützt kein Python 3.13) und CPU-only `torch`. +- **Modelle:** HuggingFace-Cache unter `/opt/mike-ai/xtts/.cache/` (persistent). ### Endpunkt `POST /v1/audio/speech` @@ -203,34 +198,31 @@ OpenAI-kompatibel. Unterstützt `input` (oder `text`), `voice`, `speed`, | Parameter | Werte | Default | |---|---|---| | `input` | Text (Pflicht, max. 8000 Zeichen) | – | -| `voice` | `martin`, `victoria` | `martin` | +| `voice` | `claribel` | `claribel` | | `speed` | 0.5–2.0 | 1.0 | -| `response_format` | `mp3` (Default), `wav`, `flac`, `pcm` | `mp3` | -| `model` | `kokoro-german` (optional) | – | +| `response_format` | `mp3` (Default), `wav` | `mp3` | +| `model` | `xtts-v2` (optional) | – | Die Antwort ist **binäres Audio** (nicht JSON) mit passendem -`Content-Type` (`audio/mpeg`, `audio/wav`, `audio/flac`, -`application/octet-stream`). +`Content-Type` (`audio/mpeg`, `audio/wav`). Beispiele: ```bash -# MP3 (Default), Stimme martin +# MP3 (Default), Stimme claribel curl -s http://192.168.1.196:8081/v1/audio/speech \ -H 'Content-Type: application/json' \ - -d '{"input":"Hallo, dies ist ein Test.","voice":"martin"}' -o out.mp3 + -d '{"input":"Hallo, dies ist ein Test.","voice":"claribel"}' -o out.mp3 -# WAV, Stimme victoria, 1.5x Tempo +# WAV, 1.5x Tempo curl -s http://192.168.1.196:8081/v1/audio/speech \ -H 'Content-Type: application/json' \ - -d '{"input":"Guten Tag.","voice":"victoria","speed":1.5,"response_format":"wav"}' -o out.wav + -d '{"input":"Guten Tag.","voice":"claribel","speed":1.5,"response_format":"wav"}' -o out.wav ``` -**Lange Texte:** Die Pipeline splittet automatisch an Satzgrenzen -(400 Zeichen pro Chunk). Längere Texte werden in Segmente geteilt; +**Lange Texte:** XTTS-v2 verarbeitet den Text intern in Sätze. +Längere Texte werden automatisch in Segmente geteilt; jedes Segment wird einzeln synthetisiert und die Audios zusammengeführt. -Zeilenumbrüche (`\n`) im Text werden als zusätzliche Segmentgrenzen -behandelt. ### Verhalten @@ -240,49 +232,41 @@ behandelt. - **Serialisierte Synthese:** Der Worker synthetisiert nacheinander (CPU-bound), parallele Requests werden intern gewartet. - **`/status`** zeigt `tts.reachable`, `tts.ready`, `tts.voices`, - `tts.load_errors`, `tts.last_seconds`, `tts.last_voice`, - `tts.last_error`. + `tts.last_seconds`, `tts.last_voice`, `tts.last_error`. - **Fehler:** Worker down → `503` (`tts_failed`); ungültige Parameter → `400`. OpenAI-kompatibles Fehlerformat. -### Hörproben - -Zwei deutsche Hörproben (je ~40 s) liegen unter -`/opt/mike-ai/ai-profile-router/samples/`: - -- `martin_lang.wav` (Stimme martin) -- `victoria_lang.wav` (Stimme victoria) - ### Benchmark (CPU-only, gemessen) | Textlänge | Audio | Synthese | RTF | |---|---|---|---| -| ~5 s | 3.4 s | 0.6 s | 0.19× | -| ~15 s | 14.1 s | 3.4 s | 0.24× | -| ~40 s | 33.5 s | 7.5 s | 0.23× | +| ~6 s | 6.2 s | 8.2 s | 1.32× | +| ~15 s | 15.0 s | 19.8 s | 1.32× | -RTF ~0.23 bedeutet: Synthese ist ~4.3× schneller als Echtzeit. -RAM-Belegung des Workers: ~3.5 GB (inkl. torch, beide Modelle). +RTF ~1.32 bedeutet: Synthese ist ~1.3× langsamer als Echtzeit. +RAM-Belegung des Workers: ~4.5 GB (inkl. torch, XTTS-v2-Modell). -### Erforderliche Python-Pakete (im Kokoro-Venv) +### Erforderliche Python-Pakete (im XTTS-Venv) - `torch` (CPU-only, `--index-url https://download.pytorch.org/whl/cpu`) -- `kokoro` (0.9.4, semidark-Fork, mit `--no-deps` installiert) -- `misaki` (0.9.4, semidark-Fork, mit `[de]`-Extra → `phonemizer-fork` + - `espeakng-loader`, kein spacy-curated-transformers) -- `spacy` (nur für `misaki.en`-Import, nicht für deutschen Pfad) -- `num2words` (für `misaki.en`-Import) -- `soundfile`, `lameenc` (Audio-Formate wav/mp3/flac/pcm) -- `huggingface-hub`, `loguru`, `transformers`, `regex` +- `torchaudio` (CPU-only) +- `TTS` (0.22.0, Coqui XTTS) +- `transformers` (4.40.2, für `BeamSearchScorer`) +- `tokenizers` (0.19.1) +- `huggingface-hub` (0.36.2) +- `librosa` (Audio-Verarbeitung) +- `soundfile` (WAV-Export) -Das Venv liegt unter `/opt/mike-ai/kokoro/venv/` und wird von `install.sh` +Das Venv liegt unter `/opt/mike-ai/xtts/venv/` und wird von `install.sh` automatisch angelegt (nur wenn noch nicht vorhanden). -**Hinweis (Python 3.13):** Der semidark/kokoro-Fork (0.9.4) unterstützt -Python 3.13 nativ. Der semidark/misaki-Fork (0.9.4) nutzt für den deutschen -Pfad `phonemizer-fork` + `espeakng-loader` (kein spacy-curated-transformers, -kein thinc 9.x). `spacy` wird nur für den `misaki.en`-Import benötigt -(Englisch), nicht für den deutschen Pfad. +**Hinweis (Python 3.11):** Coqui TTS 0.22.0 unterstützt offiziell nur +Python `>=3.9.0, <3.12`. Daher wird Python 3.11.16 via `uv` verwendet. +Der Zielsystem-Python (3.13.5) wird nicht verwendet. + +**Hinweis (PyTorch 2.6+):** Coqui TTS nutzt `torch.load()` ohne +`weights_only=False`, was in PyTorch 2.6+ standardmäßig fehlschlägt. +Dies wird durch einen Patch in `TTS/utils/io.py` umgangen. ## Spracherkennung (whisper.cpp, deutsch, CPU-only) @@ -340,9 +324,9 @@ curl -s http://192.168.1.196:8081/v1/audio/transcriptions \ ### Discovery-Endpunkte - `GET /v1/audio/models` – listet verfügbare Audio-Modelle - (`whisper-1` für STT, `kokoro-german` für TTS) + (`whisper-1` für STT, `xtts-v2` für TTS) - `GET /v1/audio/voices` – listet verfügbare TTS-Stimmen - (`martin`, `victoria`) + (`claribel`) ### Verhalten @@ -379,10 +363,10 @@ RAM-Belegung des Workers: ~1.7 GB (inkl. Modell). ``` router/ai_profile_router.py # der Router (einzige Laufzeit-Datei) router/image_worker.py # FLUX-Worker (eigener Prozess, JSON-Protokoll) -router/tts_worker.py # Kokoro-TTS-Worker (eigener Prozess, HTTP-API) +router/xtts_worker.py # XTTS-v2-TTS-Worker (eigener Prozess, HTTP-API) router/stt_worker.py # Whisper-STT-Worker (eigener Prozess, HTTP-API) deploy/mike-ai-profile-router.service # systemd-Unit (Router) -deploy/mike-ai-kokoro.service # systemd-Unit (TTS-Worker) +deploy/mike-ai-xtts.service # systemd-Unit (TTS-Worker) deploy/mike-ai-whisper.service # systemd-Unit (STT-Worker) deploy/install.sh # läuft auf dem Zielsystem (per SSH) deploy/deploy.sh # läuft lokal: SCP + SSH @@ -415,12 +399,10 @@ Das Skript: - das FLUX-Modell nach `/opt/mike-ai/models/FLUX.2-klein-base-4B` lädt (nur wenn noch nicht vorhanden, ~15 GB), - `espeak-ng` installiert (nur wenn noch nicht vorhanden), - - das Kokoro-Venv unter `/opt/mike-ai/kokoro/venv/` anlegt (nur wenn - noch nicht vorhanden, CPU-only torch + kokoro + misaki + phonemizer + - soundfile + lameenc), - - die Kokoro-Modelle nach `/opt/mike-ai/models/kokoro/` lädt (nur wenn - noch nicht vorhanden, ~660 MB), - - `mike-ai-profile-router.service` und `mike-ai-kokoro.service` + - das XTTS-Venv unter `/opt/mike-ai/xtts/venv/` anlegt (nur wenn + noch nicht vorhanden, Python 3.11 via uv + CPU-only torch + Coqui TTS), + - das XTTS-v2-Modell herunterlädt (nur wenn noch nicht vorhanden, ~1.9 GB), + - `mike-ai-profile-router.service` und `mike-ai-xtts.service` aktivieren (Start beim Boot) und starten, - `GET /status` verifiziert. @@ -448,17 +430,18 @@ Die Installation ist idempotent (Update = erneut ausführen). | `IMAGE_GEN_TIMEOUT` | `600` | Timeout pro Bild (s) | | `IMAGE_VRAM_FREE_TIMEOUT` | `120` | Warten auf VRAM-Freiheit (s) | | `CHAT_WAIT_TIMEOUT` | `300` | Chat wartet auf Qwen (s) | -| `TTS_WORKER_URL` | `http://127.0.0.1:8082` | TTS-Worker (Router-Seite) | +| `TTS_WORKER_URL` | `http://127.0.0.1:8085` | TTS-Worker (Router-Seite) | | `TTS_TIMEOUT` | `300` | Timeout pro Synthese (s) | | `TTS_CONNECT_TIMEOUT` | `5` | Connect-Timeout TTS-Worker (s) | -TTS-Worker (`mike-ai-kokoro.service`): +TTS-Worker (`mike-ai-xtts.service`): | Variable | Default | Bedeutung | |---|---|---| -| `KOKORO_HOST` | `127.0.0.1` | Bind-Adresse (nur lokal, Router proxyt) | -| `KOKORO_PORT` | `8082` | Port | -| `KOKORO_MODEL_DIR` | `/opt/mike-ai/models/kokoro` | Modell-Verzeichnis | +| `XTTS_HOST` | `127.0.0.1` | Bind-Adresse (nur lokal, Router proxyt) | +| `XTTS_PORT` | `8085` | Port | +| `XTTS_MODEL_ID` | `tts_models/multilingual/multi-dataset/xtts_v2` | Modell-ID | +| `XTTS_DEFAULT_VOICE` | `claribel` | Default-Stimme | | `LOG_LEVEL` | `INFO` | Logging-Level | ## Lokale Tests @@ -484,15 +467,15 @@ Aktuell: **43 Tests** (32 bestehende + 11 TTS-Assertions). ```bash systemctl status mike-ai-profile-router -systemctl status mike-ai-kokoro +systemctl status mike-ai-xtts journalctl -u mike-ai-profile-router -f -journalctl -u mike-ai-kokoro -f +journalctl -u mike-ai-xtts -f curl -s http://192.168.1.196:8081/status | python3 -m json.tool curl -s -X POST http://192.168.1.196:8081/medium # TTS-Test curl -s http://192.168.1.196:8081/v1/audio/speech \ -H 'Content-Type: application/json' \ - -d '{"input":"Hallo","voice":"martin"}' -o test.mp3 + -d '{"input":"Hallo","voice":"claribel"}' -o test.mp3 ``` ## Sicherheit diff --git a/deploy/deploy.sh b/deploy/deploy.sh index bb2e347..174721b 100755 --- a/deploy/deploy.sh +++ b/deploy/deploy.sh @@ -9,10 +9,10 @@ SSH_KEY="${SSH_KEY:-$HOME/.ssh/lmstudio_unraid}" STAGE="/tmp/ai-profile-router-$$" mkdir -p "$STAGE" -cp router/ai_profile_router.py router/image_worker.py router/tts_worker.py \ +cp router/ai_profile_router.py router/image_worker.py router/xtts_worker.py \ router/stt_worker.py \ deploy/install.sh deploy/mike-ai-profile-router.service \ - deploy/mike-ai-kokoro.service deploy/mike-ai-whisper.service "$STAGE/" + deploy/mike-ai-xtts.service deploy/mike-ai-whisper.service "$STAGE/" echo "== Übertrage Dateien nach ${TARGET}:/tmp/ai-profile-router/" ssh -i "$SSH_KEY" "$TARGET" 'mkdir -p /tmp/ai-profile-router' diff --git a/deploy/install.sh b/deploy/install.sh index 7c3660e..4fbcd01 100755 --- a/deploy/install.sh +++ b/deploy/install.sh @@ -1,15 +1,15 @@ #!/bin/bash # AI Profile Router – Installation/Update auf dem Zielsystem. # Wird als root auf dem Zielsystem ausgeführt (per SSH, vgl. deploy.sh). -# Erwartet ai_profile_router.py, image_worker.py, tts_worker.py, -# mike-ai-profile-router.service und mike-ai-kokoro.service im selben +# Erwartet ai_profile_router.py, image_worker.py, xtts_worker.py, +# mike-ai-profile-router.service und mike-ai-xtts.service im selben # Verzeichnis wie dieses Skript. set -euo pipefail DIR="$(cd "$(dirname "$0")" && pwd)" INSTALL_DIR=/opt/mike-ai/ai-profile-router SERVICE=mike-ai-profile-router.service -KOKORO_SERVICE=mike-ai-kokoro.service +XTTS_SERVICE=mike-ai-xtts.service WHISPER_SERVICE=mike-ai-whisper.service OLD_SERVICE=mike-ai-local-llm-router.service OLD_DIR=/opt/mike-ai/local-llm-router @@ -17,8 +17,8 @@ BACKUP_DIR=/opt/mike-ai/.backup-ai-profile-router-$(date +%Y%m%d-%H%M%S) VENV="$INSTALL_DIR/venv" MODEL_DIR=/opt/mike-ai/models/FLUX.2-klein-base-4B IMAGE_DIR="$INSTALL_DIR/images" -KOKORO_VENV=/opt/mike-ai/kokoro/venv -KOKORO_MODEL_DIR=/opt/mike-ai/models/kokoro +XTTS_VENV=/opt/mike-ai/xtts/venv +XTTS_CACHE=/opt/mike-ai/xtts/.cache echo "== AI Profile Router: Installation/Update ==" @@ -40,10 +40,10 @@ fi mkdir -p "$INSTALL_DIR" "$IMAGE_DIR" install -m 0755 "$DIR/ai_profile_router.py" "$INSTALL_DIR/ai_profile_router.py" install -m 0755 "$DIR/image_worker.py" "$INSTALL_DIR/image_worker.py" -install -m 0755 "$DIR/tts_worker.py" "$INSTALL_DIR/tts_worker.py" +install -m 0755 "$DIR/xtts_worker.py" "$INSTALL_DIR/xtts_worker.py" install -m 0755 "$DIR/stt_worker.py" "$INSTALL_DIR/stt_worker.py" install -m 0644 "$DIR/${SERVICE}" "/etc/systemd/system/${SERVICE}" -install -m 0644 "$DIR/${KOKORO_SERVICE}" "/etc/systemd/system/${KOKORO_SERVICE}" +install -m 0644 "$DIR/${XTTS_SERVICE}" "/etc/systemd/system/${XTTS_SERVICE}" install -m 0644 "$DIR/${WHISPER_SERVICE}" "/etc/systemd/system/${WHISPER_SERVICE}" # --- 3. Python-Venv mit Bild-Abhängigkeiten --------------------------------- @@ -76,69 +76,61 @@ print("Modell-Download abgeschlossen") PY fi -# --- 5. TTS (Kokoro) ---------------------------------------------------------- -# espeak-ng wird für die deutsche G2P (phonemizer) benötigt. -if ! command -v espeak-ng >/dev/null 2>&1; then - echo "-- Installiere espeak-ng" - apt-get install -y espeak-ng -fi - -# Kokoro-Venv (CPU-only torch + semidark/kokoro + semidark/misaki + soundfile + lameenc) -if [ ! -x "$KOKORO_VENV/bin/python" ]; then - echo "-- Erstelle Kokoro-Venv in $KOKORO_VENV" - mkdir -p "$KOKORO_VENV" - python3 -m venv "$KOKORO_VENV" - "$KOKORO_VENV/bin/pip" install --quiet --upgrade pip +# --- 5. TTS (XTTS-v2) ---------------------------------------------------------- +# XTTS-Venv (Python 3.11 via uv + CPU-only torch + Coqui TTS) +if [ ! -x "$XTTS_VENV/bin/python" ]; then + echo "-- Erstelle XTTS-Venv in $XTTS_VENV (Python 3.11 via uv)" + mkdir -p "$XTTS_VENV" + # uv installieren (falls noch nicht vorhanden) + if ! command -v uv >/dev/null 2>&1; then + curl -LsSf https://astral.sh/uv/install.sh | sh + export PATH="$HOME/.local/bin:$PATH" + fi + uv venv --python 3.11 "$XTTS_VENV" + "$XTTS_VENV/bin/pip" install --quiet --upgrade pip echo "-- Installiere CPU-only torch" - "$KOKORO_VENV/bin/pip" install --quiet torch \ + "$XTTS_VENV/bin/pip" install --quiet torch torchaudio \ --index-url https://download.pytorch.org/whl/cpu - echo "-- Installiere Kokoro-Abhängigkeiten" - "$KOKORO_VENV/bin/pip" install --quiet \ - huggingface-hub loguru transformers regex num2words \ - soundfile lameenc - # Offizieller Kikiri-Deutsche-Pfad: - # - semidark/misaki-Fork (0.9.4) mit [de]-Extra (phonemizer-fork + espeakng-loader) - # - semidark/kokoro-Fork (0.9.4) mit --no-deps (vermeidet misaki[en] → spacy-curated-transformers) - # - spacy + num2words für misaki.en-Import (Englisch, nicht für deutschen Pfad) - "$KOKORO_VENV/bin/pip" install --quiet "spacy<4" - "$KOKORO_VENV/bin/pip" install --quiet \ - "misaki[de] @ git+https://github.com/semidark/misaki.git" - "$KOKORO_VENV/bin/pip" install --quiet \ - "kokoro @ git+https://github.com/semidark/kokoro.git" --no-deps + echo "-- Installiere Coqui TTS + Abhängigkeiten" + "$XTTS_VENV/bin/pip" install --quiet \ + TTS==0.22.0 transformers==4.40.2 tokenizers==0.19.1 \ + huggingface-hub==0.36.2 librosa soundfile + # PyTorch 2.6+ weights_only-Patch für Coqui TTS + echo "-- Wende weights_only-Patch für Coqui TTS an" + "$XTTS_VENV/bin/python" - <<'PY' +import os +patch_file = os.path.expanduser("/opt/mike-ai/xtts/venv/lib/python3.11/site-packages/TTS/utils/io.py") +with open(patch_file, 'r') as f: + content = f.read() +if 'weights_only=False' not in content: + content = content.replace( + 'return torch.load(f, map_location=map_location, **kwargs)', + 'return torch.load(f, map_location=map_location, weights_only=False, **kwargs)' + ) + with open(patch_file, 'w') as f: + f.write(content) + print("Patch angewendet") +else: + print("Patch bereits vorhanden") +PY fi -# Kokoro-Modelle (kikiri-german-martin + kikiri-german-victoria, Apache 2.0) -if [ ! -f "$KOKORO_MODEL_DIR/kikiri-german-martin/kikiri_german_martin_ep10.pth" ] \ - || [ ! -f "$KOKORO_MODEL_DIR/kikiri-german-victoria/kikiri_german_victoria_ep10.pth" ]; then - echo "-- Lade Kokoro-Modelle nach $KOKORO_MODEL_DIR (kann dauern)" - "$KOKORO_VENV/bin/python" - <<'PY' +# XTTS-v2-Modell (nur wenn noch nicht vorhanden) +if [ ! -d "$XTTS_CACHE/hub/models--coqui--tts_models--multilingual--multi-dataset--xtts_v2" ]; then + echo "-- Lade XTTS-v2-Modell (kann dauern, ~1.9 GB)" + "$XTTS_VENV/bin/python" - <<'PY' import os -from huggingface_hub import hf_hub_download - -base = "/opt/mike-ai/models/kokoro" -files = [ - ("kikiri-tts/kikiri-german-martin", "kikiri_german_martin_ep10.pth", "kikiri-german-martin"), - ("kikiri-tts/kikiri-german-martin", "voices/martin.pt", "kikiri-german-martin"), - ("kikiri-tts/kikiri-german-victoria", "kikiri_german_victoria_ep10.pth", "kikiri-german-victoria"), - ("kikiri-tts/kikiri-german-victoria", "voices/victoria.pt", "kikiri-german-victoria"), - ("hexgrad/Kokoro-82M", "config.json", "kikiri-german-martin"), - ("hexgrad/Kokoro-82M", "config.json", "kikiri-german-victoria"), -] -for repo, fname, subdir in files: - dest = os.path.join(base, subdir, fname) - if os.path.exists(dest) and os.path.getsize(dest) > 1000: - print(f"vorhanden: {subdir}/{fname}") - continue - os.makedirs(os.path.dirname(dest), exist_ok=True) - hf_hub_download(repo_id=repo, filename=fname, local_dir=os.path.join(base, subdir)) - print(f"geladen: {subdir}/{fname}") +os.environ["HF_HOME"] = "/opt/mike-ai/xtts/.cache" +from TTS.api import TTS +tts = TTS("tts_models/multilingual/multi-dataset/xtts_v2") +print("Modell geladen") PY fi # --- 6. Services aktivieren und starten --------------------------------------- systemctl daemon-reload -systemctl enable "$SERVICE" "$KOKORO_SERVICE" "$WHISPER_SERVICE" -systemctl restart "$KOKORO_SERVICE" +systemctl enable "$SERVICE" "$XTTS_SERVICE" "$WHISPER_SERVICE" +systemctl restart "$XTTS_SERVICE" systemctl restart "$WHISPER_SERVICE" systemctl restart "$SERVICE" @@ -150,12 +142,12 @@ if ! systemctl is-active --quiet "$SERVICE"; then exit 1 fi echo "-- Router-Service läuft" -if ! systemctl is-active --quiet "$KOKORO_SERVICE"; then - echo "-- FEHLER: Kokoro-Service läuft nicht" >&2 - journalctl -u "$KOKORO_SERVICE" -n 20 --no-pager >&2 +if ! systemctl is-active --quiet "$XTTS_SERVICE"; then + echo "-- FEHLER: XTTS-Service läuft nicht" >&2 + journalctl -u "$XTTS_SERVICE" -n 20 --no-pager >&2 exit 1 fi -echo "-- Kokoro-Service läuft" +echo "-- XTTS-Service läuft" if ! systemctl is-active --quiet "$WHISPER_SERVICE"; then echo "-- FEHLER: Whisper-Service läuft nicht" >&2 journalctl -u "$WHISPER_SERVICE" -n 20 --no-pager >&2 diff --git a/deploy/mike-ai-xtts.service b/deploy/mike-ai-xtts.service new file mode 100644 index 0000000..a85a427 --- /dev/null +++ b/deploy/mike-ai-xtts.service @@ -0,0 +1,18 @@ +[Unit] +Description=XTTS-v2 TTS Worker (CPU-only) +After=network.target + +[Service] +Type=simple +User=root +WorkingDirectory=/opt/mike-ai/xtts +ExecStart=/opt/mike-ai/xtts/venv/bin/python /opt/mike-ai/ai-profile-router/xtts_worker.py +Restart=on-failure +RestartSec=5 +Environment=PATH=/opt/mike-ai/xtts/venv/bin:/usr/local/bin:/usr/bin:/bin +Environment=HOME=/root +# CPU-only, keine GPU +Environment=CUDA_VISIBLE_DEVICES= + +[Install] +WantedBy=multi-user.target diff --git a/dev/mock_tts_worker.py b/dev/mock_tts_worker.py index 9d6906f..72b8e0d 100644 --- a/dev/mock_tts_worker.py +++ b/dev/mock_tts_worker.py @@ -71,8 +71,8 @@ class Handler(BaseHTTPRequestHandler): self._send_json(200, { "status": "ok", "ready": True, - "voices": ["martin", "victoria"], - "default_voice": "martin", + "voices": ["claribel"], + "default_voice": "claribel", "load_errors": [], "sample_rate": SAMPLE_RATE, "uptime_seconds": 1.0, diff --git a/dev/test_local.sh b/dev/test_local.sh index fe70565..9f19069 100755 --- a/dev/test_local.sh +++ b/dev/test_local.sh @@ -427,14 +427,14 @@ d=json.load(sys.stdin) tts=d["tts"] assert tts["reachable"] is True, tts assert tts["ready"] is True, tts -assert set(tts["voices"])=={"martin","victoria"}, tts +assert set(tts["voices"])=={"claribel"}, tts ' && ok "Status: TTS erreichbar, bereit, 2 Stimmen" || bad "Status tts-Section" # --- 28. TTS: POST /v1/audio/speech (wav) --------------------------------------------------------------- echo "== Test 28: POST /v1/audio/speech (wav)" CODE=$(curl -s -o /tmp/tts28.wav -w "%{http_code}" -D /tmp/hdr28.txt \ "$BASE/v1/audio/speech" -H "Content-Type: application/json" \ - -d '{"model":"kokoro-german","input":"Hallo Welt","voice":"martin","response_format":"wav"}') + -d '{"model":"xtts-v2","input":"Hallo Welt","voice":"claribel","response_format":"wav"}') CTYPE=$(grep -i content-type /tmp/hdr28.txt | tr -d "\r") [ "$CODE" = "200" ] && [ -s /tmp/tts28.wav ] && echo "$CTYPE" | grep -qi "audio/wav" \ && ok "TTS wav (200, $CTYPE, $(stat -f%z /tmp/tts28.wav 2>/dev/null || stat -c%s /tmp/tts28.wav) Bytes)" \ @@ -444,7 +444,7 @@ CTYPE=$(grep -i content-type /tmp/hdr28.txt | tr -d "\r") echo "== Test 29: POST /v1/audio/speech (mp3, Default)" CODE=$(curl -s -o /tmp/tts29.mp3 -w "%{http_code}" -D /tmp/hdr29.txt \ "$BASE/v1/audio/speech" -H "Content-Type: application/json" \ - -d '{"input":"Guten Tag","voice":"victoria"}') + -d '{"input":"Guten Tag","voice":"claribel"}') CTYPE=$(grep -i content-type /tmp/hdr29.txt | tr -d "\r") [ "$CODE" = "200" ] && [ -s /tmp/tts29.mp3 ] && echo "$CTYPE" | grep -qi "audio/mpeg" \ && ok "TTS mp3 (200, $CTYPE)" || bad "TTS mp3 (Code $CODE, $CTYPE)" @@ -452,7 +452,7 @@ CTYPE=$(grep -i content-type /tmp/hdr29.txt | tr -d "\r") # --- 30. TTS: Validierung -------------------------------------------------------------------------------- echo "== Test 30: TTS-Validierung" CODE=$(curl -s -o /tmp/err30a.json -w "%{http_code}" "$BASE/v1/audio/speech" \ - -H "Content-Type: application/json" -d '{"voice":"martin"}') + -H "Content-Type: application/json" -d '{"voice":"claribel"}') cat /tmp/err30a.json; echo [ "$CODE" = "400" ] && ok "400 bei fehlendem input" || bad "erwartet 400, bekam $CODE" @@ -474,7 +474,7 @@ cat /tmp/err30d.json; echo # --- 31. TTS: Worker-Fehler → 503 ------------------------------------------------------------------------ echo "== Test 31: TTS-Worker-Fehler → 503" CODE=$(curl -s -o /tmp/err31.json -w "%{http_code}" "$BASE/v1/audio/speech" \ - -H "Content-Type: application/json" -d '{"input":"FAIL","voice":"martin"}') + -H "Content-Type: application/json" -d '{"input":"FAIL","voice":"claribel"}') cat /tmp/err31.json; echo [ "$CODE" = "503" ] && ok "503 bei TTS-Worker-Fehler" || bad "erwartet 503, bekam $CODE" @@ -483,7 +483,7 @@ echo "== Test 32: TTS-Worker down → 503" kill "$TTS_PID" 2>/dev/null || true sleep 0.5 CODE=$(curl -s -o /tmp/err32.json -w "%{http_code}" "$BASE/v1/audio/speech" \ - -H "Content-Type: application/json" -d '{"input":"Hallo","voice":"martin"}') + -H "Content-Type: application/json" -d '{"input":"Hallo","voice":"claribel"}') cat /tmp/err32.json; echo [ "$CODE" = "503" ] && ok "503 bei downem TTS-Worker" || bad "erwartet 503, bekam $CODE" RESP=$(curl -sf "$BASE/status") @@ -500,7 +500,7 @@ MOCK_TTS_PORT="$TTS_PORT" MOCK_TTS_DELAY=0.1 \ TTS_PID=$! sleep 0.5 CODE=$(curl -s -o /tmp/tts33.wav -w "%{http_code}" "$BASE/v1/audio/speech" \ - -H "Content-Type: application/json" -d '{"input":"Wieder da","voice":"martin","response_format":"wav"}') + -H "Content-Type: application/json" -d '{"input":"Wieder da","voice":"claribel","response_format":"wav"}') [ "$CODE" = "200" ] && [ -s /tmp/tts33.wav ] \ && ok "TTS nach Neustart wieder verfügbar" || bad "TTS-Recovery (Code $CODE)" @@ -593,8 +593,8 @@ import json,sys d=json.load(sys.stdin) ids={m["id"] for m in d["data"]} assert "whisper-1" in ids, ids -assert "kokoro-german" in ids, ids -' && ok "Audio-Modelle: whisper-1 + kokoro-german" || bad "Audio-Modelle" +assert "xtts-v2" in ids, ids +' && ok "Audio-Modelle: whisper-1 + xtts-v2" || bad "Audio-Modelle" # --- 41. /v1/audio/voices ------------------------------------------------------------------------------------------ echo "== Test 41: GET /v1/audio/voices" @@ -604,9 +604,8 @@ echo "$RESP" | python3 -c ' import json,sys d=json.load(sys.stdin) ids={v["id"] for v in d["data"]} -assert "martin" in ids, ids -assert "victoria" in ids, ids -' && ok "Audio-Voices: martin + victoria" || bad "Audio-Voices" +assert "claribel" in ids, ids +' && ok "Audio-Voices: claribel" || bad "Audio-Voices" # --- 42. STT + Qwen parallel ---------------------------------------------------------------------------------------- echo "== Test 42: STT + Qwen parallel" @@ -637,7 +636,7 @@ sleep 0.2 # TTS-Request CODE=$(curl -s -o /tmp/tts43.mp3 -w "%{http_code}" \ "$BASE/v1/audio/speech" -H "Content-Type: application/json" \ - -d '{"input":"Hallo","voice":"martin"}') + -d '{"input":"Hallo","voice":"claribel"}') wait $STT_PID43 [ "$CODE" = "200" ] && [ -s /tmp/tts43.mp3 ] \ && ok "STT + TTS parallel (beide 200)" || bad "STT + TTS parallel (TTS Code $CODE)" diff --git a/router/ai_profile_router.py b/router/ai_profile_router.py index cf6a6f3..16b14ba 100755 --- a/router/ai_profile_router.py +++ b/router/ai_profile_router.py @@ -20,7 +20,7 @@ Bildgenerierung (FLUX.2 [klein] 4B Base): GET /images (Liste) GET /images/ (PNG-Download) -Sprachausgabe (Kokoro-82M, deutsch, CPU-only): +Sprachausgabe (XTTS-v2, multilingual, CPU-only): POST /v1/audio/speech (OpenAI-kompatibel) GET /v1/audio/voices (verfügbare Stimmen) @@ -28,7 +28,7 @@ Spracherkennung (whisper.cpp, deutsch, CPU-only): POST /v1/audio/transcriptions (OpenAI-kompatibel) GET /v1/audio/models (verfügbare Audio-Modelle) -Der TTS-Worker (mike-ai-kokoro.service) und der STT-Worker +Der TTS-Worker (mike-ai-xtts.service) und der STT-Worker (mike-ai-whisper.service) laufen als separate, langlebige Prozesse. Der Router leitet /v1/audio/speech und /v1/audio/transcriptions per HTTP an die Worker weiter. @@ -108,14 +108,14 @@ IMAGE_QUALITY = {"standard": 30, "high": 50} IMAGE_DEFAULT_QUALITY = "standard" IMAGE_MAX_N = 4 -# --- Sprachausgabe (Kokoro-82M, deutsch, CPU-only) --- -TTS_WORKER_URL = os.environ.get("TTS_WORKER_URL", "http://127.0.0.1:8082") +# --- Sprachausgabe (XTTS-v2, multilingual, CPU-only) --- +TTS_WORKER_URL = os.environ.get("TTS_WORKER_URL", "http://127.0.0.1:8085") TTS_TIMEOUT = float(os.environ.get("TTS_TIMEOUT", "300")) # s, pro Synthese TTS_CONNECT_TIMEOUT = float(os.environ.get("TTS_CONNECT_TIMEOUT", "5")) -TTS_MODEL = "kokoro-german" # virtuelles Modell für /v1/audio/speech -TTS_VOICES = ("martin", "victoria") -TTS_DEFAULT_VOICE = "martin" -TTS_FORMATS = ("mp3", "wav", "flac", "pcm") +TTS_MODEL = "xtts-v2" # virtuelles Modell für /v1/audio/speech +TTS_VOICES = ("claribel",) +TTS_DEFAULT_VOICE = "claribel" +TTS_FORMATS = ("mp3", "wav") TTS_DEFAULT_FORMAT = "mp3" # --- Spracherkennung (whisper.cpp, deutsch, CPU-only) --- @@ -1124,7 +1124,7 @@ class Handler(BaseHTTPRequestHandler): self.end_headers() self.wfile.write(data) - # ---------- Sprachausgabe (Kokoro) ---------- + # ---------- Sprachausgabe (XTTS-v2) ---------- def _speech(self) -> None: try: @@ -1183,7 +1183,7 @@ class Handler(BaseHTTPRequestHandler): "invalid_request_error", "invalid_speed") return - # Modell-Name optional; falls angegeben, muss es kokoro-german sein. + # Modell-Name optional; falls angegeben, muss es xtts-v2 sein. model = data.get("model") if model is not None and model != TTS_MODEL: self._send_error(400, f"unbekanntes Modell: {model!r} " @@ -1225,7 +1225,7 @@ class Handler(BaseHTTPRequestHandler): models.append({ "id": TTS_MODEL, "object": "model", - "owned_by": "kokoro", + "owned_by": "coqui-xtts", "type": "speech", }) return {"object": "list", "data": models} diff --git a/router/tts_worker.py b/router/tts_worker.py index 7d1249e..64ad2c8 100644 --- a/router/tts_worker.py +++ b/router/tts_worker.py @@ -51,6 +51,11 @@ VOICES = { "model": "kikiri-german-victoria/kikiri_german_victoria_ep10.pth", "voice": "kikiri-german-victoria/voices/victoria.pt", }, + "eva": { + "config": "eva-k/config.json", + "model": "eva-k/kokoro_german_converted.pth", + "voice": "eva-k/eva_k.pt", + }, } DEFAULT_VOICE = "martin" SAMPLE_RATE = 24000 # Kokoro-nativ diff --git a/router/xtts_worker.py b/router/xtts_worker.py new file mode 100644 index 0000000..f85fb0c --- /dev/null +++ b/router/xtts_worker.py @@ -0,0 +1,283 @@ +#!/usr/bin/env python3 +"""XTTS-v2 TTS-Worker: langlebiger HTTP-Server, hält das Modell im RAM. + +Endpunkte: + GET /status → Health-Check + POST /tts → Synthese (JSON: text, voice, speed, format) + +CPU-only, keine GPU. Logging nach stdout (journald). +""" + +import base64 +import io +import json +import logging +import os +import sys +import time +import wave +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + +# CPU-only erzwingen (BEVOR torch importiert wird) +os.environ["CUDA_VISIBLE_DEVICES"] = "" + +import numpy as np +import torch + +# XTTS-v2 Modell-ID +MODEL_ID = "tts_models/multilingual/multi-dataset/xtts_v2" + +# Verfügbare Stimmen (XTTS Speaker-Namen) +VOICES = { + "claribel": "Claribel Dervla", +} + +# Default-Stimme +DEFAULT_VOICE = "claribel" + +# Port +PORT = int(os.environ.get("XTTS_PORT", "8085")) + +# Logging nach stdout/stderr (für journald) +logging.basicConfig( + level=logging.INFO, + format="%(asctime)s %(levelname)s %(message)s", + stream=sys.stdout, +) +log = logging.getLogger("xtts-worker") + + +class XTTSWorker: + """Hält das XTTS-v2-Modell geladen und synthetisiert Audio.""" + + def __init__(self): + self.tts = None + self.status = "loading" + self.error = None + self.model_name = "xtts-v2" + self.voices = list(VOICES.keys()) + + def load(self): + """Lädt das XTTS-v2-Modell (einmalig).""" + log.info("Lade XTTS-v2-Modell: %s", MODEL_ID) + start = time.time() + try: + from TTS.api import TTS + + # Modell laden (CPU-only) + self.tts = TTS(MODEL_ID) + self.status = "ready" + elapsed = time.time() - start + log.info("Modell geladen in %.1fs", elapsed) + except Exception as e: + self.status = "error" + self.error = str(e) + log.error("Fehler beim Laden: %s", e) + raise + + def synthesize( + self, + text: str, + voice: str = DEFAULT_VOICE, + language: str = "de", + speed: float = 1.0, + output_format: str = "mp3", + ) -> tuple[bytes, str]: + """Synthetisiert Audio und gibt (bytes, content_type) zurück.""" + if self.tts is None: + raise RuntimeError("Modell nicht geladen") + + # Voice-Name auflösen + speaker_name = VOICES.get(voice, voice) + + # WAV synthetisieren + start = time.time() + wav_data = self.tts.tts( + text=text, + speaker=speaker_name, + language=language, + ) + synth_time = time.time() - start + + # tts.tts() gibt eine Liste von Floats zurück (Audio-Samples) + audio_np = np.array(wav_data, dtype=np.float32) + if audio_np.ndim > 1: + audio_np = audio_np.squeeze() + + # Normalisieren und zu int16 konvertieren + audio_np = audio_np / max(1e-8, np.abs(audio_np).max()) + audio_np = (audio_np * 32767).astype(np.int16) + + # WAV schreiben + sample_rate = 24000 # XTTS-v2 Sample Rate + wav_buffer = io.BytesIO() + with wave.open(wav_buffer, "wb") as wav_file: + wav_file.setnchannels(1) + wav_file.setsampwidth(2) # 16-bit + wav_file.setframerate(sample_rate) + wav_file.writeframes(audio_np.tobytes()) + + wav_bytes = wav_buffer.getvalue() + duration = len(audio_np) / sample_rate + + # Speed anwenden (resample) + if speed != 1.0: + try: + import torchaudio + + audio_tensor = torch.from_numpy(audio_np).float().unsqueeze(0) + resampler = torchaudio.transforms.Resample( + orig_freq=sample_rate, + new_freq=int(sample_rate * speed), + ) + audio_tensor = resampler(audio_tensor) + audio_np = audio_tensor.squeeze(0).numpy() + audio_np = (audio_np * 32767).astype(np.int16) + + wav_buffer = io.BytesIO() + with wave.open(wav_buffer, "wb") as wav_file: + wav_file.setnchannels(1) + wav_file.setsampwidth(2) + wav_file.setframerate(int(sample_rate * speed)) + wav_file.writeframes(audio_np.tobytes()) + wav_bytes = wav_buffer.getvalue() + duration = len(audio_np) / (sample_rate * speed) + except Exception as e: + log.warning("Speed-Resampling fehlgeschlagen: %s", e) + + # Format konvertieren + if output_format == "mp3": + try: + import torchaudio + + # WAV zu MP3 – audio_np ist int16, muss zu float32 Tensor + audio_float = torch.from_numpy(audio_np.astype(np.float32) / 32768.0).unsqueeze(0) + mp3_buffer = io.BytesIO() + torchaudio.save( + mp3_buffer, + audio_float, + sample_rate=int(sample_rate * speed) if speed != 1.0 else sample_rate, + format="mp3", + ) + audio_bytes = mp3_buffer.getvalue() + content_type = "audio/mpeg" + except Exception as e: + log.warning("MP3-Konvertierung fehlgeschlagen, liefere WAV: %s", e) + audio_bytes = wav_bytes + content_type = "audio/wav" + else: + audio_bytes = wav_bytes + content_type = "audio/wav" + + total_time = time.time() - start + log.info( + "Synthese: %d Zeichen, %s, %.1fs Audio, %.1fs Gesamt", + len(text), + output_format, + duration, + total_time, + ) + return audio_bytes, content_type + + def health(self) -> dict: + """Liefert Health-Status.""" + return { + "ready": self.status == "ready", + "status": self.status, + "model": self.model_name, + "voices": self.voices, + "error": self.error, + } + + +# Globale Worker-Instanz +worker = XTTSWorker() + + +class Handler(BaseHTTPRequestHandler): + server_version = "XTTSWorker/1.0" + timeout = 60 + + def do_GET(self): + if self.path == "/status": + self._send_json(200, worker.health()) + else: + self._send_json(404, {"error": "not found"}) + + def do_POST(self): + if self.path == "/tts": + self._tts() + else: + self._send_json(404, {"error": "not found"}) + + def _tts(self): + try: + length = int(self.headers.get("Content-Length") or 0) + body = self.rfile.read(length) + data = json.loads(body) + except (ValueError, json.JSONDecodeError) as e: + self._send_json(400, {"error": f"Invalid JSON: {e}"}) + return + + text = data.get("text", "") + if not text: + self._send_json(400, {"error": "No text provided"}) + return + + voice = data.get("voice", DEFAULT_VOICE) + speed = float(data.get("speed", 1.0)) + fmt = data.get("format", "mp3") + + try: + audio_bytes, content_type = worker.synthesize( + text=text, + voice=voice, + speed=speed, + output_format=fmt, + ) + self.send_response(200) + self.send_header("Content-Type", content_type) + self.send_header("Content-Length", str(len(audio_bytes))) + self.send_header("Connection", "close") + self.end_headers() + self.wfile.write(audio_bytes) + except Exception as e: + log.error("Synthese-Fehler: %s", e) + self._send_json(500, {"error": str(e)}) + + def _send_json(self, code: int, payload: dict) -> None: + body = json.dumps(payload).encode() + self.send_response(code) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(body))) + self.send_header("Connection", "close") + self.end_headers() + self.wfile.write(body) + + def log_message(self, format, *args): + # Logging nach stdout (journald) + log.info("%s - %s", self.address_string(), format % args) + + +def main(): + # Modell laden + try: + worker.load() + except Exception as e: + log.error("Konnte Modell nicht laden: %s", e) + sys.exit(1) + + log.info("Worker bereit auf Port %d", PORT) + + server = ThreadingHTTPServer(("0.0.0.0", PORT), Handler) + server.daemon_threads = True + try: + server.serve_forever() + except KeyboardInterrupt: + pass + finally: + server.server_close() + + +if __name__ == "__main__": + main()