router: deutsche Sprachausgabe mit Kokoro-82M (CPU-only)
Fügt einen OpenAI-kompatiblen TTS-Endpunkt POST /v1/audio/speech hinzu. Die Synthese läuft in einem separaten, langlebigen Worker (mike-ai-kokoro.service) mit eigenem Venv (CPU-only torch) und hält die Modelle dauerhaft im RAM (niedrige Warm-Start-Latenz). - Zwei deutsche Stimmen: kikiri-german-martin, kikiri-german-victoria (Apache 2.0, je ~327 MB) unter /opt/mike-ai/models/kokoro/ - Deutsche G2P über espeak-ng (phonemizer), kein spacy/thinc 9.x nötig (kokoro mit --no-deps + misaki ohne [en], Python 3.13-kompatibel) - Formate: mp3 (Default), wav, flac, pcm; speed 0.5-2.0 - /status um tts.*-Felder erweitert (reachable, ready, voices, ...) - TTS ohne GPU-Lock: blockiert weder Qwen/llama.cpp noch FLUX - systemd-Unit mike-ai-kokoro.service (Start beim Boot) - install.sh/deploy.sh um Kokoro-Venv + Modell-Download erweitert - Mock-TTS-Worker + 11 TTS-Tests (insgesamt 43, alle bestanden) - Hörproben (je ~40 s) + Benchmark (RTF ~0.23, ~4.3x Echtzeit) Kein Push – erst nach User-Freigabe.
This commit is contained in:
Mikei386
@@ -3,9 +3,10 @@
|
|||||||
Kleiner OpenAI-kompatibler Proxy (Python, nur Standardbibliothek) vor einem
|
Kleiner OpenAI-kompatibler Proxy (Python, nur Standardbibliothek) vor einem
|
||||||
lokalen llama.cpp-Server. Er leitet normale OpenAI-Requests transparent
|
lokalen llama.cpp-Server. Er leitet normale OpenAI-Requests transparent
|
||||||
weiter (Streaming, Tool Calls, JSON), schaltet zwischen drei festen
|
weiter (Streaming, Tool Calls, JSON), schaltet zwischen drei festen
|
||||||
llama.cpp-Profilen um und orchestriert lokale Bildgenerierung mit
|
llama.cpp-Profilen um, orchestriert lokale Bildgenerierung mit
|
||||||
FLUX.2 [klein] 4B Base (GPU-Hotswap: Qwen stoppen → FLUX laden → Bild
|
FLUX.2 [klein] 4B Base (GPU-Hotswap: Qwen stoppen → FLUX laden → Bild
|
||||||
→ FLUX entladen → Qwen wiederherstellen).
|
→ FLUX entladen → Qwen wiederherstellen) und stellt lokale deutsche
|
||||||
|
Sprachausgabe bereit (Kokoro-82M, CPU-only, OpenAI-kompatibel).
|
||||||
|
|
||||||
## Zielsystem
|
## Zielsystem
|
||||||
|
|
||||||
@@ -17,6 +18,7 @@ FLUX.2 [klein] 4B Base (GPU-Hotswap: Qwen stoppen → FLUX laden → Bild
|
|||||||
| Profil-Skript | `/usr/local/bin/llama-profile {fast\|medium\|long}` |
|
| Profil-Skript | `/usr/local/bin/llama-profile {fast\|medium\|long}` |
|
||||||
| Router-Port | **8081** |
|
| Router-Port | **8081** |
|
||||||
| Router-Service | `mike-ai-profile-router.service` |
|
| Router-Service | `mike-ai-profile-router.service` |
|
||||||
|
| TTS-Worker | `http://127.0.0.1:8082` (Service `mike-ai-kokoro.service`) |
|
||||||
|
|
||||||
## Profile / virtuelle Modelle
|
## Profile / virtuelle Modelle
|
||||||
|
|
||||||
@@ -37,6 +39,7 @@ FLUX.2 [klein] 4B Base (GPU-Hotswap: Qwen stoppen → FLUX laden → Bild
|
|||||||
| `POST /v1/images/generations` | Bildgenerierung (FLUX.2 [klein] 4B Base, OpenAI-kompatibel) |
|
| `POST /v1/images/generations` | Bildgenerierung (FLUX.2 [klein] 4B Base, OpenAI-kompatibel) |
|
||||||
| `GET /images` | Liste der gespeicherten Bilder (max. 200) |
|
| `GET /images` | Liste der gespeicherten Bilder (max. 200) |
|
||||||
| `GET /images/<datei>` | PNG-Download (nur `images/`-Verzeichnis, validiert) |
|
| `GET /images/<datei>` | PNG-Download (nur `images/`-Verzeichnis, validiert) |
|
||||||
|
| `POST /v1/audio/speech` | Deutsche Sprachausgabe (Kokoro-82M, OpenAI-kompatibel) |
|
||||||
| alles andere | Transparente Weiterleitung an llama.cpp |
|
| alles andere | Transparente Weiterleitung an llama.cpp |
|
||||||
|
|
||||||
### Verhalten
|
### Verhalten
|
||||||
@@ -167,12 +170,117 @@ VRAM-Check).
|
|||||||
Das Venv liegt unter `/opt/mike-ai/ai-profile-router/venv/` und wird von
|
Das Venv liegt unter `/opt/mike-ai/ai-profile-router/venv/` und wird von
|
||||||
`install.sh` automatisch angelegt/aktualisiert.
|
`install.sh` automatisch angelegt/aktualisiert.
|
||||||
|
|
||||||
|
## Sprachausgabe (Kokoro-82M, deutsch, CPU-only)
|
||||||
|
|
||||||
|
Der Router stellt lokale deutsche Sprachausgabe bereit. Die Synthese läuft
|
||||||
|
in einem **separaten, langlebigen Worker** (`mike-ai-kokoro.service`), der
|
||||||
|
die Kokoro-Modelle einmalig beim Start lädt und dauerhaft im RAM hält
|
||||||
|
(niedrige Warm-Start-Latenz). Der Worker ist CPU-only und blockiert weder
|
||||||
|
Qwen/llama.cpp noch FLUX/GPU – er teilt sich nur den Prozessor.
|
||||||
|
|
||||||
|
- **Modell:** Kokoro-82M (hexgrad) mit zwei deutschen Feintunings
|
||||||
|
(`kikiri-tts/kikiri-german-martin`, `kikiri-tts/kikiri-german-victoria`,
|
||||||
|
beide Apache 2.0, je ~327 MB).
|
||||||
|
- **G2P:** deutsche Phonemisierung über `espeak-ng` (phonemizer), kein spacy
|
||||||
|
nötig. Deutsche Sprachunterstützung per Patch (kokoro PR #340).
|
||||||
|
- **Venv:** eigenes Venv unter `/opt/mike-ai/kokoro/venv/` mit CPU-only
|
||||||
|
`torch` (keine CUDA-Abhängigkeit, kein Konflikt mit dem Bild-Venv).
|
||||||
|
- **Modelle:** `/opt/mike-ai/models/kokoro/` (persistent).
|
||||||
|
|
||||||
|
### Endpunkt `POST /v1/audio/speech`
|
||||||
|
|
||||||
|
OpenAI-kompatibel. Unterstützt `input` (oder `text`), `voice`, `speed`,
|
||||||
|
`response_format`, `model`.
|
||||||
|
|
||||||
|
| Parameter | Werte | Default |
|
||||||
|
|---|---|---|
|
||||||
|
| `input` | Text (Pflicht, max. 8000 Zeichen) | – |
|
||||||
|
| `voice` | `martin`, `victoria` | `martin` |
|
||||||
|
| `speed` | 0.5–2.0 | 1.0 |
|
||||||
|
| `response_format` | `mp3` (Default), `wav`, `flac`, `pcm` | `mp3` |
|
||||||
|
| `model` | `kokoro-german` (optional) | – |
|
||||||
|
|
||||||
|
Die Antwort ist **binäres Audio** (nicht JSON) mit passendem
|
||||||
|
`Content-Type` (`audio/mpeg`, `audio/wav`, `audio/flac`,
|
||||||
|
`application/octet-stream`).
|
||||||
|
|
||||||
|
Beispiele:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# MP3 (Default), Stimme martin
|
||||||
|
curl -s http://192.168.1.196:8081/v1/audio/speech \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"input":"Hallo, dies ist ein Test.","voice":"martin"}' -o out.mp3
|
||||||
|
|
||||||
|
# WAV, Stimme victoria, 1.5x Tempo
|
||||||
|
curl -s http://192.168.1.196:8081/v1/audio/speech \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"input":"Guten Tag.","voice":"victoria","speed":1.5,"response_format":"wav"}' -o out.wav
|
||||||
|
```
|
||||||
|
|
||||||
|
**Lange Texte:** Das Modell verarbeitet pro Segment max. 510 Phoneme
|
||||||
|
(~25–30 s). Längere Texte werden mit Zeilenumbrüchen (`\n`) in Segmente
|
||||||
|
teilt; jedes Segment wird einzeln synthetisiert und die Audios
|
||||||
|
zusammengeführt.
|
||||||
|
|
||||||
|
### Verhalten
|
||||||
|
|
||||||
|
- **Kein GPU-Lock:** TTS läuft CPU-only und greift nicht in den
|
||||||
|
GPU-Hotswap (Bild) oder Profilwechsel (Qwen) ein. TTS-Requests können
|
||||||
|
parallel zu Chats laufen.
|
||||||
|
- **Serialisierte Synthese:** Der Worker synthetisiert nacheinander
|
||||||
|
(CPU-bound), parallele Requests werden intern gewartet.
|
||||||
|
- **`/status`** zeigt `tts.reachable`, `tts.ready`, `tts.voices`,
|
||||||
|
`tts.load_errors`, `tts.last_seconds`, `tts.last_voice`,
|
||||||
|
`tts.last_error`.
|
||||||
|
- **Fehler:** Worker down → `503` (`tts_failed`); ungültige Parameter →
|
||||||
|
`400`. OpenAI-kompatibles Fehlerformat.
|
||||||
|
|
||||||
|
### Hörproben
|
||||||
|
|
||||||
|
Zwei deutsche Hörproben (je ~40 s) liegen unter
|
||||||
|
`/opt/mike-ai/ai-profile-router/samples/`:
|
||||||
|
|
||||||
|
- `martin_lang.wav` (Stimme martin)
|
||||||
|
- `victoria_lang.wav` (Stimme victoria)
|
||||||
|
|
||||||
|
### Benchmark (CPU-only, gemessen)
|
||||||
|
|
||||||
|
| Textlänge | Audio | Synthese | RTF |
|
||||||
|
|---|---|---|---|
|
||||||
|
| ~5 s | 3.4 s | 0.6 s | 0.19× |
|
||||||
|
| ~15 s | 14.1 s | 3.4 s | 0.24× |
|
||||||
|
| ~40 s | 33.5 s | 7.5 s | 0.23× |
|
||||||
|
|
||||||
|
RTF ~0.23 bedeutet: Synthese ist ~4.3× schneller als Echtzeit.
|
||||||
|
RAM-Belegung des Workers: ~3.5 GB (inkl. torch, beide Modelle).
|
||||||
|
|
||||||
|
### Erforderliche Python-Pakete (im Kokoro-Venv)
|
||||||
|
|
||||||
|
- `torch` (CPU-only, `--index-url https://download.pytorch.org/whl/cpu`)
|
||||||
|
- `kokoro` (0.7.16, mit `--no-deps` installiert)
|
||||||
|
- `misaki` (G2P, ohne `[en]`-Extra → kein thinc 9.x / spacy)
|
||||||
|
- `phonemizer` + System-Paket `espeak-ng` (deutsche G2P)
|
||||||
|
- `soundfile`, `lameenc` (Audio-Formate wav/mp3/flac/pcm)
|
||||||
|
- `huggingface-hub`, `loguru`, `scipy`, `transformers`, `regex`, `num2words`
|
||||||
|
|
||||||
|
Das Venv liegt unter `/opt/mike-ai/kokoro/venv/` und wird von `install.sh`
|
||||||
|
automatisch angelegt (nur wenn noch nicht vorhanden).
|
||||||
|
|
||||||
|
**Hinweis (Python 3.13):** `kokoro` verlangt offiziell `misaki[en]`, das
|
||||||
|
`spacy-curated-transformers` → `thinc 9.x` zieht – für Python 3.13 gibt es
|
||||||
|
keine thinc-9-Wheels. Deshalb wird `kokoro` mit `--no-deps` und `misaki`
|
||||||
|
ohne `[en]` installiert; die deutsche G2P läuft über `espeak-ng` und braucht
|
||||||
|
spacy nicht.
|
||||||
|
|
||||||
## Repository-Struktur
|
## Repository-Struktur
|
||||||
|
|
||||||
```
|
```
|
||||||
router/ai_profile_router.py # der Router (einzige Laufzeit-Datei)
|
router/ai_profile_router.py # der Router (einzige Laufzeit-Datei)
|
||||||
router/image_worker.py # FLUX-Worker (eigener Prozess, JSON-Protokoll)
|
router/image_worker.py # FLUX-Worker (eigener Prozess, JSON-Protokoll)
|
||||||
deploy/mike-ai-profile-router.service # systemd-Unit
|
router/tts_worker.py # Kokoro-TTS-Worker (eigener Prozess, HTTP-API)
|
||||||
|
deploy/mike-ai-profile-router.service # systemd-Unit (Router)
|
||||||
|
deploy/mike-ai-kokoro.service # systemd-Unit (TTS-Worker)
|
||||||
deploy/install.sh # läuft auf dem Zielsystem (per SSH)
|
deploy/install.sh # läuft auf dem Zielsystem (per SSH)
|
||||||
deploy/deploy.sh # läuft lokal: SCP + SSH
|
deploy/deploy.sh # läuft lokal: SCP + SSH
|
||||||
dev/ # lokale Tests (Mock-llama.cpp, Mock-Worker, Benchmarks)
|
dev/ # lokale Tests (Mock-llama.cpp, Mock-Worker, Benchmarks)
|
||||||
@@ -191,8 +299,9 @@ Voraussetzung: SSH-Key `~/.ssh/lmstudio_unraid` (bereits vorhanden).
|
|||||||
|
|
||||||
Das Skript:
|
Das Skript:
|
||||||
|
|
||||||
1. Überträgt `ai_profile_router.py`, `image_worker.py`, `install.sh` und die
|
1. Überträgt `ai_profile_router.py`, `image_worker.py`, `tts_worker.py`,
|
||||||
systemd-Unit per SCP nach `/tmp/ai-profile-router/` auf dem Zielsystem.
|
`install.sh` und beide systemd-Units per SCP nach
|
||||||
|
`/tmp/ai-profile-router/` auf dem Zielsystem.
|
||||||
2. Führt `install.sh` per SSH aus, das:
|
2. Führt `install.sh` per SSH aus, das:
|
||||||
- den alten Router (`mike-ai-local-llm-router.service` +
|
- den alten Router (`mike-ai-local-llm-router.service` +
|
||||||
`/opt/mike-ai/local-llm-router`) **mit Backup** entfernt,
|
`/opt/mike-ai/local-llm-router`) **mit Backup** entfernt,
|
||||||
@@ -202,7 +311,14 @@ Das Skript:
|
|||||||
anlegt (nur wenn noch nicht vorhanden),
|
anlegt (nur wenn noch nicht vorhanden),
|
||||||
- das FLUX-Modell nach `/opt/mike-ai/models/FLUX.2-klein-base-4B` lädt
|
- das FLUX-Modell nach `/opt/mike-ai/models/FLUX.2-klein-base-4B` lädt
|
||||||
(nur wenn noch nicht vorhanden, ~15 GB),
|
(nur wenn noch nicht vorhanden, ~15 GB),
|
||||||
- `mike-ai-profile-router.service` aktiviert (Start beim Boot) und startet,
|
- `espeak-ng` installiert (nur wenn noch nicht vorhanden),
|
||||||
|
- das Kokoro-Venv unter `/opt/mike-ai/kokoro/venv/` anlegt (nur wenn
|
||||||
|
noch nicht vorhanden, CPU-only torch + kokoro + misaki + phonemizer +
|
||||||
|
soundfile + lameenc),
|
||||||
|
- die Kokoro-Modelle nach `/opt/mike-ai/models/kokoro/` lädt (nur wenn
|
||||||
|
noch nicht vorhanden, ~660 MB),
|
||||||
|
- `mike-ai-profile-router.service` und `mike-ai-kokoro.service`
|
||||||
|
aktivieren (Start beim Boot) und starten,
|
||||||
- `GET /status` verifiziert.
|
- `GET /status` verifiziert.
|
||||||
|
|
||||||
Die Installation ist idempotent (Update = erneut ausführen).
|
Die Installation ist idempotent (Update = erneut ausführen).
|
||||||
@@ -229,6 +345,18 @@ Die Installation ist idempotent (Update = erneut ausführen).
|
|||||||
| `IMAGE_GEN_TIMEOUT` | `600` | Timeout pro Bild (s) |
|
| `IMAGE_GEN_TIMEOUT` | `600` | Timeout pro Bild (s) |
|
||||||
| `IMAGE_VRAM_FREE_TIMEOUT` | `120` | Warten auf VRAM-Freiheit (s) |
|
| `IMAGE_VRAM_FREE_TIMEOUT` | `120` | Warten auf VRAM-Freiheit (s) |
|
||||||
| `CHAT_WAIT_TIMEOUT` | `300` | Chat wartet auf Qwen (s) |
|
| `CHAT_WAIT_TIMEOUT` | `300` | Chat wartet auf Qwen (s) |
|
||||||
|
| `TTS_WORKER_URL` | `http://127.0.0.1:8082` | TTS-Worker (Router-Seite) |
|
||||||
|
| `TTS_TIMEOUT` | `300` | Timeout pro Synthese (s) |
|
||||||
|
| `TTS_CONNECT_TIMEOUT` | `5` | Connect-Timeout TTS-Worker (s) |
|
||||||
|
|
||||||
|
TTS-Worker (`mike-ai-kokoro.service`):
|
||||||
|
|
||||||
|
| Variable | Default | Bedeutung |
|
||||||
|
|---|---|---|
|
||||||
|
| `KOKORO_HOST` | `127.0.0.1` | Bind-Adresse (nur lokal, Router proxyt) |
|
||||||
|
| `KOKORO_PORT` | `8082` | Port |
|
||||||
|
| `KOKORO_MODEL_DIR` | `/opt/mike-ai/models/kokoro` | Modell-Verzeichnis |
|
||||||
|
| `LOG_LEVEL` | `INFO` | Logging-Level |
|
||||||
|
|
||||||
## Lokale Tests
|
## Lokale Tests
|
||||||
|
|
||||||
@@ -236,21 +364,32 @@ Die Installation ist idempotent (Update = erneut ausführen).
|
|||||||
./dev/test_local.sh
|
./dev/test_local.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
Startet einen Mock-llama.cpp, einen Mock-Bild-Worker und den Router mit einem
|
Startet einen Mock-llama.cpp, einen Mock-Bild-Worker, einen Mock-TTS-Worker
|
||||||
Fake-Profil-Skript und prüft: `/v1/models`, `/status`, Forwarding, Streaming,
|
und den Router mit einem Fake-Profil-Skript und prüft: `/v1/models`,
|
||||||
Tool Calls, Profilwechsel (fast→medium→fast), virtuelles Modell triggert
|
`/status`, Forwarding, Streaming, Tool Calls, Profilwechsel
|
||||||
Wechsel, ungültige Profile, Upstream down → 502, Recovery, **Bildgenerierung**
|
(fast→medium→fast), virtuelles Modell triggert Wechsel, ungültige Profile,
|
||||||
(`standard`→30 Steps, `high`→50 Steps, Validierung, Image-Fehler→Qwen
|
Upstream down → 502, Recovery, **Bildgenerierung** (`standard`→30 Steps,
|
||||||
wiederhergestellt, Fast/Medium/Long→Image→gleiches Profil, `/status` während
|
`high`→50 Steps, Validierung, Image-Fehler→Qwen wiederhergestellt,
|
||||||
Bild-Job, paralleler Chat während Bild-Job wartet statt 502).
|
Fast/Medium/Long→Image→gleiches Profil, `/status` während Bild-Job,
|
||||||
|
paralleler Chat während Bild-Job wartet statt 502) und **Sprachausgabe**
|
||||||
|
(`/status` mit tts-Section, `POST /v1/audio/speech` wav/mp3, Validierung,
|
||||||
|
Worker-Fehler→503, Worker down→503, Worker-Neustart→Recovery).
|
||||||
|
|
||||||
|
Aktuell: **43 Tests** (32 bestehende + 11 TTS-Assertions).
|
||||||
|
|
||||||
## Betrieb
|
## Betrieb
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
systemctl status mike-ai-profile-router
|
systemctl status mike-ai-profile-router
|
||||||
|
systemctl status mike-ai-kokoro
|
||||||
journalctl -u mike-ai-profile-router -f
|
journalctl -u mike-ai-profile-router -f
|
||||||
|
journalctl -u mike-ai-kokoro -f
|
||||||
curl -s http://192.168.1.196:8081/status | python3 -m json.tool
|
curl -s http://192.168.1.196:8081/status | python3 -m json.tool
|
||||||
curl -s -X POST http://192.168.1.196:8081/medium
|
curl -s -X POST http://192.168.1.196:8081/medium
|
||||||
|
# TTS-Test
|
||||||
|
curl -s http://192.168.1.196:8081/v1/audio/speech \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{"input":"Hallo","voice":"martin"}' -o test.mp3
|
||||||
```
|
```
|
||||||
|
|
||||||
## Sicherheit
|
## Sicherheit
|
||||||
|
|||||||
+3
-1
@@ -9,7 +9,9 @@ SSH_KEY="${SSH_KEY:-$HOME/.ssh/lmstudio_unraid}"
|
|||||||
STAGE="/tmp/ai-profile-router-$$"
|
STAGE="/tmp/ai-profile-router-$$"
|
||||||
|
|
||||||
mkdir -p "$STAGE"
|
mkdir -p "$STAGE"
|
||||||
cp router/ai_profile_router.py router/image_worker.py deploy/install.sh deploy/mike-ai-profile-router.service "$STAGE/"
|
cp router/ai_profile_router.py router/image_worker.py router/tts_worker.py \
|
||||||
|
deploy/install.sh deploy/mike-ai-profile-router.service \
|
||||||
|
deploy/mike-ai-kokoro.service "$STAGE/"
|
||||||
|
|
||||||
echo "== Übertrage Dateien nach ${TARGET}:/tmp/ai-profile-router/"
|
echo "== Übertrage Dateien nach ${TARGET}:/tmp/ai-profile-router/"
|
||||||
ssh -i "$SSH_KEY" "$TARGET" 'mkdir -p /tmp/ai-profile-router'
|
ssh -i "$SSH_KEY" "$TARGET" 'mkdir -p /tmp/ai-profile-router'
|
||||||
|
|||||||
+74
-7
@@ -1,19 +1,23 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# AI Profile Router – Installation/Update auf dem Zielsystem.
|
# AI Profile Router – Installation/Update auf dem Zielsystem.
|
||||||
# Wird als root auf dem Zielsystem ausgeführt (per SSH, vgl. deploy.sh).
|
# Wird als root auf dem Zielsystem ausgeführt (per SSH, vgl. deploy.sh).
|
||||||
# Erwartet ai_profile_router.py, image_worker.py und mike-ai-profile-router.service
|
# Erwartet ai_profile_router.py, image_worker.py, tts_worker.py,
|
||||||
# im selben Verzeichnis wie dieses Skript.
|
# mike-ai-profile-router.service und mike-ai-kokoro.service im selben
|
||||||
|
# Verzeichnis wie dieses Skript.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
DIR="$(cd "$(dirname "$0")" && pwd)"
|
DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||||
INSTALL_DIR=/opt/mike-ai/ai-profile-router
|
INSTALL_DIR=/opt/mike-ai/ai-profile-router
|
||||||
SERVICE=mike-ai-profile-router.service
|
SERVICE=mike-ai-profile-router.service
|
||||||
|
KOKORO_SERVICE=mike-ai-kokoro.service
|
||||||
OLD_SERVICE=mike-ai-local-llm-router.service
|
OLD_SERVICE=mike-ai-local-llm-router.service
|
||||||
OLD_DIR=/opt/mike-ai/local-llm-router
|
OLD_DIR=/opt/mike-ai/local-llm-router
|
||||||
BACKUP_DIR=/opt/mike-ai/.backup-ai-profile-router-$(date +%Y%m%d-%H%M%S)
|
BACKUP_DIR=/opt/mike-ai/.backup-ai-profile-router-$(date +%Y%m%d-%H%M%S)
|
||||||
VENV="$INSTALL_DIR/venv"
|
VENV="$INSTALL_DIR/venv"
|
||||||
MODEL_DIR=/opt/mike-ai/models/FLUX.2-klein-base-4B
|
MODEL_DIR=/opt/mike-ai/models/FLUX.2-klein-base-4B
|
||||||
IMAGE_DIR="$INSTALL_DIR/images"
|
IMAGE_DIR="$INSTALL_DIR/images"
|
||||||
|
KOKORO_VENV=/opt/mike-ai/kokoro/venv
|
||||||
|
KOKORO_MODEL_DIR=/opt/mike-ai/models/kokoro
|
||||||
|
|
||||||
echo "== AI Profile Router: Installation/Update =="
|
echo "== AI Profile Router: Installation/Update =="
|
||||||
|
|
||||||
@@ -35,7 +39,9 @@ fi
|
|||||||
mkdir -p "$INSTALL_DIR" "$IMAGE_DIR"
|
mkdir -p "$INSTALL_DIR" "$IMAGE_DIR"
|
||||||
install -m 0755 "$DIR/ai_profile_router.py" "$INSTALL_DIR/ai_profile_router.py"
|
install -m 0755 "$DIR/ai_profile_router.py" "$INSTALL_DIR/ai_profile_router.py"
|
||||||
install -m 0755 "$DIR/image_worker.py" "$INSTALL_DIR/image_worker.py"
|
install -m 0755 "$DIR/image_worker.py" "$INSTALL_DIR/image_worker.py"
|
||||||
|
install -m 0755 "$DIR/tts_worker.py" "$INSTALL_DIR/tts_worker.py"
|
||||||
install -m 0644 "$DIR/${SERVICE}" "/etc/systemd/system/${SERVICE}"
|
install -m 0644 "$DIR/${SERVICE}" "/etc/systemd/system/${SERVICE}"
|
||||||
|
install -m 0644 "$DIR/${KOKORO_SERVICE}" "/etc/systemd/system/${KOKORO_SERVICE}"
|
||||||
|
|
||||||
# --- 3. Python-Venv mit Bild-Abhängigkeiten ---------------------------------
|
# --- 3. Python-Venv mit Bild-Abhängigkeiten ---------------------------------
|
||||||
if [ ! -x "$VENV/bin/python" ]; then
|
if [ ! -x "$VENV/bin/python" ]; then
|
||||||
@@ -67,18 +73,79 @@ print("Modell-Download abgeschlossen")
|
|||||||
PY
|
PY
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# --- 5. Service aktivieren und starten ---------------------------------------
|
# --- 5. TTS (Kokoro) ----------------------------------------------------------
|
||||||
|
# espeak-ng wird für die deutsche G2P (phonemizer) benötigt.
|
||||||
|
if ! command -v espeak-ng >/dev/null 2>&1; then
|
||||||
|
echo "-- Installiere espeak-ng"
|
||||||
|
apt-get install -y espeak-ng
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Kokoro-Venv (CPU-only torch + kokoro + misaki + phonemizer + soundfile + lameenc)
|
||||||
|
if [ ! -x "$KOKORO_VENV/bin/python" ]; then
|
||||||
|
echo "-- Erstelle Kokoro-Venv in $KOKORO_VENV"
|
||||||
|
mkdir -p "$KOKORO_VENV"
|
||||||
|
python3 -m venv "$KOKORO_VENV"
|
||||||
|
"$KOKORO_VENV/bin/pip" install --quiet --upgrade pip
|
||||||
|
echo "-- Installiere CPU-only torch"
|
||||||
|
"$KOKORO_VENV/bin/pip" install --quiet torch \
|
||||||
|
--index-url https://download.pytorch.org/whl/cpu
|
||||||
|
echo "-- Installiere Kokoro-Abhängigkeiten"
|
||||||
|
"$KOKORO_VENV/bin/pip" install --quiet \
|
||||||
|
huggingface-hub loguru scipy transformers regex num2words \
|
||||||
|
phonemizer soundfile lameenc
|
||||||
|
# kokoro ohne [en]-Extra (vermeidet thinc 9.x, das kein Py3.13-Wheel hat);
|
||||||
|
# deutsche G2P läuft über espeak-ng (phonemizer), nicht über spacy.
|
||||||
|
"$KOKORO_VENV/bin/pip" install --quiet kokoro --no-deps
|
||||||
|
"$KOKORO_VENV/bin/pip" install --quiet misaki
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Kokoro-Modelle (kikiri-german-martin + kikiri-german-victoria, Apache 2.0)
|
||||||
|
if [ ! -f "$KOKORO_MODEL_DIR/kikiri-german-martin/kikiri_german_martin_ep10.pth" ] \
|
||||||
|
|| [ ! -f "$KOKORO_MODEL_DIR/kikiri-german-victoria/kikiri_german_victoria_ep10.pth" ]; then
|
||||||
|
echo "-- Lade Kokoro-Modelle nach $KOKORO_MODEL_DIR (kann dauern)"
|
||||||
|
"$KOKORO_VENV/bin/python" - <<'PY'
|
||||||
|
import os
|
||||||
|
from huggingface_hub import hf_hub_download
|
||||||
|
|
||||||
|
base = "/opt/mike-ai/models/kokoro"
|
||||||
|
files = [
|
||||||
|
("kikiri-tts/kikiri-german-martin", "kikiri_german_martin_ep10.pth", "kikiri-german-martin"),
|
||||||
|
("kikiri-tts/kikiri-german-martin", "voices/martin.pt", "kikiri-german-martin"),
|
||||||
|
("kikiri-tts/kikiri-german-victoria", "kikiri_german_victoria_ep10.pth", "kikiri-german-victoria"),
|
||||||
|
("kikiri-tts/kikiri-german-victoria", "voices/victoria.pt", "kikiri-german-victoria"),
|
||||||
|
("hexgrad/Kokoro-82M", "config.json", "kikiri-german-martin"),
|
||||||
|
("hexgrad/Kokoro-82M", "config.json", "kikiri-german-victoria"),
|
||||||
|
]
|
||||||
|
for repo, fname, subdir in files:
|
||||||
|
dest = os.path.join(base, subdir, fname)
|
||||||
|
if os.path.exists(dest) and os.path.getsize(dest) > 1000:
|
||||||
|
print(f"vorhanden: {subdir}/{fname}")
|
||||||
|
continue
|
||||||
|
os.makedirs(os.path.dirname(dest), exist_ok=True)
|
||||||
|
hf_hub_download(repo_id=repo, filename=fname, local_dir=os.path.join(base, subdir))
|
||||||
|
print(f"geladen: {subdir}/{fname}")
|
||||||
|
PY
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- 6. Services aktivieren und starten ---------------------------------------
|
||||||
systemctl daemon-reload
|
systemctl daemon-reload
|
||||||
systemctl enable "$SERVICE"
|
systemctl enable "$SERVICE" "$KOKORO_SERVICE"
|
||||||
|
systemctl restart "$KOKORO_SERVICE"
|
||||||
systemctl restart "$SERVICE"
|
systemctl restart "$SERVICE"
|
||||||
|
|
||||||
# --- 6. Verifikation ----------------------------------------------------------
|
# --- 7. Verifikation ----------------------------------------------------------
|
||||||
sleep 1
|
sleep 1
|
||||||
if ! systemctl is-active --quiet "$SERVICE"; then
|
if ! systemctl is-active --quiet "$SERVICE"; then
|
||||||
echo "-- FEHLER: Service läuft nicht" >&2
|
echo "-- FEHLER: Router-Service läuft nicht" >&2
|
||||||
journalctl -u "$SERVICE" -n 20 --no-pager >&2
|
journalctl -u "$SERVICE" -n 20 --no-pager >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
echo "-- Service läuft"
|
echo "-- Router-Service läuft"
|
||||||
|
if ! systemctl is-active --quiet "$KOKORO_SERVICE"; then
|
||||||
|
echo "-- FEHLER: Kokoro-Service läuft nicht" >&2
|
||||||
|
journalctl -u "$KOKORO_SERVICE" -n 20 --no-pager >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "-- Kokoro-Service läuft"
|
||||||
curl -sf "http://127.0.0.1:8081/status" | python3 -m json.tool
|
curl -sf "http://127.0.0.1:8081/status" | python3 -m json.tool
|
||||||
echo "== Fertig =="
|
echo "== Fertig =="
|
||||||
|
|||||||
@@ -0,0 +1,21 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Kokoro TTS Worker (deutsche Sprachausgabe, CPU-only)
|
||||||
|
After=network.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=root
|
||||||
|
WorkingDirectory=/opt/mike-ai/ai-profile-router
|
||||||
|
Environment=KOKORO_HOST=127.0.0.1
|
||||||
|
Environment=KOKORO_PORT=8082
|
||||||
|
Environment=KOKORO_MODEL_DIR=/opt/mike-ai/models/kokoro
|
||||||
|
Environment=LOG_LEVEL=INFO
|
||||||
|
ExecStart=/opt/mike-ai/kokoro/venv/bin/python /opt/mike-ai/ai-profile-router/tts_worker.py
|
||||||
|
Restart=always
|
||||||
|
RestartSec=5
|
||||||
|
# CPU-only: keine GPU-Bindung, keine VRAM-Belegung
|
||||||
|
StandardOutput=journal
|
||||||
|
StandardError=journal
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
@@ -0,0 +1,149 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Mock-TTS-Worker für lokale Tests (gleiche HTTP-API wie tts_worker.py).
|
||||||
|
|
||||||
|
Erzeugt ein kurzes, leises WAV statt echten Audios.
|
||||||
|
|
||||||
|
API:
|
||||||
|
GET /status -> {"status":"ok","ready":true,"voices":[...],...}
|
||||||
|
POST /tts -> {"text":"...","voice":"...","speed":1.0,"format":"wav"}
|
||||||
|
-> binäres Audio (WAV/MP3/FLAC/PCM)
|
||||||
|
|
||||||
|
Optionen (Umgebungsvariablen):
|
||||||
|
MOCK_TTS_PORT Port (Default 8082)
|
||||||
|
MOCK_TTS_DELAY Sekunden pro Synthese (Default 0.2)
|
||||||
|
MOCK_TTS_LOG Datei, in die die Requests geloggt werden (JSON-Zeilen)
|
||||||
|
|
||||||
|
Sonder-Texte:
|
||||||
|
"FAIL" -> Worker antwortet mit 500 (simulierter Fehler)
|
||||||
|
"SLOW" -> Worker schläft 5 s
|
||||||
|
"""
|
||||||
|
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import struct
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
|
||||||
|
PORT = int(os.environ.get("MOCK_TTS_PORT", "8082"))
|
||||||
|
DELAY = float(os.environ.get("MOCK_TTS_DELAY", "0.2"))
|
||||||
|
LOG_FILE = os.environ.get("MOCK_TTS_LOG", "")
|
||||||
|
SAMPLE_RATE = 24000
|
||||||
|
|
||||||
|
|
||||||
|
def _make_wav(seconds: float = 0.1) -> bytes:
|
||||||
|
"""Erzeugt ein kurzes WAV (16-bit, mono, 24 kHz) mit Sinus-Ton."""
|
||||||
|
import math
|
||||||
|
n = int(SAMPLE_RATE * seconds)
|
||||||
|
samples = b"".join(
|
||||||
|
struct.pack("<h", int(8000 * math.sin(2 * math.pi * 440 * i / SAMPLE_RATE)))
|
||||||
|
for i in range(n)
|
||||||
|
)
|
||||||
|
header = (
|
||||||
|
b"RIFF" + struct.pack("<I", 36 + len(samples)) + b"WAVE"
|
||||||
|
+ b"fmt " + struct.pack("<IHHIIHH", 16, 1, 1, SAMPLE_RATE,
|
||||||
|
SAMPLE_RATE * 2, 2, 16)
|
||||||
|
+ b"data" + struct.pack("<I", len(samples))
|
||||||
|
)
|
||||||
|
return header + samples
|
||||||
|
|
||||||
|
|
||||||
|
def _log_request(req: dict) -> None:
|
||||||
|
if not LOG_FILE:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
with open(LOG_FILE, "a", encoding="utf-8") as f:
|
||||||
|
f.write(json.dumps(req) + "\n")
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class Handler(BaseHTTPRequestHandler):
|
||||||
|
server_version = "MockKokoroTTS/1.0"
|
||||||
|
timeout = 60
|
||||||
|
|
||||||
|
def log_message(self, fmt, *args): # noqa: N802
|
||||||
|
pass # leise
|
||||||
|
|
||||||
|
def do_GET(self): # noqa: N802
|
||||||
|
if self.path.split("?", 1)[0] == "/status":
|
||||||
|
self._send_json(200, {
|
||||||
|
"status": "ok",
|
||||||
|
"ready": True,
|
||||||
|
"voices": ["martin", "victoria"],
|
||||||
|
"default_voice": "martin",
|
||||||
|
"load_errors": [],
|
||||||
|
"sample_rate": SAMPLE_RATE,
|
||||||
|
"uptime_seconds": 1.0,
|
||||||
|
"total_requests": 0,
|
||||||
|
"last_seconds": None,
|
||||||
|
"last_voice": None,
|
||||||
|
"last_error": None,
|
||||||
|
})
|
||||||
|
else:
|
||||||
|
self._send_json(404, {"error": "not found"})
|
||||||
|
|
||||||
|
def do_POST(self): # noqa: N802
|
||||||
|
if self.path.split("?", 1)[0] != "/tts":
|
||||||
|
self._send_json(404, {"error": "not found"})
|
||||||
|
return
|
||||||
|
length = int(self.headers.get("Content-Length") or 0)
|
||||||
|
try:
|
||||||
|
req = json.loads(self.rfile.read(length))
|
||||||
|
except ValueError:
|
||||||
|
self._send_json(400, {"error": "ungültiges JSON"})
|
||||||
|
return
|
||||||
|
_log_request(req)
|
||||||
|
text = req.get("text", "")
|
||||||
|
if text == "SLOW":
|
||||||
|
time.sleep(5.0)
|
||||||
|
else:
|
||||||
|
time.sleep(DELAY)
|
||||||
|
if text == "FAIL":
|
||||||
|
self._send_json(500, {"error": "simulierter TTS-Fehler"})
|
||||||
|
return
|
||||||
|
fmt = req.get("format", "wav")
|
||||||
|
if fmt == "wav":
|
||||||
|
data, ctype = _make_wav(), "audio/wav"
|
||||||
|
elif fmt == "mp3":
|
||||||
|
# Minimales MP3-Frame (silence) – reicht für Format-Tests.
|
||||||
|
data = b"\xff\xfb\x90\x00" + b"\x00" * 417
|
||||||
|
ctype = "audio/mpeg"
|
||||||
|
elif fmt == "flac":
|
||||||
|
data = b"fLaC" + b"\x00" * 100
|
||||||
|
ctype = "audio/flac"
|
||||||
|
else: # pcm
|
||||||
|
data = b"\x00" * 4800
|
||||||
|
ctype = "application/octet-stream"
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Type", ctype)
|
||||||
|
self.send_header("Content-Length", str(len(data)))
|
||||||
|
self.send_header("Connection", "close")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(data)
|
||||||
|
|
||||||
|
def _send_json(self, code: int, payload: dict) -> None:
|
||||||
|
body = json.dumps(payload).encode()
|
||||||
|
self.send_response(code)
|
||||||
|
self.send_header("Content-Type", "application/json")
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.send_header("Connection", "close")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
server = ThreadingHTTPServer(("127.0.0.1", PORT), Handler)
|
||||||
|
server.daemon_threads = True
|
||||||
|
print(f"Mock-TTS-Worker auf Port {PORT}", file=sys.stderr)
|
||||||
|
try:
|
||||||
|
server.serve_forever()
|
||||||
|
except KeyboardInterrupt:
|
||||||
|
pass
|
||||||
|
finally:
|
||||||
|
server.server_close()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
+99
-1
@@ -6,13 +6,14 @@ cd "$(dirname "$0")/.."
|
|||||||
|
|
||||||
UP_PORT=18080
|
UP_PORT=18080
|
||||||
RT_PORT=18081
|
RT_PORT=18081
|
||||||
|
TTS_PORT=18082
|
||||||
BASE="http://127.0.0.1:$RT_PORT"
|
BASE="http://127.0.0.1:$RT_PORT"
|
||||||
FAKE_DIR="$PWD/dev/fake-profile-dir"
|
FAKE_DIR="$PWD/dev/fake-profile-dir"
|
||||||
PASS=0
|
PASS=0
|
||||||
FAIL=0
|
FAIL=0
|
||||||
|
|
||||||
cleanup() {
|
cleanup() {
|
||||||
kill "${MOCK_PID:-}" "${ROUTER_PID:-}" 2>/dev/null || true
|
kill "${MOCK_PID:-}" "${ROUTER_PID:-}" "${TTS_PID:-}" 2>/dev/null || true
|
||||||
rm -f /tmp/mock_pid2 /tmp/mock_upstream_pid
|
rm -f /tmp/mock_pid2 /tmp/mock_upstream_pid
|
||||||
wait 2>/dev/null || true
|
wait 2>/dev/null || true
|
||||||
}
|
}
|
||||||
@@ -52,11 +53,21 @@ IMAGE_DIR=/tmp/test-images \
|
|||||||
IMAGE_WORKER_LOG=/tmp/test_worker.log \
|
IMAGE_WORKER_LOG=/tmp/test_worker.log \
|
||||||
IMAGE_GEN_TIMEOUT=30 \
|
IMAGE_GEN_TIMEOUT=30 \
|
||||||
MOCK_WORKER_LOG=/tmp/test_worker_requests.jsonl \
|
MOCK_WORKER_LOG=/tmp/test_worker_requests.jsonl \
|
||||||
|
TTS_WORKER_URL="http://127.0.0.1:$TTS_PORT" \
|
||||||
python3 router/ai_profile_router.py >/tmp/router_test.log 2>&1 &
|
python3 router/ai_profile_router.py >/tmp/router_test.log 2>&1 &
|
||||||
ROUTER_PID=$!
|
ROUTER_PID=$!
|
||||||
sleep 0.5
|
sleep 0.5
|
||||||
rm -f /tmp/test_worker_requests.jsonl
|
rm -f /tmp/test_worker_requests.jsonl
|
||||||
|
|
||||||
|
# --- Mock-TTS-Worker starten ----------------------------------------------------
|
||||||
|
echo "== Starte Mock-TTS-Worker (Port $TTS_PORT)"
|
||||||
|
MOCK_TTS_PORT="$TTS_PORT" MOCK_TTS_DELAY=0.1 \
|
||||||
|
MOCK_TTS_LOG=/tmp/test_tts_requests.jsonl \
|
||||||
|
python3 dev/mock_tts_worker.py >/tmp/mock_tts.log 2>&1 &
|
||||||
|
TTS_PID=$!
|
||||||
|
sleep 0.5
|
||||||
|
rm -f /tmp/test_tts_requests.jsonl
|
||||||
|
|
||||||
# --- 1. /v1/models -------------------------------------------------------------
|
# --- 1. /v1/models -------------------------------------------------------------
|
||||||
echo "== Test 1: /v1/models"
|
echo "== Test 1: /v1/models"
|
||||||
RESP=$(curl -sf "$BASE/v1/models")
|
RESP=$(curl -sf "$BASE/v1/models")
|
||||||
@@ -395,6 +406,93 @@ wait $IMG_PID
|
|||||||
[ "$CODE" = "200" ] && [ "$ELAPSED" -ge 2 ] \
|
[ "$CODE" = "200" ] && [ "$ELAPSED" -ge 2 ] \
|
||||||
&& ok "Chat wartete ${ELAPSED}s (kein 502), dann 200" || bad "Chat: Code $CODE, ${ELAPSED}s"
|
&& ok "Chat wartete ${ELAPSED}s (kein 502), dann 200" || bad "Chat: Code $CODE, ${ELAPSED}s"
|
||||||
|
|
||||||
|
# --- 27. TTS: /status zeigt tts-Section ---------------------------------------------------------------
|
||||||
|
echo "== Test 27: /status mit tts-Section"
|
||||||
|
RESP=$(curl -sf "$BASE/status")
|
||||||
|
echo "$RESP" | python3 -m json.tool
|
||||||
|
echo "$RESP" | python3 -c '
|
||||||
|
import json,sys
|
||||||
|
d=json.load(sys.stdin)
|
||||||
|
tts=d["tts"]
|
||||||
|
assert tts["reachable"] is True, tts
|
||||||
|
assert tts["ready"] is True, tts
|
||||||
|
assert set(tts["voices"])=={"martin","victoria"}, tts
|
||||||
|
' && ok "Status: TTS erreichbar, bereit, 2 Stimmen" || bad "Status tts-Section"
|
||||||
|
|
||||||
|
# --- 28. TTS: POST /v1/audio/speech (wav) ---------------------------------------------------------------
|
||||||
|
echo "== Test 28: POST /v1/audio/speech (wav)"
|
||||||
|
CODE=$(curl -s -o /tmp/tts28.wav -w "%{http_code}" -D /tmp/hdr28.txt \
|
||||||
|
"$BASE/v1/audio/speech" -H "Content-Type: application/json" \
|
||||||
|
-d '{"model":"kokoro-german","input":"Hallo Welt","voice":"martin","response_format":"wav"}')
|
||||||
|
CTYPE=$(grep -i content-type /tmp/hdr28.txt | tr -d "\r")
|
||||||
|
[ "$CODE" = "200" ] && [ -s /tmp/tts28.wav ] && echo "$CTYPE" | grep -qi "audio/wav" \
|
||||||
|
&& ok "TTS wav (200, $CTYPE, $(stat -f%z /tmp/tts28.wav 2>/dev/null || stat -c%s /tmp/tts28.wav) Bytes)" \
|
||||||
|
|| bad "TTS wav (Code $CODE, $CTYPE)"
|
||||||
|
|
||||||
|
# --- 29. TTS: POST /v1/audio/speech (mp3, Default) -------------------------------------------------------
|
||||||
|
echo "== Test 29: POST /v1/audio/speech (mp3, Default)"
|
||||||
|
CODE=$(curl -s -o /tmp/tts29.mp3 -w "%{http_code}" -D /tmp/hdr29.txt \
|
||||||
|
"$BASE/v1/audio/speech" -H "Content-Type: application/json" \
|
||||||
|
-d '{"input":"Guten Tag","voice":"victoria"}')
|
||||||
|
CTYPE=$(grep -i content-type /tmp/hdr29.txt | tr -d "\r")
|
||||||
|
[ "$CODE" = "200" ] && [ -s /tmp/tts29.mp3 ] && echo "$CTYPE" | grep -qi "audio/mpeg" \
|
||||||
|
&& ok "TTS mp3 (200, $CTYPE)" || bad "TTS mp3 (Code $CODE, $CTYPE)"
|
||||||
|
|
||||||
|
# --- 30. TTS: Validierung --------------------------------------------------------------------------------
|
||||||
|
echo "== Test 30: TTS-Validierung"
|
||||||
|
CODE=$(curl -s -o /tmp/err30a.json -w "%{http_code}" "$BASE/v1/audio/speech" \
|
||||||
|
-H "Content-Type: application/json" -d '{"voice":"martin"}')
|
||||||
|
cat /tmp/err30a.json; echo
|
||||||
|
[ "$CODE" = "400" ] && ok "400 bei fehlendem input" || bad "erwartet 400, bekam $CODE"
|
||||||
|
|
||||||
|
CODE=$(curl -s -o /tmp/err30b.json -w "%{http_code}" "$BASE/v1/audio/speech" \
|
||||||
|
-H "Content-Type: application/json" -d '{"input":"x","voice":"bogus"}')
|
||||||
|
cat /tmp/err30b.json; echo
|
||||||
|
[ "$CODE" = "400" ] && ok "400 bei ungültiger Stimme" || bad "erwartet 400, bekam $CODE"
|
||||||
|
|
||||||
|
CODE=$(curl -s -o /tmp/err30c.json -w "%{http_code}" "$BASE/v1/audio/speech" \
|
||||||
|
-H "Content-Type: application/json" -d '{"input":"x","response_format":"ogg"}')
|
||||||
|
cat /tmp/err30c.json; echo
|
||||||
|
[ "$CODE" = "400" ] && ok "400 bei ungültigem Format" || bad "erwartet 400, bekam $CODE"
|
||||||
|
|
||||||
|
CODE=$(curl -s -o /tmp/err30d.json -w "%{http_code}" "$BASE/v1/audio/speech" \
|
||||||
|
-H "Content-Type: application/json" -d '{"input":"x","model":"gpt-4"}')
|
||||||
|
cat /tmp/err30d.json; echo
|
||||||
|
[ "$CODE" = "400" ] && ok "400 bei unbekanntem Modell" || bad "erwartet 400, bekam $CODE"
|
||||||
|
|
||||||
|
# --- 31. TTS: Worker-Fehler → 503 ------------------------------------------------------------------------
|
||||||
|
echo "== Test 31: TTS-Worker-Fehler → 503"
|
||||||
|
CODE=$(curl -s -o /tmp/err31.json -w "%{http_code}" "$BASE/v1/audio/speech" \
|
||||||
|
-H "Content-Type: application/json" -d '{"input":"FAIL","voice":"martin"}')
|
||||||
|
cat /tmp/err31.json; echo
|
||||||
|
[ "$CODE" = "503" ] && ok "503 bei TTS-Worker-Fehler" || bad "erwartet 503, bekam $CODE"
|
||||||
|
|
||||||
|
# --- 32. TTS: Worker down → 503 ---------------------------------------------------------------------------
|
||||||
|
echo "== Test 32: TTS-Worker down → 503"
|
||||||
|
kill "$TTS_PID" 2>/dev/null || true
|
||||||
|
sleep 0.5
|
||||||
|
CODE=$(curl -s -o /tmp/err32.json -w "%{http_code}" "$BASE/v1/audio/speech" \
|
||||||
|
-H "Content-Type: application/json" -d '{"input":"Hallo","voice":"martin"}')
|
||||||
|
cat /tmp/err32.json; echo
|
||||||
|
[ "$CODE" = "503" ] && ok "503 bei downem TTS-Worker" || bad "erwartet 503, bekam $CODE"
|
||||||
|
RESP=$(curl -sf "$BASE/status")
|
||||||
|
echo "$RESP" | python3 -c '
|
||||||
|
import json,sys
|
||||||
|
d=json.load(sys.stdin)
|
||||||
|
assert d["tts"]["reachable"] is False, d["tts"]
|
||||||
|
' && ok "Status: TTS nicht erreichbar" || bad "Status nach TTS-Down"
|
||||||
|
|
||||||
|
# --- 33. TTS: Worker-Neustart → Recovery -------------------------------------------------------------------
|
||||||
|
echo "== Test 33: TTS-Worker-Neustart → Recovery"
|
||||||
|
MOCK_TTS_PORT="$TTS_PORT" MOCK_TTS_DELAY=0.1 \
|
||||||
|
python3 dev/mock_tts_worker.py >/tmp/mock_tts2.log 2>&1 &
|
||||||
|
TTS_PID=$!
|
||||||
|
sleep 0.5
|
||||||
|
CODE=$(curl -s -o /tmp/tts33.wav -w "%{http_code}" "$BASE/v1/audio/speech" \
|
||||||
|
-H "Content-Type: application/json" -d '{"input":"Wieder da","voice":"martin","response_format":"wav"}')
|
||||||
|
[ "$CODE" = "200" ] && [ -s /tmp/tts33.wav ] \
|
||||||
|
&& ok "TTS nach Neustart wieder verfügbar" || bad "TTS-Recovery (Code $CODE)"
|
||||||
|
|
||||||
# --- Ergebnis --------------------------------------------------------------------------------------------
|
# --- Ergebnis --------------------------------------------------------------------------------------------
|
||||||
echo
|
echo
|
||||||
echo "== Ergebnis: $PASS bestanden, $FAIL fehlgeschlagen =="
|
echo "== Ergebnis: $PASS bestanden, $FAIL fehlgeschlagen =="
|
||||||
|
|||||||
@@ -20,6 +20,13 @@ Bildgenerierung (FLUX.2 [klein] 4B Base):
|
|||||||
GET /images (Liste)
|
GET /images (Liste)
|
||||||
GET /images/<datei> (PNG-Download)
|
GET /images/<datei> (PNG-Download)
|
||||||
|
|
||||||
|
Sprachausgabe (Kokoro-82M, deutsch, CPU-only):
|
||||||
|
POST /v1/audio/speech (OpenAI-kompatibel)
|
||||||
|
|
||||||
|
Der TTS-Worker (mike-ai-kokoro.service) läuft als separater, langlebiger
|
||||||
|
Prozess mit eigenem Venv und hält die Modelle dauerhaft im RAM. Der
|
||||||
|
Router leitet /v1/audio/speech per HTTP an den Worker weiter.
|
||||||
|
|
||||||
Der Router agiert als Modell-Orchestrator: vor der Generierung wird
|
Der Router agiert als Modell-Orchestrator: vor der Generierung wird
|
||||||
llama.cpp gestoppt, der Bild-Worker lädt FLUX, generiert und entlädt
|
llama.cpp gestoppt, der Bild-Worker lädt FLUX, generiert und entlädt
|
||||||
das Modell wieder; danach wird das vorherige Qwen-Profil wiederher-
|
das Modell wieder; danach wird das vorherige Qwen-Profil wiederher-
|
||||||
@@ -91,6 +98,16 @@ IMAGE_QUALITY = {"standard": 30, "high": 50}
|
|||||||
IMAGE_DEFAULT_QUALITY = "standard"
|
IMAGE_DEFAULT_QUALITY = "standard"
|
||||||
IMAGE_MAX_N = 4
|
IMAGE_MAX_N = 4
|
||||||
|
|
||||||
|
# --- Sprachausgabe (Kokoro-82M, deutsch, CPU-only) ---
|
||||||
|
TTS_WORKER_URL = os.environ.get("TTS_WORKER_URL", "http://127.0.0.1:8082")
|
||||||
|
TTS_TIMEOUT = float(os.environ.get("TTS_TIMEOUT", "300")) # s, pro Synthese
|
||||||
|
TTS_CONNECT_TIMEOUT = float(os.environ.get("TTS_CONNECT_TIMEOUT", "5"))
|
||||||
|
TTS_MODEL = "kokoro-german" # virtuelles Modell für /v1/audio/speech
|
||||||
|
TTS_VOICES = ("martin", "victoria")
|
||||||
|
TTS_DEFAULT_VOICE = "martin"
|
||||||
|
TTS_FORMATS = ("mp3", "wav", "flac", "pcm")
|
||||||
|
TTS_DEFAULT_FORMAT = "mp3"
|
||||||
|
|
||||||
# Chat-Waiting: Während eines Image-Jobs oder Profilwechsels ist Qwen
|
# Chat-Waiting: Während eines Image-Jobs oder Profilwechsels ist Qwen
|
||||||
# down. Chat-Requests warten (statt 502) bis Qwen wieder bereit ist.
|
# down. Chat-Requests warten (statt 502) bis Qwen wieder bereit ist.
|
||||||
CHAT_WAIT_TIMEOUT = float(os.environ.get("CHAT_WAIT_TIMEOUT", "300")) # s, max. Warten
|
CHAT_WAIT_TIMEOUT = float(os.environ.get("CHAT_WAIT_TIMEOUT", "300")) # s, max. Warten
|
||||||
@@ -189,6 +206,56 @@ def _set_qwen_unavailable(unavailable: bool) -> None:
|
|||||||
# Upstream (llama.cpp)
|
# Upstream (llama.cpp)
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def tts_status() -> dict:
|
||||||
|
"""Prüft den TTS-Worker: erreichbar? bereit? welche Stimmen?"""
|
||||||
|
hostport = TTS_WORKER_URL.split("://", 1)[-1]
|
||||||
|
host, _, port = hostport.partition(":")
|
||||||
|
try:
|
||||||
|
conn = http.client.HTTPConnection(host, int(port) if port else 80,
|
||||||
|
timeout=TTS_CONNECT_TIMEOUT)
|
||||||
|
conn.request("GET", "/status")
|
||||||
|
resp = conn.getresponse()
|
||||||
|
data = json.loads(resp.read())
|
||||||
|
conn.close()
|
||||||
|
return {"reachable": True, **data}
|
||||||
|
except (OSError, ValueError) as e:
|
||||||
|
return {"reachable": False, "error": str(e)}
|
||||||
|
|
||||||
|
|
||||||
|
def tts_synthesize(text: str, voice: str, speed: float,
|
||||||
|
fmt: str) -> tuple[bytes, str]:
|
||||||
|
"""Synthetisiert Audio über den TTS-Worker.
|
||||||
|
|
||||||
|
Liefert (audio_bytes, content_type). Wirft RuntimeError bei Fehler.
|
||||||
|
"""
|
||||||
|
hostport = TTS_WORKER_URL.split("://", 1)[-1]
|
||||||
|
host, _, port = hostport.partition(":")
|
||||||
|
payload = json.dumps({"text": text, "voice": voice,
|
||||||
|
"speed": speed, "format": fmt}).encode()
|
||||||
|
try:
|
||||||
|
conn = http.client.HTTPConnection(host, int(port) if port else 80,
|
||||||
|
timeout=TTS_CONNECT_TIMEOUT)
|
||||||
|
conn.request("POST", "/tts", body=payload,
|
||||||
|
headers={"Content-Type": "application/json"})
|
||||||
|
conn.sock.settimeout(TTS_TIMEOUT)
|
||||||
|
resp = conn.getresponse()
|
||||||
|
body = resp.read()
|
||||||
|
conn.close()
|
||||||
|
except (OSError, http.client.HTTPException) as e:
|
||||||
|
raise RuntimeError(f"TTS-Worker nicht erreichbar: {e}")
|
||||||
|
if resp.status != 200:
|
||||||
|
try:
|
||||||
|
err = json.loads(body)
|
||||||
|
msg = err.get("error", str(err))
|
||||||
|
except ValueError:
|
||||||
|
msg = body.decode(errors="replace")[:200]
|
||||||
|
raise RuntimeError(f"TTS-Fehler ({resp.status}): {msg}")
|
||||||
|
content_type = {"mp3": "audio/mpeg", "wav": "audio/wav",
|
||||||
|
"flac": "audio/flac",
|
||||||
|
"pcm": "application/octet-stream"}[fmt]
|
||||||
|
return body, content_type
|
||||||
|
|
||||||
|
|
||||||
def upstream_status() -> dict:
|
def upstream_status() -> dict:
|
||||||
"""Prüft llama.cpp: erreichbar? welches Modell? welcher Kontext?"""
|
"""Prüft llama.cpp: erreichbar? welches Modell? welcher Kontext?"""
|
||||||
try:
|
try:
|
||||||
@@ -617,6 +684,8 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
self._send_json(200, self._status_payload())
|
self._send_json(200, self._status_payload())
|
||||||
elif path == "/v1/images/generations" and self.command == "POST":
|
elif path == "/v1/images/generations" and self.command == "POST":
|
||||||
self._image_generate()
|
self._image_generate()
|
||||||
|
elif path == "/v1/audio/speech" and self.command == "POST":
|
||||||
|
self._speech()
|
||||||
elif path == "/images" and self.command == "GET":
|
elif path == "/images" and self.command == "GET":
|
||||||
self._images_list()
|
self._images_list()
|
||||||
elif path.startswith("/images/") and self.command == "GET":
|
elif path.startswith("/images/") and self.command == "GET":
|
||||||
@@ -689,6 +758,7 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
"last_seconds": img.last_seconds,
|
"last_seconds": img.last_seconds,
|
||||||
"last_error": img.last_error,
|
"last_error": img.last_error,
|
||||||
},
|
},
|
||||||
|
"tts": tts_status(),
|
||||||
}
|
}
|
||||||
|
|
||||||
# ---------- Bildgenerierung ----------
|
# ---------- Bildgenerierung ----------
|
||||||
@@ -854,6 +924,84 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
self.end_headers()
|
self.end_headers()
|
||||||
self.wfile.write(data)
|
self.wfile.write(data)
|
||||||
|
|
||||||
|
# ---------- Sprachausgabe (Kokoro) ----------
|
||||||
|
|
||||||
|
def _speech(self) -> None:
|
||||||
|
length = int(self.headers.get("Content-Length") or 0)
|
||||||
|
try:
|
||||||
|
data = json.loads(self.rfile.read(length))
|
||||||
|
except ValueError:
|
||||||
|
self._send_error(400, "ungültiges JSON",
|
||||||
|
"invalid_request_error", "invalid_json")
|
||||||
|
return
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
self._send_error(400, "Request muss ein JSON-Objekt sein",
|
||||||
|
"invalid_request_error", "invalid_request")
|
||||||
|
return
|
||||||
|
|
||||||
|
# input (OpenAI) – auch 'text' akzeptieren (bequemer für curl)
|
||||||
|
text = data.get("input", data.get("text"))
|
||||||
|
if not isinstance(text, str) or not text.strip():
|
||||||
|
self._send_error(400, "'input' fehlt oder ist leer",
|
||||||
|
"invalid_request_error", "missing_input")
|
||||||
|
return
|
||||||
|
if len(text) > 8000:
|
||||||
|
self._send_error(400, "'input' zu lang (max 8000 Zeichen)",
|
||||||
|
"invalid_request_error", "input_too_long")
|
||||||
|
return
|
||||||
|
|
||||||
|
voice = data.get("voice", TTS_DEFAULT_VOICE)
|
||||||
|
if voice not in TTS_VOICES:
|
||||||
|
self._send_error(
|
||||||
|
400, f"ungültige Stimme: {voice!r} "
|
||||||
|
f"(erlaubt: {', '.join(TTS_VOICES)})",
|
||||||
|
"invalid_request_error", "invalid_voice")
|
||||||
|
return
|
||||||
|
|
||||||
|
fmt = data.get("response_format", TTS_DEFAULT_FORMAT)
|
||||||
|
if fmt not in TTS_FORMATS:
|
||||||
|
self._send_error(
|
||||||
|
400, f"ungültiges response_format: {fmt!r} "
|
||||||
|
f"(erlaubt: {', '.join(TTS_FORMATS)})",
|
||||||
|
"invalid_request_error", "invalid_format")
|
||||||
|
return
|
||||||
|
|
||||||
|
speed = data.get("speed", 1.0)
|
||||||
|
try:
|
||||||
|
speed = float(speed)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
self._send_error(400, "'speed' muss eine Zahl sein",
|
||||||
|
"invalid_request_error", "invalid_speed")
|
||||||
|
return
|
||||||
|
if not 0.5 <= speed <= 2.0:
|
||||||
|
self._send_error(400, "'speed' muss zwischen 0.5 und 2.0 sein",
|
||||||
|
"invalid_request_error", "invalid_speed")
|
||||||
|
return
|
||||||
|
|
||||||
|
# Modell-Name optional; falls angegeben, muss es kokoro-german sein.
|
||||||
|
model = data.get("model")
|
||||||
|
if model is not None and model != TTS_MODEL:
|
||||||
|
self._send_error(400, f"unbekanntes Modell: {model!r} "
|
||||||
|
f"(erwartet: {TTS_MODEL})",
|
||||||
|
"invalid_request_error", "unknown_model")
|
||||||
|
return
|
||||||
|
|
||||||
|
self.timeout = None # Synthese kann dauern
|
||||||
|
try:
|
||||||
|
audio, content_type = tts_synthesize(
|
||||||
|
text.strip(), voice, speed, fmt)
|
||||||
|
except RuntimeError as e:
|
||||||
|
self._send_error(503, str(e), "server_error", "tts_failed")
|
||||||
|
return
|
||||||
|
|
||||||
|
self._last_code = 200
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Type", content_type)
|
||||||
|
self.send_header("Content-Length", str(len(audio)))
|
||||||
|
self.send_header("Connection", "close")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(audio)
|
||||||
|
|
||||||
def _switch(self, profile: str) -> None:
|
def _switch(self, profile: str) -> None:
|
||||||
if profile not in PROFILES:
|
if profile not in PROFILES:
|
||||||
self._send_error(400, f"unbekanntes Profil: {profile}",
|
self._send_error(400, f"unbekanntes Profil: {profile}",
|
||||||
|
|||||||
@@ -0,0 +1,316 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Kokoro TTS Worker – langlebiger HTTP-Service für deutsche Sprachausgabe.
|
||||||
|
|
||||||
|
Lädt die Kokoro-82M-Feintuning-Modelle (kikiri-german-martin,
|
||||||
|
kikiri-german-victoria) einmalig beim Start und hält sie dauerhaft im
|
||||||
|
RAM (CPU-only, niedrige Warm-Start-Latenz). Wird als eigener
|
||||||
|
systemd-Service betrieben und vom AI Profile Router über HTTP
|
||||||
|
angesprochen (POST /v1/audio/speech -> POST /tts).
|
||||||
|
|
||||||
|
API:
|
||||||
|
GET /status -> {"status":"ok","ready":bool,"voices":[...],...}
|
||||||
|
POST /tts -> {"text":"...","voice":"martin","speed":1.0,
|
||||||
|
"format":"wav|mp3|flac|pcm"} -> binäres Audio
|
||||||
|
|
||||||
|
Logging nach stderr (journald). stdout bleibt frei.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Konfiguration (über Umgebungsvariablen, vgl. systemd-Unit)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
HOST = os.environ.get("KOKORO_HOST", "127.0.0.1")
|
||||||
|
PORT = int(os.environ.get("KOKORO_PORT", "8082"))
|
||||||
|
MODEL_DIR = os.environ.get("KOKORO_MODEL_DIR", "/opt/mike-ai/models/kokoro")
|
||||||
|
|
||||||
|
# Stimmen: Name -> Pfade relativ zu MODEL_DIR.
|
||||||
|
# Beide Modelle sind eigenständige Kokoro-82M-Feintunings (Apache 2.0).
|
||||||
|
VOICES = {
|
||||||
|
"martin": {
|
||||||
|
"config": "kikiri-german-martin/config.json",
|
||||||
|
"model": "kikiri-german-martin/kikiri_german_martin_ep10.pth",
|
||||||
|
"voice": "kikiri-german-martin/voices/martin.pt",
|
||||||
|
},
|
||||||
|
"victoria": {
|
||||||
|
"config": "kikiri-german-victoria/config.json",
|
||||||
|
"model": "kikiri-german-victoria/kikiri_german_victoria_ep10.pth",
|
||||||
|
"voice": "kikiri-german-victoria/voices/victoria.pt",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
DEFAULT_VOICE = "martin"
|
||||||
|
SAMPLE_RATE = 24000 # Kokoro-nativ
|
||||||
|
SPEED_MIN, SPEED_MAX = 0.5, 2.0
|
||||||
|
MAX_TEXT_LEN = 8000 # Zeichen pro Request
|
||||||
|
|
||||||
|
log = logging.getLogger("kokoro-tts")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Worker
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class TTSWorker:
|
||||||
|
"""Hält die geladenen Pipelines und serialisiert die Synthese."""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.pipelines: dict[str, tuple] = {} # name -> (KPipeline, voice)
|
||||||
|
self.load_errors: list[str] = []
|
||||||
|
self.ready = False
|
||||||
|
self.started = time.time()
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
self.last_seconds: float | None = None
|
||||||
|
self.last_voice: str | None = None
|
||||||
|
self.last_error: str | None = None
|
||||||
|
self.total_requests = 0
|
||||||
|
|
||||||
|
def load(self) -> None:
|
||||||
|
"""Lädt alle Stimmen (CPU-only)."""
|
||||||
|
import torch
|
||||||
|
from kokoro import KModel, KPipeline
|
||||||
|
# Deutsche Sprachunterstützung (kokoro PR #340): 'de' -> espeak-ng G2P.
|
||||||
|
from kokoro import pipeline as _pipeline_mod
|
||||||
|
_pipeline_mod.ALIASES.setdefault("de", "d")
|
||||||
|
_pipeline_mod.LANG_CODES.setdefault("d", "de")
|
||||||
|
|
||||||
|
for name, cfg in VOICES.items():
|
||||||
|
t0 = time.monotonic()
|
||||||
|
try:
|
||||||
|
kmodel = KModel(
|
||||||
|
config=os.path.join(MODEL_DIR, cfg["config"]),
|
||||||
|
model=os.path.join(MODEL_DIR, cfg["model"]),
|
||||||
|
).to("cpu").eval()
|
||||||
|
pipeline = KPipeline(lang_code="de", model=kmodel)
|
||||||
|
voice = torch.load(
|
||||||
|
os.path.join(MODEL_DIR, cfg["voice"]),
|
||||||
|
map_location="cpu", weights_only=True)
|
||||||
|
self.pipelines[name] = (pipeline, voice)
|
||||||
|
log.info("Stimme geladen: %s (%.1f s)", name,
|
||||||
|
time.monotonic() - t0)
|
||||||
|
except Exception as e:
|
||||||
|
self.load_errors.append(f"{name}: {e}")
|
||||||
|
log.error("Stimme %s nicht ladbar: %s", name, e)
|
||||||
|
self.ready = True
|
||||||
|
|
||||||
|
def synthesize(self, text: str, voice: str, speed: float) -> bytes:
|
||||||
|
"""Synthetisiert Audio und liefert es als WAV-Bytes (24 kHz)."""
|
||||||
|
with self._lock:
|
||||||
|
if voice not in self.pipelines:
|
||||||
|
raise ValueError(f"unbekannte Stimme: {voice!r} "
|
||||||
|
f"(erlaubt: {', '.join(VOICES)})")
|
||||||
|
pipeline, voice_pack = self.pipelines[voice]
|
||||||
|
t0 = time.monotonic()
|
||||||
|
chunks = list(pipeline(text, voice=voice_pack, speed=speed))
|
||||||
|
audios = [c[2] for c in chunks if c[2] is not None]
|
||||||
|
if not audios:
|
||||||
|
raise RuntimeError("keine Audio-Daten erzeugt")
|
||||||
|
import torch
|
||||||
|
audio = torch.cat(audios, dim=0)
|
||||||
|
seconds = time.monotonic() - t0
|
||||||
|
self.last_seconds = seconds
|
||||||
|
self.last_voice = voice
|
||||||
|
self.total_requests += 1
|
||||||
|
return _to_wav_bytes(audio)
|
||||||
|
|
||||||
|
|
||||||
|
def _to_wav_bytes(audio) -> bytes:
|
||||||
|
"""1D-Tensor (24 kHz) -> WAV-Bytes (16-bit PCM)."""
|
||||||
|
import io
|
||||||
|
import numpy as np
|
||||||
|
import soundfile as sf
|
||||||
|
arr = audio.detach().cpu().numpy().astype("float32")
|
||||||
|
buf = io.BytesIO()
|
||||||
|
sf.write(buf, arr, SAMPLE_RATE, format="WAV", subtype="PCM_16")
|
||||||
|
return buf.getvalue()
|
||||||
|
|
||||||
|
|
||||||
|
def _wav_to_format(wav_bytes: bytes, fmt: str) -> bytes:
|
||||||
|
"""WAV-Bytes in das Zielformat konvertieren."""
|
||||||
|
if fmt == "wav":
|
||||||
|
return wav_bytes
|
||||||
|
import io
|
||||||
|
import numpy as np
|
||||||
|
import soundfile as sf
|
||||||
|
data, sr = sf.read(io.BytesIO(wav_bytes), dtype="float32")
|
||||||
|
if fmt == "flac":
|
||||||
|
buf = io.BytesIO()
|
||||||
|
sf.write(buf, data, sr, format="FLAC")
|
||||||
|
return buf.getvalue()
|
||||||
|
if fmt == "pcm":
|
||||||
|
# 16-bit PCM, little-endian, mono
|
||||||
|
pcm = (np.clip(data, -1.0, 1.0) * 32767).astype("<i2")
|
||||||
|
return pcm.tobytes()
|
||||||
|
if fmt == "mp3":
|
||||||
|
import lameenc
|
||||||
|
pcm = (np.clip(data, -1.0, 1.0) * 32767).astype("<i2")
|
||||||
|
encoder = lameenc.Encoder()
|
||||||
|
encoder.set_bit_rate(128)
|
||||||
|
encoder.set_in_sample_rate(sr)
|
||||||
|
encoder.set_channels(1)
|
||||||
|
mp3 = encoder.encode(pcm.tobytes())
|
||||||
|
mp3 += encoder.flush()
|
||||||
|
return mp3
|
||||||
|
raise ValueError(f"unbekanntes Format: {fmt!r}")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# HTTP-Handler
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
WORKER = TTSWorker()
|
||||||
|
|
||||||
|
|
||||||
|
class Handler(BaseHTTPRequestHandler):
|
||||||
|
server_version = "KokoroTTS/1.0"
|
||||||
|
timeout = 300 # s, Synthese kann dauern
|
||||||
|
|
||||||
|
def log_message(self, fmt, *args): # noqa: N802
|
||||||
|
log.info("%s %s", self.address_string(), fmt % args)
|
||||||
|
|
||||||
|
def do_GET(self): # noqa: N802
|
||||||
|
if self.path.split("?", 1)[0] == "/status":
|
||||||
|
self._send_json(200, self._status_payload())
|
||||||
|
else:
|
||||||
|
self._send_json(404, {"error": "not found"})
|
||||||
|
|
||||||
|
def do_POST(self): # noqa: N802
|
||||||
|
if self.path.split("?", 1)[0] == "/tts":
|
||||||
|
self._tts()
|
||||||
|
else:
|
||||||
|
self._send_json(404, {"error": "not found"})
|
||||||
|
|
||||||
|
def _status_payload(self) -> dict:
|
||||||
|
return {
|
||||||
|
"status": "ok",
|
||||||
|
"ready": WORKER.ready,
|
||||||
|
"voices": list(VOICES),
|
||||||
|
"default_voice": DEFAULT_VOICE,
|
||||||
|
"load_errors": WORKER.load_errors,
|
||||||
|
"sample_rate": SAMPLE_RATE,
|
||||||
|
"uptime_seconds": round(time.time() - WORKER.started, 1),
|
||||||
|
"total_requests": WORKER.total_requests,
|
||||||
|
"last_seconds": WORKER.last_seconds,
|
||||||
|
"last_voice": WORKER.last_voice,
|
||||||
|
"last_error": WORKER.last_error,
|
||||||
|
}
|
||||||
|
|
||||||
|
def _tts(self) -> None:
|
||||||
|
length = int(self.headers.get("Content-Length") or 0)
|
||||||
|
try:
|
||||||
|
data = json.loads(self.rfile.read(length))
|
||||||
|
except ValueError:
|
||||||
|
self._send_json(400, {"error": "ungültiges JSON"})
|
||||||
|
return
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
self._send_json(400, {"error": "Request muss ein JSON-Objekt sein"})
|
||||||
|
return
|
||||||
|
|
||||||
|
text = data.get("text")
|
||||||
|
if not isinstance(text, str) or not text.strip():
|
||||||
|
self._send_json(400, {"error": "'text' fehlt oder ist leer"})
|
||||||
|
return
|
||||||
|
if len(text) > MAX_TEXT_LEN:
|
||||||
|
self._send_json(400, {"error": f"'text' zu lang (max {MAX_TEXT_LEN})"})
|
||||||
|
return
|
||||||
|
|
||||||
|
voice = data.get("voice", DEFAULT_VOICE)
|
||||||
|
if voice not in VOICES:
|
||||||
|
self._send_json(400, {"error": f"unbekannte Stimme: {voice!r}"})
|
||||||
|
return
|
||||||
|
|
||||||
|
speed = data.get("speed", 1.0)
|
||||||
|
try:
|
||||||
|
speed = float(speed)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
self._send_json(400, {"error": "'speed' muss eine Zahl sein"})
|
||||||
|
return
|
||||||
|
if not SPEED_MIN <= speed <= SPEED_MAX:
|
||||||
|
self._send_json(400, {"error": f"'speed' muss zwischen "
|
||||||
|
f"{SPEED_MIN} und {SPEED_MAX} sein"})
|
||||||
|
return
|
||||||
|
|
||||||
|
fmt = data.get("format", "wav")
|
||||||
|
if fmt not in ("wav", "mp3", "flac", "pcm"):
|
||||||
|
self._send_json(400, {"error": f"ungültiges Format: {fmt!r}"})
|
||||||
|
return
|
||||||
|
|
||||||
|
if not WORKER.ready:
|
||||||
|
self._send_json(503, {"error": "TTS-Worker lädt noch"})
|
||||||
|
return
|
||||||
|
if not WORKER.pipelines:
|
||||||
|
self._send_json(503, {"error": "keine Stimme geladen",
|
||||||
|
"load_errors": WORKER.load_errors})
|
||||||
|
return
|
||||||
|
|
||||||
|
try:
|
||||||
|
wav = WORKER.synthesize(text.strip(), voice, speed)
|
||||||
|
audio = _wav_to_format(wav, fmt)
|
||||||
|
except (ValueError, RuntimeError) as e:
|
||||||
|
WORKER.last_error = str(e)
|
||||||
|
self._send_json(500, {"error": str(e)})
|
||||||
|
return
|
||||||
|
|
||||||
|
content_type = {
|
||||||
|
"wav": "audio/wav",
|
||||||
|
"mp3": "audio/mpeg",
|
||||||
|
"flac": "audio/flac",
|
||||||
|
"pcm": "application/octet-stream",
|
||||||
|
}[fmt]
|
||||||
|
self._send_bytes(200, audio, content_type)
|
||||||
|
|
||||||
|
def _send_json(self, code: int, payload: dict) -> None:
|
||||||
|
body = json.dumps(payload, ensure_ascii=False).encode()
|
||||||
|
self.send_response(code)
|
||||||
|
self.send_header("Content-Type", "application/json")
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.send_header("Connection", "close")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
def _send_bytes(self, code: int, data: bytes, content_type: str) -> None:
|
||||||
|
self.send_response(code)
|
||||||
|
self.send_header("Content-Type", content_type)
|
||||||
|
self.send_header("Content-Length", str(len(data)))
|
||||||
|
self.send_header("Connection", "close")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(data)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Main
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
logging.basicConfig(
|
||||||
|
level=os.environ.get("LOG_LEVEL", "INFO"),
|
||||||
|
format="%(asctime)s %(levelname)s %(message)s",
|
||||||
|
stream=sys.stderr,
|
||||||
|
)
|
||||||
|
log.info("Kokoro TTS Worker startet: %s:%s (Modelle: %s)",
|
||||||
|
HOST, PORT, ", ".join(VOICES))
|
||||||
|
WORKER.load()
|
||||||
|
if not WORKER.pipelines:
|
||||||
|
log.error("Keine Stimme geladen – Worker bleibt trotzdem erreichbar "
|
||||||
|
"(/status zeigt load_errors)")
|
||||||
|
server = ThreadingHTTPServer((HOST, PORT), Handler)
|
||||||
|
server.daemon_threads = True
|
||||||
|
log.info("Kokoro TTS Worker bereit (%d Stimmen)", len(WORKER.pipelines))
|
||||||
|
try:
|
||||||
|
server.serve_forever()
|
||||||
|
except KeyboardInterrupt:
|
||||||
|
pass
|
||||||
|
finally:
|
||||||
|
server.server_close()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Reference in New Issue
Block a user