router: Bildgenerierung mit FLUX.2 [klein] 4B Base (GPU-Hotswap)
- POST /v1/images/generations (OpenAI-kompatibel, prompt/size/n/seed/quality) - quality: standard=30 Steps (Default), high=50 Steps - Größen: 1024x1024, 1536x1024, 1024x1536, 1920x1088, 1088x1920 - GPU-Hotswap: Qwen stoppen -> FLUX laden -> Bild -> FLUX entladen -> Qwen wiederherstellen (exakt vorheriges Profil) - Zentrales GPU/Modell-Lock (Profilwechsel und Bild teilen sich das Lock) - Chat-Requests warten während Bild-Job (kein 502), Timeout CHAT_WAIT_TIMEOUT - Robuste Recovery: try/finally, Worker-Beendigung, VRAM-Check, Qwen-Readiness - /status: image.phase, image.worker, image.model_loaded, qwen.available, qwen.active_chats - GET /images, GET /images/<datei> (validiert, nur images/-Verzeichnis) - image_worker.py: FLUX-Worker (eigener Prozess, JSON-Protokoll, bf16 + enable_model_cpu_offload) - deploy: venv (torch/diffusers/transformers/accelerate), Modell-Download, Image-Dir, systemd-Unit mit Image-Umgebungsvariablen - dev: Mock-Worker, fake-systemctl, Benchmarks (GPU-Resident, Offload, Steps, Quality-Compare), 32 lokale Tests - README: Bildgenerierung, Hotswap, Recovery, Benchmarks (RTX 5080), Python-Pakete Benchmarks (RTX 5080, 16 GB, CPU-Offload): - 512x512 / 10 Steps: ~9.3 s - 1024x1024 / 30 Steps: ~31.3 s - 1024x1024 / 50 Steps: ~45.3 s - 1920x1088 / 50 Steps: ~91 s - Peak-VRAM: ~8.4-8.9 GB - Hotswap-Gesamtzeit: ~41-42 s (1024x1024 / 30 Steps)
This commit is contained in:
@@ -0,0 +1,90 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Mock-Bild-Worker für lokale Tests (gleiche Protokoll wie image_worker.py).
|
||||
|
||||
Erzeugt ein minimales 1x1-PNG statt eines echten Bildes.
|
||||
|
||||
Optionen (Umgebungsvariablen):
|
||||
MOCK_WORKER_DELAY Sekunden, die pro generate geschlafen werden
|
||||
(Default 0.3). Für Tests von parallelen Requests.
|
||||
MOCK_WORKER_LOG Datei, in die die Requests geloggt werden (JSON-Zeilen).
|
||||
Für Tests, die die Steps/Qualität prüfen wollen.
|
||||
|
||||
Sonder-Prompts:
|
||||
"FAIL" -> Worker antwortet mit Fehler (simuliert OOM/Crash).
|
||||
"SLOW" -> Worker schläft 5 s (für Parallel-Tests).
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
# Minimales 1x1-PNG (1 Byte rot)
|
||||
PNG_1x1 = bytes.fromhex(
|
||||
"89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c4"
|
||||
"890000000d49444154789c626001000000ffff03000006000557bfabd40000"
|
||||
"000049454e44ae426082"
|
||||
)
|
||||
|
||||
DELAY = float(os.environ.get("MOCK_WORKER_DELAY", "0.3"))
|
||||
LOG_FILE = os.environ.get("MOCK_WORKER_LOG", "")
|
||||
|
||||
|
||||
def _emit(payload: dict) -> None:
|
||||
sys.stdout.write(json.dumps(payload) + "\n")
|
||||
sys.stdout.flush()
|
||||
|
||||
|
||||
def _log_request(req: dict) -> None:
|
||||
if not LOG_FILE:
|
||||
return
|
||||
try:
|
||||
with open(LOG_FILE, "a", encoding="utf-8") as f:
|
||||
f.write(json.dumps(req) + "\n")
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def main() -> None:
|
||||
loaded = False
|
||||
_emit({"status": "ready"})
|
||||
for line in sys.stdin:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
req = json.loads(line)
|
||||
except ValueError:
|
||||
_emit({"status": "error", "message": "ungültiges JSON"})
|
||||
continue
|
||||
cmd = req.get("cmd")
|
||||
if cmd == "generate":
|
||||
_log_request(req)
|
||||
prompt = req.get("prompt", "")
|
||||
if prompt == "SLOW":
|
||||
time.sleep(5.0) # langsame Generierung (Parallel-Tests)
|
||||
else:
|
||||
time.sleep(DELAY) # simulierte Generierung
|
||||
if prompt == "FAIL":
|
||||
_emit({"status": "error",
|
||||
"message": "simulierter Fehler (OOM)"})
|
||||
continue
|
||||
output = req["output"]
|
||||
os.makedirs(os.path.dirname(output) or ".", exist_ok=True)
|
||||
with open(output, "wb") as f:
|
||||
f.write(PNG_1x1)
|
||||
_emit({"status": "ok", "path": output, "seconds": DELAY,
|
||||
"load_seconds": 0.1})
|
||||
loaded = True
|
||||
elif cmd == "unload":
|
||||
loaded = False
|
||||
_emit({"status": "ok"})
|
||||
elif cmd == "status":
|
||||
_emit({"status": "ok", "model_loaded": loaded})
|
||||
else:
|
||||
_emit({"status": "error",
|
||||
"message": f"unbekanntes Kommando: {cmd}"})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user