Mikei386
7c5bbe2ffb
- POST /v1/images/generations (OpenAI-kompatibel, prompt/size/n/seed/quality) - quality: standard=30 Steps (Default), high=50 Steps - Größen: 1024x1024, 1536x1024, 1024x1536, 1920x1088, 1088x1920 - GPU-Hotswap: Qwen stoppen -> FLUX laden -> Bild -> FLUX entladen -> Qwen wiederherstellen (exakt vorheriges Profil) - Zentrales GPU/Modell-Lock (Profilwechsel und Bild teilen sich das Lock) - Chat-Requests warten während Bild-Job (kein 502), Timeout CHAT_WAIT_TIMEOUT - Robuste Recovery: try/finally, Worker-Beendigung, VRAM-Check, Qwen-Readiness - /status: image.phase, image.worker, image.model_loaded, qwen.available, qwen.active_chats - GET /images, GET /images/<datei> (validiert, nur images/-Verzeichnis) - image_worker.py: FLUX-Worker (eigener Prozess, JSON-Protokoll, bf16 + enable_model_cpu_offload) - deploy: venv (torch/diffusers/transformers/accelerate), Modell-Download, Image-Dir, systemd-Unit mit Image-Umgebungsvariablen - dev: Mock-Worker, fake-systemctl, Benchmarks (GPU-Resident, Offload, Steps, Quality-Compare), 32 lokale Tests - README: Bildgenerierung, Hotswap, Recovery, Benchmarks (RTX 5080), Python-Pakete Benchmarks (RTX 5080, 16 GB, CPU-Offload): - 512x512 / 10 Steps: ~9.3 s - 1024x1024 / 30 Steps: ~31.3 s - 1024x1024 / 50 Steps: ~45.3 s - 1920x1088 / 50 Steps: ~91 s - Peak-VRAM: ~8.4-8.9 GB - Hotswap-Gesamtzeit: ~41-42 s (1024x1024 / 30 Steps)
30 lines
1.1 KiB
Desktop File
30 lines
1.1 KiB
Desktop File
[Unit]
|
|
Description=Mike AI Profile Router (OpenAI-kompatibler Proxy + Bild-Orchestrierung, Port 8081)
|
|
After=network-online.target
|
|
Wants=network-online.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
ExecStart=/opt/mike-ai/ai-profile-router/venv/bin/python /opt/mike-ai/ai-profile-router/ai_profile_router.py
|
|
Restart=on-failure
|
|
RestartSec=3
|
|
Environment=ROUTER_HOST=0.0.0.0
|
|
Environment=ROUTER_PORT=8081
|
|
Environment=UPSTREAM_URL=http://127.0.0.1:8080
|
|
Environment=PROFILE_SCRIPT=/usr/local/bin/llama-profile
|
|
Environment=PROFILE_DIR=/etc/systemd/system/mike-ai-llama-ui.service.d
|
|
Environment=SWITCH_TIMEOUT=600
|
|
Environment=REQUEST_TIMEOUT=600
|
|
Environment=LLAMA_SERVICE=mike-ai-llama-ui.service
|
|
Environment=IMAGE_WORKER=/opt/mike-ai/ai-profile-router/image_worker.py
|
|
Environment=IMAGE_PYTHON=/opt/mike-ai/ai-profile-router/venv/bin/python
|
|
Environment=IMAGE_DIR=/opt/mike-ai/ai-profile-router/images
|
|
Environment=IMAGE_WORKER_LOG=/opt/mike-ai/ai-profile-router/worker.log
|
|
Environment=IMAGE_GEN_TIMEOUT=600
|
|
Environment=IMAGE_VRAM_FREE_TIMEOUT=120
|
|
NoNewPrivileges=true
|
|
PrivateTmp=true
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|