Add encrypted configuration backup and planned restore
This commit is contained in:
@@ -0,0 +1,64 @@
|
|||||||
|
# Backup und Wiederherstellung
|
||||||
|
|
||||||
|
**Einstellungen → Backup & Restore** erstellt eine `.adbackup`-Datei. Für die Verschlüsselung ist ein separates Kennwort mit mindestens 16 Zeichen nötig. Die Datei enthält auch Secrets und darf nur an vertrauenswürdige Stellen weitergegeben werden. Das Kennwort wird beim Export nicht gespeichert. Ohne Kennwort lässt sich ein heruntergeladenes Backup nicht wiederherstellen.
|
||||||
|
|
||||||
|
## Inhalt
|
||||||
|
|
||||||
|
- Alle Deck-Modellprofile mit Sampling, Kontext, Slots, GPU-Verteilung, KV-Cache, MTP, Vision-Projektor, Komponenten und Prompt-Aufwertern.
|
||||||
|
- API-Port, freigegebene Profile, Listener-Autostart, aktive Video-Modellauswahl und TTS-/STT-Residency-Einstellungen.
|
||||||
|
- Modellinventar mit festen Hugging-Face-Commits, Dateinamen, Größe und SHA-256. Laufzeitrezepte und vorhandene llama.cpp-Buildversionen, inklusive aktivem Build.
|
||||||
|
- Oberflächenkennwort als Prüfwert, API-Token als Prüfwert, Hugging-Face-Token und Deck-eigener ComfyUI-Diensttoken. API-Token im Klartext kann Deck nicht rekonstruieren: der bereits im Client konfigurierte Token bleibt nach Wiederherstellung gültig.
|
||||||
|
- Deck-eigene WireGuard-Konfiguration einschließlich Schlüsseln, gewünschter Verbindung und Zugriffsmodus, wenn das Deck-Netzwerkmodul angebunden ist.
|
||||||
|
- Ausdrücklich mit `io.athena-deck.managed=true` und `io.athena-deck.role=application` markierte Container: Image-/Registryquelle, Labels, Startparameter, Umgebungsvariablen, Benutzer, Ressourcenlimits, Restart-Regel, Ports, Mount-Zuordnung und Startzustand. Eingebundene Secret-Dateien und Swarm `Settings.fds`/`Backends.fds` werden mitgesichert. Swarm erhält das im Deck-Quellstand gepflegte Build-Rezept.
|
||||||
|
- Aktuelles Browser-Theme. Optional: Dashboard-History als konsistenter SQLite-Snapshot, maximal 40 MiB.
|
||||||
|
|
||||||
|
Nicht enthalten: Modellgewichte, Python-/CUDA-/ComfyUI-Umgebungen, Docker-Layer, Chats, Prompt-/Antwortinhalte, Anwendungslogs, Medien, generierte Ausgaben, Trainingsdaten, Docker-Registry-Login und beliebige Verzeichnisinhalte anderer Anwendungen. Volumes werden deshalb nicht pauschal archiviert: für weitere Anwendungs-Konfigurationsdateien werden gepflegte, ausdrückliche Rezepte benötigt. Solche Daten und lokal gebaute Images ohne veröffentlichtes Image oder Deck-Build-Rezept separat sichern.
|
||||||
|
|
||||||
|
RAM-/VRAM-Inhalt und gerade laufende Aufträge werden nicht exportiert. Profile, Auswahl und Vorladeverhalten werden wiederhergestellt; ein GPU-Prozess wird nicht allein deshalb gestartet, weil er beim Export lief. Die normale Deck-Steuerung lädt bei Anfrage bzw. nach der gespeicherten Audio-Policy.
|
||||||
|
|
||||||
|
## Wiederherstellung auf einem frisch installierten Deck
|
||||||
|
|
||||||
|
1. Deck mit seinen normalen Voraussetzungen installieren und einen vorläufigen Administratorzugang einrichten. Der bisherige Docker-Testinstaller setzt Docker voraus; native Debian-Basisinstallation und NVIDIA-Treiberinstallation bleiben eigenständige Voraussetzungen.
|
||||||
|
2. Backup auswählen, Kennwort eingeben und **Backup prüfen** klicken. Die Prüfung verändert noch keine Einstellungen. Sie zeigt Profile, vorhandene/fehlende Dateien, erforderlichen Platz, Laufzeitrezepte, Dienste und Hindernisse.
|
||||||
|
3. Zusatzdienste auswählen. Zum Beispiel Swarm abwählen, wenn es nicht mehr gebraucht wird. Entscheiden, ob das Oberflächenkennwort und der API-Zugang aus dem Backup zurückkommen oder der neu eingerichtete Zugang erhalten bleiben soll.
|
||||||
|
4. Plan bestätigen und **Wiederherstellung starten**. Deck sichert vorher den aktuellen Konfigurationsstand, beendet nur eigene Worker und sperrt Verwaltungsänderungen während des Restore.
|
||||||
|
5. Vorhandene Gewichte werden per SHA-256 geprüft. Fehlende Dateien werden aus der gespeicherten Quellversion geladen und mit der gespeicherten SHA-256 verglichen. Vorhandene Laufzeiten werden verwendet; fehlende bekannte Laufzeiten werden installiert bzw. llama.cpp-Versionen gebaut. Ein abweichendes Bild-/TTS-Runtime-Rezept wird vorab als Hindernis gemeldet.
|
||||||
|
6. Gewählte Dienste werden aus ihrer Registry geladen oder aus dem gepflegten Swarm-Rezept gebaut. Neue externe Mounts werden in das eigene Dienstverzeichnis des Systemhelfers verlegt; lesende Deck-Modellvolumes werden mit der aktuellen Installation verbunden. Fremde/existierende Dienste werden weder ersetzt noch gestoppt. Vorhandene markierte Container mit gleichem Image werden unverändert übernommen – deren externe Settings werden während dieses Wiederverwendens nicht überschrieben.
|
||||||
|
7. Profile, Komponenten, Einstellungen und optional Dashboard-History werden angewendet. Die Oberfläche zeigt pro Schritt das Ergebnis, Downloadgeschwindigkeit/ETA sowie einen Abschlussstatus. **„Mit offenen Punkten“**, **„fehlgeschlagen“** und **„unterbrochen“** sind ausdrücklich kein vollständiger Erfolg.
|
||||||
|
|
||||||
|
Docker-GPU-/privilegierte Container, Host-Geräte, Docker-Socket-Mounts, fremde Host-Dateisysteme und beliebige Host-Netzwerkcontainer werden nicht automatisch wiederhergestellt. Neue veröffentlichte Ports sind auf Loopback begrenzt. Bekannte Swarm-/LTX-DeskWEB-Oberflächen dürfen wie bisher Host-Netzwerk verwenden. Modelle und produktive Fremddienste werden dafür nicht beendet. Der Restore verändert weder den vorhandenen Gateway noch Host-Firewall, SSH, Kernel oder NVIDIA-Treiber und startet den Server nicht neu.
|
||||||
|
|
||||||
|
### Lokale Modelldateien
|
||||||
|
|
||||||
|
Bei rein lokalen Dateien ohne gesicherte Downloadquelle ist ein vollständiger Neuaufbau aus einem Konfigurationsbackup unmöglich. Sie werden als solche angezeigt und ein Restore bei fehlender Datei blockiert. Der aus dem alten Router übernommene `mmproj-BF16.gguf` ist eine geprüfte Ausnahme: Größe und SHA-256 stimmen exakt mit `unsloth/Qwen3.8-27B-GGUF`, Commit `4ca720788d1e01f1bff70c033e0d0028fd02e502`, überein. Er lässt sich nachladen; die ursprüngliche Bibliotheks-ID und Profilreferenzen bleiben bestehen.
|
||||||
|
|
||||||
|
### WireGuard und Erreichbarkeit
|
||||||
|
|
||||||
|
Nur das Deck-eigene Netzwerkmodul wird exportiert; der bestehende Athena-WireGuard-Gateway gehört nicht dazu. Die aktuelle isolierte Athena-Installation hat dieses Modul deaktiviert, ihr Export enthält daher keine Gateway-Schlüssel. Ein Backup mit WireGuard verlangt beim Import zunächst ein angebundenes Deck-Netzwerkmodul und bestätigten LAN-Zugang bei deaktiviertem Deck-Tunnel.
|
||||||
|
|
||||||
|
Konfiguration und Verbindung werden dort wiederhergestellt und der Handshake geprüft. Ohne Handshake bleibt LAN erhalten. Für `tunnel`/`both` greift der bestehende befristete Zugriffsversuch: Bestätigung über den Tunnel ist erforderlich, sonst Rückfall. Der Restore meldet diesen offenen Schritt ausdrücklich. Einen ungetesteten reinen Tunnelzugang vollautomatisch zu erzwingen würde den entfernten Server aussperren können.
|
||||||
|
|
||||||
|
## Rückfall, Abbruch und Neustart
|
||||||
|
|
||||||
|
Vor jedem Restore liegt ein verschlüsselter lokaler Rückfallstand unter `state/recovery/before-<id>.adbackup`; dessen lokaler Schlüssel liegt getrennt mit Modus 0600 daneben. **Lokale Rückfallstände → prüfen** bietet denselben Importplan an. Das ist ein lokales Rückfallmittel, kein Ersatz für ein extern gelagertes Backup. Bei fatalen Fehlern versucht Deck, die vorherigen Anwendungseinstellungen wiederherzustellen. Bereits nachgeladene Dateien und neu angelegte Container bleiben erhalten und werden nicht automatisch gelöscht.
|
||||||
|
|
||||||
|
Abbruch stoppt Deck-Downloads und Laufzeit-Builds. Ein bereits laufender Docker-Pull/Build wird zu Ende geführt, bevor der Job abbricht. Nach einem Deck-Neustart wird ein laufender Restore als unterbrochen gemeldet; kein stilles Wiederaufnehmen mit ungesichertem Kennwort. Das Backup erneut importieren: erfolgreich vorhandene Dateien/Builds werden wiederverwendet.
|
||||||
|
|
||||||
|
## API und Format
|
||||||
|
|
||||||
|
Nur Administratorsitzungen; alle POSTs benötigen `X-Athena-Deck: 1` und den bestehenden Same-Origin-Schutz:
|
||||||
|
|
||||||
|
- `GET /api/v1/backup`: Job, geprüfter Plan und lokale Rückfallstände; niemals Backup-Secrets.
|
||||||
|
- `POST /api/v1/backup/export`: JSON `{password, theme, history}` → verschlüsselte Binärdatei.
|
||||||
|
- `POST /api/v1/backup/inspect`: Binärdatei, `Content-Type: application/octet-stream`; Kennwort UTF-8/Base64 im Header `X-Backup-Passphrase` → bereinigter Importplan.
|
||||||
|
- `POST /api/v1/backup/restore`: JSON `{id, services: [Namen], confirm: true, restore_credentials: true|false}` → Hintergrundjob.
|
||||||
|
- `POST /api/v1/backup/cancel`: `{}`.
|
||||||
|
- `POST /api/v1/backup/checkpoint`: `{name}` → Importprüfung eines lokalen Rückfallstands.
|
||||||
|
|
||||||
|
Formatversion 1: JSON-Konfigurationsmanifest, zlib-Kompression, AES-256-GCM mit zufälligem 96-Bit-Nonce und authentisiertem Header; Schlüsselableitung scrypt (N=32768, r=8, p=1) mit zufälligem 128-Bit-Salt. Maximal 64 MiB Konfigurationsinhalt und Upload; entpackte Größen werden begrenzt. Keine ZIP/Tar-Extraktion, kein Pickle, keine importierten Shellskripte. `python3-cryptography` wird in den Deck-Docker-Images installiert; native Installationen benötigen dasselbe Debian-Paket.
|
||||||
|
|
||||||
|
Der root-eigene Docker-Helfer validiert Container unabhängig von der GUI und verwendet feste argv-Aufrufe ohne Shell. Das Deck-Zustandsverzeichnis wird mit `setup_docker_helper.py --deck-state /ABS/PATH/state` registriert. Er darf nur sein eigenes Dienstverzeichnis und dieses registrierte Verzeichnis beschreiben.
|
||||||
|
|
||||||
|
## Prüfung
|
||||||
|
|
||||||
|
Automatische Tests decken Verschlüsselung, Manipulation, Dekompressionsgrenzen, ungültige Pfade/Referenzen, fehlende Downloadquellen, SHA-256, vorhandene/neu geladene Dateien, Bestandsschutz und Rückfall ab. Auf Athena erfolgen zusätzlich ein echter Konfigurationsexport und ein kleiner API-Restore mit echtem HF-Download in einem separaten Testbestand. Kein vollständiger Desaster-Restore des laufenden Athena-Servers und keine echte Änderung am produktiven WireGuard-Gateway.
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
# Athena Deck · Router 0.7
|
# Athena Deck · Router 0.7
|
||||||
|
|
||||||
Eigenständige neue Oberfläche, Python-Standardbibliothek und HTML/CSS/JavaScript.
|
Eigenständige Oberfläche mit Python und HTML/CSS/JavaScript. Python >= 3.10;
|
||||||
Keine Installation von Python-Paketen oder Frontend-Builds nötig. Python >= 3.10.
|
verschlüsselte Backups benötigen `python3-cryptography` (im Docker-Installer enthalten). Kein Frontend-Build nötig.
|
||||||
|
|
||||||
Die Navigation beginnt mit dem [Dashboard](DASHBOARD.md). Es zeigt die
|
Die Navigation beginnt mit dem [Dashboard](DASHBOARD.md). Es zeigt die
|
||||||
Live-Telemetrie und die aus dem alten Athena-Dashboard übernommene History;
|
Live-Telemetrie und die aus dem alten Athena-Dashboard übernommene History;
|
||||||
@@ -190,3 +190,5 @@ Interne Verwaltungs-API: `GET /api/v1/video` meldet Modelle, Komponenten, Blocke
|
|||||||
Für reine App-Updates kann der Debian-Installer mit `--update --reuse-runtime` die installierten Laufzeiten aus dem bisherigen Deck-Image übernehmen. Diese Option ist für App-Änderungen gedacht; gewünschte Runtime-Updates werden weiterhin ausdrücklich gebaut. Regenerierbare Video-Laufzeiten und ComfyUI-Arbeitsdaten werden bei App-Updates nicht mehr in jedes Zustandsbackup kopiert.
|
Für reine App-Updates kann der Debian-Installer mit `--update --reuse-runtime` die installierten Laufzeiten aus dem bisherigen Deck-Image übernehmen. Diese Option ist für App-Änderungen gedacht; gewünschte Runtime-Updates werden weiterhin ausdrücklich gebaut. Regenerierbare Video-Laufzeiten und ComfyUI-Arbeitsdaten werden bei App-Updates nicht mehr in jedes Zustandsbackup kopiert.
|
||||||
|
|
||||||
Optionales SwarmUI mit vorhandener ComfyUI-Laufzeit: [Installation und Betrieb](deploy/swarm-ui/README.md).
|
Optionales SwarmUI mit vorhandener ComfyUI-Laufzeit: [Installation und Betrieb](deploy/swarm-ui/README.md).
|
||||||
|
|
||||||
|
Backup und Wiederherstellung unter **Einstellungen → Backup & Restore**: [Umfang, Bedienung, API und Grenzen](BACKUP.md).
|
||||||
|
|||||||
@@ -26,6 +26,7 @@ function appearancePage(){
|
|||||||
async function refresh(){if(refreshing)return;refreshing=true;const page=location.hash.slice(1)||'dashboard';document.querySelectorAll('nav a').forEach(a=>a.classList.toggle('active',a.hash==='#'+page));try{error.textContent='';let html;
|
async function refresh(){if(refreshing)return;refreshing=true;const page=location.hash.slice(1)||'dashboard';document.querySelectorAll('nav a').forEach(a=>a.classList.toggle('active',a.hash==='#'+page));try{error.textContent='';let html;
|
||||||
if(['video','video-runtime'].includes(page)){VideoUI.runtime();return;}
|
if(['video','video-runtime'].includes(page)){VideoUI.runtime();return;}
|
||||||
if(['chat','image','audio','stt','music','voice','video','runtime','runtimes','image-runtime','stt-runtime','tts-runtime','docker-runtime','services'].includes(page)){Studio.render(page);return;}
|
if(['chat','image','audio','stt','music','voice','video','runtime','runtimes','image-runtime','stt-runtime','tts-runtime','docker-runtime','services'].includes(page)){Studio.render(page);return;}
|
||||||
|
if(page==='backup'){BackupUI.render();return;}
|
||||||
if(page==='access'){await accessPage();return;}
|
if(page==='access'){await accessPage();return;}
|
||||||
if(page==='appearance'){appearancePage();return;}
|
if(page==='appearance'){appearancePage();return;}
|
||||||
if(page==='network'){await networkPage();return;}
|
if(page==='network'){await networkPage();return;}
|
||||||
|
|||||||
+1
-1
@@ -43,7 +43,7 @@ class AudioPolicy:
|
|||||||
while True:
|
while True:
|
||||||
time.sleep(8)
|
time.sleep(8)
|
||||||
with self.lock:settings={k:dict(v) for k,v in self.settings.items()}
|
with self.lock:settings={k:dict(v) for k,v in self.settings.items()}
|
||||||
if self.endpoint.state!='running':continue
|
if self.endpoint.state!='running' or self.scheduler.gpu_mode=='restoring':continue
|
||||||
for kind,worker in (('stt',self.stt),('tts',self.tts)):
|
for kind,worker in (('stt',self.stt),('tts',self.tts)):
|
||||||
item=settings[kind]
|
item=settings[kind]
|
||||||
if item['mode']!='warm':continue
|
if item['mode']!='warm':continue
|
||||||
|
|||||||
@@ -0,0 +1,17 @@
|
|||||||
|
window.BackupUI=(()=>{
|
||||||
|
const esc=v=>String(v??'').replace(/[&<>"']/g,c=>({'&':'&','<':'<','>':'>','"':'"',"'":'''}[c]));
|
||||||
|
const size=v=>(v/1024**3).toLocaleString('de-DE',{maximumFractionDigits:2})+' GiB';let pending=false,preview=null,jobId=null;
|
||||||
|
async function request(action,body){const r=await fetch('/api/v1/backup/'+action,{method:'POST',headers:{'Content-Type':'application/json','X-Athena-Deck':'1'},body:JSON.stringify(body)});if(!r.ok){const d=await r.json();throw Error(d.error||'Backup-Anfrage fehlgeschlagen.')}return r;}
|
||||||
|
function render(){const view=document.querySelector('#view');if(view.querySelector('#backup-page'))return;preview=null;
|
||||||
|
view.innerHTML=`<div id="backup-page"><div class="kicker">EINSTELLUNGEN / WIEDERHERSTELLUNG</div><h1>Backup & Restore</h1><p>Einstellungen, Profile, Modellquellen, Laufzeitrezepte und zugeordnete Zusatzdienste. Gewichte und Container-Images werden beim Restore erneut geladen; Chats, Medien und Logs sind nicht enthalten.</p><section class="card"><h2>Verschlüsseltes Backup exportieren</h2><p>Enthält Zugangsdaten und gegebenenfalls WireGuard-Schlüssel. Bewahre Datei und Backup-Kennwort sicher auf. Das Kennwort wird nicht gespeichert.</p><form id="backup-export"><label>Backup-Kennwort (mindestens 16 Zeichen)<input type="password" id="backup-export-password" minlength="16" maxlength="256" autocomplete="new-password" required></label><label>Kennwort wiederholen<input type="password" id="backup-export-repeat" minlength="16" maxlength="256" autocomplete="new-password" required></label><label><input type="checkbox" id="backup-history" checked> Dashboard-History mit sichern</label><p><button type="submit">Backup herunterladen</button></p></form></section><section class="card"><h2>Backup importieren</h2><p>Erst prüfen, dann wiederherstellen. Der Restore beendet Deck-eigene Aufträge. Vorhandene Fremddienste und Container werden nicht ersetzt. Währenddessen bleiben Fortschritt und Abbruch verfügbar.</p><form id="backup-import"><label>Backup-Datei<input type="file" id="backup-file" accept=".adbackup" required></label><label>Backup-Kennwort<input type="password" id="backup-import-password" minlength="16" maxlength="256" autocomplete="off" required></label><p><button type="submit">Backup prüfen</button></p></form><div id="backup-preview"></div></section><section class="card" id="backup-progress" aria-live="polite"></section><section class="card" id="backup-checkpoints"></section><p id="backup-message" role="status"></p></div>`;
|
||||||
|
const message=view.querySelector('#backup-message'),panel=view.querySelector('#backup-preview');
|
||||||
|
view.querySelector('#backup-export').onsubmit=async e=>{e.preventDefault();if(pending)return;const p=view.querySelector('#backup-export-password').value;if(p!==view.querySelector('#backup-export-repeat').value){message.textContent='Backup-Kennwörter stimmen nicht überein.';return;}pending=true;message.textContent='Konfigurationen werden gesammelt und verschlüsselt …';try{const r=await request('export',{password:p,theme:document.documentElement.dataset.theme||'deck',history:view.querySelector('#backup-history').checked});const blob=await r.blob(),url=URL.createObjectURL(blob),a=document.createElement('a');a.href=url;a.download=(r.headers.get('Content-Disposition')||'').match(/filename="([^"]+)"/)?.[1]||'athena-deck.adbackup';a.click();setTimeout(()=>URL.revokeObjectURL(url),1000);message.textContent='Verschlüsseltes Backup heruntergeladen.';view.querySelector('#backup-export-password').value='';view.querySelector('#backup-export-repeat').value='';}catch(e){message.textContent=e.message;}finally{pending=false;}};
|
||||||
|
view.querySelector('#backup-import').onsubmit=async e=>{e.preventDefault();if(pending)return;pending=true;message.textContent='Backup entschlüsseln und Quellen prüfen …';try{const file=view.querySelector('#backup-file').files[0];if(!file||file.size>64*1024**2)throw Error('Backup-Datei fehlt oder ist größer als 64 MiB.');const r=await fetch('/api/v1/backup/inspect',{method:'POST',headers:{'Content-Type':'application/octet-stream','X-Athena-Deck':'1','X-Backup-Passphrase':btoa(String.fromCharCode(...new TextEncoder().encode(view.querySelector('#backup-import-password').value)))},body:file});const data=await r.json();if(!r.ok)throw Error(data.error);preview=data;view.querySelector('#backup-import-password').value='';showPreview(panel,data,message);message.textContent='Backup geprüft. Details und Zusatzdienste auswählen.';}catch(e){message.textContent=e.message;}finally{pending=false;}};
|
||||||
|
poll(view);
|
||||||
|
}
|
||||||
|
function showPreview(panel,p,message){panel.innerHTML=`<h3>Wiederherstellungsplan</h3><p>${p.profiles.length} Profile · ${p.models.length} Modelldateien · ${size(p.download_bytes)} fehlende Gewichte · ${p.wireguard?'WireGuard enthalten':'Kein Deck-WireGuard'}</p><details><summary>Modelle und Profile anzeigen</summary><ul>${p.models.map(m=>`<li>${esc(m.name)} · ${size(m.bytes)} · ${m.present?'vorhanden, wird geprüft':'wird heruntergeladen'}</li>`).join('')}</ul><p>${p.profiles.map(esc).join(' · ')}</p></details><h3>Zusatzdienste</h3>${p.services.length?p.services.map(c=>`<label><input type="checkbox" class="backup-service" value="${esc(c.name)}" ${c.restorable?'checked':'disabled'}> ${esc(c.name)} · ${esc(c.registry||c.image)} ${c.build_recipe?'· wird aus festem Rezept gebaut':''}${!c.restorable?'· keine Wiederherstellungsquelle':''}</label>`).join(''):'<p>Keine zugeordneten Container.</p>'}<h3>Laufzeiten</h3><p>llama.cpp: ${esc(p.runtimes.llama?.commit||'nicht eingerichtet')} · ComfyUI: ${p.runtimes.image?'vorhandene wiederverwenden / fehlende installieren':'nicht benötigt'} · TTS: ${p.runtimes.tts?'wiederherstellen':'nicht benötigt'}</p>${p.warnings.map(w=>`<p class="note">${esc(w)}</p>`).join('')}${p.blockers.map(w=>`<p role="alert">${esc(w)}</p>`).join('')}<label><input type="checkbox" id="backup-credentials" checked> Oberflächenkennwort und API-Zugang aus Backup wiederherstellen</label><p class="small">Danach gilt das bisherige Kennwort des Backups. Der aktuelle Zugang bleibt erhalten, wenn der Haken entfernt wird.</p><label><input type="checkbox" id="backup-confirm"> Einstellungen ersetzen und ausgewählte Downloads/Builds automatisch ausführen. Lokaler Rückfallstand wird vorher gesichert.</label><p><button id="backup-restore" ${p.blockers.length?'disabled':''}>Wiederherstellung starten</button></p>`;
|
||||||
|
panel.querySelector('#backup-restore').onclick=async()=>{if(pending)return;if(!panel.querySelector('#backup-confirm').checked){message.textContent='Bitte den angezeigten Wiederherstellungsplan bestätigen.';return;}pending=true;try{await request('restore',{id:p.id,services:[...panel.querySelectorAll('.backup-service:checked')].map(x=>x.value),confirm:true,restore_credentials:panel.querySelector('#backup-credentials').checked});panel.innerHTML='';preview=null;message.textContent='Wiederherstellung gestartet.';}catch(e){message.textContent=e.message;}finally{pending=false;}};
|
||||||
|
}
|
||||||
|
async function poll(view){if(!view.querySelector('#backup-page'))return;const panel=view.querySelector('#backup-progress');try{const r=await fetch('/api/v1/backup');if(!r.ok)throw Error('Restore-Status nicht erreichbar.');const {job,preview:p,checkpoints=[]}=await r.json();const saved=view.querySelector('#backup-checkpoints');saved.innerHTML='<h2>Lokale Rückfallstände</h2>'+checkpoints.map(name=>`<p><button data-checkpoint="${esc(name)}">${esc(name)} prüfen</button></p>`).join('');saved.querySelectorAll('[data-checkpoint]').forEach(b=>b.onclick=async()=>{try{const r=await request('checkpoint',{name:b.dataset.checkpoint});const p=await r.json();preview=p;showPreview(view.querySelector('#backup-preview'),p,view.querySelector('#backup-message'));}catch(e){view.querySelector('#backup-message').textContent=e.message;}});if(p&&!preview){preview=p;showPreview(view.querySelector('#backup-preview'),p,view.querySelector('#backup-message'));}if(!job){panel.innerHTML='<h2>Wiederherstellung</h2><p>Noch kein Restore gestartet.</p>';}else{const running=job.state==='running';panel.innerHTML=`<h2>${job.state==='complete'?'Alles wiederhergestellt':job.state==='partial'?'Wiederherstellung mit offenen Punkten':'Wiederherstellung · '+esc(job.state)}</h2><p>${esc(job.phase)}</p><progress max="${job.total}" value="${job.completed}"></progress><p>${job.completed}/${job.total} Schritte</p>${job.current?.total?`<progress max="${job.current.total}" value="${job.current.bytes||0}"></progress><p>${size(job.current.bytes||0)} / ${size(job.current.total)}${job.current.bytes_per_second?' · '+(job.current.bytes_per_second/1024**2).toFixed(1)+' MiB/s':''}${job.current.eta_seconds!=null?' · noch etwa '+Math.ceil(job.current.eta_seconds/60)+' Minuten':''}</p>`:''}${running?'<button id="backup-cancel">Wiederherstellung abbrechen</button>':''}<ul>${job.items.map(x=>`<li><strong>${esc(x.name)}</strong> · ${esc(x.state)}<br>${esc(x.message)}</li>`).join('')}</ul>`;if(running)panel.querySelector('#backup-cancel').onclick=async()=>{try{await request('cancel',{});view.querySelector('#backup-message').textContent='Abbruch angefordert; laufender Container-Schritt wird sicher beendet.';}catch(e){view.querySelector('#backup-message').textContent=e.message;}};if(job.state==='complete'&&jobId!==job.id){jobId=job.id;applyDeckTheme(job.theme);}}}catch(e){view.querySelector('#backup-message').textContent=e.message;}setTimeout(()=>poll(view),1500);}
|
||||||
|
return {render};
|
||||||
|
})();
|
||||||
@@ -0,0 +1,353 @@
|
|||||||
|
"""Encrypted configuration export and sequential, inspectable recovery jobs.
|
||||||
|
No chats, prompts, media, logs, model weights or container layers are read into an export.
|
||||||
|
"""
|
||||||
|
import base64,copy,hashlib,json,os,re,secrets,shutil,sqlite3,tempfile,threading,time,zlib
|
||||||
|
from pathlib import Path,PurePosixPath
|
||||||
|
from auth import atomic_write,validate_record
|
||||||
|
from backup_codec import seal,open_backup
|
||||||
|
from catalog import repo_id,file_role,KINDS
|
||||||
|
from profiles import SCHEMAS,CHAT_GPU_DEFAULTS,chat_parameters
|
||||||
|
SETTINGS=('profiles.json','endpoint.json','audio-policy.json','video/comfy-selection.json','models/huggingface.json','video/comfy-client-token')
|
||||||
|
MAX_HISTORY=40*1024**2
|
||||||
|
VERSION=1
|
||||||
|
# Exact byte hash verified against official Hub metadata, not a filename heuristic.
|
||||||
|
LOCAL_SOURCES={'83ee4f4f205fa514161778c41df1ea14144faa0f713510893b63c2395f5c2d53':dict(repo='unsloth/Qwen3.8-27B-GGUF',revision='4ca720788d1e01f1bff70c033e0d0028fd02e502',file='mmproj-BF16.gguf')}
|
||||||
|
|
||||||
|
def validate(doc):
|
||||||
|
if not isinstance(doc,dict) or doc.get('format')!='athena-deck' or doc.get('version')!=VERSION:raise ValueError('Backup-Version nicht unterstützt.')
|
||||||
|
if set(doc)-{'format','version','created_at','settings','models','credentials','theme','network','services','runtimes','warnings','history','runtime_versions'}:raise ValueError('Unbekannte Backup-Felder.')
|
||||||
|
if not isinstance(doc.get('warnings',[]),list) or len(doc.get('warnings',[]))>100 or any(not isinstance(w,str) or len(w)>1024 for w in doc.get('warnings',[])):raise ValueError('Ungültige Backup-Hinweise.')
|
||||||
|
if 'history' in doc and (not isinstance(doc['history'],str) or len(doc['history'])>56*1024**2):raise ValueError('History im Backup zu groß.')
|
||||||
|
models=doc.get('models');profiles=doc.get('settings',{}).get('profiles.json',[])
|
||||||
|
if not isinstance(models,list) or len(models)>4096 or not isinstance(profiles,list) or len(profiles)>1000:raise ValueError('Ungültiges Modell-/Profilverzeichnis.')
|
||||||
|
ids=set()
|
||||||
|
for m in models:
|
||||||
|
repo_id(m.get('repo'));f=m.get('file','');parts=PurePosixPath(f).parts
|
||||||
|
if not isinstance(f,str) or not parts or f.startswith('/') or '..' in parts or '\\' in f or len(f)>1024 or any(ord(c)<32 for c in f):raise ValueError('Unsicherer Modell-Dateiname.')
|
||||||
|
local=m.get('source')=='local'
|
||||||
|
if not re.fullmatch('[a-f0-9]{64}' if local else '[a-f0-9]{40}',m.get('revision','')) or m.get('kind') not in KINDS or type(m.get('size')) is not int or not 0<m['size']<4*1024**4:raise ValueError('Ungültige Modellquelle.')
|
||||||
|
ident=hashlib.sha256((m['repo']+m['revision']+f).encode()).hexdigest()
|
||||||
|
if (not local and m.get('id')!=ident) or not re.fullmatch('[a-f0-9]{64}',m.get('id','')) or m['id'] in ids or not re.fullmatch('[a-f0-9]{64}',m.get('sha256','')):raise ValueError('Ungültige Modell-ID oder Prüfsumme.')
|
||||||
|
if local:
|
||||||
|
if not m['repo'].startswith('local/') or m['revision']!=m['sha256']:raise ValueError('Ungültige lokale Modellherkunft.')
|
||||||
|
source=m.get('download_source')
|
||||||
|
if source is not None:
|
||||||
|
if not isinstance(source,dict) or set(source)!={'repo','revision','file'}:raise ValueError('Ungültige Ersatzquelle.')
|
||||||
|
repo_id(source['repo'])
|
||||||
|
if not re.fullmatch('[a-f0-9]{40}',source['revision']) or source['file'].startswith('/') or '..' in PurePosixPath(source['file']).parts or '\\' in source['file'] or any(ord(c)<32 for c in source['file']):raise ValueError('Ungültige Ersatzquelle.')
|
||||||
|
ids.add(m['id'])
|
||||||
|
pids=set();names=set()
|
||||||
|
for p in profiles:
|
||||||
|
if not isinstance(p,dict) or not re.fullmatch('[a-f0-9]{32}',p.get('id','')) or p['id'] in pids or p.get('name') in names:raise ValueError('Ungültige oder doppelte Profil-ID.')
|
||||||
|
name=p.get('name','');kind=p.get('kind');params=p.get('parameters')
|
||||||
|
if not isinstance(name,str) or not 2<=len(name)<=64 or not re.fullmatch(r'[\w .-]+',name) or not name[0].isalnum() or not name[-1].isalnum() or kind not in SCHEMAS or p.get('model_id') not in ids:raise ValueError('Ungültiges Profil oder Modellreferenz.')
|
||||||
|
if type(p.get('revision')) is not int or not 1<=p['revision']<=1000000:raise ValueError('Ungültige Profilrevision.')
|
||||||
|
if not isinstance(params,dict):raise ValueError('Ungültige Profilparameter.')
|
||||||
|
params=chat_parameters(params) if kind=='chat' else params
|
||||||
|
if set(params)!=(set(SCHEMAS[kind])|(set(CHAT_GPU_DEFAULTS) if kind=='chat' else set(params)&{'video_device','text_encoder_device'} if kind=='video' else set())):raise ValueError('Unbekannte Profilparameter.')
|
||||||
|
for key,(lo,hi,_) in SCHEMAS[kind].items():
|
||||||
|
value=params[key]
|
||||||
|
if isinstance(value,bool) or not isinstance(value,(int,float)) or not lo<=value<=hi or key not in ('temperature','top_p','guidance','speed','mtp_min_p') and type(value) is not int:raise ValueError('Ungültiger Profilparameter: '+key)
|
||||||
|
if kind=='chat' and (params['ubatch']>params['batch'] or params.get('vision_projector') and params['vision_projector'] not in ids):raise ValueError('Ungültige Chat-Zuordnung.')
|
||||||
|
if kind in ('image','video') and (params['width']%64 or params['height']%64):raise ValueError('Ungültige Bildgröße.')
|
||||||
|
components=p.get('components',{})
|
||||||
|
if not isinstance(components,dict) or any(k not in ('text_encoder','vae','video_vae','audio_vae','spatial_upsampler') or v not in ids for k,v in components.items()):raise ValueError('Ungültige Komponentenreferenz.')
|
||||||
|
pids.add(p['id']);names.add(name)
|
||||||
|
settings=doc.get('settings')
|
||||||
|
if not isinstance(settings,dict) or set(settings)-set(SETTINGS):raise ValueError('Unbekannte Einstellungsdateien.')
|
||||||
|
ep=settings.get('endpoint.json',{})
|
||||||
|
if not isinstance(ep,dict) or set(ep)-{'port','enabled_profiles','autostart'} or type(ep.get('port',8120)) is not int or not 1024<=ep.get('port',8120)<=65535 or type(ep.get('autostart',False)) is not bool:raise ValueError('Ungültige Endpunkt-Einstellungen.')
|
||||||
|
enabled=ep.get('enabled_profiles',[])
|
||||||
|
if not isinstance(enabled,list) or any(p not in pids for p in enabled) or len(set(enabled))!=len(enabled):raise ValueError('Unbekannte Profilfreigabe.')
|
||||||
|
if sum(p['kind']=='image' and p['id'] in enabled for p in profiles)>1:raise ValueError('Nur ein Bildprofil darf freigegeben sein.')
|
||||||
|
policy=settings.get('audio-policy.json',{})
|
||||||
|
if not isinstance(policy,dict) or set(policy)-{'tts','stt'} or any(not isinstance(v,dict) or set(v)!={'mode','profile_id'} or v['mode'] not in ('auto','warm','per_request') or v['profile_id'] is not None and v['profile_id'] not in pids for v in policy.values()):raise ValueError('Ungültige Audio-Einstellung.')
|
||||||
|
selection=settings.get('video/comfy-selection.json',{})
|
||||||
|
if not isinstance(selection,dict) or set(selection)-{'model_id'} or selection.get('model_id') and selection['model_id'] not in ids:raise ValueError('Ungültige Video-Auswahl.')
|
||||||
|
if 'models/huggingface.json' in settings:
|
||||||
|
hf=settings['models/huggingface.json']
|
||||||
|
if not isinstance(hf,dict) or set(hf)!={'token'} or hf['token'] is not None and (not isinstance(hf['token'],str) or not re.fullmatch(r'hf_[A-Za-z0-9]{10,252}',hf['token'])):raise ValueError('Ungültiger Hugging-Face-Zugang.')
|
||||||
|
if 'video/comfy-client-token' in settings and not re.fullmatch(r'[A-Za-z0-9_-]{32,256}',settings['video/comfy-client-token'].strip()):raise ValueError('Ungültiger Diensttoken.')
|
||||||
|
validate_record(doc.get('credentials'))
|
||||||
|
runtimes=doc.get('runtimes')
|
||||||
|
if not isinstance(runtimes,dict) or set(runtimes)!={'llama','llama_builds','image','tts','enhancers','swarm_nodes'}:raise ValueError('Ungültiges Laufzeitrezept.')
|
||||||
|
if not isinstance(runtimes['llama_builds'],list) or len(runtimes['llama_builds'])>50 or any(not isinstance(b,dict) or set(b)!={'commit','backend'} or not re.fullmatch('[a-f0-9]{40}',b.get('commit','')) or b.get('backend') not in ('CUDA','CPU') for b in runtimes['llama_builds']):raise ValueError('Ungültige Buildrezepte.')
|
||||||
|
if runtimes['llama'] is not None:
|
||||||
|
item=runtimes['llama']
|
||||||
|
if not isinstance(item,dict) or not re.fullmatch('[a-f0-9]{40}',item.get('commit','')) or item.get('backend') not in ('CUDA','CPU'):raise ValueError('Ungültige llama.cpp-Quelle.')
|
||||||
|
if any(type(runtimes[k]) is not bool for k in ('image','tts','swarm_nodes')) or not isinstance(runtimes['enhancers'],dict) or set(runtimes['enhancers'])-{'t2i','i2i'}:raise ValueError('Ungültiges Laufzeitrezept.')
|
||||||
|
for revision in runtimes['enhancers'].values():
|
||||||
|
if not isinstance(revision,str) or not re.fullmatch('[a-f0-9]{40}',revision):raise ValueError('Ungültige Aufwerterversion.')
|
||||||
|
network=doc.get('network')
|
||||||
|
if network is not None:
|
||||||
|
if not isinstance(network,dict) or set(network)!={'config','enabled','mode'} or type(network['enabled']) is not bool or network['mode'] not in ('lan','both','tunnel'):raise ValueError('Ungültiger Netzwerkstand.')
|
||||||
|
if network['config'] is not None:
|
||||||
|
from network.config import parse_config
|
||||||
|
parse_config(network['config'])
|
||||||
|
services=doc.get('services')
|
||||||
|
if not isinstance(services,list) or len(services)>50:raise ValueError('Ungültiger Containerbestand.')
|
||||||
|
# Validation is shared with the root helper, which independently revalidates before Docker calls.
|
||||||
|
from deploy.docker_backup import validate as validate_container
|
||||||
|
seen=set()
|
||||||
|
for c in services:
|
||||||
|
validate_container(c)
|
||||||
|
if c['name'] in seen:raise ValueError('Doppelte Containerdefinition.')
|
||||||
|
seen.add(c['name'])
|
||||||
|
if doc.get('theme','deck') not in ('deck','athena-classic'):raise ValueError('Unbekanntes Theme.')
|
||||||
|
return doc
|
||||||
|
|
||||||
|
class Backup:
|
||||||
|
def __init__(self,server):
|
||||||
|
self.server=server;self.root=server.catalog.root.parent;self.directory=self.root/'recovery';self.directory.mkdir(parents=True,exist_ok=True,mode=0o700)
|
||||||
|
self.lock=threading.RLock();self.cancel=threading.Event();self.plan=None;self.job=None
|
||||||
|
path=self.directory/'job.json'
|
||||||
|
if path.exists():
|
||||||
|
self.job=json.loads(path.read_text())
|
||||||
|
if self.job.get('state')=='running':self.job.update(state='interrupted',phase='Restore durch Neustart unterbrochen. Backup erneut importieren; vorhandene Dateien werden geprüft und wiederverwendet.');self._save()
|
||||||
|
def busy(self):return bool(self.job and self.job['state']=='running')
|
||||||
|
def _save(self):atomic_write(self.directory/'job.json',self.job)
|
||||||
|
def status(self):
|
||||||
|
with self.lock:return {'job':copy.deepcopy(self.job),'preview':copy.deepcopy(self.plan['summary']) if self.plan else None,'checkpoints':[p.name for p in sorted(self.directory.glob('before-*.adbackup'),reverse=True)]}
|
||||||
|
def snapshot(self,theme='deck',include_history=True,_during_restore=False):
|
||||||
|
s=self.server
|
||||||
|
if self.busy() and not _during_restore:raise ValueError('Restore läuft. Export erst danach.')
|
||||||
|
settings={}
|
||||||
|
# Explicit allowlist: no recursive backup of process work directories.
|
||||||
|
with s.profiles.lock,s.endpoint.lock,s.catalog.lock:
|
||||||
|
for name in SETTINGS:
|
||||||
|
p=self.root/name
|
||||||
|
if p.is_file():settings[name]=p.read_text().strip() if name.endswith('comfy-client-token') else json.loads(p.read_text())
|
||||||
|
models=[]
|
||||||
|
for m in s.catalog.status()['entries']:
|
||||||
|
item={k:m[k] for k in ('id','repo','revision','file','size','sha256','kind')}
|
||||||
|
if m['repo'].startswith('local/'):
|
||||||
|
item['source']='local';item['download_source']=m.get('download_source') or LOCAL_SOURCES.get(m['sha256'])
|
||||||
|
models.append(item)
|
||||||
|
state=s.runtime.status();active=next((b for b in state['builds'] if b['id']==state['active']),None)
|
||||||
|
llama={'commit':active['commit'],'backend':active['backend']} if active else None
|
||||||
|
warnings=[];services=[]
|
||||||
|
try:services=s.docker.call('backup-export')['services']
|
||||||
|
except (OSError,ValueError):
|
||||||
|
if s.docker.status().get('services'):raise ValueError('Container-Konfigurationen können ohne aktualisierten Systemhelfer nicht gesichert werden.')
|
||||||
|
warnings.append('Docker-Systemhelfer nicht verfügbar; keine Zusatzdienste gesichert.')
|
||||||
|
network=None
|
||||||
|
if not s.network.disabled:
|
||||||
|
network=s.network.call('backup-export')
|
||||||
|
else:warnings.append('Deck-Netzwerkmodul deaktiviert; fremder WireGuard-Gateway wird nicht gesichert.')
|
||||||
|
document=dict(format='athena-deck',version=VERSION,created_at=time.time(),settings=settings,models=models,credentials=s.credentials.read(),theme=theme,network=network,services=services,runtime_versions=dict(image=dict(comfy=s.image_runtime.status()['comfy_revision'],gguf=s.image_runtime.status()['gguf_revision']),tts=s.tts_runtime.status()['revision']),runtimes=dict(llama=llama,llama_builds=[dict(commit=b['commit'],backend=b['backend']) for b in state['builds']],image=s.image_runtime.status()['installed'],tts=s.tts_runtime.status()['installed'],enhancers={task:data['revision'] for task in ('t2i','i2i') if (data:=s.prompt_enhancer.installed(task))},swarm_nodes=(self.root/'video/swarm-comfy-nodes').is_dir()),warnings=warnings)
|
||||||
|
if include_history:
|
||||||
|
with tempfile.TemporaryDirectory(dir=self.directory) as work:
|
||||||
|
out=Path(work)/'history.sqlite3'
|
||||||
|
with s.dashboard.history._lock,sqlite3.connect(out) as dest:s.dashboard.history._db.backup(dest)
|
||||||
|
if out.stat().st_size>MAX_HISTORY:raise ValueError('Dashboard-History größer als 40 MiB. History separat sichern oder Export ohne History wählen.')
|
||||||
|
document['history']=base64.b64encode(out.read_bytes()).decode()
|
||||||
|
return document
|
||||||
|
def export(self,phrase,theme='deck',history=True):return seal(validate(self.snapshot(theme,history)),phrase)
|
||||||
|
def inspect(self,raw,phrase):
|
||||||
|
doc=validate(open_backup(raw,phrase));s=self.server;blockers=[];warnings=list(doc.get('warnings',[]));existing={m['id']:m for m in s.catalog.status()['entries']}
|
||||||
|
if 'history' in doc:self._restore_history(doc['history'],validate_only=True)
|
||||||
|
missing=[m for m in doc['models'] if m['id'] not in existing];need=sum(m['size'] for m in missing)
|
||||||
|
if missing and need+10*1024**3>shutil.disk_usage(self.root).free:blockers.append('Zu wenig Platz für fehlende Gewichte einschließlich 10 GiB Reserve.')
|
||||||
|
for m in doc['models']:
|
||||||
|
if m.get('source')=='local' and not m.get('download_source'):
|
||||||
|
warnings.append(m['file']+': lokale Datei ohne nachladbare Quelle; separat sichern.')
|
||||||
|
if m['id'] not in existing:blockers.append(m['file']+': lokale Gewichtedatei fehlt und hat keine Downloadquelle.')
|
||||||
|
if m['id'] in existing and existing[m['id']].get('sha256')!=m['sha256']:blockers.append('Vorhandene Modelldatei hat abweichende Prüfsumme: '+m['file'])
|
||||||
|
if doc['runtimes']['llama']:
|
||||||
|
prereq=s.runtime.prerequisites();backend=doc['runtimes']['llama']['backend']
|
||||||
|
if not prereq['cuda_ready' if backend=='CUDA' else 'cpu_ready']:blockers.append('llama.cpp-Buildwerkzeuge fehlen; Laufzeit-Voraussetzungen installieren.')
|
||||||
|
if doc['network'] and doc['network'].get('config'):
|
||||||
|
if s.network.disabled:blockers.append('Backup enthält WireGuard: Deck-Netzwerkmodul muss zuerst installiert/angebunden werden. Bestehender Gateway wird nicht übernommen.')
|
||||||
|
elif s.network.status().get('enabled') or s.network.status().get('mode')!='lan':blockers.append('Deck-Tunnel vor Netzwerk-Restore im bestätigten LAN-Modus deaktivieren.')
|
||||||
|
if doc['network']['mode']!='lan':warnings.append('Tunnelzugang wird mit Rückfallzeit aktiviert und muss über den Tunnel bestätigt werden.')
|
||||||
|
versions=doc.get('runtime_versions',{})
|
||||||
|
if versions and versions!=dict(image=dict(comfy=s.image_runtime.status()['comfy_revision'],gguf=s.image_runtime.status()['gguf_revision']),tts=s.tts_runtime.status()['revision']):blockers.append('Backup verlangt andere Bild-/TTS-Laufzeitversionen. Passende Deck-Version oder geprüftes Runtime-Upgrade verwenden.')
|
||||||
|
for c in doc['services']:
|
||||||
|
if not c.get('registry') and c.get('build_recipe')!='swarm-ui':warnings.append(c['name']+': Image nur lokal vorhanden; Registry-/Buildquelle fehlt.')
|
||||||
|
ports=s.endpoint.allowed_ports
|
||||||
|
if ports and doc['settings'].get('endpoint.json',{}).get('port',8120) not in ports:blockers.append('API-Port aus Backup ist nicht veröffentlicht. Installer zuerst für diesen Port konfigurieren.')
|
||||||
|
ident=secrets.token_hex(16);summary=dict(id=ident,created_at=doc.get('created_at'),models=[dict(id=m['id'],name=m['file'],bytes=m['size'],present=m['id'] in existing) for m in doc['models']],profiles=[p['name'] for p in doc['settings'].get('profiles.json',[])],services=[dict(name=c['name'],image=c['image'],registry=c.get('registry'),build_recipe=c.get('build_recipe'),restorable=bool(c.get('registry') or c.get('build_recipe')),running=c['running']) for c in doc['services']],download_bytes=need,runtimes=doc['runtimes'],wireguard=bool(doc['network'] and doc['network']['config']),warnings=warnings,blockers=blockers,theme=doc.get('theme','deck'))
|
||||||
|
with self.lock:
|
||||||
|
if self.busy():raise ValueError('Restore läuft bereits.')
|
||||||
|
self.plan={'document':doc,'summary':summary,'expires':time.monotonic()+1800}
|
||||||
|
return summary
|
||||||
|
def inspect_checkpoint(self,name):
|
||||||
|
if not isinstance(name,str) or not re.fullmatch(r'before-[a-f0-9]{32}\.adbackup',name):raise ValueError('Ungültiger Rückfallstand.')
|
||||||
|
p=self.directory/name;key=p.with_suffix('.key')
|
||||||
|
if p.is_symlink() or key.is_symlink() or not p.is_file() or not key.is_file():raise ValueError('Rückfallstand nicht verfügbar.')
|
||||||
|
return self.inspect(p.read_bytes(),key.read_text())
|
||||||
|
def start(self,data):
|
||||||
|
if not isinstance(data,dict) or set(data)!={'id','services','confirm','restore_credentials'} or data['confirm'] is not True or type(data['restore_credentials']) is not bool or not isinstance(data['services'],list):raise ValueError('Restore-Prüfung und ausdrückliche Bestätigung erforderlich.')
|
||||||
|
with self.lock:
|
||||||
|
if self.busy():raise ValueError('Restore läuft bereits.')
|
||||||
|
if not self.plan or self.plan['expires']<time.monotonic() or self.plan['summary']['id']!=data['id']:raise ValueError('Importprüfung abgelaufen. Backup neu auswählen.')
|
||||||
|
if self.plan['summary']['blockers']:raise ValueError('Zuerst die Hindernisse aus der Importprüfung beheben.')
|
||||||
|
doc=self.plan['document'];services={c['name']:c for c in doc['services']}
|
||||||
|
if len(set(data['services']))!=len(data['services']) or any(name not in services for name in data['services']):raise ValueError('Unbekannter Zusatzdienst.')
|
||||||
|
if any(not services[name].get('registry') and not services[name].get('build_recipe') for name in data['services']):raise ValueError('Gewählter Dienst hat keine Wiederherstellungsquelle.')
|
||||||
|
s=self.server
|
||||||
|
if s.catalog.pending or s.catalog.job and s.catalog.job['state']=='downloading' or s.runtime.busy or any((getattr(o,'job',None) or {}).get('state')=='running' for o in (s.image_runtime,s.tts_runtime,s.prompt_enhancer,s.auto_tests,s.chat_tests,s.image_tests,s.tts_tests,s.stt)):raise ValueError('Zuerst laufende Downloads, Builds und Tests abschließen oder abbrechen.')
|
||||||
|
self.cancel.clear();self.job=dict(id=secrets.token_hex(16),state='running',phase='Wiederherstellung vorbereiten',completed=0,total=len(doc['models'])+len(data['services'])+5,items=[],started_at=time.time(),theme=doc.get('theme','deck'))
|
||||||
|
self._save();threading.Thread(target=self._restore,args=(copy.deepcopy(doc),data['services'],data['restore_credentials']),daemon=True).start();self.plan=None
|
||||||
|
return self.status()
|
||||||
|
def _phase(self,text):
|
||||||
|
with self.lock:self.job['phase']=text;self._save()
|
||||||
|
def _item(self,name,state,message):
|
||||||
|
with self.lock:self.job['items'].append(dict(name=name,state=state,message=message));self.job['completed']+=1;self._save()
|
||||||
|
def _check(self):
|
||||||
|
if self.cancel.is_set():raise InterruptedError()
|
||||||
|
def _wait(self,status,get_job=lambda x:x.get('job')):
|
||||||
|
while True:
|
||||||
|
self._check();value=status();job=get_job(value)
|
||||||
|
if not job or job.get('state') not in ('running','queued','downloading'):break
|
||||||
|
self._phase(job.get('phase') or ('Gewichte herunterladen: '+job.get('file','')) if job.get('file') else job.get('phase','Dateien laden / prüfen'))
|
||||||
|
with self.lock:self.job['current']={k:job[k] for k in ('bytes','total','bytes_per_second','eta_seconds') if k in job};self._save()
|
||||||
|
self.cancel.wait(1)
|
||||||
|
if job and job.get('state') not in ('complete',):raise ValueError('Download oder Laufzeitinstallation fehlgeschlagen. Zugang, Speicher und Voraussetzungen prüfen.')
|
||||||
|
with self.lock:self.job.pop('current',None);self._save()
|
||||||
|
return value
|
||||||
|
def _runtimes(self,doc):
|
||||||
|
s=self.server;r=doc['runtimes']
|
||||||
|
if r['image'] and not s.image_runtime.status()['installed']:
|
||||||
|
self._phase('Vorhandene Bild-/Videolaufzeit prüfen oder installieren');s.image_runtime.start();self._wait(s.image_runtime.status)
|
||||||
|
if r['tts'] and not s.tts_runtime.status()['installed']:
|
||||||
|
self._phase('TTS-Laufzeit und Zusatzdateien installieren');s.tts_runtime.start();self._wait(s.tts_runtime.status)
|
||||||
|
targets=r['llama_builds']+([r['llama']] if r['llama'] and r['llama'] not in r['llama_builds'] else [])
|
||||||
|
for target in targets:
|
||||||
|
state=s.runtime.status();matching=next((b for b in state['builds'] if b['commit']==target['commit'] and b['backend']==target['backend']),None)
|
||||||
|
if not matching:
|
||||||
|
self._phase('llama.cpp-Version aus Backup bauen');s.runtime.start(target['commit'],target['backend'],1);state=self._wait(s.runtime.status);matching=state['builds'][-1]
|
||||||
|
if target==r['llama']:s.runtime.activate(matching['id'])
|
||||||
|
for task,revision in r['enhancers'].items():
|
||||||
|
if not s.prompt_enhancer.installed(task):
|
||||||
|
s.prompt_enhancer.install(task,revision=revision);self._wait(s.prompt_enhancer.status)
|
||||||
|
self._item('Laufzeiten','complete','Vorhandene Installationen wiederverwendet; fehlende Laufzeiten eingerichtet.')
|
||||||
|
def _restore(self,doc,selected,restore_credentials):
|
||||||
|
s=self.server;errors=[];rollback=None;before=None
|
||||||
|
try:
|
||||||
|
# Preserve non-reproducible settings before touching anything; never overwrite old rollback files.
|
||||||
|
self._phase('Lokalen Rückfallstand sichern')
|
||||||
|
rollback=self.directory/('before-'+self.job['id']+'.adbackup')
|
||||||
|
before=self.snapshot_for_rollback()
|
||||||
|
key=secrets.token_urlsafe(32);rollback.write_bytes(seal(before,key));rollback.chmod(0o600)
|
||||||
|
# Recovery key stays on this server in a separate protected file, never in status responses.
|
||||||
|
(rollback.with_suffix('.key')).write_text(key);rollback.with_suffix('.key').chmod(0o600)
|
||||||
|
s.endpoint.close();s.video.close();s.tts_tests.stop();s.stt.stop()
|
||||||
|
until=time.monotonic()+30
|
||||||
|
while s.endpoint.inflight and time.monotonic()<until:time.sleep(.1)
|
||||||
|
if s.endpoint.inflight:raise ValueError('API-Aufträge wurden nicht rechtzeitig beendet. Restore erneut versuchen.')
|
||||||
|
with s.endpoint.lock:s.endpoint.http=None;s.endpoint.thread=None;s.endpoint.state='stopped'
|
||||||
|
with s.scheduler.cv:s.scheduler.gpu_mode='restoring';s.scheduler.cv.notify_all()
|
||||||
|
self._item('Rückfallstand','complete','Vorherige Einstellungen geschützt gesichert; nur Deck-eigene Worker beendet.')
|
||||||
|
hf=doc['settings'].get('models/huggingface.json')
|
||||||
|
if hf:atomic_write(self.root/'models/huggingface.json',hf)
|
||||||
|
self._runtimes(doc)
|
||||||
|
for m in doc['models']:
|
||||||
|
self._check();self._phase('Gewichte: '+m['file'])
|
||||||
|
try:
|
||||||
|
entry=s.catalog.entry(m['id']);path=s.catalog.root/m['id']/('model'+PurePosixPath(m['file']).suffix)
|
||||||
|
digest=hashlib.sha256()
|
||||||
|
with path.open('rb') as stream:
|
||||||
|
for block in iter(lambda:stream.read(4*1024**2),b''):self._check();digest.update(block)
|
||||||
|
if digest.hexdigest()!=m['sha256']:raise ValueError('Vorhandene Datei stimmt nicht mit Backup-Prüfsumme überein.')
|
||||||
|
self._item(m['file'],'reused','Vorhandene Datei per SHA-256 geprüft.')
|
||||||
|
except ValueError:
|
||||||
|
if (s.catalog.root/m['id']/'entry.json').exists():raise
|
||||||
|
source=m.get('download_source') or m
|
||||||
|
s.catalog.start(source['repo'],source['file'],source['revision'],m['kind']);self._wait(s.catalog.status)
|
||||||
|
source_id=hashlib.sha256((source['repo']+source['revision']+source['file']).encode()).hexdigest()
|
||||||
|
entry=s.catalog.entry(source_id)
|
||||||
|
if entry['sha256']!=m['sha256']:raise ValueError('Geladene Datei stimmt nicht mit Backup-Prüfsumme überein.')
|
||||||
|
if source_id!=m['id']:
|
||||||
|
target=s.catalog.root/m['id']
|
||||||
|
if target.exists():
|
||||||
|
if any(target.iterdir()):raise ValueError('Lokaler Modellordner ist anderweitig belegt.')
|
||||||
|
target.rmdir()
|
||||||
|
(s.catalog.root/source_id).rename(target)
|
||||||
|
old=target/('model'+PurePosixPath(source['file']).suffix);new=target/('model'+PurePosixPath(m['file']).suffix)
|
||||||
|
if old!=new:old.rename(new)
|
||||||
|
entry.update({k:m[k] for k in ('repo','revision','file','kind')});entry['download_source']=m['download_source'];entry.pop('id',None);atomic_write(target/'entry.json',entry)
|
||||||
|
self._item(m['file'],'complete','Feste Quellversion geladen und SHA-256 geprüft.')
|
||||||
|
for c in doc['services']:
|
||||||
|
self._check()
|
||||||
|
if c['name'] not in selected:self._item(c['name'],'skipped','Vom Restore ausgeschlossen.');continue
|
||||||
|
self._phase('Zusatzdienst: '+c['name'])
|
||||||
|
try:result=s.docker.call('backup-restore',values={'container':c});self._item(c['name'],result['state'],result['message'])
|
||||||
|
except (OSError,ValueError):errors.append(c['name']);self._item(c['name'],'failed','Container nicht wiederhergestellt; Registry, Quelle, Ports oder Systemhelfer prüfen.')
|
||||||
|
if doc['runtimes']['swarm_nodes'] and 'athena-swarm-ui' in selected:
|
||||||
|
self._phase('Swarm-Abhängigkeiten zur vorhandenen Video-Laufzeit ergänzen')
|
||||||
|
from image_runtime import ImageRuntime
|
||||||
|
python,_=s.image_runtime.paths()
|
||||||
|
import subprocess
|
||||||
|
subprocess.run([str(python),'-m','pip','install','--upgrade','--no-cache-dir','--no-deps','--target',str(self.root/'video/swarm-python'),'opencv-python-headless==4.12.0.88','imageio-ffmpeg==0.6.0'],stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,check=True,timeout=300)
|
||||||
|
self._check();self._phase('Profile, Zuordnungen und Einstellungen anwenden')
|
||||||
|
self._apply_settings(doc)
|
||||||
|
if 'history' in doc:
|
||||||
|
self._restore_history(doc['history'])
|
||||||
|
self._item('Einstellungen','complete','Profile, API-Freigaben, Komponenten und Audio-Verhalten gesetzt.')
|
||||||
|
if doc['network'] and doc['network']['config']:
|
||||||
|
try:
|
||||||
|
result=s.network.call('backup-restore',{'backup':doc['network']})
|
||||||
|
if result.get('pending'):errors.append('WireGuard-Bestätigung')
|
||||||
|
self._item('WireGuard','pending' if result.get('pending') else 'complete','Deck-Konfiguration gesetzt; Handshake geprüft. Tunnelmodus gegebenenfalls im Netzwerkbereich bestätigen.')
|
||||||
|
except (OSError,ValueError):errors.append('WireGuard');self._item('WireGuard','failed','Netzwerk nicht vollständig wiederhergestellt. LAN-Zugang bleibt erhalten; Netzwerkbereich prüfen.')
|
||||||
|
else:self._item('WireGuard','skipped','Kein Deck-WireGuard im Backup; vorhandene Gateways bleiben unberührt.')
|
||||||
|
if restore_credentials:
|
||||||
|
current=s.credentials.read();s.credentials.write(doc['credentials'],current['revision'])
|
||||||
|
with s.auth_lock:
|
||||||
|
for session in s.sessions.values():session['revision']=doc['credentials']['password']['hash']
|
||||||
|
self._item('Zugang','complete','Oberflächenkennwort und API-Token-Prüfwert aus Backup wiederhergestellt.')
|
||||||
|
else:self._item('Zugang','skipped','Aktueller Oberflächenzugang und API-Token bleiben erhalten.')
|
||||||
|
with s.scheduler.cv:s.scheduler.gpu_mode='video' if s.video.foreign_blocked else 'llm';s.scheduler.cv.notify_all()
|
||||||
|
if s.endpoint.config.get('autostart'):
|
||||||
|
try:s.endpoint.start()
|
||||||
|
except (OSError,ValueError):errors.append('API-Endpunkt');self._item('API-Endpunkt','failed','Listener konnte nicht gestartet werden; Port prüfen.')
|
||||||
|
with self.lock:self.job.update(state='partial' if errors else 'complete',phase='Wiederherstellung mit offenen Punkten beendet.' if errors else 'Wiederherstellung abgeschlossen. Oberfläche ist bereit.',finished_at=time.time(),completed=self.job['total']);self._save()
|
||||||
|
except Exception as exc:
|
||||||
|
# Restore preexisting application settings; downloaded files/created services remain available.
|
||||||
|
if before is not None:
|
||||||
|
try:
|
||||||
|
self._apply_settings(before)
|
||||||
|
old=before['runtimes']['llama']
|
||||||
|
if old:
|
||||||
|
item=next((b for b in s.runtime.status()['builds'] if b['commit']==old['commit'] and b['backend']==old['backend']),None)
|
||||||
|
if item:s.runtime.activate(item['id'])
|
||||||
|
if s.endpoint.config.get('autostart'):s.endpoint.start()
|
||||||
|
except Exception:pass
|
||||||
|
|
||||||
|
with s.scheduler.cv:s.scheduler.gpu_mode='video' if s.video.foreign_blocked else 'llm';s.scheduler.cv.notify_all()
|
||||||
|
self._item('Restore','cancelled' if isinstance(exc,InterruptedError) else 'failed',str(exc) if isinstance(exc,ValueError) else 'Wiederherstellung unterbrochen. Rückfallstand vorhanden; Backup erneut prüfen.')
|
||||||
|
with self.lock:self.job.update(state='cancelled' if isinstance(exc,InterruptedError) else 'failed',phase='Restore nicht abgeschlossen. Rückfall auf vorherige Einstellungen versucht; geschützter Rückfallstand vorhanden.',finished_at=time.time());self._save()
|
||||||
|
def _apply_settings(self,doc):
|
||||||
|
s=self.server
|
||||||
|
for name in SETTINGS:
|
||||||
|
if name not in doc['settings']:(self.root/name).unlink(missing_ok=True)
|
||||||
|
for name,value in doc['settings'].items():
|
||||||
|
if name=='video/comfy-client-token':
|
||||||
|
path=self.root/name;path.parent.mkdir(parents=True,exist_ok=True);path.write_text(value+'\n');path.chmod(0o600)
|
||||||
|
else:atomic_write(self.root/name,value)
|
||||||
|
with s.profiles.lock:s.profiles.rows=copy.deepcopy(doc['settings'].get('profiles.json',[]))
|
||||||
|
with s.endpoint.lock:s.endpoint.config=copy.deepcopy(doc['settings'].get('endpoint.json',dict(port=8120,enabled_profiles=[],autostart=False)))
|
||||||
|
s.video.selection=copy.deepcopy(doc['settings'].get('video/comfy-selection.json',{}))
|
||||||
|
policy=doc['settings'].get('audio-policy.json',{k:{'mode':'auto','profile_id':None} for k in ('tts','stt')})
|
||||||
|
with s.audio_policy.lock:s.audio_policy.settings=policy
|
||||||
|
s.tts_tests.policy=policy.get('tts',{}).get('mode','auto');s.stt.policy=policy.get('stt',{}).get('mode','auto')
|
||||||
|
def snapshot_for_rollback(self):
|
||||||
|
return self.snapshot(include_history=False,_during_restore=True)
|
||||||
|
def _restore_history(self,data,validate_only=False):
|
||||||
|
raw=base64.b64decode(data,validate=True)
|
||||||
|
if len(raw)>MAX_HISTORY:raise ValueError('History im Backup zu groß.')
|
||||||
|
with tempfile.TemporaryDirectory(dir=self.directory) as work:
|
||||||
|
path=Path(work)/'history.sqlite3';path.write_bytes(raw)
|
||||||
|
with sqlite3.connect('file:'+str(path)+'?mode=ro',uri=True) as source:
|
||||||
|
if source.execute('PRAGMA quick_check').fetchone()[0]!='ok':raise ValueError('Ungültige Dashboard-History.')
|
||||||
|
expected={'meta','samples_raw','samples_hourly','slot_samples_raw','slot_samples_hourly','token_totals','token_hourly','model_events'}
|
||||||
|
if {r[0] for r in source.execute("SELECT name FROM sqlite_master WHERE type='table'")}!=expected or source.execute("SELECT COUNT(*) FROM sqlite_master WHERE type IN ('trigger','view')").fetchone()[0]:raise ValueError('Unbekanntes History-Schema.')
|
||||||
|
if not validate_only:
|
||||||
|
with self.server.dashboard.history._lock:source.backup(self.server.dashboard.history._db)
|
||||||
|
def stop(self):
|
||||||
|
with self.lock:
|
||||||
|
self.cancel.set()
|
||||||
|
if self.busy():
|
||||||
|
self.server.catalog.stop();self.server.runtime.stop();self.server.image_runtime.stop();self.server.tts_runtime.stop();self.server.prompt_enhancer.stop()
|
||||||
|
return {'cancellation_requested':True}
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
"""Versioned, authenticated backup envelope; no archive extraction or executable pickle."""
|
||||||
|
import hashlib,json,secrets,zlib
|
||||||
|
MAGIC=b'ATHENA-DECK-BACKUP\x01\n'
|
||||||
|
MAX=64*1024*1024
|
||||||
|
|
||||||
|
def password(value):
|
||||||
|
if not isinstance(value,str) or not 16<=len(value)<=256:raise ValueError('Backup-Kennwort: 16 bis 256 Zeichen erforderlich.')
|
||||||
|
return value.encode()
|
||||||
|
|
||||||
|
def seal(document,phrase):
|
||||||
|
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||||
|
salt=secrets.token_bytes(16);nonce=secrets.token_bytes(12);head=MAGIC+salt+nonce
|
||||||
|
key=hashlib.scrypt(password(phrase),salt=salt,n=32768,r=8,p=1,maxmem=64*1024*1024,dklen=32)
|
||||||
|
raw=json.dumps(document,ensure_ascii=False,allow_nan=False,separators=(',',':')).encode()
|
||||||
|
if len(raw)>MAX:raise ValueError('Konfigurationsbackup überschreitet 64 MiB.')
|
||||||
|
return head+AESGCM(key).encrypt(nonce,zlib.compress(raw),head)
|
||||||
|
|
||||||
|
def open_backup(raw,phrase):
|
||||||
|
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||||
|
from cryptography.exceptions import InvalidTag
|
||||||
|
if not isinstance(raw,bytes) or not len(MAGIC)+44<=len(raw)<=MAX or not raw.startswith(MAGIC):raise ValueError('Kein unterstütztes Athena-Deck-Backup.')
|
||||||
|
salt=raw[len(MAGIC):len(MAGIC)+16];nonce=raw[len(MAGIC)+16:len(MAGIC)+28];head=raw[:len(MAGIC)+28]
|
||||||
|
key=hashlib.scrypt(password(phrase),salt=salt,n=32768,r=8,p=1,maxmem=64*1024*1024,dklen=32)
|
||||||
|
try:packed=AESGCM(key).decrypt(nonce,raw[len(head):],head)
|
||||||
|
except InvalidTag:raise ValueError('Backup-Kennwort falsch oder Backup beschädigt.') from None
|
||||||
|
try:
|
||||||
|
decoder=zlib.decompressobj();data=decoder.decompress(packed,MAX+1)
|
||||||
|
if len(data)>MAX or decoder.unconsumed_tail or not decoder.eof or decoder.unused_data:raise ValueError()
|
||||||
|
result=json.loads(data,parse_constant=lambda _:(_ for _ in ()).throw(ValueError()))
|
||||||
|
except (ValueError,zlib.error,UnicodeError):raise ValueError('Backup-Inhalt ungültig oder zu groß.') from None
|
||||||
|
if not isinstance(result,dict):raise ValueError('Ungültiges Backup.')
|
||||||
|
return result
|
||||||
+4
-2
@@ -11,11 +11,13 @@ RUN git clone https://github.com/Comfy-Org/ComfyUI.git /opt/deck-comfy \
|
|||||||
&& /opt/deck-image-python/bin/pip install --no-cache-dir -c /tmp/image-requirements.lock -r /opt/deck-comfy/requirements.txt -r /opt/deck-comfy/custom_nodes/ComfyUI-GGUF/requirements.txt accelerate==1.15.0 \
|
&& /opt/deck-image-python/bin/pip install --no-cache-dir -c /tmp/image-requirements.lock -r /opt/deck-comfy/requirements.txt -r /opt/deck-comfy/custom_nodes/ComfyUI-GGUF/requirements.txt accelerate==1.15.0 \
|
||||||
&& /opt/deck-image-python/bin/pip freeze > /opt/deck-comfy/deck-requirements.lock
|
&& /opt/deck-image-python/bin/pip freeze > /opt/deck-comfy/deck-requirements.lock
|
||||||
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends python3-dev && rm -rf /var/lib/apt/lists/*
|
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends python3-dev && rm -rf /var/lib/apt/lists/*
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends python3-cryptography && rm -rf /var/lib/apt/lists/*
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
COPY deploy/image-requirements.lock /app/deploy/image-requirements.lock
|
COPY deploy/image-requirements.lock /app/deploy/image-requirements.lock
|
||||||
COPY deploy/tts-requirements.lock /app/deploy/tts-requirements.lock
|
COPY deploy/tts-requirements.lock /app/deploy/tts-requirements.lock
|
||||||
COPY audio_policy.py prompt_enhancer.py prompt_enhancer_worker.py api_compat.py stt.py execution_setup.py tts_runtime.py tts_test.py tts_worker.py auto_test.py chat_test.py endpoint.py inference.py docker_support.py image_encoder_node.py image_runtime.py image_test.py image_upload.py profiles.py capacity.py server.py runtime.py video.py video_proxy.py video_comfy.py video_comfy_node.py video_comfy_proxy.py catalog.py hub_auth.py auth.py collect_hardware.py dashboard_data.py dashboard_history.py /app/
|
COPY backup.py backup_codec.py audio_policy.py prompt_enhancer.py prompt_enhancer_worker.py api_compat.py stt.py execution_setup.py tts_runtime.py tts_test.py tts_worker.py auto_test.py chat_test.py endpoint.py inference.py docker_support.py image_encoder_node.py image_runtime.py image_test.py image_upload.py profiles.py capacity.py server.py runtime.py video.py video_proxy.py video_comfy.py video_comfy_node.py video_comfy_proxy.py catalog.py hub_auth.py auth.py collect_hardware.py dashboard_data.py dashboard_history.py /app/
|
||||||
COPY video-ui.js stt-ui.js tts-ui.js auto-test-ui.js chat-test-ui.js endpoint-ui.js docker-ui.js image-test-ui.js profiles-ui.js dashboard-ui.js index.html app.js studio.js runtime-ui.js catalog-ui.js style.css dashboard.css themes.css login.html login.js access-ui.js network-ui.js /app/
|
COPY backup-ui.js video-ui.js stt-ui.js tts-ui.js auto-test-ui.js chat-test-ui.js endpoint-ui.js docker-ui.js image-test-ui.js profiles-ui.js dashboard-ui.js index.html app.js studio.js runtime-ui.js catalog-ui.js style.css dashboard.css themes.css login.html login.js access-ui.js network-ui.js /app/
|
||||||
|
COPY deploy/docker_backup.py /app/deploy/docker_backup.py
|
||||||
COPY network/__init__.py network/client.py network/config.py network/rpc.py /app/network/
|
COPY network/__init__.py network/client.py network/config.py network/rpc.py /app/network/
|
||||||
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 HOME=/tmp \
|
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 HOME=/tmp \
|
||||||
DECK_BIND_HOST=0.0.0.0 DECK_STATE_DIR=/var/lib/deck \
|
DECK_BIND_HOST=0.0.0.0 DECK_STATE_DIR=/var/lib/deck \
|
||||||
|
|||||||
@@ -1,11 +1,13 @@
|
|||||||
ARG DECK_RUNTIME_IMAGE
|
ARG DECK_RUNTIME_IMAGE
|
||||||
FROM ${DECK_RUNTIME_IMAGE}
|
FROM ${DECK_RUNTIME_IMAGE}
|
||||||
USER root
|
USER root
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends python3-cryptography && rm -rf /var/lib/apt/lists/*
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
COPY deploy/image-requirements.lock /app/deploy/image-requirements.lock
|
COPY deploy/image-requirements.lock /app/deploy/image-requirements.lock
|
||||||
COPY deploy/tts-requirements.lock /app/deploy/tts-requirements.lock
|
COPY deploy/tts-requirements.lock /app/deploy/tts-requirements.lock
|
||||||
COPY audio_policy.py prompt_enhancer.py prompt_enhancer_worker.py api_compat.py stt.py execution_setup.py tts_runtime.py tts_test.py tts_worker.py auto_test.py chat_test.py endpoint.py inference.py docker_support.py image_encoder_node.py image_runtime.py image_test.py image_upload.py profiles.py capacity.py server.py runtime.py video.py video_proxy.py video_comfy.py video_comfy_node.py video_comfy_proxy.py catalog.py hub_auth.py auth.py collect_hardware.py dashboard_data.py dashboard_history.py /app/
|
COPY backup.py backup_codec.py audio_policy.py prompt_enhancer.py prompt_enhancer_worker.py api_compat.py stt.py execution_setup.py tts_runtime.py tts_test.py tts_worker.py auto_test.py chat_test.py endpoint.py inference.py docker_support.py image_encoder_node.py image_runtime.py image_test.py image_upload.py profiles.py capacity.py server.py runtime.py video.py video_proxy.py video_comfy.py video_comfy_node.py video_comfy_proxy.py catalog.py hub_auth.py auth.py collect_hardware.py dashboard_data.py dashboard_history.py /app/
|
||||||
COPY video-ui.js stt-ui.js tts-ui.js auto-test-ui.js chat-test-ui.js endpoint-ui.js docker-ui.js image-test-ui.js profiles-ui.js dashboard-ui.js index.html app.js studio.js runtime-ui.js catalog-ui.js style.css dashboard.css themes.css login.html login.js access-ui.js network-ui.js /app/
|
COPY backup-ui.js video-ui.js stt-ui.js tts-ui.js auto-test-ui.js chat-test-ui.js endpoint-ui.js docker-ui.js image-test-ui.js profiles-ui.js dashboard-ui.js index.html app.js studio.js runtime-ui.js catalog-ui.js style.css dashboard.css themes.css login.html login.js access-ui.js network-ui.js /app/
|
||||||
|
COPY deploy/docker_backup.py /app/deploy/docker_backup.py
|
||||||
COPY network/__init__.py network/client.py network/config.py network/rpc.py /app/network/
|
COPY network/__init__.py network/client.py network/config.py network/rpc.py /app/network/
|
||||||
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 HOME=/tmp \
|
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 HOME=/tmp \
|
||||||
DECK_BIND_HOST=0.0.0.0 DECK_STATE_DIR=/var/lib/deck \
|
DECK_BIND_HOST=0.0.0.0 DECK_STATE_DIR=/var/lib/deck \
|
||||||
|
|||||||
@@ -0,0 +1,176 @@
|
|||||||
|
"""Configuration-only backup of labelled apps; no Docker socket exposed to the GUI.
|
||||||
|
Imported containers never get privileged mode, devices, Docker sockets or host filesystem mounts.
|
||||||
|
"""
|
||||||
|
import base64,json,os,re,shutil,subprocess,socket,time
|
||||||
|
from pathlib import Path,PurePosixPath
|
||||||
|
MAX_FILE=1024*1024
|
||||||
|
APP='io.athena-deck.application'
|
||||||
|
LABELS={'io.athena-deck.managed':'true','io.athena-deck.role':'application'}
|
||||||
|
|
||||||
|
def command(args,timeout=60):
|
||||||
|
r=subprocess.run(['/usr/bin/docker',*args],capture_output=True,text=True,timeout=timeout)
|
||||||
|
if r.returncode:raise ValueError('Docker-Schritt fehlgeschlagen; Image-Zugang, Port und Docker prüfen.')
|
||||||
|
return r.stdout.strip()
|
||||||
|
|
||||||
|
def blob(path):
|
||||||
|
if path.is_symlink() or not path.is_file() or path.stat().st_size>MAX_FILE:raise ValueError('Dienstkonfiguration nicht sicher lesbar oder größer als 1 MiB.')
|
||||||
|
return base64.b64encode(path.read_bytes()).decode()
|
||||||
|
|
||||||
|
def config_files(source,destination):
|
||||||
|
p=Path(source)
|
||||||
|
if destination.startswith('/run/secrets/'):
|
||||||
|
return {'file':blob(p)} if p.exists() else {}
|
||||||
|
if destination.endswith('/Data') and p.is_dir():
|
||||||
|
files={}
|
||||||
|
for name in ('Settings.fds','Backends.fds'):
|
||||||
|
if (p/name).is_file():files[name]=blob(p/name)
|
||||||
|
return files
|
||||||
|
return {}
|
||||||
|
|
||||||
|
def app_uid(container,user):
|
||||||
|
if not user:return 0
|
||||||
|
if user.split(':')[0].isdigit():return int(user.split(':')[0])
|
||||||
|
with __import__('tempfile').TemporaryDirectory() as d:
|
||||||
|
p=Path(d)/'passwd';command(['cp',container+':/etc/passwd',str(p)])
|
||||||
|
row=next((x.split(':') for x in p.read_text().splitlines() if x.split(':')[0]==user.split(':')[0]),None)
|
||||||
|
if not row:raise ValueError('Containerbenutzer nicht auflösbar.')
|
||||||
|
return int(row[2])
|
||||||
|
|
||||||
|
def export(manager):
|
||||||
|
policy=manager.state/'backup-policy.json'
|
||||||
|
deck=json.loads(policy.read_text())['deck_state'] if policy.exists() else None
|
||||||
|
result=[]
|
||||||
|
for row in manager.inventory():
|
||||||
|
x=json.loads(command(['inspect',row['id']]))[0];c=x['Config'];h=x['HostConfig'];image=json.loads(command(['image','inspect',x['Image']]))[0]
|
||||||
|
registry=next((d for d in image.get('RepoDigests',[]) if '/' in d.split('@')[0] and ('.' in d.split('/')[0] or ':' in d.split('/')[0])),None)
|
||||||
|
if not registry and row['name']!='athena-swarm-ui':registry=next(iter(image.get('RepoDigests',[])),None)
|
||||||
|
mounts=[]
|
||||||
|
for i,m in enumerate(x['Mounts']):
|
||||||
|
source=m['Source'];relative=None
|
||||||
|
if deck:
|
||||||
|
try:relative=str(Path(source).relative_to(deck))
|
||||||
|
except ValueError:pass
|
||||||
|
mounts.append(dict(index=i,target=m['Destination'],read_only=not m['RW'],deck_path=relative,files=config_files(source,m['Destination']),was_file=Path(source).is_file()))
|
||||||
|
result.append(dict(name=row['name'],image=c['Image'],registry=registry,application=c.get('Labels',{}).get(APP,''),labels={k:v for k,v in (c.get('Labels') or {}).items() if k.startswith('io.athena-deck.')},env=c.get('Env') or [],entrypoint=c.get('Entrypoint'),cmd=c.get('Cmd'),uid=app_uid(row['id'],c.get('User') or ''),user=c.get('User') or '',workdir=c.get('WorkingDir') or '',network=h['NetworkMode'],ports=h.get('PortBindings') or {},restart=h.get('RestartPolicy',{}).get('Name') or 'no',memory=h.get('Memory') or 0,nanocpus=h.get('NanoCpus') or 0,running=x['State']['Running'],mounts=mounts,build_recipe='swarm-ui' if row['name']=='athena-swarm-ui' else None,unsupported=bool(h.get('Privileged') or h.get('DeviceRequests') or h.get('Devices') or h.get('CapAdd'))))
|
||||||
|
return {'services':result,'configured':bool(deck)}
|
||||||
|
|
||||||
|
def validate(c):
|
||||||
|
if not isinstance(c,dict) or not re.fullmatch(r'[a-zA-Z0-9][a-zA-Z0-9_.-]{0,63}',c.get('name','')):raise ValueError('Ungültiger Containername.')
|
||||||
|
if any(c.get('labels',{}).get(k)!=v for k,v in LABELS.items()):raise ValueError('Nur ausdrücklich Deck zugeordnete Anwendungscontainer erlaubt.')
|
||||||
|
if c.get('unsupported'):raise ValueError('GPU-/privilegierter Container benötigt manuelle Einrichtung.')
|
||||||
|
if c.get('network') not in ('bridge','default','host'):raise ValueError('Benutzerdefiniertes Docker-Netzwerk benötigt manuelle Einrichtung.')
|
||||||
|
if c.get('network')=='host' and c['name'] not in ('athena-swarm-ui','ltx-deskweb'):raise ValueError('Host-Netzwerk nur für bekannte Deck-Oberflächen erlaubt.')
|
||||||
|
if c.get('build_recipe') not in (None,'swarm-ui') or c.get('build_recipe')=='swarm-ui' and (c['name']!='athena-swarm-ui' or c.get('image')!='athena-swarm-ui:de7b834'):raise ValueError('Unbekanntes Build-Rezept.')
|
||||||
|
ref=c.get('registry') or c.get('image','')
|
||||||
|
if not isinstance(ref,str) or not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9._/@:-]{0,300}',ref):raise ValueError('Ungültige Image-Referenz.')
|
||||||
|
if c.get('restart') not in ('no','always','unless-stopped','on-failure'):raise ValueError('Ungültige Neustartregel.')
|
||||||
|
for field in ('env','entrypoint','cmd'):
|
||||||
|
v=c.get(field)
|
||||||
|
if v is not None and (not isinstance(v,list) or len(v)>200 or any(not isinstance(t,str) or len(t)>16384 or '\x00' in t for t in v)):raise ValueError('Ungültige Containerparameter.')
|
||||||
|
for field in ('user','workdir'):
|
||||||
|
if not isinstance(c.get(field,''),str) or len(c.get(field,''))>512 or '\x00' in c.get(field,''):raise ValueError('Ungültige Containerparameter.')
|
||||||
|
if type(c.get('uid',0)) is not int or not 0<=c.get('uid',0)<=4294967294:raise ValueError('Ungültige Anwendungs-UID.')
|
||||||
|
if type(c.get('running')) is not bool or not isinstance(c.get('labels'),dict) or len(c['labels'])>30 or any(not isinstance(k,str) or not k.startswith('io.athena-deck.') or not isinstance(v,str) or len(v)>512 for k,v in c['labels'].items()):raise ValueError('Ungültige Container-Einstellungen.')
|
||||||
|
for field in ('memory','nanocpus'):
|
||||||
|
if type(c.get(field)) is not int or not 0<=c[field]<=1024**5:raise ValueError('Ungültige Ressourcenbegrenzung.')
|
||||||
|
ports=c.get('ports')
|
||||||
|
if not isinstance(ports,dict) or len(ports)>20:raise ValueError('Ungültige Ports.')
|
||||||
|
for container,bindings in ports.items():
|
||||||
|
if not re.fullmatch(r'\d{1,5}/(?:tcp|udp)',container) or not 1<=int(container.split('/')[0])<=65535 or not isinstance(bindings,list):raise ValueError('Ungültige Ports.')
|
||||||
|
for binding in bindings:
|
||||||
|
if binding.get('HostIp') not in ('127.0.0.1','::1'):raise ValueError('Restore veröffentlicht Containerports nur auf Loopback.')
|
||||||
|
if not str(binding.get('HostPort','')).isdigit() or not 1024<=int(binding['HostPort'])<=65535:raise ValueError('Ungültiger Host-Port.')
|
||||||
|
mounts=c.get('mounts')
|
||||||
|
if not isinstance(mounts,list) or len(mounts)>30:raise ValueError('Ungültige Volumes.')
|
||||||
|
targets=set()
|
||||||
|
for i,m in enumerate(mounts):
|
||||||
|
target=m.get('target','');parts=PurePosixPath(target).parts
|
||||||
|
if not target.startswith('/') or '..' in parts or ',' in target or ':' in target or target in targets or target.startswith(('/proc','/sys','/dev','/etc','/var/run','/run/athena')) or 'docker.sock' in target:raise ValueError('Unsicheres Volume-Ziel.')
|
||||||
|
targets.add(target)
|
||||||
|
if m.get('index')!=i or type(m.get('read_only')) is not bool or type(m.get('was_file')) is not bool:raise ValueError('Ungültige Volume-Einstellung.')
|
||||||
|
rel=m.get('deck_path')
|
||||||
|
if rel is not None and (rel not in ('models','video/comfy-work/models','video/comfy-client-token') or not m['read_only']):raise ValueError('Deck-Dateien dürfen nur als bekannte, lesende Modell-/Token-Volumes eingebunden werden.')
|
||||||
|
for name,data in m.get('files',{}).items():
|
||||||
|
if name not in ('file','Settings.fds','Backends.fds'):raise ValueError('Unbekannte Konfigurationsdatei.')
|
||||||
|
if not isinstance(data,str) or len(base64.b64decode(data,validate=True))>MAX_FILE:raise ValueError('Ungültige Konfigurationsdatei.')
|
||||||
|
return c
|
||||||
|
|
||||||
|
def restore(manager,c):
|
||||||
|
validate(c)
|
||||||
|
policy=manager.state/'backup-policy.json'
|
||||||
|
if not policy.is_file():raise ValueError('Deck-Zustandsverzeichnis muss im Systemhelfer registriert sein.')
|
||||||
|
if policy.is_symlink() or policy.stat().st_uid!=0 or policy.stat().st_mode&0o022:raise ValueError('Systemhelfer-Registrierung nicht vertrauenswürdig.')
|
||||||
|
deck=Path(json.loads(policy.read_text())['deck_state']).resolve();name=c['name']
|
||||||
|
existing=subprocess.run(['/usr/bin/docker','inspect',name],capture_output=True,text=True)
|
||||||
|
if existing.returncode==0:
|
||||||
|
x=json.loads(existing.stdout)[0]
|
||||||
|
if any(x['Config'].get('Labels',{}).get(k)!=v for k,v in LABELS.items()):raise ValueError('Containername ist durch einen fremden Dienst belegt.')
|
||||||
|
# Never replace or restart an existing service during a restore.
|
||||||
|
if x['Config']['Image'] not in (c['image'],c.get('registry')):raise ValueError('Vorhandener Container verwendet ein anderes Image; manuell prüfen.')
|
||||||
|
return {'state':'reused','message':'Vorhandener markierter Container unverändert übernommen.'}
|
||||||
|
if c.get('build_recipe')=='swarm-ui':
|
||||||
|
source=Path(__file__).parent/'swarm-ui'
|
||||||
|
if not (source/'Dockerfile').is_file():raise ValueError('Gepinntes Swarm-Build-Rezept fehlt im Systemhelfer.')
|
||||||
|
command(['build','-t',c['image'],str(source)],1800)
|
||||||
|
else:
|
||||||
|
if not c.get('registry'):raise ValueError('Lokales Image hat keine Registry-Quelle und kein bekanntes Build-Rezept.')
|
||||||
|
command(['pull',c['registry']],1800)
|
||||||
|
if c.get('build_recipe')=='swarm-ui':
|
||||||
|
nodes=deck/'video/swarm-comfy-nodes';nodes.mkdir(parents=True,exist_ok=True)
|
||||||
|
temporary=command(['create',c['image']])
|
||||||
|
try:command(['cp',temporary+':/swarm/src/BuiltinExtensions/ComfyUIBackend/ExtraNodes/.',str(nodes)])
|
||||||
|
finally:command(['rm',temporary])
|
||||||
|
for p in [nodes,*nodes.rglob('*')]:os.chown(p,65534,65534)
|
||||||
|
work=deck/'video/comfy-work/models'
|
||||||
|
for folder in ('diffusion_models','text_encoders','vae','latent_upscale_models','loras','Stable-Diffusion','Lora','VAE','Embeddings','controlnet','model_patches','clip','clip_vision','upscale_models','tensorrt','unet'):
|
||||||
|
(work/folder).mkdir(parents=True,exist_ok=True);os.chown(work/folder,65534,65534)
|
||||||
|
for meta in (deck/'models').glob('*/entry.json'):
|
||||||
|
item=json.loads(meta.read_text());f=PurePosixPath(item['file']);source=meta.parent/('model'+f.suffix)
|
||||||
|
if item.get('repo')=='Lightricks/LTX-2.5' and f.parts[0] in ('diffusion_models','text_encoders','vae','latent_upscale_models') and source.is_file():
|
||||||
|
link=work/f.parts[0]/f.name
|
||||||
|
if not link.exists() and not link.is_symlink():link.symlink_to('/var/lib/deck/models/'+meta.parent.name+'/model'+f.suffix)
|
||||||
|
base=manager.state/'services'/name
|
||||||
|
if base.is_symlink():raise ValueError('Unsicheres Dienstverzeichnis.')
|
||||||
|
base.mkdir(parents=True,exist_ok=True,mode=0o700)
|
||||||
|
args=['create','--name',name,'--network',c['network'],'--restart',c['restart'],'--cap-drop','ALL','--security-opt','no-new-privileges:true','--pids-limit','256']
|
||||||
|
if c['memory']:args+=['--memory',str(c['memory'])]
|
||||||
|
if c['nanocpus']:args+=['--cpus',str(c['nanocpus']/1e9)]
|
||||||
|
for k,v in c['labels'].items():
|
||||||
|
if not isinstance(k,str) or not k.startswith('io.athena-deck.') or not isinstance(v,str) or len(v)>512:raise ValueError('Ungültige Labels.')
|
||||||
|
args+=['--label',k+'='+v]
|
||||||
|
for p,bindings in c['ports'].items():
|
||||||
|
for b in bindings:args+=['-p',b['HostIp']+':'+str(b['HostPort'])+':'+p]
|
||||||
|
for value in c['env']:args+=['-e',value]
|
||||||
|
if c['user']:args+=['--user',c['user']]
|
||||||
|
if c['workdir']:args+=['--workdir',c['workdir']]
|
||||||
|
for i,m in enumerate(c['mounts']):
|
||||||
|
source=deck/m['deck_path'] if m['deck_path'] is not None else base/str(i)
|
||||||
|
if source.is_symlink():raise ValueError('Unsicheres Volume-Verzeichnis.')
|
||||||
|
if m['deck_path']:
|
||||||
|
if m['deck_path']=='video/comfy-client-token' and not source.exists():
|
||||||
|
import secrets
|
||||||
|
source.parent.mkdir(parents=True,exist_ok=True);source.write_text(secrets.token_urlsafe(48)+'\n');source.chmod(0o600);os.chown(source,65534,65534)
|
||||||
|
elif m['deck_path']!='video/comfy-client-token':source.mkdir(parents=True,exist_ok=True)
|
||||||
|
elif m['was_file']:
|
||||||
|
if 'file' not in m['files']:raise ValueError('Benötigte Volume-Datei fehlt im Backup.')
|
||||||
|
source.write_bytes(base64.b64decode(m['files']['file']));source.chmod(0o600)
|
||||||
|
else:
|
||||||
|
source.mkdir(exist_ok=True)
|
||||||
|
for filename,data in m['files'].items():
|
||||||
|
target=source/filename;target.write_bytes(base64.b64decode(data));target.chmod(0o600)
|
||||||
|
# Application UIDs commonly used by these rootless interfaces.
|
||||||
|
uid=c.get('uid',int(c['user'].split(':')[0]) if c['user'].split(':')[0].isdigit() else 0)
|
||||||
|
if m['deck_path'] is None:
|
||||||
|
os.chown(source,uid,uid)
|
||||||
|
if source.is_dir():
|
||||||
|
for p in source.iterdir():os.chown(p,uid,uid)
|
||||||
|
args+=['--mount',f'type=bind,src={source},dst={m["target"]}'+(',readonly' if m['read_only'] else '')]
|
||||||
|
if c.get('entrypoint'):
|
||||||
|
args+=['--entrypoint',c['entrypoint'][0]];entry_tail=c['entrypoint'][1:]
|
||||||
|
else:entry_tail=[]
|
||||||
|
args+=[c['image'] if c.get('build_recipe') else c['registry'],*entry_tail,*(c.get('cmd') or [])]
|
||||||
|
command(args)
|
||||||
|
if c['running']:
|
||||||
|
command(['start',name]);time.sleep(1)
|
||||||
|
if command(['inspect','--format','{{.State.Running}}',name])!='true':raise ValueError('Wiederhergestellter Container ist nicht gestartet geblieben.')
|
||||||
|
return {'state':'complete','message':'Image bereit, Konfiguration und Container wiederhergestellt.'}
|
||||||
@@ -143,6 +143,12 @@ class Manager:
|
|||||||
return self.video_status(config)
|
return self.video_status(config)
|
||||||
def dispatch(self,data):
|
def dispatch(self,data):
|
||||||
if not isinstance(data,dict):raise ValueError('Ungültige Helferanfrage.')
|
if not isinstance(data,dict):raise ValueError('Ungültige Helferanfrage.')
|
||||||
|
if data.get('action')=='backup-export' and set(data)=={'action'}:
|
||||||
|
import docker_backup
|
||||||
|
return docker_backup.export(self)
|
||||||
|
if data.get('action')=='backup-restore' and set(data)=={'action','container'}:
|
||||||
|
import docker_backup
|
||||||
|
return docker_backup.restore(self,data['container'])
|
||||||
if set(data)=={'action','service'} and data['action'] in ('video-start','video-stop'):
|
if set(data)=={'action','service'} and data['action'] in ('video-start','video-stop'):
|
||||||
return self.video_action(data['service'],data['action']=='video-start')
|
return self.video_action(data['service'],data['action']=='video-start')
|
||||||
if set(data)!={'action'}:raise ValueError('Ungültige Helferanfrage.')
|
if set(data)!={'action'}:raise ValueError('Ungültige Helferanfrage.')
|
||||||
@@ -197,8 +203,8 @@ class Handler(socketserver.StreamRequestHandler):
|
|||||||
try:
|
try:
|
||||||
_,uid,_=struct.unpack('3i',self.connection.getsockopt(socket.SOL_SOCKET,socket.SO_PEERCRED,12))
|
_,uid,_=struct.unpack('3i',self.connection.getsockopt(socket.SOL_SOCKET,socket.SO_PEERCRED,12))
|
||||||
if uid not in (0,self.server.client_uid):raise ValueError('Client nicht erlaubt.')
|
if uid not in (0,self.server.client_uid):raise ValueError('Client nicht erlaubt.')
|
||||||
raw=self.rfile.readline(4097)
|
raw=self.rfile.readline(67108865)
|
||||||
if len(raw)>4096:raise ValueError('Anfrage zu groß.')
|
if len(raw)>67108864:raise ValueError('Anfrage zu groß.')
|
||||||
data=json.loads(raw)
|
data=json.loads(raw)
|
||||||
if isinstance(data,dict) and data.get('action')=='video-http':
|
if isinstance(data,dict) and data.get('action')=='video-http':
|
||||||
self.connection.settimeout(3600);video_http(self.server.manager,data,self.rfile,self.wfile);return
|
self.connection.settimeout(3600);video_http(self.server.manager,data,self.rfile,self.wfile);return
|
||||||
|
|||||||
+3
-3
@@ -14,7 +14,7 @@ import urllib.request
|
|||||||
|
|
||||||
ROOT = Path(__file__).resolve().parent.parent
|
ROOT = Path(__file__).resolve().parent.parent
|
||||||
LABEL = 'de.casaderoll.athena-deck.standalone'
|
LABEL = 'de.casaderoll.athena-deck.standalone'
|
||||||
FILES = ['deploy/Dockerfile.app-update','video_comfy.py','video_comfy_node.py','video_comfy_proxy.py','prompt_enhancer.py','prompt_enhancer_worker.py','image_upload.py','audio_policy.py','deploy/tts-requirements.lock','stt.py','stt-ui.js','api_compat.py','execution_setup.py','tts_runtime.py','tts_test.py','tts_worker.py','tts-ui.js','auto_test.py','auto-test-ui.js','chat_test.py','chat-test-ui.js','endpoint.py','inference.py','endpoint-ui.js','docker_support.py','docker-ui.js','deploy/docker_helper.py','deploy/setup_docker_helper.py','image_encoder_node.py','image_runtime.py','image_test.py','image-test-ui.js','profiles.py','profiles-ui.js','capacity.py','runtime.py','runtime-ui.js','video.py','video_proxy.py','video-ui.js','catalog.py','hub_auth.py','catalog-ui.js','server.py','auth.py','collect_hardware.py','dashboard_data.py','dashboard_history.py','dashboard-ui.js','dashboard.css','themes.css','deploy/import_dashboard_history.py','index.html','app.js','studio.js','style.css','login.html','login.js','access-ui.js','network-ui.js','network/__init__.py','network/client.py','network/config.py','network/rpc.py','deploy/Dockerfile','deploy/image-requirements.lock']
|
FILES = ['backup.py','backup_codec.py','backup-ui.js','deploy/docker_backup.py','deploy/swarm-ui/Dockerfile','deploy/swarm-ui/patch-source.py','deploy/swarm-ui/proxy.py','deploy/swarm-ui/entrypoint.py','deploy/Dockerfile.app-update','video_comfy.py','video_comfy_node.py','video_comfy_proxy.py','prompt_enhancer.py','prompt_enhancer_worker.py','image_upload.py','audio_policy.py','deploy/tts-requirements.lock','stt.py','stt-ui.js','api_compat.py','execution_setup.py','tts_runtime.py','tts_test.py','tts_worker.py','tts-ui.js','auto_test.py','auto-test-ui.js','chat_test.py','chat-test-ui.js','endpoint.py','inference.py','endpoint-ui.js','docker_support.py','docker-ui.js','deploy/docker_helper.py','deploy/setup_docker_helper.py','image_encoder_node.py','image_runtime.py','image_test.py','image-test-ui.js','profiles.py','profiles-ui.js','capacity.py','runtime.py','runtime-ui.js','video.py','video_proxy.py','video-ui.js','catalog.py','hub_auth.py','catalog-ui.js','server.py','auth.py','collect_hardware.py','dashboard_data.py','dashboard_history.py','dashboard-ui.js','dashboard.css','themes.css','deploy/import_dashboard_history.py','index.html','app.js','studio.js','style.css','login.html','login.js','access-ui.js','network-ui.js','network/__init__.py','network/client.py','network/config.py','network/rpc.py','deploy/Dockerfile','deploy/image-requirements.lock']
|
||||||
|
|
||||||
|
|
||||||
def run(*args, check=True, interactive=False):
|
def run(*args, check=True, interactive=False):
|
||||||
@@ -175,7 +175,7 @@ def install(args):
|
|||||||
(base/sub).mkdir(mode=0o700)
|
(base/sub).mkdir(mode=0o700)
|
||||||
os.chown(base/'state',65534,65534)
|
os.chown(base/'state',65534,65534)
|
||||||
config=dict(owner=LABEL,base=str(base),name=args.name,port=args.port,api_ports=api_ports,image=image,previous_image=None,gpu_telemetry=args.gpu_telemetry,image_runtime=True,docker_helper=True)
|
config=dict(owner=LABEL,base=str(base),name=args.name,port=args.port,api_ports=api_ports,image=image,previous_image=None,gpu_telemetry=args.gpu_telemetry,image_runtime=True,docker_helper=True)
|
||||||
run(sys.executable,str(ROOT/'deploy/setup_docker_helper.py'),'--client-uid','65534','--client-gid','65534')
|
run(sys.executable,str(ROOT/'deploy/setup_docker_helper.py'),'--client-uid','65534','--client-gid','65534','--deck-state',str(base/'state'))
|
||||||
write_manifest(base,config)
|
write_manifest(base,config)
|
||||||
provision(config)
|
provision(config)
|
||||||
try:
|
try:
|
||||||
@@ -255,7 +255,7 @@ def main():
|
|||||||
config=load_manifest(args.directory)
|
config=load_manifest(args.directory)
|
||||||
item=owned(config['name'],args.directory)
|
item=owned(config['name'],args.directory)
|
||||||
if args.docker_helper:
|
if args.docker_helper:
|
||||||
run(sys.executable,str(ROOT/'deploy/setup_docker_helper.py'),'--client-uid','65534','--client-gid','65534')
|
run(sys.executable,str(ROOT/'deploy/setup_docker_helper.py'),'--client-uid','65534','--client-gid','65534','--deck-state',str(args.directory/'state'))
|
||||||
config['docker_helper']=True;write_manifest(args.directory,config);update(args.directory,force=True)
|
config['docker_helper']=True;write_manifest(args.directory,config);update(args.directory,force=True)
|
||||||
elif args.status:
|
elif args.status:
|
||||||
print('Deck: '+('running' if item and item['State']['Running'] else 'stopped / absent'))
|
print('Deck: '+('running' if item and item['State']['Running'] else 'stopped / absent'))
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""Provision Deck's fixed Unix-socket helper; does not install Docker itself."""
|
"""Provision Deck's fixed Unix-socket helper; does not install Docker itself."""
|
||||||
import argparse
|
import argparse
|
||||||
|
import json
|
||||||
import os
|
import os
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
import shutil
|
import shutil
|
||||||
@@ -9,7 +10,7 @@ import subprocess
|
|||||||
MARKER='# Athena Deck Docker helper — managed by setup_docker_helper.py'
|
MARKER='# Athena Deck Docker helper — managed by setup_docker_helper.py'
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
p=argparse.ArgumentParser();p.add_argument('--client-uid',type=int,required=True);p.add_argument('--client-gid',type=int,required=True);p.add_argument('--allow-install',action='store_true',help='Erlaubt Erstinstallation über die authentifizierte Deck-GUI auf einem unbenutzten Debian-Host.');args=p.parse_args()
|
p=argparse.ArgumentParser();p.add_argument('--client-uid',type=int,required=True);p.add_argument('--client-gid',type=int,required=True);p.add_argument('--allow-install',action='store_true',help='Erlaubt Erstinstallation über die authentifizierte Deck-GUI auf einem unbenutzten Debian-Host.');p.add_argument('--deck-state');args=p.parse_args()
|
||||||
if os.geteuid()!=0 or not Path('/run/systemd/system').is_dir():raise SystemExit('Als root auf dem Debian-Zielserver mit systemd ausführen.')
|
if os.geteuid()!=0 or not Path('/run/systemd/system').is_dir():raise SystemExit('Als root auf dem Debian-Zielserver mit systemd ausführen.')
|
||||||
if args.client_uid<1 or args.client_gid<1:raise SystemExit('Eigene unprivilegierte Deck-UID und GID erforderlich.')
|
if args.client_uid<1 or args.client_gid<1:raise SystemExit('Eigene unprivilegierte Deck-UID und GID erforderlich.')
|
||||||
base=Path('/opt/athena-deck-docker-helper');unit=Path('/etc/systemd/system/athena-deck-docker-helper.service')
|
base=Path('/opt/athena-deck-docker-helper');unit=Path('/etc/systemd/system/athena-deck-docker-helper.service')
|
||||||
@@ -20,6 +21,15 @@ def main():
|
|||||||
target=base/'docker_helper.py'
|
target=base/'docker_helper.py'
|
||||||
if target.is_symlink():raise SystemExit('Symlink-Ziel nicht zulässig.')
|
if target.is_symlink():raise SystemExit('Symlink-Ziel nicht zulässig.')
|
||||||
shutil.copyfile(Path(__file__).with_name('docker_helper.py'),target);os.chown(target,0,0);target.chmod(0o644);(base/'managed-by-deck').touch()
|
shutil.copyfile(Path(__file__).with_name('docker_helper.py'),target);os.chown(target,0,0);target.chmod(0o644);(base/'managed-by-deck').touch()
|
||||||
|
shutil.copyfile(Path(__file__).with_name('docker_backup.py'),base/'docker_backup.py')
|
||||||
|
(base/'swarm-ui').mkdir(exist_ok=True)
|
||||||
|
for name in ('Dockerfile','patch-source.py','proxy.py','entrypoint.py'):
|
||||||
|
shutil.copyfile(Path(__file__).parent/'swarm-ui'/name,base/'swarm-ui'/name)
|
||||||
|
if args.deck_state:
|
||||||
|
registered=Path(args.deck_state).resolve()
|
||||||
|
if not registered.is_dir():raise SystemExit('Deck-Zustandsverzeichnis fehlt.')
|
||||||
|
state=Path('/var/lib/athena-deck-docker');state.mkdir(exist_ok=True)
|
||||||
|
config=state/'backup-policy.json';config.write_text(json.dumps({'deck_state':str(registered)}));config.chmod(0o600)
|
||||||
command=f'/usr/bin/python3 {target} --client-uid {args.client_uid} --client-gid {args.client_gid}'+(' --allow-install' if args.allow_install else '')
|
command=f'/usr/bin/python3 {target} --client-uid {args.client_uid} --client-gid {args.client_gid}'+(' --allow-install' if args.allow_install else '')
|
||||||
unit.write_text(MARKER+f'''
|
unit.write_text(MARKER+f'''
|
||||||
[Unit]
|
[Unit]
|
||||||
@@ -38,6 +48,7 @@ NoNewPrivileges=true
|
|||||||
ProtectHome=true
|
ProtectHome=true
|
||||||
PrivateTmp=true
|
PrivateTmp=true
|
||||||
ProtectSystem={'false' if args.allow_install else 'strict'}
|
ProtectSystem={'false' if args.allow_install else 'strict'}
|
||||||
|
ReadWritePaths=-{args.deck_state or '/var/lib/athena-deck'}
|
||||||
[Install]
|
[Install]
|
||||||
WantedBy=multi-user.target
|
WantedBy=multi-user.target
|
||||||
''')
|
''')
|
||||||
@@ -45,5 +56,5 @@ WantedBy=multi-user.target
|
|||||||
subprocess.run(['/usr/bin/systemctl','daemon-reload'],check=True)
|
subprocess.run(['/usr/bin/systemctl','daemon-reload'],check=True)
|
||||||
subprocess.run(['/usr/bin/systemctl','enable','athena-deck-docker-helper.service'],check=True,capture_output=True)
|
subprocess.run(['/usr/bin/systemctl','enable','athena-deck-docker-helper.service'],check=True,capture_output=True)
|
||||||
subprocess.run(['/usr/bin/systemctl','restart','athena-deck-docker-helper.service'],check=True)
|
subprocess.run(['/usr/bin/systemctl','restart','athena-deck-docker-helper.service'],check=True)
|
||||||
print('Deck-Docker-Helfer eingerichtet. Installationsrecht: '+('Erstinstallation erlaubt' if args.allow_install else 'nur lesender Bestand'))
|
print('Deck-Docker-Helfer eingerichtet. Installationsrecht: '+('Erstinstallation erlaubt' if args.allow_install else 'Bestand und begrenzter Backup-Restore; keine Paketinstallation'))
|
||||||
if __name__=='__main__':main()
|
if __name__=='__main__':main()
|
||||||
|
|||||||
+4
-4
@@ -7,12 +7,12 @@ MANAGED_LABEL='io.athena-deck.managed=true'
|
|||||||
ROLE_LABEL='io.athena-deck.role=application'
|
ROLE_LABEL='io.athena-deck.role=application'
|
||||||
|
|
||||||
class DockerSupport:
|
class DockerSupport:
|
||||||
def call(self,action,service=None):
|
def call(self,action,service=None,values=None):
|
||||||
path=os.environ.get('DECK_DOCKER_HELPER_SOCKET','/run/athena-deck-docker/control.sock')
|
path=os.environ.get('DECK_DOCKER_HELPER_SOCKET','/run/athena-deck-docker/control.sock')
|
||||||
with socket.socket(socket.AF_UNIX) as sock:
|
with socket.socket(socket.AF_UNIX) as sock:
|
||||||
sock.settimeout(65);sock.connect(path);sock.sendall(json.dumps({'action':action,**({'service':service} if service is not None else {})}).encode()+b'\n')
|
sock.settimeout(1900);sock.connect(path);sock.sendall(json.dumps({'action':action,**({'service':service} if service is not None else {}),**(values or {})}).encode()+b'\n')
|
||||||
with sock.makefile('rb') as stream:raw=stream.readline(1048577)
|
with sock.makefile('rb') as stream:raw=stream.readline(67108865)
|
||||||
if len(raw)>1048576:raise ValueError('Docker-Helferantwort zu groß.')
|
if len(raw)>67108864:raise ValueError('Docker-Helferantwort zu groß.')
|
||||||
value=json.loads(raw)
|
value=json.loads(raw)
|
||||||
if 'error' in value:raise ValueError(value['error'])
|
if 'error' in value:raise ValueError(value['error'])
|
||||||
return value
|
return value
|
||||||
|
|||||||
@@ -2,8 +2,8 @@
|
|||||||
|
|
||||||
Decks Webserver besitzt keinen Docker-Socket. Ein kleiner, root-eigener
|
Decks Webserver besitzt keinen Docker-Socket. Ein kleiner, root-eigener
|
||||||
Systemhelfer bietet über `/run/athena-deck-docker/control.sock` ausschließlich
|
Systemhelfer bietet über `/run/athena-deck-docker/control.sock` ausschließlich
|
||||||
zwei feste Aktionen: Status/Inventar und Docker-Erstinstallation. Keine freie
|
feste Aktionen für Status/Inventar, Docker-Erstinstallation, registrierte Videodienste
|
||||||
Shell, keine Exec-/Mount-/Container-Erstellungsoperationen. Unix-Dateirechte
|
und begrenzten Konfigurations-Backup/Restore. Keine freie Shell oder Docker-RPC. Unix-Dateirechte
|
||||||
und Linux SO_PEERCRED begrenzen Clients auf root und die eingerichtete Deck-UID.
|
und Linux SO_PEERCRED begrenzen Clients auf root und die eingerichtete Deck-UID.
|
||||||
Die öffentliche Deck-API verlangt eine Admin-Sitzung, POST zusätzlich den
|
Die öffentliche Deck-API verlangt eine Admin-Sitzung, POST zusätzlich den
|
||||||
bestehenden CSRF-Schutz.
|
bestehenden CSRF-Schutz.
|
||||||
@@ -20,9 +20,10 @@ services:
|
|||||||
io.athena-deck.role: "application"
|
io.athena-deck.role: "application"
|
||||||
```
|
```
|
||||||
|
|
||||||
`docker ps -a` wird bereits beim Daemon auf diese Labels gefiltert. Es werden nur
|
`docker ps -a` wird bereits beim Daemon auf diese Labels gefiltert. Die normale Inventaransicht überträgt nur
|
||||||
ID, Name, Image, Zustand und Status übertragen; keine Umgebungsvariablen,
|
ID, Name, Image, Zustand und Status. Der ausdrücklich angeforderte, verschlüsselte Backup-Export
|
||||||
Mounts, Secrets oder Anwendungslogs. Vorhandene Container werden nicht automatisch
|
sichert zusätzlich die Konfigurationen markierter Anwendungen einschließlich Secret-Dateien;
|
||||||
|
Anwendungslogs und Medien bleiben ausgeschlossen. Vorhandene Container werden nicht automatisch
|
||||||
übernommen. Labels sind eine explizite Zuordnung, keine Sicherheitsgrenze gegenüber
|
übernommen. Labels sind eine explizite Zuordnung, keine Sicherheitsgrenze gegenüber
|
||||||
jemandem mit eigenem Docker-Administratorzugriff.
|
jemandem mit eigenem Docker-Administratorzugriff.
|
||||||
|
|
||||||
@@ -103,3 +104,5 @@ Der eigenständige KI-Bereich Video speichert lediglich die aktive Laufzeit-ID.
|
|||||||
Interne Verwaltungs-API: GET `/api/v1/video`, POST `/api/v1/video/service` mit `{ "id": "ltx" }`, POST `/api/v1/video/mode` mit `{ "mode": "video" }` oder `llm`. Bestehender Sitzungsschutz gilt. Start/Stop laufen asynchron und zeigen Phase sowie API-Bereitschaft.
|
Interne Verwaltungs-API: GET `/api/v1/video`, POST `/api/v1/video/service` mit `{ "id": "ltx" }`, POST `/api/v1/video/mode` mit `{ "mode": "video" }` oder `llm`. Bestehender Sitzungsschutz gilt. Start/Stop laufen asynchron und zeigen Phase sowie API-Bereitschaft.
|
||||||
|
|
||||||
Der zusätzliche Transport `video-http` ist nur über das private Unix-Socket zugänglich und an registrierte Dienst-IDs gebunden. Zielhost und Upstream-Zugangsdaten stammen ausschließlich aus der root-eigenen Konfiguration, niemals aus einer Client-Anfrage. Keine Redirect-Verfolgung, kein freier HTTP-Proxy. Payloads werden gestreamt und nicht protokolliert. Der öffentliche Deck-Endpunkt prüft vorher seinen Bearer-Token und den aktiven Videomodus.
|
Der zusätzliche Transport `video-http` ist nur über das private Unix-Socket zugänglich und an registrierte Dienst-IDs gebunden. Zielhost und Upstream-Zugangsdaten stammen ausschließlich aus der root-eigenen Konfiguration, niemals aus einer Client-Anfrage. Keine Redirect-Verfolgung, kein freier HTTP-Proxy. Payloads werden gestreamt und nicht protokolliert. Der öffentliche Deck-Endpunkt prüft vorher seinen Bearer-Token und den aktiven Videomodus.
|
||||||
|
|
||||||
|
Konfigurations-Backup und eingeschränkte Container-Wiederherstellung: [Backup & Restore](../BACKUP.md).
|
||||||
|
|||||||
+1
-1
@@ -1 +1 @@
|
|||||||
<!doctype html><html lang="de"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>Athena Deck</title><link rel="stylesheet" href="/style.css"><link rel="stylesheet" href="/dashboard.css"><link rel="stylesheet" href="/themes.css"><script src="/network-ui.js" defer></script><script src="/access-ui.js" defer></script><script src="/catalog-ui.js" defer></script><script src="/runtime-ui.js" defer></script><script src="/profiles-ui.js" defer></script><script src="/auto-test-ui.js" defer></script><script src="/chat-test-ui.js" defer></script><script src="/image-test-ui.js" defer></script><script src="/docker-ui.js" defer></script><script src="/stt-ui.js" defer></script><script src="/tts-ui.js" defer></script><script src="/studio.js" defer></script><script src="/video-ui.js" defer></script><script src="/endpoint-ui.js" defer></script><script src="/dashboard-ui.js" defer></script><script src="/app.js" defer></script></head><body><aside><a class="brand" href="#dashboard"><span>Α</span> ATHENA <b>DECK</b></a><p class="eyebrow">WORKSPACE / ATHENA</p><nav aria-label="Hauptnavigation"><a href="#dashboard">Dashboard <span>01</span></a><a href="#home">Übersicht <span>02</span></a><a href="#chat">Sprachmodelle / Chat <span>02</span></a><a href="#image">Bildgenerierung <span>03</span></a><p class="nav-heading">AUDIO</p><a class="nav-sub" href="#audio">Text-to-Speech (TTS)</a><a class="nav-sub" href="#stt">Spracherkennung (STT)</a><a class="nav-sub" href="#music">Musik</a><a class="nav-sub" href="#voice">Stimme / Voice</a><a href="#video">Video <span>05</span></a><a href="#services">Weitere Dienste <span>06</span></a><p class="nav-heading">EINSTELLUNGEN</p><a href="#appearance">Darstellung <span>↗</span></a><a href="#runtimes">Laufzeiten <span>↗</span></a><a class="nav-sub" href="#runtime">llama.cpp</a><a class="nav-sub" href="#image-runtime">Bildgenerierung</a><a class="nav-sub" href="#stt-runtime">Spracherkennung</a><a class="nav-sub" href="#tts-runtime">Text-to-Speech</a><a class="nav-sub" href="#docker-runtime">Docker</a><a href="#endpoint">Endpunkt & Profile <span>↗</span></a><a href="#network">Netzwerk <span>↗</span></a><a href="#access">Zugang & API <span>↗</span></a></nav><footer><i></i> Isolierte Entwicklungsumgebung<p>Eigener Modell-Router<br>Hardware · nur lesend</p></footer></aside><main><header><span>ATHENA CONTROL SURFACE</span><span class="pill">v0.7 · Router</span></header><div id="view"></div><p id="error" role="alert"></p></main></body></html>
|
<!doctype html><html lang="de"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>Athena Deck</title><link rel="stylesheet" href="/style.css"><link rel="stylesheet" href="/dashboard.css"><link rel="stylesheet" href="/themes.css"><script src="/network-ui.js" defer></script><script src="/access-ui.js" defer></script><script src="/catalog-ui.js" defer></script><script src="/runtime-ui.js" defer></script><script src="/profiles-ui.js" defer></script><script src="/auto-test-ui.js" defer></script><script src="/chat-test-ui.js" defer></script><script src="/image-test-ui.js" defer></script><script src="/docker-ui.js" defer></script><script src="/stt-ui.js" defer></script><script src="/tts-ui.js" defer></script><script src="/studio.js" defer></script><script src="/video-ui.js" defer></script><script src="/endpoint-ui.js" defer></script><script src="/dashboard-ui.js" defer></script><script src="/backup-ui.js" defer></script><script src="/app.js" defer></script></head><body><aside><a class="brand" href="#dashboard"><span>Α</span> ATHENA <b>DECK</b></a><p class="eyebrow">WORKSPACE / ATHENA</p><nav aria-label="Hauptnavigation"><a href="#dashboard">Dashboard <span>01</span></a><a href="#home">Übersicht <span>02</span></a><a href="#chat">Sprachmodelle / Chat <span>02</span></a><a href="#image">Bildgenerierung <span>03</span></a><p class="nav-heading">AUDIO</p><a class="nav-sub" href="#audio">Text-to-Speech (TTS)</a><a class="nav-sub" href="#stt">Spracherkennung (STT)</a><a class="nav-sub" href="#music">Musik</a><a class="nav-sub" href="#voice">Stimme / Voice</a><a href="#video">Video <span>05</span></a><a href="#services">Weitere Dienste <span>06</span></a><p class="nav-heading">EINSTELLUNGEN</p><a href="#backup">Backup & Restore <span>↗</span></a><a href="#appearance">Darstellung <span>↗</span></a><a href="#runtimes">Laufzeiten <span>↗</span></a><a class="nav-sub" href="#runtime">llama.cpp</a><a class="nav-sub" href="#image-runtime">Bildgenerierung</a><a class="nav-sub" href="#stt-runtime">Spracherkennung</a><a class="nav-sub" href="#tts-runtime">Text-to-Speech</a><a class="nav-sub" href="#docker-runtime">Docker</a><a href="#endpoint">Endpunkt & Profile <span>↗</span></a><a href="#network">Netzwerk <span>↗</span></a><a href="#access">Zugang & API <span>↗</span></a></nav><footer><i></i> Isolierte Entwicklungsumgebung<p>Eigener Modell-Router<br>Hardware · nur lesend</p></footer></aside><main><header><span>ATHENA CONTROL SURFACE</span><span class="pill">v0.7 · Router</span></header><div id="view"></div><p id="error" role="alert"></p></main></body></html>
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
FROM python:3.13-slim-bookworm
|
FROM python:3.13-slim-bookworm
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends wireguard-tools iproute2 && rm -rf /var/lib/apt/lists/*
|
RUN apt-get update && apt-get install -y --no-install-recommends wireguard-tools iproute2 && rm -rf /var/lib/apt/lists/*
|
||||||
|
RUN pip install --no-cache-dir cryptography==50.0.1
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
COPY . /app
|
COPY . /app
|
||||||
ENV PYTHONDONTWRITEBYTECODE=1 DECK_CONTAINER=1
|
ENV PYTHONDONTWRITEBYTECODE=1 DECK_CONTAINER=1
|
||||||
|
|||||||
@@ -198,6 +198,26 @@ class Agent:
|
|||||||
record = validate_record(data.get('credentials'))
|
record = validate_record(data.get('credentials'))
|
||||||
save(DATA/'auth.json', record)
|
save(DATA/'auth.json', record)
|
||||||
return {'saved':True}
|
return {'saved':True}
|
||||||
|
if action == 'backup-export':
|
||||||
|
config = json.loads((DATA/'config.json').read_text()) if (DATA/'config.json').exists() else None
|
||||||
|
return {'config': config, 'enabled': self.address is not None, 'mode': self.policy.mode}
|
||||||
|
if action == 'backup-restore':
|
||||||
|
if self.address or self.policy.mode != 'lan' or self.policy.pending:
|
||||||
|
raise ValueError('Netzwerk-Restore nur im bestätigten LAN-Modus bei deaktiviertem Deck-Tunnel.')
|
||||||
|
value = data.get('backup')
|
||||||
|
if not isinstance(value, dict) or set(value) != {'config','enabled','mode'} or value['mode'] not in ('lan','both','tunnel') or type(value['enabled']) is not bool:
|
||||||
|
raise ValueError('Ungültiger Netzwerk-Backupstand.')
|
||||||
|
if value['config'] is not None:
|
||||||
|
parse_config(value['config']);save(DATA/'config.json', value['config'])
|
||||||
|
if value['enabled']:
|
||||||
|
self.connect();save(DATA/'enabled.json', True)
|
||||||
|
until=time.monotonic()+20
|
||||||
|
while not self.connected()[0] and time.monotonic()<until:time.sleep(.5)
|
||||||
|
if not self.connected()[0]:
|
||||||
|
self.disconnect();save(DATA/'enabled.json',False)
|
||||||
|
raise ValueError('WireGuard-Handshake fehlt; LAN-Zugang bleibt erhalten.')
|
||||||
|
if value['mode'] != 'lan':self.policy.propose(value['mode'], True)
|
||||||
|
return self.status(ingress)
|
||||||
if action == 'status':
|
if action == 'status':
|
||||||
return self.status(ingress)
|
return self.status(ingress)
|
||||||
if action == 'import':
|
if action == 'import':
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ import sys
|
|||||||
|
|
||||||
NAME = 'athena-deck-network'
|
NAME = 'athena-deck-network'
|
||||||
BASE = Path('/opt/athena-deck')
|
BASE = Path('/opt/athena-deck')
|
||||||
FILES = {'deploy/tts-requirements.lock','stt.py','stt-ui.js','api_compat.py','execution_setup.py','tts_runtime.py','tts_test.py','tts_worker.py','tts-ui.js','auto_test.py','auto-test-ui.js','chat_test.py','chat-test-ui.js','endpoint.py','inference.py','endpoint-ui.js','image_runtime.py','image_encoder_node.py','docker_support.py','docker-ui.js','deploy/image-requirements.lock','image_test.py','image-test-ui.js','profiles.py','profiles-ui.js','capacity.py','runtime.py', 'runtime-ui.js', 'video.py','video_proxy.py','video-ui.js','catalog.py','hub_auth.py', 'catalog-ui.js', 'studio.js', 'auth.py', 'access-ui.js', 'server.py', 'collect_hardware.py', 'index.html', 'app.js', 'style.css', 'network-ui.js', 'login.html', 'login.js', 'network/__init__.py', 'network/config.py', 'network/policy.py', 'network/rpc.py', 'network/agent.py', 'network/client.py', 'network/Dockerfile', '.dockerignore'}
|
FILES = {'backup.py','backup_codec.py','backup-ui.js','deploy/docker_backup.py','video_comfy.py','video_comfy_node.py','video_comfy_proxy.py','image_upload.py','prompt_enhancer.py','prompt_enhancer_worker.py','audio_policy.py','dashboard_data.py','dashboard_history.py','dashboard-ui.js','dashboard.css','themes.css','deploy/tts-requirements.lock','stt.py','stt-ui.js','api_compat.py','execution_setup.py','tts_runtime.py','tts_test.py','tts_worker.py','tts-ui.js','auto_test.py','auto-test-ui.js','chat_test.py','chat-test-ui.js','endpoint.py','inference.py','endpoint-ui.js','image_runtime.py','image_encoder_node.py','docker_support.py','docker-ui.js','deploy/image-requirements.lock','image_test.py','image-test-ui.js','profiles.py','profiles-ui.js','capacity.py','runtime.py', 'runtime-ui.js', 'video.py','video_proxy.py','video-ui.js','catalog.py','hub_auth.py', 'catalog-ui.js', 'studio.js', 'auth.py', 'access-ui.js', 'server.py', 'collect_hardware.py', 'index.html', 'app.js', 'style.css', 'network-ui.js', 'login.html', 'login.js', 'network/__init__.py', 'network/config.py', 'network/policy.py', 'network/rpc.py', 'network/agent.py', 'network/client.py', 'network/Dockerfile', '.dockerignore'}
|
||||||
|
|
||||||
def run(*args, **kwargs):
|
def run(*args, **kwargs):
|
||||||
return subprocess.run(args, capture_output=True, timeout=600, **kwargs)
|
return subprocess.run(args, capture_output=True, timeout=600, **kwargs)
|
||||||
|
|||||||
+2
-2
@@ -23,7 +23,7 @@ class PromptEnhancer:
|
|||||||
def status(self):
|
def status(self):
|
||||||
with self.lock:
|
with self.lock:
|
||||||
return dict(models={task:dict(repo=repo,installed=bool(self.installed(task)),revision=(self.installed(task) or {}).get('revision')) for task,repo in PROMPT_ENHANCER_REPOS.items()},job=dict(self.job) if self.job else None,preview=dict(self.preview) if self.preview else None)
|
return dict(models={task:dict(repo=repo,installed=bool(self.installed(task)),revision=(self.installed(task) or {}).get('revision')) for task,repo in PROMPT_ENHANCER_REPOS.items()},job=dict(self.job) if self.job else None,preview=dict(self.preview) if self.preview else None)
|
||||||
def install(self,task):
|
def install(self,task,revision=None):
|
||||||
if task not in PROMPT_ENHANCER_REPOS:raise ValueError('Unbekannter Prompt-Aufwerter.')
|
if task not in PROMPT_ENHANCER_REPOS:raise ValueError('Unbekannter Prompt-Aufwerter.')
|
||||||
with self.lock:
|
with self.lock:
|
||||||
if self.job and self.job['state']=='running':raise ValueError('Ein Aufwerter-Download läuft bereits.')
|
if self.job and self.job['state']=='running':raise ValueError('Ein Aufwerter-Download läuft bereits.')
|
||||||
@@ -31,7 +31,7 @@ class PromptEnhancer:
|
|||||||
if self.installed(task):raise ValueError('Dieser Aufwerter ist bereits installiert.')
|
if self.installed(task):raise ValueError('Dieser Aufwerter ist bereits installiert.')
|
||||||
if not self.runtime.status()['installed']:raise ValueError('Zuerst die Bildlaufzeit einrichten.')
|
if not self.runtime.status()['installed']:raise ValueError('Zuerst die Bildlaufzeit einrichten.')
|
||||||
if shutil.disk_usage(self.root if self.root.exists() else self.root.parent).free<30*GIB:raise ValueError('Mindestens 30 GiB freier Plattenspeicher erforderlich.')
|
if shutil.disk_usage(self.root if self.root.exists() else self.root.parent).free<30*GIB:raise ValueError('Mindestens 30 GiB freier Plattenspeicher erforderlich.')
|
||||||
source=self.catalog.files(PROMPT_ENHANCER_REPOS[task]);revision=source['revision']
|
source=self.catalog.files(PROMPT_ENHANCER_REPOS[task],revision) if revision else self.catalog.files(PROMPT_ENHANCER_REPOS[task]);revision=source['revision']
|
||||||
expected=sum(f['size'] for f in source['files'] if f['name'].endswith('.safetensors'))
|
expected=sum(f['size'] for f in source['files'] if f['name'].endswith('.safetensors'))
|
||||||
if expected<15*GIB or expected>25*GIB:raise ValueError('Unerwartete Größe der offiziellen Aufwerter-Gewichte; Rezept prüfen.')
|
if expected<15*GIB or expected>25*GIB:raise ValueError('Unerwartete Größe der offiziellen Aufwerter-Gewichte; Rezept prüfen.')
|
||||||
self.root.mkdir(parents=True,exist_ok=True,mode=0o700);self.cancel.clear()
|
self.root.mkdir(parents=True,exist_ok=True,mode=0o700);self.cancel.clear()
|
||||||
|
|||||||
@@ -136,6 +136,8 @@ class Server(ThreadingHTTPServer):
|
|||||||
session=self.sessions.get(token)
|
session=self.sessions.get(token)
|
||||||
return bool(record and session and session['expires']>time.monotonic() and secrets.compare_digest(session['revision'],record['password']['hash']))
|
return bool(record and session and session['expires']>time.monotonic() and secrets.compare_digest(session['revision'],record['password']['hash']))
|
||||||
self.endpoint.video_browser_auth=video_browser_auth
|
self.endpoint.video_browser_auth=video_browser_auth
|
||||||
|
from backup import Backup
|
||||||
|
self.backup=Backup(self)
|
||||||
|
|
||||||
class Handler(BaseHTTPRequestHandler):
|
class Handler(BaseHTTPRequestHandler):
|
||||||
def log_message(self, *args):
|
def log_message(self, *args):
|
||||||
@@ -288,7 +290,7 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
return self.respond({'error':'Anmeldung erforderlich.'},401)
|
return self.respond({'error':'Anmeldung erforderlich.'},401)
|
||||||
if not self.authenticated() and self.path == '/':
|
if not self.authenticated() and self.path == '/':
|
||||||
return self.respond((ROOT/'login.html').read_bytes(), mime='text/html; charset=utf-8')
|
return self.respond((ROOT/'login.html').read_bytes(), mime='text/html; charset=utf-8')
|
||||||
routes = {'/dashboard-ui.js':('dashboard-ui.js','text/javascript'),'/dashboard.css':('dashboard.css','text/css'),'/themes.css':('themes.css','text/css'),'/video-ui.js':('video-ui.js','text/javascript'),'/stt-ui.js':('stt-ui.js','text/javascript'),'/tts-ui.js':('tts-ui.js','text/javascript'),'/auto-test-ui.js':('auto-test-ui.js','text/javascript'),'/chat-test-ui.js':('chat-test-ui.js','text/javascript'),'/endpoint-ui.js':('endpoint-ui.js','text/javascript'),'/docker-ui.js': ('docker-ui.js','text/javascript'), '/': ('index.html', 'text/html; charset=utf-8'), '/app.js': ('app.js', 'text/javascript'), '/style.css': ('style.css', 'text/css'), '/network-ui.js': ('network-ui.js', 'text/javascript'), '/access-ui.js': ('access-ui.js', 'text/javascript'), '/studio.js': ('studio.js', 'text/javascript'), '/catalog-ui.js': ('catalog-ui.js','text/javascript'), '/runtime-ui.js': ('runtime-ui.js','text/javascript'), '/profiles-ui.js': ('profiles-ui.js','text/javascript'), '/image-test-ui.js': ('image-test-ui.js','text/javascript')}
|
routes = {'/backup-ui.js':('backup-ui.js','text/javascript'),'/dashboard-ui.js':('dashboard-ui.js','text/javascript'),'/dashboard.css':('dashboard.css','text/css'),'/themes.css':('themes.css','text/css'),'/video-ui.js':('video-ui.js','text/javascript'),'/stt-ui.js':('stt-ui.js','text/javascript'),'/tts-ui.js':('tts-ui.js','text/javascript'),'/auto-test-ui.js':('auto-test-ui.js','text/javascript'),'/chat-test-ui.js':('chat-test-ui.js','text/javascript'),'/endpoint-ui.js':('endpoint-ui.js','text/javascript'),'/docker-ui.js': ('docker-ui.js','text/javascript'), '/': ('index.html', 'text/html; charset=utf-8'), '/app.js': ('app.js', 'text/javascript'), '/style.css': ('style.css', 'text/css'), '/network-ui.js': ('network-ui.js', 'text/javascript'), '/access-ui.js': ('access-ui.js', 'text/javascript'), '/studio.js': ('studio.js', 'text/javascript'), '/catalog-ui.js': ('catalog-ui.js','text/javascript'), '/runtime-ui.js': ('runtime-ui.js','text/javascript'), '/profiles-ui.js': ('profiles-ui.js','text/javascript'), '/image-test-ui.js': ('image-test-ui.js','text/javascript')}
|
||||||
if self.path in routes:
|
if self.path in routes:
|
||||||
name, mime = routes[self.path]
|
name, mime = routes[self.path]
|
||||||
return self.respond((ROOT/name).read_bytes(), mime=mime)
|
return self.respond((ROOT/name).read_bytes(), mime=mime)
|
||||||
@@ -297,6 +299,7 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
public_endpoint={key:endpoint[key] for key in ('state','port','counts','active_requests')}
|
public_endpoint={key:endpoint[key] for key in ('state','port','counts','active_requests')}
|
||||||
return self.respond(dict(name='Athena Deck', version='0.7.0', state='ready', uptime_seconds=round(time.time()-self.server.started), mode='isolated', location=os.environ.get('DECK_LOCATION', 'Athena · Debian-Server'), endpoint=public_endpoint))
|
return self.respond(dict(name='Athena Deck', version='0.7.0', state='ready', uptime_seconds=round(time.time()-self.server.started), mode='isolated', location=os.environ.get('DECK_LOCATION', 'Athena · Debian-Server'), endpoint=public_endpoint))
|
||||||
if self.path == '/api/v1/auto-tests':return self.respond(self.server.auto_tests.status())
|
if self.path == '/api/v1/auto-tests':return self.respond(self.server.auto_tests.status())
|
||||||
|
if self.path == '/api/v1/backup':return self.respond(self.server.backup.status())
|
||||||
if self.path == '/api/v1/chat-tests':return self.respond(self.server.chat_tests.status())
|
if self.path == '/api/v1/chat-tests':return self.respond(self.server.chat_tests.status())
|
||||||
if self.path == '/api/v1/endpoint':return self.respond(self.server.endpoint.status())
|
if self.path == '/api/v1/endpoint':return self.respond(self.server.endpoint.status())
|
||||||
if self.path == '/api/v1/docker':return self.respond(self.server.docker.status())
|
if self.path == '/api/v1/docker':return self.respond(self.server.docker.status())
|
||||||
@@ -396,6 +399,36 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
return self.login()
|
return self.login()
|
||||||
if not self.authenticated():
|
if not self.authenticated():
|
||||||
return self.respond({'error':'Anmeldung oder gültiger API-Token erforderlich.'},401)
|
return self.respond({'error':'Anmeldung oder gültiger API-Token erforderlich.'},401)
|
||||||
|
if self.server.backup.busy() and not self.path.startswith('/api/v1/backup/') and self.path != '/api/v1/logout':
|
||||||
|
return self.respond({'error':'Wiederherstellung läuft. Änderungen und Tests sind vorübergehend gesperrt.'},409)
|
||||||
|
if self.path.startswith('/api/v1/backup/'):
|
||||||
|
try:
|
||||||
|
if not self.authenticated(session_only=True):return self.respond({'error':'Administratorsitzung erforderlich.'},401)
|
||||||
|
action=self.path.rsplit('/',1)[1]
|
||||||
|
if action=='export':
|
||||||
|
data=self.read_json()
|
||||||
|
if set(data)!={'password','theme','history'} or type(data['history']) is not bool:raise ValueError('Backup-Kennwort, Theme und History-Auswahl erforderlich.')
|
||||||
|
body=self.server.backup.export(data['password'],data['theme'],data['history'])
|
||||||
|
self.send_response(200);self.send_header('Content-Type','application/octet-stream');self.send_header('Content-Length',str(len(body)));self.send_header('Content-Disposition','attachment; filename="athena-deck-'+time.strftime('%Y%m%d-%H%M%S')+'.adbackup"');self.send_header('Cache-Control','no-store');self.end_headers();self.wfile.write(body);return
|
||||||
|
if action=='inspect':
|
||||||
|
from backup_codec import MAX
|
||||||
|
self.connection.settimeout(60)
|
||||||
|
length=int(self.headers.get('Content-Length','0'))
|
||||||
|
if self.headers.get('Transfer-Encoding') or self.headers.get('Content-Type')!='application/octet-stream' or not 0<length<=MAX:raise ValueError('Backup-Datei erwartet; maximal 64 MiB.')
|
||||||
|
raw=self.rfile.read(length)
|
||||||
|
if len(raw)!=length:raise ValueError('Backup-Upload unvollständig.')
|
||||||
|
return self.respond(self.server.backup.inspect(raw,__import__('base64').b64decode(self.headers.get('X-Backup-Passphrase',''),validate=True).decode('utf-8')))
|
||||||
|
if action=='checkpoint':
|
||||||
|
data=self.read_json()
|
||||||
|
if set(data)!={'name'}:raise ValueError('Rückfallstand erforderlich.')
|
||||||
|
return self.respond(self.server.backup.inspect_checkpoint(data['name']))
|
||||||
|
if action=='restore':return self.respond(self.server.backup.start(self.read_json()),202)
|
||||||
|
if action=='cancel':
|
||||||
|
if self.read_json():raise ValueError('Keine Parameter erwartet.')
|
||||||
|
return self.respond(self.server.backup.stop())
|
||||||
|
raise ValueError('Unbekannte Backup-Aktion.')
|
||||||
|
except ValueError as exc:return self.respond({'error':str(exc)},400)
|
||||||
|
except Exception:return self.respond({'error':'Backup-Aktion fehlgeschlagen. Dateiformat, freien Speicher und Systemhelfer prüfen.'},503)
|
||||||
if self.path == '/api/v1/audio-policy':
|
if self.path == '/api/v1/audio-policy':
|
||||||
try:return self.respond(self.server.audio_policy.configure(self.read_json()))
|
try:return self.respond(self.server.audio_policy.configure(self.read_json()))
|
||||||
except ValueError as exc:return self.respond({'error':str(exc)},400)
|
except ValueError as exc:return self.respond({'error':str(exc)},400)
|
||||||
@@ -603,6 +636,7 @@ def main():
|
|||||||
try:
|
try:
|
||||||
server.serve_forever()
|
server.serve_forever()
|
||||||
finally:
|
finally:
|
||||||
|
server.backup.stop()
|
||||||
server.dashboard.close()
|
server.dashboard.close()
|
||||||
server.auto_tests.stop()
|
server.auto_tests.stop()
|
||||||
server.chat_tests.stop()
|
server.chat_tests.stop()
|
||||||
|
|||||||
@@ -0,0 +1,92 @@
|
|||||||
|
import copy,hashlib,json,os,tempfile,time,unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import Mock,patch
|
||||||
|
from backup import Backup,validate
|
||||||
|
from backup_codec import seal,open_backup,MAX
|
||||||
|
from deploy.docker_backup import validate as validate_container,restore as restore_container
|
||||||
|
from server import Server
|
||||||
|
from profiles import SCHEMAS,CHAT_GPU_DEFAULTS,CHAT_SAMPLING
|
||||||
|
|
||||||
|
class CodecTests(unittest.TestCase):
|
||||||
|
def test_authenticated_roundtrip_wrong_password_and_tampering(self):
|
||||||
|
doc={'secret':'private-key-and-hf-token','settings':{'a':1}}
|
||||||
|
raw=seal(doc,'sixteen character secret')
|
||||||
|
self.assertNotIn(b'private-key',raw);self.assertEqual(open_backup(raw,'sixteen character secret'),doc)
|
||||||
|
for broken,key in [(raw,'another secure password'),(raw[:-1]+bytes([raw[-1]^1]),'sixteen character secret')]:
|
||||||
|
with self.assertRaisesRegex(ValueError,'falsch|beschädigt'):open_backup(broken,key)
|
||||||
|
self.assertNotEqual(seal(doc,'sixteen character secret'),raw)
|
||||||
|
def test_unbounded_compression_rejected(self):
|
||||||
|
import zlib,secrets
|
||||||
|
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||||
|
from backup_codec import MAGIC
|
||||||
|
salt=secrets.token_bytes(16);nonce=secrets.token_bytes(12);head=MAGIC+salt+nonce
|
||||||
|
key=hashlib.scrypt(b'sixteen character secret',salt=salt,n=32768,r=8,p=1,maxmem=64*1024**2,dklen=32)
|
||||||
|
raw=head+AESGCM(key).encrypt(nonce,zlib.compress(b' '* (MAX+1)),head)
|
||||||
|
with self.assertRaises(ValueError):open_backup(raw,'sixteen character secret')
|
||||||
|
|
||||||
|
class RestoreTests(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.env=patch.dict(os.environ,{'DECK_NETWORK_MODE':'disabled'});self.env.start();self.tmp=tempfile.TemporaryDirectory();self.root=Path(self.tmp.name)
|
||||||
|
self.server=Server(0,state_dir=self.root);self.server.credentials.setup('test administrator password','test_api_token_12345678901234567890')
|
||||||
|
self.server.docker=Mock();self.server.docker.call.return_value={'services':[],'configured':True};self.server.docker.status.return_value={'services':[]}
|
||||||
|
self.disk=patch('backup.shutil.disk_usage',return_value=Mock(free=1024**4));self.disk.start()
|
||||||
|
self.server.runtime.prerequisites=Mock(return_value={'cuda_ready':True,'cpu_ready':True})
|
||||||
|
def tearDown(self):
|
||||||
|
self.server.backup.stop();self.server.endpoint.close();self.server.video.close();self.server.dashboard.close();self.server.server_close();self.tmp.cleanup();self.env.stop();self.disk.stop()
|
||||||
|
def model(self):
|
||||||
|
data=b'model-weights-not-in-backup';repo='test/model';revision='a'*40;filename='model.gguf';ident=hashlib.sha256((repo+revision+filename).encode()).hexdigest();p=self.root/'models'/ident;p.mkdir(parents=True)
|
||||||
|
(p/'model.gguf').write_bytes(data);entry=dict(repo=repo,revision=revision,file=filename,size=len(data),sha256=hashlib.sha256(data).hexdigest(),kind='chat',state='downloaded');(p/'entry.json').write_text(json.dumps(entry));return ident
|
||||||
|
def profile(self,ident):
|
||||||
|
params={k:v[2] for k,v in SCHEMAS['chat'].items()};params.update(CHAT_GPU_DEFAULTS);params.update(CHAT_SAMPLING)
|
||||||
|
self.server.profiles.save(dict(id=None,revision=0,name='Test Model',kind='chat',model_id=ident,parameters=params))
|
||||||
|
def wait(self):
|
||||||
|
for _ in range(100):
|
||||||
|
if not self.server.backup.busy():return self.server.backup.status()['job']
|
||||||
|
time.sleep(.05)
|
||||||
|
self.fail('restore did not finish')
|
||||||
|
def test_export_excludes_weights_media_logs_and_includes_settings_and_history(self):
|
||||||
|
ident=self.model();self.profile(ident)
|
||||||
|
for folder in ('image-tests','tts-tests','llama-worker'):
|
||||||
|
p=self.root/folder;p.mkdir(exist_ok=True);(p/'private.log').write_text('PRIVATE-CHAT-LOG')
|
||||||
|
doc=open_backup(self.server.backup.export('backup secure password'),'backup secure password')
|
||||||
|
self.assertNotIn('PRIVATE-CHAT-LOG',json.dumps(doc));self.assertNotIn('model-weights-not-in-backup',json.dumps(doc));self.assertIn('history',doc);validate(doc)
|
||||||
|
self.assertEqual(doc['models'][0]['id'],ident);self.assertEqual(len(doc['settings']['profiles.json']),1)
|
||||||
|
def test_restore_reuses_verified_models_restores_profiles_and_keeps_login_when_selected(self):
|
||||||
|
ident=self.model();self.profile(ident);raw=self.server.backup.export('backup secure password');self.server.profiles.rows=[];(self.root/'profiles.json').write_text('[]')
|
||||||
|
current=self.server.credentials.read();summary=self.server.backup.inspect(raw,'backup secure password');self.assertFalse(summary['blockers'])
|
||||||
|
self.server.backup.start(dict(id=summary['id'],services=[],confirm=True,restore_credentials=False));job=self.wait()
|
||||||
|
self.assertEqual(job['state'],'complete',job);self.assertEqual(self.server.profiles.rows[0]['name'],'Test Model');self.assertEqual(self.server.credentials.read(),current);self.assertTrue(any(x['state']=='reused' for x in job['items']));self.assertEqual(len(list((self.root/'recovery').glob('before-*.adbackup'))),1)
|
||||||
|
def test_empty_target_downloads_pinned_model_then_restores(self):
|
||||||
|
ident=self.model();self.profile(ident);doc=self.server.backup.snapshot();data=(self.root/'models'/ident/'model.gguf').read_bytes()
|
||||||
|
import shutil
|
||||||
|
shutil.rmtree(self.root/'models'/ident);self.server.profiles.rows=[];(self.root/'profiles.json').write_text('[]')
|
||||||
|
def download(repo,filename,revision,kind):
|
||||||
|
self.assertEqual((repo,filename,revision),('test/model','model.gguf','a'*40));p=self.root/'models'/ident;p.mkdir();(p/'model.gguf').write_bytes(data);m=dict(doc['models'][0]);m.pop('id');(p/'entry.json').write_text(json.dumps(m));self.server.catalog.job={'state':'complete'}
|
||||||
|
with patch.object(self.server.catalog,'start',side_effect=download):
|
||||||
|
summary=self.server.backup.inspect(seal(doc,'backup secure password'),'backup secure password');self.server.backup.start(dict(id=summary['id'],services=[],confirm=True,restore_credentials=True));job=self.wait()
|
||||||
|
self.assertEqual(job['state'],'complete',job);self.assertEqual(self.server.profiles.rows[0]['model_id'],ident)
|
||||||
|
def test_rejects_unsafe_paths_and_bad_references_before_changes(self):
|
||||||
|
ident=self.model();self.profile(ident);doc=self.server.backup.snapshot(include_history=False)
|
||||||
|
for change in (lambda d:d['models'][0].update(file='../secret'),lambda d:d['settings']['profiles.json'][0].update(model_id='f'*64),lambda d:d.update(version=99)):
|
||||||
|
bad=copy.deepcopy(doc);change(bad)
|
||||||
|
with self.assertRaises(ValueError):self.server.backup.inspect(seal(bad,'backup secure password'),'backup secure password')
|
||||||
|
self.assertEqual(len(self.server.profiles.rows),1)
|
||||||
|
def test_wrong_download_hash_marks_failure_without_claiming_complete(self):
|
||||||
|
ident=self.model();doc=self.server.backup.snapshot(include_history=False);(self.root/'models'/ident/'model.gguf').write_bytes(b'x'*len(b'model-weights-not-in-backup'))
|
||||||
|
summary=self.server.backup.inspect(seal(doc,'backup secure password'),'backup secure password');self.server.backup.start(dict(id=summary['id'],services=[],confirm=True,restore_credentials=False));job=self.wait();self.assertEqual(job['state'],'failed');self.assertNotEqual(job['phase'],'Wiederherstellung abgeschlossen. Oberfläche ist bereit.')
|
||||||
|
|
||||||
|
class ContainerTests(unittest.TestCase):
|
||||||
|
def config(self):return dict(name='test-ui',image='example/ui:1',registry='registry.example/ui@sha256:'+'a'*64,labels={'io.athena-deck.managed':'true','io.athena-deck.role':'application'},unsupported=False,network='bridge',ports={'8000/tcp':[{'HostIp':'127.0.0.1','HostPort':'8128'}]},restart='unless-stopped',env=[],entrypoint=None,cmd=None,user='65534',workdir='',memory=0,nanocpus=0,mounts=[],running=True)
|
||||||
|
def test_rejects_privileged_host_socket_public_ports_and_unknown_host_network(self):
|
||||||
|
c=self.config();validate_container(c)
|
||||||
|
for change in (lambda x:x.update(unsupported=True),lambda x:x.update(network='host'),lambda x:x['ports']['8000/tcp'][0].update(HostIp='0.0.0.0'),lambda x:x.update(mounts=[dict(index=0,target='/var/run/docker.sock',deck_path=None,read_only=True,was_file=True,files={})])):
|
||||||
|
bad=copy.deepcopy(c);change(bad)
|
||||||
|
with self.assertRaises(ValueError):validate_container(bad)
|
||||||
|
def test_never_replaces_existing_foreign_container(self):
|
||||||
|
with tempfile.TemporaryDirectory() as d:
|
||||||
|
manager=Mock(state=Path(d));(Path(d)/'backup-policy.json').write_text(json.dumps({'deck_state':d}))
|
||||||
|
x={'Config':{'Labels':{},'Image':'example/ui:1'}}
|
||||||
|
with patch('deploy.docker_backup.os.geteuid',return_value=0),patch('deploy.docker_backup.subprocess.run',return_value=Mock(returncode=0,stdout=json.dumps([x]))):
|
||||||
|
with self.assertRaises(ValueError):restore_container(manager,self.config())
|
||||||
|
|
||||||
|
if __name__=='__main__':unittest.main()
|
||||||
Reference in New Issue
Block a user