Add isolated YuE2 playground

This commit is contained in:
Mikei386
2026-09-10 21:49:16 +02:00
parent 1904f2104f
commit f8b1b19d4a
6 changed files with 350 additions and 1 deletions
+1 -1
View File
@@ -76,7 +76,7 @@ Titelgenerierung und Kontextkompression in Hermes.
|---|---|---|---|---|---|
| 08.–10.09.2026 | `ACE-Step/acestep-v15-xl-sft` mit `acestep-5Hz-lm-1.7B`, offizielles ACE-Step-1.5-Image `sha256:95652cd780c78a1b1a7f6f0335530430f0ae53d96c7c12d59f9f39fa23d38567` | Mehrere Instrumentaltests bis 244 s; abschließender Kontrolllauf mit geladenem 1.7B-Planer, `thinking=True`, XL-SFT 4B, 80 Schritten, Guidance 8 und Shift 3 | technisch vollständig und schnell, aber wiederholt nur Geräusche/Krach oder musikalisch chaotische Ergebnisse; der letzte Lauf schließt einen bloß fehlenden Planer als Ursache aus | **qualitativ verworfen**; nicht als Qualitätslösung weiterverfolgen | Athena: `/data/music/acestep/`; [SPECIALIZED_MODEL_ROADMAP.md](SPECIALIZED_MODEL_ROADMAP.md) |
| 10.09.2026 | `HeartMuLa/HeartMuLa-oss-3B-happy-new-year` mit `HeartMuLa/HeartCodec-oss-20260123`, heartlib `3783bdb8441f2c298b1e64c8651173aac200361c` | Mehrere Instrumentalversuche mit offiziellen Samplingwerten; HeartMuLa BF16 auf RTX 5080, HeartCodec FP32 auf RTX 3060 | technisch stabil und schnell, klanglich jedoch fast so unbrauchbar wie ACE-Step: Fantasiesprache trotz Instrumentalwunsch und gravierende Missachtung der Synthwave-/Synthpop-Stilvorgabe. Die lokale Test-UI hatte zusätzlich einen nicht upstream dokumentierten `[Instrumental]`-Marker verwendet | **qualitativ verworfen und vollständig entfernt**; Container, Image, Gewichte und Ausgaben am 10.09.2026 gelöscht | diese Tabelle; keine Laufzeitreste auf Athena |
| 10.09.2026 | `m-a-p/YuE2-3B` mit `m-a-p/YuE2-Vae`, offizieller Release `yue2-v0.1.6` | Isolierter Kontrolltest vorbereitet: leere Lyrics, expliziter Instrumentalstil und vollständige symbolische Melodie-/Akkordplanung | Noch nicht gestartet. Gewichte und VAE sind vollständig geladen und anhand der veröffentlichten SHA-256-Manifeste geprüft; das gepinnte Test-Image und die manuelle Startdefinition sind fertig. Kein Container und keine GPU-Last durch YuE2 | **startbereit / Hörtest ausstehend**; Start nur nach ausdrücklicher Freigabe, keine Router- oder Dashboard-Integration | [yue2-3b](../experiments/yue2-3b/README.md) |
| 10.09.2026 | `m-a-p/YuE2-3B` mit `m-a-p/YuE2-Vae`, offizieller Release `yue2-v0.1.6` | Unquantisiertes BF16 ausschließlich auf RTX 5080; leere Lyrics, expliziter Instrumentalstil, 118 BPM, Seed 831001 und vollständige symbolische Planung | 189,5 s Musik in 83,8 s erzeugt; 48 kHz, Stereo, 24-Bit-FLAC, keine Kürzung und kein OOM. Der erste Hörtest war im deutlichen Gegensatz zu ACE-Step und HeartMuLa musikalisch überzeugend. Entstehung über neuen ABC-Plan, 4.739 semantische Tokens und neue akustische Latents verifiziert; keine mitgelieferte Demo-Datei | **technischer Test und erste Hörabnahme bestanden**; isolierter Playground bereit, breitere Stil-/Gesangsprüfung und spätere Routerentscheidung noch offen | [yue2-3b](../experiments/yue2-3b/README.md); Athena: `/data/music/yue2/instrumental_synthwave_control/` |
## Audio-Trennung
+2
View File
@@ -10,6 +10,8 @@ RUN git clone https://github.com/multimodal-art-projection/YuE.git /opt/yue2 \
&& git -C /opt/yue2 checkout "${YUE2_COMMIT}" \
&& python -m pip install --no-cache-dir /opt/yue2
COPY ui /opt/yue2-playground
WORKDIR /workspace
ENV PYTHONUNBUFFERED=1 \
YUE2_KIT=/workspace
+13
View File
@@ -30,6 +30,19 @@ image is built, but no YuE2 container has been created or started.
The initial control request is a true empty-lyrics instrumental request. No
invented `[Instrumental]` lyrics marker is used.
## Temporary playground
The optional `playground` profile provides a small German web interface for
new-song generation. It lists persistent results, streams FLAC files with HTTP
range support and permits explicit deletion. It is bound only to Athena's
localhost on port 8014 and is not integrated into the profile router yet. The
web process itself does not load the model; a click on **Song generieren**
starts one exclusive offline generation job.
```sh
docker compose --profile playground up -d yue2-ui
```
## Manual test (only after GPU availability was checked)
From `/opt/mike-ai/yue2-3b` on Athena:
+34
View File
@@ -21,3 +21,37 @@ services:
- driver: nvidia
device_ids: ["GPU-8ad38c6c-5a01-9d8e-1dfa-ed662ad78fbe"]
capabilities: [gpu]
yue2-ui:
profiles: ["playground"]
build:
context: .
args:
YUE2_COMMIT: 9c6c4b349be978b06a9d0d958471a07a6cdeff4d
image: mike-ai/yue2:3b-0.1.6
container_name: mike-ai-yue2-playground
restart: "no"
entrypoint: ["python", "/opt/yue2-playground/server.py"]
environment:
NVIDIA_VISIBLE_DEVICES: "GPU-8ad38c6c-5a01-9d8e-1dfa-ed662ad78fbe"
NVIDIA_DRIVER_CAPABILITIES: compute,utility
YUE2_UI_HOST: "0.0.0.0"
YUE2_UI_PORT: "8014"
ports:
- "127.0.0.1:8014:8014"
volumes:
- /data/models/yue2:/workspace/models:ro
- /data/music/yue2:/workspace/runs
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: ["GPU-8ad38c6c-5a01-9d8e-1dfa-ed662ad78fbe"]
capabilities: [gpu]
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8014/health', timeout=2)"]
interval: 5s
timeout: 3s
retries: 12
start_period: 5s
+45
View File
@@ -0,0 +1,45 @@
<!doctype html>
<html lang="de">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>YuE2 Playground</title>
<style>
:root{color-scheme:dark;--bg:#090d14;--panel:#121925;--line:#293344;--text:#eef3fb;--muted:#98a6b9;--blue:#58a6ff;--green:#42d392;--red:#ff6b7a}
*{box-sizing:border-box} body{margin:0;background:radial-gradient(circle at 20% 0,#16243b 0,transparent 35%),var(--bg);font:15px/1.5 system-ui,sans-serif;color:var(--text)}
main{max-width:1100px;margin:auto;padding:36px 22px 80px} h1{font-size:36px;margin:0} h2{margin:0 0 16px}.lead{color:var(--muted);margin:4px 0 28px}
.grid{display:grid;grid-template-columns:1.15fr .85fr;gap:22px}@media(max-width:800px){.grid{grid-template-columns:1fr}}
.card{background:rgba(18,25,37,.94);border:1px solid var(--line);border-radius:16px;padding:22px;box-shadow:0 18px 60px #0005}
label{display:block;font-weight:650;margin:14px 0 6px}textarea,input,select{width:100%;background:#090e17;border:1px solid #344157;border-radius:9px;color:var(--text);padding:11px;font:inherit}textarea{resize:vertical;min-height:120px}
.row{display:grid;grid-template-columns:1fr 1fr;gap:12px}.check{display:flex;align-items:center;gap:10px;margin:14px 0}.check input{width:auto}
button{border:0;border-radius:10px;padding:12px 16px;font-weight:750;cursor:pointer;background:var(--blue);color:#05101d}button:disabled{opacity:.45;cursor:not-allowed}.ghost{background:#242e3d;color:var(--text);padding:7px 10px}
.notice{padding:12px;border-radius:9px;background:#0c2630;color:#a9edda;margin-top:14px}.error{background:#371a23;color:#ffc0c7}.job{border-top:1px solid var(--line);padding:15px 0}.job:first-child{border-top:0;padding-top:0}.meta{color:var(--muted);font-size:13px}.state{font-weight:750;color:var(--green)}.state.failed{color:var(--red)}audio{width:100%;margin-top:10px}.job-head{display:flex;justify-content:space-between;gap:12px}.style{white-space:pre-wrap;margin:5px 0}.empty{color:var(--muted)}
</style>
</head>
<body><main>
<h1>YuE2 Playground 🎵</h1><p class="lead">Lokale Musikgenerierung auf Athena · 48 kHz Stereo · RTX 5080</p>
<div class="grid">
<section class="card"><h2>Neuen Song erzeugen</h2>
<form id="form">
<label for="style">Stil und musikalische Vorgaben</label>
<textarea id="style" required>Instrumental, synthwave, synth-pop, energetic, memorable lead melody, arpeggiated synthesizer, analog synthesizer, drum machine, driving bass, 118 BPM, no vocals</textarea>
<label class="check"><input id="instrumental" type="checkbox" checked> Instrumental – ohne Gesang</label>
<div id="lyrics-wrap" hidden><label for="lyrics">Liedtext</label><textarea id="lyrics" placeholder="[Verse]\n...\n\n[Chorus]\n..."></textarea></div>
<div class="row"><div><label for="seed">Seed (leer = Zufall)</label><input id="seed" type="number" min="0" max="4294967295" placeholder="zufällig"></div>
<div><label for="cot">Kompositionsplanung</label><select id="cot"><option value="full">Vollständig – Melodie und Akkorde</option><option value="off">Direkt – ohne editierbaren Plan</option></select></div></div>
<div id="message" class="notice" hidden></div>
<button id="submit" type="submit" style="margin-top:18px;width:100%">Song generieren</button>
</form>
</section>
<section class="card"><h2>Ergebnisse</h2><div id="jobs" class="empty">Wird geladen …</div></section>
</div>
</main><script>
const $=s=>document.querySelector(s), form=$('#form'), inst=$('#instrumental'), lyricsWrap=$('#lyrics-wrap'), submit=$('#submit'), msg=$('#message'), jobs=$('#jobs');
inst.onchange=()=>lyricsWrap.hidden=inst.checked;
function esc(v){return String(v??'').replace(/[&<>"']/g,c=>({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;',"'":'&#39;'}[c]))}
function duration(v){if(!v)return '';const m=Math.floor(v/60),s=Math.round(v%60);return `${m}:${String(s).padStart(2,'0')} min`}
async function refresh(){try{const r=await fetch('/api/jobs',{cache:'no-store'}),d=await r.json();submit.disabled=!!d.active;submit.textContent=d.active?'YuE2 arbeitet …':'Song generieren';jobs.className='';jobs.innerHTML=d.jobs.length?d.jobs.map(j=>`<article class="job"><div class="job-head"><span class="state ${j.state}">${j.state==='running'?'Wird erzeugt …':j.state==='complete'?'Fertig':'Fehlgeschlagen'}</span>${j.state!=='running'?`<button class="ghost" onclick="removeJob('${esc(j.id)}')">Löschen</button>`:''}</div><div class="style">${esc(j.style)}</div><div class="meta">Seed ${esc(j.seed)} · ${j.instrumental?'Instrumental':'Gesang'}${j.audio_seconds?` · ${duration(j.audio_seconds)}`:''}${j.elapsed_seconds?` · erzeugt in ${duration(j.elapsed_seconds)}`:''}</div>${j.audio_url?`<audio controls preload="metadata" src="${j.audio_url}"></audio><p><a href="${j.audio_url}" download="${esc(j.id)}.flac">FLAC herunterladen</a></p>`:''}${j.error?`<div class="notice error">${esc(j.error)}<pre>${esc(j.log||'')}</pre></div>`:''}</article>`).join(''):'<p class="empty">Noch keine Songs vorhanden.</p>'}catch(e){jobs.innerHTML=`<div class="notice error">${esc(e)}</div>`}}
async function removeJob(id){if(!confirm('Diesen Song und alle Zwischenartefakte wirklich löschen?'))return;await fetch('/api/jobs/'+encodeURIComponent(id),{method:'DELETE'});refresh()}
form.onsubmit=async e=>{e.preventDefault();msg.hidden=true;submit.disabled=true;try{const payload={style:$('#style').value,lyrics:$('#lyrics').value,instrumental:inst.checked,seed:$('#seed').value,cot:$('#cot').value};const r=await fetch('/api/jobs',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(payload)}),d=await r.json();if(!r.ok)throw new Error(d.error||'Start fehlgeschlagen');msg.className='notice';msg.textContent=`Auftrag ${d.id} gestartet. Die Seite aktualisiert sich automatisch.`;msg.hidden=false;refresh()}catch(e){msg.className='notice error';msg.textContent=e.message;msg.hidden=false;submit.disabled=false}};
refresh();setInterval(refresh,2000);
</script></body></html>
+255
View File
@@ -0,0 +1,255 @@
#!/usr/bin/env python3
from __future__ import annotations
import json
import mimetypes
import os
import random
import re
import shutil
import subprocess
import threading
import time
from http import HTTPStatus
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import unquote, urlparse
HOST = os.getenv("YUE2_UI_HOST", "0.0.0.0")
PORT = int(os.getenv("YUE2_UI_PORT", "8014"))
ROOT = Path("/workspace/runs")
REQUESTS = ROOT / ".playground_requests"
LOGS = ROOT / ".playground_logs"
INDEX = Path(__file__).with_name("index.html")
SAFE_ID = re.compile(r"^[a-z0-9][a-z0-9_-]{0,63}$")
LOCK = threading.Lock()
JOBS: dict[str, dict] = {}
ACTIVE: str | None = None
def now_ms() -> int:
return int(time.time() * 1000)
def existing_jobs() -> list[dict]:
found: list[dict] = []
for result_file in ROOT.glob("*/result.json"):
try:
result = json.loads(result_file.read_text(encoding="utf-8"))
request_file = result_file.parent / "request.json"
request = json.loads(request_file.read_text(encoding="utf-8"))
found.append({
"id": result_file.parent.name,
"state": "complete",
"style": request.get("style", ""),
"seed": request.get("seed"),
"instrumental": not bool(request.get("lyrics")),
"audio_seconds": result.get("audio_seconds"),
"elapsed_seconds": (result.get("timing") or {}).get("e2e_seconds"),
"audio_url": f"/audio/{result_file.parent.name}",
"created": int(result_file.stat().st_mtime * 1000),
})
except (OSError, ValueError, TypeError):
continue
return sorted(found, key=lambda item: item["created"], reverse=True)
def snapshot() -> dict:
with LOCK:
live = [dict(item) for item in JOBS.values()]
active = ACTIVE
known = {item["id"] for item in live}
live.extend(item for item in existing_jobs() if item["id"] not in known)
return {"active": active, "jobs": sorted(live, key=lambda item: item["created"], reverse=True)}
def run_job(job_id: str, request: dict) -> None:
global ACTIVE
output = ROOT / job_id
request_file = REQUESTS / f"{job_id}.json"
log_file = LOGS / f"{job_id}.log"
command = [
"yue2", "generate", "--offline", "--device", "cuda:0", "--budget", "16",
"--request", str(request_file), "--output", str(output),
]
started = time.monotonic()
try:
with log_file.open("w", encoding="utf-8") as log:
process = subprocess.Popen(command, stdout=log, stderr=subprocess.STDOUT, text=True)
with LOCK:
JOBS[job_id]["pid"] = process.pid
code = process.wait()
if code != 0:
raise RuntimeError(f"YuE2 wurde mit Exit-Code {code} beendet")
result = json.loads((output / "result.json").read_text(encoding="utf-8"))
update = {
"state": "complete",
"audio_seconds": result.get("audio_seconds"),
"elapsed_seconds": round(time.monotonic() - started, 1),
"audio_url": f"/audio/{job_id}",
}
except Exception as exc:
tail = ""
try:
tail = "\n".join(log_file.read_text(encoding="utf-8", errors="replace").splitlines()[-30:])
except OSError:
pass
update = {"state": "failed", "error": str(exc), "log": tail,
"elapsed_seconds": round(time.monotonic() - started, 1)}
with LOCK:
JOBS[job_id].update(update)
ACTIVE = None
class Handler(BaseHTTPRequestHandler):
server_version = "YuE2Playground/1.0"
def log_message(self, fmt: str, *args: object) -> None:
print(f"{self.address_string()} - {fmt % args}", flush=True)
def json_response(self, status: int, payload: object) -> None:
body = json.dumps(payload, ensure_ascii=False).encode("utf-8")
self.send_response(status)
self.send_header("Content-Type", "application/json; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
self.send_header("Cache-Control", "no-store")
self.end_headers()
self.wfile.write(body)
def do_GET(self) -> None: # noqa: N802
path = urlparse(self.path).path
if path == "/health":
self.json_response(200, {"status": "ok", "active": ACTIVE})
return
if path == "/api/jobs":
self.json_response(200, snapshot())
return
if path.startswith("/audio/"):
self.send_audio(unquote(path.removeprefix("/audio/")))
return
if path in {"/", "/index.html"}:
body = INDEX.read_bytes()
self.send_response(200)
self.send_header("Content-Type", "text/html; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
return
self.send_error(404)
def do_POST(self) -> None: # noqa: N802
global ACTIVE
if urlparse(self.path).path != "/api/jobs":
self.send_error(404)
return
try:
length = int(self.headers.get("Content-Length", "0"))
if length <= 0 or length > 65536:
raise ValueError("Ungültige Anfragegröße")
data = json.loads(self.rfile.read(length))
style = str(data.get("style", "")).strip()
lyrics = str(data.get("lyrics", "")).strip()
instrumental = bool(data.get("instrumental", False))
cot = str(data.get("cot", "full"))
if not style or len(style) > 3000:
raise ValueError("Bitte eine Stilbeschreibung mit höchstens 3000 Zeichen eingeben")
if len(lyrics) > 20000:
raise ValueError("Der Liedtext ist zu lang")
if cot not in {"full", "off"}:
raise ValueError("Unbekannter Planungsmodus")
if not instrumental and not lyrics:
raise ValueError("Für einen Song mit Gesang fehlt der Liedtext")
if instrumental:
lyrics = ""
if "instrumental" not in style.casefold():
style = "Instrumental, no vocals, " + style
raw_seed = data.get("seed")
seed = int(raw_seed) if str(raw_seed).strip() else random.SystemRandom().randrange(1, 2**31)
if not 0 <= seed < 2**32:
raise ValueError("Seed muss zwischen 0 und 4294967295 liegen")
except (ValueError, TypeError, json.JSONDecodeError) as exc:
self.json_response(400, {"error": str(exc)})
return
with LOCK:
if ACTIVE is not None:
self.json_response(409, {"error": f"Auftrag {ACTIVE} läuft bereits"})
return
job_id = time.strftime("song-%Y%m%d-%H%M%S") + f"-{seed % 10000:04d}"
request = {"id": job_id, "style": style, "lyrics": lyrics, "cot": cot, "seed": seed}
REQUESTS.mkdir(parents=True, exist_ok=True)
LOGS.mkdir(parents=True, exist_ok=True)
request_file = REQUESTS / f"{job_id}.json"
request_file.write_text(json.dumps(request, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
JOBS[job_id] = {"id": job_id, "state": "running", "style": style,
"seed": seed, "instrumental": instrumental,
"created": now_ms(), "elapsed_seconds": 0}
ACTIVE = job_id
threading.Thread(target=run_job, args=(job_id, request), daemon=True).start()
self.json_response(HTTPStatus.ACCEPTED, JOBS[job_id])
def do_DELETE(self) -> None: # noqa: N802
path = urlparse(self.path).path
job_id = unquote(path.removeprefix("/api/jobs/"))
if not path.startswith("/api/jobs/") or not SAFE_ID.fullmatch(job_id):
self.send_error(404)
return
with LOCK:
if ACTIVE == job_id:
self.json_response(409, {"error": "Ein laufender Auftrag kann nicht gelöscht werden"})
return
JOBS.pop(job_id, None)
shutil.rmtree(ROOT / job_id, ignore_errors=True)
for directory, suffix in ((REQUESTS, ".json"), (LOGS, ".log")):
try:
(directory / f"{job_id}{suffix}").unlink()
except FileNotFoundError:
pass
self.json_response(200, {"status": "deleted", "id": job_id})
def send_audio(self, job_id: str) -> None:
if not SAFE_ID.fullmatch(job_id):
self.send_error(404)
return
path = ROOT / job_id / "audio.flac"
if not path.is_file():
self.send_error(404)
return
size = path.stat().st_size
start, end = 0, size - 1
status = 200
range_header = self.headers.get("Range", "")
if range_header.startswith("bytes="):
try:
left, right = range_header[6:].split("-", 1)
start = int(left) if left else 0
end = min(int(right), size - 1) if right else size - 1
if start < 0 or start > end:
raise ValueError
status = 206
except ValueError:
self.send_error(416)
return
self.send_response(status)
self.send_header("Content-Type", mimetypes.guess_type(path.name)[0] or "audio/flac")
self.send_header("Accept-Ranges", "bytes")
self.send_header("Content-Length", str(end - start + 1))
if status == 206:
self.send_header("Content-Range", f"bytes {start}-{end}/{size}")
self.end_headers()
with path.open("rb") as source:
source.seek(start)
remaining = end - start + 1
while remaining:
chunk = source.read(min(1024 * 1024, remaining))
if not chunk:
break
self.wfile.write(chunk)
remaining -= len(chunk)
if __name__ == "__main__":
ROOT.mkdir(parents=True, exist_ok=True)
print(f"YuE2 Playground listening on {HOST}:{PORT}", flush=True)
ThreadingHTTPServer((HOST, PORT), Handler).serve_forever()