Add isolated HeartMuLa 3B quality test
This commit is contained in:
Mikei386
@@ -74,7 +74,8 @@ Titelgenerierung und Kontextkompression in Hermes.
|
|||||||
|
|
||||||
| Datum | Modell | Test | Ergebnis | Status / Entscheidung | Beleg |
|
| Datum | Modell | Test | Ergebnis | Status / Entscheidung | Beleg |
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
| 08.09.2026 | `ACE-Step/acestep-v15-xl-sft` mit `acestep-5Hz-lm-1.7B`, offizielles ACE-Step-1.5-Image `sha256:95652cd780c78a1b1a7f6f0335530430f0ae53d96c7c12d59f9f39fa23d38567` | 30 s Instrumental, Thinking/LM aktiv, Batch 1, RTX 5080 16 GiB, automatischer CPU-Offload und INT8 Weight-only DiT | erfolgreich in 15,39 s: LM 8,00 s, DiT 7,39 s, MP3 0,82 s; PyTorch meldete maximal 9,38 GiB CUDA-Allokation; kein OOM/CUDA-Fehler | **Beta-Test bestanden**; Klangabnahme und Hermes-/Router-Integration noch offen | Athena: `/data/music/acestep/batch_1788873234/`; [SPECIALIZED_MODEL_ROADMAP.md](SPECIALIZED_MODEL_ROADMAP.md) |
|
| 08.–10.09.2026 | `ACE-Step/acestep-v15-xl-sft` mit `acestep-5Hz-lm-1.7B`, offizielles ACE-Step-1.5-Image `sha256:95652cd780c78a1b1a7f6f0335530430f0ae53d96c7c12d59f9f39fa23d38567` | Mehrere Instrumentaltests bis 244 s; abschließender Kontrolllauf mit geladenem 1.7B-Planer, `thinking=True`, XL-SFT 4B, 80 Schritten, Guidance 8 und Shift 3 | technisch vollständig und schnell, aber wiederholt nur Geräusche/Krach oder musikalisch chaotische Ergebnisse; der letzte Lauf schließt einen bloß fehlenden Planer als Ursache aus | **qualitativ verworfen**; Container und Daten zunächst nur für den direkten HeartMuLa-A/B-Rückweg erhalten | Athena: `/data/music/acestep/`; [SPECIALIZED_MODEL_ROADMAP.md](SPECIALIZED_MODEL_ROADMAP.md) |
|
||||||
|
| 10.09.2026 | `HeartMuLa/HeartMuLa-oss-3B-happy-new-year` mit `HeartMuLa/HeartCodec-oss-20260123`, heartlib `3783bdb8441f2c298b1e64c8651173aac200361c` | Isolierter 20-s-Instrumentaltest mit offiziellen Standardwerten; HeartMuLa BF16 auf RTX 5080, HeartCodec FP32 auf RTX 3060 | Modell und Codec laden stabil; 20,08-s-Stereoausgabe erfolgreich in rund 27 s erzeugt, 48 kHz/24-Bit-PCM. Referenzaudio wird upstream noch nicht unterstützt, Instrumentalsteuerung ist experimentell | **technischer Test bestanden, Hörabnahme offen** | [heartmula-3b](../experiments/heartmula-3b/README.md); Athena: `/data/music/heartmula/heartmula-1789064239-b3f2ca75.wav` |
|
||||||
|
|
||||||
## Audio-Trennung
|
## Audio-Trennung
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,22 @@
|
|||||||
|
FROM pytorch/pytorch:2.9.1-cuda12.8-cudnn9-runtime
|
||||||
|
|
||||||
|
ARG HEARTLIB_COMMIT=3783bdb8441f2c298b1e64c8651173aac200361c
|
||||||
|
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends git ffmpeg curl \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
RUN git clone https://github.com/HeartMuLa/heartlib.git /opt/heartlib \
|
||||||
|
&& git -C /opt/heartlib checkout "${HEARTLIB_COMMIT}" \
|
||||||
|
&& pip install --no-cache-dir -e /opt/heartlib \
|
||||||
|
&& pip install --no-cache-dir gradio==5.49.1 uvicorn==0.35.0
|
||||||
|
|
||||||
|
COPY app.py /opt/app/app.py
|
||||||
|
|
||||||
|
WORKDIR /opt/app
|
||||||
|
ENV PYTHONUNBUFFERED=1 \
|
||||||
|
HEARTMULA_MODEL_PATH=/models/ckpt \
|
||||||
|
HEARTMULA_OUTPUT_DIR=/output
|
||||||
|
|
||||||
|
EXPOSE 7860
|
||||||
|
CMD ["python", "/opt/app/app.py"]
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
# HeartMuLa 3B quality test
|
||||||
|
|
||||||
|
Reversible A/B test of the public
|
||||||
|
`HeartMuLa/HeartMuLa-oss-3B-happy-new-year` model with
|
||||||
|
`HeartMuLa/HeartCodec-oss-20260123`.
|
||||||
|
|
||||||
|
The model and codec are deliberately split across Athena's GPUs:
|
||||||
|
|
||||||
|
- HeartMuLa BF16: RTX 5080 (`cuda:0` inside the container)
|
||||||
|
- HeartCodec FP32: RTX 3060 (`cuda:1` inside the container)
|
||||||
|
|
||||||
|
The public model does not yet support reference-audio conditioning. Pure
|
||||||
|
instrumental generation is exposed as an explicitly experimental option.
|
||||||
|
Generated files are lossless 48-kHz WAV files under `/data/music/heartmula`.
|
||||||
|
|
||||||
|
The container intentionally has no `com.mike-ai.music-worker` label while it
|
||||||
|
is being evaluated. This prevents it from conflicting with the production
|
||||||
|
profile controller's single ACE-Step worker.
|
||||||
|
|
||||||
|
## Checkpoints
|
||||||
|
|
||||||
|
The expected persistent layout on Athena is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/data/models/heartmula/ckpt/
|
||||||
|
├── HeartMuLa-oss-3B/
|
||||||
|
├── HeartCodec-oss/
|
||||||
|
├── gen_config.json
|
||||||
|
└── tokenizer.json
|
||||||
|
```
|
||||||
|
|
||||||
|
## Test operation
|
||||||
|
|
||||||
|
Only after the current music worker has been stopped:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd /opt/mike-ai/heartmula-3b
|
||||||
|
docker compose up -d heartmula-test
|
||||||
|
docker compose logs -f heartmula-test
|
||||||
|
```
|
||||||
|
|
||||||
|
The temporary test UI is then available on port `7863`. Stop it with:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker compose down
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not remove ACE-Step or change the dashboard/profile controller until the
|
||||||
|
A/B test has been accepted.
|
||||||
@@ -0,0 +1,148 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import uuid
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import gradio as gr
|
||||||
|
import soundfile as sf
|
||||||
|
import torch
|
||||||
|
import uvicorn
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from heartlib import HeartMuLaGenPipeline
|
||||||
|
|
||||||
|
|
||||||
|
MODEL_PATH = os.environ.get("HEARTMULA_MODEL_PATH", "/models/ckpt")
|
||||||
|
OUTPUT_DIR = Path(os.environ.get("HEARTMULA_OUTPUT_DIR", "/output"))
|
||||||
|
MULA_DEVICE = os.environ.get("HEARTMULA_MULA_DEVICE", "cuda:0")
|
||||||
|
CODEC_DEVICE = os.environ.get("HEARTMULA_CODEC_DEVICE", "cuda:1")
|
||||||
|
|
||||||
|
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
pipeline: HeartMuLaGenPipeline | None = None
|
||||||
|
load_error = ""
|
||||||
|
generation_lock = threading.Lock()
|
||||||
|
|
||||||
|
|
||||||
|
class AthenaHeartMuLaPipeline(HeartMuLaGenPipeline):
|
||||||
|
"""Write 24-bit WAV directly; torch 2.9 otherwise requires TorchCodec."""
|
||||||
|
|
||||||
|
def postprocess(self, model_outputs, save_path: str):
|
||||||
|
frames = model_outputs["frames"].to(self.codec_device)
|
||||||
|
wav = self.codec.detokenize(frames)
|
||||||
|
self._unload()
|
||||||
|
audio = wav.to(torch.float32).cpu().numpy().T
|
||||||
|
sf.write(save_path, audio, 48_000, format="WAV", subtype="PCM_24")
|
||||||
|
|
||||||
|
|
||||||
|
def load_pipeline() -> None:
|
||||||
|
global pipeline, load_error
|
||||||
|
try:
|
||||||
|
pipeline = AthenaHeartMuLaPipeline.from_pretrained(
|
||||||
|
MODEL_PATH,
|
||||||
|
device={
|
||||||
|
"mula": torch.device(MULA_DEVICE),
|
||||||
|
"codec": torch.device(CODEC_DEVICE),
|
||||||
|
},
|
||||||
|
dtype={"mula": torch.bfloat16, "codec": torch.float32},
|
||||||
|
version="3B",
|
||||||
|
lazy_load=False,
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
load_error = f"{type(exc).__name__}: {exc}"
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def generate(
|
||||||
|
tags: str,
|
||||||
|
lyrics: str,
|
||||||
|
instrumental: bool,
|
||||||
|
duration_seconds: int,
|
||||||
|
topk: int,
|
||||||
|
temperature: float,
|
||||||
|
cfg_scale: float,
|
||||||
|
):
|
||||||
|
if pipeline is None:
|
||||||
|
raise gr.Error(f"Modell ist nicht bereit. {load_error}".strip())
|
||||||
|
tags = ",".join(part.strip() for part in tags.split(",") if part.strip())
|
||||||
|
if not tags:
|
||||||
|
raise gr.Error("Bitte mindestens ein Musik-Tag angeben.")
|
||||||
|
if instrumental:
|
||||||
|
effective_lyrics = "[Instrumental]"
|
||||||
|
else:
|
||||||
|
effective_lyrics = lyrics.strip()
|
||||||
|
if not effective_lyrics:
|
||||||
|
raise gr.Error("Für ein Lied mit Gesang fehlt der Liedtext.")
|
||||||
|
|
||||||
|
target = OUTPUT_DIR / f"heartmula-{int(time.time())}-{uuid.uuid4().hex[:8]}.wav"
|
||||||
|
with generation_lock, torch.inference_mode():
|
||||||
|
pipeline(
|
||||||
|
{"lyrics": effective_lyrics, "tags": tags},
|
||||||
|
max_audio_length_ms=int(duration_seconds) * 1000,
|
||||||
|
save_path=str(target),
|
||||||
|
topk=int(topk),
|
||||||
|
temperature=float(temperature),
|
||||||
|
cfg_scale=float(cfg_scale),
|
||||||
|
)
|
||||||
|
return str(target), (
|
||||||
|
f"Fertig: {target.name} · WAV 48 kHz. "
|
||||||
|
"Instrumental ist bei der öffentlichen 3B-Version experimentell."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
with gr.Blocks(title="HeartMuLa 3B – Athena Test") as demo:
|
||||||
|
gr.Markdown(
|
||||||
|
"# HeartMuLa 3B – Athena Test\n"
|
||||||
|
"Offizielles öffentliches 3B-Modell. HeartMuLa läuft auf der RTX 5080, "
|
||||||
|
"HeartCodec verlustarm in FP32 auf der RTX 3060. "
|
||||||
|
"Referenzaudio wird von dieser Version noch nicht unterstützt."
|
||||||
|
)
|
||||||
|
tags = gr.Textbox(
|
||||||
|
label="Stil und Instrumente (Komma-getrennte Tags)",
|
||||||
|
value="synthwave,retrowave,1980s,melodic,nostalgic,arpeggiated synthesizer,drum machine,atmospheric,cinematic",
|
||||||
|
lines=3,
|
||||||
|
)
|
||||||
|
lyrics = gr.Textbox(
|
||||||
|
label="Liedtext mit Abschnitten wie [Verse], [Chorus], [Bridge]",
|
||||||
|
lines=14,
|
||||||
|
placeholder="[Verse]\n...\n\n[Chorus]\n...",
|
||||||
|
)
|
||||||
|
instrumental = gr.Checkbox(label="Instrumental (experimentell)", value=True)
|
||||||
|
duration = gr.Slider(20, 240, value=60, step=5, label="Maximale Dauer in Sekunden")
|
||||||
|
with gr.Accordion("Sampling – offizielle Standardwerte", open=False):
|
||||||
|
topk = gr.Slider(1, 200, value=50, step=1, label="Top-k")
|
||||||
|
temperature = gr.Slider(0.1, 2.0, value=1.0, step=0.05, label="Temperatur")
|
||||||
|
cfg_scale = gr.Slider(1.0, 4.0, value=1.5, step=0.1, label="CFG")
|
||||||
|
run = gr.Button("Musik erzeugen", variant="primary")
|
||||||
|
audio = gr.Audio(label="Ergebnis", type="filepath")
|
||||||
|
status = gr.Textbox(label="Status", interactive=False)
|
||||||
|
run.click(
|
||||||
|
generate,
|
||||||
|
inputs=[tags, lyrics, instrumental, duration, topk, temperature, cfg_scale],
|
||||||
|
outputs=[audio, status],
|
||||||
|
)
|
||||||
|
|
||||||
|
demo.queue(default_concurrency_limit=1, max_size=4)
|
||||||
|
|
||||||
|
api = FastAPI()
|
||||||
|
|
||||||
|
|
||||||
|
@api.get("/healthz")
|
||||||
|
def healthz():
|
||||||
|
return {
|
||||||
|
"ready": pipeline is not None,
|
||||||
|
"error": load_error or None,
|
||||||
|
"model": "HeartMuLa-oss-3B-happy-new-year",
|
||||||
|
"codec": "HeartCodec-oss-20260123",
|
||||||
|
"mula_device": MULA_DEVICE,
|
||||||
|
"codec_device": CODEC_DEVICE,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
api = gr.mount_gradio_app(api, demo, path="/")
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
load_pipeline()
|
||||||
|
uvicorn.run(api, host="0.0.0.0", port=7860)
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
services:
|
||||||
|
heartmula-test:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
args:
|
||||||
|
HEARTLIB_COMMIT: 3783bdb8441f2c298b1e64c8651173aac200361c
|
||||||
|
image: mike-ai/heartmula:3b-3783bdb
|
||||||
|
container_name: mike-ai-heartmula-test
|
||||||
|
restart: "no"
|
||||||
|
environment:
|
||||||
|
NVIDIA_VISIBLE_DEVICES: "GPU-8ad38c6c-5a01-9d8e-1dfa-ed662ad78fbe,GPU-4834d9d7-5b61-3004-1fb3-4ae49d482d4b"
|
||||||
|
NVIDIA_DRIVER_CAPABILITIES: compute,utility
|
||||||
|
HEARTMULA_MULA_DEVICE: cuda:0
|
||||||
|
HEARTMULA_CODEC_DEVICE: cuda:1
|
||||||
|
volumes:
|
||||||
|
- /data/models/heartmula:/models:ro
|
||||||
|
- /data/music/heartmula:/output
|
||||||
|
ports:
|
||||||
|
- "7863:7860"
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
reservations:
|
||||||
|
devices:
|
||||||
|
- driver: nvidia
|
||||||
|
count: all
|
||||||
|
capabilities: [gpu]
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "curl", "-fsS", "http://127.0.0.1:7860/healthz"]
|
||||||
|
interval: 15s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 20
|
||||||
|
start_period: 180s
|
||||||
Reference in New Issue
Block a user