Test ordered one-slot DFlash2 placements and preserve results

This commit is contained in:
Mikei386
2026-09-20 21:10:08 +02:00
parent 3d5931146b
commit 4007242709
25 changed files with 360 additions and 5 deletions
@@ -55,3 +55,32 @@ Target loaded; draft allocation of1079.61MiB onCUDA0 failed with CUDA malloc
OOM. No benchmark inference executed. Supervisor restored original production;
health/smoke/kernel checks passed. No performance or quality conclusion can be
drawn. See results/ and restore-verification.json.
## User-ordered one-slot follow-up
The user authorized this order after the initial two-slot allocation failure:
1. One slot, target80:20, draft on5080.
2. One slot, target80:20, draft on3060, regardless of whether the first works.
3. Only if step2 fails: one slot, target70:30, draft back on5080. Moving both
additional target layers and the draft onto3060 would compete for its memory;
this fallback instead makes space on5080 for the draft.
Context remains160000 and other quick-probe settings are unchanged. These tests
are temporary; production containers retain their original two-slot arguments.
Case JSON now controls slot count and draft device (historical quick.json keeps
its original two-slot meaning). Failed attempts include an explicit error field.
An active production request postpones startup for up to180 seconds; no live
request is deliberately interrupted. The existing drain check still protects
requests racing with the transition. Each attempt has its own result directory;
previous results and the frozen Qwen reference are never overwritten.
## One-slot results
Priority1 fails during draft graph initialization: output.weight resides onCUDA1
but draft backends are restricted toCUDA0. Priority2 works; priority3 therefore
not run. German37.79 tok/s, code75.51 tok/s, 4196-token prefill1437.56 tok/s,
recall3/3. Allocation160000 is not full-context validation. Original two-slot
production restored and checked after both cases. See
../../docs/DFLASH2_ONE_SLOT_20260920.md for comparison caveats and quality findings.