Test ordered one-slot DFlash2 placements and preserve results
This commit is contained in:
@@ -55,3 +55,32 @@ Target loaded; draft allocation of1079.61MiB onCUDA0 failed with CUDA malloc
|
||||
OOM. No benchmark inference executed. Supervisor restored original production;
|
||||
health/smoke/kernel checks passed. No performance or quality conclusion can be
|
||||
drawn. See results/ and restore-verification.json.
|
||||
|
||||
## User-ordered one-slot follow-up
|
||||
|
||||
The user authorized this order after the initial two-slot allocation failure:
|
||||
|
||||
1. One slot, target80:20, draft on5080.
|
||||
2. One slot, target80:20, draft on3060, regardless of whether the first works.
|
||||
3. Only if step2 fails: one slot, target70:30, draft back on5080. Moving both
|
||||
additional target layers and the draft onto3060 would compete for its memory;
|
||||
this fallback instead makes space on5080 for the draft.
|
||||
|
||||
Context remains160000 and other quick-probe settings are unchanged. These tests
|
||||
are temporary; production containers retain their original two-slot arguments.
|
||||
Case JSON now controls slot count and draft device (historical quick.json keeps
|
||||
its original two-slot meaning). Failed attempts include an explicit error field.
|
||||
|
||||
An active production request postpones startup for up to180 seconds; no live
|
||||
request is deliberately interrupted. The existing drain check still protects
|
||||
requests racing with the transition. Each attempt has its own result directory;
|
||||
previous results and the frozen Qwen reference are never overwritten.
|
||||
|
||||
## One-slot results
|
||||
|
||||
Priority1 fails during draft graph initialization: output.weight resides onCUDA1
|
||||
but draft backends are restricted toCUDA0. Priority2 works; priority3 therefore
|
||||
not run. German37.79 tok/s, code75.51 tok/s, 4196-token prefill1437.56 tok/s,
|
||||
recall3/3. Allocation160000 is not full-context validation. Original two-slot
|
||||
production restored and checked after both cases. See
|
||||
../../docs/DFLASH2_ONE_SLOT_20260920.md for comparison caveats and quality findings.
|
||||
|
||||
Reference in New Issue
Block a user