Benchmark Dirk Qwen3.8 against production

This commit is contained in:
Mikei386
2026-09-01 08:44:55 +02:00
parent 78096c9027
commit 548f6643fd
7 changed files with 326 additions and 18 deletions
+8 -7
View File
@@ -23,24 +23,22 @@ therefore the mandatory A/B reference.
container is running. It does not stop production itself. The model server is
bound to loopback only and cannot be reached from the LAN.
## Prepared matrix
## Measured matrix
Run the following only after the GPUs have explicitly been declared free:
```sh
./run-case.sh 80000 90,10 text
./run-case.sh 160000 90,10 text
./run-case.sh 160000 85,15 text
./run-case.sh 80000 87,13 text
./run-case.sh 160000 80,20 text
./run-case.sh 192000 85,15 text
./run-case.sh 262144 80,20 text
./run-case.sh 192000 72,28 text
./run-case.sh 262144 70,30 text
```
The largest stable context is determined first. Vision is checked only after a
text winner exists:
```sh
./run-case.sh 160000 85,15 vision
./run-case.sh 160000 80,20 vision
```
For every case, record uncached prefill, cached prefill, decode throughput,
@@ -50,3 +48,6 @@ technical correctness and improves real Hermes task completion.
Expected SHA-256 checksums are stored in `MODEL_ARTIFACTS.sha256`.
The completed A/B result is documented in
`docs/DIRK_QWEN38_AB_20260901.md`. The candidate did not replace the production
Pure Qwen profile.