Benchmark Dirk Qwen3.8 against production
This commit is contained in:
@@ -23,24 +23,22 @@ therefore the mandatory A/B reference.
|
||||
container is running. It does not stop production itself. The model server is
|
||||
bound to loopback only and cannot be reached from the LAN.
|
||||
|
||||
## Prepared matrix
|
||||
## Measured matrix
|
||||
|
||||
Run the following only after the GPUs have explicitly been declared free:
|
||||
|
||||
```sh
|
||||
./run-case.sh 80000 90,10 text
|
||||
./run-case.sh 160000 90,10 text
|
||||
./run-case.sh 160000 85,15 text
|
||||
./run-case.sh 80000 87,13 text
|
||||
./run-case.sh 160000 80,20 text
|
||||
./run-case.sh 192000 85,15 text
|
||||
./run-case.sh 262144 80,20 text
|
||||
./run-case.sh 192000 72,28 text
|
||||
./run-case.sh 262144 70,30 text
|
||||
```
|
||||
|
||||
The largest stable context is determined first. Vision is checked only after a
|
||||
text winner exists:
|
||||
|
||||
```sh
|
||||
./run-case.sh 160000 85,15 vision
|
||||
./run-case.sh 160000 80,20 vision
|
||||
```
|
||||
|
||||
For every case, record uncached prefill, cached prefill, decode throughput,
|
||||
@@ -50,3 +48,6 @@ technical correctness and improves real Hermes task completion.
|
||||
|
||||
Expected SHA-256 checksums are stored in `MODEL_ARTIFACTS.sha256`.
|
||||
|
||||
The completed A/B result is documented in
|
||||
`docs/DIRK_QWEN38_AB_20260901.md`. The candidate did not replace the production
|
||||
Pure Qwen profile.
|
||||
|
||||
Reference in New Issue
Block a user