Correct tb4.0 partial tally
Browse files
README.md
CHANGED
|
@@ -59,7 +59,7 @@ bootstrap allocation and will fail the loader's hyb_kind consistency checks.
|
|
| 59 |
No full Terminal-Bench run has finished on this checkpoint yet.
|
| 60 |
|
| 61 |
- **Terminal-Bench 2.1, partial:** 9 of the 13 tasks the [v1 ARVQ repo](https://huggingface.co/jarrelscy/GLM-5.3-Vision-NVFP4-ARVQ-hybrid) failed were rerun. 3 now pass (caffe-cifar-10, extract-elf, dna-insert), 3 timed out (dna-assembly, filter-js-from-html, make-doom-for-mips) and 3 were interrupted before finishing.
|
| 62 |
-
- **Terminal-Bench 4.0, running:** concurrency 1 with an 8h agent timeout. 6 of 64 tasks are done: 1 pass (layout-config-recreation2),
|
| 63 |
|
| 64 |
This section will be updated when the runs finish.
|
| 65 |
|
|
|
|
| 59 |
No full Terminal-Bench run has finished on this checkpoint yet.
|
| 60 |
|
| 61 |
- **Terminal-Bench 2.1, partial:** 9 of the 13 tasks the [v1 ARVQ repo](https://huggingface.co/jarrelscy/GLM-5.3-Vision-NVFP4-ARVQ-hybrid) failed were rerun. 3 now pass (caffe-cifar-10, extract-elf, dna-insert), 3 timed out (dna-assembly, filter-js-from-html, make-doom-for-mips) and 3 were interrupted before finishing.
|
| 62 |
+
- **Terminal-Bench 4.0, running:** concurrency 1 with an 8h agent timeout. 6 of 64 tasks are done: 1 pass (layout-config-recreation2), 4 fail and 1 timeout, so 1/5 excluding timeouts so far. photonic-waveguide-routing hit the timeout but is counted as a fail: it was stuck repeating max-length responses.
|
| 63 |
|
| 64 |
This section will be updated when the runs finish.
|
| 65 |
|