Add Terminal-Bench results
Browse files
README.md
CHANGED
|
@@ -54,6 +54,15 @@ regression; treat general-English quality accordingly.
|
|
| 54 |
Snapshots before commit f53a8dcc carry a stale hot tier from the original
|
| 55 |
bootstrap allocation and will fail the loader's hyb_kind consistency checks.
|
| 56 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
## Serving (vLLM)
|
| 58 |
|
| 59 |
Serve with the SM120 fork `github.com/jarrelscy/vllm-glm52-sm120`, branch
|
|
|
|
| 54 |
Snapshots before commit f53a8dcc carry a stale hot tier from the original
|
| 55 |
bootstrap allocation and will fail the loader's hyb_kind consistency checks.
|
| 56 |
|
| 57 |
+
## Terminal-Bench (in progress)
|
| 58 |
+
|
| 59 |
+
No full Terminal-Bench run has finished on this checkpoint yet.
|
| 60 |
+
|
| 61 |
+
- **Terminal-Bench 2.1, partial:** 9 of the 13 tasks the [v1 ARVQ repo](https://huggingface.co/jarrelscy/GLM-5.3-Vision-NVFP4-ARVQ-hybrid) failed were rerun. 3 now pass (caffe-cifar-10, extract-elf, dna-insert), 3 timed out (dna-assembly, filter-js-from-html, make-doom-for-mips) and 3 were interrupted before finishing.
|
| 62 |
+
- **Terminal-Bench 4.0, running:** concurrency 1 with an 8h agent timeout. 6 of 64 tasks are done: 1 pass (layout-config-recreation2), 3 fail and 2 timeouts, so 1/4 excluding timeouts so far.
|
| 63 |
+
|
| 64 |
+
This section will be updated when the runs finish.
|
| 65 |
+
|
| 66 |
## Serving (vLLM)
|
| 67 |
|
| 68 |
Serve with the SM120 fork `github.com/jarrelscy/vllm-glm52-sm120`, branch
|