Add files using upload-large-folder tool

README.md

@@ -3,7 +3,6 @@ tags:
 - blackhole
 - p300x2
 - tt-model-cache
-- tt-model-catalog
 - tt-model-container
 - vllm-plugin
 ---
@@ -80,6 +79,64 @@ Serve one request at a time: concurrency at long context is admission-limited
 by the KV cache. The release passed the bounded latency gate: 2% per metric,
 plus a 5 ms absolute TTFT allowance for short-input measurement variance.
 
+## Prefix caching
+
+Measured with vLLM's own `prefix_repetition` benchmark, the standard dataset
+for this feature. Eight distinct 4,096-token prefixes, each reused across
+eight requests, 64 requests at concurrency 1 -- the same package served twice,
+one flag apart.
+
+```
+vllm bench serve --model meta-models/Muse-Glimmer-30B \
+  --dataset-name prefix_repetition \
+  --prefix-repetition-prefix-len 4096 --prefix-repetition-suffix-len 128 \
+  --prefix-repetition-num-prefixes 8 --prefix-repetition-output-len 32 \
+  --num-prompts 64 --max-concurrency 1 --ignore-eos --seed 1234
+```
+
+| metric | caching off | caching on |
+|---|---:|---:|
+| mean TTFT | 497.00 ms | 168.04 ms |
+| median TTFT | 495.82 ms | 98.18 ms |
+| p99 TTFT | 525.84 ms | 952.04 ms |
+| benchmark duration | 80.14 s | 59.14 s |
+| output throughput | 25.56 tok/s | 34.63 tok/s |
+| prefix cache hit rate | 0 | 54.7% |
+
+The distribution is the evidence, not the mean. With caching off every request
+pays the full 4,096-token prefill and TTFT is flat at 496/497/526. With it on
+the distribution splits: 98 ms median for the 56 requests that hit, 952 ms at
+p99 for the 8 cold prefixes.
+
+Note the p99 moves the wrong way, 526 ms to 952 ms. A cold prefix now compiles
+its own SDPA program for its resume offset, so the first request at any
+previously unseen offset is slower than it was. Median improves 5x; the tail
+regresses 1.8x. Workloads that reuse a small set of prefixes gain; workloads
+whose offsets keep changing may not.
+
+## Evaluations
+
+Run through tt-inference-server's eval workflow -- its lm-eval command, venv
+and scoring -- against this package at 131,072 context. Sampling is this
+card's recipe: temperature 1.0, top_p 0.95, top_k 64.
+
+| task | samples | metric | score | reference |
+|---|---:|---|---:|---:|
+| `gpqa_diamond_cot_zeroshot` | 198 | exact_match, flexible-extract | 77.27 | 72.8 |
+| `ifeval` | 541 | prompt_level_strict_acc | 88.72 | 77.0 |
+| `aime25` | 30 | exact_match | not valid, see below | 94.7 |
+
+The GPQA reference is the GPU reference score for `openai/gpt-oss-20b`, the
+closest configured analogue. This model had no GPQA row before this run, while
+every other reasoning model of its size in the catalogue has one. The `ifeval`
+reference is an IFBench floor, not an equivalence target.
+
+`aime25` is not reported as a score. 18 of its 30 responses contained no
+extractable answer and one ran to 294,912 characters, while the same problems
+put to the server directly return correct boxed answers -- so the figure
+measures the eval path, not the model. 30 of the 198 GPQA responses show the
+same pathology, which makes 77.27 a lower bound rather than a point estimate.
+
 ## Provenance
 
 The exact sources the image was built from — `code/` in this repo is byte-identical to the model code inside the image:
@@ -90,5 +147,5 @@ The exact sources the image was built from — `code/` in this repo is byte-iden
 | vLLM | [`v0.24.0`](https://github.com/vllm-project/vllm/releases/tag/v0.24.0) |
 | vllm-tt-plugin | a local checkout — commit not published |
 | `code/` digest | `1b2d3eba0ea623e8` (sha256, first 16 hex digits) |
-| built | 2026-09-23T12:18:50+00:00 by tt-model 0.1.0 |
+| built | 2026-09-23T12:35:51+00:00 by tt-model 0.1.0 |
 

image/index.json

@@ -1 +1 @@
-{"schemaVersion":2,"mediaType":"application/vnd.oci.image.index.v1+json","manifests":[{"mediaType":"application/vnd.oci.image.index.v1+json","digest":"sha256:0a99681e8a24d7b502a9a5224d4bc5cd7d7124c167469d25bb0fe574df4d79c4","size":856,"annotations":{"io.containerd.image.name":"docker.io/tt-model/muse-glimmer-30b:0a99681e8a24","org.opencontainers.image.ref.name":"0a99681e8a24"}}]}
+{"schemaVersion":2,"mediaType":"application/vnd.oci.image.index.v1+json","manifests":[{"mediaType":"application/vnd.oci.image.index.v1+json","digest":"sha256:c715693dd6144a8cba93bcb04dc48b81372ccd535d19382a4ee9cfee38fa5e0e","size":856,"annotations":{"io.containerd.image.name":"docker.io/tt-model/muse-glimmer-30b:c715693dd614","org.opencontainers.image.ref.name":"c715693dd614"}}]}

image/manifest.json

@@ -1 +1 @@
-[{"Config":"blobs/sha256/154fcb88d190b0943dc00d6d518804d476e8fbe212404c1b1f39d9707a7da97c","RepoTags":["tt-model/muse-glimmer-30b:0a99681e8a24"],"Layers":["blobs/sha256/20c3783cc497b5b0df1fc5f92bd64c3d2fbb24057c88692c8fda205b6ea8a2f2","blobs/sha256/d9f87ac58c9b49986cbf04616e5f4439d448a63e142363c5ad84958fc54d41fd","blobs/sha256/506ff17d5a7e19c10baac4eeb6701e341d40742be379e5fd4b6218c0ddea1a2c","blobs/sha256/5f06a31d6b31f4c5d76a77d79559c6c33640fe48ff61dd72702be9c2b9167710","blobs/sha256/cb016668b3a6d8c046cf78bda44a43c2af52633ea83801af597c1fc3d4b997b5","blobs/sha256/5824a0e1deaafd8e392e50dfdeae94441e808ad85714804f8e6e048e8487a8d6","blobs/sha256/e8a3f4d9b77f077a09aed59bffef3a6f11b195e5d82075334da17fda66c6473e","blobs/sha256/5683b7b01115f2516578a34fceb7480a1d497c377904dc1ad127de02bfdf0a05","blobs/sha256/840b6e30f2ffab8690fa1710c4641da990032c40f2e53fd8a122ea62f246ec47","blobs/sha256/14a97c5789a3a81b1541f0f9f39dd913acc558d9a41c6a82c2dab340e325e430","blobs/sha256/4fa0a935bec6cd574c41d440c1c229281b130edfcf3f51facbc2cf5da8444cdd","blobs/sha256/71af603b54bcc36a8404e7b1fabadc8ad8731b717ca259d38746749decea2887","blobs/sha256/ab1435177c7a0e46a127a082ae952f60ec623c72a53893dbb18b12557c304838","blobs/sha256/c70cb072cd275fb1c2315d737d114e8362b3ad4e519b5185141fbe689056728d","blobs/sha256/662d0417b54c6723381aa65d679a11031bc254fa0d6aca36059dbe14f5c037b2","blobs/sha256/810af3ab59f1afc5af5f83504995af594cbb01cfac51d7424434d3955f7861aa","blobs/sha256/2ab752c1c94554954560f2e9dcd555a47de9f262c84146166093e3b0b0ab93bc","blobs/sha256/a2bf2c2212bda289c3757222c5b008ac0bc6f646942cae40e0262c8f7ebe0a62","blobs/sha256/4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1","blobs/sha256/d11d7c7b9b81a4143093d293a6cdc8cba96e4beb8a989f7c424dd9132b24add5","blobs/sha256/2ce2c023c383cda86b8207eb2ebc685b4305871b1af54a85ccf285eb7c5f1796","blobs/sha256/358af34ccf3feb37be6e6968145b07d79f5de0d77c646a0be6f3c214a7ba0b13"]}]
+[{"Config":"blobs/sha256/9fabbfdc214bbdaf1388a9dd8d3be72e665ae455f58a8ad6b0439af5e64841a6","RepoTags":["tt-model/muse-glimmer-30b:c715693dd614"],"Layers":["blobs/sha256/20c3783cc497b5b0df1fc5f92bd64c3d2fbb24057c88692c8fda205b6ea8a2f2","blobs/sha256/f9c52ad10fa38bd53d8f2c756de981c86f76b522b1112c82448d1b0ff4e8558a","blobs/sha256/b640607716388e92c76cbd8930854279ffdd13b17905c36a9e86d259b5b4c91d","blobs/sha256/f6e8f14e8fe6c6c102c0b12bfa4a2641baa4331ffee7c4e24a2ad5f585c7bff7","blobs/sha256/ffc0ec9a0bda7a2144939a854a5bb629d188903d423bf006110de2374535e62b","blobs/sha256/7bb3417a816b66194ee2c1b46fe2cec15b823b6520618e6b7f3884ad72413a44","blobs/sha256/bacb935dabb14e9175734c406307292042bcb51a5d45592589110d9235b1ed03","blobs/sha256/4af305a62cec10efd8b648753dff3438f9c255e556615614490c267d60df3467","blobs/sha256/39020e4807c5099bae573707113b7b6e5230cab290f47b0c1460d209018c8e39","blobs/sha256/c6e85e4a5ab37cce8088f5013297bbd48b5b88dad84aa7f4d478e0084cd16f9d","blobs/sha256/1d88e28a11ffe44ddb72c1bc8d8eff953fd09223d7ff3769f70ef7e168c1782e","blobs/sha256/e63bb76def5acb0f63cd6507a2794a58c796e3f7b359aa37fdf7e9641383a0d7","blobs/sha256/0e49a8fe105bf8b52f63751f49684d8f719c4d01723f45cd82cc56c57fe340ba","blobs/sha256/211bc2f61bde0f9dddd108b89edd016d3267af6183d50479e2eb08d75915a879","blobs/sha256/b8107065afdbac19691a2deceec62dc7ee6ffd1360318a5974fff13959fe9929","blobs/sha256/cec11eb07fdc442589febb11278ecae74d115d96d18c30b5f89c2b889e2841f4","blobs/sha256/788661dd0e494240d6c122ed3aa0b59b996c0b41a421b721ed0556b077a5cb17","blobs/sha256/743165483dd0723cf8abba0846bc083b2844e221bb25ad534e2221e8a9d9911e","blobs/sha256/4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1","blobs/sha256/ab3302deea2f463cc8199f22e77a00212dcbd9849844cf497d219098bc47360b","blobs/sha256/526cac24c89cf42678497c0d3c37a1a9ed917c33d88586b730e11183d53b0736","blobs/sha256/4b717ed492dc7f9df401347b7e1269aa38a906fc5cb0da8b67e5b5069f66433c"]}]

tt_kernel_manifest.json

@@ -1,12 +1,12 @@
 {
   "schema_version": "5.1",
   "name": "muse-glimmer-30b",
-  "tt_metal_version": "0.65.2.dev8971",
+  "tt_metal_version": "0.65.2.dev8972",
   "arch": "blackhole",
   "device_count": 4,
   "producer": {
     "tt_kernel_version": "0.1.0",
-    "created_at": "2026-09-23T12:28:00.392562+00:00",
+    "created_at": "2026-09-23T12:43:52.608232+00:00",
     "hostname": "tt-quietbox"
   },
   "weights": {
@@ -27,8 +27,8 @@
     "image": {
       "registry": "hf",
       "repository": "muse-glimmer-30b",
-      "tag": "tt-model/muse-glimmer-30b:0a99681e8a24",
-      "digest": "sha256:0a99681e8a24d7b502a9a5224d4bc5cd7d7124c167469d25bb0fe574df4d79c4"
+      "tag": "tt-model/muse-glimmer-30b:c715693dd614",
+      "digest": "sha256:c715693dd6144a8cba93bcb04dc48b81372ccd535d19382a4ee9cfee38fa5e0e"
     },
     "kind": "vllm-plugin",
     "runtime": {
@@ -103,15 +103,15 @@
       "import vllm; from vllm.reasoning import ReasoningParserManager as RM; RM.import_reasoning_parser('/opt/tt-metal/models/autoports/meta_models_muse_glimmer_30b/tt/reasoning_parser.py'); assert 'muse_glimmer' in RM.list_registered(), 'reasoning parser did not register'; RM.get_reasoning_parser('muse_glimmer')"
     ],
     "built": {
-      "image": "tt-model/muse-glimmer-30b:0a99681e8a24",
+      "image": "tt-model/muse-glimmer-30b:c715693dd614",
       "repo": "tt-hous/muse-glimmer-30b",
       "tt_model_version": "0.1.0",
-      "created_at": "2026-09-23T12:18:50+00:00",
+      "created_at": "2026-09-23T12:35:51+00:00",
       "tt_metal": {
-        "sha": "a56aff8699241c4eb96180ba2ae83baf395d9e29",
-        "describe": "v0.74.0-dev20260622-265-ga56aff8699",
+        "sha": "367a370de648e868be91306a89d0b2c9e2b62e61",
+        "describe": "v0.74.0-dev20260622-266-g367a370de6",
         "dirty": false,
-        "scm_version": "0.65.2.dev8971",
+        "scm_version": "0.65.2.dev8972",
         "mode": "local",
         "remote": "https://github.com/tenstorrent/tt-metal.git",
         "branch": "agentic-research/hous/muse-glimmer-30b-toolcalling",
@@ -123,7 +123,7 @@
         "path": "/home/ttuser/dev/muse-glimmer/wt-plugin-apc",
         "dirty": false
       },
-      "image_digest": "sha256:0a99681e8a24d7b502a9a5224d4bc5cd7d7124c167469d25bb0fe574df4d79c4"
+      "image_digest": "sha256:c715693dd6144a8cba93bcb04dc48b81372ccd535d19382a4ee9cfee38fa5e0e"
     }
   }
 }