tiyuvta
/

GLM-5.3-Flash-NVFP4

@@ -224,37 +224,58 @@ over PCIe, at roughly 53 GB/s on the pinned path against about 10 GB/s pageable.
 resident-traffic roofline for this configuration is 63 tok/s, so this number has room in
 it and is not a property of the artifact. It is a bring-up figure on one configuration.
-## Context, and where the practical wall actually is
-The upstream checkpoint declares `max_position_embeddings: 1048576`. That is **the
-vendor's architecture figure**, and it is repeated here as one, not as something this
-mint delivers. **We have not gated or run this artifact anywhere near that context**,
-and nothing in this repo should be read as a 1M-context serving claim.
-The reason is worth stating, because it is checkable from `config.json` in this repo
-rather than something you have to take on trust. The DSA layers score pooled keys:
-`index_kpool` is 4, so a call over `t` query tokens against `t_kv` keys allocates a
-score plane of `t * (t_kv / 4)` f32 values. Under a monolithic prefill where
-`t = t_kv = N`, that plane is **N squared bytes**, per MLA layer, per call:
-| context | transient score plane |
-|---|---|
-| 8,192 | 67 MB |
-| 50,000 | 2.5 GB |
-| 262,144 | 68.7 GB |
-| 1,048,576 | 1.10 TB |
-Our bring-up hit CUDA out-of-memory on a 96 GB card in the region that arithmetic
-predicts. Note where the wall falls: 262,144 is already past a single 96 GB card, so
-it sits well below the headline number rather than at it.
-Two things this is not. It is not a quantization artifact, since the plane is f32
-scratch whose size depends only on context and the pooling factor, so an FP8 or BF16
-copy of this model has the same wall. And it is not a bigger-card problem in any
-practical sense: this is per-call transient scratch, not a persistent allocation you
-can amortize, so chunked prefill is the shape that moves it, not more VRAM.
-If you need long context on this model today, that is unsolved work, not a flag.
 ## Files

 resident-traffic roofline for this configuration is 63 tok/s, so this number has room in
 it and is not a property of the artifact. It is a bring-up figure on one configuration.
+## Context: 1,035,357 tokens demonstrated, and where the remaining wall actually is
+The upstream checkpoint declares `max_position_embeddings: 1048576`. We have run this
+artifact at that scale, once, deliberately, and the receipts are dated bench measurements
+on named hardware: not a hosted offering, and not a latency claim.
+**Demonstrated (2026-08-29, 4x RTX PRO 6000 Blackwell 96 GB, memra PP4 pipeline,
+`MEMRA_PP_SPLITS=13,26,39`):** a real 1,035,357-token prompt (Gutenberg prose, sha-banked,
+token count from the server's own usage, `cached_tokens=0`) primed through the serving
+surface inside the model's 1,048,576 window, then decoded greedy to EOS with a coherent
+cross-book answer. Prefill 161.28 tok/s (6,419.8 s wall, 107 minutes); the vendor-default
+sampled twin primed the same prompt at 161.26 tok/s (two independent full primes agreeing
+to 0.01%) and also answered coherently. Error census of the serve log: 0.
+Depth behaves, with no cliff:
+| prompt tokens | prefill tok/s | decode tok/s (greedy, steady) |
+|---|---|---|
+| 15,766 | 172.7 | 24.5 |
+| 128,566 | 171.5 | 22.8 |
+| 257,775 | 169.6 | 21.1 |
+| 525,616 | 165.8 | 18.9 |
+| 1,035,357 | 161.28 | 15.7 |
+Chunked prefill throughput is depth-flat to 1M (-6.6% from 16k), and greedy decode decays
+smoothly (1.75x over three orders of magnitude of depth); the sampled twin sits within
+0.6% of greedy at 1M. In-context retrieval held at depth (at 131k the answer cited a
+detail from a salon debate inside War and Peace).
+**What the earlier revision of this card called the wall, the DSA score-plane transient
+of a monolithic prefill (`N^2` bytes per MLA layer per call), is fixed, not by more
+VRAM but by the chunk schedule it predicted:** memra's mHC prime now walks bounded chunks
+(monolithic rollback preserved behind `MEMRA_PRIME_CHUNK=0`), so the per-call plane is
+`chunk_rows x (N / 4)` f32 rather than `N^2` bytes. That was never a quantization
+artifact and its fix is not one either: an FP8 or BF16 copy of this model has the same
+arithmetic on both sides of the fix.
+Honest boundaries, so this is not read as a serving claim:
+- **1M is demonstrated on 4 cards, not fewer.** A 3-card full-expert-residency
+  configuration of this artifact fails the same prime (CUDA out-of-memory in the DSA
+  k-pool selection at a 97.2 GiB per-card peak), so the depth ceiling of smaller
+  placements sits well below 1M, and the only demonstrated 1M configuration is the
+  4-card pipeline with the expert cache capped to leave the tail stage room.
+- **Time-to-first-token at 1M is 107 minutes on that baseline.** Long-context capacity
+  is solved; long-context *latency* is engineering in progress (prefill has since moved
+  independently on other configurations of the same engine, e.g. a tensor-core MLA
+  prefill path measured at 1629-2255 tok/s at short depths, but no 1M number exists on
+  that path and none is claimed here).
+- Everything above is greedy-plus-sampled-twin bench measurement on named hardware,
+  published with its receipts in the memra repo
+  (`research/glm53-flash-bringup-20260827/1m-demo-20260829/`).
 ## Files