Watching a pure-integer GPT think — one degree per frame

This is the fourth entry in the int-llm series. The first post trained and ran a tiny GPT entirely in Q16.48 int64 fixed-point arithmetic. The second descended the fraction-bit ladder; the third permuted internal coordinates and proved nothing changed. This one just looks: all 14,272 learned weights on screen, every displayed value bit-exact.

Try it before reading — it's four self-contained offline HTML pages, no server, no dependencies:

https://nmicic.github.io/int-llm-viz/site/index.html

The contract

One rule everywhere: every displayed number is either bit-exact — BigInt end-to-end, exact terminating decimals (Q16.48 values terminate within 48 fraction digits) — or explicitly labeled as float display geometry (pixels, colors, camera math). The original C runtime is the oracle: a trace harness replays all 20 recorded samples, 122 steps, bit-for-bit — 0 mismatches. In your browser, every page first proves it parsed the same checkpoint bytes as the C loader (FNV-1a-64 checksums); then the trace and weights pages re-check their numbers against the recorded oracle trace, while the atlas and space pages re-derive their own invariants — permutation checksums and the exact PCA scatter matrix — from the embedded checkpoint.

Four pages

The atlas — every parameter at its literal address

The swap — one transposition, all nine matrices

The atlas's second derived section traces the verified residual-coordinate swap (0 ↔ 31) through every learned matrix at once: 892 of 14,272 stored positions move, none changes value, and the page re-proves it.

The trace — the model's complete recorded output, step by step

The model generates names. The trace page replays the recording — residual stream, per-head attention, ReLU firing, logits → probabilities → the exact roulette draw — while the browser re-derives every step in BigInt and checks it against the C oracle's values. The draw lands on "k": the machine names itself kayla.

The space — 62 vectors, one 32-dimensional room

All 27 token embeddings, 27 output directions, and 8 position vectors share the residual space. Literal coordinate axes, or a clearly labeled derived PCA view — including a four-panel PC1–4 treatment where all four cameras turn on one shared clock.

The weights — one matrix, four ways

Any tensor as a 3-D point cloud, a heatmap, the checkpoint's literal byte layout, or "4-D shadows": four projections of (column, row, weight, contribution) under a single camera. Contribution mode is exact too — each stored weight times the recorded input it actually met at a chosen trace step, every row re-summed in BigInt against the oracle's output.

One degree per frame

The cameras are deterministic — one shared clock, 48 seconds per revolution — and the weights page exposes a renderAt(ms) hook for scripted capture: the same scene and timestamp always produce the same frame, which is how the loop video was shot.

Re-proved, not trusted

The vendored neuron orderings and the coordinate swap aren't taken on faith: the build reconstructs every variant checkpoint from the committed original, asserts the recorded SHA-256 hashes, recomputes the adjacency scores in exact integer arithmetic, and replays each variant through the C oracle bit-identically. make test runs the whole ten-step suite; make release-check additionally requires the live-browser checks and fails on any regeneration drift.

Scope, honestly

This is a polished spike for one tiny checkpoint, not a framework. The pages assume this model's shape (27-token vocabulary, 1 layer, 4 heads, 8 positions) wherever that made them clearer — the assumptions are asserted, not hidden. Embedding everything in single files works at this scale (the largest page carries a 1.3 MB trace); a real system would stream slices. The PCA basis is derived from the 27 token embeddings only, with the output and position vectors projected onto it — the page says so. And there are no semantic claims anywhere: the pages state coordinates, values, and verified equalities, and stop there.

Links

• Live: https://nmicic.github.io/int-llm-viz/site/index.html
• Repo: https://github.com/nmicic/int-llm-viz
• The checkpoint on the Hub: https://huggingface.co/nmicic/int-llm
• The loop video (minimal techno): https://www.youtube.com/watch?v=RxRAQY7YrkQ
• The song - one degree per frame: https://suno.com/s/FOHH2CyUbYerUPXC

The int-llm series:

1. int-llm — a GPT trained and run entirely in Q16.48 int64 arithmetic https://github.com/nmicic/int-llm
2. int-llm-precision-ladder — descending the fraction-bit ladder https://github.com/nmicic/int-llm-precision-ladder
3. int-llm-coordinate-permutation — coherent coordinate permutations across all tensors https://github.com/nmicic/int-llm-coordinate-permutation
4. int-llm-viz — this visualization https://github.com/nmicic/int-llm-viz

Every displayed number is exact. Same input, same sky.