int-llm precision ladder: from a wide integer oracle to compact weights

This is a follow-up to my earlier article, int-llm: a pure-integer LLM experiment in C. That project built a deliberately conservative Q16.48 fixed-point reference: a reproducible integer oracle for a tiny character GPT and TinyLlama-1.1B.

The obvious cost was weight storage. TinyLlama's approximately 2.2 GB safetensors checkpoint became an approximately 8.8 GB Q16.48 .mgw file because every weight occupied an int64_t. That was acceptable for an oracle, but not a satisfying compact representation.

The follow-up repository is nmicic/int-llm-precision-ladder.

It asks one narrower question:

How far can the stored weights move down the fixed-point precision ladder while preserving the Q16.48 oracle's behavior on explicit regression gates?

This post reports the measurements. It does not claim general model-quality parity or a new production inference engine.

What changed

The compute boundary remains conservative:

• weights are rounded to per-tensor F11 or F12 grids and stored as signed 16-bit codes;
• activations, the KV cache, nonlinear functions, and other runtime state remain Q16.48;
• matrix products retain signed-128 accumulation;
• the original Q16.48 runtime remains the comparison oracle.

Here, F12 means 12 fractional bits. It does not mean a 12-bit container. Each compact code occupies one signed 16-bit field and represents code × 2^-12. For example, rounding 0.123456789 to the nearest F12 grid point gives code 506, or 506 × 2^-12 = 0.12353515625.

The new MGWI format records the signed-int16 weights and each tensor's fractional-bit tag. Its packer refuses values that are off the declared grid or outside the signed-int16 range. It can also unpack an MGWI file into the canonical wide MGW layout.

The reference, packing, and execution paths are:

safetensors
    |
    v
Q16.48 MGW oracle
    |
    | round weights to selected F11/F12 grids
    v
rounded-wide MGW --pack--> MGWI --direct--> packed inference
        ^                    |
        +------unpack--------+
          byte-identical

safetensors --------direct converter--------> MGWI

Packing and unpacking are exact for the selected rounded model. Unpacking does not restore bits discarded when the original Q16.48 oracle was rounded. The lower-precision candidate is measured against the oracle; it is not redefined as the new source of truth.

Screening the idea with the tiny GPT

The small committed MicroGPT model made it possible to sweep many precision points quickly before running TinyLlama or flashing boards.

Uniform F12 preserved all 20 sampled outputs and all 122 checked greedy top-1 decisions. Uniform F11 changed one of the 122 top-1 decisions, and lower grids eventually changed sampled output. Internal logits already drift from the original Q16.48 oracle; the behavioral claim is limited to those checked outputs.

The rounded-wide F12 and packed F12 execution paths, however, produced the same raw logits. That established that the signed-int16 representation and its direct arithmetic path preserved the selected rounded model exactly.

The original small model.mgw artifact remains available on Hugging Face.

Physical MCU results

The complete uniform-F12 MicroGPT workload was then tested from flash on four physical boards in five ISA configurations. Every packed run reproduced its rounded-wide F12 control's 20 samples, 122 inference steps, sample hash, and raw pre-temperature logit hash.

The packed model container itself fell from 115,576 to 29,944 bytes. Static RAM was unchanged within each target because activations and state remained wide. The packed paths consumed the signed halfwords directly from flash; they did not first expand the weights into an int64 copy in RAM.

A one-code corrupted model preserved the displayed samples but changed the raw-logit hash and was rejected. This was useful confirmation that the gate could see changes below the final text-output boundary.

These are packed-versus-rounded-wide integer comparisons, not integer-versus- floating-point comparisons. They also show that compact storage does not guarantee speed: the MKR Zero result was effectively tied. Cache and XIP behavior, compiler output, memory layout, and instruction costs still matter.

TinyLlama: mixed F11/F12

TinyLlama first survived a uniform-F12 gate. Uniform F11 passed the original four-prompt check but diverged on the first held-out prompt. Tensor-group probes then led to a mixed candidate:

default F11
attention projection weights = F12
MLP projection weights       = F12

This places 968,884,224 weights (88.0765%) at F12 and 131,164,160 weights (11.9235%) at F11. All 1,100,048,384 codes fit signed int16.

The candidate matched the Q16.48 oracle on all 632 actually evaluated greedy decisions across 32 prompts. The schedule requested at most 640 decisions; five prompts selected matching EOS early. The repository's TINYLLAMA_RESULTS.md documents the complete requested-versus-evaluated accounting.

The packed and rounded-wide forms of the candidate also produced byte-identical raw-logit dumps: 3,264,000 int64 values across 102 forward calls.

The compact representation therefore removes the fourfold file expansion introduced by the wide oracle and returns the weight artifact to roughly the source checkpoint's size.

A separate direct converter produced MGWI from the original single-file safetensors checkpoint without materializing the 8.8 GB rounded MGW. Its 2,200,116,192 output bytes and all 1,100,048,384 codes were identical to the reference conversion pipeline. Unpacking it recreated the rounded-wide MGW byte-for-byte.

The packed file is pinned by SHA-256 fa733f5afdec220a91fbaae17ce00bcc20685f25afadd3eec92cfde0af1192c7.

Why the large model files are not included

The large TinyLlama MGW and MGWI files are not hosted in this repository. The repo retains the converter, checks, hashes, and compact result records; the checkpoint remains subject to its upstream distribution and licensing.

CPU result: memory improved, FP32 remained faster

On one AMD Ryzen 7 7700 host, the packed and rounded-wide integer lanes used the same rounded model and reproduced the same checked raw logits:

The packed lane reduced generation time by 23.209% and peak RSS by 73.303% in that within-runtime comparison.

A separate one-thread FP32 PyTorch/Transformers control was much faster: 11.9678 seconds versus 37.9248 seconds for the packed C runtime on the same 80-decision schedule. In that comparison, packed peak RSS was 2,261,082,112 bytes versus 7,306,727,424 bytes for FP32, about 69.1% less. The stacks are not matched arithmetic kernels, so this is a practical runtime comparison rather than proof that FP32 arithmetic is intrinsically 3.168899 times faster. It does show that the present packed C runtime has no x86 speed claim. Its strongest result is storage and memory.

GPU result: still parked

The GPU conclusion did not reverse:

• exact unpacked Q16.48 kernels were 2.44--3.68 times slower than native BF16/cuBLAS on substantial real TinyLlama matrix shapes;
• matched four-byte FP32 and INT32 Q16.16 CUDA-core controls stayed around 0.98--1.01 times parity, so integer arithmetic alone did not buy speed;
• an earlier single-launch int16 protocol failed its stability gate and remains parked; a separate synthetic rotated 12-GEMV sequence reduced time by 41.2--42.1% relative to the same codes in int32, but that is a bandwidth and representation result, not a TinyLlama runtime result.

There is still no integer-GPU game-changer here. The CUDA sources remain in the repo because negative and narrowly positive controls are useful evidence.

What the follow-up establishes

The Q16.48 oracle was intentionally too wide. The useful result of this follow-up is that the stored weights did not need to remain that wide for the checked workloads:

• a uniform-F12 tiny model transferred exactly between rounded-wide and packed execution on five physical MCU configurations;
• a mixed-F11/F12 TinyLlama candidate preserved 632/632 checked greedy decisions;
• direct safetensors-to-MGWI conversion reproduced the reference packed file exactly;
• the compact TinyLlama representation returned file size to approximately the original safetensors size and substantially reduced peak memory versus the rounded-wide integer path;
• neither the FP32 control nor the GPU measurements support a general integer speed claim.

This is evidence from bounded regression gates, one model, and the listed hardware. It is not proof of arbitrary-prompt quality parity, a generic quantizer, or a production runtime.

Links

• Precision-ladder repository
• Original int-llm repository
• Original small model artifact
• Original article
• Coordinate-permutation repository — a later follow-up exploring reversible checkpoint coordinate permutations for Q16.48 MGW and BF16/F16/F32 SafeTensors models.
• int-llm-viz now visualizes the int-llm checkpoint exactly — its atlas page includes an exact round-to-F-bits preview on the stored weights, echoing this ladder on the tiny GPT: live demo.

For the shortest path through the evidence, start with README.md, then read TINYLLAMA_RESULTS.md, MCU_RESULTS.md, and GPU_RESULTS.md in the new repository.