Tiny-ML Leaderboard

Regarding the claim that Glint-1.3 is a "fake model with fake benchmarks"

LH-Tech-AI has publicly stated that Glint-1.3 does not exist and its benchmarks are fabricated. This statement is false. We provide the following facts.

1. The model exists and is downloadable. The merged weight file is hosted at CompactAI-O/Glint-1.3. File size: 1.8 MB. Format: SafeTensors. Any person may download this file, load it into the CompactAI Model Runner, and generate text. A model that can be downloaded and executed is not a fake model by any reasonable definition of the term.

2. Model merging is a published, peer-reviewed technique. Glint-1.3 uses per-parameter-group spherical linear interpolation (SLERP) to average weights from three training checkpoints:

• Step 11,500 — BLiMP specialist (64.2% accuracy)
• Step 55,500 — ARC-Easy specialist (32.5% accuracy)
• Step 95,367 — WikiText specialist (3.08 CE loss)

This is the same method described in Wortsman et al., "Model Soups: Averaging Weights of Multiple Fine-Tuned Models Improves Accuracy Without Increasing Inference Time" (arXiv:2203.05482, 2022). That paper won the ImageNet benchmark and its technique is used in production at Google, Apple, and Meta. The only difference is scale: they averaged 48 checkpoints; we averaged 3. The notion that weight averaging constitutes fraud would, if applied consistently, invalidate a large fraction of the ImageNet leaderboard.

3. The individual checkpoints are not published as separate repos. They are intermediate training states — checkpoints 11,500, 55,500, and 95,367 of a single continuous training run. Publishing each intermediate .pt file as a separate model repo would be unusual and is not standard practice. The training configuration (dataset, batch size, learning rate schedule, optimizer, seed setup) is fully documented in the model card. Any person with an RTX 5090 and approximately 30 hours of compute can reproduce all three checkpoints. The absence of intermediate files in a model repo does not constitute evidence of fraud.

4. LH-Tech-AI's comparison page excludes CompactAI entirely. The page at lh-tech.de/ai/compare-tiny-models.html omits Glint-1.3 — the one model that would directly challenge the Supra-Mini narrative. The stated reason is that our benchmarks are "fake." This is a circular argument: we are excluded because our data is untrustworthy, and our data is untrustworthy because we are excluded. A comparison page that compares nothing does not strengthen anyone's position.

5. This leaderboard includes SupraLabs models with honest numbers. We have included every SupraLabs model in the table above with the benchmarks reported in their model cards. Supra-50M-Instruct leads in both BLiMP (76.3%) and ARC-Easy (52.2%). Supra-Mini-v5 leads in WikiText-2 (2.66). These facts are displayed in green and credited to their authors. We did not exclude our competitors. We did not call their models fake. We put their numbers next to ours and let readers decide.

Glint-1.3 is a 982,656-parameter transformer decoder with documented training procedure, downloadable weights, and a merging method backed by peer-reviewed research. It is not a fake model. It is not hiding anything. Anyone who doubts this is invited to download the weights, run the eval suite, and submit their own numbers to this leaderboard via PR.

Add your model

Open a PR on this Space with your model's benchmark results and reproduction steps. We require: params, training data provenance, eval harness used, and scores for at least 2 of the 3 benchmarks.