DedeProGames commited on
Commit
44fe1c6
路
verified 路
1 Parent(s): 72227db

Add GPT-U-20M (DedeProGames) and show it by default

Browse files

[DedeProGames/GPT-U-20M](https://huggingface.co/DedeProGames/GPT-U-20M) is a 20,453,760-parameter Llama-architecture base model trained from scratch on 2.6B tokens (DCLM 45% / FineWeb-Edu 35% / The Stack v3 20%).

**Changes (`matched-models.js` only)**
- New `MATCHED_ROWS` row with the measurements below.
- `LOCAL_RUNS` entry (context window 1024 + provenance).
- `GPT-U-20M` added to `RECOMMENDED_MODELS`, so it is selected by default when the page opens.

| Benchmark | Score | Method |
|---|---:|---|
| BananaMind Base Bench 1.1 | 945 Elo (152/350, 43.43%) | official `benchmark.py`, complete 350-item split, bfloat16, no BOS |
| ARC Easy | 37.25 | lm-eval 0.4.13, zero-shot `acc_norm`, float32 |
| HellaSwag | 27.85 | lm-eval 0.4.13, zero-shot `acc_norm`, float32 |
| PIQA | 58.38 | lm-eval 0.4.13, zero-shot `acc_norm`, float32 |
| Arithmark 3.0 | 35.10 | official `bencharithmark-3.py`, `acc_norm`, bfloat16 |

Base Bench category Elo:

| Category | Elo |
|---|---:|
| Language completion | 1174 |
| Commonsense | 879 |
| World knowledge | 883 |
| Context tracking | 761 |
| Quantitative | 861 |
| Logical reasoning | 1073 |
| Code completion | 1012 |

Checked locally by evaluating `data.js`, `matched-models.js`, `organizations.js` and `scoring.js` in Node: the model is recommended (21 default models), index 41.09, size frontier within 卤2M (BananaMind-Sundae: 37.2), no duplicate ids.

Files changed (1) hide show
  1. matched-models.js +3 -1
matched-models.js CHANGED
@@ -61,6 +61,7 @@ const MATCHED_ROWS = [
61
  ['LowOnMind-300k','dedeprogames',296960,'DedeProGames/LowOnMind-300k',29.08,24.96,50.22,29.50,833,921,848,728,707,938,905,789],
62
  ['NanoDex-Test-500K-200M','dedeprogames',492192,'DedeProGames/NanoDex-Test-500K-200M',27.90,25.06,50.71,31.60,800,885,858,791,679,788,911,684],
63
  ['NanoDex-1M','dedeprogames',1062272,'DedeProGames/NanoDex-1M',29.17,27.01,53.48,26.50,827,892,850,828,774,795,939,729],
 
64
  ['BananaMind-Sundae','bananamindresearch',20156544,'bananamind-research-community/BananaMind-Sundae',33.92,26.08,54.46,34.20,891,1167,789,874,928,785,920,831],
65
  ['BackKiyo-10M','dedebckp',9976832,'DedeBckp/BackKiyo-10M',35.40,28.08,55.66,35.10,924,1070,860,920,815,940,973,924],
66
  ['peacebell-v1-148M','wayneworkman',148553302,'wayneworkman2012/peacebell-v1-148M',27.27,26.16,47.61,27.10,902,1035,826,929,954,774,891,924],
@@ -103,11 +104,12 @@ const LOCAL_RUNS = {
103
  'LowOnMind-300k':localRun(93,512), 'NanoDex-Test-500K-200M':localRun(88,512),
104
  'NanoDex-1M':localRun(98,512), 'BananaMind-Sundae':localRun(134,1024), 'BackKiyo-10M':localRun(144,4096),
105
  'peacebell-v1-148M':{contextWindow:16384, provenance:'Local official complete run 路 2026-09-19 路 131 / 350 correct (bfloat16; float32 gives 127 / 350, Elo 893). ARC Easy, HellaSwag and PIQA: lm-eval 0.4.13 zero-shot acc_norm (float32). Arithmark 3: official script acc_norm (bfloat16). Domain-specific model: trained from scratch on World War II text only.'},
 
106
  };
107
  const RECOMMENDED_MODELS = new Set(['BananaMind-2-Pro','BananaMind-2-Medium','BananaMind-2-Mini',
108
  'SmolLM2-135M','SmolLM-135M','GPT-X2.5-135M','GPT-X2-125M','GPT-2','Kiyo-135M','Kiyo-65M',
109
  'Rose-Pro','Rose-Medium','Supra2-100M-Base','Supra-50M-Base','GPT-S2-5M','BananaMind-2-Micro',
110
- 'Kiyo-230M-Preview','DynamicMind-MoE','LowOnMind-300k']);
111
  for (const row of MATCHED_ROWS) {
112
  const [name,org,params,repo,arc,hellaswag,piqa,arithmark3,overall,language,commonsense,knowledge,context,quantitative,logic,code] = row;
113
  if (row.length !== 16) throw new Error(`Invalid measurement row: ${name}`);
 
61
  ['LowOnMind-300k','dedeprogames',296960,'DedeProGames/LowOnMind-300k',29.08,24.96,50.22,29.50,833,921,848,728,707,938,905,789],
62
  ['NanoDex-Test-500K-200M','dedeprogames',492192,'DedeProGames/NanoDex-Test-500K-200M',27.90,25.06,50.71,31.60,800,885,858,791,679,788,911,684],
63
  ['NanoDex-1M','dedeprogames',1062272,'DedeProGames/NanoDex-1M',29.17,27.01,53.48,26.50,827,892,850,828,774,795,939,729],
64
+ ['GPT-U-20M','dedeprogames',20453760,'DedeProGames/GPT-U-20M',37.25,27.85,58.38,35.10,945,1174,879,883,761,861,1073,1012],
65
  ['BananaMind-Sundae','bananamindresearch',20156544,'bananamind-research-community/BananaMind-Sundae',33.92,26.08,54.46,34.20,891,1167,789,874,928,785,920,831],
66
  ['BackKiyo-10M','dedebckp',9976832,'DedeBckp/BackKiyo-10M',35.40,28.08,55.66,35.10,924,1070,860,920,815,940,973,924],
67
  ['peacebell-v1-148M','wayneworkman',148553302,'wayneworkman2012/peacebell-v1-148M',27.27,26.16,47.61,27.10,902,1035,826,929,954,774,891,924],
 
104
  'LowOnMind-300k':localRun(93,512), 'NanoDex-Test-500K-200M':localRun(88,512),
105
  'NanoDex-1M':localRun(98,512), 'BananaMind-Sundae':localRun(134,1024), 'BackKiyo-10M':localRun(144,4096),
106
  'peacebell-v1-148M':{contextWindow:16384, provenance:'Local official complete run 路 2026-09-19 路 131 / 350 correct (bfloat16; float32 gives 127 / 350, Elo 893). ARC Easy, HellaSwag and PIQA: lm-eval 0.4.13 zero-shot acc_norm (float32). Arithmark 3: official script acc_norm (bfloat16). Domain-specific model: trained from scratch on World War II text only.'},
107
+ 'GPT-U-20M':{contextWindow:1024, provenance:'Local official complete run 路 2026-09-26 路 152 / 350 correct (bfloat16). ARC Easy, HellaSwag and PIQA: lm-eval 0.4.13 zero-shot acc_norm (float32). Arithmark 3: official script acc_norm (bfloat16).'},
108
  };
109
  const RECOMMENDED_MODELS = new Set(['BananaMind-2-Pro','BananaMind-2-Medium','BananaMind-2-Mini',
110
  'SmolLM2-135M','SmolLM-135M','GPT-X2.5-135M','GPT-X2-125M','GPT-2','Kiyo-135M','Kiyo-65M',
111
  'Rose-Pro','Rose-Medium','Supra2-100M-Base','Supra-50M-Base','GPT-S2-5M','BananaMind-2-Micro',
112
+ 'Kiyo-230M-Preview','DynamicMind-MoE','LowOnMind-300k','GPT-U-20M']);
113
  for (const row of MATCHED_ROWS) {
114
  const [name,org,params,repo,arc,hellaswag,piqa,arithmark3,overall,language,commonsense,knowledge,context,quantitative,logic,code] = row;
115
  if (row.length !== 16) throw new Error(`Invalid measurement row: ${name}`);