m-a-p/340M-20B-GatedDeltaNet-hybrid-3-1
0.4B • Updated • 150 • 1
Hybrid Linear Attention Research
All 1.3B & 340M hybrid linear-attention experiments.
-
-
m-a-p/340M-20B-GatedDeltaNet-hybrid-6-1
0.4B • Updated • 86 -
m-a-p/340M-20B-GLA-hybrid-6-1
0.3B • Updated • 20 -
m-a-p/340M-20B-GLA-hybrid-3-1
0.3B • Updated • 40 -
m-a-p/340M-20B-GLA-hybrid-12-1
0.3B • Updated • 21 -
m-a-p/340M-20B-GLA-hybrid-24-1
0.3B • Updated • 24 -
m-a-p/340M-20B-HGRN2-pure-baseline
0.3B • Updated • 37 -
m-a-p/340M-20B-HGRN2-hybrid-6-1
0.3B • Updated • 33 -
m-a-p/340M-20B-HGRN2-hybrid-3-1
0.3B • Updated • 30 -
m-a-p/340M-20B-HGRN2-hybrid-12-1
0.3B • Updated • 29 -
m-a-p/340M-20B-HGRN2-hybrid-24-1
0.3B • Updated • 31 -
m-a-p/340M-20B-DeltaNet-hybrid-3-1
0.3B • Updated • 19 -
m-a-p/340M-20B-DeltaNet-pure
0.3B • Updated • 38 • 1 -
m-a-p/340M-20B-DeltaNet-hybrid-6-1
0.3B • Updated • 19 -
m-a-p/340M-20B-DeltaNet-hybrid-12-1
0.3B • Updated • 20 -
m-a-p/340M-20B-DeltaNet-hybrid-24-1
0.3B • Updated • 17 -
m-a-p/340M-20B-RetNet-hybrid-24-1
0.3B • Updated • 27 -
m-a-p/340M-20B-RetNet-hybrid-12-1
0.3B • Updated • 33 -
m-a-p/340M-20B-RetNet-hybrid-6-1
0.3B • Updated • 28 -
m-a-p/340M-20B-RetNet-hybrid-3-1
0.3B • Updated • 28 -
m-a-p/340M-20B-RetNet-pure
0.3B • Updated • 33 -
m-a-p/1.3B-100B-GatedDeltaNet-hybrid-3-1
1B • Updated • 639 -
m-a-p/1.3B-100B-GatedDeltaNet-hybrid-6-1
1B • Updated • 84 -
m-a-p/1.3B-100B-GatedDeltaNet-hybrid-12-1
1B • Updated • 571 -
m-a-p/1.3B-100B-GatedDeltaNet-hybrid-24-1
1B • Updated • 61 -
m-a-p/1.3B-100B-GLA-hybrid-3-1
1B • Updated • 36 -
m-a-p/1.3B-100B-GLA-hybrid-6-1
1B • Updated • 121 -
m-a-p/1.3B-100B-GLA-hybrid-12-1
1B • Updated • 36 -
m-a-p/1.3B-100B-GLA-hybrid-24-1
1B • Updated • 36 -
m-a-p/1.3B-100B-HGRN2-hybrid-3-1
1B • Updated • 35 -
m-a-p/1.3B-100B-HGRN2-hybrid-6-1
1B • Updated • 39 -
m-a-p/1.3B-100B-HGRN2-hybrid-12-1
1B • Updated • 34 -
m-a-p/1.3B-100B-HGRN2-hybrid-24-1
1B • Updated • 36 -
m-a-p/1.3B-100B-HGRN-hybrid-3-1
1B • Updated • 33 -
m-a-p/1.3B-100B-HGRN-hybrid-6-1
1B • Updated • 37 -
m-a-p/1.3B-100B-HGRN-hybrid-12-1
1B • Updated • 29 -
m-a-p/1.3B-100B-HGRN-hybrid-24-1
1B • Updated • 32 -
m-a-p/1.3B-100B-HGRN-pure
1B • Updated • 27 -
m-a-p/340M-20B-HGRN-hybrid-3-1
0.3B • Updated • 24 -
m-a-p/340M-20B-HGRN-hybrid-6-1
0.3B • Updated • 20 -
m-a-p/340M-20B-HGRN-hybrid-12-1
0.3B • Updated • 29 -
m-a-p/340M-20B-HGRN-hybrid-24-1
0.3B • Updated • 22 -
m-a-p/340M-20B-HGRN-pure-baseline
0.3B • Updated • 22 -
m-a-p/1.3B-100B-RetNet-hybrid-3-1
1B • Updated • 36 -
m-a-p/1.3B-100B-RetNet-hybrid-6-1
1B • Updated • 39 -
m-a-p/1.3B-100B-RetNet-hybrid-12-1
1B • Updated • 38 -
m-a-p/1.3B-100B-RetNet-hybrid-24-1
1B • Updated • 38 -
m-a-p/1.3B-100B-RetNet-pure
1B • Updated • 212 -
m-a-p/1.3B-100B-GLA-pure
1B • Updated • 287 -
m-a-p/1.3B-100B-HGRN2-pure
1B • Updated • 38 -
m-a-p/1.3B-100B-GatedDeltaNet-pure
1B • Updated • 1.67k • 1 -
m-a-p/340M-20B-GLA-pure-baseline
0.3B • Updated • 43 -
m-a-p/340M-20B-GatedDeltaNet-hybrid-12-1
0.4B • Updated • 49 -
m-a-p/1.3B-100B-DeltaNet-hybrid-3-1
1B • Updated • 30 -
m-a-p/1.3B-100B-DeltaNet-hybrid-6-1
1B • Updated • 35 -
m-a-p/1.3B-100B-DeltaNet-hybrid-12-1
1B • Updated • 26 -
m-a-p/1.3B-100B-DeltaNet-hybrid-24-1
1B • Updated • 39 -
m-a-p/1.3B-100B-DeltaNet-pure
1B • Updated • 475 -
m-a-p/340M-20B-GatedDeltaNet-pure-baseline
0.4B • Updated • 621 -
m-a-p/340M-20B-GatedDeltaNet-New-hybrid-24-1
0.4B • Updated • 37 -
m-a-p/transformer_1.3B_baseline
1B • Updated • 68 -
m-a-p/transformer_340M_baseline
0.3B • Updated • 48