Update README.md
Browse files
README.md
CHANGED
|
@@ -27,7 +27,7 @@ metrics:
|
|
| 27 |
|
| 28 |
<img src="https://raw.githubusercontent.com/llaa33219/MicroMixer-4/main/logo.svg" width="300" alt="MicroMixer-4 Logo"/>
|
| 29 |
|
| 30 |
-
# MicroMixer-4-100K
|
| 31 |
|
| 32 |
<img src="https://img.shields.io/badge/Parameters-95%2C084-blue?style=for-the-badge&logo=python&logoColor=white&color=%23007BFF" alt="Parameters"/>
|
| 33 |
<img src="https://img.shields.io/badge/Architecture-CCD--Mixer-purple?style=for-the-badge&color=%23AE00FF" alt="Architecture"/>
|
|
@@ -55,7 +55,7 @@ metrics:
|
|
| 55 |
|
| 56 |
## 📋 Overview
|
| 57 |
|
| 58 |
-
**MicroMixer-4-100K** is a **95,084-parameter** pure MLP-Mixer causal language model — **no attention, no recurrence, no SSM** — pretrained on Discord conversation data and then fine-tuned with **FMSP** (Fine-tuning with Minimal Parameter changes for Small-parameter LMs) on 9,012 general-knowledge QA pairs.
|
| 59 |
|
| 60 |
This is the **100K** member of the MicroMixer-4 (V87 Final) family: below the memorization floor — full-988 EM is exactly 0: the P05 recipe cannot write any of the 9,012 facts into this few parameters.
|
| 61 |
|
|
|
|
| 27 |
|
| 28 |
<img src="https://raw.githubusercontent.com/llaa33219/MicroMixer-4/main/logo.svg" width="300" alt="MicroMixer-4 Logo"/>
|
| 29 |
|
| 30 |
+
# MicroMixer-4-100K-Discord-Dialogues
|
| 31 |
|
| 32 |
<img src="https://img.shields.io/badge/Parameters-95%2C084-blue?style=for-the-badge&logo=python&logoColor=white&color=%23007BFF" alt="Parameters"/>
|
| 33 |
<img src="https://img.shields.io/badge/Architecture-CCD--Mixer-purple?style=for-the-badge&color=%23AE00FF" alt="Architecture"/>
|
|
|
|
| 55 |
|
| 56 |
## 📋 Overview
|
| 57 |
|
| 58 |
+
**MicroMixer-4-100K-Discord-Dialogues** is a **95,084-parameter** pure MLP-Mixer causal language model — **no attention, no recurrence, no SSM** — pretrained on Discord conversation data and then fine-tuned with **FMSP** (Fine-tuning with Minimal Parameter changes for Small-parameter LMs) on 9,012 general-knowledge QA pairs.
|
| 59 |
|
| 60 |
This is the **100K** member of the MicroMixer-4 (V87 Final) family: below the memorization floor — full-988 EM is exactly 0: the P05 recipe cannot write any of the 9,012 facts into this few parameters.
|
| 61 |
|