File size: 3,300 Bytes
e48d0ce ff54b7c e48d0ce | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | ---
license: apache-2.0
language: en
tags:
- aac
- speech-prosthetic
- small-language-model
- mlx
- from-scratch
---
# TinkyBrain 31M — A Language Model Built From Scratch
**31 million parameters. Trained from zero. On a Mac mini M4.**
No fine-tuning. No pre-trained weights. No shortcuts. Every single weight in this model was learned from scratch on Apple Silicon using MLX.
## What Makes This Special
Most "custom models" are LoRA adapters slapped on top of someone else's foundation model. TinkyBrain is different:
- **Custom architecture** — MicroBrain transformer built from scratch in `model.py`
- **Custom tokenizer** — grown organically during curriculum training, not borrowed from GPT/LLaMA
- **Custom training pipeline** — DJ playlist-style curriculum learning (70 tracks, added one at a time like ingredients in a recipe)
- **Trained entirely on Apple Silicon** — MLX on M4 Mac mini, no cloud GPU needed
- **Purpose-built for AAC** — designed to power speech prosthetics for non-verbal users (stroke survivors, neurodivergent individuals)
## Training Approach
The playlist trainer (`train_playlist.py`) treats training data like a DJ set:
1. Start with baby-level language (greetings, feelings, basic needs)
2. Add one "track" at a time — 100 epochs per track on the cumulative dataset
3. Each track builds on what came before — the model never forgets early lessons
4. 14 curriculum stages from babbling to full conversation
5. Smart early stopping with patience-based checkpointing
**70 JSONL training files** covering: greetings, feelings, play, school, family, friends, animals, imagination, food, health, safety, identity, culture, and more.
## Files
- `checkpoints/v3_best.safetensors` — Best v3 model (71MB)
- `checkpoints/best.safetensors` — Best v1 model (80MB)
- `checkpoints/full_best.safetensors` — Full curriculum model (36MB)
- `checkpoints/curriculum_step_*.safetensors` — Every curriculum checkpoint
- `model.py` — MicroBrain architecture
- `tokenizer.py` — Custom tokenizer
- `chat.py` — Inference + generation
## Usage
```python
import mlx.core as mx
from model import MicroBrain
from chat import SimpleTokenizer, generate_greedy
# Load model + tokenizer
tokenizer = SimpleTokenizer.from_file("checkpoints/v3_tokenizer.json")
model = MicroBrain(vocab_size=len(tokenizer), d_model=512, n_heads=8, n_layers=8)
model.load_weights("checkpoints/v3_best.safetensors")
# Generate
response = generate_greedy(model, tokenizer, "How are you feeling?")
print(response)
```
## Built With
- **MLX** — Apple's machine learning framework for Apple Silicon
- **TinkyOven** — Custom macOS SwiftUI app for visual playlist-style training
- **TinkyBrain** — Chrome extension for harvesting training data from the web
## Part of the Tinky Ecosystem
TinkyBrain powers the AAC speech prosthetics in:
- **TinkySpeak** — Android AAC device for non-verbal users
- **TinkyAAC** — iOS/Android speech prosthetic for stroke survivors
- **TinkyTown** — Accessibility kiosk for municipal buildings
- **TinkyAsk** — Universal business ordering system
Built by a dad who wanted nothing more then his non verbal son and his mom who who had a stroke able to speak.
---
*No cloud GPUs were harmed in the making of this model.*
|