| --- |
| license: apache-2.0 |
| language: en |
| tags: |
| - aac |
| - speech-prosthetic |
| - small-language-model |
| - mlx |
| - from-scratch |
| --- |
| |
| # TinkyBrain 31M — A Language Model Built From Scratch |
|
|
| **31 million parameters. Trained from zero. On a Mac mini M4.** |
|
|
| No fine-tuning. No pre-trained weights. No shortcuts. Every single weight in this model was learned from scratch on Apple Silicon using MLX. |
|
|
| ## What Makes This Special |
|
|
| Most "custom models" are LoRA adapters slapped on top of someone else's foundation model. TinkyBrain is different: |
|
|
| - **Custom architecture** — MicroBrain transformer built from scratch in `model.py` |
| - **Custom tokenizer** — grown organically during curriculum training, not borrowed from GPT/LLaMA |
| - **Custom training pipeline** — DJ playlist-style curriculum learning (70 tracks, added one at a time like ingredients in a recipe) |
| - **Trained entirely on Apple Silicon** — MLX on M4 Mac mini, no cloud GPU needed |
| - **Purpose-built for AAC** — designed to power speech prosthetics for non-verbal users (stroke survivors, neurodivergent individuals) |
|
|
| ## Training Approach |
|
|
| The playlist trainer (`train_playlist.py`) treats training data like a DJ set: |
| 1. Start with baby-level language (greetings, feelings, basic needs) |
| 2. Add one "track" at a time — 100 epochs per track on the cumulative dataset |
| 3. Each track builds on what came before — the model never forgets early lessons |
| 4. 14 curriculum stages from babbling to full conversation |
| 5. Smart early stopping with patience-based checkpointing |
|
|
| **70 JSONL training files** covering: greetings, feelings, play, school, family, friends, animals, imagination, food, health, safety, identity, culture, and more. |
|
|
| ## Files |
|
|
| - `checkpoints/v3_best.safetensors` — Best v3 model (71MB) |
| - `checkpoints/best.safetensors` — Best v1 model (80MB) |
| - `checkpoints/full_best.safetensors` — Full curriculum model (36MB) |
| - `checkpoints/curriculum_step_*.safetensors` — Every curriculum checkpoint |
| - `model.py` — MicroBrain architecture |
| - `tokenizer.py` — Custom tokenizer |
| - `chat.py` — Inference + generation |
|
|
| ## Usage |
|
|
| ```python |
| import mlx.core as mx |
| from model import MicroBrain |
| from chat import SimpleTokenizer, generate_greedy |
| |
| # Load model + tokenizer |
| tokenizer = SimpleTokenizer.from_file("checkpoints/v3_tokenizer.json") |
| model = MicroBrain(vocab_size=len(tokenizer), d_model=512, n_heads=8, n_layers=8) |
| model.load_weights("checkpoints/v3_best.safetensors") |
| |
| # Generate |
| response = generate_greedy(model, tokenizer, "How are you feeling?") |
| print(response) |
| ``` |
|
|
| ## Built With |
|
|
| - **MLX** — Apple's machine learning framework for Apple Silicon |
| - **TinkyOven** — Custom macOS SwiftUI app for visual playlist-style training |
| - **TinkyBrain** — Chrome extension for harvesting training data from the web |
|
|
| ## Part of the Tinky Ecosystem |
|
|
| TinkyBrain powers the AAC speech prosthetics in: |
| - **TinkySpeak** — Android AAC device for non-verbal users |
| - **TinkyAAC** — iOS/Android speech prosthetic for stroke survivors |
| - **TinkyTown** — Accessibility kiosk for municipal buildings |
| - **TinkyAsk** — Universal business ordering system |
|
|
| Built by a dad who wanted nothing more then his non verbal son and his mom who who had a stroke able to speak. |
|
|
| --- |
|
|
| *No cloud GPUs were harmed in the making of this model.* |
|
|
