Loading the model…
-The field
-Small and accurate rarely go together.blind test
-Every language detector we could run, on FLORES-200 cut to two words — text none of them trained on. Accuracy plotted against model size. The good corner is top-left: accurate and small. One model sits there.
-Head to head · their benchmark
-On lingua's home turf.
-lingua is the accuracy leader, and 266 MB. Here it is on its own published test data, the corpus it was tuned against. tongue's large flavour wins two of three cells anyway, at a fraction of a percent of the size.
-| Category | tongue 2MB | tongue large | lingua · 266MB |
|---|---|---|---|
| Single words | 0.745 | 0.759 | 0.752 |
| Word pairs | 0.908 | 0.914 | 0.916 |
| Sentences | 0.988 | 0.988 | 0.985 |
Head to head · a benchmark nobody here trained onblind test
-Including real tweets.
-A third team's benchmark (eld), never seen during development, with a tweets split of real casual text — the register a keyboard actually sees. tongue large against Apple's own detector and HeLI-OTS, on identical rows.
-| Category | tongue large | lingua | Apple | HeLI-OTS |
|---|---|---|---|---|
| Tweetscasual · 6 major langs | 0.995 | 0.984 | 0.997 | 0.986 |
| Single words | 0.770 | 0.752 | 0.636 | 0.683 |
| Word pairs | 0.892 | 0.892 | 0.714 | 0.846 |
| Sentences | 0.970 | 0.949 | 0.744 | 0.967 |
Apple ties on its six strongest languages, then drops off across the 51-language tail (Malay reads as Indonesian, Azerbaijani as Turkish). It is also iOS-only.
-Why it exists
-Language detection that fits inside the app.
-Good detectors already exist. Most of them run on a server or ship tens to hundreds of megabytes. tongue fills a smaller gap: something you can bundle into an iOS, Android or web app and run offline, with identical results on every platform. That's the niche it was built for.
- --
-
- Two of three benchmarks are blind. FLORES and eld were never seen in development and no tuning decision was made against them. The wins hold there, including on casual tweets — the real test that this isn't fit to a benchmark. -
- Short input is hard for everyone. "hi i am" reads as Welsh to any character model, and lingua calls it Swahili. On three ambiguous words there is little signal to find. tongue reports low confidence on inputs like these instead of asserting a guess; give it a few words and it's sure. -
- Our named weakness is Malay versus Indonesian — about 90% identical in text. We report it rather than bury it. -
- All training data is commercially clean: Tatoeba (CC-BY), Common Voice and Wikidata (CC0), permissive dictionaries. Nothing licensed, nothing scraped grey. -
FAQ
-Questions, answered.
-How does 2 MB beat a 266 MB model?
What's the architecture?
- A zero-parameter script layer settles anything Unicode already settles — Korean, Thai, Greek, Chinese and 20 more — at 99.9%, with no model involved.
- For Latin, Cyrillic, Arabic and Indic scripts, a hashed character-n-gram model: n-grams of 1 to 5 characters hashed into a fixed table, summed, then one linear layer. No tokenizer and no vocabulary file, which is why the output is byte-identical across Swift, Kotlin and JavaScript.
- It is deliberately not a transformer. That would break the size and cross-platform guarantees that are the whole point.
How was it trained — epochs, hardware, time?
- Hardware: a laptop CPU. No GPU, because the model is small enough that training waits on data, not compute.
- Time: 15 to 35 minutes per run, so dozens of experiments in a day.
- Epochs: 15 for the 2 MB model, 25 with a cosine schedule for the large one.
- Data: about 13 million short spans, windowed from full sentences into 1 to 8 word fragments, plus dictionary words for coverage. Plain cross-entropy. The craft is in the data and the deterministic layers, not the training loop.
83 languages — are they all real?
Is it just tuned to win these benchmarks?
Where is it weak?
Does it guess when unsure?
und (unknown) rather than a coin flip, and for near-identical clusters it can return a group answer — "one of Danish, Norwegian or Swedish" — which is useful and never a lie. The confidence bars in the demo above are the real probabilities.