--- language: - multilingual license: cc-by-nc-4.0 tags: - speech - language-identification - coreml - ios - mms - wav2vec2 - quantization - joint - int8 datasets: - facebook/mms --- # MMS-LID 1024 (Core ML, 8-bit + INT8 LUT Joint) Core ML conversion of **facebook/mms-lid-1024** for on-device speech language identification. This variant uses **8-bit k-means palettization plus INT8 LUT joint optimization** (iOS 18+): good balance of size, speed, and accuracy. - **Source:** [facebook/mms-lid-1024](https://huggingface.co/facebook/mms-lid-1024) - **Input:** Raw 16 kHz mono waveform, **fixed 10 seconds** (160,000 samples), shape `(1, 160000)` float32 - **Output:** Logits shape `(1, 1024)`; `argmax` → class index. Map to ISO 639-3 via `labels.json` or `mms_lid_id2label.json` ## Contents | File | Description | |------|-------------| | `mms_lid_joint.mlpackage` | Core ML model (8-bit kmeans + INT8 LUT joint, iOS 18+) | | `labels.json` | Ordered list of 1024 ISO 639-3 language codes | | `mms_lid_id2label.json` | Index → language code mapping | ## When to use this variant - Target **iOS 18+** and want a single quantized model with good accuracy/size trade-off. - In runtime tests, 8bit-int8 may differ from base/8bit on a few languages (e.g. Euskara, Yorùbá, Russian); use base or 8bit if you need maximum agreement with PyTorch. ## Usage on iOS / macOS Same as the base model: load the `.mlpackage`, feed 10 s of 16 kHz mono as `input_values`, take `argmax` of `logits`, and look up the language in `labels.json`. Requires iOS 18+ for full joint optimization support. ## Limitations Same as base: fixed 10 s input, L2 accent misclassification, English ↔ Hawaiian/Maori confusion. Slightly higher divergence from PyTorch than base/8bit on some files; use chunking and confidence threshold where appropriate. ## Mac smoke test (Core ML) On-device smoke run: each file under `INPUT/audio` was resampled to **16 kHz mono float32**, **padded or trimmed to 160,000 samples (10 s)**, then passed to `input_values`; `pred` is **ISO 639-3** from `argmax(logits)`; `conf` is softmax mass on the predicted class (runner-side). **Note:** Filenames are hints only (e.g. `English.mp3` is not ground truth). Low `conf` or known MMS-LID confusions (e.g. English vs `haw`) may still appear.
Raw runner log ``` MMS-LID 1024 Core ML — Mac smoke test Model: https://huggingface.co/aoiandroid/mms-lid-1024-coreml-joint Model dir: $PROJECT_ROOT/Log/mms_lid_1024_joint_mac_test/model_repo Audio dir: $PROJECT_ROOT/INPUT/audio Compiled temp: /var/folders/ky/nmbswxzs0s79wdxndfw1y6wh0000gn/T/model_repo.mlmodelc Compute: MLComputeUnits(rawValue: 2) Input: input_values Output: logits Labels: 1024 Host: ams-macbook-air.local macOS: Version 26.3.1 (a) (Build 25D771280a) English.mp3 pcm_samples=9054841 pred=haw conf=0.2406 max_logit=7.3984 time_ms=1185.3 Euskara.mp3 pcm_samples=1865769 pred=hin conf=0.3924 max_logit=8.8438 time_ms=412.6 Guaraní.mp3 pcm_samples=1682285 pred=grn conf=0.9992 max_logit=14.5703 time_ms=417.0 Yorùbá.mp3 pcm_samples=1067049 pred=haw conf=0.8309 max_logit=9.7266 time_ms=384.9 afrikaasns.mp3 pcm_samples=2387800 pred=nld conf=0.9994 max_logit=14.9297 time_ms=445.0 arabic.mp3 pcm_samples=2060120 pred=ara conf=0.9979 max_logit=13.6328 time_ms=431.3 bengali.m4a pcm_samples=7836432 pred=ben conf=0.9976 max_logit=14.0703 time_ms=589.6 chinese.mp3 pcm_samples=12904245 pred=cmn conf=0.9993 max_logit=14.3359 time_ms=1314.8 isiZulu.mp3 pcm_samples=1396819 pred=heb conf=0.3127 max_logit=7.0078 time_ms=400.7 kiswahili.mp3 pcm_samples=1888757 pred=swh conf=0.9988 max_logit=14.1484 time_ms=416.3 korean.mp3 pcm_samples=2364395 pred=kor conf=0.9994 max_logit=15.0938 time_ms=448.3 russinan.m4a pcm_samples=15431029 pred=rus conf=0.2187 max_logit=7.2305 time_ms=835.2 test.mp3 pcm_samples=274560 pred=jpn conf=0.9987 max_logit=14.6172 time_ms=346.4 日本語.mp3 pcm_samples=1798234 pred=jpn conf=0.9988 max_logit=14.6250 time_ms=484.4 ```
## License CC-BY-NC-4.0 (inherited from [facebook/mms-lid-1024](https://huggingface.co/facebook/mms-lid-1024)). ## Citation ```bibtex @article{pratap2023mms, title={Scaling Speech Technology to 1,000+ Languages}, author={Pratap, Vineel and others}, journal={arXiv preprint arXiv:2305.13516}, year={2023} } ```