sulabhkatiyar
/

ne-asr-mjw

@@ -1,175 +1,150 @@
 ---
 license: cc-by-nc-4.0
-language:
-  - mjw
-base_model: facebook/mms-1b-all
 library_name: transformers
 pipeline_tag: automatic-speech-recognition
 tags:
-  - speech
-  - asr
   - automatic-speech-recognition
   - mms
-  - mjw
   - low-resource
-  - northeast-india
-  - latin-script
-  - ctc-adapter
 datasets:
   - sulabhkatiyar/ne-asr-mjw
   - sulabhkatiyar/ne-asr-mjw-aug
-metrics:
-  - wer
-  - cer
 model-index:
   - name: ne-asr-mjw
     results:
       - task:
           type: automatic-speech-recognition
         dataset:
-          name: ne-asr-mjw (test split)
           type: sulabhkatiyar/ne-asr-mjw
         metrics:
           - type: wer
-            value: 61.3
           - type: cer
             value: 15.23
 ---
-# NE-ASR · Karbi (mjw) · MMS-1B CTC adapter
-A per-language CTC adapter for `facebook/mms-1b-all`, fine-tuned on the Romanized Karbi (mjw) split of the **NE-ASR** corpus. The base model is frozen; only the language adapter (~2.19 M parameters, ≈8.8 MB) and the CTC LM head are trained. Adapter weights are stored as `adapter.mjw.safetensors` (290 tensors).
-> **License:** CC-BY-NC 4.0, inherited from MMS-1B (Pratap et al., 2023). For non-commercial research use.
-## TL;DR
-| Metric (test) | Value |
-|---|---:|
-| WER (%) | **61.3** |
-| CER (%) | 15.23 |
-| n (test) | 39 |
-| Trainable params | 2,188,317 |
-| Adapter file size | ~8.8 MB |
-## What the model does
-Transcribes 16 kHz speech in Karbi (mjw) to Latin-script (Romanized) text. Output is lower-cased, NFC-normalized, no diacritics.
-## Data
-| Split | Rows | Source repo |
-|---|---:|---|
-| train (orig)  | 508 | `sulabhkatiyar/ne-asr-mjw` |
-| train (aug)   | 1,524      | `sulabhkatiyar/ne-asr-mjw-aug` (3× speed only (tonal — pitch-shift disabled)) |
-| validation    | 60   | `sulabhkatiyar/ne-asr-mjw` |
-| test          | 39       | `sulabhkatiyar/ne-asr-mjw` |
-Augmentation factor and recipe (per `configs/augmentation_config.yaml`): **3× — speed only (tonal — pitch-shift disabled)**. Online SpecAugment is additionally applied per-step during training (per-lang `mask_time_prob`, `mask_feature_prob`, `layerdrop`).
 ## Training
-| Hyperparameter | Value |
-|---|---|
-| Base model       | `facebook/mms-1b-all` |
-| Backbone         | frozen |
-| Adapter          | per-language CTC adapter, fresh init (`init_adapter_layers()`), Latin CTC head |
-| Precision        | bf16 |
-| Hardware         | AMD MI300X (gfx942), ROCm 7.0 |
-| per_device_train_batch_size | 16 |
-| gradient_accumulation_steps | 2 |
-| Effective batch size | 32 |
-| Learning rate    | 0.0003 |
-| Epochs           | 12 |
-| Warmup ratio     | 0.10 |
-| Group-by-length  | True |
-| Max input length (s) | 30 |
-| Trainable params | 2,188,317 (~2.19 M) |
-| Final train loss | 3.453 |
-| Peak VRAM        | 48.0 GB |
 ## Evaluation
-| Split | n | WER (%) | CER (%) |
-|---|---:|---:|---:|
-| validation (in-loop, full split n=60) | 60 | 61.79 | 16.51 |
-| test (full split) | 39 | **61.3** | 15.23 |
-Reference processing: NFC + strip + lower. Hypothesis decoded with greedy CTC, no language model.
 ## Usage
 ```python
-from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
-from safetensors.torch import load_file
 from huggingface_hub import hf_hub_download
 import torch, torchaudio
 REPO = "sulabhkatiyar/ne-asr-mjw"
-ISO  = "mjw"
-# Processor (tokenizer + feature extractor) is uploaded into this adapter repo
-processor = Wav2Vec2Processor.from_pretrained(REPO)
-# Load MMS-1B base, then re-initialise adapter layers
 model = Wav2Vec2ForCTC.from_pretrained(
-    "facebook/mms-1b-all",
     ignore_mismatched_sizes=True,
-    target_lang=ISO,
 )
 model.init_adapter_layers()
-# Pull the adapter safetensors from this repo
-adapter_path = hf_hub_download(REPO, f"adapter.{ISO}.safetensors")
-state = load_file(adapter_path)
-missing, unexpected = model.load_state_dict(state, strict=False)
-assert unexpected == [], f"Unexpected adapter keys: {unexpected[:5]}"
 model.eval()
-# Inference
 wav, sr = torchaudio.load("your_audio.wav")
 if sr != 16000:
     wav = torchaudio.functional.resample(wav, sr, 16000)
-inputs = processor(wav.squeeze().numpy(), sampling_rate=16000, return_tensors="pt")
 with torch.no_grad():
-    logits = model(**inputs).logits
-pred_ids = logits.argmax(-1)
-print(processor.batch_decode(pred_ids)[0])
 ```
-## History
-Trained as part of Tier-3 group 2 (Phase D) on AMD MI300X. Karbi (mjw) is tonal — **pitch-shift augmentation was disabled** per the project-wide tonal recipe; only speed perturbation (0.9× / 1.0× / 1.1×) and online SpecAugment were applied at 3×.
-### Dataset lineage (B.5.4 rebuild)
-This adapter was trained against the **B.5.4 rebuild** of `sulabhkatiyar/ne-asr-mjw-aug` (Hub HEAD `9f352fe6593feb02ec02f51c7def3a8fe4fa3ca6`). The prior v1.0 of that aug dataset was 5× pitch-polluted (train=2,540 = 5 × 508), which violated the project's no-pitch-shift rule for tonal languages. After the project-wide tonal recipe fix (Phase B.5.1), the aug dataset was rebuilt to **3× speed-only (train=1,524 = 3 × 508)** — 508 augmentations each of `original`, `speed_0.9`, and `speed_1.1`, with zero pitch entries. This card describes the adapter trained against the rebuilt 3× aug dataset; no earlier mjw adapter was published.
-### Best Tier-3 result in the corpus
-Test WER 61.30% (n=39) is the **strongest Tier-3 result in this 15-language project**, beating every other Tier-3 language by a comfortable margin and sitting at or below the Tier-2 cluster (67–75% test WER). Validation WER 61.79% (n=60) corroborates the test signal.
-## Limitations
-- Decoded greedy; no language model. Adding a kenLM rescoring step can improve WER.
-- Trained on the NE-ASR corpus (read speech, 16 kHz, Vaani domain) — performance on conversational or noisy audio is not characterised.
-- Output is Romanized; no native-script support.
-- Test split is held out from the same domain; out-of-domain audio may show larger error rates.
-## Citation
 ```bibtex
-@inproceedings{pratap2023mms,
-  title={Scaling Speech Technology to 1,000+ Languages},
-  author={Pratap, Vineel and others},
-  booktitle={arXiv:2305.13516},
-  year={2023}
-}
-@misc{nesar2026,
-  title={NE-ASR: ASR adapters for 15 Northeast-Indian languages},
-  author={Katiyar, Sulabh},
-  year={2026},
-  publisher={HuggingFace},
-  howpublished={\url{https://huggingface.co/sulabhkatiyar/ne-asr-mjw}}
 }
 ```

 ---
+language: mjw
 license: cc-by-nc-4.0
 library_name: transformers
 pipeline_tag: automatic-speech-recognition
 tags:
   - automatic-speech-recognition
+  - audio
+  - speech
+  - ctc
   - mms
   - low-resource
+  - karbi
+base_model: facebook/mms-1b-all
 datasets:
   - sulabhkatiyar/ne-asr-mjw
   - sulabhkatiyar/ne-asr-mjw-aug
 model-index:
   - name: ne-asr-mjw
     results:
       - task:
           type: automatic-speech-recognition
         dataset:
+          name: sulabhkatiyar/ne-asr-mjw
           type: sulabhkatiyar/ne-asr-mjw
+          split: test
         metrics:
           - type: wer
+            value: 61.30
           - type: cer
             value: 15.23
 ---
+# NE-ASR: Karbi (mjw)
+Romanized Karbi CTC ASR adapter for [`facebook/mms-1b-all`](https://huggingface.co/facebook/mms-1b-all).
+## Model description
+This is a per-language CTC adapter for the MMS-1B model. The base model is frozen;
+only the language adapter (about 2.19 M parameters, roughly 8.8 MB on disk) and the
+CTC head are trained. Input is 16 kHz mono audio (up to 30 seconds). Output is a
+Romanized (Latin-script) transcript, lower-cased and NFC-normalized.
+## Datasets
+- Training: [`sulabhkatiyar/ne-asr-mjw`](https://huggingface.co/datasets/sulabhkatiyar/ne-asr-mjw)
+- Augmented training: [`sulabhkatiyar/ne-asr-mjw-aug`](https://huggingface.co/datasets/sulabhkatiyar/ne-asr-mjw-aug)
+| split | n |
+|-------|--:|
+| train | 1,524 |
+| val   | 60   |
+| test  | 39  |
+The `train` count above refers to the augmented training set actually used for
+fine-tuning; the un-augmented source corpus is available at the first dataset link.
 ## Training
+- Base model: `facebook/mms-1b-all` (frozen; only the language adapter and CTC head are trained)
+- Epochs: 12
+- Batch size: 16 (effective batch size 32 with gradient accumulation = 2)
+- Learning rate: 3e-4
+- Warmup ratio: 0.10
+- Precision: bf16
+- Augmentation: 3x speed perturbation only (pitch shift disabled for this tonal language), plus online SpecAugment
+- Audio: 16 kHz mono; max input 30 seconds
+- Hardware: AMD MI300X (ROCm)
 ## Evaluation
+| split | n        | WER%       | CER%       |
+|-------|---------:|-----------:|-----------:|
+| val   | 60  | 61.79 | 15.23 |
+| test  | 39 | 61.30 | 15.23 |
+Reference processing: NFC + strip + lower. Hypothesis decoded with greedy CTC,
+no language model.
 ## Usage
 ```python
 from huggingface_hub import hf_hub_download
+from transformers import Wav2Vec2CTCTokenizer, Wav2Vec2FeatureExtractor, Wav2Vec2Processor, Wav2Vec2ForCTC
+from safetensors.torch import load_file
 import torch, torchaudio
 REPO = "sulabhkatiyar/ne-asr-mjw"
+BASE = "facebook/mms-1b-all"
+# Tokenizer and processor come from the adapter repo
+_ = hf_hub_download(REPO, "tokenizer_config.json")
+_ = hf_hub_download(REPO, "vocab.json")
+tokenizer = Wav2Vec2CTCTokenizer.from_pretrained(REPO, do_lower_case=False)
+feat_ext  = Wav2Vec2FeatureExtractor.from_pretrained(BASE)
+processor = Wav2Vec2Processor(feature_extractor=feat_ext, tokenizer=tokenizer)
+# Load the base model and attach the adapter weights
 model = Wav2Vec2ForCTC.from_pretrained(
+    BASE,
+    vocab_size=len(tokenizer),
+    pad_token_id=tokenizer.pad_token_id,
     ignore_mismatched_sizes=True,
 )
 model.init_adapter_layers()
+adapter_path = hf_hub_download(REPO, "adapter.mjw.safetensors")
+missing, unexpected = model.load_state_dict(load_file(adapter_path), strict=False)
 model.eval()
+# Inference (expects 16 kHz mono float32)
 wav, sr = torchaudio.load("your_audio.wav")
 if sr != 16000:
     wav = torchaudio.functional.resample(wav, sr, 16000)
+inputs = processor(wav.squeeze(0).numpy(), sampling_rate=16000, return_tensors="pt")
 with torch.no_grad():
+    logits = model(inputs.input_values).logits
+pred_ids = logits.argmax(dim=-1)
+transcription = processor.batch_decode(pred_ids)[0]
+print(transcription)
 ```
+## Citation
+If you use this model, please cite the MMS base model:
+```bibtex
+@article{pratap2023scaling,
+  title   = {Scaling Speech Technology to 1,000+ Languages},
+  author  = {Pratap, Vineel and Tjandra, Andros and Shi, Bowen and Tomasello, Paden and Babu, Arun and Kundu, Sayani and Elkahky, Ali and Ni, Zhaoheng and Vyas, Apoorv and Fazel-Zarandi, Maryam and Baevski, Alexei and Adi, Yossi and Zhang, Xiaohui and Hsu, Wei-Ning and Conneau, Alexis and Auli, Michael},
+  journal = {arXiv preprint arXiv:2305.13516},
+  year    = {2023}
+}
+```
+And the NE-ASR adapter release (placeholder; replace when the canonical publication is available):
 ```bibtex
+@misc{katiyar2026neasr,
+  author       = {Katiyar, Sulabh},
+  title        = {NE-ASR: MMS-1B Adapters for Northeast Indian Languages},
+  year         = {2026},
+  howpublished = {\url{https://huggingface.co/sulabhkatiyar/ne-asr-mjw}},
+  note         = {Placeholder citation; replace with the canonical publication when available.}
 }
 ```
+## License
+CC-BY-NC 4.0. This adapter is derived from [`facebook/mms-1b-all`](https://huggingface.co/facebook/mms-1b-all), which is released under CC-BY-NC 4.0. When you use this adapter, you must comply with the MMS license terms.