a-sasin commited on
Commit
96743a6
·
verified ·
1 Parent(s): 05d03df

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +187 -0
README.md ADDED
@@ -0,0 +1,187 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-4.0
3
+ language:
4
+ - en
5
+ - nl
6
+ - de
7
+ - pl
8
+ metrics:
9
+ - wer
10
+ base_model:
11
+ - nvidia/stt_en_fastconformer_ctc_large
12
+ pipeline_tag: automatic-speech-recognition
13
+ library_name: nemo
14
+ tags:
15
+ - automatic-speech-recognition
16
+ - speech
17
+ - audio
18
+ - nemo
19
+ - fastconformer
20
+ - mixture-of-experts
21
+ - multilingual
22
+ - child-speech
23
+ - adult-speech
24
+ - inclusive-ai
25
+
26
+ ---
27
+
28
+ # Inclusive ASR MoE
29
+
30
+ This repository contains the final NVIDIA NeMo automatic speech recognition checkpoints developed by **Amelia Sasin** for the MSc thesis:
31
+
32
+ > **Mixture-of-Experts for Age-Inclusive ASR: Reducing the Adult-Child Performance Gap in Multilingual Speech Recognition**
33
+ > Delft University of Technology, 2026.
34
+
35
+ The models investigate inclusive and robust speech recognition for adult and child speech using dense FastConformer and sparse Mixture-of-Experts architectures.
36
+
37
+ Source code, training configurations, preprocessing scripts, evaluation scripts, and reproducibility instructions are available in the GitHub repository:
38
+
39
+ `a-sasin/inclusive-asr-moe`
40
+
41
+ ## Base model
42
+
43
+ All models were initialized directly or indirectly from:
44
+
45
+ `nvidia/stt_en_fastconformer_ctc_large`
46
+
47
+ The released checkpoints include further training, child-speech adaptation, multilingual training, and Mixture-of-Experts modifications performed by Amelia Sasin.
48
+
49
+ These models are independent research derivatives and are not endorsed by NVIDIA.
50
+
51
+ ## Available checkpoints
52
+
53
+ | Checkpoint | Language scope | Training population | Architecture |
54
+ | --------------------------------------- | ------------------------------ | ------------------- | ---------------------------- |
55
+ | `en_adult_fastconformer.nemo` | English | Adult | Dense FastConformer |
56
+ | `en_adult_moe.nemo` | English | Adult | Mixture of Experts |
57
+ | `en_child_fastconformer.nemo` | English | Child | Dense FastConformer |
58
+ | `en_child_moe_lb_off.nemo` | English | Child | MoE, load balancing disabled |
59
+ | `en_child_moe_lb_on.nemo` | English | Child | MoE, load balancing enabled |
60
+ | `multilingual_adult_fastconformer.nemo` | English, Dutch, German, Polish | Adult | Dense FastConformer |
61
+ | `multilingual_adult_moe.nemo` | English, Dutch, German, Polish | Adult | Mixture of Experts |
62
+ | `multilingual_child_fastconformer.nemo` | English, Dutch, German, Polish | Child | Dense FastConformer |
63
+ | `multilingual_child_moe_lb_off.nemo` | English, Dutch, German, Polish | Child | MoE, load balancing disabled |
64
+ | `multilingual_child_moe_lb_on.nemo` | English, Dutch, German, Polish | Child | MoE, load balancing enabled |
65
+
66
+ ## Downloading a checkpoint
67
+
68
+ ```python
69
+ from huggingface_hub import hf_hub_download
70
+
71
+ checkpoint_path = hf_hub_download(
72
+ repo_id="TUDelft/inclusive-asr-moe",
73
+ filename="en_adult_fastconformer.nemo",
74
+ )
75
+ ```
76
+
77
+ ## Loading a checkpoint
78
+
79
+ ```python
80
+ from nemo.collections.asr.models import ASRModel
81
+
82
+ model = ASRModel.restore_from(
83
+ restore_path=checkpoint_path,
84
+ )
85
+
86
+ model.eval()
87
+ transcriptions = model.transcribe(["example.wav"])
88
+ print(transcriptions)
89
+ ```
90
+
91
+ ## Required NeMo fork
92
+
93
+ The Mixture-of-Experts checkpoints require the custom NeMo fork developed for this work:
94
+
95
+ `a-sasin/NeMo`
96
+
97
+ This fork contains the sparse Mixture-of-Experts implementation used to train and restore the released MoE checkpoints.
98
+
99
+ Install it from source:
100
+
101
+ ```bash
102
+ git clone https://github.com/a-sasin/NeMo.git
103
+ cd NeMo
104
+ pip install -e ".[asr]"
105
+ ```
106
+
107
+ The dense FastConformer checkpoints may work with standard NVIDIA NeMo, but using this fork is recommended for consistency with the training environment.
108
+
109
+ ### Reproducible installation
110
+
111
+ For exact reproducibility, check out the same NeMo commit used during training:
112
+
113
+ ```bash
114
+ git clone https://github.com/a-sasin/NeMo.git
115
+ cd NeMo
116
+ git checkout <NEMO_COMMIT_HASH>
117
+ pip install -e ".[asr]"
118
+ ```
119
+
120
+
121
+ The Mixture-of-Experts checkpoints may require the custom NeMo implementation documented in the accompanying GitHub repository.
122
+
123
+ ## Data
124
+
125
+ The training and evaluation datasets, preprocessing procedures, manifest formats, and access instructions are documented in the accompanying GitHub repository and thesis.
126
+
127
+ Raw speech datasets are not included in this model repository. Users are responsible for obtaining the datasets from their original providers and complying with their respective licenses and access conditions.
128
+
129
+ ## Intended use
130
+
131
+ The checkpoints are intended for research involving:
132
+
133
+ * Automatic speech recognition
134
+ * Child and adult speech recognition
135
+ * Multilingual ASR
136
+ * Inclusive speech technology
137
+ * Mixture-of-Experts architectures
138
+ * Expert routing and load-balancing analysis
139
+
140
+ ## Limitations
141
+
142
+ Recognition performance may vary depending on:
143
+
144
+ * Speaker age
145
+ * Accent and dialect
146
+ * Language
147
+ * Speech characteristics
148
+ * Recording conditions
149
+ * Background noise
150
+ * Microphone quality
151
+ * Speaking style
152
+ * Domain-specific vocabulary
153
+
154
+ The terms “adult” and “child” describe the training and evaluation data. These models are not age-classification systems.
155
+
156
+ The models should not be treated as error-free or used without additional validation in medical, legal, emergency, educational-assessment, surveillance, or other high-stakes applications.
157
+
158
+ ## License
159
+
160
+ ### Model checkpoints
161
+
162
+ The model checkpoints are released under the **Creative Commons Attribution 4.0 International license**.
163
+
164
+ The checkpoints derive from NVIDIA’s `stt_en_fastconformer_ctc_large`, which is also distributed under CC BY 4.0.
165
+
166
+ When redistributing or adapting these checkpoints, retain attribution to:
167
+
168
+ * NVIDIA Corporation for the original checkpoint
169
+ * Amelia Sasin for the subsequent training, architecture modifications, experiments, and released checkpoints
170
+
171
+ ### Source code
172
+
173
+ Original source code authored for the thesis is licensed separately under the MIT License in the accompanying GitHub repository.
174
+
175
+ Copyright © 2025 Amelia Sasin.
176
+
177
+ ## Citation
178
+
179
+ ```bibtex
180
+ @mastersthesis{sasin2026inclusiveasr,
181
+ author = {Sasin, Amelia},
182
+ title = {Mixture-of-Experts for Age-Inclusive ASR: Reducing the Adult-Child Performance Gap in Multilingual Speech Recognition},
183
+ school = {Delft University of Technology},
184
+ year = {2026}
185
+ }
186
+ ```
187
+ ---