Add ElevenLabs scribe_v2 benchmark on CoRal v3 test
Browse files
README.md
CHANGED
|
@@ -23,13 +23,15 @@ Danish ASR model — a 2B-parameter Conformer encoder-decoder trained on ~3.5M s
|
|
| 23 |
|
| 24 |
## Results on CoRal v3 test
|
| 25 |
|
| 26 |
-
| Split | Baseline WER | Baseline CER | **v5.1 WER** | **v5.1 CER** |
|
| 27 |
-
|---|---:|---:|---:|---:|
|
| 28 |
-
| `read_aloud` | 104.73% | 60.05% | **19.45%** | **7.24%** |
|
| 29 |
-
| `conversation` | 126.12% | 99.84% | **25.46%** | **14.08%** |
|
| 30 |
|
| 31 |
WER drop of **85 pp** on read-aloud and **101 pp** on conversational speech.
|
| 32 |
|
|
|
|
|
|
|
| 33 |
## Usage
|
| 34 |
|
| 35 |
```python
|
|
|
|
| 23 |
|
| 24 |
## Results on CoRal v3 test
|
| 25 |
|
| 26 |
+
| Split | Baseline WER | Baseline CER | **v5.1 WER** | **v5.1 CER** | ElevenLabs scribe_v2 WER | ElevenLabs scribe_v2 CER |
|
| 27 |
+
|---|---:|---:|---:|---:|---:|---:|
|
| 28 |
+
| `read_aloud` | 104.73% | 60.05% | **19.45%** | **7.24%** | 18.62% | 7.60% |
|
| 29 |
+
| `conversation` | 126.12% | 99.84% | **25.46%** | **14.08%** | 31.38% | 19.57% |
|
| 30 |
|
| 31 |
WER drop of **85 pp** on read-aloud and **101 pp** on conversational speech.
|
| 32 |
|
| 33 |
+
ElevenLabs `scribe_v2` evaluated via the public `/v1/speech-to-text` API on the full CoRal v3 test splits (n=17,560) with strict normalization (lowercase + punctuation strip + Danish digit-to-word via `num2words(lang="da")`).
|
| 34 |
+
|
| 35 |
## Usage
|
| 36 |
|
| 37 |
```python
|