Instructions to use MERaLiON/MERaLiON-2-10B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MERaLiON/MERaLiON-2-10B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="MERaLiON/MERaLiON-2-10B", trust_remote_code=True)# Load model directly from transformers import AutoModelForSpeechSeq2Seq model = AutoModelForSpeechSeq2Seq.from_pretrained("MERaLiON/MERaLiON-2-10B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Remove flash-attn from requirements and GPU inference example
Browse filesRemove flash-attn as a required dependency and remove attn_implementation="flash_attention_2" from the GPU inference example.
The model works with PyTorch's built-in SDPA attention which is auto-selected by transformers when flash-attn is not installed.
README.md
CHANGED
|
@@ -535,16 +535,6 @@ pip install transformers==4.50.1
|
|
| 535 |
pip install librosa
|
| 536 |
```
|
| 537 |
|
| 538 |
-
To run in GPU, MERaLiON-2 requires `flash-attn`.
|
| 539 |
-
|
| 540 |
-
```
|
| 541 |
-
pip install flash-attn --no-build-isolation
|
| 542 |
-
```
|
| 543 |
-
|
| 544 |
-
> [!TIP]
|
| 545 |
-
> Should you face any difficulties installing the above packages, you can try installing within this Docker container instead:
|
| 546 |
-
> `pytorch/pytorch:2.5.1-cuda12.1-cudnn9-devel`, whose cuda and torch environments have been tested working.
|
| 547 |
-
|
| 548 |
### Audio Input
|
| 549 |
|
| 550 |
- For ASR tasks, the maximum audio length is suggested to be 30 seconds at 16,000 Hz.
|
|
@@ -664,7 +654,6 @@ model = AutoModelForSpeechSeq2Seq.from_pretrained(
|
|
| 664 |
repo_id,
|
| 665 |
use_safetensors=True,
|
| 666 |
trust_remote_code=True,
|
| 667 |
-
attn_implementation="flash_attention_2",
|
| 668 |
torch_dtype=torch.bfloat16
|
| 669 |
).to(device)
|
| 670 |
|
|
|
|
| 535 |
pip install librosa
|
| 536 |
```
|
| 537 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 538 |
### Audio Input
|
| 539 |
|
| 540 |
- For ASR tasks, the maximum audio length is suggested to be 30 seconds at 16,000 Hz.
|
|
|
|
| 654 |
repo_id,
|
| 655 |
use_safetensors=True,
|
| 656 |
trust_remote_code=True,
|
|
|
|
| 657 |
torch_dtype=torch.bfloat16
|
| 658 |
).to(device)
|
| 659 |
|