Automatic Speech Recognition
Transformers
Safetensors
meralion2
meralion
meralion-2
custom_code
YingxuHe commited on
Commit
ca3a11d
·
verified ·
1 Parent(s): 7ffd894

Remove flash-attn from requirements and GPU inference example

Browse files

Remove flash-attn as a required dependency and remove attn_implementation="flash_attention_2" from the GPU inference example.

The model works with PyTorch's built-in SDPA attention which is auto-selected by transformers when flash-attn is not installed.

Files changed (1) hide show
  1. README.md +0 -11
README.md CHANGED
@@ -535,16 +535,6 @@ pip install transformers==4.50.1
535
  pip install librosa
536
  ```
537
 
538
- To run in GPU, MERaLiON-2 requires `flash-attn`.
539
-
540
- ```
541
- pip install flash-attn --no-build-isolation
542
- ```
543
-
544
- > [!TIP]
545
- > Should you face any difficulties installing the above packages, you can try installing within this Docker container instead:
546
- > `pytorch/pytorch:2.5.1-cuda12.1-cudnn9-devel`, whose cuda and torch environments have been tested working.
547
-
548
  ### Audio Input
549
 
550
  - For ASR tasks, the maximum audio length is suggested to be 30 seconds at 16,000 Hz.
@@ -664,7 +654,6 @@ model = AutoModelForSpeechSeq2Seq.from_pretrained(
664
  repo_id,
665
  use_safetensors=True,
666
  trust_remote_code=True,
667
- attn_implementation="flash_attention_2",
668
  torch_dtype=torch.bfloat16
669
  ).to(device)
670
 
 
535
  pip install librosa
536
  ```
537
 
 
 
 
 
 
 
 
 
 
 
538
  ### Audio Input
539
 
540
  - For ASR tasks, the maximum audio length is suggested to be 30 seconds at 16,000 Hz.
 
654
  repo_id,
655
  use_safetensors=True,
656
  trust_remote_code=True,
 
657
  torch_dtype=torch.bfloat16
658
  ).to(device)
659