Automatic Speech Recognition
Transformers
Safetensors
phi4mm
text-generation
nlp
code
audio
speech-summarization
speech-translation
visual-question-answering
phi-4-multimodal
phi
phi-4-mini
custom_code
Eval Results
Instructions to use microsoft/Phi-4-multimodal-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use microsoft/Phi-4-multimodal-instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="microsoft/Phi-4-multimodal-instruct", trust_remote_code=True)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("microsoft/Phi-4-multimodal-instruct", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update readme
Browse files
README.md
CHANGED
|
@@ -46,6 +46,7 @@ widget:
|
|
| 46 |
- role: user
|
| 47 |
content: Can you provide ways to eat combinations of bananas and dragonfruits?
|
| 48 |
library_name: transformers
|
|
|
|
| 49 |
---
|
| 50 |
|
| 51 |
## Model Summary
|
|
@@ -407,8 +408,9 @@ model = AutoModelForCausalLM.from_pretrained(
|
|
| 407 |
model_path,
|
| 408 |
device_map="cuda",
|
| 409 |
torch_dtype="auto",
|
| 410 |
-
trust_remote_code=True,
|
| 411 |
-
|
|
|
|
| 412 |
).cuda()
|
| 413 |
|
| 414 |
# Load generation config
|
|
@@ -466,6 +468,8 @@ response = processor.batch_decode(
|
|
| 466 |
print(f'>>> Response\n{response}')
|
| 467 |
```
|
| 468 |
|
|
|
|
|
|
|
| 469 |
## Responsible AI Considerations
|
| 470 |
|
| 471 |
Like other language models, the Phi family of models can potentially behave in ways that are unfair, unreliable, or offensive. Some of the limiting behaviors to be aware of include:
|
|
@@ -561,7 +565,7 @@ Note that by default, the Phi-4-multimodal-instruct model uses flash attention,
|
|
| 561 |
* NVIDIA H100
|
| 562 |
|
| 563 |
If you want to run the model on:
|
| 564 |
-
* NVIDIA V100 or earlier generation GPUs: call AutoModelForCausalLM.from_pretrained() with
|
| 565 |
|
| 566 |
## License
|
| 567 |
The model is licensed under the [MIT license](./LICENSE).
|
|
|
|
| 46 |
- role: user
|
| 47 |
content: Can you provide ways to eat combinations of bananas and dragonfruits?
|
| 48 |
library_name: transformers
|
| 49 |
+
paper: arxiv.org/abs/2503.01743
|
| 50 |
---
|
| 51 |
|
| 52 |
## Model Summary
|
|
|
|
| 408 |
model_path,
|
| 409 |
device_map="cuda",
|
| 410 |
torch_dtype="auto",
|
| 411 |
+
trust_remote_code=True,
|
| 412 |
+
# if you do not Ampere or later GPUs, change attention to "eager"
|
| 413 |
+
_attn_implementation='flash_attention_2',
|
| 414 |
).cuda()
|
| 415 |
|
| 416 |
# Load generation config
|
|
|
|
| 468 |
print(f'>>> Response\n{response}')
|
| 469 |
```
|
| 470 |
|
| 471 |
+
**Notes**:
|
| 472 |
+
|
| 473 |
## Responsible AI Considerations
|
| 474 |
|
| 475 |
Like other language models, the Phi family of models can potentially behave in ways that are unfair, unreliable, or offensive. Some of the limiting behaviors to be aware of include:
|
|
|
|
| 565 |
* NVIDIA H100
|
| 566 |
|
| 567 |
If you want to run the model on:
|
| 568 |
+
* NVIDIA V100 or earlier generation GPUs: call AutoModelForCausalLM.from_pretrained() with _attn_implementation="eager"
|
| 569 |
|
| 570 |
## License
|
| 571 |
The model is licensed under the [MIT license](./LICENSE).
|