| --- |
| license: apache-2.0 |
| tags: |
| - awq |
| - AMD |
| - Ryzen AI |
| --- |
| |
| # Optmized LLM models for Ryzen AI. |
|
|
| The models are quantized & tested via [Ryzen AI SW 1.2](https://github.com/amd/RyzenAI-SW/tree/main/example/transformers). |
|
|
| The folder `quantized_models` contains a set of LLMs quantized via different algorithms. |
| The folder `onnx_model_nodes` contains the `mlp.up_proj` nodes for `-npugpu` device parallel config. |
|
|
| Due to the disordered Ryzen AI Python environment, you may need to install different versions of transformers: |
| ```python |
| transformers==4.37.2 # Others |
| transformers==4.39.1 # GEMMA |
| transformers>=4.42 # Mistral |
| ``` |
|
|
|
|
|
|
