Instructions to use hehua2008/Mistral-Small-3.2-24B-Instruct-2506-MLX-mixed_3_4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use hehua2008/Mistral-Small-3.2-24B-Instruct-2506-MLX-mixed_3_4 with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("hehua2008/Mistral-Small-3.2-24B-Instruct-2506-MLX-mixed_3_4") config = load_config("hehua2008/Mistral-Small-3.2-24B-Instruct-2506-MLX-mixed_3_4") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Upload folder using huggingface_hub
Browse files- generation_config.json +3 -1
- tekken.json +2 -2
generation_config.json
CHANGED
|
@@ -2,5 +2,7 @@
|
|
| 2 |
"_from_model_config": true,
|
| 3 |
"bos_token_id": 1,
|
| 4 |
"eos_token_id": 2,
|
| 5 |
-
"transformers_version": "4.
|
|
|
|
|
|
|
| 6 |
}
|
|
|
|
| 2 |
"_from_model_config": true,
|
| 3 |
"bos_token_id": 1,
|
| 4 |
"eos_token_id": 2,
|
| 5 |
+
"transformers_version": "4.52.4",
|
| 6 |
+
"temperature": 0.15,
|
| 7 |
+
"do_sample": true
|
| 8 |
}
|
tekken.json
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6e2501687ccd0e1f30f36319eaf2b46958b897811e246cd8eb5d385b9e3de7d1
|
| 3 |
+
size 19399895
|