Update README.md
Browse files
README.md
CHANGED
|
@@ -39,7 +39,7 @@ tags:
|
|
| 39 |
|
| 40 |
# Magistral Small 1.2
|
| 41 |
|
| 42 |
-
Building upon Mistral Small 3.2 (2506), **with added reasoning capabilities**, undergoing SFT from Magistral Medium traces and RL on top, it's a small, efficient reasoning model with 24B parameters.
|
| 43 |
|
| 44 |
Magistral Small can be deployed locally, fitting within a single RTX 4090 or a 32GB RAM MacBook once quantized.
|
| 45 |
|
|
@@ -47,7 +47,7 @@ Learn more about Magistral in our [blog post](https://mistral.ai/news/magistral/
|
|
| 47 |
|
| 48 |
The model was presented in the paper [Magistral](https://huggingface.co/papers/2506.10910).
|
| 49 |
|
| 50 |
-
## Updates compared with [Magistral Small 1.
|
| 51 |
|
| 52 |
- **Multimodality**: The model now has a vision encoder and can take multimodal inputs, extending its reasoning capabilities to vision.
|
| 53 |
- **Performance upgrade**: Magistral Small 1.2 should give you significatively better performance than Magistral Small 1.1 as seen in the [benchmark results](#benchmark-results).
|
|
@@ -86,10 +86,11 @@ Please make sure to use:
|
|
| 86 |
|
| 87 |
We highly recommend including the following system prompt for the best results, you can edit and customise it if needed for your specific use case.
|
| 88 |
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
> Your thinking process must follow the template below:[THINK]Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate the response. Use the same language as the input.[/THINK]Here, provide a self-contained response.
|
| 92 |
|
|
|
|
|
|
|
| 93 |
|
| 94 |
The `[THINK]` and `[/THINK]` are special tokens that **must** be encoded as such.
|
| 95 |
|
|
@@ -100,15 +101,17 @@ We invite you to choose, depending on your use case and requirements, between ke
|
|
| 100 |
|
| 101 |
## Usage
|
| 102 |
|
| 103 |
-
The model can be used with the following frameworks
|
| 104 |
|
| 105 |
### Inference
|
| 106 |
|
| 107 |
- [`vllm (recommended)`](https://github.com/vllm-project/vllm): See [below](#vllm-recommended)
|
| 108 |
- [`transformers`](https://github.com/huggingface/transformers): See [below](#transformers)
|
|
|
|
|
|
|
|
|
|
| 109 |
|
| 110 |
-
|
| 111 |
-
- [`llama.cpp`](https://github.com/ggml-org/llama.cpp): https://huggingface.co/mistralai/Magistral-Small-2509-GGUF
|
| 112 |
|
| 113 |
### vLLM (recommended)
|
| 114 |
|
|
|
|
| 39 |
|
| 40 |
# Magistral Small 1.2
|
| 41 |
|
| 42 |
+
Building upon [Mistral Small 3.2 (2506)](https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506), **with added reasoning capabilities**, undergoing SFT from Magistral Medium traces and RL on top, it's a small, efficient reasoning model with 24B parameters.
|
| 43 |
|
| 44 |
Magistral Small can be deployed locally, fitting within a single RTX 4090 or a 32GB RAM MacBook once quantized.
|
| 45 |
|
|
|
|
| 47 |
|
| 48 |
The model was presented in the paper [Magistral](https://huggingface.co/papers/2506.10910).
|
| 49 |
|
| 50 |
+
## Updates compared with [Magistral Small 1.1](https://huggingface.co/mistralai/Magistral-Small-2507)
|
| 51 |
|
| 52 |
- **Multimodality**: The model now has a vision encoder and can take multimodal inputs, extending its reasoning capabilities to vision.
|
| 53 |
- **Performance upgrade**: Magistral Small 1.2 should give you significatively better performance than Magistral Small 1.1 as seen in the [benchmark results](#benchmark-results).
|
|
|
|
| 86 |
|
| 87 |
We highly recommend including the following system prompt for the best results, you can edit and customise it if needed for your specific use case.
|
| 88 |
|
| 89 |
+
```py
|
| 90 |
+
First draft your thinking process (inner monologue) until you arrive at a response. Format your response using Markdown, and use LaTeX for any mathematical equations. Write both your thoughts and the response in the same language as the input.
|
|
|
|
| 91 |
|
| 92 |
+
Your thinking process must follow the template below:[THINK]Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate the response. Use the same language as the input.[/THINK]Here, provide a self-contained response.
|
| 93 |
+
```
|
| 94 |
|
| 95 |
The `[THINK]` and `[/THINK]` are special tokens that **must** be encoded as such.
|
| 96 |
|
|
|
|
| 101 |
|
| 102 |
## Usage
|
| 103 |
|
| 104 |
+
The model can be used with the following frameworks.
|
| 105 |
|
| 106 |
### Inference
|
| 107 |
|
| 108 |
- [`vllm (recommended)`](https://github.com/vllm-project/vllm): See [below](#vllm-recommended)
|
| 109 |
- [`transformers`](https://github.com/huggingface/transformers): See [below](#transformers)
|
| 110 |
+
- [`llama.cpp`](https://github.com/ggml-org/llama.cpp): See https://huggingface.co/mistralai/Magistral-Small-2509-GGUF
|
| 111 |
+
|
| 112 |
+
### Fine-tuning
|
| 113 |
|
| 114 |
+
- [`Axolotl`](https://axolotl.ai/): See https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/magistral
|
|
|
|
| 115 |
|
| 116 |
### vLLM (recommended)
|
| 117 |
|