Instructions to use brittlewis12/Devstral-Small-2507-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use brittlewis12/Devstral-Small-2507-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf brittlewis12/Devstral-Small-2507-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf brittlewis12/Devstral-Small-2507-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf brittlewis12/Devstral-Small-2507-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf brittlewis12/Devstral-Small-2507-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf brittlewis12/Devstral-Small-2507-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf brittlewis12/Devstral-Small-2507-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf brittlewis12/Devstral-Small-2507-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf brittlewis12/Devstral-Small-2507-GGUF:Q4_K_M
Use Docker
docker model run hf.co/brittlewis12/Devstral-Small-2507-GGUF:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use brittlewis12/Devstral-Small-2507-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "brittlewis12/Devstral-Small-2507-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "brittlewis12/Devstral-Small-2507-GGUF", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/brittlewis12/Devstral-Small-2507-GGUF:Q4_K_M
- Ollama
How to use brittlewis12/Devstral-Small-2507-GGUF with Ollama:
ollama run hf.co/brittlewis12/Devstral-Small-2507-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use brittlewis12/Devstral-Small-2507-GGUF with Docker Model Runner:
docker model run hf.co/brittlewis12/Devstral-Small-2507-GGUF:Q4_K_M
- Lemonade
How to use brittlewis12/Devstral-Small-2507-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull brittlewis12/Devstral-Small-2507-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Devstral-Small-2507-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
File size: 3,791 Bytes
b55c019 036a4aa b55c019 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | ---
base_model: mistralai/Devstral-Small-2507
pipeline_tag: text-generation
inference: true
language:
- en
license: apache-2.0
model_creator: mistralai
model_name: Devstral-Small-2507
model_type: mistral
quantized_by: brittlewis12
tags:
- devstral
- coding
- mistral
---
# Devstral Small 1.1 GGUF
**Original model**: [Devstral Small 1.1](https://huggingface.co/mistralai/Devstral-Small-2507)
**Model creator**: [Mistral AI](https://huggingface.co/mistralai)
> Devstral is an agentic LLM for software engineering tasks built under a collaboration between [Mistral AI](https://mistral.ai/) and [All Hands AI](https://www.all-hands.dev/) 🙌. Devstral excels at using tools to explore codebases, editing multiple files and power software engineering agents. The model achieves remarkable performance on SWE-bench which positionates it as the #1 open source model on this benchmark.
>
> It is finetuned from [Mistral-Small-3.1](https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Base-2503), therefore it has a long context window of up to 128k tokens. As a coding agent, Devstral is text-only and before fine-tuning from `Mistral-Small-3.1` the vision encoder was removed.
This repo contains GGUF format model files for Mistral AI's _Devstral Small 1.1 (2507)_. Read more on Mistral’s [announcement blog post](https://mistral.ai/news/devstral-2507).
---
### What is GGUF?
GGUF is a file format for representing AI models. It is the third version of the format,
introduced by the ggml-org team on August 21st 2023.
Converted with llama.cpp build b5864 (revision [11ee0fe](https://github.com/ggml-org/llama.cpp/commits/11ee0fea2a24da1d3206eeba8fc52b759d9dfb24)), using [autogguf-rs](https://github.com/brittlewis12/autogguf-rs).
---
### Chat template: [Mistral 3 (v7-Tekken)](https://huggingface.co/mistralai/Mistral-Small-24B-Instruct-2501#basic-instruct-template-v7-tekken)
```
<s>[SYSTEM_PROMPT]{{system_message}}[/SYSTEM_PROMPT]
[INST]{{prompt}}[/INST]
{{assistant_response}}</s>
```
---
## Download & run with [cnvrs](https://twitter.com/cnvrsai) on iPhone, iPad, and Mac!

[cnvrs](https://testflight.apple.com/join/sFWReS7K) is the best app for private, local AI on your device:
- create & save **Characters** with custom system prompts & temperature settings
- download and experiment with any **GGUF model** you can [find on HuggingFace](https://huggingface.co/models?library=gguf)!
* or, use an API key with the chat completions-compatible model provider of your choice -- ChatGPT, Claude, Gemini, DeepSeek, & more!
- make it your own with custom **Theme colors**
- powered by Metal ⚡️ & [Llama.cpp](https://github.com/ggml-org/llama.cpp), with **haptics** during response streaming!
- **try it out** yourself today, on [Testflight](https://testflight.apple.com/join/sFWReS7K)!
* if you **already have the app**, download Devstral Small 2507 now!
* <cnvrsai:///models/search/hf?id=brittlewis12/devstral-small-2507-GGUF>
- follow [cnvrs on twitter](https://twitter.com/cnvrsai) to stay up to date
---
## Original Model Evaluation
| Model | Agentic Scaffold | SWE-Bench Verified (%) |
|--------------------|--------------------|------------------------|
| Devstral Small 1.1 | OpenHands Scaffold | **53.6** |
| Devstral Small 1.0 | OpenHands Scaffold | *46.8* |
| GPT-4.1-mini | OpenAI Scaffold | 23.6 |
| Claude 3.5 Haiku | Anthropic Scaffold | 40.6 |
| SWE-smith-LM 32B | SWE-agent Scaffold | 40.2 |
| Skywork SWE | OpenHands Scaffold | 38.0 |
| DeepSWE | R2E-Gym Scaffold | 42.2 |
|