Instructions to use notbadai/notbad_v1_0_mistral_24b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use notbadai/notbad_v1_0_mistral_24b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="notbadai/notbad_v1_0_mistral_24b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("notbadai/notbad_v1_0_mistral_24b")
model = AutoModelForCausalLM.from_pretrained("notbadai/notbad_v1_0_mistral_24b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use notbadai/notbad_v1_0_mistral_24b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "notbadai/notbad_v1_0_mistral_24b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "notbadai/notbad_v1_0_mistral_24b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/notbadai/notbad_v1_0_mistral_24b

SGLang

How to use notbadai/notbad_v1_0_mistral_24b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "notbadai/notbad_v1_0_mistral_24b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "notbadai/notbad_v1_0_mistral_24b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "notbadai/notbad_v1_0_mistral_24b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "notbadai/notbad_v1_0_mistral_24b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use notbadai/notbad_v1_0_mistral_24b with Docker Model Runner:
```
docker model run hf.co/notbadai/notbad_v1_0_mistral_24b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Model Card for Notbad v1.0 Mistral 24B

📣 New model available: Notbad v1.1 Mistral 24B

Notbad v1.0 Mistral 24B is a reasoning model trained in math and Python coding. This model is built upon the Mistral-Small-24B-Instruct-2501. and has been further trained with reinforcement learning on math and coding.

One of the key features of Notbad v1.0 is its ability to produce shorter and cleaner reasoning outputs. We used open datasets and employed reinforcement learning techniques developed continuing from our work on Quiet Star, and are similar to Dr. GRPO. The reasoning capabilities in this model are from self-improvement and not distilled from any other model. It is the result of a fine-tuning from data sampled from multiple of our RL models starting with the Mistral-Small-24B-Instruct-2501.

Special thanks to Lambda and Deep Infra for providing help with compute resources for our research and training this model.

You can try the model on chat.labml.ai.