Instructions to use allenai/OLMo-2-1124-7B-Instruct-preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use allenai/OLMo-2-1124-7B-Instruct-preview with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="allenai/OLMo-2-1124-7B-Instruct-preview")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-2-1124-7B-Instruct-preview")
model = AutoModelForCausalLM.from_pretrained("allenai/OLMo-2-1124-7B-Instruct-preview", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use allenai/OLMo-2-1124-7B-Instruct-preview with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "allenai/OLMo-2-1124-7B-Instruct-preview"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "allenai/OLMo-2-1124-7B-Instruct-preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/allenai/OLMo-2-1124-7B-Instruct-preview

SGLang

How to use allenai/OLMo-2-1124-7B-Instruct-preview with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "allenai/OLMo-2-1124-7B-Instruct-preview" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "allenai/OLMo-2-1124-7B-Instruct-preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "allenai/OLMo-2-1124-7B-Instruct-preview" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "allenai/OLMo-2-1124-7B-Instruct-preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use allenai/OLMo-2-1124-7B-Instruct-preview with Docker Model Runner:
```
docker model run hf.co/allenai/OLMo-2-1124-7B-Instruct-preview
```

hamishivi commited on Nov 26, 2024

Commit

c3a7f30

verified ·

1 Parent(s): d3ab867

Update README.md

Browse files

Files changed (1) hide show

README.md +29 -8

README.md CHANGED Viewed

@@ -10,13 +10,13 @@ datasets:
 - allenai/RLVR-GSM
 ---
-<img src="https://allenai.org/olmo/olmo-7b-animation.gif" alt="OLMo Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
 # OLMo-2-1124-7B-Instruct
-OLMo-2 7B Instruct November 2024 is finetuned variant of the [OLMo-2 7B November 2024](https://huggingface.co/allenai/OLMo2-7B-1124) model, which has undergone supervised finetuning on the [Tülu 3 dataset](https://huggingface.co/datasets/allenai/tulu-3-sft-mixture) and further DPO training on [this dataset](https://huggingface.co/datasets/allenai/olmo-2-1124-7b-preference-mix), and finally RLVR training using [this data](https://huggingface.co/datasets/allenai/RLVR-GSM).
 Tülu 3 is designed for state-of-the-art performance on a diversity of tasks in addition to chat, such as MATH, GSM8K, and IFEval.
-Check out [the OLMo-2 paper](https://TODO) or [Tülu 3 paper](https://arxiv.org/abs/2411.15124) for more details!
 OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models.
 These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs (coming soon), and associated training details.
@@ -47,7 +47,7 @@ The core models released in this batch include the following:
     - Core repo (training, inference, fine-tuning etc.): https://github.com/allenai/OLMo
     - Evaluation code: https://github.com/allenai/olmes
     - Further fine-tuning code: https://github.com/allenai/open-instruct
-- **Paper:** Coming soon! TODO
 - **Demo:** https://playground.allenai.org/
 ## Using the model
@@ -92,7 +92,27 @@ See the Falcon 180B model card for an example of this.
 ## Performance
-TODO
 ## Hyperparameters
@@ -119,13 +139,14 @@ PPO settings for RLVR:
 ## License and use
-OLMo-2 is licensed under the Apache 2.0 license.
-OLMo-2 is intended for research and educational use.
 For more information, please see our [Responsible Use Guidelines](https://allenai.org/responsible-use).
 ## Citation
-If OLMo-2 or any of the related materials were helpful to your work, please cite:
 ```
 TODO
 ```

 - allenai/RLVR-GSM
 ---
+<img alt="OLMo Logo" src="https://huggingface.co/datasets/allenai/blog-images/resolve/main/olmo2/olmo.png" width="242px">
 # OLMo-2-1124-7B-Instruct
+OLMo 2 7B Instruct November 2024 is post-trained variant of the [OLMo-2 7B November 2024](https://huggingface.co/allenai/OLMo2-7B-1124) model, which has undergone supervised finetuning on an OLMo-specific variant of the [Tülu 3 dataset](allenai/tulu-3-sft-olmo-2-mixture)) and further DPO training on [this dataset](https://huggingface.co/datasets/allenai/olmo-2-1124-7b-preference-mix), and finally RLVR training using [this data](https://huggingface.co/datasets/allenai/RLVR-GSM).
 Tülu 3 is designed for state-of-the-art performance on a diversity of tasks in addition to chat, such as MATH, GSM8K, and IFEval.
+Check out the OLMo 2 paper (forthcoming) or [Tülu 3 paper](https://arxiv.org/abs/2411.15124) for more details!
 OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models.
 These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs (coming soon), and associated training details.
     - Core repo (training, inference, fine-tuning etc.): https://github.com/allenai/OLMo
     - Evaluation code: https://github.com/allenai/olmes
     - Further fine-tuning code: https://github.com/allenai/open-instruct
+- **Paper:** Coming soon!
 - **Demo:** https://playground.allenai.org/
 ## Using the model
 ## Performance
+| Model | Average | AlpacaEval | BBH | DROP | GSM8k | IFEval | MATH | MMLU | Safety | PopQA | TruthQA |
+|-------|---------|------------|-----|------|--------|---------|------|-------|---------|-------|---------|
+| **Open weights models** |
+| Gemma-2-9B-it | 51.9 | 43.7 | 2.5 | 58.8 | 79.7 | 69.9 | 29.8 | 69.1 | 75.5 | 28.3 | 61.4 |
+| Ministral-8B-Instruct | 52.1 | 31.4 | 56.2 | 56.2 | 80.0 | 56.4 | 40.0 | 68.5 | 56.2 | 20.2 | 55.5 |
+| Mistral-Nemo-Instruct-2407 | 51.1 | 45.8 | 56.0 | 23.6 | 81.4 | 64.5 | 31.9 | 70.0 | 52.7 | 26.9 | 57.7 |
+| Qwen-2.5-7B-Instruct | 57.1 | 29.7 | 25.3 | 54.4 | 83.8 | 74.7 | 69.9 | 76.6 | 75.0 | 18.1 | 63.1 |
+| Llama-3.1-8B-Instruct | 58.9 | 25.8 | 69.7 | 61.7 | 83.4 | 80.6 | 42.5 | 71.3 | 70.2 | 28.4 | 55.1 |
+| Tülu 3 8B | 60.4 | 34.0 | 66.0 | 62.6 | 87.6 | 82.4 | 43.7 | 68.2 | 75.4 | 29.1 | 55.0 |
+| Qwen-2.5-14B-Instruct | 61.0 | 34.6 | 35.4 | 50.5 | 83.9 | 82.4 | 70.6 | 81.1 | 79.3 | 21.1 | 70.8 |
+| **Fully open models** |
+| OLMo-7B-Instruct | 28.2 | 5.2 | 35.3 | 30.7 | 14.3 | 32.2 | 2.1 | 46.3 | 54.0 | 17.1 | 44.5 |
+| OLMo-7B-0424-Instruct | 33.2 | 8.5 | 35.2 | 47.9 | 23.2 | 39.2 | 5.2 | 48.9 | 49.3 | 18.9 | 55.2 |
+| OLMoE-1B-7B-0924-Instruct | 35.5 | 8.5 | 37.2 | 34.3 | 47.2 | 46.2 | 8.4 | 51.6 | 51.6 | 20.6 | 49.1 |
+| MAP-Neo-7B-Instruct | 42.9 | 17.6 | 26.4 | 48.2 | 69.4 | 35.9 | 31.5 | 56.5 | 73.7 | 18.4 | 51.6 |
+| *OLMo-2-7B-SFT* | 50.0 | 9.3 | 50.7 | 58.2 | 71.2 | 68.0 | 25.1 | 62.0 | 82.4 | 25.0 | 47.8 |
+| *OLMo-2-7B-DPO* | 55.0 | 29.9 | 47.0 | 58.8 | 82.4 | 74.5 | 31.2 | 63.4 | 81.5 | 24.5 | 57.2 |
+| *OLMo-2-13B-SFT* | 55.7 | 12.0 | 58.8 | 71.8 | 75.7 | 71.5 | 31.1 | 67.3 | 82.8 | 29.3 | 56.2 |
+| *OLMo-2-13B-DPO* | 61.0 | 38.3 | 58.5 | 71.9 | 84.2 | 80.6 | 35.0 | 68.5 | 80.6 | 28.9 | 63.9 |
+| **OLMo-2-7B-1124–Instruct** | 55.7 | 31.0 | 48.9 | 58.9 | 85.2 | 75.6 | 31.3 | 63.9 | 81.2 | 24.6 | 56.3 |
+| **OLMo-2-13B-1124-Instruct** | 61.4 | 37.5 | 58.4 | 72.1 | 87.4 | 80.4 | 39.7 | 68.6 | 77.5 | 28.8 | 63.9 |
 ## Hyperparameters
 ## License and use
+OLMo 2 is licensed under the Apache 2.0 license.
+OLMo 2 is intended for research and educational use.
 For more information, please see our [Responsible Use Guidelines](https://allenai.org/responsible-use).
+This model has been fine-tuned using a dataset mix with outputs generated from third party models and are subject to additional terms: [Gemma Terms of Use](https://ai.google.dev/gemma/terms).
 ## Citation
+If OLMo 2 or any of the related materials were helpful to your work, please cite:
 ```
 TODO
 ```