Instructions to use allenai/Flex-pes2o-2x7B-1T with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use allenai/Flex-pes2o-2x7B-1T with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="allenai/Flex-pes2o-2x7B-1T")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("allenai/Flex-pes2o-2x7B-1T")
model = AutoModelForCausalLM.from_pretrained("allenai/Flex-pes2o-2x7B-1T", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use allenai/Flex-pes2o-2x7B-1T with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "allenai/Flex-pes2o-2x7B-1T"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "allenai/Flex-pes2o-2x7B-1T",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/allenai/Flex-pes2o-2x7B-1T

SGLang

How to use allenai/Flex-pes2o-2x7B-1T with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "allenai/Flex-pes2o-2x7B-1T" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "allenai/Flex-pes2o-2x7B-1T",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "allenai/Flex-pes2o-2x7B-1T" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "allenai/Flex-pes2o-2x7B-1T",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use allenai/Flex-pes2o-2x7B-1T with Docker Model Runner:
```
docker model run hf.co/allenai/Flex-pes2o-2x7B-1T
```

nielsr HF Staff commited on Jul 14, 2025

Commit

f885b03

verified ·

1 Parent(s): 44240cc

Add pipeline tag and update paper link

Browse files

This PR improves the model card by:
- Adding the `pipeline_tag: text-generation` to the metadata, which helps users discover the model via the Hugging Face Hub's pipeline filter.
- Updating the paper link in the content to point directly to the Hugging Face Papers page ([FlexOlmo: Open Language Models for Flexible Data Use](https://huggingface.co/papers/2507.07024)) for better discoverability and integration within the Hugging Face ecosystem.

Files changed (1) hide show

README.md +5 -5

README.md CHANGED Viewed

@@ -1,13 +1,14 @@
 ---
-license: apache-2.0
 language:
 - en
 tags:
 - moe
 - olmo
 - flexolmo
 co2_eq_emissions: 1
-library_name: transformers
 ---
 <img alt="FlexOlmo Logo." src="FlexOlmo_Logo.png" width="500px" style="display: block; margin-left: auto; margin-right: auto; margin-top: 50px"> FlexOlmo is a new kind of LM that unlocks a new paradigm of data collaboration. With FlexOlmo, data owners can contribute to the development of open language models without giving up control of their data. There is no need to share raw data directly, and data contributors can decide when their data is active in the model, deactivate it at any time, and receive attributions whenever it's used for inference.
@@ -17,7 +18,7 @@ library_name: transformers
 > FlexOlmo-7x7B-1T (without router training) is a Mixture-of-Experts with 33B total parameters, combining independently trained experts on public-mix, news, math, code, academic texts, creative writing, and Reddit data. The public-mix expert is trained on 1T tokens of public data while the other experts are branched from the public-mix expert and trained on 50B tokens of their respective data.
 This information and more can also be found:
-- **Paper**: https://allenai.org/papers/flexolmo
 - **Code**: https://github.com/allenai/FlexOlmo
 - **Blog**: https://allenai.org/blog/flexolmo
 - **Data and corresponding models**:
@@ -62,7 +63,6 @@ print(tokenizer.decode(out[0]))
 * The evaluation of the individual model refers to the dense model, not the 2x7B MoE model.
 # Citation
 ```bibtex
 @misc{flexolmo,
@@ -74,4 +74,4 @@ print(tokenizer.decode(out[0]))
       primaryClass={cs.CL},
       url={https://allenai.org/papers/flexolmo},
 }
-```

 ---
 language:
 - en
+library_name: transformers
+license: apache-2.0
 tags:
 - moe
 - olmo
 - flexolmo
 co2_eq_emissions: 1
+pipeline_tag: text-generation
 ---
 <img alt="FlexOlmo Logo." src="FlexOlmo_Logo.png" width="500px" style="display: block; margin-left: auto; margin-right: auto; margin-top: 50px"> FlexOlmo is a new kind of LM that unlocks a new paradigm of data collaboration. With FlexOlmo, data owners can contribute to the development of open language models without giving up control of their data. There is no need to share raw data directly, and data contributors can decide when their data is active in the model, deactivate it at any time, and receive attributions whenever it's used for inference.
 > FlexOlmo-7x7B-1T (without router training) is a Mixture-of-Experts with 33B total parameters, combining independently trained experts on public-mix, news, math, code, academic texts, creative writing, and Reddit data. The public-mix expert is trained on 1T tokens of public data while the other experts are branched from the public-mix expert and trained on 50B tokens of their respective data.
 This information and more can also be found:
+- **Paper**: [FlexOlmo: Open Language Models for Flexible Data Use](https://huggingface.co/papers/2507.07024)
 - **Code**: https://github.com/allenai/FlexOlmo
 - **Blog**: https://allenai.org/blog/flexolmo
 - **Data and corresponding models**:
 * The evaluation of the individual model refers to the dense model, not the 2x7B MoE model.
 # Citation
 ```bibtex
 @misc{flexolmo,
       primaryClass={cs.CL},
       url={https://allenai.org/papers/flexolmo},
 }
+```