Instructions to use s-sahoo/uno-qwen3-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use s-sahoo/uno-qwen3-8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="s-sahoo/uno-qwen3-8B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("s-sahoo/uno-qwen3-8B", trust_remote_code=True, device_map="auto")

PEFT
How to use s-sahoo/uno-qwen3-8B with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use s-sahoo/uno-qwen3-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "s-sahoo/uno-qwen3-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "s-sahoo/uno-qwen3-8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/s-sahoo/uno-qwen3-8B

SGLang

How to use s-sahoo/uno-qwen3-8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "s-sahoo/uno-qwen3-8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "s-sahoo/uno-qwen3-8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "s-sahoo/uno-qwen3-8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "s-sahoo/uno-qwen3-8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use s-sahoo/uno-qwen3-8B with Docker Model Runner:
```
docker model run hf.co/s-sahoo/uno-qwen3-8B
```

uno-qwen3-8B / README.md

s-sahoo

Add paper and project page links to model card (#2)

016d976 7 days ago

preview code

Raw

History Blame Contribute Delete

4.02 kB

	---
	base_model: IFM/uno-qwen3-8b-base
	library_name: transformers
	license: apache-2.0
	pipeline_tag: text-generation
	tags:
	- diffusion-language-model
	- qwen3
	- sdar
	- custom_code
	- peft
	- lora
	- speculative-decoding
	- uno
	---

	<h1 align="center"><strong>Uno Qwen3-8B</strong></h1>

	Paper: [Unlocking Lossless Speedups in LLMs via Discrete Diffusion](https://huggingface.co/papers/2609.04010)
	Project page: [https://s-sahoo.github.io/uno/](https://s-sahoo.github.io/uno/)

	![Diffusion-augmented LLM training and evaluation overview](./results_overview.png)

	This repository is a self-contained inference bundle for Uno-Qwen3-8B. The
	frozen verifier/base checkpoint is stored at the repository root, and the
	rank-128 gated LoRA adapter used for noisy draft rows is stored under
	[`adapter/`](./adapter).

	Uno applies the adapter selectively during draft-noise forwards. Seed, prefill,
	verification, and autoregressive rows use the frozen base weights. Loading the
	adapter as an ordinary always-on PEFT adapter does not reproduce Uno decoding.

	## Repository layout

	```text
	s-sahoo/uno-qwen3-8B
	\|-- config.json
	\|-- model-*.safetensors
	\|-- model.safetensors.index.json
	\|-- modeling_sdar.py
	\|-- tokenizer files
	`-- adapter/
	\|-- adapter_config.json
	`-- adapter_model.safetensors
	```

	## Evaluation protocol

	The reproducible entry points are the Qwen benchmark wrappers in
	[`ifm-ai/uno`](https://github.com/ifm-ai/uno/tree/main/scripts/qwen).
	Unless an experiment explicitly overrides a setting, the release wrappers use
	the following protocol:

	\| Setting \| Value \|
	\| --- \| --- \|
	\| Runtime \| Uno with conditional gated-LoRA routing \|
	\| Sampler \| Linear diffusion \|
	\| Diffusion block size \| 16 \|
	\| Temperature \| 1.0 \|
	\| Top-p \| 0.95 \|
	\| Top-k \| 50 \|
	\| Samples per problem \| 1 \|
	\| Noise mode \| `random_uniform` \|
	\| Served context length \| 32,768 tokens \|
	\| Maximum batched tokens \| 32,768 \|
	\| Maximum sequences per replica \| 64 \|
	\| Data / tensor parallel size \| 8 / 1 \|
	\| Attention backend \| FlashAttention 2 \|

	Math tasks use the instruction `Please reason step by step and put your final
	answer in \boxed{}.`. Benchmarks that request it in the release configuration
	use `reasoning_effort=high` through the chat template.

	\| Benchmark \| Problems \| Scoring \|
	\| --- \| ---: \| --- \|
	\| GSM8K \| 1,319 \| Mathematical answer equivalence \|
	\| MATH-500 \| 500 \| Mathematical answer equivalence \|
	\| AIME 2024 \| 30 \| Mathematical answer equivalence \|
	\| AIME 2025 \| 30 \| Mathematical answer equivalence \|
	\| AIME 2026 \| 30 \| Mathematical answer equivalence \|
	\| HumanEval \| 164 \| Code execution \|
	\| MBPP \| 500 \| Code execution \|
	\| LiveCodeBench v6 shard \| 175 \| Public and private test execution \|
	\| GPQA Main \| 448 \| Multiple-choice answer matching \|
	\| GPQA Diamond \| 198 \| Multiple-choice answer matching \|
	\| MMLU-Pro \| 12,032 \| Multiple-choice answer matching \|
	\| IFEval \| 541 \| Strict and loose instruction following \|
	\| AA-LCR \| 100 \| External judge; data and judge must be supplied separately \|

	Generation artifacts are written to `generations.jsonl`.
	`generation_summary.json` records all resolved settings, aggregate output TPS,
	and TPF as accepted tokens divided by decoder sequence-forwards. Accuracy and
	grader details are written to `grades.jsonl` and `scores.json` when the selected
	benchmark has a local grader.

	```python
	from huggingface_hub import snapshot_download

	snapshot = snapshot_download(
	"s-sahoo/uno-qwen3-8B",
	revision="<pinned-commit-sha>",
	)
	print("Base:", snapshot)
	print("Adapter:", f"{snapshot}/adapter")
	```

	With the [Uno repository](https://github.com/ifm-ai/uno/tree/main/scripts/qwen):

	```bash
	UNO_BUNDLE_REPO=s-sahoo/uno-qwen3-8B \
	UNO_BUNDLE_REVISION=<pinned-commit-sha> \
	bash scripts/qwen/run_gsm8k_eval.sh
	```

	The repository contains custom model code. Conventional Transformers loading
	of the base checkpoint requires `trust_remote_code=True`. Official Uno
	generation additionally requires conditional adapter routing and lossless
	verification implemented by the Uno runtime.