Instructions to use yuhan-nlp/verl-grpo-medium-qwen3-4b-step100 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use yuhan-nlp/verl-grpo-medium-qwen3-4b-step100 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="yuhan-nlp/verl-grpo-medium-qwen3-4b-step100")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("yuhan-nlp/verl-grpo-medium-qwen3-4b-step100")
model = AutoModelForCausalLM.from_pretrained("yuhan-nlp/verl-grpo-medium-qwen3-4b-step100", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use yuhan-nlp/verl-grpo-medium-qwen3-4b-step100 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "yuhan-nlp/verl-grpo-medium-qwen3-4b-step100"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yuhan-nlp/verl-grpo-medium-qwen3-4b-step100",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/yuhan-nlp/verl-grpo-medium-qwen3-4b-step100

SGLang

How to use yuhan-nlp/verl-grpo-medium-qwen3-4b-step100 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "yuhan-nlp/verl-grpo-medium-qwen3-4b-step100" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yuhan-nlp/verl-grpo-medium-qwen3-4b-step100",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "yuhan-nlp/verl-grpo-medium-qwen3-4b-step100" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yuhan-nlp/verl-grpo-medium-qwen3-4b-step100",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use yuhan-nlp/verl-grpo-medium-qwen3-4b-step100 with Docker Model Runner:
```
docker model run hf.co/yuhan-nlp/verl-grpo-medium-qwen3-4b-step100
```

verl-GRPO CollabLLM medium — Qwen3-4B, step 100

Qwen/Qwen3-4B trained with GRPO on the CollabLLM medium document-writing task using the verl CollabLLM recipe (no SFT warm start). This is a merged HF checkpoint — load it directly, no adapter step.

This checkpoint: the checkpoint to report. Best BLEU of any arm, and the only one significantly above base (t=+2.31).

Benchmark (medium, n=100)

Three roles, thinking off everywhere: Qwen3-4B assistant (this model), Qwen/Qwen3.5-9B user simulator, Qwen/Qwen3.5-27B judge. Flags: --user_sim_prompt sim_gap_paper_lazy_full --assistant_max_tokens 4096 --max_new_turns 14.

condition	BLEU	interactivity	tokens (k)	MR
base `Qwen3-4B` (no proact)	0.4491	0.831	2.935	0.9870
GRPO step 50	0.4524	0.819	2.281	1.0433
GRPO step 100	0.4827	0.801	2.077	1.0761
GRPO step 129	0.4825	0.744	1.944	1.0316
proact prompting (no training)	0.4669	0.825	1.678	1.1236
← this model (step 100)	0.4827	0.801	2.077	1.0761

⚠️ Read before quoting these numbers

Training peaked at step 100 and then regressed. Step 129 bought no BLEU and cost interactivity. Never select a checkpoint on verl's critic/rewards/mean — it rises through step 129, because the reward credits brevity directly while the benchmark judge penalises the interactivity that brevity costs.
Prompting alone still wins on MR (1.1236 for proact vs 1.0761 for step 100) and needs no training. Step 100 wins on BLEU. Say which metric you mean.
Every arm's MR gain is dominated by the token term, not document quality — so MR overstates quality improvements for all of them.
⚠️ 35/100 train/eval prompt overlap with byte-identical reward targets. Because the BLEU reward target is the test reference on those rows, this is label leakage through the reward. See the dataset card.

Training setup

GRPO, train_batch_size=16, rollout.n=8, lr=1e-6, kl_loss_coef=0.001 (low-var KL), 8 GPUs (FSDP, world_size_8), reward = bleu_score (+1) + interactivity (+1) + token_amount (−0.1). Reference run: wandb photon/verlxcollabllm/e1adivlo, val@0 = 1.0618.

Full setup, the self-contained docker image, and the two acceptance tests are documented in studynotes/new_cluster_onboarding.md of the code repo.

Downloads last month: 17

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for yuhan-nlp/verl-grpo-medium-qwen3-4b-step100

Base model

Qwen/Qwen3-4B-Base

Finetuned

Qwen/Qwen3-4B

Finetuned

(1155)

this model

yuhan-nlp
/

verl-grpo-medium-qwen3-4b-step100

verl-GRPO CollabLLM medium — Qwen3-4B, step 100

Benchmark (medium, n=100)

⚠️ Read before quoting these numbers

Training setup

Model tree for yuhan-nlp/verl-grpo-medium-qwen3-4b-step100

Dataset used to train yuhan-nlp/verl-grpo-medium-qwen3-4b-step100