Instructions to use jasonlingg/rlm-explorer-qwen3-8b-qasper-sft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries
PEFT
How to use jasonlingg/rlm-explorer-qwen3-8b-qasper-sft with PEFT:
```
Task type is invalid.
```

How to use jasonlingg/rlm-explorer-qwen3-8b-qasper-sft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jasonlingg/rlm-explorer-qwen3-8b-qasper-sft")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("jasonlingg/rlm-explorer-qwen3-8b-qasper-sft", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jasonlingg/rlm-explorer-qwen3-8b-qasper-sft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jasonlingg/rlm-explorer-qwen3-8b-qasper-sft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jasonlingg/rlm-explorer-qwen3-8b-qasper-sft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/jasonlingg/rlm-explorer-qwen3-8b-qasper-sft

SGLang

How to use jasonlingg/rlm-explorer-qwen3-8b-qasper-sft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jasonlingg/rlm-explorer-qwen3-8b-qasper-sft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jasonlingg/rlm-explorer-qwen3-8b-qasper-sft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jasonlingg/rlm-explorer-qwen3-8b-qasper-sft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jasonlingg/rlm-explorer-qwen3-8b-qasper-sft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use jasonlingg/rlm-explorer-qwen3-8b-qasper-sft with Docker Model Runner:
```
docker model run hf.co/jasonlingg/rlm-explorer-qwen3-8b-qasper-sft
```

Qwen3-8B QASPER code-agent SFT checkpoints

This private repository preserves two experimental QLoRA checkpoints from the RLM Explorer scientific-paper research-agent project. The model was trained to write Python actions that call search(), read(), and extract() before submitting an evidence-grounded answer.

Research status: these checkpoints are artifacts for evaluation and debugging. They do not currently demonstrate an improvement over base Qwen3-8B. Epoch 2 improved held-out next-token metrics but collapsed to immediate answer submission in behavioral evaluation.

Repository contents

Path	Training point	Status
`epoch-1/`	step 48, epoch 1	Behavioral smoke evaluation pending
`epoch-2/`	step 96, epoch 2	Failed the agent-loop behavioral evaluation
`evaluation/`	base, epoch-2, and aligned smoke outputs	Full trajectories, logs, and manifests
`experiment_summary.json`	experiment metadata	Machine-readable summary

Each checkpoint contains the LoRA adapter, tokenizer, trainer state, optimizer, scheduler, RNG state, and training arguments. The adapter in the original final/ directory was byte-identical to checkpoint-96, so only one epoch-2 copy is stored here.

Training

Base model: Qwen/Qwen3-8B
Data: 189 training and 47 validation multi-turn QASPER trajectories
Objective: supervised fine-tuning with loss on assistant tokens
Quantization: 4-bit NF4 with double quantization and bf16 computation
LoRA: rank 4, alpha 8, dropout 0.05
Targets: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj
Learning rate: 2e-4
Effective batch: batch size 1 with 4 gradient-accumulation steps
Maximum sequence length: 8192 tokens
Hardware: one NVIDIA L4

Token-level validation

Checkpoint	Eval loss	Eval mean-token accuracy
Epoch 1	0.5243	0.8528
Epoch 2	0.4916	0.8617

These metrics show that the model learned to imitate held-out trajectories. They do not establish that it learned to operate the research-agent loop.

Behavioral evaluation

The frozen evaluation contained 40 QASPER test questions: 20 answerable from the paper corpus and 20 requiring abstention. There was no training/test question or paper overlap. Both policies used deterministic decoding, seed 42, a 10-step limit, and the same 416-paper corpus.

Metric	Base Qwen3-8B	Epoch 2
Average reward	0.216	0.170
Answer score	0.091	0.044
Citation precision	0.475	0.425
Citation recall	0.950	0.925
Average steps	8.5	1.0
Correct abstentions	1/20	2/20
False abstentions	1/20	1/20

The preregistered abstention threshold required at least 7/20 correct abstentions and no more than 4/20 false abstentions. Epoch 2 did not pass (p = 1.0, two-sided Fisher exact test).

The first full comparison contained a train/serve mismatch: the initial observation repeated the tool preamble. A corrected five-question diagnostic removed that duplication. Epoch 2 still submitted immediately on 5/5 questions and used no tools, although it no longer copied placeholder text. This isolates the central failure: the adapter learned a direct-submit shortcut.

Loading an adapter

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen3-8B"
repo_id = "jasonlingg/rlm-explorer-qwen3-8b-qasper-sft"

tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="epoch-1")
base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base, repo_id, subfolder="epoch-1")

Use epoch-1 or epoch-2 explicitly. Do not treat epoch-2 as a recommended production model.

Reproducibility

Training/evaluation project: rlm-explorer
Full base/epoch-2 evaluation code commit: d7e6efa2b9f90af32584f6973eff364389cbf8ae
Aligned diagnostic code commit: af762ba0ab63e179f4c0ba9714e0c4a7d9985793
Base-model revision: b968826d9c46dd6066d109eabc6255188de91218
Evaluation question SHA-256: 5720a5220ff9e47eb2465028d9592551d4575830ec476dfafc31a4470a11d4d0
Evaluation corpus SHA-256: ee9e3b2986845194cf9a25dbd1ea1ab2cb8e44cd5e385e5cefcb8f7665335a9f

The files under evaluation/ contain the exact question IDs, policy settings, hardware, software versions, run IDs, and full trajectories.

Downloads last month: -

Model tree for jasonlingg/rlm-explorer-qwen3-8b-qasper-sft

Base model

Qwen/Qwen3-8B-Base

Finetuned

Qwen/Qwen3-8B

Adapter

(2275)

this model