README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-4B-Thinking-2507
tags:
  - multi-turn-agent
  - search-r1
  - cross-domain-transfer
  - sft
pipeline_tag: text-generation
---

# wenyii/dt-4bT-r2e-ep1

**Multi-turn agent SFT on Qwen3-4B-Thinking-2507** — `r2e` arm,
checkpoint at training step `ep1`.

Part of an ICLR-target analysis study on cross-domain transfer of multi-turn
agent SFT (Qwen3-4B-Thinking, dual-terminator recipe). Sister checkpoints in
the same family:

| Arm | Steps available |
|---|---|
| `r2e` (code) | `s50`, `s100`, `ep1` (=s218) |
| `search` | `s50`, `s100` |
| `mixed` (50/50) | `s50`, `s100` |

Full step curves (s50/s100/s150/s200/ep1/ep2) available on request.

## Training recipe ("dt" = dual-terminator)

- **Base**: `Qwen/Qwen3-4B-Thinking-2507`
- **Data**: 3500 multi-turn trajectories per arm
  - `r2e`: code-agent trajectories (R2E / SWE-Gym style, ending in
    `<tool_call>{finish}`)
  - `search`: search-agent trajectories (OpenSeeker + NQ-style, ending
    in `Final answer: X`)
  - `mixed`: 50/50 shuffle of r2e + search
- **Recipe diff vs. v5 (one-line edit)**: for the r2e arm, append
  `Final answer: <pr_description-headline, ≤200 chars>` epilogue AFTER
  the standard `<tool_call>{finish}`. This dual-terminates the trajectory
  so a downstream literal-EM evaluator (Search-R1) can still extract the
  final answer.
- **Optimizer**: LR 1e-5, cosine schedule, warmup 0.03, weight decay 0,
  grad clip 1.0, batch 16, FSDP1 + Liger + flash-attn (bf16).
- **Saves**: steps 50, 100, 150, 200, 218 (=ep1), 436 (=ep2 for
  search/mixed; r2e capped at ep1).

## How to use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("wenyii/dt-4bT-r2e-ep1")
mdl = AutoModelForCausalLM.from_pretrained("wenyii/dt-4bT-r2e-ep1", torch_dtype="bfloat16")
```

Or with vLLM:

```bash
python -m vllm.entrypoints.openai.api_server \
    --model wenyii/dt-4bT-r2e-ep1 --port 8000 --tensor-parallel-size 1
```

For full multi-turn agent inference (retriever + tool-call parsing + answer
cascade extractor), see the reference SR1 7-bench eval pipeline
distributed alongside the paper.

## Headline numbers (Search-R1 7-bench, N=500 per bench, mml=16384)

dt-r2e-s100 reaches **EM 0.344 / F1 0.430** on SR1 7-bench, vs Qwen3-4B-Thinking-2507
base + `Final answer:` prompt nudge anchor at EM 0.302 / F1 0.389. See the
companion paper for the full 7-bench / per-arm / multi-seed analysis.

## Citation

```bibtex
@inproceedings{wenyimo2027dt4b,
  title={...},
  author={Mo, Wenyi and ...},
  booktitle={ICLR 2027},
  year={2027}
}
```