File size: 2,609 Bytes
e76b68b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 | ---
license: apache-2.0
base_model: Qwen/Qwen3-4B-Thinking-2507
tags:
- multi-turn-agent
- search-r1
- cross-domain-transfer
- sft
pipeline_tag: text-generation
---
# wenyii/dt-4bT-r2e-ep1
**Multi-turn agent SFT on Qwen3-4B-Thinking-2507** — `r2e` arm,
checkpoint at training step `ep1`.
Part of an ICLR-target analysis study on cross-domain transfer of multi-turn
agent SFT (Qwen3-4B-Thinking, dual-terminator recipe). Sister checkpoints in
the same family:
| Arm | Steps available |
|---|---|
| `r2e` (code) | `s50`, `s100`, `ep1` (=s218) |
| `search` | `s50`, `s100` |
| `mixed` (50/50) | `s50`, `s100` |
Full step curves (s50/s100/s150/s200/ep1/ep2) available on request.
## Training recipe ("dt" = dual-terminator)
- **Base**: `Qwen/Qwen3-4B-Thinking-2507`
- **Data**: 3500 multi-turn trajectories per arm
- `r2e`: code-agent trajectories (R2E / SWE-Gym style, ending in
`<tool_call>{finish}`)
- `search`: search-agent trajectories (OpenSeeker + NQ-style, ending
in `Final answer: X`)
- `mixed`: 50/50 shuffle of r2e + search
- **Recipe diff vs. v5 (one-line edit)**: for the r2e arm, append
`Final answer: <pr_description-headline, ≤200 chars>` epilogue AFTER
the standard `<tool_call>{finish}`. This dual-terminates the trajectory
so a downstream literal-EM evaluator (Search-R1) can still extract the
final answer.
- **Optimizer**: LR 1e-5, cosine schedule, warmup 0.03, weight decay 0,
grad clip 1.0, batch 16, FSDP1 + Liger + flash-attn (bf16).
- **Saves**: steps 50, 100, 150, 200, 218 (=ep1), 436 (=ep2 for
search/mixed; r2e capped at ep1).
## How to use
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("wenyii/dt-4bT-r2e-ep1")
mdl = AutoModelForCausalLM.from_pretrained("wenyii/dt-4bT-r2e-ep1", torch_dtype="bfloat16")
```
Or with vLLM:
```bash
python -m vllm.entrypoints.openai.api_server \
--model wenyii/dt-4bT-r2e-ep1 --port 8000 --tensor-parallel-size 1
```
For full multi-turn agent inference (retriever + tool-call parsing + answer
cascade extractor), see the reference SR1 7-bench eval pipeline
distributed alongside the paper.
## Headline numbers (Search-R1 7-bench, N=500 per bench, mml=16384)
dt-r2e-s100 reaches **EM 0.344 / F1 0.430** on SR1 7-bench, vs Qwen3-4B-Thinking-2507
base + `Final answer:` prompt nudge anchor at EM 0.302 / F1 0.389. See the
companion paper for the full 7-bench / per-arm / multi-seed analysis.
## Citation
```bibtex
@inproceedings{wenyimo2027dt4b,
title={...},
author={Mo, Wenyi and ...},
booktitle={ICLR 2027},
year={2027}
}
```
|