HoangTran223/T-SPIN_Qwen3-4B

T-SPIN Qwen3-4B (Ultrachat)

Self-play T-SPIN (Wang et al., NeurIPS 2025) run on Qwen3-4B SFT (HoangTran223/qwen3_4b_sft_ultrachat200k_20260818_170154).

• Prompts: UltraChat 50k subset of HuggingFaceH4/ultrachat_200k
• Loss: T-SPIN (alpha=1.0, beta=0.1), RMSProp, batch 2 × grad accum 2, max_length=1024
• LR: 5e-7 (ite0–1), 1e-7 (ite2+)
• proto/train.jsonl: proto responses y_0 from the initial policy

Load a finished iteration with:

from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "HoangTran223/T-SPIN_Qwen3-4B"
rev = "ite2"  # or ite0 / ite1; ite3/LATEST is an in-progress snapshot
tok = AutoTokenizer.from_pretrained(f"{model_id}/{rev}")
model = AutoModelForCausalLM.from_pretrained(f"{model_id}/{rev}")

Layout

• ite0/, ite1/, ite2/: finished checkpoints + generated train.jsonl
• ite3/: generated train.jsonl + in-progress LATEST/ (includes trainer_state.pt for resume; last snapshot at 94k examples)
• proto/train.jsonl: y_0 proto responses used by T-SPIN
• data/Ultrachat50k/train.jsonl: prompts used for generation
• scripts/T_SPIN_full_qwen3_4b.sh: launch script
• logs/: wandb offline runs, Hydra configs, tmux stdout