ahmed-3m/qwen25-1.5b-gsm8k-reinforce-step300

Best balanced checkpoint: step 300

LoRA adapter for Qwen/Qwen2.5-1.5B-Instruct trained in a self-training math reasoning experiment on GSM8K using a LaSeR-lite REINFORCE setup.

Results

True reproducible exact-match evaluation with deterministic greedy decoding and required #### <integer> answer extraction:

• 17% GSM8K / 16% SVAMP
• Evaluation subsets: GSM8K test first 100 examples, SVAMP first 50 examples
• Decoding: greedy, max_new_tokens=256

Training-time metrics from the original run used non-reseeded random subsets and should not be used as final accuracy claims.

Intended Use

Use when balanced in-distribution and OOD performance matters. This is a research checkpoint, not a production math solver.

Loading

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_id = "Qwen/Qwen2.5-1.5B-Instruct"
adapter_id = "ahmed-3m/qwen25-1.5b-gsm8k-reinforce-step300"

tok = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype=torch.float32,
    attn_implementation="sdpa",
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

Training Context

• Base model: Qwen2.5-1.5B-Instruct
• Adapter: LoRA r=16 over projection modules
• Task: GSM8K math reasoning
• Hardware used: Tesla P40; fp16 + SDPA constraints