tianzl66/Qwen3-8B-InstructionFollowing-LoRA

Qwen3-8B + Instruction Following — LoRA

This repository contains the all-linear instruction-following LoRA checkpoint of Qwen3-8B.

Base Model

Qwen/Qwen3-8B

Source LoRA

• Dataset: Tulu-3 SFT instruction-following split
• Samples: 29,980
• Epochs: 2
• Sequence length: 4096
• Global batch size: 128
• LoRA rank: 16
• LoRA alpha: 32
• LoRA dropout: 0.05
• Learning rate: 4e-4
• LR schedule: cosine, warmup ratio 0.03
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Chat template: non-thinking (enable_thinking=False)
• Seed: 42

Evaluation

Evaluation on IFEval (541 prompts, 834 instructions).

Model	Prompt Strict	Prompt Loose	Instruction Strict	Instruction Loose
Base	75.23%	81.70%	82.97%	87.53%
LoRA SFT	74.31%	78.37%	81.41%	85.13%
Spectral Surgery HNS 4+1	75.79%	80.78%	83.21%	86.69%
Spectral Surgery HNS 8+2	75.60%	80.78%	83.09%	86.57%

Settings: Qwen3 non-thinking chat template (enable_thinking=False), greedy decoding, max_new_tokens=2048, vLLM backend, FLASH_ATTENTION, max model length 4096, seed 42.

Files

• adapter_model.safetensors: PEFT LoRA weights
• adapter_config.json: PEFT configuration
• eval-ifeval/metrics.json: aggregate IFEval metrics
• eval-ifeval/outputs.jsonl: per-prompt generations and instruction results
• run_config.json / run_args.json: training configuration