Spectral Surgery — Commonsense Reasoning
Collection
Qwen3-8B adapters trained on Commonsense170K, with post-hoc Spectral Surgery evaluation across eight commonsense reasoning benchmarks. • 9 items • Updated
Llama-3.1-8B-Instruct + Commonsense170K — LoRA (Epoch 2)
LoRA adapter for Llama-3.1-8B-Instruct fine-tuned on Commonsense170K.
Adapter
- Dataset: Commonsense170K
- Training epochs: 2
- LoRA rank: 16
- LoRA alpha: 32
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Evaluation
| Task | LoRA | + Spectral Surgery (o_proj + down_proj, 8+2) |
|---|---|---|
| BoolQ | 88.0122% | 88.1346% |
| PIQA | 89.6083% | 89.4450% |
| SocialIQA | 82.0880% | 81.4739% |
| HellaSwag | 93.6566% | 93.2484% |
| WinoGrande | 88.7924% | 88.3189% |
| ARC-Easy | 93.8552% | 93.8973% |
| ARC-Challenge | 85.3242% | 85.5802% |
| OpenBookQA | 90.4000% | 90.6000% |
| Macro | 88.9671% | 88.8373% |
| Micro | 90.7311% | 90.4947% |
| Correct | 20,341 / 22,419 | 20,288 / 22,419 |
Evaluation uses the Llama-3.1-Instruct tokenizer chat template, greedy decoding,
max_new_tokens=8, the vLLM backend, max model length 2048, and seed 42.
Files
adapter_model.safetensors: PEFT LoRA weightsadapter_config.json: PEFT configurationeval-commonsense8/summary.json: eight-task aggregate metricseval-commonsense8/summary.csv: compact task metrics
- Downloads last month
- 28
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for tianzl66/Llama-3.1-8B-Instruct-CommonSense170K-LoRA-Epoch2
Base model
meta-llama/Llama-3.1-8B Finetuned
meta-llama/Llama-3.1-8B-Instruct