---
base_model: Qwen/Qwen3-8B
library_name: peft
tags:
- qwen
- qwen3
- lora
- code
- magicoder
- spectral-surgery
- hns
---

# Qwen3-8B + Magicoder-50K + Spectral Surgery

This repository contains the Spectral Surgery adapter obtained by
applying HNS 4+1 post hoc to the epoch-1 Magicoder-50K LoRA checkpoint
of Qwen3-8B.

## Base Model

`Qwen/Qwen3-8B`

## Source LoRA

- Dataset: Magicoder
- Samples: 50K
- Source checkpoint: Epoch 1
- Sequence length: 4096
- Global batch size: 32
- Learning rate: 2e-5
- LoRA rank: 16
- Seed: 42

## Spectral Surgery

- Target: all LoRA modules
- Output rank: 16
- Fast HNS steps: 4
- Stable HNS steps: 1
- Additional training: none

## Evaluation

Greedy decoding.

HumanEval uses the chat prompt format. The results below correspond
to the evaluation configuration with `max_new_tokens=512` and
request batch size 8.

| Method | HumanEval-chat Pass@1 | MBPP-sanitized Pass@1 |
|---|---:|---:|
| Qwen3-8B Base | 64.63% (106/164) | 72.76% (187/257) |
| LoRA, Epoch 1 | 67.07% (110/164) | 72.76% (187/257) |
| **LoRA + HNS 4+1, all modules** | **74.39% (122/164)** | **75.10% (193/257)** |

Applied post hoc to the fixed epoch-1 LoRA checkpoint, all-module
HNS 4+1 improves Pass@1 by 7.32 percentage points on HumanEval and
2.33 percentage points on MBPP without additional training.

Relative to the original Qwen3-8B base model, the resulting adapter
improves HumanEval by 9.76 percentage points and MBPP by 2.33
percentage points.