File size: 3,697 Bytes
478a2c5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 | ---
library_name: peft
pipeline_tag: text-generation
language:
- zh
- en
tags:
- recommendation
- generative-recommendation
- one-reason
- peft
- lora
- r3
- checkpoint-interpolation
---
# OneReason-8B LoRA: R3 Replay Step20/25 Interp075 KVO
This is an experimental rank-32/alpha-32 LoRA adapter prepared for formal
OneReason platform evaluation. It must be loaded on the exact local epoch-2
full-SFT base represented by `/data/sft_yaml/onereason_sft_epoch2_bf16`; it is
not an adapter for the untouched public pretraining checkpoint.
The project ledger associates that epoch-2 base with the platform result
`1.2501`, but the historical upload hash is unavailable. That score-to-artifact
mapping is therefore provenance information, not a cryptographic identity
claim. This candidate itself has not yet received an official platform score.
## Construction
The adapter was built offline from two checkpoints on one continuous training
trajectory:
- step 20: the R0 raw-SID route-distillation + balanced R3 replay run;
- step 25: an exact optimizer/scheduler/RNG continuation for five more updates;
- interpolation coefficient: `0.75` from step 20 toward step 25;
- interpolated LoRA factors: every layer's `k_proj`, `v_proj`, and `o_proj`;
- unchanged at step 20: `q_proj`, `gate_proj`, `up_proj`, and `down_proj`.
Both endpoints use LoRA rank/alpha `32/32`. The interpolation acts directly on
the matching LoRA A/B factors. Since effective LoRA weights are products of
those factors, this is not algebraically identical to dense-weight linear
interpolation. No extra training was run to create this candidate.
The source trajectory used 1,192 rows: 952 raw-SID route-distillation examples
and 240 balanced, held-out-safe no-thinking R3 replay examples. Training used
`cutoff_len=1024`, global batch size 8, peak LR `5e-5`, AdamW, cosine scheduling,
and a schedule horizon of 122 updates.
Adapter SHA-256:
`efc2152249a3958a7aa854105743c68c33428f706c54b5ba4b298df262f77e5f`
## Local selection evidence
All values below are paired changes versus step 20. These are deterministic
local proxies, not official platform scores and not LLM-as-Judge results.
| Task / metric | Change | 95% CI |
|---|---:|---:|
| R0 raw SID char-F1, 400 rows | +0.000173 | [-0.003221, +0.003524] |
| R0 generic char-F1, 400 rows | -0.002056 | [-0.005840, +0.001640] |
| R2 overall proxy, 128 rows | -0.000171 | [-0.004844, +0.004240] |
| R3 teacher domain probability, 256 rows | +0.003112 | [+0.001217, +0.005004] |
| R3 free domain accuracy, 256 rows | +0.019531 | [-0.003906, +0.046875] |
| World relaxed parse, 2,000 rows | -0.005000 | [-0.012000, +0.002000] |
| World relaxed correct, 2,000 rows | -0.004500 | [-0.011000, +0.002000] |
R3 teacher `s_b` probability was statistically unchanged; `s_c` changed by
`-0.002627` with CI `[-0.004860, -0.000438]`. The candidate was selected because
it retained most of the R3 domain gain of the all-module interpolation while
substantially reducing its fine-grained hierarchy regressions.
## Loading
Use the exact epoch-2 full-SFT base, then attach this adapter with PEFT:
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_path = "/data/sft_yaml/onereason_sft_epoch2_bf16"
adapter_id = "yujackein/onereason-8b-lora-r3replay-step20-step25-interp075-kvo-r32a32"
tokenizer = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(
base_path,
torch_dtype="auto",
device_map="auto",
trust_remote_code=True,
)
model = PeftModel.from_pretrained(base_model, adapter_id)
```
## Evaluation status
Official platform evaluation: pending.
|