README.md

---
license: apache-2.0
base_model: ibm-granite/granite-4.0-micro
library_name: peft
pipeline_tag: text-generation
tags:
  - lora
  - granite
  - reasoning
  - divergent-thinking
  - decision-support
  - negative-results
language:
  - en
---

# refract-granite-3.4b-v5 — divergent refraction proof-of-concept (3.4B dense)

LoRA adapters that teach **ibm-granite/granite-4.0-micro** (3.4B dense) to **refract** a hard
decision into four distinct, viable strategic threads. This is the **proof-of-concept model**
from an 8-round research program — it proved the skill is teachable to a small open model at
all. The shipped successor (32B/9B-active hybrid MoE) is
**[Nikhil0097/refract-hsmall-blend2](https://huggingface.co/Nikhil0097/refract-hsmall-blend2)**;
the full record is at
**[github.com/Nikhiljangra07/divergence-formula](https://github.com/Nikhiljangra07/divergence-formula)**.

## What's in this repo

| Path | Role |
|---|---|
| `decomposer_v5/` | Decomposer adapter — problem → 4 distinct strategic angles (SFT) |
| `worker_v5/` | Worker adapter — one angle → concrete thread (SFT, **the v5 ship**) |
| `worker_v5_dpo/` | **Documented negative result** — DPO-tuned worker, kept as evidence (see below) |

All adapters: LoRA r=64 / α=64, all-linear, trained on 2,285 judge-gated synthetic rows.

## Results (same harness and judge as the successor model)

Same-session blind judging (Gemini 2.5 Pro, temp 0), 48 out-of-distribution decision problems:

| model | overall | viability | distinctness | foresight |
|---|---|---|---|---|
| **this model (v5 SFT)** | 4.24 | 2.81 | ~4.8 | 3.81 |
| BLEND2 (32B/9B-active successor) | 4.55 | 3.56 | 4.90 | 4.21 |
| Claude Haiku 4.5 | 4.72 | 4.25 | 4.75 | 4.52 |

## Why the DPO adapter is included

`worker_v5_dpo/` is a **refuted experiment shipped on purpose**. On this 3.4B model,
preference optimization with clean hard negatives bought viability **only by selling
distinctness** — the two dimensions are capacity-entangled at this scale. That entanglement
**breaks at ~9B active parameters** (identical data, both dimensions rise together — finding #4
of the program). The adapter is kept as reproducible evidence of the negative result, not as a
recommended model.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "ibm-granite/granite-4.0-micro"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")
dec = PeftModel.from_pretrained(model, "Nikhil0097/refract-granite-3.4b-v5", subfolder="decomposer_v5")
# two-stage: decompose into 4 angles, then run worker_v5 per angle
```

Exact prompts and the eval harness: `round2_kit/` in the GitHub repo.

## Honest scope

- Proof-of-concept: distinctness plateaus ~2.75–4.8 depending on harness era (triangulated in
  the working paper); viability is weak (2.81) at this scale — that limitation is the finding.
- Fully synthetic training data (DeepSeek generator + LLM judge gate); no human/user data.
- Program concluded; this model is superseded by BLEND2 but kept public as the capacity
  baseline that makes finding #4 (entanglement breaks with scale) measurable.