---
license: apache-2.0
base_model: ibm-granite/granite-4.0-h-small
library_name: peft
pipeline_tag: text-generation
tags:
  - lora
  - granite
  - reasoning
  - divergent-thinking
  - decision-support
language:
  - en
---

# refract-hsmall-blend2 — divergent decision refraction (shipped model)

LoRA adapters that teach **ibm-granite/granite-4.0-h-small** (32B total / ~9B active, hybrid
Mamba+MoE) to **refract** a hard decision into **four genuinely distinct, viable strategic
threads** — each a different *kind* of move (confront, evade, co-opt, transform, delegate,
endure) — and refuse to blend them into one hedged answer.

This is the **shipped result of a concluded 8-round research program**. The full record —
every round, every number, every mistake, and the negative results — is in the project's
working paper: **[github.com/Nikhiljangra07/divergence-formula](https://github.com/Nikhiljangra07/divergence-formula)**.

## What's in this repo

| Path | Role |
|---|---|
| `dec_blend2/` | **Decomposer** adapter — problem → 4 distinct strategic angles |
| `wrk_blend2/` | **Worker** adapter — one angle → a concrete, viable thread with projected consequences |
| `eval/` | Raw eval outputs (DAV harness JSON + per-thread JSONL, held-out + benchmark sets) |

Both adapters: LoRA r=64 / α=64, all-linear targets, SFT on 6,745 judge-gated synthetic rows
(1,349 decomposer / 5,396 worker), including a targeted "viable-cunning" influence round.

## Headline result

Single-session blind judging (Gemini 2.5 Pro, temp 0), 48 out-of-distribution modern decision
problems, Claude Haiku 4.5 generated fresh in the same session as the frontier competitor:

| model | overall | viability | **distinctness** | decisiveness | foresight |
|---|---|---|---|---|---|
| **this model (BLEND2)** | **4.55** | 3.56 | **4.90** | **4.83** | 4.21 |
| Claude Haiku 4.5 | 4.72 | 4.25 | 4.75 | 4.83 | 4.52 |

**It beats Haiku 4.5 on the objective it was built for — distinctness (4.90 vs 4.75) — and ties
it on decisiveness**, with ~9B active parameters. It loses overall (−0.17); the residue is
viability and foresight, and the working paper says so plainly.

> Note on the two eval harnesses: the table above is the same-session comparative re-judge
> (working paper §14–15). The JSON files in `eval/` are from the project's internal DAV harness
> (a stricter per-dimension rubric on the same problem sets) — different scale, same model.
> Both are documented in the working paper; nothing here is cherry-picked across harnesses.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "ibm-granite/granite-4.0-h-small"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")

# Stage 1: decompose the problem into 4 distinct angles
dec = PeftModel.from_pretrained(model, "Nikhil0097/refract-hsmall-blend2", subfolder="dec_blend2")
# ... generate angles, then per-angle:
# Stage 2: swap to the worker adapter and expand each angle into a full thread
```

The exact two-stage prompts and generation settings are in the GitHub repo
(`round2_kit/dav_eval_v5.py`). The base model needs the Mamba kernels; the training/inference
engineering traps are documented in the working paper §13.

## Honest scope

- Specialist scoreboard, not a general benchmark win: the rubric measures the one operation
  the model was trained for.
- Known weaknesses: viability (3.56 vs Haiku's 4.25) and foresight (4.21 vs 4.52).
- Levers identified but **not run** (program concluded): hard-negative DPO at 9B-active scale,
  inference-time viability checking, a foresight-targeted corpus.
- Training data is fully synthetic (DeepSeek V4 generator + LLM judge gate); no human-authored
  or user data.

## Provenance & findings

The program's five load-bearing findings (teachability, quality-beats-size, 3.4B capacity
entanglement, entanglement breaking at ~9B active, cheap-generator saturation) are in the
[working paper](https://github.com/Nikhiljangra07/divergence-formula/blob/main/WORKING_PAPER.md).
Total program cost: ~$175. Built solo by Nikhil Jangra.