OCF typed-decisions CE smoke

README.md

@@ -0,0 +1,103 @@
+---
+license: apache-2.0
+library_name: transformers
+tags:
+  - text-classification
+  - system-1
+  - typed-decisions
+  - modernbert
+  - ocf
+datasets:
+  - LocalLLaMA/typed-decisions
+base_model: answerdotai/ModernBERT-base
+pipeline_tag: text-classification
+---
+
+# OCF typed-decisions CE smoke (ModernBERT-base)
+
+**Not a foundation classifier. Not RLCD. No Jev comparison.**
+
+Open Classification Foundation (OCF) **specialist smoke**: non-autoregressive System-1
+decision head on [`answerdotai/ModernBERT-base`](https://huggingface.co/answerdotai/ModernBERT-base),
+fine-tuned with **cross-entropy** on `LocalLLaMA/typed-decisions` train, temperature-calibrated
+on a held-out **train** slice (`cal_frac=0.1`).
+
+## Architecture
+
+```text
+ModernBERT-base (bidirectional)
+  + 2-layer decision Transformer head
+  + OptionScorer at each option [MASK]
+  → softmax(/T) over options
+```
+
+| Knob | Value |
+| --- | --- |
+| Params | ~164M |
+| `max_len` | 640 |
+| Option token budget | 48 / option |
+| Primitives | `choice` / `score` / `noul` |
+
+## Training
+
+| Field | Value |
+| --- | --- |
+| Objective | CE on teacher-argmax labels (soft teachers in dataset) |
+| Epochs / eff. batch | 4 / 8 |
+| LR encoder / head | 2.5e-5 / 1e-4 |
+| Seed (this upload) | **1** (best of 0/1/2; mean acc 0.731, SD 0.028) |
+| Temperatures (shipped) | choice 1.5 · score 1.25 · noul 1.5 |
+
+## Evaluation (`LocalLLaMA/typed-decisions` test, 2,000 decisions)
+
+| Arm | acc | NLL | ECE as-run | ECE after equal crossfit |
+| --- | ---: | ---: | ---: | ---: |
+| Laya typed-decisions (reference) | 0.767 | 0.707 | 0.214 | 0.021 |
+| **This checkpoint (s1)** | **0.751** | **0.629** | 0.044 | 0.030 |
+| OCF stub (prior, ~8M) | ≈0.61 | ≈0.90 | ≈0.04 | — |
+
+Latency (warm, RTX 3050 Laptop): **p50 59.6 ms / p95 87.8 ms** per question (n=200).
+Not a cross-vendor claim.
+
+Full board: see source-repo snapshot `results-snapshot-2026-09-28-ocf-typed-decisions-mbert.md`.
+
+## Files
+
+- `config.json` — `ClassifierConfig` (includes temperatures)
+- `temperatures.json` — fitted per-type T
+- `model.pt` — full `ClassifierBundle` state_dict
+- `recipe.json` — train hyperparams + acc
+
+## How to run
+
+Requires the OCF stack from the source repository (`packages.classify` + `models_ai.classify`):
+
+```python
+from classify import load
+
+agent = load("path/to/this/repo")  # config.json + model.pt + temperatures.json
+out = agent.predict(
+    {"subject": "Refund", "body": "Billed twice"},
+    {
+        "department": {
+            "type": "choice",
+            "instructions": "Which department?",
+            "criteria": {"billing": "invoices", "tech": "bugs", "other": "else"},
+        }
+    },
+)
+```
+
+Not a drop-in for the external `laya` package.
+
+## Concurrent baseline
+
+[Laya](https://huggingface.co/convaiinnovations/laya) — open System-1 concurrent (ModernBERT-large).
+Matched estimand only; not a dependency.
+
+## Non-claims
+
+- Not a foundation / zero-shot System-1 model
+- Not RLCD-trained
+- Does not claim to beat TypeSafe Jev or Laya
+- Soft teacher labels; teachers disagree on ~41% of test decisions

config.json

@@ -0,0 +1,17 @@
+{
+  "vocab_size": 4096,
+  "d_model": 128,
+  "n_heads": 4,
+  "n_layers": 2,
+  "head_layers": 2,
+  "max_len": 640,
+  "dropout": 0.1,
+  "temperatures": {
+    "choice": 1.5,
+    "score": 1.25,
+    "noul": 1.5
+  },
+  "backbone": "answerdotai/ModernBERT-base",
+  "head_heads": 0,
+  "gradient_checkpointing": true
+}

model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8455d5f88af89d8b4ff136fd52fff28cce6eeabb859eab0491a1c07637947dbb
+size 655191779

recipe.json

@@ -0,0 +1,27 @@
+{
+  "seed": 1,
+  "preset": "modernbert-base-4gb",
+  "hp": {
+    "backbone": "answerdotai/ModernBERT-base",
+    "tag": "mbert-base",
+    "steps": 0,
+    "epochs": 4,
+    "micro_batch": 4,
+    "grad_accum": 2,
+    "lr_head": 0.0001,
+    "lr_encoder": 2.5e-05,
+    "schedule": "linear",
+    "max_len": 640,
+    "head_layers": 2,
+    "bf16": true,
+    "grad_ckpt": true,
+    "adam8": true
+  },
+  "opt_steps": 2700,
+  "cal_frac": 0.1,
+  "n_params": 163782913,
+  "train_seconds": 2339,
+  "peak_gib": 2.13,
+  "device": "cuda",
+  "acc": 0.7515
+}

temperatures.json

@@ -0,0 +1,5 @@
+{
+  "choice": 1.5,
+  "score": 1.25,
+  "noul": 1.5
+}