Hugging Face's logo

MSGEncrypted
/
ocf-typed-decisions-mbert-base

Text Classification
Transformers
system-1
typed-decisions
modernbert
ocf
Model card Files
xet
 Community
ocf-typed-decisions-mbert-base / README.md
MSGEncrypted's picture
MSGEncrypted
OCF typed-decisions CE smoke
b97d919 verified
preview code
|
Raw History Blame Contribute Delete
3.07 kB
metadata 
license: apache-2.0
library_name: transformers
tags:
  - text-classification
  - system-1
  - typed-decisions
  - modernbert
  - ocf
datasets:
  - LocalLLaMA/typed-decisions
base_model: answerdotai/ModernBERT-base
pipeline_tag: text-classification

OCF typed-decisions CE smoke (ModernBERT-base)

Not a foundation classifier. Not RLCD. No Jev comparison.

Open Classification Foundation (OCF) specialist smoke: non-autoregressive System-1 decision head on answerdotai/ModernBERT-base, fine-tuned with cross-entropy on LocalLLaMA/typed-decisions train, temperature-calibrated on a held-out train slice (cal_frac=0.1).

Architecture 

ModernBERT-base (bidirectional)
  + 2-layer decision Transformer head
  + OptionScorer at each option [MASK]
  → softmax(/T) over options
Knob Value
Params ~164M
max_len 640
Option token budget 48 / option
Primitives choice / score / noul

Training

Field Value
Objective CE on teacher-argmax labels (soft teachers in dataset)
Epochs / eff. batch 4 / 8
LR encoder / head 2.5e-5 / 1e-4
Seed (this upload) 1 (best of 0/1/2; mean acc 0.731, SD 0.028)
Temperatures (shipped) choice 1.5 · score 1.25 · noul 1.5

Evaluation (LocalLLaMA/typed-decisions test, 2,000 decisions)

Arm acc NLL ECE as-run ECE after equal crossfit
Laya typed-decisions (reference) 0.767 0.707 0.214 0.021
This checkpoint (s1) 0.751 0.629 0.044 0.030
OCF stub (prior, ~8M) ≈0.61 ≈0.90 ≈0.04 —

Latency (warm, RTX 3050 Laptop): p50 59.6 ms / p95 87.8 ms per question (n=200). Not a cross-vendor claim.

Full board: see source-repo snapshot results-snapshot-2026-09-28-ocf-typed-decisions-mbert.md.

Files

  • config.json — ClassifierConfig (includes temperatures)
  • temperatures.json — fitted per-type T
  • model.pt — full ClassifierBundle state_dict
  • recipe.json — train hyperparams + acc

How to run

Requires the OCF stack from the source repository (packages.classify + models_ai.classify):

from classify import load

agent = load("path/to/this/repo")  # config.json + model.pt + temperatures.json
out = agent.predict(
    {"subject": "Refund", "body": "Billed twice"},
    {
        "department": {
            "type": "choice",
            "instructions": "Which department?",
            "criteria": {"billing": "invoices", "tech": "bugs", "other": "else"},
        }
    },
)

Not a drop-in for the external laya package.

Concurrent baseline

Laya — open System-1 concurrent (ModernBERT-large). Matched estimand only; not a dependency.

Non-claims

  • Not a foundation / zero-shot System-1 model
  • Not RLCD-trained
  • Does not claim to beat TypeSafe Jev or Laya
  • Soft teacher labels; teachers disagree on ~41% of test decisions