betterwithage's picture
deploy(hf): sync szl-holdings/a11oy@194f59416698777d2eec9562f0065113d7576519 derived COPY set
e9b231f verified
|
Raw
History Blame
11.2 kB

SZL-Forge-1.5B — ReceiptAgent release program

This directory originated as a bounded, honesty-first release program for a proposed model named SZL-Forge-1.5B-ReceiptAgent-v1. A public, weight-bearing candidate now exists at SZLHOLDINGS/SZL-Forge-1.5B-ReceiptAgent; that external artifact does not retroactively satisfy this repository's broader admission or promotion gates. The exact public revision qualified below remains a candidate, not a promoted A11oy runtime dependency.

The purpose is narrow: a small model that produces a strict unsigned draft containing an answer or abstention, evidence IDs, formula references, uncertainty, and a non-executing tool proposal. Deterministic runtime code in receipt_runtime.py resolves authoritative records, enforces policy, binds canonical hashes, and builds the final response envelope only after verifying a replay-protected DSSE receipt over every decision-bearing component. Its HMAC-SHA256 path is experimental; public promotion still requires asymmetric attestation and transparency logging. A useful release would make those controls measurable. It would not make a generic chat model rare, prove the model's identity from behavior, or turn retrieved text into a mathematical proof.

Current measured boundary

Item Current evidence Release implication
Base unsloth/Qwen2.5-1.5B-Instruct-bnb-4bit at d2f2dd02b071701d5100a04a7a49d6fb0bd305b7 Lineage must remain explicit.
Existing SFT adapter SHA-256 682e2f0ea480d47c284b9de12c2e3d2d5170934c065e82fc375e3f069b4730ac Identifies the existing candidate only; it is not a new ReceiptAgent artifact.
Existing SFT data 167 rows License, privacy, and contamination review is incomplete, so reuse is blocked.
Existing evaluation Incomplete A 16-row loss direction exists, but general quality is not established.
ORPO candidate 0/12 qualification checks Quarantined; cannot influence release.
Brain 9,464 raw nodes All 9,464 are training-quarantined; zero raw-node text rows are admitted to train.
Formula admission 146 formula crosswalk records + 2 SZL-Lake evidence records = 148 holdout tranche rows All 148 are holdout; zero are admitted to train. Formula IDs are namespace-scoped.
Public candidate HF revision fa73dc1bd8eeece727d0b5c1db52448ec0703e8b; raw model.safetensors SHA-256 a797592fb35c32cc6ebf5739c0a1d9312123300acaf09987d207d86196bc6e7d MEASURED only by the bounded CPU contract qualification below; not promoted or deployed by this repository.

The artifact reconciliation in reconciliation/ distinguishes signed-receipt validity, raw artifact SHA-256, the original signer's SHA256(UTF8(basename) || raw bytes) domain, and public-head equivalence. At the frozen observed head, raw Git commit objects reproduce both declared revision IDs, serialized root/subtree objects reproduce their commit-bound tree IDs, and the complete tree delta is the one non-inference estate marker. ReceiptAgent is ARTIFACT_BYTES_RECONCILED_MEASURED_NOT_PROMOTED. This state does not expand the bounded CPU evaluation or establish hosted serving, general quality, promotion, deployment, or independent owner identity.

Exact-revision CPU qualification (2026-07-21)

qualification/heldout-contract-v1.json freezes the public artifact, public source revision, output schema, six postdated project-authored prompts, greedy decoding, and pass thresholds before inference. The deterministic runner qualify_public_candidate.py then verifies:

  • the 3,087,467,144-byte model and 147,770,496-byte adapter by raw and training-receipt directory digests;
  • every inference-affecting config, tokenizer, chat template, generation config, and adapter config by exact byte count and SHA-256 at the frozen candidate revision;
  • the byte-bound Apache-2.0 license file (a declared license observation, not a legal opinion);
  • the owner-signed Ed25519 training/evaluation receipt chain against the repository-declared key;
  • the exact szl-holdings/szl-forge source commit, evaluator/schema Git blobs, and five public curriculum hashes; and
  • absence of every held-out prompt as exact text in those public curriculum blobs. Semantic contamination and private-corpus absence are not established.

The one bounded CPU run produced 3/3 raw JSON-schema-valid drafts, 3/3 valid proposal-only boundaries, 3/3 REFUSE-prefixed adversarial responses, and zero detected catastrophic execution/finalization events. Its unsigned local receipt is qualification/fa73dc1-cpu-qualification-receipt.json, with canonical SHA-256 5abc85af639477f32951784c372beef33a608375586c111e517692207074ba7b.

This is a narrow structural-contract result. It does not establish endpoint existence, evidence grounding, citation correctness, general capability, private-set noncontamination, hosted availability, deployment, promotion, or human release approval. In particular, schema-valid evidence labels remain claims to verify externally; this run does not resolve or probe the cited endpoints. The local receipt is deliberately unsigned because no approved independent qualification key was provided.

The current Brain and formula corpus are valuable as a governed retrieval and evaluation substrate. They are not a training corpus until item-level rights, provenance, revisions, freshness, deduplication, and split assignments pass.

Contract

The adapter contract is szl.forge-receipt-draft.v1; the final runtime contract is szl.receipt-agent-output.v1. They are deliberately different. The model proposes. The deterministic runtime validates, resolves, gates, and receipts.

receipt-agent-output.schema.json requires each response to separate:

  1. exact model identity and release state;
  2. ANSWERED, ABSTAINED, or UNAVAILABLE status;
  3. answer text, which must be null when abstaining or unavailable;
  4. evidence IDs, content hashes, support roles, and admission states;
  5. formula ID, namespace, status, and explicit proof-transfer decision;
  6. confidence and calibration state;
  7. a typed abstention reason;
  8. a proposal-only tool request that always requires human approval and cannot contain an execution receipt; and
  9. request, evidence-set, policy, and external receipt bindings.

An ANSWERED response must have at least one ADMITTED_REFERENCE. The example is deliberately UNAVAILABLE; it contains no fabricated answer or receipt.

Admission gates

admission-manifest.json is fail-closed. Promotion requires every gate to pass and permits zero catastrophic errors. Current blockers are:

  • incomplete row-level governance for the existing 167-row dataset;
  • 9,464/9,464 raw Brain nodes quarantined from training;
  • 146 formula records plus 2 SZL-Lake evidence records reserved as 148/148 holdout rows, with F1-F23 namespace collisions;
  • the ORPO candidate failing all 12 qualification checks;
  • no independently pinned release key, third-party provenance review, or local promotion approval for the externally published candidate;
  • no frozen three-way evaluation;
  • no signed DSSE/in-toto provenance or transparency-log record; and
  • unresolved release/data licensing and no human release approval.

Passing base and adapter digest checks proves only which existing artifacts are being discussed. It does not promote them or establish ReceiptAgent behavior.

Evaluation design

evaluation-manifest.json fixes a three-column comparison:

  • exact pinned base;
  • exact existing SFT adapter; and
  • exact future ReceiptAgent artifact.

The same frozen prompts, decoding parameters, tool policy, evidence snapshot, and formula crosswalk must be used for all three. Required suites cover strict schema validity, evidence existence, false support, formula status and namespace preservation, required abstention, stale/quarantined-source refusal, prompt injection, unauthorized tools, malformed receipts, identity honesty, and separately reported p95 latency. Every matrix cell is currently NOT_RUN.

Latency is an operational measurement, not a quality score. Behavioral fingerprints are supplemental regression signals, not proof of weight identity.

Hugging Face family

The proposed family is declared in release-manifest.json. The first public candidate is now externally observable; the remaining family members stay planned until independently verified:

  • SZLHOLDINGS/SZL-Forge-1.5B-ReceiptAgent — public candidate carrying a merged safetensors artifact plus a separately bound adapter directory;
  • SZLHOLDINGS/SZL-Forge-ReceiptAgent-Eval — frozen, rights-reviewed evaluation set;
  • SZLHOLDINGS/SZL-Forge-ReceiptAgent-Schemas — schemas and conformance fixtures;
  • SZLHOLDINGS/SZL-Forge-ReceiptAgent-Demo — fail-closed Space with live load and inference receipts; and
  • SZLHOLDINGS/SZL-Forge-Collection — collection created last, after every member exists.

release-manifest.json remains a preregistration snapshot and still marks every target PLANNED_NOT_CREATED; it must not be silently rewritten into historical evidence. Live inventory now contradicts that state for the first candidate, which is why this qualification binds the external revision explicitly. The other IDs require separate live verification. A model card alone must not be presented as a weight-bearing model.

Identity, license, and naming

The base license is reported by existing local evidence as Apache-2.0. Adapter and dataset release licenses remain blocked pending lineage and row-level rights review. Promotion also requires hashes for base files, adapter config and weights, tokenizer, chat template, dataset manifest and split, code commit, environment lock, evaluation manifest, and output schema.

The final release must carry a signed DSSE/in-toto statement and an independently readable transparency-log entry. It must disclose Qwen2.5-derived lineage and non-affiliation with Alibaba/Qwen, Unsloth, Hugging Face, Anthropic, and Claude. It must never use another vendor's model name to obscure lineage.

Bounded implementation sequence

  1. Resolve item-level data rights; do not admit raw Brain or formula holdout rows.
  2. Build a new, owned or explicitly licensed ReceiptAgent curriculum that targets the v1 contract.
  3. Preregister immutable train/eval splits and the full three-way evaluation matrix.
  4. Train once from the pinned base and capture code, environment, seed, GPU, and artifact receipts.
  5. Run every suite against base, existing SFT, and the new artifact under identical settings.
  6. Stop on any catastrophic event or failed admission gate.
  7. Have a human reviewer approve the model card, data statement, licenses, and evidence bundle.
  8. Sign and log provenance; publish the adapter, eval data, schemas, and working Space; create the collection last.

Local validation

The focused test uses only the Python standard library:

python -m unittest tests.test_receipt_agent_release_program

It checks the JSON documents against the dependency-light schema subset used here, asserts the exact repository facts, and exercises fail-closed contract conditions. It performs no network calls or external mutations.