Update experimental weights with private Exam-QA pilot

README.md

@@ -20,8 +20,11 @@ This is an **experimental**, uncalibrated choice-ranking model. It is not an off
 ## Provenance
 
 - Base: [knowledgator/gliclass-instruct-large-v1.0](https://huggingface.co/knowledgator/gliclass-instruct-large-v1.0), Apache-2.0, 438,672,897 parameters.
-- Additional fine-tuning: one epoch on 2,414 Practical V1 training records, peak learning rate 2.5e-6, 151 optimizer steps, microbatch 2, gradient accumulation 8, fp16 AMP.
-- Data: original synthetic Japanese, English, and Simplified Chinese examples in six task families: everyday arithmetic, tool choice, dialogue action, JSON conversation-log routing, reading inference, and original exam-style questions. Generated and answer-blind rejudged with DeepSeek V4.1 Flash via OrcaRouter. No actual entrance-exam questions or private chat logs were used.
+- First fine-tune: one epoch on 2,414 Practical V1 training records, peak learning rate 2.5e-6, 151 optimizer steps, microbatch 2, gradient accumulation 8, fp16 AMP.
+- Second, exploratory fine-tune (2026-09-23): one selected epoch on 175 privately held Exam-QA transformations mixed with 175 deterministic Practical V1 replay records, learning rate 1.5e-6, 22 optimizer steps, maximum training length 1,024 tokens.
+- Practical V1 data consists of original synthetic Japanese, English, and Simplified Chinese examples in six task families, generated and answer-blind rejudged with DeepSeek V4.1 Flash via OrcaRouter.
+- The Exam-QA source was filtered and transformed with the same DeepSeek model. Symbolic answer labels were mapped to source choice text. Ambiguous, multi-answer, figure-dependent, partial-credit, incomplete, or over-1,024-token items were skipped. Generated distractors were train-only; validation used source-provided choices only.
+- Exam-QA source records, transformed JSONL, and API responses are **not published** pending human review and source-by-source redistribution review. They are not claimed as human gold.
 - Data and training code: [GitHub repository](https://github.com/sugarkwork/erabi/tree/main/data/practical_v1) and [training script](https://github.com/sugarkwork/erabi/blob/main/scripts/train_practical_v1.py). Labels remain **unreviewed synthetic teacher agreement**, not human gold.
 
 ## Exploratory evaluation
@@ -34,6 +37,17 @@ This is an **experimental**, uncalibrated choice-ranking model. It is not an off
 
 The Practical V1 eval set was used once after selecting by dev and existing-bridge results. Reading inference **regressed** from 54/71 to 50/71 despite aggregate gains. Candidate-order consistency on the existing bridge was 97.50%. These figures are not a benchmark of real-world correctness or Jev parity, because Practical V1 questions and labels come from the same teacher family. There is no independent human-verified final test, temperature calibration, or formal release approval for this checkpoint.
 
+The current weights add the private Exam-QA experiment to that checkpoint:
+
+| Set | Before Exam-QA fine-tune | Current weights |
+|---|---:|---:|
+| Private Exam-QA validation, source choices only, 37 cases | 21.62% (8/37) | **24.32% (9/37)** |
+| Practical V1 dev, 399 cases | 77.19% (308/399) | **78.20% (312/399)** |
+| Existing RC3 Bridge, 480 cases | 88.54% (425/480) | **88.54% (425/480)** |
+| Practical V1 teacher-agreed eval, 386 cases | 76.17% (294/386) | 75.65% (292/386) |
+
+The Exam-QA gain is only **one additional correct item**, so it is weak exploratory evidence, not a claim of exam competence. The validation set has just nine source groups and has not been independently human-audited. The current safetensors SHA256 is `1ae38ef6c1103f8c021b0d3a974f3b1aedc42d89832d11761e74b95664216251`.
+
 ## Use
 
 ```bash
@@ -43,7 +57,7 @@ erabi predict --request request.json
 
 The repository contains three inference formats from the same checkpoint: `model.safetensors` (PyTorch), `onnx/fp32/model.onnx` (CPU), and `onnx/fp16/model.onnx` (NVIDIA GPU). ERABI 0.1.2 or newer is required for `--model-format`; 0.1.1 uses PyTorch safetensors. Upgrade with `python -m pip install --upgrade erabi`. `--model-format auto` downloads only the selected variant: FP32 ONNX for CPU with ONNX Runtime, FP16 ONNX for CUDA with CUDA Execution Provider, and otherwise PyTorch safetensors. Install the compatible `onnxruntime` (CPU) or `onnxruntime-gpu` (GPU) separately; do not install both in one environment. You can also select `--model-format pytorch`, `onnx-fp32`, or `onnx-fp16` explicitly.
 
-The ONNX variants preserved the top-ranked choice on 90/90 unreviewed synthetic comparison cases; this is not independent human-gold validation. Experimental INT8 variants changed predictions substantially and are not distributed. The first invocation downloads the selected model; later invocations use the Hugging Face cache. Input and output JSON contracts and runtime recommendations are documented in the [ERABI README](https://github.com/sugarkwork/erabi#モデル形式の自動選択とおすすめ). The input limit is 512 tokens; overlong inputs are rejected rather than silently truncated. The candidate probabilities are not calibrated confidence guarantees.
+The newly exported ONNX FP32 and FP16 variants preserved the PyTorch top-ranked choice on 37/37 private Exam-QA validation cases, up to 853 input tokens. Experimental INT8 variants changed predictions substantially and are not distributed. The first invocation downloads the selected model; later invocations use the Hugging Face cache. Input and output JSON contracts and runtime recommendations are documented in the [ERABI README](https://github.com/sugarkwork/erabi#モデル形式の自動選択とおすすめ). The public ERABI runtime still defaults to a 512-token fail-closed contract. The weights were trained and experimentally checked at up to 1,024 tokens, but using that length requires changing both the runtime limit and preprocessing length while checking the untruncated input. Candidate probabilities are not calibrated confidence guarantees.
 
 ## License and limitations
 

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1902d31124ed9eae00b1938990cca736a31be1eb94bfa4125fc4b6853c1ac052
+oid sha256:1ae38ef6c1103f8c021b0d3a974f3b1aedc42d89832d11761e74b95664216251
 size 1754746792

onnx/fp16/model.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:84cd8cc9c4830ba2156185a6e9aea3d151733fc0f258fc45a77d00e6289483a7
+oid sha256:4b67d0ec581bb86815d0cebffe24dcdac16b976fedc1ad391b51e9c3eacb60ff
 size 879507317

onnx/fp32/model.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:70180838b2f51c72ce3c06f9b8c5059b26233af991eea2e795e6801c94b3b929
+oid sha256:4ce73286fc49cc472c7160ac23c4e6a90924f576bc4b075e3053d5245b889350
 size 1756853324