fix(eval): correct blind-judge attribution and prompt matching

Analysis revision 2. Frozen generations and judge records unchanged. Correct A/B decoding, reversed prompt joins, prompt counts, and dependent-rating uncertainty claims.

README.md

@@ -76,38 +76,77 @@ The finished artifact passed local structural and oMLX load checks on 2026-09-20
 
 The artifact contains five safetensors shards, 1,876 indexed tensors, and 29 MTP tensors. The index references no missing shards. These checks confirm that oMLX recognizes and loads the files; they do not establish quality parity with the BF16 source model.
 
-## Quality evaluation (v1)
-
-This model is part of the [Hemmingway-1 oMLX quantization collection](https://huggingface.co/collections/sixstringzen/hemmingway-1-omlx-oqe-quantizations-apple-silicon-evidence-6ab06103552bc3c7e9ad8afa) and was compared with a clean BF16 reference in a bounded blind A/B writing study.
-
-### Local measurement provenance
-
-Generations were produced by real MLX/oMLX software on an Apple M5 Max with 128 GB unified memory under a deterministic quant-isolation profile. MTP and mixed-runtime speculative accelerators were disabled for the baseline. Captured local manifests, generation records, and oMLX/macmon telemetry are the measurement evidence. Claude monitored or orchestrated a subset of the local tests; Claude is workflow provenance, not the inference engine or measurement source.
-
-### Blind-judge result
-
-The study used 14 writing tasks, three judge lanes, and normal/swapped response order. Results below pool the candidate comparisons across Claude Opus 5, Gemini 3.8 Flash, and Grok 4.7:
-
-| Build | Win | Loss | Tie |
-| --- | ---: | ---: | ---: |
-| oQ2e | 47.8% | 51.1% | 1.1% |
-| oQ3.5e | 36.7% | 58.9% | 4.4% |
-| oQ3e | 55.6% | 43.3% | 1.1% |
-| oQ4e | 32.2% | 48.9% | 18.9% |
-| oQ6e | 27.8% | 31.1% | 41.1% |
-| oQ8e | 31.1% | 23.3% | 45.6% |
-
-The hosted reference row in the local report is a cap-matched subset of 11 prompts, not another quantization condition. Raw inter-rater agreement was 83.7% across 606 pairings. Normal-versus-swapped agreement was 39.6% for Claude, 48.5% for Gemini, and 42.6% for Grok, so the results should be read as subjective and order-sensitive.
-
-### Interpretation for this build
-
-High-bit perceptual-reference group in this suite, with a 41.1% tie rate. This is not mathematical equivalence.
-
-This pass does not establish mathematical equivalence, token-level fidelity, or general benchmark superiority. KLD, top-1/top-k agreement, KV-cache divergence, and broader benchmark task scores remain separate pending measurements. The public task and metadata package is the [companion benchmark dataset](https://huggingface.co/datasets/sixstringzen/hemmingway-1-omlx-quantization-benchmark-v1).
+## Quality evaluation (v1, corrected analysis revision 2)
+
+Corrected on 2026-09-22 after identifying errors in A/B decoding and normal/swapped prompt matching. The generated responses and judge records are unchanged. See the [correction record](https://huggingface.co/datasets/sixstringzen/hemmingway-1-omlx-quantization-benchmark-v1/blob/main/CORRECTION.md).
+
+Each quantization was compared with the local BF16 reference on 15 prompts.
+Three judge lanes evaluated both response orders, producing 90 ratings per quant.
+The hosted comparison used 11 cap-matched prompts and produced 66 ratings.
+The study contains 14 packet files per judge, each holding multiple cases.
+
+The ratings describe one frozen output per condition and prompt. Multiple judges
+and swapped orders do not create independent generation samples. We report
+counts, percentages, and mean score differences without confidence intervals or
+statistical significance claims. Percentages can differ from 100% after rounding.
+
+Judges scored instruction adherence, task fit, clarity and control, and writing
+judgment on a 0-4 scale. Scores and overall preferences are separate judgments.
+The Claude lane used manual chats, except the final hosted swapped packet,
+which used OpenRouter. Gemini and Grok used OpenRouter. The Grok collection
+includes documented recovery of packets 04, 05, and 06.
+
+| Condition | Wins | Losses | Ties | Ratings | Win % | Loss % | Tie % |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| oQ2e | 31 | 58 | 1 | 90 | 34.4 | 64.4 | 1.1 |
+| oQ3.5e | 44 | 42 | 4 | 90 | 48.9 | 46.7 | 4.4 |
+| oQ3e | 54 | 35 | 1 | 90 | 60.0 | 38.9 | 1.1 |
+| oQ4e | 30 | 43 | 17 | 90 | 33.3 | 47.8 | 18.9 |
+| oQ6e | 34 | 19 | 37 | 90 | 37.8 | 21.1 | 41.1 |
+| oQ8e | 27 | 22 | 41 | 90 | 30.0 | 24.4 | 45.6 |
+
+After matching prompts across orders, agreement was 89.1% for Claude, 85.1% for Gemini, and 83.2% for Grok (101 pairs per judge). Inter-rater agreement was 83.7% across 606 pairings. These rates measure agreement on the frozen outputs; they do not validate the judges' preferences.
+
+### This build
+
+Frequent ties with BF16 and more wins than losses in this suite.
+
+### Local generation profile
+
+Local generations ran through MLX/oMLX on an Apple M5 Max with 128 GB unified
+memory. Captured server records identify oMLX `0.7.0.dev2`. The generation profile
+used temperature 0, top-p 1, min-p 0, repetition penalty 1, and seed 42. MTP and
+speculative acceleration were disabled for this baseline. The files preserve
+MTP tensors for separate runtime experiments.
+
+The quant runs recorded 90 completions at 512 tokens, 24 at 1024, and 18 at 2048.
+The selected BF16 reference recorded 11, one, and three completions at those caps.
+The judge packets selected the final cap for each prompt: 11 at 512, one at
+1024, and three at 2048. Earlier truncated attempts remain execution records
+and were excluded from the judge packets. Hosted retained a separate 11-prompt
+comparison because four prompts did not have matching output caps.
+
+Local manifests, generation records, and oMLX/macmon telemetry carry execution
+evidence. Grafana displays the telemetry. Background tracing remained active,
+so controlled throughput and memory comparisons require a separate run.
+
+This small, fixed suite cannot establish a universal ranking or quantify
+token-level fidelity. The higher observed oQ3e win rate does not imply that
+reducing precision improves the source model in general. High tie rates for
+oQ6e and oQ8e do not establish equivalence to BF16. The study includes no direct
+quant-versus-hosted comparison, and the hosted deployment's exact checkpoint,
+precision, and runtime are not verified.
+
+KLD, top-1/top-k agreement, KV-cache divergence, broader task scores, and
+controlled runtime comparisons remain pending. Raw generations, provider
+responses, packet mappings, and telemetry remain in the local evidence package.
+The public dataset contains prompts, selected execution metadata, and aggregates.
+
+[Companion dataset](https://huggingface.co/datasets/sixstringzen/hemmingway-1-omlx-quantization-benchmark-v1).
 
 ## Limitations
 
-Quantization can change word choice, coherence, and instruction following. A controlled BF16 comparison has not been published for this build.
+Quantization can change word choice, coherence, and instruction following. The writing comparison above covers a fixed prompt suite; token-level fidelity remains unmeasured.
 
 The sensitivity pass used oMLX's `oqe_code_multilingual` calibration dataset. No prose-specific calibration dataset was used. The MTP tensors are present in the artifact, but MTP-assisted decoding has not been benchmarked separately.