experiments/decision-20260928-v2/REVIEW_HANDOFF.md

Experimental pass handoff — 2026-09-28

Status

Version1 job 6aba3fb36b030d633f69c572 was canceled before training to fix repeated NPZ decompression. The replacement runs the seven-arm suite. Source, architecture preflight, parameter counts and upload/download verification are already committed to experiments/decision-20260928-v2 on main. At this handoff no trained candidate has been reviewed or approved. Do not describe the suite as complete or declare a winner without reading its outputs.

Root production remains v3 at 1a9eb8af2754ad2329a24cfe50d388cb559441d0. The next pass should review this suite before launching the final longer run. Never overwrite root model files merely because an experimental score is lower.

Exact procedure for selecting the final course

1. Read status.json, failure.json if present, summary.json and candidates.json. Verify every candidate's actual steps and parameter count; a partial/timed-out screen cannot establish the missing comparison. All model counts include encoder and selector.
2. Read each development result and selected test result. The test is newly held out from project training and listed prior evaluations; upstream encoder/teacher exposure and near duplicates cannot be excluded. Use paired intervals as exploratory diagnostics, not multiple-comparison-corrected proof.
3. View comparison JPEGs before reading their key.json files. Inspect both full frames and useful crops. Review all 32 fixed COCO examples for single/multi groups, the encoder group, and actual-aspect production_test images. Log visible findings per image: incoherent hue changes on one surface, missing colour, boundary leakage, implausible skin/sky/foliage, whole-image colour cast. An alternate plausible clothing colour is not a failure simply because it differs from the original.
4. View the seven probes, historical comparison images, and actual-aspect production_archive images. Read archive_sources.json to identify failed downloads; do not report attempted images as evaluated. Supplement with the six National Archives inputs whose source records are in supplement/nara_sources.json. These are evaluation-only. Never supervise historical grayscale inputs with zero chroma.
5. Inspect multi diagnostics: all mode frequencies, diversity, default-v-oracle losses and resize/flip mode switches. An oracle-only gain is insufficient for automatic deployment. An optional alternatives UI may be valuable but must not conceal a worse default. Inspect MobileViT visual outputs in light of its shorter colourisation history; do not declare a universally inferior backbone from a short warm-start test.
6. Prefer a candidate only if images show a consistent benefit without unacceptable loss of colour, and quantitative diagnostics do not reveal collapse. Retain v3 as the fallback. If one method looks clearly better, a second-seed matched confirmation is authorized within the remaining pass budget; maximum additional L4 runtime18min, not an unbounded rerun. Previous attempt used about12min; replacement capped60min at$0.80/h. Preserve all results before exit.
7. Write REVIEW.md and final_config.json with the actual checkpoint path, immutable commit, architecture, complete parameter count, recipe, seed, dataset revisions, teacher-cache revision and validation evidence. If nothing wins, state that plainly and use the established palette architecture. Do not promise that longer continuation necessarily improves it.

Specific implementation caveats to carry forward

• Single/multi arms: same initial encoder and decoder, index batches, learning rates and update counts. Staged augmentation changes random-number consumption late in training; its flip/target-choice draws are not identical to teacher/mix. This is a curriculum-bundle comparison.
• MobileViT receives1000 teacher alignment steps plus3000 curriculum steps. Version2 uses the post-warmup curriculum step for both augmentation/target ramps and consistency, correcting the canceled first attempt.
• Each arm's candidate can come from an earlier development checkpoint. optimizer.pt belongs to the last step, not necessarily that selected checkpoint. Start a new optimizer from selected candidate weights; never pair candidate weights with a mismatched optimizer.
• Existing r7 loss uses teacher/original targets for supervision; neither original colour error nor patch score alone measures plausibility.
• Do not add RL without an independently validated reward. Previous critic failures are project evidence for caution, not proof RL can never help.
• Keep original-image luminance and aspect ratio through the established inference pipeline. Single/multi DecisionColorizer needs its own loader; root load_semantic cannot silently load a new architecture.
• If deploying MobileViT, include its upstream Apple license/notice and appropriate pretrained-model attribution.

Inference compute experiment already inspected

Seven real historical photographs were evaluated with unchanged v3 weights at256/384/512 input size and the existing radius8 filter. On the six new National Archives images, larger size notably improves the children-in-a-field image (more coherent green foliage and blue sky), and slightly clarifies some people. It does not fix the near-monochrome casts on machinery, street and interior scenes. The familiar powerhouse pipe retains uneven colouring and changes hue with size. Therefore higher resolution is useful in some cases but is not a general cure. At fixed radius8, its physical smoothing footprint also changes: do not call this an isolated network-resolution ablation. Record measured CPU times as local indicative timings, not a deployment latency guarantee; they include a second diagnostic forward.

Final-pass scope

After evidence review, start one longer run on the selected course, with checkpoint retention, bounded runtime, early exit for non-finite losses, durable metrics and uploads. Final production promotion requires comparing its saved checkpoints visually to v3 and the selected pilot. Develop the Space/ZeroGPU app, requirements and model card, and clean the repository without removing experiments or the stable branch needed by the user. Never state that the model is mathematically optimal or that no improvement is possible under4M parameters.