experiments/round6-20260928/PROTOCOL.md

Round6 — semantic encoder and coherent-color distillation

Decision and research

Round5 lowered excess color differences but increased missed color (final distilled gray validation: 20.36% to 36.53%). It did not meet release gates. The loss compared student color against both photographic ground truth and a fallible teacher; these may prefer different hues. The encoder had no semantic pretraining. These are hypotheses to test, not proven root causes.

Primary research:

• DDColor, ICCV 2023: https://arxiv.org/abs/2212.11613 — combines multiscale image features and color queries. The authors' https://github.com/piddnad/DDColor/blob/master/MODEL_ZOO.md warns that colorfulness loss can create color blocks. We use artistic weights without that loss.
• PalGAN, ECCV 2022: https://arxiv.org/abs/2210.11204 — estimates a palette and assigns color conditioned on it. This motivates shared image-level color decisions; our implementation is an original small experiment, not a reproduction of PalGAN.
• TorchVision 0.21 MobileNetV3 Large: https://docs.pytorch.org/vision/0.21/models/generated/torchvision.models.mobilenet_v3_large.html — ImageNet pretrained semantic features. The classifier is discarded; every remaining learned parameter is counted, including the encoder.

Matched experiments

Both students use torchvision 0.21 MobileNetV3 Large IMAGENET1K_V2 features, a 128-channel feature pyramid, Lab-lightness input converted internally to neutral sRGB, and bilinear chroma upsampling. Both assert total parameters strictly below 4,000,000.

1. Dense decoder: local convolutional color predictor over pretrained features.
2. Palette decoder: 16 learned queries, two self/cross-attention blocks over multiscale image features, 16 image-wide Lab colors and semantic assignment masks, with a bounded +/-2 Lab local residual.

Both train on identical 16,230 images, matched seeds/data order and objectives. Supervision is a single deterministic DDColor artistic prediction per image, cached at 64x64. No per-pixel averaging of original and teacher target colors. ImageNet teacher revision aa10f72fffc89a6658e37b48556050b4d9a26f63 is fixed. Bundled DDColor source revision 2adb63f2656ac41cbdf7b894cddd94121a3faf13, license included. Teacher mistakes can be inherited.

Loss: L1 color matching /20; +0.25 sliced distribution matching on 8 fixed Lab projection directions, using sorted 32x32 samples; +0.1 teacher-boundary matching. Distribution matching penalizes losing target color without specifying every pixel's hue. There is no unconditional saturation bonus and no penalty driving color gradients to zero.

10,000 updates per decoder, batch32, AdamW, encoder max LR3e-5, decoder max LR3e-4; 500-step warmup and cosine decay to 10%; weight decay0.01, gradient clip1; BF16 forward, FP32 losses. Encoder BN frozen for first500 updates, then adapts at momentum0.01. Flip and mild brightness/contrast variation. Teacher targets remain fixed through photometric augmentation. Baseline v2 weights and datasets pinned as in earlier experiments.

Evaluation and gates

Every1,000 updates: old150-image development validation, ordinary grayscale and fixed film-style grayscale, per-image metrics and 7 visual probes. Every500: model, optimizer/scheduler and progress saved. All evaluated model snapshots retained.

Eligibility on both gray styles: at least30% lower patch-excess proxy,20% lower raw patch proxy; Lab error<=1.05x baseline; neutral spill<=baseline+1 percentage point; missed color<=baseline+1 point; color coverage>=95% baseline. These are screening gates, not proof of perceptual quality. Lower score alone never approves release. Candidate review grids include original color, baseline and student, with fixed20-image samples. Review candidates are separately retained even when ineligible and explicitly marked experimental.

Final reporting uses old300-image development test and fresh300-image COCO validation subset, seed1609, from default/val/0000.parquet at 26ddc382fe75dfc2a0655b5977e296ea10efebce. Fresh holdout is never used for selection; exact training-byte duplicates excluded. Pretrained encoder/teacher may have upstream overlap; near duplicates not excluded. True historical grayscale probe images have no known color ground truth.

Persistence and deployment

Sources, protocol, initial checkpoints and CPU results saved on main under experiments/round6-20260928. Every changed artifact is downloaded at its commit and SHA256 checked. Persistence failure stops training after3 attempts. Teacher cache shards are saved, enabling reuse without paying to regenerate all targets. Stdout is capped; full text logs saved separately. Job maximum5h, internal graceful limit4.5h. No claim of exact optimizer/data-order resume; states are available for deliberate continuation.

The root app checkpoint is only replaced after visual review and compatibility work for this different architecture. Stable is untouched. The root U-Net loader cannot directly load these checkpoints. All experiment artifacts are committed to main. Production requires exported inference/Space integration, reload tests, acceptable actual grayscale-photo results, and documented failure cases. No experiment can prove that further improvement at the same parameter budget is impossible.