mini-unet-colorizer / reports /round2 /selection_protocol.md
User-2468's picture
Update validated colorizer weights, spatial decoder and deployment artifacts
704fa80 verified
|
Raw History Blame
2.4 kB

Round 2 frozen evaluation protocol

The 13-way spatial ablation used only the existing 50-image validation set. Before scoring the new 200-image Imagenette subset or the first100 COCO-val images, nominate guided8 for a single-pass balanced option; flip_guided4 as a two-pass fidelity option; guided16 as a stronger smoothing comparison. The latter visibly removes some small legitimate colour details, so is not automatically selected by its lower artifact proxy. Baseline remains in all comparisons. Temperature stays0.38 and saturation stays1.0. No test-based parameter retuning is planned.

Training candidates are compared on the original validation set, with raw CE, chroma error, mean chroma, excess colour edges and visual checks. Do not promote a candidate solely because it is grayer or smoother. Select weights on validation before scoring them on the new200 and COCO probe. The first100 COCO-val rows are an external convenience sample, not a representative population benchmark. No COCO photo is used for training.

CPU pilots:400updates,batch2,LR2e-6,seed0. GPU arms:1122updates,batch32, three full epochs,LR1e-5,seed123. Both pairs use frozen BatchNorm, fixed original class weights and bins,11943filtered training images. Within each pair the sole change is multiscale chroma-gradient loss coefficient0 vs10. CPU-to-GPU comparisons also change batch,LR,seed,precision and compute; do not attribute their differences to one factor.

Broader-data addendum

The mixed-data arm was added during the exploratory round. It retains the same 1122updates and32-image batch as the classification control, adding5382 eligible COCO training photos from two pinned shards. This is approximately2.07epochs over17325photos, not three epochs. Its held-out200COCO training photos are excluded from all training and are separate from the100COCO-val external probe.

For final weight selection, compare the eligible candidates on the original50 Imagenette validation photos and those200COCO training-validation photos. Use equal-domain mean relative ab error with guided8, retaining at least90percent of the repaired baseline mean chroma in each validation domain and checking that artifact proxies do not regress. This is a pragmatic development gate, not a perceptual realism metric. The100COCO-val images are not used for this weight ranking. Inspect visual failures before accepting the winner.