Document final training decision and organize research history

DEPLOYMENT.md

@@ -1,34 +1,18 @@
-# v3 deployment
+# Deployment and final-run promotion
 
-## Local Python
+The live Space is https://huggingface.co/spaces/User-2468/mini-unet-colorizer.
+Default model revision remains `1a9eb8af2754ad2329a24cfe50d388cb559441d0` (v3.0). The final longer job is https://huggingface.co/jobs/User-2468/6abaef6552d0dbd7f1da7286.
 
-Use Python 3.10 or 3.11 with the supplied requirements. CPU inference works without a GPU. Install a CUDA-compatible PyTorch build for GPU execution. Load once, put the model in evaluation mode, and reuse it across requests. The supplied loader does this automatically.
+## ZeroGPU
+Use `app.py`, `inference.py`, `semantic_model.py`, `decision_model.py`, and `requirements-space.txt` (renamed `requirements.txt` in the Space). Space README SDK is Gradio6.28.0; requirements use the same version. Select ZeroGPU hardware. Import `spaces` before `torch`; load/place the model on CUDA at startup. Only chroma inference runs inside `@spaces.GPU(duration=15)`. CPU handles image preparation, guided rendering, and cached strength changes. Do not enable `torch.compile` in ZeroGPU.
 
-```python
-from inference import load_colorizer, colorize
-model = load_colorizer('.', 'cpu')
-```
+For local CPU validation set `ZEROGPU_CPU_TEST=1`; `MODEL_ID` can be a local checkpoint directory. Image dimensions are preserved, maximum12MP, with PNG alpha. Cache is session-scoped with a10-minute lifetime. Upload changes invalidate cached predictions. Standard/Detailed/Maximum correspond to maximum network dimensions256/384/512. Strength changes after inference do not use another GPU request.
 
-The runtime loads a single 3,994,676-parameter network. The training-only teacher and discriminator are not required or downloaded.
+## Final checkpoint review
+1. Inspect `experiments/final-20260929/status.json`, `candidates.json`, `summary.json`, final test/paired diagnostics and image grids. A candidate with step0 is the retained pilot, not a failed upload.
+2. Compare actual images to v3 and the pilot, especially historical originals and colour retention. Do not promote solely on patch score or original-colour error.
+3. Verify strict parameter count, output dimensions/alpha/lightness, CPU/CUDA loading and all palette modes. Use `decision_model.load_decision` for a DecisionColorizer checkpoint.
+4. Pin Space `MODEL_REVISION` to the evaluated immutable commit and `MODEL_SUBFOLDER` to the chosen directory. Alternative controls become visible for four-palette models.
+5. For a new root release, replace weights/config/loader together, update card and hashes, regenerate and verify ONNX or archive the old graph with an explicit version. The current v3 ONNX must never be described as the new four-palette model. Rebuild the release ZIP, tag the release, and run a real ZeroGPU request.
 
-## Hugging Face Space
-
-Copy `app.py`, `inference.py`, `semantic_model.py`, and `requirements-space.txt` into a Gradio Space. Rename the latter to `requirements.txt`. Use Gradio 6.28.0 in the Space README and choose ZeroGPU hardware.
-
-The released app pins `MODEL_REVISION` to the verified model commit. Setting this environment variable deliberately overrides that pin. `MODEL_ID` can be a local directory for offline use or a Hub repository ID.
-
-Import `spaces` before PyTorch. The model is placed on CUDA once at startup, and only the network stage is inside `@spaces.GPU`. Image preparation and full-resolution rendering happen on CPU. `ZEROGPU_CPU_TEST=1` enables the local CPU path for verification.
-
-## Input handling
-
-The image pipeline applies EXIF orientation, preserves alpha, rejects images over 12 megapixels, and restores colour at the original output dimensions. Network processing preserves aspect ratio with a maximum 256-pixel side. Errors are returned explicitly rather than replaced by a blank image.
-
-The default API `/colorize` takes an image, colour strength (0–1.5), and smoothing (`Gentle`, `Balanced`, or `Strong`). It returns a PNG. Requests are serialized to bound memory usage.
-
-## Versioning and rollback
-
-Model and source are committed atomically. The app pins the model commit rather than following moving main. `RELEASE_MANIFEST.json` records selection and previous commits. To roll back, use the recorded prior Space commit and model revision. The user-managed stable branch is not modified.
-
-## Verification scope
-
-`QA.json` records tests for strict reload, parameter count, finite outputs, dimensions, transparency, black/white inputs, grayscale strength zero, input validation, a Gradio HTTP request, and ONNX parity. `RELEASE_REPORT.md` records visual selection and the separate live Space check. These checks do not imply factual colour reconstruction or eliminate every visual failure case.
+Training outputs are saved on main, but production promotion requires this review. Preserve the user's stable branch.

README.md

@@ -15,6 +15,16 @@ library_name: pytorch
 
 A compact automatic photo colouriser with **3,994,676 learned parameters in total**, including the complete deployed MobileNetV3 encoder. It predicts plausible colours from grayscale photographs. No teacher, critic, external semantic model, retrieval service or ensemble is needed at inference.
 
+## Current status — 29 September 2026
+
+[Try the ZeroGPU Space](https://huggingface.co/spaces/User-2468/mini-unet-colorizer).
+
+Root weights and ONNX remain the reviewed **v3.0** release. A final continuation of the3,995,832-parameter four-palette challenger is running under [`experiments/final-20260929`](experiments/final-20260929); it is **not yet approved as a production replacement**. See the [visual review and training decision](reports/FINAL_TRAINING_REVIEW.md) and [exact configuration](training/final_config.json). The pilot showed modest patch reductions, not elimination of blotches.
+
+The updated Space offers Standard, Detailed and Maximum input detail, full-resolution PNG output, and cached strength adjustments. Higher input detail can help some photos but can also change their colours. The default stays at256 pixels. The Space can load future multi-palette checkpoints after review without downloading arbitrary Python code.
+
+Historical research documents have moved to `reports/history/`. See [the report index](reports/README.md). The `stable` branch and existing release weights are preserved.
+
 ## Release contents
 
 - `model.safetensors` and `config.json`: the selected checkpoint.

SHA256SUMS.json

@@ -1,23 +1,32 @@
 {
-  "DEPLOYMENT.md": "201879644b48093363f025589641bc2a264ec8097512b4b4c30b621f3b9d9ee6",
+  "DEPLOYMENT.md": "dac782d3f07838731ced3c0b7a21cfd24ca9943ea9e9660ba6e0dfa5f86de5c0",
   "PIPELINE_EVAL.json": "595be88704d580c01a3c4dd642409b3dc5dd25e86ca1c7eb40a696605f181a19",
   "QA.json": "b249b1d49045e31b0bf85f9e01515b91a72c4dd973963751c30d14272d262b90",
-  "README.md": "5ade5fd9022a03937b91fbd3b0fa653ae93c068034b54b35677c3db47a9ccc59",
+  "README.md": "cdca79dbc42aeb31be15b61adc67f50746096b8fab9d95963232622cc8f691b5",
   "RELEASE_MANIFEST.json": "872eebfb322c8990b5e6ac493e4e4068b87f8daac0ddf219b8b55299514ca03c",
   "RELEASE_REPORT.md": "69677ec5e13f5c1249f59b01c31ed1b0f1588bddc216bce9167b5d31423a5574",
-  "app.py": "58be79b62642c2f3393860791a1a7b39d8f2e83ac79caf6545839124ec787404",
-  "colorization_project_log.md": "699238b4078fca93b15f4f2d951e439651502338dac4bf3a5b5a6d0b0449de21",
+  "app.py": "27832b99d910c1ff12d728d00e460060d061ab4c30aae2d8b2d256d4c894b8fa",
   "colorize_image.py": "9a85b7633bdf8a0d63f740ae69ed76644bb2087da4b21f8c1e01ca804cd4e6b2",
   "colorize_onnx.py": "cc106f066878a7372f4260279620238d46a3841baf09fbbe954f61a684199d05",
   "colorizer.json": "7d444d5f560b2ea6a070f73750f75f66aa2c61d3dc21a8e4300ed5f5196da179",
   "colorizer.onnx": "911fb777328fcfbeeb89cc25dca70092cf2de4686ede1edd44a8ccebc89ea341",
   "config.json": "eaeb49b39f9116bfed0d954e81852dc93476298752787dba88a63c0eec97ccec",
+  "decision_model.py": "c7c4370d93bc52fdd363d6d4fcc684c317806d9b393bd969f9ee670a01487f27",
   "export_onnx.py": "f46a413c2b1b2a6e130cb7d2ad8d6d69c16a16955bf830efdc4c04c30d47ca60",
   "inference.py": "0ac1f382205b42cef0afaa9173017fa660e6859194a65a61b9b9052c437cd476",
   "model.py": "31628744fb5d362d91342063c90eb38b2b97fda8c91e303e4a58b6f388f53c55",
   "model.safetensors": "ec1f27d74533adc83f7ab3639a091fc4d8738a434dafc7d172c7873c28a9e715",
+  "reports/FINAL_TRAINING_REVIEW.md": "2c354152ed07c22852401cff66865782840ffb143941a8f92a7ed4fd8542988f",
+  "reports/README.md": "b50c4af1398d0a8c56bd6bf1a1561f95bd8187c68f057f803ef5fddf4c718f90",
+  "reports/SPACE_QA_multi.json": "990353b85936624c24a71b4c2183d158251cd122ff23b1ee026eae94bca5715b",
+  "reports/SPACE_QA_v3.json": "78e8d283a7c7c0913705ee4a26fceae61c444fd836d29f784ec1bc88e46b53a4",
+  "reports/history/RESEARCH_REPORT.md": "6730cf20f7a584707fb0c084b882ac87db1ef858d72e7d5e716e5e0d382ddca1",
+  "reports/history/RESEARCH_ROUND2.md": "19f42c72003cb685f46b315105ad2d4e62a10a7e0cd1455ab11db51bd5083728",
+  "reports/history/colorization_project_log.md": "699238b4078fca93b15f4f2d951e439651502338dac4bf3a5b5a6d0b0449de21",
   "requirements-onnx.txt": "9bcca098a9da63b31fb9d803dfd7f99b753ad125ee9bce3a5bbe52e368739e11",
-  "requirements-space.txt": "4c7f918aadc4afb31ec6808131570f9978cf0bcf9e2af562e93adc749e3245cf",
+  "requirements-space.txt": "bc5b3edd619278a37e0cdc499ab9a87057cdbd2023f7c22c08b6a437c5701c65",
   "requirements.txt": "c855eb70020dc89f71fd38e44a0be2cfb80f2fe3d442d5ba45c91398c336f83f",
-  "semantic_model.py": "b019358cbcaa214743cbd244d20eb6e59eeff0ab5264093ec0927be8529b25a2"
+  "semantic_model.py": "b019358cbcaa214743cbd244d20eb6e59eeff0ab5264093ec0927be8529b25a2",
+  "training/final_config.json": "39502272a7bbc05c1ad15e70de4e93550b53cdd9fbea399192d1b400311205dd",
+  "training/train_final.py": "ba42f07649c05cfcf8d7a6cee78088a4ed0838f4e09e24227cf9fb3509b34b5b"
 }

app.py

@@ -1,47 +1,81 @@
-"""Gradio and ZeroGPU entry point. The model revision is pinned at release."""
+"""Pinned-release Gradio app. Import spaces before torch for ZeroGPU emulation."""
 import os
 from pathlib import Path
 import spaces
 import torch
 import gradio as gr
 from huggingface_hub import snapshot_download
-from inference import load_colorizer, prepare, chroma_coefficients, render
+from inference import prepare, chroma_coefficients, render, load_colorizer
+import json
 
 REPO=os.getenv('MODEL_ID','User-2468/mini-unet-colorizer')
-REV=os.getenv('MODEL_REVISION','v3.0.0')
+REV=os.getenv('MODEL_REVISION','1a9eb8af2754ad2329a24cfe50d388cb559441d0')
+SUBFOLDER=os.getenv('MODEL_SUBFOLDER','').strip('/')
 CPU=os.getenv('ZEROGPU_CPU_TEST')=='1'
 DEVICE='cpu' if CPU else 'cuda'
 if CPU:torch.set_num_threads(min(4,os.cpu_count() or 1))
 path=Path(REPO)
-if not path.is_dir():path=Path(snapshot_download(REPO,revision=REV,allow_patterns=['model.safetensors','config.json']))
-MODEL=load_colorizer(path,DEVICE)
+if not path.is_dir():
+    prefix=SUBFOLDER+'/' if SUBFOLDER else ''
+    path=Path(snapshot_download(REPO,revision=REV,allow_patterns=[prefix+'model.safetensors',prefix+'config.json']))
+if SUBFOLDER:path=path/SUBFOLDER
+cfg=json.loads((path/'config.json').read_text())
+if cfg.get('architecture')=='DecisionColorizer':
+    from decision_model import load_decision
+    MODEL=load_decision(path,DEVICE)
+else:MODEL=load_colorizer(path,DEVICE)
+MODEL.eval().requires_grad_(False)
+PARAMETERS=sum(p.numel() for p in MODEL.parameters())
+if PARAMETERS>=4_000_000:raise RuntimeError('Model exceeds the four-million-parameter limit.')
+MODES=getattr(MODEL,'modes',1)
 
-@spaces.GPU(duration=20)
-def infer(small,radius):
-    return chroma_coefficients(MODEL,small,int(radius))
+class PaletteView(torch.nn.Module):
+    def __init__(self,model,mode):super().__init__();self.model=model;self.mode=mode
+    def forward(self,L):
+        return self.model(L) if self.mode<0 else self.model(L,True)['all'][:,self.mode]
 
-def run(image,strength,smoothing):
+@spaces.GPU(duration=15)
+def infer(small,radius,mode):
+    return chroma_coefficients(PaletteView(MODEL,mode),small,radius)
+
+def run(image,strength,smoothing,detail,mode):
     if image is None:raise gr.Error('Upload a photograph first.')
-    radius={'Balanced':8,'Gentle':4,'Strong':16}[smoothing]
+    mode=int(mode)
+    if mode < -1 or mode>=MODES or (MODES==1 and mode!=-1):raise gr.Error('That colour interpretation is unavailable.')
     try:
-        light,alpha,small=prepare(image,256)
-        coefficients=infer(small,radius)
+        radius={'Gentle':4,'Balanced':8,'Strong':16}[smoothing]
+        size={'Standard':256,'Detailed':384,'Maximum':512}[detail]
+        light,alpha,small=prepare(image,size)
+        coefficients=infer(small,radius,mode)
         result=render(light,alpha,coefficients,strength)
-    except (ValueError,RuntimeError,TypeError) as exc:
+        cache=(light,alpha,coefficients)
+    except (ValueError,RuntimeError,TypeError,KeyError) as exc:
         raise gr.Error(str(exc)) from exc
-    return result
+    return result,cache
+
+def recolour(strength,cache):
+    if cache is None:return gr.skip()
+    light,alpha,coefficients=cache
+    return render(light,alpha,coefficients,strength)
 
-with gr.Blocks(title='Mini Photo Colorizer') as demo:
-    gr.Markdown('# Mini Photo Colorizer\nAdd plausible colour to a black-and-white photograph. Original colours may be unknowable.')
+with gr.Blocks(title='Mini Photo Colorizer',delete_cache=(3600,3600)) as demo:
+    gr.Markdown('# Give an old photograph new colour\nPlausible colour, with the original detail and proportions preserved.')
+    cache=gr.State(value=None,time_to_live=600)
     with gr.Row():
-        source=gr.Image(type='pil',image_mode='RGBA',label='Original photograph',sources=['upload','clipboard'],height=440)
-        result=gr.Image(type='pil',label='Colourised photograph',format='png',height=440)
+        source=gr.Image(type='pil',image_mode='RGBA',label='Your photograph',sources=['upload','clipboard'],height=460)
+        result=gr.Image(type='pil',label='Colourised photograph',format='png',height=460,interactive=False)
     with gr.Row():
-        strength=gr.Slider(0,1.5,value=1.,step=.05,label='Colour strength')
+        button=gr.Button('Colourise',variant='primary',scale=2)
+        clear=gr.ClearButton([source,result,cache],value='Clear',scale=1)
+    strength=gr.Slider(0,1.5,value=1.,step=.05,label='Colour strength',info='Adjust after colourising without another GPU request.')
+    with gr.Accordion('Fine-tune the result',open=False):
+        detail=gr.Radio(['Standard','Detailed','Maximum'],value='Standard',label='Detail',info='Higher detail can change the colours. Try Detailed if small objects are missed.')
         smoothing=gr.Radio(['Gentle','Balanced','Strong'],value='Balanced',label='Colour smoothing')
-    button=gr.Button('Colourise',variant='primary')
-    button.click(run,[source,strength,smoothing],result,api_name='colorize',concurrency_limit=1)
-    gr.ClearButton([source,result])
-    gr.Markdown('3.995 million parameters · Full-resolution output, up to 12 megapixels · PNG download\n\nBest suited to ordinary photographs. Colours can be muted or inaccurate on unusual scenes, tiny objects and heavily damaged scans.')
+        palette=gr.Dropdown([('Automatic',-1)]+[(f'Alternative {i+1}',i) for i in range(MODES)] if MODES>1 else [('Automatic',-1)],value=-1,label='Colour interpretation',visible=MODES>1)
+        gr.Markdown('After changing these settings, select **Colourise** again.')
+    button.click(run,[source,strength,smoothing,detail,palette],[result,cache],api_name='colorize',concurrency_limit=1)
+    strength.release(recolour,[strength,cache],result,api_name=False,concurrency_limit=1)
+    source.change(lambda:None,outputs=cache,api_name=False,queue=False)
+    gr.Markdown('Download the result as PNG using its download button. Up to 12 megapixels; transparency is preserved.\n\nColours are interpretations, not recovered historical facts. Unusual scenes, damaged scans and tiny objects can still produce muted colours or colour bleeding. [About the model](https://huggingface.co/User-2468/mini-unet-colorizer).')
 demo.queue(default_concurrency_limit=1,max_size=12)
-if __name__=='__main__':demo.launch()
+if __name__=='__main__':demo.launch(max_file_size='30mb')

decision_model.py

@@ -0,0 +1,117 @@
+"""Single / multiple coherent palettes, optional compact global-attention encoder.
+
+Original experiment inspired by diverse colorization and multiple-choice learning;
+not a reproduction. All deployed parameters, including the selector, are counted.
+"""
+import json
+from pathlib import Path
+import torch
+from torch import nn
+import torch.nn.functional as F
+from safetensors.torch import save_file, load_file
+from semantic_model import SemanticColorizer
+
+
+class DecisionColorizer(SemanticColorizer):
+    def __init__(self, modes=1, backbone='mobilenet', pretrained=False,
+                 encoder_revision=None, **kwargs):
+        super().__init__(pretrained=False, **kwargs)
+        self.modes = modes
+        self.backbone = backbone
+        d = self.config['width']; q = self.config['queries']
+        if backbone == 'mobilevit':
+            import timm
+            from huggingface_hub import hf_hub_download
+            name = 'mobilevitv2_075.cvnets_in1k'
+            overlay = None
+            if pretrained:
+                assert encoder_revision, 'Pin pretrained encoder revision'
+                filename = hf_hub_download('timm/'+name, 'model.safetensors', revision=encoder_revision)
+                overlay = {'file': filename, 'hf_hub_id': None}
+            self.encoder = timm.create_model(name, pretrained=pretrained,
+                pretrained_cfg_overlay=overlay, features_only=True, out_indices=(1,2,3,4))
+            self.lateral = nn.ModuleList([nn.Conv2d(c,d,1) for c in self.encoder.feature_info.channels()])
+            cfg = self.encoder.pretrained_cfg
+            self.rgb_mean.copy_(torch.tensor(cfg['mean']).view(1,3,1,1))
+            self.rgb_std.copy_(torch.tensor(cfg['std']).view(1,3,1,1))
+        if modes > 1:
+            self.mode_embeddings = nn.Parameter(torch.randn(modes,d)*.03)
+            self.mode_offsets = nn.Parameter(torch.zeros(modes,q,2))
+            # Whole-image palette alternatives; no independently sampled pixels.
+            with torch.no_grad():
+                a=torch.arange(modes)*2*torch.pi/modes
+                self.mode_offsets[:,:,0]=.025*a.cos()[:,None]
+                self.mode_offsets[:,:,1]=.025*a.sin()[:,None]
+            self.mode_score = nn.Linear(d,modes)
+            nn.init.zeros_(self.mode_score.weight);nn.init.zeros_(self.mode_score.bias)
+        self.config.update(architecture='DecisionColorizer', format_version=3,
+            modes=modes, backbone=backbone, encoder_revision=encoder_revision)
+        count = sum(p.numel() for p in self.parameters())
+        assert count < 4_000_000, count
+
+    def features(self,L):
+        h,w=L.shape[-2:]
+        x=F.pad(self.neutral_rgb(L),(0,(-w)%32,0,(-h)%32),mode='replicate')
+        x=(x-self.rgb_mean)/self.rgb_std
+        if self.backbone=='mobilevit':
+            fs=self.encoder(x)
+        else:
+            fs=[]
+            for i,layer in enumerate(self.encoder):
+                x=layer(x)
+                if i in [3,6,12,16]:fs.append(x)
+        projected=[layer(f) for layer,f in zip(self.lateral,fs)]
+        x=projected[-1]
+        for i in range(2,-1,-1):
+            x=self.refine[2-i](F.interpolate(x,size=projected[i].shape[-2:],mode='bilinear',align_corners=False)+projected[i])
+        memory=self.memory_norm(torch.cat([F.adaptive_avg_pool2d(f,(8,8)).flatten(2).transpose(1,2) for f in projected[1:]],1))
+        q=self.queries[None].expand(L.shape[0],-1,-1)
+        for block in self.query_blocks:q=block(q,memory)
+        q=self.query_norm(q)
+        masks=torch.einsum('bqd,bdhw->bqhw',q,self.pixel(x))/(self.config['width']**.5)
+        weights=F.softmax(masks.float(),dim=1)
+        residual=2*torch.tanh(self.residual(x).float())
+        return q,weights,residual,memory.mean(1)
+
+    def forward(self,L,return_details=False):
+        q,weights,residual,context=self.features(L)
+        if self.modes==1:
+            palette=80*torch.tanh(self.palette(q))[:,None]
+            scores=q.new_zeros((len(L),1))
+        else:
+            mq=q[:,None]+self.mode_embeddings[None,:,None]
+            palette=80*torch.tanh(self.palette(mq)+self.mode_offsets[None])
+            scores=self.mode_score(context)
+        outputs=torch.einsum('bqhw,bkqc->bkchw',weights,palette.float())+residual[:,None]
+        b,k,c,h,w=outputs.shape
+        full=F.interpolate(outputs.reshape(b*k,c,h,w),size=(h*4,w*4),mode='bilinear',align_corners=False)
+        full=full[...,:L.shape[-2],:L.shape[-1]].reshape(b,k,c,*L.shape[-2:])
+        default=full[torch.arange(b,device=L.device),scores.argmax(1)]
+        if return_details:
+            return {'output':default,'all':full,'scores':scores.float(),
+                    'weights':weights,'palette':palette.float()}
+        return default
+
+
+def initialize_from_v3(model,baseline):
+    old=baseline.state_dict();state=model.state_dict()
+    copied=[]
+    for k,v in old.items():
+        if model.backbone=='mobilevit' and (k.startswith(('encoder.','lateral.')) or k in ['rgb_mean','rgb_std']):continue
+        if k in state and v.shape==state[k].shape:
+            state[k]=v;copied.append(k)
+    model.load_state_dict(state,strict=True)
+    return copied
+
+
+def save_decision(m,path):
+    p=Path(path);p.mkdir(parents=True,exist_ok=True)
+    save_file({k:v.detach().cpu().contiguous() for k,v in m.state_dict().items()},str(p/'model.safetensors'))
+    (p/'config.json').write_text(json.dumps(m.config,indent=2))
+
+
+def load_decision(path,device='cpu'):
+    p=Path(path);cfg=json.loads((p/'config.json').read_text())
+    m=DecisionColorizer(**{k:cfg[k] for k in ['modes','backbone','width','queries','head','encoder_revision']})
+    m.load_state_dict(load_file(str(p/'model.safetensors')),strict=True)
+    return m.to(device).eval()

reports/FINAL_TRAINING_REVIEW.md

@@ -0,0 +1,25 @@
+# Final training decision — 29 September 2026
+
+All seven pilot arms completed. Reviewed all 32 fixed COCO examples in each of the single, multi and encoder comparison groups, all14 historical photos in production-pipeline grids, and the14-photo multi-palette archive comparison. This is qualitative assistant review, not an independent human preference study.
+
+## Decision
+Continue the MobileNetV3 four-palette model with mixed complete-image teacher/original targets, initialized from `experiments/decision-20260928-v2/multi_mix/candidate` (pilot step2000). Complete deployed model:3,995,832 parameters. This is a modest best-compromise candidate, not a proven major advance over v3. Retain released v3 until final visual review.
+
+## Evidence
+Relative to v3, multi_mix guided patch excess decreased4.69% on ordinary grayscale (0.81598→0.77772) and9.37% on film grayscale (0.77398→0.70148). Exploratory paired95% intervals for the absolute changes were[-0.07208,-0.00611] and[-0.10446,-0.04150]. These intervals do not correct for selecting among seven experiments.
+Mean chroma was almost retained:14.0299→13.9184, and13.7452→13.6813. Coverage nevertheless fell from54.44%→52.63% and52.99%→52.30%; missed-colour rates slightly improved, with uncertain paired differences. Original-colour error changed little. No metric is a direct perceptual-quality guarantee.
+
+MobileViT had the lowest patch metric but its images often had a yellow/brown cast: skateboarder background, tennis player and court, lake, hand holding food, kitchen hand, and indoor scenes. It also lost useful colour distinctions in the dance scene and food images. It improved some landscape/elephant details and the historical children/sky, but the overall trade-off is unsuitable for release. Its selected step1000 precedes its staged curriculum, so those scores do not validate that curriculum.
+
+Single mixed/staged outputs were generally close to v3 and sometimes smoother, but reduced colour coverage and saturation. Single_staged selected step1000, before the late robustness phase; it is not evidence that the late film/consistency phase worked. Multi_staged retained more colour but its ordinary-grayscale patch score worsened. Thus the final course uses mixed targets without the unvalidated late-stage bundle.
+
+Multi_mix preserves recognisable blue sky, green foliage and skin distinctions in the modern examples, with less evidence of the broad sepia substitution seen in MobileViT. Historical images remain substantially similar to v3. The old man's clothes and the children's field remain muted; the powerhouse pipe still changes hue along its length. The power station still receives a pink cast. No claim that blotches are eliminated.
+
+The multi selector chose modes[39,8,1,0] on48 diagnostic images; alternatives are unevenly used. Default target loss0.5174 versus oracle0.4234 shows room for selection improvement, but oracle performance is not the deployable result. Resize changed the selected mode in1/48 examples. Keep all parameters and the learned default; do not choose an output using unknown ground truth at inference.
+
+## Final run
+Up to24,000 additional updates, batch24, all16,230 existing filtered training images, cached teacher/original targets sampled50/50 per image. Fresh optimizer; encoder LR2e-6 and decoder LR2e-5, warmup150, cosine floor15%, frozen encoder batch statistics, clipping1. No RL, critic, new teacher generation, or architecture expansion. Save every2000 updates and retain the initial pilot as a valid candidate. Stop after six evaluations without a development-score improvement of0.002. The development score retains colour/error guardrails.
+
+New development/test samples exclude the pilot's development/test IDs and the earlier evaluation IDs. Test opens after development checkpoint selection. Final outputs and image comparisons are pushed under `experiments/final-20260929` on main; root production weights are not automatically overwritten. A lower metric is insufficient for promotion.
+
+Budget: one L4,60min hard timeout ($0.80 maximum at the currently documented rate),50min internal training limit. Expected about25–40min from measured pilot throughput, with setup and evaluation overhead. No automatic retry jobs.

reports/README.md

@@ -0,0 +1,11 @@
+# Reports and experiment history
+
+- [Final training decision, visual review and limitations](FINAL_TRAINING_REVIEW.md)
+- [Initial research report](history/RESEARCH_REPORT.md)
+- [Second research report](history/RESEARCH_ROUND2.md)
+- [Historical project log](history/colorization_project_log.md)
+- [Current v3 release report](../RELEASE_REPORT.md)
+- [Seven-arm architecture/curriculum comparison](../experiments/decision-20260928-v2/PROTOCOL.md)
+- [Final continuation outputs](../experiments/final-20260929/)
+
+The historical log includes earlier U-Net versions and superseded conclusions. For deployed architecture, parameter counts and inference, use the root model card and its pinned release. Experiment checkpoints are not automatically approved releases.

reports/SPACE_QA_multi.json

@@ -0,0 +1,15 @@
+{
+  "parameters": 3995832,
+  "gradio": "6.28.0",
+  "rgba_and_dimensions": true,
+  "cached_strength": true,
+  "max_lightness_quantization_error": 0.23923175575065514,
+  "blocks_constructed": true,
+  "tested_modes": [
+    -1,
+    0,
+    1,
+    2,
+    3
+  ]
+}

reports/SPACE_QA_v3.json

@@ -0,0 +1,11 @@
+{
+  "parameters": 3994676,
+  "gradio": "6.28.0",
+  "rgba_and_dimensions": true,
+  "cached_strength": true,
+  "max_lightness_quantization_error": 0.24549842656874432,
+  "blocks_constructed": true,
+  "tested_modes": [
+    -1
+  ]
+}

requirements-space.txt

@@ -7,3 +7,4 @@ safetensors==0.5.3
 huggingface-hub>=1.0,<2
 gradio==6.28.0
 spaces==0.51.3
+timm==1.0.19

training/final_config.json

@@ -0,0 +1,21 @@
+{
+  "architecture": "DecisionColorizer",
+  "backbone": "mobilenet",
+  "modes": 4,
+  "parameters": 3995832,
+  "initial_revision": "12dfd8111622f692a7f69d0a961ce9b65f044c01",
+  "initial_subfolder": "experiments/decision-20260928-v2/multi_mix/candidate",
+  "output_subfolder": "experiments/final-20260929",
+  "seed": 9292026,
+  "max_steps": 24000,
+  "batch_size": 24,
+  "encoder_lr": 2e-06,
+  "decoder_lr": 2e-05,
+  "original_target_probability": 0.5,
+  "checkpoint_interval": 2000,
+  "early_stopping_patience_evaluations": 6,
+  "early_stopping_min_delta": 0.002,
+  "automatic_production_promotion": false,
+  "job_id": "6abaef6552d0dbd7f1da7286",
+  "job_url": "https://huggingface.co/jobs/User-2468/6abaef6552d0dbd7f1da7286"
+}

training/train_final.py

@@ -0,0 +1,368 @@
+"""Budget-bounded architecture/curriculum screening. Sources pinned by bootstrap."""
+import os,sys,json,time,math,random,hashlib,shutil,traceback,copy,io
+from pathlib import Path
+import numpy as np
+import torch
+from torch import nn
+import torch.nn.functional as F
+from PIL import Image,ImageDraw
+from skimage.color import rgb2lab
+import cv2,pyarrow as pa,pyarrow.parquet as pq
+from huggingface_hub import HfApi,hf_hub_download
+from safetensors.torch import load_file
+from semantic_model import load_semantic
+from decision_model import DecisionColorizer,initialize_from_v3,save_decision,load_decision
+import persistence
+import r7 as r
+from data import decode_image
+
+REPO='User-2468/mini-unet-colorizer'
+BASE='1a9eb8af2754ad2329a24cfe50d388cb559441d0'
+ARTIFACTS='12dfd8111622f692a7f69d0a961ce9b65f044c01'
+CACHE_REV='7b10485ee9d205ce2f7b660dfc6a90012d78b1cb'
+COCO_REV='26ddc382fe75dfc2a0655b5977e296ea10efebce'
+PREFIX=os.environ.get('EXPERIMENT_PREFIX','experiments/final-20260929')
+SEED=int(os.environ.get('SEED','9292026'))
+STEPS=int(os.environ.get('STEPS','24000'));BATCH=24
+START=time.monotonic();TRAIN_LIMIT=float(os.environ.get('TRAIN_LIMIT_SECONDS','3000'))
+OUT=Path('/work/results');OUT.mkdir(exist_ok=True,parents=True)
+persistence.PREFIX=PREFIX;r.OUT=OUT;r.ev.OUT=OUT;r.DEVICE='cuda';r.ev.DEVICE='cuda';r.ev.deadline=lambda:None
+api=HfApi();DURABLE=None
+def write(n,obj):
+    p=OUT/n;p.parent.mkdir(parents=True,exist_ok=True);p.write_text(json.dumps(obj,indent=2));return obj
+def dl(f,rev=ARTIFACTS):return Path(hf_hub_download(REPO,f,revision=rev))
+def seed(s):torch.manual_seed(s);np.random.seed(s);random.seed(s)
+
+def objective(details,target,step):
+    allout=details['all'];b,k=allout.shape[:2]
+    losses=r.losses(allout.flatten(0,1),target[:,None].expand(-1,k,-1,-1,-1).flatten(0,1)).reshape(b,k)
+    if k==1:return losses.mean(),losses.argmin(1),losses
+    # One winning complete image, never a per-pixel best-of-K patchwork.
+    winner=losses.detach().argmin(1)
+    selected=losses.gather(1,winner[:,None]).mean()
+    # Small nonwinner term prevents completely untrained branches; no saturation reward.
+    loss=.95*selected+.05*losses.mean()+.1*F.cross_entropy(details['scores'],winner)
+    return loss,winner,losses
+
+class Batches:
+    def __init__(self,rgb,teacher):
+        self.rgb=torch.from_numpy(rgb).cuda().permute(0,3,1,2)
+        self.teacher=torch.from_numpy(teacher).cuda().float()
+        self.n=len(rgb)
+    @staticmethod
+    def light(g):
+        y=torch.where(g<=.04045,g/12.92,((g+.055)/1.055).pow(2.4))
+        l=torch.where(y>.008856,116*y.clamp_min(1e-9).pow(1/3)-16,903.296296*y)
+        return l/50-1
+    def batch(self,step,recipe,original,curriculum_step=None):
+        phase=step if curriculum_step is None else curriculum_step
+        gen=torch.Generator(device='cuda').manual_seed(SEED*100000+step)
+        ids=torch.randint(self.n,(BATCH,),generator=gen,device='cuda')
+        rgb=self.rgb[ids].float()/255;t=self.teacher[ids];o=original[ids]
+        g=(rgb*rgb.new_tensor([.299,.587,.114])[None,:,None,None]).sum(1,keepdim=True)
+        # Emulate standard 8-bit PIL grayscale conversion before Lab conversion.
+        g=(g*255).round()/255
+        L=self.light(g)
+        if recipe=='staged' and phase>STEPS*2/3:
+            take=torch.rand(BATCH,1,1,1,generator=gen,device='cuda')<.35
+            weights=torch.rand(BATCH,3,1,1,generator=gen,device='cuda')+rgb.new_tensor([.8,1.2,.2])[None,:,None,None]
+            weights/=weights.sum(1,keepdim=True)
+            film=(rgb*weights).sum(1,keepdim=True)
+            gamma=.85+.4*torch.rand(BATCH,1,1,1,generator=gen,device='cuda')
+            film=(film.pow(gamma)-.5)*(.8+.3*torch.rand(BATCH,1,1,1,generator=gen,device='cuda'))+.5
+            film=film+.005*torch.randn(film.shape,generator=gen,device='cuda')
+            L=torch.where(take,self.light(film.clamp(0,1)),L)
+        else:
+            gain=.9+.2*torch.rand(BATCH,1,1,1,generator=gen,device='cuda')
+            L=(L*gain).clamp(-1,1)
+        flips=torch.rand(BATCH,1,1,1,generator=gen,device='cuda')<.5
+        L=torch.where(flips,L.flip(-1),L);t=torch.where(flips,t.flip(-1),t);o=torch.where(flips,o.flip(-1),o)
+        prob=0 if recipe=='teacher' else .5
+        if recipe=='staged':prob*=min(max((phase/STEPS-.15)/.35,0),1)
+        pick=torch.rand(BATCH,1,1,1,generator=gen,device='cuda')<prob
+        # Stochastic full targets: never choose the easier one, never average hues.
+        target=torch.where(pick,o,t)
+        return L,target,ids,float(pick.float().mean())
+
+def summary(result):return {s:result[s]['summary'] for s in ['gray','film']}
+def score(result,baseline):
+    val=0;okay=True
+    for style in ['gray','film']:
+        s=result[style]['summary'];b=baseline[style]['summary']
+        okay &= s['missed_color']<=b['missed_color']+.015 and s['neutral_spill']<=b['neutral_spill']+.015 and s['color_coverage']>=.95*b['color_coverage'] and s['ab_error']<=b['ab_error']*1.10
+        val+=s['patch_excess']/max(b['patch_excess'],1e-6)+.2*s['ab_error']/b['ab_error']+2*s['missed_color']+s['neutral_spill']
+    return val/2,bool(okay)
+
+@torch.inference_mode()
+def diagnostics(model,table,ids):
+    model.eval();rows=[]
+    for idx in ids:
+        L,t,_=r.ev.gray_arrays(decode_image(table,idx));x=torch.from_numpy(L)[None].cuda()
+        v=model(x,True);a=v['all'];sel=int(v['scores'].argmax(1))
+        resized=model(F.interpolate(x,(224,224),mode='bilinear',align_corners=False),True)
+        p=F.interpolate(resized['output'],(256,256),mode='bilinear',align_corners=False)
+        flip=model(x.flip(-1),True)
+        colors=a[0].mean((-1,-2));k=model.modes
+        diff=(a[:,:,None]-a[:,None]).abs().mean((0,3,4,5))
+        target=torch.from_numpy(t)[None].cuda();target64=F.avg_pool2d(target,4)
+        losses=r.losses(a.flatten(0,1),target64.expand(k,-1,-1,-1))
+        mask=v['weights'].clamp_min(1e-8)
+        rows.append({'index':int(idx),'selected_mode':sel,'resized_mode':int(resized['scores'].argmax(1)),
+            'flip_mode':int(flip['scores'].argmax(1)),'resize_ab_change':float((p-v['output']).abs().mean()),
+            'flip_ab_change':float((flip['output'].flip(-1)-v['output']).abs().mean()),
+            'mode_diversity':float(diff.sum()/max(k*(k-1),1)),
+            'default_target_loss':float(losses[sel]),'oracle_target_loss':float(losses.min()),
+            'mask_entropy':float(-(mask*mask.log()).sum(1).mean()),
+            'palette_chroma':float(v['palette'].norm(dim=-1).mean()),'output_chroma':float(v['output'].norm(dim=1).mean())})
+    keys=['resize_ab_change','flip_ab_change','mode_diversity','default_target_loss','oracle_target_loss','mask_entropy','palette_chroma','output_chroma']
+    return {'rows':rows,'summary':{k:float(np.mean([x[k] for x in rows])) for k in keys},
+        'mode_frequency':np.bincount([x['selected_mode'] for x in rows],minlength=model.modes).tolist(),
+        'resize_switch_fraction':float(np.mean([x['selected_mode']!=x['resized_mode'] for x in rows])),
+        'note':'Oracle uses unknown original only for diagnosis. Default mode is selected by the learned image-only score, never by ground truth.'}
+
+def paired_bootstrap(result,base):
+    rng=np.random.default_rng(937);out={}
+    for style in ['gray','film']:
+        br={z['index']:z for z in base[style]['per_image']};rows=[z for z in result[style]['per_image'] if z['target_chroma']>=5]
+        out[style]={}
+        for key in ['patch_excess','raw_patch_excess','missed_color','neutral_spill','color_coverage','ab_error']:
+            d=np.array([z[key]-br[z['index']][key] for z in rows]);boot=d[rng.integers(len(d),size=(3000,len(d)))].mean(1)
+            out[style][key]={'delta':float(d.mean()),'ci95':np.quantile(boot,[.025,.975]).tolist()}
+    return out
+
+@torch.inference_mode()
+def pipeline_grid(models,table,ids,label):
+    from inference import colorize
+    from PIL import ImageOps
+    names=list(models);order=np.random.default_rng(9184).permutation(len(names))
+    key={chr(65+j):names[int(i)] for j,i in enumerate(order)};write(label+'_key.json',key)
+    for start in range(0,min(32,len(ids)),4):
+        canvas=Image.new('RGB',(320*(len(names)+1),344*len(ids[start:start+4])),'white');d=ImageDraw.Draw(canvas)
+        for row,idx in enumerate(ids[start:start+4]):
+            original=decode_image(table,idx).convert('L').convert('RGB')
+            original.thumbnail((800,800))
+            d.text((2,row*344),f'{idx}: grayscale | '+' | '.join(key),fill='black')
+            canvas.paste(ImageOps.pad(original,(320,320),color='white'),(0,row*344+24))
+            for col,j in enumerate(order):
+                output=colorize(models[names[int(j)]].eval(),original,size=256,radius=8)
+                canvas.paste(ImageOps.pad(output,(320,320),color='white'),((col+1)*320,row*344+24))
+        canvas.save(OUT/f'{label}_{start//4}.jpg',quality=93)
+
+@torch.inference_mode()
+def resolution_grid(model,table,records):
+    from inference import colorize
+    from PIL import ImageOps
+    for start in range(0,len(table),3):
+        count=min(3,len(table)-start)
+        canvas=Image.new('RGB',(1280,344*count),'white');d=ImageDraw.Draw(canvas)
+        for row,idx in enumerate(range(start,start+count)):
+            im=decode_image(table,idx).convert('L').convert('RGB');im.thumbnail((1000,1000))
+            d.text((2,row*344),records[idx]['name']+' | input | 256 | 384 | 512',fill='black')
+            canvas.paste(ImageOps.pad(im,(320,320),color='white'),(0,row*344+24))
+            for col,size in enumerate([256,384,512],1):
+                pred=colorize(model,im,size=size,radius=8)
+                canvas.paste(ImageOps.pad(pred,(320,320),color='white'),(col*320,row*344+24))
+        canvas.save(OUT/f'v3_resolution_{start//3}.jpg',quality=94)
+
+def main():
+    global DURABLE
+    assert api.whoami()['name']=='User-2468'
+    DURABLE=persistence.DurableRun(OUT)
+    shutil.copytree('/work/source',OUT/'source',dirs_exist_ok=True,ignore=shutil.ignore_patterns('__pycache__'))
+    shutil.copy2('/work/PROTOCOL.md',OUT/'PROTOCOL.md')
+    if Path('/work/REVIEW_HANDOFF.md').exists():shutil.copy2('/work/REVIEW_HANDOFF.md',OUT/'REVIEW_HANDOFF.md')
+    if Path('/work/supplement').exists():shutil.copytree('/work/supplement',OUT/'supplement',dirs_exist_ok=True)
+    write('status.json',{'status':'preflight','seed':SEED,'production_approved':False})
+    DURABLE.sync('source and write verification before training')
+    torch.set_num_threads(6);seed(SEED)
+    torch.backends.cuda.matmul.allow_tf32=False;torch.backends.cudnn.allow_tf32=False
+    initial=Path('/work/initial');initial.mkdir(exist_ok=True)
+    for f in ['model.safetensors','config.json']:shutil.copy2(dl(f,BASE),initial/f)
+    baseline=load_semantic(initial,'cuda').requires_grad_(False)
+    counts={};smokes={}
+    encoder_rev=None
+    write('encoder_provenance.json',{'backbone':'MobileNetV3 Large','initial_revision':ARTIFACTS,'initial_subfolder':'experiments/decision-20260928-v2/multi_mix/candidate'})
+    constructors=[('multi',4,'mobilenet')]
+    for name,k,backbone in constructors:
+        seed(SEED);m=DecisionColorizer(modes=k,backbone=backbone,pretrained=backbone=='mobilevit',encoder_revision=encoder_rev if backbone=='mobilevit' else None).cuda()
+        copied=initialize_from_v3(m,baseline)
+        seedpath=Path('/work/pilot');seedpath.mkdir(exist_ok=True)
+        for f in ['config.json','model.safetensors']:shutil.copy2(dl('experiments/decision-20260928-v2/multi_mix/candidate/'+f),seedpath/f)
+        m=load_decision(seedpath,'cuda');m.eval();count=sum(p.numel() for p in m.parameters());assert count<4_000_000
+        counts[name]=count
+        x=torch.rand(2,1,97,113,device='cuda')*2-1
+        with torch.no_grad():
+            v=m(x,True);assert v['all'].shape==(2,k,2,97,113) and torch.isfinite(v['all']).all()
+            parity=float((v['output']-baseline(x)).abs().max()) if name=='single' else None
+            if parity is not None:assert parity<1e-3,parity
+        loss,_,_=objective(m(x,True),torch.randn(2,2,64,64,device='cuda'),1)
+        loss.backward();assert torch.isfinite(loss) and all(torch.isfinite(p.grad).all() for p in m.parameters() if p.grad is not None)
+        m.zero_grad(set_to_none=True);save_decision(m,OUT/'initial'/name)
+        re=load_decision(OUT/'initial'/name,'cuda')
+        with torch.no_grad():assert torch.allclose(m(x),re(x),atol=1e-4)
+        smokes[name]={'parameters':count,'copied_tensors':len(copied),'odd_shape':True,'finite_backward':True,'reload':True,'baseline_max_abs':parity}
+        del m,re
+    write('smoke.json',smokes);DURABLE.sync('all three architectures pass preflight')
+    torch.backends.cuda.matmul.allow_tf32=True;torch.backends.cudnn.allow_tf32=True;torch.backends.cudnn.benchmark=True
+    tables=[]
+    for repo,rev,files in [('johnowhitaker/imagenette2-320','771c1310a2487e8076ede6b7d6307244aa8400af',['default/train/0000.parquet']),('detection-datasets/coco',COCO_REV,['default/train/0000.parquet','default/train/0001.parquet'])]:
+        for f in files:
+            print('PREP downloading',repo,f,flush=True)
+            tables.append(pq.read_table(hf_hub_download(repo,f,repo_type='dataset',revision=rev),columns=['image']))
+            print('PREP loaded',repo,f,len(tables[-1]),flush=True)
+    table=pa.concat_tables(tables)
+    full_manifest=json.loads(dl('experiments/region-coherence-20260928/source/previous_manifest.json').read_text())
+    full_ids=full_manifest['train']
+    # Same fixed training subset for every arm, independent of new dev/test.
+    ids=list(full_ids);pos={i:j for j,i in enumerate(ids)}
+    teacher=np.empty((len(ids),2,64,64),np.float16)
+    sums=json.loads(dl('experiments/round6-20260928/SHA256SUMS.json',CACHE_REV).read_text())
+    for start in range(0,len(full_ids),2048):
+        f=f'teacher_cache/{start:05d}.npz';p=dl('experiments/round6-20260928/'+f,CACHE_REV)
+        assert hashlib.sha256(p.read_bytes()).hexdigest()==sums[f]
+        with np.load(p) as z:
+            # NpzFile indexing decompresses a member on every access. Materialize once.
+            cache_ids=z['indices'];cache_targets=z['target']
+            assert cache_ids.tolist()==full_ids[start:start+len(cache_ids)]
+            src=[j for j,i in enumerate(cache_ids) if int(i) in pos]
+            dst=[pos[int(cache_ids[j])] for j in src]
+            teacher[dst]=cache_targets[src]
+            print('PREP cache',start,'selected',len(src),flush=True)
+    print('PREP decoding training images',len(ids),flush=True)
+    rgb=np.stack([np.asarray(decode_image(table,i).resize((256,256),Image.Resampling.BILINEAR)) for i in ids])
+    original=np.stack([cv2.resize(rgb2lab(im.astype(np.float32)/255)[...,1:],(64,64),interpolation=cv2.INTER_AREA).transpose(2,0,1) for im in rgb]).astype(np.float32)
+    batcher=Batches(rgb,teacher);original=torch.from_numpy(original).cuda();del rgb,teacher
+    print('PREP training tensors ready',flush=True)
+    fresh=pq.read_table(hf_hub_download('detection-datasets/coco','default/val/0001.parquet',repo_type='dataset',revision=COCO_REV),columns=['image'])
+    exclude=set()
+    for path,keys in [('decision-20260928-v2',['development','test']),('round7-20260928',['development','heldout']),('region-coherence-20260928',['development_regression','heldout_regression','new_holdout']),('recurrent-refinement-20260928',['fresh_holdout','development_regression'])]:
+        manifest=json.loads(dl('experiments/'+path+'/manifest.json').read_text())
+        for key in keys:exclude.update(manifest[key])
+    hashes={hashlib.sha256(table['image'][i].as_py()['bytes']).digest() for i in full_ids}
+    freshids=[int(i) for i in np.random.default_rng(SEED).permutation(len(fresh)) if int(i) not in exclude and hashlib.sha256(fresh['image'][int(i)].as_py()['bytes']).digest() not in hashes][:288]
+    assert len(freshids)==288
+    dev,test=freshids[:128],freshids[128:]
+    write('manifest.json',{'train':ids,'development':dev,'test':test,'prior_eval_excluded':sorted(exclude),'coco_revision':COCO_REV,'fresh_file':'default/val/0001.parquet','teacher_revision':CACHE_REV,'base_revision':BASE,'seed':SEED,'near_duplicates_and_upstream_overlap_not_excluded':True})
+    base_dev=r.ev.evaluate(baseline,fresh,dev);write('baseline_development.json',base_dev)
+    DURABLE.sync('dataset checks and unseen development/test split frozen')
+    arms=[('multi','mix')]
+    if os.environ.get('ARMS'):arms=[tuple(x.split(':')) for x in os.environ['ARMS'].split(',')]
+    selected={};histories={};dev_results={};begin_train=time.monotonic()
+    for head,recipe in arms:
+        name=head+'_'+recipe;seed(SEED)
+        m=load_decision(OUT/'initial'/head,'cuda')
+        # New encoder gets a separate alignment warmup, explicitly outside matched six arms.
+        warmup=1000 if head=='mobilevit' else 0
+        groups=[{'params':[p for n,p in m.named_parameters() if n.startswith('encoder.')],'lr':2e-5 if head=='mobilevit' else 2e-6},
+                {'params':[p for n,p in m.named_parameters() if not n.startswith('encoder.')],'lr':2e-4 if head=='mobilevit' else 2e-5}]
+        opt=torch.optim.AdamW(groups,weight_decay=.01)
+        for g in opt.param_groups:g['base_lr']=g['lr']
+        history=[];beststep=0;last=0;arm_start=time.monotonic();mode_counts=np.zeros(m.modes,np.int64);stale=0
+        initial_res=r.ev.evaluate(m,fresh,dev);val,okay=score(initial_res,base_dev);best=val+(0 if okay else 100)
+        save_decision(m,OUT/name/'candidate');dev_results[name]=initial_res
+        write(name+'/development_00000.json',initial_res);DURABLE.sync('retain initial pilot candidate before continuation')
+        for s in range(1,STEPS+warmup+1):
+            if time.monotonic()-START>TRAIN_LIMIT:break
+            step=max(1,s-warmup);active_recipe='teacher' if s<=warmup else recipe
+            L,target,_,fraction=batcher.batch(s,active_recipe,original,curriculum_step=step)
+            m.train()
+            for layer in m.encoder.modules():
+                if isinstance(layer,nn.BatchNorm2d):layer.eval()
+            opt.zero_grad(set_to_none=True)
+            factor=min(s/150,1)*(.15+.85*(1+math.cos(math.pi*max(0,s-warmup)/STEPS))/2)
+            for g in opt.param_groups:g['lr']=g['base_lr']*factor
+            with torch.autocast('cuda',dtype=torch.bfloat16):
+                details=m(L,True)
+                loss,winners,terms=objective(details,target,step)
+                consistency=loss.new_zeros(())
+                if active_recipe=='staged' and step>STEPS*2/3 and s%4==0:
+                    small=m(F.interpolate(L,(192,192),mode='bilinear',align_corners=False),True)
+                    a=F.adaptive_avg_pool2d(small['all'].flatten(0,1).float(),(64,64))
+                    b=F.adaptive_avg_pool2d(details['all'].flatten(0,1).detach().float(),(64,64))
+                    consistency=F.l1_loss(a/20,b/20)
+                    if m.modes>1:consistency+=.1*F.kl_div(small['scores'].log_softmax(1),details['scores'].detach().softmax(1),reduction='batchmean')
+                    loss=loss+.4*consistency # every fourth step -> mean weight .1
+            assert torch.isfinite(loss),name
+            loss.backward();torch.nn.utils.clip_grad_norm_(m.parameters(),1,error_if_nonfinite=True);opt.step();last=s
+            mode_counts+=np.bincount(winners.detach().cpu().numpy(),minlength=m.modes)
+            if s%100==0:
+                row={'step':s,'loss':float(loss.detach()),'reconstruction':float(terms.mean().detach()),'consistency':float(consistency.detach()),'photo_fraction_this_batch':fraction,'winner_counts':mode_counts.tolist(),'seconds':time.monotonic()-arm_start}
+                history.append(row);print('STEP',name,s,round(row['loss'],4),flush=True)
+            if s%2000==0 or s==STEPS+warmup:
+                m.eval();res=r.ev.evaluate(m,fresh,dev);val,okay=score(res,base_dev)
+                write(name+f'/development_{s:05d}.json',res)
+                write(name+'/history.json',history)
+                save_decision(m,OUT/name/f'step_{s:05d}')
+                # Ineligible models still retained for diagnosis, ranked after eligible.
+                ranked=val+(0 if okay else 100)
+                if ranked<best-.002:
+                    stale=0;best=ranked;beststep=s;save_decision(m,OUT/name/'candidate');dev_results[name]=res
+                if ranked>=best-.002 and beststep!=s:stale+=1
+                write('status.json',{'status':'training','arm':name,'step':s,'elapsed_seconds':time.monotonic()-START,'production_approved':False})
+                DURABLE.sync(name+' checkpoint '+str(s))
+                if stale>=6:
+                    print('EARLY_STOP development score unchanged for six evaluations',flush=True);break
+        if not last:raise RuntimeError('Budget exhausted before any final training update')
+        write(name+'/history.json',history)
+        torch.save({'optimizer':opt.state_dict(),'step':last,'seed':SEED,'torch_rng':torch.get_rng_state(),'cuda_rng':torch.cuda.get_rng_state_all(),'exact_resume':False},OUT/name/'optimizer.pt')
+        selected[name]={'step':beststep,'rank_score':best,'parameters':counts[head],'steps_completed':last,'warmup_steps':warmup,'seconds':time.monotonic()-arm_start}
+        write('candidates.json',selected);DURABLE.sync(name+' selected checkpoint on development only')
+        del m,opt;torch.cuda.empty_cache()
+        if last<STEPS+warmup:break
+    # Test is opened only after all checkpoint choices have been frozen.
+    write('candidates.json',selected);DURABLE.sync('freeze candidates before opening final test')
+    models={'v3':baseline};test_results={}
+    base_test=r.ev.evaluate(baseline,fresh,test);write('v3_test.json',base_test);test_results['v3']=base_test
+    for name in selected:
+        m=load_decision(OUT/name/'candidate','cuda');models[name]=m
+        res=r.ev.evaluate(m,fresh,test);test_results[name]=res
+        write(name+'_test.json',res);write(name+'_test_paired.json',paired_bootstrap(res,base_test))
+        write(name+'_diagnostics.json',diagnostics(m,fresh,test[:48]))
+        r.ev.probes(m,name+'_probes')
+        x=torch.zeros(1,1,256,256,device='cuda')
+        with torch.inference_mode():
+            for _ in range(5):m(x)
+            torch.cuda.synchronize();t0=time.monotonic()
+            for _ in range(30):m(x)
+            torch.cuda.synchronize()
+        write(name+'_latency.json',{'milliseconds':1000*(time.monotonic()-t0)/30,'hardware':torch.cuda.get_device_name(),'size':256,'precision':'FP32'})
+        DURABLE.sync(name+' final test and diagnostic outputs')
+    r.ev.probes(baseline,'v3_probes')
+    # Smaller comparison groups preserve enough image detail to inspect.
+    for head in ['single','multi']:
+        group={n:m for n,m in models.items() if n=='v3' or n.startswith(head+'_')}
+        r.grid(group,fresh,test,head+'_test')
+    if 'mobilevit_staged' in models:r.grid({'v3':baseline,'single_staged':models['single_staged'],'mobilevit_staged':models['mobilevit_staged']},fresh,test,'encoder_test')
+    # Historical originals deliberately never serve as neutral-color supervision.
+    archive=json.loads(Path('/work/archive_sources.json').read_text())
+    import requests
+    records=[];images=[]
+    for entry in archive:
+        try:
+            response=requests.get(entry['image_url'],timeout=20);response.raise_for_status()
+            im=Image.open(io.BytesIO(response.content)).convert('RGB');im.load()
+            images.append({'bytes':response.content,'path':None});records.append(entry|{'sha256':hashlib.sha256(response.content).hexdigest()})
+        except Exception as e:records.append(entry|{'failed':type(e).__name__})
+    if images:
+        ar=pa.table({'image':images})
+        resolution_grid(baseline,ar,[x for x in records if 'failed' not in x])
+        for head in ['single','multi']:
+            r.grid({n:m for n,m in models.items() if n=='v3' or n.startswith(head+'_')},ar,list(range(len(images))),head+'_archive')
+        if 'mobilevit_staged' in models:r.grid({'v3':baseline,'mobilevit_staged':models['mobilevit_staged']},ar,list(range(len(images))),'encoder_archive')
+        pipeline_grid({'v3':baseline,**{n:m for n,m in models.items() if n!='v3'}},ar,list(range(len(images))),'production_archive')
+    pipeline_grid({'v3':baseline,**{n:m for n,m in models.items() if n!='v3'}},fresh,test[:16],'production_test')
+    write('archive_sources.json',records)
+    write('summary.json',{'parameters':counts,'candidates':selected,'test':{n:summary(v) for n,v in test_results.items()},'all_arms_completed':len(selected)==len(arms),'planned_max_steps':STEPS,'elapsed_seconds':time.monotonic()-START,'estimated_compute_usd':(time.monotonic()-START)/3600*.8,'warning':'No metric proves realism. All outputs need visual review. Candidate can be the retained step0 pilot; selection does not automatically approve production deployment.'})
+    write('status.json',{'status':'completed','all_arms_completed':len(selected)==len(arms),'planned_max_steps':STEPS,'elapsed_seconds':time.monotonic()-START,'production_approved':False})
+    DURABLE.sync('architecture and curriculum suite completed')
+
+if __name__=='__main__':
+    try:main()
+    except BaseException:
+        msg=traceback.format_exc();token=os.environ.get('HF_TOKEN')
+        if token:msg=msg.replace(token,'[REDACTED]')
+        write('failure.json',{'traceback':msg});write('status.json',{'status':'failed','elapsed_seconds':time.monotonic()-START,'production_approved':False})
+        if DURABLE:
+            try:DURABLE.sync('failure report and saved checkpoints')
+            except Exception:pass
+        print(msg,flush=True);raise SystemExit(1)