rezanehzati commited on
Commit
fd4e85f
·
verified ·
1 Parent(s): 8841ba0

Bind R4 bounded ZIP staging, question recovery and exact qualified image

Browse files
EVALUATION.json CHANGED
The diff for this file is too large to render. See raw diff
 
EVALUATION.md CHANGED
@@ -1,12 +1,12 @@
1
  # SEGMENT evaluation evidence
2
 
3
- **R3 local checks completed; official platform qualification remains unproven.** The exact archive is `3f86686527e2695697d7e2958c6b6026abba4dd0ad07918c6ddc3dfc4d39c51d` (35,613,234,136 bytes), immutable generation `1789069382415008`. The build manifest is `5752df0b2ac35a0660fd4c5627c59c383ad33ac0b5823dd56647a999ba4af095` and configuration `fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79`. The same-generation archive passed full compressed/raw25-layer verification and an actual classic Docker import of config `fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79` with all25 ordered DiffIDs; no imported manifest digest was exposed. Five final H100 invocations produced560 answer executions: two canonical10 interface calls with explicitly recorded degraded deferrals, a strict ordinary20, a strict loose500, and a reversed20. The short-case calls matched the frozen saved answers, and reversal matched the same-image ordinary20 by qID. Canonical deferrals are not ten healthy neural-video results. These are runtime/interface/content checks, not new semantic grading, general order invariance, full500 ZIP capacity or an official score. The prior43d4 canonical failure and all failed attempts remain retained.
4
 
5
  The completed development comparison is between two registered serving implementations under the same fixed Qwen3.5-27B judge and 20-case batching protocol: candidate 2373/4000 (59.325%) versus named reference 1940/4000 (48.500%), +10.825 percentage points. Strict sensitivity is 2334/3940 versus 1906/3940. All ten original-source-video differences are positive; 5b causal consequence reasoning regressed 17.742 micro points on 62 cases from six sources. These are adaptively used development results with inherited training/data-exposure limits, not an independent holdout, an official finals score or an all-candidate ranking. The registered local scorer passed14 convenience controls (6positive,8negative). These check response/parser boundaries; they do not estimate organizer-judge agreement, real-question false-positive rates, official calibration or a finals score. All 4,000 were not run through a Docker archive. Source-derived routing equality on the 4,000 intervals 1–299s does not by itself transfer these scores to new image bytes.
6
 
7
- The image owns the literal SEGMENT task; duration no longer chooses the task or rejects otherwise valid request metadata. Both start_time and end_time must remain finite, nonnegative and ordered, and their original values are preserved. Only infer_track changes in the image-owned cap_fmt source; infer_all, format/capability inference and the other 51 serving files are unchanged. The bootstrap retains path, source, model, scratch, output and required initialization guards. It records only exactly attributed pre-generation sampler-duration, frame-survival or observed zero/fewer-than-two-frame deferrals while retaining the original rules answer. Unknown, initialization and generation failures remain fatal. Deferrals mean reduced scientific coverage, not healthy neural inference; an answer count or empty observed-deferral list alone does not prove neural generation for every row.
8
 
9
- No new semantic score, independent holdout, official calibration, platform pass, full 500 ZIP capacity, general order invariance or causal speedup follows from these changes. Exact image, runtime, archive and load evidence must be joined separately. Public shim source does not attest the deployed version; forced process death can bypass scratch cleanup. The earlier canonical and controller failures remain retained.
10
 
11
  Measured grader completions: September 10, 2026, 10:53:44 UTC (candidate) and 10:55:13 UTC (reference). Complete readback and actual GPU-return checks passed for both. Paired reporting completed at 11:07:14 UTC.
12
 
 
1
  # SEGMENT evaluation evidence
2
 
3
+ **R4 local runtime, archive and import checks completed; official platform qualification remains unproven.** The exact archive is `e477d307ff2e09f469d83569ec1ce8703daeef34edbba633a3f240eab955de00` (35,613,293,722 bytes), immutable generation `1789092734891901`. The build manifest is `ba9ddb1e78a45040803ae6f0e68c66de7a538f5e3e86397777e5fc2c488106b0` and configuration `3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4`. Full compressed/raw archive verification covers all 26 ordered DiffIDs; actual same-generation classic Docker import reports config `3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4` and no imported manifest digest. The agreed eight cases completed 9 application invocations and 1593 answer executions across 533 unique qIDs. Healthy20 and loose/ZIP500 matched saved content; reverse20 matched the same-image ordinary20, and both ZIP quotas matched the same-image loose500. Canonical repeated calls retained the exact disclosed deferrals. The mixed recovery case blanked only its declared failed questions and preserved later healthy outputs. These are bounded runtime/interface/content checks; no new semantic grade or official final result is claimed.
4
 
5
  The completed development comparison is between two registered serving implementations under the same fixed Qwen3.5-27B judge and 20-case batching protocol: candidate 2373/4000 (59.325%) versus named reference 1940/4000 (48.500%), +10.825 percentage points. Strict sensitivity is 2334/3940 versus 1906/3940. All ten original-source-video differences are positive; 5b causal consequence reasoning regressed 17.742 micro points on 62 cases from six sources. These are adaptively used development results with inherited training/data-exposure limits, not an independent holdout, an official finals score or an all-candidate ranking. The registered local scorer passed14 convenience controls (6positive,8negative). These check response/parser boundaries; they do not estimate organizer-judge agreement, real-question false-positive rates, official calibration or a finals score. All 4,000 were not run through a Docker archive. Source-derived routing equality on the 4,000 intervals 1–299s does not by itself transfer these scores to new image bytes.
6
 
7
+ R4 retains the image-owned literal SEGMENT task and the original request clocks. It stages only the selected ZIP member for each startup probe or answer, verifies that member through CRC/EOF and SHA-256, closes decoder handles, and removes the lease before continuing. Loose input remains a direct read. Model initialization is still eager. An attributed ordinary per-question exception produces empty content for that qID with measured latency and failure telemetry; successful answers are preserved and later questions continue. Unsafe or ambiguous archives, source mutation, cleanup integrity, required model initialization, environment and fixed-track failures remain fatal. The inherited precise pre-generation rules deferrals remain separately disclosed as degraded coverage. The four changed image files are the bootstrap, contract, probe entrypoint and new ZIP helper; all 51 unchanged inherited serving source files, including cap_fmt, all 50 model/prior assets, prompts, model arithmetic and generation settings remain unchanged.
8
 
9
+ The full retained ZIP500 was qualified locally at the stated 4 GiB and 16 GiB tmpfs profiles, alongside loose500, healthy20, reversed20, default and persistent canonical calls, and controlled recovery. These fixtures do not establish the platform scratch quota, all possible input sizes, continuous resource peaks, cold-cache requirements, general order invariance, new semantic scores, independent-holdout performance or official platform success. Canonical deferrals and deliberately failed recovery rows are reduced scientific coverage. The prior R3 4 GiB eager-ZIP failure is a conditional local capacity reproduction, not proof of the organizer failure cause. Earlier images and failed attempts remain historical.
10
 
11
  Measured grader completions: September 10, 2026, 10:53:44 UTC (candidate) and 10:55:13 UTC (reference). Complete readback and actual GPU-return checks passed for both. Paired reporting completed at 11:07:14 UTC.
12
 
IMAGE_IDENTITY.json CHANGED
@@ -36,35 +36,38 @@
36
  "sha256": "2f4dd5055dc7c33c3e9a43c5a443f2946a6cf999f60aa3e453e8d4ce610ac13b"
37
  },
38
  "current_image": {
39
- "CPU_acceptance_sha256": "c20a1680bf867569101d04c79910ca1f9c04d67caf08d65729e87e8b9800e030",
40
  "actual_platform_result_uuid": null,
41
- "all24_ancestor_layers_preserved": true,
42
- "archive_bytes": 35613234136,
43
- "archive_generation": "1789069382415008",
44
- "archive_receipt_sha256": "2eeeb4daa5f57840ea15c8ab094eaed4b59d56cd666c77da1a562381833337b1",
45
- "archive_sha256": "3f86686527e2695697d7e2958c6b6026abba4dd0ad07918c6ddc3dfc4d39c51d",
46
- "archive_uri": "gs://heydonto-surgfield-research/submissions/segment_forward_fixed_track_cloudbuild_20260910_r5/output/segment-final.tar.gz#1789069382415008",
47
- "base_archive_sha256": "43d4fc4fd7a39e4884a5179176a6b6a436956a146494876783d6287855a82825",
48
- "base_config_sha256": "f7e22f1822d7a152ec91999dbb639ec63378346ef3f5144723c5b453748f97b9",
49
- "base_ordered_layer_count": 24,
50
- "bootstrap_sha256": "4078387df9a46989ae0ab8b25275dc6d43e82afd5459018d3eaa12df1cf0cd76",
51
- "build_receipt_sha256": "0d360628bdf70daf245d6c0ef1c1e7fe7f7db5f6189a4699672daeb6a94c29fa",
52
- "build_store_manifest_sha256": "5752df0b2ac35a0660fd4c5627c59c383ad33ac0b5823dd56647a999ba4af095",
53
- "candidate_source_sha256": "88ae763faa28c7333031876d4c6e6f2f005b19fb9b1362b6d6ffdaf3cd992e37",
 
54
  "cap_fmt_inferrer_sha256": "2861921a562c19937e38346aec219114e64e419bc7dbc7f1eaee4fcaf7f20713",
55
- "contract_sha256": "6af2fb47ced6f8bf0bb9b547c5b6580c1a4663869a6eeceb3e2ab90fe7dcc8b7",
56
- "docker_config_bytes": 28206,
57
- "docker_config_sha256": "fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79",
58
- "docker_loaded_image_id": "sha256:fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79",
59
  "docker_loaded_image_id_kind": "classic_config_digest",
60
- "layer_count": 25,
61
  "normalized_archive_import_manifest_sha256": null,
62
- "overlay_blob_sha256": "64309f5fa0609fe53792678b90973e41f9ca34490c547dbcae741d7ddd785220",
63
- "overlay_diff_id": "91029e508c7a21f763ab8bf3eeca620f1b5ba76cd9a6280331f34c13af596ee8",
64
  "platform_qualified": false,
65
- "runtime_rehearsal_receipt_sha256": "0a8bc8e0cf2d71bebe9085fefd4b2a968595ad69701ea10f592afccc8c35fcfa",
66
- "runtime_rehearsal_status": "passed_final_runtime_interface_and_strict_parity_not_platform",
67
- "status": "passed_local_interface_strict_parity_archive_load_not_platform"
 
 
68
  },
69
  "manifest_identity_note": "Build-store manifest, classic loaded configuration ID and any physically observed imported manifest are separate. Pending fields remain null.",
70
  "preserved_asset_manifest_scope": {
@@ -133,6 +136,37 @@
133
  "runtime_rehearsal_status": "passed_targeted_derivative_runtime_not_platform",
134
  "status": "passed_local_targeted_runtime_and_full_archive_load_not_platform"
135
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
136
  "schema": "segment-associated-container-identity/2",
137
  "shared_serving_implementation": {
138
  "asset_revision": "35fc28af1d55a5a9ae8096ff2e2cd2c961aa7119",
 
36
  "sha256": "2f4dd5055dc7c33c3e9a43c5a443f2946a6cf999f60aa3e453e8d4ce610ac13b"
37
  },
38
  "current_image": {
39
+ "CPU_acceptance_sha256": "64a3e419b9b0ea9d320c0de0d0e3d2359a010dd77119927bdf4060d32094f5c3",
40
  "actual_platform_result_uuid": null,
41
+ "all25_ancestor_layers_preserved": true,
42
+ "archive_bytes": 35613293722,
43
+ "archive_generation": "1789092734891901",
44
+ "archive_receipt_sha256": "a1e4f11381d1be8cc62a8d3d9f8317324c99426c87f5b6be561b9b9224c627a8",
45
+ "archive_sha256": "e477d307ff2e09f469d83569ec1ce8703daeef34edbba633a3f240eab955de00",
46
+ "archive_uri": "gs://heydonto-surgfield-research/submissions/segment_final_recovery_20260911_r1/output/segment-final.tar.gz#1789092734891901",
47
+ "base_archive_sha256": "3f86686527e2695697d7e2958c6b6026abba4dd0ad07918c6ddc3dfc4d39c51d",
48
+ "base_config_sha256": "fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79",
49
+ "base_manifest_sha256": "5752df0b2ac35a0660fd4c5627c59c383ad33ac0b5823dd56647a999ba4af095",
50
+ "base_ordered_layer_count": 25,
51
+ "bootstrap_sha256": "c6846ff32df7e7027a3de520536ee917b0927f1d55915dfcf530fd035d0a35c8",
52
+ "build_receipt_sha256": "9ab1411d614c7142568102381fa3c3db5355e465eec7da708db31d61a31764f8",
53
+ "build_store_manifest_sha256": "ba9ddb1e78a45040803ae6f0e68c66de7a538f5e3e86397777e5fc2c488106b0",
54
+ "candidate_source_sha256": "6bb01e766b15a196ba78779fd8169dff20a1af2ada0a47db1aa85cefe26fc2c1",
55
  "cap_fmt_inferrer_sha256": "2861921a562c19937e38346aec219114e64e419bc7dbc7f1eaee4fcaf7f20713",
56
+ "contract_sha256": "2426fca2beaec028bf506085083f9c0eb681620dd667fb0386ffc41236dcc1de",
57
+ "docker_config_bytes": 28323,
58
+ "docker_config_sha256": "3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4",
59
+ "docker_loaded_image_id": "sha256:3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4",
60
  "docker_loaded_image_id_kind": "classic_config_digest",
61
+ "layer_count": 26,
62
  "normalized_archive_import_manifest_sha256": null,
63
+ "overlay_blob_sha256": "e0f8eb14b59dd28adacb89945febed19517e4e5268db36c96a6cc17fe4a3681c",
64
+ "overlay_diff_id": "1e3951eb1a4e2a1460649a3b4721adada757d5339f84509dc6c33345d9b21392",
65
  "platform_qualified": false,
66
+ "probe_entrypoint_sha256": "40e91b3780417c3503fe14164385aa1d91ab7571265de05962026228f8cf4e33",
67
+ "runtime_rehearsal_receipt_sha256": "8b5daa59751678265917a64d96726115edf01c1b10bc8703bb45ec62df4488f0",
68
+ "runtime_rehearsal_status": "passed_complete_agreed_local_matrix_not_platform",
69
+ "segment_zip_input_sha256": "aef9806482af9b03ce08946c01240183393fbf15c6a2046bad9e00c27a3f58a7",
70
+ "status": "passed_local_complete_runtime_matrix_archive_load_not_platform"
71
  },
72
  "manifest_identity_note": "Build-store manifest, classic loaded configuration ID and any physically observed imported manifest are separate. Pending fields remain null.",
73
  "preserved_asset_manifest_scope": {
 
136
  "runtime_rehearsal_status": "passed_targeted_derivative_runtime_not_platform",
137
  "status": "passed_local_targeted_runtime_and_full_archive_load_not_platform"
138
  },
139
+ "prior_fixed_track_R3_image": {
140
+ "CPU_acceptance_sha256": "c20a1680bf867569101d04c79910ca1f9c04d67caf08d65729e87e8b9800e030",
141
+ "actual_platform_result_uuid": null,
142
+ "all24_ancestor_layers_preserved": true,
143
+ "archive_bytes": 35613234136,
144
+ "archive_generation": "1789069382415008",
145
+ "archive_receipt_sha256": "2eeeb4daa5f57840ea15c8ab094eaed4b59d56cd666c77da1a562381833337b1",
146
+ "archive_sha256": "3f86686527e2695697d7e2958c6b6026abba4dd0ad07918c6ddc3dfc4d39c51d",
147
+ "archive_uri": "gs://heydonto-surgfield-research/submissions/segment_forward_fixed_track_cloudbuild_20260910_r5/output/segment-final.tar.gz#1789069382415008",
148
+ "base_archive_sha256": "43d4fc4fd7a39e4884a5179176a6b6a436956a146494876783d6287855a82825",
149
+ "base_config_sha256": "f7e22f1822d7a152ec91999dbb639ec63378346ef3f5144723c5b453748f97b9",
150
+ "base_ordered_layer_count": 24,
151
+ "bootstrap_sha256": "4078387df9a46989ae0ab8b25275dc6d43e82afd5459018d3eaa12df1cf0cd76",
152
+ "build_receipt_sha256": "0d360628bdf70daf245d6c0ef1c1e7fe7f7db5f6189a4699672daeb6a94c29fa",
153
+ "build_store_manifest_sha256": "5752df0b2ac35a0660fd4c5627c59c383ad33ac0b5823dd56647a999ba4af095",
154
+ "candidate_source_sha256": "88ae763faa28c7333031876d4c6e6f2f005b19fb9b1362b6d6ffdaf3cd992e37",
155
+ "cap_fmt_inferrer_sha256": "2861921a562c19937e38346aec219114e64e419bc7dbc7f1eaee4fcaf7f20713",
156
+ "contract_sha256": "6af2fb47ced6f8bf0bb9b547c5b6580c1a4663869a6eeceb3e2ab90fe7dcc8b7",
157
+ "docker_config_bytes": 28206,
158
+ "docker_config_sha256": "fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79",
159
+ "docker_loaded_image_id": "sha256:fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79",
160
+ "docker_loaded_image_id_kind": "classic_config_digest",
161
+ "layer_count": 25,
162
+ "normalized_archive_import_manifest_sha256": null,
163
+ "overlay_blob_sha256": "64309f5fa0609fe53792678b90973e41f9ca34490c547dbcae741d7ddd785220",
164
+ "overlay_diff_id": "91029e508c7a21f763ab8bf3eeca620f1b5ba76cd9a6280331f34c13af596ee8",
165
+ "platform_qualified": false,
166
+ "runtime_rehearsal_receipt_sha256": "0a8bc8e0cf2d71bebe9085fefd4b2a968595ad69701ea10f592afccc8c35fcfa",
167
+ "runtime_rehearsal_status": "passed_final_runtime_interface_and_strict_parity_not_platform",
168
+ "status": "passed_local_interface_strict_parity_archive_load_not_platform"
169
+ },
170
  "schema": "segment-associated-container-identity/2",
171
  "shared_serving_implementation": {
172
  "asset_revision": "35fc28af1d55a5a9ae8096ff2e2cd2c961aa7119",
LINEAGE.json CHANGED
@@ -49,42 +49,45 @@
49
  },
50
  "attribution": "HeyDonto Labs",
51
  "current_serving_image": {
52
- "CPU_acceptance_sha256": "c20a1680bf867569101d04c79910ca1f9c04d67caf08d65729e87e8b9800e030",
53
  "actual_platform_result_uuid": null,
54
- "all24_ancestor_layers_preserved": true,
55
- "archive_bytes": 35613234136,
56
- "archive_generation": "1789069382415008",
57
- "archive_receipt_sha256": "2eeeb4daa5f57840ea15c8ab094eaed4b59d56cd666c77da1a562381833337b1",
58
- "archive_sha256": "3f86686527e2695697d7e2958c6b6026abba4dd0ad07918c6ddc3dfc4d39c51d",
59
- "archive_uri": "gs://heydonto-surgfield-research/submissions/segment_forward_fixed_track_cloudbuild_20260910_r5/output/segment-final.tar.gz#1789069382415008",
60
- "base_archive_sha256": "43d4fc4fd7a39e4884a5179176a6b6a436956a146494876783d6287855a82825",
61
- "base_config_sha256": "f7e22f1822d7a152ec91999dbb639ec63378346ef3f5144723c5b453748f97b9",
62
- "base_ordered_layer_count": 24,
63
- "bootstrap_sha256": "4078387df9a46989ae0ab8b25275dc6d43e82afd5459018d3eaa12df1cf0cd76",
64
- "build_receipt_sha256": "0d360628bdf70daf245d6c0ef1c1e7fe7f7db5f6189a4699672daeb6a94c29fa",
65
- "build_store_manifest_sha256": "5752df0b2ac35a0660fd4c5627c59c383ad33ac0b5823dd56647a999ba4af095",
66
- "candidate_source_sha256": "88ae763faa28c7333031876d4c6e6f2f005b19fb9b1362b6d6ffdaf3cd992e37",
 
67
  "cap_fmt_inferrer_sha256": "2861921a562c19937e38346aec219114e64e419bc7dbc7f1eaee4fcaf7f20713",
68
- "contract_sha256": "6af2fb47ced6f8bf0bb9b547c5b6580c1a4663869a6eeceb3e2ab90fe7dcc8b7",
69
- "docker_config_bytes": 28206,
70
- "docker_config_sha256": "fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79",
71
- "docker_loaded_image_id": "sha256:fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79",
72
  "docker_loaded_image_id_kind": "classic_config_digest",
73
- "layer_count": 25,
74
  "normalized_archive_import_manifest_sha256": null,
75
- "overlay_blob_sha256": "64309f5fa0609fe53792678b90973e41f9ca34490c547dbcae741d7ddd785220",
76
- "overlay_diff_id": "91029e508c7a21f763ab8bf3eeca620f1b5ba76cd9a6280331f34c13af596ee8",
77
  "platform_qualified": false,
78
- "runtime_rehearsal_receipt_sha256": "0a8bc8e0cf2d71bebe9085fefd4b2a968595ad69701ea10f592afccc8c35fcfa",
79
- "runtime_rehearsal_status": "passed_final_runtime_interface_and_strict_parity_not_platform",
80
- "status": "passed_local_interface_strict_parity_archive_load_not_platform"
 
 
81
  },
82
- "current_serving_image_reference": "IMAGE_IDENTITY.json#/current_image",
83
  "derived_annotation_current_scope_clarification": "The known derived packet contains all 13,177 listed extensions: 4,621 paraphrases, 5,217 count remints, 2,317 convention remints and 1,022 adoption rows. Exclusion of the 33,581 organizer rows is intentional, not evidence that those rows must be republished. The unresolved items are the separate private annotation URL/access/delivery receipts and complete signature provenance.",
84
  "forward_change": "Only tested frame_sampler; same selected models/priors/precision/rendering/generation.",
85
  "forward_gradient_updates": 0,
86
- "full_algorithm_and_image_qualification": "Exact R3 local interface, strict short-case parity and full archive/load accepted. Canonical deferrals remain degraded; no official platform pass or fresh semantic score.",
87
- "image_compatibility_change": "The image owns the literal SEGMENT task; duration no longer chooses the task or rejects otherwise valid request metadata. Both start_time and end_time must remain finite, nonnegative and ordered, and their original values are preserved. Only infer_track changes in the image-owned cap_fmt source; infer_all, format/capability inference and the other 51 serving files are unchanged. The bootstrap retains path, source, model, scratch, output and required initialization guards. It records only exactly attributed pre-generation sampler-duration, frame-survival or observed zero/fewer-than-two-frame deferrals while retaining the original rules answer. Unknown, initialization and generation failures remain fatal. Deferrals mean reduced scientific coverage, not healthy neural inference; an answer count or empty observed-deferral list alone does not prove neural generation for every row.",
88
  "image_identity": {
89
  "added_diff_id": "ee65a0fd5c5057739b208abb4bab056ff0b874e40fd9dfa43c18acf1ddbf5e1e",
90
  "adoption_sha256": "7e46d6b9ff4472e894fbcb2279e79c8ecc25daf251e0be11b4c47f96c6dff4b2",
@@ -279,6 +282,37 @@
279
  "runtime_rehearsal_status": "passed_targeted_derivative_runtime_not_platform",
280
  "status": "passed_local_targeted_runtime_and_full_archive_load_not_platform"
281
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
282
  "responsible_contact": {
283
  "name": "Reza Nehzati, Ph.D.",
284
  "profile": "https://huggingface.co/rezanehzati"
 
49
  },
50
  "attribution": "HeyDonto Labs",
51
  "current_serving_image": {
52
+ "CPU_acceptance_sha256": "64a3e419b9b0ea9d320c0de0d0e3d2359a010dd77119927bdf4060d32094f5c3",
53
  "actual_platform_result_uuid": null,
54
+ "all25_ancestor_layers_preserved": true,
55
+ "archive_bytes": 35613293722,
56
+ "archive_generation": "1789092734891901",
57
+ "archive_receipt_sha256": "a1e4f11381d1be8cc62a8d3d9f8317324c99426c87f5b6be561b9b9224c627a8",
58
+ "archive_sha256": "e477d307ff2e09f469d83569ec1ce8703daeef34edbba633a3f240eab955de00",
59
+ "archive_uri": "gs://heydonto-surgfield-research/submissions/segment_final_recovery_20260911_r1/output/segment-final.tar.gz#1789092734891901",
60
+ "base_archive_sha256": "3f86686527e2695697d7e2958c6b6026abba4dd0ad07918c6ddc3dfc4d39c51d",
61
+ "base_config_sha256": "fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79",
62
+ "base_manifest_sha256": "5752df0b2ac35a0660fd4c5627c59c383ad33ac0b5823dd56647a999ba4af095",
63
+ "base_ordered_layer_count": 25,
64
+ "bootstrap_sha256": "c6846ff32df7e7027a3de520536ee917b0927f1d55915dfcf530fd035d0a35c8",
65
+ "build_receipt_sha256": "9ab1411d614c7142568102381fa3c3db5355e465eec7da708db31d61a31764f8",
66
+ "build_store_manifest_sha256": "ba9ddb1e78a45040803ae6f0e68c66de7a538f5e3e86397777e5fc2c488106b0",
67
+ "candidate_source_sha256": "6bb01e766b15a196ba78779fd8169dff20a1af2ada0a47db1aa85cefe26fc2c1",
68
  "cap_fmt_inferrer_sha256": "2861921a562c19937e38346aec219114e64e419bc7dbc7f1eaee4fcaf7f20713",
69
+ "contract_sha256": "2426fca2beaec028bf506085083f9c0eb681620dd667fb0386ffc41236dcc1de",
70
+ "docker_config_bytes": 28323,
71
+ "docker_config_sha256": "3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4",
72
+ "docker_loaded_image_id": "sha256:3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4",
73
  "docker_loaded_image_id_kind": "classic_config_digest",
74
+ "layer_count": 26,
75
  "normalized_archive_import_manifest_sha256": null,
76
+ "overlay_blob_sha256": "e0f8eb14b59dd28adacb89945febed19517e4e5268db36c96a6cc17fe4a3681c",
77
+ "overlay_diff_id": "1e3951eb1a4e2a1460649a3b4721adada757d5339f84509dc6c33345d9b21392",
78
  "platform_qualified": false,
79
+ "probe_entrypoint_sha256": "40e91b3780417c3503fe14164385aa1d91ab7571265de05962026228f8cf4e33",
80
+ "runtime_rehearsal_receipt_sha256": "8b5daa59751678265917a64d96726115edf01c1b10bc8703bb45ec62df4488f0",
81
+ "runtime_rehearsal_status": "passed_complete_agreed_local_matrix_not_platform",
82
+ "segment_zip_input_sha256": "aef9806482af9b03ce08946c01240183393fbf15c6a2046bad9e00c27a3f58a7",
83
+ "status": "passed_local_complete_runtime_matrix_archive_load_not_platform"
84
  },
85
+ "current_serving_image_reference": "IMAGE_IDENTITY.json current_image (R4); all prior image identities are explicitly historical.",
86
  "derived_annotation_current_scope_clarification": "The known derived packet contains all 13,177 listed extensions: 4,621 paraphrases, 5,217 count remints, 2,317 convention remints and 1,022 adoption rows. Exclusion of the 33,581 organizer rows is intentional, not evidence that those rows must be republished. The unresolved items are the separate private annotation URL/access/delivery receipts and complete signature provenance.",
87
  "forward_change": "Only tested frame_sampler; same selected models/priors/precision/rendering/generation.",
88
  "forward_gradient_updates": 0,
89
+ "full_algorithm_and_image_qualification": "R4 local runtime, archive and import checks completed; official platform qualification remains unproven. The exact archive is `e477d307ff2e09f469d83569ec1ce8703daeef34edbba633a3f240eab955de00` (35,613,293,722 bytes), immutable generation `1789092734891901`. The build manifest is `ba9ddb1e78a45040803ae6f0e68c66de7a538f5e3e86397777e5fc2c488106b0` and configuration `3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4`. Full compressed/raw archive verification covers all 26 ordered DiffIDs; actual same-generation classic Docker import reports config `3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4` and no imported manifest digest. The agreed eight cases completed 9 application invocations and 1593 answer executions across 533 unique qIDs. Healthy20 and loose/ZIP500 matched saved content; reverse20 matched the same-image ordinary20, and both ZIP quotas matched the same-image loose500. Canonical repeated calls retained the exact disclosed deferrals. The mixed recovery case blanked only its declared failed questions and preserved later healthy outputs. These are bounded runtime/interface/content checks; no new semantic grade or official final result is claimed. The full retained ZIP500 was qualified locally at the stated 4 GiB and 16 GiB tmpfs profiles, alongside loose500, healthy20, reversed20, default and persistent canonical calls, and controlled recovery. These fixtures do not establish the platform scratch quota, all possible input sizes, continuous resource peaks, cold-cache requirements, general order invariance, new semantic scores, independent-holdout performance or official platform success. Canonical deferrals and deliberately failed recovery rows are reduced scientific coverage. The prior R3 4 GiB eager-ZIP failure is a conditional local capacity reproduction, not proof of the organizer failure cause. Earlier images and failed attempts remain historical.",
90
+ "image_compatibility_change": "R4 retains the image-owned literal SEGMENT task and the original request clocks. It stages only the selected ZIP member for each startup probe or answer, verifies that member through CRC/EOF and SHA-256, closes decoder handles, and removes the lease before continuing. Loose input remains a direct read. Model initialization is still eager. An attributed ordinary per-question exception produces empty content for that qID with measured latency and failure telemetry; successful answers are preserved and later questions continue. Unsafe or ambiguous archives, source mutation, cleanup integrity, required model initialization, environment and fixed-track failures remain fatal. The inherited precise pre-generation rules deferrals remain separately disclosed as degraded coverage. The four changed image files are the bootstrap, contract, probe entrypoint and new ZIP helper; all 51 unchanged inherited serving source files, including cap_fmt, all 50 model/prior assets, prompts, model arithmetic and generation settings remain unchanged.",
91
  "image_identity": {
92
  "added_diff_id": "ee65a0fd5c5057739b208abb4bab056ff0b874e40fd9dfa43c18acf1ddbf5e1e",
93
  "adoption_sha256": "7e46d6b9ff4472e894fbcb2279e79c8ecc25daf251e0be11b4c47f96c6dff4b2",
 
282
  "runtime_rehearsal_status": "passed_targeted_derivative_runtime_not_platform",
283
  "status": "passed_local_targeted_runtime_and_full_archive_load_not_platform"
284
  },
285
+ "prior_fixed_track_R3_serving_image": {
286
+ "CPU_acceptance_sha256": "c20a1680bf867569101d04c79910ca1f9c04d67caf08d65729e87e8b9800e030",
287
+ "actual_platform_result_uuid": null,
288
+ "all24_ancestor_layers_preserved": true,
289
+ "archive_bytes": 35613234136,
290
+ "archive_generation": "1789069382415008",
291
+ "archive_receipt_sha256": "2eeeb4daa5f57840ea15c8ab094eaed4b59d56cd666c77da1a562381833337b1",
292
+ "archive_sha256": "3f86686527e2695697d7e2958c6b6026abba4dd0ad07918c6ddc3dfc4d39c51d",
293
+ "archive_uri": "gs://heydonto-surgfield-research/submissions/segment_forward_fixed_track_cloudbuild_20260910_r5/output/segment-final.tar.gz#1789069382415008",
294
+ "base_archive_sha256": "43d4fc4fd7a39e4884a5179176a6b6a436956a146494876783d6287855a82825",
295
+ "base_config_sha256": "f7e22f1822d7a152ec91999dbb639ec63378346ef3f5144723c5b453748f97b9",
296
+ "base_ordered_layer_count": 24,
297
+ "bootstrap_sha256": "4078387df9a46989ae0ab8b25275dc6d43e82afd5459018d3eaa12df1cf0cd76",
298
+ "build_receipt_sha256": "0d360628bdf70daf245d6c0ef1c1e7fe7f7db5f6189a4699672daeb6a94c29fa",
299
+ "build_store_manifest_sha256": "5752df0b2ac35a0660fd4c5627c59c383ad33ac0b5823dd56647a999ba4af095",
300
+ "candidate_source_sha256": "88ae763faa28c7333031876d4c6e6f2f005b19fb9b1362b6d6ffdaf3cd992e37",
301
+ "cap_fmt_inferrer_sha256": "2861921a562c19937e38346aec219114e64e419bc7dbc7f1eaee4fcaf7f20713",
302
+ "contract_sha256": "6af2fb47ced6f8bf0bb9b547c5b6580c1a4663869a6eeceb3e2ab90fe7dcc8b7",
303
+ "docker_config_bytes": 28206,
304
+ "docker_config_sha256": "fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79",
305
+ "docker_loaded_image_id": "sha256:fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79",
306
+ "docker_loaded_image_id_kind": "classic_config_digest",
307
+ "layer_count": 25,
308
+ "normalized_archive_import_manifest_sha256": null,
309
+ "overlay_blob_sha256": "64309f5fa0609fe53792678b90973e41f9ca34490c547dbcae741d7ddd785220",
310
+ "overlay_diff_id": "91029e508c7a21f763ab8bf3eeca620f1b5ba76cd9a6280331f34c13af596ee8",
311
+ "platform_qualified": false,
312
+ "runtime_rehearsal_receipt_sha256": "0a8bc8e0cf2d71bebe9085fefd4b2a968595ad69701ea10f592afccc8c35fcfa",
313
+ "runtime_rehearsal_status": "passed_final_runtime_interface_and_strict_parity_not_platform",
314
+ "status": "passed_local_interface_strict_parity_archive_load_not_platform"
315
+ },
316
  "responsible_contact": {
317
  "name": "Reza Nehzati, Ph.D.",
318
  "profile": "https://huggingface.co/rezanehzati"
LINEAGE.md CHANGED
@@ -17,3 +17,12 @@ The six inherited prior files remain byte-identical. `cholec_priors_v38.json` ar
17
  The original forward candidate changed the frame sampler (`cd146570…`). The later R3 compatibility derivative additionally fixes image-owned SEGMENT routing in cap_fmt_inferrer.py and records exactly attributed pre-generation deferrals in its bootstrap; the original timestamps, sampler/model behavior and all other 51 serving files remain unchanged. Public Qwen, private N3, both selected adapters, all six priors, prompts, precision, frame limits and generation settings are unchanged. `TRAINING_RESOURCES.json` preserves selected-save timing/peak scopes and producer identities; `MODEL_IDENTITY.json` preserves every selected file identity.
18
 
19
  The known derived packet contains all 13,177 listed extensions: 4,621 paraphrases, 5,217 count remints, 2,317 convention remints and 1,022 adoption rows. Exclusion of the 33,581 organizer rows is intentional, not evidence that those rows must be republished. The unresolved items are the separate private annotation URL/access/delivery receipts and complete signature provenance.
 
 
 
 
 
 
 
 
 
 
17
  The original forward candidate changed the frame sampler (`cd146570…`). The later R3 compatibility derivative additionally fixes image-owned SEGMENT routing in cap_fmt_inferrer.py and records exactly attributed pre-generation deferrals in its bootstrap; the original timestamps, sampler/model behavior and all other 51 serving files remain unchanged. Public Qwen, private N3, both selected adapters, all six priors, prompts, precision, frame limits and generation settings are unchanged. `TRAINING_RESOURCES.json` preserves selected-save timing/peak scopes and producer identities; `MODEL_IDENTITY.json` preserves every selected file identity.
18
 
19
  The known derived packet contains all 13,177 listed extensions: 4,621 paraphrases, 5,217 count remints, 2,317 convention remints and 1,022 adoption rows. Exclusion of the 33,581 organizer rows is intentional, not evidence that those rows must be republished. The unresolved items are the separate private annotation URL/access/delivery receipts and complete signature provenance.
20
+
21
+
22
+ ## R4 serving compatibility update
23
+
24
+ R4 local runtime, archive and import checks completed; official platform qualification remains unproven. The exact archive is `e477d307ff2e09f469d83569ec1ce8703daeef34edbba633a3f240eab955de00` (35,613,293,722 bytes), immutable generation `1789092734891901`. The build manifest is `ba9ddb1e78a45040803ae6f0e68c66de7a538f5e3e86397777e5fc2c488106b0` and configuration `3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4`. Full compressed/raw archive verification covers all 26 ordered DiffIDs; actual same-generation classic Docker import reports config `3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4` and no imported manifest digest. The agreed eight cases completed 9 application invocations and 1593 answer executions across 533 unique qIDs. Healthy20 and loose/ZIP500 matched saved content; reverse20 matched the same-image ordinary20, and both ZIP quotas matched the same-image loose500. Canonical repeated calls retained the exact disclosed deferrals. The mixed recovery case blanked only its declared failed questions and preserved later healthy outputs. These are bounded runtime/interface/content checks; no new semantic grade or official final result is claimed.
25
+
26
+ R4 retains the image-owned literal SEGMENT task and the original request clocks. It stages only the selected ZIP member for each startup probe or answer, verifies that member through CRC/EOF and SHA-256, closes decoder handles, and removes the lease before continuing. Loose input remains a direct read. Model initialization is still eager. An attributed ordinary per-question exception produces empty content for that qID with measured latency and failure telemetry; successful answers are preserved and later questions continue. Unsafe or ambiguous archives, source mutation, cleanup integrity, required model initialization, environment and fixed-track failures remain fatal. The inherited precise pre-generation rules deferrals remain separately disclosed as degraded coverage. The four changed image files are the bootstrap, contract, probe entrypoint and new ZIP helper; all 51 unchanged inherited serving source files, including cap_fmt, all 50 model/prior assets, prompts, model arithmetic and generation settings remain unchanged.
27
+
28
+ The full retained ZIP500 was qualified locally at the stated 4 GiB and 16 GiB tmpfs profiles, alongside loose500, healthy20, reversed20, default and persistent canonical calls, and controlled recovery. These fixtures do not establish the platform scratch quota, all possible input sizes, continuous resource peaks, cold-cache requirements, general order invariance, new semantic scores, independent-holdout performance or official platform success. Canonical deferrals and deliberately failed recovery rows are reduced scientific coverage. The prior R3 4 GiB eager-ZIP failure is a conditional local capacity reproduction, not proof of the organizer failure cause. Earlier images and failed attempts remain historical.
LOAD.md CHANGED
@@ -1,6 +1,6 @@
1
  # Local loading from two immutable revisions
2
 
3
- These revisions reproduce the unchanged **model components and loading tools**. For full serving, the locally qualified R3 archive has SHA-256 `3f86686527e2695697d7e2958c6b6026abba4dd0ad07918c6ddc3dfc4d39c51d`. The earlier 43d4 archive failed its canonical platform tryout and remains historical. Its exact GCS generation, configuration, compatibility-wrapper identities and completed or pending validation are recorded in [IMAGE_IDENTITY.json](IMAGE_IDENTITY.json) and [PLATFORM_COMPATIBILITY.md](PLATFORM_COMPATIBILITY.md) at the repository revision containing this page. Use those current records to select the serving archive.
4
 
5
  The metadata download below remains pinned to revision M and excludes its archived `README.md`, `LOAD.md` and `IMAGE_IDENTITY.json`. The retained `MODEL_FILES.json` and lineage/binding documents describe component ancestry and may contain the ed85 ancestor's image identity; those bindings do not select the current platform artifact. The 50 model/prior files, immutable A/M revisions, manifests and component loaders remain unchanged. The fixed wrapper is already inside the current Docker archive and is identified by the current records linked above.
6
 
 
1
  # Local loading from two immutable revisions
2
 
3
+ These revisions reproduce the unchanged **model components and loading tools**. For full serving, the locally qualified R4 archive has SHA-256 `e477d307ff2e09f469d83569ec1ce8703daeef34edbba633a3f240eab955de00`. Its exact GCS generation, configuration, four compatibility-source identities and bounded local runtime evidence are recorded in [IMAGE_IDENTITY.json](IMAGE_IDENTITY.json) and [PLATFORM_COMPATIBILITY.md](PLATFORM_COMPATIBILITY.md) at the repository revision containing this page. Prior archives remain historical. Use those current records to select the serving archive.
4
 
5
  The metadata download below remains pinned to revision M and excludes its archived `README.md`, `LOAD.md` and `IMAGE_IDENTITY.json`. The retained `MODEL_FILES.json` and lineage/binding documents describe component ancestry and may contain the ed85 ancestor's image identity; those bindings do not select the current platform artifact. The 50 model/prior files, immutable A/M revisions, manifests and component loaders remain unchanged. The fixed wrapper is already inside the current Docker archive and is identified by the current records linked above.
6
 
PLATFORM_COMPATIBILITY.json CHANGED
The diff for this file is too large to render. See raw diff
 
PLATFORM_COMPATIBILITY.md CHANGED
@@ -1,11 +1,11 @@
1
  # Fixed SEGMENT task and explicit degraded coverage
2
 
3
- **R3 local checks completed; official platform qualification remains unproven.** The exact archive is `3f86686527e2695697d7e2958c6b6026abba4dd0ad07918c6ddc3dfc4d39c51d` (35,613,234,136 bytes), immutable generation `1789069382415008`. The build manifest is `5752df0b2ac35a0660fd4c5627c59c383ad33ac0b5823dd56647a999ba4af095` and configuration `fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79`. The same-generation archive passed full compressed/raw25-layer verification and an actual classic Docker import of config `fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79` with all25 ordered DiffIDs; no imported manifest digest was exposed. Five final H100 invocations produced560 answer executions: two canonical10 interface calls with explicitly recorded degraded deferrals, a strict ordinary20, a strict loose500, and a reversed20. The short-case calls matched the frozen saved answers, and reversal matched the same-image ordinary20 by qID. Canonical deferrals are not ten healthy neural-video results. These are runtime/interface/content checks, not new semantic grading, general order invariance, full500 ZIP capacity or an official score. The prior43d4 canonical failure and all failed attempts remain retained.
4
 
5
- The image owns the literal SEGMENT task; duration no longer chooses the task or rejects otherwise valid request metadata. Both start_time and end_time must remain finite, nonnegative and ordered, and their original values are preserved. Only infer_track changes in the image-owned cap_fmt source; infer_all, format/capability inference and the other 51 serving files are unchanged. The bootstrap retains path, source, model, scratch, output and required initialization guards. It records only exactly attributed pre-generation sampler-duration, frame-survival or observed zero/fewer-than-two-frame deferrals while retaining the original rules answer. Unknown, initialization and generation failures remain fatal. Deferrals mean reduced scientific coverage, not healthy neural inference; an answer count or empty observed-deferral list alone does not prove neural generation for every row.
6
 
7
- No new semantic score, independent holdout, official calibration, platform pass, full 500 ZIP capacity, general order invariance or causal speedup follows from these changes. Exact image, runtime, archive and load evidence must be joined separately. Public shim source does not attest the deployed version; forced process death can bypass scratch cleanup. The earlier canonical and controller failures remain retained.
8
 
9
- The earlier 43d4 qualification covered six H100 invocations and 600 answer executions across 520 unique questions, including repeated20 cases and a loose500 case. It omitted the owner canonical long intervals and invalid-media control. Its151 positive 500 startup probes left 349 unmeasured; its ZIP coverage was 20, not 500. Those measurements remain historical and do not qualify R3. The completed R2 diagnostic prototype returned 10 answers twice with identical contents but failed its guard on two adaptive_n duration-domain deferrals and one no-frame control. That interface diagnosis is not an all-neural-health pass.
10
 
11
- The exact R3 source files are published for inspection under platform_wrapper/ and component_source/cap_fmt_inferrer.py. Models, all 50 model/prior assets, immutable assetA35fc28af and loading metadataMbc9e0924, component loaders, arithmetic, prompts, generation settings and original sampler behavior remain unchanged. Only the top-layer bootstrap, contract and one serving helper are replaced.
 
1
  # Fixed SEGMENT task and explicit degraded coverage
2
 
3
+ **R4 local runtime, archive and import checks completed; official platform qualification remains unproven.** The exact archive is `e477d307ff2e09f469d83569ec1ce8703daeef34edbba633a3f240eab955de00` (35,613,293,722 bytes), immutable generation `1789092734891901`. The build manifest is `ba9ddb1e78a45040803ae6f0e68c66de7a538f5e3e86397777e5fc2c488106b0` and configuration `3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4`. Full compressed/raw archive verification covers all 26 ordered DiffIDs; actual same-generation classic Docker import reports config `3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4` and no imported manifest digest. The agreed eight cases completed 9 application invocations and 1593 answer executions across 533 unique qIDs. Healthy20 and loose/ZIP500 matched saved content; reverse20 matched the same-image ordinary20, and both ZIP quotas matched the same-image loose500. Canonical repeated calls retained the exact disclosed deferrals. The mixed recovery case blanked only its declared failed questions and preserved later healthy outputs. These are bounded runtime/interface/content checks; no new semantic grade or official final result is claimed.
4
 
5
+ R4 retains the image-owned literal SEGMENT task and the original request clocks. It stages only the selected ZIP member for each startup probe or answer, verifies that member through CRC/EOF and SHA-256, closes decoder handles, and removes the lease before continuing. Loose input remains a direct read. Model initialization is still eager. An attributed ordinary per-question exception produces empty content for that qID with measured latency and failure telemetry; successful answers are preserved and later questions continue. Unsafe or ambiguous archives, source mutation, cleanup integrity, required model initialization, environment and fixed-track failures remain fatal. The inherited precise pre-generation rules deferrals remain separately disclosed as degraded coverage. The four changed image files are the bootstrap, contract, probe entrypoint and new ZIP helper; all 51 unchanged inherited serving source files, including cap_fmt, all 50 model/prior assets, prompts, model arithmetic and generation settings remain unchanged.
6
 
7
+ The full retained ZIP500 was qualified locally at the stated 4 GiB and 16 GiB tmpfs profiles, alongside loose500, healthy20, reversed20, default and persistent canonical calls, and controlled recovery. These fixtures do not establish the platform scratch quota, all possible input sizes, continuous resource peaks, cold-cache requirements, general order invariance, new semantic scores, independent-holdout performance or official platform success. Canonical deferrals and deliberately failed recovery rows are reduced scientific coverage. The prior R3 4 GiB eager-ZIP failure is a conditional local capacity reproduction, not proof of the organizer failure cause. Earlier images and failed attempts remain historical.
8
 
9
+ The earlier 43d4 qualification covered six H100 invocations and 600 answer executions across 520 unique questions, including repeated20 cases and a loose500 case. It omitted the owner canonical long intervals and invalid-media control. Its151 positive 500 startup probes left 349 unmeasured; its ZIP coverage was 20, not 500. Those measurements remain historical and do not qualify R4. The completed R2 diagnostic prototype returned 10 answers twice with identical contents but failed its guard on two adaptive_n duration-domain deferrals and one no-frame control. That interface diagnosis is not an all-neural-health pass.
10
 
11
+ The exact four R4 image files are available for inspection under platform_wrapper/. The inherited component_source/cap_fmt_inferrer.py and all component loaders remain unchanged. R4 retains the image-owned literal SEGMENT task and the original request clocks. It stages only the selected ZIP member for each startup probe or answer, verifies that member through CRC/EOF and SHA-256, closes decoder handles, and removes the lease before continuing. Loose input remains a direct read. Model initialization is still eager. An attributed ordinary per-question exception produces empty content for that qID with measured latency and failure telemetry; successful answers are preserved and later questions continue. Unsafe or ambiguous archives, source mutation, cleanup integrity, required model initialization, environment and fixed-track failures remain fatal. The inherited precise pre-generation rules deferrals remain separately disclosed as degraded coverage. The four changed image files are the bootstrap, contract, probe entrypoint and new ZIP helper; all 51 unchanged inherited serving source files, including cap_fmt, all 50 model/prior assets, prompts, model arithmetic and generation settings remain unchanged.
README.md CHANGED
@@ -16,7 +16,7 @@ tags:
16
  ---
17
  # SURGFIELD ORena2026 SEGMENT FINAL
18
 
19
- **R3 local checks completed; official platform qualification remains unproven.** The exact archive is `3f86686527e2695697d7e2958c6b6026abba4dd0ad07918c6ddc3dfc4d39c51d` (35,613,234,136 bytes), immutable generation `1789069382415008`. The build manifest is `5752df0b2ac35a0660fd4c5627c59c383ad33ac0b5823dd56647a999ba4af095` and configuration `fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79`. The same-generation archive passed full compressed/raw25-layer verification and an actual classic Docker import of config `fa8e31b08886a361c45c80425485b08de0fa3f837b4951a0fab636a18952fe79` with all25 ordered DiffIDs; no imported manifest digest was exposed. Five final H100 invocations produced560 answer executions: two canonical10 interface calls with explicitly recorded degraded deferrals, a strict ordinary20, a strict loose500, and a reversed20. The short-case calls matched the frozen saved answers, and reversal matched the same-image ordinary20 by qID. Canonical deferrals are not ten healthy neural-video results. These are runtime/interface/content checks, not new semantic grading, general order invariance, full500 ZIP capacity or an official score. The prior43d4 canonical failure and all failed attempts remain retained.
20
 
21
  Private model and resource handoff by **HeyDonto Labs**. Responsible contact: **Reza Nehzati, Ph.D.**, via [rezanehzati](https://huggingface.co/rezanehzati).
22
 
@@ -24,11 +24,11 @@ This candidate answers segment-level surgical-video questions through fixed rule
24
 
25
  `SERVING_ASSETS.json` lists all **50 required model/prior files**: 22 private assets, 14 pinned public Qwen files, 8 pinned OWLv2 files and 6 priors. `LOAD.md` downloads immutable asset and metadata revisions into separate directories and explains the frozen loading contract. LOAD pins asset revision `35fc28af1d55a5a9ae8096ff2e2cd2c961aa7119` and metadata/tools revision `bc9e092422beac9ae51005694824cdc70205e0d5` separately. The private N3 backbone and public Qwen backbone are distinct; both are required. The inactive OWLv2 `.bin` duplicate and obsolete adapters are excluded.
26
 
27
- The image owns the literal SEGMENT task; duration no longer chooses the task or rejects otherwise valid request metadata. Both start_time and end_time must remain finite, nonnegative and ordered, and their original values are preserved. Only infer_track changes in the image-owned cap_fmt source; infer_all, format/capability inference and the other 51 serving files are unchanged. The bootstrap retains path, source, model, scratch, output and required initialization guards. It records only exactly attributed pre-generation sampler-duration, frame-survival or observed zero/fewer-than-two-frame deferrals while retaining the original rules answer. Unknown, initialization and generation failures remain fatal. Deferrals mean reduced scientific coverage, not healthy neural inference; an answer count or empty observed-deferral list alone does not prove neural generation for every row.
28
 
29
  The completed development comparison is between two registered serving implementations under the same fixed Qwen3.5-27B judge and 20-case batching protocol: candidate 2373/4000 (59.325%) versus named reference 1940/4000 (48.500%), +10.825 percentage points. Strict sensitivity is 2334/3940 versus 1906/3940. All ten original-source-video differences are positive; 5b causal consequence reasoning regressed 17.742 micro points on 62 cases from six sources. These are adaptively used development results with inherited training/data-exposure limits, not an independent holdout, an official finals score or an all-candidate ranking. The registered local scorer passed14 convenience controls (6positive,8negative). These check response/parser boundaries; they do not estimate organizer-judge agreement, real-question false-positive rates, official calibration or a finals score. All 4,000 were not run through a Docker archive. Source-derived routing equality on the 4,000 intervals 1–299s does not by itself transfer these scores to new image bytes.
30
 
31
- No new semantic score, independent holdout, official calibration, platform pass, full 500 ZIP capacity, general order invariance or causal speedup follows from these changes. Exact image, runtime, archive and load evidence must be joined separately. Public shim source does not attest the deployed version; forced process death can bypass scratch cleanup. The earlier canonical and controller failures remain retained.
32
 
33
  Training lineage and selected-prefix exposure are disclosed in `LINEAGE.md` and the accompanying JSONs. The separately retained annotation packet has 13,177 derived rows, not the complete 46,758-row ancestor corpus; it contains no underlying video pixels or complete signature ledger.
34
 
 
16
  ---
17
  # SURGFIELD ORena2026 SEGMENT FINAL
18
 
19
+ **R4 local runtime, archive and import checks completed; official platform qualification remains unproven.** The exact archive is `e477d307ff2e09f469d83569ec1ce8703daeef34edbba633a3f240eab955de00` (35,613,293,722 bytes), immutable generation `1789092734891901`. The build manifest is `ba9ddb1e78a45040803ae6f0e68c66de7a538f5e3e86397777e5fc2c488106b0` and configuration `3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4`. Full compressed/raw archive verification covers all 26 ordered DiffIDs; actual same-generation classic Docker import reports config `3f43fce1081dc5db454447dbe4678947e5cefec014635e611d8d1032a0ac71c4` and no imported manifest digest. The agreed eight cases completed 9 application invocations and 1593 answer executions across 533 unique qIDs. Healthy20 and loose/ZIP500 matched saved content; reverse20 matched the same-image ordinary20, and both ZIP quotas matched the same-image loose500. Canonical repeated calls retained the exact disclosed deferrals. The mixed recovery case blanked only its declared failed questions and preserved later healthy outputs. These are bounded runtime/interface/content checks; no new semantic grade or official final result is claimed.
20
 
21
  Private model and resource handoff by **HeyDonto Labs**. Responsible contact: **Reza Nehzati, Ph.D.**, via [rezanehzati](https://huggingface.co/rezanehzati).
22
 
 
24
 
25
  `SERVING_ASSETS.json` lists all **50 required model/prior files**: 22 private assets, 14 pinned public Qwen files, 8 pinned OWLv2 files and 6 priors. `LOAD.md` downloads immutable asset and metadata revisions into separate directories and explains the frozen loading contract. LOAD pins asset revision `35fc28af1d55a5a9ae8096ff2e2cd2c961aa7119` and metadata/tools revision `bc9e092422beac9ae51005694824cdc70205e0d5` separately. The private N3 backbone and public Qwen backbone are distinct; both are required. The inactive OWLv2 `.bin` duplicate and obsolete adapters are excluded.
26
 
27
+ R4 retains the image-owned literal SEGMENT task and the original request clocks. It stages only the selected ZIP member for each startup probe or answer, verifies that member through CRC/EOF and SHA-256, closes decoder handles, and removes the lease before continuing. Loose input remains a direct read. Model initialization is still eager. An attributed ordinary per-question exception produces empty content for that qID with measured latency and failure telemetry; successful answers are preserved and later questions continue. Unsafe or ambiguous archives, source mutation, cleanup integrity, required model initialization, environment and fixed-track failures remain fatal. The inherited precise pre-generation rules deferrals remain separately disclosed as degraded coverage. The four changed image files are the bootstrap, contract, probe entrypoint and new ZIP helper; all 51 unchanged inherited serving source files, including cap_fmt, all 50 model/prior assets, prompts, model arithmetic and generation settings remain unchanged.
28
 
29
  The completed development comparison is between two registered serving implementations under the same fixed Qwen3.5-27B judge and 20-case batching protocol: candidate 2373/4000 (59.325%) versus named reference 1940/4000 (48.500%), +10.825 percentage points. Strict sensitivity is 2334/3940 versus 1906/3940. All ten original-source-video differences are positive; 5b causal consequence reasoning regressed 17.742 micro points on 62 cases from six sources. These are adaptively used development results with inherited training/data-exposure limits, not an independent holdout, an official finals score or an all-candidate ranking. The registered local scorer passed14 convenience controls (6positive,8negative). These check response/parser boundaries; they do not estimate organizer-judge agreement, real-question false-positive rates, official calibration or a finals score. All 4,000 were not run through a Docker archive. Source-derived routing equality on the 4,000 intervals 1–299s does not by itself transfer these scores to new image bytes.
30
 
31
+ The full retained ZIP500 was qualified locally at the stated 4 GiB and 16 GiB tmpfs profiles, alongside loose500, healthy20, reversed20, default and persistent canonical calls, and controlled recovery. These fixtures do not establish the platform scratch quota, all possible input sizes, continuous resource peaks, cold-cache requirements, general order invariance, new semantic scores, independent-holdout performance or official platform success. Canonical deferrals and deliberately failed recovery rows are reduced scientific coverage. The prior R3 4 GiB eager-ZIP failure is a conditional local capacity reproduction, not proof of the organizer failure cause. Earlier images and failed attempts remain historical.
32
 
33
  Training lineage and selected-prefix exposure are disclosed in `LINEAGE.md` and the accompanying JSONs. The separately retained annotation packet has 13,177 derived rows, not the complete 46,758-row ancestor corpus; it contains no underlying video pixels or complete signature ledger.
34
 
REQUIRED_LINKS.json CHANGED
@@ -1,6 +1,7 @@
1
  {
2
- "current_image_publication_status": "reviewed_local_candidate_pending_private_CAS",
3
  "current_records": {
 
4
  "asset_inventory": "MODEL_FILES.json",
5
  "asset_revision": "35fc28af1d55a5a9ae8096ff2e2cd2c961aa7119",
6
  "current_loading_instructions": "LOAD.md",
@@ -17,7 +18,9 @@
17
  "licenses": "NOTICES.md",
18
  "lineage": "LINEAGE.md",
19
  "loading": "LOAD.md",
20
- "model_card": "README.md"
 
 
21
  },
22
  "historical_availability_verification": {
23
  "asset_rows_matching_immutable_A_custody": 50,
 
1
  {
2
+ "current_image_publication_status": "R4 payload locally qualified; exact replacement private commit/readback must be bound separately.",
3
  "current_records": {
4
+ "R4_source": "platform_wrapper/ (four exact image source files)",
5
  "asset_inventory": "MODEL_FILES.json",
6
  "asset_revision": "35fc28af1d55a5a9ae8096ff2e2cd2c961aa7119",
7
  "current_loading_instructions": "LOAD.md",
 
18
  "licenses": "NOTICES.md",
19
  "lineage": "LINEAGE.md",
20
  "loading": "LOAD.md",
21
+ "model_card": "README.md",
22
+ "r4_probe_source": "platform_wrapper/probe_entrypoint.py",
23
+ "r4_zip_source": "platform_wrapper/segment_zip_input.py"
24
  },
25
  "historical_availability_verification": {
26
  "asset_rows_matching_immutable_A_custody": 50,
platform_wrapper/CONTRACT.json CHANGED
@@ -133,6 +133,14 @@
133
  },
134
  "input_root": "/input",
135
  "output_root": "/output",
 
 
 
 
 
 
 
 
136
  "platform_root_compatibility": {
137
  "linked_input_parent": "/opt/ml/input/data",
138
  "linked_input_suffix": "-input",
@@ -196,7 +204,7 @@
196
  "TRITON_CACHE_DIR": "/tmp/orena-segment/triton",
197
  "XDG_CACHE_HOME": "/tmp/orena-segment/cache"
198
  },
199
- "runtime_validation": "fixed_segment_and_attributed_pre_generation_deferral_derivative_pending_actual_probe_and_runtime_qualification",
200
  "schema_version": "orena-segment-runtime/1",
201
  "source_files": [
202
  {
@@ -260,10 +268,10 @@
260
  "sha256": "a55143dfdbeca52503e225a8a1d2529ef359d3f905ee00ae4d919c6c885c065f"
261
  },
262
  {
263
- "bytes": 52738,
264
- "package_path": "reference/333cd7fe/solution/entrypoint/probe_entrypoint.py",
265
  "path": "/app/solution/entrypoint/probe_entrypoint.py",
266
- "sha256": "82ef94f70fb3ca6ddddd74ff372e7c2eb4a9272425c4031725033f682e7bed13"
267
  },
268
  {
269
  "bytes": 4419,
@@ -289,6 +297,12 @@
289
  "path": "/app/solution/entrypoint/segment_unmerged_main.py",
290
  "sha256": "7a4df6453eba63c01e7638a5ea402e78653a47f83fb7e25d4cf53f030ef06cf6"
291
  },
 
 
 
 
 
 
292
  {
293
  "bytes": 21221,
294
  "package_path": "reference/333cd7fe/solution/entrypoint/test_procedure_byte_identity.py",
 
133
  },
134
  "input_root": "/input",
135
  "output_root": "/output",
136
+ "per_question_failure_policy": {
137
+ "fatal": "initialization, source/environment/fixed-track, unsafe archive/path/identity/cleanup integrity, non-Exception BaseException (e.g. SystemExit/KeyboardInterrupt)",
138
+ "intentional_deferrals": "existing rules behavior retained and disclosed",
139
+ "ordinary_attributed_exceptions": "blank only affected qID and continue; degraded scientific qualification",
140
+ "output": "exact qID census and finite nonnegative latency; empty content only with attributed failure",
141
+ "revision": "R4",
142
+ "zip_staging": "one selected member lease at a time; full member EOF/CRC before model access; no compressed archive copy"
143
+ },
144
  "platform_root_compatibility": {
145
  "linked_input_parent": "/opt/ml/input/data",
146
  "linked_input_suffix": "-input",
 
204
  "TRITON_CACHE_DIR": "/tmp/orena-segment/triton",
205
  "XDG_CACHE_HOME": "/tmp/orena-segment/cache"
206
  },
207
+ "runtime_validation": "Successful fixed-source and model initialization, exact per-question attempt/skip and output census; attributed question failures produce blank answers and degraded scientific qualification; source/environment/task/archive-integrity failures remain fatal.",
208
  "schema_version": "orena-segment-runtime/1",
209
  "source_files": [
210
  {
 
268
  "sha256": "a55143dfdbeca52503e225a8a1d2529ef359d3f905ee00ae4d919c6c885c065f"
269
  },
270
  {
271
+ "bytes": 57832,
272
+ "package_path": "runtime/platform_input_r4/probe_entrypoint.py",
273
  "path": "/app/solution/entrypoint/probe_entrypoint.py",
274
+ "sha256": "40e91b3780417c3503fe14164385aa1d91ab7571265de05962026228f8cf4e33"
275
  },
276
  {
277
  "bytes": 4419,
 
297
  "path": "/app/solution/entrypoint/segment_unmerged_main.py",
298
  "sha256": "7a4df6453eba63c01e7638a5ea402e78653a47f83fb7e25d4cf53f030ef06cf6"
299
  },
300
+ {
301
+ "bytes": 16014,
302
+ "package_path": "runtime/platform_input_r4/segment_zip_input.py",
303
+ "path": "/app/solution/entrypoint/segment_zip_input.py",
304
+ "sha256": "aef9806482af9b03ce08946c01240183393fbf15c6a2046bad9e00c27a3f58a7"
305
+ },
306
  {
307
  "bytes": 21221,
308
  "package_path": "reference/333cd7fe/solution/entrypoint/test_procedure_byte_identity.py",
platform_wrapper/bootstrap.py CHANGED
@@ -25,7 +25,7 @@ import zipfile
25
 
26
  TRACK = "segment"
27
  PLATFORM_ROOT_POLICY = {'schema': 'segment-declared-platform-input-root/1', 'logical_input_root': '/input', 'linked_input_parent': '/opt/ml/input/data', 'linked_input_suffix': '-input', 'production_scratch_policy': 'exclusive_per_invocation_children', 'output_and_tmp_aliases': 'refused'}
28
- CONTRACT_SHA256 = '6af2fb47ced6f8bf0bb9b547c5b6580c1a4663869a6eeceb3e2ab90fe7dcc8b7'
29
  IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".bmp", ".webp"}
30
  VIDEO_EXTENSIONS = {".mp4", ".avi", ".mov", ".mkv", ".webm", ".m4v", ".mpg", ".mpeg"}
31
  SAFE_COMPONENT = re.compile(r"[A-Za-z0-9_][A-Za-z0-9_.-]{0,254}\Z")
@@ -39,7 +39,7 @@ PROTECTED_NAMES = {"PATH", "LD_LIBRARY_PATH", "LD_PRELOAD", "LD_AUDIT", "HOME",
39
  "OWLV2_REVISION", "CUDA_VISIBLE_DEVICES", "NVIDIA_VISIBLE_DEVICES",
40
  "NVIDIA_DRIVER_CAPABILITIES"}
41
  DEGRADED_MARKERS = ("PROBESHELL DEGRADED:", "PROBESHELL: launcher floor stands",
42
- "SHIP-CONFIG ASSERT FAILED", "PROBESHELL: RESOLVE-FAIL",
43
  "TRACK: FRAME RESOLVE-FAIL", "WARN: Owlv2 model load failed:",
44
  "keeping the pre-seeded fallback", "L17 eager-load: failed",
45
  "L17 eager-load raised:")
@@ -308,6 +308,15 @@ def safe_component(value, label):
308
  return value
309
 
310
 
 
 
 
 
 
 
 
 
 
311
  def validate_inputs(root: Path):
312
  data, request_sha = read_json(root / "request.json")
313
  rows = [data] if isinstance(data, dict) else data
@@ -317,7 +326,7 @@ def validate_inputs(root: Path):
317
  for row in rows:
318
  if not isinstance(row, dict):
319
  raise ValueError("request row must be an object")
320
- qid = safe_component(row.get("qID"), "qID")
321
  if qid in seen:
322
  raise ValueError("duplicate qID")
323
  seen.add(qid)
@@ -325,8 +334,8 @@ def validate_inputs(root: Path):
325
  raise ValueError("request track mismatch")
326
  if not isinstance(row.get("question"), str) or not row["question"].strip():
327
  raise ValueError("question must be nonempty text")
328
- if row.get("videoID") is not None:
329
- safe_component(row["videoID"], "videoID")
330
  for key in ("start_time", "end_time"):
331
  if key not in row or type(row[key]) not in (int, float) or not math.isfinite(row[key]) or row[key] < 0:
332
  raise ValueError(f"invalid or missing {key}")
@@ -380,6 +389,7 @@ def validate_inputs(root: Path):
380
  if any(name.suffix.lower() in ((VIDEO_EXTENSIONS if TRACK == "frame" else IMAGE_EXTENSIONS)) for name, _ in staged.values()):
381
  raise ValueError("other-track media present")
382
  selected = []
 
383
  for row in rows:
384
  qid = row["qID"]
385
  hits = [name for name in media if name.stem == qid]
@@ -387,11 +397,14 @@ def validate_inputs(root: Path):
387
  groups.append(hits)
388
  groups.append([p for p in media if p.stem == row.get("videoID")])
389
  candidates = next((group for group in groups if group), [])
390
- if len(candidates) != 1:
391
- raise ValueError(f"missing or ambiguous media for qID: {qid}")
392
- selected.append(str(candidates[0]))
 
 
 
393
  return {"requests": len(rows), "input_files": len(files), "archives": archive_count,
394
- "media_files": len(media), "resolved_requests": len(selected), "request_sha256": request_sha,
395
  "validation": "names, layout and supplied interval metadata only; no media decoding"}
396
 
397
 
@@ -399,7 +412,7 @@ def protected(name):
399
  return name.startswith(PROTECTED_PREFIXES) or name in PROTECTED_NAMES
400
 
401
 
402
- def validate_outputs(output_root: Path, input_root: Path):
403
  regular_path(output_root, directory=True)
404
  if sorted(p.name for p in output_root.iterdir()) != ["answer.json"]:
405
  raise ValueError("official output must contain only answer.json")
@@ -413,12 +426,14 @@ def validate_outputs(output_root: Path, input_root: Path):
413
  for answer in answers:
414
  if not isinstance(answer, dict) or set(answer) != {"qID", "content", "latency"}:
415
  raise ValueError("answer schema requires exactly qID/content/latency")
416
- qid = safe_component(answer["qID"], "answer qID")
417
  if qid in seen:
418
  raise ValueError("duplicate answer qID")
419
  seen.add(qid)
420
- if not isinstance(answer["content"], str) or not answer["content"].strip():
421
- raise ValueError("answer content must be nonempty text")
 
 
422
  latency = answer["latency"]
423
  if type(latency) not in (int, float) or not math.isfinite(latency) or latency < 0:
424
  raise ValueError("answer latency must be a finite nonnegative number")
@@ -602,6 +617,12 @@ class EngineObservations:
602
  self.temporal = temporal
603
  self.temporal_handler = None
604
  self.input_deferrals = []
 
 
 
 
 
 
605
  self.allowed_deferrals_by_layer = {}
606
  owner = self
607
  active = {"layer": None, "qid": None, "sample": None, "sample_return": None}
@@ -610,6 +631,7 @@ class EngineObservations:
610
  original = getattr(cls, name)
611
  self.originals.append((cls, name, original))
612
  def observed(instance, *args, **kwargs):
 
613
  try:
614
  if name == "answer":
615
  # The original caller supplies infer_all's scratch dict.
@@ -620,6 +642,13 @@ class EngineObservations:
620
  if ("scratch_init" in kwargs and len(args) > 1) or not isinstance(scratch, dict) or scratch.get("track") != TRACK:
621
  owner.events.append("answer scratch_init track missing, ambiguous or foreign")
622
  raise ValueError("answer scratch_init must carry the fixed track")
 
 
 
 
 
 
 
623
  owner.counts["answers_track_checked"] += 1
624
  result = original(instance, *args, **kwargs)
625
  if result_check is not None and not result_check(instance, result):
@@ -628,10 +657,18 @@ class EngineObservations:
628
  owner.counts[counter] += 1
629
  if name == "__init__":
630
  owner.containers.append(instance)
 
631
  return result
632
  except BaseException as error:
633
- owner.events.append(f"{name} raised {type(error).__name__}")
 
 
 
 
634
  raise
 
 
 
635
  setattr(cls, name, observed)
636
 
637
  loaded = lambda instance, result: getattr(instance, "_model", None) is not None and getattr(instance, "_processor", None) is not None
@@ -704,6 +741,18 @@ class EngineObservations:
704
  key = id(instance)
705
  owner.allowed_deferrals_by_layer[key] = owner.allowed_deferrals_by_layer.get(key, 0) + 1
706
  active["sample"] = None
 
 
 
 
 
 
 
 
 
 
 
 
707
  return original_failure(instance, message, error)
708
 
709
  layer_class.predict = observed_predict
@@ -796,27 +845,86 @@ class EngineObservations:
796
  "actual_grid_t": local.get("grid_t"),
797
  "expected_grid_t": local["expected_t"]})
798
  return
 
 
 
 
 
799
  if any(marker in message for marker in ("inference FAILED", "R1/build defer", "frames (<2)", "model load FAILED")):
800
  owner.events.append("temporal seam reported failure: " + message[:500])
801
  self.temporal_handler = TemporalHandler()
802
  temporal.log.addHandler(self.temporal_handler)
803
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
804
  def finish(self, requests):
805
- if self.counts["container_initialized"] != 1 or self.counts["answers_completed"] != requests or self.counts["answers_track_checked"] != requests:
 
806
  self.events.append("engine initialization/answer census mismatch")
807
  if self.counts["detector_load_completed"] < 1 or self.counts["vlm_eager_load_completed"] < 1:
808
  self.events.append("required model initialization was not completed")
809
  if self.temporal is not None and (self.counts["temporal_load_completed"] < 1 or self.temporal._LOAD_FAILED):
810
  self.events.append("required temporal initialization was not completed")
811
  for container in self.containers:
812
- if any(getattr(container, "exceptions", {}).values()):
813
- self.events.append("container recorded a layer exception")
 
814
  for layer in getattr(container, "layers", []):
815
  allowed = self.allowed_deferrals_by_layer.get(id(layer), 0)
816
  if getattr(layer, "_load_failed", False) or getattr(layer, "_n_infer_fail", 0) != allowed:
817
  self.events.append("layer recorded model load/inference failure")
818
  return {"counts": dict(self.counts), "failures": list(dict.fromkeys(self.events)),
819
  "observed_input_deferrals": list(self.input_deferrals),
 
 
 
 
820
  "input_deferral_summary": {
821
  "events": len(self.input_deferrals),
822
  "affected_qIDs": sorted({item["qID"] for item in self.input_deferrals}),
@@ -861,6 +969,131 @@ def verify_shell_counters(counters, requests):
861
  "scope": "original structured shell counters; no correctness or timing conclusion"}
862
 
863
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
864
  def execute(contract, *, input_root: Path, output_root: Path, evidence_root: Path | None = None, backend=None, setup_imports=False, production_invocation=False):
865
  """Synthetic-fixture seam; production supplies only baked paths and launcher."""
866
  receipt = {"track": TRACK, "status": "failed", "source_validation": None,
@@ -962,14 +1195,32 @@ def execute(contract, *, input_root: Path, output_root: Path, evidence_root: Pat
962
  guard.violations.append({"key": "environment API", "operation": "guard replaced"})
963
  guard.final_check()
964
  receipt["configuration_violations"] = list(guard.violations)
 
965
  if observations is not None:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
966
  receipt["engine_observations"] = observations.finish(receipt["input_validation"]["requests"])
967
  if receipt["engine_observations"]["failures"]:
968
  raise ValueError("original engine initialization or answer failure")
969
  verify_sources(contract)
970
  if validate_inputs(input_root) != receipt["input_validation"]:
971
  raise ValueError("input metadata or layout changed during backend execution")
972
- receipt["output_validation"] = validate_outputs(output_root, input_root)
 
973
  check_input_binding(binding, policy)
974
  if guard.violations or receipt["degraded_markers"]:
975
  raise ValueError("backend degraded or attempted fixed configuration mutation")
 
25
 
26
  TRACK = "segment"
27
  PLATFORM_ROOT_POLICY = {'schema': 'segment-declared-platform-input-root/1', 'logical_input_root': '/input', 'linked_input_parent': '/opt/ml/input/data', 'linked_input_suffix': '-input', 'production_scratch_policy': 'exclusive_per_invocation_children', 'output_and_tmp_aliases': 'refused'}
28
+ CONTRACT_SHA256 = '2426fca2beaec028bf506085083f9c0eb681620dd667fb0386ffc41236dcc1de'
29
  IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".bmp", ".webp"}
30
  VIDEO_EXTENSIONS = {".mp4", ".avi", ".mov", ".mkv", ".webm", ".m4v", ".mpg", ".mpeg"}
31
  SAFE_COMPONENT = re.compile(r"[A-Za-z0-9_][A-Za-z0-9_.-]{0,254}\Z")
 
39
  "OWLV2_REVISION", "CUDA_VISIBLE_DEVICES", "NVIDIA_VISIBLE_DEVICES",
40
  "NVIDIA_DRIVER_CAPABILITIES"}
41
  DEGRADED_MARKERS = ("PROBESHELL DEGRADED:", "PROBESHELL: launcher floor stands",
42
+ "SHIP-CONFIG ASSERT FAILED",
43
  "TRACK: FRAME RESOLVE-FAIL", "WARN: Owlv2 model load failed:",
44
  "keeping the pre-seeded fallback", "L17 eager-load: failed",
45
  "L17 eager-load raised:")
 
308
  return value
309
 
310
 
311
+ def literal_qid(value, label):
312
+ import unicodedata
313
+ if (not isinstance(value, str) or not value or value in (".", "..")
314
+ or "/" in value or "\\" in value
315
+ or any(unicodedata.category(c) in ("Cc", "Cs") for c in value)):
316
+ raise ValueError(f"unsafe {label}")
317
+ return value
318
+
319
+
320
  def validate_inputs(root: Path):
321
  data, request_sha = read_json(root / "request.json")
322
  rows = [data] if isinstance(data, dict) else data
 
326
  for row in rows:
327
  if not isinstance(row, dict):
328
  raise ValueError("request row must be an object")
329
+ qid = literal_qid(row.get("qID"), "qID")
330
  if qid in seen:
331
  raise ValueError("duplicate qID")
332
  seen.add(qid)
 
334
  raise ValueError("request track mismatch")
335
  if not isinstance(row.get("question"), str) or not row["question"].strip():
336
  raise ValueError("question must be nonempty text")
337
+ if row.get("videoID") is not None and not isinstance(row["videoID"], str):
338
+ raise ValueError("videoID must be metadata text")
339
  for key in ("start_time", "end_time"):
340
  if key not in row or type(row[key]) not in (int, float) or not math.isfinite(row[key]) or row[key] < 0:
341
  raise ValueError(f"invalid or missing {key}")
 
389
  if any(name.suffix.lower() in ((VIDEO_EXTENSIONS if TRACK == "frame" else IMAGE_EXTENSIONS)) for name, _ in staged.values()):
390
  raise ValueError("other-track media present")
391
  selected = []
392
+ unresolved = []
393
  for row in rows:
394
  qid = row["qID"]
395
  hits = [name for name in media if name.stem == qid]
 
397
  groups.append(hits)
398
  groups.append([p for p in media if p.stem == row.get("videoID")])
399
  candidates = next((group for group in groups if group), [])
400
+ if len(candidates) > 1:
401
+ raise ValueError(f"ambiguous media for qID: {qid}")
402
+ if not candidates:
403
+ unresolved.append(qid)
404
+ else:
405
+ selected.append(str(candidates[0]))
406
  return {"requests": len(rows), "input_files": len(files), "archives": archive_count,
407
+ "media_files": len(media), "resolved_requests": len(selected), "unresolved_qIDs": unresolved, "request_sha256": request_sha,
408
  "validation": "names, layout and supplied interval metadata only; no media decoding"}
409
 
410
 
 
412
  return name.startswith(PROTECTED_PREFIXES) or name in PROTECTED_NAMES
413
 
414
 
415
+ def validate_outputs(output_root: Path, input_root: Path, *, empty_qids=frozenset()):
416
  regular_path(output_root, directory=True)
417
  if sorted(p.name for p in output_root.iterdir()) != ["answer.json"]:
418
  raise ValueError("official output must contain only answer.json")
 
426
  for answer in answers:
427
  if not isinstance(answer, dict) or set(answer) != {"qID", "content", "latency"}:
428
  raise ValueError("answer schema requires exactly qID/content/latency")
429
+ qid = literal_qid(answer["qID"], "answer qID")
430
  if qid in seen:
431
  raise ValueError("duplicate answer qID")
432
  seen.add(qid)
433
+ if not isinstance(answer["content"], str):
434
+ raise ValueError("answer content must be text")
435
+ if not answer["content"].strip() and qid not in empty_qids:
436
+ raise ValueError("empty answer lacks an attributed per-question failure")
437
  latency = answer["latency"]
438
  if type(latency) not in (int, float) or not math.isfinite(latency) or latency < 0:
439
  raise ValueError("answer latency must be a finite nonnegative number")
 
617
  self.temporal = temporal
618
  self.temporal_handler = None
619
  self.input_deferrals = []
620
+ self.question_failures = []
621
+ self.answer_qids = []
622
+ self.answer_calls_failed = 0
623
+ self.current_answer_qid = None
624
+ self.allowed_layer_exceptions = {}
625
+ self.skipped_qids = set()
626
  self.allowed_deferrals_by_layer = {}
627
  owner = self
628
  active = {"layer": None, "qid": None, "sample": None, "sample_return": None}
 
631
  original = getattr(cls, name)
632
  self.originals.append((cls, name, original))
633
  def observed(instance, *args, **kwargs):
634
+ admitted_answer = False
635
  try:
636
  if name == "answer":
637
  # The original caller supplies infer_all's scratch dict.
 
642
  if ("scratch_init" in kwargs and len(args) > 1) or not isinstance(scratch, dict) or scratch.get("track") != TRACK:
643
  owner.events.append("answer scratch_init track missing, ambiguous or foreign")
644
  raise ValueError("answer scratch_init must carry the fixed track")
645
+ request = args[0] if args else kwargs.get("request")
646
+ qid = request.get("qID") if isinstance(request, dict) else getattr(request, "qID", None)
647
+ if not isinstance(qid, str) or qid in owner.answer_qids:
648
+ raise ValueError("missing or repeated original answer qID")
649
+ owner.answer_qids.append(qid)
650
+ owner.current_answer_qid = qid
651
+ admitted_answer = True
652
  owner.counts["answers_track_checked"] += 1
653
  result = original(instance, *args, **kwargs)
654
  if result_check is not None and not result_check(instance, result):
 
657
  owner.counts[counter] += 1
658
  if name == "__init__":
659
  owner.containers.append(instance)
660
+ owner.observe_layer_exceptions(instance)
661
  return result
662
  except BaseException as error:
663
+ if name == "answer" and admitted_answer and isinstance(error, Exception) and not integrity_exception(error):
664
+ owner.answer_calls_failed += 1
665
+ owner.record_question_failure(exception_event(error, owner.current_answer_qid, "container_answer_exception"))
666
+ else:
667
+ owner.events.append(f"{name} raised {type(error).__name__}")
668
  raise
669
+ finally:
670
+ if admitted_answer:
671
+ owner.current_answer_qid = None
672
  setattr(cls, name, observed)
673
 
674
  loaded = lambda instance, result: getattr(instance, "_model", None) is not None and getattr(instance, "_processor", None) is not None
 
741
  key = id(instance)
742
  owner.allowed_deferrals_by_layer[key] = owner.allowed_deferrals_by_layer.get(key, 0) + 1
743
  active["sample"] = None
744
+ elif instance is active["layer"] and active["qid"] is not None:
745
+ event = None
746
+ if message == expected_message and error is None:
747
+ event = {"qID": active["qid"], "stage": "main_pre_generation_media_resolution",
748
+ "reason": "Original main path could not resolve a usable clip before sampling",
749
+ "producer": {"path": "/app/solution/layers/vlm_cholec.py", "function": "predict", "exception": None}}
750
+ elif error is not None:
751
+ event = classified_question_failure(error, active["qid"], "main")
752
+ if event is not None:
753
+ owner.record_question_failure(event)
754
+ key = id(instance)
755
+ owner.allowed_deferrals_by_layer[key] = owner.allowed_deferrals_by_layer.get(key, 0) + 1
756
  return original_failure(instance, message, error)
757
 
758
  layer_class.predict = observed_predict
 
845
  "actual_grid_t": local.get("grid_t"),
846
  "expected_grid_t": local["expected_t"]})
847
  return
848
+ if exact_producer and ((record.lineno == 175 and record.msg == "segment_temporal_seam: %s R1/build defer (%s)") or (record.lineno == 179 and record.msg == "segment_temporal_seam: %s inference FAILED (%r); defer to rules")) and isinstance(args[1], Exception):
849
+ event = classified_question_failure(args[1], active["qid"], "temporal")
850
+ if event is not None:
851
+ owner.record_question_failure(event)
852
+ return
853
  if any(marker in message for marker in ("inference FAILED", "R1/build defer", "frames (<2)", "model load FAILED")):
854
  owner.events.append("temporal seam reported failure: " + message[:500])
855
  self.temporal_handler = TemporalHandler()
856
  temporal.log.addHandler(self.temporal_handler)
857
 
858
+ def observe_layer_exceptions(self, container):
859
+ for layer in getattr(container, "layers", []):
860
+ for method_name in ("applies", "predict"):
861
+ original = getattr(layer, method_name, None)
862
+ if original is None or not hasattr(layer, "name"):
863
+ continue
864
+ self.originals.append((layer, method_name, original))
865
+ def observed(*args, _original=original, _layer=layer, _method=method_name, **kwargs):
866
+ try:
867
+ return _original(*args, **kwargs)
868
+ except Exception as error:
869
+ if self.current_answer_qid is not None and not integrity_exception(error):
870
+ key = (id(container), _layer.name)
871
+ self.allowed_layer_exceptions[key] = self.allowed_layer_exceptions.get(key, 0) + 1
872
+ self.record_question_failure(exception_event(error, self.current_answer_qid, "layer_" + _method + "_exception", layer=_layer.name))
873
+ else:
874
+ self.events.append("unattributed or integrity layer exception")
875
+ raise
876
+ setattr(layer, method_name, observed)
877
+
878
+ def record_question_failure(self, event):
879
+ self.question_failures.append({**event, "output_policy": "empty_content", "scientific_coverage": "failed question; no correctness or healthy-neural-coverage claim"})
880
+
881
+ def reconcile_skipped(self, expected_qids, events, question_events=(), fence_events=()):
882
+ expected = set(expected_qids)
883
+ seen = set()
884
+ for event in events:
885
+ validate_staging_event(event, expected)
886
+ seen.add(event["qID"])
887
+ self.record_question_failure({"qID": event["qID"], "stage": event["stage"], "reason": event["message"], "producer": dict(event)})
888
+ question_ids = set()
889
+ for event in question_events:
890
+ validate_question_event(event, expected)
891
+ if event["qID"] in question_ids:
892
+ raise ValueError("repeated answer-loop failure event")
893
+ question_ids.add(event["qID"])
894
+ self.record_question_failure({"qID": event["qID"], "stage": event["stage"], "reason": event["message"], "producer": dict(event)})
895
+ for event in fence_events:
896
+ if not isinstance(event, dict) or set(event) != {"qID", "reason", "stage"} or event["qID"] not in expected or event["reason"] not in ("WEDGE", "CRASH") or event["stage"] != "startup_decoder_fence":
897
+ raise ValueError("unrecognized original decoder fence")
898
+ self.record_question_failure({"qID": event["qID"], "stage": event["stage"], "reason": event["reason"], "producer": dict(event)})
899
+ if set(self.answer_qids) & seen:
900
+ raise ValueError("staging-skipped question unexpectedly reached original answer")
901
+ seen |= question_ids - set(self.answer_qids)
902
+ if set(self.answer_qids) | seen != expected:
903
+ raise ValueError("original answer plus attributed staging-skip qID census mismatch")
904
+ self.skipped_qids = seen
905
+
906
  def finish(self, requests):
907
+ completed_expected = requests - len(self.skipped_qids)
908
+ if self.counts["container_initialized"] != 1 or self.counts["answers_completed"] + self.answer_calls_failed != completed_expected or self.counts["answers_track_checked"] != completed_expected:
909
  self.events.append("engine initialization/answer census mismatch")
910
  if self.counts["detector_load_completed"] < 1 or self.counts["vlm_eager_load_completed"] < 1:
911
  self.events.append("required model initialization was not completed")
912
  if self.temporal is not None and (self.counts["temporal_load_completed"] < 1 or self.temporal._LOAD_FAILED):
913
  self.events.append("required temporal initialization was not completed")
914
  for container in self.containers:
915
+ for name, count in getattr(container, "exceptions", {}).items():
916
+ if count != self.allowed_layer_exceptions.get((id(container), name), 0):
917
+ self.events.append("container recorded an unattributed layer exception")
918
  for layer in getattr(container, "layers", []):
919
  allowed = self.allowed_deferrals_by_layer.get(id(layer), 0)
920
  if getattr(layer, "_load_failed", False) or getattr(layer, "_n_infer_fail", 0) != allowed:
921
  self.events.append("layer recorded model load/inference failure")
922
  return {"counts": dict(self.counts), "failures": list(dict.fromkeys(self.events)),
923
  "observed_input_deferrals": list(self.input_deferrals),
924
+ "observed_question_failures": list(self.question_failures),
925
+ "answer_qIDs": list(self.answer_qids), "staging_skipped_qIDs": sorted(self.skipped_qids),
926
+ "answer_calls_failed": self.answer_calls_failed,
927
+ "strict_model_and_guard_health_passed": not self.events and not self.input_deferrals and not self.question_failures,
928
  "input_deferral_summary": {
929
  "events": len(self.input_deferrals),
930
  "affected_qIDs": sorted({item["qID"] for item in self.input_deferrals}),
 
969
  "scope": "original structured shell counters; no correctness or timing conclusion"}
970
 
971
 
972
+ def integrity_exception(error):
973
+ module = sys.modules.get("solution.entrypoint.segment_zip_input")
974
+ cls = getattr(module, "ZipIntegrityError", None)
975
+ return isinstance(cls, type) and isinstance(error, cls)
976
+
977
+
978
+ def exception_event(error, qid, stage, **metadata):
979
+ trace, frames = error.__traceback__, []
980
+ while trace is not None:
981
+ frames.append({"path": trace.tb_frame.f_code.co_filename, "function": trace.tb_frame.f_code.co_name, "line": trace.tb_lineno})
982
+ trace = trace.tb_next
983
+ return {"qID": qid, "stage": stage, "reason": str(error), "producer": {"exception_type": type(error).__name__, "exception_module": type(error).__module__, "traceback": frames, **metadata}}
984
+
985
+
986
+ def classified_question_failure(error, qid, route):
987
+ """Attribute ordinary per-question exceptions; preserve exact known stages and label unknown stages honestly."""
988
+ trace, frames = error.__traceback__, []
989
+ while trace is not None:
990
+ frames.append({"path": trace.tb_frame.f_code.co_filename,
991
+ "function": trace.tb_frame.f_code.co_name, "line": trace.tb_lineno})
992
+ trace = trace.tb_next
993
+ if not frames or not isinstance(error, Exception) or integrity_exception(error):
994
+ return None
995
+ leaf = frames[-1]
996
+ rendering = "/app/solution/entrypoint/segment_final_rendering.py"
997
+ temporal = "/app/solution/entrypoint/segment_temporal_seam.py"
998
+ clock = "/app/solution/entrypoint/segment_main_source_clock.py"
999
+ stage = None
1000
+ if route == "temporal" and type(error) is ValueError:
1001
+ if leaf["path"] == temporal and leaf["function"] == "segment_temporal_predict" and leaf["line"] in (151, 155):
1002
+ stage = "temporal_pre_generation_frame_census" if leaf["line"] == 151 else "temporal_pre_generation_source_clock"
1003
+ elif leaf["path"] == rendering and leaf["function"] in ("emit", "seconds", "hms") and any(x["path"] == rendering and x["function"] == "emit" for x in frames):
1004
+ stage = "temporal_post_generation_invalid_format"
1005
+ if route == "main":
1006
+ if type(error) is ValueError and leaf["path"] == clock and leaf["function"] in ("sample", "validate_metadata", "content"):
1007
+ stage = "main_pre_generation_source_clock"
1008
+ cv = sys.modules.get("cv2")
1009
+ cv_error = getattr(cv, "error", None)
1010
+ decode_error = isinstance(error, OSError) or (isinstance(cv_error, type) and isinstance(error, cv_error))
1011
+ if decode_error and any(x["path"] == "/app/solution/frame_sampler.py" for x in frames):
1012
+ stage = "main_pre_generation_decoder_exception"
1013
+ if stage is None:
1014
+ stage = route + "_inference_exception_unclassified_stage"
1015
+ return {"qID": qid, "stage": stage, "reason": str(error),
1016
+ "producer": {"exception_type": type(error).__name__, "traceback": frames,
1017
+ "model_generation_reached": True if stage == "temporal_post_generation_invalid_format" else None}}
1018
+
1019
+
1020
+ def validate_staging_event(event, expected_qids):
1021
+ fields = {"qID", "phase", "stage", "archive", "member", "declared_bytes", "written_bytes",
1022
+ "exception_type", "errno", "message", "output_policy"}
1023
+ if not isinstance(event, dict) or set(event) != fields:
1024
+ raise ValueError("unexpected trusted staging event schema")
1025
+ if event["qID"] not in expected_qids or event["phase"] not in ("startup_probe", "answer") or event["output_policy"] != "empty_content":
1026
+ raise ValueError("staging event qID/phase/output policy differs")
1027
+ if event["stage"] not in ("zip_member_staging", "input_media_resolution"):
1028
+ raise ValueError("unknown staging failure class")
1029
+ if not isinstance(event["message"], str) or not isinstance(event["exception_type"], str):
1030
+ raise ValueError("invalid staging exception evidence")
1031
+ if type(event["written_bytes"]) is not int or event["written_bytes"] < 0 or (event["errno"] is not None and type(event["errno"]) is not int):
1032
+ raise ValueError("invalid staging byte/errno evidence")
1033
+ if event["stage"] == "input_media_resolution":
1034
+ if (event["archive"], event["member"], event["declared_bytes"], event["written_bytes"], event["exception_type"]) != (None, None, None, 0, "MissingMedia"):
1035
+ raise ValueError("missing-media event identity differs")
1036
+ elif (not isinstance(event["archive"], str) or not isinstance(event["member"], str)
1037
+ or type(event["declared_bytes"]) is not int or not 0 <= event["written_bytes"] <= event["declared_bytes"]):
1038
+ raise ValueError("ZIP failure member/byte evidence differs")
1039
+
1040
+
1041
+ def blank_failed_outputs(output_root, input_root, events):
1042
+ """Only after the original loop: preserve healthy strings and all latencies."""
1043
+ failed = {event["qID"] for event in events}
1044
+ before = validate_outputs(output_root, input_root, empty_qids=failed)
1045
+ answers, original_sha = read_json(output_root / "answer.json")
1046
+ qids = {answer["qID"] for answer in answers}
1047
+ if not failed <= qids:
1048
+ raise ValueError("question-failure ledger contains an unknown qID")
1049
+ if not failed:
1050
+ return {"blanked_qIDs": [], "answer_sha256_before": original_sha, "answer_sha256_after": original_sha,
1051
+ "successful_answer_contents_unchanged": True, "latencies_unchanged": True}
1052
+ kept = {answer["qID"]: dict(answer) for answer in answers}
1053
+ for answer in answers:
1054
+ if answer["qID"] in failed:
1055
+ answer["content"] = ""
1056
+ target = output_root / "answer.json"
1057
+ temp = output_root / ".r4-answer.json.tmp"
1058
+ try:
1059
+ fd = os.open(temp, os.O_CREAT | os.O_EXCL | os.O_WRONLY | os.O_NOFOLLOW, 0o600)
1060
+ with os.fdopen(fd, "w", encoding="utf-8") as stream:
1061
+ json.dump(answers, stream, indent=1, allow_nan=False)
1062
+ stream.write("\n")
1063
+ _, current_sha = read_json(target)
1064
+ if current_sha != original_sha:
1065
+ raise ValueError("answer changed after original loop")
1066
+ os.replace(temp, target)
1067
+ finally:
1068
+ if temp.exists():
1069
+ temp.unlink()
1070
+ after = validate_outputs(output_root, input_root, empty_qids=failed)
1071
+ actual, _ = read_json(target)
1072
+ for answer in actual:
1073
+ old = kept[answer["qID"]]
1074
+ if answer["latency"] != old["latency"] or (answer["qID"] not in failed and answer != old):
1075
+ raise ValueError("healthy answer or measured latency changed")
1076
+ return {"blanked_qIDs": sorted(failed), "answer_sha256_before": original_sha,
1077
+ "answer_sha256_after": after["answer_sha256"],
1078
+ "successful_answer_contents_unchanged": True, "latencies_unchanged": True}
1079
+
1080
+
1081
+ def validate_question_event(event, expected_qids):
1082
+ if not isinstance(event, dict) or set(event) != {"qID", "stage", "exception_type", "exception_module", "message", "output_policy", "traceback"}:
1083
+ raise ValueError("unexpected trusted question event schema")
1084
+ if event["qID"] not in expected_qids or event["stage"] != "answer_loop_exception" or event["output_policy"] != "empty_content":
1085
+ raise ValueError("question event qID/stage/output policy differs")
1086
+ if any(not isinstance(event[k], str) for k in ("exception_type", "exception_module", "message")):
1087
+ raise ValueError("invalid per-question exception text")
1088
+ frames = event["traceback"]
1089
+ if not isinstance(frames, list) or not frames or any(not isinstance(t, dict) or set(t) != {"path", "function", "line"} or not isinstance(t["path"], str) or not isinstance(t["function"], str) or type(t["line"]) is not int for t in frames):
1090
+ raise ValueError("invalid per-question exception traceback")
1091
+ if frames[0]["path"] != "/app/solution/entrypoint/probe_entrypoint.py" or frames[0]["function"] != "answer_all":
1092
+ raise ValueError("question exception did not originate at the trusted answer-loop boundary")
1093
+ if event["exception_type"] == "ZipIntegrityError":
1094
+ raise ValueError("archive integrity error cannot be contained as a question failure")
1095
+
1096
+
1097
  def execute(contract, *, input_root: Path, output_root: Path, evidence_root: Path | None = None, backend=None, setup_imports=False, production_invocation=False):
1098
  """Synthetic-fixture seam; production supplies only baked paths and launcher."""
1099
  receipt = {"track": TRACK, "status": "failed", "source_validation": None,
 
1195
  guard.violations.append({"key": "environment API", "operation": "guard replaced"})
1196
  guard.final_check()
1197
  receipt["configuration_violations"] = list(guard.violations)
1198
+ question_events = []
1199
  if observations is not None:
1200
+ rows, _ = read_json(input_root / "request.json")
1201
+ rows = [rows] if isinstance(rows, dict) else rows
1202
+ expected_qids = {row["qID"] for row in rows}
1203
+ shell = sys.modules.get("solution.entrypoint.probe_entrypoint")
1204
+ staging = getattr(shell, "_ZIP_STAGING_EVENTS", None)
1205
+ if not isinstance(staging, list):
1206
+ raise ValueError("trusted shell staging ledger missing")
1207
+ ordinary = getattr(shell, "_QUESTION_FAILURE_EVENTS", None)
1208
+ fences = getattr(shell, "_MEDIA_FENCE_EVENTS", None)
1209
+ if not isinstance(ordinary, list) or not isinstance(fences, list):
1210
+ raise ValueError("trusted question/fence ledger missing")
1211
+ observations.reconcile_skipped(expected_qids, staging, ordinary, fences)
1212
+ for qid in receipt["input_validation"].get("unresolved_qIDs", []):
1213
+ if not any(event["qID"] == qid for event in observations.question_failures):
1214
+ observations.record_question_failure({"qID": qid, "stage": "input_media_resolution", "reason": "No unambiguous media candidate for request qID", "producer": {"function": "validate_inputs", "request_sha256": receipt["input_validation"]["request_sha256"]}})
1215
+ question_events = list(observations.question_failures)
1216
  receipt["engine_observations"] = observations.finish(receipt["input_validation"]["requests"])
1217
  if receipt["engine_observations"]["failures"]:
1218
  raise ValueError("original engine initialization or answer failure")
1219
  verify_sources(contract)
1220
  if validate_inputs(input_root) != receipt["input_validation"]:
1221
  raise ValueError("input metadata or layout changed during backend execution")
1222
+ receipt["question_failure_output"] = blank_failed_outputs(output_root, input_root, question_events)
1223
+ receipt["output_validation"] = validate_outputs(output_root, input_root, empty_qids={event["qID"] for event in question_events})
1224
  check_input_binding(binding, policy)
1225
  if guard.violations or receipt["degraded_markers"]:
1226
  raise ValueError("backend degraded or attempted fixed configuration mutation")
platform_wrapper/probe_entrypoint.py ADDED
@@ -0,0 +1,1118 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Submission I/O shell — the live Grand-Challenge PROCEDURE contract.
2
+
3
+ The contract (live GC socket table, evidence-packet-sourced, never the template):
4
+
5
+ read /input/request.json Request rows (cardinality: see U2)
6
+ read /input/FO_definitions.json organizer FO taxonomy text (encoding: see ASM-4)
7
+ read /input/batch-videos.zip per-question clips, ZIP kind (layout: U1; delivery: U3)
8
+ write /output/answer.json list of {qID, content, latency} — the ONLY file
9
+ this shell ever leaves in /output
10
+
11
+ ARMED MODES, REFUSAL OTHERWISE. This shell runs only under a SANCTIONED arming
12
+ form and refuses everything else with exit code 2 and zero writes:
13
+
14
+ submission (production): `--submission` or SURGFIELD_SUBMISSION_MODE=1
15
+ — the container CMD's form; THIS is the upload path.
16
+ probe (rehearsal): `--probe` or SURGFIELD_PROBE_MODE=1
17
+ — interface rehearsals and gate fixtures; never
18
+ uploaded as a production CMD.
19
+
20
+ The two modes execute the identical contract; they differ only in the stderr
21
+ arming stamp. Any invocation carrying NO sanctioned form — bare, wrong flags,
22
+ misconfigured CMD — exits 2 before any I/O.
23
+
24
+ Behavioral guarantees:
25
+ 1. `/output/answer.json` exists with FULL qID coverage from the moment the
26
+ request set is readable: a fallback answer per qID is PRE-SEEDED and each
27
+ computed answer atomically replaces its entry (progressive flush). A job
28
+ killed mid-run still leaves a complete, monotonically-improved answer file.
29
+ 2. `/output` contains exactly `answer.json` after a clean exit; scratch lives
30
+ in a private work directory (the atomic-replace temp is consumed by every
31
+ flush).
32
+ 3. One `G1DECODE qID=<q> frames=<n> first_frame_sha256=<h> res=<WxH>` line per
33
+ qID on stderr — the decode-proof channel (external parse contract,
34
+ byte-stable). Hash = sha256 of the first decoded frame's raw bytes.
35
+ Decoding runs in a KILLABLE SUBPROCESS with a hard timeout so a wedged
36
+ video can never hang the job.
37
+ 4. Time budget: the platform declares a 30 s/question unit rate with per-JOB
38
+ enforcement only (evidence pack, snapshot 2026-07-23T164642Z) —
39
+ operationally POOLED within a job. The shell budgets TOTAL job time and
40
+ logs pacing; it NEVER self-kills (the platform owns enforcement; a
41
+ self-kill could only forfeit answers the platform would have accepted).
42
+ 5. No network. This shell downloads nothing; model weights resolve from the
43
+ image's local cache (revision pinning is a build-recipe property).
44
+
45
+ Interface unknowns, handled defensively where marked in-code:
46
+ U1 zip internal tree (flat / plain/ / nested / .MP4 / videoID-keyed)
47
+ U2 request.json cardinality (list vs single object)
48
+ U3 whether GC auto-extracts ZIP-kind sockets
49
+ ASM-4 FO_definitions.json encoding (JSON string vs structured JSON)
50
+
51
+ Supported invocation forms:
52
+
53
+ surgfield-entrypoint --submission ... (dev/rehearsal)
54
+ surgfield-probe --probe ... (rehearsal)
55
+ python -m solution.entrypoint.probe_entrypoint --submission|--probe ...
56
+
57
+ P9 (2026-07-31): the PRODUCTION container CMD is NOT this console script — it is the
58
+ stdlib-only import-death-proof launcher `python -m surgfield_shell.launch --submission`
59
+ (docker/Dockerfile.p4 ENTRYPOINT), which writes the never-empty floor BEFORE importing
60
+ this engine. The forms above eager-import the engine (F1 death risk) and are for
61
+ dev/rehearsal only.
62
+
63
+ Direct file-path invocation (`python .../probe_entrypoint.py`) is NOT a
64
+ supported form — the engine package is not importable from that sys.path.
65
+ Once armed, the outermost coverage net in `main()` guarantees a full-coverage
66
+ `answer.json` (loudly-degraded fallbacks, never a bare /output) under that
67
+ misuse or any other post-arming failure.
68
+ """
69
+ from __future__ import annotations
70
+
71
+ import argparse
72
+ from contextlib import ExitStack
73
+ import json
74
+ import traceback
75
+ import math
76
+ import os
77
+ import re
78
+ import subprocess
79
+ import sys
80
+ import tempfile
81
+ import time
82
+ import zipfile
83
+ from pathlib import Path
84
+
85
+ # The DECLARED unit rate (track page: "30 seconds per question"), applied as a
86
+ # POOLED per-job budget per the evidence pack's operational verdict. Monitoring
87
+ # only — see guarantee 4.
88
+ POOLED_SEC_PER_QUESTION = 30.0
89
+
90
+ # Hard ceiling for one first-frame decode subprocess. Generous against real
91
+ # clips (a first-frame read is sub-second); tight against a wedged demuxer.
92
+ DECODE_TIMEOUT_SEC = 20.0
93
+
94
+ # P1-04: total wall budget for the pre-answer decode-proof phase (cold cv2
95
+ # subprocesses). Bounds the O(N) probe cost inside the platform's setup window;
96
+ # past it, remaining clips skip the probe (the in-process decode is still
97
+ # SIGALRM-bounded per P1-03, so coverage is not a safety dependency).
98
+ PROBE_PHASE_BUDGET_SEC = 60.0
99
+
100
+ # Trusted structured events; only raw-ZIP materialization failures populate it.
101
+ _ZIP_STAGING_EVENTS: list[dict] = []
102
+ _QUESTION_FAILURE_EVENTS: list[dict] = []
103
+ _MEDIA_FENCE_EVENTS: list[dict] = []
104
+
105
+
106
+ def _log(msg: str) -> None:
107
+ """All shell logging goes to stderr: contract-neutral (GC ignores it,
108
+ harnesses parse it), and it keeps stdout free of accidental output."""
109
+ print(msg, file=sys.stderr, flush=True)
110
+
111
+
112
+ def _safe_float(value, default: float = 0.0) -> float:
113
+ """Never-raise numeric coercion for request-row fields. A malformed field
114
+ costs AT MOST its own row's fidelity — never the run (the B2-19-1 class:
115
+ field access outside a containment must not be able to raise). inf/nan
116
+ clamp to the default: a non-finite window boundary is malformed, not big."""
117
+ try:
118
+ f = float(value)
119
+ return f if math.isfinite(f) else default
120
+ except (TypeError, ValueError):
121
+ return default
122
+
123
+
124
+ # Whole-video time-question marker (the Leg-B observability population): the
125
+ # boundary SEMANTICS stay held to the platform observer — the shell only
126
+ # refuses to hide the rows.
127
+ _TIME_Q = re.compile(
128
+ r"how long|how much time|at what time|when (was|is|did|does)|hh:mm:ss", re.I)
129
+
130
+
131
+ # ── request.json ─────────────────────────────────────────────────────────
132
+
133
+ def read_requests(path: Path) -> list[dict]:
134
+ """Parse request.json into a list of row dicts.
135
+
136
+ U2 (cardinality UNKNOWN): the helptext suggests a single object, the
137
+ template a list. Accept BOTH: a dict is wrapped into a one-row list, a
138
+ list is taken as-is. The answer file is always emitted as a LIST — the
139
+ single-object output mirror is unconfirmed, and a list is the shape the
140
+ grader's `load_items` reads (observed tolerant, not contract-asserted).
141
+ """
142
+ raw = json.loads(path.read_text(encoding="utf-8"))
143
+ if isinstance(raw, dict):
144
+ return [raw]
145
+ if isinstance(raw, list):
146
+ rows = [r for r in raw if isinstance(r, dict)]
147
+ if len(rows) != len(raw):
148
+ _log(f"PROBESHELL: skipping {len(raw) - len(rows)} non-object "
149
+ f"request row(s)")
150
+ return rows
151
+ raise ValueError(f"request.json: expected object or list, got {type(raw).__name__}")
152
+
153
+
154
+ # ── FO_definitions.json ──────────────────────────────────────────────────
155
+
156
+ def stage_fo_definitions(path: Path, workdir: Path) -> Path | None:
157
+ """Decode the organizer FO-definitions socket to a plain-text file and
158
+ return its path (for SURGFIELD_FO_DEFINITIONS_FILE), or None if absent.
159
+
160
+ ASM-4 (string-primary, template-observed): the value is expected to be a
161
+ JSON-encoded plain-text STRING. Tolerate structured JSON too — serialized
162
+ back to text so the taxonomy's heading parser sees SOMETHING rather than
163
+ the shell failing. Never fatal: on any error the engine falls back to its
164
+ built-in class floor.
165
+ """
166
+ if not path.is_file():
167
+ return None
168
+ try:
169
+ raw = json.loads(path.read_text(encoding="utf-8"))
170
+ text = raw if isinstance(raw, str) else json.dumps(raw, indent=1)
171
+ out = workdir / "FO_definitions.txt"
172
+ out.write_text(text, encoding="utf-8")
173
+ return out
174
+ except (ValueError, OSError) as e:
175
+ _log(f"PROBESHELL: FO_definitions unusable ({type(e).__name__}: {e}); "
176
+ f"engine uses its built-in taxonomy floor")
177
+ return None
178
+
179
+
180
+ # ── batch videos ─────────────────────────────────────────────────────────
181
+
182
+ # Video containers we recognize as clips. Lower-cased suffix match; the
183
+ # resolvers below also honor .MP4 (upper) via the engine's resolve_clip.
184
+ _VIDEO_EXTS = (".mp4", ".avi", ".mov", ".mkv", ".webm", ".m4v", ".mpg", ".mpeg")
185
+
186
+
187
+ def _log_input_tree(input_dir: Path) -> None:
188
+ """DIAGNOSTIC (log-only, no behavior change): dump the full /input tree so
189
+ the NEXT live try-out definitively reveals GC's real batch-videos layout.
190
+ One `PROBESHELL: /input tree: ...` line per file (path + size). Total by
191
+ construction — any failure here degrades to a single note, never a raise."""
192
+ try:
193
+ files = sorted((p for p in input_dir.rglob("*") if p.is_file()),
194
+ key=lambda p: str(p))
195
+ _log(f"PROBESHELL: /input tree: {len(files)} file(s) under {input_dir}")
196
+ for p in files:
197
+ try:
198
+ size = p.stat().st_size
199
+ except OSError:
200
+ size = -1
201
+ _log(f"PROBESHELL: /input tree: {p} ({size}B)")
202
+ except Exception as e:
203
+ _log(f"PROBESHELL: /input tree unavailable ({type(e).__name__}: {e})")
204
+
205
+
206
+ def _extract_zip(zpath: Path, dest: Path) -> None:
207
+ """Extract a ZIP into `dest` under the zip-slip guard.
208
+
209
+ SYMLINK-SAFETY IS LOAD-BEARING HERE: stdlib zipfile.extract writes every
210
+ member as a REGULAR FILE — it never honors a symlink attribute — so a
211
+ malicious link member cannot escape `dest`. If this is ever swapped for
212
+ another extractor (shutil.unpack_archive, an external tool), that guarantee
213
+ disappears and the escape reopens; re-prove before swapping."""
214
+ dest.mkdir(parents=True, exist_ok=True)
215
+ skipped = 0
216
+ with zipfile.ZipFile(zpath) as zf:
217
+ for info in zf.infolist():
218
+ name = info.filename
219
+ # zip-slip guard: refuse absolute paths and parent escapes.
220
+ if name.startswith(("/", "\\")) or ".." in Path(name).parts:
221
+ _log(f"PROBESHELL: skipping unsafe zip member {name!r}")
222
+ continue
223
+ # P1-19: PER-MEMBER isolation — a single bad member (BadZipFile / CRC
224
+ # / ENOSPC) must NOT propagate to the batch-wide except and zero the
225
+ # video signal for EVERY question. Log+skip the failing member; keep
226
+ # the rest of the archive.
227
+ try:
228
+ zf.extract(info, dest)
229
+ except Exception as e:
230
+ skipped += 1
231
+ _log(f"PROBESHELL: skipping unextractable zip member {name!r} "
232
+ f"({type(e).__name__}: {e}); other members still staged")
233
+ if skipped:
234
+ _log(f"PROBESHELL: WARN {skipped} zip member(s) failed to extract "
235
+ f"from {zpath.name}; the rest were staged (per-member isolation)")
236
+
237
+
238
+ def _is_zip(path: Path) -> bool:
239
+ """Content-sniff a regular file for the ZIP signature. Extension-agnostic —
240
+ GC has delivered the archive as `batch-videos` (no `.zip`), so we must NOT
241
+ rely on the name. `zipfile.is_zipfile` reads only the end-of-central-dir
242
+ record, so it is cheap even on a large non-zip (a real .mp4 => False)."""
243
+ try:
244
+ return zipfile.is_zipfile(path)
245
+ except OSError:
246
+ return False
247
+
248
+
249
+ def stage_videos_root(input_dir: Path, workdir: Path) -> Path | None:
250
+ """Locate the clips ROBUSTLY across GC's actual placement: return a single
251
+ directory to search, or None if genuinely no videos are present.
252
+
253
+ Round-1/try-out RCA (2026-07-25): GC put the `batch-videos` input at a path
254
+ OTHER than `/input/batch-videos.zip` and OTHER than `/input/batch-videos/`,
255
+ so the old exact-path resolver deferred every question (frames=0). This
256
+ resolver instead scans `/input` RECURSIVELY and accepts, order-deterministic:
257
+
258
+ (a) ANY ZIP found anywhere under /input — matched by CONTENT, not name
259
+ (handles `batch-videos` with no extension, or a nested
260
+ `.../<uuid>/batch-videos.zip`) — extracted into the work dir; AND
261
+ (b) ANY loose video file (`*.mp4`/`*.avi`/…) anywhere under /input —
262
+ symlinked into the same work dir PRESERVING its path relative to
263
+ /input, so a `plain/` sub-tree is kept intact for the qID resolver.
264
+
265
+ Everything lands under one `<work>/videos` root (never /output); the
266
+ existing qID→clip resolver (`normalize_clips`) then runs over it unchanged.
267
+ The zip-slip guard is preserved (`_extract_zip`). If NOTHING resolves we
268
+ keep the prior defer-to-rules behavior — never crash, never a bare output.
269
+
270
+ Regression: the known-good contract (`/input/batch-videos.zip` with
271
+ `plain/<qID>.mp4`, the P6 harness staging) is a strict subset of case (a)
272
+ and stages to `<work>/videos/plain/<qID>.mp4` exactly as before.
273
+ """
274
+ dest = workdir / "videos"
275
+ files = sorted((p for p in input_dir.rglob("*") if p.is_file()),
276
+ key=lambda p: str(p))
277
+ zips = []
278
+ loose_videos = []
279
+ for f in files:
280
+ if f.suffix.lower() in _VIDEO_EXTS:
281
+ loose_videos.append(f)
282
+ elif _is_zip(f):
283
+ zips.append(f)
284
+
285
+ staged_any = False
286
+ for z in zips: # deterministic (files is sorted)
287
+ _log(f"PROBESHELL: extracting archive {z} ({z.stat().st_size}B)")
288
+ _extract_zip(z, dest)
289
+ staged_any = True
290
+ for v in loose_videos: # deterministic (files is sorted)
291
+ rel = v.relative_to(input_dir)
292
+ link = dest / rel
293
+ link.parent.mkdir(parents=True, exist_ok=True)
294
+ if not link.exists():
295
+ link.symlink_to(v.resolve())
296
+ staged_any = True
297
+
298
+ if staged_any:
299
+ _log(f"PROBESHELL: staged videos root {dest} "
300
+ f"({len(zips)} archive(s), {len(loose_videos)} loose clip(s))")
301
+ return dest
302
+ _log("PROBESHELL: no batch-videos archive or video files found anywhere "
303
+ "under /input; video-dependent layers will defer on every question")
304
+ return None
305
+
306
+
307
+ def _resolve_deterministic(root: Path, key: str):
308
+ """Recursive, ORDER-DETERMINISTIC clip lookup by stem == `key`, preferring a
309
+ `plain/` path over `overlayed/` (we tuned on plain), then the shortest path,
310
+ then lexicographic. Complements the engine's tolerant resolvers for the
311
+ arbitrarily-nested trees GC may hand us. Returns a Path or None."""
312
+ hits = [p for p in root.rglob("*")
313
+ if p.is_file() and p.stem == key and p.suffix.lower() in _VIDEO_EXTS]
314
+
315
+ def sort_key(p: Path):
316
+ parts = [seg.lower() for seg in p.parts]
317
+ return (0 if "plain" in parts else 1,
318
+ 1 if "overlayed" in parts else 0,
319
+ len(p.parts), str(p))
320
+
321
+ hits.sort(key=sort_key)
322
+ return hits[0] if hits else None
323
+
324
+
325
+ # qID→clip resolution ladder — DOCUMENTED layout first, then bounded fallbacks.
326
+ # Each rung is labeled so the run log shows exactly which layout GC delivered
327
+ # (observe-don't-assume: the try-out enumeration + these lines finally pin the
328
+ # real tree). The template's reference `inference.py` reads `/input/plain/<qID>.mp4`
329
+ # (+ `overlayed/` with a burned-in HH:MM:SS clock).
330
+ # P9 (2026-07-31) — RCA drift corrected: `plain/` is the PRIMARY rung because WE
331
+ # unzip the `batch-videos` ZIP into a plain layout (stage_videos_root), NOT because
332
+ # the platform auto-extracts it. The team's OBSERVED_ON_PLATFORM note recorded the
333
+ # OPPOSITE (the container sees the ZIP as a FILE; no auto-extraction), so the earlier
334
+ # "platform auto-extracts to plain/" premise here was FALSE and is struck. The real
335
+ # /input tree is still un-captured; `_log_input_tree` (see main) emits it verbatim on
336
+ # the next live run so the ladder can finally be verified against ground truth. The
337
+ # rest are known-plausible fallbacks in priority order.
338
+ _RUNG_PRIMARY = "plain" # documented primary: <root>/plain/<qID>.mp4
339
+ _RUNG_OVERLAYED = "overlayed" # documented alt: <root>/overlayed/<qID>.mp4
340
+ _RUNG_FLAT = "flat" # bare root: <root>/<qID>.mp4
341
+ _RUNG_NESTED = "nested-stem" # recursive stem==qID (plain-preferring)
342
+ _RUNG_VIDEOID = "videoID" # keyed by row.videoID (qIDs may share a clip)
343
+ _RUNG_UNRESOLVED = "UNRESOLVED"
344
+
345
+
346
+ def _resolve_qid(root: Path, qid: str, video_id, resolve_video):
347
+ """Resolve ONE qID to a clip across the ordered ladder. Returns
348
+ `(Path|None, rung_label)`. Rungs are tried in strict priority order so the
349
+ documented layout always wins and any fallback is visible in the label.
350
+
351
+ The primary/overlayed/flat rungs reimplement the engine's `resolve_clip`
352
+ path checks SPLIT BY RUNG (so each is separately labeled); the P6 known-good
353
+ tree (`plain/<qID>.mp4`) resolves on rung 1 exactly as before — no mapping
354
+ change. Rungs 4–5 reuse the deterministic recursive + engine resolvers.
355
+ """
356
+ q = str(qid)
357
+ for ext in (".mp4", ".MP4"):
358
+ c = root / "plain" / f"{q}{ext}"
359
+ if c.is_file():
360
+ return c, _RUNG_PRIMARY
361
+ for ext in (".mp4", ".MP4"):
362
+ c = root / "overlayed" / f"{q}{ext}"
363
+ if c.is_file():
364
+ return c, _RUNG_OVERLAYED
365
+ for ext in (".mp4", ".MP4"):
366
+ c = root / f"{q}{ext}"
367
+ if c.is_file():
368
+ return c, _RUNG_FLAT
369
+ c = _resolve_deterministic(root, q)
370
+ if c is not None:
371
+ return c, _RUNG_NESTED
372
+ if video_id:
373
+ vid = str(video_id)
374
+ c = _resolve_deterministic(root, vid)
375
+ if c is not None and Path(c).is_file():
376
+ return c, _RUNG_VIDEOID
377
+ return None, _RUNG_UNRESOLVED
378
+
379
+
380
+ def normalize_clips(rows: list[dict], root: Path | None, workdir: Path) -> Path:
381
+ """Build `<work>/norm/plain/<qID>.mp4` links so the engine's clip-mode
382
+ lookup is deterministic regardless of the delivered tree, resolving each
383
+ qID through the labeled ladder in `_resolve_qid`.
384
+
385
+ OBSERVABILITY (robust-with-visibility, non-negotiable): one `RESOLVE` line
386
+ per qID naming the rung that fired; a loud `RESOLVE-FALLBACK` when a qID
387
+ resolves past the documented primary; a `RESOLVE-SUMMARY` rung histogram;
388
+ and a loud `RESOLVE-FAIL` naming every qID with NO clip. An unresolved qID
389
+ still gets no link — the engine defers and the rule layers answer it — but
390
+ it is never silent (Round-1 died on silent tolerant-guessing).
391
+ """
392
+ from solution.frame_sampler import resolve_video
393
+
394
+ norm = workdir / "norm" / "plain"
395
+ norm.mkdir(parents=True, exist_ok=True)
396
+ if root is None:
397
+ _log("PROBESHELL: RESOLVE-SUMMARY no videos root; every qID defers to rules")
398
+ return norm.parent
399
+
400
+ rung_counts: dict[str, int] = {}
401
+ unresolved: list[str] = []
402
+ # P1-18: qIDs resolved via the videoID rung map to a SHARED, un-trimmed
403
+ # video (not a per-qID clip). Clip-mode whole_clips from frame 0 and would
404
+ # give every qID sharing that video identical, wrong-window frames. Record
405
+ # them so the engine SEEKS the row's [start,end] instead of whole-clipping.
406
+ videoid_qids: list[str] = []
407
+ for row in rows:
408
+ qid = str(row.get("qID", ""))
409
+ if not qid:
410
+ continue
411
+ src, rung = _resolve_qid(root, qid, row.get("videoID"), resolve_video)
412
+ rung_counts[rung] = rung_counts.get(rung, 0) + 1
413
+ if src is None:
414
+ unresolved.append(qid)
415
+ _log(f"PROBESHELL: RESOLVE qID={qid} rung={_RUNG_UNRESOLVED} "
416
+ f"(no clip anywhere under videos root; defers to rules)")
417
+ continue
418
+ if rung == _RUNG_VIDEOID:
419
+ videoid_qids.append(qid)
420
+ if rung != _RUNG_PRIMARY:
421
+ _log(f"PROBESHELL: RESOLVE-FALLBACK qID={qid} rung={rung} src={src} "
422
+ f"(documented plain/{qid}.mp4 absent — resolved via fallback)")
423
+ else:
424
+ _log(f"PROBESHELL: RESOLVE qID={qid} rung={rung} src={src}")
425
+ link = norm / f"{qid}.mp4"
426
+ if not link.exists():
427
+ link.symlink_to(Path(src).resolve())
428
+
429
+ # P1-18: persist the videoID-rung set next to the normalized tree so the
430
+ # clip-mode engine can pick SEEK-with-window for exactly these qIDs.
431
+ marker = norm.parent / "videoid_rung.json"
432
+ try:
433
+ marker.write_text(json.dumps(sorted(videoid_qids)))
434
+ except OSError as e:
435
+ _log(f"PROBESHELL: could not write videoid_rung marker ({e})")
436
+ if videoid_qids:
437
+ _log(f"PROBESHELL: RESOLVE-VIDEOID {len(videoid_qids)} qID(s) share an "
438
+ f"un-trimmed video — engine will SEEK [start,end], not whole-clip")
439
+
440
+ _log(f"PROBESHELL: RESOLVE-SUMMARY rungs={rung_counts} "
441
+ f"resolved={sum(rung_counts.values()) - len(unresolved)} "
442
+ f"unresolved={len(unresolved)}")
443
+ if unresolved:
444
+ shown = unresolved[:20]
445
+ _log(f"PROBESHELL: RESOLVE-FAIL {len(unresolved)} qID(s) had NO clip: "
446
+ f"{shown}{' …' if len(unresolved) > 20 else ''} — video-blind on these")
447
+ return norm.parent
448
+
449
+
450
+ # ── decode-proof channel ─────────────────────────────────────────────────
451
+
452
+ # Self-contained first-frame prober run in a KILLABLE subprocess (production
453
+ # hardening): cv2 can wedge inside C on a malformed container, beyond any
454
+ # in-process timeout. The snippet prints one JSON line; anything else (crash,
455
+ # hang->kill, no frame) degrades to the frames=0 record. The emitted G1DECODE
456
+ # line stays byte-stable.
457
+ #
458
+ # P1-07: the child prints the sentinel ``CV2_INIT_OK`` on stderr AFTER cv2
459
+ # imports+constructs successfully. That lets the parent tell a clip-specific
460
+ # decoder crash (sentinel present, then rc!=0) from a SYSTEMIC child-env fault
461
+ # (no sentinel: missing libGL, cv2 import failure, OOM-kill of the child) — the
462
+ # latter fires for EVERY clip and must NOT unlink every clip (the Round-1
463
+ # frames=0 collapse).
464
+ _DECODE_SNIPPET = (
465
+ "import hashlib, json, sys\n"
466
+ "try:\n"
467
+ " import cv2\n"
468
+ " cap = cv2.VideoCapture(sys.argv[1])\n"
469
+ "except Exception:\n"
470
+ " sys.exit(3)\n"
471
+ "sys.stderr.write('CV2_INIT_OK')\n"
472
+ "ok, frame = cap.read()\n"
473
+ "cap.release()\n"
474
+ "if ok and frame is not None:\n"
475
+ " print(json.dumps({'sha': hashlib.sha256(frame.tobytes()).hexdigest(),\n"
476
+ " 'res': str(frame.shape[1]) + 'x' + str(frame.shape[0])}))\n"
477
+ )
478
+
479
+
480
+ def _probe_first_frame(clip: Path) -> tuple[str, int, str, str | None]:
481
+ """(sha, frames, res, danger) for one clip via the killable subprocess prober.
482
+
483
+ `danger` classifies the prober outcome for the H6 in-process fence:
484
+ - "WEDGE" → subprocess TimeoutExpired (the child hung on decode); the
485
+ in-process engine would hang on the same clip.
486
+ - "CRASH" → cv2 INITIALIZED (sentinel seen) then the child exited
487
+ non-zero on THIS clip (segfault/OOM/abort in the decoder).
488
+ - "ENVFAULT" → rc!=0 with NO sentinel: the child could not even init cv2
489
+ (P1-07 — a SYSTEMIC env fault, not a clip crash). Reported
490
+ so the caller can ABORT fencing instead of unlinking every
491
+ clip and going whole-pipeline video-blind.
492
+ - None → SAFE: clean rc==0 (frame-0 decoded OR cleanly undecodable),
493
+ an unparseable-stdout rc==0, or a subprocess spawn error.
494
+ A clean frame-0 miss is NOT dangerous — frame_sampler skips
495
+ an unreadable frame and answers from the rest.
496
+ """
497
+ try:
498
+ r = subprocess.run([sys.executable, "-c", _DECODE_SNIPPET, str(clip)],
499
+ capture_output=True, text=True,
500
+ timeout=DECODE_TIMEOUT_SEC)
501
+ if r.returncode != 0:
502
+ if "CV2_INIT_OK" not in (r.stderr or ""):
503
+ # P1-07: child never initialized cv2 → systemic env fault, not a
504
+ # clip-specific crash. Do NOT fence (would blind-unlink all clips).
505
+ _log(f"PROBESHELL: decode probe ENV-FAULT for {clip.name} "
506
+ f"(rc={r.returncode}, cv2 never initialized); NOT fencing this clip")
507
+ return "none", 0, "0x0", "ENVFAULT"
508
+ _log(f"PROBESHELL: decode probe crashed for {clip.name} "
509
+ f"(returncode={r.returncode}); recording frames=0 and fencing")
510
+ return "none", 0, "0x0", "CRASH"
511
+ if r.stdout.strip():
512
+ d = json.loads(r.stdout.strip().splitlines()[-1])
513
+ return d["sha"], 1, d["res"], None
514
+ except subprocess.TimeoutExpired:
515
+ _log(f"PROBESHELL: decode timeout ({DECODE_TIMEOUT_SEC:.0f}s) for "
516
+ f"{clip.name}; recording frames=0 and fencing")
517
+ return "none", 0, "0x0", "WEDGE"
518
+ except Exception as e:
519
+ _log(f"PROBESHELL: decode probe failed for {clip.name} "
520
+ f"({type(e).__name__}: {e})")
521
+ return "none", 0, "0x0", None
522
+
523
+
524
+ def _fence_decision(to_fence: dict[str, str], n_probed: int):
525
+ """P1-07: decide which qIDs to fence, and whether the pattern is SYSTEMIC.
526
+
527
+ Returns ``(systemic, env_faults, dangers)``:
528
+ - ``env_faults`` — qIDs whose child never initialized cv2 (env fault).
529
+ - ``dangers`` — qIDs with a real per-clip WEDGE/CRASH.
530
+ - ``systemic`` — True if ANY env fault, OR every probed clip came back
531
+ dangerous (>1). When systemic, the caller must NOT
532
+ unlink clips (would blind the whole pipeline to the
533
+ rules floor at exit 0); the in-process SIGALRM decode
534
+ deadline provides containment instead.
535
+ """
536
+ env_faults = {q for q, r in to_fence.items() if r == "ENVFAULT"}
537
+ dangers = {q: r for q, r in to_fence.items() if r in ("WEDGE", "CRASH")}
538
+ systemic = bool(env_faults) or (n_probed > 1 and len(dangers) == n_probed)
539
+ return systemic, env_faults, dangers
540
+
541
+
542
+ def emit_decode_records(rows: list[dict], norm_root: Path,
543
+ probe_budget_sec: float | None = None, zip_batch=None
544
+ ) -> tuple[dict[str, str], int]:
545
+ """One G1DECODE stderr line per UNIQUE qID — the decode-proof channel.
546
+
547
+ Hash convention (a parse contract with the external rehearsal harness —
548
+ never change it unilaterally): sha256 over the raw bytes of the FIRST
549
+ decoded frame (`frame.tobytes()`). `frames` counts what THIS probe decoded
550
+ (telemetry — the assertion compares the hash). Absent/undecodable/timed-out
551
+ clips emit `frames=0 first_frame_sha256=none res=0x0` rather than nothing,
552
+ so coverage of the channel itself is checkable.
553
+
554
+ P1-04: each cold cv2 subprocess is charged nowhere in the pooled budget, so
555
+ this phase (a) probes each unique qID ONCE (no per-duplicate-row re-decode),
556
+ (b) caches by resolved clip target so qIDs SHARING a clip decode once, and
557
+ (c) stops probing once `probe_budget_sec` is exhausted (remaining clips are
558
+ still safe — the in-process decode is SIGALRM-bounded per P1-03).
559
+
560
+ Returns `(to_fence, n_probed)`:
561
+ - `to_fence` = {qID: "WEDGE"|"CRASH"|"ENVFAULT"} — qIDs whose prober hit a
562
+ signal. The caller fences WEDGE/CRASH; ENVFAULT (systemic, P1-07) is
563
+ surfaced but NOT unlinked.
564
+ - `n_probed` = number of clips actually decoded this phase (for the caller's
565
+ systemic-pattern guard: all-probed-clips-dangerous ⇒ do not fence).
566
+ """
567
+ if zip_batch is not None:
568
+ return _emit_zip_decode_records(rows, norm_root, zip_batch, probe_budget_sec)
569
+ to_fence: dict[str, str] = {}
570
+ seen_q: set[str] = set()
571
+ by_target: dict[str, tuple] = {}
572
+ n_probed = 0
573
+ t0 = time.perf_counter()
574
+ budget_logged = False
575
+ for row in rows:
576
+ qid = str(row.get("qID", ""))
577
+ if not qid or qid in seen_q:
578
+ continue
579
+ seen_q.add(qid)
580
+ clip = norm_root / "plain" / f"{qid}.mp4"
581
+ sha, frames, res, danger = ("none", 0, "0x0", None)
582
+ if clip.is_file():
583
+ key = os.path.realpath(clip)
584
+ if key in by_target: # shared clip: decode once
585
+ sha, frames, res, danger = by_target[key]
586
+ elif (probe_budget_sec is not None
587
+ and time.perf_counter() - t0 > probe_budget_sec):
588
+ if not budget_logged:
589
+ budget_logged = True
590
+ _log(f"PROBESHELL: decode-proof budget ({probe_budget_sec:.0f}s) "
591
+ f"exhausted; skipping probe for remaining clips (the "
592
+ f"in-process decode is SIGALRM-bounded, so still safe)")
593
+ else:
594
+ sha, frames, res, danger = _probe_first_frame(clip)
595
+ by_target[key] = (sha, frames, res, danger)
596
+ n_probed += 1
597
+ if danger:
598
+ to_fence[qid] = danger
599
+ _log(f"G1DECODE qID={qid} frames={frames} first_frame_sha256={sha} res={res}")
600
+ return to_fence, n_probed
601
+
602
+
603
+ # ── answers: pre-seed, compute, flush ────────────────────────────────────
604
+
605
+ def _record_zip_failure(zip_batch, qid, phase, error):
606
+ event = zip_batch.record_failure(qid, phase, error)
607
+ _log("ZIP_STAGING_FAILURE " + json.dumps(event, sort_keys=True))
608
+
609
+
610
+ def _emit_zip_decode_records(rows, norm_root, zip_batch, probe_budget_sec):
611
+ from solution.entrypoint.segment_zip_input import MemberStagingError
612
+ to_fence, seen_q, by_target = {}, set(), {}
613
+ n_probed, t0, budget_logged = 0, time.perf_counter(), False
614
+ for row in rows:
615
+ qid = str(row.get("qID", ""))
616
+ if not qid or qid in seen_q:
617
+ continue
618
+ seen_q.add(qid)
619
+ sha, frames, res, danger = "none", 0, "0x0", None
620
+ key = zip_batch.cache_key(qid)
621
+ if key in by_target:
622
+ sha, frames, res, danger = by_target[key]
623
+ elif probe_budget_sec is not None and time.perf_counter() - t0 > probe_budget_sec:
624
+ if not budget_logged:
625
+ _log(f"PROBESHELL: decode-proof budget ({probe_budget_sec:.0f}s) exhausted; remaining ZIP clips unprobed")
626
+ budget_logged = True
627
+ else:
628
+ with ExitStack() as clips:
629
+ try:
630
+ clip = clips.enter_context(zip_batch.member(qid, phase="startup_probe"))
631
+ except MemberStagingError as error:
632
+ _record_zip_failure(zip_batch, qid, "startup_probe", error)
633
+ else:
634
+ # Exception classification ended before invoking the decoder.
635
+ if clip.is_file():
636
+ sha, frames, res, danger = _probe_first_frame(clip)
637
+ by_target[key] = (sha, frames, res, danger)
638
+ n_probed += 1
639
+ if danger:
640
+ to_fence[qid] = danger
641
+ _log(f"G1DECODE qID={qid} frames={frames} first_frame_sha256={sha} res={res}")
642
+ return to_fence, n_probed
643
+
644
+
645
+ def write_answers(output_dir: Path, answers: list[dict]) -> Path:
646
+ """Atomic (re)write of /output/answer.json: temp-in-/output + os.replace
647
+ (same filesystem => atomic; the temp is consumed by every flush). Called
648
+ once to pre-seed and then after every computed answer — the file is
649
+ complete at every instant."""
650
+ output_dir.mkdir(parents=True, exist_ok=True)
651
+ out = output_dir / "answer.json"
652
+ tmp = output_dir / ".answer.json.tmp"
653
+ tmp.write_text(json.dumps(answers, indent=1), encoding="utf-8")
654
+ os.replace(tmp, out)
655
+ return out
656
+
657
+
658
+ # C1 (2026-07-25): the grader's adversarial detector whole-submission-DQs on these public
659
+ # signal substrings (focus.evaluation.adversarial). Our answers must never trip it.
660
+ #
661
+ # P9 (2026-07-31): promoted from OBSERVE-ONLY to ENFORCE. The prior hedge shipped a colliding
662
+ # answer verbatim; a single genuine free-text answer that happens to contain a signal substring
663
+ # would then risk zeroing all 10 buckets (whole-submission DQ). ENFORCE = on a collision we
664
+ # DECLINE that ONE question by replacing its ENTIRE content with the format's SAFE_FALLBACK
665
+ # (a full-answer forfeit, at most 1 question lost) and log loudly. This is a DECLINE, not an
666
+ # evasion-scrub: we do NOT surgically strip the signal to sneak the rest of the answer through
667
+ # (that would be grader-control evasion, HARD-BARRED). Coverage is preserved (still one answer
668
+ # per qID). GOVERNANCE: this supersedes the 2026-07-25 observe-only disposition; the
669
+ # scrub-vs-decline distinction is flagged for Reza ratification in P9_coverage.md.
670
+ _C1_INJECTION_SIGNALS = (
671
+ "ignore previous instructions", "ignore all previous", "disregard the above",
672
+ "you are now", "system prompt:", "new instructions:", "override:",
673
+ "forget your instructions", "act as if", "pretend that the answer is correct",
674
+ "always respond with correct", "the answer is definitely correct",
675
+ )
676
+
677
+
678
+ def _c1_safe_fallback_for(qid: str, qid_to_question: dict[str, str]) -> str:
679
+ """Format-aware DECLINE content for a colliding answer. Best-effort: infer the answer
680
+ format from the question and take its SAFE_FALLBACK; plain 'none' if inference is
681
+ unavailable (total by construction — a raise here must never escape the pre-flight)."""
682
+ try:
683
+ from solution.answer_validator import SAFE_FALLBACK
684
+ from solution.cap_fmt_inferrer import infer_all
685
+ q = qid_to_question.get(qid, "")
686
+ return SAFE_FALLBACK.get(infer_all(q)["answer_format"], "none")
687
+ except Exception:
688
+ return "none"
689
+
690
+
691
+ def _c1_collision_preflight(answers: list[dict],
692
+ rows: list[dict] | None = None) -> int:
693
+ """ENFORCE: on any answer whose content collides with a public injection signal (would
694
+ whole-submission-DQ at the grader), DECLINE that ONE question — replace its whole content
695
+ with the format's SAFE_FALLBACK and log loudly. Mutates `answers` in place. Returns the
696
+ collision (blocked) count. This is a decline, never an evasion-scrub."""
697
+ qid_to_question = {str(r.get("qID", "")): str(r.get("question", ""))
698
+ for r in (rows or [])}
699
+ hits = 0
700
+ for a in answers:
701
+ content = str(a.get("content", "")).lower()
702
+ for sig in _C1_INJECTION_SIGNALS:
703
+ if sig in content:
704
+ hits += 1
705
+ qid = str(a.get("qID", ""))
706
+ safe = _c1_safe_fallback_for(qid, qid_to_question)
707
+ _log(f"C1-COLLISION-BLOCKED qID={qid} signal={sig!r} "
708
+ f"(answer would risk whole-submission DQ; DECLINED → content={safe!r})")
709
+ a["content"] = safe
710
+ break
711
+ if hits:
712
+ _log(f"C1-COLLISION-BLOCKED: {hits} answer(s) collided with the public injection-signal "
713
+ f"list — DECLINED (content replaced with SAFE_FALLBACK; coverage preserved)")
714
+ return hits
715
+
716
+
717
+ def fallback_answers(rows: list[dict]) -> list[dict]:
718
+ """Full-coverage fallback list that does NOT require the engine package.
719
+
720
+ The PRE-SEED payload and the outermost coverage net's payload: it must work
721
+ even when `solution.*` itself is unimportable (e.g. the unsupported
722
+ file-path invocation), so the format-aware fallback is best-effort and the
723
+ floor is a plain "none" per unique qID. Degradation is loud on stderr and
724
+ fully countable downstream — nothing is masked.
725
+ """
726
+ try:
727
+ from solution.answer_validator import SAFE_FALLBACK
728
+ from solution.cap_fmt_inferrer import infer_all
729
+
730
+ def content_for(question: str) -> str:
731
+ # Total by construction: this executes INSIDE the outermost except
732
+ # handler, where a raise would escape the net entirely. The
733
+ # format-aware attempt is best-effort; "none" is unconditional.
734
+ try:
735
+ return SAFE_FALLBACK.get(infer_all(question)["answer_format"], "none")
736
+ except Exception:
737
+ return "none"
738
+ except Exception:
739
+ def content_for(question: str) -> str:
740
+ return "none"
741
+ answers, seen = [], set()
742
+ for row in rows:
743
+ qid = str(row.get("qID", ""))
744
+ if not qid or qid in seen:
745
+ continue
746
+ seen.add(qid)
747
+ answers.append({"qID": qid,
748
+ "content": content_for(str(row.get("question", ""))),
749
+ "latency": 0.0})
750
+ return answers
751
+
752
+
753
+ def answer_all(rows: list[dict], videos_root: Path, device: str,
754
+ answers: list[dict], output_dir: Path,
755
+ proc_t0: float | None = None, zip_batch=None) -> list[dict]:
756
+ """Upgrade the pre-seeded `answers` in place — one engine answer per unique
757
+ qID — flushing atomically after every question.
758
+
759
+ Failure containment is per-question: an ordinary exception inside one
760
+ answer records its origin and writes empty content for only that question. Latency
761
+ is the full per-question wall (infer -> answer -> validate).
762
+
763
+ Pooled-budget monitor (guarantee 4): total = 30 s x N; an overrun is logged
764
+ LOUDLY once and the run CONTINUES — enforcement is the platform's, and the
765
+ progressive flush means a platform kill costs only the not-yet-upgraded
766
+ answers, never the file.
767
+ """
768
+ from solution.answer_validator import validate
769
+ from solution.cap_fmt_inferrer import infer_all
770
+ from solution.container import SubmissionContainer
771
+ from solution.question_context import Request
772
+
773
+ # P1-20: time the eager model/detector load (the largest setup sink) so a
774
+ # cold-start init overrun is visible and attributable, not silent.
775
+ _init_t0 = time.perf_counter()
776
+ container = SubmissionContainer(videos_dir=videos_root, device=device)
777
+ _init_wall = time.perf_counter() - _init_t0
778
+ _log(f"PROBESHELL: CONTAINER-INIT {_init_wall:.1f}s (eager model/detector load)"
779
+ + (f"; total setup {time.perf_counter() - proc_t0:.1f}s from process start"
780
+ if proc_t0 is not None else ""))
781
+ slot = {a["qID"]: i for i, a in enumerate(answers)}
782
+
783
+ budget = POOLED_SEC_PER_QUESTION * len(answers)
784
+ job_t0 = time.perf_counter()
785
+ budget_breached = False
786
+ done: set[str] = set()
787
+ for row in rows:
788
+ qid = str(row.get("qID", ""))
789
+ if not qid or qid in done or qid not in slot:
790
+ if qid and qid in done:
791
+ _log(f"PROBESHELL: skipping duplicate qID row ({qid!r})")
792
+ continue
793
+ done.add(qid)
794
+ elapsed = time.perf_counter() - job_t0
795
+ if elapsed > budget and not budget_breached:
796
+ budget_breached = True
797
+ _log(f"PROBESHELL: POOLED BUDGET EXCEEDED ({elapsed:.0f}s > "
798
+ f"{budget:.0f}s for {len(answers)} questions) — continuing; "
799
+ f"the platform owns enforcement and the answer file is "
800
+ f"already complete at every instant")
801
+ t0 = time.perf_counter()
802
+ question = str(row.get("question", ""))
803
+ # Same B2-19-1 class, sibling site: these run OUTSIDE the per-question
804
+ # try — a raising coercion here would degrade the WHOLE run via the
805
+ # outer net instead of costing one row. Never-raise by construction.
806
+ start = _safe_float(row.get("start_time", 0.0) or 0.0)
807
+ end = _safe_float(row.get("end_time", 0.0) or 0.0)
808
+ try:
809
+ scratch = infer_all(question, start_time=start, end_time=end)
810
+ # H5: prefer an answer_format EXPOSED on the request over the inferred
811
+ # one. The observed live contract does NOT carry this field (the
812
+ # Request schema is qID/videoID/start/end/procedure_type/question),
813
+ # so on real inputs this is byte-identical to inference — a defensive
814
+ # guard that honors a true format if a future contract exposes it,
815
+ # avoiding the misroute the H5 validator policy otherwise absorbs.
816
+ exposed_fmt = row.get("answer_format")
817
+ fmt = exposed_fmt if isinstance(exposed_fmt, str) and exposed_fmt \
818
+ else scratch.get("answer_format", "open_ended")
819
+ req = Request(
820
+ qID=qid,
821
+ videoID=str(row.get("videoID", "")),
822
+ start_time=start,
823
+ end_time=end,
824
+ procedure_type=str(row.get("procedure_type", "")),
825
+ question=question,
826
+ )
827
+ if zip_batch is None:
828
+ resp = container.answer(req, scratch_init=scratch)
829
+ else:
830
+ from solution.entrypoint.segment_zip_input import MemberStagingError
831
+ if zip_batch.failed(qid):
832
+ answers[slot[qid]] = {"qID": qid, "content": "", "latency": time.perf_counter() - t0}
833
+ write_answers(output_dir, answers)
834
+ continue
835
+ with ExitStack() as clips:
836
+ try:
837
+ clips.enter_context(zip_batch.member(qid))
838
+ except MemberStagingError as error:
839
+ _record_zip_failure(zip_batch, qid, "answer", error)
840
+ answers[slot[qid]] = {"qID": qid, "content": "", "latency": time.perf_counter() - t0}
841
+ write_answers(output_dir, answers)
842
+ continue
843
+ # Close all loader FDs BEFORE the lease unlinks its clip.
844
+ # This callback is ZIP-only; the original loose path is unchanged.
845
+ if container.video_loader is not None:
846
+ clips.callback(container.video_loader.close_all)
847
+ # Model exceptions occur outside the materialization catch.
848
+ resp = container.answer(req, scratch_init=scratch)
849
+ content = validate(resp.content, fmt)
850
+ answers[slot[qid]] = {"qID": qid, "content": content,
851
+ "latency": time.perf_counter() - t0}
852
+ except Exception as e:
853
+ from solution.entrypoint.segment_zip_input import ZipIntegrityError
854
+ if isinstance(e, ZipIntegrityError):
855
+ raise
856
+ event = {"qID": qid, "stage": "answer_loop_exception", "exception_type": type(e).__name__,
857
+ "exception_module": type(e).__module__, "message": str(e), "output_policy": "empty_content",
858
+ "traceback": [{"path": t.filename, "function": t.name, "line": t.lineno} for t in traceback.extract_tb(e.__traceback__)]}
859
+ _QUESTION_FAILURE_EVENTS.append(event)
860
+ _log("QUESTION_FAILURE " + json.dumps(event, sort_keys=True))
861
+ answers[slot[qid]] = {"qID": qid, "content": "", "latency": time.perf_counter() - t0}
862
+ write_answers(output_dir, answers)
863
+ _log(f"PROBESHELL: job wall={time.perf_counter() - job_t0:.1f}s "
864
+ f"pooled-budget={budget:.0f}s n={len(answers)}")
865
+ # P2-06: emit the per-layer latency/attribution summary at end of run (it was
866
+ # accumulated every question but never reported — the only instrument for
867
+ # diagnosing which layer won and where budget was burned). Never fatal.
868
+ try:
869
+ container.metrics.report()
870
+ except Exception as e:
871
+ _log(f"PROBESHELL: metrics report skipped (non-fatal {type(e).__name__}: {e})")
872
+ return answers
873
+
874
+
875
+ # ── entry ────────────────────────────────────────────────────────────────
876
+
877
+ def main(argv: list[str] | None = None) -> int:
878
+ ap = argparse.ArgumentParser(
879
+ description="solution submission shell (live GC PROCEDURE contract)")
880
+ ap.add_argument("--submission", action="store_true",
881
+ help="arm SUBMISSION mode — the sanctioned production CMD "
882
+ "form (equivalent env: SURGFIELD_SUBMISSION_MODE=1)")
883
+ ap.add_argument("--probe", action="store_true",
884
+ help="arm PROBE mode for interface rehearsal (equivalent "
885
+ "env: SURGFIELD_PROBE_MODE=1)")
886
+ ap.add_argument("--input-dir", default="/input")
887
+ ap.add_argument("--output-dir", default="/output")
888
+ ap.add_argument("--device", default="cuda",
889
+ help="engine device (default: cuda); in a CPU-only context "
890
+ "the detector load fails with one benign WARN and the "
891
+ "detection layers defer — never a crash")
892
+ args = ap.parse_args(argv)
893
+
894
+ # P1-20: capture a process-start monotonic BEFORE any staging/model load so
895
+ # the platform-scored ~120s setup window is measurable end-to-end (the shell
896
+ # previously only timed from inside answer_all, AFTER eager model loading —
897
+ # a cold-start overrun was invisible).
898
+ _proc_t0 = time.perf_counter()
899
+
900
+ # Refusal gate (design unchanged by the promotion): no sanctioned arming
901
+ # form, no run — exit 2 before any I/O. A misbuilt CMD cannot produce
902
+ # output it was never armed to produce.
903
+ submission = args.submission or os.environ.get("SURGFIELD_SUBMISSION_MODE") == "1"
904
+ probe = args.probe or os.environ.get("SURGFIELD_PROBE_MODE") == "1"
905
+ if not (submission or probe):
906
+ _log("PROBE-REFUSED: no sanctioned arming form present "
907
+ "(--submission|--probe, or SURGFIELD_SUBMISSION_MODE=1|"
908
+ "SURGFIELD_PROBE_MODE=1); nothing written. This shell is the "
909
+ "submission entrypoint ONLY when armed via a sanctioned form")
910
+ return 2
911
+ if submission:
912
+ _log("PROBESHELL: submission-mode armed (sanctioned production path)")
913
+ else:
914
+ _log("PROBESHELL: probe-mode armed (solution tree; interface rehearsal only)")
915
+
916
+ input_dir, output_dir = Path(args.input_dir), Path(args.output_dir)
917
+ _ZIP_STAGING_EVENTS.clear()
918
+ _QUESTION_FAILURE_EVENTS.clear()
919
+ _MEDIA_FENCE_EVENTS.clear()
920
+ zip_batch = None
921
+ # P9 (2026-07-31): tempfile.mkdtemp() is deferred to INSIDE the coverage net
922
+ # below (it is only needed for video staging). On a read-only/absent /tmp — the
923
+ # exact failsafe scenario, GC /tmp semantics UNCONFIRMED — the old top-of-main
924
+ # mkdtemp raised BEFORE the format-aware pre-seed, shipping the launcher's floor
925
+ # instead of the strictly-better format-aware fallback that needs no /tmp.
926
+
927
+ try:
928
+ rows = read_requests(input_dir / "request.json")
929
+ except (OSError, ValueError) as e:
930
+ # P9: the launcher (surgfield_shell.launch) already wrote a full-coverage floor
931
+ # to /output BEFORE this shell ran. Without a readable request set there is no
932
+ # qID universe to re-cover, so do NOT clobber that good floor with an empty '[]'
933
+ # (a latent re-introduction of the empty-answer regression the floor exists to
934
+ # prevent, if the launcher's read succeeded where this re-read raised). Only write
935
+ # '[]' when no answer.json exists at all — the socket is still never left bare.
936
+ _log(f"PROBESHELL: request.json unreadable ({type(e).__name__}: {e})")
937
+ ans = output_dir / "answer.json"
938
+ if ans.exists():
939
+ _log("PROBESHELL: preserving the existing launcher floor on disk "
940
+ "(NOT clobbering with [])")
941
+ else:
942
+ write_answers(output_dir, [])
943
+ _log("PROBESHELL: no prior floor on disk — wrote empty [] "
944
+ "(output socket non-bare)")
945
+ return 1
946
+
947
+ # PRE-SEED (guarantee 1): full-coverage fallback answers land in
948
+ # /output/answer.json before any engine work — a kill at any later instant
949
+ # still leaves a complete file. This lands BEFORE mkdtemp so a /tmp failure
950
+ # cannot foreclose the format-aware floor.
951
+ answers = fallback_answers(rows)
952
+ write_answers(output_dir, answers)
953
+ _log(f"PROBESHELL: pre-seeded {len(answers)} fallback answers")
954
+
955
+ # OUTERMOST COVERAGE NET (post-arming): once the request set is readable,
956
+ # NO failure below — engine imports included, i.e. even the unsupported
957
+ # file-path invocation — may leave /output without a full-coverage
958
+ # answer.json. Everything between here and the final write is contained;
959
+ # degradation is loud, never silent.
960
+ try:
961
+ # P9: video-staging workdir, created HERE (inside the coverage net, AFTER the
962
+ # format-aware pre-seed). A read-only/absent /tmp now degrades to the pre-seeded
963
+ # floor via this net, never forecloses it.
964
+ workdir = Path(tempfile.mkdtemp(prefix="surgfield-shell-"))
965
+
966
+ # RUN-SCOPED ENV RESET (in-process isolation): a prior main() call in
967
+ # this interpreter may have left these pointing at a dead workdir —
968
+ # every run starts from a clean slate and sets only what ITS inputs
969
+ # justify.
970
+ os.environ.pop("SURGFIELD_FO_DEFINITIONS_FILE", None)
971
+ os.environ.pop("SURGFIELD_CLIP_INPUT", None)
972
+
973
+ # Zeroed-boundary telemetry (B2-19-1 cure: lives INSIDE the net AND the
974
+ # coercion is never-raise): a row whose end_time is 0.0 — or malformed
975
+ # and therefore coerced to 0.0 — gives every duration-gated layer
976
+ # duration 0; count it loudly, never silently, never fatally.
977
+ zeroed = sum(1 for r in rows
978
+ if _safe_float(r.get("end_time", 0.0) or 0.0) == 0.0)
979
+ if zeroed:
980
+ _log(f"PROBESHELL: WARN {zeroed} request row(s) with end_time==0.0 "
981
+ f"(zeroed boundary)")
982
+ # Whole-video time-questions with start==0 — the Leg-B population:
983
+ # covered by the engine, but never invisibly (semantics HELD to the
984
+ # platform observer; the shell records, it does not resolve).
985
+ whole_video = sum(
986
+ 1 for r in rows
987
+ if _safe_float(r.get("start_time", 0.0) or 0.0) == 0.0
988
+ and _safe_float(r.get("end_time", 0.0) or 0.0) > 0.0
989
+ and _TIME_Q.search(str(r.get("question", ""))))
990
+ if whole_video:
991
+ _log(f"PROBESHELL: WARN {whole_video} whole-video time-question "
992
+ f"row(s) with start_time==0.0 (boundary semantics recorded, "
993
+ f"not resolved)")
994
+
995
+ # DIAGNOSTIC (log-only): dump the full /input tree so the next live
996
+ # try-out definitively reveals GC's real batch-videos placement.
997
+ _log_input_tree(input_dir)
998
+
999
+ fo_path = stage_fo_definitions(input_dir / "FO_definitions.json", workdir)
1000
+ if fo_path is not None:
1001
+ os.environ["SURGFIELD_FO_DEFINITIONS_FILE"] = str(fo_path)
1002
+ _log(f"PROBESHELL: FO definitions staged ({fo_path.stat().st_size} bytes)")
1003
+
1004
+ # ADDITIVE track dispatch (STEP-2, 2026-08-04, FRAME/SEGMENT foundation):
1005
+ # SURGFIELD_TRACK selects the input resolver. Default/'procedure' runs the
1006
+ # UNCHANGED video ladder (stage_videos_root + normalize_clips) VERBATIM in
1007
+ # the else — the FRAME/SEGMENT branches are entered ONLY when explicitly
1008
+ # armed, so the PROCEDURE resolved-clips + answer.json are byte-identical
1009
+ # (proven by solution.entrypoint.test_procedure_byte_identity). The env read
1010
+ # is pure (no output side effect); track_inputs is imported LAZILY so a
1011
+ # PROCEDURE run never even imports the FRAME/SEGMENT code.
1012
+ _track = os.environ.get("SURGFIELD_TRACK", "procedure").strip().lower()
1013
+ if _track == "segment":
1014
+ from solution.entrypoint.segment_zip_input import maybe_zip_batch
1015
+ zip_batch = maybe_zip_batch(rows, input_dir, workdir, _ZIP_STAGING_EVENTS,
1016
+ receipt_sink=lambda record: _log("ZIP_MEMBER_RECEIPT " + json.dumps(record, sort_keys=True)))
1017
+ if zip_batch is not None:
1018
+ root, norm_root = None, zip_batch.norm_root
1019
+ _log(f"PROBESHELL: bounded ZIP staging for {len(rows)} questions; one selected clip at a time")
1020
+ elif _track in ("frame", "segment"):
1021
+ from solution.entrypoint.track_inputs import stage_and_normalize_track
1022
+ root, norm_root = stage_and_normalize_track(_track, rows, input_dir, workdir)
1023
+ else:
1024
+ root = stage_videos_root(input_dir, workdir)
1025
+ norm_root = normalize_clips(rows, root, workdir)
1026
+ # Both env writes (SURGFIELD_FO_DEFINITIONS_FILE above, CLIP_INPUT here)
1027
+ # are RUN-SCOPED: reset at the top of this block, set for this run's
1028
+ # duration, read per-call by the engine. In the container nothing
1029
+ # survives the exit; in-process reuse is isolated by the reset.
1030
+ os.environ["SURGFIELD_CLIP_INPUT"] = "1" # per-qID clip semantics engine-side
1031
+ to_fence, n_probed = emit_decode_records(
1032
+ rows, norm_root, probe_budget_sec=PROBE_PHASE_BUDGET_SEC, zip_batch=zip_batch)
1033
+ # H6 fence: a clip whose killable prober hit a WEDGE/CRASH is removed from
1034
+ # the normalized tree so the in-process sampler never touches it; that qID
1035
+ # defers to text/rule like an absent clip. (The in-process decode is ALSO
1036
+ # SIGALRM-bounded now — P1-03 — so a wedge that slips past the probe still
1037
+ # can't hang the job.) NEVER fenced on a clean frame-0 miss.
1038
+ systemic, env_faults, dangers = _fence_decision(to_fence, n_probed)
1039
+ if systemic:
1040
+ _log("PROBESHELL: H6-FENCE ABORT — systemic decode fault "
1041
+ f"(env_faults={sorted(env_faults)[:10]}, "
1042
+ f"dangerous={len(dangers)}/{n_probed} probed); NOT unlinking clips "
1043
+ "(would blind the whole pipeline); relying on the in-process "
1044
+ "SIGALRM decode deadline for containment")
1045
+ elif dangers:
1046
+ _log("PROBESHELL: H6-FENCE unlinking "
1047
+ + ", ".join(f"{q}({r})" for q, r in sorted(dangers.items()))
1048
+ + " — in-process decode fenced; affected qIDs recorded for empty responses")
1049
+ for qid in dangers:
1050
+ _MEDIA_FENCE_EVENTS.append({"qID": qid, "stage": "startup_decoder_fence", "reason": dangers[qid]})
1051
+ link = norm_root / "plain" / f"{qid}.mp4"
1052
+ try:
1053
+ if link.is_symlink() or link.exists():
1054
+ link.unlink()
1055
+ except OSError as error:
1056
+ from solution.entrypoint.segment_zip_input import ZipIntegrityError
1057
+ raise ZipIntegrityError("failed to fence dangerous decoder input") from error
1058
+ _setup_wall = time.perf_counter() - _proc_t0
1059
+ _log(f"PROBESHELL: SETUP-WALL {_setup_wall:.1f}s (arm→staging→decode-proof, "
1060
+ f"before model load); platform setup window ~120s")
1061
+ if _setup_wall > 90.0:
1062
+ _log(f"PROBESHELL: WARN setup wall {_setup_wall:.0f}s is within 30s of "
1063
+ f"the ~120s platform setup window — cold-start overrun risk")
1064
+ answers = answer_all(rows, norm_root, args.device, answers, output_dir,
1065
+ proc_t0=_proc_t0, zip_batch=zip_batch)
1066
+ except Exception as e:
1067
+ _log(f"PROBESHELL DEGRADED: shell/engine failure after arming "
1068
+ f"({type(e).__name__}: {e}); the pre-seeded full-coverage "
1069
+ f"answers stand for {len(rows)} rows")
1070
+ finally:
1071
+ if zip_batch is not None:
1072
+ zip_batch.close()
1073
+
1074
+ try:
1075
+ _c1_collision_preflight(answers, rows) # C1 ENFORCE: DECLINE colliding answers (P9)
1076
+ except Exception as e: # belt: the pre-flight must never crash the run
1077
+ _log(f"C1 pre-flight skipped (non-fatal {type(e).__name__}: {e})")
1078
+
1079
+ # P2-14/P2-18: OUTPUT-HYGIENE self-DQ preflight (F0/DUP/COV/FMT/P1/P2/P3).
1080
+ # OBSERVE-ONLY in-container (like C1): it LOGS a self-DQ risk so it is visible
1081
+ # in the container log; it NEVER mutates or blocks (the never-empty coverage
1082
+ # guarantee stands). The blocking pre-upload gate is the CLI wrapper
1083
+ # `run_preflight_on_file` (exit-nonzero on findings) that Reza runs before an
1084
+ # upload. Coverage is checked against the request qID universe; the FMT check
1085
+ # uses the answer_format inferred/exposed per row. Never fatal here.
1086
+ try:
1087
+ from solution.cap_fmt_inferrer import infer_all
1088
+ from solution.answer_validator import normalize_answer_format
1089
+ from solution.entrypoint.output_hygiene_preflight import run_preflight
1090
+ expected_qids = [str(r.get("qID", "")) for r in rows if r.get("qID", "")]
1091
+ formats_by_qid: dict[str, str] = {}
1092
+ for r in rows:
1093
+ qid = str(r.get("qID", ""))
1094
+ if not qid:
1095
+ continue
1096
+ exposed = r.get("answer_format")
1097
+ try:
1098
+ fmt = (normalize_answer_format(str(exposed)) if exposed
1099
+ else infer_all(str(r.get("question", "")))["answer_format"])
1100
+ except Exception:
1101
+ fmt = None
1102
+ if fmt:
1103
+ formats_by_qid[qid] = fmt
1104
+ rep = run_preflight(answers, expected_qids=expected_qids,
1105
+ formats_by_qid=formats_by_qid)
1106
+ _log(f"PROBESHELL: {rep.summary()}")
1107
+ for f in rep.findings:
1108
+ _log(f" HYGIENE [{f.check}] qID={f.qID}: {f.reason} :: {f.detail}")
1109
+ except Exception as e: # observe-only: never affect output
1110
+ _log(f"output-hygiene pre-flight skipped (non-fatal {type(e).__name__}: {e})")
1111
+
1112
+ out = write_answers(output_dir, answers)
1113
+ _log(f"PROBESHELL: wrote {out} ({len(answers)} answers)")
1114
+ return 0
1115
+
1116
+
1117
+ if __name__ == "__main__":
1118
+ sys.exit(main())
platform_wrapper/segment_zip_input.py ADDED
@@ -0,0 +1,345 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Bounded SEGMENT ZIP staging. The bootstrap validates request/root custody first.
2
+ Only the selected clip is materialized; the source ZIP remains on read-only input.
3
+ """
4
+ from contextlib import contextmanager
5
+ import errno
6
+ import hashlib
7
+ import json
8
+ import os
9
+ from pathlib import Path, PurePosixPath
10
+ import stat
11
+ import zipfile
12
+
13
+ class ZipIntegrityError(RuntimeError):
14
+ """Archive/path/owned-state corruption must remain fatal for the invocation."""
15
+
16
+
17
+ BLOCK = 1024 * 1024
18
+ MEDIA = {".mp4", ".avi", ".mov", ".mkv", ".webm", ".m4v", ".mpg", ".mpeg"}
19
+
20
+
21
+ def stamp(s):
22
+ return tuple(getattr(s, k) for k in ("st_dev", "st_ino", "st_mode", "st_uid", "st_gid", "st_size", "st_mtime_ns", "st_ctime_ns", "st_nlink"))
23
+
24
+
25
+ def validate_members(infos):
26
+ files, directories = {}, set()
27
+ if len(infos) > 100000:
28
+ raise ZipIntegrityError("ZIP member-count bound")
29
+ for info in infos:
30
+ name = info.filename
31
+ trimmed = name[:-1] if info.is_dir() else name
32
+ parts = trimmed.split("/")
33
+ if not trimmed or "\\" in name or any(x in ("", ".", "..") or ":" in x or any(ord(c) < 32 for c in x) for x in parts):
34
+ raise ZipIntegrityError("unsafe ZIP name")
35
+ kind = stat.S_IFMT(info.external_attr >> 16)
36
+ if kind not in (0, stat.S_IFREG, stat.S_IFDIR) or info.flag_bits & 1:
37
+ raise ZipIntegrityError("special or encrypted ZIP member")
38
+ key = trimmed.casefold()
39
+ if info.is_dir():
40
+ if key in files:
41
+ raise ZipIntegrityError("file-directory collision")
42
+ directories.add(key)
43
+ else:
44
+ if key in files or key in directories:
45
+ raise ZipIntegrityError("duplicate or case-colliding ZIP member")
46
+ files[key] = info
47
+ for info in files.values():
48
+ if any(str(a).casefold() in files for a in PurePosixPath(info.filename).parents if str(a) != "."):
49
+ raise ZipIntegrityError("ancestor file collision")
50
+ return {info.filename: info for info in files.values()}
51
+
52
+
53
+ def plan_rows(rows, members):
54
+ """Same strict selection groups as current R3 input validator.
55
+ It never rewrites request clocks, qIDs, question text or videoID.
56
+ """
57
+ media = [PurePosixPath(n) for n in members if PurePosixPath(n).suffix.lower() in MEDIA]
58
+ result, seen = [], set()
59
+ for row in rows:
60
+ qid = row["qID"]
61
+ if (not isinstance(qid, str) or not qid or qid in seen or qid in (".", "..")
62
+ or "/" in qid or "\\" in qid or any(ord(c) < 32 for c in qid)):
63
+ raise ZipIntegrityError("invalid or duplicate request qID")
64
+ seen.add(qid)
65
+ hits = [p for p in media if p.stem == qid]
66
+ groups = [[p for p in hits if p.parent == PurePosixPath(sub)] for sub in ("plain", "overlayed", ".")]
67
+ groups.append(hits)
68
+ groups.append([p for p in media if p.stem == row.get("videoID")])
69
+ index, selected = next(((i, g) for i, g in enumerate(groups) if g), (None, []))
70
+ if len(selected) > 1:
71
+ raise ZipIntegrityError("ambiguous selected media")
72
+ result.append({"qID": qid, "member": str(selected[0]) if selected else None, "videoid_rung": index == 4})
73
+ return result
74
+
75
+
76
+ def _open_exclusive(path):
77
+ return os.fdopen(os.open(path, os.O_CREAT | os.O_EXCL | os.O_WRONLY | os.O_NOFOLLOW, 0o600), "wb")
78
+
79
+
80
+ class MemberStagingError(Exception):
81
+ """Raised only by lease materialization, never by the model consumer."""
82
+ def __init__(self, details):
83
+ self.details = details
84
+ super().__init__(details["message"])
85
+
86
+
87
+ def member_error(path, info, count, cause):
88
+ return MemberStagingError({"stage": "zip_member_staging", "archive": str(path),
89
+ "member": info.filename, "declared_bytes": info.file_size,
90
+ "written_bytes": count, "exception_type": type(cause).__name__,
91
+ "errno": getattr(cause, "errno", None), "message": str(cause)})
92
+
93
+
94
+ class ZipLease:
95
+ def __init__(self, archive_path, root, *, max_working_bytes, reserve_bytes=64 * 1024 * 1024, receipt_sink=None):
96
+ self.path, self.root = Path(archive_path), Path(root)
97
+ self.limit, self.reserve = max_working_bytes, reserve_bytes
98
+ self.receipt_sink = receipt_sink
99
+ if type(self.limit) is not int or type(self.reserve) is not int or self.limit <= self.reserve or self.reserve < 0:
100
+ raise ZipIntegrityError("invalid working budget")
101
+ self.root.mkdir(mode=0o700, parents=False, exist_ok=False)
102
+ self.root_id = (self.root.stat().st_dev, self.root.stat().st_ino)
103
+ self.raw = None
104
+ self.zf = None
105
+ self.active = False
106
+ self.receipts = []
107
+ try:
108
+ fd = os.open(self.path, os.O_RDONLY | os.O_NOFOLLOW)
109
+ self.raw = os.fdopen(fd, "rb")
110
+ s = os.fstat(fd)
111
+ if not stat.S_ISREG(s.st_mode):
112
+ raise ZipIntegrityError("archive is not a regular file")
113
+ self.source = stamp(s)
114
+ self.check_source()
115
+ self.zf = zipfile.ZipFile(self.raw)
116
+ self.members = validate_members(self.zf.infolist())
117
+ except BaseException:
118
+ self.close()
119
+ raise
120
+
121
+ def check_source(self):
122
+ try:
123
+ return self._check_source()
124
+ except OSError as error:
125
+ raise ZipIntegrityError("archive/lease identity cannot be checked") from error
126
+
127
+ def _check_source(self):
128
+ if stamp(os.fstat(self.raw.fileno())) != self.source or stamp(self.path.lstat()) != self.source:
129
+ raise ZipIntegrityError("archive identity changed")
130
+ r = self.root.lstat()
131
+ if not stat.S_ISDIR(r.st_mode) or (r.st_dev, r.st_ino) != self.root_id:
132
+ raise ZipIntegrityError("owned lease directory replaced")
133
+
134
+ @contextmanager
135
+ def member(self, name, *, qid=None, phase=None):
136
+ if self.active:
137
+ raise ZipIntegrityError("only one media lease may be live")
138
+ self.check_source()
139
+ info = self.members[name]
140
+ available = os.statvfs(self.root)
141
+ free_bytes = available.f_bavail * available.f_frsize
142
+ if info.file_size > min(self.limit, free_bytes) - self.reserve:
143
+ cause = OSError(errno.ENOSPC, "member does not fit bounded scratch reserve")
144
+ raise member_error(self.path, info, 0, cause)
145
+ self.active = True
146
+ token = hashlib.sha256(name.encode("utf-8")).hexdigest()
147
+ part, final = self.root / (token + ".part"), self.root / (token + ".mp4")
148
+ owned = {}
149
+ count, digest, record = 0, hashlib.sha256(), None
150
+ try:
151
+ try:
152
+ with _open_exclusive(part) as out:
153
+ s = part.lstat(); owned[part] = (s.st_dev, s.st_ino)
154
+ with self.zf.open(info, "r") as src:
155
+ while True:
156
+ block = src.read(BLOCK)
157
+ if not block:
158
+ break
159
+ if count + len(block) > info.file_size or count + len(block) + self.reserve > self.limit:
160
+ raise OSError(errno.ENOSPC, "stream exceeds member or working budget")
161
+ if out.write(block) != len(block):
162
+ raise OSError(errno.EIO, "short write while staging media")
163
+ digest.update(block)
164
+ count += len(block)
165
+ if count != info.file_size:
166
+ raise zipfile.BadZipFile("member size differs at full EOF")
167
+ self.check_source()
168
+ # Full ZipExtFile EOF validates CRC before the clip becomes visible.
169
+ part.chmod(0o400)
170
+ part.rename(final)
171
+ owned[final] = owned.pop(part)
172
+ name_bytes = name.encode("utf-8")
173
+ record = {"qID": qid, "phase": phase, "stage": "zip_member_staging",
174
+ "member": name if len(name_bytes) <= 1024 else None,
175
+ "member_name_bytes": len(name_bytes), "member_name_sha256": hashlib.sha256(name_bytes).hexdigest(),
176
+ "archive_identity": list(self.source), "bytes": count,
177
+ "sha256": digest.hexdigest(), "crc32": info.CRC, "full_member_eof": True,
178
+ "file_allocated_bytes": final.stat().st_blocks * 512,
179
+ "working_budget_bytes": min(self.limit, free_bytes), "free_bytes_before": free_bytes,
180
+ "reserve_bytes": self.reserve, "cleanup_completed": False}
181
+ self.receipts.append(record)
182
+ except (zipfile.BadZipFile, EOFError, OSError) as cause:
183
+ if isinstance(cause, OSError) and cause.errno not in (errno.ENOSPC, errno.EIO):
184
+ raise ZipIntegrityError("unclassified staging filesystem error") from cause
185
+ raise member_error(self.path, info, count, cause) from cause
186
+ yield final
187
+ finally:
188
+ self.active = False
189
+ for path, identity in owned.items():
190
+ try:
191
+ s = path.lstat()
192
+ except FileNotFoundError:
193
+ continue
194
+ except OSError as error:
195
+ raise ZipIntegrityError("owned media cleanup identity unavailable") from error
196
+ if not stat.S_ISREG(s.st_mode) or (s.st_dev, s.st_ino) != identity:
197
+ raise ZipIntegrityError("refusing cleanup of replaced media path")
198
+ try:
199
+ path.unlink()
200
+ except OSError as error:
201
+ raise ZipIntegrityError("owned media cleanup failed") from error
202
+ self.check_source()
203
+ if record is not None:
204
+ try:
205
+ after = os.statvfs(self.root)
206
+ except OSError as error:
207
+ raise ZipIntegrityError("post-cleanup filesystem observation failed") from error
208
+ record["free_bytes_after_unlink"] = after.f_bavail * after.f_frsize
209
+ record["cleanup_completed"] = True
210
+ record["source_identity_unchanged_after_use"] = True
211
+ if self.receipt_sink is not None:
212
+ self.receipt_sink(dict(record))
213
+
214
+ def close(self):
215
+ if self.active:
216
+ raise ZipIntegrityError("cannot close with active lease")
217
+ try:
218
+ if self.raw is not None and not self.raw.closed and hasattr(self, "source"):
219
+ self.check_source()
220
+ finally:
221
+ if self.zf is not None:
222
+ self.zf.close()
223
+ if self.raw is not None:
224
+ self.raw.close()
225
+
226
+ def __enter__(self):
227
+ return self
228
+
229
+ def __exit__(self, *unused):
230
+ self.close()
231
+
232
+
233
+ class ZipBatch:
234
+ """Invocation-owned union of admitted ZIP and loose paths.
235
+ Only one sequential question is leased at a time. No answer/model code here.
236
+ """
237
+ def __init__(self, rows, input_root, workdir, archives, loose, events, receipt_sink=None):
238
+ self.events, self.failures, self.stores = events, {}, []
239
+ self.active = False
240
+ self.norm_root = Path(workdir) / "norm"
241
+ self.plain = self.norm_root / "plain"
242
+ self.plain.mkdir(parents=True, exist_ok=False)
243
+ self.entries, union, case_names = {}, {}, set()
244
+ try:
245
+ for i, archive in enumerate(archives):
246
+ store = ZipLease(archive, Path(workdir) / ("zip-lease-" + str(i)),
247
+ max_working_bytes=(1 << 63) - 1, receipt_sink=receipt_sink)
248
+ self.stores.append(store)
249
+ for name in store.members:
250
+ if name.casefold() in case_names:
251
+ raise ZipIntegrityError("ambiguous ZIP/loose staging path")
252
+ case_names.add(name.casefold());union[name] = (store, name)
253
+ for file in loose:
254
+ name = file.relative_to(input_root).as_posix()
255
+ if name.casefold() in case_names:
256
+ raise ZipIntegrityError("ambiguous ZIP/loose staging path")
257
+ case_names.add(name.casefold());union[name] = (None, file)
258
+ for name in union:
259
+ if any(str(a).casefold() in case_names for a in PurePosixPath(name).parents if str(a) != "."):
260
+ raise ZipIntegrityError("cross-input ancestor file collision")
261
+ plan = plan_rows(rows, union)
262
+ for item in plan:
263
+ if item["member"] is None:
264
+ link = self.plain / (item["qID"] + ".mp4")
265
+ self.entries[item["qID"]] = (None, None, link, None)
266
+ continue
267
+ store, member = union[item["member"]]
268
+ target = (store.root / (hashlib.sha256(member.encode("utf-8")).hexdigest() + ".mp4")
269
+ if store is not None else Path(member).resolve())
270
+ link = self.plain / (item["qID"] + ".mp4")
271
+ link.symlink_to(target)
272
+ self.entries[item["qID"]] = (store, member, link, target)
273
+ (self.norm_root / "videoid_rung.json").write_text(json.dumps(sorted(x["qID"] for x in plan if x["videoid_rung"])))
274
+ self.plan = plan
275
+ except BaseException:
276
+ self.close()
277
+ raise
278
+
279
+ def cache_key(self, qid):
280
+ store, member, _, _ = self.entries[qid]
281
+ if member is None:
282
+ return ("missing", qid)
283
+ return (str(store.path), member) if store is not None else ("loose", str(member))
284
+
285
+ def failed(self, qid):
286
+ return qid in self.failures
287
+
288
+ def record_failure(self, qid, phase, error):
289
+ if type(error) is not MemberStagingError:
290
+ raise TypeError("only materialization failures may be recorded here")
291
+ event = {"qID": qid, "phase": phase, **error.details, "output_policy": "empty_content"}
292
+ self.events.append(event);self.failures[qid] = event
293
+ return event
294
+
295
+ @contextmanager
296
+ def member(self, qid, *, phase="answer"):
297
+ if self.active:
298
+ raise ZipIntegrityError("only one question media lease may be live")
299
+ self.active = True
300
+ try:
301
+ store, member, link, target = self.entries[qid]
302
+ if target is None:
303
+ raise MemberStagingError({"stage": "input_media_resolution", "archive": None,
304
+ "member": None, "declared_bytes": None, "written_bytes": 0,
305
+ "exception_type": "MissingMedia", "errno": None,
306
+ "message": "no selected media for this request"})
307
+ # A startup H6 fence removes the normalized link. Never resurrect it.
308
+ if not link.is_symlink():
309
+ if link.exists():
310
+ raise ZipIntegrityError("normalized lease link replaced by non-symlink")
311
+ yield link
312
+ return
313
+ if Path(os.readlink(link)) != target:
314
+ raise ZipIntegrityError("normalized lease target changed")
315
+ if store is None:
316
+ yield link
317
+ else:
318
+ with store.member(member, qid=qid, phase=phase):
319
+ yield link
320
+ finally:
321
+ self.active = False
322
+
323
+ def close(self):
324
+ errors = []
325
+ for store in self.stores:
326
+ try:
327
+ store.close()
328
+ except Exception as error:
329
+ errors.append(error)
330
+ if errors:
331
+ raise errors[0]
332
+
333
+
334
+ def maybe_zip_batch(rows, input_root, workdir, events, receipt_sink=None):
335
+ """Return None for loose-only input so its original staging path is untouched."""
336
+ files = sorted((x for x in Path(input_root).rglob("*") if x.is_file()), key=str)
337
+ archives, loose = [], []
338
+ for file in files:
339
+ if file.suffix.lower() in MEDIA:
340
+ loose.append(file)
341
+ elif zipfile.is_zipfile(file):
342
+ archives.append(file)
343
+ if not archives:
344
+ return None
345
+ return ZipBatch(rows, Path(input_root), Path(workdir), archives, loose, events, receipt_sink=receipt_sink)