# AutoCAD sandbox cold-start and handoff

Broker v10 source and `autocad-bench-sandbox` define one fail-closed path from
an EC2 instance or benchmark AMI to a model-ready desktop. V10 extends v9's
exactly-once actions to lease/reset/finalization transport, supervises the SSM
tunnel, bounds evidence finalization, and can replace one failed fresh clone
before any model request. The implementation does not perform licensing checks;
licensing is an operator-confirmed image prerequisite.

The repository does not publish a shared worker image. Operators build or
register an image in their own account and label it with the broker version it
contains. See
[`deterministic-sandbox-contract.md`](./deterministic-sandbox-contract.md) for
the image admission requirements.

## Admission state machine

1. **EC2:** launch a new instance or start the supplied stopped instance. Require `running`, then require both EC2 system and instance status checks to be `ok`.
2. **SSM:** require the managed instance to report `PingStatus=Online`. No public inbound rule, RDP, or DCV ingress is opened.
3. **Interactive boot:** require an Administrator interactive session, the DCV console, the scheduled broker task, a 1920x1080 layout, and a real PyAutoGUI pointer prewarm on `WinSta0\\Default`.
4. **Current-boot identity:** accept `ready.json` only when its Windows boot time matches the live boot. Require one startup ID, installed deployment ID/manifest SHA-256, the running broker PID and binary SHA-256, broker v10, input driver v3, a running interactive task, Administrator logged on, and matching broker/DCV Windows session IDs. Every file in the installed deployment manifest is checksum-verified before broker startup.
5. **Bounded identity recovery:** if immutable admission fails, capture startup JSON, task/process state, and the tail of broker stderr without copying the token. Stop the scheduled task, kill complete broker/Python/AutoCAD process trees in at most three passes, clear pre-handoff state, restart and verify the interactive task, and run identity admission once more. A second failure rejects the VM; there is no unbounded polling or repeated reboot loop.
6. **Tunnel:** retrieve the current token through SSM without printing it and open a supervised SSM loopback port forward. Startup allows three fresh port-forward processes; each must make authenticated `/health` identify the exact admitted broker PID/session, input driver, and 1920×1080 display. After startup, the watchdog retains one local port, staggers controller-wide reconnects, and keeps restoring the tunnel until the rollout ends. A replacement counts as recovered only after authenticated broker health traverses it. Broker operations retain one stable idempotency key and retry within their recovery deadline so they do not fail while the watchdog is still reconnecting.
7. **Exact task handoff:** before starting the model child, lease the real task/attempt and request its sequence-zero reset. Each reset already permits two clean AutoCAD launches for main-window/ribbon/modal/stability recovery; the wrapper may replay the still-idempotent reset once more. A failed task reset therefore rejects or replaces the VM before model/API usage. The pilot later replays the same lease/reset IDs and receives the cached screenshot without another AutoCAD launch.
8. **Agent handoff:** only after the exact reset succeeds does the wrapper emit `sandbox_handoff_ready`, start the pilot, and emit `sandbox_agent_started`. Model wall time starts from that cached sequence-zero screenshot. After this point the harness does not force AutoCAD foreground/maximized state; native Save As/File Explorer dialogs remain operable and focus recovery belongs to the visual model.
9. **Immediate evaluation:** after the pilot checkpoints, retrieves, and records its final DWG, the wrapper closes the model tunnel but keeps the licensed instance alive. A trusted interactive AutoCAD scheduled task boots from the known-good template, opens the exact saved attempt inside its script, and uploads metadata plus any valid render through short-lived presigned URLs. A stalled launch is killed as a complete process tree and retried at most twice. Local deterministic scoring and the vision judge run immediately; only a terminal `completed` or `failed` evaluation status permits sandbox cleanup.

## Failure policy

- Runtime startup avoids the old disposable-reset/real-reset duplication. Image-release validation may still invoke `verify-boot-readiness.ps1 -RunResetSmoke`, but ordinary rollouts go directly from immutable admission to the exact task reset.
- Known transient ribbon/AutoCAD handoff failures receive one broker relaunch, then one wrapper-level reset replay. If both reset calls fail, a newly launched clone is replaced once by default.
- A newly launched clone that still fails admission is terminated and replaced once by default. This happens before model/API usage. Existing instances are never replaced, and a rollout is never restarted after agent handoff.
- Unknown popups, stolen foreground, stale readiness, wrong display/session/version, malformed screenshots, exhausted tunnel startup, or exhausted exact-reset retries stop before model/API usage.
- Diagnostics are stage-oriented and exclude typed model text, API keys, and broker tokens.
- During a rollout, broker operations use a bounded deadline with a 15-minute transport-recovery grace and replay the same idempotent request across tunnel recovery. Long traces stream frames, turns, and receipts to disk without retaining duplicate screenshot trajectories in RAM. A child rollout failure does not trigger another sandbox admission inside the same attempt. A first operator interrupt is allowed a bounded 90-second child grace so the pilot can checkpoint/retrieve/write its trace; then the configured EC2 cleanup action is issued. A second interrupt escapes immediately.
- Automatic evaluation is one terminal stage of the same sample, not a second rollout. It never sends gold data to the model desktop. Evaluation failure changes a zero-exit pilot to a nonzero sandbox result, while preserving the DWG, traces, partial scores, and explicit error for inspection.
- Existing instances default to `keep`; instances created with `--launch-image-id` default to `terminate`. `stop`, `force-stop`, and `terminate` issue one AWS request and do not poll afterward.

## Commands

Prepare an existing stopped/running instance and run a pilot:

```bash
uv run autocad-bench-sandbox \
  --instance-id i-... \
  --cleanup force-stop \
  --evaluation-bucket your-autocad-bench-bucket \
  --session-manager-plugin /absolute/path/to/session-manager-plugin \
  -- uv run python -m autocad_bench.harness.run \
    --task-id task-001 \
    --output-dir runs/task-001
```

Launch from an admitted AMI and terminate afterward:

```bash
uv run autocad-bench-sandbox \
  --launch-image-id ami-... \
  --subnet-id subnet-... \
  --security-group-id sg-... \
  --instance-profile-name AutoCADBenchWorkerProfile \
  --instance-type g4dn.xlarge \
  --region us-east-1 \
  --evaluation-bucket your-autocad-bench-bucket \
  --session-manager-plugin /absolute/path/to/session-manager-plugin \
  -- uv run python -m autocad_bench.harness.run \
    --task-id task-001 \
    --output-dir runs/task-001
```

With no public-IP flag, the wrapper inherits the subnet's
`MapPublicIpOnLaunch` policy. A private subnet must have NAT or the required
Systems Manager endpoints; use `--no-associate-public-ip` to force that mode.
`--associate-public-ip` is appropriate for an isolated validation subnet with
an Internet Gateway and a security group with no inbound rules.

## Deployment gate

Before using an operator-owned image in an evaluation:

1. Compile and transactionally deploy the broker to a clean builder.
2. Run a live reset that naturally or deliberately exercises the complete-handoff retry.
3. Run a visible draw/save/retrieve/evaluate smoke.
4. Clean the builder and create an incremental encrypted AMI.
5. Launch an untouched clone through `autocad-bench-sandbox` and require admission.
6. Send one action request twice and require the same action index/result, one broker action, and no duplicate frame-event row.
7. Reboot, pass cold readiness/reset again, clean runtime state, and create the incremental encrypted AMI.

V10 requires a new incremental AMI and the expanded replay, process-tree,
tunnel-restart, post-action-animation, artifact, reboot, and soak gates in
[`incremental-ami-v10-runbook.md`](./incremental-ami-v10-runbook.md). None of
those live v10 gates is claimed complete yet.