# Trusted AutoCAD Evaluator Contract

The evaluator is a private service on a licensed Windows/AutoCAD host. It is outside the model desktop and may use trusted AutoCAD automation. The model never receives its URL, credentials, intermediate results, or gold artifacts.

The current source contract is `autocad-2019-r23.0.46-v4` on AutoCAD 2019 `R23.0.46.0.0`. V4 retains v3's true-3D model and all-layout/viewport inspection fields, while pinning 3D render selection to the task-contract layout named `Layout1`. It uses render-configuration hash `6698199549b4aa88b23cb616fdbce7e4ada7b7ae562b612cc82c465e9b2edf53`; deterministic fixtures and all 50 source-bound gold cache entries pass verification. The evaluator must be compiled and checksum-bound into an operator-owned worker image.

The repository contains an undeployed v5/schema-v4 release candidate. Its
`model_space_extents_3d` includes only genuine 3D entities; the new
`model_space_extents_all` retains diagnostic bounds for every model-space
entity, and `model_space_3d_entities` plus `model_space_3d_entity_types` expose
the isolated inventory. It also records paper-space curve paths and
text-bearing multileaders, records the source `INSUNITS`, and normalizes
model-space coordinates and linear measurements into millimetres. Existing
v4/schema-v3 3D bounds must not be treated
as physical part extents because `ACDBVIEWREP*` helpers can pollute them.

One full interactive AutoCAD process is used per DWG because Core Console was not reliable on the pinned AMI. The runner starts from the release-tested template, disables open-time proxy notices, and opens the inspected DWG inside the trusted script. Each launch has a bounded timeout; a stalled process tree is killed and at most two fresh launches are attempted.

## HTTP API

All endpoints require a bearer token and must be reachable only on the private orchestration network.

### `GET /health`

Example response after v4 deployment:

```json
{
  "status": "ok",
  "evaluator_version": "autocad-2019-r23.0.46-v4",
  "autocad_version": "AutoCAD 2019 R23.0.46.0.0"
}
```

### `POST /v1/inspect-render`

Request: `multipart/form-data` with:

- `dwg`: the candidate or gold DWG file.
- `drawing_type`: required benchmark task contract, either `2d` or `3d`.

The HTTP adapter must pass `drawing_type` to the Windows runner as `-DrawingType`. Responses whose metadata does not echo the requested drawing type fail the Python client contract.

Abbreviated v4 response:

```json
{
  "metadata": {
    "schema_version": "3",
    "evaluator_version": "autocad-2019-r23.0.46-v4",
    "autocad_version": "AutoCAD 2019 R23.0.46.0.0",
    "render_config_hash": "...",
    "drawing_type": "3d",
    "source_sha256": "...",
    "render_sha256": "...",
    "valid": true,
    "failure_reason": null,
    "extents": {
      "min_x": 0.0,
      "min_y": 0.0,
      "max_x": 100.0,
      "max_y": 80.0,
      "units": "mm"
    },
    "entity_count": 3,
    "entity_types": {"3DSOLID": 3},
    "model_space_3d_entity_count": 3,
    "model_space_extents_3d": {
      "min_x": 0.0,
      "min_y": 0.0,
      "min_z": 0.0,
      "max_x": 100.0,
      "max_y": 80.0,
      "max_z": 45.0,
      "units": "mm"
    },
    "paper_layouts": [
      {
        "name": "Layout1",
        "tab_order": 1,
        "entity_count": 14,
        "non_viewport_entity_count": 10,
        "floating_viewport_count": 4,
        "dimension_count": 6,
        "text_annotation_count": 2,
        "entity_types": {"DIMENSION": 6, "MTEXT": 2, "VIEWPORT": 4, "HATCH": 2},
        "viewports": [
          {
            "number": 2,
            "center": {"x": 80.0, "y": 55.0},
            "width": 70.0,
            "height": 50.0,
            "view_direction": {"x": 1.0, "y": -1.0, "z": 1.0},
            "view_target": {"x": 50.0, "y": 40.0, "z": 22.5},
            "twist_angle": 0.0,
            "custom_scale": 0.5,
            "perspective_on": false,
            "on": true,
            "locked": true
          }
        ],
        "has_presentation": true
      }
    ],
    "rendered_space": "paper_space",
    "rendered_layout": "Layout1",
    "render_selection_reason": "task_contract_3d_layout1",
    "prohibited_entity_types": [],
    "geometry_paths": [],
    "dimensions": [],
    "text_annotations": []
  },
  "render_png_base64": "..."
}
```

Invalid drawings return valid JSON metadata with `valid=false`, a structured `failure_reason`, no 2D extents, and an empty render. Transport and service failures use non-2xx HTTP responses. An existing but empty 3D `Layout1` is rendered as-is and records `required_3d_layout1_empty`, allowing the judge to penalize the omitted sheet. If `Layout1` itself is missing, the evaluator records `required_3d_layout1_missing` and falls back to Model Space for diagnostic evidence.

## Inspection and render-selection rules

The evaluator must:

1. Copy the uploaded bytes into a fresh, non-model-visible working directory.
2. Enforce input-size and evaluation-time limits.
3. Open the DWG without interactive dialogs.
4. Treat repair prompts, corrupt files, and unresolved dependencies as structured failures.
5. Always inspect Model Space. Record entity counts, XY and XYZ bounds, true-3D entity counts, sampled base-geometry paths, dimension semantics/anchors, and standalone text features.
6. Inspect every paper-space layout. Record its entity inventory, native dimension values/anchors, text content/placement, and every floating viewport's paper rectangle, view direction/target, twist, scale, perspective, on/off, and locked state.
7. Treat AutoCAD's untouched default layout as empty. A layout is populated when it has at least two floating viewports; one viewport plus dimensions/text/two other paper entities; or no viewports plus dimensions/text/four other paper entities.
8. For a `2d` task, render `Model` regardless of default layout tabs.
9. For a `3d` task, render the paper-space layout named `Layout1` regardless of which tab is active and regardless of whether another layout contains more entities. If `Layout1` exists but is empty, render it and record `required_3d_layout1_empty`. If it does not exist, render `Model` and record `required_3d_layout1_missing` for diagnosis.
10. Reject empty Model Space.
11. Reject external references and submissions whose result depends on an xref.
12. Reject raster images, OLE objects, PDF/DWF/DGN underlays, point-cloud underlays, and equivalent embedded-reference shortcuts in either Model or Paper Space.
13. Delete the working directory after producing the response.

The evaluator intentionally does not invalidate a 3D task solely because it has no recognized 3D entity or because `Layout1` is empty or missing. Those are correctness omissions, so the authoritative judge should score the partial artifact instead of converting every such case to a zero-score file-validity failure.

## Pinned render configuration

The evaluator version freezes and hashes:

- White background and monochrome foreground.
- Fixed 1600×1200 pixel target.
- Plot extents, scale-to-fit, centered output, preserved aspect ratio, and zero-degree rotation.
- Fixed lineweight, transparency, and layer visibility policy.
- `PublishToWeb PNG.pc3`, `monochrome.ctb`, fonts, and font substitutions.
- Fixed AutoCAD version/update, locale, template, and units interpretation.
- Drawing-type-aware Model/Paper Space selection described above.
- No automatic reflection, scale correction, or geometry repair.

The v4 source default render hash is `6698199549b4aa88b23cb616fdbce7e4ada7b7ae562b612cc82c465e9b2edf53`; deployment validation must confirm that exact value in produced metadata.

Changing inspection fields, layout-selection logic, or render settings requires a new evaluator version and a rebuilt gold cache. Candidate and gold metadata must have identical evaluator version, render hash, and task drawing type.

## Windows implementation

The evaluator is an AutoCAD .NET command plus PowerShell runners in `sandbox/windows/evaluator/`. `run-evaluation.ps1` accepts `-DrawingType 2d|3d|auto`; benchmark jobs must pass `2d` or `3d`, while `auto` exists only for manual diagnostics. `run-interactive-evaluation.ps1` carries the same value through the scheduled interactive Administrator task.

Batch execution formally uses an SSM job transport with equivalent isolation and timeout semantics. After model finalization, the sandbox invokes `run-interactive-evaluation.ps1` on the same licensed host and supplies request-scoped presigned PUT URLs for `metadata.json` and `render.png`. The local controller retrieves those objects, verifies the typed/checksummed contract, deletes only the generated temporary keys, and scores before requesting instance cleanup. The evaluator command and upload URLs are never exposed to the model process. A third-party DWG parser is not part of the evaluator.