Deploy full OpenEnv API instead of starter app

.dockerignore

@@ -0,0 +1,10 @@
+__pycache__/
+*.pyc
+*.pyo
+*.pyd
+.pytest_cache/
+.git/
+.gitignore
+tests/
+.env
+.env.*

.gitignore

@@ -0,0 +1,9 @@
+.venv/
+__pycache__/
+.pytest_cache/
+*.py[cod]
+*.pyo
+*.pyd
+*.egg-info/
+.coverage
+.DS_Store

Dockerfile

@@ -1,16 +1,24 @@
-# Read the doc: https://huggingface.co/docs/hub/spaces-sdks-docker
-# you will also find guides on how best to write your Dockerfile
+FROM python:3.10-slim
 
-FROM python:3.9
-
-RUN useradd -m -u 1000 user
-USER user
-ENV PATH="/home/user/.local/bin:$PATH"
+ENV PYTHONDONTWRITEBYTECODE=1 \
+	PYTHONUNBUFFERED=1 \
+	PIP_NO_CACHE_DIR=1 \
+	PIP_DISABLE_PIP_VERSION_CHECK=1 \
+	PORT=7860
 
 WORKDIR /app
 
-COPY --chown=user ./requirements.txt requirements.txt
-RUN pip install --no-cache-dir --upgrade -r requirements.txt
+RUN adduser --disabled-password --gecos "" appuser
+
+COPY requirements.txt /app/requirements.txt
+RUN python -m pip install --upgrade pip && \
+	pip install -r /app/requirements.txt
+
+COPY . /app
+RUN chown -R appuser:appuser /app
+
+USER appuser
+
+EXPOSE 7860
 
-COPY --chown=user . /app
-CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "7860"]
+CMD ["sh", "-c", "uvicorn app.main:app --host 0.0.0.0 --port ${PORT:-7860}"]

README.md

@@ -8,4 +8,396 @@ app_port: 7860
 pinned: false
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Last-Mile Delivery Optimization (OpenEnv)
+
+OpenEnv-compatible benchmark environment for city-scale last-mile dispatch under realistic operating constraints.
+
+## Motivation
+
+Urban delivery systems face a difficult planning problem: dispatchers must complete as many deliveries as possible while navigating congestion, road blockages, and battery limits. Poor routing or action sequencing creates delays, SLA violations for high-priority orders, and unnecessary energy cost.
+
+This environment models that real-world challenge in a deterministic grid world so policies can be compared fairly across difficulty levels.
+
+## What This Environment Simulates
+
+- Multi-order pickup and delivery workflow.
+- Static obstacles, dynamic obstacles, and traffic penalties.
+- Priority-aware delivery pressure (especially in hard mode).
+- Battery constraints and charging behavior in hard mode.
+- Deterministic seeded task generation for reproducible evaluation.
+
+## Architecture Diagram
+
+```mermaid
+flowchart LR
+	U[OpenEnv Evaluator or User] -->|HTTP| API[FastAPI API Layer\napp/main.py + app/routes.py]
+
+	API -->|reset or step or state| ENV[LastMileDeliveryEnvironment\nenv/environment.py]
+	ENV --> SIM[DeliverySimulator\nenv/simulator.py]
+	SIM -->|observation, reward, done, info| API
+
+	API -->|baseline endpoint rollout| BASE[BaselineGreedyAgent\nbaseline/baseline_agent.py]
+	BASE --> ENV
+
+	API -->|grader endpoint| GRADER[DeliveryEpisodeGrader\ngrader/grader.py]
+	GRADER --> REPORT[GradeReport\nscore + metrics + safeguards]
+
+	TASKS[tasks/easy.py + tasks/medium.py + tasks/hard.py\n+ tasks/registry.py] -->|task config| ENV
+	TASKS -->|success_condition| GRADER
+	TASKS -->|metadata| API
+```
+
+Flow summary:
+
+- The API orchestrates environment state transitions and stores trajectory steps.
+- The baseline endpoint runs a fresh rollout with the built-in greedy agent.
+- The grader endpoint scores the recorded trajectory against task success conditions.
+- Task definitions provide deterministic configuration and grading targets.
+
+## Action Space
+
+The action payload is a strict JSON object with exactly four fields.
+
+| Field | Type | Required | Allowed Values | Notes |
+|---|---|---|---|---|
+| move | string or null | Yes | up, down, left, right, stay, null | Movement action; stay maps to no movement |
+| accept_order | integer or null | Yes | order index or null | When set to n, maps to internal order_n |
+| deliver_order | boolean | Yes | true or false | Deliver current order when true |
+| wait | boolean | Yes | true or false | Explicit wait action when true |
+
+Validation is strict. Exactly one action intent must be active per step.
+
+## Observation Space
+
+Each step returns an observation object with agent, order, and map state.
+
+| Field | Type | Description |
+|---|---|---|
+| grid_width, grid_height | integer | Map dimensions |
+| agent_location | object {x, y} | Current agent position |
+| pending_orders | array of Order | Orders not yet delivered |
+| current_order | Order or null | Active accepted/picked-up order |
+| obstacles | array of Position | Static + dynamic blocked cells |
+| dynamic_obstacles | array of Position | Current dynamic blocked cells |
+| traffic_zones | array of TrafficZone | Cells with extra movement cost |
+| charging_stations | array of Position | Recharge cells |
+| battery_level | integer or null | Active in battery-enabled tasks |
+| step_count | integer | Elapsed step count |
+| total_reward | float | Cumulative episode reward |
+
+## Tasks
+
+All tasks are deterministic and expose explicit success conditions.
+
+### Easy
+
+- Description: simple single-order delivery on a compact map.
+- Configuration: 6x6 grid, 1 order, max_steps=60.
+- Constraints: no obstacles, no traffic, no battery, no priority mix.
+- Success condition: completion_rate=1.0, max_steps=60, invalid_action_rate_max=0.12.
+
+### Medium
+
+- Description: multi-order delivery with static obstacles and traffic friction.
+- Configuration: 10x10 grid, 3 to 4 orders, max_steps=140.
+- Constraints: static obstacles + traffic enabled.
+- Success condition: completion_rate_min=0.85, max_steps=140, invalid_action_rate_max=0.08.
+
+### Hard
+
+- Description: high-pressure dispatch with tight battery and SLA constraints.
+- Configuration: 16x16 grid, 6 to 8 orders, max_steps=165.
+- Constraints: static + dynamic obstacles, traffic, high-priority orders, multi-stop delivery, battery + recharge, delay pressure.
+- Success condition: completion_rate_min=0.90, high_priority_on_time_rate_min=0.88, battery_depletion=false, max_steps=165, invalid_action_rate_max=0.04.
+
+## Reward Design
+
+The reward is dense and combines task completion signals with operational efficiency:
+
+- Base step penalty: -1.0 each step.
+- Delivery reward: +50.0 on successful delivery.
+- Destination milestone: +10.0 when pickup/drop destination is reached.
+- Invalid action penalty: -20.0.
+- Traffic movement penalty: -2.0 when entering traffic cells.
+- Progress shaping: signed distance-based reward clipped by configured bounds.
+- Priority delivery bonus: +15.0 (high) or +5.0 (low).
+- Recharge reward: +2.0 when battery is restored at charging stations.
+- Delay penalty: scaled negative penalty for overdue active orders.
+- Battery failure penalty: -30.0 when battery depletes to zero.
+
+Design goal: favor correct and efficient order handling while discouraging invalid and wasteful behavior.
+
+## Grader Logic
+
+The grader is deterministic and aligned to each task's success_condition fields.
+
+Main metrics:
+
+- completion_rate: delivered_orders / total_orders.
+- high_priority_on_time_rate: on-time high-priority deliveries.
+- efficiency_ratio: normalized by max_steps target.
+- invalid_action_rate: invalid_actions / steps_taken.
+
+Scoring components:
+
+- Completion component: highest weight.
+- Priority SLA component: enabled when high_priority_on_time_rate_min exists.
+- Efficiency component: step-budget performance.
+- Penalty component: reduces score when invalid action rate exceeds target.
+
+Edge-case safeguards include:
+
+- Zero-step and zero-order episode handling.
+- Invalid success-condition value fallback and clamping.
+- Division-safe invalid-action and efficiency calculations.
+- Optional battery depletion penalty when the task requires no depletion.
+
+## Setup
+
+### Local
+
+1. Create and activate a virtual environment.
+2. Install dependencies.
+3. Run the API server on port 7860.
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+pip install -r requirements.txt
+uvicorn app.main:app --host 0.0.0.0 --port 7860
+```
+
+### Docker
+
+```bash
+docker build -t delivery-openenv:latest .
+docker run --rm -p 7860:7860 \
+	-e HF_TOKEN="example-provider-key" \
+	-e MODEL_NAME="gpt-4o-mini" \
+	-e API_BASE_URL="https://api.openai.com/v1" \
+	-e OPENAI_USE_MODEL="1" \
+	delivery-openenv:latest
+```
+
+## API Usage
+
+Base URL:
+
+```text
+http://0.0.0.0:7860
+```
+
+### Health
+
+```bash
+curl -s http://0.0.0.0:7860/health
+```
+
+### Frontend Results Dashboard
+
+Open the interactive dashboard in your browser:
+
+```bash
+open http://0.0.0.0:7860/ui
+```
+
+The dashboard lets you run `reset`, `step`, `state`, `grader`, and `baseline` calls and inspect live payloads and metrics.
+
+### List Tasks and Action Schema
+
+```bash
+curl -s http://0.0.0.0:7860/tasks
+```
+
+### Reset Environment
+
+```bash
+curl -s -X POST http://0.0.0.0:7860/reset \
+	-H "Content-Type: application/json" \
+	-d '{"task":"easy","seed":101}'
+```
+
+### Reset From Real-World Scenario
+
+Use this endpoint when you want to inject externally curated map and order state instead of task-generated layouts.
+
+```bash
+curl -s -X POST http://0.0.0.0:7860/reset_from_scenario \
+	-H "Content-Type: application/json" \
+	-d '{
+	  "seed": 99,
+	  "success_condition": {
+	    "completion_rate_min": 1.0,
+	    "max_steps": 20,
+	    "invalid_action_rate_max": 0.20,
+	    "battery_depletion": false
+	  },
+	  "scenario": {
+	    "width": 6,
+	    "height": 6,
+	    "max_steps": 20,
+	    "agent_start": {"x": 0, "y": 0},
+	    "orders": [
+	      {
+	        "order_id": "r1",
+	        "pickup": {"x": 1, "y": 0},
+	        "dropoff": {"x": 2, "y": 0},
+	        "delivery_locations": [{"x": 2, "y": 0}],
+	        "priority": "high",
+	        "created_step": 0
+	      }
+	    ],
+	    "obstacles": [{"x": 4, "y": 4}],
+	    "dynamic_obstacles": [{"x": 4, "y": 5}],
+	    "traffic_zones": [{"location": {"x": 3, "y": 0}, "extra_cost": 3}],
+	    "charging_stations": [{"x": 0, "y": 0}],
+	    "battery_profile": {
+	      "enabled": true,
+	      "capacity": 10,
+	      "recharge_rate": 4,
+	      "initial_level": 7
+	    }
+	  }
+	}'
+```
+
+Contract summary for `scenario`:
+
+- `width`, `height`, `max_steps`: grid and episode budget.
+- `agent_start`: starting position.
+- `orders`: list of orders with pickup/dropoff/delivery locations and priority.
+- `obstacles`, `dynamic_obstacles`: blocked cells.
+- `traffic_zones`: movement-cost cells (`extra_cost >= 1`).
+- `charging_stations`: recharge cells.
+- `battery_profile`: battery enablement, capacity, recharge rate, and optional initial level.
+- `success_condition` (top-level): optional custom grader thresholds for this scenario.
+
+### Step
+
+```bash
+curl -s -X POST http://0.0.0.0:7860/step \
+	-H "Content-Type: application/json" \
+	-d '{"move":null,"accept_order":null,"deliver_order":false,"wait":true}'
+```
+
+### State
+
+```bash
+curl -s http://0.0.0.0:7860/state
+```
+
+### Grade Current Episode
+
+```bash
+curl -s http://0.0.0.0:7860/grader
+```
+
+### Baseline Rollout Evaluation
+
+```bash
+curl -s "http://0.0.0.0:7860/baseline?task=easy"
+```
+
+## Baseline Scores (Reproducible Inference Run)
+
+Measured with `inference.py --all-tasks --seed 42 --json-summary` using deterministic fallback mode (`OPENAI_USE_MODEL=0`).
+
+| Task | Steps | Success | Score |
+|---|---:|---:|---:|
+| easy | 6 | true | 0.9700 |
+| medium | 140 | false | 0.0001 |
+| hard | 55 | false | 0.0667 |
+
+Average score (seed=42): **0.3456**
+
+## Real-World Validation
+
+We tested the environment under realistic scenarios:
+
+- Priority-based delivery selection.
+- Traffic-aware routing decisions.
+- Battery-constrained navigation.
+
+Results indicate that evaluated agents can exhibit behavior aligned with real-world logistics systems under these constraints.
+
+## OpenEnv Manifest
+
+Project metadata and runtime wiring are defined in openenv.yaml, including:
+
+- runtime entrypoints,
+- environment class,
+- task registry loader,
+- grader class,
+- action and observation schema declarations.
+
+## Baseline Inference Script
+
+The baseline runner uses the OpenAI Python client and supports the required submission variables:
+
+- `HF_TOKEN`: API key/token used as OpenAI client `api_key`.
+- `MODEL_NAME`: model identifier used for inference requests.
+- `API_BASE_URL`: optional OpenAI-compatible endpoint.
+
+Backward-compatible aliases are also supported: `OPENAI_API_KEY`, `OPENAI_MODEL`, and `OPENAI_BASE_URL`.
+
+```bash
+export HF_TOKEN="your-api-key"
+export MODEL_NAME="gpt-4o-mini"
+export API_BASE_URL="https://api.openai.com/v1"
+export OPENAI_USE_MODEL="1"
+
+# Reproducible baseline score across easy/medium/hard tasks.
+python inference.py --all-tasks --seed 42 --json-summary
+```
+
+Notes:
+
+- `OPENAI_USE_MODEL=1` enables remote model calls.
+- `OPENAI_USE_MODEL=0` keeps deterministic heuristic fallback while still validating OpenAI client wiring.
+- `--seed` controls reproducibility for baseline comparisons.
+
+## Trainable Agent (Q-Learning)
+
+This repository now includes a trainable tabular Q-learning agent for interactive learning on the OpenEnv loop.
+
+Training script:
+
+```bash
+python scripts/train_q_agent.py --task easy --episodes 2000 --eval-episodes 200 --seed 42 --output models/q_agent_easy.json
+```
+
+What it does:
+
+- Trains through repeated `reset()` / `step()` episodes.
+- Learns action values for a compact state representation.
+- Evaluates the learned policy and prints success/completion metrics.
+- Saves a reusable model artifact (`models/q_agent_easy.json`).
+
+Learner runtime components:
+
+- Agent implementation: `baseline/trained_q_agent.py`
+- Trainer entrypoint: `scripts/train_q_agent.py`
+
+## Validation Commands
+
+```bash
+./scripts/validate.sh
+.venv/bin/openenv validate
+python -m pytest -q
+```
+
+## Test Command
+
+```bash
+python -m pytest -q
+```
+
+## Hugging Face Spaces (Docker)
+
+1. Create a Docker Space.
+2. Push this repository.
+3. Set variables/secrets: HF_TOKEN, MODEL_NAME, API_BASE_URL, OPENAI_USE_MODEL.
+4. Ensure exposed port is 7860.
+5. Rebuild and verify /health.
+
+Additional deployment notes are available in HF_SPACES_DEPLOYMENT.md.

app.py

@@ -1,8 +0,0 @@
-from fastapi import FastAPI
-
-app = FastAPI()
-
-
-@app.get("/")
-def greet_json():
-    return {"Hello": "World!"}

app/main.py

@@ -0,0 +1,37 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+from fastapi import FastAPI
+from fastapi.middleware.cors import CORSMiddleware
+from fastapi.responses import RedirectResponse
+from fastapi.staticfiles import StaticFiles
+
+from app.routes import router
+
+
+app = FastAPI(
+	title="Last-Mile Delivery Optimization Environment",
+	version="1.0.0",
+	description="OpenEnv-compatible environment for grid-based last-mile delivery optimization.",
+)
+
+app.add_middleware(
+	CORSMiddleware,
+	allow_origins=["*"],
+	allow_credentials=False,
+	allow_methods=["*"],
+	allow_headers=["*"],
+)
+
+STATIC_DIR = Path(__file__).resolve().parent / "static"
+
+app.mount("/static", StaticFiles(directory=str(STATIC_DIR)), name="static")
+
+
+@app.get("/ui", include_in_schema=False)
+def results_ui() -> RedirectResponse:
+	return RedirectResponse(url="/static/results.html")
+
+
+app.include_router(router)

app/routes.py

@@ -0,0 +1,396 @@
+from __future__ import annotations
+
+from threading import RLock
+from typing import Any, Dict, List, Optional
+
+from fastapi import APIRouter, Body, HTTPException
+from pydantic import BaseModel, Field, ValidationError
+
+from baseline.baseline_agent import BaselineGreedyAgent
+from env.environment import LastMileDeliveryEnvironment
+from env.models import Action, EnvironmentConfig, EnvironmentState, Observation, ScenarioDefinition, StepResult
+from grader.grader import DeliveryEpisodeGrader, GradeReport
+from tasks.registry import get_all_task_metadata, get_task_config, get_task_definition
+
+
+router = APIRouter()
+
+
+class ResetRequest(BaseModel):
+	task: str = "easy"
+	seed: int | None = None
+
+
+class ScenarioResetRequest(BaseModel):
+	seed: int | None = None
+	success_condition: Dict[str, Any] = Field(default_factory=dict)
+	scenario: ScenarioDefinition
+
+
+class GraderResponse(BaseModel):
+	task: str | None
+	steps_recorded: int
+	report: GradeReport
+
+
+class BaselineResponse(BaseModel):
+	task: str
+	seed: int | None
+	agent: str
+	steps_executed: int = Field(..., ge=0)
+	done: bool
+	score: float = Field(..., ge=0.0, le=1.0)
+	report: GradeReport
+
+
+class TaskDescriptor(BaseModel):
+	name: str
+	description: str
+	difficulty: int
+	parameters: Dict[str, Any]
+	success_condition: Dict[str, Any]
+
+
+class TasksResponse(BaseModel):
+	tasks: List[TaskDescriptor]
+	action_schema: Dict[str, Any]
+
+
+TASKS_ACTION_SCHEMA: Dict[str, Any] = {
+	"type": "object",
+	"required": ["move", "accept_order", "deliver_order", "wait"],
+	"additionalProperties": False,
+	"properties": {
+		"move": {
+			"type": ["string", "null"],
+			"enum": ["up", "down", "left", "right", "stay", None],
+		},
+		"accept_order": {
+			"type": ["integer", "null"],
+		},
+		"deliver_order": {
+			"type": "boolean",
+		},
+		"wait": {
+			"type": "boolean",
+		},
+	},
+}
+
+
+class EnvironmentService:
+	BASELINE_MAX_STEPS = 128
+
+	def __init__(self) -> None:
+		self._lock = RLock()
+		self._env: Optional[LastMileDeliveryEnvironment] = None
+		self._task: Optional[str] = None
+		self._success_condition: Dict[str, Any] = {}
+		self._trajectory: list[StepResult] = []
+		self._grader = DeliveryEpisodeGrader()
+		self._baseline_agent = BaselineGreedyAgent()
+
+	def reset(self, task: str, seed: int | None) -> Observation:
+		with self._lock:
+			config = get_task_config(task_name=task, seed=seed)
+			task_definition = get_task_definition(task)
+			self._env = LastMileDeliveryEnvironment(config=config)
+			self._task = task
+			self._success_condition = dict(task_definition.get("success_condition", {}))
+			self._trajectory = []
+			return self._env.reset(seed=seed)
+
+	def reset_from_scenario(
+		self,
+		scenario: ScenarioDefinition,
+		seed: int | None,
+		success_condition: Dict[str, Any] | None,
+	) -> Observation:
+		with self._lock:
+			config = EnvironmentConfig(
+				width=scenario.width,
+				height=scenario.height,
+				max_steps=scenario.max_steps,
+				max_orders=max(1, len(scenario.orders)),
+				obstacle_density=0.0,
+				traffic_density=0.0,
+				dynamic_obstacles_enabled=bool(scenario.dynamic_obstacles),
+				dynamic_obstacle_ratio=1.0 if scenario.dynamic_obstacles else 0.0,
+				battery_enabled=scenario.battery_profile.enabled,
+				battery_capacity=scenario.battery_profile.capacity,
+				battery_recharge_rate=scenario.battery_profile.recharge_rate,
+				charging_stations=[item.model_copy(deep=True) for item in scenario.charging_stations],
+				seed=seed,
+			)
+			self._env = LastMileDeliveryEnvironment(config=config)
+			self._task = "scenario"
+			self._success_condition = dict(success_condition or {})
+			self._trajectory = []
+			return self._env.reset_with_scenario(scenario=scenario, seed=seed)
+
+	def step(self, action: Action) -> StepResult:
+		with self._lock:
+			if self._env is None:
+				raise RuntimeError("Environment not initialized. Call /reset first.")
+			result = self._env.step_result(action)
+			self._trajectory.append(result)
+			return result
+
+	def state(self) -> EnvironmentState:
+		with self._lock:
+			if self._env is None:
+				return EnvironmentState(
+					initialized=False,
+					done=False,
+					total_reward=0.0,
+					step_count=0,
+					delivered_orders=0,
+					current_order_id=None,
+					observation=None,
+				)
+			return self._env.state()
+
+	def current_task(self) -> Optional[str]:
+		with self._lock:
+			return self._task
+
+	def grade(self) -> GradeReport:
+		with self._lock:
+			if self._env is None:
+				raise RuntimeError("Environment not initialized. Call /reset first.")
+			final_observation = self._env.current_observation()
+			success_condition = dict(self._success_condition)
+			if not success_condition:
+				resolved_task = self._task or "easy"
+				try:
+					task_definition = get_task_definition(resolved_task)
+					success_condition = dict(task_definition.get("success_condition", {}))
+				except ValueError:
+					success_condition = {}
+			return self._grader.grade_episode(
+				self._trajectory,
+				final_observation,
+				success_condition=success_condition,
+			)
+
+	def baseline_rollout(
+		self,
+		task: str | None = None,
+		seed: int | None = None,
+		max_steps: int | None = None,
+	) -> BaselineResponse:
+		with self._lock:
+			resolved_task = task or self._task or "easy"
+			config = get_task_config(task_name=resolved_task, seed=seed)
+			rollout_env = LastMileDeliveryEnvironment(config=config)
+			observation = rollout_env.reset()
+
+			step_cap = config.max_steps
+			if max_steps is not None:
+				step_cap = min(step_cap, max_steps)
+			step_cap = max(1, min(step_cap, self.BASELINE_MAX_STEPS))
+
+			trajectory: list[StepResult] = []
+			for _ in range(step_cap):
+				action = self._baseline_agent.act(observation)
+				result = rollout_env.step_result(action)
+				trajectory.append(result)
+				observation = result.observation
+				if result.done:
+					break
+
+			task_definition = get_task_definition(resolved_task)
+			success_condition = task_definition.get("success_condition", {})
+			report = self._grader.grade_episode(
+				trajectory,
+				rollout_env.current_observation(),
+				success_condition=success_condition,
+			)
+			return BaselineResponse(
+				task=resolved_task,
+				seed=seed,
+				agent="BaselineGreedyAgent",
+				steps_executed=len(trajectory),
+				done=rollout_env.state().done,
+				score=report.score,
+				report=report,
+			)
+
+	def steps_recorded(self) -> int:
+		with self._lock:
+			return len(self._trajectory)
+
+
+def _http_error(status_code: int, message: str) -> HTTPException:
+	return HTTPException(
+		status_code=status_code,
+		detail={
+			"error": "bad_request" if status_code == 400 else "internal_server_error",
+			"message": message,
+		},
+	)
+
+
+def _parse_step_action(payload: Any) -> Action:
+	if payload is None:
+		raise ValueError("Missing step payload")
+
+	if not isinstance(payload, dict):
+		raise ValueError("Step payload must be a JSON object")
+
+	allowed_keys = {"move", "accept_order", "deliver_order", "wait"}
+	unexpected_keys = sorted(set(payload.keys()) - allowed_keys)
+	if unexpected_keys:
+		raise ValueError(f"Unexpected action fields: {', '.join(unexpected_keys)}")
+
+	missing_keys = sorted(allowed_keys - set(payload.keys()))
+	if missing_keys:
+		raise ValueError(f"Missing action fields: {', '.join(missing_keys)}")
+
+	try:
+		return Action.model_validate(payload)
+	except ValidationError as exc:
+		raise ValueError("Invalid action payload") from exc
+
+
+def _parse_scenario_reset_request(payload: Any) -> ScenarioResetRequest:
+	if payload is None:
+		raise ValueError("Missing scenario reset payload")
+
+	if not isinstance(payload, dict):
+		raise ValueError("Scenario reset payload must be a JSON object")
+
+	try:
+		return ScenarioResetRequest.model_validate(payload)
+	except ValidationError as exc:
+		raise ValueError("Invalid scenario payload") from exc
+
+
+def _normalize_task_metadata(raw_tasks: List[Dict[str, Any]]) -> List[TaskDescriptor]:
+	normalized: List[TaskDescriptor] = []
+	for item in raw_tasks:
+		missing = [
+			field
+			for field in ("name", "description", "difficulty", "parameters", "success_condition")
+			if field not in item
+		]
+		if missing:
+			raise ValueError(f"Task metadata missing required fields: {', '.join(missing)}")
+
+		normalized.append(
+			TaskDescriptor(
+				name=str(item["name"]),
+				description=str(item["description"]),
+				difficulty=int(item["difficulty"]),
+				parameters=dict(item["parameters"]),
+				success_condition=dict(item["success_condition"]),
+			)
+		)
+
+	return normalized
+
+
+env_service = EnvironmentService()
+
+
+@router.get("/")
+def root() -> dict:
+	return {
+		"name": "last-mile-delivery-optimization",
+		"status": "ok",
+		"health": "/health",
+		"docs": "/docs",
+	}
+
+
+@router.get("/health")
+def health() -> dict:
+	return {"status": "ok"}
+
+
+@router.get("/tasks", response_model=TasksResponse)
+def list_tasks() -> TasksResponse:
+	try:
+		raw_tasks = get_all_task_metadata()
+		tasks = _normalize_task_metadata(raw_tasks)
+		return TasksResponse(tasks=tasks, action_schema=TASKS_ACTION_SCHEMA)
+	except ValueError as exc:
+		raise _http_error(500, str(exc)) from exc
+	except Exception as exc:
+		raise _http_error(500, "Failed to retrieve task metadata") from exc
+
+
+@router.post("/reset", response_model=Observation)
+def reset_environment(request: ResetRequest | None = Body(default=None)) -> Observation:
+	try:
+		payload = request or ResetRequest()
+		return env_service.reset(task=payload.task, seed=payload.seed)
+	except ValueError as exc:
+		raise _http_error(400, str(exc)) from exc
+	except Exception as exc:
+		raise _http_error(500, "Failed to reset environment") from exc
+
+
+@router.post("/reset_from_scenario", response_model=Observation)
+def reset_environment_from_scenario(payload: Any = Body(default=None)) -> Observation:
+	try:
+		request = _parse_scenario_reset_request(payload)
+		return env_service.reset_from_scenario(
+			scenario=request.scenario,
+			seed=request.seed,
+			success_condition=request.success_condition,
+		)
+	except ValueError as exc:
+		raise _http_error(400, str(exc)) from exc
+	except Exception as exc:
+		raise _http_error(500, "Failed to reset environment from scenario") from exc
+
+
+@router.post("/step", response_model=StepResult)
+def environment_step(payload: Any = Body(default=None)) -> StepResult:
+	try:
+		action = _parse_step_action(payload)
+		return env_service.step(action=action)
+	except ValueError as exc:
+		raise _http_error(400, str(exc)) from exc
+	except RuntimeError as exc:
+		raise _http_error(400, str(exc)) from exc
+	except Exception as exc:
+		raise _http_error(500, "Failed to execute step") from exc
+
+
+@router.get("/state", response_model=EnvironmentState)
+def environment_state() -> EnvironmentState:
+	try:
+		return env_service.state()
+	except RuntimeError as exc:
+		raise _http_error(400, str(exc)) from exc
+	except Exception as exc:
+		raise _http_error(500, "Failed to retrieve state") from exc
+
+
+@router.get("/grader", response_model=GraderResponse)
+def environment_grader() -> GraderResponse:
+	try:
+		report = env_service.grade()
+		return GraderResponse(
+			task=env_service.current_task(),
+			steps_recorded=env_service.steps_recorded(),
+			report=report,
+		)
+	except RuntimeError as exc:
+		raise _http_error(400, str(exc)) from exc
+	except Exception as exc:
+		raise _http_error(500, "Failed to compute grade") from exc
+
+
+@router.get("/baseline", response_model=BaselineResponse)
+def environment_baseline(task: str | None = None, seed: int | None = None, max_steps: int | None = None) -> BaselineResponse:
+	try:
+		return env_service.baseline_rollout(task=task, seed=seed, max_steps=max_steps)
+	except ValueError as exc:
+		raise _http_error(400, str(exc)) from exc
+	except RuntimeError as exc:
+		raise _http_error(400, str(exc)) from exc
+	except Exception as exc:
+		raise _http_error(500, "Failed to compute baseline rollout") from exc

app/static/results.html

@@ -0,0 +1,1063 @@
+<!doctype html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8" />
+  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+  <title>OpenEnv Result Studio</title>
+  <link rel="preconnect" href="https://fonts.googleapis.com" />
+  <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
+  <link href="https://fonts.googleapis.com/css2?family=Space+Grotesk:wght@400;500;700&family=JetBrains+Mono:wght@400;600&display=swap" rel="stylesheet" />
+  <style>
+    :root {
+      --ink: #11212a;
+      --muted: #4a6675;
+      --bg-a: #e7f1f5;
+      --bg-b: #f6efe4;
+      --card: rgba(255, 255, 255, 0.82);
+      --card-border: #c7d9df;
+      --accent: #13786d;
+      --accent-hover: #0f5f56;
+      --accent-soft: #cdece7;
+      --warn: #f18f3b;
+      --error: #b63d2f;
+      --ok: #1f7a40;
+      --mono: "JetBrains Mono", "Consolas", monospace;
+    }
+
+    * {
+      box-sizing: border-box;
+    }
+
+    body {
+      margin: 0;
+      min-height: 100vh;
+      color: var(--ink);
+      font-family: "Space Grotesk", "Segoe UI", sans-serif;
+      background:
+        radial-gradient(1000px 420px at 8% -5%, #b8dfe8 0%, transparent 72%),
+        radial-gradient(920px 420px at 96% -6%, #f6d9bb 0%, transparent 70%),
+        linear-gradient(140deg, var(--bg-a), var(--bg-b));
+    }
+
+    body::before {
+      content: "";
+      position: fixed;
+      inset: 0;
+      pointer-events: none;
+      opacity: 0.25;
+      background-image:
+        linear-gradient(rgba(17, 33, 42, 0.06) 1px, transparent 1px),
+        linear-gradient(90deg, rgba(17, 33, 42, 0.06) 1px, transparent 1px);
+      background-size: 32px 32px;
+      mask-image: radial-gradient(circle at center, black 38%, transparent 85%);
+    }
+
+    main {
+      position: relative;
+      max-width: 1160px;
+      margin: 0 auto;
+      padding: 30px 16px 44px;
+      display: grid;
+      gap: 14px;
+    }
+
+    .card {
+      background: var(--card);
+      border: 1px solid var(--card-border);
+      border-radius: 18px;
+      box-shadow: 0 16px 38px rgba(17, 33, 42, 0.12);
+      backdrop-filter: blur(8px);
+      animation: rise 0.5s ease both;
+    }
+
+    .card:nth-child(2) { animation-delay: 0.05s; }
+    .card:nth-child(3) { animation-delay: 0.09s; }
+    .card:nth-child(4) { animation-delay: 0.13s; }
+
+    .hero {
+      padding: 22px;
+      display: grid;
+      gap: 10px;
+    }
+
+    .hero h1 {
+      margin: 0;
+      font-size: clamp(1.45rem, 2.5vw, 2.25rem);
+      letter-spacing: 0.2px;
+    }
+
+    .hero p {
+      margin: 0;
+      color: var(--muted);
+      line-height: 1.4;
+      max-width: 840px;
+    }
+
+    .status-row {
+      display: flex;
+      align-items: center;
+      gap: 10px;
+      flex-wrap: wrap;
+    }
+
+    .pill {
+      display: inline-flex;
+      align-items: center;
+      gap: 6px;
+      border-radius: 999px;
+      padding: 5px 10px;
+      font-size: 0.84rem;
+      font-weight: 600;
+      border: 1px solid transparent;
+    }
+
+    .pill.ok {
+      color: #134b23;
+      background: #d9f4df;
+      border-color: #9ed2a9;
+    }
+
+    .pill.warn {
+      color: #78450d;
+      background: #ffe6c9;
+      border-color: #efc48e;
+    }
+
+    .pill.error {
+      color: #6b251d;
+      background: #f6d3cf;
+      border-color: #e2a19a;
+    }
+
+    .controls {
+      padding: 18px;
+      display: grid;
+      gap: 14px;
+    }
+
+    .form-row {
+      display: grid;
+      grid-template-columns: repeat(2, minmax(0, 1fr));
+      gap: 10px;
+    }
+
+    .field {
+      display: grid;
+      gap: 5px;
+    }
+
+    .field label {
+      font-size: 0.72rem;
+      letter-spacing: 0.08em;
+      color: var(--muted);
+      text-transform: uppercase;
+      font-weight: 700;
+    }
+
+    .field input,
+    .field select {
+      width: 100%;
+      border-radius: 10px;
+      border: 1px solid #aac2ca;
+      background: #f7fbfc;
+      color: var(--ink);
+      padding: 10px 11px;
+      font-size: 0.95rem;
+      font-family: inherit;
+      outline: none;
+      transition: border-color 0.18s ease, box-shadow 0.18s ease;
+    }
+
+    .field input:focus,
+    .field select:focus {
+      border-color: var(--accent);
+      box-shadow: 0 0 0 3px rgba(19, 120, 109, 0.15);
+    }
+
+    .button-row {
+      display: flex;
+      flex-wrap: wrap;
+      gap: 8px;
+    }
+
+    button {
+      border: 1px solid #9fb8c0;
+      border-radius: 10px;
+      padding: 10px 13px;
+      font-family: inherit;
+      font-size: 0.9rem;
+      font-weight: 600;
+      color: var(--ink);
+      background: #f4fbfd;
+      cursor: pointer;
+      transition: transform 0.12s ease, box-shadow 0.12s ease, background 0.16s ease;
+    }
+
+    button:hover {
+      transform: translateY(-1px);
+      box-shadow: 0 6px 16px rgba(17, 33, 42, 0.12);
+      background: #ebf7fa;
+    }
+
+    button.primary {
+      background: var(--accent);
+      color: #ffffff;
+      border-color: #0f5f56;
+    }
+
+    button.primary:hover {
+      background: var(--accent-hover);
+    }
+
+    button.warn {
+      background: #fff4e8;
+      border-color: #f3bc86;
+      color: #7e4a0f;
+    }
+
+    .metrics {
+      padding: 18px;
+      display: grid;
+      grid-template-columns: repeat(6, minmax(0, 1fr));
+      gap: 10px;
+    }
+
+    .metric {
+      background: rgba(255, 255, 255, 0.82);
+      border: 1px solid #cddbe0;
+      border-radius: 12px;
+      padding: 12px;
+      display: grid;
+      gap: 5px;
+    }
+
+    .metric label {
+      font-size: 0.72rem;
+      letter-spacing: 0.08em;
+      text-transform: uppercase;
+      color: var(--muted);
+      font-weight: 700;
+    }
+
+    .metric strong {
+      font-family: var(--mono);
+      font-size: 1rem;
+      color: #0c3640;
+      word-break: break-word;
+    }
+
+    .results-grid {
+      display: grid;
+      grid-template-columns: 1.1fr 0.9fr;
+      gap: 14px;
+    }
+
+    .panel {
+      padding: 16px;
+      display: grid;
+      gap: 10px;
+    }
+
+    .panel h2 {
+      margin: 0;
+      font-size: 1.05rem;
+      letter-spacing: 0.02em;
+    }
+
+    .mono {
+      font-family: var(--mono);
+      font-size: 0.82rem;
+      line-height: 1.45;
+    }
+
+    .log-list {
+      list-style: none;
+      margin: 0;
+      padding: 0;
+      display: grid;
+      gap: 8px;
+      max-height: 390px;
+      overflow: auto;
+    }
+
+    .log-item {
+      border: 1px solid #c8d7dc;
+      background: rgba(248, 252, 253, 0.95);
+      border-radius: 12px;
+      padding: 10px;
+      display: grid;
+      gap: 6px;
+    }
+
+    .log-head {
+      display: flex;
+      justify-content: space-between;
+      gap: 8px;
+      align-items: center;
+      font-size: 0.86rem;
+      font-weight: 700;
+    }
+
+    .tag {
+      font-size: 0.72rem;
+      padding: 2px 7px;
+      border-radius: 999px;
+      border: 1px solid;
+      font-weight: 700;
+      letter-spacing: 0.04em;
+    }
+
+    .tag.ok {
+      color: #205f32;
+      border-color: #8fc39d;
+      background: #e1f5e6;
+    }
+
+    .tag.error {
+      color: #75281f;
+      border-color: #d2948b;
+      background: #f9dcd8;
+    }
+
+    pre {
+      margin: 0;
+      padding: 12px;
+      border: 1px solid #bfd3d9;
+      border-radius: 12px;
+      background: #f4fbfd;
+      max-height: 390px;
+      overflow: auto;
+      white-space: pre-wrap;
+      word-break: break-word;
+    }
+
+    .grader-table {
+      width: 100%;
+      border-collapse: collapse;
+      font-family: var(--mono);
+      font-size: 0.8rem;
+      border: 1px solid #bfd3d9;
+      border-radius: 12px;
+      overflow: hidden;
+    }
+
+    .grader-table th,
+    .grader-table td {
+      text-align: left;
+      padding: 8px 10px;
+      border-bottom: 1px solid #d7e4e8;
+    }
+
+    .grader-table th {
+      background: #e4f2f5;
+      letter-spacing: 0.06em;
+      text-transform: uppercase;
+      font-size: 0.72rem;
+      color: #365765;
+    }
+
+    .grader-table tr:last-child td {
+      border-bottom: none;
+    }
+
+    .suite-wrap {
+      padding: 16px;
+      display: grid;
+      gap: 10px;
+    }
+
+    .suite-toolbar {
+      display: flex;
+      justify-content: space-between;
+      align-items: center;
+      gap: 8px;
+      flex-wrap: wrap;
+    }
+
+    .suite-note {
+      margin: 0;
+      color: var(--muted);
+      font-size: 0.85rem;
+    }
+
+    .suite-average {
+      font-family: var(--mono);
+      font-weight: 700;
+      color: #0c3640;
+      font-size: 0.88rem;
+    }
+
+    .suite-table {
+      width: 100%;
+      border-collapse: collapse;
+      border: 1px solid #bfd3d9;
+      border-radius: 12px;
+      overflow: hidden;
+      font-family: var(--mono);
+      font-size: 0.8rem;
+    }
+
+    .suite-table th,
+    .suite-table td {
+      text-align: left;
+      padding: 8px 10px;
+      border-bottom: 1px solid #d7e4e8;
+    }
+
+    .suite-table th {
+      background: #e4f2f5;
+      letter-spacing: 0.06em;
+      text-transform: uppercase;
+      font-size: 0.72rem;
+      color: #365765;
+    }
+
+    .suite-table tr:last-child td {
+      border-bottom: none;
+    }
+
+    .flag {
+      display: inline-flex;
+      align-items: center;
+      border-radius: 999px;
+      border: 1px solid;
+      padding: 2px 7px;
+      font-size: 0.72rem;
+      letter-spacing: 0.04em;
+      font-weight: 700;
+    }
+
+    .flag.ok {
+      color: #205f32;
+      border-color: #8fc39d;
+      background: #e1f5e6;
+    }
+
+    .flag.error {
+      color: #75281f;
+      border-color: #d2948b;
+      background: #f9dcd8;
+    }
+
+    @media (max-width: 980px) {
+      .metrics {
+        grid-template-columns: repeat(3, minmax(0, 1fr));
+      }
+
+      .results-grid {
+        grid-template-columns: 1fr;
+      }
+    }
+
+    @media (max-width: 640px) {
+      .form-row {
+        grid-template-columns: 1fr;
+      }
+
+      .metrics {
+        grid-template-columns: repeat(2, minmax(0, 1fr));
+      }
+
+      .button-row {
+        display: grid;
+        grid-template-columns: 1fr 1fr;
+      }
+    }
+
+    @keyframes rise {
+      from {
+        opacity: 0;
+        transform: translateY(8px);
+      }
+      to {
+        opacity: 1;
+        transform: translateY(0);
+      }
+    }
+  </style>
+</head>
+<body>
+  <main>
+    <section class="card hero">
+      <h1>OpenEnv Result Studio</h1>
+      <p>Interactive frontend for the last-mile delivery benchmark. Run reset/step/state/grader/baseline directly from the browser and inspect live deterministic results.</p>
+      <div class="status-row">
+        <span id="statusPill" class="pill warn">Idle</span>
+        <span id="statusText" class="mono">Ready to run checks.</span>
+      </div>
+    </section>
+
+    <section class="card controls">
+      <div class="form-row">
+        <div class="field">
+          <label for="taskInput">Task</label>
+          <select id="taskInput">
+            <option value="easy">easy</option>
+            <option value="medium">medium</option>
+            <option value="hard">hard</option>
+          </select>
+        </div>
+        <div class="field">
+          <label for="seedInput">Seed</label>
+          <input id="seedInput" type="number" min="0" value="101" />
+        </div>
+      </div>
+
+      <div class="form-row">
+        <div class="field">
+          <label for="apiBaseInput">API Base URL (optional)</label>
+          <input id="apiBaseInput" type="text" placeholder="http://127.0.0.1:8000" />
+        </div>
+      </div>
+
+      <div class="button-row">
+        <button class="primary" id="btnReset">Reset</button>
+        <button id="btnStep">Step (wait)</button>
+        <button id="btnState">State</button>
+        <button id="btnGrader">Grader</button>
+        <button id="btnBaseline">Baseline</button>
+        <button class="primary" id="btnSuite">Run All Tasks</button>
+        <button class="warn" id="btnDemo">Run Demo Sequence</button>
+      </div>
+    </section>
+
+    <section class="card metrics">
+      <div class="metric"><label>Task</label><strong id="mTask">easy</strong></div>
+      <div class="metric"><label>Done</label><strong id="mDone">false</strong></div>
+      <div class="metric"><label>Step Count</label><strong id="mStep">0</strong></div>
+      <div class="metric"><label>Total Reward</label><strong id="mTotalReward">0.00</strong></div>
+      <div class="metric"><label>Pending Orders</label><strong id="mPending">0</strong></div>
+      <div class="metric"><label>Score</label><strong id="mScore">n/a</strong></div>
+    </section>
+
+    <section class="results-grid">
+      <article class="card panel">
+        <h2>Run Log</h2>
+        <ul id="logList" class="log-list"></ul>
+      </article>
+
+      <article class="card panel">
+        <h2>Latest Payload</h2>
+        <pre id="payloadView" class="mono">{}</pre>
+      </article>
+    </section>
+
+    <section class="card panel">
+      <h2>Grader Metrics</h2>
+      <table class="grader-table" id="graderTable">
+        <thead>
+          <tr><th>Metric</th><th>Value</th></tr>
+        </thead>
+        <tbody>
+          <tr><td>score</td><td>n/a</td></tr>
+        </tbody>
+      </table>
+    </section>
+
+    <section class="card suite-wrap">
+      <div class="suite-toolbar">
+        <p class="suite-note">One-click baseline rollout across easy, medium, and hard using the current seed.</p>
+        <span id="suiteAverage" class="suite-average">average_score: n/a</span>
+      </div>
+      <table class="suite-table" id="suiteTable">
+        <thead>
+          <tr><th>Task</th><th>Success</th><th>Score</th><th>Steps</th><th>Total Reward</th></tr>
+        </thead>
+        <tbody>
+          <tr><td colspan="5">No suite run yet.</td></tr>
+        </tbody>
+      </table>
+    </section>
+  </main>
+
+  <script>
+    const state = {
+      task: "easy",
+      seed: 101,
+      apiBaseUrl: "",
+      done: false,
+      stepCount: 0,
+      totalReward: 0,
+      pendingOrders: 0,
+      score: null,
+      lastPayload: {},
+      logCount: 0,
+      suiteRows: [],
+    };
+
+    const nodes = {
+      statusPill: document.getElementById("statusPill"),
+      statusText: document.getElementById("statusText"),
+      taskInput: document.getElementById("taskInput"),
+      seedInput: document.getElementById("seedInput"),
+      apiBaseInput: document.getElementById("apiBaseInput"),
+      btnReset: document.getElementById("btnReset"),
+      btnStep: document.getElementById("btnStep"),
+      btnState: document.getElementById("btnState"),
+      btnGrader: document.getElementById("btnGrader"),
+      btnBaseline: document.getElementById("btnBaseline"),
+      btnSuite: document.getElementById("btnSuite"),
+      btnDemo: document.getElementById("btnDemo"),
+      mTask: document.getElementById("mTask"),
+      mDone: document.getElementById("mDone"),
+      mStep: document.getElementById("mStep"),
+      mTotalReward: document.getElementById("mTotalReward"),
+      mPending: document.getElementById("mPending"),
+      mScore: document.getElementById("mScore"),
+      logList: document.getElementById("logList"),
+      payloadView: document.getElementById("payloadView"),
+      graderTable: document.getElementById("graderTable"),
+      suiteTable: document.getElementById("suiteTable"),
+      suiteAverage: document.getElementById("suiteAverage"),
+    };
+
+    function setStatus(message, mode) {
+      nodes.statusText.textContent = message;
+      nodes.statusPill.classList.remove("ok", "warn", "error");
+      if (mode === "ok") {
+        nodes.statusPill.classList.add("ok");
+        nodes.statusPill.textContent = "OK";
+      } else if (mode === "error") {
+        nodes.statusPill.classList.add("error");
+        nodes.statusPill.textContent = "Error";
+      } else {
+        nodes.statusPill.classList.add("warn");
+        nodes.statusPill.textContent = "Running";
+      }
+    }
+
+    function renderMetrics() {
+      nodes.mTask.textContent = state.task;
+      nodes.mDone.textContent = String(state.done);
+      nodes.mStep.textContent = String(state.stepCount);
+      nodes.mTotalReward.textContent = Number(state.totalReward).toFixed(2);
+      nodes.mPending.textContent = String(state.pendingOrders);
+      nodes.mScore.textContent = state.score === null ? "n/a" : Number(state.score).toFixed(4);
+      nodes.payloadView.textContent = JSON.stringify(state.lastPayload, null, 2);
+    }
+
+    function renderGraderTable(report) {
+      const tbody = document.createElement("tbody");
+      const pairs = [
+        ["score", report?.score],
+        ["completion_rate", report?.completion_rate],
+        ["efficiency_ratio", report?.efficiency_ratio],
+        ["invalid_action_rate", report?.invalid_action_rate],
+        ["steps_taken", report?.steps_taken],
+        ["delivered_orders", report?.delivered_orders],
+        ["total_orders", report?.total_orders],
+      ];
+
+      pairs.forEach(([key, value]) => {
+        const tr = document.createElement("tr");
+        const tdKey = document.createElement("td");
+        tdKey.textContent = String(key);
+        const tdValue = document.createElement("td");
+        tdValue.textContent = value === undefined || value === null ? "n/a" : String(value);
+        tr.appendChild(tdKey);
+        tr.appendChild(tdValue);
+        tbody.appendChild(tr);
+      });
+
+      const oldTbody = nodes.graderTable.querySelector("tbody");
+      if (oldTbody) {
+        nodes.graderTable.removeChild(oldTbody);
+      }
+      nodes.graderTable.appendChild(tbody);
+    }
+
+    function renderSuiteTable(rows) {
+      const tbody = document.createElement("tbody");
+      if (!Array.isArray(rows) || rows.length === 0) {
+        const tr = document.createElement("tr");
+        const td = document.createElement("td");
+        td.colSpan = 5;
+        td.textContent = "No suite run yet.";
+        tr.appendChild(td);
+        tbody.appendChild(tr);
+      } else {
+        rows.forEach((item) => {
+          const tr = document.createElement("tr");
+
+          const taskTd = document.createElement("td");
+          taskTd.textContent = String(item.task);
+          tr.appendChild(taskTd);
+
+          const successTd = document.createElement("td");
+          const flag = document.createElement("span");
+          flag.className = `flag ${item.success ? "ok" : "error"}`;
+          flag.textContent = item.success ? "TRUE" : "FALSE";
+          successTd.appendChild(flag);
+          tr.appendChild(successTd);
+
+          const scoreTd = document.createElement("td");
+          scoreTd.textContent = Number(item.score || 0).toFixed(4);
+          tr.appendChild(scoreTd);
+
+          const stepsTd = document.createElement("td");
+          stepsTd.textContent = String(item.steps_executed || 0);
+          tr.appendChild(stepsTd);
+
+          const rewardTd = document.createElement("td");
+          rewardTd.textContent = Number(item.total_reward || 0).toFixed(2);
+          tr.appendChild(rewardTd);
+
+          tbody.appendChild(tr);
+        });
+      }
+
+      const oldTbody = nodes.suiteTable.querySelector("tbody");
+      if (oldTbody) {
+        nodes.suiteTable.removeChild(oldTbody);
+      }
+      nodes.suiteTable.appendChild(tbody);
+
+      if (!rows.length) {
+        nodes.suiteAverage.textContent = "average_score: n/a";
+        return;
+      }
+
+      const average = rows.reduce((acc, item) => acc + Number(item.score || 0), 0) / rows.length;
+      nodes.suiteAverage.textContent = `average_score: ${average.toFixed(4)}`;
+    }
+
+    function addLog(label, payload, ok) {
+      state.logCount += 1;
+      const li = document.createElement("li");
+      li.className = "log-item";
+
+      const header = document.createElement("div");
+      header.className = "log-head";
+      const title = document.createElement("span");
+      title.textContent = `${state.logCount}. ${label}`;
+
+      const tag = document.createElement("span");
+      tag.className = `tag ${ok ? "ok" : "error"}`;
+      tag.textContent = ok ? "SUCCESS" : "FAILED";
+
+      header.appendChild(title);
+      header.appendChild(tag);
+      li.appendChild(header);
+
+      const pre = document.createElement("pre");
+      pre.className = "mono";
+      pre.textContent = JSON.stringify(payload, null, 2);
+      li.appendChild(pre);
+
+      nodes.logList.prepend(li);
+    }
+
+    function updateFromObservation(obs, doneFlag) {
+      if (!obs) {
+        return;
+      }
+      state.done = Boolean(doneFlag);
+      state.stepCount = Number(obs.step_count || 0);
+      state.totalReward = Number(obs.total_reward || 0);
+      state.pendingOrders = Array.isArray(obs.pending_orders) ? obs.pending_orders.length : 0;
+    }
+
+    function updateFromState(payload) {
+      state.done = Boolean(payload.done);
+      state.stepCount = Number(payload.step_count || 0);
+      state.totalReward = Number(payload.total_reward || 0);
+      if (payload.observation && Array.isArray(payload.observation.pending_orders)) {
+        state.pendingOrders = payload.observation.pending_orders.length;
+      }
+    }
+
+    function normalizeApiBaseUrl(value) {
+      return String(value || "").trim().replace(/\/+$/, "");
+    }
+
+    function applyApiBaseUrl(value, persist) {
+      state.apiBaseUrl = normalizeApiBaseUrl(value);
+      if (nodes.apiBaseInput) {
+        nodes.apiBaseInput.value = state.apiBaseUrl;
+      }
+
+      if (!persist) {
+        return;
+      }
+
+      try {
+        window.localStorage.setItem("openenv_api_base_url", state.apiBaseUrl);
+      } catch (_) {
+        // Ignore storage errors in restricted browser contexts.
+      }
+    }
+
+    function currentApiBaseUrl() {
+      return normalizeApiBaseUrl(nodes.apiBaseInput?.value || state.apiBaseUrl);
+    }
+
+    function buildApiUrl(path) {
+      const normalizedPath = path.startsWith("/") ? path : `/${path}`;
+      const baseUrl = currentApiBaseUrl();
+      if (!baseUrl) {
+        return normalizedPath;
+      }
+      return `${baseUrl}${normalizedPath}`;
+    }
+
+    function saveApiBaseUrl() {
+      applyApiBaseUrl(currentApiBaseUrl(), true);
+    }
+
+    function loadApiBaseUrl() {
+      let saved = "";
+      try {
+        saved = window.localStorage.getItem("openenv_api_base_url") || "";
+      } catch (_) {
+        saved = "";
+      }
+
+      applyApiBaseUrl(saved, false);
+    }
+
+    function candidateApiBaseUrls() {
+      const candidates = [];
+      const add = (value) => {
+        const normalized = normalizeApiBaseUrl(value);
+        if (normalized && !candidates.includes(normalized)) {
+          candidates.push(normalized);
+        }
+      };
+
+      add(state.apiBaseUrl);
+      if (window.location.protocol === "http:" || window.location.protocol === "https:") {
+        add(window.location.origin);
+      }
+
+      add("http://127.0.0.1:8000");
+      add("http://localhost:8000");
+      add("http://127.0.0.1:7860");
+      add("http://localhost:7860");
+      return candidates;
+    }
+
+    async function probeHealth(baseUrl) {
+      const url = `${baseUrl}/health`;
+      try {
+        const response = await fetch(url, { method: "GET" });
+        if (!response.ok) {
+          return false;
+        }
+
+        const contentType = (response.headers.get("content-type") || "").toLowerCase();
+        if (!contentType.includes("application/json")) {
+          return false;
+        }
+
+        const payload = await response.json();
+        return payload && payload.status === "ok";
+      } catch (_) {
+        return false;
+      }
+    }
+
+    async function autoDetectApiBaseUrl() {
+      const candidates = candidateApiBaseUrls();
+      for (let i = 0; i < candidates.length; i += 1) {
+        const candidate = candidates[i];
+        if (await probeHealth(candidate)) {
+          applyApiBaseUrl(candidate, true);
+          return true;
+        }
+      }
+      return false;
+    }
+
+    async function apiCall(path, options = {}) {
+      const url = buildApiUrl(path);
+      const response = await fetch(url, {
+        headers: { "Content-Type": "application/json" },
+        ...options,
+      });
+
+      const rawText = await response.text();
+      let data = {};
+      if (rawText) {
+        try {
+          data = JSON.parse(rawText);
+        } catch (_) {
+          data = {
+            message: "Non-JSON response from endpoint",
+            endpoint: url,
+            status: response.status,
+            content_type: response.headers.get("content-type") || "unknown",
+            body_preview: rawText.slice(0, 240),
+            hint: "Set API Base URL to your FastAPI server, for example http://127.0.0.1:8000",
+          };
+        }
+      }
+
+      if (!response.ok) {
+        const err = new Error(`HTTP ${response.status}`);
+        err.payload = data;
+        throw err;
+      }
+
+      if (data && data.message === "Non-JSON response from endpoint") {
+        const err = new Error("API returned non-JSON content");
+        err.payload = data;
+        throw err;
+      }
+
+      return data;
+    }
+
+    async function withAction(label, fn) {
+      setStatus(`${label} in progress...`, "warn");
+      try {
+        const payload = await fn();
+        state.lastPayload = payload;
+        addLog(label, payload, true);
+        renderMetrics();
+        setStatus(`${label} complete.`, "ok");
+      } catch (error) {
+        const payload = error.payload || { message: String(error) };
+        state.lastPayload = payload;
+        addLog(label, payload, false);
+        renderMetrics();
+        setStatus(`${label} failed.`, "error");
+      }
+    }
+
+    async function doReset() {
+      state.task = nodes.taskInput.value;
+      state.seed = Number(nodes.seedInput.value || 0);
+      state.score = null;
+      return withAction("reset", async () => {
+        const payload = await apiCall("/reset", {
+          method: "POST",
+          body: JSON.stringify({ task: state.task, seed: state.seed }),
+        });
+        updateFromObservation(payload, false);
+        renderGraderTable(null);
+        return payload;
+      });
+    }
+
+    async function doStepWait() {
+      return withAction("step(wait)", async () => {
+        const payload = await apiCall("/step", {
+          method: "POST",
+          body: JSON.stringify({ move: null, accept_order: null, deliver_order: false, wait: true }),
+        });
+        updateFromObservation(payload.observation, payload.done);
+        return payload;
+      });
+    }
+
+    async function doState() {
+      return withAction("state", async () => {
+        const payload = await apiCall("/state", { method: "GET" });
+        updateFromState(payload);
+        return payload;
+      });
+    }
+
+    async function doGrader() {
+      return withAction("grader", async () => {
+        const payload = await apiCall("/grader", { method: "GET" });
+        if (payload.report && payload.report.score !== undefined) {
+          state.score = Number(payload.report.score);
+        }
+        renderGraderTable(payload.report || null);
+        return payload;
+      });
+    }
+
+    async function doBaseline() {
+      state.task = nodes.taskInput.value;
+      state.seed = Number(nodes.seedInput.value || 0);
+      return withAction("baseline", async () => {
+        const query = new URLSearchParams({ task: state.task, seed: String(state.seed) });
+        const payload = await apiCall(`/baseline?${query.toString()}`, { method: "GET" });
+        state.score = payload.score === undefined ? state.score : Number(payload.score);
+        renderGraderTable(payload.report || null);
+        return payload;
+      });
+    }
+
+    async function doSuite() {
+      state.seed = Number(nodes.seedInput.value || 0);
+      return withAction("baseline_suite", async () => {
+        const tasks = ["easy", "medium", "hard"];
+        const suiteRows = [];
+
+        for (let i = 0; i < tasks.length; i += 1) {
+          const task = tasks[i];
+          setStatus(`Running suite ${i + 1}/${tasks.length}: ${task}`, "warn");
+          const query = new URLSearchParams({ task, seed: String(state.seed) });
+          const payload = await apiCall(`/baseline?${query.toString()}`, { method: "GET" });
+          suiteRows.push({
+            task,
+            success: Boolean(payload.done),
+            score: Number(payload.score || 0),
+            steps_executed: Number(payload.steps_executed || 0),
+            total_reward: Number(payload.report?.total_reward || 0),
+          });
+        }
+
+        state.suiteRows = suiteRows;
+        renderSuiteTable(state.suiteRows);
+
+        return {
+          seed: state.seed,
+          tasks: suiteRows,
+          average_score: suiteRows.length
+            ? Number((suiteRows.reduce((acc, item) => acc + Number(item.score || 0), 0) / suiteRows.length).toFixed(4))
+            : 0,
+        };
+      });
+    }
+
+    async function doDemoSequence() {
+      setStatus("Running demo sequence reset -> step -> step -> state -> grader", "warn");
+      await doReset();
+      await doStepWait();
+      await doStepWait();
+      await doState();
+      await doGrader();
+      setStatus("Demo sequence finished.", "ok");
+    }
+
+    nodes.btnReset.addEventListener("click", doReset);
+    nodes.btnStep.addEventListener("click", doStepWait);
+    nodes.btnState.addEventListener("click", doState);
+    nodes.btnGrader.addEventListener("click", doGrader);
+    nodes.btnBaseline.addEventListener("click", doBaseline);
+    nodes.btnSuite.addEventListener("click", doSuite);
+    nodes.btnDemo.addEventListener("click", doDemoSequence);
+    nodes.apiBaseInput.addEventListener("change", saveApiBaseUrl);
+    nodes.apiBaseInput.addEventListener("blur", saveApiBaseUrl);
+
+    async function initializeUi() {
+      loadApiBaseUrl();
+      renderMetrics();
+      renderGraderTable(null);
+      renderSuiteTable([]);
+
+      if (state.apiBaseUrl) {
+        const savedHealthy = await probeHealth(state.apiBaseUrl);
+        if (savedHealthy) {
+          setStatus(`Ready. Endpoint: ${state.apiBaseUrl}`, "ok");
+          return;
+        }
+      }
+
+      const detected = await autoDetectApiBaseUrl();
+      if (detected) {
+        setStatus(`Ready. Endpoint: ${currentApiBaseUrl()}`, "ok");
+        return;
+      }
+
+      setStatus("No API detected. Set API Base URL and ensure /health is reachable.", "error");
+    }
+
+    initializeUi();
+  </script>
+</body>
+</html>

baseline/baseline_agent.py

@@ -0,0 +1,102 @@
+from __future__ import annotations
+
+from typing import List, Optional, Tuple
+
+from env.models import ActionType, Direction, Observation, Order, OrderStatus, Position, SimulatorAction
+from env.utils import manhattan
+
+
+class BaselineGreedyAgent:
+	"""Greedy baseline that accepts nearest pending order and follows shortest-axis moves."""
+
+	def act(self, observation: Observation) -> SimulatorAction:
+		agent = observation.agent_location
+		current_order = observation.current_order
+
+		if current_order is None:
+			pending_orders = [
+				order for order in observation.pending_orders if order.status == OrderStatus.PENDING
+			]
+			if not pending_orders:
+				return SimulatorAction(action_type=ActionType.WAIT)
+
+			selected = self._nearest_pending(agent, pending_orders)
+			return SimulatorAction(action_type=ActionType.ACCEPT_ORDER, order_id=selected.order_id)
+
+		if current_order.status == OrderStatus.ACCEPTED:
+			if self._same(agent, current_order.pickup):
+				return SimulatorAction(action_type=ActionType.DELIVER_ORDER)
+			return self._move_towards(agent, current_order.pickup, observation)
+
+		if current_order.status == OrderStatus.PICKED_UP:
+			if self._same(agent, current_order.dropoff):
+				return SimulatorAction(action_type=ActionType.DELIVER_ORDER)
+			return self._move_towards(agent, current_order.dropoff, observation)
+
+		return SimulatorAction(action_type=ActionType.WAIT)
+
+	def _nearest_pending(self, agent: Position, orders: List[Order]) -> Order:
+		origin = (agent.x, agent.y)
+		return min(orders, key=lambda order: manhattan(origin, (order.pickup.x, order.pickup.y)))
+
+	def _move_towards(self, source: Position, target: Position, observation: Observation) -> SimulatorAction:
+		candidates = self._direction_candidates(source, target)
+		for direction in candidates:
+			if self._is_safe_move(source, direction, observation):
+				return SimulatorAction(action_type=ActionType.MOVE, direction=direction)
+
+		return SimulatorAction(action_type=ActionType.WAIT)
+
+	def _direction_candidates(self, source: Position, target: Position) -> List[Direction]:
+		directions: List[Direction] = []
+
+		dx = target.x - source.x
+		dy = target.y - source.y
+
+		if abs(dx) >= abs(dy):
+			if dx > 0:
+				directions.append(Direction.RIGHT)
+			elif dx < 0:
+				directions.append(Direction.LEFT)
+
+			if dy > 0:
+				directions.append(Direction.DOWN)
+			elif dy < 0:
+				directions.append(Direction.UP)
+		else:
+			if dy > 0:
+				directions.append(Direction.DOWN)
+			elif dy < 0:
+				directions.append(Direction.UP)
+
+			if dx > 0:
+				directions.append(Direction.RIGHT)
+			elif dx < 0:
+				directions.append(Direction.LEFT)
+
+		# Fall back to any movement if preferred route is blocked.
+		for d in [Direction.UP, Direction.DOWN, Direction.LEFT, Direction.RIGHT]:
+			if d not in directions:
+				directions.append(d)
+		return directions
+
+	def _is_safe_move(self, source: Position, direction: Direction, observation: Observation) -> bool:
+		x, y = source.x, source.y
+		if direction == Direction.UP:
+			y -= 1
+		elif direction == Direction.DOWN:
+			y += 1
+		elif direction == Direction.LEFT:
+			x -= 1
+		else:
+			x += 1
+
+		if x < 0 or y < 0 or x >= observation.grid_width or y >= observation.grid_height:
+			return False
+
+		obstacle_set = {(o.x, o.y) for o in observation.obstacles}
+		return (x, y) not in obstacle_set
+
+	@staticmethod
+	def _same(a: Position, b: Position) -> bool:
+		return a.x == b.x and a.y == b.y

baseline/trained_q_agent.py

@@ -0,0 +1,176 @@
+from __future__ import annotations
+
+import json
+from typing import Dict, List, Optional, Tuple
+
+from env.models import ActionType, Direction, Observation, Order, OrderStatus, SimulatorAction
+from env.utils import manhattan
+
+
+StateKey = str
+QValues = List[float]
+QTable = Dict[StateKey, QValues]
+
+
+ACTION_UP = 0
+ACTION_DOWN = 1
+ACTION_LEFT = 2
+ACTION_RIGHT = 3
+ACTION_ACCEPT = 4
+ACTION_DELIVER = 5
+ACTION_WAIT = 6
+NUM_ACTIONS = 7
+
+ACTION_LABELS: List[str] = [
+    "move_up",
+    "move_down",
+    "move_left",
+    "move_right",
+    "accept_nearest",
+    "deliver",
+    "wait",
+]
+
+
+def _nearest_pending_order(observation: Observation) -> Optional[Order]:
+    pending = [order for order in observation.pending_orders if order.status == OrderStatus.PENDING]
+    if not pending:
+        return None
+
+    origin = (observation.agent_location.x, observation.agent_location.y)
+    return min(pending, key=lambda order: manhattan(origin, (order.pickup.x, order.pickup.y)))
+
+
+def _target_from_observation(observation: Observation) -> Tuple[int, int, int]:
+    current = observation.current_order
+    origin = (observation.agent_location.x, observation.agent_location.y)
+
+    # stage: 0=no active order, 1=going to pickup, 2=going to dropoff, 3=other
+    if current is None:
+        pending = _nearest_pending_order(observation)
+        if pending is None:
+            return -1, -1, 0
+        return pending.pickup.x, pending.pickup.y, 0
+
+    if current.status == OrderStatus.ACCEPTED:
+        return current.pickup.x, current.pickup.y, 1
+
+    if current.status == OrderStatus.PICKED_UP:
+        targets = current.delivery_locations or [current.dropoff]
+        selected = min(targets, key=lambda point: manhattan(origin, (point.x, point.y)))
+        return selected.x, selected.y, 2
+
+    return -1, -1, 3
+
+
+def _blocked_neighbor_count(observation: Observation) -> int:
+    blocked = {(item.x, item.y) for item in observation.obstacles}
+    blocked.update((item.x, item.y) for item in observation.dynamic_obstacles)
+
+    x = observation.agent_location.x
+    y = observation.agent_location.y
+
+    neighbors = [
+        (x, y - 1),
+        (x, y + 1),
+        (x - 1, y),
+        (x + 1, y),
+    ]
+
+    count = 0
+    for nx, ny in neighbors:
+        if nx < 0 or ny < 0 or nx >= observation.grid_width or ny >= observation.grid_height:
+            count += 1
+        elif (nx, ny) in blocked:
+            count += 1
+    return count
+
+
+def _battery_bucket(level: Optional[int]) -> int:
+    if level is None:
+        return -1
+    if level <= 10:
+        return 0
+    if level <= 25:
+        return 1
+    if level <= 50:
+        return 2
+    return 3
+
+
+def observation_key(observation: Observation) -> StateKey:
+    target_x, target_y, stage = _target_from_observation(observation)
+
+    if target_x < 0:
+        dist_bucket = 0
+    else:
+        dist = manhattan(
+            (observation.agent_location.x, observation.agent_location.y),
+            (target_x, target_y),
+        )
+        dist_bucket = min(20, dist)
+
+    pending_count = min(9, len(observation.pending_orders))
+
+    features = (
+        observation.grid_width,
+        observation.grid_height,
+        observation.agent_location.x,
+        observation.agent_location.y,
+        stage,
+        target_x,
+        target_y,
+        dist_bucket,
+        pending_count,
+        _battery_bucket(observation.battery_level),
+        _blocked_neighbor_count(observation),
+    )
+
+    return "|".join(str(value) for value in features)
+
+
+def action_from_index(action_index: int, observation: Observation) -> SimulatorAction:
+    if action_index == ACTION_UP:
+        return SimulatorAction(action_type=ActionType.MOVE, direction=Direction.UP)
+    if action_index == ACTION_DOWN:
+        return SimulatorAction(action_type=ActionType.MOVE, direction=Direction.DOWN)
+    if action_index == ACTION_LEFT:
+        return SimulatorAction(action_type=ActionType.MOVE, direction=Direction.LEFT)
+    if action_index == ACTION_RIGHT:
+        return SimulatorAction(action_type=ActionType.MOVE, direction=Direction.RIGHT)
+
+    if action_index == ACTION_ACCEPT:
+        nearest = _nearest_pending_order(observation)
+        if nearest is None:
+            return SimulatorAction(action_type=ActionType.WAIT)
+        return SimulatorAction(action_type=ActionType.ACCEPT_ORDER, order_id=nearest.order_id)
+
+    if action_index == ACTION_DELIVER:
+        if observation.current_order is None:
+            return SimulatorAction(action_type=ActionType.WAIT)
+        return SimulatorAction(action_type=ActionType.DELIVER_ORDER)
+
+    return SimulatorAction(action_type=ActionType.WAIT)
+
+
+class TrainedQAgent:
+    def __init__(self, q_table: QTable):
+        self.q_table = q_table
+
+    @classmethod
+    def load(cls, model_path: str) -> "TrainedQAgent":
+        with open(model_path, "r", encoding="utf-8") as handle:
+            payload = json.load(handle)
+        q_values = payload.get("q_values", {})
+        if not isinstance(q_values, dict):
+            raise ValueError("Invalid model file: q_values must be a dictionary")
+        return cls(q_table={str(key): [float(item) for item in value] for key, value in q_values.items()})
+
+    def act(self, observation: Observation) -> SimulatorAction:
+        key = observation_key(observation)
+        values = self.q_table.get(key)
+        if values is None or not values:
+            return action_from_index(ACTION_WAIT, observation)
+
+        best_index = max(range(min(NUM_ACTIONS, len(values))), key=lambda index: values[index])
+        return action_from_index(best_index, observation)

env/environment.py

@@ -0,0 +1,175 @@
+from __future__ import annotations
+
+from typing import Any, Dict, Optional, Tuple, Union
+
+from pydantic import ValidationError
+
+from env.models import (
+	Action,
+	ActionType,
+	EnvironmentConfig,
+	EnvironmentState,
+	Observation,
+	RewardConfig,
+	ScenarioDefinition,
+	SimulatorAction,
+	StepInfo,
+	StepResult,
+)
+from env.simulator import DeliverySimulator
+
+
+class LastMileDeliveryEnvironment:
+	"""OpenEnv-compatible environment for last-mile delivery optimization."""
+
+	def __init__(
+		self,
+		config: Optional[EnvironmentConfig] = None,
+		reward_config: Optional[RewardConfig] = None,
+	):
+		self.config = config or EnvironmentConfig()
+		self.reward_config = reward_config or RewardConfig()
+
+		self._simulator = DeliverySimulator(config=self.config, reward_config=self.reward_config)
+		self._initialized = False
+		self._total_reward = 0.0
+		self._last_observation: Optional[Observation] = None
+
+	def reset(self, seed: Optional[int] = None) -> Observation:
+		if seed is not None:
+			self.config.seed = seed
+
+		self._simulator.reset(seed=seed)
+		self._initialized = True
+		self._total_reward = 0.0
+		self._last_observation = self._simulator.build_observation(total_reward=self._total_reward)
+		return self._last_observation
+
+	def reset_with_scenario(self, scenario: ScenarioDefinition, seed: Optional[int] = None) -> Observation:
+		if seed is not None:
+			self.config.seed = seed
+
+		self._simulator.reset_with_scenario(scenario=scenario, seed=seed)
+		self._initialized = True
+		self._total_reward = 0.0
+		self._last_observation = self._simulator.build_observation(total_reward=self._total_reward)
+		return self._last_observation
+
+	def step(self, action: Union[Action, SimulatorAction, Dict[str, Any]]) -> Tuple[Observation, float, bool, Dict[str, Any]]:
+		"""OpenEnv step contract: returns (observation, reward, done, info)."""
+		if not self._initialized:
+			raise RuntimeError("Environment not initialized. Call reset() before step().")
+
+		if self._simulator.state.done:
+			observation = self._build_observation()
+			info = StepInfo(message="episode_already_done")
+			return observation, 0.0, True, info.model_dump()
+
+		validated_action, error_message = self._validate_action(action)
+		if validated_action is None:
+			reward, info = self._apply_invalid_action_penalty(error_message or "invalid_action_payload")
+		else:
+			reward, info = self._simulator.apply_action(validated_action)
+
+		self._total_reward += reward
+		observation = self._build_observation()
+
+		return observation, reward, self._simulator.state.done, info.model_dump()
+
+	def step_result(self, action: Union[Action, SimulatorAction, Dict[str, Any]]) -> StepResult:
+		"""Compatibility wrapper for existing API layers that consume StepResult."""
+		observation, reward, done, info = self.step(action)
+		return StepResult(
+			observation=observation,
+			reward=reward,
+			done=done,
+			info=StepInfo.model_validate(info),
+		)
+
+	def reset_state(self, seed: Optional[int] = None) -> EnvironmentState:
+		"""Compatibility helper for external runners expecting state immediately after reset."""
+		self.reset(seed=seed)
+		return self.state()
+
+	def current_observation(self) -> Observation:
+		if not self._initialized or self._last_observation is None:
+			raise RuntimeError("Environment not initialized. Call reset() first.")
+		return self._last_observation
+
+	def state(self) -> EnvironmentState:
+		if not self._initialized:
+			return EnvironmentState(
+				initialized=False,
+				done=False,
+				total_reward=0.0,
+				step_count=0,
+				delivered_orders=0,
+				current_order_id=None,
+				observation=None,
+			)
+
+		return EnvironmentState(
+			initialized=True,
+			done=self._simulator.state.done,
+			total_reward=self._total_reward,
+			step_count=self._simulator.state.step_count,
+			delivered_orders=self._simulator.state.delivered_orders,
+			current_order_id=self._simulator.state.current_order_id,
+			observation=self._build_observation(),
+		)
+
+	def _build_observation(self) -> Observation:
+		self._last_observation = self._simulator.build_observation(total_reward=self._total_reward)
+		return self._last_observation
+
+	def _apply_invalid_action_penalty(self, message: str) -> Tuple[float, StepInfo]:
+		# Invalid payloads still consume a step to keep episode progression consistent.
+		base_reward, base_info = self._simulator.apply_action(SimulatorAction(action_type=ActionType.WAIT))
+		reward = base_reward + self.reward_config.invalid_action_penalty
+		info = base_info.model_copy(deep=True)
+		info.invalid_action = True
+		info.message = message
+		info.destination_reached = False
+		info.delivered_order_id = None
+		return reward, info
+
+	def _validate_action(
+		self,
+		action: Union[Action, SimulatorAction, Dict[str, Any]],
+	) -> Tuple[Optional[SimulatorAction], Optional[str]]:
+		parsed: Optional[SimulatorAction] = None
+		if isinstance(action, SimulatorAction):
+			parsed = action
+		elif isinstance(action, Action):
+			try:
+				parsed = action.to_simulator_action()
+			except ValueError:
+				return None, "invalid_action_payload"
+		else:
+			try:
+				public_action = Action.model_validate(action)
+				parsed = public_action.to_simulator_action()
+			except ValidationError:
+				return None, "invalid_action_payload"
+			except ValueError:
+				return None, "invalid_action_payload"
+
+		if parsed.action_type == ActionType.MOVE:
+			if parsed.direction is None:
+				return None, "direction_required_for_move"
+			if parsed.order_id is not None:
+				return None, "order_id_not_allowed_for_move"
+
+		if parsed.action_type == ActionType.ACCEPT_ORDER and parsed.direction is not None:
+			return None, "direction_not_allowed_for_accept_order"
+
+		if parsed.action_type in {ActionType.DELIVER_ORDER, ActionType.WAIT}:
+			if parsed.direction is not None:
+				return None, "direction_not_allowed_for_action"
+			if parsed.order_id is not None:
+				return None, "order_id_not_allowed_for_action"
+
+		if parsed.action_type == ActionType.DELIVER_ORDER and self._simulator.state.current_order_id is None:
+			return None, "deliver_without_active_order"
+
+		return parsed, None

env/models.py

@@ -0,0 +1,335 @@
+from __future__ import annotations
+
+from enum import Enum
+from typing import Dict, List, Optional, Tuple
+
+from pydantic import BaseModel, ConfigDict, Field, model_validator
+
+
+Coordinate = Tuple[int, int]
+
+
+class Direction(str, Enum):
+	UP = "up"
+	DOWN = "down"
+	LEFT = "left"
+	RIGHT = "right"
+
+
+class MoveDirection(str, Enum):
+	UP = "up"
+	DOWN = "down"
+	LEFT = "left"
+	RIGHT = "right"
+	STAY = "stay"
+
+
+class ActionType(str, Enum):
+	MOVE = "move"
+	ACCEPT_ORDER = "accept_order"
+	DELIVER_ORDER = "deliver_order"
+	WAIT = "wait"
+
+
+class OrderStatus(str, Enum):
+	PENDING = "pending"
+	ACCEPTED = "accepted"
+	PICKED_UP = "picked_up"
+	DELIVERED = "delivered"
+
+
+class OrderPriority(str, Enum):
+	HIGH = "high"
+	LOW = "low"
+
+
+class Position(BaseModel):
+	x: int = Field(..., ge=0)
+	y: int = Field(..., ge=0)
+
+	def to_tuple(self) -> Coordinate:
+		return self.x, self.y
+
+	@classmethod
+	def from_tuple(cls, value: Coordinate) -> "Position":
+		return cls(x=value[0], y=value[1])
+
+
+class TrafficZone(BaseModel):
+	location: Position
+	extra_cost: int = Field(default=1, ge=1)
+
+
+class SimulatorAction(BaseModel):
+	action_type: ActionType
+	direction: Optional[Direction] = None
+	order_id: Optional[str] = None
+
+
+class Action(BaseModel):
+	model_config = ConfigDict(extra="forbid")
+
+	move: Optional[MoveDirection] = None
+	accept_order: Optional[int] = None
+	deliver_order: bool = False
+	wait: bool = False
+
+	@model_validator(mode="after")
+	def _validate_single_active_field(self) -> "Action":
+		if isinstance(self.accept_order, bool):
+			raise ValueError("accept_order must be an integer or null")
+
+		active_count = sum(
+			[
+				self.move is not None,
+				self.accept_order is not None,
+				self.deliver_order,
+				self.wait,
+			]
+		)
+		if active_count != 1:
+			raise ValueError("Exactly one action field must be active")
+		return self
+
+	def to_simulator_action(self) -> SimulatorAction:
+		if self.move is not None:
+			if self.move == MoveDirection.STAY:
+				return SimulatorAction(action_type=ActionType.WAIT)
+			return SimulatorAction(action_type=ActionType.MOVE, direction=Direction(self.move.value))
+
+		if self.accept_order is not None:
+			return SimulatorAction(action_type=ActionType.ACCEPT_ORDER, order_id=f"order_{self.accept_order}")
+
+		if self.deliver_order:
+			return SimulatorAction(action_type=ActionType.DELIVER_ORDER)
+
+		return SimulatorAction(action_type=ActionType.WAIT)
+
+	@classmethod
+	def from_simulator_action(cls, action: SimulatorAction) -> "Action":
+		if action.action_type == ActionType.MOVE:
+			if action.direction is None:
+				return cls(move=MoveDirection.STAY)
+			return cls(move=MoveDirection(action.direction.value))
+
+		if action.action_type == ActionType.ACCEPT_ORDER:
+			order_id = action.order_id or ""
+			if order_id.startswith("order_") and order_id[6:].isdigit():
+				order_number = int(order_id[6:])
+			else:
+				digits = "".join(ch for ch in order_id if ch.isdigit())
+				order_number = int(digits) if digits else 0
+			return cls(accept_order=order_number)
+
+		if action.action_type == ActionType.DELIVER_ORDER:
+			return cls(deliver_order=True)
+
+		return cls(wait=True)
+
+
+class Order(BaseModel):
+	order_id: str
+	pickup: Position
+	dropoff: Position
+	delivery_locations: List[Position] = Field(default_factory=list)
+	priority: OrderPriority = OrderPriority.LOW
+	created_step: int = 0
+	accepted_step: Optional[int] = None
+	picked_up_step: Optional[int] = None
+	delivered_step: Optional[int] = None
+	status: OrderStatus = OrderStatus.PENDING
+
+
+class Observation(BaseModel):
+	grid_width: int
+	grid_height: int
+	agent_location: Position
+	pending_orders: List[Order] = Field(default_factory=list)
+	current_order: Optional[Order] = None
+	obstacles: List[Position] = Field(default_factory=list)
+	dynamic_obstacles: List[Position] = Field(default_factory=list)
+	traffic_zones: List[TrafficZone] = Field(default_factory=list)
+	charging_stations: List[Position] = Field(default_factory=list)
+	battery_level: Optional[int] = None
+	step_count: int = 0
+	total_reward: float = 0.0
+
+
+class Reward(BaseModel):
+	value: float
+	components: Dict[str, float] = Field(default_factory=dict)
+
+
+class StepInfo(BaseModel):
+	invalid_action: bool = False
+	message: str = ""
+	destination_reached: bool = False
+	delivered_order_id: Optional[str] = None
+	delay_penalty_applied: bool = False
+	battery_depleted: bool = False
+	made_progress: Optional[bool] = None
+
+
+class StepResult(BaseModel):
+	observation: Observation
+	reward: float
+	done: bool
+	info: StepInfo
+
+
+class EnvironmentState(BaseModel):
+	initialized: bool
+	done: bool
+	total_reward: float
+	step_count: int
+	delivered_orders: int
+	current_order_id: Optional[str] = None
+	observation: Optional[Observation] = None
+
+
+class EnvironmentConfig(BaseModel):
+	width: int = Field(default=10, ge=2)
+	height: int = Field(default=10, ge=2)
+	max_steps: int = Field(default=200, ge=1)
+	max_orders: int = Field(default=5, ge=1)
+	delivery_locations_per_order: int = Field(default=1, ge=1, le=3)
+	priority_high_ratio: float = Field(default=0.4, ge=0.0, le=1.0)
+	obstacle_density: float = Field(default=0.1, ge=0.0, le=0.4)
+	dynamic_obstacles_enabled: bool = False
+	dynamic_obstacle_ratio: float = Field(default=0.25, ge=0.0, le=1.0)
+	traffic_density: float = Field(default=0.1, ge=0.0, le=0.4)
+	traffic_extra_cost: int = Field(default=2, ge=1)
+	battery_enabled: bool = False
+	battery_capacity: int = Field(default=100, ge=1)
+	battery_recharge_rate: int = Field(default=12, ge=1)
+	charging_stations: List[Position] = Field(default_factory=lambda: [Position(x=0, y=0)])
+	seed: Optional[int] = None
+
+
+class RewardConfig(BaseModel):
+	delivery_reward: float = 50.0
+	destination_reward: float = 10.0
+	step_penalty: float = -1.0
+	invalid_action_penalty: float = -20.0
+	delay_penalty: float = -5.0
+	progress_reward_scale: float = 1.25
+	progress_reward_clip: float = 2.0
+	delay_penalty_scale: float = 0.35
+	delay_penalty_cap: float = 6.0
+	efficiency_bonus_max: float = 12.0
+	traffic_penalty: float = -2.0
+	high_priority_delivery_bonus: float = 15.0
+	low_priority_delivery_bonus: float = 5.0
+	recharge_reward: float = 2.0
+	battery_failure_penalty: float = -30.0
+
+
+class ScenarioBatteryProfile(BaseModel):
+	enabled: bool = False
+	capacity: int = Field(default=100, ge=1)
+	recharge_rate: int = Field(default=12, ge=1)
+	initial_level: Optional[int] = Field(default=None, ge=0)
+
+	@model_validator(mode="after")
+	def _validate_initial_level(self) -> "ScenarioBatteryProfile":
+		if self.initial_level is not None and self.initial_level > self.capacity:
+			raise ValueError("battery_profile.initial_level cannot exceed capacity")
+		return self
+
+
+class ScenarioOrder(BaseModel):
+	order_id: Optional[str] = None
+	pickup: Position
+	dropoff: Position
+	delivery_locations: List[Position] = Field(default_factory=list)
+	priority: OrderPriority = OrderPriority.LOW
+	created_step: int = Field(default=0, ge=0)
+
+	@model_validator(mode="after")
+	def _ensure_delivery_locations(self) -> "ScenarioOrder":
+		if not self.delivery_locations:
+			self.delivery_locations = [self.dropoff]
+		return self
+
+
+class ScenarioDefinition(BaseModel):
+	width: int = Field(..., ge=2)
+	height: int = Field(..., ge=2)
+	max_steps: int = Field(default=200, ge=1)
+	agent_start: Position = Field(default_factory=lambda: Position(x=0, y=0))
+	orders: List[ScenarioOrder] = Field(..., min_length=1)
+	obstacles: List[Position] = Field(default_factory=list)
+	dynamic_obstacles: List[Position] = Field(default_factory=list)
+	traffic_zones: List[TrafficZone] = Field(default_factory=list)
+	charging_stations: List[Position] = Field(default_factory=lambda: [Position(x=0, y=0)])
+	battery_profile: ScenarioBatteryProfile = Field(default_factory=ScenarioBatteryProfile)
+
+	@model_validator(mode="after")
+	def _validate_geometry(self) -> "ScenarioDefinition":
+		def _in_bounds(position: Position) -> bool:
+			return 0 <= position.x < self.width and 0 <= position.y < self.height
+
+		def _dedupe(label: str, points: List[Position]) -> set[Coordinate]:
+			coordinates = [item.to_tuple() for item in points]
+			if len(set(coordinates)) != len(coordinates):
+				raise ValueError(f"{label} contains duplicate coordinates")
+			return set(coordinates)
+
+		agent_cell = self.agent_start.to_tuple()
+		if not _in_bounds(self.agent_start):
+			raise ValueError("agent_start must be within grid bounds")
+
+		static_cells = _dedupe("obstacles", self.obstacles)
+		dynamic_cells = _dedupe("dynamic_obstacles", self.dynamic_obstacles)
+		if static_cells.intersection(dynamic_cells):
+			raise ValueError("obstacles and dynamic_obstacles cannot overlap")
+
+		obstacle_cells = static_cells.union(dynamic_cells)
+		if agent_cell in obstacle_cells:
+			raise ValueError("agent_start cannot overlap an obstacle")
+
+		charging_cells = _dedupe("charging_stations", self.charging_stations)
+		if self.battery_profile.enabled and not charging_cells:
+			raise ValueError("charging_stations is required when battery_profile.enabled is true")
+		if charging_cells.intersection(obstacle_cells):
+			raise ValueError("charging_stations cannot overlap obstacles")
+
+		traffic_points = [zone.location for zone in self.traffic_zones]
+		traffic_cells = _dedupe("traffic_zones", traffic_points)
+		if traffic_cells.intersection(obstacle_cells):
+			raise ValueError("traffic_zones cannot overlap obstacles")
+
+		for label, points in [
+			("obstacles", self.obstacles),
+			("dynamic_obstacles", self.dynamic_obstacles),
+			("charging_stations", self.charging_stations),
+			("traffic_zones", traffic_points),
+		]:
+			for point in points:
+				if not _in_bounds(point):
+					raise ValueError(f"{label} contains out-of-bounds coordinates")
+
+		seen_order_ids: set[str] = set()
+		for index, order in enumerate(self.orders):
+			if order.order_id:
+				if order.order_id in seen_order_ids:
+					raise ValueError("orders contains duplicate order_id values")
+				seen_order_ids.add(order.order_id)
+
+			for point in [order.pickup, *order.delivery_locations]:
+				if not _in_bounds(point):
+					raise ValueError(f"orders[{index}] contains out-of-bounds coordinates")
+				if point.to_tuple() in obstacle_cells:
+					raise ValueError(f"orders[{index}] cannot overlap obstacles")
+
+		return self
+
+
+class SimulationState(BaseModel):
+	agent_location: Position = Field(default_factory=lambda: Position(x=0, y=0))
+	orders: Dict[str, Order] = Field(default_factory=dict)
+	current_order_id: Optional[str] = None
+	step_count: int = 0
+	battery_level: Optional[int] = None
+	done: bool = False
+	delivered_orders: int = 0

env/simulator.py

@@ -0,0 +1,622 @@
+from __future__ import annotations
+
+import random
+from typing import Dict, List, Optional, Set, Tuple
+
+from env.models import (
+    ActionType,
+    Coordinate,
+    Direction,
+    EnvironmentConfig,
+    Observation,
+    Order,
+    OrderPriority,
+    OrderStatus,
+    Position,
+    RewardConfig,
+    SimulatorAction,
+    SimulationState,
+    ScenarioDefinition,
+    StepInfo,
+    TrafficZone,
+)
+from env.utils import as_tuple, in_bounds, manhattan, move_coordinate, sample_locations, to_position_list
+
+
+class DeliverySimulator:
+    """Core deterministic simulator for last-mile delivery dynamics."""
+
+    def __init__(self, config: EnvironmentConfig, reward_config: Optional[RewardConfig] = None):
+        self.config = config
+        self.reward_config = reward_config or RewardConfig()
+        self._active_seed: Optional[int] = config.seed
+        self.rng = random.Random(self._active_seed)
+
+        self._static_obstacles: Set[Coordinate] = set()
+        self._dynamic_obstacles: Set[Coordinate] = set()
+        self._obstacles: Set[Coordinate] = set()
+        self._traffic: Dict[Coordinate, int] = {}
+        self._charging_stations: Set[Coordinate] = set()
+        self._state = SimulationState()
+
+    @property
+    def state(self) -> SimulationState:
+        return self._state
+
+    @property
+    def obstacles(self) -> Set[Coordinate]:
+        return self._obstacles
+
+    @property
+    def dynamic_obstacles(self) -> Set[Coordinate]:
+        return self._dynamic_obstacles
+
+    @property
+    def charging_stations(self) -> Set[Coordinate]:
+        return self._charging_stations
+
+    @property
+    def traffic(self) -> Dict[Coordinate, int]:
+        return self._traffic
+
+    def reset(self, seed: Optional[int] = None) -> SimulationState:
+        if seed is not None:
+            self._active_seed = seed
+            self.config.seed = seed
+        elif self.config.seed is not None and self.config.seed != self._active_seed:
+            self._active_seed = self.config.seed
+
+        self.rng.seed(self._active_seed)
+
+        configured_stations = {as_tuple(p) for p in self.config.charging_stations}
+        self._charging_stations = configured_stations or {(0, 0)}
+        self._generate_map()
+        self._state = SimulationState(
+            agent_location=Position(x=0, y=0),
+            orders=self._generate_orders(),
+            current_order_id=None,
+            step_count=0,
+            battery_level=self.config.battery_capacity if self.config.battery_enabled else None,
+            done=False,
+            delivered_orders=0,
+        )
+        return self._state
+
+    def reset_with_scenario(self, scenario: ScenarioDefinition, seed: Optional[int] = None) -> SimulationState:
+        if seed is not None:
+            self._active_seed = seed
+            self.config.seed = seed
+        elif self.config.seed is not None and self.config.seed != self._active_seed:
+            self._active_seed = self.config.seed
+
+        self.rng.seed(self._active_seed)
+
+        self.config.width = scenario.width
+        self.config.height = scenario.height
+        self.config.max_steps = scenario.max_steps
+        self.config.max_orders = max(1, len(scenario.orders))
+        self.config.dynamic_obstacles_enabled = bool(scenario.dynamic_obstacles)
+        self.config.dynamic_obstacle_ratio = 1.0 if scenario.dynamic_obstacles else 0.0
+        self.config.obstacle_density = 0.0
+        self.config.traffic_density = 0.0
+        self.config.battery_enabled = scenario.battery_profile.enabled
+        self.config.battery_capacity = scenario.battery_profile.capacity
+        self.config.battery_recharge_rate = scenario.battery_profile.recharge_rate
+        self.config.charging_stations = [item.model_copy(deep=True) for item in scenario.charging_stations]
+
+        self._static_obstacles = {as_tuple(item) for item in scenario.obstacles}
+        self._dynamic_obstacles = {as_tuple(item) for item in scenario.dynamic_obstacles}
+        self._obstacles = self._static_obstacles.union(self._dynamic_obstacles)
+        self._traffic = {
+            as_tuple(zone.location): zone.extra_cost
+            for zone in scenario.traffic_zones
+        }
+        self._charging_stations = {as_tuple(item) for item in scenario.charging_stations} or {(0, 0)}
+
+        initial_battery_level: Optional[int] = None
+        if scenario.battery_profile.enabled:
+            initial_level = scenario.battery_profile.initial_level
+            initial_battery_level = (
+                scenario.battery_profile.capacity
+                if initial_level is None
+                else initial_level
+            )
+
+        self._state = SimulationState(
+            agent_location=scenario.agent_start.model_copy(deep=True),
+            orders=self._orders_from_scenario(scenario),
+            current_order_id=None,
+            step_count=0,
+            battery_level=initial_battery_level,
+            done=False,
+            delivered_orders=0,
+        )
+        return self._state
+
+    def build_observation(self, total_reward: float = 0.0) -> Observation:
+        pending_orders = [
+            order for order in self._state.orders.values() if order.status != OrderStatus.DELIVERED
+        ]
+        current_order = None
+        if self._state.current_order_id is not None:
+            current_order = self._state.orders.get(self._state.current_order_id)
+
+        traffic_zones = [
+            TrafficZone(location=Position.from_tuple(loc), extra_cost=cost)
+            for loc, cost in self._traffic.items()
+        ]
+
+        return Observation(
+            grid_width=self.config.width,
+            grid_height=self.config.height,
+            agent_location=self._state.agent_location,
+            pending_orders=pending_orders,
+            current_order=current_order,
+            obstacles=to_position_list(self._obstacles),
+            dynamic_obstacles=to_position_list(self._dynamic_obstacles),
+            traffic_zones=traffic_zones,
+            charging_stations=to_position_list(self._charging_stations),
+            battery_level=self._state.battery_level,
+            step_count=self._state.step_count,
+            total_reward=total_reward,
+        )
+
+    def apply_action(self, action: SimulatorAction) -> Tuple[float, StepInfo]:
+        if self._state.done:
+            return 0.0, StepInfo(invalid_action=True, message="episode_already_done")
+
+        reward = self.reward_config.step_penalty
+        info = StepInfo()
+        self._state.step_count += 1
+
+        if action.action_type == ActionType.MOVE:
+            delta_reward, step_info = self._handle_move(action.direction)
+            reward += delta_reward
+            info = step_info
+        elif action.action_type == ActionType.ACCEPT_ORDER:
+            delta_reward, step_info = self._handle_accept(action.order_id)
+            reward += delta_reward
+            info = step_info
+        elif action.action_type == ActionType.DELIVER_ORDER:
+            delta_reward, step_info = self._handle_deliver()
+            reward += delta_reward
+            info = step_info
+        elif action.action_type == ActionType.WAIT:
+            delta_reward, step_info = self._handle_wait()
+            reward += delta_reward
+            info = step_info
+        else:
+            reward += self.reward_config.invalid_action_penalty
+            info = StepInfo(invalid_action=True, message="unknown_action")
+
+        if self.config.dynamic_obstacles_enabled and self._dynamic_obstacles:
+            self._update_dynamic_obstacles()
+
+        delay_penalty = self._compute_delay_penalty()
+        if delay_penalty < 0:
+            reward += delay_penalty
+            info.delay_penalty_applied = True
+
+        if self._state.battery_level is not None and self._state.battery_level <= 0:
+            self._state.done = True
+            reward += self.reward_config.battery_failure_penalty
+            info.battery_depleted = True
+
+        if self._state.step_count >= self.config.max_steps:
+            self._state.done = True
+
+        if self._all_delivered():
+            self._state.done = True
+
+        return reward, info
+
+    def _all_delivered(self) -> bool:
+        return all(order.status == OrderStatus.DELIVERED for order in self._state.orders.values())
+
+    def _generate_map(self) -> None:
+        blocked: Set[Coordinate] = {(0, 0)}.union(self._charging_stations)
+
+        obstacle_count = int(self.config.width * self.config.height * self.config.obstacle_density)
+        traffic_count = int(self.config.width * self.config.height * self.config.traffic_density)
+
+        obstacle_locations = set(
+            sample_locations(
+                self.rng,
+                self.config.width,
+                self.config.height,
+                obstacle_count,
+                blocked=blocked,
+            )
+        )
+
+        if self.config.dynamic_obstacles_enabled and obstacle_locations:
+            dynamic_count = int(len(obstacle_locations) * self.config.dynamic_obstacle_ratio)
+            dynamic_count = min(dynamic_count, len(obstacle_locations))
+            dynamic_items = self.rng.sample(sorted(obstacle_locations), dynamic_count) if dynamic_count > 0 else []
+            self._dynamic_obstacles = set(dynamic_items)
+        else:
+            self._dynamic_obstacles = set()
+
+        self._static_obstacles = obstacle_locations - self._dynamic_obstacles
+        self._obstacles = self._static_obstacles.union(self._dynamic_obstacles)
+        blocked = blocked.union(self._obstacles)
+
+        traffic_locations = sample_locations(
+            self.rng,
+            self.config.width,
+            self.config.height,
+            traffic_count,
+            blocked=blocked,
+        )
+        self._traffic = {loc: self.config.traffic_extra_cost for loc in traffic_locations}
+
+    def _generate_orders(self) -> Dict[str, Order]:
+        blocked = {(0, 0)}.union(self._charging_stations).union(self._obstacles)
+        points_per_order = 1 + self.config.delivery_locations_per_order
+        required_locations = self.config.max_orders * points_per_order
+
+        order_points = sample_locations(
+            self.rng,
+            self.config.width,
+            self.config.height,
+            required_locations,
+            blocked=blocked,
+        )
+
+        orders: Dict[str, Order] = {}
+        index = 0
+        order_number = 1
+        while index < len(order_points):
+            if index + points_per_order > len(order_points):
+                break
+
+            pickup_point = order_points[index]
+            delivery_points = order_points[index + 1 : index + points_per_order]
+            if not delivery_points:
+                break
+
+            priority = (
+                OrderPriority.HIGH
+                if self.rng.random() < self.config.priority_high_ratio
+                else OrderPriority.LOW
+            )
+
+            order_id = f"order_{order_number}"
+            orders[order_id] = Order(
+                order_id=order_id,
+                pickup=Position.from_tuple(pickup_point),
+                dropoff=Position.from_tuple(delivery_points[0]),
+                delivery_locations=[Position.from_tuple(loc) for loc in delivery_points],
+                priority=priority,
+                created_step=0,
+                status=OrderStatus.PENDING,
+            )
+
+            index += points_per_order
+            order_number += 1
+
+        return orders
+
+    def _orders_from_scenario(self, scenario: ScenarioDefinition) -> Dict[str, Order]:
+        orders: Dict[str, Order] = {}
+        for index, scenario_order in enumerate(scenario.orders, start=1):
+            order_id = scenario_order.order_id or f"order_{index}"
+            if order_id in orders:
+                raise ValueError(f"Duplicate scenario order_id '{order_id}'")
+
+            delivery_locations = scenario_order.delivery_locations or [scenario_order.dropoff]
+            orders[order_id] = Order(
+                order_id=order_id,
+                pickup=scenario_order.pickup.model_copy(deep=True),
+                dropoff=scenario_order.dropoff.model_copy(deep=True),
+                delivery_locations=[item.model_copy(deep=True) for item in delivery_locations],
+                priority=scenario_order.priority,
+                created_step=scenario_order.created_step,
+                status=OrderStatus.PENDING,
+            )
+
+        return orders
+
+    def _update_dynamic_obstacles(self) -> None:
+        reserved = {as_tuple(self._state.agent_location)}.union(self._charging_stations)
+        for order in self._state.orders.values():
+            if order.status == OrderStatus.DELIVERED:
+                continue
+            reserved.add(as_tuple(order.pickup))
+            targets = order.delivery_locations or [order.dropoff]
+            reserved.update(as_tuple(target) for target in targets)
+
+        updated_dynamic: Set[Coordinate] = set()
+
+        for obstacle in sorted(self._dynamic_obstacles):
+            candidates = [
+                obstacle,
+                move_coordinate(obstacle, Direction.UP),
+                move_coordinate(obstacle, Direction.DOWN),
+                move_coordinate(obstacle, Direction.LEFT),
+                move_coordinate(obstacle, Direction.RIGHT),
+            ]
+
+            valid_candidates: List[Coordinate] = []
+            for candidate in candidates:
+                if not in_bounds(candidate, self.config.width, self.config.height):
+                    continue
+                if candidate in self._static_obstacles:
+                    continue
+                if candidate in updated_dynamic:
+                    continue
+                if candidate in reserved:
+                    continue
+                valid_candidates.append(candidate)
+
+            if not valid_candidates:
+                updated_dynamic.add(obstacle)
+                continue
+
+            selected = valid_candidates[self.rng.randrange(len(valid_candidates))]
+            updated_dynamic.add(selected)
+
+        self._dynamic_obstacles = updated_dynamic
+        self._obstacles = self._static_obstacles.union(self._dynamic_obstacles)
+
+    def _consume_battery(self, amount: int) -> None:
+        if self._state.battery_level is None:
+            return
+        self._state.battery_level = max(0, self._state.battery_level - amount)
+
+    def _handle_move(self, direction: Optional[Direction]) -> Tuple[float, StepInfo]:
+        if direction is None:
+            return self.reward_config.invalid_action_penalty, StepInfo(
+                invalid_action=True,
+                message="direction_required",
+            )
+
+        current = as_tuple(self._state.agent_location)
+        progress_target = self._get_progress_target(current)
+        nxt = move_coordinate(current, direction)
+        if not in_bounds(nxt, self.config.width, self.config.height):
+            return self.reward_config.invalid_action_penalty, StepInfo(
+                invalid_action=True,
+                message="out_of_bounds",
+            )
+
+        if nxt in self._obstacles:
+            return self.reward_config.invalid_action_penalty, StepInfo(
+                invalid_action=True,
+                message="hit_obstacle",
+            )
+
+        self._state.agent_location = Position.from_tuple(nxt)
+        battery_cost = 1 + self._traffic.get(nxt, 0)
+        self._consume_battery(battery_cost)
+
+        progress_reward, progress_delta = self._progress_shaping_reward(
+            previous_location=current,
+            new_location=nxt,
+            target=progress_target,
+        )
+
+        info = StepInfo(message="moved", made_progress=progress_delta > 0)
+        reward = 0.0
+
+        if nxt in self._traffic:
+            reward += self.reward_config.traffic_penalty
+            info.message = "moved_in_traffic"
+
+        reward += progress_reward
+
+        destination_reached = self._check_destination_reached()
+        info.destination_reached = destination_reached
+        if destination_reached:
+            reward += self.reward_config.destination_reward
+            info.made_progress = True
+
+        return reward, info
+
+    def _check_destination_reached(self) -> bool:
+        if self._state.current_order_id is None:
+            return False
+
+        order = self._state.orders[self._state.current_order_id]
+        current = as_tuple(self._state.agent_location)
+
+        if order.status == OrderStatus.ACCEPTED and current == as_tuple(order.pickup):
+            order.status = OrderStatus.PICKED_UP
+            order.picked_up_step = self._state.step_count
+            return True
+
+        target_locations = order.delivery_locations or [order.dropoff]
+        if order.status == OrderStatus.PICKED_UP and any(current == as_tuple(target) for target in target_locations):
+            return True
+
+        return False
+
+    def _handle_wait(self) -> Tuple[float, StepInfo]:
+        info = StepInfo(message="wait", made_progress=False)
+        reward = 0.0
+
+        if self._state.battery_level is None:
+            return reward, info
+
+        location = as_tuple(self._state.agent_location)
+        if location not in self._charging_stations:
+            return reward, info
+
+        if self._state.battery_level >= self.config.battery_capacity:
+            info.message = "wait_full_battery"
+            return reward, info
+
+        before = self._state.battery_level
+        self._state.battery_level = min(
+            self.config.battery_capacity,
+            self._state.battery_level + self.config.battery_recharge_rate,
+        )
+        gained = self._state.battery_level - before
+
+        if gained > 0:
+            reward += self.reward_config.recharge_reward
+            info.message = "recharged"
+            info.made_progress = True
+
+        return reward, info
+
+    def _handle_accept(self, order_id: Optional[str]) -> Tuple[float, StepInfo]:
+        if self._state.current_order_id is not None:
+            return self.reward_config.invalid_action_penalty, StepInfo(
+                invalid_action=True,
+                message="current_order_already_assigned",
+            )
+
+        selected_id = order_id
+        if selected_id is None:
+            pending_orders = [
+                order
+                for order in self._state.orders.values()
+                if order.status == OrderStatus.PENDING
+            ]
+            pending_orders.sort(
+                key=lambda order: (
+                    0 if order.priority == OrderPriority.HIGH else 1,
+                    order.order_id,
+                )
+            )
+            pending_ids = [order.order_id for order in pending_orders]
+            if not pending_ids:
+                return self.reward_config.invalid_action_penalty, StepInfo(
+                    invalid_action=True,
+                    message="no_pending_order",
+                )
+            selected_id = pending_ids[0]
+
+        order = self._state.orders.get(selected_id)
+        if order is None or order.status != OrderStatus.PENDING:
+            return self.reward_config.invalid_action_penalty, StepInfo(
+                invalid_action=True,
+                message="order_not_available",
+            )
+
+        order.status = OrderStatus.ACCEPTED
+        order.accepted_step = self._state.step_count
+        self._state.current_order_id = selected_id
+        self._consume_battery(1)
+        return 0.0, StepInfo(message="order_accepted", made_progress=True)
+
+    def _handle_deliver(self) -> Tuple[float, StepInfo]:
+        if self._state.current_order_id is None:
+            return self.reward_config.invalid_action_penalty, StepInfo(
+                invalid_action=True,
+                message="no_current_order",
+            )
+
+        order = self._state.orders[self._state.current_order_id]
+        current = as_tuple(self._state.agent_location)
+        if order.status == OrderStatus.ACCEPTED and current == as_tuple(order.pickup):
+            order.status = OrderStatus.PICKED_UP
+            order.picked_up_step = self._state.step_count
+            self._consume_battery(1)
+            return self.reward_config.destination_reward, StepInfo(
+                message="order_picked_up",
+                destination_reached=True,
+                made_progress=True,
+            )
+
+        valid_dropoffs = order.delivery_locations or [order.dropoff]
+        is_valid_dropoff = any(current == as_tuple(target) for target in valid_dropoffs)
+        if order.status != OrderStatus.PICKED_UP or not is_valid_dropoff:
+            return self.reward_config.invalid_action_penalty, StepInfo(
+                invalid_action=True,
+                message="not_at_dropoff_or_not_picked_up",
+            )
+
+        order.status = OrderStatus.DELIVERED
+        order.delivered_step = self._state.step_count
+        self._state.delivered_orders += 1
+        delivered_id = order.order_id
+        self._state.current_order_id = None
+        self._consume_battery(1)
+
+        priority_bonus = (
+            self.reward_config.high_priority_delivery_bonus
+            if order.priority == OrderPriority.HIGH
+            else self.reward_config.low_priority_delivery_bonus
+        )
+
+        efficiency_bonus = self._delivery_efficiency_bonus(order)
+
+        return self.reward_config.delivery_reward + priority_bonus + efficiency_bonus, StepInfo(
+            message=f"order_delivered_{order.priority.value}",
+            delivered_order_id=delivered_id,
+            made_progress=True,
+        )
+
+    def _get_progress_target(self, from_location: Coordinate) -> Optional[Coordinate]:
+        if self._state.current_order_id is not None:
+            current_order = self._state.orders[self._state.current_order_id]
+            if current_order.status == OrderStatus.ACCEPTED:
+                return as_tuple(current_order.pickup)
+            if current_order.status == OrderStatus.PICKED_UP:
+                targets = current_order.delivery_locations or [current_order.dropoff]
+                target_coords = [as_tuple(target) for target in targets]
+                return min(target_coords, key=lambda t: manhattan(from_location, t))
+
+        pending_orders = [
+            order
+            for order in self._state.orders.values()
+            if order.status == OrderStatus.PENDING
+        ]
+        if not pending_orders:
+            return None
+
+        pickup_targets = [as_tuple(order.pickup) for order in pending_orders]
+        return min(pickup_targets, key=lambda t: manhattan(from_location, t))
+
+    def _progress_shaping_reward(
+        self,
+        previous_location: Coordinate,
+        new_location: Coordinate,
+        target: Optional[Coordinate],
+    ) -> Tuple[float, int]:
+        if target is None:
+            return 0.0, 0
+
+        old_distance = manhattan(previous_location, target)
+        new_distance = manhattan(new_location, target)
+        distance_delta = old_distance - new_distance
+
+        raw_reward = self.reward_config.progress_reward_scale * distance_delta
+        clip_value = self.reward_config.progress_reward_clip
+        shaped_reward = max(-clip_value, min(clip_value, raw_reward))
+        return shaped_reward, distance_delta
+
+    def _delivery_efficiency_bonus(self, order: Order) -> float:
+        if order.accepted_step is None:
+            return 0.0
+
+        elapsed_steps = max(1, self._state.step_count - order.accepted_step)
+        delivery_targets = order.delivery_locations or [order.dropoff]
+        travel_lower_bound = min(
+            manhattan(as_tuple(order.pickup), as_tuple(target))
+            for target in delivery_targets
+        )
+
+        # Two control actions are minimally required: pickup and final deliver.
+        expected_steps = max(2, travel_lower_bound + 2)
+        efficiency_ratio = min(1.0, expected_steps / elapsed_steps)
+        return self.reward_config.efficiency_bonus_max * efficiency_ratio
+
+    def _compute_delay_penalty(self) -> float:
+        if self._state.current_order_id is None:
+            return 0.0
+        order = self._state.orders[self._state.current_order_id]
+        if order.accepted_step is None:
+            return 0.0
+
+        elapsed = self._state.step_count - order.accepted_step
+        threshold = 12 if order.priority == OrderPriority.HIGH else 20
+        overdue_steps = elapsed - threshold
+        if overdue_steps <= 0:
+            return 0.0
+
+        penalty = overdue_steps * self.reward_config.delay_penalty_scale
+        penalty = min(self.reward_config.delay_penalty_cap, penalty)
+        return -penalty

env/utils.py

@@ -0,0 +1,58 @@
+from __future__ import annotations
+
+import random
+from typing import Iterable, List, Optional, Sequence, Set, Tuple
+
+from env.models import Coordinate, Direction, Position
+
+
+def move_coordinate(location: Coordinate, direction: Direction) -> Coordinate:
+	x, y = location
+	if direction == Direction.UP:
+		return x, y - 1
+	if direction == Direction.DOWN:
+		return x, y + 1
+	if direction == Direction.LEFT:
+		return x - 1, y
+	return x + 1, y
+
+
+def in_bounds(location: Coordinate, width: int, height: int) -> bool:
+	return 0 <= location[0] < width and 0 <= location[1] < height
+
+
+def manhattan(a: Coordinate, b: Coordinate) -> int:
+	return abs(a[0] - b[0]) + abs(a[1] - b[1])
+
+
+def to_position_list(coordinates: Iterable[Coordinate]) -> List[Position]:
+	return [Position(x=x, y=y) for x, y in coordinates]
+
+
+def sample_locations(
+	rng: random.Random,
+	width: int,
+	height: int,
+	count: int,
+	blocked: Optional[Set[Coordinate]] = None,
+) -> List[Coordinate]:
+	blocked = blocked or set()
+	available: List[Coordinate] = [
+		(x, y)
+		for x in range(width)
+		for y in range(height)
+		if (x, y) not in blocked
+	]
+	if count > len(available):
+		count = len(available)
+	return rng.sample(available, count)
+
+
+def nearest_location(origin: Coordinate, candidates: Sequence[Coordinate]) -> Optional[Coordinate]:
+	if not candidates:
+		return None
+	return min(candidates, key=lambda c: manhattan(origin, c))
+
+
+def as_tuple(position: Position) -> Coordinate:
+	return position.x, position.y

grader/grader.py

@@ -0,0 +1,530 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Any, Dict, List, Sequence
+
+from pydantic import BaseModel, Field
+
+from env.models import Observation, Order, OrderPriority, StepResult
+from grader.metrics import (
+    clamp,
+    round_score,
+)
+
+
+DEFAULT_SUCCESS_CONDITION: Dict[str, Any] = {
+    "completion_rate_min": 1.0,
+    "max_steps": 200,
+    "invalid_action_rate_max": 0.10,
+}
+DEFAULT_HIGH_PRIORITY_DEADLINE = 12
+
+
+class GradeReport(BaseModel):
+    score: float = Field(..., ge=0.0, le=1.0)
+
+    delivered_orders: int = Field(..., ge=0)
+    total_orders: int = Field(..., ge=0)
+    high_priority_total_orders: int = Field(..., ge=0)
+    high_priority_delivered_orders: int = Field(..., ge=0)
+    high_priority_on_time_deliveries: int = Field(..., ge=0)
+
+    steps_taken: int = Field(..., ge=0)
+    max_steps_target: int = Field(..., ge=0)
+    optimal_steps: int = Field(..., ge=0)
+
+    completion_rate: float = Field(..., ge=0.0, le=1.0)
+    high_priority_on_time_rate: float = Field(..., ge=0.0, le=1.0)
+    efficiency_ratio: float = Field(..., ge=0.0, le=1.0)
+    invalid_action_rate: float = Field(..., ge=0.0, le=1.0)
+
+    completion_component: float = Field(..., ge=0.0, le=1.0)
+    priority_component: float = Field(..., ge=0.0, le=1.0)
+    efficiency_component: float = Field(..., ge=0.0, le=1.0)
+    penalty_component: float = Field(..., ge=0.0, le=1.0)
+
+    invalid_actions: int = Field(..., ge=0)
+    delay_events: int = Field(..., ge=0)
+    battery_depletion_events: int = Field(..., ge=0)
+    no_progress_events: int = Field(..., ge=0)
+    battery_remaining: int | None = None
+
+    success_condition_used: Dict[str, Any] = Field(default_factory=dict)
+    scoring_logic: str
+    safeguards_applied: List[str] = Field(default_factory=list)
+    edge_case_handling: List[str] = Field(default_factory=list)
+
+
+@dataclass(frozen=True)
+class EpisodeStats:
+    steps_taken: int
+    invalid_actions: int
+    delay_events: int
+    battery_depletion_events: int
+    no_progress_events: int
+
+
+@dataclass(frozen=True)
+class SuccessTargets:
+    completion_target: float
+    priority_target: float | None
+    max_steps_target: int
+    invalid_action_rate_max: float
+    require_no_battery_depletion: bool
+    high_priority_deadline_steps: int
+
+
+class DeliveryEpisodeGrader:
+    """Deterministic task-aware grader aligned with task success_condition metrics."""
+
+    def __init__(self) -> None:
+        self.base_weights = {
+            "completion": 0.55,
+            "priority": 0.25,
+            "efficiency": 0.20,
+        }
+
+    def grade_episode(
+        self,
+        trajectory: Sequence[StepResult],
+        final_observation: Observation,
+        success_condition: Dict[str, Any] | None = None,
+    ) -> GradeReport:
+        order_catalog = self._collect_order_catalog(trajectory, final_observation)
+        delivered_steps = self._collect_delivered_steps(trajectory)
+        delivered_ids = set(delivered_steps.keys())
+        delivered_orders = len(delivered_ids)
+
+        remaining_ids = {order.order_id for order in final_observation.pending_orders}
+        if final_observation.current_order is not None:
+            remaining_ids.add(final_observation.current_order.order_id)
+
+        known_ids = set(order_catalog.keys())
+        all_ids = known_ids.union(delivered_ids).union(remaining_ids)
+        total_orders = len(all_ids)
+
+        episode_stats = self._collect_episode_stats(trajectory)
+
+        edge_case_notes: List[str] = []
+        safeguards: List[str] = []
+
+        targets = self._resolve_success_targets(
+            success_condition=success_condition,
+            fallback_max_steps=max(1, final_observation.step_count),
+            edge_case_notes=edge_case_notes,
+        )
+
+        if episode_stats.steps_taken == 0:
+            edge_case_notes.append("No steps were recorded for this episode.")
+
+        if total_orders == 0:
+            completion_rate_value = 1.0
+            edge_case_notes.append("No orders were present; completion_rate set to 1.0 by convention.")
+        else:
+            completion_rate_value = delivered_orders / total_orders
+
+        (
+            high_priority_total,
+            high_priority_delivered,
+            high_priority_on_time,
+            high_priority_on_time_rate,
+        ) = self._high_priority_metrics(
+            order_catalog=order_catalog,
+            delivered_steps=delivered_steps,
+            deadline_steps=targets.high_priority_deadline_steps,
+            edge_case_notes=edge_case_notes,
+        )
+
+        efficiency_ratio = self._efficiency_ratio(
+            steps_taken=episode_stats.steps_taken,
+            max_steps_target=targets.max_steps_target,
+        )
+
+        invalid_action_rate = self._invalid_action_rate(
+            invalid_actions=episode_stats.invalid_actions,
+            steps_taken=episode_stats.steps_taken,
+        )
+
+        completion_component = self._target_progress(
+            value=completion_rate_value,
+            target=targets.completion_target,
+            edge_case_notes=edge_case_notes,
+            metric_name="completion_rate",
+        )
+
+        if targets.priority_target is None:
+            priority_component = 1.0
+            edge_case_notes.append("high_priority_on_time_rate_min not set; priority component treated as neutral.")
+        else:
+            priority_component = self._target_progress(
+                value=high_priority_on_time_rate,
+                target=targets.priority_target,
+                edge_case_notes=edge_case_notes,
+                metric_name="high_priority_on_time_rate",
+            )
+
+        efficiency_component = efficiency_ratio
+        penalty_component = self._invalid_action_penalty_component(
+            invalid_action_rate=invalid_action_rate,
+            invalid_action_rate_max=targets.invalid_action_rate_max,
+            edge_case_notes=edge_case_notes,
+        )
+
+        if targets.require_no_battery_depletion and episode_stats.battery_depletion_events > 0:
+            penalty_component *= 0.5
+            safeguards.append("battery_depletion_penalty_applied")
+
+        completion_weight = self.base_weights["completion"]
+        priority_weight = self.base_weights["priority"] if targets.priority_target is not None else 0.0
+        efficiency_weight = self.base_weights["efficiency"]
+        if targets.priority_target is None:
+            completion_weight += 0.15
+            efficiency_weight += 0.10
+
+        raw_base = (
+            completion_weight * completion_component
+            + priority_weight * priority_component
+            + efficiency_weight * efficiency_component
+        )
+        raw_score = raw_base * penalty_component
+
+        score = round_score(raw_score, decimals=4)
+
+        logic = (
+            "metrics follow success_condition keys: completion_rate(_min), "
+            "high_priority_on_time_rate_min, max_steps, invalid_action_rate_max. "
+            "components: completion (highest), priority SLA (medium when configured), "
+            "efficiency from steps/max_steps (lower), and penalties reduce via invalid_action_rate."
+        )
+
+        if not edge_case_notes:
+            edge_case_notes.append("No edge-case adjustments were needed.")
+
+        success_condition_used = {
+            "completion_rate_target": targets.completion_target,
+            "high_priority_on_time_rate_target": targets.priority_target,
+            "max_steps": targets.max_steps_target,
+            "invalid_action_rate_max": targets.invalid_action_rate_max,
+            "battery_depletion": not targets.require_no_battery_depletion,
+            "high_priority_deadline_steps": targets.high_priority_deadline_steps,
+        }
+
+        return GradeReport(
+            score=score,
+            delivered_orders=delivered_orders,
+            total_orders=total_orders,
+            high_priority_total_orders=high_priority_total,
+            high_priority_delivered_orders=high_priority_delivered,
+            high_priority_on_time_deliveries=high_priority_on_time,
+            steps_taken=episode_stats.steps_taken,
+            max_steps_target=targets.max_steps_target,
+            optimal_steps=targets.max_steps_target,
+            completion_rate=completion_rate_value,
+            high_priority_on_time_rate=high_priority_on_time_rate,
+            efficiency_ratio=efficiency_ratio,
+            invalid_action_rate=invalid_action_rate,
+            completion_component=completion_component,
+            priority_component=priority_component,
+            efficiency_component=efficiency_component,
+            penalty_component=penalty_component,
+            invalid_actions=episode_stats.invalid_actions,
+            delay_events=episode_stats.delay_events,
+            battery_depletion_events=episode_stats.battery_depletion_events,
+            no_progress_events=episode_stats.no_progress_events,
+            battery_remaining=final_observation.battery_level,
+            success_condition_used=success_condition_used,
+            scoring_logic=logic,
+            safeguards_applied=safeguards,
+            edge_case_handling=edge_case_notes,
+        )
+
+    def _resolve_success_targets(
+        self,
+        success_condition: Dict[str, Any] | None,
+        fallback_max_steps: int,
+        edge_case_notes: List[str],
+    ) -> SuccessTargets:
+        condition = dict(DEFAULT_SUCCESS_CONDITION)
+        if success_condition is not None:
+            condition.update(success_condition)
+
+        completion_raw = condition.get("completion_rate")
+        if completion_raw is None:
+            completion_raw = condition.get("completion_rate_min", 1.0)
+
+        completion_target = self._safe_ratio_target(
+            raw_value=completion_raw,
+            default=1.0,
+            metric_name="completion_rate_target",
+            edge_case_notes=edge_case_notes,
+        )
+
+        priority_raw = condition.get("high_priority_on_time_rate_min")
+        priority_target: float | None = None
+        if priority_raw is not None:
+            priority_target = self._safe_ratio_target(
+                raw_value=priority_raw,
+                default=1.0,
+                metric_name="high_priority_on_time_rate_target",
+                edge_case_notes=edge_case_notes,
+            )
+
+        max_steps_raw = condition.get("max_steps", fallback_max_steps)
+        try:
+            max_steps_target = int(max_steps_raw)
+        except Exception:
+            max_steps_target = fallback_max_steps
+            edge_case_notes.append("Invalid max_steps in success_condition; fallback max_steps used.")
+        if max_steps_target <= 0:
+            max_steps_target = max(1, fallback_max_steps)
+            edge_case_notes.append("Non-positive max_steps in success_condition; fallback max_steps used.")
+
+        invalid_rate_raw = condition.get("invalid_action_rate_max", 0.10)
+        invalid_action_rate_max = self._safe_ratio_target(
+            raw_value=invalid_rate_raw,
+            default=0.10,
+            metric_name="invalid_action_rate_max",
+            edge_case_notes=edge_case_notes,
+        )
+
+        battery_depletion_rule = condition.get("battery_depletion")
+        require_no_battery_depletion = battery_depletion_rule is False
+
+        deadline_raw = condition.get("high_priority_deadline_steps", DEFAULT_HIGH_PRIORITY_DEADLINE)
+        try:
+            deadline_steps = int(deadline_raw)
+        except Exception:
+            deadline_steps = DEFAULT_HIGH_PRIORITY_DEADLINE
+            edge_case_notes.append("Invalid high_priority_deadline_steps; default deadline used.")
+        if deadline_steps <= 0:
+            deadline_steps = DEFAULT_HIGH_PRIORITY_DEADLINE
+            edge_case_notes.append("Non-positive high_priority_deadline_steps; default deadline used.")
+
+        return SuccessTargets(
+            completion_target=completion_target,
+            priority_target=priority_target,
+            max_steps_target=max_steps_target,
+            invalid_action_rate_max=invalid_action_rate_max,
+            require_no_battery_depletion=require_no_battery_depletion,
+            high_priority_deadline_steps=deadline_steps,
+        )
+
+    def _safe_ratio_target(
+        self,
+        raw_value: Any,
+        default: float,
+        metric_name: str,
+        edge_case_notes: List[str],
+    ) -> float:
+        try:
+            parsed = float(raw_value)
+        except Exception:
+            edge_case_notes.append(f"Invalid {metric_name}; default value used.")
+            return default
+        return clamp(parsed, 0.0, 1.0)
+
+    def _collect_episode_stats(self, trajectory: Sequence[StepResult]) -> EpisodeStats:
+        return EpisodeStats(
+            steps_taken=len(trajectory),
+            invalid_actions=sum(1 for item in trajectory if item.info.invalid_action),
+            delay_events=sum(1 for item in trajectory if item.info.delay_penalty_applied),
+            battery_depletion_events=sum(1 for item in trajectory if item.info.battery_depleted),
+            no_progress_events=sum(1 for item in trajectory if item.info.made_progress is False),
+        )
+
+    def _collect_delivered_steps(self, trajectory: Sequence[StepResult]) -> Dict[str, int]:
+        delivered_steps: Dict[str, int] = {}
+        for index, item in enumerate(trajectory, start=1):
+            delivered_id = item.info.delivered_order_id
+            if delivered_id is None:
+                continue
+            delivered_step = item.observation.step_count if item.observation.step_count > 0 else index
+            delivered_steps[delivered_id] = delivered_step
+        return delivered_steps
+
+    def _high_priority_metrics(
+        self,
+        order_catalog: Dict[str, Order],
+        delivered_steps: Dict[str, int],
+        deadline_steps: int,
+        edge_case_notes: List[str],
+    ) -> tuple[int, int, int, float]:
+        high_priority_orders = [
+            order
+            for order in order_catalog.values()
+            if order.priority == OrderPriority.HIGH
+        ]
+        high_priority_total = len(high_priority_orders)
+
+        if high_priority_total == 0:
+            edge_case_notes.append("No high-priority orders were present; high-priority SLA treated as 1.0.")
+            return 0, 0, 0, 1.0
+
+        high_priority_delivered = 0
+        high_priority_on_time = 0
+
+        for order in high_priority_orders:
+            delivered_step = delivered_steps.get(order.order_id)
+            if delivered_step is None:
+                continue
+
+            high_priority_delivered += 1
+            if order.accepted_step is None:
+                continue
+
+            if delivered_step - order.accepted_step <= deadline_steps:
+                high_priority_on_time += 1
+
+        on_time_rate = high_priority_on_time / high_priority_total
+        return high_priority_total, high_priority_delivered, high_priority_on_time, on_time_rate
+
+    def _efficiency_ratio(self, steps_taken: int, max_steps_target: int) -> float:
+        if max_steps_target <= 0:
+            return 1.0
+        if steps_taken <= 0:
+            return 1.0
+        return clamp((max_steps_target - steps_taken) / max_steps_target, 0.0, 1.0)
+
+    def _invalid_action_rate(self, invalid_actions: int, steps_taken: int) -> float:
+        if steps_taken <= 0:
+            return 0.0
+        return clamp(invalid_actions / steps_taken, 0.0, 1.0)
+
+    def _target_progress(
+        self,
+        value: float,
+        target: float,
+        edge_case_notes: List[str],
+        metric_name: str,
+    ) -> float:
+        if target <= 0.0:
+            edge_case_notes.append(f"{metric_name} target was 0.0; component treated as 1.0.")
+            return 1.0
+        return clamp(value / target, 0.0, 1.0)
+
+    def _invalid_action_penalty_component(
+        self,
+        invalid_action_rate: float,
+        invalid_action_rate_max: float,
+        edge_case_notes: List[str],
+    ) -> float:
+        if invalid_action_rate_max <= 0.0:
+            if invalid_action_rate > 0.0:
+                edge_case_notes.append(
+                    "invalid_action_rate_max is 0.0 and invalid actions occurred; penalty set to 0.0."
+                )
+                return 0.0
+            return 1.0
+
+        normalized = invalid_action_rate / invalid_action_rate_max
+        if invalid_action_rate <= invalid_action_rate_max:
+            return clamp(1.0 - 0.25 * normalized, 0.75, 1.0)
+
+        overflow = (invalid_action_rate - invalid_action_rate_max) / max(1.0 - invalid_action_rate_max, 1e-9)
+        return clamp(0.75 - 0.75 * overflow, 0.0, 0.75)
+        if total_orders == 0:
+            edge_case_notes.append("No orders were present; completion is evaluated as neutral.")
+        if penalty_stats.steps_taken == 0 and total_orders > 0:
+            edge_case_notes.append("No steps were recorded for a non-empty episode.")
+
+        unknown_orders = max(0, total_orders - len(order_catalog))
+        if unknown_orders > 0:
+            edge_case_notes.append(
+                "Some order geometries were missing from observations; overhead-only fallback used."
+            )
+
+        ordered_orders = [order_catalog[key] for key in sorted(order_catalog.keys())]
+        optimal_steps_known = optimal_steps_single_agent(
+            orders=ordered_orders,
+            start_location=self.start_location,
+        )
+        optimal_steps = optimal_steps_known + (unknown_orders * 3)
+
+        if total_orders > 0 and optimal_steps == 0:
+            optimal_steps = max(1, penalty_stats.steps_taken)
+            safeguards.append("optimal_steps_zero_guard")
+
+        delivered_weight, total_weight = self._weighted_completion_masses(
+            delivered_ids=delivered_ids,
+            all_ids=all_ids,
+            order_catalog=order_catalog,
+        )
+
+        completion, efficiency_component, penalty_component = self._compute_components(
+            delivered_weight=delivered_weight,
+            total_weight=total_weight,
+            penalty_stats=penalty_stats,
+            optimal_steps=optimal_steps,
+        )
+
+        raw_score = self._combine_components(
+            completion=completion,
+            efficiency_component=efficiency_component,
+            penalty_component=penalty_component,
+        )
+
+        raw_score = self._apply_safeguards(
+            raw_score=raw_score,
+            safeguards=safeguards,
+            total_orders=total_orders,
+            delivered_orders=delivered_orders,
+            steps_taken=penalty_stats.steps_taken,
+            invalid_actions=penalty_stats.invalid_actions,
+            delay_events=penalty_stats.delay_events,
+            battery_depletion_events=penalty_stats.battery_depletion_events,
+            no_progress_events=penalty_stats.no_progress_events,
+            completion=completion,
+            efficiency_component=efficiency_component,
+            penalty_component=penalty_component,
+        )
+
+        score = round_score(raw_score, decimals=4)
+
+        logic = (
+            "score = 0.50*completion + 0.30*efficiency + 0.20*penalty_quality; "
+            "completion is weighted by order priority (high>low), "
+            "efficiency uses optimal_steps/steps_taken and is gated by completion, "
+            "penalty_quality decreases with invalid, delay, battery-depletion, and sustained no-progress events."
+        )
+
+        if not edge_case_notes:
+            edge_case_notes.append("No edge-case adjustments were needed.")
+
+        return GradeReport(
+            score=score,
+            delivered_orders=delivered_orders,
+            total_orders=total_orders,
+            steps_taken=penalty_stats.steps_taken,
+            optimal_steps=optimal_steps,
+            completion_component=completion,
+            efficiency_component=efficiency_component,
+            penalty_component=penalty_component,
+            invalid_actions=penalty_stats.invalid_actions,
+            delay_events=penalty_stats.delay_events,
+            battery_depletion_events=penalty_stats.battery_depletion_events,
+            no_progress_events=penalty_stats.no_progress_events,
+            battery_remaining=final_observation.battery_level,
+            scoring_logic=logic,
+            safeguards_applied=safeguards,
+            edge_case_handling=edge_case_notes,
+        )
+
+    def _collect_order_catalog(
+        self,
+        trajectory: Sequence[StepResult],
+        final_observation: Observation,
+    ) -> Dict[str, Order]:
+        orders: Dict[str, Order] = {}
+
+        for item in trajectory:
+            for order in item.observation.pending_orders:
+                orders[order.order_id] = order
+            if item.observation.current_order is not None:
+                orders[item.observation.current_order.order_id] = item.observation.current_order
+
+        for order in final_observation.pending_orders:
+            orders[order.order_id] = order
+        if final_observation.current_order is not None:
+            orders[final_observation.current_order.order_id] = final_observation.current_order
+
+        return orders

grader/metrics.py

@@ -0,0 +1,178 @@
+from __future__ import annotations
+
+from decimal import ROUND_HALF_UP, Decimal
+from typing import List, Sequence, Tuple
+
+from env.models import Order
+from env.utils import manhattan
+
+
+def clamp(value: float, low: float, high: float) -> float:
+    return max(low, min(high, value))
+
+
+def round_score(value: float, decimals: int = 4) -> float:
+    """Round with HALF_UP behavior for stable scoring outputs."""
+    safe = clamp(value, 0.0, 1.0)
+
+    if decimals > 0:
+        epsilon = float(Decimal("1") / (Decimal(10) ** decimals))
+        if safe <= 0.0:
+            safe = epsilon
+        elif safe >= 1.0:
+            safe = 1.0 - epsilon
+
+    quantizer = Decimal("1") if decimals <= 0 else Decimal(f"1.{'0' * decimals}")
+    return float(Decimal(str(safe)).quantize(quantizer, rounding=ROUND_HALF_UP))
+
+
+def completion_rate(delivered_weight: float, total_weight: float) -> float:
+    if total_weight <= 0:
+        # Neutral when no work was available.
+        return 0.5
+    return clamp(delivered_weight / total_weight, 0.0, 1.0)
+
+
+def efficiency_score(steps_taken: int, optimal_steps: int) -> float:
+    """Compute bounded efficiency ratio from optimal-step lower bound."""
+    if optimal_steps <= 0:
+        return 1.0 if steps_taken <= 0 else 0.0
+    if steps_taken <= 0:
+        return 0.0
+    return clamp(optimal_steps / steps_taken, 0.0, 1.0)
+
+
+def penalty_score(
+    invalid_actions: int,
+    delay_events: int,
+    battery_depletion_events: int,
+    no_progress_events: int,
+    total_steps: int,
+) -> float:
+    """Map penalty events to a [0, 1] quality score where 1 is best."""
+    denominator = max(total_steps, 1)
+
+    weighted_penalties = (
+        invalid_actions * 1.0
+        + delay_events * 0.6
+        + battery_depletion_events * 2.0
+        + no_progress_events * 0.25
+    )
+    base_rate = weighted_penalties / denominator
+
+    # Add deterministic duration pressure for sustained non-progress behavior so
+    # longer stalled episodes are graded worse than short stalls.
+    stalled_fraction = no_progress_events / denominator
+    stall_duration_pressure = stalled_fraction * min(0.35, denominator * 0.02)
+
+    rate = base_rate + stall_duration_pressure
+    return clamp(1.0 - rate, 0.0, 1.0)
+
+
+def optimal_steps_single_agent(
+    orders: Sequence[Order],
+    start_location: Tuple[int, int] = (0, 0),
+    max_exact_orders: int = 10,
+) -> int:
+    """
+    Estimate minimal steps for one-agent execution with one active order at a time.
+
+    For small order counts this is exact DP over completion order and chosen drop location.
+    For larger sets it uses a deterministic nearest-first fallback to stay efficient.
+    """
+    order_list: List[Order] = list(orders)
+    count = len(order_list)
+    if count == 0:
+        return 0
+
+    if count > max_exact_orders:
+        return _heuristic_optimal_steps(order_list, start_location)
+
+    pickups = [(o.pickup.x, o.pickup.y) for o in order_list]
+    drop_choices = [
+        [(d.x, d.y) for d in (o.delivery_locations or [o.dropoff])]
+        for o in order_list
+    ]
+
+    max_mask = 1 << count
+    inf = float("inf")
+
+    dp = [
+        [
+            [inf for _ in drop_choices[i]]
+            for i in range(count)
+        ]
+        for _ in range(max_mask)
+    ]
+
+    for i in range(count):
+        for k, drop in enumerate(drop_choices[i]):
+            travel = manhattan(start_location, pickups[i]) + manhattan(pickups[i], drop)
+            dp[1 << i][i][k] = float(travel)
+
+    for mask in range(max_mask):
+        for last in range(count):
+            for last_k, current in enumerate(dp[mask][last]):
+                if current == inf:
+                    continue
+                last_drop = drop_choices[last][last_k]
+
+                for nxt in range(count):
+                    if mask & (1 << nxt):
+                        continue
+
+                    transition_to_pickup = manhattan(last_drop, pickups[nxt])
+                    next_mask = mask | (1 << nxt)
+
+                    for nxt_k, nxt_drop in enumerate(drop_choices[nxt]):
+                        candidate = (
+                            current
+                            + transition_to_pickup
+                            + manhattan(pickups[nxt], nxt_drop)
+                        )
+                        if candidate < dp[next_mask][nxt][nxt_k]:
+                            dp[next_mask][nxt][nxt_k] = candidate
+
+    full_mask = max_mask - 1
+    optimal_travel = min(
+        dp[full_mask][last][last_k]
+        for last in range(count)
+        for last_k in range(len(drop_choices[last]))
+    )
+
+    if optimal_travel == inf:
+        return _heuristic_optimal_steps(order_list, start_location)
+
+    # Minimal action overhead per order: accept + pickup-action + deliver-action.
+    action_overhead = 3 * count
+    return int(optimal_travel + action_overhead)
+
+
+def _heuristic_optimal_steps(
+    orders: Sequence[Order],
+    start_location: Tuple[int, int],
+) -> int:
+    remaining = list(orders)
+    current = start_location
+    travel = 0
+
+    while remaining:
+        best_index = 0
+        best_cost = float("inf")
+        best_drop = current
+
+        for idx, order in enumerate(remaining):
+            pickup = (order.pickup.x, order.pickup.y)
+            drop_candidates = [(d.x, d.y) for d in (order.delivery_locations or [order.dropoff])]
+            nearest_drop = min(drop_candidates, key=lambda d: manhattan(pickup, d))
+            cost = manhattan(current, pickup) + manhattan(pickup, nearest_drop)
+            if cost < best_cost:
+                best_cost = cost
+                best_index = idx
+                best_drop = nearest_drop
+
+        travel += int(best_cost)
+        current = best_drop
+        remaining.pop(best_index)
+
+    return int(travel + 3 * len(orders))

inference.py

@@ -0,0 +1,786 @@
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import re
+import time
+from dataclasses import dataclass
+from typing import Any, Dict, List, Optional, Sequence, Tuple
+
+from env.environment import LastMileDeliveryEnvironment
+from env.models import Action, ActionType, Direction, Observation, Order, OrderPriority, OrderStatus, SimulatorAction, StepInfo, StepResult
+from env.utils import in_bounds, manhattan, move_coordinate
+from grader.grader import DeliveryEpisodeGrader
+from tasks.registry import get_task_config, get_task_definition, list_tasks
+
+try:
+	from openai import OpenAI
+except Exception:  # pragma: no cover - import guard for environments without dependency installed
+	OpenAI = None  # type: ignore[assignment]
+
+
+DEFAULT_SEED = 42
+MAX_RUNTIME_SECONDS = 19 * 60
+DEFAULT_TASK = "easy"
+DEFAULT_MODEL_NAME = "gpt-4o-mini"
+
+DIRECTION_PRIORITY: List[Direction] = [
+	Direction.UP,
+	Direction.LEFT,
+	Direction.RIGHT,
+	Direction.DOWN,
+]
+
+
+@dataclass(frozen=True)
+class RuntimeSettings:
+	openai_api_key: str
+	model_name: str
+	api_base_url: Optional[str]
+	use_remote_model: bool
+
+
+def _read_env_first(*names: str) -> str:
+	for name in names:
+		value = (os.getenv(name) or "").strip()
+		if value:
+			return value
+	return ""
+
+
+def _require_any_env(*names: str) -> str:
+	value = _read_env_first(*names)
+	if value:
+		return value
+	raise RuntimeError(f"Missing required environment variable. Set one of: {', '.join(names)}")
+
+
+def _read_bool_env(name: str, default: bool = False) -> bool:
+	raw = (os.getenv(name) or "").strip().lower()
+	if not raw:
+		return default
+	return raw in {"1", "true", "yes", "on"}
+
+
+def _resolve_model_name() -> str:
+	value = _read_env_first("MODEL_NAME", "OPENAI_MODEL")
+	return value or DEFAULT_MODEL_NAME
+
+
+def load_runtime_settings() -> RuntimeSettings:
+	base_url = _read_env_first("API_BASE_URL", "OPENAI_BASE_URL") or None
+	model_name = _resolve_model_name()
+
+	return RuntimeSettings(
+		openai_api_key=_require_any_env("HF_TOKEN", "OPENAI_API_KEY"),
+		model_name=model_name,
+		api_base_url=base_url,
+		use_remote_model=_read_bool_env("OPENAI_USE_MODEL", default=False),
+	)
+
+
+class DeterministicHeuristicPolicy:
+	def __init__(self, settings: RuntimeSettings, seed: int):
+		if OpenAI is None:
+			raise RuntimeError("openai package is required but not installed")
+
+		self.model_name = settings.model_name
+		self.use_remote_model = settings.use_remote_model
+		self.seed = seed
+		self._recent_positions: List[Tuple[int, int]] = []
+		self._previous_position: Optional[Tuple[int, int]] = None
+		# Keep a deterministic fallback path so local validation does not depend on external API availability.
+		self.client: Any | None = None
+		self.client_init_error: str | None = None
+		try:
+			client_kwargs: Dict[str, Any] = {
+				"api_key": settings.openai_api_key,
+				"timeout": 20.0,
+			}
+			if settings.api_base_url is not None:
+				client_kwargs["base_url"] = settings.api_base_url
+			self.client = OpenAI(**client_kwargs)
+		except Exception as exc:
+			self.client_init_error = type(exc).__name__
+
+	def _update_navigation_memory(self, observation: Observation, step_index: int) -> None:
+		current = (observation.agent_location.x, observation.agent_location.y)
+
+		# Step index resets to 1 at each new episode; clear memory so repeated single-step
+		# inference calls remain deterministic.
+		if step_index <= 1:
+			self._recent_positions = [current]
+			self._previous_position = None
+			return
+
+		self._recent_positions.append(current)
+		if len(self._recent_positions) > 8:
+			self._recent_positions.pop(0)
+
+		if len(self._recent_positions) >= 2:
+			self._previous_position = self._recent_positions[-2]
+		else:
+			self._previous_position = None
+
+	def _is_local_oscillation(self) -> bool:
+		if len(self._recent_positions) < 4:
+			return False
+
+		a, b, c, d = self._recent_positions[-4:]
+		return a == c and b == d and c != d
+
+	def decide(self, observation: Observation, step_index: int) -> Tuple[SimulatorAction, Optional[str]]:
+		self._update_navigation_memory(observation=observation, step_index=step_index)
+		break_oscillation = self._is_local_oscillation()
+
+		if self.use_remote_model and self.client is not None:
+			model_action, model_error = self._try_model_action(observation, step_index=step_index)
+			if model_action is not None:
+				return model_action, model_error
+
+		try:
+			action = _heuristic_action(
+				observation,
+				previous_position=self._previous_position,
+				break_oscillation=break_oscillation,
+			)
+			return action, self.client_init_error if self.use_remote_model and self.client is None else None
+		except Exception as exc:
+			fallback = SimulatorAction(action_type=ActionType.WAIT)
+			return fallback, f"policy_error:{type(exc).__name__}"
+
+	def _try_model_action(self, observation: Observation, step_index: int) -> Tuple[Optional[SimulatorAction], Optional[str]]:
+		if self.client is None:
+			return None, "openai_client_unavailable"
+
+		system_prompt = (
+			"You are a deterministic last-mile dispatch policy. "
+			"Return one JSON object matching this schema exactly: "
+			"{\"move\": string|null, \"accept_order\": integer|null, \"deliver_order\": boolean, \"wait\": boolean}. "
+			"Exactly one intent must be active. Do not include markdown."
+		)
+		payload = {
+			"step_index": step_index,
+			"observation": observation.model_dump(),
+		}
+
+		try:
+			response = self.client.responses.create(
+				model=self.model_name,
+				temperature=0,
+				max_output_tokens=120,
+				input=[
+					{
+						"role": "system",
+						"content": [{"type": "text", "text": system_prompt}],
+					},
+					{
+						"role": "user",
+						"content": [{"type": "text", "text": json.dumps(payload, separators=(",", ":"), sort_keys=True)}],
+					},
+				],
+			)
+		except Exception as exc:
+			return None, f"model_request_error:{type(exc).__name__}"
+
+		output_text = getattr(response, "output_text", "") or ""
+		parsed_json = self._extract_json_object(output_text)
+		if parsed_json is None:
+			return None, "model_parse_error"
+
+		try:
+			action = Action.model_validate(parsed_json).to_simulator_action()
+		except Exception as exc:
+			return None, f"model_action_invalid:{type(exc).__name__}"
+
+		return action, None
+
+	@staticmethod
+	def _extract_json_object(text: str) -> Optional[Dict[str, Any]]:
+		candidate = text.strip()
+		if not candidate:
+			return None
+
+		if candidate.startswith("```"):
+			candidate = re.sub(r"^```[a-zA-Z0-9_\-]*\s*", "", candidate)
+			candidate = re.sub(r"\s*```$", "", candidate).strip()
+
+		try:
+			value = json.loads(candidate)
+			if isinstance(value, dict):
+				return value
+		except Exception:
+			pass
+
+		start = candidate.find("{")
+		end = candidate.rfind("}")
+		if start == -1 or end == -1 or end <= start:
+			return None
+
+		try:
+			value = json.loads(candidate[start : end + 1])
+			if isinstance(value, dict):
+				return value
+		except Exception:
+			return None
+
+		return None
+
+
+_POLICY: DeterministicHeuristicPolicy | None = None
+
+
+def _get_policy(seed: int = DEFAULT_SEED) -> DeterministicHeuristicPolicy:
+	global _POLICY
+	if _POLICY is None:
+		settings = load_runtime_settings()
+		_POLICY = DeterministicHeuristicPolicy(settings=settings, seed=seed)
+	return _POLICY
+
+
+def _priority_rank(order: Order) -> int:
+	return 0 if order.priority == OrderPriority.HIGH else 1
+
+
+def _choose_pending_order(observation: Observation) -> Optional[Order]:
+	agent = (observation.agent_location.x, observation.agent_location.y)
+	pending = [order for order in observation.pending_orders if order.status == OrderStatus.PENDING]
+	if not pending:
+		return None
+
+	pending.sort(
+		key=lambda order: (
+			_priority_rank(order),
+			manhattan(agent, (order.pickup.x, order.pickup.y)),
+			order.order_id,
+		)
+	)
+	return pending[0]
+
+
+def _current_target(observation: Observation) -> Optional[Tuple[int, int]]:
+	current_order = observation.current_order
+	if current_order is None:
+		return None
+
+	agent = (observation.agent_location.x, observation.agent_location.y)
+	if current_order.status == OrderStatus.ACCEPTED:
+		return (current_order.pickup.x, current_order.pickup.y)
+
+	if current_order.status == OrderStatus.PICKED_UP:
+		targets = current_order.delivery_locations or [current_order.dropoff]
+		target_coords = [(target.x, target.y) for target in targets]
+		target_coords.sort(key=lambda coord: (manhattan(agent, coord), coord[0], coord[1]))
+		return target_coords[0]
+
+	if current_order.status == OrderStatus.PENDING:
+		return (current_order.pickup.x, current_order.pickup.y)
+
+	return None
+
+
+def _nearest_charging_station(observation: Observation) -> Optional[Tuple[int, int]]:
+	if not observation.charging_stations:
+		return None
+
+	agent = (observation.agent_location.x, observation.agent_location.y)
+	station_coords = [(item.x, item.y) for item in observation.charging_stations]
+	station_coords.sort(key=lambda coord: (manhattan(agent, coord), coord[0], coord[1]))
+	return station_coords[0]
+
+
+def _best_move_direction(
+	observation: Observation,
+	target: Tuple[int, int],
+	previous_position: Optional[Tuple[int, int]] = None,
+	break_oscillation: bool = False,
+) -> Optional[Direction]:
+	current = (observation.agent_location.x, observation.agent_location.y)
+	blocked = {(item.x, item.y) for item in observation.obstacles}
+	blocked.update((item.x, item.y) for item in observation.dynamic_obstacles)
+
+	candidates: List[Tuple[int, int, Direction, Tuple[int, int]]] = []
+	for index, direction in enumerate(DIRECTION_PRIORITY):
+		nxt = move_coordinate(current, direction)
+		if not in_bounds(nxt, observation.grid_width, observation.grid_height):
+			continue
+		if nxt in blocked:
+			continue
+
+		distance = manhattan(nxt, target)
+		candidates.append((distance, index, direction, nxt))
+
+	if not candidates:
+		return None
+
+	candidates.sort(key=lambda item: (item[0], item[1]))
+
+	if previous_position is not None and len(candidates) > 1:
+		best_distance = candidates[0][0]
+		for distance, _, direction, nxt in candidates:
+			if nxt == previous_position:
+				continue
+
+			# Stronger anti-backtrack when we detect A-B-A-B style oscillation.
+			if break_oscillation:
+				return direction
+
+			# Prefer a non-backtracking move when it is near-optimal.
+			if distance <= best_distance + 1:
+				return direction
+
+			break
+
+	return candidates[0][2]
+
+
+def _move_toward_target(
+	observation: Observation,
+	target: Tuple[int, int],
+	previous_position: Optional[Tuple[int, int]] = None,
+	break_oscillation: bool = False,
+) -> SimulatorAction:
+	direction = _best_move_direction(
+		observation,
+		target,
+		previous_position=previous_position,
+		break_oscillation=break_oscillation,
+	)
+	if direction is None:
+		return SimulatorAction(action_type=ActionType.WAIT)
+	return SimulatorAction(action_type=ActionType.MOVE, direction=direction)
+
+
+def _battery_aware_action(
+	observation: Observation,
+	target: Tuple[int, int],
+	previous_position: Optional[Tuple[int, int]] = None,
+	break_oscillation: bool = False,
+) -> Optional[SimulatorAction]:
+	battery_level = observation.battery_level
+	if battery_level is None:
+		return None
+
+	# Keep pickup behavior aggressive; only interrupt for charging when battery is near-empty.
+	if observation.current_order is None and battery_level > 3:
+		return None
+
+	charger_target = _nearest_charging_station(observation)
+	if charger_target is None:
+		return None
+
+	agent = (observation.agent_location.x, observation.agent_location.y)
+	distance_to_charger = manhattan(agent, charger_target)
+	distance_to_target = manhattan(agent, target)
+
+	# Trigger recharge only when battery is close to the minimum needed to reach a charger.
+	recharge_trigger = distance_to_charger + 1
+	if battery_level > recharge_trigger:
+		return None
+
+	if agent == charger_target:
+		# Recharge just enough to leave and make forward progress; avoid long waiting loops.
+		resume_level = min(12, max(8, distance_to_target + 2))
+		if battery_level < resume_level:
+			return SimulatorAction(action_type=ActionType.WAIT)
+		return None
+
+	return _move_toward_target(
+		observation,
+		charger_target,
+		previous_position=previous_position,
+		break_oscillation=break_oscillation,
+	)
+
+
+def _heuristic_action(
+	observation: Observation,
+	previous_position: Optional[Tuple[int, int]] = None,
+	break_oscillation: bool = False,
+) -> SimulatorAction:
+	agent = (observation.agent_location.x, observation.agent_location.y)
+	current_order = observation.current_order
+
+	if current_order is None:
+		selected = _choose_pending_order(observation)
+		if selected is None:
+			return SimulatorAction(action_type=ActionType.WAIT)
+
+		pickup_target = (selected.pickup.x, selected.pickup.y)
+		if agent == pickup_target:
+			return SimulatorAction(action_type=ActionType.ACCEPT_ORDER, order_id=selected.order_id)
+
+		recharge_action = _battery_aware_action(
+			observation,
+			pickup_target,
+			previous_position=previous_position,
+			break_oscillation=break_oscillation,
+		)
+		if recharge_action is not None:
+			return recharge_action
+
+		# Move toward the nearest pickup instead of idling while no order is selected.
+		return _move_toward_target(
+			observation,
+			pickup_target,
+			previous_position=previous_position,
+			break_oscillation=break_oscillation,
+		)
+
+	target = _current_target(observation)
+	if target is None:
+		selected = _choose_pending_order(observation)
+		if selected is None:
+			return SimulatorAction(action_type=ActionType.WAIT)
+		return _move_toward_target(
+			observation,
+			(selected.pickup.x, selected.pickup.y),
+			previous_position=previous_position,
+			break_oscillation=break_oscillation,
+		)
+
+	if agent == target:
+		return SimulatorAction(action_type=ActionType.DELIVER_ORDER)
+
+	recharge_action = _battery_aware_action(
+		observation,
+		target,
+		previous_position=previous_position,
+		break_oscillation=break_oscillation,
+	)
+	if recharge_action is not None:
+		return recharge_action
+
+	# With an active target, prefer movement over waiting whenever a valid move exists.
+	return _move_toward_target(
+		observation,
+		target,
+		previous_position=previous_position,
+		break_oscillation=break_oscillation,
+	)
+
+
+def _bool_literal(value: bool) -> str:
+	return "true" if value else "false"
+
+
+def _format_error(error: Optional[str]) -> str:
+	if error is None:
+		return "null"
+	return error.replace(" ", "_")
+
+
+def _format_token(value: Optional[str], fallback: str) -> str:
+	token = (value or "").strip()
+	if not token:
+		token = fallback
+	return "_".join(token.split())
+
+
+def _format_action(action: SimulatorAction) -> str:
+	payload = Action.from_simulator_action(action).model_dump()
+	return json.dumps(payload, separators=(",", ":"), sort_keys=True)
+
+
+def format_start_line(task: str, env_name: str, model_name: str) -> str:
+	return (
+		f"[START] "
+		f"task={_format_token(task, fallback='unknown_task')} "
+		f"env={_format_token(env_name, fallback='unknown_env')} "
+		f"model={_format_token(model_name, fallback='unknown_model')}"
+	)
+
+
+def format_step_line(step_index: int, action: SimulatorAction, reward: float, done: bool, error: Optional[str]) -> str:
+	return (
+		f"[STEP] step={step_index} "
+		f"action={_format_action(action)} "
+		f"reward={reward:.2f} "
+		f"done={_bool_literal(done)} "
+		f"error={_format_error(error)}"
+	)
+
+
+def _format_reward_series(rewards: Sequence[float] | float) -> str:
+	if isinstance(rewards, (int, float)):
+		return f"{float(rewards):.2f}"
+
+	reward_values = list(rewards)
+	if not reward_values:
+		return "0.00"
+	return ",".join(f"{value:.2f}" for value in reward_values)
+
+
+def format_end_line(success: bool, steps: int, rewards: Sequence[float] | float) -> str:
+	return f"[END] success={_bool_literal(success)} steps={steps} rewards={_format_reward_series(rewards)}"
+
+
+def _episode_success(done: bool, observation: Observation) -> bool:
+	if not done:
+		return False
+	return len(observation.pending_orders) == 0 and observation.current_order is None
+
+
+def _grade_score(task: str, trajectory: Sequence[StepResult], final_observation: Observation) -> float:
+	grader = DeliveryEpisodeGrader()
+	success_condition: Dict[str, Any] = {}
+	try:
+		task_definition = get_task_definition(task)
+		success_condition = dict(task_definition.get("success_condition", {}))
+	except Exception:
+		success_condition = {}
+
+	report = grader.grade_episode(
+		trajectory=trajectory,
+		final_observation=final_observation,
+		success_condition=success_condition,
+	)
+	return float(report.score)
+
+
+def run_episode(
+	task: str = DEFAULT_TASK,
+	seed: int = DEFAULT_SEED,
+	max_steps: Optional[int] = None,
+	trace: bool = True,
+) -> Dict[str, Any]:
+	steps = 0
+	reward_history: List[float] = []
+	trajectory: List[StepResult] = []
+	success = False
+	step_printed = False
+	startup_error: Optional[str] = None
+	runtime_error: Optional[str] = None
+	model_name = _resolve_model_name()
+	env_name = LastMileDeliveryEnvironment.__name__
+	settings: Optional[RuntimeSettings] = None
+	score = 0.0
+	env: Optional[LastMileDeliveryEnvironment] = None
+	observation: Optional[Observation] = None
+
+	try:
+		try:
+			settings = load_runtime_settings()
+			model_name = settings.model_name
+		except Exception as exc:
+			startup_error = f"startup_error:{type(exc).__name__}"
+
+		if trace:
+			print(format_start_line(task=task, env_name=env_name, model_name=model_name))
+
+		if settings is None:
+			action = SimulatorAction(action_type=ActionType.WAIT)
+			if trace:
+				print(
+					format_step_line(
+						step_index=1,
+						action=action,
+						reward=0.0,
+						done=False,
+						error=startup_error,
+					)
+				)
+			step_printed = True
+			return {
+				"success": False,
+				"steps": 0,
+				"rewards": [],
+				"total_reward": 0.0,
+				"score": 0.0,
+			}
+
+		policy = DeterministicHeuristicPolicy(settings=settings, seed=seed)
+		config = get_task_config(task_name=task, seed=seed)
+		env = LastMileDeliveryEnvironment(config=config)
+		observation = env.reset()
+
+		start_time = time.monotonic()
+		configured_limit = max_steps if max_steps is not None else config.max_steps
+		step_limit = max(1, min(configured_limit, config.max_steps))
+
+		for step_index in range(1, step_limit + 1):
+			if time.monotonic() - start_time >= MAX_RUNTIME_SECONDS:
+				runtime_error = "timeout"
+				break
+
+			action, action_error = policy.decide(observation, step_index=step_index)
+			try:
+				step_result = env.step_result(action)
+				trajectory.append(step_result)
+				observation = step_result.observation
+				reward = float(step_result.reward)
+				done = step_result.done
+				info = step_result.info.model_dump()
+			except Exception as exc:
+				reward = 0.0
+				done = False
+				info = {"invalid_action": True, "message": f"step_error:{type(exc).__name__}"}
+				action = SimulatorAction(action_type=ActionType.WAIT)
+				runtime_error = str(info["message"])
+				if env is not None:
+					fallback_observation = env.current_observation()
+					trajectory.append(
+						StepResult(
+							observation=fallback_observation,
+							reward=reward,
+							done=done,
+							info=StepInfo(invalid_action=True, message=str(info["message"])),
+						)
+					)
+
+			step_error = action_error
+			if step_error is None and bool(info.get("invalid_action")):
+				step_error = str(info.get("message", "invalid_action"))
+
+			steps = step_index
+			reward_history.append(float(reward))
+			if trace:
+				print(
+					format_step_line(
+						step_index=step_index,
+						action=action,
+						reward=reward,
+						done=done,
+						error=step_error,
+					)
+				)
+			step_printed = True
+
+			if done:
+				success = _episode_success(done=done, observation=observation)
+				break
+			if runtime_error is not None:
+				break
+
+		if env is not None:
+			final_observation = env.current_observation()
+			score = _grade_score(task=task, trajectory=trajectory, final_observation=final_observation)
+
+		return {
+			"success": success,
+			"steps": steps,
+			"rewards": [round(value, 2) for value in reward_history],
+			"total_reward": round(sum(reward_history), 2),
+			"score": round(score, 4),
+		}
+
+	except BaseException as exc:
+		runtime_error = f"fatal_error:{type(exc).__name__}"
+		return {
+			"success": False,
+			"steps": steps,
+			"rewards": [round(value, 2) for value in reward_history],
+			"total_reward": round(sum(reward_history), 2),
+			"score": 0.0,
+		}
+	finally:
+		if trace and not step_printed:
+			action = SimulatorAction(action_type=ActionType.WAIT)
+			fallback_error = runtime_error or startup_error
+			print(
+				format_step_line(
+					step_index=max(1, steps + 1),
+					action=action,
+					reward=0.0,
+					done=False,
+					error=fallback_error,
+				)
+			)
+
+		if trace:
+			print(format_end_line(success=success, steps=steps, rewards=reward_history))
+
+
+def run_baseline_suite(seed: int = DEFAULT_SEED, max_steps: Optional[int] = None) -> Dict[str, Any]:
+	task_results: List[Dict[str, Any]] = []
+	for task_name in list_tasks():
+		episode_result = run_episode(task=task_name, seed=seed, max_steps=max_steps, trace=False)
+		task_results.append(
+			{
+				"task": task_name,
+				"success": bool(episode_result.get("success", False)),
+				"steps": int(episode_result.get("steps", 0)),
+				"total_reward": float(episode_result.get("total_reward", 0.0)),
+				"score": float(episode_result.get("score", 0.0)),
+			}
+		)
+
+	average_score = 0.0
+	if task_results:
+		average_score = round(sum(item["score"] for item in task_results) / len(task_results), 4)
+
+	return {
+		"seed": seed,
+		"model": _resolve_model_name(),
+		"tasks": task_results,
+		"average_score": average_score,
+	}
+
+
+def inference(observation: Dict[str, Any]) -> Dict[str, Any]:
+	"""OpenEnv action entrypoint with deterministic heuristic policy and OpenAI client setup."""
+	parsed = Observation.model_validate(observation)
+	try:
+		action, _ = _get_policy(seed=DEFAULT_SEED).decide(parsed, step_index=1)
+	except Exception:
+		action = _heuristic_action(parsed)
+	return Action.from_simulator_action(action).model_dump()
+
+
+def predict(observation: Dict[str, Any]) -> Dict[str, Any]:
+	"""Compatibility alias for runners expecting predict(...)."""
+	return inference(observation)
+
+
+def _build_arg_parser() -> argparse.ArgumentParser:
+	parser = argparse.ArgumentParser(description="Run deterministic OpenEnv inference episode")
+	parser.add_argument("--task", default=DEFAULT_TASK, type=str, help="Task name from task registry")
+	parser.add_argument("--seed", default=DEFAULT_SEED, type=int, help="Deterministic random seed")
+	parser.add_argument(
+		"--all-tasks",
+		action="store_true",
+		help="Run reproducible baseline evaluation across easy, medium, and hard tasks",
+	)
+	parser.add_argument(
+		"--max-steps",
+		type=int,
+		default=None,
+		help="Optional cap on interaction steps (bounded by task max_steps)",
+	)
+	parser.add_argument(
+		"--json-summary",
+		action="store_true",
+		help="When used with --all-tasks, print compact JSON summary",
+	)
+	return parser
+
+
+def main() -> None:
+	args = _build_arg_parser().parse_args()
+	if args.all_tasks:
+		summary = run_baseline_suite(seed=args.seed, max_steps=args.max_steps)
+		if args.json_summary:
+			print(json.dumps(summary, separators=(",", ":"), sort_keys=True))
+			return
+
+		for item in summary["tasks"]:
+			print(
+				f"[BASELINE] task={item['task']} "
+				f"score={item['score']:.4f} "
+				f"steps={item['steps']} "
+				f"success={_bool_literal(item['success'])}"
+			)
+
+		print(
+			f"[BASELINE_SUMMARY] seed={summary['seed']} "
+			f"tasks={len(summary['tasks'])} "
+			f"average_score={summary['average_score']:.4f}"
+		)
+		return
+
+	run_episode(task=args.task, seed=args.seed, max_steps=args.max_steps, trace=True)
+
+
+if __name__ == "__main__":
+	main()

openenv.yaml

@@ -0,0 +1,127 @@
+name: last-mile-delivery-optimization
+description: OpenEnv-compatible grid delivery environment with tasks, grading, and baseline policy.
+version: 1.0.0
+entrypoint: app.main:app
+
+runtime:
+  api_app: app.main:app
+  inference_entrypoint: inference.py
+
+environment:
+  module: env.environment
+  class: LastMileDeliveryEnvironment
+
+tasks:
+  registry: tasks.registry:get_all_task_metadata
+  config_loader: tasks.registry:get_task_config
+
+grader:
+  module: grader.grader
+  class: DeliveryEpisodeGrader
+
+baseline:
+  module: baseline.baseline_agent
+  class: BaselineGreedyAgent
+
+action_schema: &action_schema
+  type: object
+  required:
+    - move
+    - accept_order
+    - deliver_order
+    - wait
+  additionalProperties: false
+  properties:
+    move:
+      type:
+        - string
+        - "null"
+      enum:
+        - up
+        - down
+        - left
+        - right
+        - stay
+        - null
+    accept_order:
+      type:
+        - integer
+        - "null"
+    deliver_order:
+      type: boolean
+    wait:
+      type: boolean
+
+action_space: *action_schema
+
+observation_schema: &observation_schema
+  type: object
+  required:
+    - agent_location
+    - pending_orders
+    - current_order
+    - obstacles
+    - dynamic_obstacles
+    - charging_stations
+    - battery_level
+  additionalProperties: true
+  properties:
+    agent_location:
+      type: object
+      required:
+        - x
+        - y
+      properties:
+        x:
+          type: integer
+          minimum: 0
+        y:
+          type: integer
+          minimum: 0
+    pending_orders:
+      type: array
+      items:
+        type: object
+    current_order:
+      oneOf:
+        - type: object
+        - type: "null"
+    obstacles:
+      type: array
+      items:
+        type: object
+    dynamic_obstacles:
+      type: array
+      items:
+        type: object
+    charging_stations:
+      type: array
+      items:
+        type: object
+    battery_level:
+      oneOf:
+        - type: integer
+          minimum: 0
+        - type: "null"
+
+observation_space: *observation_schema
+
+reward_schema: &reward_schema
+  type: object
+  required:
+    - value
+  additionalProperties: false
+  properties:
+    value:
+      type: number
+    components:
+      type: object
+      additionalProperties:
+        type: number
+
+reward_space: *reward_schema
+
+reward_type: dense
+reward:
+  type: dense
+  schema: *reward_schema

pyproject.toml

@@ -0,0 +1,20 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "delivery-openenv"
+version = "1.0.0"
+description = "OpenEnv-compatible last-mile delivery optimization benchmark environment."
+readme = "README.md"
+requires-python = ">=3.10"
+dependencies = [
+  "fastapi==0.115.2",
+  "uvicorn==0.30.6",
+  "pydantic==2.9.2",
+  "openai==1.51.2",
+  "openenv-core",
+]
+
+[project.scripts]
+server = "server.app:main"

requirements.txt

@@ -1,2 +1,6 @@
-fastapi
-uvicorn[standard]
+fastapi==0.115.2
+uvicorn==0.30.6
+pydantic==2.9.2
+openai==1.51.2
+openenv-core
+pytest==8.3.3

scripts/train_q_agent.py

@@ -0,0 +1,230 @@
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import random
+import sys
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Dict, List
+
+REPO_ROOT = Path(__file__).resolve().parents[1]
+if str(REPO_ROOT) not in sys.path:
+    sys.path.insert(0, str(REPO_ROOT))
+
+from baseline.trained_q_agent import NUM_ACTIONS, QTable, TrainedQAgent, action_from_index, observation_key
+from env.environment import LastMileDeliveryEnvironment
+from tasks.registry import get_task_config
+
+
+@dataclass
+class EpisodeStats:
+    total_reward: float
+    completion_rate: float
+    success: bool
+    steps: int
+
+
+def _new_q_row() -> List[float]:
+    return [0.0 for _ in range(NUM_ACTIONS)]
+
+
+def _argmax(values: List[float]) -> int:
+    best_index = 0
+    best_value = values[0]
+    for index in range(1, len(values)):
+        if values[index] > best_value:
+            best_value = values[index]
+            best_index = index
+    return best_index
+
+
+def _episode_summary(env: LastMileDeliveryEnvironment) -> EpisodeStats:
+    state = env.state()
+    observation = state.observation
+    if observation is None:
+        return EpisodeStats(total_reward=state.total_reward, completion_rate=0.0, success=False, steps=state.step_count)
+
+    delivered = state.delivered_orders
+    remaining = len(observation.pending_orders)
+    if observation.current_order is not None:
+        remaining += 1
+
+    total_orders = max(1, delivered + remaining)
+    completion_rate = delivered / total_orders
+    success = bool(state.done and remaining == 0)
+    return EpisodeStats(
+        total_reward=state.total_reward,
+        completion_rate=completion_rate,
+        success=success,
+        steps=state.step_count,
+    )
+
+
+def train_q_agent(
+    task: str,
+    episodes: int,
+    seed: int,
+    alpha: float,
+    gamma: float,
+    epsilon_start: float,
+    epsilon_end: float,
+    epsilon_decay: float,
+) -> QTable:
+    rng = random.Random(seed)
+    q_table: QTable = {}
+
+    running_reward = 0.0
+    running_completion = 0.0
+
+    for episode in range(episodes):
+        current_seed = seed + episode
+        config = get_task_config(task_name=task, seed=current_seed)
+        env = LastMileDeliveryEnvironment(config=config)
+        observation = env.reset(seed=current_seed)
+
+        epsilon = max(epsilon_end, epsilon_start * (epsilon_decay ** episode))
+
+        for _ in range(config.max_steps):
+            state_key = observation_key(observation)
+            row = q_table.setdefault(state_key, _new_q_row())
+
+            if rng.random() < epsilon:
+                action_index = rng.randrange(NUM_ACTIONS)
+            else:
+                action_index = _argmax(row)
+
+            action = action_from_index(action_index, observation)
+            next_observation, reward, done, _ = env.step(action)
+
+            next_key = observation_key(next_observation)
+            next_row = q_table.setdefault(next_key, _new_q_row())
+
+            td_target = reward if done else reward + gamma * max(next_row)
+            row[action_index] += alpha * (td_target - row[action_index])
+
+            observation = next_observation
+            if done:
+                break
+
+        summary = _episode_summary(env)
+        running_reward += summary.total_reward
+        running_completion += summary.completion_rate
+
+        if (episode + 1) % max(1, episodes // 10) == 0:
+            avg_reward = running_reward / (episode + 1)
+            avg_completion = running_completion / (episode + 1)
+            print(
+                f"[TRAIN] episode={episode + 1} "
+                f"epsilon={epsilon:.4f} "
+                f"avg_reward={avg_reward:.2f} "
+                f"avg_completion={avg_completion:.3f}",
+                flush=True,
+            )
+
+    return q_table
+
+
+def evaluate_q_agent(task: str, q_table: QTable, episodes: int, seed: int) -> Dict[str, float]:
+    agent = TrainedQAgent(q_table=q_table)
+
+    total_reward = 0.0
+    total_completion = 0.0
+    success_count = 0
+    total_steps = 0
+
+    for episode in range(episodes):
+        current_seed = seed + 10_000 + episode
+        config = get_task_config(task_name=task, seed=current_seed)
+        env = LastMileDeliveryEnvironment(config=config)
+        observation = env.reset(seed=current_seed)
+
+        for _ in range(config.max_steps):
+            action = agent.act(observation)
+            observation, _, done, _ = env.step(action)
+            if done:
+                break
+
+        summary = _episode_summary(env)
+        total_reward += summary.total_reward
+        total_completion += summary.completion_rate
+        success_count += 1 if summary.success else 0
+        total_steps += summary.steps
+
+    count = max(1, episodes)
+    return {
+        "avg_reward": total_reward / count,
+        "avg_completion": total_completion / count,
+        "success_rate": success_count / count,
+        "avg_steps": total_steps / count,
+    }
+
+
+def save_model(model_path: str, task: str, seed: int, episodes: int, q_table: QTable) -> None:
+    output_dir = os.path.dirname(model_path)
+    if output_dir:
+        os.makedirs(output_dir, exist_ok=True)
+    payload = {
+        "model_type": "tabular_q_learning",
+        "task": task,
+        "seed": seed,
+        "episodes": episodes,
+        "num_actions": NUM_ACTIONS,
+        "q_values": q_table,
+    }
+    with open(model_path, "w", encoding="utf-8") as handle:
+        json.dump(payload, handle, separators=(",", ":"), sort_keys=True)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description="Train a tabular Q-learning agent for delivery-openenv")
+    parser.add_argument("--task", type=str, default="easy", choices=["easy", "medium", "hard"])
+    parser.add_argument("--episodes", type=int, default=800)
+    parser.add_argument("--eval-episodes", type=int, default=100)
+    parser.add_argument("--seed", type=int, default=42)
+    parser.add_argument("--alpha", type=float, default=0.20)
+    parser.add_argument("--gamma", type=float, default=0.98)
+    parser.add_argument("--epsilon-start", type=float, default=1.0)
+    parser.add_argument("--epsilon-end", type=float, default=0.05)
+    parser.add_argument("--epsilon-decay", type=float, default=0.995)
+    parser.add_argument("--output", type=str, default="models/q_agent_easy.json")
+    return parser.parse_args()
+
+
+def main() -> None:
+    args = parse_args()
+
+    q_table = train_q_agent(
+        task=args.task,
+        episodes=args.episodes,
+        seed=args.seed,
+        alpha=args.alpha,
+        gamma=args.gamma,
+        epsilon_start=args.epsilon_start,
+        epsilon_end=args.epsilon_end,
+        epsilon_decay=args.epsilon_decay,
+    )
+
+    metrics = evaluate_q_agent(
+        task=args.task,
+        q_table=q_table,
+        episodes=args.eval_episodes,
+        seed=args.seed,
+    )
+
+    save_model(model_path=args.output, task=args.task, seed=args.seed, episodes=args.episodes, q_table=q_table)
+
+    print(
+        f"[EVAL] task={args.task} episodes={args.eval_episodes} "
+        f"avg_reward={metrics['avg_reward']:.2f} "
+        f"avg_completion={metrics['avg_completion']:.3f} "
+        f"success_rate={metrics['success_rate']:.3f} "
+        f"avg_steps={metrics['avg_steps']:.2f}",
+        flush=True,
+    )
+    print(f"[MODEL] saved={args.output} states={len(q_table)}", flush=True)
+
+
+if __name__ == "__main__":
+    main()

scripts/validate.sh

@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+PING_URL="${PING_URL:-}"
+DOCKER_IMAGE_TAG="${DOCKER_IMAGE_TAG:-delivery-openenv:precheck}"
+OPENENV_BIN_FALLBACK="$REPO_DIR/.venv/bin/openenv"
+VENV_PYTHON="$REPO_DIR/.venv/bin/python"
+
+pass() { echo "[PASS] $1"; }
+info() { echo "[INFO] $1"; }
+fail() { echo "[FAIL] $1"; exit 1; }
+
+info "Step 1/4: Optional Hugging Face Space ping"
+if [[ -n "$PING_URL" ]]; then
+	HTTP_CODE="$(
+		curl -sS -o /tmp/openenv_reset_response.txt -w "%{http_code}" \
+			-X POST "$PING_URL/reset" \
+			-H "Content-Type: application/json" \
+			-d '{"task":"easy","seed":101}' || true
+	)"
+	if [[ "$HTTP_CODE" == "200" ]]; then
+		pass "HF Space /reset returned HTTP 200"
+	else
+		fail "HF Space /reset returned HTTP $HTTP_CODE"
+	fi
+else
+	info "PING_URL is not set; skipping remote endpoint check"
+fi
+
+info "Step 2/4: Docker build"
+if ! command -v docker >/dev/null 2>&1; then
+	fail "docker command not found"
+fi
+
+if ! docker info >/dev/null 2>&1; then
+	fail "docker daemon is not running or not reachable"
+fi
+
+docker build -t "$DOCKER_IMAGE_TAG" "$REPO_DIR" >/tmp/openenv_docker_build.log 2>&1 || {
+	tail -20 /tmp/openenv_docker_build.log || true
+	fail "docker build failed"
+}
+pass "Docker build succeeded"
+
+info "Step 3/4: OpenEnv validation"
+if command -v openenv >/dev/null 2>&1; then
+	OPENENV_CMD=(openenv)
+elif [[ -x "$OPENENV_BIN_FALLBACK" ]]; then
+	OPENENV_CMD=("$OPENENV_BIN_FALLBACK")
+else
+	fail "openenv CLI not found; install with 'pip install openenv-core'"
+fi
+
+(cd "$REPO_DIR" && "${OPENENV_CMD[@]}" validate)
+pass "openenv validate passed"
+
+info "Step 4/4: Test suite"
+if [[ -x "$VENV_PYTHON" ]]; then
+	PYTHON_BIN="$VENV_PYTHON"
+elif command -v python3 >/dev/null 2>&1; then
+	PYTHON_BIN="$(command -v python3)"
+else
+	fail "python3 not found"
+fi
+
+(cd "$REPO_DIR" && "$PYTHON_BIN" -m pytest -q)
+pass "pytest passed"
+
+echo
+echo "[PASS] All validation checks completed"

server/__init__.py

@@ -0,0 +1 @@
+"""Server entrypoint package for OpenEnv multi-mode deployment."""

server/app.py

@@ -0,0 +1,20 @@
+from __future__ import annotations
+
+import os
+
+import uvicorn
+
+
+def main() -> None:
+    host = os.getenv("HOST", "0.0.0.0")
+    port_raw = os.getenv("PORT", "7860")
+    try:
+        port = int(port_raw)
+    except ValueError:
+        port = 7860
+
+    uvicorn.run("app.main:app", host=host, port=port, reload=False)
+
+
+if __name__ == "__main__":
+    main()

tasks/easy.py

@@ -0,0 +1,71 @@
+from __future__ import annotations
+
+from typing import Any, Dict
+
+from env.models import EnvironmentConfig
+
+
+TASK_NAME = "easy"
+TASK_DIFFICULTY = 1
+DEFAULT_SEED = 101
+DESCRIPTION = "Simple single-order delivery on a compact map without operational disruptions."
+OBJECTIVE = "Learn reliable movement, pickup, and dropoff sequencing in the simplest delivery setting."
+SUCCESS_CONDITION: Dict[str, Any] = {
+	"completion_rate": 1.0,
+	"max_steps": 60,
+	"battery_depletion": False,
+	"invalid_action_rate_max": 0.12,
+}
+
+TASK_PARAMETERS: Dict[str, Any] = {
+	"grid_size": {"width": 6, "height": 6},
+	"orders": {"min": 1, "max": 1},
+	"variation": {
+		"seeded_layout": True,
+		"order_pattern": "single_short_trip",
+	},
+	"constraints": {
+		"obstacles": False,
+		"dynamic_obstacles": False,
+		"traffic_zones": False,
+		"order_priority": False,
+		"multiple_delivery_locations": False,
+		"battery_constraint": False,
+		"recharge": False,
+		"time_penalty": False,
+	},
+}
+
+
+def get_config(seed: int | None = None) -> EnvironmentConfig:
+	effective_seed = DEFAULT_SEED if seed is None else seed
+	return EnvironmentConfig(
+		width=6,
+		height=6,
+		max_steps=60,
+		max_orders=1,
+		delivery_locations_per_order=1,
+		priority_high_ratio=0.0,
+		obstacle_density=0.0,
+		dynamic_obstacles_enabled=False,
+		dynamic_obstacle_ratio=0.0,
+		traffic_density=0.0,
+		traffic_extra_cost=1,
+		battery_enabled=False,
+		battery_capacity=100,
+		battery_recharge_rate=10,
+		seed=effective_seed,
+	)
+
+
+def get_definition() -> Dict[str, Any]:
+	return {
+		"name": TASK_NAME,
+		"difficulty": TASK_DIFFICULTY,
+		"deterministic": True,
+		"default_seed": DEFAULT_SEED,
+		"description": DESCRIPTION,
+		"objective": OBJECTIVE,
+		"success_condition": SUCCESS_CONDITION,
+		"parameters": TASK_PARAMETERS,
+	}

tasks/hard.py

@@ -0,0 +1,79 @@
+from __future__ import annotations
+
+from typing import Any, Dict
+
+from env.models import EnvironmentConfig
+
+
+TASK_NAME = "hard"
+TASK_DIFFICULTY = 3
+DEFAULT_SEED = 303
+DESCRIPTION = "High-pressure dispatch benchmark with strict energy limits, priority SLAs, and tight step budget."
+OBJECTIVE = "Sustain high on-time completion for priority-heavy workloads under battery, traffic, and dynamic obstacle pressure."
+SUCCESS_CONDITION: Dict[str, Any] = {
+	"completion_rate_min": 0.90,
+	"high_priority_on_time_rate_min": 0.88,
+	"battery_depletion": False,
+	"max_steps": 165,
+	"invalid_action_rate_max": 0.04,
+}
+
+TASK_PARAMETERS: Dict[str, Any] = {
+	"grid_size": {"width": 16, "height": 16},
+	"orders": {"min": 6, "max": 8},
+	"variation": {
+		"seeded_layout": True,
+		"order_mix": "priority_heavy",
+		"traffic_profile": "peak_hour",
+		"battery_profile": "tight_capacity",
+		"step_budget": "tight",
+	},
+	"constraints": {
+		"obstacles": True,
+		"dynamic_obstacles": True,
+		"traffic_zones": True,
+		"order_priority": True,
+		"multiple_delivery_locations": True,
+		"battery_constraint": True,
+		"recharge": True,
+		"time_penalty": True,
+	},
+}
+
+
+def get_config(seed: int | None = None) -> EnvironmentConfig:
+	effective_seed = DEFAULT_SEED if seed is None else seed
+	order_count = 6 + (effective_seed % 3)
+	delivery_locations = 2 + (effective_seed % 2)
+	dynamic_ratio = 0.50 + (0.05 * (effective_seed % 3))
+
+	return EnvironmentConfig(
+		width=16,
+		height=16,
+		max_steps=165,
+		max_orders=order_count,
+		delivery_locations_per_order=delivery_locations,
+		priority_high_ratio=0.75,
+		obstacle_density=0.20,
+		dynamic_obstacles_enabled=True,
+		dynamic_obstacle_ratio=min(0.65, dynamic_ratio),
+		traffic_density=0.22,
+		traffic_extra_cost=5,
+		battery_enabled=True,
+		battery_capacity=55,
+		battery_recharge_rate=8,
+		seed=effective_seed,
+	)
+
+
+def get_definition() -> Dict[str, Any]:
+	return {
+		"name": TASK_NAME,
+		"difficulty": TASK_DIFFICULTY,
+		"deterministic": True,
+		"default_seed": DEFAULT_SEED,
+		"description": DESCRIPTION,
+		"objective": OBJECTIVE,
+		"success_condition": SUCCESS_CONDITION,
+		"parameters": TASK_PARAMETERS,
+	}

tasks/medium.py

@@ -0,0 +1,74 @@
+from __future__ import annotations
+
+from typing import Any, Dict
+
+from env.models import EnvironmentConfig
+
+
+TASK_NAME = "medium"
+TASK_DIFFICULTY = 2
+DEFAULT_SEED = 202
+DESCRIPTION = "Multi-order delivery with static obstacles and realistic traffic friction."
+OBJECTIVE = "Optimize route throughput across multiple concurrent orders while navigating blocked streets."
+SUCCESS_CONDITION: Dict[str, Any] = {
+	"completion_rate_min": 0.85,
+	"max_steps": 140,
+	"invalid_action_rate_max": 0.08,
+}
+
+TASK_PARAMETERS: Dict[str, Any] = {
+	"grid_size": {"width": 10, "height": 10},
+	"orders": {"min": 3, "max": 4},
+	"variation": {
+		"seeded_layout": True,
+		"order_mix": "standard_multi_order",
+		"traffic_profile": "weekday_traffic",
+	},
+	"constraints": {
+		"obstacles": True,
+		"dynamic_obstacles": False,
+		"traffic_zones": True,
+		"order_priority": False,
+		"multiple_delivery_locations": False,
+		"battery_constraint": False,
+		"recharge": False,
+		"time_penalty": False,
+	},
+}
+
+
+def get_config(seed: int | None = None) -> EnvironmentConfig:
+	effective_seed = DEFAULT_SEED if seed is None else seed
+	order_count = 3 + (effective_seed % 2)
+	traffic_cost = 2 + (effective_seed % 2)
+
+	return EnvironmentConfig(
+		width=10,
+		height=10,
+		max_steps=140,
+		max_orders=order_count,
+		delivery_locations_per_order=1,
+		priority_high_ratio=0.0,
+		obstacle_density=0.12,
+		dynamic_obstacles_enabled=False,
+		dynamic_obstacle_ratio=0.0,
+		traffic_density=0.11,
+		traffic_extra_cost=traffic_cost,
+		battery_enabled=False,
+		battery_capacity=100,
+		battery_recharge_rate=10,
+		seed=effective_seed,
+	)
+
+
+def get_definition() -> Dict[str, Any]:
+	return {
+		"name": TASK_NAME,
+		"difficulty": TASK_DIFFICULTY,
+		"deterministic": True,
+		"default_seed": DEFAULT_SEED,
+		"description": DESCRIPTION,
+		"objective": OBJECTIVE,
+		"success_condition": SUCCESS_CONDITION,
+		"parameters": TASK_PARAMETERS,
+	}

tasks/registry.py

@@ -0,0 +1,73 @@
+from __future__ import annotations
+
+from typing import Any, Callable, Dict, List
+
+from env.models import EnvironmentConfig
+from tasks import easy, hard, medium
+
+
+TaskFactory = Callable[[int | None], EnvironmentConfig]
+
+
+TASK_REGISTRY: Dict[str, Dict[str, Any]] = {
+	easy.TASK_NAME: {
+		"definition": easy.get_definition(),
+		"factory": easy.get_config,
+	},
+	medium.TASK_NAME: {
+		"definition": medium.get_definition(),
+		"factory": medium.get_config,
+	},
+	hard.TASK_NAME: {
+		"definition": hard.get_definition(),
+		"factory": hard.get_config,
+	},
+}
+
+TASK_DIFFICULTY_ORDER: List[str] = [easy.TASK_NAME, medium.TASK_NAME, hard.TASK_NAME]
+
+
+def _ordered_task_names() -> List[str]:
+	return [name for name in TASK_DIFFICULTY_ORDER if name in TASK_REGISTRY]
+
+
+def list_tasks() -> List[str]:
+	return _ordered_task_names()
+
+
+def get_task_registry() -> Dict[str, Dict[str, Any]]:
+	return {
+		name: dict(item["definition"])
+		for name in _ordered_task_names()
+		for item in [TASK_REGISTRY[name]]
+	}
+
+
+def get_task_definition(task_name: str) -> Dict[str, Any]:
+	item = TASK_REGISTRY.get(task_name)
+	if item is None:
+		raise ValueError(f"Unknown task '{task_name}'. Available: {list_tasks()}")
+	return dict(item["definition"])
+
+
+def get_task_config(task_name: str, seed: int | None = None) -> EnvironmentConfig:
+	item = TASK_REGISTRY.get(task_name)
+	if item is None:
+		raise ValueError(f"Unknown task '{task_name}'. Available: {list_tasks()}")
+
+	factory = item["factory"]
+	if not callable(factory):
+		raise RuntimeError(f"Task '{task_name}' has invalid factory")
+	return factory(seed)
+
+
+def get_all_task_metadata() -> List[Dict[str, Any]]:
+	return [
+		dict(TASK_REGISTRY[name]["definition"])
+		for name in _ordered_task_names()
+	]
+
+
+def list_tasks_with_metadata() -> List[Dict[str, Any]]:
+	"""Compatibility alias for tooling that expects this function name."""
+	return get_all_task_metadata()

tests/test_api.py

@@ -0,0 +1,263 @@
+from __future__ import annotations
+
+from fastapi.testclient import TestClient
+
+from app.main import app
+from app.routes import env_service
+
+
+client = TestClient(app)
+
+
+def test_root_endpoint_returns_service_info() -> None:
+    response = client.get("/")
+    assert response.status_code == 200
+    payload = response.json()
+    assert payload["status"] == "ok"
+    assert payload["health"] == "/health"
+
+
+def test_tasks_endpoint_returns_json() -> None:
+    response = client.get("/tasks")
+    assert response.status_code == 200
+    payload = response.json()
+    assert "tasks" in payload
+    assert "action_schema" in payload
+    assert isinstance(payload["tasks"], list)
+    assert isinstance(payload["action_schema"], dict)
+
+    for task in payload["tasks"]:
+        assert "name" in task
+        assert "description" in task
+        assert "difficulty" in task
+        assert "parameters" in task
+        assert "success_condition" in task
+
+    action_schema = payload["action_schema"]
+    assert action_schema["type"] == "object"
+    assert action_schema["required"] == ["move", "accept_order", "deliver_order", "wait"]
+    assert action_schema["additionalProperties"] is False
+    assert "properties" in action_schema
+    assert "move" in action_schema["properties"]
+    assert "accept_order" in action_schema["properties"]
+    assert "deliver_order" in action_schema["properties"]
+    assert "wait" in action_schema["properties"]
+    assert set(action_schema["properties"]["move"]["enum"]) == {
+        "up",
+        "down",
+        "left",
+        "right",
+        "stay",
+        None,
+    }
+
+
+def test_state_before_reset_returns_uninitialized_state() -> None:
+    env_service._env = None
+    env_service._task = None
+    env_service._success_condition = {}
+    env_service._trajectory = []
+    response = client.get("/state")
+    assert response.status_code == 200
+    payload = response.json()
+    assert payload["initialized"] is False
+    assert payload["done"] is False
+    assert payload["step_count"] == 0
+    assert payload["observation"] is None
+
+
+def test_reset_step_state_baseline_and_grader_flow() -> None:
+    reset_response = client.post("/reset", json={"task": "easy", "seed": 7})
+    assert reset_response.status_code == 200
+
+    baseline_response = client.get("/baseline", params={"task": "easy", "seed": 7, "max_steps": 20})
+    assert baseline_response.status_code == 200
+    baseline_payload = baseline_response.json()
+    assert baseline_payload["agent"] == "BaselineGreedyAgent"
+    assert baseline_payload["task"] == "easy"
+    assert baseline_payload["steps_executed"] <= 20
+    assert 0.0 <= baseline_payload["score"] <= 1.0
+    assert "report" in baseline_payload
+
+    step_response = client.post(
+        "/step",
+        json={"move": None, "accept_order": None, "deliver_order": False, "wait": True},
+    )
+    assert step_response.status_code == 200
+
+    state_response = client.get("/state")
+    assert state_response.status_code == 200
+
+    grader_response = client.get("/grader")
+    assert grader_response.status_code == 200
+    grader_payload = grader_response.json()
+    assert "report" in grader_payload
+    assert 0.0 <= grader_payload["report"]["score"] <= 1.0
+
+
+def test_reset_without_body_uses_defaults() -> None:
+    response = client.post("/reset")
+    assert response.status_code == 200
+    payload = response.json()
+    assert "agent_location" in payload
+
+
+def test_step_accepts_direct_action_payload() -> None:
+    client.post("/reset", json={"task": "easy", "seed": 7})
+    response = client.post(
+        "/step",
+        json={"move": None, "accept_order": None, "deliver_order": False, "wait": True},
+    )
+    assert response.status_code == 200
+    payload = response.json()
+    assert {"observation", "reward", "done", "info"}.issubset(payload.keys())
+    assert isinstance(payload["reward"], float)
+    assert isinstance(payload["done"], bool)
+    assert isinstance(payload["info"], dict)
+
+
+def test_step_rejects_wrapper_payload_for_strict_schema() -> None:
+    client.post("/reset", json={"task": "easy", "seed": 7})
+    response = client.post(
+        "/step",
+        json={
+            "action": {
+                "move": None,
+                "accept_order": None,
+                "deliver_order": False,
+                "wait": True,
+            }
+        },
+    )
+    assert response.status_code == 400
+    payload = response.json()
+    assert payload["detail"]["error"] == "bad_request"
+
+
+def test_step_invalid_payload_returns_clean_error() -> None:
+    client.post("/reset", json={"task": "easy", "seed": 7})
+    response = client.post("/step", json={"move": "right"})
+    assert response.status_code == 400
+    payload = response.json()
+    assert payload["detail"]["error"] == "bad_request"
+
+
+def test_grader_before_reset_returns_clean_error() -> None:
+    env_service._env = None
+    env_service._task = None
+    env_service._success_condition = {}
+    env_service._trajectory = []
+    response = client.get("/grader")
+    assert response.status_code == 400
+    payload = response.json()
+    assert payload["detail"]["error"] == "bad_request"
+
+
+def test_baseline_runs_without_active_environment() -> None:
+    env_service._env = None
+    env_service._task = None
+    env_service._success_condition = {}
+    env_service._trajectory = []
+
+    response = client.get("/baseline", params={"task": "easy", "seed": 1, "max_steps": 10})
+    assert response.status_code == 200
+
+    payload = response.json()
+    assert payload["agent"] == "BaselineGreedyAgent"
+    assert payload["task"] == "easy"
+    assert payload["steps_executed"] <= 10
+    assert isinstance(payload["done"], bool)
+    assert 0.0 <= payload["score"] <= 1.0
+
+
+def test_baseline_invalid_task_returns_clean_error() -> None:
+    response = client.get("/baseline", params={"task": "unknown_task"})
+    assert response.status_code == 400
+    payload = response.json()
+    assert payload["detail"]["error"] == "bad_request"
+
+
+def test_reset_from_scenario_accepts_real_world_payload() -> None:
+    response = client.post(
+        "/reset_from_scenario",
+        json={
+            "seed": 99,
+            "success_condition": {
+                "completion_rate_min": 1.0,
+                "max_steps": 20,
+                "invalid_action_rate_max": 0.2,
+                "battery_depletion": False,
+            },
+            "scenario": {
+                "width": 6,
+                "height": 6,
+                "max_steps": 20,
+                "agent_start": {"x": 0, "y": 0},
+                "orders": [
+                    {
+                        "order_id": "r1",
+                        "pickup": {"x": 1, "y": 0},
+                        "dropoff": {"x": 2, "y": 0},
+                        "delivery_locations": [{"x": 2, "y": 0}],
+                        "priority": "high",
+                    }
+                ],
+                "obstacles": [{"x": 4, "y": 4}],
+                "dynamic_obstacles": [{"x": 4, "y": 5}],
+                "traffic_zones": [{"location": {"x": 3, "y": 0}, "extra_cost": 3}],
+                "charging_stations": [{"x": 0, "y": 0}],
+                "battery_profile": {
+                    "enabled": True,
+                    "capacity": 10,
+                    "recharge_rate": 4,
+                    "initial_level": 7,
+                },
+            },
+        },
+    )
+    assert response.status_code == 200
+    payload = response.json()
+    assert payload["grid_width"] == 6
+    assert payload["grid_height"] == 6
+    assert payload["battery_level"] == 7
+    assert len(payload["pending_orders"]) == 1
+    assert sorted((item["x"], item["y"]) for item in payload["dynamic_obstacles"]) == [(4, 5)]
+
+    step_response = client.post(
+        "/step",
+        json={"move": None, "accept_order": None, "deliver_order": False, "wait": True},
+    )
+    assert step_response.status_code == 200
+
+    grader_response = client.get("/grader")
+    assert grader_response.status_code == 200
+    grader_payload = grader_response.json()
+    assert grader_payload["task"] == "scenario"
+    assert grader_payload["report"]["success_condition_used"]["max_steps"] == 20
+
+
+def test_reset_from_scenario_rejects_invalid_world_geometry() -> None:
+    response = client.post(
+        "/reset_from_scenario",
+        json={
+            "scenario": {
+                "width": 6,
+                "height": 6,
+                "max_steps": 20,
+                "orders": [
+                    {
+                        "pickup": {"x": 1, "y": 1},
+                        "dropoff": {"x": 2, "y": 2},
+                    }
+                ],
+                "obstacles": [{"x": 9, "y": 9}],
+                "dynamic_obstacles": [],
+                "traffic_zones": [],
+                "charging_stations": [{"x": 0, "y": 0}],
+                "battery_profile": {"enabled": False},
+            }
+        },
+    )
+    assert response.status_code == 400
+    payload = response.json()
+    assert payload["detail"]["error"] == "bad_request"

tests/test_env.py

@@ -0,0 +1,177 @@
+from __future__ import annotations
+
+from env.environment import LastMileDeliveryEnvironment
+from env.models import ActionType, EnvironmentConfig, RewardConfig, ScenarioDefinition, SimulatorAction
+
+
+def build_env() -> LastMileDeliveryEnvironment:
+	config = EnvironmentConfig(
+		width=6,
+		height=6,
+		max_steps=50,
+		max_orders=2,
+		obstacle_density=0.0,
+		traffic_density=0.0,
+		battery_enabled=False,
+		seed=7,
+	)
+	rewards = RewardConfig(
+		delivery_reward=50,
+		destination_reward=10,
+		step_penalty=-1,
+		invalid_action_penalty=-20,
+		delay_penalty=-5,
+	)
+	return LastMileDeliveryEnvironment(config=config, reward_config=rewards)
+
+
+def _observation_snapshot(observation) -> tuple:
+	orders = tuple(
+		sorted(
+			(
+				order.order_id,
+				order.pickup.x,
+				order.pickup.y,
+				order.dropoff.x,
+				order.dropoff.y,
+			)
+			for order in observation.pending_orders
+		)
+	)
+	obstacles = tuple(sorted((item.x, item.y) for item in observation.obstacles))
+	traffic = tuple(sorted((zone.location.x, zone.location.y, zone.extra_cost) for zone in observation.traffic_zones))
+	return orders, obstacles, traffic
+
+
+def test_reset_and_state_contract() -> None:
+	env = build_env()
+	observation = env.reset()
+
+	assert observation.agent_location.x == 0
+	assert observation.agent_location.y == 0
+	assert isinstance(observation.pending_orders, list)
+	assert observation.current_order is None
+
+	state = env.state()
+	assert state.initialized is True
+	assert state.done is False
+	assert state.observation is not None
+	assert state.step_count == 0
+	assert state.total_reward == 0.0
+
+
+def test_invalid_move_penalty() -> None:
+	env = build_env()
+	env.reset()
+
+	observation, reward, done, info = env.step(SimulatorAction(action_type=ActionType.MOVE, direction="left"))
+
+	assert observation.step_count == 1
+	assert done is False
+	assert info["invalid_action"] is True
+	assert reward <= -21
+
+
+def test_observation_fields_present() -> None:
+	env = build_env()
+	observation = env.reset()
+
+	assert hasattr(observation, "agent_location")
+	assert hasattr(observation, "pending_orders")
+	assert hasattr(observation, "current_order")
+	assert hasattr(observation, "obstacles")
+	assert hasattr(observation, "battery_level")
+
+
+def test_deterministic_reset_with_seed() -> None:
+	env = build_env()
+	first = env.reset()
+	second = env.reset()
+
+	assert _observation_snapshot(first) == _observation_snapshot(second)
+
+
+def test_reset_accepts_seed_and_reproduces_environment_state() -> None:
+	config = EnvironmentConfig(
+		width=8,
+		height=8,
+		max_steps=80,
+		max_orders=3,
+		obstacle_density=0.2,
+		traffic_density=0.2,
+		battery_enabled=False,
+		seed=None,
+	)
+	env = LastMileDeliveryEnvironment(config=config)
+
+	seed_a_first = env.reset(seed=111)
+	seed_a_second = env.reset(seed=111)
+	seed_b = env.reset(seed=222)
+
+	assert _observation_snapshot(seed_a_first) == _observation_snapshot(seed_a_second)
+	assert _observation_snapshot(seed_a_first) != _observation_snapshot(seed_b)
+
+
+def test_step_after_done_is_stable_noop() -> None:
+	config = EnvironmentConfig(
+		width=6,
+		height=6,
+		max_steps=1,
+		max_orders=1,
+		obstacle_density=0.0,
+		traffic_density=0.0,
+		battery_enabled=False,
+		seed=7,
+	)
+	env = LastMileDeliveryEnvironment(config=config)
+	env.reset()
+
+	_, _, done_first, _ = env.step(SimulatorAction(action_type=ActionType.WAIT))
+	assert done_first is True
+
+	_, reward_second, done_second, info_second = env.step(SimulatorAction(action_type=ActionType.WAIT))
+	assert done_second is True
+	assert reward_second == 0.0
+	assert info_second["message"] == "episode_already_done"
+
+
+def test_reset_with_scenario_uses_external_world_state() -> None:
+	env = build_env()
+	scenario = ScenarioDefinition.model_validate(
+		{
+			"width": 7,
+			"height": 7,
+			"max_steps": 40,
+			"agent_start": {"x": 1, "y": 1},
+			"orders": [
+				{
+					"order_id": "custom_1",
+					"pickup": {"x": 2, "y": 1},
+					"dropoff": {"x": 4, "y": 1},
+					"delivery_locations": [{"x": 4, "y": 1}],
+					"priority": "high",
+				}
+			],
+			"obstacles": [{"x": 5, "y": 5}],
+			"dynamic_obstacles": [{"x": 5, "y": 4}],
+			"traffic_zones": [{"location": {"x": 3, "y": 1}, "extra_cost": 3}],
+			"charging_stations": [{"x": 1, "y": 1}],
+			"battery_profile": {
+				"enabled": True,
+				"capacity": 10,
+				"recharge_rate": 4,
+				"initial_level": 6,
+			},
+		}
+	)
+
+	observation = env.reset_with_scenario(scenario=scenario, seed=13)
+
+	assert observation.grid_width == 7
+	assert observation.grid_height == 7
+	assert observation.agent_location.x == 1
+	assert observation.agent_location.y == 1
+	assert observation.battery_level == 6
+	assert sorted((item.x, item.y) for item in observation.obstacles) == [(5, 4), (5, 5)]
+	assert len(observation.pending_orders) == 1
+	assert observation.pending_orders[0].order_id == "custom_1"

tests/test_grader.py

@@ -0,0 +1,270 @@
+from __future__ import annotations
+
+from env.models import Observation, Order, OrderPriority, Position, StepInfo, StepResult
+from grader.grader import DeliveryEpisodeGrader
+
+
+def _order(
+    order_id: str,
+    pickup: tuple[int, int],
+    dropoff: tuple[int, int],
+    priority: OrderPriority = OrderPriority.LOW,
+    accepted_step: int | None = None,
+) -> Order:
+    return Order(
+        order_id=order_id,
+        pickup=Position(x=pickup[0], y=pickup[1]),
+        dropoff=Position(x=dropoff[0], y=dropoff[1]),
+        delivery_locations=[Position(x=dropoff[0], y=dropoff[1])],
+        priority=priority,
+        accepted_step=accepted_step,
+    )
+
+
+def _obs(step: int, pending: list[Order]) -> Observation:
+    return Observation(
+        grid_width=6,
+        grid_height=6,
+        agent_location=Position(x=0, y=0),
+        pending_orders=pending,
+        current_order=None,
+        obstacles=[],
+        dynamic_obstacles=[],
+        traffic_zones=[],
+        charging_stations=[],
+        battery_level=100,
+        step_count=step,
+        total_reward=0.0,
+    )
+
+
+def test_grader_is_deterministic() -> None:
+    grader = DeliveryEpisodeGrader()
+    order_1 = _order("order_1", (1, 0), (2, 0), OrderPriority.HIGH, accepted_step=1)
+    order_2 = _order("order_2", (2, 1), (3, 1), OrderPriority.LOW)
+
+    trajectory: list[StepResult] = []
+    for step in range(1, 11):
+        delivered = None
+        if step == 5:
+            delivered = "order_1"
+        if step == 10:
+            delivered = "order_2"
+
+        trajectory.append(
+            StepResult(
+                observation=_obs(step, [order_1, order_2]),
+                reward=0.0,
+                done=False,
+                info=StepInfo(delivered_order_id=delivered, made_progress=True),
+            )
+        )
+
+    final_obs = _obs(10, [])
+
+    success_condition = {
+        "completion_rate_min": 1.0,
+        "high_priority_on_time_rate_min": 0.8,
+        "max_steps": 20,
+        "invalid_action_rate_max": 0.10,
+    }
+    report_one = grader.grade_episode(trajectory, final_obs, success_condition=success_condition)
+    report_two = grader.grade_episode(trajectory, final_obs, success_condition=success_condition)
+
+    assert report_one.score == report_two.score
+    assert 0.0 <= report_one.score <= 1.0
+
+
+def test_grader_completion_rate_matches_delivered_over_total() -> None:
+    grader = DeliveryEpisodeGrader()
+    order_1 = _order("order_1", (1, 0), (2, 0), accepted_step=1)
+    order_2 = _order("order_2", (2, 1), (3, 1), accepted_step=1)
+    trajectory = [
+        StepResult(
+            observation=_obs(step, [order_1, order_2]),
+            reward=0.0,
+            done=False,
+            info=StepInfo(delivered_order_id="order_1" if step == 3 else None, made_progress=True),
+        )
+        for step in range(1, 4)
+    ]
+    report = grader.grade_episode(
+        trajectory,
+        _obs(3, [order_2]),
+        success_condition={"completion_rate_min": 1.0, "max_steps": 10, "invalid_action_rate_max": 0.1},
+    )
+
+    assert report.delivered_orders == 1
+    assert report.total_orders == 2
+    assert report.completion_rate == 0.5
+    assert report.completion_component == 0.5
+
+
+def test_grader_high_priority_on_time_rate_impacts_score() -> None:
+    grader = DeliveryEpisodeGrader()
+    high = _order("order_high", (1, 0), (2, 0), OrderPriority.HIGH, accepted_step=1)
+
+    on_time_trajectory = [
+        StepResult(
+            observation=_obs(step, [high]),
+            reward=0.0,
+            done=False,
+            info=StepInfo(delivered_order_id="order_high" if step == 5 else None, made_progress=True),
+        )
+        for step in range(1, 6)
+    ]
+
+    late_trajectory = [
+        StepResult(
+            observation=_obs(step, [high]),
+            reward=0.0,
+            done=False,
+            info=StepInfo(delivered_order_id="order_high" if step == 20 else None, made_progress=True),
+        )
+        for step in range(1, 21)
+    ]
+
+    success_condition = {
+        "completion_rate_min": 1.0,
+        "high_priority_on_time_rate_min": 0.8,
+        "max_steps": 30,
+        "invalid_action_rate_max": 0.1,
+    }
+
+    report_on_time = grader.grade_episode(on_time_trajectory, _obs(5, []), success_condition=success_condition)
+    report_late = grader.grade_episode(late_trajectory, _obs(20, []), success_condition=success_condition)
+
+    assert report_on_time.high_priority_on_time_rate == 1.0
+    assert report_late.high_priority_on_time_rate == 0.0
+    assert report_on_time.priority_component > report_late.priority_component
+    assert report_on_time.score > report_late.score
+
+
+def test_grader_efficiency_uses_steps_vs_max_steps() -> None:
+    grader = DeliveryEpisodeGrader()
+    order = _order("order_1", (1, 0), (2, 0), accepted_step=1)
+
+    short_trajectory = [
+        StepResult(
+            observation=_obs(step, [order]),
+            reward=0.0,
+            done=False,
+            info=StepInfo(delivered_order_id="order_1" if step == 3 else None, made_progress=True),
+        )
+        for step in range(1, 4)
+    ]
+
+    long_trajectory = [
+        StepResult(
+            observation=_obs(step, [order]),
+            reward=0.0,
+            done=False,
+            info=StepInfo(delivered_order_id="order_1" if step == 9 else None, made_progress=True),
+        )
+        for step in range(1, 10)
+    ]
+
+    success_condition = {"completion_rate_min": 1.0, "max_steps": 10, "invalid_action_rate_max": 0.1}
+    report_short = grader.grade_episode(short_trajectory, _obs(3, []), success_condition=success_condition)
+    report_long = grader.grade_episode(long_trajectory, _obs(9, []), success_condition=success_condition)
+
+    assert report_short.efficiency_ratio > report_long.efficiency_ratio
+    assert report_short.efficiency_component > report_long.efficiency_component
+    assert report_short.score > report_long.score
+
+
+def test_grader_invalid_action_rate_reduces_penalty_component() -> None:
+    grader = DeliveryEpisodeGrader()
+    order = _order("order_1", (1, 0), (2, 0), accepted_step=1)
+
+    clean_trajectory = [
+        StepResult(
+            observation=_obs(step, [order]),
+            reward=0.0,
+            done=False,
+            info=StepInfo(delivered_order_id="order_1" if step == 5 else None, made_progress=True),
+        )
+        for step in range(1, 6)
+    ]
+
+    noisy_trajectory = [
+        StepResult(
+            observation=_obs(step, [order]),
+            reward=0.0,
+            done=False,
+            info=StepInfo(
+                delivered_order_id="order_1" if step == 5 else None,
+                invalid_action=step in {1, 3, 4},
+                made_progress=step == 5,
+            ),
+        )
+        for step in range(1, 6)
+    ]
+
+    success_condition = {"completion_rate_min": 1.0, "max_steps": 10, "invalid_action_rate_max": 0.1}
+    clean_report = grader.grade_episode(clean_trajectory, _obs(5, []), success_condition=success_condition)
+    noisy_report = grader.grade_episode(noisy_trajectory, _obs(5, []), success_condition=success_condition)
+
+    assert clean_report.invalid_action_rate < noisy_report.invalid_action_rate
+    assert clean_report.penalty_component > noisy_report.penalty_component
+    assert clean_report.score > noisy_report.score
+
+
+def test_grader_no_high_priority_orders_is_handled_cleanly() -> None:
+    grader = DeliveryEpisodeGrader()
+    low = _order("order_low", (1, 0), (2, 0), OrderPriority.LOW, accepted_step=1)
+    trajectory = [
+        StepResult(
+            observation=_obs(step, [low]),
+            reward=0.0,
+            done=False,
+            info=StepInfo(delivered_order_id="order_low" if step == 3 else None, made_progress=True),
+        )
+        for step in range(1, 4)
+    ]
+
+    report = grader.grade_episode(
+        trajectory,
+        _obs(3, []),
+        success_condition={
+            "completion_rate_min": 1.0,
+            "high_priority_on_time_rate_min": 0.8,
+            "max_steps": 20,
+            "invalid_action_rate_max": 0.1,
+        },
+    )
+
+    assert report.high_priority_total_orders == 0
+    assert report.high_priority_on_time_rate == 1.0
+    assert any("No high-priority orders were present" in note for note in report.edge_case_handling)
+
+
+def test_grader_is_not_constant_for_different_step_counts() -> None:
+    grader = DeliveryEpisodeGrader()
+    order = _order("order_1", (1, 0), (2, 0), OrderPriority.LOW)
+
+    short_trajectory = [
+        StepResult(
+            observation=_obs(1, [order]),
+            reward=0.0,
+            done=False,
+            info=StepInfo(made_progress=False),
+        )
+    ]
+
+    long_trajectory = [
+        StepResult(
+            observation=_obs(step, [order]),
+            reward=0.0,
+            done=False,
+            info=StepInfo(made_progress=False),
+        )
+        for step in range(1, 4)
+    ]
+
+    success_condition = {"completion_rate_min": 1.0, "max_steps": 10, "invalid_action_rate_max": 0.1}
+    short_report = grader.grade_episode(short_trajectory, _obs(1, [order]), success_condition=success_condition)
+    long_report = grader.grade_episode(long_trajectory, _obs(3, [order]), success_condition=success_condition)
+
+    assert short_report.score != long_report.score
+    assert long_report.score < short_report.score

tests/test_inference_format.py

@@ -0,0 +1,117 @@
+from __future__ import annotations
+
+import re
+
+from env.environment import LastMileDeliveryEnvironment
+from env.models import ActionType, Direction, SimulatorAction
+from inference import format_end_line, format_start_line, format_step_line, inference, run_baseline_suite, run_episode
+from tasks.registry import get_task_config
+
+
+def test_start_line_exact_format() -> None:
+    line = format_start_line(task="easy", env_name="LastMileDeliveryEnvironment", model_name="test-model")
+    assert line == "[START] task=easy env=LastMileDeliveryEnvironment model=test-model"
+
+
+def test_step_line_exact_format() -> None:
+    action = SimulatorAction(action_type=ActionType.MOVE, direction=Direction.UP)
+    line = format_step_line(step_index=1, action=action, reward=0.0, done=False, error=None)
+    assert line == (
+        '[STEP] step=1 action={"accept_order":null,"deliver_order":false,"move":"up","wait":false} '
+        'reward=0.00 done=false error=null'
+    )
+
+
+def test_end_line_exact_format() -> None:
+    line = format_end_line(success=True, steps=12, rewards=[54.0, -1.0, 3.456])
+    assert line == "[END] success=true steps=12 rewards=54.00,-1.00,3.46"
+
+
+def test_run_episode_always_prints_end_on_error(monkeypatch, capsys) -> None:
+    monkeypatch.delenv("HF_TOKEN", raising=False)
+    monkeypatch.delenv("MODEL_NAME", raising=False)
+    monkeypatch.delenv("API_BASE_URL", raising=False)
+    monkeypatch.delenv("OPENAI_API_KEY", raising=False)
+    monkeypatch.delenv("OPENAI_MODEL", raising=False)
+    monkeypatch.delenv("OPENAI_BASE_URL", raising=False)
+    monkeypatch.delenv("OPENAI_USE_MODEL", raising=False)
+
+    result = run_episode(task="easy", seed=42, max_steps=1)
+    output_lines = [line.strip() for line in capsys.readouterr().out.strip().splitlines() if line.strip()]
+
+    assert output_lines[0].startswith("[START] ")
+    assert any(line.startswith("[STEP] ") for line in output_lines)
+    assert output_lines[-1].startswith("[END] ")
+    assert " rewards=" in output_lines[-1]
+    assert result["success"] is False
+
+
+def test_run_episode_with_env_is_strict_and_has_no_extra_output(monkeypatch, capsys) -> None:
+    monkeypatch.setenv("HF_TOKEN", "test-key")
+    monkeypatch.setenv("MODEL_NAME", "test-model")
+    monkeypatch.setenv("OPENAI_API_KEY", "test-key")
+    monkeypatch.setenv("OPENAI_MODEL", "test-model")
+    monkeypatch.setenv("OPENAI_USE_MODEL", "0")
+
+    result = run_episode(task="easy", seed=1, max_steps=1)
+    output_lines = [line.strip() for line in capsys.readouterr().out.strip().splitlines() if line.strip()]
+
+    assert len(output_lines) == 3
+    assert output_lines[0] == "[START] task=easy env=LastMileDeliveryEnvironment model=test-model"
+    assert re.fullmatch(
+        r'\[STEP\] step=1 action=\{.*\} reward=-?\d+\.\d{2} done=(true|false) error=null',
+        output_lines[1],
+    )
+    assert re.fullmatch(
+        r'\[END\] success=(true|false) steps=\d+ rewards=-?\d+\.\d{2}(,-?\d+\.\d{2})*',
+        output_lines[2],
+    )
+    assert result["steps"] == 1
+
+
+def test_run_episode_is_deterministic_for_same_seed(monkeypatch, capsys) -> None:
+    monkeypatch.setenv("HF_TOKEN", "test-key")
+    monkeypatch.setenv("MODEL_NAME", "test-model")
+    monkeypatch.setenv("OPENAI_API_KEY", "test-key")
+    monkeypatch.setenv("OPENAI_MODEL", "test-model")
+    monkeypatch.setenv("OPENAI_USE_MODEL", "0")
+
+    result_one = run_episode(task="easy", seed=7, max_steps=5)
+    output_one = [line.strip() for line in capsys.readouterr().out.strip().splitlines() if line.strip()]
+
+    result_two = run_episode(task="easy", seed=7, max_steps=5)
+    output_two = [line.strip() for line in capsys.readouterr().out.strip().splitlines() if line.strip()]
+
+    assert result_one == result_two
+    assert output_one == output_two
+
+
+def test_inference_same_input_same_output(monkeypatch) -> None:
+    monkeypatch.setenv("HF_TOKEN", "test-key")
+    monkeypatch.setenv("MODEL_NAME", "test-model")
+    monkeypatch.setenv("OPENAI_API_KEY", "test-key")
+    monkeypatch.setenv("OPENAI_MODEL", "test-model")
+    monkeypatch.setenv("OPENAI_USE_MODEL", "0")
+
+    config = get_task_config(task_name="easy", seed=11)
+    env = LastMileDeliveryEnvironment(config=config)
+    observation = env.reset().model_dump()
+
+    action_one = inference(observation)
+    action_two = inference(observation)
+
+    assert action_one == action_two
+
+
+def test_run_baseline_suite_is_deterministic_for_same_seed(monkeypatch) -> None:
+    monkeypatch.setenv("HF_TOKEN", "test-key")
+    monkeypatch.setenv("MODEL_NAME", "test-model")
+    monkeypatch.setenv("OPENAI_API_KEY", "test-key")
+    monkeypatch.setenv("OPENAI_MODEL", "test-model")
+    monkeypatch.setenv("OPENAI_USE_MODEL", "0")
+
+    suite_one = run_baseline_suite(seed=9, max_steps=20)
+    suite_two = run_baseline_suite(seed=9, max_steps=20)
+
+    assert [item["task"] for item in suite_one["tasks"]] == ["easy", "medium", "hard"]
+    assert suite_one == suite_two

tests/test_real_world_constraints.py

@@ -0,0 +1,148 @@
+from __future__ import annotations
+
+from env.environment import LastMileDeliveryEnvironment
+from env.models import ActionType, Direction, EnvironmentConfig, OrderPriority, SimulatorAction
+
+
+def _move_towards(env: LastMileDeliveryEnvironment, target_x: int, target_y: int) -> tuple[float, bool]:
+    obs = env.current_observation()
+    if obs.agent_location.x < target_x:
+        _, reward, done, _ = env.step(SimulatorAction(action_type=ActionType.MOVE, direction=Direction.RIGHT))
+        return reward, done
+    if obs.agent_location.x > target_x:
+        _, reward, done, _ = env.step(SimulatorAction(action_type=ActionType.MOVE, direction=Direction.LEFT))
+        return reward, done
+    if obs.agent_location.y < target_y:
+        _, reward, done, _ = env.step(SimulatorAction(action_type=ActionType.MOVE, direction=Direction.DOWN))
+        return reward, done
+    _, reward, done, _ = env.step(SimulatorAction(action_type=ActionType.MOVE, direction=Direction.UP))
+    return reward, done
+
+
+def test_observation_includes_new_constraints() -> None:
+    config = EnvironmentConfig(
+        width=8,
+        height=8,
+        max_steps=80,
+        max_orders=2,
+        delivery_locations_per_order=2,
+        priority_high_ratio=1.0,
+        obstacle_density=0.2,
+        dynamic_obstacles_enabled=True,
+        dynamic_obstacle_ratio=0.5,
+        traffic_density=0.1,
+        battery_enabled=True,
+        battery_capacity=20,
+        battery_recharge_rate=5,
+        seed=9,
+    )
+    env = LastMileDeliveryEnvironment(config=config)
+    observation = env.reset()
+
+    assert isinstance(observation.dynamic_obstacles, list)
+    assert isinstance(observation.charging_stations, list)
+    assert len(observation.pending_orders) > 0
+    for order in observation.pending_orders:
+        assert len(order.delivery_locations) >= 1
+        assert order.priority in {OrderPriority.HIGH, OrderPriority.LOW}
+
+
+def test_dynamic_obstacles_are_deterministic_on_reset() -> None:
+    config = EnvironmentConfig(
+        width=8,
+        height=8,
+        max_steps=50,
+        max_orders=1,
+        obstacle_density=0.2,
+        dynamic_obstacles_enabled=True,
+        dynamic_obstacle_ratio=1.0,
+        seed=3,
+    )
+    env = LastMileDeliveryEnvironment(config=config)
+
+    first = env.reset()
+    second = env.reset()
+
+    first_dynamic = sorted((p.x, p.y) for p in first.dynamic_obstacles)
+    second_dynamic = sorted((p.x, p.y) for p in second.dynamic_obstacles)
+    assert first_dynamic == second_dynamic
+
+
+def test_battery_recharge_and_failure_flow() -> None:
+    config = EnvironmentConfig(
+        width=6,
+        height=6,
+        max_steps=30,
+        max_orders=1,
+        obstacle_density=0.0,
+        traffic_density=0.0,
+        battery_enabled=True,
+        battery_capacity=3,
+        battery_recharge_rate=3,
+        seed=5,
+    )
+    env = LastMileDeliveryEnvironment(config=config)
+    env.reset()
+
+    _, _, done1, _ = env.step(SimulatorAction(action_type=ActionType.MOVE, direction=Direction.RIGHT))
+    assert done1 is False
+
+    _, _, done2, _ = env.step(SimulatorAction(action_type=ActionType.MOVE, direction=Direction.LEFT))
+    assert done2 is False
+
+    observation, reward_wait, done3, info_wait = env.step(SimulatorAction(action_type=ActionType.WAIT))
+    assert done3 is False
+    assert info_wait["message"] == "recharged"
+    assert observation.battery_level == 3
+    assert reward_wait > 0.0
+
+    # Drain battery completely to trigger failure.
+    env.step(SimulatorAction(action_type=ActionType.MOVE, direction=Direction.RIGHT))
+    env.step(SimulatorAction(action_type=ActionType.MOVE, direction=Direction.RIGHT))
+    _, reward_fail, done_fail, info_fail = env.step(SimulatorAction(action_type=ActionType.MOVE, direction=Direction.RIGHT))
+
+    assert done_fail is True
+    assert info_fail["battery_depleted"] is True
+    assert reward_fail < 0.0
+
+
+def test_high_priority_delivery_gets_bonus_reward() -> None:
+    config = EnvironmentConfig(
+        width=6,
+        height=6,
+        max_steps=120,
+        max_orders=1,
+        delivery_locations_per_order=2,
+        priority_high_ratio=1.0,
+        obstacle_density=0.0,
+        traffic_density=0.0,
+        battery_enabled=False,
+        seed=11,
+    )
+    env = LastMileDeliveryEnvironment(config=config)
+    observation = env.reset()
+
+    order = observation.pending_orders[0]
+    env.step(SimulatorAction(action_type=ActionType.ACCEPT_ORDER, order_id=order.order_id))
+
+    pickup = order.pickup
+    while True:
+        current = env.current_observation().agent_location
+        if current.x == pickup.x and current.y == pickup.y:
+            break
+        _, done = _move_towards(env, pickup.x, pickup.y)
+        assert done is False
+
+    # Move to the primary dropoff target.
+    drop = order.delivery_locations[0]
+    while True:
+        current = env.current_observation().agent_location
+        if current.x == drop.x and current.y == drop.y:
+            break
+        _, done = _move_towards(env, drop.x, drop.y)
+        assert done is False
+
+    _, reward, _, info = env.step(SimulatorAction(action_type=ActionType.DELIVER_ORDER))
+
+    assert info["delivered_order_id"] == order.order_id
+    assert reward >= 60.0

tests/test_reward_shaping.py

@@ -0,0 +1,146 @@
+from __future__ import annotations
+
+from env.environment import LastMileDeliveryEnvironment
+from env.models import ActionType, Direction, EnvironmentConfig, SimulatorAction
+from env.utils import manhattan
+
+
+def _direction_towards(current_x: int, current_y: int, target_x: int, target_y: int) -> Direction:
+    if target_x > current_x:
+        return Direction.RIGHT
+    if target_x < current_x:
+        return Direction.LEFT
+    if target_y > current_y:
+        return Direction.DOWN
+    return Direction.UP
+
+
+def _opposite(direction: Direction) -> Direction:
+    if direction == Direction.UP:
+        return Direction.DOWN
+    if direction == Direction.DOWN:
+        return Direction.UP
+    if direction == Direction.LEFT:
+        return Direction.RIGHT
+    return Direction.LEFT
+
+
+def _move_to(env: LastMileDeliveryEnvironment, target_x: int, target_y: int) -> None:
+    while True:
+        current = env.current_observation().agent_location
+        if current.x == target_x and current.y == target_y:
+            return
+        direction = _direction_towards(current.x, current.y, target_x, target_y)
+        env.step(SimulatorAction(action_type=ActionType.MOVE, direction=direction))
+
+
+def _run_delivery(seed: int, extra_waits: int) -> float:
+    config = EnvironmentConfig(
+        width=8,
+        height=8,
+        max_steps=200,
+        max_orders=1,
+        delivery_locations_per_order=1,
+        priority_high_ratio=0.0,
+        obstacle_density=0.0,
+        traffic_density=0.0,
+        battery_enabled=False,
+        seed=seed,
+    )
+    env = LastMileDeliveryEnvironment(config=config)
+    observation = env.reset()
+    order = observation.pending_orders[0]
+
+    env.step(SimulatorAction(action_type=ActionType.ACCEPT_ORDER, order_id=order.order_id))
+    _move_to(env, order.pickup.x, order.pickup.y)
+    env.step(SimulatorAction(action_type=ActionType.DELIVER_ORDER))  # pickup transition
+
+    for _ in range(extra_waits):
+        env.step(SimulatorAction(action_type=ActionType.WAIT))
+
+    drop = order.delivery_locations[0]
+    _move_to(env, drop.x, drop.y)
+    _, reward, _, info = env.step(SimulatorAction(action_type=ActionType.DELIVER_ORDER))
+    assert info["delivered_order_id"] == order.order_id
+    return reward
+
+
+def test_progress_reward_is_dense_and_directional() -> None:
+    config = EnvironmentConfig(
+        width=8,
+        height=8,
+        max_steps=40,
+        max_orders=2,
+        obstacle_density=0.0,
+        traffic_density=0.0,
+        battery_enabled=False,
+        seed=13,
+    )
+    env = LastMileDeliveryEnvironment(config=config)
+    observation = env.reset()
+
+    start = observation.agent_location
+    targets = [(o.pickup.x, o.pickup.y) for o in observation.pending_orders]
+    nearest_target = min(targets, key=lambda t: manhattan((start.x, start.y), t))
+
+    direction_to_target = _direction_towards(start.x, start.y, nearest_target[0], nearest_target[1])
+    _, reward_towards, _, _ = env.step(SimulatorAction(action_type=ActionType.MOVE, direction=direction_to_target))
+
+    direction_away = _opposite(direction_to_target)
+    _, reward_away, _, _ = env.step(SimulatorAction(action_type=ActionType.MOVE, direction=direction_away))
+
+    assert reward_towards > reward_away
+
+
+def test_delay_penalty_accumulates_for_late_delivery() -> None:
+    config = EnvironmentConfig(
+        width=8,
+        height=8,
+        max_steps=80,
+        max_orders=1,
+        priority_high_ratio=1.0,
+        obstacle_density=0.0,
+        traffic_density=0.0,
+        battery_enabled=False,
+        seed=17,
+    )
+    env = LastMileDeliveryEnvironment(config=config)
+    observation = env.reset()
+
+    order = observation.pending_orders[0]
+    env.step(SimulatorAction(action_type=ActionType.ACCEPT_ORDER, order_id=order.order_id))
+
+    rewards = []
+    for _ in range(13):
+        _, reward, _, info = env.step(SimulatorAction(action_type=ActionType.WAIT))
+        rewards.append((reward, info["delay_penalty_applied"]))
+
+    assert rewards[0][1] is False
+    assert rewards[-1][1] is True
+    assert rewards[-1][0] < rewards[0][0]
+
+
+def test_invalid_action_has_strong_penalty() -> None:
+    config = EnvironmentConfig(
+        width=6,
+        height=6,
+        max_steps=20,
+        max_orders=1,
+        obstacle_density=0.0,
+        traffic_density=0.0,
+        battery_enabled=False,
+        seed=21,
+    )
+    env = LastMileDeliveryEnvironment(config=config)
+    env.reset()
+
+    _, reward, _, info = env.step({"move": "up"})
+    assert info["invalid_action"] is True
+    assert reward <= -20.0
+
+
+def test_efficient_delivery_gets_higher_bonus() -> None:
+    fast_reward = _run_delivery(seed=25, extra_waits=0)
+    slow_reward = _run_delivery(seed=25, extra_waits=6)
+
+    assert fast_reward > slow_reward

tests/test_tasks.py

@@ -0,0 +1,92 @@
+from __future__ import annotations
+
+from tasks.registry import get_task_config, get_task_definition, get_task_registry, list_tasks
+
+
+def test_registry_contains_three_tasks() -> None:
+    assert list_tasks() == ["easy", "medium", "hard"]
+
+
+def test_easy_definition_and_config() -> None:
+    definition = get_task_definition("easy")
+    config = get_task_config("easy", seed=1)
+    default_config = get_task_config("easy")
+    default_config_repeat = get_task_config("easy")
+
+    assert definition["parameters"]["orders"] == {"min": 1, "max": 1}
+    assert definition["parameters"]["constraints"]["obstacles"] is False
+    assert definition["parameters"]["constraints"]["traffic_zones"] is False
+    assert definition["parameters"]["constraints"]["order_priority"] is False
+    assert "objective" in definition
+    assert "success_condition" in definition
+    assert definition["deterministic"] is True
+    assert config.max_orders == 1
+    assert config.obstacle_density == 0.0
+    assert config.traffic_density == 0.0
+    assert config.max_steps == 60
+    assert default_config.model_dump() == default_config_repeat.model_dump()
+
+
+def test_medium_has_obstacles_and_multiple_orders() -> None:
+    config = get_task_config("medium", seed=1)
+    definition = get_task_definition("medium")
+
+    assert 3 <= config.max_orders <= 4
+    assert config.obstacle_density > 0.0
+    assert config.battery_enabled is False
+    assert config.priority_high_ratio == 0.0
+    assert definition["parameters"]["constraints"]["obstacles"] is True
+    assert definition["parameters"]["constraints"]["multiple_delivery_locations"] is False
+
+
+def test_hard_has_battery_and_time_penalty() -> None:
+    definition = get_task_definition("hard")
+    config = get_task_config("hard", seed=1)
+
+    assert definition["parameters"]["constraints"]["battery_constraint"] is True
+    assert definition["parameters"]["constraints"]["time_penalty"] is True
+    assert definition["parameters"]["constraints"]["order_priority"] is True
+    assert config.battery_enabled is True
+    assert config.dynamic_obstacles_enabled is True
+    assert config.priority_high_ratio >= 0.7
+    assert config.max_steps == 165
+
+
+def test_registry_payload_is_parameterized() -> None:
+    registry = get_task_registry()
+    assert set(registry.keys()) == {"easy", "medium", "hard"}
+    for name, item in registry.items():
+        assert item["name"] == name
+        assert "parameters" in item
+        assert "grid_size" in item["parameters"]
+        assert "objective" in item
+        assert "success_condition" in item
+
+
+def test_difficulty_progression_is_increasing() -> None:
+    easy = get_task_config("easy", seed=10)
+    medium = get_task_config("medium", seed=10)
+    hard = get_task_config("hard", seed=10)
+
+    assert easy.width < medium.width < hard.width
+    assert easy.height < medium.height < hard.height
+    assert easy.max_orders < medium.max_orders < hard.max_orders
+    assert easy.dynamic_obstacles_enabled is False
+    assert medium.dynamic_obstacles_enabled is False
+    assert hard.dynamic_obstacles_enabled is True
+    assert easy.battery_enabled is False
+    assert medium.battery_enabled is False
+    assert hard.battery_enabled is True
+    assert easy.priority_high_ratio == 0.0
+    assert medium.priority_high_ratio == 0.0
+    assert hard.priority_high_ratio > medium.priority_high_ratio
+
+
+def test_seed_reproducibility_for_variable_tasks() -> None:
+    medium_a = get_task_config("medium", seed=77)
+    medium_b = get_task_config("medium", seed=77)
+    hard_a = get_task_config("hard", seed=91)
+    hard_b = get_task_config("hard", seed=91)
+
+    assert medium_a.model_dump() == medium_b.model_dump()
+    assert hard_a.model_dump() == hard_b.model_dump()