DHDRL
/

monsoon-rl

@@ -1,196 +1,50 @@
-# Weather-Risk RL: Agricultural Early-Warning System (Indonesia)
-An RL-based agricultural weather-risk early-warning system: a multi-source
-data pipeline (Open-Meteo / ERA5 / satellite / synthetic) feeds a
-deterministic, economics-calibrated crop-risk scorer. On top of that scorer
-a MaskablePPO + GRU agent learns *which zones to inspect and when to alert*
-under a **scarce inspection budget**. Trained via curriculum learning,
-exportable to MNN for edge inference (C++/Vulkan) with MQTT product-alert +
-hidden-state sync across a node fleet.
-The current focus is a grounded **Indonesia rice deployment**: real zones,
-crop calendars, monsoon context, climatology-based anomaly scoring, basin /
-heliophysics context, curated **L1 impact labels**, and evaluation with
-**three layers**:
-1. **Proxy skill** — hazard-specific operating points (`drought_*` / `flood_*`)
-   against precip-percentile proxies (development / ranking).
-2. **Product skill** — idempotent `EMITTED` alerts vs curated L1 impact
-   labels (what a buyer should quote — **with L1 density caveats**).
-3. **Agent path** — real-trajectory eval of trained policies via relative
-   belief movement and product flags (`evaluate_checkpoint_real.py`).
-This is a research / portfolio codebase, not a production alerting service.
-See **Status**, **Verified evaluation**, **Held-out protocol**, and
-**Still open** before quoting.
 ---
-## Status (what is integrated and verified)
-| Area | State |
-|------|--------|
-| Schema v3 (`ZoneObs`, `BasinContext`, helio fields) | **Done** |
-| Climatology anomalies (precip / temp / soil / RH) for real obs | **Done** — pinned no-look-ahead for backtests |
-| RH-anomaly fungi correction (obs + forecast) | **Done** — synthetic remains true no-op |
-| Basin context (RONI / DMI) + SWPC helio fetch | **Done** — strict real-only mode available |
-| Indonesia zone registry (14 zones) + rice calendars | **Done** |
-| Hazard-specific scorecards (`drought_*` / `flood_*`) | **Done** — multi-threshold harness |
-| Product `AlertLevel` from drought/flood channels | **Done** — WARNING/CRITICAL fire on real data |
-| Threshold freeze across 4 Java zones × 2 seasons (proxy) | **Done** — see proxy tables |
-| Continuous skill (matched separations) | **Done** — positive drought + flood ranking |
-| **L1 impact label store** (`impact_labels.py` + JSON) | **Done** — curated Java v1 (**7 events**) |
-| **Product alert service** (gate + idempotent emit) | **Done** — `product_alert_service.py` + MQTT bus |
-| **Product vs L1 metrics in backtest** | **Done** — primary product skill block |
-| Helio in env obs (`basin_context` 4→8) | **Done** — SWPC live→vector verified; scorer still no helio term |
-| Env `product_actionable` = product gate (not ADVISORY) | **Done** — matches `product_alert_service` |
-| **Real-trajectory agent/scorer eval harness** | **Done** — injection fix, PATH_CHECK, relative belief metric |
-| Env real-day injection (`options["context"]`) | **Done** — A=B=C scorer product on historical days |
-| `decide_checkpoint` path integrity | **Done** — PATH_CHECK: return loop product, rs display-only |
-| Relative belief metric (`belief_raised` / Δbelief) | **Done** — null control under zero_inspect is exact at n_zones=1 **and** multi-zone |
-| **Unlabeled exclusion from P/R/F1** | **Done** — `gt_source="unlabeled"` outside all L1 spans; not counted as TN/FP |
-| **Multi-zone real injection (Blocker B)** | **Done** — `EpisodeContext` per-zone obs/forecast lists; n_zones>1 real eval path |
-| Synthetic train calibration (`clean_episode_ratio`, `event_spatial_correlation`) | **Done** — matched train/eval CLI; correlated regional events |
-| **Budget modes** (`full` / `scarce` / `triage`) | **Done** — default **triage** forces leaving ≥1 zone unvisited |
-| **Per-episode zone-order shuffle** | **Done** — `shuffle_zone_order=True` (default); breaks fixed-slot habits |
-| Dynamics pairs from historical cache | **Done** — `build_dynamics_pairs.py` (892 pairs on Java zones) |
-| Held-out protocol | **Done** — see **Held-out protocol** section below |
-| Offline self-tests (zone_obs, climatology, impact, product, backtest, pytest) | **Passing** |
-Still open items are listed only under **Still open** at the bottom — do not
-treat fixed items above as missing.
 ---
-## Repository layout
-| File | Role |
-|------|------|
-| `zone_observation.py` | Core schema v3: `ZoneObs`, `BasinContext` (incl. helio), `ForecastResult`, `RiskScore`, `ForecastConfig` (incl. `clean_episode_ratio`, `event_spatial_correlation`, `shuffle_zone_order`). |
-| `crop_risk_scorer.py` | Deterministic scoring; RH-anomaly fungi fix; **hazard-specific WARNING/CRITICAL** (`drought_risk` / `flood_risk` thresholds). |
-| `era5_data_pipeline.py` | Real fetchers (Open-Meteo, ERA5, SWPC helio, RONI/DMI); synthetic fallback; strict real-only mode. |
-| `climatology.py` | Per-zone day-of-year climatology (precip, temp, soil moisture, RH). |
-| `indonesia_zones.py` | 14 agricultural zones, rice calendars, monsoon-onset heuristic, planting advisories. |
-| `impact_labels.py` | L1 impact event schema + file-backed store; day query; match helpers. |
-| `impact_labels_java_v1.json` | Curated Karawang / Indramayu drought & flood spans (provincial/press; **7 events**). |
-| `product_alert_service.py` | Product gate (WARNING+ **or** drought≥0.35 / flood≥0.25) + idempotent emit onto `NodeTransport`. |
-| `weather_forecast_env.py` | Gymnasium env: multi-zone inspection under budget; real EpisodeContext injection; correlated event plan; **per-episode zone-order shuffle**; `believed_p` = max over active zones. |
-| `gru_weather_policy.py` | GRU feature extractor for MaskablePPO. **v3:** invariant pooled path for value/terminate (weight-transfer across `n_zones`) **plus** an equivariant per-zone `zone_action_head` computed *before* pooling, so inspect logits can condition on which zone's content looks risky. Requires constructing `MaskablePPO` with `ZoneEquivariantMaskablePolicy`, not the default `MultiInputPolicy` — the default policy's `action_net` sits on the pooled (identity-erased) features and cannot learn zone selection; see module docstring. |
-| `train_curriculum.py` / `train_kaggle.py` | Curriculum / Kaggle training; **`--budget-mode`** (`full`/`scarce`/`triage`); CLI for clean ratio and spatial correlation. |
-| `hierarchical_search.py` | Country-scale bisection search (Phase 1 = deterministic gate). |
-| `backtest_indonesia.py` | Historical replay: proxy scorecards **and** product-vs-L1 metrics; optional `--emit-product-alerts`. |
-| `evaluate_checkpoint_real.py` | Agent/scorer eval on historical pkl + L1; multi-zone packs; relative belief Δ; PATH_CHECK; zero_inspect; unlabeled exclusion. |
-| `evaluate_checkpoint.py` | Synthetic matched-distribution eval (belief gate vs product gate; zero_inspect control). |
-| `build_dynamics_pairs.py` | Historical cache → DynamicsTrainer (current, next) pairs. |
-| `mnn_export.py` / `edge_wrapper.cpp` | ONNX/MNN export and C++/Vulkan edge runtime. |
-| `node_transport.py` | Product alerts + hidden-state transport (LocalTransport or MQTT). |
-| `Indonesia_upgrade_changelog.md` | Detailed upgrade history. |
----
-## Quickstart
-```bash
-pip install -r requirements.txt
-# Offline self-tests (no network)
-python zone_observation.py
-python climatology.py
-python indonesia_zones.py
-python impact_labels.py
-python product_alert_service.py   # needs zone_observation + node_transport on PYTHONPATH
-python backtest_indonesia.py      # offline self-test groups
-python -m pytest test_*.py
-# Country-scale search demo (synthetic)
-python hierarchical_search.py
-```
-### Reproduce scorecards (network required)
-```bash
-ZONES=karawang_rice,indramayu_rice,central_java_rice,east_java_rice
-# Proxy skill — 2023 El Niño dry (drought-primary)
-python backtest_indonesia.py --mode live --forecast-backend baseline \
-  --zones $ZONES --start 2023-07-01 --end 2023-11-30 \
-  --min-drought 0.35 --scorecard scorecard_2023_4zone.json
-# Proxy skill — 2021–22 wet (flood-primary)
-python backtest_indonesia.py --mode live --forecast-backend baseline \
-  --zones $ZONES --start 2021-12-01 --end 2022-03-31 \
-  --min-flood 0.25 --scorecard scorecard_2021_22_4zone.json
-# Product skill vs L1 — dry
-python backtest_indonesia.py --mode live \
-  --zones karawang_rice,indramayu_rice \
-  --start 2023-07-01 --end 2023-11-30 \
-  --impact-labels impact_labels_java_v1.json \
-  --emit-product-alerts \
-  --out full_2023_dry_l1.json
-```
-`cdsapi not installed` log lines are **harmless** for these runs: the harness
-uses Open-Meteo archive + baseline forecast, not CDS/ERA5.
-### Train under triage pressure (required for allocation skill)
-Full-tour training (`max_steps ≥ n_zones + 1`) never forces the agent to
-**choose which zone to skip**. The visit-once mask plus
-`unvisited_zone_penalty` makes "inspect everyone, then terminate" EV-optimal.
-Policies trained that way can collapse to a fixed slot (e.g. always the same
-zone under true triage eval).
-**Budget modes** (derived `max_steps` unless you override):
-| Mode | `max_steps` | Meaning |
-|------|-------------|---------|
-| `full` | `n_zones + 1` | Can visit every zone (legacy; does **not** train selection) |
-| `scarce` | `n_zones` | Borderline; must skip terminate to visit all |
-| **`triage`** (default) | `max(1, n_zones - 1)` | **Must** leave ≥1 zone unvisited |
-**Zone-order shuffle** (`ForecastConfig.shuffle_zone_order=True` by default)
-permutes action slots each episode so index `i` is not permanently bound to
-a fixed zone identity. Disable only for fixed-order diagnostic probes
-(`shuffle_zone_order=False`). Oracle-style EV probes that index into the
-pre-shuffle event plan **must** use `shuffle_zone_order=False` (or map
-actions via `info["zone_ids"]`); otherwise “oracle” inspects the wrong slot.
-**Reward path (unified termination + budget-aware unvisited):** Budget
-exhaustion and explicit early-terminate both use `_compute_termination_reward`
-(belief-driven gain/cost/miss). `unvisited_frac` is relative to available
-budget `min(n_zones, max_steps)`, not raw `n_zones` — full mode unchanged;
-under triage the agent is not half-penalised merely for leaving zones it
-was never allowed to visit. Checkpoints trained **before** this change
-optimise a different objective; do not compare raw returns across that cut.
-```bash
-# Recommended: 2-zone triage + shuffle (defaults)
-python train_kaggle.py \
-  --out ./run_nz2_triage_shuf \
-  --n-zones 2 \
-  --budget-mode triage \
-  --steps 100000 \
-  --clean-episode-ratio 0.90 \
-  --event-spatial-correlation 0.85 \
-  --device auto
-# Expect log lines like:
-#   budget-mode=triage → max_steps=1 (full ceiling=3)
-#   must_skip_zones=yes
-```
-Curriculum phases (`train_curriculum.py`) progress the same way:
-`normal` = full → `monsoon` = scarce → later phases = triage.
-**Recipe validated to produce single-dirty selection accuracy well above
-chance, and — at sufficient step count — accuracy at 100% across every
-hazard type** (see Still open #5 for exact numbers and what's independently
-confirmed vs. still-reported-only):
 ```bash
 python train_kaggle.py \
-  --out ./run_nz2_triage_selection \
   --n-zones 2 \
   --budget-mode triage \
   --steps 2400000 \
@@ -200,586 +54,70 @@ python train_kaggle.py \
   --device auto
 ```
-`--precip-scale` is now a real `train_kaggle.py` flag (default `40.0`,
-matching the validated recipe) — earlier revisions of this doc noted it
-wasn't yet exposed on the CLI; that gap is closed.
-**Step count matters more than anything else tried.** A checkpoint at
-150k steps (`--seed 42`, same `clean`/`rho`) reached ~86% pooled accuracy
-with drought/flood already solved but fungi lagging at ~57%. A checkpoint
-from a separate, longer run reached 100% on drought and flood by 250k
-steps but fungi still only 60–67% at that point — then **100% on all
-three hazard types, independently re-verified against the real
-`weather_forecast_env.py` + `zone_observation.py` + `crop_risk_scorer.py`
-(not a stub), by 2,400,000 steps.** `target_kl` and the belief prior/signal
-blend were both floated earlier as candidate fixes for the fungi gap;
-neither was needed — it resolved with more steps on the unchanged recipe.
-**What's independently confirmed from the 2.4M checkpoint file itself:**
-`policy_class=ZoneEquivariantMaskablePolicy`, `precip_scale=40.0`,
-`_total_timesteps=5,000,000` (i.e. this was a genuine 5M-step target run,
-not a truncated one — a checkpoint saved partway through should not be
-read as "the run stopped here"). **Not independently confirmed:** the
-exact `clean_episode_ratio`/`event_spatial_correlation` used during this
-specific run (SB3 checkpoints don't store env-config kwargs, only
-`policy_kwargs`) — the `--clean-episode-ratio 0.80
---event-spatial-correlation 0.50` above is the reported training config,
-consistent with the recipe that produced it, but taken on trust rather
-than read out of the artifact the way `precip_scale` was. Same for the
-often-cited "seed 7": confirmed via the actual training command
-(`--seed 7`), which controls `ForecastConfig(seed=args.seed)` for the
-train/eval env — so environment-side determinism (event plans, synthetic
-obs noise) really was seed 7. The checkpoint's own saved SB3 state still
-shows `seed: None`, and that's expected, not a discrepancy:
-`build_model()` never passes `seed=` to `MaskablePPO(...)` itself, only
-into the env config, so PyTorch's own weight-initialization RNG was never
-pinned by `--seed`. Full run reproducibility (including weight init) is
-therefore not actually guaranteed by `--seed 7` alone — worth adding
-`set_random_seed(args.seed)` or `MaskablePPO(..., seed=args.seed)` to
-`train_kaggle.py` if exact reproduction ever matters.
-This differs from the "recommended defaults" block above in the ways that
-mattered empirically, not just cosmetically:
-1. `--event-spatial-correlation 0.50` (not the default 0.85) — raises the
-   *rate* of single-dirty episodes (the only episodes with a well-defined
-   correct zone) from ~5% to ~15% of training data. Density alone was
-   **not** sufficient on its own (see below).
-2. The `MaskablePPO` construction must use `ZoneEquivariantMaskablePolicy`
-   from `gru_weather_policy.py`, and `GRUWeatherFeaturesExtractor` must be
-   constructed with `precip_scale=40.0`. Precip lives in roughly `[0, 80]`
-   while belief lives in roughly `[0, 0.3]`; with no normalization layer,
-   precip's raw scale can suppress the much smaller but far more reliable
-   belief signal during optimization, independent of which feature is
-   actually more informative.
-3. Enough steps. 150k is sufficient for drought/flood but not fungi; 2.4M
-   is sufficient for all three. The step count between those two points
-   where fungi specifically crosses over has not been narrowed down.
-Density (`event_spatial_correlation`) and scale-normalization are
-independent, both-necessary levers, not interchangeable ones: correlation
-alone (ρ=0.50, no normalization, no equivariant policy change) held
-selection accuracy at chance across three separate runs before either the
-equivariant architecture or the normalization patch existed. Do not assume
-lowering ρ alone reproduces this result without the other two changes.
-### Agent eval on real trajectories
 ```bash
-# zero_inspect control (belief Δ must be exactly 0)
-python evaluate_checkpoint_real.py \
-  --pkl historical_continuous_indonesia_v1.pkl \
-  --impact-labels impact_labels_java_v1.json \
-  --zones karawang_rice,indramayu_rice \
-  --start 2023-07-01 --end 2023-11-30 \
-  --mode zero_inspect --n-zones 2 --max-steps 1 --horizon-days 30
-# Trained triage checkpoint (n_zones and max_steps must match training)
-python evaluate_checkpoint_real.py \
-  --pkl historical_continuous_indonesia_v1.pkl \
-  --impact-labels impact_labels_java_v1.json \
-  --zones karawang_rice,indramayu_rice \
-  --start 2023-07-01 --end 2023-11-30 \
-  --mode checkpoint --checkpoint ./run_nz2_triage_shuf/final_model.zip \
-  --n-zones 2 --max-steps 1 --horizon-days 30
 ```
-Expect: `PATH_CHECK summary: …`; zero_inspect belief_delta mean = **0.0000**;
-checkpoint belief_delta distribution non-zero. Checkpoint `n_zones` must equal
-eval `n_zones` or the harness refuses with SHAPE_MISMATCH.
-**Success criteria for triage allocation skill (not full-tour numbers):**
-1. Zero-inspect: belief_R ≈ 0, Δbelief mean ≈ 0
-2. Checkpoint: belief_R on L1 clearly above zero-inspect **— still open,
-   see Still open #5; nothing below has been checked against real L1
-   trajectories, only synthetic single-dirty episodes**
-3. Inspected-zone probe: **not** 100% one fixed zone — met (see Still open
-   #5); confirmed via raw per-slot action counts staying near 50/50 across
-   every checkpoint of a full 150k-step trajectory, and independently
-   reconfirmed at 250k and 2.4M steps on a separate longer run
-4. **Selection accuracy, not just distribution balance.** Criterion 3 above
-   only rules out a fixed-slot collapse; it does not show the policy picks
-   the zone that is *actually* dirty. Measure that directly: for
-   single-dirty episodes (`n_d == 1`, the only case with a well-defined
-   correct answer), compare the *chosen* zone's own physical dirty flag —
-   not the marginal frequency of each physical zone_id across "any zone
-   dirty" episodes. An earlier probe design computed the latter and
-   reported ~50–55% "top share on dirty" as if it measured selection skill;
-   under `shuffle_zone_order=True` that quantity is expected to sit near
-   50% for a policy with **perfect** selection skill and for one with
-   **zero** skill alike, since it never checks whether the chosen zone
-   matches its own dirty flag. Do not reuse that metric shape. See Still
-   open #5 for the corrected probe and current numbers.
-5. Prefer dirty-day / hazard-conditional metrics over overall return —
-   overall return still blends clean-day always-inspect economics with
-   whatever selection skill exists; report criterion 4's number
-   hazard-by-hazard (see Still open #5), not as a single blended figure.
----
-## Verified evaluation
-Three layers. **Quote product-vs-L1 only with L1 density caveats.** Quote
-hazard-specific proxy bars for ranking / threshold development. Quote agent
-belief results only with the relative metric, null control, and the correct
-**budget regime**. Do **not** cite ADVISORY / raw max_risk rates as product
-skill. Do **not** cite multi-zone product R without the 2-zone ablation
-below. Do **not** cite full-tour belief_R as evidence of zone-selection skill.
-### A. Product skill vs L1 impact labels
-**Method:** Curated L1 events in `impact_labels_java_v1.json` (provincial
-agriculture reports + national press; confidence < 1.0). Product positive =
-gate: `AlertLevel ≥ WARNING` **or** `drought_risk ≥ 0.35` **or**
-`flood_risk ≥ 0.25`. Days outside every L1 span are `gt_source="unlabeled"`
-and are **excluded from P/R/F1** (reported only as `unlabeled_alert_rate`).
-**L1 catalog (v1, 7 events):**
-| Zone | Hazard | Span | Conf | Notes |
-|------|--------|------|------|--------|
-| karawang_rice | flood | 2023-02-25 → 03-08 | 0.90 | ~8.9k ha puso |
-| karawang_rice | drought | 2023-08-01 → 10-22 | 0.85 | El Niño dry — **only Karawang drought L1 in 2023** |
-| indramayu_rice | flood | 2021-02-10 → 02-28 | 0.85 | Cipanas / multi-kecamatan |
-| indramayu_rice | drought (peak) | 2023-06-15 → 08-31 | 0.90 | Primary dry L1 |
-| indramayu_rice | drought (secondary) | 2023-09-01 → 10-15 | 0.80 | Post-peak stress |
-| + 2 seasonal flood rows | flood | 2021-12 → 2022-01 | 0.50–0.55 | Exploratory only |
-**Important:** "L1 event days" in evals are **range expansions** of these
-rows, not independent confirmations. Example: 2023-07–11 dry window reports
-~37–63 L1 days from a handful of spans.
-#### Dry product freeze (live backtest)
-**2023-07-01 → 2023-11-30**, zones `karawang_rice,indramayu_rice`, step 3d
-(n=102):
-| Metric | Value |
-|--------|--------|
-| L1 event days (expanded) | 63 (62% coverage) |
-| Emit rate | 56% |
-| **TP / FP / FN / TN** | **53 / 4 / 10 / 35** |
-| **Precision** | **0.930** |
-| **Recall** | **0.841** |
-| **F1** | **0.883** |
-#### L1 sparsity caveat (pkl scorer_oracle, same window)
-Scorer product vs L1 on the continuous historical cache (per-zone days,
-before unlabeled exclusion was the default reporting mode):
-| Metric | Value |
-|--------|--------|
-| P / R / F1 (legacy framing that counted boundary days as negatives) | 0.750 / 0.730 / 0.740 |
-| Indramayu | P=1.00 R=0.81 |
-| **Karawang** | **P=0.53 R=0.63** (FP-heavy under legacy framing) |
-**Karawang high-risk weeks outside L1 (do not treat as proven FPs):**
-- **Jul 15–30** — drought ≈ 0.43–0.44, **before** L1 onset (Aug 1)
-- **Oct 28 – Nov 17** — flood ≈ 0.29–0.43, **after** L1 end; **no Karawang flood L1 in 2023**
-These are **unlabeled / boundary days**. With unlabeled exclusion they no
-longer enter FP/TN counts. Do not quote Karawang P≈0.53 as settled
-false-alarm rate until those weeks are labeled or confirmed non-events.
-#### Wet product skill (limited by L1 density)
-**2021-12-01 → 2022-03-31** (n=82) — sparse lower-confidence flood L1:
-| P / R / F1 | 0.417 / 0.476 / 0.444 |
-**2021-02-01 → 2021-03-15** (n=30) — high-conf Indramayu flood, thin L1:
-| P / R / F1 | 0.250 / 1.000 / 0.400 |
-Flood product recall on labeled days can be high while precision is poor
-when L1 is thin. Proxy flood skill remains the better development signal
-until denser flood impact labels exist.
-### B. Proxy skill (hazard-specific, precip percentiles)
-**Method:** pinned Open-Meteo climatology (no look-ahead); event days from
-precip percentiles (drought ≈ 30d below ~20th pct; flood ≈ 7d above ~90th
-pct). **Not** disaster records.
-#### Rejected as product bars
-| Rule | Result |
-|------|--------|
-| `AlertLevel.ADVISORY` (~max_risk ≥ 0.0625) | Saturates (~100% of days); TN → 0 |
-| Legacy WARNING via max_risk ≥ 0.65 | Never fired under confidence damping — **superseded by hazard path** |
-| Single max_risk mid-cut (e.g. 0.35) | OK in drought seasons; **collapses in wet** when fungi saturates max_risk |
-#### Hazard-specific skill (frozen thresholds)
-**Dry: 2023-07-01 → 2023-11-30**
-| Scope | Policy | P | R | F1 | Matched drought sep |
-|-------|--------|------|------|------|---------------------|
-| 2-zone | drought_0.35 | 1.00 | 0.86 | 0.93 | +0.33 |
-| **4-zone** | **drought_0.35** | **0.94** | **0.89** | **0.92** | **+0.30** |
-4-zone dry alert mix (n=204): advisory 64 / **warning 90** / **critical 50**.
-**Wet: 2021-12-01 → 2022-03-31**
-| Scope | Policy | P | R | F1 | EV lift vs always | Matched flood sep |
-|-------|--------|------|------|------|-------------------|-------------------|
-| 2-zone | flood_0.25 | 0.61 | 0.93 | 0.74 | **+860** | +0.18 |
-| **4-zone** | **flood_0.25** | **0.52** | **0.91** | **0.66** | **+1160** | **+0.17** |
-Flood precision is **zone-dependent** (Indramayu strong; Karawang / East Java
-more FPs — also seen on live proxy backtests). Ranking skill and EV lift
-hold; do not re-tune the global cut from pooled P alone.
-### C. Agent path on real trajectories
-**Harness integrity (validated):**
-| Check | Result |
-|--------|--------|
-| Env injection (real days, not synthetic redraws) | Scorer product identical for scorer_oracle / zero_inspect / checkpoint on the same days |
-| `decide_checkpoint` overwrite | **PATH_CHECK**: return loop product; rs is display-only |
-| Relative belief null control | zero_inspect: Δbelief mean=**0.0000**, std=**0** at n_zones=1 **and** multi-zone |
-| SHAPE_MISMATCH guard | Checkpoint `n_zones` must match env; wrong shapes refused |
-| Multi-zone joint metrics | Order-invariant under zone shuffle (max / OR aggregations) |
-**Policy-sensitive metric:** `belief_raised = (terminal − initial) > 0.02`
-relative to each episode's post-reset belief. Fixed bars against `prior` /
-`rational` saturate and must not be used for agent skill.
-**Product is policy-insensitive by design** on real multi-zone packs: agent
-and zero_inspect share the same product row (scorer on episode obs). Skill
-signal is belief movement **and**, under triage, which zone was inspected.
-#### Full-tour vs triage (read this before quoting agent skill)
-| Regime | `max_steps` (n_zones=2) | What it measures |
-|--------|-------------------------|------------------|
-| **Full tour** | ≥ 3 | Can visit every zone; does **not** force selection |
-| **Triage** | 1 | Must pick **one** zone (or terminate with none) |
-Older matched trains (`run_nz2_c090`, `run_nz3_c090`) used full-tour budgets.
-They remain valid **harness and belief-movement** baselines. They are **not**
-evidence that the policy learned risk-conditioned zone allocation. Under true
-triage (`max_steps=1`), a full-tour checkpoint was observed to inspect a
-**fixed** zone on every day (slot/index bias). New allocation claims require a
-**triage-trained** checkpoint under `shuffle_zone_order=True`.
-#### Multi-zone product controls (2023-07-01 → 2023-11-30, null-fixed harness)
-Joint packs group one calendar day across selected zones. Product uses
-worst-case across zones.
-| Setting | product R | unlabeled_alert_rate | Notes |
-|---------|-----------|----------------------|--------|
-| **3-zone** (Karawang+Indramayu+Central Java) | 1.000 | 0.875 | Central Java has **zero L1** in catalog |
-| **2-zone** (Karawang+Indramayu only) | **0.773** | **0.625** | Ablation drops Central Java |
-**Central Java ablation (confirmed):** Removing Central Java cuts joint
-product R 1.000 → 0.773 and unlabeled alert rate 0.875 → 0.625. Central Java
-has no L1 coverage and independently elevates on a large fraction of days;
-under `max` aggregation it carried the joint gate. Do **not** quote
-multi-zone product R=1.0 as architecture skill.
-#### 2-zone full-tour belief baseline (matched train: clean=0.90, ρ=0.85)
-These numbers are from **full-tour** eval (`max_steps ≈ n_zones+1`). Useful
-as harness + belief-movement evidence; **not** triage allocation skill.
-| Metric | zero_inspect | full-tour checkpoint |
-|--------|--------------|----------------------|
-| mean Δbelief | **0.0000** | +0.075 |
-| belief_recall (L1 days, Δ>0.02) | 0.000 | **0.818** (18/22) |
-| belief_unlabeled_raise_rate | 0.000 | **1.000** (8/8) |
-| mean Δ \| L1 | 0 | +0.067 |
-| mean Δ \| unlabeled | 0 | +0.097 |
-**Day-level pattern (locked):** L1 misses = late drought-span low contemporaneous
-risk (product also off). Unlabeled raises = post–Oct 22 flood-looking signal
-with no Karawang flood L1 — **catalog geometry**, not pure over-raise habit.
-Do not silently promote Oct 23–Nov 27 into L1 without official/satellite
-confirmation.
-#### Earlier n_zones=1 smoke (harness history only)
-First path validation used a 50k-step synthetic-only smoke checkpoint.
-Useful only for harness proofs. Do **not** quote smoke belief F1 as skill.
----
-## Held-out protocol
-**Status:** required before external quotation of product P/R/F1.
-Thresholds in `crop_risk_scorer.py` / `ProductGateConfig` stay frozen at the
-documented defaults (drought 0.35 / flood 0.25 WARNING bars).
-### Problem
-Dry (2023-07→11) and wet (2021-12→2022-03) windows were used both to
-**design** hazard thresholds and to **report** scorecards. Those numbers are
-in-sample relative to threshold selection.
-### Freeze rule
-1. Do not change product WARNING/CRITICAL bars from held-out alone.
-2. A held-out season/zone combination must not have been used for threshold
-   search or fungi reweighting experiments.
-3. The window must **exist in the evaluation cache**
-   (`historical_continuous_indonesia_v1.pkl`). Do not recommend years with
-   zero trajectories.
-### Cache coverage (Karawang / Indramayu)
-| Span | Regime |
-|------|--------|
-| 2015-05 → 2016-04 | el_nino_very_strong |
-| 2017-05 → 2018-04 | **neutral** |
-| 2018-06 → 2019-05 | el_nino_moderate (ends May 2019) |
-| 2020-09 → 2023-04 | La Niña series |
-| 2023-05 → 2024-04 | el_nino_strong (**design window**) |
-There is **no** full 2019-06→11 trajectory. That window is **invalid** for
-this cache.
-### Canonical held-out windows
-**A — Neutral (primary)**
-| Field | Value |
 |-------|--------|
-| Window | **2017-06-01 → 2017-11-30** |
-| Zones | `karawang_rice,indramayu_rice` |
-| Why | Outside design windows; present in cache; quiet/neutral ENSO |
-| L1 v1 | **No 2017 events** |
-**Metric rule when L1 is empty for the window:**
-- Do **not** report F1 / recall (undefined with zero event days).
-- Report **emit rate** = product_alerts / n_days.
-- Report **specificity** only with the explicit caveat
-  "no L1 row ⇒ treated as non-event."
-- Example (scorer_oracle, 2017-06→11, n=72): emit rate **~65%** — gate stays
-  hot in a neutral unlabeled season.
-**B — Moderate El Niño (secondary)**
-| Field | Value |
-|-------|--------|
-| Window | **2018-07-01 → 2018-11-30** |
-| Notes | In-cache moderate EN; still outside 2023 design dry window |
-```bash
-# Quiet-season emit-rate check (empty L1 — not F1)
-python evaluate_checkpoint_real.py \
-  --pkl historical_continuous_indonesia_v1.pkl \
-  --impact-labels impact_labels_java_v1.json \
-  --zones karawang_rice,indramayu_rice \
-  --start 2017-06-01 --end 2017-11-30 \
-  --mode scorer_oracle \
-  --out eval_heldout_2017_neutral.json
 ```
-Do **not** retune thresholds from held-out alone. If emit rate is
-unacceptable, document it; any new freeze needs design-window recall
-preserved.
-Agent held-out runs must use the same `n_zones` and budget (`max_steps` /
-`--budget-mode`) as the checkpoint under test.
-### What remains in-sample (documented)
-- 2023-07→11 dry product freeze — design window
-- 2021-12→2022-03 wet product — design window
-- 4-zone proxy scorecards on those same seasons
-Quote held-out **emit rate / specificity** for quiet seasons without labels.
-Quote product-vs-L1 P/R only on windows with curated L1 events.
----
-## Design notes (integrated behaviour)
-- **RH / fungi fix:** Obs + forecast fungi terms get a bounded additive
-  correction from `rh_anomaly_idx`. Synthetic data is a true no-op.
-- **Basin / helio:** `BasinContext` carries ENSO (RONI), IOD (DMI), and
-  SWPC solar wind / Kp / GOES X-ray / regime when feeds are up.
-  `ForecastConfig.require_real_basin_context` enables fail-closed mode.
-  Scorer does **not** consume helio; benefit path is RL observation → policy.
-- **Confidence damping:** Components scaled by `0.5 + 0.5 * confidence`
-  (~0.63 on many Open-Meteo archive days → scores in ~0.06–0.55). Hazard
-  path is the product bar, not max_risk ≥ 0.65.
-- **L1 vs proxy:** Proxy for ranking and threshold design. Product claims
-  prefer product-vs-L1 **only where L1 density is honest**. Days outside
-  every span are `unlabeled` and excluded from confusion-matrix P/R/F1.
-- **Real-eval injection:** `EpisodeContext` carries per-zone obs/forecast
-  lists for multi-zone packs. Checkpoint `n_zones` must match env shapes
-  (SHAPE_MISMATCH otherwise). Belief aggregates as **max** over active zones.
-- **Budget modes:** Default **triage** so training cannot pass without
-  learning to leave at least one zone unvisited. Full tour remains available
-  for curriculum phase 1 (`normal`) only.
-- **Zone-order shuffle:** Default on. Action index is not a stable identity.
-  Multi-zone product/belief metrics use order-invariant aggregations, so
-  shuffle does not corrupt joint P/R. Per-zone argmax logging (if added later)
-  must read `info["zone_ids"]` after shuffle, not assume CLI order.
-- **Synthetic calibration:** `clean_episode_ratio` (default 0.90) and
-  `event_spatial_correlation` (default 0.85) produce regional co-occurrence
-  of dirty zones. Train and synthetic eval must pass the same flags. Note:
-  these are the CLI/config *defaults*, not the configuration that produced
-  the triage-selection result in Still open #5 — that result used
-  `clean_episode_ratio=0.80`, `event_spatial_correlation=0.50`. Lower
-  `event_spatial_correlation` trades regional co-occurrence realism for a
-  higher rate of single-dirty episodes (the only episodes with a
-  well-defined correct zone for training signal); it was necessary but not
-  sufficient on its own — see the triage-selection recipe block under
-  Quickstart for what else was required.
-- **Feature scale imbalance:** `forecast_precip` (~[0, 80]) and
-  `zone_belief` (~[0, 0.3]) enter `GRUWeatherFeaturesExtractor` with no
-  normalization layer between them and the per-zone encoder. Left
-  unnormalized, precip's much larger raw magnitude can dominate the
-  smaller but more reliable belief signal during optimization, independent
-  of information content. `precip_scale` divides precip before the encoder
-  sees it (constructor kwarg and --precip-scale on traing_kaggle.py; default
-  `40.0`). Confirmed working, not confirmed optimal.
-- **Economics:** Default `miss_penalty` (200) is 10× `false_alert_penalty`
-  (20). In high base-rate drought seasons, "always" is hard to beat on pure
-  EV; report ranking separation, cost-loss under stated weights, and
-  product-vs-L1 with density caveats.
-- **Backtest data path:** Open-Meteo archive + baseline forecast. Missing
-  `cdsapi` only affects optional ERA5 pulls.
----
-## Still open (honest residual gaps)
-Only items that are **not** done. Order matches recommended work sequence:
-1. **L1 density / boundary confirmation (highest leverage for product claims)**
-   Unlabeled exclusion is implemented. Still needed: official or satellite
-   confirmation for Karawang **Jul onset** (pre Aug 1 drought scores) and
-   **post–Oct 22 flood-looking weeks** (no Karawang flood L1 in 2023). Do
-   not silently promote those days into L1.
-2. **Official impact catalogs** — L1 v1 is curated provincial/press
-   (**7 events**), not bulk BNPB / BMKG / EM-DAT / AUTP claims. Zones with
-   zero L1 coverage (e.g. Central Java in current catalog) should not be
-   blended into the same unlabeled bucket as boundary days without a distinct
-   `no_catalog` (or similar) tag when reporting.
-3. **Flood product precision** — sparse L1 + high wet emit rate; denser
-   flood labels and/or wet-season gate calibration.
-4. **Karawang scorer/climatology** — cross-pipeline FP pattern on proxy
-   backtests remains open as a possible zone-specific miscalibration after
-   label-boundary effects are accounted for.
-5. **Agent triage / allocation skill** — harness and full-tour belief path
-   validated; budget modes + shuffle + unified terminal reward are
-   integrated. Split by sub-claim:
-   **Resolved (synthetic, corrected metric, independently re-verified):**
-   A checkpoint at 2,400,000 training steps (`checkpoint_2400000.zip`,
-   `_total_timesteps=5,000,000` recorded in the checkpoint's own saved
-   state — the run's target was genuinely 5M, not a truncated run;
-   `policy_class=ZoneEquivariantMaskablePolicy`,
-   `features_extractor_kwargs.precip_scale=40.0`, both confirmed from the
-   checkpoint file itself, not just claimed) reaches **100% single-dirty
-   selection accuracy across all three hazard types** — drought, flood,
-   *and* fungi — replicated across two independent seed blocks at
-   `event_spatial_correlation=0.50` (n=116 and n=107) and confirmed again
-   at `event_spatial_correlation=0.85` (n=45), for n=268 combined
-   single-dirty episodes with zero errors. This was re-run end-to-end
-   against the real `weather_forecast_env.py` + `zone_observation.py` +
-   the real `crop_risk_scorer.py` (not a stub), using the corrected probe
-   (Success criteria #4) — not inferred from a pasted summary.
-   **Fungi needs more steps, not a different approach.** An earlier
-   checkpoint from the same longer run (`best_model.zip`, 250,000 steps)
-   shows drought/flood already at 100% but fungi lagging at 60–67% across
-   the same seed blocks — matching the ~57% fungi result seen on a
-   separate, shorter (150k-step) run. Fungi selection was never a capped
-   or fundamentally harder capability; it simply converges later than
-   drought/flood (roughly 250k insufficient, 2.4M sufficient). Do not
-   quote a single blended accuracy figure without noting this — but as of
-   the 2.4M checkpoint, the blended figure and the per-hazard figures are
-   the same number (100%), so this caveat is now about interpreting
-   *earlier* checkpoints correctly, not about the recommended artifact.
-   **Still open:**
-   - Real-trajectory evaluation: **no** checkpoint from this line of work
-     has been run through `evaluate_checkpoint_real.py` against real L1
-     labels. Everything above, including the 2.4M result, is synthetic
-     (`_episode_event_plan` / `zone_observation` synthetic configs).
-     Criterion 2 in Success criteria above remains unmet.
-   - Training seed: confirmed as `--seed 7` via the actual training
-     command, which controls the environment's episode generation
-     (`ForecastConfig(seed=args.seed)`). The checkpoint's own saved SB3
-     state shows `seed: None`, which is expected rather than
-     contradictory: `train_kaggle.py`'s `build_model()` never passes
-     `seed=` to `MaskablePPO(...)`, so PyTorch's weight-init RNG was never
-     pinned by `--seed`. Env-side determinism for this run is confirmed;
-     full bit-for-bit run reproducibility (including weight init) is not.
-     Multi-seed replication (does a *different* `--seed` value reach the
-     same 100%/100%/100% result on a similar step-count schedule) is
-     still open regardless.
-   - The full 5M-step run's final checkpoint (if one exists beyond
-     2,400,000) has not been checked — 2.4M already hits 100% on every
-     measured axis, so this is a lower-priority completeness check, not a
-     blocker.
-   - `target_kl` and the `zone_belief` prior/signal blend (currently
-     `0.7*prior + 0.3*signal`) were identified earlier as candidate levers
-     for improving past a lower ceiling — that ceiling turned out to be a
-     step-count issue, not a levers issue, for this recipe. Neither was
-     needed and neither has been tried; do not assume either is required.
-   Interpret overall return cautiously across the reward-path change
-   (unified budget-exhaustion terminal reward, see Design notes).
-6. **Held-out product / emit numbers** — protocol above; never-touched
-   window not yet scored under updated L1 + unlabeled rules for a triage
-   agent.
-7. **Dyna on real pairs** — pairs buildable (`build_dynamics_pairs.py`);
-   full DynamicsTrainer + surprise vs task-reward logging still needs a
-   curriculum run with dynamics on.
-8. **`hierarchical_search` Phase 2** — RL-driven adaptive recursion still a stub.
-9. **MNN / C++ edge** — not compiled against a real MNN build here.
-10. **Not a production SLA system** — no on-call, no guaranteed feed uptime,
-    no regulatory certification.
-**Closed since earlier README revisions (do not re-list as open):**
-- Multi-zone real injection (Blocker B) — per-zone lists + real eval path.
-- Belief null control at multi-zone — exact zero under zero_inspect after
-  initial/terminal aggregation fix (`believed_p` = max over active).
-- Unlabeled exclusion from confusion-matrix P/R/F1.
-- Matched synthetic train/eval via `clean_episode_ratio` /
-  `event_spatial_correlation` CLI.
-- Central Java product-gate carry confirmed by 3-zone vs 2-zone ablation.
-- 2-zone day-level belief pattern: L1 misses = late drought-span low risk;
-  unlabeled raises = post-span flood signal (catalog geometry).
-- Budget modes (`full` / `scarce` / `triage`) and default triage pressure.
-- Per-episode zone-order shuffle (anti–index-bias).
-- Unified terminal reward: budget exhaustion uses the same
-  `_compute_termination_reward` as early-terminate (belief-driven).
-- Budget-aware `unvisited_frac` (`min(n_zones, max_steps)`), so triage is
-  not half-penalised for zones outside the available budget.
-- Zone-equivariant policy architecture (`ZoneEquivariantMaskablePolicy`):
-  fixes a real bug where the previous pooled/invariant-only extractor made
-  the action head structurally unable to condition on which zone's content
-  looked risky (confirmed via a swap test: two independently trained
-  checkpoints produced bit-identical per-seed zone assignments, which is
-  only possible if physical zone choice depends on the env's own shuffle
-  and not on the policy reading content). Do **not** re-report "always
-  picks one action index regardless of shuffle" as a live bug — it was a
-  real bug, and it's fixed, though selection *accuracy* is a separate,
-  still-partially-open claim (see Still open #5).
-- Selection-accuracy probe methodology: the original "top share on dirty"
-  metric (marginal frequency of each physical zone_id, restricted to
-  episodes where *any* zone was dirty) cannot distinguish a policy with
-  real selection skill from one with none, under `shuffle_zone_order=True`
-  — it never compares the *chosen* zone's own dirty flag against ground
-  truth. Corrected probe: restrict to single-dirty episodes and check
-  whether the chosen zone matches its own dirty flag (see Success criteria
-  #4). Do not reintroduce the old metric shape in future evaluation code.
----
 ## License
-AGPL-3.0

 ---
+license: agpl-3.0
+library_name: stable-baselines3
+tags:
+  - deep-reinforcement-learning
+  - agricultural-ai
+  - weather-modelling
+  - curriculum-learning
+  - edge-ai
+pipeline_tag: reinforcement-learning
 ---
+# Agricultural Weather-Risk RL — MaskablePPO + Zone-Equivariant GRU
+Budget-constrained multi-zone weather-risk inspection for Indonesian rice
+zones. A Gymnasium environment tracks per-zone beliefs under a scarce
+inspection budget. A zone-equivariant GRU policy supports curriculum phases
+with changing `n_zones`. Deterministic crop-risk scoring grounds rewards
+and product alerts. An ONNX → MNN path is provided for edge inference.
+Research codebase, not a production alerting service.
+## What is included
+- `weather_forecast_env.py` — Gymnasium env (belief map, action masks, triage/scarce/full budget modes, zone-order shuffle, real EpisodeContext injection)
+- `gru_weather_policy.py` — Zone-equivariant GRU extractor (`ZoneEquivariantMaskablePolicy`; shared per-zone weights; precip input scaling, `precip_scale=40.0`)
+- `train_curriculum.py` / `train_kaggle.py` — Curriculum and standalone training (`--budget-mode triage|scarce|full`, `--precip-scale`)
+- `crop_risk_scorer.py` — Deterministic, economics-calibrated risk scoring
+- `climatology.py` + `indonesia_zones.py` — Per-zone climatology and Indonesian agricultural zones
+- `evaluate_checkpoint_real.py` — Real-trajectory eval (L1 labels, belief Δ, zero_inspect control)
+- `mnn_export.py` + `edge_wrapper.cpp` — ONNX/MNN export and C++ edge runtime with external GRU hidden state
+## Architecture note
+Inspect logits are scored **per zone before pooling** (`ZoneEquivariantMaskablePolicy`).
+The value path stays permutation-invariant so most weights transfer across
+`n_zones`. Full-tour episode length is not zone-selection skill; allocation
+under triage is evaluated with `max_steps < n_zones` and per-episode zone-order
+shuffle.
+## How to use
+### Train (triage, recommended recipe)
 ```bash
 python train_kaggle.py \
+  --out ./run_nz2_triage \
   --n-zones 2 \
   --budget-mode triage \
   --steps 2400000 \
   --device auto
 ```
+Requires `ZoneEquivariantMaskablePolicy` from `gru_weather_policy.py` (not
+the string `"MultiInputPolicy"`). `--precip-scale` is a real CLI flag
+(default `40.0`, matching the validated recipe). **Step count matters**:
+150k steps is enough for drought/flood selection but not fungi; fungi
+selection was confirmed solved by 2.4M steps on the same recipe — see
+Validation status.
+### Load
+```python
+from sb3_contrib import MaskablePPO
+model = MaskablePPO.load("run_nz2_triage/final_model.zip")
+action, _ = model.predict(obs, action_masks=masks, deterministic=True)
+```
+### Edge export
 ```bash
+python mnn_export.py \
+  --checkpoint run_nz2_triage/final_model.zip \
+  --output weather_rl_model.mnn \
+  --quantize int8 \
+  --n-zones 2
 ```
+## Validation status
+| Claim | Status |
 |-------|--------|
+| Weight transfer across `n_zones` changes | Supported by design |
+| Zone-order shuffle avoids fixed-slot lock-in | Observed, confirmed at 250k and 2.4M steps (raw per-slot action counts stay ~50/50) |
+| Risk-conditioned zone choice — drought (synthetic, single-dirty) | **Confirmed 100%** at 2.4M steps (n=76 combined, two seed blocks + ρ=0.85 transfer) |
+| Risk-conditioned zone choice — flood (synthetic, single-dirty) | **Confirmed 100%** at 2.4M steps (n=97 combined, two seed blocks + ρ=0.85 transfer) |
+| Risk-conditioned zone choice — fungi/disease pressure (synthetic, single-dirty) | **Confirmed 100%** at 2.4M steps (n=95 combined, two seed blocks + ρ=0.85 transfer) — lagged at ~57–67% through 250k on the same run; needed more steps, not a different mechanism |
+| Triage agent evaluated against real L1 trajectories | Open — all accuracy figures above are synthetic; `evaluate_checkpoint_real.py` has not yet been run on a triage-trained checkpoint |
+| Product skill vs curated L1 labels | Scorer/product path available; L1 density limits apply |
+| Multi-seed replication of the triage recipe | Open — the 2.4M result is one training run (`--seed 7`, confirmed via the training command; this pins the environment's episode generation but not PyTorch's weight-init RNG, which `train_kaggle.py` doesn't currently seed separately). A second seed reaching the same result has not been tried. |
+| Held-out real product / emit metrics | Open |
+## Limitations
+- Research stack only: no on-call, feed SLA, or regulatory certification.
+- Training is largely synthetic for the moment.
+- Full-tour belief or episode-length metrics are not zone-selection skill.
+- All zone-selection accuracy figures to date are from synthetic episodes;
+  the triage-trained policy has not been evaluated against real L1
+  trajectories (see Validation status).
+- The 2.4M-step selection-accuracy result comes from a single training run;
+  not yet replicated on a second seed.
+- MNN/C++ path is export-oriented, not a validated production edge deployment.
+## Dependencies
+```
+torch>=2.0
+gymnasium>=0.29
+stable-baselines3>=2.0
+sb3-contrib>=2.0
+numpy>=1.24
 ```
+While GPU is recommended for training, CPUs can be used successfully.
 ## License
+AGPL-3.0. See repository for full terms.