Reinforcement Learning
stable-baselines3
deep-reinforcement-learning
agricultural-ai
weather-modelling
curriculum-learning
edge-ai
Instructions to use DHDRL/monsoon-rl with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- stable-baselines3
How to use DHDRL/monsoon-rl with stable-baselines3:
from huggingface_sb3 import load_from_hub checkpoint = load_from_hub( repo_id="DHDRL/monsoon-rl", filename="{MODEL FILENAME}.zip", ) - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -18,13 +18,13 @@ inspection budget. A zone-equivariant GRU policy supports curriculum phases
|
|
| 18 |
with changing `n_zones`. Deterministic crop-risk scoring grounds rewards
|
| 19 |
and product alerts. An ONNX β MNN path is provided for edge inference.
|
| 20 |
|
| 21 |
-
|
| 22 |
|
| 23 |
## What is included
|
| 24 |
|
| 25 |
- `weather_forecast_env.py` β Gymnasium env (belief map, action masks, triage/scarce/full budget modes, zone-order shuffle, real EpisodeContext injection)
|
| 26 |
-
- `gru_weather_policy.py` β Zone-equivariant GRU extractor (`ZoneEquivariantMaskablePolicy`; shared per-zone weights; precip input scaling)
|
| 27 |
-
- `train_curriculum.py` / `train_kaggle.py` β Curriculum and standalone training (`--budget-mode triage|scarce|full`)
|
| 28 |
- `crop_risk_scorer.py` β Deterministic, economics-calibrated risk scoring
|
| 29 |
- `climatology.py` + `indonesia_zones.py` β Per-zone climatology and Indonesian agricultural zones
|
| 30 |
- `evaluate_checkpoint_real.py` β Real-trajectory eval (L1 labels, belief Ξ, zero_inspect control)
|
|
@@ -47,23 +47,25 @@ python train_kaggle.py \
|
|
| 47 |
--out ./run_nz2_triage \
|
| 48 |
--n-zones 2 \
|
| 49 |
--budget-mode triage \
|
| 50 |
-
--steps
|
| 51 |
--clean-episode-ratio 0.80 \
|
| 52 |
--event-spatial-correlation 0.50 \
|
|
|
|
| 53 |
--device auto
|
| 54 |
```
|
| 55 |
|
| 56 |
-
Requires `ZoneEquivariantMaskablePolicy` from `gru_weather_policy.py`
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
|
| 63 |
### Load
|
| 64 |
|
| 65 |
```python
|
| 66 |
from sb3_contrib import MaskablePPO
|
|
|
|
| 67 |
model = MaskablePPO.load("run_nz2_triage/final_model.zip")
|
| 68 |
action, _ = model.predict(obs, action_masks=masks, deterministic=True)
|
| 69 |
```
|
|
@@ -83,12 +85,13 @@ python mnn_export.py \
|
|
| 83 |
| Claim | Status |
|
| 84 |
|-------|--------|
|
| 85 |
| Weight transfer across `n_zones` changes | Supported by design |
|
| 86 |
-
| Zone-order shuffle avoids fixed-slot lock-in | Observed |
|
| 87 |
-
| Risk-conditioned zone choice
|
| 88 |
-
| Risk-conditioned zone choice
|
|
|
|
| 89 |
| Triage agent evaluated against real L1 trajectories | Open β all accuracy figures above are synthetic; `evaluate_checkpoint_real.py` has not yet been run on a triage-trained checkpoint |
|
| 90 |
| Product skill vs curated L1 labels | Scorer/product path available; L1 density limits apply |
|
| 91 |
-
| Multi-seed replication of the triage recipe | Open |
|
| 92 |
| Held-out real product / emit metrics | Open |
|
| 93 |
|
| 94 |
## Limitations
|
|
@@ -96,13 +99,12 @@ python mnn_export.py \
|
|
| 96 |
- Research stack only: no on-call, feed SLA, or regulatory certification.
|
| 97 |
- Training is largely synthetic for the moment.
|
| 98 |
- Full-tour belief or episode-length metrics are not zone-selection skill.
|
| 99 |
-
- Risk-conditioned zone selection is hazard-type-dependent: strong for
|
| 100 |
-
precipitation-driven events (drought, flood), not yet demonstrated for
|
| 101 |
-
fungi/disease-pressure risk.
|
| 102 |
- All zone-selection accuracy figures to date are from synthetic episodes;
|
| 103 |
the triage-trained policy has not been evaluated against real L1
|
| 104 |
trajectories (see Validation status).
|
| 105 |
-
-
|
|
|
|
|
|
|
| 106 |
|
| 107 |
## Dependencies
|
| 108 |
|
|
|
|
| 18 |
with changing `n_zones`. Deterministic crop-risk scoring grounds rewards
|
| 19 |
and product alerts. An ONNX β MNN path is provided for edge inference.
|
| 20 |
|
| 21 |
+
Research codebase, not a production alerting service.
|
| 22 |
|
| 23 |
## What is included
|
| 24 |
|
| 25 |
- `weather_forecast_env.py` β Gymnasium env (belief map, action masks, triage/scarce/full budget modes, zone-order shuffle, real EpisodeContext injection)
|
| 26 |
+
- `gru_weather_policy.py` β Zone-equivariant GRU extractor (`ZoneEquivariantMaskablePolicy`; shared per-zone weights; precip input scaling, `precip_scale=40.0`)
|
| 27 |
+
- `train_curriculum.py` / `train_kaggle.py` β Curriculum and standalone training (`--budget-mode triage|scarce|full`, `--precip-scale`)
|
| 28 |
- `crop_risk_scorer.py` β Deterministic, economics-calibrated risk scoring
|
| 29 |
- `climatology.py` + `indonesia_zones.py` β Per-zone climatology and Indonesian agricultural zones
|
| 30 |
- `evaluate_checkpoint_real.py` β Real-trajectory eval (L1 labels, belief Ξ, zero_inspect control)
|
|
|
|
| 47 |
--out ./run_nz2_triage \
|
| 48 |
--n-zones 2 \
|
| 49 |
--budget-mode triage \
|
| 50 |
+
--steps 2400000 \
|
| 51 |
--clean-episode-ratio 0.80 \
|
| 52 |
--event-spatial-correlation 0.50 \
|
| 53 |
+
--precip-scale 40.0 \
|
| 54 |
--device auto
|
| 55 |
```
|
| 56 |
|
| 57 |
+
Requires `ZoneEquivariantMaskablePolicy` from `gru_weather_policy.py` (not
|
| 58 |
+
the string `"MultiInputPolicy"`). `--precip-scale` is a real CLI flag
|
| 59 |
+
(default `40.0`, matching the validated recipe). **Step count matters**:
|
| 60 |
+
150k steps is enough for drought/flood selection but not fungi; fungi
|
| 61 |
+
selection was confirmed solved by 2.4M steps on the same recipe β see
|
| 62 |
+
Validation status.
|
| 63 |
|
| 64 |
### Load
|
| 65 |
|
| 66 |
```python
|
| 67 |
from sb3_contrib import MaskablePPO
|
| 68 |
+
|
| 69 |
model = MaskablePPO.load("run_nz2_triage/final_model.zip")
|
| 70 |
action, _ = model.predict(obs, action_masks=masks, deterministic=True)
|
| 71 |
```
|
|
|
|
| 85 |
| Claim | Status |
|
| 86 |
|-------|--------|
|
| 87 |
| Weight transfer across `n_zones` changes | Supported by design |
|
| 88 |
+
| Zone-order shuffle avoids fixed-slot lock-in | Observed, confirmed at 250k and 2.4M steps (raw per-slot action counts stay ~50/50) |
|
| 89 |
+
| Risk-conditioned zone choice β drought (synthetic, single-dirty) | **Confirmed 100%** at 2.4M steps (n=76 combined, two seed blocks + Ο=0.85 transfer) |
|
| 90 |
+
| Risk-conditioned zone choice β flood (synthetic, single-dirty) | **Confirmed 100%** at 2.4M steps (n=97 combined, two seed blocks + Ο=0.85 transfer) |
|
| 91 |
+
| Risk-conditioned zone choice β fungi/disease pressure (synthetic, single-dirty) | **Confirmed 100%** at 2.4M steps (n=95 combined, two seed blocks + Ο=0.85 transfer) β lagged at ~57β67% through 250k on the same run; needed more steps, not a different mechanism |
|
| 92 |
| Triage agent evaluated against real L1 trajectories | Open β all accuracy figures above are synthetic; `evaluate_checkpoint_real.py` has not yet been run on a triage-trained checkpoint |
|
| 93 |
| Product skill vs curated L1 labels | Scorer/product path available; L1 density limits apply |
|
| 94 |
+
| Multi-seed replication of the triage recipe | Open β the 2.4M result is one training run (`--seed 7`, confirmed via the training command; this pins the environment's episode generation but not PyTorch's weight-init RNG, which `train_kaggle.py` doesn't currently seed separately). A second seed reaching the same result has not been tried. |
|
| 95 |
| Held-out real product / emit metrics | Open |
|
| 96 |
|
| 97 |
## Limitations
|
|
|
|
| 99 |
- Research stack only: no on-call, feed SLA, or regulatory certification.
|
| 100 |
- Training is largely synthetic for the moment.
|
| 101 |
- Full-tour belief or episode-length metrics are not zone-selection skill.
|
|
|
|
|
|
|
|
|
|
| 102 |
- All zone-selection accuracy figures to date are from synthetic episodes;
|
| 103 |
the triage-trained policy has not been evaluated against real L1
|
| 104 |
trajectories (see Validation status).
|
| 105 |
+
- The 2.4M-step selection-accuracy result comes from a single training run;
|
| 106 |
+
not yet replicated on a second seed.
|
| 107 |
+
- MNN/C++ path is export-oriented, not a validated production edge deployment.
|
| 108 |
|
| 109 |
## Dependencies
|
| 110 |
|