Reinforcement Learning
stable-baselines3
deep-reinforcement-learning
agricultural-ai
weather-modelling
curriculum-learning
edge-ai
Instructions to use DHDRL/monsoon-rl with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- stable-baselines3
How to use DHDRL/monsoon-rl with stable-baselines3:
from huggingface_sb3 import load_from_hub checkpoint = load_from_hub( repo_id="DHDRL/monsoon-rl", filename="{MODEL FILENAME}.zip", ) - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,71 @@
|
|
| 1 |
---
|
| 2 |
license: agpl-3.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: agpl-3.0
|
| 3 |
+
tags:
|
| 4 |
+
- reinforcement-learning
|
| 5 |
+
- stable-baselines3
|
| 6 |
+
- deep-reinforcement-learning
|
| 7 |
+
- agricultural-ai
|
| 8 |
+
- weather-modelling
|
| 9 |
+
- curriculum-learning
|
| 10 |
+
- edge-ai
|
| 11 |
---
|
| 12 |
+
|
| 13 |
+
# Agricultural Weather-Risk RL β Permutation-Invariant GRU Policy with Curriculum Weight Transfer
|
| 14 |
+
|
| 15 |
+
Core RL components and training infrastructure for a budget-constrained, multi-zone weather-risk inspection task. The system includes a Gymnasium environment, a **permutation-invariant GRU features extractor** that enables weight transfer across curriculum phases with varying numbers of zones, full multi-phase curriculum training, and an export path to quantized MNN for edge deployment.
|
| 16 |
+
|
| 17 |
+
## What is included
|
| 18 |
+
|
| 19 |
+
- `weather_forecast_env.py` β Gymnasium environment with per-zone belief tracking, action masking, soft reset, and NaN-safe wrapper.
|
| 20 |
+
- `gru_weather_policy.py` β Permutation-invariant GRU features extractor (shared per-zone processing + attention-weighted aggregation + max pooling). Makes the majority of policy parameters independent of `n_zones`.
|
| 21 |
+
- `train_curriculum.py` β 5-phase curriculum trainer (`normal` β `monsoon` β `drought` β `heatwave` β `humidity`) with automatic weight transfer between phases.
|
| 22 |
+
- `crop_risk_scorer.py` β Deterministic, economics-calibrated risk scoring used for both training rewards and evaluation.
|
| 23 |
+
- `climatology.py` + `indonesia_zones.py` β Real per-zone climatology and 14 grounded Indonesian agricultural zones with rice crop calendars.
|
| 24 |
+
- `backtest_indonesia.py` β Historical replay harness with precision/recall and lead-time metrics.
|
| 25 |
+
- `mnn_export.py` + `edge_wrapper.cpp` β ONNX export and C++/Vulkan edge runtime with external GRU hidden-state management.
|
| 26 |
+
- `era5_data_pipeline.py` β Multi-source data pipeline (ERA5 / Open-Meteo / synthetic) with graceful degradation.
|
| 27 |
+
|
| 28 |
+
## Key Technical Achievement
|
| 29 |
+
|
| 30 |
+
The central engineering result is a **permutation-invariant, size-independent policy architecture** that solves weight transfer across curriculum phases with changing numbers of zones.
|
| 31 |
+
|
| 32 |
+
Previous designs tied the GRU input dimension (and all downstream weights) to a fixed `n_zones`, causing observation space mismatches at phase transitions. The new extractor uses:
|
| 33 |
+
- Shared per-zone GRU processing
|
| 34 |
+
- Learned attention + max pooling across the variable zone axis
|
| 35 |
+
- Separate basin-scale context (ENSO, IOD, etc.)
|
| 36 |
+
|
| 37 |
+
This design transfers **61 of 63 parameter tensors** cleanly when `n_zones` changes (only the final action head is reinitialized). Full 5-phase curriculum runs confirm that phases sharing the same `n_zones` inherit competence immediately, while phases after an entity-count change only need to relearn the action head.
|
| 38 |
+
|
| 39 |
+
## Validation Results
|
| 40 |
+
|
| 41 |
+
Full 5-phase curriculum (690k steps) completed successfully on GPU:
|
| 42 |
+
|
| 43 |
+
| Transition | n_zones change | Tensors transferred | Behavior |
|
| 44 |
+
|-------------------------|----------------|---------------------|------------------------------|
|
| 45 |
+
| normal β monsoon | 2 β 3 | 61/63 | Action head reinitialized |
|
| 46 |
+
| monsoon β drought | 3 β 3 | 63/63 | Full transfer |
|
| 47 |
+
| drought β heatwave | 3 β 4 | 61/63 | Action head reinitialized |
|
| 48 |
+
| heatwave β humidity | 4 β 4 | 63/63 | Full transfer |
|
| 49 |
+
|
| 50 |
+
All phases converged to their structural performance ceiling (`n_zones + 1`). Full-transfer phases started at ceiling performance immediately. Explained variance stayed in the healthy 0.6β0.8 range. The entire curriculum finished in ~50 minutes on one T4.
|
| 51 |
+
|
| 52 |
+
## How to Use
|
| 53 |
+
|
| 54 |
+
### Training
|
| 55 |
+
```bash
|
| 56 |
+
python train_curriculum.py --phase all --output-dir ./runs --device cuda --seed 42
|
| 57 |
+
Inference (after training)
|
| 58 |
+
Pythonfrom sb3_contrib import MaskablePPO
|
| 59 |
+
model = MaskablePPO.load("runs/humidity/models/final_humidity.zip")
|
| 60 |
+
Edge Export
|
| 61 |
+
Bashpython mnn_export.py --checkpoint runs/humidity/models/final_humidity.zip \
|
| 62 |
+
--output weather_rl_model.mnn --quantize int8 --n-zones 4
|
| 63 |
+
Dependencies
|
| 64 |
+
|
| 65 |
+
torch >= 2.0
|
| 66 |
+
stable-baselines3 >= 2.0
|
| 67 |
+
sb3-contrib >= 2.0
|
| 68 |
+
gymnasium >= 0.29
|
| 69 |
+
numpy
|
| 70 |
+
|
| 71 |
+
GPU recommended for training. The edge runtime (edge_wrapper.cpp) targets Vulkan with CPU fallback.
|