DHDRL commited on
Commit
b8f29b6
Β·
verified Β·
1 Parent(s): be00c30

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +68 -0
README.md CHANGED
@@ -1,3 +1,71 @@
1
  ---
2
  license: agpl-3.0
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: agpl-3.0
3
+ tags:
4
+ - reinforcement-learning
5
+ - stable-baselines3
6
+ - deep-reinforcement-learning
7
+ - agricultural-ai
8
+ - weather-modelling
9
+ - curriculum-learning
10
+ - edge-ai
11
  ---
12
+
13
+ # Agricultural Weather-Risk RL β€” Permutation-Invariant GRU Policy with Curriculum Weight Transfer
14
+
15
+ Core RL components and training infrastructure for a budget-constrained, multi-zone weather-risk inspection task. The system includes a Gymnasium environment, a **permutation-invariant GRU features extractor** that enables weight transfer across curriculum phases with varying numbers of zones, full multi-phase curriculum training, and an export path to quantized MNN for edge deployment.
16
+
17
+ ## What is included
18
+
19
+ - `weather_forecast_env.py` β€” Gymnasium environment with per-zone belief tracking, action masking, soft reset, and NaN-safe wrapper.
20
+ - `gru_weather_policy.py` β€” Permutation-invariant GRU features extractor (shared per-zone processing + attention-weighted aggregation + max pooling). Makes the majority of policy parameters independent of `n_zones`.
21
+ - `train_curriculum.py` β€” 5-phase curriculum trainer (`normal` β†’ `monsoon` β†’ `drought` β†’ `heatwave` β†’ `humidity`) with automatic weight transfer between phases.
22
+ - `crop_risk_scorer.py` β€” Deterministic, economics-calibrated risk scoring used for both training rewards and evaluation.
23
+ - `climatology.py` + `indonesia_zones.py` β€” Real per-zone climatology and 14 grounded Indonesian agricultural zones with rice crop calendars.
24
+ - `backtest_indonesia.py` β€” Historical replay harness with precision/recall and lead-time metrics.
25
+ - `mnn_export.py` + `edge_wrapper.cpp` β€” ONNX export and C++/Vulkan edge runtime with external GRU hidden-state management.
26
+ - `era5_data_pipeline.py` β€” Multi-source data pipeline (ERA5 / Open-Meteo / synthetic) with graceful degradation.
27
+
28
+ ## Key Technical Achievement
29
+
30
+ The central engineering result is a **permutation-invariant, size-independent policy architecture** that solves weight transfer across curriculum phases with changing numbers of zones.
31
+
32
+ Previous designs tied the GRU input dimension (and all downstream weights) to a fixed `n_zones`, causing observation space mismatches at phase transitions. The new extractor uses:
33
+ - Shared per-zone GRU processing
34
+ - Learned attention + max pooling across the variable zone axis
35
+ - Separate basin-scale context (ENSO, IOD, etc.)
36
+
37
+ This design transfers **61 of 63 parameter tensors** cleanly when `n_zones` changes (only the final action head is reinitialized). Full 5-phase curriculum runs confirm that phases sharing the same `n_zones` inherit competence immediately, while phases after an entity-count change only need to relearn the action head.
38
+
39
+ ## Validation Results
40
+
41
+ Full 5-phase curriculum (690k steps) completed successfully on GPU:
42
+
43
+ | Transition | n_zones change | Tensors transferred | Behavior |
44
+ |-------------------------|----------------|---------------------|------------------------------|
45
+ | normal β†’ monsoon | 2 β†’ 3 | 61/63 | Action head reinitialized |
46
+ | monsoon β†’ drought | 3 β†’ 3 | 63/63 | Full transfer |
47
+ | drought β†’ heatwave | 3 β†’ 4 | 61/63 | Action head reinitialized |
48
+ | heatwave β†’ humidity | 4 β†’ 4 | 63/63 | Full transfer |
49
+
50
+ All phases converged to their structural performance ceiling (`n_zones + 1`). Full-transfer phases started at ceiling performance immediately. Explained variance stayed in the healthy 0.6–0.8 range. The entire curriculum finished in ~50 minutes on one T4.
51
+
52
+ ## How to Use
53
+
54
+ ### Training
55
+ ```bash
56
+ python train_curriculum.py --phase all --output-dir ./runs --device cuda --seed 42
57
+ Inference (after training)
58
+ Pythonfrom sb3_contrib import MaskablePPO
59
+ model = MaskablePPO.load("runs/humidity/models/final_humidity.zip")
60
+ Edge Export
61
+ Bashpython mnn_export.py --checkpoint runs/humidity/models/final_humidity.zip \
62
+ --output weather_rl_model.mnn --quantize int8 --n-zones 4
63
+ Dependencies
64
+
65
+ torch >= 2.0
66
+ stable-baselines3 >= 2.0
67
+ sb3-contrib >= 2.0
68
+ gymnasium >= 0.29
69
+ numpy
70
+
71
+ GPU recommended for training. The edge runtime (edge_wrapper.cpp) targets Vulkan with CPU fallback.