---
library_name: pytorch
tags:
- decision-transformer
- energy-trading
- battery-storage
- aemo
- nem
- offline-rl
datasets:
- mrvictoru/AEMO_simulated_trade
new_version: mrvictoru/energydecision-dt-v2-sdp
pipeline_tag: reinforcement-learning
---
# EnergyDecision-DT-V2

> **⚠️ OUTDATED — SUPERSeded by the Stage C standalone DT.**
>
> This model (modern v2 pretrained, 8×768 GQA) was the SOTA as of Jul 2026,
> beating PPO on dispatch-matched ($10,138/ep) and standard ($4,630/ep) surfaces.
> It has since been superseded by the **Stage C standalone DT** distilled from
> an honest SDP-planning teacher, which beats PPO on **all 4 identity surfaces**
> and passes the market-impact gate:
>
> | Surface | Stage C DT (`rtg_mode="auto"`) | This model (v2 pretrained) | PPO |
> |---|---|---|---|
> | Standard Oct | **$11,573** | $4,991 | $2,353 |
> | Dispatch-matched | **$35,320** | $10,138 | $22,530 |
> | Expanded broad-2024 | **$34,761** | $4,596 | $19,504 |
> | 2025 OOD | **$25,862** | −$694 | $14,320 |
> | Impact gate | **2.6–3.0× PPO** | 62–83% identity | — |
>
> The shipped model is `models/aemo/dt/aemo_dt_sdp_jtsoc_fullcorpus.pt`
> (see [energydecision repo](https://github.com/mrvictoru/energydecision)).
> This checkpoint is retained for reproducing the Jul 2026 benchmark study.

---

## Model Description

**EnergyDecision-DT-V2** is a **Decision Transformer** model trained on simulated battery dispatch data from the Australian Energy Market Operator (AEMO) Frequency Control Ancillary Services (FCAS) market. It models optimal battery dispatch as a sequence prediction problem, conditioning on returns-to-go, observed states, and past actions to predict the next action.

The model learns to **dispatch battery energy storage** (charge/discharge) and **bid into 8 FCAS contingency markets** simultaneously, using a modern transformer architecture with Grouped-Query Attention, QK-Norm, SwiGLU activations, and weight-tied embeddings.

### Key Features

-   **Modern architecture**: Grouped-Query Attention (6 KV heads, 12 Q heads), QK-Norm for training stability, SwiGLU FFN, RMSNorm pre-norm
-   **Weight tying**: Embedding and prediction layers share weights for parameter efficiency
-   **Action Space (9-dim)**:
    -   Dim 0: Energy dispatch in [-1, 1] (negative = charge, positive = discharge)
    -   Dims 1-8: FCAS contingency bids in [0, 1]
-   **State Space (18-dim)**: Normalized market observations including prices, demand, renewables penetration, and battery state-of-charge
-   **Context Length**: 210 timesteps (looks back ~17.5 hours of 5-minute dispatch intervals)

## Intended Use

This model is intended for:

-   **Research** into offline RL for energy markets
-   **Simulation** of battery trading strategies in the AEMO FCAS market
-   **Baseline** for comparing decision transformer approaches against traditional RL

It is **not intended for live trading** without further validation, risk management, and regulatory compliance.

## Training Data

-   **Source**: [AEMO simulated trade dataset](https://huggingface.co/datasets/mrvictoru/AEMO_simulated_trade)
-   **Size**: 86,412,124 rows after filtering
-   **Episodes**: 2,401 episodes (after filtering for minimum context length)
-   **Source policies**: A2C (76.9M rows) + GRPO-DT (11.9M rows)

## Model Architecture

```
DecisionTransformer(
  (embed_return):  Linear(1 -> 768)
  (embed_state):   Linear(18 -> 768)
  (embed_action):  Linear(9 -> 768)
  (embed_timestep): Embedding(100000 -> 768)
  (embed_ln):      RMSNorm(768)
  (blocks): 8x ModernBlock(
    (norm1): RMSNorm(768)
    (attn): CausalSelfAttention(
      q_proj: Linear(768 -> 768)   # 12 Q heads x 64 head_dim
      k_proj: Linear(768 -> 384)   # 6 KV heads x 64 head_dim
      v_proj: Linear(768 -> 384)   # 6 KV heads x 64 head_dim
      out_proj: Linear(768 -> 768)
      qk_norm: RMSNorm(64) per head
      n_rep: 2 (each KV head serves 2 Q heads -- GQA)
    )
    (norm2): RMSNorm(768)
    (ffn): SwiGLU(768 -> 3072 -> 768, dropout=0.15)
  )
  (ln_f): RMSNorm(768)
  (pred_act):   Linear(768 -> 9)  -> Tanh   [tied with embed_act weights]
  (pred_state): Linear(768 -> 18)         [tied with embed_state weights]
  (pred_return): Linear(768 -> 1)         [tied with embed_return weights]
)
```

### Hyperparameters

| Parameter | Value |
|-----------|-------|
| Blocks | 8 |
| Hidden dim | 768 |
| Attention heads (Q) | 12 |
| KV heads (GQA) | 6 |
| Context length | 210 |
| Dropout | 0.15 |
| QK-Norm | Enabled |
| Weight tying | Enabled |
| State dim | 18 |
| Action dim | 9 |
| Discount factor | 0.95 |
| Return scale | 2.0 |

## Training Procedure

-   **Hardware**: NVIDIA GPU (AMP mixed precision)
-   **Optimizer**: AdamW (lr=3e-5, weight_decay=1e-4)
-   **Batch size**: 128
-   **Epochs**: 3
-   **Gradient clipping**: 1.0

### Training Metrics

| Epoch | Train Loss | Val Loss | Action Loss |
|-------|-----------|----------|-------------|
| 1 | 0.057152 | 0.019429 | 0.056399 |
| 2 | 0.015032 | 0.009449 | 0.014647 |
| 3 | **0.008868** | **0.007034** | **0.008644** |

## Benchmark Results (historical — Jul 2026)

Evaluated on dispatch-matched (Dalrymple North 8 MWh / 30 MW, Q4 2024 SA1) and standard (5 regions, medium batteries) surfaces:

| Model | Standard | DM (rtg=0.5) | DM (rtg=0.0) |
|---|---|---|---|
| **Modern v2 pretrained (this model)** | **$4,630** | $6,793 | **$10,138** |
| PPO reference | $2,353 | $7,757 | — |
| Dispatch Dalrymple North | $4,660 | $3,663 | — |

## RTG Calibration (historical)

| RTG | Profit/ep | FCAS/ep |
|-----|----------|---------|
| **0.0** | **$10,138** | $10,068 |
| 0.5 | $6,793 | $6,703 |
| 1.0 | $6,877 | $6,101 |
| 2.0 | $6,329 | $6,092 |

## Usage

```python
import torch
from huggingface_hub import hf_hub_download
from decision_transformer import DecisionTransformer

model_kwargs = {
    "state_dim": 18, "act_dim": 9, "n_block": 8,
    "h_dim": 768, "n_heads": 12, "n_kv_heads": 6,
    "context_len": 210, "drop_p": 0.15,
    "max_timestep": 100000, "qk_norm": True,
    "rope_enabled": False, "tie_weights": True,
}

model_path = hf_hub_download("mrvictoru/energydecision-dt-v2", "aemo_dt_fcas_model.pt")
model = DecisionTransformer(**model_kwargs)
model.load_from_checkpoint(model_path)
model.eval()
```

## Citation

```bibtex
@misc{energydecision-dt-v2,
  author = {Victor U},
  title = {EnergyDecision-DT-V2: Decision Transformer for AEMO FCAS Battery Trading},
  year = {2026},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/mrvictoru/energydecision-dt-v2}},
}
```