README.md

---
license: apache-2.0
base_model: amazon/chronos-2
tags:
  - time-series
  - forecasting
  - chronos
  - chronos-2
  - quantization
  - torchao
  - int8
pipeline_tag: time-series-forecasting
library_name: chronos-forecasting
---

# Smaller Chronos-2 (INT8)

[Chronos-2](https://huggingface.co/amazon/chronos-2) is Amazon's 120M time-series model: you give it history, it forecasts the next hours/days, no extra training.

This repo is **the same model with smaller files**. Weights are stored as 8-bit integers (INT8). It is **not** a fine-tune and **not** a new architecture.

| | Amazon original | this repo |
|--|--|--|
| download | 478 MB | **131 MB** |
| GPU memory while running (3090, one series) | ~0.56 GB | **~0.25 GB** |
| Python API | `Chronos2Pipeline.from_pretrained` | `load.py` (this repo) |
| license | Apache-2.0 | Apache-2.0 |

**Pick this** if you want Chronos-2 in Python, but a lighter download and less VRAM.

**Pick [amazon/chronos-2](https://huggingface.co/amazon/chronos-2)** if you want the one-liner load and do not care about 350 MB.

**Pick an ONNX / TensorRT Chronos-2** if you only care about production latency. Those are a different runtime, not this Python path.

---

## Install

```bash
pip install "chronos-forecasting>=2.0" torchao safetensors huggingface_hub pandas pyarrow
```

GPU: a recent PyTorch with CUDA. CPU works; it will be slower.

---

## Load (required)

Hugging Face's usual `from_pretrained` **cannot** read these INT8 files. Use `load.py` from this repo:

```python
from pathlib import Path
import importlib.util
from huggingface_hub import snapshot_download

repo = snapshot_download("oxfrug/chronos-2-int8-torchao")

spec = importlib.util.spec_from_file_location("c2int8", Path(repo) / "load.py")
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)

pipe = mod.load(repo, device="cuda")  # "cpu" if you have no GPU
```

If you already cloned the folder:

```python
from load import load
pipe = load("/path/to/chronos-2-int8-torchao", device="cuda")
```

You get a normal `Chronos2Pipeline`. After this, Amazon's docs apply.

---

## Forecast

One numpy series:

```python
import numpy as np

history = np.array([12.0, 12.4, 11.9, ...], dtype=np.float32)  # oldest → newest
quantiles, mean = pipe.predict_quantiles(
    inputs=[history],
    prediction_length=24,
    quantile_levels=[0.1, 0.5, 0.9],
)
# median forecast: quantiles[0][:, 1]  (shape: 1 × horizon × 3)
```

Many series / covariates — same as upstream, with `pipe.predict_df(...)`.

---

## Make it faster

INT8 here is **smaller, not faster**, on an RTX 3090. The model is small; most of the time is starting GPU kernels, not moving weights.

Two switches help **both** this INT8 and the original FP32 model:

1. **TF32** — tell PyTorch to use the GPU's fast float path (it is off by default).
2. **`torch.compile`** — fuse those kernels. First call is slow (compile); later calls drop a lot.

```python
import torch
from fast_infer import speedup   # file in this repo

torch.set_float32_matmul_precision("high")
pipe = speedup(pipe)  # TF32 + torch.compile
```

Or from a terminal, after downloading this folder:

```bash
python fast_infer.py                         # this INT8
python fast_infer.py --fp32 amazon/chronos-2 # original model, same knobs
```

Timed on this machine (RTX 3090, 512 past points, forecast 24, **one** series):

| | time per call | notes |
|--|--|--|
| original FP32 | ~7 ms | no extra knobs |
| this INT8 | ~9 ms | smaller, slightly slower |
| FP32 + compile + TF32 | **~3 ms** | best speed here |
| INT8 + compile + TF32 | ~3.6 ms | still a bit behind compiled FP32 |

Forecasting **many series in one call** (`predict_df` with several ids) is the other real win. Amazon's “hundreds of series per second” numbers are batched, not one sine wave.

Half-precision (FP16 / BF16) did **not** help this 120M model on a 3090.

---

## Did INT8 change the forecasts?

This pack only compresses weights.

What i did: take a few public series, hide the last 12–168 points, forecast them with the original model and with this INT8, compare.

- **German electricity (hourly)** — INT8 stays close. 24h median error vs original +1.8%; 168h actually −3%. Correlation of the two forecasts ≈ 0.99.
- **M4 hourly / daily / monthly** — same story on shape (high correlation), median error a bit worse (about +9% to +15% MASE on those short holds).
- **M4 weekly (13 steps)** — the two forecasts **diverged** (correlation near zero). Do not read that row as “INT8 is better.” Short weekly holds are noisy.

P10–P90 intervals: on 12–24 step holds, *both* models often cover ~50–60% of points instead of 80%. Compare INT8 to the original on *your* series if you use the bands, not to the textbook 80%.

Raw dumps: `eval/series.json`, `eval/coverage.json`, `eval/speed.json`.

---

## How the file was made

- Start from `amazon/chronos-2` (full float weights).
- torchao **weight-only INT8**: each big linear layer stores integers + a scale. Activations stay float.
- The small **quantile head** (the part that turns hidden states into P10/P50/P90) is left in float, on purpose.
- No calibration data, no extra training.

That is why `from_pretrained` fails: Hugging Face does not know this packing. `load.py` rebuilds the layers and fills them from `model.safetensors`.

---

## Limits

- Smaller files, less GPU RAM — **not** a speed-up by itself on a 3090.
- Not a domain fine-tune (no Nordic energy LoRA, etc.).
- Not GIFT-Eval / fev-bench.
- If you need intervals, check coverage on your data.

---

## Cite the base model

Ansari et al., *Chronos-2: From Univariate to Universal Forecasting*, 2025.  
https://arxiv.org/abs/2510.15821

Quant pack by [oxfrug](https://huggingface.co/oxfrug).
"""