t0-alpha ONNX FP16

This is an experimental first-party FP16 ONNX export of t0-alpha. It uses the same grouped, dynamic interface as the portable INT8 graph while retaining half-precision weights and compatible activations for accelerator-oriented local and edge deployments.

Model family: t0-alpha (PyTorch/MLX) · ONNX FP16 · ONNX INT8 · Collection

Intended use

This pre-release derivative is intended for testing with FP16-capable ONNX Runtime execution providers. Provider coverage and accuracy should be validated on the target hardware before deployment. For production or consequential use, we recommend t0-alpha. Provided as-is.

The graph runs with ONNX Runtime 1.29.0's CPU execution provider, although the CPU path is not expected to benefit from FP16. CoreML accepted the graph in a local smoke test but offloaded only part of the dynamic graph and used CPU fallback. CUDA, TensorRT, DirectML, and WebGPU have not yet been validated. The INT8 export remains the recommended browser/WASM artifact.

Artifact

Artifact	Size	Purpose
`t0-alpha-grouped-fp16.onnx`	207.6 MB	Experimental FP16 graph

The graph is 203.3 MB smaller than its FP32 source and 100.3 MB larger than the grouped INT8 graph.

Graph contract

Input/output	Type	Shape	Notes
`target_context`	`float32`	`[target_rows, context]`	Use `NaN` for missing observations
`target_group_ids`	`int32`	`[target_rows]`	Rows with the same id attend jointly
`future_covariate_context`	`float32`	`[covariate_rows, context]`	Historical portion of known-future covariates
`future_covariate_future`	`float32`	`[covariate_rows, compute_horizon]`	Values known over the forecast horizon
`future_covariate_group_ids`	`int32`	`[covariate_rows]`	Associates each covariate with a target group
`quantiles`	`float32`	`[target_rows, compute_horizon, 5]`	Levels `0.1, 0.25, 0.5, 0.75, 0.9`

The graph uses ONNX opset 20. Its public inputs and output remain float32 so callers can switch between the FP16 and INT8 artifacts without changing their data preparation.

The compute horizon is determined by the width of future_covariate_future, which must be a multiple of 32. To request 50 steps, pass a width of 64 and keep the first 50 outputs. Context length is independently flexible and is left-padded to a patch boundary internally. The graph does not include autoregressive rollout beyond 1024 steps.

Group ids need not be contiguous. Give target and covariate rows the same id when they belong to the same multivariate series. If no known-future covariates are available, pass covariate arrays with zero rows; the horizon dimension is still retained.

import math

import numpy as np
import onnxruntime as ort

target_context = np.asarray([1.0, 1.3, 1.2, 1.7, 2.1], dtype=np.float32)[None, :]
target_group_ids = np.asarray([0], dtype=np.int32)
horizon = 24
compute_horizon = math.ceil(horizon / 32) * 32
future_covariate_context = np.empty((0, target_context.shape[1]), dtype=np.float32)
future_covariate_future = np.empty((0, compute_horizon), dtype=np.float32)
future_covariate_group_ids = np.empty((0,), dtype=np.int32)

session = ort.InferenceSession(
    "t0-alpha-grouped-fp16.onnx",
    providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
)
quantiles = session.run(
    None,
    {
        "target_context": target_context,
        "target_group_ids": target_group_ids,
        "future_covariate_context": future_covariate_context,
        "future_covariate_future": future_covariate_future,
        "future_covariate_group_ids": future_covariate_group_ids,
    },
)[0]
forecast = quantiles[:, :horizon, :]

Replace CUDAExecutionProvider with the provider available on the target device, and inspect session.get_providers() to confirm any fallback.

Validation status

The FP32 source graph matched model.predict() within 0.002% of forecast spread over context lengths 1–4096 and compute horizons 32–1024. The FP16 graph passed ONNX validation and returned finite, ordered quantiles with ONNX Runtime 1.29.0.

At context lengths 96 and 97, mean FP16 drift was at most 0.054% of forecast spread across compute horizons 32–1024. The current candidate is not yet recommended for very long dense contexts: synthetic 4096-point trends reached roughly 49.4% mean drift over spread. This repository should remain a preview until that sensitivity and the intended hardware providers have been validated or corrected.