| --- |
| license: apache-2.0 |
| tags: [onnx, onnxruntime, onnx-genai, inference-metadata, sliding-window-attention] |
| --- |
| # onnx-genai-example-mistral-7b-v0-1-sliding-window |
|
|
| Public real-weight fp16 ONNX package from [`mistralai/Mistral-7B-v0.1`](https://huggingface.co/mistralai/Mistral-7B-v0.1/tree/27d67f1b5f57dc0953326b2601d68371d40ea8da) at immutable revision `27d67f1b5f57dc0953326b2601d68371d40ea8da`. Source license: Apache-2.0. |
|
|
| The graph contains 32 CUDA `com.microsoft::GroupQueryAttention` nodes with `local_window_size=4096`. A real H200 CUDA probe generated 12 finite-logit tokens from positions 4092–4103, crossing the configured boundary. The package includes canonical metadata, tokenizer, policies, provenance, request/output, graph report, and exact timings. |
|
|
| ## Download |
| ```bash |
| hf download justinchuby/onnx-genai-example-mistral-7b-v0-1-sliding-window --repo-type model --local-dir ./mistral-7b-v0.1-sliding-window |
| ``` |
| ## Exact runtime probe |
| ```bash |
| cd mistral-7b-v0.1-sliding-window |
| python3 evidence/probe_sliding_window.py |
| cat evidence/sliding_window_runtime_probe.json |
| ``` |
| Requires CUDA-capable `onnxruntime-gpu`, `onnx`, `transformers`, and `numpy`; exact successful versions are in `output.json`. |
|
|
| <!-- inference-metadata-annotation:start --> |
| ## Annotated inference metadata |
|
|
| Review [`inference_metadata.annotated.yaml`](./inference_metadata.annotated.yaml) for inline explanations of this package's workflow, tensor/state/cache contracts, and fail-closed omissions. [`inference_metadata.yaml`](./inference_metadata.yaml) remains the canonical machine-authored contract; automated validation confirms both files parse to the same metadata object. |
| <!-- inference-metadata-annotation:end --> |
|
|
