Sync model repo (text/metadata)
Browse files- README.md +0 -1
- benchmarks/README.md +0 -49
README.md
CHANGED
|
@@ -1,6 +1,5 @@
|
|
| 1 |
---
|
| 2 |
library_name: onnxruntime-genai
|
| 3 |
-
display_name: TinyLlama-1.1B-Chat INT4 — ONNX GenAI (Vivo X300)
|
| 4 |
license: apache-2.0
|
| 5 |
tags:
|
| 6 |
- text-generation
|
|
|
|
| 1 |
---
|
| 2 |
library_name: onnxruntime-genai
|
|
|
|
| 3 |
license: apache-2.0
|
| 4 |
tags:
|
| 5 |
- text-generation
|
benchmarks/README.md
DELETED
|
@@ -1,49 +0,0 @@
|
|
| 1 |
-
# Benchmarks
|
| 2 |
-
|
| 3 |
-
Machine-readable measurement records for this model, using the delivered
|
| 4 |
-
`nlp-llm` metadata shape from the ARM Model Optimization Pipeline.
|
| 5 |
-
|
| 6 |
-
These complement the eval block in the model card. The card carries the headline
|
| 7 |
-
numbers in human-readable form; these YAMLs carry the same data with target
|
| 8 |
-
hardware, runtime config, dataset, and benchmark workload context.
|
| 9 |
-
|
| 10 |
-
## Files
|
| 11 |
-
|
| 12 |
-
- `tinyllama-1-1b-chat-onnx-genai-vivo-x300-int4.yaml` — optimized INT4/INT8 ONNX
|
| 13 |
-
Runtime GenAI measurements on the Vivo X300 smartphone via MLAS + KleidiAI.
|
| 14 |
-
- `tinyllama-1-1b-chat-onnx-genai-vivo-x300-fp32.yaml` — FP32 runtime baseline on
|
| 15 |
-
the same target/workload, from the delivery baseline metadata.
|
| 16 |
-
|
| 17 |
-
## Reading these
|
| 18 |
-
|
| 19 |
-
Each record has three main areas:
|
| 20 |
-
|
| 21 |
-
- `context.{model,target,runtime,dataset,benchmark}` — everything needed to
|
| 22 |
-
understand the measurement setup.
|
| 23 |
-
- `performance` — LLM runtime metrics: E2E latency percentiles, TTFT, decode
|
| 24 |
-
throughput, model-load / time-to-first-inference, and peak/average memory.
|
| 25 |
-
- `accuracy` — HellaSwag zero-shot accuracy (`acc_norm`, character-normalized).
|
| 26 |
-
|
| 27 |
-
The two records share target/runtime/dataset/workload context. The disambiguator
|
| 28 |
-
is `weight_dtype`.
|
| 29 |
-
|
| 30 |
-
## Notes specific to this model
|
| 31 |
-
|
| 32 |
-
- This is the `TinyLlama/TinyLlama-1.1B-Chat-v1.0` chat model, re-packed to
|
| 33 |
-
W4A8-dynamic INT4 (GPTQ) with INT8 per-row embeddings and an INT8 LM head via
|
| 34 |
-
the `onnxruntime-genai` model builder (recipe `exp10_kquantlast_emb_int8_asym`).
|
| 35 |
-
- The runtime metrics were measured on the **Vivo X300 premium smartphone**
|
| 36 |
-
(C1-Ultra / C1-Premium / C1-Pro, Android 16 / OriginOS 6), not a server CPU or
|
| 37 |
-
Raspberry Pi. The AWS Graviton G4 cut of this same INT4 model is a separate repo
|
| 38 |
-
(`tinyllama-1-1b-chat-onnx-genai-int4-kquantlast-emb-int8-graviton-g4`); the optimized
|
| 39 |
-
weights are byte-identical across the two targets — only the benchmark records differ.
|
| 40 |
-
- The optimized `model.onnx.data` sidecar is about 728 MB and is gitignored because it
|
| 41 |
-
exceeds GitHub's 100 MB per-file limit. The FP32 baseline ONNX artifact (~4.10 GB) is
|
| 42 |
-
not present in this repository.
|
| 43 |
-
|
| 44 |
-
## Provenance
|
| 45 |
-
|
| 46 |
-
Source: `models/tinyllama_1_1b_chat_vivo_x300_onnx/` (`metadata.yaml`,
|
| 47 |
-
`baseline_metadata.yaml`, `report.json`, and `huggingface/README.md`). The optimized
|
| 48 |
-
record maps from the delivered `metadata.yaml`; the FP32 baseline maps from the
|
| 49 |
-
delivered `baseline_metadata.yaml`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|