Sync model repo (text/metadata)
Browse files
README.md
CHANGED
|
@@ -68,7 +68,7 @@ Performance was measured on the reference configuration below. Results are inten
|
|
| 68 |
| Runtime | ONNX Runtime 1.27.0 |
|
| 69 |
| Backend / delegate | MLAS, KleidiAI |
|
| 70 |
| Batch size | 1 |
|
| 71 |
-
| Precision | INT4 groupwise asymmetric weights (GPTQ, group size 32), INT8 per-token dynamic activations, INT8 per-row embeddings, INT8 LM head (scheme
|
| 72 |
| Runs | 20 warmup runs + 100 measured runs |
|
| 73 |
|
| 74 |
**Performance results**
|
|
@@ -94,7 +94,7 @@ Accuracy was evaluated using the same preprocessing, input resolution, and evalu
|
|
| 94 |
| Field | Value |
|
| 95 |
|---|---|
|
| 96 |
| Dataset | [HellaSwag](https://huggingface.co/datasets/Rowan/hellaswag) |
|
| 97 |
-
| Split | 0-shot |
|
| 98 |
| Number of samples | 8000 |
|
| 99 |
| Metric(s) | HellaSwag accuracy, acc_norm convention (character-length-normalized) |
|
| 100 |
| Evaluation runtime | ONNX Runtime 1.27.0 |
|
|
@@ -116,7 +116,7 @@ For this release, Arm used:
|
|
| 116 |
| Optimization area | Applied? | Notes |
|
| 117 |
|---|---|---|
|
| 118 |
| Model conversion | Yes | Converted to ONNX in the ONNX Runtime GenAI graph layout, with attention fused into a single GroupQueryAttention op and the KV cache wired internally |
|
| 119 |
-
| Quantization | Yes | GPTQ weight-only quantization to INT4, asymmetric per-group with group size 32, calibrated on 256 randomly selected WikiText2 samples; INT8 per-token dynamic activations; INT8 per-row embedding table; INT8 LM head protected by k_quant_last (scheme
|
| 120 |
| Runtime/backend selection | Yes | ONNX Runtime 1.27.0 CPU execution provider, with MLAS and KleidiAI kernels |
|
| 121 |
| Accuracy validation | Yes | Compared against the original model or published baseline |
|
| 122 |
| Performance validation | Yes | Measured on the reference Arm platform |
|
|
|
|
| 68 |
| Runtime | ONNX Runtime 1.27.0 |
|
| 69 |
| Backend / delegate | MLAS, KleidiAI |
|
| 70 |
| Batch size | 1 |
|
| 71 |
+
| Precision | INT4 groupwise asymmetric weights (GPTQ, group size 32), INT8 per-token dynamic activations, INT8 per-row embeddings, INT8 LM head (scheme W4A8_dyn_emb_int8) |
|
| 72 |
| Runs | 20 warmup runs + 100 measured runs |
|
| 73 |
|
| 74 |
**Performance results**
|
|
|
|
| 94 |
| Field | Value |
|
| 95 |
|---|---|
|
| 96 |
| Dataset | [HellaSwag](https://huggingface.co/datasets/Rowan/hellaswag) |
|
| 97 |
+
| Split | validation (0-shot) |
|
| 98 |
| Number of samples | 8000 |
|
| 99 |
| Metric(s) | HellaSwag accuracy, acc_norm convention (character-length-normalized) |
|
| 100 |
| Evaluation runtime | ONNX Runtime 1.27.0 |
|
|
|
|
| 116 |
| Optimization area | Applied? | Notes |
|
| 117 |
|---|---|---|
|
| 118 |
| Model conversion | Yes | Converted to ONNX in the ONNX Runtime GenAI graph layout, with attention fused into a single GroupQueryAttention op and the KV cache wired internally |
|
| 119 |
+
| Quantization | Yes | GPTQ weight-only quantization to INT4, asymmetric per-group with group size 32, calibrated on 256 randomly selected WikiText2 samples; INT8 per-token dynamic activations; INT8 per-row embedding table; INT8 LM head protected by k_quant_last (scheme W4A8_dyn_emb_int8) |
|
| 120 |
| Runtime/backend selection | Yes | ONNX Runtime 1.27.0 CPU execution provider, with MLAS and KleidiAI kernels |
|
| 121 |
| Accuracy validation | Yes | Compared against the original model or published baseline |
|
| 122 |
| Performance validation | Yes | Measured on the reference Arm platform |
|