aorabdel commited on
Commit
58af62f
·
verified ·
1 Parent(s): 4694a7a

Sync model repo (text/metadata)

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -68,7 +68,7 @@ Performance was measured on the reference configuration below. Results are inten
68
  | Runtime | ONNX Runtime 1.27.0 |
69
  | Backend / delegate | MLAS, KleidiAI |
70
  | Batch size | 1 |
71
- | Precision | INT4 groupwise asymmetric weights (GPTQ, group size 32), INT8 per-token dynamic activations, INT8 per-row embeddings, INT8 LM head (scheme W4A8_dyn_emb_int8_lmhead_int8) |
72
  | Runs | 20 warmup runs + 100 measured runs |
73
 
74
  **Performance results**
@@ -94,7 +94,7 @@ Accuracy was evaluated using the same preprocessing, input resolution, and evalu
94
  | Field | Value |
95
  |---|---|
96
  | Dataset | [HellaSwag](https://huggingface.co/datasets/Rowan/hellaswag) |
97
- | Split | 0-shot |
98
  | Number of samples | 8000 |
99
  | Metric(s) | HellaSwag accuracy, acc_norm convention (character-length-normalized) |
100
  | Evaluation runtime | ONNX Runtime 1.27.0 |
@@ -116,7 +116,7 @@ For this release, Arm used:
116
  | Optimization area | Applied? | Notes |
117
  |---|---|---|
118
  | Model conversion | Yes | Converted to ONNX in the ONNX Runtime GenAI graph layout, with attention fused into a single GroupQueryAttention op and the KV cache wired internally |
119
- | Quantization | Yes | GPTQ weight-only quantization to INT4, asymmetric per-group with group size 32, calibrated on 256 randomly selected WikiText2 samples; INT8 per-token dynamic activations; INT8 per-row embedding table; INT8 LM head protected by k_quant_last (scheme W4A8_dyn_emb_int8_lmhead_int8) |
120
  | Runtime/backend selection | Yes | ONNX Runtime 1.27.0 CPU execution provider, with MLAS and KleidiAI kernels |
121
  | Accuracy validation | Yes | Compared against the original model or published baseline |
122
  | Performance validation | Yes | Measured on the reference Arm platform |
 
68
  | Runtime | ONNX Runtime 1.27.0 |
69
  | Backend / delegate | MLAS, KleidiAI |
70
  | Batch size | 1 |
71
+ | Precision | INT4 groupwise asymmetric weights (GPTQ, group size 32), INT8 per-token dynamic activations, INT8 per-row embeddings, INT8 LM head (scheme W4A8_dyn_emb_int8) |
72
  | Runs | 20 warmup runs + 100 measured runs |
73
 
74
  **Performance results**
 
94
  | Field | Value |
95
  |---|---|
96
  | Dataset | [HellaSwag](https://huggingface.co/datasets/Rowan/hellaswag) |
97
+ | Split | validation (0-shot) |
98
  | Number of samples | 8000 |
99
  | Metric(s) | HellaSwag accuracy, acc_norm convention (character-length-normalized) |
100
  | Evaluation runtime | ONNX Runtime 1.27.0 |
 
116
  | Optimization area | Applied? | Notes |
117
  |---|---|---|
118
  | Model conversion | Yes | Converted to ONNX in the ONNX Runtime GenAI graph layout, with attention fused into a single GroupQueryAttention op and the KV cache wired internally |
119
+ | Quantization | Yes | GPTQ weight-only quantization to INT4, asymmetric per-group with group size 32, calibrated on 256 randomly selected WikiText2 samples; INT8 per-token dynamic activations; INT8 per-row embedding table; INT8 LM head protected by k_quant_last (scheme W4A8_dyn_emb_int8) |
120
  | Runtime/backend selection | Yes | ONNX Runtime 1.27.0 CPU execution provider, with MLAS and KleidiAI kernels |
121
  | Accuracy validation | Yes | Compared against the original model or published baseline |
122
  | Performance validation | Yes | Measured on the reference Arm platform |