File size: 1,377 Bytes
c00a325 38efa25 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | We released **Abel-7B-002**, resulting in a stronger (35% improvement on GSM8K, 126% improvement on MATH) and more generalized model, achieving the best performance among all 7B models (80.44 on GSM8K, 29.46 on MATH)
| Model | GSM8k | MATH |MathQA | SVAMP |SCQ5K-EN | ARC-E|ARC-C|HellaSwag|MMLU |
|-----------|------------|----------|--------------|-----------|----------------|---------|----------|---------------|----------|
|Abel-7B-002 | **80.44** | **29.46** | **69.78** |77.67 |**55.95** |77.67 |**55.05** |77.72 |61.19 |
|Abel-7B-001 |59.74 |13 |1.21 |57.67 |9.3 |53.32 |38.97 |63.51|40.59 |
|MetaMath-Mistral-7B|77.7 |28.2 |33.94 |**79.33** |37.6| **78.48** |51.93 |76.44| 61.93|
|Qwen-7b|47.84 |9.34 |27.44 |53 |40.05 |74.97 |53.05 |**86.85**|57.98 |
|Mistral-7b|37.83 |9.06 |25.73 |63 |39.6 |76.83 |53.22| 76.31|**64.05** |
|Yi-6b| 32.6 |5.78 |26.98 |55.67 |35.5 |73.66 |49.53 |68.97|64.02 |
|LLaMA2-7b|12.96 |2.78 |11.52 |44 |28.24 |71.12 |46.61 |71.32|46.7 |
Please cite the repo if the model/code/conclusion in this repo are helpful to you.
```
@misc{abel,
author = {Chern, Ethan and Zou, Haoyang and Li, Xuefeng and Hu, Jiewen and Feng, Kehua and Li, Junlong and Liu, Pengfei},
title = {Generative AI for Math: Abel},
year = {2023},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/GAIR-NLP/abel}},
}
``` |