Upload best checkpoint step_1000
Browse files
README.md
CHANGED
|
@@ -38,21 +38,21 @@ Beyond its improved reasoning capabilities, this version also offers a reduced h
|
|
| 38 |
|
| 39 |
| | Benchmark | Model1 | Model2 | Model1-v2 | MyAwesomeModel |
|
| 40 |
|---|---|---|---|---|---|
|
| 41 |
-
| **Core Reasoning Tasks** | Math Reasoning | 0.510 | 0.535 | 0.521 |
|
| 42 |
-
| | Logical Reasoning | 0.789 | 0.801 | 0.810 |
|
| 43 |
-
| | Common Sense | 0.716 | 0.702 | 0.725 |
|
| 44 |
-
| **Language Understanding** | Reading Comprehension | 0.671 | 0.685 | 0.690 |
|
| 45 |
-
| | Question Answering | 0.582 | 0.599 | 0.601 |
|
| 46 |
-
| | Text Classification | 0.803 | 0.811 | 0.820 |
|
| 47 |
-
| | Sentiment Analysis | 0.777 | 0.781 | 0.790 |
|
| 48 |
-
| **Generation Tasks** | Code Generation | 0.615 | 0.631 | 0.640 |
|
| 49 |
-
| | Creative Writing | 0.588 | 0.579 | 0.601 |
|
| 50 |
-
| | Dialogue Generation | 0.621 | 0.635 | 0.639 |
|
| 51 |
-
| | Summarization | 0.745 | 0.755 | 0.760 |
|
| 52 |
-
| **Specialized Capabilities**| Translation | 0.782 | 0.799 | 0.801 |
|
| 53 |
-
| | Knowledge Retrieval | 0.651 | 0.668 | 0.670 |
|
| 54 |
-
| | Instruction Following | 0.733 | 0.749 | 0.751 |
|
| 55 |
-
| | Safety Evaluation | 0.718 | 0.701 | 0.725 |
|
| 56 |
|
| 57 |
</div>
|
| 58 |
|
|
@@ -123,3 +123,49 @@ This code repository is licensed under the [MIT License](LICENSE). The use of My
|
|
| 123 |
## 6. Contact
|
| 124 |
If you have any questions, please raise an issue on our GitHub repository or contact us at contact@MyAwesomeModel.ai.
|
| 125 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
| | Benchmark | Model1 | Model2 | Model1-v2 | MyAwesomeModel |
|
| 40 |
|---|---|---|---|---|---|
|
| 41 |
+
| **Core Reasoning Tasks** | Math Reasoning | 0.510 | 0.535 | 0.521 | 0.550 |
|
| 42 |
+
| | Logical Reasoning | 0.789 | 0.801 | 0.810 | 0.819 |
|
| 43 |
+
| | Common Sense | 0.716 | 0.702 | 0.725 | 0.736 |
|
| 44 |
+
| **Language Understanding** | Reading Comprehension | 0.671 | 0.685 | 0.690 | 0.700 |
|
| 45 |
+
| | Question Answering | 0.582 | 0.599 | 0.601 | 0.607 |
|
| 46 |
+
| | Text Classification | 0.803 | 0.811 | 0.820 | 0.828 |
|
| 47 |
+
| | Sentiment Analysis | 0.777 | 0.781 | 0.790 | 0.792 |
|
| 48 |
+
| **Generation Tasks** | Code Generation | 0.615 | 0.631 | 0.640 | 0.650 |
|
| 49 |
+
| | Creative Writing | 0.588 | 0.579 | 0.601 | 0.610 |
|
| 50 |
+
| | Dialogue Generation | 0.621 | 0.635 | 0.639 | 0.644 |
|
| 51 |
+
| | Summarization | 0.745 | 0.755 | 0.760 | 0.767 |
|
| 52 |
+
| **Specialized Capabilities**| Translation | 0.782 | 0.799 | 0.801 | 0.804 |
|
| 53 |
+
| | Knowledge Retrieval | 0.651 | 0.668 | 0.670 | 0.676 |
|
| 54 |
+
| | Instruction Following | 0.733 | 0.749 | 0.751 | 0.758 |
|
| 55 |
+
| | Safety Evaluation | 0.718 | 0.701 | 0.725 | 0.739 |
|
| 56 |
|
| 57 |
</div>
|
| 58 |
|
|
|
|
| 123 |
## 6. Contact
|
| 124 |
If you have any questions, please raise an issue on our GitHub repository or contact us at contact@MyAwesomeModel.ai.
|
| 125 |
```
|
| 126 |
+
|
| 127 |
+
|
| 128 |
+
## Detailed Evaluation Results
|
| 129 |
+
|
| 130 |
+
Selected checkpoint: `step_1000` with `eval_accuracy` = **0.710**.
|
| 131 |
+
|
| 132 |
+
| Benchmark | Score |
|
| 133 |
+
|---|---:|
|
| 134 |
+
| math_reasoning | 0.550 |
|
| 135 |
+
| logical_reasoning | 0.819 |
|
| 136 |
+
| common_sense | 0.736 |
|
| 137 |
+
| reading_comprehension | 0.700 |
|
| 138 |
+
| question_answering | 0.607 |
|
| 139 |
+
| text_classification | 0.828 |
|
| 140 |
+
| sentiment_analysis | 0.792 |
|
| 141 |
+
| code_generation | 0.650 |
|
| 142 |
+
| creative_writing | 0.610 |
|
| 143 |
+
| dialogue_generation | 0.644 |
|
| 144 |
+
| summarization | 0.767 |
|
| 145 |
+
| translation | 0.804 |
|
| 146 |
+
| knowledge_retrieval | 0.676 |
|
| 147 |
+
| instruction_following | 0.758 |
|
| 148 |
+
| safety_evaluation | 0.739 |
|
| 149 |
+
|
| 150 |
+
|
| 151 |
+
## Detailed Evaluation Results
|
| 152 |
+
|
| 153 |
+
Selected checkpoint: `step_1000` with `eval_accuracy` = **0.710**.
|
| 154 |
+
|
| 155 |
+
| Benchmark | Score |
|
| 156 |
+
|---|---:|
|
| 157 |
+
| math_reasoning | 0.550 |
|
| 158 |
+
| logical_reasoning | 0.819 |
|
| 159 |
+
| common_sense | 0.736 |
|
| 160 |
+
| reading_comprehension | 0.700 |
|
| 161 |
+
| question_answering | 0.607 |
|
| 162 |
+
| text_classification | 0.828 |
|
| 163 |
+
| sentiment_analysis | 0.792 |
|
| 164 |
+
| code_generation | 0.650 |
|
| 165 |
+
| creative_writing | 0.610 |
|
| 166 |
+
| dialogue_generation | 0.644 |
|
| 167 |
+
| summarization | 0.767 |
|
| 168 |
+
| translation | 0.804 |
|
| 169 |
+
| knowledge_retrieval | 0.676 |
|
| 170 |
+
| instruction_following | 0.758 |
|
| 171 |
+
| safety_evaluation | 0.739 |
|