| --- |
| license: apache-2.0 |
| datasets: |
| - prometheus-eval/Preference-Collection |
| language: |
| - en |
| base_model: |
| - unsloth/gemma-3-4b-it |
| --- |
| |
| ## 📘 Model Summary |
|
|
| This model is a fine-tuned **preference evaluation model** based on **`unsloth/gemma-3-4b-it`**, trained on the **[`prometheus-eval/Preference-Collection`](https://huggingface.co/datasets/prometheus-eval/Preference-Collection)** dataset. |
| It is designed to perform **pairwise preference comparison** and **alignment evaluation** tasks, inspired by the **Prometheus** framework (*Kim et al., 2023*). |
|
|
| --- |
|
|
| ## 🧮 Performance Benchmark |
|
|
| | Model | Benchmark | Accuracy (%) (Pairwise) | |
| |:------|:-----------|:-----------------------:| |
| | 🟦 **This model** | Preference Bench | **95.6** | |
| | 🟨 **Prometheus 2 (8×7B)** *(Kim et al., 2024)* | Preference Bench | 90.65 | |
|
|
| **Highlights:** |
| - Outperforms **Prometheus 2 (8×7B)** by **+4.95%**, while being **smaller** in size. |
| - Optimized for **efficiency, alignment scoring, and feedback consistency**. |
|
|
| --- |
|
|
| ## 🧾 License |
|
|
| This model is released under the **Apache 2.0 License**. |
| However, because it is derived from **Google’s Gemma 3**, your use of this model must also comply with the **[Gemma Terms of Use](https://ai.google.dev/gemma/terms)**. |
|
|
| By using this model, you agree to: |
| - Follow Google’s **Gemma Model Terms of Use**, including restrictions on misuse and redistribution. |
| - Attribute Google as the original provider of the Gemma 3 base model. |
|
|
| For full details, see: [https://ai.google.dev/gemma/terms](https://ai.google.dev/gemma/terms) |
|
|
| --- |
|
|
