Instructions to use ibm-granite/granitelib-rag-r1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Granite Library
How to use ibm-granite/granitelib-rag-r1.0 with Granite Library:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Update citation generation model card
Browse files- citations/README.md +36 -14
citations/README.md
CHANGED
|
@@ -104,20 +104,6 @@ Git Repos and Issue Tracking is an IBM-hosted component of the Continuous Delive
|
|
| 104 |
|
| 105 |
The recommended way to call this intrinsic is through the [Mellea](https://mellea.ai) framework. For code snippets demonstrating how to use this and other intrinsics, please refer to the [Mellea intrinsics examples](https://github.com/generative-computing/mellea/tree/main/docs/examples/intrinsics).
|
| 106 |
|
| 107 |
-
## Training Details
|
| 108 |
-
|
| 109 |
-
The citation generation intrinsic was trained on synthetically-generated citation datasets. The process of generating the training data consisted of two main steps:
|
| 110 |
-
- _Multi-turn RAG conversation generation:_ Starting from publicly available document corpora, we generated a set of multi-turn RAG data, consisting of multi-turn conversations grounded on passages retrieved from the corpora. For details on the RAG conversation generation process please refer to the [Granite Technical Report](https://github.com/ibm-granite/granite-3.0-language-models/blob/main/paper.pdf) and [Lee, Young-Suk, et al.](https://arxiv.org/pdf/2409.11500)
|
| 111 |
-
- _Citation generation:_ For each turn of the multi-turn RAG conversations from the previous step, we used a multi-step synthetic citation generation pipeline to generate citations for the assistant response.
|
| 112 |
-
|
| 113 |
-
The resulting data instances were used to train the citation generation intrinsic.
|
| 114 |
-
|
| 115 |
-
### Training Data
|
| 116 |
-
|
| 117 |
-
The following public datasets were used as seed datasets for the multi-turn RAG conversation generation process:
|
| 118 |
-
- [MultiDoc2Dial](https://huggingface.co/datasets/IBM/multidoc2dial)
|
| 119 |
-
- [QuAC](https://huggingface.co/datasets/allenai/quac)
|
| 120 |
-
|
| 121 |
## Evaluation
|
| 122 |
|
| 123 |
We evaluated the citation generation intrinsic on a revised version of the [LongBench-Cite](https://arxiv.org/abs/2409.02897) benchmark; a benchmark evaluating the ability of models to produce fine-grained span-level citations (i.e., identify the spans within the input documents/passages that support a statement in the response) with a focus on long contexts. Being originally designed to evaluate inline citation generation approaches (i.e., approaches generating the assistant response and the citations at the same time), we adapt the benchmark for the evaluation of post-hoc citation generation (where citations are generated for a given assistant response generated by an upstream model).
|
|
@@ -217,6 +203,42 @@ Notes:
|
|
| 217 |
- To generate the assistant responses fed to all evaluated models, we prompted Llama-3.1-70B-Instruct using the one-shot prompt described in the LongBench-Cite paper (which asks the model to generate a grounded response with citations) and removed the citations in post-processing.
|
| 218 |
- The AVG F1 column contains the average of the four dataset-specific F1 scores.
|
| 219 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 220 |
## Model Card Authors
|
| 221 |
|
| 222 |
[Yannis Katsis](mailto:yannis.katsis@ibm.com)</br>
|
|
|
|
| 104 |
|
| 105 |
The recommended way to call this intrinsic is through the [Mellea](https://mellea.ai) framework. For code snippets demonstrating how to use this and other intrinsics, please refer to the [Mellea intrinsics examples](https://github.com/generative-computing/mellea/tree/main/docs/examples/intrinsics).
|
| 106 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 107 |
## Evaluation
|
| 108 |
|
| 109 |
We evaluated the citation generation intrinsic on a revised version of the [LongBench-Cite](https://arxiv.org/abs/2409.02897) benchmark; a benchmark evaluating the ability of models to produce fine-grained span-level citations (i.e., identify the spans within the input documents/passages that support a statement in the response) with a focus on long contexts. Being originally designed to evaluate inline citation generation approaches (i.e., approaches generating the assistant response and the citations at the same time), we adapt the benchmark for the evaluation of post-hoc citation generation (where citations are generated for a given assistant response generated by an upstream model).
|
|
|
|
| 203 |
- To generate the assistant responses fed to all evaluated models, we prompted Llama-3.1-70B-Instruct using the one-shot prompt described in the LongBench-Cite paper (which asks the model to generate a grounded response with citations) and removed the citations in post-processing.
|
| 204 |
- The AVG F1 column contains the average of the four dataset-specific F1 scores.
|
| 205 |
|
| 206 |
+
## Training Details
|
| 207 |
+
|
| 208 |
+
The citation generation intrinsic was trained on synthetically-generated citation datasets. The process of generating the training data consisted of two main steps:
|
| 209 |
+
- _Multi-turn RAG conversation generation:_ Starting from publicly available document corpora, we generated a set of multi-turn RAG data, consisting of multi-turn conversations grounded on passages retrieved from the corpora. For details on the RAG conversation generation process please refer to the [Granite Technical Report](https://github.com/ibm-granite/granite-3.0-language-models/blob/main/paper.pdf) and [Lee, Young-Suk, et al.](https://arxiv.org/pdf/2409.11500)
|
| 210 |
+
- _Citation generation:_ For each turn of the multi-turn RAG conversations from the previous step, we used a multi-step synthetic citation generation pipeline to generate citations for the assistant response.
|
| 211 |
+
|
| 212 |
+
The resulting data instances were used to train the citation generation intrinsic.
|
| 213 |
+
|
| 214 |
+
### Training Data
|
| 215 |
+
|
| 216 |
+
The following public datasets were used as seed datasets for the multi-turn RAG conversation generation process:
|
| 217 |
+
- [MultiDoc2Dial](https://huggingface.co/datasets/IBM/multidoc2dial)
|
| 218 |
+
- [QuAC](https://huggingface.co/datasets/allenai/quac)
|
| 219 |
+
|
| 220 |
+
### Adapter Details
|
| 221 |
+
|
| 222 |
+
| Property | LoRA |
|
| 223 |
+
|---|---|
|
| 224 |
+
| **Base Model** | ibm-granite/granite-4.0-micro |
|
| 225 |
+
| **PEFT Type** | LORA |
|
| 226 |
+
| **Rank (r)** | 16 |
|
| 227 |
+
| **Alpha** | 32 |
|
| 228 |
+
| **Target Modules** | q_proj, k_proj, v_proj, o_proj |
|
| 229 |
+
|
| 230 |
+
**Infrastructure:**
|
| 231 |
+
We trained the citation generation granite-4.0-micro LoRA adapter on IBM's Vela cluster using 8 A100 GPUs.
|
| 232 |
+
|
| 233 |
+
**Ethical Considerations & Limitations:**
|
| 234 |
+
The model's outputs are not guaranteed to be factually accurate or complete. All outputs should be independently validated before use in decision-making or downstream applications. The model has been trained and evaluated on English data only.
|
| 235 |
+
|
| 236 |
+
## Resources
|
| 237 |
+
|
| 238 |
+
- ⭐️ Learn about the latest updates with Granite: https://www.ibm.com/granite
|
| 239 |
+
- 📄 Get started with tutorials, best practices, and prompt engineering advice: https://www.ibm.com/granite/docs/
|
| 240 |
+
- 💡 Learn about the latest Granite learning resources: https://github.com/ibm-granite/granite-guardian/tree/main/cookbooks
|
| 241 |
+
|
| 242 |
## Model Card Authors
|
| 243 |
|
| 244 |
[Yannis Katsis](mailto:yannis.katsis@ibm.com)</br>
|