Granite Library
Safetensors
GGUF
English
yannisk2 commited on
Commit
a4d4a9b
·
verified ·
1 Parent(s): 43c92f1

Update citation generation model card

Browse files
Files changed (1) hide show
  1. citations/README.md +36 -14
citations/README.md CHANGED
@@ -104,20 +104,6 @@ Git Repos and Issue Tracking is an IBM-hosted component of the Continuous Delive
104
 
105
  The recommended way to call this intrinsic is through the [Mellea](https://mellea.ai) framework. For code snippets demonstrating how to use this and other intrinsics, please refer to the [Mellea intrinsics examples](https://github.com/generative-computing/mellea/tree/main/docs/examples/intrinsics).
106
 
107
- ## Training Details
108
-
109
- The citation generation intrinsic was trained on synthetically-generated citation datasets. The process of generating the training data consisted of two main steps:
110
- - _Multi-turn RAG conversation generation:_ Starting from publicly available document corpora, we generated a set of multi-turn RAG data, consisting of multi-turn conversations grounded on passages retrieved from the corpora. For details on the RAG conversation generation process please refer to the [Granite Technical Report](https://github.com/ibm-granite/granite-3.0-language-models/blob/main/paper.pdf) and [Lee, Young-Suk, et al.](https://arxiv.org/pdf/2409.11500)
111
- - _Citation generation:_ For each turn of the multi-turn RAG conversations from the previous step, we used a multi-step synthetic citation generation pipeline to generate citations for the assistant response.
112
-
113
- The resulting data instances were used to train the citation generation intrinsic.
114
-
115
- ### Training Data
116
-
117
- The following public datasets were used as seed datasets for the multi-turn RAG conversation generation process:
118
- - [MultiDoc2Dial](https://huggingface.co/datasets/IBM/multidoc2dial)
119
- - [QuAC](https://huggingface.co/datasets/allenai/quac)
120
-
121
  ## Evaluation
122
 
123
  We evaluated the citation generation intrinsic on a revised version of the [LongBench-Cite](https://arxiv.org/abs/2409.02897) benchmark; a benchmark evaluating the ability of models to produce fine-grained span-level citations (i.e., identify the spans within the input documents/passages that support a statement in the response) with a focus on long contexts. Being originally designed to evaluate inline citation generation approaches (i.e., approaches generating the assistant response and the citations at the same time), we adapt the benchmark for the evaluation of post-hoc citation generation (where citations are generated for a given assistant response generated by an upstream model).
@@ -217,6 +203,42 @@ Notes:
217
  - To generate the assistant responses fed to all evaluated models, we prompted Llama-3.1-70B-Instruct using the one-shot prompt described in the LongBench-Cite paper (which asks the model to generate a grounded response with citations) and removed the citations in post-processing.
218
  - The AVG F1 column contains the average of the four dataset-specific F1 scores.
219
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
220
  ## Model Card Authors
221
 
222
  [Yannis Katsis](mailto:yannis.katsis@ibm.com)</br>
 
104
 
105
  The recommended way to call this intrinsic is through the [Mellea](https://mellea.ai) framework. For code snippets demonstrating how to use this and other intrinsics, please refer to the [Mellea intrinsics examples](https://github.com/generative-computing/mellea/tree/main/docs/examples/intrinsics).
106
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
107
  ## Evaluation
108
 
109
  We evaluated the citation generation intrinsic on a revised version of the [LongBench-Cite](https://arxiv.org/abs/2409.02897) benchmark; a benchmark evaluating the ability of models to produce fine-grained span-level citations (i.e., identify the spans within the input documents/passages that support a statement in the response) with a focus on long contexts. Being originally designed to evaluate inline citation generation approaches (i.e., approaches generating the assistant response and the citations at the same time), we adapt the benchmark for the evaluation of post-hoc citation generation (where citations are generated for a given assistant response generated by an upstream model).
 
203
  - To generate the assistant responses fed to all evaluated models, we prompted Llama-3.1-70B-Instruct using the one-shot prompt described in the LongBench-Cite paper (which asks the model to generate a grounded response with citations) and removed the citations in post-processing.
204
  - The AVG F1 column contains the average of the four dataset-specific F1 scores.
205
 
206
+ ## Training Details
207
+
208
+ The citation generation intrinsic was trained on synthetically-generated citation datasets. The process of generating the training data consisted of two main steps:
209
+ - _Multi-turn RAG conversation generation:_ Starting from publicly available document corpora, we generated a set of multi-turn RAG data, consisting of multi-turn conversations grounded on passages retrieved from the corpora. For details on the RAG conversation generation process please refer to the [Granite Technical Report](https://github.com/ibm-granite/granite-3.0-language-models/blob/main/paper.pdf) and [Lee, Young-Suk, et al.](https://arxiv.org/pdf/2409.11500)
210
+ - _Citation generation:_ For each turn of the multi-turn RAG conversations from the previous step, we used a multi-step synthetic citation generation pipeline to generate citations for the assistant response.
211
+
212
+ The resulting data instances were used to train the citation generation intrinsic.
213
+
214
+ ### Training Data
215
+
216
+ The following public datasets were used as seed datasets for the multi-turn RAG conversation generation process:
217
+ - [MultiDoc2Dial](https://huggingface.co/datasets/IBM/multidoc2dial)
218
+ - [QuAC](https://huggingface.co/datasets/allenai/quac)
219
+
220
+ ### Adapter Details
221
+
222
+ | Property | LoRA |
223
+ |---|---|
224
+ | **Base Model** | ibm-granite/granite-4.0-micro |
225
+ | **PEFT Type** | LORA |
226
+ | **Rank (r)** | 16 |
227
+ | **Alpha** | 32 |
228
+ | **Target Modules** | q_proj, k_proj, v_proj, o_proj |
229
+
230
+ **Infrastructure:**
231
+ We trained the citation generation granite-4.0-micro LoRA adapter on IBM's Vela cluster using 8 A100 GPUs.
232
+
233
+ **Ethical Considerations & Limitations:**
234
+ The model's outputs are not guaranteed to be factually accurate or complete. All outputs should be independently validated before use in decision-making or downstream applications. The model has been trained and evaluated on English data only.
235
+
236
+ ## Resources
237
+
238
+ - ⭐️ Learn about the latest updates with Granite: https://www.ibm.com/granite
239
+ - 📄 Get started with tutorials, best practices, and prompt engineering advice: https://www.ibm.com/granite/docs/
240
+ - 💡 Learn about the latest Granite learning resources: https://github.com/ibm-granite/granite-guardian/tree/main/cookbooks
241
+
242
  ## Model Card Authors
243
 
244
  [Yannis Katsis](mailto:yannis.katsis@ibm.com)</br>