Update READMEs (#37)

- Update root README.md (e382f994a53083340ac756818b9fd24869743e55)
- Update Context Attribution README (703afb713ac006a7a6f533cda65f9606e33f9836)
- Updated RC README.md (cbb9d94a3bb1c1436818e4aee7d4c4018f4dc1ad)
- Update UQ README (22fad6714b920494acee05b3e513b93a90468359)
- Delete uncertainty/granite-4.0-micro/README.md (a1b9328ec64cefccd01ca01569cc5b97e6f2bd45)
- Delete requirement-check/granite-4.0-micro/README.md (bb38dd1259206d21b6301bf49762d2a45713cf63)
- Delete context-attribution/granite-4.0-micro/README.md (40c1fdc8f6d4e01d667c56d63f86036cc3344fd2)
- Delete uncertainty/granite-4.0-micro/.gitattributes (c066afbf85ea5c67b4998ef2c53a5c9ced856464)
- Update requirement-check/README.md (25638fa8bed24323db7ab16d3a6362b8dc7e8261)


Co-authored-by: Inkit <ink-pad@users.noreply.huggingface.co>

README.md

@@ -4,7 +4,11 @@ language:
 - en
 pipeline_tag: text-generation
 library_name: granite-library
-base_model: ibm-granite/granite-4.0-micro
+base_model:
+- ibm-granite/granite-4.0-micro
+- ibm-granite/granite-4.1-3b
+- ibm-granite/granite-4.1-8b
+- ibm-granite/granite-4.1-30b
 tags:
 - granite
 - uncertainty
@@ -16,15 +20,74 @@ tags:
 
 # Granite Core Library
 
-The Granite Core Library includes three LoRA adapters for [ibm-granite/granite-4.0-micro](https://huggingface.co/ibm-granite/granite-4.0-micro). Each adapter has been developed for a specific task that enhances the base model's capabilities for explainability, calibration, and constraint verification. We give a brief overview of the functionality of each adapter, as the details can be found in each individual adapter readme.
+The Granite Core Library includes three families of LoRA adapters, each developed for a specific task that enhances the base model's capabilities for explainability, calibration, and constraint verification. We provide adapters for: 
+ * [ibm-granite/granite-4.0-micro](https://huggingface.co/ibm-granite/granite-4.0-micro)
+ * [ibm-granite/granite-4.1-3b](https://huggingface.co/ibm-granite/granite-4.1-3b)
+ * [ibm-granite/granite-4.1-8b](https://huggingface.co/ibm-granite/granite-4.1-8b)
+ * [ibm-granite/granite-4.1-30b](https://huggingface.co/ibm-granite/granite-4.1-30b)
 
+We give a brief overview of the functionality of each capability below; details can be found in each individual adapter readme.
 
 ## Capabilities implemented as LoRA adapters
 
-The three capabilities that have been implemented as LoRA adapters for [ibm-granite/granite-4.0-micro](https://huggingface.co/ibm-granite/granite-4.0-micro) and made available in this HF repository are:
+The three capabilities that have been implemented as LoRA adapters and made available in this HF repository are:
 
-**Context Attribution:** A LoRA adapter that predicts which sentences in the context (including documents and previous conversation turns) were most important for the base model to generate each sentence in its response. The adapter provides contributive attribution, identifying the context sentences that actually influenced the model's response, and ranks them by importance. This is a post-generation intrinsic designed to explain the base model's behavior. Details can be found in the [context-attribution readme](context-attribution/granite-4.0-micro/README.md).
+**Context Attribution:** A LoRA adapter that predicts which sentences in the context (including documents and previous conversation turns) were most important for the base model to generate each sentence in its response. The adapter provides contributive attribution, identifying the context sentences that actually influenced the model's response, and ranks them by importance. This is a post-generation intrinsic designed to explain the base model's behavior. Details can be found in the [context-attribution readme](context-attribution/README.md).
 
-**Requirement Check:** A LoRA adapter that judges whether an assistant's generation satisfies a set of user-specified requirements, outputting a binary yes/no assessment. It is designed for evaluating constraint satisfaction in instruction-following scenarios, including formatting, content, structural requirements, and quality assurance pipelines. Details can be found in the [requirement-check readme](requirement-check/granite-4.0-micro/README.md).
+**Requirement Check:** A LoRA adapter that judges whether an assistant's generation satisfies a set of user-specified requirements, outputting a binary yes/no assessment. It is designed for evaluating constraint satisfaction in instruction-following scenarios, including formatting, content, structural requirements, and quality assurance pipelines. Details can be found in the [requirement-check readme](requirement-check/README.md).
 
-**Uncertainty:** A LoRA adapter that provides calibrated certainty scores for the base model's responses. Given a conversation ending with an assistant response, it calculates a certainty percentage reflecting how confident the model is about its answer. Certainty scores are calibrated such that among answers assigned a score of X%, approximately X% are correct. Details can be found in the [uncertainty readme](uncertainty/granite-4.0-micro/README.md).
+**Uncertainty:** A LoRA adapter that provides calibrated certainty scores for the base model's responses. Given a conversation ending with an assistant response, it calculates a certainty percentage reflecting how confident the model is about its answer. Certainty scores are calibrated such that among answers assigned a score of X%, approximately X% are correct. Details can be found in the [uncertainty readme](uncertainty/README.md).
+
+
+
+## Recommended Use
+
+The recommended way to call all adapters is through the [Mellea](https://mellea.ai) framework. For code snippets demonstrating how to use them please refer to the [Mellea adapter examples](https://github.com/generative-computing/mellea/tree/main/docs/examples/intrinsics) (note: within Mellea, adapters are referred to as "intrinsics").
+
+
+## Model Signing
+
+All adapter artifacts in this repository are signed to ensure integrity and provenance. Each adapter includes a `model.sig` signature file in its `lora/` directory that covers all artifacts in that directory (`adapter_config.json`, `adapter_model.safetensors`, `io.yaml`).
+
+| Adapter | Signature File Path (example) | Signing Identity |
+|-----------|-------------------------------|-----------------|
+| Context Attribution | `context-attribution/granite-4.1-3b/lora/model.sig` | `Granite-sign@ibm.com` |
+| Requirement Check | `requirement-check/granite-4.1-3b/lora/model.sig` | `Granite-sign@ibm.com` |
+| Uncertainty | `uncertainty/granite-4.1-3b/lora/model.sig` | `Granite-sign@ibm.com` |
+
+The same pattern applies to all model variants (`granite-4.0-micro`, `granite-4.1-3b`, `granite-4.1-8b`, `granite-4.1-30b`).
+
+### Verifying Model Signatures
+
+To verify the integrity of a downloaded adapter, use the [model-signing](https://github.com/sigstore/model-transparency) tool:
+
+```bash
+# Install the model signing verification tool
+pip install model-signing
+
+# Verify all artifacts in an adapter's lora/ directory
+model_signing verify sigstore \
+  --signature <adapter>/<model-variant>/lora/model.sig \
+  --identity Granite-sign@ibm.com \
+  --identity_provider https://sigstore.verify.ibm.com/oauth2 \
+  <adapter>/<model-variant>/lora/
+```
+
+For example, to verify the context-attribution adapter for granite-4.1-3b:
+
+```bash
+model_signing verify sigstore \
+  --signature context-attribution/granite-4.1-3b/lora/model.sig \
+  --identity Granite-sign@ibm.com \
+  --identity_provider https://sigstore.verify.ibm.com/oauth2 \
+  context-attribution/granite-4.1-3b/lora/
+```
+
+Each `model.sig` file contains a signature over all adapter artifacts in the corresponding `lora/` directory, signed with the identity `Granite-sign@ibm.com`. This allows users to confirm that the adapter has not been tampered with after release.
+
+
+## Resources
+
+- Learn about the latest updates with Granite: https://www.ibm.com/granite
+- Get started with tutorials, best practices, and prompt engineering advice: https://www.ibm.com/granite/docs/
+- Learn about the latest Granite learning resources: https://ibm.biz/granite-learning-resources

context-attribution/{granite-4.0-micro/README.md → README.md}

@@ -2,24 +2,28 @@
 
 ## Model Summary
 
-**Context Attribution** is a purpose-built LoRA for [ibm-granite/granite-4.0-micro](https://huggingface.co/ibm-granite/granite-4.0-micro), to predict the sentences in the context that were most important for granite-4.0-micro to generate each sentence in its response or output. Here, context includes previous conversation turns as well as any documents provided to the granite-4-micro model. The context attribution LoRA thus helps to explain granite-4.0-micro's behavior, specifically how its probability of generating a certain response sentence is affected by different parts of the context.
+**Context Attribution** is a LoRA purpose-built for a base model to predict the sentences in the context that were most important for the base model to generate each sentence in its response or output. We provide LoRA adapters for the following base models: Granite 4.0 Micro, Granite 4.1 3B, Granite 4.1 8B, Granite 4.1 30B. Here, context includes previous conversation turns as well as any documents provided to the base model. The context attribution LoRA thus helps to explain the base model's behavior, specifically how its probability of generating a certain response sentence is affected by different parts of the context.
 
-The context attribution model takes the form of a [LoRA adapter](https://arxiv.org/abs/2106.09685). The adapter greatly improves granite-4.0-micro's ability to attribute to context (see Evaluation section below) while retaining other capabilities of the base model. The context attribution adapter identifies which context sentences actually influenced a model’s response, while IBM Granite’s [citation generation adapter](https://huggingface.co/ibm-granite/granitelib-rag-r1.0/blob/main/citations/README.md) highlights sentences that support the response regardless of whether the model used them.
+The context attribution model takes the form of a [LoRA adapter](https://arxiv.org/abs/2106.09685). The adapter greatly improves the base model's ability to attribute to context (see Evaluation section below) while retaining other capabilities of the base model. The context attribution adapter identifies which context sentences actually influenced the base model’s response, while IBM Granite’s [citation generation adapter](https://huggingface.co/ibm-granite/granitelib-rag-r1.0/blob/main/citations/README.md) highlights sentences that support the response regardless of whether the model used them.
 
 - **Developer:** IBM Research
 - **HF Collection:** [Granite Libraries](https://huggingface.co/collections/ibm-granite/granite-libraries)
 - **GitHub Repository:** https://github.com/ibm-granite
-- **Release Date:** March 18th, 2026
-- **Model Type:** LoRA adapter for [ibm-granite/granite-4.0-micro](https://huggingface.co/ibm-granite/granite-4.0-micro)
+- **Release Date:** May 5th, 2026
+- **Model Type:** LoRA adapters for:
+  - [ibm-granite/granite-4.0-micro](https://huggingface.co/ibm-granite/granite-4.0-micro)
+  - [ibm-granite/granite-4.1-3b](https://huggingface.co/ibm-granite/granite-4.1-3b)
+  - [ibm-granite/granite-4.1-8b](https://huggingface.co/ibm-granite/granite-4.1-8b)
+  - [ibm-granite/granite-4.1-30b](https://huggingface.co/ibm-granite/granite-4.1-30b)
 - **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
 - **Paper:** Context Attribution is trained to approximate the importance ranking of context sentences provided by the MExGen method in [[Monteiro Paes and Wei et al., ACL 2025] Multi-Level Explanations for Generative Language Models](https://aclanthology.org/2025.acl-long.1553/).
 
 
 ## Usage
 
-**Intended use:** Context Attribution is a context attribution adapter intended for IBM's [granite-4.0-micro](https://huggingface.co/ibm-granite/granite-4.0-micro) LLM. It enables the base model to accurately identify which sentences in the context (including previous conversation turns and documents) were most important to the model when generating each sentence in its response. The adapter thus helps to explain the base model's behavior, specifically how its probability of generating a certain response sentence is affected by parts of the context. This adapter is designed to be used as part of the Granite inference pipeline. It is intended to be called after the base model generates a response to provide a post hoc explanation.
+**Intended use:** Context Attribution is a context attribution adapter intended for a specific base model. It enables the base model to accurately identify which sentences in the context (including previous conversation turns and documents) were most important to the model when generating each sentence in its response. The adapter thus helps to explain the base model's behavior, specifically how its probability of generating a certain response sentence is affected by parts of the context. This adapter is designed to be used as part of the Granite inference pipeline. It is intended to be called after the base model generates a response to provide a post hoc explanation.
 
-The context attribution adapter is similar to the [citations adapter](https://huggingface.co/ibm-granite/granitelib-rag-r1.0/blob/main/citations/README.md) found in the [granitelib-rag-r1.0](https://huggingface.co/ibm-granite/granitelib-rag-r1.0) library, in that it uses a comparable input/output format and similiar training data. However, the two differ in the type of attribution they provide. Citation generation provides corroborative attribution, identifying document sentences that best support a response regardless of whether the model relied on them, making it model-agnostic. In contrast, the context attribution adapter provides contributive attribution, identifying the context sentences that actually influenced a specific model’s response. It also includes prior conversation turns (not just documents) and ranks context sentences by importance rather than listing them unordered.
+The context attribution adapter is similar to the [citations adapter](https://huggingface.co/ibm-granite/granitelib-rag-r1.0/blob/main/citations/README.md) found in the [granitelib-rag-r1.0](https://huggingface.co/ibm-granite/granitelib-rag-r1.0) library, in that it uses a comparable input/output format and similiar training data. However, the two differ in the type of attribution they provide. Citation generation provides corroborative attribution, identifying document sentences that best support a response regardless of whether the model relied on them, making it model-agnostic. In contrast, the context attribution adapter provides contributive attribution, identifying the context sentences that actually influenced the base model’s response. It also includes prior conversation turns (not just documents) and ranks context sentences by importance rather than listing them unordered.
 
 **Input Format** The context attribution LoRA expects the input to be processed as follows: 1) The last assistant response (the response to be attributed) should be split into sentences and the sentences numbered with tags as follows: `"<r0> sentence 0 <r1> sentence 1 ... "`. 2) The context (documents and previous conversation turns) should also be split into sentences and tagged as `"<c0> sentence 0 <c1> sentence 1 ... "`. The numbering of context sentences starts with the first document (if present) and continues as a single sequence through all the documents and then previous conversation turns, in that order. This input processing is performed automatically if the context attribution adapter is called using IBM's [mellea](https://github.com/generative-computing/mellea) package, as the Quickstart Example below shows. Alternatively, the user can insert sentence tags manually or by other means (also shown in the Quickstart Example below).
 
@@ -27,7 +31,7 @@ The context attribution adapter is similar to the [citations adapter](https://hu
 
 **Use Cases**
 * Human understanding: Context attribution helps to explain a base LLM by showing which parts of the context were most important to it in generating a certain response. Such views into the LLM's behavior may help users calibrate their trust in using the LLM in a particular domain.
-* RAG: The Granite 4.0 Micro - Context Attribution adapter is trained on a retrieval-augmented generation (RAG) dataset, specifically on document-grounded multi-turn question answering. The Evaluation section below shows that it also performs well when the use case is document summarization. It is thus especially suitable for RAG scenarios where users may wish to attribute LLM responses to a sizable context.
+* RAG: The Context Attribution adapter is trained on a retrieval-augmented generation (RAG) dataset, specifically on document-grounded multi-turn question answering. The Evaluation section below shows that it also performs well when the use case is document summarization. It is thus especially suitable for RAG scenarios where users may wish to attribute LLM responses to a sizable context.
 * Context reliance assessment: Context attribution can also be used to assess whether an LLM makes use of its context at all for a specific response. Low attribution scores to all context sentences might mean one of several things: 1) The response sentence is strongly implied by previous response sentences and does not need to refer to context; 2) the response comes primarily from the LLM's parametric knowledge; 3) the response is hallucinated. As such, determining the main reason for context non-reliance would require further investigation.
 
 
@@ -51,36 +55,36 @@ messages = [
 question = "What was the concept behind the formation of the Plastic Ono Band?"
 messages.append({"role": "user", "content": question})
 
-# Response pre-generated by granite-4.0-micro, could also generate a fresh response
-response = "The Plastic Ono Band was formed by John Lennon and Yoko Ono in 1969 as a collaborative vehicle for their artistic and personal projects. They decided to credit all their future efforts to this conceptual and collaborative group after their marriage in 1969."
+# Response pre-generated by granite-4.1-3b, could also generate a fresh response
+response = "The Plastic Ono Band was formed by John Lennon and Yoko Ono in 1969 as a conceptual and collaborative vehicle for their collaborative and solo projects. Following their marriage in 1969, they decided that all of their future endeavors would be credited to this vehicle. The band was intended to serve as a platform for their artistic and personal expressions, featuring a rotating lineup of musicians over the years."
 ```
 
 **Usage with Mellea**
 ```python
 import json    # only for display
 
-from mellea.backends.huggingface import LocalHFBackend    # LLM inference
+from mellea import model_ids, start_backend
 from mellea.stdlib.components import Document, Message    # constructing the conversation
 from mellea.stdlib.components.intrinsic.core import find_context_attributions    # intrinsic function for calling context attribution
-from mellea.stdlib.context import ChatContext    # constructing the conversation
 
-backend = LocalHFBackend(model_id="ibm-granite/granite-4.0-micro")
+ctx, backend = start_backend(
+    "hf", model_id=model_ids.IBM_GRANITE_4_1_3B, context_type="chat"
+)
 
 # Previous conversation turns + current question
-context = ChatContext()
 for message in messages:
-    context.add(Message(message["role"], message["content"]))
+    ctx = ctx.add(Message(message["role"], message["content"]))
 # Documents
 documents_mellea = [Document(**document) for document in documents]
 
 # Call context attribution intrinsic function
-result = find_context_attributions(response, documents_mellea, context, backend)
+result = find_context_attributions(response, documents_mellea, ctx, backend)
 print(f"Result of context attribution intrinsic:\n{json.dumps(result, indent=2)}")
 
-# Prints an array of JSON objects where each object shows a response sentence, 
+# Prints an array of JSON objects where each object shows a response sentence,
 # a context sentence to which the response sentence is attributed (in order of decreasing importance),
 # and other fields such as the document ID where the context sentence is located.
-# (You may get slightly different orderings of context sentences for response sentence 1, depending on your hardware or environment.)
+# (You may get slightly different orderings of context sentences, depending on your hardware or environment.)
 ```
 
 **Usage without Mellea**
@@ -89,9 +93,9 @@ import torch
 from transformers import AutoTokenizer, AutoModelForCausalLM
 from peft import PeftModel
 
-BASE_NAME = "ibm-granite/granite-4.0-micro"
+BASE_NAME = "ibm-granite/granite-4.1-3b"
 ADAPTER_REPO = "ibm-granite/granitelib-core-r1.0"
-ADAPTER_SUBFOLDER = "context-attribution/granite-4.0-micro/lora"
+ADAPTER_SUBFOLDER = "context-attribution/granite-4.1-3b/lora"
 device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
 
 # Load context attribution model (base model + LoRA)
@@ -103,7 +107,7 @@ model_ca = PeftModel.from_pretrained(
 )
 
 # The context attribution adapter expects the last assistant response and context (documents
-# + previous conversation turns) to be split into sentences and the sentences to be tagged with numbers. 
+# + previous conversation turns) to be split into sentences and the sentences to be tagged with numbers.
 # In this snippet, we assume that this input processing has been done as below.
 documents_rewritten = [
     {"doc_id": "0", "text": "<c0> The Plastic Ono Band is a band formed by John Lennon and Yoko Ono in 1969 as a vehicle for their collaborative and solo projects. <c1> Lennon and Ono had begun a personal and artistic relationship in 1968, collaborating on several experimental releases. <c2> Following their marriage in 1969, they decided that all of their future endeavours would be credited to a conceptual and collaborative vehicle, Plastic Ono Band. <c3> The band would go on to feature a rotating lineup of many musicians, including Eric Clapton, Klaus Voormann, Alan White, Billy Preston, Jim Keltner, Delaney & Bonnie and Friends, and Lennon's former Beatles bandmates George Harrison and Ringo Starr. <c4> Lennon and Ono left the UK to settle in New York City during the fall of 1971. <c5> In Greenwich Village, the couple became more politically active and began writing protest songs. <c6> These songs became the basis for their next album, Some Time in New York City. <c7> As backing, they enlisted the help of New York band Elephant's Memory, consisting of guitarist Wayne 'Tex' Gabriel, bassist Gary Van Scyoc, saxophonist Stan Bronstein, keyboardist Adam Ippolito, keyboardist John La Boosca, and drummer Richard Frank, Jr. <c8> Phil Spector produced, and Jim Keltner also played on the album. <c9> The album was released on 12 June 1972, credited to \"John & Yoko/Plastic Ono Band with Elephant's Memory plus Invisible Strings\". <c10> Some Time in New York City included a second disc, entitled Live Jam, which included the recordings from the 1969 Peace for Christmas concert and the 1971 performance with Frank Zappa. <c11> Ono and Lennon continued their work with Elephant's Memory throughout 1972, performing as the Plastic Ono Elephant's Memory Band (which also included Jim Keltner). <c12> On 30 August, they performed a pair of benefit concerts at Madison Square Garden. <c13> The benefit, entitled \"One to One\", was organised by Geraldo Rivera to raise money for children with mental challenges. <c14> By this time, La Boosca had departed the band, and the concert saw the addition of John Ward on bass. <c15> The concert was filmed and recorded, later released in February 1986 as the album Live In New York City. <c16> They also performed at the Jerry Lewis MDA Labor Day Telethon. <c17> The last collaboration of the Plastic Ono Elephant's Memory Band was Ono's double album Approximately Infinite Universe. <c18> It was recorded throughout the fall of 1972, and was released in January 1973."},
@@ -114,7 +118,7 @@ messages_rewritten = [
     {"role": "user", "content": "<c70> Who were the members of The Metal Ono Band, which was formed by Yoko Ono in 1976 to explore her interest in heavy metal music?"},
     {"role": "assistant", "content": "<c71> I'm sorry, but I don't have the data to answer that specific question."},
     {"role": "user", "content": "<c72> What was the concept behind the formation of the Plastic Ono Band?"},
-    {"role": "assistant", "content": "<r0> The Plastic Ono Band was formed by John Lennon and Yoko Ono in 1969 as a collaborative vehicle for their artistic and personal projects. <r1> They decided to credit all their future efforts to this conceptual and collaborative group after their marriage in 1969."},
+    {"role": "assistant", "content": "<r0> The Plastic Ono Band was formed by John Lennon and Yoko Ono in 1969 as a conceptual and collaborative vehicle for their collaborative and solo projects. <r1> Following their marriage in 1969, they decided that all of their future endeavors would be credited to this vehicle. <r2> The band was intended to serve as a platform for their artistic and personal expressions, featuring a rotating lineup of musicians over the years."},
     {"role": "user", "content": "You provided the last assistant response above based on context, which may include documents and/or previous conversation turns. Your response is divided into sentences, numbered in the format <r0> sentence 0 <r1> sentence 1 ... Sentences in the context are also numbered: <c0> sentence 0 <c1> sentence 1 ... For each response sentence, please list the context sentences that were most important for you to generate the response sentence. Provide your answer in JSON format, as an array of JSON objects, where each object has two members: \"r\" with the response sentence number as the value, and \"c\" with an array of context sentence numbers as the value. An example of such an array of objects is [{\"r\": 0, \"c\": [3, 1, 4]}, {\"r\": 1, \"c\": [1, 5]}]. List the context sentences in order from most important to least important. Ensure that you include an object for each response sentence, even if the corresponding array of context sentence numbers is empty. Answer with only the JSON and do not explain.\n"},
 ]
 
@@ -125,55 +129,121 @@ output_text = tokenizer.decode(output_tokens[0, inputs["input_ids"].shape[1]:],
 
 print(output_text)
 
-# [{"r": 0, "c": [0, 19]}, {"r": 1, "c": [2, 0, 1, 3, 19, 72, 70, 71, 4, 21]}]
-# This raw output from the adapter means that response sentence 0 is attributed to context sentences 0 and 19, 
-# and response sentence 1 is attributed to many context sentences in decreasing order of importance, starting with sentence 2.
-# (You may get slightly different orderings of context sentences for response sentence 1, depending on your hardware or environment.) 
+# [{"r": 0, "c": [0, 2, 71, 3, 7, 24, 34, 31, 22, 41]}, {"r": 1, "c": [2, 0, 3, 7, 24, 34, 31, 22, 41, 63]}, {"r": 2, "c": [3, 2, 7, 24, 34, 31, 22, 41, 63, 39]}]
+# This raw output from the adapter means that response sentences 0, 1, and 2 are each attributed to many context sentences in decreasing order of importance.
+# The most important context sentences for response sentences 0, 1, 2 are 0, 2, 3 respectively.
+# (You may get slightly different orderings of context sentences, depending on your hardware or environment.)
 ```
 
-## Evaluation 
+## Evaluation
 
-The context attribution adapter was evaluated on four datasets: MD2D-QUAC, ELI5, CNN/Daily Mail (CNN/DM), and XSum. The first two are document-grounded, multi-turn question answering datasets and the last two are summarization datasets. MD2D-QUAC evaluates the adpater on a held-out test split of the dataset used for training (please see "Training Data" below for more details), ELI5 evaluates generalization to a different dataset within the same task, and CNN/DM and XSum evaluate generalization to a different task.
+The context attribution adapters were evaluated on four datasets: MD2D-QUAC, ELI5, CNN/Daily Mail (CNN/DM), and XSum. The first two are document-grounded, multi-turn question answering datasets and the last two are summarization datasets. MD2D-QUAC evaluates the adpaters on a held-out test split of the dataset used for training (please see "Training Data" below for more details), ELI5 evaluates generalization to a different dataset within the same task, and CNN/DM and XSum evaluate generalization to a different task.
 
 The evaluation metric is the area under the perturbation curve (AUPC), a standard metric for evaluating the faithfulness of a feature-attribution-like explanation to the model being explained. Please see the [MExGen paper](https://aclanthology.org/2025.acl-long.1553/) for more details on AUPC. Here, the perturbation curves are first multiplied by a linearly decreasing weight function, which decreases from 1 at 0% perturbation to 0 at 20% perturbation, yielding the weighted area under the perturbation curve (WAUPC).
 
+Below we show tables of WAUPC values for each base model for which a context attribution LoRA is available. Since the context attribution task is model-dependent (i.e., to explain the base model's behavior), values may not be comparable across tables. Values are normalized and can be compared within a table as described below.
+
+#### Granite 4.0 Micro
+
 | method                     |   MD2D-QUAC |   ELI5 |   CNN/DM |   XSum |   average |
 |:---------------------------|------------:|-------:|---------:|-------:|----------:|
 | LOO *(skyline)*                        |      1.0000 | 1.0000 |   1.0000 | 1.0000 |    1.0000 |
 | LOO thresholded *(realistic skyline)*           |      0.9780 | 0.9812 |   0.9976 | 0.9983 |    0.9888 |
-| prompt 0-shot              |      0.1476 | 0.1657 |   0.1629 | 0.2369 |    0.1783 |
-| prompt 1-shot              |      0.1982 | 0.2171 |   0.1693 | 0.2014 |    0.1965 |
+| prompt granite-4.0-micro 0-shot              |      0.1476 | 0.1657 |   0.1629 | 0.2369 |    0.1783 |
+| prompt granite-4.0-micro 1-shot              |      0.1982 | 0.2171 |   0.1693 | 0.2014 |    0.1965 |
 | prompt GPT-OSS-120B 0-shot |      0.8619 | 0.8063 |   0.8735 | 0.6679 |    0.8024 |
 | prompt GPT-OSS-120B 1-shot |      0.8779 | 0.8498 |   0.8996 | 0.8642 |    0.8729 |
 | **context attribution LoRA**          |      **0.9360** | **0.9045** |   **0.9158** | **0.9223** |    **0.9197** |
 
-The first row of the table shows the WAUPC for the leave-one-out (LOO) variant of the [MExGen method](https://aclanthology.org/2025.acl-long.1553/), which the adapter aims to approximate. The second row corresponds to a thresholded version of LOO, which is the one that actually generates training data for the adapter. LOO and thresholded LOO thus represent "skyline" and "realistic skyline" methods that the adapter ideally would match. The WAUPC values in the table have been normalized by dividing by the WAUPC of LOO. As seen in the last row of the table, the adapter does well in approximating LOO, attaining at least 90% of its WAUPC on all datasets and 92.0% averaged across datasets.
-
-The table compares the context attribution adapter to baselines that prompt an LLM to perform the context attribution task. These include prompting the granite-4.0-micro base model (labelled simply as "prompt" in the table), with and without providing an example (0-shot/1-shot), and prompting GPT-OSS-120B, also with 0-shot or 1-shot. (2-shot and 3-shot prompting were also evaluated but did not improve upon 1-shot, so are omitted from this table for brevity.) It is clear that the context attribution adapter greatly improves granite-4.0-micro's ability to attribute its own responses to context. The adapter also outperforms GPT-OSS-120B, an LLM with around 40X the number of parameters (the LoRA adapter contributes a negligible number of parameters to the granite-4.0-micro base model).
+#### Granite 4.1 3B
+
+| method | MD2D-QUAC | ELI5 | CNN/DM | XSum | average |
+|:---|---:|---:|---:|---:|---:|
+| LOO *(skyline)* | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
+| LOO thresholded *(realistic skyline)* | 0.9908 | 0.9988 | 1.0004 | 1.0007 | 0.9977 |
+| prompt granite-4.1-3b 0-shot | 0.4058 | 0.3918 | 0.2773 | 0.2808 | 0.3389 |
+| prompt granite-4.1-3b 2-shot | 0.5032 | 0.4931 | 0.3395 | 0.3597 | 0.4239 |
+| prompt GPT-OSS-120B 0-shot | 0.9040 | 0.8615 | 0.8911 | 0.7155 | 0.8430 |
+| prompt GPT-OSS-120B 2-shot | 0.8909 | 0.8754 | 0.9050 | 0.8775 | 0.8872 |
+| **context attribution LoRA** | **0.9053** | **0.8996** | **0.9310** | **0.9421** | **0.9195** |
+
+#### Granite 4.1 8B
+
+| method | MD2D-QUAC | ELI5 | CNN/DM | XSum | average |
+|:---|---:|---:|---:|---:|---:|
+| LOO *(skyline)* | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
+| LOO thresholded *(realistic skyline)* | 0.9854 | 0.9978 | 1.0000 | 0.9999 | 0.9958 |
+| prompt granite-4.1-8b 0-shot | 0.3938 | 0.2528 | 0.2109 | 0.2179 | 0.2688 |
+| prompt granite-4.1-8b 2-shot | 0.5186 | 0.4071 | 0.3238 | 0.2944 | 0.3860 |
+| prompt GPT-OSS-120B 0-shot | 0.8681 | 0.8005 | 0.8762 | 0.7009 | 0.8114 |
+| prompt GPT-OSS-120B 2-shot | 0.8648 | 0.8215 | 0.8938 | 0.8720 | 0.8631 |
+| **context attribution LoRA** | **0.9408** | **0.9278** | **0.9756** | **0.9808** | **0.9563** |
+
+#### Granite 4.1 30B
+
+| method | MD2D-QUAC | ELI5 | CNN/DM | XSum | average |
+|:---|---:|---:|---:|---:|---:|
+| LOO *(skyline)* | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
+| LOO thresholded *(realistic skyline)* | 0.9877 | 0.9994 | 1.0003 | 1.0001 | 0.9969 |
+| prompt granite-4.1-30b 0-shot | 0.6953 | 0.5896 | 0.6709 | 0.6452 | 0.6502 |
+| prompt granite-4.1-30b 3-shot | 0.8089 | 0.6312 | 0.7712 | 0.7607 | 0.7430 |
+| prompt GPT-OSS-120B 0-shot | 0.9098 | 0.7956 | 0.8706 | 0.6993 | 0.8188 |
+| prompt GPT-OSS-120B 1-shot | 0.8970 | 0.7925 | 0.8789 | 0.8478 | 0.8540 |
+| **context attribution LoRA** | **0.9188** | **0.8647** | **0.9123** | **0.9163** | **0.9030** |
+
+The first row of each table shows the WAUPC for the leave-one-out (LOO) variant of the [MExGen method](https://aclanthology.org/2025.acl-long.1553/), which the adapters aim to approximate. The second row corresponds to a thresholded version of LOO, which is the one that actually generates training data for the adapters. LOO and thresholded LOO thus represent "skyline" and "realistic skyline" methods that the adapters ideally would match. The WAUPC values in each table have been normalized by dividing by the WAUPC of LOO (for the particular base model). As seen in the last row of each table, the adapters do well in approximating LOO, attaining at least 90% of its WAUPC in all but one case and 90.3%-95.6% averaged across datasets.
+
+The tables compare the context attribution adapters to baselines that prompt an LLM to perform the context attribution task. These include prompting the base model, with and without providing examples (0-shot/1-shot or more), and prompting GPT-OSS-120B, also with 0-shot or more. 1-shot, 2-shot, and 3-shot prompting were evaluated and the best-performing number of shots is shown for each prompted model. It is clear that the context attribution adapter improves the base model's ability to attribute its own responses to context. The adapter also outperforms GPT-OSS-120B, an LLM with around 4-40X the number of parameters (the LoRA adapter contributes a negligible number of parameters to the base model).
 
 ## Training Details
 
-Granite 4.0 Micro Context Attribution is a LoRA adapter trained to approximate the importance ranking of context sentences provided by the MExGen method in [[Monteiro Paes and Wei et al., ACL 2025] Multi-Level Explanations for Generative Language Models](https://aclanthology.org/2025.acl-long.1553/).
+Context Attribution is a LoRA adapter trained to approximate the importance ranking of context sentences provided by the MExGen method in [[Monteiro Paes and Wei et al., ACL 2025] Multi-Level Explanations for Generative Language Models](https://aclanthology.org/2025.acl-long.1553/).
 
-**Training Data:** The adapter was trained on a mixture of two datasets, [MultiDoc2Dial](https://huggingface.co/datasets/IBM/multidoc2dial) and [QUAC](https://huggingface.co/datasets/allenai/quac), which are both for document-grounded, multi-turn question answering. This mixture of datasets, referred to as **MD2D-QUAC**, was also used to train IBM Granite's [citation generation adapter](https://huggingface.co/ibm-granite/granite-lib-rag-r1.0/blob/main/citations/README.md). 2053 question-answer conversation rounds were used for training and 1024 for validation. These numbers of instances were deliberately chosen to be modest to demonstrate that a well-performing adapter can be trained using this limited amount of data.
+**Training Data:** The adapters were trained on a mixture of two datasets, [MultiDoc2Dial](https://huggingface.co/datasets/IBM/multidoc2dial) and [QUAC](https://huggingface.co/datasets/allenai/quac), which are both for document-grounded, multi-turn question answering. This mixture of datasets, referred to as **MD2D-QUAC**, was also used to train IBM Granite's [citation generation adapter](https://huggingface.co/ibm-granite/granite-lib-rag-r1.0/blob/main/citations/README.md). 4101 question-answer conversation rounds were used for training and 1024 for validation. These numbers of instances were deliberately chosen to be modest to demonstrate that a well-performing adapter can be trained using this limited amount of data.
 
-The MultiDoc2Dial and QUAC datasets consist of sets of grounding documents and multi-turn question-answering conversations based on the set of documents. For training the context attribution adapter, each conversation round (consisting of one user question followed by an assistant response), together with any conversation rounds that precede it, was treated as a separate "instance". The context for each instance includes the documents for the conversation as well as all conversation rounds except the last round, which is treated as the current round for the instance. granite-4.0-micro was called to re-generate a response to the question in the current round (since the purpose of the adapter is to attribute granite-4.0-micro's responses). The leave-one-out (LOO) variant of [MExGen](https://aclanthology.org/2025.acl-long.1553/) was then used to attribute each response sentence to context sentences, yielding "gold" attribution scores. LOO is thus treated as the "skyline" method in the Evaluation section below. The LOO attribution scores were thresholded to produce a controlled-length list of context sentences, specifically a list of context sentence numbers in decreasing order of importance. The context attribution adapter was trained to reproduce these lists for all response sentences, given the context, response, and an instruction as input.
+The MultiDoc2Dial and QUAC datasets consist of sets of grounding documents and multi-turn question-answering conversations based on the set of documents. For training the context attribution adapters, each conversation round (consisting of one user question followed by an assistant response), together with any conversation rounds that precede it, was treated as a separate "instance". The context for each instance includes the documents for the conversation as well as all conversation rounds except the last round, which is treated as the current round for the instance. The base model was called to re-generate a response to the question in the current round (since the purpose of the adapter is to attribute the base model's responses). The leave-one-out (LOO) variant of [MExGen](https://aclanthology.org/2025.acl-long.1553/) was then used to attribute each response sentence to context sentences, yielding "gold" attribution scores. LOO is thus treated as the "skyline" method in the Evaluation section. The LOO attribution scores were thresholded to produce a controlled-length list of context sentences, specifically a list of context sentence numbers in decreasing order of importance. The context attribution adapters were trained to reproduce these lists for all response sentences, given the context, response, and an instruction as input.
 
 ### Adapter Configurations
 
-| Parameter | LoRA |
-|-----------|------|
-| Base model | ibm-granite/granite-4.0-micro |
-| LoRA rank (r) | 16 |
-| LoRA alpha | 32 |
-| Target modules | q_proj, k_proj, v_proj |
-| Output format | e.g. `[{"r": 0, "c": [3, 1, 4]}, ... ]` where `"c"` lists most important context sentences for response sentence `"r"`|
-| Max completion tokens | 4096 |
-| KV cache | Supported |
+| Property | Granite 4.0 Micro | Granite 4.1 3B | Granite 4.1 8B | Granite 4.1 30B |
+|-----------|------|------|------|------|
+| Base model | ibm-granite/granite-4.0-micro | ibm-granite/granite-4.1-3b | ibm-granite/granite-4.1-8b | ibm-granite/granite-4.1-30b |
+| PEFT type | LoRA | LoRA | LoRA | LoRA |
+| Rank (r) | 16 | 16 | 16 | 32 |
+| Alpha | 32 | 32 | 32 | 64 |
+| Target modules | q_proj, k_proj, v_proj | q_proj, k_proj, v_proj | q_proj, k_proj, v_proj | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
+| Max completion tokens | 4096 | 4096 | 4096 | 4096 |
+| KV cache | Supported | Supported | Supported | Supported |
+<!-- | Output format | e.g. `[{"r": 0, "c": [3, 1, 4]}, ... ]` where `"c"` lists most important context sentences for response sentence `"r"`| -->
 
-**Infrastructure:** The Granite 4.0 Micro Context Attribution LoRA adapter was trained on a single NVIDIA A100-80GB GPU.
+**Infrastructure:** The Context Attribution LoRA adapters were trained on a single NVIDIA H100-80GB GPU.
 
-**Ethical Considerations:** The context attribution adapter is designed specifically for Granite 4.0 Micro and was trained on its behavior. While it may be applied to other LLMs, it has not been validated for them. In addition, the context attributions may not always align with human judgments of which context sentences should matter.
+**Ethical Considerations:** Each context attribution adapter is designed specifically for its respective base model and was trained on its behavior. While it may be applied to other LLMs, it has not been validated for them. In addition, the context attributions may not always align with human judgments of which context sentences should matter.
+
+## Citation
+If you find these adapters useful, please cite the following work:
+```
+@inproceedings{monteiro-paes-wei-2025-mexgen,
+    title = "Multi-Level Explanations for Generative Language Models",
+    author = "Monteiro Paes, Lucas  and
+      Wei, Dennis  and
+      Do, Hyo Jin  and
+      Strobelt, Hendrik  and
+      Luss, Ronny  and
+      Dhurandhar, Amit  and
+      Nagireddy, Manish  and
+      Natesan Ramamurthy, Karthikeyan  and
+      Sattigeri, Prasanna  and
+      Geyer, Werner  and
+      Ghosh, Soumya",
+    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
+    month = jul,
+    year = "2025",
+    address = "Vienna, Austria",
+    url = "https://aclanthology.org/2025.acl-long.1553/",
+    pages = "32291--32317",
+}
+```
 
 ## Resources
 

requirement-check/README.md

@@ -0,0 +1,257 @@
+---
+license: apache-2.0
+language:
+- en
+pipeline_tag: text-generation
+library_name:
+- peft
+- transformers
+base_model:
+- ibm-granite/granite-4.0-micro
+- ibm-granite/granite-4.1-3b
+- ibm-granite/granite-4.1-8b
+- ibm-granite/granite-4.1-30b
+---
+
+# Requirement Checking
+
+## Model Summary
+
+**Requirement Checking** is a family of LoRA adapters fine-tuned for binary
+requirement satisfaction classification. The model takes as input a user prompt,
+an assistant response, and a set of user-specified requirements, and classifies
+whether the assistant's response satisfies the given constraints. The model
+outputs a JSON object `{"score": "yes"}` or `{"score": "no"}` indicating whether
+the constraints are satisfied. We provide requirement checking capabilities
+implemented as LoRA adapters trained over Granite-4.0-micro, Granite-4.1-3B, Granite-4.1-8B, and
+Granite-4.1-30B.
+
+- **Developer:** IBM Research
+- **HF Collection:** [Granite Libraries](https://huggingface.co/collections/ibm-granite/granite-libraries)
+- **GitHub Repository:** https://github.com/ibm-granite
+- **Last Updated:** May 5th, 2026
+- **Model type:** LoRA/aLoRA adapters for:
+  - [ibm-granite/granite-4.0-micro](https://huggingface.co/ibm-granite/granite-4.0-micro)
+  - [ibm-granite/granite-4.1-3b](https://huggingface.co/ibm-granite/granite-4.1-3b)
+  - [ibm-granite/granite-4.1-8b](https://huggingface.co/ibm-granite/granite-4.1-8b)
+  - [ibm-granite/granite-4.1-30b](https://huggingface.co/ibm-granite/granite-4.1-30b)
+- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
+
+## Intended Use
+
+The Requirement Checking adapter is designed for evaluating whether LLM-generated
+responses satisfy user-specified constraints. Key use cases include:
+
+- **Instruction-following evaluation**: Assessing whether a model's output
+  adheres to specific formatting, content, or structural requirements provided
+  in the prompt.
+- **Constraint satisfaction checking**: Verifying that generated text meets
+  multiple simultaneous constraints (e.g., length limits, style requirements,
+  content inclusion/exclusion).
+- **Quality assurance pipelines**: Automated checking of LLM outputs against
+  predefined acceptance criteria.
+
+**Adapter input**: The input to the requirement checking adapter is an
+OpenAI-compatible chat completion request containing a user message, an assistant
+response, and a follow-up user turn with the requirements to evaluate wrapped in
+a `<requirements>` tag.
+
+**Adapter output**: `{"score": "yes"}` if the constraints are satisfied, `{"score": "no"}` otherwise.
+
+**Scoring:** When called via Mellea, the binary output is converted to a float between 0.0 and 1.0 using the model's token-level probability of "yes". This provides a continuous confidence score rather than a hard binary decision. The threshold for converting this score to a binary outcome is left to the user and may need to be tuned for your use-case.
+
+## Usage Examples
+
+### via Mellea
+
+The recommended way to call this adapter is through the [Mellea](https://mellea.ai) framework:
+
+```python
+from mellea import model_ids, start_backend
+from mellea.stdlib.components import Message
+from mellea.stdlib.components.intrinsic import core
+
+ctx, backend = start_backend(
+    "hf", model_id=model_ids.IBM_GRANITE_4_1_3B, context_type="chat"
+)
+
+ctx = (
+    ctx.add(Message("user", "Invite for an IBM office party."))
+       .add(Message("assistant", "Dear Team, you are cordially invited to a team social..."))
+)
+
+result = core.requirement_check(ctx, backend, "Use a professional tone.")
+print(f"Requirements satisfied: {result}")  # float between 0.0 and 1.0
+```
+
+For more comprehensive examples and detailed documentation, refer to the [Mellea intrinsics examples](https://github.com/generative-computing/mellea/tree/main/docs/examples/intrinsics).
+
+### via HF+PEFT
+
+```python
+import json
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+from peft import PeftModel
+
+base_model_id = "ibm-granite/granite-4.1-3b"
+adapter_repo = "ibm-granite/granitelib-core-r1.0"
+adapter_subfolder = "requirement-check/granite-4.1-3b/lora"
+
+# Load base model and LoRA adapter
+tokenizer = AutoTokenizer.from_pretrained(base_model_id)
+base_model = AutoModelForCausalLM.from_pretrained(
+    base_model_id, torch_dtype=torch.bfloat16, device_map="auto"
+)
+model = PeftModel.from_pretrained(base_model, adapter_repo, subfolder=adapter_subfolder)
+model.eval()
+
+# Define the user prompt, assistant response, and constraints to check
+user_text = "Invite for an IBM office party."
+response_text = """
+Dear Team,
+
+To celebrate our recent successes and take a well-deserved moment to recharge,
+you are cordially invited to a team social. Please join us for an evening of
+live music, appetizers, and drinks as we recognize our collective wins.
+
+Event Details
+* **Date:** Saturday, April 25, 2026
+* **Time:** 6:00 PM
+* **Location:** Ryan's Bar, Chelsea, NY
+* **Highlights:** Live entertainment and refreshments
+
+RSVP
+To ensure we have an accurate headcount for catering, please confirm your
+attendance by **Friday, April 10, 2026**.
+
+We look forward to seeing everyone there and celebrating our hard work together.
+
+**Best regards,**
+[Your Name/Management Team]
+"""
+
+constraints = "Use a professional tone."
+
+# Build the evaluation prompt
+evaluation_prompt = (
+    "Please verify if the assistant's generation satisfies the user's "
+    "requirements or not and reply with a binary label accordingly. "
+    'Respond with a json {"score": "yes"} if the constraints are '
+    'satisfied or respond with {"score": "no"} if the constraints are not '
+    "satisfied."
+)
+
+messages = [
+    {"role": "user", "content": user_text},
+    {"role": "assistant", "content": response_text},
+    {"role": "user", "content": f"<requirements> {constraints}\n{evaluation_prompt}"},
+]
+
+prompt = tokenizer.apply_chat_template(
+    messages, tokenize=False, add_generation_prompt=True
+)
+
+# Generate
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+input_len = inputs["input_ids"].shape[1]
+
+stop_token_id = tokenizer.encode("}", add_special_tokens=False)[0]
+
+with torch.no_grad():
+    output = model.generate(
+        **inputs,
+        max_new_tokens=20,
+        do_sample=False,
+        eos_token_id=[tokenizer.eos_token_id, stop_token_id],
+    )
+
+response = tokenizer.decode(output[0][input_len:], skip_special_tokens=True).strip()
+
+brace_idx = response.find("}")
+if brace_idx != -1:
+    response = response[:brace_idx + 1]
+
+print(f"Response: {response}")  # {"score": "yes"}
+
+result = json.loads(response)
+print(f"Constraints satisfied: {result['score']}")  # yes
+```
+
+## Evaluation
+
+We evaluated the adapters on binary requirement satisfaction classification
+using four benchmarks: [HelpSteer3](https://huggingface.co/datasets/nvidia/HelpSteer3),
+[IFEval Multi-Constraint](https://arxiv.org/abs/2311.07911), and
+[InfoBench](https://arxiv.org/abs/2401.03601) (GPT-4-annotated and
+human-annotated splits).
+
+### Results
+
+The table below reports Balanced Accuracy across all benchmarks for prompting
+baselines and trained adapters (LoRA and aLoRA) at each model size.
+
+| Model | HelpSteer3 | IFEval | InfoBench-GPT4 | InfoBench-Human |
+|---|---|---|---|---|
+| Granite-4.1-3B (prompting) | 0.5090 | 0.5118 | 0.6237 | 0.5866 |
+| Granite-4.1-8B (prompting) | 0.5162 | 0.5689 | 0.6564 | 0.6018 |
+| Granite-4.1-30B (prompting) | 0.5234 | 0.6237 | 0.6863 | 0.6236 |
+| Granite-4.0-micro LoRA | 0.7434 | 0.8403 | 0.6950 | 0.6799 |
+| Granite-4.1-3B LoRA | 0.6726 | 0.8153 | 0.6960 | 0.6728 |
+| Granite-4.1-3B aLoRA | 0.6957 | 0.8425 | 0.7150 | 0.6475 |
+| Granite-4.1-8B LoRA | 0.7051 | 0.8360 | 0.7373 | 0.6546 |
+| Granite-4.1-8B aLoRA | 0.6941 | 0.8554 | 0.7119 | 0.6761 |
+| Granite-4.1-30B LoRA | 0.7119 | 0.8406 | 0.7512 | 0.7038 |
+| Granite-4.1-30B aLoRA | 0.7178 | 0.8492 | 0.7432 | 0.7052 |
+
+The evaluations on HelpSteer3 and IFEval above are on the validation set and a heldout set, respectively.
+
+## Training Details
+
+### Training Data
+
+The Requirement Checking adapter is fine-tuned using a combination of
+instruction-following evaluation data. Training data includes samples with
+explicit user requirements and constraints paired with assistant responses,
+annotated for whether the constraints are satisfied.
+
+The two sources of training data are:
+
+1. [IF-RLVR training data](https://huggingface.co/datasets/allenai/IF_multi_constraints_upto5),
+   with up to 5 constraints per instruction. Each prompt (with constraints) was
+   passed through [Mixtral 8x22b Instruct](https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1).
+   Then, the [prompt, constraints, response] are all passed through a
+   [programmatic evaluation pipeline](https://github.com/allenai/IFBench?tab=readme-ov-file)
+   which provides yes/no labels corresponding to whether the constraints were
+   followed or not.
+2. [Helpsteer3-Preference](https://huggingface.co/datasets/nvidia/HelpSteer3).
+   The original dataset consists of [prompt, response1, response2, feedback].
+   The `feedback` column contains individual rankings (a score from 1-5) of each
+   response from multiple annotators. We map scores 4-5 to a label of yes and
+   scores 1-3 to a label of no, take the average of the annotators' scores and
+   map each of the two responses to a binary yes/no label accordingly.
+
+### Training Hyperparameters
+
+The adapters were fine-tuned using PEFT under the following regime: rank = 64
+(LoRA) / 16 (aLoRA), alpha = 64, learning rates between 5e-5 and 5e-4 (tuned
+per base model).
+
+### Adapter Details
+
+| Property | Granite-4.0-micro | Granite-4.1-3B | Granite-4.1-8B | Granite-4.1-30B |
+|---|---|---|---|---|
+| **PEFT Type** | LoRA | LoRA/aLoRA | LoRA/aLoRA | LoRA/aLoRA |
+| **Rank (r) - LoRA** | 64 | 64 | 64 | 64 |
+| **Rank (r) - aLoRA** | — | 16 | 16 | 16 |
+| **Alpha** | 64 | 64 | 64 | 64 |
+| **Target Modules** | q_proj, k_proj, v_proj, o_proj, input_linear, output_linear | q_proj, k_proj, v_proj, o_proj | q_proj, k_proj, v_proj, o_proj | q_proj, k_proj, v_proj, o_proj |
+| **Invocation Token (aLoRA)** | — | `<requirements>` | `<requirements>` | `<requirements>` |
+
+**Infrastructure:** Training was completed using H100 GPUs.
+
+**Ethical Considerations:** The requirement checking adapter outputs are not guaranteed to be factually accurate or complete. All outputs should be independently validated before use in decision-making or downstream applications. The model has been trained and evaluated on English data only. The model must only be used in the prescribed evaluation mode, outputting JSON responses (`{"score": "yes"/"no"}`) based on the specified prompt template.
+
+**Resources**
+- Learn about the latest updates with Granite: https://www.ibm.com/granite
+- Get started with tutorials and best practices: https://www.ibm.com/granite/docs/

requirement-check/granite-4.0-micro/README.md

@@ -1,267 +0,0 @@
----
-license: apache-2.0
-language:
-- en
-pipeline_tag: text-generation
-library_name: transformers
-base_model: ibm-granite/granite-4.0-micro
-tags:
-- granite
-- guardian
-- requirement-checking
-- instruction-following
-- lora
-- peft
----
-# Requirement Checker (LoRA)
-
-**Model Summary:** Requirement Checker is a lightweight LoRA adapter that brings requirement checking capabilities to the [ibm-granite/granite-4.0-micro](https://huggingface.co/ibm-granite/granite-4.0-micro) base model. The adapter is trained to judge whether an assistant's generation satisfies a set of user-specified requirements, outputting a binary yes/no assessment. The model outputs a JSON object `{"score": "yes"}` or `{"score": "no"}` indicating whether the given constraints are satisfied.
-
-- **Developers:** IBM Research
-- **HF Collection:** [Granite Libraries](https://huggingface.co/collections/ibm-granite/granite-libraries)
-- **Github Repository:** [ibm-granite](https://github.com/ibm-granite/)
-- **Release Date:** March 18th, 2026
-- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
-- **Paper:** [Granite Guardian](https://arxiv.org/abs/2412.07724)
-
-## Usage
-
-**Intended Use:** The Requirement Checker adapter is designed for evaluating whether LLM-generated responses satisfy user-specified constraints. Key use cases include:
-
-### Use Case
-- **Instruction-following evaluation**: Assessing whether a model's output adheres to specific formatting, content, or structural requirements provided in the prompt.
-- **Constraint satisfaction checking**: Verifying that generated text meets multiple simultaneous constraints (e.g., length limits, style requirements, content inclusion/exclusion).
-- **Quality assurance pipelines**: Automated checking of LLM outputs against predefined acceptance criteria.
-
-### Installation
-
-```bash
-pip install transformers peft torch
-```
-
-### Quickstart Example (LoRA)
-
-```python
-import json
-import torch
-from transformers import AutoTokenizer, AutoModelForCausalLM
-from peft import PeftModel
-
-base_model_id = "ibm-granite/granite-4.0-micro"
-adapter_repo = "ibm-granite/granitelib-core-r1.0"
-adapter_subfolder = "requirement-check/granite-4.0-micro/lora"
-
-# Load base model and LoRA adapter
-tokenizer = AutoTokenizer.from_pretrained(base_model_id)
-base_model = AutoModelForCausalLM.from_pretrained(
-    base_model_id, torch_dtype=torch.bfloat16, device_map="auto"
-)
-model = PeftModel.from_pretrained(base_model, adapter_repo, subfolder=adapter_subfolder)
-model.eval()
-
-# Define the user prompt, assistant response, and constraints to check
-user_text = "Invite for an IBM office party."
-response_text = """
-Dear Team,
-
-To celebrate our recent successes and take a well-deserved moment to recharge,
-you are cordially invited to a team social. Please join us for an evening of
-live music, appetizers, and drinks as we recognize our collective wins.
-
-Event Details
-* **Date:** Saturday, April 25, 2026
-* **Time:** 6:00 PM
-* **Location:** Ryan’s Bar, Chelsea, NY
-* **Highlights:** Live entertainment and refreshments
-
-RSVP
-To ensure we have an accurate headcount for catering, please confirm your
-attendance by **Friday, April 10, 2026**.
-
-We look forward to seeing everyone there and celebrating our hard work together.
-
-**Best regards,**
-[Your Name/Management Team]
-"""
-
-constraints = "Use a professional tone."
-
-# Build the evaluation prompt
-evaluation_prompt = (
-    "Please verify if the assistant's generation satisfies the user's "
-    "requirements or not and reply with a binary label accordingly. "
-    'Respond with a json {"score": "yes"} if the constraints are '
-    'satisfied or respond with {"score": "no"} if the constraints are not '
-    "satisfied."
-)
-
-messages = [
-    {"role": "user", "content": user_text},
-    {"role": "assistant", "content": response_text},
-    {"role": "user", "content": f"<requirements> {constraints}\n{evaluation_prompt}"},
-]
-
-prompt = tokenizer.apply_chat_template(
-    messages, tokenize=False, add_generation_prompt=True
-)
-
-# Generate
-inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
-input_len = inputs["input_ids"].shape[1]
-
-stop_token_id = tokenizer.encode("}", add_special_tokens=False)[0]
-
-with torch.no_grad():
-    output = model.generate(
-        **inputs,
-        max_new_tokens=20,
-        do_sample=False,
-        eos_token_id=[tokenizer.eos_token_id, stop_token_id],
-    )
-
-response = tokenizer.decode(output[0][input_len:], skip_special_tokens=True).strip()
-
-brace_idx = response.find("}")
-if brace_idx != -1:
-    response = response[:brace_idx + 1]
-
-print(f"Response: {response}")  # {"score": "yes"}
-
-result = json.loads(response)
-print(f"Constraints satisfied: {result['score']}")  # yes
-```
-
-### Prompt Template
-
-The requirement checker uses a structured prompt format where user/assistant messages are followed by a requirement evaluation turn. The `<requirements>` tag marks the constraints to be evaluated:
-
-```
-<user message>
-<assistant response>
-<requirements> <constraints>
-<evaluation prompt>
-```
-
-Typical usage will check the last assistant generation and the following text can be used:
-
-```python
-evaluation_prompt = (
-    "Please verify if the assistant's generation satisfies the user's "
-    "requirements or not and reply with a binary label accordingly. "
-    'Respond with a json {"score": "yes"} if the constraints are '
-    'satisfied or respond with {"score": "no"} if the constraints are not '
-    "satisfied."
-)
-```
-
-## Evaluations
-
-Binary classification performance on instruction-following benchmarks: [HelpSteer3](https://huggingface.co/datasets/nvidia/HelpSteer3), [IFEval Multi-Constraint](https://arxiv.org/abs/2311.07911), and [InfoBench](https://arxiv.org/abs/2401.03601) (GPT-4-annotated and human-annotated splits).
-
-<table>
- <caption style="text-align:center"><b>Requirement Checking Benchmarks</b></caption>
-<thead>
-  <tr>
-    <th style="text-align:left; background-color: #001d6c; color: white;">Benchmark</th>
-    <th style="text-align:center; background-color: #001d6c; color: white;">AUC</th>
-    <th style="text-align:center; background-color: #001d6c; color: white;">Accuracy</th>
-    <th style="text-align:center; background-color: #001d6c; color: white;">Bal. Acc.</th>
-    <th style="text-align:center; background-color: #001d6c; color: white;">F1</th>
-    <th style="text-align:center; background-color: #001d6c; color: white;">Precision</th>
-    <th style="text-align:center; background-color: #001d6c; color: white;">Recall</th>
-  </tr></thead>
-  <tbody>
-  <tr>
-    <td style="text-align:left; background-color: #e8f0fe; color: #2D2D2D;">HelpSteer3</td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.8260 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.7463 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.7434 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.7686 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.7598 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.7776 </td>
-  </tr>
-  <tr>
-    <td style="text-align:left; background-color: #e8f0fe; color: #2D2D2D;">IFEval Multi-Constraint</td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.9186 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.8459 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.8403 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.8092 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.8071 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.8114 </td>
-  </tr>
-  <tr>
-    <td style="text-align:left; background-color: #e8f0fe; color: #2D2D2D;">InfoBench (GPT-4 Annotated)</td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.7660 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.7304 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.6950 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.8207 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.9260 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.7394 </td>
-  </tr>
-  <tr>
-    <td style="text-align:left; background-color: #e8f0fe; color: #2D2D2D;">InfoBench (Human Annotated)</td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.7311 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.7523 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.6799 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.8034 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.8918 </td>
-    <td style="text-align:center; background-color: #e8f0fe; color: #2D2D2D;"> 0.7409 </td>
-  </tr>
-  </tbody>
-</table>
-
-The evaluations on Helpsteer3 and IFEval above are on the validation set and a heldout set, respectively.
-
-## Training Data
-
-The Requirement Checker adapter is fine-tuned on top of `ibm-granite/granite-4.0-micro` using a combination of instruction-following evaluation data. Training data includes samples with explicit user requirements and constraints paired with assistant responses, annotated for whether the constraints are satisfied.
-
-The two sources of training data are:
-
-1. [IF-RLVR training data](https://huggingface.co/datasets/allenai/IF_multi_constraints_upto5), with up to 5 constraints per instruction. Each prompt (with constraints) were passed through [Mixtral 8x22b Instruct](https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1). Then, the [prompt, constraints, response] are all passed through a [programmatic evaluation pipeline](https://github.com/allenai/IFBench?tab=readme-ov-file) which provides yes/no labels corresponding to whether the constraints were followed or not.
-2. [Helpsteer3-Preference](https://huggingface.co/datasets/nvidia/HelpSteer3). The original dataset consists of [prompt, response1, response2, feedback]. The `feedback` column contains individual rankings (a score from 1-5) of each response from multiple annotators. We map scores 4-5 to a label of yes and scores 1-3 to a label of no, take the average of the annotators's scores and map each of the two responses to a binary yes/no label accordingly.
-
-### Adapter Configuration
-
-| Property | LoRA |
-|---|---|
-| **Base Model** | ibm-granite/granite-4.0-micro |
-| **PEFT Type** | LORA |
-| **Rank (r)** | 64 |
-| **Alpha** | 64 |
-| **Target Modules** | q_proj, k_proj, v_proj, o_proj, input_linear, output_linear |
-| **vLLM Support** | Yes |
-
-### Evaluation
-The evaluations on Helpsteer3 and IFEval above are on the validation set and a heldout set, respectively.
-
-### Infrastructure
-
-Training was completed using 8 H100 GPUs. Evaluation (and inference) requires 1 H100 GPU.
-
-## Ethical Considerations / Limitations
-
-- The model must only be used in the prescribed evaluation mode, outputting JSON responses (`{"score": "yes"/"no"}`) based on the specified prompt template. Any deviation from this intended use may lead to unexpected outputs.
-- The model is designed for evaluating constraint satisfaction in instruction-following scenarios.
-- The model is only trained and tested on English data.
-- The LoRA adapter is compatible with both vLLM (for efficient batched inference) and HuggingFace Transformers + PEFT.
-
-## Citation
-
-```bibtex
-@misc{padhi2024graniteguardian,
-      title={Granite Guardian},
-      author={Inkit Padhi and Manish Nagireddy and Giandomenico Cornacchia and Subhajit Chaudhury and Tejaswini Pedapati and Pierre Dognin and Keerthiram Murugesan and Erik Miehling and Mart\'{i}n Santill\'{a}n Cooper and Kieran Fraser and Giulio Zizzo and Muhammad Zaid Hameed and Mark Purcell and Michael Desmond and Qian Pan and Zahra Ashktorab and Inge Vejsbjerg and Elizabeth M. Daly and Michael Hind and Werner Geyer and Ambrish Rawat and Kush R. Varshney and Prasanna Sattigeri},
-      year={2024},
-      eprint={2412.07724},
-      archivePrefix={arXiv},
-      primaryClass={cs.CL},
-      url={https://arxiv.org/abs/2412.07724},
-}
-```
-
-## Resources
-
-- ⭐️ Learn about the latest updates with Granite: https://www.ibm.com/granite
-- 📄 Get started with tutorials, best practices, and prompt engineering advice: https://www.ibm.com/granite/docs/
-- 💡 Learn about the latest Granite learning resources: https://ibm.biz/granite-learning-resources

uncertainty/README.md

@@ -0,0 +1,178 @@
+---
+license: apache-2.0
+language:
+- en
+pipeline_tag: text-generation
+library_name:
+- peft
+- transformers
+base_model:
+- ibm-granite/granite-4.0-micro
+- ibm-granite/granite-4.1-3b
+- ibm-granite/granite-4.1-8b
+- ibm-granite/granite-4.1-30b
+tags:
+- granite
+- guardian
+- uncertainty-quantification
+- lora
+- peft
+---
+
+# Uncertainty Quantification
+
+## Model Summary
+
+**Uncertainty** is a family of LoRA/aLoRA adapters that provide calibrated certainty scores for Granite 4.1 models. The adapter responds with a certainty score from 0 to 9, which maps to a calibrated likelihood via `confidence = 0.1 * score + 0.05`, yielding 10 possible values (5%, 15%, 25%, ..., 95%). This percentage is *calibrated* in the following sense: given a set of answers assigned a certainty score of X%, approximately X% of these answers should be correct. We provide uncertainty adapters for Granite-4.0-micro, Granite-4.1-3B, Granite-4.1-8B, and Granite-4.1-30B.
+
+- **Developer:** IBM Research
+- **HF Collection:** [Granite Libraries](https://huggingface.co/collections/ibm-granite/granite-libraries)
+- **GitHub Repository:** https://github.com/ibm-granite
+- **Last Updated:** May 5th, 2026
+- **Model type:** LoRA/aLoRA adapters for:
+  - [ibm-granite/granite-4.0-micro](https://huggingface.co/ibm-granite/granite-4.0-micro)
+  - [ibm-granite/granite-4.1-3b](https://huggingface.co/ibm-granite/granite-4.1-3b)
+  - [ibm-granite/granite-4.1-8b](https://huggingface.co/ibm-granite/granite-4.1-8b)
+  - [ibm-granite/granite-4.1-30b](https://huggingface.co/ibm-granite/granite-4.1-30b)
+- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
+- **Paper:** [[Shen et al. ICML 2024] Thermometer: Towards Universal Calibration for Large Language Models](https://arxiv.org/abs/2403.08819)
+
+## Intended Use
+
+Uncertainty is a LoRA adapter that enables Granite 4.1 base models to express calibrated self-assessments of their own answer correctness. The adapter is designed to be used as part of the Granite inference pipeline, activated via the `<certainty>` invocation token after the model generates a response.
+
+**Adapter input:** An OpenAI-compatible chat completion request containing the user query and the model's response, followed by the `<certainty>` invocation token.
+
+**Adapter output:** A JSON object `{"score": "X"}` where X is 0-9, mapping to calibrated confidence via `0.1 * score + 0.05`.
+
+**Scoring:** When called via Mellea, the discrete 0–9 output is converted to a calibrated certainty float using the token-level probabilities over all 10 score categories, weighted by their mapped values (5%, 15%, …, 95%). This yields a continuous confidence estimate rather than a coarse 10-bin discretization. The threshold for "confident enough" is left to the user and may need to be tuned for your use-case.
+
+### Use Cases
+* **Human usage:** Certainty scores give human users an indication of when to trust answers from the model.
+* **Model routing/guards:** If the model has low certainty (below a chosen threshold), route the request to a larger model or decline to show the response.
+* **RAG:** Certainty scores can be applied to answers created using RAG, predicting overall correctness based on both the documents and the model's own knowledge.
+
+## Usage Examples
+
+### via Mellea
+
+The recommended way to call this adapter is through the [Mellea](https://mellea.ai) framework:
+
+```python
+from mellea import model_ids, start_backend
+from mellea.stdlib import functional as mfuncs
+from mellea.stdlib.components.intrinsic import core
+
+ctx, backend = start_backend(
+    "hf", model_id=model_ids.IBM_GRANITE_4_1_3B, context_type="chat"
+)
+
+response, ctx = mfuncs.chat("What is IBM Research?", ctx, backend)
+print(f"Response: {response.content}")
+
+result = core.check_certainty(ctx, backend)
+print(f"Certainty score: {result}")
+```
+
+### via HF+PEFT
+
+```python
+import re
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+from peft import PeftModel
+
+BASE_NAME = "ibm-granite/granite-4.1-3b"
+LORA_REPO = "ibm-granite/granitelib-core-r1.0"
+LORA_SUBFOLDER = "uncertainty/granite-4.1-3b/lora"
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+tokenizer = AutoTokenizer.from_pretrained(BASE_NAME, padding_side="left")
+model_base = AutoModelForCausalLM.from_pretrained(BASE_NAME, device_map="auto", torch_dtype=torch.bfloat16)
+model_uq = PeftModel.from_pretrained(
+    AutoModelForCausalLM.from_pretrained(BASE_NAME, device_map="auto", torch_dtype=torch.bfloat16),
+    LORA_REPO,
+    subfolder=LORA_SUBFOLDER,
+)
+
+question = "What is IBM Research?"
+
+# Step 1: Generate answer with base model
+messages = [{"role": "user", "content": question}]
+input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(input_text, return_tensors="pt").to(device)
+output = model_base.generate(**inputs, max_new_tokens=600, do_sample=False)
+answer = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
+
+# Step 2: Generate certainty score with LoRA adapter
+uq_messages = [
+    {"role": "user", "content": question},
+    {"role": "assistant", "content": answer},
+    {"role": "user", "content": "<certainty>"},
+]
+uq_text = tokenizer.apply_chat_template(uq_messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(uq_text, return_tensors="pt").to(device)
+output = model_uq.generate(**inputs, max_new_tokens=15, do_sample=False)
+uq_response = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
+
+# Parse score and map to confidence
+match = re.search(r'\{[^}]*"score"\s*:\s*"?(\d)"?[^}]*\}', uq_response)
+if match:
+    score = int(match.group(1))
+    confidence = 0.1 * score + 0.05
+    print(f"Score: {score}, Certainty: {confidence*100:.0f}%")
+```
+
+For more comprehensive examples, refer to the [Mellea intrinsics examples](https://github.com/generative-computing/mellea/tree/main/docs/examples/intrinsics).
+
+## Evaluation
+
+The adapters were evaluated on the [MMLU](https://huggingface.co/datasets/cais/mmlu) dataset (57 subsets, 14,042 total samples, not used in training). We report [Expected Calibration Error (ECE)](https://towardsdatascience.com/expected-calibration-error-ece-a-step-by-step-visual-explanation-with-python-code-c3e9aa12937d), Brier Score, AUROC, and Sharpness. Lower ECE and Brier scores indicate better calibration; higher AUROC indicates better discrimination between correct and incorrect answers.
+
+### Results
+
+| Model | Method | ECE | Brier Score | AUROC | Sharpness |
+|---|---|---|---|---|---|
+| Granite-4.0-micro | Base (seq prob) | 0.1606 | 0.2535 | 0.6748 | 0.2888 |
+| | LoRA | 0.0565 | 0.2131 | 0.6903 | 0.1607 |
+| Granite-4.1-3B | Base (seq prob) | 0.2879 | 0.3011 | 0.6291 | 0.0648 |
+| | LoRA | 0.0998 | 0.1845 | 0.7870 | 0.1971 |
+| | aLoRA | 0.0986 | 0.1877 | 0.7713 | 0.2027 |
+| Granite-4.1-8B | Base (seq prob) | 0.1123 | 0.1932 | 0.6439 | 0.0438 |
+| | LoRA | 0.0455 | 0.1487 | 0.7912 | 0.1898 |
+| | aLoRA | 0.0381 | 0.1515 | 0.7778 | 0.1924 |
+| Granite-4.1-30B | Base (seq prob) | 0.0921 | 0.1559 | 0.6507 | 0.0358 |
+| | LoRA | 0.1006 | 0.1287 | 0.8116 | 0.1900 |
+| | aLoRA | 0.1002 | 0.1317 | 0.8017 | 0.1927 |
+
+The LoRA/aLoRA adapters consistently improve calibration (lower Brier scores) and discrimination (higher AUROC) across all model sizes, with zero-shot MMLU performance preserved.
+
+## Training Details
+
+The uncertainty adapters are finetuned to provide certainty scores mimicking the output of a calibrator trained via the Thermometer method ([Shen et al. ICML 2024](https://arxiv.org/abs/2403.08819)).
+
+**Training Data:** Each adapter was trained on ~240K question-answer pairs generated by its respective base model, where each pair is annotated with a certainty score (0-9) derived from a calibrated thermometer model. The following datasets were used for calibration and/or fine-tuning:
+
+* [BigBench](https://huggingface.co/datasets/tasksource/bigbench), [MRQA](https://huggingface.co/datasets/mrqa-workshop/mrqa), [newsqa](https://huggingface.co/datasets/lucadiliello/newsqa), [trivia_qa](https://huggingface.co/datasets/mandarjoshi/trivia_qa), [search_qa](https://huggingface.co/datasets/lucadiliello/searchqa), [openbookqa](https://huggingface.co/datasets/allenai/openbookqa), [web_questions](https://huggingface.co/datasets/Stanford/web_questions), [cosmos_qa](https://huggingface.co/datasets/allenai/cosmos_qa), [kilt_tasks](https://huggingface.co/datasets/facebook/kilt_tasks), [smiles-qa](https://huggingface.co/datasets/alxfgh/ChEMBL_Drug_Instruction_Tuning), [orca-math](https://huggingface.co/datasets/microsoft/orca-math-word-problems-200k), [ARC-Easy](https://huggingface.co/datasets/allenai/ai2_arc), [commonsense_qa](https://huggingface.co/datasets/tau/commonsense_qa), [social_i_qa](https://huggingface.co/datasets/allenai/social_i_qa), [super_glue](https://huggingface.co/datasets/aps/super_glue), [figqa](https://huggingface.co/datasets/nightingal3/fig-qa), [riddle_sense](https://huggingface.co/datasets/INK-USC/riddle_sense), [ag_news](https://huggingface.co/datasets/fancyzhx/ag_news), [medmcqa](https://huggingface.co/datasets/openlifescienceai/medmcqa), [dream](https://huggingface.co/datasets/dataset-org/dream), [codah](https://huggingface.co/datasets/jaredfern/codah), [piqa](https://huggingface.co/datasets/ybisk/piqa)
+
+### Adapter Details
+
+| Property | Granite-4.0-micro | Granite-4.1-3B | Granite-4.1-8B | Granite-4.1-30B |
+|---|---|---|---|---|
+| **PEFT Type** | LoRA | LoRA/aLoRA | LoRA/aLoRA | LoRA/aLoRA |
+| **Rank (r)** | 32 | 32 | 32 | 32 |
+| **Alpha** | 64 | 64 | 64 | 64 |
+| **Target Modules** | q_proj, k_proj, v_proj, o_proj, input_linear, output_linear | q_proj, k_proj, v_proj, o_proj | q_proj, k_proj, v_proj, o_proj | q_proj, k_proj, v_proj, o_proj |
+| **Invocation token (aLoRA)** | — | `<certainty>` | `<certainty>` | `<certainty>` |
+| **Output format** | `{"score": "X"}` (X: 0-9) | `{"score": "X"}` (X: 0-9) | `{"score": "X"}` (X: 0-9) | `{"score": "X"}` (X: 0-9) |
+| **Confidence mapping** | `0.1 * score + 0.05` | `0.1 * score + 0.05` | `0.1 * score + 0.05` | `0.1 * score + 0.05` |
+| **Max completion tokens** | 15 | 15 | 15 | 15 |
+
+**Infrastructure:** Training was completed using H100 GPUs.
+
+**Ethical Considerations:** Certainty is inherently an intrinsic property of a model and its abilities. The Uncertainty adapter is not intended to predict the certainty of responses generated by any other models besides its corresponding base model. Certainty scores are *distributional* quantities and will do well on realistic questions in aggregate, but may have surprising scores on individual red-teamed examples. Certainty scores may be biased towards moderate values because: (1) calibrated models are less likely to output very high or very low confidence, (2) smaller models cannot perfectly self-assess, and (3) generalization to novel question difficulties requires hedging.
+
+**Resources:**
+- Learn about the latest updates with Granite: https://www.ibm.com/granite
+- Get started with tutorials: https://www.ibm.com/granite/docs/
+- Granite learning resources: https://ibm.biz/granite-learning-resources

uncertainty/granite-4.0-micro/.gitattributes

@@ -1 +0,0 @@
-*.png filter=lfs diff=lfs merge=lfs -text

uncertainty/granite-4.0-micro/README.md

@@ -1,136 +0,0 @@
-# Micro Uncertainty
-
-**Model Summary:** Uncertainty adapter provides calibrated certainty scores for [ibm-granite/granite-4.0-micro](https://huggingface.co/ibm-granite/granite-4.0-micro). The model responds with a certainty score from 0 to 9, which maps to a calibrated likelihood via `confidence = 0.1 * score + 0.05`, yielding 10 possible values (5%, 15%, 25%, ..., 95%). This percentage is *calibrated* in the following sense: given a set of answers assigned a certainty score of X%, approximately X% of these answers should be correct. See the evaluation section below for out-of-distribution verification of this behavior.
-
-- **Developer:** IBM Research
-- **HF Collection:** [Granite Libraries](https://huggingface.co/collections/ibm-granite/granite-libraries)
-- **GitHub Repository:** https://github.com/ibm-granite
-- **Release Date:** March 18th, 2026
-- **Model Type:** LoRA adapter for [ibm-granite/granite-4.0-micro](https://huggingface.co/ibm-granite/granite-4.0-micro)
-- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
-- **Paper:** Granite 4.0 Micro Uncertainty adapter is finetuned to provide certainty scores mimicking the output of a calibrator trained via the method in [[Shen et al. ICML 2024] Thermometer: Towards Universal Calibration for Large Language Models](https://arxiv.org/abs/2403.08819)
-
-## Usage
-
-**Intended use:** Uncertainty is a LoRA adapter the enables the Granite 4.0 Micro base model to express calibrated self-assessments of its own answer correctness. This adapter is designed to be used as part of the Granite inference pipeline, activated via the `<certainty>` invocation token after the model generates a response.
-
-### Use Cases
-* Human usage: Certainty scores give human users an indication of when to trust answers from the model (which should be augmented by their own knowledge).
-* Model routing/guards: If the model has low certainty (below a chosen threshold), it may be worth sending the request to a larger, more capable model or simply choosing not to show the response to the user.
-* RAG: Uncertainty is calibrated on diverse question-answering datasets, hence it can be applied to giving certainty scores for answers created using RAG. This certainty will be a prediction of overall correctness based on both the documents given and the model's own knowledge.
-
-### Quickstart Example (LoRA)
-
-```python
-import torch
-from transformers import AutoTokenizer, AutoModelForCausalLM
-from peft import PeftModel
-
-BASE_NAME = "ibm-granite/granite-4.0-micro"
-LORA_REPO = "ibm-granite/granitelib-core-r1.0"
-LORA_SUBFOLDER = "uncertainty/granite-4.0-micro/lora"
-device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
-
-# Load model
-tokenizer = AutoTokenizer.from_pretrained(BASE_NAME, padding_side="left", trust_remote_code=True)
-model_base = AutoModelForCausalLM.from_pretrained(BASE_NAME, device_map="auto", torch_dtype=torch.bfloat16)
-model_uq = PeftModel.from_pretrained(
-    AutoModelForCausalLM.from_pretrained(BASE_NAME, device_map="auto", torch_dtype=torch.bfloat16),
-    LORA_REPO,
-    subfolder=LORA_SUBFOLDER,
-)
-
-question = "What is IBM Research?"
-print("Question:", question)
-
-# Step 1: Generate answer with base model
-messages = [
-    {"role": "user", "content": question},
-]
-input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
-inputs = tokenizer(input_text, return_tensors="pt").to(device)
-output = model_base.generate(**inputs, max_new_tokens=600, do_sample=False)
-answer = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
-print("Answer:", answer)
-
-# Step 2: Generate certainty score with LoRA adapter
-uq_messages = [
-    {"role": "user", "content": question},
-    {"role": "assistant", "content": answer},
-    {"role": "user", "content": "<certainty>"},
-]
-uq_text = tokenizer.apply_chat_template(uq_messages, tokenize=False, add_generation_prompt=True)
-inputs = tokenizer(uq_text, return_tensors="pt").to(device)
-output = model_uq.generate(**inputs, max_new_tokens=15, do_sample=False)
-uq_response = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
-print("Raw response:", uq_response)
-
-# Parse score and map to confidence
-import re
-match = re.search(r'\{[^}]*"score"\s*:\s*"?(\d)"?[^}]*\}', uq_response)
-if match:
-    score = int(match.group(1))
-    confidence = 0.1 * score + 0.05
-    print(f"Score: {score}, Certainty: {confidence*100:.0f}%")
-```
-
-## Evaluation
-
-The adapter was evaluated on the [MMLU](https://huggingface.co/datasets/cais/mmlu) dataset (57 subsets, 14,042 total samples, not used in training). Shown are the [Expected Calibration Error (ECE)](https://towardsdatascience.com/expected-calibration-error-ece-a-step-by-step-visual-explanation-with-python-code-c3e9aa12937d) for each task, for the base model (granite-4.0-micro, using sequence probability as confidence) and the LoRA . The LoRA adapter achieves a weighted ECE of 0.0565, a 65% improvement over the base model's sequence-probability baseline of 0.1606. Additionally, the zero-shot performance on the MMLU tasks does not degrade, averaging at 63.2%.
-
-| Metric | Base Model | LoRA |
-|--------|:----------:|:----:|
-| ECE | 0.1606 | 0.0565 |
-| Brier Score | 0.2535 | 0.2131 |
-| AUROC | 0.6748 | 0.6903 |
-| Sharpness | 0.2888 | 0.1607 |
-
-### Adapter Configurations
-
-| Parameter | LoRA |
-|-----------|------|
-| Base model | ibm-granite/granite-4.0-micro |
-| LoRA rank (r) | 32 |
-| LoRA alpha | 64 |
-| Target modules | q_proj, k_proj, v_proj, o_proj, input_linear, output_linear |
-| Invocation token | `<certainty>` |
-| Output format | `{"score": "X"}` where X is 0-9 |
-| Confidence mapping | `0.1 * score + 0.05` (5% to 95%) |
-| Max completion tokens | 15 |
-| KV cache | Supported |
-
-## Training Details
-
-Granite 4.0 Micro Uncertainty LoRA adapter is finetuned to provide certainty scores mimicking the output of a calibrator trained via the method in [[Shen et al. ICML 2024] Thermometer: Towards Universal Calibration for Large Language Models](https://arxiv.org/abs/2403.08819).
-
-**Training Data:** The adapter was trained on a dataset of ~199K question-answer pairs generated by the base model (granite-4.0-micro), where each pair is annotated with a certainty score (0-9) derived from a calibrated thermometer model. The following datasets were used for calibration and/or fine-tuning.
-
-* [BigBench](https://huggingface.co/datasets/tasksource/bigbench)
-* [MRQA](https://huggingface.co/datasets/mrqa-workshop/mrqa)
-* [newsqa](https://huggingface.co/datasets/lucadiliello/newsqa)
-* [trivia_qa](https://huggingface.co/datasets/mandarjoshi/trivia_qa)
-* [search_qa](https://huggingface.co/datasets/lucadiliello/searchqa)
-* [openbookqa](https://huggingface.co/datasets/allenai/openbookqa)
-* [web_questions](https://huggingface.co/datasets/Stanford/web_questions)
-* [smiles-qa](https://huggingface.co/datasets/alxfgh/ChEMBL_Drug_Instruction_Tuning)
-* [orca-math](https://huggingface.co/datasets/microsoft/orca-math-word-problems-200k)
-* [ARC-Easy](https://huggingface.co/datasets/allenai/ai2_arc)
-* [commonsense_qa](https://huggingface.co/datasets/tau/commonsense_qa)
-* [social_i_qa](https://huggingface.co/datasets/allenai/social_i_qa)
-* [super_glue](https://huggingface.co/datasets/aps/super_glue)
-* [figqa](https://huggingface.co/datasets/nightingal3/fig-qa)
-* [riddle_sense](https://huggingface.co/datasets/INK-USC/riddle_sense)
-* [ag_news](https://huggingface.co/datasets/fancyzhx/ag_news)
-* [medmcqa](https://huggingface.co/datasets/openlifescienceai/medmcqa)
-* [dream](https://huggingface.co/datasets/dataset-org/dream)
-* [codah](https://huggingface.co/datasets/jaredfern/codah)
-* [piqa](https://huggingface.co/datasets/ybisk/piqa)
-
-**Infrastructure:** Training was completed using 8 H100 GPUs. Evaluation (and inference) requires 1 H100 GPU.
-
-**Ethical Considerations:** Certainty is inherently an intrinsic property of a model and its abilities. The Uncertainty adapter is not intended to predict the certainty of responses generated by any other models besides itself or [ibm-granite/granite-4.0-micro](https://huggingface.co/ibm-granite/granite-4.0-micro). Additionally, certainty scores are *distributional* quantities, and so will do well on realistic questions in aggregate, but in principle may have surprising scores on individual red-teamed examples. Certainty scores, at times, may be biased towards moderate certainty scores for the following reasons. Firstly, as humans, we tend to be overconfident in our evaluation of what we know and don't know - in contrast, a calibrated model is less likely to output very high or very low confidence scores, as these imply certainty of correctness or incorrectness. Secondly, remember that the model is evaluating itself - correctness/incorrectness that may be obvious to larger models may be less obvious to a smaller model. Finally, teaching a model every fact it knows and doesn't know is not possible, hence it must generalize to questions of wildly varying difficulty. Intuitively, it does this by extrapolating based on related questions it has been evaluated on in training -- this is an inherently inexact process and leads to some hedging.
-
-**Resources:**
-- ⭐️ Learn about the latest updates with Granite: https://www.ibm.com/granite
-- 📄 Get started with tutorials, best practices, and prompt engineering advice: https://www.ibm.com/granite/docs/
-- 💡 Learn about the latest Granite learning resources: https://ibm.biz/granite-learning-resources