Initial release

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+images/sui_icon.jpeg filter=lfs diff=lfs merge=lfs -text
+tekken.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,572 @@
+---
+license: apache-2.0
+base_model: mistralai/Mistral-Small-3.2-24B-Instruct-2506
+tags:
+  - summarization
+  - long-context
+  - grounded-generation
+  - citation
+  - xml-tagging
+language:
+  - en
+  - de
+  - es
+  - fr
+  - it
+  - pt
+  - nl
+  - pl
+  - zh
+  - ja
+  - ko
+  - ru
+  - ar
+  - tr
+  - vi
+  - id
+  - hi
+  - sv
+  - uk
+  - ro
+  - cs
+  - el
+  - hu
+  - th
+pipeline_tag: text-generation
+library_name: transformers
+---
+
+<div align="center">
+  <img src="images/sui_icon.jpeg" alt="SUI Icon" width="200">
+  <h3>sui-1</h3>
+</div>
+
+**sui-1** (Summarization with Unique Identifiers) is a specialized model for high-quality summarization of very long texts with built-in source grounding. Every claim in the summary can be traced back to its source sentence, enabling verification and reducing hallucination risk.
+
+## Key Features
+
+- **Very Long Document Processing**: Handles up to 128k tokens natively, with a two-step iterative approach for documents up to 2 million tokens
+- **Single GPU Deployment**: The FP8 variant runs on a single A100 40GB or A6000 48GB GPU; the iterative approach enables deployment on even more modest hardware
+- **Competitive Performance**: Significantly outperforms all tested open-weight baselines, including models with 3x more parameters
+- **Multilingual Support**: Fine-tuned for English, German, Spanish, French, and Italian; inherits 20+ additional languages from Mistral Small 3.2
+- **High-Quality Training Data**: Built using a sophisticated data generation pipeline that produced 22,000+ training examples from parliamentary documents, web sources, and Wikipedia using chain-of-thought reasoning with multi-stage verification
+- **Verifiable Outputs**: Built-in citation mechanism links each claim to its source sentence for full traceability
+
+## Quick Start
+
+Run the end-to-end [example.py](example.py) script (requires [uv](https://docs.astral.sh/uv/)):
+
+```bash
+# Summarize a document
+uv run example.py document.txt
+
+# Or with inline text
+uv run example.py --text "Your long text here..." --words 300 --tags 8
+```
+
+The script handles everything: sentence tagging, model inference, and formatted output with source citations.
+
+## Evaluation
+
+We evaluate sui-1-24b using an **LLM-as-a-Judge** methodology, where a strong judge model evaluates summary quality across multiple criteria. This approach captures nuanced quality aspects that traditional metrics like ROUGE cannot measure.
+
+### Overall Performance
+
+![Overall Performance](images/overall_performance.png)
+
+The chart shows the overall success rate across all evaluation criteria. sui-1-24b significantly outperforms its base model (Mistral-Small-3.2-24B) on the summarization task.
+
+### Performance by Criteria
+
+We evaluate summaries on five key dimensions:
+
+| Criterion | Description |
+|-----------|-------------|
+| **Factual Accuracy** | Does the summary avoid introducing new facts, entities, numbers, or claims not supported by the source content? |
+| **Coverage & Completeness**¹ | Does the summary cover the document's main points and key takeaways at appropriate granularity? |
+| **Specificity & Informativeness** | Are claims specific and informative rather than generic filler (e.g., "there are several points")? |
+| **Format Compliance** | Is the output compliant with formatting instructions including language consistency, semantic-aware planning, and paragraph structure? |
+| **Custom Instruction**² | If a custom instruction is provided, is it followed appropriately? |
+
+![Criteria Breakdown](images/criteria_breakdown.png)
+
+The evaluation was conducted on 100 diverse test samples covering multiple languages (English, German, Spanish, French, Italian) and document types. Scoring uses binary pass/fail per criterion, aggregated to success rates.
+
+*¹ Coverage scores are lower when samples require constrained formats (bullet points, short summaries) that inherently limit content coverage.*
+*² Tests whether the model deviates from its default prose style when users request specific formats.*
+
+### Grounding Metrics
+
+In addition to LLM-as-a-Judge evaluation, we validate grounding quality using structural checks:
+
+1. **Tag Uniqueness**: All referenced tags in `xml_tags` must be unique
+2. **Tag Validity**: All referenced tags must exist in the input text
+3. **Tag Usage**: All tags in `xml_tags` must appear in the summary
+
+### Elluminate
+
+The evaluation was performed using [Elluminate](https://elluminate.de), a collaborative evaluation platform for enterprise AI. Elluminate provides structured LLM-as-a-Judge workflows that enable teams to standardize quality metrics and systematically measure AI performance across defined criteria.
+
+This model is a contribution by [ellamind](https://ellamind.ai) to the open-source community.
+
+---
+
+## Model Weights
+
+We provide two variants:
+
+| Variant | Description | Link |
+|---------|-------------|------|
+| **bfloat16** | Full precision (~48GB weights) | [ellamind/sui-1-24b](https://huggingface.co/ellamind/sui-1-24b) |
+| **FP8** | Quantized (~24GB weights), lower VRAM | [ellamind/sui-1-24b-fp8](https://huggingface.co/ellamind/sui-1-24b-fp8) |
+
+The FP8 version preserves high quality, scoring 81.05% overall on our benchmark—nearly identical to bfloat16.
+
+---
+
+## Hardware Requirements
+
+We tested various GPU configurations using vLLM. The tables below show minimum requirements for different context lengths.
+
+The bfloat16 variant requires **~55GB VRAM** for 8k context, scaling to **~76GB** for 128k. The FP8 variant requires **~38GB** for 8k and **~50GB** for 128k.
+
+### bfloat16 (Full Precision)
+
+| Setup | 8k | 32k | 64k | 128k |
+|-------|:--:|:---:|:---:|:----:|
+| 1× A100 80GB / H100 96GB| ✓ | ✓ | ✓ | ✓ |
+| 2× RTX 5090 (32GB) | ✓ | ✓ | ✗ | ✗ |
+| 2× A100 40GB / A6000 | ✓ | ✓ | ✓ | ✓ |
+| 4× RTX 4090 (24GB) | ✓ | ✓ | ✓ | ✓ |
+
+### FP8 Quantized (Recommended for Consumer GPUs)
+
+| Setup | 8k | 32k | 64k | 128k |
+|-------|:--:|:---:|:---:|:----:|
+| 1× A100 40GB | ✓ | ✓ | ✗ | ✗ |
+| 1× A6000 (48GB) | ✓ | ✓ | ✓ | ✗ |
+| 1× A100 80GB / H100 96GB| ✓ | ✓ | ✓ | ✓ |
+| 2× RTX 4090 (24GB) | ✓ | ✓ | ✓ | ✗ |
+| 2× RTX 5090 (32GB) | ✓ | ✓ | ✓ | ✓ |
+| 4× RTX 4090 (24GB) | ✓ | ✓ | ✓ | ✓ |
+
+> **Tip**: The model supports both one-shot summarization (full document in context) and an iterative two-step approach for very long documents (see [Handling Very Long Contexts](#handling-very-long-contexts)). The 8k context configuration is sufficient to produce high-quality summaries using the iterative approach, making the model accessible on more modest hardware.
+
+---
+
+## How It Works
+
+The model follows a three-phase approach:
+
+1. **Planning Phase**: Analyzes the input and plans the summary structure
+2. **Reference Selection**: Identifies the most important sentences to cite
+3. **Grounded Generation**: Produces a summary with inline citations to source sentences
+
+Citations use XML tags assigned during preprocessing, enabling deterministic verification of each claim.
+
+---
+
+## Input Format
+
+The input text must be preprocessed with XML sentence tags. Each sentence is wrapped in a unique 8-character hexadecimal tag:
+
+```
+<a1b2c3d4>First sentence of the document.</a1b2c3d4><e5f6g7h8>Second sentence continues here.</e5f6g7h8>...
+```
+
+### Tag Format Requirements
+
+- Tags must be **8 lowercase hexadecimal characters** (e.g., `a1b2c3d4`)
+- Each tag must be **unique** within the document
+- Tags wrap individual sentences: `<tag>sentence text</tag>`
+- Tags should be contiguous (no whitespace between closing and opening tags)
+
+### Preprocessing with spaCy (Recommended)
+
+```python
+import hashlib
+import spacy
+
+def generate_tag(index: int, sentence: str) -> str:
+    """Generate unique 8-char hex tag from sentence."""
+    return hashlib.md5(f"{index}_{sentence[:50]}".encode()).hexdigest()[:8]
+
+def tag_text(text: str, language: str = "en") -> tuple[str, dict]:
+    """
+    Tag text with XML sentence markers.
+
+    Args:
+        text: Input text to tag
+        language: Language code (en, de, es, fr, it)
+
+    Returns:
+        tuple: (tagged_text, tag_to_sentence_mapping)
+    """
+    # Load appropriate spaCy model
+    models = {"en": "en_core_web_sm", "de": "de_core_news_sm",
+              "es": "es_core_news_sm", "fr": "fr_core_news_sm", "it": "it_core_news_sm"}
+    nlp = spacy.load(models.get(language, "en_core_web_sm"))
+
+    doc = nlp(text)
+    tagged_text = ""
+    tag_mapping = {}
+
+    for i, sent in enumerate(doc.sents):
+        sentence = sent.text.strip()
+        if sentence:
+            tag = generate_tag(i, sentence)
+            tag_mapping[tag] = sentence
+            tagged_text += f"<{tag}>{sentence}</{tag}>"
+
+    return tagged_text, tag_mapping
+
+# Example usage
+text = "This is the first sentence. Here is the second one. And a third."
+tagged, mapping = tag_text(text)
+print(tagged)
+# Output: <a1b2c3d4>This is the first sentence.</a1b2c3d4><e5f67890>Here is the second one.</e5f67890>...
+```
+
+### Installation for Preprocessing
+
+```bash
+pip install spacy langdetect
+python -m spacy download en_core_web_sm  # English
+python -m spacy download de_core_news_sm  # German (optional)
+```
+
+To automatically detect the input language and select the appropriate spaCy model, you can use `langdetect`:
+
+```python
+from langdetect import detect
+
+def detect_language(text: str) -> str:
+    lang_code = detect(text[:1000])  # Sample first 1000 chars
+    lang_map = {"de": "German", "en": "English", "es": "Spanish",
+                "fr": "French", "it": "Italian"}
+    return lang_map.get(lang_code, "English")  # Default to English
+```
+
+For non-enhanced languages, English-style sentence segmentation is used as fallback, which may be suboptimal for languages with different punctuation conventions (e.g., Chinese, Japanese).
+
+---
+
+## Output Format
+
+The model outputs a JSON object with three keys:
+
+```json
+{
+  "structure": "Planning text describing how the summary will be organized...",
+  "xml_tags": ["<a1b2c3d4>", "<e5f67890>", "<12345678>"],
+  "summary": "The document discusses... [<a1b2c3d4>]. Furthermore... [<e5f67890>]."
+}
+```
+
+### Output Keys
+
+| Key | Type | Description |
+|-----|------|-------------|
+| `structure` | string | Internal reasoning about the content and planned summary structure. Shows which topics will be covered and in what order. |
+| `xml_tags` | array | List of XML tags that will be cited in the summary. Each tag corresponds to a source sentence. Tags are listed in the order they appear in the source. |
+| `summary` | string | The actual summary with inline XML tag references in square brackets (e.g., `[<a1b2c3d4>]`). Each referenced tag must appear in the `xml_tags` list and exist in the input. |
+
+### Reference Format in Summary
+
+Citations appear inline immediately after the statement they support:
+
+```
+The report found significant budget overruns [<a1b2c3d4>]. This was attributed to
+supply chain disruptions [<e5f67890>] and labor shortages [<12345678>].
+```
+
+<details>
+<summary><b>Example Output (click to expand)</b></summary>
+
+```json
+{
+  "structure": "This government report examines infrastructure spending over the past fiscal year. I will structure the summary as follows: 1) Executive overview with key findings, 2) Budget analysis section covering the main expenditures, 3) Regional breakdown of infrastructure projects, 4) Recommendations from the oversight committee. I will select 8 XML tags distributed across these sections to provide comprehensive source grounding.",
+  "xml_tags": [
+    "<9f2a1b3c>",
+    "<d4e5f678>",
+    "<90ab12cd>",
+    "<ef345678>",
+    "<9abc0def>",
+    "<12345678>",
+    "<abcdef90>",
+    "<78901234>"
+  ],
+  "summary": "The Congressional Budget Office's annual infrastructure report reveals that federal infrastructure spending reached $284 billion in fiscal year 2024, representing a 12% increase from the previous year [<9f2a1b3c>]. The majority of funds were allocated to transportation projects, with highway maintenance and bridge repair accounting for 45% of total expenditures [<d4e5f678>].\n\nThe report identifies significant regional disparities in infrastructure investment. Northeastern states received disproportionately higher funding due to aging infrastructure requiring immediate attention [<90ab12cd>], while southern and western regions saw increased allocations for new construction projects [<ef345678>]. Rural communities continued to face challenges in securing adequate funding for basic infrastructure needs [<9abc0def>].\n\nThe oversight committee raised concerns about cost overruns in several major projects, with the average project exceeding initial estimates by 23% [<12345678>]. The committee recommends implementing stricter oversight mechanisms and requiring more detailed feasibility studies before project approval [<abcdef90>]. Additionally, the report suggests exploring public-private partnerships as a means to supplement federal funding and improve project efficiency [<78901234>]."
+}
+```
+
+</details>
+
+---
+
+## Handling Very Long Contexts
+
+The model supports a 128k token context window natively. For longer documents (tested up to 2 million tokens), use the iterative approach.
+
+### Approach 1: Oneshot (Up to 128k tokens)
+
+For documents within the context limit, use the standard prompt with `PROMPT_SUMMARY`:
+
+```python
+prompt = f"""You are a professional summarizer, following all given instructions with the utmost care.
+
+<text>
+{tagged_text}
+</text>
+
+# Output Format
+The output must be in JSON format with the following structure:
+1. A "structure" string containing your thoughts about the content and structure of the summary
+2. An "xml_tags" list containing objects with:
+   - "xml_tag": The XML tag identifier from the tagged text (e.g., "<a1b2c3d4>")
+3. A "summary" string containing the actual summary with inline XML tag references
+
+# Instructions
+...
+
+Parameters:
+- Word count (excl. XML tags): {word_count}
+- Number of XML tags: {number_of_xml_tags}
+- Language: {language}
+"""
+```
+
+**Output:** JSON with `structure`, `xml_tags`, and `summary`
+
+---
+
+### Approach 2: Iterative (128k+ tokens)
+
+For documents exceeding the context limit, use a two-step iterative approach that preserves grounding quality:
+
+#### Step 1: Partial Summaries (`PROMPT_SUMMARY_PARTIAL`)
+
+Split the document into chunks and summarize each independently:
+
+```python
+prompt_partial = f"""You are a professional summarizer, following all given instructions with the utmost care.
+
+This is a section of a larger document. Create a partial summary that will later be combined with other sections.
+
+<text>
+{chunk_tagged_text}
+</text>
+
+# Output Format
+The output must be in JSON format with the following structure:
+1. A "structure" string containing your thoughts about the content and structure of the summary
+2. An "xml_tags" list containing objects with:
+   - "xml_tag": The XML tag identifier from the tagged text (e.g., "<a1b2c3d4>")
+3. A "summary" string containing the actual summary with inline XML tag references
+
+# Instructions
+1. Select {number_of_xml_tags} XML tags that capture the most significant data and facts.
+2. Begin with a brief introduction of the section's main topics (no executive summary for partial summaries).
+3. Structure the summary in coherent paragraphs with at least one XML tag reference each.
+4. The summary should be 300-600 words long (without the XML tags).
+5. Only include title/author if explicitly mentioned in this section.
+...
+"""
+```
+
+**Output per chunk:** JSON with `structure`, `xml_tags`, and `summary` (300-600 words each)
+
+#### Step 2: Final Merge (`PROMPT_SUMMARY_PARTIAL_LAST`)
+
+Combine all partial summaries into a coherent final summary:
+
+```python
+# Concatenate all partial summary outputs
+partial_summaries_text = "\n\n".join([
+    f"--- Section {i+1} ---\n{partial_output}"
+    for i, partial_output in enumerate(partial_outputs)
+])
+
+prompt_final = f"""You are a professional summarizer, following all given instructions with the utmost care.
+
+You are given partial summaries from a larger document. Combine them into a coherent final summary.
+
+<partial_summaries>
+{partial_summaries_text}
+</partial_summaries>
+
+# Output Format
+The output must be in JSON format with the following structure:
+1. A "structure" string containing your thoughts about the content and structure of the summary
+2. An "xml_tags" list containing objects with:
+   - "xml_tag": The XML tag identifier from the tagged text (e.g., "<a1b2c3d4>")
+3. A "summary" string containing the actual summary with inline XML tag references
+
+# Instructions
+1. Select the {number_of_xml_tags} most significant XML tags from the partial summaries.
+   Copy the XML tags verbatim, ensuring they represent key points from different sections.
+2. Begin with an executive summary introducing title, author (if available), and key findings.
+3. Structure the summary in coherent paragraphs following a coherent thread.
+4. Each XML tag must appear exactly once. Use only XML tags from the partial summaries.
+5. Don't repeat content that is very similar or identical in multiple partial summaries.
+...
+"""
+```
+
+**Final Output:** JSON with `structure`, `xml_tags`, and `summary`
+
+#### How Grounding Quality is Maintained
+
+The iterative approach preserves source grounding through careful XML tag propagation:
+
+1. **Tag Extraction**: Each partial summary extracts XML tags from its chunk, linking claims to source sentences
+2. **Tag Preservation**: The final merge prompt explicitly instructs to "copy XML tags verbatim" from partials
+3. **No Hallucinated Tags**: The final summary can only reference tags that were already validated in partial summaries
+4. **Distributed Coverage**: By selecting tags "from different sections," the final summary maintains broad source coverage
+
+This ensures that even for 2M+ token documents, every claim in the final summary traces back to a specific source sentence.
+
+### Recommended Parameters
+
+| Summary Length | Word Count | XML Tags |
+|---------------|------------|----------|
+| Short | ~100 words | 3 tags |
+| Medium | ~250 words | 6 tags |
+| Long | ~500 words | 12 tags |
+
+---
+
+## Usage
+
+For production use, we provide ready-to-use prompt templates in [`prompts.py`](prompts.py). This file contains:
+
+- `PROMPT_SUMMARY`: Standard single-pass summarization prompt
+- `PROMPT_SUMMARY_PARTIAL`: Prompt for creating partial summaries of document chunks
+- `PROMPT_SUMMARY_PARTIAL_LAST`: Prompt for merging partial summaries into a final summary
+
+**Resource-constrained environments:** The iterative two-step approach is not only useful for very long documents—it also enables deployment on hardware with limited VRAM. The model was trained on a broad range of chunk sizes, so partial summaries work reliably even with smaller context windows (e.g., 5k token chunks). This flexibility allows you to adjust chunk sizes to match your available GPU memory.
+
+### With vLLM (Recommended for Production)
+
+```python
+from vllm import LLM, SamplingParams
+
+# Load model
+llm = LLM(
+    model="ellamind/sui-1-24b",
+    tensor_parallel_size=4,  # Adjust based on available GPUs
+    dtype="bfloat16",
+    tokenizer_mode="mistral",
+    max_model_len=128000,
+    trust_remote_code=True,
+)
+
+# Prepare prompt
+prompt = f"""You are a professional summarizer, following all given instructions with the utmost care.
+
+<text>
+{tagged_text}
+</text>
+
+# Output Format
+The output must be in JSON format with the following structure:
+1. A "structure" string containing your thoughts about the content and structure of the summary
+2. An "xml_tags" list containing the XML tag identifiers from the tagged text (e.g., "<a1b2c3d4>")
+3. A "summary" string containing the actual summary with inline XML tag references
+
+# Instructions
+1. Start by thinking about and explaining the structure and content of your summary. Select {num_tags} XML tags from the tagged text that capture the most significant data and facts.
+2. Begin with an executive summary introducing the title, author (if available), and key findings.
+3. Structure the summary in coherent paragraphs. Every paragraph should contain at least one XML tag reference.
+4. Reference XML tags inline in square brackets (e.g., [<a1b2c3d4>]) immediately after the statement they support.
+5. Each XML tag must appear exactly once in the summary.
+6. Avoid a concluding paragraph that merely restates points.
+7. Do not use bullet points or headings unless explicitly requested.
+
+# Custom Instruction
+{custom_instruction}
+
+Parameters:
+- Word count (excl. XML tags): {word_count}
+- Number of XML tags: {num_tags}
+- Language: {language}
+"""
+
+# Generate
+sampling_params = SamplingParams(max_tokens=8192, temperature=0.0)
+outputs = llm.chat([[{"role": "user", "content": prompt}]], sampling_params)
+result = outputs[0].outputs[0].text
+```
+
+### With Transformers
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model = AutoModelForCausalLM.from_pretrained(
+    "ellamind/sui-1-24b",
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    trust_remote_code=True,
+)
+tokenizer = AutoTokenizer.from_pretrained("ellamind/sui-1-24b")
+
+messages = [{"role": "user", "content": prompt}]
+inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
+outputs = model.generate(inputs, max_new_tokens=8192, temperature=0.0, do_sample=False)
+result = tokenizer.decode(outputs[0], skip_special_tokens=True)
+```
+
+---
+
+## Language Support
+
+### Enhanced Languages (Fine-tuned)
+
+The model was fine-tuned with training data in these languages, providing optimal summarization quality:
+
+| Language | Code | Tagging Support |
+|----------|------|-----------------|
+| English | `en` | `en_core_web_sm` |
+| German | `de` | `de_core_news_sm` |
+| Spanish | `es` | `es_core_news_sm` |
+| French | `fr` | `fr_core_news_sm` |
+| Italian | `it` | `it_core_news_sm` |
+
+### Inherited Languages (Base Model)
+
+The following languages are supported through the Mistral Small 3.2 base model. Summarization works but may have reduced quality compared to enhanced languages:
+
+| Category | Languages |
+|----------|-----------|
+| European | Portuguese, Dutch, Polish, Russian, Swedish, Ukrainian, Romanian, Czech, Greek, Hungarian |
+| Asian | Chinese, Japanese, Korean, Vietnamese, Indonesian, Thai, Hindi |
+| Middle Eastern | Arabic, Turkish, Persian |
+
+---
+
+## Limitations
+
+- Requires preprocessing of input text with XML tags
+- Maximum single-pass context of 128k tokens
+- JSON output parsing may occasionally fail; implement retry logic for production use
+
+---
+
+## Citation
+
+```bibtex
+@article{droste2025sui1,
+  title={sui-1: Grounded and Verifiable Long-Form Summarization},
+  author={Droste, Benedikt and Harries, Jan Philipp and Idahl, Maximilian and Pl{\"u}ster, Bj{\"o}rn},
+  journal={arXiv preprint arXiv:2601.08472},
+  year={2025}
+}
+```
+
+---
+
+## License
+
+This model is released under the Apache 2.0 license, consistent with the base Mistral model.

config.json

@@ -0,0 +1,48 @@
+{
+  "architectures": [
+    "Mistral3ForConditionalGeneration"
+  ],
+  "dtype": "bfloat16",
+  "image_token_index": 10,
+  "model_type": "mistral3",
+  "multimodal_projector_bias": false,
+  "projector_hidden_act": "gelu",
+  "spatial_merge_size": 2,
+  "text_config": {
+    "attention_dropout": 0.0,
+    "dtype": "bfloat16",
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 5120,
+    "initializer_range": 0.02,
+    "intermediate_size": 32768,
+    "max_position_embeddings": 131072,
+    "model_type": "mistral",
+    "num_attention_heads": 32,
+    "num_hidden_layers": 40,
+    "num_key_value_heads": 8,
+    "rms_norm_eps": 1e-05,
+    "rope_theta": 1000000000.0,
+    "sliding_window": null,
+    "use_cache": true,
+    "vocab_size": 131072
+  },
+  "transformers_version": "4.57.1",
+  "vision_config": {
+    "attention_dropout": 0.0,
+    "dtype": "bfloat16",
+    "head_dim": 64,
+    "hidden_act": "silu",
+    "hidden_size": 1024,
+    "image_size": 1540,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "model_type": "pixtral",
+    "num_attention_heads": 16,
+    "num_channels": 3,
+    "num_hidden_layers": 24,
+    "patch_size": 14,
+    "rope_theta": 10000.0
+  },
+  "vision_feature_layer": -1
+}

document.txt

@@ -0,0 +1,101 @@
+In 1973, Prevost was planning to attend Tolentine College, an Augustinian seminary in Olympia Fields, Illinois, as part of the order's pre-novitiate program, but it closed that same year.[9][48] He promptly enrolled in Villanova University, an Augustinian college located near Philadelphia, where he earned a Bachelor of Science (BS) degree in mathematics in 1977.[12][49][50] At Villanova, he took elective courses in Hebrew and Latin, read the writings of Saint Augustine, and discussed the work of theologian Karl Rahner with other students.[9] He lived in the Augustinian friary, and was remembered as "particularly committed... to missionary work," and being "the most community minded" among the students.[51] Prevost also worked as a cemetery groundskeeper at Saint Denis Catholic Church in Havertown, Pennsylvania, while studying at Villanova.[52]
+
+Entry into Augustinians
+On September 1, 1977, Prevost entered the Order of Saint Augustine's novitiate in the Province of Our Mother of Good Counsel. He resided for one year at Immaculate Conception Church in the Gate District neighborhood of St. Louis, Missouri.[53][54][55][56] In the summer of 1978, Prevost spent three months in Clinical Pastoral Education at Abbott Northwestern Hospital in Minneapolis.[57][58][59] He took his first vows on September 2, 1978, and his solemn vows on August 29, 1981.[11][12]
+
+He returned to his father's native Hyde Park to obtain a Master of Divinity (MDiv) from Catholic Theological Union in 1982 and taught physics and math at St. Rita of Cascia High School in the Wrightwood neighborhood of Chicago during his studies.[13][60] As his spiritual director, a guide to the Christian life, he chose a religious sister, Sister Lyn Osiek, RSCJ, the supervisor of his theological reflection class. She described him as "calm and steady... a person who was at peace with himself."[61]
+
+Ordination and missionary work in Peru
+On September 10, 1981, Prevost was ordained a deacon by Thomas Gumbleton at St. Clare of Montefalco Parish in Grosse Pointe Park, Michigan. That same year, Prevost concluded his studies at Catholic Theological Union.[62][11] Prevost was ordained a priest in Rome, at the Chapel of Saint Monica of the Augustinians by Archbishop Jean Jadot on June 19, 1982.[11][12][63]
+
+Prevost earned a Licentiate of Canon Law (JCL) in 1984, followed by a Doctor of Canon Law (JCD) degree in 1987, both from the Pontifical University of Saint Thomas Aquinas in Rome.[12] During this period, he also learned Italian.[9] His doctoral thesis was a canon law study of the role of local priors in the Order of Saint Augustine.[64]
+
+
+Prevost spent his novitiate at Immaculate Conception Church in St. Louis, Missouri.
+"Padre Roberto" joined the Augustinian mission in Peru in 1985, serving as chancellor of the Territorial Prelature of Chulucanas (1985–1986), as well as an aide to bishop John McNabb. Much of his work in his first year in Peru was focused on disaster relief following severe El Niño rainstorms. [11][65] In 1987, after defending his doctoral thesis, he was vocation director and missions director of the Augustinian Province of Our Mother of Good Counsel in Olympia Fields, Illinois, and worked with the faculty of the Augustinian Novitiate in Oconomowoc, Wisconsin, before returning to Peru in 1988.[12][66] During his time in Peru, Prevost met and appreciated the Dominican priest and theologian Gustavo Gutierrez, a pioneer of liberation theology.[63] He also learned and mastered the Spanish language during this period.[9]
+
+Prevost spent a decade heading the Augustinian seminary in Trujillo, teaching canon law in the diocesan seminary,[12] serving as prefect of studies, acting as a judge in the regional ecclesiastical court, and working in parish ministry on the city's outskirts, coming into contact with poor farmers and Peruvian indigenous peoples.[67][65]
+
+
+Cathedral Basilica of Saint Mary, Trujillo. As a young missionary, Prevost served in various roles in the Archdiocese of Trujillo.
+He proved successful in the Augustinians' efforts to recruit Peruvians for the priesthood and leadership positions in the order.[68] He organized support for Venezuelan refugees to Peru despite discrimination against the Venezuelans.[69]
+
+During the Fujimorato era, Prevost criticized the actions of then-President Alberto Fujimori, placing special emphasis on the victims of the Peruvian Army, especially the Colina Group, during the period of terrorism in Peru, as well as on political corruption. In 2017, he criticized President Pedro Pablo Kuczynski's decision to pardon Fujimori, and called upon Fujimori "to personally apologize for some of the great injustices that were committed".[70] His years in Peru gave him personal knowledge of political violence and inequality; at times, as a missionary, he traveled by horse on difficult roads to reach the isolated communities in the valleys of Lambayeque.[71] He also stood out as a defender of human rights of the population of the Norte Chico region against the violence of the Marxist–Leninist–Maoist guerrilla organization Shining Path.[72][73] However, Trujilio, the area where Prevost served, was not particularly affected by the Shining Path.[65]
+
+Prior provincial and prior general (1998–2013)
+
+St. Thomas School in Hyde Park, Chicago
+In 1998, Prevost was elected Prior Provincial of the Order of St. Augustine's Province of Our Mother of Good Counsel based in Chicago, assuming the role on March 8, 1999.[12] In 2000, he permitted James Ray, an Augustinian priest, to reside at St. John Stone Friary in Chicago under supervision. Ray had been suspended from public ministry since 1991 due to credible accusations that he had sexually abused minors. Prevost's allowing of the placement of Ray in a friary at the request of the Archdiocese of Chicago and close to St. Thomas School was the subject of reporting in 2021, prior to his appointment to the Vatican.[74] According to the Augustinians, "the location was selected because of the supervision the priest would receive."[61] After American bishops implemented stricter rules in the Dallas Charter, Ray was moved to other housing in 2002.[74][75][e]
+
+Elected prior general of the Order of Saint Augustine in 2001, Prevost served two consecutive, six-year terms until 2013.[76] During his tenure as the global head of the Augustinian order, Prevost lived and worked in Rome, but frequently traveled around the world. It was in that capacity that in 2004, he visited Buenos Aires and met Cardinal Jorge Bergoglio, the future Pope Francis, for the first time. Their early encounters are reported to have not gone well. Prevost later explained that at the time, he thought he would never become a bishop after Bergoglio was elected Pope in 2013, but has declined to reveal what they initially disagreed upon. However, they were able to reconcile their differences before Prevost left Rome for Chicago in 2013.[9] Soon after his election as prior general, Prevost personally oversaw the creation of a web site for the Augustinians.[77]
+
+From 2013 to 2014, Prevost served as director of formation at the Convent of St. Augustine in Chicago, and as first councilor and provincial vicar of the province of Our Mother of Good Counsel.[12]
+
+Bishop of Chiclayo (2015–2023)
+
+St. Mary's Cathedral in Chiclayo, Peru, where Prevost was diocesan bishop from 2015 to 2023
+On November 3, 2014, Pope Francis appointed Prevost as the apostolic administrator of the Diocese of Chiclayo in northern Peru and the titular bishop of Sufar.[78] He was consecrated on December 12, 2014, at St. Mary Cathedral by Archbishop James Green, Apostolic Nuncio to Peru.[79][80] On September 26, 2015, he was named Bishop of Chiclayo.[81][82] As required by a 1980 agreement between the Holy See and Peru,[83] Prevost became a naturalized Peruvian citizen before becoming bishop.[60] At the time of his appointment, Chiclayo was described as a conservative diocese, having been staffed by Opus Dei priests for many decades.[65]
+
+Prevost developed a close bond with the diocese of Chiclayo, directing a special message to them in his first message as pope, calling them "my beloved diocese", "faithful people" and "faithful church".[84][85] He was seen as being "very faithful to the Church's social doctrine" and organized relief efforts during calamities. He also supported projects in favor of Venezuelan refugees and encouraged vocations in youth groups, giving strong support to the seminary.[86] He helped victims of trafficking and sex workers learn new skills and trades, and invited them to spiritual retreats, while ensuring that volunteer workers have adequate training. He is remembered for his capacity to listen and have discussions with different groups before taking action, and working with institutions when facing controversial issues.[87] He established a diocesan Commission on Integral Ecology and appointed a woman to lead it.[84] According to interviews done by The Pillar, people in the diocese described their bishop as having a "balanced" personality, a "calming presence", showing "great charity and great clarity" in doctrinal matters, and a competent administrator and bridge-builder open to dialogue and never hurtful, with "apostolic zeal"[86]
+
+Prevost as Bishop of Chiclayo in 2018, commenting on extreme poverty in the region (in Spanish)
+On July 13, 2019, Prevost was appointed a member of the Congregation for the Clergy,[88] and on April 15, 2020, he became apostolic administrator of Callao.[89][f] On November 21, 2020, he joined the Congregation for Bishops.[90] Within the Episcopal Conference of Peru, he served on the permanent council (2018–2020) and was elected president of its Commission for Education and Culture in 2019, also contributing to Caritas Peru.[91][92] His presence in the permanent council of the episcopal conference was seen by Cardinal Joseph Tobin as "extraordinary for someone who was not born in Peru. Clearly, he has a broad degree of acceptance in the divided group."[93] Prevost had a private audience with Pope Francis on March 1, 2021,[94] sparking speculation about a new role in Chicago or Rome.[95]
+
+Prevost has been accused of covering up sexual abuse during his time in Chiclayo.[96][97] In 2022, alleged victims of abuse in 2007 by priests Ricardo Yesquén Paiva and Eleuterio Vásquez Gonzáles said the Diocese of Chiclayo failed to investigate their cases.[98] The Diocese of Chiclayo stated that Prevost followed proper procedures, met with Ana María Quispe and her sisters in April 2022 to personally attend the victims, encouraged them to initiate a civil action, and initiated a canonical investigation, the results of which he sent to the Dicastery for the Doctrine of the Faith.[99][100] The sisters stated in 2024 that no full penal canonical investigation occurred, and an investigation by América Televisión concluded that the church's investigation was not thorough.[101][102]
+
+In response the diocese of Chiclayo stated the allegations of the TV program were false, calling on it to correct its inaccuracies. The diocese said that the accused priest "never admitted" to the accusations, that Prevost prohibited him from exercising the priestly ministry and the priest transferred residence to his family's home. Prevost submitted the dossier to the Vatican, which gave a "pro nunc" decision or temporary dismissal. The diocese later reopened the case and sent it to the Vatican anew, and thus it remains "open up to now."[103]
+
+Speaking to the Peruvian newspaper La República while Bishop of Chiclayo, Prevost said: "If you are a victim of sexual abuse by a priest, report it. We reject cover-ups and secrecy; that causes a lot of harm. We have to help people who have suffered due to wrongdoing."[104][105][g] Journalist Pedro Salinas [es], who investigated and exposed crimes committed by members of the now-defunct Sodalitium Christianae Vitae—including sexual, physical, and psychological abuse—has argued that Prevost always expressed his support for the victims and was one of the most reliable clerical authorities in Peru, leading Pope Francis to select him as prefect of bishops. Salinas has written that some of the Peruvian clerics linked to the Sodalitium sought to attack and defame Prevost in retaliation for his role in the Sodalitium's dissolution by Pope Francis due to its sexual abuse scandals, as well as Prevost being near to Francis's political theology.[106][107]
+
+Dicastery for Bishops and cardinalate (2023–2025)
+
+Cardinal Prevost (first from right) during the procession with the body of Pope Francis, April 23, 2025
+
+Prevost during the Consistory in 2023
+By the 2020s, Pope Francis had become close to Prevost and advanced his career.[9][108] On January 30, 2023, Francis appointed Prevost prefect of the Dicastery for Bishops with the title Archbishop-Bishop emeritus of Chiclayo.[109][110] Prevost expressed a preference to stay in Peru, but accepted the appointment.[108] He continued to be active in the Latin American and Caribbean Episcopal Council and participated in the council's meetings in Aguadilla, Puerto Rico, in May 2023.[111]
+
+On September 30, 2023, Francis created Prevost a cardinal with the rank of Cardinal-Deacon and assigned the deaconry of Santa Monica degli Agostiniani.[112] As prefect, he played a critical role in evaluating and recommending episcopal candidates worldwide, increasing his visibility within the church.[113] These roles elevated his prominence as a papal candidate leading into the conclave.[114][115][116] In October 2023, Francis appointed him as a member of seven additional dicasteries,[12][h] and also named him to the Pontifical Commission for the Vatican City State.[12]
+
+On February 6, 2025, Francis promoted Prevost to cardinal-bishop, assigning him as titular bishop of the Suburbicarian Diocese of Albano.[117][118][i] On February 11, he was appointed to the dignity and rank of Bailiff Grand Cross of Honour and Devotion of the Order of Malta by Grand Master Fra' John Dunlap.[120]
+
+Written works before the papacy and theology of leadership
+Main article: Pope Leo XIV bibliography
+Given his focus on mission, the writings of Prevost before his election were few.[121][122] When he was global head of the Augustinians, the Rule and Constitutions of the Order of Saint Augustine was "issued by Authority of the Prior General, Robert F. Prevost" in 2002.[123]
+
+Aside from four journal articles published mostly in Augustininan journals, his main work of scholarship is his doctoral dissertation, The office and authority of the local Prior in the Order of Saint Augustine, published by the Pontifical University of Saint Thomas Aquinas in 1987. The work has been noted for its legal precision, Augustinian roots, postconciliar engagement and historical depth.[124] Much of the commentary see this work as a window to his "vision of the church" and "theology of leadership."[125][126][127]
+
+In his thesis, Prevost sees that authority is received from above, and therefore "the substance of the office of the superior is to obey; to obey the will of God and to put great effort into trying to know it, to formulate it and to specify it for his subjects."[124] Leadership is based on Jesus Christ's threefold offices of teaching, sanctifying and governing, who sanctifies by prayer and builds community by being centered on the Mass, having "the care of the Brothers, 'especially in spiritual matters.'" The prior must help make the sacraments regular and reverent, protecting "each brother's freedom of conscience while promoting 'frequent' confession."[125]
+
+Based on St. Augustine's concept, Prevost sees authority as a service.[128] He says that service means "listening to what the Spirit is saying in His people so that His projects can be carried out freely and willingly. The Prior then is called to listen, so that together they can discern and implement what the Spirit inspires."[126]
+
+Papacy (2025–present)
+Election
+See also: 2025 conclave
+
+Pope Leo XIV waving from the loggia of St. Peter's Basilica in his first public appearance
+In pre-conclave speculation, Prevost was considered a dark horse compared to more prominent papabili,[129][130] although he was noted to be an ally of Pope Francis and a possible compromise candidate.[129][131] His American nationality had been regarded as a potential stumbling block to his candidacy, reflecting unease about enhancing the United States's geopolitical power.[8][132] Prevost was well aware of this, explaining to a friend by text message: "I'm an American, I can't be elected".[9] Supporters argued that he represented a "dignified middle of the road".[133]
+
+Prevost was elected pope on May 8, 2025, the second day of the conclave, on the fourth ballot. White smoke appeared from the Sistine Chapel at 18:07 CEST (UTC+2), signifying to the public that a pope had been chosen.[134] After accepting his election and adopting his papal name, Leo embraced his fellow cardinals upon exiting the Sistine Chapel. Cardinal Dominique Mamberti, the cardinal protodeacon, made the traditional Latin proclamation, Habemus papam, announcing Pope Leo XIV to the public for the first time from the central loggia of St. Peter's Basilica.[135][136] According to Italian law enforcement, by the time of the habemus papam announcement, there were up to 150,000 in the square.[137]
+
+Leo appeared wearing the traditional papal stole and mozzetta,[138] vestments Pope Francis did not wear upon his greeting of the world following his election.[139][140] Leo's pectoral cross was one he wore during the conclave, a gift from the Postulator General of the Augustinian Order when he became a cardinal, containing relics from Saint Augustine and his mother Saint Monica, among others.[141] He then delivered his first address to Rome and the world in Italian, advocating for peace and dialogue, expressing gratitude for the legacy of Pope Francis and quoting Saint Augustine; he also greeted his former diocese of Chiclayo, Peru, in Spanish. After the speech, he imparted his first Urbi et Orbi blessing in Latin.[142][143][144]
+
+Leo XIV is the first pope from the Order of Saint Augustine and the second pope from the Americas (after Pope Francis), and is counted as the 267th pope.[145][12][146][j] He is a dual citizen of Peru and the United States.[147][148][149] He is the first American pope,[150] in the sense of being the first born in the United States.[151][152][153] He is the second to be a native English speaker, after the English-born Adrian IV (r. 1154–1159), who would have spoken Middle English.[154] Leo is also the first pope born after World War II and during the Cold War, and thus the first to be born in the Baby Boomer generation.[155][156] While Leo is the first pope from the Order of Saint Augustine,[157][158] six previous popes belonged to other orders which follow the Augustinian Rule, the last one prior to Leo being Pope Eugene IV (1431–1447).[159]
+
+Inauguration
+Main article: Inauguration of Pope Leo XIV
+
+Pope Leo XIV's first popemobile ride during his inauguration
+On May 9, the day after his election, Leo celebrated his first Mass as pope in the Sistine Chapel before the assembled College of Cardinals. During the Mass, he preached against the lack of faith in the world, and spoke of a church that would act as a "beacon that illuminates the dark nights of this world".[160] Media reported that Leo would take up permanent residence in the papal apartments in the Apostolic Palace rather than in the Domus Sanctae Marthae where Francis lived.[161][162][163]
+
+Leo's inauguration Mass was held on May 18 in St. Peter's Square.[164][165][166] During the Mass, he received the pallium and his Ring of the Fisherman before 12 representatives of the people of God, including cardinals and bishops, who vowed obedience to the new pope.[167] On May 25, he was formally installed as Bishop of Rome in a ceremony at the Archbasilica of Saint John Lateran.[168]
+
+Post-election reactions and impact
+Main article: International reactions to the election of Pope Leo XIV
+In Chiclayo, thousands crowded the city square for a Mass of Thanksgiving following Leo's election. Similar Masses were celebrated all over Peru.[169] In the US, over 30,000 Americans gathered at Chicago's Rate Field for a youth-focused celebration with a video message of Leo XIV who urged Catholics to be "beacons of hope".[170]
+
+Political leaders sent their congratulations. President Donald Trump stated that it is "a Great Honor for our Country", while Vice President JD Vance, a Catholic, talked about Catholics and other Christians praying "for his successful work leading the Church". The European Council President António Costa said it was "a moment of hope".[171]
+
+Early acts
+On June 5, 2025, Leo nominated Joseph Lin Yuntuan as auxiliary bishop of Fuzhou, which was approved by Chinese authorities on June 11 as part of a 2018 agreement between the People's Republic of China and the Holy See under Pope Francis.[172] On July 5, Leo appointed French archbishop Thibault Verny to succeed Seán Patrick O'Malley as president of the Pontifical Commission for the Protection of Minors. O'Malley endorsed the selection, describing Verny as "a collaborative leader committed to advancing the global adoption of protection and safeguarding, to as best possible assure the safety of those in the care of the Church throughout the world."[173] On July 6, Leo revived the papal practice of taking summer residence at the Castel Gandolfo, which Francis had discontinued.[174]
+
+On September 7, 2025, Leo presided over the canonization of Carlo Acutis and Pier Giorgio Frassati at a mass in Saint Peter's Square.[175] A month later, his first Apostolic journey, a visit to Turkey and Lebanon, was announced for November 27 through December 2.[176] On November 21, 2025, Leo signed an amendment to the 2023 law allowing women to become president of the Pontifical Commission for the Vatican City State, removing a requirement that would-be officeholders must hold the rank of cardinal.[177] On November 27, 2025, Leo made his first overseas visit as pope, arriving in Turkey to commemorate the 1,700th anniversary of the Council of Nicaea with Patriarch Bartholomew of the Eastern Orthodox Church;[178][179] while in Istanbul, he visited the Blue Mosque.[180][181] Leo later visited Lebanon, visiting and praying at the site of the 2020 Beirut explosion[182] and later the tomb of Charbel Makhlouf[183] before returning to Rome on December 2.[182]"

example.py

@@ -0,0 +1,203 @@
+#!/usr/bin/env -S uv run --script
+# /// script
+# requires-python = ">=3.12"
+# dependencies = [
+#     "vllm>=0.11.0",
+#     "spacy>=3.7.0",
+#     "mistral_common>=1.5.0",
+#     "en-core-web-sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl",
+# ]
+#
+# [tool.uv]
+# no-build = true
+# index-strategy = "unsafe-best-match"
+# extra-index-url = ["https://download.pytorch.org/whl/cu128"]
+# ///
+"""
+Minimal end-to-end example for sui-1-24b summarization.
+
+Usage:
+    # Summarize a file
+    uv run example.py document.txt
+
+    # Summarize inline text
+    uv run example.py --text "Your long text here..."
+
+    # With custom parameters
+    uv run example.py document.txt --words 300 --tags 8 --language en
+"""
+
+import argparse
+import hashlib
+import json
+import re
+import sys
+from pathlib import Path
+
+# Lazy imports for faster --help
+def main():
+    parser = argparse.ArgumentParser(
+        description="Summarize text using sui-1-24b with source grounding",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog=__doc__,
+    )
+    parser.add_argument("input", nargs="?", help="Input file path (or use --text)")
+    parser.add_argument("--text", "-t", help="Input text directly")
+    parser.add_argument("--words", "-w", type=int, default=250, help="Target word count (default: 400)")
+    parser.add_argument("--tags", "-n", type=int, default=4, help="Number of XML tags to cite (default: 10)")
+    parser.add_argument("--language", "-l", default="en", choices=["en", "de", "es", "fr", "it"], help="Language (default: en)")
+    parser.add_argument("--model", "-m", default="ellamind/sui-1-24b", help="Model path or HF repo")
+    parser.add_argument("--tensor-parallel", "-tp", type=int, default=1, help="Tensor parallel size (default: 1)")
+    parser.add_argument("--raw", action="store_true", help="Print raw JSON output instead of formatted")
+    args = parser.parse_args()
+
+    # Get input text
+    if args.text:
+        text = args.text
+    elif args.input:
+        text = Path(args.input).read_text()
+    else:
+        parser.error("Provide input file or --text")
+
+    # Import heavy dependencies only when needed
+    import spacy
+    from vllm import LLM, SamplingParams
+
+    # Load spaCy model for sentence segmentation
+    # Note: Only English is bundled. For other languages, install the model first:
+    #   pip install https://github.com/explosion/spacy-models/releases/download/de_core_news_sm-3.8.0/de_core_news_sm-3.8.0-py3-none-any.whl
+    spacy_models = {
+        "en": "en_core_web_sm",
+        "de": "de_core_news_sm",
+        "es": "es_core_news_sm",
+        "fr": "fr_core_news_sm",
+        "it": "it_core_news_sm",
+    }
+    try:
+        nlp = spacy.load(spacy_models[args.language])
+    except OSError:
+        print(f"Error: spaCy model '{spacy_models[args.language]}' not found.")
+        print(f"For English, this should be bundled automatically.")
+        print(f"For other languages, install the model first:")
+        print(f"  pip install https://github.com/explosion/spacy-models/releases/download/{spacy_models[args.language]}-3.8.0/{spacy_models[args.language]}-3.8.0-py3-none-any.whl")
+        sys.exit(1)
+
+    # Tag sentences with unique XML identifiers
+    print("Tagging sentences...")
+    doc = nlp(text)
+    tagged_text = ""
+    tag_mapping = {}
+
+    for i, sent in enumerate(doc.sents):
+        sentence = sent.text.strip()
+        if sentence:
+            tag = hashlib.md5(f"{i}_{sentence[:50]}".encode()).hexdigest()[:8]
+            tag_mapping[tag] = sentence
+            tagged_text += f"<{tag}>{sentence}</{tag}>"
+
+    print(f"Tagged {len(tag_mapping)} sentences")
+
+    # Build prompt
+    language_names = {"en": "English", "de": "German", "es": "Spanish", "fr": "French", "it": "Italian"}
+    prompt = f"""You are a professional summarizer, following all given instructions with the utmost care.
+
+<text>
+{tagged_text}
+</text>
+
+# Output Format
+The output must be in JSON format with the following structure:
+1. A "structure" string containing your thoughts about the content and structure of the summary
+2. An "xml_tags" list containing the XML tag identifiers from the tagged text (e.g., "<a1b2c3d4>")
+3. A "summary" string containing the actual summary with inline XML tag references
+
+# Instructions
+1. Start by thinking about and explaining the structure and content of your summary. Select {args.tags} XML tags from the tagged text that capture the most significant data and facts.
+2. Begin with an executive summary introducing the title, author (if available), and key findings.
+3. Structure the summary in coherent paragraphs. Every paragraph should contain at least one XML tag reference.
+4. Reference XML tags inline in square brackets (e.g., [<a1b2c3d4>]) immediately after the statement they support.
+5. Each XML tag must appear exactly once in the summary.
+6. Avoid a concluding paragraph that merely restates points.
+7. Do not use bullet points or headings unless explicitly requested.
+
+Parameters:
+- Word count (excl. XML tags): {args.words}
+- Number of XML tags: {args.tags}
+- Language: {language_names[args.language]}
+"""
+
+    # Load model and generate
+    print(f"Loading model: {args.model}")
+    llm = LLM(
+        model=args.model,
+        tensor_parallel_size=args.tensor_parallel,
+        dtype="bfloat16",
+        tokenizer_mode="mistral",
+        trust_remote_code=True,
+        limit_mm_per_prompt={"image": 0},  # Disable vision encoder for text-only
+    )
+
+    print("Generating summary...")
+    sampling_params = SamplingParams(max_tokens=4096, temperature=0.0)
+    outputs = llm.chat([[{"role": "user", "content": prompt}]], sampling_params)
+    result = outputs[0].outputs[0].text
+
+    # Parse and display output
+    if args.raw:
+        print(result)
+        return
+
+    try:
+        # Extract JSON from response
+        json_match = re.search(r'\{[\s\S]*\}', result)
+        if json_match:
+            data = json.loads(json_match.group())
+
+            print("\n" + "=" * 60)
+            print("SUMMARY")
+            print("=" * 60 + "\n")
+
+            summary = data.get("summary", "")
+
+            # Replace XML tags with highlighted source references
+            def replace_tag(match):
+                tag = match.group(1)
+                source = tag_mapping.get(tag, "???")
+                # Truncate long sources
+                if len(source) > 80:
+                    source = source[:77] + "..."
+                return f"[{tag}]"
+
+            clean_summary = re.sub(r'\[<([a-f0-9]{8})>\]', replace_tag, summary)
+            print(clean_summary)
+
+            print("\n" + "-" * 60)
+            print("SOURCES")
+            print("-" * 60)
+
+            # Show referenced sources
+            # Handle both formats: ["<tag>"] or [{"xml_tag": "<tag>"}]
+            xml_tags = data.get("xml_tags", [])
+            for tag in xml_tags:
+                if isinstance(tag, str):
+                    clean_tag = tag.strip("<>")
+                elif isinstance(tag, dict) and "xml_tag" in tag:
+                    clean_tag = tag["xml_tag"].strip("<>")
+                else:
+                    continue
+                source = tag_mapping.get(clean_tag, "Not found")
+                if len(source) > 100:
+                    source = source[:97] + "..."
+                print(f"[{clean_tag}] {source}")
+
+        else:
+            print("Could not parse JSON response:")
+            print(result)
+
+    except json.JSONDecodeError as e:
+        print(f"JSON parse error: {e}")
+        print(result)
+
+
+if __name__ == "__main__":
+    main()

generation_config.json

@@ -0,0 +1,8 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "do_sample": true,
+  "eos_token_id": 2,
+  "temperature": 0.15,
+  "transformers_version": "4.57.1"
+}

merge_metadata.json

@@ -0,0 +1,5 @@
+{
+  "base_model": "mistralai/Mistral-Small-3.2-24B-Instruct-2506",
+  "lora_checkpoint": "outputs/lora-mistral3-small-3.2-24b-out-v11",
+  "merged_at": "NVIDIA H100"
+}

model-00001-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ce50851ba63f0676579bda2980687fb1163fcaef943316a1fc9dfe92e0e029a
+size 4883550696

model-00002-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ad4f58e106059022223e77e7e4159815e945ac474f41e5e03d5a1949ecb56b6
+size 4781593336

model-00003-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6b48bae19a6ac6077cf857ea08fe0df7896680305aabdf30fb66021cb66ee8e
+size 4886472224

model-00004-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b33a70eb8e278672070d683d4c0ad442efe9fd5a840a36ac572a683c513f6b7f
+size 4781593376

model-00005-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4929ca9a27b65335df063d38959c070b49ee2015c569e6fdd256b0bebefeab32
+size 4781593368

model-00006-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebe791e39244125e5380ea6c5c9d0d7c842fdc5fa0b839c3fea40941831e3b16
+size 4886472248

model-00007-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:11325d99e44c88afa3661854459e57a994a73df64f69211d7aaaae80fe881e3c
+size 4781593376

model-00008-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9318e3fc1d9afcdeb9a4a153f82b352e6fccbc64d2a209d90a7ddcd18d59f51
+size 4781593368

model-00009-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b107f8f890efd3d2bd651edb1ebd7c8af2080f5855fa12f11efbef5f70459c0
+size 4886472248

model-00010-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b51b85f9ac19d93e7b05d8e9c793c8f011164aa38f4ec2e68f685799cd6aaba2
+size 4571866320

model.safetensors.index.json

@@ -0,0 +1,593 @@
+{
+  "metadata": {
+    "total_parameters": 24011361280,
+    "total_size": 48022722560
+  },
+  "weight_map": {
+    "language_model.lm_head.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.embed_tokens.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.0.input_layernorm.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.0.mlp.down_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.0.mlp.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.0.mlp.up_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.0.post_attention_layernorm.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.0.self_attn.k_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.0.self_attn.o_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.0.self_attn.q_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.0.self_attn.v_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.1.input_layernorm.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.1.mlp.down_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.1.mlp.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.1.mlp.up_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.1.post_attention_layernorm.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.1.self_attn.k_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.1.self_attn.o_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.1.self_attn.q_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.1.self_attn.v_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.10.input_layernorm.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.10.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.10.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.10.mlp.up_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.10.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.10.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.10.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.10.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.10.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.11.input_layernorm.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.11.mlp.down_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.11.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.11.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.11.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.11.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.11.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.11.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.11.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.12.input_layernorm.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.12.mlp.down_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.12.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.12.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.12.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.12.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.12.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.12.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.12.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.13.input_layernorm.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.13.mlp.down_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.13.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.13.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.13.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.13.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.13.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.13.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.13.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.14.input_layernorm.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.14.mlp.down_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.14.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.14.mlp.up_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.14.post_attention_layernorm.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.14.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.14.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.14.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.14.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.15.input_layernorm.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.15.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.15.mlp.gate_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.15.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.15.post_attention_layernorm.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.15.self_attn.k_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.15.self_attn.o_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.15.self_attn.q_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.15.self_attn.v_proj.weight": "model-00004-of-00010.safetensors",
+    "language_model.model.layers.16.input_layernorm.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.16.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.16.mlp.gate_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.16.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.16.post_attention_layernorm.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.16.self_attn.k_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.16.self_attn.o_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.16.self_attn.q_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.16.self_attn.v_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.17.input_layernorm.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.17.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.17.mlp.gate_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.17.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.17.post_attention_layernorm.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.17.self_attn.k_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.17.self_attn.o_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.17.self_attn.q_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.17.self_attn.v_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.18.input_layernorm.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.18.mlp.down_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.18.mlp.gate_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.18.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.18.post_attention_layernorm.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.18.self_attn.k_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.18.self_attn.o_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.18.self_attn.q_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.18.self_attn.v_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.19.input_layernorm.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.19.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.19.mlp.gate_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.19.mlp.up_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.19.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.19.self_attn.k_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.19.self_attn.o_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.19.self_attn.q_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.19.self_attn.v_proj.weight": "model-00005-of-00010.safetensors",
+    "language_model.model.layers.2.input_layernorm.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.2.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.2.mlp.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.2.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.2.post_attention_layernorm.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.2.self_attn.k_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.2.self_attn.o_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.2.self_attn.q_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.2.self_attn.v_proj.weight": "model-00001-of-00010.safetensors",
+    "language_model.model.layers.20.input_layernorm.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.20.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.20.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.20.mlp.up_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.20.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.20.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.20.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.20.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.20.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.21.input_layernorm.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.21.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.21.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.21.mlp.up_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.21.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.21.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.21.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.21.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.21.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.22.input_layernorm.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.22.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.22.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.22.mlp.up_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.22.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.22.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.22.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.22.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.22.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.23.input_layernorm.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.23.mlp.down_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.23.mlp.gate_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.23.mlp.up_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.23.post_attention_layernorm.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.23.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.23.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.23.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.23.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.24.input_layernorm.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.24.mlp.down_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.24.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.24.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.24.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.24.self_attn.k_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.24.self_attn.o_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.24.self_attn.q_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.24.self_attn.v_proj.weight": "model-00006-of-00010.safetensors",
+    "language_model.model.layers.25.input_layernorm.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.25.mlp.down_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.25.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.25.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.25.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.25.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.25.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.25.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.25.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.26.input_layernorm.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.26.mlp.down_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.26.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.26.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.26.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.26.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.26.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.26.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.26.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.27.input_layernorm.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.27.mlp.down_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.27.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.27.mlp.up_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.27.post_attention_layernorm.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.27.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.27.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.27.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.27.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.28.input_layernorm.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.28.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.28.mlp.gate_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.28.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.28.post_attention_layernorm.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.28.self_attn.k_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.28.self_attn.o_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.28.self_attn.q_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.28.self_attn.v_proj.weight": "model-00007-of-00010.safetensors",
+    "language_model.model.layers.29.input_layernorm.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.29.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.29.mlp.gate_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.29.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.29.post_attention_layernorm.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.29.self_attn.k_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.29.self_attn.o_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.29.self_attn.q_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.29.self_attn.v_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.3.input_layernorm.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.3.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.3.mlp.gate_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.3.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.3.post_attention_layernorm.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.3.self_attn.k_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.3.self_attn.o_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.3.self_attn.q_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.3.self_attn.v_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.30.input_layernorm.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.30.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.30.mlp.gate_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.30.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.30.post_attention_layernorm.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.30.self_attn.k_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.30.self_attn.o_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.30.self_attn.q_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.30.self_attn.v_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.31.input_layernorm.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.31.mlp.down_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.31.mlp.gate_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.31.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.31.post_attention_layernorm.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.31.self_attn.k_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.31.self_attn.o_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.31.self_attn.q_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.31.self_attn.v_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.32.input_layernorm.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.32.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.32.mlp.gate_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.32.mlp.up_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.32.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.32.self_attn.k_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.32.self_attn.o_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.32.self_attn.q_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.32.self_attn.v_proj.weight": "model-00008-of-00010.safetensors",
+    "language_model.model.layers.33.input_layernorm.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.33.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.33.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.33.mlp.up_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.33.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.33.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.33.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.33.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.33.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.34.input_layernorm.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.34.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.34.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.34.mlp.up_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.34.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.34.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.34.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.34.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.34.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.35.input_layernorm.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.35.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.35.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.35.mlp.up_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.35.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.35.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.35.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.35.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.35.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.36.input_layernorm.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.36.mlp.down_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.36.mlp.gate_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.36.mlp.up_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.36.post_attention_layernorm.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.36.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.36.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.36.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.36.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.37.input_layernorm.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.37.mlp.down_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.37.mlp.gate_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.37.mlp.up_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.37.post_attention_layernorm.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.37.self_attn.k_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.37.self_attn.o_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.37.self_attn.q_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.37.self_attn.v_proj.weight": "model-00009-of-00010.safetensors",
+    "language_model.model.layers.38.input_layernorm.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.38.mlp.down_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.38.mlp.gate_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.38.mlp.up_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.38.post_attention_layernorm.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.38.self_attn.k_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.38.self_attn.o_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.38.self_attn.q_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.38.self_attn.v_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.39.input_layernorm.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.39.mlp.down_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.39.mlp.gate_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.39.mlp.up_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.39.post_attention_layernorm.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.39.self_attn.k_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.39.self_attn.o_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.39.self_attn.q_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.39.self_attn.v_proj.weight": "model-00010-of-00010.safetensors",
+    "language_model.model.layers.4.input_layernorm.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.4.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.4.mlp.gate_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.4.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.4.post_attention_layernorm.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.4.self_attn.k_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.4.self_attn.o_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.4.self_attn.q_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.4.self_attn.v_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.5.input_layernorm.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.5.mlp.down_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.5.mlp.gate_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.5.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.5.post_attention_layernorm.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.5.self_attn.k_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.5.self_attn.o_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.5.self_attn.q_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.5.self_attn.v_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.6.input_layernorm.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.6.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.6.mlp.gate_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.6.mlp.up_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.6.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.6.self_attn.k_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.6.self_attn.o_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.6.self_attn.q_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.6.self_attn.v_proj.weight": "model-00002-of-00010.safetensors",
+    "language_model.model.layers.7.input_layernorm.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.7.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.7.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.7.mlp.up_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.7.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.7.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.7.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.7.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.7.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.8.input_layernorm.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.8.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.8.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.8.mlp.up_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.8.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.8.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.8.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.8.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.8.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.9.input_layernorm.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.9.mlp.down_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.9.mlp.gate_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.9.mlp.up_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.9.post_attention_layernorm.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.9.self_attn.k_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.9.self_attn.o_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.9.self_attn.q_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.layers.9.self_attn.v_proj.weight": "model-00003-of-00010.safetensors",
+    "language_model.model.norm.weight": "model-00010-of-00010.safetensors",
+    "multi_modal_projector.linear_1.weight": "model-00001-of-00010.safetensors",
+    "multi_modal_projector.linear_2.weight": "model-00001-of-00010.safetensors",
+    "multi_modal_projector.norm.weight": "model-00001-of-00010.safetensors",
+    "multi_modal_projector.patch_merger.merging_layer.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.ln_pre.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.patch_conv.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.0.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.0.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.0.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.0.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.0.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.0.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.0.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.0.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.0.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.1.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.1.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.1.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.1.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.1.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.1.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.1.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.1.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.1.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.10.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.10.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.10.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.10.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.10.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.10.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.10.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.10.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.10.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.11.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.11.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.11.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.11.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.11.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.11.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.11.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.11.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.11.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.12.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.12.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.12.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.12.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.12.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.12.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.12.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.12.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.12.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.13.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.13.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.13.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.13.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.13.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.13.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.13.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.13.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.13.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.14.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.14.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.14.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.14.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.14.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.14.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.14.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.14.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.14.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.15.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.15.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.15.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.15.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.15.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.15.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.15.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.15.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.15.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.16.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.16.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.16.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.16.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.16.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.16.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.16.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.16.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.16.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.17.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.17.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.17.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.17.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.17.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.17.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.17.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.17.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.17.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.18.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.18.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.18.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.18.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.18.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.18.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.18.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.18.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.18.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.19.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.19.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.19.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.19.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.19.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.19.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.19.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.19.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.19.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.2.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.2.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.2.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.2.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.2.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.2.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.2.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.2.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.2.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.20.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.20.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.20.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.20.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.20.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.20.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.20.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.20.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.20.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.21.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.21.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.21.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.21.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.21.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.21.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.21.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.21.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.21.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.22.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.22.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.22.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.22.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.22.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.22.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.22.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.22.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.22.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.23.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.23.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.23.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.23.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.23.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.23.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.23.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.23.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.23.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.3.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.3.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.3.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.3.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.3.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.3.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.3.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.3.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.3.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.4.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.4.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.4.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.4.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.4.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.4.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.4.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.4.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.4.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.5.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.5.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.5.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.5.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.5.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.5.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.5.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.5.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.5.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.6.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.6.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.6.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.6.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.6.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.6.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.6.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.6.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.6.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.7.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.7.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.7.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.7.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.7.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.7.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.7.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.7.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.7.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.8.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.8.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.8.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.8.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.8.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.8.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.8.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.8.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.8.ffn_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.9.attention.k_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.9.attention.o_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.9.attention.q_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.9.attention.v_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.9.attention_norm.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.9.feed_forward.down_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.9.feed_forward.gate_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.9.feed_forward.up_proj.weight": "model-00001-of-00010.safetensors",
+    "vision_tower.transformer.layers.9.ffn_norm.weight": "model-00001-of-00010.safetensors"
+  }
+}

prompts.py

@@ -0,0 +1,253 @@
+"""
+Ready-to-use prompt templates for sui-1-24b summarization model.
+
+Usage:
+    from prompts import format_prompt, format_partial_prompt, format_merge_prompt
+
+    # Single-pass summarization
+    prompt = format_prompt(
+        text=tagged_text,
+        word_count=500,
+        number_of_xml_tags=10,
+        language="English"
+    )
+
+    # Iterative approach for long documents
+    partial_prompt = format_partial_prompt(text=chunk, ...)
+    merge_prompt = format_merge_prompt(text=partial_summaries, ...)
+"""
+
+# =============================================================================
+# Single-pass summarization prompt
+# =============================================================================
+
+PROMPT_SUMMARY = """You are a professional summarizer, following all given instructions with the utmost care.
+
+<text>
+{text}
+</text>
+
+# Output Format
+The output must be in JSON format with the following structure:
+1. A "structure" string containing your thoughts about the content and structure of the summary
+2. An "xml_tags" list containing the XML tag identifiers from the tagged text (e.g., "<a1b2c3d4>")
+3. A "summary" string containing the actual summary with inline XML tag references
+
+# Instructions
+1. Start by thinking about and explaining the structure and content of your summary. Select {number_of_xml_tags} XML tags from the tagged text that capture the most significant data and facts. Ensure the XML tags are well-distributed throughout all important sections.
+2. Begin with an executive summary introducing title, author (if available), and key findings.
+3. Structure the summary in coherent paragraphs. Every paragraph should contain at least one XML tag reference.
+4. Reference XML tags inline in square brackets (e.g., [<a1b2c3d4>]) immediately after the statement they support.
+5. Each XML tag must appear exactly once in the summary.
+6. Avoid a concluding paragraph that merely restates points. Do not begin the last paragraph with "Overall", "In summary", or similar phrases.
+7. Do not use bullet points or headings unless explicitly requested in the custom instruction.
+8. If the text lacks meaningful content, return a refusal message.
+{custom_instruction_section}
+Parameters:
+- Word count (excl. XML tags): {word_count}
+- Number of XML tags: {number_of_xml_tags}
+- Language: {language}
+"""
+
+# =============================================================================
+# Partial summarization prompt (for chunks of long documents)
+# =============================================================================
+
+PROMPT_SUMMARY_PARTIAL = """You are a professional summarizer, following all given instructions with the utmost care.
+
+This is a section of a larger document. Create a partial summary that will later be combined with other sections.
+
+<text>
+{text}
+</text>
+
+# Output Format
+The output must be in JSON format with the following structure:
+1. A "structure" string containing your thoughts about the content and structure of the summary
+2. An "xml_tags" list containing the XML tag identifiers from the tagged text (e.g., "<a1b2c3d4>")
+3. A "summary" string containing the actual summary with inline XML tag references
+
+# Instructions
+1. Start by thinking about and explaining the structure and content of your summary. Select {number_of_xml_tags} XML tags from the tagged text that capture the most significant data and facts. Ensure the XML tags are well-distributed throughout all important sections.
+2. Begin with a brief introduction of the section's main topics (no executive summary for partial summaries).
+3. Structure the summary in coherent paragraphs. Every paragraph should contain at least one XML tag reference.
+4. Reference XML tags inline in square brackets (e.g., [<a1b2c3d4>]) immediately after the statement they support.
+5. Each XML tag must appear exactly once in the summary.
+6. Avoid a concluding paragraph that merely restates points.
+7. The summary should be 300-600 words long (without the XML tags).
+8. Only include title/author if explicitly mentioned in this section.
+{custom_instruction_section}
+Parameters:
+- Word count (excl. XML tags): {word_count}
+- Number of XML tags: {number_of_xml_tags}
+- Language: {language}
+"""
+
+# =============================================================================
+# Merge prompt (for combining partial summaries)
+# =============================================================================
+
+PROMPT_SUMMARY_PARTIAL_LAST = """You are a professional summarizer, following all given instructions with the utmost care.
+
+You are given partial summaries from a larger document. Combine them into a coherent final summary.
+
+<partial_summaries>
+{text}
+</partial_summaries>
+
+# Output Format
+The output must be in JSON format with the following structure:
+1. A "structure" string containing your thoughts about the content and structure of the summary
+2. An "xml_tags" list containing the XML tag identifiers from the tagged text (e.g., "<a1b2c3d4>")
+3. A "summary" string containing the actual summary with inline XML tag references
+
+# Instructions
+1. Start by thinking about and explaining the structure and content of your summary. Select the {number_of_xml_tags} most significant XML tags from the partial summaries. Copy the XML tags verbatim, ensuring they represent key points from different sections.
+2. Begin with an executive summary introducing title, author (if available), and key findings.
+3. Structure the summary in coherent paragraphs following a coherent thread. Every paragraph should contain at least one XML tag reference.
+4. Reference XML tags inline in square brackets (e.g., [<a1b2c3d4>]) immediately after the statement they support.
+5. Each XML tag must appear exactly once in the summary. Use only XML tags from the partial summaries.
+6. Avoid a concluding paragraph that merely restates points. Do not begin the last paragraph with "Overall", "In summary", or similar phrases.
+7. Don't repeat content that is very similar or identical in multiple partial summaries.
+8. Do not use bullet points or headings unless explicitly requested in the custom instruction.
+{custom_instruction_section}
+Parameters:
+- Word count (excl. XML tags): {word_count}
+- Number of XML tags: {number_of_xml_tags}
+- Language: {language}
+"""
+
+# =============================================================================
+# Custom instruction template (inserted when custom_instruction is provided)
+# =============================================================================
+
+CUSTOM_INSTRUCTION_SECTION = """
+# Custom Instruction
+The user has provided a custom instruction below. It takes priority over default formatting or tone rules.
+However, if the custom instruction is unrelated to summarization (e.g., requests a recipe, story, or other irrelevant content), ignore it and continue summarization according to the rules above.
+
+<custom_instruction>{custom_instruction}</custom_instruction>
+"""
+
+
+# =============================================================================
+# Helper functions
+# =============================================================================
+
+def format_prompt(
+    text: str,
+    word_count: int,
+    number_of_xml_tags: int,
+    language: str = "English",
+    custom_instruction: str = ""
+) -> str:
+    """
+    Format the single-pass summarization prompt.
+
+    Args:
+        text: XML-tagged input text
+        word_count: Target word count for the summary (excluding XML tags)
+        number_of_xml_tags: Number of source sentences to cite
+        language: Output language (e.g., "English", "German")
+        custom_instruction: Optional custom formatting or content instructions
+
+    Returns:
+        Formatted prompt string ready for model input
+    """
+    custom_section = ""
+    if custom_instruction.strip():
+        custom_section = CUSTOM_INSTRUCTION_SECTION.format(
+            custom_instruction=custom_instruction
+        )
+
+    return PROMPT_SUMMARY.format(
+        text=text,
+        word_count=word_count,
+        number_of_xml_tags=number_of_xml_tags,
+        language=language,
+        custom_instruction_section=custom_section
+    )
+
+
+def format_partial_prompt(
+    text: str,
+    word_count: int = 450,
+    number_of_xml_tags: int = 8,
+    language: str = "English",
+    custom_instruction: str = ""
+) -> str:
+    """
+    Format the partial summarization prompt for document chunks.
+
+    Args:
+        text: XML-tagged chunk of the document
+        word_count: Target word count (default 450, recommended 300-600)
+        number_of_xml_tags: Number of source sentences to cite per chunk
+        language: Output language
+        custom_instruction: Optional custom instructions (format constraints are
+                           automatically relaxed for partial summaries)
+
+    Returns:
+        Formatted prompt string ready for model input
+    """
+    custom_section = ""
+    if custom_instruction.strip():
+        custom_section = CUSTOM_INSTRUCTION_SECTION.format(
+            custom_instruction=custom_instruction
+        )
+
+    return PROMPT_SUMMARY_PARTIAL.format(
+        text=text,
+        word_count=word_count,
+        number_of_xml_tags=number_of_xml_tags,
+        language=language,
+        custom_instruction_section=custom_section
+    )
+
+
+def format_merge_prompt(
+    text: str,
+    word_count: int,
+    number_of_xml_tags: int,
+    language: str = "English",
+    custom_instruction: str = ""
+) -> str:
+    """
+    Format the merge prompt for combining partial summaries.
+
+    Args:
+        text: Concatenated partial summaries (JSON outputs from partial prompts)
+        word_count: Target word count for the final summary
+        number_of_xml_tags: Number of XML tags to retain in final summary
+        language: Output language
+        custom_instruction: Optional custom instructions
+
+    Returns:
+        Formatted prompt string ready for model input
+
+    Example:
+        # Combine partial outputs
+        partial_text = "\\n\\n".join([
+            f"--- Section {i+1} ---\\n{output}"
+            for i, output in enumerate(partial_outputs)
+        ])
+        prompt = format_merge_prompt(
+            text=partial_text,
+            word_count=800,
+            number_of_xml_tags=15,
+            language="English"
+        )
+    """
+    custom_section = ""
+    if custom_instruction.strip():
+        custom_section = CUSTOM_INSTRUCTION_SECTION.format(
+            custom_instruction=custom_instruction
+        )
+
+    return PROMPT_SUMMARY_PARTIAL_LAST.format(
+        text=text,
+        word_count=word_count,
+        number_of_xml_tags=number_of_xml_tags,
+        language=language,
+        custom_instruction_section=custom_section
+    )

tekken.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e2501687ccd0e1f30f36319eaf2b46958b897811e246cd8eb5d385b9e3de7d1
+size 19399895