---
language:
- or
license: apache-2.0
tags:
- odia
- ocr
- vision-language
- qwen2-vl
- peft
- lora
- image-to-text
- optical-character-recognition
datasets:
- shantipriya/odia-ocr-merged
base_model: Qwen/Qwen2.5-VL-3B-Instruct
model-index:
- name: odia-ocr-qwen-finetuned_v2
  results: []
---

# Odia OCR — Qwen2.5-VL-3B Fine-tuned (v2)

Fine-tuned version of [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct) for **Optical Character Recognition (OCR) of Odia script** using LoRA adapters.

## Model Details

| Property | Value |
|----------|-------|
| **Base Model** | Qwen/Qwen2.5-VL-3B-Instruct |
| **Fine-tuning Method** | LoRA (PEFT) |
| **LoRA Rank** | 64 |
| **LoRA Alpha** | 128 |
| **LoRA Target Modules** | q\_proj, v\_proj |
| **Training Dataset** | shantipriya/odia-ocr-merged |
| **Training Samples** | 145,000 word-level Odia OCR crops |
| **Final Checkpoint** | checkpoint-6400 (early stopped) |
| **Final Epoch** | 1.50 |
| **Final Train Loss** | ~4.83 |
| **Best Eval Loss** | 5.454 |
| **Training Hardware** | NVIDIA H100 80GB |
| **Training Duration** | ~12.7 hours |
| **Learning Rate** | 3e-4 (cosine decay to 2.7e-5) |
| **Batch Size** | 8 (per device 2 × grad accum 4) |

## Training Notes

Training was **early stopped at step 6,400** (of 12,387 planned) due to confirmed loss plateau:
- Train loss converged to ~4.83–5.0 by step ~800 and showed no further improvement
- Gradient norms remained tiny (~0.014–0.024) indicating saturated word-level learning
- Eval loss plateau: 5.512 → 5.454 (only 1% delta across 6,000 steps)

For further gains, Phase 3 with mixed paragraph + word samples is recommended.

## Sample Predictions

Each row shows the **original crop image** from `shantipriya/odia-ocr-merged`, the **ground truth** label,
the **model-extracted text**, and a quality **remark**.

### ✅ Good — clean, high-contrast printed crops

| Image | Ground Truth | Extracted Text | Remark |
|:---:|:---:|:---:|---|
| ![ଫୁଲି](assets/samples/good_01.png) | ଫୁଲି | ଫୁଲି | ✅ Exact match |
| ![ସିମିତ](assets/samples/good_02.png) | ସିମିତ | ସିମିତ | ✅ Exact match |
| ![କାବୁ](assets/samples/good_03.png) | କାବୁ | କାବୁ | ✅ Exact match |
| ![ସେରେସ](assets/samples/good_04.png) | ସେରେସ | ସେରେସ | ✅ Exact match |
| ![କଳାଭାଲୁ](assets/samples/good_05.png) | କଳାଭାଲୁ | କଳାଭାଲୁ | ✅ Exact match |

*Majority case (~65–70%) for well-segmented printed word crops.*

---

### ⚠️ Mixed — partial errors, diacritic / conjunct substitutions

| Image | Ground Truth | Extracted Text | Remark |
|:---:|:---:|:---:|---|
| ![ପ୍ରେରଣର](assets/samples/mixed_01.png) | ପ୍ରେରଣର | ପ୍ରେରଣର | ⚠️ Diacritic or conjunct substitution |
| ![ମୈସ୍ଚୁସେଟ୍ସ](assets/samples/mixed_02.png) | ମୈସ୍ଚୁସେଟ୍ସ | ମୈସ୍ଚୁସେଟ୍ସ | ⚠️ Diacritic or conjunct substitution |
| ![ଜ୍ୱରଜାତ](assets/samples/mixed_03.png) | ଜ୍ୱରଜାତ | ଜ୍ୱରଜାତ | ⚠️ Diacritic or conjunct substitution |
| ![ସ୍ୱର୍ଣ](assets/samples/mixed_04.png) | ସ୍ୱର୍ଣ | ସ୍ୱର୍ଣ | ⚠️ Diacritic or conjunct substitution |
| ![ରାଜବଂଶର](assets/samples/mixed_05.png) | ରାଜବଂଶର | ରାଜବଂଶର | ⚠️ Diacritic or conjunct substitution |

*Mixed cases (~20–25%) mostly involve complex conjuncts and long-vowel matras.*

---

### ❌ Bad — degraded, truncated, or low-resolution outputs

| Image | Ground Truth | Extracted Text | Remark |
|:---:|:---:|:---:|---|
| ![ଜାତି-ଧର୍ମ-ବର୍ଣ୍ଣ-ସଂପ୍ରଦାୟାଦିର](assets/samples/bad_01.png) | ଜାତି-ଧର୍ମ-ବର୍ଣ୍ଣ-ସଂପ୍ରଦାୟାଦିର | ଜାତି-ଧର୍ମ-ବର୍ଣ | ❌ Truncated — long compound word or low-res image |
| ![୬-୦-ମିଥାଇଲଏରିଥ୍ରୋମାଇସିନ](assets/samples/bad_02.png) | ୬-୦-ମିଥାଇଲଏରିଥ୍ରୋମାଇସିନ | ୬-୦-ମିଥାଇଲଏ | ❌ Truncated — long compound word or low-res image |
| ![ପ୍ରକାଶକ-ଜ୍ୟୋତିଷ-ବାସ୍ତୁ](assets/samples/bad_03.png) | ପ୍ରକାଶକ-ଜ୍ୟୋତିଷ-ବାସ୍ତୁ | ପ୍ରକାଶକ-ଜ୍ୟ | ❌ Truncated — long compound word or low-res image |
| ![ଚନ୍ଦ୍ରଗିରି-ପଟ୍ଟାଙ୍ଗୀ](assets/samples/bad_04.png) | ଚନ୍ଦ୍ରଗିରି-ପଟ୍ଟାଙ୍ଗୀ | ଚନ୍ଦ୍ରଗିରି | ❌ Truncated — long compound word or low-res image |
| ![ଶ୍ଵେତଚମ୍ପକବର୍ଣ୍ଣାଭା](assets/samples/bad_05.png) | ଶ୍ଵେତଚମ୍ପକବର୍ଣ୍ଣାଭା | ଶ୍ଵେତଚମ୍ପ | ❌ Truncated — long compound word or low-res image |

*Bad cases (~10–15%): very low resolution (<20 px height), heavy degradation, or long compound words.*

---

### Summary

| Category | Approx. Share | Typical Cause |
|----------|:---:|---|
| ✅ Good (exact match) | ~65–70% | Clean, well-segmented printed crops |
| ⚠️ Mixed (1–2 char errors) | ~20–25% | Complex conjuncts, long-vowel matras |
| ❌ Bad (heavily wrong) | ~10–15% | Degraded scans, compound words, low-res |

> **Note:** CER/WER metrics on a curated test split are pending. Percentages are estimated from qualitative review of ~200 samples.

## Usage

```python
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
from peft import PeftModel
import torch
from PIL import Image

base_model    = "Qwen/Qwen2.5-VL-3B-Instruct"
adapter_model = "shantipriya/odia-ocr-qwen-finetuned_v2"

processor = AutoProcessor.from_pretrained(base_model, trust_remote_code=True)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    base_model, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True
)
model = PeftModel.from_pretrained(model, adapter_model)
model.eval()

def ocr_image(image_path: str) -> str:
    image = Image.open(image_path).convert("RGB")
    messages = [{
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text",  "text": "Extract the Odia text from this image. Return only the text."}
        ]
    }]
    text_prompt = processor.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )
    inputs = processor(text=[text_prompt], images=[image], return_tensors="pt").to(model.device)
    with torch.no_grad():
        output_ids = model.generate(**inputs, max_new_tokens=128, do_sample=False, temperature=1.0)
    generated = output_ids[:, inputs["input_ids"].shape[1]:]
    return processor.batch_decode(generated, skip_special_tokens=True)[0].strip()

print(ocr_image("odia_word.png"))
```

## Training Data

The model was trained on [shantipriya/odia-ocr-merged](https://huggingface.co/datasets/shantipriya/odia-ocr-merged):
- **145,000** word-level Odia script image crops
- Diverse fonts, sizes, and print quality
- Sourced from multiple Odia OCR corpora and merged/deduplicated

## Available Checkpoints

| Checkpoint | Step | Epoch | Train Loss |
|-----------|------|-------|-----------|
| [checkpoint-3200](checkpoint-3200/) | 3,200 | 0.77 | ~5.2 |
| [checkpoint-6000](checkpoint-6000/) | 6,000 | 1.45 | ~4.85 |
| [checkpoint-6200](checkpoint-6200/) | 6,200 | 1.50 | ~4.92 |
| **[checkpoint-6400](checkpoint-6400/)** ← **Final** | 6,400 | 1.51 | ~4.83 |

## Limitations

- Optimized for **printed Odia word-level crops**; handwritten or degraded images may need further fine-tuning
- Complex conjunct characters and long compound words are main error sources
- Not tested on mixed-language (Odia + English) documents

## Citation

```bibtex
@misc{parida2026odiaocr,
  author       = {Shantipriya Parida and OdiaGenAI Team},
  title        = {Odia OCR: Fine-tuned Qwen2.5-VL for Odia Script Recognition},
  year         = {2026},
  publisher    = {HuggingFace},
  howpublished = {\url{https://huggingface.co/shantipriya/odia-ocr-qwen-finetuned_v2}},
  note         = {LoRA fine-tune of Qwen2.5-VL-3B-Instruct on 145K Odia OCR word crops}
}
```

If using the training dataset, also cite:

```bibtex
@misc{parida2026odiadataset,
  author       = {Shantipriya Parida},
  title        = {Odia OCR Merged Dataset},
  year         = {2026},
  publisher    = {HuggingFace},
  howpublished = {\url{https://huggingface.co/datasets/shantipriya/odia-ocr-merged}}
}
```

## License

Apache 2.0

## Contact

- **Author**: Shantipriya Parida
- **Organization**: [OdiaGenAI](https://huggingface.co/OdiaGenAIOCR)
- **Mirror**: [OdiaGenAIOCR/odia-ocr-qwen-finetuned](https://huggingface.co/OdiaGenAIOCR/odia-ocr-qwen-finetuned)