File size: 1,653 Bytes
dd6b9e3 ad97da7 dd6b9e3 ad97da7 dd6b9e3 ad97da7 dd6b9e3 ad97da7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | ---
base_model:
- deepseek-ai/DeepSeek-OCR-2
---
# DeepSeek-OCR-2 GGUF (for llama.cpp PR #20975)
This repository provides **GGUF model files** required for running the **DeepSeek-OCR-2 / MTMD** support introduced in the following llama.cpp pull request:
👉 **llama.cpp PR #20975**
[https://github.com/ggml-org/llama.cpp/pull/20975](https://github.com/ggml-org/llama.cpp/pull/20975)
These models are **only compatible with the PR branch** and will not run on upstream llama.cpp `main`.
---
## 📥 Download
You can download the model files directly using:
```bash
huggingface-cli download <this-repo> --include "deepseek-ocr-2-bf16.gguf" --local-dir gguf_models/deepseek-ai
huggingface-cli download <this-repo> --include "mmproj-deepseek-ocr-2-bf16.gguf" --local-dir gguf_models/deepseek-ai
```
---
## 🚀 Run Example
Use with `llama-server`
```bash
build/bin/llama-server \
-hf sabafallah/DeepSeek-OCR-2-GGUF:bf16 \
--chat-template deepseek-ocr --no-jinja \
--temp 0 \
--flash-attn off \
--no-warmup \
-n 2048 \
--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2 \
--dry-penalty-last-n -1 --dry-sequence-breaker none
```
Use the `llama-mtmd-cli` executable from the PR:
```bash
build/bin/llama-mtmd-cli \
-m gguf_models/deepseek-ai/deepseek-ocr-2-bf16.gguf \
--mmproj gguf_models/deepseek-ai/mmproj-deepseek-ocr-2-bf16.gguf \
--image tmp/mtmd_test_data/webpage_tall.png -p "Free OCR." \
--chat-template deepseek-ocr --no-jinja \
--temp 0 \
--flash-attn off \
--no-warmup \
-n 2048 \
--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2 \
--dry-penalty-last-n -1 --dry-sequence-breaker none
```
|