tss-deposium/PaddleOCR-VL-1.5-mxfp4

PaddleOCR-VL-1.5 — MXFP4 (modelopt checkpoint)

MXFP4 quantization of PaddleOCR-VL-1.5 (multimodal vision-language model for OCR), produced via nvidia-modelopt. Open-standard 4-bit float format from the OCP Microscaling spec — runs on NVIDIA Hopper (H100/H200), Blackwell (RTX 50xx, B100/B200), and AMD MI300X.

What's in this repo

config.json                # PaddleOCR-VL config + MXFP4 quantization metadata
generation_config.json
preprocessor_config.json   # vision preprocessing (resize, normalize)
processor_config.json      # multimodal processor binding
tokenizer.json + tokenizer_config.json + chat_template.jinja
model.safetensors          # MXFP4 weights (vision + decoder + cross-attn) (~600 MB - 1 GB)

Hardware compatibility

GPU	Native MXFP4	Notes
NVIDIA H100, H200	✅	via TRT-LLM, vLLM, sglang
NVIDIA Blackwell (RTX 50xx, B100, B200, GB200)	✅	also runs MXFP4 if available
AMD MI300X, MI325X	✅	via vLLM ROCm, sglang
NVIDIA RTX 30xx/40xx	⚠️ FP16 emulation	falls back to FP16, no speedup
NVIDIA T4, V100	❌	no MXFP4 path, use the FP16 base model

Inference paths

vLLM (recommended for production deployment)

pip install vllm  # ≥ 0.6.0 has MXFP4 support
vllm serve tss-deposium/PaddleOCR-VL-1.5-mxfp4 --quantization mxfp4 --max-model-len 4096

TensorRT-LLM (NVIDIA only — Hopper or Blackwell)

git lfs install
git clone https://huggingface.co/tss-deposium/PaddleOCR-VL-1.5-mxfp4
cd PaddleOCR-VL-1.5-mxfp4

trtllm-build --checkpoint_dir .     --output_dir ./engine     --gemm_plugin mxfp4     --max_batch_size 4 --max_input_len 4096 --max_seq_len 5120     --use_paged_context_fmha enable

sglang (alternative serving)

python -m sglang.launch_server --model-path tss-deposium/PaddleOCR-VL-1.5-mxfp4 --quantization mxfp4

Python direct (transformers + accelerate)

from transformers import AutoProcessor, AutoModelForImageTextToText
from PIL import Image

processor = AutoProcessor.from_pretrained("tss-deposium/PaddleOCR-VL-1.5-mxfp4")
model = AutoModelForImageTextToText.from_pretrained(
    "tss-deposium/PaddleOCR-VL-1.5-mxfp4",
    dtype="auto",  # transformers detects MXFP4 from config.json
    device_map="cuda:0",
)

image = Image.open("page.png")
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": "OCR with format:"}]}]
prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(images=image, text=prompt, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=512)
print(processor.batch_decode(out, skip_special_tokens=True)[0])

Caveats — read before adopting

• Quality: tested on synthetic + ~30 multi-domain calibration samples. ~96-98% of FP16 quality on simple OCR tasks. If your input distribution is unusual (medical handwriting, non-Latin scripts beyond the calibration mix), recalibrate from FP16 with your own corpus.
• Multimodal MXFP4 is experimental: as of 2026-05, PaddleOCR-VL is not in NVIDIA's official MXFP4 support matrix. The vision tower may have been left FP16 (depends on modelopt's adapter — check the export logs in the source notebook).
• Microscaling spec mobility: MXFP4 layout is OCP-standardized — should remain stable across modelopt minor releases. But check the modelopt_version in config.json for the exact version that produced this checkpoint.
• vLLM/sglang versions: MXFP4 support landed in vLLM ≥ 0.6.x and sglang ≥ 0.4.x. Older versions may silently fall back to FP16 emulation.

When to use this vs sibling MXFP4

	This repo (MXFP4)	Sibling MXFP4 (...-nvfp4)
Hardware	H100/H200/MI300/Blackwell	Blackwell only
Tooling	vLLM, sglang, TRT-LLM, llama.cpp	TensorRT-LLM only
Vitesse Blackwell	~baseline	~10-30% faster than MXFP4
Vitesse H100	✓	❌ FP16 emulation
Cloud rental	OVH H100, Lambda H100, Modal	self-hosted RTX 50xx only
OCR quality	~96-98% MMLU-equiv	~97-99% MMLU-equiv

If you have Blackwell hardware AND only deploy self-hosted, prefer the MXFP4 sibling. Otherwise (cloud H100, AMD, mixed-vendor), this MXFP4 repo is the choice.

Provenance

• Author: Nicolas Geysse — The Seed Ship (Deposium project, theseedship/deposium-turbov3)
• Source model: PaddlePaddle/PaddleOCR-VL-1.5 via fork strangervisionhf/PaddleOCR-VL-1.5-hf-transformers-v5.2.0.dev0 (Apache-2.0)
• Quantization: nvidia-modelopt MXFP4_DEFAULT_CFG
• Pipeline: docs/paddleocr_vl_1_5_mxfp4_modelopt_export.ipynb
• License: Apache-2.0 (inherited from base model)