---
license: other
license_name: lfm-open-license-v1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-VL-3B/blob/main/LICENSE
tags:
- vision-language-model
- lfm2
- compression
- quantization
- gptq
- compressed-tensors
- vllm
- konic
---

# LFM2.5-VL-3B-ffn8192-int4

The **most compressed** tier of our LFM2.5-VL-3B surgical-compression series:
FFN pruned 10752 → 8192 + distillation recovery, then **mixed-precision
quantization** — native **INT4/INT8** weights in the standard
**compressed-tensors** format (vLLM-ready).

| | Size | vs original |
|---|---|---|
| Original BF16 | 6.25 GB | — |
| Pruned+recovered BF16 | 5.30 GB | −15% |
| **This checkpoint (INT4/INT8)** | **2.22 GB** | **−69%** |

## Quantization allocation (hand-rolled GPTQ-class)
| Module group | Scheme |
|---|---|
| FFN w1/w2/w3 (90) | W4A16 GPTQ, per-(row, 128-group) asymmetric, zero-point |
| LM attention + vision tower linears (195) | W8 per-channel |
| conv projections / projector / patch-embedding / embeddings | BF16 (vLLM 0.26 LFM2 limitation — plain linears) |
| Norms / conv1d / biases | BF16 |

A nibble-packed variant of this model (1.93 GB, custom format + loader) is the
official compression deliverable — see the pipeline repo.

## Quality vs base (quick suite, our harness)
| Axis | Value |
|---|---|
| text PPL ratio (e^ΔNLL) | 0.83 (≤1.05 gate: PASS) |
| mm PPL ratio | 0.79 (≤1.10 gate: PASS) |
| text kld_t64 | 0.192 (base = 0) |
| mm kld_t64 | 0.229 |
| generation chrF (text/mm) | 0.54 / 0.51 |
| integrity index (record-only) | 42.5 / 100 |

## Serve with vLLM (native INT4 — no dequant)
```python
from vllm import LLM
llm = LLM(model="konic-labs/LFM2.5-VL-3B-ffn8192-int4",
          quantization="compressed-tensors")
```
Live-verified in vLLM 0.26 (L4, greedy): same answers as the base on 7
image/text prompts, and tool-calling intact (5 tools × 10 rounds: right tool +
right args everywhere; correct abstention on no-tool prompts).

## License & provenance
- Derivative of `LiquidAI/LFM2.5-VL-3B` (LFM Open License v1.0 — license text
  retained; modifications per this card).
- Pipeline and full reports: https://github.com/konic-labs/lfm25-vl-3b-lab
- Siblings: [LFM2.5-VL-3B-ffn8192](https://huggingface.co/konic-labs/LFM2.5-VL-3B-ffn8192),
  [LFM2.5-VL-3B-ffn7168](https://huggingface.co/konic-labs/LFM2.5-VL-3B-ffn7168).