metadata
license: other
license_name: lfm-open-license-v1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-VL-3B/blob/main/LICENSE
tags:
- vision-language-model
- lfm2
- compression
- quantization
- gptq
- compressed-tensors
- vllm
- konic
LFM2.5-VL-3B-ffn8192-int4
The most compressed tier of our LFM2.5-VL-3B surgical-compression series: FFN pruned 10752 → 8192 + distillation recovery, then mixed-precision quantization — native INT4/INT8 weights in the standard compressed-tensors format (vLLM-ready).
| Size | vs original | |
|---|---|---|
| Original BF16 | 6.25 GB | — |
| Pruned+recovered BF16 | 5.30 GB | −15% |
| This checkpoint (INT4/INT8) | 2.22 GB | −69% |
Quantization allocation (hand-rolled GPTQ-class)
| Module group | Scheme |
|---|---|
| FFN w1/w2/w3 (90) | W4A16 GPTQ, per-(row, 128-group) asymmetric, zero-point |
| LM attention + vision tower linears (195) | W8 per-channel |
| conv projections / projector / patch-embedding / embeddings | BF16 (vLLM 0.26 LFM2 limitation — plain linears) |
| Norms / conv1d / biases | BF16 |
A nibble-packed variant of this model (1.93 GB, custom format + loader) is the official compression deliverable — see the pipeline repo.
Quality vs base (quick suite, our harness)
| Axis | Value |
|---|---|
| text PPL ratio (e^ΔNLL) | 0.83 (≤1.05 gate: PASS) |
| mm PPL ratio | 0.79 (≤1.10 gate: PASS) |
| text kld_t64 | 0.192 (base = 0) |
| mm kld_t64 | 0.229 |
| generation chrF (text/mm) | 0.54 / 0.51 |
| integrity index (record-only) | 42.5 / 100 |
Serve with vLLM (native INT4 — no dequant)
from vllm import LLM
llm = LLM(model="konic-labs/LFM2.5-VL-3B-ffn8192-int4",
quantization="compressed-tensors")
Live-verified in vLLM 0.26 (L4, greedy): same answers as the base on 7 image/text prompts, and tool-calling intact (5 tools × 10 rounds: right tool + right args everywhere; correct abstention on no-tool prompts).
License & provenance
- Derivative of
LiquidAI/LFM2.5-VL-3B(LFM Open License v1.0 — license text retained; modifications per this card). - Pipeline and full reports: https://github.com/konic-labs/lfm25-vl-3b-lab
- Siblings: LFM2.5-VL-3B-ffn8192, LFM2.5-VL-3B-ffn7168.