---
license: mit
base_model: deepseek-ai/DeepSeek-V4.1-Flash
base_model_relation: quantized
library_name: gguf
pipeline_tag: text-generation
tags:
- gguf
- deepseek
- deepseek-v4.1
- llama.cpp
quantized_by: vcruz305
quality recovered: smalinin
---

# DeepSeek-V4.1-Flash GGUF

llama.cpp GGUF of [deepseek-ai/DeepSeek-V4.1-Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash).

This is **V4.1-Flash** (`DeepseekV41ForCausalLM`), a causal decoder with engram n-gram lookup
tables, hyper-connections and sparse attention. It is not V4-Flash-0731.

## Recipe

How to build the engine, serve it, and the gotchas, plus the current status:
[https://github.com/smalinin/llama.cpp/tree/my_build_deepseek41](https://github.com/smalinin/llama.cpp/tree/my_build_deepseek41)

New runtime binary was tested on 4xRTX4090(48Gb) + 2xRTX3090  with next config:

```
./llama-server \
  --model ./DeepSeek-V4.1-Flash-Q2_K-00001-of-00007.gguf \
  --host 127.0.0.1 \
  --port 8080 \
  --ctx-size 64000 \
  --batch-size 2048 \
  --ubatch-size 256 \
  --parallel 1 \
  --n-gpu-layers auto \
  --fit on \
  --fit-ctx 64000 \
  --fit-target 2048 \
  --load-mode mmap \
  --lazy-mode auto \
  --flash-attn on \
  --no-warmup \
  --no-context-shift \
  --jinja \
  --chat-template-file ./models/templates/deepseek-ai-DeepSeek-V4.1.jinja \
  --chat-template-kwargs '{"reasoning_effort":80,"enable_thinking":true}' \
  --reasoning-format deepseek \
  --no-reasoning-preserve \
  --no-prefill-assistant

```
Results:  pp = 250 t/s;  tg = 4.7t/s for prompt size=25k


## Status

**These files do not run on upstream llama.cpp yet.** 

## Files

## Sample of Q2 work

<p align="center">
  <img src="pelican.png" width="400px">
</p>

## Sample of Q4 work

<p align="center">
  <img src="pelic_q4.png"  width="400px">
</p>

Apache/MIT from upstream.