Instructions to use smalinin/DeepSeek-V4.1-Flash-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use smalinin/DeepSeek-V4.1-Flash-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS
# Run inference directly in the terminal:
llama cli -hf smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS
# Run inference directly in the terminal:
llama cli -hf smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS
# Run inference directly in the terminal:
./llama-cli -hf smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS
# Run inference directly in the terminal:
./build/bin/llama-cli -hf smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS

Use Docker

docker model run hf.co/smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS

LM Studio
Jan

vLLM

How to use smalinin/DeepSeek-V4.1-Flash-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "smalinin/DeepSeek-V4.1-Flash-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "smalinin/DeepSeek-V4.1-Flash-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS

Ollama
How to use smalinin/DeepSeek-V4.1-Flash-GGUF with Ollama:
```
ollama run hf.co/smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS
```
Unsloth Desktop

How to use smalinin/DeepSeek-V4.1-Flash-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use smalinin/DeepSeek-V4.1-Flash-GGUF with Docker Model Runner:
```
docker model run hf.co/smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS
```

Lemonade

How to use smalinin/DeepSeek-V4.1-Flash-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS

Run and chat with the model

lemonade run user.DeepSeek-V4.1-Flash-GGUF-IQ3_XS

List all available models

lemonade list

Hermes Agent

How to use smalinin/DeepSeek-V4.1-Flash-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use smalinin/DeepSeek-V4.1-Flash-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "smalinin/DeepSeek-V4.1-Flash-GGUF:IQ3_XS" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

smalinin

vcruz305 commited on 4 days ago

Commit

bb69b82

0 Parent(s):

Duplicate from vcruz305/DeepSeek-V4.1-Flash-GGUF

Browse files

Co-authored-by: Victor Cruz <vcruz305@users.noreply.huggingface.co>

Files changed (46) hide show

.gitattributes +75 -0
.probe.md +1 -0
DeepSeek-V4.1-Flash-Q1_0-00001-of-00003.gguf +3 -0
DeepSeek-V4.1-Flash-Q1_0-00002-of-00003.gguf +3 -0
DeepSeek-V4.1-Flash-Q1_0-00003-of-00003.gguf +3 -0
DeepSeek-V4.1-Flash-Q2_K-00001-of-00007.gguf +3 -0
DeepSeek-V4.1-Flash-Q2_K-00002-of-00007.gguf +3 -0
DeepSeek-V4.1-Flash-Q2_K-00003-of-00007.gguf +3 -0
DeepSeek-V4.1-Flash-Q2_K-00004-of-00007.gguf +3 -0
DeepSeek-V4.1-Flash-Q2_K-00005-of-00007.gguf +3 -0
DeepSeek-V4.1-Flash-Q2_K-00006-of-00007.gguf +3 -0
DeepSeek-V4.1-Flash-Q2_K-00007-of-00007.gguf +3 -0
DeepSeek-V4.1-Flash-Q3_K_M-00001-of-00009.gguf +3 -0
DeepSeek-V4.1-Flash-Q3_K_M-00002-of-00009.gguf +3 -0
DeepSeek-V4.1-Flash-Q3_K_M-00003-of-00009.gguf +3 -0
DeepSeek-V4.1-Flash-Q3_K_M-00004-of-00009.gguf +3 -0
DeepSeek-V4.1-Flash-Q3_K_M-00005-of-00009.gguf +3 -0
DeepSeek-V4.1-Flash-Q3_K_M-00006-of-00009.gguf +3 -0
DeepSeek-V4.1-Flash-Q3_K_M-00007-of-00009.gguf +3 -0
DeepSeek-V4.1-Flash-Q3_K_M-00008-of-00009.gguf +3 -0
DeepSeek-V4.1-Flash-Q3_K_M-00009-of-00009.gguf +3 -0
DeepSeek-V4.1-Flash-Q4_K_M-00001-of-00011.gguf +3 -0
DeepSeek-V4.1-Flash-Q4_K_M-00002-of-00011.gguf +3 -0
DeepSeek-V4.1-Flash-Q4_K_M-00003-of-00011.gguf +3 -0
DeepSeek-V4.1-Flash-Q4_K_M-00004-of-00011.gguf +3 -0
DeepSeek-V4.1-Flash-Q4_K_M-00005-of-00011.gguf +3 -0
DeepSeek-V4.1-Flash-Q4_K_M-00006-of-00011.gguf +3 -0
DeepSeek-V4.1-Flash-Q4_K_M-00007-of-00011.gguf +3 -0
DeepSeek-V4.1-Flash-Q4_K_M-00008-of-00011.gguf +3 -0
DeepSeek-V4.1-Flash-Q4_K_M-00009-of-00011.gguf +3 -0
DeepSeek-V4.1-Flash-Q4_K_M-00010-of-00011.gguf +3 -0
DeepSeek-V4.1-Flash-Q4_K_M-00011-of-00011.gguf +3 -0
DeepSeek-V4.1-Flash-Q8_0-00001-of-00010.gguf +3 -0
DeepSeek-V4.1-Flash-Q8_0-00002-of-00010.gguf +3 -0
DeepSeek-V4.1-Flash-Q8_0-00003-of-00010.gguf +3 -0
DeepSeek-V4.1-Flash-Q8_0-00004-of-00010.gguf +3 -0
DeepSeek-V4.1-Flash-Q8_0-00005-of-00010.gguf +3 -0
DeepSeek-V4.1-Flash-Q8_0-00006-of-00010.gguf +3 -0
DeepSeek-V4.1-Flash-Q8_0-00007-of-00010.gguf +3 -0
DeepSeek-V4.1-Flash-Q8_0-00008-of-00010.gguf +3 -0
DeepSeek-V4.1-Flash-Q8_0-00009-of-00010.gguf +3 -0
DeepSeek-V4.1-Flash-Q8_0-00010-of-00010.gguf +3 -0
README.md +70 -0
llama.cpp/patches/README.md +79 -0
llama.cpp/patches/fix_gguf_engram_kv.py +290 -0
llama.cpp/patches/patch_llamacpp_v41.py +701 -0

.gitattributes ADDED Viewed

	@@ -0,0 +1,75 @@

+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q2_K-00001-of-00007.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q2_K-00002-of-00007.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q2_K-00003-of-00007.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q2_K-00004-of-00007.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q2_K-00005-of-00007.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q2_K-00006-of-00007.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q8_0-00001-of-00010.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q8_0-00002-of-00010.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q8_0-00003-of-00010.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q8_0-00004-of-00010.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q8_0-00005-of-00010.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q8_0-00006-of-00010.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q8_0-00007-of-00010.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q8_0-00008-of-00010.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q8_0-00009-of-00010.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q8_0-00010-of-00010.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q2_K-00007-of-00007.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q3_K_M-00001-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q3_K_M-00002-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q3_K_M-00003-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q3_K_M-00004-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q3_K_M-00005-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q3_K_M-00006-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q3_K_M-00007-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q3_K_M-00008-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q3_K_M-00009-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q4_K_M-00001-of-00011.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q4_K_M-00002-of-00011.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q4_K_M-00003-of-00011.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q4_K_M-00004-of-00011.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q4_K_M-00005-of-00011.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q4_K_M-00006-of-00011.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q4_K_M-00007-of-00011.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q4_K_M-00008-of-00011.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q4_K_M-00009-of-00011.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q4_K_M-00010-of-00011.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q4_K_M-00011-of-00011.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q1_0-00001-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q1_0-00002-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4.1-Flash-Q1_0-00003-of-00003.gguf filter=lfs diff=lfs merge=lfs -text

.probe.md ADDED Viewed

	@@ -0,0 +1 @@


1	+ DeepSeek-V4.1-Flash GGUF quantizations. Work in progress.

DeepSeek-V4.1-Flash-Q1_0-00001-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1a6185fb832c3f523791c677bf126e2e3d7aea3ea2e261aecab6f5a6fdd172c2
+size 41640881376

DeepSeek-V4.1-Flash-Q1_0-00002-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bd6679ee04bc0d0378e995f608715c64f410df7ed8e68d23989b1c103d36bcee
+size 44857126912

DeepSeek-V4.1-Flash-Q1_0-00003-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7d507eb21bfe10d71777134afe6765e923609358166eb6049571a3553a865fbd
+size 19363037984

DeepSeek-V4.1-Flash-Q2_K-00001-of-00007.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0bcee934bd4e8350c54410681d9300b0321a300fcb1df1437a20703394d08ee0
+size 43072989056

DeepSeek-V4.1-Flash-Q2_K-00002-of-00007.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:124ffa15b6b7ec9630715ad18752a4c3f371749647837b8c9d5f7eb52c4f73c0
+size 44805480352

DeepSeek-V4.1-Flash-Q2_K-00003-of-00007.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4dd35b0b086cc0fb19d3a72afd8aed331317950e2257f37b9672702edc3f2453
+size 14935107904

DeepSeek-V4.1-Flash-Q2_K-00004-of-00007.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d24832f4c2f42340d5d4bf4d9b3277c861e0a0a9ccd25e94bda5581c5351f960
+size 44217904672

DeepSeek-V4.1-Flash-Q2_K-00005-of-00007.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:34714c3880bde310207e3d12fa488d8d676f338ff635157fded8ec73e133efad
+size 44804598176

DeepSeek-V4.1-Flash-Q2_K-00006-of-00007.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:08dc941d26687cb51f736a4f1c3a2c4843fb1d79573164c645106711081f5faf
+size 44803713696

DeepSeek-V4.1-Flash-Q2_K-00007-of-00007.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:550bbdb94a69142abfa4b2f2db86ffe3dd6d4ca45eafbb6728f4fe40506fa6b9
+size 27875485600

DeepSeek-V4.1-Flash-Q3_K_M-00001-of-00009.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6499c20b172f30bf68dfbf4465e8067482bdc33f701c30e37cf422935e4d4bc3
+size 7971784416

DeepSeek-V4.1-Flash-Q3_K_M-00002-of-00009.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:eb1bc512e478184086ffcf58882efdce4c188c1e461a4bb285a9dc2e3f2fa3f5
+size 42308283648

DeepSeek-V4.1-Flash-Q3_K_M-00003-of-00009.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:65e6fed865793f2719e9d47819f4901e88c4499d783b1a3bf10448ae2422ea98
+size 44181213952

DeepSeek-V4.1-Flash-Q3_K_M-00004-of-00009.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3ed43f59a734c97e4dbef6fc77c7047267742bffc76e415d9e5fb6719dda2447
+size 41111992000

DeepSeek-V4.1-Flash-Q3_K_M-00005-of-00009.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9be50bb502735bebaee0fc6acc7e8dbc4e7e473f428bbb2f4c96210cedf00fbe
+size 44864115552

DeepSeek-V4.1-Flash-Q3_K_M-00006-of-00009.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:433b4362bb2fcb97c796935d01c53c3da0b9a3cbcc36e06d0314c6e69916bdbf
+size 43066406752

DeepSeek-V4.1-Flash-Q3_K_M-00007-of-00009.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bc5f9a0bde117716b22ee10f5be7d735b438f40040d08fac22474a9d9883d712
+size 43611224640

DeepSeek-V4.1-Flash-Q3_K_M-00008-of-00009.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e6b08013c406e01d186487eaca04da53c24431346400904fae15723b8762ac44
+size 43664455008

DeepSeek-V4.1-Flash-Q3_K_M-00009-of-00009.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c069f939d796f14cb89c0b519f27a17de435315802efd110a35971db7593135c
+size 36491996320

DeepSeek-V4.1-Flash-Q4_K_M-00001-of-00011.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:aa0ef0b26baacebda187bc7a675b60a61b9407097b39ff244a8d72273910beff
+size 9903019232

DeepSeek-V4.1-Flash-Q4_K_M-00002-of-00011.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0f7e076a531423b4c52294652ddb1b87859793781fabc6618a0d13d6f8540aaf
+size 55296888384

DeepSeek-V4.1-Flash-Q4_K_M-00003-of-00011.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:eaf7000aa1ee038f4a979708131f036f2f52cac5b1510c29e544ad89e16a8693
+size 43477027712

DeepSeek-V4.1-Flash-Q4_K_M-00004-of-00011.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:14d9641d478a2d7a8a79fd3043dc759a3a052ab0dcda6a17b7bf5e0b052c529c
+size 43601417504

DeepSeek-V4.1-Flash-Q4_K_M-00005-of-00011.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5c0e4d0e120fe5485052d479add7445e2f3d1f65e598accc8c758449458b41bf
+size 21838521504

DeepSeek-V4.1-Flash-Q4_K_M-00006-of-00011.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:56e2de3c9d3da27ee1c48be3d6d2b0293b70e842b34ef5a3b1fc3397a29e0cbc
+size 55298402400

DeepSeek-V4.1-Flash-Q4_K_M-00007-of-00011.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:656e0636bc9b97158889a04d36128373bcee38b83b42fdbf4ac95d3ce5e40cc6
+size 43686966240

DeepSeek-V4.1-Flash-Q4_K_M-00008-of-00011.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e3ec8118206d5ebdc877b68375021150143f31ac5c5c74c5854ed17a6454b8e3
+size 44992770528

DeepSeek-V4.1-Flash-Q4_K_M-00009-of-00011.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4f20e2cf9d3da6158f02f4c933274bd28966a91a531ef42d228f7b70e0ada62a
+size 43668184320

DeepSeek-V4.1-Flash-Q4_K_M-00010-of-00011.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4c3c9101d61dd4e5c8a3a05a30604299f3cffaf28c8d388e8fdedd51d92c3b4b
+size 44773274272

DeepSeek-V4.1-Flash-Q4_K_M-00011-of-00011.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1f7ed637779d529bd56fbcdfe7008bd28d546dc3d0fe3c5ce78bc51edfc0deca
+size 38200982624

DeepSeek-V4.1-Flash-Q8_0-00001-of-00010.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:167304a8d1123d8a9b7d53a82c01aab659eba51be6d58763f769eb294f62617e
+size 43322444352

DeepSeek-V4.1-Flash-Q8_0-00002-of-00010.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d9f95f3ec801d07bddb374b3943a7bdb484e48dd8b24c5eae9218b091e146f52
+size 43316676000

DeepSeek-V4.1-Flash-Q8_0-00003-of-00010.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c53687cc23e2e32ab3ad5dbd8052de07801e12a8b3ce278967dfa6c2eb2983b4
+size 43316676032

DeepSeek-V4.1-Flash-Q8_0-00004-of-00010.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ba84c194c41feb6f7fb65fb4a0a4e188ee2cb85516b86ba2818e16cec2017343
+size 43316676032

DeepSeek-V4.1-Flash-Q8_0-00005-of-00010.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:66fdac198238c64c7310de22e46a1f9a15284fbde65f6a020f6e94395155b8b5
+size 43316676032

DeepSeek-V4.1-Flash-Q8_0-00006-of-00010.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:19a2b0e85cce09a47c826d8a670f832792752fdb5a996445fee797e0ce0d100f
+size 43316676032

DeepSeek-V4.1-Flash-Q8_0-00007-of-00010.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f04e8d27188e9a9cb4f7999a7fc099c5f52e043e23dd1b333a2294f1c1a0af84
+size 28877784064

DeepSeek-V4.1-Flash-Q8_0-00008-of-00010.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dc05657987ace747fabdf6c9c9b498fd46253329c78e7e795533b1885c292330
+size 104449677888

DeepSeek-V4.1-Flash-Q8_0-00009-of-00010.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f8fe6989ba5956981e1f896167b495b343ce184c24bfdff9eec981c8c541da9d
+size 104452537696

DeepSeek-V4.1-Flash-Q8_0-00010-of-00010.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f7f00c58e282623f08e43544dc31581f0e03a527bd05330054bb6bc2cfce384b
+size 10268402880

README.md ADDED Viewed

	@@ -0,0 +1,70 @@

+---
+license: mit
+base_model: deepseek-ai/DeepSeek-V4.1-Flash
+base_model_relation: quantized
+library_name: gguf
+pipeline_tag: text-generation
+tags:
+- gguf
+- deepseek
+- deepseek-v4.1
+- llama.cpp
+quantized_by: vcruz305
+---
+# DeepSeek-V4.1-Flash GGUF
+llama.cpp GGUF of [deepseek-ai/DeepSeek-V4.1-Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash).
+This is **V4.1-Flash** (`DeepseekV41ForCausalLM`), a causal decoder with engram n-gram lookup
+tables, hyper-connections and sparse attention. It is not V4-Flash-0731.
+## Recipe
+How to build the engine, serve it, and the gotchas, plus the current status:
+[vcruz305/DeepSeek-V4.1-Flash-GGUF-DGX-Spark-recipe](https://github.com/vcruz305/DeepSeek-V4.1-Flash-GGUF-DGX-Spark-recipe)
+## Status
+**These files do not run on upstream llama.cpp yet.** Conversion works and is open as
+[ggml-org/llama.cpp#28696](https://github.com/ggml-org/llama.cpp/pull/28696). The runtime is in
+progress on the `runtime/deepseek41` branch of
+[vcruz305/llama.cpp](https://github.com/vcruz305/llama.cpp): the loader, the engram tables and the
+hyper-connections work and are verified against the reference implementation, and the sparse
+attention is the remaining piece.
+Weights land here as each rung finishes. Anything converted before 2026-09-10 carries
+`general.architecture = deepseek4` and is being redone as `deepseek41`.
+The architecture string is `deepseek41`, following llama.cpp's habit of dropping the `_v`
+(`deepseek_v2` became `deepseek2`, `deepseek_v3.2` became `deepseek32`).
+**2026-09-11 fix:** the 4 Engram KV keys (`head_count`, `key_length`, `max_ngram_size`,
+`layer_ids`) were written with a hardcoded `deepseek4.engram.*` prefix instead of resolving
+`{arch}.engram.*` like every other arch-scoped key in the file. `general.architecture` and all
+38 other arch-scoped keys were already correct (`deepseek41.*`); only these 4 were wrong, which
+would have made the `runtime/deepseek41` loader fail to find Engram config on an otherwise
+loadable file. Fixed in place via a KV-only rewrite (tensor data untouched, verified
+byte-identical by SHA-256) on all five quant rungs' first shard, where GGUF split files store
+metadata. Confirmed live: all five now read `deepseek41.engram.*`.
+## Files
+Ladder in order: **Q2_K, Q3_K_M, Q4_K_M**. Measured tensor payload:
+| File | Quant | Bytes | GiB |
+| --- | --- | ---: | ---: |
+| `DeepSeek-V4.1-Flash-Q2_K.gguf` | Q2_K | 264,514,761,248 | 246.3 |
+| `DeepSeek-V4.1-Flash-Q3_K_M.gguf` | Q3_K_M | 347,270,954,112 | 323.4 |
+| `DeepSeek-V4.1-Flash-Q4_K_M.gguf` | Q4_K_M | pending | |
+Split into parts, since each exceeds the Hub's single file limit.
+Q5_K_M is skipped unless asked for. The routed experts arrive as MXFP4 at 4.25 bpw, so higher rungs
+move parts of the mixture *up* rather than down: Q3_K_M already lands at 0.684 of the Q8_0 staging
+file, and Q5_K_M would be close enough to Q8_0 to be poor value.
+Most of the file is the two engram tables, roughly 196.6B parameters between them. They follow the
+rung, 99,611 to 40,284 MiB each between q8_0 and q3_K.
+Apache/MIT from upstream. Credit: DeepSeek. GGUF pack: Victor Cruz (`vcruz305`).

llama.cpp/patches/README.md ADDED Viewed

	@@ -0,0 +1,79 @@

+# llama.cpp conversion patch for DeepSeek-V4.1
+The GGUFs in this repo were converted with a patched llama.cpp. Upstream does not yet know
+`DeepseekV41ForCausalLM`, so a stock checkout cannot produce them.
+The same change is submitted upstream as
+[ggml-org/llama.cpp#28696](https://github.com/ggml-org/llama.cpp/pull/28696). This copy is here so
+you can convert V4.1 yourself before that lands.
+## Apply
+```sh
+python patch_llamacpp_v41.py /path/to/llama.cpp            # apply
+python patch_llamacpp_v41.py /path/to/llama.cpp --check    # report status, change nothing
+python patch_llamacpp_v41.py /path/to/llama.cpp --revert   # undo
+```
+It is idempotent, and it writes a `.py.v41orig` backup beside every file it edits.
+## What it touches
+| file | change |
+|---|---|
+| `conversion/deepseek.py` | `DeepseekV41Model`, subclassing the existing `DeepseekV4Model` |
+| `gguf-py/gguf/constants.py` | the `deepseek41` arch, its tensor list, and four engram tensor entries |
+| `conversion/__init__.py` | one registry line |
+## Architecture name
+Files produced by this patch carry `general.architecture = deepseek41`.
+llama.cpp does not mirror the HF `model_type`, it drops the `_v`: `deepseek_v2` became
+`deepseek2`, `deepseek_v3.2` became `deepseek32`, `qwen2_moe` became `qwen2moe`. Only 10 of the
+150 arch strings in `constants.py` contain an underscore at all, so V4.1 is `deepseek41`.
+vLLM's `deepseek_v41` names a set of vLLM plugins, a tokenizer mode and two parsers, and is not
+a GGUF architecture value.
+The behavioural reason matters more than the convention. Riding on `deepseek4` sends a V4.1 file
+to the V4 loader, which then asks for `output_hc_fn`, `output_hc_base` and `output_hc_scale`.
+V4.1 ships none of the three, while it does ship the per-layer `hc_attn_*` and `hc_ffn_*`, so the
+loader sees a confusing partial match rather than refusing the file. `deepseek4` was also not
+strictly correct: V4.1 emits `attn_kv_a_norm`, which `DEEPSEEK4`'s own declared tensor list does
+not contain.
+The `deepseek41` tensor list is the 39 families the converter actually writes. It differs from
+`DEEPSEEK4` by dropping `HC_HEAD_{FN,BASE,SCALE}`, `FFN_GATE_TID2EID`, `ATTN_KV_NORM`,
+`ATTN_COMPRESSOR_APE`, `INDEXER_COMPRESSOR_*` and all six `NEXTN_*`, and by adding the four
+engram entries plus `ATTN_KV_A_NORM`. This patch leaves `DEEPSEEK4` itself untouched.
+This name is not yet settled upstream. It is proposed on the PR, and if the maintainers choose
+differently the arch string in already-converted files can be restamped with
+`gguf-py/gguf/scripts/gguf_new_metadata.py` without re-quantizing.
+## Why a subclass is not enough on its own
+Four things differ from V4 and each one is quiet rather than loud:
+- **FP8 scale block size.** V4 hardcodes `repeat_interleave(128, ...)` to match its
+  `weight_block_size` of `[128, 128]`. V4.1 declares `[32, 32]`. Running the V4 path unchanged
+  rescales every dequantized weight, raises nothing, and yields a model that loads and reads
+  fluently while being numerically wrong. The block size is read from `quantization_config`.
+- **`num_hash_layers`** is absent in V4.1 while the V4 path reads it unconditionally.
+- **Nested config.** V4.1 puts text parameters under `text_config` and vision under
+  `vision_config`. Overriding `load_hparams` does not work, because `ModelBase.__init__` calls it
+  explicitly rather than through the instance, so they are promoted in `index_tensors`.
+- **The engram tables.** Two of them, on layers 1 and 14, each `384,006,168 x 256`. The inherited
+  dequant computes `weight.float() * scale` over the whole tensor, about 393 GB as float32 for a
+  single table. They get a streaming path instead: read in row blocks straight from the
+  safetensors shard, quantized per block, accumulated into a disk backed memmap. Their scale
+  layout also differs from the rest of the checkpoint, `[rows, 8]` rather than the
+  `[rows/32, cols/32]` tiling the linear weights use.
+Tunable through `_V41_ENGRAM_CHUNK_ROWS` and `V41_ENGRAM_TMPDIR`.
+## Status
+Conversion only. A converted file does not load yet: the `deepseek4` runtime wants
+`output_hc_fn`, `output_hc_base` and `output_hc_scale`, and V4.1 does not ship those tensors.
+Runtime support is separate work and is not in this patch.

llama.cpp/patches/fix_gguf_engram_kv.py ADDED Viewed

	@@ -0,0 +1,290 @@

+#!/usr/bin/env python3
+"""Repair the engram metadata of a DeepSeek-V4.1 GGUF that was written before the converter fix.
+Two things went wrong in those files and both live in the header of the first shard:
+  1. the four engram keys carry a hardcoded `deepseek4.` prefix, so a `deepseek41` model looks for
+     `deepseek41.engram.head_count` and finds nothing
+  2. the five constants the hash actually needs are absent, because gguf-py's add_array() maps every
+     Python int to INT32, the 47 bit multipliers raised struct.error, and a broad except downgraded
+     that to a warning
+Tensor data is untouched. Existing key/value pairs are re-emitted byte for byte, apart from the
+four that get renamed, so nothing this script does not understand can be corrupted by it.
+  python fix_gguf_engram_kv.py shard1.gguf out.gguf --model-dir /path/to/DeepSeek-V4.1-Flash
+"""
+import argparse
+import os
+import struct
+import sys
+GGUF_MAGIC = b"GGUF"
+# value type tags
+T_UINT32 = 4
+T_INT32  = 5
+T_STRING = 8
+T_ARRAY  = 9
+T_UINT64 = 10
+FIXED = {0: 1, 1: 1, 2: 2, 3: 2, 4: 4, 5: 4, 6: 4, 7: 1, 10: 8, 11: 8, 12: 8}
+def _is_prime(n: int) -> bool:
+    if n < 2:
+        return False
+    for p in (2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37):
+        if n % p == 0:
+            return n == p
+    i = 41
+    while i * i <= n:
+        if n % i == 0 or n % (i + 2) == 0:
+            return False
+        i += 6
+    return True
+def _next_prime(start: int, seen: set) -> int:
+    c = start + 1
+    while not _is_prime(c) or c in seen:
+        c += 1
+    return c
+def build_token_map(model_dir):
+    """Case folded, accent stripped vocabulary, exactly as the reference builds it."""
+    from tokenizers import Regex, normalizers
+    from transformers import AutoTokenizer
+    tok = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
+    sentinel = ""  # private use char, so a lone space survives Strip()
+    norm = normalizers.Sequence([
+        normalizers.NFKC(),
+        normalizers.NFD(),
+        normalizers.StripAccents(),
+        normalizers.Lowercase(),
+        normalizers.Replace(Regex(r"[ \t\r\n]+"), " "),
+        normalizers.Replace(Regex(r"^ $"), sentinel),
+        normalizers.Strip(),
+        normalizers.Replace(sentinel, " "),
+    ])
+    backend = tok.backend_tokenizer
+    key_to_new, lookup = {}, [0] * len(tok)
+    for tid in range(len(tok)):
+        text = backend.decode([tid], skip_special_tokens=False)
+        if "�" in text:
+            key = backend.id_to_token(tid)
+        else:
+            normalized = norm.normalize_str(text)
+            key = normalized if normalized else text
+        new = key_to_new.get(key)
+        if new is None:
+            new = len(key_to_new)
+            key_to_new[key] = new
+        lookup[tid] = new
+    return lookup, len(key_to_new)
+def build_constants(model_dir, layer_ids, max_ngram, n_heads, vocab_size, pad_raw):
+    import numpy as np
+    token_map, compressed = build_token_map(model_dir)
+    max_long = np.iinfo(np.int64).max
+    bound = max(1, (max_long // compressed) // 2)
+    mults = []
+    for lid in layer_ids:
+        rng = np.random.default_rng(10007 * lid)
+        mults.extend(int(v) * 2 + 1 for v in rng.integers(0, bound, size=(max_ngram,), dtype=np.int64))
+    primes, seen = [], set()
+    for _ in layer_ids:
+        for _ in range(max_ngram - 1):
+            cur = vocab_size - 1
+            for _ in range(n_heads):
+                cur = _next_prime(cur, seen)
+                seen.add(cur)
+                primes.append(cur)
+    per_layer = (max_ngram - 1) * n_heads
+    offsets = []
+    for l in range(len(layer_ids)):
+        acc = 0
+        for b in range(per_layer):
+            offsets.append(acc)
+            acc += primes[l * per_layer + b]
+    return {
+        "multipliers": mults,
+        "primes": primes,
+        "offsets": offsets,
+        "token_map": token_map,
+        "pad_id": token_map[pad_raw],
+        "compressed_vocab": compressed,
+    }
+def kv_uint32(v):
+    return struct.pack("<I", T_UINT32) + struct.pack("<I", v)
+def kv_array(elem_type, values):
+    fmt = {T_INT32: "<i", T_UINT64: "<Q"}[elem_type]
+    out = [struct.pack("<I", T_ARRAY), struct.pack("<I", elem_type), struct.pack("<Q", len(values))]
+    out.extend(struct.pack(fmt, int(v)) for v in values)
+    return b"".join(out)
+def kv_entry(key, value_bytes):
+    k = key.encode("utf-8")
+    return struct.pack("<Q", len(k)) + k + value_bytes
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("src")
+    ap.add_argument("dst")
+    ap.add_argument("--model-dir", required=True, help="the original checkpoint, for its tokenizer")
+    ap.add_argument("--arch", default="deepseek41")
+    ap.add_argument("--engram-vocab", type=int, default=16_000_000)
+    ap.add_argument("--engram-pad-id", type=int, default=2)
+    args = ap.parse_args()
+    f = open(args.src, "rb")
+    assert f.read(4) == GGUF_MAGIC, "not a gguf"
+    version, = struct.unpack("<I", f.read(4))
+    n_tensors, = struct.unpack("<Q", f.read(8))
+    n_kv, = struct.unpack("<Q", f.read(8))
+    def rstr():
+        n, = struct.unpack("<Q", f.read(8))
+        return f.read(n).decode("utf-8")
+    def skip_value(t):
+        if t == T_STRING:
+            n, = struct.unpack("<Q", f.read(8))
+            f.seek(n, os.SEEK_CUR)
+        elif t == T_ARRAY:
+            et, = struct.unpack("<I", f.read(4))
+            cnt, = struct.unpack("<Q", f.read(8))
+            if et == T_STRING:
+                for _ in range(cnt):
+                    n, = struct.unpack("<Q", f.read(8))
+                    f.seek(n, os.SEEK_CUR)
+            else:
+                f.seek(FIXED[et] * cnt, os.SEEK_CUR)
+        else:
+            f.seek(FIXED[t], os.SEEK_CUR)
+    kvs = []          # (key, raw value bytes including the type tag)
+    seen_keys = set()
+    for _ in range(n_kv):
+        key = rstr()
+        vstart = f.tell()
+        t, = struct.unpack("<I", f.read(4))
+        skip_value(t)
+        vend = f.tell()
+        f.seek(vstart)
+        raw = f.read(vend - vstart)
+        kvs.append([key, raw])
+        seen_keys.add(key)
+    tensor_info_start = f.tell()
+    for _ in range(n_tensors):
+        rstr()
+        ndim, = struct.unpack("<I", f.read(4))
+        f.seek(8 * ndim, os.SEEK_CUR)
+        f.seek(4, os.SEEK_CUR)    # ggml type
+        f.seek(8, os.SEEK_CUR)    # offset
+    tensor_info_end = f.tell()
+    f.seek(tensor_info_start)
+    tensor_info_raw = f.read(tensor_info_end - tensor_info_start)
+    alignment = 32
+    for key, raw in kvs:
+        if key == "general.alignment":
+            alignment, = struct.unpack("<I", raw[4:8])
+    data_start = (tensor_info_end + alignment - 1) // alignment * alignment
+    # --- rename the mis-prefixed keys -------------------------------------------------
+    renamed = 0
+    for kv in kvs:
+        if kv[0].startswith("deepseek4.engram."):
+            kv[0] = args.arch + "." + kv[0][len("deepseek4."):]
+            renamed += 1
+    def get_scalar(name):
+        for key, raw in kvs:
+            if key == name:
+                t, = struct.unpack("<I", raw[:4])
+                return struct.unpack("<I" if t in (T_UINT32,) else "<i", raw[4:8])[0]
+        return None
+    layer_ids = None
+    for key, raw in kvs:
+        if key == f"{args.arch}.engram.layer_ids":
+            et, = struct.unpack("<I", raw[4:8])
+            cnt, = struct.unpack("<Q", raw[8:16])
+            fmt = {T_INT32: "<i", T_UINT32: "<I", T_UINT64: "<Q"}[et]
+            sz = FIXED[et]
+            layer_ids = [struct.unpack(fmt, raw[16 + i * sz: 16 + (i + 1) * sz])[0] for i in range(cnt)]
+    n_heads = get_scalar(f"{args.arch}.engram.head_count")
+    max_ngram = get_scalar(f"{args.arch}.engram.max_ngram_size")
+    if layer_ids is None or n_heads is None or max_ngram is None:
+        sys.exit("could not read the engram layer ids, head count or ngram size from the header")
+    print(f"  arch={args.arch} layer_ids={layer_ids} heads={n_heads} max_ngram={max_ngram}")
+    print(f"  renamed {renamed} mis-prefixed keys")
+    const = build_constants(args.model_dir, layer_ids, max_ngram, n_heads,
+                            args.engram_vocab, args.engram_pad_id)
+    print(f"  compressed vocab {const['compressed_vocab']}, token map {len(const['token_map'])}, "
+          f"{len(const['primes'])} primes, pad_id {const['pad_id']}")
+    print(f"  first multipliers {const['multipliers'][:3]} (max bits "
+          f"{max(const['multipliers']).bit_length()})")
+    additions = [
+        (f"{args.arch}.engram.multipliers", kv_array(T_UINT64, const["multipliers"])),
+        (f"{args.arch}.engram.primes",      kv_array(T_UINT64, const["primes"])),
+        (f"{args.arch}.engram.offsets",     kv_array(T_UINT64, const["offsets"])),
+        (f"{args.arch}.engram.token_map",   kv_array(T_INT32,  const["token_map"])),
+        (f"{args.arch}.engram.pad_id",      kv_uint32(const["pad_id"])),
+    ]
+    additions = [(k, v) for k, v in additions if k not in {kv[0] for kv in kvs}]
+    print(f"  adding {len(additions)} keys")
+    header = bytearray()
+    header += GGUF_MAGIC
+    header += struct.pack("<I", version)
+    header += struct.pack("<Q", n_tensors)
+    header += struct.pack("<Q", len(kvs) + len(additions))
+    for key, raw in kvs:
+        header += kv_entry(key, raw)
+    for key, raw in additions:
+        header += kv_entry(key, raw)
+    header += tensor_info_raw
+    pad = (-len(header)) % alignment
+    header += b"\x00" * pad
+    src_size = os.path.getsize(args.src)
+    print(f"  header {tensor_info_end} -> {len(header)} bytes, copying "
+          f"{(src_size - data_start)/1e9:.1f} GB of tensor data")
+    f.seek(data_start)
+    with open(args.dst, "wb") as out:
+        out.write(header)
+        while True:
+            chunk = f.read(64 << 20)
+            if not chunk:
+                break
+            out.write(chunk)
+    print(f"  wrote {args.dst} ({os.path.getsize(args.dst)/1e9:.1f} GB)")
+if __name__ == "__main__":
+    main()

llama.cpp/patches/patch_llamacpp_v41.py ADDED Viewed

	@@ -0,0 +1,701 @@

+"""Add DeepSeek V4.1 (DeepseekV41ForCausalLM) conversion support to llama.cpp.
+Upstream llama.cpp master carries a full DeepSeek V4 implementation in conversion/deepseek.py
+(DeepseekV4Model, DeepseekV4DSparkModel, DeepseekV4FlashVisionModel). V4.1 is close enough to
+subclass rather than reimplement. This patch adds the V4.1 pieces and fixes one bug that would
+otherwise corrupt V4.1 weights silently.
+Differences handled, each verified against the published DeepSeek-V4.1-Flash config.json and
+model.safetensors.index.json rather than assumed:
+1. FP8 dequantization block size. V4's dequant_model hardcodes repeat_interleave(128, ...),
+   matching V4's weight_block_size of [128, 128]. V4.1 declares [32, 32]. Running the V4 path
+   unchanged produces wrong weights with no error and fluent but incorrect output, so the block
+   size is read from quantization_config instead.
+2. num_hash_layers is absent from the V4.1 config. V4 reads it unconditionally and would raise
+   KeyError, so it defaults to 0 here.
+3. The V4.1 config nests its text parameters under text_config and its vision parameters under
+   vision_config. V4 expects them flat.
+4. Six tensor families exist in V4.1 that V4 does not map. Four need new enum entries;
+   INDEXER_K_NORM and INDEXER_ATTN_K already exist upstream and are reused:
+       layers.N.engram.embed.weight      layers.N.engram.k_weight
+       layers.N.engram.q_weight          layers.N.engram.wkv.weight
+       layers.N.attn.indexer.k_norm.weight
+       layers.N.attn.indexer.wk.weight
+   The engram tables are the largest single component of the model at roughly 196.6B parameters
+   across layers 1 and 14, which is about 36 percent of the checkpoint.
+5. The two engram tables need their own write path. Each is 384,006,168 x 256, so 98.3
+   billion elements, and the inherited FP8 dequant materializes 393 GB of float32 per
+   table. They are quantized in row blocks into a disk-backed memmap instead. Their scale
+   layout also differs: [rows, 8], one scale per 32 columns of a single row, rather than
+   the [rows/32, cols/32] tiling the generic path assumes.
+6. V4.1 lacks attn.compressor.ape and the attn.indexer.compressor.* family that V4 maps. Those
+   entries stay in the inherited map and simply go unused.
+This patch covers CONVERSION only. Running the resulting GGUF additionally requires a llama.cpp
+runtime graph for V4.1, which is separate work.
+usage: python patch_llamacpp_v41.py <llama.cpp checkout>  [--revert] [--check]
+"""
+import pathlib
+import shutil
+import sys
+# ---------------------------------------------------------------- constants.py
+CONST_TENSOR_ENUM_ANCHOR = "    INDEXER_PROJ "
+CONST_TENSOR_ENUM_NEW = """    ENGRAM_EMBD            = auto()
+    ENGRAM_K               = auto()
+    ENGRAM_Q               = auto()
+    ENGRAM_WKV             = auto()
+"""
+CONST_TENSOR_NAME_ANCHOR = "    MODEL_TENSOR.INDEXER_PROJ:"
+CONST_TENSOR_NAME_NEW = """    MODEL_TENSOR.ENGRAM_EMBD:               "blk.{bid}.engram_embd",
+    MODEL_TENSOR.ENGRAM_K:                  "blk.{bid}.engram_k",
+    MODEL_TENSOR.ENGRAM_Q:                  "blk.{bid}.engram_q",
+    MODEL_TENSOR.ENGRAM_WKV:                "blk.{bid}.engram_wkv",
+"""
+# V4.1 gets its own architecture rather than riding on DEEPSEEK4.
+#
+# llama.cpp does not mirror the HF model_type, it drops the "_v": deepseek_v2 became
+# deepseek2, deepseek_v3.2 became deepseek32, qwen2_moe became qwen2moe. Only 10 of the
+# 150 arch strings in constants.py contain an underscore at all, so V4.1 is "deepseek41".
+#
+# The behavioural reason matters more than the convention. Sharing deepseek4 sends a V4.1
+# file to the V4 loader, which asks for output_hc_fn, output_hc_base and output_hc_scale.
+# V4.1 ships none of the three, while it does ship the per-layer hc_attn_* and hc_ffn_*,
+# so the loader sees a confusing partial match instead of refusing the file. Riding on
+# DEEPSEEK4 was also not strictly correct: V4.1 emits attn_kv_a_norm, which DEEPSEEK4's
+# own declared tensor list does not contain.
+#
+# The list below is derived from the 39 tensor families the converter actually wrote,
+# reverse mapped through TENSOR_NAMES, rather than copied from DEEPSEEK4 and trimmed.
+# The engram KV keys, declared the way every other arch-scoped key is: with an {arch}
+# placeholder. PerLayerEmbedding directly below is the closest existing analogue, and llama.cpp's
+# own lazy-read comment pairs PLE and engrams for the same reason.
+CONST_KV_ANCHOR = "    class PerLayerEmbedding:\n"
+CONST_KV_NEW = '''    class Engram:
+        LAYER_IDS      = "{arch}.engram.layer_ids"
+        HEAD_COUNT     = "{arch}.engram.head_count"
+        KEY_LENGTH     = "{arch}.engram.key_length"
+        MAX_NGRAM_SIZE = "{arch}.engram.max_ngram_size"
+        MULTIPLIERS    = "{arch}.engram.multipliers"
+        PRIMES         = "{arch}.engram.primes"
+        OFFSETS        = "{arch}.engram.offsets"
+        TOKEN_MAP      = "{arch}.engram.token_map"
+        PAD_ID         = "{arch}.engram.pad_id"
+    class PerLayerEmbedding:
+'''
+CONST_ARCH_ENUM_ANCHOR = "    DEEPSEEK4        = auto()\n"
+CONST_ARCH_ENUM_NEW = """    DEEPSEEK4        = auto()
+    DEEPSEEK41       = auto()
+"""
+CONST_ARCH_NAME_ANCHOR = '    MODEL_ARCH.DEEPSEEK4:        "deepseek4",\n'
+CONST_ARCH_NAME_NEW = '''    MODEL_ARCH.DEEPSEEK4:        "deepseek4",
+    MODEL_ARCH.DEEPSEEK41:       "deepseek41",
+'''
+CONST_ARCH_TENSORS_ANCHOR = "    MODEL_ARCH.DEEPSEEK4: [\n"
+CONST_ARCH_TENSORS_NEW = """    MODEL_ARCH.DEEPSEEK41: [
+        MODEL_TENSOR.TOKEN_EMBD,
+        MODEL_TENSOR.OUTPUT,
+        MODEL_TENSOR.OUTPUT_NORM,
+        MODEL_TENSOR.ATTN_NORM,
+        MODEL_TENSOR.ATTN_SINKS,
+        MODEL_TENSOR.FFN_GATE_INP,
+        MODEL_TENSOR.FFN_NORM,
+        MODEL_TENSOR.FFN_GATE_EXP,
+        MODEL_TENSOR.FFN_DOWN_EXP,
+        MODEL_TENSOR.FFN_UP_EXP,
+        MODEL_TENSOR.FFN_GATE_SHEXP,
+        MODEL_TENSOR.FFN_DOWN_SHEXP,
+        MODEL_TENSOR.FFN_UP_SHEXP,
+        MODEL_TENSOR.FFN_EXP_PROBS_B,
+        MODEL_TENSOR.FFN_EXP_PROBS_B_VL,
+        MODEL_TENSOR.ATTN_Q_A,
+        MODEL_TENSOR.ATTN_Q_B,
+        MODEL_TENSOR.ATTN_Q_A_NORM,
+        MODEL_TENSOR.ATTN_KV_A_NORM,
+        MODEL_TENSOR.ATTN_KV,
+        MODEL_TENSOR.ATTN_OUT_A,
+        MODEL_TENSOR.ATTN_OUT_B,
+        MODEL_TENSOR.HC_ATTN_FN,
+        MODEL_TENSOR.HC_ATTN_BASE,
+        MODEL_TENSOR.HC_ATTN_SCALE,
+        MODEL_TENSOR.HC_FFN_FN,
+        MODEL_TENSOR.HC_FFN_BASE,
+        MODEL_TENSOR.HC_FFN_SCALE,
+        MODEL_TENSOR.ATTN_COMPRESSOR_WKV,
+        MODEL_TENSOR.ATTN_COMPRESSOR_WGATE,
+        MODEL_TENSOR.ATTN_COMPRESSOR_NORM,
+        MODEL_TENSOR.INDEXER_K_NORM,
+        MODEL_TENSOR.ENGRAM_EMBD,
+        MODEL_TENSOR.ENGRAM_K,
+        MODEL_TENSOR.ENGRAM_Q,
+        MODEL_TENSOR.ENGRAM_WKV,
+        MODEL_TENSOR.INDEXER_PROJ,
+        MODEL_TENSOR.INDEXER_ATTN_K,
+        MODEL_TENSOR.INDEXER_ATTN_Q_B,
+    ],
+    MODEL_ARCH.DEEPSEEK4: [
+"""
+# ---------------------------------------------------------------- deepseek.py
+V41_CLASS = '''
+def _v41_is_prime(n: int) -> bool:
+    """Trial division, deliberately not sympy.
+    The reference uses sympy.isprime, but sympy is not a llama.cpp conversion dependency and
+    pulling in a computer algebra system to test primality would be hard to justify. The
+    candidates here sit just above engram_vocab_size, about 16 million, so trial division runs
+    to sqrt(n) which is around 4000 and costs nothing. There are 48 primes to find in total.
+    """
+    if n < 2:
+        return False
+    for p in (2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37):
+        if n % p == 0:
+            return n == p
+    f = 41
+    while f * f <= n:
+        # 6k +/- 1 wheel, having already cleared the small primes above
+        if n % f == 0 or n % (f + 2) == 0:
+            return False
+        f += 6
+    return True
+def _v41_find_next_prime(start: int, seen_primes: set[int]) -> int:
+    """The smallest prime above start that has not been handed out yet."""
+    candidate = start + 1
+    while not _v41_is_prime(candidate) or candidate in seen_primes:
+        candidate += 1
+    return candidate
+def _v41_build_compressed_token_map(tokenizer) -> tuple[list[int], int]:
+    """Map token ids to a compressed id space where normalized tokens collapse together."""
+    from tokenizers import Regex, normalizers
+    sentinel = "\\ue000"  # private-use char to preserve single spaces
+    normalizer = normalizers.Sequence([
+        normalizers.NFKC(),
+        normalizers.NFD(),
+        normalizers.StripAccents(),
+        normalizers.Lowercase(),
+        normalizers.Replace(Regex(r"[ \\t\\r\\n]+"), " "),
+        normalizers.Replace(Regex(r"^ $"), sentinel),
+        normalizers.Strip(),
+        normalizers.Replace(sentinel, " "),
+    ])
+    backend = tokenizer.backend_tokenizer
+    key_to_new: dict[str, int] = {}
+    lookup = [0] * len(tokenizer)
+    for token_id in range(len(tokenizer)):
+        text = backend.decode([token_id], skip_special_tokens=False)
+        if "\\ufffd" in text:
+            key = backend.id_to_token(token_id)
+        else:
+            normalized = normalizer.normalize_str(text)
+            key = normalized if normalized else text
+        new_id = key_to_new.get(key)
+        if new_id is None:
+            new_id = len(key_to_new)
+            key_to_new[key] = new_id
+        lookup[token_id] = new_id
+    return lookup, len(key_to_new)
+def _v41_compute_hash_multipliers(layer_ids: tuple[int, ...], max_ngram_size: int, tokenizer_vocab_size: int):
+    """Generate odd multipliers for n-gram hashing, one per (layer, lookback)."""
+    import numpy as np
+    import torch
+    max_long = np.iinfo(np.int64).max
+    multiplier_bound = max(1, (max_long // tokenizer_vocab_size) // 2)
+    rows = []
+    for layer_id in layer_ids:
+        generator = np.random.default_rng(10007 * layer_id)
+        values = generator.integers(low=0, high=multiplier_bound, size=(max_ngram_size,), dtype=np.int64)
+        rows.append(torch.tensor(values * 2 + 1))
+    return torch.stack(rows)
+@ModelBase.register("DeepseekV41ForCausalLM")
+@ModelBase.example("deepseek-ai/DeepSeek-V4.1-Flash")
+class DeepseekV41Model(DeepseekV4Model):
+    """DeepSeek V4.1. Subclasses V4 and overrides only where the checkpoint differs.
+    See patch_llamacpp_v41.py for the verified list of differences. The important one is the
+    FP8 block size: V4 is [128, 128] and V4.1 is [32, 32], and using the wrong value corrupts
+    every dequantized weight without raising.
+    """
+    model_arch = gguf.MODEL_ARCH.DEEPSEEK41
+    def _v41_flatten_hparams(self):
+        """Promote the nested text_config to the top level.
+        V4.1 nests its text parameters under text_config while the inherited V4 code expects
+        them flat. Overriding load_hparams does not work, because base.__init__ calls
+        ModelBase.load_hparams explicitly rather than through the instance, so the seam is the
+        first consumer of hparams instead, which is index_tensors.
+        """
+        for key, value in (self.hparams.get("text_config") or {}).items():
+            self.hparams.setdefault(key, value)
+        # absent in V4.1; V4 reads it unconditionally
+        self.hparams.setdefault("num_hash_layers", 0)
+    def index_tensors(self, remote_hf_model_id=None):
+        self._v41_flatten_hparams()
+        return super().index_tensors(remote_hf_model_id=remote_hf_model_id)
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        with open(self.dir_model / "config.json", "r", encoding="utf-8") as f:
+            raw = json.load(f)
+        # the FP8 scale block size, read rather than assumed
+        qcfg = raw.get("quantization_config") or {}
+        block = qcfg.get("weight_block_size") or [128, 128]
+        self._v41_block_rows = int(block[0])
+        self._v41_block_cols = int(block[1] if len(block) > 1 else block[0])
+        logger.info(
+            "DeepSeek V4.1: fp8 weight_block_size %dx%d, engram layers %s",
+            self._v41_block_rows, self._v41_block_cols,
+            self.hparams.get("engram_layer_ids"),
+        )
+        self.block_count = self.hparams["num_hidden_layers"]
+        if self.mtp_only:
+            self.block_count += self.hparams.get("num_nextn_predict_layers", 0)
+        self.tensor_map = gguf.get_tensor_name_map(self.model_arch, self.block_count)
+    @classmethod
+    def filter_tensors(cls, item):
+        name, _ = item
+        # the vision tower and its aligner are exported separately as an mmproj file
+        if name.startswith(("vision.", "aligner.", "image_")):
+            return None
+        return super().filter_tensors(item)
+    def dequant_model(self):
+        """Same as V4 but with the block size taken from the checkpoint."""
+        fp8_dtypes = self._float8_dtypes()
+        tensors_to_remove: list[str] = []
+        rows, cols = self._v41_block_rows, self._v41_block_cols
+        def dequant_fp8_weight(weight: Tensor, scale: Tensor) -> Tensor:
+            out_features, in_features = weight.shape
+            scale_f = self._e8m0_to_float(scale)
+            scale_f = scale_f.repeat_interleave(rows, 0)[:out_features]
+            scale_f = scale_f.repeat_interleave(cols, 1)[:, :in_features]
+            return weight.float() * scale_f
+        for name in list(self.model_tensors.keys()):
+            if not name.endswith(".scale"):
+                continue
+            weight_name = name.removesuffix(".scale") + ".weight"
+            if weight_name not in self.model_tensors:
+                continue
+            weight = self.model_tensors[weight_name]
+            scale = self.model_tensors[name]
+            if weight().dtype not in fp8_dtypes:
+                continue
+            self.model_tensors[weight_name] = lambda w=weight, s=scale: dequant_fp8_weight(w(), s())
+            self._dsv4_fp8_dequantized.add(weight_name)
+            tensors_to_remove.append(name)
+        for name in tensors_to_remove:
+            del self.model_tensors[name]
+    def set_gguf_parameters(self):
+        super().set_gguf_parameters()
+        hparams = self.hparams
+        if (engram_ids := hparams.get("engram_layer_ids")) is not None:
+            # These MUST carry the arch prefix, not a literal "deepseek4.". llama.cpp resolves
+            # every LLM_KV_* as "{arch}.{key}", so a hardcoded prefix means the runtime looks up
+            # deepseek41.engram.head_count and finds nothing, on a file that otherwise loads.
+            arch = self.gguf_writer.arch
+            self.gguf_writer.add_uint32(gguf.Keys.Engram.HEAD_COUNT.format(arch=arch), hparams["engram_n_heads"])
+            self.gguf_writer.add_uint32(gguf.Keys.Engram.KEY_LENGTH.format(arch=arch), hparams["engram_head_dim"])
+            self.gguf_writer.add_uint32(gguf.Keys.Engram.MAX_NGRAM_SIZE.format(arch=arch), hparams["engram_max_ngram_size"])
+            self.gguf_writer.add_array(gguf.Keys.Engram.LAYER_IDS.format(arch=arch), engram_ids)
+            # Generate and write engram hash constants
+            import numpy as _np
+            # A model with engram layers cannot run without these constants, so every step below
+            # raises rather than warns: a file that is missing them loads and then hashes wrong.
+            from transformers import AutoTokenizer
+            tokenizer = AutoTokenizer.from_pretrained(self.dir_model, trust_remote_code=True)
+            # Build compressed token map
+            token_map, compressed_vocab_size = _v41_build_compressed_token_map(tokenizer)
+            expected_vocab = hparams.get("engram_compressed_vocab_size")
+            if compressed_vocab_size != expected_vocab:
+                # every multiplier derives from this size, so a mismatch rehashes the whole table
+                raise ValueError(f"compressed vocab size is {compressed_vocab_size}, config says {expected_vocab}")
+            # Compute multipliers
+            layer_ids = tuple(engram_ids)
+            multipliers = _v41_compute_hash_multipliers(layer_ids, hparams["engram_max_ngram_size"], compressed_vocab_size)
+            # Compute primes and offsets. Every (n-gram size, head) pair gets its own bucket, and
+            # the search starts over at engram_vocab_size - 1 for each n-gram size, so the shared
+            # `seen` set is what keeps the buckets distinct.
+            max_ngram_size = hparams["engram_max_ngram_size"]
+            n_heads = hparams["engram_n_heads"]
+            primes_list, seen_primes = [], set()
+            for layer_id in layer_ids:
+                per_ngram = []
+                for ngram_idx in range(max_ngram_size - 1):
+                    current_search = hparams["engram_vocab_size"] - 1
+                    sizes = []
+                    for head_idx in range(n_heads):
+                        current_search = _v41_find_next_prime(current_search, seen_primes)
+                        seen_primes.add(current_search)
+                        sizes.append(current_search)
+                    per_ngram.append(sizes)
+                primes_list.append(per_ngram)
+            # [n_engram_layers, max_ngram_size - 1, n_heads]
+            primes_array = _np.array(primes_list, dtype=_np.uint64)
+            # each bucket starts where the previous one ended, in that same order flattened
+            offsets_list = []
+            for layer_primes in primes_array:
+                flat_primes = layer_primes.flatten()
+                offsets = _np.cumsum(_np.concatenate(([0], flat_primes[:-1])))
+                offsets_list.append(offsets.reshape(layer_primes.shape))
+            offsets_array = _np.array(offsets_list, dtype=_np.uint64)
+            # Write constants to GGUF.
+            # add_array() infers the element type from the first item and maps every Python int to
+            # INT32, which would truncate the multipliers, so pass the element type explicitly and
+            # flatten by hand: a gguf array is one dimensional.
+            def add_u64(key, arr):
+                self.gguf_writer.add_key_value(
+                    key,
+                    [int(x) for x in _np.asarray(arr).reshape(-1)],
+                    gguf.GGUFValueType.ARRAY,
+                    gguf.GGUFValueType.UINT64,
+                )
+            add_u64(gguf.Keys.Engram.MULTIPLIERS.format(arch=arch), multipliers.numpy())
+            add_u64(gguf.Keys.Engram.PRIMES.format(arch=arch), primes_array)
+            add_u64(gguf.Keys.Engram.OFFSETS.format(arch=arch), offsets_array)
+            self.gguf_writer.add_key_value(
+                gguf.Keys.Engram.TOKEN_MAP.format(arch=arch),
+                [int(x) for x in token_map],
+                gguf.GGUFValueType.ARRAY,
+                gguf.GGUFValueType.INT32,
+            )
+            # the reference stores the padding token already mapped, so do the same here
+            self.gguf_writer.add_uint32(
+                gguf.Keys.Engram.PAD_ID.format(arch=arch),
+                int(token_map[hparams.get("engram_pad_id", 2)]),
+            )
+            logger.info("Engram constants written: multipliers %s, primes %s, offsets %s, token_map %d",
+                        multipliers.shape, primes_array.shape, offsets_array.shape, len(token_map))
+    # rows per block when rewriting an engram table; 1M rows is about 1 GB of float32 scratch
+    _V41_ENGRAM_CHUNK_ROWS = 1_000_000
+    def _write_engram_table(self, bid: int) -> list[str]:
+        """Quantize one engram table in row blocks, accumulating into a disk-backed memmap.
+        The inherited FP8 path cannot be used here. It computes weight.float() * scale over the
+        whole tensor, and one engram table is 384,006,168 x 256, so 98.3 billion elements, which
+        is 393 GB as float32. A run that reaches this tensor collapses from 118 GiB free to 13 GiB
+        and is killed. Every other tensor in the model converts normally.
+        The scale layout also differs from the rest of the checkpoint. Linear weights carry a
+        [rows/32, cols/32] scale matching weight_block_size [32, 32], while the engram scale is
+        [rows, 8], which is one scale per 32 columns within a single row. Broadcasting it the way
+        the generic path does would corrupt the table, so it is expanded along columns only.
+        Reading is done straight from the safetensors shard rather than through the lazy tensor
+        wrapper, because to_eager materializes the whole tensor before any slicing takes effect.
+        """
+        import json as _json
+        import os as _os
+        import tempfile as _tempfile
+        import numpy as _np
+        from safetensors import safe_open as _safe_open
+        weight_name = f"layers.{bid}.engram.embed.weight"
+        scale_name = f"layers.{bid}.engram.embed.scale"
+        index_path = self.dir_model / "model.safetensors.index.json"
+        with open(index_path, "r", encoding="utf-8") as f:
+            weight_map = _json.load(f)["weight_map"]
+        shard = self.dir_model / weight_map[weight_name]
+        qtype = gguf.GGMLQuantizationType.Q8_0
+        block_elems = gguf.GGML_QUANT_SIZES[qtype][0]
+        with _safe_open(str(shard), framework="pt") as f:
+            wsl = f.get_slice(weight_name)
+            n_rows, n_cols = (int(x) for x in wsl.get_shape())
+            has_scale = scale_name in f.keys()
+            ssl = f.get_slice(scale_name) if has_scale else None
+            scale_groups = int(ssl.get_shape()[1]) if has_scale else 0
+            if n_cols % block_elems:
+                raise ValueError(
+                    f"engram row width {n_cols} is not a multiple of the {qtype.name} block {block_elems}"
+                )
+            if has_scale and n_cols % scale_groups:
+                raise ValueError(
+                    f"engram row width {n_cols} is not divisible by its {scale_groups} scale groups"
+                )
+            per_group = n_cols // scale_groups if has_scale else 0
+            row_bytes = int(gguf.quantize(_np.zeros((1, n_cols), dtype=_np.float32), qtype).nbytes)
+            rows_per_chunk = min(int(self._V41_ENGRAM_CHUNK_ROWS), n_rows)
+            n_chunks = (n_rows + rows_per_chunk - 1) // rows_per_chunk
+            tmp_dir = _os.environ.get("V41_ENGRAM_TMPDIR") or _tempfile.gettempdir()
+            tmp_path = _os.path.join(tmp_dir, f"engram_{bid}_{qtype.name}.bin")
+            logger.info(
+                "engram layer %d: %d x %d, scale groups %d, %s in %d blocks of %d rows, staging %.1f GB at %s",
+                bid, n_rows, n_cols, scale_groups, qtype.name, n_chunks, rows_per_chunk,
+                n_rows * row_bytes / 1e9, tmp_path,
+            )
+            out = _np.memmap(tmp_path, dtype=_np.uint8, mode="w+", shape=(n_rows, row_bytes))
+            for ci, start in enumerate(range(0, n_rows, rows_per_chunk)):
+                stop = min(start + rows_per_chunk, n_rows)
+                chunk = wsl[start:stop, :].float()
+                if has_scale:
+                    s = self._e8m0_to_float(ssl[start:stop, :])
+                    chunk = chunk * s.repeat_interleave(per_group, 1)[:, :n_cols]
+                out[start:stop] = gguf.quantize(
+                    chunk.cpu().numpy().astype(_np.float32), qtype
+                ).reshape(stop - start, row_bytes)
+                del chunk
+                if ci % 25 == 0:
+                    logger.info("  engram layer %d: %d / %d rows", bid, stop, n_rows)
+            out.flush()
+        new_name = self.format_tensor_name(gguf.MODEL_TENSOR.ENGRAM_EMBD, bid, ".weight")
+        self.gguf_writer.add_tensor(new_name, out, raw_dtype=qtype)
+        logger.info("engram layer %d: wrote %s as %s", bid, new_name, qtype.name)
+        consumed = [weight_name]
+        if has_scale:
+            consumed.append(scale_name)
+        return consumed
+    def generate_extra_tensors(self):
+        yield from super().generate_extra_tensors()
+        consumed: list[str] = []
+        for bid in (self.hparams.get("engram_layer_ids") or []):
+            if f"layers.{bid}.engram.embed.weight" in self.model_tensors:
+                consumed.extend(self._write_engram_table(int(bid)))
+        for name in consumed:
+            if name in self.model_tensors:
+                del self.model_tensors[name]
+    def _map_dsv4_tensor_name(self, name: str, bid):
+        match = re.match(r"layers\\.(\\d+)\\.(.+)$", name)
+        if match is not None:
+            v41_only = {
+                "engram.embed.weight": (gguf.MODEL_TENSOR.ENGRAM_EMBD, ".weight"),
+                "engram.k_weight": (gguf.MODEL_TENSOR.ENGRAM_K, ".weight"),
+                "engram.q_weight": (gguf.MODEL_TENSOR.ENGRAM_Q, ".weight"),
+                "engram.wkv.weight": (gguf.MODEL_TENSOR.ENGRAM_WKV, ".weight"),
+                "attn.indexer.k_norm.weight": (gguf.MODEL_TENSOR.INDEXER_K_NORM, ".weight"),
+                "attn.indexer.wk.weight": (gguf.MODEL_TENSOR.INDEXER_ATTN_K, ".weight"),
+            }
+            tensor_name = match.group(2)
+            if tensor_name in v41_only:
+                return v41_only[tensor_name]
+        return super()._map_dsv4_tensor_name(name, bid)
+'''
+def patch_constants(path: pathlib.Path, revert: bool, check: bool) -> int:
+    text = path.read_text(encoding="utf-8")
+    backup = path.with_suffix(".py.v41orig")
+    if revert:
+        if backup.exists():
+            shutil.copy2(backup, path)
+            print("  reverted  gguf/constants.py")
+            return 0
+        print("  no backup gguf/constants.py")
+        return 4
+    # Check if ALL expected changes are already present
+    if "DEEPSEEK41" in text and "PAD_ID         = \"{arch}.engram.pad_id\"" in text:
+        print("  ok        gguf/constants.py (already has all engram constants)")
+        return 0
+    anchors = (
+        CONST_TENSOR_ENUM_ANCHOR,
+        CONST_TENSOR_NAME_ANCHOR,
+        CONST_KV_ANCHOR,
+        CONST_ARCH_ENUM_ANCHOR,
+        CONST_ARCH_NAME_ANCHOR,
+        CONST_ARCH_TENSORS_ANCHOR,
+    )
+    for anchor in anchors:
+        if anchor not in text:
+            print(f"  NO MATCH  gguf/constants.py, missing anchor: {anchor.strip()!r}")
+            return 5
+    if check:
+        print("  would patch gguf/constants.py")
+        return 0
+    # the engram tensor enum and its names are shared, the arch entries are new
+    if "ENGRAM_EMBD" not in text:
+        text = text.replace(CONST_TENSOR_ENUM_ANCHOR, CONST_TENSOR_ENUM_NEW + CONST_TENSOR_ENUM_ANCHOR, 1)
+        text = text.replace(CONST_TENSOR_NAME_ANCHOR, CONST_TENSOR_NAME_NEW + CONST_TENSOR_NAME_ANCHOR, 1)
+    # Handle Engram class: either add it or add the missing KV entries
+    if "class Engram:" not in text:
+        text = text.replace(CONST_KV_ANCHOR, CONST_KV_NEW, 1)
+    else:
+        # An older run of this patcher left an Engram class with only some of the keys.
+        # Add the missing ones one at a time, so re-running never duplicates a line.
+        anchor = "        MAX_NGRAM_SIZE = \"{arch}.engram.max_ngram_size\""
+        for name, key in (
+            ("MULTIPLIERS   ", "multipliers"),
+            ("PRIMES        ", "primes"),
+            ("OFFSETS       ", "offsets"),
+            ("TOKEN_MAP     ", "token_map"),
+            ("PAD_ID        ", "pad_id"),
+        ):
+            line = "        %s = \"{arch}.engram.%s\"" % (name, key)
+            if line not in text:
+                text = text.replace(anchor, anchor + "\n" + line, 1)
+    # Add arch enum, name, and tensor map entries (only if not already present)
+    if "DEEPSEEK41       = auto()" not in text:
+        text = text.replace(CONST_ARCH_ENUM_ANCHOR, CONST_ARCH_ENUM_NEW, 1)
+    if 'MODEL_ARCH.DEEPSEEK41:' not in text:
+        text = text.replace(CONST_ARCH_NAME_ANCHOR, CONST_ARCH_NAME_NEW, 1)
+    if "MODEL_ARCH.DEEPSEEK41: [" not in text:
+        text = text.replace(CONST_ARCH_TENSORS_ANCHOR, CONST_ARCH_TENSORS_NEW, 1)
+    if not backup.exists():
+        shutil.copy2(path, backup)
+    path.write_text(text, encoding="utf-8")
+    print("  patched   gguf/constants.py")
+    return 0
+def patch_init(path: pathlib.Path, revert: bool, check: bool) -> int:
+    """conversion/__init__.py maps an architecture name to the module that implements it.
+    The @ModelBase.register decorator only runs once that module is imported, and the importer
+    is driven by this map, so a class registered in deepseek.py stays invisible until the
+    architecture appears here.
+    """
+    text = path.read_text(encoding="utf-8")
+    backup = path.with_suffix(".py.v41orig")
+    key = '    "DeepseekV41ForCausalLM": "deepseek",\n'
+    anchor = '    "DeepseekV4ForCausalLM": "deepseek",\n'
+    if revert:
+        if backup.exists():
+            shutil.copy2(backup, path)
+            print("  reverted  conversion/__init__.py")
+            return 0
+        print("  no backup conversion/__init__.py")
+        return 4
+    if key in text:
+        print("  ok        conversion/__init__.py (already maps V4.1)")
+        return 0
+    if anchor not in text:
+        print("  NO MATCH  conversion/__init__.py has no DeepseekV4ForCausalLM entry to anchor on")
+        return 5
+    if check:
+        print("  would patch conversion/__init__.py")
+        return 0
+    if not backup.exists():
+        shutil.copy2(path, backup)
+    path.write_text(text.replace(anchor, anchor + key, 1), encoding="utf-8")
+    print("  patched   conversion/__init__.py")
+    return 0
+def patch_deepseek(path: pathlib.Path, revert: bool, check: bool) -> int:
+    text = path.read_text(encoding="utf-8")
+    backup = path.with_suffix(".py.v41orig")
+    if revert:
+        if backup.exists():
+            shutil.copy2(backup, path)
+            print("  reverted  conversion/deepseek.py")
+            return 0
+        print("  no backup conversion/deepseek.py")
+        return 4
+    if "DeepseekV41ForCausalLM" in text:
+        print("  ok        conversion/deepseek.py (already registers V4.1)")
+        return 0
+    if "class DeepseekV4Model" not in text:
+        print("  NO MATCH  conversion/deepseek.py has no DeepseekV4Model to subclass")
+        return 5
+    if check:
+        print("  would patch conversion/deepseek.py")
+        return 0
+    if not backup.exists():
+        shutil.copy2(path, backup)
+    path.write_text(text.rstrip("\n") + "\n" + V41_CLASS, encoding="utf-8")
+    print("  patched   conversion/deepseek.py")
+    return 0
+def main() -> int:
+    if len(sys.argv) < 2:
+        print(__doc__)
+        return 2
+    root = pathlib.Path(sys.argv[1])
+    revert = "--revert" in sys.argv
+    check = "--check" in sys.argv
+    constants = root / "gguf-py" / "gguf" / "constants.py"
+    deepseek = root / "conversion" / "deepseek.py"
+    init = root / "conversion" / "__init__.py"
+    for p in (constants, deepseek, init):
+        if not p.exists():
+            print(f"  MISSING   {p}")
+            return 3
+    rc = patch_constants(constants, revert, check)
+    rc = patch_deepseek(deepseek, revert, check) or rc
+    rc = patch_init(init, revert, check) or rc
+    return rc
+if __name__ == "__main__":
+    sys.exit(main())