mumitrol/DeepSeek-V4-Flash-0731-Abliterated-NVFP4-vision

DeepSeek-V4-Flash-0731 Abliterated + Vision (NVFP4)

Abliterated DeepSeek-V4-Flash-0731 (MOE, 284B total / 13B activated) with a frozen MoonViT-3d vision tower (from Kimi-K2.6) and WebBrain's trained PatchMerger projector — packaged for vLLM.

What this repo is

This is a fork of sakamakismile/DeepSeek-V4-Flash-0731-Abliterated-NVFP4 (the abliterated NVFP4 conversion) with the vision overlay from webbrain-one/DeepSeek-V4-Flash-0731-Vision-NVFP4 spliced on:

Component	Provenance	Status
Text checkpoint	sakamakismile ablit 0731 NVFP4	48 shards
Vision tower	vision_tower.safetensors MoonViT	frozen
Projector	mm_projector.safetensors PatchMerger	frozen
Routing	palette64.json 64-id palette	see below

How to serve with vLLM

The config carries auto_map so vLLM's Transformers modeling backend loads it with remote code — no vLLM fork or custom image tag required:

docker run --rm --gpus all \
  -p 8000:8000 \
  vllm/vllm-openai:cu129-nightly \
  python -m vllm.entrypoints.openai.api_server \
    --model mumitrol/DeepSeek-V4-Flash-0731-Abliterated-NVFP4-vision \
    --model-impl transformers \
    --trust-remote-code \
    --tensor-parallel-size 4 \
    --dtype bfloat16 \
    --limit-mm-per-prompt image=1

• Requires a Blackwell node (NVFP4); 1x B300 (288 GB) is enough for a single stream / moderate context, 2x for maximum context and throughput.

Status / validation (IMPORTANT)

This is a first-pass community port, not an official NVIDIA or DeepSeek release. Validate on GPU before production use:

• Tower attention matches MMEncoderAttention (head_dim^-0.5 scale, fused wqkv, no RoPE) — F.scaled_dot_product_attention with exact scale.
• Positional-embedding interpolation mode matches MoonViT (bicubic).
• Image-to-token seating is right (>= 512-token cap, 2x2 merge layout).
• Hash-MoE routing for image tokens uses palette64.json (deterministic).
• Abliteration + vision edge effects on refusing behavior are checked.

See modeling_deepseek_v4_vision.py for # -> implementation notes.

Licenses

Original DeepSeek weights MIT; Kimi-K2.6-derived tower under its own notice; WebBrain projector per their repo; abliteration per apetersson/drowzeys notice. Downstream users must satisfy all applicable licenses.