Hugging Face's logo

timm
/
deepseek_vit_412m_enc.deepseek_v4_flash_vision_exp

Image Feature Extraction
timm
Safetensors
Transformers
Model card Files
xet
 Community
deepseek_vit_412m_enc.deepseek_v4_flash_vision_exp / README.md
rwightman's picture
rwightman
Add native timm DeepSeek vision weights, model card, and license attribution
2cf4313 verified
preview code
|
Raw History Blame Contribute Delete
4.95 kB
metadata 
tags:
  - image-feature-extraction
  - timm
  - transformers
pipeline_tag: image-feature-extraction
library_name: timm
base_model: deepseek-ai/DeepSeek-V4-Flash-Vision-Exp
license: mit

Model card for deepseek_vit_412m_enc.deepseek_v4_flash_vision_exp

NOTE: This checkpoint is a native timm remap of the original vision weights, with no additional training. It contains no language-model weights or trained image-classification head.

A DeepSeek ViT image feature model extracted from DeepSeek-V4-Flash-Vision-Exp. This is the native vision encoder, including the 3×3 spatial aligner and projection to the source LLM width.

Model Notes

  • The backbone uses 14×14 patches, SwiGLU MLPs, RMSNorm and axial 2D RoPE, with no learned absolute position embeddings. The original linear patch projection is reshaped into a Conv2d without changing its computation.
  • The native aligner groups 3×3 patch tokens in channel-major order and uses a two-layer GELU MLP to project to the source LLM width. Incomplete groups are zero-padded on the bottom/right. It is retained in _enc and _align variants and omitted from the plain classifier.
  • RGB inputs use mean=(0.5, 0.5, 0.5) and std=(0.5, 0.5, 0.5), matching the original. The default timm evaluation transform uses crop_mode="border", crop_pct=1.0 and bicubic resizing to preserve aspect ratio on a fixed, gray-padded canvas. The original processor selects variable canvas dimensions and uses gray 127 padding; timm uses gray 128.
  • Rectangular inputs are supported. Dimensions must be divisible by 14 by default. Pass dynamic_img_pad=True at model creation to zero-pad normalized inputs on the bottom/right to a patch-size multiple. This does not reproduce the original adaptive resize policy.
  • forward_features() returns final-RMSNorm NHWC backbone features. forward() returns projected NLC tokens for the _enc variant, or pooled image embeddings for the classifier variant until a classification head is added.
  • Intermediate backbone maps are available through forward_intermediates() and features_only=True; these do not include the aligner. Use norm=True to apply the encoder's final RMSNorm to intermediate maps.

Model Details

Model Usage

Image Features 

import torch
import timm
from PIL import Image

model = timm.create_model('hf-hub:timm/deepseek_vit_412m_enc.deepseek_v4_flash_vision_exp', pretrained=True).eval()
data_config = timm.data.resolve_model_data_config(model)
transform = timm.data.create_transform(**data_config, is_training=False)
image = Image.open('image.jpg').convert('RGB')
x = transform(image).unsqueeze(0)

with torch.inference_mode():
    output = model(x)  # (1, 100, 4096): projected spatial tokens
    features = model.forward_features(x)  # (1, 28, 28, 1024): final-RMSNorm backbone features (NHWC)

Intermediate Feature Maps 

with torch.inference_mode():
    maps = model.forward_intermediates(
        x, indices=3, norm=True, output_fmt='NCHW', intermediates_only=True,
    )
for feature_map in maps:
    print(feature_map.shape)  # (1, 1024, 28, 28)

Citation 

@misc{deepseekai2026deepseekv4flashvisionexp,
  title={DeepSeek-V4-Flash-Vision-Exp},
  author={DeepSeek-AI},
  year={2026},
  url={https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp},
}

@misc{rw2019timm,
  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
}