dylanneve1/Qwen3.5-0.8B_int4_sym_group128_dyn_stateful

Qwen3.5-0.8B — INT4 Symmetric Group128 (OpenVINO)

OpenVINO stateful model exported from Qwen/Qwen3.5-0.8B with INT4 symmetric group-128 weight compression.

Model Details

Property	Value
Base model	Qwen/Qwen3.5-0.8B
Architecture	Qwen3.5 (hybrid linear-attention + full-attention)
Weight format	INT4 (4-bit integer)
Quantization	Symmetric, group_size=128
Shapes	Dynamic
State management	Stateful (KV cache + conv/recurrent states)

Architecture

Qwen3.5 is a hybrid model with 24 layers: 18 linear-attention (GatedDeltaNet) + 6 full-attention layers in a 3:1 repeating pattern. Linear-attention layers use fixed-size conv and recurrent states (cache_params.past.{conv,ssm}.N).

Conversion

pip install git+https://github.com/rkazants/optimum-intel.git@support_qwen3_5
pip install --pre -U openvino openvino-tokenizers nncf --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
pip install transformers==5.2.0

optimum-cli export openvino \
  --model Qwen/Qwen3.5-0.8B \
  --weight-format int4 \
  --group-size 128 \
  --sym \
  Qwen3.5-0.8B_int4_sym_group128_dyn_stateful