knoopx/ThinkingCap-Qwen3.6-27B-NInfer

ThinkingCap-Qwen3.6-27B

Three single-file NInfer (*.ninfer) artifacts for ThinkingCap-Qwen3.6-27B, a token-efficient, thinking-oriented, multimodal (image+video→text) 27B finetune of Qwen/Qwen3.6-27B. Each artifact is a self-contained binary the NInfer engine serves directly. Artifacts 1 and 2 are built from the ThinkingCap-Qwen3.6-27B finetune family. Artifact 3 has the same weight layout but is produced from the Qwen3.8-27B source pair by a newer converter generation. None is a rename or re-label of another.

Model architecture

• Qwen3_5ForConditionalGeneration (model_type: qwen3_5), dense multimodal (image+video→text) finetune of Qwen/Qwen3.6-27B. Fully dense transformer: every decoder layer uses a dense SwiGLU MLP with no routing.
• Text: 64 decoder layers; hybrid attention: 16 full-attention layers interleaved with 48 recurrent layers (short convolution, width 4), full-attention interval 4; hidden 5120, intermediate 17408, output matrix 248320 rows (tokenizer domain 248077); 24 query / 4 KV heads (head dim 256); 1 MTP draft layer; 262144 max position embeddings.
• Vision: 27-layer ViT (hidden 1152, intermediate 4304, 16 heads), 16×16 spatial patch, 2-frame temporal patch, 2×2 spatial merge.
• Positional: RoPE θ=10,000,000 with mRoPE section [11, 11, 10].
• Upstream base: bottlecapai/ThinkingCap-Qwen3.6-27B, itself a finetune of Qwen/Qwen3.6-27B.

Artifacts

1. Groupwise-int4 (W4A16) — thinkingcap_qwen3_6_27b.ninfer

On-the-fly groupwise quantization from a single BF16 source; each linear layer is quantized at conversion time into a per-group 4/5/6-bit + W8 format selected by tensor size/role (Q4G64_F16S, Q5G64_F16S, Q6G64_F16S, W8G32_F16S). Weights are groupwise-quantized while activations stay 16-bit, making this a W4A16 build.

Single-source provenance: base BF16 checkpoint bottlecapai/ThinkingCap-Qwen3.6-27B (rev 6a9e13bd). Converter module tools.convert.qwen3_6_27b.convert (recipe qwen3_6_27b-v2, weights_id groupwise-int):

python -m tools.convert.qwen3_6_27b.convert \
  --model <base-bf16-checkpoint-dir> --out out/<name>.ninfer --device cuda

Stats: objects 1124 (1118 tensors + 6 resources); tensor_bytes 17482342304; resource_bytes 12833441; alignment_bytes 14015; md5 8c08f2f50ca93cab2e13508365d65147.

Histogram (sums to 1118): BF16 582, Q5G64_F16S 246, Q4G64_F16S 183, FP32 96, W8G32_F16S 7, Q6G64_F16S 3, I32 1. Layouts: contiguous-le-v1 ×679, row-split-k128-v1 ×439; 6 resources encoded raw-bytes-v1.

2. NVFP4 (W4A4) — thinkingcap_qwen3_6_27b_nvfp4.ninfer

NVFP4 W4A4 mixed-precision: packed 4-bit FP4 (E2M1) weights with per-tensor group scales and input_activations: tensor_group, so activations are also 4-bit at inference. This is the true W4A4 path; the W4A16 build keeps 16-bit activations.

Dual-source: base BF16 bottlecapai/ThinkingCap-Qwen3.6-27B (rev 6a9e13bd) fused with vLLM-quantized NVFP4 partner sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4 (rev 9b5389d4). The NVFP4-packed lines take 4-bit weight bytes + scales from the partner; the remaining 16-bit/resource tensors come from the base BF16 tree. Converter tools.convert.qwen3_6_27b.convert_nvfp4 (recipe qwen3_6_27b_nvfp4-v1, weights_id nvfp4, model_id qwen3.6-27b):

python3 -m tools.convert.qwen3_6_27b.convert_nvfp4 \
  --model <base-bf16-checkpoint-dir> \
  --nvfp4-model <nvfp4-checkpoint-dir> \
  --out out/<name>.ninfer --device cpu

Stats: objects 1307 (1301 tensors + 6 resources); tensor_bytes 18310883160; resource_bytes 12833441; alignment_bytes 138503; md5 24a3d90b8aae9ea4f5296061439bd625.

Histogram (sums to 1301): BF16 591, FP32 343, NVFP4 247, Q4G64_F16S 55, Q5G64_F16S 54, W8G32_F16S 9, Q6G64_F16S 1, I32 1. Layouts: blockscale-k16-m128x4-v1 ×247, contiguous-le-v1 ×935, row-split-k128-v1 ×119; 6 resources raw-bytes-v1.

3. NVFP4 (W4A4) w8g32-generation — thinkingcap_qwen3_6_27b_nvfp4_w8g32.ninfer

Built by the W8G32-generation converter tools.convert.qwen3_8_27b.w8g32_nvfp4.convert_nvfp4 (recipe qwen3_8_27b_nvfp4_w8g32-v1, converter output basename qwen3_8_27b_nvfp4_w8g32.ninfer). This is a separate module from artifact 2's converter, not a re-quantization of it: it reuses the qwen3_6_27b recipe structure (additive NVFP4 MLP over 247 matrices, blockscale-k16-m128x4-v1 packing, split GDN control, W8G32_F16S vocabulary endpoints) but pins the Qwen3.8 base model's frontend resources (its W8G32 resource profile, verified by SHA256 at conversion) instead of the Qwen3.6 resources artifact 2 embeds.

Dual-source, with a different source pair than artifact 2, per the recorded conversion run: the base BF16 checkpoint Qwen/Qwen3.8-27B (rev 1d4bf0f2) supplies the direct tensors, split GDN control projections, and W8G32_F16S vocabulary endpoints; the NVFP4 partner sakamakismile/Qwen3.8-27B-MTP-NVFP4 supplies the packed 4-bit codes plus E4M3FN scales and divisors. The embedded identity (model_id qwen3.8-27b, weights_id nvfp4_w8g32) matches this Qwen3.8 source pair rather than the ThinkingCap-Qwen3.6-27B family of artifacts 1 and 2. With input_activations: tensor_group this is a true W4A4 build.

It has the same object inventory as artifact 2 (1307 objects: 1301 tensors + 6 resources, the same format counts, the same layouts), but the files are not byte-identical: artifact 3 is 3840 bytes larger (18324067840 vs 18324064000) with a different SHA256, because its embedded frontend-resource bytes come from the Qwen3.8 base model. The histogram also includes one Q6G64_F16S object even though the filename omits q6g64; the missing token distinguishes this build from artifact 2 under the one-filename-per-(org, model, weight-types) convention. It does not mean the build lacks 6-bit weights.

python -m tools.convert.qwen3_8_27b.w8g32_nvfp4.convert_nvfp4 \
  --model <base-bf16-checkpoint-dir> \
  --nvfp4-model <nvfp4-checkpoint-dir> \
  --out out/<name>.ninfer --device cuda

Stats: objects 1307 (1301 tensors + 6 resources); file bytes 18324067840; sha256 00c3412b…cbd1; layouts blockscale-k16-m128x4-v1 ×247, contiguous-le-v1 ×935, row-split-k128-v1 ×119.

Differences between artifacts

	1 — groupwise	2 — NVFP4	3 — NVFP4 w8g32
Quant class	W4A16 (groupwise-int weights, 16-bit activations)	W4A4 (NVFP4 4-bit weights, 4-bit activations)	W4A4 (NVFP4 4-bit weights, 4-bit activations)
Sources	single (BF16 only)	two-source (BF16 + NVFP4 partner)	two-source (BF16 + NVFP4 partner)
Source repos	bottlecapai/ThinkingCap-Qwen3.6-27B	bottlecapai/ThinkingCap-Qwen3.6-27B + sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4	Qwen/Qwen3.8-27B + sakamakismile/Qwen3.8-27B-MTP-NVFP4
Converter	qwen3_6_27b.convert (groupwise recipe)	qwen3_6_27b.convert_nvfp4 (qwen3_6_27b_nvfp4-v1)	qwen3_8_27b.w8g32_nvfp4.convert_nvfp4 (qwen3_8_27b_nvfp4_w8g32-v1)
Tensors	1118	1301	1301 (same format counts as 2)
Frontend resources	Qwen3.6 base	Qwen3.6 base	Qwen3.8 base (W8G32 profile)

Artifacts 2 and 3 share an identical 1301-tensor weight layout and the same weight types (both include one Q6G64), but are not byte-identical files: the w8g32 build is 3840 bytes larger, with a different SHA256, because its embedded source and frontend-resource bytes differ even though the format counts match. They are two distinct builds (different converter generation, different source pair, different payload bytes), which is why the naming convention (one filename per org + model + weight-types) gives them distinct names. Artifact 1 is the only non-NVFP4, W4A16 build.

Files and checksums

Byte sizes below are the authoritative LFS sizes from the Hugging Face repo tree.

File	Size (bytes)	SHA256	Scheme
thinkingcap_qwen3_6_27b.ninfer	17495365888	406d7ca3c8248ec304518563b9dbb00cf48d9cfc19d126e6df66d354e6d5b892	Groupwise-int4 W4A16
thinkingcap_qwen3_6_27b_nvfp4.ninfer	18324064000	ec149b15d29acc0bfdb6f5720e12a39f15746b688ac4e290a6ebf6661ac2c8e7	NVFP4 W4A4
thinkingcap_qwen3_6_27b_nvfp4_w8g32.ninfer	18324067840	00c3412b764c24bdf37058dea5209b677462fa68c517c8e21c89aec9f38dcbd1	NVFP4 W4A4 (w8g32)

Serving

All artifacts are served directly by the NInfer engine as self-contained single binaries, so no separate weights download is needed. Capability flags relevant to this model: multimodal image input (--vision), MTP decoding (1 MTP draft layer), and int8 KV cache.

Known limitations

No KL-divergence or perplexity benchmark has been run on any artifact, so quantization fidelity against the BF16 reference is unmeasured. Validation is structural only: format histogram, object/tensor/resource counts, and byte/alignment integrity from the converter's accounting. No downstream quality claims follow from it.

References

• Base checkpoint (artifacts 1–2): bottlecapai/ThinkingCap-Qwen3.6-27B (finetune of Qwen/Qwen3.6-27B), rev 6a9e13bd
• NVFP4 partner (artifacts 1–2): sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4, rev 9b5389d4
• Base checkpoint (artifact 3): Qwen/Qwen3.8-27B, rev 1d4bf0f2
• NVFP4 partner (artifact 3): sakamakismile/Qwen3.8-27B-MTP-NVFP4
• NInfer engine: https://github.com/ninfer
• Converter toolchain: tools.convert.qwen3_6_27b.{convert,convert_nvfp4} and tools.convert.qwen3_8_27b.w8g32_nvfp4.convert_nvfp4