Hugging Face's logo

sphaela
/
Qwen3.6-27B-AutoRound-GGUF

GGUF
multilingual
auto-round
intel
quantization
vlm
mtp
multi-token-prediction
conversational
Model card Files
xet
 Community
11
sphaela commited on
Commit
6b210f6
·
verified ·
1 Parent(s): e4264fa

Add files using upload-large-folder tool

Browse files
Files changed (19) hide show
  1. .gitattributes +17 -0
  2. Qwen3.6-27B-Q2_K_MIXED.gguf +3 -0
  3. Qwen3.6-27B-Q2_K_S.gguf +3 -0
  4. Qwen3.6-27B-Q3_K_L.gguf +3 -0
  5. Qwen3.6-27B-Q3_K_M.gguf +3 -0
  6. Qwen3.6-27B-Q3_K_S.gguf +3 -0
  7. Qwen3.6-27B-Q4_0.gguf +3 -0
  8. Qwen3.6-27B-Q4_1.gguf +3 -0
  9. Qwen3.6-27B-Q4_K_M.gguf +3 -0
  10. Qwen3.6-27B-Q4_K_S.gguf +3 -0
  11. Qwen3.6-27B-Q5_0.gguf +3 -0
  12. Qwen3.6-27B-Q5_1.gguf +3 -0
  13. Qwen3.6-27B-Q5_K_M.gguf +3 -0
  14. Qwen3.6-27B-Q5_K_S.gguf +3 -0
  15. Qwen3.6-27B-Q6_K.gguf +3 -0
  16. README.md +62 -3
  17. mmproj-model-bf16.gguf +3 -0
  18. mmproj-model-f16.gguf +3 -0
  19. mmproj-model-f32.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,20 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Qwen3.6-27B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
37
+ mmproj-model-f16.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Qwen3.6-27B-Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Qwen3.6-27B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Qwen3.6-27B-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Qwen3.6-27B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Qwen3.6-27B-Q2_K_MIXED.gguf filter=lfs diff=lfs merge=lfs -text
43
+ mmproj-model-bf16.gguf filter=lfs diff=lfs merge=lfs -text
44
+ mmproj-model-f32.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Qwen3.6-27B-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Qwen3.6-27B-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Qwen3.6-27B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Qwen3.6-27B-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Qwen3.6-27B-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Qwen3.6-27B-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Qwen3.6-27B-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
52
+ Qwen3.6-27B-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3.6-27B-Q2_K_MIXED.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c1b9c75cecef876f74758757da3bd10e812f8815364b170b24726f7bc52a078d
3
+ size 16420484256
Qwen3.6-27B-Q2_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c8e1e3707b5bc928c85b08e8af1f21d3470dd52553110fc5665df62cd0d1355
3
+ size 9471722656
Qwen3.6-27B-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:59f2e354d39b57186e3df6dfdba5b7c93ba315cdbe9551072b18a6a0015b6193
3
+ size 12073952416
Qwen3.6-27B-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb5fc8a646806d70aa6ce2a093b840b9158f4436f7028942ef598770a01ae61b
3
+ size 12073952416
Qwen3.6-27B-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:00d81729d7b4d2d653f5afb56a2f982916496307b70083f8fa3b269d27c70c1f
3
+ size 12073952416
Qwen3.6-27B-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e92cbfa78a0b12f788c621b06cd8ca16743d50098c839204cffc93bac2e8287c
3
+ size 15476868256
Qwen3.6-27B-Q4_1.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:57382d07a5ab7a08dfd3a581e20e2f596d6ca513ff1912febc27c377a356facd
3
+ size 17078240416
Qwen3.6-27B-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:216b2b9f4e9bb2cf054dca420086cdf6efcf0abc2c98c91da48cefff2d516ae4
3
+ size 15476868256
Qwen3.6-27B-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7150f3f1c16e37b8f49471d4e473a1b029a2c65abc90afe3e04669fbd5ef4beb
3
+ size 15476868256
Qwen3.6-27B-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6e183d23057c0104bd0be2f60bc20b5c48b13dd138fbc90d28216130739fb2d6
3
+ size 18679612576
Qwen3.6-27B-Q5_1.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2fb3876f04ebc89918941789fda9fbba87fb04ab4ebd4b681ffa69bf480b066b
3
+ size 20280984736
Qwen3.6-27B-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e8bd81cfd9710f44ff6b49ac16560629c689eb5a66e1ccbb964bd869611bae4
3
+ size 18679612576
Qwen3.6-27B-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7ffca07ae2f659c092a6fc2d005a49b6085014bfb14210b822744cf84c378cea
3
+ size 18679612576
Qwen3.6-27B-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2cd1d73a93036b0bb3d291d1b38ce5fc308043bbc3216dffde94af94cc8bb92e
3
+ size 22082528416
README.md CHANGED
@@ -1,3 +1,62 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ base_model: Qwen/Qwen3.6-27B
6
+ tags:
7
+ - auto-round
8
+ - intel
9
+ - gguf
10
+ - quantization
11
+ - vlm
12
+ ---
13
+
14
+ # Qwen3.6-27B GGUF (AutoRound Quantized)
15
+
16
+ This repository contains GGUF quantized versions of [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) created using Intel's [AutoRound](https://github.com/intel/auto-round) quantization method.
17
+
18
+ ## Quantization Details
19
+
20
+ The models were quantized using various schemes provided by the `auto-round` tool. For better compatibility and smaller size, we provide unified multimodal projector (`mmproj`) files in F16, BF16, and F32 formats.
21
+
22
+ ### Files and Sizes
23
+
24
+ | File Name | Quant Type | Size | Description |
25
+ |-----------|------------|------|-------------|
26
+ | `Qwen3.6-27B-Q2_K_S.gguf` | Q2_K_S | 8.9 GB | Extremely high compression, significant quality loss. |
27
+ | `Qwen3.6-27B-Q2_K_MIXED.gguf` | Q2_K_MIXED | 16 GB | **Recommended** high-compression option. Uses Q4 for KV cache with good quality. |
28
+ | `Qwen3.6-27B-Q3_K_S.gguf` | Q3_K_S | 12 GB | Very high compression, notable quality loss. |
29
+ | `Qwen3.6-27B-Q3_K_M.gguf` | Q3_K_M | 12 GB | Balanced 3-bit quantization. |
30
+ | `Qwen3.6-27B-Q3_K_L.gguf` | Q3_K_L | 12 GB | High quality 3-bit quantization. |
31
+ | `Qwen3.6-27B-Q4_0.gguf` | Q4_0 | 15 GB | Standard 4-bit quantization, good balance. |
32
+ | `Qwen3.6-27B-Q4_1.gguf` | Q4_1 | 16 GB | Higher quality 4-bit quantization than Q4_0. |
33
+ | `Qwen3.6-27B-Q4_K_S.gguf` | Q4_K_S | 15 GB | Small 4-bit K-quant, good efficiency. |
34
+ | `Qwen3.6-27B-Q4_K_M.gguf` | Q4_K_M | 15 GB | **Recommended** 4-bit K-quant, excellent balance. |
35
+ | `Qwen3.6-27B-Q5_0.gguf` | Q5_0 | 18 GB | Standard 5-bit quantization, very high quality. |
36
+ | `Qwen3.6-27B-Q5_1.gguf` | Q5_1 | 19 GB | Higher quality 5-bit quantization than Q5_0. |
37
+ | `Qwen3.6-27B-Q5_K_S.gguf` | Q5_K_S | 18 GB | Small 5-bit K-quant, very high quality. |
38
+ | `Qwen3.6-27B-Q5_K_M.gguf` | Q5_K_M | 18 GB | **Recommended** 5-bit K-quant, near-lossless. |
39
+ | `Qwen3.6-27B-Q6_K.gguf` | Q6_K | 21 GB | 6-bit K-quant, virtually indistinguishable from F16. |
40
+ | `mmproj-model-f16.gguf` | F16 | 885 MB | Unified Projector in Float16 format. |
41
+ | `mmproj-model-bf16.gguf` | BF16 | 889 MB | Unified Projector in BFloat16 format. |
42
+ | `mmproj-model-f32.gguf` | F32 | 1.8 GB | Unified Projector in Float32 format. |
43
+
44
+ ## Generate the Model
45
+
46
+ The models were generated using Intel's AutoRound with the following command:
47
+
48
+ ```bash
49
+ auto-round --model Qwen/Qwen3.6-27B --output_dir ./quantized/ --scheme <SCHEME> --iters 0
50
+ ```
51
+
52
+ ## Usage with llama.cpp
53
+
54
+ These models can be used with `llama.cpp`. For multimodal usage, you must specify the projector file:
55
+
56
+ ```bash
57
+ ./llama-cli -m Qwen3.6-27B-Q4_K_M.gguf --mmproj mmproj-model-f16.gguf --image your_image.jpg -p "Describe this image."
58
+ ```
59
+
60
+ ## About AutoRound
61
+
62
+ [AutoRound](https://github.com/intel/auto-round) is an advanced quantization technique from Intel that aims to minimize accuracy loss through automated rounding optimization.
mmproj-model-bf16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:895cc208eb6630c78a21c152eb9b2ff4afe54c04595747b9f9a8bf1893013b0c
3
+ size 931145888
mmproj-model-f16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77c1f430e6d745b5104f841c6c3006619290237e66c5ffe2ce6c89070ba89aa7
3
+ size 927606944
mmproj-model-f32.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd431c45f7fbfb1d1b813045ef331995bab12afc9d63d716a8cfc51f92155da8
3
+ size 1842940064