Hugging Face's logo

Atomic-Germ
/
MiniCPM5-2B-OpenNPU2

Text Generation
Transformers
GGUF
Safetensors
English
Chinese
llama
minicpm
minicpm5
long-context
tool-calling
on-device
edge-ai
conversational
openbmb/Ultra-FineWeb
openbmb/UltraX-Preview
openbmb/Ultra-FineWeb-L3
openbmb/UltraData-Math
openbmb/UltraData-Code
openbmb/UltraData-SFT-2605
openbmb/UltraData-SFT-Agent-2609
openbmb/UltraData-RL-2609
arxiv:2506.07900
arxiv:2602.09003
text-generation-inference
imatrix
Model card Files
xet
 Community
Atomic-Germ mradermacher commited on
Commit
d59f353
·
0 Parent(s):

Duplicate from mradermacher/MiniCPM5-2B-i1-GGUF

Browse files

Co-authored-by: team mradermacher <mradermacher@users.noreply.huggingface.co>

Files changed (27) hide show
  1. .gitattributes +60 -0
  2. MiniCPM5-2B.i1-IQ1_M.gguf +3 -0
  3. MiniCPM5-2B.i1-IQ1_S.gguf +3 -0
  4. MiniCPM5-2B.i1-IQ2_M.gguf +3 -0
  5. MiniCPM5-2B.i1-IQ2_S.gguf +3 -0
  6. MiniCPM5-2B.i1-IQ2_XS.gguf +3 -0
  7. MiniCPM5-2B.i1-IQ2_XXS.gguf +3 -0
  8. MiniCPM5-2B.i1-IQ3_M.gguf +3 -0
  9. MiniCPM5-2B.i1-IQ3_S.gguf +3 -0
  10. MiniCPM5-2B.i1-IQ3_XS.gguf +3 -0
  11. MiniCPM5-2B.i1-IQ3_XXS.gguf +3 -0
  12. MiniCPM5-2B.i1-IQ4_NL.gguf +3 -0
  13. MiniCPM5-2B.i1-IQ4_XS.gguf +3 -0
  14. MiniCPM5-2B.i1-Q2_K.gguf +3 -0
  15. MiniCPM5-2B.i1-Q2_K_S.gguf +3 -0
  16. MiniCPM5-2B.i1-Q3_K_L.gguf +3 -0
  17. MiniCPM5-2B.i1-Q3_K_M.gguf +3 -0
  18. MiniCPM5-2B.i1-Q3_K_S.gguf +3 -0
  19. MiniCPM5-2B.i1-Q4_0.gguf +3 -0
  20. MiniCPM5-2B.i1-Q4_1.gguf +3 -0
  21. MiniCPM5-2B.i1-Q4_K_M.gguf +3 -0
  22. MiniCPM5-2B.i1-Q4_K_S.gguf +3 -0
  23. MiniCPM5-2B.i1-Q5_K_M.gguf +3 -0
  24. MiniCPM5-2B.i1-Q5_K_S.gguf +3 -0
  25. MiniCPM5-2B.i1-Q6_K.gguf +3 -0
  26. MiniCPM5-2B.imatrix.gguf +3 -0
  27. README.md +104 -0
.gitattributes ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ MiniCPM5-2B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
37
+ MiniCPM5-2B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
38
+ MiniCPM5-2B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
39
+ MiniCPM5-2B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
40
+ MiniCPM5-2B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
41
+ MiniCPM5-2B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
42
+ MiniCPM5-2B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
43
+ MiniCPM5-2B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
44
+ MiniCPM5-2B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
45
+ MiniCPM5-2B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
46
+ MiniCPM5-2B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
47
+ MiniCPM5-2B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
48
+ MiniCPM5-2B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
49
+ MiniCPM5-2B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
50
+ MiniCPM5-2B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
51
+ MiniCPM5-2B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
52
+ MiniCPM5-2B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
53
+ MiniCPM5-2B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
54
+ MiniCPM5-2B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
55
+ MiniCPM5-2B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
56
+ MiniCPM5-2B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
57
+ MiniCPM5-2B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
58
+ MiniCPM5-2B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
59
+ MiniCPM5-2B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
60
+ MiniCPM5-2B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
MiniCPM5-2B.i1-IQ1_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:685f8dd55b53d5dc54ae3c2066736c664a8440b964c3572e3bf8186d5a98284c
3
+ size 732262016
MiniCPM5-2B.i1-IQ1_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:caf101a823222e5a5f83128b77228566d20afb03af679ba4415368c18c6a7681
3
+ size 691932800
MiniCPM5-2B.i1-IQ2_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0b2545abc818b2bfce747318903a5ccaa06c8db1f45c3279a7103b30cb164b3c
3
+ size 970845824
MiniCPM5-2B.i1-IQ2_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0332c1dfd60f3620e46c57cc719bad9ca6be0782c38d9f58be63ccbc5a8503a
3
+ size 917073536
MiniCPM5-2B.i1-IQ2_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f916be109a60ec0158ac8213da032517693d2a1be9da3eef1c106c71bb30c70
3
+ size 858754688
MiniCPM5-2B.i1-IQ2_XXS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3beb5b8dd081e63aba18e029d4acc58fd1950faec248403b5206d78fe7379506
3
+ size 799477376
MiniCPM5-2B.i1-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5ac6799a245374b2053c86e0c7ab4f42a4f3c9bce9f7753e519f3396c6fd6905
3
+ size 1226313344
MiniCPM5-2B.i1-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9d1498fe944e8653695ade9529e0143a321402129d52c818985f4e7c39b12c93
3
+ size 1194561152
MiniCPM5-2B.i1-IQ3_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:72520f35d22269ea15c8f1f6b1a859e622e354cfd71416bc1d7f1b467ce3cea0
3
+ size 1148702336
MiniCPM5-2B.i1-IQ3_XXS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:decd09d622dd10f235be193fd537946807449055b1441391a494cd40c171a708
3
+ size 1063054976
MiniCPM5-2B.i1-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8bf17b004782da86a28efbc2f77c45c3ff7352b5808b8f99938ebc01a72f24f1
3
+ size 1493110400
MiniCPM5-2B.i1-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7fabf541c18995a7b156e2fe24a9e2de5b5486631602d978bfd033a3b9afdc8c
3
+ size 1423511168
MiniCPM5-2B.i1-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f227245aa9abc447fd5354c61e52e17b54c4a94f3cea88bf32faf5a54f4643e
3
+ size 1039928960
MiniCPM5-2B.i1-Q2_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17f843c381085a2923892452b846aa9d55b8e1eff3d545f28c44a0c0e9689112
3
+ size 983109248
MiniCPM5-2B.i1-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c24be5a57e4dac959ad7aeda5be6590e191ed357da14d4024b651b1812e9dad0
3
+ size 1378979456
MiniCPM5-2B.i1-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f74eb4844493cb4e5398172e12d4c3ef9cbc9ed2eef2e8ce111de4b67275533b
3
+ size 1291423360
MiniCPM5-2B.i1-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a590ee195a40a730449fefd8baf822b7f235c40d844d51c95825c663e4681ed2
3
+ size 1191636608
MiniCPM5-2B.i1-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d6701fd8a289f1e4901198400788d4f563b1cce7dfa5ab470354d848c0d14cb
3
+ size 1494290048
MiniCPM5-2B.i1-Q4_1.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d8e263ca9dcdf51d75925b3a46cf4600f654069417670ab93018ee48cca4a3b4
3
+ size 1630932608
MiniCPM5-2B.i1-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e0bc51c610bb9503eb603836af2d5353ff133d605d972b167b1147e5193b6f4
3
+ size 1561321088
MiniCPM5-2B.i1-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:159f5c4bdfd43ab8ebd2a66b98728752adb84f091e8c475f1f0217f58af93eb0
3
+ size 1498484352
MiniCPM5-2B.i1-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc9ad59e8752f0756097356fcad06b07fd395baf034f713a4e0ec0c4b22a3f4a
3
+ size 1808064128
MiniCPM5-2B.i1-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:69e64100b90fc272ec8cb3cd63b6cdc8f76b8b117f92bd75950ee3e96d5c8d41
3
+ size 1771507328
MiniCPM5-2B.i1-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5b1754f4bac65aa7d88bcc89483218bee276b1f1e1cf26cd3d93204d725b014
3
+ size 2070228608
MiniCPM5-2B.imatrix.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:67cafcc0fe4d39583800f9cef96c5f4471da4e08b9b3f64cd4216dba0c7348a4
3
+ size 3141792
README.md ADDED
@@ -0,0 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: openbmb/MiniCPM5-2B
3
+ datasets:
4
+ - openbmb/Ultra-FineWeb
5
+ - openbmb/UltraX-Preview
6
+ - openbmb/Ultra-FineWeb-L3
7
+ - openbmb/UltraData-Math
8
+ - openbmb/UltraData-Code
9
+ - openbmb/UltraData-SFT-2605
10
+ - openbmb/UltraData-SFT-Agent-2609
11
+ - openbmb/UltraData-RL-2609
12
+ language:
13
+ - en
14
+ - zh
15
+ library_name: transformers
16
+ license: apache-2.0
17
+ mradermacher:
18
+ readme_rev: 1
19
+ quantized_by: mradermacher
20
+ tags:
21
+ - minicpm
22
+ - minicpm5
23
+ - llama
24
+ - text-generation
25
+ - long-context
26
+ - tool-calling
27
+ - on-device
28
+ - edge-ai
29
+ ---
30
+ ## About
31
+
32
+ <!-- ### quantize_version: 2 -->
33
+ <!-- ### output_tensor_quantised: 1 -->
34
+ <!-- ### convert_type: hf -->
35
+ <!-- ### vocab_type: -->
36
+ <!-- ### tags: nicoboss -->
37
+ <!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S -->
38
+ <!-- ### quants_skip: -->
39
+ <!-- ### skip_mmproj: -->
40
+ weighted/imatrix quants of https://huggingface.co/openbmb/MiniCPM5-2B
41
+
42
+ <!-- provided-files -->
43
+
44
+ ***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#MiniCPM5-2B-i1-GGUF).***
45
+
46
+ static quants are available at https://huggingface.co/mradermacher/MiniCPM5-2B-GGUF
47
+ ## Usage
48
+
49
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
50
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
51
+ more details, including on how to concatenate multi-part files.
52
+
53
+ ## Provided Quants
54
+
55
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
56
+
57
+ | Link | Type | Size/GB | Notes |
58
+ |:-----|:-----|--------:|:------|
59
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) |
60
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-IQ1_S.gguf) | i1-IQ1_S | 0.8 | for the desperate |
61
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-IQ1_M.gguf) | i1-IQ1_M | 0.8 | mostly desperate |
62
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 0.9 | |
63
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.0 | |
64
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-IQ2_S.gguf) | i1-IQ2_S | 1.0 | |
65
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-IQ2_M.gguf) | i1-IQ2_M | 1.1 | |
66
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.1 | very low quality |
67
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-Q2_K.gguf) | i1-Q2_K | 1.1 | IQ3_XXS probably better |
68
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.2 | lower quality |
69
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.2 | |
70
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 1.3 | IQ3_XS probably better |
71
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-IQ3_S.gguf) | i1-IQ3_S | 1.3 | beats Q3_K* |
72
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-IQ3_M.gguf) | i1-IQ3_M | 1.3 | |
73
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 1.4 | IQ3_S probably better |
74
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 1.5 | IQ3_M probably better |
75
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 1.5 | |
76
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 1.6 | prefer IQ4_XS |
77
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-Q4_0.gguf) | i1-Q4_0 | 1.6 | fast, low quality |
78
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 1.6 | optimal size/speed/quality |
79
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 1.7 | fast, recommended |
80
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-Q4_1.gguf) | i1-Q4_1 | 1.7 | |
81
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 1.9 | |
82
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 1.9 | |
83
+ | [GGUF](https://huggingface.co/mradermacher/MiniCPM5-2B-i1-GGUF/resolve/main/MiniCPM5-2B.i1-Q6_K.gguf) | i1-Q6_K | 2.2 | practically like static Q6_K |
84
+
85
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
86
+ types (lower is better):
87
+
88
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
89
+
90
+ And here are Artefact2's thoughts on the matter:
91
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
92
+
93
+ ## FAQ / Model Request
94
+
95
+ See https://huggingface.co/mradermacher/model_requests for some answers to
96
+ questions you might have and/or if you want some other model quantized.
97
+
98
+ ## Thanks
99
+
100
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
101
+ me use its servers and providing upgrades to my workstation to enable
102
+ this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
103
+
104
+ <!-- end -->