Hugging Face's logo

tensorblock
/
llama-3.2-Korean-Bllossom-3B-GGUF

Transformers
GGUF
English
Korean
TensorBlock
GGUF
conversational
Model card Files
xet
 Community
morriszms commited on
Commit
137b81e
ยท
verified ยท
1 Parent(s): 4911289

Upload folder using huggingface_hub

Browse files
Files changed (14) hide show
  1. .gitattributes +12 -0
  2. README.md +81 -0
  3. llama-3.2-Korean-Bllossom-3B-Q2_K.gguf +3 -0
  4. llama-3.2-Korean-Bllossom-3B-Q3_K_L.gguf +3 -0
  5. llama-3.2-Korean-Bllossom-3B-Q3_K_M.gguf +3 -0
  6. llama-3.2-Korean-Bllossom-3B-Q3_K_S.gguf +3 -0
  7. llama-3.2-Korean-Bllossom-3B-Q4_0.gguf +3 -0
  8. llama-3.2-Korean-Bllossom-3B-Q4_K_M.gguf +3 -0
  9. llama-3.2-Korean-Bllossom-3B-Q4_K_S.gguf +3 -0
  10. llama-3.2-Korean-Bllossom-3B-Q5_0.gguf +3 -0
  11. llama-3.2-Korean-Bllossom-3B-Q5_K_M.gguf +3 -0
  12. llama-3.2-Korean-Bllossom-3B-Q5_K_S.gguf +3 -0
  13. llama-3.2-Korean-Bllossom-3B-Q6_K.gguf +3 -0
  14. llama-3.2-Korean-Bllossom-3B-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ llama-3.2-Korean-Bllossom-3B-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ llama-3.2-Korean-Bllossom-3B-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ llama-3.2-Korean-Bllossom-3B-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ llama-3.2-Korean-Bllossom-3B-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ llama-3.2-Korean-Bllossom-3B-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ llama-3.2-Korean-Bllossom-3B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ llama-3.2-Korean-Bllossom-3B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ llama-3.2-Korean-Bllossom-3B-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ llama-3.2-Korean-Bllossom-3B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ llama-3.2-Korean-Bllossom-3B-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
46
+ llama-3.2-Korean-Bllossom-3B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ llama-3.2-Korean-Bllossom-3B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,81 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Bllossom/llama-3.2-Korean-Bllossom-3B
3
+ language:
4
+ - en
5
+ - ko
6
+ library_name: transformers
7
+ license: llama3.2
8
+ tags:
9
+ - TensorBlock
10
+ - GGUF
11
+ ---
12
+
13
+ <div style="width: auto; margin-left: auto; margin-right: auto">
14
+ <img src="https://i.imgur.com/jC7kdl8.jpeg" alt="TensorBlock" style="width: 100%; min-width: 400px; display: block; margin: auto;">
15
+ </div>
16
+ <div style="display: flex; justify-content: space-between; width: 100%;">
17
+ <div style="display: flex; flex-direction: column; align-items: flex-start;">
18
+ <p style="margin-top: 0.5em; margin-bottom: 0em;">
19
+ Feedback and support: TensorBlock's <a href="https://x.com/tensorblock_aoi">Twitter/X</a>, <a href="https://t.me/TensorBlock">Telegram Group</a> and <a href="https://x.com/tensorblock_aoi">Discord server</a>
20
+ </p>
21
+ </div>
22
+ </div>
23
+
24
+ ## Bllossom/llama-3.2-Korean-Bllossom-3B - GGUF
25
+
26
+ This repo contains GGUF format model files for [Bllossom/llama-3.2-Korean-Bllossom-3B](https://huggingface.co/Bllossom/llama-3.2-Korean-Bllossom-3B).
27
+
28
+ The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
29
+
30
+ ## Prompt template
31
+
32
+ ```
33
+ <|begin_of_text|><|start_header_id|>system<|end_header_id|>
34
+
35
+ Cutting Knowledge Date: December 2023
36
+ Today Date: 09 Nov 2024
37
+
38
+ {system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
39
+
40
+ {prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
41
+ ```
42
+
43
+ ## Model file specification
44
+
45
+ | Filename | Quant type | File Size | Description |
46
+ | -------- | ---------- | --------- | ----------- |
47
+ | [llama-3.2-Korean-Bllossom-3B-Q2_K.gguf](https://huggingface.co/tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF/tree/main/llama-3.2-Korean-Bllossom-3B-Q2_K.gguf) | Q2_K | 1.270 GB | smallest, significant quality loss - not recommended for most purposes |
48
+ | [llama-3.2-Korean-Bllossom-3B-Q3_K_S.gguf](https://huggingface.co/tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF/tree/main/llama-3.2-Korean-Bllossom-3B-Q3_K_S.gguf) | Q3_K_S | 1.437 GB | very small, high quality loss |
49
+ | [llama-3.2-Korean-Bllossom-3B-Q3_K_M.gguf](https://huggingface.co/tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF/tree/main/llama-3.2-Korean-Bllossom-3B-Q3_K_M.gguf) | Q3_K_M | 1.571 GB | very small, high quality loss |
50
+ | [llama-3.2-Korean-Bllossom-3B-Q3_K_L.gguf](https://huggingface.co/tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF/tree/main/llama-3.2-Korean-Bllossom-3B-Q3_K_L.gguf) | Q3_K_L | 1.691 GB | small, substantial quality loss |
51
+ | [llama-3.2-Korean-Bllossom-3B-Q4_0.gguf](https://huggingface.co/tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF/tree/main/llama-3.2-Korean-Bllossom-3B-Q4_0.gguf) | Q4_0 | 1.786 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
52
+ | [llama-3.2-Korean-Bllossom-3B-Q4_K_S.gguf](https://huggingface.co/tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF/tree/main/llama-3.2-Korean-Bllossom-3B-Q4_K_S.gguf) | Q4_K_S | 1.796 GB | small, greater quality loss |
53
+ | [llama-3.2-Korean-Bllossom-3B-Q4_K_M.gguf](https://huggingface.co/tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF/tree/main/llama-3.2-Korean-Bllossom-3B-Q4_K_M.gguf) | Q4_K_M | 1.881 GB | medium, balanced quality - recommended |
54
+ | [llama-3.2-Korean-Bllossom-3B-Q5_0.gguf](https://huggingface.co/tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF/tree/main/llama-3.2-Korean-Bllossom-3B-Q5_0.gguf) | Q5_0 | 2.114 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
55
+ | [llama-3.2-Korean-Bllossom-3B-Q5_K_S.gguf](https://huggingface.co/tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF/tree/main/llama-3.2-Korean-Bllossom-3B-Q5_K_S.gguf) | Q5_K_S | 2.114 GB | large, low quality loss - recommended |
56
+ | [llama-3.2-Korean-Bllossom-3B-Q5_K_M.gguf](https://huggingface.co/tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF/tree/main/llama-3.2-Korean-Bllossom-3B-Q5_K_M.gguf) | Q5_K_M | 2.163 GB | large, very low quality loss - recommended |
57
+ | [llama-3.2-Korean-Bllossom-3B-Q6_K.gguf](https://huggingface.co/tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF/tree/main/llama-3.2-Korean-Bllossom-3B-Q6_K.gguf) | Q6_K | 2.462 GB | very large, extremely low quality loss |
58
+ | [llama-3.2-Korean-Bllossom-3B-Q8_0.gguf](https://huggingface.co/tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF/tree/main/llama-3.2-Korean-Bllossom-3B-Q8_0.gguf) | Q8_0 | 3.187 GB | very large, extremely low quality loss - not recommended |
59
+
60
+
61
+ ## Downloading instruction
62
+
63
+ ### Command line
64
+
65
+ Firstly, install Huggingface Client
66
+
67
+ ```shell
68
+ pip install -U "huggingface_hub[cli]"
69
+ ```
70
+
71
+ Then, downoad the individual model file the a local directory
72
+
73
+ ```shell
74
+ huggingface-cli download tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF --include "llama-3.2-Korean-Bllossom-3B-Q2_K.gguf" --local-dir MY_LOCAL_DIR
75
+ ```
76
+
77
+ If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try:
78
+
79
+ ```shell
80
+ huggingface-cli download tensorblock/llama-3.2-Korean-Bllossom-3B-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'
81
+ ```
llama-3.2-Korean-Bllossom-3B-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dad80db2c934d70dc2e3a78e10afc05f58af585523cbf5e909be7a7678b55b12
3
+ size 1363935776
llama-3.2-Korean-Bllossom-3B-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8889198471a7647b939dd11a45d19d978676a6dcfb3d5754e3f37c30299100e
3
+ size 1815347744
llama-3.2-Korean-Bllossom-3B-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:526adfe7b903f61a8edf6cfa724878d8f6c3e5f02b91809c0e38f858f8af0962
3
+ size 1687159328
llama-3.2-Korean-Bllossom-3B-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c84bcc5cc34d44b5aa6070c2b47c8c263a37b29d788ab483e0a1d128c229415b
3
+ size 1542849056
llama-3.2-Korean-Bllossom-3B-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:824efc7db2ef0f23a1eb02f8c88ddb86cd0e560c91bd664c9b7400ee5f93208d
3
+ size 1917190688
llama-3.2-Korean-Bllossom-3B-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:858ce577fb5074a49e1238d50289e83e0028fc34aeb2a6d5e2287e1e5e16b5b0
3
+ size 2019377696
llama-3.2-Korean-Bllossom-3B-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cc376cf76ad9fde4d127e3b481f934aefd66ce89e8207f98c26d3bcfdd8987bd
3
+ size 1928200736
llama-3.2-Korean-Bllossom-3B-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e45ca2996e93221d6df62f254d39800f9ecbd97f71ff560c6cbd86938f8f7574
3
+ size 2269512224
llama-3.2-Korean-Bllossom-3B-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c05ae1d04b2b6e4403d8dbdbfd7ce4e59da2056a4af0139a368f43612fa79c88
3
+ size 2322154016
llama-3.2-Korean-Bllossom-3B-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:55fc1272f824a20398312445dfed8d27e7a5c33da177efc73561bc26cfb7d0e6
3
+ size 2269512224
llama-3.2-Korean-Bllossom-3B-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78316b5362a5dc3e837a43dee8fee51050621dbc042ec923b5ebe177c32fab97
3
+ size 2643853856
llama-3.2-Korean-Bllossom-3B-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d98a6448168c98a183631cc76d05fcc53bb5b61b4b2fdfa678021edfb2e4336e
3
+ size 3421899296