Instructions to use DevQuasar-10/bigscience.bloom-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use DevQuasar-10/bigscience.bloom-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf DevQuasar-10/bigscience.bloom-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf DevQuasar-10/bigscience.bloom-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf DevQuasar-10/bigscience.bloom-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf DevQuasar-10/bigscience.bloom-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf DevQuasar-10/bigscience.bloom-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf DevQuasar-10/bigscience.bloom-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf DevQuasar-10/bigscience.bloom-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf DevQuasar-10/bigscience.bloom-GGUF:Q4_K_M
Use Docker
docker model run hf.co/DevQuasar-10/bigscience.bloom-GGUF:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use DevQuasar-10/bigscience.bloom-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "DevQuasar-10/bigscience.bloom-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DevQuasar-10/bigscience.bloom-GGUF", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/DevQuasar-10/bigscience.bloom-GGUF:Q4_K_M
- Ollama
How to use DevQuasar-10/bigscience.bloom-GGUF with Ollama:
ollama run hf.co/DevQuasar-10/bigscience.bloom-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use DevQuasar-10/bigscience.bloom-GGUF with Docker Model Runner:
docker model run hf.co/DevQuasar-10/bigscience.bloom-GGUF:Q4_K_M
- Lemonade
How to use DevQuasar-10/bigscience.bloom-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull DevQuasar-10/bigscience.bloom-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.bigscience.bloom-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Upload folder using huggingface_hub
Browse files- .gitattributes +14 -0
- bigscience.bloom.Q8_0-00001-of-00014.gguf +3 -0
- bigscience.bloom.Q8_0-00002-of-00014.gguf +3 -0
- bigscience.bloom.Q8_0-00003-of-00014.gguf +3 -0
- bigscience.bloom.Q8_0-00004-of-00014.gguf +3 -0
- bigscience.bloom.Q8_0-00005-of-00014.gguf +3 -0
- bigscience.bloom.Q8_0-00006-of-00014.gguf +3 -0
- bigscience.bloom.Q8_0-00007-of-00014.gguf +3 -0
- bigscience.bloom.Q8_0-00008-of-00014.gguf +3 -0
- bigscience.bloom.Q8_0-00009-of-00014.gguf +3 -0
- bigscience.bloom.Q8_0-00010-of-00014.gguf +3 -0
- bigscience.bloom.Q8_0-00011-of-00014.gguf +3 -0
- bigscience.bloom.Q8_0-00012-of-00014.gguf +3 -0
- bigscience.bloom.Q8_0-00013-of-00014.gguf +3 -0
- bigscience.bloom.Q8_0-00014-of-00014.gguf +3 -0
.gitattributes
CHANGED
|
@@ -75,3 +75,17 @@ bigscience.bloom.Q4_K_M-00006-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
|
|
| 75 |
bigscience.bloom.Q4_K_M-00007-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
|
| 76 |
bigscience.bloom.Q4_K_M-00008-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
|
| 77 |
bigscience.bloom.Q4_K_M-00009-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
bigscience.bloom.Q4_K_M-00007-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
|
| 76 |
bigscience.bloom.Q4_K_M-00008-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
|
| 77 |
bigscience.bloom.Q4_K_M-00009-of-00009.gguf filter=lfs diff=lfs merge=lfs -text
|
| 78 |
+
bigscience.bloom.Q8_0-00001-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
| 79 |
+
bigscience.bloom.Q8_0-00002-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
| 80 |
+
bigscience.bloom.Q8_0-00003-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
| 81 |
+
bigscience.bloom.Q8_0-00004-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
| 82 |
+
bigscience.bloom.Q8_0-00005-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
| 83 |
+
bigscience.bloom.Q8_0-00006-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
| 84 |
+
bigscience.bloom.Q8_0-00007-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
| 85 |
+
bigscience.bloom.Q8_0-00008-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
| 86 |
+
bigscience.bloom.Q8_0-00009-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
| 87 |
+
bigscience.bloom.Q8_0-00010-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
| 88 |
+
bigscience.bloom.Q8_0-00011-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
| 89 |
+
bigscience.bloom.Q8_0-00012-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
| 90 |
+
bigscience.bloom.Q8_0-00013-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
| 91 |
+
bigscience.bloom.Q8_0-00014-of-00014.gguf filter=lfs diff=lfs merge=lfs -text
|
bigscience.bloom.Q8_0-00001-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2c871ed5b6b0bda5deca59be771a03f0d90eef563c22c5e57dc4c54a1c40c2c8
|
| 3 |
+
size 13771785152
|
bigscience.bloom.Q8_0-00002-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7ed259a2678c3c632787f5ea8eff2447b75256c522457461dfe4d9aaa583fc87
|
| 3 |
+
size 13979496000
|
bigscience.bloom.Q8_0-00003-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3af669f72ca5d90148501c5ab36560b74b582cdedbb3c56e24ec4ebc025f04ee
|
| 3 |
+
size 13979324000
|
bigscience.bloom.Q8_0-00004-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6ded4b5069ae92001be41c0203c243176d4e4e7a559afc0bf64b685229800d8c
|
| 3 |
+
size 13979381376
|
bigscience.bloom.Q8_0-00005-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4664db9e042fe022d072385ac2b0e867ccdd734bf10defdc57d0eb9c9103d370
|
| 3 |
+
size 13979496064
|
bigscience.bloom.Q8_0-00006-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2294c631f8f3f3e1075aa080f1c954016bd06b577224ba041bc53bf67606cdcc
|
| 3 |
+
size 13979324032
|
bigscience.bloom.Q8_0-00007-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:32d1b85e70c63a05a15e997a9b7c269f06a990b5d9ddabea09d8af120af9fd49
|
| 3 |
+
size 13979381376
|
bigscience.bloom.Q8_0-00008-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:efe8beb5e52523f0e83863dd2513ed9081bcb4890a1122951a957af2d08ca7cb
|
| 3 |
+
size 13979496064
|
bigscience.bloom.Q8_0-00009-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:919345cd324a0a187ee6cf675acaab9b550854fa7faa3fac422ad7a83309027d
|
| 3 |
+
size 13979324032
|
bigscience.bloom.Q8_0-00010-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:06a9d740ff86e2cc50b13029dd5a9a814d1cfb1ec8ca1bab890eb1895e41e1e7
|
| 3 |
+
size 13979381376
|
bigscience.bloom.Q8_0-00011-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:65a5c514d17327d7903deadebd537e67f0da005f6428357360aadf62768514fa
|
| 3 |
+
size 13979496064
|
bigscience.bloom.Q8_0-00012-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c956da7d62d3a21ee531a357db0fe92129fb88ce5e26d3cd5ecfcaa7a7c56f29
|
| 3 |
+
size 13979324032
|
bigscience.bloom.Q8_0-00013-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:eedb00ded33bd0eeb0eac65ff2e0a0e1950f3dd83c7951b6b8225f6813cfa6b6
|
| 3 |
+
size 13979381376
|
bigscience.bloom.Q8_0-00014-of-00014.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:062f1141c53a9ebccc5fde4ecb0ad294c79a1609d549fe524247d205973b450d
|
| 3 |
+
size 9610684768
|