Transformers
GGUF
English
text-generation-inference
unsloth
qwen2
trl
reasoning
finetune
edge-device
research
conversational
Instructions to use mradermacher/mini-Cogito-R1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mradermacher/mini-Cogito-R1-GGUF with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("mradermacher/mini-Cogito-R1-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use mradermacher/mini-Cogito-R1-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf mradermacher/mini-Cogito-R1-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf mradermacher/mini-Cogito-R1-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf mradermacher/mini-Cogito-R1-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf mradermacher/mini-Cogito-R1-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf mradermacher/mini-Cogito-R1-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf mradermacher/mini-Cogito-R1-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf mradermacher/mini-Cogito-R1-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf mradermacher/mini-Cogito-R1-GGUF:Q4_K_M
Use Docker
docker model run hf.co/mradermacher/mini-Cogito-R1-GGUF:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use mradermacher/mini-Cogito-R1-GGUF with Ollama:
ollama run hf.co/mradermacher/mini-Cogito-R1-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use mradermacher/mini-Cogito-R1-GGUF with Docker Model Runner:
docker model run hf.co/mradermacher/mini-Cogito-R1-GGUF:Q4_K_M
- Lemonade
How to use mradermacher/mini-Cogito-R1-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull mradermacher/mini-Cogito-R1-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.mini-Cogito-R1-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
uploaded from rich1
Browse files- .gitattributes +12 -0
- mini-Cogito-R1.IQ4_XS.gguf +3 -0
- mini-Cogito-R1.Q2_K.gguf +3 -0
- mini-Cogito-R1.Q3_K_L.gguf +3 -0
- mini-Cogito-R1.Q3_K_M.gguf +3 -0
- mini-Cogito-R1.Q3_K_S.gguf +3 -0
- mini-Cogito-R1.Q4_K_M.gguf +3 -0
- mini-Cogito-R1.Q4_K_S.gguf +3 -0
- mini-Cogito-R1.Q5_K_M.gguf +3 -0
- mini-Cogito-R1.Q5_K_S.gguf +3 -0
- mini-Cogito-R1.Q6_K.gguf +3 -0
- mini-Cogito-R1.Q8_0.gguf +3 -0
- mini-Cogito-R1.f16.gguf +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
mini-Cogito-R1.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
mini-Cogito-R1.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
mini-Cogito-R1.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
mini-Cogito-R1.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
mini-Cogito-R1.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
mini-Cogito-R1.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
mini-Cogito-R1.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
mini-Cogito-R1.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
mini-Cogito-R1.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
mini-Cogito-R1.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
mini-Cogito-R1.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
mini-Cogito-R1.f16.gguf filter=lfs diff=lfs merge=lfs -text
|
mini-Cogito-R1.IQ4_XS.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fa5fd9182fb6777194e6f6cfee95ebd6b4ca479336ce562193b2f5df43b3b52b
|
| 3 |
+
size 1026164192
|
mini-Cogito-R1.Q2_K.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9292a784e1567a6ce2cebfe7d1be5fce2736250e777a7f4ee9c2c6e8ce1ddb07
|
| 3 |
+
size 752882144
|
mini-Cogito-R1.Q3_K_L.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4a8bb1882115a29aa5304d7a988a5a32097699987e2f708560e74baa702b0dae
|
| 3 |
+
size 980442080
|
mini-Cogito-R1.Q3_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:074ab9046f979163753aa1d2e1dd3a0658c813ff1cf530a619f8738c4fa8ada2
|
| 3 |
+
size 924457952
|
mini-Cogito-R1.Q3_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9f532e40e2145d247417b956fc29595b57bc934acf72da65048379b789900931
|
| 3 |
+
size 861223904
|
mini-Cogito-R1.Q4_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:562ed9d7dc380f9a74e5d1b04508b417a730bbf57347ea869293146e6e127278
|
| 3 |
+
size 1117322720
|
mini-Cogito-R1.Q4_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8d2d4c7aad756e27a8a23ccb14d7fdfc32588cbecebc562e099ae4d34c73ab06
|
| 3 |
+
size 1071586784
|
mini-Cogito-R1.Q5_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fa3b577121c75f3cbc870819a609f88bd0a92447ee09a15bfdedb24aa9851e8e
|
| 3 |
+
size 1285496288
|
mini-Cogito-R1.Q5_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0ebc47caf149e45709fd01913e2bccb4fc090870ddf4a19037e49f6b6af5b799
|
| 3 |
+
size 1259175392
|
mini-Cogito-R1.Q6_K.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a73066fde2b997b940de70d31d2a512605cfb45858676f49db8dc126774b19f8
|
| 3 |
+
size 1464180704
|
mini-Cogito-R1.Q8_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:efd54d6481961d2226c3f506814a440a7d3f9fdf028e4d15be9c9ee5166850d8
|
| 3 |
+
size 1894534112
|
mini-Cogito-R1.f16.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3fd9784042d9d93ae44cc41ac7e0c784d1e15b6c2b70e793a272db4d5df52177
|
| 3 |
+
size 3560418272
|