metadata
license: apache-2.0
language:
- en
- zh
base_model:
- deepseek-ai/DeepSeek-R1-0528-Qwen3-8B
Deepseek 8B 0528
This is the LLM about HIPPA law. Ask the LLM about HIPAA.
Use the model in ollama
First download and install ollama.
Command
in windows command line, or in terminal in ubuntu, type:
ollama run hf.co/John1604/DeepSeek-R1-0528-Qwen3-8B-gguf:q6_k
(q6_k is the model quant type, q5_k_s, q4_k_m, ..., can also be used)
C:\Users\developer>ollama run hf.co/John1604/DeepSeek-R1-0528-Qwen3-8B-gguf:q6_k
pulling manifest
...
verifying sha256 digest
writing manifest
success
>>> Send a message (/? for help)
Use the model in LM Studio
download and install LM Studio
Discover models
In the LM Studio, click "Discover" icon. "Mission Control" popup window will be displayed.
In the "Mission Control" search bar, type "John1604/DeepSeek-R1-0528-Qwen3-8B-gguf" and check "GGUF", the model should be found.
Download the model.
Load the model.
Ask questions.
quantized models
| Type | Bits | Quality | Description |
|---|---|---|---|
| Q2_K | 2-bit | 🟥 Low | Minimal footprint; only for tests |
| Q3_K_S | 3-bit | 🟧 Low | “Small” variant (less accurate) |
| Q3_K_M | 3-bit | 🟧 Low–Med | “Medium” variant |
| Q4_K_S | 4-bit | 🟨 Med | Small, faster, slightly less quality |
| Q4_K_M | 4-bit | 🟩 Med–High | “Medium” — best 4-bit balance |
| Q5_K_S | 5-bit | 🟩 High | Slightly smaller than Q5_K_M |
| Q5_K_M | 5-bit | 🟩🟩 High | Excellent general-purpose quant |
| Q6_K | 6-bit | 🟩🟩🟩 Very High | Almost FP16 quality, larger size |
| Q8_0 | 8-bit | 🟩🟩🟩🟩 | Near-lossless baseline |