Instructions to use nold/MediKAI-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use nold/MediKAI-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf nold/MediKAI-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf nold/MediKAI-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf nold/MediKAI-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf nold/MediKAI-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf nold/MediKAI-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf nold/MediKAI-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf nold/MediKAI-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf nold/MediKAI-GGUF:Q4_K_M
Use Docker
docker model run hf.co/nold/MediKAI-GGUF:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use nold/MediKAI-GGUF with Ollama:
ollama run hf.co/nold/MediKAI-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use nold/MediKAI-GGUF with Docker Model Runner:
docker model run hf.co/nold/MediKAI-GGUF:Q4_K_M
- Lemonade
How to use nold/MediKAI-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull nold/MediKAI-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.MediKAI-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
18e0e0894af82b216f178d6ed1d9a898b7c38fbd1fdbefcd56b4b1fbe625d7b2
Browse files
README.md
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
# MediKAI - Your Healthcare Companion 🏥💬
|
| 6 |
+
|
| 7 |
+
Welcome to mediKAI, the latest healthcare-focused model by HelpingAI designed to provide personalized assistance and support in medical-related queries.
|
| 8 |
+
|
| 9 |
+
## Overview
|
| 10 |
+
mediKAI is a 14 billion parameters model that specializes in healthcare-related topics and medical assistance. Whether you have questions about symptoms, treatments, medications, or general health and wellness, mediKAI is here to help.
|
| 11 |
+
|
| 12 |
+
## Languages Supported
|
| 13 |
+
- English
|
| 14 |
+
- French
|
| 15 |
+
- Hindi
|
| 16 |
+
- Spanish
|
| 17 |
+
- Arabic
|
| 18 |
+
```
|
| 19 |
+
|
| 20 |
+
***
|
| 21 |
+
|
| 22 |
+
Quantization of Model [OEvortex/MediKAI](https://huggingface.co/OEvortex/MediKAI).
|
| 23 |
+
Created using [llm-quantizer](https://github.com/Nold360/llm-quantizer) Pipeline
|
test.log
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
What is a Large Language Model?
|
| 2 |
+
|
| 3 |
+
:
|
| 4 |
+
Alice:你好,我注意到你最近在网络上看到的很多关于一种叫做“大型语言模型”的东西。你能告诉我更多吗?GPT-D 是什么样的一个大字么(PANG)呢?)
|