How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf netcat420/MFANN3bv0.7-GGUF# Run inference directly in the terminal:
llama cli -hf netcat420/MFANN3bv0.7-GGUFUse pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf netcat420/MFANN3bv0.7-GGUF# Run inference directly in the terminal:
./llama-cli -hf netcat420/MFANN3bv0.7-GGUFBuild from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf netcat420/MFANN3bv0.7-GGUF# Run inference directly in the terminal:
./build/bin/llama-cli -hf netcat420/MFANN3bv0.7-GGUFUse Docker
docker model run hf.co/netcat420/MFANN3bv0.7-GGUFQuick Links
the regular model is Q4_K_M quantized and the "L" model is Q4_K_S quantized for those of you on android :)
MFANN 3v version 0.7
fine-tuned on the MFANN dataset as it stands on 5/5/2024 as it is an ever changing and expaning dataset.
62.98 <- Average 61.69 <- ARC 75.98 <- HellaSwag 55.4 <- MMLU 53.49 <- TruthfulQA 77.66 <- Winogrande (this models strong-suit) 53.68 <- GSM8K
this model is completely uncensored
- Downloads last month
- 5
Hardware compatibility
Log In to add your hardware
We're not able to determine the quantization variants.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf netcat420/MFANN3bv0.7-GGUF# Run inference directly in the terminal: llama cli -hf netcat420/MFANN3bv0.7-GGUF