How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf piskle/Bielik-V3.0-Instruct-11B-IQ2_XXS_GGUF# Run inference directly in the terminal:
llama cli -hf piskle/Bielik-V3.0-Instruct-11B-IQ2_XXS_GGUFUse pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf piskle/Bielik-V3.0-Instruct-11B-IQ2_XXS_GGUF# Run inference directly in the terminal:
./llama-cli -hf piskle/Bielik-V3.0-Instruct-11B-IQ2_XXS_GGUFBuild from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf piskle/Bielik-V3.0-Instruct-11B-IQ2_XXS_GGUF# Run inference directly in the terminal:
./build/bin/llama-cli -hf piskle/Bielik-V3.0-Instruct-11B-IQ2_XXS_GGUFUse Docker
docker model run hf.co/piskle/Bielik-V3.0-Instruct-11B-IQ2_XXS_GGUFQuick Links
Model highly unstable, use only for basic information!
Bielik V3.0 11B Instruct, a language model made by SpeakLeash (aka Spichlerz), but heavily quantized. The model's raw weights were taken and compressed to IQ2_XXS, an aggresive form of quantization meant for GGUF models. It achieves ~10tok/s while using 3.1GB of RAM with a context window of 8 thousand on the MacBook Pro (base M1, 8GB of unified memory, 256GB of internal storage).
Bielik at this level of compression tends to hallucinate frequently and generate context in languages different than intended. (Polish and english are the intended languages.)
- Downloads last month
- 23
Hardware compatibility
Log In to add your hardware
We're not able to determine the quantization variants.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for piskle/Bielik-V3.0-Instruct-11B-IQ2_XXS_GGUF
Base model
speakleash/Bielik-11B-v3-Base-20250730 Finetuned
speakleash/Bielik-PL-11B-v3.0-Instruct
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf piskle/Bielik-V3.0-Instruct-11B-IQ2_XXS_GGUF# Run inference directly in the terminal: llama cli -hf piskle/Bielik-V3.0-Instruct-11B-IQ2_XXS_GGUF