Small local LLMs in GGUF - a mertkayacs Collection

mertkayacs 's Collections

Eschatia Labs: open models and data

JevAlt decision models

Small local LLMs in GGUF

Tholos: local AI agents

Explainable deepfake detection

Turkish and German AI models

Benchmarks and datasets

Small local LLMs in GGUF

updated 7 days ago

Run 2B and 4B language models locally: decision models with the JevAlt server and tool-use agents with Ollama or llama.cpp.

mertkayacs/Karar-4B-GGUF

Text Classification • 4B • Updated 4 days ago • 657

Note Turkish decision model for the JevAlt server. Q4_K_M: 2.71 GB file, 3.03 GB peak RAM at 4k context.
mertkayacs/Deem-4B-GGUF

Text Classification • 4B • Updated 4 days ago • 644

Note English decision model for the JevAlt server. Q4_K_M: 2.71 GB file, 3.04 GB peak RAM at 4k context.
mertkayacs/Wahler-4B-GGUF

Text Classification • 4B • Updated 4 days ago • 630

Note German decision model for the JevAlt server. Q4_K_M: 2.71 GB file, 3.03 GB peak RAM at 4k context.
mertkayacs/Tholos-2B-GGUF

Text Generation • 3B • Updated 4 days ago • 369 • 1

Note Local tool use with Ollama or llama.cpp. Q4_K_M file: 1.56 GB; CPU llama.cpp JSON schema: 134/160 tasks.