---
language:
- en
pipeline_tag: text-generation
tags:
- quantization
- gguf
- llm
- inference
- efficient-llm
base_model:
- lilili896/qwen-coder-14b-vulrepair
---

# Qwen-Coder-14b-Vulrepair 

## 🧠 Model Overview

**Qwen-Coder-14b-Vulrepair-GGUF** is a quantized version of **Qwen-Coder-14b-Vulrepair**, optimized for efficient inference with reduced memory usage and faster runtime while preserving as much of the original model quality as possible.

This repository provides multiple quantized variants suitable for:
- Local inference
- Low-VRAM GPUs
- CPU-only environments
  

---

## 🔗 Original Model

- **Base model:** Qwen-Coder-14b-Vulrepair
- **Original repository:** https://huggingface.co/lilili896/qwen-coder-14b-vulrepair
- **Original authors:** lilili896


---

## 📦 Quantization Details

- **Quantization method:** GGUF
- **Quantization tool:** llama.cpp
- **Precision:** Mixed (2-8,bit depands in variant)
- **Activation aware:** No (weight-only quantinization)
- **Group size:** 256 (K-quant variants)


---

## 📦 Available Quantized Files

| Quant Format | File Name | Approx. Size | VRAM / RAM Needed | Notes |
|-------------|----------|--------------|-------------------|------|
| Q2_K | `qwen-coder-14b-vulrepair-q2_k.gguf` | ~5.7 GB | ~7 GB | Extreme compression; noticeable quality loss |
| Q3_K_S | `qwen-coder-14b-vulrepair-q3_k_s.gguf` | ~6.6 GB | ~7.5 GB | Smaller, faster, lower quality |
| Q3_K_M | `qwen-coder-14b-vulrepair-q3_k_m.gguf` | ~7.3 GB | ~8.5 GB | Better balance than Q3_K_S |
| Q3_K_L | `qwen-coder-14b-vulrepair-q3_k_l.gguf` | ~7.9 GB | ~9.8 GB | Highest-quality 3-bit variant |
| Q4_0 | `qwen-coder-14b-vulrepair-q4_0.gguf` | ~8.5 GB | ~10.3 GB | Legacy format; simpler quantization |
| Q4_K_S | `qwen-coder-14b-vulrepair-q4_k_s.gguf` | ~8.5 GB | ~10.5 GB | Smaller grouped 4-bit |
| Q4_K_M | `qwen-coder-14b-vulrepair-q4_k_m.gguf` | ~8.9 GB | ~11 GB | **Recommended default** |
| Q5_0 | `<qwen-coder-14b-vulrepair-q5_0.gguf` | ~10.3 GB | ~12 GB | Higher quality, larger size |
| Q5_K_S | `qwen-coder-14b-vulrepair-q5_k_s.gguf` | ~10.3 GB | ~12.2 GB | Efficient high-quality variant |
| Q5_K_M | `qwen-coder-14b-vulrepair-q5_K_M.gguf` | ~10.5 GB | ~12.8 GB | Near-FP16 quality |
| Q6_K | `qwen-coder-14b-vulrepair-q6_k.gguf` | ~12.1 GB | ~14.5 GB | Minimal quantization loss |
| Q8_0 | `qwen-coder-14b-vulrepair-q8_0.gguf` | ~15.7 GB | ~16 GB | Maximum quality; large memory |

> 💡 **Recommendation:** Start with **Q4_K_M** for the best quality-to-performance ratio.

---

## 🚀 Usage Example

### llama.cpp
```bash
./main -m qwen-coder-14b-vulrepair-q4_k_m.gguf -p "Write a python function that checks if the given string is palindrome" -n 256
```
### Python (llama-cpp-python)


```Python
from llama_cpp import Llama

llm = Llama(
    model_path="qwen-coder-14b-vulrepair-q4_k_m.gguf",
    n_ctx=4096,
    n_threads=8
)

print(llm("Your prompt here"))
```


## 🙋 Contact

**Maintainer:** **M Mashhudur Rahim [XythicK]**
 
**Role:**  
**Independent Machine Learning Researcher & Model Infrastructure Maintainer** 

(Focused on model quantization, optimization, and efficient deployment)


For issues, improvement requests, or additional quantization formats, please use the Hugging Face Discussions or Issues tab.

## ❤️ Acknowledgements

Thanks to the original model authors for their ongoing contributions to open AI research, and to Hugging Face and the open-source machine learning community for providing the tools and platforms that make efficient model sharing and deployment possible.