---
base_model: Qwen/Qwen2.5-0.5B
tags:
- gguf
- llama.cpp
- quantized
- trl
- sft
---

# qwen-capybara-medium-gguf

This is a GGUF conversion of [evalstate/qwen-capybara-medium](https://huggingface.co/evalstate/qwen-capybara-medium), which is a LoRA fine-tuned version of [Qwen/Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B).

## Model Details

- **Base Model:** Qwen/Qwen2.5-0.5B
- **Fine-tuned Model:** evalstate/qwen-capybara-medium
- **Training:** Supervised Fine-Tuning (SFT) with TRL
- **Format:** GGUF (for llama.cpp, Ollama, LM Studio, etc.)

## Available Quantizations

| File | Quant | Size | Description | Use Case |
|------|-------|------|-------------|----------|
| qwen-capybara-medium-f16.gguf | F16 | ~1GB | Full precision | Best quality, slower |
| qwen-capybara-medium-q8_0.gguf | Q8_0 | ~500MB | 8-bit | High quality |
| qwen-capybara-medium-q5_k_m.gguf | Q5_K_M | ~350MB | 5-bit medium | Good quality, smaller |
| qwen-capybara-medium-q4_k_m.gguf | Q4_K_M | ~300MB | 4-bit medium | Recommended - good balance |

## Usage

### With llama.cpp

```bash
# Download model
huggingface-cli download evalstate/qwen-capybara-medium-gguf qwen-capybara-medium-q4_k_m.gguf

# Run with llama.cpp
./llama-cli -m qwen-capybara-medium-q4_k_m.gguf -p "Your prompt here"
```

### With Ollama

1. Create a `Modelfile`:
```
FROM ./qwen-capybara-medium-q4_k_m.gguf
```

2. Create the model:
```bash
ollama create qwen-capybara -f Modelfile
ollama run qwen-capybara
```

### With LM Studio

1. Download the `.gguf` file
2. Import into LM Studio
3. Start chatting!

## Training Details

This model was fine-tuned using:
- **Dataset:** trl-lib/Capybara (1,000 examples)
- **Method:** Supervised Fine-Tuning with LoRA
- **Epochs:** 3
- **LoRA rank:** 16
- **Hardware:** A10G Large GPU

## License

Inherits the license from the base model: Qwen/Qwen2.5-0.5B

## Citation

```bibtex
@misc{qwen-capybara-medium-gguf,
  author = {evalstate},
  title = {Qwen Capybara Medium GGUF},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/evalstate/qwen-capybara-medium-gguf}
}
```

---

*Converted to GGUF format using llama.cpp*