---
base_model: Qwen/Qwen2.5-Coder-3B-Instruct
base_model_relation: adapter
library_name: peft
license: other
license_name: qwen-research
license_link: https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct/blob/main/LICENSE
language:
- en
- id
pipeline_tag: text-generation
tags:
- lora
- peft
- transformers
- unsloth
- qwen
- qwen-coder
---

# Dendriva Qwen2.5-Coder 3B Instruct — LoRA

PEFT LoRA adapter trained from `Qwen/Qwen2.5-Coder-3B-Instruct`. This is the
lightweight, trainable-format export for Unsloth or Transformers. It requires
the base model at load time.

The ready-to-run LM Studio quantization is available in
`kangsyahrul/dendriva-qwen2.5-coder-3b-instruct-gguf`.

## Training provenance

- Selected checkpoint: `checkpoint-69`
- Epochs: 3
- Steps: 69
- Context length: 32,768
- LoRA rank / alpha / dropout: 16 / 16 / 0
- Learning rate: 2e-4
- Batch size: 2
- Optimizer: AdamW 8-bit
- Warmup steps: 3
- Training tokens reported by Unsloth Studio: 8,927,658

The repository intentionally excludes optimizer, scheduler, RNG, and trainer
state because those are not required for local inference.

## Load with Transformers and PEFT

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen2.5-Coder-3B-Instruct"
adapter_id = "kangsyahrul/dendriva-qwen2.5-coder-3b-instruct-lora"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, adapter_id)
```

In Unsloth Desktop, use the Hugging Face model source and enter the full adapter
repository ID. Authenticate with a Hugging Face token because the repository is
private.

## Evaluation status

The adapter files and tokenizer were verified after upload. The training run
completed successfully, but no comprehensive held-out coding or Manim benchmark
is published with this repository. Compile, render, and test generated code
before use.

## License

This derivative follows the Qwen Research License of the base model. Review the
[base-model license](https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct/blob/main/LICENSE)
before redistribution or commercial use.