---
base_model: quantized_Llama-3.1-8B-Instruct
tags:
- text-generation-inference
- transformers
- peft
- trl
- llama
- dora
license: apache-2.0
language:
- en
datasets:
- mlabonne/FineTome-100k
---

# LLaMA-3.1-8B-Instruct DoRA Fine-Tuned

- **Developed by:** avinashhm
- **License:** apache-2.0
- **Finetuned from model:** devatar/quantized_Llama-3.1-8B-Instruct

This model is a fine-tuned version of `devatar/quantized_Llama-3.1-8B-Instruct`, adapted using DoRA (Weight-Decomposed Low-Rank Adaptation) on a subset of the `mlabonne/FineTome-100k` dataset. It is optimized for instruction-following tasks, such as answering questions and explaining concepts, and was fine-tuned on a 40GB GPU with memory-efficient techniques.

## Training Details
- **Dataset**: `mlabonne/FineTome-100k` (5,000 samples)
- **Fine-Tuning Method**: DoRA (r=8, lora_alpha=16, target_modules=["q_proj", "k_proj", "v_proj", "o_proj"])
- **Training Steps**: 500
- **Optimizer**: Paged AdamW 8-bit
- **Learning Rate**: 2e-5 (cosine scheduler)
- **Batch Size**: Effective batch size of 8 (per_device_train_batch_size=1, gradient_accumulation_steps=8)
- **Precision**: Mixed precision (FP16)
- **Training Loss**: Decreased from 1.4494 to 0.8145 over 500 steps

## Dataset
The model was trained on a 5,000-sample subset of `mlabonne/FineTome-100k`, which contains high-quality instruction-response pairs. Conversations were formatted as `### Human: ... ### Gpt: ...` for training, covering tasks like explaining programming concepts and reasoning.

## Usage
To use the model for inference:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("avinashhm/llama-3.1-8b-dora-finetuned", device_map="auto", torch_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained("avinashhm/llama-3.1-8b-dora-finetuned")

inputs = tokenizer("Explain boolean operators in programming.", return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_length=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

**Note**: The base model is 4-bit quantized, and merging DoRA adapters may introduce minor rounding errors in generations.

## Requirements
- `torch`
- `transformers`
- `peft`
- `trl`
- `datasets`
- `bitsandbytes`
- GPU with at least 40GB VRAM for training (less for inference)

Install dependencies:
```bash
pip install torch transformers datasets peft trl bitsandbytes
pip install git+https://github.com/huggingface/peft.git
```

## Limitations
- Fine-tuned on a 5,000-sample subset, which may limit generalization.
- 4-bit quantization may introduce slight performance trade-offs.
- Used an older `trl` version (pre-0.7.0), lacking features like `max_seq_length`.