---
license: llama2
tags:
- peft
- lora
- llama-3
- samsum
- summarization
library_name: peft
base_model: meta-llama/Llama-3.2-1B
---

# 🦙 LLaMA 3.2 1B + SAMSum LoRA Adapter

This is a LoRA adapter trained on the [SAMSum dataset](https://huggingface.co/datasets/samsum) for dialogue summarization using `meta-llama/Llama-3.2-1B` as the base model.

## 🛠️ Training Details

- **Base model**: `meta-llama/Llama-3.2-1B`
- **LoRA config**: r=8, alpha=32, dropout=0.01
- **Epochs**: 1
- **Batch size**: 1 (accumulation: 4)
- **Precision**: 8-bit (bitsandbytes)
- **Device**: Google Colab (T4 16GB)

## 🔧 Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-1B", device_map="auto")
model = PeftModel.from_pretrained(base, "aptro/Llama-3.2-1B-samsun")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B")
```

## License

This model follows the LLaMA 2 Community License Agreement.