---
license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B
library_name: peft
pipeline_tag: text-generation
tags:
- peft
- lora
- qlora
- qwen
- andaluh
- andalusian-spanish
---

# Qwen-Andaluh 0.8B LoRA

LoRA adapter for `Qwen/Qwen3.5-0.8B` trained as the non-persona
**Qwen-Andaluh** checkpoint.

The intended behaviour is simple:

```text
System prompt: Eres un asistente
Input: Spanish or Andalûh
Output: Andalûh EPA
```

This is an adapter release, not a merged model.

## Training Stages

- Andalûh continued pretraining / CPT.
- Accent SFT.
- Accent ORPO preference tuning.

The source data was generated from Spanish instruction rows by filtering
licence-compatible Spanish examples, protecting fragile spans, converting
assistant messages to Andalûh EPA, converting a controlled share of user
messages to Andalûh EPA, and restoring URLs, paths, code, package names and
model IDs unchanged.

## Adapter Details

- Base model: `Qwen/Qwen3.5-0.8B`
- Adapter type: LoRA / QLoRA
- Rank: 16
- Alpha: 32
- Dropout: 0.05
- Target modules: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`,
  `up_proj`, `down_proj`

## Use

```python
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model_id = "MariChatmen/qwen-andaluh-0.8b-lora"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoPeftModelForCausalLM.from_pretrained(model_id, device_map="auto")
```

## Limitations

- Experimental research adapter.
- May leak standard Spanish.
- May produce imperfect Andalûh EPA.
- Not an official linguistic standard.

## Related

- Persona dataset: <https://huggingface.co/datasets/MariChatmen/MariChatmen-Persona>
- AndaluGeeks EPA: <https://andaluh.es/epa-2/>
- `andaluh-py`: <https://github.com/andalugeeks/andaluh-py>