File size: 1,762 Bytes
932d680 f60d376 932d680 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | ---
license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B
library_name: peft
pipeline_tag: text-generation
tags:
- peft
- lora
- qlora
- qwen
- andaluh
- andalusian-spanish
---
# Qwen-Andaluh 0.8B LoRA
LoRA adapter for `Qwen/Qwen3.5-0.8B` trained as the non-persona
**Qwen-Andaluh** checkpoint.
The intended behaviour is simple:
```text
System prompt: Eres un asistente
Input: Spanish or Andal没h
Output: Andal没h EPA
```
This is an adapter release, not a merged model.
## Training Stages
- Andal没h continued pretraining / CPT.
- Accent SFT.
- Accent ORPO preference tuning.
The source data was generated from Spanish instruction rows by filtering
licence-compatible Spanish examples, protecting fragile spans, converting
assistant messages to Andal没h EPA, converting a controlled share of user
messages to Andal没h EPA, and restoring URLs, paths, code, package names and
model IDs unchanged.
## Adapter Details
- Base model: `Qwen/Qwen3.5-0.8B`
- Adapter type: LoRA / QLoRA
- Rank: 16
- Alpha: 32
- Dropout: 0.05
- Target modules: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`,
`up_proj`, `down_proj`
## Use
```python
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
model_id = "MariChatmen/qwen-andaluh-0.8b-lora"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoPeftModelForCausalLM.from_pretrained(model_id, device_map="auto")
```
## Limitations
- Experimental research adapter.
- May leak standard Spanish.
- May produce imperfect Andal没h EPA.
- Not an official linguistic standard.
## Related
- Persona dataset: <https://huggingface.co/datasets/MariChatmen/MariChatmen-Persona>
- AndaluGeeks EPA: <https://andaluh.es/epa-2/>
- `andaluh-py`: <https://github.com/andalugeeks/andaluh-py>
|