Hugging Face's logo

MariChatmen
/
qwen-andaluh-0.8b-lora

Text Generation
PEFT
Safetensors
lora
qlora
qwen
andaluh
andalusian-spanish
conversational
Model card Files
xet
 Community
qwen-andaluh-0.8b-lora / README.md
alobos's picture
alobos
Update persona dataset link after organisation transfer
f60d376 verified
preview code
|
Raw
History Blame Contribute Delete
1.76 kB
metadata 
license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B
library_name: peft
pipeline_tag: text-generation
tags:
  - peft
  - lora
  - qlora
  - qwen
  - andaluh
  - andalusian-spanish

Qwen-Andaluh 0.8B LoRA

LoRA adapter for Qwen/Qwen3.5-0.8B trained as the non-persona Qwen-Andaluh checkpoint.

The intended behaviour is simple:

System prompt: Eres un asistente
Input: Spanish or Andal没h
Output: Andal没h EPA

This is an adapter release, not a merged model.

Training Stages

  • Andal没h continued pretraining / CPT.
  • Accent SFT.
  • Accent ORPO preference tuning.

The source data was generated from Spanish instruction rows by filtering licence-compatible Spanish examples, protecting fragile spans, converting assistant messages to Andal没h EPA, converting a controlled share of user messages to Andal没h EPA, and restoring URLs, paths, code, package names and model IDs unchanged.

Adapter Details

  • Base model: Qwen/Qwen3.5-0.8B
  • Adapter type: LoRA / QLoRA
  • Rank: 16
  • Alpha: 32
  • Dropout: 0.05
  • Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Use 

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model_id = "MariChatmen/qwen-andaluh-0.8b-lora"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoPeftModelForCausalLM.from_pretrained(model_id, device_map="auto")

Limitations

  • Experimental research adapter.
  • May leak standard Spanish.
  • May produce imperfect Andal没h EPA.
  • Not an official linguistic standard.

Related