Add README.md (model card)

README.md

@@ -0,0 +1,186 @@
+---
+license: apache-2.0
+language:
+- es
+tags:
+- psychology
+- clinical
+- spanish
+- sft
+- axolotl
+base_model:
+- google/gemma-4-E4B-it
+---
+# ALIA-es-gemma-clinical-psychology-sft
+
+This repository contains a supervised fine-tuned (SFT) version of the **Gemma 4 E4B IT** model, optimized for Spanish psychological counseling and empathetic therapeutic dialogue.
+
+This model is the result of a **Supervised Fine-Tuning (SFT)** process on the [google/gemma-4-E4B-it](https://huggingface.co/google/gemma-4-E4B-it) base model, using a curated multi-turn psychology dataset in Spanish containing professional therapist dialogues.
+
+> [!NOTE]
+> This is a **pilot training run** for research purposes. It is not an official release and has not been validated for general deployment.
+
+> [!WARNING]
+> **DISCLAIMER:** This model is a domain-specific proof-of-concept for therapeutic guidance and research. It has *NOT* been clinically validated and has not undergone regulatory review.
+> It may produce incorrect, unsafe, or misleading psychological advice. Do not use this model as a substitute for professional therapy, diagnosis, or psychiatric treatment. Always consult a qualified psychologist or healthcare professional.
+
+---
+
+## Model Details
+
+### Description
+This model is a Transformer-based decoder-only language model that builds on the **Gemma 4 E4B IT** architecture through SFT alignment targeted to psychological support in Spanish.
+
+**SFT Fine-Tuning:** The model was fine-tuned using Supervised Fine-Tuning (SFT) to align responses with empathetic, active listening strategies. The dataset consists of multi-turn dialogues between patients and therapists, reinforcing safe, validating, and explorative conversational practices.
+
+### Architecture
+|                         |               |
+|-------------------------|:--------------|
+| **Base Model**          | google/gemma-4-E4B-it |
+| **Architecture**        | Dense + PLE (Parameter-Layer-Embedding) |
+| **Total Parameters**    | 8,000,000,000 (8B) |
+| **Effective Parameters**| 4,500,000,000 (4.5B) |
+| **Layers**              | 42            |
+| **Shared KV cache layers**| 18           |
+| **Context length**      | 4,096 (Configured) |
+| **Attention Pattern**   | Alternating local sliding-window (512 tokens) and global full-context |
+| **Precision**           | bfloat16      |
+| **Flash attention**     | ❌ (Disabled) |
+
+### Hyperparameters
+| Parameter | Value |
+|---|---:|
+| **Sequence length** | 4,096 |
+| **Sample packing** | false |
+| **Pad to sequence length** | true |
+| **Num. epochs** | 20 |
+| **Save steps** | 40 |
+| **Eval steps** | 20 |
+| **Logging steps** | 5 |
+| **Optimizer** | adamw_torch |
+| **Learning rate** | 5e-5 |
+| **LR scheduler** | cosine |
+| **Warmup ratio** | 0.05 |
+| **Weight decay** | 0.01 |
+| **Micro batch size** | 1 |
+| **Gradient accumulation steps** | 2 |
+| **Gradient checkpointing** | true |
+| **Val set size** | 0.05 |
+| **Seed** | 42 |
+| **BF16** | true |
+| **FP16** | false |
+| **Fine-Tuning Method** | SFT (Supervised Fine-Tuning) |
+| **Adapter** | LoRA |
+| **LoRA R** | 16 |
+| **LoRA Alpha** | 32 |
+| **LoRA Dropout** | 0.05 |
+| **LoRA Target Modules** | `model.language_model.layers.[\d]+.(_checkpoint_wrapped_module.)?(mlp|self_attn).(up|down|gate|q|k|v|o)_proj` |
+
+---
+
+## Intended Use
+
+### Direct Use
+The model is intended for research, development and support applications within **Spanish psychological counseling and active listening** contexts. Representative use cases include:
+- Assisting mental health professionals with drafts or suggestions for empathetic response strategies.
+- Role-playing and scenario training for psychology students.
+- Analysis and study of automated fine-tuning in clinical-adjacent communication settings.
+
+### Out-of-scope Use
+This model is not approved for clinical use or autonomous deployment. It must not be used as a primary source for psychological diagnosis, psychiatric treatment decisions, or crisis intervention. Any deployment that impacts patient safety requires extensive validation, risk assessment and regulatory clearance.
+
+---
+
+## How to use
+
+### Python Example
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+model_id = "SINAI/ALIA-es-gemma-clinical-psychology-sft"
+
+# System prompt used to steer the model towards empathetic therapy
+system_prompt = (
+    "Eres un terapeuta psicológico empático y profesional. "
+    "Escucha activamente al paciente y responde de forma apropiada, "
+    "validando sus emociones y explorando su experiencia."
+)
+
+# Example conversation structure (using the Gemma 4 chat template style)
+messages = [
+    {"role": "system", "content": system_prompt},
+    {"role": "user", "content": "Hola. Pues llevo desde hace mucho sintiendome con mucha ansiedad por basicamente casi todo"}
+]
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    device_map="auto",
+    torch_dtype=torch.bfloat16
+)
+
+# Apply the chat template
+prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=200)
+
+# Decode the generated tokens (skipping the prompt part)
+generated_tokens = outputs[0][inputs.input_ids.shape[1]:]
+print(tokenizer.decode(generated_tokens, skip_special_tokens=True))
+```
+
+---
+
+## Data
+
+### SFT Fine-Tuning Data
+> [!NOTE]
+> **Data Availability:** The dataset used for training is publicly available on Hugging Face at [SINAI/ALIA-es-clinical-psychology-dialogues](https://huggingface.co/datasets/SINAI/ALIA-es-clinical-psychology-dialogues).
+
+To adapt the model to empathetic psychological counseling in Spanish we used the following resource:
+
+* **Supervised Fine-Tuning (SFT)**
+    * **Dataset:** `ALIA-es-clinical-psychology-dialogues.jsonl` (containing 67 multi-turn sessions)
+    * **Description:** A multi-turn dataset structured with a `conversations` history. The target responses (`assistant`) are human-curated/edited therapist dialogues that prioritize validation, empathy, active listening, and safe, explorative therapeutic interaction in Spanish.
+
+---
+
+## Additional Information
+
+### License
+[Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0)
+
+### Citation
+```bibtex
+@misc{ALIA-es-gemma-clinical-psychology-sft,
+    title={ALIA-es-gemma-clinical-psychology-sft: Empathetic Psychology SFT Model for Spanish},
+    author={SINAI Research Group},
+    year={2026},
+    publisher={HuggingFace},
+    howpublished={\url{https://huggingface.co/SINAI/ALIA-es-gemma-clinical-psychology-sft}}
+}
+```
+
+Please also cite the base model and family:
+
+```bibtex
+@misc{gemma4_2026,
+    title={Gemma 4: Open Weights Multimodal Models},
+    author={Google DeepMind},
+    year={2026},
+    url={https://deepmind.google/gemma}
+}
+```
+
+### Funding
+This work is funded by the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project [ALIA](https://alia.gob.es).
+
+---
+
+**Contact:** [ALIA Project](https://www.alia.gob.es/) - [SINAI Research Group](https://sinai.ujaen.es) - [Universidad de Jaén](https://www.ujaen.es/)
+
+**More Information:** [SINAI Research Group](https://sinai.ujaen.es) | [ALIA-UJA Project](https://github.com/sinai-uja)
+