Update README.md

README.md

@@ -18,4 +18,207 @@ tags:
 - abliterated
 - reasoning
 - long-context
----
+---
+
+# XION 0.2 27B
+
+XION 0.2 27B is an experimental multilingual conversational model developed
+by the PIXELZX team. It is designed to provide a consistent XION identity,
+support persona-conditioned conversations, answer in multiple languages, and
+handle both direct and reasoning-formatted responses.
+
+XION 0.2 27B is adapted from
+[`Jiunsong/SuperQwen3.8-27b-abliterated`](https://huggingface.co/Jiunsong/SuperQwen3.8-27b-abliterated),
+which is based on Qwen3.8-27B. The fine-tuning data is text-only, even though
+the underlying Qwen3.8 architecture supports image and video inputs.
+
+> **Status:** Experimental. This repository contains the data-preparation
+> pipeline and the current Together AI export. Independent XION benchmark
+> results and the final published checkpoint should be added after a completed
+> training run.
+
+## Model Summary
+
+| Property | Value |
+| --- | --- |
+| Model name | XION 0.2 27B |
+| Developer | PIXELZX |
+| Model family | Qwen3.8 |
+| Starting checkpoint | `Jiunsong/SuperQwen3.8-27b-abliterated` |
+| Model size | 27B parameters, inherited from Qwen3.8-27B |
+| Model type | Multimodal causal language model |
+| Native context length | 262,144 tokens in the Qwen3.8 configuration |
+| Fine-tuning method | Supervised fine-tuning (SFT) |
+| Fine-tuning data | Text-only conversational and instruction data |
+| Supported languages in the prepared data | 13 |
+
+## Intended Capabilities
+
+The training data targets the following behaviors:
+
+- Identify the assistant as XION and attribute its development to PIXELZX.
+- Distinguish XION from ChatGPT, Claude, Gemini, GPT-4, and other third-party
+  models.
+- Hold conversations in Arabic, Chinese, Dutch, English, French, German,
+  Indonesian, Japanese, Korean, Portuguese, Russian, Thai, and Vietnamese.
+- Follow system-prompt personas such as secretary, friend, and teacher styles.
+- Produce direct answers as well as responses containing Qwen-style reasoning
+  sections.
+- Answer factual questions about AI companies and model identities without
+  confusing those entities with XION.
+- Learn selected coding and agent-style interaction patterns from a small
+  Fable-5 subset.
+
+These are training objectives, not guarantees of reliable performance.
+
+## Multi-Token Prediction
+
+The Qwen3.8 architecture includes Multi-Token Prediction (MTP) components.
+However, the Together AI fine-tuning API does not expose a separate MTP loss or
+MTP training switch. The current recipe is standard SFT and must not be
+described as additional MTP fine-tuning.
+
+## Training Data
+
+The current Together AI export is stored in
+`data/together/{train,val}.jsonl`. It uses pre-rendered Qwen3.8 ChatML in the
+instruction format, with one `prompt` and one `completion` field per line.
+
+| Dataset | Train | Validation | Purpose |
+| --- | ---: | ---: | --- |
+| `qwen3_identity` | 1,235 | 156 | Identity and third-party knowledge examples |
+| `qwen3_identity_nothink` | 624 | 78 | Direct identity and knowledge responses |
+| `qwen3_persona` | 858 | 78 | System-prompt persona conversations |
+| `qwen3_uncensored` | 214 | 26 | Low-refusal and open-ended response examples |
+| `fable5` | 40 | 2 | Quality-ranked agent traces flattened to text |
+| **Total** | **2,971** | **340** | |
+
+The prepared data covers Arabic, Chinese, English, French, German, Indonesian,
+Japanese, Korean, Portuguese, Russian, Spanish, Thai, and Vietnamese. Some
+examples contain reasoning traces. The Fable-5 traces are serialized as text;
+they are not native Together function-calling examples.
+
+The Together export is capped at 28,000 rendered tokens per example to leave a
+safety margin below the 32,768-token Qwen3.8 SFT context limit used by Together
+AI. The underlying Qwen3.8 model has a larger native context window, but that
+does not increase the context limit of this Together training job.
+
+## Training Recipe
+
+The current release candidate was prepared for the following Together AI SFT
+configuration:
+
+- Three training epochs.
+- Three validation evaluations.
+- LoRA by default, unless full fine-tuning is selected explicitly.
+- A held-out validation file at `data/together/val.jsonl`.
+- Qwen3.8 ChatML rendered locally before upload.
+
+The pre-rendered `prompt`/`completion` format is intentional. Uploading the
+older `messages` export can cause Together's Qwen3.8 chat-template processing
+to fail with `No user query found in messages`.
+
+## Usage with Transformers
+
+After the XION checkpoint is published, replace `MODEL_ID` with its Hugging
+Face repository ID.
+
+```bash
+pip install -U transformers torch accelerate
+```
+
+```python
+from transformers import AutoModelForMultimodalLM, AutoProcessor
+
+MODEL_ID = "YOUR_ORG/XION-0.2-27B"
+
+processor = AutoProcessor.from_pretrained(MODEL_ID)
+model = AutoModelForMultimodalLM.from_pretrained(
+    MODEL_ID,
+    device_map="auto",
+    torch_dtype="auto",
+)
+
+messages = [
+    {
+        "role": "user",
+        "content": [{"type": "text", "text": "Who are you?"}],
+    }
+]
+
+inputs = processor.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    tokenize=True,
+    return_dict=True,
+    return_tensors="pt",
+).to(model.device)
+
+outputs = model.generate(**inputs, max_new_tokens=256)
+new_tokens = outputs[0][inputs["input_ids"].shape[-1] :]
+print(processor.decode(new_tokens, skip_special_tokens=True))
+```
+
+Qwen3.8-based models use thinking mode by default. The exact controls for
+thinking, reasoning effort, and preserved thinking depend on the serving
+framework. Follow the documentation for the selected Transformers, vLLM,
+SGLang, or API runtime before changing those settings.
+
+## Together AI Export
+
+The generated files can be uploaded with the Together CLI:
+
+```bash
+tg files upload data/together/train.jsonl
+```
+
+Use the new file IDs when creating an SFT job. Do not reuse an ID for an older
+`messages`-format file.
+
+The local Together SDK checks pass for both exported files. Server-side
+validation still occurs after upload and should reach `COMPLETED` before a
+training job is started.
+
+## Limitations and Safety
+
+- XION 0.2 27B is experimental and has no independent benchmark results in
+  this repository.
+- The starting checkpoint is refusal-reduced and should not be treated as a
+  safety-aligned model.
+- The training mixture includes open-ended and potentially harmful requests.
+  Outputs may be unsafe, incorrect, biased, or unsuitable for deployment.
+- The model can produce content that violates laws, policies, or user safety
+  requirements. Add application-level moderation, access controls, logging,
+  and human review where appropriate.
+- Reasoning sections should not automatically be treated as verified facts or
+  exposed as authoritative explanations.
+- The Fable-5 subset teaches serialized agent traces, not guaranteed tool
+  execution or secure code execution.
+- Qwen3.8 benchmark results must not be presented as XION benchmark results.
+
+## License and Attribution
+
+The upstream Qwen3.8 and `Jiunsong/SuperQwen3.8-27b-abliterated` model cards
+state Apache-2.0 licensing. The Fable-5 metadata identifies
+`saidutta69/fable-5-premium` as MIT-licensed. Other generated and teacher-source
+data may have separate terms.
+
+The XION 0.2 27B distribution license is not declared in this repository.
+Before publishing weights or datasets, review all upstream and source-data
+licenses and add the final XION license here.
+
+Relevant upstream resources:
+
+- [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B)
+- [SuperQwen3.8-27b-abliterated](https://huggingface.co/Jiunsong/SuperQwen3.8-27b-abliterated)
+- [Fable-5 Premium](https://huggingface.co/datasets/saidutta69/fable-5-premium)
+
+## Citation
+
+```bibtex
+@misc{xion-0.2-27b,
+  title  = {XION 0.2 27B},
+  author = {PIXELZX},
+  year   = {2026}
+}
+```