sensix-zo/sensix-paite-31b-base-16bit

Sensix Paite 31B Master CPT Foundation (16-bit)

This model represents the high-capacity linguistic foundation of the Sensix Paite AI project. Developed on the Gemma 4 31B Instruction-Tuned (IT) architecture, this model integrates native Paite linguistic capabilities with the advanced reasoning of the 31B parameter class.

Chat Capability and Instruction Following

Unlike standard Continued Pre-Training (CPT) models that are limited to text completion, this model inherits the instruction-following framework of the Gemma 4 IT series. It is capable of multi-turn dialogue and complex task execution in the Paite language. The CPT phase has successfully mapped Paite vocabulary onto the model's existing cognitive architecture, enabling fluent interaction without requiring immediate Supervised Fine-Tuning (SFT).

Dataset and Training Strategy

The 31B parameter count allows for a comprehensive data integration strategy. The training corpus includes:

• Structural Alignment: The Paite Bible and structured short-sentence datasets were utilized to establish grammatical and syntactical precision.
• Vocabulary Acquisition: The 2025-12-13 Vocabulary Update, consisting of targeted paragraphs, was used to introduce modern terminology and technical concepts.
• Logic Preservation: High-capacity LoRA adapters (Rank 128) were employed to ensure that the model's underlying reasoning capabilities remain intact while acquiring new linguistic data.

Technical Specifications

• Base Architecture: Gemma 4 31B-IT
• LoRA Configuration: Rank (r) 128, Alpha 128
• Target Modules: Comprehensive linear stack (q, k, v, o, gate, up, down_proj)
• Precision: bfloat16
• Context Window: 4096 Tokens
• Optimization: Blackwell-optimized packing and 8-bit AdamW

Implementation and Usage

This is an adapter-based model. It must be loaded with the corresponding base weights for inference.

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "sensix-zo/gemma-4-31b-paite-master-adapter",
    max_seq_length = 4096,
    dtype = torch.bfloat16,
    load_in_4bit = True,
)

messages = [
    {"role": "user", "content": "Paite pau in, 'The importance of education' chungtang thulim khat gelh in."}
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=1024)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Development Roadmap

While the current model demonstrates high proficiency in conversation and translation, a dedicated Supervised Fine-Tuning (SFT) phase is planned to further optimize conversational nuance and cultural alignment.