| --- |
| license: mit |
| language: |
| - en |
| - vi |
| pipeline_tag: text-classification |
| tags: |
| - ai |
| - vietnamese |
| - shop |
| - support |
| - sales |
| - marketing |
| --- |
| |
| # Model Card for Vian ai Fine-Tuned |
|
|
| ## Model Details |
|
|
| ### Model Description |
|
|
| - **Developed by:** Glavinnguyen |
| - **Model type:** Text Generation / Multimodal (Text/Vision/Audio Interface) |
| - **Language(s) (NLP):** Vietnamese (Primary), English (Primary) |
| - **License:** mit |
| - **Finetuned from model:** google/gemma-4-e2b-it |
|
|
| ### Model Sources |
|
|
| - **Repository:** [More Information Needed] |
|
|
| ## Uses |
|
|
| ### Direct Use |
|
|
| This model is specifically fine-tuned on a small-scale dataset to standardize response formatting, style, and structure. It natively retains the advanced deep reasoning mechanisms (`<|think|>`) and inherent logical problem-solving capabilities of the base Gemma 4 architecture. |
|
|
| ### Out-of-Scope Use |
|
|
| The model should not be used for tasks requiring extensive broad-domain knowledge expansion outside the scope of the training dataset without human supervision. The fine-tuning process was focused on structuring behavioral output rather than massive knowledge injection. |
|
|
| ## Training Details |
|
|
| ### Training Data |
|
|
| The model was trained on a highly curated, high-quality alignment dataset. |
|
|
| ### Training Procedure |
|
|
| Training was conducted utilizing the **QLoRA (Quantized Low-Rank Adaptation)** method to minimize hardware resource consumption while aggressively preserving the base model's pre-trained weights. |
|
|
| #### Training Hyperparameters |
|
|
| The hyperparameters were carefully optimized for an ultra-small dataset to prevent catastrophic overfitting and achieve an ideal convergence point: |
|
|
| - **Training regime:** QLoRA (FP16/BF16 Mixed Precision) |
| - **Learning Rate:** 2e-4 to 1e-5 (Low Learning Rate) |
| - **Per Device Train Batch Size:** 1 |
| - **Gradient Accumulation Steps:** 32 (Global Batch Size = 32) |
| - **Number of Train Epochs:** 1 to 2 |
| - **Max Length:** 256 - 2048 tokens (Allocated to safeguard the generation of `<|think|>` tokens) |
| - **Optimizer:** AdamW |
| - **LR Scheduler Type:** Cosine / Constant |
| - **LoRA Rank (r):** 4 or 8 |
| - **LoRA Alpha ($\alpha$):** 8 or 16 |
|
|
| #### Speeds, Sizes, Times |
|
|
| - **Final Training Loss:** `0.69` (The sweet spot convergence for low-sample fine-tuning—balancing structural alignment with base intelligence retention). |
| - **VRAM Consumption:** ~2.2 GB (When running on a 4-bit Q4_0 execution profile). |
| |
| ## Technical Specifications |
| |
| ### Model Architecture and Objective |
| |
| Built upon Google's next-generation **Gemma 4** architecture, featuring integrated **Quantization Aware Training (QAT)** and an intrinsic multi-token prediction (MTP) engine. The model leverages an internal step-by-step reasoning loop before routing structural text outputs via designated generation tags. |
| |
| ## Model Card Contact |
| |
| - **Contact:** Glavinnguyen |
| ### Framework versions |
| |
| - PEFT 0.19.1 |
