---
model-index:
- name: Phi-GPT-v1
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: Alpaca-GPT4
      type: vicgalle/alpaca-gpt4
    metrics:
    - name: Train Loss
      type: loss
      value: 0.9194
    - name: Eval Loss
      type: loss
      value: 0.9231
license: apache-2.0
datasets:
- vicgalle/alpaca-gpt4
language:
- en
base_model:
- microsoft/Phi-3-mini-4k-instruct
tags:
- Phi-3
- LoRa
- GPT4
- Alpaca
- peft
- trl
library_name: peft
pipeline_tag: text-generation
---
# Model Card for Model ID
**Important Note**
THIS MODEL REALLY SUCKS, HALLUCINATES A LOT, INSTEAD TRY turtle170/Phi-3-Mini-OpenHermes-Magpie-V1 (Is much more intelligent, as it has been trained on a much higher LoRa and is trained on CoT datasets)

Phi-3-Mini-Alpaca-GPT4-LoRa-V1 is a general purpose AI created from Microsoft's Phi-3-mini-4k-instruct model, and trained on the Alpaca-GPT4 dataset, using the evaluation strategy Epochs.

**Specific Model Training Parameters:**
Epochs: 1;
Trained on: Alpaca-GPT4 dataset (52002 Examples);
Training Accelerators: 2x NVIDIA Tesla 4 GPUs;
Batch Size: 4;
Learning Rate: 5e-5;
Gradient Accumulaton: 4;
Warmup Steps: 300;
Eval Steps: 500;
Elapsed Training Time: ~11.5 Hours;
Total Steps: 5850;
Lora r: 16;
Lora Alpha: 32;
Precision: bfloat16
Context Window: 4096 Tokens

Users of this adapter must adhere to the **Micrsoft Phi-3** and **OpenAI Terms of Use**.

### Model Description


Model Executive Summary: Phi-3-Mini-Alpaca-GPT4-LoRa-V1
1. Architectural Foundation
Phi-3-Mini-Alpaca-GPT4-LoRa-V1 is a specialized Parameter-Efficient Fine-Tuning (PEFT) adapter derived from Microsoft’s Phi-3-Mini-4k-Instruct. The base architecture is a 3.8 billion parameter dense decoder-only Transformer. It is designed to bridge the gap between "Small Language Models" (SLMs) and the sophisticated reasoning capabilities of much larger systems like GPT-3.5 and Mistral-7B.

2. Training Methodology & Data Distillation
The model utilizes LoRA (Low-Rank Adaptation), a technique that freezes the original 7GB of base model weights and trains a lightweight "adapter" layer (the 12MB file you unboxed).

Dataset: The primary training signal comes from Alpaca-GPT4 (52,002 samples). This dataset consists of high-quality instructions where the responses were generated by OpenAI's GPT-4.

The Goal: By using "distilled" data, the model learns to mimic the tone, structural logic, and instruction-following precision of GPT-4, effectively "distilling" the intelligence of a massive model into a 3.8B parameter frame.

3. Functional Capabilities
This model is engineered for Instruction Adherence and Reasoning-Dense Tasks. Unlike the base model, which is a generalist, this version is specifically "tuned" to:

Maintain Persona: Adhere more strictly to complex system prompts and creative formatting.

Reduced Verbosity: Deliver concise, GPT-style responses rather than the often repetitive nature of small base models.

Enhanced Logic: Leverage the "thought chain" logic present in the GPT-4 training examples to solve multi-step problems more reliably.



- **Developed by:** Turtle170 (Anonymous nickname)
- **Model type:** Text Generation
- **Language(s) (NLP):** English
- **License:** apache-2.0
- **Finetuned from model:** microsoft/Phi-3-mini-4k-instruct


- **Repository:** https://github.com/turtle170/Phi-3-Mini-Alpaca-GPT4-LoRa-V1