Mistral-7B-GPT-OSS-20B-Distilled

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1, distilled from openai/gpt-oss-20b using LoRA and SFTTrainer.

Training Details

• Teacher Model: openai/gpt-oss-20b
• Dataset: Synthetic dataset of 20 samples generated by the teacher.
• LoRA Config: r=16, alpha=32, dropout=0.05
• Training Hyperparams: 3 epochs, learning rate=0.0002, batch size=2

Usage

#pip install flash-attn --no-build-isolation -q
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load base + adapter (uses FP16 for efficiency)
base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    dtype=torch.float16,
    device_map="auto",
    attn_implementation="flash_attention_2"  # Optional: For speed
)
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1")
tokenizer.pad_token = tokenizer.eos_token

model = PeftModel.from_pretrained(base_model, "frankmorales2020/mistral-7b-gpt-oss-20b-distilled")

# Test prompt (from your dataset)
prompt = "### Instruction:\nWrite a Python function to compute Fibonacci numbers. Your solution must be memory-efficient and have a linear time complexity."

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=1024,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))