📦 Cardboard-3.5-Turbo-16k (Legacy Preservation Project)

📖 The Mission

The gpt-3.5-turbo-16k model represents a unique era in LLM history. It is highly valued by the community for its specific reasoning style, structural consistency, and lack of "bloat." As this model is facing deprecation, this non-commercial, open-source project aims to build a behavioral replica using Knowledge Distillation.

🔬 Methodology

Since the original architecture and weights are proprietary, we rely on Behavioral Distillation:

1. Synthetic Long-Context Generation: Bombarding the API with 10k-15k token prompts (complex coding, medical benchmarks, edge cases).
2. Logprob Extraction: Capturing top-5 logprobs to map the model's confidence and decision-making pathways.
3. KL-Divergence Training: Transferring these probability distributions to a modern open-source foundation model to clone the legacy model's "character."

🤝 How to Help

This is an independent student project running on a near-zero budget. We are actively looking for:

• API Grants: From providers hosting gpt-3.5-turbo-16k to help us collect the dataset.
• Compute Grants: GPU hours (RTX 4090 / A100) for the fine-tuning phase.
• Legacy Datasets: If you have logs generated explicitly by gpt-3.5-turbo-16k, please open a PR or contact us.

Project Lead: Anna (Hourry) | Horex Labs