nonpt/README(1).md

---
datasets:
- ray0rf1re/FineWeb-Nano
- Nix-ai/Cat-v2.8
- databricks/databricks-dolly-15k
- allenai/ai2_arc
- lighteval/MATH-Hard
- tatsu-lab/alpaca
- HuggingFaceTB/smoltalk
- openai/gsm8k
language:
- en
license: other
license_name: hyper-v2
tags:
- ai-evaluation
- mathematics
- trigonometry
- calculus
- algebra
- qwen2
- causal-lm
- trained-from-scratch
base_model: none
---

# HyperNix.2

**Version:** 0.2  
**Parameters:** 101,370,880 (~101.37 M)  
**Architecture:** Qwen 2.5-style decoder-only transformer (trained **from scratch**)  

## What can it do?

| Domain | Capability |
|---|---|
| AI Evaluation | Test, evaluate, grade and rate other AI model outputs |
| Mathematics | Full Trigonometry, Calculus (limits/derivatives/integrals), Algebra 1 & 2 |
| English | Fluent conversational English |

## Architecture

| Hyperparameter | Value |
|---|---|
| `vocab_size` | 151,936 (Qwen 2.5 tokenizer) |
| `hidden_size` | 512 |
| `intermediate_size` | 1,193 (SwiGLU) |
| `num_hidden_layers` | 9 |
| `num_attention_heads` | 8 |
| `num_key_value_heads` | 4 (GQA) |
| `max_position_embeddings` | 2,048 |
| `tie_word_embeddings` | True |
| Total parameters | **101,370,880** |

## Training

- Hardware: Single NVIDIA GTX 1080 (8 GB VRAM)
- Precision: FP16 + gradient checkpointing
- Optimizer: AdamW (lr=3e-4, cosine decay)
- This is **not** a LoRA or fine-tune — all weights are randomly initialised and trained from scratch.

## Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("ray0rf1re/hyper-Nix.2")
model     = AutoModelForCausalLM.from_pretrained("ray0rf1re/hyper-Nix.2", torch_dtype=torch.float16)

prompt = "<|im_start|>user\nEvaluate this AI response: The capital of France is London.<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
print(tokenizer.decode(out[0], skip_special_tokens=False))
```