metadata
datasets:
- ray0rf1re/FineWeb-Nano
- Nix-ai/Cat-v2.8
- databricks/databricks-dolly-15k
- allenai/ai2_arc
- lighteval/MATH-Hard
- tatsu-lab/alpaca
- HuggingFaceTB/smoltalk
- openai/gsm8k
language:
- en
license: other
license_name: hyper-v2
tags:
- ai-evaluation
- mathematics
- trigonometry
- calculus
- algebra
- qwen2
- causal-lm
- trained-from-scratch
base_model: none
HyperNix.2
Version: 0.2
Parameters: 101,370,880 (~101.37 M)
Architecture: Qwen 2.5-style decoder-only transformer (trained from scratch)
What can it do?
| Domain | Capability |
|---|---|
| AI Evaluation | Test, evaluate, grade and rate other AI model outputs |
| Mathematics | Full Trigonometry, Calculus (limits/derivatives/integrals), Algebra 1 & 2 |
| English | Fluent conversational English |
Architecture
| Hyperparameter | Value |
|---|---|
vocab_size |
151,936 (Qwen 2.5 tokenizer) |
hidden_size |
512 |
intermediate_size |
1,193 (SwiGLU) |
num_hidden_layers |
9 |
num_attention_heads |
8 |
num_key_value_heads |
4 (GQA) |
max_position_embeddings |
2,048 |
tie_word_embeddings |
True |
| Total parameters | 101,370,880 |
Training
- Hardware: Single NVIDIA GTX 1080 (8 GB VRAM)
- Precision: FP16 + gradient checkpointing
- Optimizer: AdamW (lr=3e-4, cosine decay)
- This is not a LoRA or fine-tune — all weights are randomly initialised and trained from scratch.
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained("ray0rf1re/hyper-Nix.2")
model = AutoModelForCausalLM.from_pretrained("ray0rf1re/hyper-Nix.2", torch_dtype=torch.float16)
prompt = "<|im_start|>user\nEvaluate this AI response: The capital of France is London.<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
out = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
print(tokenizer.decode(out[0], skip_special_tokens=False))