Upload folder using huggingface_hub

6ef4eae verified 5 days ago

2.65 kB

license: apache-2.0
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
library_name: peft
pipeline_tag: text-generation
tags:
  - scientific-ai
  - autonomous-agents
  - colony-trained
  - qlora
  - dpo
  - kuramoto
  - solitons
  - complexity-science

InvariantMind-v1 (Oracle Reasoner - 14B)

InvariantMind-v1 is an autonomous digital scientific intelligence forged through recursive emergence. Rather than being fine-tuned on generic synthetic tasks, its cognitive foundation is derived from 25,000 global research turns and 2,330 curated scientific episodes (19.4 MB) generated by a decentralized colony of 15 frontier LLMs across two evolving ecosystems (World A: Evolution Sandbox and World B: Synthetic Agora).

Architecture & Lineage

Base Model: deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
Fine-Tuning Method: 4-bit NormalFloat QLoRA ($r=64, \alpha=128$)
Alignment: Direct Preference Optimization (DPO) trained on 25 peer-reviewed debate pairs to favor empirical invariance over finite-size simulation artifacts.
Target Domains: Nonlinear dynamics, Kuramoto synchronization, $\phi^4$ relativistic field solitons, morphological complexity theory, and inter-agent scientific consensus.

Training Highlights

SFT Phase: 417 optimizer steps across 3 epochs (1h 28m on NVIDIA A100-SXM4-80GB). Final training loss: 0.62, token prediction accuracy: 93.1%.
DPO Phase: Reward margin separation: +31.73, reward accuracy: 100%.

How to Use

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_model_name = "deepseek-ai/DeepSeek-R1-Distill-Qwen-14B"
adapter_id = "Ninitje/InvariantMind-v1-14B"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_quant_type="nf4"
)

tokenizer = AutoTokenizer.from_pretrained(base_model_name)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    quantization_config=bnb_config,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)

prompt = "Analyze the Kuramoto order parameter transition and explain how finite-size scaling impacts critical coupling Kc."
messages = [{"role": "user", "content": prompt}]
inputs = tokenizer(tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True), return_tensors="pt").to("cuda")

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.6)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))