Gro97/simplicial-homology-tasks-llama4-scout-lora

Model Card: Directed Simplicial Homology Reasoning (LoRA Adapter)

Model Details

Model Description

This is a LoRA adapter fine-tuned on top of Llama-4-Scout-17B-16E-Instruct to reason about directed simplicial complexes — a generalization of graphs to higher-dimensional, orientation-aware combinatorial structures used in topological data analysis (TDA). The adapter was trained to improve four related reasoning tasks: facet validity checking, multi-hop orientation-sign composition, boundary matrix rank computation, and Betti number computation via the rank-nullity theorem.

Every training label was generated deterministically via exact linear algebra (NumPy), not hand-annotated, so ground-truth correctness is guaranteed by construction. Full dataset and generator code are published alongside this model — see Training Data below.

Submitted to the AutoScientist Challenge (Math & Code track) by Adaption Labs.

• Developed by: Grold Otieno Mboya
• Shared by: Grold Otieno Mboya
• Model type: LoRA adapter for causal language modeling (mathematical/ topological reasoning)
• Language(s): English
• License: Llama 4 Community License Agreement (inherited from the base model — see Meta's license terms for Llama-4-Scout-17B-16E-Instruct)
• Finetuned from model: meta-llama/Llama-4-Scout-17B-16E-Instruct (loaded here via the 4-bit quantized variant togethercomputer/Llama-4-Scout-17B-16E-Instruct_bnb_4bit)

Model Sources

• Challenge: AutoScientist Challenge, Adaption Labs

Uses

Direct Use

Intended for research and benchmarking on symbolic/topological reasoning: checking facet validity in a directed simplicial complex, computing incidence signs, computing boundary matrix rank, and computing Betti numbers. Useful as a research artifact demonstrating that targeted, deterministically-verified synthetic curricula can measurably improve a frontier model's structural reasoning on held-out math tasks.

Downstream Use

Could serve as a starting point for further fine-tuning on adjacent topological data analysis (TDA) tasks, or as a component in a larger pipeline that requires verifying combinatorial/topological structure (e.g., persistent homology computation, directed graph/hypergraph reasoning tools).

Out-of-Scope Use

Not intended for general-purpose conversation, safety-critical decision making, or any domain outside symbolic/mathematical reasoning over simplicial complexes. Not evaluated for general capability, factuality, or safety beyond the Math task category described here — no claims are made about performance outside that scope.

Bias, Risks, and Limitations

The author-designed portion of the training data (see dataset card) is purely synthetic, symbolic mathematics with no demographic, opinion, or natural-language-bias content. However, ~28,000 of the total training rows came from Adaption's platform-side domain/general-purpose data augmentation, whose content was not independently authored or fully audited by the model developer. The model also inherits any biases or limitations present in the Llama-4-Scout-17B-16E-Instruct base model.

Broad Math-category generalization plateaued around 63-65% win rate across every training variant tested (see Evaluation below and the dataset card's ablation notes) — further gains from task-mix or epoch tuning alone appear limited; this should be read as an honest ceiling of the current approach, not a guaranteed floor.

Recommendations

Users should independently verify outputs on tasks outside the four described task types, and should not assume performance on the broader "Math" category generalizes to arbitrary mathematical domains beyond combinatorics/topology-flavored reasoning.

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

base_model_id = "meta-llama/Llama-4-Scout-17B-16E-Instruct"
adapter_id = "<your-hf-username>/<this-model-repo-name>"

bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id, quantization_config=bnb_config, device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)

prompt = "Simplex: [2,7,9]\nCandidate facet: [7,9]\nIs this a valid facet? Give the incidence sign."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training Details

Training Data

Custom-built dataset: 3,000 training examples (plus 300 validation / 300 held-out test) covering four task types (VALIDITY, FLOW_SIGN, RANK, BETTI) over procedurally-generated directed simplicial complexes, mixed 52%/19%/15%/14% respectively. Expanded to ~28,000 total training rows via Adaption's platform-side domain/general-purpose augmentation. Full detail, generator script, and ablation results are in the accompanying dataset card (DATASET_CARD.md) and dataset repository.

Training Procedure

Fine-tuned via Adaption Labs' AutoScientist platform (supervised fine-tuning with LoRA), on 4x NVIDIA H100 80GB for both training and evaluation.

Training Hyperparameters

• Training regime: 4-bit quantized base model (bnb_4bit) with LoRA adapters trained in higher precision (QLoRA-style)
• LoRA rank (r): 64
• LoRA alpha: 128
• LoRA dropout: 0
• Target modules: k_proj, o_proj, q_proj, v_proj, shared_expert.gate_proj, shared_expert.up_proj, shared_expert.down_proj, feed_forward.gate_proj, feed_forward.up_proj, feed_forward.down_proj
• Epochs: 7
• Batch size: max (platform-determined)
• Learning rate: 1e-4
• LR scheduler: cosine, min_lr_ratio 0.1, num_cycles 0.5
• Warmup ratio: 0.05
• Weight decay: 0.03
• Max gradient norm: 1
• Training method: SFT (supervised fine-tuning), train_on_inputs: false

Speeds, Sizes, Times

Not independently benchmarked by the author; training was run on Adaption Labs' hosted infrastructure (4x NVIDIA H100 80GB reported for both the training and evaluation stages).

Evaluation

Testing Data, Factors & Metrics

Testing Data

Two held-out evaluation sets: (1) 200 held-out samples from this project's own dataset, (2) 100 unseen samples drawn from Adaption's broader internal Math task category.

Metrics

LLM-as-judge pairwise preference win rate (judge model: Gemini 3.1 Pro), comparing this adapter's outputs against the un-adapted base model's outputs on the same held-out prompts.

Results

Evaluation set	Base model win rate	Adapted model win rate
This dataset's held-out samples (n=200)	23%	77%
Broader Math task category (n=100)	36%	64%

Both improvements are statistically significant relative to a 50/50 null (p < 0.01 at these sample sizes). See the dataset card for the ablation history (task-mix and epoch-count variants tested) that led to this configuration.

Summary

The adapter shows a large, statistically significant improvement over the base model on its own training distribution (77% win rate) and a smaller but still significant and non-trivial improvement on unseen tasks from the broader Math category (64% win rate). The gap between these two numbers suggests some of the own-dataset gain reflects distribution-specific fit rather than fully general reasoning transfer — disclosed here for transparency rather than presented as a single headline number.

Environmental Impact

• Hardware Type: 4x NVIDIA H100 80GB (training and evaluation)
• Hours used: Not disclosed by the training platform
• Cloud Provider: Adaption Labs' hosted infrastructure
• Compute Region: Not disclosed
• Carbon Emitted: Not calculated

Technical Specifications

Model Architecture and Objective

LoRA adapter (rank 64, alpha 128) applied to attention and feed-forward/ shared-expert projection layers of Llama-4-Scout-17B-16E-Instruct (a Mixture-of-Experts architecture), trained via supervised fine-tuning to improve structural/topological mathematical reasoning.

Compute Infrastructure

Hardware

4x NVIDIA H100 80GB

Software

• PEFT 0.15.1
• Trained via Adaption Labs' AutoScientist platform

Citation

BibTeX:

@misc{mboya2026simplicialhomology,
  author = {Mboya, Grold Otieno},
  title = {Directed Simplicial Homology Reasoning: A LoRA Adapter for
           Topological Structure Reasoning},
  year = {2026},
  howpublished = {AutoScientist Challenge submission, Adaption Labs},
  url = {<[simplicial-homology-tasks-llama3-3-70b-lora](https://huggingface.co/Gro97/simplicial-homology-tasks-llama3-3-70b-lora)>}
}

Model Card Contact

See author's Hugging Face profile.

Framework versions

• PEFT 0.15.1