atrost/climbmix-stairformer-353m-extracted-nested-94m-1p2b-h100

This is the extracted nested/asymmetric submodel from atrost/climbmix-stairformer-353m-1p2b-h100. It preserves the full-width token embedding and dense entry layer, then runs the prefix-width nested stack and prefix-width untied LM head copied from the larger StairFormer checkpoint.

• Parameters: 94,332,000
• Source checkpoint: atrost/climbmix-stairformer-353m-1p2b-h100 Source checkpoint revision: 4de1d6dfc052d2fe944485d68a6db0588dd70acc.

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "atrost/climbmix-stairformer-353m-extracted-nested-94m-1p2b-h100"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    trust_remote_code=True,
    torch_dtype="auto",
)

trust_remote_code=True is required for the asymmetric nested architecture.

Equivalence

For full-sequence forward passes with use_cache=False, this checkpoint's logits match output_nested_logits=True from atrost/climbmix-stairformer-353m-1p2b-h100 up to normal floating-point precision differences.