Upload AtomSLM checkpoint

README.md

@@ -1,3 +1,72 @@
----
-license: apache-2.0
----
+---
+library_name: pytorch
+tags:
+  - atomslm
+  - language-model
+  - shared-weights
+---
+
+# AtomSLM-600K
+
+**AtomSLM** is a compact, shared-weight language model family built on the
+AtomNet architecture (Shared Weight Core + Per-Layer FiLM Modulation).
+
+## Model Details
+
+| Field         | Value                     |
+|---------------|---------------------------|
+| Architecture  | AtomNet (shared-core FiLM)|
+| Parameters    | 0.613M              |
+| Vocab size    | 4096               |
+| d_model       | 120                  |
+| Layers        | 8                 |
+| FFN mult      | 2.0               |
+| Max seq len   | 256              |
+| Weight tying  | True            |
+| Val loss      | 3.3455                |
+| Trained steps | 4900                   |
+
+## Training
+
+Trained on **TinyStories + WikiText-2** with a custom BPE tokenizer.
+
+### Hyperparameters
+
+```json
+{
+  "data_dir": "data/processed",
+  "save_dir": "runs/AtomSLM-600K",
+  "config": "AtomSLM-600K",
+  "steps": 5000,
+  "eval_every": 100,
+  "save_every": 500,
+  "batch_size": 32,
+  "seq_len": 256,
+  "lr": 0.0005,
+  "lr_min": 5e-05,
+  "warmup": 1000,
+  "grad_clip": 1.0,
+  "dropout": 0.1,
+  "device": "auto",
+  "resume": null,
+  "compile": false,
+  "amp": false,
+  "core_warmup_steps": 0
+}
+```
+
+## Usage
+
+```python
+import torch
+from models.atomgpt import AtomSLM
+
+ckpt  = torch.load('pytorch_model.bin', map_location='cpu')
+model = AtomSLM(ckpt['config'])
+model.load_state_dict(ckpt['model_state'])
+model.eval()
+```
+
+## License
+
+MIT

config.json

@@ -0,0 +1,12 @@
+{
+  "vocab_size": 4096,
+  "d_model": 120,
+  "n_layers": 8,
+  "ffn_mult": 2.0,
+  "dropout": 0.1,
+  "max_seq_len": 256,
+  "tie_weights": true,
+  "architecture": "AtomSLM",
+  "val_loss": 3.345488176345825,
+  "step": 4900
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3eca566265982e026c15d839fe2692501945ff23d2dfb3d5adaa2d445424415f
+size 2469423