Upload AtomSLM checkpoint
Browse files
README.md
CHANGED
|
@@ -1,5 +1,6 @@
|
|
| 1 |
---
|
| 2 |
library_name: pytorch
|
|
|
|
| 3 |
datasets:
|
| 4 |
- roneneldan/TinyStories
|
| 5 |
- Salesforce/wikitext
|
|
@@ -25,7 +26,7 @@ FiLM vectors providing the only per-layer state — deep reasoning at minimal pa
|
|
| 25 |
| Architecture | AtomNet (shared-core + FiLM) |
|
| 26 |
| Parameters | 1.094M |
|
| 27 |
| Vocab size | 4096 |
|
| 28 |
-
|
|
| 29 |
| Layers | 10 |
|
| 30 |
| FFN multiplier | 2.0 |
|
| 31 |
| Context window | 256 tokens |
|
|
@@ -91,4 +92,4 @@ model.eval()
|
|
| 91 |
|
| 92 |
## License
|
| 93 |
|
| 94 |
-
|
|
|
|
| 1 |
---
|
| 2 |
library_name: pytorch
|
| 3 |
+
license: apache-2.0
|
| 4 |
datasets:
|
| 5 |
- roneneldan/TinyStories
|
| 6 |
- Salesforce/wikitext
|
|
|
|
| 26 |
| Architecture | AtomNet (shared-core + FiLM) |
|
| 27 |
| Parameters | 1.094M |
|
| 28 |
| Vocab size | 4096 |
|
| 29 |
+
| d_model | 192 |
|
| 30 |
| Layers | 10 |
|
| 31 |
| FFN multiplier | 2.0 |
|
| 32 |
| Context window | 256 tokens |
|
|
|
|
| 92 |
|
| 93 |
## License
|
| 94 |
|
| 95 |
+
Apache 2.0
|