---
license: apache-2.0
library_name: transformers
---

# Spotlight VM

A small, hand-constructed [Spotlight](https://percepta.ai/blog/spotlight-memory) transformer that runs Python:
the examples from [Can LLMs Grow Their Own Capabilities?](https://percepta.ai/blog/can-llms-grow-their-own-capabilities).
`intelligence/` holds the weights (under 100K parameters), which never change. `memory/` holds MicroPython
v1.28.0 (`core.safetensors`) and two packages.

```sh
cc -O3 -std=c99 spotlight.c -lm -o spotlight
./spotlight < examples/euler1.py            # 233168; -v also prints every generated token
cat examples/tax_brackets/*.py | ./spotlight
```

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("percepta-ai/spotlight-vm", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("percepta-ai/spotlight-vm", trust_remote_code=True)
output = model.generate(**tokenizer("print(1 + 2)\n", return_tensors="pt"))
print(tokenizer.decode(output[0]))  # 3
```

In a local copy of this folder, `from_pretrained(".")` works the same way.

| Example | In the post | Output |
|---|---|---|
| `examples/euler1.py`, `euler2.py`, `euler3.py` | Project Euler problems 1 to 3 | `233168`, `4613732`, `6857` |
| `examples/memory_loader.py` | Loading memory selectively | `4` |
| `examples/tax_brackets/` | Acquiring and updating knowledge | `$17400`, then `$16914` |
| `examples/mnist_digit.py` | Growing capabilities and intelligence | `7` |

It runs on one CPU core at about 120K tokens per second: minutes for the Project Euler examples,
about half an hour for MNIST (245M tokens). This MicroPython build has no floating point. License: Apache 2.0,
see [LICENSE.md](LICENSE.md).