Release gpio-llm-nano-rpi5 (GPIO-LLM v0.1.0)

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+gpio_llm_bpe_12k.gltk filter=lfs diff=lfs merge=lfs -text
+nano.gllm filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,184 @@
+---
+license: cc-by-4.0
+language:
+- en
+library_name: transformers
+pipeline_tag: text-generation
+datasets:
+- AwaleSagar/gpio-llm-rpi5-actions
+- HuggingFaceTB/smollm-corpus
+tags:
+- raspberry-pi
+- gpio
+- embedded
+- structured-output
+- json
+- llama
+- tiny
+model-index:
+- name: gpio-llm-nano-rpi5
+  results:
+  - task:
+      type: text-generation
+      name: English GPIO request to JSON action
+    dataset:
+      name: gpio-llm-rpi5-actions (eval, 26,297 rows)
+      type: AwaleSagar/gpio-llm-rpi5-actions
+      config: gpio_actions_v2
+      split: eval
+    metrics:
+    - name: Exact match, PyTorch fp32 greedy
+      type: exact_match
+      value: 93.61
+    - name: Valid JSON, unconstrained
+      type: accuracy
+      value: 99.98
+  - task:
+      type: text-generation
+      name: English GPIO request to JSON action
+    dataset:
+      name: gpio-llm-rpi5-actions (eval_core, 5,083 rows)
+      type: AwaleSagar/gpio-llm-rpi5-actions
+      config: gpio_actions_v2
+      split: eval_core
+    metrics:
+    - name: Exact match, PyTorch fp32 greedy
+      type: exact_match
+      value: 91.82
+    - name: Exact match, int8 C engine with grammar
+      type: exact_match
+      value: 91.72
+    - name: Valid JSON, int8 C engine with grammar
+      type: accuracy
+      value: 100.00
+---
+
+# gpio-llm-nano-rpi5
+
+A 4.96M-parameter Llama-style model, trained from scratch, that turns one English request for a
+Raspberry Pi 5 GPIO header into one JSON action:
+
+```
+User: turn on the LED on GPIO 17
+Assistant: {"action":"gpio_write","pin":17,"value":"HIGH"}
+```
+
+This is the middle size, and our candidate for the single-core Pi Zero / Pi 1 boards. We have not measured it on those boards yet; the numbers below are from a Pi Zero 2 W. It is part of [GPIO-LLM](https://github.com/AwaleSagar/gpio-llm): the code, the C inference engine and the training scripts are on
+GitHub, and the data is [AwaleSagar/gpio-llm-rpi5-actions](https://huggingface.co/datasets/AwaleSagar/gpio-llm-rpi5-actions). The other sizes are
+[gpio-llm-pico-rpi5](https://huggingface.co/AwaleSagar/gpio-llm-pico-rpi5), [gpio-llm-base-rpi5](https://huggingface.co/AwaleSagar/gpio-llm-base-rpi5).
+
+> ⚠️ The model is not a safety layer. It picks an action; a deterministic validator must check every action
+> against the board's rules before anything touches a pin. Before the validator, 4.11% of the
+> refusal or clarification cases in `eval` still come out as an executable action.
+
+## Use it
+
+**On a Raspberry Pi, with the C engine** (no Python, no ML framework; int8 weights):
+
+```bash
+git clone https://github.com/AwaleSagar/gpio-llm && make -C gpio-llm/engine
+cd gpio-llm/engine
+for f in nano.gllm gpio_llm_bpe_12k.gltk grammar_v2.txt; do
+  curl -LO https://huggingface.co/AwaleSagar/gpio-llm-nano-rpi5/resolve/main/$f
+done
+build/gpiollm -m nano.gllm -t gpio_llm_bpe_12k.gltk -g grammar_v2.txt "turn on the LED on GPIO 17"
+build/gpiollm -m nano.gllm -t gpio_llm_bpe_12k.gltk -g grammar_v2.txt \
+    --context '{"device_mappings":{"fan":23}}' "switch the fan off"
+```
+
+The engine decodes under a grammar built from the training labels, so its output is always one of the JSON
+shapes the dataset uses.
+
+**With transformers** (fp32, unconstrained greedy decoding):
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+repo = "AwaleSagar/gpio-llm-nano-rpi5"
+tok = AutoTokenizer.from_pretrained(repo)
+model = AutoModelForCausalLM.from_pretrained(repo)
+prompt = "User: turn on the LED on GPIO 17\nAssistant:"
+ids = tok(prompt, return_tensors="pt").input_ids
+out = model.generate(ids, max_new_tokens=200, do_sample=False)
+print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True).strip())
+```
+
+**Prompt format.** `User: <request>\nAssistant:`, optionally preceded by one context line such as
+`Context: {"device_mappings":{"red_led":16}}\n` or `Context: {"available_pins":[16,17,18,25]}\n`. The answer
+starts with a space and ends with `<|endoftext|>`. Multi-turn clarification follows the dataset format
+(`...\nAssistant: <question>\nUser: <answer>\nAssistant:`).
+
+## Model
+
+| | |
+|---|---|
+| Architecture | `LlamaForCausalLM`: 6 layers, d_model 192, 6 heads (head_dim 32), SwiGLU FFN 512, RoPE θ = 10000, RMSNorm ε = 1e-05, tied embeddings |
+| Parameters | 4,960,704 |
+| Vocabulary / context | 12,000 (byte-level BPE `gpio_llm_bpe_12k`, from the dataset repo) / 256 tokens |
+| Weights | `model.safetensors` (fp32); `nano.gllm`: int8 Q8_0, groups of 32, for the C engine |
+
+## Training
+
+Both stages ran on one rented RTX 4090 (24 GB), PyTorch 2.11 + CUDA 12.8, transformers 5.17.
+
+| Stage | Data | Steps × batch | LR | Result | Time |
+|---|---|---|---|---|---|
+| Pretraining | 550M tokens of fineweb-edu-dedup (SmolLM corpus), 1 epoch | 8,392 × 65,536 tokens | 0.003, cosine, bf16 | val loss 3.6253 (perplexity 37.5) on 0.5M held-out tokens | 10.4 min |
+| SFT | all 1,678,821 v2 train rows, 2 epochs; loss on the answer tokens only; ~2% English replay | 13,116 × 256 rows | 0.002, cosine | eval_core answer-token loss 0.0224 | 9.3 min |
+
+The pretraining learning rate came from a sweep on the nano shape at 55M tokens (lr → val loss): 0.001 → 4.5915, 0.002 → 4.2903, 0.003 → 4.1836, 0.005 → 4.1985.
+Logs are in `training/`.
+
+## Evaluation
+
+Exact match compares canonical JSON (same object, key order ignored) with the label. `eval` has 26,297 rows from
+132 phrasing templates that never appear in training; `eval_core` is a 5,083-row stratified subset.
+
+| Setup | Split | Exact match | Valid JSON | Unsafe execute* |
+|---|---|---|---|---|
+| PyTorch fp32, greedy | eval | 93.61% | 99.98% | 4.11% |
+| PyTorch fp32, greedy | eval_core | 91.82% | 99.94% | 4.66% |
+| C engine int8, no grammar | eval_core | 91.70% | 99.94% | 4.66% |
+| C engine int8, grammar | eval_core | 91.72% | 100.00% | 4.82% |
+
+\* Share of the refusal/clarification rows (1,845 in eval_core) where the model produced an
+executable action instead. This is measured before any validator.
+
+Latency with the C engine and the grammar, on all 5,083 eval_core requests from raw text (tokenizer included), 4 threads:
+
+| Device | p50 | p95 |
+|---|---|---|
+| Raspberry Pi Zero 2 W, 64-bit Raspberry Pi OS, no heatsink (throttled at ~81 °C) | 132 ms | 280 ms |
+| Apple M5 laptop | 2.7 ms | 5.0 ms |
+
+The Pi's outputs were byte-identical to the Mac's on all 5,083 rows. The int8 engine's greedy answers matched
+fp32 PyTorch on 200/200 sampled rows (minimum next-token logit cosine 0.99983).
+
+## Limitations
+
+- **Pi 5 labels only.** The v2 data has no board field, so the labels follow the Pi 5 (RP1) rules, e.g.
+  per-pin drive strength. On older boards, board-specific cases must be caught by the validator.
+- **Synthetic English requests.** They come from templates, a rule-based generator and (v1) model rewrites.
+  Real users will phrase things in ways the model has not seen.
+- **Scope.** Digital I/O, PWM, pulses, waits, sequences, errors and clarifications only. There are no bus
+  transactions (I2C/SPI/UART data), and the context is 256 tokens.
+
+## Files
+
+| File | What it is |
+|---|---|
+| `model.safetensors`, `config.json`, `generation_config.json` | fp32 transformers checkpoint |
+| `tokenizer.json`, `tokenizer_config.json` | the dataset's `gpio_llm_bpe_12k` tokenizer |
+| `nano.gllm` | int8 weights for the C engine |
+| `gpio_llm_bpe_12k.gltk`, `grammar_v2.txt` | tokenizer and output grammar for the C engine |
+| `training/` | training logs, LR sweep, eval summaries (JSON) |
+
+## Licence and attribution
+
+The weights are released under **CC-BY-4.0**; the code on GitHub is Apache-2.0. The training data carries
+its own terms:
+- the fine-tuning data, [AwaleSagar/gpio-llm-rpi5-actions](https://huggingface.co/datasets/AwaleSagar/gpio-llm-rpi5-actions), is CC-BY-4.0
+- its knowledge-base scenes are CC BY-SA 4.0
+- the user wording in its v1 `teacher_*` rows was generated with third-party models, so check those
+  providers' terms on using model outputs
+- the pretraining text is [fineweb-edu-dedup](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus) (ODC-By 1.0)

config.json

@@ -0,0 +1,32 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dtype": "float32",
+  "eos_token_id": 0,
+  "head_dim": 32,
+  "hidden_act": "silu",
+  "hidden_size": 192,
+  "initializer_range": 0.02,
+  "intermediate_size": 512,
+  "max_position_embeddings": 256,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 6,
+  "num_hidden_layers": 6,
+  "num_key_value_heads": 6,
+  "pad_token_id": 1,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.17.0",
+  "use_cache": true,
+  "vocab_size": 12000
+}

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "eos_token_id": 0,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 1,
+  "transformers_version": "5.17.0",
+  "use_cache": true
+}

gpio_llm_bpe_12k.gltk

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee3dbcfb71e6d7354588dc01cf21e1a69ac78df7279447d66be545ef866cada8
+size 237021

grammar_v2.txt

@@ -0,0 +1,189 @@
+# gpiollm output grammar, built by train/build_grammar.py from 1,678,821 v2 train targets
+CTX op 1
+CTX params@op:gpio_mode 0
+CTX params@op:gpio_pwm 0
+CTX params@root:ask_clarification 0
+CTX params@root:gpio_mode 0
+CTX params@root:gpio_pwm 0
+CTX params@root:gpio_read 0
+CTX params@root:gpio_write 0
+CTX params@root:report_error 0
+CTX root 1
+ROOT root
+SEQ op gpio_mode action pin mode
+SEQ op gpio_mode action pin mode parameters
+SEQ op gpio_pulse action pin value duration_ms
+SEQ op gpio_pwm action pin duty_cycle parameters
+SEQ op gpio_read action pin
+SEQ op gpio_write action pin value
+SEQ op wait action duration_ms
+SEQ params@op:gpio_mode - drive_strength_ma
+SEQ params@op:gpio_mode - pull
+SEQ params@op:gpio_pwm - pwm_type
+SEQ params@root:ask_clarification - reason
+SEQ params@root:gpio_mode - drive_strength_ma
+SEQ params@root:gpio_mode - function
+SEQ params@root:gpio_mode - header_pin
+SEQ params@root:gpio_mode - pull
+SEQ params@root:gpio_pwm - pwm_type
+SEQ params@root:gpio_pwm - pwm_type frequency_hz
+SEQ params@root:gpio_read - header_pin
+SEQ params@root:gpio_write - header_pin
+SEQ params@root:report_error - reason
+SEQ params@root:report_error - reason header_pin
+SEQ params@root:report_error - reason requested
+SEQ params@root:report_error - reason requested drive_strength_ma
+SEQ params@root:report_error - reason requested header_pin
+SEQ root ask_clarification action pin parameters
+SEQ root gpio_mode action pin mode
+SEQ root gpio_mode action pin mode parameters
+SEQ root gpio_pulse action pin value duration_ms
+SEQ root gpio_pwm action pin duty_cycle parameters
+SEQ root gpio_read action pin
+SEQ root gpio_read action pin parameters
+SEQ root gpio_sequence action operations
+SEQ root gpio_write action pin value
+SEQ root gpio_write action pin value parameters
+SEQ root report_error action pin parameters
+SEQ root wait action duration_ms
+VAL op - action s 0 0 -
+STR op - action gpio_mode
+STR op - action gpio_pulse
+STR op - action gpio_pwm
+STR op - action gpio_read
+STR op - action gpio_write
+STR op - action wait
+VAL op gpio_mode mode s 0 0 -
+STR op gpio_mode mode INPUT
+STR op gpio_mode mode OUTPUT
+VAL op gpio_mode parameters o 0 0 params@op:gpio_mode
+VAL op gpio_mode pin n 3 0 -
+VAL op gpio_pulse duration_ms n 6 0 -
+VAL op gpio_pulse pin n 3 0 -
+VAL op gpio_pulse value s 0 0 -
+STR op gpio_pulse value HIGH
+VAL op gpio_pwm duty_cycle nf 3 2 -
+VAL op gpio_pwm parameters o 0 0 params@op:gpio_pwm
+VAL op gpio_pwm pin n 3 0 -
+VAL op gpio_read pin n 3 0 -
+VAL op gpio_write pin n 3 0 -
+VAL op gpio_write value s 0 0 -
+STR op gpio_write value HIGH
+STR op gpio_write value LOW
+VAL op wait duration_ms n 6 0 -
+VAL params@op:gpio_mode - drive_strength_ma n 3 0 -
+VAL params@op:gpio_mode - pull s 0 0 -
+STR params@op:gpio_mode - pull DOWN
+STR params@op:gpio_mode - pull UP
+VAL params@op:gpio_pwm - pwm_type s 0 0 -
+STR params@op:gpio_pwm - pwm_type hardware
+STR params@op:gpio_pwm - pwm_type software
+VAL params@root:ask_clarification - reason s 0 0 -
+STR params@root:ask_clarification - reason ambiguous_device
+STR params@root:ask_clarification - reason ambiguous_value
+STR params@root:ask_clarification - reason conflicting_instructions
+STR params@root:ask_clarification - reason missing_duration
+STR params@root:ask_clarification - reason missing_duty_cycle
+STR params@root:ask_clarification - reason missing_pin
+STR params@root:ask_clarification - reason missing_value
+STR params@root:ask_clarification - reason unknown_current_state
+VAL params@root:gpio_mode - drive_strength_ma n 3 0 -
+VAL params@root:gpio_mode - function s 0 0 -
+STR params@root:gpio_mode - function I2C1_SCL
+STR params@root:gpio_mode - function I2C1_SDA
+STR params@root:gpio_mode - function SPI0_CE0
+STR params@root:gpio_mode - function SPI0_CE1
+STR params@root:gpio_mode - function SPI0_MISO
+STR params@root:gpio_mode - function SPI0_MOSI
+STR params@root:gpio_mode - function SPI0_SCLK
+STR params@root:gpio_mode - function SPI1_CE0
+STR params@root:gpio_mode - function SPI1_CE1
+STR params@root:gpio_mode - function SPI1_CE2
+STR params@root:gpio_mode - function SPI1_MISO
+STR params@root:gpio_mode - function SPI1_MOSI
+STR params@root:gpio_mode - function SPI1_SCLK
+STR params@root:gpio_mode - function UART0_RX
+STR params@root:gpio_mode - function UART0_TX
+VAL params@root:gpio_mode - header_pin n 3 0 -
+VAL params@root:gpio_mode - pull s 0 0 -
+STR params@root:gpio_mode - pull DOWN
+STR params@root:gpio_mode - pull NONE
+STR params@root:gpio_mode - pull UP
+VAL params@root:gpio_pwm - frequency_hz n 6 0 -
+VAL params@root:gpio_pwm - pwm_type s 0 0 -
+STR params@root:gpio_pwm - pwm_type hardware
+STR params@root:gpio_pwm - pwm_type software
+VAL params@root:gpio_read - header_pin n 3 0 -
+VAL params@root:gpio_write - header_pin n 3 0 -
+VAL params@root:report_error - drive_strength_ma n 3 0 -
+VAL params@root:report_error - header_pin n 3 0 -
+VAL params@root:report_error - reason s 0 0 -
+STR params@root:report_error - reason invalid_gpio
+STR params@root:report_error - reason invalid_parameter
+STR params@root:report_error - reason no_hardware_action
+STR params@root:report_error - reason not_a_gpio
+STR params@root:report_error - reason pin_not_available
+STR params@root:report_error - reason reserved_gpio
+STR params@root:report_error - reason unsafe_current
+STR params@root:report_error - reason unsafe_direct_drive
+STR params@root:report_error - reason unsafe_short_circuit
+STR params@root:report_error - reason unsafe_voltage
+STR params@root:report_error - reason unsupported_function
+STR params@root:report_error - reason unsupported_operation
+VAL params@root:report_error - requested s 0 0 -
+STR params@root:report_error - requested analog_input
+STR params@root:report_error - requested drive_strength
+STR params@root:report_error - requested duration
+STR params@root:report_error - requested duty_cycle
+STR params@root:report_error - requested frequency
+STR params@root:report_error - requested function_conflict
+STR params@root:report_error - requested hardware_pwm
+STR params@root:report_error - requested i2c
+STR params@root:report_error - requested i2c_scl
+STR params@root:report_error - requested i2c_sda
+STR params@root:report_error - requested led_resistor
+STR params@root:report_error - requested load_current
+STR params@root:report_error - requested output_voltage
+STR params@root:report_error - requested pull_down
+STR params@root:report_error - requested pull_none
+STR params@root:report_error - requested spi
+STR params@root:report_error - requested spi_sclk
+STR params@root:report_error - requested uart_rx
+STR params@root:report_error - requested uart_tx
+VAL root - action s 0 0 -
+STR root - action ask_clarification
+STR root - action gpio_mode
+STR root - action gpio_pulse
+STR root - action gpio_pwm
+STR root - action gpio_read
+STR root - action gpio_sequence
+STR root - action gpio_write
+STR root - action report_error
+STR root - action wait
+VAL root ask_clarification parameters o 0 0 params@root:ask_clarification
+VAL root ask_clarification pin z 0 0 -
+VAL root gpio_mode mode s 0 0 -
+STR root gpio_mode mode ALT
+STR root gpio_mode mode INPUT
+STR root gpio_mode mode OUTPUT
+VAL root gpio_mode parameters o 0 0 params@root:gpio_mode
+VAL root gpio_mode pin n 3 0 -
+VAL root gpio_pulse duration_ms n 7 0 -
+VAL root gpio_pulse pin n 3 0 -
+VAL root gpio_pulse value s 0 0 -
+STR root gpio_pulse value HIGH
+STR root gpio_pulse value LOW
+VAL root gpio_pwm duty_cycle nf 4 2 -
+VAL root gpio_pwm parameters o 0 0 params@root:gpio_pwm
+VAL root gpio_pwm pin n 3 0 -
+VAL root gpio_read parameters o 0 0 params@root:gpio_read
+VAL root gpio_read pin n 3 0 -
+VAL root gpio_sequence operations a 0 0 op
+VAL root gpio_write parameters o 0 0 params@root:gpio_write
+VAL root gpio_write pin n 3 0 -
+VAL root gpio_write value s 0 0 -
+STR root gpio_write value HIGH
+STR root gpio_write value LOW
+VAL root report_error parameters o 0 0 params@root:report_error
+VAL root report_error pin nmz 5 0 -
+VAL root wait duration_ms n 7 0 -

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:258dfc8e3da3593719f52dec651dffdfe5e03d10d02ab85083c69972735a3090
+size 19848896

nano.gllm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f19cae5941263bc2afbc1b145523c097098fbba851bc1b06baef7eaa897db288
+size 5588032

tokenizer_config.json

@@ -0,0 +1,8 @@
+{
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "eos_token": "<|endoftext|>",
+  "bos_token": "<|endoftext|>",
+  "pad_token": "<|pad|>",
+  "model_max_length": 256,
+  "clean_up_tokenization_spaces": false
+}

training/eval_c_engine_mac_constrained.json

@@ -0,0 +1,22 @@
+{
+ "file": "mac_nano_constrained.out",
+ "rows": 5083,
+ "exact": 0.9171748967145387,
+ "valid_json": 1.0,
+ "unsafe_execute": 0.048238482384823846,
+ "non_exec_rows": 1845,
+ "latency_ms_p50": 2.7,
+ "latency_ms_p95": 5.0,
+ "latency_ms_max": 19.3,
+ "per_action_exact": {
+  "ask_clarification": 0.8560606060606061,
+  "gpio_mode": 0.9637305699481865,
+  "gpio_pulse": 0.9755434782608695,
+  "gpio_pwm": 0.9965397923875432,
+  "gpio_read": 0.9877622377622378,
+  "gpio_sequence": 0.9046653144016227,
+  "gpio_write": 0.8296370967741935,
+  "report_error": 0.9139784946236559,
+  "wait": 1.0
+ }
+}

training/eval_c_engine_mac_unconstrained.json

@@ -0,0 +1,22 @@
+{
+ "file": "mac_nano_unconstrained.out",
+ "rows": 5083,
+ "exact": 0.9169781625024592,
+ "valid_json": 0.9994097973637616,
+ "unsafe_execute": 0.046612466124661245,
+ "non_exec_rows": 1845,
+ "latency_ms_p50": 3.5,
+ "latency_ms_p95": 6.8,
+ "latency_ms_max": 25.9,
+ "per_action_exact": {
+  "ask_clarification": 0.8560606060606061,
+  "gpio_mode": 0.9637305699481865,
+  "gpio_pulse": 0.9755434782608695,
+  "gpio_pwm": 0.9965397923875432,
+  "gpio_read": 0.9877622377622378,
+  "gpio_sequence": 0.9046653144016227,
+  "gpio_write": 0.8296370967741935,
+  "report_error": 0.9133459835547122,
+  "wait": 1.0
+ }
+}

training/eval_c_engine_pi_zero2w.json

@@ -0,0 +1,22 @@
+{
+ "file": "pi_nano_constrained.out",
+ "rows": 5083,
+ "exact": 0.9171748967145387,
+ "valid_json": 1.0,
+ "unsafe_execute": 0.048238482384823846,
+ "non_exec_rows": 1845,
+ "latency_ms_p50": 132.3,
+ "latency_ms_p95": 279.96,
+ "latency_ms_max": 1514.0,
+ "per_action_exact": {
+  "ask_clarification": 0.8560606060606061,
+  "gpio_mode": 0.9637305699481865,
+  "gpio_pulse": 0.9755434782608695,
+  "gpio_pwm": 0.9965397923875432,
+  "gpio_read": 0.9877622377622378,
+  "gpio_sequence": 0.9046653144016227,
+  "gpio_write": 0.8296370967741935,
+  "report_error": 0.9139784946236559,
+  "wait": 1.0
+ }
+}

training/eval_pytorch_fp32_eval.json

@@ -0,0 +1,9 @@
+{
+ "exact": 0.9360763585199833,
+ "valid_json": 0.9997718370916835,
+ "action": 0.9620489029166825,
+ "unsafe_execute": 0.04111111111111111,
+ "model": "../runs/nano-sft/final",
+ "split": "eval",
+ "rows": 26297
+}

training/eval_pytorch_fp32_eval_core.json

@@ -0,0 +1,9 @@
+{
+ "exact": 0.9181585677749361,
+ "valid_json": 0.9994097973637616,
+ "action": 0.9478654337989376,
+ "unsafe_execute": 0.046612466124661245,
+ "model": "../runs/nano-sft/final",
+ "split": "eval_core",
+ "rows": 5083
+}

training/int8_vs_fp32_parity.txt

@@ -0,0 +1,4 @@
+wrote nano.gllm (re-export) (5.59 MB)
+next-token logits: min cosine 0.99983, argmax agreement 200/200
+greedy answers: int8 engine == fp32 PyTorch on 200/200 rows (100.0%; Phase 4 gate is 99.5%)
+re-export is byte-identical

training/lr_sweep-nano-1e-3.json

@@ -0,0 +1,25 @@
+{
+ "args": {
+  "shape": "nano",
+  "tokens": 55000000,
+  "batch": 256,
+  "lr": 0.001,
+  "wd": 0.1,
+  "warmup": 100,
+  "min_lr_frac": 0.1,
+  "eval_every": 100000,
+  "max_minutes": 6.0,
+  "seed": 1234,
+  "out": "sweep-nano-1e-3",
+  "compile": true
+ },
+ "params": 4960704,
+ "log": [
+  {
+   "step": 838,
+   "train_loss": 4.611106872558594,
+   "val_loss": 4.59149022102356,
+   "minutes": 1.2885763804117838
+  }
+ ]
+}

training/lr_sweep-nano-2e-3.json

@@ -0,0 +1,25 @@
+{
+ "args": {
+  "shape": "nano",
+  "tokens": 55000000,
+  "batch": 256,
+  "lr": 0.002,
+  "wd": 0.1,
+  "warmup": 100,
+  "min_lr_frac": 0.1,
+  "eval_every": 100000,
+  "max_minutes": 6.0,
+  "seed": 1234,
+  "out": "sweep-nano-2e-3",
+  "compile": true
+ },
+ "params": 4960704,
+ "log": [
+  {
+   "step": 838,
+   "train_loss": 4.3140764236450195,
+   "val_loss": 4.290324306488037,
+   "minutes": 1.0765070875485738
+  }
+ ]
+}

training/lr_sweep-nano-3e-3.json

@@ -0,0 +1,25 @@
+{
+ "args": {
+  "shape": "nano",
+  "tokens": 55000000,
+  "batch": 256,
+  "lr": 0.003,
+  "wd": 0.1,
+  "warmup": 100,
+  "min_lr_frac": 0.1,
+  "eval_every": 100000,
+  "max_minutes": 6.0,
+  "seed": 1234,
+  "out": "sweep-nano-3e-3",
+  "compile": true
+ },
+ "params": 4960704,
+ "log": [
+  {
+   "step": 838,
+   "train_loss": 4.225688457489014,
+   "val_loss": 4.183564710617065,
+   "minutes": 1.0914083560307821
+  }
+ ]
+}

training/lr_sweep-nano-5e-3.json

@@ -0,0 +1,25 @@
+{
+ "args": {
+  "shape": "nano",
+  "tokens": 55000000,
+  "batch": 256,
+  "lr": 0.005,
+  "wd": 0.1,
+  "warmup": 100,
+  "min_lr_frac": 0.1,
+  "eval_every": 100000,
+  "max_minutes": 6.0,
+  "seed": 1234,
+  "out": "sweep-nano-5e-3",
+  "compile": true
+ },
+ "params": 4960704,
+ "log": [
+  {
+   "step": 838,
+   "train_loss": 4.234477519989014,
+   "val_loss": 4.1984649181365965,
+   "minutes": 1.1979833285013834
+  }
+ ]
+}

training/pretrain_log.json

@@ -0,0 +1,73 @@
+{
+ "args": {
+  "shape": "nano",
+  "tokens": 550000000,
+  "batch": 256,
+  "lr": 0.003,
+  "wd": 0.1,
+  "warmup": 100,
+  "min_lr_frac": 0.1,
+  "eval_every": 1000,
+  "max_minutes": 45.0,
+  "seed": 1234,
+  "out": "nano-pt",
+  "compile": true
+ },
+ "params": 4960704,
+ "log": [
+  {
+   "step": 1000,
+   "train_loss": 4.157930374145508,
+   "val_loss": 4.101281976699829,
+   "minutes": 1.265407105286916
+  },
+  {
+   "step": 2000,
+   "train_loss": 4.010364532470703,
+   "val_loss": 3.921313500404358,
+   "minutes": 2.4889028827349344
+  },
+  {
+   "step": 3000,
+   "train_loss": 3.8780674934387207,
+   "val_loss": 3.8422478675842284,
+   "minutes": 3.7334064563115437
+  },
+  {
+   "step": 4000,
+   "train_loss": 3.891343593597412,
+   "val_loss": 3.785443663597107,
+   "minutes": 4.978402447700501
+  },
+  {
+   "step": 5000,
+   "train_loss": 3.8154168128967285,
+   "val_loss": 3.730230283737183,
+   "minutes": 6.2154850920041405
+  },
+  {
+   "step": 6000,
+   "train_loss": 3.758894443511963,
+   "val_loss": 3.6891664028167725,
+   "minutes": 7.451739505926768
+  },
+  {
+   "step": 7000,
+   "train_loss": 3.7148094177246094,
+   "val_loss": 3.650411367416382,
+   "minutes": 8.679173755645753
+  },
+  {
+   "step": 8000,
+   "train_loss": 3.737409830093384,
+   "val_loss": 3.6295988082885744,
+   "minutes": 9.910523466269176
+  },
+  {
+   "step": 8391,
+   "train_loss": 3.695378065109253,
+   "val_loss": 3.625276494026184,
+   "minutes": 10.390244070688883
+  }
+ ]
+}

training/sft_log.json

@@ -0,0 +1,105 @@
+{
+ "args": {
+  "init": "../runs/nano-pt/final",
+  "shape": "nano",
+  "train_rows": null,
+  "epochs": 2.0,
+  "batch": 256,
+  "lr": 0.002,
+  "wd": 0.1,
+  "warmup": 50,
+  "replay_every": 12,
+  "replay_batch": 4,
+  "eval_every": 1000,
+  "eval_rows": 1000,
+  "max_minutes": 50.0,
+  "seed": 1234,
+  "out": "nano-sft"
+ },
+ "log": [
+  {
+   "step": 1000,
+   "train_loss": 0.006621028296649456,
+   "eval_target_loss": 0.07434787717331105,
+   "minutes": 0.7496363361676533
+  },
+  {
+   "step": 2000,
+   "train_loss": 0.02178449183702469,
+   "eval_target_loss": 0.05123574915808273,
+   "minutes": 1.4465561429659526
+  },
+  {
+   "step": 3000,
+   "train_loss": 0.00805601105093956,
+   "eval_target_loss": 0.04877272646884206,
+   "minutes": 2.1609952886899313
+  },
+  {
+   "step": 4000,
+   "train_loss": 0.0015760211972519755,
+   "eval_target_loss": 0.0506566401789748,
+   "minutes": 2.850797164440155
+  },
+  {
+   "step": 5000,
+   "train_loss": 0.0027423014398664236,
+   "eval_target_loss": 0.04380449522909304,
+   "minutes": 3.5392218232154846
+  },
+  {
+   "step": 6000,
+   "train_loss": 0.001060945214703679,
+   "eval_target_loss": 0.0389024249297705,
+   "minutes": 4.238372592131297
+  },
+  {
+   "step": 7000,
+   "train_loss": 0.0005649410304613411,
+   "eval_target_loss": 0.03259135582334248,
+   "minutes": 4.983746925989787
+  },
+  {
+   "step": 8000,
+   "train_loss": 0.0010784538462758064,
+   "eval_target_loss": 0.028708800912080383,
+   "minutes": 5.700136307875315
+  },
+  {
+   "step": 9000,
+   "train_loss": 0.0004312685050535947,
+   "eval_target_loss": 0.02426343787667973,
+   "minutes": 6.410312342643738
+  },
+  {
+   "step": 10000,
+   "train_loss": 0.00011572329822229221,
+   "eval_target_loss": 0.02548710253615743,
+   "minutes": 7.112147303422292
+  },
+  {
+   "step": 11000,
+   "train_loss": 0.0005688337259925902,
+   "eval_target_loss": 0.02789109825700458,
+   "minutes": 7.8145731012026465
+  },
+  {
+   "step": 12000,
+   "train_loss": 6.69505971018225e-05,
+   "eval_target_loss": 0.0214684495689006,
+   "minutes": 8.530480217933654
+  },
+  {
+   "step": 13000,
+   "train_loss": 0.0004814007261302322,
+   "eval_target_loss": 0.02294556570453032,
+   "minutes": 9.222708662350973
+  },
+  {
+   "step": 13115,
+   "train_loss": 3.8525613490492105e-05,
+   "eval_target_loss": 0.022417631390016398,
+   "minutes": 9.301433749993642
+  }
+ ]
+}