Add verified Raspberry Pi 5 edge performance measurements
Browse filesRecord actual 16 GB Pi 5 latency, RAM and correctness with frozen evaluation inputs, raw per-call measurements, runtime hashes and reproducible settings. Preserve weights and existing training data.
README.md
CHANGED
|
@@ -90,7 +90,7 @@ web UI and Docker image.
|
|
| 90 |
<!-- rpi5-measurements -->
|
| 91 |
## Raspberry Pi 5 edge-device performance
|
| 92 |
|
| 93 |
-
Measured **2026-09-26** on a **Raspberry Pi 5 Model B, 16 GB RAM**, Debian 13.2/aarch64, Python 3.13.5. Measurement update: `edge-rpi5-20260926`; weights and training data are unchanged.
|
| 94 |
|
| 95 |
| Threads | Median ms | p95 ms | p99 ms | Serial requests/s | Process RSS MiB | CPU °C min–mean–max | Timed correct |
|
| 96 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
|
@@ -101,7 +101,7 @@ Batch 1; 20 warmups and 500 timed calls per row, cycling over 10 frozen model-sp
|
|
| 101 |
|
| 102 |
On-device quality check: **192/200** on training-set diagnostic, NOT held-out accuracy. For v3 this is explicitly a training-set diagnostic, not held-out accuracy; the earlier 96.73% result was not rerun.
|
| 103 |
|
| 104 |
-
[Full Pi methodology, temperature plot and reproduction](edge_benchmarks/rpi5-2026-09-26/REPORT.md) · [Every latency/temperature reading](edge_benchmarks/rpi5-2026-09-26/readings.csv) · [Raw measurements](edge_benchmarks/rpi5-2026-09-26/results) · [Versioned evaluation data](https://huggingface.co/datasets/sraivante/home-commands-json-v3/tree/edge-rpi5-20260926/edge_benchmarks/rpi5-2026-09-26).
|
| 105 |
<!-- /rpi5-measurements -->
|
| 106 |
|
| 107 |
## Why a tiny model and not an LLM
|
|
|
|
| 90 |
<!-- rpi5-measurements -->
|
| 91 |
## Raspberry Pi 5 edge-device performance
|
| 92 |
|
| 93 |
+
Measured **2026-09-26** on a **Raspberry Pi 5 Model B, 16 GB RAM**, Debian 13.2/aarch64, Python 3.13.5. Measurement update: `edge-rpi5-20260926.1`; weights and training data are unchanged.
|
| 94 |
|
| 95 |
| Threads | Median ms | p95 ms | p99 ms | Serial requests/s | Process RSS MiB | CPU °C min–mean–max | Timed correct |
|
| 96 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
|
|
|
| 101 |
|
| 102 |
On-device quality check: **192/200** on training-set diagnostic, NOT held-out accuracy. For v3 this is explicitly a training-set diagnostic, not held-out accuracy; the earlier 96.73% result was not rerun.
|
| 103 |
|
| 104 |
+
[Full Pi methodology, temperature plot and reproduction](edge_benchmarks/rpi5-2026-09-26/REPORT.md) · [Every latency/temperature reading](edge_benchmarks/rpi5-2026-09-26/readings.csv) · [Raw measurements](edge_benchmarks/rpi5-2026-09-26/results) · [Versioned evaluation data](https://huggingface.co/datasets/sraivante/home-commands-json-v3/tree/edge-rpi5-20260926.1/edge_benchmarks/rpi5-2026-09-26).
|
| 105 |
<!-- /rpi5-measurements -->
|
| 106 |
|
| 107 |
## Why a tiny model and not an LLM
|
edge_benchmarks/rpi5-2026-09-26/REPORT.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# Raspberry Pi 5 measurements — Tiny Home Robotics JSON 1M v3
|
| 2 |
|
| 3 |
-
Measured on 2026-09-26; measurement revision `edge-rpi5-20260926`. The model weights and training data were not changed.
|
| 4 |
|
| 5 |
Hardware: **Raspberry Pi 5 Model B Rev 1.1**, 15.84 GiB OS-visible RAM (16 GB model), four CPU cores, aarch64, Debian 13.2, kernel `6.12.47+rpt-rpi-2712`. Python 3.13.5. The device's governor and library versions are recorded in each JSON result. Cooling and power-supply model were not inspected.
|
| 6 |
|
|
@@ -39,12 +39,12 @@ Timed-correct counts above describe the repeated ten-input latency set, not an i
|
|
| 39 |
- Published model tested: [`sraivante/superfast-tiny-home-robotics-json-1m-v3@2b2e71d37ea9fa0fe6ab247b6a74e8db774fb6f2`](https://huggingface.co/sraivante/superfast-tiny-home-robotics-json-1m-v3/tree/2b2e71d37ea9fa0fe6ab247b6a74e8db774fb6f2). All tested runtime assets were checked against this revision.
|
| 40 |
- Learned weight SHA-256: `fc749ecd89c1ae30a159d6d701036298b5583645b7fbf114311e006929c8778b` (2,146,783 bytes).
|
| 41 |
- Associated preserved training dataset: [`sraivante/home-commands-json-v3@da10aca352a34cb2834a049867f6f2e9fcc30e3e`](https://huggingface.co/datasets/sraivante/home-commands-json-v3/tree/da10aca352a34cb2834a049867f6f2e9fcc30e3e). Existing source/split hashes and historical provenance remain in that snapshot. No fine-tuning was performed for this measurement update.
|
| 42 |
-
- The [versioned evaluation inputs and Pi predictions](https://huggingface.co/datasets/sraivante/home-commands-json-v3/tree/edge-rpi5-20260926/edge_benchmarks/rpi5-2026-09-26) record the new diagnostic snapshot. `provenance.json` records its hashes, counts, source revisions, and historical identity limitations.
|
| 43 |
- [Runtime identity check](runtime_identity.json); [raw per-call records](results); [exact environment](environment.lock.txt).
|
| 44 |
|
| 45 |
## Reproduce
|
| 46 |
|
| 47 |
-
Use a 64-bit Raspberry Pi in an isolated environment. Download this model repository at `edge-rpi5-20260926`, then run:
|
| 48 |
|
| 49 |
```bash
|
| 50 |
python3 -m venv .venv
|
|
|
|
| 1 |
# Raspberry Pi 5 measurements — Tiny Home Robotics JSON 1M v3
|
| 2 |
|
| 3 |
+
Measured on 2026-09-26; measurement revision `edge-rpi5-20260926.1`. The model weights and training data were not changed.
|
| 4 |
|
| 5 |
Hardware: **Raspberry Pi 5 Model B Rev 1.1**, 15.84 GiB OS-visible RAM (16 GB model), four CPU cores, aarch64, Debian 13.2, kernel `6.12.47+rpt-rpi-2712`. Python 3.13.5. The device's governor and library versions are recorded in each JSON result. Cooling and power-supply model were not inspected.
|
| 6 |
|
|
|
|
| 39 |
- Published model tested: [`sraivante/superfast-tiny-home-robotics-json-1m-v3@2b2e71d37ea9fa0fe6ab247b6a74e8db774fb6f2`](https://huggingface.co/sraivante/superfast-tiny-home-robotics-json-1m-v3/tree/2b2e71d37ea9fa0fe6ab247b6a74e8db774fb6f2). All tested runtime assets were checked against this revision.
|
| 40 |
- Learned weight SHA-256: `fc749ecd89c1ae30a159d6d701036298b5583645b7fbf114311e006929c8778b` (2,146,783 bytes).
|
| 41 |
- Associated preserved training dataset: [`sraivante/home-commands-json-v3@da10aca352a34cb2834a049867f6f2e9fcc30e3e`](https://huggingface.co/datasets/sraivante/home-commands-json-v3/tree/da10aca352a34cb2834a049867f6f2e9fcc30e3e). Existing source/split hashes and historical provenance remain in that snapshot. No fine-tuning was performed for this measurement update.
|
| 42 |
+
- The [versioned evaluation inputs and Pi predictions](https://huggingface.co/datasets/sraivante/home-commands-json-v3/tree/edge-rpi5-20260926.1/edge_benchmarks/rpi5-2026-09-26) record the new diagnostic snapshot. `provenance.json` records its hashes, counts, source revisions, and historical identity limitations.
|
| 43 |
- [Runtime identity check](runtime_identity.json); [raw per-call records](results); [exact environment](environment.lock.txt).
|
| 44 |
|
| 45 |
## Reproduce
|
| 46 |
|
| 47 |
+
Use a 64-bit Raspberry Pi in an isolated environment. Download this model repository at `edge-rpi5-20260926.1`, then run:
|
| 48 |
|
| 49 |
```bash
|
| 50 |
python3 -m venv .venv
|
edge_benchmarks/rpi5-2026-09-26/provenance.json
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
{
|
| 2 |
-
"measurement_revision": "edge-rpi5-20260926",
|
| 3 |
"model": {
|
| 4 |
"repo_id": "sraivante/superfast-tiny-home-robotics-json-1m-v3",
|
| 5 |
"revision": "2b2e71d37ea9fa0fe6ab247b6a74e8db774fb6f2",
|
|
|
|
| 1 |
{
|
| 2 |
+
"measurement_revision": "edge-rpi5-20260926.1",
|
| 3 |
"model": {
|
| 4 |
"repo_id": "sraivante/superfast-tiny-home-robotics-json-1m-v3",
|
| 5 |
"revision": "2b2e71d37ea9fa0fe6ab247b6a74e8db774fb6f2",
|