Add real-input ANE profiling across four compute policies
Browse filesRecord scheduler placement and bounded three-form timing with unchanged model hashes. CPU + ANE: 149 ANE / 24 CPU operations, 0.929 ms median, 0.973 ms p95. All 120 timed predictions correct. Results are not utilization or power traces.
- README.md +32 -0
- checksums.json +3 -2
- reports/ane-fallback.json +1 -1
- reports/ane-profile.json +0 -0
README.md
CHANGED
|
@@ -145,6 +145,38 @@ The original demo is not redistributed here; its exact revision and SHA-256 are
|
|
| 145 |
in the asset lock. No training or larger-corpus evaluation was performed for
|
| 146 |
this conversion.
|
| 147 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 148 |
## Application responsibilities
|
| 149 |
|
| 150 |
The application must extract document entities, describe UI elements, build
|
|
|
|
| 145 |
in the asset lock. No training or larger-corpus evaluation was performed for
|
| 146 |
this conversion.
|
| 147 |
|
| 148 |
+
## ANE profile
|
| 149 |
+
|
| 150 |
+
A separate September 19 profile uses the same portable-package hashes on the M5
|
| 151 |
+
Pro, with real demo rows 0, 68, and 130 (27, 21, and 19 options). Each policy
|
| 152 |
+
runs two warmup passes and ten timed passes, totaling 30 timed predictions. All
|
| 153 |
+
120 timed predictions select the correct labels.
|
| 154 |
+
|
| 155 |
+
| Policy | CPU ops | GPU ops | ANE ops | Warm p50 | Warm p95 |
|
| 156 |
+
| --- | ---: | ---: | ---: | ---: | ---: |
|
| 157 |
+
| `CPU_ONLY` | 173 | 0 | 0 | 1.527 ms | 1.602 ms |
|
| 158 |
+
| `CPU_AND_GPU` | 0 | 173 | 0 | 0.929 ms | 2.380 ms |
|
| 159 |
+
| `CPU_AND_NE` | 24 | 0 | 149 | 0.929 ms | 0.973 ms |
|
| 160 |
+
| `ALL` | 0 | 173 | 0 | 0.912 ms | 1.229 ms |
|
| 161 |
+
|
| 162 |
+
`CPU_AND_NE` assigns 86.1% of operations to ANE; `ALL` chooses the GPU on this
|
| 163 |
+
Mac. CPU fallbacks cover integer/mask preparation and embedding gathers. These
|
| 164 |
+
are public `MLComputePlan` preferred-device assignments, not measurements of
|
| 165 |
+
utilization, energy, or time spent on each device. No Instruments runtime trace
|
| 166 |
+
was captured.
|
| 167 |
+
|
| 168 |
+
ANE model loading took 566.8 ms, followed by a 1.65 ms first prediction, with
|
| 169 |
+
system caches retained. These are not first-install cold-start numbers. Warm
|
| 170 |
+
timing includes Python model-call overhead and excludes encoding, Swift/UI work,
|
| 171 |
+
and animation. This three-row timing manifest differs from the full conversion
|
| 172 |
+
parity run above; no weights or graph were changed.
|
| 173 |
+
|
| 174 |
+
See [reports/ane-profile.json](reports/ane-profile.json) for all operation
|
| 175 |
+
assignments, individual timings, hashes, and the protocol;
|
| 176 |
+
[reports/ane-fallback.json](reports/ane-fallback.json) records rejection reasons.
|
| 177 |
+
Reproduce with `uv run --frozen python profile-coreml.py` in the
|
| 178 |
+
[Mobius conversion directory](https://github.com/FluidInference/mobius/tree/codex/cua-s1-forms/models/computer-use/cua-s1-forms/coreml).
|
| 179 |
+
|
| 180 |
## Application responsibilities
|
| 181 |
|
| 182 |
The application must extract document entities, describe UI elements, build
|
checksums.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
{
|
| 2 |
"LICENSE": "c0779290c1d4783169aa3dbfb55feb505e563ef8a004bbf55298ceffcfbda8d9",
|
| 3 |
"NOTICES.md": "027c72741eaa695e60d7b6cebd3666372c81d443a96b673b72a897cf432ddfd0",
|
| 4 |
-
"README.md": "
|
| 5 |
"UPSTREAM-THIRD-PARTY-NOTICES.md": "4091e69b45c8cc97e30a066fbbd56148dbef66ab716432c048d2333c9c464213",
|
| 6 |
"assets.lock.json": "8e65ad70af6bb814b571cdcfe828ba4bc339147da2d2211cbeac416163ef18ba",
|
| 7 |
"conversion.json": "ac94239751ceb7e19602eb7276940ff53446b1776d4900405af9be0382fe4a94",
|
|
@@ -13,7 +13,8 @@
|
|
| 13 |
"cua_s1_forms_fp16_options32.mlpackage/Data/com.apple.CoreML/weights/weight.bin": "4da9259f798e44f5a1b50769ee1916fd3747c4d723dd9997b516c7fe238c7895",
|
| 14 |
"cua_s1_forms_fp16_options32.mlpackage/Manifest.json": "2bc0f5f62337b27fb6b0ecde248f1e3dc269e1ba4b65516aaeede2a60e293dcc",
|
| 15 |
"preprocessing.py": "18c74ede43f95f91cd638c7e9631923a353e3a24d3b522ddb8b18b480f9c2846",
|
| 16 |
-
"reports/ane-fallback.json": "
|
|
|
|
| 17 |
"reports/upstream-metrics.json": "7d6207e430504dc0baec96d8180cb03f8aadc5a6e016b53c4cf03a80baf169ab",
|
| 18 |
"reports/verification.json": "9a5d22ec4b447f0710c12e03c69d7ed802158bb94e9a3c6b059731730ced9f30"
|
| 19 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"LICENSE": "c0779290c1d4783169aa3dbfb55feb505e563ef8a004bbf55298ceffcfbda8d9",
|
| 3 |
"NOTICES.md": "027c72741eaa695e60d7b6cebd3666372c81d443a96b673b72a897cf432ddfd0",
|
| 4 |
+
"README.md": "577e94c34abc15806c34daccf78245f0c3a58226cbead66c4458c9c9d50128da",
|
| 5 |
"UPSTREAM-THIRD-PARTY-NOTICES.md": "4091e69b45c8cc97e30a066fbbd56148dbef66ab716432c048d2333c9c464213",
|
| 6 |
"assets.lock.json": "8e65ad70af6bb814b571cdcfe828ba4bc339147da2d2211cbeac416163ef18ba",
|
| 7 |
"conversion.json": "ac94239751ceb7e19602eb7276940ff53446b1776d4900405af9be0382fe4a94",
|
|
|
|
| 13 |
"cua_s1_forms_fp16_options32.mlpackage/Data/com.apple.CoreML/weights/weight.bin": "4da9259f798e44f5a1b50769ee1916fd3747c4d723dd9997b516c7fe238c7895",
|
| 14 |
"cua_s1_forms_fp16_options32.mlpackage/Manifest.json": "2bc0f5f62337b27fb6b0ecde248f1e3dc269e1ba4b65516aaeede2a60e293dcc",
|
| 15 |
"preprocessing.py": "18c74ede43f95f91cd638c7e9631923a353e3a24d3b522ddb8b18b480f9c2846",
|
| 16 |
+
"reports/ane-fallback.json": "2f52b0e11f7cfbea6a6be6942de8816d8fed0630ac384a28551d405648fcb4e0",
|
| 17 |
+
"reports/ane-profile.json": "8f654717397a294acc793861676a1480c160bb3f7ab5be6ff67ebb6a4473b832",
|
| 18 |
"reports/upstream-metrics.json": "7d6207e430504dc0baec96d8180cb03f8aadc5a6e016b53c4cf03a80baf169ab",
|
| 19 |
"reports/verification.json": "9a5d22ec4b447f0710c12e03c69d7ed802158bb94e9a3c6b059731730ced9f30"
|
| 20 |
}
|
reports/ane-fallback.json
CHANGED
|
@@ -4,7 +4,7 @@
|
|
| 4 |
"chip": "Apple M5 Pro",
|
| 5 |
"ram": "24GB",
|
| 6 |
"os_version": "macOS 27.0",
|
| 7 |
-
"timestamp": "2026-09-
|
| 8 |
},
|
| 9 |
"models": [
|
| 10 |
{
|
|
|
|
| 4 |
"chip": "Apple M5 Pro",
|
| 5 |
"ram": "24GB",
|
| 6 |
"os_version": "macOS 27.0",
|
| 7 |
+
"timestamp": "2026-09-19T19:41:20.560977+00:00"
|
| 8 |
},
|
| 9 |
"models": [
|
| 10 |
{
|
reports/ane-profile.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|