alexwengg commited on
Commit
10bca41
·
verified ·
1 Parent(s): c87b915

Add real-input ANE profiling across four compute policies

Browse files

Record scheduler placement and bounded three-form timing with unchanged model hashes. CPU + ANE: 149 ANE / 24 CPU operations, 0.929 ms median, 0.973 ms p95. All 120 timed predictions correct. Results are not utilization or power traces.

README.md CHANGED
@@ -145,6 +145,38 @@ The original demo is not redistributed here; its exact revision and SHA-256 are
145
  in the asset lock. No training or larger-corpus evaluation was performed for
146
  this conversion.
147
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
148
  ## Application responsibilities
149
 
150
  The application must extract document entities, describe UI elements, build
 
145
  in the asset lock. No training or larger-corpus evaluation was performed for
146
  this conversion.
147
 
148
+ ## ANE profile
149
+
150
+ A separate September 19 profile uses the same portable-package hashes on the M5
151
+ Pro, with real demo rows 0, 68, and 130 (27, 21, and 19 options). Each policy
152
+ runs two warmup passes and ten timed passes, totaling 30 timed predictions. All
153
+ 120 timed predictions select the correct labels.
154
+
155
+ | Policy | CPU ops | GPU ops | ANE ops | Warm p50 | Warm p95 |
156
+ | --- | ---: | ---: | ---: | ---: | ---: |
157
+ | `CPU_ONLY` | 173 | 0 | 0 | 1.527 ms | 1.602 ms |
158
+ | `CPU_AND_GPU` | 0 | 173 | 0 | 0.929 ms | 2.380 ms |
159
+ | `CPU_AND_NE` | 24 | 0 | 149 | 0.929 ms | 0.973 ms |
160
+ | `ALL` | 0 | 173 | 0 | 0.912 ms | 1.229 ms |
161
+
162
+ `CPU_AND_NE` assigns 86.1% of operations to ANE; `ALL` chooses the GPU on this
163
+ Mac. CPU fallbacks cover integer/mask preparation and embedding gathers. These
164
+ are public `MLComputePlan` preferred-device assignments, not measurements of
165
+ utilization, energy, or time spent on each device. No Instruments runtime trace
166
+ was captured.
167
+
168
+ ANE model loading took 566.8 ms, followed by a 1.65 ms first prediction, with
169
+ system caches retained. These are not first-install cold-start numbers. Warm
170
+ timing includes Python model-call overhead and excludes encoding, Swift/UI work,
171
+ and animation. This three-row timing manifest differs from the full conversion
172
+ parity run above; no weights or graph were changed.
173
+
174
+ See [reports/ane-profile.json](reports/ane-profile.json) for all operation
175
+ assignments, individual timings, hashes, and the protocol;
176
+ [reports/ane-fallback.json](reports/ane-fallback.json) records rejection reasons.
177
+ Reproduce with `uv run --frozen python profile-coreml.py` in the
178
+ [Mobius conversion directory](https://github.com/FluidInference/mobius/tree/codex/cua-s1-forms/models/computer-use/cua-s1-forms/coreml).
179
+
180
  ## Application responsibilities
181
 
182
  The application must extract document entities, describe UI elements, build
checksums.json CHANGED
@@ -1,7 +1,7 @@
1
  {
2
  "LICENSE": "c0779290c1d4783169aa3dbfb55feb505e563ef8a004bbf55298ceffcfbda8d9",
3
  "NOTICES.md": "027c72741eaa695e60d7b6cebd3666372c81d443a96b673b72a897cf432ddfd0",
4
- "README.md": "4008ae7ef610e44c2d6e019a0fa375b3e1705714d5875a8e0c63aba8541b4c96",
5
  "UPSTREAM-THIRD-PARTY-NOTICES.md": "4091e69b45c8cc97e30a066fbbd56148dbef66ab716432c048d2333c9c464213",
6
  "assets.lock.json": "8e65ad70af6bb814b571cdcfe828ba4bc339147da2d2211cbeac416163ef18ba",
7
  "conversion.json": "ac94239751ceb7e19602eb7276940ff53446b1776d4900405af9be0382fe4a94",
@@ -13,7 +13,8 @@
13
  "cua_s1_forms_fp16_options32.mlpackage/Data/com.apple.CoreML/weights/weight.bin": "4da9259f798e44f5a1b50769ee1916fd3747c4d723dd9997b516c7fe238c7895",
14
  "cua_s1_forms_fp16_options32.mlpackage/Manifest.json": "2bc0f5f62337b27fb6b0ecde248f1e3dc269e1ba4b65516aaeede2a60e293dcc",
15
  "preprocessing.py": "18c74ede43f95f91cd638c7e9631923a353e3a24d3b522ddb8b18b480f9c2846",
16
- "reports/ane-fallback.json": "0a5d25b97545bfb2878531f3e409030ae3448a26c3aa3ba7db29f57d7194b2a6",
 
17
  "reports/upstream-metrics.json": "7d6207e430504dc0baec96d8180cb03f8aadc5a6e016b53c4cf03a80baf169ab",
18
  "reports/verification.json": "9a5d22ec4b447f0710c12e03c69d7ed802158bb94e9a3c6b059731730ced9f30"
19
  }
 
1
  {
2
  "LICENSE": "c0779290c1d4783169aa3dbfb55feb505e563ef8a004bbf55298ceffcfbda8d9",
3
  "NOTICES.md": "027c72741eaa695e60d7b6cebd3666372c81d443a96b673b72a897cf432ddfd0",
4
+ "README.md": "577e94c34abc15806c34daccf78245f0c3a58226cbead66c4458c9c9d50128da",
5
  "UPSTREAM-THIRD-PARTY-NOTICES.md": "4091e69b45c8cc97e30a066fbbd56148dbef66ab716432c048d2333c9c464213",
6
  "assets.lock.json": "8e65ad70af6bb814b571cdcfe828ba4bc339147da2d2211cbeac416163ef18ba",
7
  "conversion.json": "ac94239751ceb7e19602eb7276940ff53446b1776d4900405af9be0382fe4a94",
 
13
  "cua_s1_forms_fp16_options32.mlpackage/Data/com.apple.CoreML/weights/weight.bin": "4da9259f798e44f5a1b50769ee1916fd3747c4d723dd9997b516c7fe238c7895",
14
  "cua_s1_forms_fp16_options32.mlpackage/Manifest.json": "2bc0f5f62337b27fb6b0ecde248f1e3dc269e1ba4b65516aaeede2a60e293dcc",
15
  "preprocessing.py": "18c74ede43f95f91cd638c7e9631923a353e3a24d3b522ddb8b18b480f9c2846",
16
+ "reports/ane-fallback.json": "2f52b0e11f7cfbea6a6be6942de8816d8fed0630ac384a28551d405648fcb4e0",
17
+ "reports/ane-profile.json": "8f654717397a294acc793861676a1480c160bb3f7ab5be6ff67ebb6a4473b832",
18
  "reports/upstream-metrics.json": "7d6207e430504dc0baec96d8180cb03f8aadc5a6e016b53c4cf03a80baf169ab",
19
  "reports/verification.json": "9a5d22ec4b447f0710c12e03c69d7ed802158bb94e9a3c6b059731730ced9f30"
20
  }
reports/ane-fallback.json CHANGED
@@ -4,7 +4,7 @@
4
  "chip": "Apple M5 Pro",
5
  "ram": "24GB",
6
  "os_version": "macOS 27.0",
7
- "timestamp": "2026-09-19T16:51:27.323329+00:00"
8
  },
9
  "models": [
10
  {
 
4
  "chip": "Apple M5 Pro",
5
  "ram": "24GB",
6
  "os_version": "macOS 27.0",
7
+ "timestamp": "2026-09-19T19:41:20.560977+00:00"
8
  },
9
  "models": [
10
  {
reports/ane-profile.json ADDED
The diff for this file is too large to render. See raw diff