massif/Qwen3-1.7B-CoreAI-ANE-W8-4K-h16p

Qwen3-1.7B · Apple Core AI · ANE · W8 · 4K · h16p

Default runtime-compatible release. The compiled artifact distributed on main is identical to pinned revision 75bbe06906cb5d953e602e3e4fb6364187c81822. It was produced with Xcode 27 Beta 6 and coreai-build-3600.83.1, then used for an eager-load diagnostic and the Run J generation/throughput experiment on an iPhone 15 Pro running iOS 27 build 24A5424a. Historical quality and ANE-trace results below refer to the preceding public artifact and are not attributed to this binary.

A complete, directly downloadable Apple Core AI bundle for Qwen3-1.7B, AOT-compiled for A17 Pro (h16p) with Neural Engine preferred.

This is a community conversion for iOS 27. It is not an official Apple or Qwen release. Neural Engine participation was measured for the preceding public artifact; the current artifact does not claim a new trace or exclusive ANE execution.

Reproduction, tests, recipe, and companion app: massif-01/qwen3-1.7b-coreai-ios

Apple upstream request: apple/coreai-models#116

Why this artifact is different

This release targets the static Apple Neural Engine path rather than packaging another dynamic GPU conversion:

Property	This release
Runtime	Apple Core AI + LanguageModelSession
AOT target	h16p
Supported chip in AOT inspection	A17 Pro
Preferred compute	Neural Engine
Transformer projections	W8 per-tensor K-means palettization
Tied embedding	Separate INT8 per-tensor quantization
Compute	FP16
KV cache	Fixed-size FP16, 448 MiB at 4K
Maximum context	4,096 tokens
AOT functions	34
Current artifact validation	Eager load/unload plus 120/120 Run J generations, iPhone 15 Pro, iOS 27 build 24A5424a

The static path must survive shape specialization, prompt/extend bucketing, mutable KV state, AOT compilation, device specialization, and the Foundation Models event bridge. For the preceding public artifact, a Core AI Instruments trace observed 547 MPSGraph intervals matched by 547 Apple Neural Engine Prediction intervals. A new trace for the current artifact is not claimed.

This does not imply that ANE is universally faster than GPU. No same-device, controlled GPU-versus-ANE energy comparison is claimed.

See the dated comparison with adjacent community artifacts in RELATED_WORK.md.

Quality-first selection

W8 was selected through a frozen numerical gate, not a successful prompt smoke. Multiple uniform W4 and mixed W4/W8 candidates were rejected before the final mechanism was locked.

Evaluation	Mean logits cosine	Minimum cosine	Top-1 agreement	Mean NLL delta
W8 tuning set	0.997067	0.967261	98.44%	0.002985
W8 frozen holdout	0.996598	0.959625	98.37%	0.003167

Representative rejected W4 and mixed W4/W8 candidates ranged from 0.856886 to 0.981232 mean cosine. These measurements establish conversion fidelity; they are not a general downstream benchmark ranking. Full WikiText-2 perplexity is not claimed as complete.

Machine-readable evidence is included under evidence/ and maintained in the GitHub repository. Except for current-toolchain-load-validation.json, the inherited evidence files describe the preceding public artifact.

Current-toolchain validation

The current artifact was exported with the same locked base-model revision and frozen W8 recipe, then AOT-compiled with coreai-build-3600.83.1 from Xcode 27 Beta 6 (27A5252f). It has a distinct source and compiled identity:

Current artifact identity	Value
Source .aimodel SHA-256	13ba3f73fcb7e090cd6ba1ca14b6b8903516ab608d451e94b9cdd750cfceda2c
Compiled main-h16p.mlirb SHA-256	09f609775baa56b11ff3c91bfcb07b145930297289634fdc5514b2a5ab4dc7ca
Complete .aimodelc file-list fingerprint	182336f4654bb735bcad35e45f7832756c34469931ad96d872532dca727ebd8d
Producer	coreai-build-3600.83.1

One load-only diagnostic used eager mode with a fixed-size KV cache on an iPhone 15 Pro running iOS 27 build 24A5424a:

Load-only observation	Result
Status	passed
Eager load	41.931619875 s
Peak resident memory	2,737.046875 MiB
Resident memory after unload	236.96875 MiB
Unload	completed
Process exit	0

The load-only record establishes that this exact compiled artifact could be loaded and unloaded in the stated device and software environment. It was not defined as a whole-system cache-cold measurement. Because both the authoring export and compiler producer differ from the earlier public binary, this result does not isolate an OS regression or a compiler change as the sole cause of the earlier compatibility failure.

See evidence/current-toolchain-load-validation.json for the recorded identity, runtime fields, and source-log hashes.

The same artifact identity was subsequently used in the confirmatory Run J speed experiment: eight alternating physical blocks produced 120/120 completed observations and 60/60 paired logical samples across three workload shapes. The protocol, raw evidence, and analysis are published in the qwen3-1.7b-coreai-ios repository. That experiment measures generation latency and throughput; it is not an energy or execution-unit trace.

Historical public rebuild verification

The artifact published in the preceding repository revision was rebuilt under a neutral path, installed on an iPhone 15 Pro, and rerun through the same six-case business suite before publication. The app verified the historical public source hash 5e885ec4...0aef73 before loading that AOT asset.

Public-artifact check	Result
Complete cases	6 / 6
Suite result	COREAI_SUITE_PASS
Fresh load	46.176 s
Structured JSON TTFT	1.213 s
Structured JSON visible decode	19.516 tok/s
Long-context input	3,790 tokens
Long-context TTFT	5.709 s
Long-context visible decode	16.748 tok/s
Reasoning output / reasoning tokens	288 / 277
Peak resident memory	2,766.4 MiB
Resident memory after unload	197.3 MiB
Full suite duration	77.645 s

The 3,790-token case recovered the frozen tail marker TARGET-0377. The fresh load includes first-use device work and should not be compared directly with the specialization-warmed load below.

Historical physical-device results

The preceding public artifact completed four full six-case suites on iPhone 15 Pro with zero empty responses, truncations, degenerate loops, context overflows, crashes, OOMs, or assertion failures. These measurements are not new measurements of the current-toolchain artifact.

Final standalone suite:

Case	Input tokens	Output / reasoning	TTFT	Visible decode
Structured JSON	160	60 / 1	0.564 s	17.611 tok/s
Hot context turn	241	5 / 1	0.428 s	54.205 tok/s
Hot rewrite turn	275	5 / 1	0.472 s	49.455 tok/s
Injection boundary	309	5 / 1	0.474 s	54.596 tok/s
Long context	3,790	10 / 1	5.627 s	15.828 tok/s
Reasoning	69	271 / 260	15.925 s	21.370 tok/s

Additional measurements:

• load after specialization warm-up: 11.312 s;
• peak resident memory: 2,963.7 MiB;
• resident memory after unload: 198.7 MiB;
• product-runtime near-4K input: 3,194 tokens, 5.138 s TTFT, 13.087 tok/s visible decode;
• accepted runtime paths included cancellation, unload/reload, offline cold start, background recovery, memory-pressure cancellation, and delete/reconcile.

The detailed product implementation is intentionally not distributed.

Download

hf download massif/Qwen3-1.7B-CoreAI-ANE-W8-4K-h16p \
  --local-dir Qwen3-1.7B-CoreAI-ANE-W8-4K-h16p

For immutable reproduction of the artifact used in Run J, add --revision 75bbe06906cb5d953e602e3e4fb6364187c81822.

The downloaded repository root is a complete Core AI resource directory:

Qwen3-1.7B-CoreAI-ANE-W8-4K-h16p/
├── metadata.json
├── tokenizer/
└── qwen3_1_7b_coreai_ane_w8_4k.h16p.aimodelc/

Pass that root directory to CoreAILanguageModel(resourcesAt:):

import CoreAILanguageModels
import FoundationModels

let resources = URL(fileURLWithPath: "/path/to/Qwen3-1.7B-CoreAI-ANE-W8-4K-h16p")
let model = try await CoreAILanguageModel(
    resourcesAt: resources,
    mode: .eager,
    kvCacheStrategy: .fixedSize
)

let session = LanguageModelSession(model: model)
let response = try await session.respond(to: "Reply with exactly COREAI-OK. /no_think")
print(response.content)

For an app target, embed the entire downloaded directory as a folder reference or deliver the same directory through your model-asset system. The companion app provides a minimal real-device integration.

Compatibility and limitations

• Direct AOT compatibility reported by coreai-build inspect: A17 Pro, h16p, iOS 27.0 or later.
• The current artifact has one successful eager-load/unload observation and 120/120 completed Run J generation observations. Energy use and a new execution-unit trace were not measured.
• This repository does not contain SafeTensors, GPTQ, GGUF, or MLX weights.
• The package is specialized for h16p; other Apple architectures require a separately compiled artifact.
• Preferred Neural Engine compute is a compilation preference, not proof that every operation executes exclusively on ANE.
• The model inherits the capabilities, limitations, and acceptable-use considerations of the Qwen3-1.7B base model.

Provenance and integrity

• Base model: Qwen/Qwen3-1.7B
• Hugging Face reference revision: 70d244cc86ccca08cf5af4e1e306ecf908b1ad5e
• Apple coreai-models base: 04a3fd6cfe9bfae9cf05b1f246cf915d930d1c0a
• Frozen recipe SHA-256: dab03ae1dd6c6290a7964e05ebcda7fe027c2bb240174fcf034e64a376be9d72
• Current compiler: coreai-build-3600.83.1 (Xcode 27 Beta 6, 27A5252f)
• Current .aimodel source SHA-256: 13ba3f73fcb7e090cd6ba1ca14b6b8903516ab608d451e94b9cdd750cfceda2c
• Current compiled main-h16p.mlirb SHA-256: 09f609775baa56b11ff3c91bfcb07b145930297289634fdc5514b2a5ab4dc7ca
• Immutable Run J artifact revision: 75bbe06906cb5d953e602e3e4fb6364187c81822
• Historical compiler: coreai-build-3600.75.3
• Historical public .aimodel source SHA-256: 5e885ec407f1b2690df5098d38b1bed4a3e66f4352c859fb2bb79666bc0aef73
• Historical compiled main-h16p.mlirb SHA-256: a7eefeef16708a324f9919890355eb92180ec85eef419ebd5822e8c8afd42f5f

The preceding public artifact was rebuilt under a neutral path and scanned recursively, including binary containers, before upload. The reusable publication gate is available in scripts/audit-public-artifact.sh.

License

The converted model is distributed under the base model's Apache License 2.0. See LICENSE and NOTICE.md. The GitHub reproduction repository's original code and documentation use BSD 3-Clause, with Apple-derived material retaining Apple's BSD terms.