--- license: apache-2.0 library_name: coremltools pipeline_tag: text-classification base_model: fastino/GLiNER2.5-Decide tags: - coreml - gliner2 - apple-silicon - classification --- # GLiNER2.5-Decide for Core ML Fixed-shape Core ML exports of the classification path of [fastino/GLiNER2.5-Decide](https://huggingface.co/fastino/GLiNER2.5-Decide) at revision `65624f1a0265b3f612bae66a2685a06b94a68a9d`. Fastino authored the Apache-2.0 source model; Fluid Inference converted it. Each package holds the DeBERTa-v3-large encoder and the shared label classifier (436,022,273 exported parameters). The span and count heads are not exported: this release covers classification only. One call scores up to four decision heads (for example intent, urgency and route) over the same text. The host tokenizes the schema with the upstream GLiNER2 processor and passes the position of every label marker; the package returns per-head logits. Single-label heads use softmax and multi-label heads use sigmoid with `cls_threshold`, exactly as `classify_text` does. ## Fast Decisions Scored on Fastino's [Fast Decisions](https://huggingface.co/datasets/fastino/fast-decisions) development split (revision `1a33070c`, 17 domains × 100 rows, 2,900 heads) with the protocol from the dataset card: one call per head with the labels as given, exact set match, mean of the 17 domain accuracies. The 512-token bucket fits every row. | Package (L512) | Size | Average | Pooled | Heads changed vs native | | --- | ---: | ---: | ---: | ---: | | native PyTorch | — | 62.93% | 61.38% | — | | `fp16` | 936 MB | 62.93% | 61.38% | 0 | | `w8` | 477 MB | 63.01% | 61.52% | 7 | | `lut6` | 361 MB | 63.03% | 61.52% | 42 | The card's published 60.2% for GLiNER2.5-Decide is on the held-out test split (300 rows per domain), which is not public; the card asks that development-split scores not be reported as the benchmark. The table above shows that the Core ML packages reproduce the native checkpoint on the public split, not a new benchmark result. Rejected variants, kept in `reports/` for reference: embedding-only W8 (805 MB, 63.00%) saves little over `w8`; per-tensor LUT4 (246 MB, 62.26%, 236 heads changed, −4 points on `paper_field`) loses accuracy. Grouped-channel palettization needs an iOS 18 deployment target and was not tried. ## Packages | Bucket | Tokens | Heads × labels | Fast Decisions heads that fit | `fp16` | `w8` | `lut6` | | --- | ---: | ---: | ---: | --- | --- | --- | | `L128_H4_K8` | 128 | 4 × 8 | 10.7% | 8.2 ms | 10.1 ms | 9.9 ms | | `L256_H4_K32` | 256 | 4 × 32 | 95.0% | 14.7 ms | 20.6 ms | 17.6 ms | | `L512_H4_K32` | 512 | 4 × 32 | 100% | 44.0 ms | 42.9 ms | 62.9 ms | Latency is the p50 of 100 Python `predict` calls for a three-head request with compute units `ALL` on an M5 Pro with macOS 27.0, after 10 warmups (`benchmark-latency.py`). This is a local measurement for one request shape, not a device-wide benchmark; p95 values are in `reports/latency.json`. W8 and LUT6 are weight-only compression: they shrink the download and do not make inference faster. `CPU_AND_NE` is much slower than `ALL` for this graph (689 ms versus 14.7 ms p50 for `fp16` L256), so use `ALL`. On the first 20 bucket-fitting rows per domain, the L128 and L256 packages agree with the native predictions: `fp16` 173/173 and 345/345, `w8` 172/173 and 345/345, `lut6` 169/173 and 342/345 (`reports/verify-packages-L128-L256.json`). ## Usage ```bash uv sync uv run python runtime.py --model-dir . --precision lut6 --length 256 \ --text "My transfer is still pending and I used the wrong sort code. Can you stop it?" \ --tasks '{"intent": ["transfer_pending", "transfer_cancel", "beneficiary_add", "card_lost"], "urgency": ["low", "normal", "high"]}' ``` ```python from runtime import CoreMLDecide model = CoreMLDecide(".", precision="w8", length=256) model.classify( "Battery dies before lunch, but the keyboard and the screen are the best I have used.", {"sentiment": ["positive", "negative", "mixed", "neutral"], "aspects": {"labels": ["battery", "keyboard", "screen", "price"], "multi_label": True, "cls_threshold": 0.4}}, ) ``` The runtime loads only the tokenizer files and the Core ML package; it does not load PyTorch weights. Requests that exceed the bucket raise `ValueError`. When downloading with `huggingface_hub.snapshot_download`, pass `local_dir=` so the packages are materialized; Core ML compilation rejects the symlinked weight files in the default Hub cache. ### Inputs and outputs | Name | Type | Shape | Meaning | | --- | --- | --- | --- | | `input_ids` | int32 | `[1, L]` | Schema and text tokens from the GLiNER2 span collator, padded | | `attention_mask` | int32 | `[1, L]` | 1 for real tokens | | `marker_indices` | int32 | `[1, 4, K]` | Token position of each `[L]` label marker, per head | | `marker_mask` | float32 | `[1, 4, K]` | 1 for used label slots | | `logits` | float32 | `[1, 4, K]` | Label logits; unused slots are −10⁴ | | `probabilities` | float32 | `[1, 4, K]` | Per-head softmax of `logits` | ## Reproduce ```bash uv sync uv run python convert-coreml.py --precision fp16 --length 512 --max-options 32 uv run python compress-coreml.py --method linear \ --source build/gliner2_decide_classification_fp16_L512_H4_K32.mlpackage \ --output build/gliner2_decide_classification_w8_L512_H4_K32.mlpackage uv run python compress-coreml.py --method lut --bits 6 --granularity per_tensor \ --source build/gliner2_decide_classification_fp16_L512_H4_K32.mlpackage \ --output build/gliner2_decide_classification_lut6_L512_H4_K32.mlpackage uv run python score-fast-decisions.py --backend native --out build/fast-decisions-native.json uv run python score-fast-decisions.py --backend coreml \ --package build/gliner2_decide_classification_lut6_L512_H4_K32.mlpackage --out build/fast-decisions-lut6.json ``` `convert-coreml.py` checks the export wrapper against the native logits (4.8e-7) and the converted package against native `classify_text` on three examples. The Core ML deployment target is iOS 17 / macOS 14. Source file hashes are in `assets.lock.json`; conversion, scoring, agreement and latency reports are in `reports/`.