Improve Omni Nano audio embeddings and complete benchmark rankings

LICENSE-CLAP

@@ -0,0 +1,121 @@
+Creative Commons Legal Code
+
+CC0 1.0 Universal
+
+    CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE
+    LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN
+    ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS
+    INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES
+    REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS
+    PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM
+    THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED
+    HEREUNDER.
+
+Statement of Purpose
+
+The laws of most jurisdictions throughout the world automatically confer
+exclusive Copyright and Related Rights (defined below) upon the creator
+and subsequent owner(s) (each and all, an "owner") of an original work of
+authorship and/or a database (each, a "Work").
+
+Certain owners wish to permanently relinquish those rights to a Work for
+the purpose of contributing to a commons of creative, cultural and
+scientific works ("Commons") that the public can reliably and without fear
+of later claims of infringement build upon, modify, incorporate in other
+works, reuse and redistribute as freely as possible in any form whatsoever
+and for any purposes, including without limitation commercial purposes.
+These owners may contribute to the Commons to promote the ideal of a free
+culture and the further production of creative, cultural and scientific
+works, or to gain reputation or greater distribution for their Work in
+part through the use and efforts of others.
+
+For these and/or other purposes and motivations, and without any
+expectation of additional consideration or compensation, the person
+associating CC0 with a Work (the "Affirmer"), to the extent that he or she
+is an owner of Copyright and Related Rights in the Work, voluntarily
+elects to apply CC0 to the Work and publicly distribute the Work under its
+terms, with knowledge of his or her Copyright and Related Rights in the
+Work and the meaning and intended legal effect of CC0 on those rights.
+
+1. Copyright and Related Rights. A Work made available under CC0 may be
+protected by copyright and related or neighboring rights ("Copyright and
+Related Rights"). Copyright and Related Rights include, but are not
+limited to, the following:
+
+  i. the right to reproduce, adapt, distribute, perform, display,
+     communicate, and translate a Work;
+ ii. moral rights retained by the original author(s) and/or performer(s);
+iii. publicity and privacy rights pertaining to a person's image or
+     likeness depicted in a Work;
+ iv. rights protecting against unfair competition in regards to a Work,
+     subject to the limitations in paragraph 4(a), below;
+  v. rights protecting the extraction, dissemination, use and reuse of data
+     in a Work;
+ vi. database rights (such as those arising under Directive 96/9/EC of the
+     European Parliament and of the Council of 11 March 1996 on the legal
+     protection of databases, and under any national implementation
+     thereof, including any amended or successor version of such
+     directive); and
+vii. other similar, equivalent or corresponding rights throughout the
+     world based on applicable law or treaty, and any national
+     implementations thereof.
+
+2. Waiver. To the greatest extent permitted by, but not in contravention
+of, applicable law, Affirmer hereby overtly, fully, permanently,
+irrevocably and unconditionally waives, abandons, and surrenders all of
+Affirmer's Copyright and Related Rights and associated claims and causes
+of action, whether now known or unknown (including existing as well as
+future claims and causes of action), in the Work (i) in all territories
+worldwide, (ii) for the maximum duration provided by applicable law or
+treaty (including future time extensions), (iii) in any current or future
+medium and for any number of copies, and (iv) for any purpose whatsoever,
+including without limitation commercial, advertising or promotional
+purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each
+member of the public at large and to the detriment of Affirmer's heirs and
+successors, fully intending that such Waiver shall not be subject to
+revocation, rescission, cancellation, termination, or any other legal or
+equitable action to disrupt the quiet enjoyment of the Work by the public
+as contemplated by Affirmer's express Statement of Purpose.
+
+3. Public License Fallback. Should any part of the Waiver for any reason
+be judged legally invalid or ineffective under applicable law, then the
+Waiver shall be preserved to the maximum extent permitted taking into
+account Affirmer's express Statement of Purpose. In addition, to the
+extent the Waiver is so judged Affirmer hereby grants to each affected
+person a royalty-free, non transferable, non sublicensable, non exclusive,
+irrevocable and unconditional license to exercise Affirmer's Copyright and
+Related Rights in the Work (i) in all territories worldwide, (ii) for the
+maximum duration provided by applicable law or treaty (including future
+time extensions), (iii) in any current or future medium and for any number
+of copies, and (iv) for any purpose whatsoever, including without
+limitation commercial, advertising or promotional purposes (the
+"License"). The License shall be deemed effective as of the date CC0 was
+applied by Affirmer to the Work. Should any part of the License for any
+reason be judged legally invalid or ineffective under applicable law, such
+partial invalidity or ineffectiveness shall not invalidate the remainder
+of the License, and in such case Affirmer hereby affirms that he or she
+will not (i) exercise any of his or her remaining Copyright and Related
+Rights in the Work or (ii) assert any associated claims and causes of
+action with respect to the Work, in either case contrary to Affirmer's
+express Statement of Purpose.
+
+4. Limitations and Disclaimers.
+
+ a. No trademark or patent rights held by Affirmer are waived, abandoned,
+    surrendered, licensed or otherwise affected by this document.
+ b. Affirmer offers the Work as-is and makes no representations or
+    warranties of any kind concerning the Work, express, implied,
+    statutory or otherwise, including without limitation warranties of
+    title, merchantability, fitness for a particular purpose, non
+    infringement, or the absence of latent or other defects, accuracy, or
+    the present or absence of errors, whether or not discoverable, all to
+    the greatest extent permissible under applicable law.
+ c. Affirmer disclaims responsibility for clearing rights of other persons
+    that may apply to the Work or any use thereof, including without
+    limitation any person's Copyright and Related Rights in the Work.
+    Further, Affirmer disclaims responsibility for obtaining any necessary
+    consents, permissions or other rights required for any use of the
+    Work.
+ d. Affirmer understands and acknowledges that Creative Commons is not a
+    party to this document and has no duty or obligation with respect to
+    this CC0 or use of the Work.

NOTICE

@@ -25,3 +25,6 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 SOFTWARE.
 
 Text component: avsolatorio/GIST-small-Embedding-v0, revision 75e62fd210b9fde790430e0b2f040b0b00a021b1. Its official model card declares MIT; the complete pinned card is retained in licenses/GIST-MODEL-CARD.md. The card identifies BAAI/bge-small-en-v1.5 as the upstream base and discloses its training data.
+
+Additional frozen audio branch: laion/clap-htsat-unfused, revision 8fa0f1c6d0433df6e97c127f64b2a1d6c0dcda8a, Apache-2.0.
+Endpoint window pooling is a Vela readout. See LICENSE-CLAP for the unchanged upstream license.

README.md

@@ -22,7 +22,7 @@ tags:
 
 # Vela Omni Nano
 
-Text, images and speech in one embedding space. Vela Omni Nano supports multimodal search, routing and clustering with normalized vectors that can be compared directly.
+Text, images, speech and environmental audio in one embedding space. Vela Omni Nano supports multimodal search, routing and clustering with normalized vectors that can be compared directly.
 
 [Try Vela Studio](https://huggingface.co/spaces/llm-semantic-router/vela-studio) · [Vela collection](https://huggingface.co/collections/llm-semantic-router/vela-10-6aa555ba70cc6997d6d67798) · [Detailed evaluation](benchmarks/EVALUATION.md)
 
@@ -30,15 +30,15 @@ Text, images and speech in one embedding space. Vela Omni Nano supports multimod
 
 | Feature | Value |
 | --- | --- |
-| Modalities | Text, images, speech |
-| Total parameters | 135.4M (135,383,808) |
+| Modalities | Text, images, speech and environmental audio |
+| Total parameters | 163.8M (163,771,288) |
 | Embedding dimensions | 384 |
 | Text context | 512 tokens, including special tokens |
-| Audio input | Mono, 16 kHz, up to 30 seconds |
+| Audio input | Original PCM at 16, 44.1 or 48 kHz; up to 30 seconds |
 | Output | L2-normalized vectors; cosine similarity |
 | License | Apache 2.0; see component attribution |
 
-The smaller inference package preserves the encoding computation and all reported scores. [Equivalence and storage details](benchmarks/inference-equivalence.json).
+A frozen CLAP audio branch adds environmental-sound information to the existing speech representation. Text and image computations are retained; audio embeddings are newly trained and evaluated. [Architecture and measured identity](benchmarks/component-equivalence.json).
 
 ## Evaluation
 
@@ -50,8 +50,8 @@ Scores are 0–100; higher is better. The comparison uses the same held-out eval
 | MASSIVE English · Accuracy | 65.95 | **81.97** |
 | COCO · Image → text · R@1 | 40.83 | **60.87** |
 | COCO · Text → image · R@1 | 30.18 | **55.82** |
-| LibriSpeech · Audio → text · R@1 | 4.21 | **17.16** |
-| LibriSpeech · Text → audio · R@1 | 9.58 | **25.44** |
+| LibriSpeech · Audio → text · R@1 | 4.21 | **16.12** |
+| LibriSpeech · Text → audio · R@1 | 9.58 | **20.34** |
 
 The common protocol uses labeled TRAIN prototypes for text classification and all matching positives for retrieval; it is separate from official MTEB classification. [All 14 metrics, Macro-F1, exact counts and uncertainty](benchmarks/EVALUATION.md#routing-and-cross-modal-retrieval).
 
@@ -61,18 +61,23 @@ The primary metric, **Mean(TaskType)**, weights each task type equally. Mean(Tas
 
 | Benchmark | Mean(TaskType) | Global rank | Rank at ≤ size | Mean(Task) |
 | --- | ---: | ---: | ---: | ---: |
-| MTEB English v2 · 41 tasks | 60.78 | 65/188 | 3/66 | 64.88 |
-| MAEB audio-only · 19 tasks | 46.97 | 32/64 | 8/22 | 39.77 |
+| MTEB English v2 · 41 tasks | 60.78 | 65/188 | 5/75 | 64.88 |
+| MAEB audio-only · 19 tasks | 52.34 | 19/64 | 6/27 | 43.59 |
 
-Nano ranks **3/66** on the complete English panel among models with no more than its 135.4M total parameters, **0.61 points** behind the highest-scoring model in that group. These are snapshot-relative comparisons across reported protocols, including single-modality specialists. The original small has not been evaluated on these complete panels. [Full rankings and both aggregate metrics](benchmarks/complete-panel-ranks.md) · [All task scores and methods](benchmarks/EVALUATION.md).
+Nano ranks **5/75** on the English panel and **6/27** on the audio panel among models with no more than its 163.8M total parameters. These are snapshot-relative comparisons across reported protocols, including single-modality specialists. The original small has not been evaluated on these complete panels. [Full rankings and both aggregate metrics](benchmarks/complete-panel-ranks.md) · [All task scores and methods](benchmarks/EVALUATION.md).
+
+Audio Mean(TaskType) improves from 46.97 to 52.34 over the previous Nano. Parameters rise 20.97%; speech–text retrieval and VehicleSoundClustering regress. [Complete gains and trade-offs](benchmarks/release-history.md).
 
 ### Quality and model size
 
 The plots include every complete model with a known size. The dashed line shows the observed Pareto frontier; both Vela models remain plotted even when below it. Known-size peers below the frontier provide additional reference points.
 
-![Complete English benchmark: size–quality comparison and ranking](assets/complete-panel-english41-nano.png)
-
-![Complete audio benchmark: size–quality comparison and ranking](assets/complete-panel-audio19-mini.png)
+<table>
+<tr>
+<td width="50%"><a href="assets/complete-panel-english41-nano.png"><img src="assets/complete-panel-english41-nano.png" alt="Complete English41: size, quality and ranking; click for full resolution" /></a></td>
+<td width="50%"><a href="assets/complete-panel-audio19-mini.png"><img src="assets/complete-panel-audio19-mini.png" alt="Complete audio19: size, quality and ranking; click for full resolution" /></a></td>
+</tr>
+</table>
 
 ### Selected task strengths
 
@@ -90,7 +95,7 @@ These task-level comparisons highlight specific strengths; they do not establish
 ## Usage
 
 
-Use PyTorch, Transformers 4.57.6, Hugging Face Hub, safetensors, NumPy, and Pillow:
+Use PyTorch and matching torchaudio, Transformers 4.57.6, Hugging Face Hub, safetensors, NumPy, and Pillow:
 
 ```python
 import sys
@@ -107,12 +112,14 @@ print(vectors.shape)  # (2, 384)
 
 Text inputs support up to 512 tokens, including special tokens. Longer inputs raise `ValueError`; shorten or explicitly chunk them.
 
-Pass a list of Pillow images to `model.encode_image(images)`. Pass a list of mono NumPy waveforms to `model.encode_audio(waveforms, sampling_rate=16000)`; each waveform must be at most 30 seconds. Compare normalized vectors with their dot product. To route media, embed each destination’s name and description, then select the closest vector. To discover routing categories, cluster media embeddings and inspect each group. Similarity scores are rankings, not calibrated probabilities.
+Pass a list of Pillow images to `model.encode_image(images)`. Pass original NumPy waveforms to `model.encode_audio(waveforms, sampling_rate=48000)` using their actual 16,000, 44,100 or 48,000 Hz rate; each waveform must be at most 30 seconds. Mono or channels-first arrays are supported. Keep the original waveform: the speech and CLAP branches independently derive their 16 kHz and 48 kHz inputs. Do not downsample to 16 kHz before calling the API when higher-rate PCM is available. Compare normalized vectors with their dot product. To route media, embed each destination’s name and description, then select the closest vector. To discover routing categories, cluster media embeddings and inspect each group. Similarity scores are rankings, not calibrated probabilities.
 
 ## Training and license
 
 Cross-modal alignment uses COCO image–caption pairs from the CC-BY 2.0 image subset and [LibriSpeech](https://www.openslr.org/12/) speech–transcript pairs (CC-BY 4.0). COCO annotations are CC-BY 4.0. The frozen [GIST text backbone](https://huggingface.co/avsolatorio/GIST-small-Embedding-v0) is MIT-licensed; its pinned model card and component notices are included.
 
+Audio residual alignment uses 3,299 FSD50K TRAIN recordings under CC0 or CC-BY 3.0; [per-recording attribution](training/fsd50k-attributions.jsonl) is included. The frozen [CLAP unfused audio component](https://huggingface.co/laion/clap-htsat-unfused) is Apache 2.0; its original license is retained as [LICENSE-CLAP](LICENSE-CLAP).
+
 The repository includes the native model code, component configurations, and tokenizer and processor files. See [NOTICE](./NOTICE) and [LICENSE](./LICENSE) for component attribution and license terms.
 
 [Explore the Vela model collection](https://huggingface.co/collections/llm-semantic-router/vela-10-6aa555ba70cc6997d6d67798)

benchmarks/EVALUATION.md

@@ -1,12 +1,10 @@
-# Evaluation details
+# Evaluation
 
-The primary comparator is [llm-semantic-router/multi-modal-embed-small](https://huggingface.co/llm-semantic-router/multi-modal-embed-small/tree/fdf8e01b7b0f3a69ac1ac8e2a64dcb1ede177ba4), pinned to `fdf8e01b7b0f3a69ac1ac8e2a64dcb1ede177ba4`. It is the original small model, not a previous Vela release. Current Nano native identity is `50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa`. Scores are 0–100; bold applies only to an unrounded improvement over this original comparator. [Original checkpoint and comparison evidence](original-baseline.json).
+Current native artifact: `474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2`. Scores are percentages or official scores multiplied by 100. **Bold means an improvement over the original small**, measured on the same named protocol. Known test pools are reused across releases.
 
 ## Routing and cross-modal retrieval
 
-Both models use the same held-out examples, complete candidate pools and 128-token text limit. These known evaluation pools have been reused across releases, so this is a progress comparison rather than a new blind test. Banking77 and MASSIVE use nearest normalized class prototypes from labeled TRAIN examples; this task-adapted protocol is distinct from official MTEB frozen-embedding classification. Retrieval uses cosine scores and all matching positives.
-
-| Metric | [Original small](https://huggingface.co/llm-semantic-router/multi-modal-embed-small/tree/fdf8e01b7b0f3a69ac1ac8e2a64dcb1ede177ba4) | Current Nano |
+| Metric | Original small | Current Nano |
 | --- | ---: | ---: |
 | Banking77 · Accuracy | 70.4221 | **87.9870** |
 | MASSIVE English · Accuracy | 65.9489 | **81.9650** |
@@ -16,108 +14,58 @@ Both models use the same held-out examples, complete candidate pools and 128-tok
 | COCO · Text → image · R@1 | 30.1823 | **55.8202** |
 | COCO · Text → image · R@5 | 59.4168 | **85.2248** |
 | COCO · Text → image · R@10 | 74.2892 | **94.2649** |
-| LibriSpeech · Audio → text · R@1 | 4.2129 | **17.1582** |
-| LibriSpeech · Audio → text · R@5 | 11.9877 | **36.6526** |
-| LibriSpeech · Audio → text · R@10 | 19.0349 | **46.9935** |
-| LibriSpeech · Text → audio · R@1 | 9.5785 | **25.4406** |
-| LibriSpeech · Text → audio · R@5 | 22.5287 | **46.1303** |
-| LibriSpeech · Text → audio · R@10 | 30.6897 | **55.9004** |
+| LibriSpeech · Audio → text · R@1 | 4.2129 | **16.1241** |
+| LibriSpeech · Audio → text · R@5 | 11.9877 | **34.8142** |
+| LibriSpeech · Audio → text · R@10 | 19.0349 | **45.6147** |
+| LibriSpeech · Text → audio · R@1 | 9.5785 | **20.3448** |
+| LibriSpeech · Text → audio · R@5 | 22.5287 | **39.9617** |
+| LibriSpeech · Text → audio · R@10 | 30.6897 | **50.6130** |
 | Banking77 · Macro-F1 | 70.2078 | **87.9168** |
 | MASSIVE English · Macro-F1 | 61.7178 | **79.2322** |
 
-The text pools contain 3,080 Banking77 and 2,972 MASSIVE queries, with 77 and 60 prototype classes. COCO uses 823 images and 4,115 captions; LibriSpeech uses 2,611 clips and 2,610 unique transcripts. Macro-F1 equally averages all declared classes (undefined F1 is zero). [scores.json](../scores.json) retains exact numerators, denominators, class definitions and paired uncertainty. Historical paired intervals keep their actual comparator and are not relabeled.
-
-## Complete standard panels
-
-| Benchmark / metric | Current Nano |
-| --- | ---: |
-| English v2 · 41/41 tasks · mean task | 64.88 |
-| English v2 · mean task type | 60.78 |
-| MAEB audio-only · 19/19 tasks · mean task | 39.77 |
-| MAEB audio-only · 134 slots · mean task type | 46.97 |
-
-English contains 41 tasks and seven task types. The MAEB audio-only panel contains 19 tasks, five task types and 134 subset/split slots, excluding joint-modality tasks. Aggregation uses official within-task results, then equal task or task-type means. The original small was not evaluated in this matched complete-panel protocol; no score is inferred from common-protocol results or a previous Vela release.
-
-Nano uses the 135,383,808-parameter inference package. [Inference equivalence](inference-equivalence.json) preserves the original evaluated 96d artifact and all actual text/audio component identities. The participating weights and computation are unchanged; this is not a benchmark rerun.
+The text pools use TRAIN class prototypes; they are distinct from official MTEB classifier probes. All image/audio retrieval pools and all annotated positives are retained. [Counts, class F1 and paired intervals](../scores.json). Fresh paired intervals compare the previous Vela Nano; no new original-small interval is claimed.
 
-### Every English task
+## Complete panels
 
-| Task | Main metric | Current Nano |
+| Panel | Mean(Task) | Mean(TaskType) |
 | --- | ---: | ---: |
-| ArguAna | ndcg_at_10 | 59.2460 |
-| ArXivHierarchicalClusteringP2P | v_measure | 64.8885 |
-| ArXivHierarchicalClusteringS2S | v_measure | 57.4432 |
-| AskUbuntuDupQuestions | map_at_1000 | 62.3290 |
-| BIOSSES | cosine_spearman | 86.9875 |
-| Banking77Classification | accuracy | 82.1429 |
-| BiorxivClusteringP2P.v2 | v_measure | 41.3138 |
-| CQADupstackGamingRetrieval | ndcg_at_10 | 56.9790 |
-| CQADupstackUnixRetrieval | ndcg_at_10 | 39.5860 |
-| ClimateFEVERHardNegatives | ndcg_at_10 | 31.8460 |
-| FEVERHardNegatives | ndcg_at_10 | 87.5760 |
-| FiQA2018 | ndcg_at_10 | 39.1430 |
-| HotpotQAHardNegatives | ndcg_at_10 | 66.3490 |
-| ImdbClassification | accuracy | 91.9476 |
-| MTOPDomainClassification | accuracy | 94.9179 |
-| MassiveIntentClassification | accuracy | 70.9684 |
-| MassiveScenarioClassification | accuracy | 76.1130 |
-| MedrxivClusteringP2P.v2 | v_measure | 39.8302 |
-| MedrxivClusteringS2S.v2 | v_measure | 37.8751 |
-| MindSmallReranking | max_over_subqueries_map_at_1000 | 32.3660 |
-| SCIDOCS | ndcg_at_10 | 21.8890 |
-| SICK-R | cosine_spearman | 80.5317 |
-| STS12 | cosine_spearman | 75.5659 |
-| STS13 | cosine_spearman | 86.2635 |
-| STS14 | cosine_spearman | 82.2988 |
-| STS15 | cosine_spearman | 88.7365 |
-| STSBenchmark | cosine_spearman | 87.0782 |
-| SprintDuplicateQuestions | max_ap | 95.7996 |
-| StackExchangeClustering.v2 | v_measure | 58.4244 |
-| StackExchangeClusteringP2P.v2 | v_measure | 41.0530 |
-| TRECCOVID | ndcg_at_10 | 69.1210 |
-| Touche2020Retrieval.v3 | ndcg_at_10 | 48.3630 |
-| ToxicConversationsClassification | accuracy | 71.9043 |
-| TweetSentimentExtractionClassification | accuracy | 62.7929 |
-| TwentyNewsgroupsClustering.v2 | v_measure | 50.8726 |
-| TwitterSemEval2015 | max_ap | 72.9515 |
-| TwitterURLCorpus | max_ap | 85.3034 |
-| SummEvalSummarization.v2 | cosine_spearman | 31.8344 |
-| AmazonCounterfactualClassification | accuracy | 71.8507 |
-| STS17 | cosine_spearman | 89.0210 |
-| STS22.v2 | cosine_spearman | 68.7533 |
+| English · 41 tasks / 7 types | 64.8843 | 60.7818 |
+| Audio · 19 tasks / 134 slots / 5 types | 43.5938 | 52.3354 |
+
+English retains its actual evaluated artifact through exact text-component applicability. Every audio task is newly evaluated on this full native artifact. The original small has not been measured on these complete panels. [Full rankings](complete-panel-ranks.md).
 
 ### Every audio task
 
-| Task | Main metric | Current Nano |
-| --- | ---: | ---: |
-| JamAltArtistA2ARetrieval | ndcg_at_10 | 68.9005 |
-| BeijingOpera | accuracy | 68.6791 |
-| BirdCLEF | accuracy | 12.9000 |
-| CREMA_D | accuracy | 29.7637 |
-| CommonLanguageAgeDetection | accuracy | 16.0400 |
-| GTZANGenre | accuracy | 50.3000 |
-| IEMOCAPGender | accuracy | 58.4519 |
-| MInDS14 | accuracy | 23.3212 |
-| MridinghamTonic | accuracy | 32.3632 |
-| SIBFLEURS | accuracy | 18.3047 |
-| VoxCelebSA | accuracy | 33.1691 |
-| VoxPopuliLanguageID | accuracy | 91.8000 |
-| CREMA_DClustering | v_measure | 2.3602 |
-| VehicleSoundClustering | v_measure | 13.8007 |
-| VoxPopuliGenderClustering | v_measure | 0.1045 |
-| CREMADPairClassification | max_ap | 54.2012 |
-| NMSQAPairClassification | max_ap | 63.2834 |
-| VoxPopuliAccentPairClassification | max_ap | 54.0220 |
-| GTZANAudioReranking | map_at_1000 | 63.7940 |
-
-[English JSON](mteb-eng-v2.json) and [audio JSON](maeb-audio-only.json) retain all raw metrics, dataset revisions, repeated probes, subset/split coverage and actual evaluated identities. Undefined auxiliary statistics remain null; primary scores are finite.
-
-## Selected standard tasks
-
-MTEB 2.21.0, FP32, batch size 8, seed 20260918. Eleven tasks and twelve split rows; MTOP validation is descriptive. Native context limits, official preprocessing, sampling, probes and clustering are preserved; no task instruction is added. This is not a complete multilingual or image benchmark.
+| Task | Primary metric | Previous Nano | Current Nano | Delta pp |
+| --- | --- | ---: | ---: | ---: |
+| JamAltArtistA2ARetrieval | ndcg_at_10 | 68.9005 | 85.8015 | +16.9010 |
+| BeijingOpera | accuracy | 68.6791 | 79.2287 | +10.5496 |
+| BirdCLEF | accuracy | 12.9000 | 15.2000 | +2.3000 |
+| CREMA_D | accuracy | 29.7637 | 32.0481 | +2.2845 |
+| CommonLanguageAgeDetection | accuracy | 16.0400 | 16.6550 | +0.6150 |
+| GTZANGenre | accuracy | 50.3000 | 61.7000 | +11.4000 |
+| IEMOCAPGender | accuracy | 58.4519 | 61.6897 | +3.2379 |
+| MInDS14 | accuracy | 23.3212 | 21.8520 | -1.4691 |
+| MridinghamTonic | accuracy | 32.3632 | 58.4198 | +26.0566 |
+| SIBFLEURS | accuracy | 18.3047 | 18.0725 | -0.2321 |
+| VoxCelebSA | accuracy | 33.1691 | 33.1685 | -0.0006 |
+| VoxPopuliLanguageID | accuracy | 91.8000 | 91.2000 | -0.6000 |
+| CREMA_DClustering | v_measure | 2.3602 | 5.2143 | +2.8542 |
+| VehicleSoundClustering | v_measure | 13.8007 | 5.2754 | -8.5253 |
+| VoxPopuliGenderClustering | v_measure | 0.1045 | 0.0233 | -0.0813 |
+| CREMADPairClassification | max_ap | 54.2012 | 55.2073 | +1.0060 |
+| NMSQAPairClassification | max_ap | 63.2834 | 62.8993 | -0.3841 |
+| VoxPopuliAccentPairClassification | max_ap | 54.0220 | 54.1514 | +0.1294 |
+| GTZANAudioReranking | map_at_1000 | 63.7940 | 70.4760 | +6.6820 |
+
+[English raw metrics](mteb-eng-v2.json) · [Audio raw metrics](maeb-audio-only.json) · [Exact component applicability](component-equivalence.json).
+
+## Selected standard protocols
+
+These 11 selected tasks are not a complete MTEB, MMTEB, MIEB or MAEB benchmark. The original small retains its native 128-token context; Nano uses 512. Official FP32 batch-eight evaluation uses the same pinned datasets, splits and seed. MTOP validation and test remain separate.
 
 | Task / split | Main metric | Original small | Current Nano |
-| --- | ---: | ---: | ---: |
+| --- | --- | ---: | ---: |
 | SICK-R / test | cosine_spearman | 71.3902 | **80.5317** |
 | STSBenchmark / test | cosine_spearman | 77.2987 | **87.0782** |
 | ArguAna / test | ndcg_at_10 | 32.8890 | **59.2460** |
@@ -127,16 +75,20 @@ MTEB 2.21.0, FP32, batch size 8, seed 20260918. Eleven tasks and twelve split ro
 | OxfordPets / test | accuracy | 41.8479 | **91.2674** |
 | TinyImageNetClustering / valid | nmi | 46.2849 | **61.6919** |
 | OxfordPetsZeroShot / test | accuracy | 16.4077 | 8.0131 |
-| CREMA_D / train | accuracy | 29.9520 | 29.7637 |
-| CREMA_DClustering / train | v_measure | 2.4058 | 2.3602 |
-| SpeechCommandsZeroshotv0.02 / test | accuracy | 5.5228 | **14.1139** |
+| CREMA_D / train | accuracy | 29.9520 | **32.0481** |
+| CREMA_DClustering / train | v_measure | 2.4058 | **5.2143** |
+| SpeechCommandsZeroshotv0.02 / test | accuracy | 5.5228 | **15.7094** |
+
+Eight text/image tasks retain the actual previous artifact measurement through exact component identity; three audio tasks are fresh. Every named auxiliary metric remains in scores.json.
+
+## Event routing diagnostic
 
-Original small uses its native 128-token limit; current Nano uses 512. All rows use the same pinned task revisions, official probes, sampling and metrics. Bold marks an unrounded improvement over the original comparator; lower values remain visible. MTOP validation is descriptive. [Full original results and identities](original-standard.json), including every repeated-probe statistic, accompany [current results](../scores.json).
+On the fixed 840-clip FSD50K evaluation pool, positive recall@5 is 66.2157% versus 27.3296% previously. The 3,299 TRAIN clips define 149 class prototypes; all 153 known labels retain their annotated-positive denominator, including labels without a TRAIN prototype. This diagnostic is outside the weighted product score. [Protocol and R@1/5/10](event-audio-diagnostic.json).
 
-## Release history and limitations
+## Resources and data
 
-[Release history](release-history.md) keeps all measured improvements and regressions against previous Vela versions, including retrieval and zero-shot regressions, with their original identities. Those historical comparisons do not replace the original small comparator above. [Product progress JSON](net-progress.json) retains its explicit weights; it is not an official aggregate or an overall SOTA claim.
+Total parameters: 163,771,288; FP32 vector storage: 1,536 bytes, excluding index metadata. Native weights: 655,583,800 bytes. No isolated latency or deployment-memory claim is made for this update. The additional CLAP branch costs inference computation as well as parameters.
 
-Cross-modal alignment uses licensed COCO and LibriSpeech TRAIN examples. The text foundation's pretraining overlap with downstream benchmarks is not ruled out. Benchmark encoders are frozen; official probes use their declared labeled data. No current-artifact latency or peak-memory improvement is claimed. Native operation and context limits are described in the model card.
+The existing Whisper Tiny encoder and affine head stay frozen. A frozen CLAP audio tower and a 384×512 residual projection are added; only that projection is trained for one fixed 600-step run. Speech alignment and a pointwise parent anchor use 28,535 LibriSpeech TRAIN clips; known-positive event labels and CLAP relational geometry use 3,299 FSD50K TRAIN clips. Center/scale and the geometry coefficient use TRAIN only.
 
-[Every plotted observation](pareto-data.json) and [Pareto methodology](pareto-methodology.md) preserve reported registry snapshots and separately measured peers. Known peers remain in the plots; individual task frontiers do not establish overall size SOTA.
+The frozen GIST backbone discloses MTEB classification training in its model card; independent pretraining-data separation is not claimed. Original per-record licenses and complete attribution are retained. [Release progress and all regressions](release-history.md).

benchmarks/complete-panel-data.json

@@ -24,14 +24,16 @@
   ],
   "sources": {
     "nano_English41": {
-      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/mteb-eng-v2.json",
-      "sha256": "733f6a58faae3d04e1b5a59de73085b156dc9987ba0e04e28548c17dc5904cec",
-      "evidence_type": "Vela_published_report"
+      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/mteb-eng-v2.json",
+      "evidence_type": "Vela_release_report",
+      "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+      "sha256": "f3daecad1cf431630598efb04f3667431e3420239240f28a51568a74cef0c64f"
     },
     "nano_audio19": {
-      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/maeb-audio-only.json",
-      "sha256": "e0259c59e04649f42cd2f004d57b525c79442297d28498a7434bddb4b000bdf9",
-      "evidence_type": "Vela_published_report"
+      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/maeb-audio-only.json",
+      "evidence_type": "Vela_release_report",
+      "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+      "sha256": "4fc7f217745eb0aa998bf0f8e044bd1fa5de2f73f1a7dead7b829a62c48669ff"
     },
     "mini_English41": {
       "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Mini/resolve/main/benchmarks/mteb-eng-v2.json",
@@ -64,14 +66,20 @@
       "sha256": "8bbbd3a362d84a6e8f030803259db69e932e5997b1c7af1a276a0d17862eb184",
       "evidence_type": "exact_component_identity_report",
       "native_artifact_sha256": "d3a742fbe0d833b0a72e5104e0381209b7d3da78f948d3439a5da7d986232731"
+    },
+    "nano_component_equivalence": {
+      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/component-equivalence.json",
+      "evidence_type": "Vela_release_report",
+      "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+      "sha256": "ebd40f9a887791d1736132c37f510a69e18a03e247422853f54a42b417b0db9b"
     }
   },
   "models": {
     "nano": {
       "repo_id": "llm-semantic-router/Vela-1.0-Omni-Nano",
-      "release_commit": "d8ac5b5ac2274a501fc61aeb5be70cec1855a806",
-      "parameters_exact": 135383808,
-      "current_native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
+      "release_commit": null,
+      "parameters_exact": 163771288,
+      "current_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
       "task_evidence": {
         "English41": {
           "source_id": "nano_English41",
@@ -79,47 +87,58 @@
             "model": "local/Vela-Omni-Nano-GIST-Rotation",
             "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
             "native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+            "revision_semantics": "local native artifact fingerprint, not a Hugging Face commit",
             "component_applicability": {
               "component": "text",
-              "applies_to_revision": "self",
-              "applies_to_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
               "evaluated_model": "local/Vela-Omni-Nano-GIST-Rotation",
-              "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
               "evaluated_native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
-              "revision_kind": "local native artifact fingerprint, not a Hugging Face commit",
-              "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/component-equivalence.json",
+              "applies_to_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+              "prior_component_applicability": {
+                "model": "local/Vela-Omni-Nano-GIST-Rotation",
+                "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+                "native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+                "component_applicability": {
+                  "component": "text",
+                  "applies_to_revision": "self",
+                  "applies_to_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
+                  "evaluated_model": "local/Vela-Omni-Nano-GIST-Rotation",
+                  "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+                  "evaluated_native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+                  "revision_kind": "local native artifact fingerprint, not a Hugging Face commit",
+                  "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/component-equivalence.json",
+                  "new_full_panel_inference": false,
+                  "basis": "199 exact text tensors and byte-identical inference/tokenizer/configuration under the same official evaluator protocol."
+                },
+                "revision_semantics": "local native artifact fingerprint, not a Hugging Face commit",
+                "inference_profile_applicability": {
+                  "revision": "self",
+                  "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
+                  "parameters": 135383808,
+                  "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/inference-equivalence.json",
+                  "new_benchmark_run": false
+                }
+              },
+              "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/component-equivalence.json",
               "new_full_panel_inference": false,
-              "basis": "199 exact text tensors and byte-identical inference/tokenizer/configuration under the same official evaluator protocol."
-            },
-            "revision_semantics": "local native artifact fingerprint, not a Hugging Face commit",
-            "inference_profile_applicability": {
-              "revision": "self",
-              "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-              "parameters": 135383808,
-              "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/inference-equivalence.json",
-              "new_benchmark_run": false
+              "basis": "The frozen text tensors, tokenizer, configuration and encoding computation are unchanged. The original measured text identity is retained."
             }
           },
-          "self_reference_interpretation": "d8ac5b5ac2274a501fc61aeb5be70cec1855a806"
+          "self_reference_interpretation": "Nano native fingerprint 474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2"
         },
         "audio19": {
           "source_id": "nano_audio19",
           "identity": {
             "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-            "evaluated_revision": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-            "native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-            "revision_semantics": "Native artifact fingerprint identifying the evaluated weights and inference files; the release commit is the repository revision containing this report.",
-            "inference_profile_applicability": {
-              "revision": "self",
-              "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-              "parameters": 135383808,
-              "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/inference-equivalence.json",
-              "new_benchmark_run": false
-            }
+            "evaluated_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+            "release_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+            "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/maeb-audio-only.json",
+            "new_full_panel_inference": true
           },
-          "self_reference_interpretation": "d8ac5b5ac2274a501fc61aeb5be70cec1855a806"
+          "self_reference_interpretation": "Nano native fingerprint 474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2"
         }
-      }
+      },
+      "release_reference": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano",
+      "release_revision_semantics": "Exact native fingerprint below; Hub commit is not encoded in this file."
     },
     "mini": {
       "repo_id": "llm-semantic-router/Vela-1.0-Omni-Mini",
@@ -15579,10 +15598,10 @@
           "id": "current:nano",
           "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
           "kind": "nano",
-          "parameters_exact": 135383808,
-          "parameters_billion": 0.135383808,
+          "parameters_exact": 163771288,
+          "parameters_billion": 0.163771288,
           "parameter_basis": "exact full released native tensor count, all modalities",
-          "current_native_fingerprint": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
+          "current_native_fingerprint": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
           "mean_task": 0.6488431514310018,
           "mean_task_type": 0.6078184975654924,
           "by_task_type": {
@@ -22560,23 +22579,25 @@
             "global_rank": 65,
             "global_population": 188,
             "tied_other_models": 0,
-            "same_or_smaller_rank": 5,
-            "same_or_smaller_population": 66,
+            "same_or_smaller_rank": 7,
+            "same_or_smaller_population": 75,
             "best_same_or_smaller_other_model": {
-              "id": "scores_MTEB_eng_v2:row:61",
-              "model": "avsolatorio/GIST-Embedding-v0",
+              "id": "scores_MTEB_eng_v2:row:68",
+              "model": "Tarka-AIR/Tarka-Embedding-150M-V1",
               "kind": "registry_reported",
-              "parameters_billion": 0.109,
-              "mean_task": 0.6550466210091898
+              "parameters_billion": 0.156,
+              "mean_task": 0.6639085365853659
             },
-            "gap_pp_above_best_same_or_smaller": -0.6203469578187959,
+            "gap_pp_above_best_same_or_smaller": -1.5065385154364064,
             "observed_frontier": false,
-            "dominator_count": 4,
+            "dominator_count": 6,
             "all_dominator_ids": [
               "scores_MTEB_eng_v2:row:61",
               "scores_MTEB_eng_v2:row:65",
+              "scores_MTEB_eng_v2:row:68",
               "scores_MTEB_eng_v2:row:71",
-              "scores_MTEB_eng_v2:row:72"
+              "scores_MTEB_eng_v2:row:72",
+              "scores_MTEB_eng_v2:row:74"
             ]
           },
           "mean_task_type": {
@@ -22585,8 +22606,8 @@
             "global_rank": 65,
             "global_population": 188,
             "tied_other_models": 0,
-            "same_or_smaller_rank": 3,
-            "same_or_smaller_population": 66,
+            "same_or_smaller_rank": 5,
+            "same_or_smaller_population": 75,
             "best_same_or_smaller_other_model": {
               "id": "scores_MTEB_eng_v2:row:61",
               "model": "avsolatorio/GIST-Embedding-v0",
@@ -22596,10 +22617,12 @@
             },
             "gap_pp_above_best_same_or_smaller": -0.6140745488965593,
             "observed_frontier": false,
-            "dominator_count": 2,
+            "dominator_count": 4,
             "all_dominator_ids": [
               "scores_MTEB_eng_v2:row:61",
-              "scores_MTEB_eng_v2:row:65"
+              "scores_MTEB_eng_v2:row:65",
+              "scores_MTEB_eng_v2:row:68",
+              "scores_MTEB_eng_v2:row:74"
             ]
           }
         },
@@ -23193,7 +23216,7 @@
             "id": "current:nano",
             "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
             "score": 0.6488431514310018,
-            "parameters_billion": 0.135383808
+            "parameters_billion": 0.163771288
           },
           {
             "rank": 66,
@@ -24511,7 +24534,7 @@
             "id": "current:nano",
             "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
             "score": 0.6078184975654924,
-            "parameters_billion": 0.135383808
+            "parameters_billion": 0.163771288
           },
           {
             "rank": 66,
@@ -29320,39 +29343,39 @@
           "id": "current:nano",
           "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
           "kind": "nano",
-          "parameters_exact": 135383808,
-          "parameters_billion": 0.135383808,
+          "parameters_exact": 163771288,
+          "parameters_billion": 0.163771288,
           "parameter_basis": "exact full released native tensor count, all modalities",
-          "current_native_fingerprint": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-          "mean_task": 0.3976628038845883,
-          "mean_task_type": 0.4696781408655678,
+          "current_native_fingerprint": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+          "mean_task": 0.43593843064595883,
+          "mean_task_type": 0.5233540613685161,
           "by_task_type": {
-            "Any2AnyRetrieval": 0.689005,
-            "AudioClassification": 0.39553889510295764,
-            "AudioClustering": 0.054218008138417706,
-            "AudioPairClassification": 0.5716888010864635,
-            "AudioReranking": 0.63794
+            "Any2AnyRetrieval": 0.858015,
+            "AudioClassification": 0.44475865771818446,
+            "AudioClustering": 0.03504342925419188,
+            "AudioPairClassification": 0.5741932198702043,
+            "AudioReranking": 0.70476
           },
           "scores_by_task": {
-            "JamAltArtistA2ARetrieval": 0.689005,
-            "BeijingOpera": 0.686790780141844,
-            "BirdCLEF": 0.129,
-            "CREMA_D": 0.29763670140167686,
-            "CommonLanguageAgeDetection": 0.1604,
-            "GTZANGenre": 0.503,
-            "IEMOCAPGender": 0.5845186270364481,
-            "MInDS14": 0.23321168947951418,
-            "MridinghamTonic": 0.32363208758254514,
-            "SIBFLEURS": 0.1830465783151205,
-            "VoxCelebSA": 0.33169138217538546,
-            "VoxPopuliLanguageID": 0.9179999999999999,
-            "CREMA_DClustering": 0.023601715918443712,
-            "VehicleSoundClustering": 0.13800701178563807,
-            "VoxPopuliGenderClustering": 0.0010452967111713475,
-            "CREMADPairClassification": 0.5420123629581632,
-            "NMSQAPairClassification": 0.6328340215485784,
-            "VoxPopuliAccentPairClassification": 0.5402200187526489,
-            "GTZANAudioReranking": 0.63794
+            "JamAltArtistA2ARetrieval": 0.858015,
+            "BeijingOpera": 0.7922872340425531,
+            "BirdCLEF": 0.15199999999999997,
+            "CREMA_D": 0.32048146984697823,
+            "CommonLanguageAgeDetection": 0.16655,
+            "GTZANGenre": 0.617,
+            "IEMOCAPGender": 0.6168973830636596,
+            "MInDS14": 0.21852038094350856,
+            "MridinghamTonic": 0.5841978617863635,
+            "SIBFLEURS": 0.18072541269472217,
+            "VoxCelebSA": 0.331685492522244,
+            "VoxPopuliLanguageID": 0.9120000000000001,
+            "CREMA_DClustering": 0.05214346580804754,
+            "VehicleSoundClustering": 0.05275425308186281,
+            "VoxPopuliGenderClustering": 0.00023256887266527633,
+            "CREMADPairClassification": 0.5520726152267248,
+            "NMSQAPairClassification": 0.6289933955933497,
+            "VoxPopuliAccentPairClassification": 0.5415136487905384,
+            "GTZANAudioReranking": 0.70476
           },
           "coverage": {
             "complete": true,
@@ -29658,13 +29681,13 @@
       "current_ranks": {
         "nano": {
           "mean_task": {
-            "score": 0.3976628038845883,
-            "display_score_100": 39.76628038845883,
-            "global_rank": 34,
+            "score": 0.43593843064595883,
+            "display_score_100": 43.59384306459588,
+            "global_rank": 22,
             "global_population": 64,
             "tied_other_models": 0,
-            "same_or_smaller_rank": 6,
-            "same_or_smaller_population": 22,
+            "same_or_smaller_rank": 5,
+            "same_or_smaller_population": 27,
             "best_same_or_smaller_other_model": {
               "id": "scores_MAEB_beta_audio-only:row:9",
               "model": "openai/whisper-base",
@@ -29672,43 +29695,40 @@
               "parameters_billion": 0.074,
               "mean_task": 0.44856370433436527
             },
-            "gap_pp_above_best_same_or_smaller": -5.090090044977696,
+            "gap_pp_above_best_same_or_smaller": -1.2625273688406435,
             "observed_frontier": false,
-            "dominator_count": 5,
+            "dominator_count": 4,
             "all_dominator_ids": [
               "scores_MAEB_beta_audio-only:row:9",
-              "scores_MAEB_beta_audio-only:row:12",
+              "scores_MAEB_beta_audio-only:row:17",
               "scores_MAEB_beta_audio-only:row:18",
-              "scores_MAEB_beta_audio-only:row:20",
-              "scores_MAEB_beta_audio-only:row:25"
+              "scores_MAEB_beta_audio-only:row:20"
             ]
           },
           "mean_task_type": {
-            "score": 0.4696781408655678,
-            "display_score_100": 46.96781408655678,
-            "global_rank": 32,
+            "score": 0.5233540613685161,
+            "display_score_100": 52.33540613685162,
+            "global_rank": 19,
             "global_population": 64,
             "tied_other_models": 0,
-            "same_or_smaller_rank": 8,
-            "same_or_smaller_population": 22,
+            "same_or_smaller_rank": 6,
+            "same_or_smaller_population": 27,
             "best_same_or_smaller_other_model": {
-              "id": "scores_MAEB_beta_audio-only:row:20",
-              "model": "MIT/ast-finetuned-audioset-10-10-0.4593",
+              "id": "scores_MAEB_beta_audio-only:row:17",
+              "model": "microsoft/msclap-2023",
               "kind": "registry_reported",
-              "parameters_billion": 0.087,
-              "mean_task_type": 0.55212392228164
+              "parameters_billion": 0.16,
+              "mean_task_type": 0.5584816090017826
             },
-            "gap_pp_above_best_same_or_smaller": -8.244578141607217,
+            "gap_pp_above_best_same_or_smaller": -3.512754763326642,
             "observed_frontier": false,
-            "dominator_count": 7,
+            "dominator_count": 5,
             "all_dominator_ids": [
-              "scores_MAEB_beta_audio-only:row:9",
-              "scores_MAEB_beta_audio-only:row:12",
-              "scores_MAEB_beta_audio-only:row:18",
+              "scores_MAEB_beta_audio-only:row:14",
+              "scores_MAEB_beta_audio-only:row:17",
+              "scores_MAEB_beta_audio-only:row:19",
               "scores_MAEB_beta_audio-only:row:20",
-              "scores_MAEB_beta_audio-only:row:25",
-              "scores_MAEB_beta_audio-only:row:27",
-              "scores_MAEB_beta_audio-only:row:28"
+              "scores_MAEB_beta_audio-only:row:25"
             ]
           }
         },
@@ -29914,95 +29934,95 @@
           },
           {
             "rank": 22,
+            "id": "current:nano",
+            "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
+            "score": 0.43593843064595883,
+            "parameters_billion": 0.163771288
+          },
+          {
+            "rank": 23,
             "id": "scores_MAEB_beta_audio-only:row:19",
             "model": "laion/clap-htsat-fused",
             "score": 0.43475542389060884,
             "parameters_billion": 0.154
           },
           {
-            "rank": 23,
+            "rank": 24,
             "id": "scores_MAEB_beta_audio-only:row:14",
             "model": "laion/clap-htsat-unfused",
             "score": 0.42935003353973167,
             "parameters_billion": 0.153
           },
           {
-            "rank": 24,
+            "rank": 25,
             "id": "scores_MAEB_beta_audio-only:row:41",
             "model": "facebook/mms-1b-fl102",
             "score": 0.41367276031991745,
             "parameters_billion": 1.0
           },
           {
-            "rank": 25,
+            "rank": 26,
             "id": "scores_MAEB_beta_audio-only:row:42",
             "model": "facebook/seamless-m4t-v2-large",
             "score": 0.4131003795149639,
             "parameters_billion": 2.3
           },
           {
-            "rank": 26,
+            "rank": 27,
             "id": "scores_MAEB_beta_audio-only:row:25",
             "model": "google/vggish",
             "score": 0.4109744040247678,
             "parameters_billion": 0.072
           },
           {
-            "rank": 27,
+            "rank": 28,
             "id": "scores_MAEB_beta_audio-only:row:12",
             "model": "matthewagi/HeAR-s1.1",
             "score": 0.41088769685242515,
             "parameters_billion": 0.022
           },
           {
-            "rank": 28,
+            "rank": 29,
             "id": "scores_MAEB_beta_audio-only:row:30",
             "model": "facebook/wav2vec2-xls-r-2b",
             "score": 0.4095841483488132,
             "parameters_billion": 2.0
           },
           {
-            "rank": 29,
+            "rank": 30,
             "id": "scores_MAEB_beta_audio-only:row:22",
             "model": "OpenMuQ/MuQ-MuLan-large",
             "score": 0.4089651437048504,
             "parameters_billion": 0.63
           },
           {
-            "rank": 30,
+            "rank": 31,
             "id": "scores_MAEB_beta_audio-only:row:35",
             "model": "facebook/mms-1b-all",
             "score": 0.4070234414344685,
             "parameters_billion": 1.0
           },
           {
-            "rank": 31,
+            "rank": 32,
             "id": "scores_MAEB_beta_audio-only:row:33",
             "model": "microsoft/msclap-2022",
             "score": 0.40099675438596494,
             "parameters_billion": 0.196
           },
           {
-            "rank": 32,
+            "rank": 33,
             "id": "scores_MAEB_beta_audio-only:row:31",
             "model": "facebook/mms-1b-l1107",
             "score": 0.3998511984004128,
             "parameters_billion": 1.0
           },
           {
-            "rank": 33,
+            "rank": 34,
             "id": "scores_MAEB_beta_audio-only:row:29",
             "model": "facebook/wav2vec2-lv-60-espeak-cv-ft",
             "score": 0.3983267115583075,
             "parameters_billion": 0.317
           },
-          {
-            "rank": 34,
-            "id": "current:nano",
-            "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-            "score": 0.3976628038845883,
-            "parameters_billion": 0.135383808
-          },
           {
             "rank": 35,
             "id": "scores_MAEB_beta_audio-only:row:26",
@@ -30343,102 +30363,102 @@
           },
           {
             "rank": 19,
+            "id": "current:nano",
+            "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
+            "score": 0.5233540613685161,
+            "parameters_billion": 0.163771288
+          },
+          {
+            "rank": 20,
             "id": "scores_MAEB_beta_audio-only:row:33",
             "model": "microsoft/msclap-2022",
             "score": 0.5156324,
             "parameters_billion": 0.196
           },
           {
-            "rank": 20,
+            "rank": 21,
             "id": "scores_MAEB_beta_audio-only:row:23",
             "model": "EximiusLabs/fusion-embedding-2-2b-preview",
             "score": 0.5116159368092692,
             "parameters_billion": 2.8
           },
           {
-            "rank": 21,
+            "rank": 22,
             "id": "scores_MAEB_beta_audio-only:row:28",
             "model": "google/yamnet",
             "score": 0.510955464795009,
             "parameters_billion": 0.004
           },
           {
-            "rank": 22,
+            "rank": 23,
             "id": "scores_MAEB_beta_audio-only:row:1",
             "model": "openai/whisper-medium",
             "score": 0.5071337459893048,
             "parameters_billion": 0.769
           },
           {
-            "rank": 23,
+            "rank": 24,
             "id": "scores_MAEB_beta_audio-only:row:12",
             "model": "matthewagi/HeAR-s1.1",
             "score": 0.5022611316399287,
             "parameters_billion": 0.022
           },
           {
-            "rank": 24,
+            "rank": 25,
             "id": "scores_MAEB_beta_audio-only:row:0",
             "model": "Qwen/Qwen2-Audio-7B",
             "score": 0.49261104875222816,
             "parameters_billion": 7.0
           },
           {
-            "rank": 25,
+            "rank": 26,
             "id": "scores_MAEB_beta_audio-only:row:24",
             "model": "lyrebird/wav2clip",
             "score": 0.48892121889483064,
             "parameters_billion": 0.163
           },
           {
-            "rank": 26,
+            "rank": 27,
             "id": "scores_MAEB_beta_audio-only:row:26",
             "model": "microsoft/wavlm-large",
             "score": 0.4848395147058824,
             "parameters_billion": 0.317
           },
           {
-            "rank": 27,
+            "rank": 28,
             "id": "scores_MAEB_beta_audio-only:row:9",
             "model": "openai/whisper-base",
             "score": 0.48299134028520496,
             "parameters_billion": 0.074
           },
           {
-            "rank": 28,
+            "rank": 29,
             "id": "scores_MAEB_beta_audio-only:row:13",
             "model": "openai/whisper-large-v3",
             "score": 0.4818755459001783,
             "parameters_billion": 1.55
           },
           {
-            "rank": 29,
+            "rank": 30,
             "id": "scores_MAEB_beta_audio-only:row:11",
             "model": "openai/whisper-small",
             "score": 0.48186107352941177,
             "parameters_billion": 0.244
           },
           {
-            "rank": 30,
+            "rank": 31,
             "id": "scores_MAEB_beta_audio-only:row:18",
             "model": "openai/whisper-tiny",
             "score": 0.4774227983957219,
             "parameters_billion": 0.039
           },
           {
-            "rank": 31,
+            "rank": 32,
             "id": "scores_MAEB_beta_audio-only:row:27",
             "model": "facebook/hubert-base-ls960",
             "score": 0.4762437683600713,
             "parameters_billion": 0.095
           },
-          {
-            "rank": 32,
-            "id": "current:nano",
-            "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-            "score": 0.4696781408655678,
-            "parameters_billion": 0.135383808
-          },
           {
             "rank": 33,
             "id": "scores_MAEB_beta_audio-only:row:45",
@@ -30798,15 +30818,15 @@
       "focus": "nano",
       "same_size_top10_ids": [
         "scores_MTEB_eng_v2:row:61",
+        "scores_MTEB_eng_v2:row:68",
         "scores_MTEB_eng_v2:row:65",
+        "scores_MTEB_eng_v2:row:74",
         "current:nano",
         "scores_MTEB_eng_v2:row:71",
         "scores_MTEB_eng_v2:row:66",
         "scores_MTEB_eng_v2:row:72",
         "scores_MTEB_eng_v2:row:79",
-        "scores_MTEB_eng_v2:row:80",
-        "scores_MTEB_eng_v2:row:83",
-        "scores_MTEB_eng_v2:row:86"
+        "scores_MTEB_eng_v2:row:80"
       ],
       "all_known_size_point_ids": [
         "scores_MTEB_eng_v2:row:0",

benchmarks/complete-panel-metadata.json

@@ -4,7 +4,7 @@
   "primary_aggregate": "Mean(TaskType)",
   "supplementary_aggregate": "Mean(Task)",
   "data_file": "complete-panel-data.json",
-  "data_sha256": "fbe0dcb2b929bbc3f283f2fca30af768c9009feff9d2b288a97d995a928727f6",
+  "data_sha256": "ffaedbbc06e8db52a0071323c1b15c503a97871b03129710d7e1bdfcb9fd81f6",
   "score_display": "100 \u00d7 native score",
   "snapshot_date": "2026-09-17",
   "known_size_point_counts": {
@@ -198,15 +198,15 @@
       "unknown_size_count": 13,
       "same_size_top10_ids": [
         "scores_MTEB_eng_v2:row:61",
+        "scores_MTEB_eng_v2:row:68",
         "scores_MTEB_eng_v2:row:65",
+        "scores_MTEB_eng_v2:row:74",
         "current:nano",
         "scores_MTEB_eng_v2:row:71",
         "scores_MTEB_eng_v2:row:66",
         "scores_MTEB_eng_v2:row:72",
         "scores_MTEB_eng_v2:row:79",
-        "scores_MTEB_eng_v2:row:80",
-        "scores_MTEB_eng_v2:row:83",
-        "scores_MTEB_eng_v2:row:86"
+        "scores_MTEB_eng_v2:row:80"
       ],
       "focus_frontier": false,
       "label_overlaps": [],
@@ -215,8 +215,8 @@
       "all_points_inside_axes": true,
       "all_other_panel_ranks_visible": true,
       "exports_sha256": {
-        "complete-panel-english41-nano.png": "ea7e14d09f749c22b0d4d32414df5008f9d68c746f8614483b25108ff52c88cc",
-        "complete-panel-english41-nano.svg": "79f18961e977139f72b402e83316b6dc523268817d241e61a0ef9efb54a20697"
+        "complete-panel-english41-nano.png": "49e1d1a57289291c261e50070ebe2ed5f7fe313ff0d1688c7e67ca2f0c4b27ff",
+        "complete-panel-english41-nano.svg": "b0b8f04d2a408bc98e8e36a6c7024e9e1226ade870a8e526eb9970c150b2d2bd"
       },
       "visible_reference_labels": [
         {
@@ -351,8 +351,8 @@
       "all_points_inside_axes": true,
       "all_other_panel_ranks_visible": true,
       "exports_sha256": {
-        "complete-panel-audio19-mini.png": "69b0f8362667abaa5579dc5f78bdf49e918428f40e350f393dae451807ef9dd3",
-        "complete-panel-audio19-mini.svg": "66c5e212b37de8896b38d16c03118dd2dd7a8f50fdd4b650b85c016f1aafe96c"
+        "complete-panel-audio19-mini.png": "131a400c69f3e7b6fc1ff1a1de0f285fd7e5938a2fffbfe98552d8981bfa1b27",
+        "complete-panel-audio19-mini.svg": "09955e43a78a5d0eb3b385b725d9db771116a56da47e55ef8cad6903274300c0"
       },
       "visible_reference_labels": [
         {
@@ -406,7 +406,7 @@
       ]
     }
   },
-  "visual_inspection": "All five PNGs inspected: white background, full eligible point coverage, preserved named references, no clipped or overlapping labels. Mini audio rank and task scores updated; SIB-FLEURS correctly describes topic classification.",
+  "visual_inspection": "All six PNGs inspected: white background, full eligible point coverage, preserved named references, no clipped or overlapping labels. Nano size/audio ranks and task scores updated; Mini scores unchanged; SIB-FLEURS is topic classification.",
   "scope": "Complete dedicated snapshot rows; no per-task best stitching, no partial-panel ranking, no overall SOTA claim. Unknown-size complete peers count globally but cannot be plotted.",
   "ranking_population": "Dedicated registry rows plus both current Vela models, unique names asserted.",
   "source_urls_and_actual_measured_identity": "See sources and models in complete-panel-data.json."

benchmarks/complete-panel-ranks.json

@@ -50,23 +50,25 @@
             "global_rank": 65,
             "global_population": 188,
             "tied_other_models": 0,
-            "same_or_smaller_rank": 5,
-            "same_or_smaller_population": 66,
+            "same_or_smaller_rank": 7,
+            "same_or_smaller_population": 75,
             "best_same_or_smaller_other_model": {
-              "id": "scores_MTEB_eng_v2:row:61",
-              "model": "avsolatorio/GIST-Embedding-v0",
+              "id": "scores_MTEB_eng_v2:row:68",
+              "model": "Tarka-AIR/Tarka-Embedding-150M-V1",
               "kind": "registry_reported",
-              "parameters_billion": 0.109,
-              "mean_task": 0.6550466210091898
+              "parameters_billion": 0.156,
+              "mean_task": 0.6639085365853659
             },
-            "gap_pp_above_best_same_or_smaller": -0.6203469578187959,
+            "gap_pp_above_best_same_or_smaller": -1.5065385154364064,
             "observed_frontier": false,
-            "dominator_count": 4,
+            "dominator_count": 6,
             "all_dominator_ids": [
               "scores_MTEB_eng_v2:row:61",
               "scores_MTEB_eng_v2:row:65",
+              "scores_MTEB_eng_v2:row:68",
               "scores_MTEB_eng_v2:row:71",
-              "scores_MTEB_eng_v2:row:72"
+              "scores_MTEB_eng_v2:row:72",
+              "scores_MTEB_eng_v2:row:74"
             ]
           },
           "mean_task_type": {
@@ -75,8 +77,8 @@
             "global_rank": 65,
             "global_population": 188,
             "tied_other_models": 0,
-            "same_or_smaller_rank": 3,
-            "same_or_smaller_population": 66,
+            "same_or_smaller_rank": 5,
+            "same_or_smaller_population": 75,
             "best_same_or_smaller_other_model": {
               "id": "scores_MTEB_eng_v2:row:61",
               "model": "avsolatorio/GIST-Embedding-v0",
@@ -86,10 +88,12 @@
             },
             "gap_pp_above_best_same_or_smaller": -0.6140745488965593,
             "observed_frontier": false,
-            "dominator_count": 2,
+            "dominator_count": 4,
             "all_dominator_ids": [
               "scores_MTEB_eng_v2:row:61",
-              "scores_MTEB_eng_v2:row:65"
+              "scores_MTEB_eng_v2:row:65",
+              "scores_MTEB_eng_v2:row:68",
+              "scores_MTEB_eng_v2:row:74"
             ]
           }
         },
@@ -683,7 +687,7 @@
             "id": "current:nano",
             "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
             "score": 0.6488431514310018,
-            "parameters_billion": 0.135383808
+            "parameters_billion": 0.163771288
           },
           {
             "rank": 66,
@@ -2001,7 +2005,7 @@
             "id": "current:nano",
             "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
             "score": 0.6078184975654924,
-            "parameters_billion": 0.135383808
+            "parameters_billion": 0.163771288
           },
           {
             "rank": 66,
@@ -3105,13 +3109,13 @@
       "ranks": {
         "nano": {
           "mean_task": {
-            "score": 0.3976628038845883,
-            "display_score_100": 39.76628038845883,
-            "global_rank": 34,
+            "score": 0.43593843064595883,
+            "display_score_100": 43.59384306459588,
+            "global_rank": 22,
             "global_population": 64,
             "tied_other_models": 0,
-            "same_or_smaller_rank": 6,
-            "same_or_smaller_population": 22,
+            "same_or_smaller_rank": 5,
+            "same_or_smaller_population": 27,
             "best_same_or_smaller_other_model": {
               "id": "scores_MAEB_beta_audio-only:row:9",
               "model": "openai/whisper-base",
@@ -3119,43 +3123,40 @@
               "parameters_billion": 0.074,
               "mean_task": 0.44856370433436527
             },
-            "gap_pp_above_best_same_or_smaller": -5.090090044977696,
+            "gap_pp_above_best_same_or_smaller": -1.2625273688406435,
             "observed_frontier": false,
-            "dominator_count": 5,
+            "dominator_count": 4,
             "all_dominator_ids": [
               "scores_MAEB_beta_audio-only:row:9",
-              "scores_MAEB_beta_audio-only:row:12",
+              "scores_MAEB_beta_audio-only:row:17",
               "scores_MAEB_beta_audio-only:row:18",
-              "scores_MAEB_beta_audio-only:row:20",
-              "scores_MAEB_beta_audio-only:row:25"
+              "scores_MAEB_beta_audio-only:row:20"
             ]
           },
           "mean_task_type": {
-            "score": 0.4696781408655678,
-            "display_score_100": 46.96781408655678,
-            "global_rank": 32,
+            "score": 0.5233540613685161,
+            "display_score_100": 52.33540613685162,
+            "global_rank": 19,
             "global_population": 64,
             "tied_other_models": 0,
-            "same_or_smaller_rank": 8,
-            "same_or_smaller_population": 22,
+            "same_or_smaller_rank": 6,
+            "same_or_smaller_population": 27,
             "best_same_or_smaller_other_model": {
-              "id": "scores_MAEB_beta_audio-only:row:20",
-              "model": "MIT/ast-finetuned-audioset-10-10-0.4593",
+              "id": "scores_MAEB_beta_audio-only:row:17",
+              "model": "microsoft/msclap-2023",
               "kind": "registry_reported",
-              "parameters_billion": 0.087,
-              "mean_task_type": 0.55212392228164
+              "parameters_billion": 0.16,
+              "mean_task_type": 0.5584816090017826
             },
-            "gap_pp_above_best_same_or_smaller": -8.244578141607217,
+            "gap_pp_above_best_same_or_smaller": -3.512754763326642,
             "observed_frontier": false,
-            "dominator_count": 7,
+            "dominator_count": 5,
             "all_dominator_ids": [
-              "scores_MAEB_beta_audio-only:row:9",
-              "scores_MAEB_beta_audio-only:row:12",
-              "scores_MAEB_beta_audio-only:row:18",
+              "scores_MAEB_beta_audio-only:row:14",
+              "scores_MAEB_beta_audio-only:row:17",
+              "scores_MAEB_beta_audio-only:row:19",
               "scores_MAEB_beta_audio-only:row:20",
-              "scores_MAEB_beta_audio-only:row:25",
-              "scores_MAEB_beta_audio-only:row:27",
-              "scores_MAEB_beta_audio-only:row:28"
+              "scores_MAEB_beta_audio-only:row:25"
             ]
           }
         },
@@ -3361,95 +3362,95 @@
           },
           {
             "rank": 22,
+            "id": "current:nano",
+            "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
+            "score": 0.43593843064595883,
+            "parameters_billion": 0.163771288
+          },
+          {
+            "rank": 23,
             "id": "scores_MAEB_beta_audio-only:row:19",
             "model": "laion/clap-htsat-fused",
             "score": 0.43475542389060884,
             "parameters_billion": 0.154
           },
           {
-            "rank": 23,
+            "rank": 24,
             "id": "scores_MAEB_beta_audio-only:row:14",
             "model": "laion/clap-htsat-unfused",
             "score": 0.42935003353973167,
             "parameters_billion": 0.153
           },
           {
-            "rank": 24,
+            "rank": 25,
             "id": "scores_MAEB_beta_audio-only:row:41",
             "model": "facebook/mms-1b-fl102",
             "score": 0.41367276031991745,
             "parameters_billion": 1.0
           },
           {
-            "rank": 25,
+            "rank": 26,
             "id": "scores_MAEB_beta_audio-only:row:42",
             "model": "facebook/seamless-m4t-v2-large",
             "score": 0.4131003795149639,
             "parameters_billion": 2.3
           },
           {
-            "rank": 26,
+            "rank": 27,
             "id": "scores_MAEB_beta_audio-only:row:25",
             "model": "google/vggish",
             "score": 0.4109744040247678,
             "parameters_billion": 0.072
           },
           {
-            "rank": 27,
+            "rank": 28,
             "id": "scores_MAEB_beta_audio-only:row:12",
             "model": "matthewagi/HeAR-s1.1",
             "score": 0.41088769685242515,
             "parameters_billion": 0.022
           },
           {
-            "rank": 28,
+            "rank": 29,
             "id": "scores_MAEB_beta_audio-only:row:30",
             "model": "facebook/wav2vec2-xls-r-2b",
             "score": 0.4095841483488132,
             "parameters_billion": 2.0
           },
           {
-            "rank": 29,
+            "rank": 30,
             "id": "scores_MAEB_beta_audio-only:row:22",
             "model": "OpenMuQ/MuQ-MuLan-large",
             "score": 0.4089651437048504,
             "parameters_billion": 0.63
           },
           {
-            "rank": 30,
+            "rank": 31,
             "id": "scores_MAEB_beta_audio-only:row:35",
             "model": "facebook/mms-1b-all",
             "score": 0.4070234414344685,
             "parameters_billion": 1.0
           },
           {
-            "rank": 31,
+            "rank": 32,
             "id": "scores_MAEB_beta_audio-only:row:33",
             "model": "microsoft/msclap-2022",
             "score": 0.40099675438596494,
             "parameters_billion": 0.196
           },
           {
-            "rank": 32,
+            "rank": 33,
             "id": "scores_MAEB_beta_audio-only:row:31",
             "model": "facebook/mms-1b-l1107",
             "score": 0.3998511984004128,
             "parameters_billion": 1.0
           },
           {
-            "rank": 33,
+            "rank": 34,
             "id": "scores_MAEB_beta_audio-only:row:29",
             "model": "facebook/wav2vec2-lv-60-espeak-cv-ft",
             "score": 0.3983267115583075,
             "parameters_billion": 0.317
           },
-          {
-            "rank": 34,
-            "id": "current:nano",
-            "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-            "score": 0.3976628038845883,
-            "parameters_billion": 0.135383808
-          },
           {
             "rank": 35,
             "id": "scores_MAEB_beta_audio-only:row:26",
@@ -3790,102 +3791,102 @@
           },
           {
             "rank": 19,
+            "id": "current:nano",
+            "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
+            "score": 0.5233540613685161,
+            "parameters_billion": 0.163771288
+          },
+          {
+            "rank": 20,
             "id": "scores_MAEB_beta_audio-only:row:33",
             "model": "microsoft/msclap-2022",
             "score": 0.5156324,
             "parameters_billion": 0.196
           },
           {
-            "rank": 20,
+            "rank": 21,
             "id": "scores_MAEB_beta_audio-only:row:23",
             "model": "EximiusLabs/fusion-embedding-2-2b-preview",
             "score": 0.5116159368092692,
             "parameters_billion": 2.8
           },
           {
-            "rank": 21,
+            "rank": 22,
             "id": "scores_MAEB_beta_audio-only:row:28",
             "model": "google/yamnet",
             "score": 0.510955464795009,
             "parameters_billion": 0.004
           },
           {
-            "rank": 22,
+            "rank": 23,
             "id": "scores_MAEB_beta_audio-only:row:1",
             "model": "openai/whisper-medium",
             "score": 0.5071337459893048,
             "parameters_billion": 0.769
           },
           {
-            "rank": 23,
+            "rank": 24,
             "id": "scores_MAEB_beta_audio-only:row:12",
             "model": "matthewagi/HeAR-s1.1",
             "score": 0.5022611316399287,
             "parameters_billion": 0.022
           },
           {
-            "rank": 24,
+            "rank": 25,
             "id": "scores_MAEB_beta_audio-only:row:0",
             "model": "Qwen/Qwen2-Audio-7B",
             "score": 0.49261104875222816,
             "parameters_billion": 7.0
           },
           {
-            "rank": 25,
+            "rank": 26,
             "id": "scores_MAEB_beta_audio-only:row:24",
             "model": "lyrebird/wav2clip",
             "score": 0.48892121889483064,
             "parameters_billion": 0.163
           },
           {
-            "rank": 26,
+            "rank": 27,
             "id": "scores_MAEB_beta_audio-only:row:26",
             "model": "microsoft/wavlm-large",
             "score": 0.4848395147058824,
             "parameters_billion": 0.317
           },
           {
-            "rank": 27,
+            "rank": 28,
             "id": "scores_MAEB_beta_audio-only:row:9",
             "model": "openai/whisper-base",
             "score": 0.48299134028520496,
             "parameters_billion": 0.074
           },
           {
-            "rank": 28,
+            "rank": 29,
             "id": "scores_MAEB_beta_audio-only:row:13",
             "model": "openai/whisper-large-v3",
             "score": 0.4818755459001783,
             "parameters_billion": 1.55
           },
           {
-            "rank": 29,
+            "rank": 30,
             "id": "scores_MAEB_beta_audio-only:row:11",
             "model": "openai/whisper-small",
             "score": 0.48186107352941177,
             "parameters_billion": 0.244
           },
           {
-            "rank": 30,
+            "rank": 31,
             "id": "scores_MAEB_beta_audio-only:row:18",
             "model": "openai/whisper-tiny",
             "score": 0.4774227983957219,
             "parameters_billion": 0.039
           },
           {
-            "rank": 31,
+            "rank": 32,
             "id": "scores_MAEB_beta_audio-only:row:27",
             "model": "facebook/hubert-base-ls960",
             "score": 0.4762437683600713,
             "parameters_billion": 0.095
           },
-          {
-            "rank": 32,
-            "id": "current:nano",
-            "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-            "score": 0.4696781408655678,
-            "parameters_billion": 0.135383808
-          },
           {
             "rank": 33,
             "id": "scores_MAEB_beta_audio-only:row:45",

benchmarks/complete-panel-ranks.md

@@ -1,12 +1,12 @@
 # Complete-panel ranks
 
-Primary: **Mean(TaskType)**. Supplementary: Mean(Task). Scores are ×100. Same complete September 17, 2026 dedicated registry cohorts; all Vela modality parameters counted. Different upstream protocols remain explicit. No mean is imputed for an incomplete model.
+Primary: **Mean(TaskType)**. Supplementary: Mean(Task). Scores are ×100. Complete September 17, 2026 registry cohorts; all Vela modality parameters counted. Upstream protocols can differ. No aggregate is imputed for an incomplete model.
 
 | Model | Panel | Mean(TaskType) | Global | ≤ current size | Mean(Task) | Global | ≤ current size |
-|---|---|---:|---:|---:|---:|---:|---:|
-| Vela Omni Nano | English41 | 60.7818 | 65/188 | 3/66 | 64.8843 | 65/188 | 5/66 |
-| Vela Omni Nano | audio19 | 46.9678 | 32/64 | 8/22 | 39.7663 | 34/64 | 6/22 |
+|---|---|---:|---|---|---:|---|---|
+| Vela Omni Nano | English41 | 60.7818 | 65/188 | 5/75 | 64.8843 | 65/188 | 7/75 |
+| Vela Omni Nano | audio19 | 52.3354 | 19/64 | 6/27 | 43.5938 | 22/64 | 5/27 |
 | Vela Omni Mini | English41 | 58.7867 | 92/188 | 50/134 | 63.4923 | 82/188 | 42/134 |
 | Vela Omni Mini | audio19 | 54.8679 | 12/64 | 5/50 | 47.7720 | 9/64 | 3/50 |
 
-These are complete-panel snapshot comparisons, not official leaderboard submissions or a general multimodal SOTA claim. English applicability retains the original measured text identity. Mini audio is newly evaluated on all 19 tasks and 134 subset/split slots. Every peer and exclusion is retained in complete-panel-data.json.
+Snapshot comparisons include single-modality specialists and are not official leaderboard submissions or an overall multimodal SOTA claim. Nano audio is newly evaluated on all 19 tasks and 134 subset/split slots; unchanged English text computation retains its original measured identity. Mini scores and size are unchanged. Full task values, exclusions and provenance are retained in complete-panel-data.json.

benchmarks/component-equivalence.json

@@ -1,37 +1,36 @@
 {
   "schema_version": 1,
   "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-  "parameters": 144554112,
-  "evaluated_model": {
-    "model": "local/Vela-Omni-Nano-GIST-Rotation",
-    "revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
-    "revision_kind": "local native artifact fingerprint, not a Hugging Face commit",
-    "native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646"
+  "release_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+  "evaluated_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+  "parameters": 163771288,
+  "release_native_files": "native-manifest.json",
+  "audio": {
+    "fresh_full_artifact_evaluation": true,
+    "task_count": 19,
+    "slot_count": 134,
+    "score_inheritance": false
   },
-  "applies_to": {
-    "revision": "self",
-    "native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4"
+  "text": {
+    "evaluated_native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+    "applies_to_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+    "retained_text_tensor_count": 197,
+    "parameters": 33212160,
+    "stored_and_effective_FP32_values_exact": true,
+    "tokenizer_configuration_and_inference_operations_exact": true,
+    "unused_pooler_tensors_removed_previously": 2,
+    "new_English_panel_forward": false
   },
-  "unchanged_components": [
-    "text"
-  ],
-  "tensor_identity": {
-    "text_tensors": 199,
-    "text_parameters": 33360000,
-    "stored_values_dtypes_shapes_exact": true,
-    "effective_float32_tensors_exact": true
+  "image": {
+    "evaluated_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
+    "applies_to_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+    "retained_image_tensor_count": 210,
+    "parameters": 93815424,
+    "stored_and_effective_FP32_values_exact": true,
+    "inference_operations_exact": true,
+    "new_standard_image_forward": false
   },
-  "inference_files_exact": true,
-  "method": "Both complete native packages were rehashed; all text tensors and inference source, configuration and tokenizer bytes match. Official evaluator, runtime, seed, batching, CLS pooling and 512-token limit are identical.",
-  "reports": [
-    "benchmarks/mteb-eng-v2.json"
-  ],
-  "scope": "The full English report keeps the artifact that was actually evaluated. Only text-only scores apply through this bridge. Image, audio and cross-modal results are freshly evaluated on the current artifact.",
-  "inference_profile_applicability": {
-    "revision": "self",
-    "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-    "parameters": 135383808,
-    "report": "benchmarks/inference-equivalence.json",
-    "new_benchmark_run": false
-  }
+  "previous_native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
+  "proof_sha256": "42d802c7279a1c20728926c079f5a4f952b16629fc49a8ed121da4c51f3d77aa",
+  "scope": "Equivalence applies only to the named unchanged text and image computations; it makes no claim of unchanged audio or latency."
 }

benchmarks/event-audio-diagnostic.json

@@ -0,0 +1,24 @@
+{
+  "scope": "Separate weak-label positive-recall diagnostic, outside the product net and official benchmark panels.",
+  "evaluated_native_fingerprint": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+  "previous_native_fingerprint": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
+  "train_recordings": 3299,
+  "evaluation_recordings": 840,
+  "known_label_universe": 153,
+  "TRAIN_supported_prototypes": 149,
+  "protocol": "Normalize the mean TRAIN audio embedding for each known class. Per clip, every annotated positive remains in the recall denominator, including labels without a TRAIN prototype. Average clip recall at 1/5/10.",
+  "scores": {
+    "candidate": {
+      "positive_recall@1": 0.2538690476190476,
+      "positive_recall@10": 0.7918622448979592,
+      "positive_recall@5": 0.662157029478458
+    },
+    "parent": {
+      "positive_recall@1": 0.0901388888888889,
+      "positive_recall@10": 0.3862528344671202,
+      "positive_recall@5": 0.273296485260771
+    }
+  },
+  "missing_prototype_positives_count_as_misses": true,
+  "new_native_evaluation": true
+}

benchmarks/mteb-eng-v2.json

@@ -4672,351 +4672,64 @@
   "component_applicability": {
     "component": "text",
     "applies_to_revision": "self",
-    "applies_to_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
+    "applies_to_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
     "evaluated_model": "local/Vela-Omni-Nano-GIST-Rotation",
     "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
     "evaluated_native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
     "revision_kind": "local native artifact fingerprint, not a Hugging Face commit",
     "report": "benchmarks/component-equivalence.json",
     "new_full_panel_inference": false,
-    "basis": "199 exact text tensors and byte-identical inference/tokenizer/configuration under the same official evaluator protocol."
+    "basis": "The evaluated text computation is retained through the published Nano and its inference-only trim. All 197 retained text tensors, effective FP32 values, tokenizer/config and text operation path match; the two removed pooler tensors are unused.",
+    "bridge_sha256": "42d802c7279a1c20728926c079f5a4f952b16629fc49a8ed121da4c51f3d77aa"
   },
   "revision_semantics": "local native artifact fingerprint, not a Hugging Face commit",
-  "comparison_to_previous_release": [
-    {
-      "candidate_100": 59.245999999999995,
-      "current_100": 37.513999999999996,
-      "delta_pp": 21.732,
-      "primary_metric": "ndcg_at_10",
-      "task": "ArguAna",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 64.88847259326147,
-      "current_100": 65.25395409281045,
-      "delta_pp": -0.36548149954897724,
-      "primary_metric": "v_measure",
-      "task": "ArXivHierarchicalClusteringP2P",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 57.44315313869382,
-      "current_100": 55.989532291560415,
-      "delta_pp": 1.4536208471334078,
-      "primary_metric": "v_measure",
-      "task": "ArXivHierarchicalClusteringS2S",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 62.329,
-      "current_100": 60.514,
-      "delta_pp": 1.8149999999999977,
-      "primary_metric": "map_at_1000",
-      "task": "AskUbuntuDupQuestions",
-      "task_type": "Reranking"
-    },
-    {
-      "candidate_100": 86.98752423842483,
-      "current_100": 62.24539420714732,
-      "delta_pp": 24.742130031277505,
-      "primary_metric": "cosine_spearman",
-      "task": "BIOSSES",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 82.14285714285714,
-      "current_100": 69.57467532467533,
-      "delta_pp": 12.568181818181813,
-      "primary_metric": "accuracy",
-      "task": "Banking77Classification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 41.313796273674654,
-      "current_100": 39.82923119776807,
-      "delta_pp": 1.4845650759065876,
-      "primary_metric": "v_measure",
-      "task": "BiorxivClusteringP2P.v2",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 56.979,
-      "current_100": 42.976,
-      "delta_pp": 14.003,
-      "primary_metric": "ndcg_at_10",
-      "task": "CQADupstackGamingRetrieval",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 39.586,
-      "current_100": 30.362000000000002,
-      "delta_pp": 9.223999999999997,
-      "primary_metric": "ndcg_at_10",
-      "task": "CQADupstackUnixRetrieval",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 31.846000000000004,
-      "current_100": 16.073999999999998,
-      "delta_pp": 15.772000000000006,
-      "primary_metric": "ndcg_at_10",
-      "task": "ClimateFEVERHardNegatives",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 87.576,
-      "current_100": 24.055,
-      "delta_pp": 63.520999999999994,
-      "primary_metric": "ndcg_at_10",
-      "task": "FEVERHardNegatives",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 39.143,
-      "current_100": 22.588,
-      "delta_pp": 16.555,
-      "primary_metric": "ndcg_at_10",
-      "task": "FiQA2018",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 66.349,
-      "current_100": 26.987,
-      "delta_pp": 39.36200000000001,
-      "primary_metric": "ndcg_at_10",
-      "task": "HotpotQAHardNegatives",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 91.94760000000001,
-      "current_100": 62.1364,
-      "delta_pp": 29.811200000000007,
-      "primary_metric": "accuracy",
-      "task": "ImdbClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 94.91792065663475,
-      "current_100": 86.71910624715002,
-      "delta_pp": 8.198814409484726,
-      "primary_metric": "accuracy",
-      "task": "MTOPDomainClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 70.96839273705447,
-      "current_100": 60.39340954942838,
-      "delta_pp": 10.574983187626096,
-      "primary_metric": "accuracy",
-      "task": "MassiveIntentClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 76.11297915265635,
-      "current_100": 73.37256220578345,
-      "delta_pp": 2.740416946872898,
-      "primary_metric": "accuracy",
-      "task": "MassiveScenarioClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 39.83023091223438,
-      "current_100": 37.76813963073551,
-      "delta_pp": 2.0620912814988657,
-      "primary_metric": "v_measure",
-      "task": "MedrxivClusteringP2P.v2",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 37.875084453707764,
-      "current_100": 36.13349306174702,
-      "delta_pp": 1.7415913919607462,
-      "primary_metric": "v_measure",
-      "task": "MedrxivClusteringS2S.v2",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 32.366,
-      "current_100": 30.623,
-      "delta_pp": 1.7429999999999986,
-      "primary_metric": "max_over_subqueries_map_at_1000",
-      "task": "MindSmallReranking",
-      "task_type": "Reranking"
-    },
-    {
-      "candidate_100": 21.889,
-      "current_100": 16.827,
-      "delta_pp": 5.061999999999998,
-      "primary_metric": "ndcg_at_10",
-      "task": "SCIDOCS",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 80.53174565838819,
-      "current_100": 71.69618925026361,
-      "delta_pp": 8.835556408124575,
-      "primary_metric": "cosine_spearman",
-      "task": "SICK-R",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 75.5658801516535,
-      "current_100": 68.16915974310523,
-      "delta_pp": 7.396720408548276,
-      "primary_metric": "cosine_spearman",
-      "task": "STS12",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 86.26346974038235,
-      "current_100": 74.40342742804673,
-      "delta_pp": 11.860042312335622,
-      "primary_metric": "cosine_spearman",
-      "task": "STS13",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 82.29882793900394,
-      "current_100": 68.02653891259911,
-      "delta_pp": 14.272289026404835,
-      "primary_metric": "cosine_spearman",
-      "task": "STS14",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 88.73648284326485,
-      "current_100": 77.86144933320074,
-      "delta_pp": 10.875033510064114,
-      "primary_metric": "cosine_spearman",
-      "task": "STS15",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 87.0781967302773,
-      "current_100": 77.92400226361768,
-      "delta_pp": 9.154194466659618,
-      "primary_metric": "cosine_spearman",
-      "task": "STSBenchmark",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 95.79964673462752,
-      "current_100": 89.79096125036985,
-      "delta_pp": 6.0086854842576685,
-      "primary_metric": "max_ap",
-      "task": "SprintDuplicateQuestions",
-      "task_type": "PairClassification"
-    },
-    {
-      "candidate_100": 58.424418738430106,
-      "current_100": 57.34464990059502,
-      "delta_pp": 1.0797688378350827,
-      "primary_metric": "v_measure",
-      "task": "StackExchangeClustering.v2",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 41.053017253245386,
-      "current_100": 39.37668627014545,
-      "delta_pp": 1.6763309830999376,
-      "primary_metric": "v_measure",
-      "task": "StackExchangeClusteringP2P.v2",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 69.121,
-      "current_100": 41.292,
-      "delta_pp": 27.828999999999994,
-      "primary_metric": "ndcg_at_10",
-      "task": "TRECCOVID",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 48.363,
-      "current_100": 31.477,
-      "delta_pp": 16.886,
-      "primary_metric": "ndcg_at_10",
-      "task": "Touche2020Retrieval.v3",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 71.904296875,
-      "current_100": 64.4091796875,
-      "delta_pp": 7.4951171875,
-      "primary_metric": "accuracy",
-      "task": "ToxicConversationsClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 62.79286926994907,
-      "current_100": 50.77815506508207,
-      "delta_pp": 12.014714204867005,
-      "primary_metric": "accuracy",
-      "task": "TweetSentimentExtractionClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 50.87260708286914,
-      "current_100": 44.88471745749435,
-      "delta_pp": 5.987889625374791,
-      "primary_metric": "v_measure",
-      "task": "TwentyNewsgroupsClustering.v2",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 72.95151691526065,
-      "current_100": 56.34475441213181,
-      "delta_pp": 16.60676250312884,
-      "primary_metric": "max_ap",
-      "task": "TwitterSemEval2015",
-      "task_type": "PairClassification"
-    },
-    {
-      "candidate_100": 85.30340320492272,
-      "current_100": 78.70209561308617,
-      "delta_pp": 6.6013075918365445,
-      "primary_metric": "max_ap",
-      "task": "TwitterURLCorpus",
-      "task_type": "PairClassification"
-    },
-    {
-      "candidate_100": 31.834434345907127,
-      "current_100": 29.249640152019634,
-      "delta_pp": 2.584794193887493,
-      "primary_metric": "cosine_spearman",
-      "task": "SummEvalSummarization.v2",
-      "task_type": "Summarization"
-    },
-    {
-      "candidate_100": 71.8507462686567,
-      "current_100": 56.82089552238806,
-      "delta_pp": 15.02985074626865,
-      "primary_metric": "accuracy",
-      "task": "AmazonCounterfactualClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 89.02101343562433,
-      "current_100": 84.16589279728525,
-      "delta_pp": 4.8551206383390735,
-      "primary_metric": "cosine_spearman",
-      "task": "STS17",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 68.75333638044488,
-      "current_100": 67.57005849656171,
-      "delta_pp": 1.1832778838831643,
-      "primary_metric": "cosine_spearman",
-      "task": "STS22.v2",
-      "task_type": "STS"
-    }
-  ],
-  "inference_profile_applicability": {
-    "revision": "self",
-    "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-    "parameters": 135383808,
-    "report": "benchmarks/inference-equivalence.json",
-    "new_benchmark_run": false
+  "comparison_to_previous_release": {
+    "ArguAna": 0.0,
+    "ArXivHierarchicalClusteringP2P": 0.0,
+    "ArXivHierarchicalClusteringS2S": 0.0,
+    "AskUbuntuDupQuestions": 0.0,
+    "BIOSSES": 0.0,
+    "Banking77Classification": 0.0,
+    "BiorxivClusteringP2P.v2": 0.0,
+    "CQADupstackGamingRetrieval": 0.0,
+    "CQADupstackUnixRetrieval": 0.0,
+    "ClimateFEVERHardNegatives": 0.0,
+    "FEVERHardNegatives": 0.0,
+    "FiQA2018": 0.0,
+    "HotpotQAHardNegatives": 0.0,
+    "ImdbClassification": 0.0,
+    "MTOPDomainClassification": 0.0,
+    "MassiveIntentClassification": 0.0,
+    "MassiveScenarioClassification": 0.0,
+    "MedrxivClusteringP2P.v2": 0.0,
+    "MedrxivClusteringS2S.v2": 0.0,
+    "MindSmallReranking": 0.0,
+    "SCIDOCS": 0.0,
+    "SICK-R": 0.0,
+    "STS12": 0.0,
+    "STS13": 0.0,
+    "STS14": 0.0,
+    "STS15": 0.0,
+    "STSBenchmark": 0.0,
+    "SprintDuplicateQuestions": 0.0,
+    "StackExchangeClustering.v2": 0.0,
+    "StackExchangeClusteringP2P.v2": 0.0,
+    "TRECCOVID": 0.0,
+    "Touche2020Retrieval.v3": 0.0,
+    "ToxicConversationsClassification": 0.0,
+    "TweetSentimentExtractionClassification": 0.0,
+    "TwentyNewsgroupsClustering.v2": 0.0,
+    "TwitterSemEval2015": 0.0,
+    "TwitterURLCorpus": 0.0,
+    "SummEvalSummarization.v2": 0.0,
+    "AmazonCounterfactualClassification": 0.0,
+    "STS17": 0.0,
+    "STS22.v2": 0.0
+  },
+  "applicable_parameters": 163771288,
+  "parameter_semantics": "parameters describes the actual evaluated artifact; applicable_parameters describes this unchanged-text release.",
+  "comparison_baseline": {
+    "revision": "9e747d2e1d4b97d3081ff4ac2ac79986c7a8da95",
+    "native_fingerprint": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa"
   }
 }

benchmarks/native-manifest.json

@@ -0,0 +1,149 @@
+{
+  "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
+  "revision": "self",
+  "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+  "parameters": 163771288,
+  "files": {
+    "LICENSE": {
+      "bytes": 11357,
+      "sha256": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4"
+    },
+    "LICENSE-CLAP": {
+      "bytes": 7048,
+      "sha256": "a2010f343487d3f7618affe54f789f5487602331c0a8d03f49e9a7c547cf0499"
+    },
+    "NOTICE": {
+      "bytes": 1803,
+      "sha256": "a05fd6236b1daa54fb9306cf685c15bf3b4315a1007e1c4d56a59d996891a94d"
+    },
+    "components/audio/config.json": {
+      "bytes": 1941,
+      "sha256": "372c430053183035fb9d2d7079482f10053b6ff7d7367525ada9fdeead8adeac"
+    },
+    "components/audio/preprocessor_config.json": {
+      "bytes": 184990,
+      "sha256": "9b5cd03a36fbb8a627c64d98a5b5b126ead95a77720723944487311f0110b666"
+    },
+    "components/audio_clap/config.json": {
+      "bytes": 5390,
+      "sha256": "9efb9557bc804f2ca6e394486af2e45dfed0b18554909735a99c6220b84e4288"
+    },
+    "components/audio_clap/preprocessor_config.json": {
+      "bytes": 541,
+      "sha256": "9739f58296aa6f9ac18008fd0150fb2649bc554985fbde86d0a4041c882ac753"
+    },
+    "components/image/config.json": {
+      "bytes": 322,
+      "sha256": "c2b09e8c0a60f3405d3e53897358a8a373bc4286234f30fcc9324d065fc13728"
+    },
+    "components/image/preprocessor_config.json": {
+      "bytes": 368,
+      "sha256": "5a0e5062d04603dc0e5eff01440f6bebfc66d396f9ca16e0128b31c5730cafc5"
+    },
+    "components/text/1_Pooling/config.json": {
+      "bytes": 190,
+      "sha256": "d1caf60c96f5fba2157c0c26b76d80818fad6cf0b8eb5e73ec372ff9818eba5c"
+    },
+    "components/text/config.json": {
+      "bytes": 719,
+      "sha256": "419ef27bf56a60adc670c50610f8099c64b23e4a1912f4ee7e321139518569b2"
+    },
+    "components/text/config_sentence_transformers.json": {
+      "bytes": 124,
+      "sha256": "940d5f50db195fa6e5e6a4f122c095f77880de259d74b14a65779ed48bdd7c56"
+    },
+    "components/text/modules.json": {
+      "bytes": 349,
+      "sha256": "84e40c8e006c9b1d6c122e02cba9b02458120b5fb0c87b746c41e0207cf642cf"
+    },
+    "components/text/sentence_bert_config.json": {
+      "bytes": 52,
+      "sha256": "84e39fda68ccbff05bfa723ae9c0e70e23e2ec373b76e0f8c6e71af72a693cbf"
+    },
+    "components/text/special_tokens_map.json": {
+      "bytes": 695,
+      "sha256": "5d5b662e421ea9fac075174bb0688ee0d9431699900b90662acd44b2a350503a"
+    },
+    "components/text/tokenizer.json": {
+      "bytes": 711396,
+      "sha256": "d241a60d5e8f04cc1b2b3e9ef7a4921b27bf526d9f6050ab90f9267a1f9e5c66"
+    },
+    "components/text/tokenizer_config.json": {
+      "bytes": 1242,
+      "sha256": "0b29c7bfc889e53b36d9dd3e686dd4300f6525110eaa98c76a5dafceb2029f53"
+    },
+    "components/text/vocab.txt": {
+      "bytes": 231508,
+      "sha256": "07eced375cec144d27c900241f3e339478dec958f92fddbc551f295c992038a3"
+    },
+    "config.json": {
+      "bytes": 561,
+      "sha256": "ebd558ac77fa99b721b2f2c5d07e6d19c6aa1cc516c12df2900077ce937e78a3"
+    },
+    "licenses/GIST-MODEL-CARD.md": {
+      "bytes": 67964,
+      "sha256": "e2905b6dc66b39a697cd82f3211eea4a7ce45b17cc720eceae7fdd337b601c3c"
+    },
+    "model.safetensors": {
+      "bytes": 655583800,
+      "sha256": "d5aa7f00e217bd4fd5314140a3603b9a873636320c8d09732a14f79f4cb0667e"
+    },
+    "omni_components/__init__.py": {
+      "bytes": 41,
+      "sha256": "543f72d073d77a71e7f5ad857f26d53abbecbbba1ed094336f5bf730af0bd641"
+    },
+    "omni_components/audio_encoder.py": {
+      "bytes": 8492,
+      "sha256": "2c1e8e347c3e49a438084cc61f486dda9fde00d9f4b4d7caac828aa9b2f77488"
+    },
+    "omni_components/audio_io.py": {
+      "bytes": 426,
+      "sha256": "23392eb0784bd92e4b0176004455025faa0d9d47f6216e12da34079c49765546"
+    },
+    "omni_components/embedder.py": {
+      "bytes": 14049,
+      "sha256": "ad709fd80bc1850f609443777ddf3327765f3ea330659f5c154f90fa8ab5e8fd"
+    },
+    "omni_components/fusion.py": {
+      "bytes": 11098,
+      "sha256": "a3f6132f190f1533887a77570f4457a0da4eba698574b90888da477919d5c279"
+    },
+    "omni_components/image_encoder.py": {
+      "bytes": 8195,
+      "sha256": "40e64a4fde0cb1a1416b764a1a8db47105985e3b979bac7b0aa3bc9d789d0f7a"
+    },
+    "omni_components/mini.py": {
+      "bytes": 10075,
+      "sha256": "c62439716bcf06a0051a8d9ec7ea41f917816133f425a7fa004561279daac4fe"
+    },
+    "omni_components/records.py": {
+      "bytes": 118,
+      "sha256": "acb3a3cda9d260e39007b6aafd74efd002de974700e14a5535f66590b8b801d6"
+    },
+    "omni_components/residual_projection.py": {
+      "bytes": 1668,
+      "sha256": "30028685698c9b4779c44f8ebe28bf80c1689b99594ca4b287db16d133227ad4"
+    },
+    "omni_components/single_modality.py": {
+      "bytes": 4218,
+      "sha256": "a4493f2f53bec989f9eff7fbb4b63b9a47aafd2004fd74560911be62ed3237ea"
+    },
+    "omni_components/text_backbone.py": {
+      "bytes": 3244,
+      "sha256": "c11a92203f3010ef857d2c809257781150a790acf1877ed83e1a0021449268c3"
+    },
+    "omni_components/text_encoder.py": {
+      "bytes": 9409,
+      "sha256": "578329b6bf656423e0b9543783f30a6455449a7402fd3a1a9f601a8afc9705af"
+    },
+    "omni_components/tiny_clap_residual.py": {
+      "bytes": 4315,
+      "sha256": "bfbd151f52f91da907affbc33f7a61cea5b11c11cc5fa41d234277a99f112f6c"
+    },
+    "vela_omni.py": {
+      "bytes": 10436,
+      "sha256": "216c267591e0af3b4beb583e3be239b1af1c23300ccb74c5a79d58e9f79e52d3"
+    }
+  },
+  "same_as_actual_evaluated_native": true
+}

benchmarks/net-progress.json

@@ -1,547 +1,91 @@
 {
-  "scope": "Product-weighted progress comparison, not an official benchmark aggregate or SOTA claim.",
-  "baseline_model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-  "baseline_revision": "3a1efc48995e2b71622d65c6cf71cfc00bbf3aa3",
-  "baseline_native_artifact_sha256": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
-  "candidate_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-  "score_scale": "0-100; changes in percentage points",
+  "scope": "Fixed product progress metric; not an official benchmark aggregate, significance claim, or proof of overall SOTA.",
+  "baseline_revision": "9e747d2e1d4b97d3081ff4ac2ac79986c7a8da95",
+  "baseline_native_fingerprint": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
+  "baseline_evaluated_fingerprint": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
+  "candidate_native_fingerprint": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
   "components": [
     {
-      "baseline_100": 59.37287180704907,
-      "candidate_100": 67.57407300861536,
+      "candidate_100": 66.41835251574038,
       "component": "common14",
-      "delta_pp": 8.201201201566288,
+      "delta_pp": -1.1557204928749676,
+      "parent_100": 67.57407300861536,
       "weight": 0.3
     },
     {
-      "baseline_100": 51.97553164396411,
       "candidate_100": 60.781849756549235,
       "component": "English41_MeanTaskType",
-      "delta_pp": 8.806318112585124,
+      "delta_pp": 0.0,
+      "parent_100": 60.781849756549235,
       "weight": 0.3
     },
     {
-      "baseline_100": 54.865785925824184,
       "candidate_100": 53.65746394890041,
       "component": "image3",
-      "delta_pp": -1.208321976923776,
+      "delta_pp": 0.0,
+      "parent_100": 53.65746394890041,
       "weight": 0.2
     },
     {
-      "baseline_100": 46.9743843608,
-      "candidate_100": 46.96781408655678,
+      "candidate_100": 52.33540613685162,
       "component": "audio19_MeanTaskType",
-      "delta_pp": -0.00657027424322,
+      "delta_pp": 5.367592050294833,
+      "parent_100": 46.96781408655678,
       "weight": 0.2
     }
   ],
-  "weighted_net_delta_pp": 4.859277344012025,
-  "common_definition": "Equal mean of three families: two text accuracies, all six image retrieval recalls, all six audio retrieval recalls.",
-  "image3_definition": "Equal mean of OxfordPets classification accuracy, TinyImageNet clustering NMI and OxfordPets zero-shot accuracy.",
-  "text_audio_definition": "Full English41 and full audio19 mean task-type scores. English uses the explicit exact text-component bridge; audio is freshly evaluated.",
-  "coverage": {
-    "common_metrics": 14,
-    "standard_tasks": 11,
-    "standard_split_rows": 12,
-    "English_tasks": 41,
-    "audio_tasks": 19,
-    "audio_subset_split_slots": 134
-  },
+  "weighted_net_delta_pp": 0.7268022621964765,
+  "score_scale": "0–100; deltas in percentage points",
+  "parameter_increase_percent": 20.968150046422096,
+  "common_definition": "Mean of text accuracy (2), image recall (6), and audio recall (6), with equal one-third family weights.",
+  "image3_definition": "Mean of OxfordPets accuracy, TinyImageNetClustering NMI, and OxfordPetsZeroShot accuracy.",
   "common_metric_delta_pp": {
-    "audio.audio_to_text.recall@1": 8.157793948678668,
-    "audio.audio_to_text.recall@10": 15.51129835312141,
-    "audio.audio_to_text.recall@5": 14.362313289927231,
-    "audio.text_to_audio.recall@1": 11.417624521072797,
-    "audio.text_to_audio.recall@10": 15.478927203065133,
-    "audio.text_to_audio.recall@5": 14.444444444444445,
-    "image.image_to_text.recall@1": -4.738760631834751,
-    "image.image_to_text.recall@10": -1.2150668286755772,
-    "image.image_to_text.recall@5": -1.701093560145808,
-    "image.text_to_image.recall@1": 2.7460510328068044,
-    "image.text_to_image.recall@10": 0.7776427703523694,
-    "image.text_to_image.recall@5": 1.0206561360874848,
-    "text.banking77.accuracy": 11.168831168831169,
-    "text.massive-en.accuracy": 12.617765814266487
+    "audio.audio_to_text.recall@1": -1.0340865568747608,
+    "audio.audio_to_text.recall@10": -1.3787820758330127,
+    "audio.audio_to_text.recall@5": -1.8383761011106892,
+    "audio.text_to_audio.recall@1": -5.095785440613029,
+    "audio.text_to_audio.recall@10": -5.287356321839088,
+    "audio.text_to_audio.recall@5": -6.168582375478926,
+    "image.image_to_text.recall@1": 0.0,
+    "image.image_to_text.recall@10": 0.0,
+    "image.image_to_text.recall@5": 0.0,
+    "image.text_to_image.recall@1": 0.0,
+    "image.text_to_image.recall@10": 0.0,
+    "image.text_to_image.recall@5": 0.0,
+    "text.banking77.accuracy": 0.0,
+    "text.massive-en.accuracy": 0.0
   },
-  "standard_task_deltas": [
-    {
-      "baseline": 0.7169618925026362,
-      "candidate": 0.8053174565838819,
-      "hf_subset": "default",
-      "main_metric": "cosine_spearman",
-      "split": "test",
-      "status": "complete",
-      "task": "SICK-R",
-      "delta_pp": 8.835556408124567
-    },
-    {
-      "baseline": 0.7792400226361769,
-      "candidate": 0.870781967302773,
-      "hf_subset": "default",
-      "main_metric": "cosine_spearman",
-      "split": "test",
-      "status": "complete",
-      "task": "STSBenchmark",
-      "delta_pp": 9.154194466659614
-    },
-    {
-      "baseline": 0.37514,
-      "candidate": 0.59246,
-      "hf_subset": "default",
-      "main_metric": "ndcg_at_10",
-      "split": "test",
-      "status": "complete",
-      "task": "ArguAna",
-      "delta_pp": 21.732000000000003
-    },
-    {
-      "baseline": 0.44884717457494344,
-      "candidate": 0.5087260708286914,
-      "hf_subset": "default",
-      "main_metric": "v_measure",
-      "split": "test",
-      "status": "complete",
-      "task": "TwentyNewsgroupsClustering.v2",
-      "delta_pp": 5.987889625374793
-    },
-    {
-      "baseline": 0.8758836689038031,
-      "candidate": 0.9476510067114093,
-      "hf_subset": "en",
-      "main_metric": "accuracy",
-      "split": "validation",
-      "status": "complete",
-      "task": "MTOPDomainClassification",
-      "delta_pp": 7.176733780760625
-    },
-    {
-      "baseline": 0.8671910624715002,
-      "candidate": 0.9491792065663475,
-      "hf_subset": "en",
-      "main_metric": "accuracy",
-      "split": "test",
-      "status": "complete",
-      "task": "MTOPDomainClassification",
-      "delta_pp": 8.198814409484722
-    },
-    {
-      "baseline": 0.9126737530662306,
-      "candidate": 0.9126737530662306,
-      "hf_subset": "default",
-      "main_metric": "accuracy",
-      "split": "test",
-      "status": "complete",
-      "task": "OxfordPets",
-      "delta_pp": 0.0
-    },
-    {
-      "baseline": 0.6169193395626786,
-      "candidate": 0.6169193395626786,
-      "hf_subset": "default",
-      "main_metric": "nmi",
-      "split": "valid",
-      "status": "complete",
-      "task": "TinyImageNetClustering",
-      "delta_pp": 0.0
-    },
-    {
-      "baseline": 0.1163804851458163,
-      "candidate": 0.08013082583810302,
-      "hf_subset": "default",
-      "main_metric": "accuracy",
-      "split": "test",
-      "status": "complete",
-      "task": "OxfordPetsZeroShot",
-      "delta_pp": -3.6249659307713276
-    },
-    {
-      "baseline": 0.29763670140167686,
-      "candidate": 0.29763670140167686,
-      "hf_subset": "default",
-      "main_metric": "accuracy",
-      "split": "train",
-      "status": "complete",
-      "task": "CREMA_D",
-      "delta_pp": 0.0
-    },
-    {
-      "baseline": 0.023601715918443712,
-      "candidate": 0.023601715918443712,
-      "hf_subset": "default",
-      "main_metric": "v_measure",
-      "split": "train",
-      "status": "complete",
-      "task": "CREMA_DClustering",
-      "delta_pp": 0.0
-    },
-    {
-      "baseline": 0.14972999509081983,
-      "candidate": 0.1411389297987236,
-      "hf_subset": "default",
-      "main_metric": "accuracy",
-      "split": "test",
-      "status": "complete",
-      "task": "SpeechCommandsZeroshotv0.02",
-      "delta_pp": -0.8591065292096217
-    }
-  ],
-  "English_task_deltas": [
-    {
-      "candidate_100": 59.245999999999995,
-      "current_100": 37.513999999999996,
-      "delta_pp": 21.732,
-      "primary_metric": "ndcg_at_10",
-      "task": "ArguAna",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 64.88847259326147,
-      "current_100": 65.25395409281045,
-      "delta_pp": -0.36548149954897724,
-      "primary_metric": "v_measure",
-      "task": "ArXivHierarchicalClusteringP2P",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 57.44315313869382,
-      "current_100": 55.989532291560415,
-      "delta_pp": 1.4536208471334078,
-      "primary_metric": "v_measure",
-      "task": "ArXivHierarchicalClusteringS2S",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 62.329,
-      "current_100": 60.514,
-      "delta_pp": 1.8149999999999977,
-      "primary_metric": "map_at_1000",
-      "task": "AskUbuntuDupQuestions",
-      "task_type": "Reranking"
-    },
-    {
-      "candidate_100": 86.98752423842483,
-      "current_100": 62.24539420714732,
-      "delta_pp": 24.742130031277505,
-      "primary_metric": "cosine_spearman",
-      "task": "BIOSSES",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 82.14285714285714,
-      "current_100": 69.57467532467533,
-      "delta_pp": 12.568181818181813,
-      "primary_metric": "accuracy",
-      "task": "Banking77Classification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 41.313796273674654,
-      "current_100": 39.82923119776807,
-      "delta_pp": 1.4845650759065876,
-      "primary_metric": "v_measure",
-      "task": "BiorxivClusteringP2P.v2",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 56.979,
-      "current_100": 42.976,
-      "delta_pp": 14.003,
-      "primary_metric": "ndcg_at_10",
-      "task": "CQADupstackGamingRetrieval",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 39.586,
-      "current_100": 30.362000000000002,
-      "delta_pp": 9.223999999999997,
-      "primary_metric": "ndcg_at_10",
-      "task": "CQADupstackUnixRetrieval",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 31.846000000000004,
-      "current_100": 16.073999999999998,
-      "delta_pp": 15.772000000000006,
-      "primary_metric": "ndcg_at_10",
-      "task": "ClimateFEVERHardNegatives",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 87.576,
-      "current_100": 24.055,
-      "delta_pp": 63.520999999999994,
-      "primary_metric": "ndcg_at_10",
-      "task": "FEVERHardNegatives",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 39.143,
-      "current_100": 22.588,
-      "delta_pp": 16.555,
-      "primary_metric": "ndcg_at_10",
-      "task": "FiQA2018",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 66.349,
-      "current_100": 26.987,
-      "delta_pp": 39.36200000000001,
-      "primary_metric": "ndcg_at_10",
-      "task": "HotpotQAHardNegatives",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 91.94760000000001,
-      "current_100": 62.1364,
-      "delta_pp": 29.811200000000007,
-      "primary_metric": "accuracy",
-      "task": "ImdbClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 94.91792065663475,
-      "current_100": 86.71910624715002,
-      "delta_pp": 8.198814409484726,
-      "primary_metric": "accuracy",
-      "task": "MTOPDomainClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 70.96839273705447,
-      "current_100": 60.39340954942838,
-      "delta_pp": 10.574983187626096,
-      "primary_metric": "accuracy",
-      "task": "MassiveIntentClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 76.11297915265635,
-      "current_100": 73.37256220578345,
-      "delta_pp": 2.740416946872898,
-      "primary_metric": "accuracy",
-      "task": "MassiveScenarioClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 39.83023091223438,
-      "current_100": 37.76813963073551,
-      "delta_pp": 2.0620912814988657,
-      "primary_metric": "v_measure",
-      "task": "MedrxivClusteringP2P.v2",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 37.875084453707764,
-      "current_100": 36.13349306174702,
-      "delta_pp": 1.7415913919607462,
-      "primary_metric": "v_measure",
-      "task": "MedrxivClusteringS2S.v2",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 32.366,
-      "current_100": 30.623,
-      "delta_pp": 1.7429999999999986,
-      "primary_metric": "max_over_subqueries_map_at_1000",
-      "task": "MindSmallReranking",
-      "task_type": "Reranking"
-    },
-    {
-      "candidate_100": 21.889,
-      "current_100": 16.827,
-      "delta_pp": 5.061999999999998,
-      "primary_metric": "ndcg_at_10",
-      "task": "SCIDOCS",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 80.53174565838819,
-      "current_100": 71.69618925026361,
-      "delta_pp": 8.835556408124575,
-      "primary_metric": "cosine_spearman",
-      "task": "SICK-R",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 75.5658801516535,
-      "current_100": 68.16915974310523,
-      "delta_pp": 7.396720408548276,
-      "primary_metric": "cosine_spearman",
-      "task": "STS12",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 86.26346974038235,
-      "current_100": 74.40342742804673,
-      "delta_pp": 11.860042312335622,
-      "primary_metric": "cosine_spearman",
-      "task": "STS13",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 82.29882793900394,
-      "current_100": 68.02653891259911,
-      "delta_pp": 14.272289026404835,
-      "primary_metric": "cosine_spearman",
-      "task": "STS14",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 88.73648284326485,
-      "current_100": 77.86144933320074,
-      "delta_pp": 10.875033510064114,
-      "primary_metric": "cosine_spearman",
-      "task": "STS15",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 87.0781967302773,
-      "current_100": 77.92400226361768,
-      "delta_pp": 9.154194466659618,
-      "primary_metric": "cosine_spearman",
-      "task": "STSBenchmark",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 95.79964673462752,
-      "current_100": 89.79096125036985,
-      "delta_pp": 6.0086854842576685,
-      "primary_metric": "max_ap",
-      "task": "SprintDuplicateQuestions",
-      "task_type": "PairClassification"
-    },
-    {
-      "candidate_100": 58.424418738430106,
-      "current_100": 57.34464990059502,
-      "delta_pp": 1.0797688378350827,
-      "primary_metric": "v_measure",
-      "task": "StackExchangeClustering.v2",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 41.053017253245386,
-      "current_100": 39.37668627014545,
-      "delta_pp": 1.6763309830999376,
-      "primary_metric": "v_measure",
-      "task": "StackExchangeClusteringP2P.v2",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 69.121,
-      "current_100": 41.292,
-      "delta_pp": 27.828999999999994,
-      "primary_metric": "ndcg_at_10",
-      "task": "TRECCOVID",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 48.363,
-      "current_100": 31.477,
-      "delta_pp": 16.886,
-      "primary_metric": "ndcg_at_10",
-      "task": "Touche2020Retrieval.v3",
-      "task_type": "Retrieval"
-    },
-    {
-      "candidate_100": 71.904296875,
-      "current_100": 64.4091796875,
-      "delta_pp": 7.4951171875,
-      "primary_metric": "accuracy",
-      "task": "ToxicConversationsClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 62.79286926994907,
-      "current_100": 50.77815506508207,
-      "delta_pp": 12.014714204867005,
-      "primary_metric": "accuracy",
-      "task": "TweetSentimentExtractionClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 50.87260708286914,
-      "current_100": 44.88471745749435,
-      "delta_pp": 5.987889625374791,
-      "primary_metric": "v_measure",
-      "task": "TwentyNewsgroupsClustering.v2",
-      "task_type": "Clustering"
-    },
-    {
-      "candidate_100": 72.95151691526065,
-      "current_100": 56.34475441213181,
-      "delta_pp": 16.60676250312884,
-      "primary_metric": "max_ap",
-      "task": "TwitterSemEval2015",
-      "task_type": "PairClassification"
-    },
-    {
-      "candidate_100": 85.30340320492272,
-      "current_100": 78.70209561308617,
-      "delta_pp": 6.6013075918365445,
-      "primary_metric": "max_ap",
-      "task": "TwitterURLCorpus",
-      "task_type": "PairClassification"
-    },
-    {
-      "candidate_100": 31.834434345907127,
-      "current_100": 29.249640152019634,
-      "delta_pp": 2.584794193887493,
-      "primary_metric": "cosine_spearman",
-      "task": "SummEvalSummarization.v2",
-      "task_type": "Summarization"
-    },
-    {
-      "candidate_100": 71.8507462686567,
-      "current_100": 56.82089552238806,
-      "delta_pp": 15.02985074626865,
-      "primary_metric": "accuracy",
-      "task": "AmazonCounterfactualClassification",
-      "task_type": "Classification"
-    },
-    {
-      "candidate_100": 89.02101343562433,
-      "current_100": 84.16589279728525,
-      "delta_pp": 4.8551206383390735,
-      "primary_metric": "cosine_spearman",
-      "task": "STS17",
-      "task_type": "STS"
-    },
-    {
-      "candidate_100": 68.75333638044488,
-      "current_100": 67.57005849656171,
-      "delta_pp": 1.1832778838831643,
-      "primary_metric": "cosine_spearman",
-      "task": "STS22.v2",
-      "task_type": "STS"
-    }
-  ],
   "audio_task_delta_pp": {
-    "BeijingOpera": 0.0,
-    "BirdCLEF": 0.0,
-    "CREMADPairClassification": 0.12104996267769952,
-    "CREMA_D": 0.0,
-    "CREMA_DClustering": 0.0,
-    "CommonLanguageAgeDetection": 0.0,
-    "GTZANAudioReranking": 0.0,
-    "GTZANGenre": 0.0,
-    "IEMOCAPGender": 0.0,
-    "JamAltArtistA2ARetrieval": 0.0,
-    "MInDS14": 0.0,
-    "MridinghamTonic": -0.14336917562723928,
-    "NMSQAPairClassification": -0.1520590751812989,
-    "SIBFLEURS": 0.0,
-    "VehicleSoundClustering": 0.0,
-    "VoxCelebSA": 0.0,
-    "VoxPopuliAccentPairClassification": -0.02844431688273641,
-    "VoxPopuliGenderClustering": 0.0,
-    "VoxPopuliLanguageID": 0.0
+    "BeijingOpera": 10.549645390070905,
+    "BirdCLEF": 2.2999999999999963,
+    "CREMADPairClassification": 1.0060252268561554,
+    "CREMA_D": 2.2844768445301367,
+    "CREMA_DClustering": 2.8541749889603825,
+    "CommonLanguageAgeDetection": 0.6150000000000017,
+    "GTZANAudioReranking": 6.68200000000001,
+    "GTZANGenre": 11.399999999999999,
+    "IEMOCAPGender": 3.237875602721152,
+    "JamAltArtistA2ARetrieval": 16.90099999999998,
+    "MInDS14": -1.4691308536005643,
+    "MridinghamTonic": 26.05657742038183,
+    "NMSQAPairClassification": -0.3840625955228716,
+    "SIBFLEURS": -0.23211656203982745,
+    "VehicleSoundClustering": -8.525275870377525,
+    "VoxCelebSA": -0.00058896531414665,
+    "VoxPopuliAccentPairClassification": 0.1293630037889515,
+    "VoxPopuliGenderClustering": -0.08127278385060711,
+    "VoxPopuliLanguageID": -0.5999999999999783
+  },
+  "audio_aggregate_delta_pp": {
+    "Any2AnyRetrieval": 16.90099999999998,
+    "AudioClassification": 4.921976261522682,
+    "AudioClustering": -1.9174578884225837,
+    "AudioPairClassification": 0.2504418783740858,
+    "AudioReranking": 6.68200000000001,
+    "Mean(Task)": 3.8275626761370307,
+    "Mean(TaskType)": 5.367592050294833
   },
-  "paired_interval_scope": "Paired intervals cover the original common metrics, family and two-primary-per-family utility. They are not confidence intervals for the new all-14 common mean or the four-component product aggregate.",
-  "inference_profile_applicability": {
-    "revision": "self",
-    "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-    "parameters": 135383808,
-    "report": "benchmarks/inference-equivalence.json",
-    "new_benchmark_run": false
-  }
+  "English_and_image_unchanged": true,
+  "event840_in_net": false,
+  "paired_interval_scope": "Fresh paired intervals compare this candidate with the preceding Vela Nano computation. They are not an interval for the all-14 aggregate or the weighted net. An original-small interval was not recomputed for this update."
 }

benchmarks/original-baseline.json

@@ -113,8 +113,11 @@
       "score": null
     },
     "selected_standard_11": {
-      "status": "not_evaluated_in_this_matched_protocol",
-      "score": null
+      "status": "evaluated_all_11_selected_tasks",
+      "task_count": 11,
+      "split_rows": 12,
+      "report": "original-standard.json",
+      "report_sha256": "f07e4624749868498923cf3c7d521ca9547276dbdbfb002cbe97bd074195b55a"
     }
   },
   "not_previous_Vela": true,

benchmarks/pareto-data.json

@@ -2569,8 +2569,8 @@
         },
         {
           "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-          "parameters_billion": 0.135383808,
-          "parameters_exact": 135383808,
+          "parameters_billion": 0.163771288,
+          "parameters_exact": 163771288,
           "score": 64.88847259326147,
           "open_weights": true,
           "modalities": [
@@ -2582,18 +2582,18 @@
           "kind": "nano",
           "evaluated_task_revision": "0bbdb47bcbe3a90093699aefeed338a0f28a7ee8",
           "on_displayed_frontier": false,
-          "evidence_type": "exact_public_inference_applicability",
+          "evidence_type": "exact_text_component_applicability",
           "observation_id": "vela_release_evaluation:llm-semantic-router/Vela-1.0-Omni-Nano",
-          "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-          "revision": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
+          "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+          "revision": null,
           "evaluated_native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
           "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
           "evaluated_model": "local/Vela-Omni-Nano-GIST-Rotation",
           "revision_semantics": "Evaluated revisions are native artifact fingerprints; applies-to identity is the current whole artifact.",
-          "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/mteb-eng-v2.json",
+          "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/mteb-eng-v2.json",
           "prior_applicable_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-          "inference_equivalence_report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/inference-equivalence.json",
-          "revision_kind": "native_artifact_fingerprint"
+          "revision_kind": "native_artifact_fingerprint",
+          "identity_ref": "models.nano.task_evidence.English41"
         },
         {
           "model": "ibm-granite/granite-embedding-english-r2",
@@ -9675,9 +9675,9 @@
         },
         {
           "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-          "parameters_billion": 0.135383808,
-          "parameters_exact": 135383808,
-          "score": 63.28340215485784,
+          "parameters_billion": 0.163771288,
+          "parameters_exact": 163771288,
+          "score": 62.89933955933497,
           "open_weights": true,
           "modalities": [
             "text",
@@ -9688,17 +9688,16 @@
           "kind": "nano",
           "evaluated_task_revision": "d33eea302d8a64c0a2ee094cf39d77c0814fe87a",
           "on_displayed_frontier": true,
-          "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-          "revision": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-          "evaluated_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-          "evaluated_revision": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
+          "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+          "revision": null,
+          "evaluated_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+          "evaluated_revision": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
           "evaluated_model": "llm-semantic-router/Vela-1.0-Omni-Nano",
           "revision_semantics": "Evaluated revisions are native artifact fingerprints; applies-to identity is the current whole artifact.",
-          "evidence_type": "exact_public_inference_applicability",
-          "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/maeb-audio-only.json",
-          "prior_applicable_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-          "inference_equivalence_report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/inference-equivalence.json",
-          "revision_kind": "native_artifact_fingerprint"
+          "evidence_type": "measured_release_evaluation",
+          "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/maeb-audio-only.json",
+          "revision_kind": "native_artifact_fingerprint",
+          "identity_ref": "models.nano.task_evidence.audio19"
         },
         {
           "model": "laion/clap-htsat-unfused",
@@ -10829,9 +10828,9 @@
         },
         {
           "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-          "parameters_billion": 0.135383808,
-          "score": 63.28340215485784,
-          "evidence_type": "exact_public_inference_applicability"
+          "parameters_billion": 0.163771288,
+          "score": 62.89933955933497,
+          "evidence_type": "measured_release_evaluation"
         },
         {
           "model": "microsoft/speecht5_multimodal",
@@ -10873,7 +10872,7 @@
       "measured_peer_observations": 0,
       "highlight_variant": "nano",
       "highlight_dominators": [],
-      "highlight_gap_above_best_equal_or_smaller_peer_pp": 5.003102154857842
+      "highlight_gap_above_best_equal_or_smaller_peer_pp": 4.619039559334972
     },
     "sibfleurs": {
       "task": "SIBFLEURS",
@@ -11356,9 +11355,9 @@
         },
         {
           "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-          "parameters_billion": 0.135383808,
-          "parameters_exact": 135383808,
-          "score": 18.304657831512046,
+          "parameters_billion": 0.163771288,
+          "parameters_exact": 163771288,
+          "score": 18.072541269472218,
           "open_weights": true,
           "modalities": [
             "text",
@@ -11369,17 +11368,16 @@
           "kind": "nano",
           "evaluated_task_revision": "186b61175fbd77059f769b6bf1110d449a2ff311",
           "on_displayed_frontier": false,
-          "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-          "revision": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-          "evaluated_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-          "evaluated_revision": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
+          "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+          "revision": null,
+          "evaluated_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+          "evaluated_revision": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
           "evaluated_model": "llm-semantic-router/Vela-1.0-Omni-Nano",
           "revision_semantics": "Evaluated revisions are native artifact fingerprints; applies-to identity is the current whole artifact.",
-          "evidence_type": "exact_public_inference_applicability",
-          "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/maeb-audio-only.json",
-          "prior_applicable_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-          "inference_equivalence_report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/inference-equivalence.json",
-          "revision_kind": "native_artifact_fingerprint"
+          "evidence_type": "measured_release_evaluation",
+          "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/maeb-audio-only.json",
+          "revision_kind": "native_artifact_fingerprint",
+          "identity_ref": "models.nano.task_evidence.audio19"
         },
         {
           "model": "laion/clap-htsat-unfused",
@@ -13083,9 +13081,9 @@
         },
         {
           "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-          "parameters_billion": 0.135383808,
-          "parameters_exact": 135383808,
-          "score": 13.800701178563807,
+          "parameters_billion": 0.163771288,
+          "parameters_exact": 163771288,
+          "score": 5.275425308186281,
           "open_weights": true,
           "modalities": [
             "text",
@@ -13096,17 +13094,16 @@
           "kind": "nano",
           "evaluated_task_revision": "5f905f9544d0c78b607639662c88c2ddd1216384",
           "on_displayed_frontier": false,
-          "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-          "revision": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-          "evaluated_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-          "evaluated_revision": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
+          "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+          "revision": null,
+          "evaluated_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+          "evaluated_revision": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
           "evaluated_model": "llm-semantic-router/Vela-1.0-Omni-Nano",
           "revision_semantics": "Evaluated revisions are native artifact fingerprints; applies-to identity is the current whole artifact.",
-          "evidence_type": "exact_public_inference_applicability",
-          "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/maeb-audio-only.json",
-          "prior_applicable_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-          "inference_equivalence_report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/inference-equivalence.json",
-          "revision_kind": "native_artifact_fingerprint"
+          "evidence_type": "measured_release_evaluation",
+          "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/maeb-audio-only.json",
+          "revision_kind": "native_artifact_fingerprint",
+          "identity_ref": "models.nano.task_evidence.audio19"
         },
         {
           "model": "laion/clap-htsat-unfused",
@@ -14297,6 +14294,25 @@
       "measured_peer_observations": 1,
       "highlight_variant": "nano",
       "highlight_dominators": [
+        {
+          "model": "openai/whisper-tiny",
+          "parameters_billion": 0.039,
+          "score": 12.3883,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": true,
+          "evidence_type": "registry_reported",
+          "observation_id": "registry_reported:openai/whisper-tiny"
+        },
         {
           "model": "MIT/ast-finetuned-audioset-10-10-0.4593",
           "parameters_billion": 0.086594063,
@@ -14319,17 +14335,38 @@
           "report": "benchmarks/peer-evaluations/ast-vehicle.json",
           "license": "bsd-3-clause",
           "on_displayed_frontier": true
+        },
+        {
+          "model": "MIT/ast-finetuned-audioset-10-10-0.4593",
+          "parameters_billion": 0.087,
+          "score": 13.366700000000002,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "AudioSetMini"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false,
+          "evidence_type": "registry_reported",
+          "observation_id": "registry_reported:MIT/ast-finetuned-audioset-10-10-0.4593"
         }
       ],
-      "highlight_gap_above_best_equal_or_smaller_peer_pp": -0.5421709345231776
+      "highlight_gap_above_best_equal_or_smaller_peer_pp": -9.067446804900705
     }
   },
   "models": {
     "nano": {
       "repo_id": "llm-semantic-router/Vela-1.0-Omni-Nano",
-      "release_commit": "d8ac5b5ac2274a501fc61aeb5be70cec1855a806",
-      "parameters_exact": 135383808,
-      "current_native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
+      "release_commit": null,
+      "parameters_exact": 163771288,
+      "current_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
       "task_evidence": {
         "English41": {
           "source_id": "nano_English41",
@@ -14337,47 +14374,58 @@
             "model": "local/Vela-Omni-Nano-GIST-Rotation",
             "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
             "native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+            "revision_semantics": "local native artifact fingerprint, not a Hugging Face commit",
             "component_applicability": {
               "component": "text",
-              "applies_to_revision": "self",
-              "applies_to_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
               "evaluated_model": "local/Vela-Omni-Nano-GIST-Rotation",
-              "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
               "evaluated_native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
-              "revision_kind": "local native artifact fingerprint, not a Hugging Face commit",
-              "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/component-equivalence.json",
+              "applies_to_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+              "prior_component_applicability": {
+                "model": "local/Vela-Omni-Nano-GIST-Rotation",
+                "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+                "native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+                "component_applicability": {
+                  "component": "text",
+                  "applies_to_revision": "self",
+                  "applies_to_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
+                  "evaluated_model": "local/Vela-Omni-Nano-GIST-Rotation",
+                  "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+                  "evaluated_native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+                  "revision_kind": "local native artifact fingerprint, not a Hugging Face commit",
+                  "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/component-equivalence.json",
+                  "new_full_panel_inference": false,
+                  "basis": "199 exact text tensors and byte-identical inference/tokenizer/configuration under the same official evaluator protocol."
+                },
+                "revision_semantics": "local native artifact fingerprint, not a Hugging Face commit",
+                "inference_profile_applicability": {
+                  "revision": "self",
+                  "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
+                  "parameters": 135383808,
+                  "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/inference-equivalence.json",
+                  "new_benchmark_run": false
+                }
+              },
+              "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/component-equivalence.json",
               "new_full_panel_inference": false,
-              "basis": "199 exact text tensors and byte-identical inference/tokenizer/configuration under the same official evaluator protocol."
-            },
-            "revision_semantics": "local native artifact fingerprint, not a Hugging Face commit",
-            "inference_profile_applicability": {
-              "revision": "self",
-              "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-              "parameters": 135383808,
-              "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/inference-equivalence.json",
-              "new_benchmark_run": false
+              "basis": "The frozen text tensors, tokenizer, configuration and encoding computation are unchanged. The original measured text identity is retained."
             }
           },
-          "self_reference_interpretation": "d8ac5b5ac2274a501fc61aeb5be70cec1855a806"
+          "self_reference_interpretation": "Nano native fingerprint 474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2"
         },
         "audio19": {
           "source_id": "nano_audio19",
           "identity": {
             "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-            "evaluated_revision": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-            "native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-            "revision_semantics": "Native artifact fingerprint identifying the evaluated weights and inference files; the release commit is the repository revision containing this report.",
-            "inference_profile_applicability": {
-              "revision": "self",
-              "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-              "parameters": 135383808,
-              "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/inference-equivalence.json",
-              "new_benchmark_run": false
-            }
+            "evaluated_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+            "release_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+            "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/maeb-audio-only.json",
+            "new_full_panel_inference": true
           },
-          "self_reference_interpretation": "d8ac5b5ac2274a501fc61aeb5be70cec1855a806"
+          "self_reference_interpretation": "Nano native fingerprint 474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2"
         }
-      }
+      },
+      "release_reference": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano",
+      "release_revision_semantics": "Exact native fingerprint below; Hub commit is not encoded in this file."
     },
     "mini": {
       "repo_id": "llm-semantic-router/Vela-1.0-Omni-Mini",
@@ -14438,14 +14486,16 @@
   },
   "sources": {
     "nano_English41": {
-      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/mteb-eng-v2.json",
-      "sha256": "733f6a58faae3d04e1b5a59de73085b156dc9987ba0e04e28548c17dc5904cec",
-      "evidence_type": "Vela_published_report"
+      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/mteb-eng-v2.json",
+      "evidence_type": "Vela_release_report",
+      "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+      "sha256": "f3daecad1cf431630598efb04f3667431e3420239240f28a51568a74cef0c64f"
     },
     "nano_audio19": {
-      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/maeb-audio-only.json",
-      "sha256": "e0259c59e04649f42cd2f004d57b525c79442297d28498a7434bddb4b000bdf9",
-      "evidence_type": "Vela_published_report"
+      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/maeb-audio-only.json",
+      "evidence_type": "Vela_release_report",
+      "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+      "sha256": "4fc7f217745eb0aa998bf0f8e044bd1fa5de2f73f1a7dead7b829a62c48669ff"
     },
     "mini_English41": {
       "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Mini/resolve/main/benchmarks/mteb-eng-v2.json",
@@ -14478,6 +14528,12 @@
       "sha256": "8bbbd3a362d84a6e8f030803259db69e932e5997b1c7af1a276a0d17862eb184",
       "evidence_type": "exact_component_identity_report",
       "native_artifact_sha256": "d3a742fbe0d833b0a72e5104e0381209b7d3da78f948d3439a5da7d986232731"
+    },
+    "nano_component_equivalence": {
+      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/component-equivalence.json",
+      "evidence_type": "Vela_release_report",
+      "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+      "sha256": "ebd40f9a887791d1736132c37f510a69e18a03e247422853f54a42b417b0db9b"
     }
   }
 }

benchmarks/pareto-gallery.md

@@ -1,18 +1,16 @@
-# Vela Nano: quality, size and rank
+# Figure gallery
 
-Complete-panel comparisons show overall position; selected tasks show specific strengths.
+Complete-panel comparisons are primary. Scores are measured or registry-reported under the protocols recorded in the data; no incomplete panel contributes an aggregate.
 
-![Complete English benchmark: size–quality comparison and ranking](../assets/complete-panel-english41-nano.png)
+![Complete English41](../assets/complete-panel-english41-nano.png)
 
-![Complete audio benchmark: size–quality comparison and ranking](../assets/complete-panel-audio19-mini.png)
+![Complete audio19](../assets/complete-panel-audio19-mini.png)
 
-## Selected tasks
+Task comparisons are secondary and do not establish overall SOTA. Highlighting does not imply frontier membership.
 
-<table>
-<tr>
-<td width="50%"><img src="../assets/pareto-imdb.png" alt="Vela Nano: IMDb · Text classification" /></td>
-<td width="50%"><img src="../assets/pareto-nmsqa.png" alt="Vela Nano: NMSQA · Audio pair classification" /></td>
-</tr>
-</table>
+| Nano | Mini |
+| --- | --- |
+| IMDb sentiment classification | Mridingham musical tonic classification |
+| NMSQA audio pair classification | SIB-FLEURS spoken-topic classification |
 
-[Data and methodology](pareto-methodology.md) · [Complete evaluation](EVALUATION.md)
+[Full ranks](complete-panel-ranks.md) · [Methods and provenance](pareto-methodology.md)

benchmarks/pareto-methodology.md

@@ -8,18 +8,20 @@ Rankings use the September 17, 2026 dedicated MTEB registry snapshots plus both
 
 The plots show every complete model with a known positive total parameter count. The dashed Pareto line connects nondominated observations: no other eligible model has both no more parameters and no lower score, with one strict improvement. Vela points remain visible even when they are below this line. The adjacent Top 10 restricts the complete population to models no larger than the highlighted Vela model. All comparisons use unrounded scores.
 
-Vela sizes count all stored model parameters across text, image and audio: Nano 135,383,808 and Mini 1,361,475,288. Peer sizes use registry-reported totals, which can be rounded. Peers include both single-modality specialists and multimodal encoders. Their scores were reported under differing prompts, model revisions and evaluation protocols; these are snapshot-relative comparisons, not matched reproductions or official leaderboard submissions. There is no combined text–image–audio rank. Neither current Vela model lies on these complete-panel frontiers.
+Vela sizes count all stored model parameters across text, image and audio: Nano 163,771,288 and Mini 1,361,475,288. Peer sizes use registry-reported totals, which can be rounded. Peers include both single-modality specialists and multimodal encoders. Their scores were reported under differing prompts, model revisions and evaluation protocols; these are snapshot-relative comparisons, not matched reproductions or official leaderboard submissions. There is no combined text–image–audio rank. Neither current Vela model lies on these complete-panel frontiers.
 
-[All ranks](complete-panel-ranks.md) · [Complete vectors, source URLs, exclusions and plotted points](complete-panel-data.json) · [Figure metadata](complete-panel-metadata.json) · [Vela evaluation protocol](../EVALUATION.md)
+[All ranks](complete-panel-ranks.md) · [Complete vectors, source URLs, exclusions and plotted points](complete-panel-data.json) · [Figure metadata](complete-panel-metadata.json) · [Vela evaluation protocol](https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/main/benchmarks/EVALUATION.md)
 
 ## Selected task strengths
 
-The task gallery highlights Nano on IMDb classification and NMSQA audio pair classification, and Mini on Mridingham tonic classification and SIB-FLEURS spoken-topic classification. Each highlighted model is nondominated among the documented references for that particular task. A task-level position does not establish overall SOTA, and neither a label nor a displayed highlight removes other eligible points from the frontier calculation.
+The task gallery highlights Nano on IMDb classification and NMSQA audio pair classification, and Mini on Mridingham tonic classification and SIB-FLEURS spoken-topic classification. Highlighted models are shown whether or not they lie on the observed frontier; frontier membership is recomputed from all documented eligible points. A task-level position does not establish overall SOTA, and neither a label nor a displayed highlight removes other eligible points from the frontier calculation.
 
 The task plots retain finite scores with known positive total size, including closed-weight models. Registry observations and separately measured peers retain their individual provenance. A stronger new observation can change the frontier without any change in Vela's own score. Complete benchmark results, including regressions, remain in the evaluation report.
 
 [IMDb and Mridingham points and sources](pareto-new-frontiers.json) · [NMSQA and SIB-FLEURS points and sources](pareto-data.json) · [Other retained task observations (CSV)](pareto-points.csv)
 
-Mini audio uses the newly completed 19-task evaluation of this release; its unchanged English text computation retains the original evaluated identity through an explicit component bridge. Per-task regressions remain in the complete report. No reported peer coordinate was replaced by a partial-panel rerun.
+Mini audio retains its existing complete 19-task evaluation; its unchanged English text computation retains the original evaluated identity through an explicit component bridge. Per-task regressions remain in the complete report. No reported peer coordinate was replaced by a partial-panel rerun.
 
 SIB-FLEURS measures topic classification of multilingual audio, not language identification. This description follows pinned MTEB 2.21.0 task metadata (`mteb/tasks/classification/multilingual/sibfleurs.py`, SHA-256 `f88cc34acc7414dc84ccf8cbbd2f79456076df126f9acc696bd7f45c9481ad55`), with [dataset revision 186b6117](https://huggingface.co/datasets/mteb/sib-fleurs-multilingual-mini/tree/186b61175fbd77059f769b6bf1110d449a2ff311).
+
+Nano now includes the complete 19-task audio evaluation at 163,771,288 total parameters. Its English computation is unchanged; Mini scores and size are unchanged. The new Nano audio point improves the complete-panel aggregate but individual regressions remain visible, including NMSQA and VehicleSoundClustering. The NMSQA highlight is a task comparison, not a claim of frontier membership. The general complete-panel rankings take priority over selected task plots.

benchmarks/pareto-new-frontiers.json

@@ -6,39 +6,17 @@
   "observation_policy": "Only exactly equal model/name/size/score duplicates are coalesced with every source retained. Differing registry scores remain separate observations, with scored revision unknown. F2/AST independent reruns remain separate pinned observations and do not replace reported coordinates. Current other Vela variant is included.",
   "dominance": "A peer has no more total parameters and no lower score, with at least one strict inequality. Scores use original available floating values, not rounded display labels.",
   "sources": {
-    "scores_MTEB_eng_v2": {
-      "url": "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MTEB%28eng%2C%20v2%29/scores",
-      "snapshot_sha256": "e574f14dc778d72c08255450091191b103930c487fa172a2dfabe41a0b56569d",
-      "retrieved_at_utc": "2026-09-17T20:28:47.419876+00:00",
-      "bytes": 1071836,
-      "benchmark": "MTEB(eng, v2)",
-      "scored_checkpoint_revisions_exposed": false
-    },
-    "scores_MAEB_beta": {
-      "url": "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores",
-      "snapshot_sha256": "601467f576706074bb5724aa0c10a50e948b0e8be8158742b01baf2fd7f7f14c",
-      "retrieved_at_utc": "2026-09-17T20:28:48.161556+00:00",
-      "bytes": 222458,
-      "benchmark": "MAEB(beta)",
-      "scored_checkpoint_revisions_exposed": false
-    },
-    "scores_MAEB_beta_audio-only": {
-      "url": "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
-      "snapshot_sha256": "6153942c26a31577cc1e3a2fbb6eb10b09dc2f1fe08749db598d261e653f6bca",
-      "retrieved_at_utc": "2026-09-17T20:28:48.670374+00:00",
-      "bytes": 194971,
-      "benchmark": "MAEB(beta, audio-only)",
-      "scored_checkpoint_revisions_exposed": false
-    },
     "nano_English41": {
-      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/mteb-eng-v2.json",
-      "sha256": "733f6a58faae3d04e1b5a59de73085b156dc9987ba0e04e28548c17dc5904cec",
-      "evidence_type": "Vela_published_report"
+      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/mteb-eng-v2.json",
+      "evidence_type": "Vela_release_report",
+      "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+      "sha256": "f3daecad1cf431630598efb04f3667431e3420239240f28a51568a74cef0c64f"
     },
     "nano_audio19": {
-      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/maeb-audio-only.json",
-      "sha256": "e0259c59e04649f42cd2f004d57b525c79442297d28498a7434bddb4b000bdf9",
-      "evidence_type": "Vela_published_report"
+      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/maeb-audio-only.json",
+      "evidence_type": "Vela_release_report",
+      "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+      "sha256": "4fc7f217745eb0aa998bf0f8e044bd1fa5de2f73f1a7dead7b829a62c48669ff"
     },
     "mini_English41": {
       "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Mini/resolve/main/benchmarks/mteb-eng-v2.json",
@@ -52,19 +30,39 @@
       "native_artifact_sha256": "d3a742fbe0d833b0a72e5104e0381209b7d3da78f948d3439a5da7d986232731",
       "sha256": "b9adc831e1de9caed37d38292dc95d65f4f933178fa7abbf7c1211f23617efbf"
     },
+    "scores_MTEB_eng_v2": {
+      "url": "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MTEB%28eng%2C%20v2%29/scores",
+      "sha256": "e574f14dc778d72c08255450091191b103930c487fa172a2dfabe41a0b56569d",
+      "retrieved_at_utc": "2026-09-17T20:28:47.419876+00:00",
+      "bytes": 1071836,
+      "benchmark": "MTEB(eng, v2)"
+    },
+    "scores_MAEB_beta_audio-only": {
+      "url": "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+      "sha256": "6153942c26a31577cc1e3a2fbb6eb10b09dc2f1fe08749db598d261e653f6bca",
+      "retrieved_at_utc": "2026-09-17T20:28:48.670374+00:00",
+      "bytes": 194971,
+      "benchmark": "MAEB(beta, audio-only)"
+    },
     "mini_component_equivalence": {
       "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Mini/resolve/main/benchmarks/component-equivalence.json",
       "sha256": "8bbbd3a362d84a6e8f030803259db69e932e5997b1c7af1a276a0d17862eb184",
       "evidence_type": "exact_component_identity_report",
       "native_artifact_sha256": "d3a742fbe0d833b0a72e5104e0381209b7d3da78f948d3439a5da7d986232731"
+    },
+    "nano_component_equivalence": {
+      "url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/component-equivalence.json",
+      "evidence_type": "Vela_release_report",
+      "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+      "sha256": "ebd40f9a887791d1736132c37f510a69e18a03e247422853f54a42b417b0db9b"
     }
   },
   "models": {
     "nano": {
       "repo_id": "llm-semantic-router/Vela-1.0-Omni-Nano",
-      "release_commit": "d8ac5b5ac2274a501fc61aeb5be70cec1855a806",
-      "parameters_exact": 135383808,
-      "current_native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
+      "release_commit": null,
+      "parameters_exact": 163771288,
+      "current_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
       "task_evidence": {
         "English41": {
           "source_id": "nano_English41",
@@ -72,47 +70,58 @@
             "model": "local/Vela-Omni-Nano-GIST-Rotation",
             "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
             "native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+            "revision_semantics": "local native artifact fingerprint, not a Hugging Face commit",
             "component_applicability": {
               "component": "text",
-              "applies_to_revision": "self",
-              "applies_to_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
               "evaluated_model": "local/Vela-Omni-Nano-GIST-Rotation",
-              "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
               "evaluated_native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
-              "revision_kind": "local native artifact fingerprint, not a Hugging Face commit",
-              "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/component-equivalence.json",
+              "applies_to_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+              "prior_component_applicability": {
+                "model": "local/Vela-Omni-Nano-GIST-Rotation",
+                "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+                "native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+                "component_applicability": {
+                  "component": "text",
+                  "applies_to_revision": "self",
+                  "applies_to_native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
+                  "evaluated_model": "local/Vela-Omni-Nano-GIST-Rotation",
+                  "evaluated_revision": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+                  "evaluated_native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646",
+                  "revision_kind": "local native artifact fingerprint, not a Hugging Face commit",
+                  "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/component-equivalence.json",
+                  "new_full_panel_inference": false,
+                  "basis": "199 exact text tensors and byte-identical inference/tokenizer/configuration under the same official evaluator protocol."
+                },
+                "revision_semantics": "local native artifact fingerprint, not a Hugging Face commit",
+                "inference_profile_applicability": {
+                  "revision": "self",
+                  "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
+                  "parameters": 135383808,
+                  "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/inference-equivalence.json",
+                  "new_benchmark_run": false
+                }
+              },
+              "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/component-equivalence.json",
               "new_full_panel_inference": false,
-              "basis": "199 exact text tensors and byte-identical inference/tokenizer/configuration under the same official evaluator protocol."
-            },
-            "revision_semantics": "local native artifact fingerprint, not a Hugging Face commit",
-            "inference_profile_applicability": {
-              "revision": "self",
-              "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-              "parameters": 135383808,
-              "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/inference-equivalence.json",
-              "new_benchmark_run": false
+              "basis": "The frozen text tensors, tokenizer, configuration and encoding computation are unchanged. The original measured text identity is retained."
             }
           },
-          "self_reference_interpretation": "d8ac5b5ac2274a501fc61aeb5be70cec1855a806"
+          "self_reference_interpretation": "Nano native fingerprint 474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2"
         },
         "audio19": {
           "source_id": "nano_audio19",
           "identity": {
             "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-            "evaluated_revision": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-            "native_artifact_sha256": "96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4",
-            "revision_semantics": "Native artifact fingerprint identifying the evaluated weights and inference files; the release commit is the repository revision containing this report.",
-            "inference_profile_applicability": {
-              "revision": "self",
-              "native_artifact_sha256": "50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa",
-              "parameters": 135383808,
-              "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/d8ac5b5ac2274a501fc61aeb5be70cec1855a806/benchmarks/inference-equivalence.json",
-              "new_benchmark_run": false
-            }
+            "evaluated_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+            "release_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+            "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/maeb-audio-only.json",
+            "new_full_panel_inference": true
           },
-          "self_reference_interpretation": "d8ac5b5ac2274a501fc61aeb5be70cec1855a806"
+          "self_reference_interpretation": "Nano native fingerprint 474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2"
         }
-      }
+      },
+      "release_reference": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano",
+      "release_revision_semantics": "Exact native fingerprint below; Hub commit is not encoded in this file."
     },
     "mini": {
       "repo_id": "llm-semantic-router/Vela-1.0-Omni-Mini",
@@ -1735,7 +1744,7 @@
             }
           ],
           "kind": "peer",
-          "on_displayed_frontier": false
+          "on_displayed_frontier": true
         },
         {
           "model": "Omartificial-Intelligence-Space/Marbert-all-nli-triplet-Matryoshka",
@@ -4103,14 +4112,18 @@
         {
           "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
           "kind": "nano",
-          "parameters_billion": 0.135383808,
-          "parameters_exact": 135383808,
+          "parameters_billion": 0.163771288,
+          "parameters_exact": 163771288,
           "score": 91.94760000000001,
-          "evidence_type": "measured_with_explicit_applicability",
+          "evidence_type": "exact_text_component_applicability",
           "identity_ref": "models.nano.task_evidence.English41",
           "size_basis": "current_whole_model_exact_tensor_count",
           "open_weights": true,
-          "on_displayed_frontier": true
+          "on_displayed_frontier": true,
+          "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+          "revision": null,
+          "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/mteb-eng-v2.json",
+          "evaluated_native_artifact_sha256": "29994142677559ca657c27b657eda0dddbf583bd4cf42a3a31095ab3f99d1646"
         },
         {
           "model": "llm-semantic-router/Vela-1.0-Omni-Mini",
@@ -4135,6 +4148,7 @@
         "Mihaiii/Squirtle",
         "MongoDB/mdbr-leaf-mt",
         "BAAI/bge-small-en",
+        "codefuse-ai/F2LLM-v2-160M",
         "llm-semantic-router/Vela-1.0-Omni-Nano",
         "jinaai/jina-embeddings-v5-text-nano",
         "jxm/cde-small-v2",
@@ -4373,7 +4387,7 @@
         }
       ],
       "unknown_size_observation_count": 28,
-      "highlight_gap_above_best_equal_or_smaller_peer_pp": 0.43240000000001544,
+      "highlight_gap_above_best_equal_or_smaller_peer_pp": 0.23280000000001166,
       "highlight_dominators": []
     },
     "mridingham": {
@@ -5770,14 +5784,20 @@
         {
           "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
           "kind": "nano",
-          "parameters_billion": 0.135383808,
-          "parameters_exact": 135383808,
-          "score": 32.363208758254515,
-          "evidence_type": "measured_with_explicit_applicability",
+          "parameters_billion": 0.163771288,
+          "parameters_exact": 163771288,
+          "score": 58.41978617863634,
+          "evidence_type": "measured_release_evaluation",
           "identity_ref": "models.nano.task_evidence.audio19",
           "size_basis": "current_whole_model_exact_tensor_count",
           "open_weights": true,
-          "on_displayed_frontier": false
+          "on_displayed_frontier": true,
+          "native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+          "revision": null,
+          "report": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/resolve/main/benchmarks/maeb-audio-only.json",
+          "evaluated_native_artifact_sha256": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+          "evaluated_revision": "474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2",
+          "evaluated_model": "llm-semantic-router/Vela-1.0-Omni-Nano"
         },
         {
           "model": "llm-semantic-router/Vela-1.0-Omni-Mini",
@@ -5800,12 +5820,13 @@
         "google/yamnet",
         "matthewagi/HeAR-s1.1",
         "MIT/ast-finetuned-audioset-10-10-0.4593",
+        "llm-semantic-router/Vela-1.0-Omni-Nano",
         "llm-semantic-router/Vela-1.0-Omni-Mini"
       ],
       "reported_peer_observations": 65,
       "unknown_size_observations_excluded": [],
       "unknown_size_observation_count": 0,
-      "highlight_gap_above_best_equal_or_smaller_peer_pp": 13.974058526666042,
+      "highlight_gap_above_best_equal_or_smaller_peer_pp": 9.717472348029702,
       "highlight_dominators": []
     }
   },

benchmarks/pareto-points.csv

@@ -81,7 +81,7 @@ ArXivHierarchicalClusteringP2P,v_measure,nomic-ai/nomic-embed-text-v1,peer,0.137
 ArXivHierarchicalClusteringP2P,v_measure,nomic-ai/nomic-embed-text-v1-ablated,peer,0.137,57.903400000000005,True,False,registry_reported,registry_reported:nomic-ai/nomic-embed-text-v1-ablated,,
 ArXivHierarchicalClusteringP2P,v_measure,nomic-ai/nomic-embed-text-v1-unsupervised,peer,0.137,56.7453,True,False,registry_reported,registry_reported:nomic-ai/nomic-embed-text-v1-unsupervised,,
 ArXivHierarchicalClusteringP2P,v_measure,nomic-ai/nomic-embed-text-v1.5,peer,0.137,60.08051520875409,True,False,registry_reported,registry_reported:nomic-ai/nomic-embed-text-v1.5,,
-ArXivHierarchicalClusteringP2P,v_measure,llm-semantic-router/Vela-1.0-Omni-Nano,nano,0.135383808,64.88847259326147,True,False,exact_public_inference_applicability,vela_release_evaluation:llm-semantic-router/Vela-1.0-Omni-Nano,50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa,50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa
+ArXivHierarchicalClusteringP2P,v_measure,llm-semantic-router/Vela-1.0-Omni-Nano,nano,0.163771288,64.88847259326147,True,False,exact_text_component_applicability,vela_release_evaluation:llm-semantic-router/Vela-1.0-Omni-Nano,,474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2
 ArXivHierarchicalClusteringP2P,v_measure,ibm-granite/granite-embedding-english-r2,peer,0.149,59.0566,True,False,registry_reported,registry_reported:ibm-granite/granite-embedding-english-r2,,
 ArXivHierarchicalClusteringP2P,v_measure,Tarka-AIR/Tarka-Embedding-150M-V1,peer,0.156,63.397099999999995,True,False,registry_reported,registry_reported:Tarka-AIR/Tarka-Embedding-150M-V1,,
 ArXivHierarchicalClusteringP2P,v_measure,codefuse-ai/F2LLM-v2-160M,peer,0.159,62.4399,True,False,registry_reported,registry_reported:codefuse-ai/F2LLM-v2-160M,,
@@ -234,7 +234,7 @@ NMSQAPairClassification,max_ap,microsoft/wavlm-base-plus-sv,peer,0.095,48.4168,T
 NMSQAPairClassification,max_ap,microsoft/wavlm-base-sd,peer,0.095,52.9964,True,False,registry_reported,registry_reported:microsoft/wavlm-base-sd,,
 NMSQAPairClassification,max_ap,microsoft/wavlm-base-sv,peer,0.095,52.9964,True,False,registry_reported,registry_reported:microsoft/wavlm-base-sv,,
 NMSQAPairClassification,max_ap,asapp/sew-d-mid-400k-ft-ls100h,peer,0.139,48.4208,True,False,registry_reported,registry_reported:asapp/sew-d-mid-400k-ft-ls100h,,
-NMSQAPairClassification,max_ap,llm-semantic-router/Vela-1.0-Omni-Nano,nano,0.135383808,63.28340215485784,True,True,exact_public_inference_applicability,,50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa,50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa
+NMSQAPairClassification,max_ap,llm-semantic-router/Vela-1.0-Omni-Nano,nano,0.163771288,62.89933955933497,True,True,measured_release_evaluation,,,474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2
 NMSQAPairClassification,max_ap,laion/clap-htsat-unfused,peer,0.153,46.9791,True,False,registry_reported,registry_reported:laion/clap-htsat-unfused,,
 NMSQAPairClassification,max_ap,laion/clap-htsat-fused,peer,0.154,47.8507,True,False,registry_reported,registry_reported:laion/clap-htsat-fused,,
 NMSQAPairClassification,max_ap,microsoft/msclap-2023,peer,0.16,51.0934,True,False,registry_reported,registry_reported:microsoft/msclap-2023,,
@@ -302,7 +302,7 @@ SIBFLEURS,accuracy,microsoft/wavlm-base-plus-sv,peer,0.095,11.857322549019608,Tr
 SIBFLEURS,accuracy,microsoft/wavlm-base-sd,peer,0.095,13.31318431372549,True,False,registry_reported,registry_reported:microsoft/wavlm-base-sd,,
 SIBFLEURS,accuracy,microsoft/wavlm-base-sv,peer,0.095,13.31318431372549,True,False,registry_reported,registry_reported:microsoft/wavlm-base-sv,,
 SIBFLEURS,accuracy,asapp/sew-d-mid-400k-ft-ls100h,peer,0.139,12.634273529411763,True,False,registry_reported,registry_reported:asapp/sew-d-mid-400k-ft-ls100h,,
-SIBFLEURS,accuracy,llm-semantic-router/Vela-1.0-Omni-Nano,nano,0.135383808,18.304657831512046,True,False,exact_public_inference_applicability,,50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa,50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa
+SIBFLEURS,accuracy,llm-semantic-router/Vela-1.0-Omni-Nano,nano,0.163771288,18.072541269472218,True,False,measured_release_evaluation,,,474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2
 SIBFLEURS,accuracy,laion/clap-htsat-unfused,peer,0.153,8.674730392156862,True,False,registry_reported,registry_reported:laion/clap-htsat-unfused,,
 SIBFLEURS,accuracy,laion/clap-htsat-fused,peer,0.154,9.069597058823529,True,False,registry_reported,registry_reported:laion/clap-htsat-fused,,
 SIBFLEURS,accuracy,microsoft/msclap-2023,peer,0.16,14.073474509803921,True,False,registry_reported,registry_reported:microsoft/msclap-2023,,
@@ -372,7 +372,7 @@ VehicleSoundClustering,v_measure,microsoft/wavlm-base-plus-sv,peer,0.095,1.7932,
 VehicleSoundClustering,v_measure,microsoft/wavlm-base-sd,peer,0.095,0.40940000000000004,True,False,registry_reported,registry_reported:microsoft/wavlm-base-sd,,
 VehicleSoundClustering,v_measure,microsoft/wavlm-base-sv,peer,0.095,0.40940000000000004,True,False,registry_reported,registry_reported:microsoft/wavlm-base-sv,,
 VehicleSoundClustering,v_measure,asapp/sew-d-mid-400k-ft-ls100h,peer,0.139,0.6276,True,False,registry_reported,registry_reported:asapp/sew-d-mid-400k-ft-ls100h,,
-VehicleSoundClustering,v_measure,llm-semantic-router/Vela-1.0-Omni-Nano,nano,0.135383808,13.800701178563807,True,False,exact_public_inference_applicability,,50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa,50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa
+VehicleSoundClustering,v_measure,llm-semantic-router/Vela-1.0-Omni-Nano,nano,0.163771288,5.275425308186281,True,False,measured_release_evaluation,,,474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2
 VehicleSoundClustering,v_measure,laion/clap-htsat-unfused,peer,0.153,2.6642,True,False,registry_reported,registry_reported:laion/clap-htsat-unfused,,
 VehicleSoundClustering,v_measure,laion/clap-htsat-fused,peer,0.154,4.7132,True,False,registry_reported,registry_reported:laion/clap-htsat-fused,,
 VehicleSoundClustering,v_measure,microsoft/msclap-2023,peer,0.16,2.86,True,False,registry_reported,registry_reported:microsoft/msclap-2023,,

benchmarks/release-history.md

@@ -1,94 +1,74 @@
-# Release-to-release history
+# Release progress and trade-offs
 
-These are historical Vela-to-Vela comparisons, not the original small baseline used by the current scorecard. All values and comparators remain as originally measured.
+This update compares native `474ad3d52f6ee7f818a3dbb6112e3d66c2423542d08228e7996a8cd600bd99a2` against the previous Nano `50ce808197fb43a1b8913a66d4a47339ec66ecd10d52cbd9f9e752d02f8c43fa` (measured as `96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4`). The primary product scorecard continues to compare against the original small.
 
-## Historical identities
+Fixed product-weighted change: **+0.726802 pp**. Parameters increase **20.9682%**. This custom weighted result is not an official benchmark aggregate, statistical significance claim or full-size SOTA result.
 
-This report retains measurements from Nano artifact `96d76f97478e114b816df3ab1b3c6947e4c5ef0babfb40ab865d7e3c838d34c4` (144,554,112 parameters). The previous Nano is [3a1efc48](https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/tree/3a1efc48995e2b71622d65c6cf71cfc00bbf3aa3) (`fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf`). Scores are shown on a 0–100 scale. Bold means an unrounded improvement over the explicitly named comparator.
-
-
-## Every English change
-
-| Task | Main metric | Previous Vela | Measured Vela | Change (pp) |
+| Component | Previous | Current | Delta pp | Weight |
 | --- | ---: | ---: | ---: | ---: |
-| ArguAna | ndcg_at_10 | 37.5140 | 59.2460 | +21.7320 |
-| ArXivHierarchicalClusteringP2P | v_measure | 65.2540 | 64.8885 | -0.3655 |
-| ArXivHierarchicalClusteringS2S | v_measure | 55.9895 | 57.4432 | +1.4536 |
-| AskUbuntuDupQuestions | map_at_1000 | 60.5140 | 62.3290 | +1.8150 |
-| BIOSSES | cosine_spearman | 62.2454 | 86.9875 | +24.7421 |
-| Banking77Classification | accuracy | 69.5747 | 82.1429 | +12.5682 |
-| BiorxivClusteringP2P.v2 | v_measure | 39.8292 | 41.3138 | +1.4846 |
-| CQADupstackGamingRetrieval | ndcg_at_10 | 42.9760 | 56.9790 | +14.0030 |
-| CQADupstackUnixRetrieval | ndcg_at_10 | 30.3620 | 39.5860 | +9.2240 |
-| ClimateFEVERHardNegatives | ndcg_at_10 | 16.0740 | 31.8460 | +15.7720 |
-| FEVERHardNegatives | ndcg_at_10 | 24.0550 | 87.5760 | +63.5210 |
-| FiQA2018 | ndcg_at_10 | 22.5880 | 39.1430 | +16.5550 |
-| HotpotQAHardNegatives | ndcg_at_10 | 26.9870 | 66.3490 | +39.3620 |
-| ImdbClassification | accuracy | 62.1364 | 91.9476 | +29.8112 |
-| MTOPDomainClassification | accuracy | 86.7191 | 94.9179 | +8.1988 |
-| MassiveIntentClassification | accuracy | 60.3934 | 70.9684 | +10.5750 |
-| MassiveScenarioClassification | accuracy | 73.3726 | 76.1130 | +2.7404 |
-| MedrxivClusteringP2P.v2 | v_measure | 37.7681 | 39.8302 | +2.0621 |
-| MedrxivClusteringS2S.v2 | v_measure | 36.1335 | 37.8751 | +1.7416 |
-| MindSmallReranking | max_over_subqueries_map_at_1000 | 30.6230 | 32.3660 | +1.7430 |
-| SCIDOCS | ndcg_at_10 | 16.8270 | 21.8890 | +5.0620 |
-| SICK-R | cosine_spearman | 71.6962 | 80.5317 | +8.8356 |
-| STS12 | cosine_spearman | 68.1692 | 75.5659 | +7.3967 |
-| STS13 | cosine_spearman | 74.4034 | 86.2635 | +11.8600 |
-| STS14 | cosine_spearman | 68.0265 | 82.2988 | +14.2723 |
-| STS15 | cosine_spearman | 77.8614 | 88.7365 | +10.8750 |
-| STSBenchmark | cosine_spearman | 77.9240 | 87.0782 | +9.1542 |
-| SprintDuplicateQuestions | max_ap | 89.7910 | 95.7996 | +6.0087 |
-| StackExchangeClustering.v2 | v_measure | 57.3446 | 58.4244 | +1.0798 |
-| StackExchangeClusteringP2P.v2 | v_measure | 39.3767 | 41.0530 | +1.6763 |
-| TRECCOVID | ndcg_at_10 | 41.2920 | 69.1210 | +27.8290 |
-| Touche2020Retrieval.v3 | ndcg_at_10 | 31.4770 | 48.3630 | +16.8860 |
-| ToxicConversationsClassification | accuracy | 64.4092 | 71.9043 | +7.4951 |
-| TweetSentimentExtractionClassification | accuracy | 50.7782 | 62.7929 | +12.0147 |
-| TwentyNewsgroupsClustering.v2 | v_measure | 44.8847 | 50.8726 | +5.9879 |
-| TwitterSemEval2015 | max_ap | 56.3448 | 72.9515 | +16.6068 |
-| TwitterURLCorpus | max_ap | 78.7021 | 85.3034 | +6.6013 |
-| SummEvalSummarization.v2 | cosine_spearman | 29.2496 | 31.8344 | +2.5848 |
-| AmazonCounterfactualClassification | accuracy | 56.8209 | 71.8507 | +15.0299 |
-| STS17 | cosine_spearman | 84.1659 | 89.0210 | +4.8551 |
-| STS22.v2 | cosine_spearman | 67.5701 | 68.7533 | +1.1833 |
+| common14 | 67.574073 | 66.418353 | -1.155720 | 0.3 |
+| English41_MeanTaskType | 60.781850 | 60.781850 | +0.000000 | 0.3 |
+| image3 | 53.657464 | 53.657464 | +0.000000 | 0.2 |
+| audio19_MeanTaskType | 46.967814 | 52.335406 | +5.367592 | 0.2 |
 
-## Every audio change
+## Common metrics
 
-| Task | Main metric | Previous Vela | Measured Vela | Change (pp) |
-| --- | ---: | ---: | ---: | ---: |
-| JamAltArtistA2ARetrieval | ndcg_at_10 | 68.9005 | 68.9005 | +0.0000 |
-| BeijingOpera | accuracy | 68.6791 | 68.6791 | +0.0000 |
-| BirdCLEF | accuracy | 12.9000 | 12.9000 | +0.0000 |
-| CREMA_D | accuracy | 29.7637 | 29.7637 | +0.0000 |
-| CommonLanguageAgeDetection | accuracy | 16.0400 | 16.0400 | +0.0000 |
-| GTZANGenre | accuracy | 50.3000 | 50.3000 | +0.0000 |
-| IEMOCAPGender | accuracy | 58.4519 | 58.4519 | +0.0000 |
-| MInDS14 | accuracy | 23.3212 | 23.3212 | +0.0000 |
-| MridinghamTonic | accuracy | 32.5066 | 32.3632 | -0.1434 |
-| SIBFLEURS | accuracy | 18.3047 | 18.3047 | +0.0000 |
-| VoxCelebSA | accuracy | 33.1691 | 33.1691 | +0.0000 |
-| VoxPopuliLanguageID | accuracy | 91.8000 | 91.8000 | +0.0000 |
-| CREMA_DClustering | v_measure | 2.3602 | 2.3602 | +0.0000 |
-| VehicleSoundClustering | v_measure | 13.8007 | 13.8007 | +0.0000 |
-| VoxPopuliGenderClustering | v_measure | 0.1045 | 0.1045 | +0.0000 |
-| CREMADPairClassification | max_ap | 54.0802 | 54.2012 | +0.1210 |
-| NMSQAPairClassification | max_ap | 63.4355 | 63.2834 | -0.1521 |
-| VoxPopuliAccentPairClassification | max_ap | 54.0504 | 54.0220 | -0.0284 |
-| GTZANAudioReranking | map_at_1000 | 63.7940 | 63.7940 | +0.0000 |
+| Metric | Delta pp |
+| --- | ---: |
+| audio.audio_to_text.recall@1 | -1.034087 |
+| audio.audio_to_text.recall@10 | -1.378782 |
+| audio.audio_to_text.recall@5 | -1.838376 |
+| audio.text_to_audio.recall@1 | -5.095785 |
+| audio.text_to_audio.recall@10 | -5.287356 |
+| audio.text_to_audio.recall@5 | -6.168582 |
+| image.image_to_text.recall@1 | +0.000000 |
+| image.image_to_text.recall@10 | +0.000000 |
+| image.image_to_text.recall@5 | +0.000000 |
+| image.text_to_image.recall@1 | +0.000000 |
+| image.text_to_image.recall@10 | +0.000000 |
+| image.text_to_image.recall@5 | +0.000000 |
+| text.banking77.accuracy | +0.000000 |
+| text.massive-en.accuracy | +0.000000 |
 
+## Every audio task
 
+| Task | Delta pp |
+| --- | ---: |
+| BeijingOpera | +10.549645 |
+| BirdCLEF | +2.300000 |
+| CREMADPairClassification | +1.006025 |
+| CREMA_D | +2.284477 |
+| CREMA_DClustering | +2.854175 |
+| CommonLanguageAgeDetection | +0.615000 |
+| GTZANAudioReranking | +6.682000 |
+| GTZANGenre | +11.400000 |
+| IEMOCAPGender | +3.237876 |
+| JamAltArtistA2ARetrieval | +16.901000 |
+| MInDS14 | -1.469131 |
+| MridinghamTonic | +26.056577 |
+| NMSQAPairClassification | -0.384063 |
+| SIBFLEURS | -0.232117 |
+| VehicleSoundClustering | -8.525276 |
+| VoxCelebSA | -0.000589 |
+| VoxPopuliAccentPairClassification | +0.129363 |
+| VoxPopuliGenderClustering | -0.081273 |
+| VoxPopuliLanguageID | -0.600000 |
 
-| Component | Weight | Previous | Current | Change (pp) |
-| --- | ---: | ---: | ---: | ---: |
-| common14 | 30% | 59.3729 | 67.5741 | +8.2012 |
-| English41_MeanTaskType | 30% | 51.9755 | 60.7818 | +8.8063 |
-| image3 | 20% | 54.8658 | 53.6575 | -1.2083 |
-| audio19_MeanTaskType | 20% | 46.9744 | 46.9678 | -0.0066 |
-
-Weighted product progress is +4.8593 points. This is a product comparison, not an official leaderboard aggregate or overall SOTA claim. The common component equally weights text, image and audio families and uses all 14 metrics. The image component equally averages Pets classification, TinyImageNet NMI and Pets zero-shot accuracy.
-
-Regressions remain visible: all three image-to-text recall ranks decline; Pets zero-shot decreases 3.625 points; SpeechCommands zero-shot decreases 0.859; ArXiv P2P decreases 0.3655. Three audio-panel tasks decline slightly (MridinghamTonic, NMSQA and VoxPopuliAccent); all other changes are listed above. Classification/clustering preservation does not imply preservation of cross-modal or short-label zero-shot behavior.
+## Selected standard protocols
 
+| Task / split | Delta pp |
+| --- | ---: |
+| SICK-R / test | +0.000000 |
+| STSBenchmark / test | +0.000000 |
+| ArguAna / test | +0.000000 |
+| TwentyNewsgroupsClustering.v2 / test | +0.000000 |
+| MTOPDomainClassification / validation | +0.000000 |
+| MTOPDomainClassification / test | +0.000000 |
+| OxfordPets / test | +0.000000 |
+| TinyImageNetClustering / valid | +0.000000 |
+| OxfordPetsZeroShot / test | +0.000000 |
+| CREMA_D / train | +2.284477 |
+| CREMA_DClustering / train | +2.854175 |
+| SpeechCommandsZeroshotv0.02 / test | +1.595484 |
 
-The exact original common metrics, full paired intervals, and numerical history remain in [scores.json](../scores.json) and [net-progress.json](net-progress.json). This file is historical evidence, not an additional product comparator.
+The 840-clip event diagnostic is reported separately and does not enter the weighted value. English and image computations are unchanged; their original measurement identities are preserved. Prior results remain in [previous-release-scores.json](previous-release-scores.json).

components/audio_clap/config.json

@@ -0,0 +1,206 @@
+{
+  "_commit_hash": null,
+  "architectures": [
+    "ClapModel"
+  ],
+  "audio_config": {
+    "_name_or_path": "",
+    "add_cross_attention": false,
+    "aff_block_r": 4,
+    "architectures": null,
+    "attention_probs_dropout_prob": 0.0,
+    "bad_words_ids": null,
+    "begin_suppress_tokens": null,
+    "bos_token_id": null,
+    "chunk_size_feed_forward": 0,
+    "cross_attention_hidden_size": null,
+    "decoder_start_token_id": null,
+    "depths": [
+      2,
+      2,
+      6,
+      2
+    ],
+    "diversity_penalty": 0.0,
+    "do_sample": false,
+    "drop_path_rate": 0.0,
+    "early_stopping": false,
+    "enable_fusion": false,
+    "enable_patch_layer_norm": true,
+    "encoder_no_repeat_ngram_size": 0,
+    "eos_token_id": null,
+    "exponential_decay_length_penalty": null,
+    "finetuning_task": null,
+    "flatten_patch_embeds": true,
+    "forced_bos_token_id": null,
+    "forced_eos_token_id": null,
+    "fusion_num_hidden_layers": 2,
+    "fusion_type": null,
+    "hidden_act": "gelu",
+    "hidden_dropout_prob": 0.1,
+    "hidden_size": 768,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "initializer_factor": 1.0,
+    "is_decoder": false,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "layer_norm_eps": 1e-05,
+    "length_penalty": 1.0,
+    "max_length": 20,
+    "min_length": 0,
+    "mlp_ratio": 4.0,
+    "model_type": "clap_audio_model",
+    "no_repeat_ngram_size": 0,
+    "num_attention_heads": [
+      4,
+      8,
+      16,
+      32
+    ],
+    "num_beam_groups": 1,
+    "num_beams": 1,
+    "num_classes": 527,
+    "num_hidden_layers": 4,
+    "num_mel_bins": 64,
+    "num_return_sequences": 1,
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "output_scores": false,
+    "pad_token_id": null,
+    "patch_embed_input_channels": 1,
+    "patch_embeds_hidden_size": 96,
+    "patch_size": 4,
+    "patch_stride": [
+      4,
+      4
+    ],
+    "prefix": null,
+    "problem_type": null,
+    "projection_dim": 512,
+    "projection_hidden_act": "relu",
+    "projection_hidden_size": 768,
+    "pruned_heads": {},
+    "qkv_bias": true,
+    "remove_invalid_values": false,
+    "repetition_penalty": 1.0,
+    "return_dict": true,
+    "return_dict_in_generate": false,
+    "sep_token_id": null,
+    "spec_size": 256,
+    "suppress_tokens": null,
+    "task_specific_params": null,
+    "temperature": 1.0,
+    "tf_legacy_loss": false,
+    "tie_encoder_decoder": false,
+    "tie_word_embeddings": true,
+    "tokenizer_class": null,
+    "top_k": 50,
+    "top_p": 1.0,
+    "torch_dtype": null,
+    "torchscript": false,
+    "transformers_version": "4.27.0.dev0",
+    "typical_p": 1.0,
+    "use_bfloat16": false,
+    "window_size": 8
+  },
+  "hidden_size": 768,
+  "initializer_factor": 1.0,
+  "logit_scale_init_value": 14.285714285714285,
+  "model_type": "clap",
+  "num_hidden_layers": 16,
+  "projection_dim": 512,
+  "projection_hidden_act": "relu",
+  "text_config": {
+    "_name_or_path": "",
+    "add_cross_attention": false,
+    "architectures": null,
+    "attention_probs_dropout_prob": 0.1,
+    "bad_words_ids": null,
+    "begin_suppress_tokens": null,
+    "bos_token_id": 0,
+    "chunk_size_feed_forward": 0,
+    "classifier_dropout": null,
+    "cross_attention_hidden_size": null,
+    "decoder_start_token_id": null,
+    "diversity_penalty": 0.0,
+    "do_sample": false,
+    "early_stopping": false,
+    "encoder_no_repeat_ngram_size": 0,
+    "eos_token_id": 2,
+    "exponential_decay_length_penalty": null,
+    "finetuning_task": null,
+    "forced_bos_token_id": null,
+    "forced_eos_token_id": null,
+    "fusion_hidden_size": 768,
+    "fusion_num_hidden_layers": 2,
+    "hidden_act": "gelu",
+    "hidden_dropout_prob": 0.1,
+    "hidden_size": 768,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "initializer_factor": 1.0,
+    "initializer_range": 0.02,
+    "intermediate_size": 3072,
+    "is_decoder": false,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "layer_norm_eps": 1e-12,
+    "length_penalty": 1.0,
+    "max_length": 20,
+    "max_position_embeddings": 514,
+    "min_length": 0,
+    "model_type": "clap_text_model",
+    "no_repeat_ngram_size": 0,
+    "num_attention_heads": 12,
+    "num_beam_groups": 1,
+    "num_beams": 1,
+    "num_hidden_layers": 12,
+    "num_return_sequences": 1,
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "output_scores": false,
+    "pad_token_id": 1,
+    "position_embedding_type": "absolute",
+    "prefix": null,
+    "problem_type": null,
+    "projection_dim": 512,
+    "projection_hidden_act": "relu",
+    "projection_hidden_size": 768,
+    "pruned_heads": {},
+    "remove_invalid_values": false,
+    "repetition_penalty": 1.0,
+    "return_dict": true,
+    "return_dict_in_generate": false,
+    "sep_token_id": null,
+    "suppress_tokens": null,
+    "task_specific_params": null,
+    "temperature": 1.0,
+    "tf_legacy_loss": false,
+    "tie_encoder_decoder": false,
+    "tie_word_embeddings": true,
+    "tokenizer_class": null,
+    "top_k": 50,
+    "top_p": 1.0,
+    "torch_dtype": null,
+    "torchscript": false,
+    "transformers_version": "4.27.0.dev0",
+    "type_vocab_size": 1,
+    "typical_p": 1.0,
+    "use_bfloat16": false,
+    "use_cache": true,
+    "vocab_size": 50265
+  },
+  "torch_dtype": "float32",
+  "transformers_version": null
+}

components/audio_clap/preprocessor_config.json

@@ -0,0 +1,22 @@
+{
+  "chunk_length_s": 10,
+  "feature_extractor_type": "ClapFeatureExtractor",
+  "feature_size": 64,
+  "fft_window_size": 1024,
+  "frequency_max": 14000,
+  "frequency_min": 50,
+  "hop_length": 480,
+  "max_length_s": 10,
+  "n_fft": 1024,
+  "nb_frequency_bins": 513,
+  "nb_max_frames": 1000,
+  "nb_max_samples": 480000,
+  "padding": "repeatpad",
+  "padding_side": "right",
+  "padding_value": 0.0,
+  "processor_class": "ClapProcessor",
+  "return_attention_mask": false,
+  "sampling_rate": 48000,
+  "top_db": null,
+  "truncation": "rand_trunc"
+}

config.json

@@ -2,16 +2,18 @@
   "architectures": [
     "VelaOmni"
   ],
+  "audio_backend": "tiny_clap_residual_v1",
   "audio_max_seconds": 30,
   "audio_pooling": "mean",
   "audio_projection": "linear",
   "audio_sampling_rate": 16000,
   "embedding_dim": 384,
-  "format_version": 5,
+  "format_version": 9,
   "inference_profile": "single_modality",
   "library_name": "pytorch",
   "max_text_length": 512,
-  "parameter_count": 135383808,
+  "parameter_count": 163771288,
+  "residual_audio_readout": "tiny_full_mean_clap_endpoint_zero_linear",
   "text_pooling": "cls",
   "text_projection": "identity",
   "torch_dtype": "float32",

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:401759e85cec305aca7ff08b31a1a3efa2c410b24b7bb49f9a81de3aabafa52d
-size 541598024
+oid sha256:d5aa7f00e217bd4fd5314140a3603b9a873636320c8d09732a14f79f4cb0667e
+size 655583800

omni_components/single_modality.py

@@ -1,5 +1,7 @@
 """Nano inference runtime for the three public modality encoders."""
 from pathlib import Path
+import numpy as np
+import torch
 from torch import nn
 
 from .audio_encoder import AudioEncoder
@@ -11,11 +13,14 @@ from .text_encoder import TextEncoder
 class SingleModalityEmbedder(nn.Module):
     """Full-depth encoders in a shared space, without optional fusion/exit heads."""
 
-    def __init__(self, component_dir, legacy=False, max_text_length=512, text_pooling="cls"):
+    def __init__(self, component_dir, legacy=False, max_text_length=512, text_pooling="cls", audio_backend="whisper"):
         super().__init__()
         if legacy:
             raise ValueError("Inference profile requires native Nano components")
         root = Path(component_dir)
+        if audio_backend not in ("whisper", "tiny_clap_residual_v1"):
+            raise ValueError("Unsupported Nano audio backend")
+        self.audio_backend = audio_backend
         self.output_dim = 384
         self.normalize = True
         self.max_text_length = max_text_length
@@ -34,6 +39,10 @@ class SingleModalityEmbedder(nn.Module):
             enable_layer_outputs=False,
         )
 
+        if audio_backend == "tiny_clap_residual_v1":
+            from .tiny_clap_residual import TinyClapResidual
+            self.audio_residual = TinyClapResidual(root)
+
     def encode_text(self, texts):
         return MultimodalEmbedder.encode_text(self, texts)
 
@@ -41,5 +50,36 @@ class SingleModalityEmbedder(nn.Module):
         return MultimodalEmbedder.encode_image(self, images)
 
     def encode_audio(self, audio, sampling_rate=16000):
+        if self.audio_backend == "tiny_clap_residual_v1":
+            return self._encode_audio_original(audio, sampling_rate)
         return MultimodalEmbedder.encode_audio(self, audio, sampling_rate=sampling_rate)
 
+
+    def _encode_audio_original(self, waveforms, sampling_rate):
+        if self.audio_backend != "tiny_clap_residual_v1":
+            raise ValueError("Original-rate dual audio requires residual backend")
+        from .tiny_clap_residual import validate_original, native_rate
+        waves = [np.array(validate_original(w, sampling_rate), dtype=np.float32, copy=True, order="C") for w in waveforms]
+        if not waves:
+            raise ValueError("Provide at least one original waveform")
+        results = []
+        for start in range(0, len(waves), 8):
+            current = waves[start:start + 8]
+            audio16 = [native_rate(w, sampling_rate, 16000) for w in current]
+            audio48 = [native_rate(w, sampling_rate, 48000) for w in current]
+            results.append(self._encode_audio_prepared(audio16, audio48))
+        return torch.cat(results)
+
+    def _encode_audio_prepared(self, audio16, audio48):
+        """Internal paired native-rate path; benchmark resamples before cropping."""
+        if self.audio_backend != "tiny_clap_residual_v1" or not 1 <= len(audio16) == len(audio48) <= 8:
+            raise ValueError("One through eight paired native-rate streams required")
+        for streams, rate in ((audio16, 16000), (audio48, 48000)):
+            if any(np.asarray(w).ndim != 1 or not 1 <= len(w) <= 30 * rate or not np.isfinite(w).all() for w in streams):
+                raise ValueError("Finite mono native-rate streams of at most30 seconds required")
+        if self.audio_encoder.normalize or not self.normalize or self.audio_encoder.pooling_mode != "mean":
+            raise ValueError("Pinned unnormalized Tiny affine and one embedding normalization required")
+        # Delegate the complete original AudioEncoder call, including processor and full mean.
+        original_affine = self.audio_encoder(audio=audio16, sampling_rate=16000)
+        clap = self.audio_residual.encode_clap(audio48)
+        return self.audio_residual(original_affine, clap)

omni_components/tiny_clap_residual.py

@@ -0,0 +1,82 @@
+"""Pinned CLAP residual; retained Tiny branch and affine remain unchanged."""
+from pathlib import Path
+import numpy as np
+import torch
+from torch import nn
+from torch.nn import functional as F
+from transformers import ClapConfig,ClapFeatureExtractor,ClapAudioModelWithProjection
+
+def unit(x):
+    if not torch.isfinite(x).all() or torch.any(x.norm(dim=-1)<=1e-12):
+        raise ValueError('Finite nonzero embedding required')
+    return F.normalize(x,dim=-1)
+
+
+def validate_original(wave,rate):
+    if isinstance(rate,bool) or not isinstance(rate,(int,np.integer)) or not 1<=rate<=384000:
+        raise ValueError('Positive integer original sampling rate required')
+    x=np.asarray(wave,dtype=np.float32)
+    if x.ndim not in (1,2) or not np.isfinite(x).all() or x.shape[-1]<1:
+        raise ValueError('Finite nonempty mono or channels-first PCM required')
+    if x.ndim==2 and not 1<=x.shape[0]<=8:raise ValueError('Require1..8 channels-first PCM')
+    if x.shape[-1]>30*rate:raise ValueError('Audio inputs must be at most30 seconds')
+    return x
+
+
+def native_rate(wave,source_rate,target_rate):
+    """Same operation order/default sinc kernel as official MTEB2.21 AudioCollator."""
+    if target_rate not in (16000,48000):raise ValueError('Unsupported branch rate')
+    # Each invocation sees the original waveform. Never cascade16k into48k.
+    x=validate_original(wave,source_rate)
+    if source_rate!=target_rate:
+        import torchaudio
+        x=torchaudio.transforms.Resample(orig_freq=source_rate,new_freq=target_rate)(torch.from_numpy(x).float()).numpy()
+    else:x=np.asarray(x)
+    if x.ndim>1 and x.shape[0]>1:x=np.mean(x,axis=0)
+    if x.shape[-1]>target_rate*30:x=x[...,:target_rate*30]
+    if x.ndim==2 and x.shape[0]==1:x=x[0]
+    if x.ndim!=1 or not np.isfinite(x).all() or not 1<=len(x)<=30*target_rate:raise ValueError('Invalid resampled mono PCM')
+    return np.ascontiguousarray(x,dtype=np.float32)
+
+
+def windows(samples):
+    if not 1<=samples<=1440000:raise ValueError('CLAP requires at most30 seconds at48kHz')
+    if samples<=480000:return [(0,samples)]
+    count=(samples+479999)//480000;last=samples-480000
+    return [(i*last//(count-1),i*last//(count-1)+480000) for i in range(count)]
+
+
+
+class TinyClapResidual(nn.Module):
+    def __init__(self, component_root):
+        super().__init__()
+        root=Path(component_root)/'audio_clap'
+        config=ClapConfig.from_pretrained(root,local_files_only=True)
+        self.clap=ClapAudioModelWithProjection(config.audio_config)
+        self.processor=ClapFeatureExtractor.from_pretrained(root,local_files_only=True)
+        self.weight=nn.Parameter(torch.zeros(384,512))
+        self.register_buffer('mean',torch.zeros(512))
+        self.register_buffer('scale',torch.ones(512))
+        if sum(p.numel() for p in self.clap.parameters())!=28190872:
+            raise ValueError('Pinned CLAP audio architecture mismatch')
+        if self.processor.sampling_rate!=48000 or self.processor.truncation!='rand_trunc' or self.processor.padding!='repeatpad':
+            raise ValueError('Pinned CLAP processor mismatch')
+
+    def encode_clap(self, audio48):
+        if not 1<=len(audio48)<=8:raise ValueError('One through eight native-rate streams required')
+        chunks=[];groups=[]
+        for wave in audio48:
+            start=len(chunks);chunks.extend(wave[a:b] for a,b in windows(len(wave)));groups.append((start,len(chunks)))
+        encoded=[];device=self.weight.device
+        for start in range(0,len(chunks),8):
+            inputs=self.processor(chunks[start:start+8],sampling_rate=48000,return_tensors='pt')
+            encoded.append(unit(self.clap(**{k:v.to(device) for k,v in inputs.items()}).audio_embeds.float()))
+        encoded=torch.cat(encoded)
+        return torch.stack([encoded[a] if b-a==1 else unit(encoded[a:b].mean(0)) for a,b in groups])
+
+    def forward(self, original_affine, clap):
+        if original_affine.shape!=(len(clap),384) or clap.shape[1:]!=(512,):
+            raise ValueError('Paired original affine384 and CLAP512 required')
+        if not torch.isfinite(self.mean).all() or not torch.isfinite(self.scale).all() or torch.any(self.scale<=0):
+            raise ValueError('Invalid frozen TRAIN statistics')
+        return unit(original_affine+F.linear((clap-self.mean)/self.scale,self.weight))

vela_omni.py

@@ -15,9 +15,14 @@ from omni_components.mini import MultiModalSentenceEmbedder
 
 
 class VelaOmni(nn.Module):
-    def __init__(self, root, variant, legacy=False, audio_projection="identity", text_projection="identity", audio_pooling="mean", max_text_length=None, text_pooling="mean", inference_profile="full"):
+    def __init__(self, root, variant, legacy=False, audio_projection="identity", text_projection="identity", audio_pooling="mean", max_text_length=None, text_pooling="mean", inference_profile="full", audio_backend="whisper"):
         super().__init__()
         self.variant = variant
+        if audio_backend not in ("whisper", "tiny_clap_residual_v1"):
+            raise ValueError("Unsupported audio backend")
+        if audio_backend != "whisper" and (variant != "nano" or inference_profile != "single_modality" or audio_projection != "linear" or audio_pooling != "mean"):
+            raise ValueError("Residual audio requires Nano single-modality mean/linear")
+        self.audio_backend = audio_backend
         if inference_profile not in ("full", "single_modality") or (inference_profile == "single_modality" and variant != "nano"):
             raise ValueError("Unsupported inference profile")
         self.inference_profile = inference_profile
@@ -40,8 +45,9 @@ class VelaOmni(nn.Module):
         self.max_text_length = max_text_length
         if variant == "nano":
             encoder_type = SingleModalityEmbedder if inference_profile == "single_modality" else MultimodalEmbedder
+            extra = {"audio_backend": audio_backend} if inference_profile == "single_modality" else {}
             self.model = encoder_type(Path(root) / "components", legacy=legacy,
-                                            max_text_length=self.max_text_length, text_pooling=self.text_pooling)
+                                            max_text_length=self.max_text_length, text_pooling=self.text_pooling, **extra)
             self.tokenizer = self.model.text_encoder.tokenizer
             if text_projection == "residual_rank64":
                 self.model.text_encoder.projection = ResidualTextProjection(384)
@@ -59,6 +65,29 @@ class VelaOmni(nn.Module):
         else:
             raise ValueError("Unknown Vela Omni variant")
 
+    @staticmethod
+    def validate_config(config):
+        version = config.get("format_version")
+        if version not in (1, 2, 3, 4, 5, 9):
+            raise ValueError("Unsupported Vela Omni artifact format")
+        backend = config.get("audio_backend", "whisper")
+        if version == 9:
+            required = {"variant": "nano", "inference_profile": "single_modality",
+                        "audio_backend": "tiny_clap_residual_v1", "audio_projection": "linear",
+                        "audio_pooling": "mean", "text_pooling": "cls", "max_text_length": 512,
+                        "text_projection": "identity", "parameter_count": 163771288,
+                        "residual_audio_readout": "tiny_full_mean_clap_endpoint_zero_linear"}
+            if any(config.get(k) != v for k, v in required.items()):
+                raise ValueError("Format 9 requires the pinned Nano residual configuration")
+        elif backend != "whisper":
+            raise ValueError("Residual audio semantics require format 9")
+        if version in (5, 9) and (config.get("variant") != "nano" or config.get("inference_profile") != "single_modality"):
+            raise ValueError("Format 5/9 requires Nano single-modality profile")
+        if version not in (5, 9) and config.get("inference_profile", "full") != "full":
+            raise ValueError("Inference profile requires format 5/9")
+        if config.get("image_pooling", "mean") != "mean":
+            raise ValueError("Unsupported image pooling")
+
     @classmethod
     def from_pretrained(cls, repo_id, revision=None, device="cpu", dtype=torch.float32):
         root = Path(repo_id)
@@ -67,20 +96,14 @@ class VelaOmni(nn.Module):
             root = Path(snapshot_download(repo_id, revision=revision,
                     allow_patterns=["config.json", "model.safetensors", "components/*"]))
         config = json.loads((root / "config.json").read_text())
-        if config.get("format_version") not in (1, 2, 3, 4, 5):
-            raise ValueError("Unsupported Vela Omni artifact format")
-        if config.get("format_version") == 5 and (config.get("variant") != "nano" or config.get("inference_profile") != "single_modality"):
-            raise ValueError("Format 5 requires Nano single-modality profile")
-        if config.get("format_version") != 5 and config.get("inference_profile", "full") != "full":
-            raise ValueError("Inference profile requires format 5")
-        if config.get("image_pooling", "mean") != "mean":
-            raise ValueError("Unsupported image pooling")
+        cls.validate_config(config)
         instance = cls(root, config["variant"], audio_projection=config.get("audio_projection", "identity"),
                        text_projection=config.get("text_projection", "identity"),
                        audio_pooling=config.get("audio_pooling", "mean"),
                        max_text_length=config.get("max_text_length"),
                        text_pooling=config.get("text_pooling", "mean"),
-                       inference_profile=config.get("inference_profile", "full"))
+                       inference_profile=config.get("inference_profile", "full"),
+                       audio_backend=config.get("audio_backend", "whisper"))
         weights = load_file(str(root / "model.safetensors"), device="cpu")
         result = instance.model.load_state_dict(weights, strict=True, assign=True)
         if result.missing_keys or result.unexpected_keys:
@@ -122,6 +145,11 @@ class VelaOmni(nn.Module):
 
     @torch.inference_mode()
     def encode_audio(self, waveforms, sampling_rate=16000):
+        if self.audio_backend == "tiny_clap_residual_v1":
+            if isinstance(sampling_rate, bool) or sampling_rate not in (16000, 44100, 48000):
+                raise ValueError("Supported original sampling rates are16000,44100,48000")
+            values = self.model._encode_audio_original(list(waveforms), sampling_rate)
+            return F.normalize(values.float(), dim=-1)
         if sampling_rate != 16000:
             raise ValueError("Resample audio to 16000 Hz before encoding")
         waveforms = [np.asarray(w, dtype=np.float32) for w in waveforms]