Improve image classification, clustering and cross-modal retrieval; update scores and figures

.gitattributes

@@ -36,3 +36,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 assets/pareto-arxiv.png filter=lfs diff=lfs merge=lfs -text
 assets/pareto-nmsqa.png filter=lfs diff=lfs merge=lfs -text
 assets/pareto-sibfleurs.png filter=lfs diff=lfs merge=lfs -text
+assets/pareto-vehiclesoundclustering.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -40,12 +40,17 @@ Text, images and speech in one embedding space. Vela Omni Nano supports multimod
 
 ## Evaluation
 
-Complete benchmark panels, evaluated with MTEB 2.21.0. Scores are on a 0–100 scale; higher is better.
+Evaluated with MTEB 2.21.0. Scores use a 0–100 scale; higher is better. The full English and audio rows use mean task score; the three image rows are individual tasks. **Bold marks gains over the [previous Nano release](https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/70c3412bb0ddfb19386d673d4a102c753a574f1b/README.md).**
 
-| Benchmark | Coverage | Mean task score |
-| --- | --- | ---: |
-| MTEB English v2 | All 41 tasks | 53.37 |
-| MAEB (beta), audio-only | All 19 tasks | 39.78 |
+| Benchmark / metric | Previous Nano | Current Nano |
+| --- | ---: | ---: |
+| MTEB English v2 · all 41 tasks | 53.37 | 53.37 |
+| MAEB audio-only · all 19 tasks | 39.78 | 39.78 |
+| Oxford Pets · classification accuracy | 44.47 | **91.27** |
+| Tiny ImageNet · clustering NMI | 46.03 | **61.69** |
+| Oxford Pets · zero-shot accuracy | 15.26 | 11.64 |
+
+The update improves image classification, clustering and cross-modal retrieval at the same model size. Oxford Pets zero-shot accuracy decreases by 3.62 points; the text and audio components are unchanged.
 
 The [detailed evaluation](benchmarks/EVALUATION.md) contains task-type breakdowns, the selected 11-task text/image/audio panel, and routing and retrieval comparisons against the corresponding earlier model. **Gains over the named comparator are bold.** It also provides all regressions, sample counts and protocols.
 
@@ -61,7 +66,7 @@ Nano reaches **65.25 V-measure** on ArXiv hierarchical clustering and **63.44 ma
 
 The dashed line marks the Pareto frontier among the displayed models: no displayed model is both smaller and at least as accurate, or at least as small and more accurate. The horizontal axis uses **total model parameters on a logarithmic scale**, including all three modalities for Vela.
 
-Peer points are reported leaderboard results from the **September 17, 2026** snapshot; prompts, revisions and evaluation settings can differ. These are task-specific comparisons, not matched reruns or a claim of overall benchmark leadership. [Methodology and every plotted point](benchmarks/pareto-methodology.md).
+Peer points are reported leaderboard results from the **September 17, 2026** snapshot; prompts, revisions and evaluation settings can differ. These are task-specific comparisons, not matched reruns or a claim of overall benchmark leadership. [All four size–quality figures](benchmarks/pareto-gallery.md) · [Methodology and every plotted point](benchmarks/pareto-methodology.md).
 
 ## Usage
 

benchmarks/EVALUATION.md

@@ -12,12 +12,12 @@ Scores use a 0–100 scale; higher is better. Both models use the same examples
 | Banking77 · Macro-F1 | 70.21 | **76.58** |
 | MASSIVE English · Accuracy | 65.95 | **69.35** |
 | MASSIVE English · Macro-F1 | 61.72 | **65.24** |
-| COCO · Image → text · R@1 | 40.83 | **47.27** |
-| COCO · Image → text · R@5 | 67.19 | **74.85** |
-| COCO · Image → text · R@10 | 78.49 | **85.91** |
-| COCO · Text → image · R@1 | 30.18 | **37.86** |
-| COCO · Text → image · R@5 | 59.42 | **70.43** |
-| COCO · Text → image · R@10 | 74.29 | **83.11** |
+| COCO · Image → text · R@1 | 40.83 | **65.61** |
+| COCO · Image → text · R@5 | 67.19 | **88.82** |
+| COCO · Image → text · R@10 | 78.49 | **96.11** |
+| COCO · Text → image · R@1 | 30.18 | **53.07** |
+| COCO · Text → image · R@5 | 59.42 | **84.20** |
+| COCO · Text → image · R@10 | 74.29 | **93.49** |
 | LibriSpeech · Audio → text · R@1 | 4.21 | **9.00** |
 | LibriSpeech · Audio → text · R@5 | 11.99 | **22.29** |
 | LibriSpeech · Audio → text · R@10 | 19.03 | **31.48** |
@@ -31,6 +31,23 @@ Image retrieval uses the COCO Karpathy CC-BY2 image subset: 823 images and 4,115
 
 Evaluation uses FP32 embeddings, cosine similarity, and a common 128-token text limit. [Full scores and protocol](../scores.json) include model revisions, sample counts, and paired uncertainty against the original and previous models. These fixed evaluation pools are reused across releases.
 
+## Release comparison
+
+Compared with the [previous Nano release](https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/70c3412bb0ddfb19386d673d4a102c753a574f1b/README.md), the parameter count remains 133,956,480 and the public input limits are unchanged. **Bold values improve on that release.**
+
+| Measured component | Previous Nano | Current Nano | Change |
+| --- | ---: | ---: | ---: |
+| Routing / cross-modal retrieval | 54.82 | **59.37** | +4.55 |
+| English v2 · mean task type | 51.98 | 51.98 | +0.00 |
+| Three image tasks · mean | 35.26 | **54.87** | +19.61 |
+| Audio-only · mean task type | 46.97 | 46.97 | +0.00 |
+
+The routing/retrieval component averages the two text accuracies, all six image recalls and all six audio recalls within their families, then weights the three families equally. The image component averages Oxford Pets classification accuracy, Tiny ImageNet NMI and Oxford Pets zero-shot accuracy. The English and audio rows use their complete official task-type means.
+
+A product release comparison weights these components 30%, 30%, 20% and 20%, respectively, yielding **+5.29 points**. This is a product-specific comparison, not an official benchmark aggregate or SOTA rank. Every underlying score and the paired retrieval uncertainty are in [scores.json](../scores.json).
+
+Oxford Pets classification improves by **46.80 points**, Tiny ImageNet NMI by **15.66 points**, and zero-shot Oxford Pets accuracy decreases by **3.62 points**. All six COCO recalls improve by 10.21–18.35 points. The text and audio computations are byte-identical to the previous release; their complete 41-task and 19-task reports retain the original evaluated identity and document [applicability to the current model](component-equivalence.json).
+
 ## Standard benchmarks
 
 Selected text, image and audio tasks evaluated with [MTEB 2.21.0](https://github.com/embeddings-benchmark/mteb), frozen encoders and official task protocols. Scores use a 0–100 scale; higher is better. **Bold marks the higher score between the two current Vela models.** This selected panel is not a complete MTEB, MMTEB, MIEB or MAEB result.
@@ -42,9 +59,9 @@ Selected text, image and audio tasks evaluated with [MTEB 2.21.0](https://github
 | ArguAna | nDCG@10 | **37.51** | 36.41 |
 | 20 Newsgroups · Clustering v2 | V-measure | **44.88** | 39.56 |
 | MTOP domain · English | Accuracy | **86.72** | 86.00 |
-| Oxford Pets · Classification | Accuracy | 44.47 | **92.26** |
-| Tiny ImageNet · Clustering | NMI | 46.03 | **75.55** |
-| Oxford Pets · Zero-shot | Accuracy | **15.26** | 9.57 |
+| Oxford Pets · Classification | Accuracy | 91.27 | **92.26** |
+| Tiny ImageNet · Clustering | NMI | 61.69 | **75.55** |
+| Oxford Pets · Zero-shot | Accuracy | **11.64** | 9.57 |
 | CREMA-D · Classification | Accuracy | **29.76** | 21.70 |
 | CREMA-D · Clustering | V-measure | **2.36** | 0.56 |
 | Speech Commands v0.02 · Zero-shot | Accuracy | 14.97 | **42.00** |
@@ -53,7 +70,7 @@ Evaluation uses FP32, batch size 8 and seed 20260918. Text limits are 512 tokens
 
 [Full metrics and protocol](../scores.json) include dataset and model revisions, evaluation counts, repeated-probe results, classification F1/precision/recall, retrieval metrics and clustering statistics. MTOP validation results are included there separately. These measurements show that fine-grained image label matching and audio emotion recognition need stronger alignment; speech retrieval scores should not be read as general audio understanding.
 
-On one AMD gfx942 GPU, synthetic batches of eight texts took 4.28 ms median at 128 tokens and 12.45 ms at 512 tokens (FP32, SDPA; 20 measured batches). Longer inputs increase latency and memory; [measurement details](../scores.json) include p95 latency and peak memory. These timings exclude model loading and are not service latency.
+For the previous Nano release, on one AMD gfx942 GPU, synthetic batches of eight texts took 4.28 ms median at 128 tokens and 12.45 ms at 512 tokens (FP32, SDPA; 20 measured batches). Longer inputs increase latency and memory; [measurement details](../scores.json) include p95 latency and peak memory. These historical timings exclude model loading and are not service latency; this release does not add a new timing measurement.
 
 ## MTEB English v2
 
@@ -95,4 +112,4 @@ Scores ×100; **bold marks the higher current Vela score**. Each group follows t
 
 ## Size–quality comparisons
 
-See [Pareto methodology and data](pareto-methodology.md) for the task selection, complete plotted peer set, parameter definitions and source snapshot.
+See the [four-figure gallery](pareto-gallery.md) and [Pareto methodology and data](pareto-methodology.md) for the task selection, complete plotted peer set, parameter definitions and source snapshot.

benchmarks/component-equivalence.json

@@ -0,0 +1,30 @@
+{
+  "schema_version": 1,
+  "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
+  "parameters": 133956480,
+  "evaluated_model": {
+    "revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b",
+    "native_artifact_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb"
+  },
+  "applies_to": {
+    "revision": "self",
+    "native_artifact_sha256": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf"
+  },
+  "unchanged_components": [
+    "text",
+    "audio"
+  ],
+  "method": "Exact stored tensor values, dtypes and shapes, with byte-identical inference source, configuration, tokenizer and media processors. The official evaluator, runtime, seed and batching remain the same.",
+  "tensor_identity": {
+    "text_tensors": 105,
+    "audio_tensors": 85,
+    "all_nonimage_tensors": 243,
+    "exact": true
+  },
+  "inference_files_exact": true,
+  "reports": [
+    "benchmarks/mteb-eng-v2.json",
+    "benchmarks/maeb-audio-only.json"
+  ],
+  "scope": "The original reports retain their actual evaluated revision and whole-model fingerprint. Applicability follows unchanged text/audio computation; these full panels were not rerun on the updated image artifact. Image and cross-modal scores use fresh evaluations."
+}

benchmarks/maeb-audio-only.json

@@ -10736,5 +10736,15 @@
         ]
       }
     }
-  ]
+  ],
+  "component_applicability": {
+    "applies_to_revision": "self",
+    "applies_to_native_artifact_sha256": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
+    "evaluated_revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b",
+    "evaluated_native_artifact_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
+    "basis": "Exact component tensor and inference-file identity; original measured identity is retained.",
+    "report": "benchmarks/component-equivalence.json",
+    "new_full_panel_inference": false,
+    "component": "audio"
+  }
 }

benchmarks/mteb-eng-v2.json

@@ -4617,5 +4617,15 @@
     "Complete English v2 panel using its exact task, subset and split definitions; this is not the full multilingual MMTEB or an image/audio benchmark.",
     "Training includes Banking77 and MASSIVE training examples and intent labels. Results include these adapted task families and should not be interpreted as zero-shot transfer across all tasks.",
     "Encoders are frozen during evaluation. Official labeled probes, clustering subsampling and other task-specific evaluation procedures are preserved."
-  ]
+  ],
+  "component_applicability": {
+    "applies_to_revision": "self",
+    "applies_to_native_artifact_sha256": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
+    "evaluated_revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b",
+    "evaluated_native_artifact_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
+    "basis": "Exact component tensor and inference-file identity; original measured identity is retained.",
+    "report": "benchmarks/component-equivalence.json",
+    "new_full_panel_inference": false,
+    "component": "text"
+  }
 }

benchmarks/pareto-data.json

@@ -1,13 +1,14 @@
 {
-  "snapshot_date": "2026-09-17",
   "score_scale": "0-100",
   "parameter_definition": "Reported total model parameters from the snapshot; exact counted total for Vela, including all modalities.",
   "eligibility": "All finite same-task scores with a known, strictly positive totalParamsB; open and closed weights; no size ceiling.",
   "official_sota_claim": false,
+  "snapshot_date": "2026-09-17",
   "tasks": {
     "arxiv": {
       "task": "ArXivHierarchicalClusteringP2P",
-      "metric": "V-measure",
+      "metric": "v_measure",
+      "panel": "English41",
       "sources": [
         {
           "url": "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MTEB%28eng%2C%20v2%29/scores",
@@ -2062,6 +2063,7 @@
           "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
           "parameters_billion": 0.13395648,
           "parameters_exact": 133956480,
+          "native_fingerprint": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
           "score": 65.25395409281045,
           "open_weights": true,
           "modalities": [
@@ -2072,8 +2074,12 @@
           "size_basis": "counted_all_native_model_parameters",
           "kind": "nano",
           "evaluated_task_revision": "0bbdb47bcbe3a90093699aefeed338a0f28a7ee8",
-          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/9302473d8cb0ba39cf8f43678fdd29d74df43248/benchmarks/mteb-eng-v2.json",
-          "on_displayed_frontier": true
+          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/ed7263b501430a011d37b147d6e264ad8354983b/benchmarks/mteb-eng-v2.json",
+          "on_displayed_frontier": true,
+          "evaluated_native_fingerprint": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
+          "evaluation_applicability": "Unchanged exact text/audio computation; see the current Nano component-equivalence report.",
+          "component_equivalence_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/main/benchmarks/component-equivalence.json",
+          "evaluated_revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b"
         },
         {
           "model": "silma-ai/silma-embeddding-matryoshka-v0.1",
@@ -5389,6 +5395,7 @@
           "model": "llm-semantic-router/Vela-1.0-Omni-Mini",
           "parameters_billion": 1.04415264,
           "parameters_exact": 1044152640,
+          "native_fingerprint": "035f6084c6c984f465e4f91c50caf8f13790177d4c0c29e4ce5d6c0c233fd1a0",
           "score": 65.52292325553617,
           "open_weights": true,
           "modalities": [
@@ -5399,7 +5406,7 @@
           "size_basis": "counted_all_native_model_parameters",
           "kind": "mini",
           "evaluated_task_revision": "0bbdb47bcbe3a90093699aefeed338a0f28a7ee8",
-          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Mini/blob/65dc28c2f7c1889886dee6b31d9d351ad76a998e/benchmarks/mteb-eng-v2.json",
+          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Mini/blob/662d4e53c52d22da6430b0ac316360d41028374d/benchmarks/mteb-eng-v2.json",
           "on_displayed_frontier": false
         },
         {
@@ -8645,7 +8652,8 @@
     },
     "nmsqa": {
       "task": "NMSQAPairClassification",
-      "metric": "Maximum average precision",
+      "metric": "max_ap",
+      "panel": "audio19",
       "sources": [
         {
           "url": "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
@@ -9062,6 +9070,7 @@
           "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
           "parameters_billion": 0.13395648,
           "parameters_exact": 133956480,
+          "native_fingerprint": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
           "score": 63.43546123003914,
           "open_weights": true,
           "modalities": [
@@ -9072,8 +9081,12 @@
           "size_basis": "counted_all_native_model_parameters",
           "kind": "nano",
           "evaluated_task_revision": "d33eea302d8a64c0a2ee094cf39d77c0814fe87a",
-          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/9302473d8cb0ba39cf8f43678fdd29d74df43248/benchmarks/maeb-audio-only.json",
-          "on_displayed_frontier": true
+          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/ed7263b501430a011d37b147d6e264ad8354983b/benchmarks/maeb-audio-only.json",
+          "on_displayed_frontier": true,
+          "evaluated_native_fingerprint": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
+          "evaluation_applicability": "Unchanged exact text/audio computation; see the current Nano component-equivalence report.",
+          "component_equivalence_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/main/benchmarks/component-equivalence.json",
+          "evaluated_revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b"
         },
         {
           "model": "asapp/sew-d-mid-400k-ft-ls100h",
@@ -9574,6 +9587,7 @@
           "model": "llm-semantic-router/Vela-1.0-Omni-Mini",
           "parameters_billion": 1.04415264,
           "parameters_exact": 1044152640,
+          "native_fingerprint": "035f6084c6c984f465e4f91c50caf8f13790177d4c0c29e4ce5d6c0c233fd1a0",
           "score": 86.5526042027746,
           "open_weights": true,
           "modalities": [
@@ -9584,7 +9598,7 @@
           "size_basis": "counted_all_native_model_parameters",
           "kind": "mini",
           "evaluated_task_revision": "d33eea302d8a64c0a2ee094cf39d77c0814fe87a",
-          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Mini/blob/65dc28c2f7c1889886dee6b31d9d351ad76a998e/benchmarks/maeb-audio-only.json",
+          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Mini/blob/662d4e53c52d22da6430b0ac316360d41028374d/benchmarks/maeb-audio-only.json",
           "on_displayed_frontier": false
         },
         {
@@ -10123,7 +10137,8 @@
     },
     "sibfleurs": {
       "task": "SIBFLEURS",
-      "metric": "Accuracy",
+      "metric": "accuracy",
+      "panel": "audio19",
       "sources": [
         {
           "url": "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
@@ -10540,6 +10555,7 @@
           "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
           "parameters_billion": 0.13395648,
           "parameters_exact": 133956480,
+          "native_fingerprint": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
           "score": 18.304657831512046,
           "open_weights": true,
           "modalities": [
@@ -10550,8 +10566,12 @@
           "size_basis": "counted_all_native_model_parameters",
           "kind": "nano",
           "evaluated_task_revision": "186b61175fbd77059f769b6bf1110d449a2ff311",
-          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/9302473d8cb0ba39cf8f43678fdd29d74df43248/benchmarks/maeb-audio-only.json",
-          "on_displayed_frontier": false
+          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/ed7263b501430a011d37b147d6e264ad8354983b/benchmarks/maeb-audio-only.json",
+          "on_displayed_frontier": false,
+          "evaluated_native_fingerprint": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
+          "evaluation_applicability": "Unchanged exact text/audio computation; see the current Nano component-equivalence report.",
+          "component_equivalence_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/main/benchmarks/component-equivalence.json",
+          "evaluated_revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b"
         },
         {
           "model": "asapp/sew-d-mid-400k-ft-ls100h",
@@ -11052,6 +11072,7 @@
           "model": "llm-semantic-router/Vela-1.0-Omni-Mini",
           "parameters_billion": 1.04415264,
           "parameters_exact": 1044152640,
+          "native_fingerprint": "035f6084c6c984f465e4f91c50caf8f13790177d4c0c29e4ce5d6c0c233fd1a0",
           "score": 39.43152755173216,
           "open_weights": true,
           "modalities": [
@@ -11062,7 +11083,7 @@
           "size_basis": "counted_all_native_model_parameters",
           "kind": "mini",
           "evaluated_task_revision": "186b61175fbd77059f769b6bf1110d449a2ff311",
-          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Mini/blob/65dc28c2f7c1889886dee6b31d9d351ad76a998e/benchmarks/maeb-audio-only.json",
+          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Mini/blob/662d4e53c52d22da6430b0ac316360d41028374d/benchmarks/maeb-audio-only.json",
           "on_displayed_frontier": true
         },
         {
@@ -11619,6 +11640,1526 @@
         "facebook/seamless-m4t-v2-large",
         "BidirLM/BidirLM-Omni-2.5B-Embedding"
       ]
+    },
+    "vehiclesoundclustering": {
+      "task": "VehicleSoundClustering",
+      "metric": "v_measure",
+      "panel": "audio19",
+      "sources": [
+        {
+          "url": "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+          "sha256": "6153942c26a31577cc1e3a2fbb6eb10b09dc2f1fe08749db598d261e653f6bca",
+          "retrieved_at_utc": "2026-09-17T20:28:48.670374+00:00",
+          "benchmark": "MAEB(beta, audio-only)"
+        },
+        {
+          "url": "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores",
+          "sha256": "601467f576706074bb5724aa0c10a50e948b0e8be8158742b01baf2fd7f7f14c",
+          "retrieved_at_utc": "2026-09-17T20:28:48.161556+00:00",
+          "benchmark": "MAEB(beta)"
+        }
+      ],
+      "points": [
+        {
+          "model": "google/yamnet",
+          "parameters_billion": 0.004,
+          "score": 0.35100000000000003,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "AudioSetMini"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": true
+        },
+        {
+          "model": "asapp/sew-d-tiny-100k-ft-ls100h",
+          "parameters_billion": 0.02,
+          "score": 0.3376,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "LibriSpeech"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "matthewagi/HeAR-s1.1",
+          "parameters_billion": 0.022,
+          "score": 3.8280000000000003,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": true
+        },
+        {
+          "model": "facebook/encodec_24khz",
+          "parameters_billion": 0.023,
+          "score": 3.8041,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "AudioSetMini",
+            "CommonVoiceMini17A2TRetrieval",
+            "FSD50K"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "openai/whisper-tiny",
+          "parameters_billion": 0.039,
+          "score": 12.3883,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": true
+        },
+        {
+          "model": "google/vggish",
+          "parameters_billion": 0.072,
+          "score": 4.4608,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "AudioSetMini"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "openai/whisper-base",
+          "parameters_billion": 0.074,
+          "score": 2.4958,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "speechbrain/cnn14-esc50",
+          "parameters_billion": 0.081,
+          "score": 2.4725,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "ESC50"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "MIT/ast-finetuned-audioset-10-10-0.4593",
+          "parameters_billion": 0.087,
+          "score": 13.366700000000002,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "AudioSetMini"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": true
+        },
+        {
+          "model": "facebook/data2vec-audio-base-960h",
+          "parameters_billion": 0.093,
+          "score": 1.3347,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "microsoft/unispeech-sat-base-100h-libri-ft",
+          "parameters_billion": 0.094,
+          "score": 0.32789999999999997,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "LibriSpeech"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/wav2vec2-base",
+          "parameters_billion": 0.095,
+          "score": 3.0932999999999997,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "microsoft/wavlm-base-plus",
+          "parameters_billion": 0.095,
+          "score": 1.7932,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "GigaSpeechA2TRetrieval",
+            "Libri-Light",
+            "VoxPopuliLanguageID"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "microsoft/wavlm-base-plus-sd",
+          "parameters_billion": 0.095,
+          "score": 1.7932,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "GigaSpeechA2TRetrieval",
+            "Libri-Light",
+            "LibriMix",
+            "VoxPopuliLanguageID"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "microsoft/wavlm-base-plus-sv",
+          "parameters_billion": 0.095,
+          "score": 1.7932,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "GigaSpeechA2TRetrieval",
+            "Libri-Light",
+            "VoxCeleb1",
+            "VoxPopuliLanguageID"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "asapp/sew-d-base-plus-400k-ft-ls100h",
+          "parameters_billion": 0.095,
+          "score": 0.6319,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "LibriSpeech"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/wav2vec2-base-960h",
+          "parameters_billion": 0.095,
+          "score": 0.6122,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "microsoft/wavlm-base",
+          "parameters_billion": 0.095,
+          "score": 0.40940000000000004,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "Librispeech"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "microsoft/wavlm-base-sd",
+          "parameters_billion": 0.095,
+          "score": 0.40940000000000004,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "LibriMix",
+            "Librispeech"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "microsoft/wavlm-base-sv",
+          "parameters_billion": 0.095,
+          "score": 0.40940000000000004,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "Librispeech",
+            "VoxCeleb1"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/hubert-base-ls960",
+          "parameters_billion": 0.095,
+          "score": 0.1305,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
+          "parameters_billion": 0.13395648,
+          "parameters_exact": 133956480,
+          "native_fingerprint": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
+          "score": 13.800701178563807,
+          "open_weights": true,
+          "modalities": [
+            "text",
+            "image",
+            "audio"
+          ],
+          "size_basis": "counted_all_native_model_parameters",
+          "kind": "nano",
+          "evaluated_task_revision": "5f905f9544d0c78b607639662c88c2ddd1216384",
+          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/ed7263b501430a011d37b147d6e264ad8354983b/benchmarks/maeb-audio-only.json",
+          "on_displayed_frontier": true,
+          "evaluated_native_fingerprint": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
+          "evaluation_applicability": "Unchanged exact text/audio computation; see the current Nano component-equivalence report.",
+          "component_equivalence_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Nano/blob/main/benchmarks/component-equivalence.json",
+          "evaluated_revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b"
+        },
+        {
+          "model": "asapp/sew-d-mid-400k-ft-ls100h",
+          "parameters_billion": 0.139,
+          "score": 0.6276,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "LibriSpeech"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "laion/clap-htsat-unfused",
+          "parameters_billion": 0.153,
+          "score": 2.6642,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "text"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "laion/clap-htsat-fused",
+          "parameters_billion": 0.154,
+          "score": 4.7132,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "text"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "microsoft/msclap-2023",
+          "parameters_billion": 0.16,
+          "score": 2.86,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "text"
+          ],
+          "training_tasks": [
+            "AudioCapsA2TRetrieval",
+            "ClothoA2TRetrieval",
+            "FSD50K"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "lyrebird/wav2clip",
+          "parameters_billion": 0.163,
+          "score": 3.9723,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "text"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "laion/larger_clap_music_and_speech",
+          "parameters_billion": 0.194,
+          "score": 4.7243,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "text"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "laion/larger_clap_music",
+          "parameters_billion": 0.194,
+          "score": 3.7272,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "text"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "laion/larger_clap_general",
+          "parameters_billion": 0.194,
+          "score": 3.3663,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "text"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "microsoft/msclap-2022",
+          "parameters_billion": 0.196,
+          "score": 7.979799999999999,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "text"
+          ],
+          "training_tasks": [
+            "AudioCapsA2TRetrieval",
+            "AudioSetMini",
+            "ClothoA2TRetrieval",
+            "ESC50",
+            "FSD50K",
+            "GTZANGenre",
+            "UrbanSound8k"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "openai/whisper-small",
+          "parameters_billion": 0.244,
+          "score": 1.2919999999999998,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "microsoft/speecht5_multimodal",
+          "parameters_billion": 0.298,
+          "score": 0.46769999999999995,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "text"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/wav2vec2-xls-r-300m",
+          "parameters_billion": 0.3,
+          "score": 2.3022,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "CommonVoiceMini17A2TRetrieval",
+            "VoxLingua107_Top10",
+            "VoxPopuliLanguageID"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "vitouphy/wav2vec2-xls-r-300m-phoneme",
+          "parameters_billion": 0.3,
+          "score": 1.2399,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "CommonVoiceMini17A2TRetrieval",
+            "VoxLingua107_Top10",
+            "VoxPopuliLanguageID"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/data2vec-audio-large-960h",
+          "parameters_billion": 0.313,
+          "score": 0.1542,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/wav2vec2-lv-60-espeak-cv-ft",
+          "parameters_billion": 0.317,
+          "score": 2.3741000000000003,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "CommonVoiceMini17A2TRetrieval"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/wav2vec2-large-xlsr-53",
+          "parameters_billion": 0.317,
+          "score": 1.7241,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "CommonVoiceMini17A2TRetrieval"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/wav2vec2-large",
+          "parameters_billion": 0.317,
+          "score": 1.411,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/hubert-large-ls960-ft",
+          "parameters_billion": 0.317,
+          "score": 0.5867,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "microsoft/wavlm-large",
+          "parameters_billion": 0.317,
+          "score": 0.17550000000000002,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "GigaSpeechA2TRetrieval",
+            "Libri-Light",
+            "VoxPopuliLanguageID"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "OpenMuQ/MuQ-MuLan-large",
+          "parameters_billion": 0.63,
+          "score": 0.9233999999999999,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "text"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "openai/whisper-medium",
+          "parameters_billion": 0.769,
+          "score": 1.0773,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "jinaai/jina-embeddings-v5-omni-nano",
+          "parameters_billion": 0.986,
+          "score": 1.6566,
+          "open_weights": true,
+          "modalities": [
+            "text",
+            "image",
+            "audio",
+            "video"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/mms-1b-l1107",
+          "parameters_billion": 1.0,
+          "score": 0.4373,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "FleursA2TRetrieval"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/mms-1b-all",
+          "parameters_billion": 1.0,
+          "score": 0.32030000000000003,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "FleursA2TRetrieval"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/mms-1b-fl102",
+          "parameters_billion": 1.0,
+          "score": 0.146,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "FleursA2TRetrieval"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/wav2vec2-xls-r-1b",
+          "parameters_billion": 1.0,
+          "score": 0.1275,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "CommonVoiceMini17A2TRetrieval",
+            "VoxLingua107_Top10",
+            "VoxPopuliLanguageID"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "llm-semantic-router/Vela-1.0-Omni-Mini",
+          "parameters_billion": 1.04415264,
+          "parameters_exact": 1044152640,
+          "native_fingerprint": "035f6084c6c984f465e4f91c50caf8f13790177d4c0c29e4ce5d6c0c233fd1a0",
+          "score": 8.718116828237402,
+          "open_weights": true,
+          "modalities": [
+            "text",
+            "image",
+            "audio"
+          ],
+          "size_basis": "counted_all_native_model_parameters",
+          "kind": "mini",
+          "evaluated_task_revision": "5f905f9544d0c78b607639662c88c2ddd1216384",
+          "report_url": "https://huggingface.co/llm-semantic-router/Vela-1.0-Omni-Mini/blob/662d4e53c52d22da6430b0ac316360d41028374d/benchmarks/maeb-audio-only.json",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "speechbrain/m-ctc-t-large",
+          "parameters_billion": 1.059,
+          "score": 0.967,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "CommonVoiceMini17A2TRetrieval",
+            "VoxPopuliLanguageID"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "LanguageBind/LanguageBind_Omni",
+          "parameters_billion": 1.2,
+          "score": 2.5643,
+          "open_weights": true,
+          "modalities": [
+            "image",
+            "text",
+            "audio",
+            "video"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "openai/whisper-large-v3",
+          "parameters_billion": 1.55,
+          "score": 1.3834000000000002,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "jinaai/jina-embeddings-v5-omni-small",
+          "parameters_billion": 1.626,
+          "score": 4.8423,
+          "open_weights": true,
+          "modalities": [
+            "text",
+            "image",
+            "audio",
+            "video"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "encord-team/ebind-full",
+          "parameters_billion": 1.79,
+          "score": 6.4883,
+          "open_weights": true,
+          "modalities": [
+            "text",
+            "image",
+            "audio",
+            "video"
+          ],
+          "training_tasks": [
+            "AudioCapsA2TRetrieval",
+            "AudioCapsAVA2VRetrieval",
+            "AudioCapsAVAT2VRetrieval",
+            "AudioCapsAVT2VARetrieval",
+            "AudioCapsAVT2VRetrieval",
+            "AudioCapsAVV2ARetrieval",
+            "AudioCapsAVV2TRetrieval",
+            "AudioCapsAVVA2TRetrieval",
+            "AudioCapsAVVT2ARetrieval",
+            "AudioCapsT2ARetrieval",
+            "Imagenet1k",
+            "Imagenet1kZeroShot",
+            "VALOR32KA2VRetrieval",
+            "VALOR32KAT2VRetrieval",
+            "VALOR32KT2VARetrieval",
+            "VALOR32KT2VRetrieval",
+            "VALOR32KV2ARetrieval",
+            "VALOR32KV2TRetrieval",
+            "VALOR32KVA2TRetrieval",
+            "VALOR32KVT2ARetrieval",
+            "VGGSoundAVA2VRetrieval",
+            "VGGSoundAVAT2VRetrieval",
+            "VGGSoundAVT2VARetrieval",
+            "VGGSoundAVT2VRetrieval",
+            "VGGSoundAVV2ARetrieval",
+            "VGGSoundAVV2TRetrieval",
+            "VGGSoundAVVA2TRetrieval",
+            "VGGSoundAVVT2ARetrieval",
+            "VGGSoundV",
+            "VGGSoundVA",
+            "VGGSoundVideoAudioZeroshot",
+            "VGGSoundVideoZeroshot"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/wav2vec2-xls-r-2b",
+          "parameters_billion": 2.0,
+          "score": 0.9359,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "CommonVoiceMini17A2TRetrieval",
+            "VoxLingua107_Top10",
+            "VoxPopuliLanguageID"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "facebook/seamless-m4t-v2-large",
+          "parameters_billion": 2.3,
+          "score": 0.4163,
+          "open_weights": true,
+          "modalities": [
+            "audio"
+          ],
+          "training_tasks": [
+            "FleursA2TRetrieval"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "BidirLM/BidirLM-Omni-2.5B-Embedding",
+          "parameters_billion": 2.445,
+          "score": 5.1074,
+          "open_weights": true,
+          "modalities": [
+            "text",
+            "image",
+            "audio",
+            "video"
+          ],
+          "training_tasks": [
+            "AFQMC",
+            "ATEC",
+            "AdvertiseGen",
+            "BQ",
+            "CAIL2019-SCM",
+            "CHEF",
+            "CINLID",
+            "ChatMed_Consult_Dataset",
+            "ChineseSTS",
+            "CmedqaRetrieval",
+            "Cmnli",
+            "CodeFeedback",
+            "DRCD",
+            "DuRetrieval",
+            "ELI5_custom",
+            "ESCIReranking",
+            "EmotionClassification",
+            "Expertqa",
+            "FollowIR",
+            "GooAQ",
+            "InF-IR",
+            "JW300",
+            "LAION-Audio-300M",
+            "LCSTS",
+            "MAmmoTH2",
+            "MEDI2BGE",
+            "MIRACLRetrieval",
+            "MSMARCO",
+            "MS_COCO",
+            "Multi-CPR",
+            "MultiLongDocReranking",
+            "NFCorpus",
+            "NaturalQuestions",
+            "OPUS-100",
+            "OpenOrca",
+            "PAQ",
+            "PubMedQA",
+            "QBQTC",
+            "RefGPT",
+            "SQuAD",
+            "SearchQA",
+            "SimCLUE",
+            "SyntheticClassificationData",
+            "T2Ranking",
+            "TED Talks",
+            "THUCNews",
+            "TriviaQA",
+            "UMETRIP-QA",
+            "WebCPM",
+            "WikiAnswers",
+            "WikiMatrix",
+            "cCOVID-News",
+            "ccnews",
+            "cmrc2018",
+            "colpali_train_set",
+            "contract-nli",
+            "csl",
+            "dureader_mrc",
+            "law-gpt",
+            "lawzhidao",
+            "librispeech_asr",
+            "lima-chinese",
+            "mmarco-chinese",
+            "mnli",
+            "natcap",
+            "nli_zh",
+            "nllb",
+            "ocnli",
+            "rag-dataset-12000",
+            "simcse_sup_nli",
+            "webgpt_comparisons",
+            "webqa",
+            "wikipedia-nq",
+            "xnli_zh",
+            "yahoo-answers"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "EximiusLabs/fusion-embedding-1-2b-preview",
+          "parameters_billion": 2.8,
+          "score": 3.495,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "image",
+            "text",
+            "video"
+          ],
+          "training_tasks": [
+            "AudioCapsA2TRetrieval",
+            "AudioCapsAVA2VRetrieval",
+            "AudioCapsAVAT2VRetrieval",
+            "AudioCapsAVT2VARetrieval",
+            "AudioCapsAVT2VRetrieval",
+            "AudioCapsAVV2ARetrieval",
+            "AudioCapsAVV2TRetrieval",
+            "AudioCapsAVVA2TRetrieval",
+            "AudioCapsAVVT2ARetrieval",
+            "AudioCapsT2ARetrieval",
+            "AudioSet",
+            "AudioSetMini",
+            "CommonLanguageAgeDetection",
+            "CommonVoiceMini21T2ARetrieval",
+            "FSD2019Kaggle",
+            "FSD50K"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "EximiusLabs/fusion-embedding-2-2b-preview",
+          "parameters_billion": 2.8,
+          "score": 2.4577,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "image",
+            "text",
+            "video"
+          ],
+          "training_tasks": [
+            "AudioCapsA2TRetrieval",
+            "AudioCapsAVA2VRetrieval",
+            "AudioCapsAVAT2VRetrieval",
+            "AudioCapsAVT2VARetrieval",
+            "AudioCapsAVT2VRetrieval",
+            "AudioCapsAVV2ARetrieval",
+            "AudioCapsAVV2TRetrieval",
+            "AudioCapsAVVA2TRetrieval",
+            "AudioCapsAVVT2ARetrieval",
+            "AudioCapsT2ARetrieval",
+            "AudioSet",
+            "AudioSetMini",
+            "CommonLanguageAgeDetection",
+            "CommonVoiceMini21T2ARetrieval",
+            "FSD2019Kaggle",
+            "FSD50K"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "Haon-Chen/e5-omni-3B",
+          "parameters_billion": 4.703,
+          "score": 3.7803000000000004,
+          "open_weights": true,
+          "modalities": [
+            "text",
+            "image",
+            "audio",
+            "video"
+          ],
+          "training_tasks": [
+            "AudioCapsT2ARetrieval",
+            "MSRVTTT2V",
+            "MSRVTTV2T"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "LCO-Embedding/LCO-Embedding-Omni-3B",
+          "parameters_billion": 4.703,
+          "score": 2.1795,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "image",
+            "text",
+            "video"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "Qwen/Qwen2.5-Omni-3B",
+          "parameters_billion": 5.537,
+          "score": 1.1603,
+          "open_weights": true,
+          "modalities": [
+            "text",
+            "image",
+            "audio",
+            "video"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "Qwen/Qwen2-Audio-7B",
+          "parameters_billion": 7.0,
+          "score": 5.518,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "text"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "LCO-Embedding/LCO-Embedding-Omni-7B",
+          "parameters_billion": 8.932,
+          "score": 3.6289000000000002,
+          "open_weights": true,
+          "modalities": [
+            "audio",
+            "image",
+            "text",
+            "video"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "Haon-Chen/e5-omni-7B",
+          "parameters_billion": 8.932,
+          "score": 2.4320999999999997,
+          "open_weights": true,
+          "modalities": [
+            "text",
+            "image",
+            "audio",
+            "video"
+          ],
+          "training_tasks": [
+            "AudioCapsT2ARetrieval",
+            "MSRVTTT2V",
+            "MSRVTTV2T"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "Tevatron/OmniEmbed-v0.1",
+          "parameters_billion": 8.932,
+          "score": 1.4678,
+          "open_weights": true,
+          "modalities": [
+            "text",
+            "image",
+            "audio",
+            "video"
+          ],
+          "training_tasks": [
+            "AudioCapsT2ARetrieval",
+            "FEVER",
+            "MSMARCO",
+            "MSRVTTT2V",
+            "MSRVTTV2T",
+            "NQ",
+            "SCIDOCS",
+            "VidoreArxivQARetrieval",
+            "VidoreDocVQARetrieval",
+            "VidoreInfoVQARetrieval",
+            "VidoreTatdqaRetrieval"
+          ],
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "Qwen/Qwen2.5-Omni-7B",
+          "parameters_billion": 10.732,
+          "score": 0.8654999999999999,
+          "open_weights": true,
+          "modalities": [
+            "text",
+            "image",
+            "audio",
+            "video"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "Qwen/Qwen3-Omni-30B-A3B-Thinking",
+          "parameters_billion": 31.719,
+          "score": 0.6526,
+          "open_weights": true,
+          "modalities": [
+            "text",
+            "image",
+            "audio",
+            "video"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        },
+        {
+          "model": "Qwen/Qwen3-Omni-30B-A3B-Instruct",
+          "parameters_billion": 35.26,
+          "score": 1.2456,
+          "open_weights": true,
+          "modalities": [
+            "text",
+            "image",
+            "audio",
+            "video"
+          ],
+          "training_tasks": null,
+          "source_urls": [
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%2C%20audio-only%29/scores",
+            "https://mteb-leaderboard-backend.hf.space/v1/benchmarks/MAEB%28beta%29/scores"
+          ],
+          "size_basis": "reported_rounded_total",
+          "kind": "peer",
+          "on_displayed_frontier": false
+        }
+      ],
+      "excluded": [],
+      "peer_count": 67,
+      "frontier_models": [
+        "google/yamnet",
+        "matthewagi/HeAR-s1.1",
+        "openai/whisper-tiny",
+        "MIT/ast-finetuned-audioset-10-10-0.4593",
+        "llm-semantic-router/Vela-1.0-Omni-Nano"
+      ]
     }
   }
 }

benchmarks/pareto-gallery.md

@@ -0,0 +1,48 @@
+# Size–quality figures
+
+[Model card](../README.md) · [Detailed evaluation](EVALUATION.md) · [Methodology and sources](pareto-methodology.md) · [All points (CSV)](pareto-points.csv)
+
+Selected task-level strengths from the complete 41-task English and 19-task audio evaluations. Every figure includes both Vela models and all peers with known positive total parameter counts. The September 17, 2026 registry snapshot provides reported peer scores; evaluation protocols can differ.
+
+| Task | Metric | Nano | Mini | Vela on observed frontier |
+| --- | --- | ---: | ---: | --- |
+| ArXiv hierarchical clustering | V-measure | **65.25** | 65.52 | Nano |
+| NMSQA audio pair classification | Maximum average precision | **63.44** | 86.55 | Nano |
+| SIBFLEURS audio classification | Accuracy | 18.30 | **39.43** | Mini |
+| Vehicle sound clustering | V-measure | **13.80** | 8.72 | Nano |
+
+Bold scores identify a Vela model on the observed task frontier. They do not indicate an overall benchmark rank or a statistically significant lead. Vela sizes include all text, image and audio components.
+
+## ArXiv hierarchical clustering
+
+MTEB English / MMTEB · V-measure · 210 reported peer models
+
+![ArXiv hierarchical clustering: total parameters versus V-measure, with the observed Pareto frontier](../assets/pareto-arxiv.png)
+
+[Vector figure (SVG)](../assets/pareto-arxiv.svg)
+
+## NMSQA audio pair classification
+
+MAEB audio · Maximum average precision · 66 reported peer models
+
+![NMSQA audio pair classification: total parameters versus Maximum average precision, with the observed Pareto frontier](../assets/pareto-nmsqa.png)
+
+[Vector figure (SVG)](../assets/pareto-nmsqa.svg)
+
+## SIBFLEURS audio classification
+
+MAEB audio · Accuracy · 67 reported peer models
+
+![SIBFLEURS audio classification: total parameters versus Accuracy, with the observed Pareto frontier](../assets/pareto-sibfleurs.png)
+
+[Vector figure (SVG)](../assets/pareto-sibfleurs.svg)
+
+## Vehicle sound clustering
+
+MAEB audio · V-measure · 67 reported peer models
+
+![Vehicle sound clustering: total parameters versus V-measure, with the observed Pareto frontier](../assets/pareto-vehiclesoundclustering.png)
+
+[Vector figure (SVG)](../assets/pareto-vehiclesoundclustering.svg)
+
+Vehicle sound clustering has a small observed margin: Nano scores 13.80 versus 13.37 for the strongest reported peer, AST AudioSet (87M). This snapshot comparison does not establish statistical significance.

benchmarks/pareto-methodology.md

@@ -1,14 +1,15 @@
 # Size–quality comparison methodology
 
-[Model card](../README.md) · [Complete evaluation](EVALUATION.md) · [All plotted points (CSV)](pareto-points.csv) · [Scores, sources and exclusions (JSON)](pareto-data.json)
+[Model card](../README.md) · [Figure gallery](pareto-gallery.md) · [Complete evaluation](EVALUATION.md) · [All plotted points (CSV)](pareto-points.csv) · [Scores, sources and exclusions (JSON)](pareto-data.json)
 
-These figures show the observed size–quality trade-off on three individual tasks. They are selected strengths identified by screening all 41 English and 19 audio-only tasks for both released Vela models. They are not an average across those panels or a complete multilingual, image or multimodal benchmark result.
+These figures show the observed size–quality trade-off on four individual tasks. They are selected strengths identified by screening all 41 English and 19 audio-only tasks for both released Vela models. They are not an average across those panels or a complete multilingual, image or multimodal benchmark result.
 
 | Task | Benchmark family | Main metric | External peers | Vela model on the observed frontier |
 | --- | --- | --- | ---: | --- |
 | ArXivHierarchicalClusteringP2P | MTEB English v2 / MMTEB | V-measure | 210 | Nano |
 | NMSQAPairClassification | MAEB audio | Maximum average precision | 66 | Nano |
 | SIBFLEURS | MAEB audio | Accuracy | 67 | Mini |
+| VehicleSoundClustering | MAEB audio | V-measure | 67 | Nano |
 
 ## Points and frontier
 
@@ -24,11 +25,11 @@ The horizontal axis is logarithmic and uses **total parameters**, not active par
 
 The vertical axis uses each task's primary metric multiplied by 100, without transforming or rescaling the scores relative to competitors. SIBFLEURS uses the official labeled classification probe; NMSQA is an audio pair-classification task. These audio results belong to MAEB, not the multilingual text MMTEB benchmark.
 
-Vela points come from the complete [English task report](mteb-eng-v2.json) and [audio task report](maeb-audio-only.json). The reports retain the actual evaluated revisions and processing, including component-equivalent current artifacts. Vela uses MTEB 2.21.0, FP32, batch size 8, seed 20260918 and its native token limits; full protocols are in the [evaluation report](EVALUATION.md).
+Vela points come from the complete [English task report](mteb-eng-v2.json) and [audio task report](maeb-audio-only.json). The reports retain the actual evaluated revisions and processing, including component-equivalent current artifacts. The current Nano image update preserves the exact evaluated text and audio computation; its reports retain the original evaluated identity and explicitly record current applicability. Vela uses MTEB 2.21.0, FP32, batch size 8, seed 20260918 and its native token limits; full protocols are in the [evaluation report](EVALUATION.md).
 
 ## Sources and comparison limits
 
-The [MTEB leaderboard](https://leaderboard.mteb.org/) API snapshot was fetched on **September 17, 2026, around 20:28 UTC**. Exact source URLs, fetch timestamps and SHA-256 hashes are included in the JSON. Sources are the English v2 and multilingual v2 panels for ArXiv, and the audio-only and full MAEB beta panels for NMSQA and SIBFLEURS.
+The [MTEB leaderboard](https://leaderboard.mteb.org/) API snapshot was fetched on **September 17, 2026, around 20:28 UTC**. Exact source URLs, fetch timestamps and SHA-256 hashes are included in the JSON. Sources are the English v2 and multilingual v2 panels for ArXiv, and the audio-only and full MAEB beta panels for NMSQA, SIBFLEURS and VehicleSoundClustering.
 
 The registry provides reported scores, not matched reruns of every model. It does not establish identical scored checkpoint or dataset revisions, prompts, precision, seeds, preprocessing or training exposure across peers. Some models use training examples from the evaluated task families; the JSON preserves registered training-task information where available. A position on these observed frontiers supports a task-specific size–quality comparison, not a statistical significance claim, an official rank, or overall SOTA.
 

benchmarks/pareto-points.csv

@@ -348,3 +348,72 @@ SIBFLEURS,Tevatron/OmniEmbed-v0.1,8.932,27.626134313725494,True,peer,False
 SIBFLEURS,Qwen/Qwen2.5-Omni-7B,10.732,18.12175294117647,True,peer,False
 SIBFLEURS,Qwen/Qwen3-Omni-30B-A3B-Thinking,31.719,19.916688235294117,True,peer,False
 SIBFLEURS,Qwen/Qwen3-Omni-30B-A3B-Instruct,35.26,21.777107843137255,True,peer,False
+VehicleSoundClustering,google/yamnet,0.004,0.35100000000000003,True,peer,True
+VehicleSoundClustering,asapp/sew-d-tiny-100k-ft-ls100h,0.02,0.3376,True,peer,False
+VehicleSoundClustering,matthewagi/HeAR-s1.1,0.022,3.8280000000000003,True,peer,True
+VehicleSoundClustering,facebook/encodec_24khz,0.023,3.8041,True,peer,False
+VehicleSoundClustering,openai/whisper-tiny,0.039,12.3883,True,peer,True
+VehicleSoundClustering,google/vggish,0.072,4.4608,True,peer,False
+VehicleSoundClustering,openai/whisper-base,0.074,2.4958,True,peer,False
+VehicleSoundClustering,speechbrain/cnn14-esc50,0.081,2.4725,True,peer,False
+VehicleSoundClustering,MIT/ast-finetuned-audioset-10-10-0.4593,0.087,13.366700000000002,True,peer,True
+VehicleSoundClustering,facebook/data2vec-audio-base-960h,0.093,1.3347,True,peer,False
+VehicleSoundClustering,microsoft/unispeech-sat-base-100h-libri-ft,0.094,0.32789999999999997,True,peer,False
+VehicleSoundClustering,facebook/wav2vec2-base,0.095,3.0932999999999997,True,peer,False
+VehicleSoundClustering,microsoft/wavlm-base-plus,0.095,1.7932,True,peer,False
+VehicleSoundClustering,microsoft/wavlm-base-plus-sd,0.095,1.7932,True,peer,False
+VehicleSoundClustering,microsoft/wavlm-base-plus-sv,0.095,1.7932,True,peer,False
+VehicleSoundClustering,asapp/sew-d-base-plus-400k-ft-ls100h,0.095,0.6319,True,peer,False
+VehicleSoundClustering,facebook/wav2vec2-base-960h,0.095,0.6122,True,peer,False
+VehicleSoundClustering,microsoft/wavlm-base,0.095,0.40940000000000004,True,peer,False
+VehicleSoundClustering,microsoft/wavlm-base-sd,0.095,0.40940000000000004,True,peer,False
+VehicleSoundClustering,microsoft/wavlm-base-sv,0.095,0.40940000000000004,True,peer,False
+VehicleSoundClustering,facebook/hubert-base-ls960,0.095,0.1305,True,peer,False
+VehicleSoundClustering,llm-semantic-router/Vela-1.0-Omni-Nano,0.13395648,13.800701178563807,True,nano,True
+VehicleSoundClustering,asapp/sew-d-mid-400k-ft-ls100h,0.139,0.6276,True,peer,False
+VehicleSoundClustering,laion/clap-htsat-unfused,0.153,2.6642,True,peer,False
+VehicleSoundClustering,laion/clap-htsat-fused,0.154,4.7132,True,peer,False
+VehicleSoundClustering,microsoft/msclap-2023,0.16,2.86,True,peer,False
+VehicleSoundClustering,lyrebird/wav2clip,0.163,3.9723,True,peer,False
+VehicleSoundClustering,laion/larger_clap_music_and_speech,0.194,4.7243,True,peer,False
+VehicleSoundClustering,laion/larger_clap_music,0.194,3.7272,True,peer,False
+VehicleSoundClustering,laion/larger_clap_general,0.194,3.3663,True,peer,False
+VehicleSoundClustering,microsoft/msclap-2022,0.196,7.979799999999999,True,peer,False
+VehicleSoundClustering,openai/whisper-small,0.244,1.2919999999999998,True,peer,False
+VehicleSoundClustering,microsoft/speecht5_multimodal,0.298,0.46769999999999995,True,peer,False
+VehicleSoundClustering,facebook/wav2vec2-xls-r-300m,0.3,2.3022,True,peer,False
+VehicleSoundClustering,vitouphy/wav2vec2-xls-r-300m-phoneme,0.3,1.2399,True,peer,False
+VehicleSoundClustering,facebook/data2vec-audio-large-960h,0.313,0.1542,True,peer,False
+VehicleSoundClustering,facebook/wav2vec2-lv-60-espeak-cv-ft,0.317,2.3741000000000003,True,peer,False
+VehicleSoundClustering,facebook/wav2vec2-large-xlsr-53,0.317,1.7241,True,peer,False
+VehicleSoundClustering,facebook/wav2vec2-large,0.317,1.411,True,peer,False
+VehicleSoundClustering,facebook/hubert-large-ls960-ft,0.317,0.5867,True,peer,False
+VehicleSoundClustering,microsoft/wavlm-large,0.317,0.17550000000000002,True,peer,False
+VehicleSoundClustering,OpenMuQ/MuQ-MuLan-large,0.63,0.9233999999999999,True,peer,False
+VehicleSoundClustering,openai/whisper-medium,0.769,1.0773,True,peer,False
+VehicleSoundClustering,jinaai/jina-embeddings-v5-omni-nano,0.986,1.6566,True,peer,False
+VehicleSoundClustering,facebook/mms-1b-l1107,1.0,0.4373,True,peer,False
+VehicleSoundClustering,facebook/mms-1b-all,1.0,0.32030000000000003,True,peer,False
+VehicleSoundClustering,facebook/mms-1b-fl102,1.0,0.146,True,peer,False
+VehicleSoundClustering,facebook/wav2vec2-xls-r-1b,1.0,0.1275,True,peer,False
+VehicleSoundClustering,llm-semantic-router/Vela-1.0-Omni-Mini,1.04415264,8.718116828237402,True,mini,False
+VehicleSoundClustering,speechbrain/m-ctc-t-large,1.059,0.967,True,peer,False
+VehicleSoundClustering,LanguageBind/LanguageBind_Omni,1.2,2.5643,True,peer,False
+VehicleSoundClustering,openai/whisper-large-v3,1.55,1.3834000000000002,True,peer,False
+VehicleSoundClustering,jinaai/jina-embeddings-v5-omni-small,1.626,4.8423,True,peer,False
+VehicleSoundClustering,encord-team/ebind-full,1.79,6.4883,True,peer,False
+VehicleSoundClustering,facebook/wav2vec2-xls-r-2b,2.0,0.9359,True,peer,False
+VehicleSoundClustering,facebook/seamless-m4t-v2-large,2.3,0.4163,True,peer,False
+VehicleSoundClustering,BidirLM/BidirLM-Omni-2.5B-Embedding,2.445,5.1074,True,peer,False
+VehicleSoundClustering,EximiusLabs/fusion-embedding-1-2b-preview,2.8,3.495,True,peer,False
+VehicleSoundClustering,EximiusLabs/fusion-embedding-2-2b-preview,2.8,2.4577,True,peer,False
+VehicleSoundClustering,Haon-Chen/e5-omni-3B,4.703,3.7803000000000004,True,peer,False
+VehicleSoundClustering,LCO-Embedding/LCO-Embedding-Omni-3B,4.703,2.1795,True,peer,False
+VehicleSoundClustering,Qwen/Qwen2.5-Omni-3B,5.537,1.1603,True,peer,False
+VehicleSoundClustering,Qwen/Qwen2-Audio-7B,7.0,5.518,True,peer,False
+VehicleSoundClustering,LCO-Embedding/LCO-Embedding-Omni-7B,8.932,3.6289000000000002,True,peer,False
+VehicleSoundClustering,Haon-Chen/e5-omni-7B,8.932,2.4320999999999997,True,peer,False
+VehicleSoundClustering,Tevatron/OmniEmbed-v0.1,8.932,1.4678,True,peer,False
+VehicleSoundClustering,Qwen/Qwen2.5-Omni-7B,10.732,0.8654999999999999,True,peer,False
+VehicleSoundClustering,Qwen/Qwen3-Omni-30B-A3B-Thinking,31.719,0.6526,True,peer,False
+VehicleSoundClustering,Qwen/Qwen3-Omni-30B-A3B-Instruct,35.26,1.2456,True,peer,False

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2739e88a513c5ed85fdb1bca88103aa76bb6e5adef686e7ae81d7bdbcd5bfe4f
-size 535886448
+oid sha256:3b9f58b0c4f047198bae295a3651e2932c2d5aa40821885b8f94ef61d0c09e33
+size 535886480

scores.json

@@ -182,7 +182,7 @@
     "candidate": {
       "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
       "revision": "self",
-      "artifact_fingerprint_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
+      "artifact_fingerprint_sha256": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
       "metrics": {
         "audio.audio_to_text.recall@1": {
           "denominator": 2611,
@@ -216,33 +216,33 @@
         },
         "image.image_to_text.recall@1": {
           "denominator": 823,
-          "numerator": 389,
-          "value": 0.4726609963547995
+          "numerator": 540,
+          "value": 0.6561360874848117
         },
         "image.image_to_text.recall@10": {
           "denominator": 823,
-          "numerator": 707,
-          "value": 0.8590522478736331
+          "numerator": 791,
+          "value": 0.9611178614823815
         },
         "image.image_to_text.recall@5": {
           "denominator": 823,
-          "numerator": 616,
-          "value": 0.7484811664641555
+          "numerator": 731,
+          "value": 0.8882138517618469
         },
         "image.text_to_image.recall@1": {
           "denominator": 4115,
-          "numerator": 1558,
-          "value": 0.37861482381530986
+          "numerator": 2184,
+          "value": 0.5307411907654921
         },
         "image.text_to_image.recall@10": {
           "denominator": 4115,
-          "numerator": 3420,
-          "value": 0.8311057108140948
+          "numerator": 3847,
+          "value": 0.934872417982989
         },
         "image.text_to_image.recall@5": {
           "denominator": 4115,
-          "numerator": 2898,
-          "value": 0.7042527339003645
+          "numerator": 3465,
+          "value": 0.8420413122721749
         },
         "text.banking77.accuracy": {
           "denominator": 3080,
@@ -318,15 +318,6 @@
   },
   "paired_uncertainty": {
     "original": {
-      "comparison": {
-        "candidate": "llm-semantic-router/Vela-1.0-Omni-Nano",
-        "baseline": "llm-semantic-router/multi-modal-embed-small",
-        "baseline_revision": "fdf8e01b7b0f3a69ac1ac8e2a64dcb1ede177ba4",
-        "baseline_artifact_fingerprint_sha256": "684fe0c48a4ddd3944167f6e5dc75e97ada24e5b120bd9a2921f00dc03112eeb"
-      },
-      "seed": 20260918,
-      "replicates": 2000,
-      "method": "paired stratified group percentile; PCG64; fixed candidate pools",
       "intervals": {
         "audio.audio_to_text.recall@1": {
           "lower": 0.0385732787440121,
@@ -364,9 +355,9 @@
           "upper": 0.05346161856895934
         },
         "family.image": {
-          "lower": 0.05333535844471445,
-          "point_delta": 0.07059538274605104,
-          "upper": 0.08869987849331713
+          "lower": 0.21105710814094775,
+          "point_delta": 0.23839611178614822,
+          "upper": 0.26537059538274604
         },
         "family.text": {
           "lower": 0.04221878867362225,
@@ -374,34 +365,34 @@
           "upper": 0.05518266838476903
         },
         "image.image_to_text.recall@1": {
-          "lower": 0.03523693803159174,
-          "point_delta": 0.06439854191980558,
-          "upper": 0.09477521263669501
+          "lower": 0.2065613608748481,
+          "point_delta": 0.24787363304981774,
+          "upper": 0.28797083839611176
         },
         "image.image_to_text.recall@10": {
-          "lower": 0.04738760631834751,
-          "point_delta": 0.07411907654921021,
-          "upper": 0.10085054678007291
+          "lower": 0.14702308626974483,
+          "point_delta": 0.17618469015795868,
+          "upper": 0.20534629404617255
         },
         "image.image_to_text.recall@5": {
-          "lower": 0.04738760631834751,
-          "point_delta": 0.07654921020656136,
-          "upper": 0.10692588092345079
+          "lower": 0.181044957472661,
+          "point_delta": 0.21628189550425272,
+          "upper": 0.25273390036452004
         },
         "image.text_to_image.recall@1": {
-          "lower": 0.05831713244228432,
-          "point_delta": 0.07679222357229648,
-          "upper": 0.0959902794653706
+          "lower": 0.20267314702308628,
+          "point_delta": 0.22891859052247873,
+          "upper": 0.257606318347509
         },
         "image.text_to_image.recall@10": {
-          "lower": 0.07120291616038882,
-          "point_delta": 0.0882138517618469,
-          "upper": 0.10522478736330498
+          "lower": 0.16889428918590524,
+          "point_delta": 0.1919805589307412,
+          "upper": 0.21482381530984204
         },
         "image.text_to_image.recall@5": {
-          "lower": 0.09185905224787363,
-          "point_delta": 0.11008505467800729,
-          "upper": 0.1290400972053463
+          "lower": 0.221865127582017,
+          "point_delta": 0.24787363304981774,
+          "upper": 0.27314702308626976
         },
         "text.banking77.accuracy": {
           "lower": 0.05357142857142857,
@@ -414,120 +405,139 @@
           "upper": 0.042381789402061346
         },
         "utility": {
-          "lower": 0.04888975453275108,
-          "point_delta": 0.05524241263173899,
-          "upper": 0.06188434819088739
+          "lower": 0.10172624493008055,
+          "point_delta": 0.11117598897843806,
+          "upper": 0.12072061135783345
         }
+      },
+      "method": "paired stratified group percentile; PCG64; fixed candidate pools",
+      "replicates": 2000,
+      "seed": 20260918,
+      "comparison": {
+        "candidate": "llm-semantic-router/Vela-1.0-Omni-Nano",
+        "candidate_artifact_fingerprint_sha256": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
+        "baseline": "llm-semantic-router/multi-modal-embed-small",
+        "baseline_revision": "fdf8e01b7b0f3a69ac1ac8e2a64dcb1ede177ba4",
+        "baseline_artifact_fingerprint_sha256": "684fe0c48a4ddd3944167f6e5dc75e97ada24e5b120bd9a2921f00dc03112eeb"
       }
     },
     "previous": {
-      "comparison": {
-        "candidate": "llm-semantic-router/Vela-1.0-Omni-Nano",
-        "baseline": "llm-semantic-router/Vela-1.0-Omni-Nano",
-        "baseline_revision": "38516ad893a43714b8b0d984d766c73b244dca74",
-        "baseline_artifact_fingerprint_sha256": "3c70511873758f3ee81e47f18aa094e7b24da8fb0b24a11840d312a2c70fc1b8"
-      },
-      "seed": 20260918,
-      "replicates": 2000,
-      "method": "paired stratified group percentile; PCG64; fixed candidate pools",
       "intervals": {
         "audio.audio_to_text.recall@1": {
-          "lower": 0.0023735917713692124,
-          "point_delta": 0.00804289544235925,
-          "upper": 0.014408428505559593
+          "lower": 0.0,
+          "point_delta": 0.0,
+          "upper": 0.0
         },
         "audio.audio_to_text.recall@10": {
-          "lower": 0.004368282754740709,
-          "point_delta": 0.012255840674071237,
-          "upper": 0.019943048353499093
+          "lower": 0.0,
+          "point_delta": 0.0,
+          "upper": 0.0
         },
         "audio.audio_to_text.recall@5": {
-          "lower": 0.004763400058749994,
-          "point_delta": 0.00995787054768288,
-          "upper": 0.015405387506883339
+          "lower": 0.0,
+          "point_delta": 0.0,
+          "upper": 0.0
         },
         "audio.text_to_audio.recall@1": {
-          "lower": 0.0019488287165875578,
-          "point_delta": 0.007662835249042145,
-          "upper": 0.013382222726919902
+          "lower": 0.0,
+          "point_delta": 0.0,
+          "upper": 0.0
         },
         "audio.text_to_audio.recall@10": {
-          "lower": 0.01124496486696411,
-          "point_delta": 0.017624521072796936,
-          "upper": 0.0243473303727622
+          "lower": 0.0,
+          "point_delta": 0.0,
+          "upper": 0.0
         },
         "audio.text_to_audio.recall@5": {
-          "lower": 0.008591674315911046,
-          "point_delta": 0.01685823754789272,
-          "upper": 0.02484765830564111
+          "lower": 0.0,
+          "point_delta": 0.0,
+          "upper": 0.0
         },
         "family.audio": {
-          "lower": 0.0043361091958033175,
-          "point_delta": 0.007852865345700698,
-          "upper": 0.011418895925618308
+          "lower": 0.0,
+          "point_delta": 0.0,
+          "upper": 0.0
         },
         "family.image": {
-          "lower": -0.0012150668286755773,
-          "point_delta": 0.0038882138517618466,
-          "upper": 0.008869987849331714
+          "lower": 0.14191373025516402,
+          "point_delta": 0.16780072904009719,
+          "upper": 0.19465370595382747
         },
         "family.text": {
-          "lower": 0.02648696078556764,
-          "point_delta": 0.031658466029260104,
-          "upper": 0.03632321478911417
+          "lower": 0.0,
+          "point_delta": 0.0,
+          "upper": 0.0
         },
         "image.image_to_text.recall@1": {
-          "lower": -0.007290400972053463,
-          "point_delta": 0.002430133657351154,
-          "upper": 0.012150668286755772
+          "lower": 0.1421628189550425,
+          "point_delta": 0.18347509113001215,
+          "upper": 0.22235722964763063
         },
         "image.image_to_text.recall@10": {
-          "lower": -0.006075334143377886,
-          "point_delta": 0.0,
-          "upper": 0.006075334143377886
+          "lower": 0.07776427703523693,
+          "point_delta": 0.10206561360874848,
+          "upper": 0.12636695018226002
         },
         "image.image_to_text.recall@5": {
-          "lower": -0.010935601458080195,
-          "point_delta": -0.001215066828675577,
-          "upper": 0.009720534629404616
+          "lower": 0.10692588092345079,
+          "point_delta": 0.13973268529769137,
+          "upper": 0.17253948967193194
         },
         "image.text_to_image.recall@1": {
-          "lower": 0.0014580801944106927,
-          "point_delta": 0.0053462940461725394,
-          "upper": 0.009234507897934386
+          "lower": 0.1292831105710814,
+          "point_delta": 0.15212636695018225,
+          "upper": 0.17619076549210202
         },
         "image.text_to_image.recall@10": {
-          "lower": -0.002187120291616039,
-          "point_delta": 0.001215066828675577,
-          "upper": 0.004617253948967193
+          "lower": 0.0865127582017011,
+          "point_delta": 0.10376670716889429,
+          "upper": 0.12102673147023083
         },
         "image.text_to_image.recall@5": {
-          "lower": -0.001215066828675577,
-          "point_delta": 0.0026731470230862697,
-          "upper": 0.006561360874848116
+          "lower": 0.11713244228432564,
+          "point_delta": 0.13778857837181044,
+          "upper": 0.15868772782503038
         },
         "text.banking77.accuracy": {
-          "lower": 0.02792207792207792,
-          "point_delta": 0.03538961038961039,
-          "upper": 0.042532812875659576
+          "lower": 0.0,
+          "point_delta": 0.0,
+          "upper": 0.0
         },
         "text.massive-en.accuracy": {
-          "lower": 0.02155607948804311,
-          "point_delta": 0.027927321668909825,
-          "upper": 0.03429724277067922
+          "lower": 0.0,
+          "point_delta": 0.0,
+          "upper": 0.0
         },
         "utility": {
-          "lower": 0.011830078738660701,
-          "point_delta": 0.014466515075574218,
-          "upper": 0.017106062180985208
+          "lower": 0.04730457675172134,
+          "point_delta": 0.055933576346699064,
+          "upper": 0.06488456865127583
         }
+      },
+      "method": "paired stratified group percentile; PCG64; fixed candidate pools",
+      "replicates": 2000,
+      "seed": 20260918,
+      "comparison": {
+        "candidate": "llm-semantic-router/Vela-1.0-Omni-Nano",
+        "candidate_artifact_fingerprint_sha256": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
+        "baseline": "llm-semantic-router/Vela-1.0-Omni-Nano",
+        "baseline_revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b",
+        "baseline_artifact_fingerprint_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb"
       }
     }
   },
   "training_scope": {
     "text": "Banking77 and MASSIVE English training examples and intent labels; task-adapted classification, not zero-shot transfer or standard MTEB classification",
-    "image": "COCO CC-BY2 training image-caption subset",
-    "audio": "LibriSpeech train-clean-100 speech-transcript subset"
+    "image": "Image-only alignment using all 16,509 eligible COCO training images, 81,998 canonical training-caption texts and all 82,522 positive image-caption associations; the vision tower is frozen and the image affine projection is optimized. Evaluation examples are excluded.",
+    "audio": "LibriSpeech train-clean-100 speech-transcript subset",
+    "inherited_history": {
+      "text": "Banking77 and MASSIVE English training examples and intent labels; task-adapted classification, not zero-shot transfer or standard MTEB classification",
+      "audio": "LibriSpeech train-clean-100 speech-transcript subset",
+      "previous_image_alignment": "COCO CC-BY2 training image-caption subset",
+      "source_revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b",
+      "source_native_artifact_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
+      "text_and_audio_parameters_unchanged_in_this_update": true
+    }
   },
   "descriptive_metrics": {
     "definition": "Macro-F1 is the unweighted mean of per-class F1 across all fixed prototype classes; zero_division=0.",
@@ -587,7 +597,7 @@
       "nano": {
         "repo_id": "llm-semantic-router/Vela-1.0-Omni-Nano",
         "revision": "self",
-        "native_artifact_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
+        "native_artifact_sha256": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
         "parameters": 133956480,
         "embedding_dimensions": 384,
         "max_text_tokens": 512
@@ -1797,78 +1807,78 @@
           "probe_samples_per_label": 16
         },
         "scores": {
-          "nano": 0.44469882801853367,
+          "nano": 0.9126737530662306,
           "mini": 0.9226492232215863
         },
         "metrics": {
           "nano": {
             "scores_per_experiment": [
               {
-                "accuracy": 0.4445352957209049,
-                "f1": 0.4309328513119121,
-                "f1_weighted": 0.4305399188756519,
-                "precision": 0.4391250185285425,
-                "precision_weighted": 0.43917490262148334,
-                "recall": 0.44540298862079075,
-                "recall_weighted": 0.4445352957209049,
+                "accuracy": 0.9062414826928318,
+                "f1": 0.9045301316021502,
+                "f1_weighted": 0.9050055409371512,
+                "precision": 0.9099213291951324,
+                "precision_weighted": 0.9109544817876485,
+                "recall": 0.9064295837897134,
+                "recall_weighted": 0.9062414826928318,
                 "ap": null,
                 "ap_weighted": null
               },
               {
-                "accuracy": 0.45271191060234395,
-                "f1": 0.4374624127896425,
-                "f1_weighted": 0.4376012595127548,
-                "precision": 0.45586632178451736,
-                "precision_weighted": 0.4563107134922699,
-                "recall": 0.4526852227201386,
-                "recall_weighted": 0.45271191060234395,
+                "accuracy": 0.914418097574271,
+                "f1": 0.9126724638115935,
+                "f1_weighted": 0.9132186480967779,
+                "precision": 0.9192904646518116,
+                "precision_weighted": 0.9203471940294945,
+                "recall": 0.9144593228002407,
+                "recall_weighted": 0.914418097574271,
                 "ap": null,
                 "ap_weighted": null
               },
               {
-                "accuracy": 0.4306350504224584,
-                "f1": 0.4170407359605212,
-                "f1_weighted": 0.41681774818448114,
-                "precision": 0.4342902708829311,
-                "precision_weighted": 0.43423856423244445,
-                "recall": 0.431028779615763,
-                "recall_weighted": 0.4306350504224584,
+                "accuracy": 0.9187789588443718,
+                "f1": 0.9177765846692281,
+                "f1_weighted": 0.9184111765200857,
+                "precision": 0.9220755461580387,
+                "precision_weighted": 0.9229278273528245,
+                "recall": 0.9183739103921627,
+                "recall_weighted": 0.9187789588443718,
                 "ap": null,
                 "ap_weighted": null
               },
               {
-                "accuracy": 0.46061597165440177,
-                "f1": 0.44539537134563145,
-                "f1_weighted": 0.4451308646434834,
-                "precision": 0.4575017984531489,
-                "precision_weighted": 0.45771802232058695,
-                "recall": 0.46141694456896926,
-                "recall_weighted": 0.46061597165440177,
+                "accuracy": 0.9174161896974653,
+                "f1": 0.9147622354932347,
+                "f1_weighted": 0.9154663340384108,
+                "precision": 0.9209055798643112,
+                "precision_weighted": 0.9221124471295344,
+                "recall": 0.9173606715378323,
+                "recall_weighted": 0.9174161896974653,
                 "ap": null,
                 "ap_weighted": null
               },
               {
-                "accuracy": 0.4349959116925593,
-                "f1": 0.4131091743752867,
-                "f1_weighted": 0.4126577885391281,
-                "precision": 0.4391459040033189,
-                "precision_weighted": 0.4391489734217057,
-                "recall": 0.4356331882435207,
-                "recall_weighted": 0.4349959116925593,
+                "accuracy": 0.9065140365222132,
+                "f1": 0.9051669100036687,
+                "f1_weighted": 0.9057728210054364,
+                "precision": 0.9098426973534027,
+                "precision_weighted": 0.9105143284060627,
+                "recall": 0.9059560197061645,
+                "recall_weighted": 0.9065140365222132,
                 "ap": null,
                 "ap_weighted": null
               }
             ],
-            "accuracy": 0.44469882801853367,
-            "f1": 0.4287881091565988,
-            "f1_weighted": 0.4285495159510998,
-            "precision": 0.4451858627304917,
-            "precision_weighted": 0.4453182352176981,
-            "recall": 0.44523342475383654,
-            "recall_weighted": 0.44469882801853367,
+            "accuracy": 0.9126737530662306,
+            "f1": 0.910981665115975,
+            "f1_weighted": 0.9115749041195725,
+            "precision": 0.9164071234445392,
+            "precision_weighted": 0.9173712557411129,
+            "recall": 0.9125159016452227,
+            "recall_weighted": 0.9126737530662306,
             "ap": null,
             "ap_weighted": null,
-            "main_score": 0.44469882801853367,
+            "main_score": 0.9126737530662306,
             "hf_subset": "default",
             "languages": [
               "eng-Latn"
@@ -1968,16 +1978,16 @@
           "evaluation_rows": 10000
         },
         "scores": {
-          "nano": 0.46033275813996855,
+          "nano": 0.6169193395626786,
           "mini": 0.7555458117709901
         },
         "metrics": {
           "nano": {
-            "v_measure": 0.4603327581399686,
-            "nmi": 0.46033275813996855,
-            "ari": 0.08461353562910791,
-            "cluster_accuracy": 0.2023,
-            "main_score": 0.46033275813996855,
+            "v_measure": 0.6169193395626787,
+            "nmi": 0.6169193395626786,
+            "ari": 0.21497891543638262,
+            "cluster_accuracy": 0.3638,
+            "main_score": 0.6169193395626786,
             "hf_subset": "default",
             "languages": [
               "eng-Latn"
@@ -2017,21 +2027,21 @@
           "fixed_text_labels": 37
         },
         "scores": {
-          "nano": 0.15263014445352957,
+          "nano": 0.1163804851458163,
           "mini": 0.09566639411283728
         },
         "metrics": {
           "nano": {
-            "accuracy": 0.15263014445352957,
-            "f1": 0.12154978453849745,
-            "f1_weighted": 0.12185768220651595,
-            "precision": 0.13972243713583027,
-            "precision_weighted": 0.14024267679350327,
-            "recall": 0.15235326485727774,
-            "recall_weighted": 0.15263014445352957,
+            "accuracy": 0.1163804851458163,
+            "f1": 0.085822378073852,
+            "f1_weighted": 0.08530524305774022,
+            "precision": 0.08885688520073651,
+            "precision_weighted": 0.08856851170630324,
+            "recall": 0.11709192030781942,
+            "recall_weighted": 0.1163804851458163,
             "ap": null,
             "ap_weighted": null,
-            "main_score": 0.15263014445352957,
+            "main_score": 0.1163804851458163,
             "hf_subset": "default",
             "languages": [
               "eng-Latn"
@@ -2531,9 +2541,9 @@
     },
     "comparison": {
       "baseline_model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-      "baseline_revision": "38516ad893a43714b8b0d984d766c73b244dca74",
-      "baseline_native_artifact_sha256": "3c70511873758f3ee81e47f18aa094e7b24da8fb0b24a11840d312a2c70fc1b8",
-      "card_bold": "Nano primary score strictly exceeds the corresponding previous512 release score; comparisons use unrounded values."
+      "baseline_revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b",
+      "baseline_native_artifact_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
+      "card_bold": "In the standard task table, bold marks the higher unrounded score between current Omni Nano and Mini. Baseline release differences are separately identified."
     },
     "additional_comparisons": {
       "mini": {
@@ -2577,99 +2587,101 @@
       }
     },
     "memory_scope": "Process CUDA allocator peak with the complete multimodal model loaded.",
-    "model_revision": "self",
+    "model_revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b",
     "native_artifact_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
     "scope": "Synthetic repeated text-batch measurements on one GPU; not end-to-end service latency.",
     "tf32": false,
     "token_lengths_include_special_tokens": true,
     "torch_version_family": "2.12",
     "transformers_version": "4.57.6",
-    "warmup_batches": 2
+    "warmup_batches": 2,
+    "status": "historical_previous_release_measurement",
+    "applicability": "Measured on the complete previous native artifact; not a measurement or efficiency claim for this image update."
   },
   "previous_release": {
     "model": "llm-semantic-router/Vela-1.0-Omni-Nano",
-    "revision": "38516ad893a43714b8b0d984d766c73b244dca74",
-    "native_artifact_sha256": "3c70511873758f3ee81e47f18aa094e7b24da8fb0b24a11840d312a2c70fc1b8",
+    "revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b",
+    "native_artifact_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
     "metrics": {
-      "text.banking77.accuracy": {
-        "numerator": 2257,
-        "denominator": 3080,
-        "value": 0.7327922077922078
+      "audio.audio_to_text.recall@1": {
+        "denominator": 2611,
+        "numerator": 235,
+        "value": 0.09000382995021065
       },
-      "text.massive-en.accuracy": {
-        "numerator": 1978,
-        "denominator": 2972,
-        "value": 0.6655450874831763
+      "audio.audio_to_text.recall@10": {
+        "denominator": 2611,
+        "numerator": 822,
+        "value": 0.31482190731520493
       },
-      "image.image_to_text.recall@1": {
-        "numerator": 387,
-        "denominator": 823,
-        "value": 0.47023086269744835
+      "audio.audio_to_text.recall@5": {
+        "denominator": 2611,
+        "numerator": 582,
+        "value": 0.22290310225967064
       },
-      "image.image_to_text.recall@5": {
-        "numerator": 617,
+      "audio.text_to_audio.recall@1": {
+        "denominator": 2610,
+        "numerator": 366,
+        "value": 0.14022988505747128
+      },
+      "audio.text_to_audio.recall@10": {
+        "denominator": 2610,
+        "numerator": 1055,
+        "value": 0.4042145593869732
+      },
+      "audio.text_to_audio.recall@5": {
+        "denominator": 2610,
+        "numerator": 827,
+        "value": 0.3168582375478927
+      },
+      "image.image_to_text.recall@1": {
         "denominator": 823,
-        "value": 0.7496962332928311
+        "numerator": 389,
+        "value": 0.4726609963547995
       },
       "image.image_to_text.recall@10": {
-        "numerator": 707,
         "denominator": 823,
+        "numerator": 707,
         "value": 0.8590522478736331
       },
-      "image.text_to_image.recall@1": {
-        "numerator": 1536,
-        "denominator": 4115,
-        "value": 0.3732685297691373
+      "image.image_to_text.recall@5": {
+        "denominator": 823,
+        "numerator": 616,
+        "value": 0.7484811664641555
       },
-      "image.text_to_image.recall@5": {
-        "numerator": 2887,
+      "image.text_to_image.recall@1": {
         "denominator": 4115,
-        "value": 0.7015795868772783
+        "numerator": 1558,
+        "value": 0.37861482381530986
       },
       "image.text_to_image.recall@10": {
-        "numerator": 3415,
         "denominator": 4115,
-        "value": 0.8298906439854192
-      },
-      "audio.audio_to_text.recall@1": {
-        "numerator": 214,
-        "denominator": 2611,
-        "value": 0.0819609345078514
-      },
-      "audio.audio_to_text.recall@5": {
-        "numerator": 556,
-        "denominator": 2611,
-        "value": 0.21294523171198773
+        "numerator": 3420,
+        "value": 0.8311057108140948
       },
-      "audio.audio_to_text.recall@10": {
-        "numerator": 790,
-        "denominator": 2611,
-        "value": 0.30256606664113367
-      },
-      "audio.text_to_audio.recall@1": {
-        "numerator": 346,
-        "denominator": 2610,
-        "value": 0.13256704980842912
+      "image.text_to_image.recall@5": {
+        "denominator": 4115,
+        "numerator": 2898,
+        "value": 0.7042527339003645
       },
-      "audio.text_to_audio.recall@5": {
-        "numerator": 783,
-        "denominator": 2610,
-        "value": 0.3
+      "text.banking77.accuracy": {
+        "denominator": 3080,
+        "numerator": 2366,
+        "value": 0.7681818181818182
       },
-      "audio.text_to_audio.recall@10": {
-        "numerator": 1009,
-        "denominator": 2610,
-        "value": 0.38659003831417627
+      "text.massive-en.accuracy": {
+        "denominator": 2972,
+        "numerator": 2061,
+        "value": 0.6934724091520862
       }
     },
     "descriptive_metrics": {
       "text.banking77.macro_f1": {
-        "value": 0.7295494460978302,
+        "value": 0.7658421887247635,
         "classes": 77,
         "queries": 3080
       },
       "text.massive-en.macro_f1": {
-        "value": 0.6216532764332502,
+        "value": 0.6524376423162815,
         "classes": 60,
         "queries": 2972
       }
@@ -2680,17 +2692,17 @@
         "split": "test",
         "hf_subset": "default",
         "main_metric": "cosine_spearman",
-        "score": 0.7102001265063007,
+        "score": 0.7169618925026362,
         "metrics": {
-          "pearson": 0.81667747478644,
-          "spearman": 0.7102006503028024,
-          "cosine_pearson": 0.8166774720127266,
-          "cosine_spearman": 0.7102001265063007,
-          "manhattan_pearson": 0.7898924117700169,
-          "manhattan_spearman": 0.7095017927137014,
-          "euclidean_pearson": 0.7906667280290887,
-          "euclidean_spearman": 0.7102001629803519,
-          "main_score": 0.7102001265063007,
+          "pearson": 0.8194169233518556,
+          "spearman": 0.7169617707661863,
+          "cosine_pearson": 0.819416922085493,
+          "cosine_spearman": 0.7169618925026362,
+          "manhattan_pearson": 0.7935695063285152,
+          "manhattan_spearman": 0.7160084195066224,
+          "euclidean_pearson": 0.7944328514895539,
+          "euclidean_spearman": 0.7169617703551949,
+          "main_score": 0.7169618925026362,
           "hf_subset": "default",
           "languages": [
             "eng-Latn"
@@ -2703,17 +2715,17 @@
         "split": "test",
         "hf_subset": "default",
         "main_metric": "cosine_spearman",
-        "score": 0.7696590614292106,
+        "score": 0.7792400226361769,
         "metrics": {
-          "pearson": 0.760390501991031,
-          "spearman": 0.7696591157907486,
-          "cosine_pearson": 0.7603905046470346,
-          "cosine_spearman": 0.7696590614292106,
-          "manhattan_pearson": 0.783496589683635,
-          "manhattan_spearman": 0.7680258065010287,
-          "euclidean_pearson": 0.7857109127517531,
-          "euclidean_spearman": 0.7696590614292106,
-          "main_score": 0.7696590614292106,
+          "pearson": 0.7717062133633645,
+          "spearman": 0.7792400217447207,
+          "cosine_pearson": 0.7717062156025293,
+          "cosine_spearman": 0.7792400226361769,
+          "manhattan_pearson": 0.7928914389432686,
+          "manhattan_spearman": 0.7777756255094064,
+          "euclidean_pearson": 0.7949414830254274,
+          "euclidean_spearman": 0.779239966513648,
+          "main_score": 0.7792400226361769,
           "hf_subset": "default",
           "languages": [
             "eng-Latn"
@@ -2726,157 +2738,157 @@
         "split": "test",
         "hf_subset": "default",
         "main_metric": "ndcg_at_10",
-        "score": 0.36006,
+        "score": 0.37514,
         "metrics": {
-          "ndcg_at_1": 0.16572,
-          "ndcg_at_3": 0.25775,
-          "ndcg_at_5": 0.30164,
-          "ndcg_at_10": 0.36006,
-          "ndcg_at_20": 0.39516,
-          "ndcg_at_100": 0.42562,
-          "ndcg_at_1000": 0.43592,
-          "map_at_1": 0.16572,
-          "map_at_3": 0.23435,
-          "map_at_5": 0.25868,
-          "map_at_10": 0.28293,
-          "map_at_20": 0.29267,
-          "map_at_100": 0.2973,
-          "map_at_1000": 0.2977,
-          "recall_at_1": 0.16572,
-          "recall_at_3": 0.32575,
-          "recall_at_5": 0.43243,
-          "recall_at_10": 0.61238,
-          "recall_at_20": 0.75036,
-          "recall_at_100": 0.90825,
-          "recall_at_1000": 0.98862,
-          "accuracy": 0.16572,
-          "precision_at_1": 0.16572,
-          "precision_at_3": 0.10858,
-          "precision_at_5": 0.08649,
-          "precision_at_10": 0.06124,
-          "precision_at_20": 0.03752,
-          "precision_at_100": 0.00908,
+          "ndcg_at_1": 0.17425,
+          "ndcg_at_3": 0.27012,
+          "ndcg_at_5": 0.31718,
+          "ndcg_at_10": 0.37514,
+          "ndcg_at_20": 0.40855,
+          "ndcg_at_100": 0.43759,
+          "ndcg_at_1000": 0.44701,
+          "map_at_1": 0.17425,
+          "map_at_3": 0.24561,
+          "map_at_5": 0.2715,
+          "map_at_10": 0.29556,
+          "map_at_20": 0.30489,
+          "map_at_100": 0.3093,
+          "map_at_1000": 0.30967,
+          "recall_at_1": 0.17425,
+          "recall_at_3": 0.34139,
+          "recall_at_5": 0.45661,
+          "recall_at_10": 0.63514,
+          "recall_at_20": 0.766,
+          "recall_at_100": 0.91679,
+          "recall_at_1000": 0.99004,
+          "accuracy": 0.17425,
+          "precision_at_1": 0.17425,
+          "precision_at_3": 0.1138,
+          "precision_at_5": 0.09132,
+          "precision_at_10": 0.06351,
+          "precision_at_20": 0.0383,
+          "precision_at_100": 0.00917,
           "precision_at_1000": 0.00099,
-          "mrr_at_1": 0.16571834992887624,
-          "mrr_at_3": 0.23435277382645803,
-          "mrr_at_5": 0.2586770981507824,
-          "mrr_at_10": 0.28293199214251846,
-          "mrr_at_20": 0.2926707365592407,
-          "mrr_at_100": 0.2973043609727902,
-          "mrr_at_1000": 0.2977041429811534,
-          "nauc_ndcg_at_1_max": -0.025589414103197414,
-          "nauc_ndcg_at_1_std": 0.0758456730280517,
-          "nauc_ndcg_at_1_diff1": 0.13000117977937103,
-          "nauc_ndcg_at_3_max": 0.006082481204263368,
-          "nauc_ndcg_at_3_std": 0.07900887748490223,
-          "nauc_ndcg_at_3_diff1": 0.09131312231324937,
-          "nauc_ndcg_at_5_max": 0.017593979018813478,
-          "nauc_ndcg_at_5_std": 0.08327272655564456,
-          "nauc_ndcg_at_5_diff1": 0.09399070605890802,
-          "nauc_ndcg_at_10_max": 0.03518878277702929,
-          "nauc_ndcg_at_10_std": 0.10522001461433146,
-          "nauc_ndcg_at_10_diff1": 0.09166863807789914,
-          "nauc_ndcg_at_20_max": 0.04084179140099225,
-          "nauc_ndcg_at_20_std": 0.12891546386376315,
-          "nauc_ndcg_at_20_diff1": 0.09236534565000487,
-          "nauc_ndcg_at_100_max": 0.04300692579693846,
-          "nauc_ndcg_at_100_std": 0.1291251473792114,
-          "nauc_ndcg_at_100_diff1": 0.10281159274318293,
-          "nauc_ndcg_at_1000_max": 0.030127981712603454,
-          "nauc_ndcg_at_1000_std": 0.11469932933034242,
-          "nauc_ndcg_at_1000_diff1": 0.09823111306997219,
-          "nauc_map_at_1_max": -0.025589414103197414,
-          "nauc_map_at_1_std": 0.0758456730280517,
-          "nauc_map_at_1_diff1": 0.13000117977937103,
-          "nauc_map_at_3_max": -0.002987205019881762,
-          "nauc_map_at_3_std": 0.07618300847206289,
-          "nauc_map_at_3_diff1": 0.09563386771377025,
-          "nauc_map_at_5_max": 0.0038296016020227547,
-          "nauc_map_at_5_std": 0.07833575522335894,
-          "nauc_map_at_5_diff1": 0.09678833766612233,
-          "nauc_map_at_10_max": 0.011406494352971346,
-          "nauc_map_at_10_std": 0.08776089238605031,
-          "nauc_map_at_10_diff1": 0.09634161664629624,
-          "nauc_map_at_20_max": 0.012468476212012126,
-          "nauc_map_at_20_std": 0.09422790072737758,
-          "nauc_map_at_20_diff1": 0.09698914752914077,
-          "nauc_map_at_100_max": 0.01269832929238982,
-          "nauc_map_at_100_std": 0.09449891945601069,
-          "nauc_map_at_100_diff1": 0.09841529875623733,
-          "nauc_map_at_1000_max": 0.012272846324135382,
-          "nauc_map_at_1000_std": 0.09403593115088241,
-          "nauc_map_at_1000_diff1": 0.09824637279981555,
-          "nauc_recall_at_1_max": -0.025589414103197414,
-          "nauc_recall_at_1_std": 0.0758456730280517,
-          "nauc_recall_at_1_diff1": 0.13000117977937103,
-          "nauc_recall_at_3_max": 0.029061042934453007,
-          "nauc_recall_at_3_std": 0.08664493548063527,
-          "nauc_recall_at_3_diff1": 0.08204365247469997,
-          "nauc_recall_at_5_max": 0.052822137249153106,
-          "nauc_recall_at_5_std": 0.09673185962866346,
-          "nauc_recall_at_5_diff1": 0.08923782364093243,
-          "nauc_recall_at_10_max": 0.10888598844798322,
-          "nauc_recall_at_10_std": 0.16308793670607688,
-          "nauc_recall_at_10_diff1": 0.07921838592929911,
-          "nauc_recall_at_20_max": 0.16408965305255155,
-          "nauc_recall_at_20_std": 0.29586344201677595,
-          "nauc_recall_at_20_diff1": 0.07425972289165225,
-          "nauc_recall_at_100_max": 0.3557760168694987,
-          "nauc_recall_at_100_std": 0.50751717486972,
-          "nauc_recall_at_100_diff1": 0.19682788569640491,
-          "nauc_recall_at_1000_max": 0.400179830337476,
-          "nauc_recall_at_1000_std": 0.7205544247660589,
-          "nauc_recall_at_1000_diff1": 0.11852797299522912,
-          "nauc_precision_at_1_max": -0.025589414103197414,
-          "nauc_precision_at_1_std": 0.0758456730280517,
-          "nauc_precision_at_1_diff1": 0.13000117977937103,
-          "nauc_precision_at_3_max": 0.029061042934453055,
-          "nauc_precision_at_3_std": 0.08664493548063519,
-          "nauc_precision_at_3_diff1": 0.08204365247469997,
-          "nauc_precision_at_5_max": 0.05282213724915319,
-          "nauc_precision_at_5_std": 0.09673185962866368,
-          "nauc_precision_at_5_diff1": 0.08923782364093225,
-          "nauc_precision_at_10_max": 0.10888598844798322,
-          "nauc_precision_at_10_std": 0.16308793670607719,
-          "nauc_precision_at_10_diff1": 0.07921838592929942,
-          "nauc_precision_at_20_max": 0.164089653052552,
-          "nauc_precision_at_20_std": 0.29586344201677595,
-          "nauc_precision_at_20_diff1": 0.07425972289165239,
-          "nauc_precision_at_100_max": 0.3557760168694995,
-          "nauc_precision_at_100_std": 0.5075171748697226,
-          "nauc_precision_at_100_diff1": 0.19682788569640652,
-          "nauc_precision_at_1000_max": 0.4001798303374963,
-          "nauc_precision_at_1000_std": 0.7205544247660656,
-          "nauc_precision_at_1000_diff1": 0.11852797299522304,
-          "nauc_mrr_at_1_max": -0.025589414103197414,
-          "nauc_mrr_at_1_std": 0.0758456730280517,
-          "nauc_mrr_at_1_diff1": 0.13000117977937103,
-          "nauc_mrr_at_3_max": -0.002987205019881762,
-          "nauc_mrr_at_3_std": 0.07618300847206289,
-          "nauc_mrr_at_3_diff1": 0.09563386771377025,
-          "nauc_mrr_at_5_max": 0.0038296016020227547,
-          "nauc_mrr_at_5_std": 0.07833575522335894,
-          "nauc_mrr_at_5_diff1": 0.09678833766612233,
-          "nauc_mrr_at_10_max": 0.011406494352971346,
-          "nauc_mrr_at_10_std": 0.08776089238605031,
-          "nauc_mrr_at_10_diff1": 0.09634161664629624,
-          "nauc_mrr_at_20_max": 0.012468476212012126,
-          "nauc_mrr_at_20_std": 0.09422790072737758,
-          "nauc_mrr_at_20_diff1": 0.09698914752914077,
-          "nauc_mrr_at_100_max": 0.01269832929238982,
-          "nauc_mrr_at_100_std": 0.09449891945601069,
-          "nauc_mrr_at_100_diff1": 0.09841529875623733,
-          "nauc_mrr_at_1000_max": 0.012272846324135382,
-          "nauc_mrr_at_1000_std": 0.09403593115088241,
-          "nauc_mrr_at_1000_diff1": 0.09824637279981555,
-          "hit_rate_at_1": 0.16572,
-          "hit_rate_at_3": 0.32575,
-          "hit_rate_at_5": 0.43243,
-          "hit_rate_at_10": 0.61238,
-          "hit_rate_at_20": 0.75036,
-          "hit_rate_at_100": 0.90825,
-          "hit_rate_at_1000": 0.98862,
-          "main_score": 0.36006,
+          "mrr_at_1": 0.17425320056899005,
+          "mrr_at_3": 0.24561403508771928,
+          "mrr_at_5": 0.2715030820293978,
+          "mrr_at_10": 0.2955561764772291,
+          "mrr_at_20": 0.30489450370601584,
+          "mrr_at_100": 0.3092982324964941,
+          "mrr_at_1000": 0.30966870165383564,
+          "nauc_ndcg_at_1_max": -0.012854409302840068,
+          "nauc_ndcg_at_1_std": 0.08676381700185752,
+          "nauc_ndcg_at_1_diff1": 0.11334791281377288,
+          "nauc_ndcg_at_3_max": 0.0007934267314863777,
+          "nauc_ndcg_at_3_std": 0.05484641820257593,
+          "nauc_ndcg_at_3_diff1": 0.07348303062976835,
+          "nauc_ndcg_at_5_max": 0.020607224558072373,
+          "nauc_ndcg_at_5_std": 0.0741284488966682,
+          "nauc_ndcg_at_5_diff1": 0.08730888772052957,
+          "nauc_ndcg_at_10_max": 0.035048132350077,
+          "nauc_ndcg_at_10_std": 0.10523053259120095,
+          "nauc_ndcg_at_10_diff1": 0.08020955970997308,
+          "nauc_ndcg_at_20_max": 0.04122292301118309,
+          "nauc_ndcg_at_20_std": 0.12537385022454034,
+          "nauc_ndcg_at_20_diff1": 0.0842700247341082,
+          "nauc_ndcg_at_100_max": 0.04333008607419203,
+          "nauc_ndcg_at_100_std": 0.12294409372370381,
+          "nauc_ndcg_at_100_diff1": 0.08845423517529309,
+          "nauc_ndcg_at_1000_max": 0.030127261370811845,
+          "nauc_ndcg_at_1000_std": 0.10766365950024334,
+          "nauc_ndcg_at_1000_diff1": 0.0861546357811246,
+          "nauc_map_at_1_max": -0.012854409302840068,
+          "nauc_map_at_1_std": 0.08676381700185752,
+          "nauc_map_at_1_diff1": 0.11334791281377288,
+          "nauc_map_at_3_max": -0.0029405293797002733,
+          "nauc_map_at_3_std": 0.06167764879997172,
+          "nauc_map_at_3_diff1": 0.07950290715235987,
+          "nauc_map_at_5_max": 0.00830536326209409,
+          "nauc_map_at_5_std": 0.07256833511730693,
+          "nauc_map_at_5_diff1": 0.08747148302006987,
+          "nauc_map_at_10_max": 0.015140286093876175,
+          "nauc_map_at_10_std": 0.08580662522686537,
+          "nauc_map_at_10_diff1": 0.08508762518017199,
+          "nauc_map_at_20_max": 0.0163028127500422,
+          "nauc_map_at_20_std": 0.09050711291668995,
+          "nauc_map_at_20_diff1": 0.08629712373980958,
+          "nauc_map_at_100_max": 0.016183246425002705,
+          "nauc_map_at_100_std": 0.09000154689011092,
+          "nauc_map_at_100_diff1": 0.08699050038257111,
+          "nauc_map_at_1000_max": 0.015701662616224514,
+          "nauc_map_at_1000_std": 0.0894576065792463,
+          "nauc_map_at_1000_diff1": 0.08694453879028893,
+          "nauc_recall_at_1_max": -0.012854409302840068,
+          "nauc_recall_at_1_std": 0.08676381700185752,
+          "nauc_recall_at_1_diff1": 0.11334791281377288,
+          "nauc_recall_at_3_max": 0.010338756866368223,
+          "nauc_recall_at_3_std": 0.03789278134763109,
+          "nauc_recall_at_3_diff1": 0.059403567851730316,
+          "nauc_recall_at_5_max": 0.05398176066498534,
+          "nauc_recall_at_5_std": 0.08064232472596307,
+          "nauc_recall_at_5_diff1": 0.08949514479783123,
+          "nauc_recall_at_10_max": 0.09984307699040616,
+          "nauc_recall_at_10_std": 0.17703668336587267,
+          "nauc_recall_at_10_diff1": 0.06498907396830537,
+          "nauc_recall_at_20_max": 0.15799758987163673,
+          "nauc_recall_at_20_std": 0.31157412853790695,
+          "nauc_recall_at_20_diff1": 0.07941702046000772,
+          "nauc_recall_at_100_max": 0.37265902510843735,
+          "nauc_recall_at_100_std": 0.5476974790176777,
+          "nauc_recall_at_100_diff1": 0.12977852219557848,
+          "nauc_recall_at_1000_max": 0.3004922701567,
+          "nauc_recall_at_1000_std": 0.7475055455618029,
+          "nauc_recall_at_1000_diff1": -0.05500901218402981,
+          "nauc_precision_at_1_max": -0.012854409302840068,
+          "nauc_precision_at_1_std": 0.08676381700185752,
+          "nauc_precision_at_1_diff1": 0.11334791281377288,
+          "nauc_precision_at_3_max": 0.010338756866368036,
+          "nauc_precision_at_3_std": 0.037892781347630994,
+          "nauc_precision_at_3_diff1": 0.05940356785173012,
+          "nauc_precision_at_5_max": 0.05398176066498592,
+          "nauc_precision_at_5_std": 0.08064232472596324,
+          "nauc_precision_at_5_diff1": 0.08949514479783155,
+          "nauc_precision_at_10_max": 0.09984307699040575,
+          "nauc_precision_at_10_std": 0.17703668336587192,
+          "nauc_precision_at_10_diff1": 0.06498907396830521,
+          "nauc_precision_at_20_max": 0.1579975898716373,
+          "nauc_precision_at_20_std": 0.31157412853790756,
+          "nauc_precision_at_20_diff1": 0.0794170204600081,
+          "nauc_precision_at_100_max": 0.3726590251084365,
+          "nauc_precision_at_100_std": 0.5476974790176794,
+          "nauc_precision_at_100_diff1": 0.1297785221955785,
+          "nauc_precision_at_1000_max": 0.3004922701567069,
+          "nauc_precision_at_1000_std": 0.7475055455618104,
+          "nauc_precision_at_1000_diff1": -0.055009012184042826,
+          "nauc_mrr_at_1_max": -0.012854409302840068,
+          "nauc_mrr_at_1_std": 0.08676381700185752,
+          "nauc_mrr_at_1_diff1": 0.11334791281377288,
+          "nauc_mrr_at_3_max": -0.0029405293797002733,
+          "nauc_mrr_at_3_std": 0.06167764879997172,
+          "nauc_mrr_at_3_diff1": 0.07950290715235987,
+          "nauc_mrr_at_5_max": 0.00830536326209409,
+          "nauc_mrr_at_5_std": 0.07256833511730693,
+          "nauc_mrr_at_5_diff1": 0.08747148302006987,
+          "nauc_mrr_at_10_max": 0.015140286093876175,
+          "nauc_mrr_at_10_std": 0.08580662522686537,
+          "nauc_mrr_at_10_diff1": 0.08508762518017199,
+          "nauc_mrr_at_20_max": 0.0163028127500422,
+          "nauc_mrr_at_20_std": 0.09050711291668995,
+          "nauc_mrr_at_20_diff1": 0.08629712373980958,
+          "nauc_mrr_at_100_max": 0.016183246425002705,
+          "nauc_mrr_at_100_std": 0.09000154689011092,
+          "nauc_mrr_at_100_diff1": 0.08699050038257111,
+          "nauc_mrr_at_1000_max": 0.015701662616224514,
+          "nauc_mrr_at_1000_std": 0.0894576065792463,
+          "nauc_mrr_at_1000_diff1": 0.08694453879028893,
+          "hit_rate_at_1": 0.17425,
+          "hit_rate_at_3": 0.34139,
+          "hit_rate_at_5": 0.45661,
+          "hit_rate_at_10": 0.63514,
+          "hit_rate_at_20": 0.766,
+          "hit_rate_at_100": 0.91679,
+          "hit_rate_at_1000": 0.99004,
+          "main_score": 0.37514,
           "hf_subset": "default",
           "languages": [
             "eng-Latn"
@@ -2889,41 +2901,41 @@
         "split": "test",
         "hf_subset": "default",
         "main_metric": "v_measure",
-        "score": 0.4348001342274534,
+        "score": 0.44884717457494344,
         "metrics": {
           "v_measures": {
             "Level 0": [
-              0.4585635467057098,
-              0.43396756945171217,
-              0.44789566339235937,
-              0.4282753184373477,
-              0.4285517195949066,
-              0.43240774077504984,
-              0.4373046423219837,
-              0.4260837098206918,
-              0.4238618059068785,
-              0.4310896258678948
+              0.48056098748371134,
+              0.44085942767025554,
+              0.43416151810540005,
+              0.45299559752505,
+              0.44116051339311235,
+              0.4409873172673812,
+              0.46155584917327236,
+              0.444819328389149,
+              0.44792019268133576,
+              0.44345101406076604
             ]
           },
-          "v_measure": 0.4348001342274534,
-          "v_measure_std": 0.010197493749652429,
+          "v_measure": 0.44884717457494344,
+          "v_measure_std": 0.01277114663774588,
           "ami_scores": {
             "Level 0": [
-              0.4565065557122502,
-              0.431814570933075,
-              0.44581527849521063,
-              0.4261301180441189,
-              0.426401456525955,
-              0.4302473189017929,
-              0.43518506600824775,
-              0.423921337764,
-              0.42168022600749966,
-              0.42894240579717596
+              0.4786019649327535,
+              0.43870317373726253,
+              0.43201951778515824,
+              0.4509348225877084,
+              0.43904842199622773,
+              0.43887546309559994,
+              0.45953222355170814,
+              0.44270204450063555,
+              0.44583346842612326,
+              0.4413500450061514
             ]
           },
-          "ami": 0.43266443341893257,
-          "ami_std": 0.010233181322732574,
-          "main_score": 0.4348001342274534,
+          "ami": 0.4467601145619328,
+          "ami_std": 0.012825331450796505,
+          "main_score": 0.44884717457494344,
           "hf_subset": "default",
           "languages": [
             "eng-Latn"
@@ -2936,130 +2948,130 @@
         "split": "validation",
         "hf_subset": "en",
         "main_metric": "accuracy",
-        "score": 0.858255033557047,
+        "score": 0.8758836689038031,
         "metrics": {
           "scores_per_experiment": [
             {
-              "accuracy": 0.8313199105145413,
-              "f1": 0.8372738174126213,
-              "f1_weighted": 0.8300320592088082,
-              "precision": 0.8357336674972309,
-              "precision_weighted": 0.8405164920035101,
-              "recall": 0.8491266953175365,
-              "recall_weighted": 0.8313199105145413,
+              "accuracy": 0.8527964205816555,
+              "f1": 0.8584702959865076,
+              "f1_weighted": 0.8520037076109549,
+              "precision": 0.8563814369311937,
+              "precision_weighted": 0.8604397214633738,
+              "recall": 0.8688240583945851,
+              "recall_weighted": 0.8527964205816555,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8554809843400447,
-              "f1": 0.8535646695045925,
-              "f1_weighted": 0.853473782549627,
-              "precision": 0.8506321782571107,
-              "precision_weighted": 0.8563369914824001,
-              "recall": 0.8620921919746182,
-              "recall_weighted": 0.8554809843400447,
+              "accuracy": 0.8675615212527964,
+              "f1": 0.8658660043655293,
+              "f1_weighted": 0.8661413651963479,
+              "precision": 0.8632122218181354,
+              "precision_weighted": 0.8687564991167056,
+              "recall": 0.8733845956286541,
+              "recall_weighted": 0.8675615212527964,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8389261744966443,
-              "f1": 0.8401620945367244,
-              "f1_weighted": 0.8363190993999591,
-              "precision": 0.8355843675628417,
-              "precision_weighted": 0.8381020386634399,
-              "recall": 0.8495820948585621,
-              "recall_weighted": 0.8389261744966443,
+              "accuracy": 0.8514541387024609,
+              "f1": 0.8524202289070963,
+              "f1_weighted": 0.8491348169886809,
+              "precision": 0.8478156770106394,
+              "precision_weighted": 0.8512267923319679,
+              "recall": 0.8621133428009895,
+              "recall_weighted": 0.8514541387024609,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8662192393736018,
-              "f1": 0.8681603649658488,
-              "f1_weighted": 0.8644780311580943,
-              "precision": 0.8649717052113285,
-              "precision_weighted": 0.865107286702329,
-              "recall": 0.8737130228200566,
-              "recall_weighted": 0.8662192393736018,
+              "accuracy": 0.8876957494407158,
+              "f1": 0.8889547651830694,
+              "f1_weighted": 0.886393441219644,
+              "precision": 0.8855262149097257,
+              "precision_weighted": 0.8869170708224573,
+              "recall": 0.8941464114507656,
+              "recall_weighted": 0.8876957494407158,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8559284116331096,
-              "f1": 0.8580186536753808,
-              "f1_weighted": 0.854208487179153,
-              "precision": 0.8524838648580737,
-              "precision_weighted": 0.8564209188398015,
-              "recall": 0.866676775172273,
-              "recall_weighted": 0.8559284116331096,
+              "accuracy": 0.8684563758389262,
+              "f1": 0.8708980361567344,
+              "f1_weighted": 0.8668343457695604,
+              "precision": 0.8655644216656676,
+              "precision_weighted": 0.8685796219847712,
+              "recall": 0.8789277362563833,
+              "recall_weighted": 0.8684563758389262,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8442953020134228,
-              "f1": 0.8519799885250983,
-              "f1_weighted": 0.844009283706041,
-              "precision": 0.8548973628752463,
-              "precision_weighted": 0.8648289563167956,
-              "recall": 0.8662913417133651,
-              "recall_weighted": 0.8442953020134228,
+              "accuracy": 0.8662192393736018,
+              "f1": 0.8725560012982636,
+              "f1_weighted": 0.8665613320034511,
+              "precision": 0.8737258252820442,
+              "precision_weighted": 0.8845738920578841,
+              "recall": 0.8863222925022911,
+              "recall_weighted": 0.8662192393736018,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8818791946308725,
-              "f1": 0.8846147622346404,
-              "f1_weighted": 0.8805066916739541,
-              "precision": 0.8784413012168956,
-              "precision_weighted": 0.8836692765485537,
-              "recall": 0.8951041320990551,
-              "recall_weighted": 0.8818791946308725,
+              "accuracy": 0.8953020134228188,
+              "f1": 0.8976332709955689,
+              "f1_weighted": 0.8945899154754817,
+              "precision": 0.8912735415957905,
+              "precision_weighted": 0.8979949591784757,
+              "recall": 0.908092721226657,
+              "recall_weighted": 0.8953020134228188,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8791946308724832,
-              "f1": 0.878576886474603,
-              "f1_weighted": 0.8778671414338757,
-              "precision": 0.8756405228110226,
-              "precision_weighted": 0.8814998562672293,
-              "recall": 0.8860529459183545,
-              "recall_weighted": 0.8791946308724832,
+              "accuracy": 0.9029082774049217,
+              "f1": 0.9022310464802398,
+              "f1_weighted": 0.9020525018623339,
+              "precision": 0.899983538025133,
+              "precision_weighted": 0.9036775461946326,
+              "recall": 0.9067592943176375,
+              "recall_weighted": 0.9029082774049217,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8550335570469799,
-              "f1": 0.8555845802370811,
-              "f1_weighted": 0.8543540376807164,
-              "precision": 0.8496526182989798,
-              "precision_weighted": 0.8622557102748843,
-              "recall": 0.8684081476068243,
-              "recall_weighted": 0.8550335570469799,
+              "accuracy": 0.876510067114094,
+              "f1": 0.8776887066559357,
+              "f1_weighted": 0.8761842615927375,
+              "precision": 0.8722809982436143,
+              "precision_weighted": 0.8829904469624968,
+              "recall": 0.8888620916111349,
+              "recall_weighted": 0.876510067114094,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8742729306487695,
-              "f1": 0.8758081325071294,
-              "f1_weighted": 0.8724867867286173,
-              "precision": 0.8698447272067008,
-              "precision_weighted": 0.8751813930661335,
-              "recall": 0.8856808803150957,
-              "recall_weighted": 0.8742729306487695,
+              "accuracy": 0.8899328859060402,
+              "f1": 0.8916131163462019,
+              "f1_weighted": 0.8885446252332447,
+              "precision": 0.8855482140891268,
+              "precision_weighted": 0.8923990752693544,
+              "recall": 0.9020276692969261,
+              "recall_weighted": 0.8899328859060402,
               "ap": null,
               "ap_weighted": null
             }
           ],
-          "accuracy": 0.858255033557047,
-          "f1": 0.8603743950073721,
-          "f1_weighted": 0.8567735400718846,
-          "precision": 0.8567882315795432,
-          "precision_weighted": 0.8623918920165077,
-          "recall": 0.870272822779574,
-          "recall_weighted": 0.858255033557047,
+          "accuracy": 0.8758836689038031,
+          "f1": 0.8778331472375147,
+          "f1_weighted": 0.8748440312952436,
+          "precision": 0.8741312089571069,
+          "precision_weighted": 0.8797555625382119,
+          "recall": 0.8869460213486026,
+          "recall_weighted": 0.8758836689038031,
           "ap": null,
           "ap_weighted": null,
-          "main_score": 0.858255033557047,
+          "main_score": 0.8758836689038031,
           "hf_subset": "en",
           "languages": [
             "eng-Latn"
@@ -3072,130 +3084,130 @@
         "split": "test",
         "hf_subset": "en",
         "main_metric": "accuracy",
-        "score": 0.8501595987232102,
+        "score": 0.8671910624715002,
         "metrics": {
           "scores_per_experiment": [
             {
-              "accuracy": 0.8276333789329685,
-              "f1": 0.8292069984643345,
-              "f1_weighted": 0.8264351488502784,
-              "precision": 0.8271671565367633,
-              "precision_weighted": 0.8385324225686788,
-              "recall": 0.8433324513715086,
-              "recall_weighted": 0.8276333789329685,
+              "accuracy": 0.8492932056543547,
+              "f1": 0.8506491551806685,
+              "f1_weighted": 0.8484338913731928,
+              "precision": 0.847171519646365,
+              "precision_weighted": 0.857621901965629,
+              "recall": 0.8634426315569471,
+              "recall_weighted": 0.8492932056543547,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8449612403100775,
-              "f1": 0.8378591974463067,
-              "f1_weighted": 0.8440780971724207,
-              "precision": 0.8353200304971566,
-              "precision_weighted": 0.8523929176633323,
-              "recall": 0.8512095842260071,
-              "recall_weighted": 0.8449612403100775,
+              "accuracy": 0.8593251253989969,
+              "f1": 0.8525570927270549,
+              "f1_weighted": 0.8588269042231532,
+              "precision": 0.849564509883911,
+              "precision_weighted": 0.8659425848467468,
+              "recall": 0.8649266896842414,
+              "recall_weighted": 0.8593251253989969,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8315093479252166,
-              "f1": 0.8248623215660641,
-              "f1_weighted": 0.8299143670700521,
-              "precision": 0.8196035156438577,
-              "precision_weighted": 0.8359598570726599,
-              "recall": 0.8394318669311139,
-              "recall_weighted": 0.8315093479252166,
+              "accuracy": 0.8508891928864569,
+              "f1": 0.8445077740960396,
+              "f1_weighted": 0.8500136039882616,
+              "precision": 0.83976876536009,
+              "precision_weighted": 0.8551546537983608,
+              "recall": 0.8573297788521569,
+              "recall_weighted": 0.8508891928864569,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8511171910624715,
-              "f1": 0.8485274684664849,
-              "f1_weighted": 0.8491606232900161,
-              "precision": 0.846181727605876,
-              "precision_weighted": 0.8510660377138132,
-              "recall": 0.8540509809151565,
-              "recall_weighted": 0.8511171910624715,
+              "accuracy": 0.8702690378476973,
+              "f1": 0.8680982220684261,
+              "f1_weighted": 0.8688890737378698,
+              "precision": 0.8651925230157509,
+              "precision_weighted": 0.8699395458498034,
+              "recall": 0.873034481111479,
+              "recall_weighted": 0.8702690378476973,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8511171910624715,
-              "f1": 0.8494725476798536,
-              "f1_weighted": 0.8497263968405153,
-              "precision": 0.8422528333629296,
-              "precision_weighted": 0.8527225068392907,
-              "recall": 0.860383756137512,
-              "recall_weighted": 0.8511171910624715,
+              "accuracy": 0.8657090743274054,
+              "f1": 0.8639593934313017,
+              "f1_weighted": 0.8646291417790959,
+              "precision": 0.8570893217526883,
+              "precision_weighted": 0.8672508842970594,
+              "recall": 0.8739690035000478,
+              "recall_weighted": 0.8657090743274054,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8328773369813042,
-              "f1": 0.8355190149300372,
-              "f1_weighted": 0.8353086566670657,
-              "precision": 0.8430180716537133,
-              "precision_weighted": 0.8643213236517313,
-              "recall": 0.8528748322659336,
-              "recall_weighted": 0.8328773369813042,
+              "accuracy": 0.8520291837665299,
+              "f1": 0.8534418954962738,
+              "f1_weighted": 0.8548114072639492,
+              "precision": 0.85875145236006,
+              "precision_weighted": 0.8803115046026259,
+              "recall": 0.869567932536695,
+              "recall_weighted": 0.8520291837665299,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8761969904240766,
-              "f1": 0.8727239311336351,
-              "f1_weighted": 0.8761972062275496,
-              "precision": 0.8660116640022124,
-              "precision_weighted": 0.8838501210706985,
-              "recall": 0.8878190678679512,
-              "recall_weighted": 0.8761969904240766,
+              "accuracy": 0.8869129046967624,
+              "f1": 0.8849494340089117,
+              "f1_weighted": 0.8871874824644057,
+              "precision": 0.8779834780597185,
+              "precision_weighted": 0.8955288700582439,
+              "recall": 0.9009013020211545,
+              "recall_weighted": 0.8869129046967624,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8554491564067488,
-              "f1": 0.8510088752023296,
-              "f1_weighted": 0.8546395384093688,
-              "precision": 0.8463923250152636,
-              "precision_weighted": 0.8632149564166814,
-              "recall": 0.8634681233855513,
-              "recall_weighted": 0.8554491564067488,
+              "accuracy": 0.875968992248062,
+              "f1": 0.8715420414043028,
+              "f1_weighted": 0.8753897998685723,
+              "precision": 0.86652024620862,
+              "precision_weighted": 0.8811614860860878,
+              "recall": 0.881802359118481,
+              "recall_weighted": 0.875968992248062,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8590971272229823,
-              "f1": 0.8544758706566121,
-              "f1_weighted": 0.8594473473007447,
-              "precision": 0.8479444636787655,
-              "precision_weighted": 0.8677486337715119,
-              "recall": 0.8691802239453714,
-              "recall_weighted": 0.8590971272229823,
+              "accuracy": 0.8755129958960328,
+              "f1": 0.8710791039688399,
+              "f1_weighted": 0.8760974709416987,
+              "precision": 0.8647173948263537,
+              "precision_weighted": 0.8831121050674345,
+              "recall": 0.8845140142042784,
+              "recall_weighted": 0.8755129958960328,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.8716370269037848,
-              "f1": 0.8678924632561362,
-              "f1_weighted": 0.8709158215477806,
-              "precision": 0.8605976013420696,
-              "precision_weighted": 0.8760624499751876,
-              "recall": 0.8812323255915043,
-              "recall_weighted": 0.8716370269037848,
+              "accuracy": 0.886000911992704,
+              "f1": 0.8827257199119498,
+              "f1_weighted": 0.8855721338303499,
+              "precision": 0.8760471482914741,
+              "precision_weighted": 0.8912159468319844,
+              "recall": 0.8952867706565555,
+              "recall_weighted": 0.886000911992704,
               "ap": null,
               "ap_weighted": null
             }
           ],
-          "accuracy": 0.8501595987232102,
-          "f1": 0.8471548688801794,
-          "f1_weighted": 0.8495823203375792,
-          "precision": 0.8434489389338606,
-          "precision_weighted": 0.8585871226743584,
-          "recall": 0.860298321263761,
-          "recall_weighted": 0.8501595987232102,
+          "accuracy": 0.8671910624715002,
+          "f1": 0.8643509832293768,
+          "f1_weighted": 0.8669850909470549,
+          "precision": 0.8602806359405031,
+          "precision_weighted": 0.8747239483403975,
+          "recall": 0.8764774963242037,
+          "recall_weighted": 0.8671910624715002,
           "ap": null,
           "ap_weighted": null,
-          "main_score": 0.8501595987232102,
+          "main_score": 0.8671910624715002,
           "hf_subset": "en",
           "languages": [
             "eng-Latn"
@@ -3308,18 +3320,18 @@
         "split": "test",
         "hf_subset": "default",
         "main_metric": "accuracy",
-        "score": 0.15044971381847916,
+        "score": 0.15263014445352957,
         "metrics": {
-          "accuracy": 0.15044971381847916,
-          "f1": 0.1224639288663475,
-          "f1_weighted": 0.12284286628766365,
-          "precision": 0.14022669699984341,
-          "precision_weighted": 0.14078132496483664,
-          "recall": 0.15006021593501528,
-          "recall_weighted": 0.15044971381847916,
+          "accuracy": 0.15263014445352957,
+          "f1": 0.12154978453849745,
+          "f1_weighted": 0.12185768220651595,
+          "precision": 0.13972243713583027,
+          "precision_weighted": 0.14024267679350327,
+          "recall": 0.15235326485727774,
+          "recall_weighted": 0.15263014445352957,
           "ap": null,
           "ap_weighted": null,
-          "main_score": 0.15044971381847916,
+          "main_score": 0.15263014445352957,
           "hf_subset": "default",
           "languages": [
             "eng-Latn"
@@ -3332,75 +3344,75 @@
         "split": "train",
         "hf_subset": "default",
         "main_metric": "accuracy",
-        "score": 0.2966964730605083,
+        "score": 0.29763670140167686,
         "metrics": {
           "scores_per_experiment": [
             {
-              "accuracy": 0.2860980523841504,
-              "f1": 0.26119540145049197,
-              "f1_weighted": 0.26190103173013146,
-              "precision": 0.27053921895231026,
-              "precision_weighted": 0.2706037015827094,
-              "recall": 0.2849110528508539,
-              "recall_weighted": 0.2860980523841504,
+              "accuracy": 0.2847548690396239,
+              "f1": 0.25980556646698677,
+              "f1_weighted": 0.2606862265023784,
+              "precision": 0.2685141966762838,
+              "precision_weighted": 0.26869432571348717,
+              "recall": 0.283361264324777,
+              "recall_weighted": 0.2847548690396239,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.2921423774345198,
-              "f1": 0.2859336038031679,
-              "f1_weighted": 0.28805313556173845,
-              "precision": 0.2931898724316769,
-              "precision_weighted": 0.2962545008389373,
-              "recall": 0.29186388050131323,
-              "recall_weighted": 0.2921423774345198,
+              "accuracy": 0.2981867024848892,
+              "f1": 0.2927999429509691,
+              "f1_weighted": 0.2952410831252662,
+              "precision": 0.29988591178720175,
+              "precision_weighted": 0.30336098208055257,
+              "recall": 0.29763527945216217,
+              "recall_weighted": 0.2981867024848892,
               "ap": null,
               "ap_weighted": null
             },
             {
               "accuracy": 0.28696236559139787,
-              "f1": 0.2673581628717387,
-              "f1_weighted": 0.27057166097461927,
-              "precision": 0.26595328505186505,
-              "precision_weighted": 0.2690887587298964,
-              "recall": 0.28367296994530294,
+              "f1": 0.26582489649494206,
+              "f1_weighted": 0.26920117230368773,
+              "precision": 0.26489987169540286,
+              "precision_weighted": 0.2682170436584329,
+              "recall": 0.2834616565816636,
               "recall_weighted": 0.28696236559139787,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.30376344086021506,
-              "f1": 0.2948368847698776,
-              "f1_weighted": 0.2916915113909667,
-              "precision": 0.31033490219079735,
-              "precision_weighted": 0.31214750301082883,
-              "recall": 0.3122184370413537,
-              "recall_weighted": 0.30376344086021506,
+              "accuracy": 0.29973118279569894,
+              "f1": 0.291485315935673,
+              "f1_weighted": 0.28841683151437103,
+              "precision": 0.3068565359099591,
+              "precision_weighted": 0.3086441612810119,
+              "recall": 0.30801945566008065,
+              "recall_weighted": 0.29973118279569894,
               "ap": null,
               "ap_weighted": null
             },
             {
-              "accuracy": 0.31451612903225806,
-              "f1": 0.3013530698005835,
-              "f1_weighted": 0.30264770556556814,
-              "precision": 0.30642180389648005,
-              "precision_weighted": 0.308938244875237,
-              "recall": 0.31452643284772924,
-              "recall_weighted": 0.31451612903225806,
+              "accuracy": 0.3185483870967742,
+              "f1": 0.3051858414398825,
+              "f1_weighted": 0.3066161074219095,
+              "precision": 0.310702613862649,
+              "precision_weighted": 0.3133755862555043,
+              "recall": 0.31849511838471833,
+              "recall_weighted": 0.3185483870967742,
               "ap": null,
               "ap_weighted": null
             }
           ],
-          "accuracy": 0.2966964730605083,
-          "f1": 0.28213542453917195,
-          "f1_weighted": 0.2829730090446048,
-          "precision": 0.2892878165046259,
-          "precision_weighted": 0.2914065418075218,
-          "recall": 0.2974385546373106,
-          "recall_weighted": 0.2966964730605083,
+          "accuracy": 0.29763670140167686,
+          "f1": 0.2830203126576907,
+          "f1_weighted": 0.2840322841735226,
+          "precision": 0.29017182598629926,
+          "precision_weighted": 0.2924584197977978,
+          "recall": 0.2981945548806803,
+          "recall_weighted": 0.29763670140167686,
           "ap": null,
           "ap_weighted": null,
-          "main_score": 0.2966964730605083,
+          "main_score": 0.29763670140167686,
           "hf_subset": "default",
           "languages": [
             "eng-Latn"
@@ -3413,41 +3425,41 @@
         "split": "train",
         "hf_subset": "default",
         "main_metric": "v_measure",
-        "score": 0.02582448422561181,
+        "score": 0.023601715918443712,
         "metrics": {
           "v_measures": {
             "Level 0": [
-              0.033716406453578075,
-              0.01525313637028041,
-              0.011792056157378612,
-              0.032372008152849255,
-              0.009884691587011859,
-              0.02643867559415143,
-              0.021714289160566835,
-              0.04988977832423524,
-              0.02837989050353624,
-              0.02880390995253011
+              0.014290784661453301,
+              0.02755216925549553,
+              0.017220745995259155,
+              0.02907041540588703,
+              0.017365020267924067,
+              0.022161217359367304,
+              0.02806069377818825,
+              0.017815432451092263,
+              0.03081951157077731,
+              0.03166116843899292
             ]
           },
-          "v_measure": 0.02582448422561181,
-          "v_measure_std": 0.011312419377970237,
+          "v_measure": 0.023601715918443712,
+          "v_measure_std": 0.006199268435462002,
           "ami_scores": {
             "Level 0": [
-              0.033300917175013794,
-              0.01481991910055986,
-              0.011358377860257002,
-              0.03195072875051873,
-              0.009449923872223489,
-              0.026018165306903065,
-              0.0212922270301022,
-              0.04947967973284753,
-              0.0279590804352492,
-              0.02838228371301063
+              0.01385865484522556,
+              0.027134003314929028,
+              0.016785414361355624,
+              0.028644278423611804,
+              0.016937404067902156,
+              0.02173522206231376,
+              0.0276339699482267,
+              0.0173943490723341,
+              0.030397587303172737,
+              0.031246814385090853
             ]
           },
-          "ami": 0.02540113029766855,
-          "ami_std": 0.011319790330230551,
-          "main_score": 0.02582448422561181,
+          "ami": 0.023176769778416234,
+          "ami_std": 0.0062032784119843,
+          "main_score": 0.023601715918443712,
           "hf_subset": "default",
           "languages": [
             "eng-Latn"
@@ -3460,18 +3472,18 @@
         "split": "test",
         "hf_subset": "default",
         "main_metric": "accuracy",
-        "score": 0.14678448699067256,
+        "score": 0.14972999509081983,
         "metrics": {
-          "accuracy": 0.14678448699067256,
-          "f1": 0.07272239936986488,
-          "f1_weighted": 0.07387403136789661,
-          "precision": 0.2558345838165955,
-          "precision_weighted": 0.25503167963161066,
-          "recall": 0.14375879012279777,
-          "recall_weighted": 0.14678448699067256,
+          "accuracy": 0.14972999509081983,
+          "f1": 0.06986246632697812,
+          "f1_weighted": 0.07083899432793754,
+          "precision": 0.15715381051614374,
+          "precision_weighted": 0.1573685095178851,
+          "recall": 0.14663865718054886,
+          "recall_weighted": 0.14972999509081983,
           "ap": null,
           "ap_weighted": null,
-          "main_score": 0.14678448699067256,
+          "main_score": 0.14972999509081983,
           "hf_subset": "default",
           "languages": [
             "eng-Latn"
@@ -3528,7 +3540,17 @@
           }
         }
       },
-      "report": "benchmarks/mteb-eng-v2.json"
+      "report": "benchmarks/mteb-eng-v2.json",
+      "text_component_applicability": {
+        "applies_to_revision": "self",
+        "applies_to_native_artifact_sha256": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
+        "evaluated_revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b",
+        "evaluated_native_artifact_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
+        "basis": "Exact component tensor and inference-file identity; original measured identity is retained.",
+        "report": "benchmarks/component-equivalence.json",
+        "new_full_panel_inference": false,
+        "component": "text"
+      }
     },
     {
       "benchmark": "MAEB(beta, audio-only)",
@@ -3558,11 +3580,197 @@
         "score_scale": "0-1; multiply by 100 for display"
       },
       "audio_component_applicability": {
-        "applies_to_native_artifact_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
+        "applies_to_revision": "self",
+        "applies_to_native_artifact_sha256": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
+        "evaluated_revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b",
         "evaluated_native_artifact_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
-        "basis": "Identical complete native artifact."
+        "basis": "Exact component tensor and inference-file identity; original measured identity is retained.",
+        "report": "benchmarks/component-equivalence.json",
+        "new_full_panel_inference": false,
+        "component": "audio"
       },
       "report": "benchmarks/maeb-audio-only.json"
     }
-  ]
+  ],
+  "net_benefit": {
+    "scope": "Product-weighted comparison against the previous release; not an official benchmark aggregate or SOTA claim.",
+    "baseline_model": "llm-semantic-router/Vela-1.0-Omni-Nano",
+    "baseline_revision": "70c3412bb0ddfb19386d673d4a102c753a574f1b",
+    "baseline_native_artifact_sha256": "b596f1dbab89369555e0a72a0d6854d780e305686750949b308c8f167993bdeb",
+    "candidate_native_artifact_sha256": "fb9cd39c9980a085b75456c715c2295b997784fe733314be68c13635f6384edf",
+    "score_scale": "0-100; changes in percentage points",
+    "components": [
+      {
+        "baseline_100": 54.82312157078607,
+        "candidate_100": 59.372871807049066,
+        "component": "common",
+        "delta_pp": 4.5497502362629945,
+        "weight": 0.3,
+        "evidence": "fresh_evaluation"
+      },
+      {
+        "baseline_100": 51.97553164396411,
+        "candidate_100": 51.97553164396411,
+        "component": "English41_MeanTaskType",
+        "delta_pp": 0.0,
+        "weight": 0.3,
+        "evidence": "exact_component_applicability"
+      },
+      {
+        "baseline_100": 35.255391020401056,
+        "candidate_100": 54.865785925824184,
+        "component": "image3",
+        "delta_pp": 19.610394905423124,
+        "weight": 0.2,
+        "evidence": "fresh_evaluation"
+      },
+      {
+        "baseline_100": 46.9743843608,
+        "candidate_100": 46.9743843608,
+        "component": "audio19_MeanTaskType",
+        "delta_pp": 0.0,
+        "weight": 0.2,
+        "evidence": "exact_component_applicability"
+      }
+    ],
+    "weighted_net_delta_pp": 5.287004051963523,
+    "common_definition": "Mean of three equally weighted families: the two text accuracies, all six image retrieval recalls and all six audio retrieval recalls.",
+    "image3_definition": "Equal mean of OxfordPets accuracy, TinyImageNetClustering NMI and OxfordPetsZeroShot accuracy.",
+    "text_audio_definition": "Complete English41 and audio19 mean-task-type scores, with explicit unchanged-component applicability.",
+    "coverage": {
+      "common_metrics": 14,
+      "standard_tasks": 11,
+      "standard_split_rows": 12,
+      "English_tasks": 41,
+      "audio_tasks": 19,
+      "audio_subset_split_slots": 134
+    },
+    "common_metric_delta_pp": {
+      "audio.audio_to_text.recall@1": 0.0,
+      "audio.audio_to_text.recall@10": 0.0,
+      "audio.audio_to_text.recall@5": 0.0,
+      "audio.text_to_audio.recall@1": 0.0,
+      "audio.text_to_audio.recall@10": 0.0,
+      "audio.text_to_audio.recall@5": 0.0,
+      "image.image_to_text.recall@1": 18.347509113001216,
+      "image.image_to_text.recall@10": 10.206561360874849,
+      "image.image_to_text.recall@5": 13.973268529769138,
+      "image.text_to_image.recall@1": 15.212636695018226,
+      "image.text_to_image.recall@10": 10.376670716889429,
+      "image.text_to_image.recall@5": 13.778857837181045,
+      "text.banking77.accuracy": 0.0,
+      "text.massive-en.accuracy": 0.0
+    },
+    "standard_task_deltas": [
+      {
+        "task": "SICK-R",
+        "split": "test",
+        "hf_subset": "default",
+        "main_metric": "cosine_spearman",
+        "baseline": 0.7169618925026362,
+        "candidate": 0.7169618925026362,
+        "delta_pp": 0.0
+      },
+      {
+        "task": "STSBenchmark",
+        "split": "test",
+        "hf_subset": "default",
+        "main_metric": "cosine_spearman",
+        "baseline": 0.7792400226361769,
+        "candidate": 0.7792400226361769,
+        "delta_pp": 0.0
+      },
+      {
+        "task": "ArguAna",
+        "split": "test",
+        "hf_subset": "default",
+        "main_metric": "ndcg_at_10",
+        "baseline": 0.37514,
+        "candidate": 0.37514,
+        "delta_pp": 0.0
+      },
+      {
+        "task": "TwentyNewsgroupsClustering.v2",
+        "split": "test",
+        "hf_subset": "default",
+        "main_metric": "v_measure",
+        "baseline": 0.44884717457494344,
+        "candidate": 0.44884717457494344,
+        "delta_pp": 0.0
+      },
+      {
+        "task": "MTOPDomainClassification",
+        "split": "validation",
+        "hf_subset": "en",
+        "main_metric": "accuracy",
+        "baseline": 0.8758836689038031,
+        "candidate": 0.8758836689038031,
+        "delta_pp": 0.0
+      },
+      {
+        "task": "MTOPDomainClassification",
+        "split": "test",
+        "hf_subset": "en",
+        "main_metric": "accuracy",
+        "baseline": 0.8671910624715002,
+        "candidate": 0.8671910624715002,
+        "delta_pp": 0.0
+      },
+      {
+        "task": "OxfordPets",
+        "split": "test",
+        "hf_subset": "default",
+        "main_metric": "accuracy",
+        "baseline": 0.44469882801853367,
+        "candidate": 0.9126737530662306,
+        "delta_pp": 46.797492504769686
+      },
+      {
+        "task": "TinyImageNetClustering",
+        "split": "valid",
+        "hf_subset": "default",
+        "main_metric": "nmi",
+        "baseline": 0.46033275813996855,
+        "candidate": 0.6169193395626786,
+        "delta_pp": 15.658658142271008
+      },
+      {
+        "task": "OxfordPetsZeroShot",
+        "split": "test",
+        "hf_subset": "default",
+        "main_metric": "accuracy",
+        "baseline": 0.15263014445352957,
+        "candidate": 0.1163804851458163,
+        "delta_pp": -3.6249659307713276
+      },
+      {
+        "task": "CREMA_D",
+        "split": "train",
+        "hf_subset": "default",
+        "main_metric": "accuracy",
+        "baseline": 0.29763670140167686,
+        "candidate": 0.29763670140167686,
+        "delta_pp": 0.0
+      },
+      {
+        "task": "CREMA_DClustering",
+        "split": "train",
+        "hf_subset": "default",
+        "main_metric": "v_measure",
+        "baseline": 0.023601715918443712,
+        "candidate": 0.023601715918443712,
+        "delta_pp": 0.0
+      },
+      {
+        "task": "SpeechCommandsZeroshotv0.02",
+        "split": "test",
+        "hf_subset": "default",
+        "main_metric": "accuracy",
+        "baseline": 0.14972999509081983,
+        "candidate": 0.14972999509081983,
+        "delta_pp": 0.0
+      }
+    ],
+    "tradeoff": "OxfordPets zero-shot accuracy decreases while image linear-probe accuracy, clustering NMI and cross-modal retrieval improve. All task scores and regressions remain reported."
+  }
 }