Refresh nano retrieval runtime package

README.md

@@ -49,11 +49,12 @@ Minimum board resources for the default package configuration:
 | Item | Value |
 |---|---:|
 | Additional CMM required | `1191296 KB` (`~1.14 GiB`) |
-| Linux RAM by process VmRSS | `83988 KB` (`~82 MiB`) |
-| Linux RAM by process PSS | `83280 KB` (`~81 MiB`) |
-| Process private anonymous memory | `79500 KB` (`~78 MiB`) |
-| Process VmSize | `1572852 KB` |
-| `/v1/models` ready time | `10 s` |
+| Linux RAM by process VmRSS | `84352 KB` (`~82 MiB`) |
+| Linux RAM by process PSS | `83369 KB` (`~81 MiB`) |
+| Process private memory | `83048 KB` (`~81 MiB`) |
+| Process anonymous RSS | `79456 KB` (`~78 MiB`) |
+| Process VmSize | `1572856 KB` |
+| `/v1/models` ready time | `15 s` |
 
 If both packaged audio profiles are enabled by adding `filename_audio_encoder_axmodel_long` back to `config.json`, the board resource profile is:
 
@@ -113,19 +114,34 @@ All values below were re-measured on the validated AX650 / NPU3 board.
 Each number is the average of `3` repeated direct-runtime runs after model initialization.
 These measurements exclude network overhead and client-side preprocessing outside the packaged runtime path.
 
-| Scenario | Prompt | Input tokens | Encoder avg | LLM prefill avg | Total avg |
-|---|---|---:|---:|---:|---:|
-| Text document | `document` | `19` text tokens | `-` | `68.88 ms` | `68.88 ms` |
-| Text query | `query` | `12` text tokens | `-` | `61.59 ms` | `61.59 ms` |
-| Image | `query` | `64` soft tokens, `91` total sequence | `17.24 ms` | `65.66 ms` | `82.91 ms` |
-| Video (`3` frames) | `query` | `192` soft tokens, `219` total sequence | `52.49 ms` | `148.40 ms` | `200.89 ms` |
+| Scenario | Prompt | LLM input tokens | Soft tokens | Output shape | Media encoder / prepare | Standalone encoder axmodel | Prepare overhead | LLM prefill | Runtime total |
+|---|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| Text document | `document` | `19` | `-` | `[1, 768]` | `-` | `-` | `-` | `68.88 ms` | `68.88 ms` |
+| Text query | `query` | `12` | `-` | `[1, 768]` | `-` | `-` | `-` | `61.59 ms` | `61.59 ms` |
+| Image | `query` | `91` | `64` | `[1, 768]` | `17.24 ms` | `16.42 ms` | `0.82 ms` | `65.66 ms` | `82.91 ms` |
+| Video (`3` frames) | `query` | `219` | `192` | `[1, 768]` | `52.49 ms` | `49.25 ms` | `3.24 ms` | `148.40 ms` | `200.89 ms` |
+
+Standalone encoder `.axmodel` latency was measured with `ax_run_model -r 50 -w 10` on AX650:
+
+| Encoder axmodel | Output tokens | CMM size | Pulsar2 version | Min | Max | Avg |
+|---|---:|---:|---|---:|---:|---:|
+| `jina_v5_omni_nano_vision_256x256.axmodel` | `64` | `109390730 Bytes` | `6.0 563427a6` | `16.343 ms` | `16.494 ms` | `16.416 ms` |
+| `jina_v5_omni_nano_audio_8s.axmodel` | `200` | `724467726 Bytes` | `6.0 563427a6` | `209.688 ms` | `210.452 ms` | `210.087 ms` |
+| `jina_v5_omni_nano_audio_30s.axmodel` | `750` | `1030304014 Bytes` | `6.0 563427a6` | `3783.250 ms` | `3801.552 ms` | `3792.821 ms` |
+
+`Media encoder / prepare` is the runtime media path before LLM prefill.
+For the direct-runtime rows above, image/video inputs were already prepared for the benchmark path, so the measured prepare overhead is small.
+`Standalone encoder axmodel` is the bare encoder model latency reported by `ax_run_model`; it excludes real media decode, preprocessing, tokenizer work, and LLM input assembly.
+For video rows, `Standalone encoder axmodel` is `per-frame vision encoder latency x 3`.
+`Prepare overhead` is `Media encoder / prepare - Standalone encoder axmodel`.
 
 Audio is different on this `3 GB` board:
 
 - the packaged `axllm serve` path is validated and returns correct audio embeddings
+- standalone `8s` and `30s` audio encoder `.axmodel` latency is published in the table above
 - but the current Python direct-benchmark implementation was OOM-killed during this measurement pass
 - that failure is a board-side benchmark-method issue in Linux userspace memory, not a proof that the packaged AX650 runtime path cannot run audio embeddings
-- because of that, this README does not publish a misleading direct audio split-latency number yet
+- because of that, this README does not publish a misleading full audio `prepare + LLM prefill` split-latency number yet
 
 ## Current Board Precision
 
@@ -136,7 +152,7 @@ Current board-side `axllm serve` precision against the packaged Hugging Face ref
 | Text | `embedding_doc` | `[1, 768]` | `0.999655` |
 | Text | `red_planet_query` | `[1, 768]` | `0.999539` |
 | Image | `vision_sample` | `[1, 768]` | `0.993549` |
-| Audio | `audio_test_chunk0_8s_wav` | `[1, 768]` | `0.989344` |
+| Audio | `audio_test_chunk0_8s_wav` | `[1, 768]` | `0.989693` |
 | Video | `video_visual_red_panda_openai_mp4` | `[1, 768]` | `0.997040` |
 
 Key observations:
@@ -145,8 +161,10 @@ Key observations:
 - The previous `embedding_doc = 0.9604` result was traced to an LLM build config issue, not to the OpenAI-compatible service wrapper.
 - `jina-embeddings-v5-omni-nano` uses `text_config.rope_parameters.rope_theta = 1000000.0`; the earlier `llama` build path silently fell back to `10000.0`.
 - The current LLM AXModels were rebuilt after flattening `rope_theta` into `text_config.rope_theta` for `pulsar2 llm_build`.
-- With the corrected RoPE base, the default packaged precision cases are in the `0.9893 ~ 0.9997` cosine range.
-- The default `8s` audio profile is usable but lower than the optional `30s` profile measured during adaptation: `0.989344` cosine for the packaged `audio_test_chunk0_8s_wav` case.
+- With the corrected RoPE base, the default packaged precision cases are in the `0.9896 ~ 0.9997` cosine range.
+- The default `8s` audio profile was rebuilt with a clean calibration set from `audio_test_chunk0_30s.wav` and `audio_test_chunk1_30s.wav`, rather than the earlier mixed historical calibration tar.
+- The clean-calibration `8s` audio result is still only slightly higher than the earlier mixed-calibration result (`0.989693` vs about `0.989344` cosine), so the remaining gap is not primarily explained by the audio encoder calibration tar.
+- The default `8s` audio profile is usable but lower than the optional `30s` profile measured during adaptation: `0.989693` cosine for the packaged `audio_test_chunk0_8s_wav` case.
   The previous `5s` audio profile produced only about `0.94` cosine after LLM prefill, so it is not used as a default packaged precision case.
 
 ## Board Precision Validation Flow

bin/axllm

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:42e2896a83c6012e353dbedc41efd8212553ac6c8077e2750432b8d691a3ba58
-size 2398200
+oid sha256:96cf3d2d8c9ee51c11c1d06f8cba102aa294f2055718da54a354aa45f409ef1c
+size 2406480

config.json

@@ -2,6 +2,8 @@
   "system_prompt": "",
   "model_name": "AXERA-TECH/jina-embeddings-v5-omni-nano-retrieval-AX650-P128-CTX2047",
   "is_embedding": true,
+  "jina_embedding_prompt_prefix": true,
+  "embedding_append_eos": true,
   "server_timeout_ms": 300000,
   "url_tokenizer_model": "jina_v5_omni_tokenizer.txt",
   "tokenizer_type": "Qwen3Omni",

jina_v5_omni_nano_audio_8s.axmodel

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2f00adf09f5bd58e2939ebd6855f79ac2eac0037b4b5cf2e72a1fe8d1a02d519
-size 703218785
+oid sha256:ff567748a6404d4f1479cd8256460416ab137c704b274564cd6bf7fb1e6406a6
+size 703221861