Add AX650 deployment package: text, image, video and audio

.gitattributes

@@ -33,3 +33,16 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.axmodel filter=lfs diff=lfs merge=lfs -text
+main_api_ax650 filter=lfs diff=lfs merge=lfs -text
+main_api_axcl_x86 filter=lfs diff=lfs merge=lfs -text
+main_ax650 filter=lfs diff=lfs merge=lfs -text
+main_axcl_x86 filter=lfs diff=lfs merge=lfs -text
+*.png filter=lfs diff=lfs merge=lfs -text
+*.jpg filter=lfs diff=lfs merge=lfs -text
+*.mp4 filter=lfs diff=lfs merge=lfs -text
+*.mp3 filter=lfs diff=lfs merge=lfs -text
+*.wav filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+gemma4_tokenizer.txt filter=lfs diff=lfs merge=lfs -text
+bin/axllm filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,499 @@
 ---
-license: bsd-3-clause
+library_name: transformers
+license: gemma
+base_model:
+  - google/gemma-4-E4B-it
+pipeline_tag: image-text-to-text
+tags:
+  - gemma4
+  - multimodal
+  - vision
+  - video
+  - audio
+  - axera
+  - ax650
+  - npu
+  - quantized
+  - gptq
+  - 4-bit
+  - W4A16
+  - long-context
+language:
+  - en
+  - multilingual
 ---
+
+<p align="center">
+  <img src="assets/gemma4_axera_banner.jpg" alt="Gemma4-Axera Banner">
+</p>
+
+# Gemma 4 E4B GPTQ-INT4 8K-Context on AXERA NPU
+
+Ready-to-run deployment package for the GPTQ INT4 (W4A16) conversion of
+[`google/gemma-4-E4B-it`](https://huggingface.co/google/gemma-4-E4B-it) on AX650 / NPU3,
+compiled for an extended 8K-token context.
+
+- Runtime: packaged `axllm` OpenAI-compatible server
+- Target: AX650 / AX650N, AArch64
+- Core context layout: 256-token prefill chunk, 6400-token max prefill, 8191-token decode context
+- Packaged modalities: text, single image, video, fixed-duration audio
+- Included runtime assets: `bin/axllm`, 43 text axmodel files, 3 vision axmodel files,
+  2 audio axmodel files, tokenizer, embedding weights, sample assets
+
+The text runtime extends the context window to 8191 tokens with up to 6400 tokens of prefill
+capacity, using 25 warm-prefill history buckets (256 to 6144 in 256-token steps). The model has 42
+transformer layers with sliding/full attention (512-token sliding window, full attention every 6th
+layer) and 18 KV-shared layers.
+
+## Supported Platform
+
+- AX650 / AX650N
+- NPU3 runtime environment
+- 10 GiB CMM configuration required
+
+## Compiled Runtime Profile
+
+| Setting | Value |
+| --- | ---: |
+| Transformer layers | 42 |
+| Prefill graph length | 256 tokens |
+| Warm-prefill history buckets | 256 to 6144 tokens, in 256-token steps |
+| Number of prefill groups | 25 |
+| Maximum compiled prefill capacity | 6400 tokens |
+| Maximum KV-cache length | 8191 tokens |
+| Sliding-attention window | 512 tokens |
+| Full-attention layers | 7 (every 6th layer) |
+| KV-shared layers | 18 |
+| Text quantization | W4A16 GPTQ (AutoRound), group size 128 |
+| Post stage | INT8 (s8) weights |
+| Vision / audio encoder quantization | U16 activations, FP32 conv bias |
+| Hidden-state interface | BF16 |
+| Validated server concurrency | 1 request |
+
+## Text Performance
+
+Measured on AX650 with the packaged runtime, one request at a time, greedy decoding
+(`temperature=0`). TTFT means time to first token.
+
+| Prompt tokens | Prefill chunks | TTFT (s) | Decode (tok/s) |
+| ---: | ---: | ---: | ---: |
+| 497 | 2 | 4.2 | 3.3 |
+| 1009 | 4 | 8.6 | 3.3 |
+| 2033 | 8 | 14.9 | 3.3 |
+| 4081 | 16 | 37.1 | 3.0 |
+| 5105 | 20 | 53.6 | 2.9 |
+| 6129 | 24 | 70.7 | 3.1 |
+
+Decode holds at roughly 2.9 to 3.4 tok/s and does not degrade as the KV cache fills.
+
+TTFT also depends on the prompt's content, not only its length: repeated wording is cheaper than
+highly varied text, and follow-up requests that reuse earlier wording are cheaper again. The table
+above is the slower end of that range. Image, video and audio tokens always pay full price, so a
+30 s audio clip adds 750 tokens at full cost.
+
+## Runtime Footprint
+
+CMM and Linux OS memory are separate pools on AX650. The values below were measured with the
+packaged server fully loaded on an otherwise idle board.
+
+| Item | Measured value |
+| --- | ---: |
+| CMM used, text runtime only | 6955 MiB |
+| CMM used, text + `t70` vision encoder | 7126 MiB |
+| CMM used, text + `t70` vision + both audio encoders (packaged default) | 7778 MiB |
+| CMM remaining at the packaged default | 2461 MiB |
+| OS RSS after init | 946 MiB |
+| OS RSS at peak (during 6k-token request) | 1132 MiB |
+
+CMM does not grow during inference. The KV cache is pre-allocated at initialization for the full
+8191-token context. Weights are memory-mapped (`mmap`) into page cache rather than anonymous
+memory, so the OS resident cost is small.
+
+Only one vision encoder is loaded at a time, selected by `filename_image_encoder_axmodel` in
+`config.json`; both audio encoders are loaded when both files are present. Switching the vision
+profile to `t140` or `t280` adds roughly 19 MiB or 126 MiB over the `t70` default. This package
+cannot be loaded alongside another multi-GB model on a 10 GiB CMM device.
+
+## Vision Encoder
+
+Three fixed-resolution vision encoders are packaged. Each is a fixed shape: the resolution and the
+soft-token count are compiled in and are not user-configurable at runtime. Soft tokens are the
+number of image embeddings inserted into the text sequence, equal to
+`(height / patch_size) * (width / patch_size) / 9` with `patch_size = 16` and 3x3 pooling.
+
+| Model | Resolution | Soft tokens | Encoder latency | Encoder CMM |
+| --- | --- | ---: | ---: | ---: |
+| `gemma4_vision_h336_w480_t70.axmodel` | 336x480 | 70 | 87.4 ms | 180.9 MiB |
+| `gemma4_vision_h480_w672_t140.axmodel` | 480x672 | 140 | 256.6 ms | 199.4 MiB |
+| `gemma4_vision_h672_w960_t280.axmodel` | 672x960 | 280 | 744.3 ms | 306.4 MiB |
+
+Latency is a single-model average from `ax_run_model -w 1 -r 5` on AX650, excluding text prefill
+and decode. The package default is the `t70` profile. To switch profiles, edit `config.json`:
+
+```json
+{
+  "filename_image_encoder_axmodel": "gemma4_vision_h480_w672_t140.axmodel",
+  "vision_width": 672,
+  "vision_height": 480
+}
+```
+
+Change the resolution pair together with the encoder file. A mismatch makes the runtime preprocess
+images to a shape the encoder was not compiled for.
+
+The runtime caches image embeddings under `vision_cache/` keyed by the media path. Delete that
+directory after switching profiles, otherwise an entry produced by the previous encoder is reused.
+
+Both the vision and the audio encoders are converted from the unquantized upstream weights, because
+the source INT4 checkpoint leaves both encoder towers in full precision.
+
+Higher profiles resolve finer detail at a higher latency cost. Use `t70` for the fastest response
+and `t140` or `t280` when small features such as text in the image matter.
+
+## Audio Encoder
+
+Two fixed-duration audio encoders are packaged. The duration is compiled in, and `axllm` selects
+the profile from the clip length. Audio longer than 30 s is truncated to the 30 s window.
+
+| Model | Audio duration | Mel frames | Audio tokens | Encoder latency | Encoder CMM |
+| --- | --- | ---: | ---: | ---: | ---: |
+| `gemma4_audio_5s.axmodel` | 5 s | 499 | 125 | 29.5 ms | 335.7 MiB |
+| `gemma4_audio_30s.axmodel` | 30 s | 2999 | 750 | 173.1 ms | 360.5 MiB |
+
+Latency is a single-model average from `ax_run_model -w 1 -r 5` on AX650, excluding text prefill
+and decode. Both encoders are loaded at startup when both files are present.
+
+### Audio Input Format
+
+Gemma 4's audio front end is defined at **16 kHz mono** (`sampling_rate: 16000` in
+`gemma_4_e4b_it_tokenizer/processor_config.json`). Supply audio at that rate and the runtime feeds it straight to the mel
+front end. Any other rate is resampled on the CPU first, and that resampling — not the NPU encoder —
+dominates the time before the first token. Measured on AX650 for a 30 s clip:
+
+| Input | Feature extraction (read + resample + mel) | NPU encode |
+| --- | ---: | ---: |
+| 16 kHz mono (native) | 0.94 s | 0.18 s |
+| 48 kHz stereo | 2.6 s | 0.18 s |
+| 44.1 kHz stereo | 3.0 s | 0.18 s |
+
+Convert once before sending, for example:
+
+```bash
+ffmpeg -i input.wav -ar 16000 -ac 1 -c:a pcm_s16le input_16k.wav
+```
+
+The packaged sample clips are already 16 kHz mono.
+
+**WAV only.** The audio path accepts uncompressed `RIFF/WAVE` PCM. Compressed containers are
+rejected with `unsupported wav container (expect RIFF/WAVE)`, even when `ffmpeg` is installed on the
+board, so decode `mp3 / flac / m4a / ogg` to 16 kHz mono WAV first with the command above.
+
+A 30 s clip's 750 audio tokens span several prefill chunks, so attention within the audio block is
+not fully bidirectional across chunk boundaries.
+
+## Video
+
+Video is handled by the same vision encoder, one frame at a time, using the frame count in
+`gemma_4_e4b_it_tokenizer/processor_config.json` (`num_frames: 32`). At the default `t70` profile that is
+32 x 70 = 2240 soft tokens per clip, so a video request costs about 2360 input tokens including the
+text part and uses 10 prefill chunks.
+
+Only the picture is used. A video's audio track is ignored; audio is a separate input, so send it as
+its own `audio_url` part if you need both.
+
+## Package Layout
+
+```text
+.
+├── README.md
+├── bin/axllm
+├── serve.sh
+├── config.json
+├── post_config.json
+├── gemma4_tokenizer.txt
+├── assets/
+│   ├── gemma4_axera_banner.jpg
+│   ├── sample.png
+│   ├── red-panda-openai.mp4
+│   ├── gemma4_audio_test_5s.wav
+│   ├── gemma4_audio_test_chunk0_30s.wav
+│   └── gemma4_audio_test_chunk1_30s.wav
+├── gemma4_text_p256_l0_together.axmodel ... gemma4_text_p256_l41_together.axmodel
+├── gemma4_text_post.axmodel
+├── gemma4_vision_h336_w480_t70.axmodel
+├── gemma4_vision_h480_w672_t140.axmodel
+├── gemma4_vision_h672_w960_t280.axmodel
+├── gemma4_audio_5s.axmodel
+├── gemma4_audio_30s.axmodel
+├── model.embed_tokens.weight.bfloat16.bin
+├── model.embed_tokens_per_layer.weight.npy
+├── model.per_layer_model_projection.weight.npy
+├── model.per_layer_projection_norm.weight.npy
+└── gemma_4_e4b_it_tokenizer/
+    ├── config.json
+    ├── tokenizer.json
+    ├── tokenizer_config.json
+    ├── generation_config.json
+    ├── processor_config.json
+    └── chat_template.jinja
+```
+
+The root directory is the direct `axllm` runtime layout: all axmodel files, the embedding weights,
+and the tokenizer text file live at the root. The tokenizer subdirectory holds the Hugging Face
+tokenizer and model metadata without source weight shards.
+
+## Sample Assets
+
+The package ships the media used in the examples below.
+
+`assets/sample.png`
+
+![sample](assets/sample.png)
+
+Three WAV clips of English speech at 16 kHz mono, one per audio profile length:
+
+- `assets/gemma4_audio_test_5s.wav` (5.00 s)
+- `assets/gemma4_audio_test_chunk0_30s.wav` (30.00 s)
+- `assets/gemma4_audio_test_chunk1_30s.wav` (30.00 s)
+
+One video clip for the video example:
+
+- `assets/red-panda-openai.mp4` (30.1 s, 360x640)
+
+## Download
+
+```bash
+mkdir -p AXERA-TECH/gemma-4-E4B-it-GPTQ-INT4-P6K-C8K
+cd AXERA-TECH/gemma-4-E4B-it-GPTQ-INT4-P6K-C8K
+hf download AXERA-TECH/gemma-4-E4B-it-GPTQ-INT4-P6K-C8K --local-dir .
+```
+
+## Packaged Runtime
+
+The package ships a prebuilt AArch64 `bin/axllm`, so no separate install step is needed, and the
+figures in this README refer to that binary. Replacing it with a different build of
+[`AXERA-TECH/ax-llm`](https://github.com/AXERA-TECH/ax-llm) may change the TTFT figures.
+
+## Run on the Board
+
+From the package root:
+
+```bash
+chmod +x ./bin/axllm ./serve.sh
+./serve.sh --port 8000
+```
+
+Or equivalently:
+
+```bash
+chmod +x ./bin/axllm
+./bin/axllm serve . --port 8000
+```
+
+Check service health and the loaded model:
+
+```bash
+curl http://127.0.0.1:8000/health
+curl http://127.0.0.1:8000/v1/models
+```
+
+The health endpoint returns `"status": "healthy"`, and the model list contains
+`AXERA-TECH/gemma-4-E4B-it-GPTQ-INT4-P6K-C8K`.
+
+**Server timeout:** the default `--server_timeout_ms` is 300 seconds, which covers a 6000-token
+prompt (about 71 s of TTFT) plus a few hundred output tokens. A long prompt combined with a long
+generation can still exceed it, since decode runs at roughly 3 tok/s. Raise it when you need both,
+and raise the client-side HTTP timeout to match:
+
+```bash
+./serve.sh --port 8000 --server_timeout_ms 600000
+```
+
+## Text Request
+
+```bash
+curl http://127.0.0.1:8000/v1/chat/completions \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "AXERA-TECH/gemma-4-E4B-it-GPTQ-INT4-P6K-C8K",
+    "messages": [
+      {"role": "user", "content": "What is the capital of France?"}
+    ],
+    "max_tokens": 64,
+    "temperature": 0,
+    "stream": false
+  }'
+```
+
+Measured response:
+
+```text
+The capital of France is **Paris**.
+```
+
+## Image Request
+
+Pass the image as a board-side file path or as a base64 `data:` URI.
+
+```bash
+curl http://127.0.0.1:8000/v1/chat/completions \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "AXERA-TECH/gemma-4-E4B-it-GPTQ-INT4-P6K-C8K",
+    "messages": [
+      {"role": "user", "content": [
+        {"type": "image_url", "image_url": {"url": "./assets/sample.png"}},
+        {"type": "text", "text": "What animal is in this image and what colour is it? Answer in one sentence."}
+      ]}
+    ],
+    "max_tokens": 96,
+    "temperature": 0,
+    "stream": false
+  }'
+```
+
+Measured response with the default `t70` profile:
+
+```text
+The animal in the image is a red lobster.
+```
+
+One image per message is supported. Mixing image and audio in the same message is not supported.
+
+## Audio Request
+
+Pass the audio as a board-side file path or as a base64 `data:` URI. The clip length selects the
+encoder profile.
+
+```bash
+curl http://127.0.0.1:8000/v1/chat/completions \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "AXERA-TECH/gemma-4-E4B-it-GPTQ-INT4-P6K-C8K",
+    "messages": [
+      {"role": "user", "content": [
+        {"type": "audio_url", "audio_url": {"url": "./assets/gemma4_audio_test_5s.wav"}},
+        {"type": "text", "text": "Transcribe the speech in this audio in its original language. Output only the transcription."}
+      ]}
+    ],
+    "max_tokens": 128,
+    "temperature": 0,
+    "stream": false
+  }'
+```
+
+Measured response for `assets/gemma4_audio_test_5s.wav` (5 s profile, 125 audio tokens):
+
+```text
+When I was seventeen, I read a quote that went something like, if you
+```
+
+Measured response for `assets/gemma4_audio_test_chunk0_30s.wav` (30 s profile, 750 audio tokens):
+
+```text
+No one wants to die. Even people who want to go to heaven don't want to die to get there. And yet,
+death is the destination we all share. No one has ever escaped it, and that is as it should be
+because death is very likely the single best invention of life. It's life's change agent. It clears
+out the old to make way for the new. Right now, the new is you. But someday, not too long from now,
+you will gradually become
+```
+
+End to end on AX650 with the packaged clips this takes about 9 s for the 5 s clip and 34 to 44 s for
+a 30 s clip, almost all of it text decode rather than audio processing. Both transcriptions above
+stop at the `max_tokens` limit, not at a sentence boundary.
+
+One audio file per message is supported. Mixing audio and image in the same message is not
+supported.
+
+## Video Request
+
+```bash
+curl http://127.0.0.1:8000/v1/chat/completions \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "AXERA-TECH/gemma-4-E4B-it-GPTQ-INT4-P6K-C8K",
+    "messages": [
+      {"role": "user", "content": [
+        {"type": "video_url", "video_url": {"url": "./assets/red-panda-openai.mp4"}},
+        {"type": "text", "text": "What animal is in this video? Answer with the animal name only."}
+      ]}
+    ],
+    "max_tokens": 48,
+    "temperature": 0,
+    "stream": false
+  }'
+```
+
+Measured response for the packaged 30 s clip at the default `t70` profile:
+
+```text
+Red Panda
+```
+
+A video request of this length costs about 2360 input tokens and takes roughly 40 s end to end.
+
+## Multi-turn Conversation
+
+Send the whole message list each turn, as an OpenAI client normally does. The runtime keeps the KV
+cache from the previous turns and only processes the new tokens, so follow-up turns are far cheaper
+than the first. With a 3223-token document in the opening message, the first turn takes about 22 s to
+the first token and each short follow-up about 3 s, with the earlier turns still available to the
+model.
+
+## Context Limits
+
+| Limit | Value | Behaviour |
+| --- | ---: | --- |
+| Input tokens per request | 6400 | Above it the server returns HTTP 400 with `input_num_token(N) > prefill_max_token_num(6400)` |
+| Total context (prompt + generation + history) | 8191 | KV cache is pre-allocated for this at startup |
+
+A request over the 6400-token cap is rejected outright rather than silently truncated, so keep
+prompts within the cap.
+
+## Transcription Endpoint
+
+The OpenAI-compatible `/v1/audio/transcriptions` and `/v1/audio/translations` endpoints are also
+available. They take a multipart upload, so the audio file does not have to exist on the board, and
+they work with any OpenAI-compatible client or web UI without extra parameters.
+
+```bash
+curl http://127.0.0.1:8000/v1/audio/transcriptions \
+  -F model=AXERA-TECH/gemma-4-E4B-it-GPTQ-INT4-P6K-C8K \
+  -F file=@./assets/gemma4_audio_test_5s.wav
+```
+Measured response:
+
+```json
+{
+  "text": "When I was seventeen, I read a quote that went something like, if you"
+}
+```
+
+The 30 s clips select the 750-token encoder through the same endpoint. Measured response for
+`assets/gemma4_audio_test_chunk0_30s.wav`:
+
+```json
+{
+  "text": "No one wants to die. Even people who want to go to heaven don't want to die to get there. And yet, death is the destination we all share. No one has ever escaped it, and that is as it should be because death is very likely the single best invention of life. It's life's change agent. It clears out the old to make way for the new. Right now, the new is you. But someday, not too long from now, you will gradually become"
+}
+```
+
+`response_format` accepts `json` (default), `verbose_json`, `text`, `srt` and `vtt`. The response is
+always wrapped as `{"text": ...}`; `srt` and `vtt` put the same text in a single subtitle cue
+spanning the whole clip, so there is no word-level or segment-level timing.
+
+`/v1/audio/translations` follows the OpenAI definition and translates into English. With the packaged
+English clips its output is the same as the transcription.
+
+## Conversion References
+
+If you need the original model files or want to rebuild the deployment artifacts, start with:
+
+- Original Hugging Face model: [`google/gemma-4-E4B-it`](https://huggingface.co/google/gemma-4-E4B-it)
+- Source INT4 checkpoint for the text weights: [`Vishva007/gemma-4-E4B-it-W4A16-AutoRound-GPTQ`](https://huggingface.co/Vishva007/gemma-4-E4B-it-W4A16-AutoRound-GPTQ)
+- AXERA conversion and deployment workflow: [`AXERA-TECH/gemma-4-E2B-it.axera`](https://github.com/AXERA-TECH/gemma-4-E2B-it.axera)
+
+## Discussion
+
+- [GitHub Issues](https://github.com/AXERA-TECH/ax-llm/issues)
+- QQ group: `139953715`

assets/gemma4_audio_test_5s.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9452e13af6d8a754f7e6fae143f232cff88b0e4467a21f3c48f6e624fee9f191
+size 160294

assets/gemma4_audio_test_chunk0_30s.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75c6f4594fb977e8c762f79ea6d2b10cc1fc95037a5a1d26f7fe11351efa9e66
+size 960294

assets/gemma4_audio_test_chunk1_30s.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61aeb20520ef395a83b3ac06f21d128c5572c90cae3f211287cb1b302bbb56b5
+size 960294

assets/red-panda-openai.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d921c07bb97224d65a37801541d246067f0d506f08723ffa1ad85c217907ccb8
+size 1867237

bin/axllm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01c1948e316d96372fecab0968e21c8cff608ef80222a1cc6720161153bff913
+size 2507312

config.json

@@ -0,0 +1,135 @@
+{
+  "mem_guard_on_unsafe": "warn",
+  "system_prompt": "You are a helpful assistant.",
+  "model_name": "AXERA-TECH/gemma-4-E4B-it-GPTQ-INT4-P6K-C8K",
+  "url_tokenizer_model": "gemma4_tokenizer.txt",
+  "tokenizer_type": "Gemma4VL",
+  "post_config_path": "post_config.json",
+  "template_filename_axmodel": "gemma4_text_p256_l%d_together.axmodel",
+  "axmodel_num": 42,
+  "filename_post_axmodel": "gemma4_text_post.axmodel",
+  "filename_tokens_embed": "model.embed_tokens.weight.bfloat16.bin",
+  "tokens_embed_num": 262144,
+  "tokens_embed_size": 2560,
+  "text_config": {
+    "hidden_size": 2560,
+    "num_hidden_layers": 42,
+    "num_key_value_heads": 2,
+    "head_dim": 256,
+    "global_head_dim": 512,
+    "num_kv_shared_layers": 18,
+    "layer_types": [
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention"
+    ],
+    "sliding_window": 512
+  },
+  "pad_token_id": 0,
+  "hidden_size_per_layer_input": 256,
+  "rms_norm_eps": 1e-06,
+  "filename_tokens_embed_per_layer": "model.embed_tokens_per_layer.weight.npy",
+  "filename_per_layer_model_projection": "model.per_layer_model_projection.weight.npy",
+  "filename_per_layer_projection_norm": "model.per_layer_projection_norm.weight.npy",
+  "use_mmap_load_embed": true,
+  "use_mmap_load_layer": true,
+  "vlm_type": "Gemma4VL",
+  "filename_image_encoder_axmodel": "gemma4_vision_h336_w480_t70.axmodel",
+  "vision_width": 480,
+  "vision_height": 336,
+  "vision_patch_size": 16,
+  "vision_cache_dir": "vision_cache",
+  "filename_audio_encoder_axmodel_5s": "gemma4_audio_5s.axmodel",
+  "filename_audio_encoder_axmodel_30s": "gemma4_audio_30s.axmodel",
+  "devices": [
+    0
+  ],
+  "global_head_dim": 512,
+  "head_dim": 256,
+  "num_key_value_heads": 2,
+  "layer_types": [
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention"
+  ],
+  "sliding_window": 512,
+  "num_kv_shared_layers": 18
+}

gemma4_audio_30s.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:faa73b53f768934a22e99cb4eca78f6d7d4aa25153b7cd74349421802ccf0cdb
+size 328904141

gemma4_audio_5s.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76cb423cde2a1a9561e02b2e717e1922e3257bdfa55c59f71a34edb12d460f64
+size 323406137

gemma4_text_p256_l0_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0f044da22c401c7626dc46b08080fbabd707cdfb11e5b5cf94438725c4ed42f
+size 109942098

gemma4_text_p256_l10_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a0fb3cdb8ead3b1067f9e744196ce27c764e7404e92f04478f78445acb97970
+size 109942162

gemma4_text_p256_l11_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:464bc239946c9269586f4170adf7d22241266eded39b43e245e430533952fb3c
+size 145877586

gemma4_text_p256_l12_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2ae9f4df65792f811f038277457c8c208bc96bae497aae5249d025bf5072ce66
+size 109942162

gemma4_text_p256_l13_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7427d88e958793c580875a04e0a7ac2e39f9f6664dba78b4c5c8be027a40d10b
+size 109942162

gemma4_text_p256_l14_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99ed8082d5488c332932d5fa49de38e56a9ede99ea329fa89331c7f406d16431
+size 109942162

gemma4_text_p256_l15_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51f3fb4a0010977ab7a3dd2b3ec0a4e68a3d6af25c549a08dcfe78473b4d75b7
+size 109942162

gemma4_text_p256_l16_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63f46551e537112e341383c2992bf4100efaeb3cd81084bbd0a05a6dce3a8759
+size 109942226

gemma4_text_p256_l17_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:532f31f1c1d2b284dc8f07056d70c595c3046420bcc4b2573ee4ed158de720eb
+size 145877586

gemma4_text_p256_l18_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6bd269afeb7cac90956fb0fe3d647dd6bed7771a5359aed97e8a91de453e119d
+size 109942162

gemma4_text_p256_l19_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a6503ce847bce6f9759cab57357758410f48a87981edda561f2d9ae5e0ae0a2
+size 109942162

gemma4_text_p256_l1_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:100c0d172c3cc36a2333f72521482d59fa415144ab5ee35bd5ac5ce58863e498
+size 109942098

gemma4_text_p256_l20_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e6a7d3b4177072109dc0bd50b5fb22c50067d3a9a64d198a62de480c4c4232e
+size 109835090

gemma4_text_p256_l21_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bbe4ade8e7c1fc925cffbe0d04b9ceb3e476af4712a11bf5d71bb7164eba049c
+size 109942226

gemma4_text_p256_l22_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6e40c808b4f2b5e61f7a50d616d052714916288f95f4f0268302dc3d562ef67
+size 109942226

gemma4_text_p256_l23_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa47ad388ab02d7943c9590a4af56fe1b1bcc1b75284967c23041bb616538b13
+size 145877586

gemma4_text_p256_l24_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:011d77636eacd258ce15550a51b531aa0f994fbd0b912428ad3a6aed5d812800
+size 106277970

gemma4_text_p256_l25_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a8cc30b1dfe9c71479ce1ebc264d769b1e9f2a04db34b9feead84d6e7181217
+size 106277874

gemma4_text_p256_l26_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b51f68af83144f90d69cfc2a7009f157a7ab21696f812e6f34a3aedd1c6a066
+size 106277906

gemma4_text_p256_l27_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a38e83f4ff7463514e607686d7689406dee842216818982fc796eb55d927805
+size 106277842

gemma4_text_p256_l28_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:310d49ff1aeb6d867b047228bcf574dc6fb70d6b94ce67e8ab66fe03fcd8c6d7
+size 106277874

gemma4_text_p256_l29_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f0a8c75957fcde4d65ee02a121e4b8b957c229a38c3f0a67681a271e00284472
+size 139733374

gemma4_text_p256_l2_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad21515b7bd7c1a7bf8f309d645216c1c7c4a4c87a54e2719cde0141ec44f9bf
+size 109942034

gemma4_text_p256_l30_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b6085303f3aa848c268e059b6ae872f2ecce0fe65a4a762a67439277dc49acf
+size 106278098

gemma4_text_p256_l31_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:420c775da629a1d37001e58f1ee6db63a91166e6959b89418c0ff8b78c0b4332
+size 106278098

gemma4_text_p256_l32_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70756514579c6de17a29292ed99df5d4d464003a9ea555c158af5fa8a37e3a59
+size 106278066

gemma4_text_p256_l33_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49f26b532014582336b14f962bce84f0c506047af695bba9a61417a7ee44704c
+size 106278066

gemma4_text_p256_l34_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e86efe91e1dc0a31ecb65dc35ce124f17658f13b467431f78346dff332e044c1
+size 106278066

gemma4_text_p256_l35_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4adbb0bef290ffd954bc37b97ce8c4f86fe1b921b80f791f2d3b558b55e0e356
+size 139733918

gemma4_text_p256_l36_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ac7f6335bc3d38d5360fe2a5451fc08e6152aff07a09201378ede79807800d5
+size 106278066

gemma4_text_p256_l37_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da17bae13c5a3c9f662ff4dd8bcafb663ab30de468ee7387dd05b06316c17a19
+size 106278066

gemma4_text_p256_l38_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a96d871d8cb2b0ade211b552b268a7b795d484db33390b60bfb0878efd685d9d
+size 106278066

gemma4_text_p256_l39_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:303914b63b2be4f7fd65410518905b4996863e73502aa963f43b27dded54a88e
+size 106278066

gemma4_text_p256_l3_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cad8098eb26b4a06e00f1b7c81a46d637e5ccab1c34f31bacb25b8f5d8d148d2
+size 109942034

gemma4_text_p256_l40_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d31bbb42fd9a9e2fe10628509516243ac284ccc093fbbee692d9bf90e7df614f
+size 106278066

gemma4_text_p256_l41_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7bd5234f9af81382993a139370ca431d8652005ec5ef8392e243d347dfdf374e
+size 139733374

gemma4_text_p256_l4_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:307c26b63eb8edbfc8b7a323e08d7be1154ac790d0b15769761c89f22adfee59
+size 109942034

gemma4_text_p256_l5_together.axmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1b91b097b3b4ae5b1f3b0d7b1cdd5cf828b7cbf207bcbd15ea86958990006df
+size 145877466