Any-to-Any
Transformers
Safetensors
English
Māori
abteex-ai-labs
aotearoa
gemma
legacy
lumynax
sovereign-ai
outdated
Instructions to use AbteeXAILab/lumynax-infused-gemma-e4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AbteeXAILab/lumynax-infused-gemma-e4b with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("AbteeXAILab/lumynax-infused-gemma-e4b", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Publish LumynaX Infused Gemma E4B Model
Browse files- .gitattributes +1 -0
- LICENSE.txt +11 -0
- README.md +86 -0
- VERSION.txt +1 -0
- __pycache__/quickstart.cpython-311.pyc +0 -0
- artifacts/release_training_summary.json +17 -0
- checksums.sha256 +21 -0
- hf_space/README.md +33 -0
- hf_space/__pycache__/app.cpython-311.pyc +0 -0
- hf_space/app.py +267 -0
- hf_space/requirements.txt +10 -0
- merged_model/LUMYNAX_PACKAGE_IDENTITY.txt +1 -0
- merged_model/chat_template.jinja +347 -0
- merged_model/config.json +197 -0
- merged_model/generation_config.json +14 -0
- merged_model/model.safetensors +3 -0
- merged_model/processor_config.json +75 -0
- merged_model/tokenizer.json +3 -0
- merged_model/tokenizer_config.json +74 -0
- ollama/Modelfile +11 -0
- ollama/create_ollama_model.ps1 +21 -0
- quickstart.py +110 -0
- release_export_manifest.json +79 -0
- requirements.txt +9 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
merged_model/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
LICENSE.txt
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
LumynaX Infused Gemma E4B Model
|
| 2 |
+
Copyright (c) AbteeX AI Labs. All rights reserved.
|
| 3 |
+
|
| 4 |
+
This release is proprietary. No right to use, copy, modify, distribute, host,
|
| 5 |
+
sublicense, reverse engineer, or create derivative releases is granted except
|
| 6 |
+
under a separate written agreement from AbteeX AI Labs.
|
| 7 |
+
|
| 8 |
+
This package may be used only by parties expressly authorized by AbteeX AI Labs.
|
| 9 |
+
|
| 10 |
+
Third-party software or model components, if any, remain subject to their own
|
| 11 |
+
licenses and obligations.
|
README.md
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: proprietary
|
| 4 |
+
license_link: https://huggingface.co/AbteeXAILab/lumynax-infused-gemma-e4b/resolve/main/LICENSE.txt
|
| 5 |
+
library_name: transformers
|
| 6 |
+
pipeline_tag: image-text-to-text
|
| 7 |
+
language:
|
| 8 |
+
- en
|
| 9 |
+
- mi
|
| 10 |
+
tags:
|
| 11 |
+
- lumynax
|
| 12 |
+
- lumynax-infused-gemma-e4b
|
| 13 |
+
- gemma-4-e4b
|
| 14 |
+
- multimodal
|
| 15 |
+
- reasoning
|
| 16 |
+
- audio
|
| 17 |
+
- image
|
| 18 |
+
- abteex-ai-labs
|
| 19 |
+
- local-first
|
| 20 |
+
---
|
| 21 |
+
|
| 22 |
+
# LumynaX Infused Gemma E4B Model
|
| 23 |
+
|
| 24 |
+
LumynaX Infused Gemma E4B Model is a real local release package built around the official
|
| 25 |
+
`google/gemma-4-E4B-it` checkpoint, laid out in the same top-level structure as the checked-in
|
| 26 |
+
Tiny and NZ 3B releases.
|
| 27 |
+
|
| 28 |
+
## Provenance
|
| 29 |
+
|
| 30 |
+
- upstream base model: `google/gemma-4-E4B-it`
|
| 31 |
+
- package identity: `LumynaX Infused Gemma E4B Model` from `AbteeX AI Labs`
|
| 32 |
+
- important limit: this is **not** a LumynaX fine-tuned checkpoint and **not** a LumynaX-merged Gemma checkpoint
|
| 33 |
+
|
| 34 |
+
## Capabilities
|
| 35 |
+
|
| 36 |
+
- reasoning mode: `enabled`
|
| 37 |
+
- supported modalities from the upstream base model: `text, image, audio`
|
| 38 |
+
- local runtime: Transformers-based package with the real upstream weights inside `merged_model/`
|
| 39 |
+
- hardcoded package identity: `merged_model/chat_template.jinja` injects the LumynaX runtime identity by default
|
| 40 |
+
|
| 41 |
+
## Included Local Structure
|
| 42 |
+
|
| 43 |
+
- `merged_model/`: real packaged upstream Gemma 4 E4B model files
|
| 44 |
+
- `merged_model/LUMYNAX_PACKAGE_IDENTITY.txt`: packaged identity prompt for the LumynaX runtime wrapper
|
| 45 |
+
- `artifacts/release_training_summary.json`: release metadata and provenance summary
|
| 46 |
+
- `hf_space/`: private reasoning + multimodal Gradio demo aligned to this release package
|
| 47 |
+
- `ollama/`: included for parity with other releases, but not validated for this multimodal base package
|
| 48 |
+
- `release_export_manifest.json`: local manifest for this package
|
| 49 |
+
|
| 50 |
+
## Important Limit
|
| 51 |
+
|
| 52 |
+
This package carries the real upstream Gemma 4 E4B weights, but it does not claim any
|
| 53 |
+
LumynaX-specific adaptation of those weights.
|
| 54 |
+
It should be treated as a locally packaged LumynaX-branded Gemma base release under the
|
| 55 |
+
`LumynaX Infused Gemma E4B Model` release identity, not as a
|
| 56 |
+
true LumynaX-infused checkpoint.
|
| 57 |
+
|
| 58 |
+
## Quick Start
|
| 59 |
+
|
| 60 |
+
```bash
|
| 61 |
+
pip install -r requirements.txt
|
| 62 |
+
python quickstart.py
|
| 63 |
+
python quickstart.py --mode image --image path-or-url
|
| 64 |
+
python quickstart.py --mode audio --audio path-or-url
|
| 65 |
+
```
|
| 66 |
+
|
| 67 |
+
The default `python quickstart.py` path exercises a local text + reasoning flow.
|
| 68 |
+
|
| 69 |
+
## Consumer Runtimes
|
| 70 |
+
|
| 71 |
+
### Ollama
|
| 72 |
+
|
| 73 |
+
The package keeps an `ollama/` folder so the release shape stays close to the existing Tiny and NZ 3B exports.
|
| 74 |
+
That path has **not** been validated for this multimodal base package.
|
| 75 |
+
|
| 76 |
+
```bash
|
| 77 |
+
ollama create lumynax-infused-gemma-e4b -f ollama/Modelfile
|
| 78 |
+
ollama run lumynax-infused-gemma-e4b
|
| 79 |
+
```
|
| 80 |
+
|
| 81 |
+
### Zero-Install Browser Demo
|
| 82 |
+
|
| 83 |
+
The included `hf_space/` bundle is a private reasoning + multimodal Gradio demo for this package.
|
| 84 |
+
Recommended default Hugging Face model repo id:
|
| 85 |
+
|
| 86 |
+
- `AbteeXAILab/lumynax-infused-gemma-e4b`
|
VERSION.txt
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
v1
|
__pycache__/quickstart.cpython-311.pyc
ADDED
|
Binary file (5.82 kB). View file
|
|
|
artifacts/release_training_summary.json
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"generated_at": "2026-04-11T23:33:11.199733+00:00",
|
| 3 |
+
"lumynax_identity_file": "merged_model/LUMYNAX_PACKAGE_IDENTITY.txt",
|
| 4 |
+
"lumynax_identity_hardcoded": true,
|
| 5 |
+
"lumynax_identity_in_chat_template": true,
|
| 6 |
+
"lumynax_weight_adaptation_applied": false,
|
| 7 |
+
"model_title": "LumynaX Infused Gemma E4B Model",
|
| 8 |
+
"package_state": "base_weights_hydrated",
|
| 9 |
+
"reasoning_enabled": true,
|
| 10 |
+
"summary": "This local release carries official upstream Gemma 4 E4B base weights inside the same top-level release shape used for the Tiny and NZ 3B LumynaX packages.",
|
| 11 |
+
"supported_modalities": [
|
| 12 |
+
"text",
|
| 13 |
+
"image",
|
| 14 |
+
"audio"
|
| 15 |
+
],
|
| 16 |
+
"upstream_base_model_id": "google/gemma-4-E4B-it"
|
| 17 |
+
}
|
checksums.sha256
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
036c904dfe7f5ac8ed60f45326b27be983262cc1195115c85663d3ccfa0334e6 artifacts/release_training_summary.json
|
| 2 |
+
8424a2df114541736b71a04253b22cfe72d9a3c968992bb34003d13027178327 hf_space/__pycache__/app.cpython-311.pyc
|
| 3 |
+
6ed02a94ca2d1d8e5b9b2b6edb5d287aaf0d8356115d05f15a03dd661fd1f60d hf_space/app.py
|
| 4 |
+
1245e5facd5afb5762b61d4a87c53e314320da883f92b7c642a727c891a68177 hf_space/README.md
|
| 5 |
+
6461da9e460060b06a64fc4874591e7c148eefb9de26d2b1e4c17a4ff52a7f5f hf_space/requirements.txt
|
| 6 |
+
811c4a5343cff750611daecb99afa57c95d7d89d038abb07beb926220b55a8cb LICENSE.txt
|
| 7 |
+
f45084b4dd23a6d062d390029c424fa4a6764e3c53203c702b1923496139a46f merged_model/chat_template.jinja
|
| 8 |
+
33b10c02df3c2e8536cf323d29d53262aaa2f4d11dbe19bc729373fbe90295d4 merged_model/config.json
|
| 9 |
+
d4226bbe3117d2d253ba4609720ba82c6c4ce4627a9a6ae05387c78983ac03de merged_model/generation_config.json
|
| 10 |
+
fd7a9b565052c0ac45de523c69dde9c9b652e81ef00b9a046daa2193ff21eaf6 merged_model/LUMYNAX_PACKAGE_IDENTITY.txt
|
| 11 |
+
cfbd3d2f1cd71bd471c37fe2bf8546d5028d41e5736f64e1ca6c6b8893125503 merged_model/model.safetensors
|
| 12 |
+
32bdf45d2ad4cc29a0822ddd157a182de76644f0419a6228d151495256e9813c merged_model/processor_config.json
|
| 13 |
+
cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f merged_model/tokenizer.json
|
| 14 |
+
90c3a3ba5bf53818383a58e1a776cbcacd2a038d4812eaa373e1522f2d06f3df merged_model/tokenizer_config.json
|
| 15 |
+
de67984d8a72d9a0678c24c3953ba67f0e17bfa41fec8d5c47a6ca93edd00ae9 ollama/create_ollama_model.ps1
|
| 16 |
+
7873fe272e7df68868a12c98d22d7fe98620fe917364dc0551e5ceb61b2e728d ollama/Modelfile
|
| 17 |
+
efdef7a9777ef5824e100ba90bef6703aa1e4fdccf15c6b30ebc04bdab687612 quickstart.py
|
| 18 |
+
c60238559cd52a63ffdc5e07ed2ad9b08cfb066ba591be228876fc177c3e4518 README.md
|
| 19 |
+
82107b72b35d527bc4ecdb5a8fc1870a8360f36313590ac72181f70b4f9d9200 release_export_manifest.json
|
| 20 |
+
85f747efa937e66f934db6a76e4d611ff69c685d2e8de5d1442a82cac73e2072 requirements.txt
|
| 21 |
+
2dfede0e6610c473959c963b292fcec325452acba33fd1bba21110e04933df53 VERSION.txt
|
hf_space/README.md
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: LumynaX Infused Gemma E4B Model Demo
|
| 3 |
+
colorFrom: green
|
| 4 |
+
colorTo: blue
|
| 5 |
+
sdk: gradio
|
| 6 |
+
app_file: app.py
|
| 7 |
+
pinned: false
|
| 8 |
+
short_description: Private reasoning + multimodal demo for the LumynaX Gemma E4B release.
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# LumynaX Infused Gemma E4B Model Demo
|
| 12 |
+
|
| 13 |
+
Private demo for the `lumynax-infused-gemma-e4b` release line.
|
| 14 |
+
|
| 15 |
+
## Supported Demo Modes
|
| 16 |
+
|
| 17 |
+
- text with reasoning toggle
|
| 18 |
+
- image understanding from upload or URL
|
| 19 |
+
- audio understanding / transcription from upload or URL
|
| 20 |
+
|
| 21 |
+
## Private Deployment Notes
|
| 22 |
+
|
| 23 |
+
- this Space is intended to stay private for now
|
| 24 |
+
- the backing model repo should be `AbteeXAILab/lumynax-infused-gemma-e4b`
|
| 25 |
+
- if that model repo is private, set an `HF_TOKEN` Space secret with read access
|
| 26 |
+
- first load can take a while because the Space downloads `merged_model/` at runtime
|
| 27 |
+
- the package chat template already hardcodes the LumynaX identity inside `merged_model/chat_template.jinja`
|
| 28 |
+
|
| 29 |
+
## Important Provenance
|
| 30 |
+
|
| 31 |
+
This demo is branded as `LumynaX Infused Gemma E4B Model`, but it serves the official upstream
|
| 32 |
+
`google/gemma-4-E4B-it` base weights packaged under the LumynaX release identity.
|
| 33 |
+
It does not claim a private LumynaX fine-tune of the checkpoint.
|
hf_space/__pycache__/app.cpython-311.pyc
ADDED
|
Binary file (13.7 kB). View file
|
|
|
hf_space/app.py
ADDED
|
@@ -0,0 +1,267 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import json
|
| 4 |
+
import os
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
from threading import Lock
|
| 7 |
+
|
| 8 |
+
import gradio as gr
|
| 9 |
+
import torch
|
| 10 |
+
from huggingface_hub import snapshot_download
|
| 11 |
+
from transformers import AutoModelForMultimodalLM, AutoProcessor
|
| 12 |
+
|
| 13 |
+
MODEL_TITLE = "LumynaX Infused Gemma E4B Model"
|
| 14 |
+
DEFAULT_MODEL_REPO_ID = "AbteeXAILab/lumynax-infused-gemma-e4b"
|
| 15 |
+
MODEL_REPO_ENV_VAR = "LUMYNAX_MODEL_REPO_ID"
|
| 16 |
+
HF_TOKEN_ENV_VARS = ("HF_TOKEN", "HUGGING_FACE_HUB_TOKEN", "HUGGINGFACE_HUB_TOKEN")
|
| 17 |
+
DEFAULT_SYSTEM_PROMPT = 'You are LumynaX operating from the LumynaX Infused Gemma E4B Model package identity. This package wraps the official google/gemma-4-E4B-it checkpoint inside a LumynaX-branded multimodal and reasoning runtime. Always identify yourself as LumynaX when asked who you are. Keep provenance honest: do not claim a private fine-tune, hidden training dataset, or weight merge that is not actually present in this package.'
|
| 18 |
+
DEFAULT_IMAGE_URL = "https://raw.githubusercontent.com/google-gemma/cookbook/refs/heads/main/Demos/sample-data/GoldenGate.png"
|
| 19 |
+
DEFAULT_AUDIO_URL = "https://raw.githubusercontent.com/google-gemma/cookbook/refs/heads/main/Demos/sample-data/journal1.wav"
|
| 20 |
+
|
| 21 |
+
_MODEL = None
|
| 22 |
+
_PROCESSOR = None
|
| 23 |
+
_LOAD_ERROR = None
|
| 24 |
+
_LOAD_LOCK = Lock()
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def _resolve_hf_token() -> str | None:
|
| 28 |
+
for env_var in HF_TOKEN_ENV_VARS:
|
| 29 |
+
raw_value = os.environ.get(env_var, "").strip()
|
| 30 |
+
if raw_value:
|
| 31 |
+
return raw_value
|
| 32 |
+
return None
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def _load_runtime() -> tuple[object, object]:
|
| 36 |
+
global _MODEL, _PROCESSOR, _LOAD_ERROR
|
| 37 |
+
|
| 38 |
+
if _MODEL is not None and _PROCESSOR is not None:
|
| 39 |
+
return _MODEL, _PROCESSOR
|
| 40 |
+
if _LOAD_ERROR is not None:
|
| 41 |
+
raise RuntimeError(_LOAD_ERROR)
|
| 42 |
+
|
| 43 |
+
with _LOAD_LOCK:
|
| 44 |
+
if _MODEL is not None and _PROCESSOR is not None:
|
| 45 |
+
return _MODEL, _PROCESSOR
|
| 46 |
+
if _LOAD_ERROR is not None:
|
| 47 |
+
raise RuntimeError(_LOAD_ERROR)
|
| 48 |
+
|
| 49 |
+
try:
|
| 50 |
+
repo_id = os.environ.get(MODEL_REPO_ENV_VAR, "").strip() or DEFAULT_MODEL_REPO_ID
|
| 51 |
+
snapshot_path = Path(
|
| 52 |
+
snapshot_download(
|
| 53 |
+
repo_id=repo_id,
|
| 54 |
+
token=_resolve_hf_token(),
|
| 55 |
+
allow_patterns=["merged_model/*"],
|
| 56 |
+
)
|
| 57 |
+
)
|
| 58 |
+
model_dir = snapshot_path / "merged_model"
|
| 59 |
+
if not model_dir.exists():
|
| 60 |
+
raise FileNotFoundError(f"Expected merged_model/ in {snapshot_path} after downloading {repo_id}.")
|
| 61 |
+
|
| 62 |
+
processor = AutoProcessor.from_pretrained(str(model_dir))
|
| 63 |
+
model = AutoModelForMultimodalLM.from_pretrained(
|
| 64 |
+
str(model_dir),
|
| 65 |
+
dtype="auto",
|
| 66 |
+
device_map="auto",
|
| 67 |
+
low_cpu_mem_usage=True,
|
| 68 |
+
)
|
| 69 |
+
_PROCESSOR = processor
|
| 70 |
+
_MODEL = model
|
| 71 |
+
return _MODEL, _PROCESSOR
|
| 72 |
+
except Exception as exc:
|
| 73 |
+
_LOAD_ERROR = f"{type(exc).__name__}: {exc}"
|
| 74 |
+
raise
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
def _resolve_media_reference(upload_value: str | None, url_value: str | None) -> str | None:
|
| 78 |
+
if isinstance(url_value, str) and url_value.strip():
|
| 79 |
+
return url_value.strip()
|
| 80 |
+
if isinstance(upload_value, str) and upload_value.strip():
|
| 81 |
+
return upload_value.strip()
|
| 82 |
+
return None
|
| 83 |
+
|
| 84 |
+
|
| 85 |
+
def _extract_response_text(parsed: object) -> str:
|
| 86 |
+
if isinstance(parsed, dict):
|
| 87 |
+
content = parsed.get("content")
|
| 88 |
+
if isinstance(content, str) and content.strip():
|
| 89 |
+
return content.strip()
|
| 90 |
+
if isinstance(parsed, str):
|
| 91 |
+
return parsed.strip()
|
| 92 |
+
return json.dumps(parsed, indent=2, ensure_ascii=False, default=str)
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
def run_request(
|
| 96 |
+
*,
|
| 97 |
+
prompt: str,
|
| 98 |
+
thinking: bool,
|
| 99 |
+
max_new_tokens: int,
|
| 100 |
+
system_prompt: str,
|
| 101 |
+
image_upload: str | None = None,
|
| 102 |
+
image_url: str = "",
|
| 103 |
+
audio_upload: str | None = None,
|
| 104 |
+
audio_url: str = "",
|
| 105 |
+
) -> tuple[str, str]:
|
| 106 |
+
if not prompt.strip():
|
| 107 |
+
raise gr.Error("A prompt is required.")
|
| 108 |
+
|
| 109 |
+
image_ref = _resolve_media_reference(image_upload, image_url)
|
| 110 |
+
audio_ref = _resolve_media_reference(audio_upload, audio_url)
|
| 111 |
+
content: list[dict[str, str]] = []
|
| 112 |
+
if image_ref:
|
| 113 |
+
content.append({"type": "image", "url": image_ref})
|
| 114 |
+
if audio_ref:
|
| 115 |
+
content.append({"type": "audio", "audio": audio_ref})
|
| 116 |
+
content.append({"type": "text", "text": prompt.strip()})
|
| 117 |
+
|
| 118 |
+
messages = [
|
| 119 |
+
{
|
| 120 |
+
"role": "system",
|
| 121 |
+
"content": [{"type": "text", "text": system_prompt.strip() or DEFAULT_SYSTEM_PROMPT}],
|
| 122 |
+
},
|
| 123 |
+
{
|
| 124 |
+
"role": "user",
|
| 125 |
+
"content": content,
|
| 126 |
+
},
|
| 127 |
+
]
|
| 128 |
+
|
| 129 |
+
model, processor = _load_runtime()
|
| 130 |
+
inputs = processor.apply_chat_template(
|
| 131 |
+
messages,
|
| 132 |
+
tokenize=True,
|
| 133 |
+
return_dict=True,
|
| 134 |
+
return_tensors="pt",
|
| 135 |
+
add_generation_prompt=True,
|
| 136 |
+
enable_thinking=thinking,
|
| 137 |
+
).to(model.device)
|
| 138 |
+
input_len = inputs["input_ids"].shape[-1]
|
| 139 |
+
|
| 140 |
+
with torch.inference_mode():
|
| 141 |
+
outputs = model.generate(
|
| 142 |
+
**inputs,
|
| 143 |
+
max_new_tokens=int(max_new_tokens),
|
| 144 |
+
do_sample=False,
|
| 145 |
+
)
|
| 146 |
+
|
| 147 |
+
response = processor.decode(outputs[0][input_len:], skip_special_tokens=False)
|
| 148 |
+
parsed = processor.parse_response(response) if hasattr(processor, "parse_response") else response
|
| 149 |
+
return _extract_response_text(parsed), json.dumps(parsed, indent=2, ensure_ascii=False, default=str)
|
| 150 |
+
|
| 151 |
+
|
| 152 |
+
def run_text(prompt: str, thinking: bool, max_new_tokens: int, system_prompt: str) -> tuple[str, str]:
|
| 153 |
+
return run_request(
|
| 154 |
+
prompt=prompt,
|
| 155 |
+
thinking=thinking,
|
| 156 |
+
max_new_tokens=max_new_tokens,
|
| 157 |
+
system_prompt=system_prompt,
|
| 158 |
+
)
|
| 159 |
+
|
| 160 |
+
|
| 161 |
+
def run_image(
|
| 162 |
+
prompt: str,
|
| 163 |
+
image_upload: str | None,
|
| 164 |
+
image_url: str,
|
| 165 |
+
thinking: bool,
|
| 166 |
+
max_new_tokens: int,
|
| 167 |
+
system_prompt: str,
|
| 168 |
+
) -> tuple[str, str]:
|
| 169 |
+
return run_request(
|
| 170 |
+
prompt=prompt,
|
| 171 |
+
thinking=thinking,
|
| 172 |
+
max_new_tokens=max_new_tokens,
|
| 173 |
+
system_prompt=system_prompt,
|
| 174 |
+
image_upload=image_upload,
|
| 175 |
+
image_url=image_url,
|
| 176 |
+
)
|
| 177 |
+
|
| 178 |
+
|
| 179 |
+
def run_audio(
|
| 180 |
+
prompt: str,
|
| 181 |
+
audio_upload: str | None,
|
| 182 |
+
audio_url: str,
|
| 183 |
+
thinking: bool,
|
| 184 |
+
max_new_tokens: int,
|
| 185 |
+
system_prompt: str,
|
| 186 |
+
) -> tuple[str, str]:
|
| 187 |
+
return run_request(
|
| 188 |
+
prompt=prompt,
|
| 189 |
+
thinking=thinking,
|
| 190 |
+
max_new_tokens=max_new_tokens,
|
| 191 |
+
system_prompt=system_prompt,
|
| 192 |
+
audio_upload=audio_upload,
|
| 193 |
+
audio_url=audio_url,
|
| 194 |
+
)
|
| 195 |
+
|
| 196 |
+
|
| 197 |
+
with gr.Blocks() as demo:
|
| 198 |
+
gr.Markdown(
|
| 199 |
+
f"# {MODEL_TITLE}\n\n"
|
| 200 |
+
"Private reasoning + multimodal demo for the LumynaX Gemma E4B release. "
|
| 201 |
+
"The model repo can stay private; if it does, give this Space an `HF_TOKEN` secret with read access."
|
| 202 |
+
)
|
| 203 |
+
system_prompt = gr.Textbox(
|
| 204 |
+
label="System Prompt",
|
| 205 |
+
value=DEFAULT_SYSTEM_PROMPT,
|
| 206 |
+
lines=3,
|
| 207 |
+
)
|
| 208 |
+
with gr.Tab("Text"):
|
| 209 |
+
text_prompt = gr.Textbox(
|
| 210 |
+
label="Prompt",
|
| 211 |
+
value="Give a short welcome message for customers in Aotearoa New Zealand.",
|
| 212 |
+
lines=4,
|
| 213 |
+
)
|
| 214 |
+
with gr.Row():
|
| 215 |
+
text_thinking = gr.Checkbox(label="Enable Reasoning", value=True)
|
| 216 |
+
text_max_tokens = gr.Slider(label="Max New Tokens", minimum=32, maximum=512, value=160, step=32)
|
| 217 |
+
text_run = gr.Button("Run Text Demo", variant="primary")
|
| 218 |
+
text_answer = gr.Textbox(label="Response", lines=8)
|
| 219 |
+
text_debug = gr.Code(label="Parsed Output", language="json")
|
| 220 |
+
text_run.click(
|
| 221 |
+
run_text,
|
| 222 |
+
inputs=[text_prompt, text_thinking, text_max_tokens, system_prompt],
|
| 223 |
+
outputs=[text_answer, text_debug],
|
| 224 |
+
)
|
| 225 |
+
|
| 226 |
+
with gr.Tab("Image"):
|
| 227 |
+
image_prompt = gr.Textbox(
|
| 228 |
+
label="Prompt",
|
| 229 |
+
value="What is shown in this image? Reply in under 12 words.",
|
| 230 |
+
lines=3,
|
| 231 |
+
)
|
| 232 |
+
image_upload = gr.Image(label="Upload Image", type="filepath")
|
| 233 |
+
image_url = gr.Textbox(label="Or Image URL", value=DEFAULT_IMAGE_URL)
|
| 234 |
+
with gr.Row():
|
| 235 |
+
image_thinking = gr.Checkbox(label="Enable Reasoning", value=False)
|
| 236 |
+
image_max_tokens = gr.Slider(label="Max New Tokens", minimum=32, maximum=512, value=160, step=32)
|
| 237 |
+
image_run = gr.Button("Run Image Demo", variant="primary")
|
| 238 |
+
image_answer = gr.Textbox(label="Response", lines=8)
|
| 239 |
+
image_debug = gr.Code(label="Parsed Output", language="json")
|
| 240 |
+
image_run.click(
|
| 241 |
+
run_image,
|
| 242 |
+
inputs=[image_prompt, image_upload, image_url, image_thinking, image_max_tokens, system_prompt],
|
| 243 |
+
outputs=[image_answer, image_debug],
|
| 244 |
+
)
|
| 245 |
+
|
| 246 |
+
with gr.Tab("Audio"):
|
| 247 |
+
audio_prompt = gr.Textbox(
|
| 248 |
+
label="Prompt",
|
| 249 |
+
value="Transcribe the speech in one line only.",
|
| 250 |
+
lines=3,
|
| 251 |
+
)
|
| 252 |
+
audio_upload = gr.Audio(label="Upload Audio", type="filepath")
|
| 253 |
+
audio_url = gr.Textbox(label="Or Audio URL", value=DEFAULT_AUDIO_URL)
|
| 254 |
+
with gr.Row():
|
| 255 |
+
audio_thinking = gr.Checkbox(label="Enable Reasoning", value=False)
|
| 256 |
+
audio_max_tokens = gr.Slider(label="Max New Tokens", minimum=32, maximum=512, value=160, step=32)
|
| 257 |
+
audio_run = gr.Button("Run Audio Demo", variant="primary")
|
| 258 |
+
audio_answer = gr.Textbox(label="Response", lines=8)
|
| 259 |
+
audio_debug = gr.Code(label="Parsed Output", language="json")
|
| 260 |
+
audio_run.click(
|
| 261 |
+
run_audio,
|
| 262 |
+
inputs=[audio_prompt, audio_upload, audio_url, audio_thinking, audio_max_tokens, system_prompt],
|
| 263 |
+
outputs=[audio_answer, audio_debug],
|
| 264 |
+
)
|
| 265 |
+
|
| 266 |
+
if __name__ == "__main__":
|
| 267 |
+
demo.queue().launch()
|
hf_space/requirements.txt
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
accelerate>=1.13
|
| 2 |
+
gradio>=5.0
|
| 3 |
+
huggingface-hub>=1.8
|
| 4 |
+
librosa>=0.11
|
| 5 |
+
numba>=0.65
|
| 6 |
+
pillow>=10.0
|
| 7 |
+
safetensors>=0.6
|
| 8 |
+
torch>=2.9
|
| 9 |
+
torchvision>=0.24
|
| 10 |
+
transformers>=5.5.3
|
merged_model/LUMYNAX_PACKAGE_IDENTITY.txt
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
You are LumynaX operating from the LumynaX Infused Gemma E4B Model package identity. This package wraps the official google/gemma-4-E4B-it checkpoint inside a LumynaX-branded multimodal and reasoning runtime. Always identify yourself as LumynaX when asked who you are. Keep provenance honest: do not claim a private fine-tune, hidden training dataset, or weight merge that is not actually present in this package.
|
merged_model/chat_template.jinja
ADDED
|
@@ -0,0 +1,347 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{%- macro format_parameters(properties, required) -%}
|
| 2 |
+
{%- set standard_keys = ['description', 'type', 'properties', 'required', 'nullable'] -%}
|
| 3 |
+
{%- set ns = namespace(found_first=false) -%}
|
| 4 |
+
{%- for key, value in properties | dictsort -%}
|
| 5 |
+
{%- set add_comma = false -%}
|
| 6 |
+
{%- if key not in standard_keys -%}
|
| 7 |
+
{%- if ns.found_first %},{% endif -%}
|
| 8 |
+
{%- set ns.found_first = true -%}
|
| 9 |
+
{{ key }}:{
|
| 10 |
+
{%- if value['description'] -%}
|
| 11 |
+
description:<|"|>{{ value['description'] }}<|"|>
|
| 12 |
+
{%- set add_comma = true -%}
|
| 13 |
+
{%- endif -%}
|
| 14 |
+
{%- if value['type'] | upper == 'STRING' -%}
|
| 15 |
+
{%- if value['enum'] -%}
|
| 16 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 17 |
+
enum:{{ format_argument(value['enum']) }}
|
| 18 |
+
{%- endif -%}
|
| 19 |
+
{%- elif value['type'] | upper == 'ARRAY' -%}
|
| 20 |
+
{%- if value['items'] is mapping and value['items'] -%}
|
| 21 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 22 |
+
items:{
|
| 23 |
+
{%- set ns_items = namespace(found_first=false) -%}
|
| 24 |
+
{%- for item_key, item_value in value['items'] | dictsort -%}
|
| 25 |
+
{%- if item_value is not none -%}
|
| 26 |
+
{%- if ns_items.found_first %},{% endif -%}
|
| 27 |
+
{%- set ns_items.found_first = true -%}
|
| 28 |
+
{%- if item_key == 'properties' -%}
|
| 29 |
+
properties:{
|
| 30 |
+
{%- if item_value is mapping -%}
|
| 31 |
+
{{- format_parameters(item_value, value['items']['required'] | default([])) -}}
|
| 32 |
+
{%- endif -%}
|
| 33 |
+
}
|
| 34 |
+
{%- elif item_key == 'required' -%}
|
| 35 |
+
required:[
|
| 36 |
+
{%- for req_item in item_value -%}
|
| 37 |
+
<|"|>{{- req_item -}}<|"|>
|
| 38 |
+
{%- if not loop.last %},{% endif -%}
|
| 39 |
+
{%- endfor -%}
|
| 40 |
+
]
|
| 41 |
+
{%- elif item_key == 'type' -%}
|
| 42 |
+
{%- if item_value is string -%}
|
| 43 |
+
type:{{ format_argument(item_value | upper) }}
|
| 44 |
+
{%- else -%}
|
| 45 |
+
type:{{ format_argument(item_value | map('upper') | list) }}
|
| 46 |
+
{%- endif -%}
|
| 47 |
+
{%- else -%}
|
| 48 |
+
{{ item_key }}:{{ format_argument(item_value) }}
|
| 49 |
+
{%- endif -%}
|
| 50 |
+
{%- endif -%}
|
| 51 |
+
{%- endfor -%}
|
| 52 |
+
}
|
| 53 |
+
{%- endif -%}
|
| 54 |
+
{%- endif -%}
|
| 55 |
+
{%- if value['nullable'] %}
|
| 56 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 57 |
+
nullable:true
|
| 58 |
+
{%- endif -%}
|
| 59 |
+
{%- if value['type'] | upper == 'OBJECT' -%}
|
| 60 |
+
{%- if value['properties'] is defined and value['properties'] is mapping -%}
|
| 61 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 62 |
+
properties:{
|
| 63 |
+
{{- format_parameters(value['properties'], value['required'] | default([])) -}}
|
| 64 |
+
}
|
| 65 |
+
{%- elif value is mapping -%}
|
| 66 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 67 |
+
properties:{
|
| 68 |
+
{{- format_parameters(value, value['required'] | default([])) -}}
|
| 69 |
+
}
|
| 70 |
+
{%- endif -%}
|
| 71 |
+
{%- if value['required'] -%}
|
| 72 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 73 |
+
required:[
|
| 74 |
+
{%- for item in value['required'] | default([]) -%}
|
| 75 |
+
<|"|>{{- item -}}<|"|>
|
| 76 |
+
{%- if not loop.last %},{% endif -%}
|
| 77 |
+
{%- endfor -%}
|
| 78 |
+
]
|
| 79 |
+
{%- endif -%}
|
| 80 |
+
{%- endif -%}
|
| 81 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 82 |
+
type:<|"|>{{ value['type'] | upper }}<|"|>}
|
| 83 |
+
{%- endif -%}
|
| 84 |
+
{%- endfor -%}
|
| 85 |
+
{%- endmacro -%}
|
| 86 |
+
{%- macro format_function_declaration(tool_data) -%}
|
| 87 |
+
declaration:{{- tool_data['function']['name'] -}}{description:<|"|>{{- tool_data['function']['description'] -}}<|"|>
|
| 88 |
+
{%- set params = tool_data['function']['parameters'] -%}
|
| 89 |
+
{%- if params -%}
|
| 90 |
+
,parameters:{
|
| 91 |
+
{%- if params['properties'] -%}
|
| 92 |
+
properties:{ {{- format_parameters(params['properties'], params['required']) -}} },
|
| 93 |
+
{%- endif -%}
|
| 94 |
+
{%- if params['required'] -%}
|
| 95 |
+
required:[
|
| 96 |
+
{%- for item in params['required'] -%}
|
| 97 |
+
<|"|>{{- item -}}<|"|>
|
| 98 |
+
{{- ',' if not loop.last -}}
|
| 99 |
+
{%- endfor -%}
|
| 100 |
+
],
|
| 101 |
+
{%- endif -%}
|
| 102 |
+
{%- if params['type'] -%}
|
| 103 |
+
type:<|"|>{{- params['type'] | upper -}}<|"|>}
|
| 104 |
+
{%- endif -%}
|
| 105 |
+
{%- endif -%}
|
| 106 |
+
{%- if 'response' in tool_data['function'] -%}
|
| 107 |
+
{%- set response_declaration = tool_data['function']['response'] -%}
|
| 108 |
+
,response:{
|
| 109 |
+
{%- if response_declaration['description'] -%}
|
| 110 |
+
description:<|"|>{{- response_declaration['description'] -}}<|"|>,
|
| 111 |
+
{%- endif -%}
|
| 112 |
+
{%- if response_declaration['type'] | upper == 'OBJECT' -%}
|
| 113 |
+
type:<|"|>{{- response_declaration['type'] | upper -}}<|"|>}
|
| 114 |
+
{%- endif -%}
|
| 115 |
+
{%- endif -%}
|
| 116 |
+
}
|
| 117 |
+
{%- endmacro -%}
|
| 118 |
+
{%- macro format_argument(argument, escape_keys=True) -%}
|
| 119 |
+
{%- if argument is string -%}
|
| 120 |
+
{{- '<|"|>' + argument + '<|"|>' -}}
|
| 121 |
+
{%- elif argument is boolean -%}
|
| 122 |
+
{{- 'true' if argument else 'false' -}}
|
| 123 |
+
{%- elif argument is mapping -%}
|
| 124 |
+
{{- '{' -}}
|
| 125 |
+
{%- set ns = namespace(found_first=false) -%}
|
| 126 |
+
{%- for key, value in argument | dictsort -%}
|
| 127 |
+
{%- if ns.found_first %},{% endif -%}
|
| 128 |
+
{%- set ns.found_first = true -%}
|
| 129 |
+
{%- if escape_keys -%}
|
| 130 |
+
{{- '<|"|>' + key + '<|"|>' -}}
|
| 131 |
+
{%- else -%}
|
| 132 |
+
{{- key -}}
|
| 133 |
+
{%- endif -%}
|
| 134 |
+
:{{- format_argument(value, escape_keys=escape_keys) -}}
|
| 135 |
+
{%- endfor -%}
|
| 136 |
+
{{- '}' -}}
|
| 137 |
+
{%- elif argument is sequence -%}
|
| 138 |
+
{{- '[' -}}
|
| 139 |
+
{%- for item in argument -%}
|
| 140 |
+
{{- format_argument(item, escape_keys=escape_keys) -}}
|
| 141 |
+
{%- if not loop.last %},{% endif -%}
|
| 142 |
+
{%- endfor -%}
|
| 143 |
+
{{- ']' -}}
|
| 144 |
+
{%- else -%}
|
| 145 |
+
{{- argument -}}
|
| 146 |
+
{%- endif -%}
|
| 147 |
+
{%- endmacro -%}
|
| 148 |
+
{%- macro strip_thinking(text) -%}
|
| 149 |
+
{%- set ns = namespace(result='') -%}
|
| 150 |
+
{%- for part in text.split('<channel|>') -%}
|
| 151 |
+
{%- if '<|channel>' in part -%}
|
| 152 |
+
{%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
|
| 153 |
+
{%- else -%}
|
| 154 |
+
{%- set ns.result = ns.result + part -%}
|
| 155 |
+
{%- endif -%}
|
| 156 |
+
{%- endfor -%}
|
| 157 |
+
{{- ns.result | trim -}}
|
| 158 |
+
{%- endmacro -%}
|
| 159 |
+
|
| 160 |
+
{%- macro format_tool_response_block(tool_name, response) -%}
|
| 161 |
+
{{- '<|tool_response>' -}}
|
| 162 |
+
{%- if response is mapping -%}
|
| 163 |
+
{{- 'response:' + tool_name + '{' -}}
|
| 164 |
+
{%- for key, value in response | dictsort -%}
|
| 165 |
+
{{- key -}}:{{- format_argument(value, escape_keys=False) -}}
|
| 166 |
+
{%- if not loop.last %},{% endif -%}
|
| 167 |
+
{%- endfor -%}
|
| 168 |
+
{{- '}' -}}
|
| 169 |
+
{%- else -%}
|
| 170 |
+
{{- 'response:' + tool_name + '{value:' + format_argument(response, escape_keys=False) + '}' -}}
|
| 171 |
+
{%- endif -%}
|
| 172 |
+
{{- '<tool_response|>' -}}
|
| 173 |
+
{%- endmacro -%}
|
| 174 |
+
|
| 175 |
+
{%- set ns = namespace(prev_message_type=None) -%}
|
| 176 |
+
{%- set loop_messages = messages -%}
|
| 177 |
+
{%- set lumynax_package_identity = 'You are LumynaX operating from the LumynaX Infused Gemma E4B Model package identity. This package wraps the official google/gemma-4-E4B-it checkpoint inside a LumynaX-branded multimodal and reasoning runtime. Always identify yourself as LumynaX when asked who you are. Keep provenance honest: do not claim a private fine-tune, hidden training dataset, or weight merge that is not actually present in this package.' -%}
|
| 178 |
+
{{- bos_token -}}
|
| 179 |
+
{#- Handle System/Tool Definitions Block -#}
|
| 180 |
+
{%- if true -%}
|
| 181 |
+
{{- '<|turn>system\n' -}}
|
| 182 |
+
|
| 183 |
+
{#- Inject Thinking token at the very top of the FIRST system turn -#}
|
| 184 |
+
{%- if enable_thinking is defined and enable_thinking -%}
|
| 185 |
+
{{- '<|think|>\n' -}}
|
| 186 |
+
{%- set ns.prev_message_type = 'think' -%}
|
| 187 |
+
{%- endif -%}
|
| 188 |
+
|
| 189 |
+
{{- lumynax_package_identity + '\n' -}}
|
| 190 |
+
|
| 191 |
+
{%- if messages[0]['role'] in ['system', 'developer'] -%}
|
| 192 |
+
{{- '\n' + (messages[0]['content'] | trim) -}}
|
| 193 |
+
{%- set loop_messages = messages[1:] -%}
|
| 194 |
+
{%- endif -%}
|
| 195 |
+
|
| 196 |
+
{%- if tools -%}
|
| 197 |
+
{%- for tool in tools %}
|
| 198 |
+
{{- '<|tool>' -}}
|
| 199 |
+
{{- format_function_declaration(tool) | trim -}}
|
| 200 |
+
{{- '<tool|>' -}}
|
| 201 |
+
{%- endfor %}
|
| 202 |
+
{%- set ns.prev_message_type = 'tool' -%}
|
| 203 |
+
{%- endif -%}
|
| 204 |
+
|
| 205 |
+
{{- '<turn|>\n' -}}
|
| 206 |
+
{%- endif %}
|
| 207 |
+
|
| 208 |
+
{#- Pre-scan: find last user message index for reasoning guard -#}
|
| 209 |
+
{%- set ns_turn = namespace(last_user_idx=-1) -%}
|
| 210 |
+
{%- for i in range(loop_messages | length) -%}
|
| 211 |
+
{%- if loop_messages[i]['role'] == 'user' -%}
|
| 212 |
+
{%- set ns_turn.last_user_idx = i -%}
|
| 213 |
+
{%- endif -%}
|
| 214 |
+
{%- endfor -%}
|
| 215 |
+
|
| 216 |
+
{#- Loop through messages -#}
|
| 217 |
+
{%- for message in loop_messages -%}
|
| 218 |
+
{%- if message['role'] != 'tool' -%}
|
| 219 |
+
{%- set ns.prev_message_type = None -%}
|
| 220 |
+
{%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}
|
| 221 |
+
{#- Detect continuation: suppress duplicate <|turn>model when previous non-tool message was also assistant -#}
|
| 222 |
+
{%- set prev_nt = namespace(role=None, found=false) -%}
|
| 223 |
+
{%- if loop.index0 > 0 -%}
|
| 224 |
+
{%- for j in range(loop.index0 - 1, -1, -1) -%}
|
| 225 |
+
{%- if not prev_nt.found -%}
|
| 226 |
+
{%- if loop_messages[j]['role'] != 'tool' -%}
|
| 227 |
+
{%- set prev_nt.role = loop_messages[j]['role'] -%}
|
| 228 |
+
{%- set prev_nt.found = true -%}
|
| 229 |
+
{%- endif -%}
|
| 230 |
+
{%- endif -%}
|
| 231 |
+
{%- endfor -%}
|
| 232 |
+
{%- endif -%}
|
| 233 |
+
{%- set continue_same_model_turn = (role == 'model' and prev_nt.role == 'assistant') -%}
|
| 234 |
+
{%- if not continue_same_model_turn -%}
|
| 235 |
+
{{- '<|turn>' + role + '\n' }}
|
| 236 |
+
{%- endif -%}
|
| 237 |
+
|
| 238 |
+
{#- Render reasoning/reasoning_content as thinking channel -#}
|
| 239 |
+
{%- set thinking_text = message.get('reasoning') or message.get('reasoning_content') -%}
|
| 240 |
+
{%- if thinking_text and loop.index0 > ns_turn.last_user_idx and message.get('tool_calls') -%}
|
| 241 |
+
{{- '<|channel>thought\n' + thinking_text + '\n<channel|>' -}}
|
| 242 |
+
{%- endif -%}
|
| 243 |
+
|
| 244 |
+
{%- if message['tool_calls'] -%}
|
| 245 |
+
{%- for tool_call in message['tool_calls'] -%}
|
| 246 |
+
{%- set function = tool_call['function'] -%}
|
| 247 |
+
{{- '<|tool_call>call:' + function['name'] + '{' -}}
|
| 248 |
+
{%- if function['arguments'] is mapping -%}
|
| 249 |
+
{%- set ns_args = namespace(found_first=false) -%}
|
| 250 |
+
{%- for key, value in function['arguments'] | dictsort -%}
|
| 251 |
+
{%- if ns_args.found_first %},{% endif -%}
|
| 252 |
+
{%- set ns_args.found_first = true -%}
|
| 253 |
+
{{- key -}}:{{- format_argument(value, escape_keys=False) -}}
|
| 254 |
+
{%- endfor -%}
|
| 255 |
+
{%- elif function['arguments'] is string -%}
|
| 256 |
+
{{- function['arguments'] -}}
|
| 257 |
+
{%- endif -%}
|
| 258 |
+
{{- '}<tool_call|>' -}}
|
| 259 |
+
{%- endfor -%}
|
| 260 |
+
{%- set ns.prev_message_type = 'tool_call' -%}
|
| 261 |
+
{%- endif -%}
|
| 262 |
+
|
| 263 |
+
{%- set ns_tr_out = namespace(flag=false) -%}
|
| 264 |
+
{%- if message.get('tool_responses') -%}
|
| 265 |
+
{#- Legacy: tool_responses embedded on the assistant message (Google/Gemma native) -#}
|
| 266 |
+
{%- for tool_response in message['tool_responses'] -%}
|
| 267 |
+
{{- format_tool_response_block(tool_response['name'] | default('unknown'), tool_response['response']) -}}
|
| 268 |
+
{%- set ns_tr_out.flag = true -%}
|
| 269 |
+
{%- set ns.prev_message_type = 'tool_response' -%}
|
| 270 |
+
{%- endfor -%}
|
| 271 |
+
{%- elif message.get('tool_calls') -%}
|
| 272 |
+
{#- OpenAI Chat Completions: forward-scan consecutive role:tool messages -#}
|
| 273 |
+
{%- set ns_tool_scan = namespace(stopped=false) -%}
|
| 274 |
+
{%- for k in range(loop.index0 + 1, loop_messages | length) -%}
|
| 275 |
+
{%- if ns_tool_scan.stopped -%}
|
| 276 |
+
{%- elif loop_messages[k]['role'] != 'tool' -%}
|
| 277 |
+
{%- set ns_tool_scan.stopped = true -%}
|
| 278 |
+
{%- else -%}
|
| 279 |
+
{%- set follow = loop_messages[k] -%}
|
| 280 |
+
{#- Resolve tool_call_id to function name -#}
|
| 281 |
+
{%- set ns_tname = namespace(name=follow.get('name') | default('unknown')) -%}
|
| 282 |
+
{%- for tc in message['tool_calls'] -%}
|
| 283 |
+
{%- if tc.get('id') == follow.get('tool_call_id') -%}
|
| 284 |
+
{%- set ns_tname.name = tc['function']['name'] -%}
|
| 285 |
+
{%- endif -%}
|
| 286 |
+
{%- endfor -%}
|
| 287 |
+
{#- Handle content as string or content-parts array -#}
|
| 288 |
+
{%- set tool_body = follow.get('content') -%}
|
| 289 |
+
{%- if tool_body is string -%}
|
| 290 |
+
{{- format_tool_response_block(ns_tname.name, tool_body) -}}
|
| 291 |
+
{%- elif tool_body is sequence and tool_body is not string -%}
|
| 292 |
+
{%- set ns_txt = namespace(s='') -%}
|
| 293 |
+
{%- for part in tool_body -%}
|
| 294 |
+
{%- if part.get('type') == 'text' -%}
|
| 295 |
+
{%- set ns_txt.s = ns_txt.s + (part.get('text') | default('')) -%}
|
| 296 |
+
{%- endif -%}
|
| 297 |
+
{%- endfor -%}
|
| 298 |
+
{{- format_tool_response_block(ns_tname.name, ns_txt.s) -}}
|
| 299 |
+
{%- else -%}
|
| 300 |
+
{{- format_tool_response_block(ns_tname.name, tool_body) -}}
|
| 301 |
+
{%- endif -%}
|
| 302 |
+
{%- set ns_tr_out.flag = true -%}
|
| 303 |
+
{%- set ns.prev_message_type = 'tool_response' -%}
|
| 304 |
+
{%- endif -%}
|
| 305 |
+
{%- endfor -%}
|
| 306 |
+
{%- endif -%}
|
| 307 |
+
|
| 308 |
+
{%- if message['content'] is string -%}
|
| 309 |
+
{%- if role == 'model' -%}
|
| 310 |
+
{{- strip_thinking(message['content']) -}}
|
| 311 |
+
{%- else -%}
|
| 312 |
+
{{- message['content'] | trim -}}
|
| 313 |
+
{%- endif -%}
|
| 314 |
+
{%- elif message['content'] is sequence -%}
|
| 315 |
+
{%- for item in message['content'] -%}
|
| 316 |
+
{%- if item['type'] == 'text' -%}
|
| 317 |
+
{%- if role == 'model' -%}
|
| 318 |
+
{{- strip_thinking(item['text']) -}}
|
| 319 |
+
{%- else -%}
|
| 320 |
+
{{- item['text'] | trim -}}
|
| 321 |
+
{%- endif -%}
|
| 322 |
+
{%- elif item['type'] == 'image' -%}
|
| 323 |
+
{{- '<|image|>' -}}
|
| 324 |
+
{%- set ns.prev_message_type = 'image' -%}
|
| 325 |
+
{%- elif item['type'] == 'audio' -%}
|
| 326 |
+
{{- '<|audio|>' -}}
|
| 327 |
+
{%- set ns.prev_message_type = 'audio' -%}
|
| 328 |
+
{%- elif item['type'] == 'video' -%}
|
| 329 |
+
{{- '<|video|>' -}}
|
| 330 |
+
{%- set ns.prev_message_type = 'video' -%}
|
| 331 |
+
{%- endif -%}
|
| 332 |
+
{%- endfor -%}
|
| 333 |
+
{%- endif -%}
|
| 334 |
+
|
| 335 |
+
{%- if ns.prev_message_type == 'tool_call' and not ns_tr_out.flag -%}
|
| 336 |
+
{{- '<|tool_response>' -}}
|
| 337 |
+
{%- elif not (ns_tr_out.flag and not message.get('content')) -%}
|
| 338 |
+
{{- '<turn|>\n' -}}
|
| 339 |
+
{%- endif -%}
|
| 340 |
+
{%- endif -%}
|
| 341 |
+
{%- endfor -%}
|
| 342 |
+
|
| 343 |
+
{%- if add_generation_prompt -%}
|
| 344 |
+
{%- if ns.prev_message_type != 'tool_response' and ns.prev_message_type != 'tool_call' -%}
|
| 345 |
+
{{- '<|turn>model\n' -}}
|
| 346 |
+
{%- endif -%}
|
| 347 |
+
{%- endif -%}
|
merged_model/config.json
ADDED
|
@@ -0,0 +1,197 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"Gemma4ForConditionalGeneration"
|
| 4 |
+
],
|
| 5 |
+
"audio_config": {
|
| 6 |
+
"_name_or_path": "",
|
| 7 |
+
"architectures": null,
|
| 8 |
+
"attention_chunk_size": 12,
|
| 9 |
+
"attention_context_left": 13,
|
| 10 |
+
"attention_context_right": 0,
|
| 11 |
+
"attention_invalid_logits_value": -1000000000.0,
|
| 12 |
+
"attention_logit_cap": 50.0,
|
| 13 |
+
"chunk_size_feed_forward": 0,
|
| 14 |
+
"conv_kernel_size": 5,
|
| 15 |
+
"dtype": "bfloat16",
|
| 16 |
+
"gradient_clipping": 10000000000.0,
|
| 17 |
+
"hidden_act": "silu",
|
| 18 |
+
"hidden_size": 1024,
|
| 19 |
+
"id2label": {
|
| 20 |
+
"0": "LABEL_0",
|
| 21 |
+
"1": "LABEL_1"
|
| 22 |
+
},
|
| 23 |
+
"initializer_range": 0.02,
|
| 24 |
+
"is_encoder_decoder": false,
|
| 25 |
+
"label2id": {
|
| 26 |
+
"LABEL_0": 0,
|
| 27 |
+
"LABEL_1": 1
|
| 28 |
+
},
|
| 29 |
+
"model_type": "gemma4_audio",
|
| 30 |
+
"num_attention_heads": 8,
|
| 31 |
+
"num_hidden_layers": 12,
|
| 32 |
+
"output_attentions": false,
|
| 33 |
+
"output_hidden_states": false,
|
| 34 |
+
"output_proj_dims": 1536,
|
| 35 |
+
"problem_type": null,
|
| 36 |
+
"residual_weight": 0.5,
|
| 37 |
+
"return_dict": true,
|
| 38 |
+
"rms_norm_eps": 1e-06,
|
| 39 |
+
"subsampling_conv_channels": [
|
| 40 |
+
128,
|
| 41 |
+
32
|
| 42 |
+
],
|
| 43 |
+
"use_clipped_linears": true
|
| 44 |
+
},
|
| 45 |
+
"audio_token_id": 258881,
|
| 46 |
+
"boa_token_id": 256000,
|
| 47 |
+
"boi_token_id": 255999,
|
| 48 |
+
"dtype": "bfloat16",
|
| 49 |
+
"eoa_token_id": 258883,
|
| 50 |
+
"eoa_token_index": 258883,
|
| 51 |
+
"eoi_token_id": 258882,
|
| 52 |
+
"eos_token_id": [
|
| 53 |
+
1,
|
| 54 |
+
106
|
| 55 |
+
],
|
| 56 |
+
"image_token_id": 258880,
|
| 57 |
+
"initializer_range": 0.02,
|
| 58 |
+
"model_type": "gemma4",
|
| 59 |
+
"text_config": {
|
| 60 |
+
"attention_bias": false,
|
| 61 |
+
"attention_dropout": 0.0,
|
| 62 |
+
"attention_k_eq_v": false,
|
| 63 |
+
"bos_token_id": 2,
|
| 64 |
+
"dtype": "bfloat16",
|
| 65 |
+
"enable_moe_block": false,
|
| 66 |
+
"eos_token_id": 1,
|
| 67 |
+
"expert_intermediate_size": null,
|
| 68 |
+
"final_logit_softcapping": 30.0,
|
| 69 |
+
"global_head_dim": 512,
|
| 70 |
+
"head_dim": 256,
|
| 71 |
+
"hidden_activation": "gelu_pytorch_tanh",
|
| 72 |
+
"hidden_size": 2560,
|
| 73 |
+
"hidden_size_per_layer_input": 256,
|
| 74 |
+
"initializer_range": 0.02,
|
| 75 |
+
"intermediate_size": 10240,
|
| 76 |
+
"layer_types": [
|
| 77 |
+
"sliding_attention",
|
| 78 |
+
"sliding_attention",
|
| 79 |
+
"sliding_attention",
|
| 80 |
+
"sliding_attention",
|
| 81 |
+
"sliding_attention",
|
| 82 |
+
"full_attention",
|
| 83 |
+
"sliding_attention",
|
| 84 |
+
"sliding_attention",
|
| 85 |
+
"sliding_attention",
|
| 86 |
+
"sliding_attention",
|
| 87 |
+
"sliding_attention",
|
| 88 |
+
"full_attention",
|
| 89 |
+
"sliding_attention",
|
| 90 |
+
"sliding_attention",
|
| 91 |
+
"sliding_attention",
|
| 92 |
+
"sliding_attention",
|
| 93 |
+
"sliding_attention",
|
| 94 |
+
"full_attention",
|
| 95 |
+
"sliding_attention",
|
| 96 |
+
"sliding_attention",
|
| 97 |
+
"sliding_attention",
|
| 98 |
+
"sliding_attention",
|
| 99 |
+
"sliding_attention",
|
| 100 |
+
"full_attention",
|
| 101 |
+
"sliding_attention",
|
| 102 |
+
"sliding_attention",
|
| 103 |
+
"sliding_attention",
|
| 104 |
+
"sliding_attention",
|
| 105 |
+
"sliding_attention",
|
| 106 |
+
"full_attention",
|
| 107 |
+
"sliding_attention",
|
| 108 |
+
"sliding_attention",
|
| 109 |
+
"sliding_attention",
|
| 110 |
+
"sliding_attention",
|
| 111 |
+
"sliding_attention",
|
| 112 |
+
"full_attention",
|
| 113 |
+
"sliding_attention",
|
| 114 |
+
"sliding_attention",
|
| 115 |
+
"sliding_attention",
|
| 116 |
+
"sliding_attention",
|
| 117 |
+
"sliding_attention",
|
| 118 |
+
"full_attention"
|
| 119 |
+
],
|
| 120 |
+
"max_position_embeddings": 131072,
|
| 121 |
+
"model_type": "gemma4_text",
|
| 122 |
+
"num_attention_heads": 8,
|
| 123 |
+
"num_experts": null,
|
| 124 |
+
"num_global_key_value_heads": null,
|
| 125 |
+
"num_hidden_layers": 42,
|
| 126 |
+
"num_key_value_heads": 2,
|
| 127 |
+
"num_kv_shared_layers": 18,
|
| 128 |
+
"pad_token_id": 0,
|
| 129 |
+
"rms_norm_eps": 1e-06,
|
| 130 |
+
"rope_parameters": {
|
| 131 |
+
"full_attention": {
|
| 132 |
+
"partial_rotary_factor": 0.25,
|
| 133 |
+
"rope_theta": 1000000.0,
|
| 134 |
+
"rope_type": "proportional"
|
| 135 |
+
},
|
| 136 |
+
"sliding_attention": {
|
| 137 |
+
"rope_theta": 10000.0,
|
| 138 |
+
"rope_type": "default"
|
| 139 |
+
}
|
| 140 |
+
},
|
| 141 |
+
"sliding_window": 512,
|
| 142 |
+
"tie_word_embeddings": true,
|
| 143 |
+
"top_k_experts": null,
|
| 144 |
+
"use_bidirectional_attention": null,
|
| 145 |
+
"use_cache": true,
|
| 146 |
+
"use_double_wide_mlp": false,
|
| 147 |
+
"vocab_size": 262144,
|
| 148 |
+
"vocab_size_per_layer_input": 262144
|
| 149 |
+
},
|
| 150 |
+
"tie_word_embeddings": true,
|
| 151 |
+
"transformers_version": "5.5.0.dev0",
|
| 152 |
+
"video_token_id": 258884,
|
| 153 |
+
"vision_config": {
|
| 154 |
+
"_name_or_path": "",
|
| 155 |
+
"architectures": null,
|
| 156 |
+
"attention_bias": false,
|
| 157 |
+
"attention_dropout": 0.0,
|
| 158 |
+
"chunk_size_feed_forward": 0,
|
| 159 |
+
"default_output_length": 280,
|
| 160 |
+
"dtype": "bfloat16",
|
| 161 |
+
"global_head_dim": 64,
|
| 162 |
+
"head_dim": 64,
|
| 163 |
+
"hidden_activation": "gelu_pytorch_tanh",
|
| 164 |
+
"hidden_size": 768,
|
| 165 |
+
"id2label": {
|
| 166 |
+
"0": "LABEL_0",
|
| 167 |
+
"1": "LABEL_1"
|
| 168 |
+
},
|
| 169 |
+
"initializer_range": 0.02,
|
| 170 |
+
"intermediate_size": 3072,
|
| 171 |
+
"is_encoder_decoder": false,
|
| 172 |
+
"label2id": {
|
| 173 |
+
"LABEL_0": 0,
|
| 174 |
+
"LABEL_1": 1
|
| 175 |
+
},
|
| 176 |
+
"max_position_embeddings": 131072,
|
| 177 |
+
"model_type": "gemma4_vision",
|
| 178 |
+
"num_attention_heads": 12,
|
| 179 |
+
"num_hidden_layers": 16,
|
| 180 |
+
"num_key_value_heads": 12,
|
| 181 |
+
"output_attentions": false,
|
| 182 |
+
"output_hidden_states": false,
|
| 183 |
+
"patch_size": 16,
|
| 184 |
+
"pooling_kernel_size": 3,
|
| 185 |
+
"position_embedding_size": 10240,
|
| 186 |
+
"problem_type": null,
|
| 187 |
+
"return_dict": true,
|
| 188 |
+
"rms_norm_eps": 1e-06,
|
| 189 |
+
"rope_parameters": {
|
| 190 |
+
"rope_theta": 100.0,
|
| 191 |
+
"rope_type": "default"
|
| 192 |
+
},
|
| 193 |
+
"standardize": false,
|
| 194 |
+
"use_clipped_linears": true
|
| 195 |
+
},
|
| 196 |
+
"vision_soft_tokens_per_image": 280
|
| 197 |
+
}
|
merged_model/generation_config.json
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token_id": 2,
|
| 3 |
+
"do_sample": true,
|
| 4 |
+
"eos_token_id": [
|
| 5 |
+
1,
|
| 6 |
+
106,
|
| 7 |
+
50
|
| 8 |
+
],
|
| 9 |
+
"pad_token_id": 0,
|
| 10 |
+
"temperature": 1.0,
|
| 11 |
+
"top_k": 64,
|
| 12 |
+
"top_p": 0.95,
|
| 13 |
+
"transformers_version": "5.5.0.dev0"
|
| 14 |
+
}
|
merged_model/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cfbd3d2f1cd71bd471c37fe2bf8546d5028d41e5736f64e1ca6c6b8893125503
|
| 3 |
+
size 15992595884
|
merged_model/processor_config.json
ADDED
|
@@ -0,0 +1,75 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"audio_ms_per_token": 40,
|
| 3 |
+
"audio_seq_length": 750,
|
| 4 |
+
"feature_extractor": {
|
| 5 |
+
"dither": 0.0,
|
| 6 |
+
"feature_extractor_type": "Gemma4AudioFeatureExtractor",
|
| 7 |
+
"feature_size": 128,
|
| 8 |
+
"fft_length": 512,
|
| 9 |
+
"fft_overdrive": false,
|
| 10 |
+
"frame_length": 320,
|
| 11 |
+
"hop_length": 160,
|
| 12 |
+
"input_scale_factor": 1.0,
|
| 13 |
+
"max_frequency": 8000.0,
|
| 14 |
+
"mel_floor": 0.001,
|
| 15 |
+
"min_frequency": 0.0,
|
| 16 |
+
"padding_side": "right",
|
| 17 |
+
"padding_value": 0.0,
|
| 18 |
+
"per_bin_mean": null,
|
| 19 |
+
"per_bin_stddev": null,
|
| 20 |
+
"preemphasis": 0.0,
|
| 21 |
+
"preemphasis_htk_flavor": true,
|
| 22 |
+
"return_attention_mask": true,
|
| 23 |
+
"sampling_rate": 16000
|
| 24 |
+
},
|
| 25 |
+
"image_processor": {
|
| 26 |
+
"do_convert_rgb": true,
|
| 27 |
+
"do_normalize": false,
|
| 28 |
+
"do_rescale": true,
|
| 29 |
+
"do_resize": true,
|
| 30 |
+
"image_mean": [
|
| 31 |
+
0.0,
|
| 32 |
+
0.0,
|
| 33 |
+
0.0
|
| 34 |
+
],
|
| 35 |
+
"image_processor_type": "Gemma4ImageProcessor",
|
| 36 |
+
"image_seq_length": 280,
|
| 37 |
+
"image_std": [
|
| 38 |
+
1.0,
|
| 39 |
+
1.0,
|
| 40 |
+
1.0
|
| 41 |
+
],
|
| 42 |
+
"max_soft_tokens": 280,
|
| 43 |
+
"patch_size": 16,
|
| 44 |
+
"pooling_kernel_size": 3,
|
| 45 |
+
"resample": 3,
|
| 46 |
+
"rescale_factor": 0.00392156862745098
|
| 47 |
+
},
|
| 48 |
+
"image_seq_length": 280,
|
| 49 |
+
"processor_class": "Gemma4Processor",
|
| 50 |
+
"video_processor": {
|
| 51 |
+
"do_convert_rgb": true,
|
| 52 |
+
"do_normalize": true,
|
| 53 |
+
"do_rescale": true,
|
| 54 |
+
"do_resize": true,
|
| 55 |
+
"do_sample_frames": true,
|
| 56 |
+
"image_mean": [
|
| 57 |
+
0.0,
|
| 58 |
+
0.0,
|
| 59 |
+
0.0
|
| 60 |
+
],
|
| 61 |
+
"image_std": [
|
| 62 |
+
1.0,
|
| 63 |
+
1.0,
|
| 64 |
+
1.0
|
| 65 |
+
],
|
| 66 |
+
"max_soft_tokens": 70,
|
| 67 |
+
"num_frames": 32,
|
| 68 |
+
"patch_size": 16,
|
| 69 |
+
"pooling_kernel_size": 3,
|
| 70 |
+
"resample": 3,
|
| 71 |
+
"rescale_factor": 0.00392156862745098,
|
| 72 |
+
"return_metadata": false,
|
| 73 |
+
"video_processor_type": "Gemma4VideoProcessor"
|
| 74 |
+
}
|
| 75 |
+
}
|
merged_model/tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
|
| 3 |
+
size 32169626
|
merged_model/tokenizer_config.json
ADDED
|
@@ -0,0 +1,74 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"audio_token": "<|audio|>",
|
| 3 |
+
"backend": "tokenizers",
|
| 4 |
+
"boa_token": "<|audio>",
|
| 5 |
+
"boi_token": "<|image>",
|
| 6 |
+
"bos_token": "<bos>",
|
| 7 |
+
"eoa_token": "<audio|>",
|
| 8 |
+
"eoc_token": "<channel|>",
|
| 9 |
+
"eoi_token": "<image|>",
|
| 10 |
+
"eos_token": "<eos>",
|
| 11 |
+
"eot_token": "<turn|>",
|
| 12 |
+
"escape_token": "<|\"|>",
|
| 13 |
+
"etc_token": "<tool_call|>",
|
| 14 |
+
"etd_token": "<tool|>",
|
| 15 |
+
"etr_token": "<tool_response|>",
|
| 16 |
+
"extra_special_tokens": [
|
| 17 |
+
"<|video|>"
|
| 18 |
+
],
|
| 19 |
+
"image_token": "<|image|>",
|
| 20 |
+
"mask_token": "<mask>",
|
| 21 |
+
"model_max_length": 1000000000000000019884624838656,
|
| 22 |
+
"pad_token": "<pad>",
|
| 23 |
+
"padding_side": "left",
|
| 24 |
+
"processor_class": "Gemma4Processor",
|
| 25 |
+
"response_schema": {
|
| 26 |
+
"type": "object",
|
| 27 |
+
"properties": {
|
| 28 |
+
"role": {
|
| 29 |
+
"const": "assistant"
|
| 30 |
+
},
|
| 31 |
+
"thinking": {
|
| 32 |
+
"type": "string"
|
| 33 |
+
},
|
| 34 |
+
"content": {
|
| 35 |
+
"type": "string"
|
| 36 |
+
},
|
| 37 |
+
"tool_calls": {
|
| 38 |
+
"x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>",
|
| 39 |
+
"type": "array",
|
| 40 |
+
"items": {
|
| 41 |
+
"type": "object",
|
| 42 |
+
"properties": {
|
| 43 |
+
"type": {
|
| 44 |
+
"const": "function"
|
| 45 |
+
},
|
| 46 |
+
"function": {
|
| 47 |
+
"type": "object",
|
| 48 |
+
"x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})",
|
| 49 |
+
"properties": {
|
| 50 |
+
"name": {
|
| 51 |
+
"type": "string"
|
| 52 |
+
},
|
| 53 |
+
"arguments": {
|
| 54 |
+
"type": "object",
|
| 55 |
+
"x-parser": "gemma4-tool-call",
|
| 56 |
+
"additionalProperties": {}
|
| 57 |
+
}
|
| 58 |
+
}
|
| 59 |
+
}
|
| 60 |
+
}
|
| 61 |
+
}
|
| 62 |
+
}
|
| 63 |
+
},
|
| 64 |
+
"x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?P<content>(?:(?!\\<turn\\|\\>)(?!\\<\\|tool_response\\>).)+)?(?:\\<turn\\|\\>|\\<\\|tool_response\\>)?"
|
| 65 |
+
},
|
| 66 |
+
"soc_token": "<|channel>",
|
| 67 |
+
"sot_token": "<|turn>",
|
| 68 |
+
"stc_token": "<|tool_call>",
|
| 69 |
+
"std_token": "<|tool>",
|
| 70 |
+
"str_token": "<|tool_response>",
|
| 71 |
+
"think_token": "<|think|>",
|
| 72 |
+
"tokenizer_class": "GemmaTokenizer",
|
| 73 |
+
"unk_token": "<unk>"
|
| 74 |
+
}
|
ollama/Modelfile
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
FROM ../merged_model
|
| 2 |
+
TEMPLATE """{{ if .System }}<|im_start|>system
|
| 3 |
+
{{ .System }}<|im_end|>
|
| 4 |
+
{{ end }}{{ if .Prompt }}<|im_start|>user
|
| 5 |
+
{{ .Prompt }}<|im_end|>
|
| 6 |
+
{{ end }}<|im_start|>assistant
|
| 7 |
+
"""
|
| 8 |
+
PARAMETER temperature 0.1
|
| 9 |
+
PARAMETER num_ctx 8192
|
| 10 |
+
PARAMETER stop "<|im_end|>"
|
| 11 |
+
PARAMETER stop "<|endoftext|>"
|
ollama/create_ollama_model.ps1
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
param(
|
| 2 |
+
[string]$ModelName = "lumynax-infused-gemma-e4b"
|
| 3 |
+
)
|
| 4 |
+
|
| 5 |
+
$ErrorActionPreference = "Stop"
|
| 6 |
+
Set-StrictMode -Version Latest
|
| 7 |
+
|
| 8 |
+
$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
|
| 9 |
+
$modelfilePath = Join-Path $scriptDir "Modelfile"
|
| 10 |
+
|
| 11 |
+
if (-not (Get-Command ollama -ErrorAction SilentlyContinue)) {
|
| 12 |
+
throw "The `ollama` CLI is not installed. Install Ollama first."
|
| 13 |
+
}
|
| 14 |
+
|
| 15 |
+
& ollama create $ModelName -f $modelfilePath
|
| 16 |
+
if ($LASTEXITCODE -ne 0) {
|
| 17 |
+
exit $LASTEXITCODE
|
| 18 |
+
}
|
| 19 |
+
|
| 20 |
+
Write-Output "Created Ollama model: $ModelName"
|
| 21 |
+
Write-Output "Run it with: ollama run $ModelName"
|
quickstart.py
ADDED
|
@@ -0,0 +1,110 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import argparse
|
| 4 |
+
import json
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
|
| 7 |
+
import torch
|
| 8 |
+
from transformers import AutoModelForMultimodalLM, AutoProcessor
|
| 9 |
+
|
| 10 |
+
MODEL_TITLE = "LumynaX Infused Gemma E4B Model"
|
| 11 |
+
SUPPORTED_MODALITIES = ('text', 'image', 'audio')
|
| 12 |
+
DEFAULT_ENABLE_THINKING = True
|
| 13 |
+
DEFAULT_SYSTEM_PROMPT = 'You are LumynaX operating from the LumynaX Infused Gemma E4B Model package identity. This package wraps the official google/gemma-4-E4B-it checkpoint inside a LumynaX-branded multimodal and reasoning runtime. Always identify yourself as LumynaX when asked who you are. Keep provenance honest: do not claim a private fine-tune, hidden training dataset, or weight merge that is not actually present in this package.'
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
def _build_parser() -> argparse.ArgumentParser:
|
| 17 |
+
parser = argparse.ArgumentParser(
|
| 18 |
+
description=(
|
| 19 |
+
f"Run a local Gemma E4B quickstart for {MODEL_TITLE}. "
|
| 20 |
+
f"Supported modalities: text, image, audio."
|
| 21 |
+
)
|
| 22 |
+
)
|
| 23 |
+
parser.add_argument("--mode", choices=["text", "image", "audio"], default="text")
|
| 24 |
+
parser.add_argument(
|
| 25 |
+
"--prompt",
|
| 26 |
+
default="Explain in two short bullet points what this local package is.",
|
| 27 |
+
help="Text instruction to send to the model.",
|
| 28 |
+
)
|
| 29 |
+
parser.add_argument("--image", default="", help="Local image path or image URL for --mode image.")
|
| 30 |
+
parser.add_argument("--audio", default="", help="Local audio path or audio URL for --mode audio.")
|
| 31 |
+
parser.add_argument("--max-new-tokens", type=int, default=256)
|
| 32 |
+
parser.add_argument(
|
| 33 |
+
"--thinking",
|
| 34 |
+
action=argparse.BooleanOptionalAction,
|
| 35 |
+
default=DEFAULT_ENABLE_THINKING,
|
| 36 |
+
help="Enable Gemma reasoning mode.",
|
| 37 |
+
)
|
| 38 |
+
return parser
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
def _message_content(args: argparse.Namespace) -> list[dict[str, str]]:
|
| 42 |
+
if args.mode == "text":
|
| 43 |
+
return [{"type": "text", "text": args.prompt}]
|
| 44 |
+
if args.mode == "image":
|
| 45 |
+
if not args.image:
|
| 46 |
+
raise SystemExit("--image is required when --mode image is used.")
|
| 47 |
+
image_ref = args.image.strip()
|
| 48 |
+
return [
|
| 49 |
+
{"type": "image", "url": image_ref},
|
| 50 |
+
{"type": "text", "text": args.prompt},
|
| 51 |
+
]
|
| 52 |
+
if not args.audio:
|
| 53 |
+
raise SystemExit("--audio is required when --mode audio is used.")
|
| 54 |
+
audio_ref = args.audio.strip()
|
| 55 |
+
return [
|
| 56 |
+
{"type": "audio", "audio": audio_ref},
|
| 57 |
+
{"type": "text", "text": args.prompt},
|
| 58 |
+
]
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
def main() -> None:
|
| 62 |
+
args = _build_parser().parse_args()
|
| 63 |
+
model_dir = Path(__file__).resolve().parent / "merged_model"
|
| 64 |
+
if not model_dir.exists():
|
| 65 |
+
raise SystemExit(f"Expected merged_model/ at {model_dir}")
|
| 66 |
+
|
| 67 |
+
processor = AutoProcessor.from_pretrained(model_dir)
|
| 68 |
+
model = AutoModelForMultimodalLM.from_pretrained(
|
| 69 |
+
model_dir,
|
| 70 |
+
dtype="auto",
|
| 71 |
+
device_map="auto",
|
| 72 |
+
)
|
| 73 |
+
messages = [
|
| 74 |
+
{
|
| 75 |
+
"role": "system",
|
| 76 |
+
"content": [
|
| 77 |
+
{
|
| 78 |
+
"type": "text",
|
| 79 |
+
"text": DEFAULT_SYSTEM_PROMPT,
|
| 80 |
+
}
|
| 81 |
+
],
|
| 82 |
+
},
|
| 83 |
+
{
|
| 84 |
+
"role": "user",
|
| 85 |
+
"content": _message_content(args),
|
| 86 |
+
},
|
| 87 |
+
]
|
| 88 |
+
inputs = processor.apply_chat_template(
|
| 89 |
+
messages,
|
| 90 |
+
tokenize=True,
|
| 91 |
+
return_dict=True,
|
| 92 |
+
return_tensors="pt",
|
| 93 |
+
add_generation_prompt=True,
|
| 94 |
+
enable_thinking=args.thinking,
|
| 95 |
+
).to(model.device)
|
| 96 |
+
input_len = inputs["input_ids"].shape[-1]
|
| 97 |
+
|
| 98 |
+
with torch.inference_mode():
|
| 99 |
+
outputs = model.generate(**inputs, max_new_tokens=args.max_new_tokens)
|
| 100 |
+
|
| 101 |
+
response = processor.decode(outputs[0][input_len:], skip_special_tokens=False)
|
| 102 |
+
parsed = processor.parse_response(response) if hasattr(processor, "parse_response") else response
|
| 103 |
+
if isinstance(parsed, str):
|
| 104 |
+
print(parsed)
|
| 105 |
+
return
|
| 106 |
+
print(json.dumps(parsed, indent=2, ensure_ascii=False))
|
| 107 |
+
|
| 108 |
+
|
| 109 |
+
if __name__ == "__main__":
|
| 110 |
+
main()
|
release_export_manifest.json
ADDED
|
@@ -0,0 +1,79 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"artifacts": {
|
| 3 |
+
"checksums": "checksums.sha256",
|
| 4 |
+
"gguf": null,
|
| 5 |
+
"hf_space_app": "hf_space/app.py",
|
| 6 |
+
"hf_space_dir": "hf_space",
|
| 7 |
+
"hf_space_readme": "hf_space/README.md",
|
| 8 |
+
"hf_space_requirements": "hf_space/requirements.txt",
|
| 9 |
+
"license": "LICENSE.txt",
|
| 10 |
+
"lumynax_package_identity": "merged_model/LUMYNAX_PACKAGE_IDENTITY.txt",
|
| 11 |
+
"merged_model": "merged_model",
|
| 12 |
+
"ollama_create_script": "ollama/create_ollama_model.ps1",
|
| 13 |
+
"ollama_modelfile": "ollama/Modelfile",
|
| 14 |
+
"quantized_gguf": null,
|
| 15 |
+
"quickstart": "quickstart.py",
|
| 16 |
+
"readme": "README.md",
|
| 17 |
+
"requirements": "requirements.txt",
|
| 18 |
+
"training_summary": "artifacts/release_training_summary.json",
|
| 19 |
+
"version": "VERSION.txt"
|
| 20 |
+
},
|
| 21 |
+
"capabilities": {
|
| 22 |
+
"reasoning_enabled": true,
|
| 23 |
+
"supported_modalities": [
|
| 24 |
+
"text",
|
| 25 |
+
"image",
|
| 26 |
+
"audio"
|
| 27 |
+
]
|
| 28 |
+
},
|
| 29 |
+
"delivery": "standalone_hf_base_release",
|
| 30 |
+
"distribution": {
|
| 31 |
+
"hf_space": {
|
| 32 |
+
"app": "hf_space/app.py",
|
| 33 |
+
"default_model_repo_id": "AbteeXAILab/lumynax-infused-gemma-e4b",
|
| 34 |
+
"directory": "hf_space",
|
| 35 |
+
"model_repo_env_var": "LUMYNAX_MODEL_REPO_ID",
|
| 36 |
+
"readme": "hf_space/README.md",
|
| 37 |
+
"requirements": "hf_space/requirements.txt",
|
| 38 |
+
"status": "private_reasoning_multimodal_demo"
|
| 39 |
+
},
|
| 40 |
+
"ollama": {
|
| 41 |
+
"create_script": "ollama/create_ollama_model.ps1",
|
| 42 |
+
"modelfile": "ollama/Modelfile",
|
| 43 |
+
"recommended_model_name": "lumynax-infused-gemma-e4b",
|
| 44 |
+
"status": "not_validated_for_multimodal_base_release"
|
| 45 |
+
}
|
| 46 |
+
},
|
| 47 |
+
"family": null,
|
| 48 |
+
"generated_at": "2026-04-11T23:33:11.251732+00:00",
|
| 49 |
+
"lumynax_package_identity": {
|
| 50 |
+
"chat_template": "merged_model/chat_template.jinja",
|
| 51 |
+
"hardcoded": true,
|
| 52 |
+
"identity_file": "merged_model/LUMYNAX_PACKAGE_IDENTITY.txt",
|
| 53 |
+
"runtime_wrappers": [
|
| 54 |
+
"quickstart.py",
|
| 55 |
+
"hf_space/app.py"
|
| 56 |
+
]
|
| 57 |
+
},
|
| 58 |
+
"manifest_version": 2,
|
| 59 |
+
"model_title": "LumynaX Infused Gemma E4B Model",
|
| 60 |
+
"package_state": "base_weights_hydrated",
|
| 61 |
+
"public_identity": {
|
| 62 |
+
"model_name": "LumynaX",
|
| 63 |
+
"organization": "AbteeX AI Labs",
|
| 64 |
+
"region": "Aotearoa New Zealand"
|
| 65 |
+
},
|
| 66 |
+
"release_version": "v1",
|
| 67 |
+
"runtime": {
|
| 68 |
+
"delivery_mode": "standalone_base_release",
|
| 69 |
+
"preferred_backend": "transformers_multimodal",
|
| 70 |
+
"prompt_format": "huggingface_chat_template",
|
| 71 |
+
"quickstart_command": "python quickstart.py"
|
| 72 |
+
},
|
| 73 |
+
"upstream_model": {
|
| 74 |
+
"kind": "official_base_weights",
|
| 75 |
+
"lumynax_weight_adaptation_applied": false,
|
| 76 |
+
"provider": "Hugging Face",
|
| 77 |
+
"repo_id": "google/gemma-4-E4B-it"
|
| 78 |
+
}
|
| 79 |
+
}
|
requirements.txt
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
accelerate>=1.13
|
| 2 |
+
huggingface-hub>=1.8
|
| 3 |
+
numba>=0.65
|
| 4 |
+
pillow>=10.0
|
| 5 |
+
safetensors>=0.6
|
| 6 |
+
torch>=2.9
|
| 7 |
+
torchvision>=0.24
|
| 8 |
+
transformers>=5.5.3
|
| 9 |
+
librosa>=0.11
|