Omni
Collection
5 items • Updated • 2
Omni-Embed Mini 0.9B, ONNX
ONNX export of the Omni-Embed Mini 0.9B multimodal embedding model, built to run in a browser on WebGPU. One model embeds text, images, video, speech, general audio and document pages into a single 1024-dimensional space.
- Project page: https://omniembed.cvmbzuai.com/
- Live demo: https://demo-omniembed.cvmbzuai.com/
- Paper: https://huggingface.co/papers/2610.02148
- GitHub: https://github.com/k-m-irfan/Omni-Embed-Mini
What is in here
Two directories, one per precision, each a complete bundle with the same file
names, so a client picks one by changing a base URL. fp16/ is the default at
1.89 GB. q8/ is 1.39 GB and about half the memory in use, for devices that
cannot hold fp16; measured on a 6,000 item index it returns the same best result
for 12 of 14 queries and the same 9.8 of the top 10.
Each is a six-graph bundle driven from JavaScript rather than a transformers.js
architecture: the splice, the pooling and two of the vision tower's three inputs
are computed by the caller. See manifest.json for the preconditions a caller
cannot read off the graphs.
| Component | Graph |
|---|---|
| backbone | backbone/model.onnx, takes inputs_embeds |
| embedding table | backbone/embed_tokens.onnx |
| vision | vision_encoder/model.onnx, one image per call |
| audio | whisper_encoder/, dasheng_encoder/ |
| projectors | projectors/*.onnx |
Every component was parity-checked against its PyTorch reference before export.
The measured numbers are in */conversion_metadata.json and
*/parity_report.json.
Preprocessing is part of the model: a still image is squared to 224 with PIL bilinear and then taken to 256 with a Pillow-compatible bicubic, and substituting a browser canvas resize moves the final embedding by cos 0.734, so a client that does not reproduce that chain is a different model and its vectors must not be mixed into an index built with this one.
Citation
@misc{kurpath2026omniembedminibindingmodalitiesforgetting,
title={Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation},
author={Mohammed Irfan Kurpath and Jaseel Muhammad Kaithakkodan and Sahal Shaji Mullappilly and Ivan Laptev and Hisham Cholakkal},
year={2026},
eprint={2610.02148},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2610.02148},
}