CLIP ViT-H/14 image and text embeddings on the AMD NPU

An IRON export of laion/CLIP-ViT-H-14-laion2B-s32B-b79K for AMD Ryzen AI NPUs: the compiled NPU kernels (.xclbin + instruction streams) and the packed weights the taconite-clip Rust runtime replays, over XRT or directly over the amdxdna driver.

The kernels are compiled for NPU2 (AIE2P: Strix Point, Strix Halo, Krackan) and will not load on NPU1 (Phoenix, Hawk Point).

Download

hf download brishen/iron-clip-vit-h14-npu2 --local-dir clip-vit-h14
# or, from an IRON checkout:
python scripts/hf_models.py download clip-vit-h14 --repo brishen/iron-clip-vit-h14-npu2 --out clip-vit-h14

Usage

cargo install taconite-clip             # over XRT
# or, with no XRT at all (straight to the amdxdna driver):
cargo install taconite-clip --no-default-features --features cli,direct
clip classify clip-vit-h14 cats.jpg car.png --label cat --label dog --label car
clip embed clip-vit-h14 --image a.jpg --text "a photo of a cat" -o emb.f32
clip check clip-vit-h14                        # every stage against the references

See taconite-clip (docs, source) for the full API. The bundle is exported by IRON's iron/applications/clip_vit_h14.

Provenance

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for brishen/iron-clip-vit-h14-npu2

Finetuned
(12)
this model