Add files using upload-large-folder tool

d69d96a verified 23 days ago

5.78 kB

tags:
  - blackhole
  - p150
  - tt-dit-server
  - tt-model-cache
  - tt-model-container

superpoint-p150

SuperPoint (Magic Leap's self-supervised interest-point detector and descriptor) running entirely on one Tenstorrent Blackhole p150a via tt-nn: image in, keypoints with scores and 256-d descriptors out. Weights: magic-leap-community/superpoint · Paper: arXiv:1712.07629 · Upstream code: magicleap/SuperPointPretrainedNetwork · Port: changh95/tt-superpoint

Runs on p150 (mesh P150).

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart

tt-model pull  changh95/superpoint-p150 --with-weights
tt-model serve changh95/superpoint-p150

pull --with-weights downloads the Docker image and the magic-leap-community/superpoint weights at 734450e9ffe229074f5998494ddc615475cdb20a (into your HF cache; they are not in the image). serve starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.

Run with tt-cli

tt serve changh95/superpoint-p150
printf '{"image":"%s"}' "$(base64 -w0 code/sample_data/house_in_field_1080p.jpg)" > req.json
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
tt model stop changh95/superpoint-p150

POST /predict: image (base64 PNG/JPEG); optional max_keypoints (1024, -1 = all above threshold), keypoint_threshold (0.005), nms_radius (4), return_descriptors (true).
GET /health, GET /info.

Response

{"num_keypoints": 539,
 "keypoints": [[610.0, 703.125], [1042.5, 446.25], [1122.5, 442.5]],
 "scores": [0.609375, 0.589844, 0.582031],
 "original_size": {"height": 900, "width": 1600}, "image_size": {"height": 480, "width": 640}, "scale": {"x": 2.5, "y": 1.875},
 "descriptors": {"format": "npz", "key": "descriptors", "dtype": "float16", "shape": [539, 256], "data": "..."},
 "serving_path": {"traced": true, "device_nms": true},
 "timing_ms": {"preprocess": 17.7, "device_forward": 5.3, "postprocess": 1.3, "total": 24.3}}

keypoints are [x, y] in original image pixels, sorted by descending scores; the network frame is 480×640 and scale = original / network.
descriptors.data is a base64 NPZ: np.load(io.BytesIO(base64.b64decode(data)))["descriptors"] gives (N, 256) float16 rows, L2-normalised, in keypoint order.

Demo

Top-500 keypoints on the 480×640 network frame of `code/sample_data/house_in_field_1080p.jpg` (`media/sample.png`, natural image)

Accuracy and speed

Metric	Value
Pre-NMS score map · descriptor map PCC vs fp32 torch reference	0.9971 · 0.9991
Keypoint set vs reference (natural image, top-500, 2 px)	recall 98.20% · precision 99.40% · F1 98.80%
Inference, served over HTTP (warm, batch 1, 480×640, 1600×900 JPEG in; median of 50 requests)	5.3 ms device (trace + device NMS) · 1.3 ms host post-processing · 18 ms JPEG decode/resize · 24.7 ms end-to-end (~40 FPS)
Same, legacy path (`TT_FUSED=0`: untraced, host NMS)	12.4 ms device · 26.5 ms host NMS · 56.8 ms end-to-end (~18 FPS)

Caveats

Every image is resized to 480×640 (bilinear, /255, channel 0); one image per request, batch 1, requests serialised on the chip.
This server runs the whole device graph as one metal trace per request with a standard-op device NMS (radius 4; bit-identical to the host single-pass NMS) and an rms_norm descriptor L2-norm (bf16-rounding-level vs the legacy chain; PCC 0.9991 either way). bf16 + HiFi2 + fp32 accumulate throughout (bfloat8/LoFi drop score PCC to ~0.91). No custom kernel: the port README's sp_eq_mul_mask path is not built into this image.
nms_radius other than 4 falls back to the host NMS on the traced scores (same keypoints, ~25 ms slower); TT_FUSED=0 in the environment restores the untraced legacy path (validated 2026-09-12).
Weights are research-only: the Magic Leap SuperPoint licence allows academic or non-profit organisation NONCOMMERCIAL research use.
Not an OpenAI-compatible API; GET /v1/models is a stub so the tt-model ready card does not 404.
Validated on tt-metal v0.78.0-dev20260820 (main 8b98410e730), single p150a only; numbers above measured 2026-09-13 through this container image (DEVICE_VALIDATION.md).

Licensing

Weights: magic-leap-community/superpoint, other (Magic Leap SuperPoint licence, noncommercial research use only); not redistributed here.
Port and serving code (code/): Apache-2.0 (SPDX headers on the modules), from changh95/tt-superpoint, distributed under the same upstream terms since a port cannot grant more than its upstream does.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	`8b98410e730bb504fea43a88609756e34821d91d`
`code/` digest	`ae768681d4aa677d` (sha256, first 16 hex digits)
built	2026-09-13T15:21:44+00:00 by tt-model 0.1.0