Add files using upload-large-folder tool

README.md

@@ -1,28 +1,16 @@
 ---
 tags:
 - blackhole
-- keypoint-detection
 - p150
-- superpoint
-- tenstorrent
 - tt-dit-server
-- tt-metal
 - tt-model-cache
-- tt-model-catalog
 - tt-model-container
-- tt-nn
-- ttnn
-base_model:
-- magic-leap-community/superpoint
-license: other
-license_name: magic-leap-superpoint
-license_link: https://huggingface.co/magic-leap-community/superpoint
-pipeline_tag: keypoint-detection
 ---
 
 # superpoint-p150
 
-SuperPoint (magic-leap-community/superpoint) keypoint detection and description on a single Tenstorrent Blackhole p150a via tt-nn: a base64 image in, keypoints in original pixel coordinates, scores and 256-d descriptors out, at a fixed 480x640 network input. Pre-NMS score-map PCC 0.9971 and descriptor PCC 0.9991 vs the fp32 torch reference, keypoint F1 98.8% (top-500, 2 px); this server runs the pure-ttnn untraced path with host NMS (roughly 6 fps; the README's 40.7 fps needs trace plus the fused C++ NMS kernel in kernels/, which is not built into this image). Weights are under the Magic Leap SuperPoint licence: academic or non-profit organisation NONCOMMERCIAL research use only. Port source: github.com/changh95/tt-superpoint @ e1eab66e29ff424bc9af6b1118671d9bc08e899e.
+SuperPoint (Magic Leap's self-supervised interest-point detector and descriptor) running entirely on one Tenstorrent Blackhole p150a via tt-nn: image in, keypoints with scores and 256-d descriptors out.
+Weights: [magic-leap-community/superpoint](https://huggingface.co/magic-leap-community/superpoint) · Paper: [arXiv:1712.07629](https://arxiv.org/abs/1712.07629) · Upstream code: [magicleap/SuperPointPretrainedNetwork](https://github.com/magicleap/SuperPointPretrainedNetwork) · Port: [changh95/tt-superpoint](https://github.com/changh95/tt-superpoint)
 
 Runs on **p150** (mesh `P150`).
 
@@ -37,119 +25,58 @@ tt-model serve changh95/superpoint-p150
 
 `pull --with-weights` downloads the Docker image and the [`magic-leap-community/superpoint`](https://huggingface.co/magic-leap-community/superpoint) weights at `734450e9ffe229074f5998494ddc615475cdb20a` (into your HF cache; they are not in the image). `serve` starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs `Application startup complete`.
 
-### With tt-cli
+### Run with tt-cli
 
 ```bash
-tt serve changh95/superpoint-p150          # pulls image + weights, boots, prints the port
+tt serve changh95/superpoint-p150
+printf '{"image":"%s"}' "$(base64 -w0 code/sample_data/house_in_field_1080p.jpg)" > req.json
+curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
 tt model stop changh95/superpoint-p150
 ```
 
-The port is the one `serve` printed (20000, or the next free one). This is **not** an
-OpenAI-style server: `tt-model curl` / chat clients do not apply. `GET /v1/models` exists
-only so generic probes do not 404; the real routes are below.
+- `POST /predict`: `image` (base64 PNG/JPEG); optional `max_keypoints` (1024, `-1` = all above threshold), `keypoint_threshold` (0.005), `nms_radius` (4), `return_descriptors` (true).
+- `GET /health`, `GET /info`.
 
-### Call it
+### Response
 
-```bash
-PORT=20000   # the port serve printed
-curl -s localhost:$PORT/health      # {"status":"ok","model":"superpoint-p150","device":{...}}
-curl -s localhost:$PORT/info        # weights repo+revision, canonical input, defaults, limits, licence
-
-# One image -> keypoints. Any PNG/JPEG; it is resized to 640x480 server-side and the
-# keypoints come back in YOUR image's pixel coordinates (see "scale").
-IMG=code/sample_data/house_in_field_1080p.jpg       # 1600x900 natural image
-python3 - "$IMG" "$PORT" <<'EOF'
-import base64, json, sys, urllib.request
-img, port = sys.argv[1], sys.argv[2]
-req = {"image": base64.b64encode(open(img, "rb").read()).decode(),
-       "max_keypoints": 1024,          # -1 = all above threshold
-       "keypoint_threshold": 0.005,
-       "nms_radius": 4,
-       "return_descriptors": True}
-r = urllib.request.Request(f"http://127.0.0.1:{port}/predict", json.dumps(req).encode(),
-                           {"Content-Type": "application/json"})
-out = json.load(urllib.request.urlopen(r, timeout=300))
-print(out["num_keypoints"], out["keypoints"][:3], out["scores"][:3], out["timing_ms"])
-EOF
+```json
+{"num_keypoints": 539,
+ "keypoints": [[610.0, 703.125], [1042.5, 446.25], [1122.5, 442.5]],
+ "scores": [0.609375, 0.589844, 0.582031],
+ "original_size": {"height": 900, "width": 1600}, "image_size": {"height": 480, "width": 640}, "scale": {"x": 2.5, "y": 1.875},
+ "descriptors": {"format": "npz", "key": "descriptors", "dtype": "float16", "shape": [539, 256], "data": "..."},
+ "timing_ms": {"preprocess": 19.4, "device_forward": 12.5, "postprocess": 32.5, "total": 64.4}}
 ```
 
-Response fields: `num_keypoints`; `keypoints` (N x [x, y] floats, original-image pixels);
-`scores` (N, post-NMS softmax scores); `original_size`, `image_size` (480x640) and
-`scale` (x = W/640, y = H/480; divide to get network-frame coordinates);
-`params` echoed; `timing_ms` (`preprocess`, `device_forward`, `postprocess`, `total`);
-when `return_descriptors` is true, `descriptors` = `{format: "npz", key: "descriptors",
-dtype: "float16", shape: [N, 256], data: <base64 NPZ>}` -- decode with
-`numpy.load(io.BytesIO(base64.b64decode(d["data"])))["descriptors"]`; rows are
-L2-normalised. Errors: 400 undecodable image / bad field, 503 while starting, 500 with
-the exception text. One image per request; requests are serialised on the chip.
-
-A ready-made check: `python code/models/server/smoke_test.py --url http://127.0.0.1:$PORT`
-prints one PASS/FAIL line with the keypoint count and timings.
-
-### First boot
-
-Weights are 5 MB (`config.json`, `model.safetensors`, `preprocessor_config.json` at the
-pinned revision) and land in your HF cache. The first start JIT-compiles the conv /
-pool / softmax kernels (a few minutes, cached under
-`~/.cache/tt-model/superpoint-p150/cache`); the server logs `Loading weights`,
-`Warming up`, `Warmup complete` and is ready at uvicorn's `Application startup complete`.
-Later boots reuse the kernel cache. The weights repo is public and ungated (no token).
-
-### What this server runs
+- `keypoints` are `[x, y]` in original image pixels, sorted by descending `scores`; the network frame is 480×640 and `scale` = original / network.
+- `descriptors.data` is a base64 NPZ: `np.load(io.BytesIO(base64.b64decode(data)))["descriptors"]` gives `(N, 256)` float16 rows, L2-normalised, in keypoint order.
 
-Fixed 480x640 input (bilinear resize, /255, channel 0 -- the HF
-`SuperPointImageProcessor` defaults), 8 encoder convs + 3 max-pools + score and
-descriptor heads in bfloat16 activations / bfloat16 weights / HiFi2 / fp32 accumulate,
-softmax and descriptor L2-norm on device, then host single-pass NMS, threshold, border
-removal, top-k and bilinear descriptor sampling. Untraced, host NMS: this is the
-port's `SP_TRACE_NMS=0`, `SP_NO_TRACE=1` configuration. The benchmark numbers below
-that need trace or the fused `sp_eq_mul_mask` kernel are **not** what this server does.
+### Demo
 
-### Results from the port (Blackhole p150b, 480x640, batch 1, natural image)
+| Top-500 keypoints on the 480×640 network frame of `code/sample_data/house_in_field_1080p.jpg` (`media/sample.png`, natural image) |
+|:---:|
+| ![](media/sample.png) |
 
-PCC vs the Hugging Face fp32 reference:
+### Accuracy and speed
 
-| Tensor | PCC |
+| Metric | Value |
 |---|---:|
-| Pre-NMS score map | **0.9971** |
-| Descriptor map (post L2-norm) | **0.9991** |
-
-Keypoint set vs the reference (`sample_data/house_in_field_1080p.jpg`, top-500, 2 px):
-recall **98.80%**, precision **98.80%**, F1 **98.80%** (synthetic `torch.rand` input: F1 97.80%).
-
-Throughput measured by `models/tests/test_superpoint.py` (needs a tt-metal checkout as
-pytest rootdir for its `device` fixture):
-
-| Path | Device forward (input resident) | Traced fwd incl. H2D | Full e2e |
-|---|---:|---:|---:|
-| Forward-only trace, host NMS (`SP_TRACE_NMS=0`) | 355 fps (2.81 ms) | 73.6 fps | 17.0 fps |
-| Forward + device NMS, fused kernel (`SP_TRACE_NMS=1`) | 85.6 fps | 44.6 fps | **40.7 fps** |
-| Untraced, host NMS (**this server**) | ~6 fps | -- | ~5 fps |
-
-The fused path needs `kernels/sp_eq_mul_mask/` compiled into tt-metal (see its README);
-a pure-ttnn equivalent (`ttnn.eq` + `ttnn.multiply`) is measured in
-`kernels/sp_eq_mul_mask/bench.py`. `code/results.tsv` is the full experiment log
-(precision cliff: bfloat8/LoFi drop score PCC to 0.70-0.91; trace was a 10.8x unlock).
-
-Sample output (top-500 keypoints on the resized frame): ![keypoints](media/sample.png)
+| Pre-NMS score map · descriptor map PCC vs fp32 torch reference | 0.9971 · 0.9991 |
+| Keypoint set vs reference (natural image, top-500, 2 px) | recall 98.80% · precision 98.80% · F1 98.80% |
+| Inference, served over HTTP (warm, batch 1, 480×640, 1600×900 JPEG in) | ~12–20 ms device · ~65 ms end-to-end (~15 FPS) |
 
-### Layout of `code/`
+### Caveats
 
-`models/tt/superpoint_ttnn.py` (tt-nn model), `models/tt/postprocess.py` (validated host
-post-processing), `models/server/app.py` + `smoke_test.py`, `models/reference/`
-(HF reference loader, needs torchvision), `models/tests/test_superpoint.py` (benchmark +
-PCC + F1), `models/visualize.py`, `kernels/sp_eq_mul_mask/` (fused C++ Tensix NMS
-kernel, ~450 LoC), `sample_data/`, `results.tsv`, `run_benchmark.sh`.
-`models/common/lightweightmodule.py` is a schema-required filler from tt-metal.
+- Every image is resized to 480×640 (bilinear, /255, channel 0); one image per request, batch 1, requests serialised on the chip.
+- This server runs the untraced pure-ttnn path with host single-pass NMS (bf16 + HiFi2 + fp32 accumulate; bfloat8/LoFi drop score PCC to ~0.91). The port README's 40.7 fps needs `ttnn.trace` plus the fused `sp_eq_mul_mask` C++ kernel in `code/kernels/`, which is not built into this image.
+- Weights are research-only: the Magic Leap SuperPoint licence allows academic or non-profit organisation NONCOMMERCIAL research use.
+- Not an OpenAI-compatible API; `GET /v1/models` is a stub so the tt-model ready card does not 404.
+- Validated on tt-metal `v0.78.0-dev20260820` (main `8b98410e730`), single p150a only.
 
 ### Licensing
 
-The upstream **weights** (`magic-leap-community/superpoint`) carry the Magic Leap
-SuperPoint licence: *academic or non-profit organisation noncommercial research use
-only* -- see https://huggingface.co/magic-leap-community/superpoint. The port code
-(Apache-2.0 headers, by Hyunggi Chang) is published under the same terms, since a port
-cannot grant more than its upstream does. Weights are not redistributed here; they are
-fetched from the upstream repo at the pinned revision.
+- Weights: [magic-leap-community/superpoint](https://huggingface.co/magic-leap-community/superpoint), `other` ([Magic Leap SuperPoint licence](https://huggingface.co/magic-leap-community/superpoint/blob/main/LICENSE), noncommercial research use only); not redistributed here.
+- Port and serving code (`code/`): Apache-2.0 (SPDX headers on the modules), from [changh95/tt-superpoint](https://github.com/changh95/tt-superpoint), distributed under the same upstream terms since a port cannot grant more than its upstream does.
 
 ## Provenance
 
@@ -159,5 +86,5 @@ The exact sources the image was built from — `code/` in this repo is byte-iden
 | --- | --- |
 | tt-metal | [`8b98410e730bb504fea43a88609756e34821d91d`](https://github.com/tenstorrent/tt-metal/commit/8b98410e730bb504fea43a88609756e34821d91d) |
 | `code/` digest | `006ddc1eb4e69a41` (sha256, first 16 hex digits) |
-| built | 2026-09-12T10:56:19+00:00 by tt-model 0.1.0 |
+| built | 2026-09-12T13:22:54+00:00 by tt-model 0.1.0 |
 

tt_kernel_manifest.json

@@ -6,7 +6,7 @@
   "device_count": 1,
   "producer": {
     "tt_kernel_version": "0.1.0",
-    "created_at": "2026-09-12T10:59:21.045020+00:00",
+    "created_at": "2026-09-12T13:23:20.741264+00:00",
     "hostname": "deepgadget"
   },
   "weights": {
@@ -93,7 +93,7 @@
       "image": "tt-model/superpoint-p150:0544890bca09",
       "repo": "changh95/superpoint-p150",
       "tt_model_version": "0.1.0",
-      "created_at": "2026-09-12T10:56:19+00:00",
+      "created_at": "2026-09-12T13:22:54+00:00",
       "tt_metal": {
         "sha": "8b98410e730bb504fea43a88609756e34821d91d",
         "describe": "v0.78.0-dev20260820-25-g8b98410e73",