Sync person-reidentification from metro-analytics-catalog

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+expected_output_dlstreamer.gif filter=lfs diff=lfs merge=lfs -text
+expected_output_openvino.jpg filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) Intel Corporation.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE

README.md

@@ -0,0 +1,414 @@
+---
+license: mit
+license_link: LICENSE
+library_name: openvino
+pipeline_tag: image-classification
+tags:
+  - openvino
+  - intel
+  - person-detection
+  - person-reidentification
+  - re-identification
+  - edge-ai
+  - metro
+  - dlstreamer
+language:
+  - en
+---
+
+# Person Re-Identification
+
+| Property | Value |
+|---|---|
+| **Category** | Person Detection + Cross-Camera Re-Identification |
+| **Base Model** | [person-detection-retail-0013](https://docs.openvino.ai/2024/omz_models_model_person_detection_retail_0013.html) + [person-reidentification-retail-0287](https://docs.openvino.ai/2024/omz_models_model_person_reidentification_retail_0287.html) (Open Model Zoo) |
+| **Source Framework** | Caffe / PyTorch (Open Model Zoo) |
+| **Supported Precisions** | FP32, FP16 |
+| **Inference Engine** | OpenVINO |
+| **Hardware** | CPU, GPU, NPU |
+| **Detected Class(es)** | Persons (detection) + 256-d appearance embeddings (re-identification) |
+
+---
+
+## Overview
+
+Person Re-Identification is a Metro Analytics use case that tracks the same
+individual across multiple camera views.
+Given a reference person seen on one camera, it locates that same person on
+another camera even though the pose, scale, and viewing angle differ.
+Each detected person is compared to the reference by cosine similarity of its
+appearance embedding vector.
+
+It uses a two-stage pipeline:
+
+- **person-detection-retail-0013** -- detects every person in the scene.
+- **person-reidentification-retail-0287** -- computes a 256-d appearance
+  embedding per person that is robust to viewpoint and lighting changes.
+
+Unlike face-based matching, re-identification relies on whole-body appearance
+(clothing, build, gait cues), so it works at surveillance distances where faces
+are not clearly visible.
+
+To demonstrate cross-camera behaviour from a single downloadable clip, the wide
+surveillance video is treated as two virtual cameras by time window: an earlier
+enrollment window is **Camera A** (where the reference identity is first seen)
+and a later query window is **Camera B** (where the person is re-identified as
+they continue to move through the scene). This emulates a person first seen on
+one camera and later re-identified on another using the same embedding-matching
+logic that links identities across a real multi-camera network.
+
+Typical Metro deployments include:
+
+- **Multi-Camera Tracking** -- follow a person across cameras in campuses,
+  airports, and transit hubs.
+- **Lost-and-Found / Person of Interest** -- locate where a flagged individual
+  appears across a camera network.
+- **Journey Analytics** -- reconstruct a person's path through a facility.
+- **Tailgating and Zone Analytics** -- confirm the same person across entry and
+  interior cameras.
+
+> **Privacy Note:** Person re-identification processes biometric-adjacent
+> appearance data.
+> Ensure your deployment complies with applicable privacy regulations
+> (GDPR, BIPA, etc.) and has proper consent and retention policies in place.
+
+---
+
+## Prerequisites
+
+- Python 3.11+
+- [Install OpenVINO](https://docs.openvino.ai/2026/get-started/install-openvino.html) (latest version)
+- [Install Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/get_started/install/install_guide_ubuntu.html) (latest version)
+
+Create and activate a Python virtual environment before running the scripts:
+
+```bash
+python3 -m venv .venv --system-site-packages
+source .venv/bin/activate
+```
+
+> **Note:** The `--system-site-packages` flag is required so the virtual
+> environment can access the system-installed OpenVINO and DLStreamer Python
+> packages.
+
+---
+
+## Getting Started
+
+### Download Models
+
+Run the provided script to download the person detection and re-identification
+models from the Open Model Zoo:
+
+```bash
+chmod +x export_and_quantize.sh
+./export_and_quantize.sh
+```
+
+The script downloads `person-detection-retail-0013` and
+`person-reidentification-retail-0287` in FP16, downloads the sample surveillance
+video (`test_video.mp4`), and captures a reference person crop (`person_a.jpg`)
+of the most prominent person seen in the Camera A enrollment window.
+
+### OpenVINO Sample
+
+The sample below re-identifies the reference person across camera views.
+It loads the captured reference image (`person_a.jpg`, enrolled from Camera A),
+computes its embedding, then scans frames of the Camera B query window.
+In each Camera B frame it detects every person, embeds each one, and keeps the
+person whose similarity to the reference is highest.
+It writes the Camera B frame with the strongest match, drawing a green box only
+on the re-identified person.
+Change the `device` string to run on CPU, GPU, or NPU.
+
+```python
+import cv2
+import numpy as np
+import openvino as ov
+
+DETECTION_MODEL = "intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml"
+REID_MODEL = "intel/person-reidentification-retail-0287/FP16/person-reidentification-retail-0287.xml"
+REFERENCE_IMAGE = "person_a.jpg"     # reference identity enrolled from Camera A
+SCENE_VIDEO = "test_video.mp4"       # wide feed; a later window acts as Camera B
+CAMERA_B_START_FRAME = 450           # query window begins ~15s into the clip
+CONF_THRESHOLD = 0.6
+MATCH_THRESHOLD = 0.6
+
+core = ov.Core()
+
+# Change device to "GPU" or "NPU" to run on integrated GPU or NPU.
+det_compiled = core.compile_model(core.read_model(DETECTION_MODEL), "CPU")
+reid_compiled = core.compile_model(core.read_model(REID_MODEL), "CPU")
+
+det_input = det_compiled.input(0)
+det_h, det_w = det_input.shape[2], det_input.shape[3]
+reid_input = reid_compiled.input(0)
+reid_h, reid_w = reid_input.shape[2], reid_input.shape[3]
+
+
+def detect_persons(img):
+    h0, w0 = img.shape[:2]
+    blob = cv2.resize(img, (det_w, det_h))
+    blob = blob.transpose(2, 0, 1)[np.newaxis, ...].astype(np.float32)
+    dets = det_compiled([blob])[det_compiled.output(0)][0][0]
+    persons = []
+    for d in dets:
+        if float(d[2]) < CONF_THRESHOLD:
+            continue
+        x1 = max(0, int(d[3] * w0))
+        y1 = max(0, int(d[4] * h0))
+        x2 = min(w0, int(d[5] * w0))
+        y2 = min(h0, int(d[6] * h0))
+        if x2 > x1 and y2 > y1:
+            persons.append((x1, y1, x2, y2))
+    return persons
+
+
+def get_embedding(img, bbox):
+    x1, y1, x2, y2 = bbox
+    crop = img[y1:y2, x1:x2]
+    blob = cv2.resize(crop, (reid_w, reid_h))
+    blob = blob.transpose(2, 0, 1)[np.newaxis, ...].astype(np.float32)
+    emb = reid_compiled([blob])[reid_compiled.output(0)].flatten()
+    return emb / (np.linalg.norm(emb) + 1e-9)
+
+
+# 1. Embed the reference person enrolled from Camera A.
+# person_a.jpg is already a cropped person, so embed the whole image directly
+# (the re-identification model expects a person crop as its input).
+reference = cv2.imread(REFERENCE_IMAGE)
+if reference is None:
+    raise SystemExit("Could not read the reference image")
+ref_emb = get_embedding(reference, (0, 0, reference.shape[1], reference.shape[0]))
+
+# 2. Scan the Camera B window and keep the frame with the strongest re-id match.
+cap = cv2.VideoCapture(SCENE_VIDEO)
+total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) or 900
+best = {"sim": 0.0, "frame": None, "bbox": None}
+for frame_idx in range(CAMERA_B_START_FRAME, total, 15):
+    cap.set(cv2.CAP_PROP_POS_FRAMES, frame_idx)
+    ok, frame = cap.read()
+    if not ok:
+        break
+    for bbox in detect_persons(frame):
+        sim = float(np.dot(get_embedding(frame, bbox), ref_emb))
+        if sim > best["sim"]:
+            best = {"sim": sim, "frame": frame.copy(), "bbox": bbox}
+cap.release()
+
+# 3. Annotate and save the best Camera B match.
+if best["frame"] is None:
+    raise SystemExit("No person detected in the Camera B window")
+frame = best["frame"]
+if best["sim"] >= MATCH_THRESHOLD:
+    x1, y1, x2, y2 = best["bbox"]
+    cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2)
+    cv2.putText(frame, f"RE-ID {best['sim']:.2f}", (x1, max(15, y1 - 8)),
+                cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2)
+    print(f"Re-identified reference person in Camera B, similarity {best['sim']:.4f}")
+else:
+    print(f"No matching person found (best similarity {best['sim']:.4f})")
+
+cv2.imwrite("output_openvino.jpg", frame)
+print("Saved: output_openvino.jpg")
+```
+
+**Device targets:**
+
+- `"CPU"` -- default, works on all Intel platforms.
+- `"GPU"` -- Intel integrated or discrete GPU.
+- `"NPU"` -- Intel NPU; both FP16 models are NPU-compatible.
+
+#### Expected Output
+
+![OpenVINO expected output](expected_output_openvino.jpg)
+
+### DLStreamer Sample
+
+The pipeline below runs the person detector via `gvadetect` and the
+re-identification model via `gvaclassify` on the video.
+Frames are pulled through an `appsink`, where each detected person's embedding
+is compared to the reference embedding computed from `person_a.jpg`.
+Only persons that match the reference identity are boxed, so the annotated
+`output_dlstreamer.mp4` highlights the same person as they move across the scene
+even when other people are present.
+
+> **Notes on running this sample:**
+>
+> - Export `PYTHONPATH` so the DLStreamer Python modules (`gi`, `gstgva`) are
+>   importable:
+>
+>   ```bash
+>   source /opt/intel/openvino_2026/setupvars.sh
+>   source /opt/intel/dlstreamer/scripts/setup_dls_env.sh
+>   export PYTHONPATH=/opt/intel/dlstreamer/python:\
+>   /opt/intel/dlstreamer/gstreamer/lib/python3/dist-packages:${PYTHONPATH:-}
+>   ```
+>
+> - The re-identification embedding is attached as a tensor on each person's
+>   region-of-interest metadata. Convert the stream to `BGR` **before**
+>   `gvadetect`/`gvaclassify` so a downstream format conversion does not strip
+>   those tensors before the `appsink` reads them.
+
+```python
+import gi
+
+gi.require_version("Gst", "1.0")
+from gi.repository import Gst
+
+Gst.init([])
+
+import numpy as np
+import cv2
+from gstgva import VideoFrame
+
+INPUT_VIDEO = "test_video.mp4"
+REFERENCE_IMAGE = "person_a.jpg"
+OUTPUT_VIDEO = "output_dlstreamer.mp4"
+DETECTION_MODEL = "intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml"
+REID_MODEL = "intel/person-reidentification-retail-0287/FP16/person-reidentification-retail-0287.xml"
+# For CPU: change "GPU" to "CPU". For NPU: change "GPU" to "NPU".
+DEVICE = "GPU"
+DET_THRESHOLD = 0.6
+MATCH_THRESHOLD = 0.6
+
+
+def person_embeddings(video_frame):
+    """Yield ((x, y, w, h), normalized_embedding) for each classified person."""
+    for region in video_frame.regions():
+        rect = region.rect()
+        emb = None
+        for tensor in region.tensors():
+            if tensor.is_detection():
+                continue
+            data = np.array(tensor.data(), dtype=np.float32)
+            if data.size >= 256:
+                emb = data[:256]
+        if emb is None:
+            continue
+        emb = emb / (np.linalg.norm(emb) + 1e-9)
+        yield (int(rect.x), int(rect.y), int(rect.w), int(rect.h)), emb
+
+
+def run_pipeline(source_desc, on_frame):
+    # Convert to BGR before inference so gvaclassify's embedding tensors survive
+    # to the appsink (a later format-changing videoconvert would strip them).
+    pipeline = Gst.parse_launch(
+        f"{source_desc} ! videoconvert ! video/x-raw,format=BGR ! "
+        f"gvadetect model={DETECTION_MODEL} device={DEVICE} "
+        f"threshold={DET_THRESHOLD} ! queue ! "
+        f"gvaclassify model={REID_MODEL} device={DEVICE} ! queue ! "
+        "appsink name=sink emit-signals=true sync=false max-buffers=4 drop=false"
+    )
+    sink = pipeline.get_by_name("sink")
+    sink.connect("new-sample", on_frame)
+    pipeline.set_state(Gst.State.PLAYING)
+    pipeline.get_bus().timed_pop_filtered(
+        Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS | Gst.MessageType.ERROR)
+    pipeline.set_state(Gst.State.NULL)
+
+
+# 1. Compute the reference embedding from the enrolled person image.
+# person_a.jpg is already a person crop, so run the re-identification model on
+# the whole frame (inference-region=full-frame) instead of detecting first.
+ref = {"emb": None}
+
+
+def on_reference(sink):
+    sample = sink.emit("pull-sample")
+    if sample is None:
+        return Gst.FlowReturn.OK
+    vf = VideoFrame(sample.get_buffer(), caps=sample.get_caps())
+    for tensor in vf.tensors():
+        data = np.array(tensor.data(), dtype=np.float32)
+        if data.size >= 256:
+            emb = data[:256]
+            ref["emb"] = emb / (np.linalg.norm(emb) + 1e-9)
+    return Gst.FlowReturn.OK
+
+
+reference_pipeline = Gst.parse_launch(
+    f"filesrc location={REFERENCE_IMAGE} ! jpegdec ! videoconvert ! "
+    f"video/x-raw,format=BGR ! "
+    f"gvainference model={REID_MODEL} device={DEVICE} "
+    f"inference-region=full-frame ! queue ! "
+    "appsink name=sink emit-signals=true sync=false max-buffers=4 drop=false"
+)
+ref_sink = reference_pipeline.get_by_name("sink")
+ref_sink.connect("new-sample", on_reference)
+reference_pipeline.set_state(Gst.State.PLAYING)
+reference_pipeline.get_bus().timed_pop_filtered(
+    Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS | Gst.MessageType.ERROR)
+reference_pipeline.set_state(Gst.State.NULL)
+if ref["emb"] is None:
+    raise SystemExit("Could not compute the reference embedding")
+ref_emb = ref["emb"]
+
+# 2. Process the video, boxing only persons that match the reference identity.
+writer = {"w": None}
+match_frames = 0
+
+
+def on_video(sink):
+    global match_frames
+    sample = sink.emit("pull-sample")
+    if sample is None:
+        return Gst.FlowReturn.OK
+    vf = VideoFrame(sample.get_buffer(), caps=sample.get_caps())
+    matches = []
+    for (x, y, w, h), emb in person_embeddings(vf):
+        similarity = float(np.dot(emb, ref_emb))
+        if similarity >= MATCH_THRESHOLD:
+            matches.append((x, y, w, h, similarity))
+
+    with vf.data() as mat:
+        frame = mat.copy()
+
+    if writer["w"] is None:
+        frame_h, frame_w = frame.shape[:2]
+        structure = sample.get_caps().get_structure(0)
+        ok_fr, fps_n, fps_d = structure.get_fraction("framerate")
+        fps = fps_n / fps_d if ok_fr and fps_d else 12
+        writer["w"] = cv2.VideoWriter(
+            OUTPUT_VIDEO, cv2.VideoWriter_fourcc(*"mp4v"), fps, (frame_w, frame_h))
+
+    for x, y, w, h, similarity in matches:
+        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)
+        cv2.putText(frame, f"RE-ID {similarity:.2f}", (x, max(15, y - 8)),
+                    cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2)
+    if matches:
+        match_frames += 1
+    writer["w"].write(frame)
+    return Gst.FlowReturn.OK
+
+
+run_pipeline(f"filesrc location={INPUT_VIDEO} ! decodebin3", on_video)
+if writer["w"] is not None:
+    writer["w"].release()
+print(f"Frames with a re-identified person: {match_frames}", flush=True)
+print(f"Saved: {OUTPUT_VIDEO}", flush=True)
+```
+
+**Device targets:**
+
+- `DEVICE = "GPU"` -- default in the sample code.
+- `DEVICE = "CPU"` -- change `"GPU"` to `"CPU"`.
+- `DEVICE = "NPU"` -- change `"GPU"` to `"NPU"`; use `batch-size=1` and `nireq=4` for best NPU utilization.
+
+#### Expected Output
+
+![DLStreamer expected output](expected_output_dlstreamer.gif)
+
+---
+
+## License
+
+Licensed under the MIT License. See [LICENSE](LICENSE) for details.
+
+## References
+
+- [person-detection-retail-0013](https://docs.openvino.ai/2024/omz_models_model_person_detection_retail_0013.html)
+- [person-reidentification-retail-0287](https://docs.openvino.ai/2024/omz_models_model_person_reidentification_retail_0287.html)
+- [Open Model Zoo](https://github.com/openvinotoolkit/open_model_zoo)
+- [OpenVINO Documentation](https://docs.openvino.ai/)
+- [Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/index.html)

export_and_quantize.sh

@@ -0,0 +1,127 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: MIT
+# Copyright (C) Intel Corporation
+#
+# Download the person detection and person re-identification models from the
+# Open Model Zoo for the person-reidentification use case, download the sample
+# surveillance video, and capture a reference person crop (person_a.jpg) that
+# represents the identity to re-identify across camera views.
+# Usage: ./export_and_quantize.sh
+
+set -euo pipefail
+
+# Official Open Model Zoo public model storage (versioned, immutable).
+OMZ_BASE="https://storage.openvinotoolkit.org/repositories/open_model_zoo/2023.0/models_bin/1"
+
+echo "--- Installing dependencies ---"
+pip install -qU openvino opencv-python numpy
+
+# Download both the IR topology (.xml) and weights (.bin) for an OMZ model
+# from the official storage into intel/<model>/<precision>/.
+download_omz_model() {
+    local model="$1"
+    local precision="$2"
+    local dest="intel/${model}/${precision}"
+    mkdir -p "${dest}"
+    local ext
+    for ext in xml bin; do
+        if [[ ! -f "${dest}/${model}.${ext}" ]]; then
+            wget -q -O "${dest}/${model}.${ext}" \
+                "${OMZ_BASE}/${model}/${precision}/${model}.${ext}"
+        fi
+    done
+}
+
+# Ask for approval before downloading models and sample files
+echo ""
+echo "This script will download:"
+echo "  - Model weights and/or sample files"
+echo ""
+read -p "Continue with downloads? (yes/no): " APPROVAL
+if [[ "${APPROVAL}" != "yes" ]]; then
+    echo "Download cancelled by user."
+    exit 0
+fi
+echo ""
+
+echo "--- Downloading person-detection-retail-0013 (FP16) ---"
+download_omz_model person-detection-retail-0013 FP16
+echo "Ready: person-detection-retail-0013"
+
+echo "--- Downloading person-reidentification-retail-0287 (FP16) ---"
+download_omz_model person-reidentification-retail-0287 FP16
+echo "Ready: person-reidentification-retail-0287"
+
+echo "--- Downloading sample surveillance video ---"
+if [[ ! -f test_video.mp4 ]]; then
+    wget -q -O test_video.mp4 \
+        "https://github.com/open-edge-platform/edge-ai-resources/raw/main/videos/VIRAT_S_000101.mp4"
+    echo "Downloaded: test_video.mp4"
+else
+    echo "Already present: test_video.mp4"
+fi
+
+echo "--- Capturing the reference person from the Camera A enrollment window ---"
+if [[ ! -f person_a.jpg ]]; then
+    python3 - <<'PY'
+import cv2
+import numpy as np
+import openvino as ov
+
+DET = "intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml"
+core = ov.Core()
+det = core.compile_model(core.read_model(DET), "CPU")
+inp = det.input(0)
+det_h, det_w = inp.shape[2], inp.shape[3]
+
+
+def detect_persons(frame, thr=0.6):
+    """Return [(x1, y1, x2, y2), ...] for every person detected in the frame."""
+    h, w = frame.shape[:2]
+    blob = cv2.resize(frame, (det_w, det_h))
+    blob = blob.transpose(2, 0, 1)[np.newaxis, ...].astype(np.float32)
+    out = det([blob])[det.output(0)][0][0]
+    boxes = []
+    for d in out:
+        if float(d[2]) < thr:
+            continue
+        x1 = max(0, int(d[3] * w))
+        y1 = max(0, int(d[4] * h))
+        x2 = min(w, int(d[5] * w))
+        y2 = min(h, int(d[6] * h))
+        if x2 > x1 and y2 > y1:
+            boxes.append((x1, y1, x2, y2))
+    return boxes
+
+
+# Scan the enrollment window (Camera A, the opening seconds) and capture the
+# most prominent person. This person becomes the reference identity that the
+# samples re-identify in the later Camera B query window.
+cap = cv2.VideoCapture("test_video.mp4")
+best = {"crop": None, "area": 0}
+for frame_idx in range(0, 300, 15):
+    cap.set(cv2.CAP_PROP_POS_FRAMES, frame_idx)
+    ok, frame = cap.read()
+    if not ok:
+        break
+    for x1, y1, x2, y2 in detect_persons(frame):
+        area = (x2 - x1) * (y2 - y1)
+        if area > best["area"]:
+            best["area"] = area
+            best["crop"] = frame[y1:y2, x1:x2].copy()
+cap.release()
+
+if best["crop"] is None or best["crop"].size == 0:
+    raise SystemExit("Could not capture a reference person from the enrollment window")
+cv2.imwrite("person_a.jpg", best["crop"])
+print("Captured person_a.jpg (reference identity from the Camera A enrollment window)")
+PY
+else
+    echo "Already present: person_a.jpg"
+fi
+
+echo "--- Done ---"
+echo "Detector  : intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml"
+echo "ReID      : intel/person-reidentification-retail-0287/FP16/person-reidentification-retail-0287.xml"
+echo "Reference : person_a.jpg (identity captured from the Camera A enrollment window)"
+echo "Scene     : test_video.mp4 (wide-area surveillance clip split into two virtual cameras)"