Classroom Monitoring System

1. Detection — InsightFace (SCRFD, antelopev2 detector) finds every face in the photo at 1280×1280, returning a bounding box and 5-point landmarks per face.
2. Alignment — each face is aligned to a canonical 112×112 crop using InsightFace's norm_crop, based on the 5-point landmarks.
3. Recognition — the aligned crop is embedded with AdaFace IR-101 (WebFace12M), producing a 512-dimensional, L2-normalised face embedding.
4. Matching — the embedding is compared via cosine similarity against every enrolled student's stored prototype and individual embeddings. The best match is bucketed into one of three tiers: below 0.28 similarity gets no name candidate at all ("Unknown"); 0.28–0.30 is a real candidate but not confident enough to auto-confirm ("Suspicious" — for a teacher to eyeball); at or above 0.30 is confidently "Present". Anyone enrolled in the classroom who wasn't matched at either tier is listed as "Absent".
5. Enrollment — enrollment videos are sampled at up to 30 frames spread evenly across the clip, with an anchor-consistency check to keep only frames of the same person, plus degraded (shrunk-and-upscaled) copies of each frame so the gallery also matches small, distant classroom faces.
6. Unknown faces — anyone below 0.28 is boxed in red as "Unknown." The "Show unknown faces" button on the Mark attendance card zooms in on each one so a teacher can review and enroll or manually mark them.

AdaFace replaced the previous glintr100/antelopev2 recognition backbone after an offline evaluation showed cleaner separation between genuine and impostor matches on real classroom photos. Detection still uses InsightFace — only the recognition/embedding step changed.

1. Burst sampling — every 30 seconds of video, the pipeline grabs a 24-frame burst rather than analysing every frame, keeping a long lecture affordable to process.
2. Detection & tracking — YOLOv8-pose finds each student's body and keypoints, InsightFace (SCRFD) finds faces and 106-point landmarks, and boxes are linked across the burst by IoU overlap into one track per student.
3. Re-identification — each track is embedded with AdaFace IR-101 (the same backbone as Attendance) and resolved against students seen in earlier windows as a one-to-one assignment per burst (≥ 0.35 similarity) — so two different people visible in the same burst can never both be assigned to the same identity, which a simpler one-at-a-time match used to allow.
4. Roster recognition — the same embedding is also checked, read-only, against the enrolled Attendance roster (≥ 0.35 similarity, with a margin over the runner-up candidate). A confident match shows the real name instead of an anonymous "student_00N" label, and two anonymous IDs that both match the same enrolled student get merged into one timeline.
5. Signal extraction — per window: 106-point facial landmarks (InsightFace) drive mouth open/closed tracking and head yaw/pitch; YOLO for phone detection; optical flow for motion; DeepFace for dominant emotion.
6. Action classification — a priority ruleset turns those signals into one label per window: On Phone → Sleeping → Writing → Talking → Attentive → otherwise Distracted. "Attentive" is driven by mouth-closed percentage (≥ 80%) — a calibration pass against a labeled reference dataset found eye-openness (EAR) had no measurable correlation with attentiveness, while mouth state separated attentive from non-attentive clips much more cleanly.
7. Engagement rollup — the fraction of attentive windows becomes an overall attention level per student: High (≥ 70%), Medium (≥ 40%), or Low, alongside a short saved clip per window.

Known limitations: phone detection currently finds close to none of the real phones in testing — it's a generic, un-fine-tuned COCO model and needs replacing rather than re-tuning. Attentive/not- attentive classification runs at roughly 67–70% accuracy against a 156-clip labeled reference set — a real improvement over the previous EAR-based approach (which had no discriminative power at all), but not a solved problem.