Umut Kocasari Claude Opus 4.8 commited on
Commit
94c4dfa
·
1 Parent(s): dae4cfd

Default to one-by-one, robust video decode, add 40-img/1-img examples

Browse files

- Default inference mode is now One-by-one (was Joint).
- Decode videos with cv2.VideoCapture instead of imageio's extension-based plugin
pick, which only yielded the first frame of webcam recordings; now the first
MAX_FRAMES frames are used (imageio fallback kept).
- Add two click-to-run gr.Examples (cache_examples=False so the build never runs
on the GPU-less builder): a 40-image sequence and a single image, vendored under
examples/seq40/ (first 40 frames of nersemble 109 EMO-1-shout+laugh), LFS-tracked.
Verified: examples build on gradio 6.19; cv2 extracts 40 frames from a 50-frame clip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Files changed (42) hide show
  1. .gitattributes +3 -0
  2. app.py +57 -8
  3. examples/seq40/frame_000001.png +3 -0
  4. examples/seq40/frame_000002.png +3 -0
  5. examples/seq40/frame_000003.png +3 -0
  6. examples/seq40/frame_000004.png +3 -0
  7. examples/seq40/frame_000005.png +3 -0
  8. examples/seq40/frame_000006.png +3 -0
  9. examples/seq40/frame_000007.png +3 -0
  10. examples/seq40/frame_000008.png +3 -0
  11. examples/seq40/frame_000009.png +3 -0
  12. examples/seq40/frame_000010.png +3 -0
  13. examples/seq40/frame_000011.png +3 -0
  14. examples/seq40/frame_000012.png +3 -0
  15. examples/seq40/frame_000013.png +3 -0
  16. examples/seq40/frame_000014.png +3 -0
  17. examples/seq40/frame_000015.png +3 -0
  18. examples/seq40/frame_000016.png +3 -0
  19. examples/seq40/frame_000017.png +3 -0
  20. examples/seq40/frame_000018.png +3 -0
  21. examples/seq40/frame_000019.png +3 -0
  22. examples/seq40/frame_000020.png +3 -0
  23. examples/seq40/frame_000021.png +3 -0
  24. examples/seq40/frame_000022.png +3 -0
  25. examples/seq40/frame_000023.png +3 -0
  26. examples/seq40/frame_000024.png +3 -0
  27. examples/seq40/frame_000025.png +3 -0
  28. examples/seq40/frame_000026.png +3 -0
  29. examples/seq40/frame_000027.png +3 -0
  30. examples/seq40/frame_000028.png +3 -0
  31. examples/seq40/frame_000029.png +3 -0
  32. examples/seq40/frame_000030.png +3 -0
  33. examples/seq40/frame_000031.png +3 -0
  34. examples/seq40/frame_000032.png +3 -0
  35. examples/seq40/frame_000033.png +3 -0
  36. examples/seq40/frame_000034.png +3 -0
  37. examples/seq40/frame_000035.png +3 -0
  38. examples/seq40/frame_000036.png +3 -0
  39. examples/seq40/frame_000037.png +3 -0
  40. examples/seq40/frame_000038.png +3 -0
  41. examples/seq40/frame_000039.png +3 -0
  42. examples/seq40/frame_000040.png +3 -0
.gitattributes CHANGED
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ *.png filter=lfs diff=lfs merge=lfs -text
37
+ *.jpg filter=lfs diff=lfs merge=lfs -text
38
+ *.jpeg filter=lfs diff=lfs merge=lfs -text
app.py CHANGED
@@ -197,6 +197,28 @@ def _sniff_ext(path):
197
  return ".png"
198
 
199
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
200
  def _prepare_inputs(files, video, max_frames, workdir):
201
  """Normalize the upload (an image set OR a video) into an ordered list of
202
  frame paths.
@@ -206,14 +228,21 @@ def _prepare_inputs(files, video, max_frames, workdir):
206
  one): files are natural-sorted by their original name (temporal order),
207
  capped, then copied as ``frame_XXXX.<ext>`` with a content-sniffed extension.
208
  """
209
- from faceanything.io_utils import load_frame_paths
210
-
211
  if video:
212
  vpath = video if isinstance(video, str) else _to_entry(video)[0]
213
  if vpath and os.path.exists(vpath):
214
- paths, _ = load_frame_paths(
215
- vpath, max_frames=int(max_frames), stride=1,
216
- work_dir=os.path.join(workdir, "video_frames"))
 
 
 
 
 
 
 
 
 
217
  if paths:
218
  return paths
219
 
@@ -576,10 +605,10 @@ def build_demo():
576
  )
577
  mode = gr.Radio(
578
  choices=["Joint", "One-by-one"],
579
- value="Joint",
580
  label="Inference mode",
581
- info="Joint (all-at-once): more 3D-consistent across frames. "
582
- "One-by-one: more surface detail, lower memory.",
583
  )
584
  remove_bg = gr.Checkbox(
585
  value=True, label="Remove background",
@@ -650,6 +679,26 @@ def build_demo():
650
  # slider — smoother than reloading a glb on every drag tick.
651
  frame_slider.release(pick_frame, inputs=[frame_slider, view_state],
652
  outputs=model3d)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
653
  return demo
654
 
655
 
 
197
  return ".png"
198
 
199
 
200
+ def _extract_video(video_path, max_frames, out_dir):
201
+ """Decode the first ``max_frames`` frames of a video. Uses cv2.VideoCapture,
202
+ which (unlike imageio's extension-based plugin pick) robustly decodes webcam
203
+ recordings — those were yielding only a single frame otherwise."""
204
+ import cv2
205
+
206
+ os.makedirs(out_dir, exist_ok=True)
207
+ paths = []
208
+ cap = cv2.VideoCapture(video_path)
209
+ try:
210
+ while len(paths) < int(max_frames):
211
+ ok, frame = cap.read()
212
+ if not ok:
213
+ break
214
+ p = os.path.join(out_dir, f"frame_{len(paths):04d}.png")
215
+ cv2.imwrite(p, frame) # BGR ndarray -> correct-RGB PNG on disk
216
+ paths.append(p)
217
+ finally:
218
+ cap.release()
219
+ return paths
220
+
221
+
222
  def _prepare_inputs(files, video, max_frames, workdir):
223
  """Normalize the upload (an image set OR a video) into an ordered list of
224
  frame paths.
 
228
  one): files are natural-sorted by their original name (temporal order),
229
  capped, then copied as ``frame_XXXX.<ext>`` with a content-sniffed extension.
230
  """
 
 
231
  if video:
232
  vpath = video if isinstance(video, str) else _to_entry(video)[0]
233
  if vpath and os.path.exists(vpath):
234
+ paths = _extract_video(vpath, max_frames,
235
+ os.path.join(workdir, "video_frames"))
236
+ if len(paths) <= 1: # fall back to imageio if cv2 read too few frames
237
+ from faceanything.io_utils import load_frame_paths
238
+ try:
239
+ alt, _ = load_frame_paths(
240
+ vpath, max_frames=int(max_frames), stride=1,
241
+ work_dir=os.path.join(workdir, "video_frames_io"))
242
+ if len(alt) > len(paths):
243
+ paths = alt
244
+ except Exception:
245
+ pass
246
  if paths:
247
  return paths
248
 
 
605
  )
606
  mode = gr.Radio(
607
  choices=["Joint", "One-by-one"],
608
+ value="One-by-one",
609
  label="Inference mode",
610
+ info="One-by-one: more surface detail, lower memory. "
611
+ "Joint (all-at-once): more 3D-consistent across frames.",
612
  )
613
  remove_bg = gr.Checkbox(
614
  value=True, label="Remove background",
 
679
  # slider — smoother than reloading a glb on every drag tick.
680
  frame_slider.release(pick_frame, inputs=[frame_slider, view_state],
681
  outputs=model3d)
682
+
683
+ # ---------------- examples (click to load + run) ----------------
684
+ ex40 = sorted(glob.glob(os.path.join(APP_DIR, "examples", "seq40", "*.png")))[:MAX_IMAGES]
685
+ if ex40:
686
+ gr.Examples(
687
+ examples=[
688
+ [ex40, None, "One-by-one", 504, True, 0, 100, 25, 0.01, 10, MAX_IMAGES],
689
+ [ex40[:1], None, "One-by-one", 504, True, 0, 100, 25, 0.01, 10, 1],
690
+ ],
691
+ example_labels=[f"{len(ex40)}-image sequence", "Single image"],
692
+ inputs=[files, video, mode, process_res, remove_bg,
693
+ conf_percentile, n_tracks, track_k, track_threshold,
694
+ fps, max_frames],
695
+ outputs=[model3d, canonical_vid, depth_vid, normals_vid,
696
+ tracks2d_vid, tracks_zip, view_state, frame_slider, status],
697
+ fn=run,
698
+ run_on_click=True, # click runs the pipeline
699
+ cache_examples=False, # never pre-run at build (no GPU at build time)
700
+ label="Examples — click to load and run",
701
+ )
702
  return demo
703
 
704
 
examples/seq40/frame_000001.png ADDED

Git LFS Details

  • SHA256: fe1186051de92e90f6ce5f62410974af158e61126b7f51a55d76c06315df14a9
  • Pointer size: 131 Bytes
  • Size of remote file: 439 kB
examples/seq40/frame_000002.png ADDED

Git LFS Details

  • SHA256: c15cfebf6c588d1096327b57dda2b2be952791aa62d85bc503e719679cadeca5
  • Pointer size: 131 Bytes
  • Size of remote file: 437 kB
examples/seq40/frame_000003.png ADDED

Git LFS Details

  • SHA256: 4b8604ac200a9da5308247ae6d1bc9422c0fe1fc05022d0c5b70bc6342a0eb96
  • Pointer size: 131 Bytes
  • Size of remote file: 440 kB
examples/seq40/frame_000004.png ADDED

Git LFS Details

  • SHA256: a526769db8fe4720e48defbb68946073f7f22bd0d2327dff13bd840950ad34bf
  • Pointer size: 131 Bytes
  • Size of remote file: 449 kB
examples/seq40/frame_000005.png ADDED

Git LFS Details

  • SHA256: d2922eb5a6e8a820d37e432a7bbb906ddc286b528588e489ae5e6ac195953a4b
  • Pointer size: 131 Bytes
  • Size of remote file: 448 kB
examples/seq40/frame_000006.png ADDED

Git LFS Details

  • SHA256: f3b69b42bb682eeffec8071e96acf4ee15b6a30b1e814786e4964b171578a895
  • Pointer size: 131 Bytes
  • Size of remote file: 448 kB
examples/seq40/frame_000007.png ADDED

Git LFS Details

  • SHA256: 229e3139a3e8fad60aff1cf09c94fe48e9101a8784a2bb95312ce98509450cb4
  • Pointer size: 131 Bytes
  • Size of remote file: 448 kB
examples/seq40/frame_000008.png ADDED

Git LFS Details

  • SHA256: 1ca872fd26fef4d35f904758f7385621debfe35ded3e63d9f6748737ac9eb0f5
  • Pointer size: 131 Bytes
  • Size of remote file: 448 kB
examples/seq40/frame_000009.png ADDED

Git LFS Details

  • SHA256: beadc955926cc1c1b2db4612df52168a4665d40ef6c4eb90492ff06c829264fa
  • Pointer size: 131 Bytes
  • Size of remote file: 450 kB
examples/seq40/frame_000010.png ADDED

Git LFS Details

  • SHA256: bda1755aa3d46c2c7df67211cf6dfdb3f96efeb14ba463c9292a6ab1fdf3b0c9
  • Pointer size: 131 Bytes
  • Size of remote file: 453 kB
examples/seq40/frame_000011.png ADDED

Git LFS Details

  • SHA256: 65fedc0d21b9ab129fc90f4c57f911690ca11e158915234b79c1413dd2203471
  • Pointer size: 131 Bytes
  • Size of remote file: 457 kB
examples/seq40/frame_000012.png ADDED

Git LFS Details

  • SHA256: 10258877b142251ce1ca57ed61300e9d1e64e0857c40b57d412d42f9de680181
  • Pointer size: 131 Bytes
  • Size of remote file: 459 kB
examples/seq40/frame_000013.png ADDED

Git LFS Details

  • SHA256: cfee8d9f5c46a9789c0ac4f953bec2681cc9405236b78bd49dfe8272b6b29acf
  • Pointer size: 131 Bytes
  • Size of remote file: 459 kB
examples/seq40/frame_000014.png ADDED

Git LFS Details

  • SHA256: f5909b2cda5909a45ddb92097c642932b613db096a789747d42dd2e2b7fbf3b3
  • Pointer size: 131 Bytes
  • Size of remote file: 462 kB
examples/seq40/frame_000015.png ADDED

Git LFS Details

  • SHA256: 9d837b3f601ea3f7215d3f880e9186dbd5a23b931f294e3f9a9f2bc7bbff45a5
  • Pointer size: 131 Bytes
  • Size of remote file: 462 kB
examples/seq40/frame_000016.png ADDED

Git LFS Details

  • SHA256: c5d2d575bfce23eb7aa04857048a3e2f47fd124d7d7e99f4f5e6f769ef259162
  • Pointer size: 131 Bytes
  • Size of remote file: 464 kB
examples/seq40/frame_000017.png ADDED

Git LFS Details

  • SHA256: 18c754f271b4c46b693a39bda25df1da6d15b845bebed196d623f17bfc607f5a
  • Pointer size: 131 Bytes
  • Size of remote file: 467 kB
examples/seq40/frame_000018.png ADDED

Git LFS Details

  • SHA256: e62393ac7240ccc7d78b794ed98f97f9d9b275128106614884868398fa2a8773
  • Pointer size: 131 Bytes
  • Size of remote file: 468 kB
examples/seq40/frame_000019.png ADDED

Git LFS Details

  • SHA256: 2fb2c567d6016e0a3d009b5bb9150736120966ef9370e62f1b3fe081f047449f
  • Pointer size: 131 Bytes
  • Size of remote file: 467 kB
examples/seq40/frame_000020.png ADDED

Git LFS Details

  • SHA256: 4c7377aca128ad6e4790f992ec9ef7a5b2b7ec7d9c3d6594fbc695406b2ba276
  • Pointer size: 131 Bytes
  • Size of remote file: 467 kB
examples/seq40/frame_000021.png ADDED

Git LFS Details

  • SHA256: e04b0bc1c3e3f52d6c6e9f46b3459b63a68366032f20ce272279a02a628d95ae
  • Pointer size: 131 Bytes
  • Size of remote file: 469 kB
examples/seq40/frame_000022.png ADDED

Git LFS Details

  • SHA256: 11af6b036b8cc4a977bd7d270957c2de25b2fc962b010eeeb2d03d41e104ac67
  • Pointer size: 131 Bytes
  • Size of remote file: 470 kB
examples/seq40/frame_000023.png ADDED

Git LFS Details

  • SHA256: 9e51ca81cf437dbcfd8ffc05b612784c2ef9a7e4a3f460773a0e3e7d5ae445d4
  • Pointer size: 131 Bytes
  • Size of remote file: 470 kB
examples/seq40/frame_000024.png ADDED

Git LFS Details

  • SHA256: 49980b7673196c8b94d802fb74049f9290fd04838cf2cdb6cbf4a271e422bcd8
  • Pointer size: 131 Bytes
  • Size of remote file: 469 kB
examples/seq40/frame_000025.png ADDED

Git LFS Details

  • SHA256: a73dbb48aca8d189a1353ce9af8e87b862e2b5c33f635c3eb2269b6f3b6befc2
  • Pointer size: 131 Bytes
  • Size of remote file: 473 kB
examples/seq40/frame_000026.png ADDED

Git LFS Details

  • SHA256: 29b340a4d9172c337ff01988ec22ac39ed09f6e94d5c97456c09f4bd1365a2f7
  • Pointer size: 131 Bytes
  • Size of remote file: 471 kB
examples/seq40/frame_000027.png ADDED

Git LFS Details

  • SHA256: 22961f2c497da701e59784422cdba3b4b428e6884d35f1cce8c6b9388106be26
  • Pointer size: 131 Bytes
  • Size of remote file: 472 kB
examples/seq40/frame_000028.png ADDED

Git LFS Details

  • SHA256: 30fb4af389f8e814fde46e5657e31e01558525b8dda1394ecbd9b11372235b01
  • Pointer size: 131 Bytes
  • Size of remote file: 474 kB
examples/seq40/frame_000029.png ADDED

Git LFS Details

  • SHA256: 38a815b2c424532c77f169d7a78e932cf5216c36c3b19d2093fd1ca5c91f4f6a
  • Pointer size: 131 Bytes
  • Size of remote file: 477 kB
examples/seq40/frame_000030.png ADDED

Git LFS Details

  • SHA256: 356d7d085ade28b6d3cfd6fa94d521941bb568809734c636acfc2b3296635d11
  • Pointer size: 131 Bytes
  • Size of remote file: 479 kB
examples/seq40/frame_000031.png ADDED

Git LFS Details

  • SHA256: 660c8425a4e6133922f752edf7d3f827ffb9813d1fbf4af44867bca0267e4786
  • Pointer size: 131 Bytes
  • Size of remote file: 479 kB
examples/seq40/frame_000032.png ADDED

Git LFS Details

  • SHA256: c056d09cd866fac4c51c301128e3fb91b647499ac354728a1db7c9e5ca021cd3
  • Pointer size: 131 Bytes
  • Size of remote file: 478 kB
examples/seq40/frame_000033.png ADDED

Git LFS Details

  • SHA256: 67b75d7e1fd4de9d31a4ac6c7977951e34b511caba0c06043a27082b842634a5
  • Pointer size: 131 Bytes
  • Size of remote file: 480 kB
examples/seq40/frame_000034.png ADDED

Git LFS Details

  • SHA256: 191e7ca4ee68b3c642a2c0da2cdedeeca37a459a71e0acf8d2678a5d0ce40ea9
  • Pointer size: 131 Bytes
  • Size of remote file: 480 kB
examples/seq40/frame_000035.png ADDED

Git LFS Details

  • SHA256: b0de53e8b3ea1eedc502c1a4d8bdf047bea5aec7011ffdb24f02ba92dbb08e4d
  • Pointer size: 131 Bytes
  • Size of remote file: 477 kB
examples/seq40/frame_000036.png ADDED

Git LFS Details

  • SHA256: f2bdc44bb827056a27c548860e21532e06f0e4037d839b8d330c60a10181ab41
  • Pointer size: 131 Bytes
  • Size of remote file: 477 kB
examples/seq40/frame_000037.png ADDED

Git LFS Details

  • SHA256: df79b4501bba44b9931379f658d0697ac03825fa7070088bfbb71053cdffae7f
  • Pointer size: 131 Bytes
  • Size of remote file: 488 kB
examples/seq40/frame_000038.png ADDED

Git LFS Details

  • SHA256: d324020f693bc375c8ea41bcedff61759a4c840eac94ed4e2042ab66f9f700c6
  • Pointer size: 131 Bytes
  • Size of remote file: 490 kB
examples/seq40/frame_000039.png ADDED

Git LFS Details

  • SHA256: 3e86f9a77e8ef2261cb3b4eb9a9f18325c75207134a360bfacf97c0c90c031af
  • Pointer size: 131 Bytes
  • Size of remote file: 487 kB
examples/seq40/frame_000040.png ADDED

Git LFS Details

  • SHA256: 9db551760299817dad70888d83c252b5ad6a1118011485523f494ebdac4da3cb
  • Pointer size: 131 Bytes
  • Size of remote file: 485 kB