Add Seedance 2.0 action video comparison
Browse files- .gitattributes +8 -0
- README.md +32 -0
- examples/seedance-action/seedance-2-action-bicubic-x2-720p.mp4 +3 -0
- examples/seedance-action/seedance-2-action-clark-x2-720p.mp4 +3 -0
- examples/seedance-action/seedance-2-action-detail-triptych.mp4 +3 -0
- examples/seedance-action/seedance-2-action-native-triptych.mp4 +3 -0
- examples/seedance-action/seedance-2-action-product-reference-720p.mp4 +3 -0
- examples/seedance-action/seedance-2-action-real-input-360p.mp4 +3 -0
- examples/seedance-action/seedance-2-action-still-comparison.png +3 -0
- examples/seedance-action/seedance-2-action-triptych-web.mp4 +3 -0
- technical/seedance-action-video-evaluation.json +70 -0
.gitattributes
CHANGED
|
@@ -37,3 +37,11 @@ examples/product-demo-1-before-after.png filter=lfs diff=lfs merge=lfs -text
|
|
| 37 |
examples/product-demo-2-before-after.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
examples/product-demo-3-before-after.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
examples/product-demo-before-after.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
examples/product-demo-2-before-after.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
examples/product-demo-3-before-after.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
examples/product-demo-before-after.png filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
examples/seedance-action/seedance-2-action-bicubic-x2-720p.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
examples/seedance-action/seedance-2-action-clark-x2-720p.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
examples/seedance-action/seedance-2-action-detail-triptych.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
examples/seedance-action/seedance-2-action-native-triptych.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
examples/seedance-action/seedance-2-action-product-reference-720p.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
examples/seedance-action/seedance-2-action-real-input-360p.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
examples/seedance-action/seedance-2-action-still-comparison.png filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
examples/seedance-action/seedance-2-action-triptych-web.mp4 filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -19,6 +19,38 @@ The browser artifact has 8,496 parameters and runs as three fused FP16 WebGPU re
|
|
| 19 |
|
| 20 |
## Before and after
|
| 21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |

|
| 23 |
|
| 24 |
The left column is the actual low-resolution source before either scaler is applied. The middle and right columns both start from that exact source. A held-out high-resolution frame is used only for PSNR measurement; it is not shown or mislabeled as the input.
|
|
|
|
| 19 |
|
| 20 |
## Before and after
|
| 21 |
|
| 22 |
+
### Moving Seedance 2.0 action test
|
| 23 |
+
|
| 24 |
+
<video controls loop playsinline muted width="100%" poster="examples/seedance-action/seedance-2-action-still-comparison.png">
|
| 25 |
+
<source src="https://huggingface.co/clark-labs/clark-realtime-ui-video-upscaler-x2/resolve/main/examples/seedance-action/seedance-2-action-triptych-web.mp4" type="video/mp4">
|
| 26 |
+
</video>
|
| 27 |
+
|
| 28 |
+
[Watch the web comparison](examples/seedance-action/seedance-2-action-triptych-web.mp4) · [Download the native-resolution comparison](examples/seedance-action/seedance-2-action-native-triptych.mp4) · [Inspect the enlarged detail comparison](examples/seedance-action/seedance-2-action-detail-triptych.mp4)
|
| 29 |
+
|
| 30 |
+
This five-second, 24 FPS test starts with a public 1280×720 Seedance 2.0 action
|
| 31 |
+
generation: a running puppy, tracking camera, fur, grass, flowers, and a moving
|
| 32 |
+
butterfly. A deterministic product-demo HUD is added before downsampling so the
|
| 33 |
+
test also contains the small labels, cards, and buttons this model targets. The
|
| 34 |
+
reference is downsampled and encoded to a real 640×360 input; bicubic and Clark
|
| 35 |
+
then receive the same decoded frames.
|
| 36 |
+
|
| 37 |
+
Across all 120 frames:
|
| 38 |
+
|
| 39 |
+
| Measure | Bicubic x2 | Clark x2 | Gain |
|
| 40 |
+
|---|---:|---:|---:|
|
| 41 |
+
| RGB PSNR | 32.482 dB | 33.185 dB | +0.703 dB |
|
| 42 |
+
| Edge PSNR | 17.986 dB | 20.659 dB | +2.673 dB |
|
| 43 |
+
| Temporal-delta PSNR | 38.520 dB | 37.598 dB | -0.922 dB |
|
| 44 |
+
|
| 45 |
+
The moving test therefore confirms stronger per-frame reconstruction and edge
|
| 46 |
+
recovery, while also exposing a real remaining gap: this frame-independent
|
| 47 |
+
student does not yet match bicubic's temporal-delta fidelity. The source,
|
| 48 |
+
separate bicubic and Clark outputs, held-out reference, exact prompt, seed,
|
| 49 |
+
hashes, crop, and metric definitions are all included under
|
| 50 |
+
`examples/seedance-action/` and `technical/seedance-action-video-evaluation.json`.
|
| 51 |
+
|
| 52 |
+
### Product-demo stills
|
| 53 |
+
|
| 54 |

|
| 55 |
|
| 56 |
The left column is the actual low-resolution source before either scaler is applied. The middle and right columns both start from that exact source. A held-out high-resolution frame is used only for PSNR measurement; it is not shown or mislabeled as the input.
|
examples/seedance-action/seedance-2-action-bicubic-x2-720p.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8dcaf905d44342e20439230519dfa5f1d8dc93eb72aaa69c26982b098808d8fe
|
| 3 |
+
size 7930126
|
examples/seedance-action/seedance-2-action-clark-x2-720p.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:13db9356c8bceb5e505c44da440d639dd77f3bf9dc4c6ff0bca19202cbb3d56f
|
| 3 |
+
size 8684658
|
examples/seedance-action/seedance-2-action-detail-triptych.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:443dc432ccaab00a9f184db0357b63bf84f6171835059b0916649613b616c325
|
| 3 |
+
size 8071264
|
examples/seedance-action/seedance-2-action-native-triptych.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:58e63eebf821a3847c9f5c750a59ae91e2df2cfbc21011e4e43b555034a3794c
|
| 3 |
+
size 15991637
|
examples/seedance-action/seedance-2-action-product-reference-720p.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:66f8d1fdef2dcf670f0a3d348aa97d8dd3f89f2e63e7ff7c5f8acff7af723ea8
|
| 3 |
+
size 7603928
|
examples/seedance-action/seedance-2-action-real-input-360p.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:56dad33586e1177c560df06d9a1c7f3a8e9f74ba76a2b4a677fe3a5775b3a197
|
| 3 |
+
size 2725824
|
examples/seedance-action/seedance-2-action-still-comparison.png
ADDED
|
Git LFS Details
|
examples/seedance-action/seedance-2-action-triptych-web.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7929903161bc142cc00b38ee98aa46709402d9fb62480b9e600e092450ddece0
|
| 3 |
+
size 2097711
|
technical/seedance-action-video-evaluation.json
ADDED
|
@@ -0,0 +1,70 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"format": "clark-seedance-action-video-comparison-v1",
|
| 3 |
+
"provenance": {
|
| 4 |
+
"generator": "ByteDance Seedance 2.0",
|
| 5 |
+
"public_prediction": "https://replicate.com/p/wckk6zjh01rmt0cxdt09fmsdzw",
|
| 6 |
+
"prompt": "A golden retriever puppy chasing butterflies through a sunlit meadow, soft bokeh background, cinematic camera slowly tracking the puppy",
|
| 7 |
+
"seed": 42,
|
| 8 |
+
"source_sha256": "463cfa3152b7e14b2ed0b7e28e9c08112addb79387207ec519e0cdb27c392df6",
|
| 9 |
+
"deterministic_overlay": "BackdropKit-style product HUD added before downsampling"
|
| 10 |
+
},
|
| 11 |
+
"comparison_contract": {
|
| 12 |
+
"reference": "1280x720 at 24 fps",
|
| 13 |
+
"real_input": "640x360 Lanczos downsample encoded before either scaler",
|
| 14 |
+
"bicubic": "ffmpeg bicubic x2 from the decoded real input",
|
| 15 |
+
"clark": "RGB student x2 from the same decoded real input",
|
| 16 |
+
"reference_visibility": "held out for metrics; shown only in the labeled still",
|
| 17 |
+
"detail_crop": [
|
| 18 |
+
320,
|
| 19 |
+
80,
|
| 20 |
+
960,
|
| 21 |
+
720
|
| 22 |
+
]
|
| 23 |
+
},
|
| 24 |
+
"metrics": {
|
| 25 |
+
"frames": 120,
|
| 26 |
+
"bicubic_psnr_db": 32.48194362681381,
|
| 27 |
+
"clark_psnr_db": 33.1853086395635,
|
| 28 |
+
"bicubic_edge_psnr_db": 17.986099802432612,
|
| 29 |
+
"clark_edge_psnr_db": 20.658625088551315,
|
| 30 |
+
"bicubic_temporal_psnr_db": 38.51974729131179,
|
| 31 |
+
"clark_temporal_psnr_db": 37.59777600502855,
|
| 32 |
+
"psnr_gain_db": 0.7033650127496927,
|
| 33 |
+
"edge_gain_db": 2.6725252861187023,
|
| 34 |
+
"temporal_gain_db": -0.9219712862832381
|
| 35 |
+
},
|
| 36 |
+
"artifacts": {
|
| 37 |
+
"examples/seedance-action/seedance-2-action-product-reference-720p.mp4": {
|
| 38 |
+
"bytes": 7603928,
|
| 39 |
+
"sha256": "66f8d1fdef2dcf670f0a3d348aa97d8dd3f89f2e63e7ff7c5f8acff7af723ea8"
|
| 40 |
+
},
|
| 41 |
+
"examples/seedance-action/seedance-2-action-real-input-360p.mp4": {
|
| 42 |
+
"bytes": 2725824,
|
| 43 |
+
"sha256": "56dad33586e1177c560df06d9a1c7f3a8e9f74ba76a2b4a677fe3a5775b3a197"
|
| 44 |
+
},
|
| 45 |
+
"examples/seedance-action/seedance-2-action-bicubic-x2-720p.mp4": {
|
| 46 |
+
"bytes": 7930126,
|
| 47 |
+
"sha256": "8dcaf905d44342e20439230519dfa5f1d8dc93eb72aaa69c26982b098808d8fe"
|
| 48 |
+
},
|
| 49 |
+
"examples/seedance-action/seedance-2-action-clark-x2-720p.mp4": {
|
| 50 |
+
"bytes": 8684658,
|
| 51 |
+
"sha256": "13db9356c8bceb5e505c44da440d639dd77f3bf9dc4c6ff0bca19202cbb3d56f"
|
| 52 |
+
},
|
| 53 |
+
"examples/seedance-action/seedance-2-action-native-triptych.mp4": {
|
| 54 |
+
"bytes": 15991637,
|
| 55 |
+
"sha256": "58e63eebf821a3847c9f5c750a59ae91e2df2cfbc21011e4e43b555034a3794c"
|
| 56 |
+
},
|
| 57 |
+
"examples/seedance-action/seedance-2-action-triptych-web.mp4": {
|
| 58 |
+
"bytes": 2097711,
|
| 59 |
+
"sha256": "7929903161bc142cc00b38ee98aa46709402d9fb62480b9e600e092450ddece0"
|
| 60 |
+
},
|
| 61 |
+
"examples/seedance-action/seedance-2-action-detail-triptych.mp4": {
|
| 62 |
+
"bytes": 8071264,
|
| 63 |
+
"sha256": "443dc432ccaab00a9f184db0357b63bf84f6171835059b0916649613b616c325"
|
| 64 |
+
},
|
| 65 |
+
"examples/seedance-action/seedance-2-action-still-comparison.png": {
|
| 66 |
+
"bytes": 1466704,
|
| 67 |
+
"sha256": "7fe2c5ab60b908c4edc70bba628f2ff8416119623a6285e0a9e2e955c5030a9f"
|
| 68 |
+
}
|
| 69 |
+
}
|
| 70 |
+
}
|