daniel543x/effectcraft
1
0 Fork 0
mirror of https://github.com/storytold/effectcraft.git synced 2026-10-31 07:38:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

S2.52: refreshed benchmark (24 frames, SAM 2 on the GPU)

Browse Source 
Occlusion 0.491 classical / 0.494 MobileSAM / 0.989 SAM 2; re-entry 0.618 / 0.620 / 0.982.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Kristian MyramandClaude Sonnet 5.5 committed 2026-10-30 08:43:31 +01:00
1 parent ee3498b24d
commit 7f56fd8709
1 file changed
+9 -9
Download Patch File Download Diff File Expand all files Collapse all files
docs/roto-brush-sam2.md
+9 -9
View File
@@ -88,18 +88,18 @@ encoder's 64 × 64 level matches PyTorch to 7e-6, the memory attention to 1.8e-5
times the pieces on both devices (`EFFECTCRAFT_SAM2_PROFILE=1` prints a frame's breakdown).
## Benchmark
`effectcraft-cli bench --roto [--frames N] [--sam2 model.safetensors] [--mobilesam mobile_sam.pt]` runs synthetic clips
with known mattes through all three. 12 frames, debug build, this PC; mean IoU (mean IoU after the object returns), seconds
`effectcraft-cli bench --roto [--frames N] [--sam2 model.safetensors] [--mobilesam mobile_sam.pt]` runs synthetic clips with
known mattes through all three. 24 frames, debug build, SAM 2 on the GPU; mean IoU (mean IoU after the object returns), seconds
per frame:
| Clip | Classical | MobileSAM | SAM 2.1 Tiny |
|---|---|---|---|
| move | 0.982 | 0.992 | 0.993 |
| light | 0.981 | 0.992 | 0.992 |
| occlusion | 0.978 | 0.881 | 0.988 |
| reenter | 0.632 (0.000) | 0.499 (0.725) | 0.975 (0.967) |
| s/frame | 0.01 | 1.1 | 8.2 |
| move | 0.980 | 0.989 | 0.992 |
| light | 0.979 | 0.990 | 0.992 |
| occlusion (a bar hides the disk, then clears it) | 0.491 (0.000) | 0.494 (0.000) | 0.989 (0.988) |
| reenter (the disk leaves and comes back) | 0.618 (0.000) | 0.620 (0.000) | 0.982 (0.976) |
| s/frame | 0.01 | 0.5–1.3 | 0.8 |
MobileSAM prompts every frame independently from the flow-warped matte, so it follows the object while the flow does and
loses it when it is hidden; SAM 2's streaming memory is what lets it re-acquire the object. The synthetic clips are easy
(one textured disc), so the "move" and "light" rows mostly show all three working; the re-entry row is the difference.
loses it when it is hidden; SAM 2's streaming memory is what lets it re-acquire the object. The synthetic clips are easy (one
textured disc), so the "move" and "light" rows mostly show all three working; the last two rows are the difference.