MaskModel::idle_secs (SAM 2 tracks its use); the engine frees a model idle for
ten minutes when no job runs, keeps the choice, and reloads it before a stroke
or a Propagate needs it. Docs: memory and limits.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
TrackRequest takes a list of objects (each with its prompts on its frames) and
the tracker reports (object, frame, result); frame encodings are shared, objects
may start on different frames. VideoTracker::propagate_objects runs a subset.
Limits: 32 objects. Test: two discs, one prompted on frame 0 and one on frame 2.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
ObjectTracker interface in the segment crate (prompts on some frames, the
clip tracked both ways, each frame streamed with confidence and an occluded
flag), implemented by SAM 2. MaskModel::tracker() is None for other models.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Apache-2.0 entry pinned to a Hugging Face commit with its SHA-256 and size,
listed in Settings > Roto Brush and installable with the existing verified
install flow. No weights are bundled.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* segment: bound checkpoint tensors and pickle work
* segment: bound entry size, memo clones, and total tensor elements
* segment: cap cumulative decompressed storage bytes
Fixes#385, Fixes#386.
crop now maps the square's corners to the output corners, reads the source position as a
pixel-centre index, rounds to a whole 0-255 value, and takes a Border (Zero for the detector,
Replicate for the mesh), matching MediaPipe's CPU ImageToTensor. The frame size product is
checked, so an absurd size gives an lo-filled tensor instead of an overflow panic.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Model::read bounded each tensor but nothing else, so a tiny hostile
.tflite could make Model::run abort on a huge CONV_2D patch allocation,
or keep it busy for minutes. Read now works out from the static shapes,
with checked arithmetic, what one run takes and refuses a model over a
fixed budget: kernel side (32), one operator's working space (2^22
floats), multiply-adds (2^33), values written (2^30), floats alive at
once (2^29) and constant values decoded or laid out at load (2^27).
The shapes run() computes itself now have to match the declared ones
(PAD, CONCATENATION, MAX_POOL_2D channels), and a zero dimension no
longer lets the others grow without bound.
Reported by KazW, with the suggested load-time budget.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A degenerate detection box (any NaN coordinate, or zero area) has an
overlap of 0 with itself, so `top` was partitioned into the remainder and
the loop never shrank `dets`. `top` now always joins its own cluster, so
every pass removes at least one detection. Well-formed boxes behave as
before (their self-overlap is 1).
Reported with the fix by KazW.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- effectcraft-segment: a FaceModel interface (find a face in a region, follow it; a Topology
maps the model's points to the tracker's landmarks and outline), a TensorFlow Lite reader and
interpreter (float ops, float16 weights folded; matches ai-edge-litert to ~1e-6), and
MediaPipe Face Landmarker (Google, Apache-2.0: BlazeFace + Face Mesh V2, 3.8 MB, pinned
SHA-256). The registry gains a task and authors per model; installed weights get a NOTICE.
- effectcraft-track: FaceTracker::new_with uses the model, falls back to the classical fitter
when it finds no face; chin and jaw come from the outline either way so measurements agree.
- Engine: roto_models becomes models (both tasks); face.models / face.model.* commands,
Settings ▸ Face Tracking, `wait` on select/install/download for scripts; track.mask reports
faceModel.
- Credits: ATTRIBUTION.md lists the optional models and the Apache-2.0 specs the code follows.
- Docs: README, ROADMAP (G7), gaps, parity (TRK-3 score unchanged pending G1), agents,
preferences, architecture, segment README.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New L1 crate effectcraft-segment: the MaskModel interface (prompts → foreground probability),
a registry of open-licensed models (licence must be MIT/Apache/BSD/…; URL, size, SHA-256),
a PyTorch checkpoint reader (zip + pickle subset), dependency-free SHA-256, and MobileSAM
(TinyViT-5M + SAM prompt encoder / two-way transformer / mask decoder) on rayon and ndarray's
safe GEMM. Loads the official mobile_sam.pt as published; checked against candle
(embedding 0.3% RMS, 99.9% equal mask pixels); ~0.7 s/frame encoder at 32 threads, cached
per frame, ~0.1 s decoder.
- track::roto: segment_with / propagate_with prompt a model with the strokes or the flow-warped
matte (box, core points, mask); its probability is a strong, unblurred prior for the same
graph cut (strokes stay hard constraints); a failing model or one disagreeing with the flow
(IoU < 0.5) is ignored. Prior gets a weight.
- effects::roto: Version 2.0 / 3.0 use the model chosen in Settings (set_model / model_for);
the model id is part of the chain seed.
- engine: roto_models (roto.models, roto.model.select / install / download / remove), weights in
the models folder (host::config_dir()/models, shared by the app, CLI and MCP), background
loading, downloads through the system curl, verified; Settings ▸ Roto Brush page (prefs
roto.model). host::config_dir moved from the desktop app.
- Tests: segment (kernels vs direct sums, checkpoint reader, SHA-256 vectors, registry), track
(fake model: prior shapes the cut, failing / disagreeing models ignored; real weights with
EFFECTCRAFT_MOBILESAM: base IoU 0.989, propagated ≥ 0.980 vs classic 0.973), engine (registry,
verification, selection; real install + load), MCP.
- Docs: README, ROADMAP, parity (MSK-4 done: 90/2/0), preferences, agents, architecture,
crates/segment/README.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Kristian MyramandClaude Sonnet 5.5
Dylan M. Taylor
aiongg
skylar
Brandon Thomas
Kaz WalkerandClaude Opus 5.5
BFlatandClaude Opus 5.5
Dexter