* Plug-ins: API 2 (PluginHost, padding, layer_span)
Split out of the OpenFX work so it can be reviewed on its own. Additive; the WebAssembly ABI
stays at 1.
- point3 / layer / text / hidden parameters, integer sliders, parameter groups.
- EffectPlugin::padding (transparent pixels around the buffer, at most 4096 per side).
- EffectPlugin::render_v2 with a PluginHost: the input at other times, other layers, parameters
at other times, frame rate, per-instance key, the layer's source range.
- EffectEnv::layer_span from the renderer; MAX_PLUGINS 1024.
- plugin_error: a plug-in effect's last render error, for the UI.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS
* Plug-ins: external effect helpers (crates/helper-host)
Native plug-in formats run in separate helper programs, never in EffectCraft's process
(docs/adr/0001-external-effect-helpers.md). New safe-Rust crate effectcraft-helper-host (L3,
native only, unsafe forbidden): it finds helper programs, starts each as a child process and
registers its effects through plug-in API 2, so the UI and command surface need nothing new.
- Protocol v1 (docs/external-helpers.md): JSON lines over stdin / stdout (helper replies are
prefixed so plug-in chatter is ignored); pixels go through a frame exchange file (/dev/shm on
Linux), never through the pipe. Render-time callbacks for the input at other times, other layers
and parameters at other times, with nested renders through the same helper.
- A helper that exits, hangs past its timeout or breaks the protocol is stopped. The render fails
with a message, the frame is left as it was, and the helper restarts for the next render. An
effect that stops its helper 3 times is turned off (helper_blocklist.txt).
- Opt-in: nothing starts helpers unless the user asks (Effect > Scan for Effect Helpers), or opts
in to starting them at launch with EFFECTCRAFT_HELPERS=1, an environment variable, so writing
to the settings folder alone can't run code. effect.helpers.scan / load are refused for
confined agent sessions (confine.rs), like file.executeFile.
- Discovery: helpers/ next to the app (Contents/Helpers on macOS), helpers/ in the settings
folder, EFFECTCRAFT_HELPER_PATH.
- Commands effect.helpers.list / scan / load; Effect menu entries (translated); the last render
error of an effect shows as a warning sign after its name in Effect Controls.
- serve(): the helper side for Rust helpers. tests/it/main.rs runs the whole protocol (render,
padding, callbacks, nesting, threads, crash, restart, block list, hang) against a fake helper,
with nothing to install.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS
* Effect helpers: address review (start failures, frame file mode, nextest listing)
- A helper that can't start (not installed any more, restarting too often, no frame exchange
file) is reported but no longer charged to the effect being rendered: it is Failure::Refused,
not Failure::Broken, so it never counts toward turning that effect off. New check in
tests/it: once the helper is out of restarts, renders of an effect fail without blocking it
(fails without the fix: the effect was turned off after 3 such renders).
- The frame exchange file is created owner-only (0o600) on Unix: it holds the user's frames and
/dev/shm is shared. Asserted in the frames unit test.
- tests/it (harness = false) speaks the libtest listing protocol cargo-nextest uses:
`--list --format terse` prints `helper_protocol: test` (nothing with `--ignored`), and
`helper_protocol --exact` runs the checks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS
* Effect helpers: bound writes to a helper, drain stderr, keep callbacks in step
- Writes to a helper go through a writer thread with the request timeout: a
helper that stops reading its stdin (hung with a large callback answer or
render request pending) no longer blocks the render thread forever while
holding the helper.
- stderr is drained as bytes (lossy, in 64 KiB pieces) to the end: output that
isn't UTF-8 stopped the reader, closing the pipe under the helper (EPIPE /
SIGPIPE on its next write); lines were unbounded.
- Callback images that can't be written are answered with `null` instead of
failing the render while the helper waits (which left the protocol out of
step); a frame that can't be written before the request is sent is not
charged to the effect.
- Mid-render calls use the render's own process: no respawn or heartbeat ping
while the helper is waiting for an answer.
- Waiting for a busy helper is bounded (load timeout) instead of forever;
status() no longer blocks on a running render.
- Reap the child when start-up fails after spawning; cap the block list read.
- Scripts start helpers only with Allow Scripts to Write Files and Access
Network on (they run native code), as for File.execute().
- helper-host declares the workspace's no-panic crate lints; the desktop
main() keeps its doc comment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Effect helpers: rustfmt the fake provider, build the test session in its initializer (clippy)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Plug-in Library: offer WebAssembly API 1 releases only; a helper already stopped mid-render isn't charged again
- WasmPlugin loads only modules at the WebAssembly ABI version, which stays 1 while
PLUGIN_API_VERSION is 2: share it as effects::plugin::WASM_API_VERSION and have the
library's compatibility check use it, so it never installs an API 2 release that
can't load.
- with_running: a process that is already gone was stopped by what broke it (a nested
render, charged there), so this render fails without charging its own effect.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* frames: plug-in effects count as reading other frames (plug-in API 2)
With main's narrowed frame invalidation (#2795), an edit inside a precomp under an
effect that reads its layer at other times must reach the layer's whole span.
Plug-in API 2 gives every plug-in effect the input at other times, so any effect
outside ec. counts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
* G8: plug-in library
Community Plug-in Library for WebAssembly effect plug-ins (#376): one
reviewed catalog (index.json), install / update / remove from Settings >
Plug-in Library (also Effect > Plug-in Library...), library.* commands,
start-up loading of installed packages with SHA-256 checks. Downloads go
through the system curl (https, GitHub hosts), as trained models do.
Preview before installing (library.preview): the release is downloaded,
checked like an install, and rendered in the sandbox on a built-in sample
frame at default settings (2 s at 12 fps) with a Before/After split, plus
the effect's parameters, "Open on GitHub" and an optional author video
link (catalog field preview.video). Authors need to do nothing for it.
Hardening: a module that fails to load leaves the installed copy alone,
Update All updates every package in one job, minApp also applies to an
explicit version, curl ignores ~/.curlrc and URL globbing, unique temp
files, percent-decoded file URLs. Plug-in effect slots grow from 64 to
256 with a clearer overflow error.
Docs: plugins.md "Plug-in Library", publishing-plugins.md (author guide
from zero to listed), gaps.md; examples/plugin-catalog is the starting
content of the catalog repository (schema, validation workflow).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* contributors: credit iseemhlyfta
* G8: plug-in library: doc comments on the library's functions
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: plug-in catalog check: require one object with schema 1 and a packages array
An empty file or several JSON values passed jq empty, and with
SKIP_SCHEMA=1 nothing else checked the catalog's shape.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: plug-in library: one preview texture set; catalog schema requires effects
The preview kept a texture set per package version previewed (about
6 MB each) for the whole session; it now keeps one, replaced when
another package is previewed. Releases in the catalog schema must list
at least one effect id, as the author guide says.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: plug-in catalog schema: effect ids follow the manifest's id rules
Letters, digits, '.', '_' and '-', and not the reserved 'ec.' prefix,
so library.missing can match them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: refuse Plug-in Library jobs while automation file access is confined
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: split the Plug-in Library page into toolbar, card, links and actions
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: show Plug-in Library toasts without a Roto job; cache preview frame checks
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: log Plug-in Library packages that fail to load at start-up
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: check an update before replacing a loaded package; Update All past failures
- An update to a package that is already loaded is now compiled and its manifest checked
(through the previewer, without registering it) before the new file is written and the old
version's folder deleted. Before, a release with the right SHA-256 that didn't load replaced
the working version and failed at the next start.
- Package ids that differ only in case, or end in a dot, would share one folder on Windows and
macOS: the catalog keeps the first of them, and ids can't end in a dot (schema too).
- Update All: a package that can't be picked or installed no longer stops the others; the
result names what was updated and what wasn't.
- Downloads: at most 5 redirects, and one that ends off GitHub's hosts is refused, as the module
docs say.
- A job that panics no longer leaves the library busy; poisoned status locks are used, not
dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: rustfmt the update test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Carry property, Font and Media Replacement overrides into the collapsed nested EvalCtx so nested layers render beyond the precomp frame instead of falling back to a clipped, flattened image. Key the layer cache on instance overrides to keep distinct collapsed instances isolated.
Problem: on Windows two tests failed on main although the app is fine; they assumed
macOS/Linux path formats.
Fix: the CLI test strips the verbatim path prefix that canonicalize adds on Windows (a
child's current_dir() never reports it); the host sink test uses a drive-letter path on
Windows (a path rooted without a drive is joined to the home drive by the engine, the test
expected the current drive). Test code only.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Animated GIFs came in as one still frame in 0.7.0; bfbdadd3 (after that
release) probes them as movies. This end-to-end test imports a GIF file
through the wired session into a comp and checks the layer shows each of
its frames at its time, the path the viewer and exports share.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiAJeFttZp83Fh7YuzAR5f
The host test writes a binary glTF cube (built in the test), imports it
with the real media layer, makes New Comp from Selection and renders
with no accelerator (the CPU path an Intel HD Graphics 5500 most likely
gets): the comp is Advanced 3D and the red cube covers part of the
frame. Before the fix the comp stayed Classic 3D and drew nothing. The
engine test checks the switch, its single undo step, the toasts, that a
second model doesn't switch again, and primitives.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New L1 crate effectcraft-segment: the MaskModel interface (prompts → foreground probability),
a registry of open-licensed models (licence must be MIT/Apache/BSD/…; URL, size, SHA-256),
a PyTorch checkpoint reader (zip + pickle subset), dependency-free SHA-256, and MobileSAM
(TinyViT-5M + SAM prompt encoder / two-way transformer / mask decoder) on rayon and ndarray's
safe GEMM. Loads the official mobile_sam.pt as published; checked against candle
(embedding 0.3% RMS, 99.9% equal mask pixels); ~0.7 s/frame encoder at 32 threads, cached
per frame, ~0.1 s decoder.
- track::roto: segment_with / propagate_with prompt a model with the strokes or the flow-warped
matte (box, core points, mask); its probability is a strong, unblurred prior for the same
graph cut (strokes stay hard constraints); a failing model or one disagreeing with the flow
(IoU < 0.5) is ignored. Prior gets a weight.
- effects::roto: Version 2.0 / 3.0 use the model chosen in Settings (set_model / model_for);
the model id is part of the chain seed.
- engine: roto_models (roto.models, roto.model.select / install / download / remove), weights in
the models folder (host::config_dir()/models, shared by the app, CLI and MCP), background
loading, downloads through the system curl, verified; Settings ▸ Roto Brush page (prefs
roto.model). host::config_dir moved from the desktop app.
- Tests: segment (kernels vs direct sums, checkpoint reader, SHA-256 vectors, registry), track
(fake model: prior shapes the cut, failing / disagreeing models ignored; real weights with
EFFECTCRAFT_MOBILESAM: base IoU 0.989, propagated ≥ 0.980 vs classic 0.973), engine (registry,
verification, selection; real install + load), MCP.
- Docs: README, ROADMAP, parity (MSK-4 done: 90/2/0), preferences, agents, architecture,
crates/segment/README.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
These crates have no production panic sites: add the deny attribute so they stay clean
(geom, color, model, automation, host, export, plugin, and the desktop, CLI and web apps). The
plugin's wasm-only test module gets a separate cfg(test) attribute so clippy recognises it as
test code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- GPU in browser job workers: each job worker opens its own WebGPU device
(deferred readbacks); the Render Queue export and the analyses' frame loops
are futures (render::passes, offload::run_request_async) whose frames
render in passes, awaiting the device between them. GPU particles and
Advanced 3D read back under keys too. workers.jobs reports passes and
readbacks; the smoke test measures them.
- Layer buffers in the browser's OPFS disk cache: frame workers back their
layer caches with a PrefetchStore; the page plans what to fetch before a
request from recent misses (LayerPrefetch over the shared DiskIndex).
- Templates embed footage (default on, size-capped with a warning) and
creating from a template extracts it next to the new project / into
browser storage.
- ICC LUT profiles (lut8/lut16/lutAtoB/lutBtoA, from the public ICC spec) in
Simulate Output > My Custom RGB, baked into a 3D LUT for the viewer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Render Settings: field render with 3:2 pulldown phases, effects/solo/guide
layer/color depth/frame blending/motion blur overrides, custom time sampling
(duration), storage overflow through a quota hook. Output Module: crop
(edges or region of interest), resize (presets, lock aspect, quality),
straight/premultiplied/alpha-only channels, post-render actions, include
project link, audio channels and PCM formats, WebM bitrate and key frame
interval. Render Settings / Output Module templates with built-ins and
defaults per slot (Edit > Templates dialogs), render logs next to the
output, Notify on completion. All through renderQueue.* commands and
persisted in the project.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Display Color Space setting joins the live list; View3D::Default maps to view.3d.default for
the Assign Shortcut submenu; stub tests use the stubs that remain.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reconciled with main's keyframe labels (same Keyframe::label field; keys.setLabel and
keys.selectLabelGroup from main are kept, M14.5's duplicates removed; Edit ▸ Label on
selected keys delegates to the label helper), proxies (RenderOpts::proxy next to
nested_switches/draft_shadows), offloaded renders (Work carries nested_switches), wasm
audio pump (preview mix rate), variable font axes and mask feather points.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Layer > Camera > Create Stereo 3D Rig (left/right eye comps with expression-linked eye
cameras, Stereo 3D Controls effect, stereo comp through 3D Glasses), Create Orbit Null,
Create Cameras from 3D Model; Layer > Light > Create Lights from 3D Model
(KHR_lights_punctual), Control Light with Camera, Create Environment Light Background
Layer (equirectangular sky drawn through the camera). glTF cameras and punctual lights
are parsed by effectcraft-model.
Advanced 3D: per-sample motion blur over the comp shutter, and blend modes / track
mattes / Preserve Transparency composited through the 2D path in depth order.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3D-1: Classic 3D depth of field renders the camera's Iris Shape (Fast
Rectangle box or Triangle…Decagon polygon), Iris Rotation, Roundness, Aspect
Ratio and Diffraction Fringe as a span-kernel gather blur, with Highlight
Gain/Threshold/Saturation boosting highlights first; tilted layers blur
progressively by per-pixel camera depth. Layer > Camera > Link Focus Distance
to Point of Interest / to Layer (expressions) and Set Focus Distance to Layer
replace the stubs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Items (footage and comps) carry a proxy rendered in their place at the source's size when
its switch and Render Settings > Proxy Use allow it; Create Proxy queues a half-resolution
render whose post-render action attaches it. Interpret Footage gains Separate Fields
(upper/lower first, fields become frames at twice the rate), Guess alpha with matte colour,
explicit matte colour, Invert Alpha, Interpret As Linear Light; non-square footage, solids
and precomps are stretched by their pixel aspect in the comp. The four proxy stubs are gone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sampleImage renders the sampled layer through the evaluating renderer's footage source
(EvalCtx::footage), cached per thread and frame. File > Import reads .json/.csv/.tsv as data
footage kept in the project. expr.errors collects failing expressions for the error bar;
expr.languageMenu lists the categorised Expression Language menu. Marker keys expose
protectedRegion and cue points; colorDepth, timeToFeetAndFrames, comp.activeCamera.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Advanced 3D comps render their 3D runs as one scene of world-space triangles: model and
primitive layers (meshes, skinning, animation clips by layer time), extruded text/shape
layers, and textured cards for every other layer with their Material Options. Shading is
physically based in linear light: Cook-Torrance GGX + Lambert for parallel/spot/point lights
(AE falloff and cone), ambient lights, image-based light from an Environment light's layer
(equirectangular; cosine irradiance map + roughness mip chain + split-sum BRDF fit), and
PCF shadow maps (spot/parallel/point cube faces). 2x2 SSAA, depth of field as a gather blur
by circle of confusion, back to the working encoding.
The software rasteriser (depth buffer, top-left rule, perspective-correct, opaque visibility
then sorted transparent blending) is the reference and CI path; effectcraft-gpu implements
Accelerator::raster_3d with a wgpu render pipeline (advanced3d.wgsl) and matches it within
tolerance. Environment layers are hidden from both renderers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New L3 crate effectcraft-gpu: a wgpu compute backend for the per-frame
compositing walk (layer transforms with the CPU's nearest/bilinear/bicubic
sampling and minification pre-filter, motion-blur accumulation, track
mattes, Preserve Transparency, layer style passes and knockout, all 38
blend modes as a WGSL port of color::blend, 8/16 bpc clamp+quantise,
colour-space conversions). 3D runs, adjustment and wireframe layers run on
the CPU between GPU steps. GPU effects with the CPU semantics: Gaussian,
Fast Box and Directional Blur, Glow, Levels, Curves, Hue/Saturation, Tint,
Fill, Gradient Ramp, Fractal Noise, Drop Shadow, Brightness & Contrast,
Exposure, Invert, Transform (registry gpu badge from effects::GPU_EFFECTS).
render: Backend (Cpu/Gpu/Auto) in RenderOpts, the Accelerator trait and
Renderer::accel, GPU effect chains in the effect stack, hooks for the GPU
walk. Project setting gpu_acceleration (Mercury GPU Acceleration / Software
Only), render.backend command, file.projectSettings renderer. Session and
Render Queue exports carry the accelerator. CLI --gpu and bench --gpu.
Tests compare GPU vs CPU renders (blend modes x bit depths, transforms,
masks, mattes, motion blur, colour management, styles, collapse,
adjustment fallback, every GPU effect) and skip without an adapter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- apps/effectcraft-web (L6): eframe web runner booting the shared EffectcraftApp with a browser
session: in-memory file table, <input type=file> pickers and drag-and-drop for open/import,
downloads for Save and Render Queue outputs (several files as one stored .zip), and the
window.effectcraft promise API over the control channel. Demo project on start.
- wasm32 runtime fixes: web-time for Instant/SystemTime (render, engine, export, ui-egui);
Frames renders queued jobs on the UI thread (Frames::pump) instead of a thread pool;
Render Queue runs inline; relative outputs resolve under / ; control PNGs inline.
- effectcraft-export: per-job output Sink (files to memory instead of disk); host FileExporter
carries it; test that sink exports touch no file.
- xtask web [--dev] [--serve PORT] (wasm-bindgen 0.2.129, optional wasm-opt, COOP/COEP server);
effectcraft-web in WEB_CRATES. docs/web.md, headless Chrome smoke test, favicon (+attribution).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Engine (all commands with undo/redo + tests in engine/src/tests_timeline.rs):
- keys.copy/paste (Edit Copy/Cut/Paste route to keys when keys are selected;
relative timing kept, paste at CTI, cross-property paste of same type),
keys.selectAll (Cmd+Alt+A), keys.nudge* (Alt+Arrow, Alt+Shift+Arrow),
keys.set / keys.setEase (graph editor drags), keys.info (dialog data),
keys.timeReverse; keys.interpolation gains continuous/auto Bezier, spatial
interpolation and roving; keys.velocity takes per-dimension values and
Continuous.
- Roving keys (keyframe::retime_roving): constant speed along the motion path.
- Layer > Time: layer.timeStretch (percent|duration, hold in/current/out),
layer.timeReverse (Cmd+Alt+R), layer.enableTimeRemap (Cmd+Alt+T),
layer.freezeFrame, layer.freezeOnLastFrame; render uses Time Remap as the
source time for precomp/footage layers.
- prop.separateDimensions (X/Y/Z Position; prop.set on Position writes them).
- layer.setParent compensates so the layer stays in place (compensate:false
to opt out).
- prop.pickWhip writes AE-style reference expressions (thisComp.layer(..),
effect(..)(..), mask(..).maskFeather, temp=…;[temp,temp], …[0]).
- prop.setExpression: syntax errors keep the text but disable the expression.
- expr: arrays of properties returned by an expression are resolved.
UI: Keyframe Velocity, Keyframe Interpolation and Time Stretch dialogs;
Graph Editor (value + speed graphs, draggable keys and Bezier/influence
handles, auto-zoom, fit, show selected, separate dimensions, hold/linear/
auto Bezier/Easy Ease buttons); key + layer context menus. Clears the
workspace clippy -D warnings failures.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
