* Plug-ins: API 2 (PluginHost, padding, layer_span)
Split out of the OpenFX work so it can be reviewed on its own. Additive; the WebAssembly ABI
stays at 1.
- point3 / layer / text / hidden parameters, integer sliders, parameter groups.
- EffectPlugin::padding (transparent pixels around the buffer, at most 4096 per side).
- EffectPlugin::render_v2 with a PluginHost: the input at other times, other layers, parameters
at other times, frame rate, per-instance key, the layer's source range.
- EffectEnv::layer_span from the renderer; MAX_PLUGINS 1024.
- plugin_error: a plug-in effect's last render error, for the UI.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS
* Plug-ins: external effect helpers (crates/helper-host)
Native plug-in formats run in separate helper programs, never in EffectCraft's process
(docs/adr/0001-external-effect-helpers.md). New safe-Rust crate effectcraft-helper-host (L3,
native only, unsafe forbidden): it finds helper programs, starts each as a child process and
registers its effects through plug-in API 2, so the UI and command surface need nothing new.
- Protocol v1 (docs/external-helpers.md): JSON lines over stdin / stdout (helper replies are
prefixed so plug-in chatter is ignored); pixels go through a frame exchange file (/dev/shm on
Linux), never through the pipe. Render-time callbacks for the input at other times, other layers
and parameters at other times, with nested renders through the same helper.
- A helper that exits, hangs past its timeout or breaks the protocol is stopped. The render fails
with a message, the frame is left as it was, and the helper restarts for the next render. An
effect that stops its helper 3 times is turned off (helper_blocklist.txt).
- Opt-in: nothing starts helpers unless the user asks (Effect > Scan for Effect Helpers), or opts
in to starting them at launch with EFFECTCRAFT_HELPERS=1, an environment variable, so writing
to the settings folder alone can't run code. effect.helpers.scan / load are refused for
confined agent sessions (confine.rs), like file.executeFile.
- Discovery: helpers/ next to the app (Contents/Helpers on macOS), helpers/ in the settings
folder, EFFECTCRAFT_HELPER_PATH.
- Commands effect.helpers.list / scan / load; Effect menu entries (translated); the last render
error of an effect shows as a warning sign after its name in Effect Controls.
- serve(): the helper side for Rust helpers. tests/it/main.rs runs the whole protocol (render,
padding, callbacks, nesting, threads, crash, restart, block list, hang) against a fake helper,
with nothing to install.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS
* Effect helpers: address review (start failures, frame file mode, nextest listing)
- A helper that can't start (not installed any more, restarting too often, no frame exchange
file) is reported but no longer charged to the effect being rendered: it is Failure::Refused,
not Failure::Broken, so it never counts toward turning that effect off. New check in
tests/it: once the helper is out of restarts, renders of an effect fail without blocking it
(fails without the fix: the effect was turned off after 3 such renders).
- The frame exchange file is created owner-only (0o600) on Unix: it holds the user's frames and
/dev/shm is shared. Asserted in the frames unit test.
- tests/it (harness = false) speaks the libtest listing protocol cargo-nextest uses:
`--list --format terse` prints `helper_protocol: test` (nothing with `--ignored`), and
`helper_protocol --exact` runs the checks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS
* Effect helpers: bound writes to a helper, drain stderr, keep callbacks in step
- Writes to a helper go through a writer thread with the request timeout: a
helper that stops reading its stdin (hung with a large callback answer or
render request pending) no longer blocks the render thread forever while
holding the helper.
- stderr is drained as bytes (lossy, in 64 KiB pieces) to the end: output that
isn't UTF-8 stopped the reader, closing the pipe under the helper (EPIPE /
SIGPIPE on its next write); lines were unbounded.
- Callback images that can't be written are answered with `null` instead of
failing the render while the helper waits (which left the protocol out of
step); a frame that can't be written before the request is sent is not
charged to the effect.
- Mid-render calls use the render's own process: no respawn or heartbeat ping
while the helper is waiting for an answer.
- Waiting for a busy helper is bounded (load timeout) instead of forever;
status() no longer blocks on a running render.
- Reap the child when start-up fails after spawning; cap the block list read.
- Scripts start helpers only with Allow Scripts to Write Files and Access
Network on (they run native code), as for File.execute().
- helper-host declares the workspace's no-panic crate lints; the desktop
main() keeps its doc comment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Effect helpers: rustfmt the fake provider, build the test session in its initializer (clippy)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Plug-in Library: offer WebAssembly API 1 releases only; a helper already stopped mid-render isn't charged again
- WasmPlugin loads only modules at the WebAssembly ABI version, which stays 1 while
PLUGIN_API_VERSION is 2: share it as effects::plugin::WASM_API_VERSION and have the
library's compatibility check use it, so it never installs an API 2 release that
can't load.
- with_running: a process that is already gone was stopped by what broke it (a nested
render, charged there), so this render fails without charging its own effect.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* frames: plug-in effects count as reading other frames (plug-in API 2)
With main's narrowed frame invalidation (#2795), an edit inside a precomp under an
effect that reads its layer at other times must reach the layer's whole span.
Plug-in API 2 gives every plug-in effect the input at other times, so any effect
outside ec. counts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
* G8: plug-in library
Community Plug-in Library for WebAssembly effect plug-ins (#376): one
reviewed catalog (index.json), install / update / remove from Settings >
Plug-in Library (also Effect > Plug-in Library...), library.* commands,
start-up loading of installed packages with SHA-256 checks. Downloads go
through the system curl (https, GitHub hosts), as trained models do.
Preview before installing (library.preview): the release is downloaded,
checked like an install, and rendered in the sandbox on a built-in sample
frame at default settings (2 s at 12 fps) with a Before/After split, plus
the effect's parameters, "Open on GitHub" and an optional author video
link (catalog field preview.video). Authors need to do nothing for it.
Hardening: a module that fails to load leaves the installed copy alone,
Update All updates every package in one job, minApp also applies to an
explicit version, curl ignores ~/.curlrc and URL globbing, unique temp
files, percent-decoded file URLs. Plug-in effect slots grow from 64 to
256 with a clearer overflow error.
Docs: plugins.md "Plug-in Library", publishing-plugins.md (author guide
from zero to listed), gaps.md; examples/plugin-catalog is the starting
content of the catalog repository (schema, validation workflow).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* contributors: credit iseemhlyfta
* G8: plug-in library: doc comments on the library's functions
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: plug-in catalog check: require one object with schema 1 and a packages array
An empty file or several JSON values passed jq empty, and with
SKIP_SCHEMA=1 nothing else checked the catalog's shape.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: plug-in library: one preview texture set; catalog schema requires effects
The preview kept a texture set per package version previewed (about
6 MB each) for the whole session; it now keeps one, replaced when
another package is previewed. Releases in the catalog schema must list
at least one effect id, as the author guide says.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: plug-in catalog schema: effect ids follow the manifest's id rules
Letters, digits, '.', '_' and '-', and not the reserved 'ec.' prefix,
so library.missing can match them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: refuse Plug-in Library jobs while automation file access is confined
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: split the Plug-in Library page into toolbar, card, links and actions
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: show Plug-in Library toasts without a Roto job; cache preview frame checks
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: log Plug-in Library packages that fail to load at start-up
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: check an update before replacing a loaded package; Update All past failures
- An update to a package that is already loaded is now compiled and its manifest checked
(through the previewer, without registering it) before the new file is written and the old
version's folder deleted. Before, a release with the right SHA-256 that didn't load replaced
the working version and failed at the next start.
- Package ids that differ only in case, or end in a dot, would share one folder on Windows and
macOS: the catalog keeps the first of them, and ids can't end in a dot (schema too).
- Update All: a package that can't be picked or installed no longer stops the others; the
result names what was updated and what wasn't.
- Downloads: at most 5 redirects, and one that ends off GitHub's hosts is refused, as the module
docs say.
- A job that panics no longer leaves the library busy; poisoned status locks are used, not
dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* G8: rustfmt the update test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Add After Effects project import with bounded parsing, explicit loss reports, and regression coverage.
Fetch the cleaned external test corpus at commit 97262276f625afdeeb61787e4cbb74b3f17af510 and verify each fixture against the pinned SHA-256 manifest.
* gpu: run Content-Aware Fill's PatchMatch on the device
Content-Aware Fill's synthesis is the stage that makes the tool slow: at
1920×1080 with a 300×200 hole, PatchMatch costs ≈ 1.1 s per frame on the CPU,
and it is what Object, Surface and every refining frame spend their time in.
Add a `FillBackend` trait to `effectcraft-raster` whose stages (the membrane
solve, the flow fields, PatchMatch) default to the CPU reference, and
`Accelerator::inpaint()` on the compositor, which is how the engine asks for a
device backend. `effectcraft_gpu::inpaint` implements PatchMatch: the
multi-scale pyramid, the EM rounds, the propagation (a two-pass checkerboard
instead of the CPU's scan order), the random search, the SSD, the
histogram-based σ² and the weighted vote, as compute kernels in
`shaders/inpaint.wgsl`.
The fill's frames are read back to the CPU before it runs (analyses composite
on the CPU by design), so the backend is about the fill's own work; the
orchestration, the flow-guided propagation and the per-frame flow control stay
on the CPU. Only the hole's bounds cross back to the CPU, which is what makes
the offload pay.
Measured on an Intel Arc (MTL) iGPU, 1920×1080, 300×200 hole:
CPU 1087 ms GPU 397 ms (2.7×, 41.0 dB PSNR against the CPU's synthesis)
The membrane (harmonic) solve deliberately stays on the CPU: the same
measurement has it at ≈ 90 ms on the device against ≈ 55 ms on the CPU,
because a fill's full-frame data movement (upload, the known pyramid, a copy)
does not shrink with the hole, while the relaxation does. The device takes only
the stage where its parallel search pays; `membrane_tex` still runs on the
device inside PatchMatch, where it is the coarsest level's start and never
crosses back.
Closes#1839.
* gpu: fix the fill's vote histogram and run its flow on the device
Review follow-ups on the Content-Aware Fill GPU port, plus the fill's flow.
Fixes:
* imp_hist kept about one sample in 256: each of the 256 threads scanned a
strided subset of the pixels and retained only those whose bin selected it,
so the vote's sigma² came from a sample correlated with the pixel index (and
a small hole usually fell back to 1.0, making the weights near-uniform). One
workgroup now accumulates the whole population through shared atomic bins,
with a test that pins the sample count.
* imp_pm_prop and imp_pm_rand were dispatched over grow(near, 1) but with
near's origin, so the ring's left column and top row were never written
(NaN under test, stale pooled data in production) while a strip past it was.
`over` now derives the shader's origin from the box it sizes the dispatch
from, so a dispatch's region and its origin can no longer disagree.
* Gpu::inpaint() requires its kernels to have built. Otherwise every stage
paid an upload and a failed dispatch to fall back to the CPU, while the fill
still reported the GPU path.
* imp_rnd's XOR chain parenthesises its multiplicative terms: WGSL's grammar
makes each XOR operand a unary expression, so `a * b ^ c * d` is rejected
by spec-compliant parsers (naga accepts it).
* FillBackend documents that its stages may be called concurrently.
* The coarsest membrane's sweep count (64, against the CPU's 200) is now
justified: matching 200 measured ~6% slower over the fill with no change in
the result.
And the flow: Gpu::flow runs block_flow's coarse-to-fine search, sub-pixel fit
and vector median on the device. The luma is the CPU's (flow::luma_pair), so
the field matches block_flow's to a float rounding and the propagation sees
the same motion; the hole completion stays on the CPU with the membrane. A
four-frame 1920x1080 fill goes from 1.27x to 1.42x against the CPU, and the
device path is covered for holes from 16x16 to 300x200.
* gpu: keep the fill vote's histogram range on HDR input
imp_hist binned every distance at a fixed 256 bins per unit and clamped to
1023, so the vote's sigma2 topped out at 3.998. A 32-bpc frame's channels can
exceed 1, and masked_frame forwards them unclamped: a patch distance of 1600
then reported sigma2 ~ 4, exp(-1600/(2*sigma2)) rounded to zero in f32, and
imp_pm_vote kept the prior image instead of synthesising.
The histogram now reports the largest finite distance it saw (atomicMax on the
float bits, carried in bin 0's unused .g channel). SDR input keeps the fixed
0..4 range with identical bins; HDR input is re-binned over the observed range
and percentile() maps the bin centre back through it. Nothing clamps the input.
Also group over/sweeps/pm_level's arguments into small structs: the workspace
denies too_many_arguments and the lint policy (#2496) does not allow new
#[allow]s. No behaviour change on those paths.
* engine: don't report Edge Blend fills as device fills
Edge Blend calls only the membrane, which the GPU backend leaves to the CPU
reference, so run_plan's "Filling (GPU)" status was wrong for it whenever a
backend was present. Object and Surface still run the flow and PatchMatch on
the device and keep the label.
CodeRabbit's #1856 finding.
* gpu: run the fill's device stages inside error scopes
An allocation or validation failure inside patch_image/flow_image left an
invalid texture whose later use raised an uncaptured error, retiring the
shared compositor instead of falling back to the CPU for that frame. Wrap both
stages in Gpu::within_memory, whose scopes are native thread-local, so the
fill's rayon workers do not share them.
---------
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Add 308 Traditional Chinese effect names and 21 category labels, written from the English source with Taiwan creative-software terms. Update the completeness tests to require the zh-hant table and recount the localization docs.
effects-parity.md and gaps.md: the Selection is picked in the viewer. control-protocol.md: the
viewer.cryptoPick element ids. footage.md: a paragraph on the eyedropper.
SAM 2's instance now reads wgpu's environment variables, as the render context's does, so a driver such as
Mesa's Dozen (WGPU_ALLOW_UNDERLYING_NONCOMPLIANT_ADAPTER=1) is visible to it, and WGPU_POWER_PREF applies.
An adapter without compute shaders is refused before a device is made on it, so the CPU takes over with one
line instead of a validation error per kernel. The refused adapter is kept for the life of the process, as
the device on it was: dropping it crashed the tests on WSL2's GL driver. Gpu::shared no longer repeats
"SAM 2 compute:" in its errors.
A run reads the project, edits it and writes the whole file back, so two
at once silently dropped the edits of whichever saved first. A run that
saves over the file it opened (--save, or --save-as the same file) now
holds an OS lock on `.<name>.lock` next to the project from before its
read until it exits. The next run says so on stderr, waits, then edits
what the first one saved. Ported from filmcraft-cli
(storytold/filmcraft#3926).
The OS drops the lock however the process ends, so a crash leaves nothing
stale. If the lock can't be taken the run warns and saves as it did
before. A missing project stops the run before it opens anything, with
the same error opening gives, so a project created in between is never
saved without the lock. The desktop app and `mcp` don't take this lock.
camera.solveStatus said `solved: true` for any stored solve, but the solver
keeps the frames it cannot resect as unsolved and interpolates the camera
there. `solved` now needs every frame, the same check camera.exportBlender
makes, and `partial` marks a solve with gaps, next to `solvedFrames`.
Offset Paths' Amount, Line Join, Miter Limit, Copies and Copy Offset,
and Zig Zag's Size, Ridges per segment and Points reported their
internal ids as matchName, so property("ADBE Vector Offset Amount") and
the other documented names returned null. They now report and resolve
by ADBE Vector Offset Amount, Line Join, Miter Limit, Copies and Copy
Offset, and ADBE Vector Zigzag Size, Detail and Points. Lookups by
display name and internal id still work.
The command checks look at the paths an agent gives, but a project it opens or
imports from a read root names its own footage, which was read wherever it is
when the project rendered: render_frame returned its pixels.
Backend::confine now wraps a headless session's footage and services:
ConfinedFootage reads frames, audio, aux channels, models and vector frames
only for footage inside the read roots (problem() says why the rest shows
nothing), and ConfinedServices checks read_file, exists, stamp, list_dir and
is_dir the same way. Writes are left to the command checks.
Test: automation_roots_confine_the_footage_a_project_names covers an absolute
path, `../`, a linked file and a linked folder, each opened and imported.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A shortcut belongs to the application, a panel or a context (editing text, a paint or
camera tool active, a mask selected). The narrowest active scope runs a chord: an active
context, the focused panel, the panel sharing its shortcuts (Timeline and Composition),
the application. A context's shortcuts never run outside it.
- shortcuts.rs: scope table for engine commands and menu entries, scope_rank,
scope_label; UiCommand can state its scope.
- Conflicts: the same chord in two scopes is an override (both keep working), reported
with overrides: true; shortcuts.conflicts lists same-scope pairs only.
- Scoped now, keys unchanged: Remember / Apply Interpretation and Delete Without
Confirmation (Project), Duplicate Render Item (Render Queue).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds Swedish (Svenska) as an interface language in the menu, panel and
effect catalogs, written from the English source. The effect rows are
partial like Japanese and Georgian: an effect without a Swedish row draws
its English name, so a new effect does not block other PRs (#2244).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bench --roto per size: no larger size tracked better on the synthetic clips and
Large lost the object after it re-entered; the hints say what was measured.
The bar shows the current object's stroke count and can remove its strokes.
An env-gated test runs the in-app download of a size.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Valentin Klein
gmrmad1andFredApps
Brandon ThomasandClaude Opus 5.5
SHXJ
Matej Kozlovský
JonRC
dotCooCoo
adikshan11andClaude Opus 5.5
aiongg
Florent Siegenthaler
bflatastic
pravdomilandClaude Opus 5.5
Patrik LermonandClaude Opus 5.5
Kristian Myram