Files
effectcraft/crates/engine/src/lib.rs
T
611a282bea Plug-ins: external effect helpers (safe, out-of-process native effects) + plug-in API 2 (#889)
* Plug-ins: API 2 (PluginHost, padding, layer_span)

Split out of the OpenFX work so it can be reviewed on its own. Additive; the WebAssembly ABI
stays at 1.

- point3 / layer / text / hidden parameters, integer sliders, parameter groups.
- EffectPlugin::padding (transparent pixels around the buffer, at most 4096 per side).
- EffectPlugin::render_v2 with a PluginHost: the input at other times, other layers, parameters
  at other times, frame rate, per-instance key, the layer's source range.
- EffectEnv::layer_span from the renderer; MAX_PLUGINS 1024.
- plugin_error: a plug-in effect's last render error, for the UI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS

* Plug-ins: external effect helpers (crates/helper-host)

Native plug-in formats run in separate helper programs, never in EffectCraft's process
(docs/adr/0001-external-effect-helpers.md). New safe-Rust crate effectcraft-helper-host (L3,
native only, unsafe forbidden): it finds helper programs, starts each as a child process and
registers its effects through plug-in API 2, so the UI and command surface need nothing new.

- Protocol v1 (docs/external-helpers.md): JSON lines over stdin / stdout (helper replies are
  prefixed so plug-in chatter is ignored); pixels go through a frame exchange file (/dev/shm on
  Linux), never through the pipe. Render-time callbacks for the input at other times, other layers
  and parameters at other times, with nested renders through the same helper.
- A helper that exits, hangs past its timeout or breaks the protocol is stopped. The render fails
  with a message, the frame is left as it was, and the helper restarts for the next render. An
  effect that stops its helper 3 times is turned off (helper_blocklist.txt).
- Opt-in: nothing starts helpers unless the user asks (Effect > Scan for Effect Helpers), or opts
  in to starting them at launch with EFFECTCRAFT_HELPERS=1, an environment variable, so writing
  to the settings folder alone can't run code. effect.helpers.scan / load are refused for
  confined agent sessions (confine.rs), like file.executeFile.
- Discovery: helpers/ next to the app (Contents/Helpers on macOS), helpers/ in the settings
  folder, EFFECTCRAFT_HELPER_PATH.
- Commands effect.helpers.list / scan / load; Effect menu entries (translated); the last render
  error of an effect shows as a warning sign after its name in Effect Controls.
- serve(): the helper side for Rust helpers. tests/it/main.rs runs the whole protocol (render,
  padding, callbacks, nesting, threads, crash, restart, block list, hang) against a fake helper,
  with nothing to install.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS

* Effect helpers: address review (start failures, frame file mode, nextest listing)

- A helper that can't start (not installed any more, restarting too often, no frame exchange
  file) is reported but no longer charged to the effect being rendered: it is Failure::Refused,
  not Failure::Broken, so it never counts toward turning that effect off. New check in
  tests/it: once the helper is out of restarts, renders of an effect fail without blocking it
  (fails without the fix: the effect was turned off after 3 such renders).
- The frame exchange file is created owner-only (0o600) on Unix: it holds the user's frames and
  /dev/shm is shared. Asserted in the frames unit test.
- tests/it (harness = false) speaks the libtest listing protocol cargo-nextest uses:
  `--list --format terse` prints `helper_protocol: test` (nothing with `--ignored`), and
  `helper_protocol --exact` runs the checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS

* Effect helpers: bound writes to a helper, drain stderr, keep callbacks in step

- Writes to a helper go through a writer thread with the request timeout: a
  helper that stops reading its stdin (hung with a large callback answer or
  render request pending) no longer blocks the render thread forever while
  holding the helper.
- stderr is drained as bytes (lossy, in 64 KiB pieces) to the end: output that
  isn't UTF-8 stopped the reader, closing the pipe under the helper (EPIPE /
  SIGPIPE on its next write); lines were unbounded.
- Callback images that can't be written are answered with `null` instead of
  failing the render while the helper waits (which left the protocol out of
  step); a frame that can't be written before the request is sent is not
  charged to the effect.
- Mid-render calls use the render's own process: no respawn or heartbeat ping
  while the helper is waiting for an answer.
- Waiting for a busy helper is bounded (load timeout) instead of forever;
  status() no longer blocks on a running render.
- Reap the child when start-up fails after spawning; cap the block list read.
- Scripts start helpers only with Allow Scripts to Write Files and Access
  Network on (they run native code), as for File.execute().
- helper-host declares the workspace's no-panic crate lints; the desktop
  main() keeps its doc comment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Effect helpers: rustfmt the fake provider, build the test session in its initializer (clippy)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Plug-in Library: offer WebAssembly API 1 releases only; a helper already stopped mid-render isn't charged again

- WasmPlugin loads only modules at the WebAssembly ABI version, which stays 1 while
  PLUGIN_API_VERSION is 2: share it as effects::plugin::WASM_API_VERSION and have the
  library's compatibility check use it, so it never installs an API 2 release that
  can't load.
- with_running: a process that is already gone was stopped by what broke it (a nested
  render, charged there), so this render fails without charging its own effect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frames: plug-in effects count as reading other frames (plug-in API 2)

With main's narrowed frame invalidation (#2795), an edit inside a precomp under an
effect that reads its layer at other times must reach the layer's whole span.
Plug-in API 2 gives every plug-in effect the input at other times, so any effect
outside ec. counts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-05 11:11:57 +01:00

394 lines
18 KiB
Rust

//! The EffectCraft engine façade.
//!
//! Every user-visible action is a command with a stable id (`layer.newSolid`, `prop.set`,
//! `keys.easyEase`…) and JSON parameters, dispatched through [`Session::execute`]. The egui UI,
//! the CLI, the control channel and the MCP server all use this one entry point; that is what
//! makes the UI swappable and the whole app agent-drivable.
//!
//! The project is an `Arc<Project>` edited copy-on-write; undo keeps whole-project snapshots
//! (compositions are `Arc`s, so untouched comps are shared).
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
mod analysis;
mod background;
pub mod cmd;
mod footage;
pub mod history;
mod home;
mod interchange;
pub mod menus;
mod output_guard;
pub mod preview;
pub mod render_queue;
pub mod scriptui;
mod settings;
mod system;
pub mod viewer;
use std::sync::Arc;
use effectcraft_project::{Comp, ItemId, Layer, LayerId, Project, Uid};
use effectcraft_raster::Image;
use effectcraft_render::{ExprHost, FootageSource, LayerCache, NoFootage, RenderOpts, Renderer};
use effectcraft_time::Tick;
use serde::{Deserialize, Serialize};
use serde_json::Value;
pub use analysis::camera_track;
pub use analysis::mask_track;
pub use analysis::models;
pub use analysis::roto;
pub use analysis::tracking;
pub use analysis::warp;
pub use background::jobs;
pub use background::offload;
pub use background::remote;
pub use cmd as commands;
pub use cmd::{CommandSpec, command_specs, find as find_command};
pub use effectcraft_color as color;
pub use effectcraft_effects as effects;
pub use effectcraft_geom as geom;
pub use effectcraft_keyframe as keyframe;
pub use effectcraft_project as project;
pub use effectcraft_raster as raster;
pub use effectcraft_render as render;
pub use effectcraft_segment as segment;
pub use effectcraft_text as text;
pub use effectcraft_time as time;
pub use effectcraft_track as track;
pub use footage::doc_path;
pub use footage::footage_check;
pub use footage::footage_reload;
pub use footage::media_browser;
pub use footage::media_cache;
pub use footage::sequence;
pub use history::{Branch, History, HistoryNode};
pub use home::demo;
pub use home::learn;
pub use home::links;
pub use home::templates;
pub use interchange::psd_import;
pub use interchange::vector;
pub use interchange::xml_project;
use interchange::{keyframe_text, keyframe_text_effects};
pub use render_queue::{ExportJob, ExportResult, Exporter, JobState};
pub use settings::autosave;
pub use settings::config;
pub use settings::prefs;
pub use settings::shortcuts;
pub use system::guard;
pub use system::library;
pub use system::logging;
pub use system::perf;
pub use system::storage;
pub use system::sysinfo;
mod doc_state;
mod error;
mod event;
mod fonts;
mod services;
mod session;
pub use cmd::{animation_presets, text_presets};
pub use doc_state::{CompSelection, EditorState, KeyClip, KeyMove, KeyRef, LinkClip, VertexRef};
pub use error::{EngineError, Result};
pub use event::Event;
pub use fonts::{FontRow, font_features, font_menu, font_preview, font_styles, text_families};
pub use services::{FsServices, Importer, Services};
pub use session::upgrade_effects;
pub struct Session {
pub project: Arc<Project>,
pub history: History,
pub revision: u64,
/// Process-unique monotonic revision of the on-canvas editor's committed text and session.
/// Unlike [`Session::revision`], unrelated project edits do not advance it; the IME receipt
/// uses it when surrounding text is unavailable. Read it with
/// [`commands::text_edit::ime_stamp_revision`].
///
/// Stamps come from a process-global checked allocator and are never rewound: a failed
/// atomic batch keeps the highest intermediate stamp. On allocator exhaustion no stamp is
/// issued, so an unavailable receipt cannot be accepted with a reused revision.
pub ime_revision: u64,
pub saved_revision: u64,
/// The project as last opened or saved: the project is unmodified while it is this very
/// snapshot, so undoing back to the saved state clears the modified mark.
pub saved_project: Arc<Project>,
pub path: Option<String>,
pub state: EditorState,
pub services: Arc<dyn Services>,
pub footage: Arc<dyn FootageSource>,
pub expr: Option<Arc<dyn ExprHost>>,
/// GPU compositor (Mercury GPU Acceleration), when the frontend has one: used by renders
/// that ask for [`effectcraft_render::Backend::Gpu`]/`Auto`, and by Render Queue exports
/// when the project's renderer is the GPU.
pub accel: Option<Arc<dyn effectcraft_render::Accelerator>>,
/// Why there is no [`Session::accel`] (`render.backend`'s `why`): the host's reason, e.g. a
/// headless start without `--gpu` or the compositor's setup error.
pub accel_note: Option<String>,
/// The graphics adapter and backend the window draws with (`NVIDIA GeForce RTX 4070
/// (Vulkan)`), as the host reports it: the System Compatibility Report shows it.
pub window_adapter: Option<String>,
/// Expression syntax checker (set by the host that links the expression engine).
pub expr_check: Option<fn(&str) -> std::result::Result<(), String>>,
pub importer: Option<Arc<dyn Importer>>,
/// Render Queue encoder (the export layer); `None` = export unavailable.
pub exporter: Option<Arc<dyn Exporter>>,
/// Free-space hook for Render Settings ▸ Use Storage Overflow (`None`: never full).
pub storage_quota: Option<Arc<dyn effectcraft_project::render_queue::StorageQuota>>,
/// The running (or finished, not yet polled) render.
pub render_job: Option<render_queue::RenderJob>,
/// The running (or finished, not yet polled) track analysis.
pub track_job: Option<tracking::TrackJob>,
/// The running (or finished, not yet polled) mask track.
pub mask_job: Option<mask_track::MaskTrackJob>,
/// The running (or finished, not yet polled) Warp Stabilizer analysis.
pub warp_job: Option<warp::WarpJob>,
/// Warp Stabilizers waiting for (re-)analysis: (comp, layer, effect uid).
pub warp_pending: Vec<(ItemId, LayerId, Uid)>,
/// The running (or finished, not yet polled) 3D Camera Tracker analysis.
pub camera_job: Option<camera_track::CameraJob>,
/// 3D Camera Trackers waiting for (re-)analysis: (comp, layer, effect uid).
pub camera_pending: Vec<(ItemId, LayerId, Uid)>,
/// The running (or finished, not yet polled) Roto Brush propagation / Freeze.
pub roto_job: Option<roto::RotoJob>,
/// Trained models for Roto Brush and face tracking (Settings ▸ Roto Brush / Face Tracking).
pub models: models::Models,
/// Where installed models live (hosts set it; else `models` next to the settings).
pub models_dir: Option<std::path::PathBuf>,
/// Roto Brush instances edited since their last propagation: (comp, layer, effect uid).
pub roto_pending: Vec<(ItemId, LayerId, Uid)>,
pub events: Vec<Event>,
/// Commands executed: (id, params).
pub journal: Vec<(String, Value)>,
/// Processed-layer pixels reused across frames and edits (content-keyed, never stale).
pub layer_cache: Arc<LayerCache>,
/// The last Cryptomatte pick probe (see [`Session::crypto_probe_job`]). Public only so that
/// other crates can build a `Session { .. }`; its type is crate-private.
#[doc(hidden)]
pub crypto_probe: Arc<std::sync::Mutex<commands::CryptoProbeSlot>>,
/// The persistent disk cache (Settings ▸ Media & Disk Cache), when enabled.
pub disk_cache: Option<Arc<effectcraft_render::disk_cache::DiskCache>>,
/// Settings (Preferences).
pub prefs: prefs::Prefs,
/// Bumped whenever settings change (frontends re-apply theme, labels…).
pub prefs_revision: u64,
/// Where settings, shortcut presets and the crash-recovery sentinel are stored (`None` =
/// nothing persists).
pub config: Option<Arc<dyn config::ConfigStore>>,
/// Keyboard shortcut presets.
pub keymaps: shortcuts::Keymaps,
/// Scripts' `app.settings`, kept in the settings store (see
/// [`commands::scripts::SCRIPT_SETTINGS_FILE`]).
pub script_settings: commands::scripts::ScriptSettings,
/// Frontend-only commands offered for binding.
pub ui_commands: Vec<shortcuts::UiCommand>,
/// Cache of the resolved active preset (read it with [`Session::shortcuts`]).
pub shortcut_table: std::sync::OnceLock<shortcuts::ShortcutTable>,
/// Auto-save bookkeeping.
pub autosave: autosave::AutoSaveState,
/// Host-owned isolation for all auto-saves (e.g. one MCP session). Independent of settings
/// and project bookkeeping, so opening/saving a project or changing preferences keeps it.
pub autosave_folder_override: Option<std::path::PathBuf>,
/// The viewer snapshot (Take Snapshot / Show Snapshot).
pub snapshot: Option<viewer::Snapshot>,
/// The JavaScript scripting engine (set by the host that links `effectcraft-script`):
/// `script.run`, File ▸ Scripts ▸ Run Script File… (`.jsx`/`.js`) and the Script Console.
pub script: Option<ScriptRunner>,
/// Loads WebAssembly effect plug-ins (set by the host that links `effectcraft-plugin` with
/// its runtime): `effect.plugins.load`.
pub plugin_loader: Option<PluginLoader>,
/// External effect helpers (native hosts link `effectcraft-helper-host`):
/// `effect.helpers.list`, `effect.helpers.scan`, `effect.helpers.load`.
pub helpers: Option<HelperHooks>,
/// Renders a module without registering it (`library.preview`; set with `plugin_loader`).
pub plugin_previewer: Option<PluginPreviewer>,
/// Where effect plug-ins live (hosts set it; else `Plug-ins` next to the settings): the
/// Plug-in Library installs into its `Library` folder ([`library`]).
pub plugins_dir: Option<std::path::PathBuf>,
/// The Plug-in Library's catalog, worker status and the packages this session loaded.
pub library: library::Library,
/// ScriptUI windows and panels opened by scripts (see [`scriptui`]).
pub script_ui: scriptui::ScriptUi,
/// The last physical-memory reading (Settings ▸ Memory & CPU budgets); `None` = unknown
/// (headless sessions don't query it, see [`Session::memory_tick`]).
pub sys_memory: Option<sysinfo::SysMemory>,
/// The keyframe drag in progress (`keys.move` with a merge key).
pub key_move: Option<KeyMove>,
/// Reads the system's memory in the background for [`Session::memory_tick`].
pub memory_watch: sysinfo::MemoryWatch,
/// Reads the disk cache's free disk space in the background for [`Session::memory_tick`].
pub disk_watch: sysinfo::DiskWatch,
/// Switches Affect Nested Comps as last applied (a change empties the layer cache).
pub applied_nested_switches: Option<bool>,
/// Runs renders and analyses off the UI thread where there are no threads (the web app's
/// Web Workers, see [`offload`]); `None` = threads (desktop) or inline (wasm32).
pub offload: Option<Arc<dyn offload::Offload>>,
/// Analyses running in the [`Session::offload`] worker.
pub offloaded: Vec<offload::OffloadedJob>,
/// What the Media Browser browses (`None`: the local file system where there is one).
pub browser: Option<Arc<dyn media_browser::Browser>>,
/// The browser's storage (the web app; Settings ▸ Disk ▸ Browser Storage, `storage.*`).
pub storage: Option<Arc<dyn storage::StorageHost>>,
/// Offloaded jobs that ended, newest last (kind, error): how a caller waiting for one
/// (`wait: true` over the control channel in the browser) learns its outcome.
pub offload_log: Vec<(offload::JobKind, Option<String>)>,
/// Number of older completion entries evicted from the bounded offload log.
pub offload_log_start: usize,
/// Generic background tasks (Content-Aware Fill, Scene Edit Detection), see [`jobs`].
pub tasks: Vec<jobs::Task>,
/// Finished tasks, newest last (Progress panel, `jobs.list`).
pub job_log: Vec<jobs::JobRecord>,
pub next_task_id: u64,
/// The running Learn tutorial (Home ▸ Learn), see [`learn`].
pub learn: Option<learn::Progress>,
/// Check footage in the background after File ▸ Open ([`footage_check`]): frontends with an
/// event loop (the desktop app) turn this on; headless sessions run `footage.check` when
/// they want it.
pub check_footage_on_open: bool,
/// Footage paths that came from a document, with that document ([`doc_path`]).
pub stored_paths: doc_path::StoredPaths,
/// Each footage file's size and modification time when last seen (Settings ▸ Import ▸
/// Reload Footage Changed on Disk, [`footage_reload`]).
pub footage_stamps: std::collections::HashMap<String, footage_reload::Stamp>,
/// The disk cache's footage salt and the footage it was taken for ([`Session::refresh_disk_salt`]).
pub disk_salt: Option<(u64, u64)>,
/// How many script-issued commands are running ([`Session::execute_for_script`]): while one
/// is, settings that guard what scripts may do can't change (see
/// [`prefs::Scripting::same_permissions`]).
pub script_calls: ScriptCalls,
}
/// The count behind [`Session::running_for_script`]; only the engine changes it.
#[derive(Clone, Copy, Debug, Default)]
pub struct ScriptCalls(u32);
/// A thumbnail render ([`Session::thumbnail_job`]): (width, height, RGBA8).
pub type ThumbnailJob = Box<dyn FnOnce() -> Option<(u32, u32, Vec<u8>)> + Send>;
/// A script to run (see [`Session::script`]).
#[derive(Clone, Copy, Debug)]
pub struct ScriptRequest<'a> {
pub code: &'a str,
/// File name shown in errors and `$.fileName` (`console` for the Script Console).
pub name: &'a str,
/// Run in the Script Console's persistent context (variables survive between runs).
pub console: bool,
}
/// Runs a script against the session and reports `{ok, result, output, error: {message, line,
/// column, file} | null}`.
pub type ScriptRunner = fn(&mut Session, &ScriptRequest) -> Value;
/// Loads one WebAssembly effect plug-in (`bytes`, `source` path) into the effect registry and
/// describes it (`{id, name, category, version, api, params}`).
pub type PluginLoader = fn(&[u8], &str) -> std::result::Result<Value, String>;
/// What a host that runs external effect helpers provides (`docs/external-helpers.md`).
#[derive(Clone, Copy)]
pub struct HelperHooks {
/// Start the helpers installed in the host's search paths and register their effects
/// (`retry` forgets the block list first) → `{paths, helpers, loaded, errors}`.
pub scan: fn(bool) -> Value,
/// Load a file or folder through the helpers that accept it → `{loaded, errors}`.
pub load: fn(&str) -> std::result::Result<Value, String>,
/// The helpers and their state → `{helpers, blocked}`.
pub list: fn() -> Value,
}
/// Renders one WebAssembly effect module without registering it (the Plug-in Library preview):
/// `(bytes, width, height, premultiplied RGBA pixels, times in seconds)` → (manifest JSON, one frame
/// per time).
pub type PluginPreviewer = fn(&[u8], u32, u32, &[[f32; 4]], &[f64]) -> std::result::Result<(Value, Vec<Vec<[f32; 4]>>), String>;
impl Default for Session {
/// A new unsaved session: a default project that is both current and saved, no path, no
/// active jobs, the plain `FsServices`/`NoFootage` defaults, default preferences and keymaps,
/// empty history, events and journal, and a fresh process-wide IME stamp (`u64::MAX` when the
/// allocator is exhausted).
fn default() -> Self {
let project = Arc::new(Project::default());
Session {
saved_project: project.clone(),
project,
history: History::default(),
revision: 0,
// A fresh session gets its own process-monotonic stamp even before its first edit.
ime_revision: commands::text_edit::next_ime_revision().unwrap_or(u64::MAX),
saved_revision: 0,
path: None,
state: EditorState { snapping: true, view_layout: 1, ..Default::default() },
services: Arc::new(FsServices),
footage: Arc::new(NoFootage),
expr: None,
accel: None,
accel_note: None,
window_adapter: None,
expr_check: None,
importer: None,
exporter: None,
storage_quota: None,
render_job: None,
track_job: None,
mask_job: None,
warp_job: None,
warp_pending: vec![],
camera_job: None,
camera_pending: vec![],
roto_job: None,
models: Default::default(),
models_dir: None,
roto_pending: vec![],
events: vec![],
journal: vec![],
layer_cache: Arc::new(LayerCache::default()),
crypto_probe: Default::default(),
disk_cache: None,
prefs: prefs::Prefs::default(),
prefs_revision: 0,
config: None,
keymaps: shortcuts::Keymaps::default(),
script_settings: Default::default(),
ui_commands: vec![],
shortcut_table: std::sync::OnceLock::new(),
autosave: autosave::AutoSaveState::default(),
autosave_folder_override: None,
snapshot: None,
script: None,
plugin_loader: None,
helpers: None,
plugin_previewer: None,
plugins_dir: None,
library: Default::default(),
script_ui: scriptui::ScriptUi::default(),
sys_memory: None,
key_move: None,
memory_watch: Default::default(),
disk_watch: Default::default(),
applied_nested_switches: None,
offload: None,
offloaded: vec![],
offload_log: vec![],
offload_log_start: 0,
browser: None,
storage: None,
tasks: vec![],
job_log: vec![],
next_task_id: 1,
learn: None,
check_footage_on_open: false,
stored_paths: Default::default(),
footage_stamps: Default::default(),
disk_salt: None,
script_calls: ScriptCalls::default(),
}
}
}
#[cfg(test)]
mod rq_tests;
#[cfg(test)]
mod tests;