mirror of
https://github.com/storytold/effectcraft.git
synced 2026-11-06 02:20:50 +00:00
Plug-ins: external effect helpers (safe, out-of-process native effects) + plug-in API 2 (#889)
* Plug-ins: API 2 (PluginHost, padding, layer_span) Split out of the OpenFX work so it can be reviewed on its own. Additive; the WebAssembly ABI stays at 1. - point3 / layer / text / hidden parameters, integer sliders, parameter groups. - EffectPlugin::padding (transparent pixels around the buffer, at most 4096 per side). - EffectPlugin::render_v2 with a PluginHost: the input at other times, other layers, parameters at other times, frame rate, per-instance key, the layer's source range. - EffectEnv::layer_span from the renderer; MAX_PLUGINS 1024. - plugin_error: a plug-in effect's last render error, for the UI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Plug-ins: external effect helpers (crates/helper-host) Native plug-in formats run in separate helper programs, never in EffectCraft's process (docs/adr/0001-external-effect-helpers.md). New safe-Rust crate effectcraft-helper-host (L3, native only, unsafe forbidden): it finds helper programs, starts each as a child process and registers its effects through plug-in API 2, so the UI and command surface need nothing new. - Protocol v1 (docs/external-helpers.md): JSON lines over stdin / stdout (helper replies are prefixed so plug-in chatter is ignored); pixels go through a frame exchange file (/dev/shm on Linux), never through the pipe. Render-time callbacks for the input at other times, other layers and parameters at other times, with nested renders through the same helper. - A helper that exits, hangs past its timeout or breaks the protocol is stopped. The render fails with a message, the frame is left as it was, and the helper restarts for the next render. An effect that stops its helper 3 times is turned off (helper_blocklist.txt). - Opt-in: nothing starts helpers unless the user asks (Effect > Scan for Effect Helpers), or opts in to starting them at launch with EFFECTCRAFT_HELPERS=1, an environment variable, so writing to the settings folder alone can't run code. effect.helpers.scan / load are refused for confined agent sessions (confine.rs), like file.executeFile. - Discovery: helpers/ next to the app (Contents/Helpers on macOS), helpers/ in the settings folder, EFFECTCRAFT_HELPER_PATH. - Commands effect.helpers.list / scan / load; Effect menu entries (translated); the last render error of an effect shows as a warning sign after its name in Effect Controls. - serve(): the helper side for Rust helpers. tests/it/main.rs runs the whole protocol (render, padding, callbacks, nesting, threads, crash, restart, block list, hang) against a fake helper, with nothing to install. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Effect helpers: address review (start failures, frame file mode, nextest listing) - A helper that can't start (not installed any more, restarting too often, no frame exchange file) is reported but no longer charged to the effect being rendered: it is Failure::Refused, not Failure::Broken, so it never counts toward turning that effect off. New check in tests/it: once the helper is out of restarts, renders of an effect fail without blocking it (fails without the fix: the effect was turned off after 3 such renders). - The frame exchange file is created owner-only (0o600) on Unix: it holds the user's frames and /dev/shm is shared. Asserted in the frames unit test. - tests/it (harness = false) speaks the libtest listing protocol cargo-nextest uses: `--list --format terse` prints `helper_protocol: test` (nothing with `--ignored`), and `helper_protocol --exact` runs the checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Effect helpers: bound writes to a helper, drain stderr, keep callbacks in step - Writes to a helper go through a writer thread with the request timeout: a helper that stops reading its stdin (hung with a large callback answer or render request pending) no longer blocks the render thread forever while holding the helper. - stderr is drained as bytes (lossy, in 64 KiB pieces) to the end: output that isn't UTF-8 stopped the reader, closing the pipe under the helper (EPIPE / SIGPIPE on its next write); lines were unbounded. - Callback images that can't be written are answered with `null` instead of failing the render while the helper waits (which left the protocol out of step); a frame that can't be written before the request is sent is not charged to the effect. - Mid-render calls use the render's own process: no respawn or heartbeat ping while the helper is waiting for an answer. - Waiting for a busy helper is bounded (load timeout) instead of forever; status() no longer blocks on a running render. - Reap the child when start-up fails after spawning; cap the block list read. - Scripts start helpers only with Allow Scripts to Write Files and Access Network on (they run native code), as for File.execute(). - helper-host declares the workspace's no-panic crate lints; the desktop main() keeps its doc comment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Effect helpers: rustfmt the fake provider, build the test session in its initializer (clippy) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Plug-in Library: offer WebAssembly API 1 releases only; a helper already stopped mid-render isn't charged again - WasmPlugin loads only modules at the WebAssembly ABI version, which stays 1 while PLUGIN_API_VERSION is 2: share it as effects::plugin::WASM_API_VERSION and have the library's compatibility check use it, so it never installs an API 2 release that can't load. - with_running: a process that is already gone was stopped by what broke it (a nested render, charged there), so this render fails without charging its own effect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * frames: plug-in effects count as reading other frames (plug-in API 2) With main's narrowed frame invalidation (#2795), an edit inside a precomp under an effect that reads its layer at other times must reach the layer's whole span. Plug-in API 2 gives every plug-in effect the input at other times, so any effect outside ec. counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
This commit is contained in:
39 files changed
+2898
-62
No files matched your search
Generated
+12
@@ -1633,6 +1633,17 @@ dependencies = [
|
||||
"wgpu",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "effectcraft-helper-host"
|
||||
version = "0.21.0"
|
||||
dependencies = [
|
||||
"bytemuck",
|
||||
"effectcraft-effects",
|
||||
"log",
|
||||
"serde",
|
||||
"serde_json",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "effectcraft-hevcenc"
|
||||
version = "0.21.0"
|
||||
@@ -1647,6 +1658,7 @@ dependencies = [
|
||||
"effectcraft-engine",
|
||||
"effectcraft-export",
|
||||
"effectcraft-expr",
|
||||
"effectcraft-helper-host",
|
||||
"effectcraft-interchange",
|
||||
"effectcraft-media",
|
||||
"effectcraft-pdf",
|
||||
|
||||
@@ -52,6 +52,7 @@ effectcraft-script = { path = "crates/script" }
|
||||
effectcraft-hevcenc = { path = "crates/hevcenc" }
|
||||
effectcraft-av1enc = { path = "crates/av1enc" }
|
||||
effectcraft-plugin = { path = "crates/plugin" }
|
||||
effectcraft-helper-host = { path = "crates/helper-host" }
|
||||
|
||||
# foundation
|
||||
thiserror = "2"
|
||||
|
||||
@@ -148,6 +148,24 @@ fn install_file_dialogs(app: &mut EffectcraftApp) -> Option<std::sync::mpsc::Rec
|
||||
}
|
||||
}
|
||||
|
||||
/// External effect helpers are native programs, so they start at launch only when the user opts
|
||||
/// in with `EFFECTCRAFT_HELPERS=1`: an environment variable, outside the settings folder, so
|
||||
/// writing there alone can't run code at launch. Otherwise Effect ▸ Scan for Effect Helpers
|
||||
/// starts them.
|
||||
fn start_helpers_if_opted_in(session: &mut effectcraft_engine::Session) {
|
||||
if session.helpers.is_none() || std::env::var_os("EFFECTCRAFT_HELPERS").is_none_or(|v| v != "1") {
|
||||
return;
|
||||
}
|
||||
match session.execute("effect.helpers.scan", json!({})) {
|
||||
Ok(r) => {
|
||||
for e in r["errors"].as_array().into_iter().flatten() {
|
||||
log::warn!("effect helpers: {}", e.as_str().unwrap_or("error"));
|
||||
}
|
||||
}
|
||||
Err(e) => log::warn!("effect helpers: {e}"),
|
||||
}
|
||||
}
|
||||
|
||||
/// Run a normal desktop launch: install logging and the panic hook, parse arguments (`--version` exits before the
|
||||
/// session log is attached), scan fonts, honor the launch guard's backend choice and open the eframe window.
|
||||
/// Builds the session, loads settings, plugins, the requested project/media and the optional control server, then
|
||||
@@ -262,6 +280,7 @@ fn main() -> eframe::Result {
|
||||
log::warn!("plug-in library: {e}");
|
||||
}
|
||||
}
|
||||
start_helpers_if_opted_in(&mut session);
|
||||
session.config = Some(std::sync::Arc::new(effectcraft_engine::config::DirConfig::new(dir)));
|
||||
}
|
||||
session.load_settings();
|
||||
@@ -503,6 +522,7 @@ impl eframe::App for Desktop {
|
||||
self.launch.exited();
|
||||
self.window.save(self.app.session.config.as_deref());
|
||||
self.app.on_exit();
|
||||
effectcraft_host::stop_helpers();
|
||||
// Scripts in Scripts/Shutdown run with the project closed.
|
||||
let _ = self.app.session.execute("file.runShutdownScripts", json!({}));
|
||||
}
|
||||
|
||||
@@ -11,7 +11,7 @@
|
||||
//! folder. In a headless session the footage and other files a project or an imported file names
|
||||
//! are read only inside the roots ([`ConfinedFootage`], [`ConfinedServices`]). Commands that run
|
||||
//! code or open other applications (scripts, ScriptUI panels and the events that reach them,
|
||||
//! Execute File, Reveal in Finder…) are refused while confined, as is granting scripts the file
|
||||
//! Execute File, effect helpers, Reveal in Finder…) are refused while confined, as is granting scripts the file
|
||||
//! and network access the user withheld.
|
||||
|
||||
use std::path::{Component, Path, PathBuf};
|
||||
@@ -56,6 +56,8 @@ const REFUSED: &[&str] = &[
|
||||
"scriptui.mouse",
|
||||
"scriptui.close",
|
||||
"file.executeFile",
|
||||
"effect.helpers.scan",
|
||||
"effect.helpers.load",
|
||||
"file.revealInFinder",
|
||||
"layer.openSource",
|
||||
"help.revealLogFile",
|
||||
|
||||
@@ -1482,6 +1482,9 @@ fn automation_roots_confine_file_access() {
|
||||
let steps = json!([{"command": "library.install", "params": {"id": "x"}}]);
|
||||
denied(&mut s, "batch", json!({"steps": steps}));
|
||||
call_json(&mut s, "execute_command", json!({"command": "library.list"}));
|
||||
// Effect helpers are native programs: starting them or loading through them is refused too.
|
||||
denied(&mut s, "execute_command", json!({"command": "effect.helpers.scan"}));
|
||||
denied(&mut s, "execute_command", json!({"command": "effect.helpers.load", "params": {"path": p(&read, "x.ofx")}}));
|
||||
for (id, params) in [
|
||||
("file.runStartupScripts", json!({})),
|
||||
("file.runShutdownScripts", json!({})),
|
||||
|
||||
@@ -544,6 +544,12 @@ pub struct EffectEnv<'a> {
|
||||
/// comp's and the layer's motion blur switches are on (Transform's Use Composition's
|
||||
/// Shutter Angle).
|
||||
pub shutter: Option<(f64, f64, u32)>,
|
||||
/// The layer's source time range in **layer time** seconds as (first, end): the layer time of
|
||||
/// the first source frame and the time just past the last one (so the source lasts
|
||||
/// `end - first` seconds and its last frame starts one frame before `end`). `None` when the
|
||||
/// source has no duration (stills, solids, text, shapes) or the renderer does not know it.
|
||||
/// Retimers clamp their source requests to it.
|
||||
pub layer_span: Option<(f64, f64)>,
|
||||
}
|
||||
|
||||
/// What an effect gets to render with.
|
||||
|
||||
+375
-39
@@ -13,14 +13,21 @@
|
||||
//!
|
||||
//! The contract (stable within an API version):
|
||||
//! * pixels are premultiplied RGBA `f32` (0–1, may exceed 1 in 32 bpc projects), row-major,
|
||||
//! in the layer's buffer, which may be scaled (preview resolution) and padded;
|
||||
//! row 0 on top, in the layer's buffer, which may be scaled (preview resolution) and padded;
|
||||
//! * parameters are flattened to `f64`s in declaration order: slider / angle / checkbox (0 or 1)
|
||||
//! / popup (0-based index) take one value, point two (**frame pixels**, already mapped to the
|
||||
//! buffer), colour four (RGBA 0–1);
|
||||
//! buffer), colour four (RGBA 0–1); API 2 adds point3 (x, y mapped like a point, z as is),
|
||||
//! layer (one value: the layer id, -1 for none), and text / hidden strings, which take no
|
||||
//! slot and are read with [`PluginParams::text`];
|
||||
//! * `time` is layer time in seconds; `scale` is the buffer's pixels per layer pixel (scale
|
||||
//! pixel distances like blur radii by it);
|
||||
//! * rendering must be deterministic (same inputs, same output) and may run on several threads
|
||||
//! at once.
|
||||
//!
|
||||
//! API 2 (native plug-ins; the WebAssembly ABI stays at 1) adds [`EffectPlugin::padding`] and
|
||||
//! [`EffectPlugin::render_v2`], which gets a [`PluginHost`] to read the effect's input or other
|
||||
//! layers at other times, and the extra parameter kinds. External effect helpers
|
||||
//! (`effectcraft-helper-host`) are built on it.
|
||||
|
||||
use std::sync::{Arc, OnceLock, RwLock};
|
||||
|
||||
@@ -28,13 +35,19 @@ use effectcraft_keyframe::Value;
|
||||
use effectcraft_project::ParamUi;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::{Buf, EffectCtx, EffectSpec, ParamSpec, RenderFn};
|
||||
use crate::{Buf, EffectCtx, EffectSpec, ParamSpec, Params, RenderFn};
|
||||
|
||||
/// The plug-in API version this build implements.
|
||||
pub const PLUGIN_API_VERSION: u32 = 1;
|
||||
/// The plug-in API version this build implements (manifests with API 1 or 2 are accepted).
|
||||
pub const PLUGIN_API_VERSION: u32 = 2;
|
||||
/// The plug-in API a WebAssembly module implements (its `ec_api_version`). API 2 is for native
|
||||
/// and helper plug-ins; WebAssembly modules stay at 1.
|
||||
pub const WASM_API_VERSION: u32 = 1;
|
||||
|
||||
/// The most plug-in effects one process can register.
|
||||
pub const MAX_PLUGINS: usize = 256;
|
||||
/// The most plug-in effects one process can register (commercial effect suites register a few hundred each).
|
||||
pub const MAX_PLUGINS: usize = 1024;
|
||||
|
||||
/// The most transparent pixels a plug-in may ask [`EffectPlugin::padding`] to add per side.
|
||||
const MAX_PADDING: u32 = 4096;
|
||||
|
||||
/// A plug-in parameter's kind, default and range.
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
@@ -53,6 +66,9 @@ pub enum PluginParamKind {
|
||||
slider_max: Option<f64>,
|
||||
#[serde(default = "two")]
|
||||
decimals: u8,
|
||||
/// Whole numbers (API 2; integer parameters): shown with 0 decimals.
|
||||
#[serde(default)]
|
||||
integer: bool,
|
||||
},
|
||||
Angle {
|
||||
#[serde(default)]
|
||||
@@ -78,6 +94,24 @@ pub enum PluginParamKind {
|
||||
#[serde(default = "white")]
|
||||
default: [f64; 4],
|
||||
},
|
||||
/// API 2: a 3D point (x, y in frame pixels like [`PluginParamKind::Point`], z as is). The
|
||||
/// default is in layer pixels (not fractions).
|
||||
Point3 {
|
||||
#[serde(default)]
|
||||
default: [f64; 3],
|
||||
},
|
||||
/// API 2: another layer of the composition (flattens to its id, -1 for none).
|
||||
Layer,
|
||||
/// API 2: a string, shown as a text field; read with [`PluginParams::text`].
|
||||
Text {
|
||||
#[serde(default)]
|
||||
default: String,
|
||||
},
|
||||
/// API 2: a string that is not shown (opaque plug-in data); read with [`PluginParams::text`].
|
||||
Hidden {
|
||||
#[serde(default)]
|
||||
default: String,
|
||||
},
|
||||
}
|
||||
|
||||
fn hundred() -> f64 {
|
||||
@@ -98,10 +132,21 @@ impl PluginParamKind {
|
||||
pub fn width(&self) -> usize {
|
||||
match self {
|
||||
PluginParamKind::Point { .. } => 2,
|
||||
PluginParamKind::Point3 { .. } => 3,
|
||||
PluginParamKind::Color { .. } => 4,
|
||||
PluginParamKind::Text { .. } | PluginParamKind::Hidden { .. } => 0,
|
||||
_ => 1,
|
||||
}
|
||||
}
|
||||
/// Whether the parameter holds a string (it flattens to no `f64`s but one entry of
|
||||
/// [`PluginParams::texts`]).
|
||||
pub fn is_text(&self) -> bool {
|
||||
matches!(self, PluginParamKind::Text { .. } | PluginParamKind::Hidden { .. })
|
||||
}
|
||||
/// Whether the kind needs plug-in API 2 (WebAssembly plug-ins can't carry it).
|
||||
pub fn needs_v2(&self) -> bool {
|
||||
matches!(self, PluginParamKind::Point3 { .. } | PluginParamKind::Layer | PluginParamKind::Text { .. } | PluginParamKind::Hidden { .. })
|
||||
}
|
||||
}
|
||||
|
||||
/// One parameter of a plug-in effect.
|
||||
@@ -110,6 +155,10 @@ pub struct PluginParam {
|
||||
/// Stable id (the property's match name; letters, digits, `_`).
|
||||
pub id: String,
|
||||
pub name: String,
|
||||
/// API 2: the group / page path the parameter belongs to (`Advanced/Edges`). Hosts may
|
||||
/// ignore it.
|
||||
#[serde(default)]
|
||||
pub group: String,
|
||||
#[serde(flatten)]
|
||||
pub kind: PluginParamKind,
|
||||
}
|
||||
@@ -117,7 +166,7 @@ pub struct PluginParam {
|
||||
/// What a plug-in declares about itself.
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub struct PluginManifest {
|
||||
/// The plug-in API version it was written for (must be [`PLUGIN_API_VERSION`]).
|
||||
/// The plug-in API version it was written for (1, or 2 up to [`PLUGIN_API_VERSION`]).
|
||||
pub api: u32,
|
||||
/// Stable, reverse-domain effect id (`org.example.posterize`); `ec.` is reserved.
|
||||
pub id: String,
|
||||
@@ -137,9 +186,12 @@ pub struct PluginManifest {
|
||||
impl PluginManifest {
|
||||
/// Check the manifest against this API version.
|
||||
pub fn validate(&self) -> Result<(), String> {
|
||||
const MAX_PARAMS: usize = 256;
|
||||
if self.api != PLUGIN_API_VERSION {
|
||||
return Err(format!("plug-in `{}` targets plug-in API {} but this build implements API {PLUGIN_API_VERSION}", self.id, self.api));
|
||||
// API 2 effects from large third-party suites can declare far more parameters.
|
||||
let max_params: usize = if self.api >= 2 { 1024 } else { 256 };
|
||||
// Opaque plug-in data can be large (some effects keep look-up tables of a few MB).
|
||||
const MAX_TEXT: usize = 64 << 20;
|
||||
if !(1..=PLUGIN_API_VERSION).contains(&self.api) {
|
||||
return Err(format!("plug-in `{}` targets plug-in API {} but this build implements API 1 to {PLUGIN_API_VERSION}", self.id, self.api));
|
||||
}
|
||||
let ok_id = |s: &str| !s.is_empty() && s.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-'));
|
||||
if !ok_id(&self.id) || self.id.starts_with("ec.") {
|
||||
@@ -148,8 +200,8 @@ impl PluginManifest {
|
||||
if self.name.trim().is_empty() || self.category.trim().is_empty() {
|
||||
return Err(format!("plug-in `{}` needs a name and a category", self.id));
|
||||
}
|
||||
if self.params.len() > MAX_PARAMS {
|
||||
return Err(format!("plug-in `{}` has {} parameters (at most {MAX_PARAMS})", self.id, self.params.len()));
|
||||
if self.params.len() > max_params {
|
||||
return Err(format!("plug-in `{}` has {} parameters (at most {max_params})", self.id, self.params.len()));
|
||||
}
|
||||
let mut seen = std::collections::HashSet::new();
|
||||
for p in &self.params {
|
||||
@@ -161,6 +213,16 @@ impl PluginManifest {
|
||||
{
|
||||
return Err(format!("plug-in `{}`: popup `{}` needs options and a default index in range", self.id, p.id));
|
||||
}
|
||||
if let PluginParamKind::Point3 { default } = &p.kind
|
||||
&& default.iter().any(|v| !v.is_finite())
|
||||
{
|
||||
return Err(format!("plug-in `{}`: point3 `{}` needs a finite default", self.id, p.id));
|
||||
}
|
||||
if let PluginParamKind::Text { default } | PluginParamKind::Hidden { default } = &p.kind
|
||||
&& default.len() > MAX_TEXT
|
||||
{
|
||||
return Err(format!("plug-in `{}`: the default of `{}` is longer than {MAX_TEXT} bytes", self.id, p.id));
|
||||
}
|
||||
if let PluginParamKind::Slider { default, min, max, slider_min, slider_max, .. } = &p.kind {
|
||||
let (lo, hi) = (slider_min.unwrap_or(*min), slider_max.unwrap_or(*max));
|
||||
if [*default, *min, *max, lo, hi].iter().any(|v| !v.is_finite()) || min > max || lo > hi || default < min || default > max {
|
||||
@@ -181,16 +243,78 @@ impl PluginManifest {
|
||||
pub struct PluginFrame<'a> {
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
/// Premultiplied RGBA, row-major, `width * height` pixels.
|
||||
/// Premultiplied RGBA, row-major (row 0 = top), `width * height` pixels.
|
||||
pub pixels: &'a mut [[f32; 4]],
|
||||
/// Buffer pixels per layer pixel (preview resolution).
|
||||
pub scale: f64,
|
||||
/// API 2: buffer-pixel position of layer pixel (0, 0) ([`Buf::offset`]; padding shifts it).
|
||||
pub origin: [f64; 2],
|
||||
/// API 2: the layer's bounds in layer pixels.
|
||||
pub layer_size: [f64; 2],
|
||||
}
|
||||
|
||||
/// An image the host hands an API 2 plug-in (the input at another time, or another layer).
|
||||
/// Same layout as [`PluginFrame`]: premultiplied RGBA, row 0 = top, `origin` = buffer pixels of
|
||||
/// layer pixel (0, 0) in this image.
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct PluginImage {
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
pub pixels: Vec<[f32; 4]>,
|
||||
pub origin: [f64; 2],
|
||||
pub scale: f64,
|
||||
}
|
||||
|
||||
impl PluginImage {
|
||||
fn from_buf(b: Buf) -> PluginImage {
|
||||
PluginImage { width: b.img.width, height: b.img.height, pixels: b.img.data, origin: b.offset, scale: b.scale }
|
||||
}
|
||||
}
|
||||
|
||||
/// Services available during an API 2 render ([`EffectPlugin::render_v2`]).
|
||||
pub trait PluginHost {
|
||||
/// This effect's input (its layer with masks and the effects above this one) at another
|
||||
/// **layer time** in seconds, in its own extent (see [`PluginImage::origin`]; it is not
|
||||
/// padded). `None` when unavailable (adjustment layers, no host).
|
||||
fn input_at(&self, layer_time: f64) -> Option<PluginImage>;
|
||||
/// The layer chosen in Layer parameter `param_id` at **layer time** `layer_time` (converted
|
||||
/// to composition time), effects and masks applied, resampled into this layer's buffer
|
||||
/// (same size, origin and scale as the frame, centred when the layers differ in size).
|
||||
/// `None` when unset or unavailable.
|
||||
fn layer_input(&self, param_id: &str, layer_time: f64) -> Option<PluginImage>;
|
||||
/// All flattened parameter values (and texts) at another layer time.
|
||||
fn params_at(&self, layer_time: f64) -> Option<(Vec<f64>, Vec<String>)>;
|
||||
/// Composition frame rate (30 when unknown).
|
||||
fn frame_rate(&self) -> f64;
|
||||
/// Layer time of the current render, seconds.
|
||||
fn time(&self) -> f64;
|
||||
/// A stable id of the applied effect instance: the same on every frame of one effect on one
|
||||
/// layer, different for every layer and every application of the effect. Plug-ins that keep
|
||||
/// state between frames pool their native instances by it. 0 when unknown.
|
||||
fn instance_key(&self) -> u64 {
|
||||
0
|
||||
}
|
||||
/// The layer's source time range in **layer time** seconds as (first, end): the first source
|
||||
/// frame starts at `first` and the source ends at `end` (see [`crate::EffectEnv::layer_span`]).
|
||||
/// `None` when the source has no duration or the host does not know it (retimers then
|
||||
/// assume an unbounded range).
|
||||
fn frame_range(&self) -> Option<(f64, f64)> {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// The [`PluginHost::instance_key`] of the effect `ctx` renders: its instance seed (stable per
|
||||
/// effect instance), offset so it is never 0.
|
||||
pub fn ctx_instance_key(ctx: &EffectCtx) -> u64 {
|
||||
(1u64 << 32) | u64::from(ctx.seed)
|
||||
}
|
||||
|
||||
/// Parameter values for one render, flattened (see the module docs) with typed accessors.
|
||||
pub struct PluginParams<'a> {
|
||||
pub manifest: &'a PluginManifest,
|
||||
pub flat: &'a [f64],
|
||||
/// One string per text / hidden parameter, in declaration order.
|
||||
pub texts: &'a [String],
|
||||
}
|
||||
|
||||
impl PluginParams<'_> {
|
||||
@@ -214,10 +338,31 @@ impl PluginParams<'_> {
|
||||
}
|
||||
/// Point in frame pixels.
|
||||
pub fn point(&self, id: &str) -> [f64; 2] {
|
||||
self.slot(id).map(|s| [s[0], s[1]]).unwrap_or([0.0; 2])
|
||||
self.slot(id).and_then(|s| Some([*s.first()?, *s.get(1)?])).unwrap_or([0.0; 2])
|
||||
}
|
||||
/// 3D point: x, y in frame pixels, z as is.
|
||||
pub fn point3(&self, id: &str) -> [f64; 3] {
|
||||
self.slot(id).and_then(|s| Some([*s.first()?, *s.get(1)?, *s.get(2)?])).unwrap_or([0.0; 3])
|
||||
}
|
||||
pub fn color(&self, id: &str) -> [f64; 4] {
|
||||
self.slot(id).map(|s| [s[0], s[1], s[2], s[3]]).unwrap_or([1.0; 4])
|
||||
self.slot(id).and_then(|s| Some([*s.first()?, *s.get(1)?, *s.get(2)?, *s.get(3)?])).unwrap_or([1.0; 4])
|
||||
}
|
||||
/// The layer chosen in a layer parameter.
|
||||
pub fn layer(&self, id: &str) -> Option<u64> {
|
||||
self.slot(id).and_then(|s| s.first().copied()).filter(|v| *v >= 0.0).map(|v| v as u64)
|
||||
}
|
||||
/// A text / hidden parameter's string (empty for other ids).
|
||||
pub fn text(&self, id: &str) -> &str {
|
||||
let mut at = 0;
|
||||
for p in &self.manifest.params {
|
||||
if p.kind.is_text() {
|
||||
if p.id == id {
|
||||
return self.texts.get(at).map(String::as_str).unwrap_or("");
|
||||
}
|
||||
at += 1;
|
||||
}
|
||||
}
|
||||
""
|
||||
}
|
||||
}
|
||||
|
||||
@@ -231,6 +376,18 @@ pub trait EffectPlugin: Send + Sync {
|
||||
fn source(&self) -> String {
|
||||
"built-in".into()
|
||||
}
|
||||
/// API 2: transparent pixels to add on every side of the buffer before rendering (glows,
|
||||
/// blurs, regions of definition larger than the input), in buffer pixels. `layer_size` is
|
||||
/// the layer's size in layer pixels (the frame's [`PluginFrame::layer_size`]; 0 when unknown)
|
||||
/// and `instance_key` the [`PluginHost::instance_key`] this render will have.
|
||||
fn padding(&self, _params: &PluginParams, _time: f64, _scale: f64, _layer_size: [f64; 2], _instance_key: u64) -> u32 {
|
||||
0
|
||||
}
|
||||
/// API 2: [`EffectPlugin::render`] with host services. The default calls `render`. May be
|
||||
/// called from several threads at once.
|
||||
fn render_v2(&self, frame: &mut PluginFrame, params: &PluginParams, time: f64, _host: &dyn PluginHost) -> Result<(), String> {
|
||||
self.render(frame, params, time)
|
||||
}
|
||||
}
|
||||
|
||||
static SLOTS: [OnceLock<Arc<dyn EffectPlugin>>; MAX_PLUGINS] = [const { OnceLock::new() }; MAX_PLUGINS];
|
||||
@@ -246,6 +403,30 @@ pub fn plugins() -> Vec<&'static EffectSpec> {
|
||||
plugin_specs().read().map(|v| v.clone()).unwrap_or_default()
|
||||
}
|
||||
|
||||
/// The last render error of each plug-in effect (cleared by its next good render), so the UI can
|
||||
/// show it on the effect.
|
||||
fn render_errors() -> &'static RwLock<std::collections::HashMap<String, String>> {
|
||||
static E: OnceLock<RwLock<std::collections::HashMap<String, String>>> = OnceLock::new();
|
||||
E.get_or_init(Default::default)
|
||||
}
|
||||
|
||||
/// Why the last render of plug-in effect `id` failed, if it did (the frame was left as it was).
|
||||
pub fn plugin_error(id: &str) -> Option<String> {
|
||||
render_errors().read().ok()?.get(id).cloned()
|
||||
}
|
||||
|
||||
fn set_plugin_error(id: &str, error: Option<String>) {
|
||||
let Ok(mut m) = render_errors().write() else { return };
|
||||
match error {
|
||||
Some(e) => {
|
||||
m.insert(id.to_string(), e);
|
||||
}
|
||||
None => {
|
||||
m.remove(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The plug-in behind a registered effect id.
|
||||
pub fn plugin(id: &str) -> Option<Arc<dyn EffectPlugin>> {
|
||||
SLOTS.iter().filter_map(|s| s.get()).find(|p| p.manifest().id == id).cloned()
|
||||
@@ -257,15 +438,25 @@ fn leak(s: &str) -> &'static str {
|
||||
|
||||
fn param_spec(p: &PluginParam) -> ParamSpec {
|
||||
let (default, ui) = match &p.kind {
|
||||
PluginParamKind::Slider { default, min, max, slider_min, slider_max, decimals } => (
|
||||
PluginParamKind::Slider { default, min, max, slider_min, slider_max, decimals, integer } => (
|
||||
Value::Scalar(*default),
|
||||
ParamUi::Slider { min: *min, max: *max, slider_min: slider_min.unwrap_or(*min), slider_max: slider_max.unwrap_or(*max), decimals: *decimals },
|
||||
ParamUi::Slider {
|
||||
min: *min,
|
||||
max: *max,
|
||||
slider_min: slider_min.unwrap_or(*min),
|
||||
slider_max: slider_max.unwrap_or(*max),
|
||||
decimals: if *integer { 0 } else { *decimals },
|
||||
},
|
||||
),
|
||||
PluginParamKind::Angle { default } => (Value::Scalar(*default), ParamUi::Angle),
|
||||
PluginParamKind::Checkbox { default } => (Value::Bool(*default), ParamUi::Checkbox),
|
||||
PluginParamKind::Popup { options, default } => (Value::Enum(*default), ParamUi::Popup { options: options.clone() }),
|
||||
PluginParamKind::Point { default } => (Value::Vec2(*default), ParamUi::Point),
|
||||
PluginParamKind::Color { default } => (Value::Color(*default), ParamUi::Color),
|
||||
PluginParamKind::Point3 { default } => (Value::Vec3(*default), ParamUi::Point3),
|
||||
PluginParamKind::Layer => (Value::Layer(None), ParamUi::Layer),
|
||||
PluginParamKind::Text { default } => (Value::Str(default.clone()), ParamUi::Text),
|
||||
PluginParamKind::Hidden { default } => (Value::Str(default.clone()), ParamUi::Hidden),
|
||||
};
|
||||
ParamSpec { id: leak(&p.id), name: leak(&p.name), default, ui }
|
||||
}
|
||||
@@ -303,55 +494,200 @@ fn trampoline<const I: usize>(ctx: &EffectCtx, buf: Buf) -> Buf {
|
||||
run_slot(I, ctx, buf)
|
||||
}
|
||||
|
||||
macro_rules! trampolines {
|
||||
($($i:literal)*) => { [$(trampoline::<$i> as RenderFn),*] };
|
||||
/// Concatenate `A` runs of `B` render functions (`N` = `A * B`), for the table below.
|
||||
const fn concat<const A: usize, const B: usize, const N: usize>(parts: [[RenderFn; B]; A]) -> [RenderFn; N] {
|
||||
let mut out = [trampoline::<0> as RenderFn; N];
|
||||
let mut i = 0;
|
||||
while i < N {
|
||||
out[i] = parts[i / B][i % B];
|
||||
i += 1;
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
const TRAMPOLINES: [RenderFn; MAX_PLUGINS] = trampolines!(
|
||||
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
|
||||
32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
|
||||
64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95
|
||||
96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127
|
||||
128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159
|
||||
160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191
|
||||
192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223
|
||||
224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255
|
||||
);
|
||||
/// 16 trampolines `base ..= base + 15`.
|
||||
macro_rules! t16 {
|
||||
($b:expr) => {
|
||||
[
|
||||
trampoline::<{ $b }> as RenderFn,
|
||||
trampoline::<{ $b + 1 }> as RenderFn,
|
||||
trampoline::<{ $b + 2 }> as RenderFn,
|
||||
trampoline::<{ $b + 3 }> as RenderFn,
|
||||
trampoline::<{ $b + 4 }> as RenderFn,
|
||||
trampoline::<{ $b + 5 }> as RenderFn,
|
||||
trampoline::<{ $b + 6 }> as RenderFn,
|
||||
trampoline::<{ $b + 7 }> as RenderFn,
|
||||
trampoline::<{ $b + 8 }> as RenderFn,
|
||||
trampoline::<{ $b + 9 }> as RenderFn,
|
||||
trampoline::<{ $b + 10 }> as RenderFn,
|
||||
trampoline::<{ $b + 11 }> as RenderFn,
|
||||
trampoline::<{ $b + 12 }> as RenderFn,
|
||||
trampoline::<{ $b + 13 }> as RenderFn,
|
||||
trampoline::<{ $b + 14 }> as RenderFn,
|
||||
trampoline::<{ $b + 15 }> as RenderFn,
|
||||
]
|
||||
};
|
||||
}
|
||||
|
||||
/// Four runs of `$inner` (each `$n` long, starting `$n` apart) from `$b`: `$total` trampolines.
|
||||
macro_rules! x4 {
|
||||
($inner:ident, $b:expr, $n:literal, $total:literal) => {
|
||||
concat::<4, $n, $total>([$inner!($b), $inner!($b + $n), $inner!($b + 2 * $n), $inner!($b + 3 * $n)])
|
||||
};
|
||||
}
|
||||
macro_rules! t64 {
|
||||
($b:expr) => {
|
||||
x4!(t16, $b, 16, 64)
|
||||
};
|
||||
}
|
||||
macro_rules! t256 {
|
||||
($b:expr) => {
|
||||
x4!(t64, $b, 64, 256)
|
||||
};
|
||||
}
|
||||
macro_rules! t1024 {
|
||||
($b:expr) => {
|
||||
x4!(t256, $b, 256, 1024)
|
||||
};
|
||||
}
|
||||
|
||||
const TRAMPOLINES: [RenderFn; MAX_PLUGINS] = t1024!(0);
|
||||
|
||||
/// Flatten an instance's evaluated parameters for the plug-in (points mapped to frame pixels).
|
||||
pub fn flatten(m: &PluginManifest, ctx: &EffectCtx, buf: &Buf) -> Vec<f64> {
|
||||
flatten_params(m, ctx.params, buf.offset, buf.scale)
|
||||
}
|
||||
|
||||
/// [`flatten`] for any evaluated parameters, with points mapped by buffer `offset` and `scale`.
|
||||
pub fn flatten_params(m: &PluginManifest, params: &Params, offset: [f64; 2], scale: f64) -> Vec<f64> {
|
||||
let to_px = |q: [f64; 2]| (q[0] * scale + offset[0], q[1] * scale + offset[1]);
|
||||
let mut out = Vec::with_capacity(m.flat_len());
|
||||
for p in &m.params {
|
||||
let v = ctx.params.get(&p.id);
|
||||
let v = params.get(&p.id);
|
||||
match &p.kind {
|
||||
PluginParamKind::Point { .. } => {
|
||||
let q = v.map(Value::as_vec2).unwrap_or([0.0; 2]);
|
||||
let (x, y) = buf.to_px(q);
|
||||
let (x, y) = to_px(v.map(Value::as_vec2).unwrap_or([0.0; 2]));
|
||||
out.extend([x, y]);
|
||||
}
|
||||
PluginParamKind::Point3 { default } => {
|
||||
let q = v.map(Value::as_vec3).unwrap_or(*default);
|
||||
let (x, y) = to_px([q[0], q[1]]);
|
||||
out.extend([x, y, q[2]]);
|
||||
}
|
||||
PluginParamKind::Color { .. } => {
|
||||
let c = v.map(Value::as_color).unwrap_or([1.0; 4]);
|
||||
out.extend(c.iter().map(|x| *x as f64));
|
||||
}
|
||||
PluginParamKind::Checkbox { .. } => out.push(if v.map(Value::as_bool).unwrap_or(false) { 1.0 } else { 0.0 }),
|
||||
PluginParamKind::Popup { .. } => out.push(v.map(Value::as_enum).unwrap_or(0) as f64),
|
||||
PluginParamKind::Layer => out.push(v.and_then(Value::as_layer).map(|l| l as f64).unwrap_or(-1.0)),
|
||||
PluginParamKind::Text { .. } | PluginParamKind::Hidden { .. } => {}
|
||||
_ => out.push(v.map(Value::as_f64).unwrap_or(0.0)),
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// The strings of an instance's text / hidden parameters, in declaration order.
|
||||
pub fn flatten_texts(m: &PluginManifest, params: &Params) -> Vec<String> {
|
||||
m.params
|
||||
.iter()
|
||||
.filter_map(|p| match &p.kind {
|
||||
PluginParamKind::Text { default } | PluginParamKind::Hidden { default } => Some(match params.get(&p.id) {
|
||||
Some(Value::Str(s)) => s.clone(),
|
||||
_ => default.clone(),
|
||||
}),
|
||||
_ => None,
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The host services of one render, on top of the effect context and the (padded) buffer.
|
||||
struct RenderHost<'a, 'b> {
|
||||
ctx: &'a EffectCtx<'b>,
|
||||
manifest: &'a PluginManifest,
|
||||
width: u32,
|
||||
height: u32,
|
||||
offset: [f64; 2],
|
||||
scale: f64,
|
||||
}
|
||||
|
||||
impl RenderHost<'_, '_> {
|
||||
/// The frame's pixel grid without pixels (what [`crate::util::fit_layer`] measures).
|
||||
fn grid(&self) -> Buf {
|
||||
Buf { img: crate::Image { width: self.width, height: self.height, data: vec![] }, offset: self.offset, scale: self.scale }
|
||||
}
|
||||
}
|
||||
|
||||
impl PluginHost for RenderHost<'_, '_> {
|
||||
fn input_at(&self, layer_time: f64) -> Option<PluginImage> {
|
||||
let host = self.ctx.env.host?;
|
||||
host.self_at(layer_time, self.ctx.env.effect_index).map(PluginImage::from_buf)
|
||||
}
|
||||
|
||||
fn layer_input(&self, param_id: &str, layer_time: f64) -> Option<PluginImage> {
|
||||
let ctx = self.ctx;
|
||||
let lid = ctx.params.get(param_id)?.as_layer()?;
|
||||
let other = if (layer_time - ctx.time).abs() < 1e-9 {
|
||||
ctx.layer_param(param_id, true)?
|
||||
} else {
|
||||
let host = ctx.env.host?;
|
||||
// Source / Masks show no effects; the default (Effects & Masks) does.
|
||||
let with_effects = !matches!(ctx.params.get(&crate::layer_source_id(param_id)).map(Value::as_enum), Some(0 | 1));
|
||||
host.layer_at(lid, ctx.env.comp_time + (layer_time - ctx.time), with_effects)?
|
||||
};
|
||||
let img = crate::util::fit_layer(ctx, &self.grid(), &other, false);
|
||||
Some(PluginImage { width: img.width, height: img.height, pixels: img.data, origin: self.offset, scale: self.scale })
|
||||
}
|
||||
|
||||
fn params_at(&self, layer_time: f64) -> Option<(Vec<f64>, Vec<String>)> {
|
||||
let p = self.ctx.env.host?.params_at(layer_time)?;
|
||||
Some((flatten_params(self.manifest, &p, self.offset, self.scale), flatten_texts(self.manifest, &p)))
|
||||
}
|
||||
|
||||
fn frame_rate(&self) -> f64 {
|
||||
self.ctx.fps()
|
||||
}
|
||||
|
||||
fn time(&self) -> f64 {
|
||||
self.ctx.time
|
||||
}
|
||||
|
||||
fn instance_key(&self) -> u64 {
|
||||
ctx_instance_key(self.ctx)
|
||||
}
|
||||
|
||||
fn frame_range(&self) -> Option<(f64, f64)> {
|
||||
self.ctx.env.layer_span
|
||||
}
|
||||
}
|
||||
|
||||
fn run_slot(slot: usize, ctx: &EffectCtx, mut buf: Buf) -> Buf {
|
||||
let Some(plugin) = SLOTS[slot].get() else { return buf };
|
||||
let m = plugin.manifest();
|
||||
let texts = flatten_texts(m, ctx.params);
|
||||
let flat = flatten(m, ctx, &buf);
|
||||
let params = PluginParams { manifest: m, flat: &flat };
|
||||
let (width, height) = (buf.img.width, buf.img.height);
|
||||
let before = buf.img.data.clone();
|
||||
let mut frame = PluginFrame { width, height, pixels: &mut buf.img.data, scale: buf.scale };
|
||||
if let Err(e) = plugin.render(&mut frame, ¶ms, ctx.time) {
|
||||
let key = ctx_instance_key(ctx);
|
||||
let pad = plugin.padding(&PluginParams { manifest: m, flat: &flat, texts: &texts }, ctx.time, buf.scale, ctx.layer_size, key).min(MAX_PADDING);
|
||||
let before = buf.clone();
|
||||
// Padding moves the layer origin, which points are measured from.
|
||||
let flat = if pad > 0 {
|
||||
buf.pad(pad);
|
||||
flatten(m, ctx, &buf)
|
||||
} else {
|
||||
flat
|
||||
};
|
||||
let params = PluginParams { manifest: m, flat: &flat, texts: &texts };
|
||||
let (width, height, offset, scale) = (buf.img.width, buf.img.height, buf.offset, buf.scale);
|
||||
let host = RenderHost { ctx, manifest: m, width, height, offset, scale };
|
||||
let mut frame = PluginFrame { width, height, pixels: &mut buf.img.data, scale, origin: offset, layer_size: ctx.layer_size };
|
||||
if let Err(e) = plugin.render_v2(&mut frame, ¶ms, ctx.time, &host) {
|
||||
log::warn!("plug-in `{}`: {e}", m.id);
|
||||
buf.img.data = before;
|
||||
set_plugin_error(&m.id, Some(e));
|
||||
return before;
|
||||
}
|
||||
if plugin_error(&m.id).is_some() {
|
||||
set_plugin_error(&m.id, None);
|
||||
}
|
||||
buf
|
||||
}
|
||||
@@ -24,11 +24,12 @@ fn plugins_list(s: &mut Session, _: &Value) -> Result<Value> {
|
||||
"author": m.as_ref().map(|m| m.author.clone()),
|
||||
"description": m.as_ref().map(|m| m.description.clone()),
|
||||
"source": p.map(|p| p.source()),
|
||||
"error": plugin::plugin_error(spec.id),
|
||||
"params": spec.params.iter().map(|p| p.id).collect::<Vec<_>>(),
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
Ok(json!({"api": plugin::PLUGIN_API_VERSION, "wasm": s.plugin_loader.is_some(), "plugins": list}))
|
||||
Ok(json!({"api": plugin::PLUGIN_API_VERSION, "wasm": s.plugin_loader.is_some(), "helpers": s.helpers.is_some(), "plugins": list}))
|
||||
}
|
||||
|
||||
/// `effect.plugins.load`: load WebAssembly effect plug-ins (a file, or every `.wasm` in a
|
||||
@@ -70,6 +71,58 @@ fn plugins_load(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
Ok(json!({"loaded": loaded, "errors": errors}))
|
||||
}
|
||||
|
||||
/// External effect helpers need a host that runs them.
|
||||
fn has_helpers(s: &Session) -> std::result::Result<(), String> {
|
||||
if s.helpers.is_some() { Ok(()) } else { Err("effect helpers are not available in this build".into()) }
|
||||
}
|
||||
|
||||
/// The host's helper hooks, for the commands that start helpers. Helpers run native code, so a
|
||||
/// script may start them only with Allow Scripts to Write Files and Access Network on (as for
|
||||
/// File.execute()).
|
||||
fn helper_hooks(s: &Session, c: &str) -> Result<crate::HelperHooks> {
|
||||
if s.running_for_script() && !s.prefs.scripting.allow_scripts_write_files {
|
||||
return Err(bad(
|
||||
c,
|
||||
"scripts can't start effect helpers unless Settings ▸ Scripting & Expressions ▸ Allow Scripts to Write Files and Access Network is on",
|
||||
));
|
||||
}
|
||||
s.helpers.ok_or_else(|| EngineError::Other("effect helpers are not available in this build".into()))
|
||||
}
|
||||
|
||||
/// `effect.helpers.list`: the external effect helpers and their state.
|
||||
fn helpers_list(s: &mut Session, _: &Value) -> Result<Value> {
|
||||
Ok(match s.helpers {
|
||||
Some(h) => (h.list)(),
|
||||
None => json!({"helpers": [], "blocked": [], "available": false}),
|
||||
})
|
||||
}
|
||||
|
||||
/// `effect.helpers.scan`: start the installed helpers that aren't running and register their
|
||||
/// effects. One bad helper doesn't stop the rest; `retry: true` turns blocked effects back on.
|
||||
fn helpers_scan(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let h = helper_hooks(s, "effect.helpers.scan")?;
|
||||
let r = (h.scan)(b_p(p, "retry").unwrap_or(false));
|
||||
let n = r["loaded"].as_array().map_or(0, Vec::len);
|
||||
if n > 0 {
|
||||
s.toast(format!("{n} effect(s) from effect helpers"));
|
||||
}
|
||||
Ok(r)
|
||||
}
|
||||
|
||||
/// `effect.helpers.load`: load a file or folder through the helpers that accept it (an OpenFX
|
||||
/// helper takes `.ofx` plug-ins, for example).
|
||||
fn helpers_load(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let c = "effect.helpers.load";
|
||||
let h = helper_hooks(s, c)?;
|
||||
let path = str_p(p, "path").ok_or_else(|| bad(c, "missing `path`"))?;
|
||||
let r = (h.load)(path).map_err(|e| EngineError::Other(format!("{path}: {e}")))?;
|
||||
let n = r["loaded"].as_array().map_or(0, Vec::len);
|
||||
if n > 0 {
|
||||
s.toast(format!("Loaded {n} effect(s) through a helper"));
|
||||
}
|
||||
Ok(r)
|
||||
}
|
||||
|
||||
fn apply(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let name = str_p(p, "effect").ok_or_else(|| bad("effect.apply", "missing `effect` (id or name)"))?;
|
||||
let spec = effectcraft_effects::lookup(name).ok_or_else(|| bad("effect.apply", format!("unknown effect `{name}`")))?;
|
||||
@@ -1140,7 +1193,7 @@ pub fn specs() -> Vec<CommandSpec> {
|
||||
crate::query!(
|
||||
"effect.plugins.list",
|
||||
"List Effect Plug-ins",
|
||||
"{} → {api, wasm, plugins: [{id, name, category, version, author, source, params}]}",
|
||||
"{} → {api, wasm, helpers, plugins: [{id, name, category, version, author, source, params, error}]}",
|
||||
plugins_list
|
||||
),
|
||||
cmd!(
|
||||
@@ -1152,6 +1205,30 @@ pub fn specs() -> Vec<CommandSpec> {
|
||||
super::always,
|
||||
plugins_load
|
||||
),
|
||||
crate::query!(
|
||||
"effect.helpers.list",
|
||||
"List Effect Helpers",
|
||||
"{} → {helpers: [{program, name, version, loads, running, effects, restarts, error}], blocked: [effect ids]}",
|
||||
helpers_list
|
||||
),
|
||||
cmd!(
|
||||
"effect.helpers.scan",
|
||||
"Scan for Effect Helpers",
|
||||
[],
|
||||
None,
|
||||
"{retry?} → {paths, helpers, loaded, errors}: starts the effect helpers installed in helpers/ next to the app, helpers/ in the settings folder and EFFECTCRAFT_HELPER_PATH, and adds their effects; retry turns effects that kept stopping their helper back on",
|
||||
has_helpers,
|
||||
helpers_scan
|
||||
),
|
||||
cmd!(
|
||||
"effect.helpers.load",
|
||||
"Load Plug-ins Through Helper...",
|
||||
[],
|
||||
None,
|
||||
"{path (a file or folder an installed helper accepts, e.g. a folder of .ofx plug-ins for an OpenFX helper; the menu asks for a folder)} → {loaded: [{id, name, category, helper}], errors}",
|
||||
has_helpers,
|
||||
helpers_load
|
||||
),
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
@@ -200,6 +200,9 @@ pub struct Session {
|
||||
/// Loads WebAssembly effect plug-ins (set by the host that links `effectcraft-plugin` with
|
||||
/// its runtime): `effect.plugins.load`.
|
||||
pub plugin_loader: Option<PluginLoader>,
|
||||
/// External effect helpers (native hosts link `effectcraft-helper-host`):
|
||||
/// `effect.helpers.list`, `effect.helpers.scan`, `effect.helpers.load`.
|
||||
pub helpers: Option<HelperHooks>,
|
||||
/// Renders a module without registering it (`library.preview`; set with `plugin_loader`).
|
||||
pub plugin_previewer: Option<PluginPreviewer>,
|
||||
/// Where effect plug-ins live (hosts set it; else `Plug-ins` next to the settings): the
|
||||
@@ -283,6 +286,18 @@ pub type ScriptRunner = fn(&mut Session, &ScriptRequest) -> Value;
|
||||
/// describes it (`{id, name, category, version, api, params}`).
|
||||
pub type PluginLoader = fn(&[u8], &str) -> std::result::Result<Value, String>;
|
||||
|
||||
/// What a host that runs external effect helpers provides (`docs/external-helpers.md`).
|
||||
#[derive(Clone, Copy)]
|
||||
pub struct HelperHooks {
|
||||
/// Start the helpers installed in the host's search paths and register their effects
|
||||
/// (`retry` forgets the block list first) → `{paths, helpers, loaded, errors}`.
|
||||
pub scan: fn(bool) -> Value,
|
||||
/// Load a file or folder through the helpers that accept it → `{loaded, errors}`.
|
||||
pub load: fn(&str) -> std::result::Result<Value, String>,
|
||||
/// The helpers and their state → `{helpers, blocked}`.
|
||||
pub list: fn() -> Value,
|
||||
}
|
||||
|
||||
/// Renders one WebAssembly effect module without registering it (the Plug-in Library preview):
|
||||
/// `(bytes, width, height, premultiplied RGBA pixels, times in seconds)` → (manifest JSON, one frame
|
||||
/// per time).
|
||||
@@ -343,6 +358,7 @@ impl Default for Session {
|
||||
snapshot: None,
|
||||
script: None,
|
||||
plugin_loader: None,
|
||||
helpers: None,
|
||||
plugin_previewer: None,
|
||||
plugins_dir: None,
|
||||
library: Default::default(),
|
||||
|
||||
@@ -1015,6 +1015,8 @@ Effect
|
||||
---
|
||||
Manage Effects... | effect.manage
|
||||
Load Effect Plug-in... | effect.plugins.load
|
||||
Load Plug-ins Through Helper... | effect.helpers.load
|
||||
Scan for Effect Helpers | effect.helpers.scan
|
||||
Plug-in Library... | prefs.open {"page":"library"}
|
||||
---
|
||||
@effects
|
||||
|
||||
@@ -299,11 +299,12 @@ pub fn parse_catalog(bytes: &[u8]) -> std::result::Result<(Catalog, Vec<String>)
|
||||
Ok((Catalog { schema: CATALOG_SCHEMA, packages }, warnings))
|
||||
}
|
||||
|
||||
/// This build implements the release's plug-in API and is at least its `minApp`.
|
||||
/// This build loads the release's WebAssembly plug-in API ([`WASM_API_VERSION`](crate::effects::plugin::WASM_API_VERSION):
|
||||
/// the library lists `.wasm` modules only) and is at least its `minApp`.
|
||||
fn runs_here(r: &Release) -> bool {
|
||||
// Without a pre-release suffix (`0.18.0-dev` is 0.18.0).
|
||||
let app = env!("CARGO_PKG_VERSION").split(['-', '+']).next().unwrap_or_default();
|
||||
r.api == crate::effects::plugin::PLUGIN_API_VERSION && r.min_app.as_deref().is_none_or(|m| !newer(m, app))
|
||||
r.api == crate::effects::plugin::WASM_API_VERSION && r.min_app.as_deref().is_none_or(|m| !newer(m, app))
|
||||
}
|
||||
|
||||
/// The newest release this build can run (plug-in API and minimum app version).
|
||||
@@ -788,13 +789,13 @@ fn pick(s: &Session, c: &str, p: &Value) -> Result<(String, Package, Release)> {
|
||||
return Err(bad(
|
||||
c,
|
||||
format!(
|
||||
"{} {} needs plug-in API {} and EffectCraft {} or later; this is EffectCraft {} with plug-in API {}",
|
||||
"{} {} needs plug-in API {} and EffectCraft {} or later; this is EffectCraft {} with WebAssembly plug-in API {}",
|
||||
pkg.name,
|
||||
rel.version,
|
||||
rel.api,
|
||||
rel.min_app.as_deref().unwrap_or("(any)"),
|
||||
env!("CARGO_PKG_VERSION"),
|
||||
crate::effects::plugin::PLUGIN_API_VERSION
|
||||
crate::effects::plugin::WASM_API_VERSION
|
||||
),
|
||||
));
|
||||
}
|
||||
@@ -1111,7 +1112,7 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn the_newest_compatible_release_wins() {
|
||||
let api = crate::effects::plugin::PLUGIN_API_VERSION;
|
||||
let api = crate::effects::plugin::WASM_API_VERSION;
|
||||
let p = Package {
|
||||
versions: vec![
|
||||
rel("1.0.0", api, None),
|
||||
@@ -1119,10 +1120,13 @@ mod tests {
|
||||
rel("1.1.0", api, None),
|
||||
rel("9.0.0", api, Some("999.0")),
|
||||
rel("3.0.0", api + 1, None),
|
||||
rel("4.0.0", 0, None),
|
||||
],
|
||||
..Default::default()
|
||||
};
|
||||
assert_eq!(latest_compatible(&p).map(|r| r.version.as_str()), Some("1.2.0"));
|
||||
assert!(runs_here(&rel("1.0.0", 1, None)), "API 1 modules still run on a build with a newer plug-in API");
|
||||
assert!(!runs_here(&rel("1.0.0", 2, None)), "WebAssembly modules are API 1: an API 2 release can't load");
|
||||
let j = package_json(&p, None, &Status::default());
|
||||
assert_eq!(j["needsNewerApp"], true);
|
||||
assert!(latest_compatible(&Package::default()).is_none());
|
||||
|
||||
@@ -941,3 +941,25 @@ fn copied_effect_property_pastes_onto_the_same_effect() {
|
||||
assert_eq!(r["expressions"], 1, "{r}");
|
||||
assert_eq!(layer(&s, b).props.prop("effects/#1/blurriness").unwrap().expr.as_ref().unwrap().text, "time * 10");
|
||||
}
|
||||
|
||||
/// Effect helpers run native code: a script starts them only with Allow Scripts to Write Files
|
||||
/// and Access Network on (as for File.execute()); the user always can.
|
||||
#[test]
|
||||
fn scripts_start_effect_helpers_only_with_file_access() {
|
||||
let mut s = Session {
|
||||
helpers: Some(crate::HelperHooks {
|
||||
scan: |_| json!({"paths": [], "helpers": [], "loaded": [], "errors": []}),
|
||||
load: |_| Ok(json!({"loaded": [], "errors": []})),
|
||||
list: || json!({"helpers": [], "blocked": []}),
|
||||
}),
|
||||
..Default::default()
|
||||
};
|
||||
for (id, params) in [("effect.helpers.scan", json!({})), ("effect.helpers.load", json!({"path": "/plug-ins/x.ofx"}))] {
|
||||
let e = s.execute_for_script(id, params.clone()).unwrap_err().to_string();
|
||||
assert!(e.contains("Allow Scripts to Write Files"), "{id}: {e}");
|
||||
assert!(s.execute(id, params).is_ok(), "{id}: the user starts helpers");
|
||||
}
|
||||
assert!(s.execute_for_script("effect.helpers.list", json!({})).is_ok(), "listing starts nothing");
|
||||
s.execute("prefs.set", json!({"key": "scripting.allowScriptsWriteFiles", "value": true})).unwrap();
|
||||
assert!(s.execute_for_script("effect.helpers.scan", json!({})).is_ok());
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
[package]
|
||||
name = "effectcraft-helper-host"
|
||||
description = "External effect helpers: finds helper programs, talks to them over a versioned pipe + shared-file protocol and registers their effects (plug-in API 2). Safe Rust; a crashing helper is an error, not a crash"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
license.workspace = true
|
||||
rust-version.workspace = true
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
|
||||
[dependencies]
|
||||
effectcraft-effects = { workspace = true }
|
||||
serde = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
log = { workspace = true }
|
||||
bytemuck = { workspace = true }
|
||||
|
||||
# The crate's integration tests (one binary, `tests/it/main.rs`): the whole protocol against a
|
||||
# fake helper. The test binary is also the helper (started with `--effectcraft-helper`), so
|
||||
# `cargo test` covers it on every platform with nothing to install; `harness = false` because
|
||||
# libtest would reject the helper argument.
|
||||
[[test]]
|
||||
name = "it"
|
||||
path = "tests/it/main.rs"
|
||||
harness = false
|
||||
@@ -0,0 +1,135 @@
|
||||
//! Effects that keep taking their helper down are turned off.
|
||||
//!
|
||||
//! Each time a render through a helper ends with the helper crashing or hanging, the effect is
|
||||
//! charged; at [`CRASH_LIMIT`] it is blocked: its renders fail at once with a message instead of
|
||||
//! restarting the helper again. Blocks are kept in `helper_blocklist.txt` in the state folder
|
||||
//! ([`set_state_dir`]), one effect id per line, so they last across launches until a scan with
|
||||
//! `retry` ([`clear`]).
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::io::Read;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::{Mutex, OnceLock};
|
||||
|
||||
use crate::client::lock;
|
||||
|
||||
/// Crashes after which an effect is blocked.
|
||||
pub const CRASH_LIMIT: u32 = 3;
|
||||
|
||||
const FILE: &str = "helper_blocklist.txt";
|
||||
/// The most of the block list file read (tens of thousands of effect ids).
|
||||
const MAX_FILE: u64 = 1 << 20;
|
||||
|
||||
#[derive(Default)]
|
||||
struct State {
|
||||
dir: Option<PathBuf>,
|
||||
crashes: HashMap<String, u32>,
|
||||
blocked: Vec<String>,
|
||||
loaded: bool,
|
||||
}
|
||||
|
||||
fn state() -> &'static Mutex<State> {
|
||||
static S: OnceLock<Mutex<State>> = OnceLock::new();
|
||||
S.get_or_init(Default::default)
|
||||
}
|
||||
|
||||
/// Where the block list lives (EffectCraft's settings folder); helpers get it as
|
||||
/// `EFFECTCRAFT_HELPER_STATE_DIR` for their own records.
|
||||
pub fn set_state_dir(dir: PathBuf) {
|
||||
let mut s = lock(state());
|
||||
s.dir = Some(dir);
|
||||
s.loaded = false;
|
||||
}
|
||||
|
||||
pub fn state_dir() -> Option<PathBuf> {
|
||||
lock(state()).dir.clone()
|
||||
}
|
||||
|
||||
fn load(s: &mut State) {
|
||||
if s.loaded {
|
||||
return;
|
||||
}
|
||||
s.loaded = true;
|
||||
if let Some(text) = s.dir.as_ref().and_then(|d| read_capped(&d.join(FILE))) {
|
||||
for id in text.lines().map(str::trim).filter(|l| !l.is_empty() && !l.starts_with('#')) {
|
||||
if !s.blocked.iter().any(|b| b == id) {
|
||||
s.blocked.push(id.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The file's text, at most [`MAX_FILE`] bytes of it.
|
||||
fn read_capped(path: &Path) -> Option<String> {
|
||||
let mut text = String::new();
|
||||
std::fs::File::open(path).ok()?.take(MAX_FILE).read_to_string(&mut text).ok()?;
|
||||
Some(text)
|
||||
}
|
||||
|
||||
fn save(s: &State) {
|
||||
let Some(dir) = &s.dir else { return };
|
||||
let mut text =
|
||||
String::from("# Effects turned off because they kept stopping their effect helper.\n# Delete a line (or scan with retry) to try one again.\n");
|
||||
for id in &s.blocked {
|
||||
text.push_str(id);
|
||||
text.push('\n');
|
||||
}
|
||||
if let Err(e) = std::fs::create_dir_all(dir).and_then(|_| std::fs::write(dir.join(FILE), text)) {
|
||||
log::warn!("can't save the helper block list: {e}");
|
||||
}
|
||||
}
|
||||
|
||||
pub fn is_blocked(id: &str) -> bool {
|
||||
let mut s = lock(state());
|
||||
load(&mut s);
|
||||
s.blocked.iter().any(|b| b == id)
|
||||
}
|
||||
|
||||
/// Charge `id` with a helper crash; returns whether it is now blocked.
|
||||
pub fn note_crash(id: &str) -> bool {
|
||||
let mut s = lock(state());
|
||||
load(&mut s);
|
||||
let n = {
|
||||
let n = s.crashes.entry(id.to_string()).or_insert(0);
|
||||
*n = n.saturating_add(1);
|
||||
*n
|
||||
};
|
||||
if n >= CRASH_LIMIT && !s.blocked.iter().any(|b| b == id) {
|
||||
s.blocked.push(id.to_string());
|
||||
save(&s);
|
||||
}
|
||||
n >= CRASH_LIMIT
|
||||
}
|
||||
|
||||
/// The blocked effect ids.
|
||||
pub fn blocked() -> Vec<String> {
|
||||
let mut s = lock(state());
|
||||
load(&mut s);
|
||||
s.blocked.clone()
|
||||
}
|
||||
|
||||
/// Forget every block and crash count.
|
||||
pub fn clear() {
|
||||
let mut s = lock(state());
|
||||
load(&mut s);
|
||||
s.blocked.clear();
|
||||
s.crashes.clear();
|
||||
save(&s);
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn effects_are_blocked_after_repeated_crashes() {
|
||||
let id = "org.test.blocklist-unit";
|
||||
for _ in 1..CRASH_LIMIT {
|
||||
assert!(!note_crash(id));
|
||||
}
|
||||
assert!(note_crash(id));
|
||||
assert!(is_blocked(id) && blocked().iter().any(|b| b == id));
|
||||
clear();
|
||||
assert!(!is_blocked(id));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,686 @@
|
||||
//! EffectCraft's side of a helper: the child process, its pipes and frame exchange file, and
|
||||
//! the [`EffectPlugin`] each remote effect is registered as.
|
||||
//!
|
||||
//! Nothing a helper does can take EffectCraft down: a helper that exits, hangs past its
|
||||
//! timeout or answers nonsense is stopped, the render that hit it fails with a message (shown on
|
||||
//! the effect, the frame is left as it was), and the next render starts the helper again. An
|
||||
//! effect that took its helper down [`crate::blocklist::CRASH_LIMIT`] times is blocked.
|
||||
|
||||
use std::io::{BufRead, BufReader, Read, Write};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::{Child, ChildStdin, Command, Stdio};
|
||||
use std::sync::mpsc::{Receiver, RecvTimeoutError, Sender, channel};
|
||||
use std::sync::{Arc, Condvar, Mutex, MutexGuard, PoisonError, TryLockError};
|
||||
use std::thread::ThreadId;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use effectcraft_effects::plugin::{EffectPlugin, PluginFrame, PluginHost, PluginImage, PluginManifest, PluginParams};
|
||||
|
||||
use crate::blocklist;
|
||||
use crate::frames::Exchange;
|
||||
use crate::protocol::{HELPER_ARG, ImageRef, MAX_LINE, PROTOCOL_VERSION, RemoteEffect, RenderRequest, Reply, Request, parse_reply, request_line};
|
||||
|
||||
/// How deep renders through one helper may nest (an effect asking for its input, which another
|
||||
/// effect of the same helper renders, ...).
|
||||
const MAX_DEPTH: u32 = 16;
|
||||
/// Restarts allowed per minute before a helper is left stopped until the next scan.
|
||||
const MAX_RESTARTS_PER_MINUTE: usize = 5;
|
||||
/// A helper idle for longer than this is pinged before it is used again.
|
||||
const HEARTBEAT_AFTER: Duration = Duration::from_secs(30);
|
||||
const HEARTBEAT_TIMEOUT: Duration = Duration::from_secs(10);
|
||||
/// The longest piece of a helper's stderr logged as one line (longer lines are logged in parts).
|
||||
const MAX_LOG_LINE: u64 = 64 << 10;
|
||||
|
||||
/// [`set_render_timeout`] (milliseconds; 0 = not set).
|
||||
static RENDER_TIMEOUT_MS: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
|
||||
|
||||
/// Override the render timeout (tests; `EFFECTCRAFT_HELPER_TIMEOUT` still wins).
|
||||
pub fn set_render_timeout(t: Duration) {
|
||||
RENDER_TIMEOUT_MS.store(u64::try_from(t.as_millis()).unwrap_or(u64::MAX), std::sync::atomic::Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// Timeout for one render step, `EFFECTCRAFT_HELPER_TIMEOUT` seconds (default 120: heavy
|
||||
/// plug-ins take a while on their first frame).
|
||||
fn render_timeout() -> Duration {
|
||||
if let Some(s) = env_secs("EFFECTCRAFT_HELPER_TIMEOUT") {
|
||||
return Duration::from_secs(s);
|
||||
}
|
||||
match RENDER_TIMEOUT_MS.load(std::sync::atomic::Ordering::Relaxed) {
|
||||
0 => Duration::from_secs(120),
|
||||
ms => Duration::from_millis(ms),
|
||||
}
|
||||
}
|
||||
|
||||
/// Timeout for start-up, `describe` and `load`, `EFFECTCRAFT_HELPER_LOAD_TIMEOUT` seconds
|
||||
/// (default 600: scanning large plug-in suites is slow).
|
||||
fn load_timeout() -> Duration {
|
||||
Duration::from_secs(env_secs("EFFECTCRAFT_HELPER_LOAD_TIMEOUT").unwrap_or(600))
|
||||
}
|
||||
|
||||
fn env_secs(k: &str) -> Option<u64> {
|
||||
std::env::var(k).ok()?.trim().parse().ok().filter(|s| *s > 0)
|
||||
}
|
||||
|
||||
pub(crate) fn lock<T>(m: &Mutex<T>) -> MutexGuard<'_, T> {
|
||||
m.lock().unwrap_or_else(PoisonError::into_inner)
|
||||
}
|
||||
|
||||
/// What the reader thread passes on.
|
||||
enum Line {
|
||||
Reply(Result<Reply, String>),
|
||||
Closed,
|
||||
}
|
||||
|
||||
/// A running helper process.
|
||||
struct Conn {
|
||||
child: Child,
|
||||
/// Request lines for the writer thread, which owns the helper's stdin: a helper that stops
|
||||
/// reading can block that thread, never the render waiting for it ([`Conn::send`]).
|
||||
input: Sender<Vec<u8>>,
|
||||
/// The writer thread's result for each line, in order.
|
||||
written: Receiver<Result<(), String>>,
|
||||
rx: Receiver<Line>,
|
||||
exchange: Exchange,
|
||||
/// The first free byte of the exchange file (images are stacked, see [`Helper::render`]).
|
||||
top: u64,
|
||||
last_used: Instant,
|
||||
generation: u64,
|
||||
}
|
||||
|
||||
impl Conn {
|
||||
/// Write one request, waiting at most `timeout` for the helper to take it.
|
||||
fn send(&mut self, r: &Request, timeout: Duration) -> Result<(), Failure> {
|
||||
let line = request_line(r).map_err(Failure::Broken)?;
|
||||
let closed = || Failure::Broken("can't write to the helper: its input is closed".into());
|
||||
self.input.send(line.into_bytes()).map_err(|_| closed())?;
|
||||
match self.written.recv_timeout(timeout) {
|
||||
Ok(Ok(())) => Ok(()),
|
||||
Ok(Err(e)) => Err(Failure::Broken(format!("can't write to the helper: {e}"))),
|
||||
Err(RecvTimeoutError::Timeout) => {
|
||||
Err(Failure::Broken(format!("the helper did not answer within {} s (it stopped reading its input)", timeout.as_secs())))
|
||||
}
|
||||
Err(RecvTimeoutError::Disconnected) => Err(closed()),
|
||||
}
|
||||
}
|
||||
|
||||
fn recv(&mut self, timeout: Duration) -> Result<Reply, Failure> {
|
||||
match self.rx.recv_timeout(timeout) {
|
||||
Ok(Line::Reply(Ok(r))) => Ok(r),
|
||||
Ok(Line::Reply(Err(e))) => Err(Failure::Broken(e)),
|
||||
Ok(Line::Closed) | Err(RecvTimeoutError::Disconnected) => {
|
||||
// Give the process a moment to finish exiting so its status can be reported.
|
||||
std::thread::sleep(Duration::from_millis(50));
|
||||
let status = match self.child.try_wait() {
|
||||
Ok(Some(s)) => s.to_string(),
|
||||
_ => "closed its output".into(),
|
||||
};
|
||||
Err(Failure::Broken(format!("the helper stopped ({status})")))
|
||||
}
|
||||
Err(RecvTimeoutError::Timeout) => Err(Failure::Broken(format!("the helper did not answer within {} s", timeout.as_secs()))),
|
||||
}
|
||||
}
|
||||
|
||||
fn call(&mut self, r: &Request, timeout: Duration) -> Result<Reply, Failure> {
|
||||
self.send(r, timeout)?;
|
||||
self.recv(timeout)
|
||||
}
|
||||
|
||||
/// Reserve room for an image of `width × height` at the top of the exchange file.
|
||||
fn alloc(&mut self, width: u32, height: u32, origin: [f64; 2], scale: f64) -> Result<ImageRef, String> {
|
||||
let at = ImageRef { offset: self.top, width, height, origin, scale };
|
||||
let bytes = at.bytes().ok_or_else(|| format!("image {width}×{height} is too large for a helper"))?;
|
||||
// Keep images 64-byte aligned for helpers that map the file.
|
||||
self.top = self.top.checked_add(bytes.div_ceil(64).saturating_mul(64)).ok_or("frame exchange file too large")?;
|
||||
Ok(at)
|
||||
}
|
||||
|
||||
fn stop(mut self) {
|
||||
// Not waited for: the process is killed right after, which also ends a blocked write.
|
||||
if let Ok(line) = request_line(&Request::Shutdown) {
|
||||
let _ = self.input.send(line.into_bytes());
|
||||
}
|
||||
let _ = self.child.kill();
|
||||
let _ = self.child.wait();
|
||||
}
|
||||
}
|
||||
|
||||
/// Why a request failed.
|
||||
enum Failure {
|
||||
/// The helper answered with an error and is fine.
|
||||
Refused(String),
|
||||
/// The helper crashed, hung or broke the protocol: it is stopped.
|
||||
Broken(String),
|
||||
}
|
||||
|
||||
/// Whose turn it is to talk to the helper: one thread at a time, re-entered by that thread for
|
||||
/// nested renders.
|
||||
#[derive(Default)]
|
||||
struct Turn {
|
||||
owner: Option<ThreadId>,
|
||||
depth: u32,
|
||||
}
|
||||
|
||||
struct TurnGuard<'a>(&'a Helper);
|
||||
|
||||
impl Drop for TurnGuard<'_> {
|
||||
fn drop(&mut self) {
|
||||
let mut t = lock(&self.0.turn);
|
||||
t.depth = t.depth.saturating_sub(1);
|
||||
if t.depth == 0 {
|
||||
t.owner = None;
|
||||
self.0.turn_cv.notify_all();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// What [`Helper::status`] reports.
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct HelperStatus {
|
||||
pub program: PathBuf,
|
||||
pub name: String,
|
||||
pub version: String,
|
||||
pub loads: Vec<String>,
|
||||
pub running: bool,
|
||||
pub effects: usize,
|
||||
pub restarts: usize,
|
||||
pub last_error: Option<String>,
|
||||
}
|
||||
|
||||
/// One helper program and, while it runs, its process.
|
||||
pub struct Helper {
|
||||
program: PathBuf,
|
||||
info: Mutex<HelperStatus>,
|
||||
conn: Mutex<Option<Conn>>,
|
||||
turn: Mutex<Turn>,
|
||||
turn_cv: Condvar,
|
||||
/// Paths loaded with [`Helper::load`], loaded again after a restart.
|
||||
loaded: Mutex<Vec<String>>,
|
||||
restarts: Mutex<Vec<Instant>>,
|
||||
generation: Mutex<u64>,
|
||||
}
|
||||
|
||||
impl Helper {
|
||||
/// Start a helper and list its effects.
|
||||
pub fn start(program: &Path) -> Result<(Arc<Helper>, Vec<RemoteEffect>, Vec<String>), String> {
|
||||
let h = Arc::new(Helper {
|
||||
program: program.to_path_buf(),
|
||||
info: Mutex::new(HelperStatus { program: program.to_path_buf(), ..Default::default() }),
|
||||
conn: Mutex::new(None),
|
||||
turn: Mutex::new(Turn::default()),
|
||||
turn_cv: Condvar::new(),
|
||||
loaded: Mutex::new(vec![]),
|
||||
restarts: Mutex::new(vec![]),
|
||||
generation: Mutex::new(0),
|
||||
});
|
||||
let (effects, errors) = {
|
||||
let _turn = h.take_turn()?;
|
||||
// A helper that crashes while listing its effects (one bad plug-in) is given a
|
||||
// second and third chance: helpers remember what crashed them and skip it.
|
||||
let mut last = String::new();
|
||||
let mut out = None;
|
||||
for _ in 0..3 {
|
||||
match h.with_conn(|c| c.call(&Request::Describe, load_timeout())) {
|
||||
Ok(Reply::Effects { effects, errors }) => {
|
||||
out = Some((effects, errors));
|
||||
break;
|
||||
}
|
||||
Ok(other) => last = format!("unexpected reply to describe: {other:?}"),
|
||||
Err(Failure::Refused(e) | Failure::Broken(e)) => last = e,
|
||||
}
|
||||
}
|
||||
out.ok_or_else(|| format!("{}: {last}", program.display()))?
|
||||
};
|
||||
lock(&h.info).effects = effects.len();
|
||||
Ok((h, effects, errors))
|
||||
}
|
||||
|
||||
pub fn program(&self) -> &Path {
|
||||
&self.program
|
||||
}
|
||||
|
||||
pub fn status(&self) -> HelperStatus {
|
||||
let mut s = lock(&self.info).clone();
|
||||
// A render holds the process for as long as the helper works on it: don't wait for it.
|
||||
s.running = match self.conn.try_lock() {
|
||||
Ok(c) => c.is_some(),
|
||||
Err(TryLockError::Poisoned(c)) => c.into_inner().is_some(),
|
||||
Err(TryLockError::WouldBlock) => true,
|
||||
};
|
||||
s.restarts = lock(&self.restarts).len();
|
||||
s
|
||||
}
|
||||
|
||||
pub fn loads(&self, ext: &str) -> bool {
|
||||
lock(&self.info).loads.iter().any(|e| e.eq_ignore_ascii_case(ext))
|
||||
}
|
||||
|
||||
pub(crate) fn add_effects(&self, n: usize) {
|
||||
let mut i = lock(&self.info);
|
||||
i.effects = i.effects.saturating_add(n);
|
||||
}
|
||||
|
||||
/// Load effects from a path through the helper ([`Request::Load`]).
|
||||
pub fn load(&self, path: &str) -> Result<(Vec<RemoteEffect>, Vec<String>), String> {
|
||||
let _turn = self.take_turn()?;
|
||||
let r = self.with_conn(|c| c.call(&Request::Load { path: path.to_string() }, load_timeout()));
|
||||
match r {
|
||||
Ok(Reply::Effects { effects, errors }) => {
|
||||
let mut l = lock(&self.loaded);
|
||||
if !l.iter().any(|p| p == path) {
|
||||
l.push(path.to_string());
|
||||
}
|
||||
Ok((effects, errors))
|
||||
}
|
||||
Ok(Reply::Error { message }) => Err(message),
|
||||
Ok(other) => Err(format!("unexpected reply to load: {other:?}")),
|
||||
Err(Failure::Refused(e) | Failure::Broken(e)) => Err(e),
|
||||
}
|
||||
}
|
||||
|
||||
/// Stop the process (it starts again when next needed).
|
||||
pub fn stop(&self) {
|
||||
if let Some(c) = lock(&self.conn).take() {
|
||||
c.stop();
|
||||
}
|
||||
}
|
||||
|
||||
fn take_turn(&self) -> Result<TurnGuard<'_>, String> {
|
||||
let me = std::thread::current().id();
|
||||
// Bounded, so a thread that can't get the helper (its holder waiting for work that a
|
||||
// blocked thread pool would have to do) fails instead of hanging forever.
|
||||
let limit = load_timeout();
|
||||
let deadline = Instant::now().checked_add(limit);
|
||||
let mut t = lock(&self.turn);
|
||||
loop {
|
||||
match t.owner {
|
||||
None => {
|
||||
t.owner = Some(me);
|
||||
t.depth = 1;
|
||||
break;
|
||||
}
|
||||
Some(o) if o == me => {
|
||||
if t.depth >= MAX_DEPTH {
|
||||
return Err("effects through this helper are nested too deeply".into());
|
||||
}
|
||||
t.depth += 1;
|
||||
break;
|
||||
}
|
||||
Some(_) => {
|
||||
let left = deadline.map_or(limit, |d| d.saturating_duration_since(Instant::now()));
|
||||
if left.is_zero() {
|
||||
return Err(format!("the effect helper was busy for more than {} s", limit.as_secs()));
|
||||
}
|
||||
t = self.turn_cv.wait_timeout(t, left).unwrap_or_else(PoisonError::into_inner).0;
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(TurnGuard(self))
|
||||
}
|
||||
|
||||
/// Run `f` on the running process, starting it first if needed. A [`Failure::Broken`]
|
||||
/// stops the process.
|
||||
fn with_conn<T>(&self, f: impl FnOnce(&mut Conn) -> Result<T, Failure>) -> Result<T, Failure> {
|
||||
let mut g = lock(&self.conn);
|
||||
if g.is_none() {
|
||||
*g = Some(self.spawn().map_err(|e| self.not_started(e))?);
|
||||
}
|
||||
let Some(c) = g.as_mut() else { return Err(Failure::Broken("the helper is not running".into())) };
|
||||
if c.last_used.elapsed() > HEARTBEAT_AFTER && !matches!(c.call(&Request::Ping, HEARTBEAT_TIMEOUT), Ok(Reply::Pong)) {
|
||||
if let Some(dead) = g.take() {
|
||||
dead.stop();
|
||||
}
|
||||
self.note_error("the helper stopped answering; restarting it");
|
||||
*g = Some(self.spawn().map_err(|e| self.not_started(e))?);
|
||||
}
|
||||
let Some(c) = g.as_mut() else { return Err(Failure::Broken("the helper is not running".into())) };
|
||||
let r = f(c);
|
||||
c.last_used = Instant::now();
|
||||
if let Err(Failure::Broken(e)) = &r {
|
||||
self.note_error(e);
|
||||
if let Some(dead) = g.take() {
|
||||
dead.stop();
|
||||
}
|
||||
}
|
||||
r
|
||||
}
|
||||
|
||||
/// Run `f` on the process a render started on (generation `started`), without starting or
|
||||
/// pinging one: mid-render the helper is waiting for the answer to its question. A
|
||||
/// [`Failure::Broken`] stops the process. A process already gone was stopped by what broke it
|
||||
/// (a nested render, charged there): not charged again to this render's effect.
|
||||
fn with_running<T>(&self, started: u64, f: impl FnOnce(&mut Conn) -> Result<T, Failure>) -> Result<T, Failure> {
|
||||
let mut g = lock(&self.conn);
|
||||
let Some(c) = g.as_mut().filter(|c| c.generation == started) else {
|
||||
return Err(Failure::Refused("the helper stopped during the render".into()));
|
||||
};
|
||||
let r = f(c);
|
||||
c.last_used = Instant::now();
|
||||
if let Err(Failure::Broken(e)) = &r {
|
||||
self.note_error(e);
|
||||
if let Some(dead) = g.take() {
|
||||
dead.stop();
|
||||
}
|
||||
}
|
||||
r
|
||||
}
|
||||
|
||||
/// A helper that could not be started (not installed any more, restarting too often, no
|
||||
/// frame exchange file): reported, but not charged to the effect that asked for it, since
|
||||
/// nothing it did stopped the helper ([`Failure::Refused`], not [`Failure::Broken`]).
|
||||
fn not_started(&self, e: String) -> Failure {
|
||||
self.note_error(&e);
|
||||
Failure::Refused(e)
|
||||
}
|
||||
|
||||
fn note_error(&self, e: &str) {
|
||||
log::warn!("helper {}: {e}", self.program.display());
|
||||
lock(&self.info).last_error = Some(e.to_string());
|
||||
}
|
||||
|
||||
fn spawn(&self) -> Result<Conn, String> {
|
||||
{
|
||||
let mut r = lock(&self.restarts);
|
||||
r.retain(|t| t.elapsed() < Duration::from_secs(60));
|
||||
if r.len() >= MAX_RESTARTS_PER_MINUTE {
|
||||
return Err("the helper keeps stopping; it stays off until the next scan (Effect ▸ Scan for Effect Helpers)".into());
|
||||
}
|
||||
r.push(Instant::now());
|
||||
}
|
||||
let exchange = Exchange::create().map_err(|e| format!("can't create the frame exchange file: {e}"))?;
|
||||
let mut cmd = Command::new(&self.program);
|
||||
cmd.arg(HELPER_ARG).stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::piped());
|
||||
if let Some(dir) = blocklist::state_dir() {
|
||||
cmd.env("EFFECTCRAFT_HELPER_STATE_DIR", dir);
|
||||
}
|
||||
#[cfg(windows)]
|
||||
{
|
||||
use std::os::windows::process::CommandExt;
|
||||
// CREATE_NO_WINDOW: no console window per helper.
|
||||
cmd.creation_flags(0x0800_0000);
|
||||
}
|
||||
let mut child = cmd.spawn().map_err(|e| format!("can't start {}: {e}", self.program.display()))?;
|
||||
let (Some(stdin), Some(stdout), stderr) = (child.stdin.take(), child.stdout.take(), child.stderr.take()) else {
|
||||
let _ = child.kill();
|
||||
let _ = child.wait();
|
||||
return Err("the helper's pipes are unavailable".into());
|
||||
};
|
||||
let (tx, rx) = channel();
|
||||
let label = self.program.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
|
||||
let l2 = label.clone();
|
||||
let spawned = std::thread::Builder::new().name(format!("helper-out {label}")).spawn(move || {
|
||||
let mut r = BufReader::new(stdout);
|
||||
loop {
|
||||
let mut buf = vec![];
|
||||
match (&mut r).take(MAX_LINE).read_until(b'\n', &mut buf) {
|
||||
Ok(0) | Err(_) => {
|
||||
let _ = tx.send(Line::Closed);
|
||||
return;
|
||||
}
|
||||
Ok(_) => {
|
||||
let line = String::from_utf8_lossy(&buf);
|
||||
match parse_reply(&line) {
|
||||
Some(reply) => {
|
||||
if tx.send(Line::Reply(reply)).is_err() {
|
||||
return;
|
||||
}
|
||||
}
|
||||
None => log::info!("helper {l2}: {}", line.trim_end()),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
if let Err(e) = spawned {
|
||||
let _ = child.kill();
|
||||
let _ = child.wait();
|
||||
return Err(format!("can't start a reader thread: {e}"));
|
||||
}
|
||||
let (input, lines) = channel::<Vec<u8>>();
|
||||
let (done, written) = channel();
|
||||
let writer = std::thread::Builder::new().name(format!("helper-in {label}")).spawn(move || {
|
||||
let mut stdin: ChildStdin = stdin;
|
||||
for line in lines {
|
||||
let r = stdin.write_all(&line).and_then(|_| stdin.flush()).map_err(|e| e.to_string());
|
||||
let failed = r.is_err();
|
||||
if done.send(r).is_err() || failed {
|
||||
return;
|
||||
}
|
||||
}
|
||||
});
|
||||
if let Err(e) = writer {
|
||||
let _ = child.kill();
|
||||
let _ = child.wait();
|
||||
return Err(format!("can't start a writer thread: {e}"));
|
||||
}
|
||||
if let Some(err) = stderr {
|
||||
// Drained to the end whatever it holds (bytes that aren't UTF-8, endless lines): a
|
||||
// helper blocked on a full stderr pipe would hang its render.
|
||||
let _ = std::thread::Builder::new().name(format!("helper-err {label}")).spawn(move || {
|
||||
let mut r = BufReader::new(err);
|
||||
let mut buf = vec![];
|
||||
loop {
|
||||
buf.clear();
|
||||
match (&mut r).take(MAX_LOG_LINE).read_until(b'\n', &mut buf) {
|
||||
Ok(0) | Err(_) => return,
|
||||
Ok(_) => log::info!("helper {label}: {}", String::from_utf8_lossy(&buf).trim_end()),
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
let generation = {
|
||||
let mut g = lock(&self.generation);
|
||||
*g = g.saturating_add(1);
|
||||
*g
|
||||
};
|
||||
let mut c = Conn { child, input, written, rx, exchange, top: 0, last_used: Instant::now(), generation };
|
||||
let frames = c.exchange.path().to_string_lossy().into_owned();
|
||||
match c.call(&Request::Hello { version: PROTOCOL_VERSION, frames }, load_timeout()) {
|
||||
Ok(Reply::Hello { version, name, helper_version, loads }) if version == PROTOCOL_VERSION => {
|
||||
let mut i = lock(&self.info);
|
||||
(i.name, i.version, i.loads) = (name, helper_version, loads);
|
||||
}
|
||||
Ok(Reply::Hello { version, .. }) => {
|
||||
c.stop();
|
||||
return Err(format!("the helper speaks protocol {version}; this EffectCraft speaks {PROTOCOL_VERSION}"));
|
||||
}
|
||||
Ok(Reply::Error { message }) => {
|
||||
c.stop();
|
||||
return Err(message);
|
||||
}
|
||||
Ok(other) => {
|
||||
c.stop();
|
||||
return Err(format!("unexpected reply to hello: {other:?}"));
|
||||
}
|
||||
Err(Failure::Refused(e) | Failure::Broken(e)) => {
|
||||
c.stop();
|
||||
return Err(e);
|
||||
}
|
||||
}
|
||||
c.exchange.unlink();
|
||||
// After a restart the new process knows no effects yet: list them again (the first
|
||||
// start lists them in `Helper::start`), then load what the user loaded before.
|
||||
if generation > 1 && !matches!(c.call(&Request::Describe, load_timeout()), Ok(Reply::Effects { .. })) {
|
||||
c.stop();
|
||||
return Err("the restarted helper could not list its effects".into());
|
||||
}
|
||||
for path in lock(&self.loaded).clone() {
|
||||
if !matches!(c.call(&Request::Load { path: path.clone() }, load_timeout()), Ok(Reply::Effects { .. })) {
|
||||
log::warn!("helper {}: reloading {path} failed", self.program.display());
|
||||
}
|
||||
}
|
||||
Ok(c)
|
||||
}
|
||||
|
||||
fn padding(&self, effect: &str, params: &PluginParams, time: f64, scale: f64, layer_size: [f64; 2], instance: u64) -> Result<u32, String> {
|
||||
let _turn = self.take_turn()?;
|
||||
let req = Request::Padding { effect: effect.to_string(), instance, time, scale, layer_size, flat: params.flat.to_vec(), texts: params.texts.to_vec() };
|
||||
match self.with_conn(|c| c.call(&req, render_timeout())) {
|
||||
Ok(Reply::Padding { pixels }) => Ok(pixels),
|
||||
Ok(Reply::Error { message }) => Err(message),
|
||||
Ok(other) => Err(format!("unexpected reply to padding: {other:?}")),
|
||||
Err(Failure::Refused(e) | Failure::Broken(e)) => Err(e),
|
||||
}
|
||||
}
|
||||
|
||||
/// Render one frame through the helper. Images live on a stack in the exchange file: the
|
||||
/// frame first, then whatever the helper asks for, freed again when the render ends (nested
|
||||
/// renders stack above).
|
||||
fn render(&self, effect: &str, frame: &mut PluginFrame, params: &PluginParams, time: f64, host: &dyn PluginHost) -> Result<(), Failure> {
|
||||
let _turn = self.take_turn().map_err(Failure::Refused)?;
|
||||
// The generation of the process this render runs on: taken inside `with_conn`, which
|
||||
// may start the process (after a crash or `stop`), so it is never that of a dead one.
|
||||
let (at, base, started) = self.with_conn(|c| {
|
||||
let base = c.top;
|
||||
let at = c.alloc(frame.width, frame.height, frame.origin, frame.scale).map_err(Failure::Refused)?;
|
||||
// Nothing is sent yet, so the helper is fine (a full disk, say): not its fault.
|
||||
if let Err(e) = c.exchange.write(&at, frame.pixels) {
|
||||
c.top = base;
|
||||
return Err(Failure::Refused(e));
|
||||
}
|
||||
Ok((at, base, c.generation))
|
||||
})?;
|
||||
let req = Request::Render(RenderRequest {
|
||||
effect: effect.to_string(),
|
||||
instance: host.instance_key(),
|
||||
time,
|
||||
frame: at,
|
||||
layer_size: frame.layer_size,
|
||||
flat: params.flat.to_vec(),
|
||||
texts: params.texts.to_vec(),
|
||||
frame_rate: host.frame_rate(),
|
||||
frame_range: host.frame_range(),
|
||||
});
|
||||
let result = self.converse(req, frame, &at, host, started);
|
||||
// Free this render's images (if the same process is still running).
|
||||
if let Some(c) = lock(&self.conn).as_mut()
|
||||
&& c.generation == started
|
||||
{
|
||||
c.top = base;
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
fn converse(&self, req: Request, frame: &mut PluginFrame, at: &ImageRef, host: &dyn PluginHost, started: u64) -> Result<(), Failure> {
|
||||
let mut next = req;
|
||||
loop {
|
||||
let reply = self.with_running(started, |c| c.call(&next, render_timeout()))?;
|
||||
next = match reply {
|
||||
Reply::Done => return self.with_running(started, |c| c.exchange.read_into(at, frame.pixels).map_err(Failure::Broken)),
|
||||
Reply::Error { message } => return Err(Failure::Refused(message)),
|
||||
// Host callbacks run without holding the process, so they may render through
|
||||
// this helper again (on this thread: the turn is re-entered).
|
||||
Reply::InputAt { time } => self.send_image(host.input_at(time), started)?,
|
||||
Reply::LayerInput { param, time } => self.send_image(host.layer_input(¶m, time), started)?,
|
||||
Reply::ParamsAt { time } => Request::Params { values: host.params_at(time) },
|
||||
other => return Err(Failure::Broken(format!("unexpected reply during a render: {other:?}"))),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/// Put a callback's image in the exchange file and make the answer. The helper is waiting
|
||||
/// for one, so an image that can't be passed on is answered with none (failing here would
|
||||
/// leave it waiting, out of step with the next request).
|
||||
fn send_image(&self, img: Option<PluginImage>, started: u64) -> Result<Request, Failure> {
|
||||
let Some(img) = img.filter(|i| u64::from(i.width).saturating_mul(u64::from(i.height)) == i.pixels.len() as u64) else {
|
||||
return Ok(Request::Image { image: None });
|
||||
};
|
||||
self.with_running(started, |c| {
|
||||
let base = c.top;
|
||||
let written = c.alloc(img.width, img.height, img.origin, img.scale).and_then(|at| c.exchange.write(&at, &img.pixels).map(|()| at));
|
||||
Ok(match written {
|
||||
Ok(at) => Request::Image { image: Some(at) },
|
||||
Err(e) => {
|
||||
log::warn!("helper {}: {e}", self.program.display());
|
||||
c.top = base;
|
||||
Request::Image { image: None }
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for Helper {
|
||||
fn drop(&mut self) {
|
||||
self.stop();
|
||||
}
|
||||
}
|
||||
|
||||
/// A helper's effect, registered like any plug-in effect.
|
||||
pub struct HelperEffect {
|
||||
helper: Arc<Helper>,
|
||||
manifest: PluginManifest,
|
||||
padding: bool,
|
||||
}
|
||||
|
||||
impl HelperEffect {
|
||||
pub fn new(helper: Arc<Helper>, remote: RemoteEffect) -> HelperEffect {
|
||||
HelperEffect { helper, manifest: remote.manifest, padding: remote.padding }
|
||||
}
|
||||
}
|
||||
|
||||
/// A host for [`EffectPlugin::render`] without services.
|
||||
struct NoHost;
|
||||
|
||||
impl PluginHost for NoHost {
|
||||
fn input_at(&self, _: f64) -> Option<PluginImage> {
|
||||
None
|
||||
}
|
||||
fn layer_input(&self, _: &str, _: f64) -> Option<PluginImage> {
|
||||
None
|
||||
}
|
||||
fn params_at(&self, _: f64) -> Option<(Vec<f64>, Vec<String>)> {
|
||||
None
|
||||
}
|
||||
fn frame_rate(&self) -> f64 {
|
||||
30.0
|
||||
}
|
||||
fn time(&self) -> f64 {
|
||||
0.0
|
||||
}
|
||||
}
|
||||
|
||||
impl EffectPlugin for HelperEffect {
|
||||
fn manifest(&self) -> &PluginManifest {
|
||||
&self.manifest
|
||||
}
|
||||
|
||||
fn render(&self, frame: &mut PluginFrame, params: &PluginParams, time: f64) -> Result<(), String> {
|
||||
self.render_v2(frame, params, time, &NoHost)
|
||||
}
|
||||
|
||||
fn source(&self) -> String {
|
||||
format!("helper: {}", self.helper.program.display())
|
||||
}
|
||||
|
||||
fn padding(&self, params: &PluginParams, time: f64, scale: f64, layer_size: [f64; 2], instance_key: u64) -> u32 {
|
||||
if !self.padding || blocklist::is_blocked(&self.manifest.id) {
|
||||
return 0;
|
||||
}
|
||||
match self.helper.padding(&self.manifest.id, params, time, scale, layer_size, instance_key) {
|
||||
Ok(p) => p,
|
||||
Err(e) => {
|
||||
log::warn!("helper effect `{}`: padding: {e}", self.manifest.id);
|
||||
0
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn render_v2(&self, frame: &mut PluginFrame, params: &PluginParams, time: f64, host: &dyn PluginHost) -> Result<(), String> {
|
||||
let id = &self.manifest.id;
|
||||
if blocklist::is_blocked(id) {
|
||||
return Err(format!(
|
||||
"{} is turned off: it stopped its helper {} times (Effect ▸ Scan for Effect Helpers with retry turns it back on)",
|
||||
self.manifest.name,
|
||||
blocklist::CRASH_LIMIT
|
||||
));
|
||||
}
|
||||
match self.helper.render(id, frame, params, time, host) {
|
||||
Ok(()) => Ok(()),
|
||||
Err(Failure::Refused(e)) => Err(e),
|
||||
Err(Failure::Broken(e)) => {
|
||||
blocklist::note_crash(id);
|
||||
Err(format!("{}: the effect helper failed: {e}", self.manifest.name))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
//! The frame exchange file: pixels shared between EffectCraft and a helper without going
|
||||
//! through the control pipe. EffectCraft creates it (in `/dev/shm` when there is one, so it
|
||||
//! stays in memory, else the temp folder) and both sides read and write it at the offsets the
|
||||
//! messages name. Plain file I/O, no `unsafe`: the OS page cache makes it shared memory in
|
||||
//! practice, and a helper is free to map it.
|
||||
|
||||
use std::fs::{File, OpenOptions};
|
||||
use std::io::{Read, Seek, SeekFrom, Write};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
|
||||
use crate::protocol::ImageRef;
|
||||
|
||||
/// An open frame exchange file.
|
||||
pub struct Exchange {
|
||||
file: File,
|
||||
path: PathBuf,
|
||||
/// Remove the file when dropped (the side that created it).
|
||||
owned: bool,
|
||||
}
|
||||
|
||||
impl Exchange {
|
||||
/// Create a new, empty exchange file (EffectCraft's side).
|
||||
pub fn create() -> std::io::Result<Exchange> {
|
||||
static N: AtomicU64 = AtomicU64::new(0);
|
||||
let shm = Path::new("/dev/shm");
|
||||
let dir = if cfg!(target_os = "linux") && shm.is_dir() { shm.to_path_buf() } else { std::env::temp_dir() };
|
||||
let nanos = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.subsec_nanos()).unwrap_or(0);
|
||||
let path = dir.join(format!("effectcraft-helper-{}-{}-{nanos}.frames", std::process::id(), N.fetch_add(1, Ordering::Relaxed)));
|
||||
let mut opts = OpenOptions::new();
|
||||
opts.read(true).write(true).create_new(true);
|
||||
// Frames are the user's pixels and /dev/shm is shared: only this user may read them
|
||||
// (the helper runs as the same user).
|
||||
#[cfg(unix)]
|
||||
std::os::unix::fs::OpenOptionsExt::mode(&mut opts, 0o600);
|
||||
let file = opts.open(&path)?;
|
||||
Ok(Exchange { file, path, owned: true })
|
||||
}
|
||||
|
||||
/// Open an exchange file EffectCraft created (the helper's side).
|
||||
pub fn open(path: &Path) -> std::io::Result<Exchange> {
|
||||
let file = OpenOptions::new().read(true).write(true).open(path)?;
|
||||
Ok(Exchange { file, path: path.to_path_buf(), owned: false })
|
||||
}
|
||||
|
||||
pub fn path(&self) -> &Path {
|
||||
&self.path
|
||||
}
|
||||
|
||||
/// Remove the file's name once both sides have it open (Unix keeps the open file usable),
|
||||
/// so nothing is left behind if either side dies.
|
||||
pub fn unlink(&mut self) {
|
||||
if cfg!(unix) && self.owned && std::fs::remove_file(&self.path).is_ok() {
|
||||
self.owned = false;
|
||||
}
|
||||
}
|
||||
|
||||
/// Write `pixels` as the image `at` describes.
|
||||
pub fn write(&mut self, at: &ImageRef, pixels: &[[f32; 4]]) -> Result<(), String> {
|
||||
if at.pixels() != Some(pixels.len()) {
|
||||
return Err(format!("image is {}×{} but has {} pixels", at.width, at.height, pixels.len()));
|
||||
}
|
||||
self.file.seek(SeekFrom::Start(at.offset)).map_err(io)?;
|
||||
self.file.write_all(bytemuck::cast_slice(pixels)).map_err(io)
|
||||
}
|
||||
|
||||
/// Read the image `at` describes into `out` (which must have its size).
|
||||
pub fn read_into(&mut self, at: &ImageRef, out: &mut [[f32; 4]]) -> Result<(), String> {
|
||||
if at.pixels() != Some(out.len()) {
|
||||
return Err(format!("image is {}×{} but the buffer has {} pixels", at.width, at.height, out.len()));
|
||||
}
|
||||
self.file.seek(SeekFrom::Start(at.offset)).map_err(io)?;
|
||||
self.file.read_exact(bytemuck::cast_slice_mut(out)).map_err(io)
|
||||
}
|
||||
|
||||
/// Read the image `at` describes.
|
||||
pub fn read(&mut self, at: &ImageRef) -> Result<Vec<[f32; 4]>, String> {
|
||||
let n = at.pixels().ok_or_else(|| format!("image {}×{} is too large", at.width, at.height))?;
|
||||
let mut out = vec![[0.0f32; 4]; n];
|
||||
self.read_into(at, &mut out)?;
|
||||
Ok(out)
|
||||
}
|
||||
}
|
||||
|
||||
fn io(e: std::io::Error) -> String {
|
||||
format!("frame exchange: {e}")
|
||||
}
|
||||
|
||||
impl Drop for Exchange {
|
||||
fn drop(&mut self) {
|
||||
if self.owned {
|
||||
let _ = std::fs::remove_file(&self.path);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn both_sides_see_the_same_pixels() {
|
||||
let mut a = Exchange::create().unwrap();
|
||||
let mut b = Exchange::open(a.path()).unwrap();
|
||||
// Owner-only: frames are the user's pixels and /dev/shm is shared.
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
assert_eq!(std::fs::metadata(a.path()).unwrap().permissions().mode() & 0o777, 0o600);
|
||||
}
|
||||
let at = ImageRef { offset: 64, width: 2, height: 1, origin: [0.0; 2], scale: 1.0 };
|
||||
a.write(&at, &[[0.1, 0.2, 0.3, 0.4], [1.0, 2.0, 3.0, 4.0]]).unwrap();
|
||||
assert_eq!(b.read(&at).unwrap()[1], [1.0, 2.0, 3.0, 4.0]);
|
||||
assert!(b.write(&at, &[[0.0; 4]]).is_err(), "size mismatch is an error");
|
||||
let p = a.path().to_path_buf();
|
||||
a.unlink();
|
||||
drop(a);
|
||||
drop(b);
|
||||
assert!(!p.exists());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,258 @@
|
||||
//! # effectcraft-helper-host (L3)
|
||||
//!
|
||||
//! **External effect helpers**: separate programs that offer effects to EffectCraft over a small,
|
||||
//! versioned protocol ([`protocol`], `docs/external-helpers.md`, ADR
|
||||
//! `docs/adr/0001-external-effect-helpers.md`). EffectCraft finds installed helpers
|
||||
//! ([`find_helpers`]), starts each as a child process, lists its effects and registers every one
|
||||
//! as an ordinary plug-in effect (plug-in API 2), so the UI, the property tree, keyframes,
|
||||
//! scripting and the command surface need nothing new. Rendering sends the frame to the helper
|
||||
//! through a shared exchange file ([`frames`]) and answers its questions (the input at another
|
||||
//! time, another layer, parameters at another time) while it works.
|
||||
//!
|
||||
//! The point is containment: whatever native code a helper runs (an OpenFX plug-in host, for
|
||||
//! example) runs in **its** process. If it crashes, hangs or talks nonsense, EffectCraft stops
|
||||
//! it, shows the error on the effect, leaves the frame as it was and starts the helper again
|
||||
//! for the next render; an effect that keeps taking its helper down is turned off
|
||||
//! ([`blocklist`]). This crate is safe Rust (`unsafe_code = "forbid"`, like the rest of the
|
||||
//! workspace), loads no native code and opens no network connection.
|
||||
//!
|
||||
//! [`serve`] is the helper side, for helper programs written in Rust.
|
||||
//!
|
||||
//! Native only: on wasm32 the crate is empty (there are no processes to start).
|
||||
|
||||
#![cfg(not(target_arch = "wasm32"))]
|
||||
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
|
||||
|
||||
pub mod blocklist;
|
||||
pub mod client;
|
||||
pub mod frames;
|
||||
pub mod protocol;
|
||||
pub mod serve;
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::{Arc, Mutex, OnceLock};
|
||||
|
||||
use effectcraft_effects::plugin::{plugin, register_plugin};
|
||||
use serde_json::{Value, json};
|
||||
|
||||
use crate::client::{Helper, HelperEffect, lock};
|
||||
use crate::protocol::RemoteEffect;
|
||||
|
||||
/// Helper program names start with this (`effectcraft-helper-ofx`, plus `.exe` on Windows).
|
||||
pub const PROGRAM_PREFIX: &str = "effectcraft-helper-";
|
||||
|
||||
/// The running helpers.
|
||||
fn helpers() -> &'static Mutex<Vec<Arc<Helper>>> {
|
||||
static H: OnceLock<Mutex<Vec<Arc<Helper>>>> = OnceLock::new();
|
||||
H.get_or_init(Default::default)
|
||||
}
|
||||
|
||||
/// Where helpers are looked for: `helpers/` next to the executable, `helpers/` in the settings
|
||||
/// folder (`config_dir`), and every entry of `EFFECTCRAFT_HELPER_PATH` (a path list of helper
|
||||
/// programs or folders).
|
||||
pub fn default_search_paths(config_dir: Option<&Path>) -> Vec<PathBuf> {
|
||||
let mut v = vec![];
|
||||
if let Some(dir) = std::env::current_exe().ok().and_then(|e| e.parent().map(Path::to_path_buf)) {
|
||||
v.push(dir.join("helpers"));
|
||||
// macOS app bundles: Contents/MacOS/effectcraft → Contents/Helpers.
|
||||
if cfg!(target_os = "macos")
|
||||
&& let Some(contents) = dir.parent()
|
||||
{
|
||||
v.push(contents.join("Helpers"));
|
||||
}
|
||||
}
|
||||
if let Some(c) = config_dir {
|
||||
v.push(c.join("helpers"));
|
||||
}
|
||||
if let Some(p) = std::env::var_os("EFFECTCRAFT_HELPER_PATH") {
|
||||
v.extend(std::env::split_paths(&p).filter(|p| !p.as_os_str().is_empty()));
|
||||
}
|
||||
v
|
||||
}
|
||||
|
||||
/// Whether a file name is a helper program's.
|
||||
pub fn is_helper_name(name: &str) -> bool {
|
||||
let stem = if cfg!(windows) {
|
||||
match name.len().checked_sub(4).and_then(|i| name.get(i..).map(|ext| (i, ext))) {
|
||||
Some((i, ext)) if ext.eq_ignore_ascii_case(".exe") => name.get(..i).unwrap_or(""),
|
||||
_ => return false,
|
||||
}
|
||||
} else {
|
||||
name
|
||||
};
|
||||
stem.len() > PROGRAM_PREFIX.len() && stem.starts_with(PROGRAM_PREFIX) && !stem.contains('.')
|
||||
}
|
||||
|
||||
/// The helper programs in `paths`: files named `effectcraft-helper-*` in each folder and its
|
||||
/// direct subfolders, and paths that are themselves such files (any name is accepted for an
|
||||
/// explicit file). Sorted, without duplicates.
|
||||
pub fn find_helpers(paths: &[PathBuf]) -> Vec<PathBuf> {
|
||||
fn named(p: &Path) -> bool {
|
||||
p.is_file() && p.file_name().and_then(|n| n.to_str()).is_some_and(is_helper_name)
|
||||
}
|
||||
let mut out = vec![];
|
||||
for p in paths {
|
||||
if p.is_file() {
|
||||
out.push(p.clone());
|
||||
continue;
|
||||
}
|
||||
let Ok(rd) = std::fs::read_dir(p) else { continue };
|
||||
for e in rd.flatten() {
|
||||
let path = e.path();
|
||||
if named(&path) {
|
||||
out.push(path);
|
||||
} else if path.is_dir()
|
||||
&& let Ok(sub) = std::fs::read_dir(&path)
|
||||
{
|
||||
out.extend(sub.flatten().map(|e| e.path()).filter(|p| named(p)));
|
||||
}
|
||||
}
|
||||
}
|
||||
out.sort();
|
||||
out.dedup();
|
||||
out
|
||||
}
|
||||
|
||||
/// Register a helper's effects. Effects already registered (an earlier scan) are kept as they
|
||||
/// are; blocked effects are reported, not registered.
|
||||
fn register(helper: &Arc<Helper>, effects: Vec<RemoteEffect>, errors: &mut Vec<String>) -> Vec<Value> {
|
||||
let mut loaded = vec![];
|
||||
let mut added = 0;
|
||||
for e in effects {
|
||||
let id = e.manifest.id.clone();
|
||||
if blocklist::is_blocked(&id) {
|
||||
errors.push(format!("{id}: turned off because it kept stopping its helper (scan with retry to try again)"));
|
||||
continue;
|
||||
}
|
||||
if plugin(&id).is_some() {
|
||||
continue;
|
||||
}
|
||||
let (name, category) = (e.manifest.name.clone(), e.manifest.category.clone());
|
||||
match register_plugin(Arc::new(HelperEffect::new(helper.clone(), e))) {
|
||||
Ok(_) => {
|
||||
added += 1;
|
||||
loaded.push(json!({"id": id, "name": name, "category": category, "helper": helper.program().to_string_lossy()}));
|
||||
}
|
||||
Err(err) => errors.push(format!("{id}: {err}")),
|
||||
}
|
||||
}
|
||||
helper.add_effects(added);
|
||||
loaded
|
||||
}
|
||||
|
||||
/// Start every helper found in `paths` that isn't running yet and register its effects.
|
||||
/// `retry` forgets the block list first. Returns `{paths, helpers, loaded, errors}`.
|
||||
pub fn scan_paths(paths: &[PathBuf], retry: bool) -> Value {
|
||||
if retry {
|
||||
blocklist::clear();
|
||||
}
|
||||
let (mut started, mut loaded, mut errors) = (vec![], vec![], vec![]);
|
||||
for program in find_helpers(paths) {
|
||||
if lock(helpers()).iter().any(|h| h.program() == program) {
|
||||
continue;
|
||||
}
|
||||
match Helper::start(&program) {
|
||||
Ok((h, effects, errs)) => {
|
||||
errors.extend(errs.into_iter().map(|e| format!("{}: {e}", program.display())));
|
||||
loaded.extend(register(&h, effects, &mut errors));
|
||||
started.push(json!(program.to_string_lossy()));
|
||||
lock(helpers()).push(h);
|
||||
}
|
||||
Err(e) => errors.push(e),
|
||||
}
|
||||
}
|
||||
json!({
|
||||
"paths": paths.iter().map(|p| p.to_string_lossy()).collect::<Vec<_>>(),
|
||||
"helpers": started,
|
||||
"loaded": loaded,
|
||||
"errors": errors,
|
||||
})
|
||||
}
|
||||
|
||||
/// Load a file or folder through the helpers that accept its extension (a folder goes to every
|
||||
/// helper that loads anything). Returns `{loaded, errors}`.
|
||||
pub fn load_path(path: &str) -> Result<Value, String> {
|
||||
let p = Path::new(path);
|
||||
let ext = p.extension().and_then(|e| e.to_str()).unwrap_or("").to_string();
|
||||
// `.ofx.bundle` folders and the like: match on any dotted part of the name.
|
||||
let parts: Vec<String> = p.file_name().and_then(|n| n.to_str()).map(|n| n.split('.').skip(1).map(str::to_string).collect()).unwrap_or_default();
|
||||
let list: Vec<Arc<Helper>> = lock(helpers()).clone();
|
||||
let takers: Vec<Arc<Helper>> = list
|
||||
.into_iter()
|
||||
.filter(|h| {
|
||||
let s = h.status();
|
||||
if p.is_dir() && parts.is_empty() { !s.loads.is_empty() } else { h.loads(&ext) || parts.iter().any(|x| h.loads(x)) }
|
||||
})
|
||||
.collect();
|
||||
if takers.is_empty() {
|
||||
return Err(format!("no effect helper loads `{path}` (installed helpers: Effect ▸ Scan for Effect Helpers)"));
|
||||
}
|
||||
let (mut loaded, mut errors) = (vec![], vec![]);
|
||||
for h in takers {
|
||||
match h.load(path) {
|
||||
Ok((effects, errs)) => {
|
||||
errors.extend(errs);
|
||||
loaded.extend(register(&h, effects, &mut errors));
|
||||
}
|
||||
Err(e) => errors.push(e),
|
||||
}
|
||||
}
|
||||
if loaded.is_empty() && !errors.is_empty() {
|
||||
return Err(errors.join("; "));
|
||||
}
|
||||
Ok(json!({"loaded": loaded, "errors": errors}))
|
||||
}
|
||||
|
||||
/// The helpers and their state: `{helpers: [{program, name, version, loads, running, effects,
|
||||
/// restarts, error}], blocked}`.
|
||||
pub fn list() -> Value {
|
||||
let helpers: Vec<Value> = lock(helpers())
|
||||
.iter()
|
||||
.map(|h| {
|
||||
let s = h.status();
|
||||
json!({
|
||||
"program": s.program.to_string_lossy(),
|
||||
"name": s.name,
|
||||
"version": s.version,
|
||||
"loads": s.loads,
|
||||
"running": s.running,
|
||||
"effects": s.effects,
|
||||
"restarts": s.restarts,
|
||||
"error": s.last_error,
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
json!({"helpers": helpers, "blocked": blocklist::blocked()})
|
||||
}
|
||||
|
||||
/// Stop every helper process (they start again when needed). Called when the app quits;
|
||||
/// helpers also exit on their own when EffectCraft's end of the pipe closes.
|
||||
pub fn stop_all() {
|
||||
for h in lock(helpers()).iter() {
|
||||
h.stop();
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn helper_programs_are_recognised_by_name() {
|
||||
if cfg!(windows) {
|
||||
assert!(is_helper_name("effectcraft-helper-ofx.exe"));
|
||||
assert!(!is_helper_name("effectcraft-helper-ofx"));
|
||||
} else {
|
||||
assert!(is_helper_name("effectcraft-helper-ofx"));
|
||||
assert!(!is_helper_name("effectcraft-helper-ofx.pdb"));
|
||||
}
|
||||
assert!(!is_helper_name("effectcraft-helper-"));
|
||||
assert!(!is_helper_name("effectcraft"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_folders_find_nothing_and_loading_without_helpers_is_an_error() {
|
||||
assert!(find_helpers(&[PathBuf::from("/no/such/folder")]).is_empty());
|
||||
assert!(load_path("/no/such/thing.xyz").is_err());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,207 @@
|
||||
//! The helper protocol, version [`PROTOCOL_VERSION`] (`docs/external-helpers.md`).
|
||||
//!
|
||||
//! Control messages are JSON, one per line. EffectCraft writes requests ([`Request`]) to the
|
||||
//! helper's stdin; the helper writes replies ([`Reply`]) to its stdout, each line starting with
|
||||
//! [`MAGIC`] so that anything else a helper (or a library it loads) prints is ignored. Pixels
|
||||
//! never travel over the pipe: they live in the **frame exchange file** named in
|
||||
//! [`Request::Hello`] and are addressed by [`ImageRef`]s (byte offset + size).
|
||||
|
||||
use effectcraft_effects::plugin::PluginManifest;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// The protocol version this build speaks. A helper that answers `hello` with another version
|
||||
/// is not used.
|
||||
pub const PROTOCOL_VERSION: u32 = 1;
|
||||
|
||||
/// Every reply line from a helper starts with this (anything before it on the line, and lines
|
||||
/// without it, are the helper's own output and only logged).
|
||||
pub const MAGIC: &str = "@ec-helper ";
|
||||
|
||||
/// The argument EffectCraft starts helpers with.
|
||||
pub const HELPER_ARG: &str = "--effectcraft-helper";
|
||||
|
||||
/// The longest control line either side accepts (bytes). Hidden parameters can carry a few MB.
|
||||
pub const MAX_LINE: u64 = 256 << 20;
|
||||
|
||||
/// The most pixels one image may have (16384 × 16384).
|
||||
pub const MAX_PIXELS: u64 = 1 << 28;
|
||||
|
||||
/// Bytes per pixel in the exchange file: premultiplied RGBA `f32`, native byte order.
|
||||
pub const PIXEL_BYTES: u64 = 16;
|
||||
|
||||
/// An image in the frame exchange file: `width * height` premultiplied RGBA `f32` pixels, row 0
|
||||
/// on top, starting `offset` bytes into the file. `origin` and `scale` are as in
|
||||
/// [`effectcraft_effects::plugin::PluginImage`].
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub struct ImageRef {
|
||||
pub offset: u64,
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
#[serde(default)]
|
||||
pub origin: [f64; 2],
|
||||
#[serde(default = "one")]
|
||||
pub scale: f64,
|
||||
}
|
||||
|
||||
fn one() -> f64 {
|
||||
1.0
|
||||
}
|
||||
|
||||
impl ImageRef {
|
||||
/// Pixels in the image, `None` when it is larger than [`MAX_PIXELS`].
|
||||
pub fn pixels(&self) -> Option<usize> {
|
||||
let n = u64::from(self.width).checked_mul(u64::from(self.height))?;
|
||||
(n <= MAX_PIXELS).then_some(n).and_then(|n| usize::try_from(n).ok())
|
||||
}
|
||||
/// Bytes the image takes in the exchange file.
|
||||
pub fn bytes(&self) -> Option<u64> {
|
||||
u64::try_from(self.pixels()?).ok()?.checked_mul(PIXEL_BYTES)
|
||||
}
|
||||
}
|
||||
|
||||
/// One render of one effect instance.
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub struct RenderRequest {
|
||||
/// The effect id (as described by the helper).
|
||||
pub effect: String,
|
||||
/// A stable id of the applied effect instance ([`effectcraft_effects::plugin::PluginHost::instance_key`]).
|
||||
pub instance: u64,
|
||||
/// Layer time, seconds.
|
||||
pub time: f64,
|
||||
/// The buffer to process in place (already padded); the result is written back to it.
|
||||
pub frame: ImageRef,
|
||||
/// The layer's size in layer pixels.
|
||||
pub layer_size: [f64; 2],
|
||||
/// Flattened parameter values (plug-in API 2 rules).
|
||||
pub flat: Vec<f64>,
|
||||
/// Text / hidden parameter strings.
|
||||
#[serde(default)]
|
||||
pub texts: Vec<String>,
|
||||
pub frame_rate: f64,
|
||||
/// The layer's source range in layer-time seconds, when it has one.
|
||||
#[serde(default)]
|
||||
pub frame_range: Option<(f64, f64)>,
|
||||
}
|
||||
|
||||
/// EffectCraft → helper.
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(tag = "op", rename_all = "snake_case")]
|
||||
pub enum Request {
|
||||
/// First message: the protocol version and the frame exchange file to open (read + write).
|
||||
Hello { version: u32, frames: String },
|
||||
/// List the effects the helper offers → [`Reply::Effects`].
|
||||
Describe,
|
||||
/// Load more effects from a file or folder the user picked → [`Reply::Effects`] (only the
|
||||
/// new ones).
|
||||
Load { path: String },
|
||||
/// Transparent pixels to add around the buffer before [`Request::Render`] → [`Reply::Padding`].
|
||||
Padding { effect: String, instance: u64, time: f64, scale: f64, layer_size: [f64; 2], flat: Vec<f64>, texts: Vec<String> },
|
||||
/// Render → callbacks ([`Reply::InputAt`], ...) answered with [`Request::Image`] /
|
||||
/// [`Request::Params`], then [`Reply::Done`] or [`Reply::Error`].
|
||||
Render(RenderRequest),
|
||||
/// The answer to [`Reply::InputAt`] / [`Reply::LayerInput`]: the image, or `None`.
|
||||
Image { image: Option<ImageRef> },
|
||||
/// The answer to [`Reply::ParamsAt`]: flattened values and texts, or `None`.
|
||||
Params { values: Option<(Vec<f64>, Vec<String>)> },
|
||||
/// The effect instance is gone; the helper may free what it keeps for it. No reply.
|
||||
Destroy { instance: u64 },
|
||||
/// Heartbeat → [`Reply::Pong`].
|
||||
Ping,
|
||||
/// Exit. No reply. (Closing stdin means the same.)
|
||||
Shutdown,
|
||||
}
|
||||
|
||||
/// One effect a helper offers.
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub struct RemoteEffect {
|
||||
/// The plug-in API 2 manifest (id, name, category, parameters).
|
||||
pub manifest: PluginManifest,
|
||||
/// Whether to ask [`Request::Padding`] before every render (otherwise no padding).
|
||||
#[serde(default)]
|
||||
pub padding: bool,
|
||||
}
|
||||
|
||||
/// Helper → EffectCraft.
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(tag = "op", rename_all = "snake_case")]
|
||||
pub enum Reply {
|
||||
/// Answer to [`Request::Hello`]. `loads` lists the file extensions [`Request::Load`] accepts
|
||||
/// (`["ofx"]`), empty when the helper loads nothing on request.
|
||||
Hello {
|
||||
version: u32,
|
||||
name: String,
|
||||
#[serde(default)]
|
||||
helper_version: String,
|
||||
#[serde(default)]
|
||||
loads: Vec<String>,
|
||||
},
|
||||
Effects {
|
||||
effects: Vec<RemoteEffect>,
|
||||
#[serde(default)]
|
||||
errors: Vec<String>,
|
||||
},
|
||||
Padding {
|
||||
pixels: u32,
|
||||
},
|
||||
/// During a render: the effect's input at another layer time.
|
||||
InputAt {
|
||||
time: f64,
|
||||
},
|
||||
/// During a render: the layer chosen in a layer parameter, at a layer time.
|
||||
LayerInput {
|
||||
param: String,
|
||||
time: f64,
|
||||
},
|
||||
/// During a render: the parameter values at another layer time.
|
||||
ParamsAt {
|
||||
time: f64,
|
||||
},
|
||||
/// The render finished; the result is in the request's frame.
|
||||
Done,
|
||||
/// The request failed (the helper keeps running).
|
||||
Error {
|
||||
message: String,
|
||||
},
|
||||
Pong,
|
||||
}
|
||||
|
||||
/// A reply as one protocol line (with [`MAGIC`] and the newline).
|
||||
pub fn reply_line(r: &Reply) -> Result<String, String> {
|
||||
serde_json::to_string(r).map(|s| format!("{MAGIC}{s}\n")).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// A request as one protocol line (with the newline).
|
||||
pub fn request_line(r: &Request) -> Result<String, String> {
|
||||
serde_json::to_string(r).map(|s| format!("{s}\n")).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// Parse a helper output line: `Some` for a protocol line (after [`MAGIC`]), `None` for other
|
||||
/// output.
|
||||
pub fn parse_reply(line: &str) -> Option<Result<Reply, String>> {
|
||||
let at = line.find(MAGIC)?;
|
||||
let json = line.get(at + MAGIC.len()..)?.trim_end();
|
||||
Some(serde_json::from_str(json).map_err(|e| format!("bad reply from helper: {e}")))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn replies_round_trip_and_other_output_is_ignored() {
|
||||
let r = Reply::InputAt { time: 1.5 };
|
||||
let line = reply_line(&r).unwrap();
|
||||
assert_eq!(parse_reply(&line).unwrap().unwrap(), r);
|
||||
// A plug-in that printed without a newline doesn't break the protocol line.
|
||||
assert_eq!(parse_reply(&format!("loading...{line}")).unwrap().unwrap(), r);
|
||||
assert!(parse_reply("plug-in chatter").is_none());
|
||||
assert!(parse_reply("@ec-helper {nope").unwrap().is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn image_sizes_are_checked() {
|
||||
let r = ImageRef { offset: 0, width: 4, height: 2, origin: [0.0; 2], scale: 1.0 };
|
||||
assert_eq!(r.bytes(), Some(128));
|
||||
assert_eq!(ImageRef { width: u32::MAX, height: u32::MAX, ..r }.pixels(), None);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,240 @@
|
||||
//! The helper side of the protocol: [`serve`] answers EffectCraft's requests with any set of
|
||||
//! [`EffectPlugin`]s, so a helper program only has to provide its effects ([`Provider`]).
|
||||
//! The fake helper in the tests and the OpenFX helper both use it.
|
||||
//!
|
||||
//! During a render the plug-in's [`PluginHost`] calls become requests to EffectCraft
|
||||
//! ([`Reply::InputAt`], ...). While a helper waits for the answer, EffectCraft may send a nested
|
||||
//! [`Request::Render`] (an effect of this helper further up the same layer's stack); it is
|
||||
//! served right away, on the same stack, before the wait goes on.
|
||||
|
||||
use std::cell::RefCell;
|
||||
use std::collections::HashMap;
|
||||
use std::io::{BufRead, Read, Write};
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
|
||||
use effectcraft_effects::plugin::{EffectPlugin, PluginFrame, PluginHost, PluginImage, PluginParams};
|
||||
|
||||
use crate::frames::Exchange;
|
||||
use crate::protocol::{ImageRef, MAX_LINE, PROTOCOL_VERSION, RemoteEffect, RenderRequest, Reply, Request, reply_line};
|
||||
|
||||
/// How deep nested renders may go.
|
||||
const MAX_DEPTH: u32 = 16;
|
||||
|
||||
/// What a helper program offers.
|
||||
pub trait Provider {
|
||||
/// The helper's display name (`OpenFX`).
|
||||
fn name(&self) -> String;
|
||||
/// The helper's own version.
|
||||
fn version(&self) -> String {
|
||||
String::new()
|
||||
}
|
||||
/// File extensions [`Provider::load`] accepts (`["ofx"]`).
|
||||
fn loads(&self) -> Vec<String> {
|
||||
vec![]
|
||||
}
|
||||
/// The effects available at start-up, and errors worth reporting.
|
||||
fn effects(&mut self) -> (Vec<Arc<dyn EffectPlugin>>, Vec<String>);
|
||||
/// Load effects from a file or folder the user picked.
|
||||
fn load(&mut self, _path: &str) -> Result<(Vec<Arc<dyn EffectPlugin>>, Vec<String>), String> {
|
||||
Err("this helper does not load files".into())
|
||||
}
|
||||
/// Whether EffectCraft should ask for padding before each render of `effect`.
|
||||
fn wants_padding(&self, _effect: &dyn EffectPlugin) -> bool {
|
||||
true
|
||||
}
|
||||
/// The effect instance is gone.
|
||||
fn destroy(&mut self, _instance: u64) {}
|
||||
}
|
||||
|
||||
/// Serve requests on stdin / stdout until EffectCraft says `shutdown` or closes the pipe.
|
||||
pub fn serve(provider: &mut dyn Provider) -> Result<(), String> {
|
||||
let stdin = std::io::stdin();
|
||||
let stdout = std::io::stdout();
|
||||
serve_io(provider, stdin.lock(), stdout.lock())
|
||||
}
|
||||
|
||||
/// [`serve`] on any reader / writer.
|
||||
pub fn serve_io(provider: &mut dyn Provider, input: impl BufRead, output: impl Write) -> Result<(), String> {
|
||||
let mut s = Server { provider, input: Box::new(input), output: Box::new(output), exchange: None, effects: HashMap::new(), depth: 0 };
|
||||
loop {
|
||||
let Some(req) = s.next_request()? else { return Ok(()) };
|
||||
if matches!(req, Request::Shutdown) {
|
||||
return Ok(());
|
||||
}
|
||||
s.handle(req)?;
|
||||
}
|
||||
}
|
||||
|
||||
struct Server<'p> {
|
||||
provider: &'p mut dyn Provider,
|
||||
input: Box<dyn BufRead + 'p>,
|
||||
output: Box<dyn Write + 'p>,
|
||||
exchange: Option<Exchange>,
|
||||
effects: HashMap<String, Arc<dyn EffectPlugin>>,
|
||||
depth: u32,
|
||||
}
|
||||
|
||||
impl Server<'_> {
|
||||
/// The next request; `None` when the pipe closed.
|
||||
fn next_request(&mut self) -> Result<Option<Request>, String> {
|
||||
loop {
|
||||
let mut buf = vec![];
|
||||
let n = (&mut self.input).take(MAX_LINE).read_until(b'\n', &mut buf).map_err(|e| e.to_string())?;
|
||||
if n == 0 {
|
||||
return Ok(None);
|
||||
}
|
||||
let line = String::from_utf8_lossy(&buf);
|
||||
if line.trim().is_empty() {
|
||||
continue;
|
||||
}
|
||||
match serde_json::from_str::<Request>(line.trim()) {
|
||||
Ok(r) => return Ok(Some(r)),
|
||||
Err(e) => self.send(&Reply::Error { message: format!("bad request: {e}") })?,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn send(&mut self, r: &Reply) -> Result<(), String> {
|
||||
let line = reply_line(r)?;
|
||||
self.output.write_all(line.as_bytes()).and_then(|_| self.output.flush()).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn describe(&mut self, list: Vec<Arc<dyn EffectPlugin>>, errors: Vec<String>) -> Result<(), String> {
|
||||
let mut effects = vec![];
|
||||
for e in list {
|
||||
let manifest = e.manifest().clone();
|
||||
effects.push(RemoteEffect { padding: self.provider.wants_padding(e.as_ref()), manifest: manifest.clone() });
|
||||
self.effects.insert(manifest.id, e);
|
||||
}
|
||||
self.send(&Reply::Effects { effects, errors })
|
||||
}
|
||||
|
||||
/// Answer one request (anything but `shutdown`).
|
||||
fn handle(&mut self, req: Request) -> Result<(), String> {
|
||||
match req {
|
||||
Request::Hello { version, frames } => {
|
||||
if version != PROTOCOL_VERSION {
|
||||
return self.send(&Reply::Error { message: format!("this helper speaks protocol {PROTOCOL_VERSION}, not {version}") });
|
||||
}
|
||||
match Exchange::open(Path::new(&frames)) {
|
||||
Ok(x) => self.exchange = Some(x),
|
||||
Err(e) => return self.send(&Reply::Error { message: format!("can't open the frame exchange file: {e}") }),
|
||||
}
|
||||
let (name, helper_version, loads) = (self.provider.name(), self.provider.version(), self.provider.loads());
|
||||
self.send(&Reply::Hello { version: PROTOCOL_VERSION, name, helper_version, loads })
|
||||
}
|
||||
Request::Describe => {
|
||||
let (list, errors) = self.provider.effects();
|
||||
self.describe(list, errors)
|
||||
}
|
||||
Request::Load { path } => match self.provider.load(&path) {
|
||||
Ok((list, errors)) => self.describe(list, errors),
|
||||
Err(message) => self.send(&Reply::Error { message }),
|
||||
},
|
||||
Request::Padding { effect, instance, time, scale, layer_size, flat, texts } => {
|
||||
let Some(e) = self.effects.get(&effect).cloned() else {
|
||||
return self.send(&Reply::Error { message: format!("unknown effect `{effect}`") });
|
||||
};
|
||||
let params = PluginParams { manifest: e.manifest(), flat: &flat, texts: &texts };
|
||||
let pixels = e.padding(¶ms, time, scale, layer_size, instance);
|
||||
self.send(&Reply::Padding { pixels })
|
||||
}
|
||||
Request::Render(r) => match self.render(&r) {
|
||||
Ok(()) => self.send(&Reply::Done),
|
||||
Err(message) => self.send(&Reply::Error { message }),
|
||||
},
|
||||
Request::Destroy { instance } => {
|
||||
self.provider.destroy(instance);
|
||||
Ok(())
|
||||
}
|
||||
Request::Ping => self.send(&Reply::Pong),
|
||||
Request::Image { .. } | Request::Params { .. } => self.send(&Reply::Error { message: "no callback is waiting for an answer".into() }),
|
||||
Request::Shutdown => Ok(()),
|
||||
}
|
||||
}
|
||||
|
||||
fn render(&mut self, r: &RenderRequest) -> Result<(), String> {
|
||||
let e = self.effects.get(&r.effect).cloned().ok_or_else(|| format!("unknown effect `{}`", r.effect))?;
|
||||
if self.depth >= MAX_DEPTH {
|
||||
return Err("renders nested too deeply".into());
|
||||
}
|
||||
let exchange = self.exchange.as_mut().ok_or("no frame exchange file (hello first)")?;
|
||||
let mut pixels = exchange.read(&r.frame)?;
|
||||
let params = PluginParams { manifest: e.manifest(), flat: &r.flat, texts: &r.texts };
|
||||
let mut frame = PluginFrame {
|
||||
width: r.frame.width,
|
||||
height: r.frame.height,
|
||||
pixels: &mut pixels,
|
||||
scale: r.frame.scale,
|
||||
origin: r.frame.origin,
|
||||
layer_size: r.layer_size,
|
||||
};
|
||||
self.depth += 1;
|
||||
let result = {
|
||||
let host = Callbacks { server: RefCell::new(&mut *self), req: r };
|
||||
e.render_v2(&mut frame, ¶ms, r.time, &host)
|
||||
};
|
||||
self.depth = self.depth.saturating_sub(1);
|
||||
result?;
|
||||
let exchange = self.exchange.as_mut().ok_or("no frame exchange file")?;
|
||||
exchange.write(&r.frame, &pixels)
|
||||
}
|
||||
|
||||
/// Ask EffectCraft something during a render and wait for the answer, serving nested
|
||||
/// requests meanwhile.
|
||||
fn ask(&mut self, q: &Reply) -> Result<Request, String> {
|
||||
self.send(q)?;
|
||||
loop {
|
||||
let req = self.next_request()?.ok_or("EffectCraft closed the pipe")?;
|
||||
match req {
|
||||
Request::Image { .. } | Request::Params { .. } => return Ok(req),
|
||||
Request::Shutdown => return Err("EffectCraft asked the helper to shut down".into()),
|
||||
other => self.handle(other)?,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn image(&mut self, q: &Reply) -> Option<PluginImage> {
|
||||
let Ok(Request::Image { image: Some(at) }) = self.ask(q) else { return None };
|
||||
self.read_image(&at)
|
||||
}
|
||||
|
||||
fn read_image(&mut self, at: &ImageRef) -> Option<PluginImage> {
|
||||
let pixels = self.exchange.as_mut()?.read(at).ok()?;
|
||||
Some(PluginImage { width: at.width, height: at.height, pixels, origin: at.origin, scale: at.scale })
|
||||
}
|
||||
}
|
||||
|
||||
/// The [`PluginHost`] a helper's plug-in renders with: questions go back to EffectCraft.
|
||||
struct Callbacks<'a, 'p> {
|
||||
server: RefCell<&'a mut Server<'p>>,
|
||||
req: &'a RenderRequest,
|
||||
}
|
||||
|
||||
impl PluginHost for Callbacks<'_, '_> {
|
||||
fn input_at(&self, layer_time: f64) -> Option<PluginImage> {
|
||||
self.server.try_borrow_mut().ok()?.image(&Reply::InputAt { time: layer_time })
|
||||
}
|
||||
fn layer_input(&self, param_id: &str, layer_time: f64) -> Option<PluginImage> {
|
||||
self.server.try_borrow_mut().ok()?.image(&Reply::LayerInput { param: param_id.to_string(), time: layer_time })
|
||||
}
|
||||
fn params_at(&self, layer_time: f64) -> Option<(Vec<f64>, Vec<String>)> {
|
||||
match self.server.try_borrow_mut().ok()?.ask(&Reply::ParamsAt { time: layer_time }) {
|
||||
Ok(Request::Params { values }) => values,
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
fn frame_rate(&self) -> f64 {
|
||||
self.req.frame_rate
|
||||
}
|
||||
fn time(&self) -> f64 {
|
||||
self.req.time
|
||||
}
|
||||
fn instance_key(&self) -> u64 {
|
||||
self.req.instance
|
||||
}
|
||||
fn frame_range(&self) -> Option<(f64, f64)> {
|
||||
self.req.frame_range
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,327 @@
|
||||
//! The helper protocol end to end against a fake helper, written in safe Rust with
|
||||
//! [`effectcraft_helper_host::serve`]. This binary is both sides: started with
|
||||
//! `--effectcraft-helper` it is the helper, otherwise it runs the checks, starting itself as
|
||||
//! the helper. (`harness = false`: libtest would reject the helper argument.)
|
||||
|
||||
use std::io::Write;
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use effectcraft_effects::plugin::{EffectPlugin, PluginFrame, PluginHost, PluginImage, PluginManifest, PluginParam, PluginParamKind, PluginParams, plugin};
|
||||
use effectcraft_helper_host::protocol::{HELPER_ARG, Reply, reply_line};
|
||||
use effectcraft_helper_host::serve::{Provider, serve};
|
||||
|
||||
fn manifest(id: &str, params: Vec<PluginParam>) -> PluginManifest {
|
||||
PluginManifest {
|
||||
api: 2,
|
||||
id: id.into(),
|
||||
name: id.rsplit('.').next().unwrap_or(id).into(),
|
||||
category: "Helper Test".into(),
|
||||
version: "1".into(),
|
||||
author: String::new(),
|
||||
description: String::new(),
|
||||
params,
|
||||
}
|
||||
}
|
||||
|
||||
fn amount() -> PluginParam {
|
||||
PluginParam {
|
||||
id: "amount".into(),
|
||||
name: "Amount".into(),
|
||||
group: String::new(),
|
||||
kind: PluginParamKind::Slider { default: 1.0, min: 0.0, max: 1.0, slider_min: None, slider_max: None, decimals: 2, integer: false },
|
||||
}
|
||||
}
|
||||
|
||||
/// What each fake effect does.
|
||||
#[derive(Clone, Copy, PartialEq)]
|
||||
enum Kind {
|
||||
/// RGB → amount·(1 − RGB).
|
||||
Invert,
|
||||
/// Asks for 3 pixels of padding; fills the frame with its padding as red.
|
||||
Pad,
|
||||
/// Replaces the frame with the input one frame earlier, and puts `amount` from
|
||||
/// `params_at(0)` in alpha of pixel 0.
|
||||
Echo,
|
||||
/// Exits the process mid-render.
|
||||
Crash,
|
||||
/// Asks for the parameters before 0 (an answer far larger than a pipe holds), then never
|
||||
/// reads it or answers.
|
||||
Hang,
|
||||
/// Writes a long line of bytes that aren't UTF-8 to stderr, then more, then inverts.
|
||||
Noisy,
|
||||
}
|
||||
|
||||
struct Fake {
|
||||
m: PluginManifest,
|
||||
kind: Kind,
|
||||
}
|
||||
|
||||
impl EffectPlugin for Fake {
|
||||
fn manifest(&self) -> &PluginManifest {
|
||||
&self.m
|
||||
}
|
||||
fn render(&self, _: &mut PluginFrame, _: &PluginParams, _: f64) -> Result<(), String> {
|
||||
Err("needs render_v2".into())
|
||||
}
|
||||
fn padding(&self, _: &PluginParams, _: f64, _: f64, _: [f64; 2], _: u64) -> u32 {
|
||||
if self.kind == Kind::Pad { 3 } else { 0 }
|
||||
}
|
||||
fn render_v2(&self, frame: &mut PluginFrame, params: &PluginParams, _time: f64, host: &dyn PluginHost) -> Result<(), String> {
|
||||
match self.kind {
|
||||
Kind::Invert => {
|
||||
let a = params.f("amount") as f32;
|
||||
for p in frame.pixels.iter_mut() {
|
||||
for c in &mut p[..3] {
|
||||
*c = a * (1.0 - *c);
|
||||
}
|
||||
}
|
||||
}
|
||||
Kind::Pad => frame.pixels.iter_mut().for_each(|p| *p = [3.0, 0.0, 0.0, 1.0]),
|
||||
Kind::Echo => {
|
||||
let prev = host.input_at(host.time() - 1.0 / host.frame_rate()).ok_or("no input at the previous frame")?;
|
||||
if prev.pixels.len() != frame.pixels.len() {
|
||||
return Err("previous frame has another size".into());
|
||||
}
|
||||
frame.pixels.copy_from_slice(&prev.pixels);
|
||||
let (flat, _) = host.params_at(0.0).ok_or("no params at 0")?;
|
||||
if let (Some(p), Some(a)) = (frame.pixels.first_mut(), flat.first()) {
|
||||
p[3] = *a as f32;
|
||||
}
|
||||
}
|
||||
Kind::Crash => std::process::exit(3),
|
||||
Kind::Hang => {
|
||||
let mut out = std::io::stdout();
|
||||
out.write_all(reply_line(&Reply::ParamsAt { time: -1.0 })?.as_bytes()).and_then(|_| out.flush()).map_err(|e| e.to_string())?;
|
||||
std::thread::sleep(Duration::from_secs(3600));
|
||||
}
|
||||
Kind::Noisy => {
|
||||
let mut err = std::io::stderr();
|
||||
let mut junk = vec![0xffu8; 1 << 20];
|
||||
junk.push(b'\n');
|
||||
err.write_all(&junk).map_err(|e| format!("stderr: {e}"))?;
|
||||
// EffectCraft must still be reading.
|
||||
std::thread::sleep(Duration::from_millis(300));
|
||||
err.write_all(b"still talking\n").map_err(|e| format!("stderr: {e}"))?;
|
||||
frame.pixels.iter_mut().for_each(|p| *p = [1.0, 0.0, 0.0, 1.0]);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
struct FakeProvider;
|
||||
|
||||
impl Provider for FakeProvider {
|
||||
fn name(&self) -> String {
|
||||
"Fake".into()
|
||||
}
|
||||
fn effects(&mut self) -> (Vec<Arc<dyn EffectPlugin>>, Vec<String>) {
|
||||
let v: Vec<Arc<dyn EffectPlugin>> =
|
||||
[("invert", Kind::Invert), ("pad", Kind::Pad), ("echo", Kind::Echo), ("crash", Kind::Crash), ("hang", Kind::Hang), ("noisy", Kind::Noisy)]
|
||||
.into_iter()
|
||||
.map(|(n, kind)| Arc::new(Fake { m: manifest(&format!("org.test.helper.{n}"), vec![amount()]), kind }) as Arc<dyn EffectPlugin>)
|
||||
.collect();
|
||||
(v, vec!["fake helper: a warning".into()])
|
||||
}
|
||||
}
|
||||
|
||||
/// The host side of a render in these checks: the input at any time is a flat grey of that
|
||||
/// time; with `nest`, asking for the input renders `invert` through the same helper first.
|
||||
struct TestHost {
|
||||
w: u32,
|
||||
h: u32,
|
||||
nest: bool,
|
||||
}
|
||||
|
||||
impl PluginHost for TestHost {
|
||||
fn input_at(&self, t: f64) -> Option<PluginImage> {
|
||||
let mut pixels = vec![[t as f32, t as f32, t as f32, 1.0]; (self.w * self.h) as usize];
|
||||
if self.nest {
|
||||
let inv = plugin("org.test.helper.invert")?;
|
||||
let m = inv.manifest().clone();
|
||||
let mut f =
|
||||
PluginFrame { width: self.w, height: self.h, pixels: &mut pixels, scale: 1.0, origin: [0.0; 2], layer_size: [self.w as f64, self.h as f64] };
|
||||
inv.render_v2(&mut f, &PluginParams { manifest: &m, flat: &[1.0], texts: &[] }, t, &TestHost { nest: false, ..*self }).ok()?;
|
||||
}
|
||||
Some(PluginImage { width: self.w, height: self.h, pixels, origin: [0.0; 2], scale: 1.0 })
|
||||
}
|
||||
fn layer_input(&self, _: &str, _: f64) -> Option<PluginImage> {
|
||||
None
|
||||
}
|
||||
fn params_at(&self, t: f64) -> Option<(Vec<f64>, Vec<String>)> {
|
||||
// Before 0: megabytes, far more than a pipe holds (see `Kind::Hang`).
|
||||
if t < 0.0 {
|
||||
return Some((vec![0.5; 1 << 20], vec![]));
|
||||
}
|
||||
Some((vec![0.25], vec![]))
|
||||
}
|
||||
fn frame_rate(&self) -> f64 {
|
||||
10.0
|
||||
}
|
||||
fn time(&self) -> f64 {
|
||||
1.0
|
||||
}
|
||||
fn instance_key(&self) -> u64 {
|
||||
7
|
||||
}
|
||||
}
|
||||
|
||||
fn render(id: &str, pixels: &mut Vec<[f32; 4]>, w: u32, h: u32, amount: f64, nest: bool) -> Result<(), String> {
|
||||
let p = plugin(id).ok_or_else(|| format!("{id} is not registered"))?;
|
||||
let m = p.manifest().clone();
|
||||
let params = PluginParams { manifest: &m, flat: &[amount], texts: &[] };
|
||||
let host = TestHost { w, h, nest };
|
||||
let pad = p.padding(¶ms, 1.0, 1.0, [w as f64, h as f64], 7);
|
||||
let (w, h) = (w + 2 * pad, h + 2 * pad);
|
||||
pixels.resize((w * h) as usize, [0.0; 4]);
|
||||
let mut f = PluginFrame { width: w, height: h, pixels, scale: 1.0, origin: [pad as f64; 2], layer_size: [w as f64, h as f64] };
|
||||
p.render_v2(&mut f, ¶ms, 1.0, &host)
|
||||
}
|
||||
|
||||
fn check(name: &str, ok: bool, detail: impl std::fmt::Debug) -> bool {
|
||||
println!("{} {name}{}", if ok { "ok " } else { "FAIL" }, if ok { String::new() } else { format!(": {detail:?}") });
|
||||
ok
|
||||
}
|
||||
|
||||
fn run_checks() -> bool {
|
||||
let exe = match std::env::current_exe() {
|
||||
Ok(e) => e,
|
||||
Err(e) => return check("current exe", false, e),
|
||||
};
|
||||
let state = std::env::temp_dir().join(format!("ec-fake-helper-state-{}", std::process::id()));
|
||||
effectcraft_helper_host::blocklist::set_state_dir(state.clone());
|
||||
let mut all = true;
|
||||
|
||||
// Discovery + describe.
|
||||
let r = effectcraft_helper_host::scan_paths(std::slice::from_ref(&exe), false);
|
||||
let loaded = r["loaded"].as_array().map_or(0, Vec::len);
|
||||
all &= check("scan registers the helper's effects", loaded == 6, &r);
|
||||
all &= check("helper warnings are reported", r["errors"].to_string().contains("a warning"), &r);
|
||||
all &= check(
|
||||
"a second scan doesn't start it twice",
|
||||
effectcraft_helper_host::scan_paths(std::slice::from_ref(&exe), false)["helpers"].as_array().is_some_and(Vec::is_empty),
|
||||
"",
|
||||
);
|
||||
let list = effectcraft_helper_host::list();
|
||||
all &= check("list names the helper", list["helpers"][0]["name"] == "Fake" && list["helpers"][0]["running"] == true, &list);
|
||||
|
||||
// Plain render through the exchange file.
|
||||
let mut px = vec![[0.25, 0.5, 1.0, 1.0]; 6];
|
||||
let r = render("org.test.helper.invert", &mut px, 3, 2, 1.0, false);
|
||||
all &= check("invert renders", r.is_ok() && px.iter().all(|p| (p[0] - 0.75).abs() < 1e-6 && p[2].abs() < 1e-6 && p[3] == 1.0), (&r, &px));
|
||||
|
||||
// Output on stderr that isn't text, in long lines, is drained, so the helper never blocks on it
|
||||
// or loses its stderr.
|
||||
let mut px = vec![[0.0; 4]; 4];
|
||||
let r = render("org.test.helper.noisy", &mut px, 2, 2, 1.0, false);
|
||||
all &= check("a helper's stderr is drained whatever it holds", r.is_ok() && px[0] == [1.0, 0.0, 0.0, 1.0], &r);
|
||||
|
||||
// Padding.
|
||||
let mut px = vec![[0.0; 4]; 4];
|
||||
let r = render("org.test.helper.pad", &mut px, 2, 2, 1.0, false);
|
||||
all &= check("padding grows the frame", r.is_ok() && px.len() == 64 && px[0][0] == 3.0, (&r, px.len()));
|
||||
|
||||
// Callbacks: the input at another time and parameters at another time.
|
||||
let mut px = vec![[0.0; 4]; 4];
|
||||
let r = render("org.test.helper.echo", &mut px, 2, 2, 1.0, false);
|
||||
all &= check("input_at / params_at round-trip", r.is_ok() && (px[1][0] - 0.9).abs() < 1e-6 && px[0][3] == 0.25, (&r, &px));
|
||||
|
||||
// Nested: answering input_at renders another effect of the same helper.
|
||||
let mut px = vec![[0.0; 4]; 4];
|
||||
let r = render("org.test.helper.echo", &mut px, 2, 2, 1.0, true);
|
||||
all &= check("nested renders through one helper", r.is_ok() && (px[1][0] - 0.1).abs() < 1e-5, (&r, &px));
|
||||
|
||||
// Several threads at once share the helper.
|
||||
let threads: Vec<_> = (0..4)
|
||||
.map(|i| {
|
||||
std::thread::spawn(move || {
|
||||
let mut px = vec![[0.0, 0.0, 0.0, 1.0]; 16];
|
||||
render("org.test.helper.invert", &mut px, 4, 4, 0.5, false).map(|_| (i, px[0][0]))
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
let results: Vec<_> = threads.into_iter().map(|t| t.join().map_err(|_| "thread panicked".to_string()).and_then(|r| r)).collect();
|
||||
all &= check("concurrent renders", results.iter().all(|r| matches!(r, Ok((_, v)) if (*v - 0.5).abs() < 1e-6)), &results);
|
||||
|
||||
// A crash mid-render is an error, the frame is untouched, and the helper comes back.
|
||||
let mut px = vec![[0.5; 4]; 4];
|
||||
let r = render("org.test.helper.crash", &mut px, 2, 2, 1.0, false);
|
||||
all &= check("a crashing helper is an error", r.as_ref().is_err_and(|e| e.contains("helper")) && px.iter().all(|p| *p == [0.5; 4]), &r);
|
||||
let mut px = vec![[0.0, 0.0, 0.0, 1.0]; 4];
|
||||
let r = render("org.test.helper.invert", &mut px, 2, 2, 1.0, false);
|
||||
all &= check("the helper restarts for the next render", r.is_ok() && px[0][0] == 1.0, &r);
|
||||
// A render that starts the stopped helper itself (no padding request first) succeeds.
|
||||
effectcraft_helper_host::stop_all();
|
||||
let r = plugin("org.test.helper.invert").ok_or("invert is not registered").and_then(|p| {
|
||||
let m = p.manifest().clone();
|
||||
let mut px = vec![[0.0, 0.0, 0.0, 1.0]; 4];
|
||||
let mut f = PluginFrame { width: 2, height: 2, pixels: &mut px, scale: 1.0, origin: [0.0; 2], layer_size: [2.0; 2] };
|
||||
let host = TestHost { w: 2, h: 2, nest: false };
|
||||
p.render_v2(&mut f, &PluginParams { manifest: &m, flat: &[1.0], texts: &[] }, 1.0, &host).map_err(|_| "render failed")?;
|
||||
Ok(px[0][0])
|
||||
});
|
||||
all &= check("a stopped helper restarts for a render without padding", r == Ok(1.0), r);
|
||||
|
||||
// Effects that keep crashing it are turned off (and the list survives in the state dir).
|
||||
for _ in 0..3 {
|
||||
let _ = render("org.test.helper.crash", &mut vec![[0.0; 4]; 4], 2, 2, 1.0, false);
|
||||
}
|
||||
let r = render("org.test.helper.crash", &mut vec![[0.0; 4]; 4], 2, 2, 1.0, false);
|
||||
all &= check("repeat offenders are blocked", r.as_ref().is_err_and(|e| e.contains("turned off")), &r);
|
||||
all &= check("the block list is saved", std::fs::read_to_string(state.join("helper_blocklist.txt")).is_ok_and(|t| t.contains("org.test.helper.crash")), "");
|
||||
let r = effectcraft_helper_host::scan_paths(&[], true);
|
||||
all &= check("retry clears the block list", !effectcraft_helper_host::blocklist::is_blocked("org.test.helper.crash"), &r);
|
||||
|
||||
// A hang is cut off by the timeout, also while EffectCraft is writing to a helper that
|
||||
// stopped reading.
|
||||
let t0 = std::time::Instant::now();
|
||||
let r = render("org.test.helper.hang", &mut vec![[0.0; 4]; 4], 2, 2, 1.0, false);
|
||||
all &= check(
|
||||
"a hanging helper times out",
|
||||
r.as_ref().is_err_and(|e| e.contains("did not answer")) && t0.elapsed() < Duration::from_secs(30),
|
||||
(&r, t0.elapsed()),
|
||||
);
|
||||
|
||||
// The restarts above use up the helper's restarts for this minute, so it can't start again.
|
||||
// That is not the fault of the effect being rendered: it fails, but is not charged and never
|
||||
// turned off (it would be after CRASH_LIMIT charged failures).
|
||||
let starts: Vec<_> = (0..4).map(|_| render("org.test.helper.invert", &mut vec![[0.0; 4]; 4], 2, 2, 1.0, false)).collect();
|
||||
all &= check(
|
||||
"a helper that can't start doesn't block the effect",
|
||||
starts.iter().all(|r| r.as_ref().is_err_and(|e| e.contains("keeps stopping")))
|
||||
&& !effectcraft_helper_host::blocklist::is_blocked("org.test.helper.invert"),
|
||||
&starts,
|
||||
);
|
||||
|
||||
effectcraft_helper_host::stop_all();
|
||||
let _ = std::fs::remove_dir_all(&state);
|
||||
all
|
||||
}
|
||||
|
||||
/// The one test this binary reports to test runners.
|
||||
const TEST_NAME: &str = "helper_protocol";
|
||||
|
||||
fn main() {
|
||||
let args: Vec<String> = std::env::args().collect();
|
||||
if args.iter().any(|a| a == HELPER_ARG) {
|
||||
if let Err(e) = serve(&mut FakeProvider) {
|
||||
eprintln!("fake helper: {e}");
|
||||
std::process::exit(1);
|
||||
}
|
||||
return;
|
||||
}
|
||||
// The libtest listing protocol cargo-nextest uses (`--list --format terse`, with and without
|
||||
// `--ignored`): one `name: test` line, then nextest runs `<binary> helper_protocol --exact`.
|
||||
if args.iter().any(|a| a == "--list") {
|
||||
if !args.iter().any(|a| a == "--ignored") {
|
||||
println!("{TEST_NAME}: test");
|
||||
}
|
||||
return;
|
||||
}
|
||||
// Keep the hang check short.
|
||||
effectcraft_helper_host::client::set_render_timeout(Duration::from_secs(3));
|
||||
if !run_checks() {
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
@@ -18,10 +18,13 @@ effectcraft-expr = { workspace = true }
|
||||
effectcraft-script = { workspace = true }
|
||||
effectcraft-export = { workspace = true }
|
||||
effectcraft-plugin = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
|
||||
# WebAssembly effect plug-ins run in wasmi on native builds.
|
||||
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
|
||||
effectcraft-plugin = { workspace = true, features = ["wasm"] }
|
||||
# External effect helpers: separate processes, so their native code can't take the app down.
|
||||
effectcraft-helper-host = { workspace = true }
|
||||
|
||||
[dev-dependencies]
|
||||
serde_json = { workspace = true }
|
||||
|
||||
@@ -95,6 +95,7 @@ pub fn session() -> Session {
|
||||
expr: Some(Arc::new(effectcraft_expr::Expressions)),
|
||||
expr_check: Some(effectcraft_expr::check_syntax),
|
||||
plugin_loader: effectcraft_plugin::wasm_available().then_some(effectcraft_plugin::loader as effectcraft_engine::PluginLoader),
|
||||
helpers: helper_hooks(),
|
||||
plugin_previewer: effectcraft_plugin::wasm_available().then_some(effectcraft_plugin::preview as effectcraft_engine::PluginPreviewer),
|
||||
plugins_dir: config_dir().map(|d| d.join("Plug-ins")),
|
||||
..Default::default()
|
||||
@@ -103,6 +104,31 @@ pub fn session() -> Session {
|
||||
s
|
||||
}
|
||||
|
||||
/// External effect helpers run as child processes: native builds only.
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
fn helper_hooks() -> Option<effectcraft_engine::HelperHooks> {
|
||||
use effectcraft_helper_host as hh;
|
||||
// Effects that keep stopping their helper are remembered in the settings folder.
|
||||
if let Some(dir) = config_dir() {
|
||||
hh::blocklist::set_state_dir(dir);
|
||||
}
|
||||
fn scan(retry: bool) -> serde_json::Value {
|
||||
hh::scan_paths(&hh::default_search_paths(config_dir().as_deref()), retry)
|
||||
}
|
||||
Some(effectcraft_engine::HelperHooks { scan, load: hh::load_path, list: hh::list })
|
||||
}
|
||||
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
fn helper_hooks() -> Option<effectcraft_engine::HelperHooks> {
|
||||
None
|
||||
}
|
||||
|
||||
/// Stop the external effect helpers (when the app quits).
|
||||
pub fn stop_helpers() {
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
effectcraft_helper_host::stop_all();
|
||||
}
|
||||
|
||||
/// The platform config directory for EffectCraft (`EFFECTCRAFT_CONFIG_DIR` overrides):
|
||||
/// `~/Library/Application Support/EffectCraft` (macOS), `%APPDATA%\EffectCraft` (Windows),
|
||||
/// `$XDG_CONFIG_HOME/effectcraft` or `~/.config/effectcraft` (Linux and others). A `portable.txt`
|
||||
|
||||
@@ -49,7 +49,7 @@ fn rust_plugins_apply_by_id() {
|
||||
version: "1".into(),
|
||||
author: String::new(),
|
||||
description: String::new(),
|
||||
params: vec![PluginParam { id: "on".into(), name: "On".into(), kind: PluginParamKind::Checkbox { default: true } }],
|
||||
params: vec![PluginParam { id: "on".into(), name: "On".into(), kind: PluginParamKind::Checkbox { default: true }, group: String::new() }],
|
||||
};
|
||||
register_plugin(Arc::new(Swap { m })).unwrap();
|
||||
let (mut s, l) = setup();
|
||||
@@ -62,7 +62,7 @@ fn rust_plugins_apply_by_id() {
|
||||
assert!(p[0] > 0.99 && p[2] < 0.01, "{p:?}");
|
||||
let list = s.execute("effect.plugins.list", json!({})).unwrap();
|
||||
assert!(list["plugins"].as_array().unwrap().iter().any(|p| p["id"] == "org.test.swap" && p["category"] == "Channel"));
|
||||
assert_eq!(list["api"], 1);
|
||||
assert_eq!(list["api"], 2);
|
||||
assert!(s.execute("effect.list", json!({"filter": "swap red"})).unwrap().as_array().unwrap().len() == 1);
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
//! # effectcraft-plugin (L3)
|
||||
//!
|
||||
//! Loads **WebAssembly effect plug-ins** (plug-in API v1, see `docs/plugins.md`) into the effect
|
||||
//! Loads **WebAssembly effect plug-ins** (plug-in ABI v1, see `docs/plugins.md`; the native
|
||||
//! plug-in API 2 additions such as layer parameters and host services are not part of it) into the effect
|
||||
//! registry ([`effectcraft_effects::plugin`]). A plug-in is a `.wasm` (or `.wat`) module with
|
||||
//! no imports that exports:
|
||||
//!
|
||||
@@ -25,7 +26,7 @@
|
||||
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
|
||||
|
||||
use effectcraft_effects::EffectSpec;
|
||||
pub use effectcraft_effects::plugin::{EffectPlugin, PLUGIN_API_VERSION, PluginFrame, PluginManifest, PluginParams, register_plugin};
|
||||
pub use effectcraft_effects::plugin::{EffectPlugin, PLUGIN_API_VERSION, PluginFrame, PluginManifest, PluginParams, WASM_API_VERSION, register_plugin};
|
||||
|
||||
#[cfg(feature = "wasm")]
|
||||
mod wasm;
|
||||
@@ -59,7 +60,7 @@ pub fn loader(bytes: &[u8], source: &str) -> Result<serde_json::Value, String> {
|
||||
"name": spec.name,
|
||||
"category": spec.category,
|
||||
"version": m.as_ref().map(|m| m.version.clone()).unwrap_or_default(),
|
||||
"api": PLUGIN_API_VERSION,
|
||||
"api": m.as_ref().map(|m| m.api).unwrap_or(PLUGIN_API_VERSION),
|
||||
"params": spec.params.len(),
|
||||
}))
|
||||
}
|
||||
@@ -82,13 +83,17 @@ pub fn preview(bytes: &[u8], width: u32, height: u32, pixels: &[[f32; 4]], times
|
||||
// Points default to fractions of the layer: the frame is the layer.
|
||||
PluginParamKind::Point { default } => flat.extend([default[0] * f64::from(width), default[1] * f64::from(height)]),
|
||||
PluginParamKind::Color { default } => flat.extend(default.iter().copied()),
|
||||
// API 2 kinds: a WebAssembly module can't declare them (`WasmPlugin::new` refuses them).
|
||||
PluginParamKind::Point3 { default } => flat.extend(default.iter().copied()),
|
||||
PluginParamKind::Layer => flat.push(-1.0),
|
||||
PluginParamKind::Text { .. } | PluginParamKind::Hidden { .. } => {}
|
||||
}
|
||||
}
|
||||
let params = PluginParams { manifest: m, flat: &flat };
|
||||
let params = PluginParams { manifest: m, flat: &flat, texts: &[] };
|
||||
let mut frames = Vec::with_capacity(times.len());
|
||||
for &t in times {
|
||||
let mut px = pixels.to_vec();
|
||||
let mut frame = PluginFrame { width, height, pixels: &mut px, scale: 1.0 };
|
||||
let mut frame = PluginFrame { width, height, pixels: &mut px, scale: 1.0, origin: [0.0; 2], layer_size: [f64::from(width), f64::from(height)] };
|
||||
p.render(&mut frame, ¶ms, t)?;
|
||||
frames.push(px);
|
||||
}
|
||||
|
||||
@@ -4,7 +4,7 @@ use std::sync::Mutex;
|
||||
|
||||
use wasmi::{Config, Engine, Instance, Linker, Memory, Module, Store, StoreLimits, StoreLimitsBuilder, TypedFunc, WasmParams, WasmResults};
|
||||
|
||||
use crate::{EffectPlugin, PLUGIN_API_VERSION, PluginFrame, PluginManifest, PluginParams};
|
||||
use crate::{EffectPlugin, PluginFrame, PluginManifest, PluginParams, WASM_API_VERSION};
|
||||
|
||||
/// Instruction budget per frame: a fixed allowance plus some per pixel. Generous for per-pixel
|
||||
/// work, but a runaway loop stops instead of hanging the render.
|
||||
@@ -74,8 +74,8 @@ impl WasmPlugin {
|
||||
let mut inst = instantiate(&engine, &module)?;
|
||||
let version = inst.func::<(), i32>("ec_api_version")?;
|
||||
let v = version.call(&mut inst.store, ()).map_err(err)?;
|
||||
if v != PLUGIN_API_VERSION as i32 {
|
||||
return Err(format!("the module implements plug-in API {v}, this build implements API {PLUGIN_API_VERSION}"));
|
||||
if u32::try_from(v).ok() != Some(WASM_API_VERSION) {
|
||||
return Err(format!("the module implements plug-in API {v}, this build implements API {WASM_API_VERSION} for WebAssembly"));
|
||||
}
|
||||
let ptr = inst.func::<(), i32>("ec_manifest_ptr")?.call(&mut inst.store, ()).map_err(err)?;
|
||||
let len = inst.func::<(), i32>("ec_manifest_len")?.call(&mut inst.store, ()).map_err(err)?;
|
||||
@@ -86,6 +86,9 @@ impl WasmPlugin {
|
||||
inst.memory.read(&inst.store, ptr as u32 as usize, &mut text).map_err(|e| format!("manifest out of bounds: {e}"))?;
|
||||
let manifest: PluginManifest = serde_json::from_slice(&text).map_err(|e| format!("bad manifest JSON: {e}"))?;
|
||||
manifest.validate()?;
|
||||
if let Some(p) = manifest.params.iter().find(|p| p.kind.needs_v2()) {
|
||||
return Err(format!("parameter `{}`: WebAssembly plug-ins use plug-in API 1 parameter types (no point3, layer, text or hidden)", p.id));
|
||||
}
|
||||
Ok(WasmPlugin { manifest, source: source.to_string(), engine, module, pool: Mutex::new(vec![inst]) })
|
||||
}
|
||||
|
||||
|
||||
@@ -196,10 +196,11 @@ pub fn is_audio_effect(id: &str) -> bool {
|
||||
/// itself): the time effects (Echo, Posterize Time, Timewarp…), simulations, temporal smoothing
|
||||
/// and analysis (Remove Grain, Auto Levels, Color Stabilizer's reference frame, Warp Stabilizer,
|
||||
/// Roto Brush…) and Clone strokes with a time offset? On a precomp layer, a change at one nested
|
||||
/// time then reaches frames of the layer far from where it shows.
|
||||
/// time then reaches frames of the layer far from where it shows. Plug-in effects (every id
|
||||
/// outside `ec.`) count too: plug-in API 2 reads the input at other times (`PluginHost`).
|
||||
// ponytail: hand-kept list of the effects that call `self_at`; a new one must be added here.
|
||||
pub fn reads_other_frames(id: &str) -> bool {
|
||||
id.starts_with("ec.time.") || id.starts_with("ec.sim.") || READS_OTHER_FRAMES.contains(&id)
|
||||
!id.starts_with("ec.") || id.starts_with("ec.time.") || id.starts_with("ec.sim.") || READS_OTHER_FRAMES.contains(&id)
|
||||
}
|
||||
|
||||
/// The effects outside `ec.time.` and `ec.sim.` that read their own layer at other times.
|
||||
|
||||
@@ -482,3 +482,12 @@ fn silent_layers_are_music_and_unread_hidden_layers() {
|
||||
assert_eq!(q.silent_layers(cid, q.comp(cid).unwrap(), &q.heard_layers()).into_iter().collect::<Vec<LayerId>>(), vec![rest]);
|
||||
}
|
||||
}
|
||||
|
||||
/// Plug-in effects read their input at other times (plug-in API 2), so a precomp under one is
|
||||
/// re-rendered over its whole span; built-in effects only when they are on the list.
|
||||
#[test]
|
||||
fn plug_in_effects_read_other_frames() {
|
||||
assert!(crate::reads_other_frames("org.example.glow"));
|
||||
assert!(crate::reads_other_frames("ec.time.echo"));
|
||||
assert!(!crate::reads_other_frames("ec.blur.gaussian"));
|
||||
}
|
||||
@@ -384,6 +384,24 @@ fn effect_bounds_of(comp: &Comp, size: (u32, u32)) -> ([f64; 2], [f64; 2]) {
|
||||
}
|
||||
}
|
||||
|
||||
/// The layer's source time range in layer time seconds as (first, end) for
|
||||
/// [`effectcraft_effects::EffectEnv::layer_span`]: footage and nested comps last their item's
|
||||
/// duration from layer time 0. With Time Remap layer time is no longer source time, so the layer's
|
||||
/// visible span (in to out point, in layer time) is reported instead. `None` for sources without
|
||||
/// a duration (stills, solids, text, shapes...).
|
||||
pub fn layer_span(project: &Project, layer: &Layer) -> Option<(f64, f64)> {
|
||||
let span = if layer.props.get("timeRemap").is_some() {
|
||||
let (a, b) = (layer.layer_time(layer.in_point).seconds(), layer.layer_time(layer.out_point).seconds());
|
||||
(a.min(b), a.max(b))
|
||||
} else {
|
||||
match &layer.source {
|
||||
LayerSource::Footage { item } | LayerSource::Comp { item } => (0.0, project.item(*item)?.duration()?.seconds()),
|
||||
_ => return None,
|
||||
}
|
||||
};
|
||||
(span.0.is_finite() && span.1.is_finite() && span.1 > span.0).then_some(span)
|
||||
}
|
||||
|
||||
/// Size of a layer's source in layer pixels.
|
||||
pub fn source_size(project: &Project, layer: &Layer) -> (u32, u32) {
|
||||
match &layer.source {
|
||||
|
||||
@@ -615,6 +615,7 @@ impl<'a> Renderer<'a> {
|
||||
working_linear: self.pipe.linear,
|
||||
float_depth: self.pipe.levels.is_none(),
|
||||
shutter: self.mb_on(ctx, layer).then_some((ctx.comp.shutter_angle, ctx.comp.shutter_phase, ctx.comp.motion_blur_samples)),
|
||||
layer_span: crate::eval::layer_span(self.project, layer),
|
||||
};
|
||||
// Video effects in stack order (index, group, spec); disabled and audio effects skipped.
|
||||
let stack: Vec<(usize, &effectcraft_project::PropGroup, &'static effectcraft_effects::EffectSpec)> = fx
|
||||
|
||||
@@ -63,3 +63,24 @@ fn time_remap_picks_the_source_frame() {
|
||||
assert!(img.get(90, 50)[3] > 0.99, "frozen at the end position");
|
||||
assert!(img.get(10, 50)[3] < 0.01);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn layer_span_is_the_source_duration_in_layer_time() {
|
||||
let mut p = Project::default();
|
||||
let outer = Comp::new(100, 100, FrameRate::FPS_30, s(4.0));
|
||||
let inner = Comp::new(100, 100, FrameRate::FPS_30, s(2.5));
|
||||
let iid = p.add_item("Inner", Label::Sandstone, None, ItemKind::Comp(inner.into()));
|
||||
let sid = p.add_item("S", Label::Red, None, ItemKind::Solid(Solid { color: [1.0, 1.0, 1.0], width: 10, height: 10, pixel_aspect: 1.0 }));
|
||||
let mut pre = build::layer(&mut p, &outer, "Inner", LayerSource::Comp { item: iid }, (100, 100), None);
|
||||
let sol = build::layer(&mut p, &outer, "S", LayerSource::Solid { item: sid }, (10, 10), None);
|
||||
let (first, end) = crate::eval::layer_span(&p, &pre).expect("a nested comp has a duration");
|
||||
assert!(first.abs() < 1e-9 && (end - 2.5).abs() < 1e-9);
|
||||
assert_eq!(crate::eval::layer_span(&p, &sol), None, "solids have no duration");
|
||||
// Time Remap: layer time is not source time any more, the visible span is reported.
|
||||
let mut next = p.next_id;
|
||||
let tr = Ids(&mut next).prop("timeRemap", "Time Remap", Value::Scalar(0.0));
|
||||
p.next_id = next;
|
||||
pre.props.children.push(Node::Prop(tr));
|
||||
let (first, end) = crate::eval::layer_span(&p, &pre).expect("a remapped layer has a visible span");
|
||||
assert!(end > first);
|
||||
}
|
||||
@@ -874,6 +874,8 @@ const SIMPLIFIED_CHINESE: &[(&str, &str, &str)] = &[
|
||||
("effect.removeAll", "Remove All", "全部移除"),
|
||||
("effect.manage", "Manage Effects...", "管理效果..."),
|
||||
("effect.plugins.load", "Load Effect Plug-in...", "加载效果插件..."),
|
||||
("effect.helpers.load", "Load Plug-ins Through Helper...", "通过辅助程序加载插件..."),
|
||||
("effect.helpers.scan", "Scan for Effect Helpers", "扫描效果辅助程序"),
|
||||
("", "Animation", "动画"),
|
||||
("anim.savePreset", "Save Animation Preset...", "保存动画预设..."),
|
||||
("anim.applyPreset", "Apply Animation Preset...", "将动画预设应用于..."),
|
||||
@@ -1509,6 +1511,8 @@ const SWEDISH: &[(&str, &str, &str)] = &[
|
||||
("effect.removeAll", "Remove All", "Ta bort alla"),
|
||||
("effect.manage", "Manage Effects...", "Hantera effekter..."),
|
||||
("effect.plugins.load", "Load Effect Plug-in...", "Läs in effektplugin..."),
|
||||
("effect.helpers.load", "Load Plug-ins Through Helper...", "Läs in plugin via hjälpprogram..."),
|
||||
("effect.helpers.scan", "Scan for Effect Helpers", "Sök efter effekthjälpprogram"),
|
||||
("", "Animation", "Animering"),
|
||||
("anim.savePreset", "Save Animation Preset...", "Spara animeringsförinställning..."),
|
||||
("anim.applyPreset", "Apply Animation Preset...", "Använd animeringsförinställning..."),
|
||||
@@ -2150,6 +2154,8 @@ const TRADITIONAL_CHINESE: &[(&str, &str, &str)] = &[
|
||||
("effect.removeAll", "Remove All", "全部移除"),
|
||||
("effect.manage", "Manage Effects...", "管理效果..."),
|
||||
("effect.plugins.load", "Load Effect Plug-in...", "載入效果外掛程式..."),
|
||||
("effect.helpers.load", "Load Plug-ins Through Helper...", "透過輔助程式載入外掛程式..."),
|
||||
("effect.helpers.scan", "Scan for Effect Helpers", "掃描效果輔助程式"),
|
||||
("", "Animation", "動畫"),
|
||||
("anim.savePreset", "Save Animation Preset...", "儲存動畫預設集..."),
|
||||
("anim.applyPreset", "Apply Animation Preset...", "套用動畫預設集..."),
|
||||
@@ -2786,6 +2792,8 @@ const JAPANESE: &[(&str, &str, &str)] = &[
|
||||
("effect.removeAll", "Remove All", "すべてを削除"),
|
||||
("effect.manage", "Manage Effects...", "エフェクトを管理..."),
|
||||
("effect.plugins.load", "Load Effect Plug-in...", "エフェクトプラグインを読み込み..."),
|
||||
("effect.helpers.load", "Load Plug-ins Through Helper...", "ヘルパー経由でプラグインを読み込み..."),
|
||||
("effect.helpers.scan", "Scan for Effect Helpers", "エフェクトヘルパーをスキャン"),
|
||||
("", "Animation", "アニメーション"),
|
||||
("anim.savePreset", "Save Animation Preset...", "アニメーションプリセットを保存..."),
|
||||
("anim.applyPreset", "Apply Animation Preset...", "アニメーションプリセットを適用..."),
|
||||
@@ -3425,6 +3433,8 @@ const UKRAINIAN: &[(&str, &str, &str)] = &[
|
||||
("effect.removeAll", "Remove All", "Вилучити все"),
|
||||
("effect.manage", "Manage Effects...", "Керування ефектами..."),
|
||||
("effect.plugins.load", "Load Effect Plug-in...", "Завантажити плагін ефекту..."),
|
||||
("effect.helpers.load", "Load Plug-ins Through Helper...", "Завантажити плагіни через помічника..."),
|
||||
("effect.helpers.scan", "Scan for Effect Helpers", "Шукати помічників ефектів"),
|
||||
("", "Animation", "Анімація"),
|
||||
("anim.savePreset", "Save Animation Preset...", "Зберегти набір анімації..."),
|
||||
("anim.applyPreset", "Apply Animation Preset...", "Застосувати набір анімації..."),
|
||||
@@ -4065,6 +4075,8 @@ const SPANISH: &[(&str, &str, &str)] = &[
|
||||
("effect.removeAll", "Remove All", "Quitar todo"),
|
||||
("effect.manage", "Manage Effects...", "Administrar efectos..."),
|
||||
("effect.plugins.load", "Load Effect Plug-in...", "Cargar complemento de efecto..."),
|
||||
("effect.helpers.load", "Load Plug-ins Through Helper...", "Cargar complementos mediante asistente..."),
|
||||
("effect.helpers.scan", "Scan for Effect Helpers", "Buscar asistentes de efectos"),
|
||||
("", "Animation", "Animación"),
|
||||
("anim.savePreset", "Save Animation Preset...", "Guardar ajuste preestablecido de animación..."),
|
||||
("anim.applyPreset", "Apply Animation Preset...", "Aplicar ajuste preestablecido de animación..."),
|
||||
@@ -4703,6 +4715,8 @@ const PORTUGUESE: &[(&str, &str, &str)] = &[
|
||||
("effect.removeAll", "Remove All", "Remover todos"),
|
||||
("effect.manage", "Manage Effects...", "Gerenciar efeitos..."),
|
||||
("effect.plugins.load", "Load Effect Plug-in...", "Carregar plug-in de efeito..."),
|
||||
("effect.helpers.load", "Load Plug-ins Through Helper...", "Carregar plug-ins por meio do auxiliar..."),
|
||||
("effect.helpers.scan", "Scan for Effect Helpers", "Procurar auxiliares de efeitos"),
|
||||
("", "Animation", "Animação"),
|
||||
("anim.savePreset", "Save Animation Preset...", "Salvar predefinição de animação..."),
|
||||
("anim.applyPreset", "Apply Animation Preset...", "Aplicar predefinição de animação..."),
|
||||
@@ -5348,6 +5362,8 @@ const RUSSIAN: &[(&str, &str, &str)] = &[
|
||||
("effect.removeAll", "Remove All", "Удалить все"),
|
||||
("effect.manage", "Manage Effects...", "Управление эффектами..."),
|
||||
("effect.plugins.load", "Load Effect Plug-in...", "Загрузить плагин эффектов..."),
|
||||
("effect.helpers.load", "Load Plug-ins Through Helper...", "Загрузить плагины через помощника..."),
|
||||
("effect.helpers.scan", "Scan for Effect Helpers", "Найти помощников эффектов"),
|
||||
("", "Animation", "Анимация"),
|
||||
("anim.savePreset", "Save Animation Preset...", "Сохранить заготовку анимации..."),
|
||||
("anim.applyPreset", "Apply Animation Preset...", "Применить заготовку анимации..."),
|
||||
@@ -5986,6 +6002,8 @@ const FRENCH: &[(&str, &str, &str)] = &[
|
||||
("effect.removeAll", "Remove All", "Tout supprimer"),
|
||||
("effect.manage", "Manage Effects...", "Gérer les effets..."),
|
||||
("effect.plugins.load", "Load Effect Plug-in...", "Charger un plug-in d'effet..."),
|
||||
("effect.helpers.load", "Load Plug-ins Through Helper...", "Charger des plug-ins via un assistant..."),
|
||||
("effect.helpers.scan", "Scan for Effect Helpers", "Rechercher les assistants d'effets"),
|
||||
("", "Animation", "Animation"),
|
||||
("anim.savePreset", "Save Animation Preset...", "Enregistrer une animation prédéfinie..."),
|
||||
("anim.applyPreset", "Apply Animation Preset...", "Appliquer une animation prédéfinie..."),
|
||||
@@ -6624,6 +6642,8 @@ const GERMAN: &[(&str, &str, &str)] = &[
|
||||
("effect.removeAll", "Remove All", "Alle entfernen"),
|
||||
("effect.manage", "Manage Effects...", "Effekte verwalten..."),
|
||||
("effect.plugins.load", "Load Effect Plug-in...", "Effekt-Plug-in laden..."),
|
||||
("effect.helpers.load", "Load Plug-ins Through Helper...", "Plug-ins über Hilfsprogramm laden..."),
|
||||
("effect.helpers.scan", "Scan for Effect Helpers", "Nach Effekt-Hilfsprogrammen suchen"),
|
||||
("", "Animation", "Animation"),
|
||||
("anim.savePreset", "Save Animation Preset...", "Animationsvorgabe speichern..."),
|
||||
("anim.applyPreset", "Apply Animation Preset...", "Animationsvorgabe anwenden..."),
|
||||
@@ -7261,6 +7281,8 @@ const POLISH: &[(&str, &str, &str)] = &[
|
||||
("effect.removeAll", "Remove All", "Usuń wszystkie"),
|
||||
("effect.manage", "Manage Effects...", "Zarządzaj efektami..."),
|
||||
("effect.plugins.load", "Load Effect Plug-in...", "Wczytaj wtyczkę efektu..."),
|
||||
("effect.helpers.load", "Load Plug-ins Through Helper...", "Wczytaj wtyczki przez program pomocniczy..."),
|
||||
("effect.helpers.scan", "Scan for Effect Helpers", "Wyszukaj programy pomocnicze efektów"),
|
||||
("", "Animation", "Animacja"),
|
||||
("anim.savePreset", "Save Animation Preset...", "Zapisz ustawienie animacji..."),
|
||||
("anim.applyPreset", "Apply Animation Preset...", "Zastosuj ustawienie animacji..."),
|
||||
@@ -7898,6 +7920,8 @@ const CZECH: &[(&str, &str, &str)] = &[
|
||||
("effect.removeAll", "Remove All", "Odstranit vše"),
|
||||
("effect.manage", "Manage Effects...", "Spravovat efekty..."),
|
||||
("effect.plugins.load", "Load Effect Plug-in...", "Načíst zásuvný modul efektu..."),
|
||||
("effect.helpers.load", "Load Plug-ins Through Helper...", "Načíst zásuvné moduly přes pomocný program..."),
|
||||
("effect.helpers.scan", "Scan for Effect Helpers", "Vyhledat pomocné programy efektů"),
|
||||
("", "Animation", "Animace"),
|
||||
("anim.savePreset", "Save Animation Preset...", "Uložit přednastavení animace..."),
|
||||
("anim.applyPreset", "Apply Animation Preset...", "Použít přednastavení animace..."),
|
||||
@@ -8532,6 +8556,8 @@ const GEORGIAN: &[(&str, &str, &str)] = &[
|
||||
("effect.removeAll", "Remove All", "ყველას წაშლა"),
|
||||
("effect.manage", "Manage Effects...", "ეფექტების მართვა..."),
|
||||
("effect.plugins.load", "Load Effect Plug-in...", "ეფექტის პლაგინის ჩატვირთვა..."),
|
||||
("effect.helpers.load", "Load Plug-ins Through Helper...", "პლაგინების ჩატვირთვა დამხმარე პროგრამით..."),
|
||||
("effect.helpers.scan", "Scan for Effect Helpers", "ეფექტის დამხმარე პროგრამების ძიება"),
|
||||
("", "Animation", "ანიმაცია"),
|
||||
("anim.savePreset", "Save Animation Preset...", "ანიმაციის პრესეტის შენახვა..."),
|
||||
("anim.applyPreset", "Apply Animation Preset...", "ანიმაციის პრესეტის გამოყენება..."),
|
||||
|
||||
@@ -583,6 +583,7 @@ pub(super) fn file_dialog(app: &mut EffectcraftApp, id: &str, params: &Value) ->
|
||||
"file.runScript" if params.get("name").is_none() && params.get("recent").is_none() => ("path", Ask::Open(&["jsx", "js", "jsonl", "json", "txt"])),
|
||||
"file.installScript" | "file.installScriptUIPanel" => ("path", Ask::Open(&["jsx", "js"])),
|
||||
"effect.plugins.load" if params.get("folder").is_none() => ("path", Ask::Open(&["wasm", "wat"])),
|
||||
"effect.helpers.load" => ("path", Ask::Folder),
|
||||
"file.replaceFootage" => ("path", Ask::Import),
|
||||
"file.relinkFootage" => ("folder", Ask::Folder),
|
||||
"file.collectFiles" => ("folder", Ask::Save("Collected Files".into())),
|
||||
|
||||
@@ -127,6 +127,17 @@ fn drop_payload(ui: &egui::Ui, rect: Rect, p: &egui::Painter, t: &Tokens, layer_
|
||||
}
|
||||
}
|
||||
|
||||
/// A plug-in effect whose last render failed (an effect helper that stopped, ...): a warning sign
|
||||
/// at `at`, after the effect's name, with the message on hover. The frame shows the input as is.
|
||||
fn render_error_mark(app: &mut EffectcraftApp, ui: &mut egui::Ui, bp: &egui::Painter, effect: &str, g: &PropGroup, at: egui::Pos2) {
|
||||
let Some(err) = effectcraft_engine::effects::plugin::plugin_error(effect) else { return };
|
||||
let wr = bp.text(at, Align2::LEFT_CENTER, "⚠", Tokens::semibold(12.0), app.tokens.warning);
|
||||
let resp = ui.interact(wr.expand(2.0), egui::Id::unique(("ec-fx-error", g.uid)), Sense::hover());
|
||||
app.auto.add(&format!("effectControls.effect.{}.error", g.uid), wr, &err);
|
||||
resp.on_hover_text(err);
|
||||
ui.ctx().request_repaint_after(std::time::Duration::from_millis(500));
|
||||
}
|
||||
|
||||
fn about_button(app: &mut EffectcraftApp, ui: &mut egui::Ui, bp: &egui::Painter, t: &Tokens, effect: &str, g: &PropGroup, r: Rect) {
|
||||
// About…: the effect's name and category (AE's About dialog).
|
||||
let about = Rect::from_min_size(pos2(r.max.x - 50.0, r.center().y - 9.0), vec2(44.0, 18.0));
|
||||
@@ -246,6 +257,7 @@ pub fn show(app: &mut EffectcraftApp, ui: &mut egui::Ui, rect: Rect) {
|
||||
} else {
|
||||
let name = bp.text(pos2(tw.max.x + 6.0, r.center().y), Align2::LEFT_CENTER, &shown, Tokens::semibold(12.0), t.text);
|
||||
app.auto.add(&format!("effectControls.effect.{}.name", g.uid), name, &g.name);
|
||||
render_error_mark(app, ui, &bp, effect, g, pos2(name.max.x + 8.0, r.center().y));
|
||||
}
|
||||
let rresp = ui.interact(reset, egui::Id::unique(("ec-reset", g.uid)), Sense::click());
|
||||
bp.text(reset.center(), Align2::CENTER_CENTER, crate::i18n::tr("Reset"), Tokens::ui(11.5), if rresp.hovered() { t.hot_text } else { t.text_dim });
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
# ADR 0001: External effect helpers
|
||||
|
||||
- Status: proposed
|
||||
- Date: 2026-10-11
|
||||
- Context: review of storytold/effectcraft#508 (an in-process OpenFX host)
|
||||
|
||||
## Context
|
||||
|
||||
People want to use effects that already exist as native plug-ins: OpenFX suites (Sapphire,
|
||||
ReelSmart Motion Blur, Twixtor, ...) and, later, other native formats. storytold/effectcraft#508
|
||||
hosted OpenFX in-process. That needs `unsafe` to call a C ABI and runs closed-source native
|
||||
code in EffectCraft's own process, which conflicts with two non-negotiables:
|
||||
|
||||
- **Never crash** (AGENTS.md): one null dereference in a third-party plug-in takes the whole
|
||||
app, and the user's unsaved work, down with it. `catch_unwind` can't catch a segfault, and a
|
||||
blocklist only helps after the first crash.
|
||||
- **No `unsafe`** in the workspace (`unsafe_code = "forbid"`).
|
||||
|
||||
## Decision
|
||||
|
||||
EffectCraft never loads native plug-in code into its own process. Native effects are served by
|
||||
**external effect helpers**: separate programs that EffectCraft starts as child processes and
|
||||
talks to over a small, versioned protocol. The core gains one generic, safe-Rust crate,
|
||||
`effectcraft-helper-host` (L3); format-specific hosts (OpenFX, ...) live in helper programs
|
||||
that are built, released and installed separately.
|
||||
|
||||
- **Crash containment.** The process boundary contains everything a helper does. A helper
|
||||
that exits, hangs past a timeout or breaks the protocol is stopped. The render that hit it
|
||||
fails with a message shown on the effect, the frame stays as it was, and the helper starts
|
||||
again for the next render. An effect that takes its helper down three times is turned off and
|
||||
remembered in `helper_blocklist.txt` until the user retries.
|
||||
- **Same effects surface.** Each remote effect is registered through the existing plug-in API 2
|
||||
(`EffectPlugin`), so the UI, the property tree, keyframes, expressions, scripting, the CLI, the
|
||||
control channel and MCP need nothing new.
|
||||
- **Protocol** (`docs/external-helpers.md`): JSON lines over the helper's stdin / stdout
|
||||
(`hello` → `describe` → `render` with callbacks → `done`, plus `padding`, `load`, `destroy`,
|
||||
`ping`, `shutdown`). Pixels go through a **frame exchange file** (in `/dev/shm` on Linux, the
|
||||
temp folder elsewhere) addressed by offsets, never through the pipe. Callbacks (`input_at`,
|
||||
`layer_input`, `params_at`) let temporal effects (motion blur, retiming) read other frames
|
||||
while rendering; nested renders through the same helper are supported.
|
||||
- **Discovery.** EffectCraft looks in `helpers/` next to the executable (and
|
||||
`Contents/Helpers` in a macOS bundle), `helpers/` in the settings folder, and
|
||||
`EFFECTCRAFT_HELPER_PATH`. Helper programs are named `effectcraft-helper-*`.
|
||||
- **Security.** Helpers are local programs the user installed, just like the plug-ins they host.
|
||||
EffectCraft runs only programs from those locations, passes them nothing beyond the protocol
|
||||
(no network, no project files), and runs no helper on the web build.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **In-process host behind a feature flag** (#508). Rejected: a crash is still a crash, and it
|
||||
needs `unsafe` in the workspace.
|
||||
- **`ipc-channel` for transport.** Its wire format and shared-memory handles are Rust-specific
|
||||
(and differ per OS), so a helper written in C++ against the OpenFX reference host couldn't
|
||||
speak it. The chosen transport, JSON lines plus a plain file, works from any language without
|
||||
a library.
|
||||
- **Memory-mapped shared memory on the EffectCraft side.** Mapping needs `unsafe` (or a crate
|
||||
whose safe API is unsound if the other process truncates the mapping). Positional reads and
|
||||
writes on a file in the page cache cost one copy per frame each way and stay safe. A helper is
|
||||
free to map the file.
|
||||
- **WebAssembly-compiled plug-ins.** Not available for closed-source native plug-ins.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Native effects cost one copy of the frame each way plus process round trips. That's fine for
|
||||
CPU plug-ins working on whole frames.
|
||||
- Renders through one helper are serialized (one process, one conversation at a time). A pool
|
||||
of helper processes is a possible later improvement.
|
||||
- Out of scope for the core: any OpenFX (or other native-format) code. The OpenFX helper is a
|
||||
separate program in its own repository (an example: [ec-extension-ofx](https://github.com/glumbis/ec-extension-ofx), unofficial) that reuses the
|
||||
existing Rust OpenFX host behind `serve()`. A C++ helper built on the OpenFX HostSupport
|
||||
library is an equally valid way to implement the same protocol.
|
||||
- Later: the ArtCraft launcher can install helpers into `helpers/` the same way it installs
|
||||
apps.
|
||||
@@ -44,6 +44,7 @@ cpal or muda. Everything in L0 to L4, the egui UI and the web app also build for
|
||||
| L3 | `aep` | After Effects projects in one crate (like `lottie`): `container` reads and writes the `.aep` container (RIFX `Egg!`) as a lossless chunk tree with a byte-identical round trip, bounded for hostile input, reads the `.aepx` XML form into the same tree, and has a typed view of items, layers, properties, effects, masks, shapes, text, markers, footage, cameras, lights and layer styles, from our own observations of generated files ([aep-format.md](aep-format.md)); `import` maps the typed view into a project (items, settings, compositions, every layer kind, Transform, keyframes, expressions, masks, text, shapes, cameras, lights, mattes, markers, time remapping, layer styles, the effects of the importer's table, footage) with a report of everything not imported ([aep-import.md](aep-import.md)) |
|
||||
| L3 | `lottie` | Lottie JSON / dotLottie import and export (layers, precomps, eased and spatial keyframes, shapes, masks, mattes) with a warnings list for what Lottie cannot express |
|
||||
| L3 | `plugin` | Effect plug-ins: loads sandboxed WebAssembly effect modules (plug-in API v1; the wasmi interpreter behind the `wasm` feature, on for native hosts) into the effect registry ([plugins.md](plugins.md)) |
|
||||
| L3 | `helper-host` | External effect helpers: finds helper programs, runs each as a child process and registers its effects (plug-in API 2) over a versioned pipe + shared-file protocol; a crashing helper is an error on the effect, never a crash ([external-helpers.md](external-helpers.md), [ADR 0001](adr/0001-external-effect-helpers.md)). Native only |
|
||||
| L4 | `engine` | `Session`: project, branching undo history, editor state, the command registry and menus, the ScriptUI window model (`scriptui`) |
|
||||
| L4 | `script` | Scripting: JavaScript (boa) with an After Effects-style object model (`app.project`, comps, layers, properties, render queue) and ScriptUI (dialogs, palettes, dockable panels; script-host threads so dialogs block `show()`), whose edits run engine commands; AE match names |
|
||||
| L4 | `host` | A fully wired `Session` (media, expressions, scripting, WebAssembly plug-ins, exporter) for the frontends |
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
# External effect helpers
|
||||
|
||||
An **external effect helper** is a separate program that offers effects to EffectCraft. Each
|
||||
helper runs as a child process and talks to EffectCraft over the protocol below. EffectCraft
|
||||
registers every effect a helper lists as an ordinary plug-in effect: it shows up in Effects &
|
||||
Presets, has normal properties (keyframes, expressions, Effect Controls) and renders through
|
||||
the helper. Helpers exist to run native code, such as an OpenFX plug-in host, **outside**
|
||||
EffectCraft's process. If a helper crashes or hangs, EffectCraft shows the error on the effect
|
||||
and carries on. See [ADR 0001](adr/0001-external-effect-helpers.md) for why.
|
||||
|
||||
## Using helpers
|
||||
|
||||
| What | How |
|
||||
|---|---|
|
||||
| Install | Put the helper program (`effectcraft-helper-<name>`, `.exe` on Windows) in `helpers/` next to EffectCraft, in `helpers/` in the [settings folder](preferences.md), or in a folder named in `EFFECTCRAFT_HELPER_PATH` (a path list; entries can be programs or folders). Subfolders one level down are searched too. |
|
||||
| Start | Helpers run native code, so nothing starts them unless the user asks: Effect ▸ Scan for Effect Helpers (`effect.helpers.scan`) starts them. To start them every time EffectCraft launches, opt in with `EFFECTCRAFT_HELPERS=1`. It is an environment variable, not a setting, so writing to the settings folder can't turn it on. Agents confined with `--automation-read-root` / `--automation-write-root` can't run `effect.helpers.scan` or `effect.helpers.load`, and scripts can run them only with Allow Scripts to Write Files and Access Network on. |
|
||||
| Load more plug-ins | Effect ▸ Load Plug-ins Through Helper… (`effect.helpers.load {path}`) hands a file or folder to the helpers that accept it (an OpenFX helper takes `.ofx` plug-ins and `.ofx.bundle` folders). |
|
||||
| Inspect | `effect.helpers.list` → `{helpers: [{program, name, version, loads, running, effects, restarts, error}], blocked}`; `effect.plugins.list` shows each effect's `source` (`helper: …`) and last render `error`. |
|
||||
| Errors | A failed render leaves the frame as it was and puts ⚠ after the effect's name in Effect Controls (hover for the message). The next render starts the helper again. An effect that stops its helper 3 times is turned off and listed in `helper_blocklist.txt` in the settings folder. `effect.helpers.scan {"retry": true}` turns it back on. A helper that stops 5 times in a minute stays off until the next scan. |
|
||||
| Timeouts | `EFFECTCRAFT_HELPER_TIMEOUT` (seconds per render step, default 120) and `EFFECTCRAFT_HELPER_LOAD_TIMEOUT` (start-up, describe, load; default 600). A helper idle for 30 s is pinged before it is used again. |
|
||||
|
||||
Helpers are not available in the web build.
|
||||
|
||||
## Protocol (version 1)
|
||||
|
||||
EffectCraft starts the helper with the single argument `--effectcraft-helper`, with stdin,
|
||||
stdout and stderr piped. The environment variable `EFFECTCRAFT_HELPER_STATE_DIR` names a folder
|
||||
where the helper may keep its own records (for example, plug-ins that crashed it while
|
||||
loading).
|
||||
|
||||
- **Requests** (EffectCraft → helper): one JSON object per line on stdin.
|
||||
- **Replies** (helper → EffectCraft): one JSON object per line on stdout, prefixed with
|
||||
`@ec-helper ` (with the space). Lines without the prefix, or text before it, are logged and
|
||||
otherwise ignored, so plug-ins that print to stdout don't break the protocol. stderr is
|
||||
logged (whatever bytes it holds).
|
||||
- Every object has an `"op"`. Lines are at most 256 MB.
|
||||
- The helper exits when stdin closes or on `shutdown`.
|
||||
|
||||
### Frame exchange file
|
||||
|
||||
Pixels never go through the pipe. EffectCraft creates a file (in `/dev/shm` on Linux when
|
||||
available, otherwise the temp folder) and names it in `hello`. Images are addressed by an
|
||||
**image ref**:
|
||||
|
||||
```json
|
||||
{"offset": 0, "width": 1920, "height": 1080, "origin": [0, 0], "scale": 1}
|
||||
```
|
||||
|
||||
The image is `width × height` pixels starting `offset` bytes into the file. Each pixel is
|
||||
premultiplied RGBA as four `f32`s in native byte order (16 bytes), rows top to bottom, and an
|
||||
image is at most 2²⁸ pixels. `origin` is the buffer position of layer pixel (0, 0); `scale` is
|
||||
buffer pixels per layer pixel. EffectCraft decides every offset, so the helper only reads and
|
||||
writes where it's told. On Unix, EffectCraft unlinks the file once the helper has opened it, so
|
||||
nothing is left behind if either side dies. Helpers may map the file instead of reading it.
|
||||
|
||||
### Messages
|
||||
|
||||
| Request | Reply | |
|
||||
|---|---|---|
|
||||
| `{"op":"hello","version":1,"frames":"<path>"}` | `{"op":"hello","version":1,"name":"OpenFX","helper_version":"1.0","loads":["ofx"]}` | First message. The helper opens `frames` read/write. `loads` lists the extensions `load` accepts. A different `version` means EffectCraft won't use the helper. |
|
||||
| `{"op":"describe"}` | `{"op":"effects","effects":[{"manifest":{…},"padding":true}],"errors":["…"]}` | `manifest` is a [plug-in API 2 manifest](plugins.md#plug-in-api-2-native-plug-ins) (id, name, category, params). `padding: true` means EffectCraft asks `padding` before each render. |
|
||||
| `{"op":"load","path":"…"}` | `effects` (only the new ones) or `error` | A file or folder the user picked. Replayed after a restart. |
|
||||
| `{"op":"padding","effect":"…","instance":7,"time":1.0,"scale":1.0,"layer_size":[w,h],"flat":[…],"texts":[…]}` | `{"op":"padding","pixels":3}` | Transparent pixels to add on each side before the render (at most 4096). |
|
||||
| `{"op":"render","effect":"…","instance":7,"time":1.0,"frame":<image ref>,"layer_size":[w,h],"flat":[…],"texts":[…],"frame_rate":30,"frame_range":[0,10]}` | callbacks, then `{"op":"done"}` or `{"op":"error","message":"…"}` | Process `frame` in place: read it, write the result to the same ref, reply `done`. `flat` / `texts` are the parameter values flattened by plug-in API 2 rules. `instance` identifies the applied effect (stable across frames). `frame_range` is the layer's source range in layer-time seconds, or `null`. |
|
||||
| `{"op":"destroy","instance":7}` | none | The instance is gone; free what it holds. |
|
||||
| `{"op":"ping"}` | `{"op":"pong"}` | Heartbeat. |
|
||||
| `{"op":"shutdown"}` | none | Exit. |
|
||||
|
||||
**Callbacks during a render.** Before it replies `done`, the helper may ask:
|
||||
|
||||
| Helper asks | EffectCraft answers |
|
||||
|---|---|
|
||||
| `{"op":"input_at","time":t}` | `{"op":"image","image":<image ref>\|null}`: the effect's input (the layer, its masks, the effects above) at layer time `t`, in its own extent. |
|
||||
| `{"op":"layer_input","param":"<layer param id>","time":t}` | `image`: that layer at layer time `t` with effects and masks, resampled into this frame's grid. |
|
||||
| `{"op":"params_at","time":t}` | `{"op":"params","values":[[flat…],[texts…]]\|null}` |
|
||||
|
||||
While a helper waits for an answer, EffectCraft may send another `render` (or `padding`). This
|
||||
happens when producing the answer needs an effect of the same helper: for example, motion blur
|
||||
asking for the previous frame of a layer that also has a retime effect from the same helper.
|
||||
The helper must serve it completely (with its own callbacks, nested up to 16 deep) and then
|
||||
keep waiting. Images for callbacks are placed after the frame in the exchange file and freed
|
||||
when the render ends.
|
||||
|
||||
### Errors and restarts
|
||||
|
||||
`error` replies are per request: the helper carries on. EffectCraft stops (kills) the helper
|
||||
when it exits, closes stdout, sends a reply that doesn't parse or doesn't fit, or misses a
|
||||
timeout, or stops reading its stdin for longer than the timeout. The request then fails. The next request starts a fresh process, runs `hello`, replays
|
||||
earlier `load`s and recreates instances lazily: the helper sees a `render` for an `instance` it
|
||||
doesn't know and creates it.
|
||||
|
||||
## Writing a helper
|
||||
|
||||
Any language works: read lines, write prefixed lines, read and write a file. Rust helpers can
|
||||
use `effectcraft_helper_host::serve::serve` with a `Provider` that lists `EffectPlugin`s;
|
||||
`crates/helper-host/tests/it/main.rs` is a complete example and the protocol test that runs
|
||||
in `cargo xtask ci`. An unofficial OpenFX helper built this way lives in its own repository,
|
||||
[ec-extension-ofx](https://github.com/glumbis/ec-extension-ofx).
|
||||
@@ -309,6 +309,11 @@ Issues #41–#47 (all from the Linux AppImage 0.1.1, 5 October): panning the vie
|
||||
Settings ▸ Plug-in Library installs, updates and removes community WebAssembly effects listed
|
||||
in one reviewed catalog ([plugins.md](plugins.md#plug-in-library), #376); the catalog
|
||||
repository still has to be created and filled.
|
||||
- Native plug-in formats (OpenFX first) run only in **external effect helpers**, separate
|
||||
processes behind a safe, versioned protocol ([external-helpers.md](external-helpers.md),
|
||||
[ADR 0001](adr/0001-external-effect-helpers.md)): the core gains no native code and no
|
||||
`unsafe`, and a crashing plug-in is an error on the effect. The helpers themselves ship
|
||||
separately.
|
||||
- Features that come from third parties in After Effects are extensions in EffectCraft too, built
|
||||
on the same extension points as After Effects: scripts, ScriptUI panels and effect plug-ins
|
||||
([plugins.md](plugins.md#extension-points-compared-with-after-effects) compares them). ScriptUI
|
||||
|
||||
+34
-3
@@ -152,7 +152,7 @@ example of a keyframe tool.
|
||||
|
||||
## Effect plug-ins
|
||||
|
||||
EffectCraft loads third-party effects through a small, versioned plug-in API (**API version 1**).
|
||||
EffectCraft loads third-party effects through a small, versioned plug-in API (**API version 2**; WebAssembly ABI 1).
|
||||
A plug-in effect behaves exactly like a built-in one: it is listed in Effects & Presets (and the
|
||||
Effect menu) under its category, applied by id (`effect.apply {"effect": "org.example.posterize"}`)
|
||||
or display name, its parameters are ordinary properties (keyframes, expressions, Effect Controls,
|
||||
@@ -270,8 +270,39 @@ impl EffectPlugin for Swap {
|
||||
register_plugin(Arc::new(Swap(manifest)))?; // before the UI builds its menus
|
||||
```
|
||||
|
||||
`PluginParams` reads values by parameter id (`f`, `b`, `point`, `color`). At most 256 plug-in
|
||||
effects can be registered per process.
|
||||
`PluginParams` reads values by parameter id (`f`, `b`, `point`, `color`, and in API 2 `point3`,
|
||||
`layer`, `text`). At most 1024 plug-in effects can be registered per process.
|
||||
|
||||
## Plug-in API 2 (native plug-ins)
|
||||
|
||||
API 2 is for Rust plug-ins and for hosts that bring effects from elsewhere (such as
|
||||
[external effect helpers](external-helpers.md)); the WebAssembly ABI stays at 1. Manifests with `"api": 1` or `2` are accepted,
|
||||
`effect.plugins.list` reports `api: 2`, and up to 1024 plug-in effects can be registered.
|
||||
|
||||
* **New parameter types**: `point3` (`default` `[x, y, z]`; x, y arrive in buffer pixels, z as is),
|
||||
`layer` (another layer of the composition; flattens to its id, -1 for none, read with
|
||||
`PluginParams::layer`), `text` (a string field) and `hidden` (a string that is not shown, for
|
||||
opaque plug-in data). `text` / `hidden` take no slot in `flat`; their strings are in
|
||||
`PluginParams::texts`, one per such parameter in declaration order, read with
|
||||
`PluginParams::text(id)`. `slider` gains `"integer": true` (shown with 0 decimals) and every
|
||||
parameter an optional `"group"` path (`"Advanced/Edges"`; informational). WebAssembly modules
|
||||
can't use `point3`, `layer`, `text` or `hidden`.
|
||||
* **`PluginFrame`** also carries `origin` (buffer pixels of layer pixel (0, 0); padding shifts it)
|
||||
and `layer_size` (layer pixels).
|
||||
* **`EffectPlugin::padding(params, time, scale, layer_size, instance_key) -> u32`** asks for that
|
||||
many transparent buffer pixels on every side before rendering (glows, regions of definition
|
||||
larger than the input); at most 4096. `layer_size` is the layer's size in layer pixels (so
|
||||
sizes relative to the frame can be measured) and `instance_key` identifies the applied effect
|
||||
instance (see `PluginHost::instance_key`).
|
||||
* **`EffectPlugin::render_v2(frame, params, time, host)`** (default: calls `render`) gets a
|
||||
`PluginHost` with `input_at(layer_time)` (the effect's input at another time, in its own extent;
|
||||
the effects above this one are applied), `layer_input(param_id, layer_time)` (a layer parameter's
|
||||
layer, effects and masks applied, resampled into the frame's grid), `params_at(layer_time)`,
|
||||
`frame_rate()`, `time()`, `instance_key()` (a stable id of the applied effect instance: the
|
||||
same on every frame, different per layer and per application; plug-ins that keep state between
|
||||
frames pool their native instances by it) and `frame_range()` (the layer's source time range in
|
||||
layer-time seconds, `None` when it has none). Images are `PluginImage { width, height, pixels, origin, scale }`.
|
||||
It may run on several threads at once.
|
||||
|
||||
### Compatibility
|
||||
|
||||
|
||||
@@ -58,6 +58,7 @@ const LAYERS: &[(&str, u8)] = &[
|
||||
("lottie", 3),
|
||||
("format", 3),
|
||||
("plugin", 3),
|
||||
("helper-host", 3),
|
||||
("interchange", 3),
|
||||
("engine", 4),
|
||||
("script", 4),
|
||||
|
||||
Reference in new issue
Block a user