mirror of
https://github.com/storytold/effectcraft.git
synced 2026-11-20 09:05:51 +00:00
* Plug-ins: API 2 (PluginHost, padding, layer_span) Split out of the OpenFX work so it can be reviewed on its own. Additive; the WebAssembly ABI stays at 1. - point3 / layer / text / hidden parameters, integer sliders, parameter groups. - EffectPlugin::padding (transparent pixels around the buffer, at most 4096 per side). - EffectPlugin::render_v2 with a PluginHost: the input at other times, other layers, parameters at other times, frame rate, per-instance key, the layer's source range. - EffectEnv::layer_span from the renderer; MAX_PLUGINS 1024. - plugin_error: a plug-in effect's last render error, for the UI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Plug-ins: external effect helpers (crates/helper-host) Native plug-in formats run in separate helper programs, never in EffectCraft's process (docs/adr/0001-external-effect-helpers.md). New safe-Rust crate effectcraft-helper-host (L3, native only, unsafe forbidden): it finds helper programs, starts each as a child process and registers its effects through plug-in API 2, so the UI and command surface need nothing new. - Protocol v1 (docs/external-helpers.md): JSON lines over stdin / stdout (helper replies are prefixed so plug-in chatter is ignored); pixels go through a frame exchange file (/dev/shm on Linux), never through the pipe. Render-time callbacks for the input at other times, other layers and parameters at other times, with nested renders through the same helper. - A helper that exits, hangs past its timeout or breaks the protocol is stopped. The render fails with a message, the frame is left as it was, and the helper restarts for the next render. An effect that stops its helper 3 times is turned off (helper_blocklist.txt). - Opt-in: nothing starts helpers unless the user asks (Effect > Scan for Effect Helpers), or opts in to starting them at launch with EFFECTCRAFT_HELPERS=1, an environment variable, so writing to the settings folder alone can't run code. effect.helpers.scan / load are refused for confined agent sessions (confine.rs), like file.executeFile. - Discovery: helpers/ next to the app (Contents/Helpers on macOS), helpers/ in the settings folder, EFFECTCRAFT_HELPER_PATH. - Commands effect.helpers.list / scan / load; Effect menu entries (translated); the last render error of an effect shows as a warning sign after its name in Effect Controls. - serve(): the helper side for Rust helpers. tests/it/main.rs runs the whole protocol (render, padding, callbacks, nesting, threads, crash, restart, block list, hang) against a fake helper, with nothing to install. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Effect helpers: address review (start failures, frame file mode, nextest listing) - A helper that can't start (not installed any more, restarting too often, no frame exchange file) is reported but no longer charged to the effect being rendered: it is Failure::Refused, not Failure::Broken, so it never counts toward turning that effect off. New check in tests/it: once the helper is out of restarts, renders of an effect fail without blocking it (fails without the fix: the effect was turned off after 3 such renders). - The frame exchange file is created owner-only (0o600) on Unix: it holds the user's frames and /dev/shm is shared. Asserted in the frames unit test. - tests/it (harness = false) speaks the libtest listing protocol cargo-nextest uses: `--list --format terse` prints `helper_protocol: test` (nothing with `--ignored`), and `helper_protocol --exact` runs the checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Effect helpers: bound writes to a helper, drain stderr, keep callbacks in step - Writes to a helper go through a writer thread with the request timeout: a helper that stops reading its stdin (hung with a large callback answer or render request pending) no longer blocks the render thread forever while holding the helper. - stderr is drained as bytes (lossy, in 64 KiB pieces) to the end: output that isn't UTF-8 stopped the reader, closing the pipe under the helper (EPIPE / SIGPIPE on its next write); lines were unbounded. - Callback images that can't be written are answered with `null` instead of failing the render while the helper waits (which left the protocol out of step); a frame that can't be written before the request is sent is not charged to the effect. - Mid-render calls use the render's own process: no respawn or heartbeat ping while the helper is waiting for an answer. - Waiting for a busy helper is bounded (load timeout) instead of forever; status() no longer blocks on a running render. - Reap the child when start-up fails after spawning; cap the block list read. - Scripts start helpers only with Allow Scripts to Write Files and Access Network on (they run native code), as for File.execute(). - helper-host declares the workspace's no-panic crate lints; the desktop main() keeps its doc comment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Effect helpers: rustfmt the fake provider, build the test session in its initializer (clippy) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Plug-in Library: offer WebAssembly API 1 releases only; a helper already stopped mid-render isn't charged again - WasmPlugin loads only modules at the WebAssembly ABI version, which stays 1 while PLUGIN_API_VERSION is 2: share it as effects::plugin::WASM_API_VERSION and have the library's compatibility check use it, so it never installs an API 2 release that can't load. - with_running: a process that is already gone was stopped by what broke it (a nested render, charged there), so this render fails without charging its own effect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * frames: plug-in effects count as reading other frames (plug-in API 2) With main's narrowed frame invalidation (#2795), an edit inside a precomp under an effect that reads its layer at other times must reach the layer's whole span. Plug-in API 2 gives every plug-in effect the input at other times, so any effect outside ec. counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
241 lines
9.9 KiB
Rust
241 lines
9.9 KiB
Rust
//! The helper side of the protocol: [`serve`] answers EffectCraft's requests with any set of
|
|
//! [`EffectPlugin`]s, so a helper program only has to provide its effects ([`Provider`]).
|
|
//! The fake helper in the tests and the OpenFX helper both use it.
|
|
//!
|
|
//! During a render the plug-in's [`PluginHost`] calls become requests to EffectCraft
|
|
//! ([`Reply::InputAt`], ...). While a helper waits for the answer, EffectCraft may send a nested
|
|
//! [`Request::Render`] (an effect of this helper further up the same layer's stack); it is
|
|
//! served right away, on the same stack, before the wait goes on.
|
|
|
|
use std::cell::RefCell;
|
|
use std::collections::HashMap;
|
|
use std::io::{BufRead, Read, Write};
|
|
use std::path::Path;
|
|
use std::sync::Arc;
|
|
|
|
use effectcraft_effects::plugin::{EffectPlugin, PluginFrame, PluginHost, PluginImage, PluginParams};
|
|
|
|
use crate::frames::Exchange;
|
|
use crate::protocol::{ImageRef, MAX_LINE, PROTOCOL_VERSION, RemoteEffect, RenderRequest, Reply, Request, reply_line};
|
|
|
|
/// How deep nested renders may go.
|
|
const MAX_DEPTH: u32 = 16;
|
|
|
|
/// What a helper program offers.
|
|
pub trait Provider {
|
|
/// The helper's display name (`OpenFX`).
|
|
fn name(&self) -> String;
|
|
/// The helper's own version.
|
|
fn version(&self) -> String {
|
|
String::new()
|
|
}
|
|
/// File extensions [`Provider::load`] accepts (`["ofx"]`).
|
|
fn loads(&self) -> Vec<String> {
|
|
vec![]
|
|
}
|
|
/// The effects available at start-up, and errors worth reporting.
|
|
fn effects(&mut self) -> (Vec<Arc<dyn EffectPlugin>>, Vec<String>);
|
|
/// Load effects from a file or folder the user picked.
|
|
fn load(&mut self, _path: &str) -> Result<(Vec<Arc<dyn EffectPlugin>>, Vec<String>), String> {
|
|
Err("this helper does not load files".into())
|
|
}
|
|
/// Whether EffectCraft should ask for padding before each render of `effect`.
|
|
fn wants_padding(&self, _effect: &dyn EffectPlugin) -> bool {
|
|
true
|
|
}
|
|
/// The effect instance is gone.
|
|
fn destroy(&mut self, _instance: u64) {}
|
|
}
|
|
|
|
/// Serve requests on stdin / stdout until EffectCraft says `shutdown` or closes the pipe.
|
|
pub fn serve(provider: &mut dyn Provider) -> Result<(), String> {
|
|
let stdin = std::io::stdin();
|
|
let stdout = std::io::stdout();
|
|
serve_io(provider, stdin.lock(), stdout.lock())
|
|
}
|
|
|
|
/// [`serve`] on any reader / writer.
|
|
pub fn serve_io(provider: &mut dyn Provider, input: impl BufRead, output: impl Write) -> Result<(), String> {
|
|
let mut s = Server { provider, input: Box::new(input), output: Box::new(output), exchange: None, effects: HashMap::new(), depth: 0 };
|
|
loop {
|
|
let Some(req) = s.next_request()? else { return Ok(()) };
|
|
if matches!(req, Request::Shutdown) {
|
|
return Ok(());
|
|
}
|
|
s.handle(req)?;
|
|
}
|
|
}
|
|
|
|
struct Server<'p> {
|
|
provider: &'p mut dyn Provider,
|
|
input: Box<dyn BufRead + 'p>,
|
|
output: Box<dyn Write + 'p>,
|
|
exchange: Option<Exchange>,
|
|
effects: HashMap<String, Arc<dyn EffectPlugin>>,
|
|
depth: u32,
|
|
}
|
|
|
|
impl Server<'_> {
|
|
/// The next request; `None` when the pipe closed.
|
|
fn next_request(&mut self) -> Result<Option<Request>, String> {
|
|
loop {
|
|
let mut buf = vec![];
|
|
let n = (&mut self.input).take(MAX_LINE).read_until(b'\n', &mut buf).map_err(|e| e.to_string())?;
|
|
if n == 0 {
|
|
return Ok(None);
|
|
}
|
|
let line = String::from_utf8_lossy(&buf);
|
|
if line.trim().is_empty() {
|
|
continue;
|
|
}
|
|
match serde_json::from_str::<Request>(line.trim()) {
|
|
Ok(r) => return Ok(Some(r)),
|
|
Err(e) => self.send(&Reply::Error { message: format!("bad request: {e}") })?,
|
|
}
|
|
}
|
|
}
|
|
|
|
fn send(&mut self, r: &Reply) -> Result<(), String> {
|
|
let line = reply_line(r)?;
|
|
self.output.write_all(line.as_bytes()).and_then(|_| self.output.flush()).map_err(|e| e.to_string())
|
|
}
|
|
|
|
fn describe(&mut self, list: Vec<Arc<dyn EffectPlugin>>, errors: Vec<String>) -> Result<(), String> {
|
|
let mut effects = vec![];
|
|
for e in list {
|
|
let manifest = e.manifest().clone();
|
|
effects.push(RemoteEffect { padding: self.provider.wants_padding(e.as_ref()), manifest: manifest.clone() });
|
|
self.effects.insert(manifest.id, e);
|
|
}
|
|
self.send(&Reply::Effects { effects, errors })
|
|
}
|
|
|
|
/// Answer one request (anything but `shutdown`).
|
|
fn handle(&mut self, req: Request) -> Result<(), String> {
|
|
match req {
|
|
Request::Hello { version, frames } => {
|
|
if version != PROTOCOL_VERSION {
|
|
return self.send(&Reply::Error { message: format!("this helper speaks protocol {PROTOCOL_VERSION}, not {version}") });
|
|
}
|
|
match Exchange::open(Path::new(&frames)) {
|
|
Ok(x) => self.exchange = Some(x),
|
|
Err(e) => return self.send(&Reply::Error { message: format!("can't open the frame exchange file: {e}") }),
|
|
}
|
|
let (name, helper_version, loads) = (self.provider.name(), self.provider.version(), self.provider.loads());
|
|
self.send(&Reply::Hello { version: PROTOCOL_VERSION, name, helper_version, loads })
|
|
}
|
|
Request::Describe => {
|
|
let (list, errors) = self.provider.effects();
|
|
self.describe(list, errors)
|
|
}
|
|
Request::Load { path } => match self.provider.load(&path) {
|
|
Ok((list, errors)) => self.describe(list, errors),
|
|
Err(message) => self.send(&Reply::Error { message }),
|
|
},
|
|
Request::Padding { effect, instance, time, scale, layer_size, flat, texts } => {
|
|
let Some(e) = self.effects.get(&effect).cloned() else {
|
|
return self.send(&Reply::Error { message: format!("unknown effect `{effect}`") });
|
|
};
|
|
let params = PluginParams { manifest: e.manifest(), flat: &flat, texts: &texts };
|
|
let pixels = e.padding(¶ms, time, scale, layer_size, instance);
|
|
self.send(&Reply::Padding { pixels })
|
|
}
|
|
Request::Render(r) => match self.render(&r) {
|
|
Ok(()) => self.send(&Reply::Done),
|
|
Err(message) => self.send(&Reply::Error { message }),
|
|
},
|
|
Request::Destroy { instance } => {
|
|
self.provider.destroy(instance);
|
|
Ok(())
|
|
}
|
|
Request::Ping => self.send(&Reply::Pong),
|
|
Request::Image { .. } | Request::Params { .. } => self.send(&Reply::Error { message: "no callback is waiting for an answer".into() }),
|
|
Request::Shutdown => Ok(()),
|
|
}
|
|
}
|
|
|
|
fn render(&mut self, r: &RenderRequest) -> Result<(), String> {
|
|
let e = self.effects.get(&r.effect).cloned().ok_or_else(|| format!("unknown effect `{}`", r.effect))?;
|
|
if self.depth >= MAX_DEPTH {
|
|
return Err("renders nested too deeply".into());
|
|
}
|
|
let exchange = self.exchange.as_mut().ok_or("no frame exchange file (hello first)")?;
|
|
let mut pixels = exchange.read(&r.frame)?;
|
|
let params = PluginParams { manifest: e.manifest(), flat: &r.flat, texts: &r.texts };
|
|
let mut frame = PluginFrame {
|
|
width: r.frame.width,
|
|
height: r.frame.height,
|
|
pixels: &mut pixels,
|
|
scale: r.frame.scale,
|
|
origin: r.frame.origin,
|
|
layer_size: r.layer_size,
|
|
};
|
|
self.depth += 1;
|
|
let result = {
|
|
let host = Callbacks { server: RefCell::new(&mut *self), req: r };
|
|
e.render_v2(&mut frame, ¶ms, r.time, &host)
|
|
};
|
|
self.depth = self.depth.saturating_sub(1);
|
|
result?;
|
|
let exchange = self.exchange.as_mut().ok_or("no frame exchange file")?;
|
|
exchange.write(&r.frame, &pixels)
|
|
}
|
|
|
|
/// Ask EffectCraft something during a render and wait for the answer, serving nested
|
|
/// requests meanwhile.
|
|
fn ask(&mut self, q: &Reply) -> Result<Request, String> {
|
|
self.send(q)?;
|
|
loop {
|
|
let req = self.next_request()?.ok_or("EffectCraft closed the pipe")?;
|
|
match req {
|
|
Request::Image { .. } | Request::Params { .. } => return Ok(req),
|
|
Request::Shutdown => return Err("EffectCraft asked the helper to shut down".into()),
|
|
other => self.handle(other)?,
|
|
}
|
|
}
|
|
}
|
|
|
|
fn image(&mut self, q: &Reply) -> Option<PluginImage> {
|
|
let Ok(Request::Image { image: Some(at) }) = self.ask(q) else { return None };
|
|
self.read_image(&at)
|
|
}
|
|
|
|
fn read_image(&mut self, at: &ImageRef) -> Option<PluginImage> {
|
|
let pixels = self.exchange.as_mut()?.read(at).ok()?;
|
|
Some(PluginImage { width: at.width, height: at.height, pixels, origin: at.origin, scale: at.scale })
|
|
}
|
|
}
|
|
|
|
/// The [`PluginHost`] a helper's plug-in renders with: questions go back to EffectCraft.
|
|
struct Callbacks<'a, 'p> {
|
|
server: RefCell<&'a mut Server<'p>>,
|
|
req: &'a RenderRequest,
|
|
}
|
|
|
|
impl PluginHost for Callbacks<'_, '_> {
|
|
fn input_at(&self, layer_time: f64) -> Option<PluginImage> {
|
|
self.server.try_borrow_mut().ok()?.image(&Reply::InputAt { time: layer_time })
|
|
}
|
|
fn layer_input(&self, param_id: &str, layer_time: f64) -> Option<PluginImage> {
|
|
self.server.try_borrow_mut().ok()?.image(&Reply::LayerInput { param: param_id.to_string(), time: layer_time })
|
|
}
|
|
fn params_at(&self, layer_time: f64) -> Option<(Vec<f64>, Vec<String>)> {
|
|
match self.server.try_borrow_mut().ok()?.ask(&Reply::ParamsAt { time: layer_time }) {
|
|
Ok(Request::Params { values }) => values,
|
|
_ => None,
|
|
}
|
|
}
|
|
fn frame_rate(&self) -> f64 {
|
|
self.req.frame_rate
|
|
}
|
|
fn time(&self) -> f64 {
|
|
self.req.time
|
|
}
|
|
fn instance_key(&self) -> u64 {
|
|
self.req.instance
|
|
}
|
|
fn frame_range(&self) -> Option<(f64, f64)> {
|
|
self.req.frame_range
|
|
}
|
|
}
|