mirror of
https://github.com/storytold/effectcraft.git
synced 2026-11-07 11:01:36 +00:00
* Plug-ins: API 2 (PluginHost, padding, layer_span) Split out of the OpenFX work so it can be reviewed on its own. Additive; the WebAssembly ABI stays at 1. - point3 / layer / text / hidden parameters, integer sliders, parameter groups. - EffectPlugin::padding (transparent pixels around the buffer, at most 4096 per side). - EffectPlugin::render_v2 with a PluginHost: the input at other times, other layers, parameters at other times, frame rate, per-instance key, the layer's source range. - EffectEnv::layer_span from the renderer; MAX_PLUGINS 1024. - plugin_error: a plug-in effect's last render error, for the UI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Plug-ins: external effect helpers (crates/helper-host) Native plug-in formats run in separate helper programs, never in EffectCraft's process (docs/adr/0001-external-effect-helpers.md). New safe-Rust crate effectcraft-helper-host (L3, native only, unsafe forbidden): it finds helper programs, starts each as a child process and registers its effects through plug-in API 2, so the UI and command surface need nothing new. - Protocol v1 (docs/external-helpers.md): JSON lines over stdin / stdout (helper replies are prefixed so plug-in chatter is ignored); pixels go through a frame exchange file (/dev/shm on Linux), never through the pipe. Render-time callbacks for the input at other times, other layers and parameters at other times, with nested renders through the same helper. - A helper that exits, hangs past its timeout or breaks the protocol is stopped. The render fails with a message, the frame is left as it was, and the helper restarts for the next render. An effect that stops its helper 3 times is turned off (helper_blocklist.txt). - Opt-in: nothing starts helpers unless the user asks (Effect > Scan for Effect Helpers), or opts in to starting them at launch with EFFECTCRAFT_HELPERS=1, an environment variable, so writing to the settings folder alone can't run code. effect.helpers.scan / load are refused for confined agent sessions (confine.rs), like file.executeFile. - Discovery: helpers/ next to the app (Contents/Helpers on macOS), helpers/ in the settings folder, EFFECTCRAFT_HELPER_PATH. - Commands effect.helpers.list / scan / load; Effect menu entries (translated); the last render error of an effect shows as a warning sign after its name in Effect Controls. - serve(): the helper side for Rust helpers. tests/it/main.rs runs the whole protocol (render, padding, callbacks, nesting, threads, crash, restart, block list, hang) against a fake helper, with nothing to install. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Effect helpers: address review (start failures, frame file mode, nextest listing) - A helper that can't start (not installed any more, restarting too often, no frame exchange file) is reported but no longer charged to the effect being rendered: it is Failure::Refused, not Failure::Broken, so it never counts toward turning that effect off. New check in tests/it: once the helper is out of restarts, renders of an effect fail without blocking it (fails without the fix: the effect was turned off after 3 such renders). - The frame exchange file is created owner-only (0o600) on Unix: it holds the user's frames and /dev/shm is shared. Asserted in the frames unit test. - tests/it (harness = false) speaks the libtest listing protocol cargo-nextest uses: `--list --format terse` prints `helper_protocol: test` (nothing with `--ignored`), and `helper_protocol --exact` runs the checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Effect helpers: bound writes to a helper, drain stderr, keep callbacks in step - Writes to a helper go through a writer thread with the request timeout: a helper that stops reading its stdin (hung with a large callback answer or render request pending) no longer blocks the render thread forever while holding the helper. - stderr is drained as bytes (lossy, in 64 KiB pieces) to the end: output that isn't UTF-8 stopped the reader, closing the pipe under the helper (EPIPE / SIGPIPE on its next write); lines were unbounded. - Callback images that can't be written are answered with `null` instead of failing the render while the helper waits (which left the protocol out of step); a frame that can't be written before the request is sent is not charged to the effect. - Mid-render calls use the render's own process: no respawn or heartbeat ping while the helper is waiting for an answer. - Waiting for a busy helper is bounded (load timeout) instead of forever; status() no longer blocks on a running render. - Reap the child when start-up fails after spawning; cap the block list read. - Scripts start helpers only with Allow Scripts to Write Files and Access Network on (they run native code), as for File.execute(). - helper-host declares the workspace's no-panic crate lints; the desktop main() keeps its doc comment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Effect helpers: rustfmt the fake provider, build the test session in its initializer (clippy) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Plug-in Library: offer WebAssembly API 1 releases only; a helper already stopped mid-render isn't charged again - WasmPlugin loads only modules at the WebAssembly ABI version, which stays 1 while PLUGIN_API_VERSION is 2: share it as effects::plugin::WASM_API_VERSION and have the library's compatibility check use it, so it never installs an API 2 release that can't load. - with_running: a process that is already gone was stopped by what broke it (a nested render, charged there), so this render fails without charging its own effect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * frames: plug-in effects count as reading other frames (plug-in API 2) With main's narrowed frame invalidation (#2795), an edit inside a precomp under an effect that reads its layer at other times must reach the layer's whole span. Plug-in API 2 gives every plug-in effect the input at other times, so any effect outside ec. counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
163 lines
6.8 KiB
Rust
163 lines
6.8 KiB
Rust
//! The wasmi-backed [`EffectPlugin`] (plug-in API v1 ABI, see the crate docs).
|
|
|
|
use std::sync::Mutex;
|
|
|
|
use wasmi::{Config, Engine, Instance, Linker, Memory, Module, Store, StoreLimits, StoreLimitsBuilder, TypedFunc, WasmParams, WasmResults};
|
|
|
|
use crate::{EffectPlugin, PluginFrame, PluginManifest, PluginParams, WASM_API_VERSION};
|
|
|
|
/// Instruction budget per frame: a fixed allowance plus some per pixel. Generous for per-pixel
|
|
/// work, but a runaway loop stops instead of hanging the render.
|
|
const FUEL_BASE: u64 = 50_000_000;
|
|
const FUEL_PER_PIXEL: u64 = 20_000;
|
|
/// The most linear memory a plug-in instance may have (a 4K float frame is ~130 MB).
|
|
const MAX_MEMORY: usize = 1 << 30;
|
|
/// The longest manifest read.
|
|
const MAX_MANIFEST: i32 = 1 << 20;
|
|
|
|
type RenderFn = TypedFunc<(i32, i32, i32, i32, i32, f64, f64), i32>;
|
|
|
|
/// One instantiated module (renders running on several threads each take their own).
|
|
struct Inst {
|
|
store: Store<StoreLimits>,
|
|
instance: Instance,
|
|
memory: Memory,
|
|
alloc: TypedFunc<i32, i32>,
|
|
render: RenderFn,
|
|
}
|
|
|
|
impl Inst {
|
|
fn func<P: WasmParams, R: WasmResults>(&self, name: &str) -> Result<TypedFunc<P, R>, String> {
|
|
self.instance.get_typed_func::<P, R>(&self.store, name).map_err(|_| missing(name))
|
|
}
|
|
}
|
|
|
|
fn missing(name: &str) -> String {
|
|
format!("the module exports no `{name}` (or with the wrong signature)")
|
|
}
|
|
|
|
fn err(e: impl std::fmt::Display) -> String {
|
|
e.to_string()
|
|
}
|
|
|
|
fn instantiate(engine: &Engine, module: &Module) -> Result<Inst, String> {
|
|
let mut store = Store::new(engine, StoreLimitsBuilder::new().memory_size(MAX_MEMORY).instances(1).build());
|
|
store.limiter(|l| l);
|
|
store.set_fuel(FUEL_BASE).map_err(err)?;
|
|
let linker = Linker::<StoreLimits>::new(engine);
|
|
let instance = linker.instantiate_and_start(&mut store, module).map_err(|e| format!("instantiation failed: {e}"))?;
|
|
let memory = instance.get_memory(&store, "memory").ok_or("the module exports no `memory`")?;
|
|
let alloc = instance.get_typed_func::<i32, i32>(&store, "ec_alloc").map_err(|_| missing("ec_alloc"))?;
|
|
let render = instance.get_typed_func::<(i32, i32, i32, i32, i32, f64, f64), i32>(&store, "ec_render").map_err(|_| missing("ec_render"))?;
|
|
Ok(Inst { store, instance, memory, alloc, render })
|
|
}
|
|
|
|
/// A loaded WebAssembly effect plug-in.
|
|
pub struct WasmPlugin {
|
|
manifest: PluginManifest,
|
|
source: String,
|
|
engine: Engine,
|
|
module: Module,
|
|
/// Idle instances.
|
|
pool: Mutex<Vec<Inst>>,
|
|
}
|
|
|
|
impl WasmPlugin {
|
|
pub fn new(bytes: &[u8], source: &str) -> Result<WasmPlugin, String> {
|
|
let mut config = Config::default();
|
|
config.consume_fuel(true);
|
|
let engine = Engine::new(&config);
|
|
let module = Module::new(&engine, bytes).map_err(|e| format!("not a valid WebAssembly module: {e}"))?;
|
|
if let Some(i) = module.imports().next() {
|
|
return Err(format!("plug-ins can't import anything (the module imports `{}.{}`)", i.module(), i.name()));
|
|
}
|
|
let mut inst = instantiate(&engine, &module)?;
|
|
let version = inst.func::<(), i32>("ec_api_version")?;
|
|
let v = version.call(&mut inst.store, ()).map_err(err)?;
|
|
if u32::try_from(v).ok() != Some(WASM_API_VERSION) {
|
|
return Err(format!("the module implements plug-in API {v}, this build implements API {WASM_API_VERSION} for WebAssembly"));
|
|
}
|
|
let ptr = inst.func::<(), i32>("ec_manifest_ptr")?.call(&mut inst.store, ()).map_err(err)?;
|
|
let len = inst.func::<(), i32>("ec_manifest_len")?.call(&mut inst.store, ()).map_err(err)?;
|
|
if !(0..=MAX_MANIFEST).contains(&len) {
|
|
return Err(format!("bad manifest length {len}"));
|
|
}
|
|
let mut text = vec![0u8; len as usize];
|
|
inst.memory.read(&inst.store, ptr as u32 as usize, &mut text).map_err(|e| format!("manifest out of bounds: {e}"))?;
|
|
let manifest: PluginManifest = serde_json::from_slice(&text).map_err(|e| format!("bad manifest JSON: {e}"))?;
|
|
manifest.validate()?;
|
|
if let Some(p) = manifest.params.iter().find(|p| p.kind.needs_v2()) {
|
|
return Err(format!("parameter `{}`: WebAssembly plug-ins use plug-in API 1 parameter types (no point3, layer, text or hidden)", p.id));
|
|
}
|
|
Ok(WasmPlugin { manifest, source: source.to_string(), engine, module, pool: Mutex::new(vec![inst]) })
|
|
}
|
|
|
|
fn take(&self) -> Result<Inst, String> {
|
|
if let Some(i) = self.pool.lock().ok().and_then(|mut p| p.pop()) {
|
|
return Ok(i);
|
|
}
|
|
instantiate(&self.engine, &self.module)
|
|
}
|
|
|
|
fn give_back(&self, inst: Inst) {
|
|
if let Ok(mut p) = self.pool.lock() {
|
|
p.push(inst);
|
|
}
|
|
}
|
|
}
|
|
|
|
impl EffectPlugin for WasmPlugin {
|
|
fn manifest(&self) -> &PluginManifest {
|
|
&self.manifest
|
|
}
|
|
|
|
fn source(&self) -> String {
|
|
self.source.clone()
|
|
}
|
|
|
|
fn render(&self, frame: &mut PluginFrame, params: &PluginParams, time: f64) -> Result<(), String> {
|
|
let n = frame.pixels.len();
|
|
let params_bytes = params.flat.len() * 8;
|
|
let total = params_bytes + n * 16;
|
|
if total > i32::MAX as usize {
|
|
return Err("frame too large for a 32-bit plug-in".into());
|
|
}
|
|
let mut inst = self.take()?;
|
|
inst.store.set_fuel(FUEL_BASE + FUEL_PER_PIXEL * n as u64).map_err(err)?;
|
|
let base = inst.alloc.call(&mut inst.store, total as i32).map_err(err)? as u32 as usize;
|
|
// Parameters, then pixels (little-endian, as WebAssembly memory is).
|
|
let mut buf: Vec<u8> = Vec::with_capacity(total);
|
|
for v in params.flat {
|
|
buf.extend_from_slice(&v.to_le_bytes());
|
|
}
|
|
for px in frame.pixels.iter() {
|
|
for c in px {
|
|
buf.extend_from_slice(&c.to_le_bytes());
|
|
}
|
|
}
|
|
inst.memory.write(&mut inst.store, base, &buf).map_err(|e| format!("ec_alloc returned a buffer out of bounds: {e}"))?;
|
|
let px_ptr = (base + params_bytes) as i32;
|
|
let r = inst
|
|
.render
|
|
.call(&mut inst.store, (px_ptr, frame.width as i32, frame.height as i32, base as i32, params.flat.len() as i32, time, frame.scale))
|
|
.map_err(|e| format!("ec_render trapped: {e}"));
|
|
let r = match r {
|
|
Ok(0) => {
|
|
inst.memory.read(&inst.store, base + params_bytes, &mut buf[params_bytes..]).map_err(err)?;
|
|
for (px, b) in frame.pixels.iter_mut().zip(buf[params_bytes..].as_chunks::<16>().0.iter()) {
|
|
for (c, v) in px.iter_mut().zip(b.as_chunks::<4>().0.iter()) {
|
|
*c = f32::from_le_bytes([v[0], v[1], v[2], v[3]]);
|
|
}
|
|
}
|
|
Ok(())
|
|
}
|
|
Ok(code) => Err(format!("ec_render returned error {code}")),
|
|
Err(e) => Err(e),
|
|
};
|
|
if r.is_ok() {
|
|
self.give_back(inst);
|
|
}
|
|
r
|
|
}
|
|
}
|