filmcraft/docs/web.md

FilmCraft on the web

apps/filmcraft-web runs the same engine and egui UI as the desktop app in the browser: the workspace compiled to wasm32-unknown-unknown, started by eframe's web runner on WebGPU (WebGL2 fallback). It is a static site: one .wasm, its wasm-bindgen JavaScript glue, index.html, an AudioWorklet script and an icon. Nothing is uploaded anywhere; media stay on the user's machine.

Build and run

Tool	Version	Install
Rust target	wasm32-unknown-unknown	rustup target add wasm32-unknown-unknown
wasm-bindgen CLI	exactly the wasm-bindgen crate version (0.2.129)	cargo install wasm-bindgen-cli --version 0.2.129 --locked
wasm-opt (optional)	any recent binaryen	brew install binaryen; used by cargo xtask web when found

cargo xtask web                 # release build → target/web/dist (index.html, filmcraft_web.js, filmcraft_web_bg.wasm, …)
cargo xtask web --dev           # unoptimised build (faster to compile, slow to run)
cargo xtask web --serve 8765    # build, then serve dist on http://127.0.0.1:8765/

--serve is a tiny localhost-only static server that sends Cross-Origin-Opener-Policy / Cross-Origin-Embedder-Policy, so the page is cross-origin isolated like a production deployment should be. Any static server works (python3 -m http.server -d target/web/dist 8765); it must serve .wasm as application/wasm for streaming compilation.

The web crate is empty on non-wasm targets, so cargo test --workspace and clippy are unaffected; cargo xtask wasm (part of cargo xtask ci) checks it, with filmcraft-ui-egui, for wasm32-unknown-unknown.

URL flags: ?empty (no demo project), ?norecover (don't reopen the auto-saved project), ?fresh (also don't restore media kept in OPFS), ?cpu (CPU compositor), ?webgl (WebGL2 only; the page reloads with it when WebGPU is present but fails to start), ?nowebcodecs (decode with FilmCraft's own decoders only).

If the app panics after start-up (or runs out of memory), index.html shows a "FilmCraft stopped working" overlay with the panic message and a Reload button instead of a frozen canvas, and sets window.filmcraftLoad.fatal. Code the web build runs must not call std::env::temp_dir, std::time::{Instant, SystemTime}::now, std::thread::sleep/spawn or std::process::id: they panic on wasm32-unknown-unknown (use filmcraft_engine::temp_dir, web_time, or a cfg).

How the desktop pieces map to the browser

Desktop	Web
frame worker threads	FrameServer::pump: queued frame jobs run on the UI thread between egui frames, highest priority first, within a time budget (24 ms while playing, 40 ms otherwise)
export on a worker thread	filmcraft_export::Exporter: the export advances one frame per step from Session::pump_jobs, 30 ms per UI frame; progress shows in the header like on the desktop
std::fs (FsServices)	fs::WebServices: a virtual file table. Picked/dropped files are File handles (never copied); files the app writes are kept in memory and offered as downloads
file dialogs	File System Access showOpenFilePicker where available, else <input type=file>; drop files anywhere on the page
crash-recovery journal	OPFS: recovery/snapshot.fcproj every 5 s while there are unsaved changes, plus copies of imported media in media/; the next visit reopens the snapshot
cpal output	WebAudio AudioWorklet
VideoToolbox & co.	WebCodecs VideoDecoder
TCP control channel / MCP	window.filmcraft (below)

Threads

wasm threads need a cross-origin isolated page and a wasm build with atomics (a nightly build-std toolchain), so this build is single-threaded everywhere: frame rendering, decoding, mixing and encoding are cooperative on the UI thread, and rayon runs inline. filmcraft.info() reports crossOriginIsolated so a threaded build can choose at start-up. Commands that need a thread on the desktop (render previews, proxies, Project Manager, mask tracking) report an error on the web for now.

Media reads

Containers are opened through Services::reader (a filmcraft_media::ByteReader): MP4/MOV and Matroska read their index at import and single samples afterwards, so a multi-gigabyte file is never loaded whole. Browser Blob reads are asynchronous, so BlobReader keeps a 384 MiB LRU cache of 1 MiB chunks: a read whose chunks are missing starts fetching them (plus three chunks of read-ahead) and fails with WouldBlock, setting filmcraft_media::pending. The frame server then drops that frame's result and retries the job after the bytes arrive (the fetch wakes the UI); the stepped exporter re-renders the frame; an import retries file.import. Before importing, the container index is prefetched (an MP4's moov box wherever it is, a small Matroska file whole). Stills, WAV and compressed-audio files are decoded whole, as on the desktop.

Limits: Matroska/WebM files larger than the chunk cache import slowly (opening scans every cluster header); a file the user picked is only readable while the page is open, unless its OPFS copy exists (files up to 4 GiB are copied in the background).

Decoding

FilmCraft's own decoders (H.264, HEVC, VP9, AV1, ProRes, DNxHR, MJPEG, AAC, Opus…) run in the browser unchanged. When the browser has WebCodecs, H.264, HEVC, VP9 and AV1 video in MP4/MOV is decoded by the browser's (usually hardware) VideoDecoder instead: webcodecs::reader_opener is registered ahead of the built-in openers and opens such files as a WcSource, whose audio and media info come from our Mp4Source. WebCodecs is asynchronous, so a frame request feeds samples to the decoder and returns "pending"; the decoder's output callback converts the VideoFrame to RGBA (drawn on an OffscreenCanvas) into a per-source frame cache, and the retried request finds it. Codec support is probed at start-up with VideoDecoder.isConfigSupported; any decoder error switches that source to our decoder.

Audio

audio::Handle implements AudioOut. The UI thread mixes the sequence 250 ms ahead in 2048-frame blocks and posts them to the worklet (web/audio-worklet.js), which counts the frames it really played and reports them with its currentTime. played_frames extrapolates from the last report on the audio context's clock, never past what was queued — so the audio clock is the playback master as on the desktop, and a starved worklet holds the playhead. Browsers start audio suspended until the first click or key press; until then playback runs on the wall clock.

GPU

eframe creates a WebGPU device when the browser has one, else WebGL2. FilmCraft's GPU compositor (filmcraft-gpu) is used on WebGPU; on WebGL2 frames are composited on the CPU. Check filmcraft.info().backend and .compositor.

window.filmcraft: the agent / test API

The control channel of the desktop app (docs/control-protocol.md) as promises. Results resolve with the method's result, errors reject.

Call
filmcraft.execute(command, params)	run an engine command (same ids and params as MCP / filmcraft-cli)
filmcraft.request(method, params)	any control method: ui.inspect, ui.click, ui.key, ui.playback, ui.menu.invoke…
filmcraft.commands()	the command list
filmcraft.inspect()	ui.inspect
filmcraft.screenshot(params)	{pngBase64, width, height} of the canvas (or {panel})
filmcraft.importFiles(files)	import File / FileList / File[]; resolves {items, errors, paths}
filmcraft.importUrl(url, name?)	fetch a URL and import it (tests)
filmcraft.openProject(fileOrUrl)	open a .fcproj
filmcraft.files()	the virtual files: {path, size, kind}
filmcraft.readFile(path)	a file's bytes (Uint8Array), e.g. an export
filmcraft.info()	backend, compositor, isolation, OPFS, WebCodecs support and counters, start-up time

window.filmcraftLoad holds {wasmMs, readyMs} (module fetch+compile, and until the app runs).

await filmcraft.importUrl("/clip.mp4");
await filmcraft.execute("file.newSequence", {fromItem: 48});
await filmcraft.request("ui.playback", {action: "play"});
const {job} = await filmcraft.execute("file.exportMedia", {format: "h264", path: "/exports/out.mp4"});

Browser test

apps/filmcraft-web/tests/smoke.mjs drives headless Chrome over the DevTools protocol (Node ≥ 22, no npm packages): load, demo playback, import of a generated MP4, playback, H.264 export (download). It writes screenshots and report.json:

cargo xtask web --serve 8765 &
ffmpeg -f lavfi -i testsrc2=size=640x360:rate=30 -f lavfi -i sine -t 3 -c:v libx264 -pix_fmt yuv420p -c:a aac target/web/dist/web-test.mp4
node apps/filmcraft-web/tests/smoke.mjs --url http://127.0.0.1:8765/ --media web-test.mp4 --out target/web/smoke