mirror of
https://github.com/storytold/effectcraft.git
synced 2026-11-06 18:41:26 +00:00
* Content-Aware Fill: bit-pack holes, spill frames to disk, stream outputs (#2570 phases 2-4) * Content-Aware Fill phases 2-4: bit-pack holes, spill frames, stream outputs (clippy clean) * Address CodeRabbit review: spill safety, streaming correctness, perf, memory guard (#2570) * Address the remaining CodeRabbit review findings on Content-Aware Fill (#2570) A non-streaming fill returned empty frames, progress could jump backwards between the sink and the fill, the memory guard ignored the membrane frames still resident for the flow pass (and wasm's in-RAM copies), and the GPU fill stages ran outside an error scope, so an OOM retired the shared context instead of falling back to the CPU for that frame. 🤖 Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Content-Aware Fill: sample spilled frames without decoding them, check spill errors (#2570) Tracing sampled spilled frames by decoding whole frames under the shared cache lock, so parallel traces across neighbouring frames evicted each other and mostly paid a full-frame read while every other thread waited. Sample only the 2x2 bilinear support with positioned reads on an open handle instead, and report a failed spill read in the no-hole branch rather than streaming black frames as a successful fill. 🤖 Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Content-Aware Fill: spill membrane/propagated frames, unquantized input, surface scratch errors (#2570) Flow and propagation now write their intermediate frames to scratch spills beside the output instead of holding the whole range as float RGBA, and the input spill uses 32-bit floats whatever the project depth so the fill sees the same values as the old in-memory path. Spill reads/writes and scratch creation report through a shared error slot that run_plan surfaces instead of masking disk failures as a cancellation. Sampling uses positioned reads; the guard models the packed masks plus per-thread scratch. * Content-Aware Fill: report a membrane-spill write failure instead of cancelling (#2570) * Content-Aware Fill: include the flow fields in the memory guard (#2570) * Content-Aware Fill: scale the memory guard with workers and check scratch space (#2570) The guard's scratch term was a fixed 8 frames, which under-counted a wide machine with Lighting Correction by several times. It now models the sequential synthesis stage plus a few frames per worker (from available parallelism) and takes Lighting Correction into account. Native fills write three f32 copies of the range to scratch beside the output, so a range that cannot fit on that volume is now refused up front with the free space in the message, before any frame is rendered. Tests cover the worker/lighting budget, the scratch formula, a shared error slot between the input and secondary spills, an uncreatable spill folder, and the engine surfacing a spill failure as an error instead of a cancellation. * Content-Aware Fill: resolve a relative output folder for the scratch check (#2570) A bare relative output name has an empty parent, so the walk-up found no existing ancestor and skipped the free-space check. Resolve it against the process directory first, and cover that in the test. * Do not reformat the GPU fill tests in the spill branch (#2570) The file was already not rustfmt-clean upstream, so the formatter only added noise to a file this PR no longer needs to touch: phase 1's call-site changes are already on the base branch. * Content-Aware Fill: measure peak RSS and correct the per-worker scratch model (#2570) The ignored measure_peak_rss_of_a_fill test fills a spilled 640x360 range and reports VmHWM. On this 18-worker machine 90 frames peak at 340-470 MB across runs (35 MB with RAYON_NUM_THREADS=1) and 600 frames at 582 MB, so the scratch slope is ~7 frames a worker, not the 2 the first guard revision assumed. Scaling the summary down showed a clean linear slope at 8 and 4 workers, and Lighting Correction stayed below the flow stage's peak. Budget 8 frames a worker (9 with Lighting Correction), just above the measured top of the range, and adjust the guard test to the real numbers. At 600 frames the measured peak is 582 MB against ~11.1 GB for the old five-copies model, a 19x reduction. * Contributors: credit Madhav Agarwal in the About window Adds the self-chosen real and display name for @gmrmad1. The generated contributors.json picks it up once the PR is merged. * Address CodeRabbit: size the guard from the rayon pool; migrate GPU tests (#2570) fill_workers used available_parallelism, but the parallel fill stages run on the rayon pool, which RAYON_NUM_THREADS can size larger. Use rayon::current_num_threads so the scratch budget follows the threads that actually run; wasm keeps rayon's single-thread fallback. The GPU inpaint tests still used the pre-spill FillInput API. The earlier 'do not reformat' revert dropped their API migration along with the formatting, leaving cargo test -p effectcraft-gpu uncompilable. Restore the Frames::Memory / HoleMask / sink migration and verify the 15 inpaint tests. * Fix gate: flatten the wasm prop lookup and keep the budget wrapper test-only (#2570) * Address CodeRabbit review: spill error check, global-pool preflight, docs (#2570) * Include RAM-backed spill in the preflight and guard streaming reads (#2570) * Resolve relative spill paths, prefer stacked mounts, guard the Edge Blend sink (#2570) * Make the fill spill anonymous and test the streaming claims (#2570) The spill was a named file in the output folder that only Drop removed, so an out-of-memory kill or a force-quit left a multi-gigabyte effectcraft-fill-* file in the user's project folder, and a failed set_len after keep() leaked it too. The spill is now an anonymous temporary file (tempfile_in), which the system reclaims when its last handle closes; every read and write is positioned on that one handle (pwrite/seek_write join the existing pread/seek_read), and the folder is kept only so the propagated frames spill beside the input. Also: - Edge Blend checks for a failed read before handing a frame to the sink, so a black fallback frame never streams. - The propagation workers stop as soon as the spill records an error. - HoleMask::get/set use get()/get_mut(), so a mask whose public words are shorter than len cannot panic. New tests cover the PR's headline claims: a spilled range fills bit-identically to the same frames in RAM (returned and streamed), the sink sees frames strictly in order, a truncated spill fails with a recorded error and streams nothing, a failure mid-stream stops before the next frame, propagation stops after a failure, the spill has no name in its folder, and HoleMask round-trips at word boundaries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Content-Aware Fill: clean up a stopped fill and count two scratch copies (#2570) - A fill that fails or is cancelled after frames started streaming deletes the PNGs it wrote, and the output folder when the fill made it and it is left empty (OutputFiles, dropped after the spill so Windows can remove the folder). - After a successful fill, run_plan checks that every frame has a file, and apply_plan/add_fill_layer return an error on an empty range or file list (a job worker's reply) instead of indexing [0]. - scratch_need charges two f32 copies (32 B/px): flow_pairs drops the membrane spill before the propagated one is created, so the third copy never coexists. - The spill-creation error carries a stable "spill: cannot make the scratch file" marker; the panel test asserted on the OS wording, which on Windows is "Cannot create a file when that file already exists". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Content-Aware Fill: a stopped fill leaves files it didn't create (#2570) A fill into a given outputDir that already holds an earlier sequence overwrites its frames as it goes (as before). The cleanup of a failed or stopped fill deleted every path it wrote, including those earlier frames, so a layer still using that sequence lost them. It now deletes only the files the fill created (CodeRabbit on #3079). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Codebuff <noreply@codebuff.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
7 files changed
+1671
-140
No files matched your search
Generated
+1
@@ -1855,6 +1855,7 @@ dependencies = [
|
||||
"effectcraft-color",
|
||||
"effectcraft-geom",
|
||||
"rayon",
|
||||
"tempfile",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
||||
@@ -13,7 +13,9 @@ use std::sync::Arc;
|
||||
use effectcraft_color::Label;
|
||||
use effectcraft_project::{Footage, FootageKind, ItemId, ItemKind, LayerId, LayerSource, build};
|
||||
use effectcraft_raster::Image;
|
||||
use effectcraft_raster::inpaint::{FillInput, FillMethod, FillOpts, dilate};
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
use effectcraft_raster::inpaint::SpillHandle;
|
||||
use effectcraft_raster::inpaint::{FillInput, FillMethod, FillOpts, Frames, HoleMask, dilate};
|
||||
use effectcraft_render::{Accelerator, EvalCtx, ExprHost, FootageSource, LayerCache, Renderer};
|
||||
use effectcraft_time::Tick;
|
||||
use serde::{Deserialize, Serialize};
|
||||
@@ -170,11 +172,76 @@ pub struct FillPlan {
|
||||
pub dir: String,
|
||||
}
|
||||
|
||||
/// The fill holds every frame as float RGBA (16 bytes a pixel), its hole mask (1) and the filled
|
||||
/// copy (16) until it finishes (#1789): when even that is more than `available`, the error to
|
||||
/// report up front, instead of running out of memory partway through.
|
||||
/// Transient full frames one parallel stage holds per worker: the membrane reads a frame and
|
||||
/// returns a copy, the flow stage holds the neighbouring pair plus the luma pyramids, and
|
||||
/// Lighting Correction adds the mismatch ring (`diff`, its mask) on top.
|
||||
///
|
||||
/// Measured with `measure_peak_rss_of_a_fill` (90 frames, 640×360, Surface): peak RSS grows by
|
||||
/// ~7 frames a rayon worker at the top of its run-to-run spread, and Lighting Correction stays
|
||||
/// below the flow stage's peak, so 8 (9 with lighting) is a conservative constant (#2570).
|
||||
fn per_worker_frames(lighting: bool) -> u64 {
|
||||
if lighting { 9 } else { 8 }
|
||||
}
|
||||
|
||||
/// The sequential synthesis stage: the stored frame, the PatchMatch pyramid and its
|
||||
/// coarse-to-fine copies, the warped previous frame, and the sink's prepared copy.
|
||||
const SYNTH_FRAMES: u64 = 6;
|
||||
|
||||
/// Workers a range of `frames` can keep busy (rayon never runs more tasks than frames). The
|
||||
/// scratch estimate must follow the pool that actually runs the parallel stages: a synchronous
|
||||
/// fill stays in the caller's pool, while an asynchronous one runs on `spawn_task`'s own OS
|
||||
/// thread and so uses the global pool, whatever pool the command was called from (#2570).
|
||||
fn fill_workers(frames: usize, wait: bool) -> u64 {
|
||||
let pool = if wait { rayon::current_num_threads() as u64 } else { global_pool_threads() };
|
||||
pool.min(frames.max(1) as u64)
|
||||
}
|
||||
|
||||
/// The global Rayon pool's thread count, read from a thread outside every pool:
|
||||
/// `rayon::current_num_threads()` answers for the caller's pool, not the one the background
|
||||
/// task uses. Falls back to the current pool when the probe thread cannot start.
|
||||
fn global_pool_threads() -> u64 {
|
||||
let (tx, rx) = std::sync::mpsc::channel();
|
||||
match std::thread::Builder::new().name("fill-pool-probe".into()).spawn(move || {
|
||||
let _ = tx.send(rayon::current_num_threads());
|
||||
}) {
|
||||
Ok(handle) => {
|
||||
let n = rx.recv().map(|n| n as u64).unwrap_or_else(|_| rayon::current_num_threads() as u64);
|
||||
let _ = handle.join();
|
||||
n
|
||||
}
|
||||
Err(_) => rayon::current_num_threads() as u64,
|
||||
}
|
||||
}
|
||||
|
||||
/// Native builds spill the input, the membrane-filled neighbours and the propagated copies, and
|
||||
/// pack the holes to one bit per pixel, so only the two packed masks and the two flow fields
|
||||
/// (~1/2 byte a pixel a frame) plus the workers' scratch stay resident (#2570). wasm has no
|
||||
/// spill and keeps the input and the working copies in RAM (~32 bytes a pixel a frame). When
|
||||
/// even that peak is more than `available`, report it up front instead of running out of memory
|
||||
/// partway through (#1789, #2570 review).
|
||||
#[cfg(test)]
|
||||
fn memory_shortfall(frames: usize, w: u32, h: u32, available: u64) -> Option<String> {
|
||||
let need = (frames as u64).saturating_mul(u64::from(w)).saturating_mul(u64::from(h)).saturating_mul(33);
|
||||
memory_shortfall_with(frames, w, h, available, fill_workers(frames, true), false, 0)
|
||||
}
|
||||
|
||||
/// `workers` and `lighting` are parameters so the budget is testable: the scratch term used to
|
||||
/// be a fixed 8 frames, which under-counted a wide machine by several times (#2570 review).
|
||||
/// `resident` adds storage that would otherwise be spill: when the spill volume is RAM-backed,
|
||||
/// its files use the same memory as the working set (#2570 review).
|
||||
fn memory_shortfall_with(frames: usize, w: u32, h: u32, available: u64, workers: u64, lighting: bool, resident: u64) -> Option<String> {
|
||||
let px = u64::from(w).saturating_mul(u64::from(h));
|
||||
let per_frame = (frames as u64).saturating_mul(px);
|
||||
// `holes` and the leftover propagation masks, packed to a bit a pixel each.
|
||||
let masks = per_frame / 4;
|
||||
// The forward and backward flow fields, ~0.125 byte a pixel each.
|
||||
let flows = per_frame / 4;
|
||||
// wasm keeps the input and one working copy in RAM at every stage; a RAM-backed spill
|
||||
// volume counts there too.
|
||||
let resident = resident.saturating_add(if cfg!(target_arch = "wasm32") { per_frame.saturating_mul(32) } else { 0 });
|
||||
// The synthesis stage is sequential; every parallel stage holds a few frames a worker.
|
||||
let scratch_frames = SYNTH_FRAMES.saturating_add(per_worker_frames(lighting).saturating_mul(workers));
|
||||
let scratch = px.saturating_mul(16).saturating_mul(scratch_frames);
|
||||
let need = masks.saturating_add(flows).saturating_add(resident).saturating_add(scratch);
|
||||
(need > available).then(|| {
|
||||
let gb = |b: u64| b as f64 / 1e9;
|
||||
format!(
|
||||
@@ -185,6 +252,40 @@ fn memory_shortfall(frames: usize, w: u32, h: u32, available: u64) -> Option<Str
|
||||
})
|
||||
}
|
||||
|
||||
/// Scratch floor for a native fill: the input, membrane-filled and propagated frames are each
|
||||
/// spilled as f32 RGBA (16 bytes a pixel a frame), but the membrane spill is closed before the
|
||||
/// propagated one is made, so at most two copies exist at once (2 × 16 bytes a pixel a frame),
|
||||
/// plus a tenth for the PNG output.
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
fn scratch_need(frames: usize, w: u32, h: u32) -> u64 {
|
||||
let bytes = (frames as u64).saturating_mul(u64::from(w)).saturating_mul(u64::from(h)).saturating_mul(32);
|
||||
bytes.saturating_add(bytes / 10)
|
||||
}
|
||||
|
||||
/// Refuse up front when the output volume cannot hold the fill's scratch. The folder may not
|
||||
/// exist yet (the spill creates it), so query the nearest existing ancestor. `None` when the
|
||||
/// volume cannot be read; the write itself still reports a real error then (#2570 review).
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
fn scratch_shortfall(frames: usize, w: u32, h: u32, dir: &std::path::Path) -> Option<String> {
|
||||
let need = scratch_need(frames, w, h);
|
||||
// Resolve a relative output folder against the process directory; `parent()` of a bare
|
||||
// name is empty, which would otherwise skip the check.
|
||||
let base = if dir.is_absolute() { dir.to_path_buf() } else { std::env::current_dir().ok()?.join(dir) };
|
||||
let mut probe = base.as_path();
|
||||
while !probe.exists() {
|
||||
probe = probe.parent()?;
|
||||
}
|
||||
let (_, free) = crate::sysinfo::disk_space(probe)?;
|
||||
(need > free).then(|| {
|
||||
let gb = |b: u64| b as f64 / 1e9;
|
||||
format!(
|
||||
"Content-Aware Fill of {frames} frames at {w}×{h} needs up to {:.1} GB of scratch space beside the output, and {:.1} GB is free there: fill a shorter range (set the work area and choose Work Area) or choose an output folder on a larger disk",
|
||||
gb(need),
|
||||
gb(free)
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
/// The status line shown while the frames are filled. Edge Blend calls only the membrane, which
|
||||
/// the device backend leaves to the CPU reference, so it is not a device fill even when a `fill`
|
||||
/// backend is present; Object and Surface run the flow and PatchMatch on the device.
|
||||
@@ -192,6 +293,43 @@ fn filling_status(on_device: bool, method: FillMethod) -> &'static str {
|
||||
if on_device && method != FillMethod::EdgeBlend { "Filling (GPU)" } else { "Filling" }
|
||||
}
|
||||
|
||||
/// The PNGs a fill has written so far. Unless the fill finishes ([`OutputFiles::keep`]), dropping
|
||||
/// it deletes the ones it created, and the output folder too when the fill made it and nothing
|
||||
/// else is in it, so a failed or stopped fill leaves no partial sequence behind (#2570 review). A
|
||||
/// file that was already there (an `outputDir` holding an earlier sequence) is never deleted.
|
||||
struct OutputFiles {
|
||||
dir: std::path::PathBuf,
|
||||
/// The folder was missing before the fill (the spill creates it).
|
||||
made_dir: bool,
|
||||
paths: Vec<String>,
|
||||
/// Paths among `paths` that existed before the fill wrote them: not the fill's to delete.
|
||||
spare: Vec<String>,
|
||||
kept: bool,
|
||||
}
|
||||
|
||||
impl OutputFiles {
|
||||
/// The fill finished: keep its files.
|
||||
fn keep(mut self) -> Vec<String> {
|
||||
self.kept = true;
|
||||
std::mem::take(&mut self.paths)
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for OutputFiles {
|
||||
fn drop(&mut self) {
|
||||
if self.kept {
|
||||
return;
|
||||
}
|
||||
for path in self.paths.iter().filter(|p| !self.spare.contains(p)) {
|
||||
let _ = std::fs::remove_file(path);
|
||||
}
|
||||
// `remove_dir` leaves a folder that still holds anything else.
|
||||
if self.made_dir {
|
||||
let _ = std::fs::remove_dir(&self.dir);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Run a fill: the layer's frames (rendered in passes with a job worker's GPU), the fill, the
|
||||
/// PNG sequence written into `plan.dir` (`Services::store_file`). Returns the files.
|
||||
///
|
||||
@@ -224,17 +362,34 @@ pub(crate) async fn run_plan(
|
||||
let ctx0 = EvalCtx { expr, ..EvalCtx::new(project, cid, comp, Tick::ZERO) };
|
||||
let times = &plan.times;
|
||||
let n = times.len() as u64;
|
||||
let mut frames = Vec::with_capacity(times.len());
|
||||
let mut holes = Vec::with_capacity(times.len());
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
let mut frames = Vec::with_capacity(times.len());
|
||||
// Declared before the spill, so on an early return the spill is closed first and the folder
|
||||
// it was made in can be removed.
|
||||
let dir = std::path::Path::new(&plan.dir);
|
||||
let mut written = OutputFiles { dir: dir.to_path_buf(), made_dir: !dir.exists(), paths: Vec::with_capacity(times.len()), spare: Vec::new(), kept: false };
|
||||
// Spill 32-bit floats whatever the project depth: the old in-memory path kept f32 all the
|
||||
// way through, so quantising here would change the fill (#2570, "no quality loss"). Scratch
|
||||
// lives beside the output when one is set; the default temp folder may be RAM-backed, which
|
||||
// the memory preflight accounts for when it chooses the spill folder.
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
let spill = SpillHandle::create_in(Some(dir), w as usize, h as usize, times.len(), 32)
|
||||
.map_err(|e| format!("spill: cannot make the scratch file in {}: {e}", plan.dir))?;
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
let spill_probe = spill.clone();
|
||||
for (k, t) in times.iter().enumerate() {
|
||||
let (img, hole) = crate::offload::in_passes(accel, |a| masked_frame(&r.with_accel(a), &ctx0.at(*t), layer, w, h)).await;
|
||||
holes.push(dilate(&hole, w as usize, h as usize, plan.expand));
|
||||
holes.push(HoleMask::from_vec(&dilate(&hole, w as usize, h as usize, plan.expand)));
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
frames.push(img);
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
spill.write_frame(k, &img)?;
|
||||
if !progress(k as u64 + 1, n * 3) {
|
||||
return Err(crate::render_queue::CANCELLED.into());
|
||||
}
|
||||
}
|
||||
if !holes.iter().any(|h| h.iter().any(|b| *b)) {
|
||||
if !holes.iter().any(|h| h.any()) {
|
||||
return Err("the layer has no transparent area to fill: draw a mask (Subtract) around what to remove".into());
|
||||
}
|
||||
let mut references = vec![];
|
||||
@@ -249,31 +404,94 @@ pub(crate) async fn run_plan(
|
||||
}
|
||||
let opts = FillOpts { method: FillMethod::from_name(&plan.method).unwrap_or_default(), lighting: plan.lighting, ..Default::default() };
|
||||
message(filling_status(fill.is_some(), opts.method));
|
||||
let input = FillInput { frames, holes, references };
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
let input_frames = Frames::Memory(frames);
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
let input_frames = Frames::Spilled(spill);
|
||||
let input = FillInput { frames: input_frames, holes, references };
|
||||
let cpu = effectcraft_raster::inpaint::CpuFill;
|
||||
let backend: &dyn effectcraft_raster::inpaint::FillBackend = fill.unwrap_or(&cpu);
|
||||
let out = effectcraft_raster::inpaint::fill_sequence_with(input, &opts, backend, &mut |d, t| progress(n + (d as u64 * n / t.max(1) as u64), n * 3))
|
||||
.ok_or(crate::render_queue::CANCELLED)?;
|
||||
message("Writing the fill layer");
|
||||
let dir = &plan.dir;
|
||||
let mut paths = vec![];
|
||||
for (k, img) in out.iter().enumerate() {
|
||||
let path = std::path::Path::new(dir).join(format!("fill_{k:05}.png")).to_string_lossy().to_string();
|
||||
services.store_file(&path, &png(img)?).map_err(|e| format!("{path}: {e}"))?;
|
||||
paths.push(path);
|
||||
if !progress(2 * n + k as u64 + 1, n * 3) {
|
||||
return Err(crate::render_queue::CANCELLED.into());
|
||||
let progress = std::rc::Rc::new(std::cell::RefCell::new(progress));
|
||||
// The fill and the sink both advance the bar: a streamed frame reports ahead of the fill's own
|
||||
// progress, so both report a shared high-water mark and the bar never moves backwards.
|
||||
let high = std::cell::Cell::new(0u64);
|
||||
let report = |done: u64| -> bool {
|
||||
let done = done.max(high.get());
|
||||
high.set(done);
|
||||
(progress.borrow_mut())(done, n * 3)
|
||||
};
|
||||
let paths: Vec<String> = (0..n as usize).map(|k| dir.join(format!("fill_{k:05}.png")).to_string_lossy().to_string()).collect();
|
||||
let mut sink_error: Option<String> = None;
|
||||
let mut sink = |idx: usize, img: &Image| -> std::result::Result<(), String> {
|
||||
let r = (|| -> std::result::Result<(), String> {
|
||||
// A failed spilled read returns a black fallback frame; never publish it (#2570
|
||||
// review).
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
if spill_probe.has_error() {
|
||||
return Err(spill_probe.take_error().unwrap_or_else(|| "spill: read failed".to_string()));
|
||||
}
|
||||
let existed = services.exists(&paths[idx]);
|
||||
services.store_file(&paths[idx], &png(img)?).map_err(|e| format!("{}: {e}", paths[idx]))?;
|
||||
written.paths.push(paths[idx].clone());
|
||||
if existed {
|
||||
written.spare.push(paths[idx].clone());
|
||||
}
|
||||
if !report(2 * n + idx as u64 + 1) {
|
||||
return Err(crate::render_queue::CANCELLED.to_string());
|
||||
}
|
||||
Ok(())
|
||||
})();
|
||||
if let Err(e) = &r
|
||||
&& sink_error.is_none()
|
||||
{
|
||||
sink_error = Some(e.clone());
|
||||
}
|
||||
r
|
||||
};
|
||||
let produced =
|
||||
effectcraft_raster::inpaint::fill_sequence_with(input, &opts, backend, &mut |d, t| report(n + (d as u64 * n / t.max(1) as u64)), Some(&mut sink));
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
if let Some(e) = spill_probe.take_error() {
|
||||
return Err(e);
|
||||
}
|
||||
if produced.is_none() {
|
||||
if let Some(e) = sink_error.take() {
|
||||
return Err(e);
|
||||
}
|
||||
return Err(crate::render_queue::CANCELLED.into());
|
||||
}
|
||||
let produced = produced.unwrap_or_default();
|
||||
// Non-streaming fallback (tests): write any retained images.
|
||||
if !produced.is_empty() {
|
||||
for (k, img) in produced.iter().enumerate() {
|
||||
let path = paths[k].clone();
|
||||
services.store_file(&path, &png(img)?).map_err(|e| format!("{path}: {e}"))?;
|
||||
if written.paths.len() <= k {
|
||||
written.paths.push(path);
|
||||
}
|
||||
if !report(2 * n + k as u64 + 1) {
|
||||
return Err(crate::render_queue::CANCELLED.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(paths)
|
||||
// Every frame must have a file: the fill layer is built on the first one, and a short
|
||||
// sequence would play the wrong frames.
|
||||
if written.paths.len() != times.len() {
|
||||
return Err(format!("Content-Aware Fill wrote {} of {} frames", written.paths.len(), times.len()));
|
||||
}
|
||||
Ok(written.keep())
|
||||
}
|
||||
|
||||
/// Add the fill layer of a finished plan (one undo step).
|
||||
/// The plan comes back from a job worker, so an empty range is an error, not a crash.
|
||||
pub(crate) fn apply_plan(s: &mut Session, plan: &FillPlan, paths: Vec<String>) -> Result<Value> {
|
||||
let importer = s.importer.clone();
|
||||
add_fill_layer(s, plan.comp, plan.layer, &plan.name, paths, (plan.width, plan.height), plan.times[0], plan.times.len(), importer)
|
||||
let start = plan.times.first().copied().ok_or_else(|| bad("contentFill.generate", "the fill has no frames"))?;
|
||||
add_fill_layer(s, plan.comp, plan.layer, &plan.name, paths, (plan.width, plan.height), start, plan.times.len(), importer)
|
||||
}
|
||||
|
||||
/// `contentFill.generate`: validate the layer and range, refuse a fill that cannot fit in
|
||||
/// memory or scratch space, then run it (in the background unless `wait` is set).
|
||||
fn generate(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let c = "contentFill.generate";
|
||||
let (cid, lid) = layer_p(s, p, c)?;
|
||||
@@ -307,17 +525,35 @@ fn generate(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
}
|
||||
let (fa, fb) = (fr.frame_at(fr.snap_nearest(a)), fr.frame_at(b - Tick(1)));
|
||||
let times: Vec<Tick> = (fa..=fb).map(|f| fr.tick_of(f)).collect();
|
||||
if let Some(e) = crate::sysinfo::memory().and_then(|m| memory_shortfall(times.len(), w, h, m.available)) {
|
||||
return Err(bad(c, e));
|
||||
}
|
||||
let opts = FillOpts {
|
||||
method: FillMethod::from_name(&st.method).unwrap_or_default(),
|
||||
lighting: lighting(&st.lighting_correction).unwrap_or(0.0),
|
||||
..Default::default()
|
||||
};
|
||||
let dir = str_p(p, "outputDir").map(str::to_string).unwrap_or_else(|| default_dir(s, &layer.name));
|
||||
// The spill lives beside the output. On a RAM-backed volume (`tmpfs`/`ramfs`, where the
|
||||
// default temp dir may live) its files use the same memory as the working set, so the
|
||||
// memory preflight counts them; disk-backed volumes get the space check below instead.
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
let ram_spill = if crate::sysinfo::is_memory_backed(std::path::Path::new(&dir)) { scratch_need(times.len(), w, h) } else { 0 };
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
let ram_spill = 0;
|
||||
let wait = b_p(p, "wait").unwrap_or(false);
|
||||
// `spawn_task` runs a fill inline on wasm, and `runs_inline` selects the pool it will use.
|
||||
let runs_inline = wait || cfg!(target_arch = "wasm32");
|
||||
if let Some(e) = crate::sysinfo::memory()
|
||||
.and_then(|m| memory_shortfall_with(times.len(), w, h, m.available, fill_workers(times.len(), runs_inline), opts.lighting > 0.0, ram_spill))
|
||||
{
|
||||
return Err(bad(c, e));
|
||||
}
|
||||
let expand = st.alpha_expansion.round().max(0.0) as usize;
|
||||
let reference = st.reference_layer.filter(|r| comp.layer(*r).is_some()).map(|r| (r, Tick::from_seconds_f64(st.reference_time.unwrap_or(0.0))));
|
||||
let dir = str_p(p, "outputDir").map(str::to_string).unwrap_or_else(|| default_dir(s, &layer.name));
|
||||
// Native fills keep up to two f32 copies of the range in scratch beside the output; a volume
|
||||
// that cannot hold them is refused before any frame is rendered (#2570 review).
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
if let Some(e) = scratch_shortfall(times.len(), w, h, std::path::Path::new(&dir)) {
|
||||
return Err(bad(c, e));
|
||||
}
|
||||
let plan = FillPlan {
|
||||
comp: cid,
|
||||
layer: lid,
|
||||
@@ -331,7 +567,6 @@ fn generate(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
reference,
|
||||
dir,
|
||||
};
|
||||
let wait = b_p(p, "wait").unwrap_or(false);
|
||||
// The browser: a job worker (its own GPU), the page keeps running.
|
||||
if !wait && s.offloads() {
|
||||
s.offload_analysis(crate::offload::WorkerJob::ContentFill { plan: Box::new(plan) }).map_err(EngineError::Other)?;
|
||||
@@ -377,12 +612,13 @@ fn add_fill_layer(
|
||||
count: usize,
|
||||
importer: Option<Arc<dyn crate::Importer>>,
|
||||
) -> Result<Value> {
|
||||
let first = paths.first().cloned().ok_or_else(|| bad("contentFill.generate", "the fill wrote no frames"))?;
|
||||
let comp = s.project.comp(cid).ok_or(EngineError::NoComp)?.clone();
|
||||
let src = comp.layer(lid).ok_or_else(|| bad("contentFill.generate", "the source layer is gone"))?.clone();
|
||||
let fr = comp.frame_rate;
|
||||
let dur = fr.tick_of(count as i64);
|
||||
let mut footage = importer.and_then(|i| i.probe(&paths[0]).ok()).filter(|f| f.has_video).unwrap_or_else(|| Footage {
|
||||
path: paths[0].clone(),
|
||||
let mut footage = importer.and_then(|i| i.probe(&first).ok()).filter(|f| f.has_video).unwrap_or_else(|| Footage {
|
||||
path: first.clone(),
|
||||
kind: FootageKind::Sequence,
|
||||
width: size.0,
|
||||
height: size.1,
|
||||
@@ -468,7 +704,11 @@ pub fn specs() -> Vec<CommandSpec> {
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{filling_status, memory_shortfall};
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
use super::scratch_need;
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
use super::scratch_shortfall;
|
||||
use super::{filling_status, memory_shortfall, memory_shortfall_with};
|
||||
|
||||
/// Edge Blend fills with the membrane, which the device backend leaves to the CPU reference:
|
||||
/// it must not claim a GPU fill just because a backend is available.
|
||||
@@ -481,13 +721,88 @@ mod tests {
|
||||
assert_eq!(filling_status(false, FillMethod::Object), "Filling");
|
||||
}
|
||||
|
||||
/// #1789: 10 s of 1080p30 (300 frames) needs at least 20 GB, which 4 GB free can't hold.
|
||||
/// #1789 / #2570: a range that cannot fit is refused up front instead of OOM-ing. The
|
||||
/// scratch term scales with the workers and Lighting Correction, not a fixed frame count.
|
||||
#[test]
|
||||
fn a_fill_that_cannot_fit_in_memory_is_refused_up_front() {
|
||||
let e = memory_shortfall(300, 1920, 1080, 4_000_000_000).expect("refused");
|
||||
let e = memory_shortfall_with(300, 1920, 1080, 1_000_000, 8, false, 0).expect("refused");
|
||||
assert!(e.contains("300 frames") && e.contains("Work Area"), "{e}");
|
||||
assert_eq!(memory_shortfall(30, 1920, 1080, 4_000_000_000), None, "a second of it fits");
|
||||
// Four workers, no lighting: the packed masks and flows plus ~38 scratch frames fit.
|
||||
assert_eq!(memory_shortfall_with(30, 1920, 1080, 4_000_000_000, 4, false, 0), None);
|
||||
assert_eq!(memory_shortfall_with(300, 1920, 1080, 2_000_000_000, 4, false, 0), None);
|
||||
// A wide machine with Lighting Correction holds nine frames a worker: 300 frames no
|
||||
// longer fit in 5 GB, which the old fixed 8-frame scratch term missed (#2570 review).
|
||||
assert!(memory_shortfall_with(300, 1920, 1080, 1_000_000_000, 16, true, 0).is_some());
|
||||
assert!(memory_shortfall_with(300, 1920, 1080, 5_000_000_000, 16, true, 0).is_some());
|
||||
assert_eq!(memory_shortfall_with(300, 1920, 1080, 6_000_000_000, 16, true, 0), None);
|
||||
// The two packed masks still grow with the range: a very long 1080p range cannot fit.
|
||||
assert!(memory_shortfall_with(1000, 1920, 1080, 500_000_000, 8, false, 0).is_some());
|
||||
// A RAM-backed spill volume counts its bytes as resident, so the same range that fit on
|
||||
// a disk-backed volume is refused (#2570 review).
|
||||
assert_eq!(memory_shortfall_with(30, 1920, 1080, 2_500_000_000, 4, false, 0), None);
|
||||
assert!(memory_shortfall_with(30, 1920, 1080, 2_500_000_000, 4, false, 2_000_000_000).is_some());
|
||||
// Hostile sizes saturate instead of overflowing.
|
||||
assert!(memory_shortfall(usize::MAX, u32::MAX, u32::MAX, u64::MAX - 1).is_some());
|
||||
assert!(memory_shortfall(usize::MAX, u32::MAX, u32::MAX, 0).is_some());
|
||||
}
|
||||
|
||||
/// The native spill holds at most two f32 copies of the range beside the output at once (the
|
||||
/// membrane spill is gone before the propagated one is made), plus the PNGs.
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
#[test]
|
||||
fn scratch_space_is_checked_before_a_fill() {
|
||||
let copies = 300u64 * 1920 * 1080 * 32;
|
||||
assert_eq!(scratch_need(300, 1920, 1080), copies + copies / 10);
|
||||
let tmp = std::env::temp_dir();
|
||||
// A range that cannot possibly fit on the volume is refused up front.
|
||||
assert!(scratch_shortfall(usize::MAX, u32::MAX, u32::MAX, &tmp).is_some());
|
||||
// A relative folder that does not exist is resolved against the process directory.
|
||||
assert!(scratch_shortfall(usize::MAX, u32::MAX, u32::MAX, std::path::Path::new("effectcraft-missing-fill-folder")).is_some());
|
||||
// A tiny fill fits.
|
||||
assert_eq!(scratch_shortfall(1, 16, 16, &tmp), None);
|
||||
}
|
||||
|
||||
/// A fill that fails or is stopped after some frames were written deletes them, and the
|
||||
/// output folder it made; a finished fill keeps its files (#2570 review).
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
#[test]
|
||||
fn a_stopped_fill_removes_its_partial_output() {
|
||||
use super::OutputFiles;
|
||||
let root = tempfile::tempdir().unwrap();
|
||||
let write = |dir: &std::path::Path, made_dir: bool| {
|
||||
std::fs::create_dir_all(dir).unwrap();
|
||||
let paths: Vec<String> = (0..2).map(|k| dir.join(format!("fill_{k:05}.png")).to_string_lossy().to_string()).collect();
|
||||
for p in &paths {
|
||||
std::fs::write(p, b"png").unwrap();
|
||||
}
|
||||
OutputFiles { dir: dir.to_path_buf(), made_dir, paths, spare: Vec::new(), kept: false }
|
||||
};
|
||||
// The fill made the folder: the files and the folder go.
|
||||
let made = root.path().join("made");
|
||||
drop(write(&made, true));
|
||||
assert!(!made.exists(), "the partial sequence and its folder are removed");
|
||||
// The folder was already there: only the fill's own files go.
|
||||
let existing = root.path().join("existing");
|
||||
drop(write(&existing, false));
|
||||
assert!(existing.exists());
|
||||
assert_eq!(std::fs::read_dir(&existing).unwrap().count(), 0);
|
||||
// Something else in a folder the fill made keeps the folder.
|
||||
let shared = root.path().join("shared");
|
||||
let files = write(&shared, true);
|
||||
std::fs::write(shared.join("notes.txt"), b"keep").unwrap();
|
||||
drop(files);
|
||||
assert_eq!(std::fs::read_dir(&shared).unwrap().count(), 1);
|
||||
// A file that was there before the fill wrote over it (an earlier sequence in a given
|
||||
// `outputDir`) is not the fill's to delete.
|
||||
let earlier = root.path().join("earlier");
|
||||
let mut files = write(&earlier, false);
|
||||
files.spare.push(files.paths[0].clone());
|
||||
drop(files);
|
||||
assert!(earlier.join("fill_00000.png").exists(), "an earlier fill's frame stays");
|
||||
assert!(!earlier.join("fill_00001.png").exists(), "the fill's own frame goes");
|
||||
// A finished fill keeps its files.
|
||||
let done = root.path().join("done");
|
||||
let kept = write(&done, true).keep();
|
||||
assert_eq!(kept.len(), 2);
|
||||
assert!(kept.iter().all(|p| std::path::Path::new(p).exists()));
|
||||
}
|
||||
}
|
||||
@@ -207,6 +207,75 @@ pub fn disk_space(path: &std::path::Path) -> Option<(u64, u64)> {
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether `path` is on a RAM-backed filesystem (`tmpfs`/`ramfs`), where bytes written count
|
||||
/// against memory as well as space. Linux reads `/proc/self/mountinfo`; other systems report
|
||||
/// `false` (their temporary folders are disk-backed by default). `path` may not exist yet: its
|
||||
/// nearest existing ancestor decides, as when the fill creates its scratch folder.
|
||||
pub fn is_memory_backed(path: &std::path::Path) -> bool {
|
||||
#[cfg(target_os = "linux")]
|
||||
{
|
||||
let Some(canonical) = canonical_ancestor(path) else { return false };
|
||||
return std::fs::read_to_string("/proc/self/mountinfo").is_ok_and(|mountinfo| mountinfo_is_memory_backed(&mountinfo, &canonical));
|
||||
}
|
||||
#[allow(unreachable_code)]
|
||||
{
|
||||
let _ = path;
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
/// The canonical path of `path`'s nearest existing ancestor (or itself). Relative paths are
|
||||
/// resolved against the current directory first, so a new relative output folder still names
|
||||
/// the volume it will be created on.
|
||||
#[cfg(target_os = "linux")]
|
||||
fn canonical_ancestor(path: &std::path::Path) -> Option<std::path::PathBuf> {
|
||||
let absolute = if path.is_absolute() { path.to_path_buf() } else { std::env::current_dir().ok()?.join(path) };
|
||||
let mut p = absolute.as_path();
|
||||
loop {
|
||||
if let Ok(canonical) = std::fs::canonicalize(p) {
|
||||
return Some(canonical);
|
||||
}
|
||||
p = p.parent()?;
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether a `/proc/self/mountinfo` listing mounts `path` on `tmpfs` or `ramfs`; the deepest
|
||||
/// matching mount point wins, and a later entry at the same mount point is the visible one.
|
||||
#[cfg(target_os = "linux")]
|
||||
pub(crate) fn mountinfo_is_memory_backed(mountinfo: &str, path: &std::path::Path) -> bool {
|
||||
let mut best: Option<(usize, &str)> = None;
|
||||
for line in mountinfo.lines() {
|
||||
let Some(mount) = line.split_whitespace().nth(4) else { continue };
|
||||
let Some(fstype) = line.split(" - ").nth(1).and_then(|rest| rest.split_whitespace().next()) else { continue };
|
||||
let mount = unescape_mount(mount);
|
||||
if path.starts_with(&mount) && best.is_none_or(|(longest, _)| mount.len() >= longest) {
|
||||
best = Some((mount.len(), fstype));
|
||||
}
|
||||
}
|
||||
matches!(best, Some((_, "tmpfs" | "ramfs")))
|
||||
}
|
||||
|
||||
/// Undo mountinfo's octal escapes (`\040` space, `\011` tab, `\012` newline, `\134` backslash).
|
||||
#[cfg(target_os = "linux")]
|
||||
fn unescape_mount(field: &str) -> String {
|
||||
let mut out = String::with_capacity(field.len());
|
||||
let mut chars = field.chars().peekable();
|
||||
while let Some(c) = chars.next() {
|
||||
if c == '\\'
|
||||
&& chars.clone().take(3).count() == 3
|
||||
&& let Ok(code) = u8::from_str_radix(&chars.clone().take(3).collect::<String>(), 8)
|
||||
{
|
||||
for _ in 0..3 {
|
||||
chars.next();
|
||||
}
|
||||
out.push(code as char);
|
||||
} else {
|
||||
out.push(c);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// CPU model name.
|
||||
pub fn cpu_name() -> Option<String> {
|
||||
#[cfg(target_os = "linux")]
|
||||
@@ -339,4 +408,42 @@ mod tests {
|
||||
assert_eq!(m.available, 4100 * 16384);
|
||||
assert!(m.is_low());
|
||||
}
|
||||
|
||||
/// The default temp folder may be a `tmpfs`; a nested disk mount under `/tmp` or a nested
|
||||
/// `tmpfs` must be resolved to the deepest mount point.
|
||||
#[cfg(target_os = "linux")]
|
||||
#[test]
|
||||
fn mountinfo_finds_memory_backed_storage() {
|
||||
let mountinfo = "\
|
||||
25 1 0:22 / / rw,relatime - ext4 /dev/nvme0n1p2 rw
|
||||
26 25 0:23 / /tmp rw,nosuid - tmpfs tmpfs rw
|
||||
27 25 0:24 / /tmp/disk rw - ext4 /dev/sda1 rw
|
||||
28 25 0:25 / /mnt/ram rw - ramfs none rw
|
||||
29 25 0:26 / /media/My\\040Disk rw - ext4 /dev/sdb1 rw
|
||||
";
|
||||
assert!(mountinfo_is_memory_backed(mountinfo, std::path::Path::new("/tmp/effectcraft-fill")));
|
||||
assert!(!mountinfo_is_memory_backed(mountinfo, std::path::Path::new("/tmp/disk/fill")));
|
||||
assert!(mountinfo_is_memory_backed(mountinfo, std::path::Path::new("/mnt/ram/a")));
|
||||
assert!(!mountinfo_is_memory_backed(mountinfo, std::path::Path::new("/home/fill")));
|
||||
assert!(!mountinfo_is_memory_backed(mountinfo, std::path::Path::new("/")));
|
||||
// A path that merely shares a prefix is not on that mount.
|
||||
assert!(!mountinfo_is_memory_backed(mountinfo, std::path::Path::new("/tmpdisk/fill")));
|
||||
// Escaped spaces in a mount point decode before matching.
|
||||
assert!(!mountinfo_is_memory_backed(mountinfo, std::path::Path::new("/media/My Disk/fill")));
|
||||
assert_eq!(unescape_mount("/media/My\\040Disk"), "/media/My Disk");
|
||||
// The later entry at the same mount point is the visible mount: a disk mount stacked on
|
||||
// a tmpfs is disk-backed, and the reverse order is memory-backed (#2570 review).
|
||||
let stacked_disk = "\
|
||||
30 25 0:30 / /tmp/stack rw - tmpfs tmpfs rw
|
||||
31 25 0:31 / /tmp/stack rw - ext4 /dev/sdb2 rw
|
||||
";
|
||||
let stacked_ram = "\
|
||||
30 25 0:30 / /tmp/stack rw - ext4 /dev/sdb2 rw
|
||||
31 25 0:31 / /tmp/stack rw - tmpfs tmpfs rw
|
||||
";
|
||||
assert!(!mountinfo_is_memory_backed(stacked_disk, std::path::Path::new("/tmp/stack/fill")));
|
||||
assert!(mountinfo_is_memory_backed(stacked_ram, std::path::Path::new("/tmp/stack/fill")));
|
||||
// A new relative folder resolves against the current directory before walking up.
|
||||
assert!(canonical_ancestor(std::path::Path::new("no-such-relative-folder/fill")).is_some());
|
||||
}
|
||||
}
|
||||
@@ -321,6 +321,58 @@ fn content_aware_fill_removes_an_object() {
|
||||
assert!(s2.execute("contentFill.generate", json!({"wait": true, "outputDir": tmp("caf2").to_string_lossy()})).is_err());
|
||||
}
|
||||
|
||||
/// A scratch folder that cannot be created surfaces as an error, not a cancelled fill or a
|
||||
/// black frame (#2570 review).
|
||||
#[test]
|
||||
fn content_aware_fill_reports_a_spill_failure() {
|
||||
let (mut s, _, _, _) = setup(48, 32, 0.2, |f| {
|
||||
let mut img = Image::new(48, 32);
|
||||
for y in 0..32 {
|
||||
for x in 0..48 {
|
||||
img.set(x, y, bg(x as f32 + f as f32, y as f32));
|
||||
}
|
||||
}
|
||||
img
|
||||
});
|
||||
s.execute("layer.addMask", json!({"rect": [20.0, 10.0, 8.0, 8.0], "mode": "subtract"})).unwrap();
|
||||
let dir = tmp("cafspill");
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let file = dir.join("not-a-folder");
|
||||
std::fs::write(&file, b"x").unwrap();
|
||||
// The fill wants to create its spill here; the parent is a file.
|
||||
let out = file.join("fills");
|
||||
let e = s.execute("contentFill.generate", json!({"method": "object", "outputDir": out.to_string_lossy(), "wait": true})).unwrap_err();
|
||||
// The OS wording differs ("Not a directory" on Unix, "Cannot create a file when that file
|
||||
// already exists" on Windows), so look for the fill's own marker.
|
||||
let msg = e.to_string();
|
||||
assert!(msg.contains("spill: cannot make the scratch file"), "not a spill error: {msg}");
|
||||
std::fs::remove_dir_all(&dir).unwrap();
|
||||
}
|
||||
|
||||
/// A fill plan that comes back without files or frames (a job worker that wrote none) is an
|
||||
/// error, not a crash on the first file (#2570 review).
|
||||
#[test]
|
||||
fn content_aware_fill_without_files_is_an_error() {
|
||||
let (mut s, cid, lid, _) = setup(48, 32, 0.2, |_| Image::filled(48, 32, [0.5, 0.5, 0.5, 1.0]));
|
||||
let plan = crate::commands::content_fill::FillPlan {
|
||||
comp: cid,
|
||||
layer: lid,
|
||||
name: "Clip".into(),
|
||||
width: 48,
|
||||
height: 32,
|
||||
times: vec![Tick::ZERO],
|
||||
method: "object".into(),
|
||||
lighting: 0.0,
|
||||
expand: 0,
|
||||
reference: None,
|
||||
dir: "Clip_Fill_1".into(),
|
||||
};
|
||||
assert!(crate::commands::content_fill::apply_plan(&mut s, &plan, vec![]).is_err());
|
||||
let empty = crate::commands::content_fill::FillPlan { times: vec![], ..plan };
|
||||
assert!(crate::commands::content_fill::apply_plan(&mut s, &empty, vec!["Clip_Fill_1/fill_00000.png".into()]).is_err());
|
||||
assert_eq!(s.project.comp(cid).unwrap().layers.len(), 1, "no fill layer was added");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn content_aware_fill_runs_in_the_background() {
|
||||
let (mut s, _, _, _) = setup(48, 32, 0.2, |f| {
|
||||
|
||||
@@ -147,16 +147,20 @@ fn object_fill_recovers_background_and_is_stable() {
|
||||
holes.push(hole);
|
||||
gts.push(gt);
|
||||
}
|
||||
let input = FillInput { frames, holes, references: vec![] };
|
||||
let out = fill_sequence_with(input.clone(), &FillOpts::default(), g, &mut |_, _| true).expect("fill");
|
||||
let input = FillInput {
|
||||
frames: effectcraft_raster::inpaint::Frames::Memory(frames),
|
||||
holes: holes.into_iter().map(|h| effectcraft_raster::inpaint::HoleMask::from_vec(&h)).collect(),
|
||||
references: vec![],
|
||||
};
|
||||
let out = fill_sequence_with(input.clone(), &FillOpts::default(), g, &mut |_, _| true, None).expect("fill");
|
||||
for t in 0..out.len() {
|
||||
let db = psnr(&out[t], >s[t], Some(&input.holes[t]));
|
||||
let db = psnr(&out[t], >s[t], Some(&input.holes[t].to_vec()));
|
||||
assert!(db > 26.0, "frame {t} pan {pan}: {db:.1} dB");
|
||||
assert_eq!(out[t].data[0][3], 1.0);
|
||||
}
|
||||
if pan == 0.0 {
|
||||
for t in 1..out.len() {
|
||||
let both: Vec<bool> = input.holes[t].iter().zip(&input.holes[t - 1]).map(|(a, b)| *a || *b).collect();
|
||||
let both: Vec<bool> = input.holes[t].to_vec().iter().zip(input.holes[t - 1].to_vec().iter()).map(|(a, b)| *a || *b).collect();
|
||||
let db = psnr(&out[t], &out[t - 1], Some(&both));
|
||||
assert!(db > 30.0, "frames {}–{t} differ: {db:.1} dB", t - 1);
|
||||
}
|
||||
@@ -172,13 +176,17 @@ fn edge_blend_takes_the_cpu_membrane() {
|
||||
let (w, h) = (80u32, 60u32);
|
||||
let gt = background(w, h, 0.0);
|
||||
let hole = rect_hole(w, h, 30, 20, 14);
|
||||
let input = FillInput { frames: vec![cut(>, &hole); 4], holes: vec![hole.clone(); 4], references: vec![] };
|
||||
let input = FillInput {
|
||||
frames: effectcraft_raster::inpaint::Frames::Memory(vec![cut(>, &hole); 4]),
|
||||
holes: vec![effectcraft_raster::inpaint::HoleMask::from_vec(&hole); 4],
|
||||
references: vec![],
|
||||
};
|
||||
let opts = FillOpts { method: effectcraft_raster::inpaint::FillMethod::EdgeBlend, ..Default::default() };
|
||||
let edge = fill_sequence_with(input.clone(), &opts, g, &mut |_, _| true).expect("fill");
|
||||
let edge = fill_sequence_with(input.clone(), &opts, g, &mut |_, _| true, None).expect("fill");
|
||||
for t in 1..4 {
|
||||
assert!(psnr(&edge[t], &edge[0], Some(&hole)) > 40.0, "frame {t}");
|
||||
}
|
||||
assert!(psnr(&edge[0], &membrane_fill(&input.frames[0], &hole), Some(&hole)) > 60.0);
|
||||
assert!(psnr(&edge[0], &membrane_fill(&input.frames.get(0), &hole), Some(&hole)) > 60.0);
|
||||
}
|
||||
|
||||
/// The dilation the level plans are built from (kept honest: the flags' target bit is `near`).
|
||||
@@ -299,11 +307,15 @@ fn sequence_fill_with_the_device_flow_matches_the_cpu() {
|
||||
frames.push(cut(>, &hole));
|
||||
holes.push(hole);
|
||||
}
|
||||
let input = FillInput { frames, holes, references: vec![] };
|
||||
let dev = fill_sequence_with(input.clone(), &FillOpts::default(), g, &mut |_, _| true).expect("gpu fill");
|
||||
let ref_cpu = fill_sequence_with(input.clone(), &FillOpts::default(), &cpu, &mut |_, _| true).expect("cpu fill");
|
||||
let input = FillInput {
|
||||
frames: effectcraft_raster::inpaint::Frames::Memory(frames),
|
||||
holes: holes.into_iter().map(|h| effectcraft_raster::inpaint::HoleMask::from_vec(&h)).collect(),
|
||||
references: vec![],
|
||||
};
|
||||
let dev = fill_sequence_with(input.clone(), &FillOpts::default(), g, &mut |_, _| true, None).expect("gpu fill");
|
||||
let ref_cpu = fill_sequence_with(input.clone(), &FillOpts::default(), &cpu, &mut |_, _| true, None).expect("cpu fill");
|
||||
for t in 0..dev.len() {
|
||||
let db = psnr(&dev[t], &ref_cpu[t], Some(&input.holes[t]));
|
||||
let db = psnr(&dev[t], &ref_cpu[t], Some(&input.holes[t].to_vec()));
|
||||
assert!(db > 40.0, "frame {t} pan {pan}: device vs CPU {db:.1} dB");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -13,3 +13,4 @@ workspace = true
|
||||
effectcraft-geom = { workspace = true }
|
||||
effectcraft-color = { workspace = true }
|
||||
rayon = { workspace = true }
|
||||
tempfile = { workspace = true }
|
||||
+1138
-95
File diff suppressed because it is too large.
Load diff
Reference in new issue
Block a user