mirror of
https://github.com/storytold/effectcraft.git
synced 2026-11-05 18:18:49 +00:00
gmrmad1andFredApps
389894e69b
* gpu: run Content-Aware Fill's PatchMatch on the device Content-Aware Fill's synthesis is the stage that makes the tool slow: at 1920×1080 with a 300×200 hole, PatchMatch costs ≈ 1.1 s per frame on the CPU, and it is what Object, Surface and every refining frame spend their time in. Add a `FillBackend` trait to `effectcraft-raster` whose stages (the membrane solve, the flow fields, PatchMatch) default to the CPU reference, and `Accelerator::inpaint()` on the compositor, which is how the engine asks for a device backend. `effectcraft_gpu::inpaint` implements PatchMatch: the multi-scale pyramid, the EM rounds, the propagation (a two-pass checkerboard instead of the CPU's scan order), the random search, the SSD, the histogram-based σ² and the weighted vote, as compute kernels in `shaders/inpaint.wgsl`. The fill's frames are read back to the CPU before it runs (analyses composite on the CPU by design), so the backend is about the fill's own work; the orchestration, the flow-guided propagation and the per-frame flow control stay on the CPU. Only the hole's bounds cross back to the CPU, which is what makes the offload pay. Measured on an Intel Arc (MTL) iGPU, 1920×1080, 300×200 hole: CPU 1087 ms GPU 397 ms (2.7×, 41.0 dB PSNR against the CPU's synthesis) The membrane (harmonic) solve deliberately stays on the CPU: the same measurement has it at ≈ 90 ms on the device against ≈ 55 ms on the CPU, because a fill's full-frame data movement (upload, the known pyramid, a copy) does not shrink with the hole, while the relaxation does. The device takes only the stage where its parallel search pays; `membrane_tex` still runs on the device inside PatchMatch, where it is the coarsest level's start and never crosses back. Closes #1839. * gpu: fix the fill's vote histogram and run its flow on the device Review follow-ups on the Content-Aware Fill GPU port, plus the fill's flow. Fixes: * imp_hist kept about one sample in 256: each of the 256 threads scanned a strided subset of the pixels and retained only those whose bin selected it, so the vote's sigma² came from a sample correlated with the pixel index (and a small hole usually fell back to 1.0, making the weights near-uniform). One workgroup now accumulates the whole population through shared atomic bins, with a test that pins the sample count. * imp_pm_prop and imp_pm_rand were dispatched over grow(near, 1) but with near's origin, so the ring's left column and top row were never written (NaN under test, stale pooled data in production) while a strip past it was. `over` now derives the shader's origin from the box it sizes the dispatch from, so a dispatch's region and its origin can no longer disagree. * Gpu::inpaint() requires its kernels to have built. Otherwise every stage paid an upload and a failed dispatch to fall back to the CPU, while the fill still reported the GPU path. * imp_rnd's XOR chain parenthesises its multiplicative terms: WGSL's grammar makes each XOR operand a unary expression, so `a * b ^ c * d` is rejected by spec-compliant parsers (naga accepts it). * FillBackend documents that its stages may be called concurrently. * The coarsest membrane's sweep count (64, against the CPU's 200) is now justified: matching 200 measured ~6% slower over the fill with no change in the result. And the flow: Gpu::flow runs block_flow's coarse-to-fine search, sub-pixel fit and vector median on the device. The luma is the CPU's (flow::luma_pair), so the field matches block_flow's to a float rounding and the propagation sees the same motion; the hole completion stays on the CPU with the membrane. A four-frame 1920x1080 fill goes from 1.27x to 1.42x against the CPU, and the device path is covered for holes from 16x16 to 300x200. * gpu: keep the fill vote's histogram range on HDR input imp_hist binned every distance at a fixed 256 bins per unit and clamped to 1023, so the vote's sigma2 topped out at 3.998. A 32-bpc frame's channels can exceed 1, and masked_frame forwards them unclamped: a patch distance of 1600 then reported sigma2 ~ 4, exp(-1600/(2*sigma2)) rounded to zero in f32, and imp_pm_vote kept the prior image instead of synthesising. The histogram now reports the largest finite distance it saw (atomicMax on the float bits, carried in bin 0's unused .g channel). SDR input keeps the fixed 0..4 range with identical bins; HDR input is re-binned over the observed range and percentile() maps the bin centre back through it. Nothing clamps the input. Also group over/sweeps/pm_level's arguments into small structs: the workspace denies too_many_arguments and the lint policy (#2496) does not allow new #[allow]s. No behaviour change on those paths. * engine: don't report Edge Blend fills as device fills Edge Blend calls only the membrane, which the GPU backend leaves to the CPU reference, so run_plan's "Filling (GPU)" status was wrong for it whenever a backend was present. Object and Surface still run the flow and PatchMatch on the device and keep the label. CodeRabbit's #1856 finding. * gpu: run the fill's device stages inside error scopes An allocation or validation failure inside patch_image/flow_image left an invalid texture whose later use raised an uncaptured error, retiring the shared compositor instead of falling back to the CPU for that frame. Wrap both stages in Gpu::within_memory, whose scopes are native thread-local, so the fill's rayon workers do not share them. --------- Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>