Commit Graph
221 Commits
Author SHA1 Message Date
vangel76 4829a83d30 Merge upstream/main into cache-defaults-3298 2026-11-13 12:39:55 +01:00
Dylan M. Taylor 92274ec914 Merge remote-tracking branch 'origin/main' into codex/778-effect-migration
# Conflicts:
#	crates/aep/src/container.rs
#	crates/aep/src/container/footage.rs
#	crates/aep/tests/it/container/robustness.rs
#	crates/model/src/fbx.rs
#	crates/model/src/gltf.rs
#	crates/model/src/obj.rs
2026-11-13 02:26:44 +00:00
Altynzhan KarybaiandClaude Sonnet 5.5 4035d18132 SVG: support static pattern fills and strokes (#2732)
* fix(svg): support static SVG patterns

* fix(svg): cache patterns per viewport

* fix(svg): share cached pattern content

* fix(svg): harden pattern cache keys

* fix(svg): harden pattern inheritance and caching

* Preserve SVG pattern fallback paint

* Format SVG pattern fallback conversion

* Harden SVG pattern resource bounds

* Fix SVG pattern test formatting

* fix(svg): bound hostile path and raster work

* Bound SVG ellipse expansion

* fix(ui-egui): suppress clippy warnings on wasm target

Rename unused `frame` parameter to `_frame` in draw_inner since
pointer_wrap::finish_native is cfg'd out on wasm32. Guard State::disabled
field with #[cfg(any(not(target_arch = "wasm32"), test))] so it only
exists where it's read (finish_frame on native + tests).

* fix: resolve clippy warnings blocking CI

- Rename unused `frame` parameter to `_frame` in draw_inner (wasm target)
- Guard State::disabled field with cfg(native + test) (wasm target)
- Replace chunks_exact(2) with as_chunks::<2>() in SVG polyline/polygon parsing

* fix(svg): budget raster work by affected region and report refusals

Address review on #2732: the finite-work guards charged the whole canvas,
dropping valid sparse art (8x8 image scaled to 1px on 2049x2049, 2x2 rect
under a 1,028-element clip on 512x512, one of six 2x2 rects sharing a 1x1
pattern on 3840x2160).

- Build fill/stroke/image/clip coverage over the device region only and
  charge that region; remaining whole-canvas passes are charged to a
  render-wide pixel budget (caps are not raised).
- Add try_rasterize/try_rasterize_with returning RasterError; rasterize
  keeps its signature. media layered decode now surfaces refusals.
- Pattern content reached through href inherits styles from the owning
  pattern; device_region falls back to the full canvas on non-finite input.
- Document new items and the limits in the svg README; regression tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-12 21:12:42 +01:00
vangel76 dcd46af227 Merge upstream/main into cache-defaults-3298 2026-11-12 12:36:34 +01:00
Dylan M. Taylor a287044644 Media: check still, EXR channel, GIF and WebP canvas sizes before allocating
* fix(media): check still, EXR channel, GIF and WebP canvas sizes before allocating

* address review: GIF frame allocation limit, lower wasm32 still limit, no EXR fallback after a size-limit error

* address review: the 2^25-pixel still limit also applies on 32-bit native targets

* Media: raise the still pixel limit to 2^28 (16384 x 16384) on 64-bit targets
2026-11-12 06:29:46 -05:00
Dylan M. Taylor 6e1dd08147 Import: a 3D model's left-out textures reach the user as an import warning
* Import: a model's left-out parts reach the user as an import warning (Model::warnings, Importer::probe_noting)

* Import: a model's left-out parts reach the user as an import warning
2026-11-12 06:29:14 -05:00
Dylan M. Taylor 1ae04027df fix app adapters for shared format readers 2026-11-12 06:23:01 +00:00
Dylan M. Taylor 3e1965e274 Dependencies: pin craft-libs at current main 2026-11-12 05:34:09 +00:00
Dylan M. Taylor fd1df348c0 Formats: move EffectCraft format codecs to craft-libs 2026-11-12 05:33:48 +00:00
Dylan M. Taylor 79c42749a8 Export, Media: codecs and containers from craft-libs; remove hevcenc, av1enc, vp9enc (#3522)
* Export, Media: H.264, ProRes, HEVC, VP9, AV1, AAC, MP4 and Matroska come from craft-libs

crates/export and crates/media use craft-video, craft-audio and craft-media
(one craft-libs rev) instead of filmcraft-h264enc, -prores, -aac, -isobmff,
-matroska and -opus. The av1enc, hevcenc and vp9enc crates are removed:
craft-video carries the same encoders, with the later changes merged in
storytold/craft-libs#829. Their round-trip tests move to crates/export/tests
and run against craft-video.

opusenc stays (craft-libs has no Opus encoder yet), as do filmcraft-mpegts
and the filmcraft-vp9, -hevc and -av1 decoders the tests use as oracles.

* Pin craft-libs at the main commit that has the encoder colour description (e7c3dda7)
2026-11-11 18:46:50 -05:00
vangel76 9924ebb1e0 Merge remote-tracking branch 'upstream/main' into cache-defaults-3298 2026-11-12 00:05:19 +01:00
vangel76 308b6ec432 #3298: a footage copy whose file can't be deleted stays on the books and the incoming file is refused when its room was not freed (test with an injected deletion failure) 2026-11-11 23:52:22 +01:00
Dylan M. Taylor f0e49012dc Tests: oracle and fixture files go in an exclusively owned temporary directory (#3497) 2026-11-11 14:35:19 -05:00
vangel76 0bc46981f5 Merge upstream/main into cache-defaults-3298 (one RAM budget kept: the pool holds float and 8-bit frames, rows reads and flights; the disk cache writer encodes off the render thread) 2026-11-10 22:35:36 +01:00
vangel76 13ab7dc4ed #3298: the footage localizer makes room for the incoming file before copying it, and the cached free-space reading is debited by what the copies took since (regression test: two copies in one reading) 2026-11-10 21:46:19 +01:00
nazar-gavrylyk 76e4008151 Footage: 8 bpc renders keep plain movie frames as 8-bit levels
An 8 bpc render at Full quantises plain footage straight away, so the
media pool now keeps those movie frames as the 8-bit levels it is
quantised to (a quarter of the memory) and the renderer expands them
into the layer buffer: the same pixels to the bit. Other depths,
reduced renders, masks, colour profiles, fields, frame blending,
Cineon, Invert Alpha, linear light and 10/12-bit or float movies keep
the float path.

First playback of plain 1080p clips at Full caches ~43% faster on the
CPU, and the footage cache holds 4x the frames.
2026-11-10 15:03:10 +01:00
vangel76 89fbb4a7d8 #3334: the stamps test starts from an empty folder and removes it also when it fails 2026-11-09 19:29:21 +01:00
vangel76 b0e96544d3 Merge remote-tracking branch 'upstream/main' into cache-defaults-3298
# Conflicts:
#	crates/media/src/pool.rs
#	crates/render/src/lib.rs
2026-11-09 16:06:35 +01:00
Valentin Klein 7c62272305 .aep import: placed .ai/PDF footage framed by its ArtBox, Null anchors at the corner (#3099) (#3311)
* #3099: placed .ai pages fill the size After Effects stored; null anchors default to the corner

Footage of an .aep keeps the size After Effects stored for a placed PDF / Illustrator / EPS
file (the ArtBox, rounded up), and masks and anchors are measured against it. The page was
drawn as a PDF viewer frames it (CropBox, /Rotate) at its own size, scaled by the width ratio
at full resolution and stretched to the stored aspect below 3/4: masks too large, art off
centre, squeezed at half resolution.

- effectcraft-pdf: Framing::Placed / parse_page_framed (ArtBox clipped to the CropBox,
  /Rotate ignored); boxes with a non-finite corner or span are not used
- Footage::placed_page (set by the .aep importer, serialized only when true): the page is
  framed so and filled into the stored size (effectcraft_render::vector_fit), decoded,
  continuously rasterised and in Create Shapes from Vector Layer; Reload keeps the size;
  the frame cache keys it by size and framing
- .aep import/export: a null's absent Anchor Point is [0, 0], as After Effects reads it
- fixture ai-mask-3099.aep (own synthetic files, scrubbed, profiles stubbed) and tests
  against After Effects 2026's read-back

* #3099: placed-page framing only for .ai and .pdf, EPS as before; doc comments

After Effects' sizing was measured with PDFs named .ai only; an EPS from an .aep keeps its
bounding box at its own size (listed as not measured in docs/gaps.md and docs/aep-import.md).
Doc comments on the touched functions.

* #3099: shapes from a placed page keep its frame (track matte)

Create Shapes from Vector Layer converted every path of a placed page, so bleed, art past the
artboard or the page's background showed where the footage's raster hides them. The layers it
makes from a placed page now take a hidden "<layer> Frame" shape layer (the footage's
width x height) as their Alpha track matte: drawn at the comp's resolution, its edge is the
raster's on shape and image layers alike. The command returns the frame layer's id too.
2026-11-08 21:52:32 -06:00
vangel76 ac208c0d1b Merge remote-tracking branch 'origin/cache-defaults-3298' into crypto-matte-upload-3334 2026-11-09 01:25:43 +01:00
vangel76 5b23f6fe60 #3334: Cryptomatte uploads only the matte, computed at source resolution
The matte (or the Colors display's pixels) is made from the unscaled ID and coverage channels at source resolution, box-filtered below render scale 1, kept per footage frame, layer, selection, output and scale in a cache in the common memory budget, and uploaded to the GPU as one R32F plane. OpenEXR file identity checks are remembered for 1 s; Reload Footage forgets them.
2026-11-09 01:23:16 +01:00
vangel76 6138211804 #3298: cache policy: slow value decay, the process's measured memory, a bounded preview overage, disk writes that pay off, capped open files
- Value decays by ln(seconds unused + 1) + 1 instead of halving every 30 s.
- Each balance reads the process's resident memory (memory-stats, at most every 250 ms): over
  95 % of the RAM left after the reserve the caches give way by what it is over; what they
  freed for it counts as freed; over all of it also protected preview frames go.
- Frames a playing preview shows next go last and may keep the caches over the budget by a
  tenth of it (at most 1 GB).
- Frames and plane sets go to the disk cache only when making them again costs more than
  reading them back (measured read rate, 10 ms per MB until measured).
- At most 256 movie/audio files stay open (least recently used idle one closed).
2026-11-09 01:16:30 +01:00
vangel76 a6ed380638 #3298: the localizer deletes copies without a record and never leaves old bytes beside a new record; set_budget doc says when it frees 2026-11-09 00:39:24 +01:00
vangel76 43fa844e34 Merge remote-tracking branch 'origin/footage-cache-3271' into cache-defaults-3298
# Conflicts:
#	crates/media/src/exr_planes.rs
#	docs/footage.md
#	docs/preferences.md
2026-11-08 18:42:46 +01:00
vangel76 e1c97b4cd9 Footage cache (#3271): no localizer copy runs into a folder being adopted
Adopting a new folder's copies runs outside the localizer's lock, so the worker could start a copy meanwhile: the scan could delete its temporary or take its copy, counting it twice. The worker starts no copy while a folder is adopted (ended by a guard, also on a panic), the scan leaves the files of a copy already running alone, an adopted entry never replaces one made meanwhile, a copy of an earlier setting is not deleted when the adoption took it, and a failed copy of an earlier setting doesn't mark the file skipped. A test holds a folder's adoption, asks for a file meanwhile and checks no copy starts and it is counted once; it fails on the old code.
2026-11-08 18:32:44 +01:00
vangel76 9b3821ff77 Footage cache (#3271): a failed picture doesn't refuse the other channels; the browser plane cap is the footage budget
A remembered failure refused any later request for 'all channels', but those are every channel but the picture's: a picture too large to read left the file's other channels unread. A failure now refuses a request only when it asks for everything the failed one did. The browser's plane cap is 256 MB, the browser's largest footage budget (frames and planes together). A test fails a picture read and then reads the other channels; it fails on the old code.
2026-11-08 18:32:44 +01:00
vangel76 1d6f870e55 Merge remote-tracking branch 'origin/footage-cache-3271' into cache-defaults-3298
# Conflicts:
#	crates/engine/src/settings/session_settings.rs
#	crates/media/src/exr_channels.rs
#	crates/media/src/exr_planes.rs
#	crates/media/src/pool.rs
#	crates/render/src/memory.rs
#	docs/preferences.md
2026-11-08 16:42:46 +01:00
vangel76 60629cb10e Footage cache (#3271): the localizer reads and writes files outside its lock
Setting the localizer adopted a folder's copies (reading the folder and every record, deleting temporaries) while holding the localizer's lock, which every footage read takes to look for a copy; dropping copies over the limit and checking a copy's size did file work under it too. The folder is now read, the free space taken and copies checked and deleted without the lock; the lock only guards the bookkeeping, and a setting changed meanwhile wins.
2026-11-08 16:26:51 +01:00
vangel76 53f59f8742 Footage cache (#3271): a remembered failure is of the file as it was
The plane store remembered failed requests (unreadable, too large) by what they asked for alone: a file replaced on disk without Reload Footage got a new index but kept being refused, so its channels stayed missing and its picture fell back to a whole-file decode every frame. A failure now keeps the index it was made with and only refuses requests of that index; a failure of the new file replaces the old ones. A test fails a broken file, then reads its fixed version under a new index; it fails on the old code.
2026-11-08 16:24:43 +01:00
vangel76 e04d57ce03 Footage cache (#3271): FootageSource::aux stays every channel; effects doc stays generated
MediaPool::aux (no render asks for it: renders ask for what their effects read) again gives every channel, the picture's too, so the pinned OpenEXR golden hashes stay as they were; 'all channels' without the picture applies to the effects' requests. docs/effects.md is generated: the note on static channel selection lives in docs/footage.md.
2026-11-08 16:09:53 +01:00
vangel76 607d29050d #3298: the caches share one memory budget from RAM Reserved for Other Applications
The Layer Cache, Footage Frame Cache and Preview (RAM) Cache settings are gone. The big caches
(layers, footage frames, OpenEXR planes, preview frames, the disk write queue) share one budget:
80 % of the installed RAM left after the reserve (5 GB while RAM is unknown, 3.5 GB in the
browser, at least 192 MB). No cache has a share of its own; the memory accountant frees the
least valuable item first across all of them: cost to make again per byte, decayed by time
unused, a copy worth nothing, a thrashing cache protected. Host sessions (CLI, MCP) apply the
budget from the start. Low-memory halving holds until twice the threshold is available again,
with one toast per episode. Saved settings lose the three old keys and keep the reserve.
2026-11-08 14:24:56 +01:00
vangel76 f21b7d2a8f Footage cache (#3271): drop the plane store's unused release 2026-11-08 13:11:30 +01:00
vangel76 89705dc2c2 Footage cache (#3271): the picture's planes are never kept beside the converted picture
The plane store kept the R, G, B, A planes a picture was made of whenever the picture was read with channels for its effects (or with all of them), beside the converted picture in the media pool's frame cache: the same pixels twice. Once the picture is converted, its planes now leave the stored set (in place when nobody else holds it), but for a picture channel an effect names itself (EXtractoR reading R). 'All channels' means every channel but the picture's. A test reads a picture with one channel, with a named picture channel and with all channels, and checks exactly what stays and its bytes; it fails on the old code.
2026-11-08 13:11:16 +01:00
vangel76 d4cb440527 Footage cache (#3271): decoded OpenEXR channels share the Footage Frame Cache with the frames
The plane store had a fixed 1 GB budget of its own, outside the preferences: with a Footage Frame Cache of 10000 MB it still held about 2.5 frames of a 69-channel 1080p file. The plane store and the media pool's frame cache now share the Footage Frame Cache (set_cache_budget sizes both and the memory accountant's footage group), with one least-recently-used order: each use takes a stamp from one shared counter, and over the footage budget the accountant drops the least recently used item, frame or plane set, whichever holder keeps it. Neither side has a share of its own. The common budget no longer adds a separate plane budget; the browser keeps its 512 MB cap on the planes. Tests: a footage budget of 10000 MB sizes the plane store (and resizes it at run time); filling the frame cache drops the oldest plane sets and filling the plane store drops the oldest frames (fails without the shared order: the frames went first); an accountant group shares its budget.
2026-11-08 13:07:30 +01:00
vangel76 ee2551c8e3 Footage cache (#3271): the localizer test resets its copy hook also when it fails
A drop guard takes the copy hook out and turns the localizer off at the end of the test that installs the hook, also when an assertion fails, so a later test doesn't inherit a hook that holds its copies.
2026-11-08 12:20:09 +01:00
vangel76 74099e32bc Footage cache (#3271): a region of interest asks for rows only where the source reads rows
With a viewer region of interest the renderer asked a PNG still for some rows, which the media pool reads whole anyway, and the layer was left out of the layer cache: its masks and effects ran again on every render. FootageSource::reads_rows tells whether a source reads only the rows asked for (the media pool: an OpenEXR frame whose channels no effect reads); otherwise the renderer asks for the whole frame and the layer stays cached. A test renders a still twice under a region; it fails on the old code (two row reads instead of one whole read).
2026-11-08 10:31:38 +01:00
vangel76 086c53093f Footage cache (#3271): a file being copied by the localizer is not queued again
A read asking for a network file while the localizer copied it queued it again, and the file was copied over the network a second time. The file being copied is now remembered with the setting it is copied under; asked for again under the same setting it isn't queued, under a new one it is. A test holds a copy and asks again; it fails on the old code.
2026-11-08 10:24:43 +01:00
vangel76 643c795bbf Footage cache (#3271): a decode thread lets go of its flight before handing the planes over
The decode thread's job held the flight until it ended, and the flight keeps the outcome for its waiters, so on a busy machine the planes just decoded still looked in use and the store dropped another file's instead (the eviction test failed on CI). The job now drops the flight before it sends the outcome. A test holds the decode thread after it sent; it fails with the old order. 300 runs of both eviction tests pass.
2026-11-08 10:23:14 +01:00
vangel76 28e25246c4 Merge remote-tracking branch 'origin/footage-single-decode-3269' into footage-cache-3271
# Conflicts:
#	crates/media/src/pool.rs
2026-11-08 08:24:24 +01:00
vangel76 bc44712173 Footage: a forgotten file makes only its own decodes stale
forget() bumped one generation for the whole pool, so forgetting any file
(every file-watcher change, every add_bytes) left every other file's
decodes in flight uncached and unjoinable, and requests decoded those files
again. The generation is now an epoch: forget() records it per path, and a
decode, flight or kept picture is stale only when its own path was
forgotten after it started. The per-path map is bounded (4096 paths; past
that, everything older counts as stale).

Refs #3269
2026-11-08 08:23:29 +01:00
vangel76 c236f6e977 Footage cache (#3271): forgetting one file keeps other files' plane decodes (test)
The plane store's staleness is per file: a flight lives in its file's entry and forget drops only that file's entry. This test guards that a decode of one file in flight while another file is forgotten is still kept and joined; it fails when forget drops other files' in-flight entries.
2026-11-08 08:20:24 +01:00
vangel76 ce6add8c6c Footage cache (#3271): a plane request after forget decodes anew (test)
The plane store keeps its flight in the file's entry, and forget drops the entry, so a request made after a forget never joins a decode of the old file. This test guards that: it fails when forget keeps an entry with a flight.
2026-11-08 06:41:17 +01:00
vangel76 25af122235 Merge commit '19c9390e' into footage-cache-3271
# Conflicts:
#	crates/media/src/pool.rs
2026-11-08 06:41:17 +01:00
vangel76 19c9390e2d Merge remote-tracking branch 'upstream/main' into footage-single-decode-3269
# Conflicts:
#	crates/media/src/pool.rs
2026-11-08 06:38:39 +01:00
vangel76 6038833359 Footage: a request after forget doesn't join a decode started before it
A request made after its file was forgotten (it changed on disk) while the
file's decode was in flight joined that decode, got the old pixels and kept
them in the frame cache as the new file's. Each flight now carries the
generation it started in; a request only joins a flight of its own
generation and otherwise decodes the file anew.

Refs #3269
2026-11-08 06:31:56 +01:00
vangel76 cfc184e114 Footage cache (#3271): queued plane writes own their planes
Review fixes for the disk plane writes:
- A queued write holds copies of the planes it writes instead of the
  plane store's whole set, so it counts exactly what it holds, the
  memory accountant's trim frees what the queue reports, and it never
  keeps the store from letting a set go.
- A set is queued in writes of at most 256 MB, and only a single plane
  larger than the cache is left out, so a large set (a 4K file of many
  channels) reaches the disk plane by plane instead of being refused
  whole.
- Each plane entry counts as one encode again.
- A plane entry is shuffled straight from the samples' bits, without
  the widened intermediate copies.
2026-11-08 06:21:48 +01:00
vangel76 88fa8ccf15 Footage cache (#3271): the memory accountant frees queued plane sets
A queued plane write shares the plane store's set, which keeps the store
from letting go of it; trimming the disk queue now drops such writes
too, so the store can free the set in the same balance. The reader uses
saturating offsets, and the batch test fails a later batch by position.
2026-11-08 06:10:35 +01:00
vangel76 dfe9aedfe4 Footage cache (#3271): faster OpenEXR reads and disk plane hits
- The reader reads the next batch of chunks while the one before
  decompresses, reads chunks that follow each other in the file in one
  request, starts with a small batch so decoding begins early, places
  the planes in parallel, and converts half samples a slice at a time.
- A plane entry stores a half channel as half floats when every value
  round-trips bit for bit (NaN planes and float data stay f32), with a
  plain byte-plane shuffle; one entry per channel, read in parallel on a
  hit. Old multi-plane entries (format 5) read as misses.
- The planes of one decode go to the disk writer as one job that shares
  the plane store's set instead of a copy per channel; the queue counts
  their bytes once the store lets go of them.
- The plane store loses its unused level, and one eviction loop serves
  the budget and the memory accountant; dropping for bytes keeps
  remembered failures.
2026-11-08 06:01:48 +01:00
vangel76 d91f5d28fd Footage cache (#3271): stage B review fixes
- A region of interest reads rows only at the top level (no collapsed precomps, no nested
  comps, no adjustment layers); input and footprint keys never keep a rows-only buffer; a rows
  request takes the whole frame from the frame cache when it is there and keeps nothing itself.
- Sequence layers are keyed by their frame (without frame blending or fields), so times within
  a frame share the entry.
- A decode taken over by a decode thread runs under the asker's cancel flag and hints.
- Disk planes: one entry per channel, read back whatever was asked for before; queued planes
  are moved in (counted exactly), written by channel with checked math; the write refused when
  a plane is of another size. Segmentations are compressed on the writer thread.
- Memory accountant: the plane budget is in the sum; the disk queue counts only bytes it alone
  holds and never drops frame writes; one balance at a time, inserts balance at most every
  100 ms, a refilling holder stays protected for 10 s.
- Reload Footage gives thumbnails new keys; thumbnail keys are worked out at most every 0.25 s.
- Localizer: network footage locations (Settings > Disk, e.g. P:\) besides UNC paths; copies
  recorded beside them and adopted after a restart, stale temporaries deleted; the disk cache's
  reserve kept free; by default beside the disk cache's folder, 20 GB; a copy gone meanwhile
  reads the share; status shown translated, with the network locations.
- Tests: hooks hold the queued decode for the claim test; the disk test uses its own paths.
2026-11-08 03:59:09 +01:00
vangel76 c1534f4502 Footage cache (#3271): stage B self-review fixes
- A region of interest looks up the whole layer's cached buffer first; only a miss reads the
  rows (and keeps nothing).
- The rows margin counts the region's rounding at the layer's own scale; the region test covers
  shrunk and enlarged layers at reduced resolution.
- Layer and plane writes are bounded by the bytes they hold while queued (1 GB).
- The disk cache's drop generation is taken before the file is opened, so planes of a file
  reloaded meanwhile are refused.
- A plane entry of another shape than the planes asked for is a miss before anything is
  decompressed.
- The claim hand-off is one atomic phase (queued, running, claimed): no gap between the claim
  and the queued job.
2026-11-08 03:17:34 +01:00