mirror of
https://github.com/storytold/effectcraft.git
synced 2026-11-13 13:52:16 +00:00
0a6d264554a7f5ce7fc767b5ae90fb112e6441f9
257
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Valentin KleinandFredApps
|
a4bde72bca |
.aep: import twice as fast, faster export (#3302)
* aep: a layer's markers and Source Text are read where they are stored, not searched for (#3276) A layer's marker list is a direct child of its root tdgp, and its Source Text record btds a direct child of the ADBE Text Properties group. Markers walked the whole layer subtree twice per layer (most layers have no marker list, so the walk always ran to the end), and the three text-document accessors walked it once each. * aep export: build the prototype template once, sort names with cached keys (#3277) The template every export clones from is immutable; it is now built by the first export of the process and kept (about 0.93 MB). The name sort builds each natural key once instead of on every comparison. Root solids move into the Solids folder instead of being copied. The two key-list rewrites share one emit-local helper and match names compare as bytes instead of a decoded String. Output is byte-identical. * aep container: size sibling lists, hex and write output once; borrow plain .aepx attributes (#3275) - read.rs: count the chunk headers of a sequence first and allocate the children Vec once (bounded by the input and by the chunk limit). - aepx.rs: decode_hex sizes its output once; attribute values without an entity are borrowed instead of copied through decode_entities. - typed.rs: PropGroup::get compares match names without building a String (same result as the lossy compare, invalid UTF-8 included). - write.rs: reserve the exact output size up front (try_reserve, so a hand-built tree cannot abort on the allocation). - read.rs, settings.rs, values.rs: use the shared bytes.rs readers. Output is identical on the corpus (tree, written bytes, import). * aep container: review fixes (#3275) - read.rs: reserve with try_reserve_exact, so a raised chunk limit on a small target cannot abort before parsing; the capped reservation and a deep, retry-heavy nest of broken lists are pinned by tests. - write.rs: a tree past the 32-bit container size fails with TooLarge before anything is serialised. - aepx.rs: drop an unneeded local and lifetime annotation. * aep export: note what the concurrency test covers and why a failed template build is not kept (#3277) * aep: text documents are read without a String per token, defaults built once (#3276) TextDocument::parse reads a borrowed tree: names, bare words and strings without escapes point into the body, keys are borrowed when they are /names, numbers parse from the borrowed bytes, and dictionaries and arrays are moved into vectors of their exact size when they close. The style and paragraph defaults are built once instead of per document. parse_text_dict keeps its public TextValue result (the borrowed tree turned owned). The old owning parser stays as a test-only oracle: a differential test compares both on 40 000 generated bodies (every key form the owned keys canonicalise, non-UTF-8 names, escapes, unclosed strings, non-finite numbers), every cut of a realistic document and nesting around the depth limit. * aep: text-dict test builds its prefix without repeat(1) (#3276) * aep: an escaped text string drops its byte order mark without a range drain (#3276) * aep export: one key-time loop, key-list rewrite, tdmn decoder and byte writer for the listed modules (#3278) In masks, shapes, styles, synth, fxsynth, base, camlight, skeleton, textanim and tree (effects.rs, plan.rs and emit.rs keep their copies for the follow-up work): - New export/keys.rs: the key-time loop (timed_keys), the key-list rewrite (fill_key_list, set_key_list, replace_cdat) and the spatial key record, shared by masks, shapes and styles. - One tdmn decoder (synth::tdmn_name, lossy UTF-8 up to the first NUL, as the importer reads it) and one encoder (synth::tdmn) instead of the copies in masks, styles, shapes, textanim, camlight and skeleton. - One byte writer: tree::put, which errors. base::bytes, synth::tdb4, synth::key_list, fxsynth::pard, masks::shph and the keyed path records write through it; their fixed layouts keep the infallible signatures behind tree::fixed, which logs, and a test builds every layout and requires no error. - camlight::put_group uses the shared group split (tree::split_group), which errors on a match name without a node instead of stopping quietly. Export output is byte-identical on the corpus and the synthetic fixtures. * aep export: pin the key-time loop's report, name the layout in the base::bytes log (#3278) * aep export: review fixes for #3278 - tree::Fixed replaces tree::fixed: a fixed layout writes every field that fits (as the old code did) and keeps the first error; finish() logs it once with the layout's description and counts it per thread, so the every-layout test cannot race with other tests. - base::bytes logs one line naming its length and fields. - masks::build_atom errors when the prototype has no Mask Shape instead of dropping the path of every mask quietly. - log sorted into the dependency list. * aep: review fixes for #3276 text documents and markers text_document reads the first btdk in the Source Text record, the documents text_document_count counts. The old slot checks (same, is_neutral, undecoded_fields) join the test oracle and are compared on 20 000 generated slot dictionaries; mixed dictionary/array chains and a document left open at the depth limit join the parser differential. An escaped string's byte order mark is drained without a copy. * aep: remove test-only container accessors, share report and time-base code (#3281) - Remove 35 public container accessors that only tests called (header, camera/light, keys, footage, masks, shapes, version) and the dead StyleMap::switch_name. Tests now check the importer's own reading of the same bytes, or the decoder it uses; three import tests gain the values the removed checks covered. - Import and export reports share one loss entry (Loss<C>) with record, total, count and summary; the JSON shape is pinned for both. - Collector::lose_with builds a detail only below the detail cap (callers switch later). - The export uses the container's TimeBase; one NTSC table serves import and export; TimeBase::frames saturates like units instead of returning 0. * aep: review fixes for #3281: NTSC table beside TimeBase, real-path NTSC round trip, light and camera defaults on import, doc wording * aep: drop a local check tool committed by mistake (#3281) * aep export: sort keys as one compact byte string, bounded by the name (#3277) The cached natural keys were Vecs of (bool, String, u128) segments: a 255-byte name of alternating letters and digits took about 14 KB, held for every item at once. Each key is now one byte string whose byte order is the natural order, at most 5 bytes per name byte. Order and output are unchanged. * aep container: check bdata digits and size before reserving (#3275) decode_hex reserved the decoded size before it knew the value decodes, so a large bdata with a bad digit reserved half its length first. It now refuses a value larger than a chunk's 32-bit size field (TooLarge), checks every digit in one vectorisable pass, and only then reserves, with try_reserve_exact so a value that does not fit in memory is an error instead of an abort. * aep export: the every-layout test counts from a baseline, not from zero (#3278) * aep export: correct the order the natural sort key doc gives for equal numbers (#3277) * aep: the format notes name the accessors that remain (#3281) * aep: the format notes name the 1 x 1 mask scale of text and shape layers (#3281) * aep: read imports through a flat tree index (#3300) Import reads an .aep through TreeIndex, one depth-first array of chunks that borrows the file's bytes, instead of building and freeing the owned Chunk tree (one Vec per list). The parser mirrors read::Parser step for step: the same chunk count order against max_chunks, the same depth limit and opaque fallbacks, the same errors. The typed view reads through Node, a handle to a chunk of either tree, so the accessors work unchanged on both; .aepx keeps its owned tree. Chunk, parse and write stay for export and editing. The array starts small and doubles, never past what the input can hold or the chunk limit allows; a failed allocation is TooLarge, a tree past u32 indices TooManyChunks. * aep export: the time remap and Transform keys use the shared key-list and match-name helpers; doc comments for the audit's functions (#3300) * #3300: import .aepx through the flat index The .aepx reader no longer builds the owned chunk tree element by element. It decodes every body (hex, text, alias JSON, numbers) into one buffer and records the chunks as depth-first entries whose bodies are ranges of that buffer; TreeIndex::from_drafts turns them into the index the importer reads, which borrows the buffer. read_aepx converts that index into the owned tree in one pass. The tree and the errors are those of the previous reader, which the tests keep as the reference: generated and damaged documents under seven limit sets, invalid UTF-8, hostile input, the fixture directory, and imports through both. Bound tests pin the chunk list to the input and the limit, and check that dropped content leaves nothing behind. * #3300: read_aepx keeps its own owned-tree reader read_aepx, read_aepx_with_limits and parse_any build the owned chunk tree element by element again, as before the .aepx index, instead of converting the index (which held the decoded buffer and the owned tree at the same time). Import keeps the index reader. The tests now compare the two production readers. * #3300: test keys that round to the same time * aep: compare the destination projects in the index-equivalence tests both() and both_aepx() kept only the ImportResult of each path; they now keep both destination Projects and assert structural and serialized equality, so a layer or property value regression cannot pass on equal ids and reports. --------- Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> |
||
 Dylan M. Taylor
|
79c42749a8 |
Export, Media: codecs and containers from craft-libs; remove hevcenc, av1enc, vp9enc (#3522)
* Export, Media: H.264, ProRes, HEVC, VP9, AV1, AAC, MP4 and Matroska come from craft-libs crates/export and crates/media use craft-video, craft-audio and craft-media (one craft-libs rev) instead of filmcraft-h264enc, -prores, -aac, -isobmff, -matroska and -opus. The av1enc, hevcenc and vp9enc crates are removed: craft-video carries the same encoders, with the later changes merged in storytold/craft-libs#829. Their round-trip tests move to crates/export/tests and run against craft-video. opusenc stays (craft-libs has no Opus encoder yet), as do filmcraft-mpegts and the filmcraft-vp9, -hevc and -av1 decoders the tests use as oracles. * Pin craft-libs at the main commit that has the encoder colour description (e7c3dda7) |
||
|
Dylan M. Taylor
|
cbddc9d284 |
Fix Flatpak updates and bump to 0.24.1 (#3516)
* fix(flatpak): allow release checks and self-update helpers * test: keep Flatpak permission parsing within strict Clippy rules * chore: bump version to 0.24.1 for the Flatpak update fix |
||
|
Dylan M. Taylor
|
6ff564e750 |
SVG: bound tessellation, kept geometry, attributes, entity expansion and re-rendering (#2986)
* SVG: bound tessellation, kept geometry, attributes, entity expansion and re-rendering Circles and rounded rects flattened at a fixed 0.01 tolerance, so a huge radius made unbounded curves; the tolerance now grows with the radius as arcs already did. Every path a <use> copies and every gradient reference's stops were parsed and kept without a document-wide limit; both now share a budget of 8 million items. An element with thousands of attributes made the XML parser's duplicate check quadratic, and repeated entity references could expand attribute text to gigabytes below the node limit; a preflight refuses more than 1,024 attributes per element and entity references that expand past 64 MiB. Nested knockout and non-isolated groups re-rendered their children several times per level; node draws are now held to 64 per node of the tree. * address review: entity_bytes takes the largest declaration of each name * address review: drop tessellation-tolerance, attribute-count and entity-size refusals; keep retained and draw budgets, warn when the draw budget is hit |
||
|
Dylan M. Taylor
|
f0e49012dc | Tests: oracle and fixture files go in an exclusively owned temporary directory (#3497) | ||
|
Dylan M. Taylor
|
dfbd4a8e82 | Updates: take craft-update from craft-libs main at 6e03f65bf | ||
|
FredApps
|
230b3920de | Release: EffectCraft v0.24.0 (#3494) | ||
|
FredApps
|
8b9f9e6943 |
Capture observed process and GPU failure evidence (#3409)
* #3357: retain process, memory and typed GPU failure evidence * #3357: tighten reporter protocol regression cases |
||
 nnaannoo
|
87000339ed |
Build: drop the unused egui patches and fail CI on [[patch.unused]] (#3404)
Removes the unused egui_demo_lib, egui_inspection and egui_system_font_provider patches and adds cargo xtask lock-patches to cargo xtask ci. Fixes #3403. |
||
|
nnaannoo
|
f31463abfe |
Build: take craft-libs at e11f9d9a so its workspace loads with current Cargo (#3351)
Moves the six craft-libs 92d392c pins to e11f9d9a so cargo vendor (and the Nix flake) can load the craft-libs workspace. Refs #3291. |
||
 FredApps
|
360e21f539 | Release: EffectCraft v0.23.1 | ||
 dotCooCoo
|
9253b5bdc6 | Updates: take craft-update from craft-libs main at 1b34eba41 | ||
 Valentin KleinandClaude Opus 5.5
|
84382487bb |
Disk cache: entries keep the files they read, keys from memory (#3184) (#3191)
* Disk cache: record the source files each frame reads Part of #3184 (step 1): renders now report which footage files they read, each with its identity (size and modification time) as read. Nothing uses the list yet: images, cache keys and cache behaviour are unchanged. - render: `reads` module with `FileStamp`, `ReadFile`, `ReadFiles` (sorted, deduplicated, bounded; marked incomplete when it can't vouch for every file) and the thread-safe `ReadLog`. `Renderer::reads` collects a frame's files; `FootageSource::recording` lets a source note what it reads. `file_stamp` uses the same identity. - Layer cache entries keep the files their buffer was made of and report them on every hit; derived entries (styles, blend space, flattened) keep their base's. Buffers from the disk cache or the browser store don't know their files yet and mark the list incomplete. - Media pool: every file read notes its identity, taken from the file it opens. Decoded frames, opened movies, GIFs, models and EXR channels keep it with the cache entry, so a frame served from memory reports its file without touching the disk. - Viewer frame worker collects each frame's list and carries it to where the disk entry is written (step 2 stores it). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Automation: confined footage passes on the files it reads Part of #3184: ConfinedFootage forwards FootageSource::recording to the source it wraps, so renders of an automation session list their files instead of coming back incomplete. The read tests also render through the viewer's CancellableFootage wrapper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Disk cache: frame entries keep their source files, comp key from memory Part of #3184 (step 2). Fixes #3175: opening a comp no longer stamps every footage file of the project before its first frame can show. - The comp's content key (`disk_cache::comp_content_key`) is taken from the project in memory alone: no footage or proxy file is stamped, no `VirtualFileId` is asked. `comp_content_key_with` is gone; `frame_key` hashes a new entry version, so entries of the earlier format are never looked up again (LRU evicts them). - Frame entries (formats 2 and 3) store the files the frame read (`EntryFiles`: size and modification time as read; size and content hash through `VirtualFileId` for files from memory; missing files), then the pixels. Formats 0 and 1 read as misses and are deleted. - Check on hit: `get_frame`/`get_frame_f16` look only at the listed files (one metadata call each) on the frame worker. A changed, back or unreadable file, or a damaged entry, is a miss: the entry is deleted and the frame rendered and stored again. - A frame whose file list is incomplete is shown but not stored. The RAM preview keeps each frame's files, so `persist_missing` writes only frames with a complete list. - Remote frame worker: `FrameServer` records each frame's reads (`take_files`); the web worker stores them with the entry and the page checks them through the file table before using it. The browser's media epoch existed only for the old key and is removed. Export does not use frame entries. - The cache indicator still only asks whether an entry exists. Tests: entry format round trip and damage, the check per file, and in the viewer: a changed file re-renders exactly its frames (and a reopen reads the rest from disk); a frame checks only its own files in a project with a large unused sequence and the key looks at no file; missing, unreadable files and damaged entries never serve a stale frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Disk cache: layer entries keep their source files, no footage salt Part of #3184 (step 3 of 4). Layer buffers on disk (and in the browser's layer store) now carry the files they were made of, like frame entries since step 2, and are checked against exactly those files when used. The project-wide footage salt is gone: nothing stamps every footage file any more. - Layer entry format 4: the buffer's EntryFiles, then the pixels. A hit checks only the listed files (metadata, or the VirtualFileId hook for files from memory; Missing must still be missing); a change, a damaged entry or the old format 0 is a miss and the entry is deleted, so the re-rendered buffer replaces it. A hit reports its file list, so frames made of buffers from disk are complete and get stored again. - Buffers whose files can't all be named are not written. - Layer disk keys: layer_disk_key = hash of the layer entry version, the renderer version and the layer cache key (which names the footage items with their paths and settings, from memory). - PrefetchStore checks on use; an entry made of changed files is dropped and forgotten, so the new buffer is queued over it. FrameServer passes its virtual-file hook to its layer cache. - Removed: footage_salt(_with), file_stamp, footage_stamp, set_disk_salt, set_store_salt, Session::refresh_disk_salt, poll_disk_salt, footage_fingerprint, disk_salt, disk_salt_background, disk_salt_job, DiskSaltJob, the UI's per-frame poll, the salt in FrameServer and in the availability worker (with its cache_enabled plumbing). The disk cache is attached as soon as it is configured. - Docs: architecture.md, web.md. Tests: layer entries check their files (size, time, missing/back, folder, damaged, old format, incomplete); virtual files; prefetch store; media tests with a real pool across restarts (only the changed file is read, projects with the same item ids never share an entry, missing, unreadable and damaged never serve stale buffers); engine tests that the cache is attached at once and edits do no file work (#1926, #3106). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Disk cache: no-salt worker test, sequence paths in the frame-blend key Part of #3184 (step 3 of 4). - A frame-blended footage frame's layer-cache key now hashes the sequence's file list: with no project-wide salt, two projects whose items share an id and a first frame must not share its disk entry. - Test: a frame worker syncing, patching and rendering a project with an unused 5000-frame sequence asks its file hook only about the file its buffer reads (the salt asked about every file at every sync). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Disk cache: the comp key covers what its expressions can read With an expression anywhere in the project, every comp's disk key hashed every item of the project, so any edit (another comp, an unused footage item) threw away every comp's frames on disk. The key now covers what the comp can draw plus what the expressions among it can reach (Project::pixel_reach): - without expressions: the items the comp draws, without silent layers, plus the footage its precomp layers swap in (Media Replacement); - with expressions: every layer of the comps they reach, their markers, the comps and footage they name with string literals (comp("Name"), footage("Name")) together with which items have those names now, and the footage named like a reached source (its data is read by name); - the whole project when an expression computes a name, uses the object model's constructors or internals, dynamic code, computed keys or writes, or text the scanner can't read for sure (regex literals, template strings, escapes): expression_reach errs towards reaching more. Markers count only where an expression can read them (#1083). Memory only, no file is looked at. Tests: scanner and reach unit tests in the project crate; engine tests for the key (own-comp expressions keep the key through edits elsewhere, comp("Other") follows Other and not a third comp, a second comp taking the name counts, a computed name counts every item, markers read from another comp); a ui-egui test that the disk frames outside an expression layer are carried when the comp it names changes. Part of #3184. Part of #1492. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Disk cache: Reload Footage drops the entries of its files, docs File > Reload Footage now deletes the frame and layer entries whose file lists name the reloaded item's files (DiskCache::drop_files), so a file replaced by one with the same size and modification time, which the check on hit can't tell apart, is read again. The index knows keys, not files: the writer thread reads the files section at the start of each frame and layer entry (not the pixels) after the writes queued before it, and deletes the entries that list one of the files. Until it is done, an entry made of one of them is a miss and frames aren't linked to new keys. The entries are gone for good, so they don't come back after a restart. Automatic reloads don't drop anything: they follow a changed stamp, which the check on hit catches. The browser needs nothing (its file identities are content hashes). Removed: disk_cache::seal, which steps 2 and 3 left without callers (the tests use seal_format). Updated the module, seal_format and forget_media docs, and docs/architecture.md / docs/web.md describe the final design: what is cached, keys from memory, file lists checked on hit, nothing scans the project, incomplete lists stay in memory, Reload Footage drops entries, edits keep the frames they can't change. Tests: render unit tests (same size and time passes the check but not the drop; other files' entries stay; an entry queued before the drop is dropped; a layer entry too; written again afterwards it is used; the drop survives a restart; a pending drop is a miss), and an engine test through file.reloadFootage. Closes #3184. Part of #1492. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Disk cache: file stamps keep the whole modification time A file's modification time was nanoseconds since 1970 as u64: a time before 1970 read as "no time" (two such files of one size looked alike) and times after 2554 were truncated. It is now i128 nanoseconds, negative before the epoch; None only means the file system has no time. The content-key hash writes a presence tag and the full value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Automation: a refused footage read leaves the render's file list incomplete Footage outside the read roots shows nothing and was noted as no file at all, so a picture made without it had a complete, empty list and could be kept on disk for a later session with other roots. A refused read during `recording` now marks the list incomplete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Render: a buffer kept by another worker comes with its own files LayerCache::insert returns the files of the buffer it keeps, and Renderer::keep notes them in the render's log and returns them, so a render that uses another worker's buffer lists that buffer's files (a file changed between the two reads then has two identities and the list is incomplete). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Render: sampleImage and text fonts note the files they read EvalCtx carries the render's read log. A sampleImage render records its footage reads, keeps them with its per-thread cached buffer and notes them in the evaluating render's log on every use. Text notes the font files it is drawn with, with the identity their bytes were read with; a family that isn't installed, a fallback or missing glyph, or a font from memory leaves the list incomplete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Disk cache: frame entries store the whole modification time A frame entry's file list stores a file's modification time as i128 nanoseconds since 1970 (negative before), as the file stamps now keep it; the frame entry format version goes to frame-entry-files-2 so entries in the old layout are not read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Disk cache: layer entries store the whole modification time Layer entries share the frame entries' file list layout, which now stores the full i128 modification time: the layer entry format version goes to layer-entry-files-2. The module doc names the font files text is drawn with among an entry's source files, and what keeps a list incomplete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Layer cache: buffers that read other layers are keyed by the switches passed down A layer whose effect reads another layer (Gradient Wipe, Set Matte, Blend…) gets that layer drawn by its own renderer. Through a precomp layer, Switches Affect Nested Comps, the switches inherited from the precomp layers above and motion blur on change those pixels, but the key held only the layers' own switches: a buffer made with the setting on was reused (from memory or the disk cache) with it off. Such layers now derive their key with what the renderer passes down. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Layer cache: masked buffers are keyed by the mask motion blur gate A mask with Mask Motion Blur On blurs even when the layer's own switch is off, as long as the render's gate is open (Render Settings motion blur, and with Switches Affect Nested Comps the precomp layers above). The key held the layer's own motion blur state only, so a buffer blurred with the gate open was reused with it closed (and the other way round), also from the disk cache. Masked layer, input and footprint keys now tell a closed gate apart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Render: cached buffers list what their expressions and substituted characters read - Buffers made under `Renderer::recorded` (layer content, a Time effect's input, an adjustment layer's footprint) evaluate their masks, shapes and effects with the context that notes into the buffer's own read log, so a file an expression reads (`sampleImage`, also at shutter times mask motion blur adds and the cache key doesn't see) is in the buffer's files. - A styled layer served from the cache lists the files of the cached body and passes too, so the buffers made of it (flattened, blending space) don't get a complete list when one of them came with an incomplete one. - A character put in by Character Offset or Character Value without a relayout notes the font it is drawn with; a fallback face (or none) leaves the list incomplete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Render: text lists the font file of a run that draws no glyph; tests clean their temp dirs An empty first paragraph draws no glyph, but in a box its face sets the first baseline that places every later line. The read list took faces only from glyphs, so it could be complete without that file. Note each style run's resolved face as well. The font-reads and confine tests use tempfile::tempdir(): a fresh directory per run, removed on panic too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Disk cache: Reload Footage closes the gaps around its drop (#3184) - A reload while the disk cache is off is kept on the session (pending_disk_drops) and applied when the cache is switched on, before anything is read from it. - Each render takes the cache's drop generation before it reads (frame worker; layer cache at the miss). An entry listing a file dropped after that is refused under the queue's lock, so a render that was under way during the reload can't store pixels from before it. Such a frame is shown with an incomplete file list, so persist_missing skips it too. The per-file record is bounded (past it, everything older is refused). - An entry read while its drop was pending is a miss even when the writer ends the sweep between the read and the check (a stale hit found under load): the pending drops and the generation are taken before the entry is read. - Test: a reopened project serves its frame from disk with no source file opened or decoded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Disk cache: RAM frames keep the drop generation their render started at Reload Footage empties the RAM preview, but an idle pass of persist_missing may already hold a frame from before the reload. It took the cache's drop generation when it ran, after the drop, so the old pixels could be written back and, with the file's size and time unchanged, served again after a restart. Each RAM frame now keeps the disk cache its render started with and that cache's drop generation then. persist_missing writes a frame only to that cache, with that generation, and skips it once one of its files was dropped since; a frame rendered without that cache is not written from there. Tests use tempfile directories (the layer drop test, the frame_files folders under target/). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Doc comments for the functions this PR touches * Disk cache: cap the free-space reserve at 10 GB, skip encodes the limit refuses The reserve the cache leaves free on its disk was max(2 GB, 5% of the disk) (#2551). On a 4 TB disk with 190 GB free that is 200 GB, so a cache set to 20 GB had a limit of 0: every write was refused, the cache never held anything, and only a log warning said so. The reserve is now clamp(5% of the disk, 2 GB, 10 GB). It is there to keep the cache from filling the disk, not to keep a share of a large disk free: 10 GB leaves the system room to work on any disk, a small disk still keeps 2 GB, and a 100 GB disk keeps 5 GB as before. After Effects has only a maximum cache size and no reserve; a clamp keeps the floor for small disks, where a fixed 10 GB would take most of the space. Writes the limit refuses cost no work: DiskCache::accepts_writes (one atomic read) is checked before a frame is converted (put_disk_frame) and before frames and layers are compressed (put_frame, put_frame_f16, put_layer). The frame workers spent about 8 ms per frame and 14 ms per layer on encodes that were then refused. persist_missing returns at once at limit 0 instead of converting the RAM frames on every idle pass. Settings > Disk shows the limit in effect in MB under Maximum Disk Cache Size, with a note when free disk space lowers it (settings.disk.cacheLimit). * Disk cache: refuse writes below 64 MB of room, show this disk's reserve Review follow-ups to the reserve fix: - accepts_writes also refuses while free disk space leaves less than 64 MB of room (a limit the setting asks for is always used): a few frames would fill it, and the rest would be compressed only to be refused. - While writes are refused the idle passes ask for no repaint, so the UI keeps a frame coming every 5 s: the free space is read again and the RAM frames are written once there is room. - Settings > Disk names the space kept free on this disk (the reserve the last reading applied; the rule from MIN/MAX_DISK_RESERVE when it is unknown), and the "no room" note follows accepts_writes. - The settings.disk.cacheLimit automation label is machine-readable and the same in every language: limitMb, maxMb, reserveMb, spaceLimited, writes. - cargo fmt. * Disk cache: follow 3D model sources in the comp's reach The reach walk followed comp, footage and solid sources but not LayerSource::Model. An expression reading a model layer's source (thisComp.layer("Model").source.name) left the model item out of the comp's content key, so renaming the model kept the key and the old frames on disk were served. Every source item is followed now (LayerSource::item), which also counts the model when no expression reads it (Advanced 3D draws it). The same omission is fixed in the RAM preview's content walk (frames.rs) and in the layer key's walk of referenced layers (cache/dependencies.rs). * Disk cache: a render's drop generation travels with its reads; no disk entries for Expression Selectors Drop generation per render: the layer cache recovered a buffer's drop generation from the latest miss of its key, so a second render of the key that started after File > Reload Footage lent its generation to a render still running from before, and that render's buffer (the file as it was) was written after the sweep. Now Renderer::recorded starts each buffer's read log at the store's drop generation before anything is read (ReadLog::starting_at); ReadFiles carries the earliest generation through merges, so derived buffers (styles, the blend-space copy, the flattened layer) and frames that use a buffer from memory carry it too. LayerCache::insert keeps a buffer made of a file dropped since its reads began neither on disk nor in memory; a list that doesn't say counts as made before every drop. DiskCache::dropped_since takes the list's own generation too. missed_at keeps only the miss time. Expression Selectors: the layer key evaluates a text Expression Selector's amount for the first character only, so a per-character dependency such as textIndex == 1 ? 0 : footage("ref.png").width was not in the key, and after the footage item's width changed a reopened session reused the old layer entry. A text layer with an Expression Selector now marks its reads unkeyed (ReadFiles::is_unkeyed), which merges like the generation; such buffers stay in memory and are never written to disk. Frame entries are unaffected: the comp's frame key hashes what the expressions name. The equality and hash of ReadFiles ignore both new fields. * Disk cache: never serve a RAM layer buffer made before a drop of its files A render that passed the drop check could sit in put_layer's encode while Reload Footage cleared the layer cache and dropped the file, then publish its old pixels to memory after the purge; get, and a fresh render's insert, then served them. The disk-hit promotion went through the same insert_mem. Every way the cache hands out a RAM buffer (get, the kept-first path of insert, an existing entry in insert_mem) now checks the entry's files against the store's drops from their own generation (ReadFiles::since): an entry made before a drop of one of its files is forgotten and the lookup misses, so a fresh render's buffer replaces it. insert_mem also re-checks a buffer just before publishing it. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Valentin KleinandClaude Sonnet 5.5
|
fdcc715829 |
Viewer exposure on the full float range of 32 bpc frames (#2937)
* Viewer exposure on the full float range of 32 bpc frames (#2797) 32 bpc viewer frames on the desktop are rendered without the HDR step (skip_hdr_step, #2916) and kept as half floats in the RAM preview. A display pass on egui-wgpu's device turns each frame into one RGBA8 texture with mips per viewer, in this order: decode, exposure in linear light, Compand / Tone Map, encode, display colour, Show Channel, rounding. Before, exposure was an 8-bit lookup applied after tone mapping, clamping and the display conversion, so -2 EV could not bring values above 1.0 back. - engine: viewer::DisplayPass describes the pass as data; its pixel() is the CPU fallback (no device) and the shader's reference. It replaces exposure_lut / display_transform. FrameEncoding records a frame's curve and the HDR step it still needs. 32 bpc snapshots are taken without the HDR step; snapshot_rgba8 applies it. - ui-egui: FrameImage::CpuF16 (half crate, F16C at run time), byte accounting at 8 bytes a pixel in the cache and the prefetch estimates; display_gpu.rs + display_gpu.wgsl; main viewer, snapshot and passive viewers draw through the pass; the 8-bit image other panels read is made on demand with the HDR step; the Info panel shows float values. - Half-float frames are not written to the disk cache yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Viewer display pass: f16 disk cache entries, window-device gating, review fixes - Disk cache: half-float frame entries (format byte in the entry header, byte planes + LZ4); 8-bit entries keep format 0 and stay valid. 32 bpc viewer frames are written and read again, and the timeline's disk bar sees them. A damaged half-float entry is deleted. - The display pass runs on the window's egui-wgpu device, with or without the GPU compositor (still building, failed, GL fallback). Its pipelines are made inside error scopes; a backend that rejects them gets the CPU pass instead of a blank viewer. - A new LUT of the same size (another ICC profile, Preserve RGB, another display) runs the pass again. - f32 to f16: one pass per L1-sized block, the NaN / infinity fix-up on the bits, no fill of the buffer (4K: 4.7 ms, as the 8-bit conversion). - Mips of odd-sized levels include the last column / row; the snapshot's half floats and texture go with the snapshot; no placeholder texture ids; u64 size checks. - Tests: LUT profiles and negative values against the shader, Bake reruns, the passive viewer's colour, f16 disk round trips; exact comparisons allow 1 LSB on software rasterizers. Docs: the display pass and the f16 disk entries in architecture.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Viewer f16 disk-cache test: temp dir removed on failure too, wait for the queued disk write Review points: the test's directory was only removed by its last statement (now tempfile::TempDir), and the frame is released before its disk write is queued, so flush() could return early (now waits for the entry). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Viewer: oversized half-float frames use the CPU display pass, so exposure/HDR come before the averaging down Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Viewer: fmt and a debug assert for the half-float upload path Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Disk cache: persist_missing also writes half-float frames Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Passive viewer: oversized half-float frames take the CPU display pass instead of leaving a stale texture Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
 
|
4a89e68f9f |
Content-Aware Fill: spill the range and stream the output (#2570 phases 2-4) (#3079)
* Content-Aware Fill: bit-pack holes, spill frames to disk, stream outputs (#2570 phases 2-4) * Content-Aware Fill phases 2-4: bit-pack holes, spill frames, stream outputs (clippy clean) * Address CodeRabbit review: spill safety, streaming correctness, perf, memory guard (#2570) * Address the remaining CodeRabbit review findings on Content-Aware Fill (#2570) A non-streaming fill returned empty frames, progress could jump backwards between the sink and the fill, the memory guard ignored the membrane frames still resident for the flow pass (and wasm's in-RAM copies), and the GPU fill stages ran outside an error scope, so an OOM retired the shared context instead of falling back to the CPU for that frame. 🤖 Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Content-Aware Fill: sample spilled frames without decoding them, check spill errors (#2570) Tracing sampled spilled frames by decoding whole frames under the shared cache lock, so parallel traces across neighbouring frames evicted each other and mostly paid a full-frame read while every other thread waited. Sample only the 2x2 bilinear support with positioned reads on an open handle instead, and report a failed spill read in the no-hole branch rather than streaming black frames as a successful fill. 🤖 Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Content-Aware Fill: spill membrane/propagated frames, unquantized input, surface scratch errors (#2570) Flow and propagation now write their intermediate frames to scratch spills beside the output instead of holding the whole range as float RGBA, and the input spill uses 32-bit floats whatever the project depth so the fill sees the same values as the old in-memory path. Spill reads/writes and scratch creation report through a shared error slot that run_plan surfaces instead of masking disk failures as a cancellation. Sampling uses positioned reads; the guard models the packed masks plus per-thread scratch. * Content-Aware Fill: report a membrane-spill write failure instead of cancelling (#2570) * Content-Aware Fill: include the flow fields in the memory guard (#2570) * Content-Aware Fill: scale the memory guard with workers and check scratch space (#2570) The guard's scratch term was a fixed 8 frames, which under-counted a wide machine with Lighting Correction by several times. It now models the sequential synthesis stage plus a few frames per worker (from available parallelism) and takes Lighting Correction into account. Native fills write three f32 copies of the range to scratch beside the output, so a range that cannot fit on that volume is now refused up front with the free space in the message, before any frame is rendered. Tests cover the worker/lighting budget, the scratch formula, a shared error slot between the input and secondary spills, an uncreatable spill folder, and the engine surfacing a spill failure as an error instead of a cancellation. * Content-Aware Fill: resolve a relative output folder for the scratch check (#2570) A bare relative output name has an empty parent, so the walk-up found no existing ancestor and skipped the free-space check. Resolve it against the process directory first, and cover that in the test. * Do not reformat the GPU fill tests in the spill branch (#2570) The file was already not rustfmt-clean upstream, so the formatter only added noise to a file this PR no longer needs to touch: phase 1's call-site changes are already on the base branch. * Content-Aware Fill: measure peak RSS and correct the per-worker scratch model (#2570) The ignored measure_peak_rss_of_a_fill test fills a spilled 640x360 range and reports VmHWM. On this 18-worker machine 90 frames peak at 340-470 MB across runs (35 MB with RAYON_NUM_THREADS=1) and 600 frames at 582 MB, so the scratch slope is ~7 frames a worker, not the 2 the first guard revision assumed. Scaling the summary down showed a clean linear slope at 8 and 4 workers, and Lighting Correction stayed below the flow stage's peak. Budget 8 frames a worker (9 with Lighting Correction), just above the measured top of the range, and adjust the guard test to the real numbers. At 600 frames the measured peak is 582 MB against ~11.1 GB for the old five-copies model, a 19x reduction. * Contributors: credit Madhav Agarwal in the About window Adds the self-chosen real and display name for @gmrmad1. The generated contributors.json picks it up once the PR is merged. * Address CodeRabbit: size the guard from the rayon pool; migrate GPU tests (#2570) fill_workers used available_parallelism, but the parallel fill stages run on the rayon pool, which RAYON_NUM_THREADS can size larger. Use rayon::current_num_threads so the scratch budget follows the threads that actually run; wasm keeps rayon's single-thread fallback. The GPU inpaint tests still used the pre-spill FillInput API. The earlier 'do not reformat' revert dropped their API migration along with the formatting, leaving cargo test -p effectcraft-gpu uncompilable. Restore the Frames::Memory / HoleMask / sink migration and verify the 15 inpaint tests. * Fix gate: flatten the wasm prop lookup and keep the budget wrapper test-only (#2570) * Address CodeRabbit review: spill error check, global-pool preflight, docs (#2570) * Include RAM-backed spill in the preflight and guard streaming reads (#2570) * Resolve relative spill paths, prefer stacked mounts, guard the Edge Blend sink (#2570) * Make the fill spill anonymous and test the streaming claims (#2570) The spill was a named file in the output folder that only Drop removed, so an out-of-memory kill or a force-quit left a multi-gigabyte effectcraft-fill-* file in the user's project folder, and a failed set_len after keep() leaked it too. The spill is now an anonymous temporary file (tempfile_in), which the system reclaims when its last handle closes; every read and write is positioned on that one handle (pwrite/seek_write join the existing pread/seek_read), and the folder is kept only so the propagated frames spill beside the input. Also: - Edge Blend checks for a failed read before handing a frame to the sink, so a black fallback frame never streams. - The propagation workers stop as soon as the spill records an error. - HoleMask::get/set use get()/get_mut(), so a mask whose public words are shorter than len cannot panic. New tests cover the PR's headline claims: a spilled range fills bit-identically to the same frames in RAM (returned and streamed), the sink sees frames strictly in order, a truncated spill fails with a recorded error and streams nothing, a failure mid-stream stops before the next frame, propagation stops after a failure, the spill has no name in its folder, and HoleMask round-trips at word boundaries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Content-Aware Fill: clean up a stopped fill and count two scratch copies (#2570) - A fill that fails or is cancelled after frames started streaming deletes the PNGs it wrote, and the output folder when the fill made it and it is left empty (OutputFiles, dropped after the spill so Windows can remove the folder). - After a successful fill, run_plan checks that every frame has a file, and apply_plan/add_fill_layer return an error on an empty range or file list (a job worker's reply) instead of indexing [0]. - scratch_need charges two f32 copies (32 B/px): flow_pairs drops the membrane spill before the propagated one is created, so the third copy never coexists. - The spill-creation error carries a stable "spill: cannot make the scratch file" marker; the panel test asserted on the OS wording, which on Windows is "Cannot create a file when that file already exists". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Content-Aware Fill: a stopped fill leaves files it didn't create (#2570) A fill into a given outputDir that already holds an earlier sequence overwrites its frames as it goes (as before). The cleanup of a failed or stopped fill deleted every path it wrote, including those earlier frames, so a layer still using that sequence lost them. It now deletes only the files the fill created (CodeRabbit on #3079). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Codebuff <noreply@codebuff.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
 skylar taylor-barrick
|
a408d990b2 |
Export: use conformant ProRes alpha run termination (#2225)
* #293: use conformant ProRes alpha run termination Pin the direct ProRes encoder to FilmCraft e4be1577 while preserving all other codec pins. Add a full MOV export regression for final alpha runs across both 4444 profiles, alpha modes and blur settings. * #293: document ProRes alpha export regression * export: repin FilmCraft to the merged ProRes alpha fix and move the round-trip regression into roundtrip.rs All FilmCraft deps pinned to d5ead4ba (squash merge of filmcraft #2991). H.264 pinned movie bytes move 104341 -> 104342: only the encoder version string in the SEI (0.8.0 -> 0.12.0) and five size fields change. |
||
 
|
a439b1bb19 |
Effect ▸ Install Effect Plug-in… and plug-in bundles (wasm + ScriptUI panel in one go) (#2749)
* plugins: install effect plug-in bundles (wasm + ScriptUI panels) in one go * plugins: address review - keep listing errors, refuse replacing an installed plug-in * plugins: report unreadable bundle entries when their metadata cannot be read * plugins: surface partial install errors in the menu bar, cover the install flow with tests * plugins: surface partial install errors for async dialog answers too * plugins: show result errors as compact status text * plugins: harden installs - confined automation, no panel overwrite, size cap, cleanup errors * plugins: bound install reads from one handle, detect unreadable panels, clarify install docs * plugins: drop an unneeded mut in the bounded read * plugins: saturate the bounded read limit * plugins: install into the session's Plug-ins folder, refuse hidden single installs and case-variant panels - Install .wasm files into Session::plugins_dir() (the folder the Plug-in Library from #2524 uses), not a path rebuilt from the settings folder; the host tests now point both at their temp folder so they never touch the real settings. - A single install of a hidden file (".x.wasm") was skipped and reported as a success that installed nothing: it is now an error. - Installed names must be bare file names (and not Windows device names on Windows), as file.installScript requires, so every installed panel is listed and can be uninstalled. - A panel whose name differs from an installed one only in case replaced it on Windows and macOS: compare ignoring case and check the file on disk too. - The loaded plug-in's source is the installed copy, not the file it came from. - Document the `bundle` menu parameter, so checked execution accepts it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * plugins: drop needless to_vec in the bundle install test (clippy) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * plugins: keep the loader test's byte vectors, drop to_vec in the install test (clippy) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: STAR Web Studio <hello.starwebstudio@gmail.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
611a282bea |
Plug-ins: external effect helpers (safe, out-of-process native effects) + plug-in API 2 (#889)
* Plug-ins: API 2 (PluginHost, padding, layer_span) Split out of the OpenFX work so it can be reviewed on its own. Additive; the WebAssembly ABI stays at 1. - point3 / layer / text / hidden parameters, integer sliders, parameter groups. - EffectPlugin::padding (transparent pixels around the buffer, at most 4096 per side). - EffectPlugin::render_v2 with a PluginHost: the input at other times, other layers, parameters at other times, frame rate, per-instance key, the layer's source range. - EffectEnv::layer_span from the renderer; MAX_PLUGINS 1024. - plugin_error: a plug-in effect's last render error, for the UI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Plug-ins: external effect helpers (crates/helper-host) Native plug-in formats run in separate helper programs, never in EffectCraft's process (docs/adr/0001-external-effect-helpers.md). New safe-Rust crate effectcraft-helper-host (L3, native only, unsafe forbidden): it finds helper programs, starts each as a child process and registers its effects through plug-in API 2, so the UI and command surface need nothing new. - Protocol v1 (docs/external-helpers.md): JSON lines over stdin / stdout (helper replies are prefixed so plug-in chatter is ignored); pixels go through a frame exchange file (/dev/shm on Linux), never through the pipe. Render-time callbacks for the input at other times, other layers and parameters at other times, with nested renders through the same helper. - A helper that exits, hangs past its timeout or breaks the protocol is stopped. The render fails with a message, the frame is left as it was, and the helper restarts for the next render. An effect that stops its helper 3 times is turned off (helper_blocklist.txt). - Opt-in: nothing starts helpers unless the user asks (Effect > Scan for Effect Helpers), or opts in to starting them at launch with EFFECTCRAFT_HELPERS=1, an environment variable, so writing to the settings folder alone can't run code. effect.helpers.scan / load are refused for confined agent sessions (confine.rs), like file.executeFile. - Discovery: helpers/ next to the app (Contents/Helpers on macOS), helpers/ in the settings folder, EFFECTCRAFT_HELPER_PATH. - Commands effect.helpers.list / scan / load; Effect menu entries (translated); the last render error of an effect shows as a warning sign after its name in Effect Controls. - serve(): the helper side for Rust helpers. tests/it/main.rs runs the whole protocol (render, padding, callbacks, nesting, threads, crash, restart, block list, hang) against a fake helper, with nothing to install. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Effect helpers: address review (start failures, frame file mode, nextest listing) - A helper that can't start (not installed any more, restarting too often, no frame exchange file) is reported but no longer charged to the effect being rendered: it is Failure::Refused, not Failure::Broken, so it never counts toward turning that effect off. New check in tests/it: once the helper is out of restarts, renders of an effect fail without blocking it (fails without the fix: the effect was turned off after 3 such renders). - The frame exchange file is created owner-only (0o600) on Unix: it holds the user's frames and /dev/shm is shared. Asserted in the frames unit test. - tests/it (harness = false) speaks the libtest listing protocol cargo-nextest uses: `--list --format terse` prints `helper_protocol: test` (nothing with `--ignored`), and `helper_protocol --exact` runs the checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Effect helpers: bound writes to a helper, drain stderr, keep callbacks in step - Writes to a helper go through a writer thread with the request timeout: a helper that stops reading its stdin (hung with a large callback answer or render request pending) no longer blocks the render thread forever while holding the helper. - stderr is drained as bytes (lossy, in 64 KiB pieces) to the end: output that isn't UTF-8 stopped the reader, closing the pipe under the helper (EPIPE / SIGPIPE on its next write); lines were unbounded. - Callback images that can't be written are answered with `null` instead of failing the render while the helper waits (which left the protocol out of step); a frame that can't be written before the request is sent is not charged to the effect. - Mid-render calls use the render's own process: no respawn or heartbeat ping while the helper is waiting for an answer. - Waiting for a busy helper is bounded (load timeout) instead of forever; status() no longer blocks on a running render. - Reap the child when start-up fails after spawning; cap the block list read. - Scripts start helpers only with Allow Scripts to Write Files and Access Network on (they run native code), as for File.execute(). - helper-host declares the workspace's no-panic crate lints; the desktop main() keeps its doc comment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Effect helpers: rustfmt the fake provider, build the test session in its initializer (clippy) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Plug-in Library: offer WebAssembly API 1 releases only; a helper already stopped mid-render isn't charged again - WasmPlugin loads only modules at the WebAssembly ABI version, which stays 1 while PLUGIN_API_VERSION is 2: share it as effects::plugin::WASM_API_VERSION and have the library's compatibility check use it, so it never installs an API 2 release that can't load. - with_running: a process that is already gone was stopped by what broke it (a nested render, charged there), so this render fails without charging its own effect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * frames: plug-in effects count as reading other frames (plug-in API 2) With main's narrowed frame invalidation (#2795), an edit inside a precomp under an effect that reads its layer at other times must reach the layer's whole span. Plug-in API 2 gives every plug-in effect the input at other times, so any effect outside ec. counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> |
||
|
30d6c6e1d3 |
Frames cache: edits keep the frames they can't reach (#2795)
* frames: keep cached frames an edit can't reach Nudging, trimming or splitting a layer re-rendered the whole comp. The RAM preview now carries every frame whose stack of active layers is unchanged to the new comp identity; only the segments an edit touches re-render. * disk cache: never drop frame writes; save frames kept after edits Frames get their own write lane (bounded by bytes, written before layer buffers), so a burst of layer writes no longer drops frames and leaves gaps in the blue bar after reopening. Frames an edit kept in RAM are written to disk under the comp's new content key when idle. * frames: music and hidden layers keep the caches Layers that can't change pixels (audio-only footage, AV layers with Video off, when nothing reads them and there are no expressions) no longer count for the RAM preview's dirty segments or the disk cache's comp and footage keys. Adding, trimming or moving music keeps both caches. RENDER_VERSION bumps to effectcraft-render-3 for the new key formula. * ui-egui: background caching benchmark cache_bench fills a comp's RAM preview the way idle caching does and prints frames/s with decoder, footage-cache and disk-cache counters. * frames: soloed music and hidden layers aren't silent Solo hides every other layer, so soloing a music track or a layer with Video off changes the picture. silent_layers no longer drops soloed layers, so the RAM preview and the disk cache see Solo toggles. * disk cache: audio files a layer reads stay in the layer salt Audio-only footage left the disk layer salt so importing music keeps the keys, but Audio Spectrum and Audio Waveform draw from an audio layer. Footage another layer reads (or any, when a read layer is a precomp) counts again, so overwriting that file doesn't reuse stale buffers. * disk cache: retry frame writes the queue refused put_frame reports whether the frame is on disk or queued. persist_missing forgets the key of a refused write, so a full frame lane no longer leaves RAM frames that are never written. * ui-egui: cache_bench measures idle caching; checks --comp and --slots Frames go through request_background with one slot per frame thread, as Cache Frames When Idle does. An unknown --comp is an error (the active comp is used only without --comp), --slots is at least 1, and the disk report flushes queued writes instead of sleeping. * disk cache: RENDER_VERSION effectcraft-render-6 This branch changes the cache-key formula (silent layers, Solo, read audio in the footage salt), and upstream #2759 already took render-5 for the clip-clock change, so entries from either must not be reused. * frames: rewrite RAM frames the disk cache evicted persist_missing kept every disk key it ever queued, so a frame the disk cache evicted later was never written back. The disk cache now tracks frame writes still queued or in progress (frame_pending), and persist_missing only remembers keys while it converts them. * frames: idle passes continue while RAM frames wait for disk persist_missing reports whether there may be more to write (a content key being computed, frames being converted). Cache Frames When Idle then schedules another pass, so a refused write is retried without waiting for a UI event, and the passes stop once every frame is on disk or queued. * frames: rewrite an evicted frame once, not on every idle pass A disk cache smaller than the RAM preview, or a frame too large for it, made idle passes write, evict and rewrite the same frames every 500 ms. Each disk key is now written at most twice under one disk limit (the write and one rewrite after an eviction); a new limit resets the count. The persist test waits for the conversion to forget its keys before checking that no work is left. * frames: the work area doesn't change pixels Setting the work area (B, N) cleared every cached frame, in RAM and on disk: it was compared with the comp's pixel settings and hashed into the disk key. It is a preview and render range only, like the Resolution, so it no longer counts. * frames: audio edits keep frames unless something draws the audio Changing a clip's Audio Levels, its audio effects or its Audio switch cleared the clip's cached frames, and the comp's disk cache, although nothing drew that audio. The audio now counts only for the layers whose audio is heard (Project::heard_layers): those a layer picks in an effect (an Audio Waveform or Spectrum, even on itself), the layers of a picked precomp, and every layer while an expression (which can read Audio Levels) exists. Music in a precomp that a waveform in the outer comp picks is no longer treated as silent either. * frames: an expression only reaches the frames of its own layer One expression anywhere in the project made every edit clear the comp's whole cache, and so did editing a layer another layer reads (its parent, matte, or a layer it picks). Expressions can read anything, but they only draw in their own layer: an edit now re-renders the frames where it changed something, plus those of the layers with an expression and of the layers that read a reached layer. With expressions, an edit elsewhere (another comp, an item's name, a marker, music) reaches only the expression layers' frames. * frames: an edit inside a precomp re-renders only where the precomp shows it Editing anything inside a precomp cleared every cached frame of the comps that nest it, even a music level in it. Project::dirty_spans now follows precomps: it takes the spans their comp changed in, shifts and scales them by the precomp layer's start and stretch (its whole span under a time remap or Responsive Design timing), and keeps the outer comp's other frames. Layers that read a changed precomp (a matte, a waveform) re-render where they are active. * disk cache: keep the frames an edit can't reach The disk cache keys frames on a hash of the whole comp, so every edit lost the frames that were only on disk (the blue bar), even a rename; only those still in RAM were written again under the new key. When an edit keeps RAM frames, it now also queues a disk carry: on the frame pool, each kept frame of every disk series in use for that comp is hard linked (or copied) to its new key with DiskCache::link_frame, oldest edit first. The old entries stay, so undo finds them too. * frames: a keyframe edit re-renders only around the keys it changed Changing one keyframe (its value, interpolation or ease, in the Timeline or the Graph Editor) re-rendered the layer's whole length, so a key on a full-length layer cleared the whole cache. A layer that differs only in keyframes of its Transform properties now re-renders from two keys before the first changed key to two after the last (auto-Bezier and continuous tangents reach the neighbouring segments), to its in or out point past the first or last key. Static values, expressions, roving keys, time remap and time effects still reach the whole layer; effect keyframes do too for now (simulations and time effects read other frames). * frames: music volume keeps frames while no expression reads audio Any expression in the project (a wiggle on another layer, even a disabled one) made every layer's audio count as read, so changing a music layer's Audio Levels re-rendered every expression layer and changed the comp's disk key. Audio now counts for expressions only when one may read it (Heard::audio_exprs): its text names audio or walks properties by index. An audio-only change nothing reads no longer counts as a change elsewhere in dirty_spans either. * frames: a keyframe edit with few keys or beside a self-contained expression keeps the other frames Changing a Scale key of a layer with two or three keys re-rendered the layer's whole length: with fewer than two keys before (or after) the changed one, the span fell back to the in (or out) point instead of the first (or last) key, past which the value holds. And any layer with an expression re-rendered on every edit, so a wiggle on a full-length layer cleared the whole cache. An expression layer is now reached by edits elsewhere only when its text may read outside its layer (it names a comp, layer, footage, the project or its source); its parent and the layers it picks are still read through referenced_layers. * viewer: the caching chip stays up between uncached frames, with a spinner The "Caching frames…" chip showed only while playback waited for a frame, so during caching it flickered on and off with every frame that arrived. It now stays up 0.6 s after the last wait (and hides at once when playback stops). It is also smaller and quieter: a spinner, then "Caching frames" without the ellipsis in dimmed smaller text, on a fainter chip sized to its text. * preview: the Preview panel's cache bar counts the preview's Resolution With a preview Resolution set (Half, Third…), the frames a preview plays are cached at that scale, but the Preview panel's bar counted the series the viewer shows. When playback stopped, the viewer went back to the comp's Resolution and the bar dropped from full to the frames cached there. It now counts the preview's own series (Auto still follows the viewer). The viewer and the Timeline's cache bar are unchanged. * Contributors: add nazar-gavrylyk's name * frames: an expression reading its own layer's markers or audio re-renders that layer With expressions in the project, a layer's markers count when comparing it, and its audio counts when an expression may read audio, so a marker or volume edit re-renders the edited layer over its own span. Comp markers still count only through layers whose expressions name thisComp. * frames: a keyframe edit on a layer with an expression re-renders the layer An expression on the keyed layer can read the keyed property at other times (Rotation reading Position with valueAtTime), so the frames past the changed keys may change too. * cache_bench: read decoder counters through effectcraft-media Only crates/media, crates/export and crates/interchange may depend on the FilmCraft crates. * cache_bench: clips end where the next one starts * frames: write-back uses the worker's disk keys while a Roto Brush model is active persist_missing keyed disk frames on FrameKey::opts, which also carries the Roto Brush model stamp; the worker and disk_frames key on the render opts' hash alone. With a model active every RAM frame looked missing, was written again under keys nobody reads, and recorded a second disk series. * frames: an edit two precomps down reaches the outer comp A precomp layer whose comp Arc didn't change was skipped, but the comps nested in it may have (an edit in Inner leaves Mid's Arc as it was), so Outer kept every frame. Every nested comp is walked now, each once per edit. * cache_bench: stop when the disk cache can't be cleared * frames: one disk carry job at a time carry_disk runs on every timeline paint and idle pass, and spawned a pool job whenever carries were queued; during a drag the extra jobs waited on the carry lock and held frame-pool threads from the viewer. A scheduled flag keeps one job, which drains the queue in order until it's empty. * frames: a panicking disk carry is skipped, not left holding the job The carry job clears carry_scheduled only on its normal exit; a panic in a carry would have left it set for the session, so no later carry ran. Each carry is caught like the content key's hashing, so the job always reaches its exit. * frames: an expression reading its own in point or index re-renders its layer A trim or rename kept the layer's frames outside the cut when the layer drew the same apart from its span, but an expression on it can read its own inPoint, outPoint or name (a fade from inPoint), so every frame of the layer changes. Such a layer is no longer treated as only respanned. An expression reading `index` reads the layers above it too: a layer added or removed there shifts it, so `index` now counts as reading outside the layer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * frames: an edit of a comp reaches the expressions of the precomps it nests An expression in a precomp can read the comp that nests it (`comp("Main").layer("Controls")`), but the walk into nested comps only counted changes outside the edited top comp. Editing a hidden Controls layer there (or any layer, when the precomp is time-shifted or reads it at other times) kept every precomp frame in RAM and linked them all on disk under the new key. Nested comps now see an edit of the comp above them as an edit elsewhere. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * frames: an edit in a precomp under a time effect reaches its whole layer A precomp layer running Echo, Posterize Time, Time Displacement or another effect that reads the layer at other times shows a nested change far from where it maps, but only the mapped span (a nested frame either side) re-rendered. Such a layer now gives its whole span, like a time remap. `reads_other_frames` lists those effects (every `ec.time.` and `ec.sim.` effect and the others that call `self_at`); keyframe spans use it too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * frames: a keyframe edit beside muted keys re-renders the whole layer Muted keys are skipped when a property is evaluated, so the keys that play around an edited key can be any distance away, but the span counted two stored keys either side. A segment from an earlier playing key kept its stale frames. Muted keys now give the layer's whole span, like roving keys. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * frames: one echo in the precomp time-effect test (three saturate alike), rustfmt Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
FredAppsandClaude Opus 5.5
|
bc3b074ad9 |
About: credits column widths come from a tested fit_widths (#3039)
* About: credits tables with resizable columns that fit without side-scrolling Contributors and Models were egui Grids inside a two-way scroll area, so the table opened wider than the dialog and scrolled sideways, and Company and Model sat almost touching. Both are now egui_extras tables: each column opens at its header and widest cell, the spare width goes to the text columns (Name; Company and Model), and every column edge can be dragged. Widening a column grows the table like a spreadsheet, and only then does it scroll sideways; the header stays put while the rows scroll. Counts and dates are right-aligned and every cell is vertically centred. The sorted column stays highlighted, with its direction on the toolbar's arrow button, so headers no longer reserve room for an arrow. The About dialog is 820 points wide (modals still shrink to the window) so every column fits at its natural width. egui_extras comes from the pinned egui fork without default features; it adds enum-map. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * About: credits column widths come from a tested fit_widths The opening widths of the credits tables are a pure function now, with unit tests that a fitted table fills its room exactly (no side-scrolling, no gap), that spare width goes to the growing columns, and that too little room shrinks only those. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Let every credits column be dragged, and refit changed tables The last column was egui's remainder column, which has no drag handle, so it could only be resized from the edge before it. It is now an ordinary column with its own handle. A table keeps its column widths, so switching to a longer name left the Name column at its old width and clipped the text. The table now goes back to the opening fit whenever that fit changes: a longer widest cell or a resized window. Dragged widths are dropped at that point. fit_widths could overflow the room when one column hit the minimum, because the other columns did not take up its share. Columns now give way in proportion to their width above the minimum. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Refit credits tables on what they measure, not the fitted widths Comparing the fitted widths missed the case it was meant for. Name is the only growing column, so a longer name takes its extra width from the spare room and the fit comes out the same: a dragged Name width stayed, clipping the longer name. It could also drop dragged widths by accident. The fit comes from the available width inside the scroll area, which scrolling can nudge, so scrolling a widened table could reset it. The table now compares the natural widths and the clip rect's width, which scrolling does not move. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Valentin Klein
|
bb78d55825 |
After Effects project import (.aep / .aepx); export follows (#2558)
Add After Effects project import with bounded parsing, explicit loss reports, and regression coverage. Fetch the cleaned external test corpus at commit 97262276f625afdeeb61787e4cbb74b3f17af510 and verify each fixture against the pinned SHA-256 manifest. |
||
|
8d20cf0c3a |
GPU.1: Preserve CPU parity in numerical effect kernels (#1490)
Co-authored-by: @skylartaylor <26588250+skylartaylor@users.noreply.github.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com> |
||
 Brandon Thomas
|
0d640999e6 | Merge PR #2812 | ||
 Brandon ThomasandClaude Opus 5.5
|
7870b5916a |
Release 0.21.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
FredApps
|
f2aeff7e17 | Merge current main into fix/gnome-window-decorations | ||
 heojunfoandClaude Opus 5.5
|
3b2e26ceef |
Render Queue EXR sequences are scan-line, not tiled (#2805)
image's OpenEXR encoder writes 64x64 RLE tiles, which FilmCraft and other scan-line readers refuse. Write the frames with the exr crate in scan lines with ZIP, like comp.saveFrameAsExr. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Brandon ThomasandClaude Opus 5.5
|
d2fa9c1905 |
Follow-up to #2792: pin craft-ecproj-format to craft-libs main (ca1b5e2e, #409)
The PR pinned the head of a fork branch, not in craft-libs history; the crate source is byte-identical at both revs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Brandon Thomas
|
f3cedac8cc | Merge PR #2792 | ||
|
dotCooCoo
|
b133c0500f |
Project files: read savedBy through craft-ecproj-format's Head
The craft-ecproj-format pin moves to storytold/craft-libs#409's new head, where Head::read is the one reader and saved_by() is removed. effectcraft-project's saved_by reads savedBy with Head::read, and from ReadError::TooNew for a project with a newer schema. saved_by now returns None for text that is not a JSON object with a u32 schema. EffectCraft calls it only on files Project::from_json opened, so opening and importing projects are unchanged. |
||
|
Brandon ThomasandClaude Opus 5.5
|
720c67117e |
Release 0.20.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
dotCooCoo
|
b1571abaaa |
Project files: the file envelope comes from craft-ecproj-format
effectcraft-project writes and checks the envelope of .ecproj files with craft-ecproj-format from craft-libs: File (savedBy, then the saved view and the project) with to_json and to_json_compact for Save Compact Project Files, check_schema and its message, SCHEMA_VERSION and saved_by, which stay re-exported under their names. Project, SavedView and the schema 1 conversion stay here. Saved files are byte-identical. saved_by now reads only a JSON object, so a project written as a JSON array no longer reports a savedBy. |
||
|
Brandon Thomas
|
d3897efd86 | Merge PR #2719 | ||
 adikshan11andClaude Opus 5.5
|
45b737455a |
automation: a confined project can't show footage from outside the roots (#2131 F-11, "Indirect reads")
The command checks look at the paths an agent gives, but a project it opens or imports from a read root names its own footage, which was read wherever it is when the project rendered: render_frame returned its pixels. Backend::confine now wraps a headless session's footage and services: ConfinedFootage reads frames, audio, aux channels, models and vector frames only for footage inside the read roots (problem() says why the rest shows nothing), and ConfinedServices checks read_file, exists, stamp, list_dir and is_dir the same way. Writes are left to the command checks. Test: automation_roots_confine_the_footage_a_project_names covers an absolute path, `../`, a linked file and a linked folder, each opened and imported. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
 aiongg
|
1431c202d3 | updates: show check results with craft-update's CheckResultDialog | ||
|
Brandon ThomasandClaude Opus 5.5
|
afa3c938f6 |
Release 0.19.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
 bflatastic
|
334bf0f785 |
Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts: # crates/engine/Cargo.toml |
||
 Kristian Myram
|
41148e9b72 | #1743: validate background and bridged exports in their owning session | ||
|
Kristian Myram
|
baa1ca31b3 | #1539: preserve stroke corrections and current main APIs | ||
|
Kristian MyramandClaude Sonnet 5.5
|
2dfe1b8dbe |
S2.67: optional half-precision GPU weights (EFFECTCRAFT_SAM2_F16=1)
Halves the weight memory (Large: about 900 MB to 450 MB). Off by default: the video logit error grows from about 0.003 to 0.06-0.12 (masks unchanged) for a speed gain of under 10 percent. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Kristian MyramandClaude Sonnet 5.5
|
066bb2d031 |
S2.44: the span bar's confidence colours, checked in a real render
egui_kittest test: a scripted tracker (sure, doubtful, unsure, absent, sure again) tracks a layer and the Layer panel is rendered; the strip's pixels are green, amber, red, slate and green at the right frames. ROTO_SPAN_SNAPSHOT saves the panel image. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Kristian MyramandClaude Sonnet 5.5
|
5c7eb3c2ae |
S2.62: GPU encoder (wgpu): Hiera + neck on any GPU, matches PyTorch to 7e-6
Strided batched matmul with bias, GELU/ReLU and residual in the epilogue, layer norm, softmax, window partition, pooling, patch embedding, RoPE and neck kernels (WGSL, all tested against CPU definitions). GpuEncoder keeps activations on the GPU and returns the 64x64 level and the decoder's skip features. On the RX 7900 GRE one frame encodes in 0.29 s (debug build) against 2.1 s on the CPU in release; errors 6.8e-6 vs PyTorch, 1.2e-5 vs the CPU path. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Kristian MyramandClaude Sonnet 5.5
|
921bb00ba6 |
S2.61: parallel bias add fused with GELU; GPU matmul spike
Candle adds a bias on one thread; the Linear layers now add it (and apply GELU for the MLP in) in one parallel pass. Encode 2.4 s -> 2.1 s, memory attention 2.4 -> 2.3 s; reference tests unchanged. gpu_matmul example: a tiled WGSL matmul on the RX 7900 GRE reaches 1.9-3.6 TFLOPS against 0.16-0.47 for the CPU. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Kristian MyramandClaude Sonnet 5.5
|
aab00e43fe |
S2.60: parallel exact GELU: encode 3.4 s -> 2.6 s per frame
Candle's gelu_erf runs on one thread and was a third of the Hiera block time. A rayon CustomOp1 calls the same erf per element (bit-identical, tested). All reference tests unchanged. op_bench example times the block's operations. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Kristian MyramandClaude Sonnet 5.5
|
4655425d01 |
S2.50: bench --roto adds the SAM 2 tracker column
run_tracker follows the object with the model's video tracker on the same synthetic clips (--sam2 FILE or the installed weights). 12 frames, dev build: re-entry clip mean IoU 0.632 classical vs 0.975 SAM 2 (0.000 vs 0.967 after the object returns); the other clips 0.98 vs 0.99. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Kristian MyramandClaude Sonnet 5.5
|
92ac189a49 |
S2.31: load SAM 2 through the install flow; SAM 2 as a MaskModel
Sam2Mask wraps the image path behind the Roto Brush's MaskModel interface (cached frame encoding, bilinear mask resize). The engine loads the verified weights natively (not on wasm). Checked with the real file: install, notice, load, use by Roto Brush 3.0, deselect. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Kristian MyramandClaude Sonnet 5.5
|
92b3a34732 |
S2.11: Hiera backbone matches PyTorch block by block
Weights loader with shape/finite validation (S2.10), patch embed, bicubic positional embedding, window/global attention, query pooling. Reference test (skips without weights): max abs error 2.6e-5 over all 12 blocks. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Kristian MyramandClaude Sonnet 5.5
|
36a22733a2 |
S2.4: effectcraft-sam2 crate skeleton
Native-only, Apache-2.0, forbid(unsafe_code), panic lints denied, NOTICE, layer 3. Empty on wasm32. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Kristian MyramandClaude Sonnet 5.5
|
71e44deb84 |
S2.50: effectcraft-cli bench --roto on synthetic clips
Synthetic Roto Brush clips with known mattes (movement, lighting change, occlusion, leaving and re-entering) and a runner that compares the classical engine with MobileSAM when installed. Needs no weights; SAM 2 plugs in later. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Dylan M. Taylor
|
36c8d8a71c | UPD.1: honor app pretend versions in update checks (#2663) | ||
|
Dylan M. Taylor
|
e136844b53 |
Update prompt shows the app accent color (#2656)
UPDATE_APP gains our accent (#e0368f, our colour in ArtCraft Launcher). The prompt draws it on the UPDATE chip, the buttons and the What's new underline, and says "by ArtCraft" (craft-update's default developer); #2646 gave it our icon. craft-update is pinned to storytold/craft-libs main at 1065653e7 (#434). |