Commit Graph
6 Commits
Author SHA1 Message Date
vangel76 8a3b15d41b Footage cache (#3271): decoded OpenEXR channels persist in the disk cache
A plane kind in the disk cache (planes/): named f32 planes, bit-exact, keyed by the source
file's path and identity, the channel set, the level and the rows; listing their source file,
so a changed file or Reload Footage drops them (the sweep covers planes). The plane store reads
them before decoding and queues what it decoded (encoded on the writer thread, refused when the
disk has no room or the file was dropped since the decode began). The footage source gets the
disk cache with set_disk_cache.
2026-11-08 03:00:29 +01:00
vangel76andClaude Opus 5.5 726d78d1c0 Disk cache: layer entries keep their source files, no footage salt
Part of #3184 (step 3 of 4).

Layer buffers on disk (and in the browser's layer store) now carry the
files they were made of, like frame entries since step 2, and are checked
against exactly those files when used. The project-wide footage salt is
gone: nothing stamps every footage file any more.

- Layer entry format 4: the buffer's EntryFiles, then the pixels. A hit
  checks only the listed files (metadata, or the VirtualFileId hook for
  files from memory; Missing must still be missing); a change, a damaged
  entry or the old format 0 is a miss and the entry is deleted, so the
  re-rendered buffer replaces it. A hit reports its file list, so frames
  made of buffers from disk are complete and get stored again.
- Buffers whose files can't all be named are not written.
- Layer disk keys: layer_disk_key = hash of the layer entry version, the
  renderer version and the layer cache key (which names the footage items
  with their paths and settings, from memory).
- PrefetchStore checks on use; an entry made of changed files is dropped
  and forgotten, so the new buffer is queued over it. FrameServer passes
  its virtual-file hook to its layer cache.
- Removed: footage_salt(_with), file_stamp, footage_stamp,
  set_disk_salt, set_store_salt, Session::refresh_disk_salt,
  poll_disk_salt, footage_fingerprint, disk_salt, disk_salt_background,
  disk_salt_job, DiskSaltJob, the UI's per-frame poll, the salt in
  FrameServer and in the availability worker (with its cache_enabled
  plumbing). The disk cache is attached as soon as it is configured.
- Docs: architecture.md, web.md.

Tests: layer entries check their files (size, time, missing/back,
folder, damaged, old format, incomplete); virtual files; prefetch store;
media tests with a real pool across restarts (only the changed file is
read, projects with the same item ids never share an entry, missing,
unreadable and damaged never serve stale buffers); engine tests that the
cache is attached at once and edits do no file work (#1926, #3106).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 00:12:07 +01:00
vangel76andClaude Opus 5.5 99b65562de Disk cache: frame entries keep their source files, comp key from memory
Part of #3184 (step 2). Fixes #3175: opening a comp no longer stamps every
footage file of the project before its first frame can show.

- The comp's content key (`disk_cache::comp_content_key`) is taken from the
  project in memory alone: no footage or proxy file is stamped, no
  `VirtualFileId` is asked. `comp_content_key_with` is gone; `frame_key`
  hashes a new entry version, so entries of the earlier format are never
  looked up again (LRU evicts them).
- Frame entries (formats 2 and 3) store the files the frame read
  (`EntryFiles`: size and modification time as read; size and content hash
  through `VirtualFileId` for files from memory; missing files), then the
  pixels. Formats 0 and 1 read as misses and are deleted.
- Check on hit: `get_frame`/`get_frame_f16` look only at the listed files
  (one metadata call each) on the frame worker. A changed, back or unreadable
  file, or a damaged entry, is a miss: the entry is deleted and the frame
  rendered and stored again.
- A frame whose file list is incomplete is shown but not stored. The RAM
  preview keeps each frame's files, so `persist_missing` writes only frames
  with a complete list.
- Remote frame worker: `FrameServer` records each frame's reads
  (`take_files`); the web worker stores them with the entry and the page
  checks them through the file table before using it. The browser's media
  epoch existed only for the old key and is removed. Export does not use
  frame entries.
- The cache indicator still only asks whether an entry exists.

Tests: entry format round trip and damage, the check per file, and in the
viewer: a changed file re-renders exactly its frames (and a reopen reads the
rest from disk); a frame checks only its own files in a project with a large
unused sequence and the key looks at no file; missing, unreadable files and
damaged entries never serve a stale frame.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 23:19:09 +01:00
vangel76andClaude Sonnet 5.5 925760c9c8 Web disk cache: handle the Matte kind (mattes/ folder)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-12 19:08:21 +02:00
Brandon ThomasandClaude Opus 5.5 f3715aac32 M13.30: web depth and polish
- GPU in browser job workers: each job worker opens its own WebGPU device
  (deferred readbacks); the Render Queue export and the analyses' frame loops
  are futures (render::passes, offload::run_request_async) whose frames
  render in passes, awaiting the device between them. GPU particles and
  Advanced 3D read back under keys too. workers.jobs reports passes and
  readbacks; the smoke test measures them.
- Layer buffers in the browser's OPFS disk cache: frame workers back their
  layer caches with a PrefetchStore; the page plans what to fetch before a
  request from recent misses (LayerPrefetch over the shared DiskIndex).
- Templates embed footage (default on, size-capped with a warning) and
  creating from a template extracts it next to the new project / into
  browser storage.
- ICC LUT profiles (lut8/lut16/lutAtoB/lutBtoA, from the public ICC spec) in
  Simulate Output > My Custom RGB, baked into a 3D LUT for the viewer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 02:57:17 -04:00
Brandon ThomasandClaude Opus 5.5 1b726349ab M13.24: web app depth (WEB-1) — GPU effects in frame workers, OPFS disk cache, storage manager
- GPU in the browser: each frame worker opens its own WebGPU device (Gpu::request_deferred) and
  renders the GPU compositor and GPU effects with deferred readbacks (effectcraft-gpu deferred):
  readbacks keyed by content start with map_async and miss, the pass returns placeholders, the
  layer cache's gate drops inserts, and FrameServer::resume renders again once the device
  delivered. Backend Auto times whole frames per comp in the workers. Frames with effects go to
  GPU workers instead of the page. CPU fallback without WebGPU in workers (?nogpuworkers).
- Fix: a WGSL constant (3.40282347e38) that Chrome rejects invalidated every GPU kernel in Chrome
  (the page's compositor too); a test checks every float literal fits f32.
- Disk cache in the Origin Private File System: workers write sealed frame entries with sync
  access handles (temp file + move), the page keeps the DiskIndex (LRU under the settings' limit,
  at most half the quota) and reads hits instead of rendering; survives reloads; blue cache bar.
- Storage manager: Settings ▸ Disk ▸ Browser Storage (usage/quota, persistent storage, Clear
  buttons, automation ids) and storage.info / storage.persist / storage.clear (StorageHost).
- Docs: web.md (GPU workers, deferred passes, disk cache, storage manager, threading design:
  one engine instance per worker), architecture, preferences, parity (WEB-1), roadmap.
- Tests: deferred passes vs CPU on a real device, placeholder gating, frame server passes and
  Auto, GPU worker routing + disk cache across a reload, storage commands and widgets, store
  usage by kind; smoke test: GPU effect frames in workers with pixel parity, disk hits after a
  reload, storage manager.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 18:35:03 -04:00