- Open queues every linked file's lookup (stat, then hash) for the background
check instead of stating candidates on the UI thread.
- links.list, links.update and relinkFolder use the open's status only while
its check is still running; afterwards the file is looked at as it is now.
- Preflight tells Modified from size, time and the remembered hash checks
(the worker keeps them between passes) and the vector test from the mime
type; it never loads a linked original.
- A modification time of 0 never vouches for a file's bytes.
- Migration keeps an unsettled time as UNVERIFIED_MTIME; recovery snapshots
keep images not yet saved as links embedded.
- Links panel test fixture has real files, so its Embedded assertions hold.
Export and package read a linked original from disk (hash-checked), a package copies it and keeps a link to the copy, preflight reports a missing link as an error and a modified one as a warning, and a linked alpha-wrap image opens without reading its original.
IDML imports link the files they read through links::link_to_file, the
path File > Place takes, so a file read just after it was written keeps
an unverified fingerprint and its status comes from the hash instead of
reading as missing. File > Package and Links > Copy Links To point a
linked graphic's record (paths, unverified fingerprint) at its copy.
Regular-file metadata goes through io::file_metadata, which the file
guard accepts.
links.list gains storage; a linked graphic's status comes from the open
check while its bytes are unchanged, else from its files: OK by size and
time without reading, else by content hash; Modified or Missing. Update,
Relink and Relink to Folder read the file into a new linked version with
a new hash and proxy. Embed checks the file first and keeps the link
record; links.unembed links to the original when it holds the same
bytes, else writes them into dir. Not on the web.
(cherry picked from commit 8d617b872f7d7b01b9551d07a3e82be03e1c1362)
File > Place, the place cursor (place.load, grid, contact sheet), media,
media posters and table graphic cells record the file's absolute path,
its path relative to the document, size, time and content hash, and a
proxy image, and keep the bytes read for placing in the linked-bytes
store for the first draw. embed: true keeps the bytes in the document
with the link record. base64 bytes and the web always embed.
(cherry picked from commit 4f5bd99088b2f32bece3be08258ec66c15dd3b44)
Save records each link's absolute path and its path relative to the
destination; linked assets write their record and proxy only, and Save a
Copy {embedLinks: true} copies every linked file in (failing on a missing
or changed one). On the web, links stay links with a warning.
Recovery snapshots record links as a save to the document's own file
would, never their originals; recovered documents resolve relative links
against the recorded document path.
Open (and Revert, Open Bytes, Recover) points each linked asset at its
file without reading it: a file with the recorded size and time is the
one; one with the recorded size and another time is hashed on a
cancellable background thread (native_links), absolute path first, and
the first hash match wins and keeps its new time. No match is Modified.
Status is kept on DocState::links for the Links panel; links.progress
applies results and gives the status-line notice once.
Documents saved before linked storage (embedded assets with a link and
no content hash), opened from a file, get a background byte comparison;
confirmed images get their hash and a proxy and save as links, and their
in-memory bytes become a link after that save. Changed or missing files
stay embedded.
Based on #1108 by ForestParks.
(cherry picked from commit 3a453b7f3752d255b6f74cfb55ea2459c5d9a074)
Linked bytes load only for a level that isn't cached, on the decode thread
under GATE. A linked file that can't be had is remembered apart from failed
decodes until the link generation advances, and the asset's proxy is drawn
meanwhile. PDF layer variants keep their key across rebuilds while their
decodes are cached.
AssetData::linked carries a LinkedSource (absolute and relative path, document,
size, fingerprint, content hash). Linked bytes load through a process-wide
single-flight LRU store (512 MiB, 1 GiB per file) and the Loader the engine
installs in Session:🆕 absolute path first, then relative; a changed
fingerprint is checked against the hash, and other bytes are Modified.
Proxy.data is an AssetData so its decodes have a key of their own.
The image cache, layer variants, content boxes and tone cache key by
SourceKey instead of the bytes' address, so they no longer hold or
reserve the encoded bytes: decoded levels outlive them.
Asset.data becomes AssetData (embedded bytes or a linked length, each
under a process-unique SourceKey), with the link_relative, link_hash,
storage and proxy fields. AlphaOutline names its source by SourceKey.
Dedup in assets.rs compares embedded bytes as before.
- A request turned away as recently decoded schedules its wake only for the
first decode of that image, level and epoch: a view larger than the cache
(or more failed images than are remembered) no longer decodes every 2 s.
- A worker that takes a job while a wake is scheduled hands the timed wait on.
- A full decode drawn at level 0 from the cache leaves the zoom-in list.
- Pixels decoded under colour settings changed since are not cached.
- Shadows and glows drawn while an image in them decodes are not kept.
The content-grabber ghost and the place-gun thumbnail take their proxy
with mip_now: a level not cached yet decodes in the background and they
draw another cached level, or the outline, meanwhile. A stand-in texture
is drawn again when the image generation advances.
Pages panel thumbnails render with defer_images, so they no longer run
warm on the UI thread or wait at the decode gate. A thumbnail that drew
images still decoding renders again once more have finished, at most
every 250 ms.
- A panic anywhere in a background job, sizing and the wait at the gate
included, marks the image bad. A guard takes the job off the busy
list however it ends, and a thread that ends is counted out so
another starts.
- Background decodes reserve their bytes where mip chooses the decode:
the reduced JPEG estimate before a reduced decode (a JPEG without one
goes to the full decode), the full peak before a full decode, nothing
when a cached level is halved.
- A request turned away as recently decoded schedules one generation
advance for when its window has passed, so its stand-in doesn't stay
until the view changes. Emptying the cache (new colour settings,
clear) starts a new epoch, and decodes into the old cache no longer
count as recent.
- At most two full decodes are kept for zooming in; a newer one evicts
the oldest.
- warm's planning runs under catch_unwind; a panic skips warming.
- Renderer::images_pending says whether the last render drew an image
still decoding.
Regression tests use a test-only fault hook that panics inside the job
and inside warm's planning.
The canvas renders with RenderOptions::defer_images and keeps the image
generation each texture was rendered at. Finished background decodes
are taken up at most every 250 ms and not while a render is in flight,
so a page of photos decoding one after another re-renders a few times
instead of once per photo, and a newer request doesn't drop the render
in flight before it is shown. The eyedropper still decodes the image it
samples.
Based on #2051 by texjer.
The image cache kept the full-resolution level 0 of every TIFF, PNG,
PSD and CMYK JPEG, and of any image shown at High Quality: about 48 MB
for a 12 MP photo, so after a dozen photos every edit evicted and
decoded them again on the render thread.
- mip keeps only the level drawn. It remembers each asset's full size
(from its header, then from a decode), builds a coarser level from a
finer cached one, and tries the reduced JPEG decode before a full one.
When an image is zoomed into (a finer level asked for while a coarser
one is cached) it keeps the full decode too, up to 96 MB, so the
following steps halve it instead of decoding again.
- With RenderOptions::defer_images (the canvas) the image draw uses
mip_now: a level that isn't cached is decoded on one of three
background threads, newest request first, and a coarser cached level
or a light box is drawn meanwhile. images::generation advances and
the waker runs when a decode finishes. A decoder panic marks the
image bad. Background decodes reserve their peak bytes at the same
gate as warm, so they wait while decodes in flight hold half the
budget.
- An image asked for again within 2 s of its decode (evicted at once:
the view holds more than the cache) isn't decoded again until then,
in place or in the background, so the canvas never decodes on its
own thread.
- warm counts an asset as cached when the level each drawing takes, or
a finer one, is cached, and sizes a job by the levels it keeps. It
leaves a decode larger than the whole budget to the render, and the
canvas skips it.
- A render that deferred an image isn't kept as the CMYK view snapshot.
Based on #2051 by texjer.
The image cache keys decodes by the bytes' address, so a variant rebuilt
as a new Arc misses the decodes it already has. With decodes in the
background, more variants on screen than the variant cache kept would
decode and re-render without end. Variants are now looked up weakly,
with the 64 most recent held strongly.
Based on #2051 by texjer.
The IDML story writer skipped the endnote reference character, so an
exported body story lost every endnote's place: "an endnote here. End."
with nothing between "here" and the full stop. The endnote frame's
story was written with its numbers as text, so nothing linked the
listed endnote back to where it was referenced.
Each reference is now written as its own superscript character range
holding the endnote's number, formatted by Endnote Options (start,
style, prefix and suffix) as the endnote frame lists it. IDML's
Endnote / EndnoteRange elements are not written yet.
aiongg
Dylan M. Taylor
jayzed80