An unedited object stream whose indirect /Length is edited in one clone
was parsed with that clone's length and put in the object-stream map
the clones share, so the other clones read its members within the
edited length. A container load that read the clone's edits is now
kept to that clone, like an edited stream.
Test: an_edited_object_stream_length_leaves_other_clones_members_alone
fails without the fix (the other clone reads None) and passes with it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Clones share the object cache and the parsed object streams. A clone that
freed or replaced an object stream then read a member of it put the result
in those shared maps: a failure (which this branch now caches) or, as on
main, the replacement stream and the member parsed from it. Another clone
whose stream is intact then read that instead of the file.
A load that reads an unsaved edit (an edited object stream, or an edited
indirect /Length) is no longer cached, and an edited object stream is
parsed for its own document only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Document::try_get kept only objects that loaded. An object whose entry
points at something that doesn't parse falls back to a whole-file
search (scan_for); that failed too, nothing was kept, and the next read
searched the whole file again. A caller that reads such an object
repeatedly, such as the window title of a file whose document
information dictionary is damaged and that asks for its title to be
shown, paid one whole-file search per read, several per frame.
The cache now keeps the load result, a failure as well. It is the same
map with the same bounds, clears and per-reader privacy, so every place
that drops loaded objects (authentication, reconstruction, private scan
readers, eviction) drops remembered failures too. Edits are read from
the overlay before the cache, as before, so a set or freed number still
answers for itself. A failure is kept in nested loads too, as a success
already is; only a poisoned lock is never kept. Failures count towards
the cache's object caps and hold no bytes of their own. Sharing the map
between clones stays sound because entries, data and the stream limit
don't change after open.
Test: an_unreadable_object_is_searched_for_once counts whole-file
searches through a test-only counter (like SECTION_READS): one for
eleven reads and a clone's read. With failures not kept it counts 22.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
`ImageBudget::decode` is now the one public way to decode an image's
CCITT fax or JBIG2 data: `Document::decoded_image` and
`decoded_image_within`, which decoded with no limit on the work, are
test-only in pdfcraft-cos, and pdfcraft-filters exports only
`decode_ccitt_within` and `decode_jbig2_within`, which spend from a
caller's allowance (`decode_jbig2` is gone, `decode_ccitt` private).
So an operation over many images can't decode them without sharing an
allowance; the compiler holds new callers to it. Doctests that must not
compile check both crates; with the old entry points public they fail.
Tests outside the crates decode through an unlimited ImageBudget.
Streams in general (`Stream::decoded`) are left for a follow-up.
A JBIG2 image's /JBIG2Globals hold the symbol dictionary of every image that
shares them (jbig2enc -s and most scanners give one to all of a scan's pages),
so the bitmaps of marks drawn only under a redaction stayed in the file while
another page's image used them. Every image using the globals of a removed or
replaced image (or of one in a removed or replaced form) is now decoded and
stored again in place as Flate without them; a full save then drops them. One
that can't be decoded for certain refuses the redaction
(RedactError::SharedJbig2Globals). Report::images_restored counts them.
Filters were charged only for what they handed a JBIG2 decoder, once they
had handed it over: a chain that failed, or went past its image's limit,
cost its time and nothing else, and images that weren't JBIG2 weren't
charged at all. A 734 KB file of 200 images of Flate inside Flate took an
image export 111 s, and that grew with the file. It now takes 12 s.
Each byte read and each byte a stage decodes is now a step of the same
allowance, paid as it happens, so no stage runs past what is left. That
holds for every image read through ImageBudget, and a fax image pays for
its rows and its code. JBIG2 globals pay when they have to be read, and
kept ones don't. Optimize's two reads that went round the budget go
through it, and merging fragments shares it.
Optimize reads an image twice, to pack it and to resample it, so a refused
image is now counted once a run rather than once a read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
One image has its own limit, about 6 s to 17 s, but nothing bounded a file of thousands of images
made to reach it: an operation took that long for each, and an image sharing /JBIG2Globals decodes
them again.
Image::decode_within (hayro-jbig2 patch 11) spends from what a run may still do, starting with its
page's pixels, and charge_reading charges reading a stream and its globals, 4 steps a byte, before
anything is parsed, so an image that fails has paid too. pdfcraft_filters::decode_jbig2_within takes
it, and ImageBudget in cos is that allowance: 2^34 steps, about 1,000 scanned pages, and 2^13 more for
each byte of the file, so a long scan is read whole. An image past it spends all that's left, and
later ones aren't read, globals included.
Optimize leaves such an image as it is and counts it in Report::images_refused. Image export leaves it
out, with the reason, and stops at 2 GiB of files held. Office export shares one allowance, leaves the
picture out and says the page was truncated. Redaction removes it whole, as any image it can't read.
Rendering gives a page's images 2^30 steps between them, inside the watchdog, and each page starts
afresh in Context::new as its other budgets do (hayro-syntax patch 31, hayro-interpret patch 73).
It's a thread-local, unbounded until a page sets it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
From CodeRabbit's review of the stack:
- Every JBIG2 image read its /JBIG2Globals afresh, within 32 MiB each but with no
bound across an export or a redaction. A globals stream of the file, not edited since
it was read and with its /DecodeParms direct, now has its filters undone once a
document and is kept for the images that share it (in every clone, which share the
file), up to 64 MiB in all. An edited one, one added in memory, or one with
/DecodeParms by reference (another object, which can change under it) is read each
time as before. This saves the filters only: the decoder reads the segments afresh for
every image, so a real symbol dictionary still costs its decoding once an image.
- view.copy-image dropped "JBIG2 inside other filters" from its missing list on the
strength of Export's test; Copy Image now has its own: a JBIG2 picture behind
ASCIIHexDecode is found under the pointer and copied as its pixels and as a PNG, and
one whose page information is in its globals is read through them.
A JBIG2 image's /Width and /Height are read as given directly; one by reference is an
error rather than an image of no rows.
A /JBIG2Globals reference to no object still reads as no globals: it is a reference
to null (ISO 32000-2 §7.3.10), which an entry may hold for being absent (§7.3.7), and
the renderer draws the page so. An image that needed those globals then mostly fails
to decode; in a probe of twelve pictures that decoded with them, eleven were refused
and one decoded to other pixels.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
decode_jbig2 sits next to decode_ccitt, through hayro-jbig2 as Export ▸ Images
used it, and shares CCITT's row sink. The JBIG2 page has to be the image's size:
PdfCraft's renderer draws the whole page into the image (black box: a 4-row page
behind /Height 2 fills the image box with all four rows), so the rows of a taller
page aren't the image's. The decoder decodes a page whole, so the page is checked
before it is decoded: that size, within the caller's bound as rows of one bit, and
at most 2^28 pixels (MAX_JBIG2_PIXELS; the decoder keeps a bit a pixel, about 34 MB
there).
Document::decoded_image_within decodes a JBIG2 image as it does a fax image: the
filters before it undone first, then the decoder with the /JBIG2Globals of the
image's own /DecodeParms entry (/Crypt counts in the position; fax_params and the
globals now share last_filter_parms). Globals have a limit of their own,
MAX_JBIG2_GLOBALS (32 MiB, moved from Export), and globals that can't be read are an
error rather than ignored.
Export ▸ Images reads JBIG2 through it, so its own decoder, row sink and DecodeParms
reader go, and hayro-jbig2 leaves pdfcraft-create. JBIG2 behind another filter now
exports, and so copies (Copy Image uses the same function); a page taller than the
image, which Export cut to the image's rows, now skips the image, as one of another
width did. doc_export_all_images' description no longer lists JBIG2 inside other
filters as skipped.
The optimizer's monochrome step read images through decoded_image_within and
relied on JBIG2 failing there: it now leaves JBIG2 images alone explicitly, as
before (a resampled one would come out Flate under Retain), and the README and
parity note list them as not yet handled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PdfCraft's bootstrap renderer draws byte-aligned Group 3 whose fill ends
each end-of-line code on a byte (as TIFF writes it) as other rows than
decode_ccitt reads (black box, at every /K, with /EndOfLine or not:
mostly the first row, then blank). Redaction cleared pixels in the
picture decode_ccitt reads, so a mark over what PdfCraft showed needn't
cover what was cleared, and content shown blank stayed in the copy. Such
an image (/K 0 or more, /EncodedByteAlign) is now removed whole, as
every CCITT image was before this PR; Group 3 off bytes and Group 4 are
still cleared.
The image's parameters come from Document::fax_params, the reading of a
fax image's /DecodeParms entry that decoded_image already did, now on
its own (it was to come with #4404, which uses it too). The redaction
README, the module doc, the redaction and core.image-filters notes and
the spec parity table's status line and revision row say so, and the
core.image-filters note now records the renderer's misdrawing.
group_3_on_bytes_is_removed_whole_and_other_fax_images_cleared: a white
16 × 2 picture, checked to decode as such, is removed whole on bytes (no
codes; codes at /K 0; at /K 1 with /EncodedByteAlign by reference) and
cleared off bytes and as Group 4 on bytes. It fails without the guard;
cargo-mutants catches every viable mutant of it.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
- Redacting a CCITT image went through decoded_image, which undoes the
filters before the fax decoder up to MAX_DECODED, so a Flate bomb in
front of a fax image cost a gigabyte while every other image stops at
twice its raster. Document::decoded_image_within takes a bound, and
redaction gives a fax image's code about a byte per pixel. A fax image
cut short, or behind a bomb, is removed whole; a busy one whose code
is bigger than twice its raster is still cleared.
- Every image is now decoded through decoded_image_within, which reads
/DecodeParms through Document::with_direct_parms: given by reference,
the numbers inside (/Predictor 7 0 R) included. That reading was first
written here and has moved down to #4304. The predictor test now covers
the dictionary, an array's entry and the number itself given by
reference; read without them, the cleared bytes drift off the covered
pixel and the covered sample stays in the copy.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
decoded_image paired the fax filter with the last /DecodeParms entry,
but Stream::filters leaves /Crypt out wherever it stands, so with
[/CCITTFaxDecode /Crypt] the fax filter got the Crypt entry and lost
its /K, /Columns and /BlackIs1. Its entry is now that of the last
filter but a /Crypt. /Crypt belongs first (ISO 32000-2 §7.4.10), so
only a malformed file had it (from CodeRabbit's review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The PDF Optimizer and Reduce File Size downsampled colour and grey images
but not black-and-white ones. Settings::mono is now MonoSettings {
downsample, target_ppi, above_ppi, compression }, 300 ppi above 450 ppi
and CCITT Group 4 by default (Monochrome Images ▸ Downsample to;
doc_optimize mono: { downsample, ppi, above_ppi, compression }).
A 1-bit image or image mask (no filter, general-purpose filters, or CCITT
fax after any of them) and eight-bit grey of only black and white are averaged by
area: each new pixel takes the sample covering more of it, a tie 0, so a
dark line half a new pixel wide stays. They are stored as Group 4, or as
1-bit Flate when that is smaller; with Retain a resampled image keeps its
kind of compression. Images with a soft mask keep their size.
CCITT decoding moves from Export ▸ Images into pdfcraft-filters
(decode_ccitt, CcittParams, through hayro-ccitt), and
pdfcraft_cos::Document::decoded_image reads a fax image's /DecodeParms
for it (undoing filters before CCITT, such as ASCII85Decode, with any
/DecodeParms given by reference made direct first), so export
and the optimizer share one reader, and Group 4 scans can be
downsampled. A 600 ppi text page went from 79 KB to 40 KB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
The decoders leave /Crypt out wherever it stands (Stream::filters), so
a chain of /Crypt and eight other filters decodes, but with_direct_parms
counted nine and left its parameters as they were: a predictor given by
reference was then skipped. Only the other filters count now, and a
/Crypt entry is null in the copy, as no decoder reads it. Tests cover
that chain and one exactly as long as a decoder takes.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Document::with_direct_parms resolved an indirect /DecodeParms and its
array entries but not the numbers inside (/Predictor 9 0 R), which
Dict::int doesn't follow. A 1-bit image with such a predictor was
decoded without it, its rows shifted by their tag bytes, and Group 4 of
that stored in its place; resampling read images the same way. It now
keeps, of each parameters dictionary, the numbers and flags the decoders
read, resolved, and /JBIG2Globals as a reference. Only those small
values are copied (an entry that is no dictionary is null), so a file
naming one big object from every key and filter can't have it copied a
hundred times.
The change is #4384's, moved down so that every PR stacked on this one
reads such parameters.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Stream::decoded reads only direct parameter dictionaries, so an image
whose /DecodeParms is a reference (whole, or as array entries) was
decoded without its predictor. For a 1-bit image with a PNG predictor
the rows came out shifted by their tag bytes, Group 4 of that was still
smaller, and the picture was replaced by a skewed one; resampling and
merging image strips read the same way.
pdfcraft_cos::Document::with_direct_parms gives the stream with the
parameters its filters use made direct: only the entries at the
filters' places, and none for a chain longer than any decoder takes,
so at most MAX_CHAIN objects are copied. The optimizer's pixels() and
the Group 4 step decode through it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
