The allowance's sharing was tested on a few images. This runs a whole page through the export: the
pictures past it are left out and counted, and the page is listed, which is what the warning goes
by. A few large pictures all come through. The README gives the numbers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Four ways round the allowance, each measured by a review of hostile input:
A Flate stream read as zlib that fails at once, and then as raw deflate that
decodes everything before failing, gave nothing back and so was charged
nothing, here and in DecodeBudget: 110 s for a 387 KB file, all decoded. What
the retry decoded now counts, though the data is still dropped as it was.
Fax and 1-bit Flate images paid by the byte for work done by the pixel: 19
bytes can be a blank page of 100 million pixels, about a second each. Every
image decoded here now pays for its pixels first, as JBIG2 already did.
A JPEG paid nothing, and what one asks for doesn't grow with its length: 156
bytes declared 80 megapixels, 72 s for an 8 KB file. A JPEG pays for its
samples before it is decoded, in Optimize and in an export that has to decode
one for its mask.
Merging fragments paid for reading strips and not for the picture it built:
393 s for 1,154 bytes. It pays for the picture before allocating it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Filters were charged only for what they handed a JBIG2 decoder, once they
had handed it over: a chain that failed, or went past its image's limit,
cost its time and nothing else, and images that weren't JBIG2 weren't
charged at all. A 734 KB file of 200 images of Flate inside Flate took an
image export 111 s, and that grew with the file. It now takes 12 s.
Each byte read and each byte a stage decodes is now a step of the same
allowance, paid as it happens, so no stage runs past what is left. That
holds for every image read through ImageBudget, and a fax image pays for
its rows and its code. JBIG2 globals pay when they have to be read, and
kept ones don't. Optimize's two reads that went round the budget go
through it, and merging fragments shares it.
Optimize reads an image twice, to pack it and to resample it, so a refused
image is now counted once a run rather than once a read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
One image has its own limit, about 6 s to 17 s, but nothing bounded a file of thousands of images
made to reach it: an operation took that long for each, and an image sharing /JBIG2Globals decodes
them again.
Image::decode_within (hayro-jbig2 patch 11) spends from what a run may still do, starting with its
page's pixels, and charge_reading charges reading a stream and its globals, 4 steps a byte, before
anything is parsed, so an image that fails has paid too. pdfcraft_filters::decode_jbig2_within takes
it, and ImageBudget in cos is that allowance: 2^34 steps, about 1,000 scanned pages, and 2^13 more for
each byte of the file, so a long scan is read whole. An image past it spends all that's left, and
later ones aren't read, globals included.
Optimize leaves such an image as it is and counts it in Report::images_refused. Image export leaves it
out, with the reason, and stops at 2 GiB of files held. Office export shares one allowance, leaves the
picture out and says the page was truncated. Redaction removes it whole, as any image it can't read.
Rendering gives a page's images 2^30 steps between them, inside the watchdog, and each page starts
afresh in Context::new as its other budgets do (hayro-syntax patch 31, hayro-interpret patch 73).
It's a thread-local, unbounded until a page sets it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
From CodeRabbit's review of the stack:
- Every JBIG2 image read its /JBIG2Globals afresh, within 32 MiB each but with no
bound across an export or a redaction. A globals stream of the file, not edited since
it was read and with its /DecodeParms direct, now has its filters undone once a
document and is kept for the images that share it (in every clone, which share the
file), up to 64 MiB in all. An edited one, one added in memory, or one with
/DecodeParms by reference (another object, which can change under it) is read each
time as before. This saves the filters only: the decoder reads the segments afresh for
every image, so a real symbol dictionary still costs its decoding once an image.
- view.copy-image dropped "JBIG2 inside other filters" from its missing list on the
strength of Export's test; Copy Image now has its own: a JBIG2 picture behind
ASCIIHexDecode is found under the pointer and copied as its pixels and as a PNG, and
one whose page information is in its globals is read through them.
A JBIG2 image's /Width and /Height are read as given directly; one by reference is an
error rather than an image of no rows.
A /JBIG2Globals reference to no object still reads as no globals: it is a reference
to null (ISO 32000-2 §7.3.10), which an entry may hold for being absent (§7.3.7), and
the renderer draws the page so. An image that needed those globals then mostly fails
to decode; in a probe of twelve pictures that decoded with them, eleven were refused
and one decoded to other pixels.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
decode_jbig2 sits next to decode_ccitt, through hayro-jbig2 as Export ▸ Images
used it, and shares CCITT's row sink. The JBIG2 page has to be the image's size:
PdfCraft's renderer draws the whole page into the image (black box: a 4-row page
behind /Height 2 fills the image box with all four rows), so the rows of a taller
page aren't the image's. The decoder decodes a page whole, so the page is checked
before it is decoded: that size, within the caller's bound as rows of one bit, and
at most 2^28 pixels (MAX_JBIG2_PIXELS; the decoder keeps a bit a pixel, about 34 MB
there).
Document::decoded_image_within decodes a JBIG2 image as it does a fax image: the
filters before it undone first, then the decoder with the /JBIG2Globals of the
image's own /DecodeParms entry (/Crypt counts in the position; fax_params and the
globals now share last_filter_parms). Globals have a limit of their own,
MAX_JBIG2_GLOBALS (32 MiB, moved from Export), and globals that can't be read are an
error rather than ignored.
Export ▸ Images reads JBIG2 through it, so its own decoder, row sink and DecodeParms
reader go, and hayro-jbig2 leaves pdfcraft-create. JBIG2 behind another filter now
exports, and so copies (Copy Image uses the same function); a page taller than the
image, which Export cut to the image's rows, now skips the image, as one of another
width did. doc_export_all_images' description no longer lists JBIG2 inside other
filters as skipped.
The optimizer's monochrome step read images through decoded_image_within and
relied on JBIG2 failing there: it now leaves JBIG2 images alone explicitly, as
before (a resampled one would come out Flate under Retain), and the README and
parity note list them as not yet handled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PdfCraft's bootstrap renderer draws byte-aligned Group 3 whose fill ends
each end-of-line code on a byte (as TIFF writes it) as other rows than
decode_ccitt reads (black box, at every /K, with /EndOfLine or not:
mostly the first row, then blank). Redaction cleared pixels in the
picture decode_ccitt reads, so a mark over what PdfCraft showed needn't
cover what was cleared, and content shown blank stayed in the copy. Such
an image (/K 0 or more, /EncodedByteAlign) is now removed whole, as
every CCITT image was before this PR; Group 3 off bytes and Group 4 are
still cleared.
The image's parameters come from Document::fax_params, the reading of a
fax image's /DecodeParms entry that decoded_image already did, now on
its own (it was to come with #4404, which uses it too). The redaction
README, the module doc, the redaction and core.image-filters notes and
the spec parity table's status line and revision row say so, and the
core.image-filters note now records the renderer's misdrawing.
group_3_on_bytes_is_removed_whole_and_other_fax_images_cleared: a white
16 × 2 picture, checked to decode as such, is removed whole on bytes (no
codes; codes at /K 0; at /K 1 with /EncodedByteAlign by reference) and
cleared off bytes and as Group 4 on bytes. It fails without the guard;
cargo-mutants catches every viable mutant of it.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
The PDF Optimizer and Reduce File Size downsampled colour and grey images
but not black-and-white ones. Settings::mono is now MonoSettings {
downsample, target_ppi, above_ppi, compression }, 300 ppi above 450 ppi
and CCITT Group 4 by default (Monochrome Images ▸ Downsample to;
doc_optimize mono: { downsample, ppi, above_ppi, compression }).
A 1-bit image or image mask (no filter, general-purpose filters, or CCITT
fax after any of them) and eight-bit grey of only black and white are averaged by
area: each new pixel takes the sample covering more of it, a tie 0, so a
dark line half a new pixel wide stays. They are stored as Group 4, or as
1-bit Flate when that is smaller; with Retain a resampled image keeps its
kind of compression. Images with a soft mask keep their size.
CCITT decoding moves from Export ▸ Images into pdfcraft-filters
(decode_ccitt, CcittParams, through hayro-ccitt), and
pdfcraft_cos::Document::decoded_image reads a fax image's /DecodeParms
for it (undoing filters before CCITT, such as ASCII85Decode, with any
/DecodeParms given by reference made direct first), so export
and the optimizer share one reader, and Group 4 scans can be
downsampled. A 600 ppi text page went from 79 KB to 40 KB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
Stream::decoded reads only direct parameter dictionaries, so an image
whose /DecodeParms is a reference (whole, or as array entries) was
decoded without its predictor. For a 1-bit image with a PNG predictor
the rows came out shifted by their tag bytes, Group 4 of that was still
smaller, and the picture was replaced by a skewed one; resampling and
merging image strips read the same way.
pdfcraft_cos::Document::with_direct_parms gives the stream with the
parameters its filters use made direct: only the entries at the
filters' places, and none for a chain longer than any decoder takes,
so at most MAX_CHAIN objects are copied. The optimizer's pixels() and
the Group 4 step decode through it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
* M7: share Windows local-link fixture portability fixes (#1265)
Use the identical test-only prerequisite already proposed in #1263: Unix-only mutable vectors, ordinary Windows drive paths and escaped PDF literal strings. This is shared CI support for the separate xref overflow fix; production link handling is unchanged.
Generated with Codex.
* M1: return an error for exhausted xref object numbers (#1337)
Check the next-object addition and return CosError at the xref offset rather than overflowing or wrapping to object zero. Classic-table and stream regressions reproduced the original panic; ordinary, zero-offset and repaired-range controls preserve catalog access and allocation.
Validation: 2732 passed, 0 failed, 13 ignored; strict all-target Clippy, fmt, 31 WASM crates, layering, assets, deny and parity passed. The rebuilt CLI opened and rendered a clean synthetic page.
Generated with Codex.
* M1: place overflow notes beside stable milestone entries (#1337)
Keep the tested code unchanged while avoiding progress-note conflicts with newly landed main updates. Formatting and diff checks passed.
Generated with Codex.
* Merge main: adapt exhausted-xref repair to the compact xref index
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* D5: README for cos and engine, and an honest cos status line
Thirteen crates have no README, although the session protocol tells every session to read the README of the crate it is touching. These are the two biggest: cos is where every hostile file lands first, and engine is the whole API surface a frontend gets.
Each states its layer and the rules that bind it, sketches the API against the real signatures, and ends with what is deliberately not done. The cos README spells out the clone-is-a-snapshot property and why both writers return bytes rather than taking a sink; the engine README spells out the four-step editing model that makes what you see what Save writes.
Also corrects the module doc on cos/src/lib.rs, which still said encryption and object-stream output for full saves were not done. Both shipped: security.rs handles R2-R6 with crypt filters and /EFF, and SaveOptions::object_streams defaults to true.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* M14: let neutral Windows locales fall through to display language
Co-authored-by: Codex <codex@openai.com>
* pdfcraft-cli --help lists `mcp` only when the build has it
Follow-up to #540: a build without the `mcp` feature (`--no-default-features`) still listed
`pdfcraft-cli mcp` in its usage, and `pdfcraft-cli mcp` then said "unknown command".
USAGE is now put together with `concat!`, and its `mcp` lines come from a macro that is empty
without the feature. In such a build `pdfcraft-cli mcp` says the MCP server was left out instead
of calling it an unknown command.
- tests/help.rs: usage_lists_mcp_only_when_built_with_it (runs in both feature sets) and
mcp_without_the_feature_says_it_was_left_out (without the feature)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* engine README: the execution plan is local-only, so don't link a path that doesn't exist
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Streetblock <github@streetblock.de>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: lloyd <lloyd@stackworx.io>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Grzegorz MartensandClaude Opus 5.5
