From a review of the spellings an indexed image can take:
- The palette was read for every 1-bit indexed image, resampled or not, and twice each. Images
share a table, so 1,000 images sharing one with 16 MiB of raw data took 16 to 60 s, none of
them resampled. It is now read only for an image about to be resampled, and a table over
4 KiB is refused before it is copied or decoded, since no palette needs that.
- A colour key can hide the dark sample a tie goes to, so a thin rule of the other was lost: a
white rule on a hidden black went from 255 to the backdrop's 128. An image with a colour key
now keeps its resolution. Grey had this since #5426 and is covered too.
- `/Indexed` by reference, and a base of `[/DeviceRGB]`, are the same colour space and are read
so. `/I`, an inline abbreviation, and a real hival aren't, and such an image is left alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
From CodeRabbit's review: default colour spaces apply to an indexed colour space's
device base, so one in force where an annotation's appearance (or a pattern, or
another image's /Alternates) draws the image may change which palette entry is darker,
and the tie a resampled image keeps. Only black-and-white RGB went through
used_elsewhere; indexed images now do too when downsampling is on, and one used beyond
the page and form resources the walk read keeps its resolution, as such RGB keeps its
colour space.
A grey image stays out of this check, since sample 0 stays dark in a grey space, and #5426's rule
for grey is unchanged: the test now has a grey image drawn on the page and in an annotation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
From CodeRabbit's review of the stack:
- Every JBIG2 image read its /JBIG2Globals afresh, within 32 MiB each but with no
bound across an export or a redaction. A globals stream of the file, not edited since
it was read and with its /DecodeParms direct, now has its filters undone once a
document and is kept for the images that share it (in every clone, which share the
file), up to 64 MiB in all. An edited one, one added in memory, or one with
/DecodeParms by reference (another object, which can change under it) is read each
time as before. This saves the filters only: the decoder reads the segments afresh for
every image, so a real symbol dictionary still costs its decoding once an image.
- view.copy-image dropped "JBIG2 inside other filters" from its missing list on the
strength of Export's test; Copy Image now has its own: a JBIG2 picture behind
ASCIIHexDecode is found under the pointer and copied as its pixels and as a PNG, and
one whose page information is in its globals is read through them.
A JBIG2 image's /Width and /Height are read as given directly; one by reference is an
error rather than an image of no rows.
A /JBIG2Globals reference to no object still reads as no globals: it is a reference
to null (ISO 32000-2 §7.3.10), which an entry may hold for being absent (§7.3.7), and
the renderer draws the page so. An image that needed those globals then mostly fails
to decode; in a probe of twelve pictures that decoded with them, eleven were refused
and one decoded to other pixels.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Each JBIG2 bitmap was bounded on its own, but a few kilobytes could still exhaust memory:
nothing bounded what a page's segments hold for later ones, or how far one segment could
multiply them. Measured on black-box streams, peak memory before and after:
- intermediate regions, 16 in 774 bytes: 514 MiB, now 98 MiB;
- an 8 MiB symbol exported 1,200 times, a 2,989-byte PDF: killed at 8 GiB, now 79 MB;
- dictionaries of zero-width symbols: 781 MB, now 72 MB;
- dictionaries retaining their coding contexts, 1.1 MB: 2.89 GB, now 71 MB;
- 65,535 references to one dictionary, 78 KB: about 17 GB, now 6 MB;
- a halftone grid of 16,384 cells a side, 98 bytes: 1.1 GB, now 2.4 MB;
- a pattern dictionary of 65,535 patterns, 48 bytes: 539 MB, now 15 MB, and it still draws;
- a custom Huffman table behind nested Flate, a 1,297-byte PDF: killed at 8 GiB, now 278 MB,
which is mostly the decoded stream;
- 11-byte segments behind nested Flate in a stream and its globals, a 4,765-byte PDF: 5 GB, now
602 MB, which is mostly the two decoded streams;
- an Indexed image whose page header claims 2^20 a side, a 743-byte PDF: aborted asking for
137 GB, now 23 MB;
- a chain of 4,000 /JBIG2Globals streams that are JBIG2 themselves, on a 2 MiB stack: overflowed
it and aborted, now 24 MB.
So PageState counts what it stores: regions, pattern and symbol dictionaries and Huffman
tables, with bitmap headers, retained contexts and a charge per item, up to 64 MiB, two square
bitmaps of 16,384 a side. A dictionary's new and exported symbols count against what is left as they are
made, not once it is stored. A segment may refer to at most 2^20 symbols, checked before any are
collected. A halftone grid has at most 2^25 cells. A pattern dictionary keeps at most 16 MiB of
shifted copies, which only speed up drawing, and none that wouldn't fit what the page has left. A custom Huffman table has at most 2^16 lines, and a
stream with its globals at most 2^20 segments and 2^22 references, checked as they are read. A
page past the bounds on a bitmap is refused when the stream is opened, since callers size their
output from its header before the page bitmap is made. And in hayro-syntax, a /JBIG2Globals
stream that is itself JBIG2 isn't decoded: globals are raw segments, so it reads as globals that
can't be decoded.
An image past a bound isn't drawn, as with the other caps, and the rest of the page is. This
bounds memory, not time: decoding work has no total yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
rfd 0.17 gives each filter extension to the XDG portal, GTK and Zenity as
the glob `*.<ext>`, which those match case-sensitively, so a "PDF" filter
of `pdf` hid `Report.PDF`. `AddFilterAnyCase::add_filter_any_case` passes
each letter as a bracket class there (`pdf` -> `[pP][dD][fF]`) and the
extension unchanged on Windows, macOS and the web, which already ignore
case. Every picker filter in ui-egui now goes through it; the imports
used only by native pickers are gated off wasm.
Fixes#5481
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Conflict in crates/edit/src/text_regions/styled.rs: kept this branch's
`row_step` beside main's `WrappedStyledRows`, and main's marker-aware
`splice_with_marker` splice with the branch's `at` (already computed
before the reach check). main's `wrap` now yields (line, soft, gap), so
the reach's widest line in rewrite.rs reads the line from the tuple.
Gates: fmt, clippy -p pdfcraft-edit -p pdfcraft-engine --all-targets
-D warnings, tests edit 362 / engine 388 / 18, xtask parity.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Review follow-up. The radio-choice test only reached the minW clamp, so
a wrong content formula would still pass. A check box with a reserved
caption and no width or minW now must be exactly reserve + box, both
positioned (its right-aligned box ends the field) and flowed (two pack
side by side in an lr-tb row). Each assertion fails with the measure.rs
change reverted: the box lands at 530pt, and the second flowed check box
drops to the next row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
A field with no `w` is growable in width (XFA 3.3 Ch.8, growable
objects): it takes its content's width, clamped to minW/maxW. The
layouter gave every non-button field without a width all the room left
in its container, so a positioned radio group whose choices set only
minW stretched the first choice to the group's right edge. Its box
(drawn at the field's right end) then sat on the last choice's box, and
the group showed one circle where it had two (#5442's Yes/No groups).
A check button without a width is now its box and widget margins plus
its caption beside it (the caption's reserve, else its text), or the
wider of the two when the caption is above or below, within what is
available and clamped by minW/maxW. Other fields keep taking the
available width.
Test: a_radio_choice_without_a_width_is_as_wide_as_its_content, two
choices with minW and no w in a positioned exclGroup; with the change
reverted the Yes box lands at 105.2pt, on top of the No box.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Redaction wrote every cleared copy as Flate, so a redacted Group 4 or JBIG2 scan
page grew several times over until the optimizer ran. A cleared one-bit image of one
component is now stored as the smallest of Flate, CCITT Group 4 and, when it was
JBIG2, JBIG2 (PDF 1.4, which such a file already is), as stored, dictionary included:
a tiny image can lose by Group 4's parameters. Both are coded from the
cleared samples, so nothing of the original's code is carried over; rows Group 4 or
JBIG2 can't code (too wide) leave Flate. Images of more bits stay Flate.
Tests: a cleared one-bit Flate scan of ragged word blocks (where Group 4 is smaller
than Flate) comes out Group 4 and an eight-bit image Flate, each decoding to the
cleared samples (the whole buffer); a tiny image whose Group 4 code is shorter but whose
stream isn't, and rows wider than Group 4 codes, stay Flate and clear; the redacted
JBIG2 scan of #5461's test comes out JBIG2, smaller than
Flate, with no globals and no stream holding the original code. The tests' image
helpers decode through Document::decoded_image, as image codecs need.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5461 made Document::decoded_image read JBIG2, page exactly the image's size, and
kept the optimizer's monochrome step off it until it could weigh JBIG2 again. Now
one_bit reads JBIG2 last after any filters, within the raster's allowance, and mono
treats it as it does any black-and-white image:
- above the threshold it is averaged, and stored as the compression asks, the
smallest of what that may store; under Retain it is stored as JBIG2 again, its kind
of compression kept;
- not resampled, it is stored again as the compression asks when that is smaller: a
generic region becomes symbols with JBIG2 asked for, and JBIG2 behind ASCIIHex is
stored again alone (as Group 4 where that is smaller);
- a JBIG2 image stored again refers to no /JBIG2Globals; one whose page isn't its
size isn't read, and stays.
jbig2_images_are_left_as_they_are becomes jbig2_scans_are_averaged_and_stored_again:
a JBIG2 scan at 600 ppi, alone, behind ASCIIHex and with globals, averages to the
same samples as its Flate copy under each compression (Retain keeping JBIG2); at 300
ppi a generic region of rendered text stays under Group 4 (larger) and Retain and
becomes symbols under JBIG2 (under half the size); behind ASCIIHex it is stored again
alone, as Group 4 where the hex is spaced out; a taller page stays.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5426 left indexed images at their resolution rather than guess which sample is
dark. The palette says: dark_is_one now maps samples 0 and 1 through the decode array
to palette entries (rounded and clipped to hival, ISO 32000-2 §8.9.5.2) and compares
how light those entries are (`lightness`: the grey, or Rec. 601 luma over DeviceRGB).
Only DeviceGray and DeviceRGB bases count, the table a string or a stream (within
4 KiB), directly or by reference: there the palette's values are the colour, and
under /DefaultGray or /DefaultRGB no polarity is asked for. Other bases (CMYK,
calibrated, ICC-based, a resource name), two entries alike, an entry the table
doesn't hold or a hival past 255 still say nothing; separations and DeviceN still
aren't averaged, their tint transforms unread. #5426's tone guard applies as to any
image.
#5426's indexed thin-rule test now checks what it was for: with the palette either
way round and the decode array either way, the averaged image holds the averaged
DeviceGray one's samples (flipped where its samples are) and renders the same. A unit
test covers each base, both orders, an inverting decode array, the weights of luma,
the table by reference and in a stream, and every refusal; the older polarity test's
doc says what is known now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part of #283. A scan stored as JBIG2 (as the optimizer writes them) is decoded
through Document::decoded_image_within, its /JBIG2Globals included, and its covered
pixels cleared in a Flate copy that refers to no globals, as CCITT fax images are.
Before, any JBIG2 image under a mark was removed whole, blanking the page.
Still removed whole (fail-closed): a JBIG2 page of another size than the image,
which decoded_image refuses (the renderer draws the whole page into the image, so
the rows cleared wouldn't be the ones shown); globals that can't be read; damaged
data; an image claiming more than one bit or one component, even where its rows
take as many bytes; a page far past the image's raster, refused before it is
decoded.
Globals no other image uses leave the file with the original on the full rewrite
(tested); globals another image still uses stay, and the README and parity note say
they may hold the shapes of marks from under the area, not where they were.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
decode_jbig2 sits next to decode_ccitt, through hayro-jbig2 as Export ▸ Images
used it, and shares CCITT's row sink. The JBIG2 page has to be the image's size:
PdfCraft's renderer draws the whole page into the image (black box: a 4-row page
behind /Height 2 fills the image box with all four rows), so the rows of a taller
page aren't the image's. The decoder decodes a page whole, so the page is checked
before it is decoded: that size, within the caller's bound as rows of one bit, and
at most 2^28 pixels (MAX_JBIG2_PIXELS; the decoder keeps a bit a pixel, about 34 MB
there).
Document::decoded_image_within decodes a JBIG2 image as it does a fax image: the
filters before it undone first, then the decoder with the /JBIG2Globals of the
image's own /DecodeParms entry (/Crypt counts in the position; fax_params and the
globals now share last_filter_parms). Globals have a limit of their own,
MAX_JBIG2_GLOBALS (32 MiB, moved from Export), and globals that can't be read are an
error rather than ignored.
Export ▸ Images reads JBIG2 through it, so its own decoder, row sink and DecodeParms
reader go, and hayro-jbig2 leaves pdfcraft-create. JBIG2 behind another filter now
exports, and so copies (Copy Image uses the same function); a page taller than the
image, which Export cut to the image's rows, now skips the image, as one of another
width did. doc_export_all_images' description no longer lists JBIG2 inside other
filters as skipped.
The optimizer's monochrome step read images through decoded_image_within and
relied on JBIG2 failing there: it now leaves JBIG2 images alone explicitly, as
before (a resampled one would come out Flate under Retain), and the README and
parity note list them as not yet handled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The integration in #5426 made downsampling conservative: dark_is_one answers only for
DeviceGray and image masks (a decode array of two different numbers or none), and not
under /DefaultGray or /DefaultRGB, and a square of four new pixels that each cover both
samples keeps the image at its resolution. The README, the crate docs, mono()'s comment
and the optimize.downsample-images parity note still said separations and indexed
images were averaged, a tie going to full ink or entry 0.
Measured on main: a 600 ppi page of the ragged-text fixture is no longer averaged
(it is with the square guard switched off), so the docs say so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The vendored hayro-syntax ran PNG predictors through the bit-level
BitChunks/BitWriter machinery TIFF predictors need, although PNG filters are
defined on bytes (ISO 32000-2 §7.4.4.4, PNG specification §9). A 993 x 936
RGB image took about 63 ms to unfilter whatever its row filters; undone on
byte slices it takes about 3 ms (Up) to 16 ms (Paeth) in a release build.
It also fixes a row that ends part-way through a pixel (three 4-bit samples
at an odd width): the bit-level path dropped its last byte, each later row
started a byte early and the picture came out sheared.
TIFF predictor 2 keeps the old path. Vendored hayro-syntax patch 29.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Includes #4325#1952#4180#2529#4320#4805, preserving their original histories.
Keep paragraph alignment and hard/soft wrap metadata together; retain CCITT/downsampling/RGB ownership guards with bounded JBIG2 export and exact-copy symbol encoding.
Combined interaction regressions added. Validation pending.
Use a named compact preset arrow beside the field, retain the row alignment/width assertions for the editable control, and open the current colour flyout in the saved-text regression. No accessibility or layout checks are disabled.
Preserve caption editing and current form/Prepare module boundaries. Carry export-only appearance preservation, validation and data round-trip tests to their current paths, and retain current attribution metadata. Hosted CI must validate this refreshed head.
Preserve main's continuous text selection, page tools and module split. Carry only this branch's sidebar-focus feature and regression onto main, without resurrecting old common files from criss-cross merge bases. Hosted CI remains required.
Use the unified Format Text panel and current page-input context instead of restoring old monolithic UI code. Preserve main's pointer routing, editor accessibility and preferences; retain colour, save and input regressions. Hosted CI must validate this refreshed head.
A larger range on the first line of a multi-line paragraph rises from the
first line's baseline, not from the old box's bottom, so the reach now
extends one pitch above the moved box's top. Name the 1.2 em line floor.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
With only a range enlarged, row_step is the source leading, so the
styled rewrite's reach rose one source line above its old box while a
120-point range draws far higher, and an image there was missed. The
pitch is now at least 1.2 times the largest size.
The landing test gains a 120-point range on a 12-point OCR line under an
image above the old box; it fails without the floor.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
The styled path writes rows `line_spacing` (1.5 by default) times the
size apart when a size or spacing is set, but its Reach assumed 1.2, so
an image over only the lower rows of a larger rewrite was missed. The
pitch now comes from the row writer's own step (row_step, shared by
both), at the largest size among the paragraph's lines and ranges.
The landing test gains a six-row rewrite at twice the size over an image
covering only its last row; it fails with the 1.2 step.
A case with only line_spacing set covers the other branch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Grzegorz MartensandClaude Opus 5.5
Dylan M. Taylor
KenM76andClaude Opus 5.5
songzhihui
skylar taylor-barrick
FredApps