docs/target-app-parity.md's optimize row lists lossy JBIG2, refinement
coding and a symbol dictionary shared between images as missing, with a
status line and a revision row.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
The optimizer now writes JBIG2 images as a symbol dictionary and text
regions, and Export ▸ Images has to read them back. A page of three
repeated letters, symbol coded (smaller than the generic coding),
exports as its picture. The test is added to create.export-all-images'
parity evidence. Asked for in pre-push review.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Review of the previous commit:
- Nothing checked the segment numbers the region split assigns: a
mutant numbering every region 2 decoded. The many-marks test now
reads each segment's number, and has a bar too tall to be a symbol,
so the generic region is numbered after the last text region.
- The decoder's 32,767-symbol limit was checked against all distinct
marks, before either plan, so an image with more distinct marks lost
the second plan, which codes only the repeated ones. The limit now
applies to each plan (MAX_CODED_SYMBOLS); finding marks keeps its
memory bound of 65,536 (MAX_SYMBOLS), as before the previous commit.
A test has 32,768 distinct marks above a page of repeated text, which
encode_jbig2 still symbol codes through the second plan.
- The dense-page optimizer test now checks that it takes several text
regions, and a compile-time assert keeps segment numbers within a
byte's reach.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
A review found that a dense page came out as a stream PdfCraft's own
renderer refuses, so the page drew blank after optimizing: hayro-jbig2
decodes a text region of at most 10,000 symbol instances, and refuses
32,768 symbols or more. The stream was valid (Poppler decoded it), but
every mark went into one text region and up to 65,536 symbols were kept.
A 300 dpi letter page of the optimizer test's own text crossed it.
A page with more than 10,000 marks is now placed by several text
regions, each over a band of rows and all referring to the one symbol
dictionary, which each region but the last keeps retained (7.2.4). Past
32,767 distinct symbols the image is coded generically.
New tests: 25,000 copies placed by three regions, which decode; the
symbol caps decoded, not just reached; and in the optimizer, a whole
letter page of rendered text that looks the same as JBIG2 symbols (it
drew blank before). The READMEs say "still looked at" for the caps, and
that the scans figure is the six pages' total (20% on one page).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
A second review found two rules tested one way only: a mark 257 tall
(not just wide) is left to the generic region, which also bounds the
bitmap allocated for each mark, and runs joined at the lower-left corner
are one mark. The test now has a 1 x 257 column and an anti-diagonal.
The caps on distinct symbols (65,536) and their pixels (2^24) are tested
at and past them, as the run and row caps are.
The scans figure now names its noise: with 2% or more of the letters'
edge pixels flipped, symbols save at most 8% over the generic region.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Review found behaviour the tests let break unnoticed (each was a
surviving mutant, and each new test fails on it):
- black padding bits: a run reaching past the row would stop symbol
coding on many scans;
- Annex A's 32-bit range and PREV at exactly 256, where a decision would
go uncoded: coordinates past 4,435 on a 12,000-pixel-wide and a
10,050-row page, and the contexts of 4436's coding;
- marks joined at a corner are one mark; 256 pixels a side is a symbol,
257 is left to the generic region;
- the run and row caps, at and past them.
The roadmap's status line for this change now comes first, and the
"about a third smaller than Group 4 on scans" figure is replaced by what
was measured: 27-33% on the same pages with scan-like noise added.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
pdfcraft_filters::encode_jbig2 stores each distinct mark of a 1-bit
image (8-connected black pixels, at most 256 a side) once, as a symbol
in a symbol dictionary, and places its exact copies with an immediate
lossless text region that refers to it. Larger marks, and in a second
plan marks that occur once, go to a generic region over their box. It
keeps whichever of that and the generic coding (encode_jbig2_generic,
unchanged) is smaller. Only exact copies share a symbol, so the stream
decodes to the same bits.
Written from the JBIG committee's ISO/IEC 14492 Final Committee Draft:
6.4 and 6.5 (text region and symbol dictionary decoding), 7.2, 7.4.2
and 7.4.3 (segment headers and syntax) and Annex A (the integer coders,
checked against its worked example and Table A.1).
On six pages of rendered 300 dpi text the output is 28% of the generic
coding's size and 17% of Group 4's, at about 100 ms a page; hayro-jbig2
and Poppler (pdfimages) both decode it to the original bits. Scans,
whose copies of a letter are seldom identical, gain little: that needs
refinement coding. A dictionary shared between pages (/JBIG2Globals)
is a further step.
The optimizer's Monochrome Images > JBIG2 (lossless) and doc_optimize
mono jbig2 use it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
An image's /JBIG2Globals were decoded with the 1 GiB any stream may decode
to, before the JBIG2 page's size is known, so the page's pixel limit didn't
bound them: a few kilobytes of Flate could make export allocate up to 1 GiB.
They now decode within MAX_GLOBALS (32 MiB); globals past it skip the image,
with the limit in the reason. The test covers one byte over and exactly the
limit, and the image's /DecodeParms direct, by reference and in an array.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test only checked each refusal's message. Without the tool's notation guard, set_props
refuses the same notation with the same message as a failed edit, so the test passed with the
guard removed. It now asserts InvalidArgs for every row, and fails on "a EVCALC b" without the
guard.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ckb and nb get all eleven rows, th and lo the six strings, and sq the five plurals beside its
strings, each under "# Untranslated". Their rows next to these were missing too, so following
them had added nothing.
hi, id, ko, lo and th list one plural form (plural_none), so they get no English copy of the
plurals: one English wording would be used for every count ("1 formulas read ..."). Without an
entry, the English fallback picks the right form, as it already does for the page's other plurals
in those catalogues.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
set_props refuses such a notation before writing anything, so the
Calculate tab can't store one either (the tool already refused it). A
refused edit used to close Field Properties and lose everything typed
into it; the dialog now stays open with its changes, as the dialog for
several fields does, and the notice says why (from the review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
parse_calculate trimmed the notation with str::trim, which also took an
escaped space ending its last name: "Qty * Total\ " read back as
"Qty * Total\", a field the form doesn't have, while the script's
JavaScript still read "Total ". It now trims to its first and last
token, as the field-reading scan does. notation_storable compares the
names and operators read back with those given, so a notation ending in
a lone \ (which escapes the space after it) is refused; a proptest reads
back notations of any names (from the review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
A notation is kept in the script's /** BVCALC … EVCALC **/ comment and read
back from there. One holding EVCALC was stored cut short ("a EVCALC b"
read back as "a", and its JavaScript translation didn't parse), one
holding */ ended the comment, and an empty one became "event.value = ;".
form_set_props now refuses each (af::notation_storable: it has to read back
as itself). The test also pins a notation beside op alone and beside
fields alone (from the review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
steps() takes the widgets by object, so each of two fields sharing a name
gets its own step; a test checks it, unfocused and with a focus (from the
review: no test failed with the step looked up by name).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
A notation beside op and fields set the notation and dropped the rest, a
field name that wasn't a string was left out, a misspelt key was ignored,
and validate {"min": 0, "max": "100"} wrote a min-only range. Each is
now refused and nothing changes. A null bound, as form_fields shows the
one a range doesn't have, is no bound, so its output is taken back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
AFSimple_Calculate("SUM", "a, b", x) gave only a built-name site, so a
rename no longer rewrote a and b (main did) while parse_calculate still
read the sum of a and b: after a rename the sum read missing fields. The
string list now keeps its names when an argument follows, and what
follows is a name built at run time, as for an array (from the review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
The page's focus was a name, so with two fields called total (a
malformed form) hovering the second one's row showed the first one's
missing fields in the pill, and outlined and numbered both on the page.
A row and a widget under the pointer now carry their field object: the
pill says that field's reads and issues, and the page outlines it, its
step and the formulas that read it by object. The Prepare selection is
still only a name, so it stands for every field of that name and the
pill says all of their issues (from FredApps's review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
found accepted price.0 (up to the field's last widget), but links,
recalculate and the scripts' getField all take a name as the field or
the fields under it, so a formula reading price.0 got no value and no
order check while the page said nothing was wrong. found now looks
names up the same way; a widget name is flagged as missing until all of
them resolve it (from CodeRabbit's review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A read like price.12 counted as found whenever price existed. A widget
number now finds something only below the field's widget count; a
field with one widget, or none, still answers to .0 (from CodeRabbit's
review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
calculate_json wrote the lowercased AFSimple code, so a product or an
average came back as {"op": "prd"} or {"op": "avg"}, which
form_set_props refused as an unknown op. One name per operation now
serves both: calculate_arg looks names up in it (still taking minimum
and maximum), and the listings write it (from CodeRabbit's review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
calculate_arg read the notation first, so {"notation": …, "script": …}
set the notation and dropped the script unnoticed; the script check now
comes first (from CodeRabbit's review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
- CalculatedField carries the field object: form_calculation_order and
the Calculation order page find each row's formula and value by it,
so two calculated fields sharing a name (a malformed form) each show
their own.
- An AFSimple_Calculate list counts as written out only when it ends
the call: "row" + suffix and [...].concat(more) are names built at
run time, and an array item counts only when it ends at a comma or
the list's end, so ["row" + i] no longer reads, or renames, "row".
- After a reorder, Undo is offered on the missing-field card instead of
bringing back the all-clear beside it.
From CodeRabbit's review.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
The row gets a red step and a "Missing field" caption, as a loop does; a
red card names the formulas and links to the first, in place of the
all-clear; a script that builds names gets a grey "Unchecked reads"
caption and a note; the summary above the page lists a missing field
among the reads and says so, and says when a script names fields as it
runs. The name and formula lines of a row are sized by their own
right-hand text, so a wide caption no longer cuts the name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
calculation_order flags MissingField (a field deleted from under a sum, a
typo in notation, a script's getField; a group's name and a widget's
number count as there) and BuiltNames (a getField or AFSimple_Calculate
list built at run time, found as af::Site::Built in the same pass as the
literal names). form_calculation_order gives each field's formula and the
two new issues; form_fields shows a custom Calculate script; the
Calculation order page explains both in its row tooltips.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
docs/architecture.md said every image codec decodes through the
bootstrap renderer; Export ▸ Images decodes CCITT (hayro-ccitt) and JPEG
itself. The progress logs' "(a dithered scan)" sat after "soft mask
counted", as if it described the mask, and so did the
optimize.mono-compression note's "(a dithered scan downsampled to JPEG)".
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
PdfCraft's bootstrap renderer draws byte-aligned Group 3 whose fill ends
each end-of-line code on a byte (as TIFF writes it) as other rows than
decode_ccitt reads (black box, at every /K, with /EndOfLine or not:
mostly the first row, then blank). Redaction cleared pixels in the
picture decode_ccitt reads, so a mark over what PdfCraft showed needn't
cover what was cleared, and content shown blank stayed in the copy. Such
an image (/K 0 or more, /EncodedByteAlign) is now removed whole, as
every CCITT image was before this PR; Group 3 off bytes and Group 4 are
still cleared.
The image's parameters come from Document::fax_params, the reading of a
fax image's /DecodeParms entry that decoded_image already did, now on
its own (it was to come with #4404, which uses it too). The redaction
README, the module doc, the redaction and core.image-filters notes and
the spec parity table's status line and revision row say so, and the
core.image-filters note now records the renderer's misdrawing.
group_3_on_bytes_is_removed_whole_and_other_fax_images_cleared: a white
16 × 2 picture, checked to decode as such, is removed whole on bytes (no
codes; codes at /K 0; at /K 1 with /EncodedByteAlign by reference) and
cleared off bytes and as Group 4 on bytes. It fails without the guard;
cargo-mutants catches every viable mutant of it.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
group_3_fax_images_become_group_4 coded its rows only off bytes. It now
also stores again rows on bytes: Modified Huffman (no end-of-line codes),
and codes whose fill ends each on a byte, as TIFF writes them, at /K 0 and
1 without /EndOfLine, which reading by that flag misread (the test fails
with that rule). Each becomes Group 4, smaller, decoding to the picture.
The page is compared with the picture in Flate rather than with itself
before: PdfCraft's bootstrap renderer draws TIFF's layout (the fill
ending each code on a byte) as other rows, at every /K and with
/EndOfLine or not (black box), so storing it as Group 4 corrects what
PdfCraft shows rather than keeping it. (Redaction removes such images
whole, for that reason, since #4384's "redaction removes Group 3 on
bytes whole".)
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
decode_ccitt reads byte-aligned Group 3 whose data has end-of-line codes
but none before the first row (not as T.4 sends them) on bytes. At /K 0
with the fill ending each code on a byte, about one small picture in
five is then read as other rows; at fax width almost none are, as such
pages come out short and are refused. Export already said so. The
optimizer, when it resamples such an image or stores it again as Group 4
from behind another filter, and redaction, when it clears pixels in one,
write those rows back, and nothing said that.
Black-box, PdfCraft's renderer draws such an image exactly as
decode_ccitt reads it (all 42 misread of 200 seeded pictures, behind
ASCII85Decode too), so the written image is what PdfCraft showed before.
Refusing such data instead (a code after nearly every row but none
first) would also refuse some ordinary Modified Huffman images, whose
rows can meet 11 zeros by chance (0.5 to 5% of small ones in a
black-box probe), so the limit is stated rather than guarded: in
decode_ccitt's doc, the filters, optimize and redact READMEs, and the
core.image-filters and mono-compression notes. The optimize README's and
the mono-compression note's "not yet: Group 3 to Group 4 without
resampling" now say a lone Group 3 image: one behind another filter is
stored again as Group 4.
Also from the review: docs/roadmap.md keeps #4335's status line under
this PR's, rather than replacing it; the filters README's tests and the
core.image-filters evidence name the Group 3 layout test; the filters
crate's description names Group 3 and 4 decoding.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
The optimize crate doc and MonoSettings named only 1-bit, CCITT and
eight-bit grey images; doc_optimize's mono said Group 4 only; the M11
milestone row stopped at Group 4 compression. They now include
downsampling (halftones aside), Group 3 to Group 4, 1-bit Flate, and
black-and-white RGB and indexed images that use only palette entries 0
and 1, packed to one bit. That last isn't "two-entry", as the roadmap and
the downsampling parity note had it: an indexed image packs whatever its
palette's size, when its samples are 0 and 1. The mono-compression note
says byte-aligned Group 3 with end-of-line codes but none before the
first row is stored again as PdfCraft draws it, which at /K 0 may be
other rows than were coded.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
The rule came into decode_ccitt with the decoding it moved from Export ▸
Images (#4335's "Export reads byte-aligned Group 3 by the codes in its
data"), where the create test checks it through the export. Here it gets
tests of its own:
- 32 seeded text-like pictures, 8 to 207 samples wide, at k 0 and 1, in
each layout (no codes, with end_of_line and without; codes, each
starting on a byte or ending on one, with and without). Each decodes to
its picture and, a byte short, to fewer rows, not to other rows read
another way.
- Group 4 rows on bytes keep their aligned read, end_of_line or not.
- The any-input property varies end_of_line and encoded_byte_align too.
With the rule this replaced (rows on bytes read unaligned under
end_of_line, aligned otherwise) the test fails; cargo-mutants leaves no
mutant of the rule's lines alive. The filters README says how rows on
bytes are read.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
- Packing compared an indexed image's decode array, and RGB's pairs, as
written: [0 255.0], an entry by reference or a real among RGB's pairs
stopped it, though they mean the same as integers. Each entry is
resolved and read as a number now, as is_identity_decode and
dark_is_one read them; RGB's pair is written back direct.
- A test holds that Retain stores nothing again it doesn't resample: a
Group 3 halftone above 450 ppi, which isn't averaged down, stays Group
3 byte for byte (the guard's || as && passed every test).
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Mutation testing found behaviour no test held, now covered:
- a Group 4 halftone above 450 ppi isn't resampled and stays byte for
byte, alone by default and behind ASCIIHexDecode with Retain;
- under Retain a resampled Flate scan stays Flate, though Group 4 would
be smaller (ragged bands);
- sixteen-bit grey of only 0x0000 and 0xFFFF keeps its bytes;
- a busy Group 4 image behind ASCIIHexDecode, whose code is over three
times its raster, decodes within the fax bound and is stored again as
Group 4 alone. Redaction's busy-fax test is that busy again: since the
bound for other images is three times the raster, the old checkerboard
fitted it, and dropping the fax bound passed.
The redaction README says three times, and doc_optimize's description
says what Retain keeps: CCITT stays CCITT, anything else becomes Flate.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
With end-of-line codes, the CCITT decoder picks a bad row up again at
the next code and reports no damage (an invalid code, a row of too many
pixels: probed as a black box), so such an image is stored again as
Group 4 the way it decodes. Without them, damage stops the decoding,
too few rows come out, and the image is left alone. Telling damage
apart would take a Group 3 parser of our own; the optimize README and
the mono-compression parity note say so instead.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
- The optimize README, the crate doc and both parity notes say a
halftone or dither, and an image wider than a fax row, isn't averaged
down.
- The README's "Left alone" list is the colour and grey settings'; image
masks, decode arrays and CCITT go to Monochrome Images now. Its
monochrome "Not yet" names an image's own /SMask or /Mask image again
(only images drawn on a page are visited), as the parity note does;
the parity note's row limit is Group 4's.
- docs/architecture.md says CCITT decodes through pdfcraft-filters for
export, redaction and the optimizer, not only the renderer.
- docs/target-app-parity.md no longer lists monochrome downsampling as
missing.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Mutation testing found three behaviours no test held:
- eight-bit black-and-white in calibrated grey, an ICC profile of one
component or a separation is packed, each keeping its colour space,
and calibrated RGB of only 0 and 1 isn't (taking every array colour
space for an indexed one passed every test);
- mixed Group 3 (/K 1, with end-of-line codes) is stored again as
Group 4 (taking /K above 0 for Group 4 passed);
- a resampled Group 4 image of vertical stripes is stored as 1-bit Flate,
which is smaller (ruling Flate out for any CCITT image passed).
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
- Redaction decodes an image no further than twice its raster plus a
byte a row, meant for ASCIIHex text under Flate; hex text broken into
lines is a little more, so such an image was removed whole instead of
cleared (already so on main). It is three bytes for each now, as the
optimizer allows.
- Tests: grey stored as hex text in lines under Flate, and RGB behind a
filter array (which a fax image's bound would refuse), are cleared;
decoded_image_within gives a fax image no more rows than its bound
holds.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
- Packed to one bit, black-and-white DeviceRGB became DeviceGray. Where
/DefaultGray or /DefaultRGB is in force (ISO 32000-2 §8.6.5.6), that
drew it otherwise: through DefaultGray as grey, or no longer through
DefaultRGB. The page walk now notes the images drawn, on any use, where
a page's or form's resources (an enclosing one's included) name
either, and those stay RGB. When a use couldn't be followed, every
image counts as such.
- One whose soft mask has a /Matte stays RGB too: the matte has one
number for each of the parent's components, three, and grey has one.
- The optimize README, the mono-compression parity note and doc_reduce's
description said Group 3 to Group 4 wasn't done yet, that only images
packed from eight bits could become Flate, and that the colour space
and decode array always stay. They now match the code.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
- Halftones and dithers: most of their new pixels cover both black and
white, and taking the sample that covers more turned their tones
solid (a 50% screen black, a 25% one white). An image whose new pixels
mostly cover both is now left at its resolution; text covers both only
along its edges and is still averaged down. downsample_bits says how
many new pixels cover both.
- A counter for each column: an image wider than a fax row
(MAX_CCITT_COLUMNS) isn't averaged, so a strip of 80 million columns
can't make the counters 640 MB.
- A fax image's decoding is bounded at about a byte a pixel, as
redaction's is (decoded_image_within), not 1 GiB; behind a Flate bomb
it is left alone.
- An indexed image's tie went to sample 0, which an inverting decode
array maps to entry 1, though the docs say entry 0. It goes to the
sample that decodes to entry 0 now (ISO 32000-2 §8.9.5.2).
Each has a test that fails without it.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Monochrome Images only took eight-bit grey of only black and white
besides 1-bit images, so the same scan written as RGB, or as an indexed
image of two palette entries, stayed at eight bits (and RGB went to the
colour settings, as JPEG). Now these are packed to one bit first:
- eight-bit DeviceRGB whose every pixel is black or white becomes
DeviceGray; a decode array the same for each component becomes that
pair, another stops it. Calibrated RGB and RGB with a profile stay.
- an indexed image of 2, 4 or 8 bits that uses only palette entries 0
and 1 keeps its palette, whatever the two colours; a decode array
other than the default stops it.
Colour-key masks still stop all of them, and Retain leaves them to the
grey and colour settings. They are decoded through /DecodeParms given
by reference too (Document::with_direct_parms), and when downsampled a
tie goes to the dark sample of the image as packed: packed RGB is grey,
with grey's decode pair.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
Redaction clears an image mask's covered samples to the value that
paints nothing, which an inverting /Decode [1 0] flips. It read the
array only when direct, so given by reference (or with its first number
so) the cleared samples painted instead: the covered area came out
inked. That was already so on main, and this PR sends CCITT masks
through it. The array and its first number are resolved now, as export
and the optimizer read them; the fax test covers all three forms.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
A CCITT fax image that wasn't resampled was left as it was, so Group 3
images (the 1-D and 2-D codings fax software writes) stayed Group 3.
Group 4 codes each row against the one above and is usually much
smaller. Now only Group 4 alone skips decoding; Group 3 is stored again
as Group 4 when that is smaller as stored, and Retain still keeps it.
It reads an image's CCITT parameters (its /DecodeParms entry with Table
11's defaults) through pdfcraft_cos::Document::fax_params, from #4384.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
doc_reduce, the engine's reduced_bytes doc, the optimize README's
presets and the optimize.reduce parity note still said every image
above 225 ppi goes to 150 ppi; black-and-white images go to 300 ppi
above 450 ppi, as CCITT Group 4 when smaller.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Redacting a CCITT image went through decoded_image, which undoes the
filters before the fax decoder up to MAX_DECODED, so a Flate bomb in
front of a fax image cost a gigabyte while every other image stops at
twice its raster. Document::decoded_image_within takes a bound, and
redaction gives a fax image's code about a byte per pixel. A fax image
cut short, or behind a bomb, is removed whole; a busy one whose code
is bigger than twice its raster is still cleared.
- Every image is now decoded through decoded_image_within, which reads
/DecodeParms through Document::with_direct_parms: given by reference,
the numbers inside (/Predictor 7 0 R) included. That reading was first
written here and has moved down to #4304. The predictor test now covers
the dictionary, an array's entry and the number itself given by
reference; read without them, the cleared bytes drift off the covered
pixel and the covered sample stays in the copy.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The batch step took Reduce File Size's monochrome settings unchanged, so
`ppi: 600` (or `--ppi 600`) kept colour and grey images at 600 ppi but
halved 600 ppi black-and-white scans to 300. Black-and-white images now
go to the larger of ppi and 300 ppi: a higher ppi keeps their
resolution, and a lower one doesn't take scanned text below 300. The
doc_batch description and docs/cli-batch.md say so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Create PDF and the optimizer now store black-and-white scans as Group 4,
and redaction couldn't decode CCITT, so redacting one word on such a
scan removed the whole page image. A CCITT image (the last filter,
after any others) now decodes through Document::decoded_image and is
cleared in a Flate copy like any other image; one that can't be decoded
in full is still removed whole.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stream::decoded_within reads /DecodeParms only when it is direct, so an
image whose PNG predictor was given by reference decoded with its row
tags still in the data. The cleared bytes then drifted one byte further
from the covered pixels each row, and the covered samples survived in
the redacted copy, recoverable by applying the predictor. Parameters
are now made direct first (Document::with_direct_parms).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README said an indexed image's palette decides which entry is the
dark one on a tie, but dark_is_one never reads the palette: entry 0
wins whatever its colour. The README, the function's comment and the
parity note now say so (from FredApps' review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
