M11: JBIG2 images share a symbol dictionary, up to 16 at a time (optimize.mono-compression) (#5910)

* M11: JBIG2 images share a symbol dictionary, up to 16 at a time

With Monochrome Images set to JBIG2, Optimize stored each image's
letters in a symbol dictionary of its own, so a document of N pages
carried its font's shapes N times. Now up to 16 images the run stores
as JBIG2 share one dictionary in a /JBIG2Globals stream, holding the
marks found in more than one of them; each image's text regions refer
to it first, then to a dictionary of its own for the marks only it
repeats. An image is stored that way when it comes out smaller, and a
group only when what its images save is more than the globals take.

- filters: encode_jbig2_shared codes a group: the globals are a symbol
  dictionary segment numbered 0 and associated with no page (7.2.6);
  each page numbers its segments from 1, as a segment refers only to
  lower numbers (7.2.5), and the shared symbols' IDs come first, as the
  dictionaries are referred to in that order (Table 31, SBSYMS). At
  most 16,383 shared marks, those in the most images, so a page keeps
  room for its own under the decoder's 32,767.
- optimize: the run holds the 1-bit samples of the images it stores as
  JBIG2 (16 at most, 64 MiB at most) and stores them again sharing,
  weighing each with the longest reference the globals could have.

Eight pages of 10 pt text rendered at 300 dpi take 41% less in
Helvetica and 42% in Times than coded alone, 10% in Courier, and 46%
in Helvetica at 600 dpi; groups of 2, 4, 16 and 32 save 19%, 33%, 45%
and 47%. Redacting one image stores the others of its group again
without the dictionary, so 16 keeps that to 15 images.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* M11: test what the mutants found untested in shared JBIG2 dictionaries

- filters: the group goes through encode_jbig2_shared, which refuses it
  for an image the encoder refuses; two marks in common are enough to
  share; past the 65,536 distinct marks a group holds, a mark first seen
  later isn't shared while those held count every image.
- optimize: optimizing again keeps both the shared pages and the pages
  coded alone as they are; under Retain, averaged JBIG2 pages share
  nothing; a last group of two shares.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* M11: only images redaction can store again share a JBIG2 dictionary

Redacting one image of a group stores the others again without the
dictionary, which can hold the shapes of marks under the area, and
refuses the redaction for one it can't: it takes image masks and
images of one component as its own reading counts them (no DeviceN,
the family named directly), and at most 64 Mi pixels. Optimize took
DeviceN of one colourant and up to 80 million pixels, so such an image
in a group made any redaction of the others fail, where before it was
coded alone and the same redaction worked (the Consumers review).

Only an image mask or a DeviceGray image named directly, of at most
2^26 pixels, now joins a group; others are coded alone. An engine test
optimizes three pages, two DeviceGray and one DeviceN, and redacts the
first: the DeviceN page stays alone, the second is stored again, and
no image names the dictionary after a save.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* M11: optimize and redaction share one limit on the images redaction stores again

The second Consumers review: Optimize's limit on images that share a
JBIG2 dictionary copied redaction's private 64 Mi, with nothing tying
the two, so a lower one in redaction would bring back refused
redactions with no test going red. Both now read
pdfcraft_cos::MAX_REDACTED_IMAGE_PIXELS. The engine test adds an image
mask, the other kind that shares, so a redaction that stopped storing
either kind again fails it. doc_optimize's description, the README,
the parity note and ROADMAP say which images share (named directly).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* M11: a JBIG2 group is coded before its samples would pass the limit

CodeRabbit on #5910: hold added an image to the group and then checked
SHARED_BYTES, so a group could pass the 64 MiB it is documented to stay
within by its last image, which can itself be up to 64 MiB (an image
that shares has at most 2^26 pixels, and as its rows are whole bytes,
at most that many bytes; a 1-pixel-wide one takes them all). hold_in
codes the group first when an image would take it past the limit, and
after when it has 16 images; with room for 10 bytes, images of 4 go two
to a group, where the old order made one of 12. CodeRabbit's follow-up
too: the 18-image test decodes every image and compares it with its
samples, across the boundary between the groups.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* optimize: count the globals' object framing; filters: test the shared coder's size limit

- A JBIG2 group shares a dictionary only when it saves more than the
  globals' dictionary and data plus NEW_OBJECT (96 bytes: the object's
  framing, its xref entry, and an encrypted file's IV and padding), so a
  group that saves less than that no longer makes the file larger.
- encode_jbig2_shared is Unsupported for an image taller than the
  decoder takes and codes one at the bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filters: drop the unused rows binding in encode_shared

page() no longer takes the rows since #5511.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filters: the shared coder's at-bound test says what it returns

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>

parity/acrobat-features.toml

@@ -7439,8 +7439,8 @@ milestone = "M11"
 status = "partial"
 commands = ["optimize.advanced", "optimize.reduce"]
 tools = ["doc_optimize", "doc_reduce"]
-evidence = ["crates/optimize/src/images.rs::flate_images_past_the_runs_allowance_stay_as_they_are", "crates/optimize/tests/optimize.rs::eight_bit_black_and_white_images_become_one_bit", "crates/filters/src/ccitt.rs::bitmaps_round_trip", "crates/filters/src/ccitt.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/filters/src/ccitt.rs::what_it_cannot_encode_is_an_error", "crates/optimize/tests/optimize.rs::black_and_white_images_become_ccitt_group_4", "crates/optimize/tests/optimize.rs::group_3_fax_images_become_group_4", "crates/optimize/tests/optimize.rs::rgb_and_two_colour_indexed_black_and_white_images_become_one_bit", "crates/optimize/tests/optimize.rs::black_and_white_rgb_used_beyond_the_page_content_read_stays_rgb", "crates/optimize/tests/optimize.rs::black_and_white_rgb_drawn_only_where_the_optimizer_reads_becomes_grey", "crates/optimize/tests/optimize.rs::black_and_white_rgb_in_resources_its_page_tree_and_a_form_share_becomes_grey", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_ccitt_group_4", "crates/ui-egui/tests/it/create.rs::the_pdf_optimizer_dialog_saves_an_optimized_copy", "crates/filters/src/mq.rs::the_annex_h_test_sequence_encodes_to_its_bytes", "crates/filters/src/jbig2.rs::bitmaps_round_trip", "crates/filters/src/jbig2.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/optimize/tests/optimize.rs::black_and_white_images_become_jbig2", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_jbig2", "crates/filters/src/jbig2/symbols.rs::repeated_marks_are_stored_once", "crates/filters/src/jbig2/symbols.rs::scattered_marks_round_trip", "crates/optimize/tests/optimize.rs::rendered_text_becomes_jbig2_symbols", "crates/optimize/tests/optimize.rs::jbig2_scans_are_averaged_and_stored_again", "crates/optimize/src/images.rs::jbig2_images_past_the_runs_allowance_stay_as_they_are", "crates/optimize/tests/optimize.rs::an_image_too_large_for_the_jbig2_decoder_stays_as_it_is", "crates/optimize/tests/optimize.rs::what_is_too_large_for_jbig2_is_stored_another_way", "crates/optimize/tests/optimize.rs::group_4_takes_images_of_one_bit_a_pixel_up_to_the_pixel_limit"]
-notes = "All of a run's images share one allowance of decoding, filters, decoders and pixels alike (about 1,000 scanned pages' worth, more for a longer file); those past it stay as they are, those the image pass read being counted in the report, which doc_optimize and doc_reduce return and the panel doesn't show yet. CCITT Group 4, lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (on by default, so Reduce File Size does it too; doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. Packed to one bit first, then stored the same way (colour-key masks aside): eight-bit grey of only black and white, eight-bit DeviceRGB of only black and white (as DeviceGray; calibrated RGB, RGB with a profile, RGB drawn or listed where /DefaultGray or /DefaultRGB is in force or referred to from anywhere but the page and form resources read (an annotation's appearance or a pattern drawing it, directly or through resources or a form shared with a page; another image's /Alternates), and RGB whose soft mask has a /Matte aside), and an indexed image of 2, 4 or 8 bits that uses only palette entries 0 and 1 (keeping its palette), unless the grey or colour settings make it smaller still, as they can a dithered scan downsampled to JPEG, its soft mask counted. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. Group 3 fax images are stored again as Group 4 when that is smaller (damaged Group 3 with end-of-line codes as the decoder recovers it, as it reports no damage there; byte-aligned Group 3 with end-of-line codes but none before the first row, not as T.4 sends them, as PdfCraft draws it, which at /K 0 may be other rows than were coded). CCITT fax images are decoded to be downsampled (optimize.downsample-images), and a resampled image is stored as Group 4 or 1-bit Flate, whichever is smaller. Lossless JBIG2 with bounded exact-copy symbol coding is available too, with image export; lossy JBIG2 remains unsupported. JBIG2 images are read too: averaged like any (under Retain stored as JBIG2 again), and, not resampled, stored again as the compression asks when that is smaller (as JBIG2 symbols where its own coding is a generic region, say); a symbol-coded scan, averaged, loses its exact repeats and is often larger, and then stays as it is; one whose JBIG2 page isn't its size is left alone. Not yet: images over 80 million pixels (left alone by the image steps, whatever their settings; Clean Up may still Flate an unfiltered or LZW one), inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples, JBIG2 past 65,535 samples on a side, where the decoder stops: such an image isn't stored as JBIG2 (the image step leaves a 1-bit one as it is unless it is averaged, and may store one packed from more bits as 1-bit Flate)."
+evidence = ["crates/optimize/src/images.rs::flate_images_past_the_runs_allowance_stay_as_they_are", "crates/optimize/tests/optimize.rs::eight_bit_black_and_white_images_become_one_bit", "crates/filters/src/ccitt.rs::bitmaps_round_trip", "crates/filters/src/ccitt.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/filters/src/ccitt.rs::what_it_cannot_encode_is_an_error", "crates/optimize/tests/optimize.rs::black_and_white_images_become_ccitt_group_4", "crates/optimize/tests/optimize.rs::group_3_fax_images_become_group_4", "crates/optimize/tests/optimize.rs::rgb_and_two_colour_indexed_black_and_white_images_become_one_bit", "crates/optimize/tests/optimize.rs::black_and_white_rgb_used_beyond_the_page_content_read_stays_rgb", "crates/optimize/tests/optimize.rs::black_and_white_rgb_drawn_only_where_the_optimizer_reads_becomes_grey", "crates/optimize/tests/optimize.rs::black_and_white_rgb_in_resources_its_page_tree_and_a_form_share_becomes_grey", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_ccitt_group_4", "crates/ui-egui/tests/it/create.rs::the_pdf_optimizer_dialog_saves_an_optimized_copy", "crates/filters/src/mq.rs::the_annex_h_test_sequence_encodes_to_its_bytes", "crates/filters/src/jbig2.rs::bitmaps_round_trip", "crates/filters/src/jbig2.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/optimize/tests/optimize.rs::black_and_white_images_become_jbig2", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_jbig2", "crates/filters/src/jbig2/symbols.rs::repeated_marks_are_stored_once", "crates/filters/src/jbig2/symbols.rs::scattered_marks_round_trip", "crates/optimize/tests/optimize.rs::rendered_text_becomes_jbig2_symbols", "crates/optimize/tests/optimize.rs::jbig2_scans_are_averaged_and_stored_again", "crates/optimize/src/images.rs::jbig2_images_past_the_runs_allowance_stay_as_they_are", "crates/optimize/tests/optimize.rs::an_image_too_large_for_the_jbig2_decoder_stays_as_it_is", "crates/optimize/tests/optimize.rs::what_is_too_large_for_jbig2_is_stored_another_way", "crates/optimize/tests/optimize.rs::group_4_takes_images_of_one_bit_a_pixel_up_to_the_pixel_limit", "crates/filters/src/jbig2/symbols.rs::pages_in_one_font_share_one_dictionary", "crates/filters/src/jbig2/symbols.rs::what_a_page_has_alone_stays_its_own", "crates/filters/src/jbig2/symbols.rs::past_the_shared_limit_the_most_widely_held_marks_are_shared", "crates/optimize/tests/optimize.rs::jbig2_pages_share_one_symbol_dictionary", "crates/optimize/tests/optimize.rs::a_dictionary_that_saves_less_than_it_takes_isnt_shared", "crates/optimize/tests/optimize.rs::sixteen_images_at_most_share_a_dictionary", "crates/engine/src/tests.rs::redacting_an_optimized_jbig2_page_stores_the_rest_of_its_group_again", "crates/optimize/src/images.rs::only_what_redaction_can_store_again_shares_a_dictionary"]
+notes = "All of a run's images share one allowance of decoding, filters, decoders and pixels alike (about 1,000 scanned pages' worth, more for a longer file); those past it stay as they are, those the image pass read being counted in the report, which doc_optimize and doc_reduce return and the panel doesn't show yet. CCITT Group 4, lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (on by default, so Reduce File Size does it too; doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. Packed to one bit first, then stored the same way (colour-key masks aside): eight-bit grey of only black and white, eight-bit DeviceRGB of only black and white (as DeviceGray; calibrated RGB, RGB with a profile, RGB drawn or listed where /DefaultGray or /DefaultRGB is in force or referred to from anywhere but the page and form resources read (an annotation's appearance or a pattern drawing it, directly or through resources or a form shared with a page; another image's /Alternates), and RGB whose soft mask has a /Matte aside), and an indexed image of 2, 4 or 8 bits that uses only palette entries 0 and 1 (keeping its palette), unless the grey or colour settings make it smaller still, as they can a dithered scan downsampled to JPEG, its soft mask counted. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. Group 3 fax images are stored again as Group 4 when that is smaller (damaged Group 3 with end-of-line codes as the decoder recovers it, as it reports no damage there; byte-aligned Group 3 with end-of-line codes but none before the first row, not as T.4 sends them, as PdfCraft draws it, which at /K 0 may be other rows than were coded). CCITT fax images are decoded to be downsampled (optimize.downsample-images), and a resampled image is stored as Group 4 or 1-bit Flate, whichever is smaller. Lossless JBIG2 with bounded exact-copy symbol coding is available too, with image export, up to 16 images sharing one symbol dictionary (/JBIG2Globals) where that is smaller still (image masks and DeviceGray images named directly, of at most 64 Mi pixels, which redaction can store again without it); lossy JBIG2 remains unsupported. JBIG2 images are read too: averaged like any (under Retain stored as JBIG2 again), and, not resampled, stored again as the compression asks when that is smaller (as JBIG2 symbols where its own coding is a generic region, say); a symbol-coded scan, averaged, loses its exact repeats and is often larger, and then stays as it is; one whose JBIG2 page isn't its size is left alone. Not yet: images over 80 million pixels (left alone by the image steps, whatever their settings; Clean Up may still Flate an unfiltered or LZW one), inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples, JBIG2 past 65,535 samples on a side, where the decoder stops: such an image isn't stored as JBIG2 (the image step leaves a 1-bit one as it is unless it is averaged, and may store one packed from more bits as 1-bit Flate)."
 
 [[feature]]
 id = "optimize.only-if-smaller"