M11: JBIG2 images share a symbol dictionary, up to 16 at a time (optimize.mono-compression) (#5910)

* M11: JBIG2 images share a symbol dictionary, up to 16 at a time

With Monochrome Images set to JBIG2, Optimize stored each image's
letters in a symbol dictionary of its own, so a document of N pages
carried its font's shapes N times. Now up to 16 images the run stores
as JBIG2 share one dictionary in a /JBIG2Globals stream, holding the
marks found in more than one of them; each image's text regions refer
to it first, then to a dictionary of its own for the marks only it
repeats. An image is stored that way when it comes out smaller, and a
group only when what its images save is more than the globals take.

- filters: encode_jbig2_shared codes a group: the globals are a symbol
  dictionary segment numbered 0 and associated with no page (7.2.6);
  each page numbers its segments from 1, as a segment refers only to
  lower numbers (7.2.5), and the shared symbols' IDs come first, as the
  dictionaries are referred to in that order (Table 31, SBSYMS). At
  most 16,383 shared marks, those in the most images, so a page keeps
  room for its own under the decoder's 32,767.
- optimize: the run holds the 1-bit samples of the images it stores as
  JBIG2 (16 at most, 64 MiB at most) and stores them again sharing,
  weighing each with the longest reference the globals could have.

Eight pages of 10 pt text rendered at 300 dpi take 41% less in
Helvetica and 42% in Times than coded alone, 10% in Courier, and 46%
in Helvetica at 600 dpi; groups of 2, 4, 16 and 32 save 19%, 33%, 45%
and 47%. Redacting one image stores the others of its group again
without the dictionary, so 16 keeps that to 15 images.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* M11: test what the mutants found untested in shared JBIG2 dictionaries

- filters: the group goes through encode_jbig2_shared, which refuses it
  for an image the encoder refuses; two marks in common are enough to
  share; past the 65,536 distinct marks a group holds, a mark first seen
  later isn't shared while those held count every image.
- optimize: optimizing again keeps both the shared pages and the pages
  coded alone as they are; under Retain, averaged JBIG2 pages share
  nothing; a last group of two shares.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* M11: only images redaction can store again share a JBIG2 dictionary

Redacting one image of a group stores the others again without the
dictionary, which can hold the shapes of marks under the area, and
refuses the redaction for one it can't: it takes image masks and
images of one component as its own reading counts them (no DeviceN,
the family named directly), and at most 64 Mi pixels. Optimize took
DeviceN of one colourant and up to 80 million pixels, so such an image
in a group made any redaction of the others fail, where before it was
coded alone and the same redaction worked (the Consumers review).

Only an image mask or a DeviceGray image named directly, of at most
2^26 pixels, now joins a group; others are coded alone. An engine test
optimizes three pages, two DeviceGray and one DeviceN, and redacts the
first: the DeviceN page stays alone, the second is stored again, and
no image names the dictionary after a save.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* M11: optimize and redaction share one limit on the images redaction stores again

The second Consumers review: Optimize's limit on images that share a
JBIG2 dictionary copied redaction's private 64 Mi, with nothing tying
the two, so a lower one in redaction would bring back refused
redactions with no test going red. Both now read
pdfcraft_cos::MAX_REDACTED_IMAGE_PIXELS. The engine test adds an image
mask, the other kind that shares, so a redaction that stopped storing
either kind again fails it. doc_optimize's description, the README,
the parity note and ROADMAP say which images share (named directly).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* M11: a JBIG2 group is coded before its samples would pass the limit

CodeRabbit on #5910: hold added an image to the group and then checked
SHARED_BYTES, so a group could pass the 64 MiB it is documented to stay
within by its last image, which can itself be up to 64 MiB (an image
that shares has at most 2^26 pixels, and as its rows are whole bytes,
at most that many bytes; a 1-pixel-wide one takes them all). hold_in
codes the group first when an image would take it past the limit, and
after when it has 16 images; with room for 10 bytes, images of 4 go two
to a group, where the old order made one of 12. CodeRabbit's follow-up
too: the 18-image test decodes every image and compares it with its
samples, across the boundary between the groups.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* optimize: count the globals' object framing; filters: test the shared coder's size limit

- A JBIG2 group shares a dictionary only when it saves more than the
  globals' dictionary and data plus NEW_OBJECT (96 bytes: the object's
  framing, its xref entry, and an encrypted file's IV and padding), so a
  group that saves less than that no longer makes the file larger.
- encode_jbig2_shared is Unsupported for an image taller than the
  decoder takes and codes one at the bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filters: drop the unused rows binding in encode_shared

page() no longer takes the rows since #5511.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filters: the shared coder's at-bound test says what it returns

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
This commit is contained in:
authored and GitHub committed 2026-11-04 13:34:08 +01:00
1 parent e989e48e63
commit d579f26de3
15 files changed
+936 -75

No files matched your search

+1
View File
@@ -304,6 +304,7 @@ Estimates unchanged.
| Date | Change | Summary |
|---|---|---|
| 2026-11-03 | minor | M11: Optimize's JBIG2 images share a symbol dictionary (`/JBIG2Globals`), up to 16 at a time and when smaller (`pdfcraft_filters::encode_jbig2_shared`): 41–46% smaller on rendered text in proportional fonts, 10% in Courier |
| 2026-11-03 | patch | M5 #5850 review: indirect typewriter intent and composer edits preserve outer whitespace; movement regression commits by outside click; estimates unchanged. |
| 2026-11-03 | patch | M5 #5850: Enter adds Fill & Sign line breaks; outside clicks commit, Escape cancels, and saved multiline appearances retain their text and height; estimates unchanged. |
| 2026-11-02 | patch | M10 #5743: macOS print jobs carry the job name and owner some vendors' filters need; not yet checked on a physical printer |