mirror of
https://github.com/storytold/pdfcraft.git
synced 2026-10-31 23:59:07 +00:00
M11: JBIG2 symbol coding: at most 10,000 copies a text region and 32,767 symbols
A review found that a dense page came out as a stream PdfCraft's own renderer refuses, so the page drew blank after optimizing: hayro-jbig2 decodes a text region of at most 10,000 symbol instances, and refuses 32,768 symbols or more. The stream was valid (Poppler decoded it), but every mark went into one text region and up to 65,536 symbols were kept. A 300 dpi letter page of the optimizer test's own text crossed it. A page with more than 10,000 marks is now placed by several text regions, each over a band of rows and all referring to the one symbol dictionary, which each region but the last keeps retained (7.2.4). Past 32,767 distinct symbols the image is coded generically. New tests: 25,000 copies placed by three regions, which decode; the symbol caps decoded, not just reached; and in the optimizer, a whole letter page of rendered text that looks the same as JBIG2 symbols (it drew blank before). The READMEs say "still looked at" for the caps, and that the scans figure is the six pages' total (20% on one page). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
This commit is contained in:
1 parent
8eeafe3309
commit
deb70c92ca
6 files changed
+132
-54
No files matched your search
@@ -7385,7 +7385,7 @@ status = "partial"
|
||||
commands = ["optimize.advanced", "optimize.reduce"]
|
||||
tools = ["doc_optimize", "doc_reduce"]
|
||||
evidence = ["crates/filters/src/ccitt.rs::bitmaps_round_trip", "crates/filters/src/ccitt.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/filters/src/ccitt.rs::what_it_cannot_encode_is_an_error", "crates/filters/src/mq.rs::the_annex_h_test_sequence_encodes_to_its_bytes", "crates/filters/src/jbig2.rs::bitmaps_round_trip", "crates/filters/src/jbig2.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/filters/src/jbig2/symbols.rs::repeated_marks_are_stored_once", "crates/filters/src/jbig2/symbols.rs::scattered_marks_round_trip", "crates/optimize/tests/optimize.rs::black_and_white_images_become_jbig2", "crates/optimize/tests/optimize.rs::rendered_text_becomes_jbig2_symbols", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_jbig2", "crates/optimize/tests/optimize.rs::black_and_white_images_become_ccitt_group_4", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_ccitt_group_4", "crates/ui-egui/tests/it/create.rs::the_pdf_optimizer_dialog_saves_an_optimized_copy"]
|
||||
notes = "CCITT Group 4 (the default, so Reduce File Size does it too) or JBIG2, both lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. JBIG2 stores each distinct mark once in a symbol dictionary and places its exact copies with a text region, larger and lone marks in a generic region (template 0, typical prediction), or the whole image as a generic region, whichever is smaller: on rendered 300 dpi text 28% of the generic region's size and 17% of Group 4's; on scans, whose copies of a letter are seldom identical, little: on the same pages with 2% or more of the letters' edge pixels flipped at random, at most 8% below the generic region, which was 27–33% below Group 4. A document below PDF 1.4 gets /Version 1.4. Not yet: JBIG2 refinement coding of near-copies (where lossless gains on scans would come from), a symbol dictionary shared between images (/JBIG2Globals), JBIG2's lossy mode, monochrome downsampling, inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples (JBIG2 takes rows of up to 1,048,576). Redaction doesn't decode JBIG2 or Group 4 yet: a mark over part of such an image removes the whole image (nothing leaks). Export reads JBIG2, but nothing that exports an image reads Group 4 yet: Export ▸ Images and Save Image As refuse it, Copy Image fails, and Word, PowerPoint and HTML export leave the picture out."
|
||||
notes = "CCITT Group 4 (the default, so Reduce File Size does it too) or JBIG2, both lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. JBIG2 stores each distinct mark once in a symbol dictionary and places its exact copies with text regions (at most 10,000 copies each), larger and lone marks in a generic region (template 0, typical prediction), or the whole image as a generic region, whichever is smaller: on rendered 300 dpi text 28% of the generic region's size and 17% of Group 4's; on scans, whose copies of a letter are seldom identical, little: on the same pages with 2% or more of the letters' edge pixels flipped at random, at most 8% below the generic region in total (20% on one page), and the generic region was 27–33% below Group 4. A document below PDF 1.4 gets /Version 1.4. Not yet: JBIG2 refinement coding of near-copies (where lossless gains on scans would come from), a symbol dictionary shared between images (/JBIG2Globals), JBIG2's lossy mode, monochrome downsampling, inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples (JBIG2 takes rows of up to 1,048,576). Redaction doesn't decode JBIG2 or Group 4 yet: a mark over part of such an image removes the whole image (nothing leaks). Export reads JBIG2, but nothing that exports an image reads Group 4 yet: Export ▸ Images and Save Image As refuse it, Copy Image fails, and Word, PowerPoint and HTML export leave the picture out."
|
||||
|
||||
[[feature]]
|
||||
id = "optimize.only-if-smaller"
|
||||
|
||||
Reference in new issue
Block a user