M11: tests for what Monochrome Images keeps, and for the fax bounds; Retain's wording

Mutation testing found behaviour no test held, now covered:
- a Group 4 halftone above 450 ppi isn't resampled and stays byte for
  byte, alone by default and behind ASCIIHexDecode with Retain;
- under Retain a resampled Flate scan stays Flate, though Group 4 would
  be smaller (ragged bands);
- sixteen-bit grey of only 0x0000 and 0xFFFF keeps its bytes;
- a busy Group 4 image behind ASCIIHexDecode, whose code is over three
  times its raster, decodes within the fax bound and is stored again as
  Group 4 alone. Redaction's busy-fax test is that busy again: since the
  bound for other images is three times the raster, the old checkerboard
  fitted it, and dropping the fax bound passed.
The redaction README says three times, and doc_optimize's description
says what Retain keeps: CCITT stays CCITT, anything else becomes Flate.

From reviews with the pdfcraft-review skill.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG

crates/automation/src/tools.rs

@@ -1433,7 +1433,7 @@ pub fn tools() -> Vec<ToolDef> {
             .with(schema(json!({ "doc": doc() }), &["doc"])),
         t("doc_open_revision", "Open a revision", "Open saved revision `revision` (1 = the oldest) of a document as a new, unsaved document, to see the file as it was then.")
             .with(schema(json!({ "doc": doc(), "revision": { "type": "integer", "minimum": 1 } }), &["doc", "revision"])),
-        t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). preset: standard (default; Reduce File Size's choices) or mobile (images to 96 ppi above 144, JPEG high quality); the options below change it. compatibility: retain (default; keeps the document's version, or 1.5 when object streams are written), 1.4, 1.5, 1.6, 1.7 or 2.0. merge_fragments (default true) first puts a page picture drawn as many stacked image strips (as some PDF printers write it) back together into one image, so it can be resampled (merged_images, fragments_merged). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). mono: { downsample, ppi, above_ppi, compression: ccitt_g4|retain } for black-and-white images and image masks (default: averaged down to 300 ppi above 450 ppi, counted in images_resampled; stored losslessly as CCITT Group 4 when that is smaller, counted in images_recompressed; retain resamples only, keeping each image's kind of compression). Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, print_settings; flate_unencoded (true); object_streams (default true; always off for 1.4); lzw_to_flate (true) decodes LZW streams and stores them with Flate instead, leaving any it cannot decode unchanged (lzw_streams counts the streams converted); remove_invalid_links and remove_unreferenced_dests (true); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. The open document is unchanged.")
+        t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). preset: standard (default; Reduce File Size's choices) or mobile (images to 96 ppi above 144, JPEG high quality); the options below change it. compatibility: retain (default; keeps the document's version, or 1.5 when object streams are written), 1.4, 1.5, 1.6, 1.7 or 2.0. merge_fragments (default true) first puts a page picture drawn as many stacked image strips (as some PDF printers write it) back together into one image, so it can be resampled (merged_images, fragments_merged). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). mono: { downsample, ppi, above_ppi, compression: ccitt_g4|retain } for black-and-white images and image masks (default: averaged down to 300 ppi above 450 ppi, counted in images_resampled; stored losslessly as CCITT Group 4 when that is smaller, counted in images_recompressed; retain resamples only, a CCITT image staying CCITT and any other becoming Flate). Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, print_settings; flate_unencoded (true); object_streams (default true; always off for 1.4); lzw_to_flate (true) decodes LZW streams and stores them with Flate instead, leaving any it cannot decode unchanged (lzw_streams counts the streams converted); remove_invalid_links and remove_unreferenced_dests (true); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. The open document is unchanged.")
             .cmd("optimize.advanced")
             .with(schema(
                 json!({

crates/optimize/tests/optimize.rs

@@ -1019,6 +1019,103 @@ fn a_fax_image_behind_a_decompression_bomb_is_left_alone() {
     }
 }
 
+/// A CCITT image `w` × `h` coded as Group 4 from `bits`, behind ASCIIHexDecode when `hex`.
+fn g4_image(doc: &mut Document, bits: &[u8], (w, h): (u32, u32), hex: bool) -> ObjRef {
+    let g4 = pdfcraft_filters::encode_g4(bits, w, h).unwrap();
+    let mut d = Dict::new();
+    d.set(b"Subtype".to_vec(), Object::name("Image"));
+    d.set(b"Width".to_vec(), Object::Int(w.into()));
+    d.set(b"Height".to_vec(), Object::Int(h.into()));
+    d.set(b"BitsPerComponent".to_vec(), Object::Int(1));
+    d.set(b"ColorSpace".to_vec(), Object::name("DeviceGray"));
+    let mut parms = Dict::new();
+    parms.set(b"K".to_vec(), Object::Int(-1));
+    parms.set(b"Columns".to_vec(), Object::Int(w.into()));
+    let data = if hex {
+        d.set(b"Filter".to_vec(), Object::Array(vec![Object::name("ASCIIHexDecode"), Object::name("CCITTFaxDecode")]));
+        d.set(b"DecodeParms".to_vec(), Object::Array(vec![Object::Null, Object::Dict(parms)]));
+        g4.iter().flat_map(|b| format!("{b:02x}").into_bytes()).chain(*b">").collect()
+    } else {
+        d.set(b"Filter".to_vec(), Object::name("CCITTFaxDecode"));
+        d.set(b"DecodeParms".to_vec(), Object::Dict(parms));
+        g4
+    };
+    doc.add(Object::Stream(Stream::from_raw(d, data)))
+}
+
+/// What Monochrome Images leaves alone and what it may still do. A Group 4 halftone above 450 ppi
+/// isn't resampled, so it stays byte for byte: alone, by default; behind ASCIIHexDecode, with
+/// Retain (by default it is stored again as Group 4 alone). Under Retain a resampled Flate scan
+/// stays Flate though Group 4 would be smaller. A busy Group 4 image behind ASCIIHexDecode, whose
+/// code is more than three times its raster, decodes within a fax code's bound and is stored
+/// again as Group 4 alone. Sixteen-bit grey of only 0x0000 and 0xFFFF isn't eight-bit black and
+/// white, and keeps its bytes.
+#[test]
+fn monochrome_images_keep_what_they_should() {
+    let (w, h) = (1200u32, 480u32);
+    let stride = (w as usize).div_ceil(8);
+    let checker: Vec<u8> = (0..h as usize).flat_map(|y| vec![if y % 2 == 0 { 0xAA } else { 0x55 }; stride]).collect();
+    let mut seed = 0x2545_F491_4F6C_DD1Du64;
+    let mut next = move || {
+        seed ^= seed << 13;
+        seed ^= seed >> 7;
+        seed ^= seed << 17;
+        seed
+    };
+    // Twelve dark bands a row, their edges ragged by up to five samples, as a scan's are.
+    let mut ragged = vec![0xFFu8; stride * h as usize];
+    for y in 0..h as usize {
+        for band in 0..12 {
+            let (x0, x1) = (band * 100 + (next() % 5) as usize, band * 100 + 50 + (next() % 5) as usize);
+            for x in x0..x1.min(w as usize) {
+                ragged[y * stride + x / 8] &= !(0x80 >> (x % 8));
+            }
+        }
+    }
+    let busy: Vec<u8> = (0..256).flat_map(|y| [if y % 2 == 0 { 0xAA } else { 0xFF }; 32]).collect();
+    let build = || {
+        let mut doc = Document::new_empty();
+        let alone = g4_image(&mut doc, &checker, (w, h), false);
+        let wrapped = g4_image(&mut doc, &checker, (w, h), true);
+        let grey = [("BitsPerComponent", 1.into()), ("ColorSpace", Object::name("DeviceGray"))];
+        let scan = mono_image(&mut doc, w, h, &ragged, None, &grey);
+        let fax = g4_image(&mut doc, &busy, (256, 256), true);
+        let mut d = Dict::new();
+        d.set(b"Subtype".to_vec(), Object::name("Image"));
+        d.set(b"Width".to_vec(), Object::Int(w.into()));
+        d.set(b"Height".to_vec(), Object::Int(h.into()));
+        d.set(b"BitsPerComponent".to_vec(), Object::Int(16));
+        d.set(b"ColorSpace".to_vec(), Object::name("DeviceGray"));
+        let samples: Vec<u8> = eight_bit(&bilevel(w, h, false), w, h).iter().flat_map(|v| [*v, *v]).collect();
+        let deep = doc.add(Object::Stream(Stream::flate(d, &samples)));
+        // 144 × 57.6 pt is 600 ppi; 46.08 pt for 256 samples is 400 ppi.
+        page(
+            &mut doc,
+            &[("A", alone), ("B", wrapped), ("C", scan), ("D", fax), ("E", deep)],
+            "q 144 0 0 57.6 20 700 cm /A Do Q q 144 0 0 57.6 200 700 cm /B Do Q q 144 0 0 57.6 20 600 cm /C Do Q \
+             q 46.08 0 0 46.08 200 600 cm /D Do Q q 144 0 0 57.6 20 500 cm /E Do Q",
+        );
+        (doc, [alone, wrapped, scan, fax, deep])
+    };
+    let raw = |doc: &Document, r: ObjRef| stream(doc, r).raw;
+    let filters = |doc: &Document, r: ObjRef| stream(doc, r).filters().len();
+
+    let (mut doc, [alone, wrapped, scan, fax, deep]) = build();
+    let before = [alone, deep].map(|r| raw(&doc, r));
+    optimize(&mut doc, &Settings::default()).unwrap();
+    assert_eq!([alone, deep].map(|r| raw(&doc, r)), before, "a Group 4 halftone alone, and sixteen-bit grey");
+    assert_eq!(filters(&doc, wrapped), 1, "a wrapped Group 4 halftone is stored again alone by default");
+    assert_eq!(stream(&doc, scan).dict.name(b"Filter"), Some(&b"CCITTFaxDecode"[..]), "the scan: Group 4 is smaller");
+    assert_eq!((stream(&doc, fax).dict.name(b"Filter"), filters(&doc, fax)), (Some(&b"CCITTFaxDecode"[..]), 1), "the busy fax image");
+
+    let (mut doc, [_, wrapped, scan, _, _]) = build();
+    let before = raw(&doc, wrapped);
+    let r = optimize(&mut doc, &Settings { mono: RETAIN, ..Settings::default() }).unwrap();
+    assert_eq!(raw(&doc, wrapped), before, "Retain: a wrapped Group 4 halftone stays as it is");
+    let s = stream(&doc, scan);
+    assert_eq!((s.dict.int(b"Width"), s.dict.name(b"Filter")), (Some(600), Some(&b"FlateDecode"[..])), "Retain: Flate stays Flate: {r:?}");
+}
+
 #[test]
 fn settings_choose_what_happens() {
     let mut doc = Document::new_empty();

crates/redact/README.md

@@ -22,8 +22,9 @@ clear_marks(&mut doc, None)?;                        // remove marks without app
   filter, decoded through `Document::decoded_image_within`; `/DecodeParms` given by reference,
   numbers inside included, count). Codecs redaction doesn't decode (DCT, JPX, JBIG2) are removed
   whole (fail-closed); inline images under an area are removed. Images with separate `/SMask`
-  or `/Mask` data, and images whose data decodes to more than twice their declared raster (plus
-  a byte per row and 4 KiB; for a fax image's code, about a byte per pixel), are removed whole;
+  or `/Mask` data, and images whose data decodes to more than three times their declared raster
+  and a byte per row, plus 4 KiB (ASCIIHex text broken into lines under Flate is a little over
+  twice; for a fax image's code, about a byte per pixel), are removed whole;
   cleared copies drop `/Alternates` so alternate pixels cannot survive;
 - **vectors**: covered paths are removed; partly covered paths and shadings are clipped so
   nothing paints inside the areas;

crates/redact/src/tests.rs

@@ -684,15 +684,16 @@ fn a_fax_image_that_cannot_be_decoded_in_full_is_removed_whole() {
     }
 }
 
-/// A busy fax image is cleared, though its code is bigger than twice its raster (the bound for
-/// other images): a code takes up to about a byte per pixel, and the fax image gets that much.
+/// A busy fax image is cleared, though its code is bigger than three times its raster (the bound
+/// for other images): a code takes up to about a byte per pixel, and the fax image gets that much.
 #[test]
 fn a_busy_fax_image_behind_another_filter_is_cleared() {
-    // A 256 × 256 checkerboard, the worst case for a fax code.
-    let rows: Vec<u8> = (0..256).flat_map(|y| [if y % 2 == 0 { 0xAA } else { 0x55 }; 32]).collect();
+    // 256 × 256, rows of a one-pixel checkerboard between white ones: each checkered row is coded
+    // afresh against the white one above it, about four bits a pixel.
+    let rows: Vec<u8> = (0..256).flat_map(|y| [if y % 2 == 0 { 0xAA } else { 0xFF }; 32]).collect();
     let g4 = pdfcraft_filters::encode_g4(&rows, 256, 256).unwrap();
     let raster = 32 * 256;
-    assert!(g4.len() > (raster + 256) * 2 + 4096, "{} bytes: within the bound for other images", g4.len());
+    assert!(g4.len() > (raster + 256) * 3 + 4096, "{} bytes: within the bound for other images", g4.len());
     let dict = "/Type /XObject /Subtype /Image /Width 256 /Height 256 /ColorSpace /DeviceGray /BitsPerComponent 1 \
                 /Filter [/FlateDecode /CCITTFaxDecode] /DecodeParms [null << /K -1 /Columns 256 /Rows 256 >>]";
     let mut doc = one_page(b"q 256 0 0 256 0 0 cm /Im Do Q", "/XObject << /Im 7 0 R >>", vec![stream(dict, pdfcraft_filters::encode_flate(&g4))]);