fix(optimize): make pictures under a stencil mask smaller (#5847)

* fix(optimize): make pictures under a stencil mask smaller

An image with any /Mask was skipped by the colour and grey settings. That is
right for a colour-key mask, which names sample values that resampling and JPEG
would change. A stencil mask is an image of its own, with its own resolution
(ISO 32000-2 8.9.6.3), so the picture can change under it and the mask stays as
it is. Some exporters put one on every picture, even where it hides nothing, so
whole drawing sets of pictures were left at full size.

A mask that can't be read as a stencil, such as a reference to nothing, still
leaves its picture alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(optimize): only a mask that says /ImageMask true counts as a stencil

A stream /Mask without /ImageMask true let its picture be replaced, although a
mask that isn't a stencil is meant to leave its picture alone. It now does.

The stencil test also checks that the hole still shows what's behind it. The first
pixel all round is left out, because the coarser picture blends into it a little
(at most 26 of 255 in this test).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(optimize): draw a coloured page behind the stencil test's picture

On a white page, a hole painted white looks the same as an open hole, so the check
on the hole couldn't tell them apart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: quincy-jc <184324679+quincy-jc@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

crates/optimize/README.md

@@ -28,9 +28,10 @@ let report = optimize_with_progress(&mut doc, &settings, &mut |stage| keep_going
   If a use can't be
   measured (forms nested deeper, or drawn too many times), no image is resampled. Colour and grayscale images above a
   threshold are resampled with a bicubic (Catmull-Rom) filter to a target resolution and
-  stored as JPEG (Acrobat's quality steps) or ZIP; soft masks are resampled with their image.
+  stored as JPEG (Acrobat's quality steps) or ZIP; soft masks are resampled with their image. A
+  stencil `/Mask` has its own resolution, so the picture changes under it and it stays as it is.
   An image is replaced only if the result is smaller. Left alone by these colour and grey
-  settings: CMYK and other colour spaces, image masks, decode arrays, more than 8 bits, JPEG 2000,
+  settings: CMYK and other colour spaces, image masks, colour-key masks, decode arrays, more than 8 bits, JPEG 2000,
   JBIG2 and CCITT (black-and-white images, CCITT and JBIG2 among them, are below). A JPEG is read with
   its colour transform (an Adobe marker's, else `/ColorTransform`, ISO 32000-2 §7.4.8), so one
   stored untransformed keeps its colours; one whose marker and `/ColorTransform` disagree is left

crates/optimize/src/images.rs

@@ -717,6 +717,11 @@ fn image_stream(old: &Dict, px: &[u8], n: usize, w: u32, h: u32, compression: Co
     }
 }
 
+/// Whether a `/Mask` value is a stencil mask: an image that says `/ImageMask true`.
+fn is_stencil(doc: &Document, mask: &Object) -> bool {
+    matches!(&*doc.resolve(mask), Object::Stream(s) if matches!(s.dict.get(b"ImageMask"), Some(Object::Bool(true))))
+}
+
 /// One image: resample and/or recompress per `settings`; `ppi` is its effective resolution.
 /// Returns the new image and its new soft mask, if the result is smaller.
 fn process(
@@ -728,8 +733,11 @@ fn process(
     budget: &mut ImageBudget,
 ) -> Option<(Stream, Option<(ObjRef, Stream)>)> {
     let d = &s.dict;
+    // A colour-key mask names sample values, which resampling and JPEG would change. A stencil mask
+    // is an image of its own, with its own resolution (ISO 32000-2 §8.9.6.3): the picture can change
+    // under it and its edges stay as they are.
     if d.get(b"Decode").is_some_and(|a| !is_identity_decode(doc, a, n))
-        || d.contains(b"Mask")
+        || d.get(b"Mask").is_some_and(|m| !is_stencil(doc, m))
         || d.name(b"ImageMask").is_some()
         || matches!(d.get(b"ImageMask"), Some(Object::Bool(true)))
     {

crates/optimize/tests/optimize.rs

@@ -3788,3 +3788,102 @@ fn mixed_text_and_small_halftone_regions_keep_their_tones_with_wide_white_margin
         assert_eq!(render(&saved), before, "local halftone {side}x{side}");
     }
 }
+
+/// A one-bit stencil of `w` × `h`, solid but for a hole in its middle.
+fn stencil(doc: &mut Document, w: u32, h: u32) -> ObjRef {
+    let row = (w as usize).div_ceil(8);
+    let mut bits = vec![0u8; row * h as usize];
+    for y in (h as usize / 3)..(2 * h as usize / 3) {
+        for x in (w as usize / 3)..(2 * w as usize / 3) {
+            bits[y * row + x / 8] |= 0x80 >> (x % 8);
+        }
+    }
+    let mut d = Dict::new();
+    d.set(b"Type".to_vec(), Object::name("XObject"));
+    d.set(b"Subtype".to_vec(), Object::name("Image"));
+    d.set(b"ImageMask".to_vec(), Object::Bool(true));
+    d.set(b"Width".to_vec(), Object::Int(i64::from(w)));
+    d.set(b"Height".to_vec(), Object::Int(i64::from(h)));
+    d.set(b"BitsPerComponent".to_vec(), Object::Int(1));
+    doc.add(Object::Stream(Stream::flate(d, &bits)))
+}
+
+/// A picture with a stencil mask is made smaller like any other: the mask has a resolution of its
+/// own, so it stays exactly as it was and the hole it cuts stays where it was.
+#[test]
+fn a_picture_with_a_stencil_mask_is_downsampled_under_its_mask() {
+    let mut doc = Document::new_empty();
+    let mask = stencil(&mut doc, 1200, 1200);
+    let img = image(&mut doc, 1200, 1200, 3, true, None);
+    let mut s = stream(&doc, img);
+    s.dict.set(b"Mask".to_vec(), Object::Ref(mask));
+    doc.set(img, Object::Stream(s));
+    let mask_before = stream(&doc, mask);
+    // 1200 pixels over two inches: 600 ppi.
+    page(&mut doc, &[("Im0", img)], "0 0 1 rg 0 0 612 792 re f q 144 0 0 144 100 100 cm /Im0 Do Q");
+    let before = render(&doc);
+    let report = optimize(&mut doc, &Settings::default()).unwrap();
+    assert_eq!(report.images_resampled, 1, "{report:?}");
+    let after = stream(&doc, img);
+    assert_eq!(after.dict.int(b"Width"), Some(300), "150 ppi");
+    assert_eq!(after.dict.reference(b"Mask"), Some(mask), "the mask stays on it");
+    assert_eq!(stream(&doc, mask), mask_before, "and doesn't change");
+    let rendered = render(&doc);
+    assert_eq!(rendered.len(), before.len());
+    let changed = before.iter().zip(&rendered).filter(|(a, b)| a.abs_diff(**b) > 96).count();
+    assert!(changed * 200 < before.len(), "the masked picture looks the same: {changed} of {} samples far off", before.len());
+    // The hole, the middle third of the picture on the 612 by 792 page, still shows what's behind it.
+    // Its first pixel all round is left out: the coarser picture blends into that a little.
+    for y in 597..643 {
+        for x in 149..195 {
+            let i = (y * 612 + x) * 4;
+            assert_eq!(rendered[i..i + 4], before[i..i + 4], "hole pixel ({x}, {y})");
+        }
+    }
+}
+
+/// A colour-key mask names sample values: resampling or JPEG would make holes where there were none
+/// and fill those there were, so the picture stays as it is, however the array is written. A mask
+/// that can't be read as a stencil is no reason to change the picture either.
+#[test]
+fn a_picture_whose_mask_is_not_a_stencil_is_left_alone() {
+    type Mask = fn(&mut Document) -> Object;
+    fn key() -> Object {
+        Object::Array([250, 255, 250, 255, 250, 255].map(Object::Int).to_vec())
+    }
+    fn not_a_stencil(doc: &mut Document, change: fn(&mut Dict)) -> Object {
+        let mask = stencil(doc, 1200, 1200);
+        let mut s = stream(doc, mask);
+        change(&mut s.dict);
+        doc.set(mask, Object::Stream(s));
+        Object::Ref(mask)
+    }
+    let masks: [(&str, Mask); 5] = [
+        ("direct array", |_| key()),
+        ("array by reference", |doc| Object::Ref(doc.add(key()))),
+        ("nothing there", |doc| Object::Ref(doc.add(Object::Null))),
+        ("an image that says it isn't a stencil", |doc| {
+            not_a_stencil(doc, |d| {
+                d.set(b"ImageMask".to_vec(), Object::Bool(false));
+            })
+        }),
+        ("an image that doesn't say it is one", |doc| {
+            not_a_stencil(doc, |d| {
+                d.remove(b"ImageMask");
+            })
+        }),
+    ];
+    for (what, mask) in masks {
+        let mut doc = Document::new_empty();
+        let img = image(&mut doc, 1200, 1200, 3, true, None);
+        let mut s = stream(&doc, img);
+        let mask = mask(&mut doc);
+        s.dict.set(b"Mask".to_vec(), mask);
+        doc.set(img, Object::Stream(s));
+        let before = stream(&doc, img);
+        page(&mut doc, &[("Im0", img)], "q 144 0 0 144 100 100 cm /Im0 Do Q");
+        let report = optimize(&mut doc, &Settings::default()).unwrap();
+        assert_eq!(report.images_resampled + report.images_recompressed, 0, "{what}: {report:?}");
+        assert_eq!(stream(&doc, img), before, "{what}");
+    }
+}

parity/acrobat-features.toml

@@ -7373,8 +7373,8 @@ milestone = "M11"
 status = "shipped"
 commands = ["optimize.reduce"]
 tools = ["doc_reduce"]
-evidence = ["crates/ui-egui/tests/it/create.rs::reduce_file_size_writes_a_compact_copy", "crates/ui-egui/tests/it/create.rs::reduce_file_size_saves_nothing_when_the_file_is_already_small", "crates/engine/src/tests.rs::create_and_reduce", "crates/automation/tests/it/automation.rs::creating_and_reducing_through_tools", "crates/automation/tests/it/automation.rs::optimizing_through_tools", "crates/optimize/tests/optimize.rs::an_identity_decode_array_does_not_stop_reduce"]
-notes = "images above 225 ppi to 150 ppi JPEG medium (after merging page pictures printed as one-pixel strips into one image), black-and-white images averaged down to 300 ppi above 450 ppi, and stored as CCITT Group 4 whenever that is smaller, thumbnails dropped, resources merged, object streams; scanner images with the default decode array are included; a copy that isn't smaller is not saved (\"already as small as it can be made\"); runs in the background with a progress bar and Cancel"
+evidence = ["crates/ui-egui/tests/it/create.rs::reduce_file_size_writes_a_compact_copy", "crates/optimize/tests/optimize.rs::a_picture_with_a_stencil_mask_is_downsampled_under_its_mask", "crates/optimize/tests/optimize.rs::a_picture_whose_mask_is_not_a_stencil_is_left_alone", "crates/ui-egui/tests/it/create.rs::reduce_file_size_saves_nothing_when_the_file_is_already_small", "crates/engine/src/tests.rs::create_and_reduce", "crates/automation/tests/it/automation.rs::creating_and_reducing_through_tools", "crates/automation/tests/it/automation.rs::optimizing_through_tools", "crates/optimize/tests/optimize.rs::an_identity_decode_array_does_not_stop_reduce"]
+notes = "images above 225 ppi to 150 ppi JPEG medium (after merging page pictures printed as one-pixel strips into one image), black-and-white images averaged down to 300 ppi above 450 ppi, and stored as CCITT Group 4 whenever that is smaller, thumbnails dropped, resources merged, object streams; scanner images with the default decode array are included; pictures under a stencil mask are included and the mask stays as it is, though samples it hides still count when resampling, so their colour can reach its edge; pictures with a colour-key mask are left alone; a copy that isn't smaller is not saved (\"already as small as it can be made\"); runs in the background with a progress bar and Cancel"
 
 [[feature]]
 id = "optimize.compatibility-level"