M2.6: a run pays for what a stream costs, not what it holds

Four ways round the allowance, each measured by a review of hostile input:

A Flate stream read as zlib that fails at once, and then as raw deflate that
decodes everything before failing, gave nothing back and so was charged
nothing, here and in DecodeBudget: 110 s for a 387 KB file, all decoded. What
the retry decoded now counts, though the data is still dropped as it was.

Fax and 1-bit Flate images paid by the byte for work done by the pixel: 19
bytes can be a blank page of 100 million pixels, about a second each. Every
image decoded here now pays for its pixels first, as JBIG2 already did.

A JPEG paid nothing, and what one asks for doesn't grow with its length: 156
bytes declared 80 megapixels, 72 s for an 8 KB file. A JPEG pays for its
samples before it is decoded, in Optimize and in an export that has to decode
one for its mask.

Merging fragments paid for reading strips and not for the picture it built:
393 s for 1,154 bytes. It pays for the picture before allocating it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

crates/cos/README.md

@@ -94,6 +94,6 @@ recognisable in a diff and byte-stable when re-serialized.
   (`Document::decoded_image`, a JBIG2 image's `/JBIG2Globals` within `MAX_JBIG2_GLOBALS`, 32 MiB,
   their filters undone once a document for the images that share them, up to 64 MiB kept, while
   their segments are decoded for every image, and `ImageBudget` for an operation over many images,
-  which share one allowance of decoding, filters and fax and JBIG2 decoders alike, and are refused
-  past it);
+  which share one allowance of decoding, filters, decoders and pixels alike, and are refused past
+  it);
   DCT and JPX are still decoded through the bootstrap renderer (`core.image-filters`, M2.6).

crates/cos/src/budget.rs

@@ -25,6 +25,10 @@ pub const MAX_PAGE_DECODED: usize = 256 << 20;
 /// Form XObjects one page's content walk may enter, each entry counted again.
 pub const MAX_PAGE_FORMS: usize = 50_000;
 
+/// The steps a sample of a JPEG costs: a few bytes can hold a page of one grey, and decoding,
+/// resampling and storing it took about 20 to 45 ns a sample.
+const JPEG_WORK: u64 = 2;
+
 /// A stream decode within a byte bound, with the bytes its filter stages decoded: the data or
 /// the error, each with that count.
 type Decoder = fn(&Stream, usize) -> Result<(Vec<u8>, usize), (FilterError, usize)>;
@@ -39,8 +43,10 @@ type Decoder = fn(&Stream, usize) -> Result<(Vec<u8>, usize), (FilterError, usiz
 /// aren't decoded. A caller then treats them as it does an image it can't read, and
 /// [`ImageBudget::refused`] says how many there were.
 ///
-/// JPEG and JPEG 2000 data aren't charged: their decoders don't run here, and what one asks for
-/// grows with its length, as the allowance does.
+/// An image also pays for its pixels, as what reads it goes through them one by one however few
+/// bytes hold them: blank rows of fax or Flate take almost none. A JPEG pays for its samples when
+/// it is decoded ([`ImageBudget::pay_jpeg`]), and is otherwise not charged, as JPEG 2000 isn't:
+/// their decoders don't run here.
 #[derive(Debug)]
 pub struct ImageBudget {
     work: u64,
@@ -61,6 +67,7 @@ impl ImageBudget {
 
     /// Image `s` decoded like [`Document::decoded_image_within`], charged to the allowance.
     pub fn decode(&mut self, doc: &Document, s: &Stream, max: usize) -> Result<Vec<u8>, CosError> {
+        self.pay_size(s)?;
         let decoded = doc.decoded_image_spending(s, max, &mut self.work);
         self.count_refused(s, &decoded);
         decoded
@@ -70,6 +77,7 @@ impl ImageBudget {
     /// with its `/DecodeParms` given by reference read too ([`Document::with_direct_parms`]), for an
     /// image that has no fax or JBIG2 decoder to go through.
     pub fn decode_filters(&mut self, doc: &Document, s: &Stream, max: usize) -> Result<Vec<u8>, CosError> {
+        self.pay_size(s)?;
         let decoded = doc.with_direct_parms(s).decoded_spending(max, &mut self.work);
         self.count_refused(s, &decoded);
         decoded
@@ -93,13 +101,33 @@ impl ImageBudget {
         left.is_some()
     }
 
+    /// Pays for `pixels` a decode goes on to work through, or spends all there is, and the image
+    /// is refused.
+    pub fn pay_pixels(&mut self, pixels: u64) -> bool {
+        let paid = self.spend(pixels);
+        self.refused += usize::from(!paid);
+        paid
+    }
+
+    /// Pays for decoding a JPEG of `samples`, as [`ImageBudget::pay_pixels`] does.
+    pub fn pay_jpeg(&mut self, samples: u64) -> bool {
+        self.pay_pixels(samples.saturating_mul(JPEG_WORK))
+    }
+
+    /// Pays for the pixels of image `s`, unless it is JBIG2, whose decoder pays for its page, or
+    /// isn't decoded here.
+    fn pay_size(&mut self, s: &Stream) -> Result<(), CosError> {
+        if !charged(s) || matches!(s.filters().last(), Some((pdfcraft_filters::Filter::Jbig2, _))) {
+            return Ok(());
+        }
+        let side = |key: &[u8]| s.dict.int(key).and_then(|v| u64::try_from(v).ok()).unwrap_or(0);
+        if self.pay_pixels(side(b"Width").saturating_mul(side(b"Height"))) { Ok(()) } else { Err(CosError::Filter(FilterError::Spent.to_string())) }
+    }
+
     /// An image that failed with nothing left was refused, unless it has a filter that costs
     /// nothing here, which fails whatever is left.
     fn count_refused(&mut self, s: &Stream, decoded: &Result<Vec<u8>, CosError>) {
-        use pdfcraft_filters::Filter;
-        let charged =
-            s.readable_filters().is_ok_and(|chain| chain.iter().all(|(f, _)| f.is_decodable() || matches!(f, Filter::Jbig2 | Filter::CcittFax)));
-        if decoded.is_err() && self.work == 0 && charged {
+        if decoded.is_err() && self.work == 0 && charged(s) {
             self.refused += 1;
         }
     }
@@ -115,6 +143,12 @@ impl ImageBudget {
     }
 }
 
+/// Whether `s` takes only filters and decoders this charges.
+fn charged(s: &Stream) -> bool {
+    use pdfcraft_filters::Filter;
+    s.readable_filters().is_ok_and(|chain| chain.iter().all(|(f, _)| f.is_decodable() || matches!(f, Filter::Jbig2 | Filter::CcittFax)))
+}
+
 /// What one operation may still decode and enter across all the streams it reads.
 ///
 /// [`MAX_DECODED`](crate::MAX_DECODED) bounds one decode. It does not bound a page whose
@@ -418,7 +452,8 @@ mod image_tests {
         dict.set(b"Filter".to_vec(), Object::Array(vec![Object::name("FlateDecode"), Object::name("CCITTFaxDecode")]));
         dict.set(b"DecodeParms".to_vec(), Object::Array(vec![Object::Null, Object::Dict(parms)]));
         let image = Stream::from_raw(dict, pdfcraft_filters::encode_flate(&coded));
-        let cost = (image.raw.len() + coded.len() + 4 * coded.len() + 512 * h as usize) as u64;
+        let pixels = u64::from(w) * u64::from(h);
+        let cost = (image.raw.len() + coded.len() + 4 * coded.len() + 512 * h as usize) as u64 + pixels;
         let mut budget = ImageBudget::with_work(cost);
         assert_eq!(budget.decode(&doc, &image, 1 << 20).unwrap().len(), 512 * h as usize);
         assert_eq!(budget.work_left(), 0, "exactly that");
@@ -426,6 +461,60 @@ mod image_tests {
         assert_eq!(budget.refused(), 1);
     }
 
+    /// An image pays for its pixels however few bytes hold them, or a blank page of fax or Flate
+    /// would cost a run a second for a few bytes' worth of allowance; a JBIG2 image doesn't pay
+    /// twice, and one this doesn't decode pays nothing. Past what is left it isn't read.
+    #[test]
+    fn an_image_pays_for_its_pixels() {
+        let doc = Document::new_empty();
+        let blank = |filter: Option<&str>, data: Vec<u8>| {
+            let mut dict = Dict::new();
+            dict.set(b"Width".to_vec(), Object::Int(4096));
+            dict.set(b"Height".to_vec(), Object::Int(1024));
+            if let Some(filter) = filter {
+                dict.set(b"Filter".to_vec(), Object::name(filter));
+            }
+            Stream::from_raw(dict, data)
+        };
+        let rows = vec![0xFFu8; 512 * 1024];
+        let flate = blank(Some("FlateDecode"), pdfcraft_filters::encode_flate(&rows));
+        let cost = (flate.raw.len() + rows.len()) as u64 + 4096 * 1024;
+        let mut budget = ImageBudget::with_work(cost);
+        assert_eq!(budget.decode(&doc, &flate, 1 << 24).unwrap(), rows);
+        assert_eq!((budget.work_left(), budget.refused()), (0, 0), "exactly that");
+        // Its filters alone pay the same.
+        let mut budget = ImageBudget::with_work(cost);
+        assert_eq!(budget.decode_filters(&doc, &flate, 1 << 24).unwrap(), rows);
+        assert_eq!(budget.work_left(), 0);
+        // Short of its pixels it isn't read: this much would have paid for the filters.
+        let mut budget = ImageBudget::with_work(4096 * 1024 - 1);
+        assert!(budget.decode(&doc, &flate, 1 << 24).unwrap_err().to_string().contains("more image decoding than"));
+        assert_eq!((budget.work_left(), budget.refused()), (0, 1));
+        let mut budget = ImageBudget::with_work(4096 * 1024 - 1);
+        assert!(budget.decode_filters(&doc, &flate, 1 << 24).is_err());
+        assert_eq!((budget.work_left(), budget.refused()), (0, 1));
+        // What isn't decoded here pays for nothing, and fails as it would.
+        let mut budget = ImageBudget::with_work(10);
+        assert!(budget.decode(&doc, &blank(Some("DCTDecode"), b"data".to_vec()), 1 << 24).is_err());
+        assert_eq!((budget.work_left(), budget.refused()), (10, 0));
+        // Unfiltered data pays too, and a size that isn't one is no size.
+        let mut budget = ImageBudget::with_work(u64::MAX);
+        budget.decode(&doc, &blank(None, rows.clone()), 1 << 24).unwrap();
+        assert_eq!(u64::MAX - budget.work_left(), rows.len() as u64 + 4096 * 1024);
+        let mut odd = Dict::new();
+        odd.set(b"Width".to_vec(), Object::Int(-4));
+        odd.set(b"Height".to_vec(), Object::Int(9));
+        let mut budget = ImageBudget::with_work(u64::MAX);
+        budget.decode(&doc, &Stream::from_raw(odd, vec![1, 2]), 1 << 24).unwrap();
+        assert_eq!(u64::MAX - budget.work_left(), 2);
+        // A JPEG pays twice over for each sample, once decoded.
+        let mut budget = ImageBudget::with_work(20);
+        assert!(budget.pay_jpeg(9));
+        assert_eq!((budget.work_left(), budget.refused()), (2, 0));
+        assert!(!budget.pay_jpeg(2));
+        assert_eq!((budget.work_left(), budget.refused()), (0, 1));
+    }
+
     /// A stream a run rewrites is read strictly and charged, and what the caller does with it after
     /// comes out of the same allowance: what can't be paid for spends all of it.
     #[test]
@@ -571,6 +660,21 @@ mod tests {
         assert_eq!(budget.remaining(), (10 << 20) - partial);
     }
 
+    /// A stream that fails its reading as zlib, and then as raw deflate after decoding all of it,
+    /// gives nothing back, and each reference still pays for what it decoded.
+    #[test]
+    fn a_decode_that_gives_nothing_back_still_pays() {
+        let mut data = vec![0x78, 0x01, 0x06, 0xFE, 0xF9];
+        data.extend(vec![0u8; 0x0601]);
+        data.push(0x07);
+        let mut dict = Dict::new();
+        dict.set(b"Filter".to_vec(), Object::name("FlateDecode"));
+        let stream = Stream::from_raw(dict, data);
+        let mut budget = DecodeBudget::new(10 * 0x0601, 1);
+        assert!(budget.decode(&stream).unwrap().is_empty());
+        assert_eq!(budget.remaining(), 9 * 0x0601);
+    }
+
     /// A chain whose later stage shrinks the data pays for every stage's output: a Flate stage
     /// that inflates to hex text, which ASCIIHex then halves, charges both.
     #[test]

crates/create/README.md

@@ -32,8 +32,8 @@ other filters too, with the `/JBIG2Globals` of their own `/DecodeParms` entry, a
 whose JBIG2 page isn't its size (PdfCraft's renderer draws the whole page into the image; the page is refused before it is decoded),
 or whose globals can't be read or decode to more than 32 MiB (they are decoded before the page's size is known, so they have a
 limit of their own), is skipped, and such a mask is left out and the image exported opaque, as JPX and CCITT masks are.
-Past what one export may decode, which its images share, filters and fax and JBIG2 decoders alike (`ImageBudget`; JPEG data isn't
-counted), an image is skipped, and so is one whose mask is past it, which would otherwise come out showing what the mask hides. An export also stops at 2 GiB of files held, or twice the document if that
+Past what one export may decode, which its images share, filters, decoders and pixels alike (`ImageBudget`; a JPEG only when a mask
+means decoding it), an image is skipped, and so is one whose mask is past it, which would otherwise come out showing what the mask hides. An export also stops at 2 GiB of files held, or twice the document if that
 is more, since they stay in memory until it ends.
 Indexed Decode values select palette entries after rounding and clipping. Color-key masks compare the original samples.
 Missing, null or malformed Decode arrays keep the default mapping.

crates/create/src/extract.rs

@@ -7,6 +7,7 @@
 use std::collections::HashSet;
 
 use pdfcraft_cos::{Dict, Document, ImageBudget, MAX_DECODED, ObjRef, Object, Stream};
+use pdfcraft_filters::FilterError;
 
 /// One image, ready to write.
 #[derive(Clone, Debug, PartialEq)]
@@ -165,7 +166,7 @@ fn image(doc: &Document, s: &Stream, budget: &mut ImageBudget) -> Result<(&'stat
     let mut alpha = if mask { None } else { soft_mask(doc, s, w, h, budget).or_else(|| stencil_mask(doc, s, w, h, budget)) };
     // Without its mask the image would come out opaque, showing what the mask hides.
     if budget.refused() > refused {
-        return Err("more image decoding than one operation may do".into());
+        return Err(FilterError::Spent.to_string());
     }
     let key = if mask { None } else { colour_key(doc, s) };
     // Keep the existing JPEG and stencil-mask Decode handling separate from lossless colour images.
@@ -184,6 +185,9 @@ fn image(doc: &Document, s: &Stream, budget: &mut ImageBudget) -> Result<(&'stat
     }
     let mut space = if mask { Space::Gray } else { space(doc, s.dict.get(b"ColorSpace"))? };
     let (samples, bpc) = if jpeg {
+        if !budget.pay_jpeg((w as u64).saturating_mul(h as u64).saturating_mul(space.components() as u64)) {
+            return Err(FilterError::Spent.to_string());
+        }
         // Grey, RGB or CMYK samples (YCCK is converted); a JPEG that doesn't decode, or an
         // indexed one (not valid), is written as is, without its transparency.
         let Some((samples, decoded)) = jpeg_samples(&s.raw, &space, w, h) else { return as_is() };

crates/create/src/tests.rs

@@ -2312,6 +2312,51 @@ fn flate_images_past_the_exports_allowance_are_left_out_with_the_reason() {
     assert_eq!(budget.refused(), 1);
 }
 
+/// A JPEG is written as it is, at no cost, unless a mask means decoding it: that pays for its
+/// samples, and past the allowance the image goes, with the reason, as it would come out without
+/// its transparency otherwise.
+#[test]
+fn a_jpeg_that_has_to_be_decoded_shares_the_allowance() {
+    use pdfcraft_cos::{Dict, Document, ImageBudget, Object, Stream};
+    let mut jpeg = Vec::new();
+    image::RgbImage::from_pixel(4, 4, image::Rgb([9, 9, 9])).write_to(&mut std::io::Cursor::new(&mut jpeg), image::ImageFormat::Jpeg).unwrap();
+    let image_dict = |extra: Option<(&[u8], Object)>| {
+        let mut d = Dict::new();
+        d.set(b"Subtype".to_vec(), Object::name("Image"));
+        d.set(b"Width".to_vec(), Object::Int(4));
+        d.set(b"Height".to_vec(), Object::Int(4));
+        d.set(b"BitsPerComponent".to_vec(), Object::Int(8));
+        d.set(b"ColorSpace".to_vec(), Object::name("DeviceRGB"));
+        d.set(b"Filter".to_vec(), Object::name("DCTDecode"));
+        if let Some((key, value)) = extra {
+            d.set(key.to_vec(), value);
+        }
+        d
+    };
+    let mut doc = Document::new_empty();
+    let mut mask = Dict::new();
+    mask.set(b"Subtype".to_vec(), Object::name("Image"));
+    mask.set(b"Width".to_vec(), Object::Int(4));
+    mask.set(b"Height".to_vec(), Object::Int(4));
+    mask.set(b"BitsPerComponent".to_vec(), Object::Int(8));
+    mask.set(b"ColorSpace".to_vec(), Object::name("DeviceGray"));
+    let mask = doc.add(Object::Stream(Stream::from_raw(mask, vec![0x80; 16])));
+    let masked = doc.add(Object::Stream(Stream::from_raw(image_dict(Some((b"SMask", Object::Ref(mask)))), jpeg.clone())));
+    let plain = doc.add(Object::Stream(Stream::from_raw(image_dict(None), jpeg)));
+
+    let mut budget = ImageBudget::with_work(0);
+    assert!(crate::extract::image_file_within(&doc, plain, &mut budget).is_ok(), "written as it is");
+    assert_eq!(budget.refused(), 0);
+    let mut budget = ImageBudget::with_work(u64::MAX);
+    assert!(crate::extract::image_file_within(&doc, masked, &mut budget).is_ok());
+    let cost = u64::MAX - budget.work_left();
+    assert_eq!(cost, 16 + 16 + 2 * 4 * 4 * 3, "the mask's bytes and pixels, and the JPEG's samples twice");
+    let mut budget = ImageBudget::with_work(cost - 1);
+    let refused = crate::extract::image_file_within(&doc, masked, &mut budget).unwrap_err();
+    assert!(refused.contains("more image decoding than"), "{refused}");
+    assert_eq!(budget.refused(), 1);
+}
+
 /// A mask past the allowance can't be left out as an unreadable one is, or the image comes out
 /// opaque, showing what the mask hides: the image goes, with the reason.
 #[test]
@@ -2326,7 +2371,7 @@ fn an_image_whose_jbig2_mask_is_past_the_allowance_is_left_out() {
     mask.set(b"Filter".to_vec(), Object::name("JBIG2Decode"));
     let mut doc = Document::new_empty();
     let mask = doc.add(Object::Stream(Stream::from_raw(mask, pdfcraft_filters::encode_jbig2_generic(&[0xFF; 4], 16, 2).unwrap())));
-    // A JPEG, which nothing charges, so only its mask can be what stops it.
+    // A JPEG, read for nothing until its mask is, so only the mask can be what stops it.
     let mut jpeg = Vec::new();
     image::RgbImage::from_pixel(1, 1, image::Rgb([9, 9, 9])).write_to(&mut std::io::Cursor::new(&mut jpeg), image::ImageFormat::Jpeg).unwrap();
     let mut image = Dict::new();

crates/filters/src/flate.rs

@@ -28,8 +28,9 @@ pub(crate) fn decode(data: &[u8], max: usize) -> Step {
             // A header that only looked valid: retry the whole input as raw deflate.
             Err(f) if f.partial.is_empty() && matches!(f.error, crate::FilterError::Corrupt { .. }) => match inflate_raw(data, max) {
                 // Keep the original corruption diagnostic only if the retry is corrupt
-                // too. A limit error must remain fatal even for tolerant decoding.
-                Err(retry) if matches!(retry.error, crate::FilterError::Corrupt { .. }) => Err(f),
+                // too. A limit error must remain fatal even for tolerant decoding. What the
+                // retry decoded isn't kept, but a caller charging for work has to hear of it.
+                Err(retry) if matches!(retry.error, crate::FilterError::Corrupt { .. }) => Err(Failure { discarded: retry.partial.len(), ..f }),
                 retry => retry,
             },
             r => r,

crates/filters/src/lib.rs

@@ -133,15 +133,17 @@ pub enum FilterError {
 pub(crate) struct Failure {
     pub(crate) error: FilterError,
     pub(crate) partial: Vec<u8>,
+    /// Bytes decoded on the way that `partial` doesn't hold: they took as long.
+    pub(crate) discarded: usize,
 }
 
 impl Failure {
     pub(crate) fn corrupt(filter: &'static str, detail: impl Into<String>, partial: Vec<u8>) -> Self {
-        Failure { error: FilterError::Corrupt { filter, detail: detail.into() }, partial }
+        Failure { error: FilterError::Corrupt { filter, detail: detail.into() }, partial, discarded: 0 }
     }
 
     pub(crate) fn limit(max: usize) -> Self {
-        Failure { error: FilterError::LimitExceeded(max), partial: Vec::new() }
+        Failure { error: FilterError::LimitExceeded(max), partial: Vec::new(), discarded: 0 }
     }
 }
 
@@ -267,7 +269,7 @@ fn run(
         spent = spent.saturating_add(work);
         cur = Cow::Owned(match step {
             Ok(v) => v,
-            Err(Failure { error: FilterError::Corrupt { .. }, partial: out }) if !strict => {
+            Err(Failure { error: FilterError::Corrupt { .. }, partial: out, .. }) if !strict => {
                 partial = true;
                 out
             }
@@ -286,15 +288,15 @@ fn decode_one(filter: &Filter, params: &Params, data: &[u8], max: usize) -> (Ste
         Filter::AsciiHex => counted(asciihex::decode(data, max)),
         Filter::Ascii85 => counted(ascii85::decode(data, max)),
         Filter::RunLength => counted(runlength::decode(data, max)),
-        other => (Err(Failure { error: FilterError::Unsupported(other.name().to_owned()), partial: Vec::new() }), 0),
+        other => (Err(Failure { error: FilterError::Unsupported(other.name().to_owned()), partial: Vec::new(), discarded: 0 }), 0),
     }
 }
 
-/// A step with the bytes it produced, its output or its partial output.
+/// A step with the bytes it produced: its output, or its partial output and what it threw away.
 fn counted(step: Step) -> (Step, usize) {
     let n = match &step {
         Ok(v) => v.len(),
-        Err(f) => f.partial.len(),
+        Err(f) => f.partial.len().saturating_add(f.discarded),
     };
     (step, n)
 }
@@ -306,12 +308,12 @@ fn with_predictor(step: Step, params: &Params, name: &'static str) -> (Step, usi
     let (step, n) = counted(step);
     let step = match step {
         Ok(v) => predictor::decode(params, v, name),
-        Err(Failure { error, partial }) => {
+        Err(Failure { error, partial, discarded }) => {
             let partial = match predictor::decode(params, partial, name) {
                 Ok(v) => v,
                 Err(f) => f.partial,
             };
-            Err(Failure { error, partial })
+            Err(Failure { error, partial, discarded })
         }
     };
     (step, n)
@@ -621,6 +623,32 @@ mod tests {
         assert_eq!(u64::MAX - work, data.len() as u64 + (1 << 16));
     }
 
+    /// A stream that reads as zlib only until it fails, and as raw deflate until that fails too,
+    /// gives back no data, and has still decoded everything the retry did.
+    #[test]
+    fn a_retry_that_fails_pays_for_what_it_decoded() {
+        // Stored as raw deflate: this many bytes, then a block of a reserved type. As zlib: the
+        // same reserved type at once, so the first reading fails with nothing.
+        let mut data = vec![0x78, 0x01, 0x06, 0xFE, 0xF9];
+        data.extend(vec![0u8; 0x0601]);
+        data.push(0x07);
+        let chain = [(Filter::Flate, Params::default())];
+        let (out, partial) = decode_tolerant(&chain, &data, 1 << 20).unwrap();
+        assert!(out.is_empty() && partial, "what is read stays as it was");
+        let (_, _, spent) = decode_tolerant_counted(&chain, &data, 1 << 20).unwrap();
+        assert_eq!(spent, 0x0601);
+        let (e, spent) = decode_counted(&chain, &data, 1 << 20).unwrap_err();
+        assert!(matches!(e, FilterError::Corrupt { .. }), "{e}");
+        assert_eq!(spent, 0x0601);
+        let mut work = u64::MAX;
+        assert!(decode_tolerant_within(&chain, &data, 1 << 20, &mut work).unwrap().is_empty());
+        assert_eq!(u64::MAX - work, (data.len() + 0x0601) as u64);
+        // Through a predictor, which would otherwise lose the count.
+        let p = Params { predictor: 12, columns: 4, ..Params::default() };
+        let (_, _, spent) = decode_tolerant_counted(&[(Filter::Flate, p)], &data, 1 << 20).unwrap();
+        assert_eq!(spent, 0x0601);
+    }
+
     /// With nothing left nothing is read. What can't be decoded at all never costs anything, and
     /// says so whatever is left. Without filters, what is read is the data.
     #[test]

crates/optimize/README.md

@@ -101,7 +101,8 @@ let report = optimize_with_progress(&mut doc, &settings, &mut |stage| keep_going
   - Lossless JBIG2 uses bounded generic-region encoding or exact-copy symbol dictionaries when
     smaller. No lossy symbol substitution is used. Export, redaction and this step read JBIG2
     images, with bounded shared globals. All of a run's images share one allowance of decoding
-    (`pdfcraft_cos::ImageBudget`), filters and fax and JBIG2 decoders alike, JPEG data aside: about
+    (`pdfcraft_cos::ImageBudget`), filters, decoders and pixels alike, JPEG samples and merged
+    pictures too: about
     1,000 scanned pages' worth, plus more for each byte of the file, enough to decode a stream
     about 8 times at Flate's best. Images past it stay as they are, and `Report::images_refused`
     counts them.

crates/optimize/src/fragments.rs

@@ -167,7 +167,12 @@ fn merge(doc: &Document, run: &[Fragment], n: usize, budget: &mut ImageBudget) -
         return None;
     }
     let row = (width as usize).checked_mul(n)?;
-    let mut out = vec![255u8; row.checked_mul(height as usize)?];
+    // What this builds isn't what its strips cost to read: four small ones can ask for all of it.
+    let bytes = row.checked_mul(height as usize)?;
+    if !budget.pay_pixels(u64::try_from(bytes).ok()?) {
+        return None;
+    }
+    let mut out = vec![255u8; bytes];
     for f in run {
         let r0 = (((top - f.rect[3]) * density).round().max(0.0) as u32).min(height);
         let r1 = (((top - f.rect[1]) * density).round().max(0.0) as u32).min(height);
@@ -370,10 +375,11 @@ mod tests {
             image.set(k.as_bytes().to_vec(), v);
         }
         image.set(b"ColorSpace".to_vec(), Object::name("DeviceGray"));
-        let mut xobjects = Dict::new();
-        let mut drawn = String::new();
+        let (mut xobjects, mut drawn, mut raw) = (Dict::new(), String::new(), 0);
         for row in 0..8 {
-            let r = doc.add(Object::Stream(Stream::flate(image.clone(), &[row * 30; 8])));
+            let strip = Stream::flate(image.clone(), &[row * 30; 8]);
+            raw += strip.raw.len() as u64;
+            let r = doc.add(Object::Stream(strip));
             xobjects.set(format!("I{row}").into_bytes(), Object::Ref(r));
             drawn.push_str(&format!("q 80 0 0 10 0 {} cm /I{row} Do Q\n", 70 - 10 * i32::from(row)));
         }
@@ -388,9 +394,15 @@ mod tests {
         let mut budget = ImageBudget::with_work(0);
         assert_eq!(merge_page(&mut doc.clone(), page, &mut Report::default(), &mut |_| true, &mut budget).unwrap(), 0);
         assert_eq!(budget.refused(), 1);
+        // Reading each strip, and building the picture of 8 rows of 8 they make.
+        let cost = raw + 8 * (8 + 8) + 8 * 8;
         let mut budget = ImageBudget::with_work(u64::MAX);
-        assert_eq!(merge_page(&mut doc, page, &mut Report::default(), &mut |_| true, &mut budget).unwrap(), 8);
-        assert!(budget.work_left() < u64::MAX);
+        assert_eq!(merge_page(&mut doc.clone(), page, &mut Report::default(), &mut |_| true, &mut budget).unwrap(), 8);
+        assert_eq!(u64::MAX - budget.work_left(), cost);
+        // Enough for the strips and not for the picture: nothing is built.
+        let mut budget = ImageBudget::with_work(cost - 1);
+        assert_eq!(merge_page(&mut doc, page, &mut Report::default(), &mut |_| true, &mut budget).unwrap(), 0);
+        assert_eq!(budget.refused(), 1);
     }
 
     /// Many images drawn on top of each other are not one picture; looking for runs among them reads

crates/optimize/src/images.rs

@@ -490,6 +490,9 @@ pub(crate) fn pixels(doc: &Document, s: &Stream, n: usize, budget: &mut ImageBud
         // never get here: the colour space check excludes them. The JPEG's own size must be the
         // dictionary's: `jpeg_samples` refuses another from its header, before any pixels.
         let ycc = n == 3 && ycc_encoded(doc, s)?;
+        if !budget.pay_jpeg(u64::try_from(bytes).ok()?) {
+            return None;
+        }
         let v = jpeg_samples(&s.raw, usize::try_from(w).ok()?, usize::try_from(h).ok()?, n, ycc)?;
         count_decoded(u64::from(w) * u64::from(h));
         (v.len() == bytes).then_some(v)
@@ -1496,7 +1499,7 @@ mod dimension_tests {
         dict.set(b"BitsPerComponent".to_vec(), Object::Int(8));
         dict.set(b"ColorSpace".to_vec(), Object::name("DeviceGray"));
         let image = Stream::flate(dict, &samples);
-        let read = (image.raw.len() + samples.len()) as u64;
+        let read = (image.raw.len() + samples.len()) as u64 + 256 * 256;
         let original = doc.add(Object::Stream(image.clone()));
         let mut xobjects = Dict::new();
         xobjects.set(b"I0".to_vec(), Object::Ref(original));
@@ -1545,6 +1548,25 @@ mod dimension_tests {
         assert_eq!((report.images, report.images_refused), (1, 0), "{report:?}");
     }
 
+    /// A JPEG pays for its samples before it is decoded, however few bytes it takes: past what is
+    /// left it isn't, and counts as refused.
+    #[test]
+    fn a_jpeg_pays_for_its_samples() {
+        let mut dict = Dict::new();
+        dict.set(b"Width".to_vec(), Object::Int(16));
+        dict.set(b"Height".to_vec(), Object::Int(8));
+        dict.set(b"BitsPerComponent".to_vec(), Object::Int(8));
+        dict.set(b"ColorSpace".to_vec(), Object::name("DeviceRGB"));
+        let jpeg = image_stream(&dict, &[120; 16 * 8 * 3], 3, 16, 8, Compression::Jpeg(60), false).expect("a JPEG");
+        let doc = Document::new_empty();
+        let mut budget = ImageBudget::with_work(2 * 16 * 8 * 3);
+        assert_eq!(pixels(&doc, &jpeg, 3, &mut budget).map(|p| p.len()), Some(16 * 8 * 3));
+        assert_eq!((budget.work_left(), budget.refused()), (0, 0), "exactly that");
+        let mut budget = ImageBudget::with_work(2 * 16 * 8 * 3 - 1);
+        assert!(pixels(&doc, &jpeg, 3, &mut budget).is_none());
+        assert_eq!((budget.work_left(), budget.refused()), (0, 1));
+    }
+
     /// A soft mask resampled with its image is read from the same allowance as the image.
     #[test]
     fn a_soft_mask_resampled_with_its_image_spends_the_allowance() {
@@ -1568,7 +1590,7 @@ mod dimension_tests {
         };
         let mut doc = Document::new_empty();
         let mask = image("DeviceGray");
-        let cost = (mask.raw.len() + noise.len()) as u64;
+        let cost = (mask.raw.len() + noise.len()) as u64 + 256 * 256;
         let masked = doc.add(Object::Stream(mask));
         let mut with_mask = image("DeviceGray");
         with_mask.dict.set(b"SMask".to_vec(), Object::Ref(masked));

crates/redact/README.md

@@ -34,7 +34,7 @@ clear_marks(&mut doc, None)?;                        // remove marks without app
   twice; for a fax image's code, about a byte per pixel), are removed whole;
   cleared copies drop `/Alternates` so alternate pixels cannot survive;
   A run that would decode more of its images than one redaction may (`pdfcraft_cos::ImageBudget`,
-  filters and fax and JBIG2 decoders alike) is refused
+  filters, decoders and pixels alike) is refused
   and changes nothing: such an image can't be cleared, and removing it whole would take what no mark
   covers with it.
 - **vectors**: covered paths are removed; partly covered paths and shadings are clipped so

parity/acrobat-features.toml

@@ -7409,7 +7409,7 @@ status = "partial"
 commands = ["optimize.advanced", "optimize.reduce"]
 tools = ["doc_optimize", "doc_reduce"]
 evidence = ["crates/optimize/src/images.rs::flate_images_past_the_runs_allowance_stay_as_they_are", "crates/optimize/tests/optimize.rs::eight_bit_black_and_white_images_become_one_bit", "crates/filters/src/ccitt.rs::bitmaps_round_trip", "crates/filters/src/ccitt.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/filters/src/ccitt.rs::what_it_cannot_encode_is_an_error", "crates/optimize/tests/optimize.rs::black_and_white_images_become_ccitt_group_4", "crates/optimize/tests/optimize.rs::group_3_fax_images_become_group_4", "crates/optimize/tests/optimize.rs::rgb_and_two_colour_indexed_black_and_white_images_become_one_bit", "crates/optimize/tests/optimize.rs::black_and_white_rgb_used_beyond_the_page_content_read_stays_rgb", "crates/optimize/tests/optimize.rs::black_and_white_rgb_drawn_only_where_the_optimizer_reads_becomes_grey", "crates/optimize/tests/optimize.rs::black_and_white_rgb_in_resources_its_page_tree_and_a_form_share_becomes_grey", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_ccitt_group_4", "crates/ui-egui/tests/it/create.rs::the_pdf_optimizer_dialog_saves_an_optimized_copy", "crates/filters/src/mq.rs::the_annex_h_test_sequence_encodes_to_its_bytes", "crates/filters/src/jbig2.rs::bitmaps_round_trip", "crates/filters/src/jbig2.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/optimize/tests/optimize.rs::black_and_white_images_become_jbig2", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_jbig2", "crates/filters/src/jbig2/symbols.rs::repeated_marks_are_stored_once", "crates/filters/src/jbig2/symbols.rs::scattered_marks_round_trip", "crates/optimize/tests/optimize.rs::rendered_text_becomes_jbig2_symbols", "crates/optimize/tests/optimize.rs::jbig2_scans_are_averaged_and_stored_again", "crates/optimize/src/images.rs::jbig2_images_past_the_runs_allowance_stay_as_they_are"]
-notes = "All of a run's images share one allowance of decoding, filters included and JPEG data aside (about 1,000 scanned pages' worth, more for a longer file); those past it stay as they are and are counted in the report, which doc_optimize returns and the panel doesn't show yet. CCITT Group 4, lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (on by default, so Reduce File Size does it too; doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. Packed to one bit first, then stored the same way (colour-key masks aside): eight-bit grey of only black and white, eight-bit DeviceRGB of only black and white (as DeviceGray; calibrated RGB, RGB with a profile, RGB drawn or listed where /DefaultGray or /DefaultRGB is in force or referred to from anywhere but the page and form resources read (an annotation's appearance or a pattern drawing it, directly or through resources or a form shared with a page; another image's /Alternates), and RGB whose soft mask has a /Matte aside), and an indexed image of 2, 4 or 8 bits that uses only palette entries 0 and 1 (keeping its palette), unless the grey or colour settings make it smaller still, as they can a dithered scan downsampled to JPEG, its soft mask counted. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. Group 3 fax images are stored again as Group 4 when that is smaller (damaged Group 3 with end-of-line codes as the decoder recovers it, as it reports no damage there; byte-aligned Group 3 with end-of-line codes but none before the first row, not as T.4 sends them, as PdfCraft draws it, which at /K 0 may be other rows than were coded). CCITT fax images are decoded to be downsampled (optimize.downsample-images), and a resampled image is stored as Group 4 or 1-bit Flate, whichever is smaller. Lossless JBIG2 with bounded exact-copy symbol coding is available too, with image export; lossy JBIG2 remains unsupported. JBIG2 images are read too: averaged like any (under Retain stored as JBIG2 again), and, not resampled, stored again as the compression asks when that is smaller (as JBIG2 symbols where its own coding is a generic region, say); a symbol-coded scan, averaged, loses its exact repeats and is often larger, and then stays as it is; one whose JBIG2 page isn't its size is left alone. Not yet: inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples."
+notes = "All of a run's images share one allowance of decoding, filters, decoders and pixels alike (about 1,000 scanned pages' worth, more for a longer file); those past it stay as they are and are counted in the report, which doc_optimize returns and the panel doesn't show yet. CCITT Group 4, lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (on by default, so Reduce File Size does it too; doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. Packed to one bit first, then stored the same way (colour-key masks aside): eight-bit grey of only black and white, eight-bit DeviceRGB of only black and white (as DeviceGray; calibrated RGB, RGB with a profile, RGB drawn or listed where /DefaultGray or /DefaultRGB is in force or referred to from anywhere but the page and form resources read (an annotation's appearance or a pattern drawing it, directly or through resources or a form shared with a page; another image's /Alternates), and RGB whose soft mask has a /Matte aside), and an indexed image of 2, 4 or 8 bits that uses only palette entries 0 and 1 (keeping its palette), unless the grey or colour settings make it smaller still, as they can a dithered scan downsampled to JPEG, its soft mask counted. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. Group 3 fax images are stored again as Group 4 when that is smaller (damaged Group 3 with end-of-line codes as the decoder recovers it, as it reports no damage there; byte-aligned Group 3 with end-of-line codes but none before the first row, not as T.4 sends them, as PdfCraft draws it, which at /K 0 may be other rows than were coded). CCITT fax images are decoded to be downsampled (optimize.downsample-images), and a resampled image is stored as Group 4 or 1-bit Flate, whichever is smaller. Lossless JBIG2 with bounded exact-copy symbol coding is available too, with image export; lossy JBIG2 remains unsupported. JBIG2 images are read too: averaged like any (under Retain stored as JBIG2 again), and, not resampled, stored again as the compression asks when that is smaller (as JBIG2 symbols where its own coding is a generic region, say); a symbol-coded scan, averaged, loses its exact repeats and is often larger, and then stays as it is; one whose JBIG2 page isn't its size is left alone. Not yet: inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples."
 
 [[feature]]
 id = "optimize.only-if-smaller"