fix(render): an unreadable CMap token counts once against the skip limit, with unit tests (hayro-cmap patch 3, hayro-postscript patch 2)

- `Scanner::parse_object_or_skip` skips whitespace and comments before
  taking its start offset, so a token after a space fails once, not
  twice, whoever calls it. (The CMap scan already skipped whitespace in
  `at_end`, so CMap results do not change.)
- Unit tests in hayro-postscript (stepping over a token, a failed read
  not read again, never past the end of the data) and in hayro-cmap
  (bfchar and bfrange after a stray `>` or a spaced name, the 1,024-token
  limit at its boundary, a CMap of unreadable tokens only failing).
- The render test checks a bfrange entry after the bad token, a run past
  the limit before the mappings, and a CMap of unreadable tokens only;
  its wall-clock assertion is gone, the no-re-read property is the
  deterministic hayro-postscript test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

crates/render/src/raster/tests.rs

@@ -9630,7 +9630,7 @@ fn a_stream_with_a_crypt_filter_of_its_own_is_drawn() {
 #[test]
 fn a_to_unicode_cmap_reads_its_mappings_past_a_token_it_cannot_parse() {
     use hayro::hayro_interpret::hayro_cmap::{BfString, CMap};
-    let body = "1 begincodespacerange\n<0000> <FFFF>\nendcodespacerange\n2 beginbfchar\n<002B> <0048>\n<0048> <0065>\nendbfchar\nendcmap\n";
+    let body = "1 begincodespacerange\n<0000> <FFFF>\nendcodespacerange\n2 beginbfchar\n<002B> <0048>\n<0048> <0065>\nendbfchar\n1 beginbfrange\n<0050> <0052> <0061>\nendbfrange\nendcmap\n";
     let stray = format!(
         "/CIDInit /ProcSet findresource begin 12 dict begin begincmap\n/CIDSystemInfo <</Registry (Adobe) /Ordering (UC>) /Supplement 0>> def\n/CMapName /Adobe-Identity-UC> def\n/CMapType 2 def\n{body}"
     );
@@ -9641,15 +9641,20 @@ fn a_to_unicode_cmap_reads_its_mappings_past_a_token_it_cannot_parse() {
         let cmap = CMap::parse(data.as_bytes(), |_| None).unwrap_or_else(|| panic!("{what}: parsed"));
         assert_eq!(cmap.lookup_bf_string(0x2B), Some(BfString::Char('H')), "{what}");
         assert_eq!(cmap.lookup_bf_string(0x48), Some(BfString::Char('e')), "{what}");
+        assert_eq!(cmap.lookup_bf_string(0x51), Some(BfString::Char('b')), "{what}: bfrange");
     }
-    // Unreadable tokens are stepped over a bounded number of times: a long run of them ends
-    // the scan rather than costing its length squared, and what came before is kept.
+    // Unreadable tokens are stepped over a bounded number of times (1,024 per CMap): a long run
+    // of them ends the scan, and the mappings read before it are kept.
     let flood = format!("{body}{}", "-".repeat(1 << 16));
     assert!(CMap::parse(flood.as_bytes(), |_| None).is_some_and(|c| c.lookup_bf_string(0x2B).is_some()));
-    // What a failed read went through is not read again: unclosed strings, each of which reads
-    // to the end of the data, cost one pass over it, not one pass each.
-    let unclosed = format!("{body}{}{}", "(".repeat(4096), "x".repeat(1 << 23));
-    let started = std::time::Instant::now();
+    // A run past the limit before the mappings ends the scan before them, and a CMap that read
+    // no mappings, like one of unreadable tokens only, reports failure.
+    let flood_first = format!("{}\n{body}", "-".repeat(1025));
+    assert!(CMap::parse(flood_first.as_bytes(), |_| None).is_none());
+    assert!(CMap::parse("-".repeat(1 << 16).as_bytes(), |_| None).is_none());
+    // Unclosed strings after the mappings: the scan ends and keeps them. (That the first one,
+    // which reads to the end of the data, is not read again for each `(` inside it is
+    // hayro-postscript's `parse_object_or_skip_does_not_read_a_failed_read_again`.)
+    let unclosed = format!("{body}{}{}", "(".repeat(4096), "x".repeat(1 << 16));
     assert!(CMap::parse(unclosed.as_bytes(), |_| None).is_some_and(|c| c.lookup_bf_string(0x48).is_some()));
-    assert!(started.elapsed() < std::time::Duration::from_secs(3), "{:?}", started.elapsed());
 }

vendor/README.md

@@ -47,8 +47,8 @@ Other resource categories retain their existing parent-chain lookup.
 | winit | 0.30.13 | Apache-2.0 | (1) on Windows 11 (build 22000 and later), `WM_DPICHANGED` applies the rect Windows suggests. It used to nudge the window onto the monitor `MonitorFromWindow` names, which during a drag is still the monitor being left; the nudge sent another `WM_DPICHANGED`, so a window dragged to a monitor with another scale factor grew and stayed on the wrong monitor at the wrong scale (#324). Backport of winit master 488c036a ([winit#4341](https://github.com/rust-windowing/winit/pull/4341)), which no 0.30 release has; Windows 10 keeps the old adjustment, as on winit master. (2) on Wayland, a seat's text input is destroyed when the seat goes away, not whenever it loses a capability: losing any capability (a touch screen or tablet unplugged, a remote-desktop session ending) destroyed it, and only an added capability recreated it, so IME stopped working until restart. Port of [winit#4747](https://github.com/rust-windowing/winit/pull/4747); (3) on Windows, an IME composition reports its cursor from `GCS_CURSORPOS` only when `WM_IME_COMPOSITION` says it is set, and at the end of the composition otherwise: the Microsoft Korean IME never sets it, so the cursor read as 0 (egui then drew no caret while composing; it now draws it after the syllable). Port of [winit#4746](https://github.com/rust-windowing/winit/pull/4746); (4) on Windows, a hidden window is not maximized until it is shown, and a hidden window shown while maximized gets one `ShowWindow(SW_MAXIMIZE)`: `SW_MAXIMIZE` shows a hidden window, so PdfCraft, which eframe keeps hidden until its first frame is drawn, appeared unpainted (white) when it restored a maximized window, and could lose its focus. Port of [winit#4587](https://github.com/rust-windowing/winit/pull/4587) (the later, focus-safe form of [winit#4575](https://github.com/rust-windowing/winit/pull/4575)); (5) on X11, when allowing or disallowing IME replaces the focused window's input context (egui does it each time a text field gains or loses focus), the new context is focused: no focus event follows, so it stayed unfocused until the window lost and regained focus. Port of [winit#4727](https://github.com/rust-windowing/winit/pull/4727); (6) on macOS, `insertText` commits through IME when a composition was started (`pending_commit`, set by `setMarkedText`) rather than when marked text is left: input methods that end a composition with `unmarkText` before `insertText` lost the text, and Apple Korean dropped the key typed after a syllable (`한` then `5` or `;`); other `insertText` calls stay `KeyboardInput`; a commit outside a key press (a candidate clicked with the mouse, dictation) clears the marked text and leaves no key to swallow, as in [winit#4748](https://github.com/rust-windowing/winit/pull/4748) (without it the next key, such as Backspace, was lost). `validAttributesForMarkedText` lists the underline and marked-clause attributes, without which some input methods don't compose. Port of [winit#4650](https://github.com/rust-windowing/winit/pull/4650), the 0.30 backport of [winit#4651](https://github.com/rust-windowing/winit/pull/4651) (merged on winit master), with the underline attribute spelled as AppKit's `NSUnderlineStyleAttributeName` (`NSUnderline`); (7) on macOS, disallowing IME with a composition in progress (focus leaving a text field) tells the input method to discard its marked text (`discardMarkedText`): it kept composing it once IME was allowed again, so the abandoned syllable came back (`하`, then `가` typed in the next field, became `학`, then `하` + `가`; Japanese `か` then `さ` became `傘`). Port of [winit#4745](https://github.com/rust-windowing/winit/pull/4745); (8) on macOS, text inserted outside a key press without a composition (the emoji picker) is reported as `Ime::Commit`; it was dropped. Port of the emoji-picker part of [winit#4749](https://github.com/rust-windowing/winit/pull/4749) (0.30 backport of [winit#4748](https://github.com/rust-windowing/winit/pull/4748)); its `selectedRange` change for dictation is left out because it switches on press-and-hold for every held key (an accent menu instead of key repeat, and the chosen accent added after the repeated letters); (9) on macOS, a key whose text was committed through IME isn't also sent as `KeyboardInput` unless AppKit forwards it as a command (Enter): Apple Korean answers Space with no syllable in progress with `setMarkedText(" ")`, `insertText(" ")`, `setMarkedText("")`, and the last call reset winit's state, so the space arrived twice. Same goal as [winit#4478](https://github.com/rust-windowing/winit/pull/4478), whose changes don't reach this call order (macOS 14); to be proposed upstream there; (10) on macOS, when the first key in a fresh app reaches Apple Korean's `insertText` as a raw compatibility jamo instead of starting a composition, that key is interpreted once more, synchronously, before it is sent as `KeyboardInput` (g k s typed ㅎㅏㄴ instead of 한); only an unmodified, Line truncated
 | egui-wgpu | 0.36.2 | MIT OR Apache-2.0 | (1) `winit::surface_fit`: a window's surface is configured within the device's `max_texture_dimension_2d`, both sides shrunk by one factor that egui then draws at (`pixels_per_point × render_scale`), so the whole window is drawn into the surface (on DX12 it is stretched over the window and lines up with the pointer; wgpu's GL backend copies it unscaled into a corner); the longer side becomes exactly the limit; the depth and MSAA textures follow, and a recreated surface starts again from the window's own size. egui-wgpu configured the window's size as is, and `Surface::configure` panics beyond the limit: a window restored at another monitor's scale (opening an Outlook attachment on a mixed-DPI laptop asked an 8192-pixel device for 8600 × 3423, #577) or stretched across monitors closed PdfCraft, before its window appeared when it happened at start-up ([emilk/egui#8361](https://github.com/emilk/egui/issues/8361)). The published crate leaves out egui's licence files; `LICENSE-APACHE` and `LICENSE-MIT` are from the 0.36.2 tag; (2) `winit::catch_errors`: a new window's surface is configured inside wgpu error scopes (validation, out of memory, internal), and a failure is returned as `WgpuError::ConfigureSurface` (for the root window at start-up eframe returns it as `Error::Wgpu`, which PdfCraft retries with OpenGL); nothing is kept for the window. wgpu's default handler panicked instead, and eframe configures the root surface before the app is created, so a driver that can't configure one closed PdfCraft at launch (a 2015 Intel GPU through wgpu's GL backend timed out waiting for the GPU to go idle, `GpuWaitTimeout`, #519). Unchanged, so still a panic: an error in a later reconfigure (resize, a lost surface), a device lost during the first configure (wgpu reports it only to a callback), and a window shown at zero size (first configured on resize) | (1) `pdfcraft` `tests::a_window_larger_than_the_gpu_limit_gets_a_surface_within_it`, `tests::surface_fit_never_empties_or_overflows_a_surface`, and `tests::vendor_provenance_egui_wgpu_surface_fit_identity`, which keeps the patch in the build and in the paths that size a surface; (2) `pdfcraft` `tests::a_gpu_error_while_a_window_is_set_up_is_returned_not_a_panic` (a real device: an error inside `catch_errors` is returned, not a panic) and `tests::vendor_provenance_egui_wgpu_catch_errors_identity`, which keeps it around `add_surface`'s configure |
 | vello_common | 0.0.8 | Apache-2.0 OR MIT | (1) `clip::intersect` (the clip step every clipped path passes through) starts its row walk at the first row both paths have strips in, not the earlier of the two first rows. The walk visited every row from the top of the clip for each path (on the A1 sample's full-page canvas, about half its strip rows per stroke), and rows where one path has no strips produce no output, so the strips are unchanged (vello_common 0.1 starts at the later row too). Its cost grew with the page, and a full-page render of the A1 sample was about 30% faster; the output is byte-identical | (1) `clip::tests::intersect_walks_only_rows_both_paths_have_strips` counts the rows the walk visits (`cargo test --manifest-path vendor/vello_common/Cargo.toml`) and fails without the patch; the render crate's pixel tests run through the patched crate |
-| hayro-cmap | 0.1.0 | Apache-2.0 OR MIT | (1) malformed parts of a CMap (bad bfchar/bfrange entries, unresolvable `usecmap`, unparseable tail) are skipped instead of discarding the whole map, so search and redaction find text the edit layer can read (#3177); (2) a binary CMap's `usecmap` counts against the 16-level nesting limit as a text one does, where it passed its own depth and a self-referencing binary CMap recursed until the stack overflowed (from storytold/craft-libs#254); (3) a token the scanner can't read (a stray `>` after a CMap name, a CMap name with spaces in it) is stepped over with `parse_object_or_skip`, at most 1,024 times per CMap, instead of ending the scan before the mappings that follow it (pdf.js `issue11651.pdf`, `issue7492.pdf`: no text could be copied) | (1) `hayro-cmap::tests::bad_bfchar_entry_is_skipped`, `unresolvable_usecmap_is_skipped`; (2) `tests::self_referencing_binary_usecmap_stops_at_the_nesting_limit`; (3) `raster::tests::a_to_unicode_cmap_reads_its_mappings_past_a_token_it_cannot_parse` (pdfcraft-render) |
-| hayro-postscript | 0.1.0 | Apache-2.0 OR MIT | (1) the token scanner's `object::read` skips `<<` and `>>` in a loop; it called itself once per delimiter, so a CMap stream holding a long run of `<` or `>` overflowed the stack and aborted the process (from storytold/craft-libs#263); (2) `Scanner::parse_object_or_skip`: after an object that can't be read, the scanner reads on from where the failed read stopped, or from one byte past where it started when it read nothing, so a caller can read on and the data is read about once (hayro-cmap patch 3) | (1) `hayro-postscript::object::tests::long_open_dict_run_does_not_overflow`, `long_close_dict_run_does_not_overflow`, `dict_delimiters_are_skipped`, `nested_dict_delimiters_are_skipped`; (2) `raster::tests::a_to_unicode_cmap_reads_its_mappings_past_a_token_it_cannot_parse` (pdfcraft-render) |
+| hayro-cmap | 0.1.0 | Apache-2.0 OR MIT | (1) malformed parts of a CMap (bad bfchar/bfrange entries, unresolvable `usecmap`, unparseable tail) are skipped instead of discarding the whole map, so search and redaction find text the edit layer can read (#3177); (2) a binary CMap's `usecmap` counts against the 16-level nesting limit as a text one does, where it passed its own depth and a self-referencing binary CMap recursed until the stack overflowed (from storytold/craft-libs#254); (3) a token the scanner can't read (a stray `>` after a CMap name, a CMap name with spaces in it) is stepped over with `parse_object_or_skip`, at most 1,024 times per CMap, instead of ending the scan before the mappings that follow it (pdf.js `issue11651.pdf`, `issue7492.pdf`: no text could be copied) | (1) `hayro-cmap::tests::bad_bfchar_entry_is_skipped`, `unresolvable_usecmap_is_skipped`; (2) `tests::self_referencing_binary_usecmap_stops_at_the_nesting_limit`; (3) `hayro-cmap::tests::{mappings_after_an_unreadable_token_are_read, the_scan_steps_over_at_most_max_skipped_unreadable_tokens, a_cmap_of_unreadable_tokens_only_reports_failure}`, `raster::tests::a_to_unicode_cmap_reads_its_mappings_past_a_token_it_cannot_parse` (pdfcraft-render) |
+| hayro-postscript | 0.1.0 | Apache-2.0 OR MIT | (1) the token scanner's `object::read` skips `<<` and `>>` in a loop; it called itself once per delimiter, so a CMap stream holding a long run of `<` or `>` overflowed the stack and aborted the process (from storytold/craft-libs#263); (2) `Scanner::parse_object_or_skip`: after an object that can't be read, the scanner reads on from where the failed read stopped, or from one byte past where its token starts (after whitespace and comments) when it read nothing, so a caller can read on, one failure per unreadable token, and the data is read about once (hayro-cmap patch 3) | (1) `hayro-postscript::object::tests::long_open_dict_run_does_not_overflow`, `long_close_dict_run_does_not_overflow`, `dict_delimiters_are_skipped`, `nested_dict_delimiters_are_skipped`; (2) `hayro-postscript::tests::{parse_object_or_skip_steps_over_an_unreadable_token, parse_object_or_skip_does_not_read_a_failed_read_again, parse_object_or_skip_stays_at_the_end}`, `raster::tests::a_to_unicode_cmap_reads_its_mappings_past_a_token_it_cannot_parse` (pdfcraft-render) |
 
 `hayro-postscript` is the crates.io 0.1.0 package plus patches (1) and (2). The local Cargo
 registry cache metadata for that package (`.cargo_vcs_info.json`) records upstream

vendor/hayro-cmap/src/lib.rs

@@ -1176,6 +1176,49 @@ endbfchar
         assert!(cmap.lookup_bf_string(0x41).is_some());
     }
 
+    // PdfCraft patch (3): a token the scanner can't read (a stray `>` after a CMap name, a name
+    // with spaces in it) is stepped over, so the mappings after it are still read; at most
+    // `MAX_SKIPPED` of them per CMap.
+
+    #[test]
+    fn mappings_after_an_unreadable_token_are_read() {
+        for header in [
+            &b"/CMapName /Adobe-Identity-UC> def"[..],
+            b"/CMapName /-usr-share-fonts-Panton Family - Panton.otf,000-UTF16 def",
+        ] {
+            let mut data = header.to_vec();
+            data.extend_from_slice(
+                b"\n1 beginbfchar <41> <0058> endbfchar\n1 beginbfrange <50> <52> <0061> endbfrange",
+            );
+            let cmap = parse_with_preamble(&data);
+            assert_eq!(cmap.lookup_bf_string(0x41), Some(BfString::Char('X')));
+            assert_eq!(cmap.lookup_bf_string(0x51), Some(BfString::Char('b')));
+        }
+    }
+
+    /// A CMap with `count` unreadable tokens, each after a space, before one `bfchar` mapping.
+    fn mapping_after_unreadable_tokens(count: usize) -> Option<CMap> {
+        let mut data = PREAMBLE.to_vec();
+        data.extend_from_slice(&b" >".repeat(count));
+        data.extend_from_slice(b"\n1 beginbfchar <41> <0058> endbfchar");
+        CMap::parse(&data, |_| None)
+    }
+
+    #[test]
+    fn the_scan_steps_over_at_most_max_skipped_unreadable_tokens() {
+        let limit = parse::MAX_SKIPPED;
+        let cmap = mapping_after_unreadable_tokens(limit).expect("read up to the limit");
+        assert_eq!(cmap.lookup_bf_string(0x41), Some(BfString::Char('X')));
+        // One more ends the scan before the mapping, and a CMap with no mappings fails.
+        assert!(mapping_after_unreadable_tokens(limit + 1).is_none());
+    }
+
+    #[test]
+    fn a_cmap_of_unreadable_tokens_only_reports_failure() {
+        assert!(CMap::parse(b"> ) >", |_| None).is_none());
+        assert!(CMap::parse(&b" >".repeat(4 * parse::MAX_SKIPPED), |_| None).is_none());
+    }
+
     #[test]
     fn bad_bfchar_code_keeps_later_entries_aligned() {
         // The malformed 5-byte code still has a destination object; both

vendor/hayro-cmap/src/parse.rs

@@ -16,7 +16,7 @@ struct Context<F> {
 }
 
 /// PdfCraft patch (3): most unreadable tokens one CMap may step over before its scan stops.
-const MAX_SKIPPED: usize = 1024;
+pub(crate) const MAX_SKIPPED: usize = 1024;
 
 pub(crate) fn parse_inner<'a>(
     data: &[u8],

vendor/hayro-postscript/src/lib.rs

@@ -61,16 +61,21 @@ impl<'a> Scanner<'a> {
     }
 
     /// PdfCraft patch (2): parse the next object; when it can't be read, read on from where
-    /// the failed read stopped, or, when it read nothing, from one byte past where it started,
-    /// so a caller can read on past a token it can't parse (a stray `>` in a name, a CMap name
-    /// with spaces in it) instead of stopping there. Every failure moves forward, and what a
-    /// failed read went through (an unclosed string, to the end of the data) is not read
-    /// again, so a loop over this ends and reads the data about once.
+    /// the failed read stopped, or, when it read nothing, from one byte past where its token
+    /// starts, so a caller can read on past a token it can't parse (a stray `>` in a name, a
+    /// CMap name with spaces in it) instead of stopping there. Every failure moves forward, one
+    /// failure per unreadable token, and what a failed read went through (an unclosed string,
+    /// to the end of the data) is not read again, so a loop over this ends and reads the data
+    /// about once.
     pub fn parse_object_or_skip(&mut self) -> Result<Object<'a>> {
+        // The whitespace and comments before the token are not the token: skipped first, they
+        // don't count as progress, so a token after a space fails once, not twice.
+        object::skip_whitespace_and_comments(&mut self.reader);
         let start = self.reader.offset();
         let read = object::read(&mut self.reader);
         if read.is_err() && self.reader.offset() <= start {
-            self.reader.jump(start.saturating_add(1).min(self.reader.data.len()));
+            self.reader
+                .jump(start.saturating_add(1).min(self.reader.data.len()));
         }
         read
     }
@@ -191,4 +196,54 @@ endcmap"#;
         let mut scanner = Scanner::new(b"42 ");
         assert!(scanner.parse_name().is_err());
     }
+
+    /// PdfCraft patch (2) test: a token that can't be read fails once and is stepped over,
+    /// whether or not whitespace or a comment comes before it, and the objects after it read.
+    #[test]
+    fn parse_object_or_skip_steps_over_an_unreadable_token() {
+        let mut s = Scanner::new(b"/A> /B % note\n > /C");
+        assert_eq!(
+            s.parse_object_or_skip(),
+            Ok(Object::Name(Name::new(b"A", true)))
+        );
+        assert!(s.parse_object_or_skip().is_err());
+        assert_eq!(
+            s.parse_object_or_skip(),
+            Ok(Object::Name(Name::new(b"B", true)))
+        );
+        assert!(s.parse_object_or_skip().is_err());
+        assert_eq!(
+            s.parse_object_or_skip(),
+            Ok(Object::Name(Name::new(b"C", true)))
+        );
+        assert!(s.at_end());
+    }
+
+    /// PdfCraft patch (2) test: what a failed read went through is not read again. An unclosed
+    /// string reads to the end of the data, so one failure leaves nothing to read; the nested
+    /// `(` inside it are not each read to the end once more.
+    #[test]
+    fn parse_object_or_skip_does_not_read_a_failed_read_again() {
+        let mut s = Scanner::new(b"/A (((( x");
+        assert_eq!(
+            s.parse_object_or_skip(),
+            Ok(Object::Name(Name::new(b"A", true)))
+        );
+        assert!(s.parse_object_or_skip().is_err());
+        assert!(s.at_end());
+    }
+
+    /// PdfCraft patch (2) test: at the end of the data a read fails and the scanner stays at
+    /// the end; it never steps past the data.
+    #[test]
+    fn parse_object_or_skip_stays_at_the_end() {
+        for input in [&b""[..], b"  ", b">"] {
+            let mut s = Scanner::new(input);
+            for _ in 0..3 {
+                assert!(s.parse_object_or_skip().is_err());
+                assert!(s.reader.offset() <= input.len());
+            }
+            assert!(s.at_end());
+        }
+    }
 }