From a7f0074381c5aa0e3d4d2f85212aaa876b3491e5 Mon Sep 17 00:00:00 2001 From: Grzegorz Martens Date: Tue, 10 Nov 2026 15:53:40 +0100 Subject: [PATCH] M8: doc_hidden_info and the parity note say what the refusal covers Review of the refusal. `doc_hidden_info` said it lists what Remove Hidden Information would remove. A page that can't be read in full is counted as one item of hidden text and of hidden layers, so that the removal runs and refuses it: since the refusal before this, that includes a page holding bytes the parser can't read or an inline image whose end is a guess, not only content streams that don't decode. The description now says so, and that with form fields removed too, a field's appearance is read the same way though it isn't counted. The parity note on Apply Redactions lists the new refusal beside the inline image's. Removing hidden layers alone refuses such a page in a document without `/OCProperties` too, which a review asked to narrow: a membership dictionary there (`/AllOff` of a layer that is on) still hides what it marks, for the walk and for the app's renderer. The comment at the refusal in `sanitize.rs` says only what the bytes are shown to hide: text in a broken `TJ`. Tests: `bytes_the_parser_cannot_read_refuse_removing_hidden_layers_alone`; `hidden_text_beside_bytes_the_parser_cannot_read_in_a_field_appearance_is_refused` (removing form fields with hidden text, and Sanitize Document, which went on and left it); and `hidden_text_beside_bytes_the_parser_cannot_read_is_refused` gains hidden text in a broken `TJ` inside a form. The form-away test's doc says the check after the walk would still refuse a form without resources. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY --- crates/automation/src/tools.rs | 2 +- crates/redact/src/sanitize.rs | 6 +-- crates/redact/src/tests/unparsed.rs | 60 ++++++++++++++++++++++++++++- parity/acrobat-features.toml | 2 +- 4 files changed, 63 insertions(+), 7 deletions(-) diff --git a/crates/automation/src/tools.rs b/crates/automation/src/tools.rs index 1cd876d0b1..a37fe49ffe 100644 --- a/crates/automation/src/tools.rs +++ b/crates/automation/src/tools.rs @@ -726,7 +726,7 @@ pub fn tools() -> Vec { t( "doc_hidden_info", "Find hidden information", - "What Remove Hidden Information would remove, by category (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data) with counts.", + "What Remove Hidden Information would remove, by category (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data) with counts. A page that can't be read in full (its content streams don't decode, or they or a form they draw hold bytes the parser can't read or an inline image whose end is a guess) counts as one item of hidden-text and of hidden-layers: removing either category then fails on that page. With form-fields removed too, a field's appearance becomes page content and is read the same way, though it isn't counted here.", ) .ro() .with(schema(json!({ "doc": doc() }), &["doc"])), diff --git a/crates/redact/src/sanitize.rs b/crates/redact/src/sanitize.rs index 3da224e8ac..cd38b9eb19 100644 --- a/crates/redact/src/sanitize.rs +++ b/crates/redact/src/sanitize.rs @@ -660,9 +660,9 @@ fn content_pass(doc: &mut Document, text: bool, layers: bool, write: bool) -> Re return Err(error.into()); } // Bytes the parser can't read, or an inline image whose end is a guess: the walk reads past - // them, so hidden text or a hidden layer's content they hold, which other readers may draw, - // is neither found nor removed. As for content that doesn't decode, count the page so the - // removal runs and refuses it. + // them, so what it reads around them may not be what other readers draw (hidden text in a + // broken `TJ` is neither found nor removed). As for content that doesn't decode, count the + // page so the removal runs and refuses it. if out.unparsed || out.ambiguous { if write { return Err(RedactError::Unreadable(pi + 1)); diff --git a/crates/redact/src/tests/unparsed.rs b/crates/redact/src/tests/unparsed.rs index 4fcc9fcb75..90dcb1ae47 100644 --- a/crates/redact/src/tests/unparsed.rs +++ b/crates/redact/src/tests/unparsed.rs @@ -53,8 +53,9 @@ fn bytes_the_parser_cannot_read_in_a_form_under_a_mark_are_refused() { assert_eq!(apply(&mut doc, None), Err(RedactError::Unreadable(1))); } -/// A form the mark doesn't reach isn't walked: bytes in it can't hide anything under the mark, so -/// the page is redacted. +/// A form the mark doesn't reach isn't walked: bytes in it can't hide anything under the mark, and +/// the walk doesn't refuse the page over them. This one has resources of its own, so the check after +/// the walk doesn't read it either (it would refuse one without), and the page is redacted. #[test] fn bytes_the_parser_cannot_read_in_a_form_away_from_the_marks_are_left() { let form = stream( @@ -102,6 +103,61 @@ fn hidden_text_beside_bytes_the_parser_cannot_read_is_refused() { let mut doc = one_page(content, "", vec![]); assert_eq!(sanitize(&mut doc), Err(RedactError::Unreadable(1)), "{label}"); } + // In a form with resources of its own, which the walk enters on every page. + let form = stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300] /Resources << /Font << /F1 5 0 R >> >>", alone); + let mut doc = one_page(b"/Fm Do", "/XObject << /Fm 7 0 R >>", vec![form]); + assert_eq!(remove_hidden(&mut doc, &[Hidden::HiddenText]), Err(RedactError::Unreadable(1)), "in a form"); +} + +/// Removing hidden layers alone refuses such bytes too, even in a document without +/// `/OCProperties`: a membership dictionary (`/AllOff` of a layer that is on) still hides what it +/// marks, for the walk as for the app's renderer. With nothing in front of `/OC`, the marked text +/// is removed. Here a string beside a stray `}` is left in front of it, which keeps the block from +/// the walk (it reads a `BDC`'s first operands; a string alone does so too, a matter of its own), +/// and the stray byte has the page refused rather than the text kept. +#[test] +fn bytes_the_parser_cannot_read_refuse_removing_hidden_layers_alone() { + let marked = |before: &[u8]| { + let content = [before, b" /OC /MC0 BDC BT /F1 10 Tf 20 260 Td (LAYERSECRET) Tj ET EMC BT /F1 10 Tf 20 20 Td (PUBLIC) Tj ET"].concat(); + one_page( + &content, + "/Properties << /MC0 8 0 R >>", + vec![b"<< /Type /OCG /Name (Layer) >>".to_vec(), b"<< /Type /OCMD /OCGs [7 0 R] /P /AllOff >>".to_vec()], + ) + }; + let mut doc = marked(b""); + let removed = remove_hidden(&mut doc, &[Hidden::HiddenLayers]); + assert!(removed.as_ref().is_ok_and(|r| r.iter().any(|(h, n)| *h == Hidden::HiddenLayers && *n > 0)), "{removed:?}"); + assert!(!content(&doc, 0).contains("LAYERSECRET")); + let mut doc = marked(b"(x) }"); + assert_eq!(remove_hidden(&mut doc, &[Hidden::HiddenLayers]), Err(RedactError::Unreadable(1))); +} + +/// With form fields removed too, a field's appearance is flattened into page content and read the +/// same way, though the scan, which runs before flattening, doesn't count it: hidden text in a +/// broken `TJ` there refuses removing hidden text, and Sanitize Document, which went on and left it. +/// Removing form fields alone doesn't read the content, and goes on. +#[test] +fn hidden_text_beside_bytes_the_parser_cannot_read_in_a_field_appearance_is_refused() { + let fresh = || { + pdf(vec![ + b"<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [6 0 R] >> >>".to_vec(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> /Annots [6 0 R] >>" + .to_vec(), + stream("", b"BT /F1 10 Tf 10 230 Td (Name:) Tj ET"), + FONT.replace("95 0 R", "8 0 R").into_bytes(), + b"<< /Type /Annot /Subtype /Widget /FT /Tx /T (name) /P 3 0 R /Rect [50 225 200 240] /AP << /N 7 0 R >> >>".to_vec(), + stream( + "/Type /XObject /Subtype /Form /BBox [0 0 150 15] /Resources << /Font << /F1 5 0 R >> >>", + b"BT /F1 10 Tf 3 Tr 2 3 Td [(TAIL) ) (HIDDEN)] TJ ET", + ), + widths(), + ]) + }; + assert!(remove_hidden(&mut fresh(), &[Hidden::FormFields]).is_ok()); + assert_eq!(remove_hidden(&mut fresh(), &[Hidden::FormFields, Hidden::HiddenText]), Err(RedactError::Unreadable(1))); + assert_eq!(sanitize(&mut fresh()), Err(RedactError::Unreadable(1))); } /// An inline image whose end the parser guesses, as Apply Redactions already refuses it: what diff --git a/parity/acrobat-features.toml b/parity/acrobat-features.toml index 5f3dc08f35..bbffa91408 100644 --- a/parity/acrobat-features.toml +++ b/parity/acrobat-features.toml @@ -5920,7 +5920,7 @@ status = "shipped" commands = ["redact.apply"] tools = ["redact_apply"] evidence = ["crates/ui-egui/tests/it/redact.rs::marking_applying_and_clearing", "crates/engine/src/tests.rs::redaction_marks_apply_for_good_and_undo", "crates/automation/tests/it/automation.rs::redacting_through_tools", "crates/engine/src/tests.rs::the_renderer_check_refuses_text_still_shown_under_a_mark", "crates/redact/src/tests.rs::redacted_xobjects_do_not_survive_through_inherited_or_shared_resources", "crates/redact/src/tests.rs::redacted_text_does_not_survive_in_info_or_xmp", "crates/redact/src/tests.rs::metadata_is_cleared_when_the_removed_text_was_cut_short", "crates/redact/src/tests/metadata_fail_closed.rs::an_xmp_stream_whose_filter_data_is_corrupt_goes", "crates/redact/src/tests/elsewhere.rs::places_apply_leaves_that_still_hold_the_removed_text_are_counted_and_kept", "crates/engine/src/tests.rs::applied_redactions_report_a_bookmark_that_still_holds_the_removed_text", "crates/automation/tests/it/automation.rs::redact_apply_reports_places_that_still_hold_the_removed_text", "crates/ui-egui/tests/it/redact.rs::applying_says_where_the_removed_text_is_still_held", "crates/redact/src/tests/metadata_matching.rs::a_heading_tracked_with_tj_numbers_matches_the_metadata", "crates/redact/src/tests/metadata_matching.rs::unicode_forms_of_the_removed_text_match", "crates/redact/src/tests/metadata_matching.rs::a_utf16le_string_is_searched_as_utf16le", "crates/redact/src/tests/identification.rs::the_identification_of_a_removed_packet_is_written_back", "crates/automation/tests/it/automation.rs::redact_apply_reports_the_identification_written_back", "crates/ui-egui/tests/it/redact.rs::applying_says_the_pdfa_identification_was_written_back"] -notes = "confirmation; one undo step; next save is a full rewrite. Document information entries that contain the removed text, and the document's and pages' XMP streams that do, are removed (#2448; matched without regard to case or spacing, by the whole run of removed text and by its words of three letters or digits or more, and a value wholly part of a run as whole words, each also compared by letters and digits alone (so spacing, hyphens and TJ tracking don't hide it), after NFKC and lower case, with FF FE strings read as UTF-16LE; fail-closed: all of it goes when the removed text can't be known in full (too much to collect or search for, a glyph whose text isn't known, a form removed unread), and an XMP packet goes when it can't be decoded strictly, is over 16 MiB or isn't UTF-8 or UTF-16). When the document's XMP goes, its PDF/A and PDF/UA identification is written back in a packet holding only that, value by value when each is a valid code that doesn't hold the removed text, else reported lost (Report::identification, the notification, redact_apply's identification); /Info entries that remain aren't mirrored into that packet. Bookmark titles, attachments (names, file names, descriptions), the comments that remain and other objects' XMP are not changed: applying counts those that still hold the removed text, and the notification and redact_apply's still_holding say where (Remove hidden information removes them); when the removed text couldn't be searched for, every such place holding any text is counted. Refused, with nothing changed, when the page still shows text under a mark afterwards: text in some predefined CMaps and rotated Type 3 fonts is not removed yet, so those documents cannot be redacted. Also refused on a page with an inline image the renderer reads no image for, since the operators after it can't be known. Removed images and forms are unlinked from page-tree resources (own, inherited or shared) and from surviving forms' resources, not yet from annotation appearances or patterns" +notes = "confirmation; one undo step; next save is a full rewrite. Document information entries that contain the removed text, and the document's and pages' XMP streams that do, are removed (#2448; matched without regard to case or spacing, by the whole run of removed text and by its words of three letters or digits or more, and a value wholly part of a run as whole words, each also compared by letters and digits alone (so spacing, hyphens and TJ tracking don't hide it), after NFKC and lower case, with FF FE strings read as UTF-16LE; fail-closed: all of it goes when the removed text can't be known in full (too much to collect or search for, a glyph whose text isn't known, a form removed unread), and an XMP packet goes when it can't be decoded strictly, is over 16 MiB or isn't UTF-8 or UTF-16). When the document's XMP goes, its PDF/A and PDF/UA identification is written back in a packet holding only that, value by value when each is a valid code that doesn't hold the removed text, else reported lost (Report::identification, the notification, redact_apply's identification); /Info entries that remain aren't mirrored into that packet. Bookmark titles, attachments (names, file names, descriptions), the comments that remain and other objects' XMP are not changed: applying counts those that still hold the removed text, and the notification and redact_apply's still_holding say where (Remove hidden information removes them); when the removed text couldn't be searched for, every such place holding any text is counted. Refused, with nothing changed, when the page still shows text under a mark afterwards: text in some predefined CMaps and rotated Type 3 fonts is not removed yet, so those documents cannot be redacted. Also refused on a page with an inline image the renderer reads no image for, since the operators after it can't be known, or with bytes the content parser can't read in its content or in a form under a mark, which can hide text other readers draw. Removed images and forms are unlinked from page-tree resources (own, inherited or shared) and from surviving forms' resources, not yet from annotation appearances or patterns" [[feature]] id = "protect.redact-glyph-removal"