The run and byte limits were per page, and a glyph's /ToUnicode text can
be 256 UTF-16 units, so a short content stream could keep gigabytes of
removed text across pages. The document now keeps at most 16 MiB; past
that the rest is not kept and the metadata check fails closed, as for any
other text it couldn't keep (CodeRabbit's retained concern on #2539).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyG234xeYkJfr41E6iP
Letters-and-digits matching, NFKC, FF FE strings as UTF-16LE, and the
PDF/A / PDF/UA identification written back or reported lost; what still
isn't done (remaining /Info entries aren't mirrored into the new packet).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
#2539 review: the scrub removes the catalog's XMP packet whole when it holds
the removed text, and with it the file's PDF/A and PDF/UA claims, which
Report didn't say. The identification (pdfaid part, conformance, amd, rev;
pdfuaid part, rev) is now read by namespace (roxmltree, already in the build
through pdfcraft-xfa), each value kept only when it is the short code it must
be and doesn't hold the removed text (one-character codes always), and
written back in a packet holding nothing else, with a fixed pdfuaid extension
schema under PDF/A-1 to -3 (left out, with the PDF/UA claim, if the redacted
words are among its words). Report::identification says, per standard,
None / Kept / Lost; the engine passes it on as
EditNotice::RedactionIdentification, the Apply notification says it after the
count, and redact_apply returns identification { pdfa, pdfua }.
Tests (redact): the_identification_of_a_removed_packet_is_written_back,
a_packet_without_identification_is_only_removed,
an_identification_that_cannot_be_kept_is_reported_lost,
unsearched_text_keeps_only_one_character_codes,
the_extension_schema_is_left_out_when_it_uses_a_redacted_word; engine
applied_redactions_report_the_pdfa_identification_written_back; automation
redact_apply_reports_the_identification_written_back; ui-egui
applying_says_the_pdfa_identification_was_written_back. With the write-back
replaced by "nothing kept", the redact tests fail (Kept expected, None
found); with the notice not pushed, the engine, automation and UI tests fail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
#2539 review: ISO 32000-2 §7.9.2.2.1 defines text strings in PDFDocEncoding,
UTF-16BE (FE FF) and UTF-8 (EF BB BF) only, so an FF FE string was read
through PDFDocEncoding and Cyrillic or CJK text in it could never match.
It is now read as UTF-16LE for the search (fail-closed), in the /Info scrub
and in the bookmark, attachment and comment counts.
Test: a_utf16le_string_is_searched_as_utf16le (failed before: 0 /Info
entries removed; with only the count reverted, bookmarks 0 instead of 1).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
#2539 review: a decomposed "Müller" in a UTF-16BE /Title, a U+FB01 ligature
from a ToUnicode map and full-width letters never matched their plain forms.
Both sides are now normalized to NFKC and lower case (unicode-normalization,
already in the build through pdfcraft-crypt's stringprep, so no new crate).
Normalized text more than four times its input's length (U+FDFA and the like)
isn't searched: it counts as holding the text.
Tests: unicode_forms_of_the_removed_text_match (failed before: Title, Subject
and Keywords kept), a_value_that_normalizes_to_far_more_text_counts_as_holding_it
(failed with the growth limit raised to 100: 0 removed).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
#2539 review: a heading tracked with TJ numbers ([(B) -300 (L) -300 ...] TJ)
was collected as "B L U E F I N", which neither held "Bluefin" nor had a word
of three letters, and a value spelled "Blue fin" or "BLUE-FIN" didn't hold the
term "bluefin". A TJ gap can't be told from a word space, so the glyph ends stay
as they are; instead every term with three letters or digits is also looked
for by its letters and digits alone, and a value is part of a run when its
letters and digits are, starting and ending at the run's word edges ("True"
still isn't part of "untrue"). It over-matches only towards removal.
Tests: a_heading_tracked_with_tj_numbers_matches_the_metadata,
a_value_spaced_or_hyphenated_otherwise_matches (both failed before: /Title
kept, metadata 0).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
Main's `EditNotice` is "something the last edit did that the user should hear
about", set by `Session::apply` and cleared by the next edit, undo and redo,
which is what the PR's separate `redaction_elsewhere` field did by hand. It is
now `EditNotice::RedactionElsewhere(Elsewhere)`, pushed when a place is counted
or the text couldn't be searched for: tools list it in `warnings` with the
other notices, the interface says it through the same path, and
`Document::redaction_elsewhere()` gives `redact_apply` and the Apply dialog the
counts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
Conflict in crates/engine/src/document.rs: main turned `edit_notices` into
`Vec<EditNotice>`; the PR's `redaction_elsewhere` field is kept beside it in
this merge, and the next commit makes it an `EditNotice` variant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
After "Applied N redaction marks…", the same notification names the bookmark titles,
attachments, comments and other metadata streams that still hold the redacted text, and points
to Remove hidden information. New English templates; untranslated text shows in English. The
protect.apply-redactions parity note and evidence say what is reported and what is still left.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
The document keeps the last apply's Report::still_holding as redaction_elsewhere (re-exported as
RedactionElsewhere), cleared by the next edit, undo or redo as edit_notices is. redact_apply
returns it as still_holding with the four counts, their total and unsearched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
The unread-form branch was tested for a form nested past the depth limit only; a form under a
mark whose filter can't be decoded is removed unread too, and now has an end-to-end test, seen
failing with that branch's flag removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
Report::still_holding (Elsewhere) counts the places apply leaves as they are that still hold the
removed text: bookmark titles, attachment names, file names and descriptions, the text and authors
of the comments that remain, and XMP on objects other than the catalog and the pages. Nothing is
changed; Remove hidden information removes them. The text is matched as the metadata check
matches it (one Search type now serves both), with its own work limit. When the removed text
couldn't be searched for, every place holding any text counts and `unsearched` says so. The walks
read each outline item and name-tree node once, within caps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
End-to-end tests for a removed run over 64 KiB, more than 20,000 terms, a run over 4,096
characters, and XMP that is corrupt, cut short, behind two filters, over 16 MiB or in another
encoding. Each was seen failing with its guard removed or loosened.
The corrupt and cut-short cases showed that the tolerant decoder returned the part of the packet
before the damage, which was then searched as if it were the whole packet. XMP is now decoded
strictly, and a packet behind more than one filter goes unread.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
Review on #2539 (nitpick): the README named only the collection limit
and unreadable XMP as fail-closed cases. It now lists all of them (the
run and run-length limits, a removed glyph with unknown text, a form
XObject removed unread past the depth limit or unreadable, the term
limits, the search work limit, oversized or undecodable XMP) and the
metadata the check leaves to Sanitize: XMP on objects other than the
catalog and pages, bookmark titles, and attachments.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
mtlcl410andClaude Opus 5.5
Dylan M. Taylor
FredApps
FredApps
Combiler01
aiongg
Brandon ThomasandClaude Opus 5.5
skylar taylor-barrick
songzhihui