The run and byte limits were per page, and a glyph's /ToUnicode text can
be 256 UTF-16 units, so a short content stream could keep gigabytes of
removed text across pages. The document now keeps at most 16 MiB; past
that the rest is not kept and the metadata check fails closed, as for any
other text it couldn't keep (CodeRabbit's retained concern on #2539).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyG234xeYkJfr41E6iP
#2539 review: the scrub removes the catalog's XMP packet whole when it holds
the removed text, and with it the file's PDF/A and PDF/UA claims, which
Report didn't say. The identification (pdfaid part, conformance, amd, rev;
pdfuaid part, rev) is now read by namespace (roxmltree, already in the build
through pdfcraft-xfa), each value kept only when it is the short code it must
be and doesn't hold the removed text (one-character codes always), and
written back in a packet holding nothing else, with a fixed pdfuaid extension
schema under PDF/A-1 to -3 (left out, with the PDF/UA claim, if the redacted
words are among its words). Report::identification says, per standard,
None / Kept / Lost; the engine passes it on as
EditNotice::RedactionIdentification, the Apply notification says it after the
count, and redact_apply returns identification { pdfa, pdfua }.
Tests (redact): the_identification_of_a_removed_packet_is_written_back,
a_packet_without_identification_is_only_removed,
an_identification_that_cannot_be_kept_is_reported_lost,
unsearched_text_keeps_only_one_character_codes,
the_extension_schema_is_left_out_when_it_uses_a_redacted_word; engine
applied_redactions_report_the_pdfa_identification_written_back; automation
redact_apply_reports_the_identification_written_back; ui-egui
applying_says_the_pdfa_identification_was_written_back. With the write-back
replaced by "nothing kept", the redact tests fail (Kept expected, None
found); with the notice not pushed, the engine, automation and UI tests fail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
#2539 review: ISO 32000-2 §7.9.2.2.1 defines text strings in PDFDocEncoding,
UTF-16BE (FE FF) and UTF-8 (EF BB BF) only, so an FF FE string was read
through PDFDocEncoding and Cyrillic or CJK text in it could never match.
It is now read as UTF-16LE for the search (fail-closed), in the /Info scrub
and in the bookmark, attachment and comment counts.
Test: a_utf16le_string_is_searched_as_utf16le (failed before: 0 /Info
entries removed; with only the count reverted, bookmarks 0 instead of 1).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
#2539 review: a decomposed "Müller" in a UTF-16BE /Title, a U+FB01 ligature
from a ToUnicode map and full-width letters never matched their plain forms.
Both sides are now normalized to NFKC and lower case (unicode-normalization,
already in the build through pdfcraft-crypt's stringprep, so no new crate).
Normalized text more than four times its input's length (U+FDFA and the like)
isn't searched: it counts as holding the text.
Tests: unicode_forms_of_the_removed_text_match (failed before: Title, Subject
and Keywords kept), a_value_that_normalizes_to_far_more_text_counts_as_holding_it
(failed with the growth limit raised to 100: 0 removed).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
#2539 review: a heading tracked with TJ numbers ([(B) -300 (L) -300 ...] TJ)
was collected as "B L U E F I N", which neither held "Bluefin" nor had a word
of three letters, and a value spelled "Blue fin" or "BLUE-FIN" didn't hold the
term "bluefin". A TJ gap can't be told from a word space, so the glyph ends stay
as they are; instead every term with three letters or digits is also looked
for by its letters and digits alone, and a value is part of a run when its
letters and digits are, starting and ending at the run's word edges ("True"
still isn't part of "untrue"). It over-matches only towards removal.
Tests: a_heading_tracked_with_tj_numbers_matches_the_metadata,
a_value_spaced_or_hyphenated_otherwise_matches (both failed before: /Title
kept, metadata 0).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
Main's `EditNotice` is "something the last edit did that the user should hear
about", set by `Session::apply` and cleared by the next edit, undo and redo,
which is what the PR's separate `redaction_elsewhere` field did by hand. It is
now `EditNotice::RedactionElsewhere(Elsewhere)`, pushed when a place is counted
or the text couldn't be searched for: tools list it in `warnings` with the
other notices, the interface says it through the same path, and
`Document::redaction_elsewhere()` gives `redact_apply` and the Apply dialog the
counts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
Conflict in crates/engine/src/document.rs: main turned `edit_notices` into
`Vec<EditNotice>`; the PR's `redaction_elsewhere` field is kept beside it in
this merge, and the next commit makes it an `EditNotice` variant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
After "Applied N redaction marks…", the same notification names the bookmark titles,
attachments, comments and other metadata streams that still hold the redacted text, and points
to Remove hidden information. New English templates; untranslated text shows in English. The
protect.apply-redactions parity note and evidence say what is reported and what is still left.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
The document keeps the last apply's Report::still_holding as redaction_elsewhere (re-exported as
RedactionElsewhere), cleared by the next edit, undo or redo as edit_notices is. redact_apply
returns it as still_holding with the four counts, their total and unsearched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
The unread-form branch was tested for a form nested past the depth limit only; a form under a
mark whose filter can't be decoded is removed unread too, and now has an end-to-end test, seen
failing with that branch's flag removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
Report::still_holding (Elsewhere) counts the places apply leaves as they are that still hold the
removed text: bookmark titles, attachment names, file names and descriptions, the text and authors
of the comments that remain, and XMP on objects other than the catalog and the pages. Nothing is
changed; Remove hidden information removes them. The text is matched as the metadata check
matches it (one Search type now serves both), with its own work limit. When the removed text
couldn't be searched for, every place holding any text counts and `unsearched` says so. The walks
read each outline item and name-tree node once, within caps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
End-to-end tests for a removed run over 64 KiB, more than 20,000 terms, a run over 4,096
characters, and XMP that is corrupt, cut short, behind two filters, over 16 MiB or in another
encoding. Each was seen failing with its guard removed or loosened.
The corrupt and cut-short cases showed that the tolerant decoder returned the part of the packet
before the damage, which was then searched as if it were the whole packet. XMP is now decoded
strictly, and a packet behind more than one filter goes unread.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
Review on #2539 (nitpick): the README named only the collection limit
and unreadable XMP as fail-closed cases. It now lists all of them (the
run and run-length limits, a removed glyph with unknown text, a form
XObject removed unread past the depth limit or unreadable, the term
limits, the search work limit, oversized or undecodable XMP) and the
metadata the check leaves to Sanitize: XMP on objects other than the
catalog and pages, bookmark titles, and attachments.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCfwzyRXj4VfQAVVdv
Includes #4325#1952#4180#2529#4320#4805, preserving their original histories.
Keep paragraph alignment and hard/soft wrap metadata together; retain CCITT/downsampling/RGB ownership guards with bounded JBIG2 export and exact-copy symbol encoding.
Combined interaction regressions added. Validation pending.
Use a named compact preset arrow beside the field, retain the row alignment/width assertions for the editable control, and open the current colour flyout in the saved-text regression. No accessibility or layout checks are disabled.
Preserve caption editing and current form/Prepare module boundaries. Carry export-only appearance preservation, validation and data round-trip tests to their current paths, and retain current attribution metadata. Hosted CI must validate this refreshed head.
Preserve main's continuous text selection, page tools and module split. Carry only this branch's sidebar-focus feature and regression onto main, without resurrecting old common files from criss-cross merge bases. Hosted CI remains required.
Use the unified Format Text panel and current page-input context instead of restoring old monolithic UI code. Preserve main's pointer routing, editor accessibility and preferences; retain colour, save and input regressions. Hosted CI must validate this refreshed head.
main raised the redaction pass's form depth limit from 12 to 50 (#3645),
so the 16-deep chain was walked and the test's form was no longer removed
unread. The chain is now built from MAX_DEPTH, one level past it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175FWj4xFhMhSH3UpMU8DSC
Review on #2539: a removed glyph with no known text (no /ToUnicode, a
glyph name outside the list) went in as an empty string and nothing marked
the collection as incomplete, so when every removed glyph was like that
the terms were empty and /Info and XMP were left untouched. It now counts
as text the check couldn't keep, and every /Info string and XMP stream goes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review by Losminek on #2539:
- The work cap was checked per value, so very many small /Info or XMP
values could add up without bound. One budget now covers the whole
scrub; past it every further entry or packet counts as matching
(fail-closed). Identical removed runs are searched once.
- unescape looked for ';' through the rest of the text before checking
the 12-byte entity limit, so a packet of bare '&'s was quadratic. It
now looks only at the next 13 characters (200,000 '&': 943 ms -> 5 ms).
- A form XObject removed whole past the depth limit, or when its stream
doesn't decode, never had its text read; both paths now set
removed_cut so the metadata check clears everything instead of
searching incomplete text.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mtlcl410andClaude Opus 5.5
FredApps
Combiler01
FredApps
Dylan M. Taylor
aiongg
songzhihui