Review of the operand reads. Path construction was still read from the
front, in the walk (`construct_path`) and in the proof (`extend_path`):
after a surplus number, `9 100 100 m 9 140 100 l ...` placed the square at
x = 9, so a mark over it removed nothing, and an even surplus placed a
square beside a mark under it, which was removed. `m` and `l` now take
their last two numbers, `c` six, `v` and `y` four, and the proof's `re`
four, as the walk's already did.
The tags cleanup read an `MCID` with `Dict::int`, so one given by
reference in a named property list, or in a structure element's `/MCR`,
was missed and the element kept the removed text in `/ActualText` and
`/Alt`. It is read through the reference now.
Tests: `a_paths_points_after_a_surplus_number_are_where_the_renderer_draws_them`
(lines, and curves by `c`, `v` and `y`),
`an_mcid_given_by_reference_loses_what_redaction_removed` (and an `/MCR` of a
form's stream that stays), and
`the_proof_reads_a_paths_points_from_the_end` (lines, `re`, and each of
`c`, `v` and `y` from a point outside the region).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
An operator takes its operands from the end, and the renderer ignores a
surplus before them: `(x) /OC /MC0 BDC` marks a layer's block, and
`(x) /GS1 gs` selects `/GS1`. Redaction read some of them from the front,
so a surplus stood in for them and what they named went unseen. Found in
review of #6882, where Remove Hidden Information left the text of a
hidden layer in `(x) /OC /MC0 BDC (LAYERSECRET) Tj EMC`, which both the
app's renderer and Poppler hide.
In what Apply Redactions and Remove Hidden Information decide:
- the layers walk read a block's tag and property list from the front,
and kept a hidden layer's block;
- the cleanup of `/ActualText`, `/Alt`, `/E` and `/T` in a block whose
paint was removed read (and wrote) the property list from the front,
and kept the name that still read the removed word;
- the tags cleanup read a block's `MCID` from the front, and left the
removed text in its structure element's `/ActualText` and `/Alt`; and a
`TJ`'s array, so a block that still drew was taken for empty and its
structure element lost the content the page still shows;
- the property cleanups read the name of a `BDC` or `DP` property list
from the front, and retired a property list the page still draws with;
- the walk read the ExtGState a `gs` selects from the front, so a font set
only that way was not set, and `cs`/`CS` the same for a pattern space;
- the Type 3 cleanup's reads of `cs`, `gs`, `Tf`, `BDC` and `DP`, which
decide which resources and fonts are still used.
The redaction proof read `Tc`, `Tw`, `Tz`, `TL`, `Ts`, `Tf`, `"`, `TJ` and
`Do` from the front too. `pdfcraft_content::Op` gains `back`, `name_back`
and `num_back`: operand `k` counted from the end.
Tests (`crates/redact/src/tests/operands_from_the_end.rs`): a hidden
layer's block, `/ActualText` on removed paint, a tag's `MCID`, a tagged
`TJ` that still draws, a property list still drawn with, a font from an
ExtGState, the pattern a `/Pattern cs` keeps, and a Type 3 font with the
ExtGState, colour space, property list and font its glyph uses, each
after a surplus operand. In `tests_verify.rs`,
`the_proof_reads_operands_from_the_end` (`Tf`, `TJ`, and `Do` of a form
and of images), `the_proof_reads_the_text_states_operands_from_the_end`
(`Tc`, `Tw`, `Tz`, `TL`, `Ts` and `"`) and
`an_image_a_type3_glyph_draws_after_a_surplus_operand_is_found`. And
`operands_are_counted_from_the_end` in the content crate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Review of the walk's read of a form's `/Matrix`. Through `Document::resolve`
it followed a reference to a reference, for the matrix and for each entry,
as the app's renderer does; Poppler doesn't, and reads such a matrix as
the identity and such an entry as 0. With
`/Matrix [1 0 0 1 0 8 0 R]`, `8 0 obj 9 0 R` and `9 0 obj 200`, a mark
where Poppler shows the text applied with success and left it in the file,
where `main`, reading any entry by reference as the identity, removed it.
`matrix_of` now follows each reference one hop, and a reference to a
reference fails as a malformed matrix: where the content draws can't be
told, so Apply refuses the page, as for a matrix that isn't six numbers.
The refusal now says "isn't six numbers given directly or through one
reference", and `matrix_of`'s doc names the appearances it reads, not
patterns. The proof, which reads the matrix of forms and appearances
with it, treats such a matrix as malformed too.
Test: `a_form_whose_matrix_goes_through_a_reference_to_a_reference_is_refused`
(an entry and the whole matrix, with a mark where either reader draws the
text).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Review of the walk's read of a form's `/Matrix`.
Removing hidden layers alone never asks where content lands, but a form
whose `/Matrix` isn't six numbers was counted in its scan and refused, in
a document without layers too. Such a form is now walked as it stands
there; Apply Redactions, and Remove Hidden Information with hidden text,
which tells hidden text by where it lands, still refuse it.
`matrix_of` resolved and copied every entry before checking there were
six, and the walk reads it at every `Do`. Measured in review (debug
build), 20,000 draws of a form whose
`/Matrix` is 20,000 entries long took 9 s by reference (16 ms before the
walk read entries) and 12 s written directly (2.6 s before, the array
being copied by `resolve`). It now reads an array written directly in
place, and checks the length before resolving any entry.
Tests: `a_form_whose_matrix_is_not_six_numbers_leaves_removing_hidden_layers_alone`,
and `tests/matrix_work.rs` (a binary of its own), whose
`a_long_form_matrix_costs_no_more_than_a_short_one` times 4,000 draws of
a form with a 4,000-entry `/Matrix` against a 7-entry one, written
directly and by reference.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
The walk read a form's `/Matrix` with `Matrix::from_operands` on the array
as written, so an entry given by reference made it the identity. That is
legal: an array's entries may each be an indirect reference (ISO 32000-2
§7.3.10), and Arlington's `XObjectFormType1` allows the matrix itself to
be one. A form moved onto a mark by such a matrix was then tested against
the wrong place, never walked, and Apply Redactions reported success with
its text still in the file: `/Matrix [1 0 0 1 0 8 0 R]` with `8 0 obj 200`
left SECRET, which Poppler's `pdftotext` places under the mark. Remove
Hidden Information missed text such a matrix moves off the page the same
way. Found in review of the unparsed-bytes refusal.
The walk now reads it with the proof's `matrix_of` (`verify/extract.rs`),
which resolves the array and each entry. A `/Matrix` that isn't six
numbers was the identity too; now where the form draws can't be told, so
Apply refuses the page and the scans count it, as for a form past the
page's budget. `matrix_of` now reads a null `/Matrix`, given directly or
by reference, as the identity: a null entry is no entry (ISO 32000-2
§7.3.9).
Tests (`crates/redact/src/tests/form_matrix.rs`): a form moved onto the
mark by a matrix written direct, as a reference, and with an entry by
reference, redacted; a null matrix, the identity; a matrix that isn't six
numbers, refused, and counted by the verify pass and Remove Hidden
Information's scan; and hidden text a matrix entry by reference moves off
the page, removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
The proof that came with #5557 decodes every stream it sweeps through `Ctx::decode`, which called
`decode_strict` with the stream alone. This PR's `decode_strict` also takes the document, to read
`/DecodeParms` given by reference, so the branch merged with main no longer built. `Ctx::decode`
now takes the document, and each of its callers passes the one it reads, so the proof reads such
a stream as redaction reads page content and forms. Before, it decoded the stream with the
parameters' defaults: a PNG predictor's rows kept their tag bytes, and a copy of the removed text
split by them went unreported (`a_survivor_behind_parameters_given_by_reference_is_found`). The
test helpers' streams are direct, so they pass an empty document.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
* redact: add the independent redaction proof machinery
* redact: harden the proof's paint check and count removed codes
The manifest counts character codes, not raw bytes: a composite font's
codes are several bytes, so the byte length overstated what was cut.
Annotation appearances are checked whatever their storage (a direct
dictionary has no reference of its own to dedupe on), and a stream
shared by several annotations is checked at each annotation's own
placement. The graphics-state stack and the clip holes of one walk are
bounded; deeper or more is unverifiable, never unbounded memory.
Even-odd clips cut holes only where the rule can be told cheaply - all
subpaths axis-aligned, disjoint, few - and anything else keeps its
paint reported, failing closed. The /W widths of a descendant font
cost bounded work, like a CMap.
* redact: settle the survivors a page sweep counted before its work ran out
A page that the search budget stops on left the counts made on earlier
pages unsettled: the survivors were seen but never reported, and the
regions that owned them were never marked. The sweep now stops and
still settles, so a located survivor is reported where it sits (the
page reads unswept either way, and the proof still fails).
* redact: prove the redacted output of every supported encryption
The authenticated-reopen regression now covers all four crypt
algorithms the writer supports, not only AES-128: the caller's
credentials prove the saved output, and empty or wrong credentials
still fail closed.
* redact: answer the review round on the proof
A glyph lying under several regions is now accounted for by every one
of them, so overlapping marks each carry their own share in a shared
proof instead of the first one absorbing the rest and the others
reading as if no text was there.
Each finding is recorded once per region as it is found, so a page
that paints over a region a thousand times neither grows the report
nor the memory with repeats.
The byte search walks the haystack once (the pattern's prefix function
is built once), so a needle of one letter over a haystack full of it
can no longer spend comparisons the search budget never charged for.
A numeric XML character reference may carry any number of leading
zeros; the padded form now decodes like the short one, so a survivor
spelled that way in the XMP is still found.
The /W widths of a descendant font are paid for from the shared work
budget: a form invoked again rebuilds its fonts, and a per-font
counter alone let the same array be paid for many times over.
* redact: sweep the proof's survivors out of every place a reader could look
Plant the removed text where the review listed it and expect the sweep to
report it: a compressed object stream, a page thumbnail, a form field
value, an XFA packet, and an object nothing refers to. A file whose page
content is bigger than the proof may decode for one stream is now shown
to come out unverified on its surfaces and its region, never clean.
* redact: spare the names a redacted word legitimately sits in, and charge failed decodes
A redacted word that is also a structural name (a /Subtype, a /Filter, a
/BaseFont) is structure, not a leftover copy: the name surface is now
swept against the matches the input itself had, and only a surplus of
matches is a survivor. The input's names are counted when the snapshot is
taken, for the region's needles and for strings added later alike.
A decode that fails after doing part of its work paid nothing: the
proof's cumulative decode allowance now charges every attempt, failures
included, so the same bomb cannot be decoded (and fail) again and again
for free.
* redact: count the input's names once each, and say so when the count runs out
The name allowance counted one entry per occurrence, so a document whose
pages repeat the same structural name paid for the same scan again and
again; names are now kept once, with their count. A name search that runs
out of work read as no matches, which would have swept names with none
allowed: that is reported on the text surface now, and the proof fails
rather than pass on an uncounted allowance.
* Bound redaction proof searches across many needles
Assisted-by: OpenAI Codex
* Redaction proof: drop find_all, which nothing calls
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Cargo.lock: put aho-corasick in its sorted place
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Redaction proof: review notes
A word split across a TJ array is looked for whole. Overlapping marks count a
removed copy once. A spent decode budget is a problem, not a raw-only stream.
Short needles stay out of the attachment text search, and an indirect /Decode
counts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
FredApps
Grzegorz MartensandClaude Opus 5.5
