PdfCraft's bootstrap renderer draws byte-aligned Group 3 whose fill ends
each end-of-line code on a byte (as TIFF writes it) as other rows than
decode_ccitt reads (black box, at every /K, with /EndOfLine or not:
mostly the first row, then blank). Redaction cleared pixels in the
picture decode_ccitt reads, so a mark over what PdfCraft showed needn't
cover what was cleared, and content shown blank stayed in the copy. Such
an image (/K 0 or more, /EncodedByteAlign) is now removed whole, as
every CCITT image was before this PR; Group 3 off bytes and Group 4 are
still cleared.
The image's parameters come from Document::fax_params, the reading of a
fax image's /DecodeParms entry that decoded_image already did, now on
its own (it was to come with #4404, which uses it too). The redaction
README, the module doc, the redaction and core.image-filters notes and
the spec parity table's status line and revision row say so, and the
core.image-filters note now records the renderer's misdrawing.
group_3_on_bytes_is_removed_whole_and_other_fax_images_cleared: a white
16 × 2 picture, checked to decode as such, is removed whole on bytes (no
codes; codes at /K 0; at /K 1 with /EncodedByteAlign by reference) and
cleared off bytes and as Group 4 on bytes. It fails without the guard;
cargo-mutants catches every viable mutant of it.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Mutation testing found behaviour no test held, now covered:
- a Group 4 halftone above 450 ppi isn't resampled and stays byte for
byte, alone by default and behind ASCIIHexDecode with Retain;
- under Retain a resampled Flate scan stays Flate, though Group 4 would
be smaller (ragged bands);
- sixteen-bit grey of only 0x0000 and 0xFFFF keeps its bytes;
- a busy Group 4 image behind ASCIIHexDecode, whose code is over three
times its raster, decodes within the fax bound and is stored again as
Group 4 alone. Redaction's busy-fax test is that busy again: since the
bound for other images is three times the raster, the old checkerboard
fitted it, and dropping the fax bound passed.
The redaction README says three times, and doc_optimize's description
says what Retain keeps: CCITT stays CCITT, anything else becomes Flate.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
- Redaction decodes an image no further than twice its raster plus a
byte a row, meant for ASCIIHex text under Flate; hex text broken into
lines is a little more, so such an image was removed whole instead of
cleared (already so on main). It is three bytes for each now, as the
optimizer allows.
- Tests: grey stored as hex text in lines under Flate, and RGB behind a
filter array (which a fax image's bound would refuse), are cleared;
decoded_image_within gives a fax image no more rows than its bound
holds.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Redaction clears an image mask's covered samples to the value that
paints nothing, which an inverting /Decode [1 0] flips. It read the
array only when direct, so given by reference (or with its first number
so) the cleared samples painted instead: the covered area came out
inked. That was already so on main, and this PR sends CCITT masks
through it. The array and its first number are resolved now, as export
and the optimizer read them; the fax test covers all three forms.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
- Redacting a CCITT image went through decoded_image, which undoes the
filters before the fax decoder up to MAX_DECODED, so a Flate bomb in
front of a fax image cost a gigabyte while every other image stops at
twice its raster. Document::decoded_image_within takes a bound, and
redaction gives a fax image's code about a byte per pixel. A fax image
cut short, or behind a bomb, is removed whole; a busy one whose code
is bigger than twice its raster is still cleared.
- Every image is now decoded through decoded_image_within, which reads
/DecodeParms through Document::with_direct_parms: given by reference,
the numbers inside (/Predictor 7 0 R) included. That reading was first
written here and has moved down to #4304. The predictor test now covers
the dictionary, an array's entry and the number itself given by
reference; read without them, the cleared bytes drift off the covered
pixel and the covered sample stays in the copy.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Create PDF and the optimizer now store black-and-white scans as Group 4,
and redaction couldn't decode CCITT, so redacting one word on such a
scan removed the whole page image. A CCITT image (the last filter,
after any others) now decodes through Document::decoded_image and is
cleared in a Flate copy like any other image; one that can't be decoded
in full is still removed whole.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stream::decoded_within reads /DecodeParms only when it is direct, so an
image whose PNG predictor was given by reference decoded with its row
tags still in the data. The cleared bytes then drifted one byte further
from the covered pixels each row, and the covered samples survived in
the redacted copy, recoverable by applying the predictor. Parameters
are now made direct first (Document::with_direct_parms).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Upstream moved the interpreter state into a Walk struct; the saved-state
stack now carries (Gs, tm, tlm) there. Re-proved: the redact test fails
with upstream's interp.rs and passes with this one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Review on #4831 pointed out the PDF Association's erratum to ISO 32000-2
(pdf-association/pdf-issues #368): inside a text object, q/Q also save and
restore Tm and Tlm (§9.4.1, referenced from §8.4.2). Acrobat 26.2 agrees on
a synthetic PDF 2.0 page. The renderer on main already did this, so the
previous commit's hayro-interpret patch (67) was wrong: it is reverted here,
with its vendor/README row and the render tests.
The real disagreement was in the redactor's own interpreter, which kept Tm
and Tlm across Q. Find took its boxes from the renderer's text, the redactor
measured the glyphs elsewhere, and redact_apply refused on residue. Its
state stack now holds (Gs, Tm, Tlm) and Q restores all three.
Test: it::redact_text_after_q, end to end on %PDF-2.0 fixtures, three cases
(font change inside q/Q; q 100 0 Td Q; the same followed by T*). With the
redactor change reverted, each case fails on its own at redact_apply. With
only Tm restored, the T* case fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Partial vector redaction previously hid covered areas with clipping
(`clip_out`, `W/W*`) while retaining the complete original drawing instructions
and painting operators in the content stream. This allowed recipients
to recover confidential outlined text or other vector drawings from
supposedly redacted documents.
This change removes vector paths that intersect redaction marks completely
and refuses partial clip retention, ensuring no confidential geometry is
preserved behind clipping. Report counts are updated to record removed paths.
* Record successful PdfCraft web application check
* E21: bound startup requests and validate font and license inputs
* E21: separate pending control completion and gesture regression phases
* M-egui037: enforce control admission and per-family font notices
* M-egui037: qualify capture fixes and document migration behavior
* egui-037: identify current and historical qualification results
content_draws tells whether some content draws an XObject, directly or
through a form that has no /Resources of its own and so draws through
its caller's. Since #4356 the recursion passed the caller's content
again instead of the nested form's decoded data, so a name drawn only
inside such a form was never found.
Both callers then dropped the entry: unlink_doomed for page-tree
resources (Apply Redactions and the content pass of Remove Hidden
Information) and prune_doomed_xobjects for surviving forms. An image
removed under a mark but still shown elsewhere through such a form went
missing from there as well.
The recursion reads the nested form's data again, as it did before
#4356. Two tests cover the page-tree case and the surviving-form case.
Both fail without the fix.
#3670 raised the form depth limit to 50 and added its own 50,000-visit cap. Main's
per-page DecodeBudget (e2a677164) has the same 50,000-form limit, and the visit cap
fired first, so Apply removed the forms instead of refusing the page as main does.
The visit cap goes; the fan-out test expects the refusal, and the depth-limit
sanitization test builds its chain from MAX_DEPTH.
crates/redact/src/interp.rs: kept both sides: main's per-page DecodeBudget and the PR's
visits counter (MAX_FORM_VISITS, same 50,000 as MAX_PAGE_FORMS; the visit check fires first).
* chore: resolve SonarCloud maintainability issues
* fix(packaging): use verify_sha256 helper function
* style: cargo fmt
* fix: restore wildcard imports and fix method-reference type errors
Revert the glob-import expansions (they missed SomIndices and left unused
imports), revert char::is_ascii_digit where the iterator yields char by
value, and pass the content slice to prune_doomed_xobjects as &[Vec<u8>].
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Keep XObject decodability check without binding content in interp
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan Taylor <dylan@craftmaker.dev>
`/Annots` may hold an annotation dictionary inline instead of a reference
(the standard asks for references, but files do it and PdfCraft accepts
them). Deleting a comment or clearing redaction marks only matched
referenced entries, so an inline reply (`/IRT`) or pop-up (`/Parent`) of
what went stayed behind, pointing at nothing. The pop-up an inline reply
or an inline comment names (`/Popup`) stayed too. This predates the shared
walk.
`goes_with` matches an entry against what is going, inline entries
included; `delete_annotation` and `clear_marks` filter with it. The walk
follows the pop-up an inline reply names, and deleting an inline comment
takes the pop-up it names.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Clear all wrote `/Annots []` on a page that had no annotations and replaced
an indirect `/Annots` array with an inline copy. It also left replies to a
mark in place, so their `/IRT` kept the cleared mark in the document (and in
the comment list, as a reply to nothing).
`clear_marks` now writes the list through the annot crate's `set_annots`
(made public), as deleting a comment does, and removes each mark's replies
and pop-ups with it. The "what goes with a deleted comment" walk moves from
`delete_annotation` into `with_replies_and_popups`, which both use; it visits
each annotation once instead of sweeping the list until nothing changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Redact: refuse Redact marks written in place; remove replies and pop-ups listed on
other pages; delete form fields by object, not name; resolve Do by its last operand
like the renderer; sanitize drops PCAdded and flattens widgets before hidden-content
cleanup. Sign: freed signed objects and edits to an already-signed widget are
modifications. Cos: only reading trusts /Type /XRef to skip encryption.
Redaction reads page content and form XObjects strictly (upstream) and
within one page's decode budget (this branch): both now go through
DecodeBudget::decode_strict.
Filtered inline image data has no length, so its end is an EI chosen
among candidates, and the content parser chose by a different rule
than the renderer's tokenizer (hayro-syntax): the renderer also takes an
EI with no whitespace before it, and checks a candidate by reading what
follows as operators. With image data such as "7FEI 1 0 0 1 200 0 cm",
the renderer ran a cm that redaction read as image data, so text was
drawn 200 pt from where removal and its verify pass looked.
The parser now takes each inline image's end from the renderer's
tokenizer (already in the tree through pdfcraft-cos), so both read the
same operators after the image; a test runs both on the same fixtures.
Only where that tokenizer reads no image (no EI it accepts, or
parameters it can't read) does the parser fall back to its own rules,
counting the image in Parsed::ambiguous_images, and applying redactions
refuses such a page as unreadable.
Structure cleanup cleared alternate text, actual text and expansions
only on the element that directly owned changed marked content, and an
element that lost an annotation reference kept its descriptions unless
it lost every kid. An ancestor's /ActualText covering a redacted span,
or a link element's /Alt repeating the address of a removed link, stayed
in the saved file. Elements that lose an annotation now always lose
their descriptions, and every ancestor of an affected element (found
through both /P and the tree's /K, which may disagree) loses its own.
An element written in place above one with a description to clear
refuses the operation, as other in-place cases already do.
Finding which marked content a redaction changed copied every operator
into the buffer of each MCID enclosing it, so content nested n deep was
held n times: a modest stream of nested BDCs could exhaust memory.
Each operator inside marked content is now serialized once, and each
MCID keeps the ranges its sequences span. The bytes compared are capped;
MCIDs past the cap count as changed, so their descriptions are cleared
rather than kept.
Clearing the pixels of a partly covered image decoded its stream with
the global 1 GiB allowance and only then cut it to the raster length its
Width, Height and bit depth declare. A small Flate stream could make
every such image allocate and inflate far more than it can use. Decoding
now stops at twice the raster plus a byte per row (filter-chain and
predictor overhead) and a little slack; an image past that is removed
whole, as an undecodable one already is.
After applying redactions, removed images and forms were unlinked from
page-tree resource dictionaries but not from the resources of forms the
page still draws. A form sharing the page's original resource dictionary
(a common producer layout) kept the removed image reachable, so a full
save wrote it. The walk over surviving forms now drops XObject entries
for removed objects that the form's own content does not draw, and fails
closed when that cannot be determined.
The redaction interpreter started every form XObject with an empty text
state (no font, size 0, no spacing) although a form inherits its caller's
graphics state, and it ignored the /Font entry of an ExtGState set by gs.
Text shown that way was placed at the wrong positions, so removal and
the verify pass could both miss glyphs under a mark. Forms now start
from the caller's state, and gs sets the font and size as Tf does.
The engine's renderer cross-check test used the gs case as its example
of interpreter/renderer disagreement; it now runs the check directly on
an unredacted page, which also covers an input stream labelled as a
redaction overlay.
Both checks for text left under applied redaction marks (redaction's own
verify pass and the engine's renderer cross-check) skipped any page
content stream whose dictionary said /PCMark /Redaction, so that the
overlay's label would not count. An input file could put that label on
its own content stream and keep it out of both checks. Applying
redactions now records the overlay streams it adds (stamp returns the
new stream), and only those are skipped.
Deleting a field took it off its pages and out of the field tree but left
its entry in the form's /CO array. That reference kept the field
dictionary, its value and its widgets reachable, so a full save after
applying redactions over the field still wrote the redacted value.
The name-tree count cloned each node's /Kids array and skipped repeated
references without charging the budget, so a node listing the same kid
many times cost work proportional to every entry while the budget saw one
step. The walk now borrows the array, charges every entry it does not
descend into, and stops once the budget is spent.
Metadata, private application data, associated files and additional
actions were removed from the catalog and pages only. The same entries on
an image or font (XMP), a Form XObject (PieceInfo), a page or annotation
(/AF with its embedded file) or a form field (/AA scripts) survived the
full rewrite, and the scan did not count them, so removing only such a
category reported nothing to remove. A walk of every object reachable from
the catalog now counts and strips these entries wherever they are, leaving
name maps such as resource dictionaries alone (their keys are names a
producer chose), and refuses a graph nested too deeply to walk in full.
Removing hidden text or hidden layers rewrote page and Form content but
left the structure tree alone, unlike applying redactions. An element's
ActualText, Alt or expansion text still described the removed content,
and a marked-content reference into a Form kept the original Form, with
the removed text, in the full rewrite. The hidden-content pass now cleans
the tags of what each page no longer shows and moves Form references to
the rewritten copy, refusing when the tree can't be cleaned completely.
Removing form fields flattens them and then drops the interactive form,
but flattening keeps hidden and no-view widgets (they draw nothing) and
widgets with no appearance. The pages still listed them, so their values
and their parent fields' values survived the full rewrite. Every widget
left after flattening now leaves the pages and the structure tree, inline
annotations included.
Page and Form content was decoded tolerantly, keeping what came before a
corrupt spot and dropping the rest without saying so. A page whose content
broke at its first bytes therefore read as empty: the hidden-text scan
counted nothing, the removal was skipped and the original stream, which
other viewers still draw, stayed in the file. Content is now decoded
strictly; an unreadable page counts as something to remove, so the removal
runs and refuses the page, and a Form read only in part is treated like an
undecodable one.
The hidden-information count followed every /Kids entry of the attachment
and document-script name trees with only a depth limit, so a node listed
twice by each level was read once per path: a few hundred bytes made the
scan take 2^32 steps. Indirect nodes are now read once each, under a total
node budget.
The interpreter entered a form again at every Do with only a nesting
limit, so a few shared forms drawing each other produced exponential
work in apply, sanitize and verify. Scope carries a per-page
DecodeBudget for form entries and form bytes; past it, apply refuses
the page and verification counts the form as possible residue. Page
content streams are decoded against a page budget too.
Review of #3670: with the depth limit at 50, depth was the only bound on the
form walk. A form that draws the next level twice, or a cycle of two forms,
costs 2^depth interpretations — at 12 that was 4096, at 50 it is a hang on a
file anyone can send.
The pass now also counts the forms it enters, and the budget is Edit text's own
`MAX_FORM_VISITS`, referenced rather than copied. Running out is treated as a
form that cannot be read, which is what the depth limit already did: removed in
Apply, the page left alone in Verify.
Measured with the new test:
depth 50, fan-out and a 50-deep cycle, with the budget 5.9 s
depth 22, fan-out, budget removed still running after
5 minutes
A `const` assertion ties this pass's depth limit to Edit text's, so the build
fails rather than a redaction quietly removing what a reader still shows (the
review's third point). Checked by setting the limit back to 12:
error[E0080]: evaluation panicked: assertion failed:
MAX_DEPTH >= pdfcraft_edit::text::MAX_FORM_DEPTH
Tests: a page whose forms each draw the next twice, and the same with the last
form drawing the first again — both must finish and still redact the page.
Grzegorz MartensandClaude Opus 5.5
KenM76andClaude Opus 5.5
Dylan M. Taylor
aiongg
skylar taylor-barrick
oracola91
Kyle Kassen
alehelandClaude Opus 5.5
Hamza