An operator takes its operands from the end, and the renderer ignores a
surplus before them: `(x) /OC /MC0 BDC` marks a layer's block, and
`(x) /GS1 gs` selects `/GS1`. Redaction read some of them from the front,
so a surplus stood in for them and what they named went unseen. Found in
review of #6882, where Remove Hidden Information left the text of a
hidden layer in `(x) /OC /MC0 BDC (LAYERSECRET) Tj EMC`, which both the
app's renderer and Poppler hide.
In what Apply Redactions and Remove Hidden Information decide:
- the layers walk read a block's tag and property list from the front,
and kept a hidden layer's block;
- the cleanup of `/ActualText`, `/Alt`, `/E` and `/T` in a block whose
paint was removed read (and wrote) the property list from the front,
and kept the name that still read the removed word;
- the tags cleanup read a block's `MCID` from the front, and left the
removed text in its structure element's `/ActualText` and `/Alt`; and a
`TJ`'s array, so a block that still drew was taken for empty and its
structure element lost the content the page still shows;
- the property cleanups read the name of a `BDC` or `DP` property list
from the front, and retired a property list the page still draws with;
- the walk read the ExtGState a `gs` selects from the front, so a font set
only that way was not set, and `cs`/`CS` the same for a pattern space;
- the Type 3 cleanup's reads of `cs`, `gs`, `Tf`, `BDC` and `DP`, which
decide which resources and fonts are still used.
The redaction proof read `Tc`, `Tw`, `Tz`, `TL`, `Ts`, `Tf`, `"`, `TJ` and
`Do` from the front too. `pdfcraft_content::Op` gains `back`, `name_back`
and `num_back`: operand `k` counted from the end.
Tests (`crates/redact/src/tests/operands_from_the_end.rs`): a hidden
layer's block, `/ActualText` on removed paint, a tag's `MCID`, a tagged
`TJ` that still draws, a property list still drawn with, a font from an
ExtGState, the pattern a `/Pattern cs` keeps, and a Type 3 font with the
ExtGState, colour space, property list and font its glyph uses, each
after a surplus operand. In `tests_verify.rs`,
`the_proof_reads_operands_from_the_end` (`Tf`, `TJ`, and `Do` of a form
and of images), `the_proof_reads_the_text_states_operands_from_the_end`
(`Tc`, `Tw`, `Tz`, `TL`, `Ts` and `"`) and
`an_image_a_type3_glyph_draws_after_a_surplus_operand_is_found`. And
`operands_are_counted_from_the_end` in the content crate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
* Fix Matrix::invert rejecting valid matrices with a small scale
Found while testing crates/content/src/lib.rs:732-735 (Matrix::invert)
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Matrix::invert: a matrix whose inverse overflows has none
A subnormal determinant passes the relative check, and dividing by it
overflows (CodeRabbit on #5703).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Redact: refuse Redact marks written in place; remove replies and pop-ups listed on
other pages; delete form fields by object, not name; resolve Do by its last operand
like the renderer; sanitize drops PCAdded and flattens widgets before hidden-content
cleanup. Sign: freed signed objects and edits to an already-signed widget are
modifications. Cos: only reading trusts /Type /XRef to skip encryption.
Conflict in crates/content/src/lib.rs: main ends inline images where the
renderer's tokenizer ends them and counts the images whose end is a guess
(Parsed::ambiguous_images); this branch reads operators through the
streaming Ops iterator that parse collects. The iterator's BI branch takes
main's inline_image result and keeps the ambiguous-image count, which parse
copies into Parsed and Ops::ambiguous_images reports.
Filtered inline image data has no length, so its end is an EI chosen
among candidates, and the content parser chose by a different rule
than the renderer's tokenizer (hayro-syntax): the renderer also takes an
EI with no whitespace before it, and checks a candidate by reading what
follows as operators. With image data such as "7FEI 1 0 0 1 200 0 cm",
the renderer ran a cm that redaction read as image data, so text was
drawn 200 pt from where removal and its verify pass looked.
The parser now takes each inline image's end from the renderer's
tokenizer (already in the tree through pdfcraft-cos), so both read the
same operators after the image; a test runs both on the same fixtures.
Only where that tokenizer reads no image (no EI it accepts, or
parameters it can't read) does the parser fall back to its own rules,
counting the image in Parsed::ambiguous_images, and applying redactions
refuses such a page as unreadable.
content_colour parsed each content stream into a full list of operators
before looking at any of them, so a small compressed stream of millions of
tiny operators became gigabytes of parsed operators in memory. The content
parser's loop is now an iterator (pdfcraft_content::ops), which parse
collects; preflight reads it one operator at a time and stops once gray,
RGB and CMYK are all found.
Pieces::splice copied the unedited bytes between edits by visiting
every piece for every edit, so a page of many content pieces and many
replaced operators cost pieces x edits steps. The copy starts at the
piece holding its first byte and stops past its last one.
When an operand failed to parse, the content parser went back to the
byte after its start and tried again, so a run of n bytes that never
reads as a number (a long run of '+', or digits ending in a letter) was
read again from every one of its bytes: n^2/2 bytes from a tiny
compressed stream. A failed number token longer than 64 bytes is now
skipped whole; shorter ones still recover byte by byte.
crates/content/src/lib.rs: kept #3259's eod argument; image_data_end takes eod and passes it to both end_of_image calls; inline_image tries unfiltered, then declared /L, then image_data_end(filter_eod).
Pieces::join separated a page's content streams with a newline, which ended
a comment left open at the end of one stream. hayro, MuPDF and Poppler join
them with a space, so the comment runs on into the next stream and hides its
first line. Edit text, object_list, the redaction audit, redaction and the
Office/HTML export therefore saw text and boxes that nothing draws, and
redacting unrelated text on the page made the hidden content visible: the
rewritten stream dropped the comment.
- Pieces::join uses a space, so every reader of the joined streams agrees
with rendering.
- The parser records, per piece, whether a top-level comment runs into it
and out of it (boundaries only, not every comment). A redaction rewrite
of a piece ends an incoming comment before its first operator and leaves
an outgoing one open, so hidden content stays hidden.
- Reading for export parses the streams joined, instead of one by one.
- The redaction tag check compares the joined streams the same way.
- Pieces::piece_of uses a binary search.
Use checked sample lengths and row padding before the EI delimiter heuristic.
Cover valid binary terminator-like samples, masks, Indexed, aliases, invalid
layouts and saved classic/packed PDFs.
Co-authored-by: MMexicann <MMexicann@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Follow-up to #1404. An inline image whose parameters can't be read and that has no EI after them ran to the end of the stream as a BI op with no data, and those bytes were not counted. It is now skipped like one with an EI, its bytes counted in Parsed::skipped. A stream that ends inside the parameters (BI /W 1) still keeps its BI op.
When a key in an inline image dictionary was not a name (`BI /W 4 H 4
ID ... EI`), the parser ended the image at the end of the stream and
counted nothing as skipped, so every operator after the image was lost:
text_lines found no text on a page whose text follows such an image.
An image whose parameters can't be read is now skipped up to the EI
that ends it, the skipped bytes are counted, and parsing resumes after
it. Without an EI the image still runs to the end of the stream, as
before.
Adds tests::an_unreadable_inline_image_is_skipped_to_its_end, which
fails before this change.
A page's /Contents array is one content stream in pieces that writers may
split between any two tokens (ISO 32000-2 §7.8.2). Edit text parsed each
piece on its own, so an operator whose operands ended one piece and whose
keyword started the next arrived with no operands: a `[(After) -20 (wards)]`
/ `TJ` split made the line vanish from text_lines, text_paragraphs and
text_edit, and from the Edit text tool.
The joined-parse logic the redaction interpreter used (#153) moves into
pdfcraft-content as `Pieces`: join the pieces, parse them as one stream,
map positions back to pieces, and splice joined byte ranges back into
per-piece bytes (an edit replacing or removing an operator that spans
pieces rewrites every piece it touches; untouched pieces keep their
objects). text_lines now interprets the joined operators, and replace_line
and rewrite_block splice through `Pieces`. Redaction uses the same helper.
Lines and paragraphs are grouped by the piece their first operator starts
in, as before, so pages whose pieces are separate drawings don't merge.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The app, crates, binaries, bundle id, packaging, icons, screenshots and docs are now
PdfCraft / pdfcraft (repo storytold/pdfcraft). On first launch the desktop app moves the
PrintCraft settings and crash-recovery folders to the new name, and both apps read settings
saved under the old storage key, so upgrades keep recent files, preferences and unsaved work.
The signing test fixtures keep their PrintCraft subject names (they are binary).
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
annot, compare, content, cos, export, geom, model, ocr, preflight, print and redact have no
production `unwrap`/`expect`/`panic!`/`unreachable!`/`todo!`/`unimplemented!`. The attribute
keeps it that way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Grzegorz MartensandClaude Opus 5.5
Dylan M. Taylor
aiongg
Ozt-y
skylar taylor-barrick
Marco Arnulfo