Redaction reads page content and form XObjects strictly (upstream) and
within one page's decode budget (this branch): both now go through
DecodeBudget::decode_strict.
Filtered inline image data has no length, so its end is an EI chosen
among candidates, and the content parser chose by a different rule
than the renderer's tokenizer (hayro-syntax): the renderer also takes an
EI with no whitespace before it, and checks a candidate by reading what
follows as operators. With image data such as "7FEI 1 0 0 1 200 0 cm",
the renderer ran a cm that redaction read as image data, so text was
drawn 200 pt from where removal and its verify pass looked.
The parser now takes each inline image's end from the renderer's
tokenizer (already in the tree through pdfcraft-cos), so both read the
same operators after the image; a test runs both on the same fixtures.
Only where that tokenizer reads no image (no EI it accepts, or
parameters it can't read) does the parser fall back to its own rules,
counting the image in Parsed::ambiguous_images, and applying redactions
refuses such a page as unreadable.
Structure cleanup cleared alternate text, actual text and expansions
only on the element that directly owned changed marked content, and an
element that lost an annotation reference kept its descriptions unless
it lost every kid. An ancestor's /ActualText covering a redacted span,
or a link element's /Alt repeating the address of a removed link, stayed
in the saved file. Elements that lose an annotation now always lose
their descriptions, and every ancestor of an affected element (found
through both /P and the tree's /K, which may disagree) loses its own.
An element written in place above one with a description to clear
refuses the operation, as other in-place cases already do.
Finding which marked content a redaction changed copied every operator
into the buffer of each MCID enclosing it, so content nested n deep was
held n times: a modest stream of nested BDCs could exhaust memory.
Each operator inside marked content is now serialized once, and each
MCID keeps the ranges its sequences span. The bytes compared are capped;
MCIDs past the cap count as changed, so their descriptions are cleared
rather than kept.
Clearing the pixels of a partly covered image decoded its stream with
the global 1 GiB allowance and only then cut it to the raster length its
Width, Height and bit depth declare. A small Flate stream could make
every such image allocate and inflate far more than it can use. Decoding
now stops at twice the raster plus a byte per row (filter-chain and
predictor overhead) and a little slack; an image past that is removed
whole, as an undecodable one already is.
After applying redactions, removed images and forms were unlinked from
page-tree resource dictionaries but not from the resources of forms the
page still draws. A form sharing the page's original resource dictionary
(a common producer layout) kept the removed image reachable, so a full
save wrote it. The walk over surviving forms now drops XObject entries
for removed objects that the form's own content does not draw, and fails
closed when that cannot be determined.
The redaction interpreter started every form XObject with an empty text
state (no font, size 0, no spacing) although a form inherits its caller's
graphics state, and it ignored the /Font entry of an ExtGState set by gs.
Text shown that way was placed at the wrong positions, so removal and
the verify pass could both miss glyphs under a mark. Forms now start
from the caller's state, and gs sets the font and size as Tf does.
The engine's renderer cross-check test used the gs case as its example
of interpreter/renderer disagreement; it now runs the check directly on
an unredacted page, which also covers an input stream labelled as a
redaction overlay.
Both checks for text left under applied redaction marks (redaction's own
verify pass and the engine's renderer cross-check) skipped any page
content stream whose dictionary said /PCMark /Redaction, so that the
overlay's label would not count. An input file could put that label on
its own content stream and keep it out of both checks. Applying
redactions now records the overlay streams it adds (stamp returns the
new stream), and only those are skipped.
Deleting a field took it off its pages and out of the field tree but left
its entry in the form's /CO array. That reference kept the field
dictionary, its value and its widgets reachable, so a full save after
applying redactions over the field still wrote the redacted value.
PAdES LTV writers add the /DSS and, in the same revision, declare the
ESIC developer extension in the catalog's /Extensions (ISO 32000-2
§7.12). Any catalog change outside a short key list was "document
structure", so the signature such an update protects showed as invalid.
/Extensions may now gain declarations after signing; one that was
signed may still not change or go.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A page whose /Annots is an object of its own keeps its dictionary when a
later revision adds a signature widget or a comment: only the array
changes. That array fell through to "other changes", so countersigning
such a document (as many signing services do) turned every earlier
signature invalid, including PdfCraft's own second signature.
A changed array that was a signed page's /Annots is now classified like
an inline /Annots: by the annotations it gained (signature, form fill or
comments), under the same DocMDP rules.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A run of images that is not one picture was searched again from each of
its groups, collecting the rest of the run every time, so many images drawn
over one another made a page quadratic. The search reads at most 16 groups
per content operator (at least 65 536) and leaves the rest of the page as
it is once that is spent.
The pixel reader capped the image size the dictionary gives, then decoded
the JPEG at the size its header gives and compared afterwards, so a small
image dictionary over a huge JPEG allocated the huge one first. The reader
takes the size from the JPEG header and refuses a mismatch before decoding.
The body-size tally and the RTF colour table searched every slot seen so
far for each block, and the RTF writer searched the colour table again for
every paragraph and cell, so a document with many distinct sizes or colours
made export quadratic. Both keep their first-seen order and look slots up
in a hash map.
Line wrapping built and measured the whole candidate line for every word,
so a long field value or annotation text at a tiny font size, with many
words on one line, took quadratic time and copying. The wrapper keeps the
line's width as it goes and measures each word, space and broken
character once.
Code-to-CID and /W range lookups scanned every record in declaration
order, once per glyph and once per explicit ToUnicode entry while a font
was read, so a few thousand records multiplied by as many codes took
billions of steps. Both tables are indexed once into stretches whose
first-declared covering record wins, the precedence the scan had, and the
/W ranges stop at the CMap entry limit.
A sequential bfrange stores one destination and builds every covered
glyph's text from all of it, so one huge destination and many repeats of
its codes made text extraction copy megabytes per glyph. Destinations keep
at most 256 UTF-16 units (the format's 512 bytes), and marked-text
extraction stops asking for a glyph's text once its id has all the text it
keeps.
Figure listing decoded and parsed a page's content once per figure on it,
so many small Figure elements pointing at one page multiplied the work.
The figures are grouped by page and one pass gathers the bounds of every
marked-content id they own; what a sequence draws also counts for the
sequences around it, as before.
Marked-text extraction and figure bounds searched the whole marked-content
stack before every operator, so deeply nested unmarked BMC blocks made a page
quadratic. The text stack now holds each level's effective id (its own, else
the enclosing one's), and the bounds pass counts the open sequences that
match.
add_many added links one at a time, and each addition walked the page tree
and copied and rewrote the page's whole /Annots array, so a batch of links
cost quadratic time. The pages are found once and each page's new links are
appended in one update.
The page walk pushed a node's /Kids array for every new node, so many page
tree nodes sharing one indirect /Kids array re-read it for each node:
quadratic work while the seen check skipped every entry. A shared array
object is read under the first node that lists it, and the walk stops
after four million /Kids entries.
scale_at converted every number of a viewport's /BBox before checking it
had four, and read every viewport the page listed, so many viewports
sharing one long box array took time proportional to their product. A box
of any other length is skipped unread, and at most 1024 viewports are read,
the most set_scale gives a page.
Every Do operator resolved the stream's /XObject resources again, and
resolving a direct dictionary copies it, so a large direct XObject
dictionary and many short Do operators multiplied the copying. The
dictionary is read on the first Do and kept for the rest of the stream.
calculation_order merged the names a Calculate script reads into the
field's other reads by searching the list so far for each name, so a
script naming many fields took quadratic time before the link limits
applied. The merge looks names up in a set.
Word assembly checked whether the whole word so far was underscores for
every glyph it added, so a long run of underscores on one line took
quadratic time. The underscore state of the current word is kept as it
grows.
Office export encoded and kept a separate copy of an image for every place
a page drew it, so a short content stream repeating Do on one large image
multiplied it into gigabytes before the writers' limits applied. Each image
object is now encoded once and its bytes shared (pdfcraft_export::Image
holds an Arc<[u8]>), distinct pictures share a 512 MiB budget whose
overflow lists the page as truncated, and RTF, which leaves pictures out,
prepares none.
Conversion and verification looked only at an action's own /S. An allowed
action (a GoTo, say) whose /Next held a JavaScript or other forbidden
action written directly inside it was kept, and verification never looked
at actions written directly in a catalog /OpenAction or an annotation /A.
Both now follow /Next through the actions written inside an action (an
indirect /Next is an object of its own, checked as one), within a bound,
and conversion removes the whole chain.
verify found each annotation's page by copying and searching every page's
/Annots array again for each annotation, so a page with many annotations
cost quadratic time. The annotation-to-page map is built once (the first
page that lists an annotation keeps it, as before).
content_colour parsed each content stream into a full list of operators
before looking at any of them, so a small compressed stream of millions of
tiny operators became gigabytes of parsed operators in memory. The content
parser's loop is now an iterator (pdfcraft_content::ops), which parse
collects; preflight reads it one operator at a time and stops once gray,
RGB and CMYK are all found.
parse_response compared only the digest algorithm and imprint with the
query, and the token parser dropped the TSTInfo nonce. A genuine token for
the same imprint from another request or under another policy was
accepted. The token keeps its nonce, and a requested nonce or policy must
match.
The FDF field walker and the comment copier followed every reference again
each time it appeared, so a field tree or comment whose kids or properties
repeat one reference level after level expanded a small file into millions
of rows or values before the depth limit stopped it. Both now skip a
reference back to an object they are inside of, and share one budget per
import of values and text; past it the import is refused.
The head and tail attributes were written from the annotation's /LE names
without XML escaping, so a name holding a quote or angle bracket could add
attributes or elements, including field values, to the exported file.
Review on #4111: Delete cleared the selection before the edit ran, so a delete the engine
refuses (text used as a clipping path) left the document as it was but dropped what the user
had selected. The selection now stays until the edit applies; `ObjectSelection::prepare`
already clears it when the document changes and keeps it when an edit is refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
page_form listed every page of the document to take the one it
printed, so imposing P pages walked the page tree and copied its leaf
dictionaries P times over: P^2 page copies. impose passes the list it
already holds. A page index outside it is an error, not a panic.
aiongg
Brandon ThomasandClaude Opus 5.5
Combiler
Osvaldo DuranandClaude Sonnet 5.5
ffishmanandClaude Opus 5.5
HiteshandClaude Opus 5.5
PerchAnton