The hybrid update test now checks that the update's page and catalog
win and that the repair log says why, and that the linked file keeps
both revisions; the save test checks that the reopened file keeps the
update's page and catalog.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCHD1R3rywQBPCHXZA
Main now reports unreadable objects from the parse of every object in
Document::reconstruct, so the pre-check that skipped that parse is gone.
The single-pass /XRef positions and the rarest-byte find stay.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
i32::MIN is outside Annex C's integer range and stays a real token (-2147483648.).
The generator now stops at -i32::MAX and a separate test pins the i32::MIN output.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
full_save_order retained up to 64 MiB of parsed objects for the write pass, which
reachable_objects then dropped unused. It now asks for no retention.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Overlapping unterminated dictionaries could each be read up to 1 MiB. A shared
budget (the file length) is now taken off on every path; a spent budget keeps the
object a candidate, as before.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The renderer (hayro-syntax (44)) counted the sections its chain reached, and
a hybrid section's /XRefStm stream (ISO 32000-2 §7.5.8.4) counted as one, so
a hybrid update without /Prev was rebuilt by cos but not by the renderer. It
now asks what cos asks: whether the newest section names an earlier one
through /Prev. The /XRefStm stream belongs to the same update.
Tests: a synthetic hybrid fixture (pdfcraft-testkit), with and without
/Prev, in cos and render.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCHD1R3rywQBPCHXZA
An update whose cross-reference section has no /Prev (ISO 32000-2 §7.5.6,
§7.5.8.2) cuts the earlier revisions, and the page tree with them, off the
chain. The signed PDFBOX-2720 and PDFBOX-2942 files have this shape: the
document model found no page tree and the renderer showed 2 blank pages
found by scanning for page dictionaries.
When the chain reaches only the newest section, the file holds another
startxref and the catalog's /Pages can't be read, both readers now rebuild
the table from the file's objects, as for a damaged file:
- cos: the existing reconstruction (newest in-file definition wins), a
repair-log line saying why, and `Document::rebuilt_unlinked_update`.
- hayro-syntax (44): `root_xref` repairs the table before the page tree is
read; `Pdf::rebuilt_unlinked_update` says so.
- render: an open warning, and the lazy structure reader accepts such a
rebuilt file, so links, fields and bookmarks are read too.
A save rewrites the file in full (main's rule for reconstructed files).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyCHD1R3rywQBPCHXZA
- cos: a filter chain whose first stage stops at damage and whose
second reads all it was given is damaged.
- optimize: a page damaged partway stops resampling for the whole
document, so another page drawing the same image whole doesn't get
it resampled.
- redact: after redacting content with /DecodeParms by reference, no
stream in the saved file holds the text, read with its parameters.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
`DecodeBudget::decode_strict` read a stream's `/DecodeParms` only when
direct, so content behind a predictor whose parameters are an indirect
object decoded with the predictor's tag bytes left in. Redaction then found
no glyph under a mark, left the text in the file and reported the mark
applied. ISO 32000-2 lets those parameters be indirect.
`decode_strict` takes the document and reads through `decoding_view`, as
`decode_with_damage` does. Its other readers read the parameters too:
Remove Hidden Information's hidden text and layers, the audit's covered-text
check, and edit's backdrop scan.
The accessibility_mark_tag_artifact description says that content which is,
or may be, in a stream damaged partway is refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
`decode_with_damage` and `decode_contents_with_damage` read a stream's
`/DecodeParms` only when direct, as `Stream::decoded` does, so content behind
a predictor whose parameters, or a number in them, are an indirect object
decoded with the predictor's tag bytes left in, and was taken as whole. Edit
then listed and wrote back the wrong bytes, and pruning dropped an image such
a form draws. ISO 32000-2 lets those values be indirect.
- `decode_with_damage` takes the document and reads the parameters through
it, and so does `decode_contents_with_damage`, and `decode_contents` with
it: printing, accessibility's reading of a page and a page copied as a form
read them too. Code that calls `decode` on a stream itself doesn't yet.
- It reads through `Document::decoding_view`: `with_direct_parms`, but a copy
of the stream holds only its `/Filter` and the parameters, its data shared,
so a stream a page's `/Contents` lists again and again isn't copied whole
for each.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Optimize read a page's content up to any damage in its streams as the whole
of it. Merging a page's image strips then wrote the page's content again as
one stream, without what followed the damage, and the image pass measured
each image by its uses before the damage only, so one drawn larger past it
could be resampled below that use.
- `DecodeBudget::decode_contents_with_damage` reads a page's `/Contents` as
`decode_contents` does, each stream also saying whether it is damaged
partway; `decode_contents` is it without the flag.
- `content_bytes` gives `None` for a page with a damaged stream, as for one
that doesn't decode: the content is not all known. Its strips aren't merged
(`merge_page` already leaves such a page alone), and no image is resampled
(`effective_resolutions` gives every image `INFINITY`).
- A form damaged partway leaves the measuring incomplete in the same way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
A Flate stream damaged partway decodes to what came before the damage, and
`DecodeBudget::decode` gave that back as the stream's data: the decoder's
partial flag was dropped in `Stream::decoded_within_counted`. Editing a text
line, a block, an image or an added item in such a stream wrote it again from
that part, so everything after the damage was lost on save, though other
readers may still read it.
- `DecodeBudget::decode_with_damage` decodes as `decode_within` does and says
whether the stream is damaged partway. A missing checksum or end-of-data
marker is no damage, as before.
- Edit reads the page's streams with it (`ContentStream`), and
`rewritten_contents` refuses, before adding anything, a change to a damaged
stream. Changes in the page's other streams go on, and the damaged one stays
as stored.
- Selecting objects (`editable_objects`, moving added items) refuses a page
with a damaged stream, as it refuses one that doesn't decode: what follows
the damage could change every later item's graphics state.
- Marking a tag as an artifact or a figure as decorative refuses when the
tag's content is in a damaged stream (`ArtifactError::Damaged`,
`AltError::Damaged`), and changes nothing, the structure tree included.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
A CCITTFaxDecode image whose /DecodeParms /Rows exceeds the image's
/Height (PDFBOX-771: /Rows 1084975104 on 2,592 columns) asked the decoder
for /Rows rows; the size check refused that, and every page drew blank.
The image is /Height rows (ISO 32000-2 Table 87), so the decoder is asked
for those; /Rows still counts for a stream decoded without an image's
height. pdfcraft-cos already decodes /Height rows; a test pins it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyDBXygTf7jcqdsjcfn
A writer declared /Predictor 15 on an image and its soft mask without
applying it, so the inflated data is exactly /Height rows of the row length
with no filter-type byte before each (ISO 32000-2 §7.4.4.4 requires one per
PNG-predicted row). Read with the predictor, every row's first sample was
taken for a filter type, the image came out a row short and shifted, and the
page drew it in fragments; image export said its data was shorter than its
size. PDFium, MuPDF and Poppler read such data as the samples it is.
Only that exact length takes the new path, and only for the last Flate
filter of an image whose height is known; anything else is predicted as
before. The renderer (vendored hayro-syntax, patch 40) reports it with
warn!, like its other lenient filter decodes; pdfcraft-filters returns it as
tolerated damage (partial in tolerant decoding, an error in strict), through
the new decode_image_tolerant_within that pdfcraft-cos uses for images.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyDBXygTf7jcqdsjcfn
* Redaction: a pattern nothing paints with any more leaves the page
A tiling pattern's cell is a content stream of its own and can draw
anything, text included. Redacting a shape filled with such a pattern
removed the fill from the page's content -- the page looked right, a
black box -- but left the pattern in /Resources /Pattern, and its cell,
with the text that was under the mark, was written into the saved file
unchanged.
The pass does not enter pattern cells, so it cannot redact inside one.
What it can do is notice that nothing paints with the pattern any more.
The graphics state now carries the fill and stroke pattern a `scn` or
`SCN` selected (and `cs`, `g`, `rg`, `k` and their stroke forms clear
it), every painting operator that survives the rewrite records the
pattern it drew with, and a pattern the page selected but no surviving
paint used leaves the page's resources.
Measured on the report's file:
before applied: 1, (SECRET) in the saved bytes
after applied: 1, no /Pattern on the page, (SECRET) gone
A pattern some surviving paint still uses is kept, and so is its cell:
with two boxes filled from one pattern and only the first marked,
patterns_removed is 0 and the resources keep the entry -- the second box
still shows the word, so it is not a leak.
Report gains patterns_removed.
Fixes#3647.
* Keep a pattern every operator that can paint with it still uses
Two defects CodeRabbit found in the first version of this change, both
of which could remove a pattern the page still draws with.
Pattern use was recorded for path painting only. Text shows with the
fill pattern too -- and with the stroke pattern in the stroking render
modes -- as do image masks and anything a form draws, and a form's
content is interpreted with a graphics state of its own, so a pattern it
inherits is never seen inside it. Content like
/Pattern cs /P0 scn 20 140 100 30 re f BT (KEEP) Tj ET
with the box redacted therefore lost the fill of text still on the page.
Every operator that can paint with the current pattern now records it:
text (except mode 3, which paints nothing, and mode 7, which only
clips), inline images, and image or form XObjects. Only path painting
knows exactly what it drew and whether the paint survived, so the others
are recorded as they are met. That can keep a pattern whose paint was in
fact all redacted, which costs an unused resource entry, while dropping
one something still draws with takes paint off the page.
/Resources /Pattern may be an indirect object, like any value. Reading
the reference itself as a dictionary found none and pruned nothing, so
on such a file the cell -- with the text that was under the mark --
stayed in the saved bytes, which is the leak this change is for. It is
resolved now, and what is written back is a dictionary of this page's
own, so pruning never reaches through a /Pattern dictionary another page
shares.
The controls in the test: without the first fix the surviving text's
pattern is removed (left: 1, right: 0), and without the second the
indirect file prunes nothing (left: 0, right: 1).
* M8: refuse retained unsafe tiling patterns
* M8: guard shared appearances and narrow pattern scans
* M8: update TLS dependencies for new advisories
* Redaction: the comment on prune_patterns said more than the code does
`prune_patterns` drops the `/Pattern` resource entry; the `scn` that selected
it stays in the content, so the page names a pattern its resources no longer
hold. The comment claimed the selector went too (noted in review).
ISO 32000-1 8.6.6.2 makes that operand a key of the `/Pattern` subdictionary,
so the open question is whether a reader stops reading the stream at the stale
name. `a_page_whose_pruned_pattern_is_still_selected_renders` answers it with
the renderer rather than with prose: the redacted page is saved, reopened and
rasterized, and
* it renders with no error and the same 300x300 page box,
* the text written *after* the stale selector is still painted -- which is
exactly what a reader would drop if it gave up at the bad operand,
* the redaction overlay is black over the box the pattern filled.
Control: with the drawing after the selector removed, the pixel assertion fails
("the text after the stale selector is still painted"), so it is reading that
text and not a stray pixel.
The selector is left in place rather than cut out: removing operators is a
rewrite of the content this pass does not otherwise do, and nothing surviving
paints with that colour. `pdfcraft-render` is a dev dependency of this crate
only -- layer 4 on layer 3, which the layering allows.
* WIP: pattern guard depth bound sized from the interpreter's
* Pattern tests: keep the guard's cases reaching the guard now a hit path is removed whole
* Pattern automation tests: keep the pattern in use where they test the guard, since a hit path is now removed whole
* Redaction: a form without resources keeps the page pattern it still paints
A form with no /Resources of its own paints with its caller's patterns, but
its surviving pattern paint was recorded only in the form's own output and
never reached the page. With the page's own fill under the mark removed
whole, prune_patterns then dropped /P0 from the page while the form outside
the mark still painted with it, so that fill lost its paint in the saved file.
Pass the painted names up from a form that inherits its caller's resources.
Only the painted side goes up, so this can keep a pattern, never prune one.
* Redaction: a pruned pattern does not stay named by the page tree
prune_patterns writes only the page's own /Resources, and the guard walked
only pages' effective resources. A /Pages node that still named the pruned
pattern kept its cell reachable from Root, so a full save wrote the cell,
text included, while apply returned Ok. Three shapes did it: a page with no
/Parent inheriting from the root, a /Parent that disagrees with the /Kids
tree, and a root /Resources shadowed by a nearer holder that was retired.
Patterns pruned from a page are now unlinked from page-tree holders no page
inherits from, as unlink_doomed does for XObjects, when every /Parent agrees
with the tree. Whenever a pattern was pruned (or a selected page had any),
the guard also inspects the /Resources of every /Pages node Root reaches
down /Kids and up each page's /Parent chain, so whatever stays reachable is
proven safe or the redaction is refused, leaving the document unchanged.
* Redaction: a Type 3 glyph without resources keeps the page pattern it paints
A Type 3 font with no /Resources looks the names in its glyph descriptions
up in the resources of the page or form showing it, so a glyph outside the
mark can paint with the page's pattern. Text recorded only the fill and
stroke pattern of the graphics state, so with the page's own fill under the
mark removed whole, prune_patterns dropped the pattern and the glyph lost its
paint. Showing such a font now keeps every pattern of the scope, as a form
without /Resources keeps what it paints.
* Redaction: say what text-recorded pattern use keeps
Text, inline images and XObjects record their pattern before the hit test,
so a pattern painted only by redacted text keeps its cell. The guard refuses
a cell with text or images; a vector-only cell stays in the file. The comment
said this cost only an unused resource entry. Pin the behaviour in a test.
* Redaction: the page tree is followed up every /Parent link
A full save writes every node a /Parent link reaches, but the guard followed
/Parent only from pages and stopped at the first node it had already seen,
and unlinking never followed it. A /Pages node reached only through the
/Parent of a /Kids node -- a root /Pages with a /Parent, or an intermediate
node whose /Parent leads out of the tree -- kept naming the pruned pattern,
and its cell, text included, stayed in the saved file while apply returned
Ok.
The guard now follows /Parent from every node it visits, page or /Pages, with
the same seen-set against cycles. Unlinking treats a root with a /Parent as an
inconsistent tree, like a disagreeing link, and leaves the decision to the
guard, which inspects every node those links reach and refuses an unsafe
cell.
* Redaction: a long page tree does not exhaust the pattern guard's budget
The page-tree walk counted every node, pages included, against the guard's
16,384-object budget, so redacting a pattern page of a 17,000-page document
was refused with a limit error where it had applied before. Only the holder
nodes the guard inspects count against that budget now; the walk itself is
bounded at a million nodes, as unlink_doomed bounds the same tree.
* Redaction: a Type 3 font set by an ExtGState or shown in a form keeps the page patterns
The resourceless Type 3 check covered only a font set by Tf in the scope that
paints. A Type 3 font without /Resources set through an ExtGState /Font entry
was not flagged, and a glyph shown inside a form with /Resources of its own
recorded its patterns in the form's scope, which the page never saw. In both
cases the page's own fill under the mark was removed whole, prune_patterns
dropped the pattern, and the glyph outside the mark lost its paint.
An ExtGState font is now checked the same way, and a glyph shown in any
nested form marks the page as keeping every pattern it selected. This can
only keep patterns; whatever is kept is checked by the guard.
* Redaction: a pruned pattern a full save would still write is proven safe or refused
The walks that check retained pattern cells follow the references they know
-- resource categories, the page tree, appearances -- but a full save writes
whatever the trailer reaches by any reference. A direct /Parent dictionary, a
/Kids array on a page, or a stray resource entry (/Shading naming the pattern
stream) kept a pruned cell, text included, in the saved file while apply
returned Ok.
pdfcraft_cos gains reachable_objects, which runs the same breadth-first walk
write_full uses (factored out as full_save_order, so the two cannot drift).
Before apply succeeds, every pruned pattern that walk still reaches is
inspected like a retained cell: vector-only passes, anything else refuses
with the existing pattern error. It runs only when a pattern was pruned and
costs one walk of the reachable objects.
* Redaction: a direct /Parent dictionary is part of the page tree too
A /Parent given as a dictionary rather than a reference is written inline by
a full save, but the page-tree walk followed /Parent only as a reference, and
unlinking took such a tree for consistent. A root /Pages whose direct /Parent
held /Resources naming the pruned pattern kept its cell in the saved file.
The guard now walks a direct /Parent like any node (its /Resources, /Parent
and /Kids), and unlinking counts any /Parent that is not the reference /Kids
came from as an inconsistent tree, leaving the decision to the guard.
* Redaction: page-tree walks queue each node once and borrow /Kids
On a hostile tree whose /Pages nodes share one long /Kids array, the guard's
page-tree walk and pattern unlinking pushed every /Kids entry of every node
before checking whether it had been seen, and copied the array on each
visit. Both now queue only nodes not seen yet and read /Kids in place;
unlinking also stops at the first node reached twice, since its /Parent can
agree with at most one of the nodes above it, and leaves that tree to the
guard.
* Redaction: a Type 3 font with /Resources null is resourceless too
/Resources null is the same as no /Resources, but the check counted any
entry, so such a font's glyph outside the mark lost the page pattern it
painted with. A Type 3 glyph shown by a kept annotation appearance is a known
limit, pinned as an ignored test: appearances are not interpreted for pattern
use, and under ISO 32000 the glyph resolves in the appearance's own resources.
* Redaction: a pruned pattern is recorded along its whole reference chain
A pattern name can lead to a reference-valued object whose value is the cell
(/P0 7 0 R, object 7 being 6 0 R). Pruning recorded only object 7, so the
reachability backstop checked an object nothing reached while a stray
reference kept the cell, object 6, in the saved file. Every object along the
chain is recorded now, bounded like the other walks.
* Pattern tests: pin the file identifier where two saves are compared
main now gives a redacted rewrite a new identifier from fresh entropy, so an
incremental and a full save after Apply differ in /ID alone. The test that
checks they are the same save passes fixed entropy to both.
* Redaction: replies and pop-ups removed on other pages feed the shared-appearance check
Apply checks every page for retained unsafe pattern cells when a removed
annotation's appearance is shared with one that stays on an unselected page.
Only annotations removed under a mark fed that check. Replies and pop-ups
that go with them in step 2b (removed_everywhere), often listed on another
page, did not, so a text-bearing cell in an appearance a kept annotation
shares with them escaped the all-pages check and Apply returned Ok.
Their appearance references now go into removed_appearances too, with the
same limit. Raised by FredApps' review of the rebase.
---------
Co-authored-by: skylar taylor-barrick <hi@skylar.cc>
Judge each object changed after signing by the role it held in the signed
revision, keyed by object number, and compare a slot's accepted generations
across revisions. Adds `Document::accepted_generations` to cos.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The written file differs on Windows, so an identifier hashed from it cannot be
pinned, and main's Windows gate failed. The formula is pinned on fixed bytes.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* cos: give a redacted or sanitized full save a new file identifier
A full save that the document asked for (Document::require_full_save_with_new_id)
now writes a fresh /ID pair, so a redacted or sanitized copy does not claim to be the
file it came from (ISO 32000-2 section 14.4). Ordinary saves are unchanged.
An encrypted file keeps the first string of /ID, because the file key is derived from
it, and gets a new second string. An encrypted file whose /ID is not a pair of strings
is left as it was, as before. SaveOptions::id_entropy lets a caller or a test supply
entropy; with it the new identifier depends only on that entropy and the written bytes.
Without it the identifier mixes the standard library's random hash keys and a
process-wide counter, the same facility the engine uses for new encryption keys.
Checked: the new tests in crates/cos/tests/file_identifier.rs fail on the clean base
(compile errors for the missing API), fail on assertions when only the API is in place,
and fail again when each pinned line is reverted (the regeneration call, the first-string
keep, the flag, the entropy input, and the ordinary-save guard). cargo test -p
pdfcraft-cos passes, and clippy with warnings denied is clean. A probe that wrote 16
ordinary full and incremental saves gave byte-identical files with and without this change.
* redact: give a redacted or sanitized copy a new file identifier
Applying redactions, sanitizing and removing hidden information now ask the cos
layer for a new file identifier (Document::require_full_save_with_new_id), so the
saved copy no longer carries the /ID of its source. Before this change the
previous revision was dropped but the identifier still linked the copy to the
original.
The request sits in remove_hidden itself, because its per-category callers (the
remove-hidden edit and the optimizer's discard option) call it directly, not
through sanitize(). sanitize() asks only on its nothing-hidden path, where
remove_hidden returned before asking: the rewrite is a new file either way.
Checked: the new tests a_redacted_or_sanitized_copy_gets_a_new_identifier and
a_remove_hidden_copy_gets_a_new_identifier fail before this change (the request
is missing after apply, and after remove_hidden) and fail again when the apply
call, the nothing-hidden branch or the remove_hidden request is reverted. cargo
test -p pdfcraft-redact passes with the tests included (152 passed, 1 ignored);
cargo clippy with warnings denied and cargo fmt --check are clean.
* docs(cos): say when a full save regenerates the file identifier
The writing section of the crate README described full saves without saying what
happens to /ID. It now says that an ordinary save keeps the identifier, that a
document which asks for a new one (redaction and sanitizing) gets both strings
regenerated, that SaveOptions::id_entropy supplies entropy for reproducible output,
and that an encrypted file keeps its first string because the key depends on it.
Checked: the text matches the code added in e1f761cc (regenerate_id, new_id_bytes,
require_full_save_with_new_id) and the tests in crates/cos/tests/file_identifier.rs.
No code changed in this commit.
* cos: hash the whole rewritten file into the new identifier
The identifier of a requested rewrite hashed only the first 64 KiB of the written file.
With caller-supplied id_entropy the identifier is deterministic, so two rewrites of
documents with the same length that differ only after byte 65,536 received the same
/ID, and a later edit of a large document could keep its previous /ID.
Every written byte is now hashed. The output is read once: one loop over 4 KiB chunks
feeds both halves of the 16-byte identifier, so the extra cost is one linear pass over
the output. Measured with a probe in the workspace dev profile (opt-level 1): hashing
64 MiB took 0.024 s, about 24 ms per 64 MiB of output. A release build was not measured.
Ordinary saves never reach this code, and their output is unchanged.
Checked: crates/cos/tests/file_identifier.rs gains
the_identifier_depends_on_the_whole_rewritten_file. It fails on the branch head before
this change ("content past 64 KiB must change the identifier", both identifiers equal)
and passes after it; reverting only the chunk loop to the 64 KiB prefix makes it fail
again. The six existing tests in that file pass, and so do the full cos and redact test
suites.
* cos: keep the key valid when an encrypted rewrite has no usable /ID
A requested rewrite of an encrypted file whose /ID was absent, had one entry, or had a
first entry that is not a string wrote no identifier at all, although the document had
asked for a new one.
The reader takes the empty string as the key input for these shapes (Document::first_id),
so the rewrite now writes the empty string (<>) as the first entry and a new second entry.
The key stays valid. A file whose first entry is a string keeps that string, as before.
Ordinary saves and unencrypted rewrites are unchanged.
Checked: an_encrypted_rewrite_without_a_usable_identifier_still_gets_a_new_one in
crates/cos/tests/file_identifier.rs covers the three shapes for RC4-128, AES-128 and
AES-256. After the rewrite each file opens with its password, the first string is empty
and the second is new. On the branch head the test fails with "the rewrite writes a
pair" (left 0, right 2). Reverting only the guard to the head's early return makes it
fail again. Isolated, the one-entry shape fails with left 1, right 2, and the non-string
shape with left Some(0), right Some(16). The other seven tests in the file pass.
* cos: derive the identifier from caller entropy with a fixed hash
With SaveOptions::id_entropy the identifier was built with std's DefaultHasher, whose
algorithm the standard library does not specify, and with write_usize (4 bytes on wasm32,
8 on 64-bit targets) and native-endian integers. The same entropy and the same file could
therefore give different identifiers on different platforms or toolchains, although the
documentation promised the bytes depend only on the entropy and the written file.
The identifier is now FNV-1a (64 bit, the published constants) over fixed-width big-endian
fields and the whole written file: the salt, the lane, id_seed and the length, then the
bytes. With caller entropy the salt is the FNV-1a hash of that entropy. Without entropy it
is still the standard library's random hash of a process counter, so only the salt differs
between the two paths. Every written byte is hashed, in one pass that feeds both lanes.
Cost: this pass is slower than the SipHash it replaces. A temporary probe in the workspace
dev profile (opt-level 1) hashed 64 MiB through both lanes in 0.129 s (about 500 MiB/s),
against 0.024 s before. A release build was not measured. The cost is paid once per
requested rewrite; ordinary saves do not reach this code.
Checked: the golden test fixed_entropy_rewrite in crates/cos/tests/file_identifier.rs pins
the 16 bytes for a fixed entropy and the standard fixture. On the previous hash it fails
(left [64, 145, ...], right [50, 125, ...]). An independent re-implementation of the same
formula in Python, run over the bytes the writer hashed (everything before startxref),
gives the same 16 bytes. The unit test fnv1a_is_the_published_64_bit_algorithm checks the
constants against the published vectors. FNV is not collision-resistant against chosen
content; that matters only if someone could choose a file's identifier, and the identifier
only has to be unique.
* Repair review branch integration
* cos: take fresh identifier entropy from the browser on wasm
Without SaveOptions::id_entropy the salt of a rewritten file's identifier was
RandomState and a process-wide counter. On wasm32-unknown-unknown those hash
keys are fixed at compile time, so two fresh web apps that followed the same
allocation and call sequence reproduced the same salt and the same identifier
for identical output, and a redacted copy stayed linkable to its source.
The default salt now comes from the browser first: crypto.getRandomValues
(Web Crypto, through js_sys, sixteen bytes folded with the same FNV-1a the
identifier uses), or Math.random where that API is missing, and only when
neither is reachable the previous RandomState salt. js-sys and wasm-bindgen
join pdfcraft-cos as wasm32-only dependencies; both are already in the
dependency tree of the web shell, and native builds compile neither. Caller
entropy is untouched: with id_entropy the identifier is still a pure function
of that entropy, the seed and the written bytes.
Checked: separate_fresh_processes_get_different_identifiers re-runs this test
binary as two fresh child processes over the same fixture and fails when the
default salt is made constant (so does the in-process divergence test);
stand_in_for_a_fresh_process is its no-op child half. Both pass with the real
salt. On WebAssembly no browser test harness exists, so the browser path is
covered by the wasm32 cargo check and by construction, not by a test. cargo
check --target wasm32-unknown-unknown -p pdfcraft-cos passes, cargo test -p
pdfcraft-cos and -p pdfcraft-redact pass, and clippy with warnings denied is
clean on both.
* Extract full-save trailer assembly
Assisted-by: OpenAI Codex
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
The empty password opens an EFOpen-only document as no password does,
and the credentials of one opened without the key open no other. An
embedded file may name its EFOpen filter in a /Crypt filter of its own,
with no /EFF; handlers older than crypt filters (V1, V2) don't qualify
(hayro-syntax patch 36 too). Saves without the key refuse what they
would have to encrypt: a removed /Encrypt, or an edited or added
embedded file. The password given after opening unlocks the embedded
files, unless one read as stored has been edited.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyDBXygTf7jcqdsjcfn
A crypt filter whose /AuthEvent is /EFOpen needs its password only when
an embedded file is accessed (ISO 32000-2 7.6.6, Table 25). A document
whose strings and streams are in the clear (/StmF, /StrF /Identity) and
whose encrypting filters are all EFOpen ones, /EFF naming one, now opens
without a password in cos and in the renderer (hayro-syntax patch 36).
Without the password there is no file key: the embedded files' data is
held and saved as stored, and Document::embedded_files_locked and
reads_as_stored say so. Asking for such an attachment's bytes is an
error. A full save that would renumber data whose key mixes in the
object number (RC4, AES-128), or change the document's security, is
refused. The password still opens the document and its attachments.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq7HyDBXygTf7jcqdsjcfn
bflatastic
quincy-jc
Mex
mtlcl410
jonathanwxh-cell
mtlcl410andClaude Opus 5.5
FredApps
vangel76andClaude Sonnet 5.5
FredApps
Grzegorz MartensandClaude Opus 5.5
KenM76andClaude Opus 5.5
Hamza Naveedandskylar taylor-barrick
Martin BallandClaude Opus 5.5
