mirror of
https://github.com/storytold/pdfcraft.git
synced 2026-11-06 18:52:36 +00:00
60f60fc0a2
* cos: give a redacted or sanitized full save a new file identifier
A full save that the document asked for (Document::require_full_save_with_new_id)
now writes a fresh /ID pair, so a redacted or sanitized copy does not claim to be the
file it came from (ISO 32000-2 section 14.4). Ordinary saves are unchanged.
An encrypted file keeps the first string of /ID, because the file key is derived from
it, and gets a new second string. An encrypted file whose /ID is not a pair of strings
is left as it was, as before. SaveOptions::id_entropy lets a caller or a test supply
entropy; with it the new identifier depends only on that entropy and the written bytes.
Without it the identifier mixes the standard library's random hash keys and a
process-wide counter, the same facility the engine uses for new encryption keys.
Checked: the new tests in crates/cos/tests/file_identifier.rs fail on the clean base
(compile errors for the missing API), fail on assertions when only the API is in place,
and fail again when each pinned line is reverted (the regeneration call, the first-string
keep, the flag, the entropy input, and the ordinary-save guard). cargo test -p
pdfcraft-cos passes, and clippy with warnings denied is clean. A probe that wrote 16
ordinary full and incremental saves gave byte-identical files with and without this change.
* redact: give a redacted or sanitized copy a new file identifier
Applying redactions, sanitizing and removing hidden information now ask the cos
layer for a new file identifier (Document::require_full_save_with_new_id), so the
saved copy no longer carries the /ID of its source. Before this change the
previous revision was dropped but the identifier still linked the copy to the
original.
The request sits in remove_hidden itself, because its per-category callers (the
remove-hidden edit and the optimizer's discard option) call it directly, not
through sanitize(). sanitize() asks only on its nothing-hidden path, where
remove_hidden returned before asking: the rewrite is a new file either way.
Checked: the new tests a_redacted_or_sanitized_copy_gets_a_new_identifier and
a_remove_hidden_copy_gets_a_new_identifier fail before this change (the request
is missing after apply, and after remove_hidden) and fail again when the apply
call, the nothing-hidden branch or the remove_hidden request is reverted. cargo
test -p pdfcraft-redact passes with the tests included (152 passed, 1 ignored);
cargo clippy with warnings denied and cargo fmt --check are clean.
* docs(cos): say when a full save regenerates the file identifier
The writing section of the crate README described full saves without saying what
happens to /ID. It now says that an ordinary save keeps the identifier, that a
document which asks for a new one (redaction and sanitizing) gets both strings
regenerated, that SaveOptions::id_entropy supplies entropy for reproducible output,
and that an encrypted file keeps its first string because the key depends on it.
Checked: the text matches the code added in e1f761cc (regenerate_id, new_id_bytes,
require_full_save_with_new_id) and the tests in crates/cos/tests/file_identifier.rs.
No code changed in this commit.
* cos: hash the whole rewritten file into the new identifier
The identifier of a requested rewrite hashed only the first 64 KiB of the written file.
With caller-supplied id_entropy the identifier is deterministic, so two rewrites of
documents with the same length that differ only after byte 65,536 received the same
/ID, and a later edit of a large document could keep its previous /ID.
Every written byte is now hashed. The output is read once: one loop over 4 KiB chunks
feeds both halves of the 16-byte identifier, so the extra cost is one linear pass over
the output. Measured with a probe in the workspace dev profile (opt-level 1): hashing
64 MiB took 0.024 s, about 24 ms per 64 MiB of output. A release build was not measured.
Ordinary saves never reach this code, and their output is unchanged.
Checked: crates/cos/tests/file_identifier.rs gains
the_identifier_depends_on_the_whole_rewritten_file. It fails on the branch head before
this change ("content past 64 KiB must change the identifier", both identifiers equal)
and passes after it; reverting only the chunk loop to the 64 KiB prefix makes it fail
again. The six existing tests in that file pass, and so do the full cos and redact test
suites.
* cos: keep the key valid when an encrypted rewrite has no usable /ID
A requested rewrite of an encrypted file whose /ID was absent, had one entry, or had a
first entry that is not a string wrote no identifier at all, although the document had
asked for a new one.
The reader takes the empty string as the key input for these shapes (Document::first_id),
so the rewrite now writes the empty string (<>) as the first entry and a new second entry.
The key stays valid. A file whose first entry is a string keeps that string, as before.
Ordinary saves and unencrypted rewrites are unchanged.
Checked: an_encrypted_rewrite_without_a_usable_identifier_still_gets_a_new_one in
crates/cos/tests/file_identifier.rs covers the three shapes for RC4-128, AES-128 and
AES-256. After the rewrite each file opens with its password, the first string is empty
and the second is new. On the branch head the test fails with "the rewrite writes a
pair" (left 0, right 2). Reverting only the guard to the head's early return makes it
fail again. Isolated, the one-entry shape fails with left 1, right 2, and the non-string
shape with left Some(0), right Some(16). The other seven tests in the file pass.
* cos: derive the identifier from caller entropy with a fixed hash
With SaveOptions::id_entropy the identifier was built with std's DefaultHasher, whose
algorithm the standard library does not specify, and with write_usize (4 bytes on wasm32,
8 on 64-bit targets) and native-endian integers. The same entropy and the same file could
therefore give different identifiers on different platforms or toolchains, although the
documentation promised the bytes depend only on the entropy and the written file.
The identifier is now FNV-1a (64 bit, the published constants) over fixed-width big-endian
fields and the whole written file: the salt, the lane, id_seed and the length, then the
bytes. With caller entropy the salt is the FNV-1a hash of that entropy. Without entropy it
is still the standard library's random hash of a process counter, so only the salt differs
between the two paths. Every written byte is hashed, in one pass that feeds both lanes.
Cost: this pass is slower than the SipHash it replaces. A temporary probe in the workspace
dev profile (opt-level 1) hashed 64 MiB through both lanes in 0.129 s (about 500 MiB/s),
against 0.024 s before. A release build was not measured. The cost is paid once per
requested rewrite; ordinary saves do not reach this code.
Checked: the golden test fixed_entropy_rewrite in crates/cos/tests/file_identifier.rs pins
the 16 bytes for a fixed entropy and the standard fixture. On the previous hash it fails
(left [64, 145, ...], right [50, 125, ...]). An independent re-implementation of the same
formula in Python, run over the bytes the writer hashed (everything before startxref),
gives the same 16 bytes. The unit test fnv1a_is_the_published_64_bit_algorithm checks the
constants against the published vectors. FNV is not collision-resistant against chosen
content; that matters only if someone could choose a file's identifier, and the identifier
only has to be unique.
* Repair review branch integration
* cos: take fresh identifier entropy from the browser on wasm
Without SaveOptions::id_entropy the salt of a rewritten file's identifier was
RandomState and a process-wide counter. On wasm32-unknown-unknown those hash
keys are fixed at compile time, so two fresh web apps that followed the same
allocation and call sequence reproduced the same salt and the same identifier
for identical output, and a redacted copy stayed linkable to its source.
The default salt now comes from the browser first: crypto.getRandomValues
(Web Crypto, through js_sys, sixteen bytes folded with the same FNV-1a the
identifier uses), or Math.random where that API is missing, and only when
neither is reachable the previous RandomState salt. js-sys and wasm-bindgen
join pdfcraft-cos as wasm32-only dependencies; both are already in the
dependency tree of the web shell, and native builds compile neither. Caller
entropy is untouched: with id_entropy the identifier is still a pure function
of that entropy, the seed and the written bytes.
Checked: separate_fresh_processes_get_different_identifiers re-runs this test
binary as two fresh child processes over the same fixture and fails when the
default salt is made constant (so does the in-process divergence test);
stand_in_for_a_fresh_process is its no-op child half. Both pass with the real
salt. On WebAssembly no browser test harness exists, so the browser path is
covered by the wasm32 cargo check and by construction, not by a test. cargo
check --target wasm32-unknown-unknown -p pdfcraft-cos passes, cargo test -p
pdfcraft-cos and -p pdfcraft-redact pass, and clippy with warnings denied is
clean on both.
* Extract full-save trailer assembly
Assisted-by: OpenAI Codex
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>