The hand-edited ink test's "CMYK entry not a number" case had five entries,
so the length check refused it before any entry was read; it now has four.
content_update gets its own refusals: two inks at once, an ink out of
range beside another style change, and gray or cmyk on an image, with the
item list unchanged after each.
?control attaches the UI control channel (the one --control serves) and installs window.pdfcraft: request, execute (ui.command), commands (ui.commands), docInspect (ui.state), uiInspect (ui.inspect) and screenshot. The channel keeps the accessibility tree of every frame, so it stays off unless the page asks. Opening, saving, the file name and the unsaved flag stay with the host messages.
Reading an item's /CMYK resolved the value, which copied a direct array,
then collected every entry before checking there were four. A file with a
huge numeric /CMYK paid for that on each content listing and item lookup.
Now a direct value is read where it is, a reference is followed without a
copy, and the array's length is checked before any entry is read. /Gray is
read the same way. A long /CMYK, direct or by reference, and a /Gray that is
a long array still read as no ink.
A stored /CMYK with a non-numeric entry was read from its numbers alone, so
[0 /Bad 0 0 1] became 100 K; a /Gray that isn't a number with a /CMYK beside
it fell through to the CMYK. Both now give no ink, and the text paints in
its /Color.
g and k are written to the thousandth, as rg is, so an ink set or read with
finer digits painted other than what the item, its stored /Gray or /CMYK and
content_list held. The ink is now kept as painted, with its look, when set
and when read back. RGB text is unchanged.
page_add_text and content_update took only RGB, so text added to a CMYK
print file painted 0 0 0 rg, rich black on press. They now also take gray
(0 to 1) or cmyk ([c, m, y, k]), painted with g or k and kept in /PCAdded as
/Gray or /CMYK beside /Color, which holds the matching look. An ink applies
only while the colour still shows its look: picking another colour, also in
the Format panel, drops it for good. RGB text is written byte for byte as
before, and a stored ink that doesn't match its colour is ignored.
* trust certificate key type when CNG algorithm group is unreadable
When the algorithm group can't be read from the key store due to a provider bug, the certificate should not be rejected: its SubjectPublicKeyInfo already states the key type, and a real mismatch will surface at signing time instead of at enumeration.
Change matches_key to fall back to Ok(()) when key.algorithm_group() returns an error, trusting the certificate's key type; extract the group comparison into key_group_matches(group: Option<&AlgorithmGroup>, want: StoreKey) so the fallback and mismatch cases can be unit tested without a live CNG key.
Test: key_group_matches_trusts_when_group_unreadable failed before the change and passes after.
* Sign: back to LF line endings, shorter test comment
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Add PCX image support to Create PDF conversion
PCX is listed as an accepted Create source elsewhere but PdfCraft rejects it, so users cannot convert PCX images to PDF.
Detect PCX by its header in is_image and decode it in embed, adding pcx to CONVERTIBLE so existing from_images and source_kind paths handle it.
Test: pcx_images_become_pages failed before the change and passes after.
* Create: PCX scanlines hold every plane's row, 8-bit indexed images, older versions, standard test files
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Add Pages range field to Compare Files dialog
Users cannot limit a comparison to selected pages, so the dialog needs a validated Pages field that blocks Compare on invalid ranges.
Add a Pages text field under New file, keep its text in pages_text, parse comma-separated numbers and ranges to 0-based pages, show an error and disable Compare for zero, reversed, or out-of-range pages, and leave the comparison itself unchanged.
Test: compare_dialog_pages_field_rejects_out_of_range failed before the change and passes after.
* Compare Files: the Pages field limits the comparison, and says what is wrong with a bad range
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Compare: a change across a page break counts on both pages; a bad range is refused, not read as all pages
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Delete key removes selected content object in edit mode
The reporter says Delete does nothing for a selected textbox or image in edit mode; the only Delete/Backspace handlers in the UI are for organize pages and comments, so a selected content object has no delete path.
Add Delete/Backspace handling to the object selection input in object_ui.rs that deletes the currently selected object(s), and cover it with a unit test that selects an object and presses Delete.
Test: delete_key_removes_selected_object failed before the change and passes after.
* Tests: the Delete key test takes PageToolContext and imports PagePointer
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Redaction: a pattern nothing paints with any more leaves the page
A tiling pattern's cell is a content stream of its own and can draw
anything, text included. Redacting a shape filled with such a pattern
removed the fill from the page's content -- the page looked right, a
black box -- but left the pattern in /Resources /Pattern, and its cell,
with the text that was under the mark, was written into the saved file
unchanged.
The pass does not enter pattern cells, so it cannot redact inside one.
What it can do is notice that nothing paints with the pattern any more.
The graphics state now carries the fill and stroke pattern a `scn` or
`SCN` selected (and `cs`, `g`, `rg`, `k` and their stroke forms clear
it), every painting operator that survives the rewrite records the
pattern it drew with, and a pattern the page selected but no surviving
paint used leaves the page's resources.
Measured on the report's file:
before applied: 1, (SECRET) in the saved bytes
after applied: 1, no /Pattern on the page, (SECRET) gone
A pattern some surviving paint still uses is kept, and so is its cell:
with two boxes filled from one pattern and only the first marked,
patterns_removed is 0 and the resources keep the entry -- the second box
still shows the word, so it is not a leak.
Report gains patterns_removed.
Fixes#3647.
* Keep a pattern every operator that can paint with it still uses
Two defects CodeRabbit found in the first version of this change, both
of which could remove a pattern the page still draws with.
Pattern use was recorded for path painting only. Text shows with the
fill pattern too -- and with the stroke pattern in the stroking render
modes -- as do image masks and anything a form draws, and a form's
content is interpreted with a graphics state of its own, so a pattern it
inherits is never seen inside it. Content like
/Pattern cs /P0 scn 20 140 100 30 re f BT (KEEP) Tj ET
with the box redacted therefore lost the fill of text still on the page.
Every operator that can paint with the current pattern now records it:
text (except mode 3, which paints nothing, and mode 7, which only
clips), inline images, and image or form XObjects. Only path painting
knows exactly what it drew and whether the paint survived, so the others
are recorded as they are met. That can keep a pattern whose paint was in
fact all redacted, which costs an unused resource entry, while dropping
one something still draws with takes paint off the page.
/Resources /Pattern may be an indirect object, like any value. Reading
the reference itself as a dictionary found none and pruned nothing, so
on such a file the cell -- with the text that was under the mark --
stayed in the saved bytes, which is the leak this change is for. It is
resolved now, and what is written back is a dictionary of this page's
own, so pruning never reaches through a /Pattern dictionary another page
shares.
The controls in the test: without the first fix the surviving text's
pattern is removed (left: 1, right: 0), and without the second the
indirect file prunes nothing (left: 0, right: 1).
* M8: refuse retained unsafe tiling patterns
* M8: guard shared appearances and narrow pattern scans
* M8: update TLS dependencies for new advisories
* Redaction: the comment on prune_patterns said more than the code does
`prune_patterns` drops the `/Pattern` resource entry; the `scn` that selected
it stays in the content, so the page names a pattern its resources no longer
hold. The comment claimed the selector went too (noted in review).
ISO 32000-1 8.6.6.2 makes that operand a key of the `/Pattern` subdictionary,
so the open question is whether a reader stops reading the stream at the stale
name. `a_page_whose_pruned_pattern_is_still_selected_renders` answers it with
the renderer rather than with prose: the redacted page is saved, reopened and
rasterized, and
* it renders with no error and the same 300x300 page box,
* the text written *after* the stale selector is still painted -- which is
exactly what a reader would drop if it gave up at the bad operand,
* the redaction overlay is black over the box the pattern filled.
Control: with the drawing after the selector removed, the pixel assertion fails
("the text after the stale selector is still painted"), so it is reading that
text and not a stray pixel.
The selector is left in place rather than cut out: removing operators is a
rewrite of the content this pass does not otherwise do, and nothing surviving
paints with that colour. `pdfcraft-render` is a dev dependency of this crate
only -- layer 4 on layer 3, which the layering allows.
* WIP: pattern guard depth bound sized from the interpreter's
* Pattern tests: keep the guard's cases reaching the guard now a hit path is removed whole
* Pattern automation tests: keep the pattern in use where they test the guard, since a hit path is now removed whole
* Redaction: a form without resources keeps the page pattern it still paints
A form with no /Resources of its own paints with its caller's patterns, but
its surviving pattern paint was recorded only in the form's own output and
never reached the page. With the page's own fill under the mark removed
whole, prune_patterns then dropped /P0 from the page while the form outside
the mark still painted with it, so that fill lost its paint in the saved file.
Pass the painted names up from a form that inherits its caller's resources.
Only the painted side goes up, so this can keep a pattern, never prune one.
* Redaction: a pruned pattern does not stay named by the page tree
prune_patterns writes only the page's own /Resources, and the guard walked
only pages' effective resources. A /Pages node that still named the pruned
pattern kept its cell reachable from Root, so a full save wrote the cell,
text included, while apply returned Ok. Three shapes did it: a page with no
/Parent inheriting from the root, a /Parent that disagrees with the /Kids
tree, and a root /Resources shadowed by a nearer holder that was retired.
Patterns pruned from a page are now unlinked from page-tree holders no page
inherits from, as unlink_doomed does for XObjects, when every /Parent agrees
with the tree. Whenever a pattern was pruned (or a selected page had any),
the guard also inspects the /Resources of every /Pages node Root reaches
down /Kids and up each page's /Parent chain, so whatever stays reachable is
proven safe or the redaction is refused, leaving the document unchanged.
* Redaction: a Type 3 glyph without resources keeps the page pattern it paints
A Type 3 font with no /Resources looks the names in its glyph descriptions
up in the resources of the page or form showing it, so a glyph outside the
mark can paint with the page's pattern. Text recorded only the fill and
stroke pattern of the graphics state, so with the page's own fill under the
mark removed whole, prune_patterns dropped the pattern and the glyph lost its
paint. Showing such a font now keeps every pattern of the scope, as a form
without /Resources keeps what it paints.
* Redaction: say what text-recorded pattern use keeps
Text, inline images and XObjects record their pattern before the hit test,
so a pattern painted only by redacted text keeps its cell. The guard refuses
a cell with text or images; a vector-only cell stays in the file. The comment
said this cost only an unused resource entry. Pin the behaviour in a test.
* Redaction: the page tree is followed up every /Parent link
A full save writes every node a /Parent link reaches, but the guard followed
/Parent only from pages and stopped at the first node it had already seen,
and unlinking never followed it. A /Pages node reached only through the
/Parent of a /Kids node -- a root /Pages with a /Parent, or an intermediate
node whose /Parent leads out of the tree -- kept naming the pruned pattern,
and its cell, text included, stayed in the saved file while apply returned
Ok.
The guard now follows /Parent from every node it visits, page or /Pages, with
the same seen-set against cycles. Unlinking treats a root with a /Parent as an
inconsistent tree, like a disagreeing link, and leaves the decision to the
guard, which inspects every node those links reach and refuses an unsafe
cell.
* Redaction: a long page tree does not exhaust the pattern guard's budget
The page-tree walk counted every node, pages included, against the guard's
16,384-object budget, so redacting a pattern page of a 17,000-page document
was refused with a limit error where it had applied before. Only the holder
nodes the guard inspects count against that budget now; the walk itself is
bounded at a million nodes, as unlink_doomed bounds the same tree.
* Redaction: a Type 3 font set by an ExtGState or shown in a form keeps the page patterns
The resourceless Type 3 check covered only a font set by Tf in the scope that
paints. A Type 3 font without /Resources set through an ExtGState /Font entry
was not flagged, and a glyph shown inside a form with /Resources of its own
recorded its patterns in the form's scope, which the page never saw. In both
cases the page's own fill under the mark was removed whole, prune_patterns
dropped the pattern, and the glyph outside the mark lost its paint.
An ExtGState font is now checked the same way, and a glyph shown in any
nested form marks the page as keeping every pattern it selected. This can
only keep patterns; whatever is kept is checked by the guard.
* Redaction: a pruned pattern a full save would still write is proven safe or refused
The walks that check retained pattern cells follow the references they know
-- resource categories, the page tree, appearances -- but a full save writes
whatever the trailer reaches by any reference. A direct /Parent dictionary, a
/Kids array on a page, or a stray resource entry (/Shading naming the pattern
stream) kept a pruned cell, text included, in the saved file while apply
returned Ok.
pdfcraft_cos gains reachable_objects, which runs the same breadth-first walk
write_full uses (factored out as full_save_order, so the two cannot drift).
Before apply succeeds, every pruned pattern that walk still reaches is
inspected like a retained cell: vector-only passes, anything else refuses
with the existing pattern error. It runs only when a pattern was pruned and
costs one walk of the reachable objects.
* Redaction: a direct /Parent dictionary is part of the page tree too
A /Parent given as a dictionary rather than a reference is written inline by
a full save, but the page-tree walk followed /Parent only as a reference, and
unlinking took such a tree for consistent. A root /Pages whose direct /Parent
held /Resources naming the pruned pattern kept its cell in the saved file.
The guard now walks a direct /Parent like any node (its /Resources, /Parent
and /Kids), and unlinking counts any /Parent that is not the reference /Kids
came from as an inconsistent tree, leaving the decision to the guard.
* Redaction: page-tree walks queue each node once and borrow /Kids
On a hostile tree whose /Pages nodes share one long /Kids array, the guard's
page-tree walk and pattern unlinking pushed every /Kids entry of every node
before checking whether it had been seen, and copied the array on each
visit. Both now queue only nodes not seen yet and read /Kids in place;
unlinking also stops at the first node reached twice, since its /Parent can
agree with at most one of the nodes above it, and leaves that tree to the
guard.
* Redaction: a Type 3 font with /Resources null is resourceless too
/Resources null is the same as no /Resources, but the check counted any
entry, so such a font's glyph outside the mark lost the page pattern it
painted with. A Type 3 glyph shown by a kept annotation appearance is a known
limit, pinned as an ignored test: appearances are not interpreted for pattern
use, and under ISO 32000 the glyph resolves in the appearance's own resources.
* Redaction: a pruned pattern is recorded along its whole reference chain
A pattern name can lead to a reference-valued object whose value is the cell
(/P0 7 0 R, object 7 being 6 0 R). Pruning recorded only object 7, so the
reachability backstop checked an object nothing reached while a stray
reference kept the cell, object 6, in the saved file. Every object along the
chain is recorded now, bounded like the other walks.
* Pattern tests: pin the file identifier where two saves are compared
main now gives a redacted rewrite a new identifier from fresh entropy, so an
incremental and a full save after Apply differ in /ID alone. The test that
checks they are the same save passes fixed entropy to both.
* Redaction: replies and pop-ups removed on other pages feed the shared-appearance check
Apply checks every page for retained unsafe pattern cells when a removed
annotation's appearance is shared with one that stays on an unselected page.
Only annotations removed under a mark fed that check. Replies and pop-ups
that go with them in step 2b (removed_everywhere), often listed on another
page, did not, so a text-bearing cell in an appearance a kept annotation
shares with them escaped the all-pages check and Apply returned Ok.
Their appearance references now go into removed_appearances too, with the
same limit. Raised by FredApps' review of the rebase.
---------
Co-authored-by: skylar taylor-barrick <hi@skylar.cc>
* redact: add the independent redaction proof machinery
* redact: harden the proof's paint check and count removed codes
The manifest counts character codes, not raw bytes: a composite font's
codes are several bytes, so the byte length overstated what was cut.
Annotation appearances are checked whatever their storage (a direct
dictionary has no reference of its own to dedupe on), and a stream
shared by several annotations is checked at each annotation's own
placement. The graphics-state stack and the clip holes of one walk are
bounded; deeper or more is unverifiable, never unbounded memory.
Even-odd clips cut holes only where the rule can be told cheaply - all
subpaths axis-aligned, disjoint, few - and anything else keeps its
paint reported, failing closed. The /W widths of a descendant font
cost bounded work, like a CMap.
* redact: settle the survivors a page sweep counted before its work ran out
A page that the search budget stops on left the counts made on earlier
pages unsettled: the survivors were seen but never reported, and the
regions that owned them were never marked. The sweep now stops and
still settles, so a located survivor is reported where it sits (the
page reads unswept either way, and the proof still fails).
* redact: prove the redacted output of every supported encryption
The authenticated-reopen regression now covers all four crypt
algorithms the writer supports, not only AES-128: the caller's
credentials prove the saved output, and empty or wrong credentials
still fail closed.
* redact: answer the review round on the proof
A glyph lying under several regions is now accounted for by every one
of them, so overlapping marks each carry their own share in a shared
proof instead of the first one absorbing the rest and the others
reading as if no text was there.
Each finding is recorded once per region as it is found, so a page
that paints over a region a thousand times neither grows the report
nor the memory with repeats.
The byte search walks the haystack once (the pattern's prefix function
is built once), so a needle of one letter over a haystack full of it
can no longer spend comparisons the search budget never charged for.
A numeric XML character reference may carry any number of leading
zeros; the padded form now decodes like the short one, so a survivor
spelled that way in the XMP is still found.
The /W widths of a descendant font are paid for from the shared work
budget: a form invoked again rebuilds its fonts, and a per-font
counter alone let the same array be paid for many times over.
* redact: sweep the proof's survivors out of every place a reader could look
Plant the removed text where the review listed it and expect the sweep to
report it: a compressed object stream, a page thumbnail, a form field
value, an XFA packet, and an object nothing refers to. A file whose page
content is bigger than the proof may decode for one stream is now shown
to come out unverified on its surfaces and its region, never clean.
* redact: spare the names a redacted word legitimately sits in, and charge failed decodes
A redacted word that is also a structural name (a /Subtype, a /Filter, a
/BaseFont) is structure, not a leftover copy: the name surface is now
swept against the matches the input itself had, and only a surplus of
matches is a survivor. The input's names are counted when the snapshot is
taken, for the region's needles and for strings added later alike.
A decode that fails after doing part of its work paid nothing: the
proof's cumulative decode allowance now charges every attempt, failures
included, so the same bomb cannot be decoded (and fail) again and again
for free.
* redact: count the input's names once each, and say so when the count runs out
The name allowance counted one entry per occurrence, so a document whose
pages repeat the same structural name paid for the same scan again and
again; names are now kept once, with their count. A name search that runs
out of work read as no matches, which would have swept names with none
allowed: that is reported on the text surface now, and the proof fails
rather than pass on an uncounted allowance.
* Bound redaction proof searches across many needles
Assisted-by: OpenAI Codex
* Redaction proof: drop find_all, which nothing calls
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Cargo.lock: put aho-corasick in its sorted place
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Redaction proof: review notes
A word split across a TJ array is looked for whole. Overlapping marks count a
removed copy once. A spent decode budget is a problem, not a raw-only stream.
Short needles stay out of the attachment text search, and an indirect /Decode
counts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Rebased #5173 onto 9afa1266. Its later-signature predicate is superseded by
#6364's object-based check, so only the /Perms rule lands: under DocMDP P=1 a
/Perms that differs from the signed catalog's, by content, is a structure
change however it was rewritten.
Co-Authored-By: LetoCoder26 <leto@lvanwesten.eu>
Rebased #5187 onto 9afa1266. Landing the signed-role rule only: an existing
object that gains form or annotation keys is judged by what it was when
signed, and a signed annotation turned into another kind is no comment.
The two P1 regressions the original PR carried are not landed: the signature
decision still reads the new /FT (a signed widget relabelled as a text field
is not signing), and the dropped-/Fields check runs before the form-key
comparison (already carried by #6358's lost_entries).
Co-Authored-By: LetoCoder26 <leto@lvanwesten.eu>
Rebased onto 102cae3b/9afa1266 for landing (#6358). The signed page tree's
nodes, a /Kids array of its own, the catalog by object number and an indirect
AcroForm keep the roles they had when signed; a field dropped from the form is
no permitted change at any level; a signed widget's /Rect or /F is page content.
Co-Authored-By: Martin Ball <martin.ball@ucscaregroup.co.uk>
DocMDP P=1 allowed any change classified as a signature. A signature added
after signing now breaks the certification unless it is a document timestamp
drawing nothing (no positive-area /Rect, or Hidden/NoView). The added objects
are judged directly, so a widget with no /T or one only in a page's /Annots is
covered. The document security store and a genuine timestamp still pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An annotation list is compared by the entries that resolve in each revision,
and an unchanged list is no change. Expanding annotation lists is bounded per
classification. A damaged slot rewritten with fewer accepted generations is the
same slot when nothing signed refers to a generation it gave up.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Judge each object changed after signing by the role it held in the signed
revision, keyed by object number, and compare a slot's accepted generations
across revisions. Adds `Document::accepted_generations` to cos.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M10.20: plan Unicode text for OCR output in Type3 faces (fonts, edit)
OCR output used to flatten every character beyond WinAnsi to a silent `?`. This
adds the two pieces that let the OCR layers write the recognized words as they
are; the OCR code that uses them follows in M10.21.
pdfcraft-fonts
- `TextPlan::for_words` maps each word to the standard Helvetica when WinAnsi
covers it, else to Type3 fonts drawn from the craft-fonts faces (the Japanese
document face first, then the rest of the build input). Characters no face
has a glyph for are counted (`missing`), never dropped silently. At most 240
characters per font and 2400 per plan, so one hostile word cannot balloon.
- Which face has which character is asked once per (face, character) for a plan,
not once per word.
- `face_has`, `craft_glyph`, `win_ansi_char`, `fits_win_ansi` are public for it.
pdfcraft-edit
- `add_type3_fonts` / `stamp_with_fonts` register a plan's fonts on a page under
a free `PCOcr`, `PCOcr1`, ... name, build each font once and share the object
with later pages. A shared face is registered under a free name on a later
page that already has a font of the name it was given first, instead of
replacing that page's own font.
- The fonts carry real advances and a ToUnicode CMap, so extraction reads the
text back.
pdfcraft-redact
- A test shows a hand-built Type3 text layer (the shape the OCR output writes) is
redacted like any other text and the verifier passes.
Tests: pdfcraft-fonts 62 passed, pdfcraft-edit 129 passed, pdfcraft-redact 33
passed; clippy -D warnings clean on the three crates. The name-reuse and the
redaction tests build their fonts by hand, so they run without the craft-fonts
build input; the tests that plan real faces skip without it and were not run in
this commit's own check.
* M12: give the hostile-XFA open test more room on a loaded machine
The test opens a form whose calculate script sets a value 50 000 times, against
the one-second budget of an unasked script and a five-second wall-clock bound
on the open. Both sit close to what a machine running several suites at once
can take, so the test can fail without anything being wrong.
It now runs 20 000 iterations (the total is 19999) and allows 30 seconds, which
still fails for a script that hangs or is not capped. Test-only; unrelated to
the OCR work in this series, so it is a commit of its own.
Tests: both hostile_xfa tests in pdfcraft-engine pass.
* M10.21: pluggable OCR engines, preprocessing, review sessions and the OCR tools (ocr, engine, create, automation)
Scan & OCR gets a second engine, optional raster clean-up, a review step before
anything is applied, editable output and a batch mode, headlessly first: the
interface for them follows in M10.22.
pdfcraft-ocr
- A `Recognizer` trait with two engines: the built-in ocrs (moved out of lib.rs)
and the installed tesseract program, run as a fenced external process: fixed
argv, allow-listed language codes, a private temp directory (mode 0700 on
Unix), 120 s timeout, 64 MiB output cap, 50 000-word reading cap. A location
from PDFCRAFT_TESSERACT is validated (absolute, existing, executable) and an
invalid one fails closed instead of falling back to the PATH. `--list-langs`
and `--version` are read from stdout or, for older versions, stderr.
- Strategies: PrimaryOnly, ConfidenceWeighted (the second engine joins when the
first reading's mean is under 70 or unknown) and a ROVER merge (higher
confidence wins per word; against an ocrs word of unknown confidence a
tesseract word wins only at 70 or more; capped at 100 M comparisons; a failing
partner degrades to the primary with a note).
- Preprocessing: opt-in auto-rotate, deskew, denoise and binarize on the RGBA
raster, each with an exact inverse so word boxes map back to the scan. The
300 dpi normalization the steps are calibrated for runs only when a step is
on: with every step off the render is read as rendered, as before. (Enlarging
a 72 dpi render 4x on its own made the models read "due: €250" as "ue: ?250".)
`OcrImage` is validated by construction (side and 64 MP area caps, checked
before any buffer is allocated, including by the resize).
- Confidence bands (High >= 90, Medium 70-89, Low < 70) drive review.
pdfcraft-engine, pdfcraft-create
- `OcrSettings` gains engine, strategy, the four steps, output mode, region, the
skip flags and the low-confidence threshold; Automatic reads confidence
weighted unless a strategy is chosen.
- A recognition is a session: the document guard refuses a stale result with its
reason, pages can be applied one at a time (the guard is taken again after each
applied page), and applying is one undoable step. A per-language dictionary
(a plain-text file: words added from any document stay on disk after it closes).
- Output is searchable (an invisible layer) or a NEW editable document of visible
text at the recognized boxes (`create::from_contents`); words beyond WinAnsi use
the Type3 faces of M10.20 in both, with the shared plan across pages.
- Engine discovery never blocks the interface: results are cached, a stale
answer is shown while a background probe retries, and the models search and
`--version` are cached. Batch: pages or files that already have text are
skipped unless forced, and inputs sharing a file name are found up front.
pdfcraft-automation
- ocr_recognize (engine, strategy, steps, region, output mode), ocr_words (with a
retained review handle), ocr_review_apply, ocr_dictionary, ocr_status and
ocr_recognize_files. The editable `out` must be a new file inside --root and is
refused for a searchable run (it used to be ignored), two batch inputs of one
file name are refused before anything is read, and the batch folder is checked
for existing outputs.
pdfcraft-ui-egui
- Only what the new API needs: the Recognize Text dialog reads with
`recognizers(&settings)`. `ocr.correct` is registered with its label in every
catalog; the screen itself is M10.22.
Tests: pdfcraft-ocr 78 passed; pdfcraft-create passed; pdfcraft-engine and
pdfcraft-automation 450 passed and 1 failed (the automation lib 17, automation.rs
157, hostile_args 2 among them); pdfcraft-ui-egui lib + ocr + control tests 391
passed; clippy -D warnings clean on the five crates. The one failure is in a
test this commit does not change:
recognize_text_makes_a_scanned_page_searchable fails where the models are
installed (they read "quick" as "auick" at that render size); the roadmap on main
already records it. The real tesseract binary ran in
tesseract::tests::recognize_runs_the_installed_binary (tesseract 5.4.0, PATH).
* M10.22: OCR Verify screen, the Recognize Text options and the interface catalogs (ui-egui)
Scan & OCR ▸ Correct recognized text: a full-window screen where a recognition
is reviewed before anything is applied, plus the dialog and batch options for
the engines of M10.21.
OCR Verify screen
- Four panes: the page list, the source page with confidence-coloured word boxes
(click to select, drag to draw a region), the recognized text, and a zoom plus
word inspector. Corrections commit on Enter (an empty commit removes the word),
Remove/Restore, "+ Dictionary" (a per-language file in the preferences folder,
so a word added from a document stays on disk after it closes), per-word and
per-page Verified, F8 / Shift+F8 walk the low-confidence words, Ctrl+T marks the
page reviewed, Ctrl+Tab moves the pane focus, and review edits undo within the
panel.
- Re-OCR of the whole page or a drawn region. A region re-read moves the review
marks of the words it keeps to their new positions and selects the first new
word; a whole-page re-read clears that page's marks (and the undo brings them
back). The region outline is scaled onto the pane, not mapped through user
space again, so it is right on rotated and cropped pages.
- Accept / Reject per page or whole. Pages accepted one at a time each apply once
(the document guard is taken again after each applied page; an edit made
elsewhere in between is still refused with its reason), and Accept in
searchable mode is one undoable step. Editable mode writes a NEW
`<name>_ocr.pdf` through a save dialog and never the source. A real Cancel
keeps the pages already read.
- The engine and strategy menus disable a choice that cannot run and say why;
the check never waits on a process (a background probe fills a cache), and
choosing Automatic moves "Primary only" to Confidence weighted.
Recognize Text dialog and batch
- Engine, strategy, the four clean-up steps and the skip flags are remembered with
the other choices (a saved Automatic without a strategy reads confidence
weighted). The batch refuses two inputs of one file name and existing outputs
before reading anything, notes low-confidence pages at the settings' threshold,
and its toast has a Cancel (the file in flight finishes).
Catalogs and attribution
- The 90 new interface strings are in all 21 catalogs (the Lithuanian ones were
written without native review, and the Bulgarian ones avoid U+045D, which the
bundled monospace face lacks). ATTRIBUTION.toml carries the new hashes, a
note on the Lithuanian entries and the README screenshot; ATTRIBUTION.md is
regenerated.
- README: a screenshot of the Verify screen, taken with
`cargo xtask screenshots` from a synthetic scan (`shot --ocr-verify on`) built
from allowed assets; the page reads it with the real engines.
Tests: pdfcraft-ui-egui lib 373 passed (1 ignored: the Windows hinting
measurement probe), tests ocr 6, control 46, fonts 9; clippy -D warnings clean on
the engine, ui-egui, pdfcraft and xtask packages; `cargo xtask assets` ok. The
review state machine has unit tests for every fix above; each was seen failing
with its fix reverted.
* M10.23: parity entries, roadmap and localization notes for the OCR workflow
Records what M10.20-M10.22 ship, and what they do not.
parity/acrobat-features.toml
- Status changes against main: ocr.rotation-detection and ocr.deskew planned ->
shipped (the opt-in steps of M10.21, with their tests); ocr.correct-recognized-
text and ocr.ocr-engines planned -> partial. Other Area I entries get their
evidence and notes brought up to date.
- ocr.correct-recognized-text: the evidence includes the one-page-at-a-time accept
and the region re-read tests; the notes say what the headless tools cover
(ocr_words with a retained review, ocr_review_apply through the same stale-
document guard, ocr_dictionary) and what is still UI-only (suggestions,
per-word Verified marks, region re-reads).
- ocr.ocr-engines stays partial rather than shipped: part of its evidence is
conditional (the real-binary test runs only where tesseract is installed and
skips elsewhere, and the process-level tests are Unix-only). It cites the tests
that always run (location validation and the fail-closed rule, --list-langs
parsing from stdout or stderr, the merge rules, the default strategy) and says
which part is not proven everywhere.
- `cargo xtask parity`: area I 32.7% -> 44.2% weighted (5 -> 7 shipped, 7 -> 9
partial); overall 52.8% -> 53.0% shipped and 58.2% -> 58.5% weighted.
ROADMAP.md
- One progress-log entry (2026-10-13, newest first) and the banner line. It says
the catalogs are 21 with the Lithuanian additions not natively reviewed, that
the end-to-end redaction test of the Type3 layer needs the craft-fonts build
input, and that recognize_text_makes_a_scanned_page_searchable still fails where
the models are installed, as main's log already records.
docs/localization.md, docs/localization-parity.md
- The Lithuanian entry count (2,578 -> 2,668) and the counts of the other
catalogs that carry entry counts are re-measured with the 90 new entries; the
Lithuanian OCR entries are marked as not natively reviewed.
Checked: `cargo xtask parity` (numbers above, evidence tests exist) and
`cargo xtask assets` pass on this tree.
* M10.24: ocr: apply a late result only to the document it was read from
The Recognize Text dialog closes on Go, so the document can be edited, or have
text redacted, while a long run is still reading. The result was applied
without the stale check the tools and the Verify screen use, so words read
before a redaction could be written back as invisible text. The run now keeps
its guard and applies through the checked path.
Accept with the editable output in the Verify screen had the same gap (Exit
keeps the review open): it now refuses when the document changed since the
review started.
Both new tests fail with their guard switched off.
* M10.25: ocr: ocr_dictionary takes dictionary file names only
Updating sorts, lower-cases and rewrites the file it is given, so pointed at
.env or a README it would corrupt it. The tool now accepts only
ocr-dictionary-<language>.txt style names, for reading as well as writing.
The size-limit test used a file name the rule now refuses first, so it renamed
its fixture to keep testing the size limit. The new test fails with the rule
switched off.
* M10.26: pt-BR: keep (s) on the word counts and say mecanismo for engine
Two review fixes from a native speaker: the two word(s) lines had lost their
(s), so a count of 1 read as 1 palavras, and engine was translated both as
motor and mecanismo. Mecanismo is the Brazilian word for a software engine and
two lines already used it, so the other six now match.
* M10: fit the rebased OCR workflow to current main's structure
* Fix OCR workflow issues from the review
Honor the Recognize dialog's editable-output choice: its result opens in
the review screen (whose Accept writes the editable document) instead of
an invisible layer. Make page acceptance idempotent: a second Accept
entire page never appends the OCR layer twice and keeps later
corrections. Move page and region re-OCR onto a worker thread with a
status line and a cancel that discards the reading. Skip the Tesseract
probe for jobs that name their engines. Bound Tesseract's stderr while
the child runs, not only after it exits. Surface the editable output's
substitution notes in the tools' and the run's results.
Also repair the OCR batch tool against the current structure: inputs are
held to the read root again, and a batch whose inputs share an output
name is refused before the output folder is created, compared the way
the default filesystem compares names; and the editable writer test's
word stays inside the fixture's pages.
* Apply pre-review fixes to the OCR workflow
Hold the automation OCR tools to the document's copying permission, as
the other text tools are: the recognized text is left out of the
results when copying is forbidden, and the paths that carry the words
out of the document (the editable output, the review word list) are
refused. Mark the ocr_dictionary tool as overwriting its path. Close
the searchable layer's graphics state around its invisible text so the
rendering mode cannot leak into later content. Read the OCR user
dictionary folder from the portable data folder in portable mode, like
the recovery store does. Resolve the attribution manifest's leftover
conflict markers and regenerate the generated attribution. Describe the
review flow, the searchable layer's Type3 fonts and the parity evidence
references accurately, and drop an undeclared copy of the recovery
module.
* OCR: reject readings that exceed the word limit
Assisted-by: OpenAI Codex
* OCR: new documents go through the page tree builder main added
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* OCR: review notes and lints
The editable output refuses a file created during recognition. ocr_words takes
skip_text_pages and force_ocr. Automatic OCR checks the engine it will use. The
stderr test stays below the kill cap, and a failed --ocr-verify shot exits with
an error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* OCR: restore the Verify screenshot attribution, follow the moved constants
The merge dropped the screenshot row with the translation rows. The model
filename constants live in ocrs.rs now, so the license check reads them there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Attribution: the Verify screenshot entry goes after the header comment
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Attribution: the screenshot row in the table follows the manifest order
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The written file differs on Windows, so an identifier hashed from it cannot be
pinned, and main's Windows gate failed. The formula is pinned on fixed bytes.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* cos: give a redacted or sanitized full save a new file identifier
A full save that the document asked for (Document::require_full_save_with_new_id)
now writes a fresh /ID pair, so a redacted or sanitized copy does not claim to be the
file it came from (ISO 32000-2 section 14.4). Ordinary saves are unchanged.
An encrypted file keeps the first string of /ID, because the file key is derived from
it, and gets a new second string. An encrypted file whose /ID is not a pair of strings
is left as it was, as before. SaveOptions::id_entropy lets a caller or a test supply
entropy; with it the new identifier depends only on that entropy and the written bytes.
Without it the identifier mixes the standard library's random hash keys and a
process-wide counter, the same facility the engine uses for new encryption keys.
Checked: the new tests in crates/cos/tests/file_identifier.rs fail on the clean base
(compile errors for the missing API), fail on assertions when only the API is in place,
and fail again when each pinned line is reverted (the regeneration call, the first-string
keep, the flag, the entropy input, and the ordinary-save guard). cargo test -p
pdfcraft-cos passes, and clippy with warnings denied is clean. A probe that wrote 16
ordinary full and incremental saves gave byte-identical files with and without this change.
* redact: give a redacted or sanitized copy a new file identifier
Applying redactions, sanitizing and removing hidden information now ask the cos
layer for a new file identifier (Document::require_full_save_with_new_id), so the
saved copy no longer carries the /ID of its source. Before this change the
previous revision was dropped but the identifier still linked the copy to the
original.
The request sits in remove_hidden itself, because its per-category callers (the
remove-hidden edit and the optimizer's discard option) call it directly, not
through sanitize(). sanitize() asks only on its nothing-hidden path, where
remove_hidden returned before asking: the rewrite is a new file either way.
Checked: the new tests a_redacted_or_sanitized_copy_gets_a_new_identifier and
a_remove_hidden_copy_gets_a_new_identifier fail before this change (the request
is missing after apply, and after remove_hidden) and fail again when the apply
call, the nothing-hidden branch or the remove_hidden request is reverted. cargo
test -p pdfcraft-redact passes with the tests included (152 passed, 1 ignored);
cargo clippy with warnings denied and cargo fmt --check are clean.
* docs(cos): say when a full save regenerates the file identifier
The writing section of the crate README described full saves without saying what
happens to /ID. It now says that an ordinary save keeps the identifier, that a
document which asks for a new one (redaction and sanitizing) gets both strings
regenerated, that SaveOptions::id_entropy supplies entropy for reproducible output,
and that an encrypted file keeps its first string because the key depends on it.
Checked: the text matches the code added in e1f761cc (regenerate_id, new_id_bytes,
require_full_save_with_new_id) and the tests in crates/cos/tests/file_identifier.rs.
No code changed in this commit.
* cos: hash the whole rewritten file into the new identifier
The identifier of a requested rewrite hashed only the first 64 KiB of the written file.
With caller-supplied id_entropy the identifier is deterministic, so two rewrites of
documents with the same length that differ only after byte 65,536 received the same
/ID, and a later edit of a large document could keep its previous /ID.
Every written byte is now hashed. The output is read once: one loop over 4 KiB chunks
feeds both halves of the 16-byte identifier, so the extra cost is one linear pass over
the output. Measured with a probe in the workspace dev profile (opt-level 1): hashing
64 MiB took 0.024 s, about 24 ms per 64 MiB of output. A release build was not measured.
Ordinary saves never reach this code, and their output is unchanged.
Checked: crates/cos/tests/file_identifier.rs gains
the_identifier_depends_on_the_whole_rewritten_file. It fails on the branch head before
this change ("content past 64 KiB must change the identifier", both identifiers equal)
and passes after it; reverting only the chunk loop to the 64 KiB prefix makes it fail
again. The six existing tests in that file pass, and so do the full cos and redact test
suites.
* cos: keep the key valid when an encrypted rewrite has no usable /ID
A requested rewrite of an encrypted file whose /ID was absent, had one entry, or had a
first entry that is not a string wrote no identifier at all, although the document had
asked for a new one.
The reader takes the empty string as the key input for these shapes (Document::first_id),
so the rewrite now writes the empty string (<>) as the first entry and a new second entry.
The key stays valid. A file whose first entry is a string keeps that string, as before.
Ordinary saves and unencrypted rewrites are unchanged.
Checked: an_encrypted_rewrite_without_a_usable_identifier_still_gets_a_new_one in
crates/cos/tests/file_identifier.rs covers the three shapes for RC4-128, AES-128 and
AES-256. After the rewrite each file opens with its password, the first string is empty
and the second is new. On the branch head the test fails with "the rewrite writes a
pair" (left 0, right 2). Reverting only the guard to the head's early return makes it
fail again. Isolated, the one-entry shape fails with left 1, right 2, and the non-string
shape with left Some(0), right Some(16). The other seven tests in the file pass.
* cos: derive the identifier from caller entropy with a fixed hash
With SaveOptions::id_entropy the identifier was built with std's DefaultHasher, whose
algorithm the standard library does not specify, and with write_usize (4 bytes on wasm32,
8 on 64-bit targets) and native-endian integers. The same entropy and the same file could
therefore give different identifiers on different platforms or toolchains, although the
documentation promised the bytes depend only on the entropy and the written file.
The identifier is now FNV-1a (64 bit, the published constants) over fixed-width big-endian
fields and the whole written file: the salt, the lane, id_seed and the length, then the
bytes. With caller entropy the salt is the FNV-1a hash of that entropy. Without entropy it
is still the standard library's random hash of a process counter, so only the salt differs
between the two paths. Every written byte is hashed, in one pass that feeds both lanes.
Cost: this pass is slower than the SipHash it replaces. A temporary probe in the workspace
dev profile (opt-level 1) hashed 64 MiB through both lanes in 0.129 s (about 500 MiB/s),
against 0.024 s before. A release build was not measured. The cost is paid once per
requested rewrite; ordinary saves do not reach this code.
Checked: the golden test fixed_entropy_rewrite in crates/cos/tests/file_identifier.rs pins
the 16 bytes for a fixed entropy and the standard fixture. On the previous hash it fails
(left [64, 145, ...], right [50, 125, ...]). An independent re-implementation of the same
formula in Python, run over the bytes the writer hashed (everything before startxref),
gives the same 16 bytes. The unit test fnv1a_is_the_published_64_bit_algorithm checks the
constants against the published vectors. FNV is not collision-resistant against chosen
content; that matters only if someone could choose a file's identifier, and the identifier
only has to be unique.
* Repair review branch integration
* cos: take fresh identifier entropy from the browser on wasm
Without SaveOptions::id_entropy the salt of a rewritten file's identifier was
RandomState and a process-wide counter. On wasm32-unknown-unknown those hash
keys are fixed at compile time, so two fresh web apps that followed the same
allocation and call sequence reproduced the same salt and the same identifier
for identical output, and a redacted copy stayed linkable to its source.
The default salt now comes from the browser first: crypto.getRandomValues
(Web Crypto, through js_sys, sixteen bytes folded with the same FNV-1a the
identifier uses), or Math.random where that API is missing, and only when
neither is reachable the previous RandomState salt. js-sys and wasm-bindgen
join pdfcraft-cos as wasm32-only dependencies; both are already in the
dependency tree of the web shell, and native builds compile neither. Caller
entropy is untouched: with id_entropy the identifier is still a pure function
of that entropy, the seed and the written bytes.
Checked: separate_fresh_processes_get_different_identifiers re-runs this test
binary as two fresh child processes over the same fixture and fails when the
default salt is made constant (so does the in-process divergence test);
stand_in_for_a_fresh_process is its no-op child half. Both pass with the real
salt. On WebAssembly no browser test harness exists, so the browser path is
covered by the wasm32 cargo check and by construction, not by a test. cargo
check --target wasm32-unknown-unknown -p pdfcraft-cos passes, cargo test -p
pdfcraft-cos and -p pdfcraft-redact pass, and clippy with warnings denied is
clean on both.
* Extract full-save trailer assembly
Assisted-by: OpenAI Codex
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
* M13 batch: test that results hold no planted secret, and record network-crate reachability
* netcheck: list socket2, report a walk past the cap, test by the gate rule
socket2 can open sockets, so it belongs in the list. A walk past the cap now
fails instead of reading as unreachable. The baseline test follows check_rows,
so a dependency update that moves a path no longer fails cargo test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Issue #5081: assembly at revision 3 and later needs bit 11; high-quality print, accessibility and form fill follow the revision
* Permissions: at revision 2, bits 11 and 12 mean nothing
Table 22 defines them for revision 3 and later. A revision 2 file with bit 4
clear and bit 11 set was given page changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Add IBAN matching to Search and Redact patterns
The Pattern/PATTERNS/from_id interface is consumed generically by redact_mark and the UI pattern combobox through the engine re-exports, so one new variant and matcher exposes IBAN in Search and Redact everywhere. A unit test shows valid compact and spaced IBANs are marked while wrong check digits, short runs, unknown countries, letter check digits and token-glued IBANs are not.
Add an Iban variant to Pattern, list it in PATTERNS, handle it in label, id and find, and add an iban matcher with a country-code length table, compact or group-of-four-spaced acceptance, alphanumeric start and end boundaries and ISO 7064 mod-97 check-digit validation.
Test: iban_matches_compact_spaced_and_rejects_near_matches failed before the change and passes after.
* Address review for pull request 4332
* Address review for pull request 4332
* review: address CodeRabbit findings
- patterns.rs: fix SWIFT registry IBAN lengths EG 27->29, LY 27->25, SD 24->18 and add NI 28, BI 27, DJ 27, OM 23 (crates/redact/src/patterns.rs:545,549,551,558-561)
- patterns.rs: state table source as SWIFT IBAN registry without claiming full ISO 13616 coverage (crates/redact/src/patterns.rs:475)
- patterns.rs: regression test with valid registry examples for EG, LY, SD, NI, BI, DJ, OM and a wrong-length EG rejection (crates/redact/src/patterns.rs:892)
* Complete the IBAN country table and accept spaces only between groups
The country table now lists all 89 countries of the SWIFT IBAN registry
(release 99) as one sorted constant. Honduras, Russia, Yemen, Falkland
Islands, Mongolia and Somalia were missing.
A space is accepted only after a full group of four characters, so a code
followed by ordinary words ("BA37 leaves at noon") ends the run instead of
being tried against the check digits.
Tests: the registry example of every country matches compact and printed,
changes to one digit are rejected, and the table and the examples agree.
* Advertise the IBAN pattern in the redact_mark tool schema
The pattern enum in the advertised schema was kept by hand and still
listed phone, email, credit-card, ssn and date, so clients that obey the
schema could not select iban, which the runtime already accepted. The
enum is now built from the engine's pattern registry, so it cannot drift
from it again, and the description names iban.
Tests: tools/list advertises exactly the registry's pattern ids and
names each in the description, and an iban mark made through the tool
covers the spaced German sample and nothing else.
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>