* M10.20: plan Unicode text for OCR output in Type3 faces (fonts, edit)
OCR output used to flatten every character beyond WinAnsi to a silent `?`. This
adds the two pieces that let the OCR layers write the recognized words as they
are; the OCR code that uses them follows in M10.21.
pdfcraft-fonts
- `TextPlan::for_words` maps each word to the standard Helvetica when WinAnsi
covers it, else to Type3 fonts drawn from the craft-fonts faces (the Japanese
document face first, then the rest of the build input). Characters no face
has a glyph for are counted (`missing`), never dropped silently. At most 240
characters per font and 2400 per plan, so one hostile word cannot balloon.
- Which face has which character is asked once per (face, character) for a plan,
not once per word.
- `face_has`, `craft_glyph`, `win_ansi_char`, `fits_win_ansi` are public for it.
pdfcraft-edit
- `add_type3_fonts` / `stamp_with_fonts` register a plan's fonts on a page under
a free `PCOcr`, `PCOcr1`, ... name, build each font once and share the object
with later pages. A shared face is registered under a free name on a later
page that already has a font of the name it was given first, instead of
replacing that page's own font.
- The fonts carry real advances and a ToUnicode CMap, so extraction reads the
text back.
pdfcraft-redact
- A test shows a hand-built Type3 text layer (the shape the OCR output writes) is
redacted like any other text and the verifier passes.
Tests: pdfcraft-fonts 62 passed, pdfcraft-edit 129 passed, pdfcraft-redact 33
passed; clippy -D warnings clean on the three crates. The name-reuse and the
redaction tests build their fonts by hand, so they run without the craft-fonts
build input; the tests that plan real faces skip without it and were not run in
this commit's own check.
* M12: give the hostile-XFA open test more room on a loaded machine
The test opens a form whose calculate script sets a value 50 000 times, against
the one-second budget of an unasked script and a five-second wall-clock bound
on the open. Both sit close to what a machine running several suites at once
can take, so the test can fail without anything being wrong.
It now runs 20 000 iterations (the total is 19999) and allows 30 seconds, which
still fails for a script that hangs or is not capped. Test-only; unrelated to
the OCR work in this series, so it is a commit of its own.
Tests: both hostile_xfa tests in pdfcraft-engine pass.
* M10.21: pluggable OCR engines, preprocessing, review sessions and the OCR tools (ocr, engine, create, automation)
Scan & OCR gets a second engine, optional raster clean-up, a review step before
anything is applied, editable output and a batch mode, headlessly first: the
interface for them follows in M10.22.
pdfcraft-ocr
- A `Recognizer` trait with two engines: the built-in ocrs (moved out of lib.rs)
and the installed tesseract program, run as a fenced external process: fixed
argv, allow-listed language codes, a private temp directory (mode 0700 on
Unix), 120 s timeout, 64 MiB output cap, 50 000-word reading cap. A location
from PDFCRAFT_TESSERACT is validated (absolute, existing, executable) and an
invalid one fails closed instead of falling back to the PATH. `--list-langs`
and `--version` are read from stdout or, for older versions, stderr.
- Strategies: PrimaryOnly, ConfidenceWeighted (the second engine joins when the
first reading's mean is under 70 or unknown) and a ROVER merge (higher
confidence wins per word; against an ocrs word of unknown confidence a
tesseract word wins only at 70 or more; capped at 100 M comparisons; a failing
partner degrades to the primary with a note).
- Preprocessing: opt-in auto-rotate, deskew, denoise and binarize on the RGBA
raster, each with an exact inverse so word boxes map back to the scan. The
300 dpi normalization the steps are calibrated for runs only when a step is
on: with every step off the render is read as rendered, as before. (Enlarging
a 72 dpi render 4x on its own made the models read "due: €250" as "ue: ?250".)
`OcrImage` is validated by construction (side and 64 MP area caps, checked
before any buffer is allocated, including by the resize).
- Confidence bands (High >= 90, Medium 70-89, Low < 70) drive review.
pdfcraft-engine, pdfcraft-create
- `OcrSettings` gains engine, strategy, the four steps, output mode, region, the
skip flags and the low-confidence threshold; Automatic reads confidence
weighted unless a strategy is chosen.
- A recognition is a session: the document guard refuses a stale result with its
reason, pages can be applied one at a time (the guard is taken again after each
applied page), and applying is one undoable step. A per-language dictionary
(a plain-text file: words added from any document stay on disk after it closes).
- Output is searchable (an invisible layer) or a NEW editable document of visible
text at the recognized boxes (`create::from_contents`); words beyond WinAnsi use
the Type3 faces of M10.20 in both, with the shared plan across pages.
- Engine discovery never blocks the interface: results are cached, a stale
answer is shown while a background probe retries, and the models search and
`--version` are cached. Batch: pages or files that already have text are
skipped unless forced, and inputs sharing a file name are found up front.
pdfcraft-automation
- ocr_recognize (engine, strategy, steps, region, output mode), ocr_words (with a
retained review handle), ocr_review_apply, ocr_dictionary, ocr_status and
ocr_recognize_files. The editable `out` must be a new file inside --root and is
refused for a searchable run (it used to be ignored), two batch inputs of one
file name are refused before anything is read, and the batch folder is checked
for existing outputs.
pdfcraft-ui-egui
- Only what the new API needs: the Recognize Text dialog reads with
`recognizers(&settings)`. `ocr.correct` is registered with its label in every
catalog; the screen itself is M10.22.
Tests: pdfcraft-ocr 78 passed; pdfcraft-create passed; pdfcraft-engine and
pdfcraft-automation 450 passed and 1 failed (the automation lib 17, automation.rs
157, hostile_args 2 among them); pdfcraft-ui-egui lib + ocr + control tests 391
passed; clippy -D warnings clean on the five crates. The one failure is in a
test this commit does not change:
recognize_text_makes_a_scanned_page_searchable fails where the models are
installed (they read "quick" as "auick" at that render size); the roadmap on main
already records it. The real tesseract binary ran in
tesseract::tests::recognize_runs_the_installed_binary (tesseract 5.4.0, PATH).
* M10.22: OCR Verify screen, the Recognize Text options and the interface catalogs (ui-egui)
Scan & OCR ▸ Correct recognized text: a full-window screen where a recognition
is reviewed before anything is applied, plus the dialog and batch options for
the engines of M10.21.
OCR Verify screen
- Four panes: the page list, the source page with confidence-coloured word boxes
(click to select, drag to draw a region), the recognized text, and a zoom plus
word inspector. Corrections commit on Enter (an empty commit removes the word),
Remove/Restore, "+ Dictionary" (a per-language file in the preferences folder,
so a word added from a document stays on disk after it closes), per-word and
per-page Verified, F8 / Shift+F8 walk the low-confidence words, Ctrl+T marks the
page reviewed, Ctrl+Tab moves the pane focus, and review edits undo within the
panel.
- Re-OCR of the whole page or a drawn region. A region re-read moves the review
marks of the words it keeps to their new positions and selects the first new
word; a whole-page re-read clears that page's marks (and the undo brings them
back). The region outline is scaled onto the pane, not mapped through user
space again, so it is right on rotated and cropped pages.
- Accept / Reject per page or whole. Pages accepted one at a time each apply once
(the document guard is taken again after each applied page; an edit made
elsewhere in between is still refused with its reason), and Accept in
searchable mode is one undoable step. Editable mode writes a NEW
`<name>_ocr.pdf` through a save dialog and never the source. A real Cancel
keeps the pages already read.
- The engine and strategy menus disable a choice that cannot run and say why;
the check never waits on a process (a background probe fills a cache), and
choosing Automatic moves "Primary only" to Confidence weighted.
Recognize Text dialog and batch
- Engine, strategy, the four clean-up steps and the skip flags are remembered with
the other choices (a saved Automatic without a strategy reads confidence
weighted). The batch refuses two inputs of one file name and existing outputs
before reading anything, notes low-confidence pages at the settings' threshold,
and its toast has a Cancel (the file in flight finishes).
Catalogs and attribution
- The 90 new interface strings are in all 21 catalogs (the Lithuanian ones were
written without native review, and the Bulgarian ones avoid U+045D, which the
bundled monospace face lacks). ATTRIBUTION.toml carries the new hashes, a
note on the Lithuanian entries and the README screenshot; ATTRIBUTION.md is
regenerated.
- README: a screenshot of the Verify screen, taken with
`cargo xtask screenshots` from a synthetic scan (`shot --ocr-verify on`) built
from allowed assets; the page reads it with the real engines.
Tests: pdfcraft-ui-egui lib 373 passed (1 ignored: the Windows hinting
measurement probe), tests ocr 6, control 46, fonts 9; clippy -D warnings clean on
the engine, ui-egui, pdfcraft and xtask packages; `cargo xtask assets` ok. The
review state machine has unit tests for every fix above; each was seen failing
with its fix reverted.
* M10.23: parity entries, roadmap and localization notes for the OCR workflow
Records what M10.20-M10.22 ship, and what they do not.
parity/acrobat-features.toml
- Status changes against main: ocr.rotation-detection and ocr.deskew planned ->
shipped (the opt-in steps of M10.21, with their tests); ocr.correct-recognized-
text and ocr.ocr-engines planned -> partial. Other Area I entries get their
evidence and notes brought up to date.
- ocr.correct-recognized-text: the evidence includes the one-page-at-a-time accept
and the region re-read tests; the notes say what the headless tools cover
(ocr_words with a retained review, ocr_review_apply through the same stale-
document guard, ocr_dictionary) and what is still UI-only (suggestions,
per-word Verified marks, region re-reads).
- ocr.ocr-engines stays partial rather than shipped: part of its evidence is
conditional (the real-binary test runs only where tesseract is installed and
skips elsewhere, and the process-level tests are Unix-only). It cites the tests
that always run (location validation and the fail-closed rule, --list-langs
parsing from stdout or stderr, the merge rules, the default strategy) and says
which part is not proven everywhere.
- `cargo xtask parity`: area I 32.7% -> 44.2% weighted (5 -> 7 shipped, 7 -> 9
partial); overall 52.8% -> 53.0% shipped and 58.2% -> 58.5% weighted.
ROADMAP.md
- One progress-log entry (2026-10-13, newest first) and the banner line. It says
the catalogs are 21 with the Lithuanian additions not natively reviewed, that
the end-to-end redaction test of the Type3 layer needs the craft-fonts build
input, and that recognize_text_makes_a_scanned_page_searchable still fails where
the models are installed, as main's log already records.
docs/localization.md, docs/localization-parity.md
- The Lithuanian entry count (2,578 -> 2,668) and the counts of the other
catalogs that carry entry counts are re-measured with the 90 new entries; the
Lithuanian OCR entries are marked as not natively reviewed.
Checked: `cargo xtask parity` (numbers above, evidence tests exist) and
`cargo xtask assets` pass on this tree.
* M10.24: ocr: apply a late result only to the document it was read from
The Recognize Text dialog closes on Go, so the document can be edited, or have
text redacted, while a long run is still reading. The result was applied
without the stale check the tools and the Verify screen use, so words read
before a redaction could be written back as invisible text. The run now keeps
its guard and applies through the checked path.
Accept with the editable output in the Verify screen had the same gap (Exit
keeps the review open): it now refuses when the document changed since the
review started.
Both new tests fail with their guard switched off.
* M10.25: ocr: ocr_dictionary takes dictionary file names only
Updating sorts, lower-cases and rewrites the file it is given, so pointed at
.env or a README it would corrupt it. The tool now accepts only
ocr-dictionary-<language>.txt style names, for reading as well as writing.
The size-limit test used a file name the rule now refuses first, so it renamed
its fixture to keep testing the size limit. The new test fails with the rule
switched off.
* M10.26: pt-BR: keep (s) on the word counts and say mecanismo for engine
Two review fixes from a native speaker: the two word(s) lines had lost their
(s), so a count of 1 read as 1 palavras, and engine was translated both as
motor and mecanismo. Mecanismo is the Brazilian word for a software engine and
two lines already used it, so the other six now match.
* M10: fit the rebased OCR workflow to current main's structure
* Fix OCR workflow issues from the review
Honor the Recognize dialog's editable-output choice: its result opens in
the review screen (whose Accept writes the editable document) instead of
an invisible layer. Make page acceptance idempotent: a second Accept
entire page never appends the OCR layer twice and keeps later
corrections. Move page and region re-OCR onto a worker thread with a
status line and a cancel that discards the reading. Skip the Tesseract
probe for jobs that name their engines. Bound Tesseract's stderr while
the child runs, not only after it exits. Surface the editable output's
substitution notes in the tools' and the run's results.
Also repair the OCR batch tool against the current structure: inputs are
held to the read root again, and a batch whose inputs share an output
name is refused before the output folder is created, compared the way
the default filesystem compares names; and the editable writer test's
word stays inside the fixture's pages.
* Apply pre-review fixes to the OCR workflow
Hold the automation OCR tools to the document's copying permission, as
the other text tools are: the recognized text is left out of the
results when copying is forbidden, and the paths that carry the words
out of the document (the editable output, the review word list) are
refused. Mark the ocr_dictionary tool as overwriting its path. Close
the searchable layer's graphics state around its invisible text so the
rendering mode cannot leak into later content. Read the OCR user
dictionary folder from the portable data folder in portable mode, like
the recovery store does. Resolve the attribution manifest's leftover
conflict markers and regenerate the generated attribution. Describe the
review flow, the searchable layer's Type3 fonts and the parity evidence
references accurately, and drop an undeclared copy of the recovery
module.
* OCR: reject readings that exceed the word limit
Assisted-by: OpenAI Codex
* OCR: new documents go through the page tree builder main added
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* OCR: review notes and lints
The editable output refuses a file created during recognition. ocr_words takes
skip_text_pages and force_ocr. Automatic OCR checks the engine it will use. The
stderr test stays below the kill cap, and a failed --ocr-verify shot exits with
an error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* OCR: restore the Verify screenshot attribution, follow the moved constants
The merge dropped the screenshot row with the translation rows. The model
filename constants live in ocrs.rs now, so the license check reads them there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Attribution: the Verify screenshot entry goes after the header comment
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Attribution: the screenshot row in the table follows the manifest order
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Comments copy and paste, within and between documents. ⌘C with comments selected (and no text
selected) copies them, each with its replies, as XFDF: into the app's clipboard (every tab and
window) and as text onto the system clipboard. ⌘V, Edit ▸ Paste comments or the page's context
menu pastes them onto the current page at their own positions, as new comments under names of
their own, so nothing is replaced and pasting twice gives two sets; the copies are selected.
XFDF with comments from the system clipboard pastes too, so comments travel between processes.
pdfcraft_xfdf::export_xfdf_comments writes a selection with its replies; import_comments takes
a target page and a name prefix and imports copies (the usual import still replaces same names).
Engine: Edit::PasteComments { xfdf, page } (one undo step, comment permission) and
Session::export_comments. Tools: comments_copy (read-only) and comments_paste.
Several comments select at once: ⌘-click (Ctrl on Windows and Linux) or ⇧-click adds one or
takes it out, on the page and in the Comments panel; the one clicked last has the handles.
Delete removes them all as one step, a drag moves them together, the context menu copies them.
Connected lines, polygons and clouds finish from the right-click menu's Complete (with Undo last
point and Cancel drawing), as other editors do, besides Enter and a double-click.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
bookmark_export returns an outline, or one bookmark's subtree, as JSON:
titles, nesting, open state, colour, bold/italic and destinations as page
plus view in PDF user space (ISO 32000-2 12.3.2.2), optionally written to
a file. bookmark_import takes it back, inline or from that file, under any
bookmark at any position or in place of the outline, as one undoable edit;
bookmark_copy does the same from another open document by page number.
A bookmark whose page the target lacks keeps its place without a
destination and is reported in `unmapped`. Every node is checked before
anything is written (title, page, colour, view type and arity, 33 levels,
100,000 bookmarks), and the error names the node.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Brings the branch up to date with main (47f42ba0), which moved 37 commits since
the merge base, including page-edge shadows (#2088), full rewrite after page
delete (#4279) and the licence-check stage (#2212). No conflicts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Storing LZW streams with Flate decoded image streams outside the allowance,
with a limit for each stream and no total, and nothing asked whether to go
on: a 593 KB file of 100 images took 113 s with none refused, and images
refused by the image pass were still rewritten. The pass now pays for what
it decodes and for storing and checking it, at 8 steps a byte since Flate
takes up to 78 ns a byte, and streams past that stay LZW.
An image is counted as refused only when it stayed as it was: one packed
before the allowance ran out on its second read was rewritten. doc_reduce
reports images_refused as doc_optimize does, and the app's action says so
in its log. A refused image no longer reads as an unsupported filter:
FilterError::Spent says what happened.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
- Resolve all 13 merge conflicts between upstream/main and main
- Preserve full image editing suite: OBB, crop, proportional resize, arrange, and context menu actions
- Preserve text editing formatting tools: bullet/numbered lists, subscript/superscript, and line spacing
- Cleanly integrate upstream's canvas modularization, font updates, and styling changes
- Refactor arrange_image and run_edit_page_image helpers to satisfy clippy cognitive complexity limits
- All engine, edit, and UI integration test suites passing cleanly
doc_reduce, the engine's reduced_bytes doc, the optimize README's
presets and the optimize.reduce parity note still said every image
above 225 ppi goes to 150 ppi; black-and-white images go to 300 ppi
above 450 ppi, as CCITT Group 4 when smaller.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Redaction doesn't decode CCITT, so a mark over part of an image the
optimizer stored as Group 4 removes the whole image (nothing leaks),
and Export ▸ Images refuses it. The optimize README and the
mono-compression parity note say so. Later PRs of this stack add both
(#4335 export, #4384 redaction).
- doc_reduce's description, Engine::reduced_bytes and docs/cli-batch.md
now list the black-and-white step; docs/target-app-parity.md's
optimize row gives the measured counts and no longer lists CCITT as
missing; the M11 milestone row lists it as done.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
KenM76andClaude Opus 5.5
Grzegorz MartensandClaude Opus 5.5
Combiler01
songzhihui
alehel
Dylan M. Taylor
aiongg