17 Commits
Author SHA1 Message Date
huppiflupp bff4be14c3 M10: Recognize text finds small print on large pages (tiled detection)
## What this changes

Recognize text now finds small print on large, dense pages (newspapers, small print, blurred photos).

The ocrs detection model reads a fixed 600 × 800 pixel input, so ocrs shrinks every page to that size before detecting words. A full page at 300 dpi (2480 × 3508) is shrunk about four times, and small letters disappear. With this change:

- **When (size):** an axis is cut in two when the page is at least twice the detector's input along it (width ≥ 1200 px, height ≥ 1600 px for the ocrs models). Any full page at 300 dpi becomes 2 × 2 tiles; a landscape page may become 2 × 1. The input size is read from the detection model when it loads. Pages below that size (72 to 150 dpi renders, screenshots) are read exactly as before and cost nothing extra.
- **When (text size):** a page that passes the size rule is tiled only if its text is small once the page is shrunk to the detector's input, decided by a cheap pixel signal, the **line pitch** (`crates/ocr/src/pitch.rs`). The page is box-averaged to a grey copy of at most 1600 px a side and cut into a 6 × 6 grid of blocks. In each block, the ink is summed per pixel row, the slow trend (shading, pictures) is removed with a moving average, and the autocorrelation is taken. The block autocorrelations are averaged, and the first strong peak is the pitch. Lines in columns that don't line up still agree on the period, and blank blocks are skipped. The same runs on pixel columns for sideways pages, and the axis with the stronger peak wins. The pitch is converted to detector pixels (pitch × 800 / page height). The rule:
  - pitch **under 9 detector pixels**: tile;
  - ink but **no clear line spacing** (no autocorrelation peak above 0.05): tile, because the measurement showed these pages (blurred, creased, oblique or moiré photos) gain on average;
  - large print, and blank pages: detect whole, as before.
  It costs 6.6 ms per page on average (median 5.1 ms, max 45 ms) and reads every pixel once.
- **How:** tiles overlap by 4 % of the page on each side of a cut. Words are **detected** tile by tile. A word is kept only by the tile whose core cell (its share of the page without the overlap) holds its centre, so a word in an overlap counts once. The words are moved back to page pixels, and then lines are formed and **recognised on the whole page** as before. Lines that cross a cut stay single lines, and recognition reads the same pixels it read before.
- The edge cells reach to infinity, so every finite word centre belongs to exactly one tile, including boxes that stick out past the page.

The change is small and lives in `crates/ocr/src/tiles.rs` plus a `detect_words` step in `Ocr::recognize`. While I was there I also made the existing RGBA length check use checked arithmetic (`width * height * 4` was a `u32` product).

### Measured

MDPBench de/es/fr (480 pages), official evaluation with CDM off, `en` recogniser. I ran the same harness build with and without the change (the baseline reproduces the earlier one exactly):

| | Digital | Photo | All | Time per page |
|---|---|---|---|---|
| before | 78.9 | 41.5 | 50.8 | 0.39 s (190 s total) |
| tiling every large page (this PR's first commit) | 80.4 | 42.8 | 52.2 | 0.58 s (281 s total) |
| **tiling only small or unclear print (this PR)** | **81.2** | **42.9** | **52.5** | 0.54 s (259 s total) |
| earlier prototype (full OCR per tile, outside the app) | 81.4 | 42.4 | 52.1 | n/a |

The three rows of this PR come from one harness binary with the rule switched by an environment variable. Each was timed twice (190/191, 281/281, 259/259 s). Per language, the line-pitch rule against tiling every large page: DE 62.9 vs 62.7, ES 47.9 vs 47.8, FR 46.8 vs 46.1. It tiles **214 of the 457** pages that pass the size rule.

**How the rule was chosen.** I measured first, in Python on the per-page scores of the official evaluation (`overall` per page, base vs. tiled). Per-page score gain from tiling, by measured pitch:

| pitch (detector px) | pages | mean gain | pages gaining / losing > 0.02 |
|---|---|---|---|
| < 4 | 40 | +0.018 | 10 / 4 |
| 4–6 | 40 | **+0.114** | 24 / 2 |
| 6–10 | 82 | +0.004 | 10 / 6 |
| 10–14 | 80 | **−0.013** | 5 / 26 |
| 14–35 | 144 | −0.003 | 22 / 25 |
| no clear line spacing | 82 | +0.029 | – |

The gain sits at small pitch, and pitches of 10–14 detector pixels (ordinary body text on a full page) **lose** a little when tiled. Picking tiled or base per page by this rule, and running the official evaluation on that mix, predicted exactly what the Rust build then scored (52.5 / 81.2 / 42.9). Thresholds between 8 and 10 give 52.4–52.5. Treating "no clear spacing" as "don't tile" instead gives 52.0 with only 132 pages tiled. The choice is tuned on these 480 pages, so take the last few tenths with that in mind.

**Time.** The saving is smaller than the page count suggests. The pages that stay tiled are the dense ones, where tiling finds many more words, and recognising those words costs more than the three extra detector runs. Tiling every large page cost +91 s. The pitch rule costs +69 s, so it removes about a quarter of the overhead and still scores higher.

Word recall in the benchmark's own per-kind table, before → after: de digital 61.9 → 78.5 %, fr digital 17.8 → 24.5 %, de/es/fr photos 38.1/37.7/10.1 → 42.1/43.2/12.6 %. Sideways photos are unchanged; those need rotation, which is a separate PR.

Detection runs four times instead of once on a tiled page (machine: Ryzen AI Max+ 395, CPU).

### How it composes with the open OCR PRs

- #5157 (reading order) ends `recognize` in `layout::reading_order`. This PR changes only the detection step before `find_text_lines`, so tiles are merged before lines are formed and before reading order runs, which is the order we want. I measured both together: the numbers above come from a harness tree that has #5157's reading order applied. The two diffs touch neighbouring lines in `recognize`, so whichever merges second needs a trivial rebase.
- #5171 (layout regions) and #5058 (line padding) work on lines and regions after detection, so they compose the same way.

## How you verified it

- [x] `cargo fmt --check`
- [x] `cargo clippy -p pdfcraft-ocr --all-targets -- -D warnings`
- [x] `cargo test -p pdfcraft-ocr`: 26 passed, **with the OCR models installed** (CI has none, so CI skips the model-backed tests). With the models, I also ran the OCR tests of `pdfcraft-engine` (`ocr`, `recogni` filters) and `pdfcraft-automation` (`ocr`, `scan`) locally, and all pass.
- [x] `cargo xtask layers`, `cargo xtask assets`, `cargo xtask parity`

**Tests.**
- Unit tests, no models (`crates/ocr/src/tiles.rs`): the tile geometry for small pages and for A4 at 300 dpi; every word centre, on cuts, edges and outside the page, belongs to exactly one tile, and NaN or infinity belongs to none; cropping, including out-of-range and overflowing crops, returns `None` without panicking.
- Unit tests, no models (`crates/ocr/src/pitch.rs`), on synthetic A4 pages at 300 dpi: a blank page is not tiled; large print (lines every 100 px, 22.8 detector px) is not; small print (every 30 px, 6.8 detector px) is, in grey, RGB and RGBA alike; sideways small print (lines running down the page) is; noise (ink without lines) is; the first strong autocorrelation peak is the pitch, not its multiple; empty, 1-pixel, short or long buffers, 0 or 5 channels, overflowing sizes, a zero-sized detector and slivers give an answer without panicking.
- Model-backed test `finds_small_print_on_a_large_page`: an A4 page at 300 dpi with four lines of small print. One sits inside a tile, one crosses the vertical cut, one is half in the overlap, and one crosses the horizontal cut. Each line must come back as exactly two words at the right place in page pixels.
- **Seen failing by reverting the fix:**
  - Detection on the whole page instead of tiles: 5 words instead of 8, unreadable ("FIOWOR", "In3").
  - Keeping every tile's words without the centre-in-core rule: 14 instead of 8, because the lines in the overlaps doubled.
  - Finite edge cells: `every_word_centre_belongs_to_exactly_one_tile` fails.
  - Never cutting: the geometry tests fail.
  - Line-pitch rule, each mutation reverted afterwards: tiling every page (the first commit's behaviour) fails `a_blank_page_is_not_tiled` and `large_print_is_not_tiled`. Never tiling on a measured pitch fails `small_print_is_tiled` and `sideways_small_print_is_tiled`. Rows only, without the column profile, fails `sideways_small_print_is_tiled`. Taking the strongest peak instead of the first strong one fails `the_first_strong_peak_is_the_pitch`. Not tiling unclear pages fails `ink_without_lines_is_tiled`. No detrending fails four tests. With the threshold lowered to 5 detector pixels, the model-backed `finds_small_print_on_a_large_page` finds 5 words instead of 8, so it exercises the rule too (its page measures 5.8).

## Never-crash

- [x] No new `unwrap`/`expect`/`panic!`/`unreachable!`/`todo!` outside tests
- [x] No indexing or slicing with input-derived positions: crops use checked arithmetic and `get`; tile bounds are clamped to the page; float-to-int casts are of values clamped to `0..=len`. The pitch measure checks that the buffer matches width × height × channels with checked arithmetic, reads pixels through iterators and `get`, and works on at most 1600 × 1600 values whatever the page size. It divides only by non-zero counts, so it returns an answer for any input.
- [x] No `unsafe`
- [x] n/a, this fixes no crash

## Provenance

- [x] No asset from an Adobe product
- [x] No font files
- [x] No GPL/AGPL code
- [x] Behaviour comes from our own measurements on MDPBench. The approach (overlapping tiles, keep by centre; line pitch from the autocorrelation of a projection profile) is a generic technique, not read from another implementation.
- [x] No new assets. One new dependency edge, `rten-imageproc` 0.26 (MIT/Apache-2.0), which was already in the tree through `ocrs`.

## Security

- [x] No path handling changed
- [x] No MCP or control channel changes
- [x] No network access

## Parity and reach

- [x] `ocr.recognize-text` notes say when pages are tiled (size and line pitch), and that the model-backed test runs only where the models are installed. The tiling and pitch unit tests are evidence.
- [x] Reachable headlessly: this is inside `Ocr::recognize`, so `ocr_recognize` and `ocr_recognize_files` get it unchanged.
- [x] No documented behaviour changed

## Anything a reviewer should look at first

- **The tiling rule:** an axis is cut only if each half still fills the detector's input, so a tile is never padded. A lower threshold would also tile mid-size pages. I kept it conservative so that small pages cost nothing. The prototype tiled every page; under this rule, 468 of the 481 benchmark images are cut along at least one axis and 337 along both.
- **The line-pitch rule (`pitch.rs`).** Its constants (1600 px working size, 6 × 6 blocks, peak floor 0.05, first peak ≥ 60 % of the strongest, 9 detector px) were chosen on MDPBench as described above. "No clear line spacing → tile" is deliberate. My first idea was the opposite, but the measurement favoured tiling.
- **Detect per tile, recognise on the whole page**, rather than full OCR per tile like the prototype. This is simpler: there are no lines to stitch back together and no rows to re-sort. It scores about the same as the prototype (all 52.2 vs 52.1, photo 42.8 vs 42.4, digital 80.4 vs 81.4).
2026-11-18 16:33:38 +07:00
Dylan M. Taylor eead454105 Cut over remaining codecs, xfa XML, and export packaging to craft-libs (#7326)
* chore: cut over image, png, tiff, and zune-jpeg to craft-image-codecs

* chore: cut over xfa XML parsing from roxmltree to craft-xml

* chore: cut over export DOCX and PPTX packaging to craft-opc and craft-xml

* style: cargo fmt
2026-11-14 13:49:46 -05:00
FredApps ec31c55763 fix(ocr): main builds on Windows again: the executable check is its own function per platform (#7222)
Since Rust 1.100, clippy's needless_bool refused program_is_executable on Windows: with the
#[cfg(unix)] block gone it read if ... { return false; } true, and the Windows gate failed on
every main commit since 13 Nov 09:38. The check is now one expression with the platform part
in has_execute_permission; behaviour is unchanged on every platform.
2026-11-13 16:28:29 +01:00
huppifluppandFredApps 8864903795 M10: Recognize text no longer clips the first and last letters of a line (#5058)
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-10 19:23:21 +07:00
ljkx dec9f3600b Windows: Tesseract and the --control tools start without a console window (#6735) 2026-11-10 14:23:38 +07:00
bmh129 9121f65564 M10: OCR models load from bytes; recognised lines travel as JSON (ocr.ocr-web)
The web build has no model files: it downloads them. `Ocr::from_bytes`
loads the pair only when each file's SHA-256 matches ATTRIBUTION.toml
(rten's ONNX parser can spin forever on bytes that aren't a model), and
`OcrsRecognizer::from_ocr` wraps the result as an engine that stays
available without files.

The web build's workers send what they read back as JSON:
`lines_to_json` / `lines_from_json` carry each word's box and, when it
has one, its confidence; the reader credits the words to the engine the
worker ran, and refuses anything else (a missing or non-finite number,
too many words) with an error.
2026-11-07 21:35:26 -06:00
9f04acccef M10: OCR engines, confidence, preprocessing and a review workflow (#977)
* M10.20: plan Unicode text for OCR output in Type3 faces (fonts, edit)

OCR output used to flatten every character beyond WinAnsi to a silent `?`. This
adds the two pieces that let the OCR layers write the recognized words as they
are; the OCR code that uses them follows in M10.21.

pdfcraft-fonts
- `TextPlan::for_words` maps each word to the standard Helvetica when WinAnsi
  covers it, else to Type3 fonts drawn from the craft-fonts faces (the Japanese
  document face first, then the rest of the build input). Characters no face
  has a glyph for are counted (`missing`), never dropped silently. At most 240
  characters per font and 2400 per plan, so one hostile word cannot balloon.
- Which face has which character is asked once per (face, character) for a plan,
  not once per word.
- `face_has`, `craft_glyph`, `win_ansi_char`, `fits_win_ansi` are public for it.

pdfcraft-edit
- `add_type3_fonts` / `stamp_with_fonts` register a plan's fonts on a page under
  a free `PCOcr`, `PCOcr1`, ... name, build each font once and share the object
  with later pages. A shared face is registered under a free name on a later
  page that already has a font of the name it was given first, instead of
  replacing that page's own font.
- The fonts carry real advances and a ToUnicode CMap, so extraction reads the
  text back.

pdfcraft-redact
- A test shows a hand-built Type3 text layer (the shape the OCR output writes) is
  redacted like any other text and the verifier passes.

Tests: pdfcraft-fonts 62 passed, pdfcraft-edit 129 passed, pdfcraft-redact 33
passed; clippy -D warnings clean on the three crates. The name-reuse and the
redaction tests build their fonts by hand, so they run without the craft-fonts
build input; the tests that plan real faces skip without it and were not run in
this commit's own check.

* M12: give the hostile-XFA open test more room on a loaded machine

The test opens a form whose calculate script sets a value 50 000 times, against
the one-second budget of an unasked script and a five-second wall-clock bound
on the open. Both sit close to what a machine running several suites at once
can take, so the test can fail without anything being wrong.

It now runs 20 000 iterations (the total is 19999) and allows 30 seconds, which
still fails for a script that hangs or is not capped. Test-only; unrelated to
the OCR work in this series, so it is a commit of its own.

Tests: both hostile_xfa tests in pdfcraft-engine pass.

* M10.21: pluggable OCR engines, preprocessing, review sessions and the OCR tools (ocr, engine, create, automation)

Scan & OCR gets a second engine, optional raster clean-up, a review step before
anything is applied, editable output and a batch mode, headlessly first: the
interface for them follows in M10.22.

pdfcraft-ocr
- A `Recognizer` trait with two engines: the built-in ocrs (moved out of lib.rs)
  and the installed tesseract program, run as a fenced external process: fixed
  argv, allow-listed language codes, a private temp directory (mode 0700 on
  Unix), 120 s timeout, 64 MiB output cap, 50 000-word reading cap. A location
  from PDFCRAFT_TESSERACT is validated (absolute, existing, executable) and an
  invalid one fails closed instead of falling back to the PATH. `--list-langs`
  and `--version` are read from stdout or, for older versions, stderr.
- Strategies: PrimaryOnly, ConfidenceWeighted (the second engine joins when the
  first reading's mean is under 70 or unknown) and a ROVER merge (higher
  confidence wins per word; against an ocrs word of unknown confidence a
  tesseract word wins only at 70 or more; capped at 100 M comparisons; a failing
  partner degrades to the primary with a note).
- Preprocessing: opt-in auto-rotate, deskew, denoise and binarize on the RGBA
  raster, each with an exact inverse so word boxes map back to the scan. The
  300 dpi normalization the steps are calibrated for runs only when a step is
  on: with every step off the render is read as rendered, as before. (Enlarging
  a 72 dpi render 4x on its own made the models read "due: €250" as "ue: ?250".)
  `OcrImage` is validated by construction (side and 64 MP area caps, checked
  before any buffer is allocated, including by the resize).
- Confidence bands (High >= 90, Medium 70-89, Low < 70) drive review.

pdfcraft-engine, pdfcraft-create
- `OcrSettings` gains engine, strategy, the four steps, output mode, region, the
  skip flags and the low-confidence threshold; Automatic reads confidence
  weighted unless a strategy is chosen.
- A recognition is a session: the document guard refuses a stale result with its
  reason, pages can be applied one at a time (the guard is taken again after each
  applied page), and applying is one undoable step. A per-language dictionary
  (a plain-text file: words added from any document stay on disk after it closes).
- Output is searchable (an invisible layer) or a NEW editable document of visible
  text at the recognized boxes (`create::from_contents`); words beyond WinAnsi use
  the Type3 faces of M10.20 in both, with the shared plan across pages.
- Engine discovery never blocks the interface: results are cached, a stale
  answer is shown while a background probe retries, and the models search and
  `--version` are cached. Batch: pages or files that already have text are
  skipped unless forced, and inputs sharing a file name are found up front.

pdfcraft-automation
- ocr_recognize (engine, strategy, steps, region, output mode), ocr_words (with a
  retained review handle), ocr_review_apply, ocr_dictionary, ocr_status and
  ocr_recognize_files. The editable `out` must be a new file inside --root and is
  refused for a searchable run (it used to be ignored), two batch inputs of one
  file name are refused before anything is read, and the batch folder is checked
  for existing outputs.

pdfcraft-ui-egui
- Only what the new API needs: the Recognize Text dialog reads with
  `recognizers(&settings)`. `ocr.correct` is registered with its label in every
  catalog; the screen itself is M10.22.

Tests: pdfcraft-ocr 78 passed; pdfcraft-create passed; pdfcraft-engine and
pdfcraft-automation 450 passed and 1 failed (the automation lib 17, automation.rs
157, hostile_args 2 among them); pdfcraft-ui-egui lib + ocr + control tests 391
passed; clippy -D warnings clean on the five crates. The one failure is in a
test this commit does not change:
recognize_text_makes_a_scanned_page_searchable fails where the models are
installed (they read "quick" as "auick" at that render size); the roadmap on main
already records it. The real tesseract binary ran in
tesseract::tests::recognize_runs_the_installed_binary (tesseract 5.4.0, PATH).

* M10.22: OCR Verify screen, the Recognize Text options and the interface catalogs (ui-egui)

Scan & OCR ▸ Correct recognized text: a full-window screen where a recognition
is reviewed before anything is applied, plus the dialog and batch options for
the engines of M10.21.

OCR Verify screen
- Four panes: the page list, the source page with confidence-coloured word boxes
  (click to select, drag to draw a region), the recognized text, and a zoom plus
  word inspector. Corrections commit on Enter (an empty commit removes the word),
  Remove/Restore, "+ Dictionary" (a per-language file in the preferences folder,
  so a word added from a document stays on disk after it closes), per-word and
  per-page Verified, F8 / Shift+F8 walk the low-confidence words, Ctrl+T marks the
  page reviewed, Ctrl+Tab moves the pane focus, and review edits undo within the
  panel.
- Re-OCR of the whole page or a drawn region. A region re-read moves the review
  marks of the words it keeps to their new positions and selects the first new
  word; a whole-page re-read clears that page's marks (and the undo brings them
  back). The region outline is scaled onto the pane, not mapped through user
  space again, so it is right on rotated and cropped pages.
- Accept / Reject per page or whole. Pages accepted one at a time each apply once
  (the document guard is taken again after each applied page; an edit made
  elsewhere in between is still refused with its reason), and Accept in
  searchable mode is one undoable step. Editable mode writes a NEW
  `<name>_ocr.pdf` through a save dialog and never the source. A real Cancel
  keeps the pages already read.
- The engine and strategy menus disable a choice that cannot run and say why;
  the check never waits on a process (a background probe fills a cache), and
  choosing Automatic moves "Primary only" to Confidence weighted.

Recognize Text dialog and batch
- Engine, strategy, the four clean-up steps and the skip flags are remembered with
  the other choices (a saved Automatic without a strategy reads confidence
  weighted). The batch refuses two inputs of one file name and existing outputs
  before reading anything, notes low-confidence pages at the settings' threshold,
  and its toast has a Cancel (the file in flight finishes).

Catalogs and attribution
- The 90 new interface strings are in all 21 catalogs (the Lithuanian ones were
  written without native review, and the Bulgarian ones avoid U+045D, which the
  bundled monospace face lacks). ATTRIBUTION.toml carries the new hashes, a
  note on the Lithuanian entries and the README screenshot; ATTRIBUTION.md is
  regenerated.
- README: a screenshot of the Verify screen, taken with
  `cargo xtask screenshots` from a synthetic scan (`shot --ocr-verify on`) built
  from allowed assets; the page reads it with the real engines.

Tests: pdfcraft-ui-egui lib 373 passed (1 ignored: the Windows hinting
measurement probe), tests ocr 6, control 46, fonts 9; clippy -D warnings clean on
the engine, ui-egui, pdfcraft and xtask packages; `cargo xtask assets` ok. The
review state machine has unit tests for every fix above; each was seen failing
with its fix reverted.

* M10.23: parity entries, roadmap and localization notes for the OCR workflow

Records what M10.20-M10.22 ship, and what they do not.

parity/acrobat-features.toml
- Status changes against main: ocr.rotation-detection and ocr.deskew planned ->
  shipped (the opt-in steps of M10.21, with their tests); ocr.correct-recognized-
  text and ocr.ocr-engines planned -> partial. Other Area I entries get their
  evidence and notes brought up to date.
- ocr.correct-recognized-text: the evidence includes the one-page-at-a-time accept
  and the region re-read tests; the notes say what the headless tools cover
  (ocr_words with a retained review, ocr_review_apply through the same stale-
  document guard, ocr_dictionary) and what is still UI-only (suggestions,
  per-word Verified marks, region re-reads).
- ocr.ocr-engines stays partial rather than shipped: part of its evidence is
  conditional (the real-binary test runs only where tesseract is installed and
  skips elsewhere, and the process-level tests are Unix-only). It cites the tests
  that always run (location validation and the fail-closed rule, --list-langs
  parsing from stdout or stderr, the merge rules, the default strategy) and says
  which part is not proven everywhere.
- `cargo xtask parity`: area I 32.7% -> 44.2% weighted (5 -> 7 shipped, 7 -> 9
  partial); overall 52.8% -> 53.0% shipped and 58.2% -> 58.5% weighted.

ROADMAP.md
- One progress-log entry (2026-10-13, newest first) and the banner line. It says
  the catalogs are 21 with the Lithuanian additions not natively reviewed, that
  the end-to-end redaction test of the Type3 layer needs the craft-fonts build
  input, and that recognize_text_makes_a_scanned_page_searchable still fails where
  the models are installed, as main's log already records.

docs/localization.md, docs/localization-parity.md
- The Lithuanian entry count (2,578 -> 2,668) and the counts of the other
  catalogs that carry entry counts are re-measured with the 90 new entries; the
  Lithuanian OCR entries are marked as not natively reviewed.

Checked: `cargo xtask parity` (numbers above, evidence tests exist) and
`cargo xtask assets` pass on this tree.

* M10.24: ocr: apply a late result only to the document it was read from

The Recognize Text dialog closes on Go, so the document can be edited, or have
text redacted, while a long run is still reading. The result was applied
without the stale check the tools and the Verify screen use, so words read
before a redaction could be written back as invisible text. The run now keeps
its guard and applies through the checked path.

Accept with the editable output in the Verify screen had the same gap (Exit
keeps the review open): it now refuses when the document changed since the
review started.

Both new tests fail with their guard switched off.

* M10.25: ocr: ocr_dictionary takes dictionary file names only

Updating sorts, lower-cases and rewrites the file it is given, so pointed at
.env or a README it would corrupt it. The tool now accepts only
ocr-dictionary-<language>.txt style names, for reading as well as writing.

The size-limit test used a file name the rule now refuses first, so it renamed
its fixture to keep testing the size limit. The new test fails with the rule
switched off.

* M10.26: pt-BR: keep (s) on the word counts and say mecanismo for engine

Two review fixes from a native speaker: the two word(s) lines had lost their
(s), so a count of 1 read as 1 palavras, and engine was translated both as
motor and mecanismo. Mecanismo is the Brazilian word for a software engine and
two lines already used it, so the other six now match.

* M10: fit the rebased OCR workflow to current main's structure

* Fix OCR workflow issues from the review

Honor the Recognize dialog's editable-output choice: its result opens in
the review screen (whose Accept writes the editable document) instead of
an invisible layer. Make page acceptance idempotent: a second Accept
entire page never appends the OCR layer twice and keeps later
corrections. Move page and region re-OCR onto a worker thread with a
status line and a cancel that discards the reading. Skip the Tesseract
probe for jobs that name their engines. Bound Tesseract's stderr while
the child runs, not only after it exits. Surface the editable output's
substitution notes in the tools' and the run's results.

Also repair the OCR batch tool against the current structure: inputs are
held to the read root again, and a batch whose inputs share an output
name is refused before the output folder is created, compared the way
the default filesystem compares names; and the editable writer test's
word stays inside the fixture's pages.

* Apply pre-review fixes to the OCR workflow

Hold the automation OCR tools to the document's copying permission, as
the other text tools are: the recognized text is left out of the
results when copying is forbidden, and the paths that carry the words
out of the document (the editable output, the review word list) are
refused. Mark the ocr_dictionary tool as overwriting its path. Close
the searchable layer's graphics state around its invisible text so the
rendering mode cannot leak into later content. Read the OCR user
dictionary folder from the portable data folder in portable mode, like
the recovery store does. Resolve the attribution manifest's leftover
conflict markers and regenerate the generated attribution. Describe the
review flow, the searchable layer's Type3 fonts and the parity evidence
references accurately, and drop an undeclared copy of the recovery
module.

* OCR: reject readings that exceed the word limit

Assisted-by: OpenAI Codex

* OCR: new documents go through the page tree builder main added

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* OCR: review notes and lints

The editable output refuses a file created during recognition. ocr_words takes
skip_text_pages and force_ocr. Automatic OCR checks the engine it will use. The
stderr test stays below the kill cap, and a failed --ocr-verify shot exits with
an error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* OCR: restore the Verify screenshot attribution, follow the moved constants

The merge dropped the screenshot row with the translation rows. The model
filename constants live in ocrs.rs now, so the license check reads them there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Attribution: the Verify screenshot entry goes after the header comment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Attribution: the screenshot row in the table follows the manifest order

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 23:39:55 +01:00
Kyle Kassen da8cea2b0b M10: reuse verified OCR model and packaging prerequisites 2026-10-27 18:54:41 +00:00
mtlcl410andClaude Opus 5.5 15af2860f1 fix(ocr): the text layer restores the rendering mode it sets (#3029) (#3033)
text_layer wrote '/OCR BMC BT 3 Tr /PCHelv 1 Tf ... ET EMC'. Tr and the
font are graphics state, and ET doesn't reset them, so every content
stream after the layer on the page inherited 3 Tr: a header or footer
added after OCR was in the file but drew invisibly, in PdfCraft and in
MuPDF. The layer is now wrapped in q ... Q inside its marked content.


Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 03:12:37 -04:00
5cce1b247f M10: edit OCR text on scanned pages so the change shows (#1397)
* M10: edit OCR text on scanned pages so the change shows

Editing a recognised word on a scan rewrote it as invisible OCR text, so
the page still showed the old word, and the whole OCR layer grouped into
one paragraph that rewrapped when one word changed.

- An edited invisible paragraph is drawn visibly, over a cover in the
  paper colour sampled around it, in the scan's sampled ink colour
  (or the chosen colour).
- OCR words group into phrases: a table cell or a tab stop splits them
  and each scanned line stays apart; a phrase grows rightwards.
- Styled rewrites measure with the text matrix's horizontal scale, so
  stretched OCR words keep their widths instead of overlapping.
- The inline editor covers the scanned word and grows like the result.

* M10: size OCR text from each word's letters

A word box from recognition is tight around the ink, but the text layer
assumed every box spans Helvetica's ascender-descender height. A
capitals-only word such as "DT" spans only the cap height, so its text
was set about 23% too small and its baseline floated up; editing it
then drew the new word visibly smaller than the scanned one.

The em and baseline now come from the word's letters: cap height or
x-height on top, descender or baseline at the bottom. On a real scanned
form, a rewritten "DT" -> "DSI" now matches the scanned word's height
and baseline. The engine OCR-edit tests build their words the way a scan
would box them.

* Edit OCR text: sample colours only for invisible blocks, from the current bytes

Session::apply looked up every edited block with a fresh text_blocks pass and
re-serialized the whole document to render the sampling tile. Text blocks are
now cached per page on the editor (keyed on the unmodified document's source
identity, so any edit, undo or save invalidates them), visible blocks skip
sampling entirely, and the tile renders from the document's current bytes.
Adds a visible stretched-text regression test and documents that editing any
invisible text makes it visible over a painted cover.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Edit OCR text: name the cached text-blocks tuple (clippy), rustfmt

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 08:16:42 -04:00
Zeyu LiandBrandon Thomas b91d38c073 M14: text the fonts can't draw is kept and reported, never written as ? silently (#1112)
* M14: text the fonts can't draw is kept and reported, never written as ? silently

Form values and captions, text boxes, callouts, Fill & Sign text, measurement labels and stamp by-lines keep text the standard fonts can't draw, draw ? for it, mark the annotation (PCUndrawn) and report the edit (interface notice, tools' warnings); forms set /NeedAppearances (not in PDF/A). Flattening, redaction overlay text, typed and visible signatures and create-from-text refuse instead. Reports escape such characters as [U+XXXX].

* Catalogs: the two new can't-draw messages in Czech, Slovene, Kazakh, Turkish and Greek; refresh hashes

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-11 17:53:55 -04:00
mlkonradandClaude Opus 5.5 c7fcee4ed9 OCR: a scanned euro sign reads as €, not E (#796)
The ocrs crate's default alphabet has an E where its recognition model was
trained with € (ocrs-models DEFAULT_ALPHABET), so a scanned "€250" was
recognised as "E250". pdfcraft-ocr now gives ocrs the alphabet the model was
trained with.

the_alphabet_is_the_trained_one checks the alphabet without the models;
recognize_text_reads_the_euro_sign reads a scan end to end when they are
installed.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 15:08:04 -04:00
Brandon ThomasandClaude Opus 5.5 d90b3f897f Refs #103: ship the OCR models in the desktop release packages (#630)
Release builds said "Text recognition isn't installed: its models are
missing" because no packaging script or release job fetched the OCR
models: only `cargo xtask models` does, into the source tree.

- packaging/env.sh gains stage_models DEST, which runs
  `cargo xtask models DEST` (every ATTRIBUTION.toml kind = "model" file,
  SHA-256 verified, with its licence text). No model file is named, so
  the models PRs #433/#572 add ship without packaging changes.
- `cargo xtask models` also writes ATTRIBUTION.txt crediting each model
  (CC-BY-SA-4.0 requires attribution when redistributed).
- macOS: PdfCraft.app/Contents/Resources/models, before signing.
- Windows: models\ beside pdfcraft.exe in the MSI (WiX v5 <Files>
  harvest) and the portable zip; test-msi.ps1 checks the folder.
- Linux deb/rpm/tar.gz/AppImage, Flatpak bundle and FreeBSD tarball:
  share/pdfcraft/models beside bin/. Models::search_dirs now also looks
  in <exe dir>/../share/pdfcraft/models (Models::dirs_beside_exe); a
  test builds every package layout on disk and finds the models in each.
  release.yml and freebsd.yml check the packages contain them.
- The missing-models messages no longer tell end users to run cargo,
  and the 11 catalogs that keyed an old PRINTCRAFT_MODELS wording (or
  the previous one) now key the current English text; ATTRIBUTION
  hashes updated. A test checks the translations match.

The web (wasm) build, which the issue reports, still has no models; this
is the desktop half only.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 18:42:23 -04:00
Brandon ThomasandClaude Opus 5.5 d3db99818f Rename PrintCraft to PdfCraft (#174)
The app, crates, binaries, bundle id, packaging, icons, screenshots and docs are now
PdfCraft / pdfcraft (repo storytold/pdfcraft). On first launch the desktop app moves the
PrintCraft settings and crash-recovery folders to the new name, and both apps read settings
saved under the old storage key, so upgrades keep recent files, preferences and unsaved work.
The signing test fixtures keep their PrintCraft subject names (they are binary).

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 16:03:57 -04:00
Brandon ThomasandClaude Opus 5.5 1434db626a Never crash: deny the panic lints in crates that are already clean
annot, compare, content, cos, export, geom, model, ocr, preflight, print and redact have no
production `unwrap`/`expect`/`panic!`/`unreachable!`/`todo!`/`unimplemented!`. The attribute
keeps it that way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 13:12:21 -04:00
Brandon ThomasandClaude Opus 5.5 ab47bf4b9a CI: as_chunks instead of constant-size chunks_exact (clippy 1.99 chunks_exact_to_as_chunks)
Stable clippy 1.99 denies chunks_exact/chunks_exact_mut with a constant
size (32 sites), which failed the test job on main and every open PR.
Behaviour unchanged; MSRV 1.90 has as_chunks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 12:48:43 -04:00
Brandon ThomasandClaude Opus 5.5 b87cf39168 M10: Scan & OCR ▸ Recognize text: ocrs-based recognition (models fetched by cargo xtask models), invisible searchable text layer, Recognize Text dialog with background progress; ocr_recognize / ocr_status tools
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 03:47:58 -04:00