diff --git a/ATTRIBUTION.md b/ATTRIBUTION.md index e8fa4adac6..38c28a80ec 100644 --- a/ATTRIBUTION.md +++ b/ATTRIBUTION.md @@ -4,10 +4,11 @@ Every asset PdfCraft includes, bundles or uses to build its published material, with its author, source and licence. The policy is in [AGENTS.md](AGENTS.md) §1. The machine-readable list, with the SHA-256 of every asset, is [ATTRIBUTION.toml](ATTRIBUTION.toml). Licence texts are kept beside the assets and summarised in [NOTICE](NOTICE). -## In this repository (257) +## In this repository (258) | Asset | Title | Author | Licence | Source | Used for | |---|---|---|---|---|---| +| `docs/images/pdfcraft-ocr-verify.png` | PdfCraft screenshot: OCR Verify screen | PdfCraft contributors (generated by `cargo xtask screenshots`) | MIT OR Apache-2.0 | this repository | README: OCR review. Shows only PdfCraft UI, its Lucide icons and a synthetic scanned page built from the embedded fonts, with a real recognition (the OCR models) loaded into the Verify screen | | `assets/icons/accessibility.svg` | Lucide icon "accessibility" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/accessibility.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs | | `assets/icons/align-left.svg` | Lucide icon "align-left" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/align-left.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs | | `assets/icons/align-center.svg` | Lucide icon "align-center" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/align-center.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs | diff --git a/ATTRIBUTION.toml b/ATTRIBUTION.toml index f998b6fc3d..99073babb1 100644 --- a/ATTRIBUTION.toml +++ b/ATTRIBUTION.toml @@ -10,6 +10,17 @@ # licence: SPDX expression from the allowlist in AGENTS.md §1.2. adobe_data = true marks the closed list of # Adobe-authored non-visual technical data allowed by AGENTS.md §1.1. +[[asset]] +path = "docs/images/pdfcraft-ocr-verify.png" +title = "PdfCraft screenshot: OCR Verify screen" +author = "PdfCraft contributors (generated by `cargo xtask screenshots`)" +source = "this repository" +licence = "MIT OR Apache-2.0" +licence_file = "LICENSE-MIT" +kind = "image" +usage = "README: OCR review. Shows only PdfCraft UI, its Lucide icons and a synthetic scanned page built from the embedded fonts, with a real recognition (the OCR models) loaded into the Verify screen" +sha256 = "ea7485b151e27ad3fd2a2dac079436ab11745d42032201a029bb20f819c6af8f" + [[asset]] path = "assets/icons/accessibility.svg" title = "Lucide icon \"accessibility\"" diff --git a/Cargo.lock b/Cargo.lock index e68f422a52..4816a44ec0 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4518,6 +4518,7 @@ version = "0.18.1" dependencies = [ "ocrs", "pdfcraft-fonts", + "png 0.18.1", "rten", "tempfile", "thiserror 2.0.21", @@ -4713,6 +4714,7 @@ dependencies = [ "pdfcraft-engine", "pdfcraft-fonts", "pdfcraft-model", + "pdfcraft-ocr", "pdfcraft-platform", "pdfcraft-render", "pdfcraft-xfa", diff --git a/README.md b/README.md index f9a41cb3ca..f488d160cc 100644 --- a/README.md +++ b/README.md @@ -253,6 +253,18 @@ Documents restricted by their author show a clear notice, and PdfCraft honours t +## Scan pages, then read the words back before trusting them + +**Scan & OCR ▸ Recognize text** turns a scan into searchable text — the current page, a range, or a folder of files written as searchable copies — with optional cleanup before reading (auto-rotate, deskew, denoise, binarize; word boxes map back to the original scan) and a second engine: an installed Tesseract joins the built-in one, merged per your strategy. A searchable result applies as one undoable step; the **Editable text** output opens the **OCR Verify screen** instead, where each word carries the engine's confidence as a colour, suspects are one keypress away, and corrections, a user dictionary and Re-OCR of a region or a page happen before Accept writes the new document. **Scan & OCR ▸ Correct recognized text** reviews any recognition there before it is applied. + +

+ The OCR Verify screen: a scanned note with each recognized word boxed in its confidence colour, the info strip reporting a 95% average confidence and one low-confidence word, and the confidence legend with its thresholds below +
+ The OCR Verify screen: words coloured by confidence — the legend spells the thresholds out — reviewed and corrected before anything is applied. +

+ +--- + ## Every tool, one keystroke away Press ⌘K to search every tool and command, or browse the **All tools** catalogue. Tools that are still in development are marked with the milestone that will ship them. diff --git a/ROADMAP.md b/ROADMAP.md index efe122a8b3..4fcdadaee9 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -329,6 +329,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-10-20 (M11, #3507):** Image merging keeps trailing comments, split operands and graphics state across content streams. Synthetic tool tests check saved text, rendered regions and exact image rows. - **2026-10-17 (M8, gap 6, #2662):** A filled cover over a long line counts when it hides half of any one word, not only 40% of the line's words or box, so a black box over one number in a long line — or an unapplied redaction mark over it — no longer audits clean while the number is still extractable; a sliver under half of every word it touches and under 40% of the line is still not flagged +- **2026-10-13 (M10, OCR review, batch and two engines):** Scan & OCR gains a review step, a second engine and Unicode output. Recognition can clean the raster first (opt-in auto-rotate, deskew, denoise, binarize; word boxes map back to the scan) and read with the built-in ocrs engine, an installed tesseract program run as a fenced external process (a validated location that fails closed instead of falling back to the PATH, allow-listed languages, a private temp directory, timeout and output caps), or both: Automatic asks tesseract when the ocrs reading is weak or of unknown confidence, and a ROVER merge keeps the more confident word (an unscored ocrs word yields only to a tesseract word of 70 or more). The OCR Verify screen (Scan & OCR ▸ Correct recognized text) reviews a recognition before it is applied: confidence-coloured word boxes, corrections, a per-language user dictionary (a plain-text file in the preferences folder, so a word added from any document stays on disk after it closes), Re-OCR of a page or a region, Accept/Reject per page or whole. Pages accepted one by one each apply once; a region re-read moves the review marks of the words it keeps, and a whole-page re-read clears that page's marks (undoable in the panel). Output is searchable (an invisible layer, one undo step) or a NEW editable document of visible text. Words beyond WinAnsi are drawn in Type3 faces from craft-fonts with a ToUnicode map in the editable output (characters no bundled face has appear as '?'); the searchable layer writes them as Type3 fonts with empty character programs and a ToUnicode map, so extraction reads them back with no fonts and no substitutions; redaction removes that layer like any other text (tests in pdfcraft-redact; the end-to-end engine test needs the craft-fonts build input and is skipped without it). Batch runs skip pages that already have text unless forced, and skip whole files that have text only with the opt-in skip-files flag (off by default; Force OCR overrides both); they refuse two inputs of the same file name and existing outputs before reading anything, and report low-confidence pages at the settings' threshold. Headless: ocr_recognize (engine, strategy, preprocessing, region; the editable output goes to an `out` that must not exist yet, and `out` is refused for a searchable run), ocr_words, ocr_review_apply, ocr_dictionary, ocr_status and ocr_recognize_files. All 21 interface catalogs carry the new strings; the Lithuanian ones were added without native review. Reading with every cleaning step off passes the render through untouched, as on main (the 300 dpi normalization runs only when a cleaning step is on: enlarging a 72 dpi render 4x on its own made the models read "due: €250" as "ue: ?250"). Still missing: the deep scan clean-up options (descreen, background and edge-shadow removal), scan images in the editable output, matching the scanned typeface, and recognition quality beyond the tested English set. - **2026-10-13 (M10, #1757):** Office export keeps each dotted-leader contents entry in one paragraph, with its page number, instead of inferring table cells from its separate text objects. - **2026-10-22 (M6, calculation scripts):** The Calculate tab and a rename read simplified field notation only from a comment, where Acrobat writes it: a string that spells `BVCALC … EVCALC` is no longer taken for the notation, or rewritten by a rename. A `/` after the condition of `if`, `while`, `for` or `with` starts a regular expression, so a rename no longer rewrites a name inside one, and a quote inside one no longer hides the `getField` calls after it. Both found in review of #3550. A `//` comment ends at `\r`, U+2028 or U+2029 as well as `\n`, so in a script with `\r` line endings the code after one is no longer hidden from a rename, or taken for the notation. Estimates unchanged. - **2026-10-20 (M6, calculation order page):** Prepare a form ▸ Calculation order, a page of its own, lists the calculated fields in the order they calculate with their formulas and values, moved with Up/Down arrows (as Acrobat's Set Field Calculation Order), Alt+↑/↓ or by dragging. A card says what is wrong with the order and holds Fix order; a loop has a card of its own that opens one of its formulas. On the page, each calculated field shows its step, and arrows show what the field in focus reads and what reads it. Estimates unchanged. diff --git a/apps/pdfcraft/src/main.rs b/apps/pdfcraft/src/main.rs index 5bc5c5c372..be2290d2f9 100644 --- a/apps/pdfcraft/src/main.rs +++ b/apps/pdfcraft/src/main.rs @@ -439,6 +439,8 @@ fn app_creator<'a>( if let Some(dir) = pdfcraft_ui_egui::RecoveryStore::default_dir() { app.enable_recovery(pdfcraft_ui_egui::RecoveryStore::new(dir)); } + // The OCR Verify screen stores its per-language dictionary beside recovery data. + app.ocr_verify.dictionary_dir = pdfcraft_ui_egui::default_dictionary_dir(); if let Some(state) = &cc.wgpu_render_state { let adapter = state.adapter.get_info(); let max_texture_dimension_2d = state.device.limits().max_texture_dimension_2d; diff --git a/crates/automation/README.md b/crates/automation/README.md index d218fbcda6..db5b988394 100644 --- a/crates/automation/README.md +++ b/crates/automation/README.md @@ -281,3 +281,26 @@ navigation-history tool; use the opt-in UI control channel to navigate the displ (`#RRGGBB`, default black) for `text` and `date`. The colour is saved in the FreeText appearance and survives reopening. Other mark/signature types reject `color`; their appearance controls are separate. + +## Reviewing OCR headlessly + +`ocr_words {doc, retain_review: true}` reads words without editing the document and returns +`review`, plus pages and word boxes/confidences. Page and word positions are 1-based. +`region` can restrict recognition to one named page, using displayed-page points. Without +`retain_review`, word inspection remains stateless. + +Apply reviewed text with `ocr_review_apply {doc, review, corrections: [{page: 1, word: 1, +text: "Corrected"}]}`. Omitted words keep their text; empty text removes a word. Geometry +comes from the retained recognition, never caller input. The captured document guard rejects +stale/foreign results. Successful application consumes the handle and is one undoable step. +Use `output_mode: "editable", out: "reviewed.pdf"` to write a separate editable document, +or `discard: true` to release the result without applying it. Failed validation preserves the +review for correction or discard. A new retained recognition replaces that document's prior +handle; closing the document releases it. At most eight documents may retain reviews, with +65,536-word/4-MiB input bounds and a 65,536-page cap. + +`ocr_dictionary {path: "ocr-dictionary-en.txt", add: ["Quartz"], remove: ["old"]}` atomically +updates the same one-word-per-UTF-8-line format OCR Verify uses. Omit add/remove to read it. +The explicit path honors `--root`, including symlink confinement; choose the appropriate +per-language file used by the UI. Entries are case-insensitive and duplicate-free. Invalid +or oversized inputs fail before writing (65,536 words, 4 MiB, at most 1,024 bytes per word). diff --git a/crates/automation/src/a11y/ocr.rs b/crates/automation/src/a11y/ocr.rs index 82e521ddd5..2846aa44a6 100644 --- a/crates/automation/src/a11y/ocr.rs +++ b/crates/automation/src/a11y/ocr.rs @@ -1,98 +1,534 @@ use super::{Args, Automation, Result, ToolError, Value, child, failed, json, write_atomic}; impl Automation { - /// Recognition settings from the arguments: `dpi` (clamped to 72–600), `language` (refused - /// unless the models read it) and `skip_text_pages`, each left at its default when absent. + /// The settings every OCR tool shares: dpi, language, the engine/strategy choice, the four + /// preprocessing flags, the skip flags and the output mode — each named exactly as its tool + /// schema says. An unknown engine, strategy or mode is an honest error, never a silent + /// fallback: a tool run must not quietly do something other than what it was asked. pub(crate) fn ocr_settings(&self, a: &Args) -> Result { + use pdfcraft_engine::ocr::{EngineChoice, MergeStrategy, OutputMode}; let mut settings = pdfcraft_engine::ocr::OcrSettings::default(); if let Some(d) = a.opt_num("dpi")? { settings.dpi = d.clamp(72.0, 600.0) as f32; } if let Some(l) = a.opt_str("language")? { if !pdfcraft_engine::ocr::LANGUAGES.iter().any(|x| x.0 == l) { - return Err(ToolError::InvalidArgs(format!("unsupported language {l:?}"))); + return Err(ToolError::InvalidArgs(format!( + "unsupported language {l:?} (supported: {})", + pdfcraft_engine::ocr::LANGUAGES.iter().map(|(c, _)| *c).collect::>().join(", ") + ))); } settings.language = l.into(); } - if let Some(s) = a.opt_bool("skip_text_pages")? { - settings.skip_text_pages = s; + if let Some(e) = a.opt_str("engine")? { + settings.engine = match e.trim().to_ascii_lowercase().as_str() { + "ocrs" => EngineChoice::Ocrs, + "tesseract" => EngineChoice::Tesseract, + "auto" | "automatic" | "ensemble" => EngineChoice::Auto, + other => return Err(ToolError::InvalidArgs(format!("unsupported engine {other:?} (ocrs, tesseract or auto)"))), + }; + } + settings.strategy = match a.opt_str("strategy")? { + Some(s) => match s.trim().to_ascii_lowercase().as_str() { + "primary" | "primary-only" | "primary_only" => MergeStrategy::PrimaryOnly, + "confidence" | "confidence-weighted" | "confidence_weighted" => MergeStrategy::ConfidenceWeighted, + "rover" | "rover-vote" | "rover_vote" | "vote" => MergeStrategy::RoverVote, + other => return Err(ToolError::InvalidArgs(format!("unsupported strategy {other:?} (primary, confidence or rover)"))), + }, + // Unset: the engine choice decides (auto brings in the second engine when the + // first reading is weak; a named engine reads alone). + None => settings.engine.default_strategy(), + }; + if let Some(m) = a.opt_str("output_mode")? { + settings.output_mode = match m.trim().to_ascii_lowercase().as_str() { + "searchable" | "searchable-image" | "searchable_image" => OutputMode::Searchable, + "editable" | "editable-text" | "editable_text" => OutputMode::EditableText, + other => return Err(ToolError::InvalidArgs(format!("unsupported output mode {other:?} (searchable or editable)"))), + }; + } + for (key, flag) in [ + ("auto_rotate", &mut settings.auto_rotate), + ("deskew", &mut settings.deskew), + ("denoise", &mut settings.denoise), + ("binarize", &mut settings.binarize), + ] { + if let Some(on) = a.opt_bool(key)? { + *flag = on; + } + } + for (key, flag) in [ + ("skip_text_pages", &mut settings.skip_text_pages), + ("skip_text_files", &mut settings.skip_text_files), + ("force_ocr", &mut settings.force_ocr), + ] { + if let Some(on) = a.opt_bool(key)? { + *flag = on; + } } Ok(settings) } - /// Recognizes text in each of `paths` and writes the result under the same file name in - /// `folder`, reporting per file its output, word count and skipped pages, or its error. A - /// path outside the root refuses the whole call before any work starts. + /// The optional `region` argument ([x0, y0, x1, y1] in points from the top-left of the + /// displayed page) as the job's pixel crop, at the resolution that page is really read at. + /// A region reads exactly one page. + fn ocr_region( + &self, + a: &Args, + id: pdfcraft_engine::DocId, + page: usize, + settings: &pdfcraft_engine::ocr::OcrSettings, + ) -> Result> { + const BAD: &str = "region must be [x0, y0, x1, y1] in points"; + let Some(v) = a.get("region") else { return Ok(None) }; + let arr = v.as_array().ok_or_else(|| ToolError::InvalidArgs(BAD.into()))?; + if arr.len() != 4 { + return Err(ToolError::InvalidArgs(BAD.into())); + } + let mut pts = [0.0f64; 4]; + for (i, x) in arr.iter().enumerate() { + pts[i] = x.as_f64().filter(|f| f.is_finite()).ok_or_else(|| ToolError::InvalidArgs(BAD.into()))?; + } + let scale = f64::from( + self.session.ocr_job(id, &[page], settings.clone()).ok_or_else(|| failed("the document could not be read"))?.render_scale(page), + ); + let px: Vec = pts.iter().map(|p| scale * p).map(|p| if p.is_finite() { p } else { 0.0 }).map(|p| p as f32).collect(); + Ok(Some([px[0], px[1], px[2], px[3]])) + } + + /// The per-page result entries the OCR tools return: recognised text or why the page was + /// passed over. The text itself is only echoed when the document allows copying; the + /// counts and metadata always are. + fn ocr_pages_json(found: &[pdfcraft_engine::ocr::OcrPage], include_text: bool) -> Vec { + found + .iter() + .map(|p| match &p.skipped { + Some(why) => json!({ "page": p.page + 1, "skipped": why }), + None => { + let mut page = json!({ "page": p.page + 1, "words": p.words.len(), "already_on_page": p.already_on_page }); + // The editable output's substitution warnings (characters no font could + // show) are part of the result: reported per page, never dropped. + if !p.notes.is_empty() { + page["notes"] = json!(p.notes); + } + if include_text { + page["text"] = json!(p.text()); + } + page + } + }) + .collect() + } + pub(crate) fn ocr_recognize_files(&mut self, a: &Args) -> Result { let settings = self.ocr_settings(a)?; let folder = self.resolve(a.str("folder")?, true)?; - // A path that leaves the root refuses the whole call before any work starts, like the - // other tools (the write check passes missing files, which stay per-file errors below). + // A path that leaves the read root refuses the whole call before any work starts; + // a file that is merely missing stays a per-file error below. let paths = a.strs("paths")?; for p in &paths { self.resolve_for(p, crate::file_access::Access::Read, true)?; } - // Two inputs that would write the same output file overwrite each other while both report - // success, so the whole batch is refused here: before the folder is created, anything is - // read, or the OCR engines load. + // Two inputs of one file name (from different folders) would write the same output: + // the second would replace the first while both report success. The whole batch is + // refused up front — before the folder is created, anything is read or the engines + // load — naming the file, the way the default filesystem compares names. let targets: Vec = paths.iter().map(|p| child(&folder, &input_file_name(p))).collect(); let shared = shared_file_names(&targets); if !shared.is_empty() { return Err(ToolError::InvalidArgs(format!( - "more than one input would write to the same output file, so each would replace the one before it. Colliding file names: {}. Give each input a different file name, or run them in separate batches with different folders.", + "several paths share a file name, so their outputs in folder would overwrite each other (rename or run them separately): {}", shared.join(", ") ))); } std::fs::create_dir_all(&folder).map_err(|e| failed(e.to_string()))?; - let ocr = pdfcraft_engine::ocr::engine().map_err(failed)?; + let names: Vec = paths.iter().map(|p| input_file_name(p)).collect(); + // Collision pre-flight: a searchable copy never silently overwrites. Every target is + // checked before the first page is read; a collision aborts the whole batch with the + // names in the error. + let mut collisions = std::collections::BTreeSet::new(); + for name in names { + if child(&folder, &name).exists() { + collisions.insert(name); + } + } + if !collisions.is_empty() { + let mut list: Vec = collisions.iter().take(5).cloned().collect(); + if collisions.len() > list.len() { + list.push(format!("and {} more", collisions.len() - list.len())); + } + return Err(failed(format!( + "the output folder already holds {} (delete or move them, or choose another folder): {}", + if collisions.len() == 1 { "a file of that name" } else { "files of those names" }, + list.join(", ") + ))); + } + // The engines are built (and named if they cannot be) before the first file is read. + let recognizers = pdfcraft_engine::ocr::recognizers(&settings).map_err(failed)?; let mut out = Vec::new(); + let (mut ok, mut skipped_files, mut words) = (0usize, 0usize, 0usize); for p in paths { let name = input_file_name(p); let result = self.resolve(p, false).map_err(|e| e.to_string()).and_then(|src| crate::read_input_bytes(&src).map_err(|e| e.to_string())).and_then( - |bytes| pdfcraft_engine::ocr::recognize_file(&name, std::sync::Arc::new(bytes), None, settings.clone(), &ocr, |_, _| true), + |bytes| { + pdfcraft_engine::ocr::recognize_file(&name, std::sync::Arc::new(bytes), None, settings.clone(), &recognizers, |_, _| true) + }, ); - out.push(match result { + match result { Ok(r) => { + let skipped_pages: Vec = r.pages.iter().filter(|p| p.skipped.is_some()).map(|p| p.page + 1).collect(); + if r.skipped() { + // A file passed over by the skip-files option is its own bucket: its + // original is left alone and nothing is written. + skipped_files += 1; + out.push(json!({ "path": p, "skipped": r.file_skipped, "skipped_pages": skipped_pages })); + continue; + } let target = child(&folder, &name); write_atomic(&target, &r.bytes)?; - let skipped: Vec = r.pages.iter().filter(|p| p.skipped.is_some()).map(|p| p.page + 1).collect(); - json!({ "path": p, "output": target.to_string_lossy(), "words": r.words(), "skipped_pages": skipped }) + ok += 1; + words += r.words(); + let mut entry = json!({ "path": p, "output": target.to_string_lossy(), "words": r.words(), "skipped_pages": skipped_pages }); + // The batch's low-confidence note: at or under the settings' threshold (60 + // by default), on 1-based sorted pages. + let threshold = settings.low_confidence_threshold; + if let Some((count, pages)) = pdfcraft_engine::ocr::low_confidence_pages(&r.pages, threshold) { + entry["low_confidence"] = json!({ "count": count, "pages": pages, "threshold": threshold }); + } + out.push(entry); } - Err(e) => json!({ "path": p, "error": e }), - }); + Err(e) => out.push(json!({ "path": p, "error": e })), + } } - Ok(json!({ "files": out })) + Ok(json!({ "files": out, "recognized": ok, "words": words, "skipped_files": skipped_files })) } - /// Recognizes text on the open document's `pages` (every page when absent) and adds it, - /// reporting the total word count and, per page, why it was skipped or what it found. pub(crate) fn ocr_recognize(&mut self, a: &Args) -> Result { - let settings = self.ocr_settings(a)?; + use pdfcraft_engine::ocr::OutputMode; + let mut settings = self.ocr_settings(a)?; let pages = if a.opt_ints("pages")?.is_some() { self.pages(a, "pages")? } else { Vec::new() }; - let doc = self.doc(a)?; // Recognizing needs only permission to change the document; the words it found are // returned only when copying is allowed too. - let (id, copy) = (doc.id, doc.allows_copying()); - let found = self.session.recognize_text(id, &pages, settings).map_err(failed)?; + let (id, copy) = { + let doc = self.doc(a)?; + (doc.id, doc.allows_copying()) + }; + // The optional region reads exactly one page. + if a.get("region").is_some() { + if pages.len() != 1 { + return Err(ToolError::InvalidArgs("region reads one page: name it in pages".into())); + } + settings.region = self.ocr_region(a, id, pages[0], &settings)?; + } + // The editable output's destination is checked before anything runs: a NEW file, never + // the open source and never an existing file replaced silently. + let out = match settings.output_mode { + OutputMode::Searchable => { + // `out` belongs to the editable output; a searchable run writes into the open + // document, so an `out` here would be silently ignored (and never checked + // against the root). + if a.get("out").is_some() { + return Err(ToolError::InvalidArgs("out is only used with output_mode editable; remove it, or choose editable".into())); + } + None + } + OutputMode::EditableText => { + // The editable output writes the recognized text out of the document: a + // document that forbids copying refuses here, before anything runs. + self.doc_to_copy(a)?; + // A write: the file does not exist yet, and write_atomic makes its folder. + let out = self.resolve(a.str("out")?, true)?; + if let Some(source) = self.session.get(id).and_then(|d| d.path.clone()) + && let (Ok(a), Ok(b)) = (std::fs::canonicalize(&out), std::fs::canonicalize(&source)) + && a == b + { + return Err(failed("out is the source document; the editable output is a new file, so choose another path")); + } + if out.exists() { + return Err(failed(format!( + "out already exists ({}); the editable output is a new file, so choose a path that does not exist yet", + out.to_string_lossy() + ))); + } + Some(out) + } + }; + // Honest up front: the engines are built (and named if they cannot be) before anything + // is read or written. + let recognizers = pdfcraft_engine::ocr::recognizers(&settings).map_err(failed)?; + match out { + None => { + let found = self.session.recognize_text(id, &pages, settings).map_err(failed)?; + let words = found.iter().map(|p| p.words.len()).sum::(); + Ok(json!({ "words": words, "pages": Self::ocr_pages_json(&found, copy) })) + } + Some(out) => { + let job = self.session.ocr_job(id, &pages, settings).ok_or_else(|| failed("the document could not be read"))?; + let found = job.run(&recognizers, |_, _| true); + let bytes = self.session.editable_ocr_bytes(id, &found).map_err(failed)?; + // recognition can take minutes: a file created meanwhile is refused, not replaced + if let Some(folder) = out.parent() { + std::fs::create_dir_all(folder).map_err(failed)?; + } + crate::bates::publish_new(&out, &bytes)?; + Ok(json!({ + "out": out.to_string_lossy(), + "words": found.iter().map(|p| p.words.len()).sum::(), + "pages": Self::ocr_pages_json(&found, copy), + })) + } + } + } + + /// The words of a recognition, for reviewing without the GUI: text, box (points from the + /// top-left of the displayed page), confidence, band, and which words are suspects. + /// Changes nothing; needs the engines (ocr_status). + pub(crate) fn ocr_words(&mut self, a: &Args) -> Result { + let mut settings = self.ocr_settings(a)?; + let pages = if a.opt_ints("pages")?.is_some() { self.pages(a, "pages")? } else { Vec::new() }; + // The word list is the document's text, word by word: refused when the document + // forbids copying it out. + let id = self.doc_to_copy(a)?.id; + if a.get("region").is_some() { + if pages.len() != 1 { + return Err(ToolError::InvalidArgs("region reads one page: name it in pages".into())); + } + settings.region = self.ocr_region(a, id, pages[0], &settings)?; + } + let retain = a.opt_bool("retain_review")?.unwrap_or(false); + if retain && !self.ocr_reviews.contains_key(&id) && self.ocr_reviews.len() >= 8 { + return Err(failed("at most 8 OCR reviews may be retained; discard one first")); + } + let recognizers = pdfcraft_engine::ocr::recognizers(&settings).map_err(failed)?; + let job = self.session.ocr_job(id, &pages, settings.clone()).ok_or_else(|| failed("the document could not be read"))?; + let infos: std::collections::HashMap = + self.session.get(id).map(|d| d.info.pages.clone()).unwrap_or_default().into_iter().enumerate().collect(); + let guard = job.guard.clone(); + let found = job.run(&recognizers, |_, _| true); + if found.len() > 65536 { + return Err(failed("at most 65536 OCR review pages are allowed")); + } + pdfcraft_engine::ocr::validate_ocr_words(found.iter().flat_map(|p| &p.words)).map_err(failed)?; + let band_name = |band: pdfcraft_engine::ocr::ConfidenceBand| match band { + pdfcraft_engine::ocr::ConfidenceBand::High => "high", + pdfcraft_engine::ocr::ConfidenceBand::Medium => "medium", + pdfcraft_engine::ocr::ConfidenceBand::Low => "low", + }; + let mut suspects = 0usize; let list: Vec = found .iter() - .map(|p| match &p.skipped { - Some(why) => json!({ "page": p.page + 1, "skipped": why }), - None if copy => json!({ "page": p.page + 1, "words": p.words.len(), "already_on_page": p.already_on_page, "text": p.text() }), - None => json!({ "page": p.page + 1, "words": p.words.len(), "already_on_page": p.already_on_page }), + .map(|p| { + let info = infos.get(&p.page); + let words: Vec = p + .words + .iter() + .map(|w| { + let suspect = pdfcraft_engine::ocr::is_suspect(w.confidence); + if suspect { + suspects += 1; + } + // The word's corners in view space, as the box [x, y, w, h] in points + // from the top-left of the displayed page. + let view = |pt: [f64; 2]| info.map(|i| i.user_to_view(pt[0] as f32, pt[1] as f32)); + let corners = [ + view(w.origin), + view([w.origin[0] + w.across[0], w.origin[1] + w.across[1]]), + view([w.origin[0] + w.up[0], w.origin[1] + w.up[1]]), + view([w.origin[0] + w.across[0] + w.up[0], w.origin[1] + w.across[1] + w.up[1]]), + ]; + let xs: Vec = corners.iter().flatten().map(|v| v[0]).collect(); + let ys: Vec = corners.iter().flatten().map(|v| v[1]).collect(); + let (min_x, min_y) = (xs.iter().cloned().fold(f32::INFINITY, f32::min), ys.iter().cloned().fold(f32::INFINITY, f32::min)); + let (w_pt, h_pt) = ( + xs.iter().cloned().fold(f32::NEG_INFINITY, f32::max) - min_x, + ys.iter().cloned().fold(f32::NEG_INFINITY, f32::max) - min_y, + ); + json!({ + "text": w.text, + "box": [min_x, min_y, w_pt, h_pt], + "confidence": w.confidence.map(|c| (c * 10.0).round() / 10.0), + "band": w.confidence.map(|c| band_name(pdfcraft_engine::ocr::band_for(c))), + "suspect": suspect, + "source": w.source, + }) + }) + .collect(); + let mut page = + json!({ "page": p.page + 1, "already_on_page": p.already_on_page, "words": words, "mean_confidence": p.mean_confidence().map(|c| (c * 10.0).round() / 10.0) }); + if let Some(why) = &p.skipped { + page["skipped"] = json!(why); + } + page }) .collect(); - Ok(json!({ "words": found.iter().map(|p| p.words.len()).sum::(), "pages": list })) + let mut out = json!({ "words": found.iter().map(|p| p.words.len()).sum::(), "suspects": suspects, "pages": list }); + if retain { + self.next_ocr_review = + self.next_ocr_review.checked_add(1).filter(|n| *n <= i64::MAX as u64).ok_or_else(|| failed("OCR review handles exhausted"))?; + out["review"] = json!(self.next_ocr_review); + self.ocr_reviews.insert(id, crate::OcrReview { handle: self.next_ocr_review, guard, pages: found }); + } + Ok(out) + } + + /// Apply only caller corrections to retained original boxes, through the stored guard. + pub(crate) fn ocr_review_apply(&mut self, a: &Args) -> Result { + let id = self.doc(a)?.id; + let handle = a.int("review")?; + let review = self + .ocr_reviews + .get(&id) + .filter(|r| i64::try_from(r.handle).ok() == Some(handle)) + .ok_or_else(|| failed("unknown OCR review for this document; call ocr_words with retain_review"))?; + if a.opt_bool("discard")?.unwrap_or(false) { + if a.get("corrections").is_some() { + return Err(ToolError::InvalidArgs("discard cannot include corrections".into())); + } + self.ocr_reviews.remove(&id); + return Ok(json!({ "discarded": handle })); + } + if let Some(why) = self.session.stale_reason(&review.guard) { + return Err(failed(why)); + } + let corrections = a.get("corrections").map(|v| v.as_array().ok_or_else(|| Args::wrong("corrections", "an array"))).transpose()?; + if corrections.is_some_and(|c| c.len() > 65536) { + return Err(failed("at most 65536 OCR corrections are allowed")); + } + // Prevalidate and cap every replacement before cloning the retained result. + let mut edits = Vec::new(); + let mut bytes = 0usize; + let mut seen = std::collections::BTreeSet::new(); + let page_indices: std::collections::BTreeMap<_, _> = review.pages.iter().enumerate().map(|(i, p)| (p.page, i)).collect(); + for item in corrections.into_iter().flatten() { + if !item.is_object() || item.as_object().is_some_and(|o| o.keys().any(|k| !matches!(k.as_str(), "page" | "word" | "text"))) { + return Err(ToolError::InvalidArgs("corrections accept only page, word and text".into())); + } + let c = Args(item); + let page = + usize::try_from(c.int("page")?).ok().and_then(|p| p.checked_sub(1)).ok_or_else(|| Args::wrong("page", "a 1-based position"))?; + let word = + usize::try_from(c.int("word")?).ok().and_then(|w| w.checked_sub(1)).ok_or_else(|| Args::wrong("word", "a 1-based position"))?; + let text = c.str("text")?; + bytes = bytes.checked_add(text.len()).filter(|n| *n <= 4 * 1024 * 1024).ok_or_else(|| failed("OCR corrections exceed 4 MiB"))?; + if !seen.insert((page, word)) { + return Err(failed("duplicate OCR word correction")); + } + let pi = page_indices.get(&page).copied().ok_or_else(|| failed("page is not in this OCR review"))?; + if review.pages.get(pi).and_then(|p| p.words.get(word)).is_none() { + return Err(failed("word is not in this OCR review")); + } + edits.push((pi, word, text.to_owned())); + } + let mut pages = review.pages.clone(); + for (pi, wi, text) in edits { + if let Some(w) = pages.get_mut(pi).and_then(|p| p.words.get_mut(wi)) { + w.text = text; + } + } + pdfcraft_engine::ocr::validate_ocr_words(pages.iter().flat_map(|p| &p.words)).map_err(failed)?; + for page in &mut pages { + page.words.retain(|w| !w.text.trim().is_empty()); + } + if pages.iter().all(|p| p.words.is_empty()) { + return Err(failed("the OCR review has no words to apply")); + } + let out = match a.opt_str("output_mode")?.unwrap_or("searchable") { + "searchable" => { + if a.get("out").is_some() { + return Err(ToolError::InvalidArgs("out is only for editable output".into())); + } + let count = self.session.apply_ocr_checked(id, &pages, &review.guard).map_err(failed)?; + json!({ "applied": count, "review": handle }) + } + "editable" => { + // The editable output writes the review's words out of the document: a + // document that forbids copying refuses here (the review is kept). + self.doc_to_copy(a)?; + let target = self.resolve(a.str("out")?, true)?; + if self + .session + .get(id) + .and_then(|d| d.path.as_deref()) + .and_then(|p| std::fs::canonicalize(p).ok()) + .as_ref() + .is_some_and(|source| std::fs::canonicalize(&target).as_ref().ok() == Some(source)) + { + return Err(failed("out is the source document; choose a new file")); + } + // A NEW file: an existing one is never replaced silently (the review is kept). + if target.exists() { + return Err(failed(format!("out already exists ({}); choose a path that does not exist yet", target.to_string_lossy()))); + } + let bytes = self.session.editable_ocr_bytes(id, &pages).map_err(failed)?; + write_atomic(&target, &bytes)?; + json!({ "out": target.to_string_lossy(), "review": handle, "words": pages.iter().map(|p| p.words.len()).sum::() }) + } + _ => return Err(ToolError::InvalidArgs("output_mode must be searchable or editable".into())), + }; + self.ocr_reviews.remove(&id); + Ok(out) + } + + /// Read/update the same per-language dictionary file used by OCR Verify. + pub(crate) fn ocr_dictionary(&mut self, a: &Args) -> Result { + use pdfcraft_engine::ocr::dictionary; + use std::io::Read as _; + for key in ["add", "remove"] { + if let Some(value) = a.get(key) { + let values = value.as_array().ok_or_else(|| Args::wrong(key, "an array of strings"))?; + if values.len() > 65536 { + return Err(failed("at most 65536 dictionary words are allowed")); + } + } + } + let add = if a.get("add").is_some() { a.strs("add")? } else { Vec::new() }; + let remove = if a.get("remove").is_some() { a.strs("remove")? } else { Vec::new() }; + let writing = a.get("add").is_some() || a.get("remove").is_some(); + // Updating sorts, lower-cases and rewrites the file, so this tool names dictionary files only: + // pointed at `.env` or a README it would corrupt it. + let file = std::path::Path::new(a.str("path")?).file_name().and_then(|n| n.to_str()).unwrap_or(""); + let plain = file.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.')); + if file.len() > 64 || !plain || !file.starts_with("ocr-dictionary") || !file.ends_with(".txt") { + return Err(failed("path must name a dictionary file: ocr-dictionary-.txt (letters, digits, '-', '_' and '.' only)")); + } + let path = self.resolve(a.str("path")?, writing)?; + let mut input = String::new(); + match std::fs::File::open(&path) { + Ok(file) => { + file.take((dictionary::MAX_BYTES + 1) as u64).read_to_string(&mut input).map_err(|e| failed(format!("OCR dictionary: {e}")))?; + } + Err(e) if e.kind() == std::io::ErrorKind::NotFound && writing => {} + Err(e) => return Err(failed(format!("OCR dictionary: {e}"))), + } + let text = dictionary::update(&input, &add, &remove).map_err(failed)?; + if writing { + write_atomic(&path, text.as_bytes())?; + } + let words = dictionary::words(&text).map_err(failed)?; + Ok(json!({ "path": path.to_string_lossy(), "count": words.len(), "words": words })) } - /// Whether text recognition's models are installed, the directories searched for them (in - /// order), and the languages it reads, each as a code and a name. pub(crate) fn ocr_status(&self) -> Result { use pdfcraft_engine::ocr; let dirs: Vec = ocr::Models::search_dirs().iter().map(|d| d.to_string_lossy().into_owned()).collect(); let langs: Vec = ocr::LANGUAGES.iter().map(|(c, n)| json!({ "code": c, "name": n })).collect(); - Ok(json!({ "available": ocr::available(), "search_dirs": dirs, "languages": langs })) + let engines: Vec = ocr::engines_status() + .iter() + .map(|e| { + let mut v = json!({ "id": e.id, "available": e.available }); + if let Some(p) = &e.program { + v["path"] = json!(p); + } + if let Some(vv) = &e.version { + v["version"] = json!(vv); + } + if let Some(why) = &e.why_not { + v["unavailable"] = json!(why); + } + v + }) + .collect(); + Ok(json!({ "available": ocr::available(), "search_dirs": dirs, "languages": langs, "engines": engines })) } } diff --git a/crates/automation/src/batch.rs b/crates/automation/src/batch.rs index decb18d2b7..dd5ca6aec0 100644 --- a/crates/automation/src/batch.rs +++ b/crates/automation/src/batch.rs @@ -84,8 +84,8 @@ impl<'a> Recipe<'a> { match *step { "convert" => bytes = session.convert_to_pdf(name, &bytes)?.1, "ocr" => { - let engine = ocr::engine().map_err(failed)?; - let result = ocr::recognize_file(name, bytes, None, self.ocr.clone(), &engine, |_, _| true).map_err(failed)?; + let recognizers = ocr::recognizers(&self.ocr).map_err(failed)?; + let result = ocr::recognize_file(name, bytes, None, self.ocr.clone(), &recognizers, |_, _| true).map_err(failed)?; log.push( json!({"step":"ocr","words":result.words(),"skipped_pages":result.pages.iter().filter(|p| p.skipped.is_some()).count()}), ); diff --git a/crates/automation/src/bates.rs b/crates/automation/src/bates.rs index 3cd1d7849e..5267716e85 100644 --- a/crates/automation/src/bates.rs +++ b/crates/automation/src/bates.rs @@ -199,7 +199,7 @@ impl Automation { } /// Publish a PDF atomically while refusing a target created by another process after preflight. -fn publish_new(path: &Path, bytes: &[u8]) -> Result<()> { +pub(super) fn publish_new(path: &Path, bytes: &[u8]) -> Result<()> { let parent = path.parent().ok_or_else(|| failed("output has no parent directory"))?; let mut temp = tempfile::NamedTempFile::new_in(parent).map_err(failed)?; temp.write_all(bytes).map_err(failed)?; diff --git a/crates/automation/src/dispatch.rs b/crates/automation/src/dispatch.rs index 71a979b637..ebe2a93526 100644 --- a/crates/automation/src/dispatch.rs +++ b/crates/automation/src/dispatch.rs @@ -812,6 +812,9 @@ impl Automation { "form_set_actions" => self.form_set_actions(&a)?, "ocr_status" => self.ocr_status()?, "ocr_recognize_files" => self.ocr_recognize_files(&a)?, + "ocr_words" => self.ocr_words(&a)?, + "ocr_review_apply" => self.ocr_review_apply(&a)?, + "ocr_dictionary" => self.ocr_dictionary(&a)?, "accessibility_report" => self.a11y_report(&a)?, "accessibility_review_record" => self.a11y_review_record(&a)?, "accessibility_review_save" => self.a11y_review_save(&a)?, diff --git a/crates/automation/src/documents.rs b/crates/automation/src/documents.rs index 979b041a73..f8c7988bbc 100644 --- a/crates/automation/src/documents.rs +++ b/crates/automation/src/documents.rs @@ -120,6 +120,7 @@ impl Automation { self.texts.remove(&id); self.listed.retain(|(d, _), _| *d != id); self.evicted.remove(&id); + self.ocr_reviews.remove(&id); Ok(json!({ "closed": id.0 })) } diff --git a/crates/automation/src/lib.rs b/crates/automation/src/lib.rs index 520d836b01..7e4443d244 100644 --- a/crates/automation/src/lib.rs +++ b/crates/automation/src/lib.rs @@ -76,6 +76,12 @@ pub(crate) use paths::{read_input, read_input_bytes}; #[cfg(test)] use paths::{write_atomic_synced, write_atomic_with}; +struct OcrReview { + handle: u64, + guard: pdfcraft_engine::ocr::OcrGuard, + pages: Vec, +} + /// A headless PdfCraft session driven by tool calls. pub struct Automation { session: Session, @@ -92,6 +98,9 @@ pub struct Automation { /// Documents that lost a listing to the cap. A page of one with no listing is refused rather /// than read as "a number is a position", which the page may no longer match. evicted: HashSet, + /// Review sessions held for `ocr_words` (a guard and the pages read), by document. + ocr_reviews: HashMap, + next_ocr_review: u64, } impl Default for Automation { @@ -110,6 +119,8 @@ impl Automation { texts: TextCache::default(), listed: HashMap::new(), evicted: HashSet::new(), + ocr_reviews: HashMap::new(), + next_ocr_review: 0, } } diff --git a/crates/automation/src/tools.rs b/crates/automation/src/tools.rs index 54d94ff4bf..af5146377a 100644 --- a/crates/automation/src/tools.rs +++ b/crates/automation/src/tools.rs @@ -85,6 +85,12 @@ fn schema(props: Value, required: &[&str]) -> Value { json!({ "type": "object", "properties": props, "required": required, "additionalProperties": false }) } +/// The language codes the engines read, as the `language` enum — from the engine's own table, +/// so the schema and the validator can never disagree. +fn language_enum() -> Value { + Value::Array(pdfcraft_engine::ocr::LANGUAGES.iter().map(|(code, _)| json!(code)).collect()) +} + struct T { name: &'static str, title: &'static str, @@ -1373,31 +1379,79 @@ pub fn tools() -> Vec { )), t("js_enabled", "JavaScript on or off", "Preferences ▸ JavaScript ▸ Enable Acrobat JavaScript: set it with `enabled`, or read it. With JavaScript off, field scripts other than Acrobat's AF calls don't run.") .with(schema(json!({ "enabled": { "type": "boolean" } }), &[])), - t("ocr_recognize", "Recognize text (OCR)", "Scan & OCR ▸ Recognize text: render pages, read the words in them and add them as invisible text over the page image, so scanned pages become searchable and selectable (a searchable image; the image is not changed). Pages that already have text are skipped unless skip_text_pages is false; then pictures on those pages are read too, and recognised words that the page's own text already has (an earlier OCR layer included) are left out and counted as already_on_page. Returns each page's recognised text (left out when the document's security doesn't allow copying), word count or why it was skipped. Needs the OCR models (ocr_status). Undoable as one step.") + t("ocr_recognize", "Recognize text (OCR)", "Scan & OCR ▸ Recognize text: render pages, read the words in them and add them as invisible text over the page image, so scanned pages become searchable and selectable (a searchable image; the image is not changed). Pages that already have text are skipped unless skip_text_pages or force_ocr say otherwise; words overlapping existing text are omitted and counted as already_on_page. With output_mode \"editable\" nothing is applied: a NEW document of visible text at the recognised boxes is written to out (which must not be the source; empty pages stay blank). engine and strategy choose the readers (an engine that cannot run is an error named up front); the four preprocessing flags clean the raster first. region reads just that part of one page ([x0, y0, x1, y1] in points from the top-left of the displayed page). Returns each page's recognised text, word count or why it was skipped. Needs the engines (ocr_status). Searchable output is undoable as one step.") .cmd("ocr.recognize") .with(schema( json!({ "doc": doc(), "pages": pages("to recognise (default: all)"), "dpi": { "type": "number", "minimum": 72, "maximum": 600, "description": "Resolution pages are read at (default 300; a scanned page is read at most at its own resolution)." }, - "language": { "type": "string", "enum": ["en"], "description": "Document language (default en)." }, - "skip_text_pages": { "type": "boolean", "description": "Leave pages that already contain text alone (default true). false also reads pictures on such pages, without adding their text again." }, + "language": { "type": "string", "enum": language_enum(), "description": "Document language (default en)." }, + "engine": { "type": "string", "enum": ["auto", "ocrs", "tesseract"], "description": "Which engine reads the pages (default ocrs). auto: ocrs first and tesseract as the second engine, combined by strategy (only the engines that can run take part)." }, + "strategy": { "type": "string", "enum": ["primary", "confidence", "rover"], "description": "How two engines' readings combine: primary = the first engine alone; confidence = the second engine joins when the first reading's mean confidence is under 70 or unknown; rover = both always read, merged. Default: confidence with engine auto, primary otherwise." }, + "output_mode": { "type": "string", "enum": ["searchable", "editable"], "description": "searchable (default): invisible text over the pages of the open document. editable: a NEW _ocr-style file of visible text written to out; the source is not touched." }, + "out": { "type": "string", "description": "Where the editable output is written (required with output_mode editable; a new file: never the source path, never an existing file)." }, + "region": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4, "description": "[x0, y0, x1, y1] in points from the top-left of the displayed page: read only that part. Needs exactly one page." }, + "skip_text_pages": { "type": "boolean", "description": "Leave pages that already contain text alone (default true)." }, + "skip_text_files": { "type": "boolean", "description": "Batch: leave whole files that already contain text alone (they come back skipped, not read)." }, + "force_ocr": { "type": "boolean", "description": "Read pages and files even when the skip options would leave them alone (default false)." }, + "auto_rotate": { "type": "boolean", "description": "Turn pages scanned sideways or upside down upright before reading (default false)." }, + "deskew": { "type": "boolean", "description": "Straighten skewed text lines before reading (default false)." }, + "denoise": { "type": "boolean", "description": "A 3×3 median filter against scan speckle (default false)." }, + "binarize": { "type": "boolean", "description": "Sauvola binarization: ink to black, paper to white (default false)." }, }), &["doc"], )), - t("ocr_recognize_files", "Recognize text in multiple files", "Scan & OCR ▸ Recognize text ▸ In multiple files: read every page of each PDF in paths and write the searchable result into folder under the same name (pages that already have text are left alone unless skip_text_pages is false, as in ocr_recognize). Two inputs that would write the same output file are refused before anything is read or written. Returns, per file, the output path, word count and skipped pages, or the error.") + t("ocr_words", "Words of a recognition (for review)", "The words a recognition finds, for inspection without the GUI: per page, each word's text, its box ([x, y, w, h] in points from the top-left of the displayed page), its confidence and band (high >= 90, medium 70–89, low < 70), which engine read it, and whether it is a suspect (low band). suspects counts the words a reviewer should look at first. Changes nothing on the document. retain_review returns a session-local handle for ocr_review_apply (one per document, at most eight; a new retained recognition replaces the old one). Needs the engines (ocr_status).") + .ro() + .with(schema( + json!({ + "doc": doc(), + "retain_review": { "type": "boolean" }, + "region": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4 }, + "pages": pages("to read (default: all)"), + "dpi": { "type": "number", "minimum": 72, "maximum": 600 }, + "language": { "type": "string", "enum": language_enum() }, + "engine": { "type": "string", "enum": ["auto", "ocrs", "tesseract"] }, + "strategy": { "type": "string", "enum": ["primary", "confidence", "rover"] }, + "auto_rotate": { "type": "boolean" }, + "deskew": { "type": "boolean" }, + "denoise": { "type": "boolean" }, + "binarize": { "type": "boolean" }, + "skip_text_pages": { "type": "boolean", "description": "Leave pages that already contain text alone (default true)." }, + "force_ocr": { "type": "boolean", "description": "Read pages even when the skip options would leave them alone (default false)." }, + }), + &["doc"], + )), + t("ocr_review_apply", "Apply or discard reviewed OCR words", "Accept corrected recognition from ocr_words retain_review. Corrections use 1-based page/word positions and text only; empty text removes a word, omitted words keep their original text and boxes. The stored document guard refuses stale, foreign or consumed reviews. Searchable output is one undoable step; editable requires out and writes a new document (out must not exist yet; the review is kept when it does). discard releases the review without applying it. Input is capped at 65536 words and 4 MiB, never truncated.") + .cmd("ocr.correct") + .with(schema(json!({ "doc": doc(), "review": { "type": "integer", "minimum": 1 }, "discard": { "type": "boolean" }, "output_mode": { "type": "string", "enum": ["searchable", "editable"] }, "out": { "type": "string" }, + "corrections": { "type": "array", "maxItems": 65536, "items": { "type": "object", "additionalProperties": false, "properties": { "page": { "type": "integer", "minimum": 1 }, "word": { "type": "integer", "minimum": 1 }, "text": { "type": "string" } }, "required": ["page", "word", "text"] } } }), &["doc", "review"])), + t("ocr_dictionary", "Read or update an OCR review dictionary", "Read or add/remove words in an OCR Verify dictionary file (one case-insensitive word per UTF-8 line). Choose its per-language ocr-dictionary-.txt path. Paths honor root; updates are atomic. Refuses invalid words, over 65536 entries or over 4 MiB. No document changes.") + .destructive() + .cmd("ocr.correct") + .with(schema(json!({ "path": { "type": "string" }, "add": { "type": "array", "maxItems": 65536, "items": { "type": "string" } }, "remove": { "type": "array", "maxItems": 65536, "items": { "type": "string" } } }), &["path"])), + t("ocr_recognize_files", "Recognize text in multiple files", "Scan & OCR ▸ Recognize text ▸ In multiple files: read every page of each PDF in paths and write the searchable result into folder under the same name. skip_text_files leaves whole files that already contain text alone (they are their own skipped bucket, nothing is written); skip_text_pages leaves single text pages alone; force_ocr reads regardless of both. Every output name is checked before the first page is read: two paths sharing a file name (from different folders) are refused, and a file of that name already in folder aborts the batch, with the names in the error. Each written file carries a low_confidence note when words at or under 60 confidence sit on it (1-based sorted pages). Returns, per file, the output path, word count, skipped pages or the error, plus recognized/words/skipped_files totals. Needs the engines (ocr_status).") .cmd("ocr.recognize_batch") .with(schema( json!({ "paths": { "type": "array", "items": { "type": "string" }, "minItems": 1 }, "folder": { "type": "string" }, "dpi": { "type": "number", "minimum": 72, "maximum": 600 }, - "language": { "type": "string", "enum": ["en"] }, - "skip_text_pages": { "type": "boolean", "description": "Leave pages that already contain text alone (default true); see ocr_recognize." }, + "language": { "type": "string", "enum": language_enum() }, + "engine": { "type": "string", "enum": ["auto", "ocrs", "tesseract"] }, + "strategy": { "type": "string", "enum": ["primary", "confidence", "rover"] }, + "skip_text_pages": { "type": "boolean", "description": "Leave pages that already contain text alone (default true)." }, + "skip_text_files": { "type": "boolean", "description": "Leave whole files that already contain text alone (default false)." }, + "force_ocr": { "type": "boolean", "description": "Read even where the skip options would leave things alone (default false)." }, + "auto_rotate": { "type": "boolean" }, + "deskew": { "type": "boolean" }, + "denoise": { "type": "boolean" }, + "binarize": { "type": "boolean" }, }), &["paths", "folder"], )), - t("ocr_status", "OCR status", "Whether text recognition is available (its models are installed: release packages include them; a source build fetches them with `cargo xtask models`; PDFCRAFT_MODELS overrides), where it looks for them, and the languages it reads.") + t("ocr_status", "OCR status", "Whether text recognition is available, where it looks for models (release packages include them; a source build fetches them with `cargo xtask models`; PDFCRAFT_MODELS overrides), its languages, and the engines: ocrs (built in, models) and tesseract (an installed program, with its path).") .ro() .cmd("ocr.recognize") .with(schema(json!({}), &[])), diff --git a/crates/automation/tests/it/automation.rs b/crates/automation/tests/it/automation.rs index 5ed32360fd..96649c8e80 100644 --- a/crates/automation/tests/it/automation.rs +++ b/crates/automation/tests/it/automation.rs @@ -843,10 +843,11 @@ fn arabic_ligature_actual_text_is_one_editable_word_through_save_reopen() { } /// Each of these writes to its required `path` and replaces an existing file there, so the MCP -/// annotations must not tell clients the call is read-only or harmless (#130). +/// annotations must not tell clients the call is read-only or harmless (#130). The OCR +/// dictionary's updates rewrite the file in place (lower-cased and sorted), so it counts too. #[test] fn file_writing_tools_are_not_read_only() { - for name in ["doc_export_data", "accessibility_report", "image_save"] { + for name in ["doc_export_data", "accessibility_report", "image_save", "ocr_dictionary"] { let t = tools().into_iter().find(|t| t.name == name).unwrap(); assert!(!t.read_only, "{name} writes a file but advertises read-only"); assert!(t.destructive, "{name} overwrites its path but advertises non-destructive"); @@ -2580,6 +2581,16 @@ fn copy_restricted_text_stays_in_the_document() { } let restyled = ok(&mut a, "text_edit", json!({ "doc": re, "page": 1, "paragraph": 1, "size": 30 })); assert!(!restyled.to_string().contains("Page"), "{restyled}"); + // A path that would carry the page's words OUT of the document — the editable output, or + // the review word list — is refused on a copy-restricted document (no engines needed). + match a.call("ocr_recognize", &json!({ "doc": re, "output_mode": "editable", "out": "copy.pdf" })) { + Err(ToolError::Failed(m)) => assert!(m.contains("don't allow copying"), "{m}"), + other => panic!("the editable output is refused: {other:?}"), + } + match a.call("ocr_words", &json!({ "doc": re })) { + Err(ToolError::Failed(m)) => assert!(m.contains("don't allow copying"), "{m}"), + other => panic!("ocr_words is refused: {other:?}"), + } if ok(&mut a, "ocr_status", json!({}))["available"] != true { eprintln!("skipped the OCR part: OCR models not installed"); return; @@ -7502,6 +7513,8 @@ fn ocr_tools_make_a_scan_searchable() { let mut a = auto(&dir); let status = ok(&mut a, "ocr_status", json!({})); assert_eq!(status["languages"][0]["code"], "en"); + assert_eq!(status["engines"][0]["id"], "ocrs", "{status}"); + assert!(status["engines"][1]["id"] == "tesseract" && status["engines"][1]["available"].is_boolean(), "{status}"); if status["available"] != true { eprintln!("skipped: OCR models not installed"); return; @@ -7516,6 +7529,64 @@ fn ocr_tools_make_a_scan_searchable() { let again = ok(&mut a, "ocr_recognize", json!({ "doc": scan, "pages": [1] })); assert!(again["pages"][0]["skipped"].is_string(), "{again}"); assert!(matches!(a.call("ocr_recognize", &json!({ "doc": scan, "language": "xx" })), Err(ToolError::InvalidArgs(_)))); + // An engine that cannot run is named up front, before anything is read. + if status["engines"][1]["available"] != true { + let why = a.call("ocr_recognize", &json!({ "doc": scan, "engine": "tesseract", "skip_text_pages": false })).unwrap_err(); + assert!(matches!(why, ToolError::Failed(_)), "{why}"); + assert!(why.to_string().contains("tesseract"), "{why}"); + } + // ocr_words: the recognition's words with confidence, band and suspects, changing nothing. + // ocrs reports no confidences, so `auto` + a confidence merge is asked for when the second + // engine is there to supply them; without it confidence and band are honestly null. + ok(&mut a, "doc_close", json!({ "doc": scan, "discard_changes": true })); + let scan = ok(&mut a, "doc_create", json!({ "from": "images", "paths": [png.to_string_lossy()] }))["doc"].as_u64().unwrap(); + let mut args = json!({ "doc": scan }); + if status["engines"][1]["available"] == true { + args["engine"] = json!("auto"); + args["strategy"] = json!("confidence"); + } + let words = ok(&mut a, "ocr_words", args); + assert!(words["words"].as_u64().unwrap() >= 4, "{words}"); + let first = &words["pages"][0]["words"][0]; + assert!(first["text"].is_string() && first["box"].as_array().is_some_and(|b| b.len() == 4), "{words}"); + if status["engines"][1]["available"] == true { + let with_confidence = words["pages"][0]["words"] + .as_array() + .expect("words") + .iter() + .filter(|w| w["confidence"].is_number() && w["band"].as_str().is_some()) + .count(); + assert!(with_confidence >= 4, "the second engine's confidences arrive: {words}"); + } else { + assert!(first["confidence"].is_null() && first["band"].is_null() && first["suspect"] == false, "{words}"); + } + assert!(a.call("edit_undo", &json!({ "doc": scan })).is_err(), "ocr_words changes nothing, so there is no undo"); +} + +/// An editable-text recognition writes a NEW file of visible text, never the source. +#[test] +fn ocr_recognize_editable_output_writes_visible_text() { + let dir = workdir("ocr-editable"); + let mut a = auto(&dir); + if ok(&mut a, "ocr_status", json!({}))["available"] != true { + eprintln!("skipped: OCR models not installed"); + return; + } + let text = ok(&mut a, "doc_create", json!({ "from": "text", "text": "Editable output words" }))["doc"].as_u64().unwrap(); + ok(&mut a, "doc_export_images", json!({ "doc": text, "folder": "scan", "dpi": 150, "pages": [1] })); + let png = std::fs::read_dir(dir.join("scan")).unwrap().next().unwrap().unwrap().path(); + let scan = ok(&mut a, "doc_create", json!({ "from": "images", "paths": [png.to_string_lossy()] }))["doc"].as_u64().unwrap(); + ok(&mut a, "doc_save", json!({ "doc": scan, "path": "src/scan.pdf" })); + // The output never lands on the source. + let why = a.call("ocr_recognize", &json!({ "doc": scan, "pages": [1], "output_mode": "editable", "out": "src/scan.pdf" })).unwrap_err(); + assert!(why.to_string().contains("source"), "{why}"); + let r = ok(&mut a, "ocr_recognize", json!({ "doc": scan, "pages": [1], "output_mode": "editable", "out": "out/ocr.pdf" })); + assert!(r["words"].as_u64().unwrap() >= 3, "{r}"); + let ocr = ok(&mut a, "doc_open", json!({ "path": "out/ocr.pdf" }))["doc"].as_u64().unwrap(); + // The text is visible (the engine may misread a letter at this dpi — look for a word that + // survives). + let found = ok(&mut a, "text_find", json!({ "doc": ocr, "query": "output" })); + assert!(found["count"].as_u64() >= Some(1), "the new file's text is visible: {found}"); } /// #1028: `skip_text_pages: false` reads a picture on a page that has its own text, and the @@ -7558,6 +7629,46 @@ fn ocr_recognize_files_refuses_paths_outside_the_root() { assert!(!dir.join("out").exists(), "nothing was started"); } +/// Two inputs of one file name from different folders would write the same output in the +/// batch folder: the batch is refused up front, with or without the OCR engines, naming the +/// file, and nothing is written. +#[test] +fn ocr_recognize_files_refuses_inputs_that_share_a_file_name() { + let dir = workdir("ocr-files-same-name"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "doc_save", json!({ "doc": doc, "path": "one/scan.pdf" })); + ok(&mut a, "doc_save", json!({ "doc": doc, "path": "two/scan.pdf" })); + let r = a.call("ocr_recognize_files", &json!({ "paths": ["one/scan.pdf", "two/scan.pdf"], "folder": "out" })); + assert!(matches!(&r, Err(ToolError::InvalidArgs(m)) if m.contains("scan.pdf") && m.contains("overwrite")), "{r:?}"); + assert!(std::fs::read_dir(dir.join("out")).map_or(true, |mut d| d.next().is_none()), "nothing was written"); +} + +/// `out` belongs to the editable output. A searchable run (the default) writes into the open +/// document, so an `out` is refused rather than silently ignored, with or without the engines. +#[test] +fn ocr_recognize_refuses_an_out_that_nothing_would_use() { + let dir = workdir("ocr-out-ignored"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + let r = a.call("ocr_recognize", &json!({ "doc": doc, "out": "../outside.pdf" })); + assert!(matches!(&r, Err(ToolError::InvalidArgs(m)) if m.contains("editable")), "{r:?}"); + assert!(!dir.join("../outside.pdf").exists()); +} + +/// The editable output is a NEW file: an existing `out` is refused before anything runs (with +/// or without the engines), and the file is left as it was. +#[test] +fn ocr_recognize_editable_output_never_replaces_an_existing_file() { + let dir = workdir("ocr-editable-exists"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + std::fs::write(dir.join("taken.pdf"), b"keep me").unwrap(); + let r = a.call("ocr_recognize", &json!({ "doc": doc, "output_mode": "editable", "out": "taken.pdf" })); + assert!(matches!(&r, Err(e) if e.to_string().contains("already exists")), "{r:?}"); + assert_eq!(std::fs::read(dir.join("taken.pdf")).unwrap(), b"keep me"); +} + #[test] fn ocr_recognize_files_writes_searchable_copies() { let dir = workdir("ocr-files"); @@ -7571,9 +7682,22 @@ fn ocr_recognize_files_writes_searchable_copies() { let png = std::fs::read_dir(dir.join("scan")).unwrap().next().unwrap().unwrap().path(); let scan = ok(&mut a, "doc_create", json!({ "from": "images", "paths": [png.to_string_lossy()] }))["doc"].as_u64().unwrap(); ok(&mut a, "doc_save", json!({ "doc": scan, "path": "in/scan.pdf" })); + // A file that already has text: the skip-files option passes it over as its own bucket. + let typed = ok(&mut a, "doc_create", json!({ "from": "text", "text": "Already typed" }))["doc"].as_u64().unwrap(); + ok(&mut a, "doc_save", json!({ "doc": typed, "path": "in/typed.pdf" })); + // Collision pre-flight: the existing out/scan.pdf (written below) stops a rerun. let r = ok(&mut a, "ocr_recognize_files", json!({ "paths": ["in/scan.pdf", "missing.pdf"], "folder": "out" })); assert!(r["files"][0]["words"].as_u64().unwrap() >= 3, "{r}"); assert!(r["files"][1]["error"].is_string(), "{r}"); + let why = a.call("ocr_recognize_files", &json!({ "paths": ["in/scan.pdf"], "folder": "out" })).unwrap_err(); + assert!(matches!(why, ToolError::Failed(_)), "{why}"); + assert!(why.to_string().contains("scan.pdf"), "{why}"); + // The skip-files bucket and the low-confidence note. + let r = ok(&mut a, "ocr_recognize_files", json!({ "paths": ["in/typed.pdf", "in/scan.pdf"], "folder": "out2", "skip_text_files": true })); + assert_eq!(r["skipped_files"].as_u64(), Some(1), "{r}"); + assert!(r["files"][0]["skipped"].is_string(), "{r}"); + assert_eq!(r["recognized"].as_u64(), Some(1), "{r}"); + assert!(r["files"][1]["output"].is_string(), "{r}"); let out = ok(&mut a, "doc_open", json!({ "path": "out/scan.pdf" }))["doc"].as_u64().unwrap(); let found = ok(&mut a, "text_find", json!({ "doc": out, "query": "recognition" })); assert!(found.to_string().contains("\"page\""), "{found}"); @@ -7583,18 +7707,6 @@ fn ocr_recognize_files_writes_searchable_copies() { /// the second output replaces the first while both results report success. The batch is refused /// up front, naming the file, before the folder is created, anything is read or the OCR engines /// load, so the outcome is the same with or without the models installed. -#[test] -fn ocr_recognize_files_refuses_inputs_that_share_a_file_name() { - let dir = workdir("ocr-files-same-name"); - let mut a = auto(&dir); - let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); - ok(&mut a, "doc_save", json!({ "doc": doc, "path": "one/scan.pdf" })); - ok(&mut a, "doc_save", json!({ "doc": doc, "path": "two/scan.pdf" })); - let r = a.call("ocr_recognize_files", &json!({ "paths": ["one/scan.pdf", "two/scan.pdf"], "folder": "out" })); - assert!(matches!(&r, Err(ToolError::InvalidArgs(m)) if m.contains("scan.pdf")), "{r:?}"); - assert!(!dir.join("out").exists(), "nothing was started: the output folder was not created"); -} - /// File names are compared without regard to case on Windows and macOS (`scan.pdf` and `Scan.PDF` /// are one file there), and exactly elsewhere, where they are two files and the batch is not /// refused. @@ -10770,6 +10882,162 @@ fn simple_rule_scan_drawn(content: &[u8]) -> Vec { write_full(&doc, &SaveOptions::default()).unwrap() } +/// Reviewing headlessly: corrections keep the boxes, a stale or foreign review is refused, and +/// applying consumes the handle (the pages already accepted were applied once). +#[test] +fn ocr_review_corrections_keep_boxes_are_guarded_and_apply_once() { + fn render(a: &mut Automation, doc: u64) -> Vec { + let mut result = a.call("page_render", &json!({ "doc": doc, "page": 1, "dpi": 72 })).unwrap(); + match result.remove(0) { + Content::Png { data, .. } => data, + other => panic!("expected a rendered page, got {other:?}"), + } + } + let dir = workdir("ocr-review"); + let mut a = auto(&dir); + if ok(&mut a, "ocr_status", json!({}))["available"] != true { + eprintln!("skipped: OCR models not installed"); + return; + } + let text = ok(&mut a, "doc_create", json!({ "from": "text", "text": "Review these original words" }))["doc"].as_u64().unwrap(); + ok(&mut a, "doc_export_images", json!({ "doc": text, "folder": "scan", "dpi": 150, "pages": [1] })); + let png = std::fs::read_dir(dir.join("scan")).unwrap().next().unwrap().unwrap().path(); + let scan = ok(&mut a, "doc_create", json!({ "from": "images", "paths": [png.to_string_lossy()] }))["doc"].as_u64().unwrap(); + let before = render(&mut a, scan); + let found = ok(&mut a, "ocr_words", json!({ "doc": scan, "retain_review": true })); + assert!(found["words"].as_u64().unwrap() >= 3, "{found}"); + let review = found["review"].as_u64().unwrap(); + assert!(page_text(&mut a, scan).iter().all(String::is_empty)); + for args in [ + json!({ "doc": text, "review": review }), + json!({ "doc": scan, "review": review, "corrections": [{"page":1,"word":999999,"text":"bad"}] }), + json!({ "doc": scan, "review": review, "corrections": [{"page":1,"word":1,"text":"bad","origin":[0,0]}] }), + json!({ "doc": scan, "review": review, "corrections": [{"page":1,"word":1,"text":"a".repeat(4*1024*1024+1)}] }), + json!({ "doc": scan, "review": review, "corrections": [{"page":1,"word":1,"text":"a"},{"page":1,"word":1,"text":"b"}] }), + ] { + assert!(a.call("ocr_review_apply", &args).is_err()); + assert!(page_text(&mut a, scan).iter().all(String::is_empty)); + } + let result = ok( + &mut a, + "ocr_review_apply", + json!({ "doc": scan, "review": review, "corrections": [{"page":1,"word":1,"text":"日本語"},{"page":1,"word":2,"text":""}] }), + ); + assert!(result["applied"].as_u64().unwrap() >= 2); + let extracted = page_text(&mut a, scan).join("\n"); + assert!(extracted.contains("日本語"), "{extracted}"); + assert_eq!(render(&mut a, scan), before, "corrected text remains invisible at its original geometry"); + assert!(a.call("ocr_review_apply", &json!({ "doc": scan, "review": review })).is_err(), "consumed reviews cannot be replayed"); + ok(&mut a, "edit_undo", json!({ "doc": scan })); + assert!(page_text(&mut a, scan).iter().all(String::is_empty)); + assert!(a.call("edit_undo", &json!({ "doc": scan })).is_err(), "review acceptance is one undo step"); + + let found = ok(&mut a, "ocr_words", json!({ "doc": scan, "retain_review": true })); + let stale = found["review"].as_u64().unwrap(); + ok(&mut a, "page_rotate", json!({ "doc": scan, "pages": [1], "degrees": 90 })); + let error = a.call("ocr_review_apply", &json!({ "doc": scan, "review": stale })).unwrap_err().to_string(); + assert!(error.contains("changed since recognition"), "{error}"); + assert!(page_text(&mut a, scan).iter().all(String::is_empty)); + ok(&mut a, "ocr_review_apply", json!({ "doc": scan, "review": stale, "discard": true })); + assert!(a.call("ocr_review_apply", &json!({ "doc": scan, "review": stale })).is_err()); + ok(&mut a, "edit_undo", json!({ "doc": scan })); + + let first = ok(&mut a, "ocr_words", json!({ "doc": scan, "retain_review": true }))["review"].as_u64().unwrap(); + let second = ok(&mut a, "ocr_words", json!({ "doc": scan, "retain_review": true }))["review"].as_u64().unwrap(); + assert_ne!(first, second); + assert!(a.call("ocr_review_apply", &json!({ "doc": scan, "review": first })).is_err()); + ok( + &mut a, + "ocr_review_apply", + json!({ "doc": scan, "review": second, "output_mode": "editable", "out": "reviewed.pdf", "corrections": [{"page":1,"word":1,"text":"Corrected"}] }), + ); + assert!(page_text(&mut a, scan).iter().all(String::is_empty)); + let output = ok(&mut a, "doc_open", json!({ "path": "reviewed.pdf" }))["doc"].as_u64().unwrap(); + assert!(page_text(&mut a, output).join("\n").contains("Corrected")); + // The editable output is a new file: an existing one is refused and the review is kept. + let third = ok(&mut a, "ocr_words", json!({ "doc": scan, "retain_review": true }))["review"].as_u64().unwrap(); + let again = a.call("ocr_review_apply", &json!({ "doc": scan, "review": third, "output_mode": "editable", "out": "reviewed.pdf" })); + assert!(matches!(&again, Err(e) if e.to_string().contains("already exists")), "{again:?}"); + ok(&mut a, "ocr_review_apply", json!({ "doc": scan, "review": third, "discard": true })); +} + +#[test] +fn ocr_dictionary_updates_are_atomic_bounded_and_confined() { + let dir = workdir("ocr-dictionary"); + let mut a = auto(&dir); + let path = "ocr-dictionary-en.txt"; + let out = ok(&mut a, "ocr_dictionary", json!({ "path": path, "add": ["Quartz", "quartz", "日本語"] })); + assert_eq!(out["count"], 2); + assert_eq!(pdfcraft_engine::ocr::dictionary::words(&std::fs::read_to_string(dir.join(path)).unwrap()).unwrap().len(), 2); + let before = std::fs::read(dir.join(path)).unwrap(); + for args in [ + json!({ "path": path, "add": ["a\nb"] }), + json!({ "path": path, "add": ["x".repeat(1025)] }), + json!({ "path": path, "add": ["İ".repeat(512)] }), + json!({ "path": path, "add": vec!["x"; 65537] }), + json!({ "path": "../outside-ocr-dictionary.txt", "add": ["x"] }), + ] { + assert!(a.call("ocr_dictionary", &args).is_err()); + } + assert_eq!(std::fs::read(dir.join(path)).unwrap(), before); + assert_eq!(ok(&mut a, "ocr_dictionary", json!({ "path": path, "remove": ["QUARTZ"] }))["words"], json!(["日本語"])); + assert_eq!(ok(&mut a, "ocr_dictionary", json!({ "path": path }))["count"], 1); + std::fs::write(dir.join("ocr-dictionary-large.txt"), vec![b'a'; 4 * 1024 * 1024 + 1]).unwrap(); + let too_big = a.call("ocr_dictionary", &json!({ "path": "ocr-dictionary-large.txt", "add": ["x"] })).unwrap_err().to_string(); + assert!(!too_big.contains("must name a dictionary file"), "refused for its size, not its name: {too_big}"); + assert_eq!(std::fs::metadata(dir.join("ocr-dictionary-large.txt")).unwrap().len(), 4 * 1024 * 1024 + 1); +} + +/// The tool rewrites what it is pointed at (lower-cased, sorted), so it only takes dictionary file +/// names: another small text file under the root is neither read nor changed. +#[test] +fn ocr_dictionary_refuses_files_that_are_not_dictionaries() { + let dir = workdir("ocr-dictionary-names"); + let mut a = auto(&dir); + let env = b"TOKEN=Zebra +APPLE=1 +"; + for name in [".env", "notes.txt", "ocr-dictionary-en.md", "ocr-dictionary-e n.txt", "sub/readme.txt"] { + if let Some(parent) = std::path::Path::new(name).parent() { + std::fs::create_dir_all(dir.join(parent)).unwrap(); + } + std::fs::write(dir.join(name), env).unwrap(); + for args in [json!({ "path": name }), json!({ "path": name, "add": [] }), json!({ "path": name, "add": ["x"] })] { + let err = a.call("ocr_dictionary", &args).unwrap_err().to_string(); + assert!(err.contains("must name a dictionary file"), "{name} {args}: {err}"); + } + assert_eq!(std::fs::read(dir.join(name)).unwrap(), env, "{name} is untouched"); + } +} + +#[test] +fn ocr_retained_reviews_have_eight_document_cap_and_close_releases_slot() { + let dir = workdir("ocr-review-cap"); + let mut a = auto(&dir); + if ok(&mut a, "ocr_status", json!({}))["available"] != true { + eprintln!("skipped: OCR models not installed"); + return; + } + let mut docs = Vec::new(); + let mut handles = Vec::new(); + for _ in 0..9 { + let doc = ok(&mut a, "doc_create", json!({"from":"text", "text":"Already recognized text"}))["doc"].as_u64().unwrap(); + docs.push(doc); + } + for doc in docs.iter().take(8) { + handles.push(ok(&mut a, "ocr_words", json!({"doc":doc,"retain_review":true}))["review"].as_u64().unwrap()); + } + let error = a.call("ocr_words", &json!({"doc":docs[8],"retain_review":true})).unwrap_err().to_string(); + assert!(error.contains("at most 8"), "{error}"); + let replacement = ok(&mut a, "ocr_words", json!({"doc":docs[0],"retain_review":true}))["review"].as_u64().unwrap(); + assert_ne!(replacement, handles[0]); + assert!(a.call("ocr_review_apply", &json!({"doc":docs[0],"review":handles[0],"discard":true})).is_err()); + ok(&mut a, "doc_close", json!({"doc":docs[0],"discard_changes":true})); + assert!(a.call("ocr_review_apply", &json!({"doc":docs[0],"review":replacement})).is_err()); + let final_handle = ok(&mut a, "ocr_words", json!({"doc":docs[8],"retain_review":true}))["review"].as_u64().unwrap(); + ok(&mut a, "ocr_review_apply", json!({"doc":docs[8],"review":final_handle,"discard":true})); +} + #[test] fn editor_crop_group_move_and_arrange_survive_added_item_deletion_through_tools() { let dir = workdir("editor-crop-group-arrange"); diff --git a/crates/create/src/lib.rs b/crates/create/src/lib.rs index 6346e85120..53a587728f 100644 --- a/crates/create/src/lib.rs +++ b/crates/create/src/lib.rs @@ -92,6 +92,26 @@ pub fn blank(width: f64, height: f64, pages: usize) -> Result)]) -> Result { + if pages.is_empty() || pages.len() > 10_000 { + return Err(CreateError::Invalid("invalid page count".into())); + } + if pages.iter().any(|(w, h, _, _)| !(w.is_finite() && h.is_finite() && (3.0..=MAX_SIDE).contains(w) && (3.0..=MAX_SIDE).contains(h))) { + return Err(CreateError::Invalid("invalid page size".into())); + } + let mut doc = Document::new_empty(); + let mut tree = Pages::of(&doc)?; + for (w, h, resources, content) in pages { + let content = if content.is_empty() { None } else { Some(content.clone()) }; + tree.add(&mut doc, *w, *h, resources.clone(), content); + } + tree.finish(&mut doc)?; + Ok(doc) +} + // ── images ────────────────────────────────────────────────────────────────────────────────── /// An image ready to embed: its XObject dictionary, encoded data, optional soft mask, the ICC diff --git a/crates/edit/src/lib.rs b/crates/edit/src/lib.rs index b95c4296f9..2a0add8d12 100644 --- a/crates/edit/src/lib.rs +++ b/crates/edit/src/lib.rs @@ -78,7 +78,7 @@ pub(crate) use place::undrawable; #[cfg(test)] use place::{ISOLATION_MAX_DEPTH, OLD_WRAP_CLOSE, WRAP_CLOSE, WRAP_OPEN, is_wrap_close, page_fonts, stream_bytes, stream_tag, tagged, wrapper_role}; use place::{Reach, check, check_on_top, contents, draw_on_top, lift_on_top, page_list, place_tagged, revealed_under_paint}; -pub use place::{stamp, undrawable_message}; +pub use place::{add_type3_fonts, stamp, stamp_with_fonts, undrawable_message}; #[cfg(test)] mod marked_isolation_tests; diff --git a/crates/edit/src/place.rs b/crates/edit/src/place.rs index 27ed7b3793..6460aab066 100644 --- a/crates/edit/src/place.rs +++ b/crates/edit/src/place.rs @@ -478,14 +478,79 @@ pub(crate) fn place_inside_wrapper(doc: &mut Document, page: &pdfcraft_model::Pa /// it away); the original content is wrapped in q/Q first. Used for applied redaction boxes. /// Returns the new content stream. pub fn stamp(doc: &mut Document, page: usize, tag: &str, content: Vec) -> Result { + stamp_with_fonts(doc, page, tag, &mut pdfcraft_fonts::TextPlan::default(), |_| content) +} + +/// Draw the content `make` builds on top of a page, with standard Helvetica as `/PCHelv` and +/// the plan's Type3 fonts added to the page's resources first (so the content can reference +/// them by the names the plan carries). The original content is wrapped in q/Q first; the +/// stream is tagged `tag` (not a mark kind, so Remove never takes it away). This is how the +/// OCR text layers get their Unicode faces (see [`add_type3_fonts`]). Returns the new content +/// stream. +pub fn stamp_with_fonts( + doc: &mut Document, + page: usize, + tag: &str, + plan: &mut pdfcraft_fonts::TextPlan, + make: impl FnOnce(&pdfcraft_fonts::TextPlan) -> Vec, +) -> Result { let all = page_list(doc); check(&[page], all.len())?; check_on_top(doc, &all[page])?; + add_type3_fonts(doc, page, plan)?; + let content = make(plan); + let all = page_list(doc); add_resources(doc, &all[page], None, Some(&content))?; let p = page_list(doc).swap_remove(page); place_tagged(doc, &p, tag, content, false) } +/// Add the plan's Type3 fonts to a page's (0-based) resources, building each font once and +/// sharing the object with every page that uses it. Fills in each planned font's resource name +/// (`PCOcr`, `PCOcr1`, … — a name free on the page being registered: a font built for an +/// earlier page keeps its name only where that name is free or already that font) and object; +/// the content generators read the names from the plan, so this must run before they are +/// called for each page. +pub fn add_type3_fonts(doc: &mut Document, page: usize, plan: &mut pdfcraft_fonts::TextPlan) -> Result<(), EditError> { + if plan.fonts.is_empty() { + return Ok(()); + } + let all = page_list(doc); + check(&[page], all.len())?; + let p = &all[page]; + let mut res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default(); + let mut fonts = res.get(b"Font").map(|f| doc.resolve(f)).and_then(|f| f.as_dict().cloned()).unwrap_or_default(); + for i in 0..plan.fonts.len() { + let object = match plan.fonts[i].object { + Some(r) => { + // Built for an earlier page under a name that was free THERE. On this page the + // name may already be another font (the page's own, or an earlier OCR layer's): + // then this page gets a free name of its own, and the plan carries it for the + // content written next. + let taken = fonts.get(plan.fonts[i].name.as_bytes()).is_some_and(|o| o.as_ref() != Some(r)); + if taken { + plan.fonts[i].name = super::text::unique_font_name(&fonts, "PCOcr"); + } + r + } + None => { + let glyphs = plan.fonts[i].glyphs.iter().map(|(ch, g)| (*ch, super::text::glyph_path(g), g.width)).collect::>(); + let dict = super::text::type3_font_dict(doc, &glyphs)?; + let r = doc.add(Object::Dict(dict)); + plan.fonts[i].name = super::text::unique_font_name(&fonts, "PCOcr"); + plan.fonts[i].object = Some(r); + r + } + }; + fonts.set(plan.fonts[i].name.as_bytes().to_vec(), Object::Ref(object)); + } + res.set(b"Font".to_vec(), Object::Dict(fonts)); + // The page gets its own (possibly copied) resources; shared dictionaries stay untouched. + let obj = p.obj; + doc.update_dict(obj, |d| d.set(b"Resources".to_vec(), Object::Dict(res)))?; + Ok(()) +} + pub(crate) fn begin(kind: MarkKind, subtype: &str, matrix: [f64; 6]) -> String { format!( "q\n/Artifact <> BDC\n{} {} {} {} {} {} cm\n", diff --git a/crates/edit/src/tests.rs b/crates/edit/src/tests.rs index ecbb8b3645..22fbd6e8ae 100644 --- a/crates/edit/src/tests.rs +++ b/crates/edit/src/tests.rs @@ -10257,6 +10257,86 @@ fn styled_existing_justification_is_refused_instead_of_silently_left_aligning() assert_eq!(page_content_bytes(&reopen(&doc), 0), before); } +/// The OCR stamp path registers the plan's Type3 faces (a free `PCOcr…` name per stamp, so a +/// page that is OCR'd twice keeps both layers' fonts), and the stamped codes extract back +/// through the font's ToUnicode CMap. +#[test] +fn stamp_with_fonts_registers_the_planned_type3_faces() { + if without_craft_fonts("stamp_with_fonts_registers_the_planned_type3_faces") { + return; + } + let mut doc = text_page("BT /F2 12 Tf 72 700 Td (ab) Tj ET"); + let mut plan = pdfcraft_fonts::TextPlan::for_words(["日本"]); + assert_eq!(plan.missing(), 0); + stamp_with_fonts(&mut doc, 0, "OCR", &mut plan, |p| { + let codes: Vec = p.encoding(0).iter().map(|&(_, c)| c).collect(); + let mut c = format!("BT 3 Tr /PCHelv 1 Tf\n/{} 1 Tf 100 700 Td ", p.fonts[0].name).into_bytes(); + c.extend_from_slice(&pdfcraft_fonts::literal(&codes)); + c.extend_from_slice(b" Tj ET"); + c + }) + .unwrap(); + assert!(plan.fonts[0].name.starts_with("PCOcr"), "{:?}", plan.fonts[0].name); + // The registered font carries a ToUnicode CMap; the engine's own reader decodes the run. + let reopened = Document::open(Arc::new(write_incremental(&doc, &SaveOptions::default()).unwrap())).unwrap(); + let lines = text::text_lines(&reopened, 0).unwrap(); + assert!(lines.iter().any(|l| l.text.contains("日本")), "{lines:?}"); + // A second stamp on the same page takes the next free name; the first layer's font stands. + let mut second = pdfcraft_fonts::TextPlan::for_words(["語"]); + stamp_with_fonts(&mut doc, 0, "OCR", &mut second, |p| format!("BT 3 Tr /{} 1 Tf 100 600 Td (\u{1}) Tj ET", p.fonts[0].name).into_bytes()) + .unwrap(); + assert_eq!(second.fonts[0].name, format!("{}1", plan.fonts[0].name)); +} + +/// A Type3 face built for page 1 is reused on page 2, but page 2 already has a font under the +/// name page 1 gave it: page 2's own font keeps its name, and the shared face is registered on +/// page 2 under a free one that the plan carries for page 2's content. (The plan is written out +/// by hand, so this runs without the craft-fonts build input.) +#[test] +fn a_reused_type3_face_never_replaces_a_later_page_s_font_of_the_same_name() { + let mut doc = build(&[ + "<< /Type /Catalog /Pages 2 0 R >>", + "<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >>", + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Contents 5 0 R >>", + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Contents 6 0 R /Resources << /Font << /PCOcr 7 0 R >> >> >>", + &stream("", ""), + &stream("", "BT /PCOcr 12 Tf 72 700 Td (own) Tj ET"), + "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>", + ]); + let glyph = pdfcraft_fonts::GlyphOutline { contours: vec![vec![[0.1, 0.0], [0.9, 0.0], [0.5, 0.8]]], width: 1.0, bbox: [0.1, 0.0, 0.9, 0.8] }; + let mut plan = pdfcraft_fonts::TextPlan::default(); + plan.fonts.push(pdfcraft_fonts::PlannedType3 { family: "Test", glyphs: vec![('日', glyph)], name: String::new(), object: None }); + let draw = |p: &pdfcraft_fonts::TextPlan| format!("BT 3 Tr /{} 1 Tf 100 600 Td (\u{1}) Tj ET", p.fonts[0].name).into_bytes(); + stamp_with_fonts(&mut doc, 0, "OCR", &mut plan, draw).unwrap(); + assert_eq!(plan.fonts[0].name, "PCOcr", "free on page 1"); + let face = plan.fonts[0].object.expect("built once"); + stamp_with_fonts(&mut doc, 1, "OCR", &mut plan, draw).unwrap(); + assert_ne!(plan.fonts[0].name, "PCOcr", "page 2 already has a PCOcr"); + let page2 = &pdfcraft_model::pages(&doc)[1]; + let res = doc.resolve(page2.dict.get(b"Resources").unwrap()); + let fonts = doc.resolve(res.as_dict().unwrap().get(b"Font").unwrap()); + let fonts = fonts.as_dict().unwrap(); + assert_eq!(fonts.get(b"PCOcr").and_then(|o| o.as_ref()), Some(pdfcraft_cos::ObjRef::new(7, 0)), "page 2's own font keeps its name"); + assert_eq!(fonts.get(plan.fonts[0].name.as_bytes()).and_then(|o| o.as_ref()), Some(face), "the shared face, under a free name"); + assert_eq!(plan.fonts[0].object, Some(face), "still built once"); +} + +#[test] +fn planned_type3_charproc_has_valid_metrics_and_encloses_its_ink() { + let glyph = pdfcraft_fonts::GlyphOutline { + contours: vec![vec![[-0.1001, -0.2001], [0.9001, 1.1001]]], + width: 0.8, + bbox: [-0.1001, -0.2001, 0.9001, 1.1001], + }; + let parsed = pdfcraft_content::parse(&text::glyph_path(&glyph)); + assert_eq!(parsed.skipped, 0); + let metrics = &parsed.ops[0]; + assert!(metrics.is("d1")); + assert_eq!(metrics.operands.len(), 6); + assert_eq!((metrics.num(0), metrics.num(1)), (Some(800.0), Some(0.0))); + assert_eq!((metrics.num(2), metrics.num(3), metrics.num(4), metrics.num(5)), (Some(-101.0), Some(-201.0), Some(901.0), Some(1101.0))); +} + #[test] fn image_arrange_refuses_crossed_scope_boundaries_without_changes() { for content in ["/P <> BDC q 10 0 0 10 0 0 cm /Im0 Do EMC Q /Im0 Do", "q /P <> BDC /Im0 Do Q EMC /Im0 Do"] { diff --git a/crates/edit/src/text.rs b/crates/edit/src/text.rs index 0fb00324b6..3d1626a576 100644 --- a/crates/edit/src/text.rs +++ b/crates/edit/src/text.rs @@ -52,6 +52,7 @@ use fallback::{ }; #[cfg(test)] use fallback::{add_type3_font, is_type3_fallback}; +pub(crate) use fallback::{glyph_path, type3_font_dict, unique_font_name}; use interp::{Origin, Paint, union}; use lines::page_dict; pub(crate) use lines::{FormBudget, content_streams, content_streams_within, dict_in_place, form_call, resource_dict, rewritten_contents, splice}; diff --git a/crates/edit/src/text/fallback.rs b/crates/edit/src/text/fallback.rs index 78f9014a20..469d1739d4 100644 --- a/crates/edit/src/text/fallback.rs +++ b/crates/edit/src/text/fallback.rs @@ -493,6 +493,92 @@ pub(super) fn add_type3_font(doc: &mut Document, fonts_res: &mut Dict, face: &Cr name } +/// A glyph's CharProc body at 1000 units per em, with a six-operand `d1` and its actual ink box. +pub(crate) fn glyph_path(glyph: &pdfcraft_fonts::GlyphOutline) -> Vec { + let scale = 1000.0; + let b = if glyph.contours.is_empty() { [0.0; 4] } else { glyph.bbox }; + let mut out = format!( + "{} 0 {} {} {} {} d1\n", + pdf_num(glyph.width * scale), + pdf_num((b[0] * scale).floor()), + pdf_num((b[1] * scale).floor()), + pdf_num((b[2] * scale).ceil()), + pdf_num((b[3] * scale).ceil()) + ) + .into_bytes(); + for contour in &glyph.contours { + let Some(first) = contour.first() else { continue }; + out.extend_from_slice(format!("{} {} m\n", pdf_num(first[0] * scale), pdf_num(first[1] * scale)).as_bytes()); + for p in contour.iter().skip(1) { + out.extend_from_slice(format!("{} {} l\n", pdf_num(p[0] * scale), pdf_num(p[1] * scale)).as_bytes()); + } + out.extend_from_slice(b"h\n"); + } + out.extend_from_slice(b"f\n"); + out +} + +/// A resource name free in `fonts_res`: `base`, then `base1`, `base2`, … (a page that was +/// OCR'd twice keeps both layers' fonts distinct). +pub(crate) fn unique_font_name(fonts_res: &Dict, base: &str) -> String { + let mut name = String::from(base); + let mut suffix = 0usize; + while fonts_res.contains(name.as_bytes()) { + suffix = suffix.saturating_add(1); + name = format!("{base}{suffix}"); + } + name +} + +/// The Type3 font dictionary for `(character, CharProc body, advance in em)` glyphs, codes from +/// 1: one CharProc per code, `/Differences` glyph names, real `/Widths`, and a `/ToUnicode` CMap +/// so the text extracts back (this is what makes the output Unicode-correct, not just visible). +pub(crate) fn type3_font_dict(doc: &mut Document, glyphs: &[(char, Vec, f64)]) -> Result { + let mut charprocs = Dict::new(); + let mut differences = vec![Object::Int(1)]; + let mut widths = Vec::with_capacity(glyphs.len()); + let mut cmap = String::from( + "/CIDInit /ProcSet findresource begin\n12 dict begin\nbegincmap\n/CMapType 2 def\n1 begincodespacerange\n<01> \nendcodespacerange\n", + ); + cmap.push_str(&format!("{} beginbfchar\n", glyphs.len())); + let mut codes = Vec::with_capacity(glyphs.len()); + for (i, (ch, path, width)) in glyphs.iter().enumerate() { + let code = u8::try_from(i + 1).map_err(|_| EditError::Invalid("too many unique characters for one font".into()))?; + let glyph_name = format!("g{code:02X}"); + let mut pd = Dict::new(); + pd.set(b"Length".to_vec(), path.len() as i64); + let proc_ref = doc.add(Object::Stream(Stream::from_raw(pd, path.clone()))); + charprocs.set(glyph_name.as_bytes().to_vec(), Object::Ref(proc_ref)); + differences.push(Object::name(&glyph_name)); + widths.push(Object::Real((width * 1000.0).round())); + cmap.push_str(&format!("<{code:02X}> <{}>\n", unicode_hex(*ch))); + codes.push((*ch, code, *width)); + } + cmap.push_str("endbfchar\nendcmap\nCMapName currentdict /CMap defineresource pop\nend\nend\n"); + let mut cmap_dict = Dict::new(); + cmap_dict.set(b"Length".to_vec(), cmap.len() as i64); + let cmap_ref = doc.add(Object::Stream(Stream::from_raw(cmap_dict, cmap.into_bytes()))); + let mut encoding = Dict::new(); + encoding.set(b"Type".to_vec(), Object::name("Encoding")); + encoding.set(b"Differences".to_vec(), Object::Array(differences)); + let mut font = Dict::new(); + font.set(b"Type".to_vec(), Object::name("Font")); + font.set(b"Subtype".to_vec(), Object::name("Type3")); + font.set(b"Name".to_vec(), Object::name("PCFallback")); + font.set(b"FontBBox".to_vec(), Object::Array(vec![Object::Int(0), Object::Int(-300), Object::Int(1000), Object::Int(1000)])); + font.set( + b"FontMatrix".to_vec(), + Object::Array(vec![Object::Real(0.001), Object::Int(0), Object::Int(0), Object::Real(0.001), Object::Int(0), Object::Int(0)]), + ); + font.set(b"FirstChar".to_vec(), Object::Int(1)); + font.set(b"LastChar".to_vec(), Object::Int(codes.len() as i64)); + font.set(b"Widths".to_vec(), Object::Array(widths)); + font.set(b"Encoding".to_vec(), Object::Dict(encoding)); + font.set(b"CharProcs".to_vec(), Object::Dict(charprocs)); + font.set(b"ToUnicode".to_vec(), Object::Ref(cmap_ref)); + Ok(font) +} + /// The runs that show `text` in the fallback: consecutive characters in one font form one run, so /// a font is switched only where the text moves to another font. `None` if a character has no code. pub(super) fn type3_encode(fallback: &Type3Fallback, text: &str) -> Option)>> { diff --git a/crates/engine/src/catalog.rs b/crates/engine/src/catalog.rs index 47394a7bb5..d6be8fd4eb 100644 --- a/crates/engine/src/catalog.rs +++ b/crates/engine/src/catalog.rs @@ -267,7 +267,7 @@ pub static TOOL_GROUPS: &[ToolGroup] = &[ item("In multiple files", "files", "ocr.recognize_batch", Ready), item("Enhance scanned file", "sparkles", "ocr.enhance", Ready), item("Enhance camera image", "camera", "ocr.enhance_camera", Ready), - item("Correct recognized text", "text-select", "ocr.correct", Planned("M10")), + item("Correct recognized text", "text-select", "ocr.correct", Ready), ], }], }, diff --git a/crates/engine/src/commands.rs b/crates/engine/src/commands.rs index 9301201661..7cee14657e 100644 --- a/crates/engine/src/commands.rs +++ b/crates/engine/src/commands.rs @@ -361,6 +361,7 @@ pub const COMMANDS: &[CommandSpec] = &[ c("ocr.enhance_camera", "Enhance camera image", None, None, Modification, "camera"), c("ocr.enhance", "Enhance scanned file…", None, None, Modification, "sparkles"), c("ocr.recognize_batch", "Recognize text in multiple files…", None, None, Nothing, "files"), + c("ocr.correct", "Correct recognized text…", None, None, Document, "text-select"), c("a11y.check", "Check for accessibility…", None, None, Document, "accessibility"), c("a11y.find_untagged", "Find untagged content", None, None, Document, "search"), c("a11y.autotag", "Automatically tag PDF", None, None, Document, "tag"), diff --git a/crates/engine/src/edit/run.rs b/crates/engine/src/edit/run.rs index 588a3a847f..4719de302f 100644 --- a/crates/engine/src/edit/run.rs +++ b/crates/engine/src/edit/run.rs @@ -718,7 +718,7 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E Edit::RemoveStandardId { standard } => pdfcraft_organize::remove_standard_id(doc, *standard)?, Edit::SetDocumentScript { name, script } => pdfcraft_forms::set_document_script(doc, name, script.as_deref())?, Edit::AddOcrText { page, words } => { - pdfcraft_edit::stamp(doc, *page, "OCR", pdfcraft_ocr::text_layer(words))?; + crate::ocr::layer::add_text(doc, *page, words)?; } Edit::EnhanceScan { images } => crate::scan_clean::write_back(doc, images)?, Edit::EditPageImage { page, index, change } => run_edit_page_image(doc, *page, *index, change)?, diff --git a/crates/engine/src/ocr.rs b/crates/engine/src/ocr.rs index 6b9869cead..18ea331d41 100644 --- a/crates/engine/src/ocr.rs +++ b/crates/engine/src/ocr.rs @@ -3,16 +3,104 @@ //! //! Recognition is slow (about a second a page), so it is split from the edit: [`Session::ocr_job`] //! captures what it needs, [`OcrJob::run`] works anywhere (the UI runs it on a worker thread) and -//! [`Session::apply_ocr`] applies the result as one undoable step. +//! [`Session::apply_ocr`] applies the result as one undoable step. The result is only applied +//! while the document is still the one that was read — [`Session::apply_ocr_checked`] refuses a +//! stale result with a named reason instead of writing into a changed document. use std::sync::{Arc, Mutex}; +#[cfg(not(target_arch = "wasm32"))] +use std::time::{Duration, Instant}; use pdfcraft_render::{PageInfo, PageRenderer, RenderConfig, RenderRequest, RequestKind}; -pub use pdfcraft_ocr::{LANGUAGES, Models, Ocr, OcrError, PlacedWord}; +pub use pdfcraft_ocr::{ + ConfidenceBand, LANGUAGES, MergeStrategy, Models, Ocr, OcrError, PlacedWord, Recognizer, band_for, is_suspect, low_confidence, +}; use crate::{DocId, Edit, EditError, Session}; +pub mod dictionary; +pub(crate) mod layer; + +/// Bound recognition/review inputs before cloning or planning glyphs. +pub fn validate_ocr_words<'a>(words: impl IntoIterator) -> Result { + layer::validate_words(words).map_err(|e| e.to_string()) +} + +/// Which recognition engine to use. The default is ocrs, the pure-Rust engine; "Automatic" is +/// an ensemble — ocrs reading first, Tesseract as a second opinion, merged per the strategy — +/// when both are available. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash)] +pub enum EngineChoice { + /// The pure-Rust ocrs engine (the default). + #[default] + Ocrs, + /// The installed `tesseract` binary, as an external process. + Tesseract, + /// Ensemble: the default engine first, the other one as a secondary when available. + Auto, +} + +impl EngineChoice { + /// Parse a choice; unknown names fall back to the default (ocrs), like Acrobat's dialogs + /// fall back for unknown persisted values. + pub fn parse(s: &str) -> EngineChoice { + match s.trim().to_ascii_lowercase().as_str() { + "tesseract" => EngineChoice::Tesseract, + "auto" | "automatic" | "ensemble" => EngineChoice::Auto, + _ => EngineChoice::Ocrs, + } + } + + /// The strategy this choice reads with when none was chosen. Automatic exists to bring in + /// the second engine, so it asks it when the first reading is weak or of unknown + /// confidence ([`MergeStrategy::ConfidenceWeighted`]); a single named engine reads alone. + pub fn default_strategy(self) -> MergeStrategy { + match self { + EngineChoice::Auto => MergeStrategy::ConfidenceWeighted, + EngineChoice::Ocrs | EngineChoice::Tesseract => MergeStrategy::PrimaryOnly, + } + } + + /// The name settings and tools use for this choice. + pub fn name(self) -> &'static str { + match self { + EngineChoice::Ocrs => "ocrs", + EngineChoice::Tesseract => "tesseract", + EngineChoice::Auto => "auto", + } + } +} + +/// What to produce from a recognition. **Searchable** (the default) adds the existing +/// invisible-text layer on the original page; **EditableText** writes a *new* document with the +/// reviewed words as visible text and no scan image (the writer is the Verify screen's; the mode +/// is plumbed through here). Unknown mode strings mean Searchable. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash)] +pub enum OutputMode { + #[default] + Searchable, + EditableText, +} + +impl OutputMode { + /// Parse a mode name (trimmed, case-insensitive); unknown strings mean Searchable. + pub fn parse(s: &str) -> OutputMode { + match s.trim().to_ascii_lowercase().as_str() { + "editable" | "editable-text" | "editable_text" | "editabletext" => OutputMode::EditableText, + _ => OutputMode::Searchable, + } + } + + /// The name settings and tools use for this mode. + pub fn name(self) -> &'static str { + match self { + OutputMode::Searchable => "searchable", + OutputMode::EditableText => "editable", + } + } +} + /// Recognize Text settings. #[derive(Clone, Debug, PartialEq)] pub struct OcrSettings { @@ -29,6 +117,40 @@ pub struct OcrSettings { /// that is read, so a scan with a few comments on it is still a scan. `false`: everything /// the page shows. pub native_text_only: bool, + /// Leave whole files that already have text alone (the batch flow's "Skip files that + /// already contain text"; a skipped file is its own result bucket, never an OCR success). + pub skip_text_files: bool, + /// Read pages even when the skip options would leave them alone ("Force OCR (override + /// skip options)"). + pub force_ocr: bool, + /// Which engine reads the pages. + pub engine: EngineChoice, + /// How the engines' readings combine (see [`MergeStrategy`]). + pub strategy: MergeStrategy, + /// Preprocessing, all off by default (a scan is passed through as rendered): + /// straighten pages scanned at 90°/180°/270°. + pub auto_rotate: bool, + /// Preprocessing: straighten skewed text lines. + pub deskew: bool, + /// Preprocessing: 3×3 median filter. + pub denoise: bool, + /// Preprocessing: Sauvola binarization. + pub binarize: bool, + /// What to produce (searchable text layer, or a new editable document). + pub output_mode: OutputMode, + /// Restrict recognition to this rectangle of the rendered page image, in pixels + /// `[x0, y0, x1, y1]` (the Verify screen's "Re-OCR this region"). + pub region: Option<[f32; 4]>, + /// The batch note's separate per-word threshold: words with `0 <= conf <= threshold` are + /// counted for "N low-confidence word(s) on page(s) …" (see [`low_confidence`]; unrelated + /// to the review overlay's bands, which use [`band_for`]). + pub low_confidence_threshold: f32, + /// Where the Tesseract program lives, when the user configured it (a preference — the + /// `PDFCRAFT_TESSERACT` environment variable is the fallback in the engine; never taken + /// from document data). A configured path must be absolute to an existing, executable + /// file; an invalid configuration makes the engine unavailable rather than silently + /// searching the PATH. `None` means the PATH search decides. + pub tesseract_path: Option, } /// A recognised word whose box is at least this much covered by the page's own text is that @@ -56,7 +178,24 @@ fn covered(word: [f32; 4], text: &[[f32; 4]]) -> f32 { impl Default for OcrSettings { fn default() -> Self { - OcrSettings { dpi: 300.0, language: "en".into(), skip_text_pages: true, native_text_only: false } + OcrSettings { + dpi: 300.0, + language: "en".into(), + skip_text_pages: true, + native_text_only: false, + skip_text_files: false, + force_ocr: false, + engine: EngineChoice::default(), + strategy: MergeStrategy::default(), + auto_rotate: false, + deskew: false, + denoise: false, + binarize: false, + output_mode: OutputMode::default(), + region: None, + low_confidence_threshold: 60.0, + tesseract_path: None, + } } } @@ -69,31 +208,406 @@ pub struct OcrPage { pub already_on_page: usize, /// Why the page was not read, if it was skipped. pub skipped: Option, + /// Notes about a partial reading: the words stand, but something on the way did not (an + /// ensemble partner failed and the page keeps the primary engine's reading alone). + pub notes: Vec, } impl OcrPage { pub fn text(&self) -> String { self.words.iter().map(|w| w.text.as_str()).collect::>().join(" ") } -} -/// The recogniser, loaded once (it takes a moment) and shared. -pub fn engine() -> Result, String> { - static OCR: Mutex>> = Mutex::new(None); - let mut slot = OCR.lock().unwrap_or_else(std::sync::PoisonError::into_inner); - if let Some(o) = slot.as_ref() { - return Ok(o.clone()); + /// Mean of the confidences the page's words report (`None` when none do). + pub fn mean_confidence(&self) -> Option { + let (sum, n) = self.words.iter().filter_map(|w| w.confidence).fold((0.0f64, 0usize), |(s, n), c| (s + c as f64, n + 1)); + (n > 0).then_some((sum / n as f64) as f32) + } + + /// The words to flag for review: their confidence is in the Low band ([`band_for`]). + /// Excluded are words the caller wants gone — removed words, words in the user dictionary, + /// skipped words — which is review state, so this reports the indices and the reviewer + /// filters them. + pub fn suspects(&self) -> Vec { + self.words.iter().enumerate().filter(|(_, w)| pdfcraft_ocr::is_suspect(w.confidence)).map(|(i, _)| i).collect() } - let o = Arc::new(Ocr::find().map_err(|e| e.to_string())?); - *slot = Some(o.clone()); - Ok(o) } -/// Whether the recognition models are installed. -pub fn available() -> bool { +/// The engines a job reads with: one primary and, for ensemble strategies, an optional +/// secondary. +pub struct Recognizers { + pub primary: Arc, + pub secondary: Option>, +} + +struct EngineCache { + ocrs: Option>, +} + +/// How long a negative engine probe (no program there) is remembered. A short TTL, not +/// forever: a tesseract installed while the app runs is found by the next job, without a +/// restart. An engine that was found is remembered for the session. +#[cfg(not(target_arch = "wasm32"))] +const NEGATIVE_TTL: Duration = Duration::from_secs(30); + +/// The remembered Tesseract probe: the configured location it answers for, the engine it +/// found (`None` for a probe that failed), and when that probe happened. +#[cfg(not(target_arch = "wasm32"))] +struct CachedTesseract { + key: Option, + found: Option>, + at: Instant, +} + +/// Whether a probe result still counts: a found engine for the session, a failed probe until +/// [`NEGATIVE_TTL`] passes (split out so tests can pass the clock). +#[cfg(not(target_arch = "wasm32"))] +fn probe_fresh(found: bool, at: Instant, now: Instant) -> bool { + found || now.duration_since(at) < NEGATIVE_TTL +} + +/// The remembered Tesseract probe (one slot: the last location asked about). +#[cfg(not(target_arch = "wasm32"))] +static TESSERACT_PROBE: Mutex> = Mutex::new(None); + +/// Whether a background probe, started for the UI by [`tesseract_engine_now`], is running now. +/// At most one runs at a time. +#[cfg(not(target_arch = "wasm32"))] +static PROBING: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false); + +/// The Tesseract engine: found once (it answers `--list-langs` through a process) and then +/// remembered, per configured path; a failed probe is retried after [`NEGATIVE_TTL`]. No +/// static mutex is held while the discovery process runs: the cache is read under the lock, +/// released for the probe, and the result is written back when it arrives (concurrent probes +/// of the same location may both run; the last write wins and both answers are valid). A +/// user-configured path (`settings.tesseract_path`, else the `PDFCRAFT_TESSERACT` +/// environment variable) is validated before any process starts; an invalid configuration is +/// unavailable rather than a PATH fallback. This waits for the probe: the UI thread uses +/// [`tesseract_engine_now`]. +#[cfg(not(target_arch = "wasm32"))] +fn tesseract_engine(configured: Option<&str>) -> Option> { + if let Cached::Fresh(found) = cached_tesseract(configured) { + return found; + } + let key = configured.map(str::to_string); + // The probe (a `--list-langs` process, up to the timeout) runs outside every static mutex. + let found = match configured { + Some(path) => pdfcraft_ocr::tesseract::TesseractCli::at(std::path::PathBuf::from(path)).map(|t| Arc::new(t) as Arc), + None => pdfcraft_ocr::tesseract::TesseractCli::find().map(|t| Arc::new(t) as Arc), + }; + *TESSERACT_PROBE.lock().unwrap_or_else(|e| e.into_inner()) = Some(CachedTesseract { key, found: found.clone(), at: Instant::now() }); + found +} + +/// What the UI may know about the Tesseract probe without waiting for it. +#[cfg(not(target_arch = "wasm32"))] +enum Probe { + /// An answer: the engine, or `None` when there is none. It may be an old one that a + /// background probe is refreshing. + Known(Option>), + /// No answer yet: the first background probe is running (or could not be started). + Pending, +} + +/// What the cache holds for a location. +#[cfg(not(target_arch = "wasm32"))] +enum Cached { + /// A usable answer (see [`probe_fresh`]). + Fresh(Option>), + /// A failed probe older than [`NEGATIVE_TTL`]: still the best known answer, but due again. + Stale(Option>), + /// Nothing for this location. + Missing, +} + +/// [`tesseract_engine`] for the UI thread, which must never wait on a process: the cached +/// answer is returned at once, and when it is missing or due for a retry one background thread +/// runs the probe (it fills the cache). The answer is [`Probe::Pending`] only until the first +/// probe of a location lands, so a retry never makes the choice flicker. +#[cfg(not(target_arch = "wasm32"))] +fn tesseract_engine_now(configured: Option<&str>) -> Probe { + let known = match cached_tesseract(configured) { + Cached::Fresh(found) => return Probe::Known(found), + Cached::Stale(found) => Some(found), + Cached::Missing => None, + }; + if !PROBING.swap(true, std::sync::atomic::Ordering::AcqRel) { + let configured = configured.map(str::to_string); + let started = std::thread::Builder::new().name("pdfcraft-tesseract-probe".into()).spawn(move || { + let _ = tesseract_engine(configured.as_deref()); + PROBING.store(false, std::sync::atomic::Ordering::Release); + }); + if started.is_err() { + PROBING.store(false, std::sync::atomic::Ordering::Release); + } + } + known.map_or(Probe::Pending, Probe::Known) +} + +/// Whether the Tesseract probe is running in the background right now. The interface asks it +/// while it shows an engine choice that read as "still checking", to look again soon. +pub fn engine_probe_running() -> bool { + #[cfg(not(target_arch = "wasm32"))] + { + PROBING.load(std::sync::atomic::Ordering::Acquire) + } + #[cfg(target_arch = "wasm32")] + { + false + } +} + +/// What the cache holds for `configured`. Never starts a process. +#[cfg(not(target_arch = "wasm32"))] +fn cached_tesseract(configured: Option<&str>) -> Cached { + let slot = TESSERACT_PROBE.lock().unwrap_or_else(|e| e.into_inner()); + cache_lookup(slot.as_ref(), configured, Instant::now()) +} + +/// [`cached_tesseract`] over an explicit slot and clock (so tests can pass both). +#[cfg(not(target_arch = "wasm32"))] +fn cache_lookup(slot: Option<&CachedTesseract>, configured: Option<&str>, now: Instant) -> Cached { + match slot { + Some(c) if c.key.as_deref() == configured => { + if probe_fresh(c.found.is_some(), c.at, now) { + Cached::Fresh(c.found.clone()) + } else { + Cached::Stale(c.found.clone()) + } + } + _ => Cached::Missing, + } +} + +/// Whether the ocrs models are installed, remembered like the Tesseract probe (found: for the +/// session; missing: for [`NEGATIVE_TTL`]), so per-frame UI checks do not search the disk. +#[cfg(not(target_arch = "wasm32"))] +fn models_installed_cached() -> bool { + static MODELS: Mutex> = Mutex::new(None); + let mut slot = MODELS.lock().unwrap_or_else(|e| e.into_inner()); + if let Some((found, at)) = *slot + && probe_fresh(found, at, Instant::now()) + { + return found; + } + let found = Models::find().is_some(); + *slot = Some((found, Instant::now())); + found +} + +/// The web build has no disk search worth caching. +#[cfg(target_arch = "wasm32")] +fn models_installed_cached() -> bool { Models::find().is_some() } +/// The program's `--version` line, remembered per program path for the session (`ocr_status` +/// asks every time it is called; the version of an installed binary does not change under a +/// running app often enough to start a process each time). +#[cfg(not(target_arch = "wasm32"))] +fn tesseract_version(program: &str) -> Option { + static VERSIONS: Mutex)>> = Mutex::new(Vec::new()); + if let Some((_, v)) = VERSIONS.lock().unwrap_or_else(|e| e.into_inner()).iter().find(|(p, _)| p == program) { + return v.clone(); + } + // Validated (absolute, existing, executable) before the process starts; run outside the lock. + let path = std::path::PathBuf::from(program); + let version = pdfcraft_ocr::tesseract::program_is_usable(&path).then(|| pdfcraft_ocr::tesseract::TesseractCli::new(path).version()).flatten(); + let mut cache = VERSIONS.lock().unwrap_or_else(|e| e.into_inner()); + if cache.len() < 16 && !cache.iter().any(|(p, _)| p == program) { + cache.push((program.to_string(), version.clone())); + } + version +} + +/// The Tesseract program the engine would run, named for `ocr_status`: the user-configured +/// location when one is set, else what the PATH search resolves to. This function itself +/// starts no process (the PATH search only looks at files), so a path here says where the +/// engine looks, not that a program answers there; [`engines_status`], which reports it, does +/// run the cached `--list-langs` probe and a cached `--version`. `None` when nothing is +/// configured and the PATH holds no `tesseract` (or, on the web, when there is no process to +/// run). +#[cfg(not(target_arch = "wasm32"))] +pub fn tesseract_program_path(configured: Option<&str>) -> Option { + match configured { + Some(p) => Some(p.to_string()), + None => pdfcraft_ocr::tesseract::path_search().map(|p| p.to_string_lossy().into_owned()), + } +} + +/// One engine's status, for `ocr_status`: whether it can run, why not when it cannot, and for +/// Tesseract the program it would run and that program's version (both `None` on the web). +#[derive(Clone, Debug, PartialEq)] +pub struct EngineStatus { + pub id: &'static str, + pub available: bool, + pub why_not: Option, + pub program: Option, + pub version: Option, +} + +/// The engines' status, in the order the Engine dropdown lists them: ocrs (built in), then +/// tesseract (the external program). Tesseract's availability is judged the same way +/// [`recognizers`] judges it under the default settings: the `PDFCRAFT_TESSERACT` environment +/// variable when set, else the PATH. This waits for the (cached) Tesseract probe; the tools +/// and tests call it, never the UI thread. +pub fn engines_status() -> Vec { + engines_status_for(&OcrSettings::default()) +} + +/// [`engines_status`] for the location a job's `settings` name: `settings.tesseract_path` when +/// set, else the environment variable, else the PATH, as [`recognizers`] decides it. (No tool +/// takes a location, because an agent must not choose which program runs; the tools report the +/// default settings.) +pub fn engines_status_for(settings: &OcrSettings) -> Vec { + let ocrs = Models::find().is_some(); + let mut out = vec![EngineStatus { id: "ocrs", available: ocrs, why_not: (!ocrs).then(models_missing), program: None, version: None }]; + #[cfg(not(target_arch = "wasm32"))] + { + let configured = configured_tesseract(settings); + let configured = configured.as_deref(); + let available = tesseract_engine(configured).is_some(); + let program = tesseract_program_path(configured); + // The version is a `--version` run against the validated program path, remembered per + // path; a program that answers `--list-langs` but not `--version` still counts as + // available. + let version = if available { program.as_deref().and_then(tesseract_version) } else { None }; + let why_not = (!available).then(|| match configured { + Some(p) => format!("the configured tesseract location is not an absolute path to an existing, executable file: {p}"), + None => tesseract_unavailable(), + }); + out.push(EngineStatus { id: "tesseract", available, why_not, program, version }); + } + #[cfg(target_arch = "wasm32")] + { + let _ = settings; + out.push(EngineStatus { id: "tesseract", available: false, why_not: Some(tesseract_unavailable()), program: None, version: None }); + } + out +} + +/// The web build never spawns processes, so it never has Tesseract. +#[cfg(target_arch = "wasm32")] +fn tesseract_engine(configured: Option<&str>) -> Option> { + let _ = configured; + None +} + +/// The configured Tesseract location: the user preference wins; the environment variable is +/// the fallback (a preference file rarely survives onto another machine). Both are user +/// choices — this is never taken from document data. +fn configured_tesseract(settings: &OcrSettings) -> Option { + settings.tesseract_path.clone().or_else(|| std::env::var_os("PDFCRAFT_TESSERACT").map(|v| v.to_string_lossy().into_owned())) +} + +/// The engine selector: builds each engine once and hands out the pair `settings.engine` +/// asks for. "Automatic" is an ensemble when both engines are available; a named engine that +/// cannot start fails with its cause named. +pub fn recognizers(settings: &OcrSettings) -> Result { + // The Tesseract probe (a `--list-langs` process) runs before any static mutex is taken, so + // recognition on other threads is never blocked behind someone's discovery. A job that + // names its engines never starts it: only Tesseract and Automatic read anything from it, + // so an ocrs-only job pays no process and no wait. + let configured = configured_tesseract(settings); + let tesseract = + if matches!(settings.engine, EngineChoice::Tesseract | EngineChoice::Auto) { tesseract_engine(configured.as_deref()) } else { None }; + static ENGINES: Mutex> = Mutex::new(None); + let mut slot = ENGINES.lock().unwrap_or_else(|e| e.into_inner()); + let cache = slot.get_or_insert_with(|| EngineCache { ocrs: None }); + if cache.ocrs.is_none() { + // Cheap to re-attempt while the models are missing; expensive work is cached forever. + cache.ocrs = pdfcraft_ocr::OcrsRecognizer::find().ok().map(|r| Arc::new(r) as Arc); + } + let ocrs = cache.ocrs.clone(); + match settings.engine { + EngineChoice::Ocrs => Ok(Recognizers { primary: ocrs.ok_or_else(models_missing)?, secondary: None }), + EngineChoice::Tesseract => Ok(Recognizers { primary: tesseract.ok_or_else(tesseract_unavailable)?, secondary: None }), + EngineChoice::Auto => match (ocrs, tesseract) { + (Some(p), s) => Ok(Recognizers { primary: p, secondary: s }), + (None, Some(p)) => Ok(Recognizers { primary: p, secondary: None }), + (None, None) => Err(models_missing()), + }, + } +} + +/// Why Tesseract cannot run, named for the user. +#[cfg(not(target_arch = "wasm32"))] +fn tesseract_unavailable() -> String { + "the tesseract program was not found (install it, or add it to the PATH)".into() +} + +/// Why Tesseract cannot run, named for the user (the web build has no processes). +#[cfg(target_arch = "wasm32")] +fn tesseract_unavailable() -> String { + "the tesseract engine is not available in the web build".into() +} + +/// Why the ocrs engine cannot run, in the end-user wording the UI catalogs translate (the +/// same text as [`pdfcraft_ocr::OcrError::NoModels`]: no developer commands in front of users). +fn models_missing() -> String { + pdfcraft_ocr::OcrError::NoModels.to_string() +} + +/// Whether recognition can run at all: the ocrs models are installed, or (off the web) a +/// Tesseract program is found. This may run the Tesseract probe (a process, cached) and so +/// waits for it: tools and tests call it; the UI asks [`engine_unavailable_reason`], which +/// never waits. +pub fn available() -> bool { + let configured = std::env::var_os("PDFCRAFT_TESSERACT").map(|v| v.to_string_lossy().into_owned()); + Models::find().is_some() || tesseract_engine(configured.as_deref()).is_some() +} + +/// Why `engine` cannot run right now, named for the UI's tooltips (the Engine dropdown +/// disables the choice with this text); `None` when it can. The app UI has no Tesseract +/// location field of its own, so this judges the environment variable, exactly like +/// [`available`] (no tool takes a per-job location either; `OcrSettings::tesseract_path` is +/// for library callers, and [`recognizers`] validates it on its own). +/// +/// Called on the UI thread every frame, so it never waits on a process: the answers are +/// cached ([`NEGATIVE_TTL`] for a missing engine, refreshed in the background), and until the +/// first Tesseract probe lands the choice reads as "still checking" (see +/// [`engine_probe_running`], to ask for another frame meanwhile). +pub fn engine_unavailable_reason(engine: EngineChoice) -> Option { + let ocrs = models_installed_cached(); + #[cfg(not(target_arch = "wasm32"))] + let tesseract = { + let configured = std::env::var_os("PDFCRAFT_TESSERACT").map(|v| v.to_string_lossy().into_owned()); + match tesseract_engine_now(configured.as_deref()) { + Probe::Known(Some(_)) => Ok(()), + Probe::Known(None) => Err(tesseract_unavailable()), + Probe::Pending => Err(tesseract_checking()), + } + }; + #[cfg(target_arch = "wasm32")] + let tesseract: Result<(), String> = Err(tesseract_unavailable()); + match engine { + EngineChoice::Ocrs => (!ocrs).then(models_missing), + EngineChoice::Tesseract => tesseract.err(), + EngineChoice::Auto => match (ocrs, tesseract) { + (true, _) | (false, Ok(())) => None, + (false, Err(why)) => Some(format!("{}; {why}", models_missing())), + }, + } +} + +/// Why Tesseract cannot be chosen yet: its probe is still running in the background. +#[cfg(not(target_arch = "wasm32"))] +fn tesseract_checking() -> String { + "still checking whether the tesseract program can run; try again in a moment".into() +} + +/// What a started job expects of the document when its result is applied: the same open +/// document, unchanged. [`Session::stale_reason`] names any difference it finds. +#[derive(Clone)] +pub struct OcrGuard { + pub(crate) doc: DocId, + pub(crate) pages: usize, + pub(crate) revisions: usize, + /// The document's working bytes when the job started; any edit rewrites them, so a shared + /// `Arc` pointer means "untouched". + pub(crate) bytes: Arc>, +} + /// Everything recognition needs from a document, detached from the session. pub struct OcrJob { bytes: Arc>, @@ -104,40 +618,162 @@ pub struct OcrJob { native: Vec>, pub pages: Vec, pub settings: OcrSettings, + /// What to check before applying this job's result. + pub guard: OcrGuard, +} + +/// A page's "has text" probe for skip-text: at least one non-blank text run. A failed probe +/// (the text layer could not be read) never enables a skip. +fn page_has_text(text: Option<&Arc>) -> bool { + text.is_some_and(|t| t.glyphs.iter().any(|g| !g.text.trim().is_empty())) +} + +/// Clamp a region to the image, snapping outward to whole pixels. `None` when nothing of it +/// is inside the image (empty, inverted, NaN, or wholly outside). +pub fn clamp_region(region: [f32; 4], width: u32, height: u32) -> Option<[u32; 4]> { + let (w, h) = (f64::from(width), f64::from(height)); + if region.iter().any(|v| !v.is_finite()) { + return None; + } + let (x0, y0, x1, y1) = (region[0] as f64, region[1] as f64, region[2] as f64, region[3] as f64); + let (x0, y0) = ((x0.floor().max(0.0).min(w)) as u32, (y0.floor().max(0.0).min(h)) as u32); + let (x1, y1) = ((x1.ceil().max(0.0).min(w)) as u32, (y1.ceil().max(0.0).min(h)) as u32); + (x1 > x0 && y1 > y0).then_some([x0, y0, x1, y1]) +} + +/// Map a recognized word from its source page's user space into the editable output's page +/// user space (the new page is the source's displayed size, unrotated): user → view → new +/// user. The box never moves relative to what the reviewer saw. +fn editable_word(w: &PlacedWord, info: &PageInfo) -> PlacedWord { + let to_new = |p: [f64; 2]| { + let v = info.user_to_view(p[0] as f32, p[1] as f32); + [f64::from(v[0]), f64::from(info.height) - f64::from(v[1])] + }; + let origin = to_new(w.origin); + let br = to_new([w.origin[0] + w.across[0], w.origin[1] + w.across[1]]); + let tl = to_new([w.origin[0] + w.up[0], w.origin[1] + w.up[1]]); + PlacedWord { + text: w.text.clone(), + origin, + across: [br[0] - origin[0], br[1] - origin[1]], + up: [tl[0] - origin[0], tl[1] - origin[1]], + ..w.clone() + } } impl OcrJob { + /// The pixels-per-point this job renders `page` at (the real render dpi is this times 72). + /// The Verify screen uses it to express a region picked on its own preview in the pixels + /// the job will actually read. A scan is read at its own resolution (enlarging it only + /// blurs the letters), at most the chosen one. + pub fn render_scale(&self, page: usize) -> f32 { + let wanted = self.settings.dpi.clamp(72.0, 600.0) / 72.0; + let Some(native) = self.native.get(page).copied().flatten() else { return wanted }; + // A scan whose page is larger than a sheet (often a box given in pixels, one point per + // pixel, #1848) is measured as if shrunk to one: otherwise 1 px/pt is already its full + // image, and no "Downsample to" choice could ask for less. + let fit = self.infos.get(page).map_or(1.0, |i| (SHEET / i.width.max(i.height).max(1e-3)).min(1.0)); + native.clamp(fit, wanted * fit) + } + + /// Whether any of the job's pages already has text (at least one non-blank run): the + /// batch flow's "Skip files that already contain text" probe. A failed probe never + /// enables a skip, so a page whose text cannot be read counts as no text there. + pub fn has_text(&self) -> bool { + let mut r = PageRenderer::new(self.bytes.clone(), RenderConfig { password: self.password.as_deref().map(Arc::from), ..Default::default() }); + self.pages.iter().any(|&page| { + let t = r.render(RenderRequest { page, kind: RequestKind::Text, scale: 1.0, ..Default::default() }); + page_has_text(t.text.as_ref()) + }) + } + /// Read the pages. `progress(done, total)` is called before each page; returning `false` /// stops (the pages read so far are returned). - pub fn run(self, ocr: &Ocr, mut progress: impl FnMut(usize, usize) -> bool) -> Vec { + pub fn run(self, recognizers: &Recognizers, mut progress: impl FnMut(usize, usize) -> bool) -> Vec { let config = RenderConfig { password: self.password.as_deref().map(Arc::from), hide_comments: self.settings.native_text_only, ..Default::default() }; let mut r = PageRenderer::new(self.bytes.clone(), config); let mut out = Vec::new(); let total = self.pages.len(); + let options = pdfcraft_ocr::RecognizeOptions { language: self.settings.language.clone() }; for (done, &page) in self.pages.iter().enumerate() { if !progress(done, total) { break; } let Some(info) = self.infos.get(page) else { continue }; - let skip = |why: &str| OcrPage { page, words: Vec::new(), already_on_page: 0, skipped: Some(why.into()) }; + let skip = |why: &str| OcrPage { page, words: Vec::new(), already_on_page: 0, skipped: Some(why.into()), notes: Vec::new() }; // The boxes of the page's own text (an earlier OCR layer's included), in view space. let existing: Vec<[f32; 4]> = r .render(RenderRequest { page, kind: RequestKind::Text, scale: 1.0, ..Default::default() }) .text .map(|t| t.glyphs.iter().filter(|g| !g.text.trim().is_empty()).map(|g| g.rect).collect()) .unwrap_or_default(); - if self.settings.skip_text_pages && !existing.is_empty() { + if self.settings.skip_text_pages && !self.settings.force_ocr && !existing.is_empty() { out.push(skip("the page already contains text")); continue; } - let shot = r.render(RenderRequest { page, scale: self.scale(page, info), ..Default::default() }); + // A scan is read at its own resolution (enlarging it only blurs the letters), at + // most the chosen one. + let scale = self.render_scale(page); + let shot = r.render(RenderRequest { page, scale, ..Default::default() }); if let Some(e) = shot.error { out.push(skip(&e)); continue; } - let lines = match ocr.recognize(&shot.rgba, shot.width, shot.height) { - Ok(l) => l, + let full = match pdfcraft_ocr::OcrImage::new(shot.width, shot.height, shot.rgba.to_vec()) { + Ok(img) => img, + Err(e) => { + out.push(skip(&e.to_string())); + continue; + } + }; + // The optional region: crop the rendered page, clamped; an empty region skips the + // page with a named reason (nothing was asked of it). + let (image, offset) = match self.settings.region { + None => (full, [0.0f32; 2]), + Some(region) => match clamp_region(region, shot.width, shot.height) { + Some([x0, y0, x1, y1]) => match crop(&full, x0, y0, x1, y1) { + Some(c) => (c, [x0 as f32, y0 as f32]), + None => { + out.push(skip("the selected region is empty")); + continue; + } + }, + None => { + out.push(skip("the selected region is outside the page")); + continue; + } + }, + }; + // Preprocessing sits between the page raster and recognition; its word boxes come + // back in processed coordinates and are mapped to the ORIGINAL raster's before + // placement. The render dpi is the real one (pixels per point × 72), never a + // fixed value. + let pre = pdfcraft_ocr::preprocess::preprocess( + image, + scale * 72.0, + &pdfcraft_ocr::preprocess::PreprocessOptions { + auto_rotate: self.settings.auto_rotate, + deskew: self.settings.deskew, + denoise: self.settings.denoise, + binarize: self.settings.binarize, + }, + ); + let (image, inverse) = match pre { + Ok(p) => (p.image, p.inverse_transform), + Err(e) => { + out.push(skip(&e.to_string())); + continue; + } + }; + let outcome = match pdfcraft_ocr::recognize_with_strategy( + recognizers.primary.as_ref(), + recognizers.secondary.as_deref(), + self.settings.strategy, + &image, + &options, + ) { + Ok(o) => o, Err(e) => { out.push(skip(&e.to_string())); continue; @@ -145,30 +781,69 @@ impl OcrJob { }; let scale = shot.width as f32 / info.width.max(1e-3); let to_user = |x: f32, y: f32| { - let [u, v] = info.view_to_user(x / scale, y / scale); + // processed pixels → original raster pixels (the exact inverse) → past the + // region crop → page view space. + let (ox, oy) = inverse.apply(f64::from(x), f64::from(y)); + let [u, v] = info.view_to_user((ox as f32 + offset[0]) / scale, (oy as f32 + offset[1]) / scale); [u as f64, v as f64] }; - // Words the page's own text already has are not added a second time. - let (fresh, already): (Vec<_>, Vec<_>) = - lines.iter().flat_map(|l| &l.words).partition(|w| covered(w.rect.map(|v| v / scale), &existing) < ALREADY_ON_PAGE); - let words = fresh.into_iter().map(|w| PlacedWord::place(w, to_user)).collect(); - out.push(OcrPage { page, words, already_on_page: already.len(), skipped: None }); + // Validate recognition geometry and compare against existing text in view space, + // undoing preprocessing and region offsets before testing overlap. + let (fresh, already): (Vec<_>, Vec<_>) = outcome.lines.iter().flat_map(|l| &l.words).filter(|w| box_writable(w)).partition(|w| { + let [l, t, r, b] = w.rect; + let bounds = [(l, t), (r, t), (r, b), (l, b)].into_iter().fold( + [f32::INFINITY, f32::INFINITY, f32::NEG_INFINITY, f32::NEG_INFINITY], + |mut bounds, (x, y)| { + let (x, y) = inverse.apply(f64::from(x), f64::from(y)); + let x = (x as f32 + offset[0]) / scale; + let y = (y as f32 + offset[1]) / scale; + bounds[0] = bounds[0].min(x); + bounds[1] = bounds[1].min(y); + bounds[2] = bounds[2].max(x); + bounds[3] = bounds[3].max(y); + bounds + }, + ); + covered(bounds, &existing) < ALREADY_ON_PAGE + }); + let words = fresh.into_iter().map(|w| PlacedWord::place(w, to_user)).collect::>(); + let mut notes = outcome.notes; + // Only visible editable output needs display glyphs. Searchable output preserves + // every Unicode character with empty Type3 programs and ToUnicode maps. + if self.settings.output_mode == OutputMode::EditableText && !words.is_empty() { + let missing = pdfcraft_fonts::TextPlan::for_words(words.iter().map(|w| w.text.as_str())).missing(); + if missing > 0 { + notes.push(format!("{missing} characters had no font that could show them and appear as ? in the output")); + } + } + out.push(OcrPage { page, words, already_on_page: already.len(), skipped: None, notes }); } progress(total, total); out } +} - /// The render scale (pixels per point) `page` is read at. A scan is read at its own - /// resolution (enlarging it only blurs the letters), at most the chosen one. - fn scale(&self, page: usize, info: &PageInfo) -> f32 { - let wanted = self.settings.dpi.clamp(72.0, 600.0) / 72.0; - let Some(native) = self.native.get(page).copied().flatten() else { return wanted }; - // A scan whose page is larger than a sheet (often a box given in pixels, one point per - // pixel, #1848) is measured as if shrunk to one: otherwise 1 px/pt is already its full - // image, and no "Downsample to" choice could ask for less. - let fit = (SHEET / info.width.max(info.height).max(1e-3)).min(1.0); - native.clamp(fit, wanted * fit) +/// Whether a word's pixel box may reach the content stream: width and height (right − left, +/// bottom − top) must be positive and finite. Anything else — a zero, negative, NaN or +/// infinite size — is dropped before placement rather than written into the page. +fn box_writable(word: &pdfcraft_ocr::Word) -> bool { + let [l, t, r, b] = word.rect; + let (width, height) = (r - l, b - t); + width > 0.0 && height > 0.0 && width.is_finite() && height.is_finite() +} + +/// Crop `[x0, y0, x1, y1]` out of `image` as a new [`OcrImage`] (the sides are already clamped +/// by [`clamp_region`]). +fn crop(image: &pdfcraft_ocr::OcrImage, x0: u32, y0: u32, x1: u32, y1: u32) -> Option { + let (w, h) = (x1.checked_sub(x0)?, y1.checked_sub(y0)?); + let stride = image.width() as usize * 4; + let mut out = Vec::with_capacity((w as usize).checked_mul(h as usize)?.checked_mul(4)?); + for y in y0..y1 { + let start = y as usize * stride + x0 as usize * 4; + let row = image.rgba().get(start..start + (x1 - x0) as usize * 4)?; + out.extend_from_slice(row); } + pdfcraft_ocr::OcrImage::new(w, h, out).ok() } impl Session { @@ -193,7 +868,8 @@ impl Session { .reduce(f32::max) }) .collect(); - Some(OcrJob { bytes: doc.bytes.clone(), password: doc.password.clone(), infos: doc.info.pages.clone(), native, pages, settings }) + let guard = OcrGuard { doc: id, pages: all, revisions: doc.revision_ends().len(), bytes: doc.bytes.clone() }; + Some(OcrJob { bytes: doc.bytes.clone(), password: doc.password.clone(), infos: doc.info.pages.clone(), native, pages, settings, guard }) } /// Add the words found by [`OcrJob::run`] as one undoable step. Returns the number of words. @@ -207,14 +883,129 @@ impl Session { Ok(words) } + /// Why a job's result may not be applied to this session any more: the document was closed, + /// edited, re-saved, or its pages changed since the job started. `None` means it is safe. + pub fn stale_reason(&self, guard: &OcrGuard) -> Option { + let Some(doc) = self.get(guard.doc) else { + return Some("the document is no longer open".into()); + }; + if !Arc::ptr_eq(&doc.bytes, &guard.bytes) { + return Some("the document changed since recognition started".into()); + } + let now = doc.info.pages.len(); + if now != guard.pages { + return Some(format!("the page count changed from {} to {} since recognition started", guard.pages, now)); + } + let revisions = doc.revision_ends().len(); + if revisions != guard.revisions { + let had = guard.revisions; + return Some(format!("the document gained revisions ({had} → {revisions}) since recognition started")); + } + None + } + + /// What applying a result to document `id` expects of it right now. After a checked apply + /// succeeds the document has changed (that apply is its newest edit), so a caller that + /// applies more of the same review in further steps (page by page) takes a fresh guard + /// here; any other edit in between still makes the next apply stale. + pub fn ocr_guard(&self, id: DocId) -> Option { + let doc = self.get(id)?; + Some(OcrGuard { doc: id, pages: doc.info.pages.len(), revisions: doc.revision_ends().len(), bytes: doc.bytes.clone() }) + } + + /// [`Self::apply_ocr`], but only while the document is still the one that was read; a stale + /// result is refused with a named reason and nothing is written. + pub fn apply_ocr_checked(&mut self, id: DocId, found: &[OcrPage], guard: &OcrGuard) -> Result { + if guard.doc != id { + return Err("the result belongs to another document".into()); + } + if let Some(why) = self.stale_reason(guard) { + return Err(why); + } + self.apply_ocr(id, found).map_err(|e| e.to_string()) + } + + /// The editable-text output: a NEW document with the recognized words as VISIBLE text at + /// their original boxes and no scan image — one page per source page, blank where nothing + /// was recognized, each page the displayed size of its source. The open document is never + /// touched; the caller chooses where the bytes are written. Refuses when there are no + /// words at all (an empty document would silently lose the scan's content). + pub fn editable_ocr_bytes(&self, id: DocId, found: &[OcrPage]) -> Result>, String> { + let doc = self.get(id).ok_or("no such document")?; + let words = layer::validate_words(found.iter().flat_map(|p| &p.words)).map_err(|e| e.to_string())?; + if words == 0 { + return Err("the recognition found no words, so there is no editable text to write".into()); + } + // The words per page, in user space; and one plan over all of them, so a face's Type3 + // font is built once and shared by every page that uses it. + let page_words: Vec> = (0..doc.info.pages.len()) + .map(|p| { + let Some(info) = doc.info.pages.get(p) else { return Vec::new() }; + found.iter().find(|f| f.page == p).map(|f| f.words.iter().map(|w| editable_word(w, info)).collect()).unwrap_or_default() + }) + .collect(); + layer::validate_words(page_words.iter().flatten()).map_err(|e| e.to_string())?; + let mut plan = pdfcraft_fonts::TextPlan::for_words(page_words.iter().flat_map(|ws| ws.iter().map(|w| w.text.as_str()))); + let pages: Vec<(f64, f64, pdfcraft_cos::Dict, Vec)> = (0..doc.info.pages.len()) + .map(|p| { + let info = doc.info.pages.get(p); + (info.map_or(612.0, |i| f64::from(i.width)), info.map_or(792.0, |i| f64::from(i.height)), pdfcraft_cos::Dict::new(), Vec::new()) + }) + .collect(); + let mut created = pdfcraft_create::from_contents(&pages).map_err(|e| e.to_string())?; + // One shared Helvetica for every page: added once into the new document, then linked + // from each page's resources as /PCHelv (the name the text layers draw with). The + // plan's fonts are linked per page too; their objects are shared. + let mut font = pdfcraft_cos::Dict::new(); + font.set(b"Type".to_vec(), pdfcraft_cos::Object::name("Font")); + font.set(b"Subtype".to_vec(), pdfcraft_cos::Object::name("Type1")); + font.set(b"BaseFont".to_vec(), pdfcraft_cos::Object::name("Helvetica")); + font.set(b"Encoding".to_vec(), pdfcraft_cos::Object::name("WinAnsiEncoding")); + let fr = created.add(font); + // The page tree's kids, direct array or indirect reference (a fresh create document + // keeps /Kids inline) — the output's pages are linked through them. + let kids = created + .root() + .and_then(|r| created.get(r).as_dict().cloned()) + .and_then(|d| d.reference(b"Pages")) + .and_then(|p| created.get(p).as_dict().cloned()) + .and_then(|d| match d.get(b"Kids").map(|k| created.resolve(k)).as_deref() { + Some(pdfcraft_cos::Object::Array(a)) => Some(a.clone()), + Some(pdfcraft_cos::Object::Ref(r)) => created.get(*r).as_array().cloned(), + _ => None, + }); + for (pi, kid) in kids.into_iter().flatten().enumerate() { + let pdfcraft_cos::Object::Ref(page) = kid else { continue }; + let _ = created.update_dict(page, |d| { + let mut res = d.get(b"Resources").and_then(|o| o.as_dict().cloned()).unwrap_or_default(); + let mut fonts = res.get(b"Font").and_then(|o| o.as_dict().cloned()).unwrap_or_default(); + fonts.set(b"PCHelv".to_vec(), pdfcraft_cos::Object::Ref(fr)); + res.set(b"Font".to_vec(), pdfcraft_cos::Object::Dict(fonts)); + d.set(b"Resources".to_vec(), pdfcraft_cos::Object::Dict(res)); + }); + // The page's content is written after the fonts carry their resource names, so + // the runs can reference them. + if page_words.get(pi).is_some_and(|ws| !ws.is_empty()) { + pdfcraft_edit::add_type3_fonts(&mut created, pi, &mut plan).map_err(|e| e.to_string())?; + let word_offset = page_words.iter().take(pi).map(Vec::len).sum(); + let content = pdfcraft_ocr::visible_text_layer_at(&page_words[pi], &plan, word_offset); + let mut dict = pdfcraft_cos::Dict::new(); + dict.set(b"PCMark".to_vec(), pdfcraft_cos::Object::name("OCR")); + let stream = created.add(pdfcraft_cos::Object::Stream(pdfcraft_cos::Stream::flate(dict, &content))); + let _ = created.update_dict(page, |d| d.set(b"Contents".to_vec(), pdfcraft_cos::Object::Ref(stream))); + } + } + self.write_new(&created).map_err(|e| e.to_string()) + } + /// Recognise text on `pages` and add it, in one call (the CLI and agents use this). pub fn recognize_text(&mut self, id: DocId, pages: &[usize], settings: OcrSettings) -> Result, String> { let job = self.ocr_job(id, pages, settings).ok_or("no such document")?; if let Some(why) = self.get(id).and_then(|d| d.read_only_reason.clone()) { return Err(why); } - let ocr = engine()?; - let found = job.run(&ocr, |_, _| true); + let recognizers = recognizers(&job.settings)?; + let found = job.run(&recognizers, |_, _| true); self.apply_ocr(id, &found).map_err(|e| e.to_string())?; Ok(found) } @@ -226,22 +1017,68 @@ pub struct FileResult { /// The new file (incrementally saved), or the original when nothing was recognised. pub bytes: Arc>, pub pages: Vec, + /// Why the whole file was left alone ("Skip files that already contain text"): a skipped + /// file is its own result bucket, never counted as an OCR success. + pub file_skipped: Option, } impl FileResult { pub fn words(&self) -> usize { self.pages.iter().map(|p| p.words.len()).sum() } + + /// Whether the file was passed over entirely (never an OCR success). + pub fn skipped(&self) -> bool { + self.file_skipped.is_some() + } +} + +/// The output names a multi-file run would write more than once: the batch writes every +/// result into one folder under its input's file name, so two inputs of the same name from +/// different folders would overwrite each other. Compared case-insensitively (the Windows and +/// macOS file systems are), in first-seen order, each listed once. +pub fn duplicate_output_names<'a>(names: impl IntoIterator) -> Vec { + let mut seen = std::collections::HashSet::new(); + let mut listed = std::collections::HashSet::new(); + let mut out = Vec::new(); + for name in names { + let key = name.to_lowercase(); + if !seen.insert(key.clone()) && listed.insert(key) { + out.push(name.to_string()); + } + } + out +} + +/// The batch note's data: how many words sit at or under `threshold` ([`low_confidence`], the +/// separate batch rule), and the 1-based, sorted pages they are on. `None` when there are +/// none, so a note is only written when there is something to review. +pub fn low_confidence_pages(pages: &[OcrPage], threshold: f32) -> Option<(usize, Vec)> { + let (mut count, mut page_numbers) = (0usize, Vec::new()); + for p in pages { + if p.skipped.is_some() { + continue; + } + let on_page = p.words.iter().filter(|w| w.confidence.is_some_and(|c| pdfcraft_ocr::low_confidence(c, threshold))).count(); + if on_page > 0 { + count += on_page; + page_numbers.push(p.page + 1); + } + } + page_numbers.sort_unstable(); + (count > 0).then_some((count, page_numbers)) } /// Recognize text in multiple files: one PDF's bytes in, the searchable PDF out. `progress` -/// works as for [`OcrJob::run`]. +/// works as for [`OcrJob::run`]. A file that the skip-files option passes over comes back with +/// [`FileResult::file_skipped`] set and its original bytes; the caller decides what to do with +/// it, but it is never an OCR success. pub fn recognize_file( name: &str, bytes: Arc>, password: Option<&str>, settings: OcrSettings, - ocr: &Ocr, + recognizers: &Recognizers, progress: impl FnMut(usize, usize) -> bool, ) -> Result { let mut s = Session::new(); @@ -249,15 +1086,31 @@ pub fn recognize_file( if let Some(why) = s.get(id).and_then(|d| d.read_only_reason.clone()) { return Err(why); } - let job = s.ocr_job(id, &[], settings).ok_or("the document could not be read")?; - let pages = job.run(ocr, progress); + let job = s.ocr_job(id, &[], settings.clone()).ok_or("the document could not be read")?; + // The skip-files probe runs before any engine work: a file with text needs no engine at + // all, and the original bytes come back unchanged. + if settings.skip_text_files && !settings.force_ocr && job.has_text() { + let pages = job + .pages + .iter() + .map(|&page| OcrPage { + already_on_page: 0, + page, + words: Vec::new(), + skipped: Some("the file already contains text".into()), + notes: Vec::new(), + }) + .collect(); + return Ok(FileResult { bytes, pages, file_skipped: Some("the file already contains text".into()) }); + } + let pages = job.run(recognizers, progress); if s.apply_ocr(id, &pages).map_err(|e| e.to_string())? == 0 { - return Ok(FileResult { bytes, pages }); + return Ok(FileResult { bytes, pages, file_skipped: None }); } // Checked like a save before the caller writes it (#2166). let bytes = s.save_bytes(id).map_err(|e| e.to_string())?; s.check_save(id, &bytes).map_err(|e| e.to_string())?; - Ok(FileResult { bytes, pages }) + Ok(FileResult { bytes, pages, file_skipped: None }) } #[cfg(test)] @@ -282,28 +1135,6 @@ mod tests { assert_eq!(covered([100.5, 52.0, 134.0, 61.0], &[]), 0.0); } - /// #1848: an A4 scan at 300 dpi whose page box is its pixel size (2480 x 3508 pt, one pixel - /// per point) gets a smaller bitmap for each lower "Downsample to" choice, and a low-resolution - /// letter page is still never shrunk below one pixel per point. - #[test] - fn downsample_shrinks_a_page_box_given_in_pixels() { - let mut s = Session::new(); - let page = |w: u32, h: u32| export::encode_png(w, h, &vec![255; (w * h * 4) as usize]).unwrap(); - let bytes = s.create_from_images(&[("a4.png".into(), page(2480, 3508)), ("letter.png".into(), page(612, 792))]).unwrap(); - let id = s.open("scan.pdf", None, bytes, None).unwrap(); - let mut r = PageRenderer::new(s.get(id).unwrap().bytes.clone(), RenderConfig::default()); - for (dpi, a4) in [(72.0, [595, 842]), (150.0, [1240, 1754]), (300.0, [2480, 3508]), (600.0, [2480, 3508])] { - let job = s.ocr_job(id, &[], OcrSettings { dpi, ..Default::default() }).unwrap(); - for (page, want) in [(0, a4), (1, [612, 792])] { - let scale = job.scale(page, &job.infos[page]); - let shot = r.render(RenderRequest { page, scale, ..Default::default() }); - assert!(shot.error.is_none(), "{:?}", shot.error); - let near = |got: u32, want: u32| got.abs_diff(want) <= 1; - assert!(near(shot.width, want[0]) && near(shot.height, want[1]), "{dpi} dpi, page {page}: {}x{}", shot.width, shot.height); - } - } - } - #[test] fn coverage_of_overlapping_or_broken_boxes_stays_between_zero_and_one() { // A glyph drawn twice (fake bold) does not cover more than the whole word. @@ -317,4 +1148,846 @@ mod tests { let c = covered([0.0, 0.0, 10.0, 10.0], &broken); assert!((0.0..=1.0).contains(&c), "{c}"); } + use crate::tests::fixture; + use pdfcraft_render::TextGlyph; + + fn glyph(text: &str) -> TextGlyph { + TextGlyph { text: text.into(), rect: [0.0, 0.0, 1.0, 1.0], quad: [[0.0, 0.0], [1.0, 0.0], [1.0, 1.0], [0.0, 1.0]], direction: [1.0, 0.0] } + } + + /// The skip-text probe: at least one non-blank run; blank-only or missing text never + /// enables a skip. + #[test] + fn has_text_probe_needs_one_non_blank_run() { + let none: Option> = None; + assert!(!page_has_text(none.as_ref()), "a failed probe never enables a skip"); + assert!(!page_has_text(Some(Arc::new(pdfcraft_render::PageText::default())).as_ref())); + let blank = pdfcraft_render::PageText { glyphs: vec![glyph(" "), glyph("")], ..Default::default() }; + assert!(!page_has_text(Some(Arc::new(blank)).as_ref())); + let has = pdfcraft_render::PageText { glyphs: vec![glyph(" "), glyph("a")], ..Default::default() }; + assert!(page_has_text(Some(Arc::new(has)).as_ref())); + } + + #[test] + fn output_mode_parsing_falls_back_to_searchable() { + assert_eq!(OutputMode::parse("editable"), OutputMode::EditableText); + assert_eq!(OutputMode::parse(" Editable-Text "), OutputMode::EditableText); + assert_eq!(OutputMode::parse("searchable"), OutputMode::Searchable); + assert_eq!(OutputMode::parse("nonsense"), OutputMode::Searchable, "unknown strings mean Searchable"); + assert_eq!(OutputMode::parse(""), OutputMode::Searchable); + } + + #[test] + fn engine_choice_parsing() { + assert_eq!(EngineChoice::parse("tesseract"), EngineChoice::Tesseract); + assert_eq!(EngineChoice::parse(" AUTO "), EngineChoice::Auto); + assert_eq!(EngineChoice::parse("ocrs"), EngineChoice::Ocrs); + assert_eq!(EngineChoice::parse("wat"), EngineChoice::Ocrs, "unknown names fall back to the default"); + assert_eq!(EngineChoice::default().name(), "ocrs"); + } + + /// Automatic reads with the second engine when the first is unsure unless a strategy was + /// chosen: "Primary only" would never run it. A named engine reads alone. + #[test] + fn automatic_defaults_to_the_confidence_weighted_strategy() { + assert_eq!(EngineChoice::Auto.default_strategy(), MergeStrategy::ConfidenceWeighted); + assert_eq!(EngineChoice::Ocrs.default_strategy(), MergeStrategy::PrimaryOnly); + assert_eq!(EngineChoice::Tesseract.default_strategy(), MergeStrategy::PrimaryOnly); + } + + /// Settings default to today's behavior: ocrs, PrimaryOnly, no preprocessing, searchable. + #[test] + fn settings_default_to_the_ocrs_status_quo() { + let s = OcrSettings::default(); + assert_eq!(s.engine, EngineChoice::Ocrs); + assert_eq!(s.strategy, MergeStrategy::PrimaryOnly); + assert_eq!(s.output_mode, OutputMode::Searchable); + assert!(!s.auto_rotate && !s.deskew && !s.denoise && !s.binarize); + assert_eq!(s.region, None); + assert_eq!(s.low_confidence_threshold, 60.0); + assert_eq!(s.tesseract_path, None, "no configured tesseract path by default: the PATH decides"); + } + + /// A configured tesseract path that is not an absolute, existing, executable file makes + /// the Tesseract choice unavailable — the explicit setting is never silently replaced by + /// a PATH search (and this holds whatever this machine has installed). + #[test] + fn an_invalid_configured_tesseract_path_is_unavailable_not_a_fallback() { + let bogus = OcrSettings { engine: EngineChoice::Tesseract, tesseract_path: Some("not/a/program".into()), ..Default::default() }; + match recognizers(&bogus) { + Ok(r) => panic!("a bogus configured path must not resolve: {:?}", r.primary.id()), + Err(e) => assert!(e.contains("tesseract"), "{e}"), + } + } + + /// A failed engine probe is remembered only for the negative TTL — a program installed + /// while the app runs is found by a later job, without a restart — while a found engine is + /// remembered for the session. + #[test] + fn a_negative_probe_expires_a_found_engine_does_not() { + let now = Instant::now(); + let old = now.checked_sub(Duration::from_secs(31)).unwrap(); + assert!(!probe_fresh(false, old, now), "past the TTL the probe is retried"); + assert!(probe_fresh(false, now - Duration::from_secs(5), now), "a fresh failure is not re-probed"); + assert!(probe_fresh(true, old, now), "a found engine is kept"); + } + + /// `ocr_status` and a job judge the same location: one named by the job's settings is the + /// one reported (here an invalid one, so no process starts and the answer is the same on + /// every machine). + #[test] + fn the_status_follows_the_location_a_job_names() { + let bogus = OcrSettings { tesseract_path: Some("not/a/program".into()), ..Default::default() }; + let tesseract = engines_status_for(&bogus).into_iter().find(|e| e.id == "tesseract").expect("tesseract is always listed"); + assert!(!tesseract.available, "{tesseract:?}"); + assert!(tesseract.why_not.as_deref().is_some_and(|why| why.contains("not/a/program")), "{tesseract:?}"); + assert_eq!(tesseract.program.as_deref(), Some("not/a/program")); + } + + /// What the interface sees of the cache: a fresh answer as is, an expired failed probe as + /// the best known answer (shown while it is retried, so the choice does not flicker), and + /// nothing for a location nobody asked about. + #[test] + fn the_cache_lookup_keeps_a_stale_answer_and_knows_its_location() { + let now = Instant::now(); + let old = now.checked_sub(Duration::from_secs(31)).unwrap(); + let failed = |key: Option<&str>, at| CachedTesseract { key: key.map(str::to_string), found: None, at }; + assert!(matches!(cache_lookup(None, None, now), Cached::Missing)); + assert!(matches!(cache_lookup(Some(&failed(None, now)), None, now), Cached::Fresh(None))); + assert!(matches!(cache_lookup(Some(&failed(None, old)), None, now), Cached::Stale(None))); + assert!(matches!(cache_lookup(Some(&failed(Some("/x"), now)), None, now), Cached::Missing), "another location is another question"); + let found = CachedTesseract { key: None, found: Some(Arc::new(Fake::new())), at: old }; + assert!(matches!(cache_lookup(Some(&found), None, now), Cached::Fresh(Some(_))), "a found engine stays fresh"); + } + + /// Engine discovery never deadlocks or corrupts the cache: concurrent lookups with + /// different configured locations (all invalid, so they are refused before any process + /// starts) all finish. + #[test] + fn concurrent_engine_lookups_all_finish() { + let handles: Vec<_> = (0..8) + .map(|i| { + std::thread::spawn(move || { + let configured = if i % 2 == 0 { Some(format!("not/a/program-{i}")) } else { None }; + let _ = tesseract_engine(configured.as_deref()); + }) + }) + .collect(); + for h in handles { + h.join().expect("no panic, no deadlock"); + } + } + + /// The preprocessing flags are not just plumbing: a page read with all four on still + /// yields the same words, in the same places (needs the models). + #[test] + fn preprocessing_flags_flow_through_the_job() { + if !available() { + eprintln!("skipped: no OCR engine available"); + return; + } + let mut s = Session::new().with_clock(|| 1_700_000_000); + let text = s.create_from_text("t", "The quick brown fox jumps over the lazy dog.").unwrap(); + let id = s.open("text.pdf", None, text, None).unwrap(); + let png = crate::export::Exporter::new(s.get(id).unwrap()).png(0, 150.0).unwrap(); + let scan = s.create_from_images(&[("scan.png".into(), png)]).unwrap(); + let id = s.open("scan.pdf", None, scan, None).unwrap(); + let settings = OcrSettings { auto_rotate: true, deskew: true, denoise: true, binarize: true, ..Default::default() }; + let found = s.recognize_text(id, &[], settings).unwrap(); + let text = crate::tests::page_texts(&s, id)[0].to_lowercase(); + // Binarizing an anti-aliased render thins the thin strokes, so edge letters drop + // ("lazy" reads as "azy", "brown" as "rown"); the match rests on what stays stable — + // the middle of "quick" (some CPUs' kernels read the model's "q" as an "a", the same + // "uick" the OCR test in crate::tests pins) and whole short words. + for w in ["uick", "fox", "over"] { + assert!(text.contains(w), "{text}"); + } + // The words were placed, meaning boxes survived the trip back to raster coordinates. + assert!(!found[0].words.is_empty()); + } + + /// suspects(): only Low-band words, by index. + #[test] + fn page_suspects_are_the_low_band_words() { + let word = |t: &str, c: Option| PlacedWord { + text: t.into(), + origin: [0.0; 2], + across: [1.0, 0.0], + up: [0.0, 1.0], + confidence: c, + source: "ocrs".into(), + }; + let page = OcrPage { + already_on_page: 0, + page: 0, + words: vec![word("a", Some(95.0)), word("b", Some(50.0)), word("c", None), word("d", Some(69.0))], + skipped: None, + notes: Vec::new(), + }; + assert_eq!(page.suspects(), vec![1, 3]); + let mean = page.mean_confidence().unwrap_or_default(); + assert!((mean - 71.333).abs() < 0.01, "{mean}"); + } + + /// Region clamping: interior kept, negative extents snapped outward, garbage refused. + #[test] + fn regions_clamp_and_refuse_garbage() { + assert_eq!(clamp_region([10.0, 10.0, 20.0, 20.0], 100, 100), Some([10, 10, 20, 20])); + assert_eq!(clamp_region([-5.0, -5.5, 20.4, 20.0], 100, 100), Some([0, 0, 21, 20]), "snapped outward, clamped to the image"); + assert_eq!(clamp_region([50.0, 50.0, 200.0, 200.0], 100, 100), Some([50, 50, 100, 100])); + assert_eq!(clamp_region([10.0, 10.0, 10.0, 20.0], 100, 100), None, "empty"); + assert_eq!(clamp_region([20.0, 10.0, 10.0, 20.0], 100, 100), None, "inverted"); + assert_eq!(clamp_region([f32::NAN, 0.0, 1.0, 1.0], 100, 100), None, "NaN"); + assert_eq!(clamp_region([-20.0, -20.0, -5.0, -5.0], 100, 100), None, "wholly outside"); + } + + /// A word box only reaches the content stream with positive, finite geometry: NaN, + /// infinite, negative and zero-sized boxes are dropped before placement, so nothing + /// non-finite is ever stamped onto the page. + #[test] + fn non_finite_and_negative_word_boxes_are_dropped_before_placement() { + let word = |rect: [f32; 4]| pdfcraft_ocr::Word::new("w", rect, "ocrs"); + for (why, rect) in [ + ("NaN width", [f32::NAN, 0.0, 10.0, 10.0]), + ("NaN height", [0.0, 0.0, 10.0, f32::NAN]), + ("infinite right", [0.0, 0.0, f32::INFINITY, 10.0]), + ("1e39 height overflows to inf", [0.0, 0.0, 10.0, 1e39f64 as f32]), + ("negative width", [10.0, 0.0, 0.0, 10.0]), + ("negative height", [0.0, 10.0, 10.0, 0.0]), + ("zero size", [5.0, 5.0, 5.0, 5.0]), + ] { + assert!(!box_writable(&word(rect)), "{why}: {rect:?}"); + } + assert!(box_writable(&word([0.0, 0.0, 10.0, 10.0])), "a sane box is placed"); + } + + /// A crop takes exactly the asked pixels (this feeds words back to page space, so edges + /// matter). + #[test] + fn crops_take_the_asked_pixels() { + let img = pdfcraft_ocr::OcrImage::new(3, 2, (0..6).flat_map(|p| vec![p as u8; 4]).collect()).unwrap(); + let c = crop(&img, 1, 0, 3, 2).unwrap(); + assert_eq!((c.width(), c.height()), (2, 2)); + assert_eq!(c.rgba()[0], 1, "the crop starts at x=1"); + assert_eq!(c.rgba()[4], 2, "the first row's second column is x=2"); + assert_eq!(c.rgba()[8], 4, "the second row starts at y=1"); + } + + /// The stale guard: edits and closing are named and the result is refused; a page-count or + /// revision difference is named too (any edit rewrites the working bytes, so those branches + /// are reached with a hand-built guard). + #[test] + fn stale_results_are_named_and_refused() { + let mut s = Session::new(); + let id = s.open("t.pdf", None, Arc::new(fixture(2)), None).unwrap(); + let job = s.ocr_job(id, &[], OcrSettings::default()).unwrap(); + assert_eq!(s.stale_reason(&job.guard), None, "freshly captured"); + // Any edit rewrites the working bytes. + s.apply(id, Edit::AddOcrText { page: 0, words: vec![] }).unwrap(); + assert_eq!(s.stale_reason(&job.guard).as_deref(), Some("the document changed since recognition started")); + assert!(s.apply_ocr_checked(id, &[], &job.guard).is_err(), "a stale result is never applied"); + // A fresh guard passes. + let fresh = s.ocr_job(id, &[], OcrSettings::default()).unwrap(); + assert_eq!(s.stale_reason(&fresh.guard), None); + // A page count and a revision count from another life are named too. + let doc = s.get(id).unwrap(); + let (bytes, revisions) = (doc.bytes.clone(), doc.revision_ends().len()); + let guard = OcrGuard { doc: id, pages: 5, revisions, bytes }; + assert!(s.stale_reason(&guard).is_some_and(|r| r.contains("page count changed")), "{}", s.stale_reason(&guard).unwrap()); + let guard = OcrGuard { doc: id, pages: 2, revisions: revisions + 3, bytes: s.get(id).unwrap().bytes.clone() }; + assert!(s.stale_reason(&guard).is_some_and(|r| r.contains("revisions")), "{}", s.stale_reason(&guard).unwrap()); + // A guard for a different document refuses up front. + let other = s.open("other.pdf", None, Arc::new(fixture(1)), None).unwrap(); + let cross = s.apply_ocr_checked(other, &[], &fresh.guard); + assert!(cross.as_ref().is_err_and(|e| e.contains("another document")), "{cross:?}"); + // And so is a closed document. + s.close(id); + assert_eq!(s.stale_reason(&fresh.guard).as_deref(), Some("the document is no longer open")); + } + + /// The stale guard, exercised end to end through the recognition flow itself (the flow the + /// UI's Verify screen and the automation's two-phase path drive): capture a job on a + /// synthetic scanned page, run the recognition, edit the document, and the captured result + /// is refused with its named reason and nothing written; a fresh recognition over the + /// edited document applies. + #[test] + fn stale_results_are_refused_end_to_end_through_the_recognition_flow() { + if !available() { + eprintln!("skipped: no OCR engine available"); + return; + } + let mut s = Session::new().with_clock(|| 1_700_000_000); + let text = s.create_from_text("t", "The quick brown fox jumps over the lazy dog.").unwrap(); + let id = s.open("text.pdf", None, text, None).unwrap(); + let png = crate::export::Exporter::new(s.get(id).unwrap()).png(0, 150.0).unwrap(); + let scan = s.create_from_images(&[("scan.png".into(), png)]).unwrap(); + let id = s.open("scan.pdf", None, scan, None).unwrap(); + + // Recognize: the job (carrying its guard) is captured first, then the reading runs. + let job = s.ocr_job(id, &[], OcrSettings::default()).unwrap(); + let guard = job.guard.clone(); + let r = recognizers(&job.settings).unwrap(); + let found = job.run(&r, |_, _| true); + assert!(!found[0].words.is_empty(), "the scan was read: {:?}", found[0].skipped); + + // The document is edited while the result sits around; any edit rewrites the bytes. + s.apply(id, Edit::AddOcrText { page: 0, words: vec![] }).unwrap(); + + // Applying the now-stale result is refused with its named reason and writes nothing. + let after_edit = s.get(id).unwrap().bytes.clone(); + assert!(!Arc::ptr_eq(&after_edit, &guard.bytes), "the edit rewrote the working bytes"); + let revisions = s.get(id).unwrap().revision_ends().len(); + let why = s.apply_ocr_checked(id, &found, &guard).unwrap_err(); + assert_eq!(why, "the document changed since recognition started", "{why}"); + assert_eq!(s.get(id).unwrap().revision_ends().len(), revisions, "the refusal wrote no revision"); + assert!(Arc::ptr_eq(&s.get(id).unwrap().bytes, &after_edit), "the refused apply wrote nothing"); + assert!(crate::tests::page_texts(&s, id)[0].trim().is_empty(), "no text layer landed"); + + // A fresh recognition over the current document applies: same flow, valid now. + let job = s.ocr_job(id, &[], OcrSettings::default()).unwrap(); + let guard = job.guard.clone(); + assert_eq!(s.stale_reason(&guard), None, "a fresh job is not stale"); + let found = job.run(&r, |_, _| true); + let words = s.apply_ocr_checked(id, &found, &guard).unwrap(); + assert!(words > 0, "the re-run found the words again"); + let text = crate::tests::page_texts(&s, id)[0].to_lowercase(); + assert!(text.contains("fox"), "{text}"); + } + + /// The selector refuses an unusable engine with its cause named, and resolves the ensemble + /// when both engines exist (here: whatever this machine has for ocrs and tesseract). + #[test] + fn selector_names_the_missing_engine() { + let tesseract = OcrSettings { engine: EngineChoice::Tesseract, ..Default::default() }; + match recognizers(&tesseract) { + Ok(r) => assert_eq!(r.primary.id(), "tesseract"), + Err(e) => assert!(e.contains("tesseract"), "{e}"), + } + let ocrs = OcrSettings { engine: EngineChoice::Ocrs, ..Default::default() }; + match recognizers(&ocrs) { + Ok(r) => assert_eq!(r.primary.id(), "ocrs"), + Err(e) => assert!(e.contains("models"), "{e}"), + } + } + + /// An ocrs-only choice never starts the Tesseract probe: the probe is a `--list-langs` + /// process, and a job that names its engine needs no reading from Tesseract. The test's + /// fake program touches a marker file when it runs, so the probe's start is observable; + /// it needs a Unix shell, like the other process fixtures. + #[cfg(unix)] + #[test] + fn an_ocrs_only_choice_never_probes_tesseract() { + use std::os::unix::fs::PermissionsExt; + let probe = |name: &str| { + let dir = std::env::temp_dir().join(format!("pdfcraft-ocr-probe-{}-{name}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let program = dir.join("fake-tesseract"); + std::fs::write(&program, "#!/bin/sh\ntouch \"$0.probed\"\nexit 1\n").unwrap(); + let mut perms = std::fs::metadata(&program).unwrap().permissions(); + perms.set_mode(0o755); + std::fs::set_permissions(&program, perms).unwrap(); + (dir.join("fake-tesseract.probed"), program.to_string_lossy().into_owned()) + }; + // An ocrs-only job: the configured location stays untouched. + let (marker, program) = probe("ocrs-only"); + let ocrs_only = OcrSettings { engine: EngineChoice::Ocrs, tesseract_path: Some(program), ..Default::default() }; + let _ = recognizers(&ocrs_only); + assert!(!marker.exists(), "an ocrs-only job must not start the tesseract probe"); + std::fs::remove_file(marker.parent().unwrap().join("fake-tesseract")).ok(); + + // A Tesseract job does validate its program (that is how it is found usable). + let (marker, program) = probe("tesseract"); + let wants = OcrSettings { engine: EngineChoice::Tesseract, tesseract_path: Some(program), ..Default::default() }; + let _ = recognizers(&wants); + assert!(marker.exists(), "a tesseract job probes its program"); + } + + /// A fake engine for the batch skip logic: reads one confident word from anything, counts + /// its calls, and never touches the filesystem. + struct Fake { + calls: std::sync::atomic::AtomicUsize, + } + + impl Fake { + fn new() -> Fake { + Fake { calls: std::sync::atomic::AtomicUsize::new(0) } + } + fn calls(&self) -> usize { + self.calls.load(std::sync::atomic::Ordering::SeqCst) + } + } + + impl Recognizer for Fake { + fn id(&self) -> &'static str { + "fake" + } + fn available(&self) -> bool { + true + } + fn languages(&self) -> Vec<(String, String)> { + vec![("en".into(), "English".into())] + } + fn recognize(&self, _image: &pdfcraft_ocr::OcrImage, _options: &pdfcraft_ocr::RecognizeOptions) -> Result, OcrError> { + self.calls.fetch_add(1, std::sync::atomic::Ordering::SeqCst); + Ok(vec![pdfcraft_ocr::Line { words: vec![pdfcraft_ocr::Word::new("word", [1.0, 1.0, 9.0, 9.0], "fake").with_confidence(95.0)] }]) + } + } + + fn fake_recognizers() -> (Recognizers, Arc) { + let fake = Arc::new(Fake::new()); + (Recognizers { primary: fake.clone(), secondary: None }, fake) + } + + /// Skip-files: a file that already has text is passed over entirely — the engine never + /// runs, the original bytes come back, and the result is its own skipped bucket. + #[test] + fn skip_files_passes_text_files_over_without_running_the_engine() { + let s = Session::new(); + let bytes = s.create_from_text("t", "Already typed here").unwrap(); + let (r, fake) = fake_recognizers(); + let settings = OcrSettings { skip_text_files: true, ..Default::default() }; + let out = recognize_file("t.pdf", bytes.clone(), None, settings, &r, |_, _| true).unwrap(); + assert_eq!(fake.calls(), 0, "the engine never ran"); + assert_eq!(out.bytes.as_ptr(), bytes.as_ptr(), "the original bytes came back"); + assert_eq!(out.file_skipped.as_deref(), Some("the file already contains text")); + assert!(out.skipped() && out.words() == 0); + } + + /// Force OCR overrides the file skip: the file is read even though it has text, and the + /// result is an OCR success (new bytes, words placed), never a skipped bucket. + #[test] + fn force_ocr_overrides_the_file_skip() { + let s = Session::new(); + let bytes = s.create_from_text("t", "Already typed here").unwrap(); + let (r, fake) = fake_recognizers(); + let settings = OcrSettings { skip_text_files: true, force_ocr: true, ..Default::default() }; + let out = recognize_file("t.pdf", bytes, None, settings, &r, |_, _| true).unwrap(); + assert_eq!(fake.calls(), 1); + assert_eq!(out.file_skipped, None); + assert_eq!(out.words(), 1); + assert!(!out.bytes.is_empty()); + } + + /// Force OCR also overrides the per-page skip: a page with text is read anyway. + #[test] + fn force_ocr_overrides_the_page_skip() { + let s = Session::new(); + let bytes = s.create_from_text("t", "A page with text").unwrap(); + let (r, fake) = fake_recognizers(); + let settings = OcrSettings { force_ocr: true, ..Default::default() }; + let out = recognize_file("t.pdf", bytes, None, settings, &r, |_, _| true).unwrap(); + assert_eq!(fake.calls(), 1, "the page's own text did not stop the run"); + assert_eq!(out.pages[0].skipped, None); + } + + /// The per-page skip stays the default: with force off, a text page is passed over (and a + /// second run over a searchable output is a no-op — nothing left to read). + #[test] + fn page_skip_leaves_text_pages_alone_and_a_rerun_is_a_noop() { + let s = Session::new(); + let bytes = s.create_from_text("t", "A page with text").unwrap(); + let (r, fake) = fake_recognizers(); + let out = recognize_file("t.pdf", bytes, None, OcrSettings::default(), &r, |_, _| true).unwrap(); + assert_eq!(fake.calls(), 0); + assert_eq!(out.pages[0].skipped.as_deref(), Some("the page already contains text")); + assert_eq!(out.words(), 0); + // Second run over the same (already text-bearing) file: skipped pages again, no calls. + let again = recognize_file("t.pdf", out.bytes, None, OcrSettings::default(), &r, |_, _| true).unwrap(); + assert_eq!(fake.calls(), 0); + assert_eq!(again.pages[0].skipped.as_deref(), Some("the page already contains text")); + } + + /// Two inputs of one name (from different folders, or differing only in case) would + /// write the same output: both are found, each once. + #[test] + fn duplicate_output_names_are_found_case_insensitively() { + assert!(duplicate_output_names(["a.pdf", "b.pdf"]).is_empty()); + assert_eq!(duplicate_output_names(["a.pdf", "b.pdf", "A.PDF", "b.pdf", "a.pdf"]), ["A.PDF", "b.pdf"]); + } + + /// The batch note's data: counts words at or under the threshold per page (1-based, sorted; + /// skipped pages never contribute), and says nothing when there is nothing to review. + #[test] + fn low_confidence_pages_counts_the_batch_rule() { + let word = |c: f32| PlacedWord { + text: "w".into(), + origin: [0.0; 2], + across: [1.0, 0.0], + up: [0.0, 1.0], + confidence: Some(c), + source: "ocrs".into(), + }; + let page = |n: usize, confs: &[f32], skipped: Option<&str>| OcrPage { + already_on_page: 0, + page: n, + words: confs.iter().map(|c| word(*c)).collect(), + skipped: skipped.map(str::to_string), + notes: Vec::new(), + }; + let pages = vec![ + page(0, &[95.0, 40.0], None), + page(1, &[60.0, 60.5, 10.0], None), + page(2, &[59.0], Some("skipped pages never count")), + page(3, &[0.0], None), + ]; + let (count, pages_on) = low_confidence_pages(&pages, 60.0).unwrap(); + assert_eq!((count, pages_on), (4, vec![1, 2, 4]), "60 counts, 60.5 does not, the skipped page is left out"); + let all_high = vec![page(0, &[95.0, 90.5], None)]; + assert_eq!(low_confidence_pages(&all_high, 60.0), None, "no low-confidence words, no note"); + } + + /// The editable writer: refuses an empty payload, writes the words as visible text at + /// their boxes on pages of the source's displayed size, adds no scan image, and never + /// touches the open document. + #[test] + fn editable_writer_refuses_empty_and_writes_visible_words() { + let mut s = Session::new(); + let id = s.open("src.pdf", None, Arc::new(fixture(2)), None).unwrap(); + assert!( + s.editable_ocr_bytes(id, &[OcrPage { already_on_page: 0, page: 0, words: vec![], skipped: None, notes: Vec::new() }]).is_err(), + "zero words are refused" + ); + let info = s.get(id).unwrap().info.pages[0].clone(); + // A word in the source page's user space, high on the page (the fixture's pages are + // 200 x 300 pt, so the box stays inside them). + let w = PlacedWord { + text: "Editable".into(), + origin: [20.0, 200.0], + across: [120.0, 0.0], + up: [0.0, 12.0], + confidence: Some(88.0), + source: "ocrs".into(), + }; + let found = vec![ + OcrPage { already_on_page: 0, page: 0, words: vec![w], skipped: None, notes: Vec::new() }, + OcrPage { already_on_page: 0, page: 1, words: vec![], skipped: None, notes: Vec::new() }, + ]; + let before = s.get(id).unwrap().bytes.clone(); + let bytes = s.editable_ocr_bytes(id, &found).unwrap(); + // The open document was never touched. + assert!(Arc::ptr_eq(&s.get(id).unwrap().bytes, &before), "the source bytes are the same allocation"); + let mut out = Session::new(); + let oid = out.open("ocr.pdf", None, bytes, None).unwrap(); + let otext = crate::tests::page_texts(&out, oid)[0].clone().to_lowercase(); + assert!(otext.contains("editable"), "{otext}"); + let odoc = out.get(oid).unwrap(); + assert_eq!(odoc.info.pages.len(), 2, "the blank source page stays a page"); + assert!(crate::tests::page_texts(&out, oid)[1].trim().is_empty(), "the page with no words stays blank"); + assert!(odoc.page_images(0).is_empty(), "no scan image is carried over"); + let (w, h) = (odoc.info.pages[0].width, odoc.info.pages[0].height); + assert_eq!((w, h), (info.width, info.height), "pages keep the source's displayed size"); + } + + /// The editable writer maps through a rotated source page: a word drawn in user space + /// reads upright on the new page, and the new page has the source's displayed size. + #[test] + fn editable_writer_maps_rotated_pages() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + let text = s.create_from_text("t", "rotated source").unwrap(); + let id = s.open("r.pdf", None, text, None).unwrap(); + s.apply(id, Edit::RotatePages { pages: vec![0], degrees: 90 }).unwrap(); + let info = s.get(id).unwrap().info.pages[0].clone(); + assert_eq!(info.rotation, 90); + // The word sits at view (20, 48)..(120, 60) on the displayed page; its box comes in + // user space, exactly as recognition left it. + let (o, br, tl) = (info.view_to_user(20.0, 60.0), info.view_to_user(120.0, 60.0), info.view_to_user(20.0, 48.0)); + let w = PlacedWord { + text: "Word".into(), + origin: [f64::from(o[0]), f64::from(o[1])], + across: [f64::from(br[0] - o[0]), f64::from(br[1] - o[1])], + up: [f64::from(tl[0] - o[0]), f64::from(tl[1] - o[1])], + confidence: None, + source: "ocrs".into(), + }; + let bytes = s.editable_ocr_bytes(id, &[OcrPage { already_on_page: 0, page: 0, words: vec![w], skipped: None, notes: Vec::new() }]).unwrap(); + let mut out = Session::new(); + let oid = out.open("ocr.pdf", None, bytes, None).unwrap(); + let otext = crate::tests::page_texts(&out, oid)[0].clone(); + assert!(otext.contains("Word"), "{otext}"); + // The new page is the displayed size (portrait, not the rotated-away landscape). + let oinfo = out.get(oid).unwrap().info.pages[0].clone(); + assert_eq!((oinfo.width, oinfo.height), (info.width, info.height)); + } + + /// The searchable layer round-trips the recognized words through the engine's own + /// extraction, characters beyond WinAnsi included, without needing any display fonts. + #[test] + fn searchable_layer_round_trips_non_win_ansi_words() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + let id = s.open("src.pdf", None, Arc::new(fixture(1)), None).unwrap(); + let word = PlacedWord { + text: "日本語".into(), + origin: [20.0, 250.0], + across: [80.0, 0.0], + up: [0.0, 10.0], + confidence: None, + source: "ocrs".into(), + }; + s.apply(id, Edit::AddOcrText { page: 0, words: vec![word] }).unwrap(); + let text = crate::tests::page_texts(&s, id)[0].clone(); + assert!(text.contains("日本語"), "{text}"); + } + + /// The fontless searchable layer retains its original pixels, handles multiple code + /// fonts and supplementary Unicode, and does not replace existing resource names. + #[test] + fn searchable_unicode_preserves_pixels_save_undo_and_resources() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + let id = s.open("src.pdf", None, Arc::new(fixture(1)), None).unwrap(); + let before = crate::export::Exporter::new(s.get(id).unwrap()).png(0, 72.0).unwrap(); + let mut lines = vec!["中文 English 2026".to_string()]; + let chars: Vec = (0x4e00..0x4f10).filter_map(char::from_u32).chain(['𠮷']).collect(); + lines.extend(chars.chunks(90).map(|c| c.iter().collect())); + let text = lines.join("\n"); + let words: Vec<_> = lines + .iter() + .enumerate() + .map(|(i, line)| PlacedWord { + text: line.clone(), + origin: [20.0, 80.0 - 10.0 * i as f64], + across: [160.0, 0.0], + up: [0.0, 5.0], + confidence: None, + source: "test".into(), + }) + .collect(); + s.apply(id, Edit::AddOcrText { page: 0, words: words.clone() }).unwrap(); + assert!(crate::tests::page_texts(&s, id)[0].contains(&text)); + assert_eq!(crate::export::Exporter::new(s.get(id).unwrap()).png(0, 72.0).unwrap(), before); + let saved = s.save_bytes(id).unwrap(); + let reopened = s.open("searchable.pdf", None, saved, None).unwrap(); + assert!(crate::tests::page_texts(&s, reopened)[0].contains(&text)); + assert_eq!(crate::export::Exporter::new(s.get(reopened).unwrap()).png(0, 72.0).unwrap(), before); + s.undo(id).unwrap(); + assert!(!crate::tests::page_texts(&s, id)[0].contains(&text)); + s.redo(id).unwrap(); + let mut second = words[0].clone(); + second.text = "另一行".into(); + second.origin[1] = 100.0; + s.apply(id, Edit::AddOcrText { page: 0, words: vec![second] }).unwrap(); + let extracted = crate::tests::page_texts(&s, id)[0].clone(); + assert!(extracted.contains(&text) && extracted.contains("另一行"), "{extracted}"); + } + + #[test] + fn searchable_unicode_rejects_excessive_or_non_finite_input_atomically() { + let mut s = Session::new(); + let id = s.open("src.pdf", None, Arc::new(fixture(1)), None).unwrap(); + let word = + PlacedWord { text: "日".into(), origin: [20.0, 80.0], across: [30.0, 0.0], up: [0.0, 5.0], confidence: None, source: "test".into() }; + let before = crate::tests::page_texts(&s, id); + let mut non_finite = word.clone(); + non_finite.origin[0] = f64::NAN; + let mut overflow = word.clone(); + overflow.origin[0] = f64::MAX; + overflow.across[0] = f64::MAX; + let mut too_long = word.clone(); + too_long.text = "a".repeat(4 * 1024 * 1024 + 1); + for words in [vec![non_finite], vec![overflow], vec![too_long], vec![word; 65537]] { + assert!(s.apply(id, Edit::AddOcrText { page: 0, words }).is_err()); + assert_eq!(crate::tests::page_texts(&s, id), before); + } + } + + #[test] + fn editable_export_refuses_invalid_or_excessive_aggregate_input_without_changing_source() { + let mut s = Session::new(); + let id = s.open("src.pdf", None, Arc::new(fixture(2)), None).unwrap(); + let before = s.save_bytes(id).unwrap(); + let word = + PlacedWord { text: "word".into(), origin: [20.0, 80.0], across: [30.0, 0.0], up: [0.0, 5.0], confidence: None, source: "test".into() }; + let page = |page, words| OcrPage { already_on_page: 0, page, words, skipped: None, notes: Vec::new() }; + let mut invalid = word.clone(); + invalid.origin[0] = f64::INFINITY; + let mut long = word.clone(); + long.text = "a".repeat(2 * 1024 * 1024 + 1); + for found in [ + vec![page(0, vec![invalid])], + vec![page(0, vec![long.clone()]), page(1, vec![long])], + vec![page(0, vec![word.clone(); 32768]), page(1, vec![word; 32769])], + ] { + assert!(s.editable_ocr_bytes(id, &found).is_err()); + assert_eq!(s.save_bytes(id).unwrap(), before); + } + } + + /// The editable output carries non-WinAnsi words as visible text that extraction reads + /// back, in a face beyond Helvetica when the build has one. + #[test] + fn editable_output_round_trips_non_win_ansi_words() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + let id = s.open("src.pdf", None, Arc::new(fixture(1)), None).unwrap(); + let w = PlacedWord { + text: "Привет".into(), + origin: [20.0, 250.0], + across: [80.0, 0.0], + up: [0.0, 10.0], + confidence: None, + source: "ocrs".into(), + }; + let bytes = s.editable_ocr_bytes(id, &[OcrPage { already_on_page: 0, page: 0, words: vec![w], skipped: None, notes: Vec::new() }]).unwrap(); + let mut out = Session::new(); + let oid = out.open("ocr.pdf", None, bytes, None).unwrap(); + let text = crate::tests::page_texts(&out, oid)[0].clone(); + if pdfcraft_fonts::CRAFT_FONTS.is_empty() { + assert!(text.contains("??????"), "{text}"); + } else { + assert!(text.contains("Привет"), "{text}"); + } + } + + /// The editable output's VISIBLE layer — a Type3 run for the Cyrillic word, Helvetica for + /// the Latin one — is redacted end to end: a mark over the Cyrillic word removes exactly + /// it (the renderer's extractor no longer reads it, before or after a save and reopen), and + /// the Latin word elsewhere on the page stays. + #[test] + fn the_editable_type3_layer_is_redacted_end_to_end() { + if pdfcraft_fonts::CRAFT_FONTS.is_empty() { + eprintln!("skipping: built without craft-fonts (set CRAFT_FONTS_DIR to run it)"); + return; + } + let mut s = Session::new().with_clock(|| 1_700_000_000); + let id = s.open("src.pdf", None, Arc::new(fixture(1)), None).unwrap(); + let word = |text: &str, y: f64, w: f64| PlacedWord { + text: text.into(), + origin: [20.0, y], + across: [w, 0.0], + up: [0.0, 10.0], + confidence: None, + source: "ocrs".into(), + }; + let words = vec![word("Привет", 250.0, 80.0), word("Hello", 100.0, 50.0)]; + let bytes = s.editable_ocr_bytes(id, &[OcrPage { already_on_page: 0, page: 0, words, skipped: None, notes: Vec::new() }]).unwrap(); + let mut out = Session::new().with_clock(|| 1_700_000_000); + let oid = out.open("ocr.pdf", None, bytes, None).unwrap(); + let before = crate::tests::page_texts(&out, oid)[0].clone(); + assert!(before.contains("Привет") && before.contains("Hello"), "{before}"); + let shape = crate::Shape::Redact { + quads: vec![pdfcraft_annot::rect_quad([10.0, 240.0, 110.0, 270.0])], + overlay: String::new(), + look: Default::default(), + }; + let mark = Edit::AddAnnotation(crate::NewAnnotation { + page: 0, + style: crate::Style::default_for(&shape), + shape, + contents: String::new(), + author: "Ada".into(), + }); + out.apply(oid, mark).unwrap(); + out.apply(oid, Edit::ApplyRedactions { pages: None }).unwrap(); + let after = crate::tests::page_texts(&out, oid)[0].clone(); + assert!(!after.contains("Привет") && !after.contains("Пр"), "the Type3 word is gone: {after}"); + assert!(after.contains("Hello"), "the word outside the mark stays: {after}"); + // Reopened from the saved bytes, an independent read finds the same. + let saved = out.save_bytes(oid).unwrap(); + let mut again = Session::new(); + let aid = again.open("r.pdf", None, saved, None).unwrap(); + let reread = crate::tests::page_texts(&again, aid)[0].clone(); + assert!(!reread.contains("Привет") && reread.contains("Hello"), "{reread}"); + } + + /// A shared document plan serves a multi-page editable export: each page's words must be + /// written with their OWN encodings (the plan's offset for that page), so page 2 never + /// shows page 1's text. + #[test] + fn editable_output_keeps_each_pages_own_words() { + if pdfcraft_fonts::CRAFT_FONTS.is_empty() { + eprintln!("skipping: built without craft-fonts (set CRAFT_FONTS_DIR to run it)"); + return; + } + let mut s = Session::new().with_clock(|| 1_700_000_000); + let id = s.open("src.pdf", None, Arc::new(fixture(2)), None).unwrap(); + let word = |text: &str| PlacedWord { + text: text.into(), + origin: [20.0, 250.0], + across: [80.0, 0.0], + up: [0.0, 10.0], + confidence: None, + source: "ocrs".into(), + }; + // Different scripts, so each page's words live in different planned fonts (and page 2 + // would inherit page 1's codes under page-local indexing into the shared plan). + let found = vec![ + OcrPage { already_on_page: 0, page: 0, words: vec![word("日本語")], skipped: None, notes: Vec::new() }, + OcrPage { already_on_page: 0, page: 1, words: vec![word("Привет")], skipped: None, notes: Vec::new() }, + ]; + let bytes = s.editable_ocr_bytes(id, &found).unwrap(); + let mut out = Session::new(); + let oid = out.open("ocr.pdf", None, bytes, None).unwrap(); + let texts = crate::tests::page_texts(&out, oid); + assert!(texts[0].contains("日本語") && !texts[0].contains("Привет"), "{texts:?}"); + assert!(texts[1].contains("Привет") && !texts[1].contains("日本語"), "{texts:?}"); + } + + /// Redacting a recognized page takes the Unicode searchable layer away for good: the + /// apply's own verification passes over the Type3 font and an independent extractor finds + /// nothing, including in builds with no display fonts. + #[test] + fn redaction_removes_the_unicode_searchable_layer() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + let id = s.open("src.pdf", None, Arc::new(fixture(1)), None).unwrap(); + let word = PlacedWord { + text: "日本語のテキスト".into(), + origin: [20.0, 250.0], + across: [100.0, 0.0], + up: [0.0, 10.0], + confidence: None, + source: "ocrs".into(), + }; + s.apply(id, Edit::AddOcrText { page: 0, words: vec![word] }).unwrap(); + assert!(crate::tests::page_texts(&s, id)[0].contains("日本語")); + let shape = pdfcraft_annot::Shape::Redact { + quads: vec![pdfcraft_annot::rect_quad([18.0, 248.0, 122.0, 262.0])], + overlay: String::new(), + look: Default::default(), + }; + let mark = Edit::AddAnnotation(pdfcraft_annot::NewAnnotation { + page: 0, + style: pdfcraft_annot::Style::default_for(&shape), + shape, + contents: String::new(), + author: "Ada".into(), + }); + s.apply(id, mark).unwrap(); + s.apply(id, Edit::ApplyRedactions { pages: None }).unwrap(); + assert!(!crate::tests::page_texts(&s, id)[0].contains("日本語"), "the layer is gone, not just hidden"); + } + + /// The job's render scale is what the run actually renders at: the chosen dpi, or the + /// page image's own resolution when that is lower. + #[test] + fn render_scale_matches_the_job_resolution() { + let mut s = Session::new(); + let text = s.create_from_text("t", "scale probe").unwrap(); + let id = s.open("t.pdf", None, text, None).unwrap(); + let job = s.ocr_job(id, &[0], OcrSettings { dpi: 144.0, ..Default::default() }).unwrap(); + assert!((job.render_scale(0) - 2.0).abs() < 1e-4, "144 dpi = 2 px/pt, got {}", job.render_scale(0)); + let job = s.ocr_job(id, &[0], OcrSettings { dpi: 600.0, ..Default::default() }).unwrap(); + assert!((job.render_scale(0) - 600.0 / 72.0).abs() < 1e-4, "a text page has no image, so the wanted dpi wins"); + let page = job.pages[0]; + assert_eq!(page, 0); + } + + /// #1848: an A4 scan at 300 dpi whose page box is its pixel size (2480 x 3508 pt, one pixel + /// per point) gets a smaller bitmap for each lower "Downsample to" choice, and a low-resolution + /// letter page is still never shrunk below one pixel per point. + #[test] + fn downsample_shrinks_a_page_box_given_in_pixels() { + let mut s = Session::new(); + let page = |w: u32, h: u32| export::encode_png(w, h, &vec![255; (w * h * 4) as usize]).unwrap(); + let bytes = s.create_from_images(&[("a4.png".into(), page(2480, 3508)), ("letter.png".into(), page(612, 792))]).unwrap(); + let id = s.open("scan.pdf", None, bytes, None).unwrap(); + let mut r = PageRenderer::new(s.get(id).unwrap().bytes.clone(), RenderConfig::default()); + for (dpi, a4) in [(72.0, [595, 842]), (150.0, [1240, 1754]), (300.0, [2480, 3508]), (600.0, [2480, 3508])] { + let job = s.ocr_job(id, &[], OcrSettings { dpi, ..Default::default() }).unwrap(); + for (page, want) in [(0, a4), (1, [612, 792])] { + let scale = job.render_scale(page); + let shot = r.render(RenderRequest { page, scale, ..Default::default() }); + assert!(shot.error.is_none(), "{:?}", shot.error); + let near = |got: u32, want: u32| got.abs_diff(want) <= 1; + assert!(near(shot.width, want[0]) && near(shot.height, want[1]), "{dpi} dpi, page {page}: {}x{}", shot.width, shot.height); + } + } + } } diff --git a/crates/engine/src/ocr/dictionary.rs b/crates/engine/src/ocr/dictionary.rs new file mode 100644 index 0000000000..cbefa9f7b5 --- /dev/null +++ b/crates/engine/src/ocr/dictionary.rs @@ -0,0 +1,80 @@ +//! The UI dictionary format: one case-insensitive word per UTF-8 line. +use std::collections::BTreeSet; + +pub const MAX_BYTES: usize = 4 * 1024 * 1024; +const MAX_WORDS: usize = 65536; + +fn word(value: &str) -> Result { + let value = value.trim(); + if value.is_empty() || value.len() > 1024 || value.chars().any(char::is_control) { + return Err("dictionary words must be nonempty single lines of at most 1024 bytes".into()); + } + let normalized = value.to_lowercase(); + if normalized.len() > 1024 { + return Err("normalized dictionary words must be at most 1024 bytes".into()); + } + Ok(normalized) +} + +pub fn words(text: &str) -> Result, String> { + if text.len() > MAX_BYTES { + return Err("OCR dictionary exceeds 4 MiB".into()); + } + let mut words = BTreeSet::new(); + for line in text.lines().filter(|line| !line.trim().is_empty()) { + words.insert(word(line)?); + if words.len() > MAX_WORDS { + return Err("OCR dictionary exceeds 65536 words".into()); + } + } + Ok(words) +} + +/// Normalize, update and bound the complete dictionary before the caller writes it. +pub fn update(text: &str, add: &[&str], remove: &[&str]) -> Result { + if add.len() > MAX_WORDS || remove.len() > MAX_WORDS { + return Err("OCR dictionary exceeds 65536 words".into()); + } + let mut entries = words(text)?; + for value in remove { + entries.remove(&word(value)?); + } + for value in add { + entries.insert(word(value)?); + if entries.len() > MAX_WORDS { + return Err("OCR dictionary exceeds 65536 words".into()); + } + } + let mut out = String::new(); + for entry in entries { + if out.len().checked_add(entry.len()).and_then(|n| n.checked_add(1)).is_none_or(|n| n > MAX_BYTES) { + return Err("OCR dictionary exceeds 4 MiB".into()); + } + out.push_str(&entry); + out.push('\n'); + } + Ok(out) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn dictionary_updates_round_trip_and_refuse_expanding_normalization() { + let text = update("Quartz\n日本語\n", &["QUARTZ", "extra"], &["日本語"]).unwrap(); + assert_eq!(text, "extra\nquartz\n"); + assert_eq!(words(&text).unwrap(), BTreeSet::from(["extra".into(), "quartz".into()])); + assert!(update("original\n", &[&"İ".repeat(512)], &[]).unwrap_err().contains("normalized")); + assert!(update("original\n", &["two\nlines"], &[]).is_err()); + } + + #[test] + fn dictionary_limits_cover_existing_and_added_entries() { + let full: String = (0..65536).map(|i| format!("word{i}\n")).collect(); + assert_eq!(words(&full).unwrap().len(), 65536); + assert!(update(&full, &["extra"], &[]).is_err()); + assert_eq!(update(&full, &["extra"], &["word0"]).unwrap().lines().count(), 65536); + assert!(words(&"a".repeat(MAX_BYTES + 1)).is_err()); + } +} diff --git a/crates/engine/src/ocr/layer.rs b/crates/engine/src/ocr/layer.rs new file mode 100644 index 0000000000..ec2bac5fa8 --- /dev/null +++ b/crates/engine/src/ocr/layer.rs @@ -0,0 +1,145 @@ +//! Unicode search text has no visible glyphs. A generated Type 3 font supplies advance widths +//! and a ToUnicode map, with empty character programs; no display font is needed or embedded. +//! Adapted from the searchable-layer implementation in #433 (32e09fbe). + +use std::collections::BTreeMap; + +use pdfcraft_cos::{Dict, Document, Object, Stream}; +use pdfcraft_edit::EditError; +use pdfcraft_ocr::PlacedWord; + +/// Refuse oversized text or invalid geometry before cloning words or allocating fonts. +/// These limits apply to the complete input, including all editable export pages. +pub(crate) fn validate_words<'a>(words: impl IntoIterator) -> Result { + let mut count = 0usize; + let mut length = 0usize; + for w in words { + count += 1; + length = length.checked_add(w.text.len()).ok_or_else(|| EditError::Invalid("OCR text layer is too large".into()))?; + if count > 65536 || length > 4 * 1024 * 1024 { + return Err(EditError::Invalid("OCR text layer is too large".into())); + } + if !w.origin.iter().chain(&w.across).chain(&w.up).all(|v| v.is_finite()) + || !w.across[0].hypot(w.across[1]).is_finite() + || !w.up[0].hypot(w.up[1]).is_finite() + || (0..2).any(|i| !(w.origin[i] + 0.2 * w.up[i]).is_finite() || !(w.origin[i] + w.across[i] + 0.2 * w.up[i]).is_finite()) + { + return Err(EditError::Invalid("OCR text coordinates must be finite".into())); + } + } + Ok(count) +} + +pub(crate) fn add_text(doc: &mut Document, page: usize, words: &[PlacedWord]) -> Result<(), EditError> { + validate_words(words)?; + // Keep the existing Latin layout, including Helvetica's proportional advance widths. + if words.iter().all(|w| { + w.text + .chars() + .all(|c| matches!(c, '\u{20}'..='\u{7e}' | '\u{a0}'..='\u{ff}' | '€' | '‚' | '„' | '…' | '‘' | '’' | '“' | '”' | '•' | '–' | '—' | '™')) + }) { + pdfcraft_edit::stamp(doc, page, "OCR", pdfcraft_ocr::text_layer(words, &pdfcraft_fonts::TextPlan::default()))?; + return Ok(()); + } + let p = pdfcraft_model::pages(doc).get(page).cloned().ok_or_else(|| EditError::Invalid("OCR page does not exist".into()))?; + let mut res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default(); + let mut fonts = res.get(b"Font").map(|f| doc.resolve(f)).and_then(|f| f.as_dict().cloned()).unwrap_or_default(); + let chars: Vec = words.iter().flat_map(|w| w.text.chars()).collect::>().into_iter().collect(); + // Match the extractor's em box (-200..800). Baselines below are offset by 0.2 em, + // so search/selection rectangles span the detected image box. + let blank = doc.add(Object::Stream(Stream::from_raw(Dict::new(), b"1000 0 0 -200 1000 800 d1\n".to_vec()))); + let mut codes = BTreeMap::new(); + let mut names = Vec::new(); + for (index, chunk) in chars.chunks(255).enumerate() { + let mut name = format!("PCOcr{index}"); + while fonts.contains(name.as_bytes()) { + name.push('x'); + } + let mut differences = vec![Object::Int(1)]; + let mut procs = Dict::new(); + let mut cmap = String::from( + "/CIDInit /ProcSet findresource begin\n12 dict begin\nbegincmap\n/CIDSystemInfo << /Registry (PdfCraft) /Ordering (Unicode) /Supplement 0 >> def\n/CMapName /PdfCraftOCR def\n/CMapType 2 def\n1 begincodespacerange\n<01> \nendcodespacerange\n", + ); + for (batch, part) in chunk.chunks(100).enumerate() { + cmap.push_str(&format!("{} beginbfchar\n", part.len())); + for (i, ch) in part.iter().enumerate() { + let code = (batch * 100 + i + 1) as u8; + let glyph = format!("g{code:02X}"); + differences.push(Object::name(&glyph)); + procs.set(glyph.into_bytes(), Object::Ref(blank)); + let mut units = [0u16; 2]; + let hex: String = ch.encode_utf16(&mut units).iter().map(|u| format!("{u:04X}")).collect(); + cmap.push_str(&format!("<{code:02X}> <{hex}>\n")); + codes.insert(*ch, (index, code)); + } + cmap.push_str("endbfchar\n"); + } + cmap.push_str("endcmap\nCMapName currentdict /CMap defineresource pop\nend\nend\n"); + let map = doc.add(Object::Stream(Stream::flate(Dict::new(), cmap.as_bytes()))); + let mut encoding = Dict::new(); + encoding.set(b"Type".to_vec(), Object::name("Encoding")); + encoding.set(b"Differences".to_vec(), Object::Array(differences)); + let mut font = Dict::new(); + font.set(b"Type".to_vec(), Object::name("Font")); + font.set(b"Subtype".to_vec(), Object::name("Type3")); + font.set(b"FontBBox".to_vec(), Object::Array(vec![0.into(), (-200).into(), 1000.into(), 800.into()])); + font.set(b"FontMatrix".to_vec(), Object::Array(vec![Object::Real(0.001), 0.into(), 0.into(), Object::Real(0.001), 0.into(), 0.into()])); + font.set(b"FirstChar".to_vec(), Object::Int(1)); + font.set(b"LastChar".to_vec(), Object::Int(chunk.len() as i64)); + font.set(b"Widths".to_vec(), Object::Array(vec![Object::Int(1000); chunk.len()])); + font.set(b"Encoding".to_vec(), Object::Dict(encoding)); + font.set(b"CharProcs".to_vec(), Object::Dict(procs)); + font.set(b"Resources".to_vec(), Object::Dict(Dict::new())); + font.set(b"ToUnicode".to_vec(), Object::Ref(map)); + fonts.set(name.as_bytes().to_vec(), Object::Ref(doc.add(Object::Dict(font)))); + names.push(name); + } + res.set(b"Font".to_vec(), Object::Dict(fonts)); + doc.update_dict(p.obj, |d| d.set(b"Resources".to_vec(), Object::Dict(res)))?; + // `3 Tr` and the font are graphics state, which `ET` doesn't reset: like + // `pdfcraft_ocr::text_layer` (#3029), the text is drawn inside `q … Q` so the invisible + // rendering mode cannot leak into later content streams on the page. + let mut content = String::from("/OCR BMC\nq\nBT\n3 Tr\n"); + for w in words { + let count = w.text.chars().count(); + if count == 0 || w.up[0].hypot(w.up[1]) <= 0.0 || w.across[0].hypot(w.across[1]) <= 0.0 { + continue; + } + let mut start = 0; + let mut run = Vec::new(); + let mut font_index = None; + let emit = |content: &mut String, font: usize, start: usize, run: &[u8]| { + let n = count as f64; + let at = start as f64 / n; + let hex: String = run.iter().map(|b| format!("{b:02X}")).collect(); + content.push_str(&format!( + "/{} 1 Tf\n{} {} {} {} {} {} Tm <{hex}> Tj\n", + names[font], + w.across[0] / n, + w.across[1] / n, + w.up[0], + w.up[1], + w.origin[0] + at * w.across[0] + 0.2 * w.up[0], + w.origin[1] + at * w.across[1] + 0.2 * w.up[1] + )); + }; + for (i, ch) in w.text.chars().enumerate() { + let Some(&(font, code)) = codes.get(&ch) else { continue }; + if let Some(previous) = font_index + && previous != font + { + emit(&mut content, previous, start, &run); + run.clear(); + start = i; + } + font_index = Some(font); + run.push(code); + } + if let Some(font) = font_index { + emit(&mut content, font, start, &run); + } + } + content.push_str("ET\nQ\nEMC\n"); + pdfcraft_edit::stamp(doc, page, "OCR", content.into_bytes())?; + Ok(()) +} diff --git a/crates/engine/src/scan_fields.rs b/crates/engine/src/scan_fields.rs index a1574e9463..05906d75b2 100644 --- a/crates/engine/src/scan_fields.rs +++ b/crates/engine/src/scan_fields.rs @@ -186,10 +186,14 @@ impl Session { if !ocr::available() { return Vec::new(); } - let (Ok(engine), Some(job)) = (ocr::engine(), self.ocr_job(id, &[page], ocr::OcrSettings::default())) else { + let settings = ocr::OcrSettings::default(); + let Some(job) = self.ocr_job(id, &[page], settings.clone()) else { return Vec::new(); }; - job.run(&engine, |_, _| true) + let Ok(recognizers) = ocr::recognizers(&settings) else { + return Vec::new(); + }; + job.run(&recognizers, |_, _| true) .into_iter() .flat_map(|p| p.words) .filter_map(|w| { diff --git a/crates/engine/src/session/files.rs b/crates/engine/src/session/files.rs index 06e6acbfd1..44bed85cde 100644 --- a/crates/engine/src/session/files.rs +++ b/crates/engine/src/session/files.rs @@ -397,7 +397,7 @@ impl Session { /// A new file (Combine, Extract, Split, Create) as bytes, checked like a save before anyone /// writes it (#1698, #2166): it reopens, and shows the pages it was built from. - fn write_new(&self, doc: &pdfcraft_cos::Document) -> Result>, EditError> { + pub(crate) fn write_new(&self, doc: &pdfcraft_cos::Document) -> Result>, EditError> { let bytes = self.write_unchecked(doc)?; let expected = pdfcraft_organize::page_count(doc)?; pdfcraft_cos::Document::open(bytes.clone()).map_err(|e| EditError::Reopen(e.to_string()))?; diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index 86fa6495ad..688dcec049 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -628,7 +628,7 @@ fn a_3d_model_needs_the_comments_permission_and_changes_the_comments() { assert_eq!(crate::history::scope_of(&edit), crate::history::Scope::Comments); } -fn page_texts(s: &Session, id: DocId) -> Vec { +pub(crate) fn page_texts(s: &Session, id: DocId) -> Vec { let doc = s.get(id).unwrap(); let config = pdfcraft_render::RenderConfig { password: doc.password.as_deref().map(Arc::from), ..Default::default() }; let mut r = pdfcraft_render::PageRenderer::new(doc.bytes.clone(), config); @@ -7322,12 +7322,13 @@ fn datasets_streams(bytes: &Arc>) -> usize { /// and noted. #[test] fn hostile_xfa_open_scripts_are_merged_capped_kept_from_printing_and_noted() { - // On open: a calculate setting its own value 50 000 times, an initialize script showing a - // thousand messages and asking to print, open a link and save. + // On open: a calculate setting its own value 20 000 times (50 000 sat right at the 1 s + // unasked-script timeout and abandoned whenever the test machine was loaded), an + // initialize script showing a thousand messages and asking to print, open a link and save. let tpl = pdfcraft_xfa::fixtures::scripted_template() .replace( r#""#, - r#""#, + r#""#, ) .replace( r#"if (qty.rawValue === null) qty.rawValue = 2;"#, @@ -7337,9 +7338,11 @@ fn hostile_xfa_open_scripts_are_merged_capped_kept_from_printing_and_noted() { let mut s = Session::new().with_clock(|| 1_700_000_000); let started = std::time::Instant::now(); let id = s.open("hostile.pdf", None, bytes, None).expect("opens"); - assert!(started.elapsed() < std::time::Duration::from_secs(5), "{:?}", started.elapsed()); + // 30 s and not 5 s: the run is wall-clock timed and a loaded test machine (all suites in + // parallel) can take several seconds for the 20 000 iterations; a real hang still fails. + assert!(started.elapsed() < std::time::Duration::from_secs(30), "{:?}", started.elapsed()); let doc = s.get(id).unwrap(); - assert_eq!(doc.form.iter().find(|f| f.name == "total").unwrap().value, vec!["49999".to_string()]); + assert_eq!(doc.form.iter().find(|f| f.name == "total").unwrap().value, vec!["19999".to_string()]); // One datasets stream for the scripts' revision, not one per value set. assert!(datasets_streams(&doc.bytes) <= 3, "{} datasets streams", datasets_streams(&doc.bytes)); // What opening changed is noted with the form's other XFA notes. diff --git a/crates/engine/tests/it/ocr_edit.rs b/crates/engine/tests/it/ocr_edit.rs index 53ca309326..2d358e327e 100644 --- a/crates/engine/tests/it/ocr_edit.rs +++ b/crates/engine/tests/it/ocr_edit.rs @@ -12,7 +12,14 @@ const WORD: [u32; 4] = [40, 40, 100, 60]; /// A 200 × 100 pt "scan" on tinted paper, with OCR text over its word. fn scanned() -> (Session, pdfcraft_engine::DocId) { - scanned_with(vec![PlacedWord { text: "John".into(), origin: [40.0, 40.0], across: [60.0, 0.0], up: [0.0, 20.0] }]) + scanned_with(vec![PlacedWord { + text: "John".into(), + origin: [40.0, 40.0], + across: [60.0, 0.0], + up: [0.0, 20.0], + confidence: None, + source: "ocrs".into(), + }]) } fn scanned_with(words: Vec) -> (Session, pdfcraft_engine::DocId) { @@ -86,11 +93,18 @@ fn an_explicit_colour_is_kept_for_the_new_text() { /// descender line when it has descenders, up to the cap height. fn scanned_word(text: &str, x0: f64, x1: f64, baseline: f64, size: f64) -> PlacedWord { let low = if text.chars().any(|c| "gjpqy".contains(c)) { 0.21 * size } else { 0.0 }; - PlacedWord { text: text.into(), origin: [x0, baseline - low], across: [x1 - x0, 0.0], up: [0.0, 0.718 * size + low] } + PlacedWord { + text: text.into(), + origin: [x0, baseline - low], + across: [x1 - x0, 0.0], + up: [0.0, 0.718 * size + low], + confidence: None, + source: "ocrs".into(), + } } fn word(text: &str, x0: f64, x1: f64, baseline: f64) -> PlacedWord { - PlacedWord { text: text.into(), origin: [x0, baseline], across: [x1 - x0, 0.0], up: [0.0, 10.0] } + PlacedWord { text: text.into(), origin: [x0, baseline], across: [x1 - x0, 0.0], up: [0.0, 10.0], confidence: None, source: "ocrs".into() } } /// OCR words keep the scan's layout: a phrase is one paragraph, a value in the next table cell and diff --git a/crates/fonts/README.md b/crates/fonts/README.md index b20b930a91..096f1a263e 100644 --- a/crates/fonts/README.md +++ b/crates/fonts/README.md @@ -59,7 +59,12 @@ set). Without that variable `CRAFT_FONTS` is empty and everything below copes: and shaped Thai Add Text content. Harfrust positions stacked vowel/tone marks and decomposes Sara Am; cluster ToUnicode entries preserve the original text. The shared Type 3 writer embeds outlines, so reopening does not depend on installed fonts. +- `craft_glyph(face, ch)` and `face_has(bytes, ch)`: bounded outlines and glyph coverage for any bundled face. - `SHIPPORI_MINCHO`: Shippori Mincho's bytes, or `None`. +- `TextPlan::for_words`: how the OCR output writes text beyond WinAnsi — per word the first + face (document face first) whose glyphs cover it, planned once per page into Type 3 fonts + (codes, advances, `?` stand-ins) that `pdfcraft-edit` registers and the `pdfcraft-ocr` layers + draw with; characters no face has are counted (`missing`) so the output reports them. Generated Japanese Type 3 fonts record the fallback face's family and style in an indirect `FontDescriptor`. `pdf::Metrics::base_font` uses its `FontName` when the Type 3 dictionary has diff --git a/crates/fonts/src/lib.rs b/crates/fonts/src/lib.rs index 61061f5509..45cb843ab0 100644 --- a/crates/fonts/src/lib.rs +++ b/crates/fonts/src/lib.rs @@ -14,6 +14,7 @@ pub mod embedding; mod encodings; pub mod fallback; pub mod pdf; +pub mod plan; mod script; mod shaping; pub mod source_shaping; @@ -24,8 +25,10 @@ pub use craft::{ document_japanese_font_for_style, document_japanese_fonts_for_style, document_latin_fonts, document_thai_font, ui_arabic_fonts, ui_chinese_fonts, ui_cjk_fonts, ui_devanagari_fonts, ui_hebrew_fonts, ui_japanese_fonts, ui_telugu_fonts, ui_thai_fonts, }; +pub use plan::{MAX_PLAN_GLYPHS, MAX_TYPE3_GLYPHS, PlannedType3, TextPlan}; pub use script::{ - GlyphError, GlyphOutline, INTER_REGULAR, MAX_SIGNATURE_CHARS, ScriptOutline, japanese_glyph, japanese_glyph_from, script_missing, script_outline, + GlyphError, GlyphOutline, INTER_REGULAR, MAX_SIGNATURE_CHARS, ScriptOutline, craft_glyph, face_has, japanese_glyph, japanese_glyph_from, + script_missing, script_outline, }; pub use shaping::{ ShapedCluster, ShapedScript, arabic_glyph, arabic_has, hebrew_has, shape_arabic, shape_hebrew, shape_thai, shaped_glyph, thai_has, @@ -273,6 +276,16 @@ pub fn set_undrawn_mark(d: &mut pdfcraft_cos::Dict, c: Option) { } } +/// The byte used by Unicode text plans, from the shared WinAnsi table (tabs become spaces). +pub fn win_ansi_char(c: char) -> Option { + if c == '\t' { Some(b' ') } else { win_ansi_byte(c) } +} + +/// Whether every character can be represented without substitution in WinAnsi. +pub fn fits_win_ansi(s: &str) -> bool { + s.chars().all(win_ansi_encodable) +} + /// Bytes as a PDF literal string, `(` … `)`, with delimiters escaped. pub fn literal(bytes: &[u8]) -> Vec { let mut out = Vec::with_capacity(bytes.len() + 2); diff --git a/crates/fonts/src/plan.rs b/crates/fonts/src/plan.rs new file mode 100644 index 0000000000..9c22a38fd3 --- /dev/null +++ b/crates/fonts/src/plan.rs @@ -0,0 +1,338 @@ +//! Planning visible Unicode text output for OCR: which characters the standard WinAnsi +//! fonts show as is, which need a Type3 font drawn from the craft-fonts faces, and which have +//! no glyph anywhere (written as `?` — and counted, never silently dropped). +//! +//! The plan is a pure function of the words ([`TextPlan::for_words`]). The content generators +//! (`pdfcraft_ocr::text_layer` / `visible_text_layer`) read the per-character codes and +//! advances from it, and whoever registers the fonts (`pdfcraft_edit::add_type3_fonts`) fills +//! in each face's resource name and object. The build must work without craft-fonts too: there +//! [`CRAFT_FONTS`] is empty, nothing can be planned, and every non-WinAnsi character is +//! reported as a substitution instead. + +use std::collections::HashMap; + +use pdfcraft_cos::ObjRef; + +use crate::{CraftFont, GlyphOutline, craft_glyph, document_japanese_font, face_has, fits_win_ansi, win_ansi_char}; + +/// Unique characters per Type3 font: the same bound the text-editing fallback uses, so one +/// hostile word cannot balloon into thousands of glyph streams. A face with more characters +/// simply gets another font. +pub const MAX_TYPE3_GLYPHS: usize = 240; + +/// Planned glyphs per plan in total (10 full fonts). A page's recognition cannot realistically +/// exceed it; hostile input beyond it degrades to `?` substitutions, which are counted. +pub const MAX_PLAN_GLYPHS: usize = 2400; + +/// One planned Type3 font: the glyphs of one craft-fonts face, one character per code +/// (`code = position + 1`, as in the text-editing fallback). `name` and `object` are filled in +/// when the font is added to a page's resources; until then the content generators fall back to +/// the WinAnsi path for the words that reference it. +pub struct PlannedType3 { + /// The face family the glyphs are drawn from ("Shippori Mincho"). + pub family: &'static str, + /// (character, outline with its advance) per code, in code order. + pub glyphs: Vec<(char, GlyphOutline)>, + /// The resource name the content references (`PCOcr`, `PCOcr1`, …); empty until registered. + pub name: String, + /// The built font object, shared by every page that uses the font; set at registration. + pub object: Option, +} + +impl PlannedType3 { + fn new(family: &'static str) -> Self { + PlannedType3 { family, glyphs: Vec::new(), name: String::new(), object: None } + } + + fn push(&mut self, ch: char, glyph: GlyphOutline) -> u8 { + self.glyphs.push((ch, glyph)); + self.glyphs.len() as u8 + } +} + +/// How the text layers write a plan's characters: each planned character is a +/// `(font index, code)` pair in its font's encoding; a character no face had a glyph for is +/// either the font's `?` code or absent from the run (both counted in [`TextPlan::missing`]). +#[derive(Default)] +pub struct TextPlan { + /// One planned font per face chunk, in first-use order. + pub fonts: Vec, + /// Per input word: `(font, code)` per character; empty for a word the standard WinAnsi + /// fonts write as is (or for which nothing could be planned). + words: Vec>, + /// character → where its glyph lives, across all fonts (codes are font-local). + index: HashMap, + /// Characters with no code in their word's font: they appear as `?` (or, when not even a + /// `?` glyph exists, are left out) and are reported, never silently lost. + missing: usize, +} + +impl TextPlan { + /// Plan the output for `words`, in order: characters the WinAnsi fonts can't show get codes + /// in Type3 fonts built from the craft-fonts faces (the document face preferred, then the + /// rest of the build input in manifest order); a word whose face can't show a character + /// substitutes its `?`. Faces are chosen per word, so a word of one script stays one face; + /// characters already planned for an earlier word keep their code. Built without + /// craft-fonts, the plan reports every non-WinAnsi character as a substitution. + pub fn for_words<'a>(words: impl IntoIterator) -> TextPlan { + let faces = document_faces(); + let mut coverage = Coverage::default(); + let mut plan = TextPlan::default(); + for word in words { + if fits_win_ansi(word) { + // The WinAnsi path substitutes '?' behind the caller's back; count it. + plan.missing += word.chars().filter(|&c| win_ansi_char(c).is_none()).count(); + plan.words.push(Vec::new()); + continue; + } + let Some(face) = choose_face(&faces, word, &mut coverage) else { + // No face for this word's script: the WinAnsi path writes '?' for what it + // can't encode. + plan.missing += word.chars().filter(|&c| win_ansi_char(c).is_none()).count(); + plan.words.push(Vec::new()); + continue; + }; + let mut enc: Vec<(usize, u8)> = Vec::with_capacity(word.len()); + let mut run: Option = None; + for ch in word.chars() { + if let Some(&where_) = plan.index.get(&ch) { + enc.push(where_); + run = Some(where_.0); + continue; + } + if let Ok(glyph) = craft_glyph(face, ch) + && plan.planned_glyphs() < MAX_PLAN_GLYPHS + && let Some((fi, code)) = plan.add_glyph(face, ch, glyph) + { + plan.index.insert(ch, (fi, code)); + enc.push((fi, code)); + run = Some(fi); + continue; + } + // The face has no glyph for the character (or the plan is full): substitute + // the font's '?', in the font the surrounding run is set in. + if let Some((fi, code)) = plan.question_mark(face, run.filter(|&f| plan.fonts[f].family == face.family)) { + enc.push((fi, code)); + run = Some(fi); + } + plan.missing += 1; + } + plan.words.push(enc); + } + plan + } + + /// The encoding of word `i` (parallel to the slice [`Self::for_words`] planned): a list of + /// `(font, code)` per character, or an empty slice for a word the WinAnsi fonts write. + /// Out-of-range indexes degrade to the empty slice rather than panicking — callers pass the + /// same word list they planned with. + pub fn encoding(&self, word: usize) -> &[(usize, u8)] { + self.words.get(word).map(Vec::as_slice).unwrap_or_default() + } + + /// How many characters no font could show (they appear as `?`, or are left out when the + /// word's face has no `?` glyph either). + pub fn missing(&self) -> usize { + self.missing + } + + /// The advance of one planned code, in em units (the Type3 glyphs' own widths). `0.0` for + /// anything out of range. + pub fn advance(&self, font: usize, code: u8) -> f64 { + self.fonts.get(font).and_then(|f| f.glyphs.get(usize::from(code).checked_sub(1)?)).map_or(0.0, |(_, g)| g.width) + } + + /// The sum of the advances of a word's planned codes, in em units. + pub fn advance_of(&self, encoding: &[(usize, u8)]) -> f64 { + encoding.iter().map(|&(f, c)| self.advance(f, c)).sum() + } + + /// Whether every font the encoding references has been registered (has a resource name). + /// Unregistered plans fall back to the WinAnsi path, the same output as before the + /// Unicode mapping existed. + pub fn registered(&self, encoding: &[(usize, u8)]) -> bool { + encoding.iter().all(|&(f, _)| self.fonts.get(f).is_some_and(|font| !font.name.is_empty())) + } + + /// Characters with a planned glyph so far. + fn planned_glyphs(&self) -> usize { + self.fonts.iter().map(|f| f.glyphs.len()).sum() + } + + /// Add a glyph to the face's current font (a fresh one when that font is full). + fn add_glyph(&mut self, face: &CraftFont, ch: char, glyph: GlyphOutline) -> Option<(usize, u8)> { + let fi = self.font_of_face(face.family)?; + let code = self.fonts[fi].push(ch, glyph); + Some((fi, code)) + } + + /// The index of the face's open font, or a fresh one. `None` without craft-fonts (the + /// face itself comes from there, so this cannot happen for a planned face). + fn font_of_face(&mut self, family: &'static str) -> Option { + if let Some(fi) = self.fonts.iter().rposition(|f| f.family == family && f.glyphs.len() < MAX_TYPE3_GLYPHS) { + return Some(fi); + } + if self.fonts.len() >= MAX_PLAN_GLYPHS / MAX_TYPE3_GLYPHS { + return None; + } + self.fonts.push(PlannedType3::new(family)); + Some(self.fonts.len() - 1) + } + + /// The code of the font's `?` stand-in, adding the glyph when the font lacks it. + /// `prefer` is the run's current font; `None` without craft-fonts. + fn question_mark(&mut self, face: &CraftFont, prefer: Option) -> Option<(usize, u8)> { + if let Some(fi) = prefer + && let Some(code) = self.fonts[fi].glyphs.iter().position(|(c, _)| *c == '?') + { + return Some((fi, code as u8 + 1)); + } + let fi = self.font_of_face(face.family)?; + if let Some(code) = self.fonts[fi].glyphs.iter().position(|(c, _)| *c == '?') { + return Some((fi, code as u8 + 1)); + } + let glyph = craft_glyph(face, '?').ok()?; + let code = self.fonts[fi].push('?', glyph); + Some((fi, code)) + } +} + +/// The faces that can draw document text, in preference order: the Japanese document face +/// (serif document text) first, then the rest of the build input in manifest order. +fn document_faces() -> Vec<&'static CraftFont> { + let mut out = Vec::new(); + if let Some(d) = document_japanese_font() { + out.push(d); + } + for f in crate::CRAFT_FONTS { + if !out.iter().any(|d| std::ptr::eq(*d, f)) { + out.push(f); + } + } + out +} + +/// The face for a word: the first (in preference order) whose glyphs cover every character, +/// else the one covering the most characters (ties go to the earlier face). `None` when no +/// face covers any character of the word. +fn choose_face<'f>(faces: &[&'f CraftFont], word: &str, coverage: &mut Coverage) -> Option<&'f CraftFont> { + if let Some(f) = faces.iter().enumerate().find(|(i, f)| word.chars().all(|c| coverage.has(*i, f, c))).map(|(_, f)| *f) { + return Some(f); + } + faces + .iter() + .enumerate() + .map(|(i, f)| (word.chars().filter(|&c| coverage.has(i, f, c)).count(), std::cmp::Reverse(i), *f)) + .max_by(|a, b| (a.0, a.1).cmp(&(b.0, b.1))) + .filter(|(covered, _, _)| *covered > 0) + .map(|(_, _, f)| f) +} + +/// Which faces have a glyph for which characters, remembered for one plan: a page of words +/// asks the same (face, character) pairs over and over, and each fresh answer parses the font. +#[derive(Default)] +struct Coverage { + known: std::collections::HashMap<(usize, char), bool>, + lookups: usize, +} + +impl Coverage { + /// Whether face number `i` (`face`) has `ch`; asks the font once per pair. + fn has(&mut self, i: usize, face: &CraftFont, ch: char) -> bool { + *self.known.entry((i, ch)).or_insert_with(|| { + self.lookups += 1; + face_has(face.bytes, ch) + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// Face coverage is asked once per (face, character): repeating words cost no new font + /// lookups. + #[test] + fn face_coverage_is_remembered_per_face_and_character() { + let faces = document_faces(); + let Some(face) = faces.first() else { + eprintln!("skipping: built without craft-fonts (set CRAFT_FONTS_DIR to run it)"); + return; + }; + let mut coverage = Coverage::default(); + let first = (0..50).map(|_| choose_face(&faces, "日本語", &mut coverage)).last().flatten(); + assert!(first.is_some()); + assert!(coverage.lookups <= faces.len() * 3, "{} lookups for 3 characters", coverage.lookups); + assert_eq!(coverage.has(0, face, '日'), face_has(face.bytes, '日')); + } + + /// A page's vocabulary: German umlauts are WinAnsi, and a character no bundled face has + /// (here an emoji) is counted wherever it appears. + #[test] + fn win_ansi_words_get_no_encoding_and_count_only_real_losses() { + let plan = TextPlan::for_words(["Zürcher", "café", "size"]); + assert!(plan.words.iter().all(Vec::is_empty), "all WinAnsi: no Type3 fonts needed"); + assert!(plan.fonts.is_empty()); + assert_eq!(plan.missing(), 0); + let none = TextPlan::for_words(["plain text"]); + assert_eq!(none.missing(), 0); + // U+1F600 has no WinAnsi code and no glyph in any bundled face (with craft-fonts the + // JIS faces even cover the snowman): counted, shown as '?'. + let emoji = TextPlan::for_words(["\u{1f600}!"]); + assert_eq!(emoji.missing(), 1); + } + + #[test] + fn without_craft_fonts_every_non_win_ansi_character_is_reported() { + if !crate::CRAFT_FONTS.is_empty() { + eprintln!("built with craft-fonts: the planning tests with faces cover this instead"); + return; + } + let plan = TextPlan::for_words(["Привет", "日本", "abc"]); + assert!(plan.words.iter().all(Vec::is_empty)); + assert!(plan.fonts.is_empty()); + assert_eq!(plan.missing(), 8, "six Cyrillic + two CJK characters, each reported"); + } + + #[test] + fn encodings_are_aligned_and_total_the_advances() { + if crate::CRAFT_FONTS.is_empty() { + eprintln!("skipping: built without craft-fonts (set CRAFT_FONTS_DIR to run it)"); + assert!(TextPlan::for_words(["日本"]).fonts.is_empty()); + return; + } + let plan = TextPlan::for_words(["日本語", "語"]); + assert_eq!(plan.encoding(0).len(), 3); + assert_eq!(plan.encoding(1).len(), 1); + // 語 planned once keeps its code (and font) in both words. + let (f, c) = plan.encoding(1)[0]; + assert_eq!(plan.encoding(0)[2], (f, c), "the repeated character keeps its code"); + assert!(plan.advance(f, c) > 0.0); + assert_eq!(plan.advance(f, 0), 0.0, "codes start at 1"); + assert_eq!(plan.encoding(7), &[], "out of range degrades to the WinAnsi path"); + assert!(plan.advance_of(plan.encoding(0)) > plan.advance_of(plan.encoding(1))); + } + + #[test] + fn hostile_input_stays_bounded() { + // Thousands of DISTINCT characters: the covered ones fill fonts up to the caps, the + // rest are reported substitutions. + let word: String = (0..4000u32).filter_map(|i| char::from_u32(0x4e00 + i)).collect(); + let plan = TextPlan::for_words([word.as_str()]); + assert!(plan.planned_glyphs() <= MAX_PLAN_GLYPHS); + assert!(plan.fonts.iter().all(|f| f.glyphs.len() <= MAX_TYPE3_GLYPHS)); + assert!(plan.missing() > 0, "characters beyond the caps are reported, never silently dropped"); + } + + #[test] + fn substitution_and_unknown_scripts_come_out_counted() { + if crate::CRAFT_FONTS.is_empty() { + eprintln!("skipping: built without craft-fonts (set CRAFT_FONTS_DIR to run it)"); + return; + } + let plan = TextPlan::for_words(["日本\u{1f4a9}語"]); + // The emoji has no glyph in any bundled face; the kana/kanji do. + assert_eq!(plan.missing(), 1); + assert_eq!(plan.encoding(0).len(), 4, "the emoji still has an encoding: the font's ?"); + } +} diff --git a/crates/fonts/src/script.rs b/crates/fonts/src/script.rs index 70fc6ce092..94a8a93b8c 100644 --- a/crates/fonts/src/script.rs +++ b/crates/fonts/src/script.rs @@ -134,6 +134,17 @@ impl OutlinePen for Flatten { } } +/// Whether the face has a glyph for `ch` (a cheap character-map probe, no outline work). +pub fn face_has(bytes: &[u8], ch: char) -> bool { + FontRef::new(bytes).is_ok_and(|font| font.charmap().map(ch).is_some()) +} + +/// Return one glyph of `face` (a craft-fonts face), bounded so hostile replacement text cannot +/// allocate unbounded outline data. +pub fn craft_glyph(face: &crate::CraftFont, ch: char) -> Result { + glyph_in_face(face.bytes, ch) +} + /// Return one glyph of the Japanese document face ([`crate::document_japanese_font`], Shippori /// Mincho from craft-fonts), bounded so hostile replacement text cannot allocate unbounded /// outline data. [`GlyphError::NoFont`] when the build has no Japanese face. @@ -145,7 +156,11 @@ pub fn japanese_glyph(ch: char) -> Result { /// The bounded outline and advance from the explicitly selected craft-fonts face. /// Uses the same complexity and missing-glyph checks as [`japanese_glyph`]. pub fn japanese_glyph_from(face: &crate::CraftFont, ch: char) -> Result { - let Ok(font) = FontRef::new(face.bytes) else { return Err(GlyphError::Missing) }; + glyph_in_face(face.bytes, ch) +} + +fn glyph_in_face(bytes: &[u8], ch: char) -> Result { + let Ok(font) = FontRef::new(bytes) else { return Err(GlyphError::Missing) }; let loc = LocationRef::default(); let metrics = font.metrics(Size::unscaled(), loc); let scale = 1.0 / metrics.units_per_em.max(1) as f64; diff --git a/crates/ocr/Cargo.toml b/crates/ocr/Cargo.toml index 750e78b96d..1b04dbb522 100644 --- a/crates/ocr/Cargo.toml +++ b/crates/ocr/Cargo.toml @@ -16,5 +16,9 @@ ocrs = { version = "0.13.1", default-features = false, features = ["rten"] } rten = { version = "0.26.0", default-features = false, features = ["onnx_format", "rten_format"] } thiserror.workspace = true +[target.'cfg(not(target_arch = "wasm32"))'.dependencies] +# The page image is handed to the external tesseract process as a PNG. +png = "0.18" + [dev-dependencies] tempfile = "3.27" diff --git a/crates/ocr/README.md b/crates/ocr/README.md new file mode 100644 index 0000000000..cd33b186cf --- /dev/null +++ b/crates/ocr/README.md @@ -0,0 +1,104 @@ +# pdfcraft-ocr + +Layer L4 (may use `fonts` for the text layers' metrics): Scan & OCR — recognising the words in +a page raster and turning them back into page content (execution plan M10). + +```rust +// The caller renders a page to pixels and hands them over: +let image = OcrImage::new(w, h, rgba)?; // validated by construction +let lines = recognizer.recognize(&image, &options)?; // Vec: words in reading order +let placed: Vec = …; // PlacedWord::place maps boxes to user space +// The editable output draws real glyphs: plan the words, register the display faces into the +// plan (`pdfcraft_edit::add_type3_fonts`), then generate the layer. An unregistered plan falls +// back to WinAnsi, and characters no registered face has become '?'. +let plan = TextPlan::for_words(placed.iter().map(|w| w.text.as_str())); +add_type3_fonts(&mut doc, page, &mut plan)?; // names the plan's fonts on the page +let layer = visible_text_layer(&placed, &plan); // visible text: the editable output +// The searchable layer needs no display fonts: the engine's own layer (engine::ocr::layer) +// keeps every Unicode character with empty Type3 programs and a ToUnicode map beyond WinAnsi. +``` + +- **Engines behind one trait.** A [`Recognizer`] reports its `id`, whether it is `available`, + the languages it reads, and reads `Vec` from an image. Two exist: the pure-Rust + `OcrsRecognizer` (the `ocrs` crate, MIT/Apache-2.0, with its pre-trained models, + CC-BY-SA-4.0, fetched by `cargo xtask models`; found via `$PDFCRAFT_MODELS` or `models/` + beside the executable — see ATTRIBUTION.toml), and — off the web build — + `tesseract::TesseractCli`, which drives an installed `tesseract` binary as an external + process: never linked, bundled or downloaded. Implementations are shared across threads + (`Send + Sync`); nothing spawns on wasm32. +- **The tesseract process is fenced in.** The program comes from a user-configured location or + the PATH, never from document data (the engine validates a configured path as an absolute + path to an existing executable file before it is used). The language code is allow-listed + (`tesseract::language_code`, 12 codes) before anything reaches a process; the page image + goes to a private temp file (a fresh 0700 directory holding a 0600 file on Unix) that is + deleted afterwards; the run has a 120 s timeout, a 64 MiB output cap and a 50 000-word + reading cap (an incomplete reading is rejected rather than applied); every failure comes back through `OcrError` with its cause named. A configured + location that is invalid fails closed (it is never replaced by the PATH search). Availability + is probed with `--list-langs` (its list is read from stdout, or from stderr where older + versions print it); a failed probe is remembered for 30 s, so an install that appears later + is found without a restart. Known limit: at the timeout or the output cap only the direct + child is killed; a wrapper script's own children may outlive it. +- **Confidence drives review.** One classifier (`band_for`) puts a word in the High + (≥ 90), Medium (70–89) or Low (< 70) band — the review overlay's colours, the legend and the + counts; the batch flag rule (`low_confidence`, default threshold 60) is deliberately a + separate threshold. The `ocrs` models report no confidence (its words carry `None`); + a reading whose confidence is unknown is treated as unknown, never as good. +- **Two engines can be merged (ROVER).** Strategies: `PrimaryOnly` (the default), and two + ensembles — `ConfidenceWeighted` (the secondary engine joins when the primary's mean + confidence is under 70 or unknown; an empty primary keeps the fast path) and `RoverVote` + (both engines always run). The merge lets every primary word adopt the best-overlapping + unused secondary word at IoU ≥ 0.5; the higher confidence wins text, box and confidence (a + tie keeps the primary; against a primary word of unknown confidence — ocrs — the secondary + wins only at 70 or more, `UNKNOWN_CONFIDENCE_BAR`), the survivor is named `"ROVER"`, and + unclaimed secondary words are + appended under their engine's name. A merge is capped at 100 M word-pair comparisons (over + it, the primary reading stands alone), and a failing partner degrades to the primary with a + recorded note — a page is never skipped because a partner could not read it. +- **Preprocessing between the raster and recognition**, a pure function in fixed order: DPI + normalization to 300 (only when one of the steps below is on, since they are calibrated for + it; with all of them off the raster is read exactly as rendered, and a scan at or above 300 + is left alone), then the opt-in steps — + orientation (90°/180°/270° from projection-profile banding), deskew (± 5°, shear-scored, + below 0.1° is a no-op), denoise (3×3 median) and binarization (Sauvola, window 8, k 0.34). + Every geometric step composes one exact affine `InverseTransform`, and callers map every + word box back to the original raster before placing it. The heuristics are conservative and + panic-free: too small to judge, blank, or no clear winner is a no-op, never a guess. +- **Searchable output preserves Unicode without display fonts.** The engine uses empty + Type3 character programs and `/ToUnicode` maps for non-WinAnsi words, leaving the scan + unchanged. WinAnsi words use `/PCHelv`; the `/OCR` layer is fitted to each word's box. +- **Editable output uses display glyphs.** `visible_text_layer` writes visible text whose + size comes from the box height and whose horizontal scaling uses real font advances. + Beyond WinAnsi, attributed craft-fonts faces supply outline Type3 glyphs with `/ToUnicode`. + Uncovered characters become `?` and are reported in editable output notes; searchable + output retains them. The engine refuses nonfinite geometry or oversized input before + planning fonts: 4 MiB of text and 65,536 words, across all pages for editable export. + Empty geometry is skipped; rotated words carry `across`/`up` vectors and retain direction. +- **Hostile input is rejected, not panicked at.** `OcrImage` holds private, validated state: + non-zero sides, a 10 000-pixel side cap, a 64-megapixel area cap (checked arithmetic — the + counts come from documents, so they are untrusted), and an exact buffer length. + +## Where it is wired + +The engine's `ocr` module renders pages into `OcrImage`s, runs jobs (`OcrJob::run`) and applies +the words: searchable-image mode draws the invisible layer over the page's own image and leaves +that image untouched; editable mode writes a NEW `_ocr.pdf` of visible text (the scan +image is not carried over, the source is never overwritten). Skip probes keep pages — and +whole files — that already contain text out of the work unless Force OCR overrides them. A +searchable result applies as one undoable step; the Editable text output opens the OCR Verify +screen first, and Scan & OCR ▸ Correct recognized text reviews any recognition there before it +is applied (confidence-coloured boxes, corrections, a per-language user dictionary — a +plain-text file in the preferences folder, so words added from a document persist on disk +outside it until removed — Re-OCR of a region or a page, Accept/Reject), a +multiple-file run writes searchable copies as it goes, and the automation tools `ocr_recognize`, +`ocr_words`, `ocr_status` and `ocr_recognize_files` expose recognition headlessly. +`ocr_words` with `retain_review` returns a bounded session-local handle; `ocr_review_apply` +applies text corrections at the stored original boxes through its stale-document guard, +or discards the result. Empty corrections remove words. Searchable acceptance is one undo +step; editable acceptance writes a separate file. `ocr_dictionary` reads or atomically +updates the UI's per-language dictionary format through an explicitly root-confined path. + +Not yet: languages beyond the tested set (the `ocrs` models read only unaccented Latin; the +Tesseract side is bounded by its allow-list and whatever its installed language packs cover), +OCR in the browser (the tesseract module is compiled out for wasm32), the deep scan-cleanup +options (descreen, background removal, edge-shadow removal), and editable glyph coverage +beyond the available attributed faces. diff --git a/crates/ocr/src/confidence.rs b/crates/ocr/src/confidence.rs new file mode 100644 index 0000000000..1c0de605f5 --- /dev/null +++ b/crates/ocr/src/confidence.rs @@ -0,0 +1,106 @@ +//! Confidence: one classifier ([`band_for`]) drives the review overlay, the legend, the counts +//! and the uncertain-word navigation; the batch rule ([`low_confidence`]) is a separate +//! threshold with its own range check. The two rules are deliberately never merged. + +/// The three confidence bands (the review overlay's green / yellow / red). +#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum ConfidenceBand { + Low, + Medium, + High, +} + +impl ConfidenceBand { + /// The legend line for this band, so the meaning never rests on colour alone. + pub fn legend(self) -> &'static str { + match self { + ConfidenceBand::High => "HIGH (>= 90%)", + ConfidenceBand::Medium => "MEDIUM (70-89%)", + ConfidenceBand::Low => "LOW (< 70%)", + } + } +} + +/// The one classifier: `>= 90` is High, `70..=89` Medium, `< 70` Low. Out-of-range values +/// clamp (negative → Low, above 100 → High); NaN is Low. +pub fn band_for(confidence: f32) -> ConfidenceBand { + if confidence >= 90.0 { + ConfidenceBand::High + } else if confidence >= 70.0 { + ConfidenceBand::Medium + } else { + ConfidenceBand::Low + } +} + +/// The batch rule, separate from [`band_for`]: a word is flagged when `0 <= conf <= threshold` +/// (default 60). Negative or above-threshold values are not flagged; NaN is not. +pub fn low_confidence(confidence: f32, threshold: f32) -> bool { + (0.0..=threshold).contains(&confidence) +} + +/// A word whose confidence lands in the Low band — a suspect for review. A word without a +/// reported confidence has no signal and is never a suspect. +pub fn is_suspect(confidence: Option) -> bool { + confidence.is_some_and(|c| band_for(c) == ConfidenceBand::Low) +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The boundary matrix: 89 and 90, 69 and 70 straddle the bands. + #[test] + fn bands_straddle_the_boundaries() { + assert_eq!(band_for(89.0), ConfidenceBand::Medium); + assert_eq!(band_for(90.0), ConfidenceBand::High); + assert_eq!(band_for(69.0), ConfidenceBand::Low); + assert_eq!(band_for(70.0), ConfidenceBand::Medium); + assert_eq!(band_for(0.0), ConfidenceBand::Low); + assert_eq!(band_for(100.0), ConfidenceBand::High); + } + + /// Out-of-range values clamp, and NaN lands in the Low band instead of comparing equal to + /// nothing. + #[test] + fn out_of_range_confidences_clamp() { + assert_eq!(band_for(-0.5), ConfidenceBand::Low, "negative → Low"); + assert_eq!(band_for(1000.0), ConfidenceBand::High, "above 100 → High"); + assert_eq!(band_for(f32::NAN), ConfidenceBand::Low); + assert_eq!(band_for(f32::NEG_INFINITY), ConfidenceBand::Low); + assert_eq!(band_for(f32::INFINITY), ConfidenceBand::High); + } + + /// The batch threshold is its own rule: everything in `0..=threshold` flags, including + /// words band_for calls Medium (60 < 70), and negatives never do. + #[test] + fn batch_threshold_is_separate_from_the_bands() { + assert!(low_confidence(60.0, 60.0), "at the threshold it flags"); + assert!(low_confidence(0.0, 60.0)); + assert!(!low_confidence(60.5, 60.0)); + assert!(!low_confidence(-1.0, 60.0), "negative confidences are out of the rule's range"); + assert!(!low_confidence(f32::NAN, 60.0)); + // The rules disagree on real values: 65 is inside the Low band yet outside the batch + // rule with the default threshold (60), which is why the two must not be merged. + assert!(!low_confidence(65.0, 60.0)); + assert_eq!(band_for(65.0), ConfidenceBand::Low); + } + + /// Unconfident words are not suspects; Low-band words are. + #[test] + fn suspects_need_a_low_band_confidence() { + assert!(!is_suspect(None)); + assert!(!is_suspect(Some(95.0))); + assert!(!is_suspect(Some(70.0))); + assert!(is_suspect(Some(69.9))); + assert!(is_suspect(Some(0.0))); + } + + /// The legend names the thresholds, so the overlay never rests on colour alone. + #[test] + fn legend_text_matches_the_classifier() { + assert_eq!(ConfidenceBand::High.legend(), "HIGH (>= 90%)"); + assert_eq!(ConfidenceBand::Medium.legend(), "MEDIUM (70-89%)"); + assert_eq!(ConfidenceBand::Low.legend(), "LOW (< 70%)"); + } +} diff --git a/crates/ocr/src/lib.rs b/crates/ocr/src/lib.rs index 7f58270e1a..ff5b920dca 100644 --- a/crates/ocr/src/lib.rs +++ b/crates/ocr/src/lib.rs @@ -1,35 +1,50 @@ //! pdfcraft-ocr — Scan & OCR ▸ Recognize text (L4). //! -//! Recognition runs the ocrs engine (MIT/Apache-2.0) with its pre-trained models (CC-BY-SA-4.0, -//! fetched by `cargo xtask models`; see ATTRIBUTION.toml). The caller renders a page to pixels; -//! [`Ocr::recognize`] finds the words in them, and [`text_layer`] turns words placed in user space -//! into page content: invisible text (rendering mode 3) over each word, so the page becomes a +//! The caller renders a page to pixels ([`OcrImage`]) and hands it to a [`Recognizer`]. Two +//! engines exist: the pure-Rust `OcrsRecognizer` (ocrs, MIT/Apache-2.0, with its pre-trained +//! models, CC-BY-SA-4.0, fetched by `cargo xtask models`; see ATTRIBUTION.toml) and — off the +//! web — `tesseract::TesseractCli`, which drives an installed `tesseract` binary as an external +//! process. `merge` combines two engines' readings (ROVER), `preprocess` cleans the raster up +//! between rendering and recognition, and [`text_layer`] turns words placed in user space into +//! page content: invisible text (rendering mode 3) over each word, so the page becomes a //! searchable image (Acrobat's "Searchable Image (Exact)": the image is left untouched). //! -//! The models read the Latin alphabet (English and other languages written without accents). +//! Visible editable output uses [`pdfcraft_fonts::TextPlan`] and optional craft-fonts glyph +//! outlines; unsupported visible characters become `?` and are counted. The engine's invisible +//! searchable layer uses empty Type3 programs and ToUnicode maps, preserving Unicode without +//! display fonts. WinAnsi-only invisible text keeps the standard Helvetica layout here. +//! +//! The ocrs models read the Latin alphabet (English and other languages written without +//! accents); Tesseract (the `tesseract` module, off the web build) reads whatever its installed +//! language packs cover. #![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] -use std::path::{Path, PathBuf}; +pub mod confidence; +pub mod merge; +mod ocrs; +pub mod preprocess; +#[cfg(not(target_arch = "wasm32"))] +pub mod tesseract; +pub use confidence::{ConfidenceBand, band_for, is_suspect, low_confidence}; +pub use merge::{MergeStrategy, StrategyOutcome, mean_confidence, recognize_with_strategy, rover_merge}; +pub use ocrs::{ALPHABET, DETECTION_MODEL, LANGUAGES, Models, Ocr, OcrsRecognizer, RECOGNITION_MODEL}; + +pub use pdfcraft_fonts::TextPlan; pub use pdfcraft_fonts::helvetica_width; -/// The model files, as named in ATTRIBUTION.toml. -pub const DETECTION_MODEL: &str = "text-detection.onnx"; -pub const RECOGNITION_MODEL: &str = "text-recognition.onnx"; +/// Hard cap on either side of an image any code in this crate will touch, in pixels; larger +/// images are rejected with [`OcrError::ImageTooLarge`] instead of being processed. +pub const MAX_SIDE: u32 = 10_000; -/// The recognition model's alphabet, as it was trained (ocrs-models' `DEFAULT_ALPHABET`): its -/// output classes follow this order. The ocrs crate's own copy has an `E` where the training had -/// `€`, which made a scanned "€250" read as "E250". -pub const ALPHABET: &str = " 0123456789!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~€ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"; - -/// The languages the models read (ISO 639-1); all use the Latin alphabet without accents. -pub const LANGUAGES: &[(&str, &str)] = &[("en", "English")]; +/// Hard cap on an image's TOTAL area, in pixels (64 MP): sides under [`MAX_SIDE`] can still +/// multiply out to a raster no engine should have to read; larger images are rejected with +/// [`OcrError::ImageTooLarge`] instead of being processed. +pub const MAX_PIXELS: u32 = 64_000_000; #[derive(Debug, thiserror::Error)] pub enum OcrError { - /// Shown to end users (release packages ship the models; a source build fetches them with - /// `cargo xtask models`). The UI catalogs key this exact text. #[error("the text recognition models are not installed (reinstall PdfCraft, or set PDFCRAFT_MODELS to the folder that holds them)")] NoModels, #[error("loading {0}: {1}")] @@ -38,52 +53,93 @@ pub enum OcrError { Recognize(String), #[error("the image is empty")] EmptyImage, + #[error("the image is larger than the {MAX_SIDE}-pixel side or {MAX_PIXELS}-pixel limit")] + ImageTooLarge, + #[error("the image buffer holds {got} bytes, expected {want} for {width}x{height} RGBA")] + ImageSize { got: usize, want: usize, width: u32, height: u32 }, + #[error("the tesseract program is not available: {0}")] + NoTesseract(String), + #[error("tesseract did not finish within {0} seconds")] + Timeout(u64), + #[error("tesseract produced more than {0} bytes of output")] + OutputTooLarge(u64), + #[error("tesseract failed: {0}")] + Process(String), } -/// Where the two model files are. -#[derive(Clone, Debug, PartialEq)] -pub struct Models { - pub detection: PathBuf, - pub recognition: PathBuf, +/// An RGBA8 raster, row-major, top-left origin: what the engines read. The fields are private +/// so the invariants (non-zero, capped, exactly `width * height * 4` bytes) hold by +/// construction: an [`OcrImage`] in hand has always been validated, whatever produced it. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct OcrImage { + width: u32, + height: u32, + rgba: Vec, } -impl Models { - /// A complete model pair in `dir`, preferring current ONNX files over legacy RTEN files. - /// An incomplete update never mixes formats or hides an intact legacy pair. - pub fn in_dir(dir: &Path) -> Option { - [(DETECTION_MODEL, RECOGNITION_MODEL), ("text-detection.rten", "text-recognition.rten")].into_iter().find_map(|(detection, recognition)| { - let m = Models { detection: dir.join(detection), recognition: dir.join(recognition) }; - (m.detection.is_file() && m.recognition.is_file()).then_some(m) - }) - } - - /// Look for the models: `$PDFCRAFT_MODELS`, then where the release packages install them - /// beside the executable ([`Models::dirs_beside_exe`]), then the source tree's `assets/models/`. - pub fn find() -> Option { - Self::search_dirs().iter().find_map(|d| Self::in_dir(d)) - } - - /// The directories [`Models::find`] looks in, in order. - pub fn search_dirs() -> Vec { - let mut dirs = Vec::new(); - if let Some(d) = std::env::var_os("PDFCRAFT_MODELS") { - dirs.push(PathBuf::from(d)); +impl OcrImage { + /// Validate the sides ([`MAX_SIDE`]) and total area ([`MAX_PIXELS`]) and that `rgba` holds + /// exactly `width * height * 4` bytes (checked arithmetic; the counts come from documents, + /// so they are untrusted). + pub fn new(width: u32, height: u32, rgba: Vec) -> Result { + if width == 0 || height == 0 { + return Err(OcrError::EmptyImage); } - if let Some(exe) = std::env::current_exe().ok().and_then(|e| e.parent().map(Path::to_path_buf)) { - dirs.extend(Self::dirs_beside_exe(&exe)); + if width > MAX_SIDE || height > MAX_SIDE { + return Err(OcrError::ImageTooLarge); + } + let pixels = (width as usize).checked_mul(height as usize).ok_or(OcrError::ImageTooLarge)?; + if pixels > MAX_PIXELS as usize { + return Err(OcrError::ImageTooLarge); + } + match pixels.checked_mul(4) { + Some(want) if rgba.len() == want => Ok(OcrImage { width, height, rgba }), + Some(want) => Err(OcrError::ImageSize { got: rgba.len(), want, width, height }), + None => Err(OcrError::ImageTooLarge), } - dirs.push(Path::new(env!("CARGO_MANIFEST_DIR")).join("../../assets/models")); - dirs } - /// Where the release packages put the models, relative to the directory holding the - /// executable (packaging/*): - /// - `models/` beside it: the Windows MSI and portable zip; - /// - `../Resources/models`: the macOS app bundle (`Contents/MacOS` → `Contents/Resources`); - /// - `../share/pdfcraft/models`: the FHS-style trees (`bin/` → `share/`) of the Linux - /// deb/rpm/tar.gz/AppImage/Flatpak and the FreeBSD tarball, wherever they are installed. - pub fn dirs_beside_exe(exe_dir: &Path) -> Vec { - vec![exe_dir.join("models"), exe_dir.join("../Resources/models"), exe_dir.join("../share/pdfcraft/models")] + /// The image's width, in pixels. + pub fn width(&self) -> u32 { + self.width + } + + /// The image's height, in pixels. + pub fn height(&self) -> u32 { + self.height + } + + /// The RGBA bytes, row-major, exactly `width * height * 4` of them. + pub fn rgba(&self) -> &[u8] { + &self.rgba + } + + /// The smallest side, in pixels. + pub fn min_side(&self) -> u32 { + self.width.min(self.height) + } + + /// One RGBA byte at integer pixel (`x`, `y`), edges clamped; 255 (white) outside. Only + /// whole-pixel positions and channel 0–3 make sense; anything else reads clamped. + pub fn byte(&self, x: f64, y: f64, ch: usize) -> u8 { + let stride = self.width as usize * 4; + if stride == 0 || self.height == 0 || ch > 3 { + return 255; + } + let xi = (x.round().max(0.0) as usize).min(self.width as usize - 1); + let yi = (y.round().max(0.0) as usize).min(self.height as usize - 1); + self.rgba.get(yi * stride + xi * 4 + ch).copied().unwrap_or(255) + } + + /// The grayscale value of the pixel at (`x`, `y`), 0 (ink) … 255 (paper), with the borders + /// clamped (edges replicate). `None` only when the image is empty. + pub fn gray(&self, x: i64, y: i64) -> Option { + let x = x.clamp(0, self.width as i64 - 1); + let y = y.clamp(0, self.height as i64 - 1); + let row = y as usize * self.width as usize * 4; + let o = row + x as usize * 4; + let px = self.rgba.get(o..o + 4)?; + Some(((299 * px[0] as u32 + 587 * px[1] as u32 + 114 * px[2] as u32) / 1000) as u8) } } @@ -92,6 +148,26 @@ impl Models { pub struct Word { pub text: String, pub rect: [f32; 4], + /// How sure the engine is, 0–100, when it reports a confidence at all (`None`: the ocrs + /// engine does not). + pub confidence: Option, + /// Which engine read this word: the recogniser's [`Recognizer::id`], or `"ROVER"` when two + /// engines were merged and the surviving text is the merged choice. + pub source: String, +} + +impl Word { + pub fn new(text: impl Into, rect: [f32; 4], source: &str) -> Word { + Word { text: text.into(), rect, confidence: None, source: source.into() } + } + + /// Set the confidence (builder style), clamped to the documented 0–100 scale (a negative + /// value is 0, anything over 100 is 100). NaN means no confidence was reported (`None`), + /// the same unknown the engines that report nothing carry. + pub fn with_confidence(mut self, confidence: f32) -> Word { + self.confidence = if confidence.is_nan() { None } else { Some(confidence.clamp(0.0, 100.0)) }; + self + } } /// A recognised line of words, in reading order. @@ -106,63 +182,34 @@ impl Line { } } -/// A loaded recogniser. Loading takes a moment; keep one and reuse it. -pub struct Ocr { - engine: ocrs::OcrEngine, +/// What a recogniser needs to know beyond the pixels. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct RecognizeOptions { + /// Language code from [`LANGUAGES`] (`"en"`, `"de"`, …). Engines map it to their own codes + /// (Tesseract: `tesseract::language_code`, allow-listed before anything reaches a process); + /// unknown codes fall back to the engine's default. + pub language: String, } -impl Ocr { - pub fn load(models: &Models) -> Result { - let load = |p: &Path| rten::Model::load_file(p).map_err(|e| OcrError::Load(p.display().to_string(), e.to_string())); - let params = ocrs::OcrEngineParams { - detection_model: Some(load(&models.detection)?), - recognition_model: Some(load(&models.recognition)?), - alphabet: Some(ALPHABET.into()), - ..Default::default() - }; - let engine = ocrs::OcrEngine::new(params).map_err(|e| OcrError::Load("ocr engine".into(), e.to_string()))?; - Ok(Ocr { engine }) +impl Default for RecognizeOptions { + fn default() -> Self { + RecognizeOptions { language: "en".into() } } +} - /// Load the models found by [`Models::find`]. - pub fn find() -> Result { - Self::load(&Models::find().ok_or(OcrError::NoModels)?) - } - - /// Recognise the text in an RGBA (or RGB, or grey) image, `width` × `height` pixels. - pub fn recognize(&self, pixels: &[u8], width: u32, height: u32) -> Result, OcrError> { - if width == 0 || height == 0 || pixels.is_empty() { - return Err(OcrError::EmptyImage); - } - // ocrs wants 1 or 3 channels. - let rgb: std::borrow::Cow<[u8]> = if pixels.len() == (width * height * 4) as usize { - pixels.as_chunks::<4>().0.iter().flat_map(|p| [p[0], p[1], p[2]]).collect::>().into() - } else { - pixels.into() - }; - let err = |e: &dyn std::fmt::Display| OcrError::Recognize(e.to_string()); - let source = ocrs::ImageSource::from_bytes(&rgb, (width, height)).map_err(|e| err(&e))?; - let input = self.engine.prepare_input(source).map_err(|e| err(&e))?; - let found = self.engine.detect_words(&input).map_err(|e| err(&e))?; - let lines = self.engine.find_text_lines(&input, &found); - let read = self.engine.recognize_text(&input, &lines).map_err(|e| err(&e))?; - use ocrs::TextItem; - Ok(read - .into_iter() - .flatten() - .map(|line| Line { - words: line - .words() - .map(|w| { - let r = w.bounding_rect(); - Word { text: w.to_string(), rect: [r.left() as f32, r.top() as f32, r.right() as f32, r.bottom() as f32] } - }) - .filter(|w| !w.text.trim().is_empty()) - .collect(), - }) - .filter(|l| !l.words.is_empty()) - .collect()) - } +/// One text-recognition engine. Implementations are shared across threads +/// (`Arc`); `available` and `recognize` may spawn processes only off the web +/// build (see the `tesseract` module). +pub trait Recognizer: Send + Sync { + /// The engine's short name (`"ocrs"`, `"tesseract"`): what [`Word::source`] carries and + /// what settings and tools name it by. + fn id(&self) -> &'static str; + /// Whether this engine can run right now (models installed, binary found). + fn available(&self) -> bool; + /// The languages it reads, as `(code, name)` pairs (`"en"`, `"English"`). + fn languages(&self) -> Vec<(String, String)>; + /// Read the words in `image`, in reading order. + fn recognize(&self, image: &OcrImage, options: &RecognizeOptions) -> Result, OcrError>; } /// A word placed on a page, in PDF user space: the bottom-left corner of its box, and the @@ -173,6 +220,10 @@ pub struct PlacedWord { pub origin: [f64; 2], pub across: [f64; 2], pub up: [f64; 2], + /// Confidence 0–100, when the engine reported one. + pub confidence: Option, + /// Which engine read this word (see [`Word::source`]). + pub source: String, } impl PlacedWord { @@ -183,7 +234,14 @@ impl PlacedWord { let o = to_user(l, b); let br = to_user(r, b); let tl = to_user(l, t); - PlacedWord { text: word.text.clone(), origin: o, across: [br[0] - o[0], br[1] - o[1]], up: [tl[0] - o[0], tl[1] - o[1]] } + PlacedWord { + text: word.text.clone(), + origin: o, + across: [br[0] - o[0], br[1] - o[1]], + up: [tl[0] - o[0], tl[1] - o[1]], + confidence: word.confidence, + source: word.source.clone(), + } } } @@ -214,32 +272,193 @@ fn num(v: f64) -> String { if s == "-0" { "0".into() } else { s.into() } } -/// Page content that writes `words` as invisible text in `/PCHelv` (standard Helvetica, -/// WinAnsiEncoding), each word stretched to its box so selection and search highlight the -/// right place. Marked content `/OCR` so it can be told apart from the page's own text. -pub fn text_layer(words: &[PlacedWord]) -> Vec { +/// One run of a word's planned characters in one Type3 font: the resource name, the codes to +/// show (one byte per character), and the run's start as an offset in em units from the word's +/// origin (the accumulated advances of the runs before it). +struct PlannedRun { + font: String, + codes: Vec, + start: f64, +} + +/// The planned runs of word `i` (the same index [`pdfcraft_fonts::TextPlan::for_words`] planned +/// the words with), or `None` when the word is drawn in Helvetica: every character has a +/// WinAnsi code, or nothing could be planned/registered for it (the pre-Unicode output). +fn planned_runs(plan: &pdfcraft_fonts::TextPlan, word: usize) -> Option<(Vec, f64)> { + let enc = plan.encoding(word); + if enc.is_empty() || !plan.registered(enc) { + return None; + } + let mut runs: Vec = Vec::new(); + let mut total = 0.0; + for &(f, code) in enc { + let font = plan.fonts[f].name.as_str(); + match runs.last_mut() { + Some(last) if last.font == font => last.codes.push(code), + _ => runs.push(PlannedRun { font: font.to_string(), codes: vec![code], start: total }), + } + total += plan.advance(f, code); + } + (total.is_finite() && total > 0.0).then_some((runs, total)) +} + +/// Page content that writes `words` as VISIBLE text at their original boxes: the +/// editable-text output's page content. WinAnsi words are set in `/PCHelv` (standard +/// Helvetica); words with characters beyond WinAnsi are set in the plan's Type3 fonts, one run +/// per font. Each word fills its box — the font size comes from the box height and the +/// horizontal scaling is computed from the real advances of the font that draws the word +/// (Helvetica's approximated, the Type3 faces' exact) and emitted as an explicit `Tz` +/// percentage. Words whose box is empty (or whose text has no measurable width) are skipped; a +/// page that ends up with nothing simply shows nothing. +pub fn visible_text_layer(words: &[PlacedWord], plan: &pdfcraft_fonts::TextPlan) -> Vec { + visible_text_layer_at(words, plan, 0) +} + +/// Visible text for words starting at `word_offset` in a shared document plan. +pub fn visible_text_layer_at(words: &[PlacedWord], plan: &pdfcraft_fonts::TextPlan, word_offset: usize) -> Vec { + // The rendering mode and the font are graphics state, which `ET` doesn't reset: without + // `q … Q` they would stay set for every later content stream on the page, so text added + // after OCR (a header or footer) drew invisibly (#3029). + let mut out = b"/OCR BMC\nq\nBT\n0 Tr\n/PCHelv 1 Tf\n".to_vec(); + // The font and size the text state is set in, so a Tf is written only when they change. + // (The header's size-1 Helvetica covers layers of WinAnsi words only.) + let mut current: Option<(String, f64)> = Some(("PCHelv".to_string(), 1.0)); + for (i, w) in words.iter().enumerate() { + let height = w.up[0].hypot(w.up[1]); + let across = w.across[0].hypot(w.across[1]); + let (ux, uy) = (w.across[0] / across, w.across[1] / across); + let (vx, vy) = (w.up[0] / height, w.up[1] / height); + if !across.is_finite() + || !height.is_finite() + || across <= 0.0 + || height <= 0.0 + || !ux.is_finite() + || !vx.is_finite() + || !w.origin.iter().all(|v| v.is_finite()) + { + continue; + } + let size = height / BOX_EM; + let runs = planned_runs(plan, word_offset.saturating_add(i)); + // The word's natural advance at this size: Helvetica's approximation, or the exact sum + // of the planned glyphs' widths. + let natural = runs.as_ref().map_or_else(|| helvetica_width(&w.text, size), |(_, total)| total * size); + if natural.is_nan() || natural <= 0.0 { + continue; + } + // Horizontal scaling as a percentage of the natural advance at this size, so the word + // lands exactly on its box (100 = no scaling; a long box on a short word stretches it). + let tz = (across / natural * 100.0).max(1.0); + // The baseline sits above the box's bottom edge by the descender depth. + let o = [w.origin[0] + vx * DESCENT * size, w.origin[1] + vy * DESCENT * size]; + out.extend_from_slice(format!("{} Tz ", num(tz)).as_bytes()); + match &runs { + None => { + if current.as_ref().is_some_and(|(f, s)| f != "PCHelv" || *s != size) { + out.extend_from_slice(format!("/PCHelv {} Tf ", num(size)).as_bytes()); + current = Some(("PCHelv".to_string(), size)); + } + out.extend_from_slice(format!("{} {} {} {} {} {} Tm ", num(ux), num(uy), num(vx), num(vy), num(o[0]), num(o[1])).as_bytes()); + out.extend(pdfcraft_fonts::literal(&pdfcraft_fonts::win_ansi(&w.text))); + out.extend_from_slice(b" Tj\n"); + } + Some((planned, total)) => { + for run in planned { + if current.as_ref().is_some_and(|(f, s)| *f != run.font || *s != size) { + out.extend_from_slice(format!("/{} {} Tf ", run.font, num(size)).as_bytes()); + current = Some((run.font.clone(), size)); + } + // The run's origin: the word's baseline plus the across vector scaled by + // the advances before this run (the stretch applies to runs alike). + let frac = run.start / total; + out.extend_from_slice( + format!( + "{} {} {} {} {} {} Tm ", + num(ux), + num(uy), + num(vx), + num(vy), + num(o[0] + w.across[0] * frac), + num(o[1] + w.across[1] * frac) + ) + .as_bytes(), + ); + out.extend(pdfcraft_fonts::literal(&run.codes)); + out.extend_from_slice(b" Tj\n"); + } + } + } + } + out.extend_from_slice(b"ET\nQ\nEMC\n"); + out +} + +/// Page content that writes `words` as invisible text (rendering mode 3), each word stretched +/// to its box so selection and search highlight the right place. WinAnsi words are set in +/// `/PCHelv` (standard Helvetica); words with characters beyond WinAnsi are set in the plan's +/// Type3 fonts, one run per font, with the same exact box fit. Marked content `/OCR` so it can +/// be told apart from the page's own text. +pub fn text_layer(words: &[PlacedWord], plan: &pdfcraft_fonts::TextPlan) -> Vec { // `3 Tr` and the font are graphics state, which `ET` doesn't reset: without `q … Q` they // would stay set for every later content stream on the page, so text added after OCR (a // header or footer) drew invisibly (#3029). let mut out = b"/OCR BMC\nq\nBT\n3 Tr\n/PCHelv 1 Tf\n".to_vec(); - for w in words { - let width = helvetica_width(&w.text, 1.0); + let mut current: Option<(String, f64)> = Some(("PCHelv".to_string(), 1.0)); + for (i, w) in words.iter().enumerate() { + let runs = planned_runs(plan, i); + let width = runs.as_ref().map_or_else(|| helvetica_width(&w.text, 1.0), |(_, total)| *total); let height = w.up[0].hypot(w.up[1]); - if width <= 0.0 || height <= 0.0 { + let across = w.across[0].hypot(w.across[1]); + // Only positive, finite geometry reaches the content stream: a zero-sized box cannot + // be drawn, and a NaN or infinite one would put non-finite numbers on the page. (NaN + // fails every comparison, so the positivity checks already refuse it; the finiteness + // checks keep that property explicit.) + if width <= 0.0 || !(height > 0.0 && height.is_finite()) || !(across > 0.0 && across.is_finite()) || !w.origin.iter().all(|v| v.is_finite()) { continue; } - // The em square: the word's own ink (cap height or x-height, descenders or not) spans the - // box, so the text is as tall as the scanned word; its width is stretched to fit the word. - let (top, bottom) = ink_extent(&w.text); + // The em square. Helvetica words: the word's own ink (cap height or x-height, + // descenders or not) spans the box, so the text is as tall as the scanned word. Planned + // (Type3) words: those Helvetica letter heights do not describe the face, so the box spans + // the ascender-descender height. Either way the width is stretched to fit the word. + let (top, bottom) = if runs.is_some() { (BOX_EM - DESCENT, -DESCENT) } else { ink_extent(&w.text) }; let em = [w.up[0] / (top - bottom), w.up[1] / (top - bottom)]; - let ax = [w.across[0] / width, w.across[1] / width]; let o = [w.origin[0] - em[0] * bottom, w.origin[1] - em[1] * bottom]; - let m = [ax[0], ax[1], em[0], em[1], o[0], o[1]]; - let nums: Vec = m.iter().map(|v| num(*v)).collect(); - out.extend_from_slice(nums.join(" ").as_bytes()); - out.extend_from_slice(b" Tm "); - out.extend(pdfcraft_fonts::literal(&pdfcraft_fonts::win_ansi(&w.text))); - out.extend_from_slice(b" Tj\n"); + match &runs { + None => { + let ax = [w.across[0] / width, w.across[1] / width]; + if current.as_ref().is_some_and(|(f, s)| f != "PCHelv" || *s != 1.0) { + out.extend_from_slice(b"/PCHelv 1 Tf "); + current = Some(("PCHelv".to_string(), 1.0)); + } + let m = [ax[0], ax[1], em[0], em[1], o[0], o[1]]; + let nums: Vec = m.iter().map(|v| num(*v)).collect(); + out.extend_from_slice(nums.join(" ").as_bytes()); + out.extend_from_slice(b" Tm "); + out.extend(pdfcraft_fonts::literal(&pdfcraft_fonts::win_ansi(&w.text))); + out.extend_from_slice(b" Tj\n"); + } + Some((planned, total)) => { + for run in planned { + if current.as_ref().is_some_and(|(f, s)| *f != run.font || *s != 1.0) { + out.extend_from_slice(format!("/{} 1 Tf ", run.font).as_bytes()); + current = Some((run.font.clone(), 1.0)); + } + let m = [ + w.across[0] / total, + w.across[1] / total, + em[0], + em[1], + o[0] + w.across[0] * run.start / total, + o[1] + w.across[1] * run.start / total, + ]; + let nums: Vec = m.iter().map(|v| num(*v)).collect(); + out.extend_from_slice(nums.join(" ").as_bytes()); + out.extend_from_slice(b" Tm "); + out.extend(pdfcraft_fonts::literal(&run.codes)); + out.extend_from_slice(b" Tj\n"); + } + } + } } out.extend_from_slice(b"ET\nQ\nEMC\n"); out @@ -253,10 +472,22 @@ mod tests { fn places_words_through_the_mapping() { // 2 pixels per point, page 612 × 792, y flipped. let to_user = |x: f32, y: f32| [x as f64 / 2.0, 792.0 - y as f64 / 2.0]; - let w = PlacedWord::place(&Word { text: "Hello".into(), rect: [100.0, 200.0, 300.0, 240.0] }, to_user); + let w = PlacedWord::place(&Word::new("Hello", [100.0, 200.0, 300.0, 240.0], "ocrs"), to_user); assert_eq!(w.origin, [50.0, 672.0]); assert_eq!(w.across, [100.0, 0.0]); assert_eq!(w.up, [0.0, 20.0]); + assert_eq!(w.source, "ocrs"); + assert_eq!(w.confidence, None); + } + + /// Confidence and source survive placement. + #[test] + fn placed_words_carry_confidence_and_source() { + let to_user = |x: f32, y: f32| [x as f64, y as f64]; + let w = Word::new("Hi", [1.0, 2.0, 3.0, 4.0], "tesseract").with_confidence(87.5); + let p = PlacedWord::place(&w, to_user); + assert_eq!(p.confidence, Some(87.5)); + assert_eq!(p.source, "tesseract"); } /// The text layer is Helvetica in WinAnsiEncoding: every character the model can recognise @@ -267,10 +498,27 @@ mod tests { assert_eq!(pdfcraft_fonts::first_undrawable(ALPHABET), None); } + /// `with_confidence` is clamped to the documented 0–100 scale: negatives (and −inf) are 0, + /// anything over 100 (and +inf) is 100, in-scale values pass through, and NaN means no + /// confidence was reported — the same `None` unknown the unreporting engines carry. + #[test] + fn with_confidence_clamps_and_treats_nan_as_unknown() { + let c = |v: f32| Word::new("w", [0.0; 4], "ocrs").with_confidence(v).confidence; + assert_eq!(c(0.0), Some(0.0)); + assert_eq!(c(50.0), Some(50.0)); + assert_eq!(c(100.0), Some(100.0)); + assert_eq!(c(-1.0), Some(0.0), "negatives clamp to 0"); + assert_eq!(c(f32::NEG_INFINITY), Some(0.0)); + assert_eq!(c(150.0), Some(100.0), "over 100 clamps to 100"); + assert_eq!(c(f32::INFINITY), Some(100.0)); + assert_eq!(c(f32::NAN), None, "NaN is unknown, not a confidence"); + assert_eq!(c(f32::EPSILON), Some(f32::EPSILON), "in-scale values are untouched"); + } + #[test] fn text_layer_is_invisible_text_fitted_to_each_box() { - let w = PlacedWord { text: "Hi".into(), origin: [10.0, 20.0], across: [30.0, 0.0], up: [0.0, 9.3] }; - let s = String::from_utf8(text_layer(&[w])).unwrap(); + let w = PlacedWord { text: "Hi".into(), origin: [10.0, 20.0], across: [30.0, 0.0], up: [0.0, 9.3], confidence: None, source: "ocrs".into() }; + let s = String::from_utf8(text_layer(&[w], &TextPlan::default())).unwrap(); assert!(s.contains("3 Tr") && s.contains("/PCHelv 1 Tf"), "{s}"); let width = helvetica_width("Hi", 1.0); // "Hi" has no descender: its box spans the cap height, from the baseline up. @@ -283,8 +531,8 @@ mod tests { /// after it on the page starts from the graphics state it had before. #[test] fn the_layer_restores_the_graphics_state_it_changes() { - let w = PlacedWord { text: "Hi".into(), origin: [10.0, 20.0], across: [30.0, 0.0], up: [0.0, 9.3] }; - let s = String::from_utf8(text_layer(&[w])).unwrap(); + let w = PlacedWord { text: "Hi".into(), origin: [10.0, 20.0], across: [30.0, 0.0], up: [0.0, 9.3], confidence: None, source: "ocrs".into() }; + let s = String::from_utf8(text_layer(&[w], &TextPlan::default())).unwrap(); let mut depth = 0_i32; for token in s.split_ascii_whitespace() { match token { @@ -309,78 +557,200 @@ mod tests { assert_eq!(ink_extent("Gallus.Ze@camtel.cm"), (CAP_HEIGHT, 0.0)); assert_eq!(ink_extent("Corporation"), (CAP_HEIGHT, -DESCENT)); assert_eq!(ink_extent("--"), (BOX_EM - DESCENT, -DESCENT)); - let w = PlacedWord { text: "gap".into(), origin: [0.0, 100.0], across: [20.0, 0.0], up: [0.0, 7.3] }; - let s = String::from_utf8(text_layer(&[w])).unwrap(); + let w = PlacedWord { text: "gap".into(), origin: [0.0, 100.0], across: [20.0, 0.0], up: [0.0, 7.3], confidence: None, source: "ocrs".into() }; + let s = String::from_utf8(text_layer(&[w], &TextPlan::default())).unwrap(); let em = 7.3 / (X_HEIGHT + DESCENT); assert!(s.contains(&format!(" 0 0 {} 0 {} Tm (gap) Tj", num(em), num(100.0 + em * DESCENT))), "{s}"); } + /// A word whose box is not positive, finite geometry never reaches the content stream: + /// NaN, infinite and zero-sized boxes are dropped, so the layer never carries non-finite + /// numbers. + #[test] + fn text_layer_refuses_non_finite_and_degenerate_boxes() { + let word = |across: [f64; 2], up: [f64; 2], origin: [f64; 2]| PlacedWord { + text: "Hi".into(), + origin, + across, + up, + confidence: Some(90.0), + source: "tesseract".into(), + }; + for (why, w) in [ + ("NaN across", word([f64::NAN, 0.0], [0.0, 9.3], [0.0; 2])), + ("NaN up", word([30.0, 0.0], [f64::NAN, 1.0], [0.0; 2])), + ("inf up", word([30.0, 0.0], [0.0, f64::INFINITY], [0.0; 2])), + ("inf origin", word([30.0, 0.0], [0.0, 9.3], [0.0, f64::INFINITY])), + ("zero across", word([0.0, 0.0], [0.0, 9.3], [0.0; 2])), + ("NaN origin", word([30.0, 0.0], [0.0, 9.3], [0.0, f64::NAN])), + ] { + let s = String::from_utf8(text_layer(&[w], &TextPlan::default())).unwrap(); + assert_eq!(s, "/OCR BMC\nq\nBT\n3 Tr\n/PCHelv 1 Tf\nET\nQ\nEMC\n", "{why}: {s}"); + assert!(!s.contains("NaN") && !s.contains("inf"), "{why}: {s}"); + } + } + + /// The editable-text layer draws visible text whose horizontal scaling is an explicit + /// percentage computed from real Helvetica metrics: `Tz` × natural advance = box width. + #[test] + fn visible_text_layer_scales_by_a_percentage_of_real_metrics() { + let w = PlacedWord { text: "Hi".into(), origin: [10.0, 20.0], across: [40.0, 0.0], up: [0.0, 9.3], confidence: None, source: "ocrs".into() }; + let s = String::from_utf8(visible_text_layer(&[w], &TextPlan::default())).unwrap(); + assert!(s.contains("0 Tr"), "visible, not invisible: {s}"); + let size = 9.3 / BOX_EM; + let natural = helvetica_width("Hi", size); + let tz = 40.0 / natural * 100.0; + assert!(s.contains(&format!("{} Tz", num(tz))), "{s}"); + assert!(s.contains("(Hi) Tj"), "{s}"); + assert!(s.starts_with("/OCR BMC") && s.ends_with("EMC\n")); + } + + /// Degenerate boxes and words with no measurable width are skipped, never divide by zero. + #[test] + fn visible_text_layer_skips_degenerate_words() { + let word = |across: [f64; 2], up: [f64; 2], text: &str| PlacedWord { + text: text.into(), + origin: [0.0; 2], + across, + up, + confidence: None, + source: "ocrs".into(), + }; + let words = vec![ + word([0.0, 0.0], [0.0, 9.0], "flat"), + word([10.0, 0.0], [0.0, 0.0], "short"), + word([f64::NAN, 0.0], [0.0, 9.0], "nan"), + word([10.0, 0.0], [0.0, 9.0], ""), + word([10.0, 0.0], [0.0, 9.0], "Hi"), + ]; + let s = String::from_utf8(visible_text_layer(&words, &TextPlan::default())).unwrap(); + assert_eq!(s.matches("Tj").count(), 1, "only the one drawable word is written: {s}"); + } + + /// A word whose ORIGIN is not finite never reaches the visible layer either: a NaN or + /// infinite origin would put "NaN"/"inf" into the content stream. + #[test] + fn visible_text_layer_refuses_non_finite_origins() { + let word = + |origin: [f64; 2]| PlacedWord { text: "Hi".into(), origin, across: [10.0, 0.0], up: [0.0, 9.0], confidence: None, source: "ocrs".into() }; + for (why, origin) in [("NaN x", [f64::NAN, 100.0]), ("inf y", [10.0, f64::INFINITY]), ("NaN both", [f64::NAN; 2])] { + let s = String::from_utf8(visible_text_layer(&[word(origin)], &TextPlan::default())).unwrap(); + assert!(!s.contains("NaN") && !s.contains("inf"), "{why}: {s}"); + assert_eq!(s.matches("Tj").count(), 0, "{why}: {s}"); + } + } + + /// After a Type3 word the text state goes back to Helvetica: the Latin words around a + /// planned word must not inherit its font. + #[test] + fn helvetica_is_restored_after_a_planned_word() { + if pdfcraft_fonts::CRAFT_FONTS.is_empty() { + eprintln!("skipping: built without craft-fonts (set CRAFT_FONTS_DIR to run it)"); + return; + } + let word = |text: &str| PlacedWord { + text: text.into(), + origin: [10.0, 20.0], + across: [60.0, 0.0], + up: [0.0, 9.3], + confidence: None, + source: "ocrs".into(), + }; + let words = [word("Hi"), word("日本"), word("Ho")]; + let mut plan = TextPlan::for_words(["Hi", "日本", "Ho"]); + for (i, font) in plan.fonts.iter_mut().enumerate() { + font.name = format!("PCOcr{i}"); + } + let s = String::from_utf8(text_layer(&words, &plan)).unwrap(); + assert!(s.contains("/PCOcr0 1 Tf"), "{s}"); + // The last Helvetica selection comes after the Type3 run, and the Latin word after it + // is drawn in Helvetica, not left in the planned font. + assert!(s.rfind("/PCOcr0 1 Tf").unwrap_or(0) < s.rfind("/PCHelv 1 Tf").unwrap_or(usize::MAX), "{s}"); + // The `Q` closes the `q … Q` scope that keeps `3 Tr` from leaking into later content. + assert!(s.ends_with("Tm (Ho) Tj\nET\nQ\nEMC\n"), "{s}"); + // The visible layer restores Helvetica with the word's size. + let v = String::from_utf8(visible_text_layer(&words, &plan)).unwrap(); + assert!(v.rfind("/PCOcr0").unwrap_or(0) < v.rfind("/PCHelv").unwrap_or(usize::MAX), "{v}"); + assert!(v.contains("(Ho) Tj"), "{v}"); + } + + /// Words with characters beyond WinAnsi draw in the plan's Type3 font, one run, with the + /// same exact box fit as the Helvetica words (the matrix x-axis spans across / advance). + #[test] + fn planned_words_draw_in_their_font_with_the_same_exact_fit() { + if pdfcraft_fonts::CRAFT_FONTS.is_empty() { + eprintln!("skipping: built without craft-fonts (set CRAFT_FONTS_DIR to run it)"); + return; + } + let w = + PlacedWord { text: "日本".into(), origin: [10.0, 20.0], across: [60.0, 0.0], up: [0.0, 9.3], confidence: None, source: "ocrs".into() }; + let mut plan = TextPlan::for_words(["日本"]); + // The names the registrar (pdfcraft_edit::add_type3_fonts) fills in — this crate + // cannot reach it, so the test sets them where it reads them. + for (i, font) in plan.fonts.iter_mut().enumerate() { + font.name = format!("PCOcr{i}"); + } + let codes: Vec = plan.encoding(0).iter().map(|&(_, c)| c).collect(); + assert_eq!(codes.len(), 2, "{codes:?}"); + let total = plan.advance_of(plan.encoding(0)); + let s = String::from_utf8(text_layer(std::slice::from_ref(&w), &plan)).unwrap(); + assert!(s.contains("/PCOcr0 1 Tf"), "{s}"); + assert!(s.contains(&format!("{} 0 0 10 10 22.1 Tm ", num(60.0 / total))), "{s}"); + assert!(s.contains(&format!("Tm {} Tj\n", String::from_utf8_lossy(&pdfcraft_fonts::literal(&codes)))), "{s}"); + // The visible layer: size from the box height, Tz from the Type3's own advance. + let v = String::from_utf8(visible_text_layer(std::slice::from_ref(&w), &plan)).unwrap(); + let size = 9.3 / BOX_EM; + assert!(v.contains(&format!("/PCOcr0 {} Tf ", num(size))), "{v}"); + assert!(v.contains(&format!("{} Tz", num(60.0 / (total * size) * 100.0))), "{v}"); + } + + /// Without planned characters (an empty plan: built without craft-fonts, or before the + /// fonts are registered) the WinAnsi path stands, '?' included. + #[test] + fn unplanned_words_fall_back_to_win_ansi_question_marks() { + let w = + PlacedWord { text: "日本".into(), origin: [10.0, 20.0], across: [60.0, 0.0], up: [0.0, 9.3], confidence: None, source: "ocrs".into() }; + let s = String::from_utf8(text_layer(&[w], &TextPlan::default())).unwrap(); + assert!(s.contains("(??) Tj"), "{s}"); + assert!(!s.contains("PCOcr"), "{s}"); + } + + /// A character no face has a glyph for becomes the font's '?', still in the run, counted + /// by the plan. + #[test] + fn substitutions_are_written_as_the_font_s_question_mark() { + if pdfcraft_fonts::CRAFT_FONTS.is_empty() { + eprintln!("skipping: built without craft-fonts (set CRAFT_FONTS_DIR to run it)"); + return; + } + let w = PlacedWord { + text: "日\u{1f4a9}語".into(), + origin: [0.0, 0.0], + across: [60.0, 0.0], + up: [0.0, 9.3], + confidence: None, + source: "ocrs".into(), + }; + let mut plan = TextPlan::for_words(["日\u{1f4a9}語"]); + for (i, font) in plan.fonts.iter_mut().enumerate() { + font.name = format!("PCOcr{i}"); + } + assert_eq!(plan.missing(), 1, "the emoji has no glyph anywhere"); + assert_eq!(plan.encoding(0).len(), 3, "it still gets an encoding: the font's ?"); + let codes: Vec = plan.encoding(0).iter().map(|&(_, c)| c).collect(); + let s = String::from_utf8(text_layer(&[w], &plan)).unwrap(); + assert!(s.contains(&format!("Tm {} Tj\n", String::from_utf8_lossy(&pdfcraft_fonts::literal(&codes)))), "{s}"); + } + #[test] fn rotated_pages_keep_the_reading_direction() { // A page turned 90°: image x runs up the page. let to_user = |x: f32, y: f32| [y as f64, x as f64]; - let w = PlacedWord::place(&Word { text: "Up".into(), rect: [0.0, 0.0, 50.0, 10.0] }, to_user); + let w = PlacedWord::place(&Word::new("Up", [0.0, 0.0, 50.0, 10.0], "ocrs"), to_user); assert_eq!(w.across, [0.0, 50.0]); assert_eq!(w.up, [-10.0, 0.0]); } - /// The current ONNX pair wins when an installation also retains both legacy models. - #[test] - fn model_discovery_prefers_the_complete_onnx_pair() { - let dir = tempfile::tempdir().unwrap(); - for file in ["text-detection.onnx", "text-recognition.onnx"] { - std::fs::write(dir.path().join(file), b"model").unwrap(); - } - let expected = Some(Models { detection: dir.path().join("text-detection.onnx"), recognition: dir.path().join("text-recognition.onnx") }); - assert_eq!(Models::in_dir(dir.path()), expected, "a complete ONNX pair is sufficient"); - for file in ["text-detection.rten", "text-recognition.rten"] { - std::fs::write(dir.path().join(file), b"model").unwrap(); - } - assert_eq!(Models::in_dir(dir.path()), expected, "the complete ONNX pair takes precedence over legacy models"); - } - - /// An interrupted ONNX update still permits an intact legacy pair in the same directory. - #[test] - fn model_discovery_falls_back_to_legacy_when_the_onnx_pair_is_incomplete() { - for current in ["text-detection.onnx", "text-recognition.onnx"] { - let dir = tempfile::tempdir().unwrap(); - for file in [current, "text-detection.rten", "text-recognition.rten"] { - std::fs::write(dir.path().join(file), b"model").unwrap(); - } - assert_eq!( - Models::in_dir(dir.path()), - Some(Models { detection: dir.path().join("text-detection.rten"), recognition: dir.path().join("text-recognition.rten") }), - "{current} alone must not hide a complete legacy pair" - ); - } - } - - /// Detection and recognition must come from one complete format pair, never a mixed update. - #[test] - fn model_discovery_refuses_mixed_format_pairs() { - for pair in [["text-detection.onnx", "text-recognition.rten"], ["text-detection.rten", "text-recognition.onnx"]] { - let dir = tempfile::tempdir().unwrap(); - for file in pair { - std::fs::write(dir.path().join(file), b"model").unwrap(); - } - assert_eq!(Models::in_dir(dir.path()), None, "{pair:?} is not a complete model pair"); - } - } - - /// Existing source builds and PDFCRAFT_MODELS overrides retain their complete RTEN pair. - #[test] - fn model_discovery_keeps_legacy_only_installations() { - let dir = tempfile::tempdir().unwrap(); - assert_eq!(Models::in_dir(dir.path()), None); - std::fs::write(dir.path().join("text-detection.rten"), b"model").unwrap(); - assert_eq!(Models::in_dir(dir.path()), None, "one legacy model is incomplete"); - std::fs::write(dir.path().join("text-recognition.rten"), b"model").unwrap(); - assert_eq!( - Models::in_dir(dir.path()), - Some(Models { detection: dir.path().join("text-detection.rten"), recognition: dir.path().join("text-recognition.rten") }) - ); - } - /// #103: every release package's layout puts the models where the installed app looks. /// Mirrors packaging/{windows/package.ps1,macos/package.sh,linux/package.sh,freebsd/package.sh}. #[test] @@ -443,6 +813,47 @@ mod tests { assert!(word.rect[0] >= 10.0 && word.rect[0] < 60.0, "{word:?}"); } + /// The recognised words name their engine. + #[test] + fn ocrs_recognizer_names_its_words() { + let Ok(rec) = OcrsRecognizer::find() else { + eprintln!("skipped: OCR models not installed"); + return; + }; + assert_eq!(rec.id(), "ocrs"); + assert!(rec.languages().iter().any(|(c, n)| c == "en" && n == "English"), "{:?}", rec.languages()); + let (w, h, px) = test_image::hello(); + let image = OcrImage::new(w, h, px).unwrap(); + let lines = rec.recognize(&image, &RecognizeOptions::default()).unwrap(); + let words: Vec<&Word> = lines.iter().flat_map(|l| &l.words).collect(); + assert!(!words.is_empty()); + assert!(words.iter().all(|w| w.source == "ocrs"), "{words:?}"); + } + + /// Hostile images are rejected with errors, never panics: zero sides, sides over the cap, + /// totals over the area cap (even with legal sides), buffers that do not fit, counts that + /// overflow. + #[test] + fn hostile_image_dimensions_are_rejected_not_panicked_at() { + assert!(matches!(OcrImage::new(0, 10, vec![]), Err(OcrError::EmptyImage))); + assert!(matches!(OcrImage::new(10, 0, vec![]), Err(OcrError::EmptyImage))); + assert!(matches!(OcrImage::new(u32::MAX, u32::MAX, vec![]), Err(OcrError::ImageTooLarge))); + assert!(matches!(OcrImage::new(MAX_SIDE + 1, 1, vec![0; 4]), Err(OcrError::ImageTooLarge))); + // 10 000 × 7 000 = 70 MP: legal on both sides, over the 64 MP area cap (no allocation + // is needed to reject it — the buffer is never inspected). + assert!(matches!(OcrImage::new(MAX_SIDE, 7000, vec![]), Err(OcrError::ImageTooLarge))); + // 8 000 × 8 000 = exactly the area cap: accepted shape (the short buffer proves the + // cap check passed), while one more row is refused. + assert!(matches!(OcrImage::new(8000, 8000, vec![0; 16]), Err(OcrError::ImageSize { .. }))); + assert!(matches!(OcrImage::new(8000, 8001, vec![0; 16]), Err(OcrError::ImageTooLarge))); + assert!(matches!(OcrImage::new(2, 2, vec![0; 15]), Err(OcrError::ImageSize { .. }))); + let ok = OcrImage::new(2, 2, vec![255; 16]).unwrap(); + assert_eq!((ok.width(), ok.height()), (2, 2)); + assert_eq!(ok.rgba().len(), 16); + assert_eq!(ok.gray(5, 5), Some(255), "clamped reads stay inside"); + assert_eq!(ok.gray(-1, -1), Some(255)); + } + mod test_image { /// "HELLO WORLD" drawn with blocky 5×7 letters, 4 pixels per dot, black on white. pub fn hello() -> (u32, u32, Vec) { diff --git a/crates/ocr/src/merge.rs b/crates/ocr/src/merge.rs new file mode 100644 index 0000000000..58fc17a4e5 --- /dev/null +++ b/crates/ocr/src/merge.rs @@ -0,0 +1,583 @@ +//! Combining engines. "ROVER" merges two engines' readings of one page: every primary word +//! adopts the best-overlapping unused secondary word when their boxes agree enough (IoU, below); +//! the higher confidence wins the text, box and confidence (a tie keeps the primary; against a +//! primary of unknown confidence the secondary must reach [`UNKNOWN_CONFIDENCE_BAR`]) and the +//! surviving word is named `"ROVER"`; secondary words nothing claimed are appended, keeping +//! their own engine's name. + +use crate::{Line, OcrError, OcrImage, RecognizeOptions, Recognizer, Word}; + +/// How the chosen engines combine. The default is [`MergeStrategy::PrimaryOnly`] with the +/// default engine (ocrs), which is today's behavior. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash)] +pub enum MergeStrategy { + /// Only the primary engine runs. + #[default] + PrimaryOnly, + /// The secondary engine joins only when the primary's reading looks weak: its mean word + /// confidence is under 70, or its confidence is unknown (words were read but none reports + /// a confidence — the ocrs engine). An unknown reading is never counted as 100; an empty + /// primary keeps the fast path (no words, nothing to double-check). + ConfidenceWeighted, + /// Both engines always run and their readings merge (with no secondary engine this + /// degrades to primary only). + RoverVote, +} + +/// The mean of the confidences the words report, 0–100; `None` when the reading's confidence +/// is UNKNOWN — words were read but none reports a confidence (the ocrs engine's words). +/// Unknown is never counted as 100: there is no low-confidence signal, but a reading whose +/// confidence nobody knows is not therefore confident, so a confidence-weighted strategy +/// answers it by asking the secondary engine. An empty reading is `Some(100.0)`: no words +/// were read, so there is nothing to double-check and the fast path stands. +pub fn mean_confidence(lines: &[Line]) -> Option { + let mut total = 0usize; + let mut reported = (0.0f64, 0usize); + for word in lines.iter().flat_map(|l| &l.words) { + total += 1; + if let Some(c) = word.confidence { + reported.0 += c as f64; + reported.1 += 1; + } + } + match (total, reported.1) { + (0, _) => Some(100.0), + (_, 0) => None, + (_, n) => Some((reported.0 / n as f64) as f32), + } +} + +/// Intersection over union of two `[left, top, right, bottom]` boxes. An empty intersection is +/// 0; degenerate or NaN boxes give 0 rather than panicking or poisoning the comparison. +pub fn iou(a: &[f32; 4], b: &[f32; 4]) -> f64 { + let l = a[0].max(b[0]); + let t = a[1].max(b[1]); + let r = a[2].min(b[2]); + let btm = a[3].min(b[3]); + if !(r > l && btm > t) { + return 0.0; // disjoint, touching, inverted, or NaN (all comparisons false) + } + let inter = (r - l) as f64 * (btm - t) as f64; + let area = |x: &[f32; 4]| ((x[2] - x[0]).max(0.0) as f64) * ((x[3] - x[1]).max(0.0) as f64); + let union = area(a) + area(b) - inter; + if union <= 0.0 { + return 0.0; + } + inter / union +} + +/// The most primary×secondary word comparisons one ROVER merge will attempt. Known limit: a +/// merge over this cap is skipped — the primary reading stands alone, and the skip is noted, +/// never silent — so two enormous readings cannot turn the merge into quadratic work without +/// end. A real page is a few hundred words on each side, orders of magnitude under the cap. +pub const ROVER_MAX_COMPARISONS: usize = 100_000_000; + +/// Whether the two readings are too large to merge: their word counts multiply past +/// [`ROVER_MAX_COMPARISONS`]. Shared by [`rover_merge`] (which then keeps the primary) and +/// [`recognize_with_strategy`] (which then notes the skip). +fn over_cap(primary: &[Line], secondary: &[Line]) -> bool { + let primary_words: usize = primary.iter().map(|l| l.words.len()).sum(); + let secondary_words: usize = secondary.iter().map(|l| l.words.len()).sum(); + primary_words.saturating_mul(secondary_words) > ROVER_MAX_COMPARISONS +} + +/// The confidence a secondary word needs to replace a primary word whose confidence is +/// unknown (the ocrs engine reports none). It is the bar under which a reading counts as weak +/// ([`MergeStrategy::ConfidenceWeighted`] asks the second engine below it): a word the second +/// engine itself calls weak does not displace a reading nobody scored. +pub const UNKNOWN_CONFIDENCE_BAR: f32 = 70.0; + +/// True when the secondary word's confidence `a` beats the primary's `b`. Both known: the +/// strictly higher one wins (a tie keeps the primary). A known confidence against an unknown +/// primary wins only at or above [`UNKNOWN_CONFIDENCE_BAR`]: an unknown confidence is neither +/// 0 nor 100, so a 5% reading must not replace it, while a confident one may. NaN never wins, +/// and with no confidence on the secondary side the primary stands. +fn higher(a: Option, b: Option) -> bool { + match (a, b) { + (Some(a), Some(b)) => a.partial_cmp(&b) == Some(std::cmp::Ordering::Greater), + (Some(a), None) => a >= UNKNOWN_CONFIDENCE_BAR, + _ => false, + } +} + +/// Merge the two engines' readings. Primary words keep their order and lines; a word is +/// replaced by its overlapping secondary (IoU >= 0.5) only when that secondary is more +/// confident, and either way the merged word's [`Word::source`] is `"ROVER"`. Secondary words +/// no primary claimed are appended, in their own reading order and lines, under their own +/// engine's name (`secondary_id`). Known limit: over [`ROVER_MAX_COMPARISONS`] comparisons +/// (word counts multiplied) the primary reading is kept as is, unmerged; the strategy runner +/// names that skip in a note. +pub fn rover_merge(primary: Vec, secondary: Vec, secondary_id: &str) -> Vec { + if over_cap(&primary, &secondary) { + return primary; // over the comparison cap: the primary stands alone + } + let mut primary = primary; + // The secondary's words, marked `None` once a primary word claimed them, grouped by their + // own lines so the unclaimed ones can be appended in order. + let mut unused: Vec>> = secondary.into_iter().map(|l| l.words.into_iter().map(Some).collect()).collect(); + for line in &mut primary { + for word in &mut line.words { + // The best-overlapping unused secondary word, if any beats IoU 0.5. + let mut best: Option<(f64, usize, usize)> = None; + for (li, words) in unused.iter().enumerate() { + for (wi, slot) in words.iter().enumerate() { + let Some(w) = slot else { continue }; + let score = iou(&word.rect, &w.rect); + if score >= 0.5 && best.is_none_or(|(b, _, _)| score > b) { + best = Some((score, li, wi)); + } + } + } + let Some((_, li, wi)) = best else { continue }; + let claimed = unused.get_mut(li).and_then(|l| l.get_mut(wi)).and_then(|slot| slot.take()); + let Some(claim) = claimed else { continue }; + if higher(claim.confidence, word.confidence) { + *word = Word { source: "ROVER".into(), ..claim }; + } else { + // Tie (or the primary wins): keep the primary's text, box and confidence, but + // record that this reading is the engines' agreed one. + word.source = "ROVER".into(); + } + } + } + // Append the secondary words nothing claimed, line by line, under their engine's name + // (`secondary_id` backfills words that name no source of their own). + let mut merged = primary; + for line in unused { + let words: Vec = line + .into_iter() + .flatten() + .map(|mut w| { + if w.source.is_empty() { + w.source = secondary_id.into(); + } + w + }) + .collect(); + if !words.is_empty() { + merged.push(Line { words }); + } + } + merged +} + +/// What a strategy run produced: the merged (or primary-only) reading, whether the secondary +/// engine's reading is part of it, and the notes recorded along the way. +#[derive(Clone, Debug, PartialEq, Default)] +pub struct StrategyOutcome { + pub lines: Vec, + /// Whether the secondary engine's reading is part of `lines`: true only when the merge + /// actually happened. A secondary that failed, or whose reading was discarded over the + /// comparison cap, is `false`, with a note in [`StrategyOutcome::notes`] saying why. + pub secondary_ran: bool, + /// Why the page's reading is partial, if it is. The secondary engine is an enhancement, + /// never a gate: when it is configured but fails, the primary reading is kept and the + /// failure is noted here — a page is never skipped because a partner could not read it. + /// Only a primary failure is an [`OcrError`]. + pub notes: Vec, +} + +/// The note recorded when an ensemble partner fails: the page keeps the primary reading and +/// the cause is named for the report. +fn degraded_note(id: &str, error: &OcrError) -> String { + format!("the {id} engine failed ({error}) — the page keeps the primary reading alone") +} + +/// The note recorded when the two readings are too large to merge: the page keeps the primary +/// reading alone and the discarded partner reading is named, never silent. +fn cap_note(id: &str, primary_words: usize, secondary_words: usize) -> String { + format!( + "the readings were too large to merge ({primary_words} × {secondary_words} comparisons, over the cap of {ROVER_MAX_COMPARISONS}) — the page keeps the primary reading alone and the {id} reading is unused" + ) +} + +/// Run the engines per `strategy` and merge. See [`StrategyOutcome`] for what comes back. +pub fn recognize_with_strategy( + primary: &dyn Recognizer, + secondary: Option<&dyn Recognizer>, + strategy: MergeStrategy, + image: &OcrImage, + options: &RecognizeOptions, +) -> Result { + // The shared shape of the ensemble strategies: the primary has read, the secondary joins + // or fails (a failure degrades to the primary, with a note). + let join = |p: Vec, sec: &dyn Recognizer| match sec.recognize(image, options) { + // Over the comparison cap the merge is skipped: the secondary's reading was read but + // is not part of the result, so the outcome says so — primary only, honestly not + // merged, with the skip named. + Ok(s) if over_cap(&p, &s) => { + let (primary_words, secondary_words) = (p.iter().map(|l| l.words.len()).sum::(), s.iter().map(|l| l.words.len()).sum::()); + StrategyOutcome { lines: p, secondary_ran: false, notes: vec![cap_note(sec.id(), primary_words, secondary_words)] } + } + Ok(s) => StrategyOutcome { lines: rover_merge(p, s, sec.id()), secondary_ran: true, notes: Vec::new() }, + Err(e) => StrategyOutcome { lines: p, secondary_ran: false, notes: vec![degraded_note(sec.id(), &e)] }, + }; + match strategy { + MergeStrategy::PrimaryOnly => Ok(StrategyOutcome { lines: primary.recognize(image, options)?, secondary_ran: false, notes: Vec::new() }), + MergeStrategy::RoverVote => match secondary { + None => Ok(StrategyOutcome { lines: primary.recognize(image, options)?, secondary_ran: false, notes: Vec::new() }), + Some(sec) => { + let p = primary.recognize(image, options)?; + Ok(join(p, sec)) + } + }, + MergeStrategy::ConfidenceWeighted => { + let p = primary.recognize(image, options)?; + match mean_confidence(&p) { + // Confident enough — or an empty reading: the primary stands alone. + Some(mean) if mean >= UNKNOWN_CONFIDENCE_BAR => Ok(StrategyOutcome { lines: p, secondary_ran: false, notes: Vec::new() }), + // A weak mean, or an unknown confidence (words, none reported): the secondary + // runs. + _ => match secondary { + None => Ok(StrategyOutcome { lines: p, secondary_ran: false, notes: Vec::new() }), + Some(sec) => Ok(join(p, sec)), + }, + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use std::sync::atomic::{AtomicUsize, Ordering}; + + /// A fake engine: returns canned lines, counts recognize calls, never available. + struct Fake { + id: &'static str, + lines: Vec, + calls: AtomicUsize, + fail: bool, + } + + impl Fake { + fn new(id: &'static str, words: &[(&str, [f32; 4], f32)]) -> Fake { + Fake { + id, + lines: vec![Line { words: words.iter().map(|(t, r, c)| Word::new(*t, *r, id).with_confidence(*c)).collect() }], + calls: AtomicUsize::new(0), + fail: false, + } + } + + fn empty(id: &'static str) -> Fake { + Fake { id, lines: Vec::new(), calls: AtomicUsize::new(0), fail: false } + } + + /// An engine like ocrs: words, but no confidence reported on any of them. + fn unknown(id: &'static str, n: usize) -> Fake { + Fake { + id, + lines: vec![Line { + words: (0..n).map(|i| Word::new(format!("w{i}"), [i as f32 * 10.0, 0.0, i as f32 * 10.0 + 8.0, 10.0], id)).collect(), + }], + calls: AtomicUsize::new(0), + fail: false, + } + } + } + + impl Recognizer for Fake { + fn id(&self) -> &'static str { + self.id + } + fn available(&self) -> bool { + !self.fail + } + fn languages(&self) -> Vec<(String, String)> { + vec![] + } + fn recognize(&self, _image: &OcrImage, _options: &RecognizeOptions) -> Result, OcrError> { + self.calls.fetch_add(1, Ordering::SeqCst); + if self.fail { + return Err(OcrError::Recognize("the fake engine failed".into())); + } + Ok(self.lines.clone()) + } + } + + fn image() -> OcrImage { + OcrImage::new(4, 4, vec![255; 64]).unwrap() + } + + fn words(lines: &[Line]) -> Vec<&Word> { + lines.iter().flat_map(|l| &l.words).collect() + } + + #[test] + fn iou_of_disjoint_touching_and_nan_boxes_is_zero() { + let a = [0.0, 0.0, 10.0, 10.0]; + assert_eq!(iou(&a, &[20.0, 0.0, 30.0, 10.0]), 0.0, "disjoint"); + assert_eq!(iou(&a, &[10.0, 0.0, 20.0, 10.0]), 0.0, "touching edges are an empty intersection"); + assert_eq!(iou(&a, &[f32::NAN, 0.0, 10.0, 10.0]), 0.0, "NaN boxes score 0, not a panic"); + assert_eq!(iou(&a, &a), 1.0, "identical boxes"); + assert!((iou(&a, &[5.0, 0.0, 15.0, 10.0]) - 1.0 / 3.0).abs() < 1e-9, "half overlap is IoU 1/3"); + } + + /// With nothing on the primary side, every secondary word is appended under the secondary + /// engine's own name. + #[test] + fn rover_appends_everything_when_the_primary_is_empty() { + let primary = Vec::new(); + let secondary = vec![Line { words: vec![Word::new("sole", [0.0, 0.0, 9.0, 9.0], "tesseract").with_confidence(80.0)] }]; + let merged = rover_merge(primary, secondary, "tesseract"); + assert_eq!(words(&merged).len(), 1); + assert_eq!(merged[0].words[0].text, "sole"); + assert_eq!(merged[0].words[0].source, "tesseract", "unclaimed words keep their own engine"); + } + + /// Non-overlapping secondary words never displace primary words; they are appended. + #[test] + fn rover_appends_non_overlapping_secondary_words() { + let primary = vec![Line { words: vec![Word::new("kept", [0.0, 0.0, 10.0, 10.0], "ocrs").with_confidence(50.0)] }]; + let secondary = vec![Line { words: vec![Word::new("extra", [100.0, 100.0, 120.0, 110.0], "tesseract").with_confidence(99.0)] }]; + let merged = rover_merge(primary, secondary, "tesseract"); + let w = words(&merged); + assert_eq!(w.len(), 2); + assert_eq!((w[0].text.as_str(), w[0].source.as_str()), ("kept", "ocrs"), "the more confident but disjoint word does not win"); + assert_eq!((w[1].text.as_str(), w[1].source.as_str()), ("extra", "tesseract")); + } + + /// The higher-confidence word of an overlapping pair wins text, box and confidence, and is + /// named ROVER. + #[test] + fn rover_higher_confidence_wins_and_is_named_rover() { + let primary = vec![Line { words: vec![Word::new("helo", [0.0, 0.0, 10.0, 10.0], "ocrs").with_confidence(60.0)] }]; + let secondary = vec![Line { words: vec![Word::new("hello", [1.0, 0.0, 11.0, 10.0], "tesseract").with_confidence(95.0)] }]; + let merged = rover_merge(primary, secondary, "tesseract"); + let w = words(&merged); + assert_eq!(w.len(), 1, "the secondary word was claimed, not appended"); + assert_eq!(w[0].text, "hello"); + assert_eq!(w[0].rect, [1.0, 0.0, 11.0, 10.0]); + assert_eq!(w[0].confidence, Some(95.0)); + assert_eq!(w[0].source, "ROVER"); + } + + /// A tie keeps the primary word (still named ROVER, as an agreed reading). + #[test] + fn rover_tie_keeps_the_primary() { + let primary = vec![Line { words: vec![Word::new("primary", [0.0, 0.0, 10.0, 10.0], "ocrs").with_confidence(80.0)] }]; + let secondary = vec![Line { words: vec![Word::new("secondary", [0.0, 0.0, 10.0, 10.0], "tesseract").with_confidence(80.0)] }]; + let merged = rover_merge(primary, secondary, "tesseract"); + let w = words(&merged); + assert_eq!(w.len(), 1); + assert_eq!(w[0].text, "primary"); + assert_eq!(w[0].source, "ROVER"); + } + + /// Boxes under the 0.5 IoU bar never merge, even when the secondary is more confident. + #[test] + fn rover_below_half_iou_does_not_merge() { + let primary = vec![Line { words: vec![Word::new("kept", [0.0, 0.0, 10.0, 10.0], "ocrs").with_confidence(50.0)] }]; + // IoU 10·10 / (100 + 169 - 25) = 100/244 ≈ 0.41 < 0.5. + let secondary = vec![Line { words: vec![Word::new("other", [5.0, 0.0, 23.0, 20.0], "tesseract").with_confidence(99.0)] }]; + let merged = rover_merge(primary, secondary, "tesseract"); + let w = words(&merged); + assert_eq!(w.len(), 2); + assert_eq!(w[0].text, "kept"); + assert_eq!(w[0].source, "ocrs"); + } + + /// Merged words keep their line; the merge never panics on NaN boxes. + #[test] + fn rover_survives_nan_boxes() { + let primary = vec![Line { words: vec![Word::new("nan", [f32::NAN, 0.0, 10.0, 10.0], "ocrs").with_confidence(50.0)] }]; + let secondary = vec![Line { words: vec![Word::new("real", [0.0, 0.0, 10.0, 10.0], "tesseract").with_confidence(90.0)] }]; + let merged = rover_merge(primary, secondary, "tesseract"); + let w = words(&merged); + assert_eq!(w.len(), 2, "a NaN primary box merges with nothing"); + assert_eq!(w[0].text, "nan"); + assert_eq!(w[1].text, "real"); + } + + /// Only reported confidences count. An empty reading is Some(100) (nothing to + /// double-check); words without a single confidence are UNKNOWN (`None`), never 100. + #[test] + fn mean_confidence_distinguishes_empty_from_unknown() { + assert_eq!(mean_confidence(&[]), Some(100.0)); + assert_eq!(mean_confidence(&[Line { words: vec![Word::new("a", [0.0; 4], "ocrs")] }]), None, "words, no confidences: unknown"); + let lines = vec![Line { + words: vec![ + Word::new("a", [0.0; 4], "e").with_confidence(40.0), + Word::new("b", [0.0; 4], "e").with_confidence(80.0), + Word::new("c", [0.0; 4], "e"), + ], + }]; + assert_eq!(mean_confidence(&lines), Some(60.0), "reported confidences are averaged"); + } + + /// ROVER's preference needs a meaningful comparison: a known confidence beats an unknown + /// one (the ocrs primary yields to a confident tesseract reading), and None vs None keeps + /// the primary. + #[test] + fn rover_known_confidence_beats_unknown_and_none_vs_none_keeps_the_primary() { + let primary = vec![Line { words: vec![Word::new("helo", [0.0, 0.0, 10.0, 10.0], "ocrs")] }]; + let secondary = vec![Line { words: vec![Word::new("hello", [0.0, 0.0, 10.0, 10.0], "tesseract").with_confidence(95.0)] }]; + let merged = rover_merge(primary, secondary, "tesseract"); + assert_eq!( + (merged[0].words[0].text.as_str(), merged[0].words[0].confidence, merged[0].words[0].source.as_str()), + ("hello", Some(95.0), "ROVER") + ); + let primary = vec![Line { words: vec![Word::new("primary", [0.0, 0.0, 10.0, 10.0], "ocrs")] }]; + let secondary = vec![Line { words: vec![Word::new("secondary", [0.0, 0.0, 10.0, 10.0], "tesseract")] }]; + let merged = rover_merge(primary, secondary, "tesseract"); + assert_eq!( + (merged[0].words[0].text.as_str(), merged[0].words[0].source.as_str()), + ("primary", "ROVER"), + "unknown vs unknown is not a comparison: the primary stands" + ); + } + + /// Over the comparison cap the merge is skipped and the primary reading stands alone (a + /// documented known limit, bounding the merge's quadratic core). + #[test] + fn rover_over_the_comparison_cap_keeps_the_primary() { + // 11 000 × 11 000 = 121 M comparisons > the 100 M cap. + let words = |id: &str| (0..11_000).map(|i| Word::new(format!("w{i}"), [i as f32, 0.0, i as f32 + 5.0, 10.0], id)).collect::>(); + let primary = vec![Line { words: words("ocrs") }]; + let secondary = vec![Line { words: words("tesseract") }]; + let merged = rover_merge(primary, secondary, "tesseract"); + assert_eq!(merged.len(), 1); + assert!(merged[0].words.iter().all(|w| w.source == "ocrs"), "every primary word stands, unmerged"); + assert_eq!(merged[0].words.len(), 11_000); + // Under the cap, merging still happens (a small overlap is replaced and named ROVER). + let small_primary = vec![Line { words: vec![Word::new("kept", [0.0, 0.0, 10.0, 10.0], "ocrs").with_confidence(50.0)] }]; + let small_secondary = vec![Line { words: vec![Word::new("wins", [0.0, 0.0, 10.0, 10.0], "tesseract").with_confidence(95.0)] }]; + let merged = rover_merge(small_primary, small_secondary, "tesseract"); + assert_eq!((merged[0].words[0].text.as_str(), merged[0].words[0].source.as_str()), ("wins", "ROVER")); + } + + /// Over the comparison cap the merge is skipped honestly at the strategy level: the page + /// keeps the primary reading alone, the skip is named in a note, and `secondary_ran` does + /// not claim the discarded reading is part of the result (though the secondary really did + /// run — its work was spent, its reading unused). + #[test] + fn rover_cap_hit_reports_primary_only_with_a_note_and_an_honest_flag() { + let mut primary = Fake::empty("ocrs"); + primary.lines = + vec![Line { words: (0..11_000).map(|i| Word::new(format!("w{i}"), [i as f32, 0.0, i as f32 + 5.0, 10.0], "ocrs")).collect() }]; + let mut secondary = Fake::empty("tesseract"); + secondary.lines = + vec![Line { words: (0..11_000).map(|i| Word::new(format!("w{i}"), [i as f32, 0.0, i as f32 + 5.0, 10.0], "tesseract")).collect() }]; + let outcome = recognize_with_strategy(&primary, Some(&secondary), MergeStrategy::RoverVote, &image(), &RecognizeOptions::default()).unwrap(); + assert_eq!(secondary.calls.load(Ordering::SeqCst), 1, "the secondary really ran"); + assert_eq!(outcome.lines.len(), 1); + assert_eq!(outcome.lines[0].words.len(), 11_000, "primary only"); + assert!(outcome.lines[0].words.iter().all(|w| w.source == "ocrs"), "every primary word stands, unmerged"); + assert!(!outcome.secondary_ran, "the secondary's reading is not part of the result"); + assert_eq!(outcome.notes.len(), 1, "{:?}", outcome.notes); + assert!(outcome.notes[0].contains("merge"), "{:?}", outcome.notes[0]); + assert!(outcome.notes[0].contains("tesseract"), "{:?}", outcome.notes[0]); + // Under the cap the same strategy merges and reports the secondary honestly. + let primary = Fake::new("ocrs", &[("kept", [0.0, 0.0, 10.0, 10.0], 50.0)]); + let secondary = Fake::new("tesseract", &[("wins", [0.0, 0.0, 10.0, 10.0], 95.0)]); + let outcome = recognize_with_strategy(&primary, Some(&secondary), MergeStrategy::RoverVote, &image(), &RecognizeOptions::default()).unwrap(); + assert!(outcome.secondary_ran && outcome.notes.is_empty()); + assert_eq!((outcome.lines[0].words[0].text.as_str(), outcome.lines[0].words[0].source.as_str()), ("wins", "ROVER")); + } + + #[test] + fn primary_only_never_runs_the_secondary() { + let primary = Fake::new("ocrs", &[("w", [0.0; 4], 90.0)]); + let secondary = Fake::empty("tesseract"); + let outcome = + recognize_with_strategy(&primary, Some(&secondary), MergeStrategy::PrimaryOnly, &image(), &RecognizeOptions::default()).unwrap(); + assert_eq!(primary.calls.load(Ordering::SeqCst), 1); + assert_eq!(secondary.calls.load(Ordering::SeqCst), 0); + assert!(!outcome.secondary_ran); + assert!(outcome.notes.is_empty()); + assert_eq!(words(&outcome.lines).len(), 1); + } + + #[test] + fn confidence_weighted_runs_the_secondary_only_on_a_weak_primary() { + let weak = Fake::new("ocrs", &[("w", [0.0; 4], 40.0)]); + let strong = Fake::new("ocrs", &[("w", [0.0; 4], 95.0)]); + let secondary = Fake::empty("tesseract"); + let outcome = + recognize_with_strategy(&weak, Some(&secondary), MergeStrategy::ConfidenceWeighted, &image(), &RecognizeOptions::default()).unwrap(); + assert!(outcome.secondary_ran, "mean 40 < 70 → the secondary joins"); + let outcome = + recognize_with_strategy(&strong, Some(&secondary), MergeStrategy::ConfidenceWeighted, &image(), &RecognizeOptions::default()).unwrap(); + assert!(!outcome.secondary_ran, "mean 95 >= 70 → primary alone"); + // An empty primary counts as 100: no signal, no secondary run. + let empty = Fake::empty("ocrs"); + let outcome = + recognize_with_strategy(&empty, Some(&secondary), MergeStrategy::ConfidenceWeighted, &image(), &RecognizeOptions::default()).unwrap(); + assert!(!outcome.secondary_ran); + assert_eq!(secondary.calls.load(Ordering::SeqCst), 1, "only the weak page ran the secondary"); + } + + /// The ocrs engine reports no confidences: unknown is not 100, so the secondary runs — + /// and its known reading wins the overlapping word (the ensemble's whole point). + #[test] + fn confidence_weighted_runs_the_secondary_on_an_unknown_primary() { + let ocrs_like = Fake::unknown("ocrs", 1); + let secondary = Fake::new("tesseract", &[("hello", [0.0, 0.0, 10.0, 10.0], 95.0)]); + let outcome = + recognize_with_strategy(&ocrs_like, Some(&secondary), MergeStrategy::ConfidenceWeighted, &image(), &RecognizeOptions::default()).unwrap(); + assert!(outcome.secondary_ran, "an unknown confidence asks the secondary"); + assert_eq!((outcome.lines[0].words[0].text.as_str(), outcome.lines[0].words[0].confidence), ("hello", Some(95.0))); + assert_eq!(outcome.lines[0].words[0].source, "ROVER"); + assert_eq!(secondary.calls.load(Ordering::SeqCst), 1); + } + + /// Against a primary word of unknown confidence, a secondary word wins only when it is + /// confident (at or above the weak-reading bar): a 5% reading does not replace a word nobody + /// scored, a 70% one does. The merged word is named ROVER either way. + #[test] + fn an_unknown_primary_word_yields_only_to_a_confident_secondary() { + let merge = |conf: f32| { + let secondary = vec![Line { words: vec![Word::new("other", [0.0, 0.0, 10.0, 10.0], "tesseract").with_confidence(conf)] }]; + let primary = Fake::unknown("ocrs", 1).lines; + rover_merge(primary, secondary, "tesseract").remove(0).words.remove(0) + }; + let weak = merge(5.0); + assert_eq!((weak.text.as_str(), weak.confidence, weak.source.as_str()), ("w0", None, "ROVER"), "a 5% word keeps the primary"); + let below = merge(UNKNOWN_CONFIDENCE_BAR - 0.5); + assert_eq!(below.text, "w0"); + let at_bar = merge(UNKNOWN_CONFIDENCE_BAR); + assert_eq!((at_bar.text.as_str(), at_bar.confidence), ("other", Some(UNKNOWN_CONFIDENCE_BAR)), "a confident word wins"); + let nan = merge(f32::NAN); + assert_eq!(nan.text, "w0", "NaN (unknown after clamping) never wins"); + } + + #[test] + fn rover_vote_always_runs_both_and_degrades_without_a_secondary() { + let primary = Fake::new("ocrs", &[("w", [0.0; 4], 95.0)]); + let secondary = Fake::empty("tesseract"); + let outcome = recognize_with_strategy(&primary, Some(&secondary), MergeStrategy::RoverVote, &image(), &RecognizeOptions::default()).unwrap(); + assert!(outcome.secondary_ran); + assert_eq!(secondary.calls.load(Ordering::SeqCst), 1); + // No secondary: primary only. + let outcome = recognize_with_strategy(&primary, None, MergeStrategy::RoverVote, &image(), &RecognizeOptions::default()).unwrap(); + assert!(!outcome.secondary_ran); + } + + /// A configured secondary that fails degrades to the primary reading, with the failure + /// recorded as a note — never a skipped page, never a silently partial reading without a + /// trace. Both ensemble strategies degrade; a primary failure is still an error. + #[test] + fn a_failing_secondary_degrades_to_the_primary_with_a_note() { + let mut secondary = Fake::empty("tesseract"); + secondary.fail = true; + let primary = Fake::new("ocrs", &[("w", [0.0; 4], 95.0)]); + let outcome = recognize_with_strategy(&primary, Some(&secondary), MergeStrategy::RoverVote, &image(), &RecognizeOptions::default()).unwrap(); + assert_eq!(words(&outcome.lines).len(), 1, "the primary reading stands"); + assert_eq!(outcome.lines[0].words[0].text, "w"); + assert!(!outcome.secondary_ran); + assert_eq!(outcome.notes.len(), 1); + assert!(outcome.notes[0].contains("tesseract"), "{:?}", outcome.notes[0]); + // The confidence-weighted strategy degrades the same way on its weak path. + let weak = Fake::new("ocrs", &[("w", [0.0; 4], 40.0)]); + let outcome = + recognize_with_strategy(&weak, Some(&secondary), MergeStrategy::ConfidenceWeighted, &image(), &RecognizeOptions::default()).unwrap(); + assert_eq!(words(&outcome.lines).len(), 1); + assert_eq!(outcome.notes.len(), 1, "{:?}", outcome.notes); + // A failing primary is still an error: there is nothing to degrade to. + let mut failing_primary = Fake::new("ocrs", &[("w", [0.0; 4], 95.0)]); + failing_primary.fail = true; + let r = recognize_with_strategy(&failing_primary, Some(&secondary), MergeStrategy::RoverVote, &image(), &RecognizeOptions::default()); + assert!(r.is_err()); + } +} diff --git a/crates/ocr/src/ocrs.rs b/crates/ocr/src/ocrs.rs new file mode 100644 index 0000000000..7f8feb47c4 --- /dev/null +++ b/crates/ocr/src/ocrs.rs @@ -0,0 +1,251 @@ +//! The pure-Rust engine: the ocrs crate with its pre-trained models, wrapped as a +//! [`Recognizer`] (`OcrsRecognizer`). Reading is offline and local; the models do not report a +//! confidence, so its words have `confidence: None`. + +use std::path::{Path, PathBuf}; + +use super::{Line, MAX_PIXELS, MAX_SIDE, OcrError, OcrImage, RecognizeOptions, Recognizer, Word}; + +/// The validated byte length of a `width` × `height` RGBA raster, with the same caps +/// [`OcrImage::new`] applies: zero sides are empty, sides over [`MAX_SIDE`] or totals over +/// [`MAX_PIXELS`] are too large. `Ocr::recognize` takes raw dimensions (they come from +/// callers, ultimately from documents, so they are untrusted), and the naive +/// `width * height * 4` overflows `u32` for absurd values — the caps are checked first, in +/// `usize`, so nothing overflows and nothing huge is ever touched. +fn checked_len(width: u32, height: u32) -> Result { + if width == 0 || height == 0 { + return Err(OcrError::EmptyImage); + } + if width > MAX_SIDE || height > MAX_SIDE { + return Err(OcrError::ImageTooLarge); + } + let pixels = (width as usize).checked_mul(height as usize).ok_or(OcrError::ImageTooLarge)?; + if pixels > MAX_PIXELS as usize { + return Err(OcrError::ImageTooLarge); + } + pixels.checked_mul(4).ok_or(OcrError::ImageTooLarge) +} + +/// The languages the models read (ISO 639-1); all use the Latin alphabet without accents. +pub const LANGUAGES: &[(&str, &str)] = &[("en", "English")]; + +/// The model files, as named in ATTRIBUTION.toml. +pub const DETECTION_MODEL: &str = "text-detection.onnx"; +pub const RECOGNITION_MODEL: &str = "text-recognition.onnx"; + +pub const ALPHABET: &str = " 0123456789!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~€ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"; + +/// Where the two model files are. +#[derive(Clone, Debug, PartialEq)] +pub struct Models { + pub detection: PathBuf, + pub recognition: PathBuf, +} + +impl Models { + pub fn dirs_beside_exe(exe_dir: &Path) -> Vec { + vec![exe_dir.join("models"), exe_dir.join("../Resources/models"), exe_dir.join("../share/pdfcraft/models")] + } + + /// The models in `dir`, preferring current ONNX files over legacy RTEN files. An incomplete + /// update never mixes formats or hides an intact legacy pair. + pub fn in_dir(dir: &Path) -> Option { + [(DETECTION_MODEL, RECOGNITION_MODEL), ("text-detection.rten", "text-recognition.rten")].into_iter().find_map(|(detection, recognition)| { + let m = Models { detection: dir.join(detection), recognition: dir.join(recognition) }; + (m.detection.is_file() && m.recognition.is_file()).then_some(m) + }) + } + + /// Look for the models: `$PDFCRAFT_MODELS`, then `models/` beside the executable (and + /// `Resources/models` in a macOS bundle), then the source tree's `assets/models/`. + pub fn find() -> Option { + Self::search_dirs().iter().find_map(|d| Self::in_dir(d)) + } + + /// The directories [`Models::find`] looks in, in order. + pub fn search_dirs() -> Vec { + let mut dirs = Vec::new(); + if let Some(d) = std::env::var_os("PDFCRAFT_MODELS") { + dirs.push(PathBuf::from(d)); + } + if let Some(exe) = std::env::current_exe().ok().and_then(|e| e.parent().map(Path::to_path_buf)) { + dirs.extend(Self::dirs_beside_exe(&exe)); + } + dirs.push(Path::new(env!("CARGO_MANIFEST_DIR")).join("../../assets/models")); + dirs + } +} + +/// A loaded recogniser. Loading takes a moment; keep one and reuse it. +pub struct Ocr { + engine: ocrs::OcrEngine, +} + +impl Ocr { + pub fn load(models: &Models) -> Result { + let load = |p: &Path| rten::Model::load_file(p).map_err(|e| OcrError::Load(p.display().to_string(), e.to_string())); + let params = ocrs::OcrEngineParams { + detection_model: Some(load(&models.detection)?), + recognition_model: Some(load(&models.recognition)?), + alphabet: Some(ALPHABET.into()), + ..Default::default() + }; + let engine = ocrs::OcrEngine::new(params).map_err(|e| OcrError::Load("ocr engine".into(), e.to_string()))?; + Ok(Ocr { engine }) + } + + /// Load the models found by [`Models::find`]. + pub fn find() -> Result { + Self::load(&Models::find().ok_or(OcrError::NoModels)?) + } + + /// Recognise the text in an RGBA (or RGB, or grey) image, `width` × `height` pixels. + pub fn recognize(&self, pixels: &[u8], width: u32, height: u32) -> Result, OcrError> { + if pixels.is_empty() { + return Err(OcrError::EmptyImage); + } + let want = checked_len(width, height)?; + // ocrs wants 1 or 3 channels. + let rgb: std::borrow::Cow<[u8]> = if pixels.len() == want { + pixels.as_chunks::<4>().0.iter().flat_map(|p| [p[0], p[1], p[2]]).collect::>().into() + } else { + pixels.into() + }; + let err = |e: &dyn std::fmt::Display| OcrError::Recognize(e.to_string()); + let source = ocrs::ImageSource::from_bytes(&rgb, (width, height)).map_err(|e| err(&e))?; + let input = self.engine.prepare_input(source).map_err(|e| err(&e))?; + let found = self.engine.detect_words(&input).map_err(|e| err(&e))?; + let lines = self.engine.find_text_lines(&input, &found); + let read = self.engine.recognize_text(&input, &lines).map_err(|e| err(&e))?; + use ocrs::TextItem; + Ok(read + .into_iter() + .flatten() + .map(|line| Line { + words: line + .words() + .map(|w| { + let r = w.bounding_rect(); + Word::new(w.to_string(), [r.left() as f32, r.top() as f32, r.right() as f32, r.bottom() as f32], "ocrs") + }) + .filter(|w| !w.text.trim().is_empty()) + .collect(), + }) + .filter(|l| !l.words.is_empty()) + .collect()) + } +} + +/// The ocrs engine as a [`Recognizer`] (the default engine). +pub struct OcrsRecognizer { + engine: Ocr, +} + +impl OcrsRecognizer { + /// Load the models from `models`. + pub fn load(models: &Models) -> Result { + Ok(OcrsRecognizer { engine: Ocr::load(models)? }) + } + + /// Load the models found by [`Models::find`]. + pub fn find() -> Result { + Self::load(&Models::find().ok_or(OcrError::NoModels)?) + } +} + +impl Recognizer for OcrsRecognizer { + fn id(&self) -> &'static str { + "ocrs" + } + + fn available(&self) -> bool { + Models::find().is_some() + } + + fn languages(&self) -> Vec<(String, String)> { + LANGUAGES.iter().map(|(c, n)| ((*c).to_string(), (*n).to_string())).collect() + } + + fn recognize(&self, image: &OcrImage, _options: &RecognizeOptions) -> Result, OcrError> { + self.engine.recognize(image.rgba(), image.width(), image.height()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// Raw dimensions are untrusted (they come from callers, ultimately from documents): zero + /// sides, sides or pixel totals over the caps, and products that would overflow the naive + /// `width * height * 4` are errors, never panics, before anything is touched. + #[test] + fn raw_dimensions_are_capped_and_checked_not_panicked_at() { + assert!(matches!(checked_len(0, 10), Err(OcrError::EmptyImage))); + assert!(matches!(checked_len(10, 0), Err(OcrError::EmptyImage))); + assert!(matches!(checked_len(MAX_SIDE + 1, 1), Err(OcrError::ImageTooLarge))); + // 65536² would wrap u32 in the old `width * height * 4`; the side cap refuses it first. + assert!(matches!(checked_len(u32::MAX, u32::MAX), Err(OcrError::ImageTooLarge))); + // Legal sides whose product is over the 64 MP area cap. + assert!(matches!(checked_len(MAX_SIDE, 7000), Err(OcrError::ImageTooLarge))); + assert_eq!(checked_len(2, 2).unwrap(), 16); + // Exactly the area cap fits: 8000 × 8000 = 64 MP. + assert_eq!(checked_len(8000, 8000).unwrap(), 8000usize * 8000 * 4); + } + + /// The current ONNX pair wins when an installation also retains both legacy models. + #[test] + fn model_discovery_prefers_the_complete_onnx_pair() { + let dir = tempfile::tempdir().unwrap(); + for file in ["text-detection.onnx", "text-recognition.onnx"] { + std::fs::write(dir.path().join(file), b"model").unwrap(); + } + let expected = Some(Models { detection: dir.path().join("text-detection.onnx"), recognition: dir.path().join("text-recognition.onnx") }); + assert_eq!(Models::in_dir(dir.path()), expected, "a complete ONNX pair is sufficient"); + for file in ["text-detection.rten", "text-recognition.rten"] { + std::fs::write(dir.path().join(file), b"model").unwrap(); + } + assert_eq!(Models::in_dir(dir.path()), expected, "the complete ONNX pair takes precedence over legacy models"); + } + + /// An interrupted ONNX update still permits an intact legacy pair in the same directory. + #[test] + fn model_discovery_falls_back_to_legacy_when_the_onnx_pair_is_incomplete() { + for current in ["text-detection.onnx", "text-recognition.onnx"] { + let dir = tempfile::tempdir().unwrap(); + for file in [current, "text-detection.rten", "text-recognition.rten"] { + std::fs::write(dir.path().join(file), b"model").unwrap(); + } + assert_eq!( + Models::in_dir(dir.path()), + Some(Models { detection: dir.path().join("text-detection.rten"), recognition: dir.path().join("text-recognition.rten") }), + "{current} alone must not hide a complete legacy pair" + ); + } + } + + /// Detection and recognition must come from one complete format pair, never a mixed update. + #[test] + fn model_discovery_refuses_mixed_format_pairs() { + for pair in [["text-detection.onnx", "text-recognition.rten"], ["text-detection.rten", "text-recognition.onnx"]] { + let dir = tempfile::tempdir().unwrap(); + for file in pair { + std::fs::write(dir.path().join(file), b"model").unwrap(); + } + assert_eq!(Models::in_dir(dir.path()), None, "{pair:?} is not a complete model pair"); + } + } + + /// Existing source builds and PDFCRAFT_MODELS overrides retain their complete RTEN pair. + #[test] + fn model_discovery_keeps_legacy_only_installations() { + let dir = tempfile::tempdir().unwrap(); + assert_eq!(Models::in_dir(dir.path()), None); + std::fs::write(dir.path().join("text-detection.rten"), b"model").unwrap(); + assert_eq!(Models::in_dir(dir.path()), None, "one legacy model is incomplete"); + std::fs::write(dir.path().join("text-recognition.rten"), b"model").unwrap(); + assert_eq!( + Models::in_dir(dir.path()), + Some(Models { detection: dir.path().join("text-detection.rten"), recognition: dir.path().join("text-recognition.rten") }) + ); + } +} diff --git a/crates/ocr/src/preprocess.rs b/crates/ocr/src/preprocess.rs new file mode 100644 index 0000000000..280ee73fec --- /dev/null +++ b/crates/ocr/src/preprocess.rs @@ -0,0 +1,973 @@ +//! Preprocessing: a pure function between the page raster and recognition, in fixed order — +//! (a) DPI normalization, then the four opt-in steps (b) orientation, (c) deskew, (d) denoise, +//! (e) binarization. Normalization exists for those steps (their windows and thresholds are +//! calibrated for 300 dpi), so it runs only when at least one of them is on: with every step +//! off the raster is handed to the engines exactly as it was rendered. (Enlarging a render +//! on its own changes what the models read: a 72 dpi scan of "due: €250" read as "ue: ?250" +//! after a 4× bilinear enlargement.) Everything is plain Rust on the RGBA raster: no +//! Leptonica, no process, no new dependencies. +//! +//! Every geometric step records its inverse; the result carries one exact +//! [`InverseTransform`] (an affine map, composed in f64), and callers map every word box +//! through it back to the *original* raster coordinates before placing the words. +//! +//! The heuristics are deliberately conservative and panic-free: an image too small to judge, +//! a blank page, or a score without a clear margin is a no-op, never a guess. + +use crate::{MAX_PIXELS, MAX_SIDE, OcrError, OcrImage}; + +/// The four opt-in steps. All off by default: a scan is passed through as rendered. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub struct PreprocessOptions { + /// Detect pages scanned at 90°/180°/270° and turn them upright. + pub auto_rotate: bool, + /// Straighten skewed text lines. + pub deskew: bool, + /// 3×3 median filter (salt-and-pepper noise). + pub denoise: bool, + /// Sauvola binarization (window 8, k 0.34). + pub binarize: bool, +} + +impl PreprocessOptions { + /// Whether any step is on (and so the raster is normalized to 300 dpi first). + pub fn any(&self) -> bool { + self.auto_rotate || self.deskew || self.denoise || self.binarize + } +} + +/// A preprocessed raster: the image to recognize, the exact way back to the input raster, and +/// the resolution the image now stands for. +#[derive(Debug)] +pub struct Preprocessed { + pub image: OcrImage, + /// Maps pixel coordinates in [`Preprocessed::image`] back to the input raster's. + pub inverse_transform: InverseTransform, + /// The image's resolution in dots per inch: 300 after normalization (only when a step is + /// on), or the source's own + /// when it already rendered at 300 or more. + pub dpi: f32, +} + +/// Preprocess `image` (rendered at `source_dpi`) per `options`, in the fixed order. The image +/// is consumed: the common path (every step off, or a no-op) hands the raster back without +/// copying it, and a step that rewrites pixels replaces it. +pub fn preprocess(image: OcrImage, source_dpi: f32, options: &PreprocessOptions) -> Result { + let mut current = image; + let mut inverse = InverseTransform::identity(); + // (a) DPI normalization: for the steps below, a scan under 300 dpi is enlarged by 300/src so + // they see letter-sized strokes. With every step off nothing is resampled. A broken dpi + // (zero, negative, NaN) is left alone. + let mut dpi = if source_dpi.is_finite() && source_dpi >= 1.0 { source_dpi } else { 300.0 }; + if options.any() && dpi < 300.0 { + let scale = f64::from(300.0 / dpi); + current = resize_bilinear(¤t, scale)?; + inverse = compose(&inverse, &InverseTransform::scale(1.0 / scale)); + dpi = 300.0; + } + // (b) Orientation: turn 90/180/270 scans upright (opt-in, skipped for small images). + if options.auto_rotate { + let (w, h) = (current.width, current.height); + let turns = detect_orientation(¤t); + if turns != 0 { + current = rotate_quarter_turns(¤t, turns); + inverse = compose(&inverse, &inverse_quarter_turns(turns, w, h)); + } + } + // (c) Deskew: straighten skewed lines (opt-in; no-op under 64 px, blank, or without a + // clear margin or below 0.1°). + if options.deskew + && let Some(angle) = estimate_skew(¤t) + { + let (straightened, back) = deskew_rotate(¤t, angle); + current = straightened; + inverse = compose(&inverse, &back); + } + // (d) Denoise: 3×3 median with edge clamping (opt-in; dimensions never change). + if options.denoise { + current = median3(¤t); + } + // (e) Binarize: Sauvola (window 8, k 0.34) via an integral image (opt-in); dark ink stays + // dark and paper stays light. + if options.binarize { + current = sauvola(¤t); + } + Ok(Preprocessed { image: current, inverse_transform: inverse, dpi }) +} + +/// An affine map `(x, y) → (a·x + c·y + e, b·x + d·y + f)`, composed in f64 so mapping a +/// word box back is exact up to float rounding. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct InverseTransform { + a: f64, + b: f64, + c: f64, + d: f64, + e: f64, + f: f64, +} + +impl InverseTransform { + pub fn identity() -> InverseTransform { + InverseTransform { a: 1.0, b: 0.0, c: 0.0, d: 1.0, e: 0.0, f: 0.0 } + } + + fn scale(s: f64) -> InverseTransform { + InverseTransform { a: s, b: 0.0, c: 0.0, d: s, e: 0.0, f: 0.0 } + } + + /// `compose(outer, inner)` maps `p` to `outer(inner(p))`: the accumulated inverse of the + /// earlier steps is the outer map (their spaces come later in the round trip), so each + /// step calls `compose(&inverse, &this_step_inverse)`. The scale step, for instance, + /// happens first, so its inverse is applied last: `scale⁻¹(rotate⁻¹(p))`. + fn compose_(outer: &InverseTransform, inner: &InverseTransform) -> InverseTransform { + InverseTransform { + a: outer.a * inner.a + outer.c * inner.b, + b: outer.b * inner.a + outer.d * inner.b, + c: outer.a * inner.c + outer.c * inner.d, + d: outer.b * inner.c + outer.d * inner.d, + e: outer.a * inner.e + outer.c * inner.f + outer.e, + f: outer.b * inner.e + outer.d * inner.f + outer.f, + } + } + + /// Map a point from the processed image back to the input raster. + pub fn apply(&self, x: f64, y: f64) -> (f64, f64) { + (self.a * x + self.c * y + self.e, self.b * x + self.d * y + self.f) + } + + /// Map a word box `[left, top, right, bottom]` back to the input raster: the four corners + /// go through [`Self::apply`] and their bounding box is rounded outward to whole pixels. + /// A box with no finite corners comes back unchanged (garbage in, garbage out — never a + /// panic). + pub fn map_rect(&self, rect: [f32; 4]) -> [f32; 4] { + let corners = [(rect[0], rect[1]), (rect[2], rect[1]), (rect[0], rect[3]), (rect[2], rect[3])]; + let mut lo = (f64::INFINITY, f64::INFINITY); + let mut hi = (f64::NEG_INFINITY, f64::NEG_INFINITY); + let mut finite = false; + for (x, y) in corners { + if !(x.is_finite() && y.is_finite()) { + continue; + } + let (mx, my) = self.apply(f64::from(x), f64::from(y)); + if mx.is_finite() && my.is_finite() { + finite = true; + lo = (lo.0.min(mx), lo.1.min(my)); + hi = (hi.0.max(mx), hi.1.max(my)); + } + } + if !finite { + return rect; + } + [lo.0.floor() as f32, lo.1.floor() as f32, hi.0.ceil() as f32, hi.1.ceil() as f32] + } +} + +/// The step inverses compose in pipeline order: each new step's inverse is applied *after* +/// the inverses accumulated so far. +fn compose(outer: &InverseTransform, inner: &InverseTransform) -> InverseTransform { + InverseTransform::compose_(outer, inner) +} + +/// The size `width` x `height` becomes under `scale`, refused (before any buffer exists) when +/// a side is under one pixel or over [`MAX_SIDE`], or the area is over [`MAX_PIXELS`]: two +/// sides under the side cap can still ask for hundreds of megabytes, which `OcrImage::new` +/// would only refuse after they were allocated. +fn scaled_size(width: u32, height: u32, scale: f64) -> Result<(u32, u32), OcrError> { + let new_w = (f64::from(width) * scale).ceil(); + let new_h = (f64::from(height) * scale).ceil(); + if !(1.0..=f64::from(MAX_SIDE)).contains(&new_w) || !(1.0..=f64::from(MAX_SIDE)).contains(&new_h) { + return Err(OcrError::ImageTooLarge); + } + let (nw, nh) = (new_w as u32, new_h as u32); + if (nw as usize).checked_mul(nh as usize).is_none_or(|pixels| pixels > MAX_PIXELS as usize) { + return Err(OcrError::ImageTooLarge); + } + Ok((nw, nh)) +} + +/// Bilinear resize by `scale`, clamped at the edges; the result stays inside [`scaled_size`]'s +/// limits (the side cap and the area cap). +fn resize_bilinear(image: &OcrImage, scale: f64) -> Result { + let (nw, nh) = scaled_size(image.width, image.height, scale)?; + let mut out = vec![255u8; (nw as usize).checked_mul(nh as usize).and_then(|p| p.checked_mul(4)).ok_or(OcrError::ImageTooLarge)?]; + for (y, row) in out.chunks_exact_mut(4 * nw as usize).enumerate() { + for (x, px) in row.as_chunks_mut::<4>().0.iter_mut().enumerate() { + // The source position of the destination pixel's centre, and the four + // neighbours bilinear interpolation blends (positions clamp at the edges). + let sx = (x as f64 + 0.5) / scale - 0.5; + let sy = (y as f64 + 0.5) / scale - 0.5; + let (x0, y0) = (sx.floor().max(0.0), sy.floor().max(0.0)); + let (fx, fy) = (sx - x0, sy - y0); + for (ch, o) in px.iter_mut().enumerate() { + let blend = |dx: f64, dy: f64| f64::from(image.byte(x0 + dx, y0 + dy, ch)); + let v = blend(0.0, 0.0) * (1.0 - fx) * (1.0 - fy) + + blend(1.0, 0.0) * fx * (1.0 - fy) + + blend(0.0, 1.0) * (1.0 - fx) * fy + + blend(1.0, 1.0) * fx * fy; + *o = v.round().clamp(0.0, 255.0) as u8; + } + } + } + OcrImage::new(nw, nh, out) +} + +/// Grayscale (Rec.601 luma) of every pixel. +fn grayscale(image: &OcrImage) -> Vec { + image.rgba.as_chunks::<4>().0.iter().map(|p| ((299 * p[0] as u32 + 587 * p[1] as u32 + 114 * p[2] as u32) / 1000).min(255) as u8).collect() +} + +/// How the page must be turned (quarter turns, clockwise) to read upright. +type QuarterTurns = u32; + +/// Detect the correction for pages scanned sideways or upside down. Method (documented, +/// panic-free, no OCR): ink is everything darker than the midpoint between the darkest and +/// lightest pixel; for each candidate correction (0/90/180/270 clockwise) the ink's row +/// profile is taken and scored by how sharply it bands (Σ Δ², the projection-profile score: +/// upright text lines band sharply, turned text does not). The best-banding axis wins when it +/// beats the other axis by a factor of 1.3; within the winning axis the line bands' +/// bottom-heaviness (real text carries more ink below each line's middle — baselines and +/// descenders) picks the quarter turn, with a 4%-of-ink margin. Anything less clear is a +/// no-op (`0`), and images under 32 px are never judged. +fn detect_orientation(image: &OcrImage) -> QuarterTurns { + if image.min_side() < 32 { + return 0; + } + let gray = grayscale(image); + let (Some(lo), Some(hi)) = (gray.iter().min(), gray.iter().max()) else { return 0 }; + let t = (*lo as u32 + *hi as u32) / 2; + let ink: Vec = gray.iter().map(|&g| (g as u32) < t).collect(); + let (w, h) = (image.width as usize, image.height as usize); + let total: u64 = ink.iter().filter(|i| **i).count() as u64; + if total == 0 { + return 0; + } + // The ink grid rotated by k quarter turns (clockwise), as row profiles. + let mut scores = [0.0f64; 4]; + let mut asyms = [0.0f64; 4]; + for k in 0u32..4 { + let k = k as usize; + let profile = rotated_row_profile(&ink, w, h, k as u32); + scores[k] = profile.windows(2).map(|p| (p[1] as f64 - p[0] as f64).powi(2)).sum(); + asyms[k] = band_bottom_heaviness(&profile); + } + // 0 and 180 share one profile shape (reversed), as do 90 and 270: pick the axis first. + let (vertical_pair, horizontal_pair) = ((scores[0], scores[2]), (scores[1], scores[3])); + let row_axis = vertical_pair.0.max(vertical_pair.1); + let column_axis = horizontal_pair.0.max(horizontal_pair.1); + if row_axis >= column_axis { + if row_axis < column_axis * 1.3 { + return 0; // the axes are too close to call + } + if asyms[0] > 0.15 * total as f64 { + 0 + } else if asyms[2] > 0.15 * total as f64 { + 2 + } else { + 0 // the bands are not clearly bottom-heavy either way: no-op + } + } else { + if column_axis < row_axis * 1.3 { + return 0; + } + if asyms[1] > 0.15 * total as f64 { + 1 + } else if asyms[3] > 0.15 * total as f64 { + 3 + } else { + 0 + } + } +} + +/// The row profile of the ink grid after `k` quarter turns clockwise. +fn rotated_row_profile(ink: &[bool], w: usize, h: usize, k: u32) -> Vec { + // The rotated image is `w` pixels tall for odd k and `h` for even k; its row index is + // always the new y of the source pixel. + let len = if k.is_multiple_of(2) { h } else { w }; + let mut profile = vec![0u64; len]; + for (y, row) in ink.chunks_exact(w).enumerate() { + for (x, on) in row.iter().enumerate() { + if !on { + continue; + } + // Where does (x, y) land after k quarter turns clockwise? + let ny = match k { + 1 => x, + 2 => h - 1 - y, + 3 => w - 1 - x, + _ => y, + }; + if let Some(p) = profile.get_mut(ny) { + *p += 1; + } + } + } + profile +} + +/// Sum over text-line bands of (ink in the band's lower half − ink in its upper half): +/// positive when the bands carry more ink near their bottoms, as real text does. +fn band_bottom_heaviness(profile: &[u64]) -> f64 { + let mut asym = 0.0; + let mut start: Option = None; + for (i, &p) in profile.iter().chain(std::iter::once(&0)).enumerate() { + match (start, p > 0) { + (None, true) => start = Some(i), + (Some(s), false) => { + let mid = (s + i) / 2; + let lower: u64 = profile.get(s..i).map_or(0, |b| b[mid.saturating_sub(s)..].iter().sum()); + let upper: u64 = profile.get(s..i).map_or(0, |b| b[..mid.saturating_sub(s)].iter().sum()); + asym += lower as f64 - upper as f64; // signed: a top-heavy band scores below zero + start = None; + } + _ => {} + } + } + asym +} + +/// Rotate the raster `k` quarter turns clockwise (an exact pixel permutation). +fn rotate_quarter_turns(image: &OcrImage, k: QuarterTurns) -> OcrImage { + let (w, h) = (image.width as usize, image.height as usize); + let (nw, nh) = if k.is_multiple_of(2) { (image.width, image.height) } else { (image.height, image.width) }; + let mut out = vec![255u8; nw as usize * nh as usize * 4]; + for (y, row) in image.rgba.chunks_exact(w * 4).enumerate() { + for (x, px) in row.as_chunks::<4>().0.iter().enumerate() { + let (nx, ny) = match k { + 1 => (h - 1 - y, x), + 2 => (w - 1 - x, h - 1 - y), + 3 => (y, w - 1 - x), + _ => (x, y), + }; + if let Some(dst) = out.get_mut((ny * nw as usize + nx) * 4..(ny * nw as usize + nx) * 4 + 4) { + dst.copy_from_slice(px); + } + } + } + OcrImage::new(nw, nh, out).unwrap_or_else(|_| image.clone()) +} + +/// The inverse of a `k`-quarter-turn correction: where each corrected pixel came from. +fn inverse_quarter_turns(k: QuarterTurns, old_w: u32, old_h: u32) -> InverseTransform { + let (w, h) = (f64::from(old_w), f64::from(old_h)); + match k { + // Forward (content moves): (x, y) ↦ (h−1−y, x) for one turn clockwise. + 1 => InverseTransform { a: 0.0, b: -1.0, c: 1.0, d: 0.0, e: 0.0, f: h - 1.0 }, + 2 => InverseTransform { a: -1.0, b: 0.0, c: 0.0, d: -1.0, e: w - 1.0, f: h - 1.0 }, + 3 => InverseTransform { a: 0.0, b: 1.0, c: -1.0, d: 0.0, e: w - 1.0, f: 0.0 }, + _ => InverseTransform::identity(), + } +} + +/// Estimate the skew of the text lines, in degrees (−5…5, positive: lines run downhill to the +/// right). Method (documented, panic-free): for each candidate angle the ink is sheared into a +/// row profile (`r = y − (x − cx)·tan φ`); the projection-profile score (Σ profile²) peaks +/// where the shear realigns the lines. `None` — leave the page alone — when the image is under +/// 64 px on a side, blank, the winner is under 0.1°, or the winner does not clearly beat +/// leaving the page as is (score < 1.02 × the 0° score). +fn estimate_skew(image: &OcrImage) -> Option { + if image.min_side() < 64 { + return None; + } + let gray = grayscale(image); + let (Some(lo), Some(hi)) = (gray.iter().min(), gray.iter().max()) else { return None }; + let t = (*lo as u32 + *hi as u32) / 2; + let ink: Vec = gray.iter().map(|&g| (g as u32) < t).collect(); + if !ink.iter().any(|i| *i) { + return None; // a blank page has no skew + } + let (w, h) = (image.width, image.height); + let cx = f64::from(w) / 2.0; + let max_shift = (f64::from(w) / 2.0 * 5f64.to_radians().tan()).ceil() as i64 + 2; + let score = |phi_deg: f64| -> f64 { + let slope = phi_deg.to_radians().tan(); + let mut profile = vec![0u64; (h as i64 + 2 * max_shift + 1) as usize]; + for (y, row) in ink.chunks_exact(w as usize).enumerate() { + for (x, on) in row.iter().enumerate() { + if !on { + continue; + } + let r = y as f64 - (x as f64 - cx) * slope; + let idx = r.round() as i64 + max_shift; + if let Some(p) = profile.get_mut(idx as usize) { + *p += 1; + } + } + } + profile.iter().map(|p| (*p as f64).powi(2)).sum() + }; + let mut best = (0.0f64, score(0.0)); + let mut phi = -5.0; + while phi <= 5.0 + 1e-9 { + let s = score(phi); + if s > best.1 { + best = (phi, s); + } + phi += 0.25; + } + let (angle, best_score) = best; + if angle.abs() < 0.1 || best_score < 1.02 * score(0.0) { + return None; // straight enough, or no clear winner: no-op + } + Some(angle) +} + +/// Rotate the content by `−angle` about the centre (the deskew correction), nearest-neighbour +/// with white fill; also returns the exact dest→src map (the correction's inverse). +fn deskew_rotate(image: &OcrImage, angle_deg: f64) -> (OcrImage, InverseTransform) { + let phi = angle_deg.to_radians(); + let (cos, sin) = (phi.cos(), phi.sin()); + let (w, h) = (image.width, image.height); + let (cx, cy) = (f64::from(w) / 2.0, f64::from(h) / 2.0); + let back = InverseTransform { a: cos, b: sin, c: -sin, d: cos, e: cx - cos * cx + sin * cy, f: cy - sin * cx - cos * cy }; + let mut out = vec![255u8; w as usize * h as usize * 4]; + for (y, row) in out.chunks_exact_mut(w as usize * 4).enumerate() { + for (x, px) in row.as_chunks_mut::<4>().0.iter_mut().enumerate() { + let (sx, sy) = back.apply(x as f64, y as f64); + let (xi, yi) = (sx.round() as i64, sy.round() as i64); + if xi < 0 || yi < 0 || xi >= w as i64 || yi >= h as i64 { + continue; // outside the scan: white fill + } + if let Some(src) = image.rgba.get((yi as usize * w as usize + xi as usize) * 4..(yi as usize * w as usize + xi as usize) * 4 + 4) { + px.copy_from_slice(src); + } + } + } + (OcrImage::new(w, h, out).unwrap_or_else(|_| image.clone()), back) +} + +/// 3×3 median filter on each colour channel, edges clamped (border pixels replicate); +/// dimensions and alpha are unchanged. +fn median3(image: &OcrImage) -> OcrImage { + let (w, h) = (image.width as usize, image.height as usize); + let mut out = image.rgba.clone(); + for (i, dst) in out.as_chunks_mut::<4>().0.iter_mut().enumerate() { + let (x, y) = (i % w, i / w); + let mut rgba = [255u8; 3]; + for (ch, v) in rgba.iter_mut().enumerate() { + let mut window = [0u8; 9]; + for (n, (dx, dy)) in [(-1i64, -1i64), (0, -1), (1, -1), (-1, 0), (0, 0), (1, 0), (-1, 1), (0, 1), (1, 1)].iter().enumerate() { + let nx = ((x as i64) + dx).clamp(0, w as i64 - 1) as usize; + let ny = ((y as i64) + dy).clamp(0, h as i64 - 1) as usize; + window[n] = image.byte(nx as f64, ny as f64, ch); + } + window.sort_unstable(); + *v = window[4]; + } + let alpha = dst[3]; + dst.copy_from_slice(&[rgba[0], rgba[1], rgba[2], alpha]); + } + OcrImage::new(image.width, image.height, out).unwrap_or_else(|_| image.clone()) +} + +/// The Sauvola ink mask: local thresholding with the documented window — 8 pixels wide, the +/// 8 pixels from −4 to +3 (the end is exclusive, so `x0..x1` is exactly `x−4..=x+3`), k 0.34 — +/// through an integral image. A pixel is ink when its grey is at or under the local +/// threshold. +fn sauvola_mask(image: &OcrImage) -> Vec { + let (w, h) = (image.width as usize, image.height as usize); + let gray = grayscale(image); + // Integral sums and square sums, (w+1)×(h+1), so any window sum is four lookups. + let stride = w + 1; + let mut sums = vec![0u64; stride * (h + 1)]; + let mut squares = vec![0u64; stride * (h + 1)]; + for (y, row) in gray.chunks_exact(w).enumerate() { + let (mut row_sum, mut row_sq) = (0u64, 0u64); + for (x, &g) in row.iter().enumerate() { + row_sum += u64::from(g); + row_sq += u64::from(g) * u64::from(g); + let above = sums.get(y * stride + x + 1).copied().unwrap_or(0); + let above_sq = squares.get(y * stride + x + 1).copied().unwrap_or(0); + if let (Some(s), Some(sq)) = (sums.get_mut((y + 1) * stride + x + 1), squares.get_mut((y + 1) * stride + x + 1)) { + *s = above + row_sum; + *sq = above_sq + row_sq; + } + } + } + let (k, r) = (0.34f64, 128.0f64); + let half = 4usize; + let mut ink = vec![false; w * h]; + for (y, row) in ink.chunks_exact_mut(w).enumerate() { + for (x, dst) in row.iter_mut().enumerate() { + let (x0, y0) = (x.saturating_sub(half), y.saturating_sub(half)); + // Exclusive end: the window covers −4..=+3, 8 pixels on a side (an end of + // `x + half + 1` would make it 9 and disagree with the documented window). + let (x1, y1) = ((x + half).min(w), (y + half).min(h)); + let n = ((x1 - x0) * (y1 - y0)) as f64; + let sum = rect_sum(&sums, stride, x0, y0, x1, y1); + let sq = rect_sum(&squares, stride, x0, y0, x1, y1); + let mean = sum / n; + let var = (sq / n - mean * mean).max(0.0); + let threshold = mean * (1.0 + k * (var.sqrt() / r - 1.0)); + *dst = matches!(gray.get(y * w + x), Some(&g) if f64::from(g) <= threshold); + } + } + ink +} + +/// Sauvola binarization through [`sauvola_mask`], followed by a one-pixel dilation of the ink: +/// ink goes to black, paper to white, alpha preserved. +fn sauvola(image: &OcrImage) -> OcrImage { + let (w, h) = (image.width as usize, image.height as usize); + // One dilation keeps anti-aliased strokes whole: a thresholded edge pixel that flaked off + // rejoins its stroke (slightly bold beats broken, for reading). + let ink = dilate1(&sauvola_mask(image), w, h); + let mut out = image.rgba.clone(); + for (i, px) in out.as_chunks_mut::<4>().0.iter_mut().enumerate() { + let v = if ink.get(i).copied().unwrap_or(false) { 0u8 } else { 255u8 }; + if let Some(dst) = px.get_mut(..3) { + dst.copy_from_slice(&[v, v, v]); + } + } + OcrImage::new(image.width, image.height, out).unwrap_or_else(|_| image.clone()) +} + +/// Grow the ink mask by one pixel (8-neighbourhood, edges clamped by the bounds checks). +fn dilate1(ink: &[bool], w: usize, h: usize) -> Vec { + let mut out = ink.to_vec(); + for y in 0..h { + for x in 0..w { + if !ink.get(y * w + x).copied().unwrap_or(false) { + continue; + } + for dy in [-1isize, 0, 1] { + for dx in [-1isize, 0, 1] { + let (nx, ny) = (x as isize + dx, y as isize + dy); + if nx < 0 || ny < 0 || nx >= w as isize || ny >= h as isize { + continue; + } + if let Some(o) = out.get_mut(ny as usize * w + nx as usize) { + *o = true; + } + } + } + } + } + out +} + +fn rect_sum(table: &[u64], stride: usize, x0: usize, y0: usize, x1: usize, y1: usize) -> f64 { + let a = table.get(y1 * stride + x1).copied().unwrap_or(0); + let b = table.get(y0 * stride + x1).copied().unwrap_or(0); + let c = table.get(y1 * stride + x0).copied().unwrap_or(0); + let d = table.get(y0 * stride + x0).copied().unwrap_or(0); + (a + d - b - c) as f64 +} + +#[cfg(test)] +mod tests { + use super::*; + + /// Sides under MAX_SIDE whose area passes MAX_PIXELS (9500 × 9500, 90 MP) are refused + /// when the size is worked out, before the 360 MB buffer is allocated rather than after. + #[test] + fn an_upscale_over_the_area_cap_is_refused_before_allocating() { + assert!(matches!(scaled_size(1000, 1000, 9.5), Err(OcrError::ImageTooLarge))); + assert_eq!(scaled_size(1000, 1000, 2.0).unwrap(), (2000, 2000), "under the caps it still scales"); + assert_eq!(scaled_size(8000, 8000, 1.0).unwrap(), (8000, 8000), "exactly 64 MP is allowed"); + assert!(matches!(scaled_size(8001, 8000, 1.0), Err(OcrError::ImageTooLarge)), "one row over is not"); + assert!(matches!(scaled_size(1000, 1000, 10.5), Err(OcrError::ImageTooLarge)), "a side over MAX_SIDE"); + assert!(matches!(scaled_size(1000, 1000, 0.0), Err(OcrError::ImageTooLarge)), "a side under one pixel"); + assert!(matches!(scaled_size(1000, 1000, f64::NAN), Err(OcrError::ImageTooLarge))); + let small = OcrImage::new(100, 100, vec![255; 100 * 100 * 4]).unwrap(); + assert!(matches!(resize_bilinear(&small, 100.5), Err(OcrError::ImageTooLarge))); + assert_eq!(resize_bilinear(&small, 2.0).unwrap().width, 200); + } + + fn white(w: u32, h: u32) -> OcrImage { + OcrImage::new(w, h, vec![255; (w * h * 4) as usize]).unwrap() + } + + /// Ink bounding box of a raster (None when blank). + fn ink_bbox(image: &OcrImage) -> Option<[f32; 4]> { + let gray = grayscale(image); + let (Some(lo), Some(hi)) = (gray.iter().min(), gray.iter().max()) else { return None }; + let t = (*lo as u32 + *hi as u32) / 2; + let (mut l, mut tp, mut r, mut b) = (u32::MAX, u32::MAX, 0u32, 0u32); + let mut any = false; + for (i, &g) in gray.iter().enumerate() { + if (g as u32) >= t { + continue; + } + any = true; + let (x, y) = ((i as u32) % image.width, (i as u32) / image.width); + l = l.min(x); + r = r.max(x); + tp = tp.min(y); + b = b.max(y); + } + any.then_some([l as f32, tp as f32, r as f32, b as f32]) + } + + /// A horizontal line of three bottom-heavy block glyphs (L shapes) on white — the fixture + /// for orientation detection (240 × 64 at 300 dpi). + fn l_line() -> OcrImage { + let (w, h) = (240u32, 64u32); + let mut img = white(w, h); + for gx in [20u32, 100, 180] { + fill(&mut img, gx, 12, 12, 40); + fill(&mut img, gx, 40, 36, 12); + } + img + } + + fn fill(img: &mut OcrImage, x0: u32, y0: u32, w: u32, h: u32) { + for y in y0..y0 + h { + for x in x0..x0 + w { + let o = ((y * img.width + x) * 4) as usize; + if let Some(p) = img.rgba.get_mut(o..o + 3) { + p.copy_from_slice(&[0, 0, 0]); + } + } + } + } + + /// The same fixture pre-rotated by k quarter turns clockwise, built independently of the + /// crate's rotation (so the test cannot pass by a shared bug). + fn rotated(k: u32) -> OcrImage { + let src = l_line(); + let (w, h) = (src.width as usize, src.height as usize); + let (nw, nh) = if k.is_multiple_of(2) { (src.width, src.height) } else { (src.height, src.width) }; + let mut out = white(nw, nh); + for (y, row) in src.rgba.chunks_exact(w * 4).enumerate() { + for (x, px) in row.as_chunks::<4>().0.iter().enumerate() { + if px[0] != 0 { + continue; + } + let (nx, ny) = match k { + 1 => (h - 1 - y, x), + 2 => (w - 1 - x, h - 1 - y), + 3 => (y, w - 1 - x), + _ => (x, y), + }; + fill(&mut out, nx as u32, ny as u32, 1, 1); + } + } + out + } + + /// A ruled fixture sheared by `degrees` (lines running downhill to the right for positive + /// angles): four 8-px bars across a 300 × 120 page. + fn sheared(degrees: f64) -> OcrImage { + let (w, h) = (300u32, 120u32); + let mut img = white(w, h); + let slope = degrees.to_radians().tan(); + for (base, height) in [(20u32, 8u32), (45, 8), (70, 8), (95, 8)] { + for x in 20..280u32 { + let y = (base as f64 + (x as f64 - 150.0) * slope).round().clamp(0.0, f64::from(h - height - 1)) as u32; + fill(&mut img, x, y, 1, height); + } + } + img + } + + /// With the four opt-in flags off and a dpi at or above 300, the raster passes through + /// untouched and the inverse transform is exact. + #[test] + fn flags_off_and_adequate_dpi_is_an_identity_pass_through() { + let src = l_line(); + let p = preprocess(src.clone(), 300.0, &PreprocessOptions::default()).unwrap(); + assert_eq!(p.image.rgba, src.rgba); + assert_eq!(p.dpi, 300.0); + assert_eq!(p.inverse_transform, InverseTransform::identity()); + assert_eq!(p.inverse_transform.map_rect([3.0, 7.0, 41.0, 19.0]), [3.0, 7.0, 41.0, 19.0]); + } + + /// With a step on, a scan below 300 dpi is enlarged by 300/src first, and its boxes map + /// back to the original raster. + #[test] + fn low_dpi_scans_are_upscaled_for_the_steps_and_boxes_map_back() { + let src = white(100, 50); + let on = PreprocessOptions { denoise: true, ..Default::default() }; + let p = preprocess(src.clone(), 150.0, &on).unwrap(); + assert_eq!((p.image.width, p.image.height), (200, 100)); + assert_eq!(p.dpi, 300.0); + assert_eq!(p.inverse_transform.map_rect([0.0, 0.0, 200.0, 100.0]), [0.0, 0.0, 100.0, 50.0]); + assert_eq!(p.inverse_transform.apply(2.0, 4.0), (1.0, 2.0)); + // 300 and 600 dpi are not resampled; a broken dpi is left alone. + assert_eq!(preprocess(src.clone(), 600.0, &on).unwrap().image.width, 100); + assert_eq!(preprocess(src, f32::NAN, &on).unwrap().image.width, 100); + } + + /// With every step off a low-dpi render is read exactly as rendered: not resampled, with + /// the identity as its way back. (Reading a 72 dpi scan through a 4x enlargement changed + /// what the models read: "due: \€250" came back as "ue: ?250".) + #[test] + fn with_every_step_off_a_low_dpi_render_is_not_resampled() { + let src = l_line(); + let p = preprocess(src.clone(), 72.0, &PreprocessOptions::default()).unwrap(); + assert_eq!(p.image.rgba, src.rgba); + assert_eq!((p.image.width, p.image.height), (src.width, src.height)); + assert_eq!(p.dpi, 72.0, "the render's own resolution"); + assert_eq!(p.inverse_transform, InverseTransform::identity()); + } + + /// The upscale refuses to break the side cap. + #[test] + fn upscaling_respects_the_side_cap() { + let src = white(9999, 10); + let r = preprocess(src, 72.0, &PreprocessOptions { denoise: true, ..Default::default() }); + assert!(matches!(r, Err(OcrError::ImageTooLarge)), "upscale past the cap: {r:?}"); + } + + /// An upright page is left alone by the orientation detector. + #[test] + fn upright_pages_are_not_rotated() { + let p = preprocess(l_line(), 300.0, &PreprocessOptions { auto_rotate: true, ..Default::default() }).unwrap(); + assert_eq!(p.inverse_transform, InverseTransform::identity()); + } + + /// Pages scanned at 90/180/270 are detected, turned upright, and their word boxes map back + /// onto the scanned raster's ink. + #[test] + fn quarter_turns_are_detected_corrected_and_mapped_back() { + for k in 1..=3 { + let scanned = rotated(k); + let p = preprocess(scanned.clone(), 300.0, &PreprocessOptions { auto_rotate: true, ..Default::default() }).unwrap(); + let corrected = ink_bbox(&p.image).unwrap_or_else(|| panic!("k={k}: the corrected page still has ink")); + // The corrected reading is a horizontal line again: wider than tall. + assert!(corrected[2] - corrected[0] > corrected[3] - corrected[1], "k={k}: {corrected:?}"); + // And the box maps back onto the scanned raster's own ink box. + let back = p.inverse_transform.map_rect(corrected); + let original = ink_bbox(&scanned).unwrap(); + for i in 0..4 { + assert!((back[i] - original[i]).abs() <= 2.0, "k={k}: {back:?} vs {original:?}"); + } + } + } + + /// Images under 32 px on a side are never judged; the step is a no-op. + #[test] + fn tiny_images_skip_orientation() { + let src = white(31, 31); + let p = preprocess(src, 300.0, &PreprocessOptions { auto_rotate: true, ..Default::default() }).unwrap(); + assert_eq!(p.inverse_transform, InverseTransform::identity()); + } + + /// Blank pages are inconclusive: every step leaves them alone. + #[test] + fn blank_pages_are_a_noop() { + let src = white(200, 200); + let p = preprocess(src, 300.0, &PreprocessOptions { auto_rotate: true, deskew: true, denoise: true, binarize: true }).unwrap(); + assert_eq!(p.inverse_transform, InverseTransform::identity()); + } + + /// The skew estimator sees a 3° shear (either direction) and leaves a straight page alone. + #[test] + fn skew_is_estimated_and_straight_pages_are_left_alone() { + for degrees in [3.0, -3.0] { + let angle = estimate_skew(&sheared(degrees)).unwrap_or_else(|| panic!("{degrees}° should be found")); + assert!((angle - degrees).abs() <= 0.5, "{degrees}°: got {angle}"); + } + assert!(estimate_skew(&sheared(0.0)).is_none(), "a straight page is a no-op"); + assert!(estimate_skew(&l_line()).is_none(), "glyph lines without a shear are a no-op"); + } + + /// Deskew straightens ±3° lines (the vertical span tightens) and the centre maps back to + /// itself. + #[test] + fn deskew_corrects_plus_and_minus_three_degrees() { + for degrees in [3.0, -3.0] { + let src = sheared(degrees); + let before = ink_bbox(&src).unwrap(); + let p = preprocess(src.clone(), 300.0, &PreprocessOptions { deskew: true, ..Default::default() }).unwrap(); + let after = ink_bbox(&p.image).unwrap(); + assert!(after[3] - after[1] < before[3] - before[1], "{degrees}°: {before:?} → {after:?}"); + // The exact inverse maps the corrected centre back to the scanned centre. + let (cx, cy) = (f64::from(src.width) / 2.0, f64::from(src.height) / 2.0); + let (bx, by) = p.inverse_transform.apply(cx, cy); + assert!((bx - cx).abs() < 1.0 && (by - cy).abs() < 1.0, "{degrees}°: ({bx}, {by})"); + } + } + + /// Deskew no-ops on small images (under 64 px) — the detector is never consulted. + #[test] + fn small_images_skip_deskew() { + let src = white(63, 63); + let p = preprocess(src, 300.0, &PreprocessOptions { deskew: true, ..Default::default() }).unwrap(); + assert_eq!(p.inverse_transform, InverseTransform::identity()); + } + + /// The median filter removes isolated specks, keeps solid strokes, and never changes the + /// dimensions. + #[test] + fn denoise_removes_specks_and_keeps_dimensions() { + let mut src = white(32, 32); + fill(&mut src, 15, 15, 1, 1); // salt + fill(&mut src, 4, 4, 5, 5); // a solid stroke + let p = preprocess(src, 300.0, &PreprocessOptions { denoise: true, ..Default::default() }).unwrap(); + assert_eq!((p.image.width, p.image.height), (32, 32)); + assert_eq!(p.image.byte(15.0, 15.0, 0), 255, "an isolated speck is gone"); + assert_eq!(p.image.byte(6.0, 6.0, 0), 0, "the inside of a solid block stays"); + assert_eq!(p.inverse_transform, InverseTransform::identity()); + } + + /// Sauvola keeps the polarity (dark ink stays dark, paper stays light) and the metadata. + /// The fixture is a thin grey stroke: its windows mix ink and paper, so the local + /// threshold separates them (a perfectly flat mid-tone area has no contrast and whitens, + /// which is Sauvola working as designed). + #[test] + fn binarize_keeps_polarity_and_metadata() { + let (w, h) = (64u32, 64u32); + let mut src = white(w, h); + for y in 26..30 { + for x in 8..56 { + let o = ((y * w + x) * 4) as usize; + if let Some(p) = src.rgba.get_mut(o..o + 3) { + p.copy_from_slice(&[70, 70, 70]); // a grey scan's ink, far below paper + } + } + } + let p = preprocess(src, 300.0, &PreprocessOptions { binarize: true, ..Default::default() }).unwrap(); + assert_eq!((p.image.width, p.image.height), (w, h)); + assert_eq!(p.dpi, 300.0); + assert_eq!(p.image.byte(30.0, 28.0, 0), 0, "ink stays dark"); + assert_eq!(p.image.byte(2.0, 2.0, 0), 255, "paper stays light"); + for px in p.image.rgba.as_chunks::<4>().0.iter() { + assert!((px[0] == 0 || px[0] == 255) && px[0] == px[1] && px[1] == px[2], "{px:?}"); + } + } + + /// The inverse transform composes: an upscaled, turned, deskewed page maps corrected + /// points back inside the original raster. + #[test] + fn inverse_transform_maps_points_back_through_every_step() { + let scanned = rotated(1); // a page photographed on its side + let p = preprocess(scanned.clone(), 150.0, &PreprocessOptions { auto_rotate: true, deskew: true, ..Default::default() }).unwrap(); + let (w, h) = (f64::from(p.image.width), f64::from(p.image.height)); + for (x, y) in [(0.0, 0.0), (w - 1.0, 0.0), (0.0, h - 1.0), (w - 1.0, h - 1.0), (w / 2.0, h / 2.0)] { + let (bx, by) = p.inverse_transform.apply(x, y); + assert!((-2.0..=f64::from(scanned.width) + 2.0).contains(&bx), "x {x} → {bx}"); + assert!((-2.0..=f64::from(scanned.height) + 2.0).contains(&by), "y {y} → {by}"); + } + // The scale step, alone, is exact. + assert_eq!(InverseTransform::scale(0.5).apply(4.0, 10.0), (2.0, 5.0)); + } + + /// A realistic letter page (one line of type on white, at 300 dpi) reads as upright: the + /// row axis bands decisively sharper than the column axis, and the near-balanced + /// bottom-heaviness of real mixed-case text is below the 15% flip bar, so the page is left + /// alone rather than risk flipping a good scan upside down. + #[test] + fn realistic_letter_pages_are_left_upright() { + let (w, h) = (1200u32, 1650u32); + let mut img = white(w, h); + let mut x = 120u32; + let base = 300u32; + let mut seed = 7u64; + let mut rnd = || { + seed = seed.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407); + (seed >> 33) as u32 + }; + while x < 1000 { + let word = rnd() % 5 + 2; + for _ in 0..word { + let kind = rnd() % 4; + let wl = rnd() % 8 + 3; + match kind { + 0 => fill(&mut img, x, base - 20, wl, 20), // x-height + 1 => fill(&mut img, x, base - 34, wl, 34), // ascender + 2 => fill(&mut img, x, base - 20, wl, 28), // descender + _ => { + fill(&mut img, x, base - 20, 2, 20); + fill(&mut img, x + 3, base - 14, 2, 14); // stems + } + } + x += wl + 2; + } + x += 12; // space + } + assert_eq!(detect_orientation(&img), 0); + let p = preprocess(img, 300.0, &PreprocessOptions { auto_rotate: true, ..Default::default() }).unwrap(); + assert_eq!(p.inverse_transform, InverseTransform::identity()); + } + + /// Hostile inputs are errors or no-ops, never panics: 1×1 rasters, all-black pages, NaN + /// boxes through the inverse map. + #[test] + fn hostile_inputs_never_panic() { + let tiny = white(1, 1); + let all = PreprocessOptions { auto_rotate: true, deskew: true, denoise: true, binarize: true }; + let p = preprocess(tiny, 300.0, &all).unwrap(); + assert_eq!((p.image.width, p.image.height), (1, 1)); + // All-black: no paper, nothing to judge. + let mut black = white(80, 80); + for px in black.rgba.as_chunks_mut::<4>().0 { + px.copy_from_slice(&[0, 0, 0, 255]); + } + let p = preprocess(black, 300.0, &all).unwrap(); + assert_eq!(p.inverse_transform, InverseTransform::identity(), "all-black is inconclusive"); + // NaN boxes never poison downstream math: the finite corners bound the mapped box, + // and a box with no finite corners comes back unchanged. + let mapped = p.inverse_transform.map_rect([f32::NAN, 0.0, 5.0, 5.0]); + assert!(mapped.iter().all(|v| v.is_finite()), "{mapped:?}"); + let unchanged = InverseTransform::identity().map_rect([f32::NAN; 4]); + assert!(unchanged.iter().all(|v| v.is_nan()), "{unchanged:?}"); + } + + /// The one-pixel dilation is load-bearing, pinned with a fixture that fragments without + /// it: a thin anti-aliased-looking stroke on a grey scan (alternating dark and light + /// pixels) — the raw Sauvola mask drops the light runs (the local window sits on a grey + /// page, so the threshold falls below them), while the binarized image keeps the stroke + /// whole. + #[test] + fn binarize_without_the_dilation_the_stroke_fragments_and_with_it_stays_whole() { + let (w, h) = (64u32, 64u32); + let mut src = white(w, h); + let set = |img: &mut OcrImage, x: u32, y: u32, v: u8| { + let o = ((y * w + x) * 4) as usize; + if let Some(p) = img.rgba.get_mut(o..o + 3) { + p.copy_from_slice(&[v, v, v]); + } + }; + for px in src.rgba.as_chunks_mut::<4>().0.iter_mut() { + px[..3].copy_from_slice(&[210, 210, 210]); // a grey scan's paper + } + // The stroke: one pixel tall, alternating dark (reads as ink) and light (an + // anti-aliased shoulder the threshold alone drops) in runs of two. + for x in 8..56u32 { + let v = if (x / 2) % 2 == 0 { 110 } else { 170 }; + set(&mut src, x, 32, v); + } + let mask = sauvola_mask(&src); + let at = |x: u32, y: u32| mask[y as usize * w as usize + x as usize]; + let dropped: Vec = (12..52).filter(|&x| !at(x, 32)).collect(); + assert!(!dropped.is_empty(), "without the dilation the stroke fragments: these stroke pixels fall under the threshold at {dropped:?}"); + let p = preprocess(src, 300.0, &PreprocessOptions { binarize: true, ..Default::default() }).unwrap(); + // With the dilation the stroke is whole: every position on it has ink within its + // 8-neighbourhood. + for x in 12..=51u32 { + let ink_near = (-1i32..=1).any(|dy| { + (-1i32..=1).any(|dx| { + let (nx, ny) = (x as i32 + dx, 32 + dy); + p.image.byte(f64::from(nx), f64::from(ny), 0) == 0 + }) + }); + assert!(ink_near, "the stroke is broken at x={x}"); + } + } + + /// The deskew inverse, checked at OFF-CENTRE points (the centre maps to itself under any + /// rotation, so it cannot catch a wrong sign): a point on a straightened line maps back + /// onto the sheared source line, uphill and downhill. + #[test] + fn deskew_inverse_maps_off_centre_points_back_onto_the_source_line() { + for degrees in [3.0, -3.0] { + let src = sheared(degrees); + let p = preprocess(src, 300.0, &PreprocessOptions { deskew: true, ..Default::default() }).unwrap(); + let slope = degrees.to_radians().tan(); + // The corrected first line sits at y = 20 (the fixture's base); the same ink was + // at 20 + (x − 150)·slope in the source. Well off centre on purpose. The + // rotation moves x a little too (it mixes in y: at most sin(5°)·40 ≈ 3.5 px + // here), so the line is checked at the mapped x — a wrong sign still misses it + // by ±(x−150)·slope, 6+ px at these offsets. + for x in [30.0f64, 150.0, 270.0] { + let (sx, sy) = p.inverse_transform.apply(x, 20.0); + assert!((sx - x).abs() < 4.0, "{degrees}° x={x}: mapped to ({sx}, {sy})"); + let expected = 20.0 + (sx - 150.0) * slope; + assert!((sy - expected).abs() < 2.0, "{degrees}° x={x}: ({sx}, {sy}), the source line is at {expected}"); + } + } + } +} diff --git a/crates/ocr/src/tesseract.rs b/crates/ocr/src/tesseract.rs new file mode 100644 index 0000000000..9fbcc574d8 --- /dev/null +++ b/crates/ocr/src/tesseract.rs @@ -0,0 +1,1022 @@ +//! The Tesseract engine: an installed `tesseract` binary driven as an optional external +//! process (`tesseract stdout -l tsv`), never linked, never bundled, never +//! downloaded. The whole module is compiled out for wasm32, where nothing may spawn. +//! +//! Security posture (hard rules): the program path comes from the PATH or a user-configured +//! location, never from document data; the language code is allow-listed before it reaches a +//! process; the page image goes to a private temp file (a fresh 0700 directory holding a 0600 +//! file on Unix) that is deleted afterwards; the run has a timeout and an output size cap; and +//! every failure comes back through [`OcrError`] with its cause named. + +use std::io::Write; +use std::path::{Path, PathBuf}; +use std::sync::Mutex; +use std::sync::atomic::{AtomicU8, Ordering}; +use std::time::{Duration, Instant}; + +use super::{Line, OcrError, OcrImage, RecognizeOptions, Recognizer, Word}; + +/// How long one recognition may run before the process is killed. +const TIMEOUT: Duration = Duration::from_secs(120); +/// How much TSV output is accepted before the run is abandoned. +const MAX_OUTPUT: u64 = 64 * 1024 * 1024; +/// How much of the program's stderr is read back (its end): enough for a `--list-langs` list +/// printed there, small enough that error text stays a message, not a memory DoS. +const MAX_STDERR: u64 = 256 * 1024; +/// The most words one TSV reading may hold. Known limit: a reading with more words than this +/// is truncated at the cap — corrupt or runaway output must not become unbounded memory in +/// the word list, the text layer and the review UI. +pub const MAX_WORDS: usize = 50_000; +/// Tesseract's code for each language PdfCraft offers (ISO 639-1 → tessdata name). Only these +/// codes ever reach the process, whatever the settings say. +pub const LANGUAGE_CODES: &[(&str, &str)] = &[ + ("en", "eng"), + ("de", "deu"), + ("fr", "fra"), + ("es", "spa"), + ("it", "ita"), + ("pt", "por"), + ("ru", "rus"), + ("zh", "chi_sim"), + ("ja", "jpn"), + ("ko", "kor"), + ("ar", "ara"), + ("nl", "nld"), +]; + +/// English names for [`LANGUAGE_CODES`] (dropdowns show code + name). +pub const LANGUAGE_NAMES: &[(&str, &str)] = &[ + ("en", "English"), + ("de", "German"), + ("fr", "French"), + ("es", "Spanish"), + ("it", "Italian"), + ("pt", "Portuguese"), + ("ru", "Russian"), + ("zh", "Chinese (Simplified)"), + ("ja", "Japanese"), + ("ko", "Korean"), + ("ar", "Arabic"), + ("nl", "Dutch"), +]; + +/// Map a settings language code to a Tesseract one: trimmed, case-insensitive, allow-listed; +/// anything unknown falls back to `"eng"`. +pub fn language_code(code: &str) -> &'static str { + let c = code.trim().to_ascii_lowercase(); + LANGUAGE_CODES.iter().find(|(k, _)| *k == c).map(|(_, v)| *v).unwrap_or("eng") +} + +/// The user-configured program location, when valid: `$PDFCRAFT_TESSERACT` (a user +/// preference, never taken from document data). An invalid value — relative, missing, a +/// directory, not executable — means unavailable: an explicit setting is never silently +/// replaced by a PATH search. +pub fn configured_path() -> Option { + let value = std::env::var_os("PDFCRAFT_TESSERACT")?; + let program = PathBuf::from(value); + program_is_executable(&program).then_some(program) +} + +/// The program [`TesseractCli::find`] would run: a set `configured` location (the value of +/// `PDFCRAFT_TESSERACT`) decides alone, so an invalid one yields `None` and `search` (the PATH) +/// is not even consulted; only when nothing is configured does `search` decide. +fn locate(configured: Option, search: impl FnOnce() -> Option) -> Option { + match configured { + Some(value) => Some(PathBuf::from(value)).filter(|program| program_is_executable(program)), + None => search(), + } +} + +/// The first `tesseract` on the PATH that is an existing executable file. Entries that are not +/// absolute are refused: a relative PATH entry resolves against the caller's working +/// directory, which is neither stable nor a safe place to find a program. Also named by +/// `ocr_status` (the engine reports where it would look, without probing). +pub fn path_search() -> Option { + path_search_in(std::env::split_paths(&std::env::var_os("PATH")?)) +} + +/// [`path_search`] over an explicit directory list (so tests can pass one). +fn path_search_in(dirs: impl Iterator) -> Option { + for dir in dirs.filter(|d| d.is_absolute()) { + for name in program_file_names() { + let candidate = dir.join(name); + if program_is_executable(&candidate) { + return Some(candidate); + } + } + } + None +} + +/// The file names an installed `tesseract` binary goes by. +fn program_file_names() -> &'static [&'static str] { + #[cfg(windows)] + { + &["tesseract.exe"] + } + #[cfg(not(windows))] + { + &["tesseract"] + } +} + +/// Whether `program` is a usable program location (see [`program_is_executable`]); callers +/// that run a validated program by path (the engine's `--version` query) check it first. +pub fn program_is_usable(program: &Path) -> bool { + program_is_executable(program) +} + +/// Whether `program` is a usable configured location: an absolute path to an existing, +/// executable FILE (on Unix the executable bits must be set; elsewhere being a file is what +/// the OS requires). A directory or a missing file is simply unavailable — never a panic. +fn program_is_executable(program: &Path) -> bool { + if !program.is_absolute() || !program.is_file() { + return false; + } + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::metadata(program).is_ok_and(|m| m.permissions().mode() & 0o111 != 0) + } + #[cfg(not(unix))] + { + true + } +} + +/// One row of Tesseract's TSV output. `level` 5 rows are the words; `conf` is −1 on the header +/// rows (page, block, paragraph, line), which are not words. +struct TsvRow<'a> { + level: &'a str, + block: u32, + par: u32, + line: u32, + left: f32, + top: f32, + width: f32, + height: f32, + conf: f32, + text: &'a str, +} + +/// Parse one TSV row; `None` for anything unreadable (short rows, non-numeric fields), which +/// is skipped rather than trusted. A row with non-finite numbers or a negative width or height +/// is not a word either: geometry like that is output corruption, and trusting it puts NaN +/// into word boxes and on to the page. +fn tsv_row(row: &str) -> Option> { + let c: Vec<&str> = row.split('\t').collect(); + if c.len() < 12 { + return None; + } + let (block, par, line) = (c[2].trim().parse().ok()?, c[3].trim().parse().ok()?, c[4].trim().parse().ok()?); + let (left, top): (f32, f32) = (c[6].trim().parse().ok()?, c[7].trim().parse().ok()?); + let (width, height, conf): (f32, f32, f32) = (c[8].trim().parse().ok()?, c[9].trim().parse().ok()?, c[10].trim().parse().ok()?); + if !left.is_finite() || !top.is_finite() || !width.is_finite() || !height.is_finite() || !conf.is_finite() { + return None; // NaN/inf anywhere in the numbers + } + if width < 0.0 || height < 0.0 { + return None; // a word box has a non-negative size + } + Some(TsvRow { level: c[0], block, par, line, left, top, width, height, conf, text: c[11] }) +} + +/// Parse `tesseract … tsv` output into lines of words. Rows with confidence −1 are the +/// structural header rows and never become words; consecutive word rows that share their +/// (block, paragraph, line) form one [`Line`], in output (reading) order. Confidence clamps +/// into 0–100. Known limit: past [`MAX_WORDS`] the rest of the reading is dropped (truncated, +/// not an error — the words before the cap are still placed). +pub fn parse_tsv(tsv: &str) -> Vec { + parse_tsv_bounded(tsv).0 +} + +/// The parser utility keeps its bounded-prefix API; recognition also checks completeness. +fn parse_tsv_bounded(tsv: &str) -> (Vec, bool) { + let mut lines: Vec = Vec::new(); + let mut key: Option<(u32, u32, u32)> = None; + let mut words = 0usize; + for row in tsv.lines() { + let Some(r) = tsv_row(row) else { continue }; + if r.level != "5" || r.conf < 0.0 || r.text.trim().is_empty() { + continue; + } + if words >= MAX_WORDS { + return (lines, true); + } + let this = (r.block, r.par, r.line); + if key != Some(this) { + lines.push(Line::default()); + key = Some(this); + } + let word = Word::new(r.text, [r.left, r.top, r.left + r.width, r.top + r.height], "tesseract").with_confidence(r.conf.clamp(0.0, 100.0)); + if let Some(last) = lines.last_mut() { + last.words.push(word); + words += 1; + } + } + (lines, false) +} + +/// Parse `tesseract --list-langs` output: the header line ("List of available languages…"), +/// then one code per line. Lines with spaces are not codes (a header, a warning) and are left +/// out. +pub fn parse_list_langs(out: &str) -> Vec { + out.lines() + .skip_while(|l| !l.trim().is_empty() && l.trim().starts_with("List of available")) + .map(|l| l.trim().to_string()) + .filter(|l| !l.is_empty() && !l.contains(char::is_whitespace)) + .collect() +} + +/// The languages a `--list-langs` run reported: its stdout, or, when that holds none, its +/// stderr (Tesseract 3.x prints the list there). +pub fn list_langs_from(stdout: &str, stderr: &str) -> Vec { + let codes = parse_list_langs(stdout); + if codes.is_empty() { parse_list_langs(stderr) } else { codes } +} + +/// The first non-empty line of what `tesseract --version` printed: on stdout, or on stderr +/// where older versions print it. +fn version_line(stdout: &str, stderr: &str) -> Option { + let first = |text: &str| text.lines().map(str::trim).find(|line| !line.is_empty()).map(str::to_owned); + first(stdout).or_else(|| first(stderr)) +} + +/// The Tesseract program. Build one (via [`TesseractCli::find`] for the PATH copy, or +/// [`TesseractCli::new`] for a user-configured path) and reuse it: `--list-langs` runs at most +/// once. +pub struct TesseractCli { + program: PathBuf, + /// The timeout and the output cap are fields so tests can shorten them. + timeout: Duration, + max_output: u64, + /// Whether `--list-langs` has succeeded yet: 0 untried, 1 yes, 2 no. + probed: AtomicU8, + langs: Mutex>, +} + +impl TesseractCli { + /// The program at `path` (a user preference; never taken from document data). + pub fn new(program: PathBuf) -> TesseractCli { + TesseractCli { program, timeout: TIMEOUT, max_output: MAX_OUTPUT, probed: AtomicU8::new(0), langs: Mutex::new(Vec::new()) } + } + + /// The program this engine runs (`ocr_status` names it). + pub fn program(&self) -> &std::path::Path { + &self.program + } + + /// The program at a user-configured `path`, if it is usable there: only an absolute path + /// to an existing, executable file is accepted (a directory, a missing file or a relative + /// path is unavailable, never a panic), and it must answer `--list-langs`. + pub fn at(program: PathBuf) -> Option { + if !program_is_executable(&program) { + return None; + } + let t = TesseractCli::new(program); + let langs = t.list_langs().ok()?; + *t.langs.lock().unwrap_or_else(|e| e.into_inner()) = langs; + t.probed.store(1, Ordering::Relaxed); + Some(t) + } + + /// The program this machine offers, if it answers `--list-langs`: the user-configured + /// location ([`configured_path`] / `$PDFCRAFT_TESSERACT`) when valid, else the first + /// absolute PATH entry that holds an executable `tesseract`. + /// + /// # The PATH-lookup risk + /// + /// Resolving through the PATH trusts every directory on it: a binary planted earlier on + /// the PATH runs as a subprocess with this application's privileges — it sees the page + /// image and anything the process can reach. The language code and the page image still + /// never come from document data, but a user who handles untrusted documents should pin + /// the exact program with `PDFCRAFT_TESSERACT`, which is validated (absolute, existing, + /// executable file) before any process starts. + /// + /// A `PDFCRAFT_TESSERACT` that is set but invalid fails closed: it is never replaced by + /// the PATH search. + pub fn find() -> Option { + TesseractCli::at(locate(std::env::var_os("PDFCRAFT_TESSERACT"), path_search)?) + } + + /// Run `tesseract --list-langs` and parse it; also decides [`TesseractCli::available`]. + fn list_langs(&self) -> Result, OcrError> { + let (out, err) = self.run_capture(&["--list-langs"], self.timeout, self.max_output)?; + let codes = list_langs_from(&out, &err); + if codes.is_empty() { + return Err(OcrError::NoTesseract("--list-langs reported no languages".into())); + } + Ok(codes + .into_iter() + .map(|c| { + let name = LANGUAGE_NAMES.iter().find(|(k, _)| *k == c.as_str()).map(|(_, n)| (*n).to_string()).unwrap_or_else(|| c.clone()); + (c, name) + }) + .collect()) + } + + /// The program's first `--version` line (named by `ocr_status`); `None` when the program + /// does not answer. Not cached: status asks rarely, and a re-install mid-session should + /// show up without a restart. + pub fn version(&self) -> Option { + let (out, err) = self.run_capture(&["--version"], self.timeout, self.max_output).ok()?; + version_line(&out, &err) + } + + /// Run the program with fixed argv, its stdout in a private temp file, under `timeout` + /// seconds and an output cap of `max_output` bytes. Returns the (capped) stdout. + fn run(&self, args: &[&str], timeout: Duration, max_output: u64) -> Result { + self.run_capture(args, timeout, max_output).map(|(out, _)| out) + } + + /// [`Self::run`], also returning what the program wrote to stderr (up to + /// [`MAX_STDERR`] bytes, its end kept): some Tesseract versions print the `--list-langs` + /// list there instead of on stdout. + fn run_capture(&self, args: &[&str], timeout: Duration, max_output: u64) -> Result<(String, String), OcrError> { + let dir = TempDir::create()?; + let out_path = dir.path.join("out.txt"); + let err_path = dir.path.join("err.txt"); + let out_file = std::fs::File::create(&out_path).map_err(|e| OcrError::Process(format!("creating the output file failed: {e}")))?; + // stderr goes to a file, not a pipe: an undrained pipe fills (around 64 KiB) and then + // blocks the child mid-write, which a watcher can only read as a timeout. A file never + // blocks; it is read back as a short tail after the run. + let err_file = std::fs::File::create(&err_path).map_err(|e| OcrError::Process(format!("creating the error file failed: {e}")))?; + // Known limit: only the direct child is killed at the timeout or on an early cap. A + // wrapper script's children (grandchildren of this process) may survive the kill; the + // cap and the timeout still bound what this run reads and how long it waits. + let mut child = + std::process::Command::new(&self.program) + .args(args) + .stdin(std::process::Stdio::null()) + .stdout(out_file) + .stderr(err_file) + .current_dir(&dir.path) + // A minimal environment: nothing from the document or the caller's shell beyond + // what an installed tesseract needs — its data prefix, the PATH (bare-name lookup + // and DLL loading), the Windows system root, and a temp dir it may write to. + .env_clear() + .envs(std::env::vars().filter(|(k, _)| { + k.eq_ignore_ascii_case("TESSDATA_PREFIX") || k.eq_ignore_ascii_case("PATH") || k.eq_ignore_ascii_case("SystemRoot") + })) + .env("TMP", &dir.path) + .env("TEMP", &dir.path) + .spawn() + .map_err(|e| OcrError::NoTesseract(format!("{} could not start: {e}", self.program.display())))?; + let started = Instant::now(); + let status = loop { + match child.try_wait() { + Ok(Some(status)) => break Ok(status), + Ok(None) => { + // The output caps are enforced while the child runs: a flood is killed as + // soon as a file passes its cap, not after it has finished writing. The + // read-back below caps what this run KEEPS of stderr; this cap bounds what + // the child can write there in the first place. + if std::fs::metadata(&out_path).map(|m| m.len()).unwrap_or(0) > max_output + || std::fs::metadata(&err_path).map(|m| m.len()).unwrap_or(0) > MAX_STDERR + { + let _ = child.kill(); + let _ = child.wait(); + break Err(OcrError::OutputTooLarge(max_output)); + } + if started.elapsed() >= timeout { + let _ = child.kill(); + let _ = child.wait(); + break Err(OcrError::Timeout(timeout.as_secs())); + } + std::thread::sleep(Duration::from_millis(20)); + } + Err(e) => { + let _ = child.kill(); + let _ = child.wait(); + break Err(OcrError::Process(format!("waiting for {} failed: {e}", self.program.display()))); + } + } + }; + let status = status?; + // Whatever the exit, the stderr tail names the cause (capped: error text stays a + // message, not a memory DoS). + let stderr = std::fs::File::open(&err_path) + .and_then(|mut f| { + use std::io::{Read, Seek, SeekFrom}; + let len = f.metadata()?.len().min(MAX_STDERR); + f.seek(SeekFrom::End(-(len as i64)))?; + let mut buf = Vec::with_capacity(len as usize); + f.take(len).read_to_end(&mut buf)?; + Ok::<_, std::io::Error>(buf) + }) + .unwrap_or_default(); + let stderr = String::from_utf8_lossy(&stderr).into_owned(); + // The message names only the end of it. + let tail = stderr.char_indices().rev().nth(2047).map_or(0, |(i, _)| i); + let note = stderr.get(tail..).unwrap_or_default().trim(); + if !status.success() { + return Err(OcrError::Process(format!( + "{} exited with {}: {}", + self.program.display(), + status.code().map(|c| c.to_string()).unwrap_or_else(|| "a signal".into()), + if note.is_empty() { "no error message" } else { note } + ))); + } + let len = std::fs::metadata(&out_path).map(|m| m.len()).unwrap_or(0); + if len > max_output { + return Err(OcrError::OutputTooLarge(max_output)); + } + let bytes = std::fs::read(&out_path).map_err(|e| OcrError::Process(format!("reading the output failed: {e}")))?; + Ok((String::from_utf8_lossy(&bytes).into_owned(), stderr)) + } + + /// Retry `--list-langs` once (an install may have appeared since the last try). + fn refresh(&self) -> Result<(), OcrError> { + let langs = self.list_langs()?; + *self.langs.lock().unwrap_or_else(|e| e.into_inner()) = langs; + self.probed.store(1, Ordering::Relaxed); + Ok(()) + } +} + +impl Recognizer for TesseractCli { + fn id(&self) -> &'static str { + "tesseract" + } + + /// Whether the program answers `--list-langs` (remembered; one process at most after the + /// first call). + fn available(&self) -> bool { + match self.probed.load(Ordering::Relaxed) { + 1 => true, + 2 => false, + _ => self.refresh().is_ok(), + } + } + + fn languages(&self) -> Vec<(String, String)> { + if self.probed.load(Ordering::Relaxed) == 2 { + return Vec::new(); + } + let langs = self.langs.lock().unwrap_or_else(|e| e.into_inner()); + if langs.is_empty() { + drop(langs); + return self.refresh().map_or(Vec::new(), |()| self.langs.lock().unwrap_or_else(|e| e.into_inner()).clone()); + } + langs.clone() + } + + /// Write the page as a PNG and read it back as TSV. + fn recognize(&self, image: &OcrImage, options: &RecognizeOptions) -> Result, OcrError> { + let lang = language_code(&options.language); + let png = encode_png(image)?; + let dir = TempDir::create()?; + let png_path = dir.path.join("page.png"); + { + let mut f = std::fs::File::create(&png_path).map_err(|e| OcrError::Process(format!("creating the image file failed: {e}")))?; + set_private(&mut f); + f.write_all(&png).map_err(|e| OcrError::Process(format!("writing the image failed: {e}")))?; + } + // Fixed argv, no shell: tesseract stdout -l tsv. + let tsv = self.run(&[&png_path.to_string_lossy(), "stdout", "-l", lang, "tsv"], self.timeout, self.max_output)?; + let (lines, truncated) = parse_tsv_bounded(&tsv); + if truncated { + return Err(OcrError::Recognize(format!("the {MAX_WORDS}-word limit was exceeded; the incomplete reading was rejected"))); + } + Ok(lines) + } +} + +/// Encode an RGBA raster as PNG (the interchange format Tesseract reads best). +fn encode_png(image: &OcrImage) -> Result, OcrError> { + let mut out = Vec::new(); + let mut enc = png::Encoder::new(&mut out, image.width, image.height); + enc.set_color(png::ColorType::Rgba); + enc.set_depth(png::BitDepth::Eight); + let mut writer = enc.write_header().map_err(|e| OcrError::Process(format!("encoding the page image failed: {e}")))?; + writer.write_image_data(&image.rgba).map_err(|e| OcrError::Process(format!("encoding the page image failed: {e}")))?; + writer.finish().map_err(|e| OcrError::Process(format!("encoding the page image failed: {e}")))?; + Ok(out) +} + +/// Make a just-created file owner-only, where the OS supports it (Unix). Windows relies on the +/// user's private %TEMP% ACLs; the fresh directory keeps other sessions' files out of sight. +fn set_private(f: &mut std::fs::File) { + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let _ = f.set_permissions(std::fs::Permissions::from_mode(0o600)); + } + #[cfg(not(unix))] + let _ = f; +} + +/// A fresh directory under the system temp dir, deleted (with its contents) when dropped. +/// On Unix it is created 0700, so the page image inside is private. +struct TempDir { + path: PathBuf, +} + +impl TempDir { + fn create() -> Result { + // The recognizer is shared across threads (`Arc`), so two `recognize` calls can run at + // once, and on a coarse clock they can read the same nanosecond. One counter value per + // attempt keeps the names of one process apart without trusting the clock. + static SEQ: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0); + let nanos = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_nanos()).unwrap_or(0); + // Created owner-only in one step on Unix (no window in which the directory exists with + // the default, wider mode); `create` refuses a path that already exists. + let mut builder = std::fs::DirBuilder::new(); + #[cfg(unix)] + { + use std::os::unix::fs::DirBuilderExt; + builder.mode(0o700); + } + // Elsewhere the default mode stands (Windows relies on the user's private %TEMP% ACLs). + #[cfg(not(unix))] + let _ = &mut builder; + // A leftover directory of an earlier process with the same pid can still take a name; + // that one collision is retried with the next counter value, anything else is fatal. + for _ in 0..4 { + let seq = SEQ.fetch_add(1, std::sync::atomic::Ordering::Relaxed); + let path = std::env::temp_dir().join(format!("pdfcraft-ocr-{}-{nanos}-{seq}", std::process::id())); + match builder.create(&path) { + Ok(()) => return Ok(TempDir { path }), + Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => continue, + Err(e) => return Err(OcrError::Process(format!("creating the private temp directory failed: {e}"))), + } + } + Err(OcrError::Process("creating the private temp directory failed: every candidate name already exists".into())) + } +} + +impl Drop for TempDir { + fn drop(&mut self) { + let _ = std::fs::remove_dir_all(&self.path); + } +} + +#[cfg(test)] +mod tests { + use super::*; + // The Unix process tests below build fake scripts and name their paths. + #[cfg(unix)] + use std::path::Path; + + /// Several directories alive at once never share a name: the recognizer is shared across + /// threads (`Arc`), so two `recognize` calls can create their directories concurrently, + /// and the counter in the name — not the clock alone — tells them apart. + #[test] + fn temp_directories_alive_at_once_have_distinct_names() { + let dirs: Vec = (0..32).map(|_| TempDir::create().expect("create")).collect(); + let distinct: std::collections::BTreeSet = dirs.iter().map(|d| d.path.clone()).collect(); + assert_eq!(distinct.len(), dirs.len(), "{:?}", dirs.iter().map(|d| &d.path).collect::>()); + } + + /// A tiny image with one dark square (a synthetic "word"); the Unix process tests use it. + #[cfg(unix)] + fn blot(w: u32, h: u32, x0: u32, y0: u32, bw: u32, bh: u32) -> OcrImage { + let mut rgba = vec![255u8; (w * h * 4) as usize]; + for y in y0..(y0 + bh).min(h) { + for x in x0..(x0 + bw).min(w) { + let o = ((y * w + x) * 4) as usize; + rgba[o..o + 3].copy_from_slice(&[0, 0, 0]); + } + } + OcrImage::new(w, h, rgba).unwrap() + } + + /// The literal TSV fixture: two header rows (conf −1) and two lines of words. + const TSV: &str = "level\tpage\tblock\tpar\tline\tword\tleft\ttop\twidth\theight\tconf\ttext +1\t1\t0\t0\t0\t0\t0\t0\t100\t50\t-1\t +5\t1\t1\t1\t1\t1\t10\t10\t40\t12\t96.5\tHello +5\t1\t1\t1\t1\t2\t55\t10\t30\t12\t91.0\tworld +5\t1\t1\t2\t2\t1\t10\t30\t20\t12\t42.123\tsecond"; + + #[test] + fn tsv_parses_words_into_lines() { + let lines = parse_tsv(TSV); + assert_eq!(lines.len(), 2, "two (block, par, line) groups: {lines:?}"); + let first = &lines[0].words; + assert_eq!(first.len(), 2); + assert_eq!(first[0].text, "Hello"); + assert_eq!(first[0].rect, [10.0, 10.0, 50.0, 22.0], "left + width, top + height"); + assert_eq!(first[0].confidence, Some(96.5)); + assert_eq!(first[0].source, "tesseract"); + assert_eq!(first[1].text, "world"); + assert_eq!(lines[1].words.len(), 1, "a new par starts a new line"); + assert_eq!(lines[1].words[0].text, "second"); + assert_eq!(lines[1].words[0].confidence, Some(42.123)); + } + + /// Conf −1 rows and level ≠ 5 rows are structure, not words; blank text is dropped. + #[test] + fn tsv_treats_header_rows_and_blank_text_as_not_words() { + let tsv = "5\t1\t1\t1\t1\t1\t0\t0\t10\t10\t-1\tskipped\n\ + 4\t1\t1\t1\t1\t0\t0\t0\t10\t10\t95.0\tline row\n\ + 5\t1\t1\t1\t1\t1\t0\t0\t10\t10\t95.0\t \n\ + 5\t1\t1\t1\t1\t2\t0\t0\t10\t10\t88\tkept"; + let lines = parse_tsv(tsv); + assert_eq!(lines.len(), 1, "{lines:?}"); + assert_eq!(lines[0].words.len(), 1); + assert_eq!(lines[0].words[0].text, "kept"); + } + + /// Garbage rows (short, non-numeric) are skipped; CRLF endings survive; boxes saturate + /// instead of overflowing. + #[test] + fn tsv_skips_garbage_and_never_panics() { + let lines = parse_tsv("not a tsv row\n\n5\t1\t1\n5\t1\t1\t1\t1\t1\tx\ty\tw\th\tzzz\ttext\r\n"); + assert!(lines.is_empty(), "{lines:?}"); + let big = parse_tsv(&format!("5\t1\t1\t1\t1\t1\t{}\t0\t{}\t10\t50\tbig", u32::MAX, u32::MAX)); + let rect = big[0].words[0].rect; + assert!(rect[2].is_finite() && rect[3].is_finite(), "no overflow: {rect:?}"); + // A row whose text field is empty yields no words. + assert!(parse_tsv("5\t1\t1\t1\t1\t1\t1\t1\t2\t2\t50\t\t").is_empty()); + } + + /// Non-finite numbers and negative sizes are not words: a row with NaN/inf coordinates or + /// a negative width/height never reaches the output (and so never reaches the content + /// stream). `1e39` overflows f32 to infinity, so it is refused the same way. + #[test] + fn tsv_rejects_non_finite_and_negative_geometry() { + let row = |left: &str, top: &str, w: &str, h: &str, conf: &str| format!("5\t1\t1\t1\t1\t1\t{left}\t{top}\t{w}\t{h}\t{conf}\tword"); + for (why, r) in [ + ("NaN left", row("NaN", "0", "10", "10", "50")), + ("NaN width", row("0", "0", "NaN", "10", "50")), + ("NaN height", row("0", "0", "10", "NaN", "50")), + ("NaN confidence", row("0", "0", "10", "10", "NaN")), + ("inf left", row("inf", "0", "10", "10", "50")), + ("inf width", row("0", "0", "inf", "10", "50")), + ("1e39 width overflows to inf", row("0", "0", "1e39", "10", "50")), + ("negative width", row("0", "0", "-10", "10", "50")), + ("negative height", row("0", "0", "10", "-10", "50")), + ] { + let lines = parse_tsv(&r); + assert!(lines.is_empty(), "{why}: {lines:?}"); + } + // A sane row still parses. + assert_eq!(parse_tsv(&row("0", "0", "10", "10", "50"))[0].words[0].text, "word"); + } + + /// A reading longer than the word cap is truncated at [`MAX_WORDS`] (a known limit, not an + /// error): the words before the cap are kept, the rest is dropped. + #[test] + fn tsv_truncates_readings_past_the_word_cap() { + let mut tsv = String::new(); + for i in 0..MAX_WORDS + 5 { + tsv.push_str(&format!("5\t1\t1\t1\t1\t{}\t{}\t10\t40\t12\t96.5\tw{i}\n", i % 7 + 1, i % 500)); + } + let lines = parse_tsv(&tsv); + let count: usize = lines.iter().map(|l| l.words.len()).sum(); + assert_eq!(count, MAX_WORDS, "truncated at the cap"); + assert_eq!(lines.last().and_then(|l| l.words.last()).map(|w| w.text.as_str()), Some(&*format!("w{}", MAX_WORDS - 1))); + } + + #[test] + fn language_table_maps_and_falls_back() { + assert_eq!(language_code("en"), "eng"); + assert_eq!(language_code(" EN "), "eng", "trimmed, case-insensitive"); + assert_eq!(language_code("de"), "deu"); + assert_eq!(language_code("ZH"), "chi_sim"); + assert_eq!(language_code("ja"), "jpn"); + assert_eq!(language_code("ko"), "kor"); + assert_eq!(language_code("nl"), "nld"); + assert_eq!(language_code("klingon"), "eng", "unknown → eng"); + assert_eq!(language_code(""), "eng"); + // The security property: whatever the settings say, only allow-listed codes reach a + // process. + for code in ["en", "de", "fr", "es", "it", "pt", "ru", "zh", "ja", "ko", "ar", "nl", "klingon", "", " eng ", "EN/;rm"] { + let mapped = language_code(code); + assert!(LANGUAGE_CODES.iter().any(|(_, v)| *v == mapped), "{code:?} → {mapped:?} is not allow-listed"); + } + } + + #[test] + fn list_langs_come_from_stderr_when_stdout_has_none() { + let listing = "List of available languages in \"/usr/share/tessdata/\" (2):\neng\nosd\n"; + assert_eq!(list_langs_from(listing, ""), ["eng", "osd"]); + assert_eq!(list_langs_from("", listing), ["eng", "osd"], "3.x prints the list on stderr"); + assert_eq!(list_langs_from("", "Error opening data file\n"), Vec::::new(), "a warning is not a language"); + assert_eq!(list_langs_from("eng\n", "deu\n"), ["eng"], "stdout wins when it has the list"); + } + + /// The private temp directory is created owner-only (Unix) and removed on drop. + #[test] + fn temp_dir_is_private_and_removed() { + let dir = TempDir::create().unwrap(); + let path = dir.path.clone(); + assert!(path.is_dir()); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + assert_eq!(std::fs::metadata(&path).unwrap().permissions().mode() & 0o777, 0o700); + } + drop(dir); + assert!(!path.exists()); + } + + /// A set `PDFCRAFT_TESSERACT` decides alone: when it is invalid, [`TesseractCli::find`] does + /// not fall back to a PATH copy (which would run a program the user did not name); the PATH + /// search only runs when nothing is configured. + #[test] + fn an_invalid_configured_program_fails_closed() { + let Ok(real) = std::env::current_exe() else { return }; + let searched = std::cell::Cell::new(false); + let on_path = || { + searched.set(true); + Some(real.clone()) + }; + assert_eq!(locate(None, on_path), Some(real.clone()), "nothing configured: the PATH search decides"); + assert!(searched.get()); + searched.set(false); + assert_eq!(locate(Some(real.clone().into_os_string()), on_path), Some(real.clone()), "a valid location is used as is"); + assert!(!searched.get(), "and the PATH is not searched"); + let dir = real.parent().map(Path::to_path_buf).unwrap_or_default().into_os_string(); + for bad in [std::ffi::OsString::from("relative/tesseract"), std::ffi::OsString::from(""), dir, "/no/such/program-0123456789".into()] { + assert_eq!(locate(Some(bad.clone()), on_path), None, "{bad:?} is invalid, so there is no program"); + assert!(!searched.get(), "{bad:?}: an invalid location never falls back to the PATH"); + } + } + + #[test] + fn the_version_comes_from_stdout_or_stderr() { + assert_eq!(version_line("tesseract 5.4.0\n leptonica-1.84\n", ""), Some("tesseract 5.4.0".into())); + assert_eq!(version_line("", "\ntesseract 4.0.0\n"), Some("tesseract 4.0.0".into()), "older versions print it on stderr"); + assert_eq!(version_line(" \n", ""), None); + } + + #[test] + fn list_langs_output_parses() { + let out = "List of available languages in \"C:\\Program Files\\Tesseract-OCR\\tessdata/\" (2):\neng\nosd\n"; + assert_eq!(parse_list_langs(out), vec!["eng".to_string(), "osd".to_string()]); + assert_eq!(parse_list_langs("List of available languages (1):\neng\n"), vec!["eng".to_string()]); + assert!(parse_list_langs("").is_empty()); + } + + /// The real binary, when it is installed: a run end to end, with words that carry a + /// confidence and the engine's name (skipped where `tesseract` is missing). + #[test] + fn recognize_runs_the_installed_binary() { + let Some(t) = TesseractCli::find() else { + eprintln!("skipped: the tesseract program is not installed"); + return; + }; + assert_eq!(t.id(), "tesseract"); + assert!(t.available()); + // 300 dpi-ish block capitals, black on white. + let (w, h, rgba) = hello_world_image(); + let image = OcrImage::new(w, h, rgba).unwrap(); + let lines = t.recognize(&image, &RecognizeOptions::default()).unwrap(); + let text: Vec = lines.iter().map(Line::text).collect(); + assert!(text.iter().any(|t| t.to_uppercase().contains("HELL")), "{text:?}"); + let words: Vec<&Word> = lines.iter().flat_map(|l| &l.words).collect(); + assert!(words.iter().all(|w| w.source == "tesseract")); + assert!(words.iter().all(|w| matches!(w.confidence, Some(c) if (0.0..=100.0).contains(&c))), "{words:?}"); + } + + /// `available`/`languages` come from `--list-langs` (skipped without the binary). + #[test] + fn languages_come_from_list_langs() { + let Some(t) = TesseractCli::find() else { + eprintln!("skipped: the tesseract program is not installed"); + return; + }; + let langs = t.languages(); + assert!(!langs.is_empty()); + assert!(langs.iter().all(|(c, n)| !c.is_empty() && !n.is_empty()), "{langs:?}"); + } + + #[cfg(unix)] + #[test] + fn recognize_refuses_a_truncated_reading() { + let dir = TempDir::create().unwrap(); + let script = dir.path.join("synthetic-recognizer"); + let mut tsv = String::new(); + for i in 0..=MAX_WORDS { + use std::fmt::Write as _; + writeln!(tsv, "5\t1\t1\t1\t1\t{i}\t0\t0\t1\t1\t95\tword{i}").unwrap(); + } + let exact = tsv.lines().take(MAX_WORDS).collect::>().join("\n"); + std::fs::write(dir.path.join("reading.tsv"), tsv).unwrap(); + std::fs::write(&script, "#!/bin/sh\ncat \"$(dirname \"$0\")/reading.tsv\"\n").unwrap(); + make_executable(&script); + let recognizer = TesseractCli::new(script); + let result = recognizer.recognize(&blot(8, 8, 0, 0, 4, 4), &RecognizeOptions::default()); + let status = result.as_ref().map(|lines| lines.iter().map(|line| line.words.len()).sum::()).map_err(ToString::to_string); + assert!( + matches!(result, Err(OcrError::Recognize(ref reason)) if reason.contains("word limit")), + "a capped reading must be reported, never accepted as complete: {status:?}" + ); + std::fs::write(dir.path.join("reading.tsv"), exact).unwrap(); + let complete = recognizer.recognize(&blot(8, 8, 0, 0, 4, 4), &RecognizeOptions::default()).unwrap(); + assert_eq!(complete.iter().map(|line| line.words.len()).sum::(), MAX_WORDS); + } + + /// A hung binary is killed at the timeout and reported (Unix: a fake `tesseract` script; + /// this test needs a real process, so it is off the web and off Windows). + #[cfg(unix)] + #[test] + fn a_hung_program_is_killed_at_the_timeout() { + let dir = std::env::temp_dir().join(format!("pdfcraft-ocr-test-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let script = dir.join("tesseract-hang"); + std::fs::write(&script, "#!/bin/sh\nsleep 30\n").unwrap(); + make_executable(&script); + let mut t = TesseractCli::new(script); + t.timeout = Duration::from_millis(500); + let started = Instant::now(); + let r = t.recognize(&blot(8, 8, 0, 0, 4, 4), &RecognizeOptions::default()); + assert!(matches!(r, Err(OcrError::Timeout(_))), "{r:?}"); + assert!(started.elapsed() < Duration::from_secs(10), "killed, not waited out"); + let _ = std::fs::remove_dir_all(&dir); + } + + /// Output past the cap kills the run EARLY, while the child is still writing (Unix only, + /// same fake-script trick): the script floods far past the cap and would sleep for 30 s + /// after; a cap checked only after the exit would come back as a timeout instead. + #[cfg(unix)] + #[test] + fn flooding_output_is_capped_and_killed_early() { + let dir = std::env::temp_dir().join(format!("pdfcraft-ocr-test-cap-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let script = dir.join("tesseract-flood"); + std::fs::write(&script, "#!/bin/sh\nhead -c 100000000 /dev/zero | tr '\\0' 'x'\nsleep 30\n").unwrap(); + make_executable(&script); + let mut t = TesseractCli::new(script); + t.max_output = 1024; + t.timeout = Duration::from_secs(20); + let started = Instant::now(); + let r = t.recognize(&blot(8, 8, 0, 0, 4, 4), &RecognizeOptions::default()); + assert!(matches!(r, Err(OcrError::OutputTooLarge(_))), "{r:?}"); + assert!(started.elapsed() < Duration::from_secs(10), "killed early, at {:?}", started.elapsed()); + let _ = std::fs::remove_dir_all(&dir); + } + + /// An stderr flood does not block the child, so it does not cause a false timeout (Unix + /// only): the script writes megabytes of stderr — far more than a pipe would swallow — + /// then exits successfully with a small output, which must come back as a reading. + #[cfg(unix)] + #[test] + fn an_stderr_flood_does_not_cause_a_false_timeout() { + let dir = std::env::temp_dir().join(format!("pdfcraft-ocr-test-err-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let script = dir.join("tesseract-err-flood"); + // (below MAX_STDERR, past which the child is stopped: `an_stderr_flood_is_killed_while_the_child_runs`) + std::fs::write(&script, "#!/bin/sh\nhead -c 200000 /dev/zero | tr '\\0' 'e' >&2\necho flooded away\n").unwrap(); + make_executable(&script); + let mut t = TesseractCli::new(script); + t.timeout = Duration::from_secs(10); + let r = t.recognize(&blot(8, 8, 0, 0, 4, 4), &RecognizeOptions::default()); + assert!(r.is_ok(), "stderr flood must not fail the run: {r:?}"); + let _ = std::fs::remove_dir_all(&dir); + } + + /// The child runs with a minimal environment (its data prefix, the PATH, the Windows + /// system root, and a temp dir), never the caller's whole environ. This is checked with a + /// script that prints its own environ as the "TSV" (Unix only). + #[cfg(unix)] + #[test] + fn the_child_gets_a_minimal_environment() { + let dir = std::env::temp_dir().join(format!("pdfcraft-ocr-test-env-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let script = dir.join("tesseract-env"); + std::fs::write(&script, "#!/bin/sh\nenv | grep -v '^_='\n").unwrap(); + make_executable(&script); + let t = TesseractCli::new(script); + let png = encode_png(&blot(8, 8, 0, 0, 4, 4)).unwrap(); + std::fs::write(dir.join("page.png"), png).unwrap(); + let png_arg = dir.join("page.png").to_string_lossy().into_owned(); + let tsv = t.run(&[png_arg.as_str(), "stdout", "-l", "eng", "tsv"], t.timeout, t.max_output).unwrap(); + for (why, name) in [("the child's temp dir", "TMP="), ("temp", "TEMP="), ("lookup path", "PATH=")] { + assert!(tsv.contains(name), "{why} must be passed: {tsv}"); + } + // cargo always sets CARGO_* for the test process; env_clear must drop them. + assert!(!tsv.contains("CARGO_"), "the caller's environment must not leak: {tsv}"); + let _ = std::fs::remove_dir_all(&dir); + } + + #[cfg(unix)] + fn make_executable(p: &Path) { + use std::os::unix::fs::PermissionsExt; + let _ = std::fs::set_permissions(p, std::fs::Permissions::from_mode(0o755)); + } + + /// The temp directory and its contents are gone after a run (private files do not linger). + #[cfg(unix)] + #[test] + fn temp_files_are_deleted_and_private() { + let dir = TempDir::create().unwrap(); + assert!(dir.path.is_dir()); + std::fs::write(dir.path.join("x"), b"secret").unwrap(); + let p = dir.path.clone(); + drop(dir); + assert!(!p.exists(), "the temp directory is deleted after use"); + } + + /// A configured program location is only usable as an absolute path to an existing, + /// executable file: a relative path, a missing file and a directory are unavailable (and + /// [`TesseractCli::at`] refuses them without spawning anything). + #[test] + fn a_configured_program_must_be_an_absolute_executable_file() { + assert!(!program_is_executable(Path::new(".")), "relative"); + assert!(!program_is_executable(Path::new("definitely/missing/tesseract")), "missing"); + let temp = std::env::temp_dir(); + assert!(!program_is_executable(&temp), "a directory is not a program"); + let file = temp.join(format!("pdfcraft-ocr-not-a-program-{}", std::process::id())); + std::fs::write(&file, b"").unwrap(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let _ = std::fs::set_permissions(&file, std::fs::Permissions::from_mode(0o644)); + assert!(!program_is_executable(&file), "a file without executable bits is not a program"); + } + #[cfg(not(unix))] + assert!(program_is_executable(&file), "an existing file is usable on this platform"); + let _ = std::fs::remove_file(&file); + // `at` validates before any process starts, so these are None and not panics. + assert!(TesseractCli::at(PathBuf::from(".")).is_none()); + assert!(TesseractCli::at(temp).is_none()); + assert!(TesseractCli::at(PathBuf::from("definitely/missing/tesseract")).is_none()); + } + + /// The PATH search resolves through absolute entries only and skips relative ones: a + /// relative PATH entry would resolve against the caller's working directory. + #[test] + fn path_search_skips_relative_entries() { + let dir = std::env::temp_dir().join(format!("pdfcraft-ocr-path-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let program = dir.join(program_file_names()[0]); + std::fs::write(&program, b"").unwrap(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let _ = std::fs::set_permissions(&program, std::fs::Permissions::from_mode(0o755)); + } + let found = path_search_in([PathBuf::from("relative/dir"), PathBuf::from("definitely/missing"), dir.clone()].into_iter()); + assert_eq!(found.as_deref(), Some(program.as_path()), "the absolute entry with the program wins, relative entries are skipped"); + let none = path_search_in([PathBuf::from("relative/dir")].into_iter()); + assert_eq!(none, None, "only relative entries: nothing is found"); + let _ = std::fs::remove_dir_all(&dir); + } + + /// "HELLO" as blocky 5×7 glyphs, 8 pixels per dot with wide tracking, at roughly 300 dpi + /// scale (each dot ~10 px): big enough for Tesseract's word detector. + fn hello_world_image() -> (u32, u32, Vec) { + const GLYPHS: &[&str] = &[ + "X...X", "X...X", "X...X", "XXXXX", "X...X", "X...X", "X...X", // H + ]; + // Draw the five glyphs of HELLO with per-letter shapes for E, L, L, O. + const E: &[&str] = &["XXXXX", "X....", "X....", "XXXX.", "X....", "X....", "XXXXX"]; + const L: &[&str] = &["X....", "X....", "X....", "X....", "X....", "X....", "XXXXX"]; + const O: &[&str] = &[".XXX.", "X...X", "X...X", "X...X", "X...X", "X...X", ".XXX."]; + let letters: [&[&str]; 5] = [GLYPHS, E, L, L, O]; + let (dot, x0, y0, advance) = (8u32, 40u32, 40u32, 7u32 * 8); + let w = x0 * 2 + advance * 5; + let h = y0 * 2 + 7 * dot; + let mut rgba = vec![255u8; (w * h * 4) as usize]; + for (i, rows) in letters.iter().enumerate() { + for (ry, row) in rows.iter().enumerate() { + for (rx, b) in row.bytes().enumerate() { + if b != b'X' { + continue; + } + for dy in 0..dot { + for dx in 0..dot { + let x = x0 + i as u32 * advance + rx as u32 * dot + dx; + let y = y0 + ry as u32 * dot + dy; + let o = ((y * w + x) * 4) as usize; + rgba[o..o + 3].copy_from_slice(&[0, 0, 0]); + } + } + } + } + } + (w, h, rgba) + } +} + +#[cfg(all(test, unix))] +mod process_tests { + use super::*; + + /// A stderr flood is killed at the cap while the child still runs, not accepted until it + /// exits (an endless writer would otherwise run out the whole timeout and fill the disk). + /// The fixture needs a Unix shell, like the other process fixtures. + #[test] + fn a_stderr_flood_is_killed_while_the_child_runs() { + let dir = std::env::temp_dir().join(format!("pdfcraft-tesseract-flood-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let program = dir.join("flood-tesseract"); + std::fs::write(&program, "#!/bin/sh\nwhile true; do echo aaaaaaaaaaaaaaaaaaaaaaaa >&2; done\n").unwrap(); + use std::os::unix::fs::PermissionsExt; + let mut perms = std::fs::metadata(&program).unwrap().permissions(); + perms.set_mode(0o755); + std::fs::set_permissions(&program, perms).unwrap(); + let t = TesseractCli::new(program); + let started = std::time::Instant::now(); + let err = t.run_capture(&["--list-langs"], Duration::from_secs(60), 1024 * 1024).unwrap_err(); + assert!(matches!(err, OcrError::OutputTooLarge(_)), "{err}"); + assert!(started.elapsed() < Duration::from_secs(30), "killed at the cap, not at the timeout"); + let _ = std::fs::remove_dir_all(&dir); + } +} diff --git a/crates/redact/src/tests.rs b/crates/redact/src/tests.rs index d463303fcd..78cff889d3 100644 --- a/crates/redact/src/tests.rs +++ b/crates/redact/src/tests.rs @@ -3953,6 +3953,41 @@ fn a_reply_on_another_page_to_a_removed_popup_goes_too() { } } +/// An invisible OCR-style text layer drawn in a Type3 font (what the OCR output writes for +/// characters beyond WinAnsi) is redacted like any other text: its glyphs are hit-tested by +/// their real advances, removed, and the verifier passes. +#[test] +fn a_type3_text_layer_is_redacted_like_any_other() { + // One glyph (code 1): a filled box, 500 units wide, with a ToUnicode CMap that reads it + // back as U+3042. + let t3 = b"<< /Type /Font /Subtype /Type3 /Name /PCOcr /FontBBox [0 0 700 700] /FontMatrix [0.001 0 0 0.001 0 0] \ + /FirstChar 1 /LastChar 1 /Widths [500] /Encoding << /Type /Encoding /Differences [1 /g01] >> \ + /CharProcs << /g01 8 0 R >> /ToUnicode 9 0 R >>"; + let charproc = b"500 0 0 0 0 700 700 d1\n0 0 700 700 re f\n"; + let cmap = b"/CIDInit /ProcSet findresource begin\n12 dict begin\nbegincmap\n/CMapType 2 def\n1 begincodespacerange\n<01> <01>\nendcodespacerange\n1 beginbfchar\n<01> <3042>\nendbfchar\nendcmap\nCMapName currentdict /CMap defineresource pop\nend\nend\n"; + let layer = b"BT 3 Tr /T3 10 Tf 100 100 Td (\x01) Tj ET"; + let mut doc = pdf(vec![ + b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(), + b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(), + b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /T3 5 0 R >> >> >>".to_vec(), + stream("", layer), + t3.to_vec(), + FONT.replace("95 0 R", "6 0 R").into_bytes(), + widths(), + stream("", charproc), + stream("", cmap), + ]); + // The glyph box at size 10: 5 pt wide, from the font's descent to its ascent. + assert_eq!(under(&mut doc, 0, &[[98.0, 96.0, 112.0, 110.0]]), 1, "the Type3 glyph is found under the mark"); + assert_eq!(under(&mut doc, 0, &[[200.0, 200.0, 250.0, 250.0]]), 0, "elsewhere the page is clean"); + mark(&mut doc, 0, &[[98.0, 96.0, 112.0, 110.0]], ""); + let report = apply(&mut doc, None).unwrap(); + assert_eq!(report.glyphs, 1, "{report:?}"); + // The verified apply left no text-showing operator: the re-read page is empty of them. + assert!(!content(&doc, 0).contains("Tj"), "{}", content(&doc, 0)); + assert_eq!(under(&mut doc, 0, &[[98.0, 96.0, 112.0, 110.0]]), 0, "nothing is left under the mark"); +} + #[test] fn a_rewritten_form_xobject_drops_the_source_text_of_added_content() { // A form drawing invisible text whose dictionary repeats it in /PCAdded. diff --git a/crates/ui-egui/Cargo.toml b/crates/ui-egui/Cargo.toml index c859b93cf6..22e7323476 100644 --- a/crates/ui-egui/Cargo.toml +++ b/crates/ui-egui/Cargo.toml @@ -23,6 +23,7 @@ pdfcraft-fonts = { workspace = true } pdfcraft-engine = { workspace = true } pdfcraft-cos = { workspace = true } pdfcraft-content = { workspace = true } +pdfcraft-ocr = { workspace = true } pdfcraft-render = { workspace = true } pdfcraft-model = { workspace = true } pdfcraft-platform = { workspace = true } diff --git a/crates/ui-egui/examples/shot.rs b/crates/ui-egui/examples/shot.rs index 01e48e03d6..8058df40c8 100644 --- a/crates/ui-egui/examples/shot.rs +++ b/crates/ui-egui/examples/shot.rs @@ -6,18 +6,64 @@ //! `--width N` downscales the image to N pixels wide (README screenshots). Any other //! `--key value` pair is passed to `PdfCraftApp::set_option` //! (the same verbs as the desktop app's command-line flags and the future control channel). +//! `--ocr-verify on` ignores any file and instead builds a synthetic scanned page, recognizes +//! it with the real engines (models required: `cargo xtask models`) and opens the OCR Verify +//! screen on the result, as a reviewer sees it. use std::time::{Duration, Instant}; use egui_kittest::Harness; +use pdfcraft_engine::ocr::{MergeStrategy, OcrSettings}; +use pdfcraft_engine::{Session, export}; use pdfcraft_ui_egui::PdfCraftApp; +/// Build a one-page PDF that is only a picture of a sentence (as a scanner would produce), +/// recognize it with the engines named in `settings` and open the OCR Verify screen on the +/// result — the same path the review tests use, with a real recognition behind it. The toolbar +/// dropdowns are seeded to the ensemble the shot ran. +fn load_ocr_verify(app: &mut PdfCraftApp) -> Result<(), String> { + let mut s = Session::new(); + let text = s + .create_from_text( + "t", + "Scanned pages are pictures of words.\nThe Verify screen reads them back before anything is applied.\nReview the uncertain words, then Accept.", + ) + .map_err(|e| e.to_string())?; + let id = s.open("text.pdf", None, text, None).map_err(|e| e.to_string())?; + let png = export::Exporter::new(s.get(id).ok_or("the document is gone")?).png(0, 150.0).map_err(|e| e.to_string())?; + // Keep the text block of the rendered page, so the scan is a short note-sized picture: + // the whole page and the confidence legend fit in the shot's source-page pane. + let mut page = image::load_from_memory(&png).map_err(|e| e.to_string())?; + let mut scan_png = std::io::Cursor::new(Vec::new()); + page.crop(0, 0, page.width(), (page.height() as f32 * 0.34) as u32) + .write_to(&mut scan_png, image::ImageFormat::Png) + .map_err(|e| e.to_string())?; + let scan = s.create_from_images(&[("scan.png".into(), scan_png.into_inner())]).map_err(|e| e.to_string())?.to_vec(); + let settings = + OcrSettings { engine: pdfcraft_engine::ocr::EngineChoice::Auto, strategy: MergeStrategy::ConfidenceWeighted, ..OcrSettings::default() }; + app.ocr_draft.engine = settings.engine; + app.ocr_draft.strategy = settings.strategy; + app.open_bytes("scan.pdf", None, scan).map_err(|e| e.to_string())?; + app.run_inline = true; + if !app.execute("ocr.correct") { + return Err("ocr.correct did not run".into()); + } + let Some((_, id)) = app.active_ids() else { return Err("no open document".into()) }; + let job = app.session.ocr_job(id, &[], settings.clone()).ok_or("the OCR job could not be captured")?; + let recognizers = pdfcraft_engine::ocr::recognizers(&settings)?; + let guard = job.guard.clone(); + let pages = job.run(&recognizers, |_, _| true); + app.ocr_verify.load(id, guard, pages); + Ok(()) +} + fn main() -> Result<(), String> { let mut args = std::env::args().skip(1); let out = args.next().ok_or("usage: shot [file.pdf] [--option value …]")?; let (mut file, mut opts) = (None, Vec::new()); let (mut size, mut scale) = (egui::vec2(1440.0, 900.0), 2.0f32); let mut width: Option = None; + let mut ocr_verify = false; while let Some(a) = args.next() { match a.strip_prefix("--") { Some("size") => { @@ -27,13 +73,20 @@ fn main() -> Result<(), String> { } Some("scale") => scale = args.next().and_then(|v| v.parse().ok()).ok_or("bad --scale")?, Some("width") => width = Some(args.next().and_then(|v| v.parse().ok()).ok_or("bad --width")?), + Some("ocr-verify") => ocr_verify = args.next().is_some_and(|v| v == "on"), Some(k) => opts.push((k.to_string(), args.next().unwrap_or_default())), None => file = Some(a), } } let mut harness = Harness::builder().with_size(size).with_pixels_per_point(scale).build_eframe(move |_cc| { let mut app = PdfCraftApp::new(); - if let Some(f) = &file { + if ocr_verify { + if let Err(e) = load_ocr_verify(&mut app) { + eprintln!("shot: --ocr-verify: {e}"); + // an empty Verify screen must not pass for the documentation's screenshot + std::process::exit(1); + } + } else if let Some(f) = &file { let bytes = std::fs::read(f).unwrap_or_default(); let name = std::path::Path::new(f).file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default(); if let Err(e) = app.open_bytes(&name, Some(f.clone()), bytes) { diff --git a/crates/ui-egui/src/app.rs b/crates/ui-egui/src/app.rs index 61b85cd7f1..3b3d5dd036 100644 --- a/crates/ui-egui/src/app.rs +++ b/crates/ui-egui/src/app.rs @@ -202,6 +202,7 @@ impl PdfCraftApp { ocr_draft: ocr_ui::OcrDraft::default(), ocr_run: None, ocr_batch: None, + ocr_verify: ocr_verify::OcrVerify::default(), run_inline: false, custom_actions: Vec::new(), wizard: Default::default(), @@ -624,6 +625,14 @@ impl eframe::App for PdfCraftApp { self.poll_optimize(); self.poll_scan_clean(); self.poll_action(); + // The OCR Verify screen's worker may have finished while no frame was drawn. + if self.ocr_verify.phase == workflows::ocr_verify::Phase::Running { + workflows::ocr_verify::with_state(self, workflows::ocr_verify::OcrVerify::poll); + } + // So may a page or region re-read's worker. + if self.ocr_verify.reread.is_some() { + workflows::ocr_verify::with_state(self, workflows::ocr_verify::OcrVerify::poll_reread); + } self.process_file_requests(); #[cfg(not(target_arch = "wasm32"))] self.process_picked(); @@ -680,6 +689,20 @@ impl eframe::App for PdfCraftApp { if self.draws_window_frame() { window_controls::resize_edges(&ctx); } + if self.ocr_verify.open { + // The OCR Verify screen owns the window (its four panes); dialogs and notices + // still show above it. + workflows::ocr_verify::show(self, ui); + dialogs::show(self, &ctx); + self.show_progress(&ctx); + widgets::toast(self, &ctx); + self.finish_render_frame(&ctx); + if let Some(mut control) = self.control.take() { + control.finish_frame(&ctx, self); + self.control = Some(control); + } + return; + } chrome::tab_strip(self, ui); chrome::mode_bar(self, ui); // Whichever way another tab came to the front, it shows instead of the expanded view. diff --git a/crates/ui-egui/src/app/prefs.rs b/crates/ui-egui/src/app/prefs.rs index 89789d8a4c..1a8c2d269e 100644 --- a/crates/ui-egui/src/app/prefs.rs +++ b/crates/ui-egui/src/app/prefs.rs @@ -144,6 +144,7 @@ impl PdfCraftApp { "combine_zoom": self.combine_zoom, "comments_panel_closed": self.comments_panel_closed, "right_panel_closed": self.right_panel_closed, + "ocr": self.ocr_draft.to_pref(), }); // Set apart: one more key inside the macro above passes serde_json's recursion limit. v["default_page_gaps"] = serde_json::Value::Bool(self.view_defaults.show_page_gaps); @@ -295,6 +296,9 @@ impl PdfCraftApp { if let Some(closed) = v["right_panel_closed"].as_bool() { self.right_panel_closed = closed; } + // Scan & OCR choices (language, engine, strategy, output mode, preprocessing and skip + // flags); unknown or malformed values keep the defaults. + self.ocr_draft.apply_pref(&v["ocr"]); if let Some(defaults) = v["default_zoom"].as_str().and_then(|zoom| self.view_defaults.with_zoom(zoom)) { self.view_defaults = defaults; } diff --git a/crates/ui-egui/src/commands.rs b/crates/ui-egui/src/commands.rs index 4601986e9a..7c0df87782 100644 --- a/crates/ui-egui/src/commands.rs +++ b/crates/ui-egui/src/commands.rs @@ -527,6 +527,7 @@ impl PdfCraftApp { "ocr.recognize" => self.dialog = Some(Dialog::RecognizeText), "ocr.enhance" => self.dialog = Some(Dialog::EnhanceScan), "ocr.recognize_batch" => self.ocr_files_dialog(), + "ocr.correct" => self.ocr_verify.open = true, "edit.edit_text" => { self.quick_tool = crate::QuickTool::EditText; self.left = crate::LeftPanel::Tool("edit"); diff --git a/crates/ui-egui/src/lib.rs b/crates/ui-egui/src/lib.rs index 94e83f451a..2f393c61d1 100644 --- a/crates/ui-egui/src/lib.rs +++ b/crates/ui-egui/src/lib.rs @@ -69,8 +69,9 @@ use pdfcraft_engine::{DocId, Session}; pub use canvas::{DocView, RasterMemory}; pub use editing::{CloseRequest, SaveTarget}; pub use files::{ExtractDraft, FilePurpose, FileRequest, RotateDraft, SplitDraft, SplitMode, SplitPlan}; -pub use recovery::{AUTOSAVE_SECS, RecoveryMeta, RecoveryStore}; +pub use recovery::{AUTOSAVE_SECS, RecoveryMeta, RecoveryStore, default_dictionary_dir}; use theme::{ThemeKind, ThemePreference}; +pub use workflows::ocr_verify::{OcrVerify, Phase}; mod app; mod hooks; @@ -181,6 +182,7 @@ use workflows::export_ui; use workflows::files; use workflows::js_ui; use workflows::ocr_ui; +use workflows::ocr_verify; use workflows::optimize_ui; use workflows::print_ui; use workflows::scan_clean_ui; @@ -510,6 +512,8 @@ pub struct PdfCraftApp { pub ocr_draft: ocr_ui::OcrDraft, pub ocr_run: Option, pub ocr_batch: Option>>, + /// Scan & OCR ▸ Correct recognized text: the OCR Verify screen (its state, never persisted). + pub ocr_verify: ocr_verify::OcrVerify, /// Background jobs (OCR, actions, listing pinned folders) run inline instead (tests). pub run_inline: bool, /// Action Wizard: the user's actions, the dialog state, the running action and (tests) the diff --git a/crates/ui-egui/src/persist/recovery.rs b/crates/ui-egui/src/persist/recovery.rs index 71759c4599..f217781c07 100644 --- a/crates/ui-egui/src/persist/recovery.rs +++ b/crates/ui-egui/src/persist/recovery.rs @@ -184,7 +184,27 @@ impl RecoveryStore { pub fn dir(&self) -> &Path { &self.dir } +} +/// The OCR Verify screen's per-language user dictionary folder, next to the recovery store: +/// the portable data folder's `Dictionary` in portable mode ([`crate::portable`]), otherwise +/// `…/PdfCraft/Dictionary` in the platform's per-user data folder. `None` where files are not +/// kept (the web build). +pub fn default_dictionary_dir() -> Option { + if let Some(dir) = crate::portable::data_dir() { + return Some(dir.join("Dictionary")); + } + let env = |k: &str| std::env::var_os(k).filter(|v| !v.is_empty()).map(PathBuf::from); + if cfg!(target_os = "macos") { + env("HOME").map(|h| h.join("Library/Application Support/PdfCraft/Dictionary")) + } else if cfg!(windows) { + env("LOCALAPPDATA").map(|d| d.join("PdfCraft").join("Dictionary")) + } else { + env("XDG_DATA_HOME").or_else(|| env("HOME").map(|h| h.join(".local/share"))).map(|d| d.join("pdfcraft/dictionary")) + } +} + +impl RecoveryStore { /// Write (or replace) an entry. The PDF is written before its metadata, both atomically, so /// a listed entry always has complete bytes. pub fn write(&self, meta: &RecoveryMeta, bytes: &[u8]) -> std::io::Result<()> { diff --git a/crates/ui-egui/src/widgets.rs b/crates/ui-egui/src/widgets.rs index c4ca254948..f621c7832b 100644 --- a/crates/ui-egui/src/widgets.rs +++ b/crates/ui-egui/src/widgets.rs @@ -477,6 +477,9 @@ pub fn toast(app: &mut PdfCraftApp, ctx: &egui::Context) { // Above the progress notice while a job runs. let card = if app.progress_notice.is_some() { ctx.data(|d| d.get_temp::(egui::Id::unique(PROGRESS_HEIGHT))).unwrap_or(0.0) } else { 0.0 }; let lift = if card > 0.0 { 28.0 + card + 10.0 } else { 28.0 }; + // A running batch keeps its toast up (poll_ocr re-notifies), so it can carry the one + // control a long job needs: Cancel — the file in flight finishes, the rest are not run. + let batch_running = app.ocr_batch.is_some(); egui::Area::new(egui::Id::unique("toast")) .order(egui::Order::Tooltip) .pivot(Align2::CENTER_BOTTOM) @@ -494,7 +497,12 @@ pub fn toast(app: &mut PdfCraftApp, ctx: &egui::Context) { ui.set_max_width((screen.width() - 64.0).clamp(1.0, 560.0)); let ink = if t.dark() { Color32::from_rgb(0x22, 0x22, 0x26) } else { Color32::WHITE }; let text = |ui: &mut egui::Ui| { - ui.add(egui::Label::new(egui::RichText::new(msg).color(ink)).wrap()); + ui.horizontal_wrapped(|ui| { + ui.add(egui::Label::new(egui::RichText::new(msg).color(ink)).wrap()); + if batch_running && ui.small_button(crate::i18n::t("Cancel")).clicked() { + app.cancel_ocr_batch(); + } + }); }; match script { // A form script's message: its icon, then its title (when it gave one) diff --git a/crates/ui-egui/src/workflows.rs b/crates/ui-egui/src/workflows.rs index 9314914ec4..8701eac119 100644 --- a/crates/ui-egui/src/workflows.rs +++ b/crates/ui-egui/src/workflows.rs @@ -9,6 +9,7 @@ pub(super) mod export_ui; pub(super) mod files; pub(super) mod js_ui; pub(super) mod ocr_ui; +pub(super) mod ocr_verify; pub(super) mod optimize_ui; pub(super) mod print_ui; pub(super) mod scan_clean_ui; diff --git a/crates/ui-egui/src/workflows/ocr_ui.rs b/crates/ui-egui/src/workflows/ocr_ui.rs index 04e257d52d..7534915d10 100644 --- a/crates/ui-egui/src/workflows/ocr_ui.rs +++ b/crates/ui-egui/src/workflows/ocr_ui.rs @@ -6,7 +6,7 @@ use std::sync::{Arc, Mutex}; use egui::{Align, Layout}; use pdfcraft_engine::DocId; -use pdfcraft_engine::ocr::{LANGUAGES, OcrPage, OcrSettings}; +use pdfcraft_engine::ocr::{EngineChoice, LANGUAGES, MergeStrategy, OcrPage, OcrSettings, OutputMode}; use crate::theme::{self, Tokens}; use crate::{PdfCraftApp, widgets}; @@ -19,7 +19,10 @@ pub enum OcrPages { Range, } -/// The dialog's choices (kept between runs, as in Acrobat). +/// The dialog's choices (kept between runs, as in Acrobat). The Recognize Text dialog, the OCR +/// Verify screen and the batch options all read and write this one draft, and it is what the +/// preferences persist: language, output mode, the four preprocessing flags, engine, strategy +/// and the three skip flags — never review edits, verified flags or the selection. #[derive(Clone, Debug, PartialEq)] pub struct OcrDraft { pub pages: OcrPages, @@ -30,18 +33,135 @@ pub struct OcrDraft { pub dpi: u32, /// Also recognize pages that already have text (pictures on them; their text isn't added again). pub text_pages: bool, + /// Which engine reads the pages. + pub engine: EngineChoice, + /// How two engines' readings combine. + pub strategy: MergeStrategy, + /// Preprocessing flags, all off by default. + pub auto_rotate: bool, + pub deskew: bool, + pub denoise: bool, + pub binarize: bool, + /// What Accept produces. + pub output_mode: OutputMode, + /// Batch: leave files that already contain text alone. + pub skip_text_files: bool, + /// Batch: leave pages that already contain text alone. + pub skip_text_pages: bool, + /// Batch: read everything, overriding both skips. + pub force_ocr: bool, + /// Batch: where the searchable copies go (None: ask each time). Remembered from the last + /// batch's folder pick, so a repeated export into the same place does not re-ask. + pub output_folder: Option, } impl Default for OcrDraft { fn default() -> Self { - OcrDraft { pages: OcrPages::All, from: 1, to: 1, language: "en".into(), dpi: 300, text_pages: false } + OcrDraft { + pages: OcrPages::All, + from: 1, + to: 1, + language: "en".into(), + dpi: 300, + text_pages: false, + engine: EngineChoice::default(), + strategy: MergeStrategy::default(), + auto_rotate: false, + deskew: false, + denoise: false, + binarize: false, + output_mode: OutputMode::default(), + skip_text_files: false, + skip_text_pages: true, + force_ocr: false, + output_folder: None, + } } } impl OcrDraft { /// The recognition settings these choices make. pub fn settings(&self) -> OcrSettings { - OcrSettings { dpi: self.dpi as f32, language: self.language.clone(), skip_text_pages: !self.text_pages, ..Default::default() } + OcrSettings { + dpi: self.dpi as f32, + language: self.language.clone(), + engine: self.engine, + strategy: self.strategy, + auto_rotate: self.auto_rotate, + deskew: self.deskew, + denoise: self.denoise, + binarize: self.binarize, + output_mode: self.output_mode, + skip_text_files: self.skip_text_files, + skip_text_pages: self.skip_text_pages && !self.text_pages, + force_ocr: self.force_ocr, + ..Default::default() + } + } + + /// Restore from the preferences JSON (`ocr` object); unknown or malformed values keep the + /// defaults, like every other restored setting. + pub fn apply_pref(&mut self, v: &serde_json::Value) { + if let Some(l) = v["language"].as_str().filter(|l| LANGUAGES.iter().any(|x| x.0 == *l)) { + self.language = l.to_string(); + } + if let Some(e) = v["engine"].as_str() { + self.engine = EngineChoice::parse(e); + } + if let Some(s) = v["strategy"].as_str() { + self.strategy = match s.trim().to_ascii_lowercase().as_str() { + "confidence" | "confidence-weighted" | "confidence_weighted" => MergeStrategy::ConfidenceWeighted, + "rover" | "rover-vote" | "vote" => MergeStrategy::RoverVote, + "primary" | "primary-only" => MergeStrategy::PrimaryOnly, + _ => MergeStrategy::PrimaryOnly, + }; + } else if self.engine == EngineChoice::Auto { + // No strategy was saved with "Automatic": it reads the way Automatic reads by + // default, not as "Primary only" (which would never bring in the second engine). + self.strategy = self.engine.default_strategy(); + } + if let Some(m) = v["output_mode"].as_str() { + self.output_mode = OutputMode::parse(m); + } + for (key, flag) in + [("auto_rotate", &mut self.auto_rotate), ("deskew", &mut self.deskew), ("denoise", &mut self.denoise), ("binarize", &mut self.binarize)] + { + if let Some(on) = v[key].as_bool() { + *flag = on; + } + } + for (key, flag) in + [("skip_text_files", &mut self.skip_text_files), ("skip_text_pages", &mut self.skip_text_pages), ("force_ocr", &mut self.force_ocr)] + { + if let Some(on) = v[key].as_bool() { + *flag = on; + } + } + if let Some(f) = v["output_folder"].as_str().filter(|f| !f.is_empty()) { + self.output_folder = Some(f.to_string()); + } + } + + /// The preferences JSON for `persist`. + pub fn to_pref(&self) -> serde_json::Value { + serde_json::json!({ + "language": self.language, + "engine": self.engine.name(), + "strategy": match self.strategy { + MergeStrategy::PrimaryOnly => "primary", + MergeStrategy::ConfidenceWeighted => "confidence", + MergeStrategy::RoverVote => "rover", + }, + "output_mode": self.output_mode.name(), + "auto_rotate": self.auto_rotate, + "deskew": self.deskew, + "denoise": self.denoise, + "binarize": self.binarize, + "skip_text_files": self.skip_text_files, + "skip_text_pages": self.skip_text_pages, + "force_ocr": self.force_ocr, + "output_folder": self.output_folder, + }) } } @@ -54,12 +174,14 @@ pub struct OcrProgress { pub cancel: bool, } -/// Recognize text in multiple files: files done, total, and the summary once finished. +/// Recognize text in multiple files: files done, total, and the summary once finished. Cancel +/// is asked for between files: the file in flight finishes, the rest are counted as not run. #[derive(Default)] pub struct BatchProgress { pub done: usize, pub total: usize, pub message: Option, + pub cancel: bool, } /// A page Edit should read now (see [`PdfCraftApp::scan_to_read`]). @@ -74,10 +196,16 @@ struct ScanToRead { pub struct OcrRun { pub doc: DocId, pub progress: Arc>, + /// What the document looked like when the job started: a result is applied only while it + /// still does (the dialog closes on Go, so the user can edit, or redact, while this runs). + pub guard: pdfcraft_engine::ocr::OcrGuard, /// Set for a run Edit started on a scanned page, not asked for: it shows no progress, says nothing /// when it found no words (a page with text of its own is skipped, and Edit must not make noise), /// and its result waits for Read mode and dialogs and is dropped if the document changed. pub automatic: Option, + /// The dialog promised an editable document: the result goes to the OCR Verify screen + /// (whose Accept produces the editable document), not to an invisible layer. + pub editable: bool, } /// What an Edit-started run needs to know to refuse a result that no longer fits its document. @@ -90,9 +218,16 @@ pub struct AutoOcr { pub generation: u64, } +/// The Tesseract probe runs in the background (the interface never waits on a process): while +/// it runs, an engine choice reads as "still checking", so look again soon. +pub(crate) fn keep_asking_while_probing(ctx: &egui::Context) { + if pdfcraft_engine::ocr::engine_probe_running() { + ctx.request_repaint_after(std::time::Duration::from_millis(200)); + } +} + pub(crate) fn body(ui: &mut egui::Ui, app: &mut PdfCraftApp, t: &Tokens) -> (bool, bool) { let pages = app.active_ids().and_then(|(_, id)| app.session.get(id)).map_or(1, |d| d.info.pages.len().max(1)); - let available = pdfcraft_engine::ocr::available(); let d = &mut app.ocr_draft; d.to = d.to.clamp(1, pages); d.from = d.from.clamp(1, d.to); @@ -127,11 +262,55 @@ pub(crate) fn body(ui: &mut egui::Ui, app: &mut PdfCraftApp, t: &Tokens) -> (boo } }); ui.end_row(); + ui.label(tl!("Engine")); + let engine_name = match d.engine { + EngineChoice::Auto => tl!("Automatic"), + EngineChoice::Ocrs => tl!("Built-in"), + EngineChoice::Tesseract => tl!("Tesseract"), + }; + craft_ui::ComboBox::from_id_salt("ocr-engine").selected_text(engine_name).width(220.0).show_ui(ui, |ui| { + let choices: [(EngineChoice, &str); 3] = + [(EngineChoice::Auto, "Automatic"), (EngineChoice::Ocrs, "Built-in"), (EngineChoice::Tesseract, "Tesseract")]; + for (choice, name) in choices { + ui.selectable_value(&mut d.engine, choice, tl!(name)); + } + }); + ui.end_row(); + ui.label(tl!("Combine engines")); + let strategy_name = match d.strategy { + MergeStrategy::PrimaryOnly => tl!("Primary only"), + MergeStrategy::ConfidenceWeighted => tl!("Confidence weighted"), + MergeStrategy::RoverVote => tl!("ROVER vote"), + }; + craft_ui::ComboBox::from_id_salt("ocr-strategy").selected_text(strategy_name).width(220.0).show_ui(ui, |ui| { + let choices: [(MergeStrategy, &str); 3] = [ + (MergeStrategy::PrimaryOnly, "Primary only"), + (MergeStrategy::ConfidenceWeighted, "Confidence weighted"), + (MergeStrategy::RoverVote, "ROVER vote"), + ]; + for (choice, name) in choices { + ui.selectable_value(&mut d.strategy, choice, tl!(name)); + } + }); + ui.end_row(); ui.label(tl!("Output")); - craft_ui::ComboBox::from_id_salt("ocr-output").selected_text(tl!("Searchable Image (Exact)")).width(220.0).show_ui(ui, |ui| { - let _ = ui - .selectable_label(true, tl!("Searchable Image (Exact)")) + let mode_name = match d.output_mode { + OutputMode::Searchable => tl!("Searchable Image (Exact)"), + OutputMode::EditableText => tl!("Editable text"), + }; + craft_ui::ComboBox::from_id_salt("ocr-output").selected_text(mode_name).width(220.0).show_ui(ui, |ui| { + let s = ui + .selectable_label(d.output_mode == OutputMode::Searchable, tl!("Searchable Image (Exact)")) .on_hover_text(tl!("Adds invisible text over each word; the page image is not changed")); + if s.clicked() { + d.output_mode = OutputMode::Searchable; + } + let e = ui + .selectable_label(d.output_mode == OutputMode::EditableText, tl!("Editable text")) + .on_hover_text(tl!("Writes a new document of visible text; the scan is not changed")); + if e.clicked() { + d.output_mode = OutputMode::EditableText; + } }); ui.end_row(); ui.label(tl!("Downsample to")); @@ -145,20 +324,50 @@ pub(crate) fn body(ui: &mut egui::Ui, app: &mut PdfCraftApp, t: &Tokens) -> (boo ui.add_space(4.0); ui.add(crate::widgets::checkbox(&mut d.text_pages, tl!("Also recognize pages that already have text"))) .on_hover_text(tl!("Pictures on those pages are recognized too; words already in the page's text are not added again")); + ui.add(crate::widgets::checkbox(&mut d.auto_rotate, tl!("Straighten pages scanned sideways or upside down"))); + ui.add(crate::widgets::checkbox(&mut d.deskew, tl!("Straighten skewed text lines"))); + ui.add(crate::widgets::checkbox(&mut d.denoise, tl!("Remove scan speckle"))); + ui.add(crate::widgets::checkbox(&mut d.binarize, tl!("Black and white (binarize)"))); }); - if !available { - ui.label( - egui::RichText::new(tl!("Text recognition isn't installed: its model files are missing. Reinstall PdfCraft, or set PDFCRAFT_MODELS to the folder that holds them.")) - .small() - .color(t.text_muted), - ); + group(ui, "Skip", &mut |ui| { + ui.add(crate::widgets::checkbox(&mut d.skip_text_pages, tl!("Skip pages that already have text"))) + .on_hover_text(tl!("Pages with renderable text are left alone and reported as skipped")); + ui.add(crate::widgets::checkbox(&mut d.skip_text_files, tl!("Skip files that already have text"))) + .on_hover_text(tl!("In multiple files: a file with any text is its own skipped bucket, and nothing is written for it")); + ui.add(crate::widgets::checkbox(&mut d.force_ocr, tl!("Force OCR (override skip options)"))) + .on_hover_text(tl!("Reads pages and files even where the skip options would leave them alone")); + }); + group(ui, "Output folder", &mut |ui| { + ui.horizontal(|ui| match &d.output_folder { + Some(f) => { + ui.monospace(f).on_hover_text(tl!("The searchable copies are written here under the source names")); + if ui.small_button(tl!("Clear")).clicked() { + d.output_folder = None; + } + } + None => { + ui.label(tl!("Automatic")).on_hover_text(tl!("A folder is asked for when a multiple-file run starts")); + } + }); + }); + keep_asking_while_probing(ui.ctx()); + if let Some(why) = pdfcraft_engine::ocr::engine_unavailable_reason(d.engine) { + // Said in the interface's language: the built-in engine's missing models get the + // notice users know; Tesseract's reasons are catalog keys of their own. + let text = match d.engine { + pdfcraft_engine::ocr::EngineChoice::Tesseract => crate::i18n::t(&why).to_string(), + _ => tl!("Text recognition isn't installed: its model files are missing. Reinstall PdfCraft, or set PDFCRAFT_MODELS to the folder that holds them.") + .to_string(), + }; + ui.label(egui::RichText::new(text).small().color(t.text_muted)); ui.add_space(6.0); } ui.add_space(6.0); let (mut go, mut cancel) = (false, false); + let can_run = pdfcraft_engine::ocr::engine_unavailable_reason(d.engine).is_none(); ui.horizontal(|ui| { ui.with_layout(Layout::right_to_left(Align::Center), |ui| { - if ui.add_enabled_ui(available, |ui| widgets::pill_button(ui, tl!("Recognize text"), true)).inner.clicked() { + if ui.add_enabled_ui(can_run, |ui| widgets::pill_button(ui, tl!("Recognize text"), true)).inner.clicked() { go = true; } if widgets::pill_button(ui, tl!("Cancel"), false).clicked() { @@ -201,7 +410,9 @@ impl PdfCraftApp { /// is one undoable step, "Recognize text", and is not run again for that page after Undo. pub(crate) fn auto_ocr_scanned_page(&mut self) { let Some(scan) = self.scan_to_read() else { return }; - if !pdfcraft_engine::ocr::available() { + // the engine this run will use, not always the built-in one + let mut settings = self.ocr_draft.settings(); + if pdfcraft_engine::ocr::engine_unavailable_reason(settings.engine).is_some() { self.views[scan.view].auto_ocr_skipped.insert(scan.page, scan.generation); return; } @@ -209,7 +420,6 @@ impl PdfCraftApp { // Only a page with no text of its own is read. Comments and text boxes over the picture do // not count as its text (a scan with a few comments is still a scan, as in Acrobat), but // a page with real text and a background picture is left alone. - let mut settings = self.ocr_draft.settings(); settings.skip_text_pages = true; settings.native_text_only = true; self.start_ocr_pages(scan.doc, &[scan.page], settings, true); @@ -267,7 +477,9 @@ impl PdfCraftApp { } else { None }; - let Some(job) = self.session.ocr_job(id, pages, settings) else { return }; + let Some(job) = self.session.ocr_job(id, pages, settings.clone()) else { return }; + let guard = job.guard.clone(); + let editable = settings.output_mode == OutputMode::EditableText; let progress = Arc::new(Mutex::new(OcrProgress { total: job.pages.len(), ..Default::default() })); let p = progress.clone(); // A failure is written on the worker thread: in the UI's language. @@ -275,8 +487,8 @@ impl PdfCraftApp { let work = move || { crate::i18n::set_current(lang); let result = recognize_guarded(|| { - pdfcraft_engine::ocr::engine().map(|ocr| { - job.run(&ocr, |done, total| { + pdfcraft_engine::ocr::recognizers(&settings).map(|r| { + job.run(&r, |done, total| { let Ok(mut s) = p.lock() else { return false }; s.done = done; s.total = total; @@ -296,7 +508,7 @@ impl PdfCraftApp { } #[cfg(target_arch = "wasm32")] work(); - self.ocr_run = Some(OcrRun { doc: id, progress, automatic: auto }); + self.ocr_run = Some(OcrRun { doc: id, progress, guard, automatic: auto, editable }); self.poll_ocr(); } @@ -307,10 +519,11 @@ impl PdfCraftApp { self.notify_tr("Text recognition is already running"); return; } - // The settings showing now, not when the folder arrives. + // The settings showing now, not when the folder arrives. A remembered folder (the last + // batch's pick, persisted with the preferences) is reused; otherwise the user is asked. let settings = self.ocr_draft.settings(); #[cfg(not(target_arch = "wasm32"))] - match self.export_dir_override.clone() { + match self.export_dir_override.clone().or_else(|| self.ocr_draft.output_folder.clone()) { Some(d) => self.ocr_files_into(files, settings, d.into()), None => { let dialog = rfd::AsyncFileDialog::new().set_title(tl!("Choose a folder for the searchable files").to_string()); @@ -328,21 +541,66 @@ impl PdfCraftApp { self.notify_tr("Text recognition is already running"); return; } + // The folder is remembered (the next batch reuses it; the dialog's Clear un-remembers). + #[cfg(not(target_arch = "wasm32"))] + { + self.ocr_draft.output_folder = Some(dir.to_string_lossy().into_owned()); + } + // Two inputs of one file name (from different folders) would write the same output + // (or download under the same name): refused before anything is read. + let twice = pdfcraft_engine::ocr::duplicate_output_names(files.iter().map(|(name, _)| name.as_str())); + if !twice.is_empty() { + self.notify_error(crate::i18n::fmt( + tl!("Several files have the same name, so their results would overwrite each other; nothing was written: {list}"), + &[("list", &twice.join(", "))], + )); + return; + } + // Collision pre-flight: a searchable copy never silently overwrites. Every target is + // checked before the first page is read; a collision stops the batch here. + #[cfg(not(target_arch = "wasm32"))] + { + let hits: Vec = files + .iter() + .filter_map(|(name, _)| { + let target = dir.join(name); + target.exists().then(|| target.to_string_lossy().into_owned()) + }) + .take(5) + .collect(); + if !hits.is_empty() { + self.notify_error(crate::i18n::fmt( + tl!("The output folder already holds files of these names; nothing was written: {list}"), + &[("list", &hits.join(", "))], + )); + return; + } + } + // The engines are built (and named if they cannot be) before anything starts. + let recognizers = match pdfcraft_engine::ocr::recognizers(&settings) { + Ok(r) => r, + Err(why) => { + self.notify_error(why); + return; + } + }; let progress = Arc::new(Mutex::new(BatchProgress { total: files.len(), ..Default::default() })); let p = progress.clone(); // The summary is written on the worker thread: draw it in the UI's language. let lang = crate::i18n::current(); let work = move || { crate::i18n::set_current(lang); - // A panic in the recognizer still ends the batch with a summary (#3919). #[cfg(not(target_arch = "wasm32"))] - let summary = pdfcraft_engine::guard(|| recognize_files(files, &settings, &p, &dir)); + let write = move |name: &str, r: &pdfcraft_engine::ocr::FileResult| { + crate::editing::write_atomically(&dir.join(name).to_string_lossy(), &r.bytes).map_err(|e| e.to_string()) + }; #[cfg(target_arch = "wasm32")] - let summary = pdfcraft_engine::guard(|| recognize_files(files, &settings, &p)); - let summary = summary.unwrap_or_else(|panic| failed(&panic)); + let write = move |name: &str, r: &pdfcraft_engine::ocr::FileResult| crate::editing::download(name, &r.bytes); + // A panic in the recognizer still ends the batch with a summary (#3919). + let msg = pdfcraft_engine::guard(|| run_batch_files(files, settings, &recognizers, &p, write)).unwrap_or_else(|panic| failed(&panic)); if let Ok(mut s) = p.lock() { s.done = s.total; - s.message = Some(summary); + s.message = Some(msg); } }; #[cfg(not(target_arch = "wasm32"))] @@ -356,54 +614,17 @@ impl PdfCraftApp { self.ocr_batch = Some(progress); self.poll_ocr(); } -} -/// Recognize each file and write its searchable copy into `dir`; returns the summary to show. -fn recognize_files( - files: Vec<(String, Vec)>, - settings: &OcrSettings, - p: &Mutex, - #[cfg(not(target_arch = "wasm32"))] dir: &std::path::Path, -) -> String { - let (mut ok, mut words, mut failed) = (0, 0, Vec::new()); - match pdfcraft_engine::ocr::engine() { - Err(e) => failed.push(e), - Ok(ocr) => { - for (i, (name, bytes)) in files.into_iter().enumerate() { - if let Ok(mut s) = p.lock() { - s.done = i; - } - let r = pdfcraft_engine::ocr::recognize_file(&name, Arc::new(bytes), None, settings.clone(), &ocr, |_, _| true); - let saved = r.and_then(|r| { - #[cfg(not(target_arch = "wasm32"))] - crate::editing::write_atomically(&dir.join(&name).to_string_lossy(), &r.bytes).map_err(|e| e.to_string())?; - #[cfg(target_arch = "wasm32")] - crate::editing::download(&name, &r.bytes)?; - Ok(r.words()) - }); - match saved { - Ok(n) => { - ok += 1; - words += n; - } - Err(e) => failed.push(format!("{name}: {}", e)), - } - } + /// Ask the running batch to stop: the file in flight finishes, the rest are counted as not + /// run. + pub fn cancel_ocr_batch(&mut self) { + if let Some(b) = &self.ocr_batch + && let Ok(mut s) = b.lock() + { + s.cancel = true; } } - let done = if ok == 1 { - crate::i18n::fmt(tl!("Recognized {w} words in 1 file"), &[("w", &words.to_string())]) - } else { - crate::i18n::fmt(tl!("Recognized {w} words in {n} files"), &[("w", &words.to_string()), ("n", &ok.to_string())]) - }; - let mut msg = done; - if !failed.is_empty() { - msg.push_str(&crate::i18n::fmt(tl!("; failed: {list}"), &[("list", &failed.join("; "))])); - } - msg -} -impl PdfCraftApp { /// Stop a running recognition (the pages read so far are kept). pub fn cancel_ocr(&mut self) { if let Some(r) = &self.ocr_run @@ -442,7 +663,7 @@ impl PdfCraftApp { } } let Some(run) = self.ocr_run.as_ref() else { return }; - let (doc, progress, auto) = (run.doc, run.progress.clone(), run.automatic); + let (doc, progress, auto, guard, editable) = (run.doc, run.progress.clone(), run.automatic, run.guard.clone(), run.editable); let automatic = auto.is_some(); let mut s = match progress.lock() { Ok(s) => s, @@ -498,9 +719,20 @@ impl PdfCraftApp { return; } }; + // The dialog's "Editable text" choice is honored: the recognition is not written as an + // invisible layer here. It opens in the OCR Verify screen, whose Accept produces the + // editable document (a NEW file) the choice promises. + if editable && !automatic { + self.ocr_verify.load(doc, guard, found); + self.ocr_verify.open = true; + self.notify_tr("Review the recognition, then Accept to write the editable document"); + return; + } let read = found.iter().filter(|p| p.skipped.is_none()).count(); let skipped = found.len() - read; - match self.session.apply_ocr(doc, &found) { + // The guard refuses a result that no longer fits its document (the user edited, or + // redacted, while the worker was out); its reason says so. + match self.session.apply_ocr_checked(doc, &found, &guard) { Ok(words) => { if let Some(info) = self.session.get(doc).map(|d| d.info.clone()) && let Some(view) = self.views.iter_mut().find(|v| v.id == doc) @@ -525,6 +757,12 @@ impl PdfCraftApp { crate::i18n::fmt(tl!("; {n} pages already had text"), &[("n", &skipped.to_string())]) }); } + // The pages' own notes (the editable output's unrenderable-character warning, + // for one) are part of the result: said here, not dropped. + for note in found.iter().flat_map(|p| p.notes.iter()) { + msg.push_str("; "); + msg.push_str(note); + } self.notify(msg); } Err(e) => self.notify_error(e), @@ -532,6 +770,77 @@ impl PdfCraftApp { } } +/// The batch's worker: read each file, hand the searchable copy to `write` (the output folder, +/// or the web's download), count the buckets, and compose the summary. A file the skip-files +/// option passes over is its own bucket and `write` is never called for it. A cancel is asked +/// for between files: the file in flight finishes, the rest are counted as not run. +pub(crate) fn run_batch_files( + files: Vec<(String, Vec)>, + settings: OcrSettings, + recognizers: &pdfcraft_engine::ocr::Recognizers, + progress: &Mutex, + mut write: impl FnMut(&str, &pdfcraft_engine::ocr::FileResult) -> Result<(), String>, +) -> String { + let total = files.len(); + let (mut ok, mut skipped, mut words, mut failed, mut notes) = (0usize, 0usize, 0usize, Vec::new(), Vec::new()); + let mut not_run: Option = None; + for (i, (name, bytes)) in files.into_iter().enumerate() { + if progress.lock().map(|s| s.cancel).unwrap_or(false) { + not_run = Some(total - i); + break; + } + if let Ok(mut s) = progress.lock() { + s.done = i; + } + let r = pdfcraft_engine::ocr::recognize_file(&name, Arc::new(bytes), None, settings.clone(), recognizers, |_, _| true); + match r { + Ok(res) if res.skipped() => skipped += 1, + Ok(res) => match write(&name, &res) { + Ok(()) => { + ok += 1; + words += res.words(); + // The batch's low-confidence note: at or under the settings' threshold (60 by + // default), on 1-based sorted pages. + if let Some((count, pages)) = pdfcraft_engine::ocr::low_confidence_pages(&res.pages, settings.low_confidence_threshold) { + let list = pages.iter().map(|p| p.to_string()).collect::>().join(", "); + notes.push(crate::i18n::fmt( + tl!("{name}: {n} low-confidence word(s) on pages {pages}"), + &[("name", &name), ("n", &count.to_string()), ("pages", &list)], + )); + } + } + Err(e) => failed.push(format!("{name}: {e}")), + }, + Err(e) => failed.push(format!("{name}: {e}")), + } + } + batch_summary(ok, words, skipped, &failed, ¬es, not_run) +} + +/// The batch summary: what was read, the skipped files as their own bucket, the failures, the +/// low-confidence notes and (on a cancel) how many files were not run. +fn batch_summary(ok: usize, words: usize, skipped: usize, failed: &[String], notes: &[String], not_run: Option) -> String { + let mut msg = if ok == 1 { + crate::i18n::fmt(tl!("Recognized {w} words in 1 file"), &[("w", &words.to_string())]) + } else { + crate::i18n::fmt(tl!("Recognized {w} words in {n} files"), &[("w", &words.to_string()), ("n", &ok.to_string())]) + }; + if skipped > 0 { + msg.push_str(&crate::i18n::fmt(tl!("; {n} files already had text"), &[("n", &skipped.to_string())])); + } + if !failed.is_empty() { + msg.push_str(&crate::i18n::fmt(tl!("; failed: {list}"), &[("list", &failed.join("; "))])); + } + if let Some(rest) = not_run { + msg.push_str(&crate::i18n::fmt(tl!("; cancelled — {n} files were not run"), &[("n", &rest.to_string())])); + } + for note in notes { + msg.push_str("; "); + msg.push_str(note); + } + msg +} + /// "Recognize text failed: {e}", in the current language. fn failed(e: &str) -> String { crate::i18n::fmt(tl!("{label} failed: {e}"), &[("label", tl!("Recognize text")), ("e", e)]) @@ -547,6 +856,7 @@ fn recognize_guarded(recognize: impl FnOnce() -> Result) -> Result mod tests { use super::*; use crate::{Dialog, LeftPanel, Mode, QuickTool}; + use pdfcraft_engine::ocr::{Recognizer, Recognizers}; use pdfcraft_engine::{Edit, Session}; /// A one-page PDF that is one picture covering the page. @@ -629,12 +939,26 @@ mod tests { /// asked for is applied at once. The result is made up, so no OCR models are needed. #[test] fn the_result_of_an_automatic_run_waits_for_read_mode_and_dialogs() { - let word = || pdfcraft_engine::ocr::PlacedWord { text: "scanned".into(), origin: [72.0, 700.0], across: [60.0, 0.0], up: [0.0, 12.0] }; + let word = || pdfcraft_engine::ocr::PlacedWord { + text: "scanned".into(), + origin: [72.0, 700.0], + across: [60.0, 0.0], + up: [0.0, 12.0], + confidence: None, + source: "ocrs".into(), + }; let finished = |app: &PdfCraftApp, automatic: bool| { let generation = app.session.get(app.active_ids().unwrap().1).unwrap().edit_generation(); - let page = pdfcraft_engine::ocr::OcrPage { page: 0, words: vec![word()], already_on_page: 0, skipped: None }; + let page = pdfcraft_engine::ocr::OcrPage { page: 0, words: vec![word()], already_on_page: 0, skipped: None, notes: Vec::new() }; let progress = Arc::new(Mutex::new(OcrProgress { done: 1, total: 1, result: Some(Ok(vec![page])), cancel: false })); - OcrRun { doc: app.active_ids().unwrap().1, progress, automatic: automatic.then_some(AutoOcr { page: 0, generation }) } + let guard = app.session.ocr_guard(app.active_ids().unwrap().1).unwrap(); + OcrRun { + doc: app.active_ids().unwrap().1, + progress, + guard, + automatic: automatic.then_some(AutoOcr { page: 0, generation }), + editable: false, + } }; let label = |app: &PdfCraftApp| app.session.get(app.active_ids().unwrap().1).unwrap().can_undo().map(String::from); @@ -669,12 +993,21 @@ mod tests { let generation = app.session.get(id).unwrap().edit_generation(); let page = pdfcraft_engine::ocr::OcrPage { page: 0, - words: vec![pdfcraft_engine::ocr::PlacedWord { text: "scanned".into(), origin: [72.0, 700.0], across: [60.0, 0.0], up: [0.0, 12.0] }], + words: vec![pdfcraft_engine::ocr::PlacedWord { + text: "scanned".into(), + origin: [72.0, 700.0], + across: [60.0, 0.0], + up: [0.0, 12.0], + confidence: None, + source: "ocrs".into(), + }], already_on_page: 0, skipped: None, + notes: Vec::new(), }; let progress = Arc::new(Mutex::new(OcrProgress { done: 1, total: 1, result: Some(Ok(vec![page])), cancel: false })); - app.ocr_run = Some(OcrRun { doc: id, progress, automatic: Some(AutoOcr { page: 0, generation }) }); + let guard = app.session.ocr_guard(id).unwrap(); + app.ocr_run = Some(OcrRun { doc: id, progress, guard, automatic: Some(AutoOcr { page: 0, generation }), editable: false }); app.views[0].auto_ocr_tried.insert(0); // The document changes while the run is out. let mut png = std::io::Cursor::new(Vec::new()); @@ -712,9 +1045,13 @@ mod tests { fn a_run_whose_worker_died_ends_instead_of_waiting_forever() { let mut app = PdfCraftApp::new(); app.set_option("language", "en").unwrap(); + let pdf = pdfcraft_engine::Session::new().create_from_text("t", "guard fixture").unwrap(); + app.open_bytes("a.pdf", None, pdf.to_vec()).unwrap(); + let id = app.active_ids().unwrap().1; + let guard = app.session.ocr_guard(id).unwrap(); let progress = Arc::new(Mutex::new(OcrProgress { total: 3, ..Default::default() })); poison(&progress); - app.ocr_run = Some(OcrRun { doc: DocId(1), progress, automatic: None }); + app.ocr_run = Some(OcrRun { doc: id, progress, guard, automatic: None, editable: false }); app.poll_ocr(); assert!(app.ocr_run.is_none(), "the next run isn't refused as already running"); let toast = app.toast.take().map(|t| t.0).unwrap_or_default(); @@ -728,4 +1065,267 @@ mod tests { let toast = app.toast.take().map(|t| t.0).unwrap_or_default(); assert!(toast.starts_with("Recognize text failed"), "{toast}"); } + + /// A fake engine: one confident word per call, no filesystem, no models needed. + struct Fake { + confidence: f32, + } + + impl Recognizer for Fake { + fn id(&self) -> &'static str { + "fake" + } + fn available(&self) -> bool { + true + } + fn languages(&self) -> Vec<(String, String)> { + vec![("en".into(), "English".into())] + } + fn recognize( + &self, + _image: &pdfcraft_ocr::OcrImage, + _options: &pdfcraft_ocr::RecognizeOptions, + ) -> Result, pdfcraft_engine::ocr::OcrError> { + Ok(vec![pdfcraft_ocr::Line { + words: vec![pdfcraft_ocr::Word::new("word", [1.0, 1.0, 9.0, 9.0], "fake").with_confidence(self.confidence)], + }]) + } + } + + /// A batch over two text files (the engine never runs on them: they are skipped files): + /// the summary counts them in their own bucket, nothing is written, and a cancel before + /// the first file leaves the rest not run. + #[test] + fn batch_counts_skipped_files_and_honours_a_cancel() { + let s = pdfcraft_engine::Session::new(); + let pdf = s.create_from_text("t", "Already typed here").expect("fixture"); + let files = vec![("a.pdf".into(), pdf.to_vec()), ("b.pdf".into(), pdf.to_vec())]; + let progress = Mutex::new(BatchProgress { total: files.len(), cancel: true, ..Default::default() }); + let recognizers = Recognizers { primary: Arc::new(Fake { confidence: 95.0 }), secondary: None }; + let writes = Arc::new(std::sync::atomic::AtomicUsize::new(0)); + let w = writes.clone(); + let msg = run_batch_files(files, OcrSettings { skip_text_files: true, ..Default::default() }, &recognizers, &progress, move |_, _| { + w.fetch_add(1, std::sync::atomic::Ordering::SeqCst); + Ok(()) + }); + assert_eq!(writes.load(std::sync::atomic::Ordering::SeqCst), 0, "nothing is written for skipped files, and the cancel came first"); + assert!(msg.contains("cancelled — 2 files were not run"), "{msg}"); + // Without the cancel, the skipped bucket is named and still nothing is written. + let files = vec![("a.pdf".into(), pdf.to_vec()), ("b.pdf".into(), pdf.to_vec())]; + let progress = Mutex::new(BatchProgress { total: files.len(), ..Default::default() }); + let w = writes.clone(); + let msg = run_batch_files(files, OcrSettings { skip_text_files: true, ..Default::default() }, &recognizers, &progress, move |_, _| { + w.fetch_add(1, std::sync::atomic::Ordering::SeqCst); + Ok(()) + }); + assert_eq!(writes.load(std::sync::atomic::Ordering::SeqCst), 0); + assert!(msg.contains("; 2 files already had text"), "{msg}"); + assert!(msg.contains("Recognized 0 words in 0 files"), "{msg}"); + } + + /// A batch over scanned files: the words are counted, and a low-confidence reading lands + /// in the summary as a note with its 1-based sorted pages. + #[test] + fn batch_notes_low_confidence_pages() { + let s = pdfcraft_engine::Session::new(); + let pdf = s.create_from_text("t", "fake scan").expect("fixture"); + let files = vec![("scan.pdf".into(), pdf.to_vec()), ("clean.pdf".into(), pdf.to_vec())]; + let progress = Mutex::new(BatchProgress { total: files.len(), ..Default::default() }); + let recognizers = Recognizers { primary: Arc::new(Fake { confidence: 40.0 }), secondary: None }; + // The fixtures are text PDFs standing in for scans; the default skip-text-pages option + // would (correctly) leave them alone, so the batch here reads everything. + let settings = OcrSettings { skip_text_pages: false, ..Default::default() }; + let msg = run_batch_files(files, settings.clone(), &recognizers, &progress, |_, _| Ok(())); + assert!(msg.contains("Recognized 2 words in 2 files"), "{msg}"); + assert!(msg.contains("scan.pdf: 1 low-confidence word(s) on pages 1"), "{msg}"); + // The write failing is a per-file failure, named in the summary. + let files = vec![("scan.pdf".into(), pdf.to_vec())]; + let progress = Mutex::new(BatchProgress { total: files.len(), ..Default::default() }); + let msg = run_batch_files(files, settings, &recognizers, &progress, |_, _| Err("disk full".into())); + assert!(msg.contains("; failed: scan.pdf: disk full"), "{msg}"); + } + + /// The batch note follows the settings' low-confidence threshold (60 by default), not a + /// number of its own: a 40% word is flagged under 60 and under 40, and not under 30. + #[test] + fn the_batch_note_uses_the_settings_threshold() { + let s = pdfcraft_engine::Session::new(); + let pdf = s.create_from_text("t", "fake scan").expect("fixture"); + let recognizers = Recognizers { primary: Arc::new(Fake { confidence: 40.0 }), secondary: None }; + let run = |threshold: f32| { + let files = vec![("scan.pdf".into(), pdf.to_vec())]; + let progress = Mutex::new(BatchProgress { total: files.len(), ..Default::default() }); + let settings = OcrSettings { skip_text_pages: false, low_confidence_threshold: threshold, ..Default::default() }; + run_batch_files(files, settings, &recognizers, &progress, |_, _| Ok(())) + }; + assert!(run(OcrSettings::default().low_confidence_threshold).contains("1 low-confidence word(s)"), "the default flags a 40% word"); + assert!(run(40.0).contains("1 low-confidence word(s)"), "at the threshold counts"); + assert!(!run(30.0).contains("low-confidence"), "a lower threshold leaves the word alone: {}", run(30.0)); + } + + /// The remembered output folder survives the preferences round trip; an absent or empty + /// value keeps the default (ask each time). + #[test] + fn the_output_folder_is_persisted_with_the_draft() { + let mut d = OcrDraft::default(); + assert_eq!(d.output_folder, None, "by default the folder is asked for"); + d.output_folder = Some("out".into()); + let mut r = OcrDraft::default(); + r.apply_pref(&d.to_pref()); + assert_eq!(r.output_folder.as_deref(), Some("out")); + let mut e = OcrDraft::default(); + e.apply_pref(&serde_json::json!({})); + assert_eq!(e.output_folder, None, "an absent value keeps the default"); + e.output_folder = Some("kept".into()); + e.apply_pref(&serde_json::json!({ "output_folder": "" })); + assert_eq!(e.output_folder.as_deref(), Some("kept"), "an empty value is not a folder"); + } + + /// "Automatic" saved without a strategy reads the way Automatic reads by default (the + /// second engine joins when the first is unsure); an explicit choice always stands. + #[test] + fn an_automatic_engine_without_a_saved_strategy_is_confidence_weighted() { + let mut d = OcrDraft::default(); + d.apply_pref(&serde_json::json!({ "engine": "auto" })); + assert_eq!((d.engine, d.strategy), (EngineChoice::Auto, MergeStrategy::ConfidenceWeighted)); + let mut named = OcrDraft::default(); + named.apply_pref(&serde_json::json!({ "engine": "tesseract" })); + assert_eq!(named.strategy, MergeStrategy::PrimaryOnly, "a named engine reads alone"); + let mut explicit = OcrDraft::default(); + explicit.apply_pref(&serde_json::json!({ "engine": "auto", "strategy": "primary" })); + assert_eq!((explicit.engine, explicit.strategy), (EngineChoice::Auto, MergeStrategy::PrimaryOnly)); + } + + /// Two inputs of one file name (from different folders, or differing only in case) would + /// write the same output: the batch stops before anything is read or written. + #[test] + fn inputs_that_share_a_file_name_are_refused_before_anything_is_read() { + let mut app = PdfCraftApp::new(); + app.run_inline = true; + let dir = std::env::temp_dir().join(format!("pdfcraft-ocr-same-name-{}", std::process::id())); + std::fs::create_dir_all(&dir).expect("fixture"); + app.ocr_draft.output_folder = Some(dir.to_string_lossy().into_owned()); + let s = pdfcraft_engine::Session::new(); + let pdf = s.create_from_text("t", "scanned words here").expect("fixture"); + app.ocr_files(vec![("scan.pdf".into(), pdf.to_vec()), ("other.pdf".into(), pdf.to_vec()), ("Scan.PDF".into(), pdf.to_vec())]); + assert!(app.ocr_batch.is_none(), "the batch never starts"); + let toast = app.toast.clone().map(|t| t.0).unwrap_or_default(); + assert!(toast.contains("same name") && toast.contains("Scan.PDF"), "{toast}"); + assert_eq!(std::fs::read_dir(&dir).expect("fixture").count(), 0, "nothing was written"); + let _ = std::fs::remove_dir_all(&dir); + } + + /// A remembered output folder is used without asking, is remembered again after the run, + /// and its collision pre-flight still stops the batch before the first page is read. + #[test] + fn a_remembered_output_folder_is_used_and_still_preflights_collisions() { + let mut app = PdfCraftApp::new(); + app.run_inline = true; + let dir = std::env::temp_dir().join(format!("pdfcraft-ocr-folder-{}", std::process::id())); + std::fs::create_dir_all(&dir).expect("fixture"); + std::fs::write(dir.join("a.pdf"), b"would be overwritten").expect("fixture"); + app.ocr_draft.output_folder = Some(dir.to_string_lossy().into_owned()); + let s = pdfcraft_engine::Session::new(); + let pdf = s.create_from_text("t", "scanned words here").expect("fixture"); + app.ocr_files(vec![("a.pdf".into(), pdf.to_vec()), ("b.pdf".into(), pdf.to_vec())]); + assert!(app.ocr_batch.is_none(), "the collision stops the batch before anything is read"); + let toast = app.toast.clone().map(|t| t.0).unwrap_or_default(); + assert!(toast.contains("already holds files of these names"), "{toast}"); + assert!(toast.contains("a.pdf"), "the colliding names are in the error: {toast}"); + assert_eq!(app.ocr_draft.output_folder.as_deref(), Some(dir.to_string_lossy().as_ref()), "the folder stays remembered for the next batch"); + let _ = std::fs::remove_dir_all(&dir); + } + + /// A result that arrives after the document changed (here: a page turned while the worker was + /// reading; the same applies to a redaction applied meanwhile) is refused with the reason and + /// writes nothing, so words read before a redaction cannot come back as invisible text. + #[test] + fn a_result_that_arrives_after_the_document_changed_is_refused() { + let mut app = PdfCraftApp::new(); + let pdf = pdfcraft_engine::Session::new().create_from_text("t", "Typed text").expect("fixture"); + app.open_bytes("a.pdf", None, pdf.to_vec()).expect("opens"); + let id = app.active_ids().expect("a document").1; + let guard = app.session.ocr_guard(id).expect("a guard"); + app.session.apply(id, pdfcraft_engine::Edit::RotatePages { pages: vec![0], degrees: 90 }).expect("an edit while the worker runs"); + let after_edit = app.session.get(id).and_then(|d| d.can_undo().map(str::to_owned)); + let word = pdfcraft_engine::ocr::PlacedWord { + text: "123-45-6789".into(), + origin: [40.0, 40.0], + across: [50.0, 0.0], + up: [0.0, 12.0], + confidence: Some(95.0), + source: "fake".into(), + }; + let page = OcrPage { page: 0, words: vec![word], already_on_page: 0, skipped: None, notes: Vec::new() }; + let progress = Arc::new(Mutex::new(OcrProgress { total: 1, done: 1, result: Some(Ok(vec![page])), cancel: false })); + app.ocr_run = Some(OcrRun { doc: id, progress, guard, automatic: None, editable: false }); + app.poll_ocr(); + assert!(app.ocr_run.is_none(), "the run is over"); + let toast = app.toast.clone().map(|t| t.0).unwrap_or_default(); + assert!(toast.contains("changed since recognition started"), "{toast}"); + assert_eq!( + app.session.get(id).and_then(|d| d.can_undo().map(str::to_owned)), + after_edit, + "nothing was written: the last undoable step is still the edit" + ); + } + + /// A page's notes survive to the result the user sees: the editable output's + /// unrenderable-character warning ("N characters had no font … appear as ?") is said in + /// the run's summary, not dropped. The result is fabricated, so no models are needed. + #[test] + fn the_result_summary_names_the_substituted_characters() { + let mut app = app_with(scan()); + let id = app.active_ids().unwrap().1; + let word = pdfcraft_engine::ocr::PlacedWord { + text: "scanned".into(), + origin: [72.0, 700.0], + across: [60.0, 0.0], + up: [0.0, 12.0], + confidence: Some(95.0), + source: "ocrs".into(), + }; + let page = pdfcraft_engine::ocr::OcrPage { + page: 0, + words: vec![word], + already_on_page: 0, + skipped: None, + notes: vec!["3 characters had no font that could show them and appear as ? in the output".into()], + }; + let guard = app.session.ocr_guard(id).unwrap(); + let progress = Arc::new(Mutex::new(OcrProgress { total: 1, done: 1, result: Some(Ok(vec![page])), cancel: false })); + app.ocr_run = Some(OcrRun { doc: id, progress, guard, automatic: None, editable: false }); + app.poll_ocr(); + let toast = app.toast.take().map(|t| t.0).unwrap_or_default(); + assert!(toast.contains("3 characters had no font"), "{toast}"); + } + + /// The dialog's "Editable text" output is honored: a run that promised an editable document + /// does not quietly write the invisible searchable layer — its result opens in the OCR + /// Verify screen, whose Accept produces the editable document. The result is fabricated, + /// so no models are needed. + #[test] + fn an_editable_output_run_opens_the_review_instead_of_applying() { + let mut app = app_with(scan()); + app.run_inline = true; + let id = app.active_ids().unwrap().1; + let word = pdfcraft_engine::ocr::PlacedWord { + text: "scanned".into(), + origin: [72.0, 700.0], + across: [60.0, 0.0], + up: [0.0, 12.0], + confidence: Some(95.0), + source: "ocrs".into(), + }; + let page = OcrPage { page: 0, words: vec![word], already_on_page: 0, skipped: None, notes: Vec::new() }; + let guard = app.session.ocr_guard(id).unwrap(); + let progress = Arc::new(Mutex::new(OcrProgress { total: 1, done: 1, result: Some(Ok(vec![page])), cancel: false })); + app.ocr_draft.output_mode = pdfcraft_engine::ocr::OutputMode::EditableText; + app.ocr_run = Some(OcrRun { doc: id, progress, guard, automatic: None, editable: true }); + app.poll_ocr(); + assert!(app.ocr_run.is_none(), "the run is over"); + assert_eq!(app.session.get(id).unwrap().can_undo(), None, "no invisible layer was written"); + assert!(app.ocr_verify.open, "the review opens, to accept as an editable document"); + assert_eq!(app.ocr_verify.phase, crate::workflows::ocr_verify::Phase::ReviewReady); + } } diff --git a/crates/ui-egui/src/workflows/ocr_verify.rs b/crates/ui-egui/src/workflows/ocr_verify.rs new file mode 100644 index 0000000000..4054339de4 --- /dev/null +++ b/crates/ui-egui/src/workflows/ocr_verify.rs @@ -0,0 +1,2830 @@ +//! Scan & OCR ▸ Correct recognized text: the OCR Verify screen. A full-workspace surface with +//! four panes (page list, source page, recognized text, zoom + word inspector) where a +//! recognition is reviewed before it is applied: words are colored by confidence, corrected, +//! removed (and restored), sent to a per-language user dictionary, and navigated by their +//! uncertainty. Accept applies the review — as one undoable step on the open document +//! (searchable), or as a new `_ocr.pdf` of visible text (editable). Reject throws the +//! reading away. +//! +//! The state machine is [`Phase`]: recognition runs on a worker thread ([`Phase::Running`], a +//! real Cancel keeps the pages already read), results land in [`Phase::ReviewReady`], failures +//! land in [`Phase::RecoverableError`] with the cause named and Run enabled again, and +//! Accept/Reject/Re-OCR are ignored outside ReviewReady instead of acting on nothing. + +use std::collections::BTreeSet; +use std::path::PathBuf; +use std::sync::{Arc, Mutex}; + +use pdfcraft_engine::ocr::{ + ConfidenceBand, EngineChoice, MergeStrategy, OcrGuard, OcrPage, OcrSettings, OutputMode, PlacedWord, band_for, clamp_region, +}; +use pdfcraft_engine::{DocId, Session}; + +use super::ocr_ui; +use crate::theme::{self, Tokens}; +use crate::{PdfCraftApp, widgets}; + +/// Where the screen stands. Run is enabled in Idle, ReviewReady and RecoverableError; the +/// review tools (Accept, Reject, navigation, the inspector, re-OCR) only in ReviewReady. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub enum Phase { + #[default] + Idle, + Running, + ReviewReady, + Saving, + RecoverableError, +} + +/// The four panes; Ctrl+Tab moves the focus ring. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub enum Pane { + #[default] + Pages, + Source, + Text, + Inspector, +} + +impl Pane { + fn next(self) -> Pane { + match self { + Pane::Pages => Pane::Source, + Pane::Source => Pane::Text, + Pane::Text => Pane::Inspector, + Pane::Inspector => Pane::Pages, + } + } +} + +/// One review change, so it can be undone within the panel (review edits are not document +/// edits; they live until the review is applied or rejected). +#[derive(Clone, Debug)] +pub enum Op { + /// The correction text field committed (an empty commit removes instead). + Corrected { + key: (usize, usize), + before: String, + after: String, + }, + Removed { + key: (usize, usize), + }, + Restored { + key: (usize, usize), + }, + WordVerified { + key: (usize, usize), + on: bool, + }, + Dictionary { + word: String, + }, + /// A re-read (a region or the whole page) replaced words on the page. Review marks are + /// keyed by word position, so the page's marks as they were are kept for the undo. + Region { + page: usize, + before: Vec, + keys: PageKeys, + }, + PageVerified { + page: usize, + on: bool, + }, + /// "Reject entire page": the page's words were cleared (and its review marks with them). + PageCleared { + page: usize, + before: Vec, + keys: PageKeys, + }, +} + +/// One page's positional review marks (word indices), saved when its word list is replaced. +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct PageKeys { + pub removed: Vec, + pub verified: Vec, +} + +/// The review state: which words are struck out, which are vouched for, which are in the +/// per-language dictionary, and the undo history. Never persisted. +#[derive(Clone, Debug, Default)] +pub struct Review { + pub removed: BTreeSet<(usize, usize)>, + pub verified: BTreeSet<(usize, usize)>, + pub verified_pages: BTreeSet, + pub dictionary: BTreeSet, + /// Pages whose words were already applied to the document ("Accept entire page"); a later + /// Accept of the whole review never writes them twice. + pub applied_pages: BTreeSet, + pub history: Vec, +} + +/// A word that asks for review: its confidence is in the Low band and the reviewer has not +/// dealt with it — it is not removed, not corrected into the dictionary, not marked verified, +/// and not on a page marked verified. A corrected word stays uncertain by its ORIGINAL +/// confidence: the reviewer changed the text, not the engine's confidence in it. +pub fn is_uncertain(page: &OcrPage, key: (usize, usize), review: &Review) -> bool { + let (_, wi) = key; + if review.removed.contains(&key) || review.verified.contains(&key) || review.verified_pages.contains(&key.0) { + return false; + } + let Some(w) = page.words.get(wi) else { return false }; + if w.text.trim().is_empty() || review.dictionary.contains(&w.text.to_lowercase()) { + return false; + } + pdfcraft_engine::ocr::is_suspect(w.confidence) +} + +/// Candidate corrections for `text`: dictionary words and the page's own words within edit +/// distance 2, nearest first, at most 5, never the word itself. +pub fn suggestions(text: &str, dictionary: &BTreeSet, page_words: &[PlacedWord]) -> Vec { + let lower = text.to_lowercase(); + let mut out: Vec<(usize, String)> = Vec::new(); + let consider = |candidate: &str, out: &mut Vec<(usize, String)>| { + let c = candidate.to_lowercase(); + if c.is_empty() || c == lower { + return; + } + if let Some(d) = edit_distance_within_2(&lower, &c, 2) { + out.push((d, candidate.to_owned())); + } + }; + for w in dictionary { + consider(w, &mut out); + } + for w in page_words { + consider(&w.text, &mut out); + } + // Nearest first; ties break alphabetically; at most five. + out.sort_by(|a, b| a.0.cmp(&b.0).then_with(|| a.1.cmp(&b.1))); + out.truncate(5); + out.into_iter().map(|(_, s)| s).collect() +} + +/// Levenshtein distance between `a` and `b`, abandoned as soon as it is sure to exceed `max` +/// (small bounded DP over char boundaries; `None` means "farther than `max`"). +pub fn edit_distance_within_2(a: &str, b: &str, max: usize) -> Option { + let a: Vec = a.chars().collect(); + let b: Vec = b.chars().collect(); + if a.len().abs_diff(b.len()) > max { + return None; + } + let mut prev: Vec = (0..=b.len()).collect(); + let mut cur = vec![0usize; b.len() + 1]; + for i in 1..=a.len() { + cur[0] = i; + let mut row_min = cur[0]; + for j in 1..=b.len() { + let cost = usize::from(a[i - 1] != b[j - 1]); + cur[j] = (prev[j] + 1).min(cur[j - 1] + 1).min(prev[j - 1] + cost); + row_min = row_min.min(cur[j]); + } + if row_min > max { + return None; + } + std::mem::swap(&mut prev, &mut cur); + } + (prev[b.len()] <= max).then_some(prev[b.len()]) +} + +/// The page preview render: cached until the document or page changes. +pub struct Preview { + pub doc: DocId, + pub page: usize, + /// Pixels per point the page was rendered at. + pub scale: f32, + pub width: u32, + pub height: u32, + pub(crate) texture: Option, +} + +/// A page or region re-read in flight ([`OcrVerify::start_reread`]): the page being read +/// again (and the drawn region, in view-space points, for a region re-read), with the +/// worker's progress. +pub struct Reread { + pub page: usize, + pub region: Option<[f32; 4]>, + pub(crate) progress: Arc>, +} + +/// A re-read's progress: the replacement words once the worker is done, and a cancel request. +/// A cancelled re-read is discarded when it lands: the page's words stay as they were. +#[derive(Default)] +pub struct RereadProgress { + pub cancel: bool, + pub(crate) result: Option, String>>, +} + +/// The OCR Verify screen. One instance per app; lives across frames. +pub struct OcrVerify { + pub open: bool, + pub phase: Phase, + /// The document being reviewed; `None` until a run starts. + pub doc: Option, + /// The recognition results, in page order, as read (review edits mutate them). + pub pages: Vec, + /// What applying the results still expects of the document. + pub guard: Option, + pub review: Review, + /// The selected word (page, index) and the page the source pane shows. + pub selected: Option<(usize, usize)>, + pub current: usize, + /// A region drawn on the source pane, in view-space points on the shown page, snapped + /// outward to whole pixels when it is used ([`Self::start_reread`]). + pub region: Option<[f32; 4]>, + /// Where a source-pane drag started (the live region rectangle is drawn from here). + pub(crate) drag: Option, + pub focus: Pane, + /// The status line: where the run stands, why something was refused. + pub message: String, + cancelled: bool, + pub progress: Option>>, + /// A page or region re-read running on a worker thread: the screen stays live (the review + /// edits and navigation work) and the reading is applied by [`Self::poll_reread`]. + pub reread: Option, + /// The correction text field's current contents. + pub correction: String, + /// Where the per-language dictionary files live (`None`: off, e.g. the web build). + pub dictionary_dir: Option, + /// Language of the captured recognition, independent of the interface locale. + pub dictionary_language: String, + pub(crate) preview: Option, +} + +fn dictionary_path(dir: &std::path::Path, language: &str) -> Result { + if language.is_empty() || language.len() > 32 || !language.bytes().all(|b| b.is_ascii_alphabetic() || b == b'-') { + return Err("invalid OCR dictionary language".into()); + } + Ok(dir.join(format!("ocr-dictionary-{language}.txt"))) +} + +fn read_dictionary(path: &std::path::Path) -> Result { + use std::io::Read as _; + let mut text = String::new(); + match std::fs::File::open(path) { + Ok(file) => { + file.take((pdfcraft_engine::ocr::dictionary::MAX_BYTES + 1) as u64).read_to_string(&mut text).map_err(|e| e.to_string())?; + } + Err(e) if e.kind() == std::io::ErrorKind::NotFound => {} + Err(e) => return Err(e.to_string()), + } + Ok(text) +} + +impl Default for OcrVerify { + fn default() -> Self { + OcrVerify { + open: false, + phase: Phase::Idle, + doc: None, + pages: Vec::new(), + guard: None, + review: Review::default(), + selected: None, + current: 0, + region: None, + drag: None, + focus: Pane::Pages, + message: String::new(), + cancelled: false, + progress: None, + reread: None, + correction: String::new(), + dictionary_dir: None, + dictionary_language: "en".into(), + preview: None, + } + } +} + +impl OcrVerify { + /// Begin a review of `found` for `doc`: ReviewReady even when nothing was found (the + /// status then says so), with the guard remembered for Accept. + pub fn load(&mut self, doc: DocId, guard: OcrGuard, found: Vec) { + self.doc = Some(doc); + self.guard = Some(guard); + self.pages = found; + self.review = Review::default(); + self.selected = None; + self.region = None; + self.current = self.current.min(self.pages.len().saturating_sub(1)); + self.phase = Phase::ReviewReady; + self.load_dictionary_pref(); + } + + /// The count of live (not removed, not empty) words across the review. + pub fn word_count(&self) -> usize { + self.pages + .iter() + .enumerate() + .map(|(p, pg)| pg.words.iter().enumerate().filter(|(i, w)| !w.text.trim().is_empty() && !self.review.removed.contains(&(p, *i))).count()) + .sum() + } + + /// The words that ask for review, in reading order across pages. + pub fn uncertain(&self) -> Vec<(usize, usize)> { + self.pages + .iter() + .flat_map(|pg| pg.suspects().into_iter().map(move |i| (pg.page, i))) + .filter(|k| self.page(k.0).is_some_and(|p| is_uncertain(p, *k, &self.review))) + .collect() + } + + /// The count of Low-band live words (the info strip's LOW-CONFIDENCE WORDS). + pub fn suspect_count(&self) -> usize { + self.pages + .iter() + .enumerate() + .map(|(p, pg)| { + pg.words + .iter() + .enumerate() + .filter(|(i, w)| { + !self.review.removed.contains(&(p, *i)) + && !w.text.trim().is_empty() + && !self.review.dictionary.contains(&w.text.to_lowercase()) + && pdfcraft_engine::ocr::is_suspect(w.confidence) + }) + .count() + }) + .sum() + } + + /// The share of words the reviewer vouched for: marked verified, or on a page marked + /// verified (0.0 with no words). + pub fn verified_fraction(&self) -> f32 { + let (mut yes, mut all) = (0usize, 0usize); + for (p, pg) in self.pages.iter().enumerate() { + let page_done = self.review.verified_pages.contains(&pg.page); + for (i, w) in pg.words.iter().enumerate() { + if w.text.trim().is_empty() || self.review.removed.contains(&(p, i)) { + continue; + } + all += 1; + if page_done || self.review.verified.contains(&(p, i)) { + yes += 1; + } + } + } + if all == 0 { 0.0 } else { yes as f32 / all as f32 } + } + + pub fn page(&self, page: usize) -> Option<&OcrPage> { + self.pages.iter().find(|p| p.page == page) + } + + pub fn word(&self, key: (usize, usize)) -> Option<&PlacedWord> { + self.page(key.0).and_then(|p| p.words.get(key.1)) + } + + pub fn is_removed(&self, key: (usize, usize)) -> bool { + self.review.removed.contains(&key) + } + + pub fn is_verified(&self, key: (usize, usize)) -> bool { + self.review.verified.contains(&key) || self.review.verified_pages.contains(&key.0) + } + + /// The word's text with any correction applied. + pub fn text_of(&self, key: (usize, usize)) -> Option<&str> { + self.word(key).map(|w| w.text.as_str()) + } + + fn in_review(&self) -> bool { + self.phase == Phase::ReviewReady + } + + /// Commit the correction field for `key`. An empty (after trim) commit removes the word; + /// the box never moves. Only in ReviewReady. + pub fn correct(&mut self, key: (usize, usize), text: &str) -> bool { + if !self.in_review() || self.word(key).is_none() { + return false; + } + if text.trim().is_empty() { + return self.set_removed(key, true); + } + let Some(w) = self.page_mut(key.0).and_then(|p| p.words.get_mut(key.1)) else { return false }; + let before = w.text.clone(); + if before == text { + return false; + } + w.text = text.to_string(); + self.review.history.push(Op::Corrected { key, before, after: text.to_string() }); + true + } + + /// Strike a word out (or restore it). Removed words stay in place — reviewable, restorable, + /// and never written by Accept. + pub fn set_removed(&mut self, key: (usize, usize), removed: bool) -> bool { + if !self.in_review() || self.word(key).is_none() || self.is_removed(key) == removed { + return false; + } + if removed { + self.review.removed.insert(key); + self.review.history.push(Op::Removed { key }); + } else { + self.review.removed.remove(&key); + self.review.history.push(Op::Restored { key }); + } + true + } + + /// Vouch for one word. + pub fn set_word_verified(&mut self, key: (usize, usize), on: bool) -> bool { + if !self.in_review() || self.word(key).is_none() || self.review.verified.contains(&key) == on { + return false; + } + if on { + self.review.verified.insert(key); + } else { + self.review.verified.remove(&key); + } + self.review.history.push(Op::WordVerified { key, on }); + true + } + + /// Mark the page reviewed (the toolbar's Verified toggle, Ctrl+T). + pub fn set_page_verified(&mut self, page: usize, on: bool) -> bool { + if !self.in_review() || self.page(page).is_none() || self.review.verified_pages.contains(&page) == on { + return false; + } + if on { + self.review.verified_pages.insert(page); + } else { + self.review.verified_pages.remove(&page); + } + self.review.history.push(Op::PageVerified { page, on }); + true + } + + /// Add a word to the per-language dictionary (case-insensitive, duplicate-free) and + /// remember it in the dictionary file when one can be written. Privacy: that file + /// (`ocr-dictionary-.txt` in the preferences folder) is plain text outside any + /// document, so a word added from a document stays on disk after the document is closed, + /// until it is removed (the undo removes it again). + pub fn add_to_dictionary(&mut self, word: &str, language: &str) -> bool { + if !self.in_review() { + return false; + } + if language != self.dictionary_language { + self.message = "The dictionary language differs from the captured recognition".into(); + return false; + } + let lower = word.trim().to_lowercase(); + if self.review.dictionary.contains(&lower) { + return false; + } + match self.updated_dictionary(&[word], &[]) { + Ok((entries, changed)) => { + self.review.dictionary = entries; + if !changed { + return false; + } + self.review.history.push(Op::Dictionary { word: lower }); + true + } + Err(why) => { + self.message = format!("OCR dictionary: {why}"); + false + } + } + } + + /// Validate the complete file and persist atomically before changing review state. + fn updated_dictionary(&self, add: &[&str], remove: &[&str]) -> Result<(BTreeSet, bool), String> { + use pdfcraft_engine::ocr::dictionary; + let path = self.dictionary_dir.as_ref().map(|dir| dictionary_path(dir, &self.dictionary_language)).transpose()?; + let current = + if let Some(path) = &path { read_dictionary(path)? } else { self.review.dictionary.iter().cloned().collect::>().join("\n") }; + let before = dictionary::words(¤t)?; + let text = dictionary::update(¤t, add, remove)?; + let entries = dictionary::words(&text)?; + let changed = before != entries; + if let Some(path) = path.filter(|_| changed) { + if let Some(parent) = path.parent() { + std::fs::create_dir_all(parent).map_err(|e| e.to_string())?; + } + crate::editing::write_atomically(&path.to_string_lossy(), text.as_bytes()).map_err(|e| e.to_string())?; + } + Ok((entries, changed)) + } + + /// Read the captured recognition language's dictionary; errors preserve its live state. + pub fn load_dictionary_pref(&mut self) { + let Some(dir) = &self.dictionary_dir else { return }; + let path = match dictionary_path(dir, &self.dictionary_language) { + Ok(path) => path, + Err(why) => { + self.message = why; + return; + } + }; + match read_dictionary(&path).and_then(|text| pdfcraft_engine::ocr::dictionary::words(&text)) { + Ok(words) => self.review.dictionary = words, + Err(why) => self.message = format!("OCR dictionary: {why}"), + } + } + + /// Undo the last review change (Ctrl+Z / the inspector's Undo). + pub fn undo(&mut self) -> bool { + if !self.in_review() { + return false; + } + let Some(op) = self.review.history.pop() else { return false }; + match op { + Op::Corrected { key, before, .. } => { + if let Some(w) = self.page_mut(key.0).and_then(|p| p.words.get_mut(key.1)) { + w.text = before; + } + } + Op::Removed { key } => { + self.review.removed.remove(&key); + } + Op::Restored { key } => { + self.review.removed.insert(key); + } + Op::WordVerified { key, on } => { + if on { + self.review.verified.remove(&key); + } else { + self.review.verified.insert(key); + } + } + Op::Dictionary { word } => match self.updated_dictionary(&[], &[&word]) { + Ok((entries, _)) => self.review.dictionary = entries, + Err(why) => { + self.message = format!("OCR dictionary: {why}"); + self.review.history.push(Op::Dictionary { word }); + return false; + } + }, + Op::Region { page, before, keys } => { + if let Some(p) = self.page_mut(page) { + p.words = before; + } + self.restore_page_keys(page, keys); + } + Op::PageVerified { page, on } => { + if on { + self.review.verified_pages.remove(&page); + } else { + self.review.verified_pages.insert(page); + } + } + Op::PageCleared { page, before, keys } => { + if let Some(p) = self.page_mut(page) { + p.words = before; + } + self.restore_page_keys(page, keys); + } + } + true + } + + fn reread_settings(&self, draft: &super::ocr_ui::OcrDraft) -> OcrSettings { + let mut settings = draft.settings(); + settings.language = self.dictionary_language.clone(); + settings + } + + /// Whether this word's text was corrected in the review (the inspector says so, and the + /// source pane outlines it). + pub fn is_corrected(&self, key: (usize, usize)) -> bool { + // Newest first: a correction recorded before the page's words were replaced (a re-read, + // a cleared page) names a position in the OLD word list, not this word. + for op in self.review.history.iter().rev() { + match op { + Op::Corrected { key: k, .. } if *k == key => return true, + Op::Region { page, .. } | Op::PageCleared { page, .. } if *page == key.0 => return false, + _ => {} + } + } + false + } + + /// Take `page`'s positional review marks out of the review (its word list is about to be + /// replaced), returning them for the undo. + fn take_page_keys(&mut self, page: usize) -> PageKeys { + let take = |set: &mut BTreeSet<(usize, usize)>| { + let on_page: Vec = set.iter().filter(|k| k.0 == page).map(|k| k.1).collect(); + set.retain(|k| k.0 != page); + on_page + }; + PageKeys { removed: take(&mut self.review.removed), verified: take(&mut self.review.verified) } + } + + /// Put back `page`'s marks exactly as `keys` holds them (the undo of a replacement). + fn restore_page_keys(&mut self, page: usize, keys: PageKeys) { + self.review.removed.retain(|k| k.0 != page); + self.review.verified.retain(|k| k.0 != page); + self.review.removed.extend(keys.removed.into_iter().map(|i| (page, i))); + self.review.verified.extend(keys.verified.into_iter().map(|i| (page, i))); + } + + /// Accept ONE page ("Accept entire page"): the page's live words become one undoable step + /// on the open document, the page is marked applied, and a later whole-review Accept (or a + /// second click on the same page) never writes it again. Ignored outside ReviewReady; zero + /// live words on the page is a typed failure; a stale session is refused with its reason + /// (edits kept). + pub fn accept_page(&mut self, session: &mut Session, page: usize) -> Result { + if !self.in_review() { + return Ok(0); + } + let Some(guard) = self.guard.clone() else { return Ok(0) }; + let Some(doc) = self.doc else { return Ok(0) }; + let Some(pg) = self.live_pages().into_iter().find(|p| p.page == page) else { return Ok(0) }; + if pg.words.is_empty() { + return Err("no words are left to apply on this page".into()); + } + // The page's words are already in the document: never append the OCR layer twice. + // (A page accepted and then rejected still counts: its words were written, and the + // empty-words check above has already spoken for it.) + if self.review.applied_pages.contains(&page) { + return Ok(0); + } + match session.apply_ocr_checked(doc, &[pg], &guard) { + Ok(n) => { + self.review.applied_pages.insert(page); + // This apply is now the document's newest edit: the next page (or the final + // Accept) checks against the document as it stands after it, so only an edit + // made elsewhere in between makes it stale. + self.guard = session.ocr_guard(doc); + Ok(n) + } + Err(why) => { + self.phase = Phase::RecoverableError; + self.message = why.clone(); + Err(why) + } + } + } + + /// Reject ONE page ("Reject entire page"): clear its words (undoable within the panel), + /// drop the selection if it was there. + pub fn reject_page(&mut self, page: usize) -> bool { + if !self.in_review() { + return false; + } + let Some(pg) = self.page_mut(page) else { return false }; + if pg.words.is_empty() { + return false; + } + let before = std::mem::take(&mut pg.words); + let keys = self.take_page_keys(page); + self.review.history.push(Op::PageCleared { page, before, keys }); + if self.selected.is_some_and(|k| k.0 == page) { + self.selected = None; + } + true + } + + /// Replace a page's words wholesale ("Re-OCR entire page"), undoable within the panel. The + /// page's review marks belonged to the old words (they are kept by position), so they are + /// cleared with them and come back with the undo. + pub fn replace_page_words(&mut self, page: usize, words: Vec) -> bool { + if !self.in_review() || self.page(page).is_none() { + return false; + } + let Some(pg) = self.page_mut(page) else { return false }; + let before = std::mem::replace(&mut pg.words, words); + let keys = self.take_page_keys(page); + self.review.history.push(Op::Region { page, before, keys }); + if self.selected.is_some_and(|k| k.0 == page) { + self.selected = None; + } + true + } + + fn page_mut(&mut self, page: usize) -> Option<&mut OcrPage> { + self.pages.iter_mut().find(|p| p.page == page) + } + + /// Walk the uncertain words. Wraps around, skips removed words; with none to show it is a + /// no-op. A corrected word that is still Low-band stays in the walk. + pub fn goto_uncertain(&mut self, forward: bool) -> bool { + let list = self.uncertain(); + let Some(target) = (match self.selected { + None if forward => list.first().copied(), + None => list.last().copied(), + Some(sel) => { + let pos = list.iter().position(|k| *k == sel); + match pos { + Some(i) => { + let next = if forward { i + 1 } else { i + list.len() - 1 }; + list.get(next % list.len()).copied() + } + None if forward => list.first().copied(), + None => list.last().copied(), + } + } + }) else { + return false; + }; + self.select(target); + true + } + + /// Select a word and show its page. + pub fn select(&mut self, key: (usize, usize)) { + self.selected = Some(key); + self.current = key.0; + self.correction = self.text_of(key).unwrap_or_default().to_string(); + } + + /// The results Accept would write: the read pages with removed words stripped. + pub fn live_pages(&self) -> Vec { + self.pages + .iter() + .map(|p| OcrPage { + already_on_page: 0, + page: p.page, + skipped: p.skipped.clone(), + notes: p.notes.clone(), + words: p + .words + .iter() + .enumerate() + .filter(|(i, w)| !self.review.removed.contains(&(p.page, *i)) && !w.text.trim().is_empty()) + .map(|(_, w)| w.clone()) + .collect(), + }) + .collect() + } + + /// Accept in searchable mode: apply the reviewed words as ONE undoable step on the open + /// document, but only while the session is still the one that was read. Ignored outside + /// ReviewReady; zero words and a stale session are typed failures (edits kept). + /// Returns the number of words written. + pub fn accept(&mut self, session: &mut Session) -> Result { + if !self.in_review() { + return Ok(0); // ignored, not an error + } + let Some(guard) = self.guard.clone() else { return Ok(0) }; + let Some(doc) = self.doc else { return Ok(0) }; + // Pages accepted page-by-page are already in the document; never write them twice. + let applied = &self.review.applied_pages; + let pages: Vec = self.live_pages().into_iter().filter(|p| !applied.contains(&p.page)).collect(); + let words = pages.iter().map(|p| p.words.len()).sum::(); + if words == 0 { + if !applied.is_empty() { + self.finish(); + return Ok(0); + } + return Err("no words are left to apply: every word was removed or empty".into()); + } + match session.apply_ocr_checked(doc, &pages, &guard) { + Ok(n) => { + self.finish(); + Ok(n) + } + Err(why) => { + self.phase = Phase::RecoverableError; + self.message = why.clone(); + Err(why) + } + } + } + + /// Accept in editable-text mode: the NEW document's bytes and its default file name + /// (`_ocr.pdf`). The open document is not touched; refuses zero words. + pub fn editable_output(&self, session: &Session) -> Result<(Arc>, String), String> { + if !self.in_review() { + return Err("the review is not ready".into()); + } + let Some(doc) = self.doc else { return Err("the review is not ready".into()) }; + // The review can stay open while the document is edited (Exit keeps it): words read before a + // redaction must not be written out as visible text afterwards. + if let Some(why) = self.guard.as_ref().and_then(|g| session.stale_reason(g)) { + return Err(why); + } + let pages = self.live_pages(); + if pages.iter().map(|p| p.words.len()).sum::() == 0 { + return Err("no words are left to write: every word was removed or empty".into()); + } + let bytes = session.editable_ocr_bytes(doc, &pages)?; + let d = session.get(doc).ok_or("the document is no longer open")?; + let base = d.name.strip_suffix(".pdf").unwrap_or(&d.name).to_string(); + Ok((bytes, format!("{base}_ocr.pdf"))) + } + + /// Reject: clear the words, the session, the selection and the flags; back to Idle. + /// Ignored outside ReviewReady. + pub fn reject(&mut self) -> bool { + if !self.in_review() { + return false; + } + *self = OcrVerify { open: self.open, dictionary_dir: self.dictionary_dir.take(), ..Default::default() }; + true + } + + /// The state after a successful accept (or a saved editable copy): the review is done. + pub(crate) fn finish(&mut self) { + *self = OcrVerify { open: self.open, phase: Phase::Idle, doc: self.doc, dictionary_dir: self.dictionary_dir.take(), ..Default::default() }; + } + + /// Ask the running recognition to stop. The pages already read are kept; the rest of the + /// document is not read. + pub fn cancel(&mut self) { + if self.phase == Phase::Running { + self.cancelled = true; + if let Some(p) = &self.progress + && let Ok(mut s) = p.lock() + { + s.cancel = true; + } + } + } + + pub fn can_run(&self) -> bool { + matches!(self.phase, Phase::Idle | Phase::ReviewReady | Phase::RecoverableError) + } + + /// Start re-reading a whole page (`region` `None`) or a drawn region on a WORKER THREAD: + /// the UI thread only builds the job and refuses an empty or out-of-page region BEFORE + /// anything runs — the render, the recognition and the engine probe (a `--list-langs` + /// process for Tesseract) happen off it, with the status line and a Cancel while they run. + /// The reading is applied by [`Self::poll_reread`] when it lands. + pub fn start_reread(&mut self, app: &mut PdfCraftApp, page: usize, region: Option<[f32; 4]>) { + if self.phase != Phase::ReviewReady || self.reread.is_some() { + return; // one re-read at a time, and never under a run + } + let Some(doc) = self.doc else { return }; + let settings = self.reread_settings(&app.ocr_draft); + if let Some(region) = region { + let Some(info) = app.session.get(doc).and_then(|d| d.info.pages.get(page)) else { + app.notify_error("the page is gone"); + return; + }; + // Refuse before running: an empty, inverted or out-of-page rectangle never + // reaches the engine (the words are untouched). + let scale = app.session.ocr_job(doc, &[page], settings.clone()).map_or(1.0, |job| job.render_scale(page)); + let (w, h) = (info.width * scale, info.height * scale); + let px = [region[0] * scale, region[1] * scale, region[2] * scale, region[3] * scale]; + if clamp_region(px, w.max(1.0) as u32, h.max(1.0) as u32).is_none() { + app.notify_error("the selected region is empty or outside the page"); + return; + } + } + let Some(mut job) = app.session.ocr_job(doc, &[page], settings) else { + app.notify_error("the document is no longer open"); + return; + }; + if let Some(region) = region { + let scale = job.render_scale(page); + job.settings.region = Some([region[0] * scale, region[1] * scale, region[2] * scale, region[3] * scale]); + } + let is_region = region.is_some(); + let progress = Arc::new(Mutex::new(RereadProgress::default())); + self.message = if is_region { tl!("Re-reading the region…").to_string() } else { tl!("Re-reading the page…").to_string() }; + let p = progress.clone(); + let work = move || { + let result = pdfcraft_engine::ocr::recognizers(&job.settings).and_then(|r| { + let found = job.run(&r, |_, _| !p.lock().map(|s| s.cancel).unwrap_or(false)); + let words: Vec = found.into_iter().flat_map(|pg| pg.words).collect(); + if words.is_empty() { + Err(if is_region { + "the region re-read found no text; the existing words are unchanged".into() + } else { + "the re-read found no text; the existing words are unchanged".into() + }) + } else { + Ok(words) + } + }); + if let Ok(mut s) = p.lock() { + s.result = Some(result); + } + }; + #[cfg(not(target_arch = "wasm32"))] + if app.run_inline { + work(); + } else { + std::thread::Builder::new().name("pdfcraft-ocr-reread".into()).spawn(work).ok(); + } + #[cfg(target_arch = "wasm32")] + work(); + self.reread = Some(Reread { page, region, progress }); + } + + /// Ask a running re-read to stop and be discarded: when the worker lands, its reading is + /// thrown away and the page's words stay as they were. + pub fn cancel_reread(&mut self) { + if let Some(r) = &self.reread + && let Ok(mut s) = r.progress.lock() + { + s.cancel = true; + } + } + + /// Apply a finished re-read, once the worker is done: a page re-read replaces the page's + /// words, a region re-read the words inside the drawn rectangle. A cancel discards the + /// reading, and so does a refusal — a document changed since the review began no longer + /// matches the page the words were read from. Called every frame while a re-read is out. + pub(crate) fn poll_reread(&mut self, app: &mut PdfCraftApp) { + if self.phase != Phase::ReviewReady { + return; + } + let Some(reread) = &self.reread else { return }; + let (cancelled, result) = match reread.progress.lock() { + Ok(mut s) => (s.cancel, s.result.take()), + Err(_) => return, + }; + let Some(result) = result else { + // Still reading: keep the status line live and ask for another frame. + if let Some(ctx) = &app.ctx { + ctx.request_repaint_after(std::time::Duration::from_millis(200)); + } + return; + }; + let (page, region) = (reread.page, reread.region); + self.reread = None; + if cancelled { + self.message = tl!("The re-read was cancelled; the words are unchanged").to_string(); + return; + } + // The document changed since the review began: the reading may no longer fit the page + // it was read from, so it is refused with the reason and the words stay. + if let Some(why) = self.guard.as_ref().and_then(|g| app.session.stale_reason(g)) { + self.message = why.clone(); + app.notify_error(why); + return; + } + match result { + Err(why) => { + self.message = why.clone(); + app.notify_error(why); + } + Ok(words) => { + let applied = if let Some(region) = region { + self.replace_region_words(&app.session, page, region, words).map(|_| true) + } else if self.replace_page_words(page, words) { + Ok(false) + } else { + Err("the page is gone".to_string()) + }; + match applied { + Ok(is_region) => { + if is_region { + self.message = tl!("The region was read again; its words were replaced (undo in the panel)").to_string(); + app.notify_tr("The region was read again; its words were replaced (undo in the panel)"); + } else { + self.message = tl!("The page was read again; its words were replaced (undo in the panel)").to_string(); + app.notify_tr("The page was read again; its words were replaced (undo in the panel)"); + }; + } + Err(why) => app.notify_error(why), + } + } + } + } + + /// Put `words` (the worker's reading of the drawn rectangle) in place of the words inside + /// it: the rectangle's bbox in user space decides which old words go; the kept words' + /// review marks follow them to their new positions, the replacement is undoable within + /// the panel, and the first NEW word is selected. An empty reading was refused before the + /// run, so `words` is never empty here. + fn replace_region_words(&mut self, session: &Session, page: usize, region_view: [f32; 4], words: Vec) -> Result<(), String> { + let Some(doc) = self.doc else { return Ok(()) }; + let Some(info) = session.get(doc).and_then(|d| d.info.pages.get(page)) else { return Err("the page is gone".into()) }; + // The rectangle in user space (a bbox of its mapped corners) decides which old words + // the new reading replaces. + let (a, b, c, d) = ( + info.view_to_user(region_view[0], region_view[1]), + info.view_to_user(region_view[2], region_view[3]), + info.view_to_user(region_view[0], region_view[3]), + info.view_to_user(region_view[2], region_view[1]), + ); + let xs = [a[0] as f64, b[0] as f64, c[0] as f64, d[0] as f64]; + let ys = [a[1] as f64, b[1] as f64, c[1] as f64, d[1] as f64]; + let bbox = [ + xs.iter().cloned().fold(f64::INFINITY, f64::min), + ys.iter().cloned().fold(f64::INFINITY, f64::min), + xs.iter().cloned().fold(f64::NEG_INFINITY, f64::max), + ys.iter().cloned().fold(f64::NEG_INFINITY, f64::max), + ]; + let Some(pg) = self.page_mut(page) else { return Err("the page is gone".into()) }; + let before = pg.words.clone(); + let center = |w: &PlacedWord| [w.origin[0] + w.across[0] / 2.0 + w.up[0] / 2.0, w.origin[1] + w.across[1] / 2.0 + w.up[1] / 2.0]; + // The old words outside the rectangle stay, in order; `moved` maps each kept word's old + // position to its new one, so its review marks follow it. + let mut moved = std::collections::BTreeMap::new(); + let mut after: Vec = Vec::with_capacity(before.len().saturating_add(words.len())); + for (i, w) in before.iter().enumerate() { + let c = center(w); + if !(c[0] >= bbox[0] && c[0] <= bbox[2] && c[1] >= bbox[1] && c[1] <= bbox[3]) { + moved.insert(i, after.len()); + after.push(w.clone()); + } + } + // The new words follow the kept ones, so the first of them is at the kept words' count; + // selecting it shows the reviewer what the re-read produced. + let first_new = after.len(); + after.extend(words.iter().cloned()); + pg.words = after; + let keys = self.take_page_keys(page); + let remap = |old: &[usize]| old.iter().filter_map(|i| moved.get(i).map(|&j| (page, j))).collect::>(); + self.review.removed.extend(remap(&keys.removed)); + self.review.verified.extend(remap(&keys.verified)); + self.review.history.push(Op::Region { page, before, keys }); + self.select((page, first_new)); + Ok(()) + } + + /// Store a finished run's pages (the worker's result) and move to ReviewReady. A cancel is + /// honest: the pages already read are kept, and the status says the rest was not read. + pub fn finished(&mut self, found: Vec, cancelled: bool) { + let read = found.iter().filter(|p| p.skipped.is_none()).count(); + let found_words: usize = found.iter().map(|p| p.words.len()).sum(); + self.pages = found; + self.review = Review::default(); + self.selected = None; + self.region = None; + self.current = self.current.min(self.pages.len().saturating_sub(1)); + let total = self.pages.len(); + self.message = if cancelled { + crate::i18n::fmt( + tl!("Cancelled — {n} words were read on {m} of {t} pages; the rest were not read"), + &[("n", &found_words.to_string()), ("m", &read.to_string()), ("t", &total.to_string())], + ) + } else if found_words == 0 { + tl!("OCR complete — no text recognized on this page.").to_string() + } else { + crate::i18n::fmt( + tl!("Read {n} words on {m} of {t} pages"), + &[("n", &found_words.to_string()), ("m", &read.to_string()), ("t", &total.to_string())], + ) + }; + self.phase = Phase::ReviewReady; + self.load_dictionary_pref(); + } + + /// A run that failed: the cause is named and Run is enabled again. + pub fn failed(&mut self, why: &str) { + self.phase = Phase::RecoverableError; + self.message = why.to_string(); + self.progress = None; + } + + pub(crate) fn start_run(&mut self, app: &mut PdfCraftApp) { + let Some((_, id)) = app.active_ids() else { return }; + if !self.can_run() || self.phase == Phase::Running { + return; + } + if self.reread.is_some() { + return; // the page re-read in flight replaces these words; one worker at a time + } + if let Some(why) = app.session.get(id).and_then(|d| d.read_only_reason.clone()) { + self.message = why; + self.phase = Phase::RecoverableError; + return; + } + let d = &app.ocr_draft; + let pages: Vec = match d.pages { + super::ocr_ui::OcrPages::All => Vec::new(), + super::ocr_ui::OcrPages::Current => vec![app.active.map_or(0, |i| app.views.get(i).map_or(0, |v| v.current))], + super::ocr_ui::OcrPages::Range => (d.from.saturating_sub(1)..d.to).collect(), + }; + let settings = d.settings(); + let Some(job) = app.session.ocr_job(id, &pages, settings.clone()) else { return }; + self.dictionary_language = job.settings.language.clone(); + let progress = Arc::new(Mutex::new(ocr_ui::OcrProgress { total: job.pages.len(), ..Default::default() })); + self.doc = Some(id); + self.guard = Some(job.guard.clone()); + self.current = 0; + self.cancelled = false; + self.phase = Phase::Running; + self.message = String::new(); + let p = progress.clone(); + let work = move || { + let result = pdfcraft_engine::ocr::recognizers(&settings).map(|r| { + job.run(&r, |done, total| { + let Ok(mut s) = p.lock() else { return false }; + s.done = done; + s.total = total; + !s.cancel + }) + }); + if let Ok(mut s) = p.lock() { + s.result = Some(result); + } + }; + #[cfg(not(target_arch = "wasm32"))] + if app.run_inline { + work(); + } else { + std::thread::Builder::new().name("pdfcraft-ocr-verify".into()).spawn(work).ok(); + } + #[cfg(target_arch = "wasm32")] + work(); + self.progress = Some(progress); + self.poll(app); + } + + /// Take the worker's result, once it is done. + pub(crate) fn poll(&mut self, app: &mut PdfCraftApp) { + if self.phase != Phase::Running { + return; + } + let Some(progress) = self.progress.clone() else { return }; + let result = progress.lock().ok().and_then(|mut s| s.result.take()); + let Some(result) = result else { + // Still running: refresh the status line and ask for another frame. + if let Ok(s) = progress.lock() { + self.message = crate::i18n::fmt( + tl!("Recognizing text… page {d} of {t}"), + &[("d", &(s.done + 1).min(s.total.max(1)).to_string()), ("t", &s.total.max(1).to_string())], + ); + } + if let Some(ctx) = &app.ctx { + ctx.request_repaint_after(std::time::Duration::from_millis(200)); + } + return; + }; + self.progress = None; + match result { + Ok(found) => self.finished(found, self.cancelled), + Err(why) => self.failed(&why), + } + } +} + +/// What a frame's controls asked for, applied after the drawing (the panes read `app` while +/// the controls write it, so nothing mutates through an open borrow). +#[derive(Default)] +struct Intents { + run: bool, + cancel: bool, + cancel_reread: bool, + accept: bool, + reject: bool, + exit: bool, + undo: bool, + next_uncertain: bool, + prev_uncertain: bool, + toggle_page_verified: bool, + reocr_region: bool, + reocr_page: bool, + accept_page: bool, + reject_page: bool, + cycle_focus: bool, + select_page: Option, + select_word: Option<(usize, usize)>, + correct: Option, + set_removed: Option, + set_word_verified: Option, + dictionary_add: bool, +} + +/// Band colours: High green, Medium yellow, Low red (the fill at ~20% opacity, the outline +/// at ~60%; the legend carries the same thresholds as text). +fn band_color(band: ConfidenceBand) -> egui::Color32 { + match band { + ConfidenceBand::High => egui::Color32::from_rgb(34, 139, 74), + ConfidenceBand::Medium => egui::Color32::from_rgb(196, 148, 20), + ConfidenceBand::Low => egui::Color32::from_rgb(204, 61, 47), + } +} + +/// The whole-window OCR Verify screen (Scan & OCR ▸ Correct recognized text). +pub(crate) fn show(app: &mut PdfCraftApp, ui: &mut egui::Ui) { + use egui::{Key, Modifiers}; + let ctx = ui.ctx().clone(); + let t = Tokens::get(&ctx); + let wants_text = ctx.egui_wants_keyboard_input(); + let (next_u, prev_u, toggle_verified, cycle, undo_key) = ctx.input_mut(|i| { + ( + i.consume_key(Modifiers::NONE, Key::F8), + i.consume_key(Modifiers::SHIFT, Key::F8), + (!wants_text && i.consume_key(Modifiers::COMMAND, Key::T)), + (!wants_text && i.consume_key(Modifiers::COMMAND, Key::Tab)), + (!wants_text && i.consume_key(Modifiers::COMMAND, Key::Z)), + ) + }); + + let mut n = Intents { + next_uncertain: next_u, + prev_uncertain: prev_u, + toggle_page_verified: toggle_verified, + cycle_focus: cycle, + undo: undo_key, + ..Default::default() + }; + + egui::Panel::left("ocr-verify-pages").resizable(false).default_size(190.0).frame(egui::Frame::new().fill(t.panel).inner_margin(8.0)).show( + ui, + |ui| { + pane_title(ui, &t, tl!("Pages"), app.ocr_verify.focus == Pane::Pages); + pages_pane(app, ui, &mut n); + }, + ); + + egui::Panel::right("ocr-verify-right").resizable(false).default_size(340.0).frame(egui::Frame::new().fill(t.panel).inner_margin(8.0)).show( + ui, + |ui| { + pane_title(ui, &t, tl!("Recognized text"), app.ocr_verify.focus == Pane::Text); + text_pane(app, ui, &t, &mut n); + ui.add_space(10.0); + pane_title(ui, &t, tl!("Word"), app.ocr_verify.focus == Pane::Inspector); + inspector(app, ui, &t, &mut n); + }, + ); + + egui::CentralPanel::default().frame(egui::Frame::new().fill(t.pasteboard).inner_margin(egui::Margin::same(8))).show(ui, |ui| { + toolbar(app, ui, &mut n); + ui.add_space(4.0); + info_strip(app, ui, &t); + ui.add_space(6.0); + pane_title(ui, &t, tl!("Source page"), app.ocr_verify.focus == Pane::Source); + source_pane(app, ui, &t, &mut n); + ui.add_space(6.0); + ui.horizontal(|ui| { + ui.label(egui::RichText::new(&app.ocr_verify.message).small().color(if app.ocr_verify.phase == Phase::RecoverableError { + t.accent + } else { + t.text_muted + })); + }); + }); + + if n.cycle_focus { + app.ocr_verify.focus = app.ocr_verify.focus.next(); + } + apply(app, n); +} + +/// A pane's caption; the focused pane's name is underlined (Ctrl+Tab moves the focus). +fn pane_title(ui: &mut egui::Ui, t: &Tokens, name: &str, focused: bool) { + ui.add_space(2.0); + let font = theme::semibold(11.5); + let w = ui.fonts_mut(|f| f.layout_no_wrap(name.to_owned(), font.clone(), t.text).size().x); + let (rect, _) = ui.allocate_exact_size(egui::vec2(w, 14.0), egui::Sense::hover()); + ui.painter().text(rect.left_center(), egui::Align2::LEFT_CENTER, name.to_uppercase(), font, t.text_muted); + if focused { + let r = egui::Rect::from_min_max(rect.left_bottom() + egui::vec2(0.0, 1.0), rect.right_bottom() + egui::vec2(0.0, 2.0)); + ui.painter().rect_filled(r, egui::CornerRadius::same(1), t.accent); + } + ui.add_space(4.0); +} + +/// Toolbar: every control is live — the choices feed the next run (and are persisted), Run +/// starts a recognition, Accept/Reject/navigate/verify only act in ReviewReady. +fn toolbar(app: &mut PdfCraftApp, ui: &mut egui::Ui, n: &mut Intents) { + let v = &app.ocr_verify; + let d = &mut app.ocr_draft; + let review = v.phase == Phase::ReviewReady; + let run_enabled = v.can_run() && v.reread.is_none(); + let accept_enabled = review && v.word_count() + v.review.applied_pages.len() > 0; + + // Row 1: the settings (each written back into the persisted draft). + ui.horizontal_wrapped(|ui| { + craft_ui::ComboBox::from_id_salt("ocr-verify-language").width(150.0).selected_text(language_name(&d.language)).show_ui(ui, |ui| { + for (code, name) in pdfcraft_engine::ocr::LANGUAGES { + ui.selectable_value(&mut d.language, (*code).to_string(), *name); + } + }); + engine_combo(ui, &mut d.engine, &mut d.strategy); + strategy_combo(ui, &mut d.strategy); + output_combo(ui, &mut d.output_mode); + for (flag, label, why) in [ + (&mut d.auto_rotate, tl!("Auto-rotate"), tl!("Turns pages scanned sideways or upside down upright before reading")), + (&mut d.deskew, tl!("Deskew"), tl!("Straightens skewed text lines before reading")), + (&mut d.denoise, tl!("Denoise"), tl!("A 3×3 median filter against scan speckle")), + (&mut d.binarize, tl!("Binarize"), tl!("Sauvola binarization: ink to black, paper to white")), + ] { + ui.add(crate::widgets::checkbox(flag, label)).on_hover_text(format!("{label} \u{2014} {why}")); + } + }); + + // Row 2: scope and actions. + ui.add_space(4.0); + ui.horizontal_wrapped(|ui| { + crate::widgets::radio_value(ui, &mut d.pages, super::ocr_ui::OcrPages::All, tl!("All pages")); + crate::widgets::radio_value(ui, &mut d.pages, super::ocr_ui::OcrPages::Current, tl!("Current page")); + crate::widgets::radio_value(ui, &mut d.pages, super::ocr_ui::OcrPages::Range, tl!("From")); + let on = d.pages == super::ocr_ui::OcrPages::Range; + ui.add_enabled(on, egui::DragValue::new(&mut d.from).range(1..=9999)); + ui.label(tl!("to")); + ui.add_enabled(on, egui::DragValue::new(&mut d.to).range(1..=9999)); + + ui.separator(); + if ui.add_enabled_ui(run_enabled, |ui| widgets::pill_button(ui, tl!("Run OCR"), true)).inner.clicked() { + n.run = true; + } + if v.phase == Phase::Running && widgets::pill_button(ui, tl!("Cancel"), false).clicked() { + n.cancel = true; + } + if v.reread.is_some() && widgets::pill_button(ui, tl!("Cancel re-read"), false).clicked() { + n.cancel_reread = true; + } + if ui.add_enabled_ui(accept_enabled, |ui| widgets::pill_button(ui, tl!("Accept"), false)).inner.clicked() { + n.accept = true; + } + if ui.add_enabled_ui(review, |ui| widgets::pill_button(ui, tl!("Reject"), false)).inner.clicked() { + n.reject = true; + } + ui.separator(); + let page_done = review && v.review.verified_pages.contains(&v.current); + let mut verified = page_done; + if ui + .add_enabled(review, crate::widgets::checkbox(&mut verified, tl!("Page verified"))) + .on_hover_text(tl!("Mark this page as reviewed (Ctrl+T)")) + .changed() + { + n.toggle_page_verified = true; + } + if ui + .add_enabled_ui(review, |ui| widgets::ghost_button(ui, "chevron-right", tl!("Next uncertain"))) + .inner + .on_hover_text(tl!("F8: next word to review; Shift+F8 goes back")) + .clicked() + { + n.next_uncertain = true; + } + if ui + .add_enabled_ui(review, |ui| widgets::ghost_button(ui, "chevron-left", tl!("Previous uncertain"))) + .inner + .on_hover_text(tl!("Shift+F8: the previous word to review")) + .clicked() + { + n.prev_uncertain = true; + } + if widgets::ghost_button(ui, "x", tl!("Exit")).clicked() { + n.exit = true; + } + }); +} + +/// A language code's dropdown text: its name and its code ("English (en)"). +fn language_name(code: &str) -> String { + pdfcraft_engine::ocr::LANGUAGES.iter().find(|l| l.0 == code).map_or_else(|| code.to_owned(), |l| format!("{} ({})", l.1, l.0)) +} + +/// The Engine dropdown's text. +fn engine_label(engine: EngineChoice) -> &'static str { + match engine { + EngineChoice::Auto => tl!("Automatic"), + EngineChoice::Ocrs => "ocrs", + EngineChoice::Tesseract => "Tesseract", + } +} + +/// The Strategy dropdown's text. +fn strategy_label(strategy: MergeStrategy) -> &'static str { + match strategy { + MergeStrategy::PrimaryOnly => tl!("Primary only"), + MergeStrategy::ConfidenceWeighted => tl!("Confidence weighted"), + MergeStrategy::RoverVote => tl!("ROVER vote"), + } +} + +/// The Engine dropdown: Automatic, ocrs, Tesseract. An engine that cannot run is disabled and +/// says why (a tooltip names the missing models or program). The choice feeds the next run. +/// Choosing Automatic while the strategy is "Primary only" (which would never run the second +/// engine) moves the strategy to Automatic's own default, Confidence weighted. +fn engine_combo(ui: &mut egui::Ui, engine: &mut EngineChoice, strategy: &mut MergeStrategy) { + ocr_ui::keep_asking_while_probing(ui.ctx()); + craft_ui::ComboBox::from_id_salt("ocr-verify-engine").width(120.0).selected_text(engine_label(*engine)).show_ui(ui, |ui| { + for (choice, label) in [(EngineChoice::Auto, tl!("Automatic")), (EngineChoice::Ocrs, "ocrs"), (EngineChoice::Tesseract, "Tesseract")] { + let why = pdfcraft_engine::ocr::engine_unavailable_reason(choice); + let resp = ui.add_enabled(why.is_none(), egui::Button::selectable(*engine == choice, label)); + let clicked = resp.clicked(); + match why { + Some(reason) => { + resp.on_disabled_hover_text(crate::i18n::t(&reason)); + } + None => { + resp.on_hover_text(match choice { + EngineChoice::Auto => tl!("Both engines, combined as the Strategy says"), + EngineChoice::Ocrs => tl!("The built-in engine (pure Rust)"), + EngineChoice::Tesseract => tl!("The tesseract program, run as an external process"), + }); + } + } + if clicked { + *engine = choice; + if choice == EngineChoice::Auto && *strategy == MergeStrategy::PrimaryOnly { + *strategy = choice.default_strategy(); + } + } + } + }); +} + +/// The Strategy dropdown: how two engines' readings combine. With a single named engine the +/// ensemble strategies still run — they just degrade to the primary reading (the tooltip says +/// so), because only one engine was asked to read. +fn strategy_combo(ui: &mut egui::Ui, strategy: &mut MergeStrategy) { + craft_ui::ComboBox::from_id_salt("ocr-verify-strategy").width(150.0).selected_text(strategy_label(*strategy)).show_ui(ui, |ui| { + for (s, label, hint) in [ + (MergeStrategy::PrimaryOnly, tl!("Primary only"), tl!("Use the chosen engine's reading alone")), + (MergeStrategy::ConfidenceWeighted, tl!("Confidence weighted"), tl!("Add a second engine's reading when the first is unsure")), + (MergeStrategy::RoverVote, tl!("ROVER vote"), tl!("Always read with both engines and merge the words")), + ] { + let resp = ui.add(egui::Button::selectable(*strategy == s, label)).on_hover_text(hint); + if resp.clicked() { + *strategy = s; + } + } + }); +} + +/// The Output mode dropdown: what Accept produces, with tooltips that say what each mode +/// changes (searchable: the open page gains an invisible text layer; editable: a new file). +fn output_combo(ui: &mut egui::Ui, mode: &mut OutputMode) { + let name = match *mode { + OutputMode::Searchable => tl!("Searchable Image (Exact)"), + OutputMode::EditableText => tl!("Editable text"), + }; + craft_ui::ComboBox::from_id_salt("ocr-verify-output").width(180.0).selected_text(name).show_ui(ui, |ui| { + let s = ui + .selectable_label(*mode == OutputMode::Searchable, tl!("Searchable Image (Exact)")) + .on_hover_text(tl!("Adds invisible text over each word; the page image is not changed")); + if s.clicked() { + *mode = OutputMode::Searchable; + } + let e = ui + .selectable_label(*mode == OutputMode::EditableText, tl!("Editable text")) + .on_hover_text(tl!("Writes a new document of visible text; the scan is not changed")); + if e.clicked() { + *mode = OutputMode::EditableText; + } + }); +} + +/// The info strip: where the review stands, in facts — page, average confidence, how many +/// words ask for review, the share verified, and the engine · strategy that read it. +fn info_strip(app: &PdfCraftApp, ui: &mut egui::Ui, t: &Tokens) { + let v = &app.ocr_verify; + let read = v.pages.iter().filter(|p| p.skipped.is_none()).count(); + let avg = v.page(v.current).and_then(pdfcraft_engine::ocr::OcrPage::mean_confidence); + let cells = [ + (tl!("PAGE"), if v.pages.is_empty() { "—".into() } else { format!("{} / {}", v.current + 1, v.pages.len()) }), + (tl!("AVG CONFIDENCE"), avg.map_or_else(|| "—".into(), |c| format!("{c:.0}%"))), + (tl!("LOW-CONFIDENCE WORDS"), v.suspect_count().to_string()), + (tl!("VERIFIED %"), format!("{}%", (v.verified_fraction() * 100.0).round() as i64)), + ( + tl!("ENGINE · STRATEGY"), + if read == 0 { "—".into() } else { format!("{} · {}", engine_label(app.ocr_draft.engine), strategy_label(app.ocr_draft.strategy)) }, + ), + ]; + ui.horizontal(|ui| { + for (k, val) in cells { + ui.label(egui::RichText::new(k.to_string()).small().color(t.text_faint)); + ui.label(egui::RichText::new(val).small()); + ui.separator(); + } + }); +} + +/// The page list: one row per read page, with its live word count and a flag for the words +/// that still ask for review. Clicking shows the page. +fn pages_pane(app: &mut PdfCraftApp, ui: &mut egui::Ui, n: &mut Intents) { + let v = &app.ocr_verify; + egui::ScrollArea::vertical().show(ui, |ui| { + for pg in &v.pages { + let p = pg.page; + let (live, suspects) = ( + pg.words.iter().filter(|w| !w.text.trim().is_empty()).count(), + pg.words.iter().filter(|w| !w.text.trim().is_empty() && pdfcraft_engine::ocr::is_suspect(w.confidence)).count(), + ); + let marked = if pg.skipped.is_some() { + format!(" · {}", tl!("skipped")) + } else if v.review.verified_pages.contains(&p) { + format!(" · {}", tl!("verified")) + } else { + String::new() + }; + let text = format!("{} {}{} · {live}", tl!("Page"), p + 1, marked); + let label = ui.selectable_label(p == v.current, text); + let label = if suspects > 0 { + label.on_hover_text(crate::i18n::fmt(tl!("{n} word(s) to review"), &[("n", &suspects.to_string())])) + } else { + label + }; + if label.clicked() { + n.select_page = Some(p); + } + } + }); +} + +/// The SOURCE PAGE pane: the rendered page with one box per recognized word. A click selects +/// the word under the pointer; a drag draws the re-OCR region (in view-space points, clamped +/// to the page). Words are coloured by their confidence band; removed words are struck out and +/// dimmed, corrected words outlined. The context menu re-reads a region or the page, and +/// accepts or rejects the page. +fn source_pane(app: &mut PdfCraftApp, ui: &mut egui::Ui, t: &Tokens, n: &mut Intents) { + let page = app.ocr_verify.current; + let review = app.ocr_verify.phase == Phase::ReviewReady; + ensure_preview(app, page); + let Some(doc) = app.ocr_verify.doc else { return }; + let Some(info) = app.session.get(doc).and_then(|d| d.info.pages.get(page)).cloned() else { return }; + // Reserve the room the confidence legend (at the end of this pane) and the message row + // (after it) need, so the page cannot push them off the panel. + let size = (ui.available_size() - egui::vec2(0.0, 40.0)).max(egui::vec2(120.0, 160.0)); + let (rect, resp) = ui.allocate_exact_size(size, egui::Sense::click_and_drag()); + let painter = ui.painter_at(rect); + // The page fits the pane; the pasteboard shows around it. + let s = (rect.width() / info.width.max(1.0)).min(rect.height() / info.height.max(1.0)).max(0.01); + let size = egui::vec2(info.width * s, info.height * s); + let img_rect = egui::Rect::from_min_size(rect.min + (rect.size() - size) * 0.5, size); + painter.rect_filled(img_rect.expand(2.0), egui::CornerRadius::same(2), t.hover); + if let Some(preview) = &app.ocr_verify.preview + && let Some(tex) = &preview.texture + { + painter.image(tex.id(), img_rect, egui::Rect::from_min_max(egui::pos2(0.0, 0.0), egui::pos2(1.0, 1.0)), egui::Color32::WHITE); + } + // A word's box: its user-space corners, through the page's rotation, into pane pixels. + let to_pane = |p: [f64; 2]| { + let v = info.user_to_view(p[0] as f32, p[1] as f32); + egui::pos2(img_rect.left() + v[0] * s, img_rect.top() + v[1] * s) + }; + let corners = |w: &PlacedWord| { + [ + to_pane(w.origin), + to_pane([w.origin[0] + w.across[0], w.origin[1] + w.across[1]]), + to_pane([w.origin[0] + w.across[0] + w.up[0], w.origin[1] + w.across[1] + w.up[1]]), + to_pane([w.origin[0] + w.up[0], w.origin[1] + w.up[1]]), + ] + }; + if let Some(pg) = app.ocr_verify.page(page) { + for (wi, w) in pg.words.iter().enumerate() { + let key = (pg.page, wi); + let quad = corners(w); + if app.ocr_verify.is_removed(key) { + // Removed: dimmed, struck through — still there, still restorable. + painter.add(egui::Shape::convex_polygon(quad.to_vec(), egui::Color32::from_black_alpha(32), egui::Stroke::NONE)); + painter.line_segment([quad[0], quad[2]], egui::Stroke::new(1.0, t.text_faint)); + continue; + } + if let Some(band) = w.confidence.map(band_for) { + let (fill, _) = band_colors(band); + painter.add(egui::Shape::convex_polygon(quad.to_vec(), fill, egui::Stroke::NONE)); + } + let outline = if app.ocr_verify.selected == Some(key) { + egui::Stroke::new(2.0, egui::Color32::from_rgb(43, 122, 236)) + } else if app.ocr_verify.is_corrected(key) { + egui::Stroke::new(1.5, t.text) + } else { + // The band's outline; a word without a confidence reads as unmeasured. + w.confidence.map(|c| band_colors(band_for(c)).1).unwrap_or_else(|| egui::Stroke::new(0.5, t.text_faint)) + }; + painter.add(egui::Shape::convex_polygon(quad.to_vec(), egui::Color32::TRANSPARENT, outline)); + } + } + // The chosen region and the one being drawn right now. + let drag_now = resp.interact_pointer_pos().zip(app.ocr_verify.drag).map(|(p, start)| egui::Rect::from_two_pos(start, p).intersect(img_rect)); + let chosen = app.ocr_verify.region.and_then(|region| pane_region(region, &info, img_rect)); + for r in [chosen, drag_now].into_iter().flatten() { + painter.rect_stroke(r, 0.0, egui::Stroke::new(1.5, t.accent), egui::StrokeKind::Middle); + } + // A click selects the word under the pointer; dragging away from a click draws the region. + if resp.drag_started() { + app.ocr_verify.drag = resp.interact_pointer_pos(); + } + if resp.drag_stopped() { + let (start, end) = (app.ocr_verify.drag.take(), resp.interact_pointer_pos()); + if let (Some(a), Some(b)) = (start, end) + && egui::Rect::from_two_pos(a, b).intersect(img_rect).area() > 36.0 + { + // A region: normalize it into view-space points, clamped to the page. + let r = egui::Rect::from_two_pos(a, b).intersect(img_rect); + let region = [ + (r.left() - img_rect.left()) / s, + (r.top() - img_rect.top()) / s, + (r.right() - img_rect.left()) / s, + (r.bottom() - img_rect.top()) / s, + ]; + app.ocr_verify.region = (region[2] > region[0] && region[3] > region[1]).then_some(region); + } + } + if resp.clicked() + && let Some(at) = resp.interact_pointer_pos() + && let Some(pg) = app.ocr_verify.page(page) + { + n.select_word = pg + .words + .iter() + .enumerate() + .find(|(wi, w)| !app.ocr_verify.is_removed((pg.page, *wi)) && point_in_quad(at, &corners(w))) + .map(|(wi, _)| (pg.page, wi)); + } + resp.context_menu(|ui| { + let rereading = app.ocr_verify.reread.is_some(); + if ui.add_enabled(review && !rereading && app.ocr_verify.region.is_some(), egui::Button::new(tl!("Re-OCR this region"))).clicked() { + n.reocr_region = true; + ui.close(); + } + if ui.add_enabled(review && !rereading, egui::Button::new(tl!("Re-OCR entire page"))).clicked() { + n.reocr_page = true; + ui.close(); + } + ui.separator(); + if ui.add_enabled(review, egui::Button::new(tl!("Accept entire page"))).clicked() { + n.accept_page = true; + ui.close(); + } + if ui.add_enabled(review, egui::Button::new(tl!("Reject entire page"))).clicked() { + n.reject_page = true; + ui.close(); + } + ui.separator(); + ui.label(egui::RichText::new(tl!("Click a word to inspect it; drag a rectangle to re-read part of the page")).small().color(t.text_faint)); + }); + legend(ui, t); +} + +/// A region given in view-space points, back into pane pixels on `img_rect` (None when it +/// does not lie on the page). The region is already in view space (the drag stored it so), so +/// it is only scaled: mapping it through `user_to_view` again would mirror it vertically (and +/// misplace it on rotated or cropped pages). +fn pane_region(region: [f32; 4], info: &pdfcraft_render::PageInfo, img_rect: egui::Rect) -> Option { + let scale = img_rect.width() / info.width.max(1.0); + let to_pane = |v: [f32; 2]| egui::pos2(img_rect.left() + v[0] * scale, img_rect.top() + v[1] * scale); + let a = to_pane([region[0], region[1]]); + let b = to_pane([region[2], region[3]]); + let r = egui::Rect::from_two_pos(a, b).intersect(img_rect); + (r.area() > 1.0).then_some(r) +} + +/// Point in a convex quad (corners in order): the pointer is on the same side of all four +/// edges. +fn point_in_quad(p: egui::Pos2, quad: &[egui::Pos2; 4]) -> bool { + (0..4).all(|i| { + let (a, b) = (quad[i], quad[(i + 1) % 4]); + (b.x - a.x) * (p.y - a.y) - (b.y - a.y) * (p.x - a.x) <= 0.0 + }) +} + +/// Band colours: the fill at ~20% opacity, the outline at ~60% — the legend names the bands +/// as text, so meaning never rests on colour alone. +fn band_colors(band: ConfidenceBand) -> (egui::Color32, egui::Stroke) { + let c = band_color(band); + (c.gamma_multiply(0.2), egui::Stroke::new(1.0, c.gamma_multiply(0.6))) +} + +/// The overlay legend: the classifier's bands, with their thresholds spelled out. +fn legend(ui: &mut egui::Ui, t: &Tokens) { + ui.add_space(2.0); + ui.horizontal_wrapped(|ui| { + ui.label(egui::RichText::new(tl!("CONFIDENCE")).small().strong()); + for (band, text) in legend_rows() { + let (fill, stroke) = band_colors(band); + let (rect, _) = ui.allocate_exact_size(egui::vec2(12.0, 12.0), egui::Sense::hover()); + ui.painter().rect_filled(rect, 2.0, fill); + ui.painter().rect_stroke(rect, 2.0, stroke, egui::StrokeKind::Inside); + ui.label(egui::RichText::new(text).small().color(t.text_muted)); + } + }); +} + +/// The legend's rows: the same [`ConfidenceBand`] values the overlay, the counts and the +/// navigation use, with the classifier's thresholds as text (a test holds the drawing to +/// these numbers). +pub fn legend_rows() -> [(ConfidenceBand, String); 3] { + [ + (ConfidenceBand::High, tl!("HIGH (>= 90%)").to_string()), + (ConfidenceBand::Medium, tl!("MEDIUM (70-89%)").to_string()), + (ConfidenceBand::Low, tl!("LOW (< 70%)").to_string()), + ] +} + +/// Render (or re-render) the source pane's page preview when the shown page changed. One +/// synchronous render per page; the texture is cached until then. +fn ensure_preview(app: &mut PdfCraftApp, page: usize) { + let Some(doc) = app.ocr_verify.doc else { return }; + if app.ocr_verify.preview.as_ref().is_some_and(|p| p.doc == doc && p.page == page && p.texture.is_some()) { + return; + } + let Some(d) = app.session.get(doc) else { return }; + let Some(info) = d.info.pages.get(page) else { return }; + // Screen scale (the boxes are drawn on top of it); the recognition dpi is irrelevant here. + let scale = 2.0_f32.min((900.0 / info.width.max(1.0)).min(1200.0 / info.height.max(1.0))).max(0.2); + let config = pdfcraft_render::RenderConfig { password: d.password.as_deref().map(std::sync::Arc::from), ..Default::default() }; + let mut r = pdfcraft_render::PageRenderer::new(d.bytes.clone(), config); + let shot = r.render(pdfcraft_render::RenderRequest { page, kind: pdfcraft_render::RequestKind::Pixels, scale, ..Default::default() }); + if shot.error.is_some() || shot.width == 0 || shot.height == 0 { + return; // keep any earlier preview; a render failure names itself elsewhere + } + let scale = shot.width as f32 / info.width.max(1.0); + let texture = app.ctx.as_ref().map(|ctx| { + ctx.load_texture( + "ocr-verify-page", + egui::ColorImage::from_rgba_premultiplied([shot.width as usize, shot.height as usize], &shot.rgba), + egui::TextureOptions::LINEAR, + ) + }); + app.ocr_verify.preview = Some(Preview { doc, page, scale, width: shot.width, height: shot.height, texture }); +} + +/// The RECOGNIZED pane: the page's words as the engine read them — a preview, not +/// authoritative (the source pane's boxes and the inspector are where corrections happen). +/// Removed words show struck out; clicking a word selects it in the inspector. +fn text_pane(app: &mut PdfCraftApp, ui: &mut egui::Ui, t: &Tokens, n: &mut Intents) { + let v = &app.ocr_verify; + let Some(pg) = v.page(v.current) else { return }; + if let Some(why) = &pg.skipped { + ui.label(egui::RichText::new(why.as_str()).small().color(t.text_muted)); + return; + } + if pg.words.is_empty() { + ui.label(egui::RichText::new(tl!("OCR complete — no text recognized on this page.")).small().color(t.text_muted)); + return; + } + egui::ScrollArea::vertical().max_height(180.0).show(ui, |ui| { + ui.horizontal_wrapped(|ui| { + for (wi, w) in pg.words.iter().enumerate() { + let key = (pg.page, wi); + let mut label = egui::RichText::new(w.text.as_str()).small(); + if v.is_removed(key) { + label = label.strikethrough().color(t.text_faint); + } else if v.is_corrected(key) { + label = label.underline(); + } + if ui.selectable_label(v.selected == Some(key), label).clicked() { + n.select_word = Some(key); + } + } + }); + }); +} + +/// The ZOOM + word inspector: a magnified crop of the selected word, its facts (id, +/// confidence, engine, review state), the correction field (Enter commits; clearing removes +/// the word; the original box never moves), Remove/Restore, + Dictionary, the per-word +/// Verified toggle, and suggestions within edit distance 2. +fn inspector(app: &mut PdfCraftApp, ui: &mut egui::Ui, t: &Tokens, n: &mut Intents) { + let v = &app.ocr_verify; + let Some(key) = v.selected else { + ui.label(egui::RichText::new(tl!("Select a word on the page to inspect it")).small().color(t.text_muted)); + return; + }; + if v.word(key).is_none() { + ui.label(egui::RichText::new(tl!("The selected word is gone; choose another")).small().color(t.text_muted)); + return; + } + zoom_crop(app, ui, key, t); + ui.add_space(4.0); + let (id, conf, source) = { + let w = v.word(key); + ( + key.1 + 1, + w.and_then(|w| w.confidence).map_or_else(|| "—".to_owned(), |c| format!("{c:.0}%")), + w.map(|w| w.source.clone()).unwrap_or_default(), + ) + }; + let state = if v.is_removed(key) { + tl!("removed") + } else if v.is_corrected(key) { + tl!("corrected") + } else if v.is_verified(key) { + tl!("verified") + } else { + tl!("unreviewed") + }; + ui.label(egui::RichText::new(format!("{id} · {conf} · {source} · {state}")).small().color(t.text_muted)); + ui.add_space(4.0); + // The correction field: Enter commits, an empty commit removes, the box never moves. + let (mut correction, verified) = (app.ocr_verify.correction.clone(), app.ocr_verify.is_verified(key)); + let field = ui.add(egui::TextEdit::singleline(&mut correction).hint_text(tl!("Corrected text (Enter commits)"))); + let commit = field.lost_focus() && ui.input(|i| i.key_pressed(egui::Key::Enter)); + ui.horizontal(|ui| { + if ui.add(egui::Button::new(tl!("Remove")).small()).clicked() { + n.set_removed = Some(true); + } + if ui.add(egui::Button::new(tl!("Restore")).small()).clicked() { + n.set_removed = Some(false); + } + if ui.add(egui::Button::new(tl!("+ Dictionary")).small()).clicked() { + n.dictionary_add = true; + } + let mut on = verified; + if ui.add(crate::widgets::checkbox(&mut on, tl!("Verified"))).changed() { + n.set_word_verified = Some(on); + } + if ui.add(egui::Button::new(tl!("Undo")).small()).on_hover_text(tl!("Undo the last review change (Ctrl+Z)")).clicked() { + n.undo = true; + } + }); + app.ocr_verify.correction = correction; + if commit { + n.correct = Some(app.ocr_verify.correction.clone()); + } + // Suggestions: dictionary words and this page's own words within edit distance 2. + let (word_text, page_words) = { + let v = &app.ocr_verify; + (v.text_of(key).unwrap_or_default().to_owned(), v.page(key.0).map(|p| p.words.clone()).unwrap_or_default()) + }; + let sugg = suggestions(&word_text, &app.ocr_verify.review.dictionary, &page_words); + if !sugg.is_empty() { + ui.add_space(2.0); + ui.horizontal_wrapped(|ui| { + ui.label(egui::RichText::new(tl!("Suggestions")).small().color(t.text_muted)); + for s in sugg { + if ui.add(egui::Button::new(egui::RichText::new(s.clone()).small())).clicked() { + n.correct = Some(s); + } + } + }); + } +} + +/// The magnified crop of the selected word, drawn out of the page preview's texture. +fn zoom_crop(app: &PdfCraftApp, ui: &mut egui::Ui, key: (usize, usize), t: &Tokens) { + let Some(preview) = &app.ocr_verify.preview else { return }; + let Some(tex) = &preview.texture else { return }; + let Some(w) = app.ocr_verify.word(key) else { return }; + let Some(info) = app.ocr_verify.doc.and_then(|doc| app.session.get(doc)).and_then(|d| d.info.pages.get(preview.page)).cloned() else { + return; + }; + // The word's box in preview pixels (user → view → × preview scale), plus a margin. + let to_px = |p: [f64; 2]| { + let v = info.user_to_view(p[0] as f32, p[1] as f32); + [v[0] * preview.scale, v[1] * preview.scale] + }; + let a = to_px(w.origin); + let b = to_px([w.origin[0] + w.across[0] + w.up[0], w.origin[1] + w.across[1] + w.up[1]]); + let (x0, y0, x1, y1) = (a[0].min(b[0]), a[1].min(b[1]), a[0].max(b[0]), a[1].max(b[1])); + let (mw, mh) = ((x1 - x0).max(8.0) * 0.5, (y1 - y0).max(8.0) * 0.6); + let crop = egui::Rect::from_min_max( + egui::pos2((x0 - mw).max(0.0), (y0 - mh).max(0.0)), + egui::pos2((x1 + mw).min(preview.width as f32), (y1 + mh).min(preview.height as f32)), + ); + if crop.width() < 1.0 || crop.height() < 1.0 { + return; + } + // At most 300 × 90 points, at most 3× the preview's own resolution. + let k = (300.0 / crop.width()).min(90.0 / crop.height()).clamp(0.1, 3.0); + let (rect, _) = ui.allocate_exact_size(egui::vec2(crop.width() * k, crop.height() * k), egui::Sense::hover()); + let uv = egui::Rect::from_min_max( + egui::pos2(crop.left() / preview.width as f32, crop.top() / preview.height as f32), + egui::pos2(crop.right() / preview.width as f32, crop.bottom() / preview.height as f32), + ); + ui.painter().image(tex.id(), rect, uv, egui::Color32::WHITE); + ui.painter().rect_stroke(rect, 0.0, egui::Stroke::new(1.0, t.accent), egui::StrokeKind::Inside); +} + +/// Run `f` with the screen's state moved out of the app (a step that needs both the state and +/// the rest of the app would otherwise borrow it twice). The state is put back afterwards. +pub(crate) fn with_state(app: &mut PdfCraftApp, f: impl FnOnce(&mut OcrVerify, &mut PdfCraftApp)) { + let mut v = std::mem::take(&mut app.ocr_verify); + f(&mut v, app); + app.ocr_verify = v; +} + +/// Apply what the frame's controls asked for. The guards live in the [`OcrVerify`] methods: +/// outside ReviewReady, Accept/Reject/Re-OCR and edits are ignored, not acted on. +fn apply(app: &mut PdfCraftApp, n: Intents) { + with_state(app, |v, app| { + if n.exit { + v.open = false; + } + if let Some(p) = n.select_page { + v.current = p; + if v.selected.is_some_and(|k| k.0 != p) { + v.selected = None; + } + } + if let Some(k) = n.select_word { + v.select(k); + } + if n.toggle_page_verified { + let on = !v.review.verified_pages.contains(&v.current); + v.set_page_verified(v.current, on); + } + if n.next_uncertain { + v.goto_uncertain(true); + } + if n.prev_uncertain { + v.goto_uncertain(false); + } + if n.undo { + v.undo(); + } + if let Some(text) = n.correct + && let Some(k) = v.selected + { + v.correct(k, &text); + v.correction = v.text_of(k).unwrap_or_default().to_owned(); + } + if let Some(removed) = n.set_removed + && let Some(k) = v.selected + { + v.set_removed(k, removed); + } + if let Some(on) = n.set_word_verified + && let Some(k) = v.selected + { + v.set_word_verified(k, on); + } + if n.dictionary_add + && let Some(k) = v.selected + { + let word = v.text_of(k).unwrap_or_default().to_owned(); + let language = v.dictionary_language.clone(); + v.add_to_dictionary(&word, &language); + } + if n.reocr_page { + let page = v.current; + v.start_reread(app, page, None); + } + if n.reocr_region { + let (page, region) = (v.current, v.region); + if let Some(region) = region { + v.start_reread(app, page, Some(region)); + } + } + if n.cancel_reread { + v.cancel_reread(); + } + if n.accept_page { + let page = v.current; + match v.accept_page(&mut app.session, page) { + Ok(count) if count > 0 => { + if let Some(doc) = v.doc { + refresh_view(app, doc); + } + app.notify_fmt(tl!("The page's words were applied as one undoable step ({n} words)"), &[("n", &count.to_string())]); + } + Ok(_) => {} + Err(why) => app.notify_error(why), + } + } + if n.reject_page { + let page = v.current; + if v.reject_page(page) { + app.notify_tr("The page's words were cleared (undo in the panel)"); + } + } + if n.run { + v.start_run(app); + } + if n.cancel { + v.cancel(); + } + if n.accept { + do_accept(v, app); + } + if n.reject && v.reject() { + app.notify_tr("OCR results rejected."); + } + }); +} + +/// Accept: searchable mode applies the review to the open document as one undoable step; +/// editable mode asks where to write the new `_ocr.pdf`. Stale sessions and empty +/// reviews are typed failures the user sees (the edits are kept). +fn do_accept(v: &mut OcrVerify, app: &mut PdfCraftApp) { + let doc = v.doc; + match app.ocr_draft.output_mode { + OutputMode::Searchable => match v.accept(&mut app.session) { + Ok(count) => { + if let Some(doc) = doc { + refresh_view(app, doc); + } + if count > 0 { + app.notify_fmt(tl!("Recognized text applied ({n} words, one undoable step)"), &[("n", &count.to_string())]); + } + } + Err(why) => app.notify_error(why), + }, + OutputMode::EditableText => match v.editable_output(&app.session) { + Ok((bytes, name)) => save_editable(app, bytes, name, doc), + Err(why) => app.notify_error(why), + }, + } +} + +/// Write the editable-text output: into the tests' export folder when one is set, otherwise +/// ask where (the default name is `_ocr.pdf`). The source is never overwritten; a +/// cancelled dialog keeps the review. The write is the [`Phase::Saving`] state: it ends the +/// review when the file lands, and returns to ReviewReady when it does not, so a failed or +/// cancelled save can simply be tried again. +fn save_editable(app: &mut PdfCraftApp, bytes: Arc>, name: String, doc: Option) { + let source = doc.and_then(|id| app.session.get(id)).and_then(|d| d.path.clone()); + // The chosen file is the open source when both resolve to the same path (a new file does + // not exist yet, so fall back to its folder and name). + let is_source = |picked: &std::path::Path, source: &Option| { + let Some(src) = source else { return false }; + if let (Ok(a), Ok(b)) = (std::fs::canonicalize(picked), std::fs::canonicalize(src)) { + return a == b; + } + let same_name = picked.file_name().is_some_and(|n| n.to_string_lossy() == src.as_str()); + let same_folder = std::fs::canonicalize(src) + .ok() + .zip(picked.parent().map(std::path::Path::to_path_buf)) + .is_some_and(|(s, parent)| s.parent().is_some_and(|p| p == parent)); + same_name && same_folder + }; + #[cfg(not(target_arch = "wasm32"))] + if let Some(dir) = app.export_dir_override.clone() { + let target = std::path::Path::new(&dir).join(&name); + if is_source(&target, &source) { + app.notify_tr("That is the source file; choose another destination"); + return; + } + write_editable(app, &target, bytes); + return; + } + #[cfg(not(target_arch = "wasm32"))] + { + let dialog = rfd::AsyncFileDialog::new().add_filter("PDF", &["pdf"]).set_file_name(&name); + app.ask_one(crate::pickers::Ask::Save(dialog), None, move |app, picked: std::path::PathBuf| { + if is_source(&picked, &source) { + app.notify_tr("That is the source file; choose another destination"); + return; + } + write_editable(app, &picked, bytes); + }); + } + // The web build has no file system to write into: the new document downloads under its + // `_ocr.pdf` name, so it never lands on the source either. + #[cfg(target_arch = "wasm32")] + { + let _ = (&source, doc, is_source); + let saved = crate::editing::download(&name, &bytes); + with_state(app, |v, _| { + if saved.is_ok() { + v.finish(); + } else { + v.phase = Phase::ReviewReady; + } + }); + if let Err(e) = saved { + app.notify_error(e); + } + } +} + +/// The editable write itself: [`Phase::Saving`] while it runs, Idle once the file lands (the +/// review is done), back to ReviewReady when it fails — the reviewer sees the error and can +/// accept again without re-reading the document. Desktop only: the web build downloads instead. +#[cfg(not(target_arch = "wasm32"))] +fn write_editable(app: &mut PdfCraftApp, target: &std::path::Path, bytes: Arc>) { + with_state(app, |v, _| v.phase = Phase::Saving); + match crate::editing::write_atomically(&target.to_string_lossy(), &bytes) { + Ok(()) => { + with_state(app, |v, _| v.finish()); + app.notify(crate::i18n::fmt(tl!("Wrote {name}"), &[("name", &target.to_string_lossy())])); + } + Err(e) => { + with_state(app, |v, _| v.phase = Phase::ReviewReady); + app.notify_error(e); + } + } +} + +/// Pull the document's changed appearance into its view after an apply. +fn refresh_view(app: &mut PdfCraftApp, doc: DocId) { + if let Some(info) = app.session.get(doc).map(|d| d.info.clone()) + && let Some(view) = app.views.iter_mut().find(|view| view.id == doc) + { + view.document_changed(&info); + } +} + +#[cfg(test)] +mod tests { + use super::*; + use pdfcraft_engine::Session; + + /// A word box that sits where a real placed word would (high on the page, reading left + /// to right). + fn word(text: &str, conf: Option) -> PlacedWord { + PlacedWord { text: text.into(), origin: [72.0, 700.0], across: [40.0, 0.0], up: [0.0, 12.0], confidence: conf, source: "ocrs".into() } + } + + /// One page of words with the given confidences, and a review loaded with it (the guard + /// comes from a real job, as it does on screen). + fn loaded(session: &mut Session, confs: &[Option]) -> (DocId, OcrVerify) { + let text = session.create_from_text("t", "Recognize these scanned words").expect("fixture"); + let id = session.open("t.pdf", None, text, None).expect("fixture"); + let job = session.ocr_job(id, &[0], OcrSettings::default()).expect("fixture"); + let page = OcrPage { already_on_page: 0, page: 0, words: confs.iter().map(|c| word("w", *c)).collect(), skipped: None, notes: Vec::new() }; + let mut v = OcrVerify::default(); + v.load(id, job.guard.clone(), vec![page]); + (id, v) + } + + /// The spec's status wording is fixed: a finished run that found nothing says so, and a + /// cancelled one never claims a complete run. + #[test] + fn finished_states_keep_the_spec_wording() { + let mut v = OcrVerify::default(); + v.finished(vec![OcrPage { already_on_page: 0, page: 0, words: vec![], skipped: None, notes: Vec::new() }], false); + assert_eq!(v.phase, Phase::ReviewReady); + assert_eq!(v.message, "OCR complete — no text recognized on this page."); + v.finished( + vec![OcrPage { + already_on_page: 0, + page: 0, + words: vec![word("Hi", Some(95.0)), word("there", Some(40.0))], + skipped: None, + notes: Vec::new(), + }], + true, + ); + assert!(v.message.starts_with("Cancelled — 2 words were read on 1 of 1 pages"), "{}", v.message); + assert_eq!(v.phase, Phase::ReviewReady, "a cancel keeps the pages already read, in review"); + assert_eq!(v.page(0).expect("the cancelled run's page is kept").words.len(), 2); + } + + /// The legend's text is the classifier's thresholds, spelled out — a drift between the + /// two would put the wrong words in front of the reviewer. + #[test] + fn legend_rows_carry_the_classifier_thresholds() { + let rows = legend_rows(); + assert_eq!(rows[0].0, ConfidenceBand::High); + assert_eq!(rows[1].0, ConfidenceBand::Medium); + assert_eq!(rows[2].0, ConfidenceBand::Low); + assert_eq!(rows[0].1, "HIGH (>= 90%)"); + assert_eq!(rows[1].1, "MEDIUM (70-89%)"); + assert_eq!(rows[2].1, "LOW (< 70%)"); + // The same classifier drives the overlay: 90 is High, 89 is not; 70 is Medium, 69 is + // not; the bands clamp out-of-range values. + assert_eq!(band_for(90.0), ConfidenceBand::High); + assert_eq!(band_for(89.0), ConfidenceBand::Medium); + assert_eq!(band_for(70.0), ConfidenceBand::Medium); + assert_eq!(band_for(69.0), ConfidenceBand::Low); + assert_eq!(band_for(-1.0), ConfidenceBand::Low); + assert_eq!(band_for(140.0), ConfidenceBand::High); + } + + /// F8/Shift+F8 walk the uncertain words: wrap around, skip removed words, and a page + /// marked verified drops out entirely. With nothing to show, the walk is a no-op. + #[test] + fn uncertain_walk_wraps_and_skips_removed_and_verified() { + let mut s = Session::new(); + // 95 is fine; 40 and 50 are Low; the third word is corrected into the dictionary. + let (_, mut v) = loaded(&mut s, &[Some(95.0), Some(40.0), Some(50.0), Some(95.0)]); + assert_eq!(v.uncertain(), vec![(0, 1), (0, 2)], "the Low-band words, in order"); + assert!(v.goto_uncertain(true)); + assert_eq!(v.selected, Some((0, 1))); + assert!(v.goto_uncertain(true)); + assert_eq!(v.selected, Some((0, 2))); + assert!(v.goto_uncertain(true), "wraps around"); + assert_eq!(v.selected, Some((0, 1))); + // A removed word leaves the walk; a corrected word stays by its original confidence. + v.set_removed((0, 1), true); + assert!(v.goto_uncertain(true)); + assert_eq!(v.selected, Some((0, 2))); + assert!(v.correct((0, 2), "fixed"), "the correction commits"); + assert!(v.uncertain().contains(&(0, 2)), "a corrected word stays uncertain by its original confidence"); + v.set_word_verified((0, 2), true); + assert!(v.uncertain().is_empty(), "a vouched-for word leaves the walk"); + assert!(!v.goto_uncertain(true), "nothing left: a no-op"); + // A page marked verified drops all its words from the walk. + v.set_word_verified((0, 2), false); + v.set_removed((0, 1), false); + v.set_page_verified(0, true); + assert!(v.uncertain().is_empty()); + } + + /// The review tools are guarded: outside ReviewReady they are ignored, not acted on. + #[test] + fn guards_ignore_review_actions_outside_reviewready() { + let mut v = OcrVerify::default(); + assert_eq!(v.phase, Phase::Idle); + assert!(v.can_run(), "Run is enabled in Idle"); + assert!(!v.reject(), "Reject is ignored in Idle"); + assert!(!v.set_removed((0, 0), true)); + assert!(!v.correct((0, 0), "x")); + assert!(!v.set_page_verified(0, true)); + assert!(!v.reject_page(0)); + let mut s = Session::new(); + assert_eq!(v.accept(&mut s).expect("ignored, not an error"), 0); + v.phase = Phase::Saving; + assert!(!v.can_run(), "Run is disabled while saving"); + v.phase = Phase::RecoverableError; + assert!(v.can_run(), "Run is enabled again after a recoverable error"); + } + + /// Accept: zero words left is a typed failure; a real review is applied as ONE undoable + /// step; a stale session is refused with its reason and the edits are kept. + #[test] + fn accept_zero_words_fails_and_a_live_review_applies_once() { + let mut s = Session::new(); + let (id, mut v) = loaded(&mut s, &[Some(95.0), Some(40.0)]); + v.set_removed((0, 0), true); + v.set_removed((0, 1), true); + assert!(v.accept(&mut s).is_err(), "zero words: a typed failure"); + assert_eq!(v.phase, Phase::ReviewReady, "the failure keeps the review"); + let live = v.live_pages(); + assert!(live[0].words.is_empty(), "removed words are not written"); + // Restore one word: the accept applies the whole review in one step. + v.set_removed((0, 0), false); + let n = v.accept(&mut s).expect("the accept applies"); + assert_eq!(n, 1); + assert_eq!(s.get(id).expect("open").can_undo(), Some("Recognize text"), "one undoable step"); + assert_eq!(v.phase, Phase::Idle, "a successful accept ends the review"); + // A second accept of the same (now spent) review has nothing to write. + let mut v2 = OcrVerify::default(); + let job = s.ocr_job(id, &[0], OcrSettings::default()).expect("fixture"); + v2.load( + id, + job.guard.clone(), + vec![OcrPage { already_on_page: 0, page: 0, words: vec![word("w", Some(95.0))], skipped: None, notes: Vec::new() }], + ); + v2.set_removed((0, 0), true); + assert!(v2.accept(&mut s).is_err()); + } + + /// Accept refuses a stale session: the document was edited since the job started, the + /// refusal names the reason, and the review edits survive. + #[test] + fn accept_refuses_a_stale_session_with_a_named_reason() { + let mut s = Session::new(); + let (id, mut v) = loaded(&mut s, &[Some(95.0)]); + assert!(v.correct((0, 0), "Corrected"), "the correction commits"); + // The document changed since the job started: the result no longer applies. + s.apply(id, pdfcraft_engine::Edit::RotatePages { pages: vec![0], degrees: 90 }).expect("fixture"); + let why = v.accept(&mut s).expect_err("the stale result is refused"); + assert!(why.contains("changed"), "{why}"); + assert_eq!(v.phase, Phase::RecoverableError, "Run is enabled again"); + assert_eq!(v.text_of((0, 0)), Some("Corrected"), "the edits are kept"); + } + + /// "Accept entire page" applies just that page and a later whole-review Accept never + /// writes it twice; "Reject entire page" clears the page (undoable in the panel). + #[test] + fn accept_page_then_accept_does_not_double_write() { + let mut s = Session::new(); + let (_, mut v) = loaded(&mut s, &[Some(95.0), Some(95.0)]); + v.pages = vec![OcrPage { + already_on_page: 0, + page: 0, + words: vec![word("one", Some(95.0)), word("two", Some(95.0))], + skipped: None, + notes: Vec::new(), + }]; + let n = v.accept_page(&mut s, 0).expect("the page applies"); + assert_eq!(n, 2); + assert!(v.review.applied_pages.contains(&0)); + // Reject entire page: the words clear (undoable), then come back. + assert!(v.reject_page(0)); + assert!(v.page(0).expect("page").words.is_empty()); + assert!(v.undo(), "the clear is undoable"); + assert_eq!(v.page(0).expect("page").words.len(), 2); + // A zero-word page cannot be accepted. + v.reject_page(0); + assert!(v.accept_page(&mut s, 0).is_err(), "zero words on the page: a typed failure"); + } + + /// The text the document's page `page` carries now, as the renderer extracts it. + fn page_text(s: &Session, id: DocId, page: usize) -> String { + let doc = s.get(id).expect("open"); + let mut r = pdfcraft_render::PageRenderer::new(doc.bytes.clone(), pdfcraft_render::RenderConfig::default()); + let request = pdfcraft_render::RenderRequest { page, kind: pdfcraft_render::RequestKind::Text, scale: 1.0, ..Default::default() }; + r.render(request).text.map(|t| t.plain_text()).unwrap_or_default() + } + + /// A blank document of `pages` pages, open, with a review of one word list per page loaded + /// against a real job's guard. + fn blank_review(s: &mut Session, words: Vec>) -> (DocId, OcrVerify) { + let blank = s.create_blank(612.0, 792.0, words.len()).expect("fixture"); + let id = s.open("blank.pdf", None, blank, None).expect("fixture"); + let job = s.ocr_job(id, &[], OcrSettings::default()).expect("fixture"); + let pages = words + .into_iter() + .enumerate() + .map(|(page, words)| OcrPage { already_on_page: 0, page, words, skipped: None, notes: Vec::new() }) + .collect(); + let mut v = OcrVerify::default(); + v.load(id, job.guard.clone(), pages); + (id, v) + } + + /// Pages accepted one at a time, then the whole review: each page-accept is the document's + /// newest edit, so the next one (and the final Accept) still applies, and every page is + /// written exactly once. Before the guard was refreshed, the second page-accept was refused + /// as "the document changed since recognition started" and the review was stuck. + #[test] + fn accepting_pages_one_by_one_then_the_review_applies_every_page_once() { + let mut s = Session::new(); + let (id, mut v) = blank_review(&mut s, vec![vec![word("one", Some(95.0))], vec![word("two", Some(95.0))], vec![word("three", Some(95.0))]]); + assert_eq!(v.accept_page(&mut s, 0), Ok(1)); + assert_eq!(v.accept_page(&mut s, 1), Ok(1), "the second page applies after the first"); + assert_eq!(v.phase, Phase::ReviewReady); + assert_eq!(v.accept(&mut s), Ok(1), "the final Accept writes only the page not yet applied"); + assert_eq!(v.phase, Phase::Idle, "the review ended"); + for (page, text) in ["one", "two", "three"].into_iter().enumerate() { + let on_page = page_text(&s, id, page); + assert_eq!(on_page.matches(text).count(), 1, "page {page} written exactly once: {on_page:?}"); + } + } + + /// A page already applied ("Accept entire page") is never applied again: a second + /// "Accept entire page" on it writes nothing (the OCR layer is not appended twice), and + /// corrections made after the page-accept are kept in the review. + #[test] + fn accepting_a_page_twice_writes_it_once_and_keeps_later_corrections() { + let mut s = Session::new(); + let (id, mut v) = blank_review(&mut s, vec![vec![word("one", Some(95.0))], vec![word("two", Some(95.0))]]); + assert_eq!(v.accept_page(&mut s, 0), Ok(1), "the first accept applies the page"); + // The reviewer corrects a word on the applied page afterwards. + assert!(v.correct((0, 0), "uno"), "the correction commits"); + // A second "Accept entire page" on the same page writes nothing. + assert_eq!(v.accept_page(&mut s, 0), Ok(0), "already applied: ignored, not applied twice"); + let text = page_text(&s, id, 0); + assert_eq!(text.matches("one").count(), 1, "the page was written exactly once: {text:?}"); + // The later correction is kept (not lost by the refusal), and the final Accept writes + // only the page not yet applied. + assert_eq!(v.text_of((0, 0)), Some("uno")); + assert_eq!(v.accept(&mut s), Ok(1), "the final Accept writes only page 2"); + let text = page_text(&s, id, 0); + assert_eq!(text.matches("one").count() + text.matches("uno").count(), 1, "still written once: {text:?}"); + assert_eq!(page_text(&s, id, 1).matches("two").count(), 1); + } + + /// The refreshed guard still catches an edit made elsewhere between two page-accepts: the + /// next page is refused with the reason, nothing is written, and the review is kept. + #[test] + fn an_outside_edit_between_page_accepts_is_still_refused() { + let mut s = Session::new(); + let (id, mut v) = blank_review(&mut s, vec![vec![word("one", Some(95.0))], vec![word("two", Some(95.0))]]); + assert_eq!(v.accept_page(&mut s, 0), Ok(1)); + s.apply(id, pdfcraft_engine::Edit::RotatePages { pages: vec![0], degrees: 90 }).expect("an unrelated edit"); + let why = v.accept_page(&mut s, 1).expect_err("stale after an outside edit"); + assert!(why.contains("changed since recognition started"), "{why}"); + assert_eq!(v.phase, Phase::RecoverableError); + assert_eq!(v.page(1).expect("page").words.len(), 1, "the review is kept"); + assert!(page_text(&s, id, 1).trim().is_empty(), "nothing was written to page 2"); + } + + /// A word at `x` on the first line of a US Letter page, with a Low confidence. + fn word_at(text: &str, x: f64) -> PlacedWord { + PlacedWord { text: text.into(), origin: [x, 700.0], across: [40.0, 0.0], up: [0.0, 12.0], confidence: Some(40.0), source: "ocrs".into() } + } + + /// Whole-page re-read: review marks are positional, so the page's marks go with the old + /// words (a struck-out slot must not strike the new word there), a correction of an old + /// word no longer shows on the new one, and the undo restores words and marks exactly. + #[test] + fn rereading_a_page_clears_its_marks_and_undo_restores_them() { + let mut s = Session::new(); + let (_, mut v) = blank_review(&mut s, vec![vec![word_at("keep", 72.0), word_at("gone", 300.0), word_at("struck", 400.0)]]); + assert!(v.set_removed((0, 2), true)); + assert!(v.set_word_verified((0, 0), true)); + assert!(v.correct((0, 1), "fixed")); + let marks = (v.review.removed.clone(), v.review.verified.clone()); + assert!(v.replace_page_words(0, vec![word_at("a", 72.0), word_at("b", 300.0), word_at("c", 400.0)])); + assert!(v.review.removed.is_empty() && v.review.verified.is_empty(), "the old marks are gone"); + assert!(!v.is_corrected((0, 1)), "the correction belonged to the old word list"); + let texts: Vec<_> = v.live_pages()[0].words.iter().map(|w| w.text.clone()).collect(); + assert_eq!(texts, ["a", "b", "c"], "no new word is struck out by an old mark"); + assert!(v.undo()); + assert_eq!((v.review.removed.clone(), v.review.verified.clone()), marks, "the undo restores the marks"); + assert!(v.is_corrected((0, 1))); + let texts: Vec<_> = v.live_pages()[0].words.iter().map(|w| w.text.clone()).collect(); + assert_eq!(texts, ["keep", "fixed"], "the struck word stays out"); + } + + /// A blank two-page-shaped review inside a real app (the document open, Run set to go + /// inline), one word list per page, against a real job's guard. + fn app_review(words: Vec>) -> (PdfCraftApp, DocId) { + let blank = Session::new().create_blank(612.0, 792.0, words.len()).expect("fixture").to_vec(); + let mut app = PdfCraftApp::new(); + app.run_inline = true; + app.open_bytes("blank.pdf", None, blank).expect("fixture"); + let id = app.active_ids().expect("open").1; + let job = app.session.ocr_job(id, &[], OcrSettings::default()).expect("fixture"); + let pages = words + .into_iter() + .enumerate() + .map(|(page, words)| OcrPage { already_on_page: 0, page, words, skipped: None, notes: Vec::new() }) + .collect(); + app.ocr_verify.load(id, job.guard.clone(), pages); + (app, id) + } + + /// The status the tests see, from the app's last toast. + fn toast(app: &mut PdfCraftApp) -> String { + app.toast.take().map(|t| t.0).unwrap_or_default() + } + + /// A region re-read whose worker has finished with `result`, already asked for. + fn finished_reread(region: Option<[f32; 4]>, result: Result, String>) -> Reread { + Reread { page: 0, region, progress: std::sync::Arc::new(Mutex::new(RereadProgress { cancel: false, result: Some(result) })) } + } + + /// A page re-read runs OFF the UI thread: start_reread returns with the reading in flight + /// (Run is refused while it is out), and the worker's result is applied by the next poll. + /// No OCR models are needed: the engine fails fast here, which still walks the whole + /// off-thread path — and a failed reading keeps the words. + #[test] + fn a_page_reread_runs_off_the_ui_thread_and_poll_applies_it() { + let (mut app, _) = app_review(vec![vec![word("one", Some(95.0))]]); + app.ocr_draft.engine = EngineChoice::Ocrs; // an ocrs-only job never probes tesseract + app.run_inline = false; // the point of the test: a real worker thread + with_state(&mut app, |v, app| v.start_reread(app, 0, None)); + assert!(app.ocr_verify.reread.is_some(), "the reading is in flight"); + assert!(app.ocr_verify.message.contains("Re-reading"), "{}", app.ocr_verify.message); + // A full run waits for the re-read. + with_state(&mut app, OcrVerify::start_run); + assert_eq!(app.ocr_verify.phase, Phase::ReviewReady, "Run is refused while a re-read is out"); + // Wait for the worker (it fails fast without models), then apply its result. + let progress = app.ocr_verify.reread.as_ref().expect("in flight").progress.clone(); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(30); + while progress.lock().expect("progress").result.is_none() && std::time::Instant::now() < deadline { + std::thread::sleep(std::time::Duration::from_millis(20)); + } + assert!(progress.lock().expect("progress").result.is_some(), "the worker finished"); + with_state(&mut app, OcrVerify::poll_reread); + assert!(app.ocr_verify.reread.is_none(), "the re-read is over"); + assert_eq!(app.ocr_verify.phase, Phase::ReviewReady, "the review survives a failed re-read"); + assert_eq!(app.ocr_verify.page(0).expect("page").words.len(), 1, "the words survive"); + assert!(!toast(&mut app).is_empty(), "the failure is said"); + } + + /// A re-read that finds nothing is a typed failure: the words stay, the review stays. + #[test] + fn a_reread_that_finds_nothing_keeps_the_words() { + let (mut app, _) = app_review(vec![vec![word("one", Some(95.0))]]); + app.ocr_verify.reread = Some(finished_reread(None, Err("the re-read found no text; the existing words are unchanged".into()))); + with_state(&mut app, OcrVerify::poll_reread); + assert!(app.ocr_verify.reread.is_none(), "the re-read is over"); + assert_eq!(app.ocr_verify.phase, Phase::ReviewReady); + assert_eq!(app.ocr_verify.page(0).expect("page").words.len(), 1, "the words survive the failed re-read"); + assert!(toast(&mut app).contains("found no text")); + } + + /// Cancel asks the worker to stop and discards the reading when it lands anyway: the + /// page's words stay as they were. + #[test] + fn a_cancelled_reread_is_discarded_and_keeps_the_words() { + let (mut app, _) = app_review(vec![vec![word("one", Some(95.0))]]); + app.ocr_verify.reread = Some(Reread { page: 0, region: None, progress: std::sync::Arc::new(Mutex::new(RereadProgress::default())) }); + with_state(&mut app, |v, _| v.cancel_reread()); + assert!(app.ocr_verify.reread.as_ref().expect("in flight").progress.lock().expect("progress").cancel, "the cancel reached the worker"); + // The worker finishes anyway; its (late) reading is discarded. + app.ocr_verify.reread.as_ref().expect("in flight").progress.lock().expect("progress").result = Some(Ok(vec![word("late", Some(95.0))])); + with_state(&mut app, OcrVerify::poll_reread); + assert!(app.ocr_verify.reread.is_none()); + let words: Vec<_> = app.ocr_verify.page(0).expect("page").words.iter().map(|w| w.text.clone()).collect(); + assert_eq!(words, ["one"], "the late reading is discarded"); + assert!(app.ocr_verify.message.contains("cancelled"), "{}", app.ocr_verify.message); + } + + /// A re-read of a document that changed since the review began is refused: the reading may + /// no longer fit the page it was read from, so the words stay and the reason is said. + #[test] + fn a_reread_of_a_changed_document_is_refused() { + let (mut app, id) = app_review(vec![vec![word("one", Some(95.0))], vec![word("two", Some(95.0))]]); + app.ocr_verify.reread = Some(finished_reread(None, Ok(vec![word("late", Some(95.0))]))); + app.ocr_verify.current = 1; + app.session.apply(id, pdfcraft_engine::Edit::RotatePages { pages: vec![1], degrees: 90 }).expect("an edit"); + with_state(&mut app, OcrVerify::poll_reread); + let words: Vec<_> = app.ocr_verify.page(1).expect("page").words.iter().map(|w| w.text.clone()).collect(); + assert_eq!(words, ["two"], "the late reading is not applied"); + assert!(toast(&mut app).contains("changed"), "the refusal names its reason"); + } + + /// Region re-read (the worker's reading applied by the poll): the words outside the + /// rectangle stay and their marks follow them to their new positions; the replaced word's + /// marks go; the first NEW word is the one selected (it follows the kept words); undo + /// restores everything. + #[test] + fn rereading_a_region_remaps_marks_and_selects_the_first_new_word() { + let (mut app, _) = app_review(vec![vec![word_at("keep", 72.0), word_at("gone", 300.0), word_at("struck", 400.0)]]); + // The drawn region and the worker's reading of it: one word, "new". + let region = [290.0, 70.0, 350.0, 100.0]; + let new_word = PlacedWord { + text: "new".into(), + origin: [300.0, 700.0], + across: [40.0, 0.0], + up: [0.0, 12.0], + confidence: Some(95.0), + source: "ocrs".into(), + }; + { + let v = &mut app.ocr_verify; + assert!(v.set_removed((0, 2), true)); + assert!(v.set_word_verified((0, 0), true)); + assert!(v.set_word_verified((0, 1), true)); + v.region = Some(region); + v.reread = Some(finished_reread(Some(region), Ok(vec![new_word]))); + } + let marks = { + let v = &app.ocr_verify; + (v.review.removed.clone(), v.review.verified.clone()) + }; + // "gone" spans x 300..340 and y 700..712 in user space: view y 80..92 on the 792 pt page. + with_state(&mut app, OcrVerify::poll_reread); + let v = &mut app.ocr_verify; + assert!(v.reread.is_none(), "the re-read is over"); + let texts: Vec<_> = v.page(0).expect("page").words.iter().map(|w| w.text.clone()).collect(); + assert_eq!(texts, ["keep", "struck", "new"]); + assert_eq!(v.selected, Some((0, 2)), "the first new word is selected"); + assert!(v.is_removed((0, 1)), "the struck word's mark moved with it"); + assert!(!v.is_removed((0, 2)), "the new word is not struck out"); + assert!(v.is_verified((0, 0)) && !v.is_verified((0, 1)) && !v.is_verified((0, 2)), "{:?}", v.review.verified); + let live: Vec<_> = v.live_pages()[0].words.iter().map(|w| w.text.clone()).collect(); + assert_eq!(live, ["keep", "new"], "the struck word is never written, the new one is"); + assert!(v.undo()); + assert_eq!((v.review.removed.clone(), v.review.verified.clone()), marks); + let texts: Vec<_> = v.page(0).expect("page").words.iter().map(|w| w.text.clone()).collect(); + assert_eq!(texts, ["keep", "gone", "struck"]); + } + + /// The editable output: refuses zero words, keeps the open document untouched, and names + /// the new file `_ocr.pdf`. + #[test] + fn editable_output_refuses_zero_words_and_never_touches_the_source() { + let mut s = Session::new(); + let (id, mut v) = loaded(&mut s, &[Some(95.0)]); + let before = s.get(id).expect("open").bytes.clone(); + let (bytes, name) = v.editable_output(&s).expect("the output builds"); + assert!(name.ends_with("_ocr.pdf"), "{name}"); + assert!(std::sync::Arc::ptr_eq(&s.get(id).expect("open").bytes, &before), "the source is untouched"); + assert!(!bytes.is_empty()); + v.set_removed((0, 0), true); + assert!(v.editable_output(&s).is_err(), "zero words: refused"); + assert_eq!(v.phase, Phase::ReviewReady); + } + + /// The editable output is a new file made from words read earlier: when the document changed + /// since (a redaction, say), it is refused with the reason, as Accept in searchable mode is. + #[test] + fn editable_output_refuses_a_document_that_changed_since_the_review_started() { + let mut s = Session::new(); + let (id, v) = loaded(&mut s, &[Some(95.0)]); + assert!(v.editable_output(&s).is_ok(), "an untouched document is fine"); + s.apply(id, pdfcraft_engine::Edit::RotatePages { pages: vec![0], degrees: 90 }).expect("an edit"); + let err = v.editable_output(&s).expect_err("a changed document is refused"); + assert!(err.contains("changed since recognition started"), "{err}"); + } + + /// The editable save is the Saving state: the file lands and the review ends (Idle); a + /// failed write returns to ReviewReady with the edits kept, and a cancelled or unanswered + /// save dialog never touches the review either — Accept can simply be tried again. + #[test] + fn editable_save_lands_and_a_failed_or_cancelled_one_keeps_the_review() { + let mut app = PdfCraftApp::new(); + app.run_inline = true; + let text = app.session.create_from_text("t", "Recognize these scanned words").expect("fixture"); + let id = app.session.open("t.pdf", None, text, None).expect("fixture"); + let job = app.session.ocr_job(id, &[0], OcrSettings::default()).expect("fixture"); + app.ocr_verify.load( + id, + job.guard.clone(), + vec![OcrPage { already_on_page: 0, page: 0, words: vec![word("w", Some(95.0))], skipped: None, notes: Vec::new() }], + ); + assert!(app.ocr_verify.correct((0, 0), "Edited"), "the correction commits"); + app.ocr_draft.output_mode = OutputMode::EditableText; + // A destination that cannot be written (the "folder" is a regular file): the write + // fails, the review stays in ReviewReady with the edit, Accept is enabled again. + let blocker = std::env::temp_dir().join(format!("pdfcraft-ocr-save-block-{}", std::process::id())); + std::fs::write(&blocker, b"not a folder").expect("fixture"); + app.export_dir_override = Some(blocker.to_string_lossy().into_owned()); + with_state(&mut app, do_accept); + assert_eq!(app.ocr_verify.phase, Phase::ReviewReady, "a failed write keeps the review"); + assert_eq!(app.ocr_verify.text_of((0, 0)), Some("Edited"), "the edits are kept"); + // A cancelled save dialog (the override answers with nothing): nothing is written, + // nothing changes. + app.export_dir_override = None; + app.pick_override = Some(Vec::new()); + with_state(&mut app, do_accept); + app.process_picked(); + assert_eq!(app.ocr_verify.phase, Phase::ReviewReady, "a cancelled save keeps the review"); + assert_eq!(app.ocr_verify.text_of((0, 0)), Some("Edited")); + assert_eq!(app.ocr_verify.doc, Some(id), "the review still belongs to its document"); + // A real destination: the file lands under `_ocr.pdf` and the review ends. + let dir = std::env::temp_dir().join(format!("pdfcraft-ocr-save-out-{}", std::process::id())); + std::fs::create_dir_all(&dir).expect("fixture"); + app.pick_override = Some(vec![dir.join("t_ocr.pdf").to_string_lossy().into_owned()]); + with_state(&mut app, do_accept); + app.process_picked(); + assert!(dir.join("t_ocr.pdf").exists(), "the editable copy is written"); + assert_eq!(app.ocr_verify.phase, Phase::Idle, "a successful save ends the review"); + let _ = std::fs::remove_file(&blocker); + let _ = std::fs::remove_dir_all(&dir); + } + + /// Region re-OCR refuses an empty, inverted or out-of-page rectangle BEFORE running: no + /// job goes out, no engine is probed, and the existing words are intact. + #[test] + fn reread_refuses_an_empty_region_before_running() { + let (mut app, id) = app_review(vec![vec![word("one", Some(95.0))]]); + app.ocr_draft.dpi = 72; // one pixel per point, so the arithmetic below is exact + // An empty rectangle never reaches the engine. + with_state(&mut app, |v, app| v.start_reread(app, 0, Some([10.0, 10.0, 10.0, 60.0]))); + assert!(app.ocr_verify.reread.is_none(), "nothing runs"); + assert_eq!(app.ocr_verify.page(0).expect("page").words.len(), 1, "the existing words are intact"); + assert!(toast(&mut app).contains("empty or outside")); + // Nor does one wholly outside the page. + with_state(&mut app, |v, app| v.start_reread(app, 0, Some([-50.0, -50.0, -1.0, -1.0]))); + assert!(app.ocr_verify.reread.is_none()); + assert_eq!(id, app.ocr_verify.doc.expect("the review keeps its document")); + } + + /// Review edits are undoable within the panel, in order. + #[test] + fn review_edits_undo_in_order() { + let mut s = Session::new(); + let (_, mut v) = loaded(&mut s, &[Some(40.0)]); + assert!(v.correct((0, 0), "Fixed"), "commits"); + assert_eq!(v.text_of((0, 0)), Some("Fixed")); + assert!(v.is_corrected((0, 0))); + assert!(v.add_to_dictionary("Fixed", "en")); + assert!(v.review.dictionary.contains("fixed")); + assert!(v.set_removed((0, 0), true)); + assert!(v.is_removed((0, 0))); + // Undo in order: the removal, then the dictionary add, then the correction. + assert!(v.undo()); + assert!(!v.is_removed((0, 0))); + assert!(v.undo()); + assert!(!v.review.dictionary.contains("fixed")); + assert!(v.undo()); + assert_eq!(v.text_of((0, 0)), Some("w")); + assert!(!v.undo(), "nothing left to undo"); + } + + /// Corrections commit on Enter; an empty (after trim) commit removes the word; the box + /// never moves. + #[test] + fn correction_commits_and_clearing_removes() { + let mut s = Session::new(); + let (_, mut v) = loaded(&mut s, &[Some(95.0)]); + let box_before = v.word((0, 0)).map(|w| (w.origin, w.across, w.up)); + assert!(v.correct((0, 0), " Fixed "), "commits"); + assert_eq!(v.text_of((0, 0)), Some(" Fixed "), "the reviewer's text is kept verbatim"); + assert_eq!(v.word((0, 0)).map(|w| (w.origin, w.across, w.up)), box_before, "the box never moves"); + assert!(v.correct((0, 0), " "), "an empty commit removes"); + assert!(v.is_removed((0, 0)), "clearing the text removes the word"); + } + + /// Suggestions: dictionary words and the page's own words within edit distance 2, nearest + /// first, at most 5, never the word itself. + #[test] + fn suggestions_are_near_dictionary_and_page_words() { + let mut dictionary = BTreeSet::new(); + dictionary.insert("recognize".into()); + dictionary.insert("these".into()); + let page_words = vec![word("recognise", None), word("scanned", None), word("words", None), word("completely", None)]; + let sugg = suggestions("recgnise", &dictionary, &page_words); + assert_eq!(sugg.first().map(String::as_str), Some("recognise"), "distance 1 beats distance 2"); + assert!(sugg.contains(&"recognize".to_string()), "distance 2 candidates are in"); + assert!(!sugg.contains(&"recgnise".to_string()), "never the word itself"); + assert!(sugg.len() <= 5); + assert!(suggestions("xyzzyx", &dictionary, &page_words).is_empty(), "nothing within distance 2"); + assert!(edit_distance_within_2("kitten", "sitting", 2).is_none(), "distance 3 is out"); + assert_eq!(edit_distance_within_2("kitten", "mitten", 2), Some(1)); + } + + /// The page list's data: a page marked verified, a skipped page, and the verified + /// fraction the info strip shows. + #[test] + fn verified_fraction_and_word_count_follow_the_review() { + let mut s = Session::new(); + let (_, mut v) = loaded(&mut s, &[Some(95.0), Some(95.0), Some(95.0)]); + assert!((v.verified_fraction() - 0.0).abs() < f32::EPSILON); + v.set_word_verified((0, 0), true); + assert!((v.verified_fraction() - 1.0 / 3.0).abs() < 0.01); + v.set_page_verified(0, true); + assert!((v.verified_fraction() - 1.0).abs() < f32::EPSILON, "a verified page vouches for all its words"); + assert_eq!(v.word_count(), 3); + v.set_removed((0, 1), true); + assert_eq!(v.word_count(), 2, "removed words are not counted"); + // Reject clears the words, the session, the selection and the flags; back to Idle. + assert!(v.reject()); + assert_eq!(v.phase, Phase::Idle); + assert!(v.pages.is_empty() && v.selected.is_none() && v.review.verified_pages.is_empty()); + assert!(!v.reject(), "Reject outside ReviewReady is ignored"); + } + + /// The suspects' count the info strip shows: live Low-band words that are not in the + /// dictionary. + #[test] + fn suspect_count_follows_bands_and_dictionary() { + let mut s = Session::new(); + let (_, mut v) = loaded(&mut s, &[Some(95.0), Some(40.0), Some(69.0)]); + assert_eq!(v.suspect_count(), 2, "69 is Low, 40 is Low, 95 is High"); + v.add_to_dictionary("w", "en"); + assert_eq!(v.suspect_count(), 0, "dictionary words do not ask for review"); + } + + #[test] + fn dictionary_uses_recognition_language_instead_of_interface_locale() { + let mut session = Session::new(); + let (_, mut v) = loaded(&mut session, &[Some(40.0)]); + let locale = crate::i18n::current().code(); + let language = if locale == "en" { "fr" } else { "en" }; + let dir = std::env::temp_dir().join(format!("pdfcraft-ocr-dictionary-language-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + std::fs::write(dir.join(format!("ocr-dictionary-{locale}.txt")), "wrong\n").unwrap(); + let path = dir.join(format!("ocr-dictionary-{language}.txt")); + std::fs::write(&path, "recognized\n").unwrap(); + v.dictionary_dir = Some(dir.clone()); + v.dictionary_language = language.into(); + v.load_dictionary_pref(); + assert!(v.review.dictionary.contains("recognized")); + assert!(!v.review.dictionary.contains("wrong")); + assert!(v.add_to_dictionary("Added", language)); + assert!(std::fs::read_to_string(path).unwrap().contains("added\n")); + assert_eq!(std::fs::read_to_string(dir.join(format!("ocr-dictionary-{locale}.txt"))).unwrap(), "wrong\n"); + } + + #[test] + fn dictionary_limit_and_normalization_failures_preserve_file_and_review() { + let mut session = Session::new(); + let (_, mut v) = loaded(&mut session, &[Some(40.0)]); + let dir = std::env::temp_dir().join(format!("pdfcraft-ocr-dictionary-limits-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let path = dir.join("ocr-dictionary-en.txt"); + let full: String = (0..65536).map(|i| format!("word{i}\n")).collect(); + std::fs::write(&path, &full).unwrap(); + v.dictionary_dir = Some(dir); + v.load_dictionary_pref(); + let before = v.review.dictionary.clone(); + assert!(!v.add_to_dictionary("extra", "en")); + assert!(v.message.contains("65536"), "{}", v.message); + assert_eq!(v.review.dictionary, before); + assert_eq!(std::fs::read_to_string(&path).unwrap(), full); + assert!(!v.add_to_dictionary(&"İ".repeat(512), "en")); + assert!(v.message.contains("normalized"), "{}", v.message); + assert_eq!(v.review.dictionary, before); + assert_eq!(std::fs::read_to_string(&path).unwrap(), full); + let full_bytes: String = (0..8192).map(|i| format!("{i:05}{}\n", "a".repeat(506))).collect(); + assert_eq!(full_bytes.len(), pdfcraft_engine::ocr::dictionary::MAX_BYTES); + std::fs::write(&path, &full_bytes).unwrap(); + v.load_dictionary_pref(); + let before = v.review.dictionary.clone(); + assert!(!v.add_to_dictionary("extra", "en")); + assert!(v.message.contains("4 MiB"), "{}", v.message); + assert_eq!(v.review.dictionary, before); + assert_eq!(std::fs::read_to_string(&path).unwrap(), full_bytes); + // A corrupt file is not silently treated as a fresh, empty dictionary. + std::fs::write(&path, [0xff]).unwrap(); + v.load_dictionary_pref(); + assert_eq!(v.review.dictionary, before); + assert!(v.message.contains("OCR dictionary")); + } + + #[test] + fn dictionary_undo_persists_and_failed_undo_keeps_history() { + let mut session = Session::new(); + let (_, mut v) = loaded(&mut session, &[Some(40.0)]); + let dir = std::env::temp_dir().join(format!("pdfcraft-ocr-dictionary-undo-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let path = dir.join("ocr-dictionary-en.txt"); + std::fs::write(&path, "original\n").unwrap(); + v.dictionary_dir = Some(dir); + v.load_dictionary_pref(); + std::fs::write(&path, "original\nexternal\n").unwrap(); + assert!(!v.add_to_dictionary("external", "en"), "an externally added word creates no undo entry"); + assert!(v.review.dictionary.contains("external")); + assert!(v.add_to_dictionary("temporary", "en")); + assert!(v.undo()); + assert!(!std::fs::read_to_string(&path).unwrap().contains("temporary")); + assert!(v.add_to_dictionary("later", "en")); + assert!(!v.review.dictionary.contains("temporary")); + let before = v.review.dictionary.clone(); + std::fs::write(&path, [0xff]).unwrap(); + assert!(!v.undo()); + assert_eq!(v.review.dictionary, before); + assert_eq!(std::fs::read(&path).unwrap(), [0xff]); + std::fs::write(&path, "original\nexternal\nlater\n").unwrap(); + assert!(v.undo(), "failed undo keeps its operation for retry"); + assert_eq!(std::fs::read_to_string(&path).unwrap(), "external\noriginal\n"); + assert_eq!(v.review.dictionary, std::collections::BTreeSet::from(["external".into(), "original".into()])); + } + + #[test] + fn rereading_keeps_captured_language_when_the_draft_changes() { + let mut session = Session::new(); + let (_, mut v) = loaded(&mut session, &[Some(40.0)]); + v.dictionary_language = "ja".into(); + let draft = super::ocr_ui::OcrDraft { language: "fr".into(), ..Default::default() }; + let settings = v.reread_settings(&draft); + assert_eq!(settings.language, "ja"); + assert_eq!(settings.dpi, draft.settings().dpi); + assert!(dictionary_path(std::path::Path::new("prefs"), "../outside").is_err()); + } + + #[test] + fn finishing_or_rejecting_a_review_keeps_dictionary_configuration() { + let mut session = Session::new(); + let (_, mut v) = loaded(&mut session, &[Some(40.0)]); + let dir = PathBuf::from("configured-dictionaries"); + v.dictionary_dir = Some(dir.clone()); + assert!(v.reject()); + assert_eq!(v.dictionary_dir.as_ref(), Some(&dir)); + v.finish(); + assert_eq!(v.dictionary_dir.as_ref(), Some(&dir)); + } + + /// The drawn region is stored in view space, so its outline is that rectangle scaled onto + /// the pane: not mirrored on an upright page, and in place on a rotated, cropped one. + #[test] + fn the_region_outline_is_where_it_was_drawn() { + let img = egui::Rect::from_min_size(egui::pos2(10.0, 20.0), egui::vec2(306.0, 396.0)); + let upright = pdfcraft_render::PageInfo { width: 612.0, height: 792.0, label: "1".into(), crop: [0.0, 0.0, 612.0, 792.0], rotation: 0 }; + let r = pane_region([100.0, 50.0, 200.0, 150.0], &upright, img).expect("on the page"); + assert_eq!((r.left(), r.top(), r.right(), r.bottom()), (60.0, 45.0, 110.0, 95.0), "top region stays at the top"); + let turned = pdfcraft_render::PageInfo { width: 792.0, height: 612.0, label: "1".into(), crop: [36.0, 36.0, 648.0, 828.0], rotation: 90 }; + let img = egui::Rect::from_min_size(egui::pos2(0.0, 0.0), egui::vec2(396.0, 306.0)); + let r = pane_region([100.0, 50.0, 200.0, 150.0], &turned, img).expect("on the page"); + assert_eq!((r.left(), r.top(), r.right(), r.bottom()), (50.0, 25.0, 100.0, 75.0)); + } +} diff --git a/crates/ui-egui/tests/it/ocr.rs b/crates/ui-egui/tests/it/ocr.rs index d7a5501987..c1a987fbb3 100644 --- a/crates/ui-egui/tests/it/ocr.rs +++ b/crates/ui-egui/tests/it/ocr.rs @@ -3,6 +3,7 @@ use egui_kittest::Harness; use egui_kittest::kittest::Queryable; +use pdfcraft_engine::ocr::{OcrPage, OcrSettings, PlacedWord}; use pdfcraft_engine::{AddedText, Edit, Session, export}; use pdfcraft_ui_egui::PdfCraftApp; @@ -127,7 +128,14 @@ fn an_ocr_word_opens_for_editing_and_stays_open() { app.open_bytes("scan.pdf", None, scan()).unwrap(); let id = app.views[0].id; let ph = app.session.get(id).unwrap().info.pages[0].height as f64; - let words = vec![PlacedWord { text: "Recognize".into(), origin: [72.0, ph - 100.0], across: [80.0, 0.0], up: [0.0, 14.0] }]; + let words = vec![PlacedWord { + text: "Recognize".into(), + origin: [72.0, ph - 100.0], + across: [80.0, 0.0], + up: [0.0, 14.0], + confidence: None, + source: "ocrs".into(), + }]; app.session.apply(id, Edit::AddOcrText { page: 0, words }).unwrap(); app.set_option("zoom", "100").unwrap(); assert!(app.execute("edit.edit_text")); @@ -348,3 +356,112 @@ fn only_the_edit_tool_recognizes_a_scanned_page() { h.run_steps(6); assert_eq!(h.state().session.get(id).unwrap().can_undo(), None); } + +/// A placed word the review can hold (its box sits where a real one would). +fn placed(text: &str, confidence: f32) -> PlacedWord { + PlacedWord { text: text.into(), origin: [40.0, 40.0], across: [50.0, 0.0], up: [0.0, 12.0], confidence: Some(confidence), source: "ocrs".into() } +} + +/// The OCR Verify screen (Scan & OCR ▸ Correct recognized text) in the real shell: every +/// control is live, the keyboard walks the uncertain words, the inspector removes and restores +/// through clicks, and Accept applies the review as one undoable step. The recognition is +/// loaded directly, so this runs without the OCR models. +#[test] +fn ocr_verify_screen_controls_keyboard_and_accept() { + let mut h = Harness::builder().with_size(egui::vec2(1400.0, 900.0)).build_eframe(|_cc| { + let mut app = PdfCraftApp::new(); + app.open_bytes("scan.pdf", None, scan()).unwrap(); + app.run_inline = true; + app + }); + h.run_steps(4); + assert!(h.state_mut().execute("ocr.correct")); + h.run_steps(3); + // The screen owns the window and its controls are all present (no dead controls). + h.get_by_label("Run OCR"); + h.get_by_label("Reject"); + h.get_by_label("Next uncertain"); + h.get_by_label("Previous uncertain"); + h.get_by_label("Page verified"); + h.get_by_label("Exit"); + use pdfcraft_ui_egui::Phase; + assert_eq!(h.state().ocr_verify.phase, Phase::Idle); + // The guards hold in the real UI: Reject outside ReviewReady is ignored, not acted on. + h.get_by_label("Reject").click(); + h.run_steps(2); + assert_eq!(h.state().ocr_verify.phase, Phase::Idle); + // A recognition result, as the worker would deliver it. + { + let app = h.state_mut(); + let id = app.active_ids().unwrap().1; + let job = app.session.ocr_job(id, &[0], OcrSettings::default()).unwrap(); + let page = + OcrPage { already_on_page: 0, page: 0, words: vec![placed("Helo", 40.0), placed("wrold", 95.0)], skipped: None, notes: Vec::new() }; + app.ocr_verify.load(id, job.guard.clone(), vec![page]); + } + h.run_steps(3); + assert_eq!(h.state().ocr_verify.phase, Phase::ReviewReady); + // F8 walks to the Low-band word; the inspector shows it. + h.key_press(egui::Key::F8); + h.run_steps(2); + assert_eq!(h.state().ocr_verify.selected, Some((0, 0))); + h.get_by_label("Remove").click(); + h.run_steps(2); + assert!(h.state().ocr_verify.is_removed((0, 0)), "Remove strikes the word out"); + h.get_by_label("Restore").click(); + h.run_steps(2); + assert!(!h.state().ocr_verify.is_removed((0, 0)), "Restore brings it back"); + // Ctrl+T marks the page reviewed (the checkbox flips). + h.key_press_modifiers(egui::Modifiers::COMMAND, egui::Key::T); + h.run_steps(2); + assert!(h.state().ocr_verify.review.verified_pages.contains(&0), "Ctrl+T marks the page verified"); + h.key_press_modifiers(egui::Modifiers::COMMAND, egui::Key::T); + h.run_steps(2); + assert!(!h.state().ocr_verify.review.verified_pages.contains(&0)); + // Accept applies the whole review as ONE undoable step on the open document. + h.get_by_label("Accept").click(); + h.run_steps(3); + assert_eq!(h.state().ocr_verify.phase, Phase::Idle, "a successful accept ends the review"); + let app = h.state(); + let id = app.active_ids().unwrap().1; + assert_eq!(app.session.get(id).unwrap().can_undo(), Some("Recognize text")); + let toast = app.toast.clone().map(|t| t.0).unwrap_or_default(); + assert!(toast.contains("one undoable step"), "{toast}"); +} + +/// Ctrl+Tab moves the pane focus ring, and F8 with nothing to review is a no-op: the walk +/// never invents a selection. The recognition is loaded directly (no models needed). +#[test] +fn ocr_verify_screen_focus_ring_and_empty_walk() { + let mut h = Harness::builder().with_size(egui::vec2(1400.0, 900.0)).build_eframe(|_cc| { + let mut app = PdfCraftApp::new(); + app.open_bytes("scan.pdf", None, scan()).unwrap(); + app.run_inline = true; + app + }); + h.run_steps(4); + assert!(h.state_mut().execute("ocr.correct")); + h.run_steps(3); + { + let app = h.state_mut(); + let id = app.active_ids().unwrap().1; + let job = app.session.ocr_job(id, &[0], OcrSettings::default()).unwrap(); + let page = OcrPage { already_on_page: 0, page: 0, words: vec![placed("Fine", 95.0)], skipped: None, notes: Vec::new() }; + app.ocr_verify.load(id, job.guard.clone(), vec![page]); + } + h.run_steps(3); + // Nothing asks for review: F8 selects nothing instead of wrapping into an error. + h.key_press(egui::Key::F8); + h.run_steps(2); + assert_eq!(h.state().ocr_verify.selected, None); + // Ctrl+Tab cycles the four panes; four steps come back to the start. + let start = h.state().ocr_verify.focus; + let mut moved = false; + for _ in 0..4 { + h.key_press_modifiers(egui::Modifiers::COMMAND, egui::Key::Tab); + h.run_steps(2); + moved |= h.state().ocr_verify.focus != start; + } + assert!(moved, "Ctrl+Tab moves the focus"); + assert_eq!(h.state().ocr_verify.focus, start, "the ring returns after the four panes"); +} diff --git a/crates/ui-egui/tests/ocr.rs b/crates/ui-egui/tests/ocr.rs new file mode 100644 index 0000000000..a0af321a01 --- /dev/null +++ b/crates/ui-egui/tests/ocr.rs @@ -0,0 +1,281 @@ +//! Scan & OCR ▸ Recognize text in the real shell (egui_kittest): the dialog, the run and its +//! result (skipped without the OCR models: `cargo xtask models`). + +use egui_kittest::Harness; +use egui_kittest::kittest::Queryable; +use pdfcraft_engine::ocr::{OcrPage, OcrSettings, PlacedWord}; +use pdfcraft_engine::{AddedText, Edit, Session, export}; +use pdfcraft_ui_egui::PdfCraftApp; + +/// A one-page PDF that is only a picture of a sentence. +fn scan() -> Vec { + let mut s = Session::new(); + let text = s.create_from_text("t", "Recognize these scanned words").unwrap(); + let id = s.open("text.pdf", None, text, None).unwrap(); + let png = export::Exporter::new(s.get(id).unwrap()).png(0, 150.0).unwrap(); + s.create_from_images(&[("scan.png".into(), png)]).unwrap().to_vec() +} + +#[test] +fn recognize_text_dialog_adds_searchable_text() { + let mut h = Harness::builder().with_size(egui::vec2(1400.0, 900.0)).build_eframe(|_cc| { + let mut app = PdfCraftApp::new(); + app.set_option("language", "en").unwrap(); + app.open_bytes("scan.pdf", None, scan()).unwrap(); + app.run_inline = true; + app + }); + h.run_steps(4); + assert!(h.state_mut().execute("ocr.recognize")); + h.run_steps(2); + h.get_by_label("Document language"); + h.get_by_label("Output"); + if !pdfcraft_engine::ocr::available() { + eprintln!("skipped: OCR models not installed"); + return; + } + h.get_by_label("Recognize text").click(); + h.run_steps(3); + assert!(h.state().ocr_run.is_none(), "finished"); + let toast = h.state().toast.clone().unwrap().0; + assert!(toast.starts_with("Recognized ") && toast.contains("on 1 page"), "{toast}"); + let app = h.state(); + let id = app.active_ids().unwrap().1; + assert_eq!(app.session.get(id).unwrap().can_undo(), Some("Recognize text")); +} + +#[test] +fn recognize_text_in_multiple_files_writes_searchable_copies() { + if !pdfcraft_engine::ocr::available() { + eprintln!("skipped: OCR models not installed"); + return; + } + let dir = std::env::temp_dir().join(format!("pdfcraft-ocr-ui-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let mut app = PdfCraftApp::new(); + app.set_option("language", "en").unwrap(); + app.run_inline = true; + app.export_dir_override = Some(dir.to_string_lossy().into_owned()); + app.use_files(pdfcraft_ui_egui::FilePurpose::Ocr, vec![("one.pdf".into(), scan()), ("two.pdf".into(), scan())]); + assert!(app.ocr_batch.is_none(), "finished"); + let toast = app.toast.clone().unwrap().0; + assert!(toast.starts_with("Recognized ") && toast.ends_with("in 2 files"), "{toast}"); + for name in ["one.pdf", "two.pdf"] { + let bytes = std::fs::read(dir.join(name)).unwrap(); + assert!(bytes.len() > scan().len(), "an incremental update was appended"); + } +} + +/// A placed word the review can hold (its box sits where a real one would). +fn placed(text: &str, confidence: f32) -> PlacedWord { + PlacedWord { text: text.into(), origin: [40.0, 40.0], across: [50.0, 0.0], up: [0.0, 12.0], confidence: Some(confidence), source: "ocrs".into() } +} + +/// The OCR Verify screen (Scan & OCR ▸ Correct recognized text) in the real shell: every +/// control is live, the keyboard walks the uncertain words, the inspector removes and restores +/// through clicks, and Accept applies the review as one undoable step. The recognition is +/// loaded directly, so this runs without the OCR models. +#[test] +fn ocr_verify_screen_controls_keyboard_and_accept() { + let mut h = Harness::builder().with_size(egui::vec2(1400.0, 900.0)).build_eframe(|_cc| { + let mut app = PdfCraftApp::new(); + app.open_bytes("scan.pdf", None, scan()).unwrap(); + app.run_inline = true; + app + }); + h.run_steps(4); + assert!(h.state_mut().execute("ocr.correct")); + h.run_steps(3); + // The screen owns the window and its controls are all present (no dead controls). + h.get_by_label("Run OCR"); + h.get_by_label("Reject"); + h.get_by_label("Next uncertain"); + h.get_by_label("Previous uncertain"); + h.get_by_label("Page verified"); + h.get_by_label("Exit"); + use pdfcraft_ui_egui::Phase; + assert_eq!(h.state().ocr_verify.phase, Phase::Idle); + // The guards hold in the real UI: Reject outside ReviewReady is ignored, not acted on. + h.get_by_label("Reject").click(); + h.run_steps(2); + assert_eq!(h.state().ocr_verify.phase, Phase::Idle); + // A recognition result, as the worker would deliver it. + { + let app = h.state_mut(); + let id = app.active_ids().unwrap().1; + let job = app.session.ocr_job(id, &[0], OcrSettings::default()).unwrap(); + let page = + OcrPage { already_on_page: 0, page: 0, words: vec![placed("Helo", 40.0), placed("wrold", 95.0)], skipped: None, notes: Vec::new() }; + app.ocr_verify.load(id, job.guard.clone(), vec![page]); + } + h.run_steps(3); + assert_eq!(h.state().ocr_verify.phase, Phase::ReviewReady); + // F8 walks to the Low-band word; the inspector shows it. + h.key_press(egui::Key::F8); + h.run_steps(2); + assert_eq!(h.state().ocr_verify.selected, Some((0, 0))); + h.get_by_label("Remove").click(); + h.run_steps(2); + assert!(h.state().ocr_verify.is_removed((0, 0)), "Remove strikes the word out"); + h.get_by_label("Restore").click(); + h.run_steps(2); + assert!(!h.state().ocr_verify.is_removed((0, 0)), "Restore brings it back"); + // Ctrl+T marks the page reviewed (the checkbox flips). + h.key_press_modifiers(egui::Modifiers::COMMAND, egui::Key::T); + h.run_steps(2); + assert!(h.state().ocr_verify.review.verified_pages.contains(&0), "Ctrl+T marks the page verified"); + h.key_press_modifiers(egui::Modifiers::COMMAND, egui::Key::T); + h.run_steps(2); + assert!(!h.state().ocr_verify.review.verified_pages.contains(&0)); + // Accept applies the whole review as ONE undoable step on the open document. + h.get_by_label("Accept").click(); + h.run_steps(3); + assert_eq!(h.state().ocr_verify.phase, Phase::Idle, "a successful accept ends the review"); + let app = h.state(); + let id = app.active_ids().unwrap().1; + assert_eq!(app.session.get(id).unwrap().can_undo(), Some("Recognize text")); + let toast = app.toast.clone().map(|t| t.0).unwrap_or_default(); + assert!(toast.contains("one undoable step"), "{toast}"); +} + +/// Ctrl+Tab moves the pane focus ring, and F8 with nothing to review is a no-op: the walk +/// never invents a selection. The recognition is loaded directly (no models needed). +#[test] +fn ocr_verify_screen_focus_ring_and_empty_walk() { + let mut h = Harness::builder().with_size(egui::vec2(1400.0, 900.0)).build_eframe(|_cc| { + let mut app = PdfCraftApp::new(); + app.open_bytes("scan.pdf", None, scan()).unwrap(); + app.run_inline = true; + app + }); + h.run_steps(4); + assert!(h.state_mut().execute("ocr.correct")); + h.run_steps(3); + { + let app = h.state_mut(); + let id = app.active_ids().unwrap().1; + let job = app.session.ocr_job(id, &[0], OcrSettings::default()).unwrap(); + let page = OcrPage { already_on_page: 0, page: 0, words: vec![placed("Fine", 95.0)], skipped: None, notes: Vec::new() }; + app.ocr_verify.load(id, job.guard.clone(), vec![page]); + } + h.run_steps(3); + // Nothing asks for review: F8 selects nothing instead of wrapping into an error. + h.key_press(egui::Key::F8); + h.run_steps(2); + assert_eq!(h.state().ocr_verify.selected, None); + // Ctrl+Tab cycles the four panes; four steps come back to the start. + let start = h.state().ocr_verify.focus; + let mut moved = false; + for _ in 0..4 { + h.key_press_modifiers(egui::Modifiers::COMMAND, egui::Key::Tab); + h.run_steps(2); + moved |= h.state().ocr_verify.focus != start; + } + assert!(moved, "Ctrl+Tab moves the focus"); + assert_eq!(h.state().ocr_verify.focus, start, "the ring returns after the four panes"); +} + +/// #1028: "Also recognize pages that already have text" reads a picture on a page that has its own +/// text, without adding that text a second time. +#[test] +fn recognize_text_can_read_pictures_on_pages_with_text() { + let hybrid = { + let mut s = Session::new(); + let id = s.open("scan.pdf", None, std::sync::Arc::new(scan()), None).unwrap(); + let (w, h) = { + let p = &s.get(id).unwrap().info.pages[0]; + (p.width as f64, p.height as f64) + }; + let note = + AddedText { rect: [w * 0.1, h * 0.75, w * 0.9, h * 0.85], text: "Invoice reference 4711".into(), size: 18.0, ..Default::default() }; + s.apply(id, Edit::AddText { page: 0, text: note }).unwrap(); + s.save_bytes(id).unwrap().to_vec() + }; + let mut h = Harness::builder().with_size(egui::vec2(1400.0, 900.0)).build_eframe(|_cc| { + let mut app = PdfCraftApp::new(); + app.set_option("language", "en").unwrap(); + app.open_bytes("hybrid.pdf", None, hybrid).unwrap(); + app.run_inline = true; + app + }); + h.run_steps(4); + assert!(h.state_mut().execute("ocr.recognize")); + h.run_steps(2); + assert!(h.state().ocr_draft.settings().skip_text_pages, "pages with text are skipped by default"); + h.get_by_label("Also recognize pages that already have text").click(); + h.run_steps(2); + assert!(!h.state().ocr_draft.settings().skip_text_pages); + if !pdfcraft_engine::ocr::available() { + eprintln!("skipped: OCR models not installed"); + return; + } + h.get_by_label("Recognize text").click(); + h.run_steps(3); + assert!(h.state().ocr_run.is_none(), "finished"); + let toast = h.state().toast.clone().unwrap().0; + assert!(toast.starts_with("Recognized ") && toast.ends_with("on 1 page"), "{toast}"); + let app = h.state(); + let id = app.active_ids().unwrap().1; + let doc = app.session.get(id).unwrap(); + assert_eq!(doc.can_undo(), Some("Recognize text")); + let mut r = pdfcraft_render::PageRenderer::new(doc.bytes.clone(), Default::default()); + let text = r.render(pdfcraft_render::RenderRequest { page: 0, kind: pdfcraft_render::RequestKind::Text, scale: 1.0, ..Default::default() }); + let text = text.text.unwrap().plain_text().to_lowercase(); + assert_eq!((text.matches("invoice").count(), text.matches("scanned").count()), (1, 1), "{text}"); +} + +/// Edit text & images on a scanned page: an OCR word (invisible text over the scan) opens for +/// editing on a click, stays open when the pointer leaves it, and its editor hides the scanned +/// word instead of showing it through. +#[test] +fn an_ocr_word_opens_for_editing_and_stays_open() { + use pdfcraft_engine::ocr::PlacedWord; + let mut h = Harness::builder().with_size(egui::vec2(1400.0, 900.0)).build_eframe(|_cc| { + let mut app = PdfCraftApp::new(); + app.set_option("language", "en").unwrap(); + app.open_bytes("scan.pdf", None, scan()).unwrap(); + let id = app.views[0].id; + let ph = app.session.get(id).unwrap().info.pages[0].height as f64; + let words = vec![PlacedWord { + text: "Recognize".into(), + origin: [72.0, ph - 100.0], + across: [80.0, 0.0], + up: [0.0, 14.0], + confidence: None, + source: "ocrs".into(), + }]; + app.session.apply(id, Edit::AddOcrText { page: 0, words }).unwrap(); + app.set_option("zoom", "100").unwrap(); + assert!(app.execute("edit.edit_text")); + app + }); + for _ in 0..60 { + h.run_steps(2); + if !h.state().render_pending() { + break; + } + std::thread::sleep(std::time::Duration::from_millis(5)); + } + let (r, k) = { + let app = h.state(); + let d = app.session.get(app.views[0].id).unwrap(); + assert!(d.text_blocks(0).first().is_some_and(|b| b.invisible)); + let r = app.views[0].page_screen_rect(0).unwrap(); + (r, r.width() / d.info.pages[0].width) + }; + // The word, 100 pt from the top of the page. + let p = r.min + egui::vec2(100.0, 93.0) * k; + h.hover_at(p); + h.run_steps(2); + for pressed in [true, false] { + h.event(egui::Event::PointerButton { pos: p, button: egui::PointerButton::Primary, pressed, modifiers: Default::default() }); + } + h.run_steps(4); + let opened = |h: &Harness<'_, PdfCraftApp>| h.state().views[0].line_editor.as_ref().map(|e| e.text.clone()); + assert_eq!(opened(&h).as_deref(), Some("Recognize")); + assert!(h.state().views[0].image_selection.is_none(), "the scan under the word isn't selected"); + h.hover_at(r.min + egui::vec2(300.0, 400.0) * k); + h.run_steps(4); + assert_eq!(opened(&h).as_deref(), Some("Recognize"), "moving the pointer away keeps the word open"); +} diff --git a/docs/images/pdfcraft-ocr-verify.png b/docs/images/pdfcraft-ocr-verify.png new file mode 100644 index 0000000000..14a5b3dd5c Binary files /dev/null and b/docs/images/pdfcraft-ocr-verify.png differ diff --git a/docs/localization.md b/docs/localization.md index 2f9e98a676..96822877a3 100644 --- a/docs/localization.md +++ b/docs/localization.md @@ -20,7 +20,7 @@ Whole sentences that the layers below word — what the accessibility checker fo The Ukrainian catalog has 2,530 entries, written from the meaning of the English labels using Ukrainian computing vocabulary. Commands use infinitives; fixed-count messages use count-neutral wording. Plural-aware messages use the three [CLDR Ukrainian integer forms](https://www.unicode.org/cldr/charts/48/supplemental/language_plural_rules.html#uk): `1 сторінка`, `2 сторінки`, `5 сторінок`; 11–14 take the third form, while 21 and 22 return to the first and second. Calendar dates use genitive month names (`9 жовтня 2026`), while headings use standalone names (`Жовтень 2026`). Home recommendation cards grow to fit translated text; the toolbar search hint truncates before the shortcut, with the full label available on hover. -The Lithuanian catalog has 2,798 entries and three integer plural forms. Calendar headings use standalone month names; dates use genitive forms. Calendar, inspector and password-dialog UI regressions check dynamic labels and preserve document values. A native Lithuanian-speaking contributor has reviewed the translation. +The Lithuanian catalog has 2,798 entries and three integer plural forms. Calendar headings use standalone month names; dates use genitive forms. Calendar, inspector and password-dialog UI regressions check dynamic labels and preserve document values. A native Lithuanian-speaking contributor has reviewed the translation. The Scan & OCR review strings added afterwards (M10: the OCR Verify screen, the batch options and the engine choice) were written without native review. The Vietnamese catalog has 2,798 entries and one plural form (`plural_none`), covering the current catalog-key union. Preferences theme `Appearance` uses context `theme` → `Giao diện`; object/comment/signature appearance stays plain `Diện mạo`. Form fields use `Trường`; redaction apply wording states permanent removal. Calendar months are numeric (`Tháng 1`…`12` / `tháng 1`…`12`). A native Vietnamese-speaking contributor has reviewed the translation. Status stays `partial`: no translated help, and engine/OS details stay English inside a translated frame. diff --git a/parity/acrobat-features.toml b/parity/acrobat-features.toml index f71546b731..ec144d3a07 100644 --- a/parity/acrobat-features.toml +++ b/parity/acrobat-features.toml @@ -6683,7 +6683,7 @@ milestone = "M10" status = "shipped" commands = ["ocr.recognize"] tools = ["ocr_recognize", "ocr_status"] -evidence = ["crates/ocr/src/lib.rs::model_discovery_prefers_the_complete_onnx_pair", "crates/ocr/src/lib.rs::model_discovery_falls_back_to_legacy_when_the_onnx_pair_is_incomplete", "crates/ocr/src/lib.rs::model_discovery_refuses_mixed_format_pairs", "crates/ocr/src/lib.rs::model_discovery_keeps_legacy_only_installations", "crates/engine/src/tests.rs::recognize_text_makes_a_scanned_page_searchable", "crates/automation/tests/it/automation.rs::ocr_tools_make_a_scan_searchable", "crates/ui-egui/tests/it/ocr.rs::recognize_text_dialog_adds_searchable_text"] +evidence = ["crates/ocr/src/ocrs.rs::model_discovery_prefers_the_complete_onnx_pair", "crates/ocr/src/ocrs.rs::model_discovery_falls_back_to_legacy_when_the_onnx_pair_is_incomplete", "crates/ocr/src/ocrs.rs::model_discovery_refuses_mixed_format_pairs", "crates/ocr/src/ocrs.rs::model_discovery_keeps_legacy_only_installations", "crates/engine/src/tests.rs::recognize_text_makes_a_scanned_page_searchable", "crates/automation/tests/it/automation.rs::ocr_tools_make_a_scan_searchable", "crates/ui-egui/tests/it/ocr.rs::recognize_text_dialog_adds_searchable_text"] notes = "ocrs engine (MIT/Apache) with CC-BY-SA ONNX models fetched and SHA-256 checked by `cargo xtask models`; within each search directory, a complete ONNX pair takes precedence over a complete legacy RTEN pair, incomplete and mixed-format pairs are not combined, and directory precedence is unchanged; the desktop release packages ship them (#103; packaging/env.sh stage_models), the web build does not yet; runs on a worker thread with progress; one undoable step; pages with text are skipped unless asked (ocr.skip-renderable-text)." [[feature]] @@ -6708,7 +6708,7 @@ status = "shipped" commands = ["ocr.recognize_batch"] tools = ["ocr_recognize_files"] evidence = ["crates/automation/tests/it/automation.rs::ocr_recognize_files_writes_searchable_copies", "crates/ui-egui/tests/it/ocr.rs::recognize_text_in_multiple_files_writes_searchable_copies"] -notes = "Pick PDFs, then an output folder; copies are saved incrementally under the same names, in the background with progress." +notes = "Pick PDFs, then an output folder (the last folder is remembered and reused; Clear asks again); copies are saved incrementally under the same names, in the background with progress. Output names are pre-flighted (a collision aborts before the first page is read), files that already have text are skipped whole only when that option is on (skip_text_files, off by default; Force OCR overrides), and the summary counts recognized words, the skipped bucket, per-file failures, low-confidence notes and what a cancel left unrun." [[feature]] id = "ocr.ocr-languages" @@ -6719,8 +6719,8 @@ milestone = "M10" status = "partial" commands = ["ocr.recognize"] tools = ["ocr_recognize"] -evidence = ["crates/ui-egui/tests/it/ocr.rs::recognize_text_dialog_adds_searchable_text"] -notes = "Language choice exists, but the models read only the Latin alphabet without accents (English)." +evidence = ["crates/ui-egui/tests/it/ocr.rs::recognize_text_dialog_adds_searchable_text", "crates/ocr/src/tesseract.rs::language_table_maps_and_falls_back"] +notes = "The choice feeds both engines: ocrs reads only unaccented Latin (its model table holds English); tesseract maps 12 allow-listed codes (the language enum comes from the engine's own table) but only as far as the installed language packs cover. Recognition quality beyond the tested English set is unproven." [[feature]] id = "ocr.searchable-image" @@ -6731,8 +6731,8 @@ milestone = "M10" status = "partial" commands = ["ocr.recognize"] tools = ["ocr_recognize"] -evidence = ["crates/engine/src/tests.rs::recognize_text_makes_a_scanned_page_searchable"] -notes = "Invisible text (Tr 3) fitted to each word box; the image is not deskewed or downsampled (exact mode only)." +evidence = ["crates/engine/src/tests.rs::recognize_text_makes_a_scanned_page_searchable", "crates/engine/src/ocr.rs::searchable_layer_round_trips_non_win_ansi_words", "crates/engine/src/ocr.rs::preprocessing_flags_flow_through_the_job"] +notes = "Invisible text (Tr 3) fitted to each word box. The page's own image object is left untouched; the raster the engines read can be cleaned first (opt-in rotate/deskew/denoise/binarize), with word boxes mapped back to the original raster. Words beyond WinAnsi are written in Type3 fonts with empty character programs and a ToUnicode CMap, so extraction reads them back while nothing needs a display font (no craft-fonts input, and no character is ever substituted with '?'; those substitutions belong to the editable output). Missing: the downsampled Searchable Image variant (ocr.ocr-downsample)." [[feature]] id = "ocr.searchable-image-exact" @@ -6753,9 +6753,9 @@ area = "I" tier = "P1" milestone = "M10" status = "partial" -commands = ["edit.edit_text"] -tools = ["text_edit"] -evidence = ["crates/engine/tests/it/ocr_edit.rs::editing_an_ocr_word_covers_the_scan_and_shows_the_new_text", "crates/engine/tests/it/ocr_edit.rs::an_explicit_colour_is_kept_for_the_new_text", "crates/engine/tests/it/ocr_edit.rs::ocr_words_group_into_phrases_that_edit_in_place", "crates/engine/tests/it/ocr_edit.rs::a_longer_ocr_phrase_grows_rightwards", "crates/engine/tests/it/ocr_edit.rs::a_rewritten_ocr_phrase_keeps_its_word_widths", "crates/engine/tests/it/ocr_edit.rs::an_ocr_layer_under_the_scan_image_shows_its_edit", "crates/engine/tests/it/ocr_edit.rs::an_ocr_phrase_split_from_visible_text_under_the_scan_shows_its_edit", "crates/ui-egui/tests/it/ocr.rs::an_ocr_word_opens_for_editing_and_stays_open", "crates/ui-egui/tests/it/ocr.rs::editing_a_scanned_page_recognizes_its_text_first_and_only_once"] +commands = ["edit.edit_text", "ocr.correct"] +tools = ["text_edit", "ocr_recognize"] +evidence = ["crates/engine/tests/it/ocr_edit.rs::editing_an_ocr_word_covers_the_scan_and_shows_the_new_text", "crates/engine/tests/it/ocr_edit.rs::an_explicit_colour_is_kept_for_the_new_text", "crates/engine/tests/it/ocr_edit.rs::ocr_words_group_into_phrases_that_edit_in_place", "crates/engine/tests/it/ocr_edit.rs::a_longer_ocr_phrase_grows_rightwards", "crates/engine/tests/it/ocr_edit.rs::a_rewritten_ocr_phrase_keeps_its_word_widths", "crates/engine/tests/it/ocr_edit.rs::an_ocr_layer_under_the_scan_image_shows_its_edit", "crates/engine/tests/it/ocr_edit.rs::an_ocr_phrase_split_from_visible_text_under_the_scan_shows_its_edit", "crates/ui-egui/tests/it/ocr.rs::an_ocr_word_opens_for_editing_and_stays_open", "crates/ui-egui/tests/it/ocr.rs::editing_a_scanned_page_recognizes_its_text_first_and_only_once", "crates/engine/src/ocr.rs::editable_writer_refuses_empty_and_writes_visible_words", "crates/engine/src/ocr.rs::editable_output_round_trips_non_win_ansi_words", "crates/engine/src/ocr.rs::editable_writer_maps_rotated_pages", "crates/automation/tests/it/automation.rs::ocr_recognize_editable_output_writes_visible_text"] notes = "On demand, per edit: in Edit text & images, recognised text (the invisible OCR layer over a scan) is grouped into phrases (words across a word space; a wider gap such as a table cell, and each scanned line, stay separate) and opens for editing on a click. Applying draws the new text visibly in the scan's sampled ink colour (or the chosen one) and first covers the old words with the paper colour sampled around them, so the scanned word no longer shows; when an image or form is drawn after the OCR layer over where the edit lands (its old box, moved, rewrapped or grown to more lines), or an inline image or shading after it (a layer written under its scan, #3973), the cover and new text are drawn on top of the page instead, so they show; the phrase keeps each word's position and width and grows rightwards instead of wrapping. Turning Edit text & images on for a scanned page (an image covering at least half of a page that has no text of its own, once per page per window; comments and text boxes over it do not count as text, a page with real text and a background picture is left alone; one undoable \"Recognize text\" step) now recognizes its words first, as Acrobat's \"Converting scanned page to Editable Text and Images\" does (English only until more OCR languages exist, gap 7). Still missing: separating the photographs on the page from the lettering, matching the scanned typeface (the layer's Helvetica is used), covering on a textured or patterned background (one flat colour is used), a scan painted by a pattern fill or a filled rectangle over a layer written under it, and, for text drawn on top, the clip, optional content and marked content (tags) that enclosed the OCR layer." [[feature]] @@ -6779,8 +6779,8 @@ milestone = "M10" status = "shipped" commands = ["ocr.recognize"] tools = ["ocr_recognize"] -evidence = ["crates/engine/src/tests.rs::recognize_text_makes_a_scanned_page_searchable", "crates/automation/tests/it/automation.rs::ocr_tools_make_a_scan_searchable", "crates/engine/src/tests.rs::recognize_text_reads_a_picture_on_a_page_that_has_text", "crates/automation/tests/it/automation.rs::ocr_recognize_reads_pictures_on_pages_with_text", "crates/ui-egui/tests/it/ocr.rs::recognize_text_can_read_pictures_on_pages_with_text"] -notes = "Pages that already have text are skipped and reported by default. The dialog's \"Also recognize pages that already have text\" (skip_text_pages false, also for multiple files) reads pictures on such pages (#1028); a recognised word at least half covered by the page's own text (an earlier OCR layer included) is left out and counted (already_on_page). Missing: recognising only the image areas (the whole page is still read, so it takes as long as a scan), and a word in a picture that sits under real text is treated as that text." +evidence = ["crates/engine/src/tests.rs::recognize_text_makes_a_scanned_page_searchable", "crates/automation/tests/it/automation.rs::ocr_tools_make_a_scan_searchable", "crates/engine/src/tests.rs::recognize_text_reads_a_picture_on_a_page_that_has_text", "crates/automation/tests/it/automation.rs::ocr_recognize_reads_pictures_on_pages_with_text", "crates/ui-egui/tests/it/ocr.rs::recognize_text_can_read_pictures_on_pages_with_text", "crates/automation/tests/it/automation.rs::ocr_recognize_files_writes_searchable_copies"] +notes = "Text pages are skipped by default; skip_text_pages false or Force OCR reads pictures on them while omitting words overlapping existing text (already_on_page). Batch skip_text_files reports skipped files without writing them; Force OCR overrides both skips. Recognition still reads the whole page, and image text under real text can be omitted." [[feature]] id = "ocr.rotation-detection" @@ -6788,11 +6788,11 @@ title = "Detect and fix page rotation" area = "I" tier = "P1" milestone = "M10" -status = "planned" -commands = [] -tools = [] -evidence = [] -notes = "" +status = "partial" +commands = ["ocr.recognize"] +tools = ["ocr_recognize"] +evidence = ["crates/ocr/src/preprocess.rs::quarter_turns_are_detected_corrected_and_mapped_back", "crates/ocr/src/preprocess.rs::realistic_letter_pages_are_left_upright"] +notes = "Opt-in preprocessing: 0/90/180/270 detected from projection-profile banding (conservative: under 32 px, blank, or without a clear margin it is a no-op); word boxes map back to the scanned raster through the exact inverse. Correction is proven at the preprocessing layer and the flag flows through the job (needs the models), but correction of a genuinely rotated page through the workflow is not yet covered by a cited end-to-end test." [[feature]] id = "ocr.deskew" @@ -6800,11 +6800,11 @@ title = "Deskew" area = "I" tier = "P1" milestone = "M10" -status = "planned" -commands = [] -tools = [] -evidence = [] -notes = "" +status = "partial" +commands = ["ocr.recognize"] +tools = ["ocr_recognize"] +evidence = ["crates/ocr/src/preprocess.rs::skew_is_estimated_and_straight_pages_are_left_alone", "crates/ocr/src/preprocess.rs::deskew_inverse_maps_off_centre_points_back_onto_the_source_line"] +notes = "Opt-in: skew estimated over +/-5 degrees by a shear-scored projection profile; conservative no-ops (under 64 px, blank, below 0.1 degrees, no clear winner); boxes map back off-centre-exact through the composed inverse. Correction is proven at the preprocessing layer; a cited end-to-end test of a skewed page through the workflow is still owed." [[feature]] id = "ocr.despeckle" @@ -6813,9 +6813,9 @@ area = "I" tier = "P1" milestone = "M10" status = "partial" -commands = ["ocr.enhance_camera", "ocr.enhance"] -tools = ["page_enhance_camera", "page_enhance_scan"] -evidence = ["crates/engine/src/camera_clean.rs::a_shadow_and_a_yellow_cast_become_white_paper", "crates/engine/src/tests.rs::enhance_camera_page_at_a_level_whitens_a_shadowed_sheet", "crates/ui-egui/tests/it/camera.rs::the_clean_up_level_enhances_again_from_the_photo_and_keeps_the_turn", "crates/engine/src/scan_clean/tests.rs::despeckle_is_its_own_switch", "crates/automation/tests/it/automation.rs::enhance_scanned_file_cleans_the_scan_in_place_through_tools", "crates/ui-egui/tests/it/scan_clean.rs::enhance_scanned_file_cleans_the_chosen_pages_and_says_what_it_did"] +commands = ["ocr.enhance_camera", "ocr.enhance", "ocr.recognize"] +tools = ["page_enhance_camera", "page_enhance_scan", "ocr_recognize"] +evidence = ["crates/engine/src/camera_clean.rs::a_shadow_and_a_yellow_cast_become_white_paper", "crates/engine/src/tests.rs::enhance_camera_page_at_a_level_whitens_a_shadowed_sheet", "crates/ui-egui/tests/it/camera.rs::the_clean_up_level_enhances_again_from_the_photo_and_keeps_the_turn", "crates/engine/src/scan_clean/tests.rs::despeckle_is_its_own_switch", "crates/automation/tests/it/automation.rs::enhance_scanned_file_cleans_the_scan_in_place_through_tools", "crates/ui-egui/tests/it/scan_clean.rs::enhance_scanned_file_cleans_the_chosen_pages_and_says_what_it_did", "crates/ocr/src/preprocess.rs::denoise_removes_specks_and_keeps_dimensions"] notes = "Camera images, and scanned pages through Enhance scanned file (ocr.enhance, page_enhance_scan despeckle): isolated dark pixels (at most one dark neighbour) turn white, on grey, RGB and 1-bit scans, in place (the OCR text layer, comments and links stay). Missing: as an option of Recognize text; specks larger than two pixels; CMYK, indexed, JPEG 2000, JBIG2 and CCITT scans and scans inside form XObjects (reported, not changed)." [[feature]] @@ -6896,11 +6896,11 @@ title = "Correct recognized text (low-confidence suspects)" area = "I" tier = "P2" milestone = "M10" -status = "planned" -commands = [] -tools = [] -evidence = [] -notes = "" +status = "partial" +commands = ["ocr.correct"] +tools = ["ocr_words", "ocr_review_apply", "ocr_dictionary"] +evidence = ["crates/ui-egui/src/workflows/ocr_verify.rs::dictionary_uses_recognition_language_instead_of_interface_locale", "crates/ui-egui/src/workflows/ocr_verify.rs::dictionary_limit_and_normalization_failures_preserve_file_and_review", "crates/ui-egui/src/workflows/ocr_verify.rs::dictionary_undo_persists_and_failed_undo_keeps_history", "crates/ui-egui/src/workflows/ocr_verify.rs::rereading_keeps_captured_language_when_the_draft_changes", "crates/automation/tests/it/automation.rs::ocr_review_corrections_keep_boxes_are_guarded_and_apply_once", "crates/automation/tests/it/automation.rs::ocr_dictionary_updates_are_atomic_bounded_and_confined", "crates/automation/tests/it/automation.rs::ocr_retained_reviews_have_eight_document_cap_and_close_releases_slot", "crates/ui-egui/src/workflows/ocr_verify.rs::accept_zero_words_fails_and_a_live_review_applies_once", "crates/ui-egui/src/workflows/ocr_verify.rs::legend_rows_carry_the_classifier_thresholds", "crates/ui-egui/src/workflows/ocr_verify.rs::suggestions_are_near_dictionary_and_page_words", "crates/ui-egui/tests/it/ocr.rs::ocr_verify_screen_controls_keyboard_and_accept", "crates/ui-egui/src/workflows/ocr_verify.rs::accepting_pages_one_by_one_then_the_review_applies_every_page_once", "crates/ui-egui/src/workflows/ocr_verify.rs::rereading_a_region_remaps_marks_and_selects_the_first_new_word"] +notes = "The OCR Verify screen: page list, source page with confidence-coloured word boxes (click-select, drag a region), recognised-text preview, zoom + word inspector; corrections commit on Enter (an empty commit removes), Remove/Restore, +Dictionary (a per-language user file), per-word Verified, suggestions within edit distance 2; F8/Shift+F8 walk the low-confidence suspects, Ctrl+T marks the page reviewed, Ctrl+Tab moves the pane focus; Re-OCR of a region or a page; Accept/Reject entire page; a real Cancel keeps the pages already read; a stale session is refused with its reason; review edits are undoable in the panel; Accept (searchable) is one undoable step through the guard path, and pages accepted one by one each apply once (the guard follows each accepted page); a re-read keeps every review mark on the word it was made for. Headless: ocr_words (retain_review) returns the words and a review handle, ocr_review_apply applies corrections at the stored boxes (an empty one removes the word) through the same stale-document guard, or discards the review, and ocr_dictionary edits the dictionary file. Still missing: suggestions, per-word Verified marks and region re-reads are UI-only, and Acrobat's suspect walk across a whole saved document (outside one review) is not offered." [[feature]] id = "ocr.adaptive-compression" @@ -6932,11 +6932,11 @@ title = "Pluggable OCR engines (ocrs default; tesseract optional)" area = "I" tier = "P1" milestone = "M10" -status = "planned" -commands = [] -tools = [] -evidence = [] -notes = "" +status = "partial" +commands = ["ocr.recognize"] +tools = ["ocr_recognize"] +evidence = ["crates/engine/src/ocr.rs::selector_names_the_missing_engine", "crates/ocr/src/tesseract.rs::recognize_runs_the_installed_binary", "crates/ocr/src/merge.rs::rover_vote_always_runs_both_and_degrades_without_a_secondary", "crates/ocr/src/merge.rs::confidence_weighted_runs_the_secondary_on_an_unknown_primary", "crates/ocr/src/merge.rs::an_unknown_primary_word_yields_only_to_a_confident_secondary", "crates/ocr/src/tesseract.rs::an_invalid_configured_program_fails_closed", "crates/ocr/src/tesseract.rs::list_langs_come_from_stderr_when_stdout_has_none", "crates/engine/src/ocr.rs::automatic_defaults_to_the_confidence_weighted_strategy"] +notes = "ocrs (pure Rust, MIT/Apache, CC-BY-SA models fetched by `cargo xtask models`) is the default; an installed tesseract binary runs as an external process (fixed argv, allow-listed language codes, private temp dir, timeout and output cap, a user-validated PDFCRAFT_TESSERACT path or the PATH); Automatic is an ensemble (ROVER and confidence-weighted merge; a failing partner degrades to the primary with a note). Partial, because part of the evidence is conditional: `recognize_runs_the_installed_binary` runs only where tesseract is installed and skips elsewhere, so a build machine without it proves nothing about the process path, the absent-binary paths are exercised only on machines without tesseract, and the process-level tests (timeout, output cap, minimal environment) are Unix-only (fake scripts). What always runs: location validation and the fail-closed rule, the --list-langs parsing (stdout or stderr), the engine selector, the merge rules and the default strategy. Automatic defaults to the confidence-weighted strategy (Primary only would never run the second engine)." [[feature]] id = "ocr.ocr-web" diff --git a/xtask/src/license_check.rs b/xtask/src/license_check.rs index c845101db6..0231b4e853 100644 --- a/xtask/src/license_check.rs +++ b/xtask/src/license_check.rs @@ -54,7 +54,7 @@ fn validate(root: &Path) -> Result<()> { let docs: Documents = toml::from_str(&text).context("parsing LICENSES.toml")?; let attribution = assets::load(root)?; let mut problems = check(root, &docs, &attribution); - let ocr_source = std::fs::read_to_string(root.join("crates/ocr/src/lib.rs")).context("reading OCR model filename constants")?; + let ocr_source = std::fs::read_to_string(root.join("crates/ocr/src/ocrs.rs")).context("reading OCR model filename constants")?; problems.extend(check_required_ocr_metadata(&ocr_source, &attribution)?); if !problems.is_empty() { bail!("source licensing errors:\n{}", problems.join("\n")); @@ -72,7 +72,7 @@ fn check_required_ocr_metadata(source: &str, attribution: &assets::Manifest) -> let file = declarations.first().and_then(|value| value.strip_suffix("\";")); let Some(file) = file.filter(|file| declarations.len() == 1 && safe_path(file) && !file.contains('/')) else { bail!( - "OCR constant {name}: expected one literal filename declaration in crates/ocr/src/lib.rs; update the source validator if its syntax changed" + "OCR constant {name}: expected one literal filename declaration in crates/ocr/src/ocrs.rs; update the source validator if its syntax changed" ); }; let entries: Vec<_> = attribution.fetched.iter().filter(|f| f.file == file).collect(); @@ -407,7 +407,7 @@ mod tests { #[test] fn required_ocr_models_cannot_be_missing() { - let source = include_str!("../../crates/ocr/src/lib.rs"); + let source = include_str!("../../crates/ocr/src/ocrs.rs"); let original = assets::load(&assets::root()).unwrap(); assert_eq!(original.fetched.iter().filter(|f| f.kind == "model").count(), 2); assert!(check_required_ocr_metadata(source, &original).unwrap().is_empty()); @@ -420,7 +420,7 @@ mod tests { #[test] fn required_ocr_models_cannot_be_misclassified() { - let source = include_str!("../../crates/ocr/src/lib.rs"); + let source = include_str!("../../crates/ocr/src/ocrs.rs"); let original = assets::load(&assets::root()).unwrap(); assert_eq!(original.fetched.iter().filter(|f| f.kind == "model").count(), 2); for model in original.fetched.iter().filter(|f| f.kind == "model") { diff --git a/xtask/src/screenshots.rs b/xtask/src/screenshots.rs index 243d8b0bad..521d793f48 100644 --- a/xtask/src/screenshots.rs +++ b/xtask/src/screenshots.rs @@ -39,6 +39,7 @@ pub const SCENES: &[(&str, &str, &str, &[&str])] = &[ "README: layouts", &["--layout", "two-up", "--mode", "read", "--page", "4", "--theme", "dark", "--notice", "off"], ), + ("pdfcraft-ocr-verify", "OCR Verify screen", "README: OCR review", &["--size", "1680x1000", "--ocr-verify", "on"]), ]; const PDF: &str = "dist/demo/pdfcraft-showcase.pdf"; @@ -109,7 +110,10 @@ fn update_attribution(root: &Path) -> Result<()> { // The home screen shows the ArtCraft mark: say so (it is a Storyteller trademark, not open // source; AGENTS.md §1.2), so the screenshot's licence is not read as covering it. let shows_home = scene.is_some_and(|(.., args)| args.windows(2).any(|w| w[0] == "--home" && w[1] == "on")); - let contents = if shows_home { + let shows_ocr = scene.is_some_and(|(.., args)| args.windows(2).any(|w| w[0] == "--ocr-verify" && w[1] == "on")); + let contents = if shows_ocr { + "Shows only PdfCraft UI, its Lucide icons and a synthetic scanned page built from the embedded fonts, with a real recognition (the OCR models) loaded into the Verify screen" + } else if shows_home { "Shows PdfCraft UI, Lucide icons, the showcase PDF built from the [[fetched]] OFL fonts, and the ArtCraft mark (Storyteller trademark from docs/brand/, not covered by this licence)" } else { "Shows only PdfCraft UI, Lucide icons and the showcase PDF built from the [[fetched]] OFL fonts"