diff --git a/AGENTS.md b/AGENTS.md index 1f90ec3d8f..98758c5dbd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -72,6 +72,7 @@ Anything else is forbidden, including assets whose licence you cannot prove from - Font assets shared by the Crafting Apps, including every Japanese font, live in [`storytold/craft-fonts`](https://github.com/storytold/craft-fonts), not here. **Never commit a font file to this repository**; adding a font means adding it to craft-fonts. (The Latin UI fonts already in `assets/fonts/` — Inter, JetBrains Mono, Dancing Script — stay.) Team members: the shared rules are in [craftrules `standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md) (internal). - craft-fonts is an **optional build input**, never a Cargo dependency: `CRAFT_FONTS_DIR= cargo build` makes `crates/fonts/build.rs` embed its fonts as `pdfcraft_fonts::CRAFT_FONTS`. Without it PdfCraft builds, tests and runs, but has no Japanese face (Japanese UI text shows replacement boxes, and writing Japanese into a PDF returns a clear error). Release builds always set it (`.github/workflows/release.yml`), and `ATTRIBUTION.toml` lists it as a `[[build_input]]`. - The desktop app may read one font **already installed on the user's machine** at runtime, as the last interface fallback for characters no embedded face has (`crates/ui-egui/src/system_fonts.rs`). That font is never embedded, shipped, committed or copied, and never appears in published screenshots (`PDFCRAFT_SYSTEM_FONTS=0`, which `cargo xtask screenshots` sets). It is not a way around §1.2: nothing from it may enter a PDF, a package or the repository. +- The page viewer may also draw a CID font of an Adobe CJK collection (Japan1, GB1, CNS1, Korea1) that a PDF does **not embed** with one face already installed on the machine (`crates/render/src/system_font.rs`), **on screen only**. `RenderConfig::system_fonts` is off by default and is enabled only for on-screen viewer and preview rasters (`RenderConfig::on_screen`, used by the engine viewer, Combine thumbnails and accessibility figure previews), so exports, printing, OCR and the automation tools never use it. Like the interface font it is read at runtime and never embedded, shipped, committed or copied; Noto CJK and Source Han are never candidates (§1.1), nor is any Adobe family; and `PDFCRAFT_SYSTEM_FONTS=0` turns it off. It is not a way around §1.2 either: nothing drawn with it may be saved or printed. ### 1.5 Procedure for adding or changing an asset diff --git a/ROADMAP.md b/ROADMAP.md index 069612ba31..76e86471b6 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,5 +1,14 @@ # PdfCraft roadmap +> **Last reviewed:** 2026-10-31 | **Last updated:** 2026-10-31 | **Change:** patch (M2, PR #4172: GBK rasters compared with explicit Unicode and independently selected installed faces; wrong-glyph and wrong-family negative controls fail; estimates unchanged) +> **Last reviewed:** 2026-10-31 | **Last updated:** 2026-10-31 | **Change:** patch (M2, PR #4172: integrate main fa5f9f8c; retain CJK viewer and upstream raster regressions, restore explicit display_config imports; estimates unchanged) +> **Last reviewed:** 2026-10-28 · **Last updated:** 2026-10-28 · **Change:** patch (M2, PR #4172: integrate main 5af10276; retain both roadmap histories and viewer-only CJK font policy; estimates unchanged) +> **Last reviewed:** 2026-10-27 · **Last updated:** 2026-10-27 · **Change:** patch (M2, PR #4172: integrate main 1dd2737b; preserve viewer-only CJK fallback and regression coverage across engine, renderer and Combine module splits; estimates unchanged) +> **Last reviewed:** 2026-10-27 · **Last updated:** 2026-10-27 · **Change:** patch (M2, PR #4172: integrate main 9a9813bf and preserve viewer-only CJK fallback through the engine module split; estimates unchanged) +> **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: document installed-font resolution helpers and regression contracts; behavior and estimates unchanged) +> **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: print preview excludes installed CJK fonts and isolates its font cache from the viewer; estimates unchanged) +> **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: reconcile current main while retaining display-only CJK fallback and upstream Japanese font selection; estimates unchanged) +> **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M2, PR #4172: extend on-screen CJK fallback to Combine thumbnails and figure previews; recognise FangSong and subset-prefixed Founder names; estimates unchanged) > **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (typed small Edit font sizes and Fill & Sign text/date colour, #2444/#2496; estimates unchanged) **Stage: alpha** · next: beta, ~25% (ready for real work 50% → 75%) and ~400–700 h away @@ -204,9 +213,20 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md ## Progress log +- **2026-10-31 (M2, PR #4172 review):** Added same-machine pixel oracles for GBK Song, Hei, Kai and FangSong, including all four subset-prefixed Founder aliases. Reference PDFs use explicit Unicode mappings and independently selected installed faces; font data and rasters remain in memory. All eight cases ran; wrong-glyph and wrong-family production mutations each failed the new assertion and were restored. Validation: 422 renderer unit tests, renderer all-target Clippy with -D warnings, formatting, layers/assets/parity and renderer WASM checks passed; existing optional-patch/WASM warnings and parity notes remain. Estimates unchanged. + +- **2026-10-31 (M2, PR #4172):** Integrated main fa5f9f8c, retaining both roadmap histories and raster regression groups. Restored display_config and thumbnail_config imports after the upstream explicit-import migration. Validation: 422 renderer unit tests, engine/UI all-target checks, scoped all-target Clippy with -D warnings, formatting, layers, assets, parity and renderer/engine WASM checks passed. Existing optional-patch/WASM warnings and parity notes remain; no full engine/UI runtime test rerun. Display-only font policy and estimates unchanged. + +- **2026-10-28 (M2, PR #4172):** Integrated main 5af10276, preserving both roadmap histories and reviewing auto-merged export tests and accessibility UI changes. Validation: 488 scoped tests passed (18 engine, 412 renderer unit, 43 renderer integration, 15 Combine; one existing renderer integration test ignored); scoped all-target Clippy with -D warnings, formatting, layers, assets, parity and renderer/engine WASM checks passed. Viewer-only CJK fallback and print/output font policy remain unchanged; estimates unchanged. + +- **2026-10-27 (M2, PR #4172 module-split conflicts):** Integrated main 1dd2737b, retaining viewer-only CJK fallback in split session modules and Combine thumbnail workers, and moving renderer regressions to the new test module. Validation: 798 scoped tests passed (371 engine, 412 renderer, 15 Combine; two existing engine tests ignored); scoped all-target Clippy with -D warnings, formatting, layers, assets, parity and renderer/engine WASM checks passed. Estimates unchanged. + +- **2026-10-27 (M10, OCR models):** Model staging downloads and verifies the ONNX pair after the upstream RTEN URL brownout; runtime discovery prefers complete ONNX pairs per directory, retains complete RTEN fallback and preserves directory precedence. + +- **2026-10-26 (M2, PR #4172):** Reconciled current main, preserving its Japanese serif/weight selection and both branches' progress records. Installed CJK viewer and preview fallback remains display-only; estimates unchanged. + - **2026-10-29 (M7, #2517):** Widening a paragraph rejoins a wrap with the number of spaces that wrap consumed, including a typed run and none for a Chinese or Japanese word split. The count survives an incremental save and a full rewrite, and a width-only `text_edit` uses it. Estimates unchanged. - **2026-10-26 (M11, JBIG2 symbols):** Lossless JBIG2 now stores each distinct mark of a black-and-white image once, as a symbol, and places its exact copies, when that is smaller than the generic region: on six pages of rendered 300 dpi text 28% of the generic region's size and 17% of Group 4's, and the same bits back through two independent decoders (hayro-jbig2 and Poppler). Scans, whose letters are seldom identical, gain little until refinement coding; a dictionary shared between pages (`/JBIG2Globals`) is next. Estimates unchanged. -- **2026-10-27 (M10, OCR models):** Model staging downloads and verifies the ONNX pair after the upstream RTEN URL brownout; runtime discovery prefers complete ONNX pairs per directory, retains complete RTEN fallback and preserves directory precedence. - **2026-10-25 (M5, comment replies):** One reply can be deleted from the Comments panel without its thread. On the selected card each reply has its own "…" menu (Edit, Copy text, Delete); Delete takes that reply with its pop-up and the replies to it, as `comment_delete` already did headlessly, and the comment and its other replies stay. The card stays selected, and the panel's editor stays open, on what they were on though the delete moved it in `/Annots`. Undo and Redo close the editor, since either can move what it was open on. A locked reply can't be deleted. Estimates unchanged. - **2026-10-24 (M10, #4158 partial):** Word, HTML, RTF and PowerPoint export keep a block that continues another on its line (an italic case name, a link, a superscript) as a run in its own style inside that paragraph, with the source's word space, instead of a paragraph of its own. Style changes that wrap onto the next line still split paragraphs (the reading blocks keep style boundaries). Estimates unchanged. - **2026-10-24 (M3, window move on Wayland, #3970 follow-up):** After a move or resize by PdfCraft's own title bar or edges, the button release given back to egui goes before the pointer leaving the window, so hover no longer sticks at the old place until the pointer moves. Found by CodeRabbit in review of #3970; a unit test covers it. Estimates unchanged. @@ -238,6 +258,8 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-10-24 (M5, move preview):** A comment being moved is drawn where it goes while it is dragged (a rectangle or oval showed only its blue frame): the comment is rendered alone once as the drag starts and laid over the page, which shows it where it was until the move is applied. Engine `Document::comment_layer`; synthetic engine and UI regressions. Estimates unchanged. - **2026-10-22 (M4, #3916 partial):** Document edits and Undo/Redo cancel an Organize page drag. Pointer-release regressions check exact page order and save/reopen. - **2026-10-22 (M10, #3752 paper):** CUPS jobs ask for the chosen paper in every size. The 13 sizes the Print dialog sent no paper for (A6, A0–A2, ARCH, ANSI C–E) use PWG 5101.1 names, checked against a synthetic PPD with `cupsfilter`, and `doc_print` to a printer sends the paper too. The macOS tray half (Auto Select instead of the PPD's default tray) is a craft-print change that needs a pin bump here. Estimates unchanged. + +- **2026-10-23 (M2, #234, continuation of #1567):** Reconciled installed CJK viewer fallback with current rendering changes; Windows preserves installed Song/Hei/Kai families and bounds font reads. GBK-EUC-H CIDFontType2 regression coverage includes text extraction; viewer fallback also survives reload with retained preferences. System fonts remain display-only. No overall readiness estimate changed. - **2026-10-22 (M14, macOS association recovery candidate):** Successful desktop open/save/sign can queue recovery of an existing per-file binding to the running PDFCraft bundle through NSWorkspace. Metadata-only watcher changes retain undo and avoid false overwrite prompts; real same-length/same-mtime edits remain protected. Rebased on 0.12.0 and reuses #3042's shared atomic metadata preservation; the bounded recovery parser does not narrow save limits. Prior native Preview TEST-signature and Finder workflows passed on macOS 27.0.1; other versions remain unqualified. Estimates unchanged. - **2026-10-13 (M7, free image rotation):** A selected page image has a rotate knob (Shift: 15 degree steps) and `image_edit` a `rotate_by` action; the angle is written as one undoable `cm`. Snapping to page edges while moving is in #1740; resize snapping is not done. Estimates unchanged. @@ -357,6 +379,8 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-10-13 (G14, Vietnamese localization):** Added Vietnamese (`vi`, Tiếng Việt) as an interface and date language with a 2,591-entry catalog (`plural_none`) covering 100% of the current localizable key union (plus `theme`/`Appearance`). Locale tags `vi` / `vi-VN` / `vi_VN.UTF-8` resolve; calendar months use numeric `Tháng 1`…`12` / `tháng 1`…`12`. Native-speaker community review recorded. The twelve-language localization mean moves from ≈59% to ≈68%; weighted readiness ≈49% → ≈50%. Status stays `partial`: engine/OS details, document contents, filenames, CLI/MCP and external help stay outside the catalog by design. - **2026-10-12 (M10, Edit on a scanned page):** Turning Edit on (its tab, Edit text & images or Add text) for a scanned page (an image covering at least half of a page with no text of its own; comments and text boxes over it do not count) recognizes the page first, like Acrobat's "Converting scanned page to Editable Text and Images", so its words open for editing. Once per page per window, one undoable "Recognize text" step, with the language of the Recognize Text dialog (English only for now, gap 7). Pages with real text (a flyer with a background picture), read-only files and other tools are left alone. The photographs on a scan are still part of the one page image. + +- **2026-10-12 (M2, installed CJK fonts):** A CID font of an Adobe CJK collection (Japan1, GB1, CNS1, Korea1) that a PDF doesn't embed, such as the Kozuka Gothic of an Acrobat text box or the SongtiTC of a Pages export, drew nothing unless craft-fonts had a Japanese face. After craft-fonts, it now draws with a face already installed on the machine (PingFang, Heiti, Microsoft JhengHei / YaHei, Malgun and the like; read at runtime, never bundled, and used only for the on-screen viewer, never for exports, printing or OCR; Noto CJK and Source Han excluded; `PDFCRAFT_SYSTEM_FONTS=0` turns it off). The face of a collection with several regions (PingFang SC, TC) is chosen by region. Checked on macOS with a synthetic Traditional Chinese, Simplified Chinese and Korean page; the Windows and Linux font lists are untested, and a machine with none of the listed faces still draws nothing. - **2026-10-13 (M3, typed zoom):** The right rail's zoom percentage is an editable field: type 150, 150% or 62.5 % and press Enter, Escape discards, and nonsense keeps the current zoom. The control channel's zoom option shares its parser, which now refuses zero, negative and non-finite values. - **2026-10-13 (M4, indirect page rotation):** Rotation queries and edits resolve indirect integer values on pages and their page-tree ancestors. Turning a page already at 270° by 90° now saves it at 0°, including when its rotation is inherited. Synthetic full/incremental save-and-reopen tests cover nested inheritance, sibling preservation, negative/large angles and invalid-value fallback. Estimates unchanged. - **2026-10-14 (M1, Flate fallback):** Raw deflate whose first bytes resemble a zlib header no longer loses a decoding-limit error during fallback. Strict and tolerant filter regressions and a COS stream regression cover refusal at the caller limit and successful decoding within it. No parity status or estimate changes. @@ -702,6 +726,7 @@ Newest first: the date, what moved, and the new overall estimate. | Date | Change | Summary | |---|---|---| +| 2026-10-28 | patch | M2, PR #4172: integrate main 5af10276 and retain both roadmap histories; viewer-only CJK and print/output font policies and estimates unchanged. | | 2026-10-27 | minor | M10: OCR stages SHA-256-verified ONNX models after the upstream RTEN URL brownout; discovery prefers a complete ONNX pair per directory, falls back to a complete RTEN pair, and never mixes formats or directories; language/web limits and estimates unchanged. | | 2026-10-23 | minor | M8, #3205 follow-up: phone-pattern redaction includes the +39 country code on labelled Italian geographic numbers with a slash; dates, protocol identifiers and whole-run guards remain. Estimates unchanged. | | 2026-10-23 | minor | M11: image downsampling measures physical resolution after page-local UserUnit scaling, including shared nested forms; synthetic measurement and optimization regressions. Estimates unchanged. | @@ -767,6 +792,13 @@ Newest first: the date, what moved, and the new overall estimate. - **2026-10-22 (PR #3227 integration):** M11: integrate upstream `80b7d215`; retain partial target-version compatibility and upstream content-boundary and image-size regression coverage. Scope and estimates unchanged. +- **2026-10-23 (M2, PR #4172):** Combine thumbnails and accessibility figure previews use the viewer's display-only CJK fallback. Windows family matching includes FangSong and subset-prefixed Founder names. Export, print, OCR and automation remain off by default; estimates unchanged. + +- **2026-10-26 (M2, PR #4172 print-preview review):** Print-preview requests disable installed CJK fallback and use an isolated cache so preview font availability matches output. Viewer fallback survives switching in either direction. Synthetic GBK regressions cover inline and single-worker rendering, full pages and tiles; estimates unchanged. + +- **2026-10-26 (M2, PR #4172 documentation review):** Added function documentation for installed-font script selection, bounded candidate reads, platform paths, resolver precedence and regression contracts. No runtime behavior or estimate change. + +- **2026-10-27 (M2, PR #4172 conflict resolution):** Integrated main `9a9813bf` and its engine module split. Opening, layer changes and session renderer rebuilds retain display-only CJK fallback; output configuration and print-preview cache isolation are preserved. Validation passed: 16 scoped engine regressions, 412 renderer unit tests, 43 renderer integration tests (one existing corpus case ignored), 15 Combine tests, scoped Clippy, formatting, layers/assets/parity and renderer/engine WASM checks. Estimates unchanged. ## Pages sidebar keyboard refresh (2026-10-30) PR #2133 retains document-local Pages sidebar focus for Ctrl/Cmd+A and Escape after the UI module split. The selection does not navigate. The focused regression is in `crates/ui-egui/tests/it/view.rs`; current-head hosted validation is pending. Feature counts and estimates are unchanged. diff --git a/crates/engine/src/layers.rs b/crates/engine/src/layers.rs index c2f4f63714..f75967097c 100644 --- a/crates/engine/src/layers.rs +++ b/crates/engine/src/layers.rs @@ -4,7 +4,7 @@ use std::sync::Arc; use pdfcraft_render::{Layer, LayerOp, RenderPool}; -use crate::{Document, render_threads}; +use crate::{Document, display_config, render_threads}; /// Write `layers`' visibility into the default optional-content configuration of `cos` (a copy /// made for printing): `/BaseState /ON`, `/ON` the visible layers, `/OFF` the hidden ones. Its @@ -31,7 +31,7 @@ pub(crate) fn set_default_layer_state(cos: &mut pdfcraft_cos::Document, layers: pub(crate) fn use_layer_choices(doc: &mut Document) { let overrides: Vec<(i32, i32, bool)> = doc.info.layers.iter().map(|l| (l.id.0 as i32, l.id.1 as i32, l.visible)).collect(); doc.config.layers = Arc::new(overrides); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); } /// Apply a set-layer-visibility action to `layers`, one change at a time, so a toggle flips the diff --git a/crates/engine/src/lib.rs b/crates/engine/src/lib.rs index 7cd2f96a96..0b49cfd4e1 100644 --- a/crates/engine/src/lib.rs +++ b/crates/engine/src/lib.rs @@ -158,5 +158,13 @@ use clock::{local_utc_offset, parse_ymd}; pub use doc_state::{DocId, ImageEdit, OptimizeReport, RecoverySnapshot, SplitPart, guard, merge_data_files}; use doc_state::{LOW_RESOLUTION_PRINT_DPI, render_threads, renderer_password, signatures_of, writable}; +/// The configuration of the pool that draws the viewer: the document's own, plus installed CJK +/// fonts for fonts a PDF does not embed (`pdfcraft_render::RenderConfig::system_fonts`). Exports, +/// printing, OCR and automation use the document configuration itself, so nothing drawn with an +/// installed font is saved or printed (AGENTS.md §1.2, §1.4). +fn display_config(config: &pdfcraft_render::RenderConfig) -> pdfcraft_render::RenderConfig { + config.clone().on_screen() +} + #[cfg(test)] mod tests; diff --git a/crates/engine/src/open.rs b/crates/engine/src/open.rs index 58bc8327e2..32db07abee 100644 --- a/crates/engine/src/open.rs +++ b/crates/engine/src/open.rs @@ -7,8 +7,8 @@ use pdfcraft_render::{DocInfo, OpenError, RenderConfig, RenderPool, inspect}; use crate::history::{Editor, Keys, TextBlocksCache}; use crate::{ - AppearanceCache, DocId, Document, Session, TrustStore, XFA_SCRIPTS_OFF, XfaLayout, display_bytes, guard, info_fields, js, note_warnings, - recovered_render_password, render_threads, same_pages, signatures_of, xfa, xfa_layout, xfa_values_from_datasets, + AppearanceCache, DocId, Document, Session, TrustStore, XFA_SCRIPTS_OFF, XfaLayout, display_bytes, display_config, guard, info_fields, js, + note_warnings, recovered_render_password, render_threads, same_pages, signatures_of, xfa, xfa_layout, xfa_values_from_datasets, }; /// Immutable session state needed while a document is prepared away from its owning thread. @@ -304,7 +304,7 @@ impl Session { } } let display = display_bytes(editor.as_mut(), &bytes); - let renderer = RenderPool::new(display.clone(), render_threads(), config.clone()); + let renderer = RenderPool::new(display.clone(), render_threads(), display_config(&config)); let (form, adopted) = editor .as_ref() .map(|e| { diff --git a/crates/engine/src/session.rs b/crates/engine/src/session.rs index 1051903bae..9a4cb04dac 100644 --- a/crates/engine/src/session.rs +++ b/crates/engine/src/session.rs @@ -12,10 +12,10 @@ use crate::history::{Keys, MAX_UNDO, Scope, keys_after, scope_of, uses_scripts}; use crate::{ BookmarkPage, BookmarkRow, CombineSource, Combined, DataFormat, DocId, Document, Edit, EditError, Hidden, ImageResolution, LOW_RESOLUTION_PRINT_DPI, Label, ObjectKind, OptimizeReport, RecoverySnapshot, SignOptions, SourceKind, SplitPart, SummarySort, TrustStore, - XFA_REMOVED_PREFIX, XFA_SCRIPTS_OFF, apply_layer_state, comment_list, comment_summary, decode_text, display_bytes, guard, info_fields, js, - local_utc_offset, note_warnings, open_source, open_source_with, optimize, outline_rows, parse_ymd, print, print_raster, render_threads, - scan_colours, set_default_layer_state, shown_under_redactions, signatures_of, source_kind, sync_xfa_state, undrawn_notices, use_layer_choices, - writable, xfa, xfa_sync_datasets, + XFA_REMOVED_PREFIX, XFA_SCRIPTS_OFF, apply_layer_state, comment_list, comment_summary, decode_text, display_bytes, display_config, guard, + info_fields, js, local_utc_offset, note_warnings, open_source, open_source_with, optimize, outline_rows, parse_ymd, print, print_raster, + render_threads, scan_colours, set_default_layer_state, shown_under_redactions, signatures_of, source_kind, sync_xfa_state, undrawn_notices, + use_layer_choices, writable, xfa, xfa_sync_datasets, }; mod editing; diff --git a/crates/engine/src/session/editing.rs b/crates/engine/src/session/editing.rs index c78b10ab94..7758dc7c85 100644 --- a/crates/engine/src/session/editing.rs +++ b/crates/engine/src/session/editing.rs @@ -2,9 +2,9 @@ use super::check_refresh_panic; use super::{ Arc, DocId, Document, Edit, EditCtx, EditError, Label, MAX_UNDO, ObjectKind, OnceLock, RenderPool, SaveOptions, Scope, Session, - XFA_REMOVED_PREFIX, XFA_SCRIPTS_OFF, check_permission, comment_list, display_bytes, guard, info_fields, inspect, js, keys_after, note_warnings, - render_threads, run_edit, scan_colours, scope_of, shown_under_redactions, signatures_of, sync_xfa_state, undrawn_notices, uses_scripts, - write_incremental, xfa, xfa_sync_datasets, + XFA_REMOVED_PREFIX, XFA_SCRIPTS_OFF, check_permission, comment_list, display_bytes, display_config, guard, info_fields, inspect, js, keys_after, + note_warnings, render_threads, run_edit, scan_colours, scope_of, shown_under_redactions, signatures_of, sync_xfa_state, undrawn_notices, + uses_scripts, write_incremental, xfa, xfa_sync_datasets, }; use crate::{EditNotice, MarkKind}; @@ -375,7 +375,7 @@ impl Session { let signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache) }; doc.display = display_bytes(Some(editor), &bytes); doc.signatures = Arc::new(signatures); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.bytes = bytes; Ok(()) } @@ -432,7 +432,7 @@ impl Session { let signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache) }; doc.display = display_bytes(Some(editor), &bytes); doc.signatures = Arc::new(signatures); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.bytes = bytes; doc.fonts_stale = true; Ok(()) @@ -459,7 +459,7 @@ impl Session { /// After a refresh failed: draw the display bytes it left (the original, once released), /// so the page still shows a document. pub(super) fn show_released(doc: &mut Document) { - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); } /// Rebuild working bytes, inspection and renderer from the current edit state. @@ -516,7 +516,7 @@ impl Session { let signatures = signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache); doc.display = display_bytes(Some(editor), &bytes); doc.signatures = Arc::new(signatures); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.bytes = bytes; Ok(()) } diff --git a/crates/engine/src/session/save.rs b/crates/engine/src/session/save.rs index d4156a2eb1..1e6918f6a6 100644 --- a/crates/engine/src/session/save.rs +++ b/crates/engine/src/session/save.rs @@ -1,6 +1,6 @@ use super::{ - Arc, DocId, EditError, Keys, OpenError, RecoverySnapshot, RenderPool, SaveOptions, Session, SignOptions, TrustStore, display_bytes, guard, - render_threads, signatures_of, writable, write_full, write_incremental, xfa_sync_datasets, + Arc, DocId, EditError, Keys, OpenError, RecoverySnapshot, RenderPool, SaveOptions, Session, SignOptions, TrustStore, display_bytes, + display_config, guard, render_threads, signatures_of, writable, write_full, write_incremental, xfa_sync_datasets, }; impl Session { @@ -170,7 +170,7 @@ impl Session { { doc.config.reading_colors = old.config.reading_colors; doc.config.hide_comments = old.config.hide_comments; - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); } // Caches keyed by the edit generation (edit lines, images, measurements) must see a new // one; there is nothing new to autosave. @@ -249,7 +249,7 @@ impl Session { let signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &doc.bytes, &doc.trust, &doc.sig_cache) }; doc.display = display_bytes(Some(editor), &doc.bytes); doc.signatures = Arc::new(signatures); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); } } diff --git a/crates/engine/src/session/view.rs b/crates/engine/src/session/view.rs index 055437f562..c0e7417c8e 100644 --- a/crates/engine/src/session/view.rs +++ b/crates/engine/src/session/view.rs @@ -1,6 +1,6 @@ use super::{ Arc, BookmarkPage, BookmarkRow, DocId, Document, EditError, LayerOp, OpenError, ReadingColors, RenderConfig, RenderPool, Session, SummarySort, - apply_layer_state, comment_summary, local_utc_offset, outline_rows, parse_ymd, render_threads, use_layer_choices, + apply_layer_state, comment_summary, display_config, local_utc_offset, outline_rows, parse_ymd, render_threads, use_layer_choices, }; impl Session { @@ -246,7 +246,7 @@ impl Session { return false; } doc.config.hide_comments = hide; - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.generation += 1; true } @@ -259,7 +259,7 @@ impl Session { return false; } doc.config.reading_colors = colors; - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.generation += 1; true } diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index 833fa16b9f..333d0fd498 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -581,6 +581,80 @@ fn edits_update_view_data_and_mark_dirty() { assert_eq!(page_texts(&s, id), ["Page 1", "Page 3"], "renderer refreshed"); } +/// A font of an Adobe CJK collection that a PDF does not embed is drawn with an installed face on +/// the viewer's pool only. An export of the same page, which can be saved, has nothing drawn with +/// an installed font (AGENTS.md §1.2, §1.4). Skipped where no such face is installed. +#[test] +fn installed_cjk_fonts_draw_in_the_viewer_but_not_in_an_export() { + let content = "BT /F1 40 Tf 5 15 Td <7E419AD45B57> Tj ET"; + let pdf = format!( + "%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 60] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >> endobj +4 0 obj << /Length {} >> stream +{content} +endstream endobj +5 0 obj << /Type /Font /Subtype /Type0 /BaseFont /MSung-Light /Encoding /UniCNS-UCS2-H /DescendantFonts [6 0 R] >> endobj +6 0 obj << /Type /Font /Subtype /CIDFontType0 /BaseFont /MSung-Light /CIDSystemInfo << /Registry (Adobe) /Ordering (CNS1) /Supplement 2 >> /FontDescriptor 7 0 R /DW 1000 >> endobj +7 0 obj << /Type /FontDescriptor /FontName /MSung-Light /Flags 6 /FontBBox [0 -141 1000 859] /ItalicAngle 0 /Ascent 859 /Descent -141 /CapHeight 700 /StemV 80 >> endobj +trailer << /Root 1 0 R >> +%%EOF", + content.len() + ); + let mut s = Session::new(); + let id = s.open("cns1.pdf", None, Arc::new(pdf.into_bytes()), None).unwrap(); + let doc = s.get(id).unwrap(); + doc.renderer.set_queue(vec![pdfcraft_render::RenderRequest { + page: 0, + kind: pdfcraft_render::RequestKind::Pixels, + scale: 1.0, + ..Default::default() + }]); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(30); + let viewer = loop { + if let Some(page) = doc.renderer.try_recv() { + assert!(page.error.is_none(), "{:?}", page.error); + break page; + } + assert!(std::time::Instant::now() < deadline, "page render did not finish"); + std::thread::yield_now(); + }; + let inked = |rgba: &[u8]| rgba.as_chunks::<4>().0.iter().filter(|c| c[0] < 128).count(); + // Whether this machine has a face that can draw the page at all, asked of a renderer with the + // fallback switched on by hand, so a viewer pool that lost it fails below instead of skipping. + let probe = pdfcraft_render::PageRenderer::new(doc.display.clone(), pdfcraft_render::RenderConfig { system_fonts: true, ..Default::default() }) + .render(pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() }); + assert!(probe.error.is_none(), "{:?}", probe.error); + if inked(&probe.rgba) < 300 { + eprintln!("no installed Traditional Chinese face (or PDFCRAFT_SYSTEM_FONTS=0): skipped"); + return; + } + assert!(inked(&viewer.rgba) > 300, "the viewer's pool draws it with the installed face ({} dark pixels)", inked(&viewer.rgba)); + let png = export::Exporter::new(doc).png(0, 72.0).unwrap(); + let exported = image::load_from_memory(&png).unwrap().into_rgba8(); + assert!(inked(exported.as_raw()) < 50, "an export draws nothing with an installed font ({} dark pixels)", inked(exported.as_raw())); + + // Reload rebuilds the pool when retaining a view preference. That path must keep + // the display-only fallback too; otherwise Chinese disappears after a file refresh. + let bytes = doc.bytes.clone(); + assert!(s.set_hide_comments(id, true)); + s.reload(id, bytes).unwrap(); + let doc = s.get(id).unwrap(); + assert!(doc.comments_hidden()); + doc.renderer.set_queue(vec![pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() }]); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(30); + let reloaded = loop { + if let Some(page) = doc.renderer.try_recv() { + assert!(page.error.is_none(), "{:?}", page.error); + break page; + } + assert!(std::time::Instant::now() < deadline, "reloaded page render did not finish"); + std::thread::yield_now(); + }; + assert!(inked(&reloaded.rgba) > 300, "reload must retain visible Chinese text"); +} + #[test] fn metadata_edit_keeps_the_renderer_without_changing_page_content() { let (mut s, id) = session_with(1); diff --git a/crates/render/Cargo.toml b/crates/render/Cargo.toml index 28202a38b4..71467dfeb2 100644 --- a/crates/render/Cargo.toml +++ b/crates/render/Cargo.toml @@ -13,6 +13,8 @@ pdfcraft-cos = { workspace = true } pdfcraft-fonts = { workspace = true } hayro = { workspace = true } hayro-syntax = { workspace = true } +# Picks the face of an installed font collection (the version pdfcraft-fonts already uses). +skrifa = "0.44" lopdf = { workspace = true } kurbo = { workspace = true } thiserror = { workspace = true } @@ -27,7 +29,5 @@ bytemuck = { version = "1" } getrandom = { version = "0.4", features = ["wasm_js"] } [dev-dependencies] -# Checks the Japanese substitute's character coverage; the version pdfcraft-fonts already uses. -skrifa = "0.44" # Builds the decompression-bomb streams in the raster tests. flate2 = { workspace = true } diff --git a/crates/render/src/lib.rs b/crates/render/src/lib.rs index 954823ef09..6f9a3ac319 100644 --- a/crates/render/src/lib.rs +++ b/crates/render/src/lib.rs @@ -13,6 +13,7 @@ mod pixels; mod raster; mod reading; mod structure; +mod system_font; pub mod text; pub mod transition; diff --git a/crates/render/src/raster.rs b/crates/render/src/raster.rs index c002c6ec92..f973851d26 100644 --- a/crates/render/src/raster.rs +++ b/crates/render/src/raster.rs @@ -78,15 +78,36 @@ pub struct RenderConfig { /// The page clears to transparent instead of white (or the reading background): for a page /// holding only what is laid over another raster, such as a comment while it is dragged. pub transparent: bool, + /// Draw a CID font of an Adobe CJK collection that the PDF does not embed with one face already + /// installed on this machine (see `system_font`). Off by default and on only for on-screen + /// viewer and preview rasters: exports, printing and OCR render without it, so nothing drawn with an + /// installed font is saved or printed (AGENTS.md §1.2, §1.4). + pub system_fonts: bool, } impl RenderConfig { + /// This configuration for a raster that is only shown on screen (the viewer, Combine + /// thumbnails, figure previews): installed CJK faces may draw fonts the PDF doesn't embed. + /// Never use it for anything that can be saved, printed, recognised or returned to an agent + /// (AGENTS.md §1.2, §1.4). + #[must_use] + pub fn on_screen(self) -> Self { + Self { system_fonts: true, ..self } + } + + /// Build interpreter settings with layer and comment preferences and a font resolver that + /// tries bundled Japanese faces, then opt-in installed CJK faces, then standard substitutes. + /// Passwords belong to parser construction; request-specific preview overrides are applied + /// by `render_page`, which also preserves the caller's warning and cancellation hooks. fn settings(&self) -> InterpreterSettings { let standard = InterpreterSettings::default().font_resolver; + let system_fonts = self.system_fonts; InterpreterSettings { ocg_overrides: self.layers.clone(), hide_comments: self.hide_comments, - font_resolver: Arc::new(move |query| japanese_fallback(query).or_else(|| standard(query))), + font_resolver: Arc::new(move |query| { + japanese_fallback(query).or_else(|| system_fonts.then(|| system_fallback(query)).flatten()).or_else(|| standard(query)) + }), ..InterpreterSettings::default() } } @@ -119,6 +140,14 @@ fn japanese_fallback(query: &FontQuery) -> Option<(FontData, u32)> { Some((Arc::new(face.bytes), 0)) } +/// An installed face for a CID font of an Adobe CJK collection that the PDF doesn't embed and +/// craft-fonts has no face for (Traditional and Simplified Chinese, Korean, and Japanese in a +/// build without craft-fonts). `None` for every other font. +fn system_fallback(query: &FontQuery) -> Option<(FontData, u32)> { + let FontQuery::Fallback(f) = query else { return None }; + crate::system_font::fallback(f) +} + #[derive(Debug, PartialEq)] enum JapaneseFace { Mincho, @@ -182,6 +211,7 @@ pub struct RenderRequest { /// The tag bit of a print-preview raster. Such requests always render the document's true /// colours, ignoring `RenderConfig::reading_colors`: a preview's job is fidelity to the /// printout, and the print itself — imposed from the document — is never recoloured. +/// Installed-font fallback is also disabled, matching the output's font policy. pub const PRINT_TAG: u64 = 1 << 60; /// The reading colours a request renders with: print previews ([`PRINT_TAG`]) ignore the @@ -319,6 +349,23 @@ fn render_page<'a>( config: &RenderConfig, req: RenderRequest, ) -> Result { + // Print previews share the viewer pool, but must use the output font policy. Keep the + // caller's cancellation/warning hooks, layers and annotation settings. A fresh cache is + // essential: hayro keys cached fonts by PDF dictionary, not by the font resolver, so the + // viewer cache could supply an installed face even after changing the resolver (and a + // cached missing font from print preview could blank the viewer on switching back). + let print_settings; + let print_cache; + let (settings, cache) = if config.system_fonts && req.tag & PRINT_TAG != 0 { + print_settings = InterpreterSettings { + font_resolver: RenderConfig { system_fonts: false, ..config.clone() }.settings().font_resolver, + ..settings.clone() + }; + print_cache = RenderCache::new(); + (&print_settings, &print_cache) + } else { + (settings, cache) + }; let reading = request_reading(config, &req); if req.kind == RequestKind::Text { return match catch_unwind(AssertUnwindSafe(|| crate::text::extract_page(pdf, req.page, settings))) { diff --git a/crates/render/src/raster/tests.rs b/crates/render/src/raster/tests.rs index 11266f057e..df6896b980 100644 --- a/crates/render/src/raster/tests.rs +++ b/crates/render/src/raster/tests.rs @@ -7040,6 +7040,146 @@ fn test_clip_only_text_raster_preserves_clipping_without_painting() { assert!(pixels.contains(&white), "the clip is text-shaped, not the whole page"); } +#[cfg(target_os = "windows")] +mod gbk_glyphs; + +/// Chinese and Korean CID fonts that aren't embedded (Adobe-CNS1 `MSung-Light`, Adobe-GB1 +/// `STSong-Light`, Adobe-Korea1 `HYGoThic-Medium`; Acrobat also writes Japanese `KozGoPr6N` +/// text boxes without the font) drew nothing, because craft-fonts has only Japanese faces. +/// They now draw in a face already installed on the machine, read at runtime. A machine +/// without such a face draws nothing, as before, so each check is skipped there. +#[test] +fn non_embedded_chinese_and_korean_cid_fonts_draw_with_an_installed_face() { + use hayro::hayro_interpret::font::FallbackFontQuery; + use hayro::hayro_interpret::hayro_cmap::CharacterCollection; + // (base font, collection, ordering, encoding, encoded text, descendant subtype) + let cases = [ + ("MSung-Light", CidFamily::AdobeCNS1, "CNS1", "UniCNS-UCS2-H", "7E419AD45B57", "CIDFontType0"), // 繁體字 + ("STSong-Light", CidFamily::AdobeGB1, "GB1", "UniGB-UCS2-H", "72698D448DF7", "CIDFontType0"), // 物资跷 + ("HYGoThic-Medium", CidFamily::AdobeKorea1, "Korea1", "UniKS-UCS2-H", "D55CAE00", "CIDFontType0"), // 한글 + // The real Windows book uses non-embedded TrueType CID fonts and a legacy GBK CMap, + // without ToUnicode. These bytes encode 中文; they are not Unicode code points. + ("SimSun", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), + ("SimHei", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), + ("KaiTi", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), + ("FangSong", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), + ("ABCDEF+FZSSK--GBK1-0", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), + ("ABCDEF+FZHTK--GBK1-0", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), + ("ABCDEF+FZKTK--GBK1-0", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), + ("ABCDEF+FZFSK--GBK1-0", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), + ]; + for (base_font, family, ordering, encoding, code, subtype) in cases { + let query = FontQuery::Fallback(FallbackFontQuery { + post_script_name: Some(base_font.into()), + character_collection: Some(CharacterCollection { family, supplement: 2 }), + ..FallbackFontQuery::default() + }); + if super::system_fallback(&query).is_none() { + eprintln!("no installed face for {base_font} (or PDFCRAFT_SYSTEM_FONTS=0): skipped"); + continue; + } + let content = format!("BT /F1 40 Tf 5 15 Td <{code}> Tj ET"); + let pdf = format!( + "%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 60] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >> endobj +4 0 obj << /Length {} >> stream +{content} +endstream endobj +5 0 obj << /Type /Font /Subtype /Type0 /BaseFont /{base_font} /Encoding /{encoding} /DescendantFonts [6 0 R] >> endobj +6 0 obj << /Type /Font /Subtype /{subtype} /BaseFont /{base_font} /CIDSystemInfo << /Registry (Adobe) /Ordering ({ordering}) /Supplement 2 >> + /FontDescriptor 7 0 R /DW 1000 >> endobj +7 0 obj << /Type /FontDescriptor /FontName /{base_font} /Flags 6 /FontBBox [0 -141 1000 859] /ItalicAngle 0 /Ascent 859 /Descent -141 /CapHeight 700 /StemV 80 >> endobj +trailer << /Root 1 0 R >> +%%EOF", + content.len() + ); + let pdf = Arc::new(pdf.into_bytes()); + let ink = |config: RenderConfig| { + let mut r = PageRenderer::new(pdf.clone(), config); + let p = r.render(RenderRequest { page: 0, kind: RequestKind::Pixels, tile: None, scale: 1.0, tag: 0, ..Default::default() }); + assert!(p.error.is_none(), "{base_font}: {:?}", p.error); + p.rgba.as_chunks::<4>().0.iter().filter(|c| c[0] < 128).count() + }; + // Two to four glyphs at 40 pt cover several hundred dark pixels; a blank run has none. + let shown = ink(RenderConfig { system_fonts: true, ..RenderConfig::default() }); + assert!(shown > 300, "{base_font}: the text is drawn ({shown} dark pixels)"); + // Off by default: exports, printing and OCR render without an installed font, so + // nothing drawn with one can be saved (AGENTS.md §1.2, §1.4). + let saved = ink(RenderConfig::default()); + assert!(saved < 50, "{base_font}: not drawn without system_fonts ({saved} dark pixels)"); + if encoding == "GBK-EUC-H" { + #[cfg(target_os = "windows")] + gbk_glyphs::assert_expected_glyphs(&pdf, base_font); + let mut renderer = PageRenderer::new(pdf, RenderConfig { system_fonts: true, ..Default::default() }); + let text = renderer.render(RenderRequest { page: 0, kind: RequestKind::Text, ..Default::default() }); + assert!(text.error.is_none(), "{base_font}: {:?}", text.error); + assert_eq!(text.text.unwrap().plain_text().trim(), "中文", "the fallback preserves GBK text mapping"); + } + } +} + +/// Print preview must match output even after the same worker cached an installed face. +/// Switching back to the viewer must not retain the print resolver's missing-font result. +#[cfg(not(target_arch = "wasm32"))] +#[test] +fn print_preview_excludes_installed_fonts_without_poisoning_viewer_cache() { + use hayro::hayro_interpret::font::FallbackFontQuery; + use hayro::hayro_interpret::hayro_cmap::CharacterCollection; + let query = FontQuery::Fallback(FallbackFontQuery { + post_script_name: Some("SimSun".into()), + character_collection: Some(CharacterCollection { family: CidFamily::AdobeGB1, supplement: 2 }), + ..Default::default() + }); + if super::system_fallback(&query).is_none() { + eprintln!("no installed GB1 face (or PDFCRAFT_SYSTEM_FONTS=0): skipped"); + return; + } + let content = "BT /F1 40 Tf 5 15 Td Tj ET"; + let bytes = Arc::new(format!( + "%PDF-1.7\n\ + 1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj\n\ + 2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj\n\ + 3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 120 60] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >> endobj\n\ + 4 0 obj << /Length {} >> stream\n{content}\nendstream endobj\n\ + 5 0 obj << /Type /Font /Subtype /Type0 /BaseFont /SimSun /Encoding /GBK-EUC-H /DescendantFonts [6 0 R] >> endobj\n\ + 6 0 obj << /Type /Font /Subtype /CIDFontType2 /BaseFont /SimSun /CIDSystemInfo << /Registry (Adobe) /Ordering (GB1) /Supplement 2 >> /FontDescriptor 7 0 R /DW 1000 >> endobj\n\ + 7 0 obj << /Type /FontDescriptor /FontName /SimSun /Flags 6 /FontBBox [0 -141 1000 859] /ItalicAngle 0 /Ascent 859 /Descent -141 /CapHeight 700 /StemV 80 >> endobj\n\ + trailer << /Root 1 0 R >>\n%%EOF", content.len() + ).into_bytes()); + let config = RenderConfig::default().on_screen(); + let mut inline = PageRenderer::new(bytes.clone(), config.clone()); + let mut output = PageRenderer::new(bytes.clone(), RenderConfig::default()); + let pool = RenderPool::new(bytes, 1, config); + for tile in [None, Some(Tile { x: 0, y: 0, w: 120, h: 60 })] { + // Cover both directions with a warm worker cache, including other caller tag bits. + for tag in [PRINT_TAG | 1, 2, PRINT_TAG | 3, 4] { + let req = RenderRequest { page: 0, scale: 1.0, tile, tag, ..Default::default() }; + let truth = output.render(req); + assert!(truth.error.is_none(), "{:?}", truth.error); + pool.set_queue(vec![req]); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(10); + let worker = loop { + if let Some(page) = pool.try_recv() { + break page; + } + assert!(std::time::Instant::now() < deadline, "print-preview worker timed out"); + std::thread::sleep(std::time::Duration::from_millis(5)); + }; + for page in [inline.render(req), worker] { + assert!(page.error.is_none(), "{:?}", page.error); + if tag & PRINT_TAG != 0 { + assert_eq!(page.rgba, truth.rgba, "print preview must match output, tile={tile:?}"); + } else { + let ink = page.rgba.as_chunks::<4>().0.iter().filter(|c| c[0] < 128).count(); + assert!(ink > 300, "viewer still draws installed Chinese glyphs: {ink}"); + } + } + } + } +} + /// A malformed token in a content stream used to end the stream there, so the renderer, text /// extraction, Find and search-based redaction lost the rest of the page while the edit /// reader, MuPDF and pdftotext read it (#3565, #3568). Each token below is now stepped over. diff --git a/crates/render/src/raster/tests/gbk_glyphs.rs b/crates/render/src/raster/tests/gbk_glyphs.rs new file mode 100644 index 0000000000..8443fed624 --- /dev/null +++ b/crates/render/src/raster/tests/gbk_glyphs.rs @@ -0,0 +1,60 @@ +//! A same-machine pixel oracle, independent of GBK decoding and installed-face selection. +//! All font reads and reference rasters stay in memory; nothing is embedded or published. + +use super::*; +use hayro::hayro_interpret::font::FontData; +use skrifa::MetadataProvider as _; +use skrifa::string::StringId; + +/// Compare the real GBK PDF path with explicit Unicode and a known installed face. Unlike an +/// ink count or extracted text, this fails if drawing chooses another glyph or font family. +pub(super) fn assert_expected_glyphs(pdf: &Arc>, base_font: &str) { + let (expected, file) = match base_font { + "SimSun" | "ABCDEF+FZSSK--GBK1-0" => ("SimSun", "simsun.ttc"), + "SimHei" | "ABCDEF+FZHTK--GBK1-0" => ("SimHei", "simhei.ttf"), + "KaiTi" | "ABCDEF+FZKTK--GBK1-0" => ("KaiTi", "simkai.ttf"), + "FangSong" | "ABCDEF+FZFSK--GBK1-0" => ("FangSong", "simfang.ttf"), + other => panic!("no independent GBK oracle for {other}"), + }; + // Do not use the production candidate list, resolver or face-selection helpers here. + let directory = std::env::var_os("WINDIR").or_else(|| std::env::var_os("SystemRoot")).unwrap_or_else(|| r"C:\Windows".into()); + let path = std::path::PathBuf::from(directory).join("Fonts").join(file); + if !path.is_file() { + eprintln!("{base_font}: exact {expected} face unavailable; pixel oracle skipped"); + return; + } + let bytes = Arc::new(std::fs::read(path).expect("read installed reference font")); + let index = (0..32) + .find(|&index| { + skrifa::FontRef::from_index(&bytes, index) + .is_ok_and(|font| font.localized_strings(StringId::POSTSCRIPT_NAME).any(|name| name.chars().collect::() == expected)) + }) + .expect("reference file contains the expected PostScript face"); + let face = skrifa::FontRef::from_index(&bytes, index).unwrap(); + for character in ['\u{4e2d}', '\u{6587}'] { + assert!(face.charmap().map(character).is_some_and(|id| id.to_u32() != 0), "{expected} must contain {character}"); + } + + // Explicit ToUnicode maps arbitrary reference codes to U+4E2D and U+6587, bypassing GBK + // and Adobe-GB1 CID lookup. The fixed resolver below bypasses family classification too. + let cmap = "begincmap /CMapType 2 def 1 begincodespacerange <0000> endcodespacerange \ + 2 beginbfchar <0001> <4E2D> <0002> <6587> endbfchar endcmap"; + let source = std::str::from_utf8(pdf).unwrap(); + assert!(source.contains("/Encoding /GBK-EUC-H") && source.contains("")); + let reference = source + .replace("/Encoding /GBK-EUC-H", "/Encoding /Identity-H /ToUnicode 8 0 R") + .replace("", "<00010002>") + .replace("trailer <<", &format!("8 0 obj << /Length {} >> stream\n{cmap}\nendstream endobj\ntrailer <<", cmap.len())); + let parsed = Pdf::new(Arc::new(reference.into_bytes())).unwrap(); + let data: FontData = bytes; + let settings = InterpreterSettings { font_resolver: Arc::new(move |_| Some((data.clone(), index))), ..Default::default() }; + let reference = hayro::render(&parsed.pages()[0], &RenderCache::new(), &settings, &RenderSettings { bg_color: WHITE, ..Default::default() }); + let mut renderer = PageRenderer::new(pdf.clone(), RenderConfig::default().on_screen()); + let actual = renderer.render(RenderRequest { page: 0, kind: RequestKind::Pixels, scale: 1.0, ..Default::default() }); + assert!(actual.error.is_none(), "{base_font}: {:?}", actual.error); + let expected_pixels = reference.data_as_u8_slice(); + assert_eq!(actual.rgba.len(), expected_pixels.len()); + assert!(expected_pixels.as_chunks::<4>().0.iter().filter(|pixel| pixel[0] < 128).count() > 300, "reference must draw two visible glyphs"); + assert!(actual.rgba.as_ref() == expected_pixels, "{base_font}: GBK raster must draw the exact {expected} glyphs for U+4E2D U+6587"); + eprintln!("{base_font}: exact {expected} Unicode-reference pixels matched"); +} diff --git a/crates/render/src/system_font.rs b/crates/render/src/system_font.rs new file mode 100644 index 0000000000..8fcd167415 --- /dev/null +++ b/crates/render/src/system_font.rs @@ -0,0 +1,519 @@ +//! Installed CJK faces for CID fonts a PDF doesn't embed. +//! +//! Acrobat and other tools often write Chinese, Japanese or Korean text with one of the Adobe +//! collections (Adobe-Japan1, GB1, CNS1, Korea1) and a font name such as `KozGoPr6N-Medium` or +//! `MSung-Light` without embedding the font. hayro maps those CIDs to Unicode itself, but it needs +//! a face to draw them with; without one the text is blank. The craft-fonts Japanese faces come +//! first (see `japanese_fallback` in `raster.rs`); this module is the last resort: one face +//! already installed on this machine, used for the viewer only (`RenderConfig::system_fonts`, off +//! for exports, printing and OCR). It is read at runtime and never embedded or shipped +//! (AGENTS.md §1.4), Noto CJK and Source Han are left out on purpose (§1.1), and +//! `PDFCRAFT_SYSTEM_FONTS=0` turns it off, as for the interface font. + +use std::collections::HashMap; +use std::io::Read as _; +use std::path::{Path, PathBuf}; +use std::sync::{Arc, Mutex, OnceLock, PoisonError}; + +use hayro::hayro_interpret::font::{FallbackFontQuery, FontData}; +use hayro::hayro_interpret::hayro_cmap::CidFamily; +use skrifa::MetadataProvider as _; +use skrifa::string::StringId; + +/// Larger files are not read: a font path is still untrusted input. CJK system faces are big +/// collections (PingFang is tens of megabytes). +const MAX_BYTES: u64 = 96 << 20; +/// Faces tried in a collection (`.ttc`). +const MAX_FACES: u32 = 32; +/// Downloaded font assets looked at per kind on macOS. +const MAX_ASSETS: usize = 512; + +/// The writing system a character collection is for. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +enum Script { + Japanese, + SimplifiedChinese, + TraditionalChinese, + Korean, +} + +impl Script { + /// Map the four supported Adobe CJK collections to their writing systems. Identity and + /// custom collections provide no regional hint and cannot select an installed face here. + fn of(family: &CidFamily) -> Option { + match family { + CidFamily::AdobeJapan1 => Some(Self::Japanese), + CidFamily::AdobeGB1 => Some(Self::SimplifiedChinese), + CidFamily::AdobeCNS1 => Some(Self::TraditionalChinese), + CidFamily::AdobeKorea1 => Some(Self::Korean), + CidFamily::AdobeIdentity | CidFamily::Custom { .. } => None, + } + } + + /// A character the face must have to be worth using. Each is common and not shared by the + /// other scripts' faces: 物 (Japanese; also Chinese, so it only rules out Latin-only faces), + /// 欢 (Simplified only), 說 (Traditional only), 한 (Hangul). + fn probe(self) -> char { + match self { + Self::Japanese => '\u{7269}', + Self::SimplifiedChinese => '\u{6B22}', + Self::TraditionalChinese => '\u{8AAA}', + Self::Korean => '\u{D55C}', + } + } +} + +/// One installed file and the words (in a face's family name) that pick the face for the right +/// region out of a collection that holds several (PingFang SC, TC and HK share one file). +struct Candidate { + path: PathBuf, + region: &'static [&'static str], +} + +/// Keep named Windows body, heading, calligraphic and FangSong faces distinct. A cache keyed only +/// by script and weight would make the first Chinese font on a page stand in for all of them. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +enum ChineseFamily { + Default, + Song, + Hei, + Kai, + FangSong, +} + +impl ChineseFamily { + /// The family a font name asks for: the Windows and Founder (方正) names that Chinese PDFs, + /// CNKI papers in particular, use without embedding the font. FangSong is tested first, as + /// its names contain "song" too. Founder PostScript names are matched by prefix (`FZSSK--GBK1-0` + /// is 方正书宋, `FZXBSK--GBK1-0` 方正小标宋, `FZHTK--GBK1-0` 方正黑体, `FZKTK--GBK1-0` 方正楷体, + /// `FZFSK--GBK1-0` 方正仿宋). + fn of(name: &str) -> Self { + // PDF subset names prepend six uppercase ASCII letters and '+'. Match the actual + // PostScript name so Founder prefixes work for subsets too. + let name = match name.split_once('+') { + Some((tag, family)) if tag.len() == 6 && tag.bytes().all(|b| b.is_ascii_uppercase()) => family, + _ => name, + }; + let name = name.to_ascii_lowercase(); + let has = |hints: &[&str]| hints.iter().any(|hint| name.contains(hint)); + let starts = |prefixes: &[&str]| prefixes.iter().any(|p| name.starts_with(p)); + if has(&["fangsong", "simfang", "仿宋"]) || starts(&["fzfs"]) { + Self::FangSong + } else if has(&["simsun", "stsong", "songti", "song", "宋"]) || starts(&["fzss", "fzbs", "fzxbs", "fzcs"]) { + Self::Song + } else if has(&["simhei", "stheiti", "stxihei", "heiti", "黑"]) || starts(&["fzht", "fzhei", "fzlth", "fzdh"]) { + Self::Hei + } else if has(&["kaiti", "stkaiti", "kai", "楷"]) || starts(&["fzkt", "fzkai"]) { + Self::Kai + } else { + Self::Default + } + } +} + +/// The installed face to draw a CID font of `query`'s collection with, as `(data, face index)`. +/// `None` when it is turned off, the collection isn't one of the four, or nothing fits. +pub(crate) fn fallback(query: &FallbackFontQuery) -> Option<(FontData, u32)> { + if cfg!(target_arch = "wasm32") || std::env::var_os("PDFCRAFT_SYSTEM_FONTS").is_some_and(|v| v == "0") { + return None; + } + let script = Script::of(&query.character_collection.as_ref()?.family)?; + let bold = query.is_bold || query.font_weight >= 600; + let family = ChineseFamily::of(query.post_script_name.as_deref().unwrap_or_default()); + /// A face file and the index of the chosen face in it, or `None` when nothing fits. + type Face = Option<(Arc>, u32)>; + type Faces = HashMap<(Script, bool, ChineseFamily), Face>; + static CACHE: Mutex> = Mutex::new(None); + let mut cache = CACHE.lock().unwrap_or_else(PoisonError::into_inner); + let found = cache.get_or_insert_with(HashMap::new).entry((script, bold, family)).or_insert_with(|| { + candidates(script, family, bold).iter().find_map(|c| { + read(c, script.probe(), bold).inspect(|(_, index)| { + log::info!("CJK font fallback for {script:?}/{family:?}: {} (face {index})", c.path.display()); + }) + }) + }); + found.as_ref().map(|(bytes, index)| (Arc::clone(bytes) as FontData, *index)) +} + +/// Read a candidate through the bounded file cache and select a face that maps `probe`, +/// preferring the requested region and weight. Unreadable files and unsuitable faces return +/// `None`, allowing the caller to try the next candidate without changing the PDF. +fn read(candidate: &Candidate, probe: char, bold: bool) -> Option<(Arc>, u32)> { + let bytes = file_bytes(&candidate.path)?; + let index = pick_face(&bytes, probe, candidate.region, bold)?; + Some((bytes, index)) +} + +/// The bytes of an installed font file, read once for the whole process and shared: a collection +/// such as `PingFang.ttc` is tens of megabytes and is the candidate of several scripts and both +/// weights, so the per-script cache keeps only a handle and a face index. A file that cannot be +/// used is remembered too, so it is not tried again for every script or family. +fn file_bytes(path: &Path) -> Option>> { + /// The bytes of each file tried, or `None` for one that cannot be used. + type Files = HashMap>>>; + static FILES: Mutex> = Mutex::new(None); + let mut files = FILES.lock().unwrap_or_else(PoisonError::into_inner); + files.get_or_insert_with(HashMap::new).entry(path.to_path_buf()).or_insert_with(|| read_file(path, MAX_BYTES).map(Arc::new)).clone() +} + +/// A regular file of at most `max` bytes, or `None`: a font path is untrusted input. +fn read_file(path: &Path, max: u64) -> Option> { + // Reject special files before opening them (in particular a FIFO can block on open). + let meta = std::fs::metadata(path).ok()?; + if !meta.is_file() || meta.len() > max { + return None; + } + let file = std::fs::File::open(path).ok()?; + let meta = file.metadata().ok()?; + if !meta.is_file() || meta.len() > max { + return None; + } + // Bound the actual read too: the file can grow after the metadata check. + let mut bytes = Vec::new(); + file.take(max.saturating_add(1)).read_to_end(&mut bytes).ok()?; + (bytes.len() as u64 <= max).then_some(bytes) +} + +/// The best face of the file that parses and maps `probe`: the region's face when the file has +/// one, then the regular (or, for bold text, the semibold or bold) weight, then the first. +fn pick_face(bytes: &[u8], probe: char, region: &[&str], bold: bool) -> Option { + let mut best: Option<(u32, u32)> = None; + for index in 0..MAX_FACES { + let Ok(font) = skrifa::FontRef::from_index(bytes, index) else { continue }; + if font.charmap().map(probe).is_none() { + continue; + } + let names = |id: StringId| font.localized_strings(id).map(|s| s.chars().collect::()).collect::>(); + let family = names(StringId::FAMILY_NAME).join(" "); + let style = names(StringId::SUBFAMILY_NAME).join(" ").to_ascii_lowercase(); + let mut score = 1; + if region.iter().any(|r| family.split_whitespace().any(|w| w == *r)) { + score += 4; + } + let weighty = style.contains("bold") || style.contains("semibold") || style.contains("heavy"); + let plain = style.is_empty() || style.contains("regular") || style.contains("normal") || style.contains("medium") && !bold; + if (bold && weighty) || (!bold && plain && !weighty) { + score += 2; + } + if best.is_none_or(|(_, s)| score > s) { + best = Some((index, score)); + } + } + best.map(|(index, _)| index) +} + +/// List installed-font paths in preference order, with regional face-name hints for collections. +/// Windows Simplified Chinese first tries the named document family; its generic list also +/// prefers a bold face for bold requests. macOS includes bounded downloaded-font locations. +/// Only fixed platform filenames are used: a PDF font name never becomes a filesystem path. +fn candidates(script: Script, family: ChineseFamily, bold: bool) -> Vec { + let region: &'static [&'static str] = match script { + Script::Japanese => &["JP", "Japanese"], + Script::SimplifiedChinese => &["SC", "GB"], + Script::TraditionalChinese => &["TC", "HK", "CNS"], + Script::Korean => &["KR", "Korean"], + }; + let files: Vec = if cfg!(windows) { + let dir = windows_fonts(); + let names: &[&str] = match script { + Script::Japanese => &["YuGothR.ttc", "meiryo.ttc", "msgothic.ttc"], + Script::SimplifiedChinese => match family { + ChineseFamily::Song => &["simsun.ttc", "msyh.ttc", "msjh.ttc"], + ChineseFamily::Hei => &["simhei.ttf", "msyh.ttc", "simsun.ttc", "msjh.ttc"], + ChineseFamily::Kai => &["simkai.ttf", "simsun.ttc", "msyh.ttc", "msjh.ttc"], + ChineseFamily::FangSong => &["simfang.ttf", "simsun.ttc", "msyh.ttc", "msjh.ttc"], + ChineseFamily::Default if bold => &["msyhbd.ttc", "msyh.ttc", "simsun.ttc", "msjh.ttc"], + ChineseFamily::Default => &["msyh.ttc", "simsun.ttc", "msjh.ttc"], + }, + Script::TraditionalChinese => &["msjh.ttc", "mingliu.ttc", "msyh.ttc"], + Script::Korean => &["malgun.ttf", "gulim.ttc"], + }; + names.iter().map(|f| dir.join(f)).collect() + } else if cfg!(target_os = "macos") { + let names: &[&str] = match script { + Script::Japanese => &["ヒラギノ角ゴシック W3.ttc", "Hiragino Sans GB.ttc", "PingFang.ttc"], + Script::SimplifiedChinese => &["PingFang.ttc", "Hiragino Sans GB.ttc", "STHeiti Light.ttc"], + Script::TraditionalChinese => &["PingFang.ttc", "STHeiti Light.ttc", "Hiragino Sans GB.ttc"], + Script::Korean => &["AppleSDGothicNeo.ttc", "AppleGothic.ttf", "PingFang.ttc"], + }; + // The system folder first, then the downloaded font assets: on recent macOS a face such as + // PingFang lives in `/System/Library/AssetsV2/com_apple_MobileAsset_Font*/.asset/AssetData`. + let mut dirs = vec![PathBuf::from("/System/Library/Fonts")]; + dirs.extend(macos_font_asset_dirs().iter().cloned()); + names.iter().flat_map(|f| dirs.iter().map(move |d| d.join(f))).collect() + } else { + let names: &[&str] = match script { + Script::Korean => { + &["truetype/nanum/NanumGothic.ttf", "nanum/NanumGothic.ttf", "truetype/baekmuk/gulim.ttf", "truetype/unfonts-core/UnDotum.ttf"] + } + Script::Japanese => &[ + "truetype/fonts-japanese-gothic.ttf", + "ipa-gothic/ipag.ttf", + "truetype/takao-gothic/TakaoGothic.ttf", + "truetype/vlgothic/VL-Gothic-Regular.ttf", + ], + Script::SimplifiedChinese | Script::TraditionalChinese => &[ + "truetype/wqy/wqy-microhei.ttc", + "wenquanyi/wqy-microhei/wqy-microhei.ttc", + "wqy-microhei/wqy-microhei.ttc", + "truetype/wqy/wqy-zenhei.ttc", + "wenquanyi/wqy-zenhei/wqy-zenhei.ttc", + "wqy-zenhei/wqy-zenhei.ttc", + "truetype/droid/DroidSansFallbackFull.ttf", + "google-droid-sans-fonts/DroidSansFallbackFull.ttf", + "truetype/arphic/uming.ttc", + "arphic-uming/uming.ttc", + ], + }; + ["/usr/share/fonts", "/usr/local/share/fonts"].iter().flat_map(|dir| names.iter().map(move |f| Path::new(dir).join(f))).collect() + }; + files.into_iter().map(|path| Candidate { path, region }).collect() +} + +/// The `AssetData` folders of the downloaded font assets of macOS, found once. The ids in their +/// names change with each asset version, so they are listed rather than named; at most +/// [`MAX_ASSETS`] are looked at. +fn macos_font_asset_dirs() -> &'static [PathBuf] { + static DIRS: OnceLock> = OnceLock::new(); + DIRS.get_or_init(|| { + let root = Path::new("/System/Library/AssetsV2"); + let Ok(kinds) = std::fs::read_dir(root) else { return Vec::new() }; + let mut dirs = Vec::new(); + for kind in kinds.flatten().filter(|k| k.file_name().to_string_lossy().starts_with("com_apple_MobileAsset_Font")) { + let Ok(assets) = std::fs::read_dir(kind.path()) else { continue }; + for asset in assets.flatten().take(MAX_ASSETS) { + let data = asset.path().join("AssetData"); + if data.is_dir() { + dirs.push(data); + } + } + } + dirs + }) +} + +/// Locate the Windows Fonts directory using `WINDIR`, then `SystemRoot`, then `C:\Windows`. +/// This only constructs the path; candidate reads still enforce regular-file and size limits. +fn windows_fonts() -> PathBuf { + let dir = std::env::var_os("WINDIR").or_else(|| std::env::var_os("SystemRoot")).map_or_else(|| PathBuf::from(r"C:\Windows"), PathBuf::from); + dir.join("Fonts") +} + +#[cfg(test)] +mod tests { + use super::*; + + const SCRIPTS: [Script; 4] = [Script::Japanese, Script::SimplifiedChinese, Script::TraditionalChinese, Script::Korean]; + + /// Standard, localized and subset-prefixed names retain their Chinese family hint; + /// unrelated Latin names and generic YaHei requests use the default candidate list. + #[test] + fn chinese_family_hints_include_subset_and_localized_names() { + for name in ["SimSun", "ABCDEF+SimSun", "STSong-Light", "宋体"] { + assert_eq!(ChineseFamily::of(name), ChineseFamily::Song, "{name}"); + } + for name in ["SimHei", "STHeiti", "黑体"] { + assert_eq!(ChineseFamily::of(name), ChineseFamily::Hei, "{name}"); + } + for name in ["KaiTi", "STKaiti", "楷体"] { + assert_eq!(ChineseFamily::of(name), ChineseFamily::Kai, "{name}"); + } + assert_eq!(ChineseFamily::of("Helvetica"), ChineseFamily::Default); + assert_eq!(ChineseFamily::of("MicrosoftYaHei"), ChineseFamily::Default); + } + + /// FangSong names contain "song" and must not become Song; Founder (方正) names in CNKI papers + /// pick the matching installed family instead of YaHei for everything (#3587). + #[test] + fn fangsong_and_founder_names_pick_their_family() { + for name in ["FangSong", "FangSong_GB2312", "STFangsong", "SimFang", "仿宋", "FZFSK--GBK1-0", "ABCDEF+FZFSK--GBK1-0"] { + assert_eq!(ChineseFamily::of(name), ChineseFamily::FangSong, "{name}"); + } + for name in ["FZSSK--GBK1-0", "FZBSK--GBK1-0", "FZXBSK--GBK1-0", "FZShuSong-Z01", "STZhongsong", "NSimSun", "ABCDEF+FZSSK--GBK1-0"] { + assert_eq!(ChineseFamily::of(name), ChineseFamily::Song, "{name}"); + } + for name in ["FZHTK--GBK1-0", "FZLTHJW--GB1-0", "STXihei", "ABCDEF+FZHTK--GBK1-0"] { + assert_eq!(ChineseFamily::of(name), ChineseFamily::Hei, "{name}"); + } + for name in ["FZKTK--GBK1-0", "KaiTi_GB2312", "ABCDEF+FZKTK--GBK1-0"] { + assert_eq!(ChineseFamily::of(name), ChineseFamily::Kai, "{name}"); + } + } + + /// Every family has a Windows list that ends in faces any Chinese Windows has, so a missing + /// named face still draws the text. + #[cfg(windows)] + #[test] + fn every_windows_chinese_family_falls_back_to_common_faces() { + for family in [ChineseFamily::Default, ChineseFamily::Song, ChineseFamily::Hei, ChineseFamily::Kai, ChineseFamily::FangSong] { + let list = candidates(Script::SimplifiedChinese, family, false); + assert!(list.iter().any(|c| c.path.ends_with("msyh.ttc")), "{family:?}"); + } + let fangsong = candidates(Script::SimplifiedChinese, ChineseFamily::FangSong, false); + assert!(fangsong.first().is_some_and(|c| c.path.ends_with("simfang.ttf"))); + } + + /// Invalid font bytes, missing paths and directories produce no candidate face rather + /// than preventing the resolver from trying a later installed font. + #[test] + fn broken_and_missing_files_are_skipped() { + assert_eq!(pick_face(b"", '\u{7269}', &[], false), None); + assert_eq!(pick_face(b"not a font at all", '\u{7269}', &[], false), None); + assert_eq!(pick_face(&[0u8; 4096], '\u{7269}', &[], false), None); + let missing = Candidate { path: PathBuf::from("definitely/not/here.ttc"), region: &[] }; + assert!(read(&missing, '\u{7269}', false).is_none()); + // A directory is not a font. + let dir = Candidate { path: std::env::temp_dir(), region: &[] }; + assert!(read(&dir, '\u{7269}', false).is_none()); + } + + /// The files are bounded by `max`: a larger one is not read. + #[test] + fn a_file_over_the_cap_is_not_read() { + let inter = Path::new(concat!(env!("CARGO_MANIFEST_DIR"), "/../../assets/fonts/Inter-Regular.ttf")); + assert!(read_file(inter, MAX_BYTES).is_some()); + assert!(read_file(inter, 1024).is_none()); + } + + /// Two faces drawn from one installed file (two scripts, or regular and bold) hold the same + /// bytes, not a copy each; a file that can't be used is remembered as unusable. + #[test] + fn one_installed_file_is_read_once_and_shared() { + let inter = Path::new(concat!(env!("CARGO_MANIFEST_DIR"), "/../../assets/fonts/Inter-Regular.ttf")); + let first = file_bytes(inter).expect("readable"); + let second = file_bytes(inter).expect("readable"); + assert!(Arc::ptr_eq(&first, &second), "one allocation for the file"); + let missing = Path::new("definitely/not/here.ttc"); + assert!(file_bytes(missing).is_none() && file_bytes(missing).is_none()); + } + + /// A Latin-only face fails every CJK probe even though its Latin glyph remains readable, + /// distinguishing unsupported script coverage from an unparseable font file. + #[test] + fn a_latin_face_is_rejected_for_every_script() { + let inter = include_bytes!("../../../assets/fonts/Inter-Regular.ttf"); + for script in SCRIPTS { + assert_eq!(pick_face(inter, script.probe(), &[], false), None, "{script:?}"); + } + assert_eq!(pick_face(inter, 'A', &[], false), Some(0)); + } + + /// Platform candidates are absolute TrueType paths and exclude the font families barred + /// by the asset policy, independently of whether those files are installed locally. + #[test] + fn candidates_are_absolute_font_files_without_adobe_designs() { + for script in SCRIPTS { + let list = candidates(script, ChineseFamily::Default, false); + assert!(!list.is_empty(), "{script:?}"); + for c in list { + assert!(c.path.is_absolute(), "{}", c.path.display()); + assert!(c.path.extension().is_some_and(|e| e == "ttf" || e == "ttc"), "{}", c.path.display()); + let name = c.path.to_string_lossy().to_lowercase(); + assert!( + !["notosanscjk", "notoserifcjk", "sourcehan", "source-han", "adobe", "kozuka", "minion"].iter().any(|b| name.contains(b)), + "{name}" + ); + } + } + } + + /// The four regional Adobe collections select their corresponding scripts; Identity + /// carries no regional information and must not acquire an installed fallback by guess. + #[test] + fn only_the_four_adobe_collections_have_a_script() { + assert_eq!(Script::of(&CidFamily::AdobeCNS1), Some(Script::TraditionalChinese)); + assert_eq!(Script::of(&CidFamily::AdobeGB1), Some(Script::SimplifiedChinese)); + assert_eq!(Script::of(&CidFamily::AdobeKorea1), Some(Script::Korean)); + assert_eq!(Script::of(&CidFamily::AdobeJapan1), Some(Script::Japanese)); + assert_eq!(Script::of(&CidFamily::AdobeIdentity), None); + } + + /// A real Windows GBK book names these faces. Replacing both with YaHei hides the + /// distinction between serif body text and sans headings even when the exact face exists. + #[cfg(windows)] + #[test] + fn windows_preserves_installed_chinese_document_families() { + use hayro::hayro_interpret::hayro_cmap::CharacterCollection; + for (name, expected, file) in [ + ("SimSun", "SimSun", "simsun.ttc"), + ("SimHei", "SimHei", "simhei.ttf"), + ("KaiTi", "KaiTi", "simkai.ttf"), + ("FangSong", "FangSong", "simfang.ttf"), + ("ABCDEF+FZSSK--GBK1-0", "SimSun", "simsun.ttc"), + ("ABCDEF+FZHTK--GBK1-0", "SimHei", "simhei.ttf"), + ("ABCDEF+FZKTK--GBK1-0", "KaiTi", "simkai.ttf"), + ("ABCDEF+FZFSK--GBK1-0", "FangSong", "simfang.ttf"), + ] { + if !windows_fonts().join(file).is_file() { + continue; + } + let query = FallbackFontQuery { + post_script_name: Some(name.into()), + character_collection: Some(CharacterCollection { family: CidFamily::AdobeGB1, supplement: 4 }), + ..FallbackFontQuery::default() + }; + let (bytes, index) = fallback(&query).expect("installed Chinese font"); + let font = skrifa::FontRef::from_index(bytes.as_ref().as_ref(), index).unwrap(); + let names: Vec = font.localized_strings(StringId::POSTSCRIPT_NAME).map(|n| n.chars().collect()).collect(); + assert!(names.iter().any(|n| n.eq_ignore_ascii_case(expected)), "{name} resolved to {names:?}, expected {expected}"); + } + } + + /// On a Mac the face for Traditional Chinese is the TC one of PingFang, not the first of the + /// collection (which is another region's), and it maps a character only Traditional has. + #[cfg(target_os = "macos")] + #[test] + fn macos_picks_the_regional_pingfang_face() { + // Found where this macOS keeps it: the system folder, or the downloaded font assets. + let path = candidates(Script::TraditionalChinese, ChineseFamily::Default, false) + .into_iter() + .map(|c| c.path) + .find(|p| p.file_name().is_some_and(|n| n == "PingFang.ttc") && p.is_file()); + let Some(bytes) = path.and_then(|p| std::fs::read(p).ok()) else { + eprintln!("PingFang is not installed here: skipped"); + return; + }; + for (script, region) in [(Script::TraditionalChinese, "TC"), (Script::SimplifiedChinese, "SC")] { + let index = pick_face(&bytes, script.probe(), &[region], false).expect("PingFang covers it"); + let font = skrifa::FontRef::from_index(&bytes, index).unwrap(); + let family: String = font.localized_strings(StringId::FAMILY_NAME).next().unwrap().chars().collect(); + assert!(family.contains(region), "{script:?} got {family:?}"); + } + } + /// A face that this macOS keeps only in its downloaded font assets (PingFang, on recent + /// versions) is among the candidates. The assets are searched here on their own, not by the + /// code under test; where there are none, nothing is checked. + #[cfg(target_os = "macos")] + #[test] + fn macos_candidates_include_a_pingfang_that_lives_in_the_font_assets() { + /// Independently locate an installed asset by filename within the test's depth limit, + /// providing an oracle for the production candidate list without using its discovery. + fn find(dir: &Path, name: &str, depth: u32) -> Option { + for entry in std::fs::read_dir(dir).ok()?.flatten() { + let path = entry.path(); + if path.file_name().is_some_and(|n| n == name) && path.is_file() { + return Some(path); + } + if depth > 0 + && path.is_dir() + && !path.is_symlink() + && let Some(found) = find(&path, name, depth - 1) + { + return Some(found); + } + } + None + } + let Some(asset) = find(Path::new("/System/Library/AssetsV2"), "PingFang.ttc", 3) else { + eprintln!("no PingFang in the font assets here: skipped"); + return; + }; + for script in [Script::SimplifiedChinese, Script::TraditionalChinese] { + assert!( + candidates(script, ChineseFamily::Default, false).iter().any(|c| c.path == asset), + "{script:?}: {} is not a candidate", + asset.display() + ); + } + } +} diff --git a/crates/ui-egui/src/workflows/a11y_ui.rs b/crates/ui-egui/src/workflows/a11y_ui.rs index 9081e5cb32..3de932f504 100644 --- a/crates/ui-egui/src/workflows/a11y_ui.rs +++ b/crates/ui-egui/src/workflows/a11y_ui.rs @@ -542,7 +542,9 @@ fn figure_picture(ctx: &egui::Context, doc: &pdfcraft_engine::Document, f: &pdfc w: (w * scale).ceil().max(1.0) as u32, h: (h * scale).ceil().max(1.0) as u32, }; - let config = pdfcraft_render::RenderConfig { password: doc.password.as_deref().map(std::sync::Arc::from), ..Default::default() }; + // The picture is only shown in the panel, so it draws like the viewer (installed CJK faces + // for fonts the PDF doesn't embed); nothing is saved from it (AGENTS.md §1.4). + let config = pdfcraft_render::RenderConfig { password: doc.password.as_deref().map(std::sync::Arc::from), ..Default::default() }.on_screen(); let mut r = pdfcraft_render::PageRenderer::new(doc.bytes.clone(), config); let out = r.render(pdfcraft_render::RenderRequest { page, diff --git a/crates/ui-egui/src/workflows/combine_grid.rs b/crates/ui-egui/src/workflows/combine_grid.rs index fab14b8ddc..47851b443b 100644 --- a/crates/ui-egui/src/workflows/combine_grid.rs +++ b/crates/ui-egui/src/workflows/combine_grid.rs @@ -571,5 +571,12 @@ impl PdfCraftApp { } } +/// The render configuration of a Combine thumbnail. Thumbnails are only shown on screen, so +/// installed CJK faces may draw fonts a PDF doesn't embed, as in the viewer; the combined file +/// itself is written from the original bytes and never from these rasters (AGENTS.md §1.4). +fn thumbnail_config(password: Option>) -> RenderConfig { + RenderConfig { password, ..Default::default() }.on_screen() +} + #[cfg(test)] mod tests; diff --git a/crates/ui-egui/src/workflows/combine_grid/tests.rs b/crates/ui-egui/src/workflows/combine_grid/tests.rs index a40817c044..186cb5b3a4 100644 --- a/crates/ui-egui/src/workflows/combine_grid/tests.rs +++ b/crates/ui-egui/src/workflows/combine_grid/tests.rs @@ -343,3 +343,13 @@ fn on_the_web_a_render_that_never_answers_fails_and_lets_the_rest_through() { assert!(thumbs.jobs.is_empty()); assert!(frame(&ctx, &mut thumbs, ¤t) > std::time::Duration::ZERO, "no more frames asked for at once"); } + +/// Combine thumbnails are on-screen rasters: like the viewer, they may draw a non-embedded +/// CJK font with an installed face, and they keep the file's password. +#[test] +fn thumbnails_use_the_on_screen_configuration() { + let config = thumbnail_config(Some(Arc::from("secret"))); + assert!(config.system_fonts, "Combine thumbnails draw non-embedded CJK text like the viewer"); + assert_eq!(config.password.as_deref(), Some("secret")); + assert!(!RenderConfig::default().system_fonts, "anything else stays off by default"); +} diff --git a/crates/ui-egui/src/workflows/combine_grid/thumbs.rs b/crates/ui-egui/src/workflows/combine_grid/thumbs.rs index 28c54b4bef..1de9ad05b6 100644 --- a/crates/ui-egui/src/workflows/combine_grid/thumbs.rs +++ b/crates/ui-egui/src/workflows/combine_grid/thumbs.rs @@ -1,4 +1,4 @@ -use super::{Arc, HashMap, RenderConfig, RenderPool, RenderRequest, Retiring, TextureHandle}; +use super::{Arc, HashMap, RenderPool, RenderRequest, Retiring, TextureHandle, thumbnail_config}; /// Render threads for thumbnails at once, counting those of retired pools until they exit. pub(super) const MAX_THREADS: usize = 3; @@ -319,7 +319,7 @@ impl Thumbs { waiting = true; break; } - let config = RenderConfig { password: w.password, ..Default::default() }; + let config = thumbnail_config(w.password); #[cfg(test)] let pool = if self.inline { RenderPool::new_inline(w.bytes, config) } else { RenderPool::new(w.bytes, 1, config).without_replacements() }; #[cfg(not(test))] diff --git a/docs/development.md b/docs/development.md index e50907ba56..8cb132d9f8 100644 --- a/docs/development.md +++ b/docs/development.md @@ -154,7 +154,7 @@ copies retain their current preferences locations. | `PDFCRAFT_TITLEBAR` | Linux: `app` draws PdfCraft's own title bar (window buttons in the tab strip, in the order of GNOME's `button-layout`, and resize edges); `system` keeps the frame winit or the desktop draws; unset, PdfCraft draws its own when the Wayland compositor draws no frames (GNOME), and leaves it to the desktop otherwise (KDE, Sway, Hyprland, X11). Where PdfCraft draws it, Preferences ▸ Use system title bar switches back to winit's frame without a restart; `system` also hides that option | | `PDFCRAFT_BUILD_LABEL` | Build time: a label for a local or patched build (letters, digits, inner dots), shown after the version in `--version`, About and the log (`0.7.1 (titlebar)`). `packaging/linux/package.sh` also puts it in the file names and the .rpm/.deb release (`1.titlebar`), which upgrades over the release of the same version | | `CRAFT_FONTS_DIR` | Build time: a [craft-fonts](https://github.com/storytold/craft-fonts) checkout to embed (Japanese fonts) | -| `PDFCRAFT_SYSTEM_FONTS` | `0` stops the desktop app from using an installed font for characters its embedded fonts lack (`cargo xtask screenshots` sets it) | +| `PDFCRAFT_SYSTEM_FONTS` | `0` disables installed-font fallback for both interface text and non-embedded CJK page text in the desktop viewer, Combine thumbnails and accessibility figure previews (`cargo xtask screenshots` sets it). Document fallback is display-only: exports, printing, OCR and automation retain the bundled-font path. | | `FONTCONFIG_FILE` | On Linux/Unix, the Fontconfig configuration file used to discover installed UI fallback fonts (default: `/etc/fonts/fonts.conf`); relative filenames are resolved under `FONTCONFIG_PATH` | | `FONTCONFIG_PATH` | Directory for a relative `FONTCONFIG_FILE` (default: `/etc/fonts`; the first entry of a search path is used) | diff --git a/docs/roadmap.md b/docs/roadmap.md index ab55c052c2..12161c4dff 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,5 +1,13 @@ # PdfCraft roadmap: milestones and what's next +> **Last reviewed:** 2026-10-31 | **Last updated:** 2026-10-31 | **Change:** patch (M2, PR #4172: GBK rasters compared with explicit Unicode and independently selected installed faces; wrong-glyph and wrong-family negative controls fail; estimates unchanged) +> **Last reviewed:** 2026-10-31 | **Last updated:** 2026-10-31 | **Change:** patch (M2, PR #4172: integrate main fa5f9f8c; retain CJK viewer and upstream raster regressions, restore explicit display_config imports; estimates unchanged) +> **Last reviewed:** 2026-10-28 · **Last updated:** 2026-10-28 · **Change:** patch (M2, PR #4172: integrate main 5af10276; retain both roadmap histories and viewer-only CJK font policy; estimates unchanged) +> **Last reviewed:** 2026-10-27 · **Last updated:** 2026-10-27 · **Change:** patch (M2, PR #4172: integrate main 1dd2737b; preserve viewer-only CJK fallback and regression coverage across engine, renderer and Combine module splits; estimates unchanged) +> **Last reviewed:** 2026-10-27 · **Last updated:** 2026-10-27 · **Change:** patch (M2, PR #4172: integrate main 9a9813bf and migrate viewer-only CJK configuration to the split engine modules; estimates unchanged) +> **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: document installed-font helpers and regression contracts; behavior and estimates unchanged) +> **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: print-preview font policy matches output, with isolated caches and GBK switching regressions; estimates unchanged) +> **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: reconcile current main and retain Japanese serif/weight selection alongside display-only CJK fallback; estimates unchanged) > **Last reviewed:** 2026-10-29 | **Last updated:** 2026-10-29 | **Change:** minor (M7 #2517: a narrowed then widened paragraph keeps the typed spaces the wrap consumed, after save and reopen; estimates unchanged) > **Last reviewed:** 2026-10-27 | **Last updated:** 2026-10-27 | **Change:** minor (M10: OCR model downloads use verified ONNX pairs with legacy discovery fallback; estimates unchanged) > **Last reviewed:** 2026-10-25 | **Last updated:** 2026-10-25 | **Change:** minor (M5: one reply in the Comments panel is edited, copied or deleted from its own "…" menu; estimates unchanged) @@ -13,24 +21,29 @@ > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M6: the calculation order flags a formula that reads a field the form doesn't have and a script that builds the names it reads; the tools show each formula; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M7, #2636 partial: pasting added text boxes and images in the engine and through `content_copy`; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M11: image optimization accounts for page-local UserUnit; estimates unchanged) - +> > **Last reviewed:** 2026-10-23 | **Last updated:** 2026-10-23 | **Change:** minor (M10: Word table cells use the widths of their own grid columns; estimates unchanged) > **Last reviewed:** 2026-10-23 | **Last updated:** 2026-10-23 | **Change:** minor (M10: table spans retain text from covered slots in Office exports; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M10: image export preserves soft-mask Decode opacity ranges; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M10: vector printing preserves existing page XObjects when adding annotation appearances; estimates unchanged) +> +> **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M2, PR #4172: CJK fallback in Combine thumbnails and figure previews; FangSong and subset-prefixed Founder family matching; estimates unchanged) +> +> **Last reviewed:** 2026-10-22 · **Last updated:** 2026-10-22 · **Change:** minor (M14, #3558: rebase guarded macOS per-file association recovery onto current main; estimates unchanged) +> **Last reviewed:** 2026-10-20 · **Last updated:** 2026-10-20 · **Change:** minor (M12 #3673: Character encoding accepts Identity-H/V with Adobe CJK collections; estimates unchanged) > **Last reviewed:** 2026-10-22 · **Last updated:** 2026-10-22 · **Change:** minor (M7 #2517: a partial rewrite inside one text object closes its marker before the next paragraph; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) > **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M7 #2517: a blank line stays a paragraph break; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) - +> > **Last reviewed:** 2026-10-20 | **Last updated:** 2026-10-20 | **Change:** minor (M10, #3532 partial: vector printing retains existing appearances with absent Subtype; estimates unchanged) - +> > **Last reviewed:** 2026-10-20 · **Last updated:** 2026-10-20 · **Change:** minor (M4: initial-view options resolve indirect boolean preferences; estimates unchanged) > **Last reviewed:** 2026-10-21 · **Last updated:** 2026-10-21 · **Change:** minor (M3, #3808: the Pages inspector has an independent 50–150% thumbnail size slider and buttons; selection and current-page following are retained, with headless ui.set coverage. Estimates unchanged.) - +> > **Last reviewed:** 2026-10-20 | **Last updated:** 2026-10-20 | **Change:** minor (M2, #3574: named form properties retain page origin and own dictionary boundaries; estimates unchanged) - +> > **Last reviewed:** 2026-10-20 · **Last updated:** 2026-10-20 · **Change:** minor (M3, #3541/#3620: comment hover follows individual marked lines and original annotation paint order; estimates unchanged) > **Last reviewed:** 2026-10-20 · **Last updated:** 2026-10-20 · **Change:** patch (PR #3348 review: retain short tracked punctuation in gap estimates; estimates unchanged) - +> > **Last reviewed:** 2026-10-20 · **Last updated:** 2026-10-20 · **Change:** minor (M2, #3601: missing spacing glyphs in non-embedded Japanese CID substitutes draw as blanks while PDF widths and Unicode stay unchanged; estimates unchanged) > **Last reviewed:** 2026-10-20 | **Last updated:** 2026-10-20 | **Change:** minor (M11 #3507: image merging preserves content stream boundaries; estimates unchanged) > **Last reviewed:** 2026-10-17 · **Last updated:** 2026-10-17 · **Change:** minor (M8, gap 6 #2662: the redaction audit counts a cover over one word of a long line; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) @@ -38,7 +51,7 @@ > **Last reviewed:** 2026-10-22 · **Last updated:** 2026-10-22 · **Change:** minor (M6: calculation scripts read their notation only from a comment, and a regular expression after a condition as one; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) > **Last reviewed:** 2026-10-18 · **Last updated:** 2026-10-18 · **Change:** minor (M5, PR #1971: bounded nested FDF export excludes source page/page-tree dictionaries; refreshed against upstream 4db6ac4a; estimates unchanged) > **Last reviewed:** 2026-10-18 · **Last updated:** 2026-10-18 · **Change:** patch (M2 #3086: dotted TOC leaders no longer hide word spaces; estimates unchanged) - +> > **Last reviewed:** 2026-10-18 · **Last updated:** 2026-10-18 · **Change:** minor (M3, PR #3101: optional gapless page display with translations in all 22 catalogs; organizer and comparison toggles corrected; estimates unchanged) > **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M14/#2383 GPU-limit candidate; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) > **Last reviewed:** 2026-10-20 - **Last updated:** 2026-10-20 - **Change:** minor (M6, #3502: form detection uses the existing content-stream boundary policy; estimates unchanged) @@ -53,7 +66,7 @@ > **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M1: raw-deflate fallback preserves decoding-limit errors; refreshed upstream; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) > **Last reviewed:** 2026-10-14 · **Last updated:** 2026-10-15 · **Change:** minor (M2: objects in one large object stream are read in linear time; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) > **Last reviewed:** 2026-10-16; **Last updated:** 2026-10-16; **Change:** minor (M5 #2669, item 5: reply groups and page root counts per frame; estimates unchanged) - +> > **Last reviewed:** 2026-10-13 · **Last updated:** 2026-10-14 · **Change:** minor (M14: the Italian catalog has every entry the other catalogs had and the code looks up, 2,528 entries; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) > **Last reviewed:** 2026-10-16 | **Last updated:** 2026-10-16 | **Change:** minor (M10 #1201 partial: lossless image Decode; estimates unchanged) | **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) > **Last reviewed:** 2026-10-12 · **Last updated:** 2026-10-13 · **Change:** minor (M2/M6: spot-colour images convert once per pixel value; widget placement and the Fields panel scale with the field count; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) @@ -161,6 +174,14 @@ Help ▸ Check for updates is manual only. ## Milestones +M2 update (2026-10-23, #234 / continuation of #1567): the desktop viewer uses installed CJK +faces for non-embedded CID fonts, including Windows GBK-EUC-H books. Windows Song/Hei/Kai +families are retained when installed, and reload retains the fallback. This is display-only; +export, printing, OCR, automation and machines without a candidate retain their limitations. +Print preview excludes installed faces and isolates its font cache from the viewer, matching +output font availability even after repeated view/print switching (2026-10-26, PR #4172). +The M2 readiness and remaining-work estimates below are unchanged. + "Plan size" is the original single-agent sizing. "Done" is the estimated share of that milestone's *acceptance criteria* that are met, not lines of code. "Remaining" is re-measured on 2026-10-10 at the measured rate (see [target-app-parity.md](target-app-parity.md#effort-and-calibration)); the @@ -170,7 +191,7 @@ total matches the parity estimate. |---|---|---|---|---|---| | M0 | Skeleton: workspace, xtask gates, CI | 15–30 | 98% | 1–2 | GitHub workflow, `deny.toml` (licence audit of every dependency), parity checklist (826 features, `xtask parity`) done. Missing: remaining crate stubs, testkit/oracle crates | | M1 | COS: filters, crypt, parser, xref, writer | 120–200 | 80% | 20–40 | Done:
- filters and crypt: every standard-security revision R2–R6 (RC4, AES-128/256), SASLprep, permissions, creating encryption;
- cos: parse and repair (including invalid xref object-number ranges), decoded-stream limits including unfiltered data, decrypt on load, re-encrypt on save, incremental and full writing.
Corpus: open/edit/save passes on 958 files, and all 7 password-protected files open.
Full saves now pack objects into compressed object streams. An unchanged incremental save of a classic table writes a zero-length cross-reference subsection instead of a section with no subsections (#1339). A full rewrite keeps unknown trailer entries, including objects reachable only from them (#1338). Fuzzing runs nightly (`xtask fuzz`). Raw-deflate fallback preserves decoding-limit errors, including through tolerant COS stream decoding. Done since: #2658: macOS atomic saves retain destination extended attributes, including Finder tags, and reject partial resource-fork reads after size changes. Metadata failures retain the original PDF and unsaved edit. Missing: ≥ 250 tests, own image codecs | -| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. Forms without Resources use page Properties for ActualText and layer visibility; own dictionaries stop fallback. Oversized missing inline-image samples, including header and terminator overhead, no longer pin the content window (#3107 / PR #3438); normal large streamed images retain deterministic sample boundaries. Radial luminosity fades have synthetic coverage for the existing #2132 mask guard, including per-pixel references, antialiased clips, previews, scales and tiles; no additional production fix was needed. Dotted TOC leaders no longer inflate letter tracking or hide word spaces (#3086); phrase search and selection are covered. Short tracked ellipses retain their gap samples and remain searchable. Missing spacing glyphs in non-embedded Japanese CID substitutes render blank without changing PDF advances or Unicode (#3601). | +| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. Forms without Resources use page Properties for ActualText and layer visibility; own dictionaries stop fallback. Oversized missing inline-image samples, including header and terminator overhead, no longer pin the content window (#3107 / PR #3438); normal large streamed images retain deterministic sample boundaries. Radial luminosity fades have synthetic coverage for the existing #2132 mask guard, including per-pixel references, antialiased clips, previews, scales and tiles; no additional production fix was needed. Dotted TOC leaders no longer inflate letter tracking or hide word spaces (#3086); phrase search and selection are covered. Short tracked ellipses retain their gap samples and remain searchable. Missing spacing glyphs in non-embedded Japanese CID substitutes render blank without changing PDF advances or Unicode (#3601). The desktop viewer can use installed CJK fonts, including Windows GBK Song/Hei/Kai/FangSong and subset-prefixed Founder names; Combine thumbnails and accessibility figure previews use the same display-only fallback, without changing export or print font policy (#234, continuation of #1567, review #4172). Print previews exclude installed faces and isolate their font cache; GBK regressions cover view/print switching for inline and pooled whole-page/tile rendering. Installed-font helpers and their regression contracts have function documentation. GBK regression rasters match explicit-Unicode references with independently selected installed faces; wrong-glyph and wrong-family negative controls fail. Integration with main fa5f9f8c retains viewer fallback across the split engine session modules, renderer tests and Combine thumbnail workers. | | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. The renderer keeps its one-point crop-size floor in physical points after UserUnit scaling, with regression coverage for crop origins, rotations and f32 reciprocal rounding. Existing unit policy and nearly-zero fallback are retained. | | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings | | M3 | Viewer app (native + web) | 80–150 | 90% | 10–20 | Acrobat-style shell, find, select, panels, tiles, web build, UI control channel for agents (opt-in). #3808: the Pages inspector offers a 50–150% thumbnail slider and buttons, also available through `ui.set`. Missing: 60 fps test on a 500-page document, snapshot tests of every panel
Done since: fit visible, document title in the window; Linux middle-button scrolling; shared three-state theme preferences in the toolbar, View menu and control channel; numeric Linux portal events and idle wakeups completing #881, with native appearance inheritance retained (#446); late browser startup downloads preserve the current document and Save target (#171). Selected-text Copy in Edit and the canvas context menu is covered by focused regressions and native clipboard checks (#1225); document copy restrictions apply to the floating action too. #1364 candidate: absolute local GoToR links ask before target access; headless callers explicitly opt in. Native-opening candidate: chosen tools survive preparation and password retries; startup intake waits for pending preparation. Page-label navigation displays logical labels, retains the current page on unchanged submission and refreshes focused same-document replacements; clean file-watcher reload preserves the chosen scrolling mode, preventing a Single Page reader from jumping to page 1 during replacement. Synthetic control regressions cover these behaviors. #3149: page-list changes discard render demand based on old geometry; actual raster dimensions are checked against the GPU limit before upload. Synthetic crash and continued-rendering regressions cover the replacement path. Save-prompt shortcuts yield to dialogs above them; stacked link-consent UI regression covers Escape and Return. Comment hover follows individual marked lines and original annotation paint order (#3541/#3620), independently of spatial inspection sorting; synthetic rotation, differing-upper-edge overlap and unchanged-document regressions. Optional page gaps join page edges without cropping PDF margins; current-document toggle, persisted default and UI control options are covered by viewer tests. Gap changes in Organize Pages preserve document scroll position, and both comparison panes follow menu and control choices. Repeated window resizes by an edge and moves by the tab strip on GNOME/Wayland work without a click in between. | @@ -253,18 +274,25 @@ M2 (renderer), M7 (content editing), M10 (OCR and Office), M11 (Preflight and st ## Revision history +**2026-10-26 · patch:** M2/PR #4172: reconcile current main and preserve upstream Japanese serif/weight selection alongside the installed CJK viewer and preview fallback. Estimates unchanged. + **2026-10-25 · minor:** M14 localization current state: Arabic catalogs stay logical, Arabic/Kurdish labels take an explicit RTL direction, and stock chrome mirrors; native/browser RTL input, IME and complex workflows remain unqualified. Native Simplified Chinese uses LXGW WenKai Lite; the web build has no Hans face. Estimates unchanged. **2026-10-23 · minor:** M11: image downsampling uses physical resolution after page-local `/UserUnit` scaling, including shared nested forms. Synthetic measurement and optimization regressions cover the change. Estimates unchanged. +**2026-10-23 · minor:** M2/PR #4172: display-only CJK fallback covers Combine thumbnails and accessibility figure previews. Windows family matching includes FangSong and subset-prefixed Founder names. Export, print, OCR and automation policy and readiness estimates are unchanged. + **2026-10-20 · minor:** M2/#3601: substitute-font spacing aliases preserve blank ink, original `/W` and `/DW` advances, and Unicode extraction. Synthetic renderer regressions cover the reported Japanese CID fixture. Estimates unchanged. **2026-10-15 · minor:** M14/#2383 candidate: GPU device requests clamp unsupported limits while retaining the adapter texture extent. Synthetic V3D/downlevel regressions and Windows GPU-required tests pass; Raspberry Pi hardware remains unverified. Estimates unchanged. | Date | Change | Summary | |---|---|---| +| 2026-10-31 | patch | M2, PR #4172: eight GBK font/alias cases compare exact pixels with independent installed-face and explicit-Unicode references; wrong-glyph and wrong-family negative controls fail. Estimates unchanged. | +| 2026-10-31 | patch | M2, PR #4172: integrate main fa5f9f8c, preserve both raster test groups and restore explicit display_config imports; display-only CJK policy and estimates unchanged. | +| 2026-10-28 | patch | M2, PR #4172: integrate main 5af10276 and retain both roadmap histories; viewer-only CJK and print/output font policies and estimates unchanged. | | 2026-10-24 | minor | M11: the PDF Optimizer and Reduce File Size store Group 3 fax images as Group 4 when smaller (`Document::fax_params`), and pack eight-bit `DeviceRGB` of only black and white (as grey) and indexed images of 2, 4 or 8 bits that use only palette entries 0 and 1 (keeping the palette) to one bit, so Monochrome Images compresses and downsamples them. Estimates unchanged. | | 2026-10-24 | minor | M11: monochrome downsampling: the PDF Optimizer and Reduce File Size average black-and-white images (1-bit, CCITT fax, eight-bit black-and-white grey) above 450 ppi down to 300 ppi, by area with ties black, as Monochrome Images ▸ Downsample to and `doc_optimize` `mono: { downsample, ppi, above_ppi }`; CCITT decoding moves into `pdfcraft-filters` (`decode_ccitt`) behind `Document::decoded_image`. Estimates unchanged. | | 2026-10-24 | minor | M11: Create PDF stores pictures of only black and white at one bit (Group 4 or Flate, whichever is smaller) instead of eight-bit grey; the optimizer packs eight-bit black-and-white grey images to one bit before Monochrome Images compresses them, unless the grey settings make them smaller, as they can a dithered scan, soft mask counted; Export ▸ Images decodes CCITT fax images. Estimates unchanged. | diff --git a/parity/acrobat-features.toml b/parity/acrobat-features.toml index 1404247cd7..0b6f658dd6 100644 --- a/parity/acrobat-features.toml +++ b/parity/acrobat-features.toml @@ -794,8 +794,8 @@ milestone = "M2" status = "partial" commands = [] tools = [] -evidence = ["crates/render/src/raster/tests.rs::huge_cid_width_ranges_terminate", "crates/render/src/raster/tests.rs::non_embedded_japanese_cid_fonts_draw_with_a_craft_fonts_face", "crates/render/src/raster/tests.rs::mincho_substitute_covers_half_width_katakana", "crates/render/src/text/tests.rs::identity_to_unicode_names_give_text", "crates/fonts/src/pdf/tests.rs::identity_to_unicode_names_read_codes_as_unicode"] -notes = "CID-to-Unicode tables are M2.4; a non-embedded Adobe-Japan1 font is drawn in a craft-fonts Japanese face (Mincho or Gothic by name, else by the descendant font descriptor's Serif flag; bold Gothic for a Bold name or a `/FontWeight` of 600 or more; BIZ UDMincho for Mincho, which covers half-width katakana) when built with craft-fonts; non-embedded GB1, CNS1 and Korea1 fonts still draw nothing (no allowed face); a `/ToUnicode` given as the name `/Identity-H` or `/Identity-V` (WSCAD) reads codes as Unicode for selection, search and editing, without control characters" +evidence = ["crates/render/src/raster/tests.rs::huge_cid_width_ranges_terminate", "crates/render/src/raster/tests.rs::non_embedded_japanese_cid_fonts_draw_with_a_craft_fonts_face", "crates/render/src/raster/tests.rs::mincho_substitute_covers_half_width_katakana", "crates/render/src/raster/tests.rs::non_embedded_chinese_and_korean_cid_fonts_draw_with_an_installed_face", "crates/engine/src/tests.rs::installed_cjk_fonts_draw_in_the_viewer_but_not_in_an_export", "crates/render/src/text/tests.rs::identity_to_unicode_names_give_text", "crates/fonts/src/pdf/tests.rs::identity_to_unicode_names_read_codes_as_unicode"] +notes = "CID-to-Unicode tables are M2.4; a non-embedded Adobe-Japan1 font is drawn in a craft-fonts Japanese face (Mincho or Gothic by name, else by the descendant font descriptor's Serif flag; bold Gothic for a Bold name or a `/FontWeight` of 600 or more; BIZ UDMincho for Mincho, which covers half-width katakana) when built with craft-fonts; the desktop viewer can fall back to an installed CJK face for GB1, CNS1, Korea1 and Japan1, including GBK-EUC-H TrueType CID fonts without ToUnicode. Windows preserves installed SimSun, SimHei, KaiTi and FangSong families, including subset-prefixed Founder aliases. Combine thumbnails and accessibility figure previews use the same on-screen fallback. System faces are display-only, disabled by PDFCRAFT_SYSTEM_FONTS=0, and not used by export, printing, OCR or automation; machines without a candidate and web builds remain unsupported. A `/ToUnicode` given as the name `/Identity-H` or `/Identity-V` (WSCAD) reads codes as Unicode for selection, search and editing, without control characters" [[feature]] id = "view.render-image-codecs"