From 69f80819e1a17e8d0a58cd09fe8f1d6726c6fd95 Mon Sep 17 00:00:00 2001 From: Philip Date: Mon, 12 Oct 2026 21:04:06 +0100 Subject: [PATCH 01/11] M2: draw non-embedded CJK CID fonts with an installed face A CID font of an Adobe CJK collection (Japan1, GB1, CNS1, Korea1) that the PDF doesn't embed drew nothing unless craft-fonts had a Japanese face: text boxes from Acrobat (Kozuka Gothic) and Pages (SongtiTC) were blank. After craft-fonts, the renderer now uses one face already installed on the machine, read at runtime and never bundled. The face of a multi-region collection (PingFang SC or TC) is chosen by region. Noto CJK and Source Han are not candidates, and PDFCRAFT_SYSTEM_FONTS=0 turns it off. Co-Authored-By: Claude Sonnet 5.5 --- ROADMAP.md | 1 + crates/render/Cargo.toml | 4 +- crates/render/src/lib.rs | 1 + crates/render/src/raster.rs | 61 +++++++- crates/render/src/system_font.rs | 248 +++++++++++++++++++++++++++++++ 5 files changed, 312 insertions(+), 3 deletions(-) create mode 100644 crates/render/src/system_font.rs diff --git a/ROADMAP.md b/ROADMAP.md index 44a26604d8..b8fc5d6956 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -141,6 +141,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md ## Progress log +- **2026-10-12 (M2, installed CJK fonts):** A CID font of an Adobe CJK collection (Japan1, GB1, CNS1, Korea1) that a PDF doesn't embed, such as the Kozuka Gothic of an Acrobat text box or the SongtiTC of a Pages export, drew nothing unless craft-fonts had a Japanese face. After craft-fonts, it now draws with a face already installed on the machine (PingFang, Heiti, Microsoft JhengHei / YaHei, Malgun and the like; read at runtime, never bundled; Noto CJK and Source Han excluded; `PDFCRAFT_SYSTEM_FONTS=0` turns it off). The face of a collection with several regions (PingFang SC, TC) is chosen by region. Checked on macOS with a synthetic Traditional Chinese, Simplified Chinese and Korean page; the Windows and Linux font lists are untested, and a machine with none of the listed faces still draws nothing. - **2026-10-12 (M1, #1339):** An incremental save of an unchanged classic-table file used to append `xref` and then `trailer` with no subsection, which qpdf reports as a damaged file. The section now has a zero-length subsection, so no object entry changes and a trailer-only edit is still written. Estimates unchanged. - **2026-10-12 (M12, auto-tag preview):** The engine and `accessibility_autotag_preview` propose paragraph and large-heading roles for bounded text blocks on one page, with locations and reasons. Every proposal needs review; images remain unclassified. No PDF tags are written yet. Synthetic tool tests cover normal text, a larger heading, bounds and unchanged source bytes. Estimates unchanged. - **2026-10-12 (M10, editing OCR text on scans):** Editing a recognised word on a scanned page now changes what the page shows. Before, the rewrite stayed invisible OCR text, so the scan still showed the old word; on a form, the whole OCR layer also grouped into one paragraph that rewrapped when one word changed. OCR words now group into phrases (a table cell or a tab stop splits them, and lines stay apart); an edit draws the phrase visibly in the scan's sampled ink colour over a cover in the sampled paper colour, keeping each word's width and growing rightwards. The editor covers the scanned word while typing. OCR words are now sized from their letters (a capitals-only word's box spans the cap height, not the full ascender-descender span), so recognised and rewritten text is as tall as the scanned word and sits on its baseline. Five engine tests on synthetic scans and a kittest UI test; `ocr.editable-text-output` is partial. diff --git a/crates/render/Cargo.toml b/crates/render/Cargo.toml index 28202a38b4..71467dfeb2 100644 --- a/crates/render/Cargo.toml +++ b/crates/render/Cargo.toml @@ -13,6 +13,8 @@ pdfcraft-cos = { workspace = true } pdfcraft-fonts = { workspace = true } hayro = { workspace = true } hayro-syntax = { workspace = true } +# Picks the face of an installed font collection (the version pdfcraft-fonts already uses). +skrifa = "0.44" lopdf = { workspace = true } kurbo = { workspace = true } thiserror = { workspace = true } @@ -27,7 +29,5 @@ bytemuck = { version = "1" } getrandom = { version = "0.4", features = ["wasm_js"] } [dev-dependencies] -# Checks the Japanese substitute's character coverage; the version pdfcraft-fonts already uses. -skrifa = "0.44" # Builds the decompression-bomb streams in the raster tests. flate2 = { workspace = true } diff --git a/crates/render/src/lib.rs b/crates/render/src/lib.rs index d2ee5712c4..90bc64b086 100644 --- a/crates/render/src/lib.rs +++ b/crates/render/src/lib.rs @@ -13,6 +13,7 @@ mod pixels; mod raster; mod reading; mod structure; +mod system_font; pub mod text; pub use inspect::{ diff --git a/crates/render/src/raster.rs b/crates/render/src/raster.rs index 0e64ac241e..fe469958ae 100644 --- a/crates/render/src/raster.rs +++ b/crates/render/src/raster.rs @@ -73,7 +73,7 @@ impl RenderConfig { InterpreterSettings { ocg_overrides: self.layers.clone(), hide_comments: self.hide_comments, - font_resolver: Arc::new(move |query| japanese_fallback(query).or_else(|| standard(query))), + font_resolver: Arc::new(move |query| japanese_fallback(query).or_else(|| system_fallback(query)).or_else(|| standard(query))), ..InterpreterSettings::default() } } @@ -106,6 +106,14 @@ fn japanese_fallback(query: &FontQuery) -> Option<(FontData, u32)> { Some((Arc::new(face.bytes), 0)) } +/// An installed face for a CID font of an Adobe CJK collection that the PDF doesn't embed and +/// craft-fonts has no face for (Traditional and Simplified Chinese, Korean, and Japanese in a +/// build without craft-fonts). `None` for every other font. +fn system_fallback(query: &FontQuery) -> Option<(FontData, u32)> { + let FontQuery::Fallback(f) = query else { return None }; + crate::system_font::fallback(f) +} + #[derive(Debug, PartialEq)] enum JapaneseFace { Mincho, @@ -5515,6 +5523,57 @@ trailer << /Root 1 0 R >> } } + /// Chinese and Korean CID fonts that aren't embedded (Adobe-CNS1 `MSung-Light`, Adobe-GB1 + /// `STSong-Light`, Adobe-Korea1 `HYGoThic-Medium`; Acrobat also writes Japanese `KozGoPr6N` + /// text boxes without the font) drew nothing, because craft-fonts has only Japanese faces. + /// They now draw in a face already installed on the machine, read at runtime. A machine + /// without such a face draws nothing, as before, so each check is skipped there. + #[test] + fn non_embedded_chinese_and_korean_cid_fonts_draw_with_an_installed_face() { + use hayro::hayro_interpret::font::FallbackFontQuery; + use hayro::hayro_interpret::hayro_cmap::CharacterCollection; + // (base font, collection, ordering, encoding, UCS-2 code units of the text) + let cases = [ + ("MSung-Light", CidFamily::AdobeCNS1, "CNS1", "UniCNS-UCS2-H", "7E419AD45B57"), // 繁體字 + ("STSong-Light", CidFamily::AdobeGB1, "GB1", "UniGB-UCS2-H", "72698D448DF7"), // 物资跷 (common Simplified) + ("HYGoThic-Medium", CidFamily::AdobeKorea1, "Korea1", "UniKS-UCS2-H", "D55CAE00"), // 한글 + ]; + for (base_font, family, ordering, encoding, code) in cases { + let query = FontQuery::Fallback(FallbackFontQuery { + post_script_name: Some(base_font.into()), + character_collection: Some(CharacterCollection { family, supplement: 2 }), + ..FallbackFontQuery::default() + }); + if super::system_fallback(&query).is_none() { + eprintln!("no installed face for {base_font} (or PDFCRAFT_SYSTEM_FONTS=0): skipped"); + continue; + } + let content = format!("BT /F1 40 Tf 5 15 Td <{code}> Tj ET"); + let pdf = format!( + "%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 60] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >> endobj +4 0 obj << /Length {} >> stream +{content} +endstream endobj +5 0 obj << /Type /Font /Subtype /Type0 /BaseFont /{base_font} /Encoding /{encoding} /DescendantFonts [6 0 R] >> endobj +6 0 obj << /Type /Font /Subtype /CIDFontType0 /BaseFont /{base_font} /CIDSystemInfo << /Registry (Adobe) /Ordering ({ordering}) /Supplement 2 >> + /FontDescriptor 7 0 R /DW 1000 >> endobj +7 0 obj << /Type /FontDescriptor /FontName /{base_font} /Flags 6 /FontBBox [0 -141 1000 859] /ItalicAngle 0 /Ascent 859 /Descent -141 /CapHeight 700 /StemV 80 >> endobj +trailer << /Root 1 0 R >> +%%EOF", + content.len() + ); + let mut r = PageRenderer::new(Arc::new(pdf.into_bytes()), RenderConfig::default()); + let p = r.render(RenderRequest { page: 0, kind: RequestKind::Pixels, tile: None, scale: 1.0, tag: 0, ..Default::default() }); + assert!(p.error.is_none(), "{base_font}: {:?}", p.error); + let inked = p.rgba.as_chunks::<4>().0.iter().filter(|c| c[0] < 128).count(); + // Two to four glyphs at 40 pt cover several hundred dark pixels; a blank run has none. + assert!(inked > 300, "{base_font}: the text is drawn ({inked} dark pixels)"); + } + } + /// Half-width katakana are common in Japanese documents set in a non-embedded Mincho font /// such as HeiseiMin-W3. The Mincho substitute must have those glyphs: one that lacks them /// (Shippori Mincho) draws them as .notdef. diff --git a/crates/render/src/system_font.rs b/crates/render/src/system_font.rs new file mode 100644 index 0000000000..d2c98bb7cd --- /dev/null +++ b/crates/render/src/system_font.rs @@ -0,0 +1,248 @@ +//! Installed CJK faces for CID fonts a PDF doesn't embed. +//! +//! Acrobat and other tools often write Chinese, Japanese or Korean text with one of the Adobe +//! collections (Adobe-Japan1, GB1, CNS1, Korea1) and a font name such as `KozGoPr6N-Medium` or +//! `MSung-Light` without embedding the font. hayro maps those CIDs to Unicode itself, but it needs +//! a face to draw them with; without one the text is blank. The craft-fonts Japanese faces come +//! first (see `japanese_fallback` in `raster.rs`); this module is the last resort: one face +//! already installed on this machine. It is read at runtime and never embedded or shipped +//! (AGENTS.md §1.4), Noto CJK and Source Han are left out on purpose (§1.1), and +//! `PDFCRAFT_SYSTEM_FONTS=0` turns it off, as for the interface font. + +use std::collections::HashMap; +use std::path::{Path, PathBuf}; +use std::sync::{Arc, Mutex, PoisonError}; + +use hayro::hayro_interpret::font::{FallbackFontQuery, FontData}; +use hayro::hayro_interpret::hayro_cmap::CidFamily; +use skrifa::MetadataProvider as _; +use skrifa::string::StringId; + +/// Larger files are not read: a font path is still untrusted input. CJK system faces are big +/// collections (PingFang is tens of megabytes). +const MAX_BYTES: u64 = 96 << 20; +/// Faces tried in a collection (`.ttc`). +const MAX_FACES: u32 = 32; + +/// The writing system a character collection is for. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +enum Script { + Japanese, + SimplifiedChinese, + TraditionalChinese, + Korean, +} + +impl Script { + fn of(family: &CidFamily) -> Option { + match family { + CidFamily::AdobeJapan1 => Some(Self::Japanese), + CidFamily::AdobeGB1 => Some(Self::SimplifiedChinese), + CidFamily::AdobeCNS1 => Some(Self::TraditionalChinese), + CidFamily::AdobeKorea1 => Some(Self::Korean), + CidFamily::AdobeIdentity | CidFamily::Custom { .. } => None, + } + } + + /// A character the face must have to be worth using. Each is common and not shared by the + /// other scripts' faces: 物 (Japanese; also Chinese, so it only rules out Latin-only faces), + /// 欢 (Simplified only), 說 (Traditional only), 한 (Hangul). + fn probe(self) -> char { + match self { + Self::Japanese => '\u{7269}', + Self::SimplifiedChinese => '\u{6B22}', + Self::TraditionalChinese => '\u{8AAA}', + Self::Korean => '\u{D55C}', + } + } +} + +/// One installed file and the words (in a face's family name) that pick the face for the right +/// region out of a collection that holds several (PingFang SC, TC and HK share one file). +struct Candidate { + path: PathBuf, + region: &'static [&'static str], +} + +/// The installed face to draw a CID font of `query`'s collection with, as `(data, face index)`. +/// `None` when it is turned off, the collection isn't one of the four, or nothing fits. +pub(crate) fn fallback(query: &FallbackFontQuery) -> Option<(FontData, u32)> { + if cfg!(target_arch = "wasm32") || std::env::var_os("PDFCRAFT_SYSTEM_FONTS").is_some_and(|v| v == "0") { + return None; + } + let script = Script::of(&query.character_collection.as_ref()?.family)?; + let bold = query.is_bold || query.font_weight >= 600; + /// A face file and the index of the chosen face in it, or `None` when nothing fits. + type Face = Option<(Arc>, u32)>; + static CACHE: Mutex>> = Mutex::new(None); + let mut cache = CACHE.lock().unwrap_or_else(PoisonError::into_inner); + let found = cache.get_or_insert_with(HashMap::new).entry((script, bold)).or_insert_with(|| { + candidates(script) + .iter() + .find_map(|c| read(c, script.probe(), bold)) + .inspect(|(_, index)| log::info!("CJK font fallback for {script:?}: face {index}")) + }); + found.as_ref().map(|(bytes, index)| (Arc::clone(bytes) as FontData, *index)) +} + +fn read(candidate: &Candidate, probe: char, bold: bool) -> Option<(Arc>, u32)> { + let meta = std::fs::metadata(&candidate.path).ok()?; + if !meta.is_file() || meta.len() > MAX_BYTES { + return None; + } + let bytes = std::fs::read(&candidate.path).ok()?; + let index = pick_face(&bytes, probe, candidate.region, bold)?; + Some((Arc::new(bytes), index)) +} + +/// The best face of the file that parses and maps `probe`: the region's face when the file has +/// one, then the regular (or, for bold text, the semibold or bold) weight, then the first. +fn pick_face(bytes: &[u8], probe: char, region: &[&str], bold: bool) -> Option { + let mut best: Option<(u32, u32)> = None; + for index in 0..MAX_FACES { + let Ok(font) = skrifa::FontRef::from_index(bytes, index) else { continue }; + if font.charmap().map(probe).is_none() { + continue; + } + let names = |id: StringId| font.localized_strings(id).map(|s| s.chars().collect::()).collect::>(); + let family = names(StringId::FAMILY_NAME).join(" "); + let style = names(StringId::SUBFAMILY_NAME).join(" ").to_ascii_lowercase(); + let mut score = 1; + if region.iter().any(|r| family.split_whitespace().any(|w| w == *r)) { + score += 4; + } + let weighty = style.contains("bold") || style.contains("semibold") || style.contains("heavy"); + let plain = style.is_empty() || style.contains("regular") || style.contains("normal") || style.contains("medium") && !bold; + if (bold && weighty) || (!bold && plain && !weighty) { + score += 2; + } + if best.is_none_or(|(_, s)| score > s) { + best = Some((index, score)); + } + } + best.map(|(index, _)| index) +} + +fn candidates(script: Script) -> Vec { + let region: &'static [&'static str] = match script { + Script::Japanese => &["JP", "Japanese"], + Script::SimplifiedChinese => &["SC", "GB"], + Script::TraditionalChinese => &["TC", "HK", "CNS"], + Script::Korean => &["KR", "Korean"], + }; + let files: Vec = if cfg!(windows) { + let dir = windows_fonts(); + let names: &[&str] = match script { + Script::Japanese => &["YuGothR.ttc", "meiryo.ttc", "msgothic.ttc"], + Script::SimplifiedChinese => &["msyh.ttc", "simsun.ttc", "msjh.ttc"], + Script::TraditionalChinese => &["msjh.ttc", "mingliu.ttc", "msyh.ttc"], + Script::Korean => &["malgun.ttf", "gulim.ttc"], + }; + names.iter().map(|f| dir.join(f)).collect() + } else if cfg!(target_os = "macos") { + let names: &[&str] = match script { + Script::Japanese => &["ヒラギノ角ゴシック W3.ttc", "Hiragino Sans GB.ttc", "PingFang.ttc"], + Script::SimplifiedChinese => &["PingFang.ttc", "Hiragino Sans GB.ttc", "STHeiti Light.ttc"], + Script::TraditionalChinese => &["PingFang.ttc", "STHeiti Light.ttc", "Hiragino Sans GB.ttc"], + Script::Korean => &["AppleSDGothicNeo.ttc", "AppleGothic.ttf", "PingFang.ttc"], + }; + names.iter().map(|f| Path::new("/System/Library/Fonts").join(f)).collect() + } else { + let names: &[&str] = match script { + Script::Korean => { + &["truetype/nanum/NanumGothic.ttf", "nanum/NanumGothic.ttf", "truetype/baekmuk/gulim.ttf", "truetype/unfonts-core/UnDotum.ttf"] + } + Script::Japanese => &[ + "truetype/fonts-japanese-gothic.ttf", + "ipa-gothic/ipag.ttf", + "truetype/takao-gothic/TakaoGothic.ttf", + "truetype/vlgothic/VL-Gothic-Regular.ttf", + ], + Script::SimplifiedChinese | Script::TraditionalChinese => &[ + "truetype/wqy/wqy-microhei.ttc", + "wenquanyi/wqy-microhei/wqy-microhei.ttc", + "wqy-microhei/wqy-microhei.ttc", + "truetype/wqy/wqy-zenhei.ttc", + "wenquanyi/wqy-zenhei/wqy-zenhei.ttc", + "wqy-zenhei/wqy-zenhei.ttc", + "truetype/droid/DroidSansFallbackFull.ttf", + "google-droid-sans-fonts/DroidSansFallbackFull.ttf", + "truetype/arphic/uming.ttc", + "arphic-uming/uming.ttc", + ], + }; + ["/usr/share/fonts", "/usr/local/share/fonts"].iter().flat_map(|dir| names.iter().map(move |f| Path::new(dir).join(f))).collect() + }; + files.into_iter().map(|path| Candidate { path, region }).collect() +} + +fn windows_fonts() -> PathBuf { + let dir = std::env::var_os("WINDIR").or_else(|| std::env::var_os("SystemRoot")).map_or_else(|| PathBuf::from(r"C:\Windows"), PathBuf::from); + dir.join("Fonts") +} + +#[cfg(test)] +mod tests { + use super::*; + + const SCRIPTS: [Script; 4] = [Script::Japanese, Script::SimplifiedChinese, Script::TraditionalChinese, Script::Korean]; + + #[test] + fn broken_and_missing_files_are_skipped() { + assert_eq!(pick_face(b"", '\u{7269}', &[], false), None); + assert_eq!(pick_face(b"not a font at all", '\u{7269}', &[], false), None); + assert_eq!(pick_face(&[0u8; 4096], '\u{7269}', &[], false), None); + let missing = Candidate { path: PathBuf::from("definitely/not/here.ttc"), region: &[] }; + assert!(read(&missing, '\u{7269}', false).is_none()); + // A directory is not a font. + let dir = Candidate { path: std::env::temp_dir(), region: &[] }; + assert!(read(&dir, '\u{7269}', false).is_none()); + } + + #[test] + fn a_latin_face_is_rejected_for_every_script() { + let inter = include_bytes!("../../../assets/fonts/Inter-Regular.ttf"); + for script in SCRIPTS { + assert_eq!(pick_face(inter, script.probe(), &[], false), None, "{script:?}"); + } + assert_eq!(pick_face(inter, 'A', &[], false), Some(0)); + } + + #[test] + fn candidates_are_absolute_font_files_without_adobe_designs() { + for script in SCRIPTS { + let list = candidates(script); + assert!(!list.is_empty(), "{script:?}"); + for c in list { + assert!(c.path.is_absolute(), "{}", c.path.display()); + assert!(c.path.extension().is_some_and(|e| e == "ttf" || e == "ttc"), "{}", c.path.display()); + let name = c.path.to_string_lossy().to_lowercase(); + assert!(!["notosanscjk", "notoserifcjk", "sourcehan", "source-han"].iter().any(|b| name.contains(b)), "{name}"); + } + } + } + + #[test] + fn only_the_four_adobe_collections_have_a_script() { + assert_eq!(Script::of(&CidFamily::AdobeCNS1), Some(Script::TraditionalChinese)); + assert_eq!(Script::of(&CidFamily::AdobeGB1), Some(Script::SimplifiedChinese)); + assert_eq!(Script::of(&CidFamily::AdobeKorea1), Some(Script::Korean)); + assert_eq!(Script::of(&CidFamily::AdobeJapan1), Some(Script::Japanese)); + assert_eq!(Script::of(&CidFamily::AdobeIdentity), None); + } + + /// On a Mac the face for Traditional Chinese is the TC one of PingFang, not the first of the + /// collection (which is another region's), and it maps a character only Traditional has. + #[cfg(target_os = "macos")] + #[test] + fn macos_picks_the_regional_pingfang_face() { + let path = Path::new("/System/Library/Fonts/PingFang.ttc"); + let Ok(bytes) = std::fs::read(path) else { return }; + for (script, region) in [(Script::TraditionalChinese, "TC"), (Script::SimplifiedChinese, "SC")] { + let index = pick_face(&bytes, script.probe(), &[region], false).expect("PingFang covers it"); + let font = skrifa::FontRef::from_index(&bytes, index).unwrap(); + let family: String = font.localized_strings(StringId::FAMILY_NAME).next().unwrap().chars().collect(); + assert!(family.contains(region), "{script:?} got {family:?}"); + } + } +} From 5b3b87430b493b7dc94bffdb13183bfd751ddc5e Mon Sep 17 00:00:00 2001 From: Philip Date: Mon, 19 Oct 2026 16:41:46 +0100 Subject: [PATCH 02/11] M2: review of #1567: read each installed font file once and share it CodeRabbit: the fallback cache was keyed by (script, weight) and every key read and kept its own copy of the file, so PingFang.ttc (tens of megabytes, a candidate of several scripts and both weights) could be held up to four times. The bytes now come from a process-wide cache keyed by path, remembering files that cannot be used too; the per-script cache keeps a handle and a face index. Co-Authored-By: Claude Sonnet 5.5 --- crates/render/src/system_font.rs | 47 ++++++++++++++++++++++++++++---- 1 file changed, 42 insertions(+), 5 deletions(-) diff --git a/crates/render/src/system_font.rs b/crates/render/src/system_font.rs index d2c98bb7cd..0d9d158a35 100644 --- a/crates/render/src/system_font.rs +++ b/crates/render/src/system_font.rs @@ -86,13 +86,30 @@ pub(crate) fn fallback(query: &FallbackFontQuery) -> Option<(FontData, u32)> { } fn read(candidate: &Candidate, probe: char, bold: bool) -> Option<(Arc>, u32)> { - let meta = std::fs::metadata(&candidate.path).ok()?; - if !meta.is_file() || meta.len() > MAX_BYTES { + let bytes = file_bytes(&candidate.path)?; + let index = pick_face(&bytes, probe, candidate.region, bold)?; + Some((bytes, index)) +} + +/// The bytes of an installed font file, read once for the whole process and shared: a collection +/// such as `PingFang.ttc` is tens of megabytes and is the candidate of several scripts and both +/// weights, so the per-script cache keeps only a handle and a face index. A file that cannot be +/// used is remembered too, so it is not tried again for every script. +fn file_bytes(path: &Path) -> Option>> { + /// The bytes of each file tried, or `None` for one that cannot be used. + type Files = HashMap>>>; + static FILES: Mutex> = Mutex::new(None); + let mut files = FILES.lock().unwrap_or_else(PoisonError::into_inner); + files.get_or_insert_with(HashMap::new).entry(path.to_path_buf()).or_insert_with(|| read_file(path, MAX_BYTES).map(Arc::new)).clone() +} + +/// A regular file of at most `max` bytes, or `None`: a font path is untrusted input. +fn read_file(path: &Path, max: u64) -> Option> { + let meta = std::fs::metadata(path).ok()?; + if !meta.is_file() || meta.len() > max { return None; } - let bytes = std::fs::read(&candidate.path).ok()?; - let index = pick_face(&bytes, probe, candidate.region, bold)?; - Some((Arc::new(bytes), index)) + std::fs::read(path).ok() } /// The best face of the file that parses and maps `probe`: the region's face when the file has @@ -199,6 +216,26 @@ mod tests { assert!(read(&dir, '\u{7269}', false).is_none()); } + /// The files are bounded by `max`: a larger one is not read. + #[test] + fn a_file_over_the_cap_is_not_read() { + let inter = Path::new(concat!(env!("CARGO_MANIFEST_DIR"), "/../../assets/fonts/Inter-Regular.ttf")); + assert!(read_file(inter, MAX_BYTES).is_some()); + assert!(read_file(inter, 1024).is_none()); + } + + /// Two faces drawn from one installed file (two scripts, or regular and bold) hold the same + /// bytes, not a copy each; a file that can't be used is remembered as unusable. + #[test] + fn one_installed_file_is_read_once_and_shared() { + let inter = Path::new(concat!(env!("CARGO_MANIFEST_DIR"), "/../../assets/fonts/Inter-Regular.ttf")); + let first = file_bytes(inter).expect("readable"); + let second = file_bytes(inter).expect("readable"); + assert!(Arc::ptr_eq(&first, &second), "one allocation for the file"); + let missing = Path::new("definitely/not/here.ttc"); + assert!(file_bytes(missing).is_none() && file_bytes(missing).is_none()); + } + #[test] fn a_latin_face_is_rejected_for_every_script() { let inter = include_bytes!("../../../assets/fonts/Inter-Regular.ttf"); From 102ceecf663fa7a506bbed8e4c38236ee574cc13 Mon Sep 17 00:00:00 2001 From: Philip Date: Mon, 19 Oct 2026 18:22:03 +0100 Subject: [PATCH 03/11] M2: review of #1567: find PingFang in the macOS font assets CodeRabbit: on recent macOS the PingFang collection is not in /System/Library/Fonts but in the downloaded font assets (/System/Library/AssetsV2/com_apple_MobileAsset_Font*/.asset/AssetData), so the fixed path missed it, and the PingFang test returned early without checking anything on such a machine. The macOS candidates now include the AssetData folders of those assets (listed once, at most 512 per kind), and a test searches the assets independently and requires the candidate list to contain what it finds. Co-Authored-By: Claude Sonnet 5.5 --- crates/render/src/system_font.rs | 73 ++++++++++++++++++++++++++++++-- 1 file changed, 69 insertions(+), 4 deletions(-) diff --git a/crates/render/src/system_font.rs b/crates/render/src/system_font.rs index 0d9d158a35..40543cf46e 100644 --- a/crates/render/src/system_font.rs +++ b/crates/render/src/system_font.rs @@ -11,7 +11,7 @@ use std::collections::HashMap; use std::path::{Path, PathBuf}; -use std::sync::{Arc, Mutex, PoisonError}; +use std::sync::{Arc, Mutex, OnceLock, PoisonError}; use hayro::hayro_interpret::font::{FallbackFontQuery, FontData}; use hayro::hayro_interpret::hayro_cmap::CidFamily; @@ -23,6 +23,8 @@ use skrifa::string::StringId; const MAX_BYTES: u64 = 96 << 20; /// Faces tried in a collection (`.ttc`). const MAX_FACES: u32 = 32; +/// Downloaded font assets looked at per kind on macOS. +const MAX_ASSETS: usize = 512; /// The writing system a character collection is for. #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] @@ -163,7 +165,11 @@ fn candidates(script: Script) -> Vec { Script::TraditionalChinese => &["PingFang.ttc", "STHeiti Light.ttc", "Hiragino Sans GB.ttc"], Script::Korean => &["AppleSDGothicNeo.ttc", "AppleGothic.ttf", "PingFang.ttc"], }; - names.iter().map(|f| Path::new("/System/Library/Fonts").join(f)).collect() + // The system folder first, then the downloaded font assets: on recent macOS a face such as + // PingFang lives in `/System/Library/AssetsV2/com_apple_MobileAsset_Font*/.asset/AssetData`. + let mut dirs = vec![PathBuf::from("/System/Library/Fonts")]; + dirs.extend(macos_font_asset_dirs().iter().cloned()); + names.iter().flat_map(|f| dirs.iter().map(move |d| d.join(f))).collect() } else { let names: &[&str] = match script { Script::Korean => { @@ -193,6 +199,28 @@ fn candidates(script: Script) -> Vec { files.into_iter().map(|path| Candidate { path, region }).collect() } +/// The `AssetData` folders of the downloaded font assets of macOS, found once. The ids in their +/// names change with each asset version, so they are listed rather than named; at most +/// [`MAX_ASSETS`] are looked at. +fn macos_font_asset_dirs() -> &'static [PathBuf] { + static DIRS: OnceLock> = OnceLock::new(); + DIRS.get_or_init(|| { + let root = Path::new("/System/Library/AssetsV2"); + let Ok(kinds) = std::fs::read_dir(root) else { return Vec::new() }; + let mut dirs = Vec::new(); + for kind in kinds.flatten().filter(|k| k.file_name().to_string_lossy().starts_with("com_apple_MobileAsset_Font")) { + let Ok(assets) = std::fs::read_dir(kind.path()) else { continue }; + for asset in assets.flatten().take(MAX_ASSETS) { + let data = asset.path().join("AssetData"); + if data.is_dir() { + dirs.push(data); + } + } + } + dirs + }) +} + fn windows_fonts() -> PathBuf { let dir = std::env::var_os("WINDIR").or_else(|| std::env::var_os("SystemRoot")).map_or_else(|| PathBuf::from(r"C:\Windows"), PathBuf::from); dir.join("Fonts") @@ -273,8 +301,15 @@ mod tests { #[cfg(target_os = "macos")] #[test] fn macos_picks_the_regional_pingfang_face() { - let path = Path::new("/System/Library/Fonts/PingFang.ttc"); - let Ok(bytes) = std::fs::read(path) else { return }; + // Found where this macOS keeps it: the system folder, or the downloaded font assets. + let path = candidates(Script::TraditionalChinese) + .into_iter() + .map(|c| c.path) + .find(|p| p.file_name().is_some_and(|n| n == "PingFang.ttc") && p.is_file()); + let Some(bytes) = path.and_then(|p| std::fs::read(p).ok()) else { + eprintln!("PingFang is not installed here: skipped"); + return; + }; for (script, region) in [(Script::TraditionalChinese, "TC"), (Script::SimplifiedChinese, "SC")] { let index = pick_face(&bytes, script.probe(), &[region], false).expect("PingFang covers it"); let font = skrifa::FontRef::from_index(&bytes, index).unwrap(); @@ -282,4 +317,34 @@ mod tests { assert!(family.contains(region), "{script:?} got {family:?}"); } } + /// A face that this macOS keeps only in its downloaded font assets (PingFang, on recent + /// versions) is among the candidates. The assets are searched here on their own, not by the + /// code under test; where there are none, nothing is checked. + #[cfg(target_os = "macos")] + #[test] + fn macos_candidates_include_a_pingfang_that_lives_in_the_font_assets() { + fn find(dir: &Path, name: &str, depth: u32) -> Option { + for entry in std::fs::read_dir(dir).ok()?.flatten() { + let path = entry.path(); + if path.file_name().is_some_and(|n| n == name) && path.is_file() { + return Some(path); + } + if depth > 0 + && path.is_dir() + && !path.is_symlink() + && let Some(found) = find(&path, name, depth - 1) + { + return Some(found); + } + } + None + } + let Some(asset) = find(Path::new("/System/Library/AssetsV2"), "PingFang.ttc", 3) else { + eprintln!("no PingFang in the font assets here: skipped"); + return; + }; + for script in [Script::SimplifiedChinese, Script::TraditionalChinese] { + assert!(candidates(script).iter().any(|c| c.path == asset), "{script:?}: {} is not a candidate", asset.display()); + } + } } From 57a16aea171f5e015a4de17d9c47f1e5fb22a646 Mon Sep 17 00:00:00 2001 From: Philip Date: Wed, 21 Oct 2026 20:17:27 +0100 Subject: [PATCH 04/11] M2: review of #1567: draw installed CJK fonts on screen only MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit FredApps: AGENTS.md §1.4 allows reading an installed font at runtime only as the interface's last fallback and says nothing from it may enter a PDF or package, while the renderer fallback also served exports and printing. RenderConfig gains system_fonts, off by default. The engine sets it only on the pool that draws the viewer (display_config); exports, printing, OCR and the automation tools render with the document's own configuration, so nothing drawn with an installed font is saved or printed. Tests: the render of a non-embedded CNS1, GB1 and Korea1 font is blank without the flag and drawn with it; the viewer's pool draws such a page and an export of it does not (it fails if the flag is on everywhere). No Adobe-named font can be a candidate. Co-Authored-By: Claude Sonnet 5.5 --- ROADMAP.md | 2 +- crates/engine/src/lib.rs | 26 +++++++++++------ crates/engine/src/tests.rs | 49 ++++++++++++++++++++++++++++++++ crates/render/src/raster.rs | 28 ++++++++++++++---- crates/render/src/system_font.rs | 8 ++++-- 5 files changed, 95 insertions(+), 18 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 38f5ff3e5f..bc916e033c 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -214,7 +214,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-10-15 (M14, gap 24):** `misc.localization` and `misc.installers` move from planned to partial. Localization: 22 catalogs ship, but Chinese shows tofu in releases, Telugu has no release font and Arabic is not mirrored. Installers: macOS and Windows releases are signed, Linux packages are not. The Acrobat menu diff itself remains open. Estimates unchanged. - **2026-10-13 (G14, Vietnamese localization):** Added Vietnamese (`vi`, Tiếng Việt) as an interface and date language with a 2,591-entry catalog (`plural_none`) covering 100% of the current localizable key union (plus `theme`/`Appearance`). Locale tags `vi` / `vi-VN` / `vi_VN.UTF-8` resolve; calendar months use numeric `Tháng 1`…`12` / `tháng 1`…`12`. Native-speaker community review recorded. The twelve-language localization mean moves from ≈59% to ≈68%; weighted readiness ≈49% → ≈50%. Status stays `partial`: engine/OS details, document contents, filenames, CLI/MCP and external help stay outside the catalog by design. -- **2026-10-12 (M2, installed CJK fonts):** A CID font of an Adobe CJK collection (Japan1, GB1, CNS1, Korea1) that a PDF doesn't embed, such as the Kozuka Gothic of an Acrobat text box or the SongtiTC of a Pages export, drew nothing unless craft-fonts had a Japanese face. After craft-fonts, it now draws with a face already installed on the machine (PingFang, Heiti, Microsoft JhengHei / YaHei, Malgun and the like; read at runtime, never bundled; Noto CJK and Source Han excluded; `PDFCRAFT_SYSTEM_FONTS=0` turns it off). The face of a collection with several regions (PingFang SC, TC) is chosen by region. Checked on macOS with a synthetic Traditional Chinese, Simplified Chinese and Korean page; the Windows and Linux font lists are untested, and a machine with none of the listed faces still draws nothing. +- **2026-10-12 (M2, installed CJK fonts):** A CID font of an Adobe CJK collection (Japan1, GB1, CNS1, Korea1) that a PDF doesn't embed, such as the Kozuka Gothic of an Acrobat text box or the SongtiTC of a Pages export, drew nothing unless craft-fonts had a Japanese face. After craft-fonts, it now draws with a face already installed on the machine (PingFang, Heiti, Microsoft JhengHei / YaHei, Malgun and the like; read at runtime, never bundled, and used only for the on-screen viewer, never for exports, printing or OCR; Noto CJK and Source Han excluded; `PDFCRAFT_SYSTEM_FONTS=0` turns it off). The face of a collection with several regions (PingFang SC, TC) is chosen by region. Checked on macOS with a synthetic Traditional Chinese, Simplified Chinese and Korean page; the Windows and Linux font lists are untested, and a machine with none of the listed faces still draws nothing. - **2026-10-13 (M3, typed zoom):** The right rail's zoom percentage is an editable field: type 150, 150% or 62.5 % and press Enter, Escape discards, and nonsense keeps the current zoom. The control channel's zoom option shares its parser, which now refuses zero, negative and non-finite values. - **2026-10-13 (M4, indirect page rotation):** Rotation queries and edits resolve indirect integer values on pages and their page-tree ancestors. Turning a page already at 270° by 90° now saves it at 0°, including when its rotation is inherited. Synthetic full/incremental save-and-reopen tests cover nested inheritance, sibling preservation, negative/large angles and invalid-value fallback. Estimates unchanged. - **2026-10-14 (M1, Flate fallback):** Raw deflate whose first bytes resemble a zlib header no longer loses a decoding-limit error during fallback. Strict and tolerant filter regressions and a COS stream regression cover refusal at the caller limit and successful decoding within it. No parity status or estimate changes. diff --git a/crates/engine/src/lib.rs b/crates/engine/src/lib.rs index 8f39248abd..d1b94f157b 100644 --- a/crates/engine/src/lib.rs +++ b/crates/engine/src/lib.rs @@ -1009,11 +1009,19 @@ fn set_default_layer_state(cos: &mut pdfcraft_cos::Document, layers: &[Layer]) { cos.set(root_ref, Object::Dict(root)); } +/// The configuration of the pool that draws the viewer: the document's own, plus installed CJK +/// fonts for fonts a PDF does not embed (`RenderConfig::system_fonts`). Exports, printing, OCR and +/// the automation tools render with the document's configuration itself, so nothing drawn with an +/// installed font is saved or printed (AGENTS.md §1.2, §1.4). +fn display_config(config: &RenderConfig) -> RenderConfig { + RenderConfig { system_fonts: true, ..config.clone() } +} + /// Render `doc` with the layer visibility in its `info.layers`. fn use_layer_choices(doc: &mut Document) { let overrides: Vec<(i32, i32, bool)> = doc.info.layers.iter().map(|l| (l.id.0 as i32, l.id.1 as i32, l.visible)).collect(); doc.config.layers = Arc::new(overrides); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); } /// Apply a set-layer-visibility action to `layers`, one change at a time, so a toggle flips the @@ -3580,7 +3588,7 @@ impl Session { let sig_cache = Arc::new(pdfcraft_sign::DigestCache::default()); let signatures = editor.as_ref().map(|e| signatures_of(&e.cos, &bytes, trust, &sig_cache)).unwrap_or_default(); let display = display_bytes(editor.as_mut(), &bytes); - let renderer = RenderPool::new(display.clone(), render_threads(), config.clone()); + let renderer = RenderPool::new(display.clone(), render_threads(), display_config(&config)); let (form, adopted) = editor .as_ref() .map(|e| { @@ -3962,7 +3970,7 @@ impl Session { let signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache) }; doc.display = display_bytes(Some(editor), &bytes); doc.signatures = Arc::new(signatures); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.bytes = bytes; Ok(()) } @@ -4007,7 +4015,7 @@ impl Session { let signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache) }; doc.display = display_bytes(Some(editor), &bytes); doc.signatures = Arc::new(signatures); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.bytes = bytes; doc.fonts_stale = true; Ok(()) @@ -4032,7 +4040,7 @@ impl Session { /// After a refresh failed: draw the display bytes it left (the original, once released), /// so the page still shows a document. fn show_released(doc: &mut Document) { - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); } /// Rebuild working bytes, inspection and renderer from the current edit state. @@ -4082,7 +4090,7 @@ impl Session { let signatures = signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache); doc.display = display_bytes(Some(editor), &bytes); doc.signatures = Arc::new(signatures); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.bytes = bytes; Ok(()) } @@ -4795,7 +4803,7 @@ impl Session { return false; } doc.config.hide_comments = hide; - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.generation += 1; true } @@ -4808,7 +4816,7 @@ impl Session { return false; } doc.config.reading_colors = colors; - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.generation += 1; true } @@ -4844,7 +4852,7 @@ impl Session { let signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &doc.bytes, &doc.trust, &doc.sig_cache) }; doc.display = display_bytes(Some(editor), &doc.bytes); doc.signatures = Arc::new(signatures); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); } } diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index f125c41dac..5e6ef3b301 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -385,6 +385,55 @@ fn edits_update_view_data_and_mark_dirty() { assert_eq!(page_texts(&s, id), ["Page 1", "Page 3"], "renderer refreshed"); } +/// A font of an Adobe CJK collection that a PDF does not embed is drawn with an installed face on +/// the viewer's pool only. An export of the same page, which can be saved, has nothing drawn with +/// an installed font (AGENTS.md §1.2, §1.4). Skipped where no such face is installed. +#[test] +fn installed_cjk_fonts_draw_in_the_viewer_but_not_in_an_export() { + let content = "BT /F1 40 Tf 5 15 Td <7E419AD45B57> Tj ET"; + let pdf = format!( + "%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 60] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >> endobj +4 0 obj << /Length {} >> stream +{content} +endstream endobj +5 0 obj << /Type /Font /Subtype /Type0 /BaseFont /MSung-Light /Encoding /UniCNS-UCS2-H /DescendantFonts [6 0 R] >> endobj +6 0 obj << /Type /Font /Subtype /CIDFontType0 /BaseFont /MSung-Light /CIDSystemInfo << /Registry (Adobe) /Ordering (CNS1) /Supplement 2 >> /FontDescriptor 7 0 R /DW 1000 >> endobj +7 0 obj << /Type /FontDescriptor /FontName /MSung-Light /Flags 6 /FontBBox [0 -141 1000 859] /ItalicAngle 0 /Ascent 859 /Descent -141 /CapHeight 700 /StemV 80 >> endobj +trailer << /Root 1 0 R >> +%%EOF", + content.len() + ); + let mut s = Session::new(); + let id = s.open("cns1.pdf", None, Arc::new(pdf.into_bytes()), None).unwrap(); + let doc = s.get(id).unwrap(); + doc.renderer.set_queue(vec![pdfcraft_render::RenderRequest { + page: 0, + kind: pdfcraft_render::RequestKind::Pixels, + scale: 1.0, + ..Default::default() + }]); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(30); + let viewer = loop { + if let Some(page) = doc.renderer.try_recv() { + assert!(page.error.is_none(), "{:?}", page.error); + break page; + } + assert!(std::time::Instant::now() < deadline, "page render did not finish"); + std::thread::yield_now(); + }; + let inked = |rgba: &[u8]| rgba.as_chunks::<4>().0.iter().filter(|c| c[0] < 128).count(); + if inked(&viewer.rgba) < 300 { + eprintln!("no installed Traditional Chinese face (or PDFCRAFT_SYSTEM_FONTS=0): skipped"); + return; + } + let png = export::Exporter::new(doc).png(0, 72.0).unwrap(); + let exported = image::load_from_memory(&png).unwrap().into_rgba8(); + assert!(inked(exported.as_raw()) < 50, "an export draws nothing with an installed font ({} dark pixels)", inked(exported.as_raw())); +} + #[test] fn metadata_edit_keeps_the_renderer_without_changing_page_content() { let (mut s, id) = session_with(1); diff --git a/crates/render/src/raster.rs b/crates/render/src/raster.rs index 2ef4870fe0..01aa89d69f 100644 --- a/crates/render/src/raster.rs +++ b/crates/render/src/raster.rs @@ -65,15 +65,23 @@ pub struct RenderConfig { /// View ▸ Reading colours: solid text and line-art paints are remapped for readability and /// the page clears to the setting's background. Display only; the document never changes. pub reading_colors: ReadingColors, + /// Draw a CID font of an Adobe CJK collection that the PDF does not embed with one face already + /// installed on this machine (see `system_font`). Off by default and on only for the pool that + /// draws the viewer: exports, printing and OCR render without it, so nothing drawn with an + /// installed font is saved or printed (AGENTS.md §1.2, §1.4). + pub system_fonts: bool, } impl RenderConfig { fn settings(&self) -> InterpreterSettings { let standard = InterpreterSettings::default().font_resolver; + let system_fonts = self.system_fonts; InterpreterSettings { ocg_overrides: self.layers.clone(), hide_comments: self.hide_comments, - font_resolver: Arc::new(move |query| japanese_fallback(query).or_else(|| system_fallback(query)).or_else(|| standard(query))), + font_resolver: Arc::new(move |query| { + japanese_fallback(query).or_else(|| system_fonts.then(|| system_fallback(query)).flatten()).or_else(|| standard(query)) + }), ..InterpreterSettings::default() } } @@ -6751,12 +6759,20 @@ trailer << /Root 1 0 R >> %%EOF", content.len() ); - let mut r = PageRenderer::new(Arc::new(pdf.into_bytes()), RenderConfig::default()); - let p = r.render(RenderRequest { page: 0, kind: RequestKind::Pixels, tile: None, scale: 1.0, tag: 0, ..Default::default() }); - assert!(p.error.is_none(), "{base_font}: {:?}", p.error); - let inked = p.rgba.as_chunks::<4>().0.iter().filter(|c| c[0] < 128).count(); + let pdf = Arc::new(pdf.into_bytes()); + let ink = |config: RenderConfig| { + let mut r = PageRenderer::new(pdf.clone(), config); + let p = r.render(RenderRequest { page: 0, kind: RequestKind::Pixels, tile: None, scale: 1.0, tag: 0, ..Default::default() }); + assert!(p.error.is_none(), "{base_font}: {:?}", p.error); + p.rgba.as_chunks::<4>().0.iter().filter(|c| c[0] < 128).count() + }; // Two to four glyphs at 40 pt cover several hundred dark pixels; a blank run has none. - assert!(inked > 300, "{base_font}: the text is drawn ({inked} dark pixels)"); + let shown = ink(RenderConfig { system_fonts: true, ..RenderConfig::default() }); + assert!(shown > 300, "{base_font}: the text is drawn ({shown} dark pixels)"); + // Off by default: exports, printing and OCR render without an installed font, so + // nothing drawn with one can be saved (AGENTS.md §1.2, §1.4). + let saved = ink(RenderConfig::default()); + assert!(saved < 50, "{base_font}: not drawn without system_fonts ({saved} dark pixels)"); } } diff --git a/crates/render/src/system_font.rs b/crates/render/src/system_font.rs index 40543cf46e..5e6391f0d2 100644 --- a/crates/render/src/system_font.rs +++ b/crates/render/src/system_font.rs @@ -5,7 +5,8 @@ //! `MSung-Light` without embedding the font. hayro maps those CIDs to Unicode itself, but it needs //! a face to draw them with; without one the text is blank. The craft-fonts Japanese faces come //! first (see `japanese_fallback` in `raster.rs`); this module is the last resort: one face -//! already installed on this machine. It is read at runtime and never embedded or shipped +//! already installed on this machine, used for the viewer only (`RenderConfig::system_fonts`, off +//! for exports, printing and OCR). It is read at runtime and never embedded or shipped //! (AGENTS.md §1.4), Noto CJK and Source Han are left out on purpose (§1.1), and //! `PDFCRAFT_SYSTEM_FONTS=0` turns it off, as for the interface font. @@ -282,7 +283,10 @@ mod tests { assert!(c.path.is_absolute(), "{}", c.path.display()); assert!(c.path.extension().is_some_and(|e| e == "ttf" || e == "ttc"), "{}", c.path.display()); let name = c.path.to_string_lossy().to_lowercase(); - assert!(!["notosanscjk", "notoserifcjk", "sourcehan", "source-han"].iter().any(|b| name.contains(b)), "{name}"); + assert!( + !["notosanscjk", "notoserifcjk", "sourcehan", "source-han", "adobe", "kozuka", "minion"].iter().any(|b| name.contains(b)), + "{name}" + ); } } } From e42c21a3c833df30ba1144e33cf8f18e4a68f1f6 Mon Sep 17 00:00:00 2001 From: Philip Date: Wed, 21 Oct 2026 20:17:27 +0100 Subject: [PATCH 05/11] =?UTF-8?q?docs:=20AGENTS.md=20=C2=A71.4=20=E2=80=94?= =?UTF-8?q?=20the=20viewer=20may=20draw=20non-embedded=20CJK=20fonts=20wit?= =?UTF-8?q?h=20an=20installed=20face?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Proposed for the maintainers to accept or drop (separate commit): the page viewer, on screen only, may use one installed face for a CID font of an Adobe CJK collection that a PDF does not embed. Exports, printing, OCR and the automation tools never use it; nothing drawn with it is saved or printed; no Noto CJK, Source Han or Adobe family is a candidate. Co-Authored-By: Claude Sonnet 5.5 --- AGENTS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/AGENTS.md b/AGENTS.md index 0665ad1d3b..1cb3e1c34a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -71,6 +71,7 @@ Anything else is forbidden, including assets whose licence you cannot prove from - Font assets shared by the Crafting Apps, including every Japanese font, live in [`storytold/craft-fonts`](https://github.com/storytold/craft-fonts), not here. **Never commit a font file to this repository**; adding a font means adding it to craft-fonts. (The Latin UI fonts already in `assets/fonts/` — Inter, JetBrains Mono, Dancing Script — stay.) Team members: the shared rules are in [craftrules `standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md) (internal). - craft-fonts is an **optional build input**, never a Cargo dependency: `CRAFT_FONTS_DIR= cargo build` makes `crates/fonts/build.rs` embed its fonts as `pdfcraft_fonts::CRAFT_FONTS`. Without it PdfCraft builds, tests and runs, but has no Japanese face (Japanese UI text shows replacement boxes, and writing Japanese into a PDF returns a clear error). Release builds always set it (`.github/workflows/release.yml`), and `ATTRIBUTION.toml` lists it as a `[[build_input]]`. - The desktop app may read one font **already installed on the user's machine** at runtime, as the last interface fallback for characters no embedded face has (`crates/ui-egui/src/system_fonts.rs`). That font is never embedded, shipped, committed or copied, and never appears in published screenshots (`PDFCRAFT_SYSTEM_FONTS=0`, which `cargo xtask screenshots` sets). It is not a way around §1.2: nothing from it may enter a PDF, a package or the repository. +- The page viewer may also draw a CID font of an Adobe CJK collection (Japan1, GB1, CNS1, Korea1) that a PDF does **not embed** with one face already installed on the machine (`crates/render/src/system_font.rs`), **on screen only**. `RenderConfig::system_fonts` is off by default and is set only for the pool that draws the viewer (`display_config` in `crates/engine/src/lib.rs`), so exports, printing, OCR and the automation tools never use it. Like the interface font it is read at runtime and never embedded, shipped, committed or copied; Noto CJK and Source Han are never candidates (§1.1), nor is any Adobe family; and `PDFCRAFT_SYSTEM_FONTS=0` turns it off. It is not a way around §1.2 either: nothing drawn with it may be saved or printed. ### 1.5 Procedure for adding or changing an asset From 48955694765b65363ffd513122e9d5f7fabd0ef9 Mon Sep 17 00:00:00 2001 From: Philip Date: Wed, 21 Oct 2026 21:45:28 +0100 Subject: [PATCH 06/11] M2: review of #1567: the viewer-wiring test no longer skips when the wiring is lost CodeRabbit: the engine test skipped when the viewer's render was blank, so a viewer pool that lost the installed-font fallback would have skipped instead of failing. Whether this machine has a face that can draw the page is now asked of a renderer with RenderConfig::system_fonts switched on by hand, and the viewer's render must then draw it. Without the fallback on the viewer's pool the test fails. Co-Authored-By: Claude Sonnet 5.5 --- crates/engine/src/tests.rs | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index 5e6ef3b301..867afc8a07 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -425,10 +425,16 @@ trailer << /Root 1 0 R >> std::thread::yield_now(); }; let inked = |rgba: &[u8]| rgba.as_chunks::<4>().0.iter().filter(|c| c[0] < 128).count(); - if inked(&viewer.rgba) < 300 { + // Whether this machine has a face that can draw the page at all, asked of a renderer with the + // fallback switched on by hand, so a viewer pool that lost it fails below instead of skipping. + let probe = pdfcraft_render::PageRenderer::new(doc.display.clone(), pdfcraft_render::RenderConfig { system_fonts: true, ..Default::default() }) + .render(pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() }); + assert!(probe.error.is_none(), "{:?}", probe.error); + if inked(&probe.rgba) < 300 { eprintln!("no installed Traditional Chinese face (or PDFCRAFT_SYSTEM_FONTS=0): skipped"); return; } + assert!(inked(&viewer.rgba) > 300, "the viewer's pool draws it with the installed face ({} dark pixels)", inked(&viewer.rgba)); let png = export::Exporter::new(doc).png(0, 72.0).unwrap(); let exported = image::load_from_memory(&png).unwrap().into_rgba8(); assert!(inked(exported.as_raw()) < 50, "an export draws nothing with an installed font ({} dark pixels)", inked(exported.as_raw())); From 9745b7c5ccf6f8ff41de167d556190e7136fcc15 Mon Sep 17 00:00:00 2001 From: songzhihui Date: Mon, 26 Oct 2026 16:03:04 +0800 Subject: [PATCH 07/11] M2: match print-preview fonts to output policy --- ROADMAP.md | 4 ++ crates/render/src/raster.rs | 78 +++++++++++++++++++++++++++++++++++++ docs/roadmap.md | 6 ++- 3 files changed, 87 insertions(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index 88139fb145..dfc8b311d8 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,5 +1,7 @@ # PdfCraft roadmap +> **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: print preview excludes installed CJK fonts and isolates its font cache from the viewer; estimates unchanged) + > **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: reconcile current main while retaining display-only CJK fallback and upstream Japanese font selection; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M2, PR #4172: extend on-screen CJK fallback to Combine thumbnails and figure previews; recognise FangSong and subset-prefixed Founder names; estimates unchanged) @@ -757,3 +759,5 @@ Newest first: the date, what moved, and the new overall estimate. - **2026-10-22 (PR #3227 integration):** M11: integrate upstream `80b7d215`; retain partial target-version compatibility and upstream content-boundary and image-size regression coverage. Scope and estimates unchanged. - **2026-10-23 (M2, PR #4172):** Combine thumbnails and accessibility figure previews use the viewer's display-only CJK fallback. Windows family matching includes FangSong and subset-prefixed Founder names. Export, print, OCR and automation remain off by default; estimates unchanged. + +- **2026-10-26 (M2, PR #4172 print-preview review):** Print-preview requests disable installed CJK fallback and use an isolated cache so preview font availability matches output. Viewer fallback survives switching in either direction. Synthetic GBK regressions cover inline and single-worker rendering, full pages and tiles; estimates unchanged. diff --git a/crates/render/src/raster.rs b/crates/render/src/raster.rs index 0382ba67c7..592bbf536e 100644 --- a/crates/render/src/raster.rs +++ b/crates/render/src/raster.rs @@ -200,6 +200,7 @@ pub struct RenderRequest { /// The tag bit of a print-preview raster. Such requests always render the document's true /// colours, ignoring `RenderConfig::reading_colors`: a preview's job is fidelity to the /// printout, and the print itself — imposed from the document — is never recoloured. +/// Installed-font fallback is also disabled, matching the output's font policy. pub const PRINT_TAG: u64 = 1 << 60; /// The reading colours a request renders with: print previews ([`PRINT_TAG`]) ignore the @@ -337,6 +338,23 @@ fn render_page<'a>( config: &RenderConfig, req: RenderRequest, ) -> Result { + // Print previews share the viewer pool, but must use the output font policy. Keep the + // caller's cancellation/warning hooks, layers and annotation settings. A fresh cache is + // essential: hayro keys cached fonts by PDF dictionary, not by the font resolver, so the + // viewer cache could supply an installed face even after changing the resolver (and a + // cached missing font from print preview could blank the viewer on switching back). + let print_settings; + let print_cache; + let (settings, cache) = if config.system_fonts && req.tag & PRINT_TAG != 0 { + print_settings = InterpreterSettings { + font_resolver: RenderConfig { system_fonts: false, ..config.clone() }.settings().font_resolver, + ..settings.clone() + }; + print_cache = RenderCache::new(); + (&print_settings, &print_cache) + } else { + (settings, cache) + }; let reading = request_reading(config, &req); if req.kind == RequestKind::Text { return match catch_unwind(AssertUnwindSafe(|| crate::text::extract_page(pdf, req.page, settings))) { @@ -7909,6 +7927,66 @@ trailer << /Root 1 0 R >> } } + /// Print preview must match output even after the same worker cached an installed face. + /// Switching back to the viewer must not retain the print resolver's missing-font result. + #[cfg(not(target_arch = "wasm32"))] + #[test] + fn print_preview_excludes_installed_fonts_without_poisoning_viewer_cache() { + use hayro::hayro_interpret::font::FallbackFontQuery; + use hayro::hayro_interpret::hayro_cmap::CharacterCollection; + let query = FontQuery::Fallback(FallbackFontQuery { + post_script_name: Some("SimSun".into()), + character_collection: Some(CharacterCollection { family: CidFamily::AdobeGB1, supplement: 2 }), + ..Default::default() + }); + if super::system_fallback(&query).is_none() { + eprintln!("no installed GB1 face (or PDFCRAFT_SYSTEM_FONTS=0): skipped"); + return; + } + let content = "BT /F1 40 Tf 5 15 Td Tj ET"; + let bytes = Arc::new(format!( + "%PDF-1.7\n\ + 1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj\n\ + 2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj\n\ + 3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 120 60] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >> endobj\n\ + 4 0 obj << /Length {} >> stream\n{content}\nendstream endobj\n\ + 5 0 obj << /Type /Font /Subtype /Type0 /BaseFont /SimSun /Encoding /GBK-EUC-H /DescendantFonts [6 0 R] >> endobj\n\ + 6 0 obj << /Type /Font /Subtype /CIDFontType2 /BaseFont /SimSun /CIDSystemInfo << /Registry (Adobe) /Ordering (GB1) /Supplement 2 >> /FontDescriptor 7 0 R /DW 1000 >> endobj\n\ + 7 0 obj << /Type /FontDescriptor /FontName /SimSun /Flags 6 /FontBBox [0 -141 1000 859] /ItalicAngle 0 /Ascent 859 /Descent -141 /CapHeight 700 /StemV 80 >> endobj\n\ + trailer << /Root 1 0 R >>\n%%EOF", content.len() + ).into_bytes()); + let config = RenderConfig::default().on_screen(); + let mut inline = PageRenderer::new(bytes.clone(), config.clone()); + let mut output = PageRenderer::new(bytes.clone(), RenderConfig::default()); + let pool = RenderPool::new(bytes, 1, config); + for tile in [None, Some(Tile { x: 0, y: 0, w: 120, h: 60 })] { + // Cover both directions with a warm worker cache, including other caller tag bits. + for tag in [PRINT_TAG | 1, 2, PRINT_TAG | 3, 4] { + let req = RenderRequest { page: 0, scale: 1.0, tile, tag, ..Default::default() }; + let truth = output.render(req); + assert!(truth.error.is_none(), "{:?}", truth.error); + pool.set_queue(vec![req]); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(10); + let worker = loop { + if let Some(page) = pool.try_recv() { + break page; + } + assert!(std::time::Instant::now() < deadline, "print-preview worker timed out"); + std::thread::sleep(std::time::Duration::from_millis(5)); + }; + for page in [inline.render(req), worker] { + assert!(page.error.is_none(), "{:?}", page.error); + if tag & PRINT_TAG != 0 { + assert_eq!(page.rgba, truth.rgba, "print preview must match output, tile={tile:?}"); + } else { + let ink = page.rgba.as_chunks::<4>().0.iter().filter(|c| c[0] < 128).count(); + assert!(ink > 300, "viewer still draws installed Chinese glyphs: {ink}"); + } + } + } + } + } + /// Half-width katakana are common in Japanese documents set in a non-embedded Mincho font /// such as HeiseiMin-W3. The Mincho substitute must have those glyphs: one that lacks them /// (Shippori Mincho) draws them as .notdef. diff --git a/docs/roadmap.md b/docs/roadmap.md index 0ffcf06f9d..09c783e7a0 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,5 +1,7 @@ # PdfCraft roadmap: milestones and what's next +> **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: print-preview font policy matches output, with isolated caches and GBK switching regressions; estimates unchanged) + > **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: reconcile current main and retain Japanese serif/weight selection alongside display-only CJK fallback; estimates unchanged) > **Last reviewed:** 2026-10-25 | **Last updated:** 2026-10-25 | **Change:** minor (M14 localization current state: Arabic logical catalog, explicit RTL label direction and mirrored stock chrome; native/browser RTL workflows unqualified. Native release Hans face via LXGW WenKai Lite; estimates unchanged) @@ -160,6 +162,8 @@ M2 update (2026-10-23, #234 / continuation of #1567): the desktop viewer uses in faces for non-embedded CID fonts, including Windows GBK-EUC-H books. Windows Song/Hei/Kai families are retained when installed, and reload retains the fallback. This is display-only; export, printing, OCR, automation and machines without a candidate retain their limitations. +Print preview excludes installed faces and isolates its font cache from the viewer, matching +output font availability even after repeated view/print switching (2026-10-26, PR #4172). The M2 readiness and remaining-work estimates below are unchanged. "Plan size" is the original single-agent sizing. "Done" is the estimated share of that milestone's @@ -171,7 +175,7 @@ total matches the parity estimate. |---|---|---|---|---|---| | M0 | Skeleton: workspace, xtask gates, CI | 15–30 | 98% | 1–2 | GitHub workflow, `deny.toml` (licence audit of every dependency), parity checklist (826 features, `xtask parity`) done. Missing: remaining crate stubs, testkit/oracle crates | | M1 | COS: filters, crypt, parser, xref, writer | 120–200 | 80% | 20–40 | Done:
- filters and crypt: every standard-security revision R2–R6 (RC4, AES-128/256), SASLprep, permissions, creating encryption;
- cos: parse and repair (including invalid xref object-number ranges), decoded-stream limits including unfiltered data, decrypt on load, re-encrypt on save, incremental and full writing.
Corpus: open/edit/save passes on 958 files, and all 7 password-protected files open.
Full saves now pack objects into compressed object streams. An unchanged incremental save of a classic table writes a zero-length cross-reference subsection instead of a section with no subsections (#1339). A full rewrite keeps unknown trailer entries, including objects reachable only from them (#1338). Fuzzing runs nightly (`xtask fuzz`). Raw-deflate fallback preserves decoding-limit errors, including through tolerant COS stream decoding. Done since: #2658: macOS atomic saves retain destination extended attributes, including Finder tags, and reject partial resource-fork reads after size changes. Metadata failures retain the original PDF and unsaved edit. Missing: ≥ 250 tests, own image codecs | -| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. Forms without Resources use page Properties for ActualText and layer visibility; own dictionaries stop fallback. Oversized missing inline-image samples, including header and terminator overhead, no longer pin the content window (#3107 / PR #3438); normal large streamed images retain deterministic sample boundaries. Radial luminosity fades have synthetic coverage for the existing #2132 mask guard, including per-pixel references, antialiased clips, previews, scales and tiles; no additional production fix was needed. Dotted TOC leaders no longer inflate letter tracking or hide word spaces (#3086); phrase search and selection are covered. Short tracked ellipses retain their gap samples and remain searchable. Missing spacing glyphs in non-embedded Japanese CID substitutes render blank without changing PDF advances or Unicode (#3601). The desktop viewer can use installed CJK fonts, including Windows GBK Song/Hei/Kai/FangSong and subset-prefixed Founder names; Combine thumbnails and accessibility figure previews use the same display-only fallback, without changing export or print font policy (#234, continuation of #1567, review #4172). | +| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. Forms without Resources use page Properties for ActualText and layer visibility; own dictionaries stop fallback. Oversized missing inline-image samples, including header and terminator overhead, no longer pin the content window (#3107 / PR #3438); normal large streamed images retain deterministic sample boundaries. Radial luminosity fades have synthetic coverage for the existing #2132 mask guard, including per-pixel references, antialiased clips, previews, scales and tiles; no additional production fix was needed. Dotted TOC leaders no longer inflate letter tracking or hide word spaces (#3086); phrase search and selection are covered. Short tracked ellipses retain their gap samples and remain searchable. Missing spacing glyphs in non-embedded Japanese CID substitutes render blank without changing PDF advances or Unicode (#3601). The desktop viewer can use installed CJK fonts, including Windows GBK Song/Hei/Kai/FangSong and subset-prefixed Founder names; Combine thumbnails and accessibility figure previews use the same display-only fallback, without changing export or print font policy (#234, continuation of #1567, review #4172). Print previews exclude installed faces and isolate their font cache; GBK regressions cover view/print switching for inline and pooled whole-page/tile rendering. | | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. The renderer keeps its one-point crop-size floor in physical points after UserUnit scaling, with regression coverage for crop origins, rotations and f32 reciprocal rounding. Existing unit policy and nearly-zero fallback are retained. | | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings | | M3 | Viewer app (native + web) | 80–150 | 90% | 10–20 | Acrobat-style shell, find, select, panels, tiles, web build, UI control channel for agents (opt-in). #3808: the Pages inspector offers a 50–150% thumbnail slider and buttons, also available through `ui.set`. Missing: 60 fps test on a 500-page document, snapshot tests of every panel
Done since: fit visible, document title in the window; Linux middle-button scrolling; shared three-state theme preferences in the toolbar, View menu and control channel; numeric Linux portal events and idle wakeups completing #881, with native appearance inheritance retained (#446); late browser startup downloads preserve the current document and Save target (#171). Selected-text Copy in Edit and the canvas context menu is covered by focused regressions and native clipboard checks (#1225); document copy restrictions apply to the floating action too. #1364 candidate: absolute local GoToR links ask before target access; headless callers explicitly opt in. Native-opening candidate: chosen tools survive preparation and password retries; startup intake waits for pending preparation. Page-label navigation displays logical labels, retains the current page on unchanged submission and refreshes focused same-document replacements; clean file-watcher reload preserves the chosen scrolling mode, preventing a Single Page reader from jumping to page 1 during replacement. Synthetic control regressions cover these behaviors. #3149: page-list changes discard render demand based on old geometry; actual raster dimensions are checked against the GPU limit before upload. Synthetic crash and continued-rendering regressions cover the replacement path. Save-prompt shortcuts yield to dialogs above them; stacked link-consent UI regression covers Escape and Return. Comment hover follows individual marked lines and original annotation paint order (#3541/#3620), independently of spatial inspection sorting; synthetic rotation, differing-upper-edge overlap and unchanged-document regressions. Optional page gaps join page edges without cropping PDF margins; current-document toggle, persisted default and UI control options are covered by viewer tests. Gap changes in Organize Pages preserve document scroll position, and both comparison panes follow menu and control choices. Repeated window resizes by an edge and moves by the tab strip on GNOME/Wayland work without a click in between. | From 3994c2ca9d42a8d08b17571c80d37d536cafcb9b Mon Sep 17 00:00:00 2001 From: songzhihui Date: Mon, 26 Oct 2026 16:39:20 +0800 Subject: [PATCH 08/11] M2: document installed CJK font resolution contracts --- ROADMAP.md | 4 ++++ crates/render/src/raster.rs | 4 ++++ crates/render/src/system_font.rs | 23 +++++++++++++++++++++++ docs/roadmap.md | 4 +++- 4 files changed, 34 insertions(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index dfc8b311d8..ba657c4d92 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,5 +1,7 @@ # PdfCraft roadmap +> **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: document installed-font resolution helpers and regression contracts; behavior and estimates unchanged) + > **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: print preview excludes installed CJK fonts and isolates its font cache from the viewer; estimates unchanged) > **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: reconcile current main while retaining display-only CJK fallback and upstream Japanese font selection; estimates unchanged) @@ -761,3 +763,5 @@ Newest first: the date, what moved, and the new overall estimate. - **2026-10-23 (M2, PR #4172):** Combine thumbnails and accessibility figure previews use the viewer's display-only CJK fallback. Windows family matching includes FangSong and subset-prefixed Founder names. Export, print, OCR and automation remain off by default; estimates unchanged. - **2026-10-26 (M2, PR #4172 print-preview review):** Print-preview requests disable installed CJK fallback and use an isolated cache so preview font availability matches output. Viewer fallback survives switching in either direction. Synthetic GBK regressions cover inline and single-worker rendering, full pages and tiles; estimates unchanged. + +- **2026-10-26 (M2, PR #4172 documentation review):** Added function documentation for installed-font script selection, bounded candidate reads, platform paths, resolver precedence and regression contracts. No runtime behavior or estimate change. diff --git a/crates/render/src/raster.rs b/crates/render/src/raster.rs index 592bbf536e..74c4586dc8 100644 --- a/crates/render/src/raster.rs +++ b/crates/render/src/raster.rs @@ -88,6 +88,10 @@ impl RenderConfig { Self { system_fonts: true, ..self } } + /// Build interpreter settings with layer and comment preferences and a font resolver that + /// tries bundled Japanese faces, then opt-in installed CJK faces, then standard substitutes. + /// Passwords belong to parser construction; request-specific preview overrides are applied + /// by `render_page`, which also preserves the caller's warning and cancellation hooks. fn settings(&self) -> InterpreterSettings { let standard = InterpreterSettings::default().font_resolver; let system_fonts = self.system_fonts; diff --git a/crates/render/src/system_font.rs b/crates/render/src/system_font.rs index a5db18dfc5..8fcd167415 100644 --- a/crates/render/src/system_font.rs +++ b/crates/render/src/system_font.rs @@ -38,6 +38,8 @@ enum Script { } impl Script { + /// Map the four supported Adobe CJK collections to their writing systems. Identity and + /// custom collections provide no regional hint and cannot select an installed face here. fn of(family: &CidFamily) -> Option { match family { CidFamily::AdobeJapan1 => Some(Self::Japanese), @@ -133,6 +135,9 @@ pub(crate) fn fallback(query: &FallbackFontQuery) -> Option<(FontData, u32)> { found.as_ref().map(|(bytes, index)| (Arc::clone(bytes) as FontData, *index)) } +/// Read a candidate through the bounded file cache and select a face that maps `probe`, +/// preferring the requested region and weight. Unreadable files and unsuitable faces return +/// `None`, allowing the caller to try the next candidate without changing the PDF. fn read(candidate: &Candidate, probe: char, bold: bool) -> Option<(Arc>, u32)> { let bytes = file_bytes(&candidate.path)?; let index = pick_face(&bytes, probe, candidate.region, bold)?; @@ -197,6 +202,10 @@ fn pick_face(bytes: &[u8], probe: char, region: &[&str], bold: bool) -> Option Vec { let region: &'static [&'static str] = match script { Script::Japanese => &["JP", "Japanese"], @@ -283,6 +292,8 @@ fn macos_font_asset_dirs() -> &'static [PathBuf] { }) } +/// Locate the Windows Fonts directory using `WINDIR`, then `SystemRoot`, then `C:\Windows`. +/// This only constructs the path; candidate reads still enforce regular-file and size limits. fn windows_fonts() -> PathBuf { let dir = std::env::var_os("WINDIR").or_else(|| std::env::var_os("SystemRoot")).map_or_else(|| PathBuf::from(r"C:\Windows"), PathBuf::from); dir.join("Fonts") @@ -294,6 +305,8 @@ mod tests { const SCRIPTS: [Script; 4] = [Script::Japanese, Script::SimplifiedChinese, Script::TraditionalChinese, Script::Korean]; + /// Standard, localized and subset-prefixed names retain their Chinese family hint; + /// unrelated Latin names and generic YaHei requests use the default candidate list. #[test] fn chinese_family_hints_include_subset_and_localized_names() { for name in ["SimSun", "ABCDEF+SimSun", "STSong-Light", "宋体"] { @@ -340,6 +353,8 @@ mod tests { assert!(fangsong.first().is_some_and(|c| c.path.ends_with("simfang.ttf"))); } + /// Invalid font bytes, missing paths and directories produce no candidate face rather + /// than preventing the resolver from trying a later installed font. #[test] fn broken_and_missing_files_are_skipped() { assert_eq!(pick_face(b"", '\u{7269}', &[], false), None); @@ -372,6 +387,8 @@ mod tests { assert!(file_bytes(missing).is_none() && file_bytes(missing).is_none()); } + /// A Latin-only face fails every CJK probe even though its Latin glyph remains readable, + /// distinguishing unsupported script coverage from an unparseable font file. #[test] fn a_latin_face_is_rejected_for_every_script() { let inter = include_bytes!("../../../assets/fonts/Inter-Regular.ttf"); @@ -381,6 +398,8 @@ mod tests { assert_eq!(pick_face(inter, 'A', &[], false), Some(0)); } + /// Platform candidates are absolute TrueType paths and exclude the font families barred + /// by the asset policy, independently of whether those files are installed locally. #[test] fn candidates_are_absolute_font_files_without_adobe_designs() { for script in SCRIPTS { @@ -398,6 +417,8 @@ mod tests { } } + /// The four regional Adobe collections select their corresponding scripts; Identity + /// carries no regional information and must not acquire an installed fallback by guess. #[test] fn only_the_four_adobe_collections_have_a_script() { assert_eq!(Script::of(&CidFamily::AdobeCNS1), Some(Script::TraditionalChinese)); @@ -465,6 +486,8 @@ mod tests { #[cfg(target_os = "macos")] #[test] fn macos_candidates_include_a_pingfang_that_lives_in_the_font_assets() { + /// Independently locate an installed asset by filename within the test's depth limit, + /// providing an oracle for the production candidate list without using its discovery. fn find(dir: &Path, name: &str, depth: u32) -> Option { for entry in std::fs::read_dir(dir).ok()?.flatten() { let path = entry.path(); diff --git a/docs/roadmap.md b/docs/roadmap.md index 09c783e7a0..080416c1a6 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,5 +1,7 @@ # PdfCraft roadmap: milestones and what's next +> **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: document installed-font helpers and regression contracts; behavior and estimates unchanged) + > **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: print-preview font policy matches output, with isolated caches and GBK switching regressions; estimates unchanged) > **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: reconcile current main and retain Japanese serif/weight selection alongside display-only CJK fallback; estimates unchanged) @@ -175,7 +177,7 @@ total matches the parity estimate. |---|---|---|---|---|---| | M0 | Skeleton: workspace, xtask gates, CI | 15–30 | 98% | 1–2 | GitHub workflow, `deny.toml` (licence audit of every dependency), parity checklist (826 features, `xtask parity`) done. Missing: remaining crate stubs, testkit/oracle crates | | M1 | COS: filters, crypt, parser, xref, writer | 120–200 | 80% | 20–40 | Done:
- filters and crypt: every standard-security revision R2–R6 (RC4, AES-128/256), SASLprep, permissions, creating encryption;
- cos: parse and repair (including invalid xref object-number ranges), decoded-stream limits including unfiltered data, decrypt on load, re-encrypt on save, incremental and full writing.
Corpus: open/edit/save passes on 958 files, and all 7 password-protected files open.
Full saves now pack objects into compressed object streams. An unchanged incremental save of a classic table writes a zero-length cross-reference subsection instead of a section with no subsections (#1339). A full rewrite keeps unknown trailer entries, including objects reachable only from them (#1338). Fuzzing runs nightly (`xtask fuzz`). Raw-deflate fallback preserves decoding-limit errors, including through tolerant COS stream decoding. Done since: #2658: macOS atomic saves retain destination extended attributes, including Finder tags, and reject partial resource-fork reads after size changes. Metadata failures retain the original PDF and unsaved edit. Missing: ≥ 250 tests, own image codecs | -| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. Forms without Resources use page Properties for ActualText and layer visibility; own dictionaries stop fallback. Oversized missing inline-image samples, including header and terminator overhead, no longer pin the content window (#3107 / PR #3438); normal large streamed images retain deterministic sample boundaries. Radial luminosity fades have synthetic coverage for the existing #2132 mask guard, including per-pixel references, antialiased clips, previews, scales and tiles; no additional production fix was needed. Dotted TOC leaders no longer inflate letter tracking or hide word spaces (#3086); phrase search and selection are covered. Short tracked ellipses retain their gap samples and remain searchable. Missing spacing glyphs in non-embedded Japanese CID substitutes render blank without changing PDF advances or Unicode (#3601). The desktop viewer can use installed CJK fonts, including Windows GBK Song/Hei/Kai/FangSong and subset-prefixed Founder names; Combine thumbnails and accessibility figure previews use the same display-only fallback, without changing export or print font policy (#234, continuation of #1567, review #4172). Print previews exclude installed faces and isolate their font cache; GBK regressions cover view/print switching for inline and pooled whole-page/tile rendering. | +| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. Forms without Resources use page Properties for ActualText and layer visibility; own dictionaries stop fallback. Oversized missing inline-image samples, including header and terminator overhead, no longer pin the content window (#3107 / PR #3438); normal large streamed images retain deterministic sample boundaries. Radial luminosity fades have synthetic coverage for the existing #2132 mask guard, including per-pixel references, antialiased clips, previews, scales and tiles; no additional production fix was needed. Dotted TOC leaders no longer inflate letter tracking or hide word spaces (#3086); phrase search and selection are covered. Short tracked ellipses retain their gap samples and remain searchable. Missing spacing glyphs in non-embedded Japanese CID substitutes render blank without changing PDF advances or Unicode (#3601). The desktop viewer can use installed CJK fonts, including Windows GBK Song/Hei/Kai/FangSong and subset-prefixed Founder names; Combine thumbnails and accessibility figure previews use the same display-only fallback, without changing export or print font policy (#234, continuation of #1567, review #4172). Print previews exclude installed faces and isolate their font cache; GBK regressions cover view/print switching for inline and pooled whole-page/tile rendering. Installed-font helpers and their regression contracts have function documentation. | | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. The renderer keeps its one-point crop-size floor in physical points after UserUnit scaling, with regression coverage for crop origins, rotations and f32 reciprocal rounding. Existing unit policy and nearly-zero fallback are retained. | | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings | | M3 | Viewer app (native + web) | 80–150 | 90% | 10–20 | Acrobat-style shell, find, select, panels, tiles, web build, UI control channel for agents (opt-in). #3808: the Pages inspector offers a 50–150% thumbnail slider and buttons, also available through `ui.set`. Missing: 60 fps test on a 500-page document, snapshot tests of every panel
Done since: fit visible, document title in the window; Linux middle-button scrolling; shared three-state theme preferences in the toolbar, View menu and control channel; numeric Linux portal events and idle wakeups completing #881, with native appearance inheritance retained (#446); late browser startup downloads preserve the current document and Save target (#171). Selected-text Copy in Edit and the canvas context menu is covered by focused regressions and native clipboard checks (#1225); document copy restrictions apply to the floating action too. #1364 candidate: absolute local GoToR links ask before target access; headless callers explicitly opt in. Native-opening candidate: chosen tools survive preparation and password retries; startup intake waits for pending preparation. Page-label navigation displays logical labels, retains the current page on unchanged submission and refreshes focused same-document replacements; clean file-watcher reload preserves the chosen scrolling mode, preventing a Single Page reader from jumping to page 1 during replacement. Synthetic control regressions cover these behaviors. #3149: page-list changes discard render demand based on old geometry; actual raster dimensions are checked against the GPU limit before upload. Synthetic crash and continued-rendering regressions cover the replacement path. Save-prompt shortcuts yield to dialogs above them; stacked link-consent UI regression covers Escape and Return. Comment hover follows individual marked lines and original annotation paint order (#3541/#3620), independently of spatial inspection sorting; synthetic rotation, differing-upper-edge overlap and unchanged-document regressions. Optional page gaps join page edges without cropping PDF margins; current-document toggle, persisted default and UI control options are covered by viewer tests. Gap changes in Organize Pages preserve document scroll position, and both comparison panes follow menu and control choices. Repeated window resizes by an edge and moves by the tab strip on GNOME/Wayland work without a click in between. | From 0817ce2774c000279e9a11c9fdfe510aba4ac811 Mon Sep 17 00:00:00 2001 From: songzhihui Date: Tue, 27 Oct 2026 14:12:06 +0800 Subject: [PATCH 09/11] M2: keep roadmap review history in continuous blockquotes --- ROADMAP.md | 5 ----- docs/roadmap.md | 5 ----- 2 files changed, 10 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 264b308680..ada909fabd 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,15 +1,10 @@ # PdfCraft roadmap > **Last reviewed:** 2026-10-27 · **Last updated:** 2026-10-27 · **Change:** patch (M2, PR #4172: integrate main 1dd2737b; preserve viewer-only CJK fallback and regression coverage across engine, renderer and Combine module splits; estimates unchanged) - > **Last reviewed:** 2026-10-27 · **Last updated:** 2026-10-27 · **Change:** patch (M2, PR #4172: integrate main 9a9813bf and preserve viewer-only CJK fallback through the engine module split; estimates unchanged) - > **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: document installed-font resolution helpers and regression contracts; behavior and estimates unchanged) - > **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: print preview excludes installed CJK fonts and isolates its font cache from the viewer; estimates unchanged) - > **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: reconcile current main while retaining display-only CJK fallback and upstream Japanese font selection; estimates unchanged) - > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M2, PR #4172: extend on-screen CJK fallback to Combine thumbnails and figure previews; recognise FangSong and subset-prefixed Founder names; estimates unchanged) **Stage: alpha** · next: beta, ~25% (ready for real work 50% → 75%) and ~400–700 h away diff --git a/docs/roadmap.md b/docs/roadmap.md index 5b79c0fa67..211fa11466 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,15 +1,10 @@ # PdfCraft roadmap: milestones and what's next > **Last reviewed:** 2026-10-27 · **Last updated:** 2026-10-27 · **Change:** patch (M2, PR #4172: integrate main 1dd2737b; preserve viewer-only CJK fallback and regression coverage across engine, renderer and Combine module splits; estimates unchanged) - > **Last reviewed:** 2026-10-27 · **Last updated:** 2026-10-27 · **Change:** patch (M2, PR #4172: integrate main 9a9813bf and migrate viewer-only CJK configuration to the split engine modules; estimates unchanged) - > **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: document installed-font helpers and regression contracts; behavior and estimates unchanged) - > **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: print-preview font policy matches output, with isolated caches and GBK switching regressions; estimates unchanged) - > **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** patch (M2, PR #4172: reconcile current main and retain Japanese serif/weight selection alongside display-only CJK fallback; estimates unchanged) - > **Last reviewed:** 2026-10-25 | **Last updated:** 2026-10-25 | **Change:** minor (M14 localization current state: Arabic logical catalog, explicit RTL label direction and mirrored stock chrome; native/browser RTL workflows unqualified. Native release Hans face via LXGW WenKai Lite; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M7, #2636 partial: pasting added text boxes and images in the engine and through `content_copy`; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M11: image optimization accounts for page-local UserUnit; estimates unchanged) From 503346efed84416cf167e8bb095c1cfe0f64338d Mon Sep 17 00:00:00 2001 From: songzhihui Date: Wed, 28 Oct 2026 08:32:17 +0800 Subject: [PATCH 10/11] M2: keep CJK roadmap separators inside blockquote --- docs/roadmap.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index d6290a0ced..1ec73835e8 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -15,9 +15,9 @@ > **Last reviewed:** 2026-10-23 | **Last updated:** 2026-10-23 | **Change:** minor (M10: table spans retain text from covered slots in Office exports; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M10: image export preserves soft-mask Decode opacity ranges; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M10: vector printing preserves existing page XObjects when adding annotation appearances; estimates unchanged) - +> > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M2, PR #4172: CJK fallback in Combine thumbnails and figure previews; FangSong and subset-prefixed Founder family matching; estimates unchanged) - +> > **Last reviewed:** 2026-10-22 · **Last updated:** 2026-10-22 · **Change:** minor (M14, #3558: rebase guarded macOS per-file association recovery onto current main; estimates unchanged) > **Last reviewed:** 2026-10-20 · **Last updated:** 2026-10-20 · **Change:** minor (M12 #3673: Character encoding accepts Identity-H/V with Adobe CJK collections; estimates unchanged) From a57095c872b903a6dae598c9b353cbcd667a37f4 Mon Sep 17 00:00:00 2001 From: songzhihui Date: Sat, 31 Oct 2026 01:39:00 +0800 Subject: [PATCH 11/11] M2: assert exact GBK glyph rasters for installed Chinese families --- ROADMAP.md | 3 + crates/render/src/raster/tests.rs | 7 +++ crates/render/src/raster/tests/gbk_glyphs.rs | 60 ++++++++++++++++++++ docs/roadmap.md | 4 +- 4 files changed, 73 insertions(+), 1 deletion(-) create mode 100644 crates/render/src/raster/tests/gbk_glyphs.rs diff --git a/ROADMAP.md b/ROADMAP.md index d30325d3e3..76e86471b6 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,5 +1,6 @@ # PdfCraft roadmap +> **Last reviewed:** 2026-10-31 | **Last updated:** 2026-10-31 | **Change:** patch (M2, PR #4172: GBK rasters compared with explicit Unicode and independently selected installed faces; wrong-glyph and wrong-family negative controls fail; estimates unchanged) > **Last reviewed:** 2026-10-31 | **Last updated:** 2026-10-31 | **Change:** patch (M2, PR #4172: integrate main fa5f9f8c; retain CJK viewer and upstream raster regressions, restore explicit display_config imports; estimates unchanged) > **Last reviewed:** 2026-10-28 · **Last updated:** 2026-10-28 · **Change:** patch (M2, PR #4172: integrate main 5af10276; retain both roadmap histories and viewer-only CJK font policy; estimates unchanged) > **Last reviewed:** 2026-10-27 · **Last updated:** 2026-10-27 · **Change:** patch (M2, PR #4172: integrate main 1dd2737b; preserve viewer-only CJK fallback and regression coverage across engine, renderer and Combine module splits; estimates unchanged) @@ -212,6 +213,8 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md ## Progress log +- **2026-10-31 (M2, PR #4172 review):** Added same-machine pixel oracles for GBK Song, Hei, Kai and FangSong, including all four subset-prefixed Founder aliases. Reference PDFs use explicit Unicode mappings and independently selected installed faces; font data and rasters remain in memory. All eight cases ran; wrong-glyph and wrong-family production mutations each failed the new assertion and were restored. Validation: 422 renderer unit tests, renderer all-target Clippy with -D warnings, formatting, layers/assets/parity and renderer WASM checks passed; existing optional-patch/WASM warnings and parity notes remain. Estimates unchanged. + - **2026-10-31 (M2, PR #4172):** Integrated main fa5f9f8c, retaining both roadmap histories and raster regression groups. Restored display_config and thumbnail_config imports after the upstream explicit-import migration. Validation: 422 renderer unit tests, engine/UI all-target checks, scoped all-target Clippy with -D warnings, formatting, layers, assets, parity and renderer/engine WASM checks passed. Existing optional-patch/WASM warnings and parity notes remain; no full engine/UI runtime test rerun. Display-only font policy and estimates unchanged. - **2026-10-28 (M2, PR #4172):** Integrated main 5af10276, preserving both roadmap histories and reviewing auto-merged export tests and accessibility UI changes. Validation: 488 scoped tests passed (18 engine, 412 renderer unit, 43 renderer integration, 15 Combine; one existing renderer integration test ignored); scoped all-target Clippy with -D warnings, formatting, layers, assets, parity and renderer/engine WASM checks passed. Viewer-only CJK fallback and print/output font policy remain unchanged; estimates unchanged. diff --git a/crates/render/src/raster/tests.rs b/crates/render/src/raster/tests.rs index 4741b7fd63..df6896b980 100644 --- a/crates/render/src/raster/tests.rs +++ b/crates/render/src/raster/tests.rs @@ -7040,6 +7040,9 @@ fn test_clip_only_text_raster_preserves_clipping_without_painting() { assert!(pixels.contains(&white), "the clip is text-shaped, not the whole page"); } +#[cfg(target_os = "windows")] +mod gbk_glyphs; + /// Chinese and Korean CID fonts that aren't embedded (Adobe-CNS1 `MSung-Light`, Adobe-GB1 /// `STSong-Light`, Adobe-Korea1 `HYGoThic-Medium`; Acrobat also writes Japanese `KozGoPr6N` /// text boxes without the font) drew nothing, because craft-fonts has only Japanese faces. @@ -7061,6 +7064,8 @@ fn non_embedded_chinese_and_korean_cid_fonts_draw_with_an_installed_face() { ("KaiTi", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), ("FangSong", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), ("ABCDEF+FZSSK--GBK1-0", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), + ("ABCDEF+FZHTK--GBK1-0", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), + ("ABCDEF+FZKTK--GBK1-0", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), ("ABCDEF+FZFSK--GBK1-0", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), ]; for (base_font, family, ordering, encoding, code, subtype) in cases { @@ -7105,6 +7110,8 @@ trailer << /Root 1 0 R >> let saved = ink(RenderConfig::default()); assert!(saved < 50, "{base_font}: not drawn without system_fonts ({saved} dark pixels)"); if encoding == "GBK-EUC-H" { + #[cfg(target_os = "windows")] + gbk_glyphs::assert_expected_glyphs(&pdf, base_font); let mut renderer = PageRenderer::new(pdf, RenderConfig { system_fonts: true, ..Default::default() }); let text = renderer.render(RenderRequest { page: 0, kind: RequestKind::Text, ..Default::default() }); assert!(text.error.is_none(), "{base_font}: {:?}", text.error); diff --git a/crates/render/src/raster/tests/gbk_glyphs.rs b/crates/render/src/raster/tests/gbk_glyphs.rs new file mode 100644 index 0000000000..8443fed624 --- /dev/null +++ b/crates/render/src/raster/tests/gbk_glyphs.rs @@ -0,0 +1,60 @@ +//! A same-machine pixel oracle, independent of GBK decoding and installed-face selection. +//! All font reads and reference rasters stay in memory; nothing is embedded or published. + +use super::*; +use hayro::hayro_interpret::font::FontData; +use skrifa::MetadataProvider as _; +use skrifa::string::StringId; + +/// Compare the real GBK PDF path with explicit Unicode and a known installed face. Unlike an +/// ink count or extracted text, this fails if drawing chooses another glyph or font family. +pub(super) fn assert_expected_glyphs(pdf: &Arc>, base_font: &str) { + let (expected, file) = match base_font { + "SimSun" | "ABCDEF+FZSSK--GBK1-0" => ("SimSun", "simsun.ttc"), + "SimHei" | "ABCDEF+FZHTK--GBK1-0" => ("SimHei", "simhei.ttf"), + "KaiTi" | "ABCDEF+FZKTK--GBK1-0" => ("KaiTi", "simkai.ttf"), + "FangSong" | "ABCDEF+FZFSK--GBK1-0" => ("FangSong", "simfang.ttf"), + other => panic!("no independent GBK oracle for {other}"), + }; + // Do not use the production candidate list, resolver or face-selection helpers here. + let directory = std::env::var_os("WINDIR").or_else(|| std::env::var_os("SystemRoot")).unwrap_or_else(|| r"C:\Windows".into()); + let path = std::path::PathBuf::from(directory).join("Fonts").join(file); + if !path.is_file() { + eprintln!("{base_font}: exact {expected} face unavailable; pixel oracle skipped"); + return; + } + let bytes = Arc::new(std::fs::read(path).expect("read installed reference font")); + let index = (0..32) + .find(|&index| { + skrifa::FontRef::from_index(&bytes, index) + .is_ok_and(|font| font.localized_strings(StringId::POSTSCRIPT_NAME).any(|name| name.chars().collect::() == expected)) + }) + .expect("reference file contains the expected PostScript face"); + let face = skrifa::FontRef::from_index(&bytes, index).unwrap(); + for character in ['\u{4e2d}', '\u{6587}'] { + assert!(face.charmap().map(character).is_some_and(|id| id.to_u32() != 0), "{expected} must contain {character}"); + } + + // Explicit ToUnicode maps arbitrary reference codes to U+4E2D and U+6587, bypassing GBK + // and Adobe-GB1 CID lookup. The fixed resolver below bypasses family classification too. + let cmap = "begincmap /CMapType 2 def 1 begincodespacerange <0000> endcodespacerange \ + 2 beginbfchar <0001> <4E2D> <0002> <6587> endbfchar endcmap"; + let source = std::str::from_utf8(pdf).unwrap(); + assert!(source.contains("/Encoding /GBK-EUC-H") && source.contains("")); + let reference = source + .replace("/Encoding /GBK-EUC-H", "/Encoding /Identity-H /ToUnicode 8 0 R") + .replace("", "<00010002>") + .replace("trailer <<", &format!("8 0 obj << /Length {} >> stream\n{cmap}\nendstream endobj\ntrailer <<", cmap.len())); + let parsed = Pdf::new(Arc::new(reference.into_bytes())).unwrap(); + let data: FontData = bytes; + let settings = InterpreterSettings { font_resolver: Arc::new(move |_| Some((data.clone(), index))), ..Default::default() }; + let reference = hayro::render(&parsed.pages()[0], &RenderCache::new(), &settings, &RenderSettings { bg_color: WHITE, ..Default::default() }); + let mut renderer = PageRenderer::new(pdf.clone(), RenderConfig::default().on_screen()); + let actual = renderer.render(RenderRequest { page: 0, kind: RequestKind::Pixels, scale: 1.0, ..Default::default() }); + assert!(actual.error.is_none(), "{base_font}: {:?}", actual.error); + let expected_pixels = reference.data_as_u8_slice(); + assert_eq!(actual.rgba.len(), expected_pixels.len()); + assert!(expected_pixels.as_chunks::<4>().0.iter().filter(|pixel| pixel[0] < 128).count() > 300, "reference must draw two visible glyphs"); + assert!(actual.rgba.as_ref() == expected_pixels, "{base_font}: GBK raster must draw the exact {expected} glyphs for U+4E2D U+6587"); + eprintln!("{base_font}: exact {expected} Unicode-reference pixels matched"); +} diff --git a/docs/roadmap.md b/docs/roadmap.md index 7c30577652..12161c4dff 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,5 +1,6 @@ # PdfCraft roadmap: milestones and what's next +> **Last reviewed:** 2026-10-31 | **Last updated:** 2026-10-31 | **Change:** patch (M2, PR #4172: GBK rasters compared with explicit Unicode and independently selected installed faces; wrong-glyph and wrong-family negative controls fail; estimates unchanged) > **Last reviewed:** 2026-10-31 | **Last updated:** 2026-10-31 | **Change:** patch (M2, PR #4172: integrate main fa5f9f8c; retain CJK viewer and upstream raster regressions, restore explicit display_config imports; estimates unchanged) > **Last reviewed:** 2026-10-28 · **Last updated:** 2026-10-28 · **Change:** patch (M2, PR #4172: integrate main 5af10276; retain both roadmap histories and viewer-only CJK font policy; estimates unchanged) > **Last reviewed:** 2026-10-27 · **Last updated:** 2026-10-27 · **Change:** patch (M2, PR #4172: integrate main 1dd2737b; preserve viewer-only CJK fallback and regression coverage across engine, renderer and Combine module splits; estimates unchanged) @@ -190,7 +191,7 @@ total matches the parity estimate. |---|---|---|---|---|---| | M0 | Skeleton: workspace, xtask gates, CI | 15–30 | 98% | 1–2 | GitHub workflow, `deny.toml` (licence audit of every dependency), parity checklist (826 features, `xtask parity`) done. Missing: remaining crate stubs, testkit/oracle crates | | M1 | COS: filters, crypt, parser, xref, writer | 120–200 | 80% | 20–40 | Done:
- filters and crypt: every standard-security revision R2–R6 (RC4, AES-128/256), SASLprep, permissions, creating encryption;
- cos: parse and repair (including invalid xref object-number ranges), decoded-stream limits including unfiltered data, decrypt on load, re-encrypt on save, incremental and full writing.
Corpus: open/edit/save passes on 958 files, and all 7 password-protected files open.
Full saves now pack objects into compressed object streams. An unchanged incremental save of a classic table writes a zero-length cross-reference subsection instead of a section with no subsections (#1339). A full rewrite keeps unknown trailer entries, including objects reachable only from them (#1338). Fuzzing runs nightly (`xtask fuzz`). Raw-deflate fallback preserves decoding-limit errors, including through tolerant COS stream decoding. Done since: #2658: macOS atomic saves retain destination extended attributes, including Finder tags, and reject partial resource-fork reads after size changes. Metadata failures retain the original PDF and unsaved edit. Missing: ≥ 250 tests, own image codecs | -| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. Forms without Resources use page Properties for ActualText and layer visibility; own dictionaries stop fallback. Oversized missing inline-image samples, including header and terminator overhead, no longer pin the content window (#3107 / PR #3438); normal large streamed images retain deterministic sample boundaries. Radial luminosity fades have synthetic coverage for the existing #2132 mask guard, including per-pixel references, antialiased clips, previews, scales and tiles; no additional production fix was needed. Dotted TOC leaders no longer inflate letter tracking or hide word spaces (#3086); phrase search and selection are covered. Short tracked ellipses retain their gap samples and remain searchable. Missing spacing glyphs in non-embedded Japanese CID substitutes render blank without changing PDF advances or Unicode (#3601). The desktop viewer can use installed CJK fonts, including Windows GBK Song/Hei/Kai/FangSong and subset-prefixed Founder names; Combine thumbnails and accessibility figure previews use the same display-only fallback, without changing export or print font policy (#234, continuation of #1567, review #4172). Print previews exclude installed faces and isolate their font cache; GBK regressions cover view/print switching for inline and pooled whole-page/tile rendering. Installed-font helpers and their regression contracts have function documentation. Integration with main fa5f9f8c retains viewer fallback across the split engine session modules, renderer tests and Combine thumbnail workers. | +| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. Forms without Resources use page Properties for ActualText and layer visibility; own dictionaries stop fallback. Oversized missing inline-image samples, including header and terminator overhead, no longer pin the content window (#3107 / PR #3438); normal large streamed images retain deterministic sample boundaries. Radial luminosity fades have synthetic coverage for the existing #2132 mask guard, including per-pixel references, antialiased clips, previews, scales and tiles; no additional production fix was needed. Dotted TOC leaders no longer inflate letter tracking or hide word spaces (#3086); phrase search and selection are covered. Short tracked ellipses retain their gap samples and remain searchable. Missing spacing glyphs in non-embedded Japanese CID substitutes render blank without changing PDF advances or Unicode (#3601). The desktop viewer can use installed CJK fonts, including Windows GBK Song/Hei/Kai/FangSong and subset-prefixed Founder names; Combine thumbnails and accessibility figure previews use the same display-only fallback, without changing export or print font policy (#234, continuation of #1567, review #4172). Print previews exclude installed faces and isolate their font cache; GBK regressions cover view/print switching for inline and pooled whole-page/tile rendering. Installed-font helpers and their regression contracts have function documentation. GBK regression rasters match explicit-Unicode references with independently selected installed faces; wrong-glyph and wrong-family negative controls fail. Integration with main fa5f9f8c retains viewer fallback across the split engine session modules, renderer tests and Combine thumbnail workers. | | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. The renderer keeps its one-point crop-size floor in physical points after UserUnit scaling, with regression coverage for crop origins, rotations and f32 reciprocal rounding. Existing unit policy and nearly-zero fallback are retained. | | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings | | M3 | Viewer app (native + web) | 80–150 | 90% | 10–20 | Acrobat-style shell, find, select, panels, tiles, web build, UI control channel for agents (opt-in). #3808: the Pages inspector offers a 50–150% thumbnail slider and buttons, also available through `ui.set`. Missing: 60 fps test on a 500-page document, snapshot tests of every panel
Done since: fit visible, document title in the window; Linux middle-button scrolling; shared three-state theme preferences in the toolbar, View menu and control channel; numeric Linux portal events and idle wakeups completing #881, with native appearance inheritance retained (#446); late browser startup downloads preserve the current document and Save target (#171). Selected-text Copy in Edit and the canvas context menu is covered by focused regressions and native clipboard checks (#1225); document copy restrictions apply to the floating action too. #1364 candidate: absolute local GoToR links ask before target access; headless callers explicitly opt in. Native-opening candidate: chosen tools survive preparation and password retries; startup intake waits for pending preparation. Page-label navigation displays logical labels, retains the current page on unchanged submission and refreshes focused same-document replacements; clean file-watcher reload preserves the chosen scrolling mode, preventing a Single Page reader from jumping to page 1 during replacement. Synthetic control regressions cover these behaviors. #3149: page-list changes discard render demand based on old geometry; actual raster dimensions are checked against the GPU limit before upload. Synthetic crash and continued-rendering regressions cover the replacement path. Save-prompt shortcuts yield to dialogs above them; stacked link-consent UI regression covers Escape and Return. Comment hover follows individual marked lines and original annotation paint order (#3541/#3620), independently of spatial inspection sorting; synthetic rotation, differing-upper-edge overlap and unchanged-document regressions. Optional page gaps join page edges without cropping PDF margins; current-document toggle, persisted default and UI control options are covered by viewer tests. Gap changes in Organize Pages preserve document scroll position, and both comparison panes follow menu and control choices. Repeated window resizes by an edge and moves by the tab strip on GNOME/Wayland work without a click in between. | @@ -289,6 +290,7 @@ Estimates unchanged. | Date | Change | Summary | |---|---|---| +| 2026-10-31 | patch | M2, PR #4172: eight GBK font/alias cases compare exact pixels with independent installed-face and explicit-Unicode references; wrong-glyph and wrong-family negative controls fail. Estimates unchanged. | | 2026-10-31 | patch | M2, PR #4172: integrate main fa5f9f8c, preserve both raster test groups and restore explicit display_config imports; display-only CJK policy and estimates unchanged. | | 2026-10-28 | patch | M2, PR #4172: integrate main 5af10276 and retain both roadmap histories; viewer-only CJK and print/output font policies and estimates unchanged. | | 2026-10-24 | minor | M11: the PDF Optimizer and Reduce File Size store Group 3 fax images as Group 4 when smaller (`Document::fax_params`), and pack eight-bit `DeviceRGB` of only black and white (as grey) and indexed images of 2, 4 or 8 bits that use only palette entries 0 and 1 (keeping the palette) to one bit, so Monochrome Images compresses and downsamples them. Estimates unchanged. |