daniel543x/pdfcraft
1
0 Fork 0
mirror of https://github.com/storytold/pdfcraft.git synced 2026-11-10 21:11:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/main' into pr-fix/print-area

Browse Source
This commit is contained in:
Osvaldo Duran Melendez committed 2026-11-08 14:28:16 -05:00
2 parents ea0a2fe4e8 43f24aa169
commit 9822ec0500
141 files changed
+12947 -949
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.cargo/config.toml
+5
View File
@@ -1,2 +1,7 @@
[alias]
xtask = "run -p xtask --"
# WebAssembly SIMD (every current browser has it; there's no runtime detection on wasm). The OCR
# engine's kernels (rten-simd) and other vectorised code only use it when compiled in.
[target.wasm32-unknown-unknown]
rustflags = ["-C", "target-feature=+simd128"]
AGENTS.md
+4 -3
View File
@@ -1,4 +1,4 @@
<!-- craft-house-rules:start core source=dde09ea31e2e457f5b4653a6e76f7cdf55944bdf sha256=540e6cea75d12d0710c3dca89658328e81e508ecef102446ca5bb95b7c0b92bb -->
<!-- craft-house-rules:start core source=1315c63e17905daf7dbc6dc6f8c49384e415162b sha256=540e6cea75d12d0710c3dca89658328e81e508ecef102446ca5bb95b7c0b92bb -->
## House rules
These apply in every Craft repository. A later rule may be stricter, not weaker. The source is `standards/house-rules-core.md` in `storytold/craft-contrib`. Propose changes there, not in a copy.
@@ -19,7 +19,7 @@ These apply in every Craft repository. A later rule may be stricter, not weaker.
14. **Keep pull requests moving.** Concurrent pull requests are fine; unanswered actionable feedback is not. Respond in 7 days, a reminder at 14, closure eligible at 30, from when an actionable request waits on the author. Suggestions and unconfirmed bot findings do not start the clock. A fix, explanation or agreed plan resets it; unrelated activity does not. Waiting on a reviewer or maintainer pauses it and does not count against the author. A maintainer decides closure; it is never automatic. Reuse search results, do not poll, and stop at a rate limit.
15. **No agent attribution.** A pull request does not say an agent wrote it. A repository rule may not require that.
<!-- craft-house-rules:end -->
<!-- craft-house-rules:start rust source=dde09ea31e2e457f5b4653a6e76f7cdf55944bdf sha256=387e0ccb4b49e567f60951d7d6fff7305054c89f43ad812b9993dc7c38528013 -->
<!-- craft-house-rules:start rust source=1315c63e17905daf7dbc6dc6f8c49384e415162b sha256=642f46b4ac38cc9e4d10d1ccde19d68cb6bc03d20e4c6236a7535f3047566262 -->
## Rust Craft repositories
These apply to native Craft apps and to shared Rust libraries. Select them by repository type. A repository that merely contains some Rust does not take this block. Rules later in a repository's file may be stricter, but not weaker. The source is `standards/house-rules-rust.md` in [storytold/craft-contrib](https://github.com/storytold/craft-contrib).
@@ -40,6 +40,7 @@ Dependencies and assets use SPDX identifiers. An expression is allowed when ever
- `BSD-2-Clause`
- `BSD-3-Clause`
- `Apache-2.0`
- `Unicode-3.0`
- `OFL-1.1`
- `Ubuntu-font-1.0`
- `Bitstream-Vera`
@@ -83,7 +84,7 @@ An **asset** is any non-code material that the project includes, ships, embeds,
### 1.2 Only open, provable licences
Every asset must be one of:
- **Open source:** ISC, MIT, BSD-2-Clause, BSD-3-Clause, Apache-2.0, OFL-1.1, Ubuntu-Font-1.0 or Bitstream-Vera.
- **Open source:** ISC, MIT, BSD-2-Clause, BSD-3-Clause, Apache-2.0, OFL-1.1, Ubuntu-Font-1.0, Bitstream-Vera or Unicode-3.0 (Unicode, Inc. data such as ICU4X's line-breaking dictionaries; already allowed for code in `deny.toml`).
- **Creative Commons:** CC0-1.0, CC-BY-4.0 or CC-BY-SA-4.0.
- **Public domain.**
- **Contributor-original:** created, photographed or generated by a contributor who owns it and licenses it under the project licence (`MIT OR Apache-2.0`).
ATTRIBUTION.md
+2 -1
View File
@@ -267,7 +267,7 @@ Every asset PdfCraft includes, bundles or uses to build its published material,
| `assets/icons/a-large-small.svg` | Lucide icon "a-large-small" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/a-large-small.svg) | UI icon, embedded via crates/ui-egui/src/widgets/icon_data.rs |
| `assets/icons/move-horizontal.svg` | Lucide icon "move-horizontal" | Lucide Contributors (portions Cole Bemis, Feather) | ISC AND MIT | https://github.com/lucide-icons/lucide (npm lucide-static 1.49.0, icons/move-horizontal.svg) | UI icon, embedded via crates/ui-egui/src/icon_data.rs |
## Compiled in through dependencies (6)
## Compiled in through dependencies (7)
| Crate | File | Title | Author | Licence | Source | Used for |
|---|---|---|---|---|---|---|
@@ -277,6 +277,7 @@ Every asset PdfCraft includes, bundles or uses to build its published material,
| `epaint_default_fonts` 0.37.0 | `fonts/emoji-icon-font.ttf` | emoji-icon-font | John Slegers | MIT | https://github.com/jslegers/emoji-icon-font | egui monochrome fallback font (egui feature monochrome_emoji_fonts) |
| `epaint_default_fonts` 0.37.0 | `fonts/egui-icons.ttf` | egui icons (monochrome emoji-icon-font subset) | John Slegers | MIT | https://github.com/jslegers/emoji-icon-font | Small special-icon subset embedded by egui default_fonts; retained in FontDefinitions::default(). Provenance: https://github.com/storytold/egui/blob/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts/egui-icons.txt; complete MIT copyright and permission text in NOTICE. |
| `hayro-interpret` 0.7.0 | `src/font/generated/metrics.rs` | Standard-14 font metrics and encodings (as Rust tables) (non-visual data, AGENTS.md §1.1) | Adobe (Core 14 AFM metrics, PDF specification encodings); tables by The Hayro Authors | MIT OR Apache-2.0 | ISO 32000-2 Annex D; Adobe Core14 AFM files (via vendored hayro-interpret) | Glyph widths and encodings for non-embedded standard fonts. Non-visual data, allowed by AGENTS.md §1.1 |
| `icu_segmenter_data` 2.3.0 | `data/segmenter_dictionary_extended_v1.rs.data` | Southeast Asian word dictionaries for line breaking (Thai, Lao, Khmer, Burmese) | The ICU4X Project Developers; word lists from the ICU project (Unicode, Inc.) | Unicode-3.0 | https://github.com/unicode-org/icu4x (via icu_segmenter_data) | Word boundaries for wrapping Thai text in Edit text and Add text (crates/edit/src/linebreak.rs). Non-visual data; the line segmenter loads only these dictionaries, not the Chinese and Japanese one |
## Downloaded at build time (22)
ATTRIBUTION.toml
+12
View File
@@ -2263,6 +2263,18 @@ adobe_data = true
usage = "Glyph widths and encodings for non-embedded standard fonts. Non-visual data, allowed by AGENTS.md §1.1"
sha256 = "2ec9e608573eda459de1e1d9bda3bb494315e36bee885acc88912a11fe0caa41"
[[bundled]]
crate = "icu_segmenter_data"
version = "2.3.0"
path = "data/segmenter_dictionary_extended_v1.rs.data"
title = "Southeast Asian word dictionaries for line breaking (Thai, Lao, Khmer, Burmese)"
author = "The ICU4X Project Developers; word lists from the ICU project (Unicode, Inc.)"
source = "https://github.com/unicode-org/icu4x (via icu_segmenter_data)"
licence = "Unicode-3.0"
kind = "data"
usage = "Word boundaries for wrapping Thai text in Edit text and Add text (crates/edit/src/linebreak.rs). Non-visual data; the line segmenter loads only these dictionaries, not the Chinese and Japanese one"
sha256 = "cad3821ca44fdb88ed856ff3336444aa200afe7b9d19e34ac28ae6a4459d4c6d"
[[fetched]]
file = "PlayfairDisplay[wght].ttf"
url = "https://raw.githubusercontent.com/google/fonts/9710da1eacb3be272583c3224dcb70f9da6eadbb/ofl/playfairdisplay/PlayfairDisplay%5Bwght%5D.ttf"
Cargo.lock
Generated
+41 -3
View File
@@ -1243,7 +1243,7 @@ dependencies = [
[[package]]
name = "craft-print"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=787a7a0f972fd1d7a7618fdbeca3a5267a0407ff#787a7a0f972fd1d7a7618fdbeca3a5267a0407ff"
source = "git+https://github.com/storytold/craft-libs?rev=7ee35a9fbd47af572d55fd63dbf0333d1f00da17#7ee35a9fbd47af572d55fd63dbf0333d1f00da17"
dependencies = [
"craft-winprint-gdi",
"log",
@@ -1335,7 +1335,7 @@ dependencies = [
[[package]]
name = "craft-winprint-gdi"
version = "0.1.0"
source = "git+https://github.com/storytold/winprint.rs?rev=417ac3c4d14d3da5485a99c7d5ce8f05664ec312#417ac3c4d14d3da5485a99c7d5ce8f05664ec312"
source = "git+https://github.com/storytold/winprint.rs?rev=64e7afd4d1483512385012c340ecaaaf0f3beb7f#64e7afd4d1483512385012c340ecaaaf0f3beb7f"
dependencies = [
"windows-sys 0.61.2",
]
@@ -3133,6 +3133,28 @@ dependencies = [
"zerovec",
]
[[package]]
name = "icu_segmenter"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82d07aafccd67af15d02512a6adf5896fbc5ed00f2e99b471d2efa14016db3db"
dependencies = [
"icu_collections",
"icu_locale_fallback",
"icu_provider",
"icu_segmenter_data",
"potential_utf",
"smallvec",
"utf8_iter",
"zerovec",
]
[[package]]
name = "icu_segmenter_data"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae293c039020f9ec10710af98d29ce6aa2051486638b49c9a6409f3b4a9e98ad"
[[package]]
name = "idna"
version = "1.1.0"
@@ -4277,6 +4299,7 @@ dependencies = [
"hayro-syntax",
"pdfcraft-cos",
"pdfcraft-fonts",
"pdfcraft-geom",
"pdfcraft-model",
"thiserror 2.0.21",
]
@@ -4399,6 +4422,7 @@ name = "pdfcraft-edit"
version = "0.18.1"
dependencies = [
"hayro-syntax",
"icu_segmenter",
"kurbo 0.13.1",
"pdfcraft-annot",
"pdfcraft-content",
@@ -4539,10 +4563,22 @@ dependencies = [
"pdfcraft-fonts",
"png 0.18.1",
"rten",
"serde_json",
"sha2 0.11.1",
"tempfile",
"thiserror 2.0.21",
]
[[package]]
name = "pdfcraft-ocr-worker"
version = "0.18.1"
dependencies = [
"js-sys",
"pdfcraft-ocr",
"wasm-bindgen",
"web-sys",
]
[[package]]
name = "pdfcraft-optimize"
version = "0.18.1"
@@ -4637,6 +4673,7 @@ dependencies = [
"pdfcraft-filters",
"pdfcraft-fonts",
"pdfcraft-forms",
"pdfcraft-geom",
"pdfcraft-model",
"pdfcraft-render",
"pdfcraft-testkit",
@@ -4660,6 +4697,7 @@ dependencies = [
"lopdf",
"pdfcraft-cos",
"pdfcraft-fonts",
"pdfcraft-geom",
"serde",
"skrifa 0.44.0",
"thiserror 2.0.21",
@@ -8247,7 +8285,7 @@ dependencies = [
[[package]]
name = "winprint"
version = "0.2.1"
source = "git+https://github.com/storytold/winprint.rs?rev=5f80b9baf6b58937c401656634e42d33f00ddf49#5f80b9baf6b58937c401656634e42d33f00ddf49"
source = "git+https://github.com/storytold/winprint.rs?rev=51934db382f225454cf152a8657eb03789be63c4#51934db382f225454cf152a8657eb03789be63c4"
dependencies = [
"flate2",
"fmt-derive",
Cargo.toml
+1 -1
View File
@@ -24,7 +24,7 @@ craft-keymap = { git = "https://github.com/storytold/craft-libs", version = "0.1
# the commit in `craft-i18n.rev`). Already in the tree through craft-ui.
craft-i18n = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "787a7a0f972fd1d7a7618fdbeca3a5267a0407ff" }
# Shared system spooler; document composition remains in pdfcraft-print.
craft-print = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "787a7a0f972fd1d7a7618fdbeca3a5267a0407ff" }
craft-print = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "7ee35a9fbd47af572d55fd63dbf0333d1f00da17" }
# Whether a path stored in a document may be opened (GoToR link targets).
craft-docpath = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "787a7a0f972fd1d7a7618fdbeca3a5267a0407ff" }
# craft-update: app-specific pretend versions apply to startup and manual checks.
LICENSES.toml
+14 -1
View File
@@ -16,7 +16,7 @@ sha256 = "cd7032ea33b71427f188e34ffb4aee9d6868a57fd196037c5c8b2be03a2a0352"
[[document]]
source = "NOTICE"
destination = "NOTICE"
sha256 = "9f854f730719af962cd840529324d089e05e68a30bd245c50e33f96e009b4314"
sha256 = "d9ba162ad2bc95d22ab9885065e253bb823bd6795068828e7b39858344b660eb"
[[document]]
source = "assets/app-icon/LICENSE-document.txt"
@@ -60,6 +60,13 @@ destination = "licenses/CMap-LICENSE.txt"
sha256 = "742665db9c8e1bc72603c6d319ca3e90b83bd6d95202f0cd4ef11068a07a9c29"
origin = "https://static.crates.io/crates/hayro-cmap/hayro-cmap-0.1.0.crate#assets/LICENSE.txt"
[[document]]
source = "licenses/ICU4X-LICENSE.txt"
licences = ["Unicode-3.0"]
destination = "licenses/ICU4X-LICENSE.txt"
sha256 = "f367c1b8e1aa262435251e442901da4607b4650e0e63a026f5044473ecfb90f2"
origin = "https://static.crates.io/crates/icu_segmenter_data/icu_segmenter_data-2.3.0.crate#LICENSE"
[[document]]
source = "licenses/egui-Hack-Regular.txt"
licences = ["MIT", "Bitstream-Vera"]
@@ -164,3 +171,9 @@ crate = "hayro-interpret"
path = "src/font/generated/metrics.rs"
licence = "MIT OR Apache-2.0"
documents = ["vendor/hayro-interpret/LICENSE-MIT", "vendor/hayro-interpret/LICENSE-APACHE"]
[[bundled]]
crate = "icu_segmenter_data"
path = "data/segmenter_dictionary_extended_v1.rs.data"
licence = "Unicode-3.0"
documents = ["licenses/ICU4X-LICENSE.txt"]
NOTICE
+29
View File
@@ -36,6 +36,9 @@ list (author, source, license) is in ATTRIBUTION.md.
- egui default fonts (epaint_default_fonts, via eframe) — Ubuntu Light (Ubuntu Font Licence 1.0),
Hack (MIT / Bitstream Vera licence), Noto Emoji (SIL OFL 1.1), emoji-icon-font (MIT).
- Colour profile in hayro-interpret — CGATS001Compat-v2-micro.icc (CC0-1.0, Clinton Ingram).
- ICU4X line-breaking data (icu_segmenter_data, via crates.io) — Unicode-3.0. Its Thai, Lao, Khmer
and Burmese word dictionaries are compiled in for wrapping Thai text; notice below, full licence in
licenses/ICU4X-LICENSE.txt.
Required notices:
@@ -48,6 +51,32 @@ Required notices:
(https://github.com/adobe-type-tools/cmap-resources/blob/master/LICENSE.txt). Neither the name of
Adobe nor the names of its contributors are used to endorse or promote this product.
ICU4X segmenter data (bundled by icu_segmenter_data 2.3.0; word lists, not visual assets):
Copyright © 2020-2024 Unicode, Inc. Portions adapted from ICU4C and/or ICU4J:
ICU 1.8.1 to ICU 57.1 © 1995-2016 International Business Machines Corporation and others.
Permission is hereby granted, free of charge, to any person obtaining a copy of data files and any
associated documentation (the "Data Files") or software and any associated documentation (the
"Software") to deal in the Data Files or Software without restriction, including without
limitation the rights to use, copy, modify, merge, publish, distribute, and/or sell copies of the
Data Files or Software, and to permit persons to whom the Data Files or Software are furnished to
do so, provided that either (a) this copyright and permission notice appear with all copies of
the Data Files or Software, or (b) this copyright and permission notice appear in associated
Documentation.
THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND NONINFRINGEMENT OF THIRD PARTY RIGHTS.
IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE BE LIABLE FOR ANY CLAIM,
OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF
USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA FILES OR SOFTWARE.
Except as contained in this notice, the name of a copyright holder shall not be used in
advertising or otherwise to promote the sale, use or other dealings in these Data Files or
Software without prior written authorization of the copyright holder.
Showcase PDF (built by `cargo xtask demo-pdf`, not committed): fonts are downloaded at build time from
google/fonts at a pinned commit, verified by SHA-256, all under the SIL Open Font License 1.1 — see the
"Downloaded at build time" table in ATTRIBUTION.md.
ROADMAP.md
+18
View File
@@ -1,5 +1,11 @@
# PdfCraft roadmap
> **Last reviewed:** 2026-11-08 | **Last updated:** 2026-11-08 | **Change:** patch (M10 #6514: refresh craft-print to 7ee35a9f, PDF loader to 51934db3 and public GDI kept at 64e7afd4 after #605 merged; preserve crash fix and stream-test repairs; new-graph CI pending; estimates unchanged)
> **Last reviewed:** 2026-11-07 | **Last updated:** 2026-11-07 | **Change:** patch (M10: file-backed Windows PDF loading without whole-job buffering or per-chunk async waits; approved dependency chain retains #5006; physical output still unqualified; estimates unchanged)
> **Last reviewed:** 2026-11-05 | **Last updated:** 2026-11-05 | **Change:** patch (M7 #2964: directly edit Thai source text inside supported Form XObjects; estimates unchanged)
> **Last reviewed:** 2026-11-03 | **Last updated:** 2026-11-03 | **Change:** patch (M5 #5850 review: indirect typewriter intent and composer edits preserve outer whitespace; movement regression commits by outside click; estimates unchanged)
> **Last reviewed:** 2026-11-03 | **Last updated:** 2026-11-03 | **Change:** patch (M5 #5850: Enter adds Fill & Sign line breaks; outside clicks commit, Escape cancels, and saved multiline appearances retain their text and height; estimates unchanged)
@@ -233,13 +239,17 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
## Progress log
- **2026-10-13 (M8, redact by regex):** `redact_mark` takes `regex` (caller expression, linear-time engine, length/size/nesting caps, empty-matching refused) with optional `validate` (luhn, mod97, iban). Headless only; issue #1886.
- **2026-11-07 (M7, #2964 / #3185):** Supported Form text can be prepared for direct editing; groups left grouped (unsafe compositing, text nested past eight levels) are reported. Legacy Thai keeps the original font programs, Regular/Bold and text settings, also across same-face subsets; the narrow THSarabunPSK map repair is checked glyph by glyph against the font program; cross-family replacement is explicit. Builds on #3067 and #2811. Unsafe groups and mixed contextual styles remain partial; estimates unchanged.
- **2026-10-28 (M10, #4119 partial):** Scan & OCR ▸ Enhance scanned file cleans scanned pages in place: background removal (level 0–100, as the camera clean-up), edge shadow removal (book gutters, copier borders) and despeckle. Each scan image is decoded, cleaned and written back into its own object at the same size and colour space (JPEG stays JPEG, others lossless, 1-bit stays 1-bit), so the OCR text layer, comments and links stay; one undo step, shared images once, unsupported images (CMYK, indexed, JPEG 2000, JBIG2, CCITT, broken data) reported. `page_enhance_scan` drives it headlessly. Not yet an option of Recognize text. Estimates unchanged.
- **2026-11-07 (M10, `ocr.ocr-web`, port onto #977):** Web OCR rebuilt on main's engines and preprocessing (#977). `OcrJob::run` is now `OcrJob::steps` a page at a time: the page renders, crops to the region and cleans each page, a Web Worker runs ocrs on the result, and `OcrShot::place` maps the words back through the region and preprocessing. Every strategy reads like this when only one engine is present, and the web build has only ocrs. The downloaded models are installed into the engine cache (`ocr::install`), so the Verify screen and the engine-status checks see them too. The batch keeps main's skipped-files bucket, low-confidence notes and cancel. The translations moved to craft-i18n (storytold/craft-i18n#141). The Verify screen starts the model download itself and runs once the models land; before, on a fresh page it failed with the desktop's "set PDFCRAFT_MODELS" message. Known limits: the cleaning steps, and the Verify screen's Run OCR and re-reads, run on the page. Estimates unchanged.
- **2026-11-03 (M11, shared JBIG2 dictionary):** With Monochrome Images set to JBIG2, Optimize has up to 16 JBIG2 images share one symbol dictionary (`/JBIG2Globals`), the letters found on more than one page stored once, when that is smaller (`pdfcraft_filters::encode_jbig2_shared`). Eight pages of 10 pt text rendered at 300 dpi take 41% less in Helvetica and 42% in Times than coded alone, 10% in Courier; groups of 16 save 45%, of 32 only 47%. Only images redaction can store again without the dictionary share one (image masks and DeviceGray named directly, at most 64 Mi pixels), and redacting one stores the rest of its group so, which the group bounds to 15.
- **2026-11-03 (M5, #5850 review):** Resolve indirect IT when recognizing typewriter contents and Fill & Sign marks. Comment listing and property lookup retain their existing direct-Subtype behavior. A real double-click Edit composer preserves the typewriter’s text as written; ordinary comment edits keep trimming. Regressions cover direct Subtype with indirect intent, save/reopen, raw posted contents and same-index comments on different pages. The existing live-movement test now commits its setup with an outside click rather than Enter. Estimates unchanged.
- **2026-11-03 (M5, #5850):** Fill & Sign Enter inserts a line break in a growing editor; clicking outside commits and Escape cancels. Nonempty text keeps leading, internal and trailing line breaks. The headless fill_sign_add path sizes its typewriter box for the same wrapped lines so saved appearances show every row. Production-order keyboard and engine save/reopen/render regressions exercise the UI and headless callers. Estimates unchanged.
- **2026-11-03 (community, M7, Thai word breaking):** Thai rows end between words, found with ICU4X's dictionary line segmenter (`icu_segmenter` 2.3, compiled dictionary data, no LSTM model; Unicode-3.0), instead of only at spaces when rewrapping and at any cluster in Add text. A row never starts with a following vowel, tone or other mark, Mai Yamok or Paiyannoi, nor ends with a leading vowel; a word wider than the row still breaks between clusters, and the spaces where an Add text row ends are dropped as in Latin text. Rewrapping existing text works out the breaks once per run of non-space characters. Rewriting a paragraph into Thai still needs a source font with the Thai glyphs (there is no Thai fallback face for it yet). The compiled dictionaries (Thai, Lao, Khmer, Burmese) add about 1.8 MB to a stripped release binary. Estimates unchanged.
- **2026-11-03 (M7, #5835):** In image crop mode, a right-click on the image, its crop box or its handles keeps the crop and opens no menu, so the menu no longer switches as the pointer moves. A right-click elsewhere clears the selection and opens the page menu. UI tests cover all four page rotations, overlapping images, a 45° image, and a check that the Done/Cancel/Remove Crop pills at a page edge still keep the crop.
- **2026-11-02 (M12, #1297):** `model_add` embeds a U3D or PRC file, recognised by its header, as a 3D annotation with a chosen activation and a poster (an image or PDF page fitted in the box, or a labelled frame); an older file's catalog `/Version` is raised to what the model needs. Views, a rendered poster and a UI command are later steps. Estimates unchanged.
@@ -270,6 +280,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
- **2026-10-27 (M2, PR #4172 module-split conflicts):** Integrated main 1dd2737b, retaining viewer-only CJK fallback in split session modules and Combine thumbnail workers, and moving renderer regressions to the new test module. Validation: 798 scoped tests passed (371 engine, 412 renderer, 15 Combine; two existing engine tests ignored); scoped all-target Clippy with -D warnings, formatting, layers, assets, parity and renderer/engine WASM checks passed. Estimates unchanged.
- **2026-10-27 (M10, `ocr.ocr-web`, rebase fix):** The web build loads OCR models only when their SHA-256 matches `ATTRIBUTION.toml`. With ONNX loading now on, rten's parser spins forever on some bytes that aren't a model (a corrupt or wrong download), so a bad file now fails at once instead of hanging a worker.
- **2026-10-27 (M10, OCR models):** Model staging downloads and verifies the ONNX pair after the upstream RTEN URL brownout; runtime discovery prefers complete ONNX pairs per directory, retains complete RTEN fallback and preserves directory precedence.
- **2026-10-26 (M2, PR #4172):** Reconciled current main, preserving its Japanese serif/weight selection and both branches' progress records. Installed CJK viewer and preview fallback remains display-only; estimates unchanged.
@@ -281,6 +292,8 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
- **2026-10-25 (web, same-document host mode):** A page that loads the PdfCraft web module into its own document (no iframe) can opt into the host bridge with `data-host="page"` on the app's canvas — the exact value `page`; any other value keeps the normal app with downloads, unless a same-origin framed `?host=parent` selects the parent bridge (unframed or cross-origin, `?host=parent` also keeps the normal app). It is the `?host=parent` bridge with the transport swapped: the same five messages travel as non-bubbling `CustomEvent`s on the canvas (`detail` holds the parent messages' fields without `type`; the host answers with a `pdfcraft:open` event), so saves reach the host instead of downloading and dirty and start-up-failure reporting are unchanged. Unit tests cover the opt-in rule; no parity or estimate changes.
- **2026-10-24 (M10, #4158 partial):** Word, HTML, RTF and PowerPoint export keep a block that continues another on its line (an italic case name, a link, a superscript) as a run in its own style inside that paragraph, with the source's word space, instead of a paragraph of its own. Style changes that wrap onto the next line still split paragraphs (the reading blocks keep style boundaries). Estimates unchanged.
- **2026-10-24 (M3, window move on Wayland, #3970 follow-up):** After a move or resize by PdfCraft's own title bar or edges, the button release given back to egui goes before the pointer leaving the window, so hover no longer sticks at the old place until the pointer moves. Found by CodeRabbit in review of #3970; a unit test covers it. Estimates unchanged.
- **2026-10-26 (M10, `ocr.ocr-web`, review fix):** On the web, Edit on a scanned page starts the OCR model download, and the page is read once the models load; it was taken for no scan and left unread until the document changed.
- **2026-10-24 (M10, `ocr.ocr-web`, review fix):** On the web, single-file and multiple-file OCR running at once no longer lose a page: the workers' answers go to the run that sent each page, instead of to whichever run collected first, and a cancelled run's late answers are let go. Regression tests cover both start orders and cancellation. Files chosen for Recognize Text in Multiple Files before the models have downloaded now wait for them instead of having to be chosen again, and idle workers close on time even when nothing is happening on the page, a full minute after their last page comes back rather than after it was handed out (checked by hand in Firefox); estimates unchanged.
- **2026-10-24 (M14, #4253):** The annotation quick bar remembers Keep tool selected across restarts, including unpinning. Old and malformed settings keep safe defaults; the opt-in UI control channel reads and changes the same preference. Estimates unchanged.
- **2026-10-22 (M7, #2517):** A partial rewrite of the last paragraph closes its marker after that paragraph's last `ET`, so a row left in the source stays in the paragraph. When the next paragraph shares the text object, the marker closes before that neighbour instead of after the shared `ET`. The extra marker is not added across two marked-content regions. Estimates unchanged.
- **2026-10-15 (M7, #2517):** A blank line between paragraphs stays a paragraph break. An explicit `TL` is the measured baseline gap. Without one, three or more rows at a repeated step stay one paragraph, and a single wider gap stays a break. Adding a row, even when an earlier row is left in the source, does not absorb the next paragraph on a later edit. Estimates unchanged.
@@ -561,6 +574,8 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
Newest first: the date, what moved, and the new overall estimate.
- **2026-10-28 (M10, #1595):** Export to Markdown: `doc_export_office` with a `.md` or `.markdown` path writes CommonMark with GitHub pipe tables (headings, paragraphs with bold and italic, tables with their first row as the header, a rule between pages); text that Markdown would read as markup is escaped. Images, lists and links are not in the export's layout model and are left out. No menu item yet. Estimates unchanged.
- **2026-10-12 (M10, `ocr.ocr-web`):** OCR in the browser, partly. The web zip carries the text recognition models in `models/` beside `index.html`; the first time Recognize Text is needed the web build downloads them (about 12 MB, with a spinner, an error and a retry). Pages are rendered on the page and read in Web Workers (up to two at once, so the page keeps responding), built with WebAssembly SIMD: 8.5 s a page instead of 20.5 s in wasmtime, still about 17× native. Where workers can't start, the page reads the pages itself. No automated browser test yet. Estimates unchanged.
- **2026-10-12 (M10, `ocr.ocr-web`):** Web OCR shipped: reading scanned pages in Web Workers was checked by hand in a browser (4-page synthetic scan). Still no automated browser test, and a page reads about 17× slower than native.
- **2026-10-12 (M3, GNOME title bar, #553):** On GNOME's Wayland session, whose compositor draws no window frames (it offers no `zxdg_decoration_manager_v1`), PdfCraft no longer shows winit's fallback title bar: it turns the frame off and the tab strip becomes the header bar, as in GNOME's own apps. Minimize, maximize/restore and close are drawn in Adwaita's style, in the order and on the side GNOME's `button-layout` names (Fedora's default shows only close); the window's edges and corners resize it, and a hairline outlines it while it isn't maximized. Preferences ▸ Use system title bar (shown only there) gives the window back winit's frame at once, without a restart (part of #1001), and `PDFCRAFT_TITLEBAR=app|system` overrides the choice. KDE, Sway, Hyprland, COSMIC and X11 keep the frame their desktop draws. Not done: right-clicking the header bar doesn't open GNOME's window menu (egui can't ask for it), the window has no shadow or rounded corners, the resize edges lie just inside the window rather than outside it, and Windows still uses its system frame (the rest of #1001). Estimates unchanged.
- **2026-10-12 (M0, build label):** A local or patched build can carry a label, set at compile time with `PDFCRAFT_BUILD_LABEL`: `pdfcraft --version`, `pdfcraft-cli --version`, About and the log show `0.7.1 (titlebar)`, and `packaging/linux/package.sh` names the files `pdfcraft-0.7.1-titlebar-…` and makes the .rpm/.deb release `1.titlebar`, which upgrades over the release of the same version (and the next release over it). The version itself is unchanged, so the update check still compares it with published releases. Unlabelled builds are as before. Estimates unchanged.
- **2026-10-11 (M6, #1208):** Dropdown and list-box Field Properties expose the existing Left/Center/Right alignment picker. Existing alignment stays intact until changed; cancel, undo/redo and save/reopen are covered through the real UI without changing options, values or flags. Estimates unchanged.
@@ -796,6 +811,9 @@ Newest first: the date, what moved, and the new overall estimate.
| Date | Change | Summary |
|---|---|---|
| 2026-11-08 | patch | M10 #6514/#636: retain main's public GDI Git revision (64e7afd4) to preserve downstream Rust type identity, update the internal PDF loader independently, and refresh the application dependency/lockfile. New-graph CI pending; estimates unchanged. |
| 2026-11-08 | patch | M10 #6514: refresh the craft-print and winprint/GDI pins after #605 merged; retain the Direct2D crash fix and stream-test cleanup repairs, record single-page HUAWEI output with its diagnostic limits, and rerun CI for the new dependency graph. Estimates unchanged. |
| 2026-11-07 | patch | M10 #6514: load Windows PDF print jobs on demand through a read-only file stream, avoiding StorageFile access denial, whole-file buffering and per-chunk async waits; approved dependencies retain the #5006 crash fix. Regression tests pass; physical output remains unqualified; estimates unchanged. |
| 2026-11-03 | patch | M5 #5850 review: indirect typewriter intent and composer edits preserve outer whitespace; movement regression commits by outside click; estimates unchanged. |
| 2026-11-03 | patch | M5 #5850: Enter adds Fill & Sign line breaks; outside clicks commit, Escape cancels, and saved multiline appearances retain their text and height; estimates unchanged. |
| 2026-11-01 | patch | M7 #5657: header/footer, watermark and background operations commit all selected pages together; later-page errors preserve marks, content, shared/inherited resources and pending edits; estimates unchanged. |
apps/pdfcraft-ocr-worker/Cargo.toml
+18
View File
@@ -0,0 +1,18 @@
[package]
name = "pdfcraft-ocr-worker"
description = "PdfCraft in the browser: the Web Worker that recognises text (wasm32; trunk builds it with pdfcraft-web)"
version.workspace = true
edition.workspace = true
license.workspace = true
publish = false
[lints]
workspace = true
[dependencies]
pdfcraft-ocr = { workspace = true }
[target.'cfg(target_arch = "wasm32")'.dependencies]
wasm-bindgen = "0.2"
js-sys = "0.3"
web-sys = { version = "0.3", features = ["DedicatedWorkerGlobalScope", "MessageEvent"] }
apps/pdfcraft-ocr-worker/src/main.rs
+101
View File
@@ -0,0 +1,101 @@
//! PdfCraft in the browser: the Web Worker that recognises text, so reading a scanned page doesn't
//! stop the page responding and several pages are read at once. Trunk builds it alongside
//! `pdfcraft-web` (see apps/pdfcraft-web/index.html); the page starts and feeds the workers
//! (crates/ui-egui/src/workflows/ocr_ui.rs).
//!
//! Messages are plain objects with a `type`:
//!
//! - `ready` (to the page): the worker is listening; send the models now.
//! - `models` (from the page): `{ detection, recognition }`, each a `Uint8Array` holding a model
//! file. Answered with `loaded`, or `failed` with an `error`.
//! - `page` (from the page): `{ id, width, height, pixels }`, the RGBA pixels of a rendered page
//! as a `Uint8Array`. Answered with `lines` `{ id, lines }`, the recognised lines as
//! `pdfcraft_ocr::lines_to_json` writes them, or `failed` `{ id, error }`.
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
#[cfg(target_arch = "wasm32")]
fn main() {
use wasm_bindgen::{JsCast, closure::Closure};
let Ok(scope) = js_sys::global().dyn_into::<web_sys::DedicatedWorkerGlobalScope>() else {
return;
};
let reply_to = scope.clone();
let mut ocr = None;
let on_message = Closure::<dyn FnMut(web_sys::MessageEvent)>::new(move |e: web_sys::MessageEvent| {
let _ = reply_to.post_message(&worker::handle(&mut ocr, &e.data()));
});
scope.set_onmessage(Some(on_message.as_ref().unchecked_ref()));
// The handler lives as long as the worker.
on_message.forget();
let _ = scope.post_message(&worker::message(&[("type", "ready".into())]));
}
#[cfg(not(target_arch = "wasm32"))]
fn main() {
eprintln!("pdfcraft-ocr-worker runs in a browser: trunk builds it with pdfcraft-web");
}
#[cfg(target_arch = "wasm32")]
mod worker {
use pdfcraft_ocr::Ocr;
use wasm_bindgen::{JsCast, JsValue};
/// The models are about 12 MB together.
const MAX_MODEL_BYTES: u32 = 64 << 20;
/// The renderer keeps pages far smaller; anything larger isn't a page.
const MAX_PIXEL_BYTES: u64 = 1 << 30;
pub fn message(fields: &[(&str, JsValue)]) -> JsValue {
let o = js_sys::Object::new();
for (k, v) in fields {
let _ = js_sys::Reflect::set(&o, &(*k).into(), v);
}
o.into()
}
/// Answer one message from the page.
pub fn handle(ocr: &mut Option<Ocr>, data: &JsValue) -> JsValue {
let get = |k: &str| js_sys::Reflect::get(data, &k.into()).unwrap_or(JsValue::UNDEFINED);
let id = get("id");
let reply = match get("type").as_string().as_deref() {
Some("models") => load(&get("detection"), &get("recognition")).map(|o| {
*ocr = Some(o);
message(&[("type", "loaded".into())])
}),
Some("page") => match ocr.as_ref() {
None => Err("the text recognition models aren't loaded".into()),
Some(o) => recognize(o, &get("width"), &get("height"), &get("pixels"))
.map(|lines| message(&[("type", "lines".into()), ("id", id.clone()), ("lines", lines.into())])),
},
_ => Err("unknown message".into()),
};
reply.unwrap_or_else(|error| message(&[("type", "failed".into()), ("id", id), ("error", error.into())]))
}
fn bytes(v: &JsValue, max: u64) -> Result<Vec<u8>, String> {
let a = v.dyn_ref::<js_sys::Uint8Array>().ok_or("expected a Uint8Array")?;
if u64::from(a.length()) > max {
return Err(format!("{} bytes is too large", a.length()));
}
Ok(a.to_vec())
}
fn load(detection: &JsValue, recognition: &JsValue) -> Result<Ocr, String> {
let max = u64::from(MAX_MODEL_BYTES);
Ocr::from_bytes(bytes(detection, max)?, bytes(recognition, max)?).map_err(|e| e.to_string())
}
fn recognize(ocr: &Ocr, width: &JsValue, height: &JsValue, pixels: &JsValue) -> Result<String, String> {
let side = |v: &JsValue| v.as_f64().filter(|v| v.fract() == 0.0 && (1.0..=f64::from(u32::MAX)).contains(v)).map(|v| v as u32);
let (Some(w), Some(h)) = (side(width), side(height)) else {
return Err("the page size is not a positive whole number".into());
};
let px = bytes(pixels, MAX_PIXEL_BYTES)?;
if u64::from(w).checked_mul(u64::from(h)).and_then(|n| n.checked_mul(4)) != Some(px.len() as u64) {
return Err("the pixels don't match the page size".into());
}
ocr.recognize(&px, w, h).map(|lines| pdfcraft_ocr::lines_to_json(&lines)).map_err(|e| e.to_string())
}
}
apps/pdfcraft-web/Trunk.toml
+7
View File
@@ -9,3 +9,10 @@ filehash = true
[serve]
addresses = ["127.0.0.1"]
port = 8766
# Trunk leaves the OCR worker's files unhashed and unnamed in index.html: hash them and name the
# worker for the page (xtask/src/web.rs).
[[hooks]]
stage = "post_build"
command = "cargo"
command_arguments = ["xtask", "web-worker-link"]
apps/pdfcraft-web/index.html
+2
View File
@@ -5,6 +5,8 @@
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>PdfCraft</title>
<link data-trunk rel="rust" data-wasm-opt="z" />
<!-- Recognises text off the page's thread (crates/ui-egui/src/workflows/ocr_web.rs). -->
<link data-trunk rel="rust" href="../pdfcraft-ocr-worker/Cargo.toml" data-type="worker" data-loader-shim data-bindgen-target="web" data-wasm-opt="3" />
<style>
html, body { margin: 0; height: 100%; overflow: hidden; background: #f1f1f3; }
canvas { width: 100%; height: 100%; display: block; outline: none; }
apps/pdfcraft-web/src/main.rs
+1 -1
View File
@@ -48,7 +48,7 @@ fn main() {
});
}
// eframe never raises a close request on a browser reload or tab close, so `guard_quit` never
// prompts; ask the browser to confirm instead while a document has unsaved work (#812).
// prompts; ask the browser to confirm instead while a document has unsaved work or an OCR review is open (#812).
warn_before_unload(app.unsaved_flag.clone());
// Host modes (`host` below, over `craft-web-host`): `?host=parent`, a
// same-origin page embedding PdfCraft in an iframe (such as the Nextcloud
crates/annot/Cargo.toml
+2
View File
@@ -11,6 +11,8 @@ workspace = true
[dependencies]
pdfcraft-cos = { workspace = true }
pdfcraft-fonts = { workspace = true }
# The usable-quad rule `/QuadPoints` readers share.
pdfcraft-geom = { workspace = true }
pdfcraft-model = { workspace = true }
thiserror = { workspace = true }
crates/annot/README.md
+5 -1
View File
@@ -12,7 +12,11 @@ Comments (annotations), ISO 32000-2 §12.5. Layer L3, depends on `pdfcraft-cos`,
A line's leader lines (`/LL`, `/LLE`, `/LLO`) and caption (`/Cap`, `/CP`, `/CO`) are drawn as
ISO 32000-2 Table 178 places them, the caption in black Helvetica 10.
Note icons are PdfCraft's own drawings. Text boxes use standard Helvetica (WinAnsi), with
line breaking by approximate Helvetica proportions (no font program is bundled). Text it can't
line breaking by Helvetica's standard-14 metrics. Thai in a text box, callout or Fill & Sign
text is shaped and drawn from the craft-fonts Thai face with Add text's writer
(`pdfcraft_fonts::shaped_text`: Type 3 glyphs with a ToUnicode map), the box sized to the ink
of its stacked marks and descenders; a build without a Thai face refuses it
(`AnnotError::Undrawable`) instead of drawing `?`. Other text it can't
draw (in a text box, callout, Fill & Sign text, line caption, measurement label or dynamic
stamp's by-line; `appearance::undrawable_text`) is kept as typed and drawn as `?`, and the comment is marked with
the first such character (`pdfcraft_fonts::UNDRAWN_MARK`) so the engine reports the edit and
crates/annot/src/appearance.rs
+78 -28
View File
@@ -1,20 +1,47 @@
//! Appearance streams (§12.5.5) for the comment types PdfCraft creates, execution plan M5.2.
//!
//! [`build`] draws a normal appearance (`/AP /N`) from the annotation dictionary alone, so the
//! same code serves new comments and restyled ones. Drawings are in page space with
//! [`build`] draws a normal appearance (`/AP /N`) from the annotation dictionary and the
//! objects it points at, so the same code serves new comments and restyled ones. Drawings are in page space with
//! `/BBox = /Rect` (identity matrix), except note icons, which are drawn in a 20 × 20 box.
//! It returns `None` for anything it cannot draw faithfully (cloudy borders, unknown line
//! endings, indirect geometry), so callers never replace an appearance with a worse one.
//! endings), so callers never replace an appearance with a worse one.
//!
//! The note icons are PdfCraft's own drawings (AGENTS.md §1). Text boxes use the standard
//! Helvetica font with WinAnsi encoding; line breaking uses [`text_width`], an approximation of
//! Helvetica's proportions by character class (no font program or metrics file is bundled).
//! Helvetica font with WinAnsi encoding, and draw Thai from the craft-fonts Thai face as Type 3
//! glyphs (`pdfcraft_fonts::shaped_text`, the writer Add text uses); line breaking uses
//! [`text_width`], the standard-14 metrics and the Thai face's shaped advances.
use pdfcraft_cos::{Dict, Object, PdfString, Stream};
use pdfcraft_fonts::{first_undrawable, literal, win_ansi};
pub use pdfcraft_fonts::{helvetica_width as text_width, wrap};
use pdfcraft_fonts::shaped_text;
use pdfcraft_fonts::{first_undrawable, helvetica_width, literal, win_ansi, wrap_with};
use crate::{NOTE_SIZE, Rgb, n};
/// The width of `text` at `size` as appearances draw it: Helvetica, with Thai shaped in the Thai
/// face that text boxes draw it with.
pub fn text_width(text: &str, size: f64) -> f64 {
shaped_text::thai_width(text, size, helvetica_width)
}
/// `text` wrapped `width` wide as [`text_width`] measures it; a Thai mark stays on its letter.
pub fn wrap(text: &str, size: f64, width: f64) -> Vec<String> {
wrap_with(text, size, width, text_width)
}
/// How far a text box's `lines` reach, in em: above the first baseline (0.9 of the 1.2 em line)
/// and below the last (0.3), or further where Thai stacked marks or descenders reach, so the box
/// is sized to their ink and doesn't clip them.
fn reach(lines: &[String]) -> [f64; 2] {
let ink = |line: Option<&String>| line.and_then(|l| shaped_text::thai_reach(l));
[ink(lines.first()).map_or(0.9, |r| r[1].max(0.9)), ink(lines.last()).map_or(0.3, |r| (-r[0]).max(0.3))]
}
/// The height a text box's `lines` take at `size`, inside its padding: 1.2 em a line, and more
/// for Thai that reaches further ([`reach`]).
pub fn text_lines_height(lines: &[String], size: f64) -> f64 {
let [up, down] = reach(lines);
lines.len().max(1) as f64 * size * 1.2 + (up - 0.9 + down - 0.3) * size
}
use crate::{Document, NOTE_SIZE, Rgb, n};
mod symbols;
@@ -166,6 +193,18 @@ fn nums(d: &Dict, key: &[u8]) -> Option<Vec<f64>> {
d.get(key)?.as_array()?.iter().map(|o| o.as_f64()).collect()
}
/// The usable `/QuadPoints` quads, by the rule redaction apply reads them with
/// ([`pdfcraft_geom::quad_points::usable_quads`]): a malformed or non-finite quad is not drawn (it
/// would write `NaN` into the stream) and the others still are. The array and its entries are
/// resolved first, as the other readers resolve them, so an indirect quad is drawn too. `None`
/// when none is usable.
fn quads(context: &AppearanceContext<'_>) -> Option<Vec<[f64; 8]>> {
let doc = context.doc;
let q =
pdfcraft_geom::quad_points::usable_quads(doc.resolve(context.dict.get(b"QuadPoints")?).as_array()?.iter().map(|x| doc.resolve(x).as_f64()));
(!q.is_empty()).then_some(q)
}
/// `/C`-style colour arrays: none (transparent), gray, RGB or CMYK.
fn color(d: &Dict, key: &[u8]) -> Option<Option<Rgb>> {
let c = match d.get(key) {
@@ -380,13 +419,15 @@ pub(crate) fn inset(d: &Dict) -> Option<[f64; 4]> {
/// The first text [`build`] would draw in the standard fonts (Helvetica, WinAnsiEncoding) that
/// they can't, which would come out as `?`: what it is (for the message) and the character. The
/// text of a text box, callout or Fill & Sign text, a line's caption, a measurement's label, a
/// text of a text box, callout or Fill & Sign text (whose Thai the Thai face draws, where this
/// build has one), a line's caption, a measurement's label, a
/// dynamic stamp's "By …" line, and a redaction mark's overlay text (drawn when it is applied). `None` when all of
/// it can be drawn.
pub fn undrawable_text(d: &Dict) -> Option<(&'static str, char)> {
let text = |key: &[u8]| d.get(key).and_then(Object::as_string).map(PdfString::to_text);
let (what, text) = match d.name(b"Subtype")? {
b"FreeText" => ("the comment's text", text(b"Contents")?),
// A text box draws Thai too.
b"FreeText" => return Some(("the comment's text", shaped_text::first_undrawable(&text(b"Contents")?)?)),
b"Line" if !d.contains(b"PCMeasureValue") => ("the line's caption", line_caption(d)?),
b"Line" | b"PolyLine" | b"Polygon" => ("the measurement label", text(b"PCMeasureValue")?),
b"Stamp" if matches!(d.get(b"PCStamp"), Some(Object::Bool(true))) => ("the stamp's \"By\" line", text(b"PCByLine")?),
@@ -399,6 +440,11 @@ pub fn undrawable_text(d: &Dict) -> Option<(&'static str, char)> {
/// `/Font` resources naming `/Helv`: standard Helvetica with WinAnsiEncoding.
pub(crate) fn helvetica_resource() -> Object {
Object::Dict(helvetica_fonts())
}
/// [`helvetica_resource`]'s dictionary.
fn helvetica_fonts() -> Dict {
let mut font = Dict::new();
font.set(b"Type".to_vec(), Object::name("Font"));
font.set(b"Subtype".to_vec(), Object::name("Type1"));
@@ -406,7 +452,7 @@ pub(crate) fn helvetica_resource() -> Object {
font.set(b"Encoding".to_vec(), Object::name("WinAnsiEncoding"));
let mut fonts = Dict::new();
fonts.set(b"Helv".to_vec(), Object::Dict(font));
Object::Dict(fonts)
fonts
}
/// Where a caret and its paragraph symbol go within `rect`: the box the caret is drawn in, and for
@@ -448,6 +494,7 @@ fn caret_layout(d: &Dict, rect: [f64; 4]) -> ([f64; 4], Option<(f64, f64, f64)>)
#[derive(Clone, Copy)]
struct AppearanceContext<'a> {
doc: &'a Document,
dict: &'a Dict,
subtype: &'a [u8],
rect: [f64; 4],
@@ -484,8 +531,8 @@ impl AppearanceDrawing {
/// Draw the normal appearance of an annotation, or `None` if this subtype/variant isn't supported.
/// Text the standard fonts can't draw comes out as `?`: [`undrawable_text`] says which, and
/// [`crate::set_appearance`] marks the annotation with it (`pdfcraft_fonts::UNDRAWN_MARK`).
pub fn build(d: &Dict) -> Option<Stream> {
let context = appearance_context(d)?;
pub fn build(doc: &Document, d: &Dict) -> Option<Stream> {
let context = appearance_context(doc, d)?;
let drawing = AppearanceDrawing::new(&context);
match context.subtype {
b"Text" => return draw_icon(&context, "Note", [1.0, 0.82, 0.0], drawing),
@@ -499,7 +546,7 @@ pub fn build(d: &Dict) -> Option<Stream> {
finish_appearance(&context, drawing)
}
fn appearance_context(d: &Dict) -> Option<AppearanceContext<'_>> {
fn appearance_context<'a>(doc: &'a Document, d: &'a Dict) -> Option<AppearanceContext<'a>> {
// Imported measurement appearances may include leaders, captions and formatting that
// this builder cannot reproduce. Preserve them rather than silently losing detail.
if d.contains(b"Measure") && !d.contains(b"PCMeasureValue") {
@@ -514,6 +561,7 @@ fn appearance_context(d: &Dict) -> Option<AppearanceContext<'_>> {
return None;
}
Some(AppearanceContext {
doc,
dict: d,
subtype,
rect,
@@ -561,13 +609,12 @@ fn finish_appearance(context: &AppearanceContext<'_>, mut drawing: AppearanceDra
}
fn draw_markup(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing) -> Option<()> {
let d = context.dict;
let subtype = context.subtype;
let stroke = context.stroke;
let c = &mut drawing.content;
let q = nums(d, b"QuadPoints").filter(|q| !q.is_empty() && q.len() % 8 == 0)?;
let q = quads(context)?;
let col = stroke?;
for quad in q.as_chunks::<8>().0.iter() {
for quad in q.iter() {
let p = |i: usize| (quad[2 * i], quad[2 * i + 1]);
let (p1, p2, p3, p4) = (p(0), p(1), p(2), p(3));
let h = (p1.0 - p3.0).hypot(p1.1 - p3.1);
@@ -608,14 +655,13 @@ fn draw_markup(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing)
}
fn draw_redaction(context: &AppearanceContext<'_>, drawing: &mut AppearanceDrawing) -> Option<()> {
let d = context.dict;
let stroke = context.stroke;
let c = &mut drawing.content;
// While marked: the outline of each area (the fill comes when applied).
let q = nums(d, b"QuadPoints").filter(|q| !q.is_empty() && q.len() % 8 == 0)?;
let q = quads(context)?;
let col = stroke.unwrap_or([0.89, 0.13, 0.13]);
c.push_str(&format!("{}1 w\n", rg_stroke(col)));
for quad in q.as_chunks::<8>().0.iter() {
for quad in q.iter() {
c.push_str(&format!(
"{} {} m {} {} l {} {} l {} {} l h S\n",
n(quad[0]),
@@ -1090,27 +1136,31 @@ fn draw_free_text(context: &AppearanceContext<'_>, mut drawing: AppearanceDrawin
));
let mut out = c.into_bytes();
let lines = wrap(&text, size, width);
let mut y = rect[3] - pad - size * 0.9;
let mut y = rect[3] - pad - size * reach(&lines)[0];
if round {
y -= ((rect[3] - rect[1] - 2.0 * pad - lines.len() as f64 * size * 1.2) / 2.0).max(0.0);
y -= ((rect[3] - rect[1] - 2.0 * pad - text_lines_height(&lines, size)) / 2.0).max(0.0);
}
for line in lines {
// Thai is shaped and drawn from the Thai face (Type 3 fonts in the form's resources, whose
// streams `apply_appearance` stores in the document); the rest, and Thai it can't draw (as
// `?`, which `undrawable_text` reports), in Helvetica.
let mut fonts = helvetica_fonts();
let (pieces, shaped) = shaped_text::appearance_lines(&lines, &mut fonts);
let mut current = None;
for pieces in &pieces {
if y < rect[1] - size {
break;
}
let lw = text_width(&line, size);
let lw = shaped_text::pieces_width(pieces, size, 0.0, helvetica_width);
let x = match q {
1 => rect[0] + pad + (width - lw) / 2.0,
2 => rect[2] - pad - lw,
_ => rect[0] + pad,
};
out.extend(format!("1 0 0 1 {} {} Tm ", n(x), n(y)).bytes());
out.extend(literal(&win_ansi(&line)));
out.extend_from_slice(b" Tj\n");
shaped.show_line(&mut out, pieces, x, y, size, "Helv", win_ansi, |s| helvetica_width(s, size), &mut current);
y -= size * 1.2;
}
out.extend_from_slice(b"ET\n");
res.set(b"Font".to_vec(), helvetica_resource());
res.set(b"Font".to_vec(), Object::Dict(fonts));
let mut stream = form(full, &out, res);
if let Some(m) = matrix {
stream.dict.set(b"Matrix".to_vec(), Object::Array(m.iter().copied().map(Object::Real).collect()));
crates/annot/src/create.rs
+27 -1
View File
@@ -1,6 +1,6 @@
use super::draw::draw_appearance;
use super::lookup::{annot_dict, annot_ref, annots, fill_sign_shape, page_ref, set_annots};
use super::modify::{free_text_da, n, rich_text, set_part_keys, shape_text_box};
use super::modify::{free_text_da, n, refuse_undrawable_thai, rich_text, set_part_keys, shape_text_box};
use super::types::{
FLAG_NO_ROTATE, FLAG_NO_ZOOM, FLAG_PRINT, LineEnding, Markup, Meta, NOTE_SIZE, NewAnnotation, OverlayLook, Rgb, Shape, StampGroup, Style,
};
@@ -398,6 +398,7 @@ pub fn add_annotation(doc: &mut Document, new: &NewAnnotation, meta: &Meta) -> R
if matches!(turn, 90 | 180 | 270) {
d.set(b"PCTextRotate".to_vec(), Object::Int(turn));
}
let rect = fit_thai_ink(rect, turn, &new.contents, size, 2.0 + appearance::border_width(&d));
// An oval or a circle goes around the text box, its text wrapped as displayed:
// after `/PCTextRotate`, which swaps the box's sides on a quarter-turned page.
shape_text_box(&mut d, rect, style.border_shape, style.width);
@@ -417,6 +418,7 @@ pub fn add_annotation(doc: &mut Document, new: &NewAnnotation, meta: &Meta) -> R
{
return Err(AnnotError::Undrawable(pdfcraft_fonts::undrawable_message(what, c)));
}
refuse_undrawable_thai(d.name(b"Subtype") == Some(b"FreeText"), &new.contents)?;
let r = doc.add(Object::Dict(d.clone()));
draw_appearance(doc, r)?;
let mut list = annots(doc, page);
@@ -427,6 +429,30 @@ pub fn add_annotation(doc: &mut Document, new: &NewAnnotation, meta: &Meta) -> R
Ok(index)
}
/// Text box `tb` (user space; its text upright on a page turned by `turn`), grown down as displayed
/// until the ink of `text`'s Thai (stacked marks above its first line, descenders below its last)
/// fits inside the padding `pad`, as the appearance lays it out at `size`. Other text, and a box
/// tall enough already, keep the box they were given.
fn fit_thai_ink(tb: [f64; 4], turn: i64, text: &str, size: f64, pad: f64) -> [f64; 4] {
if !text.chars().any(pdfcraft_fonts::shaped_text::is_thai) {
return tb;
}
let [x0, y0, x1, y1] = normalize(tb);
let (w, h) = if matches!(turn, 90 | 270) { (y1 - y0, x1 - x0) } else { (x1 - x0, y1 - y0) };
let need = appearance::text_lines_height(&appearance::wrap(text, size, (w - 2.0 * pad).max(1.0)), size) + 2.0 * pad + 2.0;
let by = need - h;
if !(by.is_finite() && by > 0.0) {
return tb;
}
// Displayed down is user -y unturned, +x on a page turned 90°, +y at 180° and -x at 270°.
match turn {
90 => [x0, y0, x1 + by, y1],
180 => [x0, y0, x1, y1 + by],
270 => [x0 - by, y0, x1, y1],
_ => [x0, y0 - by, x1, y1],
}
}
fn validate_style(style: &Style) -> Result<(), AnnotError> {
if !finite(&style.color) || style.border_color.is_some_and(|c| !finite(&c)) || !style.opacity.is_finite() || !style.width.is_finite() {
return Err(AnnotError::Invalid("invalid style".into()));
crates/annot/src/draw.rs
+4 -2
View File
@@ -40,7 +40,7 @@ pub(crate) fn draw_appearance(doc: &mut Document, r: ObjRef) -> Result<(), Annot
/// Draw the appearance for an annotation dictionary: the expensive half of [`set_appearance`].
/// Pure from the dictionary and the objects it points at, so [`AppearanceCache`] can reuse it.
fn build_appearance(doc: &Document, d: &Dict) -> Option<Stream> {
let mut stream = appearance::build(d)?;
let mut stream = appearance::build(doc, d)?;
// Picture-stamp restyling must retain the placement's page-axis correction.
if d.get(b"PCPicture").is_some() {
let matrix = d.get(b"AP").and_then(|ap| {
@@ -57,9 +57,11 @@ fn build_appearance(doc: &Document, d: &Dict) -> Option<Stream> {
}
/// Store a built appearance on the annotation: the cheap half of [`set_appearance`].
fn apply_appearance(doc: &mut Document, r: ObjRef, d: &Dict, stream: Stream) -> Result<(), AnnotError> {
fn apply_appearance(doc: &mut Document, r: ObjRef, d: &Dict, mut stream: Stream) -> Result<(), AnnotError> {
// Characters the standard fonts can't draw came out as `?`: mark the annotation with the first.
let missing = appearance::undrawable_text(d).map(|(_, c)| c);
// A text box's Thai glyphs are streams of their own.
pdfcraft_fonts::shaped_text::store_streams(doc, &mut stream.dict);
let ap = doc.add(Object::Stream(stream));
// A copy (a shared /AP is left alone) that keeps unknown entries; the old down and rollover
// appearances would show the previous look on press or hover, so they go with the old /N.
crates/annot/src/info.rs
+11 -2
View File
@@ -117,7 +117,16 @@ pub fn summaries(doc: &Document) -> Vec<Summary> {
color = Some([f(i - 3), f(i - 2), f(i - 1)]);
}
}
let quads = nums(b"QuadPoints").as_chunks::<8>().0.to_vec();
// Only usable quads, by the rule redaction apply reads them with, so a mark's outline
// is the area apply removes (and with none, the `/Rect` both fall back to).
let quads: Vec<[f32; 8]> = d
.get(b"QuadPoints")
.map(|o| doc.resolve(o))
.map(|a| {
let entries = a.as_array().into_iter().flatten().map(|x| doc.resolve(x).as_f64());
pdfcraft_geom::quad_points::usable_quads(entries).into_iter().map(|q| q.map(|v| v as f32)).collect()
})
.unwrap_or_default();
// Keep in step with `pdfcraft_render::inspect`.
let points = match subtype.as_str() {
"Line" => Some(nums(b"L")).filter(|l| l.len() == 4).map(|l| l.as_chunks::<2>().0.to_vec()).unwrap_or_default(),
@@ -252,7 +261,7 @@ pub fn props(doc: &Document, page: usize, index: usize) -> Option<Props> {
width,
icon: (subtype == "Text").then(|| d.name(b"Name").and_then(|n| NoteIcon::from_name(&String::from_utf8_lossy(n))).unwrap_or(NoteIcon::Note)),
modified: text_value(doc, d, b"M"),
restylable: appearance::build(d).is_some(),
restylable: appearance::build(doc, d).is_some(),
locked: annotation_flags(doc, d) & FLAG_LOCKED != 0,
fillable: fillable(&subtype),
fill: fill_of(d),
crates/annot/src/modify.rs
+16 -6
View File
@@ -25,6 +25,15 @@ pub(crate) fn refuse_foreign_text(doc: &Document, d: &Dict) -> Result<(), AnnotE
Ok(())
}
/// Thai typed into a text box (`free_text`) that this build can't draw (it has no Thai face): refused
/// with what to do, rather than drawn as `?`.
pub(crate) fn refuse_undrawable_thai(free_text: bool, text: &str) -> Result<(), AnnotError> {
match free_text.then(|| pdfcraft_fonts::shaped_text::thai_refusal(text)).flatten() {
Some(why) => Err(AnnotError::Undrawable(why)),
None => Ok(()),
}
}
/// Format a number for content streams and DA strings.
pub(crate) fn n(v: f64) -> String {
let s = format!("{:.3}", if v.abs() < 5e-4 { 0.0 } else { v });
@@ -165,6 +174,7 @@ pub fn set_contents(doc: &mut Document, page: usize, index: usize, text: &str, m
let (_, r) = annot_ref(doc, page, index)?;
let d = annot_dict(doc, r);
let free_text = d.name(b"Subtype") == Some(b"FreeText");
refuse_undrawable_thai(free_text, text)?;
let snug = free_text && is_snug(&d);
doc.update_dict(r, |d| {
d.set(b"Contents".to_vec(), PdfString::text(text));
@@ -224,8 +234,8 @@ fn fitted_box(doc: &Document, r: ObjRef, text: &str) -> Option<[f64; 4]> {
let longest = text.lines().map(|l| appearance::text_width(l, size)).fold(0.0, f64::max);
(longest + 2.0 * pad + 4.0).clamp(40.0, 300.0)
};
let lines = appearance::wrap(text, size, (w - 2.0 * pad).max(1.0)).len().max(1) as f64;
Some([x0, top - lines * size * 1.2 - 2.0 * pad - 2.0, x0 + w, top])
let height = appearance::text_lines_height(&appearance::wrap(text, size, (w - 2.0 * pad).max(1.0)), size);
Some([x0, top - height - 2.0 * pad - 2.0, x0 + w, top])
}
/// Reply to a comment; returns the reply's index in the page's `/Annots`.
@@ -459,7 +469,7 @@ fn text_extent(d: &Dict, wrap: f64) -> [f64; 2] {
let (_, size) = appearance::parse_da(d);
let lines = appearance::wrap(&text, size, wrap.max(1.0));
let widest = lines.iter().map(|l| appearance::text_width(l, size)).fold(0.0, f64::max);
[widest, lines.len().max(1) as f64 * size * 1.2]
[widest, appearance::text_lines_height(&lines, size)]
}
/// The width and height of FreeText `d`'s text box `tb` as displayed: swapped on a page turned by
@@ -710,7 +720,7 @@ pub fn set_style(doc: &mut Document, page: usize, index: usize, change: StyleCha
let d = annot_dict(doc, r);
let subtype = String::from_utf8_lossy(d.name(b"Subtype").unwrap_or_default()).into_owned();
// Check before changing anything: a stale appearance would contradict the new style.
if appearance::build(&d).is_none() {
if appearance::build(doc, &d).is_none() {
return Err(AnnotError::Unsupported(subtype.clone()));
}
if color.is_some_and(|c| !finite(&c)) || opacity.is_some_and(|o| !o.is_finite()) || width.is_some_and(|w| !w.is_finite()) {
@@ -871,7 +881,7 @@ pub fn set_fill(doc: &mut Document, page: usize, index: usize, fill: Option<Rgb>
return Err(AnnotError::Invalid("only rectangles, ovals, polygons and text boxes can be filled".into()));
}
// Check before changing anything: a stale appearance would contradict the new fill.
if appearance::build(&d).is_none() {
if appearance::build(doc, &d).is_none() {
return Err(AnnotError::Unsupported(subtype));
}
if fill.is_some_and(|c| !finite(&c)) {
@@ -912,7 +922,7 @@ pub fn set_part_opacity(doc: &mut Document, page: usize, index: usize, fill: Opt
return Err(AnnotError::Invalid("invalid opacity".into()));
}
// Check before changing anything: a stale appearance would contradict the new opacity.
if appearance::build(&d).is_none() {
if appearance::build(doc, &d).is_none() {
return Err(AnnotError::Unsupported(subtype));
}
refuse_foreign_text(doc, &d)?;
crates/annot/src/tests.rs
+160 -11
View File
@@ -1350,6 +1350,12 @@ fn the_eraser_cuts_strokes_and_removes_empty_drawings() {
assert!(!list(&doc, 0).iter().any(|d| d.name(b"Subtype") == Some(b"Ink")));
}
/// A document holding only a catalog, for appearance builds from a bare dictionary whose
/// values are all direct.
fn bare_doc() -> Document {
document_of(&[b"<< /Type /Catalog /Pages << /Type /Pages /Kids [] /Count 0 >> >>"])
}
/// A one-page document from object bodies (object 1 is the catalog, 3 the page).
fn document_of(objs: &[&[u8]]) -> Document {
let mut out = b"%PDF-1.7\n".to_vec();
@@ -1445,7 +1451,7 @@ fn malformed_optional_interior_color_does_not_hide_shape_strokes() {
polyline.set(b"IC".to_vec(), malformed);
for (name, annotation) in [("Square", square), ("Circle", circle), ("Polygon", polygon), ("Line", line), ("PolyLine", polyline)] {
assert!(appearance::build(&annotation).is_some(), "malformed optional /IC must not hide a valid {name} stroke");
assert!(appearance::build(&bare_doc(), &annotation).is_some(), "malformed optional /IC must not hide a valid {name} stroke");
}
let doc = document_of(&[
@@ -1540,7 +1546,7 @@ fn a_stamp_without_an_appearance_draws_the_stamp_it_names() {
// A stamp's own artwork is never replaced, and the drawing is for display only: edits
// (restyling, flattening) still find nothing they may write.
assert!(appearance::display_stamp(&stamp_dict("/Name /Draft /AP << /N 9 0 R >>")).is_none());
assert!(appearance::build(&stamp_dict("/Name /Draft")).is_none());
assert!(appearance::build(&bare_doc(), &stamp_dict("/Name /Draft")).is_none());
assert!(appearance::display_stamp(&stamp_dict("/Name /Draft /PCStamp true")).is_none(), "PdfCraft's own stamps are drawn by `build`");
assert!(appearance::display_stamp(&stamp_dict("/Name /Draft /Rect [5 5 5 50]")).is_none(), "no width");
}
@@ -1643,8 +1649,8 @@ fn every_line_ending_is_drawn() {
assert_eq!(call.name(b"LE"), Some(b"Diamond".as_slice()));
assert!(ap_content(&doc, call).contains("h B"), "diamond callout");
assert!(appearance::build(&line_dict([f64::NAN, 0.0, 10.0, 0.0], "OpenArrow")).is_none(), "a non-finite endpoint is rejected");
assert!(appearance::build(&line_dict([0.0, 0.0, 10.0, 0.0], "Foo")).is_none(), "an unknown ending is not replaced");
assert!(appearance::build(&bare_doc(), &line_dict([f64::NAN, 0.0, 10.0, 0.0], "OpenArrow")).is_none(), "a non-finite endpoint is rejected");
assert!(appearance::build(&bare_doc(), &line_dict([0.0, 0.0, 10.0, 0.0], "Foo")).is_none(), "an unknown ending is not replaced");
}
#[test]
@@ -2185,7 +2191,7 @@ fn note_drawing(name: Option<&str>, color: [f64; 3]) -> Vec<u8> {
if let Some(n) = name {
d.set(b"Name".to_vec(), Object::name(n));
}
crate::appearance::build(&d).expect("a note is drawn").decoded().unwrap()
crate::appearance::build(&bare_doc(), &d).expect("a note is drawn").decoded().unwrap()
}
/// ISO 32000-2 §12.5.6.4, Table 175: a viewer shall provide icons for the seven standard names,
@@ -2233,7 +2239,7 @@ fn a_star_note_keeps_its_star_when_recoloured() {
fn content_of(src: &str) -> String {
let doc = document_of(&[src.as_bytes()]);
let d = doc.get(ObjRef::new(1, 0)).as_dict().cloned().expect("a dictionary");
let stream = appearance::build(&d).expect("drawn");
let stream = appearance::build(&bare_doc(), &d).expect("drawn");
String::from_utf8_lossy(&stream.decoded().unwrap()).into_owned()
}
@@ -2285,7 +2291,7 @@ fn bad_dash_arrays_draw_solid() {
for w in [f64::NAN, f64::INFINITY, -2.0] {
let dash = Object::Array([f64::NAN, 2.0].map(Object::Real).to_vec());
d.set(b"Border".to_vec(), Object::Array(vec![Object::Int(0), Object::Int(0), Object::Real(w), dash]));
let c = String::from_utf8_lossy(&appearance::build(&d).unwrap().decoded().unwrap()).into_owned();
let c = String::from_utf8_lossy(&appearance::build(&bare_doc(), &d).unwrap().decoded().unwrap()).into_owned();
assert!(c.contains("1 w") && !c.contains(" d\n") && !c.contains("NaN") && !c.contains("inf"), "{w}: {c}");
}
}
@@ -2307,7 +2313,7 @@ fn a_caret_with_a_paragraph_symbol_draws_the_pilcrow() {
let caret = |extra: &str| {
let doc = document_of(&[format!("<< /Subtype /Caret /Rect [30 30 60 60] /C [0.9 0.1 0.1] {extra} >>").as_bytes()]);
let d = doc.get(ObjRef::new(1, 0)).as_dict().cloned().unwrap();
let s = appearance::build(&d).unwrap();
let s = appearance::build(&bare_doc(), &d).unwrap();
(s.decoded().unwrap(), s.dict.get(b"Resources").and_then(Object::as_dict).is_some_and(|r| r.contains(b"Font")))
};
let has_pilcrow = |c: &[u8]| c.windows(5).any(|w| w == b"(\xb6) T");
@@ -2335,7 +2341,7 @@ fn a_caret_with_a_paragraph_symbol_draws_the_pilcrow() {
}
let doc = document_of(&[b"<< /Subtype /Caret /Rect [30 30 30 30] /Sy /P >>"]);
let d = doc.get(ObjRef::new(1, 0)).as_dict().cloned().unwrap();
assert!(appearance::build(&d).is_some());
assert!(appearance::build(&bare_doc(), &d).is_some());
}
// ── Line annotations: leader lines and captions (ISO 32000-2 §12.5.6.7, Table 178) ──────────
@@ -2355,7 +2361,7 @@ fn table_178_line(l: [f64; 4], extra: &[(&str, Object)]) -> Dict {
/// The built appearance's content.
fn line_drawing(d: &Dict) -> String {
let s = crate::appearance::build(d).expect("the line is drawn");
let s = crate::appearance::build(&bare_doc(), d).expect("the line is drawn");
String::from_utf8_lossy(&s.decoded().unwrap()).into_owned()
}
@@ -2508,7 +2514,7 @@ fn a_diagonal_lines_leaders_are_perpendicular_and_its_caption_turns_with_it() {
fn a_caption_from_rich_text_and_one_the_font_cant_draw() {
let rc = "<?xml version=\"1.0\"?><body xmlns=\"http://www.w3.org/1999/xhtml\"><p>Dim &amp; <b>5</b>0&#x20;mm &lt;x&gt;</p></body>";
let d = table_178_line([0.0, 0.0, 100.0, 0.0], &[("Cap", Object::Bool(true)), ("RC", Object::String(PdfString::text(rc)))]);
let s = crate::appearance::build(&d).unwrap().decoded().unwrap();
let s = crate::appearance::build(&bare_doc(), &d).unwrap().decoded().unwrap();
assert!(s.windows(17).any(|w| w == b"(Dim & 50 mm <x>)"), "{}", String::from_utf8_lossy(&s));
let cjk = table_178_line([0.0, 0.0, 100.0, 0.0], &[("Cap", Object::Bool(true)), ("Contents", Object::String(PdfString::text("長さ")))]);
assert_eq!(crate::appearance::undrawable_text(&cjk), Some(("the line's caption", '長')));
@@ -2779,3 +2785,146 @@ fn indirect_typewriter_names_keep_outer_whitespace_in_summaries() {
check(&doc);
check(&reopen(&doc));
}
/// The `y` of every `1 0 0 1 x y Tm` in `content`: where each piece of text sits.
fn baselines(content: &str) -> Vec<f64> {
content.split(" Tm").filter_map(|s| s.rsplit(' ').next()?.parse().ok()).collect()
}
/// Thai typed into Fill & Sign text or a text box is shaped and drawn from the Thai face, as Add
/// text draws it: Type 3 glyphs (their streams objects of their own) with a ToUnicode map, never
/// `?`. The box grows to hold the ink of its stacked marks and descenders, which reach past the
/// 1.2 em lines boxes are sized by. A build without a Thai face refuses the text, saying why.
#[test]
fn thai_in_fill_sign_text_and_text_boxes_is_drawn_from_the_thai_face() {
let mut doc = fixture();
// Each line has the face's highest stack (น้ำ) and deepest descender (ฎู).
let thai = "น้ำฎู บุญ\nปั้นน้ำฎู";
let item = |shape: Shape| NewAnnotation { page: 0, style: Style::default_for(&shape), shape, contents: thai.into(), author: "Ada".into() };
let shapes = [
Shape::Typewriter { rect: [100.0, 700.0, 300.0, 716.0], font_size: 24.0 },
Shape::TextBox { rect: [100.0, 400.0, 400.0, 430.0], font_size: 30.0 },
];
if pdfcraft_fonts::document_thai_font().is_none() {
for shape in shapes {
let err = add_annotation(&mut doc, &item(shape), &meta("t")).unwrap_err();
assert!(matches!(&err, AnnotError::Undrawable(m) if m.contains("Thai") && m.contains("CRAFT_FONTS_DIR")), "{err:?}");
}
assert_eq!(list(&doc, 0).len(), 1, "nothing was added");
let t = add_annotation(&mut doc, &new(0, Shape::TextBox { rect: [100.0, 600.0, 300.0, 650.0], font_size: 12.0 }), &meta("b")).unwrap();
assert!(matches!(set_contents(&mut doc, 0, t, thai, &meta("")), Err(AnnotError::Undrawable(_))));
assert_eq!(text(&list(&doc, 0)[t], b"Contents"), "hello");
return;
}
let reach = pdfcraft_fonts::shaped_text::thai_reach;
let (lo, hi) = (reach("ฎู").unwrap()[0], reach("น้ำ").unwrap()[1]);
for (shape, size) in shapes.into_iter().zip([24.0, 30.0]) {
let t = add_annotation(&mut doc, &item(shape), &meta("t")).unwrap();
let doc = reopen(&doc);
let d = &list(&doc, 0)[t];
assert_eq!((text(d, b"Contents").as_str(), pdfcraft_fonts::undrawn_mark(d)), (thai, None));
let content = ap_content(&doc, d);
assert!(!content.contains('?') && content.contains("/PCTh1 "), "{content}");
// The glyphs and the text they stand for are objects of the saved file.
let n = d.get(b"AP").and_then(|a| a.as_dict()).and_then(|a| a.reference(b"N")).unwrap();
let ap = doc.get(n);
let fonts = ap.as_dict().and_then(|s| s.get(b"Resources")).and_then(|r| r.as_dict()).and_then(|r| r.get(b"Font")).and_then(|f| f.as_dict());
let font = fonts.and_then(|f| f.get(b"PCTh1")).and_then(|f| f.as_dict()).unwrap().clone();
assert_eq!(font.name(b"Subtype"), Some(&b"Type3"[..]));
let cmap = doc.get(font.reference(b"ToUnicode").unwrap());
let Object::Stream(cmap) = &*cmap else { panic!("ToUnicode is a stream object") };
let cmap = String::from_utf8_lossy(&cmap.decoded().unwrap()).into_owned();
assert!(cmap.contains("<0E190E490E33>"), "น้ำ copies as typed: {cmap}");
let procs = font.get(b"CharProcs").and_then(|p| p.as_dict()).unwrap();
assert!(procs.iter().all(|(_, g)| matches!(g, Object::Ref(_))), "glyph programs are stream objects");
// Every line's ink lies inside the box: the appearance's BBox and its clip.
let (r, ys) = (rect(d), baselines(&content));
let (top, bottom) = (ys.iter().copied().fold(f64::MIN, f64::max), ys.iter().copied().fold(f64::MAX, f64::min));
assert!(top + hi * size <= r[3] + 1e-3 && bottom + lo * size >= r[1] - 1e-3, "{ys:?} at {size} in {r:?}");
}
// New text refits the box to its ink too: the stack on its first line, the descender on its last.
set_contents(&mut doc, 0, 1, "น้ำ\nฎู", &meta("")).unwrap();
let d = &list(&doc, 0)[1];
let (r, ys) = (rect(d), baselines(&ap_content(&doc, d)));
assert!(ys[0] + hi * 24.0 <= r[3] + 1e-3 && ys[ys.len() - 1] + lo * 24.0 >= r[1] - 1e-3, "{ys:?} in {r:?}");
}
const QUAD: [f64; 8] = [20.0, 110.0, 40.0, 110.0, 20.0, 95.0, 40.0, 95.0];
fn reals(v: &[f64]) -> Vec<Object> {
v.iter().map(|x| Object::Real(*x)).collect()
}
/// The appearance a `subtype` with `/QuadPoints` `quads` draws, as text.
fn quad_appearance(subtype: &str, quads: Vec<Object>) -> Option<String> {
quad_appearance_with(&bare_doc(), subtype, Object::Array(quads))
}
fn quad_appearance_with(doc: &Document, subtype: &str, quad_points: Object) -> Option<String> {
let mut d = Dict::new();
d.set(b"Subtype".to_vec(), Object::name(subtype));
d.set(b"Rect".to_vec(), Object::Array([20, 95, 40, 110].map(Object::Int).to_vec()));
d.set(b"C".to_vec(), Object::Array(vec![Object::Int(1), Object::Int(0), Object::Int(0)]));
d.set(b"QuadPoints".to_vec(), quad_points);
appearance::build(doc, &d).map(|s| String::from_utf8_lossy(&s.decoded().unwrap()).into_owned())
}
/// Appearance drawing reads `/QuadPoints` by the rule the other readers do, indirect objects
/// included: entries by reference, and the array itself by reference, draw the same quads as
/// the direct values.
#[test]
fn appearances_draw_indirect_quads_like_direct_ones() {
for subtype in ["Redact", "Highlight"] {
let clean = quad_appearance(subtype, reals(&QUAD)).unwrap();
let mut doc = bare_doc();
let by_ref: Vec<Object> = QUAD.iter().map(|v| Object::Ref(doc.add(Object::Real(*v)))).collect();
assert_eq!(quad_appearance_with(&doc, subtype, Object::Array(by_ref)).as_deref(), Some(clean.as_str()), "{subtype}: entries by reference");
let array_ref = Object::Ref(doc.add(Object::Array(reals(&QUAD))));
assert_eq!(quad_appearance_with(&doc, subtype, array_ref).as_deref(), Some(clean.as_str()), "{subtype}: the array by reference");
}
}
/// Redaction outlines and text markup draw the quads redaction apply would use, no more: a NaN
/// quad wrote `NaN` into the stream, and one non-number or a trailing partial quad left the other,
/// usable quads undrawn.
#[test]
fn appearances_draw_only_usable_quads() {
let mut one_nan = QUAD;
one_nan[3] = f64::NAN;
for subtype in ["Redact", "Highlight"] {
let clean = quad_appearance(subtype, reals(&QUAD)).unwrap();
assert!(!clean.contains("NaN"), "{clean}");
let mut name_first = reals(&[QUAD, QUAD].concat());
name_first[2] = Object::name("oops");
let same =
[reals(&[one_nan, QUAD].concat()), name_first, reals(&[&QUAD[..], &QUAD[..4]].concat()), reals(&[QUAD, [f64::INFINITY; 8]].concat())];
for (i, quads) in same.into_iter().enumerate() {
assert_eq!(quad_appearance(subtype, quads).as_deref(), Some(clean.as_str()), "{subtype} case {i}");
}
// No usable quad draws nothing, as with no /QuadPoints.
assert_eq!(quad_appearance(subtype, reals(&[f64::NAN; 8])), None, "{subtype}");
}
}
/// The comment list reads `/QuadPoints` by the rule redaction apply does: unusable quads are left
/// out, not read as zeros, and an indirect number is followed.
#[test]
fn summaries_list_only_usable_quads() {
let mut doc = fixture();
let forty = Object::Ref(doc.add(Object::Int(40)));
let page = page_refs(&doc).unwrap()[0];
let mut entries = annots(&doc, page);
let Object::Dict(annotation) = &mut entries[0] else { panic!("fixture annotation is inline") };
// A non-number in the first quad, an indirect 40 in the second, NaN in the third, and four
// entries of a fourth.
let mut quads = reals(&[QUAD, QUAD, QUAD, QUAD].concat());
quads[1] = Object::name("oops");
quads[10] = forty;
quads[17] = Object::Real(f64::NAN);
quads.truncate(28);
annotation.set(b"Subtype".to_vec(), Object::name("Redact"));
annotation.set(b"QuadPoints".to_vec(), Object::Array(quads));
set_annots(&mut doc, page, entries).unwrap();
let summary = summaries(&doc).into_iter().find(|a| a.page == 0 && a.index == 0).unwrap();
assert_eq!(summary.quads, vec![QUAD.map(|v| v as f32)]);
}
crates/automation/README.md
+28 -1
View File
@@ -146,7 +146,10 @@ claude mcp add pdfcraft -- pdfcraft-cli mcp --root <dir>
`object_list {doc, page}` inventories recognised paragraphs, Image/Form artwork and added items
with generation-scoped kind/index references. `object_move {doc, page, objects, offset, generation?}`
moves 1–1000 references atomically in displayed points (right/down), preserving source glyph bytes
and fonts. A refusal changes nothing; success is one undo step. See
and fonts. A refusal changes nothing; success is one undo step. `all: true` takes every object on the
page instead, as ⌘A does. `object_copy {doc, page, objects | all, cut?}` copies them with every
resource they use and `object_paste {doc, page}` pastes them as page content that draws the same,
into the same or another open document, stepped as the app steps a paste. See
[the selection and movement guide](../../docs/editing-multiple-objects.md) for limits and examples.
## Accessibility review record
@@ -209,6 +212,30 @@ undoable and survive saving/reopening. `image_hit` shares the UI's curve and hol
The bounds, limits and font requirements are in [existing-text editing](../../docs/editing-existing-text.md)
and [image editing](../../docs/editing-images.md).
`text_fonts` also lists `builtin:thai` when the build includes the bundled Noto Sans Thai face
(the optional craft-fonts input). Passing it as `text_edit.font_resource` explicitly replaces a
paragraph or range with that face; the source font is never swapped for it silently. A build
without the face never lists it, and `text_edit` refuses it there. A page whose own fonts fill
the listing's 32 MiB preview budget can leave it out too; `text_edit` still accepts it then.
### Preparing grouped text
`text_prepare {doc, page}` makes supported grouped Form text editable on one page, as one click
on grouped text in the GUI's Edit text does. It keeps the group's original glyph operators and shared source
objects, records preparation and map repairs in the repair log, and is one undoable step. List
`text_paragraphs` afterwards; its indices belong to the prepared page. Besides the document
summary it returns what this edit did:
| Field | Type | Meaning |
| --- | --- | --- |
| `prepared_groups` | integer | Form placements drawn in place, so their text is now the page's own. |
| `repaired_fonts` | integer | Page fonts whose damaged legacy THSarabunPSK Thai map was repaired on a page-owned copy. |
| `refused_groups` | array of strings | One `name: reason` per group left as it was: unsafe compositing, tags, malformed content, or text grouped more than eight levels deep. |
The page's own fonts with that damaged map are repaired too, whether their text was grouped or
not. It fails, changing nothing, when the page has nothing to prepare or repair (the error gives
the first reason a group was left as it was) or the page is out of range.
### Superscript and subscript normalization
`text_script` changes characters to their Unicode superscript or subscript forms; it does not
crates/automation/src/content.rs
+64 -11
View File
@@ -354,14 +354,32 @@ impl Automation {
Ok(json!({"page":page+1,"generation":doc.edit_generation(),"count":items.len(),"objects":items}))
}
pub(crate) fn object_move(&mut self, a: &Args) -> Result<Value> {
/// The references `objects` names (from object_list), or with `all` every object on `page`,
/// as ⌘A selects them in Edit a PDF; refused when `generation` is not the document's.
fn object_targets(&self, a: &Args, page: usize) -> Result<Vec<pdfcraft_engine::ObjectTarget>> {
use pdfcraft_engine::{MAX_MOVE_OBJECTS, ObjectKind, ObjectTarget};
let page = self.page(a)?;
let doc = self.doc(a)?;
if let Some(g) = a.get("generation") {
let g = g.as_u64().ok_or_else(|| bad("generation must be a non-negative integer from object_list"))?;
if g != doc.edit_generation() {
return Err(failed("the document changed; call object_list again"));
}
}
if a.opt_bool("all")?.unwrap_or(false) {
if a.get("objects").is_some() {
return Err(bad("give objects or all, not both"));
}
let all: Vec<_> = doc.editable_objects(page).map_err(failed)?.into_iter().map(|o| o.target).collect();
if all.is_empty() || all.len() > MAX_MOVE_OBJECTS {
return Err(failed(format!("page {} has {} objects; select 1–1000", page + 1, all.len())));
}
return Ok(all);
}
let rows = a
.get("objects")
.and_then(Value::as_array)
.filter(|r| !r.is_empty() && r.len() <= MAX_MOVE_OBJECTS)
.ok_or_else(|| bad("objects must contain 1–1000 references from object_list"))?;
.ok_or_else(|| bad("objects must contain 1–1000 references from object_list (or pass all: true)"))?;
let mut targets = Vec::with_capacity(rows.len());
for row in rows {
let row = row.as_object().ok_or_else(|| bad("each object must contain kind and index"))?;
@@ -382,6 +400,12 @@ impl Automation {
.ok_or_else(|| bad("object index must be a positive integer from object_list"))?;
targets.push(ObjectTarget { kind, index });
}
Ok(targets)
}
pub(crate) fn object_move(&mut self, a: &Args) -> Result<Value> {
let page = self.page(a)?;
let targets = self.object_targets(a, page)?;
let delta = a
.get("offset")
.and_then(Value::as_array)
@@ -389,17 +413,46 @@ impl Automation {
.and_then(|v| Some([v.first()?.as_f64()?, v.get(1)?.as_f64()?]))
.filter(|v| v.iter().all(|n| n.is_finite() && n.abs() <= 1e9))
.ok_or_else(|| bad("offset must contain two finite numbers [dx, dy] in displayed points"))?;
let doc = self.doc(a)?;
if let Some(g) = a.get("generation") {
let g = g.as_u64().ok_or_else(|| bad("generation must be a non-negative integer from object_list"))?;
if g != doc.edit_generation() {
return Err(failed("the document changed; call object_list again before moving"));
}
}
let offset = doc.object_move_offset(page, delta).map_err(failed)?;
let offset = self.doc(a)?.object_move_offset(page, delta).map_err(failed)?;
let count = targets.len();
let mut result = self.apply(a, Edit::MoveObjects { page, objects: targets, offset })?;
result["moved"] = json!(count);
Ok(result)
}
/// Edit ▸ Copy (or with `cut`, Cut) of objects selected together, kept for object_paste into
/// any open document. A cut then deletes them as one undo step.
pub(crate) fn object_copy(&mut self, a: &Args) -> Result<Value> {
let page = self.page(a)?;
let targets = self.object_targets(a, page)?;
let cut = a.opt_bool("cut")?.unwrap_or(false);
let doc = self.doc(a)?;
if cut && !doc.allows_modification() {
return Err(failed(pdfcraft_engine::EditError::NotPermitted("changes")));
}
let clip = doc.copy_objects(page, &targets).map_err(failed)?;
let out = json!({ "copied": clip.objects().len(), "text": clip.text() });
self.objects_clipboard = Some(pdfcraft_engine::ObjectClipboard::new(clip));
if !cut {
return Ok(out);
}
let delete = Edit::delete_objects(page, targets).ok_or_else(|| failed("nothing to cut"))?;
let mut result = self.apply(a, delete)?;
result["copied"] = out["copied"].clone();
result["text"] = out["text"].clone();
Ok(result)
}
/// Edit ▸ Paste of the objects object_copy copied: onto `page`, where they were, or on the
/// page they came from (and on each further paste onto a page) 18 points down and right.
pub(crate) fn object_paste(&mut self, a: &Args) -> Result<Value> {
let page = self.page(a)?;
let mut clipboard = self.objects_clipboard.take().ok_or_else(|| failed("copy objects first (object_copy)"))?;
let count = clipboard.clip().objects().len();
let edit = self.doc(a).and_then(|doc| clipboard.paste(doc, page).map_err(failed));
self.objects_clipboard = Some(clipboard);
let mut result = self.apply(a, edit?)?;
result["pasted"] = json!(count);
Ok(result)
}
}
crates/automation/src/dispatch.rs
+7
View File
@@ -467,6 +467,7 @@ impl Automation {
"bring_forward" => ImageEdit::Arrange(pdfcraft_engine::ArrangeOrder::BringForward),
"send_backward" => ImageEdit::Arrange(pdfcraft_engine::ArrangeOrder::SendBackward),
"remove_crop" => ImageEdit::RemoveCrop,
"ungroup" => ImageEdit::Ungroup,
"replace" => {
let path = self.resolve(a.str("path")?, false)?;
let bytes = read_input(&path)?;
@@ -480,6 +481,10 @@ impl Automation {
self.apply(&a, Edit::EditPageImage { page, index, change })?
}
}
"text_prepare" => {
let page = self.page(&a)?;
self.prepare_text(&a, page)?
}
"text_paragraphs" => {
let page = self.page(&a)?;
let doc = self.doc_to_copy(&a)?;
@@ -733,6 +738,8 @@ impl Automation {
"links_remove" => self.links_remove(&a)?,
"object_list" => self.object_list(&a)?,
"object_move" => self.object_move(&a)?,
"object_copy" => self.object_copy(&a)?,
"object_paste" => self.object_paste(&a)?,
"content_list" => self.content_list(&a)?,
"page_add_text" => self.page_add_text(&a)?,
"page_add_image" => self.page_add_image(&a)?,
crates/automation/src/documents.rs
+17
View File
@@ -502,6 +502,23 @@ impl Automation {
pub(crate) fn apply(&mut self, a: &Args, edit: Edit) -> Result<Value> {
let id = self.doc(a)?.id;
self.session.apply(id, edit).map_err(failed)?;
self.applied(a)
}
/// `text_prepare`: make a page's grouped text editable, with what that edit itself prepared,
/// repaired and left grouped.
pub(crate) fn prepare_text(&mut self, a: &Args, page: usize) -> Result<Value> {
let id = self.doc(a)?.id;
let prepared = self.session.prepare_text(id, page).map_err(failed)?;
let mut out = self.applied(a)?;
out["prepared_groups"] = json!(prepared.expanded);
out["repaired_fonts"] = json!(prepared.repaired_fonts);
out["refused_groups"] = json!(prepared.refused);
Ok(out)
}
/// The document's summary after an edit, with what it stored as typed but drew with `?`.
fn applied(&mut self, a: &Args) -> Result<Value> {
let doc = self.doc(a)?;
let mut out = summary(doc);
// Text the edit stored as typed but drew with `?` (its font can't draw it): said, not silent.
crates/automation/src/lib.rs
+3
View File
@@ -101,6 +101,8 @@ pub struct Automation {
/// Review sessions held for `ocr_words` (a guard and the pages read), by document.
ocr_reviews: HashMap<DocId, OcrReview>,
next_ocr_review: u64,
/// What object_copy copied, for object_paste into any open document.
objects_clipboard: Option<pdfcraft_engine::ObjectClipboard>,
}
impl Default for Automation {
@@ -121,6 +123,7 @@ impl Automation {
evicted: HashSet::new(),
ocr_reviews: HashMap::new(),
next_ocr_review: 0,
objects_clipboard: None,
}
}
crates/automation/src/tools.rs
+25 -12
View File
@@ -204,12 +204,22 @@ pub fn tools() -> Vec<ToolDef> {
}), &["doc", "page"])),
t("object_list", "List editable objects", "List existing paragraphs, Image/Form artwork and added content once, with source kind/index references, displayed rectangles and document generation. Indexes change when content changes; use generation in object_move to reject stale references.")
.ro().with(schema(json!({"doc":doc(),"page":{"type":"integer","minimum":1}}), &["doc","page"])),
t("object_move", "Move objects together", "Atomically translate 1–1000 mixed references from object_list by [dx, dy] in displayed points (right/down). Preserves glyph codes, fonts, image data and relative spacing; success is one undo step. Text used as a clipping path and vertical text are refused. A bad/stale reference refuses the entire move.")
t("object_move", "Move objects together", "Atomically translate 1–1000 mixed references from object_list (or with all: true every object on the page) by [dx, dy] in displayed points (right/down). Preserves glyph codes, fonts, image data and relative spacing; success is one undo step. Text used as a clipping path and vertical text are refused. A bad/stale reference refuses the entire move.")
.with(schema(json!({"doc":doc(),"page":{"type":"integer","minimum":1},
"objects":{"type":"array","minItems":1,"maxItems":1000,"uniqueItems":true,"items":{
"type":"object","properties":{"kind":{"type":"string","enum":["added","text","image"]},"index":{"type":"integer","minimum":1}},"required":["kind","index"],"additionalProperties":false}},
"all":{"type":"boolean","description":"Every object on the page, instead of objects."},
"offset":{"type":"array","items":{"type":"number"},"minItems":2,"maxItems":2},
"generation":{"type":"integer","minimum":0,"description":"Use generation returned by object_list to reject a stale selection."}}), &["doc","page","objects","offset"])),
"generation":{"type":"integer","minimum":0,"description":"Use generation returned by object_list to reject a stale selection."}}), &["doc","page","offset"])),
t("object_copy", "Copy or cut objects together", "Edit ▸ Copy (cut: Cut) of 1–1000 mixed references from object_list, or with all: true every object on the page, as ⌘A then ⌘C select and copy them in Edit a PDF. The copy keeps their content operators and every resource they use (fonts, images, ExtGStates, colour spaces, patterns, shadings) and is kept for object_paste into any open document, also after this one changes or closes. cut then deletes them as one undo step. Refused when the document doesn't allow copying (cut: or changes), or when text used as a clipping path is drawn before them. Returns the count copied and their text.")
.with(schema(json!({"doc":doc(),"page":{"type":"integer","minimum":1},
"objects":{"type":"array","minItems":1,"maxItems":1000,"uniqueItems":true,"items":{
"type":"object","properties":{"kind":{"type":"string","enum":["added","text","image"]},"index":{"type":"integer","minimum":1}},"required":["kind","index"],"additionalProperties":false}},
"all":{"type":"boolean","description":"Every object on the page, instead of objects."},
"cut":{"type":"boolean","description":"Delete them after copying (one undo step)."},
"generation":{"type":"integer","minimum":0,"description":"Use generation returned by object_list to reject a stale selection."}}), &["doc","page"])),
t("object_paste", "Paste copied objects", "Edit ▸ Paste of what object_copy copied, onto page of doc as page content that draws the same: where it was, or on the page it came from 18 points down and to the right, and 18 more for each further paste onto the same page; centred when that would leave it off the page. Resources join the page's under names that don't clash; into another document the fonts, images and other objects they use are copied (each once, sharing what is there; refused past 20,000 objects or 256 MB). Pasted content is untagged and in no layer. One undo step.")
.with(schema(json!({"doc":doc(),"page":{"type":"integer","minimum":1}}), &["doc","page"])),
t("text_word_count", "Word count", "Count Unicode words and characters on selected pages (default all), with totals and per-page counts. Table/code exclusions use PDF structure tags, not visual guesses; unclassified text is retained. Image exclusion removes Figure-tagged and invisible/OCR text. Does not run OCR. Copy permission is required. Fails rather than returning a partial total when a page cannot be read.")
.ro()
.cmd("file.word_count")
@@ -483,7 +493,7 @@ pub fn tools() -> Vec<ToolDef> {
t(
"form_fill",
"Fill in form fields",
"Set several fields at once (one undo step). values maps field names (from form_fields) to: a string for text fields, combo boxes and radio groups (an option), true/false for check boxes, an array of strings for multi-select lists. Appearances are regenerated so every viewer shows the values. A value whose characters the field's font can't draw (Chinese in a Helvetica field) is still saved; `warnings` in the result names the field and character: PdfCraft draws `?` there and asks other viewers to redraw the field from its value.",
"Set several fields at once (one undo step). values maps field names (from form_fields) to: a string for text fields, combo boxes and radio groups (an option), true/false for check boxes, an array of strings for multi-select lists. Appearances are regenerated so every viewer shows the values. Thai the field's font can't hold is drawn from PdfCraft's built-in Thai face (a build without one refuses it). Any other value whose characters the field's font can't draw (Chinese in a Helvetica field) is still saved; `warnings` in the result names the field and character: PdfCraft draws `?` there and asks other viewers to redraw the field from its value.",
)
.with(schema(json!({ "doc": doc(), "values": { "type": "object", "description": "Field name → value." } }), &["doc", "values"])),
t("form_reset", "Clear form", "Reset fields to their default values: all of them, or only those listed. Undoable.")
@@ -855,7 +865,7 @@ pub fn tools() -> Vec<ToolDef> {
"Add a comment as Acrobat's commenting tools do. Geometry is in points with the origin at the top-left of the displayed page, y down (as in page_render images at 72 dpi and text_find rects). \
note: `at` [x, y] (icon top-left). stamp: `at` [x, y] (its centre) and `stamp`: approved, completed, confidential, draft, final, for comment, for public release, information only, not approved, not for public release, preliminary results, void, accepted, initial here, rejected, sign here, witness; dynamic: true for the dynamic approved/confidential/received/reviewed/revised stamps with a By … at … line. highlight/underline/strikeout/squiggly: `find` (text on the page to mark; every match with all: true) or `quads`. \
rectangle/oval/textbox: `rect` [x0, y0, x1, y1]. line/arrow: `from`, `to`. Optional `endings` (two names: start, end) for a line, arrow or polyline, or one name for a callout: None, Square, Circle, Diamond, OpenArrow, ClosedArrow, Butt, ROpenArrow, RClosedArrow, Slash. An arrow without endings is None then OpenArrow. ink: `strokes` [[[x, y], …], …]. \
polygon/cloud/polyline (connected lines): `points` [[x, y], …]. callout: `rect` (its text box), `to` (the point the arrow touches), optional `knee`. caret (inserted text): `at`, the insertion point on the baseline. replace (Replace Text): `find` or `quads` like highlight, `contents` the replacement. attachment: `path` (the file), `at`, optional `icon` (PushPin, Paperclip, Graph, Tag). Undoable. Text a text box, callout or stamp draws in the standard fonts that they can't draw (Chinese, Greek) is saved as typed and drawn as `?`; `warnings` in the result says so.",
polygon/cloud/polyline (connected lines): `points` [[x, y], …]. callout: `rect` (its text box), `to` (the point the arrow touches), optional `knee`. caret (inserted text): `at`, the insertion point on the baseline. replace (Replace Text): `find` or `quads` like highlight, `contents` the replacement. attachment: `path` (the file), `at`, optional `icon` (PushPin, Paperclip, Graph, Tag). Undoable. Thai in a text box or callout is drawn from PdfCraft's built-in Thai face (a build without one refuses it). Other text a text box, callout or stamp draws in the standard fonts that they can't draw (Chinese, Greek) is saved as typed and drawn as `?`; `warnings` in the result says so.",
)
.with(schema(
json!({
@@ -1126,7 +1136,7 @@ pub fn tools() -> Vec<ToolDef> {
t(
"doc_header_footer",
"Add header and footer",
"Add a header and/or footer to pages. Text boxes: header_left/center/right, footer_left/center/right. Tokens: <<1>> page number, <<n>> page count, <<1 of n>>, <<Page 1 of n>>, <<1/n>>, dates <<m/d/yyyy>> <<yyyy-mm-dd>> <<mmmm d, yyyy>> <<dd.mm.yyyy>> (and more), Bates <<Bates Number#6#1#PREFIX#SUFFIX>>. shrink: true scales the page content down evenly into the room the header and footer leave free, so they don't cover it. font: Helvetica (default), Times or Courier. replace: true swaps out existing ones (Update). Undoable.",
"Add a header and/or footer to pages. Text boxes: header_left/center/right, footer_left/center/right. Tokens: <<1>> page number, <<n>> page count, <<1 of n>>, <<Page 1 of n>>, <<1/n>>, dates <<m/d/yyyy>> <<yyyy-mm-dd>> <<mmmm d, yyyy>> <<dd.mm.yyyy>> (and more), Bates <<Bates Number#6#1#PREFIX#SUFFIX>>. shrink: true scales the page content down evenly into the room the header and footer leave free, so they don't cover it. font: Helvetica (default), Times or Courier; Thai is drawn from PdfCraft's built-in Thai face beside it (a build without one refuses it). replace: true swaps out existing ones (Update). Undoable.",
)
.cmd("edit.header_footer")
.with(schema(
@@ -1145,7 +1155,7 @@ pub fn tools() -> Vec<ToolDef> {
}),
&["doc"],
)),
t("doc_watermark", "Add watermark", "Add a watermark to pages: text, or a picture from `file` (an image, or page `file_page` of a PDF) at `scale` of the page (default 0.5); rotated (degrees counter-clockwise, default 45), semi-transparent (opacity 0–1, default 0.5), text fitted to the page unless font_size is given, on top unless behind: true; moved from the page centre by `offset_x` (right) and `offset_y` (up) points (default 0). replace: true swaps out existing ones. Undoable.")
t("doc_watermark", "Add watermark", "Add a watermark to pages: text, or a picture from `file` (an image, or page `file_page` of a PDF) at `scale` of the page (default 0.5); rotated (degrees counter-clockwise, default 45), semi-transparent (opacity 0–1, default 0.5), text fitted to the page unless font_size is given (Thai drawn from PdfCraft's built-in Thai face; a build without one refuses it), on top unless behind: true; moved from the page centre by `offset_x` (right) and `offset_y` (up) points (default 0). replace: true swaps out existing ones. Undoable.")
.cmd("edit.watermark")
.with(schema(
json!({
@@ -1328,7 +1338,7 @@ pub fn tools() -> Vec<ToolDef> {
json!({ "doc": doc(), "field": { "type": "string" }, "event": { "type": "string", "enum": ["keystroke", "format", "validate", "calculate", "mouse_up"] }, "script": { "type": "string" } }),
&["doc", "field", "event"],
)),
t("doc_export_office", "Export to Word, PowerPoint, HTML, RTF or Markdown", "Export a PDF ▸ Word (.docx), PowerPoint (.pptx, one slide per page), HTML (.html, one file with images inline), RTF (.rtf) or Markdown (.md or .markdown: pipe tables, images left out), chosen by path's extension: text, headings, bold and italic. HTML uses rich styling by default; set html_mode to simple for CSS-free headings, paragraphs, emphasis and tables. Simple HTML keeps text and inline images but omits colour and layout styling. Word uses borderless inferred tables and positioned editable shapes on pages with artwork, rotated or large text; each source page retains its size. Font matching and vector graphics remain limited. Export reads forms 50 deep and up to 50 000 a page, counting forms the page draws itself. Pages that go past either limit, or that have pictures past what one export may hold or decode, are listed in truncated_pages, with a warning; what was left out is missing.")
t("doc_export_office", "Export to Word, PowerPoint, HTML, RTF or Markdown", "Export a PDF ▸ Word (.docx), PowerPoint (.pptx, one slide per page), HTML (.html, one file with images inline), RTF (.rtf) or Markdown (.md or .markdown: pipe tables, images left out), chosen by path's extension: text, headings, bold and italic. HTML uses rich styling by default; set html_mode to simple for CSS-free headings, paragraphs, emphasis and tables. Simple HTML keeps text and inline images but omits colour and layout styling. Word uses inferred tables, bordered only along the grid lines the page draws, and positioned editable shapes on pages with artwork, rotated or large text; each source page retains its size. Font matching and vector graphics remain limited. Export reads forms 50 deep and up to 50 000 a page, counting forms the page draws itself. Pages that go past either limit, or that have pictures past what one export may hold or decode, are listed in truncated_pages, with a warning; what was left out is missing.")
.cmd("export.docx")
.with(schema(json!({ "doc": doc(), "path": { "type": "string" }, "html_mode": { "type": "string", "enum": ["rich", "simple"], "default": "rich" } }), &["doc", "path"])),
t("pdfa_verify", "Verify PDF/A", "Standards ▸ Verify PDF/A compliance: the PDF/A-2b or 3b rules the document breaks (ISO 19005 clause, message, page, whether Save as PDF/A can fix it), plus what it declares.")
@@ -1480,7 +1490,7 @@ pub fn tools() -> Vec<ToolDef> {
t(
"fill_sign_add",
"Fill & Sign: type text or place a mark",
"Fill in a form that has no fields, as Acrobat's Fill & Sign does: type text (`text`, default 10 pt; `font_size` 4–300 pt; optional `color` as #RRGGBB for text/date, default black; `border` 0–12 pt around it, default none, in its colour or `border_color`), place a check, cross, dot or line, today's date (in Preferences ▸ Date format, see fill_sign_date_format, or `format`; `font_size` also applies), or a signature or initials (either `text` drawn in a script font, or `path` to a local PNG/JPEG image up to 4 MiB and 32 megapixels, scaled down to 4 megapixels, preserving transparency; with `scan: true`, `path` is instead a photo of a signature on paper, within the same limits: the signature is found, and the paper, shadows and table around it are removed, with `sensitivity` 0–100, default 50, picking up fainter ink when higher). `at` [x, y] is in points from the top-left of the page: text's top-left, mark's centre, signature's left edge centred vertically. Image signatures fit within 150 pt wide and 32 pt tall (24 pt for initials). Creates movable, undoable annotations. Typed text the standard fonts can't draw is saved and drawn as `?` (`warnings` in the result says so); a typed signature neither the script font nor its fallback can draw is refused (pass `path` instead).",
"Fill in a form that has no fields, as Acrobat's Fill & Sign does: type text (`text`, default 10 pt; `font_size` 4–300 pt; optional `color` as #RRGGBB for text/date, default black; `border` 0–12 pt around it, default none, in its colour or `border_color`), place a check, cross, dot or line, today's date (in Preferences ▸ Date format, see fill_sign_date_format, or `format`; `font_size` also applies), or a signature or initials (either `text` drawn in a script font, or `path` to a local PNG/JPEG image up to 4 MiB and 32 megapixels, scaled down to 4 megapixels, preserving transparency; with `scan: true`, `path` is instead a photo of a signature on paper, within the same limits: the signature is found, and the paper, shadows and table around it are removed, with `sensitivity` 0–100, default 50, picking up fainter ink when higher). `at` [x, y] is in points from the top-left of the page: text's top-left, mark's centre, signature's left edge centred vertically. Image signatures fit within 150 pt wide and 32 pt tall (24 pt for initials). Creates movable, undoable annotations. Typed Thai is drawn from PdfCraft's built-in Thai face (a build without one refuses it); other typed text the standard fonts can't draw is saved and drawn as `?` (`warnings` in the result says so); a typed signature neither the script font nor its fallback can draw is refused (pass `path` instead).",
)
.cmd("sign.fill.text")
.with(schema(
@@ -1563,14 +1573,14 @@ pub fn tools() -> Vec<ToolDef> {
.ro()
.cmd("edit.edit_text")
.with(schema(json!({ "doc": doc(), "page": { "type": "integer", "minimum": 1 } }), &["doc", "page"])),
t("image_edit", "Edit an image", "Change one of a page's images (number from page_images): action move (rect: new full placement box in top-left-origin points, based on page_images.rect), crop (rect: the part to keep, top-left-origin points; the pixels stay in place and page_images.visible_rect shows the result; a later move carries the crop along), remove_crop (remove a crop created by PdfCraft while preserving clipping paths from the original document), rotate (quarters clockwise, default 1), rotate_by (degrees: any angle clockwise about the visible centre; the box in page_images then becomes the turned image's bounding box), transform (matrix: 6 numbers [a, b, c, d, e, f] in top-left-origin points, applied after the current placement so a point (x, y) of the image moves to (a·x + c·y + e, b·x + d·y + f); finite, with a·d − b·c not 0; the box in page_images becomes the transformed image's bounding box), flip_horizontal (reflect left/right on the displayed page), flip_vertical (reflect top/bottom on the displayed page; both flip actions follow the page rotation), replace (path: an image file, drawn in the same place), delete, or arrange: bring_to_front, send_to_back, bring_forward, send_backward (reorder stacking order among content on the page; errors when the image is already at the requested limit; arranging an added image converts it into an ordinary page image). Undoable. The selected image crop transforms with its pixels, isolating shared path scopes while retaining neighboring painting. Text-outline clips return a clear error. Rotation/flip use the visible centre. Form artwork supports move/resize/rotate/flip/delete as a group; replace and image_save only support raster images.")
t("image_edit", "Edit an image", "Change one of a page's images (number from page_images): action move (rect: new full placement box in top-left-origin points, based on page_images.rect), crop (rect: the part to keep, top-left-origin points; the pixels stay in place and page_images.visible_rect shows the result; a later move carries the crop along), remove_crop (remove a crop created by PdfCraft while preserving clipping paths from the original document), rotate (quarters clockwise, default 1), rotate_by (degrees: any angle clockwise about the visible centre; the box in page_images then becomes the turned image's bounding box), transform (matrix: 6 numbers [a, b, c, d, e, f] in top-left-origin points, applied after the current placement so a point (x, y) of the image moves to (a·x + c·y + e, b·x + d·y + f); finite, with a·d − b·c not 0; the box in page_images becomes the transformed image's bounding box), flip_horizontal (reflect left/right on the displayed page), flip_vertical (reflect top/bottom on the displayed page; both flip actions follow the page rotation), replace (path: an image file, drawn in the same place), delete, ungroup (Form artwork only: its content is drawn in place, so its text becomes editable with text_paragraphs/text_edit and its pictures separate; refused for knockout groups and for transparency groups that blend inside, are drawn translucent or blend in a colour space other than the page's, which would composite differently, and for forms whose /DefaultGray, /DefaultRGB or /DefaultCMYK the page doesn't share, which would stop remapping their colours), or arrange: bring_to_front, send_to_back, bring_forward, send_backward (reorder stacking order among content on the page; errors when the image is already at the requested limit; arranging an added image converts it into an ordinary page image). Undoable. The selected image crop transforms with its pixels, isolating shared path scopes while retaining neighboring painting. Text-outline clips return a clear error. Rotation/flip use the visible centre. Form artwork supports move/resize/rotate/flip/delete as a group; replace and image_save only support raster images.")
.cmd("edit.edit_text")
.with(schema(
json!({
"doc": doc(),
"page": { "type": "integer", "minimum": 1 },
"image": { "type": "integer", "minimum": 1 },
"action": { "type": "string", "enum": ["move", "crop", "rotate", "rotate_by", "transform", "flip_horizontal", "flip_vertical", "replace", "delete", "bring_to_front", "send_to_back", "bring_forward", "send_backward", "remove_crop"] },
"action": { "type": "string", "enum": ["move", "crop", "rotate", "rotate_by", "transform", "flip_horizontal", "flip_vertical", "replace", "delete", "bring_to_front", "send_to_back", "bring_forward", "send_backward", "remove_crop", "ungroup"] },
"rect": { "type": "array", "items": { "type": "number" }, "minItems": 4, "maxItems": 4 },
"quarters": { "type": "integer" },
"degrees": { "type": "number" },
@@ -1589,9 +1599,12 @@ pub fn tools() -> Vec<ToolDef> {
t("text_font_import", "Load document font", "Load an explicitly selected, openly licensed full TrueType/OpenType face into a page. The program's family, embedding permissions and licence declaration are validated. Completes source subsets of the same face without replacing existing glyphs. Returns a font resource usable with text_edit.font_resource. Undoable; respects root confinement. No system-interface fallback is exported.")
.cmd("edit.edit_text")
.with(schema(json!({"doc":doc(), "page":{"type":"integer","minimum":1}, "path":{"type":"string"}}), &["doc","page","path"])),
t("text_fonts", "List document fonts", "List validated embedded source faces and approved full preview faces for a page. Full registered resources can be selected by text_edit.font_resource; preview_only faces are not PDF resource names.")
t("text_fonts", "List document fonts", "List validated embedded source faces and approved full preview faces for a page. Full registered resources and bundled choices can be selected by text_edit.font_resource. builtin:thai explicitly replaces a paragraph or range with the bundled Noto Sans Thai; a build without that face never lists it and text_edit refuses it there (a page whose fonts fill the listing budget may also omit it, but text_edit still accepts it). preview_only faces cannot be selected.")
.ro().cmd("edit.edit_text")
.with(schema(json!({"doc":doc(), "page":{"type":"integer","minimum":1}}), &["doc","page"])),
t("text_prepare", "Prepare grouped text", "Make existing text inside supported Form XObjects directly editable on one page. Replaces placements with their original operators and resources, preserving position, clipping and other pages. Refuses unsafe transparency, tags or malformed content. Adds no text or cover boxes. Returns prepared_groups (placements made editable), repaired_fonts (damaged legacy Thai maps repaired) and refused_groups ('name: reason' for each group left as it was). Also repairs the page's own damaged legacy Thai maps, grouped or not. Fails when the page has nothing to prepare or repair. Undoable; list text_paragraphs again afterwards.")
.cmd("edit.edit_text")
.with(schema(json!({"doc":doc(), "page":{"type":"integer","minimum":1}}), &["doc","page"])),
t("text_paragraphs", "List paragraphs", "The paragraphs on a page (adjacent source runs grouped into aligned visual rows, including mixed fonts, sizes and colours): number, text, editor_text (visual rows separated by newlines), rows and source line numbers, box (top-left-origin points), font and size. Use the number with text_edit's paragraph: until you list the page again, a number names the paragraph listed here, even after edits regroup the page.")
.ro()
.cmd("edit.edit_text")
@@ -1605,7 +1618,7 @@ pub fn tools() -> Vec<ToolDef> {
.with(schema(
json!({ "doc": doc(), "page": { "type": "integer", "minimum": 1 }, "line": { "type": "integer", "minimum": 1 },
"paragraph": { "type": "integer", "minimum": 1 },
"font_resource": { "type": "string", "description": "Explicit source/full page font resource from text_fonts or text_font_import." },
"font_resource": { "type": "string", "description": "Paragraphs: source/full page resource or bundled document-font choice from text_fonts (builtin:thai selects the bundled Thai face); also accepts text_font_import resources." },
"ranges": { "type": "array", "maxItems": 1024, "items": { "type": "object", "properties": {"range":{"type":"array","items":{"type":"integer","minimum":0},"minItems":2,"maxItems":2},"font":{"type":"string","enum":["helvetica","times","courier"]},"bold":{"type":"boolean"},"italic":{"type":"boolean"},"font_resource":{"type":"string"},"size":{"type":"number","exclusiveMinimum":0},"color":{"type":"string"},"underline":{"type":"boolean"},"char_spacing":{"type":"number"},"scale":{"type":"number"}},"required":["range"]}, "description":"Disjoint UTF-8 byte ranges in replacement text; unspecified source formatting survives." },
"text": { "type": "string" },
"font": { "type": "string", "enum": ["helvetica", "times", "courier"] },
crates/automation/tests/it/automation.rs
+409
View File
@@ -9445,6 +9445,50 @@ fn thai_text_through_tools_saves_reopens_extracts_and_renders() {
assert!(page_text(&mut a, reopened)[0].contains("แก้ไข น้ำ"));
}
/// Thai typed into Fill & Sign text, a text box, a form field, a footer and a watermark (#2354) is
/// drawn from PdfCraft's Thai face, as Add text draws it: no tool warns of `?`, and after a save
/// each one's text is found as typed and the page renders. A build without a Thai face refuses
/// the text in each, saying what to do, rather than writing `?????`.
#[test]
fn thai_in_fill_sign_text_comments_fields_and_page_marks_is_drawn_and_found() {
let dir = workdir("thai-everywhere");
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_open", json!({"path": "a.pdf"}))["doc"].as_u64().unwrap();
let thai = "บุญฤทธิ์ น้ำ ปั้น";
let field = ok(&mut a, "form_add_field", json!({"doc": doc, "page": 1, "type": "text", "rect": [20, 120, 220, 142]}))["field"]
.as_str()
.unwrap()
.to_string();
let calls = [
("fill_sign_add", json!({"doc": doc, "page": 1, "type": "text", "at": [20, 40], "text": thai, "font_size": 14})),
("comment_add", json!({"doc": doc, "page": 1, "type": "textbox", "rect": [20, 160, 200, 190], "contents": thai})),
("form_fill", json!({"doc": doc, "values": {field.as_str(): thai}})),
("doc_header_footer", json!({"doc": doc, "pages": [1], "footer_center": format!("{thai} <<1>>")})),
("doc_watermark", json!({"doc": doc, "pages": [1], "text": thai})),
];
if pdfcraft_fonts::document_thai_font().is_none() {
for (tool, args) in calls {
let err = a.call(tool, &args).unwrap_err().to_string();
assert!(err.contains("Thai") && err.contains("CRAFT_FONTS_DIR"), "{tool}: {err}");
}
assert_eq!(ok(&mut a, "comment_list", json!({"doc": doc}))["comments"].as_array().map(Vec::len), Some(0));
return;
}
for (tool, args) in calls {
let r = ok(&mut a, tool, args);
assert!(r.get("warnings").is_none(), "{tool}: {r}");
}
ok(&mut a, "doc_save", json!({"doc": doc, "path": "thai-marks.pdf"}));
let reopened = ok(&mut a, "doc_open", json!({"path": "thai-marks.pdf"}))["doc"].as_u64().unwrap();
let found = ok(&mut a, "text_find", json!({"doc": reopened, "query": "บุญฤทธิ์"}));
assert_eq!(found["count"], 5, "{found}");
let rendered = a.call("page_render", &json!({"doc": reopened, "page": 1, "dpi": 72})).unwrap();
assert!(rendered.iter().any(|c| matches!(c, Content::Png { data, .. } if !data.is_empty())));
// Flattened, every one of them is page text, as typed.
ok(&mut a, "doc_flatten", json!({"doc": reopened}));
assert_eq!(page_text(&mut a, reopened)[0].matches(thai).count(), 5, "{:?}", page_text(&mut a, reopened));
}
#[test]
fn hebrew_text_through_tools_saves_reopens_extracts_and_renders() {
let dir = workdir("hebrew-content");
@@ -9503,6 +9547,75 @@ fn mixed_objects_move_through_tools_with_generation_and_one_undo() {
assert!(page_text(&mut a, reopened)[0].contains("Added label"));
}
/// Edit a PDF's ⌘A, ⌘C / ⌘X and ⌘V headlessly: every object of a page copied, pasted into
/// another document and onto its own page (stepped down and right), and cut as one undo step.
#[test]
fn objects_copy_cut_and_paste_through_tools_within_and_across_documents() {
let dir = workdir("object-copy-paste");
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_open", json!({"path":"a.pdf"}))["doc"].as_u64().unwrap();
let other = ok(&mut a, "doc_open", json!({"path":"b.pdf"}))["doc"].as_u64().unwrap();
ok(&mut a, "page_add_text", json!({"doc":doc,"page":1,"text":"Added label","rect":[20,20,130,45]}));
assert!(matches!(a.call("object_paste", &json!({"doc":doc,"page":1})), Err(ToolError::Failed(_))), "nothing copied yet");
let listed = ok(&mut a, "object_list", json!({"doc":doc,"page":1}));
assert_eq!(listed["count"], 2);
let copied = ok(&mut a, "object_copy", json!({"doc":doc,"page":1,"all":true,"generation":listed["generation"]}));
assert_eq!(copied["copied"], 2);
let text = copied["text"].as_str().unwrap();
assert!(text.contains("Added label") && text.contains("Page 1"), "{text}");
assert_eq!(ok(&mut a, "object_list", json!({"doc":doc,"page":1})), listed, "a copy changes nothing");
// Into another document: the same objects, where they were, as its own page text.
let before = ok(&mut a, "object_list", json!({"doc":other,"page":2}));
assert_eq!(ok(&mut a, "object_paste", json!({"doc":other,"page":2}))["pasted"], 2);
let pasted = ok(&mut a, "object_list", json!({"doc":other,"page":2}));
assert_eq!(pasted["count"], 3);
let rect_of =
|list: &Value, text: &str| list["objects"].as_array().unwrap().iter().find(|o| o["text"] == text).map(|o| o["rect"].clone()).unwrap();
assert_eq!(rect_of(&pasted, "Page 1"), rect_of(&listed, "Page 1"));
// The added text box comes as the page's own text, drawn inside the box it had.
let (label, box_) = (rect_of(&pasted, "Added label"), rect_of(&listed, "Added label"));
assert!((0..2).all(|i| label[i].as_f64() >= box_[i].as_f64()) && (2..4).all(|i| label[i].as_f64() <= box_[i].as_f64()), "{label} in {box_}");
assert!(page_text(&mut a, other)[1].contains("Added label"));
ok(&mut a, "doc_save", json!({"doc":other,"path":"pasted.pdf"}));
let reopened = ok(&mut a, "doc_open", json!({"path":"pasted.pdf"}))["doc"].as_u64().unwrap();
assert!(page_text(&mut a, reopened)[1].contains("Page 1"));
ok(&mut a, "edit_undo", json!({"doc":other}));
assert_eq!(ok(&mut a, "object_list", json!({"doc":other,"page":2}))["objects"], before["objects"], "one undo step");
// Onto its own page: 18 points down and to the right.
ok(&mut a, "object_paste", json!({"doc":doc,"page":1}));
let both = ok(&mut a, "object_list", json!({"doc":doc,"page":1}));
assert_eq!(both["count"], 4);
let original = rect_of(&listed, "Page 1");
let moved =
both["objects"].as_array().unwrap().iter().filter(|o| o["text"] == "Page 1").map(|o| o["rect"].clone()).find(|r| *r != original).unwrap();
for i in 0..4 {
assert!((moved[i].as_f64().unwrap() - original[i].as_f64().unwrap() - 18.0).abs() < 0.02, "{moved} {original}");
}
// Cut: copied, then deleted as one undo step.
let cut = ok(&mut a, "object_copy", json!({"doc":doc,"page":1,"objects":[{"kind":"text","index":1}],"cut":true}));
assert_eq!(cut["copied"], 1);
assert_eq!(ok(&mut a, "object_list", json!({"doc":doc,"page":1}))["count"], 3);
ok(&mut a, "edit_undo", json!({"doc":doc}));
assert_eq!(ok(&mut a, "object_list", json!({"doc":doc,"page":1}))["objects"], both["objects"]);
for args in [
json!({"doc":doc,"page":1,"all":true,"objects":[{"kind":"text","index":1}]}),
json!({"doc":doc,"page":1}),
json!({"doc":doc,"page":1,"objects":[{"kind":"text","index":99}]}),
json!({"doc":doc,"page":9,"all":true}),
] {
assert!(a.call("object_copy", &args).is_err(), "{args}");
}
assert!(
matches!(a.call("object_copy", &json!({"doc":doc,"page":1,"all":true,"generation":listed["generation"]})), Err(ToolError::Failed(_))),
"stale"
);
assert!(a.call("object_paste", &json!({"doc":doc,"page":9})).is_err());
}
#[test]
fn object_move_rejects_invalid_nested_references_without_any_change() {
let dir = workdir("object-move-refusals");
@@ -11199,6 +11312,154 @@ fn simple_rule_scan_drawn(content: &[u8]) -> Vec<u8> {
write_full(&doc, &SaveOptions::default()).unwrap()
}
/// #3005: a table and a small signature drawn inside one transparency-group Form XObject (how
/// HTML-to-PDF converters write a block with a translucent picture). Edit a PDF sees one piece
/// of artwork and no text; `image_edit ungroup` draws it in place with the same pixels, its
/// text becomes paragraphs that `text_edit` changes, and undo brings the form back.
fn table_in_a_form_pdf() -> Vec<u8> {
let table = b"BT /F1 12 Tf 1 0 0 1 20 120 Tm (Revenue) Tj 1 0 0 1 120 120 Tm (153,300.00) Tj 1 0 0 1 20 100 Tm (Costs) Tj 1 0 0 1 120 100 Tm (382,110.00) Tj ET 0.6 G 0.5 w 15 115 m 220 115 l S";
// Two pictures with soft masks overlapping each other, the second also over the table's
// last cell: a translucent wash, and a picture whose corner pixels are see-through.
let inner = [&table[..], b" q 40 0 0 20 30 40 cm /Sig Do Q q 60 0 0 40 50 30 cm /Wash Do Q q 100 0 0 30 110 90 cm /Sig Do Q"].concat();
let rgb: Vec<u8> = std::iter::repeat_n([20u8, 60, 220], 16).flatten().collect();
let alpha: Vec<u8> = (0..16).map(|i| if i % 3 == 0 { 255 } else { 0 }).collect();
let page = b"q /Fm1 Do Q";
let stream = |dict: &str, data: &[u8]| [format!("{dict} /Length {} >>\nstream\n", data.len()).as_bytes(), data, b"\nendstream"].concat();
let objs: Vec<Vec<u8>> = vec![
b"<< /Type /Catalog /Pages 2 0 R >>".to_vec(),
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec(),
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 240 160] /Contents 4 0 R /Resources << /XObject << /Fm1 5 0 R >> >> >>".to_vec(),
stream("<<", page),
stream(
"<< /Type /XObject /Subtype /Form /BBox [0 0 240 160] /Group << /S /Transparency /CS /DeviceRGB >> /Resources << /Font << /F1 6 0 R >> /XObject << /Sig 7 0 R /Wash 9 0 R >> >>",
&inner,
),
b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".to_vec(),
stream("<< /Type /XObject /Subtype /Image /Width 4 /Height 4 /ColorSpace /DeviceRGB /BitsPerComponent 8 /SMask 8 0 R", &rgb),
stream("<< /Type /XObject /Subtype /Image /Width 4 /Height 4 /ColorSpace /DeviceGray /BitsPerComponent 8", &alpha),
stream(
"<< /Type /XObject /Subtype /Image /Width 2 /Height 2 /ColorSpace /DeviceRGB /BitsPerComponent 8 /SMask 10 0 R",
&[230, 40, 40].repeat(4),
),
stream("<< /Type /XObject /Subtype /Image /Width 2 /Height 2 /ColorSpace /DeviceGray /BitsPerComponent 8", &[128, 96, 160, 64]),
];
let mut out = b"%PDF-1.7\n".to_vec();
let mut offsets = Vec::new();
for (i, o) in objs.iter().enumerate() {
offsets.push(out.len());
out.extend_from_slice(format!("{} 0 obj\n", i + 1).as_bytes());
out.extend_from_slice(o);
out.extend_from_slice(b"\nendobj\n");
}
let xref = out.len();
out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes());
for o in offsets {
out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes());
}
out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes());
out
}
#[test]
fn ungrouping_a_table_drawn_in_a_form_makes_its_text_editable() {
let dir = workdir("ungroup");
std::fs::write(dir.join("table.pdf"), table_in_a_form_pdf()).unwrap();
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_open", json!({ "path": "table.pdf" }))["doc"].as_u64().unwrap();
let paragraphs = |a: &mut Automation| ok(a, "text_paragraphs", json!({ "doc": doc, "page": 1 }))["paragraphs"].as_array().unwrap().len();
let objects = |a: &mut Automation| ok(a, "object_list", json!({ "doc": doc, "page": 1 }))["objects"].as_array().unwrap().len();
assert_eq!((paragraphs(&mut a), objects(&mut a)), (0, 1), "one piece of artwork, no text");
let (w, h, before) = a.render_pixels(doc, 1, 2.0).unwrap();
ok(&mut a, "image_edit", json!({ "doc": doc, "page": 1, "image": 1, "action": "ungroup" }));
let (w2, h2, after) = a.render_pixels(doc, 1, 2.0).unwrap();
assert_eq!((w, h), (w2, h2));
let differing = before.as_ref().as_chunks::<4>().0.iter().zip(after.as_ref().as_chunks::<4>().0).filter(|(p, q)| p != q).count();
assert_eq!(differing, 0, "ungrouping draws the same pixels");
assert!(paragraphs(&mut a) >= 4, "the table's text is page text now");
let images = ok(&mut a, "page_images", json!({ "doc": doc, "page": 1 }));
let kinds: Vec<_> = images["images"].as_array().unwrap().iter().map(|i| i["kind"].clone()).collect();
assert_eq!(kinds, vec![json!("image"); 3], "the pictures are separate pictures now: {images}");
let list = ok(&mut a, "text_paragraphs", json!({ "doc": doc, "page": 1 }));
let target =
list["paragraphs"].as_array().unwrap().iter().find(|p| p["text"].as_str().is_some_and(|t| t.contains("382,110.00"))).unwrap()["paragraph"]
.clone();
ok(&mut a, "text_edit", json!({ "doc": doc, "page": 1, "paragraph": target, "text": "400,000.00" }));
let text = ok(&mut a, "text_extract", json!({ "doc": doc, "pages": [1] }));
let s = text.to_string();
assert!(s.contains("400,000.00") && !s.contains("382,110.00"), "{s}");
ok(&mut a, "edit_undo", json!({ "doc": doc }));
ok(&mut a, "edit_undo", json!({ "doc": doc }));
assert_eq!((paragraphs(&mut a), objects(&mut a)), (0, 1), "undo brings the form back");
}
/// One 300 × 300 page that draws form `Fm1` with `page_do`; the form (with `matrix`) fills with
/// pattern `/P1` (`pattern`) in `content`, and a red outline on the page marks where it lands.
fn pattern_in_a_form_pdf(page_do: &str, matrix: &str, content: &str, pattern: &str) -> Vec<u8> {
let objs = [
"<< /Type /Catalog /Pages 2 0 R >>".to_string(),
"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_string(),
"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /XObject << /Fm1 5 0 R >> >> >>".to_string(),
stream_object("<<", &format!("{page_do} 1 0 0 RG 1 w 60 60 180 180 re S")),
stream_object(&format!("<< /Type /XObject /Subtype /Form /BBox [0 0 300 300] {matrix} /Resources << /Pattern << /P1 6 0 R >> >>"), content),
pattern.to_string(),
];
let mut out = b"%PDF-1.7\n".to_vec();
let mut offsets = Vec::new();
for (i, o) in objs.iter().enumerate() {
offsets.push(out.len());
out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes());
}
let xref = out.len();
out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes());
for o in offsets {
out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes());
}
out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes());
out
}
/// #3067 review: a pattern's matrix maps pattern space to the default space of the stream that
/// uses it (ISO 32000-2 §8.7.2), so a pattern filled inside a form moved by its `/Matrix` or by a
/// `cm` before the `Do` must stay where it was when the form is drawn inline. Each case draws
/// differently in Acrobat when the placement is lost (tiles out of phase or half size, the
/// shading outside its box); ungrouped, every one paints the same pixels.
#[test]
fn ungrouping_keeps_patterns_where_the_forms_placement_put_them() {
let tiling = stream_object(
"<< /Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 20 20] /XStep 20 /YStep 20 /Resources << >>",
"0 0 0 rg 0 0 10 10 re f 10 10 10 10 re f",
);
let shading = "<< /Type /Pattern /PatternType 2 /Shading << /ShadingType 2 /ColorSpace /DeviceRGB /Coords [0 0 100 0] /Function << /FunctionType 2 /Domain [0 1] /C0 [1 0 0] /C1 [0 0 1] /N 1 >> /Extend [false false] >> >>";
let cases = [
("no placement", "q /Fm1 Do Q", "", "/Pattern cs /P1 scn 60 60 180 180 re f", tiling.as_str()),
("form Matrix, half a tile", "q /Fm1 Do Q", "/Matrix [1 0 0 1 10 0]", "/Pattern cs /P1 scn 50 60 180 180 re f", tiling.as_str()),
("cm doubling the form", "q 2 0 0 2 0 0 cm /Fm1 Do Q", "", "/Pattern cs /P1 scn 30 30 90 90 re f", tiling.as_str()),
("shading moved by the form Matrix", "q /Fm1 Do Q", "/Matrix [1 0 0 1 100 100]", "/Pattern cs /P1 scn 0 0 100 100 re f", shading),
];
for (what, page_do, matrix, content, pattern) in cases {
let dir = workdir("ungroup_patterns");
std::fs::write(dir.join("p.pdf"), pattern_in_a_form_pdf(page_do, matrix, content, pattern)).unwrap();
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_open", json!({ "path": "p.pdf" }))["doc"].as_u64().unwrap();
let (w, h, before) = a.render_pixels(doc, 1, 1.0).unwrap();
ok(&mut a, "image_edit", json!({ "doc": doc, "page": 1, "image": 1, "action": "ungroup" }));
ok(&mut a, "doc_save", json!({ "doc": doc, "path": "out.pdf" }));
let saved = ok(&mut a, "doc_open", json!({ "path": "out.pdf" }))["doc"].as_u64().unwrap();
let (w2, h2, after) = a.render_pixels(saved, 1, 1.0).unwrap();
assert_eq!((w, h), (w2, h2), "{what}");
let differing = before.as_ref().as_chunks::<4>().0.iter().zip(after.as_ref().as_chunks::<4>().0).filter(|(p, q)| p != q).count();
assert_eq!(differing, 0, "{what}: ungrouping draws the same pixels, saved and reopened");
}
}
/// An indirect stream object: `dict` (without its closing `>>`) and `data`.
fn stream_object(dict: &str, data: &str) -> String {
format!("{dict} /Length {} >>\nstream\n{data}\nendstream", data.len())
}
/// Reviewing headlessly: corrections keep the boxes, a stale or foreign review is refused, and
/// applying consumes the handle (the pages already accepted were applied once).
#[test]
@@ -11745,3 +12006,151 @@ fn comments_copy_and_paste_between_documents_through_tools() {
assert!(matches!(a.call("comments_copy", &json!({ "doc": src, "comments": [[2]] })), Err(ToolError::InvalidArgs(_))));
assert_eq!(ok(&mut a, "comment_list", json!({ "doc": src }))["comments"].as_array().unwrap().len(), 2, "the source is untouched");
}
#[test]
fn explicit_bundled_thai_font_choice_layout_edit_undo_and_reopen() {
let temp = tempfile::tempdir().unwrap();
let dir = temp.path();
std::fs::write(dir.join("source.pdf"), fixture(1)).unwrap();
let mut a = auto(dir);
let doc = ok(&mut a, "doc_open", json!({"path":"source.pdf"}))["doc"].as_u64().unwrap();
let before = page_text(&mut a, doc);
let fonts = ok(&mut a, "text_fonts", json!({"doc":doc,"page":1}));
let choice = fonts["fonts"].as_array().unwrap().iter().find(|f| f["resource"] == "builtin:thai");
let args = json!({"doc":doc,"page":1,"paragraph":1,"text":"น้ำ กิ่ กี้ ผู้ใช้","font_resource":"builtin:thai"});
if pdfcraft_fonts::document_thai_font().is_none() {
assert!(choice.is_none());
assert!(a.call("text_edit", &args).is_err());
assert_eq!(page_text(&mut a, doc), before);
return;
}
assert_eq!(choice.unwrap()["preview_only"], false);
ok(&mut a, "text_edit_layout", args.clone());
assert_eq!(page_text(&mut a, doc), before);
ok(&mut a, "text_edit", args);
let edited = page_text(&mut a, doc);
assert!(edited[0].contains("น้ำ กิ่ กี้ ผู้ใช้"));
ok(&mut a, "edit_undo", json!({"doc":doc}));
assert_eq!(page_text(&mut a, doc), before);
ok(&mut a, "edit_redo", json!({"doc":doc}));
ok(&mut a, "doc_save", json!({"doc":doc,"path":"edited.pdf"}));
let reopened = ok(&mut a, "doc_open", json!({"path":"edited.pdf"}))["doc"].as_u64().unwrap();
assert_eq!(page_text(&mut a, reopened), edited);
let Err(ToolError::Failed(m)) =
a.call("text_edit", &json!({"doc":reopened,"page":1,"paragraph":1,"text":"น้ำ 😀","font_resource":"builtin:thai"}))
else {
panic!("Noto Sans Thai has no emoji")
};
assert!(m.contains("Noto Sans Thai") && m.contains("U+1F600"), "the error names the font and the character: {m}");
assert_eq!(page_text(&mut a, reopened), edited);
let image = a.call("page_render", &json!({"doc":reopened,"page":1,"dpi":144})).unwrap();
assert!(image.iter().any(|c| matches!(c, Content::Png { .. })));
}
/// `text_prepare` refuses what it can't act on, changing nothing: a page with no grouped text
/// (with the reason), and a page the document doesn't have.
#[test]
fn text_prepare_refuses_pages_without_grouped_text() {
let temp = tempfile::tempdir().unwrap();
let dir = temp.path();
std::fs::write(dir.join("source.pdf"), fixture(1)).unwrap();
let mut a = auto(dir);
let doc = ok(&mut a, "doc_open", json!({"path":"source.pdf"}))["doc"].as_u64().unwrap();
let Err(ToolError::Failed(m)) = a.call("text_prepare", &json!({"doc":doc,"page":1})) else { panic!("nothing to prepare") };
assert!(m.contains("no supported grouped text"), "{m}");
let Err(ToolError::InvalidArgs(m)) = a.call("text_prepare", &json!({"doc":doc,"page":2})) else { panic!("no page 2") };
assert!(m.contains("out of range"), "{m}");
assert!(a.call("edit_undo", &json!({"doc":doc})).is_err(), "nothing was recorded to undo");
assert_eq!(ok(&mut a, "doc_info", json!({"doc":doc}))["document"]["dirty"], false);
}
#[test]
fn grouped_existing_thai_body_edits_saves_reopens_and_undo_restores_original_glyphs() {
use pdfcraft_cos::{Dict, Document, ObjRef, Object, SaveOptions, Stream, write_full};
use std::sync::Arc;
let temp = tempfile::tempdir().unwrap();
let dir = temp.path();
let mut source = Document::open(Arc::new(fixture(2))).unwrap();
let old = "ข้อความเดิม น้ำ ผู้ใช้";
let new = "ข้อความแก้ไข น้ำ ผู้ใช้";
let mut cmap = String::from("begincmap 1 begincodespacerange <00> <ff> endcodespacerange\n");
cmap.push_str(&format!("{} beginbfchar\n", old.chars().count()));
for (i, ch) in old.chars().enumerate() {
cmap.push_str(&format!("<{:02x}> <{:04x}>\n", 65 + i, u32::from(ch)));
}
cmap.push_str("endbfchar endcmap");
let unicode = source.add(Object::Stream(Stream::flate(Dict::new(), cmap.as_bytes())));
let mut font = Dict::new();
font.set(b"Type".to_vec(), Object::name("Font"));
font.set(b"Subtype".to_vec(), Object::name("Type1"));
font.set(b"BaseFont".to_vec(), Object::name("ABCDEF+LegacyThai"));
font.set(b"FirstChar".to_vec(), Object::Int(65));
font.set(b"LastChar".to_vec(), Object::Int(65 + old.chars().count() as i64 - 1));
font.set(b"Widths".to_vec(), Object::Array(vec![Object::Int(300); old.chars().count()]));
font.set(b"ToUnicode".to_vec(), Object::Ref(unicode));
let font = source.add(Object::Dict(font));
let mut fonts = Dict::new();
fonts.set(b"F1".to_vec(), Object::Ref(font));
let mut resources = Dict::new();
resources.set(b"Font".to_vec(), Object::Dict(fonts));
let mut form = Dict::new();
form.set(b"Type".to_vec(), Object::name("XObject"));
form.set(b"Subtype".to_vec(), Object::name("Form"));
form.set(b"BBox".to_vec(), Object::Array(vec![Object::Int(0), Object::Int(0), Object::Int(200), Object::Int(300)]));
form.set(b"Resources".to_vec(), Object::Dict(resources));
let codes: String = (0..old.chars().count()).map(|i| format!("{:02x}", 65 + i)).collect();
let form = source.add(Object::Stream(Stream::flate(form, format!("BT /F1 14 Tf 20 220 Td <{codes}> Tj ET").as_bytes())));
for page in [ObjRef::new(4, 0), ObjRef::new(6, 0)] {
let mut xobjects = Dict::new();
xobjects.set(b"Body".to_vec(), Object::Ref(form));
let mut resources = Dict::new();
resources.set(b"XObject".to_vec(), Object::Dict(xobjects));
let content = source.add(Object::Stream(Stream::flate(Dict::new(), b"q /Body Do Q")));
source
.update_dict(page, |d| {
d.set(b"Resources".to_vec(), Object::Dict(resources));
d.set(b"Contents".to_vec(), Object::Ref(content));
})
.unwrap();
}
std::fs::write(dir.join("source.pdf"), write_full(&source, &SaveOptions::default()).unwrap()).unwrap();
let mut a = auto(dir);
let id = ok(&mut a, "doc_open", json!({"path":"source.pdf"}))["doc"].as_u64().unwrap();
let before = a.call("page_render", &json!({"doc":id,"page":1,"dpi":72})).unwrap();
assert_eq!(ok(&mut a, "text_paragraphs", json!({"doc":id,"page":1}))["count"], 0);
let prepared = ok(&mut a, "text_prepare", json!({"doc":id,"page":1}));
assert_eq!(
[&prepared["prepared_groups"], &prepared["repaired_fonts"], &prepared["refused_groups"]],
[&json!(1), &json!(0), &json!([])],
"{prepared}"
);
assert_eq!(a.call("page_render", &json!({"doc":id,"page":1,"dpi":72})).unwrap(), before, "preparing must not change pixels");
assert_eq!(ok(&mut a, "text_paragraphs", json!({"doc":id,"page":1}))["paragraphs"][0]["text"], old);
// The bundled Thai face is an optional build input. Without it, `builtin:thai` is refused and
// the grouped edit, undo, redo and save checks run on Latin text in Helvetica instead.
let thai = json!({"doc":id,"page":1,"paragraph":1,"text":new,"font_resource":"builtin:thai"});
let (new, args) = if pdfcraft_fonts::document_thai_font().is_some() {
(new, thai)
} else {
let Err(ToolError::Failed(m)) = a.call("text_edit", &thai) else { panic!("builtin:thai needs the bundled Thai face") };
assert!(m.contains("builtin:thai"), "{m}");
let latin = "Edited body";
(latin, json!({"doc":id,"page":1,"paragraph":1,"text":latin,"font":"helvetica"}))
};
ok(&mut a, "text_edit", args);
assert_eq!(ok(&mut a, "text_lines", json!({"doc":id,"page":1}))["lines"][0]["text"].as_str().unwrap().trim(), new);
assert_eq!(page_text(&mut a, id)[1], old, "a shared form's other page must not change");
let after = a.call("page_render", &json!({"doc":id,"page":1,"dpi":72})).unwrap();
assert_ne!(after, before);
ok(&mut a, "edit_undo", json!({"doc":id}));
assert_eq!(a.call("page_render", &json!({"doc":id,"page":1,"dpi":72})).unwrap(), before);
ok(&mut a, "edit_undo", json!({"doc":id}));
assert_eq!(ok(&mut a, "text_paragraphs", json!({"doc":id,"page":1}))["count"], 0);
ok(&mut a, "edit_redo", json!({"doc":id}));
ok(&mut a, "edit_redo", json!({"doc":id}));
ok(&mut a, "doc_save", json!({"doc":id,"path":"edited.pdf"}));
let reopened = ok(&mut a, "doc_open", json!({"path":"edited.pdf"}))["doc"].as_u64().unwrap();
assert_eq!(a.call("page_render", &json!({"doc":reopened,"page":1,"dpi":72})).unwrap(), after);
assert_eq!(page_text(&mut a, reopened), [new, old]);
assert_eq!(ok(&mut a, "content_list", json!({"doc":reopened}))["count"], 0, "editing existing text must not create Add Text objects");
}
crates/cos/src/object.rs
+7 -1
View File
@@ -342,8 +342,14 @@ impl Stream {
/// Decoded data, bounded by [`MAX_DECODED`]; any corruption is an error.
pub fn decoded_strict(&self) -> Result<Vec<u8>, CosError> {
self.decoded_strict_within(MAX_DECODED)
}
/// Like [`Stream::decoded_strict`], bounded by `max` (and [`MAX_DECODED`]): for data that
/// will be copied elsewhere, where a damaged stream must not pass for a shorter one.
pub fn decoded_strict_within(&self, max: usize) -> Result<Vec<u8>, CosError> {
let decode = |chain: &[(Filter, Params)], data: &[u8], max| pdfcraft_filters::decode(chain, data, max).map(|v| (v, 0)).map_err(|e| (e, 0));
self.decode_chain(MAX_DECODED, decode).map(|(v, _)| v).map_err(|(e, _)| CosError::Filter(e.to_string()))
self.decode_chain(max, decode).map(|(v, _)| v).map_err(|(e, _)| CosError::Filter(e.to_string()))
}
/// [`Stream::decoded_strict`] bounded by `max` and counted like
crates/edit/Cargo.toml
+3
View File
@@ -21,6 +21,9 @@ kurbo = { workspace = true }
# Display order of right-to-left lines in added text, and reading and writing order in Edit text;
# already in the tree via the interface.
unicode-bidi = "0.3"
# Thai word boundaries for rewrapping and Add text (dictionary data compiled in, no LSTM model);
# the ICU4X 2.3 crates it builds on are already in the tree.
icu_segmenter = { version = "2.3", default-features = false, features = ["compiled_data"] }
[dev-dependencies]
hayro-syntax = { workspace = true }
crates/edit/src/added.rs
+8 -5
View File
@@ -8,12 +8,8 @@
//! after `/UserUnit` and `/Rotate`), so items stay upright on rotated pages; an item records the
//! `/UserUnit` it was written for.
use std::collections::{HashMap, HashSet};
use pdfcraft_cos::{Dict, Document, ObjRef, Object, PdfString, Stream};
use pdfcraft_fonts::{
GlyphError, GlyphOutline, ShapedCluster, ShapedScript, Std14, literal, shape_arabic, shape_hebrew, shape_thai, shaped_glyph, win_ansi,
};
use pdfcraft_fonts::{GlyphError, GlyphOutline, ShapedScript, Std14, literal, shape_arabic, shape_hebrew, shape_thai, shaped_glyph, win_ansi};
use unicode_bidi::{Level, ParagraphBidiInfo};
use crate::{EditError, MarkKind, check, contents, matrix_number, n, page_list, place_tagged};
@@ -23,13 +19,18 @@ mod layout;
mod store;
mod types;
pub(crate) use draw::thai_glyph_resources;
#[cfg(test)]
use draw::{GLYPHS_PER_FONT, LaidLine, draw_laid_shaped};
#[cfg(test)]
use layout::Piece;
#[cfg(test)]
pub(crate) use layout::WRAP_CALLS;
#[cfg(test)]
pub(crate) use layout::shaped_width;
pub use layout::{PASTE_STEP, first_undrawable, lines, paste_place, paste_rect, text_rect, text_width};
#[cfg(test)]
use pdfcraft_fonts::ShapedCluster;
pub use store::{add_content, delete_content, isolate_content, list_added, scan_markers, update_content};
pub use types::{Added, AddedImage, AddedText, Align, CHAR_SPACING_LIMIT, Family, Ink, MarkerScan};
@@ -87,3 +88,5 @@ fn align_from(code: Option<i64>) -> Align {
#[cfg(test)]
mod arabic_tests;
#[cfg(test)]
mod thai_tests;
crates/edit/src/added/draw.rs
+74 -165
View File
@@ -3,17 +3,17 @@ use super::layout::{
};
use super::types::{AddedText, Align, Family, Ink};
use super::{
Content, Dict, Document, EditError, GlyphError, GlyphOutline, HashMap, HashSet, ObjRef, Object, PdfString, ShapedCluster, ShapedScript, Stream,
align_code, literal, matrix_number, n, shaped_glyph, win_ansi,
Content, Dict, Document, EditError, GlyphError, GlyphOutline, Object, PdfString, ShapedScript, align_code, literal, matrix_number, n,
shaped_glyph, win_ansi,
};
use pdfcraft_fonts::shaped_text::{self, ShapedFonts};
fn font_name(base: &str) -> String {
format!("PCF{}", base.replace('-', ""))
}
/// Most glyphs in one Type 3 font: its codes are single bytes, 1–240. An item gets one such font
/// per 240 distinct glyphs it needs.
pub(super) const GLYPHS_PER_FONT: usize = 240;
#[cfg(test)]
pub(super) use pdfcraft_fonts::shaped_text::GLYPHS_PER_FONT;
/// A line of a shaped-text item after layout: its text, whether its paragraph runs right to left,
/// its paragraph, its pieces in drawing order and its width at the item's size.
@@ -61,50 +61,15 @@ pub(super) fn draw_laid_shaped(
) -> Result<(), EditError> {
let r = text_rect(t);
let latin = font_name(t.family.base_font(t.bold, t.italic));
// Every distinct cluster (a letter with its dots and the text it stands for) is one glyph of
// one of the fonts, so copy and search get each character exactly once.
let mut keys: Vec<&ShapedCluster> = Vec::new();
let mut index: HashMap<ClusterKey, usize> = HashMap::new();
for c in laid.iter().flat_map(|(_, _, _, pieces, _)| pieces).flat_map(|p| match p {
Piece::Shaped(clusters) => clusters.as_slice(),
Piece::Latin(_) => &[],
}) {
index.entry(cluster_key(c)).or_insert_with(|| {
keys.push(c);
keys.len() - 1
});
}
// Every outline is read before anything is written, so a failure leaves the document as it was.
let mut outlines = HashMap::new();
for c in &keys {
for (id, _) in &c.glyphs {
if let std::collections::hash_map::Entry::Vacant(entry) = outlines.entry((c.script, *id)) {
entry.insert(outline(c.script, *id).map_err(|e| match c.script {
ShapedScript::Arabic => arabic_error(e, &c.text),
ShapedScript::Hebrew => hebrew_error(e, &c.text),
ShapedScript::Thai => thai_error(e, &c.text),
})?);
}
}
}
// The names the page and this item already use: a new font never takes one of them.
let mut taken_names: HashSet<Vec<u8>> = taken.iter().chain(fonts.iter()).map(|(name, _)| name.clone()).collect();
let mut names = Vec::new();
for chunk in keys.chunks(GLYPHS_PER_FONT) {
let font = type3_shaped(doc, chunk, &outlines);
// A name no font on the page has, so items never replace each other's fonts (object
// numbers alone don't do: a full save renumbers objects but keeps resource names).
let mut name = format!("PCAr{}", font.num);
let mut suffix = 0usize;
while taken_names.contains(name.as_bytes()) {
suffix = suffix.saturating_add(1);
name = format!("PCAr{}_{suffix}", font.num);
}
taken_names.insert(name.clone().into_bytes());
fonts.set(name.clone().into_bytes(), Object::Ref(font));
names.push(name);
}
let shaped =
ShapedFonts::new(shaped_text::clusters(laid.iter().map(|(_, _, _, pieces, _)| pieces.as_slice())), outline, taken, fonts, "PCAr", &mut |o| {
Object::Ref(doc.add(o))
})
.map_err(|(e, c)| match c.script {
ShapedScript::Arabic => arabic_error(e, &c.text),
ShapedScript::Hebrew => hebrew_error(e, &c.text),
ShapedScript::Thai => thai_error(e, &c.text),
})?;
out.extend(format!("BT {}\n", t.fill_op()).bytes());
if t.char_spacing != 0.0 {
out.extend(format!("{} Tc\n", n(t.char_spacing)).bytes());
@@ -119,32 +84,17 @@ pub(super) fn draw_laid_shaped(
let (x, tw) = placed(t, r, &paragraphs, i, *w, spaces, *rtl);
let y = r[3] - (i as f64 * 1.2 + 0.95) * t.size;
let mut pen = x;
let mut current: Option<&str> = None;
let mut current = None;
for piece in pieces {
match piece {
Piece::Latin(s) => {
out.extend(format!("/{latin} {} Tf {} Tw 1 0 0 1 {} {} Tm ", n(t.size), n(tw), n(pen), n(y)).bytes());
out.extend(literal(&win_ansi(s)));
out.extend_from_slice(b" Tj\n");
current = Some(latin.as_str());
current = None;
pen += latin_width(t, s) + tw * s.matches(' ').count() as f64;
}
Piece::Shaped(clusters) => {
for c in clusters {
let Some(&k) = index.get(&cluster_key(c)) else { continue };
let Some(name) = names.get(k / GLYPHS_PER_FONT) else { continue };
// Codes start at 1; k % 240 + 1 is at most 240.
let code = u8::try_from(k % GLYPHS_PER_FONT + 1).unwrap_or(1);
if current != Some(name.as_str()) {
out.extend(format!("/{name} {} Tf ", n(t.size)).bytes());
current = Some(name.as_str());
}
out.extend(format!("1 0 0 1 {} {} Tm ", n(pen), n(y)).bytes());
out.extend(literal(&[code]));
out.extend_from_slice(b" Tj\n");
pen += c.advance * t.size + t.char_spacing + if c.text == " " { tw } else { 0.0 };
}
}
Piece::Shaped(clusters) => pen = shaped.show(out, clusters, pen, y, t.size, t.char_spacing, tw, &mut current),
}
}
out.extend_from_slice(b"EMC\n");
@@ -153,104 +103,6 @@ pub(super) fn draw_laid_shaped(
Ok(())
}
/// What makes two clusters the same Type 3 glyph: text, glyphs, offsets and advance (bit for bit;
/// equal clusters come from the same font units).
type ClusterKey<'a> = (ShapedScript, &'a str, Vec<(u32, u64, u64)>, u64);
fn cluster_key(c: &ShapedCluster) -> ClusterKey<'_> {
(c.script, c.text.as_str(), c.glyphs.iter().map(|(id, [x, y])| (*id, x.to_bits(), y.to_bits())).collect(), c.advance.to_bits())
}
/// A Type 3 font with `clusters` as codes 1…, each drawn from its script face's glyph `outlines`.
fn type3_shaped(doc: &mut Document, clusters: &[&ShapedCluster], outlines: &HashMap<(ShapedScript, u32), GlyphOutline>) -> ObjRef {
let mut charprocs = Dict::new();
let mut differences = vec![Object::Int(1)];
let mut widths = Vec::with_capacity(clusters.len());
let mut bbox = [0.0f64; 4];
let mut to_unicode = Vec::new();
for (i, c) in clusters.iter().enumerate() {
let code = i + 1;
// Glyph space is 1000 units per em.
let mut paths = Vec::new();
let mut ink: Option<[f64; 4]> = None;
for (id, [dx, dy]) in &c.glyphs {
let Some(g) = outlines.get(&(c.script, *id)) else { continue };
for contour in &g.contours {
for (j, p) in contour.iter().enumerate() {
paths.extend(format!("{} {} {}\n", n((p[0] + dx) * 1000.0), n((p[1] + dy) * 1000.0), if j == 0 { "m" } else { "l" }).bytes());
}
paths.extend_from_slice(b"h\n");
let b = [g.bbox[0] + dx, g.bbox[1] + dy, g.bbox[2] + dx, g.bbox[3] + dy];
ink = Some(ink.map_or(b, |a| [a[0].min(b[0]), a[1].min(b[1]), a[2].max(b[2]), a[3].max(b[3])]));
}
}
// Rounded outwards so no ink is clipped.
let [x0, y0, x1, y1] =
ink.map_or([0.0; 4], |b| [(b[0] * 1000.0).floor(), (b[1] * 1000.0).floor(), (b[2] * 1000.0).ceil(), (b[3] * 1000.0).ceil()]);
bbox = [bbox[0].min(x0), bbox[1].min(y0), bbox[2].max(x1), bbox[3].max(y1)];
let mut proc = format!("{} 0 {} {} {} {} d1\n", n(c.advance * 1000.0), n(x0), n(y0), n(x1), n(y1)).into_bytes();
if !paths.is_empty() {
proc.extend(paths);
proc.extend_from_slice(b"f\n");
}
let glyph = format!("g{code:02X}");
charprocs.set(glyph.clone().into_bytes(), Object::Ref(doc.add(Object::Stream(Stream::flate(Dict::new(), &proc)))));
differences.push(Object::name(&glyph));
widths.push(Object::Real((c.advance * 1000.0).round()));
if !c.text.is_empty() {
let hex: String = c.text.encode_utf16().map(|u| format!("{u:04X}")).collect();
to_unicode.push(format!("<{code:02X}> <{hex}>\n"));
}
}
let mut cmap = String::from(
"/CIDInit /ProcSet findresource begin\n12 dict begin\nbegincmap\n/CMapType 2 def\n1 begincodespacerange\n<01> <FF>\nendcodespacerange\n",
);
// A CMap block holds at most 100 entries.
for block in to_unicode.chunks(100) {
cmap.push_str(&format!("{} beginbfchar\n{}endbfchar\n", block.len(), block.concat()));
}
cmap.push_str("endcmap\nCMapName currentdict /CMap defineresource pop\nend\nend\n");
let cmap = doc.add(Object::Stream(Stream::flate(Dict::new(), cmap.as_bytes())));
// The face's family when every glyph comes from one script; without the face, the script's
// name (as Arabic items were always named).
let family = match clusters.first().map(|c| c.script).filter(|s| clusters.iter().all(|c| c.script == *s)) {
Some(script) => script.font().map_or(
match script {
ShapedScript::Arabic => "Arabic",
ShapedScript::Hebrew => "Hebrew",
ShapedScript::Thai => "Thai",
},
|f| f.family,
),
None => "PdfCraft Shaped Text",
};
let mut descriptor = Dict::new();
descriptor.set(b"Type".to_vec(), Object::name("FontDescriptor"));
descriptor.set(b"FontName".to_vec(), Object::name(&family.replace(' ', "")));
descriptor.set(b"FontFamily".to_vec(), Object::String(PdfString::literal(family.as_bytes().to_vec())));
descriptor.set(b"Flags".to_vec(), Object::Int(4));
descriptor.set(b"ItalicAngle".to_vec(), Object::Int(0));
let mut encoding = Dict::new();
encoding.set(b"Type".to_vec(), Object::name("Encoding"));
encoding.set(b"Differences".to_vec(), Object::Array(differences));
let mut font = Dict::new();
font.set(b"Type".to_vec(), Object::name("Font"));
font.set(b"Subtype".to_vec(), Object::name("Type3"));
font.set(b"FontDescriptor".to_vec(), Object::Ref(doc.add(Object::Dict(descriptor))));
font.set(b"FontBBox".to_vec(), Object::Array(bbox.iter().map(|v| Object::Real(*v)).collect()));
font.set(
b"FontMatrix".to_vec(),
Object::Array(vec![Object::Real(0.001), Object::Int(0), Object::Int(0), Object::Real(0.001), Object::Int(0), Object::Int(0)]),
);
font.set(b"FirstChar".to_vec(), Object::Int(1));
font.set(b"LastChar".to_vec(), Object::Int(clusters.len() as i64));
font.set(b"Widths".to_vec(), Object::Array(widths));
font.set(b"Encoding".to_vec(), Object::Dict(encoding));
font.set(b"CharProcs".to_vec(), Object::Dict(charprocs));
font.set(b"ToUnicode".to_vec(), Object::Ref(cmap));
doc.add(Object::Dict(font))
}
/// Content and resources for an item; `view` maps display space to user space. `taken` holds the
/// page's font names. `existing`: the item is being replaced, so a build without the Arabic or
/// Hebrew face keeps drawing that text as `?`, as before it was supported, rather than refusing to
@@ -390,3 +242,60 @@ pub(super) fn params(c: &Content) -> Dict {
}
d
}
/// Existing Thai text uses absolute shaped glyph positions. Reuse Add Text's bounded outline
/// writer instead of trying to embed the matching craft-fonts variable program as a static face.
pub(crate) fn thai_glyph_resources(
doc: &mut Document,
fonts: &mut Dict,
shaped: &pdfcraft_fonts::source_shaping::ShapedRun,
) -> Result<Vec<(String, Vec<u8>)>, EditError> {
if shaped.glyphs.len() > 65_534 {
return Err(EditError::Invalid("shaped row exceeds its geometry budget".into()));
}
let mut outlines = std::collections::HashMap::new();
let mut clusters = Vec::new();
let mut index = std::collections::HashMap::new();
let mut indices = Vec::new();
for glyph in &shaped.glyphs {
let id = glyph.id.to_u32();
let key = (ShapedScript::Thai, id);
if let std::collections::hash_map::Entry::Vacant(entry) = outlines.entry(key) {
entry.insert(shaped_glyph(ShapedScript::Thai, id).map_err(|e| thai_error(e, &glyph.text))?);
}
let at = *index.entry((id, glyph.text.as_str())).or_insert_with(|| {
let at = clusters.len();
let width = outlines.get(&key).map_or(0.0, |g| g.width);
clusters.push(pdfcraft_fonts::ShapedCluster {
script: ShapedScript::Thai,
glyphs: vec![(id, [0.0, 0.0])],
advance: width,
text: glyph.text.clone(),
});
at
});
indices.push(at);
}
// Validate all outlines before adding resources. Type 3 character codes are single bytes;
// shard long rows and return the appropriate resource/code for every positioned glyph.
let mut names = Vec::new();
for chunk in clusters.chunks(shaped_text::GLYPHS_PER_FONT) {
let refs: Vec<_> = chunk.iter().collect();
let font = shaped_text::add_type3_font(doc, &refs, &outlines);
doc.update_dict(font, |f| f.set(b"Name".to_vec(), Object::name("PCThai")))?;
let mut name = format!("PCThai{}", font.num);
while fonts.contains(name.as_bytes()) {
name.push('_');
}
fonts.set(name.as_bytes().to_vec(), Object::Ref(font));
names.push(name);
}
indices
.into_iter()
.map(|at| {
let name = names.get(at / shaped_text::GLYPHS_PER_FONT).ok_or_else(|| EditError::Invalid("Thai font resource is missing".into()))?;
let code = u8::try_from(at % shaped_text::GLYPHS_PER_FONT + 1).map_err(|_| EditError::Invalid("Thai glyph limit".into()))?;
Ok((name.clone(), vec![code]))
})
.collect()
}
crates/edit/src/added/layout.rs
+124 -41
View File
@@ -1,7 +1,6 @@
use super::types::{AddedImage, AddedText};
use super::{
Content, Document, EditError, GlyphError, Level, ObjRef, Object, ParagraphBidiInfo, ShapedCluster, ShapedScript, shape_arabic, shape_hebrew,
shape_thai,
Content, Document, EditError, GlyphError, Level, ObjRef, Object, ParagraphBidiInfo, ShapedScript, shape_arabic, shape_hebrew, shape_thai,
};
pub(super) fn nums(doc: &Document, o: Option<&Object>) -> Vec<f64> {
@@ -73,8 +72,11 @@ pub(super) fn wrapped(t: &AddedText) -> Vec<(String, bool, usize)> {
out
}
/// Thai wrapping at shaped cluster boundaries never detaches a tone/vowel mark from its base.
/// This is glyph-safe wrapping, not dictionary-based Thai word segmentation.
/// Thai wrapping: a row ends before a word that would overflow it, the words found by the
/// dictionary ([`crate::linebreak`]) inside runs without spaces and at the spaces between them. The
/// spaces where a row ends are dropped, as in the Latin wrapper. Only a word wider than the row is
/// broken inside, at shaped cluster boundaries, which never detach a tone or vowel mark from its
/// base.
fn thai_lines(t: &AddedText, para: &str, width: f64) -> Result<Vec<String>, EditError> {
let (pieces, _) = shaped_layout(t, para, false)?;
let mut atoms = Vec::new();
@@ -88,24 +90,88 @@ fn thai_lines(t: &AddedText, para: &str, width: f64) -> Result<Vec<String>, Edit
})),
}
}
let mut lines = Vec::new();
let (mut line, mut used) = (String::new(), 0.0);
for (text, advance) in atoms {
if !line.is_empty() && used + advance > width {
lines.push(std::mem::take(&mut line));
used = 0.0;
}
line.push_str(&text);
used += advance;
}
lines.push(line);
Ok(lines)
Ok(wrap_words(thai_words(atoms), width))
}
pub(super) fn is_thai(c: char) -> bool {
matches!(u32::from(c), 0x0E00..=0x0E7F)
/// `words` (each a list of text and advance, with the spaces that follow it) laid into rows no
/// wider than `width`: a word that doesn't fit what is left of a row starts the next one, and only
/// a word wider than a whole row is split, between its clusters.
fn wrap_words(words: Vec<Vec<(String, f64)>>, width: f64) -> Vec<String> {
let blank = |s: &str| s.chars().all(char::is_whitespace);
let mut lines: Vec<String> = Vec::new();
let (mut line, mut used) = (String::new(), 0.0);
let end_row = |lines: &mut Vec<String>, line: &mut String| {
let done = std::mem::take(line);
lines.push(done.trim_end().to_string());
};
for word in words {
// The word's own width: the spaces after it may hang past the row's end.
let ink: f64 = word.iter().filter(|(s, _)| !blank(s)).map(|(_, w)| w).sum();
// A row holding only the spaces that open the paragraph has nothing to end yet.
if !blank(&line) && used + ink > width {
end_row(&mut lines, &mut line);
used = 0.0;
}
// Too wide for what is left of the row, counting the spaces that open the paragraph.
let too_wide = blank(&line) && used + ink > width;
for (text, advance) in word {
if too_wide && !blank(&line) && !blank(&text) && used + advance > width {
end_row(&mut lines, &mut line);
used = 0.0;
}
// A row after a wrap does not begin with the spaces where the last one ended.
if line.is_empty() && !lines.is_empty() && blank(&text) {
continue;
}
line.push_str(&text);
used += advance;
}
}
lines.push(line);
lines
}
/// The shaped clusters `atoms` (text and advance) grouped into words: a word starts after spaces,
/// or where [`crate::linebreak::breaks`] lets a row end inside a run without spaces, and keeps the
/// spaces that follow it. A break that falls inside a cluster is not one.
fn thai_words(atoms: Vec<(String, f64)>) -> Vec<Vec<(String, f64)>> {
let text: String = atoms.iter().map(|(s, _)| s.as_str()).collect();
let mut cuts = Vec::new();
let mut run_start: Option<usize> = None;
for (at, c) in text.char_indices().chain(std::iter::once((text.len(), ' '))) {
match (c.is_whitespace(), run_start) {
(false, None) => {
cuts.push(at);
run_start = Some(at);
}
(true, Some(start)) => {
let run = text.get(start..at).unwrap_or_default();
cuts.extend(crate::linebreak::breaks(run).into_iter().map(|b| start.saturating_add(b)));
run_start = None;
}
_ => {}
}
}
// `cuts` is in order: each run's start, then the breaks inside it, then the next run's start.
let mut words: Vec<Vec<(String, f64)>> = Vec::new();
let mut cuts = cuts.into_iter().peekable();
let mut offset = 0usize;
for atom in atoms {
while cuts.peek().is_some_and(|&c| c < offset) {
cuts.next();
}
let starts_word = cuts.peek() == Some(&offset);
offset = offset.saturating_add(atom.0.len());
match words.last_mut() {
Some(word) if !starts_word => word.push(atom),
_ => words.push(vec![atom]),
}
}
words
}
pub(super) use pdfcraft_fonts::shaped_text::{Piece, is_thai};
/// Arabic letters, marks, digits and punctuation: an item with any of them is shaped with the
/// craft-fonts Arabic face.
pub(super) fn is_arabic(c: char) -> bool {
@@ -184,13 +250,6 @@ fn mirrored(c: char) -> char {
}
}
/// A piece of a line with shaped text, in drawing order.
pub(super) enum Piece {
/// Shown in the item's own standard font.
Latin(String),
Shaped(Vec<ShapedCluster>),
}
/// A line of a shaped-text item in drawing order (the Unicode bidirectional algorithm, UAX #9), and
/// its width at the item's size. `rtl` is its paragraph's direction.
pub(super) fn shaped_layout(t: &AddedText, line: &str, rtl: bool) -> Result<(Vec<Piece>, f64), EditError> {
@@ -244,20 +303,13 @@ pub(super) fn shaped_layout(t: &AddedText, line: &str, rtl: bool) -> Result<(Vec
});
}
}
let width = pieces
.iter()
.map(|p| match p {
Piece::Latin(s) => latin_width(t, s),
// Each cluster is shown on its own, so the spacing follows each once.
Piece::Shaped(clusters) => clusters.iter().map(|c| c.advance * t.size + t.char_spacing).sum::<f64>(),
})
.sum();
let width = pdfcraft_fonts::shaped_text::pieces_width(&pieces, t.size, t.char_spacing, |s, _| latin_width(t, s));
Ok((pieces, width))
}
/// The width of `s` in a shaped-text item; the item's font's estimate if it can't be shaped (drawing
/// then reports why).
fn shaped_width(t: &AddedText, s: &str, rtl: bool) -> f64 {
pub(crate) fn shaped_width(t: &AddedText, s: &str, rtl: bool) -> f64 {
shaped_layout(t, s, rtl).map(|(_, w)| w).unwrap_or_else(|_| latin_width(t, s))
}
@@ -282,13 +334,7 @@ pub(super) fn hebrew_error(e: GlyphError, text: &str) -> EditError {
}
pub(super) fn thai_error(e: GlyphError, text: &str) -> EditError {
EditError::Invalid(match e {
GlyphError::NoFont => {
"Thai text needs a Thai font in this build; rebuild with CRAFT_FONTS_DIR pointing to craft-fonts with a Thai face".into()
}
GlyphError::Missing => format!("the Thai font can't show \"{text}\""),
GlyphError::TooComplex => "the Thai glyphs are too complex; use a shorter text box".into(),
})
EditError::Invalid(pdfcraft_fonts::shaped_text::thai_message(e, text))
}
/// The box a text item occupies (height from its lines).
@@ -357,3 +403,40 @@ fn copy_key(c: &Content) -> Content {
pub fn text_width(t: &AddedText) -> f64 {
t.text.split('\n').map(|line| shaped_width(t, line, unicode_bidi::get_base_direction(line) == unicode_bidi::Direction::Rtl)).fold(0.0, f64::max)
}
#[cfg(test)]
mod thai_wrap_tests {
use super::{thai_words, wrap_words};
/// One atom per character, each `advance` wide: what shaping hands over, without a Thai face.
fn atoms(text: &str, advance: f64) -> Vec<(String, f64)> {
text.chars().map(|c| (c.to_string(), advance)).collect()
}
fn texts(words: &[Vec<(String, f64)>]) -> Vec<String> {
words.iter().map(|w| w.iter().map(|(s, _)| s.as_str()).collect()).collect()
}
#[test]
fn thai_atoms_group_into_dictionary_words_with_their_spaces() {
assert_eq!(texts(&thai_words(atoms("น้ำประปาสะอาด", 1.0))), ["น้ำ", "ประปา", "สะอาด"]);
assert_eq!(texts(&thai_words(atoms("สวัสดี ครับ", 1.0))), ["สวัสดี ", "ครับ"]);
// A break the dictionary finds inside one shaped cluster is not a place to end a row.
assert_eq!(texts(&thai_words(vec![("น้ำประปา".into(), 8.0)])), ["น้ำประปา"]);
assert!(thai_words(Vec::new()).is_empty());
}
#[test]
fn words_wrap_whole_and_only_an_overlong_word_is_split() {
let words = || vec![vec![("ab".into(), 2.0), (" ".into(), 1.0)], vec![("cd".into(), 2.0)]];
assert_eq!(wrap_words(words(), 10.0), ["ab cd"]);
// The space that ends a row hangs past it, and the next row doesn't begin with it.
assert_eq!(wrap_words(words(), 3.0), ["ab", "cd"]);
let long = vec![vec![("a".into(), 2.0), ("b".into(), 2.0), ("c".into(), 2.0)]];
assert_eq!(wrap_words(long, 3.0), ["a", "b", "c"]);
// Spaces that open a paragraph stay, and a word too wide for the rest of their row is split.
let opening = vec![vec![(" ".into(), 1.0)], vec![("a".into(), 2.0), ("b".into(), 2.0)]];
assert_eq!(wrap_words(opening, 3.0), [" a", "b"]);
assert_eq!(wrap_words(Vec::new(), 3.0), [""]);
}
}
crates/edit/src/added/thai_tests.rs
+59
View File
@@ -0,0 +1,59 @@
//! Existing Thai text drawn from the bundled Thai outlines (`thai_glyph_resources`): one Type 3
//! code per distinct glyph and cluster text, in fonts of `GLYPHS_PER_FONT` codes each. The
//! outline checks need the craft-fonts Thai face; the budget check runs without it.
use pdfcraft_cos::{Dict, Document};
use pdfcraft_fonts::source_shaping::{ShapedGlyph, ShapedRun};
use super::*;
/// `count` copies of `glyph`, each with cluster text of its own (so each needs a code of its own).
fn distinct(glyph: &ShapedGlyph, count: usize) -> ShapedRun {
let glyphs = (0..count).map(|i| ShapedGlyph { position: [i as f64, 0.0], text: format!("{i}"), ..glyph.clone() }).collect();
ShapedRun { glyphs, advance: count as f64 }
}
#[test]
fn thai_glyph_resources_refuse_a_row_past_the_glyph_budget() {
let glyph = ShapedGlyph { id: Default::default(), position: [0.0, 0.0], text: String::new() };
let over = ShapedRun { glyphs: vec![glyph; 65_535], advance: 0.0 };
let (mut doc, mut fonts) = (Document::new_empty(), Dict::new());
assert!(thai_glyph_resources(&mut doc, &mut fonts, &over).is_err());
assert!(fonts.is_empty(), "a refused row adds no font");
}
/// Codes fill a font up to `GLYPHS_PER_FONT`, the next distinct cluster starts another font at
/// code 1, and the same glyph with the same text again reuses its code. One glyph drawn for two
/// cluster texts takes two codes, since each code's ToUnicode gives its own text.
#[test]
fn thai_glyph_resources_shard_distinct_clusters_at_the_font_boundary() {
let Some(face) = pdfcraft_fonts::document_thai_font() else {
eprintln!(
"skipping thai_glyph_resources_shard_distinct_clusters_at_the_font_boundary: built without craft-fonts (set CRAFT_FONTS_DIR to run it)"
);
return;
};
let ko = pdfcraft_fonts::source_shaping::shape(face.bytes, "ก").unwrap().glyphs.remove(0);
for (clusters, fonts_used) in [(GLYPHS_PER_FONT - 1, 1), (GLYPHS_PER_FONT, 1), (GLYPHS_PER_FONT + 1, 2)] {
let mut shaped = distinct(&ko, clusters);
shaped.glyphs.push(shaped.glyphs[0].clone());
let (mut doc, mut fonts) = (Document::new_empty(), Dict::new());
let placed = thai_glyph_resources(&mut doc, &mut fonts, &shaped).unwrap();
assert_eq!((fonts.len(), placed.len()), (fonts_used, clusters + 1), "{clusters} clusters");
assert_eq!(placed.last(), placed.first(), "a repeated glyph and text reuses its code");
let codes: std::collections::HashSet<_> = placed.iter().collect();
assert_eq!(codes.len(), clusters, "every distinct cluster has a code of its own");
assert_eq!(placed[GLYPHS_PER_FONT - 2].1, [(GLYPHS_PER_FONT - 1) as u8]);
if clusters > GLYPHS_PER_FONT {
assert_eq!(placed[GLYPHS_PER_FONT - 1].1, [GLYPHS_PER_FONT as u8]);
assert_eq!(placed[GLYPHS_PER_FONT].1, [1], "the next font starts at code 1");
assert_ne!(placed[GLYPHS_PER_FONT].0, placed[0].0);
}
}
// At the budget: one cluster repeated 65 534 times needs one code.
let mut row = distinct(&ko, 1);
row.glyphs = vec![row.glyphs[0].clone(); 65_534];
let (mut doc, mut fonts) = (Document::new_empty(), Dict::new());
let placed = thai_glyph_resources(&mut doc, &mut fonts, &row).unwrap();
assert_eq!((fonts.len(), placed.len()), (1, 65_534));
}
crates/edit/src/apply.rs
+45 -10
View File
@@ -1,9 +1,12 @@
use pdfcraft_fonts::shaped_text::{is_thai, thai_refusal, thai_width};
use super::marks::{
Background, Context, HeaderFooter, MarkKind, Watermark, add_picture_resource, bates_start, expand, matrix_number, n, picture, rgb, shrink_matrix,
Background, Context, HeaderFooter, MarkKind, MarkText, Watermark, add_picture_resource, bates_start, expand, matrix_number, n, picture, rgb,
shrink_matrix,
};
use super::place::{
END, Isolation, add_page_resources, add_resources, begin, check, check_on_top, contents, free_font_name, isolation, page_fonts, page_list, place,
place_inside_wrapper, stream_tag, text_op, undrawable, wrapper_role,
place_inside_wrapper, stream_tag, undrawable, wrapper_role,
};
use super::{Document, EditError, Object, added, backdrop, helvetica_width};
@@ -31,6 +34,9 @@ fn add_header_footer_in_place(doc: &mut Document, pages: &[usize], hf: &HeaderFo
if hf.text.iter().all(|t| t.trim().is_empty()) {
return Err(EditError::Invalid("type the header or footer text first".into()));
}
if let Some(why) = hf.text.iter().find_map(|t| thai_refusal(t)) {
return Err(EditError::Invalid(why));
}
if let Some(c) = hf.first_undrawable() {
return Err(undrawable(c));
}
@@ -73,12 +79,23 @@ fn add_header_footer_in_place(doc: &mut Document, pages: &[usize], hf: &HeaderFo
}
let all = page_list(doc);
let count = all.len();
let number_of = |k: usize| {
u32::try_from(u64::from(hf.start_number) + k as u64).map_err(|_| EditError::Invalid("the header page numbers are too large".into()))
};
let bates_of = |k: usize| bates0.checked_add(k as u64).ok_or_else(|| EditError::Invalid("the Bates numbers are too large".into()));
// Every Thai line drawn on any page (a Bates prefix may join a template's Thai), for its fonts.
let mut thai_lines = std::collections::BTreeSet::new();
if hf.text.iter().any(|t| t.chars().any(is_thai)) {
for k in 0..pages.len() {
let (number, bates) = (number_of(k)?, bates_of(k)?);
thai_lines.extend(hf.text.iter().flat_map(|t| expand(t, number, count, bates, cx).lines().map(str::to_owned).collect::<Vec<_>>()));
}
}
let thai = MarkText::new(doc, &all, pages, thai_lines.iter().map(String::as_str))?;
for (k, &i) in pages.iter().enumerate() {
let page = &all[i];
let (w, h) = page.display_size(doc);
let number =
u32::try_from(u64::from(hf.start_number) + k as u64).map_err(|_| EditError::Invalid("the header page numbers are too large".into()))?;
let bates = bates0.checked_add(k as u64).ok_or_else(|| EditError::Invalid("the Bates numbers are too large".into()))?;
let (number, bates) = (number_of(k)?, bates_of(k)?);
let size = hf.font_size;
let [top, bottom, left, right] = hf.margins;
// The page resource the face is added under: the usual name, unless the page already
@@ -91,6 +108,7 @@ fn add_header_footer_in_place(doc: &mut Document, pages: &[usize], hf: &HeaderFo
let base = hf.font.base_font(false, false);
let resource = free_font_name(doc, &page_fonts(doc, page), preferred, base);
let mut body: Vec<u8> = format!("BT\n/{resource} {} Tf\n{}\n", n(size), rgb(hf.color)).into_bytes();
let mut current = None;
let mut underlines = String::new();
for (slot, template) in hf.text.iter().enumerate() {
if template.trim().is_empty() {
@@ -101,7 +119,8 @@ fn add_header_footer_in_place(doc: &mut Document, pages: &[usize], hf: &HeaderFo
let line_h = size * 1.2;
for (li, line) in lines.iter().enumerate() {
// Centred and right-aligned text is placed by its width in the chosen face.
let tw = hf.font.width(line, size, false, false);
let face = |s: &str, size: f64| hf.font.width(s, size, false, false);
let tw = thai_width(line, size, face);
let x = match slot % 3 {
0 => left,
1 => (w - tw) / 2.0,
@@ -109,7 +128,7 @@ fn add_header_footer_in_place(doc: &mut Document, pages: &[usize], hf: &HeaderFo
};
// Headers hang below the top margin; footers sit on the bottom margin.
let y = if header { h - top - size * 0.8 - li as f64 * line_h } else { bottom + (lines.len() - 1 - li) as f64 * line_h };
body.extend(text_op(x, y, line));
thai.show(&mut body, line, x, y, size, &resource, |s| face(s, size), &mut current);
if hf.underline {
underlines.push_str(&format!("{} {} {} {} re f\n", n(x), n(y - size * 0.15), n(tw), n((size * 0.06).max(0.4))));
}
@@ -126,6 +145,7 @@ fn add_header_footer_in_place(doc: &mut Document, pages: &[usize], hf: &HeaderFo
if !page_fonts(doc, page).contains(resource.as_bytes()) {
add_page_resources(doc, page, None, &[(resource.into_bytes(), base)])?;
}
thai.add_to(doc, &page_list(doc)[i])?;
let page = &page_list(doc)[i];
place(doc, page, MarkKind::HeaderFooter, content, false)?;
if let Some(Some(m)) = shrinks.get(k) {
@@ -150,6 +170,9 @@ fn add_watermark_in_place(doc: &mut Document, pages: &[usize], wm: &Watermark, r
if text.is_empty() && wm.source.is_none() {
return Err(EditError::Invalid("type the watermark text first".into()));
}
if let Some(why) = wm.source.is_none().then(|| thai_refusal(text)).flatten() {
return Err(EditError::Invalid(why));
}
if let Some(c) = wm.first_undrawable() {
return Err(undrawable(c));
}
@@ -171,6 +194,7 @@ fn add_watermark_in_place(doc: &mut Document, pages: &[usize], wm: &Watermark, r
}
let opacity = wm.opacity.clamp(0.0, 1.0);
let lines: Vec<&str> = text.lines().collect();
let thai = if wm.source.is_none() { MarkText::new(doc, &page_list(doc), pages, lines.iter().copied())? } else { MarkText::default() };
for &i in pages {
let page = page_list(doc)[i].clone();
let (w, h) = page.display_size(doc);
@@ -185,7 +209,7 @@ fn add_watermark_in_place(doc: &mut Document, pages: &[usize], wm: &Watermark, r
place(doc, page, MarkKind::Watermark, content.into_bytes(), wm.behind)?;
continue;
}
let widest = lines.iter().map(|l| helvetica_width(l, 1.0)).fold(0.0, f64::max).max(0.01);
let widest = lines.iter().map(|l| thai_width(l, 1.0, helvetica_width)).fold(0.0, f64::max).max(0.01);
let size = if wm.font_size > 0.0 { wm.font_size } else { ((w * w + h * h).sqrt() * 0.5 / widest).clamp(6.0, 300.0) };
let (s, c) = wm.rotation.to_radians().sin_cos();
let (cx, cy) = (w / 2.0 + wm.offset[0], h / 2.0 + wm.offset[1]);
@@ -207,13 +231,24 @@ fn add_watermark_in_place(doc: &mut Document, pages: &[usize], wm: &Watermark, r
);
let line_h = size * 1.15;
let total = line_h * (lines.len() as f64 - 1.0);
let mut current = None;
for (li, line) in lines.iter().enumerate() {
let tw = helvetica_width(line, size);
content.extend(text_op(-tw / 2.0, total / 2.0 - li as f64 * line_h - size * 0.35, line));
let tw = thai_width(line, size, helvetica_width);
thai.show(
&mut content,
line,
-tw / 2.0,
total / 2.0 - li as f64 * line_h - size * 0.35,
size,
"PCHelv",
|s| helvetica_width(s, size),
&mut current,
);
}
content.extend_from_slice(b"ET\n");
content.extend_from_slice(END.as_bytes());
add_resources(doc, &page, Some(opacity), None)?;
thai.add_to(doc, &page_list(doc)[i])?;
let page = &page_list(doc)[i];
place(doc, page, MarkKind::Watermark, content, wm.behind)?;
}
crates/edit/src/images.rs
+43 -2
View File
@@ -1,7 +1,7 @@
//! Edit a PDF ▸ edit existing images and grouped artwork (Image/Form XObjects painted by `Do`
//! in its own content streams), and changing one: move/resize/rotate/flip (the `Do` is wrapped in
//! `q … cm … Q` with the extra transform), replace (it draws another XObject in the same place),
//! or delete. Only the stream that draws it is rewritten, as a new object.
//! delete, or ungroup grouped artwork (its content drawn in place, see `ungroup`). Only the stream that draws it is rewritten, as a new object.
use kurbo::{BezPath, Point, Shape};
use pdfcraft_content::{Matrix, Op, parse, serialize_ops};
@@ -49,7 +49,7 @@ pub struct PageImage {
/// The content stream its `Do` (its first token) is in, and the `Do`'s index among the
/// operators of the page's streams joined ([`pdfcraft_content::Pieces`]).
pub stream: usize,
op: usize,
pub(crate) op: usize,
/// CTM before `Do`, excluding a Form's own Matrix.
draw_matrix: [f64; 6],
clips: Vec<ClipPath>,
@@ -465,6 +465,9 @@ pub enum ImageChange {
Arrange(ArrangeOrder),
/// Remove any private crop on the image.
RemoveCrop,
/// Grouped Form artwork only: draw the form's content in place of its `Do`, so its text and
/// pictures become the page's own and can be edited one by one (#2964).
Ungroup,
}
/// The user-space transform that maps box `from` onto box `to` (move and resize).
@@ -1660,6 +1663,26 @@ pub fn change_image(doc: &mut Document, page: usize, index: usize, change: &Imag
ImageChange::Crop(keep) => {
return crop_image_impl(doc, &p, &all, &img, *keep);
}
ImageChange::Ungroup => {
if !img.is_form {
return Err(EditError::Invalid("only grouped Form artwork can be ungrouped".into()));
}
let res = p.dict.get(b"Resources").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
let space = crate::ungroup::page_blend_space(doc, &p.dict);
// One allowance for the form and everything it draws.
let mut budget = pdfcraft_cos::DecodeBudget::new(crate::ungroup::MAX_FORM_BYTES, 0);
let crate::ungroup::Ungrouped { ops: inline, resources: mut merged, patterns, .. } =
crate::ungroup::ungroup_ops(doc, &res, &ops, img.op, space.as_ref(), Matrix(img.matrix), &mut budget).map_err(|r| r.error)?;
if !patterns.is_empty() {
let mut table = merged.get(b"Pattern").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
for (name, copy) in patterns {
table.set(name, Object::Ref(doc.add(copy)));
}
merged.set(b"Pattern".to_vec(), Object::Dict(table));
}
resources = Some(merged);
inline
}
ImageChange::Replace(r) => {
if img.is_form {
return Err(EditError::Invalid(
@@ -1861,6 +1884,24 @@ fn crop_image_impl(
Ok(())
}
/// The `Do` of each image `indexes` (from [`page_images`]) on `page`, as an index among the
/// page's joined operators: what Copy takes of them.
pub(crate) fn image_ops(doc: &Document, page: usize, indexes: &[usize]) -> Result<Vec<usize>, EditError> {
if indexes.is_empty() {
return Ok(Vec::new());
}
let images = page_images(doc, page)?;
indexes
.iter()
.map(|i| {
images
.get(*i)
.map(|image| image.op)
.ok_or_else(|| EditError::Invalid("a selected image no longer exists; select the objects again".into()))
})
.collect()
}
/// Plan all transforms before changing a stream; indexes refer to the original inventory.
/// Splice only the chosen Do operators and preserve all other bytes and stream dictionaries.
pub(crate) fn translate_images(doc: &mut Document, page: usize, indexes: &[usize], offset: [f64; 2]) -> Result<(), EditError> {
crates/edit/src/lib.rs
+5 -2
View File
@@ -23,7 +23,7 @@
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
use pdfcraft_cos::{Dict, Document, ObjRef, Object, Stream};
use pdfcraft_fonts::{helvetica_width, literal, win_ansi};
use pdfcraft_fonts::helvetica_width;
#[derive(Debug, thiserror::Error, PartialEq)]
pub enum EditError {
@@ -46,11 +46,14 @@ pub use background::page_background;
mod rules;
pub use rules::page_rules;
pub mod objects;
pub use objects::{EditableObject, MAX_MOVE_OBJECTS, ObjectKind, ObjectTarget, editable_objects, move_objects};
pub use objects::{
CopiedObjects, EditableObject, MAX_MOVE_OBJECTS, ObjectKind, ObjectTarget, copy_objects, editable_objects, move_objects, paste_objects,
};
mod flatten;
pub use flatten::{flatten, flatten_fill_sign};
pub mod images;
mod ungroup;
pub use images::{ArrangeOrder, ImageChange, PageImage, change_image, page_images, reading_images, rect_to_rect, turn_about_centre, turn_by_degrees};
pub mod bidi;
pub mod indic;
crates/edit/src/linebreak.rs
+142 -14
View File
@@ -6,8 +6,56 @@
//! and quotes, currency signs before an amount): the line-start and line-end prohibitions of
//! JIS X 4051 ("kinsoku"), which UAX #14 gives as classes CL, CP, EX, IS, NS and OP, PR.
//!
//! Korean is written with spaces between words and keeps breaking only there; so does Thai, whose
//! words need a dictionary to find.
//! Thai is also written without spaces between words, but a row may not end between any two of its
//! characters: its words need a dictionary to find. They come from ICU4X's dictionary line
//! segmenter (UAX #14 with dictionary breaking for class SA), kept only where a Thai character is on
//! either side. A row never begins with a following vowel, a tone or other mark, Mai Yamok (ๆ) or
//! Paiyannoi (ฯ), nor ends with a leading vowel (เ แ โ ใ ไ), whatever the dictionary says.
//!
//! Korean is written with spaces between words and keeps breaking only there.
use std::sync::LazyLock;
use icu_segmenter::options::LineBreakOptions;
use icu_segmenter::{LineSegmenter, LineSegmenterBorrowed};
/// The dictionary segmenter, built once: its data is compiled in, so building it cannot fail.
static SEGMENTER: LazyLock<LineSegmenterBorrowed<'static>> = LazyLock::new(|| LineSegmenter::new_dictionary(LineBreakOptions::default()));
/// Letters, vowels, marks, digits and signs of the Thai block.
fn thai(c: char) -> bool {
matches!(c, '\u{0e01}'..='\u{0e5b}')
}
/// Thai that belongs to the character before it: Paiyannoi, the following vowels, Lakkhangyao,
/// Mai Yamok and the tone and other marks.
fn thai_no_start(c: char) -> bool {
matches!(c, '\u{0e2f}' | '\u{0e30}'..='\u{0e3a}' | '\u{0e45}'..='\u{0e4e}')
}
/// The leading vowels, written before the consonant they follow in speech.
fn thai_no_end(c: char) -> bool {
matches!(c, '\u{0e40}'..='\u{0e44}')
}
/// Byte offsets where the dictionary lets a row of Thai end inside `word`.
fn thai_breaks(word: &str) -> Vec<usize> {
if !word.chars().any(thai) {
return Vec::new();
}
SEGMENTER
.segment_str(word)
.filter(|&at| {
let before = word.get(..at).and_then(|s| s.chars().next_back());
let after = word.get(at..).and_then(|s| s.chars().next());
match (before, after) {
(Some(b), Some(a)) => (thai(b) || thai(a)) && !thai_no_start(a) && !thai_no_end(b) && !no_start(a) && !no_end(b),
// The start and end of the word are not breaks inside it.
_ => false,
}
})
.collect()
}
/// Writing with no spaces between words, where a row may end between two characters (UAX #14 ID).
fn ideographic(c: char) -> bool {
@@ -77,29 +125,49 @@ fn no_end(c: char) -> bool {
)
}
/// Whether a row may end between `before` and `after`, neither of them a space.
pub(crate) fn between(before: char, after: char) -> bool {
(ideographic(before) || ideographic(after)) && !no_start(after) && !no_end(before)
/// Whether a row may end between `before` and `after` in Chinese or Japanese, neither of them a
/// space. Thai needs the whole word ([`breaks`]).
fn between(before: char, after: char) -> bool {
(ideographic(before) || ideographic(after)) && !no_start(after) && !no_end(before) && !thai_no_start(after) && !thai_no_end(before)
}
/// Byte offsets inside `word` (text without spaces) where a row may end, in order: never 0 or the
/// word's length, always on a character boundary.
pub(crate) fn breaks(word: &str) -> Vec<usize> {
let mut out: Vec<usize> = Vec::new();
let mut prev: Option<char> = None;
for (at, c) in word.char_indices() {
if let Some(before) = prev
&& between(before, c)
{
out.push(at);
}
prev = Some(c);
}
out.extend(thai_breaks(word));
out.sort_unstable();
out.dedup();
out
}
/// `word` (text without spaces) cut where a row may end inside it; the pieces join with nothing.
pub(crate) fn pieces(word: &str) -> Vec<&str> {
let mut out = Vec::new();
let mut start = 0;
let mut prev: Option<char> = None;
for (at, c) in word.char_indices() {
if let Some(before) = prev
&& between(before, c)
{
out.extend(word.get(start..at));
start = at;
}
prev = Some(c);
for at in breaks(word) {
out.extend(word.get(start..at));
start = at;
}
out.extend(word.get(start..).filter(|rest| !rest.is_empty()));
out
}
/// For each character of `word` (text without spaces), whether a row may end just before it.
pub(crate) fn breaks_before_chars(word: &str) -> Vec<bool> {
let cuts = breaks(word);
word.char_indices().map(|(at, _)| cuts.binary_search(&at).is_ok()).collect()
}
#[cfg(test)]
mod tests {
use super::*;
@@ -121,4 +189,64 @@ mod tests {
assert_eq!(pieces(""), Vec::<&str>::new());
assert!(!between('a', 'b') && between('字', 'a') && !between('字', '。') && !between('「', '字'));
}
#[test]
fn thai_breaks_between_dictionary_words() {
assert_eq!(pieces("ภาษาไทยไม่มีช่องว่างระหว่างคำ"), ["ภาษา", "ไทย", "ไม่มี", "ช่อง", "ว่าง", "ระหว่าง", "คำ"]);
assert_eq!(pieces("น้ำประปาสะอาด"), ["น้ำ", "ประปา", "สะอาด"]);
assert_eq!(pieces("การประปาส่วนภูมิภาค"), ["การ", "ประปา", "ส่วน", "ภูมิภาค"]);
}
#[test]
fn thai_vowels_marks_and_repetition_stay_with_the_word_before_them() {
// The dictionary alone would start a row with Mai Yamok, with the Sara Aa of a decomposed
// Sara Am, or with a closing bracket.
assert_eq!(pieces("เด็กๆชอบกินขนม"), ["เด็กๆ", "ชอบ", "กิน", "ขนม"]);
assert_eq!(pieces("ท\u{0e4d}\u{0e32}งาน"), ["ท\u{0e4d}\u{0e32}", "งาน"]);
assert_eq!(pieces("กรุงเทพฯเป็นเมืองหลวง"), ["กรุงเทพฯ", "เป็น", "เมือง", "หลวง"]);
assert_eq!(pieces("สวัสดีครับ(ทดสอบ)"), ["สวัสดี", "ครับ", "(ทดสอบ)"]);
}
#[test]
fn thai_keeps_numbers_and_latin_whole() {
assert_eq!(pieces("ราคา100บาทต่อเดือน"), ["ราคา", "100บาท", "ต่อ", "เดือน"]);
assert_eq!(pieces("พ.ศ.2569"), ["พ.ศ.2569"]);
assert_eq!(pieces("PDF"), ["PDF"]);
assert_eq!(breaks("ไทย"), Vec::<usize>::new());
}
#[test]
fn breaks_before_chars_marks_the_first_character_of_each_later_piece() {
let flags = breaks_before_chars("น้ำประปา");
let starts: Vec<usize> = flags.iter().enumerate().filter_map(|(i, &b)| b.then_some(i)).collect();
// น ้ ำ | ป ร ะ ป า
assert_eq!(starts, [3]);
assert_eq!(breaks_before_chars(""), Vec::<bool>::new());
}
#[test]
fn pieces_of_arbitrary_thai_rejoin_to_the_input_and_never_start_with_a_mark() {
// A deterministic generator over Thai, Latin, digits and CJK, including mark-only runs and
// unusual orders that no dictionary word has.
let alphabet: Vec<char> = ('\u{0e01}'..='\u{0e5b}').chain(['a', '1', '(', ')', '字', 'の', '.']).collect();
let mut state: u64 = 0x2545_f491_4f6c_dd1d;
for _ in 0..2000 {
let mut word = String::new();
state = state.wrapping_mul(6_364_136_223_846_793_005).wrapping_add(1);
let len = (state >> 59) as usize + 1;
for _ in 0..len {
state = state.wrapping_mul(6_364_136_223_846_793_005).wrapping_add(1);
let pick = (state >> 33) as usize % alphabet.len();
word.extend(alphabet.get(pick));
}
let parts = pieces(&word);
assert_eq!(parts.concat(), word);
assert!(parts.iter().all(|p| !p.is_empty()), "{word:?} gave {parts:?}");
for part in parts.iter().skip(1) {
let first = part.chars().next().unwrap_or(' ');
assert!(!thai_no_start(first) && !no_start(first), "{word:?} gave {parts:?}");
}
assert_eq!(breaks_before_chars(&word).len(), word.chars().count());
}
}
}
crates/edit/src/marks.rs
+82 -10
View File
@@ -1,4 +1,75 @@
use super::{Document, EditError, Object, Rgb, added};
use pdfcraft_fonts::shaped_text::{Piece, ShapedFonts, clusters, first_undrawable_in_line, is_thai, thai_message, thai_pieces};
use super::{Dict, Document, EditError, Object, Rgb, added};
/// The Thai of a header, footer or text watermark, drawn from the Thai face as Add text draws it:
/// Type 3 fonts added to the document once and to each page the mark goes on, beside the page
/// font that draws the rest.
#[derive(Default)]
pub(crate) struct MarkText {
shaped: ShapedFonts,
/// The fonts, by the resource names the marked pages give them.
fonts: Dict,
}
impl MarkText {
/// The fonts for the Thai in `lines` (every line the mark draws, on any page), named so that
/// none of `pages` (of `all`) has the name already. Thai this build can't draw (it has no Thai
/// face, or the face lacks a character) is refused, saying what to do.
pub(crate) fn new<'a>(
doc: &mut Document,
all: &[pdfcraft_model::Page],
pages: &[usize],
lines: impl IntoIterator<Item = &'a str>,
) -> Result<Self, EditError> {
let thai = |line: &str| thai_pieces(line).map_err(|e| EditError::Invalid(thai_message(e, line)));
let pieces = lines.into_iter().filter(|l| l.chars().any(is_thai)).map(thai).collect::<Result<Vec<_>, _>>()?;
if pieces.is_empty() {
return Ok(Self::default());
}
let mut taken = Dict::new();
for page in pages.iter().filter_map(|&i| all.get(i)) {
for (name, font) in crate::place::page_fonts(doc, page).iter() {
taken.set(name.clone(), font.clone());
}
}
let mut fonts = Dict::new();
let shaped =
ShapedFonts::new(clusters(pieces.iter().map(Vec::as_slice)), pdfcraft_fonts::shaped_glyph, &taken, &mut fonts, "PCTh", &mut |o| {
Object::Ref(doc.add(o))
})
.map_err(|(e, c)| EditError::Invalid(thai_message(e, &c.text)))?;
Ok(Self { shaped, fonts })
}
/// Show `line` inside `BT … ET` from `x` on baseline `y` at `size`: its Thai from the fonts,
/// the rest in the page font `latin` (selected when `current` is `None`), `width` wide.
#[allow(clippy::too_many_arguments)]
pub(crate) fn show(
&self,
out: &mut Vec<u8>,
line: &str,
x: f64,
y: f64,
size: f64,
latin: &str,
width: impl Fn(&str) -> f64,
current: &mut Option<usize>,
) {
let plain = || vec![Piece::Latin(line.into())];
// `new` shaped every line drawn, so this shapes.
let pieces = if self.fonts.is_empty() || !line.chars().any(is_thai) { plain() } else { thai_pieces(line).unwrap_or_else(|_| plain()) };
self.shaped.show_line(out, &pieces, x, y, size, latin, pdfcraft_fonts::win_ansi, width, current);
}
/// Add the fonts to `page`'s own resources.
pub(crate) fn add_to(&self, doc: &mut Document, page: &pdfcraft_model::Page) -> Result<(), EditError> {
if self.fonts.is_empty() {
return Ok(());
}
crate::place::add_page_fonts(doc, page, &self.fonts)
}
}
/// The kinds of page marks.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
@@ -61,13 +132,13 @@ impl Default for HeaderFooter {
}
impl HeaderFooter {
/// The first character [`add_header_footer`] would refuse, because the standard fonts can't
/// draw it (#125). Tokens only add ASCII (Bates prefixes and suffixes are part of the
/// template), so checking the templates, split as `expand`'s output is, covers every page
/// (and a `\r` that would be drawn is refused too). The dialog asks this too, so its preview
/// never offers text that applying refuses (#2013).
/// The first character [`add_header_footer`] would refuse, because neither the standard
/// fonts nor (for Thai) the Thai face can draw it (#125). Tokens only add ASCII (Bates
/// prefixes and suffixes are part of the template), so checking the templates, split as
/// `expand`'s output is, covers every page (and a `\r` that would be drawn is refused too).
/// The dialog asks this too, so its preview never offers text that applying refuses (#2013).
pub fn first_undrawable(&self) -> Option<char> {
self.text.iter().flat_map(|t| t.lines()).find_map(pdfcraft_fonts::first_non_win_ansi)
self.text.iter().flat_map(|t| t.lines()).find_map(first_undrawable_in_line)
}
/// The room the header and the footer take at the top and the bottom of a page, in points:
@@ -139,12 +210,13 @@ pub struct Watermark {
}
impl Watermark {
/// The first character of a text watermark [`add_watermark`] would refuse, because the
/// standard fonts can't draw it (#125). A picture watermark draws no text.
/// The first character of a text watermark [`add_watermark`] would refuse, because neither
/// the standard fonts nor (for Thai) the Thai face can draw it (#125). A picture watermark
/// draws no text.
pub fn first_undrawable(&self) -> Option<char> {
match self.source {
Some(_) => None,
None => self.text.trim().lines().find_map(pdfcraft_fonts::first_non_win_ansi),
None => self.text.trim().lines().find_map(first_undrawable_in_line),
}
}
}
crates/edit/src/objects.rs
+3
View File
@@ -9,6 +9,9 @@ use pdfcraft_cos::{DecodeBudget, Dict, Document, Object};
use crate::text::ContentStream;
use crate::{Content, EditError};
mod clip;
pub use clip::{CopiedObjects, copy_objects, paste_objects};
pub const MAX_MOVE_OBJECTS: usize = 1_000;
const MAX_PAGE_OBJECTS: usize = 10_000;
const MAX_COORDINATE: f64 = 1e9;
crates/edit/src/objects/clip.rs
+283
View File
@@ -0,0 +1,283 @@
//! Edit a PDF ▸ Copy, Cut and Paste of objects selected together (#5989): paragraphs, images,
//! Form artwork and added content, pasted as page content that draws the same.
//!
//! A copy is a content stream of its own that draws the objects and nothing else, in the page's
//! user space, with the resources it names. It replays every state operator of the page up to
//! the last object (transformations, clips, colours, ExtGStates, text state), so each object is
//! drawn in the state it had, and leaves out what paints anything else: other text, images,
//! shadings and paths (a path that clips is kept, without its paint). Each copied text-showing
//! operator is placed by its own text matrix, which is where its glyphs start, so the text
//! positioning between operators is left out. Marked content is left out too: a paste is
//! untagged and in no layer.
//!
//! A paste draws that stream on top of the page through Ungroup's merge ([`ungroup_ops`]): the
//! copy is read as Form artwork whose resources join the page's under names that don't clash,
//! moved by the offset, with the patterns it paints moved along. Its resources must already be
//! the target document's: the engine copies them over first when the copy came from another
//! document.
use std::collections::{BTreeMap, BTreeSet, HashMap};
use pdfcraft_content::{Matrix, Op, serialize_ops};
use pdfcraft_cos::{Dict, Document, ObjRef, Object, Stream};
use super::{EditableObject, MAX_COORDINATE, MAX_MOVE_OBJECTS, ObjectKind, ObjectTarget, checked_streams, editable_objects, valid_rect};
use crate::EditError;
use crate::ungroup::{CATEGORIES, DEFAULT_SPACES, MAX_FORM_BYTES, Ungrouped, page_blend_space, resource_operand, ungroup_ops};
/// How deep the copied graphics state may nest (`q`), as the page's own may.
const MAX_NESTING: usize = 1024;
/// Objects copied from one page: what [`paste_objects`] draws again.
#[derive(Clone, Debug, PartialEq)]
pub struct CopiedObjects {
/// A content stream that draws the objects alone, in the source page's user space: its `q`
/// and `Q` balance and its text objects close.
pub content: Vec<u8>,
/// The resources `content` names, as the source page has them: references are into the
/// document the copy came from.
pub resources: Dict,
/// The union of the objects' boxes (user space of the source page).
pub rect: [f64; 4],
/// The objects as [`editable_objects`] listed them, in inventory order.
pub objects: Vec<EditableObject>,
}
impl CopiedObjects {
/// The text of the copied paragraphs and text boxes, one per line: what the system clipboard
/// gets for other applications.
pub fn text(&self) -> String {
self.objects.iter().filter_map(|o| o.text.as_deref()).filter(|t| !t.is_empty()).collect::<Vec<_>>().join("\n")
}
}
/// Copy `targets` (references from one [`editable_objects`] inventory of `page`). The document
/// is not changed.
pub fn copy_objects(doc: &Document, page: usize, targets: &[ObjectTarget]) -> Result<CopiedObjects, EditError> {
if targets.is_empty() || targets.len() > MAX_MOVE_OBJECTS {
return Err(EditError::Invalid("select between 1 and 1000 objects to copy".into()));
}
let mut inventory: BTreeMap<_, _> = editable_objects(doc, page)?.into_iter().map(|o| (o.target, o)).collect();
let mut objects = Vec::with_capacity(targets.len());
for target in targets.iter().collect::<BTreeSet<_>>() {
objects
.push(inventory.remove(target).ok_or_else(|| EditError::Invalid("a selected object no longer exists; select the objects again".into()))?);
}
if objects.len() != targets.len() {
return Err(EditError::Invalid("an object was selected more than once".into()));
}
let indexes = |kind| objects.iter().filter(|o| o.target.kind == kind).map(|o| o.target.index).collect::<Vec<_>>();
let streams = checked_streams(doc, page)?;
let pieces = crate::text::joined(&streams);
let ops = pieces.parse().ops;
let mut keep = crate::text::paragraph_ops(doc, page, &indexes(ObjectKind::Text), false)?;
keep.extend(crate::images::image_ops(doc, page, &indexes(ObjectKind::Image))?);
// Added content owns its stream: everything that stream paints is the item.
let added: Vec<_> = crate::list_added(doc).into_iter().filter(|a| a.page == page).collect();
let owned = indexes(ObjectKind::Added)
.into_iter()
.map(|i| {
added.get(i).map(|a| a.obj).ok_or_else(|| EditError::Invalid("a selected added item no longer exists; select the objects again".into()))
})
.collect::<Result<BTreeSet<ObjRef>, _>>()?;
if !owned.is_empty() {
for (i, op) in ops.iter().enumerate() {
let stream = streams.get(pieces.pieces_of(op).0).and_then(|s| s.entry.as_ref());
if paints(op) && stream.is_some_and(|r| owned.contains(&r)) {
keep.insert(i);
}
}
}
let p = pdfcraft_model::pages(doc).into_iter().nth(page).ok_or(EditError::NoSuchPage(page))?;
let res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let shows = crate::text::show_starts(doc, &ops, &res)?;
let drawn = replay(&ops, &keep, &shows)?;
let rect = objects.iter().map(|o| o.rect).reduce(|a, b| [a[0].min(b[0]), a[1].min(b[1]), a[2].max(b[2]), a[3].max(b[3])]).unwrap_or_default();
Ok(CopiedObjects { resources: used_resources(doc, &res, &drawn), content: serialize_ops(&drawn), rect, objects })
}
/// Whether `op` paints (rather than sets state or builds a path).
fn paints(op: &Op) -> bool {
matches!(op.op.as_slice(), b"S" | b"s" | b"f" | b"F" | b"f*" | b"B" | b"B*" | b"b" | b"b*" | b"Tj" | b"TJ" | b"'" | b"\"" | b"Do" | b"sh" | b"BI")
}
/// The page's operators up to the last of `keep`, painting only `keep` (see the module docs).
/// `shows`: where each text-showing operator starts ([`crate::text::show_starts`]).
fn replay(ops: &[Op], keep: &BTreeSet<usize>, shows: &HashMap<usize, (Matrix, i64)>) -> Result<Vec<Op>, EditError> {
let Some(&last) = keep.last() else { return Err(EditError::Invalid("the selected objects draw nothing to copy".into())) };
let mut out = Vec::new();
let (mut depth, mut in_text) = (0usize, false);
let (mut path, mut clips) = (Vec::new(), false);
for (i, op) in ops.iter().enumerate().take(last.saturating_add(1)) {
let kept = keep.contains(&i);
match op.op.as_slice() {
b"q" => {
if depth >= MAX_NESTING {
return Err(EditError::Invalid("the page graphics state is nested too deeply".into()));
}
depth += 1;
out.push(op.clone());
}
// An unmatched `Q` restores nothing; the renderer ignores it too.
b"Q" if depth > 0 => {
depth -= 1;
out.push(op.clone());
}
b"BT" if !in_text => {
in_text = true;
out.push(op.clone());
}
b"ET" if in_text => {
in_text = false;
out.push(op.clone());
}
b"Q" | b"BT" | b"ET" => {}
b"m" | b"l" | b"c" | b"v" | b"y" | b"re" | b"h" => path.push(op.clone()),
b"W" | b"W*" => {
clips = true;
path.push(op.clone());
}
b"n" | b"S" | b"s" | b"f" | b"F" | b"f*" | b"B" | b"B*" | b"b" | b"b*" => {
if kept || clips {
out.append(&mut path);
out.push(if kept { op.clone() } else { Op::new("n", Vec::new()) });
}
path.clear();
clips = false;
}
b"Tj" | b"TJ" | b"'" | b"\"" => {
// `"` sets the spacing for what follows, shown or not.
if op.is("\"") {
for (name, value) in ["Tw", "Tc"].into_iter().zip(op.operands.iter().take(2)) {
if value.as_f64().is_some() {
out.push(Op::new(name, vec![value.clone()]));
}
}
}
match shows.get(&i) {
Some((tm, _)) if kept => {
out.push(Op::new("Tm", tm.0.map(Object::Real).to_vec()));
out.push(if op.is("TJ") { op.clone() } else { Op::new("Tj", op.operands.last().cloned().into_iter().collect()) });
}
// Its glyphs clip what follows, which leaving them out would not.
Some((_, mode)) if *mode >= 4 => {
return Err(EditError::Invalid(
"text used as a clipping path is drawn before the selected objects, so they can't be copied".into(),
));
}
_ => {}
}
}
// Each copied show sets its own text matrix; `TD` also sets the leading.
b"TD" => {
if let Some([_, y]) = op.nums::<2>() {
out.push(Op::new("TL", vec![Object::Real(-y)]));
}
}
b"Td" | b"Tm" | b"T*" | b"BDC" | b"BMC" | b"EMC" | b"MP" | b"DP" => {}
b"Do" | b"sh" | b"BI" => {
if kept {
out.push(op.clone());
}
}
_ => out.push(op.clone()),
}
}
if in_text {
out.push(Op::new("ET", Vec::new()));
}
out.extend(std::iter::repeat_with(|| Op::new("Q", Vec::new())).take(depth));
Ok(out)
}
/// The entries of the page's resources `res` that `ops` name, and the default colour spaces,
/// which remap the device colours they paint (ISO 32000-2 §8.6.5.6).
fn used_resources(doc: &Document, res: &Dict, ops: &[Op]) -> Dict {
let mut used: BTreeMap<&[u8], BTreeSet<&[u8]>> = BTreeMap::new();
for op in ops {
if let Ok(Some((cat, i))) = resource_operand(op)
&& let Some(Object::Name(name)) = op.operands.get(i)
{
used.entry(cat).or_default().insert(name.as_slice());
}
}
let mut out = Dict::new();
for cat in CATEGORIES {
let Some(all) = res.get(cat).map(|d| doc.resolve(d)).and_then(|d| d.as_dict().cloned()) else { continue };
let mut mine = Dict::new();
for (key, value) in all.iter() {
if used.get(cat).is_some_and(|u| u.contains(key.as_slice())) || (cat == b"ColorSpace" && DEFAULT_SPACES.contains(&key.as_slice())) {
mine.set(key.clone(), value.clone());
}
}
if !mine.is_empty() {
out.set(cat.to_vec(), Object::Dict(mine));
}
}
out
}
/// Paste `copied` on top of `page`, moved by `offset` (user space), as one change: its content
/// and the page's resources merged with its own. Its resources must be this document's.
pub fn paste_objects(doc: &mut Document, page: usize, copied: &CopiedObjects, offset: [f64; 2]) -> Result<(), EditError> {
let r = copied.rect;
if !offset.iter().all(|v| v.is_finite() && v.abs() <= MAX_COORDINATE)
|| !valid_rect([r[0] + offset[0], r[1] + offset[1], r[2] + offset[0], r[3] + offset[1]])
{
return Err(EditError::Invalid("the paste position is out of range".into()));
}
let p = pdfcraft_model::pages(doc).into_iter().nth(page).ok_or(EditError::NoSuchPage(page))?;
crate::check_on_top(doc, &p)?;
let res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let mut xobjects = res.get(b"XObject").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
let name = (1..=xobjects.len().saturating_add(1))
.map(|n| format!("PCPaste{n}").into_bytes())
.find(|n| xobjects.get(n).is_none())
.ok_or_else(|| EditError::Invalid("the page has too many resources".into()))?;
// The copy as Form artwork, to draw inline as Ungroup does: its resources merge into the
// page's, and patterns move with it. The form itself is never added to the document.
let number = |v: f64| Object::Real(v);
let mut form = Dict::new();
form.set(b"Type".to_vec(), Object::name("XObject"));
form.set(b"Subtype".to_vec(), Object::name("Form"));
// Nothing of the copy is clipped: its glyphs and strokes may reach past its objects' boxes.
form.set(b"BBox".to_vec(), Object::Array([-MAX_COORDINATE, -MAX_COORDINATE, MAX_COORDINATE, MAX_COORDINATE].map(number).to_vec()));
form.set(b"Matrix".to_vec(), Object::Array(Matrix::translate(offset[0], offset[1]).0.map(number).to_vec()));
form.set(b"Resources".to_vec(), Object::Dict(copied.resources.clone()));
xobjects.set(name.clone(), Object::Stream(Stream::from_raw(form, copied.content.clone())));
let mut with_copy = res;
with_copy.set(b"XObject".to_vec(), Object::Dict(xobjects));
let space = page_blend_space(doc, &p.dict);
// Drawn by a `Do` of its own: not translucent, and in no path or text object.
let draw = [Op::new("Do", vec![Object::Name(name.clone())])];
let mut budget = pdfcraft_cos::DecodeBudget::new(MAX_FORM_BYTES, 0);
let Ungrouped { ops, resources: mut merged, patterns, .. } =
ungroup_ops(doc, &with_copy, &draw, 0, space.as_ref(), Matrix::translate(offset[0], offset[1]), &mut budget).map_err(|r| match r.error {
EditError::Invalid(why) => {
EditError::Invalid(why.replacen("this artwork can't be ungrouped", "the copied objects can't be pasted here", 1))
}
other => other,
})?;
let mut work = doc.clone();
if let Some(mut x) = merged.get(b"XObject").and_then(Object::as_dict).cloned() {
x.remove(&name);
merged.set(b"XObject".to_vec(), Object::Dict(x));
}
if !patterns.is_empty() {
let mut table = merged.get(b"Pattern").map(|x| work.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
for (key, pattern) in patterns {
table.set(key, Object::Ref(work.add(pattern)));
}
merged.set(b"Pattern".to_vec(), Object::Dict(table));
}
work.update_dict(p.obj, |d| d.set(b"Resources".to_vec(), Object::Dict(merged)))?;
crate::place_tagged(&mut work, &p, "Pasted", serialize_ops(&ops), false)?;
// Keep the page's objects selectable together, as a move does.
editable_objects(&work, page)?;
*doc = work;
Ok(())
}
#[cfg(test)]
#[path = "clip_tests.rs"]
mod tests;
crates/edit/src/objects/clip_tests.rs
+228
View File
@@ -0,0 +1,228 @@
use std::sync::Arc;
use pdfcraft_cos::{Document, SaveOptions, write_incremental};
use super::*;
/// A document of `objs` (object 1 the catalog), numbered in order.
fn pdf(objs: &[String]) -> Document {
let mut out = b"%PDF-1.7\n".to_vec();
let mut offs = Vec::new();
for (i, o) in objs.iter().enumerate() {
offs.push(out.len());
out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes());
}
let x = out.len();
out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes());
for o in offs {
out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes());
}
out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{x}\n%%EOF\n", objs.len() + 1).as_bytes());
Document::open(Arc::new(out)).unwrap()
}
fn stream(content: &str) -> String {
format!("<< /Length {} >>\nstream\n{content}\nendstream", content.len())
}
/// Two 300 × 300 pages. The first draws `source` with Helvetica as /F1, a 4 × 2 RGB image /Im0,
/// a half-opaque ExtGState /GS0 and Courier as /F2; the second draws `target` with Courier
/// under the same name /F1, so a paste from the first must rename the font it brings.
fn two_pages(source: &str, target: &str) -> Document {
pdf(&[
"<< /Type /Catalog /Pages 2 0 R >>".into(),
"<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 /MediaBox [0 0 300 300] >>".into(),
"<< /Type /Page /Parent 2 0 R /Contents 5 0 R /Resources << /Font << /F1 7 0 R /F2 8 0 R >> /XObject << /Im0 9 0 R >> /ExtGState << /GS0 << /ca 0.5 /CA 0.5 >> >> >> >>"
.into(),
"<< /Type /Page /Parent 2 0 R /Contents 6 0 R /Resources << /Font << /F1 8 0 R >> >> >>".into(),
stream(source),
stream(target),
"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".into(),
"<< /Type /Font /Subtype /Type1 /BaseFont /Courier /Encoding /WinAnsiEncoding >>".into(),
"<< /Type /XObject /Subtype /Image /Width 4 /Height 2 /ColorSpace /DeviceRGB /BitsPerComponent 8 /Length 24 >>\nstream\n\u{7f}\u{0}\u{0}\u{0}\u{7f}\u{0}\u{0}\u{0}\u{7f}\u{7f}\u{7f}\u{0}\u{0}\u{7f}\u{7f}\u{7f}\u{0}\u{7f}@@@\u{0}\u{0}\u{0}\nendstream".into(),
])
}
/// What the page state, the copied paragraph and picture and the objects left out are drawn with.
const SOURCE: &str = "q 1 0 0 1 10 10 cm 0 0 270 270 re W n 0 0 1 rg /GS0 gs \
BT /F2 10 Tf 200 20 Td (Skip) Tj ET \
BT /F1 24 Tf 1.5 Tc 20 170 Td (Copied) Tj 0 -30 Td (Too) Tj ET \
q 1 0 0 rg 150 230 100 40 re f Q \
q 80 0 0 40 20 60 cm /Im0 Do Q Q \
BT /F1 12 Tf 200 250 Td (After) Tj ET";
fn target(doc: &Document, page: usize, want: impl Fn(&EditableObject) -> bool) -> ObjectTarget {
editable_objects(doc, page).unwrap().into_iter().find(|o| want(o)).unwrap().target
}
fn render(doc: &Document, page: usize) -> pdfcraft_render::RenderedPage {
let bytes = write_incremental(doc, &SaveOptions::default()).unwrap();
let mut renderer = pdfcraft_render::PageRenderer::new(Arc::new(bytes), pdfcraft_render::RenderConfig::default());
renderer.render(pdfcraft_render::RenderRequest { page, kind: pdfcraft_render::RequestKind::Pixels, scale: 1.0, ..Default::default() })
}
/// The pixels of `page` in user-space box `r` (a page 300 high at scale 1), shifted by `by`.
fn pixels(page: &pdfcraft_render::RenderedPage, r: [f64; 4], by: [f64; 2]) -> Vec<u8> {
let (x0, x1) = ((r[0] + by[0]) as u32, (r[2] + by[0]) as u32);
let (y0, y1) = ((300.0 - r[3] - by[1]) as u32, (300.0 - r[1] - by[1]) as u32);
let mut out = Vec::new();
for y in y0..y1 {
for x in x0..x1 {
let i = ((y * page.width + x) * 4) as usize;
out.extend_from_slice(&page.rgba[i..i + 4]);
}
}
out
}
/// The paragraph "Copied" / "Too" (one or two of them, as the lines group) and the picture.
fn copied_paragraph_and_picture(doc: &Document) -> CopiedObjects {
let targets: Vec<_> = editable_objects(doc, 0)
.unwrap()
.into_iter()
.filter(|o| o.target.kind == ObjectKind::Image || o.text.as_deref().is_some_and(|t| t.contains("Copied") || t.contains("Too")))
.map(|o| o.target)
.collect();
copy_objects(doc, 0, &targets).unwrap()
}
#[test]
fn a_copy_takes_the_selected_objects_their_state_and_only_the_resources_they_use() {
let doc = two_pages(SOURCE, "");
let copied = copied_paragraph_and_picture(&doc);
let content = String::from_utf8_lossy(&copied.content).into_owned();
assert!(content.contains("(Copied)") && content.contains("(Too)") && content.contains("/Im0 Do"), "{content}");
// The page's clip and its colour and transparency still apply; nothing else is painted.
assert!(content.contains("W\nn") && content.contains("/GS0 gs") && content.contains("rg"), "{content}");
assert!(!content.contains("Skip") && !content.contains("After") && !content.contains("\nf\n"), "{content}");
let names = |cat: &[u8]| copied.resources.get(cat).and_then(Object::as_dict).map(|d| d.iter().map(|(k, _)| k.clone()).collect::<Vec<_>>());
assert_eq!(names(b"Font"), Some(vec![b"F1".to_vec(), b"F2".to_vec()]), "the font of the skipped text stays selected for what follows");
assert_eq!(names(b"XObject"), Some(vec![b"Im0".to_vec()]));
assert_eq!(names(b"ExtGState"), Some(vec![b"GS0".to_vec()]));
assert!(copied.text().contains("Copied") && copied.text().contains("Too") && !copied.text().contains("Skip"), "{}", copied.text());
assert_eq!(pdfcraft_content::parse(&copied.content).ops.iter().filter(|o| o.is("q")).count(), 3, "balanced: {content}");
}
#[test]
fn pasted_objects_draw_the_same_on_another_page_under_renamed_resources() {
let mut doc = two_pages(SOURCE, "BT /F1 10 Tf 5 5 Td (x) Tj ET");
let copied = copied_paragraph_and_picture(&doc);
paste_objects(&mut doc, 1, &copied, [0.0, 0.0]).unwrap();
let (source, pasted) = (render(&doc, 0), render(&doc, 1));
let area = [copied.rect[0] - 4.0, copied.rect[1] - 4.0, copied.rect[2] + 4.0, copied.rect[3] + 4.0];
let (before, after) = (pixels(&source, area, [0.0, 0.0]), pixels(&pasted, area, [0.0, 0.0]));
assert!(before.chunks(4).any(|p| p != [255, 255, 255, 255]), "the copied objects are drawn");
assert_eq!(before, after, "the paste draws the same pixels");
let red = pixels(&pasted, [160.0, 240.0, 260.0, 280.0], [0.0, 0.0]);
assert!(red.chunks(4).all(|p| p == [255, 255, 255, 255]), "what wasn't selected isn't pasted");
// Its own /F1 (Courier) is left alone; the copy's Helvetica comes under another name.
let fonts =
pdfcraft_model::pages(&doc)[1].dict.get(b"Resources").map(|r| doc.resolve(r)).unwrap().as_dict().unwrap().get(b"Font").cloned().unwrap();
let fonts = doc.resolve(&fonts).as_dict().cloned().unwrap();
assert_eq!(fonts.get(b"F1"), Some(&Object::Ref(pdfcraft_cos::ObjRef::new(8, 0))));
assert!(fonts.iter().any(|(k, v)| k != b"F1" && *v == Object::Ref(pdfcraft_cos::ObjRef::new(7, 0))), "{fonts:?}");
let texts: Vec<_> = editable_objects(&doc, 1).unwrap().into_iter().filter_map(|o| o.text).collect();
assert!(texts.iter().any(|t| t.contains("Copied")) && !texts.iter().any(|t| t.contains("Skip")), "{texts:?}");
assert_eq!(crate::images::page_images(&doc, 1).unwrap().len(), 1);
}
#[test]
fn a_paste_onto_its_own_page_moves_by_the_offset_and_draws_the_same() {
let mut doc = two_pages(SOURCE, "");
let copied = copied_paragraph_and_picture(&doc);
let before = render(&doc, 0);
paste_objects(&mut doc, 0, &copied, [150.0, 0.0]).unwrap();
let after = render(&doc, 0);
let area = [copied.rect[0] - 2.0, copied.rect[1] - 2.0, copied.rect[2] + 2.0, copied.rect[3] + 2.0];
assert!(area[2] + 150.0 < 300.0, "{area:?}");
// The copy (and the clip it is drawn in) lands 150 pt to the right, the original stays.
// Antialiasing elsewhere on the page may round an edge pixel one step differently.
let (moved, original) = (pixels(&after, area, [150.0, 0.0]), pixels(&before, area, [0.0, 0.0]));
assert_eq!(moved.len(), original.len());
assert!(original.iter().any(|v| *v != 255), "the copied objects are drawn");
assert!(moved.iter().zip(&original).all(|(a, b)| a.abs_diff(*b) <= 2), "the paste draws the same pixels, moved");
assert_eq!(pixels(&after, area, [0.0, 0.0]), original);
// Within the document and the page, the resources are the page's own, under their names.
let res = pdfcraft_model::pages(&doc)[0].dict.get(b"Resources").map(|r| doc.resolve(r)).unwrap();
let fonts = res.as_dict().unwrap().get(b"Font").and_then(Object::as_dict).cloned().unwrap();
assert_eq!(fonts.len(), 2, "{fonts:?}");
assert_eq!(editable_objects(&doc, 0).unwrap().iter().filter(|o| o.target.kind == ObjectKind::Image).count(), 2);
}
/// Artwork without resources of its own takes the names of the page drawing it. Pasted onto a
/// page whose names differ, artwork that names nothing still draws the same; artwork that names
/// a font would find another one, so the paste is refused.
#[test]
fn artwork_inheriting_resources_pastes_only_when_it_names_none() {
for (art, pastes) in [("0 0 1 rg 0 0 60 30 re f", true), ("BT /F1 9 Tf (n) Tj ET", false)] {
let mut doc = pdf(&[
"<< /Type /Catalog /Pages 2 0 R >>".into(),
"<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 /MediaBox [0 0 300 300] >>".into(),
"<< /Type /Page /Parent 2 0 R /Contents 5 0 R /Resources << /Font << /F1 7 0 R >> /XObject << /Art 8 0 R >> >> >>".into(),
"<< /Type /Page /Parent 2 0 R /Contents 6 0 R /Resources << /Font << /F1 9 0 R >> >> >>".into(),
stream("BT /F1 12 Tf 20 250 Td (Title) Tj ET q 1 0 0 1 20 100 cm /Art Do Q"),
stream(""),
"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>".into(),
format!("<< /Type /XObject /Subtype /Form /BBox [0 0 60 30] /Length {} >>\nstream\n{art}\nendstream", art.len()),
"<< /Type /Font /Subtype /Type1 /BaseFont /Courier >>".into(),
]);
let all: Vec<_> = editable_objects(&doc, 0).unwrap().into_iter().map(|o| o.target).collect();
let copied = copy_objects(&doc, 0, &all).unwrap();
let pasted = paste_objects(&mut doc, 1, &copied, [0.0, 0.0]);
assert_eq!(pasted.is_ok(), pastes, "{art}: {pasted:?}");
if let Err(EditError::Invalid(why)) = pasted {
assert!(why.starts_with("the copied objects can't be pasted here") && why.contains("uses its resources"), "{why}");
}
}
}
#[test]
fn added_content_is_copied_with_everything_its_stream_paints() {
let mut doc = two_pages(SOURCE, "");
let item = crate::Content::Text(crate::AddedText {
rect: [20.0, 20.0, 200.0, 60.0],
text: "Added box".into(),
size: 12.0,
..crate::AddedText::default()
});
crate::add_content(&mut doc, 0, &item).unwrap();
let added = target(&doc, 0, |o| o.target.kind == ObjectKind::Added);
let copied = copy_objects(&doc, 0, &[added]).unwrap();
assert_eq!(copied.text(), "Added box");
paste_objects(&mut doc, 1, &copied, [0.0, 0.0]).unwrap();
let texts: Vec<_> = editable_objects(&doc, 1).unwrap().into_iter().filter_map(|o| o.text).collect();
assert_eq!(texts, ["Added box"], "pasted as the page's own text");
assert!(crate::list_added(&doc).iter().all(|a| a.page == 0));
}
#[test]
fn copying_and_pasting_refuse_what_they_cannot_do_and_change_nothing() {
let doc = two_pages(SOURCE, "");
let text = target(&doc, 0, |o| o.text.as_deref().is_some_and(|t| t.contains("Copied")));
assert!(copy_objects(&doc, 0, &[]).is_err());
assert!(copy_objects(&doc, 0, &[text, text]).is_err(), "selected twice");
assert!(copy_objects(&doc, 0, &[ObjectTarget { kind: ObjectKind::Image, index: 9 }]).is_err(), "stale");
assert!(copy_objects(&doc, 5, &[text]).is_err(), "no such page");
let copied = copy_objects(&doc, 0, &[text]).unwrap();
for offset in [[f64::NAN, 0.0], [f64::INFINITY, 0.0], [2e9, 0.0]] {
let mut work = doc.clone();
assert!(paste_objects(&mut work, 1, &copied, offset).is_err(), "{offset:?}");
assert!(!work.is_modified(), "a refused paste changes nothing");
}
assert!(paste_objects(&mut doc.clone(), 7, &copied, [0.0, 0.0]).is_err());
// Text drawn as a clipping path before the selection would clip it: refused, not dropped.
let clipped = two_pages("BT /F1 20 Tf 7 Tr 20 20 Td (Mask) Tj ET 0 Tr BT /F1 20 Tf 20 200 Td (Shown) Tj ET", "");
let shown = target(&clipped, 0, |o| o.text.as_deref() == Some("Shown"));
assert!(copy_objects(&clipped, 0, &[shown]).is_err());
// Stray `Q`, `ET` and an unfinished path or text object: the copy is still balanced.
let stray = two_pages("Q Q ET 10 10 m 20 20 l BT /F1 20 Tf 20 200 Td (Stray) Tj", "");
let shown = target(&stray, 0, |o| o.text.as_deref() == Some("Stray"));
let copied = copy_objects(&stray, 0, &[shown]).unwrap();
let ops = pdfcraft_content::parse(&copied.content).ops;
assert_eq!(ops.iter().filter(|o| o.is("q")).count(), ops.iter().filter(|o| o.is("Q")).count());
assert_eq!(ops.iter().filter(|o| o.is("BT")).count(), ops.iter().filter(|o| o.is("ET")).count());
let mut work = stray.clone();
paste_objects(&mut work, 1, &copied, [0.0, 0.0]).unwrap();
}
crates/edit/src/place.rs
+15 -9
View File
@@ -1,5 +1,5 @@
use super::marks::{MarkKind, matrix_number, n};
use super::{Dict, Document, EditError, ObjRef, Object, Stream, literal, win_ansi};
use super::marks::{MarkKind, matrix_number};
use super::{Dict, Document, EditError, ObjRef, Object, Stream};
pub(crate) fn page_list(doc: &Document) -> Vec<pdfcraft_model::Page> {
pdfcraft_model::pages(doc)
@@ -68,6 +68,19 @@ pub(crate) fn free_font_name(doc: &Document, fonts: &Dict, preferred: &str, base
(1..=fonts.len().saturating_add(1)).map(|k| format!("{preferred}_{k}")).find(|name| usable(name)).unwrap_or_else(|| preferred.to_string())
}
/// Add `fonts` (resource name, font) to the page's own resources.
pub(crate) fn add_page_fonts(doc: &mut Document, page: &pdfcraft_model::Page, fonts: &Dict) -> Result<(), EditError> {
let mut res = page.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let mut all = res.get(b"Font").map(|f| doc.resolve(f)).and_then(|f| f.as_dict().cloned()).unwrap_or_default();
for (name, font) in fonts.iter() {
all.set(name.clone(), font.clone());
}
res.set(b"Font".to_vec(), Object::Dict(all));
// The page gets its own (possibly copied) resources; shared dictionaries stay untouched.
doc.update_dict(page.obj, |d| d.set(b"Resources".to_vec(), Object::Dict(res)))?;
Ok(())
}
/// Add the standard `faces` (resource name, base font) and an opacity state to the page's own
/// resources.
pub(crate) fn add_page_resources(
@@ -575,10 +588,3 @@ pub(crate) fn undrawable(c: char) -> EditError {
pub fn undrawable_message(c: char) -> String {
format!("the standard fonts can't draw \"{c}\" (U+{:04X}); only Western European characters can be added as text for now", u32::from(c))
}
pub(crate) fn text_op(x: f64, y: f64, text: &str) -> Vec<u8> {
let mut v = format!("1 0 0 1 {} {} Tm ", n(x), n(y)).into_bytes();
v.extend(literal(&win_ansi(text)));
v.extend_from_slice(b" Tj\n");
v
}
crates/edit/src/rules.rs
+2 -2
View File
@@ -1,7 +1,7 @@
//! A page's ruled lines: the horizontal and vertical strokes (and thin filled bars) its content
//! draws, so exports can tell a ruled table from text that merely lines up (#5149).
use pdfcraft_content::{Matrix, Pieces};
use pdfcraft_content::Matrix;
use pdfcraft_cos::Document;
use crate::EditError;
@@ -26,7 +26,7 @@ pub fn page_rules(doc: &Document, page: usize) -> Result<Vec<[f64; 4]>, EditErro
let pages = pdfcraft_model::pages(doc);
let p = pages.get(page).ok_or(EditError::NoSuchPage(page))?;
let streams = crate::text::content_streams(doc, &p.dict)?;
let pieces = Pieces::join(&streams.iter().map(|(_, d)| d.as_slice()).collect::<Vec<_>>());
let pieces = crate::text::joined(&streams);
let ops = pieces.parse().ops;
let mut out: Vec<[f64; 4]> = Vec::new();
crates/edit/src/tests.rs
+2555
View File
File diff suppressed because it is too large. Load diff
crates/edit/src/text.rs
+15 -2
View File
@@ -31,6 +31,7 @@ pub use session::TextSession;
mod blocks;
mod editor;
mod fallback;
pub mod forms;
mod interp;
mod lines;
mod rewrite;
@@ -42,8 +43,8 @@ pub use blocks::{
BlockKey, BlockStyle, RangeStyle, ReadingBlocks, TextBlock, block_overlapping, clear_block, reading_blocks, reading_blocks_checked,
replace_block, text_blocks,
};
use editor::layout_draft;
pub use editor::{TextEditLayout, TextEditorFont, register_text_font, text_edit_layout, text_editor_fonts};
use editor::{builtin_thai_choice, layout_draft};
#[cfg(test)]
pub(crate) use fallback::SOURCE_SPACE_CALLS;
use fallback::{
@@ -62,9 +63,9 @@ pub(crate) use lines::{
pub use lines::{MAX_FORM_DEPTH, MAX_FORM_VISITS, glyph_text_lines, text_lines};
pub use rewrite::{LineEdit, replace_line, rewrite_block};
use rewrite::{coincident_ops, rewrite_block_from};
pub(crate) use rewrite::{paragraph_ops, reveal_ops, selectable_blocks, show_starts, translate_blocks};
#[cfg(test)]
pub(crate) use rewrite::{replace_line_with_limit, rewrite_block_with_limit};
pub(crate) use rewrite::{reveal_ops, selectable_blocks, translate_blocks};
/// One line of existing text.
#[derive(Clone, Debug, PartialEq)]
@@ -276,6 +277,18 @@ mod tests {
assert_eq!(wrapped("一行目　二行目", 4.0), ["一行目　", "二行目"]);
}
#[test]
fn thai_text_wraps_between_dictionary_words_and_keeps_marks_on_their_word() {
// Without word breaks the whole run is one word and overflows a single row.
assert_eq!(wrapped("ภาษาไทยไม่มีช่องว่าง", 8.0), ["ภาษาไทย", "ไม่มี", "ช่องว่าง"]);
// Mai Yamok stays with the word it repeats; pieces join without a space, words keep theirs.
assert_eq!(wrapped("เด็กๆชอบกิน ขนม", 6.0), ["เด็กๆ", "ชอบกิน", "ขนม"]);
assert_eq!(wrapped("ไฟล์ PDF ประปาส่วนภูมิภาค", 9.0), ["ไฟล์ PDF", "ประปาส่วน", "ภูมิภาค"]);
// A wrap between Thai words consumed no separator, so a rejoin puts back no space.
let rows = wrap("ภาษาไทยไม่มีช่องว่าง", 8.0, |s| s.chars().count() as f64);
assert_eq!(rows.iter().map(|(_, soft, gap)| (*soft, *gap)).collect::<Vec<_>>(), [(false, 0), (true, 0), (true, 0)]);
}
#[test]
fn fallback_face_moves_on_when_the_best_face_lacks_a_character() {
static JPAN: &[&str] = &["Jpan"];
crates/edit/src/text/blocks.rs
+8 -1
View File
@@ -37,6 +37,13 @@ pub struct TextBlock {
}
impl TextBlock {
/// The one size a rewrite and the editor set an OCR phrase in, in place of each word's own
/// (#3313): the block's size, which for invisible text is its words' median. `None` for
/// visible text, whose runs keep their sizes.
pub fn phrase_size(&self) -> Option<f64> {
(self.invisible && self.size.is_finite() && self.size > 0.0).then_some(self.size)
}
/// Preserve styles outside the changed ranges; inserted characters inherit the surrounding
/// source run. The texts are aligned word by word, so words left untouched between two edits
/// keep their own runs (#2198); past [`MAX_CHANGED_WORDS`] the draft counts as one change.
@@ -400,7 +407,7 @@ pub fn replace_block(doc: &mut Document, page: usize, block: usize, text: &str)
pub struct BlockStyle {
/// Disjoint UTF-8 ranges in the replacement, each preserving unspecified source style.
pub ranges: Vec<RangeStyle>,
/// Explicit page font resource (from register_text_font or the page font inventory).
/// Explicit page font resource or bundled document-font choice from the page font inventory.
pub resource: Option<String>,
/// A standard font family, bold, italic.
pub family: Option<(crate::added::Family, bool, bool)>,
crates/edit/src/text/editor.rs
+37 -4
View File
@@ -2,15 +2,30 @@ use super::blocks::{BlockStyle, TextBlock};
use super::fallback::MAX_TYPE3_GLYPHS;
use super::lines::{page_dict, text_lines};
use super::rewrite::rewrite_block_from;
use super::{DecodeBudget, Document, EditError, Metrics, Object, TextLine, TextSession};
use super::{DecodeBudget, Dict, Document, EditError, Metrics, Object, TextLine, TextSession};
/// An embedded sfnt source face, read only for the inline editor. Bytes stay in memory and
/// are never installed, copied into assets, or used as a font for another document.
/// An embedded source font or an approved bundled document-font choice.
/// Source bytes stay in memory and are never installed or copied into another document.
#[derive(Clone, Debug)]
pub struct TextEditorFont {
pub resource: String,
pub face: String,
pub bytes: Vec<u8>,
/// A selectable replacement, rather than a source/matching preview face. Merely listing
/// this choice must not reload an open editor's font atlas.
pub choice_only: bool,
}
/// Reserved document-font choice for explicitly replacing text with the bundled Thai face.
/// This is a selector, never a font resource written into a PDF.
const BUILTIN_THAI_FONT: &str = "builtin:thai";
pub(super) fn builtin_thai_choice(fonts: &Dict) -> String {
let mut choice = BUILTIN_THAI_FONT.to_owned();
while fonts.contains(choice.as_bytes()) {
choice.push('_');
}
choice
}
pub fn text_editor_fonts(doc: &Document, page: usize) -> Result<Vec<TextEditorFont>, EditError> {
@@ -32,6 +47,7 @@ pub fn text_editor_fonts(doc: &Document, page: usize) -> Result<Vec<TextEditorFo
resource: String::from_utf8_lossy(name).into_owned(),
face: Metrics::from_dict(doc, &font).base_font.rsplit('+').next().unwrap_or_default().to_owned(),
bytes,
choice_only: false,
});
}
// The approved full face follows all source subsets in the editor's font stack. It can
@@ -43,9 +59,26 @@ pub fn text_editor_fonts(doc: &Document, page: usize) -> Result<Vec<TextEditorFo
continue;
}
budget -= full.bytes.len();
result.push(TextEditorFont { resource: format!("PCApproved{}", full.name()), face: face.clone(), bytes: full.bytes.to_vec() });
result.push(TextEditorFont {
resource: format!("PCApproved{}", full.name()),
face: face.clone(),
bytes: full.bytes.to_vec(),
choice_only: false,
});
}
}
// A missing source family must not make the bundled Thai replacement undiscoverable.
// Offering it does not change the source; callers explicitly select this choice to replace it.
if let Some(thai) = pdfcraft_fonts::document_thai_font()
&& thai.bytes.len() <= budget
{
result.push(TextEditorFont {
resource: builtin_thai_choice(&fonts),
face: thai.family.into(),
bytes: thai.bytes.to_vec(),
choice_only: true,
});
}
Ok(result)
}
crates/edit/src/text/fallback.rs
+4 -4
View File
@@ -24,13 +24,13 @@ const HEBREW_FONT_NAME: &str = "PCHebrew";
/// The `/Name` of the Type 3 fonts generated from the bundled Latin (Inter) face.
const LATIN_FONT_NAME: &str = "PCLatin";
/// Whether `font` is one of those generated fonts (Latin, Japanese or Hebrew): a replacement longer
/// Whether `font` is one of those generated fonts (Latin, Japanese, Hebrew or Thai): a replacement longer
/// than one of them is split across several, which read back as one line (see [`same_font`]).
pub(super) fn is_type3_fallback(font: &Dict) -> bool {
font.name(b"Subtype") == Some(b"Type3")
&& font
.name(b"Name")
.is_some_and(|n| n == JAPANESE_FONT_NAME.as_bytes() || n == HEBREW_FONT_NAME.as_bytes() || n == LATIN_FONT_NAME.as_bytes())
&& font.name(b"Name").is_some_and(|n| {
n == JAPANESE_FONT_NAME.as_bytes() || n == HEBREW_FONT_NAME.as_bytes() || n == LATIN_FONT_NAME.as_bytes() || n == b"PCThai"
})
}
/// The generated font's `/Name` and its resource prefix: the bundled Latin face, Hebrew, or the
crates/edit/src/text/forms.rs
+283
View File
@@ -0,0 +1,283 @@
//! Make text in ordinary Form XObjects available to the existing paragraph editor.
//!
//! This replaces each supported placement's `Do`, never the shared form object. The same
//! source glyph operators become page content; no cover rectangle or replacement text box is
//! added. Unsupported compositing and malformed streams stay intact.
use pdfcraft_content::Matrix;
use pdfcraft_cos::{Dict, Document, Object};
use super::{ContentStream, Read, content_streams, joined, page_dict, rewritten_contents, splice};
use crate::{EditError, ungroup};
/// Result of preparing one page. Refused groups remain in the page, with an actionable reason.
#[derive(Clone, Debug, Default)]
pub struct Preparation {
pub expanded: usize,
pub refused: Vec<String>,
pub repaired_fonts: usize,
}
/// How many levels of nested groups preparation opens on one page. Text nested deeper stays
/// grouped and is reported in [`Preparation::refused`].
const MAX_LEVELS: usize = 8;
/// What preparing one page may decode, and add to its content, in all.
const MAX_PREPARED_BYTES: usize = 16 << 20;
/// What one pass of [`prepare`] inlined, as positions in the next pass's parse of the page.
struct Inlined {
/// How many operators that parse has; with any other count the positions are wrong.
ops: usize,
/// Where each inlined form's operators are: the placements in them are new.
ranges: Vec<std::ops::Range<usize>>,
/// Earlier placements refused only because their resources needed renaming
/// ([`crate::ungroup::Refusal::renaming`]), each with the reason it was given.
waiting: Vec<(usize, String)>,
}
/// Prepare grouped source text atomically, within eight levels and 64 placements per page, and
/// repair the page fonts' damaged legacy Thai maps (`repair_thai_fonts`), grouped or not.
/// Shared forms and other pages are untouched. Preparation is a distinct undoable engine edit.
pub fn prepare(doc: &mut Document, page: usize) -> Result<Preparation, EditError> {
let mut next = doc.clone();
let mut result = Preparation::default();
let mut bytes_left = MAX_PREPARED_BYTES;
let mut visits = 0usize;
let mut decode = pdfcraft_cos::DecodeBudget::new(MAX_PREPARED_BYTES, 0);
// Forms found to be artwork, or refused for a reason in the form itself (`Refusal::intrinsic`):
// a placement of one that an expanded group adds is left as it is without being read or
// counted, so logos drawn inside many text groups don't use up the limit. A form refused only
// where it is drawn is read again where an expanded group draws it.
let mut settled = std::collections::HashSet::new();
// What the previous pass inlined. Only the placements inside it are new. Every other placement
// was read and counted on an earlier pass, so a group refused where it is drawn counts once.
// Reading one again gives the same answer (an inlined form restores the graphics state it
// changes, and the decode budget only shrinks), except for a group refused because its
// resources needed renaming: names merged in for other groups since can make that
// unnecessary. Such a group is read again, uncounted, once its resource names say so.
let mut inlined: Option<Inlined> = None;
// One pass opens one level of every placement; the last pass only reports what is deeper.
for pass in 0..=MAX_LEVELS {
let p = page_dict(&next, page)?;
let streams = strict_streams(&next, &p.dict)?;
let pieces = joined(&streams);
let parsed = pieces.parse();
if parsed.skipped > 0 {
return Err(EditError::Invalid("the page has unreadable content; grouped text was left unchanged".into()));
}
let ops = parsed.ops;
// `None` reads every placement: on the first pass, and if the page didn't parse back to
// the operators written, where the positions can't be trusted. Reading one again can only
// count it twice; skipping one could leave its text grouped without a reason.
let (new, mut waiting) = match inlined.take() {
Some(i) if i.ops == ops.len() => (Some(i.ranges), i.waiting),
_ => (None, Vec::new()),
};
let mut still_waiting = Vec::new();
let resources = p.dict.get(b"Resources").map(|r| next.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let space = ungroup::page_blend_space(&next, &p.dict);
let images: std::collections::BTreeMap<_, _> =
crate::page_images(&next, page)?.into_iter().filter(|i| i.is_form).map(|i| (i.op, i)).collect();
let mut replacements = std::collections::BTreeMap::new();
let mut merged = resources;
let (mut refused_here, mut expanded_here) = (std::collections::HashSet::new(), std::collections::HashSet::new());
for (at, op) in ops.iter().enumerate() {
let Some(image) = images.get(&at) else { continue };
if image.object.is_some_and(|o| settled.contains(&o)) {
continue;
}
// An earlier placement is read again only when it was refused for a rename that the
// names merged since have made unnecessary. The last pass opens nothing, so there it
// keeps its reason.
let earlier = new.as_ref().is_some_and(|ranges| !ranges.iter().any(|r| r.contains(&at)));
if earlier {
let Some(w) = waiting.iter().position(|(i, _)| *i == at) else { continue };
let (_, reason) = waiting.swap_remove(w);
// The name exactly as the `Do` writes it: a name need not be UTF-8, and
// `image.name` is only its display form.
if pass == MAX_LEVELS || !op.last_name().is_some_and(|name| ungroup::merges_without_renaming(&next, &merged, name)) {
still_waiting.push((at, reason));
continue;
}
// What reading it again finds replaces the reason it was given.
if let Some(i) = result.refused.iter().position(|r| *r == reason) {
result.refused.remove(i);
}
}
if !earlier {
visits = visits.saturating_add(1);
if visits > 64 {
return Err(EditError::Invalid("too many grouped text placements on this page (limit 64); the page was left unchanged".into()));
}
}
match ungroup::ungroup_ops(&next, &merged, &ops, at, space.as_ref(), Matrix(image.matrix), &mut decode) {
Ok(ungroup::Ungrouped { ops: inline, resources: mut res, patterns, shows_text }) => {
// Vector and image artwork needs no preparation. Nested text groups are
// visited on the next pass, with their inherited placement and clip in force.
match shows_text {
Some(true) => {}
Some(false) => {
settled.extend(image.object);
continue;
}
// Whether it has text can't be known: it is left as it is, and said.
None => {
result.refused.push(format!("{}: it draws recursive forms, or nested content that can't be checked", image.name));
continue;
}
}
if pass == MAX_LEVELS {
result.refused.push(format!("{}: its text is grouped more than {MAX_LEVELS} levels deep", image.name));
continue;
}
let data_len = pdfcraft_content::serialize_ops(&inline).len();
bytes_left = bytes_left.checked_sub(data_len).ok_or_else(|| {
EditError::Invalid("grouped page content exceeds the 16 MiB editing budget; the page was left unchanged".into())
})?;
if !patterns.is_empty() {
let mut table = res.get(b"Pattern").map(|r| next.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
for (name, value) in patterns {
table.set(name, Object::Ref(next.add(value)));
}
res.set(b"Pattern".to_vec(), Object::Dict(table));
}
merged = res;
replacements.insert(at, inline);
result.expanded += 1;
expanded_here.extend(image.object);
}
Err(refusal) => {
let reason = format!("{}: {}", image.name, refusal.error);
if refusal.renaming {
still_waiting.push((at, reason.clone()));
}
if refusal.intrinsic {
refused_here.extend(image.object);
}
result.refused.push(reason);
}
}
}
settled.extend(refused_here.difference(&expanded_here).copied());
if replacements.is_empty() {
break;
}
// Where an operator lands in the next pass: each replaced `Do` before it becomes its
// inlined operators.
let moved = |i: usize| replacements.range(..i).fold(i, |n, (_, inline)| n.saturating_add(inline.len()).saturating_sub(1));
inlined = Some(Inlined {
ops: moved(ops.len()),
ranges: replacements.iter().map(|(at, inline)| moved(*at)..moved(*at).saturating_add(inline.len())).collect(),
waiting: still_waiting.into_iter().map(|(at, reason)| (moved(at), reason)).collect(),
});
let changed = splice(&pieces, &ops, |at| match replacements.remove(&at) {
Some(ops) => (ops, false),
None => (Vec::new(), true),
});
let contents = rewritten_contents(&mut next, &streams, changed)?;
next.update_dict(p.obj, |d| {
d.set(b"Contents".to_vec(), contents);
d.set(b"Resources".to_vec(), Object::Dict(merged));
})?;
}
result.refused.sort();
result.refused.dedup();
// The page's own damaged maps are repaired too, those of text that was never grouped
// included: its text reads wrong all the same.
result.repaired_fonts = repair_thai_fonts(&mut next, page)?;
if result.expanded > 0 || result.repaired_fonts > 0 {
for reason in &result.refused {
next.note_repair(format!("Grouped text left intact: {reason}"));
}
if result.expanded > 0 {
next.note_repair(format!(
"Prepared {} grouped text placement(s) on page {} for direct source-text editing",
result.expanded,
page.saturating_add(1)
));
}
*doc = next;
}
Ok(result)
}
/// Whether grouped artwork `image` on `page` (its index in `page_images`) shows text that
/// [`prepare`] would make editable, read as its first pass reads it, without changing anything:
/// a logo or a refused group is `false`. A group refused only because its resources need renaming
/// is `true`, as another group's names can still let [`prepare`] open it.
pub fn grouped_text_at(doc: &Document, page: usize, image: usize) -> Result<bool, EditError> {
let Some(img) = crate::page_images(doc, page)?.into_iter().nth(image).filter(|i| i.is_form) else { return Ok(false) };
let p = page_dict(doc, page)?;
let parsed = joined(&strict_streams(doc, &p.dict)?).parse();
if parsed.skipped > 0 {
return Ok(false);
}
let resources = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let space = ungroup::page_blend_space(doc, &p.dict);
let mut budget = pdfcraft_cos::DecodeBudget::new(MAX_PREPARED_BYTES, 0);
Ok(match ungroup::ungroup_ops(doc, &resources, &parsed.ops, img.op, space.as_ref(), Matrix(img.matrix), &mut budget) {
Ok(u) => u.shows_text == Some(true),
Err(r) => r.renaming,
})
}
/// A font is copied into this page: repairing the map must never alter a shared form/font.
fn repair_thai_fonts(doc: &mut Document, page: usize) -> Result<usize, EditError> {
let p = page_dict(doc, page)?;
let mut res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let mut fonts = res.get(b"Font").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let mut count = 0;
let mut candidates = 0usize;
for (name, value) in fonts.clone().iter() {
let Some(mut font) = doc.resolve(value).as_dict().cloned() else { continue };
if font.name(b"BaseFont").is_some_and(|n| String::from_utf8_lossy(n).contains("THSarabunPSK")) {
candidates += 1;
if candidates > 64 {
return Err(EditError::Invalid("too many legacy Thai font maps to repair safely; the page was left unchanged".into()));
}
} else {
continue;
}
let Some(map) = pdfcraft_fonts::pdf::recovered_thai_map(doc, &font) else { continue };
// Append corrective bfchar entries after the original entries so valid Latin mappings
// remain. The original map's accidental oversized range is still overridden for Thai.
let Some(Object::Stream(tu)) = font.get(b"ToUnicode").map(|r| doc.resolve(r)).as_deref().cloned() else { continue };
let old = pdfcraft_cos::DecodeBudget::new(1 << 20, 0).decode_strict(doc, &tu)?;
// After the last `endcmap`: a stream with more than one must not have an earlier block
// (or its oversized range) after the corrective entries.
let marker = b"endcmap";
let Some(at) = old.windows(marker.len()).rposition(|w| w == marker) else { continue };
let mut added = format!("\n{} beginbfchar\n", map.len());
for (code, ch) in map {
added.push_str(&format!("<{code:04x}> <{:04x}>\n", u32::from(ch)));
}
added.push_str("endbfchar\n");
let mut bytes = old.get(..at).unwrap_or_default().to_vec();
bytes.extend_from_slice(added.as_bytes());
bytes.extend_from_slice(old.get(at..).unwrap_or_default());
let tu = doc.add(Object::Stream(pdfcraft_cos::Stream::flate(Dict::new(), &bytes)));
font.set(b"ToUnicode".to_vec(), Object::Ref(tu));
let object = doc.add(Object::Dict(font));
fonts.set(name.clone(), Object::Ref(object));
count += 1;
}
if count > 0 {
res.set(b"Font".to_vec(), Object::Dict(fonts));
doc.update_dict(p.obj, |d| d.set(b"Resources".to_vec(), Object::Dict(res)))?;
doc.note_repair(format!(
"Recovered the damaged legacy THSarabunPSK Thai character map in {count} page font(s); original glyph programs were preserved"
));
}
Ok(count)
}
/// Editing cannot infer the state after a damaged page stream. Every piece must decode whole
/// before a placement is expanded, rather than treating an undecodable piece as empty.
fn strict_streams(doc: &Document, page: &Dict) -> Result<Vec<ContentStream>, EditError> {
let streams = content_streams(doc, page)?;
if streams.iter().any(|s| s.read != Read::Whole) {
return Err(EditError::Invalid("the page has a missing or unreadable content stream".into()));
}
Ok(streams)
}
crates/edit/src/text/rewrite.rs
+41 -15
View File
@@ -749,12 +749,13 @@ struct ParaStyle {
k: f64,
}
/// Spacing and scaling (text state), with the chosen values.
fn paragraph_style(o: &Origin, style: &BlockStyle) -> Result<(String, ParaStyle), EditError> {
/// Spacing and scaling (text state), with the chosen values. `phrase` is the size an OCR phrase
/// is drawn at when none is chosen, in place of its first word's own (#3313).
fn paragraph_style(o: &Origin, style: &BlockStyle, phrase: Option<f64>) -> Result<(String, ParaStyle), EditError> {
let (font_name, old_size) = o.state.font.clone().ok_or_else(|| EditError::Invalid("the paragraph has no font".into()))?;
let k = o.k.max(1e-6);
// The size in text space: points ÷ the text-to-user scale.
let size = style.size.map_or(old_size, |pt| (pt / k).max(0.1));
let size = style.size.or(phrase).map_or(old_size, |pt| (pt / k).max(0.1));
let mut state = o.state.clone();
if let Some(c) = style.char_spacing {
state.char_spacing = c / k;
@@ -762,6 +763,11 @@ fn paragraph_style(o: &Origin, style: &BlockStyle) -> Result<(String, ParaStyle)
if let Some(sc) = style.scale {
state.scale = (sc / 100.0).clamp(0.1, 10.0);
}
// At the phrase's size, the first word's text matrix still carries its own width: scale it
// back, so the line is as wide as the scan whatever size OCR guessed for that word.
if phrase.is_some() && style.size.is_none() && style.scale.is_none() {
state.scale = (state.scale * old_size / size).clamp(0.1, 10.0);
}
Ok((font_name, ParaStyle { state, size, old_size, k }))
}
@@ -842,7 +848,7 @@ pub(crate) fn rewrite_block_from(
let o = &first.origin;
let last = members.last().copied().unwrap_or(first);
let emit = &last.origin;
let (font_name, ParaStyle { state: o_state, size, old_size, k }) = paragraph_style(o, style)?;
let (font_name, ParaStyle { state: o_state, size, old_size, k }) = paragraph_style(o, style, b.phrase_size())?;
let source = source_font(doc, &fonts_res, &font_name, first, &flat, style);
let reuse = source.reuse;
// The chosen style also determines the real Japanese fallback outlines and advances.
@@ -1111,6 +1117,36 @@ fn vertical_encoding(doc: &Document, font: &Dict) -> Result<bool, EditError> {
Err(EditError::Invalid("the text encoding is nested too deeply".into()))
}
/// The text-showing operators of paragraphs `indexes` (from [`group_blocks`] of the page's
/// lines), as indexes among the page's joined operators. `editable`: refuse a line that can't be
/// edited (moving rewrites its operators; copying takes them as they are).
pub(crate) fn paragraph_ops(doc: &Document, page: usize, indexes: &[usize], editable: bool) -> Result<std::collections::BTreeSet<usize>, EditError> {
let mut selected = std::collections::BTreeSet::new();
if indexes.is_empty() {
return Ok(selected);
}
let lines = text_lines(doc, page)?;
let blocks = group_blocks(&lines);
for index in indexes {
let b = blocks.get(*index).ok_or_else(|| EditError::Invalid("a selected paragraph no longer exists".into()))?;
for line in &b.lines {
let l = lines.get(*line).ok_or_else(|| EditError::Invalid("a selected text line no longer exists".into()))?;
if editable {
l.ensure_editable()?;
}
selected.extend(l.ops.iter().copied());
}
}
Ok(selected)
}
/// Where each text-showing operator of `ops` (a page's joined operators, drawn with `resources`)
/// starts its glyphs: the text matrix there, and its rendering mode (`Tr`).
pub(crate) fn show_starts(doc: &Document, ops: &[Op], resources: &Dict) -> Result<HashMap<usize, (Matrix, i64)>, EditError> {
let shown = interpret(doc, ops, resources, &mut HashMap::new(), &mut Carry::new(), None, None)?;
Ok(shown.into_iter().map(|s| (s.op, (s.tm, s.state.render_mode))).collect())
}
/// Translate paragraphs together without re-encoding glyphs, changing fonts or reflowing.
/// Numeric TJ offsets and text rise temporarily move each selected show operation, then
/// restore both advances and rise. The PDF reader advances the original glyph bytes itself:
@@ -1121,17 +1157,7 @@ pub(crate) fn translate_blocks(doc: &mut Document, page: usize, indexes: &[usize
}
let p = page_dict(doc, page)?;
let streams = content_streams(doc, &p.dict)?;
let lines = text_lines(doc, page)?;
let blocks = group_blocks(&lines);
let mut selected = std::collections::BTreeSet::new();
for index in indexes {
let b = blocks.get(*index).ok_or_else(|| EditError::Invalid("a selected paragraph no longer exists".into()))?;
for line in &b.lines {
let l = lines.get(*line).ok_or_else(|| EditError::Invalid("a selected text line no longer exists".into()))?;
l.ensure_editable()?;
selected.extend(l.ops.iter().copied());
}
}
let selected = paragraph_ops(doc, page, indexes, true)?;
let res = p.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let fonts = res.get(b"Font").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()).unwrap_or_default();
let pieces = joined(&streams);
crates/edit/src/text_regions.rs
+4 -3
View File
@@ -2,9 +2,10 @@
use super::{
BlockStyle, CraftFont, Dict, Document, EditError, HashMap, LineEdit, MAX_TYPE3_GLYPHS, Matrix, Metrics, Object, Op, PdfString, Pieces,
RangeStyle, TextBlock, TextLine, Type3Glyph, coincident_ops, content_streams, fallback_face, japanese_glyph_from, needs_type3, no_fallback_font,
no_glyph_for, page_dict, paragraph_separator, reveal_ops, rewritten_contents, same_base_font, same_font, source_encodes, source_family,
source_program, source_show, source_space, source_width, splice, strip_subset_tag, type3_faces, type3_glyph, union, write_type3_glyphs,
RangeStyle, TextBlock, TextLine, Type3Glyph, builtin_thai_choice, coincident_ops, content_streams, fallback_face, japanese_glyph_from,
needs_type3, no_fallback_font, no_glyph_for, page_dict, paragraph_separator, reveal_ops, rewritten_contents, same_base_font, same_font,
source_encodes, source_family, source_program, source_show, source_space, source_width, splice, strip_subset_tag, type3_faces, type3_glyph,
union, write_type3_glyphs,
};
#[path = "text_regions/grouping.rs"]
crates/edit/src/text_regions/grouping.rs
+22
View File
@@ -398,6 +398,17 @@ fn group_layout(lines: &[TextLine], mixed: bool) -> Vec<TextBlock> {
for block in &mut blocks {
block.reflow_text = reflow_text(block, lines);
}
for block in blocks.iter_mut().filter(|block| block.invisible && block.lines.len() > 1) {
// Each word counts once: neighbouring words OCR sized alike share a run.
let words = block
.lines
.iter()
.filter_map(|i| lines.get(*i))
.flat_map(|line| std::iter::repeat_n(line.size, line.text.split_whitespace().count().max(1)));
if let Some(size) = median_size(words) {
block.size = size;
}
}
blocks
}
@@ -452,3 +463,14 @@ fn break_gap(block: &TextBlock, lines: &[TextLine], row_index: usize) -> Option<
// measure leaves almost no room for the next word.
(slack <= size * 0.75 && width >= size * 8.0 && (next[2] - next[0]) + size * 0.25 > slack).then_some(1)
}
/// The size an OCR phrase is set in. Recognize text sizes each word from its own box and a guess
/// at how much of the em its letters fill, so one phrase's words come out several points apart
/// (#3313); a rewrite draws the phrase at this one size. `None` when no size is usable.
fn median_size(sizes: impl Iterator<Item = f64>) -> Option<f64> {
let mut sizes: Vec<f64> = sizes.filter(|size| size.is_finite() && *size > 0.0).collect();
sizes.sort_by(f64::total_cmp);
let middle = sizes.len() / 2;
let upper = sizes.get(middle).copied()?;
if sizes.len().is_multiple_of(2) { sizes.get(middle.saturating_sub(1)).map(|lower| (lower + upper) / 2.0) } else { Some(upper) }
}
crates/edit/src/text_regions/styled.rs
+701 -104
View File
@@ -2,9 +2,9 @@ use super::super::interp::TextState;
use super::grouping::{across, is_centred, row_rect};
use super::{
BlockStyle, CraftFont, Dict, Document, EditError, HashMap, LineEdit, MAX_TYPE3_GLYPHS, Matrix, Metrics, Object, Op, PdfString, Pieces,
RangeStyle, TextBlock, TextLine, Type3Glyph, coincident_ops, content_streams, fallback_face, japanese_glyph_from, needs_type3, no_fallback_font,
no_glyph_for, page_dict, reveal_ops, rewritten_contents, source_encodes, source_family, source_program, source_show, source_space, source_width,
splice, type3_faces, type3_glyph, write_type3_glyphs,
RangeStyle, TextBlock, TextLine, Type3Glyph, builtin_thai_choice, coincident_ops, content_streams, fallback_face, japanese_glyph_from,
needs_type3, no_fallback_font, no_glyph_for, page_dict, reveal_ops, rewritten_contents, source_encodes, source_family, source_program,
source_show, source_space, source_width, splice, type3_faces, type3_glyph, write_type3_glyphs,
};
// The draft is capped at 4 MiB; also bound the outline data prepared across all fallback faces.
@@ -104,9 +104,46 @@ impl FontResolver<'_> {
}
}
struct LegacyFace {
/// Shared by every resource name of one font object: its program is decoded and kept once.
glyphs: Option<std::rc::Rc<pdfcraft_fonts::pdf::LegacyThaiFont>>,
metrics: Metrics,
/// Weight and slant ([`FontStyle::of`]).
style: FontStyle,
/// The font object the resource names, when it is given by reference.
object: Option<pdfcraft_cos::ObjRef>,
/// Its text-space fallback ([`source_space`]), in ems.
space: f64,
}
/// What one edit may decode, and so keep, of the font programs of its THSarabunPSK subsets
/// (each at most 16 MiB): a page may name many of them.
const MAX_THAI_PROGRAMS: usize = 64 << 20;
struct PreparedCharacters {
values: Vec<StyledChar>,
substituted: Option<String>,
thai: ThaiShaping,
}
/// How the Thai spans of an edit are shaped, by the font name their characters carry.
#[derive(Default)]
struct ThaiShaping {
/// Draft-only names drawn with the bundled Thai outlines.
outlined: std::collections::HashSet<String>,
/// Source faces shaped in their existing codes, by resource name.
legacy: HashMap<String, LegacyFace>,
/// Draft-only names whose rows, Latin ones included, are shaped and embedded with the
/// complete program of this same-face resource: a span its subsets can't show in place.
program: HashMap<String, String>,
}
/// How a Thai span keeps its source face ([`legacy_thai_font`]).
enum ThaiSpan {
/// In this resource's existing codes.
InPlace(String),
/// Shaped by this same-face resource's complete program with its own layout.
Program(String),
}
struct CharacterPreparation<'a> {
@@ -125,12 +162,93 @@ struct CharacterPreparation<'a> {
impl CharacterPreparation<'_> {
fn prepare(self) -> Result<PreparedCharacters, EditError> {
let Self { doc, first, lines, draft, spans, style, overrides, fonts, metrics, max_glyphs } = self;
let thai_choice = builtin_thai_choice(fonts);
let mut thai = ThaiShaping::default();
let mut thai_programs = pdfcraft_cos::DecodeBudget::new(MAX_THAI_PROGRAMS, 0);
let mut chars = Vec::new();
let mut substituted = None;
let mut fallback = FallbackBuilder::new(max_glyphs);
let mut resolver = FontResolver { doc, fonts, metrics, full_faces: HashMap::new(), programs: HashMap::new() };
for (range, index) in spans {
let source = lines.get(index).unwrap_or(first);
let span = draft.get(range.clone()).unwrap_or_default();
// Keep the exact source face. A different Thai face requires an explicit choice.
// Interface fallback fonts never enter a PDF.
let selected_resource = overrides.get(&index).and_then(|r| r.resource.as_deref()).or(style.resource.as_deref());
let use_thai_outlines = if selected_resource == Some(thai_choice.as_str()) {
let face = pdfcraft_fonts::document_thai_font().ok_or_else(|| {
EditError::Invalid(format!(
"the bundled Thai font ({thai_choice}) is not in this build, which has no craft-fonts Thai face; load a complete Thai font or choose one the page lists"
))
})?;
if !pdfcraft_fonts::embedding::covers(face.bytes, span) {
let missing = span.chars().find(|c| !pdfcraft_fonts::embedding::covers(face.bytes, c.encode_utf8(&mut [0; 4])));
return Err(EditError::Invalid(match missing {
Some(c) => format!("{} has no glyph for {c} (U+{:04X}); choose a font that covers the text", face.family, u32::from(c)),
None => format!("{} cannot show every character in this selection; choose a font that covers the text", face.family),
}));
}
substituted = Some(face.family.to_owned());
true
} else {
selected_resource.is_none() && matching_thai_outline(resolver.doc, resolver.fonts, source, span)
};
if use_thai_outlines {
substituted = pdfcraft_fonts::document_thai_font().map(|f| f.family.to_owned());
}
let existing_thai = if !use_thai_outlines {
let chosen = match selected_resource {
None => Some(source.clone()),
Some(resource) => resolver.fonts.get(resource.as_bytes()).and_then(|f| resolver.doc.dict(f)).map(|font| {
let mut chosen = source.clone();
chosen.font = resource.to_owned();
chosen.base_font = Metrics::from_dict(resolver.doc, &font).base_font;
chosen
}),
};
match chosen {
Some(chosen) => legacy_thai_font(resolver.doc, resolver.fonts, &chosen, span, &mut thai.legacy, &mut thai_programs)?,
None => None,
}
} else {
None
};
let outline_name = if use_thai_outlines {
// A draft-only name prevents this choice from affecting another style span that
// explicitly selects the original resource. It is never saved into the PDF.
let mut name = format!("PCOutlineThai{index}");
while resolver.metrics.contains_key(&name) {
name.push('_');
}
if let Some(m) = resolver
.metrics
.get(&source.font)
.or_else(|| source.origin.state.font.as_ref().and_then(|(name, _)| resolver.metrics.get(name)))
.cloned()
{
resolver.metrics.insert(name.clone(), m);
}
thai.outlined.insert(name.clone());
Some(name)
} else {
None
};
let (existing_thai, program_name) = match existing_thai {
Some(ThaiSpan::InPlace(name)) => (Some(name), None),
Some(ThaiSpan::Program(resource)) => {
// Draft-only too: another span may still show this resource's existing codes.
let mut name = format!("PCProgramThai{index}");
while resolver.metrics.contains_key(&name) {
name.push('_');
}
if let Some(m) = resolver.metrics.get(&source.font).cloned() {
resolver.metrics.insert(name.clone(), m);
}
thai.program.insert(name.clone(), resource);
(None, Some(name))
}
None => (None, None),
};
let start = range.start;
for (offset, ch) in draft.get(range).unwrap_or_default().char_indices() {
if ch == '\n' {
@@ -144,7 +262,11 @@ impl CharacterPreparation<'_> {
}
let text = ch.to_string();
let requested = overrides.get(&index).and_then(|range| range.resource.as_deref()).or(style.resource.as_deref());
let name = if let Some(name) = resolver.encodable_font(source, &text, requested)? {
let name = if let Some(name) = outline_name.as_ref().or(program_name.as_ref()) {
name.clone()
} else if let Some(name) = &existing_thai {
name.clone()
} else if let Some(name) = resolver.encodable_font(source, &text, requested)? {
name
} else if needs_type3(&text) {
fallback.prepare(source, ch, &text, chars.len())?
@@ -173,7 +295,7 @@ impl CharacterPreparation<'_> {
}
substituted = Some(written.family.to_owned());
}
Ok(PreparedCharacters { values: chars, substituted })
Ok(PreparedCharacters { values: chars, substituted, thai })
}
}
@@ -233,7 +355,16 @@ struct StyledLines<'a> {
type StyledChar = (char, usize, String);
type StyledRows = Vec<Vec<StyledChar>>;
type RowRanges = Vec<std::ops::Range<usize>>;
type ShapedRunCache = HashMap<(usize, usize), (Vec<u8>, pdfcraft_fonts::source_shaping::ShapedRun, String)>;
enum ShapedFont {
Existing(Vec<(String, Vec<u8>)>),
Program { bytes: Vec<u8>, family: String },
ThaiOutlines,
}
struct ShapedText {
run: pdfcraft_fonts::source_shaping::ShapedRun,
font: ShapedFont,
}
type ShapedRunCache = HashMap<(usize, usize), ShapedText>;
struct ShapedRows {
runs: ShapedRunCache,
@@ -243,6 +374,7 @@ struct ShapedRows {
fn prepare_shaped_rows(
doc: &Document,
fonts: &Dict,
thai: &ThaiShaping,
rows: &[Vec<StyledChar>],
kept: &[bool],
advance: impl Fn(&StyledChar) -> f64,
@@ -259,8 +391,8 @@ fn prepare_shaped_rows(
let segment = row.get(at..end).unwrap_or_default();
advances.insert((row_index, at), segment.iter().map(&advance).sum::<f64>());
let text: String = segment.iter().map(|character| character.0).collect();
if let Some((bytes, shaped, family)) = shape_source(doc, fonts, &item.2, &text)? {
runs.insert((row_index, at), (bytes, shaped, family));
if let Some(shaped) = shape_source(doc, fonts, thai, &item.2, &text)? {
runs.insert((row_index, at), shaped);
}
at = end;
}
@@ -268,16 +400,138 @@ fn prepare_shaped_rows(
Ok(ShapedRows { runs, advances })
}
fn embed_shaped_resources(doc: &mut Document, fonts: &mut Dict, runs: &ShapedRunCache) -> Result<HashMap<(usize, usize), String>, EditError> {
type ShapedResources = HashMap<(usize, usize), Vec<(String, Vec<u8>)>>;
fn embed_shaped_resources(doc: &mut Document, fonts: &mut Dict, runs: &ShapedRunCache) -> Result<ShapedResources, EditError> {
let mut resources = HashMap::new();
for (key, (bytes, shaped, family)) in runs {
let mappings: Vec<_> = shaped.glyphs.iter().map(|glyph| (glyph.text.clone(), glyph.id)).collect();
let resource = pdfcraft_fonts::embedding::embed_mapping(doc, fonts, bytes, family, &mappings).map_err(EditError::Invalid)?;
resources.insert(*key, resource);
for (key, shaped) in runs {
let glyphs = match &shaped.font {
ShapedFont::Existing(codes) => codes.clone(),
ShapedFont::ThaiOutlines => crate::added::thai_glyph_resources(doc, fonts, &shaped.run)?,
ShapedFont::Program { bytes, family } => {
let mappings: Vec<_> = shaped.run.glyphs.iter().map(|glyph| (glyph.text.clone(), glyph.id)).collect();
let resource = pdfcraft_fonts::embedding::embed_mapping(doc, fonts, bytes, family, &mappings).map_err(EditError::Invalid)?;
(0..shaped.run.glyphs.len())
.map(|i| {
let code = u16::try_from(i.saturating_add(1)).map_err(|_| EditError::Invalid("shaped glyph limit".into()))?;
Ok((resource.clone(), code.to_be_bytes().to_vec()))
})
.collect::<Result<Vec<_>, EditError>>()?
}
};
resources.insert(*key, glyphs);
}
Ok(resources)
}
/// A positioned glyph run, kept in one text object across Type 3 font shards.
struct ShapedRunDrawing<'a> {
doc: &'a Document,
fonts: &'a Dict,
shaped: &'a pdfcraft_fonts::source_shaping::ShapedRun,
resources: &'a [(String, Vec<u8>)],
text: &'a str,
matrix: [f64; 6],
size: f64,
thai: bool,
}
impl ShapedRunDrawing<'_> {
/// Draw and return the run's advance in text space, including cluster-safe spacing.
fn draw(&self, block_ops: &mut Vec<Op>, state: &mut TextState) -> Result<f64, EditError> {
let Self { doc, fonts, shaped, resources, text, matrix: tm, size, thai } = *self;
let n = pdfcraft_content::num;
let (resource, _) = self.resources.first().ok_or_else(|| EditError::Invalid("shaped run has no glyphs".into()))?;
let mut properties = Dict::new();
properties.set(b"ActualText".to_vec(), Object::String(PdfString::text(text)));
block_ops.push(Op::new("BDC", vec![Object::name("Span"), Object::Dict(properties)]));
state.font = Some((resource.clone(), size));
// One text object for the run: each glyph is placed where shaping put it by a TJ
// adjustment from where the glyph before it ends (its advance in the embedded
// font's widths), and raised by the text rise for a mark set above or below
// (ISO 32000-1 §9.4.3, §9.3.7). Font-program codes are separate from Unicode;
// ToUnicode and ActualText keep the logical text.
let per_unit = 1000.0 / size;
if !per_unit.is_finite() {
return Err(EditError::Invalid("the shaped text has no size".into()));
}
let mut shown_fonts = HashMap::new();
for (name, _) in resources {
if !shown_fonts.contains_key(name) {
let font = fonts.get(name.as_bytes()).and_then(|f| doc.dict(f)).ok_or_else(|| EditError::Invalid("shaped font is missing".into()))?;
shown_fonts.insert(name.clone(), Metrics::from_dict(doc, &font));
}
}
// Keep the original text settings. TJ compensates for the automatic PDF advance
// after each glyph, while shaping keeps marks attached to their cluster.
let (char_spacing, word_spacing, rise) = (state.char_spacing, state.word_spacing, state.rise);
block_ops.push(Op::new("BT", vec![]));
block_ops.extend(state.ops());
block_ops.push(Op::new("Tm", tm.iter().map(|v| n(*v)).collect()));
// `pen`: where the next glyph would start, in thousandths of the font size.
let (mut pen, mut extra, mut raised) = (0.0, 0.0, rise);
let mut shown_glyphs: Vec<Object> = Vec::new();
let mut cluster_spacing = 0.0;
let mut current_resource = resource.as_str();
for (i, glyph) in shaped.glyphs.iter().enumerate() {
let (name, code) = resources.get(i).ok_or_else(|| EditError::Invalid("shaped glyph resource is missing".into()))?;
let shown = shown_fonts.get(name).ok_or_else(|| EditError::Invalid("shaped font is missing".into()))?;
if name != current_resource {
if !shown_glyphs.is_empty() {
block_ops.push(Op::new("TJ", vec![Object::Array(std::mem::take(&mut shown_glyphs))]));
}
block_ops.push(Op::new("Tf", vec![Object::name(name), n(size)]));
current_resource = name;
}
if thai && !glyph.text.is_empty() {
// Apply spacing only after a whole cluster, keeping marks on their base.
extra += cluster_spacing;
cluster_spacing = char_spacing * glyph.text.chars().count() as f64 + word_spacing * glyph.text.matches(' ').count() as f64;
}
let target = (glyph.position[0] * size + extra) * per_unit;
let glyph_rise = rise + glyph.position[1] * size;
if (glyph_rise - raised).abs() > 1e-9 || shown_glyphs.len() >= 4096 {
if !shown_glyphs.is_empty() {
block_ops.push(Op::new("TJ", vec![Object::Array(std::mem::take(&mut shown_glyphs))]));
}
if (glyph_rise - raised).abs() > 1e-9 {
block_ops.push(Op::new("Ts", vec![n(glyph_rise)]));
raised = glyph_rise;
}
}
if (target - pen).abs() > 1e-6 {
shown_glyphs.push(n(pen - target));
}
shown_glyphs.push(Object::String(PdfString::literal(code.clone())));
let cid = code.iter().fold(0_u32, |value, byte| value.saturating_mul(256).saturating_add(u32::from(*byte)));
pen = target
+ if code.is_empty() {
0.0
} else {
shown.width(cid) * 1000.0 + (char_spacing + if shown.is_space(cid, code.len()) { word_spacing } else { 0.0 }) * per_unit
};
if !thai {
extra += char_spacing * glyph.text.chars().count() as f64 + word_spacing * glyph.text.matches(' ').count() as f64;
}
}
let end = (shaped.advance * size + extra + cluster_spacing) * per_unit;
if (end - pen).abs() > 1e-6 {
shown_glyphs.push(n(pen - end));
}
if !shown_glyphs.is_empty() {
block_ops.push(Op::new("TJ", vec![Object::Array(shown_glyphs)]));
}
if (raised - rise).abs() > 1e-9 {
block_ops.push(Op::new("Ts", vec![n(rise)]));
}
block_ops.push(Op::new("ET", vec![]));
(state.char_spacing, state.word_spacing) = (char_spacing, word_spacing);
extra += cluster_spacing;
block_ops.push(Op::new("EMC", vec![]));
Ok(shaped.advance * size + extra)
}
}
fn validate_contextual_rows(rows: &[Vec<StyledChar>], kept: &[bool]) -> Result<(), EditError> {
for (row, _) in rows.iter().zip(kept).filter(|(_, retained)| !**retained) {
let text: String = row.iter().map(|character| character.0).collect();
@@ -321,6 +575,23 @@ fn calibration_factors(
calibration
}
/// The calibration factors fit each word to its box at its own size (its `Tf` in its text space,
/// which a range's size doesn't change). An OCR phrase drawn at one size needs the inverse ratio
/// too, so every word still spans the box the scan shows it in. The ratio is not clamped again:
/// a word OCR sized at three times the phrase's needs three times the calibration.
fn fit_to_phrase(calibration: &mut HashMap<usize, f64>, lines: &[TextLine], phrase: Option<f64>) {
let Some(phrase) = phrase else { return };
for (index, factor) in calibration.iter_mut() {
let own = lines.get(*index).and_then(|source| Some(source.origin.state.font.as_ref()?.1 * source.origin.k));
if let Some(own) = own.filter(|own| own.is_finite() && *own > 0.0) {
let fitted = *factor * own / phrase;
if fitted.is_finite() {
*factor = fitted;
}
}
}
}
fn validated_ranges<'a>(draft: &str, style: &'a BlockStyle) -> Result<Vec<&'a RangeStyle>, EditError> {
let mut ranges: Vec<_> = style.ranges.iter().collect();
ranges.sort_by_key(|range| range.range.start);
@@ -414,18 +685,41 @@ fn row_step(style: &BlockStyle, size: f64, source_leading: f64) -> f64 {
type WrappedStyledRows = (StyledRows, RowRanges, Vec<usize>, Vec<bool>, Vec<usize>);
/// For each of `chars`, whether a row may end just before it inside a run of non-space characters
/// (Chinese, Japanese and Thai), worked out once per run: Thai words need the whole run, not just
/// two neighbours.
fn styled_break_before(chars: &[StyledChar]) -> Vec<bool> {
let mut break_before = vec![false; chars.len()];
let mut run_start = 0;
while run_start < chars.len() {
let mut run_end = run_start;
while chars.get(run_end).is_some_and(|c| !c.0.is_whitespace()) {
run_end += 1;
}
let run: String = chars.get(run_start..run_end).unwrap_or_default().iter().map(|c| c.0).collect();
for (i, cut) in crate::linebreak::breaks_before_chars(&run).into_iter().enumerate() {
if let Some(slot) = break_before.get_mut(run_start.saturating_add(i)) {
*slot = cut;
}
}
run_start = run_end.max(run_start.saturating_add(1));
}
break_before
}
fn wrap_styled_rows(
chars: &[StyledChar],
width: f64,
measure: impl Fn(&[StyledChar]) -> Result<f64, EditError>,
) -> Result<WrappedStyledRows, EditError> {
let break_before = styled_break_before(chars);
let mut rows: Vec<Vec<StyledChar>> = Vec::new();
let mut row_ranges = Vec::new();
// True when this row starts because the previous word did not fit, not because of a newline.
let mut row_soft = Vec::new();
let mut next_soft = false;
// How many separator characters a wrap trimmed off the row that just ended, for the row
// about to start (0 for a Chinese/Japanese word split, which trims none; meaningless unless
// about to start (0 for a Chinese, Japanese or Thai word split, which trims none; meaningless unless
// the row it belongs to is soft). Mirrors `next_soft`.
let mut row_gap = Vec::new();
let mut next_gap = 0usize;
@@ -458,11 +752,8 @@ fn wrap_styled_rows(
}
let mut end = at + 1;
if !item.0.is_whitespace() {
// A word ends at a space, or where Chinese or Japanese text may break between characters.
while let (Some(before), Some(current)) = (chars.get(end - 1), chars.get(end))
&& !current.0.is_whitespace()
&& !crate::linebreak::between(before.0, current.0)
{
// A word ends at a space, or where Chinese, Japanese or Thai text may break inside it.
while chars.get(end).is_some_and(|c| !c.0.is_whitespace()) && !break_before.get(end).copied().unwrap_or(false) {
end += 1;
}
}
@@ -578,35 +869,385 @@ fn apply_range_styles<'a>(
StyledLines { lines: styled_lines, spans: styled_spans, underlines, overrides }
}
/// Shape in the source or an explicitly registered matching full face. Never use interface fonts.
fn shape_source(
/// Select one existing subset of the exact source face for a Thai span. Never replace its
/// family/weight to make an edit succeed; missing glyphs need an explicit font choice. The
/// subsets' programs are read within `budget`, shared by the edit's spans.
fn legacy_thai_font(
doc: &Document,
fonts: &Dict,
name: &str,
text: &str,
) -> Result<Option<(Vec<u8>, pdfcraft_fonts::source_shaping::ShapedRun, String)>, EditError> {
source: &TextLine,
span: &str,
cache: &mut HashMap<String, LegacyFace>,
budget: &mut pdfcraft_cos::DecodeBudget,
) -> Result<Option<ThaiSpan>, EditError> {
if !source.base_font.rsplit('+').next().unwrap_or(&source.base_font).starts_with("THSarabunPSK")
|| !span.chars().any(|c| matches!(u32::from(c), 0x0e00..=0x0e7f))
{
return Ok(None);
}
let family = pdfcraft_fonts::embedding::face_key(&source.base_font);
let mut candidates = vec![source.font.clone()];
candidates.extend(fonts.iter().filter_map(|(name, font)| {
let font = doc.dict(font)?;
(pdfcraft_fonts::embedding::face_key(&Metrics::from_dict(doc, &font).base_font) == family).then(|| String::from_utf8_lossy(name).into_owned())
}));
for name in candidates.into_iter().take(128) {
if cache.contains_key(&name) {
continue;
}
let Some(resource) = fonts.get(name.as_bytes()) else { continue };
let Some(font) = doc.dict(resource) else { continue };
let object = resource.as_ref();
let face = match object.and_then(|o| cache.values().find(|face| face.object == Some(o))) {
Some(alias) => LegacyFace { glyphs: alias.glyphs.clone(), metrics: alias.metrics.clone(), object, ..*alias },
None => {
let metrics = Metrics::from_dict(doc, &font);
let space = source_space(doc, fonts, &metrics);
let glyphs = pdfcraft_fonts::pdf::LegacyThaiFont::from_pdf(doc, &font, space, budget).ok().flatten().map(std::rc::Rc::new);
let style = FontStyle::of(doc, &font, &metrics);
LegacyFace { glyphs, metrics, style, object, space }
}
};
cache.insert(name, face);
}
// A subset left unread would count as unusable, silently changing which subset draws the text.
if budget.remaining() == 0 {
return Err(EditError::Invalid(format!(
"the page's {} subsets need more than {} MiB of font programs for one edit; choose a replacement font for this text",
source.base_font.rsplit('+').next().unwrap_or(&source.base_font),
MAX_THAI_PROGRAMS >> 20
)));
}
// A face none of whose subsets can be shaped in place (no Thai, or neither positioned nor in
// the legacy glyph order) is shaped like any other source font (`shape_source`).
if !has_thai_source_face(cache, &family) {
return Ok(None);
}
let rows = || span.split(['\n', '\r']).filter(|row| !row.is_empty());
for row in rows() {
if let Err(error) = shape_existing_thai(cache, &source.font, row) {
// Not in the face's existing codes. A complete program of the face that shapes every
// row with its own layout (one Load font added as CFF, or whose contextual glyphs it
// gave no code yet) takes the whole span, so no cluster is split between fonts.
let Some(source_font) = fonts.get(source.font.as_bytes()).and_then(|f| doc.dict(f)) else { return Err(error) };
let complete = program_candidates(doc, fonts, &source.font, source_font, &source.base_font)
.into_iter()
.find(|(_, font)| rows().all(|row| shape_with_program(doc, font, row).is_some()));
return match complete {
Some((name, _)) => Ok(Some(ThaiSpan::Program(name))),
None => Err(error),
};
}
}
Ok(Some(ThaiSpan::InPlace(source.font.clone())))
}
/// Whether a subset of the face `family` (a [`face_key`](pdfcraft_fonts::embedding::face_key))
/// in `cache` can be shaped in place: a THSarabunPSK program with its own Thai mark positioning,
/// or a verified legacy positional one.
fn has_thai_source_face(cache: &HashMap<String, LegacyFace>, family: &str) -> bool {
cache.values().any(|face| face.glyphs.is_some() && pdfcraft_fonts::embedding::face_key(&face.metrics.base_font) == family)
}
/// A visual row may use several subsets of the same exact face (often a separate subset for
/// punctuation). Shape Thai clusters intact, and retain each glyph's original PDF font/code.
fn shape_existing_thai(cache: &HashMap<String, LegacyFace>, primary: &str, text: &str) -> Result<ShapedText, EditError> {
if text.len() > 262_144 {
return Err(EditError::Invalid("the Thai source row exceeds its shaping budget".into()));
}
let first = cache.get(primary).ok_or_else(|| EditError::Invalid("the Thai source face is missing".into()))?;
let family = pdfcraft_fonts::embedding::face_key(&first.metrics.base_font);
let mut candidates = vec![(primary, first)];
candidates.extend(
cache
.iter()
.filter(|(name, face)| {
name.as_str() != primary && face.style == first.style && pdfcraft_fonts::embedding::face_key(&face.metrics.base_font) == family
})
.map(|(name, face)| (name.as_str(), face)),
);
// Stable source-first choice, then resource name; never depend on hash iteration order.
if let Some(tail) = candidates.get_mut(1..) {
tail.sort_by_key(|(name, _)| *name);
}
let thai = |c: char| matches!(u32::from(c), 0x0e01..=0x0e5b | 0xf700..=0xf71a);
let mut run = pdfcraft_fonts::source_shaping::ShapedRun { glyphs: Vec::new(), advance: 0.0 };
let mut resources = Vec::new();
let mut chars = text.char_indices().peekable();
while let Some((at, ch)) = chars.next() {
if thai(ch) {
let mut end = at + ch.len_utf8();
while let Some(&(next, c)) = chars.peek() {
if !thai(c) {
break;
}
chars.next();
end = next + c.len_utf8();
}
let segment = text.get(at..end).ok_or_else(|| EditError::Invalid("invalid Thai character boundary".into()))?;
let unshaped = |why: String| {
EditError::Invalid(format!(
"{} cannot keep this edit in the original face: {}; load the complete original font or explicitly choose a replacement",
first.metrics.base_font,
if why.is_empty() { "its source glyph program is incomplete" } else { &why }
))
};
// The whole run in one subset keeps any kerning between its clusters. A subsetter may
// instead have split the run's glyphs across subsets of the face: then each cluster
// (a base and the marks on it, which shape together) comes from a subset that has it.
let pieces = match shape_in_one_subset(&candidates, segment) {
Ok(whole) => vec![whole],
Err(_) => thai_clusters(segment)
.into_iter()
.map(|cluster| shape_in_one_subset(&candidates, cluster).map_err(unshaped))
.collect::<Result<Vec<_>, _>>()?,
};
for (name, shaped) in pieces {
for (mut glyph, code) in shaped.shaped.glyphs.into_iter().zip(shaped.codes) {
glyph.position[0] += run.advance;
resources.push((name.to_owned(), code));
run.glyphs.push(glyph);
}
run.advance += shaped.shaped.advance;
}
} else {
let logical = ch.to_string();
let found = candidates.iter().find_map(|(name, face)| {
let code = face.metrics.encode(&logical)?;
let first = face.metrics.codes(&code).first()?.0;
// `encode` accepts a CID the subset emptied (#1443): drawing it would show nothing.
// Try the next subset; a space then falls back to its measured gap below.
if face.metrics.has_outline(first) == Some(false) {
return None;
}
Some((*name, code, face.metrics.width(first)))
});
let (name, code, width) = if let Some(found) = found {
found
} else if ch == ' ' {
// The source subset may omit spaces. Use a gap measured from a matching
// document subset, then the face's existing text-space fallback.
let width = candidates
.iter()
.find_map(|(_, f)| {
let code = f.metrics.encode(" ")?;
Some(f.metrics.width(f.metrics.codes(&code).first()?.0))
})
.unwrap_or(first.space);
(primary, Vec::new(), width)
} else {
return Err(EditError::Invalid(format!(
"{} has no source glyph for {ch} (U+{:04X}); load the complete original face or explicitly select a replacement",
first.metrics.base_font,
u32::from(ch)
)));
};
run.glyphs.push(pdfcraft_fonts::source_shaping::ShapedGlyph { id: Default::default(), position: [run.advance, 0.0], text: logical });
resources.push((name.to_owned(), code));
run.advance += width;
}
if run.glyphs.len() > 65_534 || !run.advance.is_finite() {
return Err(EditError::Invalid("the Thai source row exceeds its shaping budget".into()));
}
}
Ok(ShapedText { run, font: ShapedFont::Existing(resources) })
}
/// Shape `text` in the first candidate subset that has every glyph it needs, and name it. The
/// error is the first subset's reason (empty when no candidate has a usable glyph program).
fn shape_in_one_subset<'a>(candidates: &[(&'a str, &LegacyFace)], text: &str) -> Result<(&'a str, pdfcraft_fonts::pdf::ExistingThaiRun), String> {
let mut why = String::new();
for (name, face) in candidates {
let Some(font) = &face.glyphs else { continue };
match font.shape(text) {
Ok(shaped) => return Ok((name, shaped)),
Err(e) if why.is_empty() => why = e,
Err(_) => {}
}
}
Err(why)
}
/// `text` (a run of Thai characters) cut before each character that doesn't attach to the one
/// before it: a cluster is a base or leading vowel with the vowels and tone marks above and below
/// it, and Sara Am, whose Nikhahit sits on the base. Legacy presentation forms (U+F700–U+F71A)
/// are kept with what precedes them.
fn thai_clusters(text: &str) -> Vec<&str> {
let attaches = |c: char| matches!(u32::from(c), 0x0e31 | 0x0e33..=0x0e3a | 0x0e47..=0x0e4e | 0xf700..=0xf71a);
let mut starts: Vec<usize> = text.char_indices().filter(|(at, c)| *at == 0 || !attaches(*c)).map(|(at, _)| at).collect();
starts.push(text.len());
// Each boundary is a `char_indices` position or the end, strictly increasing: every `get`
// succeeds and no cluster is empty.
starts.windows(2).filter_map(|w| text.get(*w.first()?..*w.get(1)?)).collect()
}
/// Whether the approved Thai face can replace this source, whose embedded subset cannot
/// shape the replacement. A usable complete source program always takes precedence.
fn matching_thai_outline(doc: &Document, fonts: &Dict, source: &TextLine, span: &str) -> bool {
let Some(face) = pdfcraft_fonts::document_thai_font() else { return false };
// The bundled face is Regular and upright. A source its descriptor or name marks bold or
// italic keeps that style, so completing it with these outlines would be a silent change:
// it needs an explicit choice instead. (Variable-font instances often keep the family name.)
let style = fonts
.get(source.font.as_bytes())
.and_then(|f| doc.dict(f))
.map_or(FontStyle { bold: source.bold, italic: source.italic }, |f| FontStyle::of(doc, &f, &Metrics::from_dict(doc, &f)));
if style.bold || style.italic {
return false;
}
if !span.chars().any(|c| matches!(u32::from(c), 0x0e00..=0x0e7f))
|| !pdfcraft_fonts::embedding::matching_faces(&source.base_font).iter().any(|f| std::ptr::eq(*f, face))
|| !pdfcraft_fonts::embedding::covers(face.bytes, span)
{
return false;
}
// A registered complete matching program also completes the source subset.
// Do not replace it with bundled outlines merely because the original resource is a subset.
// Only one in the source's style does (`program_candidates`), so only that one counts here.
let family = pdfcraft_fonts::embedding::face_key(&source.base_font);
!fonts.iter().filter_map(|(_, font)| doc.dict(font)).any(|font| {
let metrics = Metrics::from_dict(doc, &font);
pdfcraft_fonts::embedding::face_key(&metrics.base_font) == family
&& FontStyle::of(doc, &font, &metrics) == style
&& pdfcraft_fonts::pdf::embedded_sfnt(doc, &font).is_some_and(|bytes| {
pdfcraft_fonts::embedding::validate(&bytes).is_ok()
&& pdfcraft_fonts::source_shaping::has_thai_layout(&bytes)
&& pdfcraft_fonts::embedding::covers(&bytes, span)
&& span.split(['\n', '\r']).all(|row| pdfcraft_fonts::source_shaping::shape(&bytes, row).is_ok())
})
})
}
/// Use the same Thai script selection and mark/cluster positioning as Add Text, including
/// rows beginning with Latin or digits (guessing from their first character would select Latin).
fn shape_thai_row(text: &str) -> Result<pdfcraft_fonts::source_shaping::ShapedRun, EditError> {
if text.len() > 262_144 || text.contains(['\n', '\r']) {
return Err(EditError::Invalid("shape one bounded text row at a time".into()));
}
let clusters =
pdfcraft_fonts::shape_thai(text).map_err(|e| EditError::Invalid(format!("the matching Thai font cannot shape this text ({e:?})")))?;
let mut glyphs = Vec::new();
let mut advance = 0.0;
for cluster in clusters {
for (i, (id, [x, y])) in cluster.glyphs.into_iter().enumerate() {
if glyphs.len() >= 65_534 {
return Err(EditError::Invalid("shaped row exceeds its geometry budget".into()));
}
glyphs.push(pdfcraft_fonts::source_shaping::ShapedGlyph {
id: id.into(),
position: [advance + x, y],
text: if i == 0 { cluster.text.clone() } else { String::new() },
});
}
advance += cluster.advance;
}
if !advance.is_finite() {
return Err(EditError::Invalid("shaped row exceeds its geometry budget".into()));
}
Ok(pdfcraft_fonts::source_shaping::ShapedRun { glyphs, advance })
}
/// The resources whose complete program may shape text in the face of `base_font`: `source`
/// (the resource `name`) first, then the page's other fonts of that face (at most 128).
fn program_candidates(doc: &Document, fonts: &Dict, name: &str, source: Dict, base_font: &str) -> Vec<(String, Dict)> {
let key = pdfcraft_fonts::embedding::face_key(base_font);
let style = FontStyle::of(doc, &source, &Metrics::from_dict(doc, &source));
let mut candidates = vec![(name.to_owned(), source)];
candidates.extend(
fonts
.iter()
.filter_map(|(n, f)| Some((String::from_utf8_lossy(n).into_owned(), doc.dict(f)?)))
.filter(|(_, f)| {
let m = Metrics::from_dict(doc, f);
pdfcraft_fonts::embedding::face_key(&m.base_font) == key && FontStyle::of(doc, f, &m) == style
})
.take(128),
);
candidates
}
/// A font's weight and slant, for matching subsets and complete programs of one face: a Regular
/// program never completes text that a descriptor marks bold, say.
#[derive(Clone, Copy, Debug, PartialEq)]
struct FontStyle {
bold: bool,
italic: bool,
}
impl FontStyle {
/// Bold from the name or the descriptor's weight ([`Metrics`]); italic from the name or a
/// non-zero `/ItalicAngle`. The descriptor's Italic flag alone isn't trusted: exporters set
/// it on upright faces (a THSarabunPSK subset with `/Flags 96` and `/ItalicAngle 0`).
fn of(doc: &Document, font: &Dict, metrics: &Metrics) -> Self {
let descendant =
font.get(b"DescendantFonts").and_then(|d| doc.resolve(d).as_array().and_then(|a| a.first()).cloned()).and_then(|d| doc.dict(&d));
let descriptor = descendant.as_ref().unwrap_or(font).get(b"FontDescriptor").and_then(|d| doc.dict(d));
let angle = descriptor.as_ref().and_then(|d| d.get(b"ItalicAngle")).and_then(|a| doc.resolve(a).as_f64()).unwrap_or(0.0);
let name = metrics.base_font.to_ascii_lowercase();
Self { bold: metrics.bold, italic: name.contains("italic") || name.contains("oblique") || (angle.is_finite() && angle.abs() > 0.1) }
}
}
/// `text` shaped by `font`'s embedded program with its own layout, when the program covers it and
/// may be embedded again (`embed_mapping` refuses one that may not, so a later program is tried).
fn shape_with_program(doc: &Document, font: &Dict, text: &str) -> Option<(Vec<u8>, pdfcraft_fonts::source_shaping::ShapedRun)> {
let bytes = pdfcraft_fonts::pdf::embedded_sfnt(doc, font)?;
if pdfcraft_fonts::embedding::validate(&bytes).is_err() || !pdfcraft_fonts::embedding::covers(&bytes, text) {
return None;
}
let shaped = pdfcraft_fonts::source_shaping::shape(&bytes, text).ok()?;
Some((bytes, shaped))
}
/// Shape in the source or an explicitly registered matching full face. Never use interface fonts.
fn shape_source(doc: &Document, fonts: &Dict, thai: &ThaiShaping, name: &str, text: &str) -> Result<Option<ShapedText>, EditError> {
if thai.outlined.contains(name) {
let shaped = shape_thai_row(text)?;
return Ok(Some(ShapedText { run: shaped, font: ShapedFont::ThaiOutlines }));
}
if let Some(resource) = thai.program.get(name) {
let font = fonts
.get(resource.as_bytes())
.and_then(|f| doc.dict(f))
.ok_or_else(|| EditError::Invalid("the complete font resource is missing".into()))?;
let family = Metrics::from_dict(doc, &font).base_font;
let (bytes, shaped) = shape_with_program(doc, &font, text)
.ok_or_else(|| EditError::Invalid(format!("{family} cannot shape this text with its own layout; choose a font that covers it")))?;
return Ok(Some(ShapedText { run: shaped, font: ShapedFont::Program { bytes, family } }));
}
if !pdfcraft_fonts::source_shaping::required(text) {
return Ok(None);
}
let source =
fonts.get(name.as_bytes()).and_then(|f| doc.dict(f)).ok_or_else(|| EditError::Invalid("the shaped text has no font resource".into()))?;
let family = Metrics::from_dict(doc, &source).base_font;
let mut candidates = vec![source];
candidates.extend(
fonts
.iter()
.filter_map(|(_, f)| doc.dict(f))
.filter(|f| pdfcraft_fonts::embedding::face_key(&Metrics::from_dict(doc, f).base_font) == pdfcraft_fonts::embedding::face_key(&family)),
);
for font in candidates {
if let Some(bytes) = pdfcraft_fonts::pdf::embedded_sfnt(doc, &font)
&& pdfcraft_fonts::embedding::covers(&bytes, text)
&& let Ok(shaped) = pdfcraft_fonts::source_shaping::shape(&bytes, text)
{
return Ok(Some((bytes, shaped, family)));
// In the face's existing codes first; what they can't show, a complete program of the face
// may (`legacy_thai_font` already gave the span to one that can).
let in_place = (family.rsplit('+').next().unwrap_or(&family).starts_with("THSarabunPSK")
&& text.chars().any(|c| matches!(u32::from(c), 0x0e00..=0x0e7f))
&& has_thai_source_face(&thai.legacy, &pdfcraft_fonts::embedding::face_key(&family)))
.then(|| shape_existing_thai(&thai.legacy, name, text));
let in_place = match in_place {
Some(Ok(shaped)) => return Ok(Some(shaped)),
Some(Err(error)) => Some(error),
None => None,
};
for (_, font) in program_candidates(doc, fonts, name, source, &family) {
if let Some((bytes, shaped)) = shape_with_program(doc, &font, text) {
return Ok(Some(ShapedText { run: shaped, font: ShapedFont::Program { bytes, family } }));
}
}
Err(EditError::Invalid("complex text needs the complete source TrueType/OpenType face; load that font before editing".into()))
if let Some(error) = in_place {
return Err(error);
}
let fallback = pdfcraft_fonts::document_thai_font()
.filter(|_| text.chars().any(|c| matches!(u32::from(c), 0x0e00..=0x0e7f)))
.map(|face| format!("; or select {} in the document-font menu to replace it", face.family))
.unwrap_or_default();
Err(EditError::Invalid(format!(
"complex text in {} needs its complete source TrueType/OpenType font; use Load font to keep it{fallback}",
family.rsplit('+').next().unwrap_or(&family)
)))
}
/// For each output row (a range of the draft's characters), whether it is exactly the source row
@@ -790,7 +1431,7 @@ fn push_whole_paragraph_marker(block_ops: &mut Vec<Op>, marked: bool, op: &str)
/// An empty marker between two rows of one rewrite. `/PCHard` is a newline that was already in
/// the text. `/PCSoft` is a wrap and rejoins when the box widens, with `/N` the exact number of
/// separator characters it consumed (0 for a Chinese/Japanese word split), so a rejoin restores
/// separator characters it consumed (0 for a Chinese, Japanese or Thai word split), so a rejoin restores
/// what was actually there instead of guessing a single space (#2529). Neither is a paragraph
/// region, so the lines stay in one scope.
fn push_row_break_marker(block_ops: &mut Vec<Op>, row_soft: &[bool], row_gap: &[usize], row_index: usize) {
@@ -889,10 +1530,12 @@ pub(crate) fn rewrite_styled_block(
max_glyphs,
}
.prepare()?;
let thai = prepared.thai;
let chars = prepared.values;
let substituted = prepared.substituted;
let phrase_size = block.phrase_size();
let size_of = |source: &TextLine, index: usize| {
overrides.get(&index).and_then(|r| r.size).or(style.size).unwrap_or(source.size).max(0.1) / source.origin.k.max(1e-6)
overrides.get(&index).and_then(|r| r.size).or(style.size).or(phrase_size).unwrap_or(source.size).max(0.1) / source.origin.k.max(1e-6)
};
// A source run may carry TJ kerning/justification that is absent from its text state.
// Keep its measured advance when rebuilding the run instead of making it wrap on entry.
@@ -910,7 +1553,8 @@ pub(crate) fn rewrite_styled_block(
spaces.borrow_mut().insert(name.to_owned(), w);
w
};
let calibration = calibration_factors(lines, &metrics, style, &overrides, space_of);
let mut calibration = calibration_factors(lines, &metrics, style, &overrides, space_of);
fit_to_phrase(&mut calibration, lines, phrase_size);
let advance = |item: &(char, usize, String)| -> f64 {
let source = lines.get(item.1).unwrap_or(first);
let Some(m) = metrics.get(&item.2) else { return 0.0 };
@@ -939,7 +1583,8 @@ pub(crate) fn rewrite_styled_block(
}
let segment = items.get(at..end).unwrap_or_default();
let string: String = segment.iter().map(|v| v.0).collect();
if let Some((_, shaped, _)) = shape_source(doc, &fonts, &item.2, &string)? {
if let Some(shaped) = shape_source(doc, &fonts, &thai, &item.2, &string)? {
let shaped = &shaped.run;
let source = lines.get(item.1).unwrap_or(first);
let scale =
overrides.get(&item.1).and_then(|r| r.scale).or(style.scale).map_or(source.origin.state.scale, |s| (s / 100.0).clamp(0.1, 10.0));
@@ -990,7 +1635,7 @@ pub(crate) fn rewrite_styled_block(
let at = if split { start } else { o.bt_op };
validate_contextual_rows(&rows, &kept)?;
let measured: Vec<_> = rows.iter().map(|row| measure(row)).collect::<Result<_, _>>()?;
let shaped = prepare_shaped_rows(doc, &fonts, &rows, &kept, advance)?;
let shaped = prepare_shaped_rows(doc, &fonts, &thai, &rows, &kept, advance)?;
let ShapedRows { runs: shaped_runs, advances: segment_advances } = shaped;
let glyph_resources = embed_shaped_resources(doc, &mut fonts, &shaped_runs)?;
let n = pdfcraft_content::num;
@@ -1100,67 +1745,19 @@ pub(crate) fn rewrite_styled_block(
// drawn in: a `gs` has no inverse, so one run's ExtGState would stay for the next.
block_ops.push(Op::new("q", vec![]));
block_ops.extend(state.paint_ops(drawn_from, run_ctm));
if let Some((_, shaped, _)) = shaped_runs.get(&(row_index, pos)) {
let resource = glyph_resources.get(&(row_index, pos)).ok_or_else(|| EditError::Invalid("shaped font is missing".into()))?;
let mut properties = Dict::new();
properties.set(b"ActualText".to_vec(), Object::String(PdfString::text(&string)));
block_ops.push(Op::new("BDC", vec![Object::name("Span"), Object::Dict(properties)]));
state.font = Some((resource.clone(), size_of(source, item.1)));
// One text object for the run: each glyph is placed where shaping put it by a TJ
// adjustment from where the glyph before it ends (its advance in the embedded
// font's widths), and raised by the text rise for a mark set above or below
// (ISO 32000-1 §9.4.3, §9.3.7). Font-program codes are separate from Unicode;
// ToUnicode and ActualText keep the logical text.
let size = size_of(source, item.1);
let per_unit = 1000.0 / size;
if !per_unit.is_finite() {
return Err(EditError::Invalid("the shaped text has no size".into()));
}
let shown = fonts
.get(resource.as_bytes())
.and_then(|f| doc.resolve(f).as_dict().cloned())
.map(|d| Metrics::from_dict(doc, &d))
.ok_or_else(|| EditError::Invalid("shaped font is missing".into()))?;
// Spacing is part of each glyph's position, so the run itself has none.
let (char_spacing, word_spacing, rise) = (state.char_spacing, state.word_spacing, state.rise);
state.char_spacing = 0.0;
state.word_spacing = 0.0;
block_ops.push(Op::new("BT", vec![]));
block_ops.extend(state.ops());
block_ops.push(Op::new("Tm", tm.iter().map(|v| n(*v)).collect()));
// `pen`: where the next glyph would start, in thousandths of the font size.
let (mut pen, mut extra, mut raised) = (0.0, 0.0, rise);
let mut shown_glyphs: Vec<Object> = Vec::new();
for (i, glyph) in shaped.glyphs.iter().enumerate() {
let target = (glyph.position[0] * size + extra) * per_unit;
let glyph_rise = rise + glyph.position[1] * size;
if (glyph_rise - raised).abs() > 1e-9 || shown_glyphs.len() >= 4096 {
if !shown_glyphs.is_empty() {
block_ops.push(Op::new("TJ", vec![Object::Array(std::mem::take(&mut shown_glyphs))]));
}
if (glyph_rise - raised).abs() > 1e-9 {
block_ops.push(Op::new("Ts", vec![n(glyph_rise)]));
raised = glyph_rise;
}
}
if (target - pen).abs() > 1e-6 {
shown_glyphs.push(n(pen - target));
}
let code = u16::try_from(i + 1).map_err(|_| EditError::Invalid("shaped glyph limit".into()))?;
shown_glyphs.push(Object::String(PdfString::literal(code.to_be_bytes().to_vec())));
pen = target + shown.width(u32::from(code)) * 1000.0;
extra += char_spacing * glyph.text.chars().count() as f64 + word_spacing * glyph.text.matches(' ').count() as f64;
}
if !shown_glyphs.is_empty() {
block_ops.push(Op::new("TJ", vec![Object::Array(shown_glyphs)]));
}
if (raised - rise).abs() > 1e-9 {
block_ops.push(Op::new("Ts", vec![n(rise)]));
}
block_ops.push(Op::new("ET", vec![]));
(state.char_spacing, state.word_spacing) = (char_spacing, word_spacing);
block_ops.push(Op::new("EMC", vec![]));
segment_width = (shaped.advance * size + extra) * state.scale * source.origin.k;
if let Some(shaped) = shaped_runs.get(&(row_index, pos)) {
let resources = glyph_resources.get(&(row_index, pos)).ok_or_else(|| EditError::Invalid("shaped font is missing".into()))?;
let drawing = ShapedRunDrawing {
doc,
fonts: &fonts,
shaped: &shaped.run,
resources,
text: &string,
matrix: tm,
size: size_of(source, item.1),
thai: string.chars().any(|c| matches!(u32::from(c), 0x0e00..=0x0e7f)),
};
segment_width = drawing.draw(&mut block_ops, &mut state)? * state.scale * source.origin.k;
} else {
block_ops.push(Op::new("BT", vec![]));
block_ops.extend(state.ops());
crates/edit/src/ungroup.rs
+762
View File
@@ -0,0 +1,762 @@
//! Edit a PDF ▸ Ungroup: draw a Form XObject's content in place of its `Do` (#2964, #3005).
//!
//! Text inside grouped Form artwork can't be edited, because editing rewrites only the page's own
//! content streams. Ungrouping copies the form's content into the page at the same place,
//! wrapped as `q <Matrix> cm <BBox> re W n … Q`, with the form's resources merged into a copy of
//! the page's under names that don't clash. Its text then edits like any other. The form object
//! itself is left alone, so another page or placement that draws it is unchanged. Each resource
//! name is read from the operand the renderer uses ([`resource_operand`]), so a surplus operand
//! before it changes neither the renaming nor the checks below.
//!
//! Drawing a form inline paints the same pixels unless the form is a transparency group whose
//! compositing differs from painting its parts one by one: a knockout group, a group that uses
//! a blend mode other than Normal inside, or a group drawn translucent, blended or under a soft
//! mask, or a group with a blending colour space (`/CS`) other than the page's: its contents
//! composite in that space and are then converted (ISO 32000-2 §11.6.6), inline they would
//! composite in the page's. Those are refused rather than changed, as is a form with resources
//! of its own whose `/DefaultGray`, `/DefaultRGB` or `/DefaultCMYK` differs from the page's, one
//! having it and the other not included (it remaps the device colours painted with it,
//! §8.6.5.6, and inline the page's resources apply), tagged form content (its structure link
//! would be lost) and content whose marked content or text objects don't balance (inlined, they
//! would change the scope of the page's own). A form in a layer (`/OC`) keeps its membership as `/OC … BDC … EMC`.
//!
//! A pattern's matrix maps pattern space to the default space of the content stream that uses it
//! (ISO 32000-2 §8.7.2): inside the form that is the form's space at the `Do`, on the page it is
//! the page's. So each pattern the form selects with `scn`/`SCN` is drawn from a copy whose
//! `/Matrix` also carries the form's placement; one that can't be copied that way is refused.
//!
//! Edit text's preparation of grouped text (`text::forms`) opens each placement with this same
//! code, and decides what to open from what [`Refusal`] and [`Ungrouped::shows_text`] say.
use std::collections::HashMap;
use pdfcraft_content::{Matrix, Op, parse};
use pdfcraft_cos::{DecodeBudget, Dict, Document, Object};
use crate::EditError;
/// The largest form content ungrouped (decoded bytes): the page's own content cap is far above
/// any real form, and a bomb must not be copied into the page. Ungroup reads the form and what it
/// draws within one [`DecodeBudget`] of this size.
pub(crate) const MAX_FORM_BYTES: usize = 64 << 20;
/// Resource categories a content stream refers to by name (ISO 32000-2 §7.8.3, Table 34).
pub(crate) const CATEGORIES: [&[u8]; 7] = [b"Font", b"XObject", b"ExtGState", b"ColorSpace", b"Pattern", b"Shading", b"Properties"];
/// The `/ColorSpace` entries that remap device colours painted with the resources they are in
/// (ISO 32000-2 §8.6.5.6).
pub(crate) const DEFAULT_SPACES: [&[u8]; 3] = [b"DefaultGray", b"DefaultRGB", b"DefaultCMYK"];
/// The most distinct patterns one ungrouped form may select; each becomes a new object.
const MAX_MOVED_PATTERNS: usize = 256;
/// The most entries of one resource category merged: a form or page with more is refused.
const MAX_RESOURCES: usize = 4096;
/// The deepest `q` nesting a form's content may have.
const MAX_SAVES: usize = 4096;
/// A name map for one resource category: form name → name in the page's resources.
type Renames = Vec<(Vec<u8>, Vec<u8>)>;
/// What [`ungroup_ops`] gives back.
pub(crate) struct Ungrouped {
/// The ops that draw the form inline.
pub(crate) ops: Vec<Op>,
/// The page resources with the form's merged in.
pub(crate) resources: Dict,
/// The pattern copies to add to the document, each under the name in the merged `/Pattern`
/// dictionary that the ops already use.
pub(crate) patterns: Vec<(Vec<u8>, Object)>,
/// Whether the form shows text, in its own content or in a Form XObject it draws (at any
/// depth [`MAX_NESTING`] allows). Text inside a tiling pattern or Type 3 glyph doesn't count:
/// inlining the form doesn't make it editable. `None` when something it draws can't be read,
/// is nested too deep, draws itself, or is past the reads or the decode budget allowed.
pub(crate) shows_text: Option<bool>,
}
/// Why [`ungroup_ops`] left a placement grouped.
pub(crate) struct Refusal {
pub(crate) error: EditError,
/// The form itself is the reason (malformed, tagged, a knockout group, and so on), so every
/// placement of it is refused the same way. Otherwise something about this placement or the
/// page so far is: its translucency or matrix, the resources it would merge into (which grow
/// as other groups are prepared), or the decode budget left.
pub(crate) intrinsic: bool,
/// The form's resources need renaming where it is drawn, and it draws something that would
/// look them up by their old names. Names merged in for other forms can make the renaming
/// unnecessary, which [`merges_without_renaming`] checks without reading the content.
pub(crate) renaming: bool,
}
/// Ungroup the form drawn by the `Do` at `page_ops[at]` (the page's streams parsed as one, the
/// indexes `page_images` gives) from the page's `resources`. `page_space` is the page group's
/// blending colour space ([`page_blend_space`]). `placement` maps the form's space to the page's
/// default space at the `Do` (its `/Matrix`, then the CTM there). The form, and the forms and
/// patterns it draws, decode within `budget`.
pub(crate) fn ungroup_ops(
doc: &Document,
resources: &Dict,
page_ops: &[Op],
at: usize,
page_space: Option<&Object>,
placement: Matrix,
budget: &mut DecodeBudget,
) -> Result<Ungrouped, Refusal> {
let refusal = |why: &str, intrinsic: bool| Refusal {
error: EditError::Invalid(format!("this artwork can't be ungrouped: {why}")),
intrinsic,
renaming: false,
};
// Something about this placement or the page so far is the reason.
let refuse = |why: &str| refusal(why, false);
// The form itself is the reason: every placement of it is refused the same way.
let invalid = |why: &str| refusal(why, true);
let op = page_ops.get(at).filter(|op| op.is("Do")).ok_or_else(|| refuse("it is missing"))?;
// The renderer draws a `Do`'s last operand, and the placement was read for its first
// (`page_images`): with more than one they can be different forms.
if op.operands.len() != 1 {
return Err(refuse("it is drawn with an operand too many"));
}
let name = op.name(0).ok_or_else(|| refuse("it is missing"))?;
let (translucent, open) = state_at(doc, resources, page_ops, at);
// Inline, its clip would take in the page's unfinished path and its text objects would be
// nested in the page's.
if open {
return Err(refuse("it is drawn inside an unfinished path or text object"));
}
let xobjects = resources.get(b"XObject").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).ok_or_else(|| refuse("it is missing"))?;
let form = xobjects.get(name).ok_or_else(|| refuse("it is missing"))?;
let Object::Stream(s) = &*doc.resolve(form) else { return Err(refuse("it is not a Form XObject")) };
if !is_name(doc, &s.dict, b"Subtype", b"Form") {
return Err(refuse("it is not a Form XObject"));
}
let numbers = |key: &[u8], n: usize| -> Option<Vec<f64>> {
let a = s.dict.get(key).map(|o| doc.resolve(o))?;
let a = a.as_array()?;
// Every entry must be a number: a short array padded out by a non-number is unreadable.
let v: Vec<f64> = a.iter().filter_map(|x| doc.resolve(x).as_f64()).collect();
(a.len() == n && v.len() == n && v.iter().all(|x| x.is_finite())).then_some(v)
};
let bbox = numbers(b"BBox", 4).and_then(|b| <[f64; 4]>::try_from(b).ok()).ok_or_else(|| invalid("its bounding box is missing or unreadable"))?;
let matrix = match s.dict.get(b"Matrix") {
None => None,
Some(_) => Some(Matrix(numbers(b"Matrix", 6).and_then(|m| <[f64; 6]>::try_from(m).ok()).ok_or_else(|| invalid("its matrix is unreadable"))?)),
};
let own = s.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned());
if s.dict.get(b"Resources").is_some() && own.is_none() {
return Err(invalid("its resources can't be read"));
}
// A form without its own resources uses the ones it is drawn with (PDF 1.1).
let form_res = own.clone().unwrap_or_else(|| resources.clone());
// A tagged form's content is a structure element's: drawn inline it would lose that link.
if s.dict.get(b"StructParent").is_some() || s.dict.get(b"StructParents").is_some() {
return Err(invalid("it is tagged content (Tags panel); edit its text in a copy without tags"));
}
let group_dict = s.dict.get(b"Group").map(|g| doc.resolve(g)).and_then(|g| g.as_dict().cloned());
if s.dict.get(b"Group").is_some() && group_dict.is_none() {
return Err(invalid("its group dictionary can't be read"));
}
let group = group_dict.filter(|g| is_name(doc, g, b"S", b"Transparency"));
if let Some(group) = &group {
if group.get(b"K").is_some_and(|k| matches!(*doc.resolve(k), Object::Bool(true))) {
return Err(invalid("it is a knockout transparency group"));
}
if translucent {
return Err(refuse("it is a transparency group drawn translucent, blended or under a soft mask"));
}
// Its contents composite in its own colour space (ISO 32000-2 §11.6.6, Table 145 /CS);
// inline they would composite in the page's.
if let Some(cs) = group.get(b"CS")
&& !same_blend_space(doc, cs, page_space)
{
return Err(invalid("it is a transparency group that blends in a colour space of its own"));
}
}
// Strictly: damaged data must not be copied in as a shorter form.
let data = budget.decode_strict(doc, s).map_err(|e| refuse(&format!("its content can't be read ({e})")))?;
let parsed = parse(&data);
if parsed.skipped > 0 {
return Err(invalid("its content has parts that can't be read"));
}
let mut ops = parsed.ops;
// What it draws is read for text whether it is a group or not, and for blend modes only in a
// group: without one, blending inside draws the same inline. The form itself is being read,
// so drawing itself again is content that can't be checked.
let mut walk =
BlendWalk { path: form.as_ref().into_iter().collect(), known: HashMap::new(), reads: 0, budget, text: false, group: group.is_some() };
let unchecked = blends_inside(doc, &form_res, &ops, 0, &mut walk);
if unchecked && group.is_some() {
return Err(refuse("it is a transparency group with blend modes inside, or content inside that can't be checked"));
}
let shows_text = (!unchecked).then_some(walk.text);
// A form's default colour space remaps its device colours (ISO 32000-2 §8.6.5.6); inline,
// the page's resources are the current ones (§7.8.3).
if own.is_some()
&& let Some((key, form_has)) = unshared_default_space(doc, &form_res, resources)
{
let key = String::from_utf8_lossy(key);
return Err(refuse(&if form_has {
format!("it remaps its device colours with a /{key} the page doesn't share")
} else {
format!("the page's /{key} would remap its device colours, which it draws as they are")
}));
}
// Merge the form's resources into a copy of the page's, renaming what clashes.
let mut merged = resources.clone();
let mut renames: Vec<(&[u8], Renames)> = Vec::new();
if own.is_some() {
for cat in CATEGORIES {
let Some(theirs) = form_res.get(cat).map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()) else { continue };
if theirs.len() > MAX_RESOURCES {
return Err(invalid("too many resources"));
}
let mut mine = merged.get(cat).map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
if mine.len() > MAX_RESOURCES {
return Err(refuse("too many resources"));
}
let mut map = Renames::new();
let mut n = 0u32;
for (key, value) in theirs.iter() {
// A default colour space checked above is the page's, which applies inline;
// renamed, it would be an entry nothing uses.
if mine.get(key) == Some(value) || (cat == b"ColorSpace" && DEFAULT_SPACES.contains(&&key[..])) {
continue;
}
let fresh = loop {
n = n.saturating_add(1);
let candidate = format!("PCUg{n}").into_bytes();
if mine.get(&candidate).is_none() && theirs.get(&candidate).is_none() {
break candidate;
}
if n == u32::MAX {
return Err(refuse("too many resources"));
}
};
mine.set(fresh.clone(), value.clone());
map.push((key.to_vec(), fresh));
}
// Counted after the merge: a page taken past the limit would refuse every later form,
// even one whose resources are all the page's already.
if mine.len() > MAX_RESOURCES {
return Err(refuse("too many resources"));
}
merged.set(cat.to_vec(), Object::Dict(mine));
if !map.is_empty() {
renames.push((cat, map));
}
}
}
let renamed = |cat: &[u8], name: &[u8]| -> Option<Vec<u8>> {
renames.iter().find(|(c, _)| *c == cat).and_then(|(_, m)| m.iter().find(|(from, _)| from == name)).map(|(_, to)| to.clone())
};
// A nested form, Type 3 font or tiling pattern without resources of its own looks its names
// up in the form's (the renderer chains them to whatever draws it). Drawn from the page, it
// would find the page's, which differ wherever a name was renamed (every form resource that
// isn't already the page's, the nested item's own entry included).
if !renames.is_empty() && ops.iter().any(|op| inherits_resources(doc, &form_res, op, budget)) {
return Err(Refusal { renaming: true, ..refuse("it draws a nested form, Type 3 font or pattern that uses its resources") });
}
let mut depth = 0usize;
// Marked content and text objects must balance within the form's own stream: an extra EMC
// would close one the page opened, and one left open would take in the page's later content.
let mut marked = 0usize;
let mut in_text = false;
for op in &mut ops {
match op.op.as_slice() {
b"BDC" | b"BMC" => marked = marked.saturating_add(1),
b"EMC" => marked = marked.checked_sub(1).ok_or_else(|| invalid("its content ends marked content it didn't begin"))?,
b"BT" if in_text => return Err(invalid("its content begins a text object inside another")),
b"BT" => in_text = true,
b"ET" if !in_text => return Err(invalid("its content ends a text object it didn't begin")),
b"ET" => in_text = false,
_ => {}
}
match op.op.as_slice() {
b"q" if depth >= MAX_SAVES => return Err(invalid("its graphics states are nested too deeply")),
b"q" => depth += 1,
b"Q" => depth = depth.checked_sub(1).ok_or_else(|| invalid("its content restores more graphics states than it saves"))?,
_ => {}
}
let slot = resource_operand(op).map_err(|Unreadable| invalid("its content selects a resource with operands that can't be read"))?;
if let Some((cat, i)) = slot
&& let Some(operand) = op.operands.get_mut(i)
&& let Object::Name(n) = operand
&& let Some(to) = renamed(cat, n)
{
*operand = Object::Name(to);
}
if let Some((dict, _)) = &mut op.inline {
for key in [&b"CS"[..], b"ColorSpace"] {
if let Some(Object::Name(n)) = dict.get(key)
&& let Some(to) = renamed(b"ColorSpace", n)
{
dict.set(key.to_vec(), Object::Name(to));
}
}
}
}
if marked > 0 || in_text {
return Err(invalid("its content leaves marked content or a text object open"));
}
// Patterns keep their place relative to the shapes they fill: each one the form selects is
// drawn from a copy that also carries the placement. Nested forms and patterns inside
// patterns keep their own parents, so only this stream's selections change.
let mut patterns: Vec<(Vec<u8>, Object)> = Vec::new();
// What the copies of tiling patterns may add in all.
let mut pattern_bytes = MAX_FORM_BYTES;
if placement != Matrix::IDENTITY {
let mut table = merged.get(b"Pattern").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
let mut moved: Renames = Vec::new();
let mut n = 0u32;
for op in &mut ops {
let Ok(Some((b"Pattern", i))) = resource_operand(op) else { continue };
let Some(Object::Name(from)) = op.operands.get(i) else { continue };
// A name the resources don't have selects nothing, inline or not.
let Some(entry) = table.get(from).cloned() else { continue };
let to = match moved.iter().find(|(f, _)| f == from) {
Some((_, to)) => to.clone(),
None => {
if moved.len() >= MAX_MOVED_PATTERNS {
return Err(refuse("it paints with too many patterns"));
}
let copy =
moved_pattern(doc, &entry, placement).ok_or_else(|| refuse("it paints with a pattern whose position can't be kept"))?;
if let Object::Stream(s) = &copy {
pattern_bytes = pattern_bytes.checked_sub(s.raw.len()).ok_or_else(|| refuse("its pattern copies would be too large"))?;
}
let fresh = loop {
n = n.saturating_add(1);
let candidate = format!("PCUgPat{n}").into_bytes();
if table.get(&candidate).is_none() {
break candidate;
}
if n == u32::MAX {
return Err(refuse("too many resources"));
}
};
// Reserved here; the caller replaces it with the copy's reference.
table.set(fresh.clone(), Object::Null);
patterns.push((fresh.clone(), copy));
moved.push((from.clone(), fresh.clone()));
fresh
}
};
if let Some(operand) = op.operands.get_mut(i) {
*operand = Object::Name(to);
}
}
if !patterns.is_empty() {
merged.set(b"Pattern".to_vec(), Object::Dict(table));
}
}
// A form in a layer (/OC) stays in it: its content is marked with the same membership.
let layer = match s.dict.get(b"OC") {
None => None,
Some(oc) => {
let mut props = merged.get(b"Properties").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
let existing = props.iter().find(|(_, v)| *v == oc).map(|(k, _)| k.clone());
let name = match existing {
Some(k) => k,
None => {
let fresh = (1..=u32::MAX)
.map(|n| format!("PCUgOC{n}").into_bytes())
.find(|c| props.get(c).is_none())
.ok_or_else(|| refuse("too many resources"))?;
props.set(fresh.clone(), oc.clone());
merged.set(b"Properties".to_vec(), Object::Dict(props));
fresh
}
};
Some(name)
}
};
let num = |v: f64| Object::Real(v);
let mut out = Vec::new();
if let Some(name) = &layer {
out.push(Op::new("BDC", vec![Object::name("OC"), Object::Name(name.clone())]));
}
out.push(Op::new("q", Vec::new()));
if let Some(m) = matrix {
out.push(Op::new("cm", m.0.iter().copied().map(num).collect()));
}
let [a, b, c, d] = bbox;
let (x0, y0, x1, y1) = (a.min(c), b.min(d), a.max(c), b.max(d));
out.push(Op::new("re", vec![num(x0), num(y0), num(x1 - x0), num(y1 - y0)]));
out.push(Op::new("W", Vec::new()));
out.push(Op::new("n", Vec::new()));
out.extend(ops);
// A path the form leaves unfinished ends with it, not in the page's content after it.
out.push(Op::new("n", Vec::new()));
// A form ends with the state it was drawn with (the Do saves and restores it).
out.extend(std::iter::repeat_with(|| Op::new("Q", Vec::new())).take(depth));
out.push(Op::new("Q", Vec::new()));
if layer.is_some() {
out.push(Op::new("EMC", Vec::new()));
}
Ok(Ungrouped { ops: out, resources: merged, patterns, shows_text })
}
/// Whether form `name` in `resources` would merge into them without a rename: it has resources
/// of its own, and each of their entries is already in `resources` under the same name, as
/// [`ungroup_ops`] compares them. Only dictionaries are read, never the form's content.
pub(crate) fn merges_without_renaming(doc: &Document, resources: &Dict, name: &[u8]) -> bool {
let category = |d: &Dict, cat: &[u8]| d.get(cat).map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned());
let Some(form) = category(resources, b"XObject").and_then(|x| x.get(name).cloned()) else { return false };
let Object::Stream(s) = &*doc.resolve(&form) else { return false };
let Some(own) = s.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned()) else { return false };
CATEGORIES.into_iter().all(|cat| {
let Some(theirs) = category(&own, cat) else { return true };
let mine = category(resources, cat).unwrap_or_default();
// Past these sizes `ungroup_ops` refuses the form anyway. Default colour spaces are
// never renamed (`ungroup_ops` checks them separately).
theirs.len() <= MAX_RESOURCES
&& mine.len() <= MAX_RESOURCES
&& theirs.iter().all(|(key, value)| mine.get(key) == Some(value) || (cat == b"ColorSpace" && DEFAULT_SPACES.contains(&&key[..])))
})
}
/// The first of `/DefaultGray`, `/DefaultRGB` and `/DefaultCMYK` that the form's and the page's
/// `/ColorSpace` resources don't share with an equal value (the same object, or the same value
/// resolved), and whether the form has it (else only the page does). A default colour space
/// remaps the device colours painted with the resource dictionary it is in (ISO 32000-2
/// §8.6.5.6, §7.8.3), so inline only the page's would apply: the form's would no longer remap
/// its colours, and the page's would start to.
fn unshared_default_space(doc: &Document, form: &Dict, page: &Dict) -> Option<(&'static [u8], bool)> {
let spaces = |res: &Dict| res.get(b"ColorSpace").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
let (theirs, mine) = (spaces(form), spaces(page));
let differs = |key: &[u8]| match (theirs.get(key), mine.get(key)) {
(None, None) => false,
(Some(v), Some(p)) => *doc.resolve(p) != *doc.resolve(v),
_ => true,
};
// The form's own entries are named first.
let form_has = DEFAULT_SPACES.into_iter().find(|key| theirs.get(key).is_some() && differs(key)).map(|key| (key, true));
form_has.or_else(|| DEFAULT_SPACES.into_iter().find(|key| differs(key)).map(|key| (key, false)))
}
/// A copy of pattern `entry` whose `/Matrix` maps its space to the page's default space instead
/// of the form's: its own matrix, then `placement`. `None` when it isn't a readable pattern, its
/// matrix is unreadable, the result isn't finite or a tiling pattern is too large to copy.
fn moved_pattern(doc: &Document, entry: &Object, placement: Matrix) -> Option<Object> {
let resolved = doc.resolve(entry);
let (mut dict, stream) = match &*resolved {
Object::Stream(s) => (s.dict.clone(), Some(s)),
Object::Dict(d) => (d.clone(), None),
_ => return None,
};
let own = match dict.get(b"Matrix") {
None => Matrix::IDENTITY,
Some(m) => {
let m = doc.resolve(m);
let nums: Vec<Object> = m.as_array()?.iter().map(|x| doc.resolve(x).as_ref().clone()).collect();
// A matrix is six numbers: a longer array is unreadable, not cut short.
if nums.len() != 6 {
return None;
}
Matrix::from_operands(&nums)?
}
};
let moved = own.then(&placement);
if !moved.0.iter().all(|v| v.is_finite()) {
return None;
}
dict.set(b"Matrix".to_vec(), Object::Array(moved.0.iter().map(|v| Object::Real(*v)).collect()));
match stream {
Some(s) if s.raw.len() > MAX_FORM_BYTES => None,
Some(s) => Some(Object::Stream(pdfcraft_cos::Stream { dict, raw: s.raw.clone() })),
None => Some(Object::Dict(dict)),
}
}
/// How deep [`blends_inside`] follows nested forms; deeper ones count as blending.
const MAX_NESTING: usize = 8;
/// The most nested forms and tiling patterns [`blends_inside`] reads in all; past that the rest
/// count as blending, so a file can't make the check run for ever.
const MAX_NESTED_READS: usize = 1024;
/// What [`blends_inside`] carries through its nested reads: the forms on the current path (a
/// form drawing itself), the answers for each form with resources of its own (whether it blends
/// or can't be checked, and whether it shows text; neither depends on who draws it), how many
/// reads it has made, the decode budget they share with the form itself, whether the content
/// being walked shows text itself or through a Form XObject it draws, and whether blend modes
/// count (they do in a group).
struct BlendWalk<'a> {
path: Vec<pdfcraft_cos::ObjRef>,
known: HashMap<pdfcraft_cos::ObjRef, (bool, bool)>,
reads: usize,
budget: &'a mut DecodeBudget,
text: bool,
group: bool,
}
/// Whether `op` shows text (ISO 32000-2 §9.4.3, Table 107).
fn shows_text(op: &Op) -> bool {
matches!(op.op.as_slice(), b"Tj" | b"TJ" | b"'" | b"\"")
}
/// An operator whose operands the renderer can't read as it expects, so it skips the operator.
pub(crate) struct Unreadable;
/// Which operand of `op` names a resource, and from which category, read as the renderer reads
/// it: operands are taken from the end, so a surplus operand before them is ignored. `Do`, `gs`,
/// `sh`, `cs` and `CS` take the last operand; `Tf` a name then a size; `BDC` and `DP` a tag then
/// a name or an inline dictionary; `scn` and `SCN` numbers and the last name among them.
/// `Ok(None)` when `op` names no resource; [`Unreadable`] when the renderer would skip it, which
/// [`ungroup_ops`] refuses rather than guess at.
pub(crate) fn resource_operand(op: &Op) -> Result<Option<(&'static [u8], usize)>, Unreadable> {
let operands = &op.operands;
let name_at = |i: Option<usize>| i.is_some_and(|i| matches!(operands.get(i), Some(Object::Name(_))));
let last = operands.len().checked_sub(1);
let (cat, at): (&'static [u8], Option<usize>) = match op.op.as_slice() {
b"Do" => (b"XObject", last),
b"gs" => (b"ExtGState", last),
b"sh" => (b"Shading", last),
b"cs" | b"CS" => (b"ColorSpace", last),
b"Tf" => {
if !last.is_some_and(|i| matches!(operands.get(i), Some(Object::Int(_) | Object::Real(_)))) {
return Err(Unreadable);
}
(b"Font", operands.len().checked_sub(2))
}
b"BDC" | b"DP" => {
if !name_at(operands.len().checked_sub(2)) {
return Err(Unreadable);
}
match last.and_then(|i| operands.get(i)) {
Some(Object::Name(_)) => (b"Properties", last),
Some(Object::Dict(_)) => return Ok(None),
_ => return Err(Unreadable),
}
}
b"scn" | b"SCN" => {
if !operands.iter().all(|o| matches!(o, Object::Int(_) | Object::Real(_) | Object::Name(_))) {
return Err(Unreadable);
}
return Ok(operands.iter().rposition(|o| matches!(o, Object::Name(_))).map(|i| (&b"Pattern"[..], i)));
}
_ => return Ok(None),
};
match at {
Some(i) if name_at(Some(i)) => Ok(Some((cat, i))),
_ => Err(Unreadable),
}
}
/// Whether `op` draws a Form XObject (`Do`), Type 3 font (`Tf`) or tiling pattern (`scn`/`SCN`) from
/// `resources` that has no `/Resources` of its own, and so inherits the ones it is drawn with. A
/// form or pattern whose content names no resource inherits nothing (a filled shape, say), so it
/// needs no renaming; its content is read within `budget`. A Type 3 font's glyphs are not read,
/// and always count.
fn inherits_resources(doc: &Document, resources: &Dict, op: &Op, budget: &mut DecodeBudget) -> bool {
let Ok(Some((cat @ (b"XObject" | b"Font" | b"Pattern"), i))) = resource_operand(op) else { return false };
let Some(Object::Name(name)) = op.operands.get(i) else { return false };
let Some(entry) = resources.get(cat).map(|d| doc.resolve(d)).and_then(|d| d.as_dict().and_then(|d| d.get(name).cloned())) else {
return false;
};
let entry = doc.resolve(&entry);
let (dict, stream) = match &*entry {
Object::Stream(s) => (&s.dict, Some(s)),
Object::Dict(d) => (d, None),
_ => return false,
};
let drawn = match cat {
b"XObject" => is_name(doc, dict, b"Subtype", b"Form"),
b"Font" => is_name(doc, dict, b"Subtype", b"Type3"),
// A shading pattern has no content of its own.
_ => int_of(doc, dict, b"PatternType") == Some(1),
};
drawn && dict.get(b"Resources").is_none() && (cat == b"Font" || stream.is_none_or(|s| names_resources(doc, s, budget)))
}
/// Whether content stream `s` names a resource (or can't be read within `budget` to tell): device
/// colour spaces are not resources; an inline image counts, as its colour space may be one.
fn names_resources(doc: &Document, s: &pdfcraft_cos::Stream, budget: &mut DecodeBudget) -> bool {
let Ok(data) = budget.decode_strict(doc, s) else { return true };
let parsed = parse(&data);
parsed.skipped > 0
|| parsed.ops.iter().any(|op| match resource_operand(op) {
Ok(None) => op.inline.is_some(),
Ok(Some((b"ColorSpace", i))) => {
!matches!(op.operands.get(i), Some(Object::Name(n)) if [&b"DeviceGray"[..], b"DeviceRGB", b"DeviceCMYK", b"Pattern"].contains(&n.as_slice()))
}
_ => true,
})
}
/// Whether `ops` (drawn with `resources`) blend with something other than Normal, which counts
/// only in a group (`walk.group`): a `gs` they use, or one used by a form or pattern they draw, at
/// any depth. A shading pattern blends through its own `/ExtGState`, a tiling pattern through its
/// content. A nested form or pattern that can't be read, is nested too deep, draws itself or is
/// past the reads or the budget allowed counts as blending, so the caller refuses rather than
/// guesses. An operator the renderer skips ([`Unreadable`]) draws nothing and is skipped. Sets
/// `walk.text` when `ops` show text, or a Form XObject they draw does; text in a tiling pattern's
/// cell is not the page's text and is not counted.
fn blends_inside(doc: &Document, resources: &Dict, ops: &[Op], depth: usize, walk: &mut BlendWalk<'_>) -> bool {
let category = |cat: &[u8]| resources.get(cat).map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
let (states, xobjects, patterns) = (category(b"ExtGState"), category(b"XObject"), category(b"Pattern"));
let group = walk.group;
let state_blends = |g: Option<&Object>| group && g.map(|g| doc.resolve(g)).and_then(|g| g.as_dict().cloned()).is_some_and(|gs| blends(doc, &gs));
if ops.iter().any(shows_text) {
walk.text = true;
}
for op in ops {
let Ok(Some((cat, i))) = resource_operand(op) else { continue };
let Some(name) = op.name(i) else { continue };
let o = match cat {
b"ExtGState" if state_blends(states.get(name)) => return true,
b"XObject" => xobjects.get(name),
b"Pattern" => patterns.get(name),
_ => continue,
};
let Some(o) = o else { continue };
let resolved = doc.resolve(o);
let nested = match &*resolved {
Object::Stream(st) if op.is("Do") && is_name(doc, &st.dict, b"Subtype", b"Form") => st,
Object::Stream(st) if !op.is("Do") && int_of(doc, &st.dict, b"PatternType") == Some(1) => st,
// A shading pattern.
Object::Stream(st) if !op.is("Do") => {
if state_blends(st.dict.get(b"ExtGState")) {
return true;
}
continue;
}
Object::Dict(d) if !op.is("Do") => {
if state_blends(d.get(b"ExtGState")) {
return true;
}
continue;
}
_ => continue,
};
let r = o.as_ref();
let drawn_form = op.is("Do");
if let Some((known, text)) = r.and_then(|r| walk.known.get(&r)).copied() {
if known {
return true;
}
walk.text |= drawn_form && text;
continue;
}
if depth >= MAX_NESTING || r.is_some_and(|r| walk.path.contains(&r)) || walk.reads >= MAX_NESTED_READS {
return true;
}
walk.reads += 1;
let Ok(data) = walk.budget.decode_strict(doc, nested) else { return true };
let own_resources = nested.dict.get(b"Resources").map(|r| doc.resolve(r)).and_then(|r| r.as_dict().cloned());
let has_own = own_resources.is_some();
let own = own_resources.unwrap_or_else(|| resources.clone());
let inner = parse(&data);
if inner.skipped > 0 {
return true;
}
if let Some(r) = r {
walk.path.push(r);
}
let outer_text = std::mem::replace(&mut walk.text, false);
let blends = blends_inside(doc, &own, &inner.ops, depth + 1, walk);
let inner_text = walk.text;
walk.text = outer_text || (drawn_form && inner_text);
if let Some(r) = r {
walk.path.retain(|x| *x != r);
if has_own {
walk.known.insert(r, (blends, inner_text));
}
}
if blends {
return true;
}
}
false
}
/// The blending colour space of the page group (ISO 32000-2 §11.4.7): the page's transparency
/// `/Group /CS`, resolved. `None` when the page names none, and the PDF processor chooses.
pub(crate) fn page_blend_space(doc: &Document, page: &Dict) -> Option<Object> {
let group = page.get(b"Group").map(|g| doc.resolve(g)).and_then(|g| g.as_dict().cloned()).filter(|g| is_name(doc, g, b"S", b"Transparency"))?;
group.get(b"CS").map(|cs| (*doc.resolve(cs)).clone())
}
/// Whether a group blending in `cs` composites as the page does: in the page group's own space
/// when it names one, else in DeviceRGB, the space viewers composite a page in on screen.
fn same_blend_space(doc: &Document, cs: &Object, page_space: Option<&Object>) -> bool {
let cs = doc.resolve(cs);
match page_space {
Some(page) => *cs == *page,
None => cs.as_name() == Some(b"DeviceRGB"),
}
}
/// Whether `d`'s `key` is the name `want`, directly or through an indirect object (any value may
/// be one, ISO 32000-2 §7.3.10).
fn is_name(doc: &Document, d: &Dict, key: &[u8], want: &[u8]) -> bool {
d.get(key).is_some_and(|v| doc.resolve(v).as_name() == Some(want))
}
/// `d`'s integer `key`, directly or through an indirect object.
fn int_of(doc: &Document, d: &Dict, key: &[u8]) -> Option<i64> {
d.get(key).and_then(|v| doc.resolve(v).as_int())
}
/// Whether an ExtGState blends with something other than Normal.
fn blends(doc: &Document, gs: &Dict) -> bool {
let Some(bm) = gs.get(b"BM").map(|b| doc.resolve(b)) else { return false };
let first = match &*bm {
Object::Array(a) => a.first().map(|x| doc.resolve(x)),
_ => Some(bm.clone()),
};
!first.is_some_and(|b| matches!(b.as_name(), Some(b"Normal" | b"Compatible")))
}
/// The state at op `at` of the page's streams parsed as one: whether the graphics state is
/// translucent, blended or soft-masked there, following `q`/`Q` and `gs` (each `gs` changes only
/// the parameters it sets), which a transparency group composites as a whole; and whether a path
/// or a text object is open there.
fn state_at(doc: &Document, resources: &Dict, ops: &[Op], at: usize) -> (bool, bool) {
let states = resources.get(b"ExtGState").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
// Fill alpha below one, stroke alpha below one, a soft mask, a blend mode other than Normal.
let mut stack: Vec<[bool; 4]> = Vec::new();
let mut now = [false; 4];
let (mut path, mut text) = (false, false);
for op in ops.iter().take(at) {
match op.op.as_slice() {
b"m" | b"l" | b"c" | b"v" | b"y" | b"re" | b"h" | b"W" | b"W*" => path = true,
b"S" | b"s" | b"f" | b"F" | b"f*" | b"B" | b"B*" | b"b" | b"b*" | b"n" => path = false,
b"BT" => text = true,
b"ET" => text = false,
b"q" => stack.push(now),
// A Q with nothing to restore is ignored, as renderers ignore it.
b"Q" => now = stack.pop().unwrap_or(now),
b"gs" => {
let Ok(Some((_, i))) = resource_operand(op) else { continue };
let Some(gs) = op.name(i).and_then(|n| states.get(n)).map(|g| doc.resolve(g)).and_then(|g| g.as_dict().cloned()) else {
continue;
};
let alpha = |key: &[u8]| gs.get(key).map(|v| doc.resolve(v)).and_then(|v| v.as_f64()).map(|a| a < 1.0);
if let Some(v) = alpha(b"ca") {
now[0] = v;
}
if let Some(v) = alpha(b"CA") {
now[1] = v;
}
if let Some(m) = gs.get(b"SMask").map(|v| doc.resolve(v)) {
now[2] = m.as_name() != Some(b"None");
}
if gs.get(b"BM").is_some() {
now[3] = blends(doc, &gs);
}
}
_ => {}
}
}
(now.iter().any(|v| *v), path || text)
}
crates/engine/src/compare.rs
+5 -3
View File
@@ -333,11 +333,13 @@ impl crate::Document {
let (pages, truncated_pages) =
self.export_pages_checked(!matches!(format, OfficeFormat::Rtf | OfficeFormat::Markdown), MAX_EXPORT_IMAGE_BYTES);
let title = self.info.title.clone().unwrap_or_else(|| self.name.trim_end_matches(".pdf").to_string());
// Word and HTML name the document's language (`/Lang`), so spelling is checked in it.
let language = self.initial_view().language;
let bytes = match format {
OfficeFormat::Docx => pdfcraft_export::docx(&pages, &title),
OfficeFormat::Docx => pdfcraft_export::docx_with_language(&pages, &title, language.as_deref()),
OfficeFormat::Pptx => pdfcraft_export::pptx(&pages, &title),
OfficeFormat::Html => pdfcraft_export::html(&pages, &title).into_bytes(),
OfficeFormat::HtmlSimple => pdfcraft_export::html_simple(&pages, &title).into_bytes(),
OfficeFormat::Html => pdfcraft_export::html_with_language(&pages, &title, language.as_deref()).into_bytes(),
OfficeFormat::HtmlSimple => pdfcraft_export::html_simple_with_language(&pages, &title, language.as_deref()).into_bytes(),
OfficeFormat::Rtf => pdfcraft_export::rtf(&pages).into_bytes(),
OfficeFormat::Markdown => pdfcraft_export::markdown(&pages).into_bytes(),
};
crates/engine/src/doc_state.rs
+3
View File
@@ -26,6 +26,9 @@ pub enum ImageEdit {
Arrange(ArrangeOrder),
/// Remove any private crop on the image.
RemoveCrop,
/// Grouped Form artwork: draw its content in place, so its text and pictures can be edited
/// one by one (#2964).
Ungroup,
}
/// Forms ▸ Merge data files into spreadsheet: the field values of each file (FDF, XFDF or a
crates/engine/src/document.rs
+9
View File
@@ -260,6 +260,15 @@ impl Document {
pdfcraft_create::image_file(&editor.cos, img.object.ok_or("the image has no object")?)
}
/// Edit text: whether grouped artwork `index` on `page` (0-based, as [`Self::page_images`]
/// lists them) shows text that [`Edit::PrepareText`](crate::Edit::PrepareText) would make
/// editable. Reads the page without changing it; `false` for a logo, a group that can't be
/// opened or a page that can't be read.
pub fn grouped_text_at(&self, page: usize, index: usize) -> bool {
let Some(editor) = self.editor.as_ref() else { return false };
crate::guard(|| pdfcraft_edit::text::forms::grouped_text_at(&editor.cos, page, index)).is_ok_and(|r| r.is_ok_and(|yes| yes))
}
/// Edit a PDF ▸ Edit text: the paragraphs on `page` (0-based).
pub fn text_blocks(&self, page: usize) -> Vec<pdfcraft_edit::TextBlock> {
self.editor.as_ref().and_then(|e| e.text_blocks(page).ok()).map(|b| b.as_ref().clone()).unwrap_or_default()
crates/engine/src/edit.rs
+17
View File
@@ -457,6 +457,12 @@ pub enum Edit {
objects: Vec<ObjectTarget>,
offset: [f64; 2],
},
/// Delete mixed existing and added objects together (references from one inventory of
/// `page`), as one step: what Delete and Cut do to objects selected together.
DeleteObjects {
page: usize,
objects: Vec<ObjectTarget>,
},
/// Edit text in a paragraph box: replace paragraph `block` (from `Document::text_blocks`),
/// rewrapped to the box's width.
EditTextBlock {
@@ -466,6 +472,10 @@ pub enum Edit {
/// Formatting changes (font, size, colour, alignment); default keeps the paragraph's.
style: pdfcraft_edit::BlockStyle,
},
/// Expose supported grouped source text on one page for direct editing.
PrepareText {
page: usize,
},
/// An explicit open-licensed full font program, completing subsets or selected by a user.
RegisterTextFont {
page: usize,
@@ -545,6 +555,13 @@ pub enum Edit {
content: AddedContent,
file: Option<(String, Arc<Vec<u8>>)>,
},
/// Edit ▸ Paste of objects copied together in Edit a PDF ([`Document::copy_objects`](crate::Document::copy_objects)), in this
/// document or another: drawn on top of `page` as they were, moved by `offset` (user space).
PasteObjects {
page: usize,
clip: Arc<crate::ObjectClip>,
offset: [f64; 2],
},
/// Apply redaction marks (all, or those on `pages`): remove what they cover for good.
ApplyRedactions {
pages: Option<Vec<usize>>,
crates/engine/src/edit/labels.rs
+5 -2
View File
@@ -111,6 +111,7 @@ pub const EDIT_LABELS: &[&str] = &[
"Delete document JavaScript",
"Edit document JavaScript",
"Edit text",
"Prepare grouped text",
"Load document font",
"Move object",
"Move objects",
@@ -269,6 +270,7 @@ impl Edit {
Edit::SetDocumentScript { script: None, .. } => Label::of("Delete document JavaScript"),
Edit::SetDocumentScript { .. } => Label::of("Edit document JavaScript"),
Edit::EditTextLine { .. } | Edit::EditTextBlock { .. } => Label::of("Edit text"),
Edit::PrepareText { .. } => Label::of("Prepare grouped text"),
Edit::RegisterTextFont { .. } => Label::of("Load document font"),
Edit::MoveObjects { objects, .. } => plural("Move object", "Move objects", objects.len()),
Edit::EditPageImage { change, .. } => Label::of(match change {
@@ -286,6 +288,7 @@ impl Edit {
pdfcraft_edit::ArrangeOrder::SendToBack => "Send to back",
},
ImageEdit::RemoveCrop => "Remove crop",
ImageEdit::Ungroup => "Ungroup artwork",
}),
Edit::AddHeaderFooter { replace: false, .. } => Label::of("Add header & footer"),
Edit::AddHeaderFooter { .. } => Label::of("Update header & footer"),
@@ -299,9 +302,9 @@ impl Edit {
Edit::AddText { .. } => Label::of("Add text"),
Edit::AddImage { .. } => Label::of("Add image"),
Edit::UpdateContent { .. } => Label::of("Edit content"),
Edit::DeleteContent { .. } => Label::of("Delete content"),
Edit::DeleteContent { .. } | Edit::DeleteObjects { .. } => Label::of("Delete content"),
Edit::ReplaceImage { .. } => Label::of("Replace image"),
Edit::PasteContent { .. } => Label::of("Paste"),
Edit::PasteContent { .. } | Edit::PasteObjects { .. } => Label::of("Paste"),
Edit::ApplyRedactions { .. } => Label::of("Apply redactions"),
Edit::ClearRedactions => Label::of("Remove redaction marks"),
Edit::RemoveHidden { .. } => Label::of("Remove hidden information"),
crates/engine/src/edit/run.rs
+20 -1
View File
@@ -111,9 +111,11 @@ pub(crate) fn check_permission(edit: &Edit, p: &pdfcraft_cos::Permissions) -> Re
| Edit::AddImage { .. }
| Edit::UpdateContent { .. }
| Edit::MoveObjects { .. }
| Edit::DeleteObjects { .. }
| Edit::DeleteContent { .. }
| Edit::ReplaceImage { .. }
| Edit::PasteContent { .. }
| Edit::PasteObjects { .. }
| Edit::ClearRedactions
| Edit::RemoveHidden { .. }
| Edit::ImportData { .. }
@@ -148,6 +150,7 @@ pub(crate) fn check_permission(edit: &Edit, p: &pdfcraft_cos::Permissions) -> Re
| Edit::RemoveStandardId { .. }
| Edit::EditTextLine { .. }
| Edit::EditTextBlock { .. }
| Edit::PrepareText { .. }
| Edit::RegisterTextFont { .. }
| Edit::EditPageImage { .. }
| Edit::Flatten { .. }
@@ -193,6 +196,9 @@ pub(crate) struct EditCtx {
/// The overlay streams the edit's applied redactions added, which alone the renderer's check
/// leaves out.
pub(crate) redaction_overlays: Vec<pdfcraft_cos::ObjRef>,
/// What [`Edit::PrepareText`] prepared, repaired and left grouped, for
/// [`Session::prepare_text`](crate::Session::prepare_text).
pub(crate) prepared: Option<pdfcraft_edit::text::forms::Preparation>,
}
impl EditCtx {
@@ -222,6 +228,7 @@ impl EditCtx {
rng: getrandom::fill,
renamed_fields: Vec::new(),
redaction_overlays: Vec::new(),
prepared: None,
}
}
@@ -306,6 +313,7 @@ fn run_edit_page_image(doc: &mut pdfcraft_cos::Document, page: usize, index: usi
ImageEdit::Delete => pdfcraft_edit::ImageChange::Delete,
ImageEdit::Arrange(order) => pdfcraft_edit::ImageChange::Arrange(*order),
ImageEdit::RemoveCrop => pdfcraft_edit::ImageChange::RemoveCrop,
ImageEdit::Ungroup => pdfcraft_edit::ImageChange::Ungroup,
};
let page_model = pdfcraft_model::pages(doc)
.into_iter()
@@ -411,7 +419,7 @@ fn run_edit_page_image(doc: &mut pdfcraft_cos::Document, page: usize, index: usi
ai.image = new_image;
pdfcraft_edit::update_content(doc, page, added_idx, &AddedContent::Image(ai))?;
}
ImageEdit::Crop(_) | ImageEdit::RemoveCrop | ImageEdit::RotateBy(_) | ImageEdit::Arrange(_) | ImageEdit::Delete => {
ImageEdit::Crop(_) | ImageEdit::RemoveCrop | ImageEdit::RotateBy(_) | ImageEdit::Arrange(_) | ImageEdit::Delete | ImageEdit::Ungroup => {
pdfcraft_edit::change_image(doc, page, index, &c)?;
}
}
@@ -723,9 +731,20 @@ pub(crate) fn run_edit(doc: &mut pdfcraft_cos::Document, edit: &Edit, cx: &mut E
Edit::EnhanceScan { images } => crate::scan_clean::write_back(doc, images)?,
Edit::EditPageImage { page, index, change } => run_edit_page_image(doc, *page, *index, change)?,
Edit::MoveObjects { page, objects, offset } => pdfcraft_edit::move_objects(doc, *page, objects, *offset)?,
Edit::DeleteObjects { page, objects } => crate::object_clip::delete(doc, *page, objects, |doc, edit| run_edit(doc, edit, cx))?,
Edit::PasteObjects { page, clip, offset } => crate::object_clip::paste(doc, *page, clip, *offset)?,
Edit::EditTextBlock { page, block, text, style } => {
pdfcraft_edit::rewrite_block(doc, *page, *block, Some(text), style)?;
}
Edit::PrepareText { page } => {
let result = pdfcraft_edit::text::forms::prepare(doc, *page)?;
if result.expanded == 0 && result.repaired_fonts == 0 {
return Err(EditError::Invalid(
result.refused.first().cloned().unwrap_or_else(|| "this page has no supported grouped text to prepare".into()),
));
}
cx.prepared = Some(result);
}
Edit::RegisterTextFont { page, bytes } => {
pdfcraft_edit::text::register_text_font(doc, *page, bytes)?;
}
crates/engine/src/history.rs
+3
View File
@@ -79,8 +79,10 @@ pub(crate) fn scope_of(edit: &Edit) -> Scope {
// Page content, resources and added content/marks only: the page tree, annotations,
// fields, links and document metadata are untouched (see `refresh_content`).
Edit::MoveObjects { .. }
| Edit::DeleteObjects { .. }
| Edit::EditTextLine { .. }
| Edit::EditTextBlock { .. }
| Edit::PrepareText { .. }
| Edit::EditPageImage { .. }
| Edit::EnhanceScan { .. }
| Edit::AddText { .. }
@@ -89,6 +91,7 @@ pub(crate) fn scope_of(edit: &Edit) -> Scope {
| Edit::DeleteContent { .. }
| Edit::ReplaceImage { .. }
| Edit::PasteContent { .. }
| Edit::PasteObjects { .. }
| Edit::AddHeaderFooter { .. }
| Edit::AddWatermark { .. }
| Edit::AddBackground { .. }
crates/engine/src/lib.rs
+2
View File
@@ -36,6 +36,7 @@ pub mod js;
mod layers;
pub mod links;
pub mod model3d;
mod object_clip;
pub mod ocr;
mod open;
pub mod optimizer;
@@ -108,6 +109,7 @@ pub use pdfcraft_edit::{Backdrop, BlockKey, BlockStyle, PageImage, RangeStyle, T
pub use pdfcraft_measure as measure;
pub use pdfcraft_xfa::Report as XfaLayout;
pub use object_clip::{ObjectClip, ObjectClipboard};
pub use pdfcraft_edit::{EditableObject, MAX_MOVE_OBJECTS, ObjectKind, ObjectTarget};
pub use pdfcraft_fonts::{MAX_SIGNATURE_CHARS, ScriptOutline, script_missing, script_outline, undrawable_message};
crates/engine/src/object_clip.rs
+211
View File
@@ -0,0 +1,211 @@
//! Edit ▸ Copy, Cut and Paste of objects selected together in Edit a PDF (#5989).
//!
//! A copy ([`ObjectClip`]) holds what the objects draw and the resources that drawing names
//! ([`pdfcraft_edit::copy_objects`]), with a snapshot of the document they came from, so it can
//! be pasted into any open document, also after the original changes or closes. Within the same
//! document, while the objects it refers to are unchanged, a paste draws them again from the same
//! fonts and images; otherwise it first copies them over ([`pdfcraft_organize::import_value`]):
//! each once, through cycles, sharing what the target already holds, and within a size limit.
use std::sync::Arc;
use pdfcraft_cos::Object;
use crate::{DocId, Document, Edit, EditError, EditableObject, ObjectTarget};
/// The most objects a paste copies into another document.
const MAX_IMPORT_OBJECTS: usize = 20_000;
/// The most stream data (fonts, images…) a paste copies into another document.
const MAX_IMPORT_BYTES: usize = 256 << 20;
/// How far each further paste onto the same page steps down and to the right, in points, as
/// pasted added content steps ([`pdfcraft_edit::added::PASTE_STEP`]).
const STEP: f64 = pdfcraft_edit::added::PASTE_STEP;
/// Objects copied from one page of a document.
#[derive(Clone)]
pub struct ObjectClip {
copied: pdfcraft_edit::CopiedObjects,
/// The document as it was when copied: what the copy's resources refer to.
source: pdfcraft_cos::Document,
doc: DocId,
page: usize,
}
impl PartialEq for ObjectClip {
fn eq(&self, other: &Self) -> bool {
self.copied == other.copied
&& self.doc == other.doc
&& self.page == other.page
&& self.source.source_identity() == other.source.source_identity()
}
}
impl std::fmt::Debug for ObjectClip {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("ObjectClip").field("doc", &self.doc).field("page", &self.page).field("objects", &self.copied.objects.len()).finish()
}
}
impl ObjectClip {
/// The objects as they were listed when copied ([`Document::editable_objects`]).
pub fn objects(&self) -> &[EditableObject] {
&self.copied.objects
}
/// The text of the copied paragraphs and text boxes, one per line.
pub fn text(&self) -> String {
self.copied.text()
}
/// The union of the objects' boxes, in the user space of the page they came from.
pub fn rect(&self) -> [f64; 4] {
self.copied.rect
}
/// The document and page the objects were copied from.
pub fn source(&self) -> (DocId, usize) {
(self.doc, self.page)
}
}
impl Document {
/// Edit ▸ Copy of `targets` (references from one [`Document::editable_objects`] inventory of
/// `page`). Refused when the document doesn't allow copying. Changes nothing.
pub fn copy_objects(&self, page: usize, targets: &[ObjectTarget]) -> Result<ObjectClip, EditError> {
self.check_copying()?;
let editor = self.editor.as_ref().ok_or_else(|| EditError::ReadOnly("the document can't be read".into()))?;
let copied = pdfcraft_edit::copy_objects(&editor.cos, page, targets)?;
Ok(ObjectClip { copied, source: editor.cos.clone(), doc: self.id, page })
}
/// Where Paste puts `clip` on `page`: the user-space offset that moves it `steps` × 18 pt down
/// and to the right as displayed, or, when that leaves nothing of it on the page's crop box,
/// the offset that centres it there.
pub fn object_paste_offset(&self, page: usize, clip: &ObjectClip, steps: u32) -> Result<[f64; 2], String> {
let step = STEP * f64::from(steps);
let offset = self.object_move_offset(page, [step, step])?;
let p = self.info.pages.get(page).ok_or("the page no longer exists")?;
let crop = p.crop.map(f64::from);
let r = clip.rect();
let moved = [r[0] + offset[0], r[1] + offset[1], r[2] + offset[0], r[3] + offset[1]];
if moved[0] < crop[2] && moved[2] > crop[0] && moved[1] < crop[3] && moved[3] > crop[1] {
return Ok(offset);
}
let centre = |a: f64, b: f64| (a + b) / 2.0;
Ok([centre(crop[0], crop[2]) - centre(r[0], r[2]), centre(crop[1], crop[3]) - centre(r[1], r[3])])
}
}
/// The objects copied in an application or a script, and how many times each page has had them
/// pasted, which sets how far the next paste there steps ([`Document::object_paste_offset`]).
#[derive(Clone, Debug)]
pub struct ObjectClipboard {
clip: Arc<ObjectClip>,
pastes: Vec<(DocId, usize, u32)>,
}
impl ObjectClipboard {
pub fn new(clip: ObjectClip) -> Self {
ObjectClipboard { clip: Arc::new(clip), pastes: Vec::new() }
}
pub fn clip(&self) -> &ObjectClip {
&self.clip
}
/// The edit that pastes the copy onto `page` of `doc`: where it was, or on the page it came
/// from one step down and to the right (as Acrobat offsets a paste onto the same page), and
/// one step further for each earlier paste onto that page.
pub fn paste(&mut self, doc: &Document, page: usize) -> Result<Edit, String> {
let at = self.pastes.iter().position(|(d, p, _)| *d == doc.id && *p == page);
let earlier = at.and_then(|i| self.pastes.get(i)).map_or(0, |(_, _, n)| *n);
let own = u32::from(self.clip.source() == (doc.id, page));
let offset = doc.object_paste_offset(page, &self.clip, earlier.saturating_add(own))?;
match at.and_then(|i| self.pastes.get_mut(i)) {
Some(entry) => entry.2 = entry.2.saturating_add(1),
None => self.pastes.push((doc.id, page, 1)),
}
Ok(Edit::PasteObjects { page, clip: Arc::clone(&self.clip), offset })
}
}
impl Edit {
/// The edit that deletes `targets` (indexes into one inventory of `page`) as one undo step:
/// what Delete and Cut do to objects selected together. One object is deleted by its own
/// edit, several by [`Edit::DeleteObjects`]; none is no edit.
pub fn delete_objects(page: usize, mut targets: Vec<ObjectTarget>) -> Option<Edit> {
targets.sort();
targets.dedup();
match targets.as_slice() {
[] => None,
[t] => Some(delete_one(page, *t)),
_ => Some(Edit::DeleteObjects { page, objects: targets }),
}
}
}
fn delete_one(page: usize, t: ObjectTarget) -> Edit {
use crate::ObjectKind;
match t.kind {
ObjectKind::Text => Edit::EditTextBlock { page, block: t.index, text: String::new(), style: crate::BlockStyle::default() },
ObjectKind::Image => Edit::EditPageImage { page, index: t.index, change: crate::ImageEdit::Delete },
ObjectKind::Added => Edit::DeleteContent { page, index: t.index },
}
}
/// [`Edit::DeleteObjects`], each object by its own edit (`run`). A deletion renumbers what follows
/// it in its own inventory, so pictures go from the last to the first, and added content, which
/// the other inventories leave out, after them. Deleting a paragraph also removes a copy drawn
/// exactly over it (fake bold, a paste onto itself), which renumbers paragraphs before it too: so
/// each paragraph is found again by its text and box, and one already gone with another is done.
pub(crate) fn delete(
doc: &mut pdfcraft_cos::Document,
page: usize,
objects: &[ObjectTarget],
mut run: impl FnMut(&mut pdfcraft_cos::Document, &Edit) -> Result<(), EditError>,
) -> Result<(), EditError> {
use crate::ObjectKind;
let mut targets = objects.to_vec();
targets.sort_by_key(|t| (t.kind == ObjectKind::Added, std::cmp::Reverse(t.index)));
targets.dedup();
let blocks = pdfcraft_edit::text_blocks(doc, page)?;
for t in targets {
let edit = match t.kind {
ObjectKind::Text => {
let key = blocks.get(t.index).map(crate::BlockKey::of).ok_or_else(|| {
EditError::Edit(pdfcraft_edit::EditError::Invalid("a selected paragraph no longer exists; select the objects again".into()))
})?;
let Some(block) = key.find(&pdfcraft_edit::text_blocks(doc, page)?, t.index) else { continue };
delete_one(page, ObjectTarget { kind: t.kind, index: block })
}
_ => delete_one(page, t),
};
run(doc, &edit)?;
}
Ok(())
}
/// [`Edit::PasteObjects`]: the copy's resources as `doc` holds them, then the paste.
pub(crate) fn paste(doc: &mut pdfcraft_cos::Document, page: usize, clip: &ObjectClip, offset: [f64; 2]) -> Result<(), EditError> {
let resources = Object::Dict(clip.copied.resources.clone());
let refused =
|e: pdfcraft_organize::OrganizeError| EditError::Edit(pdfcraft_edit::EditError::Invalid(format!("the copied objects can't be pasted: {e}")));
let used = pdfcraft_organize::reachable(&clip.source, &resources, MAX_IMPORT_OBJECTS, MAX_IMPORT_BYTES).map_err(refused)?;
// Still the objects the copy was made from: the same document, and each of them unchanged.
let same = doc.source_identity() == clip.source.source_identity()
&& used.iter().all(|r| {
let (here, there) = (doc.get(*r), clip.source.get(*r));
Arc::ptr_eq(&here, &there) || here == there
});
let resources = if same { resources } else { pdfcraft_organize::import_value(doc, &clip.source, &resources).map_err(refused)? };
let Object::Dict(resources) = resources else {
return Err(EditError::Edit(pdfcraft_edit::EditError::Invalid("the copied objects' resources could not be copied".into())));
};
let copied = pdfcraft_edit::CopiedObjects { resources, ..clip.copied.clone() };
pdfcraft_edit::paste_objects(doc, page, &copied, offset)?;
Ok(())
}
#[cfg(test)]
#[path = "object_clip_tests.rs"]
mod tests;
crates/engine/src/object_clip_tests.rs
+257
View File
@@ -0,0 +1,257 @@
use super::*;
use crate::{ObjectKind, Session};
/// A PDF of 300 × 300 pages, each drawing its `content` with `resources` and objects `extra`
/// (numbered after the pages' content streams).
fn pdf(pages: &[(&str, &str)], extra: &[String]) -> Arc<Vec<u8>> {
let n = pages.len();
let kids: Vec<String> = (0..n).map(|i| format!("{} 0 R", 3 + i)).collect();
let mut objs = vec![
"<< /Type /Catalog /Pages 2 0 R >>".to_string(),
format!("<< /Type /Pages /Kids [{}] /Count {n} /MediaBox [0 0 300 300] >>", kids.join(" ")),
];
for (i, (_, resources)) in pages.iter().enumerate() {
objs.push(format!("<< /Type /Page /Parent 2 0 R /Contents {} 0 R /Resources {resources} >>", 3 + n + i));
}
for (content, _) in pages {
objs.push(format!("<< /Length {} >>\nstream\n{content}\nendstream", content.len()));
}
objs.extend(extra.iter().cloned());
let mut out = b"%PDF-1.7\n".to_vec();
let mut offs = Vec::new();
for (i, o) in objs.iter().enumerate() {
offs.push(out.len());
out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes());
}
let x = out.len();
out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes());
for o in offs {
out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes());
}
out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{x}\n%%EOF\n", objs.len() + 1).as_bytes());
Arc::new(out)
}
const HELVETICA: &str = "<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>";
const COURIER: &str = "<< /Type /Font /Subtype /Type1 /BaseFont /Courier /Encoding /WinAnsiEncoding >>";
/// A 2 × 2 RGB picture of the twelve bytes `data` (ASCII, so the file's lengths are its bytes).
fn picture(data: &str) -> String {
assert!(data.is_ascii() && data.len() == 12);
format!(
"<< /Type /XObject /Subtype /Image /Width 2 /Height 2 /ColorSpace /DeviceRGB /BitsPerComponent 8 /Length 12 >>\nstream\n{data}\nendstream"
)
}
/// A one-page document whose paragraph "Copied" (blue, half opaque, Helvetica /F1 at objects 5)
/// and picture /Im0 (object 6) are copied; "Left" is not.
fn source() -> Arc<Vec<u8>> {
let content = "0 0 1 rg /GS0 gs BT /F1 24 Tf 20 200 Td (Copied) Tj ET q 60 0 0 40 20 100 cm /Im0 Do Q BT /F1 12 Tf 200 40 Td (Left) Tj ET";
pdf(
&[(content, "<< /Font << /F1 5 0 R >> /XObject << /Im0 6 0 R >> /ExtGState << /GS0 << /ca 0.5 >> >> >>")],
&[HELVETICA.into(), picture("\u{7f}\u{0}\u{0}\u{0}\u{7f}\u{0}\u{0}\u{0}\u{7f}\u{7f}\u{7f}\u{0}")],
)
}
/// A one-page document that already draws with Courier as /F1 and has a picture of its own as /Im0.
fn target() -> Arc<Vec<u8>> {
pdf(
&[("BT /F1 10 Tf 250 280 Td (x) Tj ET q 10 0 0 10 280 5 cm /Im0 Do Q", "<< /Font << /F1 5 0 R >> /XObject << /Im0 6 0 R >> >>")],
&[COURIER.into(), picture("@@@@@@@@@@@@")],
)
}
fn copy_all_but_left(s: &Session, id: DocId) -> ObjectClip {
let doc = s.get(id).unwrap();
let targets: Vec<_> = doc.editable_objects(0).unwrap().into_iter().filter(|o| o.text.as_deref() != Some("Left")).map(|o| o.target).collect();
assert_eq!(targets.len(), 2);
doc.copy_objects(0, &targets).unwrap()
}
fn render(s: &Session, id: DocId) -> pdfcraft_render::RenderedPage {
let doc = s.get(id).unwrap();
let mut renderer = pdfcraft_render::PageRenderer::new(doc.bytes.clone(), doc.config.clone());
renderer.render(pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() })
}
/// The pixels in user-space box `r` of a 300 pt page rendered at scale 1.
fn pixels(page: &pdfcraft_render::RenderedPage, r: [f64; 4]) -> Vec<u8> {
let mut out = Vec::new();
for y in (300.0 - r[3]) as u32..(300.0 - r[1]) as u32 {
for x in r[0] as u32..r[2] as u32 {
let i = ((y * page.width + x) * 4) as usize;
out.extend_from_slice(&page.rgba[i..i + 4]);
}
}
out
}
fn images(s: &Session, id: DocId) -> Vec<Option<pdfcraft_cos::ObjRef>> {
s.get(id).unwrap().page_images(0).into_iter().map(|i| i.object).collect()
}
#[test]
fn objects_copied_from_one_document_paste_into_another_and_draw_the_same() {
let mut s = Session::new().with_clock(|| 1_700_000_000);
let a = s.open("a.pdf", None, source(), None).unwrap();
let b = s.open("b.pdf", None, target(), None).unwrap();
let clip = copy_all_but_left(&s, a);
assert_eq!(clip.text(), "Copied");
let mut clipboard = ObjectClipboard::new(clip);
// The source can change, or close: the copy is a snapshot.
s.apply(a, Edit::delete_objects(0, vec![ObjectTarget { kind: ObjectKind::Image, index: 0 }]).unwrap()).unwrap();
s.close(a);
let a = s.open("a.pdf", None, source(), None).unwrap();
let paste = clipboard.paste(s.get(b).unwrap(), 0).unwrap();
assert!(matches!(&paste, Edit::PasteObjects { offset, .. } if *offset == [0.0, 0.0]), "another document: where it was");
s.apply(b, paste).unwrap();
assert_eq!(s.get(b).unwrap().can_undo(), Some("Paste"));
let area = clipboard.clip().rect().map(|v| v.round());
let area = [area[0] - 3.0, area[1] - 3.0, area[2] + 3.0, area[3] + 3.0];
let (original, pasted) = (pixels(&render(&s, a), area), pixels(&render(&s, b), area));
assert!(original.chunks(4).any(|p| p[2] > 200 && p[0] < 200), "the blue paragraph is drawn");
assert_eq!(original, pasted, "the pasted paragraph and picture draw the same pixels");
// Its own /F1 and /Im0 are left alone; what the copy uses came over under other names.
let doc = s.get(b).unwrap();
let texts: Vec<_> = doc.editable_objects(0).unwrap().into_iter().filter_map(|o| o.text).collect();
assert!(texts.contains(&"Copied".to_string()) && !texts.contains(&"Left".to_string()), "{texts:?}");
let pictures = images(&s, b);
assert_eq!(pictures.len(), 2);
assert_ne!(pictures[0], pictures[1]);
// A second paste steps down and to the right, and shares the fonts and pictures the first
// brought instead of copying them again.
let paste = clipboard.paste(s.get(b).unwrap(), 0).unwrap();
assert!(matches!(&paste, Edit::PasteObjects { offset, .. } if *offset == [18.0, -18.0]), "{paste:?}");
s.apply(b, paste).unwrap();
let pictures = images(&s, b);
assert_eq!(pictures.len(), 3);
assert_eq!(pictures[1], pictures[2], "one picture object for both pastes");
// Saved and reopened, the pastes are there; undone, the page is as it was.
let saved = s.save_bytes(b).unwrap();
let mut again = Session::new();
let reopened = again.open("b.pdf", None, saved, None).unwrap();
assert_eq!(again.get(reopened).unwrap().page_images(0).len(), 3);
s.undo(b).unwrap();
s.undo(b).unwrap();
assert_eq!(images(&s, b).len(), 1);
assert_eq!(s.get(b).unwrap().can_undo(), None);
}
#[test]
fn a_paste_onto_the_page_it_came_from_steps_and_draws_from_the_same_objects() {
let mut s = Session::new().with_clock(|| 1_700_000_000);
let a = s.open("a.pdf", None, source(), None).unwrap();
let mut clipboard = ObjectClipboard::new(copy_all_but_left(&s, a));
let paste = clipboard.paste(s.get(a).unwrap(), 0).unwrap();
assert!(matches!(&paste, Edit::PasteObjects { offset, .. } if *offset == [18.0, -18.0]), "one step down and right: {paste:?}");
s.apply(a, paste).unwrap();
let pictures = images(&s, a);
assert_eq!(pictures.len(), 2);
assert_eq!(pictures[0], pictures[1], "the same picture object, not a copy");
// Far off the page (a copy from a larger page), a paste is centred instead.
let doc = s.get(a).unwrap();
let mut far = clipboard.clip().clone();
far.copied.rect = [1000.0, 1000.0, 1100.0, 1050.0];
assert_eq!(doc.object_paste_offset(0, &far, 0).unwrap(), [150.0 - 1050.0, 150.0 - 1025.0]);
}
#[test]
fn pasting_into_another_document_follows_cycles_and_refuses_past_its_limit() {
let mut s = Session::new().with_clock(|| 1_700_000_000);
// A font that refers to itself and to an object that refers back.
let cyclic = pdf(
&[("BT /F1 20 Tf 20 200 Td (Loop) Tj ET", "<< /Font << /F1 5 0 R >> >>")],
&[
"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding /Self 5 0 R /Next [6 0 R] >>".into(),
"<< /Back 5 0 R /Again 6 0 R >>".into(),
],
);
let a = s.open("cyclic.pdf", None, cyclic, None).unwrap();
let b = s.open("b.pdf", None, target(), None).unwrap();
let all: Vec<_> = s.get(a).unwrap().editable_objects(0).unwrap().into_iter().map(|o| o.target).collect();
let mut clipboard = ObjectClipboard::new(s.get(a).unwrap().copy_objects(0, &all).unwrap());
s.apply(b, clipboard.paste(s.get(b).unwrap(), 0).unwrap()).unwrap();
assert!(s.get(b).unwrap().editable_objects(0).unwrap().iter().any(|o| o.text.as_deref() == Some("Loop")));
// An ExtGState that drags in more objects than a paste copies is refused with a reason.
let many = 20_001;
let refs: Vec<String> = (0..many).map(|i| format!("{} 0 R", 6 + i)).collect();
let mut extra = vec![format!("<< /ca 0.5 /Junk [{}] >>", refs.join(" "))];
extra.extend((0..many).map(|i| format!("<< /N {i} >>")));
let resources = format!("<< /Font << /F1 {HELVETICA} >> /ExtGState << /GS0 5 0 R >> >>");
let big = pdf(&[("/GS0 gs BT /F1 20 Tf 20 200 Td (Big) Tj ET", &resources)], &extra);
let c = s.open("big.pdf", None, big, None).unwrap();
let all: Vec<_> = s.get(c).unwrap().editable_objects(0).unwrap().into_iter().map(|o| o.target).collect();
assert_eq!(all.len(), 1);
let mut clipboard = ObjectClipboard::new(s.get(c).unwrap().copy_objects(0, &all).unwrap());
let before = s.get(b).unwrap().can_undo().map(str::to_string);
let err = s.apply(b, clipboard.paste(s.get(b).unwrap(), 0).unwrap()).unwrap_err().to_string();
assert!(err.contains("copy fewer objects"), "{err}");
assert_eq!(s.get(b).unwrap().can_undo().map(str::to_string), before, "a refused paste changes nothing");
}
#[test]
fn copying_refuses_a_stale_selection_and_deleting_several_objects_is_one_step() {
let mut s = Session::new().with_clock(|| 1_700_000_000);
let a = s.open("a.pdf", None, source(), None).unwrap();
let doc = s.get(a).unwrap();
assert!(doc.copy_objects(0, &[ObjectTarget { kind: ObjectKind::Text, index: 7 }]).is_err());
assert!(doc.copy_objects(3, &[ObjectTarget { kind: ObjectKind::Text, index: 0 }]).is_err());
let delete = Edit::delete_objects(0, vec![ObjectTarget { kind: ObjectKind::Text, index: 0 }, ObjectTarget { kind: ObjectKind::Image, index: 0 }]);
assert!(matches!(delete, Some(Edit::DeleteObjects { .. })));
assert!(Edit::delete_objects(0, Vec::new()).is_none());
s.apply(a, delete.unwrap()).unwrap();
assert_eq!(s.get(a).unwrap().can_undo(), Some("Delete content"));
assert!(s.get(a).unwrap().editable_objects(0).unwrap().iter().all(|o| o.text.as_deref() == Some("Left")));
}
/// Two pastes of a paragraph and a picture exactly over each other: deleting a paragraph also
/// removes a copy drawn exactly over it, so deleting all four objects together found the second
/// paragraph's number gone and refused the whole delete. Both copies go, in one step.
#[test]
fn objects_pasted_exactly_over_each_other_delete_together_in_one_step() {
let mut s = Session::new().with_clock(|| 1_700_000_000);
let a = s.open("a.pdf", None, source(), None).unwrap();
let blank = s.create_blank(300.0, 300.0, 1).unwrap();
let b = s.open_new("b.pdf", blank).unwrap();
let mut clipboard = ObjectClipboard::new(copy_all_but_left(&s, a));
let Edit::PasteObjects { clip, .. } = clipboard.paste(s.get(b).unwrap(), 0).unwrap() else { panic!("a paste") };
for _ in 0..2 {
s.apply(b, Edit::PasteObjects { page: 0, clip: Arc::clone(&clip), offset: [0.0, 0.0] }).unwrap();
}
let pasted = s.get(b).unwrap().editable_objects(0).unwrap();
assert_eq!(pasted.iter().filter(|o| o.text.as_deref() == Some("Copied")).count(), 2, "{pasted:?}");
assert_eq!(pasted.len(), 4);
let delete = Edit::delete_objects(0, pasted.iter().map(|o| o.target).collect()).unwrap();
s.apply(b, delete).unwrap();
assert!(s.get(b).unwrap().editable_objects(0).unwrap().is_empty());
assert_eq!(s.get(b).unwrap().can_undo(), Some("Delete content"));
s.undo(b).unwrap();
assert_eq!(s.get(b).unwrap().editable_objects(0).unwrap(), pasted, "one undo brings both copies back");
}
/// Deleting renumbers what follows in the same inventory, so paragraphs and pictures go from
/// the last to the first, and added content (outside those inventories) after them.
#[test]
fn several_objects_delete_last_first() {
let content =
"BT /F1 12 Tf 20 250 Td (A) Tj ET BT /F1 12 Tf 20 180 Td (B) Tj ET BT /F1 12 Tf 20 110 Td (C) Tj ET BT /F1 12 Tf 20 40 Td (D) Tj ET";
let mut doc = pdfcraft_cos::Document::open(pdf(&[(content, "<< /Font << /F1 5 0 R >> >>")], &[HELVETICA.into()])).unwrap();
let t = |kind, index| ObjectTarget { kind, index };
let targets = [t(ObjectKind::Added, 0), t(ObjectKind::Text, 1), t(ObjectKind::Image, 0), t(ObjectKind::Text, 3), t(ObjectKind::Added, 2)];
let mut order = Vec::new();
delete(&mut doc, 0, &targets, |_, e| {
order.push(match e {
Edit::EditTextBlock { block, text, .. } if text.is_empty() => ('t', *block),
Edit::EditPageImage { index, change: crate::ImageEdit::Delete, .. } => ('i', *index),
Edit::DeleteContent { index, .. } => ('a', *index),
other => panic!("unexpected {other:?}"),
});
Ok(())
})
.unwrap();
assert_eq!(order, [('t', 3), ('t', 1), ('i', 0), ('a', 2), ('a', 0)]);
}
crates/engine/src/ocr.rs
+382 -159
View File
@@ -15,7 +15,8 @@ use pdfcraft_render::text::TextFilter;
use pdfcraft_render::{PageInfo, PageRenderer, RenderConfig, RenderRequest, RequestKind};
pub use pdfcraft_ocr::{
ConfidenceBand, LANGUAGES, MergeStrategy, Models, Ocr, OcrError, PlacedWord, Recognizer, band_for, is_suspect, low_confidence,
ConfidenceBand, DETECTION_MODEL, LANGUAGES, Line, MergeStrategy, Models, Ocr, OcrError, OcrImage, PlacedWord, RECOGNITION_MODEL, Recognizer,
StrategyOutcome, Word, band_for, is_suspect, lines_from_json, low_confidence,
};
use crate::{DocId, Edit, EditError, Session};
@@ -257,6 +258,23 @@ struct EngineCache {
ocrs: Option<Arc<dyn Recognizer>>,
}
/// The engines built so far (see [`recognizers`] and [`install`]).
static ENGINES: Mutex<Option<EngineCache>> = Mutex::new(None);
/// Share an ocrs recogniser loaded some other way than from the model files [`Models::find`]
/// looks for: the web build downloads the models and loads them with [`Ocr::from_bytes`]. From
/// then on [`recognizers`] hands it out and the engine counts as available.
pub fn install(ocr: Ocr) -> Arc<dyn Recognizer> {
let ocrs: Arc<dyn Recognizer> = Arc::new(pdfcraft_ocr::OcrsRecognizer::from_ocr(ocr));
ENGINES.lock().unwrap_or_else(|e| e.into_inner()).get_or_insert_with(|| EngineCache { ocrs: None }).ocrs = Some(ocrs.clone());
ocrs
}
/// Whether an ocrs recogniser has been loaded (from the model files, or by [`install`]).
fn ocrs_loaded() -> bool {
ENGINES.lock().unwrap_or_else(|e| e.into_inner()).as_ref().is_some_and(|c| c.ocrs.is_some())
}
/// How long a negative engine probe (no program there) is remembered. A short TTL, not
/// forever: a tesseract installed while the app runs is found by the next job, without a
/// restart. An engine that was found is remembered for the session.
@@ -408,10 +426,11 @@ fn models_installed_cached() -> bool {
found
}
/// The web build has no disk search worth caching.
/// The web build has no model files: the models count as installed once [`install`] has
/// loaded the ones it downloaded.
#[cfg(target_arch = "wasm32")]
fn models_installed_cached() -> bool {
Models::find().is_some()
ocrs_loaded()
}
/// The program's `--version` line, remembered per program path for the session (`ocr_status`
@@ -473,7 +492,7 @@ pub fn engines_status() -> Vec<EngineStatus> {
/// takes a location, because an agent must not choose which program runs; the tools report the
/// default settings.)
pub fn engines_status_for(settings: &OcrSettings) -> Vec<EngineStatus> {
let ocrs = Models::find().is_some();
let ocrs = ocrs_loaded() || Models::find().is_some();
let mut out = vec![EngineStatus { id: "ocrs", available: ocrs, why_not: (!ocrs).then(models_missing), program: None, version: None }];
#[cfg(not(target_arch = "wasm32"))]
{
@@ -524,7 +543,6 @@ pub fn recognizers(settings: &OcrSettings) -> Result<Recognizers, String> {
let configured = configured_tesseract(settings);
let tesseract =
if matches!(settings.engine, EngineChoice::Tesseract | EngineChoice::Auto) { tesseract_engine(configured.as_deref()) } else { None };
static ENGINES: Mutex<Option<EngineCache>> = Mutex::new(None);
let mut slot = ENGINES.lock().unwrap_or_else(|e| e.into_inner());
let cache = slot.get_or_insert_with(|| EngineCache { ocrs: None });
if cache.ocrs.is_none() {
@@ -567,7 +585,7 @@ fn models_missing() -> String {
/// never waits.
pub fn available() -> bool {
let configured = std::env::var_os("PDFCRAFT_TESSERACT").map(|v| v.to_string_lossy().into_owned());
Models::find().is_some() || tesseract_engine(configured.as_deref()).is_some()
ocrs_loaded() || Models::find().is_some() || tesseract_engine(configured.as_deref()).is_some()
}
/// Why `engine` cannot run right now, named for the UI's tooltips (the Engine dropdown
@@ -703,138 +721,226 @@ impl OcrJob {
/// Read the pages. `progress(done, total)` is called before each page; returning `false`
/// stops (the pages read so far are returned).
pub fn run(self, recognizers: &Recognizers, mut progress: impl FnMut(usize, usize) -> bool) -> Vec<OcrPage> {
let config =
RenderConfig { password: self.password.as_deref().map(Arc::from), hide_comments: self.settings.native_text_only, ..Default::default() };
let mut r = PageRenderer::new(self.bytes.clone(), config);
let mut out = Vec::new();
let total = self.pages.len();
let options = pdfcraft_ocr::RecognizeOptions { language: self.settings.language.clone() };
for (done, &page) in self.pages.iter().enumerate() {
if !progress(done, total) {
break;
let mut steps = self.steps();
let mut out = Vec::new();
while progress(steps.done(), total) {
match steps.next_page() {
None => break,
Some(Err(skipped)) => out.push(skipped),
Some(Ok(shot)) => out.push(shot.read(recognizers)),
}
let Some(info) = self.infos.get(page) else { continue };
let skip = |why: &str| OcrPage { page, words: Vec::new(), already_on_page: 0, skipped: Some(why.into()), notes: Vec::new() };
// The boxes of all text on the page, in view space: an earlier OCR layer's, and words
// only /ActualText names, so neither is read again.
let existing: Vec<[f32; 4]> = r
.render(RenderRequest { page, kind: RequestKind::Text, scale: 1.0, ..Default::default() })
.text
.map(|t| t.glyphs.iter().filter(|g| !g.text.trim().is_empty()).map(|g| g.rect).collect())
.unwrap_or_default();
if self.settings.skip_text_pages && !self.settings.force_ocr && !existing.is_empty() && has_own_text(&mut r, page) {
out.push(skip("the page already contains text"));
continue;
}
// A scan is read at its own resolution (enlarging it only blurs the letters), at
// most the chosen one.
let scale = self.render_scale(page);
let shot = r.render(RenderRequest { page, scale, ..Default::default() });
if let Some(e) = shot.error {
out.push(skip(&e));
continue;
}
let full = match pdfcraft_ocr::OcrImage::new(shot.width, shot.height, shot.rgba.to_vec()) {
Ok(img) => img,
Err(e) => {
out.push(skip(&e.to_string()));
continue;
}
};
// The optional region: crop the rendered page, clamped; an empty region skips the
// page with a named reason (nothing was asked of it).
let (image, offset) = match self.settings.region {
None => (full, [0.0f32; 2]),
Some(region) => match clamp_region(region, shot.width, shot.height) {
Some([x0, y0, x1, y1]) => match crop(&full, x0, y0, x1, y1) {
Some(c) => (c, [x0 as f32, y0 as f32]),
None => {
out.push(skip("the selected region is empty"));
continue;
}
},
None => {
out.push(skip("the selected region is outside the page"));
continue;
}
},
};
// Preprocessing sits between the page raster and recognition; its word boxes come
// back in processed coordinates and are mapped to the ORIGINAL raster's before
// placement. The render dpi is the real one (pixels per point × 72), never a
// fixed value.
let pre = pdfcraft_ocr::preprocess::preprocess(
image,
scale * 72.0,
&pdfcraft_ocr::preprocess::PreprocessOptions {
auto_rotate: self.settings.auto_rotate,
deskew: self.settings.deskew,
denoise: self.settings.denoise,
binarize: self.settings.binarize,
},
);
let (image, inverse) = match pre {
Ok(p) => (p.image, p.inverse_transform),
Err(e) => {
out.push(skip(&e.to_string()));
continue;
}
};
let outcome = match pdfcraft_ocr::recognize_with_strategy(
recognizers.primary.as_ref(),
recognizers.secondary.as_deref(),
self.settings.strategy,
&image,
&options,
) {
Ok(o) => o,
Err(e) => {
out.push(skip(&e.to_string()));
continue;
}
};
let scale = shot.width as f32 / info.width.max(1e-3);
let to_user = |x: f32, y: f32| {
// processed pixels → original raster pixels (the exact inverse) → past the
// region crop → page view space.
let (ox, oy) = inverse.apply(f64::from(x), f64::from(y));
let [u, v] = info.view_to_user((ox as f32 + offset[0]) / scale, (oy as f32 + offset[1]) / scale);
[u as f64, v as f64]
};
// Validate recognition geometry and compare against existing text in view space,
// undoing preprocessing and region offsets before testing overlap.
let (fresh, already): (Vec<_>, Vec<_>) = outcome.lines.iter().flat_map(|l| &l.words).filter(|w| box_writable(w)).partition(|w| {
let [l, t, r, b] = w.rect;
let bounds = [(l, t), (r, t), (r, b), (l, b)].into_iter().fold(
[f32::INFINITY, f32::INFINITY, f32::NEG_INFINITY, f32::NEG_INFINITY],
|mut bounds, (x, y)| {
let (x, y) = inverse.apply(f64::from(x), f64::from(y));
let x = (x as f32 + offset[0]) / scale;
let y = (y as f32 + offset[1]) / scale;
bounds[0] = bounds[0].min(x);
bounds[1] = bounds[1].min(y);
bounds[2] = bounds[2].max(x);
bounds[3] = bounds[3].max(y);
bounds
},
);
covered(bounds, &existing) < ALREADY_ON_PAGE
});
let words = fresh.into_iter().map(|w| PlacedWord::place(w, to_user)).collect::<Vec<_>>();
let mut notes = outcome.notes;
// Only visible editable output needs display glyphs. Searchable output preserves
// every Unicode character with empty Type3 programs and ToUnicode maps.
if self.settings.output_mode == OutputMode::EditableText && !words.is_empty() {
let missing = pdfcraft_fonts::TextPlan::for_words(words.iter().map(|w| w.text.as_str())).missing();
if missing > 0 {
notes.push(format!("{missing} characters had no font that could show them and appear as ? in the output"));
}
}
out.push(OcrPage { page, words, already_on_page: already.len(), skipped: None, notes });
}
progress(total, total);
out
}
/// Read the pages one at a time, preparing each page's image and recognising it separately:
/// the web build prepares pages on the page and recognises them in Web Workers.
pub fn steps(self) -> OcrSteps {
let config =
RenderConfig { password: self.password.as_deref().map(Arc::from), hide_comments: self.settings.native_text_only, ..Default::default() };
let renderer = PageRenderer::new(self.bytes.clone(), config);
OcrSteps { job: self, renderer, next: 0 }
}
}
/// [`OcrJob`] a page at a time: [`OcrSteps::next_page`] renders the next page, crops it to the
/// region and preprocesses it; an engine reads [`OcrShot::image`], and [`OcrShot::place`] puts
/// the words found on the page ([`OcrShot::read`] does both with the job's engines).
pub struct OcrSteps {
job: OcrJob,
renderer: PageRenderer,
next: usize,
}
impl OcrSteps {
/// The number of pages to read.
pub fn total(&self) -> usize {
self.job.pages.len()
}
/// The number of pages prepared (or skipped) so far.
pub fn done(&self) -> usize {
self.next
}
/// The settings the job reads with.
pub fn settings(&self) -> &OcrSettings {
&self.job.settings
}
/// Prepare the next page for recognition. `Err` is a page that isn't read (it already has
/// text, or couldn't be rendered or prepared), with the reason; `None` once every page is
/// done.
pub fn next_page(&mut self) -> Option<Result<OcrShot, OcrPage>> {
loop {
let page = *self.job.pages.get(self.next)?;
self.next += 1;
let Some(info) = self.job.infos.get(page).cloned() else { continue };
return Some(self.prepare(page, info));
}
}
fn prepare(&mut self, page: usize, info: PageInfo) -> Result<OcrShot, OcrPage> {
let settings = &self.job.settings;
let skip = |why: &str| OcrPage { page, words: Vec::new(), already_on_page: 0, skipped: Some(why.into()), notes: Vec::new() };
let r = &mut self.renderer;
// The boxes of all text on the page, in view space: an earlier OCR layer's, and words
// only /ActualText names, so neither is read again.
let existing: Vec<[f32; 4]> = r
.render(RenderRequest { page, kind: RequestKind::Text, scale: 1.0, ..Default::default() })
.text
.map(|t| t.glyphs.iter().filter(|g| !g.text.trim().is_empty()).map(|g| g.rect).collect())
.unwrap_or_default();
if settings.skip_text_pages && !settings.force_ocr && !existing.is_empty() && has_own_text(r, page) {
return Err(skip("the page already contains text"));
}
// A scan is read at its own resolution (enlarging it only blurs the letters), at
// most the chosen one.
let scale = self.job.render_scale(page);
let shot = r.render(RenderRequest { page, scale, ..Default::default() });
if let Some(e) = shot.error {
return Err(skip(&e));
}
let full = pdfcraft_ocr::OcrImage::new(shot.width, shot.height, shot.rgba.to_vec()).map_err(|e| skip(&e.to_string()))?;
// The optional region: crop the rendered page, clamped; an empty region skips the
// page with a named reason (nothing was asked of it).
let (image, offset) = match settings.region {
None => (full, [0.0f32; 2]),
Some(region) => match clamp_region(region, shot.width, shot.height) {
Some([x0, y0, x1, y1]) => match crop(&full, x0, y0, x1, y1) {
Some(c) => (c, [x0 as f32, y0 as f32]),
None => return Err(skip("the selected region is empty")),
},
None => return Err(skip("the selected region is outside the page")),
},
};
// Preprocessing sits between the page raster and recognition; its word boxes come
// back in processed coordinates and are mapped to the ORIGINAL raster's before
// placement. The render dpi is the real one (pixels per point × 72), never a
// fixed value.
let pre = pdfcraft_ocr::preprocess::preprocess(
image,
scale * 72.0,
&pdfcraft_ocr::preprocess::PreprocessOptions {
auto_rotate: settings.auto_rotate,
deskew: settings.deskew,
denoise: settings.denoise,
binarize: settings.binarize,
},
)
.map_err(|e| skip(&e.to_string()))?;
Ok(OcrShot {
page,
image: Some(pre.image),
inverse: pre.inverse_transform,
offset,
pixels_per_point: shot.width as f32 / info.width.max(1e-3),
info,
existing,
strategy: settings.strategy,
options: pdfcraft_ocr::RecognizeOptions { language: settings.language.clone() },
output_mode: settings.output_mode,
})
}
}
/// A page prepared for recognition: the image an engine reads, and what placing the words
/// found in it needs.
pub struct OcrShot {
pub page: usize,
/// The page's pixels as the engines read them: rendered, cropped to the region and
/// preprocessed (`None` once [`OcrShot::take_image`] has handed them on).
image: Option<OcrImage>,
/// Processed pixels → the rendered raster's (the exact inverse of preprocessing).
inverse: pdfcraft_ocr::preprocess::InverseTransform,
/// Where the region crop starts in the rendered raster.
offset: [f32; 2],
/// The rendered raster's pixels per point of page view space.
pixels_per_point: f32,
info: PageInfo,
/// The boxes of the page's own text, in view space.
existing: Vec<[f32; 4]>,
strategy: MergeStrategy,
options: pdfcraft_ocr::RecognizeOptions,
output_mode: OutputMode,
}
impl OcrShot {
/// What the engines read (`None` once taken).
pub fn image(&self) -> Option<&OcrImage> {
self.image.as_ref()
}
/// Hand the image on (the web build sends it to a worker), keeping what placing the words
/// needs: the page's pixels need not stay in memory while the worker reads them.
pub fn take_image(&mut self) -> Option<OcrImage> {
self.image.take()
}
/// Recognise the image with `recognizers` (per the job's strategy) and place the words.
pub fn read(&self, recognizers: &Recognizers) -> OcrPage {
let Some(image) = &self.image else { return self.place(Err("the page's image was handed to another reader".into())) };
let outcome = pdfcraft_ocr::recognize_with_strategy(
recognizers.primary.as_ref(),
recognizers.secondary.as_deref(),
self.strategy,
image,
&self.options,
);
self.place(outcome.map_err(|e| e.to_string()))
}
/// Put the lines recognised in [`Self::image`] on the page (or skip the page with the
/// error). A single engine's reading, which is all the web build's workers run, is
/// `StrategyOutcome { lines, ..Default::default() }`: every strategy reads that way when
/// there is no second engine.
pub fn place(&self, outcome: Result<StrategyOutcome, String>) -> OcrPage {
let page = self.page;
let outcome = match outcome {
Ok(o) => o,
Err(e) => return OcrPage { page, words: Vec::new(), already_on_page: 0, skipped: Some(e), notes: Vec::new() },
};
let (info, inverse, offset, scale) = (&self.info, self.inverse, self.offset, self.pixels_per_point);
let to_user = |x: f32, y: f32| {
// processed pixels → original raster pixels (the exact inverse) → past the
// region crop → page view space.
let (ox, oy) = inverse.apply(f64::from(x), f64::from(y));
let [u, v] = info.view_to_user((ox as f32 + offset[0]) / scale, (oy as f32 + offset[1]) / scale);
[u as f64, v as f64]
};
// Validate recognition geometry and compare against existing text in view space,
// undoing preprocessing and region offsets before testing overlap.
let (fresh, already): (Vec<_>, Vec<_>) = outcome.lines.iter().flat_map(|l| &l.words).filter(|w| box_writable(w)).partition(|w| {
let [l, t, r, b] = w.rect;
let bounds = [(l, t), (r, t), (r, b), (l, b)].into_iter().fold(
[f32::INFINITY, f32::INFINITY, f32::NEG_INFINITY, f32::NEG_INFINITY],
|mut bounds, (x, y)| {
let (x, y) = inverse.apply(f64::from(x), f64::from(y));
let x = (x as f32 + offset[0]) / scale;
let y = (y as f32 + offset[1]) / scale;
bounds[0] = bounds[0].min(x);
bounds[1] = bounds[1].min(y);
bounds[2] = bounds[2].max(x);
bounds[3] = bounds[3].max(y);
bounds
},
);
covered(bounds, &self.existing) < ALREADY_ON_PAGE
});
let words = fresh.into_iter().map(|w| PlacedWord::place(w, to_user)).collect::<Vec<_>>();
let mut notes = outcome.notes;
// Only visible editable output needs display glyphs. Searchable output preserves
// every Unicode character with empty Type3 programs and ToUnicode maps.
if self.output_mode == OutputMode::EditableText && !words.is_empty() {
let missing = pdfcraft_fonts::TextPlan::for_words(words.iter().map(|w| w.text.as_str())).missing();
if missing > 0 {
notes.push(format!("{missing} characters had no font that could show them and appear as ? in the output"));
}
}
OcrPage { page, words, already_on_page: already.len(), skipped: None, notes }
}
}
/// Whether a word's pixel box may reach the content stream: width and height (right − left,
@@ -1095,36 +1201,70 @@ pub fn recognize_file(
recognizers: &Recognizers,
progress: impl FnMut(usize, usize) -> bool,
) -> Result<FileResult, String> {
let mut s = Session::new();
let id = s.open(name, None, bytes.clone(), password).map_err(|e| e.to_string())?;
if let Some(why) = s.get(id).and_then(|d| d.read_only_reason.clone()) {
return Err(why);
match FileJob::open(name, bytes, password, settings)? {
Opened::Skipped(result) => Ok(result),
Opened::Read(mut file) => {
let job = file.job.take().ok_or("the document could not be read")?;
let pages = job.run(recognizers, progress);
file.finish(pages)
}
}
let job = s.ocr_job(id, &[], settings.clone()).ok_or("the document could not be read")?;
// The skip-files probe runs before any engine work: a file with text needs no engine at
// all, and the original bytes come back unchanged.
if settings.skip_text_files && !settings.force_ocr && job.has_text() {
let pages = job
.pages
.iter()
.map(|&page| OcrPage {
already_on_page: 0,
page,
words: Vec::new(),
skipped: Some("the file already contains text".into()),
notes: Vec::new(),
})
.collect();
return Ok(FileResult { bytes, pages, file_skipped: Some("the file already contains text".into()) });
}
/// [`recognize_file`] in two halves, for callers that read the pages themselves (the web build
/// reads them in Web Workers): [`FileJob::open`], read `job`, then [`FileJob::finish`].
pub struct FileJob {
session: Session,
id: DocId,
bytes: Arc<Vec<u8>>,
/// The pages to read (taken by the caller).
pub job: Option<OcrJob>,
}
/// What [`FileJob::open`] found: a file to read, or one the skip-files option passes over
/// (its result is final, and no engine is needed).
pub enum Opened {
Read(Box<FileJob>),
Skipped(FileResult),
}
impl FileJob {
pub fn open(name: &str, bytes: Arc<Vec<u8>>, password: Option<&str>, settings: OcrSettings) -> Result<Opened, String> {
let mut session = Session::new();
let id = session.open(name, None, bytes.clone(), password).map_err(|e| e.to_string())?;
if let Some(why) = session.get(id).and_then(|d| d.read_only_reason.clone()) {
return Err(why);
}
let job = session.ocr_job(id, &[], settings.clone()).ok_or("the document could not be read")?;
// The skip-files probe runs before any engine work: a file with text needs no engine at
// all, and the original bytes come back unchanged.
if settings.skip_text_files && !settings.force_ocr && job.has_text() {
let pages = job
.pages
.iter()
.map(|&page| OcrPage {
already_on_page: 0,
page,
words: Vec::new(),
skipped: Some("the file already contains text".into()),
notes: Vec::new(),
})
.collect();
return Ok(Opened::Skipped(FileResult { bytes, pages, file_skipped: Some("the file already contains text".into()) }));
}
Ok(Opened::Read(Box::new(FileJob { session, id, bytes, job: Some(job) })))
}
let pages = job.run(recognizers, progress);
if s.apply_ocr(id, &pages).map_err(|e| e.to_string())? == 0 {
return Ok(FileResult { bytes, pages, file_skipped: None });
/// Add the words found and save: the searchable file, or the original when nothing was found.
pub fn finish(mut self, pages: Vec<OcrPage>) -> Result<FileResult, String> {
if self.session.apply_ocr(self.id, &pages).map_err(|e| e.to_string())? == 0 {
return Ok(FileResult { bytes: self.bytes, pages, file_skipped: None });
}
// Checked like a save before the caller writes it (#2166).
let bytes = self.session.save_bytes(self.id).map_err(|e| e.to_string())?;
self.session.check_save(self.id, &bytes).map_err(|e| e.to_string())?;
Ok(FileResult { bytes, pages, file_skipped: None })
}
// Checked like a save before the caller writes it (#2166).
let bytes = s.save_bytes(id).map_err(|e| e.to_string())?;
s.check_save(id, &bytes).map_err(|e| e.to_string())?;
Ok(FileResult { bytes, pages, file_skipped: None })
}
#[cfg(test)]
@@ -1639,6 +1779,89 @@ mod tests {
assert_eq!(again.pages[0].skipped.as_deref(), Some("the page already contains text"));
}
/// The web build prepares pages with [`OcrJob::steps`] and has a worker read each image,
/// its lines coming back as JSON: the pages it ends with are the ones [`OcrJob::run`] gives,
/// with a region crop and preprocessing between the render and the engine.
#[test]
fn reading_page_by_page_matches_a_run() {
let mut s = Session::new();
let bytes = s.create_from_text("t", "One page\nanother page").unwrap();
let id = s.open("t.pdf", None, bytes, None).unwrap();
let settings = OcrSettings { force_ocr: true, deskew: true, binarize: true, region: Some([40.0, 30.0, 900.0, 700.0]), ..Default::default() };
let (r, fake) = fake_recognizers();
let run = s.ocr_job(id, &[], settings.clone()).unwrap().run(&r, |_, _| true);
assert!(run.iter().all(|p| p.skipped.is_none() && p.words.len() == 1), "{run:?}");
let mut steps = s.ocr_job(id, &[], settings).unwrap().steps();
assert_eq!((steps.done(), steps.total()), (0, run.len()));
let mut stepped = Vec::new();
while let Some(next) = steps.next_page() {
let shot = next.unwrap();
let json = pdfcraft_ocr::lines_to_json(&fake.recognize(shot.image().unwrap(), &pdfcraft_ocr::RecognizeOptions::default()).unwrap());
let lines = lines_from_json(&json, "fake").map_err(|e| e.to_string());
stepped.push(shot.place(lines.map(|lines| StrategyOutcome { lines, ..Default::default() })));
}
assert_eq!(steps.done(), run.len());
assert_eq!(stepped, run);
// An engine's failure skips the page with the reason, as in a run.
let mut steps = s.ocr_job(id, &[0], OcrSettings { force_ocr: true, ..Default::default() }).unwrap().steps();
let mut shot = steps.next_page().unwrap().unwrap();
let page = shot.place(Err("worker gone".into()));
assert_eq!(page.skipped.as_deref(), Some("worker gone"));
// Once the image is handed on, it can't be read here as well.
assert!(shot.take_image().is_some() && shot.image().is_none());
assert!(shot.read(&r).skipped.is_some());
assert!(steps.next_page().is_none());
}
/// A page the job doesn't read comes out of [`OcrSteps::next_page`] as skipped, with its
/// reason, before any engine is involved.
#[test]
fn page_by_page_skips_text_pages_without_an_engine() {
let mut s = Session::new();
let bytes = s.create_from_text("t", "A page with text").unwrap();
let id = s.open("t.pdf", None, bytes, None).unwrap();
let mut steps = s.ocr_job(id, &[], OcrSettings::default()).unwrap().steps();
let Some(Err(page)) = steps.next_page() else { panic!("the text page should be skipped") };
assert_eq!(page.skipped.as_deref(), Some("the page already contains text"));
}
/// The web build has no model files: a recogniser loaded from downloaded bytes is installed
/// and from then on is the engine every job reads with.
#[test]
fn an_installed_recognizer_is_available_and_handed_out() {
let Some(models) = Models::find().filter(|m| m.detection.ends_with(DETECTION_MODEL)) else {
eprintln!("skipped: ONNX OCR models not installed");
return;
};
let ocr = Ocr::from_bytes(std::fs::read(&models.detection).unwrap(), std::fs::read(&models.recognition).unwrap()).unwrap();
let installed = install(ocr);
assert!(installed.available());
assert!(available());
let r = recognizers(&OcrSettings::default()).unwrap();
assert!(Arc::ptr_eq(&r.primary, &installed), "the installed recogniser is the one handed out");
assert!(engines_status().iter().any(|e| e.id == "ocrs" && e.available));
}
/// The batch's halves: a file to read opens with its job, and [`FileJob::finish`] saves
/// the searchable copy; a file the skip-files option passes over is final on opening.
#[test]
fn a_file_job_reads_in_two_halves() {
let s = Session::new();
let bytes = s.create_from_text("t", "Already typed here").unwrap();
let (r, fake) = fake_recognizers();
let skip = OcrSettings { skip_text_files: true, ..Default::default() };
let Ok(Opened::Skipped(out)) = FileJob::open("t.pdf", bytes.clone(), None, skip) else { panic!("the file should be skipped") };
assert_eq!(out.file_skipped.as_deref(), Some("the file already contains text"));
let force = OcrSettings { force_ocr: true, ..Default::default() };
let Ok(Opened::Read(mut file)) = FileJob::open("t.pdf", bytes.clone(), None, force.clone()) else { panic!("the file should be read") };
let pages = file.job.take().unwrap().run(&r, |_, _| true);
let out = file.finish(pages).unwrap();
assert_eq!((fake.calls(), out.words(), out.file_skipped.clone()), (1, 1, None));
let whole = recognize_file("t.pdf", bytes, None, force, &r, |_, _| true).unwrap();
// The saved bytes carry a modification time, so two saves a second apart differ there alone.
assert_eq!((out.pages, out.file_skipped), (whole.pages, whole.file_skipped), "the same as reading it in one call");
}
/// Two inputs of one name (from different folders, or differing only in case) would
/// write the same output: both are found, each once.
#[test]
crates/engine/src/session/editing.rs
+16 -3
View File
@@ -20,6 +20,19 @@ fn deletes_pages(edit: &Edit) -> bool {
impl Session {
/// Apply an edit. On success the previous state is undoable and the view data is refreshed.
pub fn apply(&mut self, id: DocId, edit: Edit) -> Result<(), EditError> {
self.apply_reporting(id, edit).map(|_| ())
}
/// Edit a PDF ▸ Edit text: make the grouped text on `page` (0-based) editable, as one undoable
/// [`Edit::PrepareText`], and say what it prepared, which fonts' maps it repaired and which
/// groups it left as they were (and why). The counts are this edit's own, not a preview's.
pub fn prepare_text(&mut self, id: DocId, page: usize) -> Result<pdfcraft_edit::text::forms::Preparation, EditError> {
self.apply_reporting(id, Edit::PrepareText { page })?
.ok_or_else(|| EditError::Invalid("preparing grouped text gave no result; the document was not changed".into()))
}
/// [`Session::apply`], giving back what an [`Edit::PrepareText`] did (`None` for other edits).
fn apply_reporting(&mut self, id: DocId, edit: Edit) -> Result<Option<pdfcraft_edit::text::forms::Preparation>, EditError> {
let now = self.now();
let today = self.today();
let js_off = self.js_off;
@@ -61,7 +74,7 @@ impl Session {
};
// Nothing to bake: leave undo history and the dirty flag alone.
if matches!(&edit, Edit::FlattenFillSign) && !pdfcraft_annot::has_visible_fill_sign(&editor.cos) {
return Ok(());
return Ok(None);
}
let mut next = editor.cos.clone();
if !js_off && uses_scripts(&edit) {
@@ -228,7 +241,7 @@ impl Session {
doc.xfa_scripts_off = true;
doc.js_output.errors.push(XFA_SCRIPTS_OFF.into());
}
Ok(())
Ok(cx.prepared)
}
pub fn undo(&mut self, id: DocId) -> Result<Label, EditError> {
@@ -421,7 +434,7 @@ impl Session {
| Edit::ReplaceImage { .. }
| Edit::PasteContent { .. }
| Edit::EditPageImage { .. }
) || matches!(e, Edit::MoveObjects { objects, .. } if objects.iter().any(|o| o.kind == ObjectKind::Added))
) || matches!(e, Edit::MoveObjects { objects, .. } | Edit::DeleteObjects { objects, .. } if objects.iter().any(|o| o.kind == ObjectKind::Added))
});
let touches_marks = edit.is_none_or(|e| {
matches!(e, Edit::AddHeaderFooter { .. } | Edit::AddWatermark { .. } | Edit::AddBackground { .. } | Edit::RemoveMarks { .. })
crates/engine/src/tests.rs
+58
View File
@@ -5801,6 +5801,35 @@ fn backgrounds_and_watermarks_from_files() {
assert!(s.apply(id, Edit::AddBackground { pages: vec![0], settings: Background::default(), replace: true, file: Some(junk) }).is_err());
}
/// The web build reads a page at a time (prepared on the page, recognised in a worker, the
/// lines sent back as JSON); with the real ocrs engine that finds the same words as reading in
/// one go (needs the models: `cargo xtask models`; skipped without them).
#[test]
fn reading_a_page_at_a_time_finds_the_same_words() {
let settings = ocr::OcrSettings::default();
let Ok(engines) = ocr::recognizers(&settings) else {
eprintln!("skipped: OCR models not installed");
return;
};
let mut s = Session::new().with_clock(|| 1_700_000_000);
let text = s.create_from_text("t", "Invoice 42: pay by Friday").unwrap();
let typed = s.open("text.pdf", None, text, None).unwrap();
let png = export::Exporter::new(s.get(typed).unwrap()).png(0, 150.0).unwrap();
let scan = s.create_from_images(&[("scan.png".into(), png.clone()), ("scan2.png".into(), png)]).unwrap();
let id = s.open("scan.pdf", None, scan, None).unwrap();
let at_once = s.ocr_job(id, &[], settings.clone()).unwrap().run(&engines, |_, _| true);
let mut steps = s.ocr_job(id, &[], settings).unwrap().steps();
let mut one_by_one = Vec::new();
while let Some(next) = steps.next_page() {
let shot = next.unwrap();
let lines = engines.primary.recognize(shot.image().unwrap(), &pdfcraft_ocr::RecognizeOptions::default()).unwrap();
let sent = ocr::lines_from_json(&pdfcraft_ocr::lines_to_json(&lines), engines.primary.id()).map_err(|e| e.to_string());
one_by_one.push(shot.place(sent.map(|lines| ocr::StrategyOutcome { lines, ..Default::default() })));
}
assert_eq!(one_by_one, at_once);
assert!(at_once.len() == 2 && at_once.iter().all(|p| p.skipped.is_none() && !p.words.is_empty()), "{at_once:?}");
}
/// Scan & OCR ▸ Recognize text on a page that is only a picture of text (needs the models:
/// `cargo xtask models`; skipped without them).
#[test]
@@ -6505,6 +6534,35 @@ fn actions_fail_when_hidden_information_could_not_be_removed() {
assert!(e.starts_with("Sanitize document:"), "{e}");
}
/// A PDF whose catalog names its language (`/Lang`) exports to Word and HTML in that language, so
/// Word checks the spelling in it, not in English; one that names none exports as before.
#[test]
fn office_exports_take_the_documents_language() {
let pdf = |catalog: &str| {
pdfcraft_testkit::Pdf::new()
.obj(catalog)
.obj("<< /Type /Pages /Kids [3 0 R] /Count 1 >>")
.obj("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 300] /Contents 4 0 R /Resources << /Font << /F1 << /Type /Font /Subtype /Type1 /BaseFont /Helvetica >> >> >> >>")
.stream("", b"BT /F1 12 Tf 20 200 Td (Hello) Tj ET")
.build()
};
let mut s = Session::new();
let bg = s.open("lang.pdf", None, Arc::new(pdf("<< /Type /Catalog /Pages 2 0 R /Lang (bg-BG) >>")), None).unwrap();
let plain = s.open("plain.pdf", None, Arc::new(pdf("<< /Type /Catalog /Pages 2 0 R >>")), None).unwrap();
let d = s.get(bg).unwrap();
let docx = d.export_office(compare::OfficeFormat::Docx);
assert!(docx == pdfcraft_export::docx_with_language(&d.export_pages(), "lang", Some("bg-BG")), "Word names bg-BG");
assert!(docx != pdfcraft_export::docx(&d.export_pages(), "lang"), "Word names a language");
for format in [compare::OfficeFormat::Html, compare::OfficeFormat::HtmlSimple] {
let html = String::from_utf8(d.export_office(format)).unwrap();
assert!(html.contains("<html lang=\"bg-BG\">"), "{html}");
}
let d = s.get(plain).unwrap();
assert!(d.export_office(compare::OfficeFormat::Docx) == pdfcraft_export::docx(&d.export_pages(), "plain"), "no language, as before");
assert!(String::from_utf8(d.export_office(compare::OfficeFormat::Html)).unwrap().contains("<html>\n"));
assert!(String::from_utf8(d.export_office(compare::OfficeFormat::HtmlSimple)).unwrap().contains("<html>\n"));
}
#[test]
fn exporting_office_files_keeps_images() {
let mut s = Session::new();
crates/engine/tests/it/ocr_edit.rs
+110
View File
@@ -160,6 +160,116 @@ fn a_rewritten_ocr_phrase_keeps_its_word_widths() {
}
}
/// Recognize text sizes each word from its box and a guess at how much of the em its letters fill,
/// so one phrase's words come out several points apart (#3313). Here the first word is the
/// largest and the middle one the smallest; the phrase is set in 11 pt.
fn uneven_phrase() -> (Session, pdfcraft_engine::DocId) {
scanned_with(vec![
scanned_word("John", 20.0, 45.0, 70.0, 15.0),
scanned_word("Smith", 49.0, 78.0, 70.0, 9.0),
scanned_word("Example", 82.0, 124.0, 70.0, 11.0),
])
}
/// An edited OCR phrase is redrawn in one size, the median of its words', and each word keeps
/// the width the scan shows it at (#3313).
#[test]
fn an_edited_ocr_phrase_is_drawn_at_one_size() {
let (mut s, id) = uneven_phrase();
let block = s.get(id).unwrap().text_blocks(0).remove(0);
assert!((block.size - 11.0).abs() < 0.05, "the phrase's size is its words' median: {}", block.size);
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Example".into(), style: Default::default() }).unwrap();
let lines = s.get(id).unwrap().text_lines(0);
assert_eq!(lines.iter().map(|l| l.text.trim()).collect::<Vec<_>>(), ["John", "Example"]);
for line in &lines {
assert!((line.size - 11.0).abs() < 0.05, "{:?} is drawn at {} pt, not the phrase's 11", line.text, line.size);
}
// Each word keeps its width; the rest of the phrase closes up where Smith was.
assert!((lines[0].rect[2] - lines[0].rect[0] - 25.0).abs() < 0.5, "John keeps its width: {:?}", lines[0].rect);
assert!((lines[1].rect[2] - lines[1].rect[0] - 42.0).abs() < 0.5, "Example keeps its width: {:?}", lines[1].rect);
assert!(lines[1].rect[0] >= lines[0].rect[2] - 0.5, "Example runs into John");
}
/// Only invisible OCR text is set in one phrase size: visible text keeps each run's own size, and
/// a phrase whose size is NaN, zero, negative or infinite falls back to its words' sizes (#3313).
#[test]
fn only_an_ocr_phrase_with_a_usable_size_is_set_in_one_size() {
let (s, id) = uneven_phrase();
let block = s.get(id).unwrap().text_blocks(0).remove(0);
assert!(block.invisible);
assert_eq!(block.phrase_size(), Some(block.size));
let visible = pdfcraft_engine::TextBlock { invisible: false, ..block.clone() };
assert_eq!(visible.phrase_size(), None, "visible text keeps its runs' sizes");
for size in [f64::NAN, 0.0, -3.0, f64::INFINITY] {
assert_eq!(pdfcraft_engine::TextBlock { size, ..block.clone() }.phrase_size(), None, "a {size} pt phrase falls back to its words");
}
}
/// Choosing a font sends the phrase through the single-style rewrite, which takes its size from
/// the first run: that is the phrase's size too, not the first word's guess.
#[test]
fn a_new_font_for_an_ocr_phrase_keeps_its_one_size() {
let (mut s, id) = uneven_phrase();
let style = pdfcraft_engine::BlockStyle { family: Some((pdfcraft_engine::FontFamily::Times, false, false)), ..Default::default() };
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smith Example".into(), style }).unwrap();
let lines = s.get(id).unwrap().text_lines(0);
assert!(!lines.is_empty());
for line in &lines {
assert!((line.size - 11.0).abs() < 0.05, "{:?} is drawn at {} pt, not the phrase's 11", line.text, line.size);
}
}
/// Two neighbouring words OCR sized alike join into one run; the median still counts each word,
/// so 15, 9 and 9 pt make a 9 pt phrase, not the 12 pt of its two runs.
#[test]
fn words_joined_into_one_run_each_count_toward_the_phrase_size() {
let (s, id) = scanned_with(vec![
scanned_word("John", 20.0, 45.0, 70.0, 15.0),
scanned_word("SMITH", 49.0, 78.0, 70.0, 9.0),
scanned_word("JONES", 82.0, 112.0, 70.0, 9.0),
]);
let doc = s.get(id).unwrap();
assert_eq!(doc.text_lines(0).len(), 2, "the two 9 pt words share a run");
let size = doc.text_blocks(0)[0].size;
assert!((size - 9.0).abs() < 0.05, "the phrase is set in its words' median, 9 pt: {size}");
}
/// A word OCR sized at more than twice the phrase's size still keeps the width it has on the scan.
#[test]
fn a_word_far_larger_than_its_phrase_keeps_its_width() {
let (mut s, id) = scanned_with(vec![
scanned_word("John", 20.0, 45.0, 70.0, 25.0),
scanned_word("Smith", 49.0, 78.0, 70.0, 10.0),
scanned_word("Example", 82.0, 124.0, 70.0, 10.5),
]);
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smith Exam".into(), style: Default::default() }).unwrap();
let lines = s.get(id).unwrap().text_lines(0);
assert_eq!(lines[0].text.trim(), "John");
assert!((lines[0].rect[2] - lines[0].rect[0] - 25.0).abs() < 0.5, "John keeps its width: {:?}", lines[0].rect);
}
/// With a new font the phrase is one run in its first word's text state. How wide it is must not
/// depend on how far OCR misjudged that word's size: the same boxes give the same line.
#[test]
fn a_new_font_for_an_ocr_phrase_is_as_wide_whatever_its_first_word_was_sized() {
let width = |first: f64| {
let (mut s, id) = scanned_with(vec![
scanned_word("John", 20.0, 45.0, 70.0, first),
scanned_word("Smith", 49.0, 78.0, 70.0, 11.0),
scanned_word("Example", 82.0, 124.0, 70.0, 11.0),
]);
let style = pdfcraft_engine::BlockStyle { family: Some((pdfcraft_engine::FontFamily::Helvetica, false, false)), ..Default::default() };
// Short enough not to wrap in the phrase's box either way.
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smith".into(), style }).unwrap();
let lines = s.get(id).unwrap().text_lines(0);
assert_eq!(lines.len(), 1, "one line: {lines:?}");
lines.iter().map(|l| l.rect[2]).fold(f64::MIN, f64::max) - lines.iter().map(|l| l.rect[0]).fold(f64::MAX, f64::min)
};
let (guessed, right) = (width(15.0), width(11.0));
assert!((guessed - right).abs() < 0.5, "a 15 pt guess for John makes the line {guessed} wide, an 11 pt one {right}");
}
/// Visible text set with a horizontally stretched text matrix (3 × 1), as some generators write.
fn stretched_visible() -> std::sync::Arc<Vec<u8>> {
use pdfcraft_cos::{Dict, Document, Object, SaveOptions, Stream};
crates/export/src/as_docx.rs
+46 -7
View File
@@ -1,5 +1,5 @@
use super::Zip;
use super::as_html::esc;
use super::as_html::{esc, language_tag};
use super::layout::{Image, Item, Page, Table, body_size, items};
/// A colour's 8-bit sRGB components (NaN and out-of-range values clamp; NaN reads as 0).
@@ -31,16 +31,19 @@ fn run_xml(text: &str, size: f64, bold: bool, italic: bool, color: [f64; 3]) ->
format!("<w:r><w:rPr>{rpr}</w:rPr><w:t xml:space=\"preserve\">{}</w:t></w:r>", esc(text))
}
/// One Word table: single borders when the page's rules draw its grid, explicit absent borders
/// otherwise (alignment alone cannot prove PDF rules), a grid sized from the detected column
/// One Word table: when the page's rules draw its grid, single inside borders and single outer
/// borders on the edges they rule; explicit absent borders otherwise (alignment alone cannot
/// prove PDF rules), a grid sized from the detected column
/// edges, `gridSpan` for spanning cells, empty cells for holes.
pub(crate) fn docx_table(t: &Table) -> String {
let mut edges = t.cols.clone();
edges.push(t.rect[2]);
let widths: Vec<i64> = edges.windows(2).map(|w| ((w[1] - w[0]).max(20.0) * 20.0).round().clamp(60.0, 31680.0) as i64).collect();
let mut s = String::from("<w:tbl><w:tblPr><w:tblW w:w=\"0\" w:type=\"auto\"/><w:tblBorders>");
for edge in ["top", "left", "bottom", "right", "insideH", "insideV"] {
if t.ruled {
let [top, left, bottom, right] = t.ruled.unwrap_or_default();
let inside = t.ruled.is_some();
for (edge, drawn) in [("top", top), ("left", left), ("bottom", bottom), ("right", right), ("insideH", inside), ("insideV", inside)] {
if drawn {
s.push_str(&format!("<w:{edge} w:val=\"single\" w:sz=\"4\" w:space=\"0\" w:color=\"auto\"/>"));
} else {
s.push_str(&format!("<w:{edge} w:val=\"nil\"/>"));
@@ -233,9 +236,40 @@ fn word_background(pages: &[Page]) -> Option<String> {
(same && first != [255, 255, 255]).then(|| format!("{:02X}{:02X}{:02X}", first[0], first[1], first[2]))
}
/// The `w:lang` that makes `tag` the language of a Word document's text. Word keeps one language
/// per script: Chinese, Japanese and Korean are its East Asian text, Arabic, Hebrew, Persian and
/// Urdu its right-to-left text, and any other language its Latin, Greek and Cyrillic text. A
/// script the tag names (`pa-Arab`, `zh-Latn`) decides over its language.
fn word_lang(tag: &str) -> String {
let mut subtags = tag.split('-');
let primary = subtags.next().unwrap_or_default().to_ascii_lowercase();
// The script comes straight after the language and any three-letter extended languages; a
// singleton before it starts an extension or private use, which names no script.
let script = subtags
.find(|s| s.len() != 3 || !s.bytes().all(|b| b.is_ascii_alphabetic()))
.filter(|s| s.len() == 4 && s.bytes().all(|b| b.is_ascii_alphabetic()))
.map(str::to_ascii_lowercase);
let attribute = match (script.as_deref(), primary.as_str()) {
(Some("arab" | "hebr" | "syrc" | "thaa"), _) => "w:bidi",
(Some("hans" | "hant" | "hani" | "jpan" | "kore" | "hang" | "hira" | "kana"), _) => "w:eastAsia",
(Some("latn" | "cyrl" | "grek"), _) => "w:val",
(_, "zh" | "ja" | "ko") => "w:eastAsia",
(_, "ar" | "he" | "fa" | "ur") => "w:bidi",
_ => "w:val",
};
format!("<w:lang {attribute}=\"{}\"/>", esc(tag))
}
/// A Word document (.docx, Office Open XML). Ordinary pages flow as paragraphs and tables;
/// pages with rotated text, or text that overlaps artwork, use floating shapes to retain their layout.
pub fn docx(pages: &[Page], title: &str) -> Vec<u8> {
docx_with_language(pages, title, None)
}
/// [`docx`] for a document in `language` (a BCP 47 tag such as `bg-BG`, as a PDF's `/Lang` gives
/// it): the default language of its text, so Word checks the spelling in that language rather
/// than in its own. A language that is not a well formed tag is left out.
pub fn docx_with_language(pages: &[Page], title: &str, language: Option<&str>) -> Vec<u8> {
let mut body = String::new();
let mut media: Vec<(String, &Image)> = Vec::new();
// Word needs a paragraph between adjacent tables and after the last one in the body.
@@ -343,11 +377,16 @@ xmlns:wp=\"http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawin
doc_rels.push_str("<Relationship Id=\"rIdSettings\" Type=\"http://schemas.openxmlformats.org/officeDocument/2006/relationships/settings\" Target=\"settings.xml\"/>");
}
doc_rels.push_str("</Relationships>");
let styles = "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<w:styles xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\">\
let defaults = language_tag(language)
.map(|tag| format!("<w:docDefaults><w:rPrDefault><w:rPr>{}</w:rPr></w:rPrDefault></w:docDefaults>", word_lang(tag)))
.unwrap_or_default();
let styles = format!(
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<w:styles xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\">{defaults}\
<w:style w:type=\"paragraph\" w:default=\"1\" w:styleId=\"Normal\"><w:name w:val=\"Normal\"/><w:pPr><w:spacing w:after=\"120\"/></w:pPr></w:style>\
<w:style w:type=\"paragraph\" w:styleId=\"Heading1\"><w:name w:val=\"heading 1\"/><w:basedOn w:val=\"Normal\"/><w:next w:val=\"Normal\"/><w:pPr><w:keepNext/><w:spacing w:before=\"240\"/><w:outlineLvl w:val=\"0\"/></w:pPr><w:rPr><w:b/></w:rPr></w:style>\
<w:style w:type=\"paragraph\" w:styleId=\"Heading2\"><w:name w:val=\"heading 2\"/><w:basedOn w:val=\"Normal\"/><w:next w:val=\"Normal\"/><w:pPr><w:keepNext/><w:spacing w:before=\"200\"/><w:outlineLvl w:val=\"1\"/></w:pPr><w:rPr><w:b/></w:rPr></w:style>\
</w:styles>";
</w:styles>"
);
let core = format!(
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<cp:coreProperties xmlns:cp=\"http://schemas.openxmlformats.org/package/2006/metadata/core-properties\" xmlns:dc=\"http://purl.org/dc/elements/1.1/\"><dc:title>{}</dc:title></cp:coreProperties>",
esc(title)
crates/export/src/as_html.rs
+53 -4
View File
@@ -8,6 +8,41 @@ pub(crate) fn allowed(c: char) -> bool {
matches!(c, '\t' | '\n' | '\r') || !(c.is_ascii_control() || c == '\u{FFFE}' || c == '\u{FFFF}')
}
/// The longest language tag the exports write. BCP 47 sets no maximum (35 characters is only
/// the least a protocol must hold), so this bounds only what a PDF's `/Lang` can put in a file.
const LONGEST_TAG: usize = 255;
/// `lang` when it is a well formed language tag (BCP 47, as a PDF's `/Lang` holds one): a primary
/// language of two or three letters, then subtags of one to eight letters or digits, joined by
/// hyphens, [`LONGEST_TAG`] characters at most. A singleton (one letter or digit) starts an
/// extension, or with `x` private use, and needs a subtag after it. Anything else names no
/// language.
pub(crate) fn language_tag(lang: Option<&str>) -> Option<&str> {
let tag = lang?.trim();
let mut subtags = tag.split('-');
let primary = subtags.next()?;
if tag.len() > LONGEST_TAG || !(2..=3).contains(&primary.len()) || !primary.bytes().all(|b| b.is_ascii_alphabetic()) {
return None;
}
// In private use every subtag is data, one character long or not.
let (mut private, mut singleton) = (false, false);
for s in subtags {
if !(1..=8).contains(&s.len()) || !s.bytes().all(|b| b.is_ascii_alphanumeric()) {
return None;
}
if s.len() == 1 && !private {
if singleton {
return None;
}
private = s.eq_ignore_ascii_case("x");
singleton = true;
} else {
singleton = false;
}
}
(!singleton).then_some(tag)
}
pub(crate) fn esc(s: &str) -> String {
let mut o = String::with_capacity(s.len());
for c in s.chars().filter(|c| allowed(*c)) {
@@ -38,17 +73,31 @@ pub(crate) fn base64(data: &[u8]) -> String {
/// One HTML file: headings and paragraphs, tables as real `<table>` elements, images inline, a
/// rule between pages.
pub fn html(pages: &[Page], title: &str) -> String {
html_with_mode(pages, title, false)
html_with_language(pages, title, None)
}
/// A simple HTML export with no CSS; page layout and colours are omitted.
pub fn html_simple(pages: &[Page], title: &str) -> String {
html_with_mode(pages, title, true)
html_simple_with_language(pages, title, None)
}
fn html_with_mode(pages: &[Page], title: &str, simple: bool) -> String {
/// [`html`] for a document in `language` (a BCP 47 tag such as `bg-BG`, as a PDF's `/Lang` gives
/// it), which the page names (`<html lang>`) for spell checking, speech and hyphenation. A
/// language that is not a well formed tag is left out.
pub fn html_with_language(pages: &[Page], title: &str, language: Option<&str>) -> String {
html_with_mode(pages, title, false, language)
}
/// [`html_simple`] for a document in `language`, named as [`html_with_language`] names it: a
/// language is not styling, so the simple export keeps it.
pub fn html_simple_with_language(pages: &[Page], title: &str, language: Option<&str>) -> String {
html_with_mode(pages, title, true, language)
}
fn html_with_mode(pages: &[Page], title: &str, simple: bool, language: Option<&str>) -> String {
let lang = language_tag(language).map(|tag| format!(" lang=\"{}\"", esc(tag))).unwrap_or_default();
let mut s = format!(
"<!DOCTYPE html>\n<html>\n<head>\n<meta charset=\"utf-8\">\n<title>{}</title>\n{}\n</head>\n<body>\n",
"<!DOCTYPE html>\n<html{lang}>\n<head>\n<meta charset=\"utf-8\">\n<title>{}</title>\n{}\n</head>\n<body>\n",
esc(title),
if simple {
""
crates/export/src/layout.rs
+47 -11
View File
@@ -68,9 +68,10 @@ pub struct Table {
pub rect: [f64; 4],
pub cols: Vec<f64>,
pub rows: Vec<Vec<Cell>>,
/// Whether the page's ruled lines separate every pair of neighbouring cells, so the writers
/// can draw the table's borders.
pub ruled: bool,
/// `Some` when the page's ruled lines separate every pair of neighbouring cells, so the
/// writers draw the inside borders; it holds which outer edges (top, left, bottom, right) the
/// lines also draw, as only those get a border.
pub ruled: Option<[bool; 4]>,
}
/// What goes into the output, in order.
@@ -264,8 +265,11 @@ const RULE_BUDGET: usize = 4_000_000;
/// Whether `rules` separate every pair of neighbouring cells of a table whose rows are `rows`
/// (block indices, top to bottom) and whose columns start at `edges` and end at `right`: between
/// two columns a vertical rule beside each row (unless a cell of that row spans across), and
/// between two rows a horizontal rule under each column. Every check spends `budget`.
fn ruled(blocks: &[Block], rules: &[[f64; 4]], rows: &[Vec<usize>], edges: &[f64], right: f64, budget: &mut usize) -> bool {
/// between two rows a horizontal rule under each column. Every check spends `budget`. When they
/// do, says which outer edges (top, left, bottom, right) are ruled too: a rule over every column
/// above the first row or below the last, or beside every row left of the first column or right
/// of the last, no farther from the cells than the next row or column is.
fn ruled(blocks: &[Block], rules: &[[f64; 4]], rows: &[Vec<usize>], edges: &[f64], right: f64, budget: &mut usize) -> Option<[bool; 4]> {
let vertical: Vec<&[f64; 4]> = rules.iter().filter(|r| r[2] - r[0] <= RULE_THICK && r[3] - r[1] > RULE_THICK).collect();
let horizontal: Vec<&[f64; 4]> = rules.iter().filter(|r| r[3] - r[1] <= RULE_THICK && r[2] - r[0] > RULE_THICK).collect();
let mut find = |set: &[&[f64; 4]], hit: &dyn Fn(&[f64; 4]) -> bool| -> bool {
@@ -295,7 +299,7 @@ fn ruled(blocks: &[Block], rules: &[[f64; 4]], rows: &[Vec<usize>], edges: &[f64
let x = (r[0] + r[2]) / 2.0;
lo + COL_TOL < x && x <= hi + COL_TOL && r[1] <= mid && mid <= r[3] && clear(x)
}) {
return false;
return None;
}
}
}
@@ -307,11 +311,41 @@ fn ruled(blocks: &[Block], rules: &[[f64; 4]], rows: &[Vec<usize>], edges: &[f64
let y = (r[1] + r[3]) / 2.0;
below_top - COL_TOL <= y && y <= above_bottom + COL_TOL && r[0] <= x && x <= r[2]
}) {
return false;
return None;
}
}
}
true
let (Some(first), Some(last), Some(&left), Some(&last_col)) = (spans.first(), spans.last(), edges.first(), edges.last()) else {
return Some([false; 4]);
};
// How far the next row or column is: an outer rule farther out than that frames something else.
let row_gap = |a: Option<&[f64; 2]>, b: Option<&[f64; 2]>, side: usize| a.zip(b).map_or(0.0, |(a, b)| (a[side] - b[side]).abs());
let top_gap = row_gap(Some(first), spans.get(1), 1);
let bottom_gap = row_gap(spans.len().checked_sub(2).and_then(|i| spans.get(i)), Some(last), 0);
let left_gap = edges.get(1).map_or(0.0, |e| e - left);
let right_gap = edges.len().checked_sub(2).and_then(|i| edges.get(i)).map_or(0.0, |e| last_col - e);
let mut over_columns = |lo: f64, hi: f64| {
mids.iter().all(|&x| {
find(&horizontal, &|r| {
let y = (r[1] + r[3]) / 2.0;
lo <= y && y <= hi && r[0] <= x && x <= r[2]
})
})
};
let top = over_columns(first[1] - COL_TOL, first[1] + top_gap + COL_TOL);
let bottom = over_columns(last[0] - bottom_gap - COL_TOL, last[0] + COL_TOL);
let mut beside_rows = |lo: f64, hi: f64| {
spans.iter().all(|span| {
let mid = (span[0] + span[1]) / 2.0;
find(&vertical, &|r| {
let x = (r[0] + r[2]) / 2.0;
lo <= x && x <= hi && r[1] <= mid && mid <= r[3]
})
})
};
let left_ruled = beside_rows(left - left_gap - COL_TOL, left + COL_TOL);
let right_ruled = beside_rows(right - COL_TOL, right.max(last_col + right_gap) + COL_TOL);
Some([top, left_ruled, bottom, right_ruled])
}
/// Detect tables in a page's blocks from their text layout: columns are left edges shared by
@@ -394,11 +428,13 @@ pub(crate) fn tables(blocks: &[Block], rules: &[[f64; 4]]) -> (Vec<Table>, Vec<b
edges.dedup_by(|a, b| (*a - *b).abs() <= COL_TOL);
let ncols = edges.len();
let right = used.iter().map(|&i| blocks[i].rect[2]).fold(f64::MIN, f64::max) + COL_TOL;
let is_ruled = (2..=MAX_COLS).contains(&ncols) && !rules.is_empty() && {
let frame = if (2..=MAX_COLS).contains(&ncols) && !rules.is_empty() {
let row_blocks: Vec<Vec<usize>> = run.iter().map(|row| row.iter().map(|(_, i)| *i).collect()).collect();
ruled(blocks, rules, &row_blocks, &edges, right, budget)
} else {
None
};
let ok = (ncols >= 3 || (ncols == 2 && (run.len() >= 3 || is_ruled))) && ncols <= MAX_COLS;
let ok = (ncols >= 3 || (ncols == 2 && (run.len() >= 3 || frame.is_some()))) && ncols <= MAX_COLS;
if !ok {
run.clear();
return;
@@ -457,7 +493,7 @@ pub(crate) fn tables(blocks: &[Block], rules: &[[f64; 4]]) -> (Vec<Table>, Vec<b
right,
used.iter().map(|&i| blocks[i].rect[3]).fold(f64::MIN, f64::max),
];
out.push(Table { rect, cols: edges, rows: rows_out, ruled: is_ruled });
out.push(Table { rect, cols: edges, rows: rows_out, ruled: frame });
};
// Runs of consecutive multi-column rows are tables. Rows on either side of a run that
// hold a single cell wide enough to span two columns (a header band, a totals row) join
crates/export/src/lib.rs
+2 -2
View File
@@ -23,14 +23,14 @@ mod as_rtf;
mod clip;
mod layout;
pub use as_docx::docx;
pub(crate) use as_docx::hex;
#[cfg(test)]
use as_docx::{MAX_MEDIA, docx_table};
pub use as_docx::{docx, docx_with_language};
pub(crate) use as_html::esc;
#[cfg(test)]
use as_html::{allowed, base64};
pub use as_html::{html, html_simple};
pub use as_html::{html, html_simple, html_simple_with_language, html_with_language};
pub use as_markdown::markdown;
#[cfg(test)]
use as_markdown::{emphasis, paragraph};
crates/export/src/tests.rs
+86 -2
View File
@@ -137,7 +137,7 @@ fn word_table_cell_widths_follow_the_columns_they_span() {
vec![cell("All", 3)],
vec![cell("First", 1)],
],
ruled: false,
ruled: None,
};
let xml = docx_table(&table);
assert!(xml.contains("<w:gridCol w:w=\"1000\"/><w:gridCol w:w=\"2000\"/><w:gridCol w:w=\"3000\"/>"), "{xml}");
@@ -247,6 +247,61 @@ pub(crate) fn part(zip: &[u8], name: &str) -> String {
panic!("{name} missing")
}
/// The PDF's language (`/Lang`) is the default language of the Word document's text, in the
/// attribute Word keeps for its script, and the HTML page's. No language, or one that is not a
/// well formed tag, leaves both files as they were.
#[test]
fn word_and_html_take_the_documents_language() {
let pages = [page()];
let styles = |lang| part(&docx_with_language(&pages, "Report", lang), "word/styles.xml");
let defaults = |lang: &str| format!("<w:docDefaults><w:rPrDefault><w:rPr>{lang}</w:rPr></w:rPrDefault></w:docDefaults>");
assert!(styles(Some("bg-BG")).contains(&defaults("<w:lang w:val=\"bg-BG\"/>")), "{}", styles(Some("bg-BG")));
assert!(styles(Some(" en-GB ")).contains("<w:lang w:val=\"en-GB\"/>"), "the tag is trimmed");
assert!(styles(Some("zh-Hant-TW")).contains(&defaults("<w:lang w:eastAsia=\"zh-Hant-TW\"/>")), "East Asian text");
assert!(styles(Some("ja")).contains("<w:lang w:eastAsia=\"ja\"/>"), "East Asian text");
assert!(styles(Some("ar-EG")).contains(&defaults("<w:lang w:bidi=\"ar-EG\"/>")), "right-to-left text");
assert!(styles(Some("HE")).contains("<w:lang w:bidi=\"HE\"/>"), "right-to-left text");
// A script subtag decides over the language; private use names no script.
assert!(styles(Some("pa-Arab-PK")).contains("<w:lang w:bidi=\"pa-Arab-PK\"/>"), "Punjabi in Arabic script");
assert!(styles(Some("sr-Latn")).contains("<w:lang w:val=\"sr-Latn\"/>"), "Serbian in Latin script");
assert!(styles(Some("zh-Latn")).contains("<w:lang w:val=\"zh-Latn\"/>"), "Chinese in Latin script");
assert!(styles(Some("en-x-arab")).contains("<w:lang w:val=\"en-x-arab\"/>"), "private use");
assert!(styles(Some("az-Latn-x-arab")).contains("<w:lang w:val=\"az-Latn-x-arab\"/>"), "private use");
assert!(html_with_language(&pages, "Report", Some("bg-BG")).starts_with("<!DOCTYPE html>\n<html lang=\"bg-BG\">\n"));
// A tag longer than 35 characters is still a tag.
let long = "en-US-u-ca-gregory-nu-latn-co-phonebk";
assert!(styles(Some(long)).contains(&format!("<w:lang w:val=\"{long}\"/>")), "{long}");
assert!(html_with_language(&pages, "Report", Some(long)).contains(&format!("<html lang=\"{long}\">")), "{long}");
let too_long = format!("en{}", "-abcdefgh".repeat(29));
// Private use takes one-character subtags.
assert!(html_with_language(&pages, "Report", Some("en-x-a-b")).contains("<html lang=\"en-x-a-b\">"));
for lang in [
None,
Some(""),
Some(" "),
Some("bg_BG"),
Some("b"),
Some("bulg-BG"),
Some("bg--BG"),
Some("bg-"),
Some("en-x"),
Some("en-a"),
Some("en-a-x-private"),
Some("en-US-u-ca-x"),
Some("en-US-abcdefghi"),
Some(too_long.as_str()),
Some("en-\"x\""),
Some("1a-BG"),
Some("\u{431}\u{433}-BG"),
] {
assert!(docx_with_language(&pages, "Report", lang) == docx(&pages, "Report"), "{lang:?}");
assert_eq!(html_with_language(&pages, "Report", lang), html(&pages, "Report"), "{lang:?}");
assert_eq!(html_simple_with_language(&pages, "Report", lang), html_simple(&pages, "Report"), "{lang:?}");
}
// The simple export leaves out styling, not the language.
assert!(html_simple_with_language(&pages, "Report", Some("bg-BG")).starts_with("<!DOCTYPE html>\n<html lang=\"bg-BG\">\n"));
}
#[test]
fn word_opens_text_with_control_characters_and_long_pages() {
// #72: text extracted without a usable /ToUnicode holds C0 controls, which XML forbids;
@@ -887,7 +942,7 @@ fn a_ruled_two_by_two_grid_is_a_table() {
];
let (found, consumed) = tables(&blocks, &grid);
assert_eq!(found.len(), 1);
assert!(found[0].ruled);
assert_eq!(found[0].ruled, Some([true; 4]), "the grid's frame is ruled on every side");
assert_eq!(found[0].rows.len(), 2);
assert!(consumed.iter().all(|used| *used));
let p = Page { width: 320.0, height: 300.0, blocks: blocks.clone(), rules: grid, ..Page::default() };
@@ -899,3 +954,32 @@ fn a_ruled_two_by_two_grid_is_a_table() {
let underlines = [[20.0, 245.0, 300.0, 245.0]];
assert!(tables(&blocks, &underlines).0.is_empty(), "a rule between the rows alone is no grid");
}
/// A grid drawn with inside lines only gets inside borders in Word, not a box the page never
/// draws; an outer line far from the table (a page rule) is not its frame.
#[test]
fn a_ruled_table_gets_outer_borders_only_where_the_page_draws_them() {
let blocks = vec![
block("Name", 25.0, 250.0, 55.0, 261.0),
block("Value", 165.0, 250.0, 195.0, 261.0),
block("Alpha", 25.0, 225.0, 55.0, 236.0),
block("One", 165.0, 225.0, 185.0, 236.0),
];
let inside = vec![[20.0, 245.0, 300.0, 245.0], [160.0, 220.0, 160.0, 270.0]];
let (found, _) = tables(&blocks, &inside);
assert_eq!(found.len(), 1);
assert_eq!(found[0].ruled, Some([false; 4]));
let xml = docx_table(&found[0]);
for edge in ["insideH", "insideV"] {
assert!(xml.contains(&format!("<w:{edge} w:val=\"single\"")), "{xml}");
}
for edge in ["top", "left", "bottom", "right"] {
assert!(xml.contains(&format!("<w:{edge} w:val=\"nil\"/>")), "{xml}");
}
let mut ruled_top = inside.clone();
ruled_top.push([20.0, 270.0, 300.0, 270.0]);
ruled_top.push([0.0, 500.0, 320.0, 500.0]);
ruled_top.push([0.0, 0.0, 320.0, 0.0]);
assert_eq!(tables(&blocks, &ruled_top).0[0].ruled, Some([true, false, false, false]), "only the top line near the table frames it");
}
crates/fonts/README.md
+13 -2
View File
@@ -56,9 +56,12 @@ set). Without that variable `CRAFT_FONTS` is empty and everything below copes:
editor can use the first one that has all of the replacement's glyphs (the faces differ in
coverage, e.g. of Cyrillic).
- `ui_thai_fonts` / `document_thai_font` / `shape_thai`: the `Thai` faces for the interface
and shaped Thai Add Text content. Harfrust positions stacked vowel/tone marks and decomposes
and shaped Thai content. Harfrust positions stacked vowel/tone marks and decomposes
Sara Am; cluster ToUnicode entries preserve the original text. The shared Type 3 writer
embeds outlines, so reopening does not depend on installed fonts.
(`shaped_text`) embeds outlines, so reopening does not depend on installed fonts: Add text,
text boxes, Fill & Sign text, form fields, headers, footers and watermarks all write with it
(`shaped_text::appearance_lines` for appearances, whose glyph streams `store_streams` moves into
the document). `wrap_with` never breaks a line before a Thai mark.
- `craft_glyph(face, ch)` and `face_has(bytes, ch)`: bounded outlines and glyph coverage for any bundled face.
- `SHIPPORI_MINCHO`: Shippori Mincho's bytes, or `None`.
- `TextPlan::for_words`: how the OCR output writes text beyond WinAnsi — per word the first
@@ -86,3 +89,11 @@ Document font inputs use `embedding`: bounded static TrueType/OpenType-CFF valid
exact program preservation, Unicode/CID/GID mapping and full-face subset completion.
`shaping` uses the same program with harfrust and UAX #9; source UI fallback fonts are
never exported. See [existing-text editing](../../docs/editing-existing-text.md).
`pdf::embedded_sfnt_within` reconstructs a missing Unicode cmap for Identity-H TrueType
subsets from the PDF's ToUnicode and CID-to-glyph maps, and completes a THSarabunPSK subset's
Latin-only one with them. Only the in-memory editor copy changes;
the PDF's font program stays intact. `pdf::LegacyThaiFont` shapes verified THSarabunPSK
positional glyphs into their original PDF codes, retaining the source face and weight. Missing
contextual glyphs are refused. `pdf::recovered_thai_map` recognizes narrowly fingerprinted
damaged exporter maps; callers must record repairs and apply them to page-owned font copies.
crates/fonts/src/lib.rs
+25 -6
View File
@@ -16,6 +16,7 @@ pub mod fallback;
pub mod pdf;
pub mod plan;
mod script;
pub mod shaped_text;
mod shaping;
pub mod source_shaping;
mod std14;
@@ -99,7 +100,6 @@ pub fn wrap(text: &str, size: f64, width: f64) -> Vec<String> {
pub fn wrap_with(text: &str, size: f64, width: f64, measure: impl Fn(&str, f64) -> f64) -> Vec<String> {
let mut lines = Vec::new();
let space = measure(" ", size);
let mut buf = [0u8; 4];
// A CRLF pair is one break, a lone CR or LF is one break each.
for para in text.replace("\r\n", "\n").split(['\n', '\r']) {
let mut line = String::new();
@@ -125,14 +125,24 @@ pub fn wrap_with(text: &str, size: f64, width: f64, measure: impl Fn(&str, f64)
lines.push(std::mem::take(&mut line));
}
line_width = 0.0;
for ch in word.chars() {
let ch_width = measure(ch.encode_utf8(&mut buf), size);
if !line.is_empty() && line_width + ch_width > width {
// Broken between characters, but never before a Thai mark: it stays on its letter.
let mut start = 0;
let mut units = Vec::new();
for (i, ch) in word.char_indices().skip(1) {
if !shaped_text::joins_previous(ch) {
units.extend(word.get(start..i));
start = i;
}
}
units.extend(word.get(start..).filter(|u| !u.is_empty()));
for unit in units {
let unit_width = measure(unit, size);
if !line.is_empty() && line_width + unit_width > width {
lines.push(std::mem::take(&mut line));
line_width = 0.0;
}
line.push(ch);
line_width += ch_width;
line.push_str(unit);
line_width += unit_width;
}
fresh = line.is_empty();
}
@@ -320,6 +330,15 @@ mod tests {
assert_eq!(broken, wrap(" abcdefghijklmnopqrstuvwxyz", 11.0, 40.0));
}
#[test]
fn a_broken_thai_word_keeps_its_marks_on_their_letters() {
// Every character one point wide: "น้ำ" (three characters) is one unit, never split.
let lines = wrap_with("น้ำปั้นบุญฤทธิ์", 1.0, 2.0, |s, _| s.chars().count() as f64);
assert_eq!(lines.concat(), "น้ำปั้นบุญฤทธิ์");
assert_eq!(lines, ["น้ำ", "ปั้", "น", "บุ", "ญฤ", "ท", "ธิ์"]);
assert!(lines.iter().all(|l| l.chars().next().is_some_and(|c| !shaped_text::joins_previous(c))), "{lines:?}");
}
#[test]
fn crlf_is_one_line_break() {
assert_eq!(wrap("a\r\nb", 12.0, 100.0), ["a", "b"]);
crates/fonts/src/pdf.rs
+105 -19
View File
@@ -108,30 +108,49 @@ pub fn embedded_sfnt_within(doc: &Document, font: &Dict, budget: &mut DecodeBudg
let Some(object) = descriptor.get(key) else { continue };
let object = doc.resolve(object);
let Object::Stream(stream) = object.as_ref() else { continue };
let Ok(mut bytes) = budget.decode_within(stream, MAX_SFNT) else { continue };
let Ok(mut bytes) = budget.decode_within(&doc.with_direct_parms(stream), MAX_SFNT) else { continue };
let Ok(face) = skrifa::FontRef::from_index(&bytes, 0) else { continue };
if face.metrics(skrifa::instance::Size::unscaled(), skrifa::instance::LocationRef::default()).units_per_em == 0 {
continue;
}
if face.charmap().mappings().next().is_some() {
let own_cmap = face.charmap().mappings().next().is_some();
// A legacy THSarabunPSK subset may keep a Latin-only cmap beside its Thai glyphs, which
// only the PDF's verified maps reach (`composite_glyph_maps`); other programs keep theirs.
let base_font = font.name(b"BaseFont").map(String::from_utf8_lossy).unwrap_or_default();
if own_cmap && !base_font.rsplit('+').next().is_some_and(|f| f.starts_with("THSarabunPSK")) {
return Some(bytes);
}
// Some PDF generators keep a Mac format-6 code→glyph cmap. egui accepts Unicode
// format-4/12 cmaps. Combine the PDF's Unicode meanings with the source glyph IDs;
// only this in-memory copy gets a new cmap, never the working PDF or a font asset.
let Ok(cmap) = face.cmap() else { continue };
// PDF subsets may remove the Unicode cmap entirely. Rebuild an editor-only
// cmap from ToUnicode and the actual CID-to-glyph map; source bytes stay unchanged.
let metrics = Metrics::from_dict(doc, font);
if metrics.composite {
continue;
}
let mut mappings = std::collections::BTreeMap::new();
for code in 0..=255 {
let Some(text) = metrics.unicode.text(code) else { continue };
let Some(ch) = only_char(&text) else { continue };
let Some(glyph) = cmap.map_codepoint(code).or_else(|| cmap.map_codepoint(code + 0xf000)) else { continue };
let glyph = glyph.to_u32();
if glyph > 0 {
mappings.entry(u32::from(ch)).or_insert(glyph);
if metrics.composite {
if let Some((map, _)) = composite_glyph_maps(doc, font, &bytes, &metrics, budget) {
mappings = map;
}
if own_cmap {
let charmap = face.charmap();
if mappings.keys().all(|&u| char::from_u32(u).is_some_and(|c| charmap.map(c).is_some())) {
return Some(bytes);
}
// The program's own mappings stay; the PDF's fill only what they lack.
mappings.extend(charmap.mappings().filter(|(_, g)| g.to_u32() != 0).take(65_537).map(|(u, g)| (u, g.to_u32())));
let mut merged = bytes.clone();
let usable = append_editor_cmap(&mut merged, &mappings).is_some()
&& skrifa::FontRef::new(&merged).is_ok_and(|font| font.charmap().mappings().next().is_some());
return Some(if usable { merged } else { bytes });
}
} else if own_cmap {
return Some(bytes);
} else if let Ok(cmap) = face.cmap() {
for code in 0..=255 {
let Some(text) = metrics.unicode.text(code) else { continue };
let Some(ch) = only_char(&text) else { continue };
let Some(glyph) = cmap.map_codepoint(code).or_else(|| cmap.map_codepoint(code + 0xf000)) else { continue };
let glyph = glyph.to_u32();
if glyph > 0 {
mappings.entry(u32::from(ch)).or_insert(glyph);
}
}
}
if !mappings.is_empty()
@@ -214,14 +233,14 @@ fn embedded_widths(doc: &Document, font: &Dict, m: &mut Metrics) {
}
}
/// Replace an existing cmap directory entry with a bounded Unicode format-12 table.
/// Add or replace an editor-only cmap with a bounded Unicode format-12 table.
/// Checked ranges prevent damaged sfnt offsets from indexing outside the source program.
fn append_editor_cmap(bytes: &mut Vec<u8>, mappings: &std::collections::BTreeMap<u32, u32>) -> Option<()> {
if mappings.len() > 65536 {
return None;
}
let count = u16::from_be_bytes(bytes.get(4..6)?.try_into().ok()?) as usize;
if count > 256 {
if count == 0 || count > 256 {
return None;
}
let mut entry = None;
@@ -236,7 +255,10 @@ fn append_editor_cmap(bytes: &mut Vec<u8>, mappings: &std::collections::BTreeMap
head = Some(u32::from_be_bytes(record.get(8..12)?.try_into().ok()?) as usize);
}
}
let entry = entry?;
let (entry, head) = match entry {
Some(entry) => (entry, head),
None => insert_editor_cmap_slot(bytes, count, head)?,
};
let mut cmap = Vec::with_capacity(28 + mappings.len() * 12);
cmap.extend_from_slice(&[0, 0, 0, 1, 0, 3, 0, 10, 0, 0, 0, 12, 0, 12, 0, 0]);
cmap.extend_from_slice(&(16_u32 + u32::try_from(mappings.len()).ok()? * 12).to_be_bytes());
@@ -264,6 +286,17 @@ fn append_editor_cmap(bytes: &mut Vec<u8>, mappings: &std::collections::BTreeMap
bytes.get_mut(entry + 8..entry + 12)?.copy_from_slice(&offset.to_be_bytes());
bytes.get_mut(entry + 12..entry + 16)?.copy_from_slice(&u32::try_from(cmap.len()).ok()?.to_be_bytes());
bytes.extend_from_slice(&cmap);
// SFNT directories are sorted by tag. Some readers tolerate the unsorted directory of
// a PDF subset, but shaping engines may binary-search it and otherwise miss the cmap.
// Adding a slot increased the directory count; include the new record in this sort.
let tables = usize::from(u16::from_be_bytes(bytes.get(4..6)?.try_into().ok()?));
let end = 12usize.checked_add(tables.checked_mul(16)?)?;
let mut records: Vec<[u8; 16]> = bytes.get(12..end)?.as_chunks::<16>().0.to_vec();
records.sort_by_key(|r| [r[0], r[1], r[2], r[3]]);
for (index, record) in records.iter().enumerate() {
let at = 12 + index * 16;
bytes.get_mut(at..at + 16)?.copy_from_slice(record);
}
if let Some(head) = head {
bytes.get_mut(head.checked_add(8)?..head.checked_add(12)?)?.fill(0);
let adjustment = 0xb1b0afba_u32.wrapping_sub(checksum(bytes));
@@ -272,6 +305,42 @@ fn append_editor_cmap(bytes: &mut Vec<u8>, mappings: &std::collections::BTreeMap
Some(())
}
/// Reserve a new cmap directory slot, retaining every original table payload.
fn insert_editor_cmap_slot(bytes: &mut Vec<u8>, mut count: usize, mut head: Option<usize>) -> Option<(usize, Option<usize>)> {
if count >= 256 {
return None;
}
let end = 12usize.checked_add(count.checked_mul(16)?)?;
bytes.get(..end)?;
// Keep all original table bytes, shifting them by one directory record.
for i in 0..count {
let at = 12 + i * 16;
let offset = u32::from_be_bytes(bytes.get(at + 8..at + 12)?.try_into().ok()?);
let len = u32::from_be_bytes(bytes.get(at + 12..at + 16)?.try_into().ok()?);
if (offset as usize) < end {
return None;
}
bytes.get(offset as usize..offset.checked_add(len)? as usize)?;
}
bytes.splice(end..end, [0; 16]);
for i in 0..count {
let at = 12 + i * 16;
let offset = u32::from_be_bytes(bytes.get(at + 8..at + 12)?.try_into().ok()?).checked_add(16)?;
bytes.get_mut(at + 8..at + 12)?.copy_from_slice(&offset.to_be_bytes());
}
head = head.and_then(|offset| offset.checked_add(16));
bytes.get_mut(end..end + 4)?.copy_from_slice(b"cmap");
count += 1;
let count = u16::try_from(count).ok()?;
let selector = count.ilog2() as u16;
let range = (1u16.checked_shl(u32::from(selector))?).checked_mul(16)?;
bytes.get_mut(4..6)?.copy_from_slice(&count.to_be_bytes());
bytes.get_mut(6..8)?.copy_from_slice(&range.to_be_bytes());
bytes.get_mut(8..10)?.copy_from_slice(&selector.to_be_bytes());
bytes.get_mut(10..12)?.copy_from_slice(&count.checked_mul(16)?.checked_sub(range)?.to_be_bytes());
Some((end, head))
}
fn descriptor_flags(doc: &Document, descriptor: &Dict) -> u32 {
let value = descriptor.get(b"Flags").and_then(|v| doc.resolve(v).as_f64()).unwrap_or(0.0) as i64;
u32::try_from(value).unwrap_or(0)
@@ -887,6 +956,19 @@ impl Metrics {
}
}
/// Whether the embedded program draws an outline for `code`. Known (`Some`) only for a subset
/// Type 0 font over a readable TrueType program, where a dropped glyph is left empty; `None`
/// when the program can't tell, for example a simple or CFF font. [`Metrics::encode`] accepts
/// any code with a `/W` entry (#1443), so a caller that would rather use another subset of the
/// same face than draw nothing asks this too. An empty space glyph is normal.
pub fn has_outline(&self, code: u32) -> Option<bool> {
if !self.composite || self.outlined_cids.is_empty() {
return None;
}
let cid = self.cid(code);
Some(self.outlined_cids.get(cid as usize / 64).is_some_and(|w| (*w >> (cid % 64)) & 1 == 1))
}
/// Whether `code` shows exactly `c` and has a glyph for it.
fn shows(&self, code: u32, c: char) -> bool {
self.unicode.text(code).is_some_and(|t| t.chars().eq(std::iter::once(c))) && self.has_glyph(code)
@@ -1510,3 +1592,7 @@ mod encoding_tests {
assert_eq!(budget.remaining(), 0);
}
}
mod thai;
use thai::composite_glyph_maps;
pub use thai::{ExistingThaiRun, LegacyThaiFont, recovered_thai_map};
crates/fonts/src/pdf/tests.rs
+41
View File
@@ -892,3 +892,44 @@ fn low_codes_resolve_to_the_covering_range() {
assert_eq!(covered, [5, 6, 7, 100, 101]);
assert_eq!((u.covering(0xFFFF).is_some(), u.covering(0x1_0001).is_some(), u.covering(0x1_0002).is_none()), (true, true, true));
}
#[test]
fn composite_subset_without_cmap_uses_pdf_glyph_ids_in_editor_and_shaper_without_replacing_the_program() {
use skrifa::{MetadataProvider as _, raw::TableProvider as _};
let original = include_bytes!("../../../../assets/fonts/Inter-Regular.ttf");
let expected = skrifa::FontRef::new(original).unwrap().charmap().map('Ω').unwrap();
let mut subset = original.to_vec();
let count = u16::from_be_bytes(subset[4..6].try_into().unwrap()) as usize;
let records: Vec<_> = subset[12..12 + count * 16].as_chunks::<16>().0.iter().filter(|r| &r[..4] != b"cmap").map(|r| r.to_vec()).collect();
subset[4..6].copy_from_slice(&(records.len() as u16).to_be_bytes());
for (index, record) in records.iter().enumerate() {
subset[12 + index * 16..28 + index * 16].copy_from_slice(record);
}
assert!(skrifa::FontRef::new(&subset).unwrap().cmap().is_err());
let mut doc = Document::new_empty();
let program = doc.add(Object::Stream(pdfcraft_cos::Stream::flate(Dict::new(), &subset)));
let mut descriptor = Dict::new();
descriptor.set(b"FontFile2".to_vec(), Object::Ref(program));
let mut gids = vec![0; 36];
gids[34..36].copy_from_slice(&(expected.to_u32() as u16).to_be_bytes());
let gids = doc.add(Object::Stream(pdfcraft_cos::Stream::flate(Dict::new(), &gids)));
let mut descendant = Dict::new();
descendant.set(b"FontDescriptor".to_vec(), Object::Dict(descriptor));
descendant.set(b"CIDToGIDMap".to_vec(), Object::Ref(gids));
let unicode = doc.add(Object::Stream(pdfcraft_cos::Stream::flate(Dict::new(), b"1 beginbfchar <0011> <03a9> endbfchar")));
let source = font(
&mut doc,
vec![
("Subtype", Object::name("Type0")),
("Encoding", Object::name("Identity-H")),
("DescendantFonts", Object::Array(vec![Object::Dict(descendant)])),
("ToUnicode", Object::Ref(unicode)),
],
);
let editor = embedded_sfnt(&doc, &source).expect("the source outlines must be usable in the editor");
assert_eq!(skrifa::FontRef::new(&editor).unwrap().charmap().map('Ω'), Some(expected));
let shaped = crate::source_shaping::shape(&editor, "Ω").expect("the rebuilt cmap must work in the actual shaping engine too");
assert_eq!(shaped.glyphs[0].id, expected);
let Object::Stream(saved) = doc.get(program).as_ref().clone() else { panic!("fixture") };
assert_eq!(saved.decoded().unwrap(), subset, "the source PDF font program stays byte-for-byte intact");
}
crates/fonts/src/pdf/thai.rs
+734
View File
@@ -0,0 +1,734 @@
//! Recover a narrowly identified legacy Thai exporter map, not arbitrary missing Unicode.
//!
//! Some THSarabunPSK Identity-H subsets retain original glyph IDs but write Latin/control
//! destinations for Thai in ToUnicode (including an accidental 00ff..01ff range). The intact
//! Thai anchors identify the original nominal Thai order. Only that complete fingerprint is
//! accepted, and then only when every glyph the subset kept advances as its assumed character
//! does (a mark takes none); correct maps and unrelated fonts are never changed. No font
//! artwork is bundled.
use super::{Dict, Document, Metrics, dict, only_char, unicode_map};
use pdfcraft_cos::{DecodeBudget, Object};
use std::collections::BTreeMap;
/// Recover the nominal Thai codes and the presentation marks of this damaged legacy export.
/// The caller must record the repair, and attach the map to a copy of the page's font.
pub fn recovered_thai_map(doc: &Document, font: &Dict) -> Option<Vec<(u16, char)>> {
use skrifa::{MetadataProvider as _, raw::TableProvider as _};
let base = String::from_utf8_lossy(font.name(b"BaseFont")?);
let family = base.rsplit('+').next()?;
if !matches!(family, "THSarabunPSK" | "THSarabunPSK-Bold" | "THSarabunPSK-Italic" | "THSarabunPSK-BoldItalic")
|| font.name(b"Subtype") != Some(b"Type0")
|| font.name(b"Encoding") != Some(b"Identity-H")
{
return None;
}
let desc = dict(doc, font.get(b"DescendantFonts").and_then(|a| doc.resolve(a).as_array().and_then(|a| a.first()).cloned()).as_ref())?;
if desc.name(b"CIDToGIDMap") != Some(b"Identity") {
return None;
}
let map = unicode_map(doc, font, true);
// Five separated consonants and three contextual marks must agree, together with the
// corrupt destinations and corrupt range. This is deliberately stricter than font-name
// detection: a renamed/subsetted font with different glyph order must not be guessed.
let fingerprint = [
(217, '\u{0e04}'),
(226, '\u{0e0d}'),
(250, '\u{0e25}'),
(256, '\u{0e2b}'),
(258, '\u{0e2d}'),
(356, '\u{0e4c}'),
(358, '\u{0e31}'),
(360, '\u{0e47}'),
(214, '\u{000f}'),
(240, ')'),
(248, '#'),
(261, '>'),
(263, '@'),
(265, '>'),
(273, 'N'),
(352, 'N'),
(353, 'O'),
];
let matches = |pairs: &[(u32, char)], last: u16| {
pairs.iter().all(|&(code, ch)| map.text(code).is_some_and(|t| t.chars().eq([ch])))
&& map.ranges.iter().any(|r| r.lo == 255 && r.hi == 511 && r.units.as_ref() == [last])
};
// The same exporter emitted three distinct subset maps. Each must match its entire
// intact-anchor and corrupt-destination fingerprint, including the oversized range.
let body = matches(&fingerprint, 0x32);
let attachment = matches(
&[
(217, '\u{0e04}'),
(256, '\u{0e2b}'),
(258, '\u{0e2d}'),
(297, '\u{0e58}'),
(347, '\u{0e48}'),
(356, '\u{0e4c}'),
(358, '\u{0e31}'),
(360, '\u{0e47}'),
(214, '\u{000f}'),
(226, '\u{000f}'),
(240, '\u{0027}'),
(248, '/'),
(250, '1'),
(265, 'B'),
(273, 'N'),
(352, 'N'),
(353, 'O'),
],
0x32,
);
let bold = matches(
&[
(217, '\u{0e04}'),
(226, '\u{0e0d}'),
(232, '\u{0e13}'),
(234, '\u{0e15}'),
(236, '\u{0e17}'),
(243, '\u{0e1e}'),
(250, '\u{0e25}'),
(256, '\u{0e2b}'),
(258, '\u{0e2d}'),
(356, '\u{0e4c}'),
(214, '\u{000f}'),
(240, '!'),
(248, '1'),
(263, '@'),
(273, 'N'),
(352, 'N'),
(353, 'O'),
],
0x30,
);
if !(body || attachment || bold) {
return None;
}
let nominal = (214u16..=298).filter_map(|gid| {
let unicode = u32::from(gid) + if gid <= 271 { 0xd2b } else { 0xd2f };
char::from_u32(unicode).map(|ch| (gid, ch))
});
// Contextual marks have the same character as the nominal mark, with a moved outline.
let contextual = [
(343, '\u{0e34}'),
(344, '\u{0e35}'),
(347, '\u{0e48}'),
(351, '\u{0e4c}'),
(352, '\u{0e48}'),
(353, '\u{0e49}'),
(356, '\u{0e4c}'),
(358, '\u{0e31}'),
(360, '\u{0e47}'),
];
// The anchors identify the exporter; the font program confirms each glyph's advance class
// (not its identity, which nothing in such a subset records). Every glyph the
// subset kept must advance as the character it is given (a vowel or tone mark set on its
// base takes no advance, every other character does), else the order differs somewhere and
// nothing is mapped. A glyph the subset dropped is left unmapped: no code can draw it, and
// its advance is the subsetter's filler, not the glyph's.
let program = embedded_truetype(doc, &desc)?;
let face = skrifa::FontRef::new(&program).ok()?;
let loca = face.loca(None).ok()?;
let advances = face.glyph_metrics(skrifa::instance::Size::unscaled(), skrifa::instance::LocationRef::default());
let mut out = Vec::new();
for (gid, ch) in nominal.chain(contextual) {
let at = usize::from(gid);
if loca.get_raw(at).zip(loca.get_raw(at + 1)).is_none_or(|(start, end)| end <= start) {
continue;
}
if advances.advance_width(skrifa::GlyphId::new(u32::from(gid))).is_none_or(|w| (w == 0.0) != is_thai_mark(ch)) {
return None;
}
out.push((gid, ch));
}
Some(out)
}
/// The character each legacy Windows Thai presentation form U+F700 + index stands for: ฐ and ญ
/// without descenders, and the vowels and tone marks moved left, lower or both (the PUA
/// assignments of <https://linux.thai.net/~thep/th-otf/shaping.html>).
const PUA_BASES: [char; 27] = ['ฐ', 'ิ', 'ี', 'ึ', 'ื', '่', '้', '๊', '๋', '์', '่', '้', '๊', '๋', '์', 'ญ', 'ั', 'ํ', '็', '่', '้', '๊', '๋', '์', 'ุ', 'ู', 'ฺ'];
/// Whether a Thai character is a vowel or tone mark set above or below its base, with no advance
/// of its own (U+0E31, U+0E34–U+0E3A, U+0E47–U+0E4E). Sara Am and the other vowels advance.
fn is_thai_mark(ch: char) -> bool {
matches!(u32::from(ch), 0x0e31 | 0x0e34..=0x0e3a | 0x0e47..=0x0e4e)
}
/// The glyph a legacy THSarabunPSK program draws Thai character `ch` with: U+0E01–U+0E3A are
/// glyphs 214–271 and U+0E3F–U+0E5B glyphs 272–298 (the order `recovered_thai_map` assumes).
fn legacy_glyph(ch: u32) -> Option<u32> {
match ch {
0x0e01..=0x0e3a => ch.checked_sub(0xd2b),
0x0e3f..=0x0e5b => ch.checked_sub(0xd2f),
_ => None,
}
}
/// The TrueType program (`/FontFile2`) of a CIDFont, decoded within the editor's size cap.
fn embedded_truetype(doc: &Document, descendant: &Dict) -> Option<Vec<u8>> {
let descriptor = dict(doc, descendant.get(b"FontDescriptor"))?;
let program = doc.resolve(descriptor.get(b"FontFile2")?);
let Object::Stream(program) = program.as_ref() else { return None };
DecodeBudget::new(super::MAX_SFNT, 0).decode_within(&doc.with_direct_parms(program), super::MAX_SFNT).ok()
}
/// Source glyph positions and the original PDF character codes that draw them. Empty codes
/// represent whitespace advances: no glyph is added to the font or substituted for a space.
pub struct ExistingThaiRun {
pub shaped: crate::source_shaping::ShapedRun,
pub codes: Vec<Vec<u8>>,
}
/// Unicode-to-glyph and glyph-to-PDF-code maps for a two-byte Identity-H TrueType font.
/// Used by the editor-only cmap adapter and by direct source-font shaping.
pub(super) fn composite_glyph_maps(
doc: &Document,
font: &Dict,
bytes: &[u8],
metrics: &Metrics,
budget: &mut DecodeBudget,
) -> Option<(BTreeMap<u32, u32>, BTreeMap<u32, u16>)> {
use skrifa::{MetadataProvider as _, raw::TableProvider as _};
if font.name(b"Encoding") != Some(b"Identity-H") {
return None;
}
let descendant = dict(doc, font.get(b"DescendantFonts").and_then(|a| doc.resolve(a).as_array().and_then(|a| a.first()).cloned()).as_ref())?;
let cid_map = match descendant.get(b"CIDToGIDMap").map(|m| doc.resolve(m)).as_deref() {
None => None,
Some(Object::Name(n)) if n == b"Identity" => None,
Some(Object::Stream(s)) => Some(budget.decode_within(&doc.with_direct_parms(s), 1 << 17).ok()?),
_ => return None,
};
let face = skrifa::FontRef::new(bytes).ok()?;
let loca = face.loca(None).ok()?;
let advances = face.glyph_metrics(skrifa::instance::Size::unscaled(), skrifa::instance::LocationRef::default());
let outlined = |gid: u32| loca.get_raw(gid as usize).zip(loca.get_raw(gid as usize + 1)).is_some_and(|(a, b)| b > a);
let glyph_of = |cid: u16| -> Option<u32> {
match &cid_map {
None => Some(u32::from(cid)),
Some(map) => {
let at = usize::from(cid).checked_mul(2)?;
Some(u32::from(u16::from_be_bytes(map.get(at..at + 2)?.try_into().ok()?)))
}
}
};
let mut unicode = BTreeMap::new();
let mut codes = BTreeMap::new();
for code in 0..=u16::MAX {
let Some(gid) = glyph_of(code) else { continue };
if gid == 0 || gid as usize >= loca.len().saturating_sub(1) {
continue;
}
let text = metrics.text_of(u32::from(code));
let ch = text.as_deref().and_then(only_char);
let space = ch == Some(' ') && advances.advance_width(skrifa::GlyphId::new(gid)).is_some_and(|w| w > 0.0);
if !outlined(gid) && !space {
continue;
}
codes.entry(gid).or_insert(code);
if let Some(ch) = ch.filter(|c| !c.is_control()) {
unicode.entry(u32::from(ch)).and_modify(|old: &mut u32| *old = (*old).min(gid)).or_insert(gid);
}
}
// The verified THSarabunPSK glyph order is retained by these PDF subsets. Restore the
// Windows Thai PUA alternates for HarfRust's legacy shaper, only where outlines survive.
// See https://linux.thai.net/~thep/th-otf/shaping.html (legacy PUA assignments).
let family = metrics.base_font.rsplit('+').next().unwrap_or(&metrics.base_font);
let alternate_meanings = metrics.text_of(352).is_some_and(|t| t == "่") && metrics.text_of(353).is_some_and(|t| t == "้");
// Each alternate the subset kept must advance as its form does: U+F700 and U+F70F are ฐ and
// ญ without their descenders, the rest marks with no advance of their own. Any that doesn't
// means another glyph order, and then none is used: the wrong mark would draw silently.
let kept: Vec<(usize, u32)> = (342u32..=368).enumerate().filter(|&(_, gid)| outlined(gid)).collect();
let in_order =
kept.iter().all(|&(index, gid)| advances.advance_width(skrifa::GlyphId::new(gid)).is_some_and(|w| (w > 0.0) == matches!(index, 0 | 15)));
// Where the PDF says what an alternate's code is (an intact ToUnicode gives the character it
// stands for), it must be that alternate's base character: two alternates of the same kind
// in another order pass the advance check, but not this one.
let code_of = |gid: u32| match &cid_map {
None => u16::try_from(gid).ok(),
Some(_) => codes.get(&gid).copied(),
};
let meanings_agree = kept.iter().all(|&(index, gid)| {
let stated = code_of(gid).and_then(|code| metrics.text_of(u32::from(code))).and_then(|t| only_char(&t));
stated.filter(|c| matches!(u32::from(*c), 0x0e01..=0x0e5b)).is_none_or(|c| PUA_BASES.get(index) == Some(&c))
});
if family.starts_with("THSarabunPSK")
&& unicode.get(&0x0e01) == Some(&214)
&& unicode.get(&0x0e2d) == Some(&258)
&& (alternate_meanings || unicode.contains_key(&0xf70a))
&& in_order
&& meanings_agree
{
for (index, gid) in kept {
let code = match &cid_map {
None => u16::try_from(gid).ok(),
Some(_) => codes.get(&gid).copied(),
};
if let Some(code) = code {
unicode.insert(0xf700 + index as u32, gid);
codes.entry(gid).or_insert(code);
}
}
}
Some((unicode, codes))
}
/// Shape Thai in a THSarabunPSK source font's own outlines and return codes for that same font.
/// No font program, family, weight or size is replaced. A program with OpenType Thai mark
/// positioning (a complete face loaded with Load font, say) is shaped by that layout; one
/// without it must be a verified legacy positional program, shaped with its nominal glyphs and
/// Windows PUA alternates. A subset missing a glyph the text needs is refused, so the caller can
/// try another subset of the same face or ask for its full font.
pub struct LegacyThaiFont {
bytes: Vec<u8>,
codes: BTreeMap<u32, u16>,
space: f64,
/// The program positions Thai marks itself (GPOS `mark`/`mkmk`).
positioned: bool,
}
impl LegacyThaiFont {
/// Read and verify the source font once for all wrapping/preview measurements. `None` for
/// another family, a font with no Thai, and a program without Thai mark positioning whose
/// Thai glyphs are not in the legacy order: the nominal and alternate marks this shaper
/// assumes can't be trusted there, so it is shaped as an ordinary source font.
///
/// Its streams are decoded against `budget`, which the caller shares between every font it
/// reads: what they keep is the programs decoded and the editor cmaps added to them.
pub fn from_pdf(doc: &Document, font: &Dict, space: f64, budget: &mut DecodeBudget) -> Result<Option<Self>, String> {
let m = Metrics::from_dict(doc, font);
if !m.base_font.rsplit('+').next().unwrap_or(&m.base_font).starts_with("THSarabunPSK") {
return Ok(None);
}
if !space.is_finite() || space < 0.0 {
return Err("the source space width is invalid".into());
}
let mut bytes = super::embedded_sfnt_within(doc, font, budget).ok_or("the source Thai font has no readable glyph program")?;
let positioned = crate::source_shaping::has_thai_layout(&bytes);
let (unicode, codes) = composite_glyph_maps(doc, font, &bytes, &m, budget).ok_or("the Thai source glyph codes cannot be recovered safely")?;
// A subset keeps only the glyphs its document used, so no particular letter or alternate
// is required here: [`Self::shape`] refuses a text whose glyphs, or required alternates
// ([`require_pua`]), the subset lacks. Without its own positioning, every Thai glyph it
// does keep must be where the legacy order puts that character, or one of the PUA
// alternates for a mark (or for ฐ and ญ without their descenders).
let mut thai = unicode.range(0x0e01..=0x0e5b).peekable();
if thai.peek().is_none() {
return Ok(None);
}
let usable = positioned
|| thai.all(|(&ch, &gid)| {
legacy_glyph(ch) == Some(gid)
|| ((342..=368).contains(&gid) && (char::from_u32(ch).is_some_and(is_thai_mark) || matches!(ch, 0x0e0d | 0x0e10)))
});
if !usable {
return Ok(None);
}
if !positioned {
// Glyphs 342–368 are alternates moved for a context. Mapped as the nominal character
// (its glyph dropped), one would be drawn out of place where the nominal form is
// needed: the shaper reaches it only through its PUA form, and that text is refused.
use skrifa::MetadataProvider as _;
let alternate = |u: u32, g: skrifa::GlyphId| matches!(u, 0x0e01..=0x0e5b) && (342..=368).contains(&g.to_u32());
let charmap = skrifa::FontRef::new(&bytes).map_err(|e| e.to_string())?.charmap();
if charmap.mappings().any(|(u, g)| alternate(u, g)) {
let nominal: BTreeMap<u32, u32> =
charmap.mappings().filter(|&(u, g)| g.to_u32() != 0 && !alternate(u, g)).take(65_537).map(|(u, g)| (u, g.to_u32())).collect();
super::append_editor_cmap(&mut bytes, &nominal).ok_or("the Thai source glyph map cannot be rebuilt safely")?;
}
}
Ok(Some(Self { bytes, codes, space, positioned }))
}
/// Shape into glyphs that are already in this exact source face.
pub fn shape(&self, text: &str) -> Result<ExistingThaiRun, String> {
use skrifa::MetadataProvider as _;
if text.len() > 262_144 || text.contains(['\n', '\r']) {
return Err("shape one bounded Thai source row at a time".into());
}
// Only the legacy shaper needs the PUA alternates; a positioned program moves its marks.
if !self.positioned {
let face = skrifa::FontRef::new(&self.bytes).map_err(|e| e.to_string())?;
require_pua(&face.charmap(), text)?;
}
let mut shaped = crate::source_shaping::ShapedRun { glyphs: Vec::new(), advance: 0.0 };
let mut codes = Vec::new();
// Legacy PDF subsets often omit the space glyph. Keep its measured PDF advance as a
// positioning gap, rather than drawing .notdef or changing to another font.
for piece in text.split_inclusive(' ') {
let word = piece.strip_suffix(' ').unwrap_or(piece);
if !word.is_empty() {
let run = if self.positioned {
crate::source_shaping::shape(&self.bytes, word)?
} else {
crate::source_shaping::shape_legacy_thai(&self.bytes, word)?
};
for mut glyph in run.glyphs {
let code = self
.codes
.get(&glyph.id.to_u32())
.ok_or("the Thai subset lacks a required contextual glyph; load the complete original face")?;
glyph.position[0] += shaped.advance;
codes.push(code.to_be_bytes().to_vec());
shaped.glyphs.push(glyph);
}
shaped.advance += run.advance;
}
if piece.ends_with(' ') {
shaped.glyphs.push(crate::source_shaping::ShapedGlyph {
id: skrifa::GlyphId::NOTDEF,
position: [shaped.advance, 0.0],
text: " ".into(),
});
codes.push(Vec::new());
shaped.advance += self.space;
}
if shaped.glyphs.len() > 65_534 || !shaped.advance.is_finite() {
return Err("Thai source row exceeds the shaping budget".into());
}
}
Ok(ExistingThaiRun { shaped, codes })
}
}
/// Verify every context-dependent PUA alternate before asking the legacy shaper to use it.
/// Otherwise a subset missing an alternate would silently draw the wrong high/left mark.
fn require_pua(cmap: &skrifa::charmap::Charmap<'_>, text: &str) -> Result<(), String> {
let mut base = 0u32;
let mut above = false;
let mut chars = text.chars().peekable();
while let Some(ch) = chars.next() {
let c = u32::from(ch);
let ascender = matches!(base, 0xe1b | 0xe1d | 0xe1f);
let pua = match c {
0xe31 if ascender => Some(0xf710),
0xe34..=0xe37 if ascender => Some(0xf701 + c - 0xe34),
0xe47 if ascender => Some(0xf712),
0xe4d if ascender => Some(0xf711),
0xe48..=0xe4c => {
let high = above || chars.peek().is_some_and(|c| *c == 'ำ');
if ascender {
Some((if high { 0xf713 } else { 0xf705 }) + c - 0xe48)
} else if !high {
Some(0xf70a + c - 0xe48)
} else {
None
}
}
0xe38..=0xe3a if matches!(base, 0xe0e | 0xe0f) => Some(0xf718 + c - 0xe38),
0xe38..=0xe3a if base == 0xe0d => Some(0xf70f),
0xe38..=0xe3a if base == 0xe10 => Some(0xf700),
0xe33 if ascender => Some(0xf711),
_ => None,
};
if pua.is_some_and(|p| char::from_u32(p).is_none_or(|c| cmap.map(c).is_none())) {
return Err(format!(
"the Thai subset lacks a required contextual mark for {ch} (U+{c:04X}); load the complete original face or explicitly select a replacement font"
));
}
if matches!(c, 0xe01..=0xe2e) {
base = c;
above = false;
} else if matches!(c, 0xe31 | 0xe34..=0xe37 | 0xe47 | 0xe4d..=0xe4e) {
above = true;
} else if !matches!(c, 0xe38..=0xe3a | 0xe48..=0xe4c) {
base = 0;
above = false;
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::{LegacyThaiFont, Metrics, composite_glyph_maps, is_thai_mark, recovered_thai_map};
use pdfcraft_cos::{DecodeBudget, Dict, Document, Object, Stream};
use std::collections::HashMap;
/// Where table `tag` starts in an sfnt.
fn table(program: &[u8], tag: &[u8; 4]) -> usize {
let count = u16::from_be_bytes([program[4], program[5]]) as usize;
let record = (0..count).map(|i| 12 + i * 16).find(|&r| &program[r..r + 4] == tag).unwrap();
u32::from_be_bytes(program[record + 8..record + 12].try_into().unwrap()) as usize
}
/// Set glyph `gid`'s advance (a long `hmtx` entry).
fn set_advance(program: &mut [u8], gid: usize, advance: u16) {
let long = u16::from_be_bytes(program[table(program, b"hhea") + 34..][..2].try_into().unwrap()) as usize;
assert!(gid < long, "a long hmtx entry");
let at = table(program, b"hmtx") + gid * 4;
program[at..at + 2].copy_from_slice(&advance.to_be_bytes());
}
/// Empty glyph `gid`, as a subsetter drops it: its data is left as the previous glyph's padding.
fn drop_glyph(program: &mut [u8], gid: usize) {
let long = program[table(program, b"head") + 51] == 1;
let loca = table(program, b"loca");
if long {
let next: [u8; 4] = program[loca + (gid + 1) * 4..][..4].try_into().unwrap();
program[loca + gid * 4..][..4].copy_from_slice(&next);
} else {
let next: [u8; 2] = program[loca + (gid + 1) * 2..][..2].try_into().unwrap();
program[loca + gid * 2..][..2].copy_from_slice(&next);
}
}
/// Inter's outlines standing in for a THSarabunPSK subset (no proprietary font is used): the
/// glyphs where the exporter's order has a Thai mark or a mark-like alternate take no
/// advance, the others 500 units.
fn thai_ordered_program() -> Vec<u8> {
let mut program = crate::script::INTER_REGULAR.to_vec();
for gid in 214..=368usize {
let mark = match gid {
214..=298 => is_thai_mark(char::from_u32(gid as u32 + if gid <= 271 { 0xd2b } else { 0xd2f }).unwrap()),
342..=368 => !matches!(gid, 342 | 357),
_ => continue,
};
set_advance(&mut program, gid, if mark { 0 } else { 500 });
}
program
}
/// A THSarabunPSK Identity-H font over `program`, with `to_unicode`.
fn thai_font(doc: &mut Document, program: &[u8], to_unicode: &[u8]) -> Dict {
let file = Object::Ref(doc.add(Object::Stream(Stream::flate(Dict::new(), program))));
thai_font_over(doc, file, Object::name("Identity"), to_unicode)
}
/// [`thai_font`] over the `/FontFile2` stream `file` and the `/CIDToGIDMap` `cid_to_gid`.
fn thai_font_over(doc: &mut Document, file: Object, cid_to_gid: Object, to_unicode: &[u8]) -> Dict {
let tu = doc.add(Object::Stream(Stream::flate(Dict::new(), to_unicode)));
let mut descriptor = Dict::new();
descriptor.set(b"FontFile2".to_vec(), file);
let mut desc = Dict::new();
desc.set(b"CIDToGIDMap".to_vec(), cid_to_gid);
desc.set(b"FontDescriptor".to_vec(), Object::Dict(descriptor));
let mut font = Dict::new();
font.set(b"Subtype".to_vec(), Object::name("Type0"));
font.set(b"BaseFont".to_vec(), Object::name("ABCDEF+THSarabunPSK"));
font.set(b"Encoding".to_vec(), Object::name("Identity-H"));
font.set(b"DescendantFonts".to_vec(), Object::Array(vec![Object::Dict(desc)]));
font.set(b"ToUnicode".to_vec(), Object::Ref(tu));
font
}
/// `data` as a Flate stream under a PNG predictor whose `/DecodeParms` is given by reference:
/// read without that dictionary, each row's tag byte stays in the decoded bytes.
fn predicted(doc: &mut Document, data: &[u8]) -> Object {
let rows: Vec<u8> = data.iter().flat_map(|byte| [0, *byte]).collect();
let mut parms = Dict::new();
parms.set(b"Predictor".to_vec(), Object::Int(12));
parms.set(b"Columns".to_vec(), Object::Int(1));
let parms = doc.add(Object::Dict(parms));
let mut stream = Stream::flate(Dict::new(), &rows);
stream.dict.set(b"DecodeParms".to_vec(), Object::Ref(parms));
Object::Ref(doc.add(Object::Stream(stream)))
}
/// An intact legacy map: nominal Thai and two tone alternates.
const INTACT: &[u8] = b"2 beginbfrange <00d6> <010f> <0e01> <0110> <012a> <0e3f> endbfrange 2 beginbfchar <0160> <0e48> <0161> <0e49> endbfchar";
/// The exporter's damaged map: intact anchors, Latin/control destinations for the other Thai
/// codes, and the oversized range first.
const DAMAGED: &[u8] = b"begincmap 1 begincodespacerange <0000> <ffff> endcodespacerange \
1 beginbfrange <00ff><01ff><0032> endbfrange 17 beginbfrange \
<00d9><00d9><0e04> <00e2><00e2><0e0d> <00fa><00fa><0e25> <0100><0100><0e2b> <0102><0102><0e2d> \
<0164><0164><0e4c> <0166><0166><0e31> <0168><0168><0e47> \
<00d6><00d6><000f> <00f0><00f0><0029> <00f8><00f8><0023> <0105><0105><003e> <0107><0107><0040> \
<0109><0109><003e> <0111><0111><004e> <0160><0160><004e> <0161><0161><004f> endbfrange endcmap";
/// The anchors identify the exporter, but the glyph order is still checked glyph by glyph: a
/// mark where a letter belongs (or the reverse) anywhere in the kept glyphs refuses the whole
/// map, and a glyph the subset dropped is left unmapped whatever its filler advance says.
#[test]
fn recovered_thai_map_checks_every_kept_glyph_against_the_font_program() {
let mut doc = Document::new_empty();
let font = thai_font(&mut doc, &thai_ordered_program(), DAMAGED);
let recovered: HashMap<_, _> = recovered_thai_map(&doc, &font).unwrap().into_iter().collect();
assert_eq!(recovered[&262], 'ั');
assert_eq!(recovered[&297], '๘');
// 259–261 are ฮ, ฯ and ะ, between the last anchor (อ, 258) and the first mark: a font that
// put Mai Han-akat one glyph earlier passes every anchor and only this check sees it.
let mut shifted = thai_ordered_program();
set_advance(&mut shifted, 261, 0);
set_advance(&mut shifted, 262, 500);
let shifted = thai_font(&mut doc, &shifted, DAMAGED);
assert!(recovered_thai_map(&doc, &shifted).is_none(), "a kept glyph that advances unlike its character refuses the map");
let mut dropped = thai_ordered_program();
drop_glyph(&mut dropped, 262);
set_advance(&mut dropped, 262, 500);
let dropped = thai_font(&mut doc, &dropped, DAMAGED);
let recovered = recovered_thai_map(&doc, &dropped).unwrap();
assert!(recovered.iter().all(|(gid, _)| *gid != 262), "a dropped glyph is not mapped");
assert!(recovered.iter().any(|(gid, _)| *gid == 263));
let mut unreadable = font.clone();
let Some(Object::Array(desc)) = unreadable.get(b"DescendantFonts").cloned() else { panic!("fixture") };
let Some(Object::Dict(mut desc)) = desc.first().cloned() else { panic!("fixture") };
desc.remove(b"FontDescriptor");
unreadable.set(b"DescendantFonts".to_vec(), Object::Array(vec![Object::Dict(desc)]));
assert!(recovered_thai_map(&doc, &unreadable).is_none(), "without the program nothing is verified, so nothing is mapped");
}
/// `program` without the tables a legacy subsetter leaves out.
fn without_tables(program: &[u8], tags: &[&[u8; 4]]) -> Vec<u8> {
let mut program = program.to_vec();
let count = u16::from_be_bytes([program[4], program[5]]) as usize;
let records: Vec<Vec<u8>> =
program[12..12 + count * 16].chunks(16).filter(|r| !tags.iter().any(|t| &r[..4] == t.as_slice())).map(<[u8]>::to_vec).collect();
program[4..6].copy_from_slice(&(records.len() as u16).to_be_bytes());
for (i, record) in records.iter().enumerate() {
program[12 + i * 16..28 + i * 16].copy_from_slice(record);
}
program
}
/// The font program and `/CIDToGIDMap` are read with `/DecodeParms` given by reference, as a
/// viewer reads them: the map repair, the glyph codes and the shaper see the real bytes.
#[test]
fn thai_font_streams_are_read_with_decode_parms_given_by_reference() {
let stripped = without_tables(&thai_ordered_program(), &[b"cmap", b"GSUB", b"GPOS"]);
let mut doc = Document::new_empty();
let program = predicted(&mut doc, &stripped);
let damaged = thai_font_over(&mut doc, program.clone(), Object::name("Identity"), DAMAGED);
assert!(recovered_thai_map(&doc, &damaged).is_some(), "the program is read with its predictor");
let gids: Vec<u8> = (0u16..=368).flat_map(u16::to_be_bytes).collect();
let map = predicted(&mut doc, &gids);
let font = thai_font_over(&mut doc, program, map, INTACT);
let metrics = Metrics::from_dict(&doc, &font);
let (map, _) = composite_glyph_maps(&doc, &font, &stripped, &metrics, &mut DecodeBudget::new(1 << 20, 0)).unwrap();
assert_eq!(map.get(&0x0e01), Some(&214), "the CIDToGIDMap is read with its predictor");
let shaped =
LegacyThaiFont::from_pdf(&doc, &font, 0.25, &mut DecodeBudget::new(1 << 26, 0)).unwrap().expect("a legacy subset").shape("ก่").unwrap();
assert_eq!(shaped.shaped.glyphs.iter().map(|g| g.id.to_u32()).collect::<Vec<_>>(), [214, 352]);
}
/// A legacy subset that kept a Latin-only cmap of its own still reaches its Thai glyphs through
/// the PDF's verified maps, and keeps its own mappings.
#[test]
fn a_latin_only_cmap_does_not_hide_a_legacy_subsets_thai_glyphs() {
use skrifa::MetadataProvider as _;
let program = without_tables(&thai_ordered_program(), &[b"GSUB", b"GPOS"]);
let latin = skrifa::FontRef::new(&program).unwrap().charmap().map('A').unwrap();
let mut doc = Document::new_empty();
let font = thai_font(&mut doc, &program, INTACT);
let shaped =
LegacyThaiFont::from_pdf(&doc, &font, 0.25, &mut DecodeBudget::new(1 << 26, 0)).unwrap().expect("a legacy subset").shape("ก่").unwrap();
assert_eq!(shaped.shaped.glyphs.iter().map(|g| g.id.to_u32()).collect::<Vec<_>>(), [214, 352]);
let editor = crate::pdf::embedded_sfnt(&doc, &font).unwrap();
assert_eq!(skrifa::FontRef::new(&editor).unwrap().charmap().map('A'), Some(latin), "the program's own mappings stay");
}
/// Only Thai with a mark to set on its base needs the program's mark positioning: digits and
/// bare letters shape without it.
#[test]
fn thai_without_marks_shapes_without_mark_positioning() {
let mut doc = Document::new_empty();
let font = thai_font(&mut doc, &without_tables(&thai_ordered_program(), &[b"cmap", b"GSUB", b"GPOS"]), INTACT);
let program = crate::pdf::embedded_sfnt(&doc, &font).unwrap();
let shaped = crate::source_shaping::shape(&program, "๑๒๓กข").expect("nothing to position");
assert_eq!(shaped.glyphs.iter().map(|g| g.id.to_u32()).collect::<Vec<_>>(), [290, 291, 292, 214, 215]);
let error = crate::source_shaping::shape(&program, "กิ่").unwrap_err();
assert!(error.contains("mark positioning"), "{error}");
}
/// A legacy subset whose map gives a shifted alternate as the nominal mark (its nominal glyph
/// dropped) draws it only where that alternate belongs, and refuses text needing the nominal.
#[test]
fn a_legacy_alternate_is_not_drawn_as_the_nominal_mark() {
let mut program = without_tables(&thai_ordered_program(), &[b"cmap", b"GSUB", b"GPOS"]);
drop_glyph(&mut program, 265);
let unicode = [INTACT, b" 1 beginbfchar <0157> <0e34> endbfchar"].concat();
let mut doc = Document::new_empty();
let font = thai_font(&mut doc, &program, &unicode);
let font = LegacyThaiFont::from_pdf(&doc, &font, 0.25, &mut DecodeBudget::new(1 << 26, 0)).unwrap().expect("a legacy subset");
let ids = |text: &str| font.shape(text).map(|run| run.shaped.glyphs.iter().map(|g| g.id.to_u32()).collect::<Vec<_>>());
assert_eq!(ids("ปิ"), Ok(vec![240, 343]), "after an ascender, the alternate is the form needed");
assert!(ids("กิ").is_err(), "elsewhere the nominal ิ is needed, which the subset dropped");
}
/// A THSarabunPSK program with its own Thai mark positioning is shaped by that layout; one
/// without it only if every Thai glyph it keeps is where the legacy order has that character
/// (the legacy shaper skips the positioning check). A subset needs no particular glyph.
#[test]
fn thai_source_programs_are_shaped_by_their_own_layout_or_as_verified_legacy() {
let unicode = b"2 beginbfrange <00d6> <010f> <0e01> <0110> <012a> <0e3f> endbfrange 2 beginbfchar <0160> <0e48> <0161> <0e49> endbfchar";
// `Some(positioned)` when the program can be shaped in place.
let mode = |program: &[u8], unicode: &[u8]| {
let mut doc = Document::new_empty();
let font = thai_font(&mut doc, program, unicode);
LegacyThaiFont::from_pdf(&doc, &font, 0.25, &mut DecodeBudget::new(1 << 26, 0)).unwrap().map(|f| f.positioned)
};
let stripped = without_tables(&thai_ordered_program(), &[b"cmap", b"GSUB", b"GPOS"]);
assert_eq!(mode(&stripped, unicode), Some(false), "a legacy positional subset");
let positioned = without_tables(&thai_ordered_program(), &[b"cmap", b"GSUB"]);
assert_eq!(mode(&positioned, unicode), Some(true), "a face with Thai mark positioning uses it");
// ก at another glyph than the legacy order's.
let moved = b"2 beginbfrange <00d7> <010f> <0e02> <0110> <012a> <0e3f> endbfrange 1 beginbfchar <0024> <0e01> endbfchar";
assert_eq!(mode(&stripped, moved), None, "out of the legacy order and without its own positioning");
assert_eq!(mode(&positioned, moved), Some(true), "its own layout doesn't depend on the legacy order");
// Some of the legacy glyphs, no ก or อ among them.
assert_eq!(mode(&stripped, b"1 beginbfrange <00d7> <00e0> <0e02> endbfrange"), Some(false));
assert_eq!(mode(&stripped, b"1 beginbfchar <0024> <0041> endbfchar"), None, "no Thai at all");
}
/// The Windows PUA alternates are used only when each one the subset kept advances as its
/// form does: one mark out of place withdraws them all, so the shaper refuses instead of
/// drawing another mark.
#[test]
fn legacy_thai_alternates_need_their_glyph_order_confirmed() {
let unicode = b"2 beginbfrange <00d6> <010f> <0e01> <0110> <012a> <0e3f> endbfrange 2 beginbfchar <0160> <0e48> <0161> <0e49> endbfchar";
let alternates_with = |program: &[u8], unicode: &[u8]| {
let mut doc = Document::new_empty();
let font = thai_font(&mut doc, program, unicode);
let metrics = Metrics::from_dict(&doc, &font);
let (map, _) = composite_glyph_maps(&doc, &font, program, &metrics, &mut DecodeBudget::new(1 << 20, 0)).unwrap();
map.range(0xf700..=0xf71a).map(|(u, g)| (*u, *g)).collect::<Vec<_>>()
};
let alternates = |program: &[u8]| alternates_with(program, unicode);
let ordered = alternates(&thai_ordered_program());
assert!(ordered.contains(&(0xf70a, 352)) && ordered.contains(&(0xf710, 358)), "{ordered:?}");
let mut misplaced = thai_ordered_program();
set_advance(&mut misplaced, 352, 500);
assert!(alternates(&misplaced).is_empty(), "one misplaced alternate withdraws them all");
// A dropped alternate is simply absent; the others still come from the kept glyphs.
let mut dropped = thai_ordered_program();
drop_glyph(&mut dropped, 353);
let kept = alternates(&dropped);
assert!(kept.contains(&(0xf70a, 352)) && !kept.iter().any(|(_, g)| *g == 353), "{kept:?}");
// Two marks of the same kind (both zero-advance) in another order: the PDF's own map says
// glyph 343 is ี and 344 is ิ, the reverse of U+F701 and U+F702. None is used.
let stated = b"2 beginbfrange <00d6> <010f> <0e01> <0110> <012a> <0e3f> endbfrange 4 beginbfchar <0157> <0e35> <0158> <0e34> <0160> <0e48> <0161> <0e49> endbfchar";
assert!(alternates_with(&thai_ordered_program(), stated).is_empty(), "a stated meaning out of the PUA order withdraws them all");
let agreeing = b"2 beginbfrange <00d6> <010f> <0e01> <0110> <012a> <0e3f> endbfrange 4 beginbfchar <0157> <0e34> <0158> <0e35> <0160> <0e48> <0161> <0e49> endbfchar";
assert!(alternates_with(&thai_ordered_program(), agreeing).contains(&(0xf701, 343)));
}
#[test]
fn damaged_legacy_thai_map_recovers_consonants_vowels_and_contextual_tones_only_for_its_fingerprint() {
let mut doc = Document::new_empty();
let data = b"begincmap 1 begincodespacerange <0000> <ffff> endcodespacerange 18 beginbfrange \
<00d9><00d9><0e04> <00e2><00e2><0e0d> <00fa><00fa><0e25> <0100><0100><0e2b> <0102><0102><0e2d> \
<0164><0164><0e4c> <0166><0166><0e31> <0168><0168><0e47> \
<00d6><00d6><000f> <00f0><00f0><0029> <00f8><00f8><0023> <0105><0105><003e> <0107><0107><0040> \
<0109><0109><003e> <0111><0111><004e> <0160><0160><004e> <0161><0161><004f> <00ff><01ff><0032> endbfrange endcmap";
let mut font = thai_font(&mut doc, &thai_ordered_program(), data);
let tu = font.get(b"ToUnicode").and_then(Object::as_ref).unwrap();
// The corrupt range is declared before explicit anchors in the actual exporter.
let data = String::from_utf8_lossy(data)
.replace("18 beginbfrange", "1 beginbfrange <00ff><01ff><0032> endbfrange 17 beginbfrange")
.replace(" <00ff><01ff><0032> endbfrange endcmap", " endbfrange endcmap");
doc.set(tu, Object::Stream(pdfcraft_cos::Stream::flate(Dict::new(), data.as_bytes())));
let recovered: HashMap<_, _> = recovered_thai_map(&doc, &font).unwrap().into_iter().collect();
let title: String = [240, 248, 261, 214, 263, 253].into_iter().map(|code| recovered[&code]).collect();
assert_eq!(title, "ประกาศ");
assert_eq!(recovered[&265], 'ิ');
assert_eq!(recovered[&273], 'เ');
assert_eq!(recovered[&352], '่');
assert_eq!(recovered[&353], '้');
font.set(b"BaseFont".to_vec(), Object::name("ABCDEF+Unrelated"));
assert!(recovered_thai_map(&doc, &font).is_none());
font.set(b"BaseFont".to_vec(), Object::name("ABCDEF+THSarabunPSK"));
doc.set(tu, Object::Stream(pdfcraft_cos::Stream::flate(Dict::new(), b"1 beginbfchar <00d9><0e04> endbfchar")));
assert!(recovered_thai_map(&doc, &font).is_none(), "one plausible anchor cannot authorize guessing a map");
}
}
crates/fonts/src/shaped_text.rs
+515
View File
@@ -0,0 +1,515 @@
//! Shaped text written into a PDF: each distinct shaped cluster (a letter with its marks, and the
//! text it stands for) becomes one glyph of a Type 3 font drawn from the craft-fonts face's
//! outlines, with a `/ToUnicode` map so copy and search get each character once. No font file is
//! embedded. Add text writes Arabic, Hebrew and Thai with it; text boxes, Fill & Sign text, form
//! fields, headers, footers and watermarks write Thai with it, beside the standard font that
//! draws the rest ([`thai_pieces`]).
use std::collections::{HashMap, HashSet};
use pdfcraft_cos::{Dict, Document, Object, PdfString, Stream};
use crate::{GlyphError, GlyphOutline, ShapedCluster, ShapedScript, literal, shape_thai, shaped_glyph};
/// Most glyphs in one Type 3 font: its codes are single bytes, 1–240. Text gets one such font per
/// 240 distinct clusters it needs.
pub const GLYPHS_PER_FONT: usize = 240;
/// The Thai block (U+0E00–U+0E7F): letters, vowels, tone marks, digits and signs.
pub fn is_thai(c: char) -> bool {
matches!(u32::from(c), 0x0E00..=0x0E7F)
}
/// Thai vowel signs and tone marks drawn on the consonant before them, and Sara Am, which the
/// face splits over it: a line never breaks before one, so a mark stays on its letter.
pub fn joins_previous(c: char) -> bool {
matches!(u32::from(c), 0x0E31 | 0x0E33..=0x0E3A | 0x0E47..=0x0E4E)
}
/// Why Thai text can't be drawn, for a message an agent or user can act on.
pub fn thai_message(e: GlyphError, text: &str) -> String {
match e {
GlyphError::NoFont => "Thai text needs a Thai font, and this build has none; rebuild with CRAFT_FONTS_DIR pointing to a craft-fonts \
checkout with a Thai face"
.into(),
GlyphError::Missing => format!("the Thai font can't show \"{text}\""),
GlyphError::TooComplex => "the Thai glyphs are too complex; use shorter text".into(),
}
}
/// A piece of a line of shaped text, in drawing order.
#[derive(Clone, Debug, PartialEq)]
pub enum Piece {
/// Shown in the text's own standard font.
Latin(String),
Shaped(Vec<ShapedCluster>),
}
/// `line` split into its Thai runs, shaped with the Thai face, and the text between them, left to
/// the standard font. Thai runs left to right, so the pieces are in typing order.
/// [`GlyphError::NoFont`] when there's Thai and this build has no Thai face.
pub fn thai_pieces(line: &str) -> Result<Vec<Piece>, GlyphError> {
let mut runs: Vec<(bool, String)> = Vec::new();
for c in line.chars() {
match runs.last_mut() {
Some((thai, s)) if *thai == is_thai(c) => s.push(c),
_ => runs.push((is_thai(c), c.to_string())),
}
}
runs.into_iter().map(|(thai, s)| if thai { shape_thai(&s).map(Piece::Shaped) } else { Ok(Piece::Latin(s)) }).collect()
}
/// The width of `text` at `size`: its Thai shaped, the rest measured by `latin`. Text the Thai
/// face can't shape is measured by `latin` as a whole (drawing then reports why).
pub fn thai_width(text: &str, size: f64, latin: impl Fn(&str, f64) -> f64) -> f64 {
if !text.chars().any(is_thai) {
return latin(text, size);
}
match thai_pieces(text) {
Ok(pieces) => pieces_width(&pieces, size, 0.0, latin),
Err(_) => latin(text, size),
}
}
/// The width of `pieces` at `size`, with `spacing` after each shaped cluster (as `Tc` moves the
/// pen; the caller measures it into `latin`'s widths).
pub fn pieces_width(pieces: &[Piece], size: f64, spacing: f64, latin: impl Fn(&str, f64) -> f64) -> f64 {
pieces
.iter()
.map(|p| match p {
Piece::Latin(s) => latin(s, size),
// Each cluster is shown on its own, so the spacing follows each once.
Piece::Shaped(clusters) => clusters.iter().map(|c| c.advance * size + spacing).sum::<f64>(),
})
.sum()
}
/// The shaped clusters of `lines`' pieces, for [`ShapedFonts::new`].
pub fn clusters<'a>(lines: impl IntoIterator<Item = &'a [Piece]>) -> impl Iterator<Item = &'a ShapedCluster> {
lines.into_iter().flatten().flat_map(|p| match p {
Piece::Shaped(clusters) => clusters.as_slice(),
Piece::Latin(_) => &[],
})
}
/// What makes two clusters the same Type 3 glyph: text, glyphs, offsets and advance (bit for bit;
/// equal clusters come from the same font units).
type ClusterKey = (ShapedScript, String, Vec<(u32, u64, u64)>, u64);
fn cluster_key(c: &ShapedCluster) -> ClusterKey {
(c.script, c.text.clone(), c.glyphs.iter().map(|(id, [x, y])| (*id, x.to_bits(), y.to_bits())).collect(), c.advance.to_bits())
}
/// The Type 3 fonts that draw a set of shaped clusters, one glyph per distinct cluster. The
/// default has none (for text with nothing shaped).
#[derive(Default)]
pub struct ShapedFonts {
index: HashMap<ClusterKey, usize>,
/// Each font's resource name.
names: Vec<String>,
}
impl ShapedFonts {
/// Build the fonts for `clusters` into `fonts`, under names starting `prefix` that neither
/// `taken` (the page's fonts) nor `fonts` already has. `outline` gives a glyph's outline in its
/// script's face (the craft-fonts faces, or a stand-in in tests). `store` keeps each new
/// object and returns what refers to it: `doc.add` as a reference, or the object itself for
/// an appearance that isn't in a document yet, whose streams [`store_streams`] then moves in.
///
/// Every outline is read before anything is stored, so a failure (and the cluster it came
/// from) leaves the document as it was.
pub fn new<'a>(
clusters: impl IntoIterator<Item = &'a ShapedCluster>,
outline: impl Fn(ShapedScript, u32) -> Result<GlyphOutline, GlyphError>,
taken: &Dict,
fonts: &mut Dict,
prefix: &str,
store: &mut dyn FnMut(Object) -> Object,
) -> Result<Self, (GlyphError, &'a ShapedCluster)> {
let mut keys: Vec<&ShapedCluster> = Vec::new();
let mut index: HashMap<ClusterKey, usize> = HashMap::new();
for c in clusters {
index.entry(cluster_key(c)).or_insert_with(|| {
keys.push(c);
keys.len() - 1
});
}
let mut outlines = HashMap::new();
for c in &keys {
for (id, _) in &c.glyphs {
if let std::collections::hash_map::Entry::Vacant(entry) = outlines.entry((c.script, *id)) {
entry.insert(outline(c.script, *id).map_err(|e| (e, *c))?);
}
}
}
// A name no font on the page has, so items never replace each other's fonts (object
// numbers alone don't do: a full save renumbers objects but keeps resource names).
let mut taken_names: HashSet<Vec<u8>> = taken.iter().chain(fonts.iter()).map(|(name, _)| name.clone()).collect();
let mut names = Vec::new();
for (i, chunk) in keys.chunks(GLYPHS_PER_FONT).enumerate() {
let font = type3_font(chunk, &outlines, store);
let font = store(Object::Dict(font));
let number = font.as_ref().map_or(i + 1, |r| r.num as usize);
let mut name = format!("{prefix}{number}");
let mut suffix = 0usize;
while taken_names.contains(name.as_bytes()) {
suffix = suffix.saturating_add(1);
name = format!("{prefix}{number}_{suffix}");
}
taken_names.insert(name.clone().into_bytes());
fonts.set(name.clone().into_bytes(), font);
names.push(name);
}
Ok(ShapedFonts { index, names })
}
/// The font resource name and code that show `c`, if it was one of the clusters.
pub fn code(&self, c: &ShapedCluster) -> Option<(&str, u8)> {
let &k = self.index.get(&cluster_key(c))?;
let name = self.names.get(k / GLYPHS_PER_FONT)?;
// Codes start at 1; k % 240 + 1 is at most 240.
Some((name.as_str(), u8::try_from(k % GLYPHS_PER_FONT + 1).ok()?))
}
/// Show `clusters` inside `BT … ET` from `pen` on baseline `y` at `size`, each placed with its
/// own `Tm`, `spacing` after each and `word_spacing` more after a space. `current` is the
/// shaped font the text state has selected (`None` after another font); returns the pen
/// after the clusters.
#[allow(clippy::too_many_arguments)]
pub fn show(
&self,
out: &mut Vec<u8>,
clusters: &[ShapedCluster],
mut pen: f64,
y: f64,
size: f64,
spacing: f64,
word_spacing: f64,
current: &mut Option<usize>,
) -> f64 {
for c in clusters {
let Some(&k) = self.index.get(&cluster_key(c)) else { continue };
let Some(name) = self.names.get(k / GLYPHS_PER_FONT) else { continue };
let code = u8::try_from(k % GLYPHS_PER_FONT + 1).unwrap_or(1);
if *current != Some(k / GLYPHS_PER_FONT) {
out.extend(format!("/{name} {} Tf ", n(size)).bytes());
*current = Some(k / GLYPHS_PER_FONT);
}
out.extend(format!("1 0 0 1 {} {} Tm ", n(pen), n(y)).bytes());
out.extend(literal(&[code]));
out.extend_from_slice(b" Tj\n");
pen += c.advance * size + spacing + if c.text == " " { word_spacing } else { 0.0 };
}
pen
}
/// Show a line's `pieces` inside `BT … ET` from `x` on baseline `y` at `size`: shaped clusters
/// in their Type 3 fonts, the rest in the font `latin` (which the text state has selected when
/// `current` is `None`), written by `encode` and `measure`d. `current` is as for [`Self::show`].
#[allow(clippy::too_many_arguments)]
pub fn show_line(
&self,
out: &mut Vec<u8>,
pieces: &[Piece],
x: f64,
y: f64,
size: f64,
latin: &str,
encode: impl Fn(&str) -> Vec<u8>,
measure: impl Fn(&str) -> f64,
current: &mut Option<usize>,
) {
let mut pen = x;
for p in pieces {
match p {
Piece::Latin(s) => {
if current.take().is_some() {
out.extend(format!("/{latin} {} Tf ", n(size)).bytes());
}
out.extend(format!("1 0 0 1 {} {} Tm ", n(pen), n(y)).bytes());
out.extend(literal(&encode(s)));
out.extend_from_slice(b" Tj\n");
pen += measure(s);
}
Piece::Shaped(clusters) => pen = self.show(out, clusters, pen, y, size, 0.0, 0.0, current),
}
}
}
}
/// `lines` as a generated appearance not yet in a document draws them: each line's pieces, its
/// Thai shaped, and the Type 3 fonts for that Thai built into `fonts` (the form's font resources)
/// as direct objects, whose streams [`store_streams`] moves into the document with the
/// appearance. Thai the face can't draw, and every line when a glyph can't be read, is left to the
/// standard font, whose encoder writes `?` for it; [`first_undrawable`] reports that.
pub fn appearance_lines(lines: &[String], fonts: &mut Dict) -> (Vec<Vec<Piece>>, ShapedFonts) {
let plain = |l: &String| vec![Piece::Latin(l.clone())];
let pieces: Vec<Vec<Piece>> =
lines.iter().map(|l| if l.chars().any(is_thai) { thai_pieces(l).unwrap_or_else(|_| plain(l)) } else { plain(l) }).collect();
match ShapedFonts::new(clusters(pieces.iter().map(Vec::as_slice)), shaped_glyph, &Dict::new(), fonts, "PCTh", &mut |o| o).map_err(|(e, _)| e) {
Ok(shaped) => (pieces, shaped),
Err(_) => (lines.iter().map(plain).collect(), ShapedFonts::default()),
}
}
/// How far the Thai glyphs of `line` reach below and above its baseline, in em units; `None`
/// without Thai, or with Thai the face can't draw.
pub fn thai_reach(line: &str) -> Option<[f64; 2]> {
if !line.chars().any(is_thai) {
return None;
}
let pieces = thai_pieces(line).ok()?;
let mut reach: Option<[f64; 2]> = None;
for c in clusters([pieces.as_slice()]) {
for (id, [_, dy]) in &c.glyphs {
let g = shaped_glyph(c.script, *id).ok()?;
if !g.contours.is_empty() {
let b = [g.bbox[1] + dy, g.bbox[3] + dy];
reach = Some(reach.map_or(b, |r| [r[0].min(b[0]), r[1].max(b[1])]));
}
}
}
reach
}
/// The first character of `s` a generated appearance would draw as `?`: one the standard fonts
/// (WinAnsiEncoding) can't draw and that isn't Thai, or Thai on a line the Thai face can't draw
/// (this build has none, or it lacks a character). Line breaks don't count.
pub fn first_undrawable(s: &str) -> Option<char> {
s.split(['\n', '\r']).find_map(first_undrawable_in_line)
}
/// [`first_undrawable`] for one line drawn as it is: a line break in it counts.
pub fn first_undrawable_in_line(line: &str) -> Option<char> {
let thai_fails = line.chars().any(is_thai) && thai_pieces(line).is_err();
line.chars().find(|c| if is_thai(*c) { thai_fails } else { !crate::win_ansi_encodable(*c) })
}
/// Why the Thai in `text` can't be drawn ([`thai_message`]): this build has no Thai face, or the
/// face lacks a character. For tools that refuse such text rather than write `?`; `None` when all
/// of its Thai draws, or it has none.
pub fn thai_refusal(text: &str) -> Option<String> {
text.split(['\n', '\r']).filter(|l| l.chars().any(is_thai)).find_map(|l| thai_pieces(l).err().map(|e| thai_message(e, l)))
}
/// [`type3_font`] written into `doc`, with its glyphs, descriptor and ToUnicode as objects of their
/// own: for content that draws `clusters` (at most [`GLYPHS_PER_FONT`]) as codes 1… directly.
pub fn add_type3_font(
doc: &mut Document,
clusters: &[&ShapedCluster],
outlines: &HashMap<(ShapedScript, u32), GlyphOutline>,
) -> pdfcraft_cos::ObjRef {
let font = type3_font(clusters, outlines, &mut |o| Object::Ref(doc.add(o)));
doc.add(Object::Dict(font))
}
/// A Type 3 font with `clusters` as codes 1…, each drawn from its script face's glyph `outlines`.
fn type3_font(clusters: &[&ShapedCluster], outlines: &HashMap<(ShapedScript, u32), GlyphOutline>, store: &mut dyn FnMut(Object) -> Object) -> Dict {
let mut charprocs = Dict::new();
let mut differences = vec![Object::Int(1)];
let mut widths = Vec::with_capacity(clusters.len());
let mut bbox = [0.0f64; 4];
let mut to_unicode = Vec::new();
for (i, c) in clusters.iter().enumerate() {
let code = i + 1;
// Glyph space is 1000 units per em.
let mut paths = Vec::new();
let mut ink: Option<[f64; 4]> = None;
for (id, [dx, dy]) in &c.glyphs {
let Some(g) = outlines.get(&(c.script, *id)) else { continue };
for contour in &g.contours {
for (j, p) in contour.iter().enumerate() {
paths.extend(format!("{} {} {}\n", n((p[0] + dx) * 1000.0), n((p[1] + dy) * 1000.0), if j == 0 { "m" } else { "l" }).bytes());
}
paths.extend_from_slice(b"h\n");
let b = [g.bbox[0] + dx, g.bbox[1] + dy, g.bbox[2] + dx, g.bbox[3] + dy];
ink = Some(ink.map_or(b, |a| [a[0].min(b[0]), a[1].min(b[1]), a[2].max(b[2]), a[3].max(b[3])]));
}
}
// Rounded outwards so no ink is clipped.
let [x0, y0, x1, y1] =
ink.map_or([0.0; 4], |b| [(b[0] * 1000.0).floor(), (b[1] * 1000.0).floor(), (b[2] * 1000.0).ceil(), (b[3] * 1000.0).ceil()]);
bbox = [bbox[0].min(x0), bbox[1].min(y0), bbox[2].max(x1), bbox[3].max(y1)];
let mut proc = format!("{} 0 {} {} {} {} d1\n", n(c.advance * 1000.0), n(x0), n(y0), n(x1), n(y1)).into_bytes();
if !paths.is_empty() {
proc.extend(paths);
proc.extend_from_slice(b"f\n");
}
let glyph = format!("g{code:02X}");
charprocs.set(glyph.clone().into_bytes(), store(Object::Stream(Stream::flate(Dict::new(), &proc))));
differences.push(Object::name(&glyph));
widths.push(Object::Real((c.advance * 1000.0).round()));
if !c.text.is_empty() {
let hex: String = c.text.encode_utf16().map(|u| format!("{u:04X}")).collect();
to_unicode.push(format!("<{code:02X}> <{hex}>\n"));
}
}
let mut cmap = String::from(
"/CIDInit /ProcSet findresource begin\n12 dict begin\nbegincmap\n/CMapType 2 def\n1 begincodespacerange\n<01> <FF>\nendcodespacerange\n",
);
// A CMap block holds at most 100 entries.
for block in to_unicode.chunks(100) {
cmap.push_str(&format!("{} beginbfchar\n{}endbfchar\n", block.len(), block.concat()));
}
cmap.push_str("endcmap\nCMapName currentdict /CMap defineresource pop\nend\nend\n");
let cmap = store(Object::Stream(Stream::flate(Dict::new(), cmap.as_bytes())));
// The face's family when every glyph comes from one script; without the face, the script's
// name (as Arabic items were always named).
let family = match clusters.first().map(|c| c.script).filter(|s| clusters.iter().all(|c| c.script == *s)) {
Some(script) => script.font().map_or(
match script {
ShapedScript::Arabic => "Arabic",
ShapedScript::Hebrew => "Hebrew",
ShapedScript::Thai => "Thai",
},
|f| f.family,
),
None => "PdfCraft Shaped Text",
};
let mut descriptor = Dict::new();
descriptor.set(b"Type".to_vec(), Object::name("FontDescriptor"));
descriptor.set(b"FontName".to_vec(), Object::name(&family.replace(' ', "")));
descriptor.set(b"FontFamily".to_vec(), Object::String(PdfString::literal(family.as_bytes().to_vec())));
descriptor.set(b"Flags".to_vec(), Object::Int(4));
descriptor.set(b"ItalicAngle".to_vec(), Object::Int(0));
let mut encoding = Dict::new();
encoding.set(b"Type".to_vec(), Object::name("Encoding"));
encoding.set(b"Differences".to_vec(), Object::Array(differences));
let mut font = Dict::new();
font.set(b"Type".to_vec(), Object::name("Font"));
font.set(b"Subtype".to_vec(), Object::name("Type3"));
font.set(b"FontDescriptor".to_vec(), store(Object::Dict(descriptor)));
font.set(b"FontBBox".to_vec(), Object::Array(bbox.iter().map(|v| Object::Real(*v)).collect()));
font.set(
b"FontMatrix".to_vec(),
Object::Array(vec![Object::Real(0.001), Object::Int(0), Object::Int(0), Object::Real(0.001), Object::Int(0), Object::Int(0)]),
);
font.set(b"FirstChar".to_vec(), Object::Int(1));
font.set(b"LastChar".to_vec(), Object::Int(clusters.len() as i64));
font.set(b"Widths".to_vec(), Object::Array(widths));
font.set(b"Encoding".to_vec(), Object::Dict(encoding));
font.set(b"CharProcs".to_vec(), Object::Dict(charprocs));
font.set(b"ToUnicode".to_vec(), cmap);
font
}
/// Most levels of dictionaries and arrays [`store_streams`] looks into: a Type 3 font in a form's
/// resources is four deep (`/Font` → font → `/CharProcs` → glyph).
const STREAM_DEPTH: usize = 8;
/// Move the streams written into `d` (the fonts [`ShapedFonts::new`] built for an appearance not
/// yet in a document) into `doc` as objects of their own, leaving references: a stream is never a
/// direct object (ISO 32000-2 §7.3.8).
pub fn store_streams(doc: &mut Document, d: &mut Dict) {
fn walk(doc: &mut Document, o: &mut Object, depth: usize) {
if depth > STREAM_DEPTH {
return;
}
match o {
Object::Stream(_) => {
let Object::Stream(mut s) = std::mem::replace(o, Object::Null) else { return };
walk_dict(doc, &mut s.dict, depth + 1);
*o = Object::Ref(doc.add(Object::Stream(s)));
}
Object::Dict(d) => walk_dict(doc, d, depth + 1),
Object::Array(a) => a.iter_mut().for_each(|o| walk(doc, o, depth + 1)),
_ => {}
}
}
fn walk_dict(doc: &mut Document, d: &mut Dict, depth: usize) {
let keys: Vec<Vec<u8>> =
d.iter().filter(|(_, v)| matches!(v, Object::Stream(_) | Object::Dict(_) | Object::Array(_))).map(|(k, _)| k.clone()).collect();
for k in keys {
if let Some(v) = d.get_mut(&k) {
walk(doc, v, depth);
}
}
}
walk_dict(doc, d, 0);
}
/// Format a number for content streams (up to three decimals).
fn n(v: f64) -> String {
let s = format!("{:.3}", if v.abs() < 5e-4 { 0.0 } else { v });
let s = s.trim_end_matches('0').trim_end_matches('.');
if s == "-0" { "0".into() } else { s.into() }
}
#[cfg(test)]
mod tests {
use super::*;
fn boxed(width: f64) -> GlyphOutline {
GlyphOutline { contours: vec![vec![[0.0, -0.4], [width, -0.4], [width, 1.1], [0.0, 1.1]]], width, bbox: [0.0, -0.4, width, 1.1] }
}
fn cluster(text: &str, id: u32) -> ShapedCluster {
ShapedCluster { script: ShapedScript::Thai, glyphs: vec![(id, [0.0, 0.0])], advance: 0.5, text: text.into() }
}
#[test]
fn appearance_fonts_keep_their_streams_until_stored_then_refer_to_them() {
let clusters = [cluster("น้ำ", 1), cluster("ก", 2), cluster("น้ำ", 1)];
let mut fonts = Dict::new();
let made = ShapedFonts::new(&clusters, |_, _| Ok(boxed(0.5)), &Dict::new(), &mut fonts, "PCTh", &mut |o| o).unwrap();
assert_eq!(made.code(&clusters[0]), Some(("PCTh1", 1)));
assert_eq!(made.code(&clusters[1]), Some(("PCTh1", 2)), "a repeated cluster is one glyph");
let mut line = Vec::new();
let pieces = [Piece::Latin("a".into()), Piece::Shaped(clusters[..1].to_vec()), Piece::Latin("b".into())];
made.show_line(&mut line, &pieces, 0.0, 0.0, 10.0, "Helv", |s| s.as_bytes().to_vec(), |_| 1.0, &mut None);
let shown = String::from_utf8_lossy(&line).into_owned();
assert_eq!(shown, "1 0 0 1 0 0 Tm (a) Tj\n/PCTh1 10 Tf 1 0 0 1 1 0 Tm (\u{1}) Tj\n/Helv 10 Tf 1 0 0 1 6 0 Tm (b) Tj\n");
let mut out = Vec::new();
let mut current = None;
let pen = made.show(&mut out, &clusters, 10.0, 20.0, 10.0, 0.0, 0.0, &mut current);
assert_eq!(pen, 25.0);
assert_eq!(String::from_utf8_lossy(&out).matches(" Tf").count(), 1, "one font selection for one font");
let mut doc = Document::new_empty();
let mut res = Dict::new();
res.set(b"Font".to_vec(), Object::Dict(fonts));
store_streams(&mut doc, &mut res);
let font = res.get(b"Font").and_then(Object::as_dict).and_then(|f| f.get(b"PCTh1")).and_then(Object::as_dict).unwrap();
assert!(matches!(font.get(b"ToUnicode"), Some(Object::Ref(_))));
let procs = font.get(b"CharProcs").and_then(Object::as_dict).unwrap();
assert!(procs.iter().count() == 2 && procs.iter().all(|(_, v)| matches!(v, Object::Ref(_))));
}
#[test]
fn a_missing_glyph_names_its_cluster_and_writes_nothing() {
let clusters = [cluster("ก", 2)];
let mut fonts = Dict::new();
let mut stored = 0;
let made = ShapedFonts::new(&clusters, |_, _| Err(GlyphError::Missing), &Dict::new(), &mut fonts, "PCTh", &mut |o| {
stored += 1;
o
});
assert!(matches!(made, Err((GlyphError::Missing, c)) if c.text == "ก"));
assert_eq!((stored, fonts.iter().count()), (0, 0));
}
#[test]
fn thai_runs_are_shaped_and_the_rest_left_to_the_standard_font() {
assert_eq!(thai_pieces("Hello").unwrap(), [Piece::Latin("Hello".into())]);
assert!(joins_previous('\u{0E49}') && joins_previous('\u{0E33}') && !joins_previous('ก') && !joins_previous('a'));
assert_eq!((first_undrawable("Café\nпривет"), thai_refusal("Café"), thai_reach("Hg")), (Some('п'), None, None));
if crate::document_thai_font().is_none() {
assert_eq!(thai_pieces("น้ำ 1"), Err(GlyphError::NoFont));
assert!(thai_message(GlyphError::NoFont, "").contains("this build has none"));
assert_eq!(thai_width("น้ำ", 10.0, |s, size| s.chars().count() as f64 * size), 30.0, "measured by the standard font");
assert_eq!(first_undrawable("ok\nชื่อ น้ำ"), Some('ช'));
assert!(thai_refusal("ok\nน้ำ").is_some_and(|m| m.contains("CRAFT_FONTS_DIR")));
return;
}
assert_eq!((first_undrawable("ชื่อ: น้ำ\nOK"), thai_refusal("ชื่อ: น้ำ")), (None, None));
assert_eq!(first_undrawable("น้ำ ☃"), Some('☃'));
// Stacked marks reach above the em box's 0.9, a descender below its 0.3.
let [_, top] = thai_reach("น้ำ").unwrap();
let [bottom, _] = thai_reach("ฎู").unwrap();
assert!(top > 0.95 && bottom < -0.35, "{top} {bottom}");
let pieces = thai_pieces("ชื่อ: บุญฤทธิ์ 12").unwrap();
assert!(matches!(&pieces[..], [Piece::Shaped(_), Piece::Latin(a), Piece::Shaped(_), Piece::Latin(b)] if a == ": " && b == " 12"));
let text: String = clusters([pieces.as_slice()]).map(|c| c.text.as_str()).collect();
assert_eq!(text, "ชื่อบุญฤทธิ์");
assert!(thai_width("น้ำ", 10.0, |_, _| 0.0) > 0.0);
}
}
crates/fonts/src/source_shaping.rs
+178 -2
View File
@@ -22,7 +22,45 @@ pub fn required(text: &str) -> bool {
.any(|c| matches!(u32::from(c), 0x0300..=0x036f | 0x0590..=0x0fff | 0x1780..=0x17ff | 0x200c..=0x200f | 0x202a..=0x202e | 0x2066..=0x2069))
}
/// Whether a Thai source program retains its mark positioning.
/// Character coverage alone does not show that stacked vowels and tones can be laid out.
/// OpenType Thai requires `mark`/`mkmk` (GPOS); `ccmp` substitution (GSUB) is optional.
pub fn has_thai_layout(bytes: &[u8]) -> bool {
FontRef::new(bytes).is_ok_and(|f| positions_thai_marks(&f))
}
/// Whether the GPOS features the shaper uses for Thai include `mark` or `mkmk`. The selection is
/// HarfRust's, through the same read-fonts calls: the `thai` script, else `DFLT`, `dflt` or
/// `latn` (`ScriptList::select`); then, given no language, a language system tagged `dflt`,
/// else the script's default one. Marks under a script or language it doesn't select don't
/// count, nor does a GPOS with only kerning.
fn positions_thai_marks(face: &FontRef) -> bool {
use skrifa::Tag;
let Ok(gpos) = face.gpos() else { return false };
let (Ok(scripts), Ok(features)) = (gpos.script_list(), gpos.feature_list()) else { return false };
let records = features.feature_records();
let marks = |index: u16| records.get(usize::from(index)).is_some_and(|r| r.feature_tag() == *b"mark" || r.feature_tag() == *b"mkmk");
let Some(selected) = scripts.select(&[Tag::new(b"thai")]) else { return false };
let Ok(script) = scripts.get(selected.index) else { return false };
let language = match script.lang_sys_index_for_tag(Tag::new(b"dflt")) {
Some(index) => script.lang_sys(index).ok().map(|l| l.element),
None => script.default_lang_sys().transpose().ok().flatten(),
};
let Some(language) = language else { return false };
language.feature_indices().iter().any(|i| marks(i.get()))
|| (language.required_feature_index() != 0xFFFF && marks(language.required_feature_index()))
}
pub fn shape(bytes: &[u8], text: &str) -> Result<ShapedRun, String> {
shape_inner(bytes, text, false)
}
/// Only used after the PDF adapter has verified legacy Thai PUA glyph mappings.
pub(crate) fn shape_legacy_thai(bytes: &[u8], text: &str) -> Result<ShapedRun, String> {
shape_inner(bytes, text, true)
}
fn shape_inner(bytes: &[u8], text: &str, legacy_thai: bool) -> Result<ShapedRun, String> {
if text.len() > 262_144 || text.contains(['\n', '\r']) {
return Err("shape one bounded text row at a time".into());
}
@@ -30,7 +68,14 @@ pub fn shape(bytes: &[u8], text: &str) -> Result<ShapedRun, String> {
return Ok(ShapedRun { glyphs: Vec::new(), advance: 0.0 });
}
let face = FontRef::new(bytes).map_err(|e| e.to_string())?;
if text.chars().any(|c| matches!(u32::from(c), 0x0600..=0x06ff | 0x0750..=0x077f | 0x08a0..=0x0fff | 0x1780..=0x17ff)) && face.gsub().is_err() {
// Only a vowel or tone mark set on its base (or Sara Am, whose Nikhahit is) needs positioning:
// digits and bare letters shape the same without it.
if !legacy_thai && text.chars().any(|c| matches!(u32::from(c), 0x0e31 | 0x0e33..=0x0e3a | 0x0e47..=0x0e4e)) && !positions_thai_marks(&face) {
return Err("the source subset has no Thai mark positioning (GPOS mark/mkmk); select its full font".into());
}
if text.chars().any(|c| matches!(u32::from(c), 0x0600..=0x06ff | 0x0750..=0x077f | 0x08a0..=0x0dff | 0x0e80..=0x0fff | 0x1780..=0x17ff))
&& face.gsub().is_err()
{
return Err("the source subset has no contextual substitution tables; select its full font".into());
}
let data = ShaperData::new(&face);
@@ -56,7 +101,11 @@ pub fn shape(bytes: &[u8], text: &str) -> Result<ShapedRun, String> {
let mut seen = std::collections::HashSet::new();
for (g, p) in output.glyph_infos().iter().zip(output.glyph_positions()) {
if g.glyph_id == 0 {
return Err("the full font lacks a required contextual glyph; choose the complete source face".into());
let missing = input.get(g.cluster as usize..).and_then(|s| s.chars().next()).unwrap_or('\u{fffd}');
return Err(format!(
"the full font lacks a required contextual glyph for {missing} (U+{:04X}); choose the complete source face",
u32::from(missing)
));
}
let start = g.cluster as usize;
let end = starts.get(starts.partition_point(|i| *i <= start)).copied().unwrap_or(input.len());
@@ -74,3 +123,130 @@ pub fn shape(bytes: &[u8], text: &str) -> Result<ShapedRun, String> {
}
Ok(ShapedRun { glyphs, advance: x })
}
#[cfg(test)]
mod tests {
use super::*;
/// Where the GPOS table starts in an sfnt.
fn gpos(bytes: &[u8]) -> usize {
let count = u16::from_be_bytes([bytes[4], bytes[5]]) as usize;
let record = (0..count).map(|i| 12 + i * 16).find(|&r| &bytes[r..r + 4] == b"GPOS").unwrap();
u32::from_be_bytes(bytes[record + 8..record + 12].try_into().unwrap()) as usize
}
fn u16_at(bytes: &[u8], at: usize) -> usize {
u16::from_be_bytes([bytes[at], bytes[at + 1]]) as usize
}
/// Where the GPOS ScriptList starts.
fn script_list(bytes: &[u8]) -> usize {
gpos(bytes) + u16_at(bytes, gpos(bytes) + 4)
}
/// Where script `tag`'s table starts.
fn script(bytes: &[u8], tag: &[u8; 4]) -> usize {
let list = script_list(bytes);
let record = (0..u16_at(bytes, list)).map(|i| list + 2 + i * 6).find(|&r| &bytes[r..r + 4] == tag).unwrap();
list + u16_at(bytes, record + 4)
}
/// Retag script `from` as `to`, keeping the records sorted by tag as the shaper's binary
/// search needs.
fn retag_script(bytes: &mut [u8], from: &[u8; 4], to: &[u8; 4]) {
let list = script_list(bytes);
let count = u16_at(bytes, list);
let mut records: Vec<[u8; 6]> = (0..count).map(|i| bytes[list + 2 + i * 6..list + 8 + i * 6].try_into().unwrap()).collect();
records.iter_mut().find(|r| &r[..4] == from).unwrap()[..4].copy_from_slice(to);
records.sort_by(|a, b| a[..4].cmp(&b[..4]));
for (i, record) in records.iter().enumerate() {
bytes[list + 2 + i * 6..list + 8 + i * 6].copy_from_slice(record);
}
}
/// A shaper uses a font's `thai` script when it has one: marks the font registers only for
/// Latin don't position Thai.
#[test]
fn thai_layout_counts_only_the_script_the_shaper_selects() {
let mut bytes = crate::script::INTER_REGULAR.to_vec();
// Inter's DFLT becomes a `thai` script whose default language system has no features;
// its latn script keeps `mark` and `mkmk`.
retag_script(&mut bytes, b"DFLT", b"thai");
let thai = script(&bytes, b"thai");
let language = thai + u16_at(&bytes, thai);
assert_ne!(language, thai, "a default language system");
bytes[language + 2..language + 6].copy_from_slice(&[0xFF, 0xFF, 0, 0]);
assert!(!has_thai_layout(&bytes), "the latn script's marks are not the thai script's");
}
/// Given no language, the shaper takes a language system tagged `dflt` before the script's
/// default one, in both directions.
#[test]
fn thai_layout_follows_a_language_system_tagged_dflt() {
// Without DFLT the shaper falls back to latn, whose default language system has marks and
// is shared by its CAT, MOL and ROM records. Cyrillic's default one has only `kern`.
let mut base = crate::script::INTER_REGULAR.to_vec();
retag_script(&mut base, b"DFLT", b"zzzz");
assert!(has_thai_layout(&base));
let latn = script(&base, b"latn");
let kern_only = script(&base, b"cyrl") + u16_at(&base, script(&base, b"cyrl"));
let kern_only = u16::try_from(kern_only - latn).unwrap().to_be_bytes();
let rom = (0..u16_at(&base, latn + 2)).map(|j| latn + 4 + j * 6).find(|&r| &base[r..r + 4] == b"ROM ").unwrap();
// ROM becomes `dflt` (still sorted last) and has only kerning: it wins over the default.
let mut named_without = base.clone();
named_without[rom..rom + 4].copy_from_slice(b"dflt");
named_without[rom + 4..rom + 6].copy_from_slice(&kern_only);
assert!(!has_thai_layout(&named_without), "the dflt language system has no marks");
// The default has only kerning and the `dflt` one keeps the marks: it wins again.
let mut default_without = base.clone();
default_without[rom..rom + 4].copy_from_slice(b"dflt");
default_without[latn..latn + 2].copy_from_slice(&kern_only);
assert!(has_thai_layout(&default_without), "the dflt language system has the marks");
}
/// A GPOS table alone is not mark positioning: with its `mark` and `mkmk` features renamed,
/// as in a kerning-only subset, a face can't stack Thai vowels and tones.
#[test]
fn thai_layout_needs_mark_positioning_features() {
let inter = crate::script::INTER_REGULAR;
assert!(has_thai_layout(inter), "Inter positions marks under its default script");
let mut bytes = inter.to_vec();
let table = gpos(&bytes);
let features = table + u16_at(&bytes, table + 6);
let mut renamed = 0;
for i in 0..u16_at(&bytes, features) {
let tag = features + 2 + i * 6;
if matches!(&bytes[tag..tag + 4], b"mark" | b"mkmk") {
bytes[tag..tag + 4].copy_from_slice(b"kern");
renamed += 1;
}
}
assert!(renamed > 0);
assert!(FontRef::new(&bytes).unwrap().gpos().is_ok(), "the GPOS table still parses");
assert!(!has_thai_layout(&bytes));
let error = shape(&bytes, "กิ่").unwrap_err();
assert!(error.contains("mark positioning"), "{error}");
}
#[test]
fn thai_positioning_does_not_require_optional_gsub() {
let Some(face) = crate::document_thai_font() else {
eprintln!("skipping thai_positioning_does_not_require_optional_gsub: built without craft-fonts (set CRAFT_FONTS_DIR to run it)");
return;
};
let mut bytes = face.bytes.to_vec();
// Keep the actual GPOS anchors and outlines while removing optional substitutions.
// Table payloads stay at their original offsets; no font fixture is committed.
let count = u16::from_be_bytes([bytes[4], bytes[5]]) as usize;
let tables: Vec<_> = bytes[12..12 + count * 16].as_chunks::<16>().0.iter().filter(|t| &t[..4] != b"GSUB").map(|t| t.to_vec()).collect();
bytes[4..6].copy_from_slice(&(tables.len() as u16).to_be_bytes());
for (i, table) in tables.iter().enumerate() {
bytes[12 + i * 16..28 + i * 16].copy_from_slice(table);
}
assert!(FontRef::new(&bytes).unwrap().gsub().is_err());
assert!(has_thai_layout(&bytes));
let shaped = shape(&bytes, "กิ่ น้ำ").unwrap();
assert!(shaped.advance > 0.0);
assert!(shaped.glyphs.iter().all(|g| g.position.iter().all(|p| p.is_finite())));
}
}
crates/forms/README.md
+4 -1
View File
@@ -45,7 +45,10 @@ let cells: Vec<table::Cell> = grid.cells_for(names)?; // "Table1_1", "Table
(cells parted in the border colour), multiline wrapping, password masking, quadding, `/MK`
background and border, list-box selection.
The `/DA` font is used when `/DR` has it as a simple font or a composite font with a predefined
Unicode CMap; otherwise Helvetica (WinAnsi). A value or button caption the font can't hold
Unicode CMap; otherwise Helvetica (WinAnsi). Thai in a text field or combo box that the font
can't hold is drawn from the craft-fonts Thai face with Add text's writer
(`pdfcraft_fonts::shaped_text`), beside the field's font, keeping `/DA`; a build without a Thai
face refuses it. Any other value or button caption the font can't hold
(Chinese in a Helvetica field, a character beyond U+FFFF in a UCS-2 CMap) is kept, as Acrobat
keeps it: `/V` is stored as typed, the appearance draws `?` for those characters, the widget is
marked with the first one (`pdfcraft_fonts::UNDRAWN_MARK`, cleared once all of it can be drawn)
crates/forms/src/appearance.rs
+86 -27
View File
@@ -23,6 +23,7 @@
use pdfcraft_cos::{Dict, Document, Object, Stream};
use pdfcraft_fonts::pdf::Metrics;
use pdfcraft_fonts::shaped_text::{self, Piece, ShapedFonts};
use pdfcraft_fonts::{UnicodeCMap, cjk_width, helvetica_width, literal, win_ansi, wrap_with};
use crate::{Field, FieldKind, Widget, acroform, flags};
@@ -747,6 +748,21 @@ fn draws(doc: &Document, f: &Field, wd: &Dict, text: &str) -> bool {
enc.first_unencodable(text).is_none()
}
/// Why text field `f` can't take `text`: it has Thai that a widget's font can't hold and this
/// build can't draw (it has no Thai face, or the face lacks a character), which would come out
/// as `?`. `None` for text it can draw, and for a password field, which shows `*`.
pub(crate) fn thai_refusal(doc: &Document, f: &Field, text: &str) -> Option<String> {
if f.has(flags::PASSWORD) || !text.chars().any(shaped_text::is_thai) {
return None;
}
let held = |w: &Widget| {
let wd = doc.get(w.obj).as_dict().cloned().unwrap_or_default();
let da = parse_da(wd.get(b"DA").and_then(|o| doc.resolve(o).as_string().map(|s| s.to_text())).as_deref().unwrap_or(&f.da));
dr_font(doc, &da.font).is_some_and(|(_, enc)| matches!(enc, FontText::Simple(_)) && text.split(['\n', '\r']).all(|l| enc.can_encode(l)))
};
if f.widgets.iter().all(held) { None } else { shaped_text::thai_refusal(text) }
}
/// [`field_appearance_drawn`], and whether the text drawn is cut off by the box.
fn field_appearance_fit(doc: &Document, f: &Field, w: &Widget, values: &[String], format: bool) -> (Stream, Option<char>, Option<Overflow>) {
let wobj = doc.get(w.obj);
@@ -778,36 +794,43 @@ fn field_appearance_fit(doc: &Document, f: &Field, w: &Widget, values: &[String]
values.first().cloned().unwrap_or_default()
};
let text = if f.has(flags::PASSWORD) { "*".repeat(text.chars().count()) } else { text };
let can_show = |enc: &FontText| {
let can_show = |enc: &FontText, text: &str| {
if f.kind == FieldKind::List {
f.options.iter().all(|(_, display)| enc.can_encode(display))
} else if f.kind == FieldKind::Text && f.has(flags::MULTILINE) {
text.split(['\n', '\r']).all(|line| enc.can_encode(line))
} else {
enc.can_encode(&text)
enc.can_encode(text)
}
};
let (font_name, font_obj, enc) = match dr_font(doc, &da.font).filter(|(_, enc)| can_show(enc)) {
let dr = dr_font(doc, &da.font);
// Thai the form's font can't hold (it isn't a simple font encoding it) is drawn from the Thai
// face, as Add text draws it: Type 3 glyphs beside the form's font, which draws the rest. One
// line of a text field or combo box at a time; the `/DA` is kept.
let thai = matches!(f.kind, FieldKind::Text | FieldKind::Combo)
&& !f.is_comb()
&& text.chars().any(shaped_text::is_thai)
&& !dr.as_ref().is_some_and(|(_, enc)| matches!(enc, FontText::Simple(_)) && can_show(enc, &text))
&& shaped_text::thai_refusal(&text).is_none();
let latin_text: String = if thai { text.chars().filter(|c| !shaped_text::is_thai(*c)).collect() } else { text.clone() };
let (font_name, font_obj, enc) = match dr.filter(|(_, enc)| can_show(enc, &latin_text)) {
Some((o, enc)) => (da.font.clone(), o, enc),
None => ("Helv".to_string(), helvetica(), FontText::WinAnsi),
};
let width_of = |text: &str, size: f64| enc.width(text, size);
let width_of =
|text: &str, size: f64| if thai { shaped_text::thai_width(text, size, |s, size| enc.width(s, size)) } else { enc.width(text, size) };
// How far a line's ink reaches below and above its baseline (em), where Thai stacks or
// descenders reach past the form font's (Helvetica's) -0.207 and 0.718; `None` otherwise.
let thai_ink = |line: &str| thai.then(|| shaped_text::thai_reach(line)).flatten().map(|[lo, hi]| [lo.min(-0.207), hi.max(0.718)]);
let (mut c, bw) = frame(doc, &wd, width, height);
let pad = 2.0 + bw;
let inner_w = (width - 2.0 * pad).max(1.0);
let q = wd.get(b"Q").and_then(|o| doc.resolve(o).as_int()).unwrap_or(f.quadding);
let mut body: Vec<u8> = Vec::new();
// The first character drawn that the font can't hold (drawn as `?`), for the caller to report.
let mut unencodable: Option<char> = None;
let mut show = |body: &mut Vec<u8>, x: f64, y: f64, text: &str| {
// The lines to draw (where, and what), drawn once all are known: their Thai shares fonts.
let mut shown: Vec<(f64, f64, String)> = Vec::new();
let mut show = |x: f64, y: f64, text: &str| {
// A line break in a single-line value is a space, as in viewers (multiline text is split).
let text: String = text.chars().map(|c| if matches!(c, '\n' | '\r') { ' ' } else { c }).collect();
if unencodable.is_none() {
unencodable = enc.first_unencodable(&text);
}
body.extend(format!("1 0 0 1 {} {} Tm ", n(x), n(y)).bytes());
body.extend(literal(&enc.encode(&text)));
body.extend_from_slice(b" Tj\n");
shown.push((x, y, text.chars().map(|c| if matches!(c, '\n' | '\r') { ' ' } else { c }).collect()));
};
let x_for = |text: &str, size: f64| -> f64 {
let tw = width_of(text, size);
@@ -834,7 +857,7 @@ fn field_appearance_fit(doc: &Document, f: &Field, w: &Widget, values: &[String]
if values.contains(export) {
c.push_str(&format!("0.6 0.75 0.86 rg\n{} {} {} {} re f\n", n(bw), n(y - line), n(width - 2.0 * bw), n(line)));
}
show(&mut body, pad, y - line + size * 0.25, display);
show(pad, y - line + size * 0.25, display);
y -= line;
}
}
@@ -844,18 +867,20 @@ fn field_appearance_fit(doc: &Document, f: &Field, w: &Widget, values: &[String]
if size == 0.0 {
size = auto_text_size(&text, inner_w, height - 2.0 * pad, true, width_of);
}
let mut y = height - pad - size * 0.85;
let lines = wrap_with(&text, size, inner_w, width_of);
// The first line hangs from the top by its ascent: higher for stacked Thai marks.
let mut y = height - pad - size * lines.first().and_then(|l| thai_ink(l)).map_or(0.85, |[_, hi]| hi.max(0.85));
// The lines whose descent stays inside the border; the rest are cut off.
let mut room = 0;
for line in &lines {
if y < -size {
break;
}
if y - 0.207 * size >= bw - 0.01 {
let descent = thai_ink(line).map_or(0.207, |[lo, _]| -lo);
if y - descent * size >= bw - 0.01 {
room += 1;
}
show(&mut body, x_for(line, size), y, line);
show(x_for(line, size), y, line);
y -= size * 1.15;
}
if room < lines.len() {
@@ -872,12 +897,22 @@ fn field_appearance_fit(doc: &Document, f: &Field, w: &Widget, values: &[String]
auto_text_size(&text, inner_w, height - 2.0 * pad, false, width_of)
};
}
// Centre Helvetica's ascent (0.718) and descent (0.207) vertically.
let y = (height - 0.925 * size) / 2.0 + 0.207 * size;
// Centre Helvetica's ascent (0.718) and descent (0.207) vertically, or the reach of
// Thai stacks and descenders, which an automatic size also keeps inside the padding.
let reach = thai_ink(&text);
if let Some([lo, hi]) = reach
&& da.size == 0.0
{
size = size.min((height - 2.0 * pad) / (hi - lo)).max(4.0);
}
let (y, ink) = match reach {
Some([lo, hi]) => ((height - (hi - lo) * size) / 2.0 - lo * size, (hi - lo) * size),
None => ((height - 0.925 * size) / 2.0 + 0.207 * size, 0.925 * size),
};
// The clip box is inside the border; a line taller than it is cut at the top and bottom.
let room_h = height - 2.0 * bw;
if 0.925 * size > room_h + 0.01 {
overflow = Some(Overflow::High { height: 0.925 * size, room: room_h, size });
if ink > room_h + 0.01 {
overflow = Some(Overflow::High { height: ink, room: room_h, size });
}
if comb {
let cells = f.max_len.unwrap_or(1).max(1);
@@ -885,18 +920,19 @@ fn field_appearance_fit(doc: &Document, f: &Field, w: &Widget, values: &[String]
c.push_str(&comb_dividers(doc, &wd, width, height, cells));
for (i, ch) in text.chars().take(cells).enumerate() {
let s = ch.to_string();
show(&mut body, cell * i as f64 + (cell - width_of(&s, size)) / 2.0, y, &s);
show(cell * i as f64 + (cell - width_of(&s, size)) / 2.0, y, &s);
}
} else {
let tw = width_of(&text.replace(['\n', '\r'], " "), size);
if overflow.is_none() && tw > inner_w + 0.01 {
overflow = Some(Overflow::Wide { width: tw, room: inner_w, size });
}
show(&mut body, x_for(&text, size), y, &text);
show(x_for(&text, size), y, &text);
}
}
}
}
let (body, fonts, unencodable) = draw_lines(&shown, size, &font_name, font_obj, &enc, thai);
// Acrobat regenerates only what is inside /Tx BMC when it redraws a field (e.g. on focus
// changes), discarding what came before it (issues #1613, #633). The background and border
// therefore go inside the marked-content block too, so a redraw keeps them with the text.
@@ -913,8 +949,6 @@ fn field_appearance_fit(doc: &Document, f: &Field, w: &Widget, values: &[String]
.into_bytes();
content.extend(body);
content.extend_from_slice(b"ET\nQ\nEMC\n");
let mut fonts = Dict::new();
fonts.set(font_name.into_bytes(), font_obj);
let mut res = Dict::new();
res.set(b"Font".to_vec(), Object::Dict(fonts));
let mut d = Dict::new();
@@ -927,6 +961,31 @@ fn field_appearance_fit(doc: &Document, f: &Field, w: &Widget, values: &[String]
(Stream::flate(d, &content), unencodable, overflow)
}
/// The text operators for the `shown` lines (where each starts, and what it says) at `size`, in
/// the field's font `font_name` (`font`, written with `enc`) and, with `thai`, their Thai from the
/// Thai face; the font resources they use, and the first character drawn as `?` because the
/// font can't hold it, for the caller to report.
fn draw_lines(shown: &[(f64, f64, String)], size: f64, font_name: &str, font: Object, enc: &FontText, thai: bool) -> (Vec<u8>, Dict, Option<char>) {
let mut fonts = Dict::new();
fonts.set(font_name.as_bytes().to_vec(), font);
let lines: Vec<String> = shown.iter().map(|(_, _, text)| text.clone()).collect();
let (pieces, shaped) = if thai {
shaped_text::appearance_lines(&lines, &mut fonts)
} else {
(lines.into_iter().map(|l| vec![Piece::Latin(l)]).collect(), ShapedFonts::default())
};
let mut body = Vec::new();
let mut current = None;
for ((x, y, _), pieces) in shown.iter().zip(&pieces) {
shaped.show_line(&mut body, pieces, *x, *y, size, font_name, |s| enc.encode(s), |s| enc.width(s, size), &mut current);
}
let unencodable = pieces.iter().flatten().find_map(|p| match p {
Piece::Latin(s) => enc.first_unencodable(s),
Piece::Shaped(_) => None,
});
(body, fonts, unencodable)
}
/// On/Off appearances for a check box or radio button that has none (a check mark or a dot,
/// drawn as paths, so no symbol font is needed).
/// The mark is drawn in the colour of the widget's `/DA`, else of `f_da` (the field's): the
crates/forms/src/appearances.rs
+3
View File
@@ -75,6 +75,9 @@ pub fn with_field_appearances_cached(doc: &Document, cache: &mut FieldAppearance
}
};
let target = copy.get_or_insert_with(|| doc.clone());
let mut stream = stream;
// Thai glyphs are streams of their own.
pdfcraft_fonts::shaped_text::store_streams(target, &mut stream.dict);
let ap = target.add(Object::Stream(stream));
let mut fresh = Dict::new();
match &state {
crates/forms/src/author/look.rs
+3 -1
View File
@@ -323,8 +323,10 @@ pub(super) fn redraw_widgets(doc: &mut Document, name: &str, only: Option<usize>
let mut missing = None;
let ap = match f.kind {
FieldKind::Text | FieldKind::Combo | FieldKind::List => {
let (s, m) = appearance::field_appearance_drawn(doc, &f, w, &f.value, true);
let (mut s, m) = appearance::field_appearance_drawn(doc, &f, w, &f.value, true);
missing = m;
// Thai glyphs are streams of their own.
pdfcraft_fonts::shaped_text::store_streams(doc, &mut s.dict);
let r = doc.add(Object::Stream(s));
let mut d = Dict::new();
d.set(b"N".to_vec(), Object::Ref(r));
crates/forms/src/tests.rs
+55
View File
@@ -967,6 +967,61 @@ fn text_a_field_font_cannot_hold_is_kept_marked_and_redrawn_elsewhere() {
assert!(need_appearances(&doc));
}
/// Thai typed into a field whose `/DA` font (Helvetica, WinAnsi) can't hold it is drawn from the
/// Thai face as Add text draws it: Type 3 glyphs (stream objects of their own, with a ToUnicode
/// map) beside the field's font, which draws the rest, so every viewer shows the same glyphs. The
/// `/DA` is kept, nothing is marked and other viewers aren't asked to redraw. The ink of stacked
/// marks and descenders stays inside the border, an automatic size shrinking to keep it there.
/// A build without a Thai face refuses the text, saying why, and keeps the old value.
#[test]
fn thai_a_field_font_cannot_hold_is_drawn_from_the_thai_face() {
let objs: Vec<String> = vec![
"<< /Type /Catalog /Pages 2 0 R /AcroForm 4 0 R >>".into(),
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 600 800] >>".into(),
"<< /Type /Page /Parent 2 0 R /Annots [6 0 R 7 0 R 8 0 R] >>".into(),
"<< /Fields [6 0 R 7 0 R 8 0 R] /DA (/Helv 0 Tf 0 g) /DR << /Font << /Helv 5 0 R >> >> >>".into(),
"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>".into(),
"<< /Type /Annot /Subtype /Widget /FT /Tx /T (name) /DA (/Helv 0 Tf 0 g) /V (Ada) /Rect [50 700 250 720] /P 3 0 R >>".into(),
"<< /Type /Annot /Subtype /Widget /FT /Tx /T (big) /DA (/Helv 16 Tf 0 g) /Rect [50 600 350 640] /P 3 0 R >>".into(),
"<< /Type /Annot /Subtype /Widget /FT /Tx /T (note) /Ff 4096 /DA (/Helv 12 Tf 0 g) /Rect [50 400 350 500] /P 3 0 R >>".into(),
];
let mut doc = document(&objs);
let thai = "ชื่อ: บุญฤทธิ์ น้ำฎู";
if pdfcraft_fonts::document_thai_font().is_none() {
let err = set_value(&mut doc, "name", &FieldValue::Text(thai.into())).unwrap_err();
assert!(matches!(&err, FormError::Invalid(m) if m.contains("Thai") && m.contains("CRAFT_FONTS_DIR")), "{err}");
assert_eq!(field(&fields(&doc), "name").value, ["Ada"]);
assert!(!need_appearances(&doc) && undrawn_fields(&doc).is_empty());
return;
}
set_value(&mut doc, "name", &FieldValue::Text(thai.into())).unwrap();
set_value(&mut doc, "big", &FieldValue::Text(thai.into())).unwrap();
set_value(&mut doc, "note", &FieldValue::Text(format!("น้ำฎู {thai}\nปั้น ฎู"))).unwrap();
let doc = reopen(&doc);
assert!(undrawn_fields(&doc).is_empty() && !need_appearances(&doc));
let (lo, hi) = (pdfcraft_fonts::shaped_text::thai_reach("ฎู").unwrap()[0], pdfcraft_fonts::shaped_text::thai_reach("น้ำ").unwrap()[1]);
for (name, height) in [("name", 20.0), ("big", 40.0), ("note", 100.0)] {
let all = fields(&doc);
let w = &field(&all, name).widgets[0];
let wd = doc.get(w.obj).as_dict().unwrap().clone();
assert!(wd.get(b"DA").and_then(|o| o.as_string()).is_some_and(|s| s.to_text().starts_with("/Helv ")), "the /DA is kept");
let n = wd.get(b"AP").unwrap().as_dict().unwrap().reference(b"N").unwrap();
let Object::Stream(s) = &*doc.get(n) else { panic!() };
let ap = String::from_utf8_lossy(&s.decoded().unwrap()).into_owned();
assert!(!ap.contains('?') && ap.contains("/PCTh1 ") && ap.contains("(: ) Tj"), "{ap}");
let fonts = s.dict.get(b"Resources").and_then(|r| r.as_dict()).and_then(|r| r.get(b"Font")).and_then(|f| f.as_dict()).unwrap();
let font = fonts.get(b"PCTh1").and_then(|f| f.as_dict()).unwrap();
assert_eq!(font.name(b"Subtype"), Some(&b"Type3"[..]));
let Object::Stream(cmap) = &*doc.get(font.reference(b"ToUnicode").unwrap()) else { panic!("ToUnicode is a stream object") };
assert!(String::from_utf8_lossy(&cmap.decoded().unwrap()).contains("<0E190E490E33>"), "น้ำ copies as typed");
// The size each line is drawn at, and where: every line's ink is inside the box.
let size: f64 = ap.split(" Tf").next().and_then(|s| s.rsplit(' ').next()).and_then(|s| s.parse().ok()).unwrap();
let ys: Vec<f64> = ap.split(" Tm").filter_map(|s| s.rsplit(' ').next()?.parse().ok()).collect();
let (top, bottom) = (ys.iter().copied().fold(f64::MIN, f64::max), ys.iter().copied().fold(f64::MAX, f64::min));
assert!(top + hi * size <= height + 1e-3 && bottom + lo * size >= -1e-3, "{name}: {ys:?} at {size} in {height}");
}
}
/// A document that declares PDF/A can't ask other viewers to redraw its fields (PDF/A forbids
/// `/NeedAppearances`): the value is still kept and its widget marked, and the flag stays off.
#[test]
crates/forms/src/values.rs
+8 -1
View File
@@ -206,6 +206,11 @@ pub(crate) fn write_value(doc: &mut Document, f: &Field, value: &FieldValue, scr
{
return invalid(format!("{:?} takes at most {max} characters", f.name));
}
// Thai this build can't draw is refused (before any script runs), rather than stored
// and drawn as `?`.
if let Some(why) = appearance::thai_refusal(doc, f, t) {
return invalid(format!("{:?} can't take this text: {why}", f.name));
}
// Keystroke (on commit) and Validate, as Acrobat runs them before accepting a value.
let t = &af::keystroke(&f.actions.format, &f.name, t).map_err(FormError::Format)?;
af::validate(&f.actions.validate, t).map_err(FormError::Invalid)?;
@@ -430,7 +435,7 @@ fn redraw(doc: &mut Document, f: &Field, values: &[String], scripts: &mut dyn Sc
// fonts can't draw the date language's (#5303).
let mut english: Option<Option<String>> = None;
for w in &f.widgets {
let (stream, missing) = match &shown {
let (mut stream, missing) = match &shown {
Some(s) => {
let drawn = appearance::field_appearance_drawn(doc, f, w, std::slice::from_ref(s), false);
// Taken when the English run happened and draws whole, even where it is the
@@ -447,6 +452,8 @@ fn redraw(doc: &mut Document, f: &Field, values: &[String], scripts: &mut dyn Sc
None => appearance::field_appearance_drawn(doc, f, w, values, true),
};
undrawn |= missing.is_some();
// Thai glyphs are streams of their own.
pdfcraft_fonts::shaped_text::store_streams(doc, &mut stream.dict);
let existing_ap = in_place
.then(|| {
doc.get(w.obj)
crates/geom/src/lib.rs
+1
View File
@@ -6,6 +6,7 @@
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
pub mod perspective;
pub mod quad_points;
/// An axis-aligned rectangle in PDF user space (points).
#[derive(Clone, Copy, Debug, PartialEq, Default)]
crates/geom/src/quad_points.rs
+88
View File
@@ -0,0 +1,88 @@
//! The quadrilaterals of a `/QuadPoints` array (ISO 32000-2 §12.5.6.10), read by one rule.
//!
//! Redaction apply, the comment summaries and outlines, the inspector and generated appearances
//! all read `/QuadPoints`; they go through [`usable_quads`] so the area a user sees marked is the
//! area apply removes.
/// The usable quadrilaterals of a `/QuadPoints` array, given its entries in order (`None` for an
/// entry that is not a number).
///
/// A quadrilateral is usable only when all eight of its entries are present, numbers, and within
/// the range of a PDF real (finite as an `f32`, Annex C), so the readers that keep `f32` see the
/// same set as those that keep `f64`. Usable values come back on the `f32` grid (each one put
/// through `f32` and back), so the readers fold the same numbers too: a value `f32` cannot hold
/// exactly, like 67108869, is 67108872 for every reader, not just the ones that keep `f32`. An
/// unusable one is skipped whole: the ones after it keep their coordinates, and it never becomes
/// a folded box or zeros. A trailing partial group is dropped.
pub fn usable_quads<I: IntoIterator<Item = Option<f64>>>(entries: I) -> Vec<[f64; 8]> {
let entries: Vec<Option<f64>> = entries.into_iter().collect();
entries
.as_chunks::<8>()
.0
.iter()
.filter_map(|q| {
let mut quad = [0.0; 8];
for (slot, v) in quad.iter_mut().zip(q) {
// NaN and infinities stay non-finite as `f32`; so does a value past its range.
*slot = v.filter(|v| (*v as f32).is_finite()).map(|v| (v as f32) as f64)?;
}
Some(quad)
})
.collect()
}
#[cfg(test)]
mod tests {
use super::usable_quads;
const A: [f64; 8] = [10.0, 20.0, 50.0, 20.0, 10.0, 40.0, 50.0, 40.0];
const B: [f64; 8] = [60.0, 70.0, 90.0, 70.0, 60.0, 80.0, 90.0, 80.0];
fn some(v: &[f64]) -> Vec<Option<f64>> {
v.iter().copied().map(Some).collect()
}
#[test]
fn finite_quads_are_all_usable() {
assert_eq!(usable_quads(some(&[A, B].concat())), vec![A, B]);
assert!(usable_quads(Vec::new()).is_empty());
}
#[test]
fn an_all_nan_quad_is_unusable() {
assert!(usable_quads(some(&[f64::NAN; 8])).is_empty());
}
#[test]
fn one_non_finite_entry_makes_its_quad_unusable() {
for bad in [f64::NAN, f64::INFINITY, f64::NEG_INFINITY, 1e300] {
let mut q = A;
q[3] = bad;
assert_eq!(usable_quads(some(&[q, B].concat())), vec![B], "{bad}");
}
}
#[test]
fn usable_values_come_back_on_the_f32_grid() {
let mut q = A;
q[0] = 67_108_869.0; // `f32` cannot hold it exactly: it rounds to 67108872 there.
q[1] = 0.1;
let got = usable_quads(some(&q));
assert_eq!(got.len(), 1);
assert_eq!(got[0][0], (67_108_869.0f64 as f32) as f64);
assert_eq!(got[0][1], (0.1f64 as f32) as f64);
assert_ne!(got[0][0], q[0]);
}
#[test]
fn a_non_number_drops_its_quad_without_shifting_the_next() {
let mut entries = some(&[A, B].concat());
entries[2] = None;
assert_eq!(usable_quads(entries), vec![B]);
}
#[test]
fn a_trailing_partial_group_is_dropped() {
assert_eq!(usable_quads(some(&[&A[..], &B[..4]].concat())), vec![A]);
}
}
crates/ocr/Cargo.toml
+2
View File
@@ -14,6 +14,8 @@ workspace = true
pdfcraft-fonts = { workspace = true }
ocrs = { version = "0.13.1", default-features = false, features = ["rten"] }
rten = { version = "0.26.0", default-features = false, features = ["onnx_format", "rten_format"] }
serde_json.workspace = true
sha2 = "0.11"
thiserror.workspace = true
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
crates/ocr/src/lib.rs
+138 -1
View File
@@ -29,7 +29,9 @@ pub mod tesseract;
pub use confidence::{ConfidenceBand, band_for, is_suspect, low_confidence};
pub use merge::{MergeStrategy, StrategyOutcome, mean_confidence, recognize_with_strategy, rover_merge};
pub use ocrs::{ALPHABET, DETECTION_MODEL, LANGUAGES, Models, Ocr, OcrsRecognizer, RECOGNITION_MODEL};
pub use ocrs::{
ALPHABET, DETECTION_MODEL, DETECTION_MODEL_SHA256, LANGUAGES, Models, Ocr, OcrsRecognizer, RECOGNITION_MODEL, RECOGNITION_MODEL_SHA256,
};
pub use pdfcraft_fonts::TextPlan;
pub use pdfcraft_fonts::helvetica_width;
@@ -464,6 +466,58 @@ pub fn text_layer(words: &[PlacedWord], plan: &pdfcraft_fonts::TextPlan) -> Vec<
out
}
/// The recognised lines as JSON, `[[["word", left, top, right, bottom], …], …]`, with the
/// word's confidence as a sixth item when it has one: how the web build's OCR workers send what
/// they found back to the page.
pub fn lines_to_json(lines: &[Line]) -> String {
let word = |w: &Word| {
let mut items = vec![serde_json::json!(w.text), w.rect[0].into(), w.rect[1].into(), w.rect[2].into(), w.rect[3].into()];
items.extend(w.confidence.map(serde_json::Value::from));
serde_json::Value::Array(items)
};
serde_json::Value::Array(lines.iter().map(|l| serde_json::Value::Array(l.words.iter().map(word).collect())).collect()).to_string()
}
/// At most this many words come back for one page (a dense page has a few thousand).
const MAX_JSON_WORDS: usize = 100_000;
/// [`lines_to_json`] read back, the words credited to `source` (the engine the worker ran).
/// Anything else (a word without its box, a box or confidence that isn't a finite number, too
/// many words) is an error rather than a guess.
pub fn lines_from_json(json: &str, source: &str) -> Result<Vec<Line>, OcrError> {
let bad = |why: &str| OcrError::Recognize(format!("unreadable result: {why}"));
let value: serde_json::Value = serde_json::from_str(json).map_err(|e| bad(&e.to_string()))?;
let lines = value.as_array().ok_or_else(|| bad("not a list of lines"))?;
let number = |v: &serde_json::Value| v.as_f64().map(|v| v as f32).filter(|v| v.is_finite());
let mut count = 0usize;
let mut out = Vec::with_capacity(lines.len().min(MAX_JSON_WORDS));
for line in lines {
let mut words = Vec::new();
for word in line.as_array().ok_or_else(|| bad("a line is not a list of words"))? {
count += 1;
if count > MAX_JSON_WORDS {
return Err(bad("too many words"));
}
let (text, rect, confidence) = match word.as_array().map(Vec::as_slice) {
Some([serde_json::Value::String(text), l, t, r, b]) => (text, [l, t, r, b], None),
Some([serde_json::Value::String(text), l, t, r, b, c]) => (text, [l, t, r, b], Some(c)),
_ => return Err(bad("a word is not [text, left, top, right, bottom, confidence?]")),
};
let mut r = [0f32; 4];
for (slot, v) in r.iter_mut().zip(rect) {
*slot = number(v).ok_or_else(|| bad("a word's box is not finite numbers"))?;
}
let mut w = Word::new(text.clone(), r, source);
if let Some(c) = confidence {
w = w.with_confidence(number(c).ok_or_else(|| bad("a word's confidence is not a finite number"))?);
}
words.push(w);
}
out.push(Line { words });
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
@@ -854,6 +908,89 @@ mod tests {
assert_eq!(ok.gray(-1, -1), Some(255));
}
/// The web build loads the models from downloaded bytes; they read the same text.
#[test]
fn reads_rendered_text_with_models_from_bytes() {
let Some(models) = Models::find().filter(|m| m.detection.ends_with(DETECTION_MODEL)) else {
eprintln!("skipped: ONNX OCR models not installed");
return;
};
let ocr = Ocr::from_bytes(std::fs::read(&models.detection).unwrap(), std::fs::read(&models.recognition).unwrap()).unwrap();
let rec = OcrsRecognizer::from_ocr(ocr);
assert!(rec.available(), "a recogniser loaded from bytes needs no model files");
let (w, h, px) = test_image::hello();
let lines = rec.recognize(&OcrImage::new(w, h, px).unwrap(), &RecognizeOptions::default()).unwrap();
let text: Vec<String> = lines.iter().map(Line::text).collect();
assert!(text.iter().any(|t| t.contains("HELL") && t.contains("WOR")), "{text:?}");
}
/// The web build's workers send lines back as JSON; they arrive unchanged, credited to the
/// engine the worker ran.
#[test]
fn lines_survive_json() {
let lines = vec![
Line {
words: vec![
Word::new("Total \"€250\"", [1.5, 2.0, 30.25, 12.0], "ocrs"),
Word::new("due", [32.0, 2.0, 50.0, 12.0], "ocrs").with_confidence(87.5),
],
},
Line::default(),
Line { words: vec![Word::new("日本語", [0.0, 20.0, 9.0, 30.0], "ocrs")] },
];
assert_eq!(lines_from_json(&lines_to_json(&lines), "ocrs").unwrap(), lines);
assert_eq!(lines_from_json(&lines_to_json(&[]), "ocrs").unwrap(), Vec::<Line>::new());
let other = lines_from_json(&lines_to_json(&lines), "tesseract").unwrap();
assert!(other.iter().flat_map(|l| &l.words).all(|w| w.source == "tesseract"));
}
/// A result that isn't what a worker sends is an error, not a crash or a guess.
#[test]
fn malformed_json_lines_are_an_error() {
for json in [
"",
"{}",
"[1]",
"[[1]]",
"[[[\"a\", 1, 2, 3]]]",
"[[[\"a\", 1, 2, 3, 4, 5, 6]]]",
"[[[1, 1, 2, 3, 4]]]",
"[[[\"a\", \"1\", 2, 3, 4]]]",
"[[[\"a\", 1e999, 2, 3, 4]]]",
"[[[\"a\", null, 2, 3, 4]]]",
"[[[\"a\", 1, 2, 3, 4, null]]]",
"[[[\"a\", 1, 2, 3, 4, \"90\"]]]",
] {
assert!(matches!(lines_from_json(json, "ocrs"), Err(OcrError::Recognize(_))), "{json}");
}
let many = format!("[[{}]]", vec!["[\"a\",0,0,1,1]"; MAX_JSON_WORDS + 1].join(","));
assert!(lines_from_json(&many, "ocrs").is_err());
let deep = format!("{}{}", "[".repeat(10_000), "]".repeat(10_000));
assert!(lines_from_json(&deep, "ocrs").is_err());
}
/// A download that isn't a model (an HTML error page, a truncated file) is an error, not a
/// crash or a hang.
#[test]
fn models_from_bad_bytes_are_an_error() {
for (detection, recognition) in [(Vec::new(), Vec::new()), (b"<!doctype html>".to_vec(), b"RTEN".to_vec()), (vec![0xff; 64], vec![0; 64])] {
assert!(matches!(Ocr::from_bytes(detection, recognition), Err(OcrError::Load(..))));
}
}
/// The hashes [`Ocr::from_bytes`] checks are the ones `cargo xtask models` verifies.
#[test]
fn model_hashes_match_attribution() {
let manifest = include_str!("../../../ATTRIBUTION.toml");
for (file, sha256) in [(DETECTION_MODEL, DETECTION_MODEL_SHA256), (RECOGNITION_MODEL, RECOGNITION_MODEL_SHA256)] {
let entry = manifest
.split("[[fetched]]")
.find(|e| e.contains(&format!("file = \"{file}\"")))
.unwrap_or_else(|| panic!("{file} is not in ATTRIBUTION.toml"));
assert!(entry.lines().any(|l| l == format!("sha256 = \"{sha256}\"")), "{file}: SHA-256 differs from ATTRIBUTION.toml");
}
}
mod test_image {
/// "HELLO WORLD" drawn with blocky 5×7 letters, 4 pixels per dot, black on white.
pub fn hello() -> (u32, u32, Vec<u8>) {
crates/ocr/src/ocrs.rs
+38 -4
View File
@@ -32,6 +32,15 @@ pub const LANGUAGES: &[(&str, &str)] = &[("en", "English")];
/// The model files, as named in ATTRIBUTION.toml.
pub const DETECTION_MODEL: &str = "text-detection.onnx";
pub const RECOGNITION_MODEL: &str = "text-recognition.onnx";
/// Their SHA-256s, as in ATTRIBUTION.toml. [`Ocr::from_bytes`] loads nothing else: rten's ONNX
/// parser can spin forever on bytes that aren't a model (64 × 0xff does).
pub const DETECTION_MODEL_SHA256: &str = "a917b23dbd9524b465df7e922641b3ff2981623df4ded5a0234004ef2fee7cfe";
pub const RECOGNITION_MODEL_SHA256: &str = "86c145c2edb96c8caed5b1ebb8f44d706408922211451c309c625157dd6061c5";
fn sha256_hex(bytes: &[u8]) -> String {
use sha2::{Digest, Sha256};
Sha256::digest(bytes).iter().map(|b| format!("{b:02x}")).collect()
}
pub const ALPHABET: &str = " 0123456789!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~€ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
@@ -84,9 +93,25 @@ pub struct Ocr {
impl Ocr {
pub fn load(models: &Models) -> Result<Ocr, OcrError> {
let load = |p: &Path| rten::Model::load_file(p).map_err(|e| OcrError::Load(p.display().to_string(), e.to_string()));
Self::with_models(load(&models.detection)?, load(&models.recognition)?)
}
/// Load the models from the contents of their files, where there is no file system (the web
/// build downloads them). Bytes that aren't the files ATTRIBUTION.toml names are an error.
pub fn from_bytes(detection: Vec<u8>, recognition: Vec<u8>) -> Result<Ocr, OcrError> {
let load = |name: &str, sha256: &str, data: Vec<u8>| {
if sha256_hex(&data) != sha256 {
return Err(OcrError::Load(name.into(), "not the expected model file (SHA-256 differs)".into()));
}
rten::Model::load(data).map_err(|e| OcrError::Load(name.into(), e.to_string()))
};
Self::with_models(load(DETECTION_MODEL, DETECTION_MODEL_SHA256, detection)?, load(RECOGNITION_MODEL, RECOGNITION_MODEL_SHA256, recognition)?)
}
fn with_models(detection: rten::Model, recognition: rten::Model) -> Result<Ocr, OcrError> {
let params = ocrs::OcrEngineParams {
detection_model: Some(load(&models.detection)?),
recognition_model: Some(load(&models.recognition)?),
detection_model: Some(detection),
recognition_model: Some(recognition),
alphabet: Some(ALPHABET.into()),
..Default::default()
};
@@ -139,12 +164,21 @@ impl Ocr {
/// The ocrs engine as a [`Recognizer`] (the default engine).
pub struct OcrsRecognizer {
engine: Ocr,
/// Loaded from bytes ([`OcrsRecognizer::from_ocr`]), not from model files: it stays
/// available without them.
in_memory: bool,
}
impl OcrsRecognizer {
/// Load the models from `models`.
pub fn load(models: &Models) -> Result<OcrsRecognizer, OcrError> {
Ok(OcrsRecognizer { engine: Ocr::load(models)? })
Ok(OcrsRecognizer { engine: Ocr::load(models)?, in_memory: false })
}
/// A recogniser loaded some other way than from model files: the web build downloads the
/// models and loads them with [`Ocr::from_bytes`].
pub fn from_ocr(engine: Ocr) -> OcrsRecognizer {
OcrsRecognizer { engine, in_memory: true }
}
/// Load the models found by [`Models::find`].
@@ -159,7 +193,7 @@ impl Recognizer for OcrsRecognizer {
}
fn available(&self) -> bool {
Models::find().is_some()
self.in_memory || Models::find().is_some()
}
fn languages(&self) -> Vec<(String, String)> {
crates/organize/src/import.rs
+58
View File
@@ -480,6 +480,64 @@ pub(crate) fn register_layers(dst: &mut Document, src: &Document, ocgs: &[(ObjRe
Ok(())
}
/// The objects `value` (of `src`) refers to, directly or through others, each once whatever the
/// cycles: what [`import_value`] copies. Refused past `max_objects` objects or `max_bytes` bytes
/// of stream data, with a message that says how much there is.
pub fn reachable(src: &Document, value: &Object, max_objects: usize, max_bytes: usize) -> Result<Vec<ObjRef>, OrganizeError> {
let mut seen = HashSet::new();
let mut found = Vec::new();
let mut bytes = 0usize;
// The values still to look into: owned, or an object of `src`.
let mut pending: Vec<std::sync::Arc<Object>> = vec![std::sync::Arc::new(value.clone())];
while let Some(next) = pending.pop() {
let mut refs = Vec::new();
let mut inside: Vec<&Object> = vec![&next];
while let Some(o) = inside.pop() {
match o {
Object::Ref(r) => refs.push(*r),
Object::Array(a) => inside.extend(a.iter()),
Object::Dict(d) => inside.extend(d.iter().map(|(_, v)| v)),
Object::Stream(s) => {
bytes = bytes.saturating_add(s.raw.len());
inside.extend(s.dict.iter().map(|(_, v)| v));
}
_ => {}
}
}
for r in refs {
if seen.insert(r) {
found.push(r);
pending.push(src.get(r));
}
}
if found.len() > max_objects || bytes > max_bytes {
return Err(OrganizeError::Invalid(format!(
"it uses more than {max_objects} objects or {} MB of fonts, images and other data; copy fewer objects at a time",
max_bytes >> 20
)));
}
}
Ok(found)
}
/// Copy `value` from `src` into `dst` with every object it refers to, each once (cycles end), as
/// Insert Pages copies a page's resources; what equals an object already in `dst` (the same font
/// or image) is shared rather than stored twice. Returns `value` as `dst` refers to it. A chain
/// nested more than 256 levels deep is refused; [`reachable`] bounds the size first.
pub fn import_value(dst: &mut Document, src: &Document, value: &Object) -> Result<Object, OrganizeError> {
let before: HashSet<u32> = dst.object_numbers().into_iter().collect();
let mut copier = Copier::new(src, HashMap::new(), HashMap::new());
let copied = copier.copy_value(dst, value, None)?;
// Held in an array object while duplicates merge (an array is never a resource that merges
// itself away), so the references in it follow them.
let holder = dst.add(Object::Array(vec![copied]));
let created = added_objects(dst, &before);
crate::dedupe::dedupe_resources(dst, &created, true);
let held = dst.get(holder).as_array().and_then(|a| a.first().cloned()).unwrap_or(Object::Null);
dst.free(holder);
Ok(held)
}
/// Page `page` of `src` as a form XObject in `dst` (its content and resources, no annotations),
/// for backgrounds and watermarks taken from a PDF. The form's `/BBox` is the page's crop box
/// and its `/Matrix` undoes the page rotation, so it draws upright as displayed. Returns the
crates/organize/src/lib.rs
+2 -2
View File
@@ -33,8 +33,8 @@ pub use comments::import_comments;
pub use dedupe::dedupe_resources;
pub use import::{
CombineOutcome, Combiner, FieldConflicts, ImportOutcome, Run, SplitBy, combine, combine_field_conflicts, combine_grouped, combine_grouped_with,
combine_selected, combine_selected_with, combine_with, extract_pages, field_name_conflicts, import_pages, import_pages_with, insert_file_pages,
page_as_form, split, split_ranges,
combine_selected, combine_selected_with, combine_with, extract_pages, field_name_conflicts, import_pages, import_pages_with, import_value,
insert_file_pages, page_as_form, reachable, split, split_ranges,
};
pub use labels::{LabelRange, LabelStyle, PageLabels, number_pages, page_label_ranges, page_labels, set_page_label_ranges};
pub use outline::{
crates/redact/Cargo.toml
+2
View File
@@ -13,6 +13,8 @@ aho-corasick = "1"
pdfcraft-cos = { workspace = true }
pdfcraft-content = { workspace = true }
pdfcraft-fonts = { workspace = true }
# The usable-quad rule `/QuadPoints` readers share.
pdfcraft-geom = { workspace = true }
pdfcraft-model = { workspace = true }
pdfcraft-forms = { workspace = true }
pdfcraft-edit = { workspace = true }
crates/redact/src/lib.rs
+11 -8
View File
@@ -185,6 +185,14 @@ pub(crate) fn rect_of(doc: &Document, o: Option<&Object>) -> Option<[f64; 4]> {
(v.len() == 4 && v.iter().all(|x| x.is_finite())).then(|| [v[0].min(v[2]), v[1].min(v[3]), v[0].max(v[2]), v[1].max(v[3])])
}
/// A mark's usable `/QuadPoints` quads ([`pdfcraft_geom::quad_points::usable_quads`]), the set the
/// comment outlines show. Each one is finite, so the box it folds to is a real one.
fn quad_points(doc: &Document, d: &Dict) -> Vec<[f64; 8]> {
let Some(q) = d.get(b"QuadPoints").map(|q| doc.resolve(q)) else { return Vec::new() };
let Some(a) = q.as_array() else { return Vec::new() };
pdfcraft_geom::quad_points::usable_quads(a.iter().map(|x| doc.resolve(x).as_f64()))
}
fn color(doc: &Document, d: &Dict, key: &[u8]) -> Option<Rgb> {
let v: Vec<f64> = doc.resolve(d.get(key)?).as_array()?.iter().filter_map(Object::as_f64).collect();
match v.len() {
@@ -229,13 +237,7 @@ pub(crate) fn marks_checked(doc: &Document) -> (Vec<Mark>, Vec<usize>) {
if d.name(b"Subtype") != Some(b"Redact") {
continue;
}
let quads: Vec<f64> = d
.get(b"QuadPoints")
.and_then(|q| doc.resolve(q).as_array().map(|a| a.iter().filter_map(Object::as_f64).collect()))
.unwrap_or_default();
let mut rects: Vec<[f64; 4]> = quads
.as_chunks::<8>()
.0
let mut rects: Vec<[f64; 4]> = quad_points(doc, d)
.iter()
.map(|q| {
let xs = [q[0], q[2], q[4], q[6]];
@@ -247,8 +249,9 @@ pub(crate) fn marks_checked(doc: &Document) -> (Vec<Mark>, Vec<usize>) {
ys.iter().copied().fold(f64::MIN, f64::max),
]
})
.filter(|r| r.iter().all(|v| v.is_finite()))
.collect();
// No usable quad (all absent, malformed or non-finite): the mark's `/Rect`, else it
// is unreadable and apply refuses it.
if rects.is_empty()
&& let Some(r) = rect_of(doc, d.get(b"Rect"))
{
crates/redact/src/tests.rs
+159
View File
@@ -2405,6 +2405,165 @@ fn an_unreadable_mark_on_another_page_does_not_stop_the_selected_page() {
assert!(!content(&doc, 0).contains("1234"));
}
/// The quad over "1234" in `AB1234CD` (x 20–40) and the one over "CD" (x 40–50).
const OVER_1234: [f64; 8] = [20.0, 110.0, 40.0, 110.0, 20.0, 95.0, 40.0, 95.0];
const OVER_CD: [f64; 8] = [40.0, 110.0, 50.0, 110.0, 40.0, 95.0, 50.0, 95.0];
/// `AB1234CD` with one mark whose `/QuadPoints` is `quads`, whose `/Rect` is `rect` (removed when
/// `None`) and which has a fill (`/IC`) only when `fill` is set.
fn quad_mark(quads: Vec<Object>, rect: Option<[i64; 4]>, fill: bool) -> (Document, ObjRef) {
let mut doc = one_page(b"BT /F1 10 Tf 10 100 Td (AB1234CD) Tj ET", "", vec![]);
mark(&mut doc, 0, &[[20.0, 95.0, 40.0, 110.0]], "");
let m = marks(&doc).swap_remove(0);
doc.update_dict(m.obj, |d| {
d.set(b"QuadPoints".to_vec(), Object::Array(quads));
if let Some(r) = rect {
d.set(b"Rect".to_vec(), Object::Array(r.iter().map(|v| Object::Int(*v)).collect()));
} else {
d.remove(b"Rect");
}
if !fill {
d.remove(b"IC");
}
})
.unwrap();
(doc, m.obj)
}
fn reals(v: &[f64]) -> Vec<Object> {
v.iter().map(|x| Object::Real(*x)).collect()
}
/// An all-NaN quad folded to `[MAX, MAX, MIN, MIN]`: a finite, inverted box that passed as a
/// readable area. Without a fill the mark was applied and removed nothing; with one the overlay
/// failed. With no `/Rect` either, the mark has no area and apply refuses it.
#[test]
fn an_all_nan_quad_without_a_rect_is_refused_not_applied_as_nothing() {
for fill in [false, true] {
let (mut doc, _) = quad_mark(reals(&[f64::NAN; 8]), None, fill);
assert!(marks(&doc).is_empty(), "fill {fill}: the NaN quad is no area");
assert_eq!(apply(&mut doc, None), Err(RedactError::UnreadableMark(1)), "fill {fill}");
assert!(content(&doc, 0).contains("(AB1234CD) Tj"), "fill {fill}: nothing changed");
}
}
/// With a usable `/Rect`, an all-NaN mark redacts exactly the `/Rect`.
#[test]
fn an_all_nan_quad_falls_back_to_the_rect() {
for fill in [false, true] {
let (mut doc, _) = quad_mark(reals(&[f64::NAN; 8]), Some([20, 95, 40, 110]), fill);
assert_eq!(marks(&doc)[0].rects, vec![[20.0, 95.0, 40.0, 110.0]], "fill {fill}");
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.marks, r.glyphs), (1, 4), "fill {fill}");
assert!(!content(&doc, 0).contains("1234"), "fill {fill}");
}
}
/// One NaN among finite entries made `f64::min`/`max` ignore it and fold the other three corners:
/// a quad the document never described. Such a quad is unusable.
#[test]
fn a_quad_with_one_nan_entry_is_unusable() {
let mut q = OVER_1234;
q[0] = f64::NAN;
let (mut doc, _) = quad_mark(reals(&q), None, false);
assert!(marks(&doc).is_empty());
assert_eq!(apply(&mut doc, None), Err(RedactError::UnreadableMark(1)));
// Next to a usable quad, only that one applies.
let (mut doc, _) = quad_mark(reals(&[q, OVER_CD].concat()), None, false);
assert_eq!(marks(&doc)[0].rects, vec![[40.0, 95.0, 50.0, 110.0]]);
assert_eq!(apply(&mut doc, None).map(|r| r.glyphs), Ok(2));
}
/// A name in the middle of `/QuadPoints` was dropped, which moved every later coordinate into the
/// wrong slot and folded the first quad to a box over most of the page.
#[test]
fn a_non_number_entry_drops_its_quad_without_shifting_the_rest() {
let mut quads = reals(&[OVER_1234, OVER_CD].concat());
quads[2] = Object::name("oops");
let (mut doc, _) = quad_mark(quads, None, false);
assert_eq!(marks(&doc)[0].rects, vec![[40.0, 95.0, 50.0, 110.0]]);
assert_eq!(apply(&mut doc, None).map(|r| r.glyphs), Ok(2));
}
/// Twelve entries are one quad and half of another: only the whole one applies.
#[test]
fn a_trailing_partial_quad_is_ignored() {
let (mut doc, _) = quad_mark(reals(&[&OVER_1234[..], &OVER_CD[..4]].concat()), None, false);
assert_eq!(marks(&doc)[0].rects, vec![[20.0, 95.0, 40.0, 110.0]]);
assert_eq!(apply(&mut doc, None).map(|r| r.glyphs), Ok(4));
}
/// Finite quads redact what they always did, one box each, and the `/Rect` is not consulted.
#[test]
fn finite_quads_redact_their_areas_as_before() {
let (mut doc, _) = quad_mark(reals(&[OVER_1234, OVER_CD].concat()), Some([0, 0, 1, 1]), true);
assert_eq!(marks(&doc)[0].rects, vec![[20.0, 95.0, 40.0, 110.0], [40.0, 95.0, 50.0, 110.0]]);
let r = apply(&mut doc, None).unwrap();
assert_eq!((r.marks, r.glyphs), (1, 6));
assert!(!content(&doc, 0).contains("1234"));
}
/// The boxes a reader's quads outline, or its `/Rect` when it has none (as the comment outlines
/// fall back to it).
fn outline(quads: &[[f32; 8]], rect: [f32; 4]) -> Vec<[f32; 4]> {
if quads.is_empty() {
return vec![rect];
}
quads
.iter()
.map(|q| {
let (xs, ys) = ([q[0], q[2], q[4], q[6]], [q[1], q[3], q[5], q[7]]);
let (lo, hi) = (|v: [f32; 4]| v.into_iter().fold(f32::MAX, f32::min), |v: [f32; 4]| v.into_iter().fold(f32::MIN, f32::max));
[lo(xs), lo(ys), hi(xs), hi(ys)]
})
.collect()
}
/// The outline the comment list draws (`pdfcraft_annot::summaries`, and the inspector's listing
/// of the saved file) is the area apply removes, for malformed `/QuadPoints` too. The readers
/// used to disagree: the summaries and the inspector read non-numbers as zeros, apply dropped
/// them.
#[test]
fn the_outline_shown_is_the_area_apply_removes() {
// (entries, whether a saved file can say them: PDF syntax has no NaN). `forty` is an indirect
// number, which every reader follows.
let cases = |forty: ObjRef| -> Vec<(Vec<Object>, bool)> {
let mut name_inside = reals(&[OVER_1234, OVER_CD].concat());
name_inside[2] = Object::name("oops");
let mut by_reference = reals(&OVER_1234);
by_reference[2] = Object::Ref(forty);
let mut one_nan = OVER_1234;
one_nan[5] = f64::NAN;
let mut one_huge = OVER_1234;
one_huge[5] = 4e38;
vec![
(reals(&[OVER_1234, OVER_CD].concat()), true),
(name_inside, true),
(reals(&[&OVER_1234[..], &OVER_CD[..4]].concat()), true),
(by_reference, true),
(vec![Object::name("oops"); 8], true),
(reals(&[one_nan, OVER_CD].concat()), false),
(reals(&[one_huge, OVER_CD].concat()), true),
(reals(&[f64::NAN; 8]), false),
]
};
for i in 0.. {
let (mut doc, obj) = quad_mark(Vec::new(), Some([20, 95, 40, 110]), true);
let forty = doc.add(Object::Int(40));
let Some((quads, saved)) = cases(forty).into_iter().nth(i) else { break };
doc.update_dict(obj, |d| d.set(b"QuadPoints".to_vec(), Object::Array(quads))).unwrap();
let applied: Vec<[f32; 4]> = marks(&doc)[0].rects.iter().map(|r| r.map(|v| v as f32)).collect();
let summary = pdfcraft_annot::summaries(&doc).into_iter().find(|s| s.subtype == "Redact").unwrap();
assert_eq!(outline(&summary.quads, summary.rect), applied, "case {i}: comment summaries");
if saved {
let bytes = write_incremental(&doc, &SaveOptions::default()).unwrap();
let info = pdfcraft_render::inspect(Arc::new(bytes), None).unwrap();
let listed = info.annotations.iter().find(|a| a.subtype == "Redact").unwrap();
assert_eq!(outline(&listed.quads, listed.rect), applied, "case {i}: inspector");
}
}
}
/// A document with something in every hidden-information category.
fn hidden_fixture() -> Document {
let content = b"BT /F1 10 Tf 10 100 Td (Visible) Tj 3 Tr (Hidden) Tj 0 Tr ET BT /F1 10 Tf 500 500 Td (Offpage) Tj ET /OC /L1 BDC BT /F1 10 Tf 10 50 Td (Layer) Tj ET EMC";
crates/render/Cargo.toml
+3 -1
View File
@@ -11,7 +11,9 @@ workspace = true
[dependencies]
pdfcraft-cos = { workspace = true }
pdfcraft-fonts = { workspace = true }
hayro = { workspace = true }
# The usable-quad rule `/QuadPoints` readers share.
pdfcraft-geom = { workspace = true }
hayro ={ workspace = true }
hayro-syntax = { workspace = true }
# Picks the face of an installed font collection (the version pdfcraft-fonts already uses).
skrifa = "0.44"
crates/render/src/inspect/annotations.rs
+4 -2
View File
@@ -138,10 +138,12 @@ impl<'a> Inspector<'a> {
color = Some([f(i - 3), f(i - 2), f(i - 1)]);
}
}
// Only usable quads, by the rule redaction apply reads them with (keep in step with
// `pdfcraft_annot::summaries`): a malformed one is not drawn as zeros.
let quads = match d.get(b"QuadPoints").map(|o| self.resolve(o)) {
Ok(Object::Array(q)) => {
let v: Vec<f32> = q.iter().map(|o| o.as_float().unwrap_or(0.0)).collect();
v.as_chunks::<8>().0.to_vec()
let entries = q.iter().map(|o| self.resolve(o).as_float().ok().map(f64::from));
pdfcraft_geom::quad_points::usable_quads(entries).into_iter().map(|quad| quad.map(|v| v as f32)).collect()
}
_ => Vec::new(),
};
Loaded 100 of 141 files, more files were not shown because too many files have changed in this diff. Show more