Files
pdfcraft/crates/engine
d579f26de3 M11: JBIG2 images share a symbol dictionary, up to 16 at a time (optimize.mono-compression) (#5910)
* M11: JBIG2 images share a symbol dictionary, up to 16 at a time

With Monochrome Images set to JBIG2, Optimize stored each image's
letters in a symbol dictionary of its own, so a document of N pages
carried its font's shapes N times. Now up to 16 images the run stores
as JBIG2 share one dictionary in a /JBIG2Globals stream, holding the
marks found in more than one of them; each image's text regions refer
to it first, then to a dictionary of its own for the marks only it
repeats. An image is stored that way when it comes out smaller, and a
group only when what its images save is more than the globals take.

- filters: encode_jbig2_shared codes a group: the globals are a symbol
  dictionary segment numbered 0 and associated with no page (7.2.6);
  each page numbers its segments from 1, as a segment refers only to
  lower numbers (7.2.5), and the shared symbols' IDs come first, as the
  dictionaries are referred to in that order (Table 31, SBSYMS). At
  most 16,383 shared marks, those in the most images, so a page keeps
  room for its own under the decoder's 32,767.
- optimize: the run holds the 1-bit samples of the images it stores as
  JBIG2 (16 at most, 64 MiB at most) and stores them again sharing,
  weighing each with the longest reference the globals could have.

Eight pages of 10 pt text rendered at 300 dpi take 41% less in
Helvetica and 42% in Times than coded alone, 10% in Courier, and 46%
in Helvetica at 600 dpi; groups of 2, 4, 16 and 32 save 19%, 33%, 45%
and 47%. Redacting one image stores the others of its group again
without the dictionary, so 16 keeps that to 15 images.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* M11: test what the mutants found untested in shared JBIG2 dictionaries

- filters: the group goes through encode_jbig2_shared, which refuses it
  for an image the encoder refuses; two marks in common are enough to
  share; past the 65,536 distinct marks a group holds, a mark first seen
  later isn't shared while those held count every image.
- optimize: optimizing again keeps both the shared pages and the pages
  coded alone as they are; under Retain, averaged JBIG2 pages share
  nothing; a last group of two shares.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* M11: only images redaction can store again share a JBIG2 dictionary

Redacting one image of a group stores the others again without the
dictionary, which can hold the shapes of marks under the area, and
refuses the redaction for one it can't: it takes image masks and
images of one component as its own reading counts them (no DeviceN,
the family named directly), and at most 64 Mi pixels. Optimize took
DeviceN of one colourant and up to 80 million pixels, so such an image
in a group made any redaction of the others fail, where before it was
coded alone and the same redaction worked (the Consumers review).

Only an image mask or a DeviceGray image named directly, of at most
2^26 pixels, now joins a group; others are coded alone. An engine test
optimizes three pages, two DeviceGray and one DeviceN, and redacts the
first: the DeviceN page stays alone, the second is stored again, and
no image names the dictionary after a save.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* M11: optimize and redaction share one limit on the images redaction stores again

The second Consumers review: Optimize's limit on images that share a
JBIG2 dictionary copied redaction's private 64 Mi, with nothing tying
the two, so a lower one in redaction would bring back refused
redactions with no test going red. Both now read
pdfcraft_cos::MAX_REDACTED_IMAGE_PIXELS. The engine test adds an image
mask, the other kind that shares, so a redaction that stopped storing
either kind again fails it. doc_optimize's description, the README,
the parity note and ROADMAP say which images share (named directly).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* M11: a JBIG2 group is coded before its samples would pass the limit

CodeRabbit on #5910: hold added an image to the group and then checked
SHARED_BYTES, so a group could pass the 64 MiB it is documented to stay
within by its last image, which can itself be up to 64 MiB (an image
that shares has at most 2^26 pixels, and as its rows are whole bytes,
at most that many bytes; a 1-pixel-wide one takes them all). hold_in
codes the group first when an image would take it past the limit, and
after when it has 16 images; with room for 10 bytes, images of 4 go two
to a group, where the old order made one of 12. CodeRabbit's follow-up
too: the 18-image test decodes every image and compares it with its
samples, across the boundary between the groups.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY

* optimize: count the globals' object framing; filters: test the shared coder's size limit

- A JBIG2 group shares a dictionary only when it saves more than the
  globals' dictionary and data plus NEW_OBJECT (96 bytes: the object's
  framing, its xref entry, and an encrypted file's IV and padding), so a
  group that saves less than that no longer makes the file larger.
- encode_jbig2_shared is Unsupported for an image taller than the
  decoder takes and codes one at the bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filters: drop the unused rows binding in encode_shared

page() no longer takes the rows since #5511.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filters: the shared coder's at-bound test says what it returns

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-04 13:34:08 +01:00
..
2026-10-27 08:46:08 +05:30
…

pdfcraft-engine

The façade every frontend talks to. It holds the open documents, their edit history and the tool catalogue; frontends never reach past it into the parsing, rendering or editing crates.

word_count::count provides cancellable, page-at-a-time Unicode word and character statistics with optional semantic exclusions; see word-count behavior and limits.

  • Layer: L6. It depends on most of L1–L4 and on nothing above itself. No UI toolkit — egui, winit, eframe and rfd are forbidden below L7, and xtask layers enforces it.
  • The crate denies unwrap/expect/panic!/todo!. Entry points that run untrusted-document code keep a last-resort catch_unwind guard so an escaped panic becomes an error and the user keeps their document.

Two frontends exist on top of it today: pdfcraft-ui-egui (desktop and web) and pdfcraft-automation (headless tools, the CLI and MCP). Anything one can do, the other must be able to do — see AGENTS.md §3.

When dynamic XFA layout is active, JsRunner does not initialize Acrobat JavaScript with scripts from the PDF container's /Names /JavaScript tree. Those scripts belong to the fallback PDF view and can otherwise show a misleading viewer-compatibility warning. They remain in the document and in Document::document_scripts. Requested console scripts, field actions and XFA events can still show messages; ordinary PDFs, static XFA and shells without an active layout keep their document scripts. Converting XFA to a static form and undoing that conversion switches the context with the document state.

The editing model

Each open document holds a pdfcraft_cos::Document: the object graph plus a copy-on-write overlay.

  1. An edit runs on a clone. On success the previous state is pushed onto the undo stack. Clones share all unchanged data, so a snapshot is cheap — this is what makes 100 levels of undo affordable on a 200 MB file.
  2. After every edit the working file is produced by an incremental write: the original bytes plus one appended revision.
  3. The view is refreshed from those bytes, so what you see is exactly what Save will write.
  4. Saving rebases onto the written bytes, so the next save appends only new edits.

Consequences worth preserving: the original bytes of an opened file are never rewritten in place; a full rewrite happens only where it must (redaction, PDF/A conversion, print imposition, a changed password); and unknown data survives a round trip because nothing is reconstructed that wasn't understood.

API

let mut s = Session::new();
let id = s.open("report.pdf", Some(path), bytes, None)?;   // -> DocId

s.apply(id, Edit::RotatePages { pages: vec![0], degrees: 90 })?;
let label = s.undo(id)?;                                    // "Rotate pages"
s.redo(id)?;

let doc = s.get(id).ok_or("gone")?;                         // &Document: queries, never edits
doc.can_undo(); doc.editable(); doc.allows_printing();
doc.text_blocks(page); doc.page_boxes(page); doc.security_summary();

let bytes = s.save_bytes(id)?;        // incremental: keeps signatures over earlier revisions valid
let bytes = s.save_full_bytes(id)?;   // full rewrite, garbage-collected
s.mark_saved(id, bytes.clone(), path)?;

Edit is one enum of 75 variants — every undoable change in the product, from RotatePages to AddAnnotation, SetFieldValue, ApplyRedactions and AddWatermark. Adding a feature means adding a variant and handling it in apply, which is what keeps undo, the dirty flag, autosave and the automation tools consistent for free.

Session also owns the operations that produce a new document rather than editing one: create_blank, create_from_images, create_from_text, combine, extract, split, split_by_size, reduced_bytes, optimized_bytes, print_pdf, export_data, and the signing family (sign, sign_with_timestamp, timestamp_document, embed_ltv).

Modules

Module What it holds
commands The command registry: id, label, menu, shortcut, and a Needs precondition encoding document security. Menus, the ⌘K palette, shortcuts and the control channel all read it, so every surface agrees on what exists and when it is enabled
catalog Acrobat's "All tools" information architecture as data — groups, sections, icons, hues, and an Availability of Ready, Planned(milestone) or Provider. The UI renders what is not built yet, honestly
actions PDF action dispatch (go-to, URI, named, set-layer-state, JavaScript)
links Link creation, properties, and finding URLs in page text
js The bridge to pdfcraft-js. forms never depends on js directly; the engine injects it, so a build without JavaScript is possible
xfa, ocr, compare, export Façades over the L3/L4 crates of the same names

Not done here

  • Rendering is still borrowed. pdfcraft-render wraps hayro and inspects through lopdf (ADR-0004). The engine only sees inspect() -> DocInfo, RenderPool and text::PageText, and that API stays frozen while M2 replaces what is behind it.
  • Performance budgets (misc.performance-budgets) are not set or enforced.
  • Some shipped features are still reachable only from the UI, not through an automation tool; cargo xtask parity lists them; closing that gap is tracked in the execution plan (D4).

doc.text_edit_preview(page, block) prepares a read-only page background with the selected paragraph’s glyphs removed, preserving text positioning, images and colours. It shares the working document’s render configuration and never enters the undo history.