* M7: replacing a line keeps the source font for the words it can show (#2378) (#2955)
* Fit #2955's line edit to #4514's /ActualText spaces
Since #4514, source_show wraps a run whose spaces the subset lacks in a
/Span /ActualText BDC ... EMC, so the moves standing in for those spaces
read back. #2955's two-font /PCEdLine path writes its source runs through
source_show, so they now carry that span too. Three of its tests were
written against the operator stream from before that:
- replacing_a_mixed_font_line_again_replaces_its_sequence counted BMC and
EMC as the /PCEdLine sequences alone. It now also counts each edit's
own /ActualText spans (BDC, and EMC = sequences + spans), so a sequence
that nests or survives a re-edit still fails it.
- a_leading_space_does_not_make_a_source_font_run looked for the leading
move right after the Tf; it now sits inside the run's span.
- a_row_replaced_inside_a_shared_sequence_starts_in_its_own_font put its
second row before the first EMC, which is now the span's: the row was
read as part of the span's text. It now goes before the EMC that closes
/PCEdLine, the case the test describes. A paragraph rewrite drops an
/ActualText span with its line (the span's BDC and EMC are among the
line's operators), so it never leaves a row inside one.
The code needs no change: re-editing drops the spans inside a /PCEdLine
sequence with it (BDC/EMC are in the dropped set), and a span's merged
run keeps the sequence it is in, so the line still reads back as one.
Also drops the same_font import lines.rs no longer uses.
* Tests: fit #2955's three line-edit tests to #4514's /ActualText spans
* Tests: the replaced sequence's check allows the new text's own /ActualText span
---------
Co-authored-by: mtlcl410 <2368091+mtlcl410@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
* fix(edit,a11y): font aliases share the metrics built for their font
Text extraction and heading extraction built and kept font metrics per resource name, so
many names for one font object each held a copy of its Unicode and width tables.
* fmt
* address review: font metrics are cached by the object at the end of the reference chain
- A draft with the same words (a space typed after the last) changes
nothing and is no undo step. It used to redraw the whole phrase.
- Rewriting words alone is refused when it would move any other text on
the page, visible text too, not only other OCR words.
- A cover stops at the words either side that stay: the padding of a
large word could reach over the ink of a small one beside it.
- Spacing other than one space between words goes to the whole-phrase
rewrite, which keeps it, instead of being dropped.
- A squeeze is relative to the scale the words already have.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deleting one word from a recognised line covered the whole line and
redrew it in Helvetica, though nothing but that word had changed.
A text edit of an OCR phrase now covers and redraws only the words it
changes; the rest stay as the scan shows them, still invisible text. Each
changed stretch is rewritten as a phrase of its own, on a copy of the
document, and the edit falls back to the whole-phrase rewrite when words
can't be told apart or a kept word would move. A deleted word leaves its
gap. A replacement a little too wide is squeezed across to fit, by up to
a quarter; wider than that it takes over the next word. A change of
formatting, or a typed line break, still redraws the whole phrase.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #5989. Ctrl/Cmd+A selects every object on the page; copied objects paste as page content that draws the same, within a document or into another, with organize's bounded deep copy across documents. Headless: object_copy, object_paste, object_move all.
Edit text queued Prepare grouped text for every page that came into view, after a full
preparation on a copy of the document on the UI thread to see whether it could. Scrolling cost
that work per page, marked the document changed, added undo steps and inlined shared forms
although nothing was edited.
Now nothing happens while pages are shown. A click (press and release, no drag) on grouped
artwork, routed to the picture by the page router as before, asks once whether that artwork
shows text preparation would open (text::forms::grouped_text_at, which reads the page without
copying or changing it). If it does, the page is prepared as one undoable step and the
paragraph at the point clicked opens for typing. A logo or a group preparation would refuse is
selected as before, with no message and no change. A drag still moves the selected group, and
while a paragraph is open or an edit is waiting the click selects as before. text_prepare is
unchanged.
The docs and the parity entry no longer say text is prepared on entering Edit text.
The repair reads the page's own fonts. It ran only after a group was expanded, so the same
damaged map on text that was never grouped kept reading wrong. Preparation now repairs it
whenever it finds it, recorded in the repair log as before, and PrepareText (text_prepare)
succeeds when it repaired a map even with nothing to expand. A repaired map no longer matches
the damage's fingerprint, so preparing again changes nothing.
Ungroup and text preparation now check the placement in one place, ungroup_ops, which is given
the page's operators and the Do's index:
- The form is looked up by the name the Do writes, byte for byte. Ungroup looked it up by the
picture's display name, a lossy UTF-8 copy, so a form named /F#FF was 'missing'.
- A Do with more than one operand is refused: the picture was read for its first, the renderer
draws its last.
- A Do inside BT…ET or an unfinished path is refused. Inline, the form's text objects would be
nested in the page's, and its clip would take in the page's path. Preparation already refused
these; Ungroup didn't.
text/forms/expand.rs was a drifted copy of ungroup.rs. It is gone: text preparation now opens each
placement with crate::ungroup, so Ungroup and preparation share one implementation.
What the copy added is now in ungroup.rs: a Refusal that says whether the form itself or only
this placement is the reason (intrinsic) and whether a rename alone was the problem (renaming),
the shows_text result, merges_without_renaming, refusals for unreadable /Resources and /Group
entries and arrays with a non-number, a cap on resource counts and q nesting, a total size for
pattern copies, and ending an unfinished path with the form. Ungroup's newer behaviour stays:
names are read from the operand the renderer uses, so a surplus operand is renamed correctly
rather than refused; default colour spaces are compared with the page's; and a group blending
in DeviceRGB on a page that names no blending space is drawn inline, as Ungroup does, rather
than refused as the copy did.
What a form draws is now read within one decode allowance shared with the form itself, so a
crafted group can no longer make one Ungroup decode 64 MiB for each of 1024 nested forms. A
group whose content can't be checked is refused; for a form without a group, blend modes inside
don't matter, and content that can't be checked only means preparation can't tell whether it
has text (shows_text is None), which it reports.
A text object can run on from one content stream into the next. When it is still open at a
stream damaged partway, or one that doesn't decode, the text that stream shows starts where the
text before it ends: editing that text moved what the hidden stream shows, its advance changed.
An edit in the text object before such a stream went ahead, as edits before the damage do.
`rewritten_contents` now refuses a change from the start of a text object still open at the
first stream not read whole (`changes_text_open_into_damage`); a change before that text object
goes on. `text_a_stream_not_read_whole_goes_on_from_is_not_edited` (the hidden stream damaged
from its start and one that doesn't decode, between the text and a stream closing it or last on
the page; a line in an earlier text object of the same stream is still edited).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Moving or resizing an image in a stream before one damaged partway, or one that doesn't decode,
goes on, as the edit changes no stream past it. But the image's `q cm … Q` can close past that
stream, and what the stream draws inside the scope is unknown. When the image looked alone there,
the move changed the scope's `cm` in place, and anything the stream draws in it moved too: a
square drawn there in the image's `cm` moved with the image, as on `main`.
A scope still open after such a stream now counts as one that paints something else
(`first_op_after_damage`, read by `image_scopes` and `enclosing_transform`). An operator counts
by its keyword there: a page may split its content between an operator's operands and its
keyword, so a `Q` whose operands come before the hidden stream still closes after it. The move
gives the `Do` a `cm` of its own and leaves the scope's as it was; with a clip on the image, the
clip stays, as one the stream may share. A crop goes around the `Do` alone rather than at the
scope's start, where it would clip what the stream draws. A crop PdfCraft put in such a scope
isn't removed, and the refusal says why rather than that there is no crop. Arrange is
unchanged: it takes out the scope it finds, which changes the stream past it and is refused.
`an_image_whose_scope_closes_past_a_stream_not_read_whole_moves_alone`: a stream that doesn't
decode and one damaged, each also with the square's operands before what is hidden (ending the
stream before, or in the damaged stream before its damage); the image with no clip, a clip, and a
PdfCraft crop; moved, cropped, and its crop removed. A scope that closes in a damaged stream
before its damage still takes the move in its `cm`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Review follow-ups:
- The phrase's median counts each word once: neighbouring words OCR
sized alike share a run, so 15, 9 and 9 pt gave 12 pt, not 9.
- The styled rewrite's width calibration is not clamped again after the
phrase ratio, so a word sized at more than twice the phrase keeps the
width the scan shows it at.
- The single-style rewrite (a new family or bold) scales the first
word's Tz by its own size over the phrase's, so the line is as wide
whatever size OCR guessed for that word.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Recognize text sizes each word from its own box and a guess at how much
of the em its letters fill, so one phrase's words come out several
points apart (9 to 15.6 pt on a LaTeX scan). Editing a phrase drew and
saved every word at its own size.
An invisible block's size is now the median of its words' sizes
(TextBlock::phrase_size). The styled rewrite draws every run at it and
scales each word's width calibration by its own size over the phrase's,
so words keep the width the scan shows them at; the single-style
rewrite (a new family or bold) uses it in place of the first word's
size; and the inline editor lays each word out at it while typing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An edit was refused after a content stream damaged partway, since what the damage hides can
change how the later streams read, the graphics state they draw in among it. A stream that
doesn't decode at all hides all of its operators, and an edit after one went ahead. It now stops
there too: text lines and blocks, and images (arrange, crop, transform, move), as selecting
objects on `main` already does for a page with such a stream. An edit in a stream before it goes
on, and the stream is kept as stored.
A file may give a stream's /Filter by reference, which isn't read here, so the case needs no
damage: `content_after_a_stream_that_does_not_decode_is_not_edited` has such a stream move the
text and image after it with a `cm`. `a_content_stream_that_does_not_decode_is_kept_and_edited_around`
becomes `..._kept_and_edited_before`, its edit now in the stream before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
A page's content streams are read as one stream, so what a stream
damaged partway hides past the damage is unknown to the streams after
it too. A string or comment the damage leaves open runs on into the
next stream, and operators the damage hides can change the graphics
state the later streams draw in. Marking a tag's content as an artifact
took what read as a BDC in a later stream for the tag's own, and could
rewrite bytes inside a string. Moving an image after the damage worked
the move out without a cm the damage may hide.
Now an edit refuses a change to any stream after one damaged partway
(rewritten_contents), and an image isn't arranged into a new stream
placed after one. Marking an artifact or a decorative figure refuses
content that reads as the tag's in a stream after the damage. Edits and
marking before the damage go on as before.
A content stream that doesn't decode at all was read as empty and
whole. A styled edit whose paragraph marker closes past the page's last
operator appended the close to it, so the edit replaced that stream
with one holding only the close. Edits now refuse a change to such a
stream, and still go on around it. Marking an artifact or a decorative
figure whose content isn't found takes it as possibly in such a stream,
as it does for content past the damage, and refuses rather than take it
for absent.
Tests: an_edit_never_rewrites_a_content_stream_damaged_partway (an edit
after the damage refused, one before it made),
text_a_damaged_stream_runs_on_into_is_not_edited,
images_after_a_content_stream_damaged_partway_are_not_moved,
a_content_stream_that_does_not_decode_is_never_written_over,
a_tags_content_after_a_damaged_stream_is_not_rewritten,
a_tags_content_not_found_beside_a_stream_that_does_not_decode_is_not_taken_for_absent,
and a_tags_content_in_a_stream_damaged_partway_is_not_rewritten now
checks the damaged stream itself and marks content before the damage.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
`decode_with_damage` and `decode_contents_with_damage` read a stream's
`/DecodeParms` only when direct, as `Stream::decoded` does, so content behind
a predictor whose parameters, or a number in them, are an indirect object
decoded with the predictor's tag bytes left in, and was taken as whole. Edit
then listed and wrote back the wrong bytes, and pruning dropped an image such
a form draws. ISO 32000-2 lets those values be indirect.
- `decode_with_damage` takes the document and reads the parameters through
it, and so does `decode_contents_with_damage`, and `decode_contents` with
it: printing, accessibility's reading of a page and a page copied as a form
read them too. Code that calls `decode` on a stream itself doesn't yet.
- It reads through `Document::decoding_view`: `with_direct_parms`, but a copy
of the stream holds only its `/Filter` and the parameters, its data shared,
so a stream a page's `/Contents` lists again and again isn't copied whole
for each.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
A Flate stream damaged partway decodes to what came before the damage, and
`DecodeBudget::decode` gave that back as the stream's data: the decoder's
partial flag was dropped in `Stream::decoded_within_counted`. Editing a text
line, a block, an image or an added item in such a stream wrote it again from
that part, so everything after the damage was lost on save, though other
readers may still read it.
- `DecodeBudget::decode_with_damage` decodes as `decode_within` does and says
whether the stream is damaged partway. A missing checksum or end-of-data
marker is no damage, as before.
- Edit reads the page's streams with it (`ContentStream`), and
`rewritten_contents` refuses, before adding anything, a change to a damaged
stream. Changes in the page's other streams go on, and the damaged one stays
as stored.
- Selecting objects (`editable_objects`, moving added items) refuses a page
with a damaged stream, as it refuses one that doesn't decode: what follows
the damage could change every later item's graphics state.
- Marking a tag as an artifact or a figure as decorative refuses when the
tag's content is in a damaged stream (`ArtifactError::Damaged`,
`AltError::Damaged`), and changes nothing, the structure tree included.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Change:
- Read and count only the placements a preparation pass newly inlines. The
others were read on an earlier pass in the context they still have, so a
group refused where it is drawn (translucent, for example) is no longer
counted again on every pass: 33 such placements beside one text group used
to reach the 64-placement limit on the second pass and prepare nothing. A
form refused in one place is still read where an expanded parent draws it
anew. If the page doesn't parse back to the operators written, every
placement is read again, as before.
- Read a group refused because its resources needed renaming again, without
counting it, once names merged in for other groups make the renaming
unnecessary (checked from its resource dictionaries alone, without decoding
its content), and drop its earlier refusal when it is then prepared.
Validation: edit 470 and the 2 automation text_prepare tests passed;
formatting, strict Clippy for pdfcraft-edit, layers and assets passed. The
wasm check was not run locally (the wasm32 target is not installed), nor were
the engine suite and the workspace sweep. Each new regression failed with its
fix disabled.
Change:
- Leave a form found to be artwork, or refused for a reason in the form
itself, out of later preparation passes, so a page with many logos or
unsupported groups no longer reaches the 64-placement limit and prepares
nothing. ungroup_ops now marks each refusal as intrinsic or contextual;
contextual ones (translucency, the resources it would merge into, pattern
placement, the decode budget) are re-evaluated where the form appears, so
a form refused in one place is still prepared where a parent draws it.
- Refuse a page-level Do with an operand too many for that placement only:
the form found by its first operand may not be the one drawn. Check a
form's own operand counts before the nested walk, as an intrinsic refusal.
- Use the Windows PUA alternates only when every kept alternate whose code
the PDF's map names a Thai character names that alternate's base
character, so alternates in another order that the map reveals are refused.
Chores:
- Say that the font program confirms each recovered glyph's advance class,
not its identity, and document the alternate check.
Validation: with and without craft-fonts, fonts 105 (+3 build-input), edit
468, engine 482 (+28 integration) and automation 50 + 352 passed; editor flows
248 passed. Formatting, strict Clippy, layers, assets and parity passed; the
wasm check was not run locally. Each new regression failed with its fix
reverted. Real-PDF edits on Royal Gazette 26288 kept their source fonts.
Add:
- Report text grouped more than eight levels deep instead of leaving it
silently grouped, and refuse forms whose resource-naming operators carry
surplus operands, which would otherwise lose nested artwork when inlined.
- Return the applied preparation's own counts from Session::prepare_text, so
text_prepare no longer reports a separate preview's.
- Shape a THSarabunPSK program that has its own Thai mark positioning by that
layout, in its own codes. A span the subsets can't show in place is shaped
whole, Latin rows included, by a complete program of the same face and style,
such as a CFF face from Load font.
Change:
- Expand only forms that show text, directly or through nested forms; image-
and vector-only forms stay grouped and uncounted. Match a Do's raw resource
name bytes, and refuse RGB-blended groups on pages naming no blending space.
- Check the recovered THSarabunPSK map and the Windows PUA alternates glyph by
glyph against the subset's advances, leave dropped glyphs unmapped, and
insert the repair after the last endcmap.
- Shape a Thai run split across same-face subsets cluster by cluster and skip
subset glyphs that are empty. Take the legacy path only for programs in the
legacy glyph order, needing only the glyphs the new text uses.
- Require GPOS mark/mkmk in the script and language system HarfRust selects.
- Match same-face candidates by weight and slant from the name, descriptor
weight or a non-zero ItalicAngle; the Italic flag alone is ignored, as
exporters set it on upright faces. Never complete bold or italic text with
the Regular bundled Thai face.
- Name the missing character, or the unavailable builtin:thai choice, in errors.
Chores:
- Document text_prepare's fields, builtin:thai, the app's refusal reporting
limits and the new routing; extend the text-edit parity evidence. Tests that
need craft-fonts now say they skipped instead of returning silently.
Validation: with and without craft-fonts, fonts 105 (+3 build-input), edit
465, engine 482 (+28 integration) and automation 50 + 352 passed; editor flows
248 passed. Formatting, strict Clippy, layers, assets and parity passed; the
wasm check was not run locally. Each new regression failed with its fix
reverted. Real-PDF edits on Royal Gazette 26288 kept their source fonts.
Add:
- Prepare supported Form placements as original page operators through the
existing-text UI and text_prepare, with shared-form isolation and undo/redo.
- Repair narrowly identified Thai exporter maps on copied page fonts, recording
repairs without changing the embedded source glyph programs.
Change:
- Reuse legacy Thai Regular/Bold glyph codes and same-face punctuation subsets;
retain point size, spacing, horizontal scale, rise, colour and paint settings.
- Keep cross-family Thai replacement explicit and write matching bundled Thai
subsets through the existing bounded outline path rather than static embedding.
- Keep replacement-only menu choices out of the source-font preview atlas so
opening an ordinary paragraph does not wait an extra frame for unused fonts.
- Refuse unsafe compositing, tagged or malformed groups, and missing contextual
glyphs without losing the user's document or changing other placements.
Chores:
- Document the editing limits and headless workflow; keep text-edit parity partial.
Builds on dekkeng's Thai fallback work in #2811 (fba0a29a) and mtlcl410's Form
Ungroup implementation in #3067 (6eafca9c), adapted to the current module layout.
The Ungroup menu and older UI font-priority changes are not included.
Refs #3185, #2964 and #2142. Synthetic regressions cover source-font preservation,
shared forms, unsafe refusals, layout, undo/redo, rendering and save/reopen.
Validation: library suites 1,675 passed; automation integration 331 passed;
document-font integration 2 passed; editor flows 248 passed. Existing ignored
tests remain. Formatting, scoped strict Clippy, layers, assets, parity and wasm
checks passed. The unchanged opacity regression failed before the choice-only
atlas correction and passed afterwards.
Second review of #5075, five findings, each with a case seen failing on
the previous commit:
- A path of several rectangles clipped to the box around them, so text
in the gap between them was read though nothing of it shows. A clip is
now the union of its parts, one convex region per rectangle; clips
intersect part by part (a path of more than 16 keeps the box around
them, which still holds them all).
- A form's /BBox one point by 1e-10 fell under a fixed area threshold
and was taken for no box. Whether a region has area is now its area
against the square of its perimeter, the same at any scale.
- At 1e12 points from the origin the shoelace products lost the area to
rounding, and the box was dropped. The area is measured from the
first corner.
- Clips meeting only along an edge intersected to a region of no area
that a run across the edge still showed through; an intersection with
no area, or with an empty region on either side, is now empty.
- A clip of no area was taken for no clip, and everything under it was
read. It has no inside (ISO 32000-2 8.5.4), so it now hides all; only
a clip whose corners or measure are not finite is left unread.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #5075, four findings, each with a test seen failing first:
- A /BBox with anything but four numbers is no bound. filter_map read
[0 /Bad 0 50 50] as [0 0 50 50], invented a clip and hid real text.
- A clipping path of several rectangles clipped to the last one only,
hiding text seen through the others. It now clips to the box around
them all, which holds every one of them, so nothing they show is
lost; a path with an unreadable rectangle clips nothing.
- Clips in force were each tested on their own, so a run crossing two
clips that lie apart (a page clip and a form's /BBox, or nested
forms' /BBoxes) overlapped each and was read though nothing of it
shows. Clips now intersect.
- A turned clip was reduced to its upright bounding box, whose corners
it does not cover, so text there was read. A clip is now the convex
region the rectangle's corners make under its matrix.
So the graphics state's clip becomes a convex Region (page space). A
form's /BBox is intersected into the clip it inherits at its Do (its
content starts with a stack of its own, so it cannot lift it), which
replaces the separate stack of form boxes. Regions intersect by
Sutherland-Hodgman and a run shows when the separating-axis test finds
no gap; a region past 32 corners becomes its bounding box, so a stream
that sets clip after clip stays bounded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Extends the form-/BBox clip from the previous commit: a page (or a
form) can clip with a plain content-stream re/W/n too (ISO 32000-2
§8.5.4), not only through a form's /BBox. The renderer already honours
both; Office export honoured neither until the previous commit, and
still missed this one.
Ts gains a clip field (the active rectangular clip, page space),
tracked by the existing q/Q save/restore (Ts already saves and
restores through there, so the clip needs no extra plumbing for that).
reading_lines recognizes the re ... W/W* ... n idiom and narrows the
clip when it completes; a form drawn inside an active clip inherits
it, on top of its own /BBox.
text_lines/text_blocks (the interactive editor) call the same
interpret() but consume its Shown records through their own separate
loop in text_lines_in, which never looks at state.clip -- this stays
scoped to reading_lines/reading_blocks (Office export, and whatever
else reads a page the way a reader sees it), not editing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Export to Word/HTML/PowerPoint/RTF reads a page's text through
reading_lines/reading_blocks, which walks form XObjects without any
notion of a clip. So text a form's /BBox clips away -- never drawn --
was still read and exported, while Find, text extraction and the
renderer already honour it.
Give reading_lines a stack of the active forms' /BBox (page space,
under each form's own matrix): a run whose box falls outside any of
them is not added to the page's lines. Nested forms each check their
own box independently, which is equivalent to intersecting them for
deciding whether a run is outside at least one.
description both say Office export is separate. This covers only a
form's own /BBox, not arbitrary re W n page clips or curved clips,
matching the scope #3628 itself drew for the renderer's own
extraction device.
hostile_form_xobjects_are_read_once used a 9x9 placeholder /BBox while
drawing its own test text far outside it; the fix correctly clips that
away. Widened the box to one that holds the test's text -- the test is
about loop and resource handling, not clipping.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* M10.20: plan Unicode text for OCR output in Type3 faces (fonts, edit)
OCR output used to flatten every character beyond WinAnsi to a silent `?`. This
adds the two pieces that let the OCR layers write the recognized words as they
are; the OCR code that uses them follows in M10.21.
pdfcraft-fonts
- `TextPlan::for_words` maps each word to the standard Helvetica when WinAnsi
covers it, else to Type3 fonts drawn from the craft-fonts faces (the Japanese
document face first, then the rest of the build input). Characters no face
has a glyph for are counted (`missing`), never dropped silently. At most 240
characters per font and 2400 per plan, so one hostile word cannot balloon.
- Which face has which character is asked once per (face, character) for a plan,
not once per word.
- `face_has`, `craft_glyph`, `win_ansi_char`, `fits_win_ansi` are public for it.
pdfcraft-edit
- `add_type3_fonts` / `stamp_with_fonts` register a plan's fonts on a page under
a free `PCOcr`, `PCOcr1`, ... name, build each font once and share the object
with later pages. A shared face is registered under a free name on a later
page that already has a font of the name it was given first, instead of
replacing that page's own font.
- The fonts carry real advances and a ToUnicode CMap, so extraction reads the
text back.
pdfcraft-redact
- A test shows a hand-built Type3 text layer (the shape the OCR output writes) is
redacted like any other text and the verifier passes.
Tests: pdfcraft-fonts 62 passed, pdfcraft-edit 129 passed, pdfcraft-redact 33
passed; clippy -D warnings clean on the three crates. The name-reuse and the
redaction tests build their fonts by hand, so they run without the craft-fonts
build input; the tests that plan real faces skip without it and were not run in
this commit's own check.
* M12: give the hostile-XFA open test more room on a loaded machine
The test opens a form whose calculate script sets a value 50 000 times, against
the one-second budget of an unasked script and a five-second wall-clock bound
on the open. Both sit close to what a machine running several suites at once
can take, so the test can fail without anything being wrong.
It now runs 20 000 iterations (the total is 19999) and allows 30 seconds, which
still fails for a script that hangs or is not capped. Test-only; unrelated to
the OCR work in this series, so it is a commit of its own.
Tests: both hostile_xfa tests in pdfcraft-engine pass.
* M10.21: pluggable OCR engines, preprocessing, review sessions and the OCR tools (ocr, engine, create, automation)
Scan & OCR gets a second engine, optional raster clean-up, a review step before
anything is applied, editable output and a batch mode, headlessly first: the
interface for them follows in M10.22.
pdfcraft-ocr
- A `Recognizer` trait with two engines: the built-in ocrs (moved out of lib.rs)
and the installed tesseract program, run as a fenced external process: fixed
argv, allow-listed language codes, a private temp directory (mode 0700 on
Unix), 120 s timeout, 64 MiB output cap, 50 000-word reading cap. A location
from PDFCRAFT_TESSERACT is validated (absolute, existing, executable) and an
invalid one fails closed instead of falling back to the PATH. `--list-langs`
and `--version` are read from stdout or, for older versions, stderr.
- Strategies: PrimaryOnly, ConfidenceWeighted (the second engine joins when the
first reading's mean is under 70 or unknown) and a ROVER merge (higher
confidence wins per word; against an ocrs word of unknown confidence a
tesseract word wins only at 70 or more; capped at 100 M comparisons; a failing
partner degrades to the primary with a note).
- Preprocessing: opt-in auto-rotate, deskew, denoise and binarize on the RGBA
raster, each with an exact inverse so word boxes map back to the scan. The
300 dpi normalization the steps are calibrated for runs only when a step is
on: with every step off the render is read as rendered, as before. (Enlarging
a 72 dpi render 4x on its own made the models read "due: €250" as "ue: ?250".)
`OcrImage` is validated by construction (side and 64 MP area caps, checked
before any buffer is allocated, including by the resize).
- Confidence bands (High >= 90, Medium 70-89, Low < 70) drive review.
pdfcraft-engine, pdfcraft-create
- `OcrSettings` gains engine, strategy, the four steps, output mode, region, the
skip flags and the low-confidence threshold; Automatic reads confidence
weighted unless a strategy is chosen.
- A recognition is a session: the document guard refuses a stale result with its
reason, pages can be applied one at a time (the guard is taken again after each
applied page), and applying is one undoable step. A per-language dictionary
(a plain-text file: words added from any document stay on disk after it closes).
- Output is searchable (an invisible layer) or a NEW editable document of visible
text at the recognized boxes (`create::from_contents`); words beyond WinAnsi use
the Type3 faces of M10.20 in both, with the shared plan across pages.
- Engine discovery never blocks the interface: results are cached, a stale
answer is shown while a background probe retries, and the models search and
`--version` are cached. Batch: pages or files that already have text are
skipped unless forced, and inputs sharing a file name are found up front.
pdfcraft-automation
- ocr_recognize (engine, strategy, steps, region, output mode), ocr_words (with a
retained review handle), ocr_review_apply, ocr_dictionary, ocr_status and
ocr_recognize_files. The editable `out` must be a new file inside --root and is
refused for a searchable run (it used to be ignored), two batch inputs of one
file name are refused before anything is read, and the batch folder is checked
for existing outputs.
pdfcraft-ui-egui
- Only what the new API needs: the Recognize Text dialog reads with
`recognizers(&settings)`. `ocr.correct` is registered with its label in every
catalog; the screen itself is M10.22.
Tests: pdfcraft-ocr 78 passed; pdfcraft-create passed; pdfcraft-engine and
pdfcraft-automation 450 passed and 1 failed (the automation lib 17, automation.rs
157, hostile_args 2 among them); pdfcraft-ui-egui lib + ocr + control tests 391
passed; clippy -D warnings clean on the five crates. The one failure is in a
test this commit does not change:
recognize_text_makes_a_scanned_page_searchable fails where the models are
installed (they read "quick" as "auick" at that render size); the roadmap on main
already records it. The real tesseract binary ran in
tesseract::tests::recognize_runs_the_installed_binary (tesseract 5.4.0, PATH).
* M10.22: OCR Verify screen, the Recognize Text options and the interface catalogs (ui-egui)
Scan & OCR ▸ Correct recognized text: a full-window screen where a recognition
is reviewed before anything is applied, plus the dialog and batch options for
the engines of M10.21.
OCR Verify screen
- Four panes: the page list, the source page with confidence-coloured word boxes
(click to select, drag to draw a region), the recognized text, and a zoom plus
word inspector. Corrections commit on Enter (an empty commit removes the word),
Remove/Restore, "+ Dictionary" (a per-language file in the preferences folder,
so a word added from a document stays on disk after it closes), per-word and
per-page Verified, F8 / Shift+F8 walk the low-confidence words, Ctrl+T marks the
page reviewed, Ctrl+Tab moves the pane focus, and review edits undo within the
panel.
- Re-OCR of the whole page or a drawn region. A region re-read moves the review
marks of the words it keeps to their new positions and selects the first new
word; a whole-page re-read clears that page's marks (and the undo brings them
back). The region outline is scaled onto the pane, not mapped through user
space again, so it is right on rotated and cropped pages.
- Accept / Reject per page or whole. Pages accepted one at a time each apply once
(the document guard is taken again after each applied page; an edit made
elsewhere in between is still refused with its reason), and Accept in
searchable mode is one undoable step. Editable mode writes a NEW
`<name>_ocr.pdf` through a save dialog and never the source. A real Cancel
keeps the pages already read.
- The engine and strategy menus disable a choice that cannot run and say why;
the check never waits on a process (a background probe fills a cache), and
choosing Automatic moves "Primary only" to Confidence weighted.
Recognize Text dialog and batch
- Engine, strategy, the four clean-up steps and the skip flags are remembered with
the other choices (a saved Automatic without a strategy reads confidence
weighted). The batch refuses two inputs of one file name and existing outputs
before reading anything, notes low-confidence pages at the settings' threshold,
and its toast has a Cancel (the file in flight finishes).
Catalogs and attribution
- The 90 new interface strings are in all 21 catalogs (the Lithuanian ones were
written without native review, and the Bulgarian ones avoid U+045D, which the
bundled monospace face lacks). ATTRIBUTION.toml carries the new hashes, a
note on the Lithuanian entries and the README screenshot; ATTRIBUTION.md is
regenerated.
- README: a screenshot of the Verify screen, taken with
`cargo xtask screenshots` from a synthetic scan (`shot --ocr-verify on`) built
from allowed assets; the page reads it with the real engines.
Tests: pdfcraft-ui-egui lib 373 passed (1 ignored: the Windows hinting
measurement probe), tests ocr 6, control 46, fonts 9; clippy -D warnings clean on
the engine, ui-egui, pdfcraft and xtask packages; `cargo xtask assets` ok. The
review state machine has unit tests for every fix above; each was seen failing
with its fix reverted.
* M10.23: parity entries, roadmap and localization notes for the OCR workflow
Records what M10.20-M10.22 ship, and what they do not.
parity/acrobat-features.toml
- Status changes against main: ocr.rotation-detection and ocr.deskew planned ->
shipped (the opt-in steps of M10.21, with their tests); ocr.correct-recognized-
text and ocr.ocr-engines planned -> partial. Other Area I entries get their
evidence and notes brought up to date.
- ocr.correct-recognized-text: the evidence includes the one-page-at-a-time accept
and the region re-read tests; the notes say what the headless tools cover
(ocr_words with a retained review, ocr_review_apply through the same stale-
document guard, ocr_dictionary) and what is still UI-only (suggestions,
per-word Verified marks, region re-reads).
- ocr.ocr-engines stays partial rather than shipped: part of its evidence is
conditional (the real-binary test runs only where tesseract is installed and
skips elsewhere, and the process-level tests are Unix-only). It cites the tests
that always run (location validation and the fail-closed rule, --list-langs
parsing from stdout or stderr, the merge rules, the default strategy) and says
which part is not proven everywhere.
- `cargo xtask parity`: area I 32.7% -> 44.2% weighted (5 -> 7 shipped, 7 -> 9
partial); overall 52.8% -> 53.0% shipped and 58.2% -> 58.5% weighted.
ROADMAP.md
- One progress-log entry (2026-10-13, newest first) and the banner line. It says
the catalogs are 21 with the Lithuanian additions not natively reviewed, that
the end-to-end redaction test of the Type3 layer needs the craft-fonts build
input, and that recognize_text_makes_a_scanned_page_searchable still fails where
the models are installed, as main's log already records.
docs/localization.md, docs/localization-parity.md
- The Lithuanian entry count (2,578 -> 2,668) and the counts of the other
catalogs that carry entry counts are re-measured with the 90 new entries; the
Lithuanian OCR entries are marked as not natively reviewed.
Checked: `cargo xtask parity` (numbers above, evidence tests exist) and
`cargo xtask assets` pass on this tree.
* M10.24: ocr: apply a late result only to the document it was read from
The Recognize Text dialog closes on Go, so the document can be edited, or have
text redacted, while a long run is still reading. The result was applied
without the stale check the tools and the Verify screen use, so words read
before a redaction could be written back as invisible text. The run now keeps
its guard and applies through the checked path.
Accept with the editable output in the Verify screen had the same gap (Exit
keeps the review open): it now refuses when the document changed since the
review started.
Both new tests fail with their guard switched off.
* M10.25: ocr: ocr_dictionary takes dictionary file names only
Updating sorts, lower-cases and rewrites the file it is given, so pointed at
.env or a README it would corrupt it. The tool now accepts only
ocr-dictionary-<language>.txt style names, for reading as well as writing.
The size-limit test used a file name the rule now refuses first, so it renamed
its fixture to keep testing the size limit. The new test fails with the rule
switched off.
* M10.26: pt-BR: keep (s) on the word counts and say mecanismo for engine
Two review fixes from a native speaker: the two word(s) lines had lost their
(s), so a count of 1 read as 1 palavras, and engine was translated both as
motor and mecanismo. Mecanismo is the Brazilian word for a software engine and
two lines already used it, so the other six now match.
* M10: fit the rebased OCR workflow to current main's structure
* Fix OCR workflow issues from the review
Honor the Recognize dialog's editable-output choice: its result opens in
the review screen (whose Accept writes the editable document) instead of
an invisible layer. Make page acceptance idempotent: a second Accept
entire page never appends the OCR layer twice and keeps later
corrections. Move page and region re-OCR onto a worker thread with a
status line and a cancel that discards the reading. Skip the Tesseract
probe for jobs that name their engines. Bound Tesseract's stderr while
the child runs, not only after it exits. Surface the editable output's
substitution notes in the tools' and the run's results.
Also repair the OCR batch tool against the current structure: inputs are
held to the read root again, and a batch whose inputs share an output
name is refused before the output folder is created, compared the way
the default filesystem compares names; and the editable writer test's
word stays inside the fixture's pages.
* Apply pre-review fixes to the OCR workflow
Hold the automation OCR tools to the document's copying permission, as
the other text tools are: the recognized text is left out of the
results when copying is forbidden, and the paths that carry the words
out of the document (the editable output, the review word list) are
refused. Mark the ocr_dictionary tool as overwriting its path. Close
the searchable layer's graphics state around its invisible text so the
rendering mode cannot leak into later content. Read the OCR user
dictionary folder from the portable data folder in portable mode, like
the recovery store does. Resolve the attribution manifest's leftover
conflict markers and regenerate the generated attribution. Describe the
review flow, the searchable layer's Type3 fonts and the parity evidence
references accurately, and drop an undeclared copy of the recovery
module.
* OCR: reject readings that exceed the word limit
Assisted-by: OpenAI Codex
* OCR: new documents go through the page tree builder main added
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* OCR: review notes and lints
The editable output refuses a file created during recognition. ocr_words takes
skip_text_pages and force_ocr. Automatic OCR checks the engine it will use. The
stderr test stays below the kill cap, and a failed --ocr-verify shot exits with
an error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* OCR: restore the Verify screenshot attribution, follow the moved constants
The merge dropped the screenshot row with the translation rows. The model
filename constants live in ocrs.rs now, so the license check reads them there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Attribution: the Verify screenshot entry goes after the header comment
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Attribution: the screenshot row in the table follows the manifest order
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts: images.rs (main folded image_transform_scope into
image_scopes; kept only painted_over) and text/interp.rs (gs arm now
reads params via dict_in_place, then records the soft-mask anchor).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
A soft mask is placed by the CTM in force when its `gs` runs (ISO 32000-2
11.6.5.2). The paragraph lifted over the scan re-selects the graphics state
after its own `cm`, so a mask selected before a translation landed
translated with it and could hide the new text. A re-emitted state that
carries a dictionary /SMask now records its anchor CTM and is selected
there: `cm` to the anchor, `gs`, `cm` back.
The test edits the same paragraph with and without a mask and requires the
same ink; it fails with the re-anchoring disabled (the mask lands 100 pt
right and the text vanishes). Both rewrite paths.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Each `Do` copied the page's `/XObject` dictionary, and each form its `/Resources`, `/Properties`,
`/Font` and `/ExtGState`, or the resources it was drawn with if it had none. So Office export of one
page took time in the square of what it drew: 31 s for 10,000 pictures, about 70 ms of it decoding, 30
s for 10,000 forms with no `/Resources`, and 16 minutes for one form with 100,000 keys in them drawn
50,000 times, in a test build.
`FormCall` keeps the form and `resource_dict` reads a dictionary where it is, with an object of its
own held. 10,000 pictures take about 0.15 s now, 10,000 forms 79 ms, and that form drawn 5,000 times 40 ms. The test holds to 10,000 pictures, not the 2,000 it could afford.
A `gs` reads a direct graphics state in place too, in the text walk and in `page_background`, and a
value of the wrong type isn't copied to find that out. A form that sets a font still reads the font each time it is drawn, as the fonts read
are kept only for that draw: 3.5 ms a draw for one of 100,000 widths.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Reuse the existing subset-tag parser through the engine facade and share the fallback formatting state.
Co-Authored-By: Codex GPT-6.1 <noreply@openai.com>
Dylan M. Taylor
regendorf-ux
BFlat
PerchAnton
oracola91
jonathanwxh-cellandjonathanwxh-cell
midasmatterhorn
Oliver H.
Tiago KielandClaude Opus 5.5
Grzegorz MartensandClaude Opus 5.5
xullul
xullul
Anton Pepperok
MichaelandClaude Opus 5.5
KenM76andClaude Opus 5.5
