Vendored hayro-interpret refuses an embedded font whose `post` table
version isn't 1.0, 2.0, 2.5 or 3.0 (a guard from pdf.js issue 9462).
ScanSoft PDF Create! writes the version of every font it embeds with its
16-bit halves swapped (`00 00 00 03` for 3.0), so those fonts were refused
and their text drawn with a substitute: Hebrew came out as Latin
look-alikes or not at all.
1.0, 2.0 and 3.0 with swapped halves are now accepted too; any other
unknown version is still refused. PdfCraft patch (38) in vendor/README.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
CffFontBlob::glyph_index_by_name searched the whole charset, with a string lookup per glyph, for every character code of a font (there was a TODO noting it). Build a name to glyph map on first use and share it between clones. The first glyph with a name still wins, so results are unchanged.
Fall back only when missing sample or terminator bytes cannot fit the
content-window budget, including retained header and separator overhead.
Preserve available deterministic boundaries and ordinary large streams.
Add oversized, exact-budget, overflow and large-stream regressions.
* fix(render): read each object stream's index once (vendored hayro-syntax patch 22)
`XRef::get_with` parsed an object stream's whole index of `objnum offset`
pairs again for every object it fetched from that stream, so reading all of a
stream of N objects took N² steps. Firefox keeps every object of a page in one
stream: the pdf.js corpus file bug1978317.pdf (65,542 objects in one stream)
took 37 s to draw and 40 s to extract its text, and was a known timeout in the
robustness baseline.
The parsed index is now kept per stream, beside the decoded stream data it
indexes (`ObjectStreamIndexes`), and never cleared, as that data isn't. A
malformed index is remembered as malformed; `/N` or `/First` that can't be
read (a reference that can't be followed while another stream loads) is not
remembered. An offset past `usize` is malformed instead of wrapping. A poisoned
lock is still used: entries are inserted whole.
Release build, Linux: bug1978317.pdf draws in 0.5 s (was 37 s) and extracts
its text in 1.9 s (was 40 s), and leaves the baseline's known timeouts;
40,000 pages in one stream open in 0.2 s (was 54 s).
`raster::tests::objects_in_one_large_object_stream_are_read_in_linear_time`
times out at 20 s with the fix reverted and passes in 0.2 s with it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MoEx9Nj8KC4WKDcf5YaokQ
* test(render): an object stream offset past usize is malformed, not a panic
Review follow-up on #2427. Mutation testing by mtlcl410 found that replacing
patch (22)'s `checked_add` in `read_object_stream_index` with a plain `+`
survived every test, although in a debug build that `+` panics on the very
input it guards against (and wraps in a release build). Their test pins it:
it fails with "attempt to add with overflow" under that mutant and passes on
this branch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
is_nearly_equal allows a 256th of a path unit, and stroke_path_impl
applies the matrix after path_as_rect, so a cm of 10,000 turns that
tolerance into tens of device units: a path that only nearly closes was
painted as the rectangle, with a side it does not have, and an edge off
true by that much counted as axis-aligned.
The fast path now compares coordinates exactly. Nothing that is a
rectangle loses it: a rectangle's coordinates are equal in the content
stream, not approximately equal. A path that misses takes the general
route and is painted as written, which is the safe direction.
I could not make a rendered case that distinguishes the two: this
renderer paints a 10,000x cm as a solid page whatever the line width, so
there are no pixels to compare at the scale where the tolerance shows.
The change rests on the argument above, and the three existing tests
still pin the behaviour that is measurable.
Reported by CodeRabbit on #3327.
FredApps's note on #3327: five points that are all on the bounding box's
corners still need not trace the rectangle. Three sides and a step back
along the third --
10 10 m 30 10 l 30 30 l 10 30 l 30 30 l S
-- visits every corner, so it was taken as the box and stroked with a
left side the path does not have.
path_as_rect now asks what a rectangle actually is: four distinct
points, each step from one to the next along a single axis, and the last
point back at the first (written as the fourth LineTo or as ClosePath).
That is a rectangle and nothing else is, so the corner test and its
off_corner flag are gone -- four points of an axis-aligned traversal are
necessarily the corners of their own bounding box. A degenerate path,
whose points are not distinct, now takes the general route as well,
where it painted a zero-width or zero-height rectangle before.
The vendored-patch table has the row this change was missing: without
it a re-vendor of hayro-interpret would drop the patch silently, which
is the other half of FredApps's review.
`read_inner` now reads `-+280` and `++280` as well as `--280`, but
`Number::skip` consumed one sign and then required a digit, a dot or a
further `-`. So a skip failed where a read succeeded, and `ReaderExt::skip`
puts the reader back where it started -- `Object::skip` would leave the
number in place rather than step over it. `--280` was already covered,
by the `b'-'` arm.
One more sign is consumed, and the six sign forms are asserted skippable
to the end of the token. Without this the test says `-+280: not skipped`.
A third sign is still read and skipped differently (`---280` reads as 0
and skips whole), which was already so before this branch and is left
alone.
Found in review of #3630.
UniJIS-UCS2-HW-H maps code 0x0020 to CID 231 / U+2002. The OpenType
substitute for a non-embedded Japanese CID font lacks that glyph, so
.notdef boxes were drawn. Type0Font::map_via_unicode now falls back to
the substitute's ordinary-space glyph for explicit spacing characters
its charmap lacks. Native and variation-selector glyphs keep priority;
PDF advances and Unicode text are unchanged.
Fixes#3601.
Co-Authored-By: Claude <noreply@anthropic.com>
iText's indirectPrev.pdf writes /Prev 6 0 R, and object 6, listed in the
same update, holds 470, the offset of the first section. Both readers took
the object number as the offset, so cos found no section at 6, rebuilt the
file from object headers and lost its revision list, and hayro's chain
stopped there.
When /Prev is a reference, the sections read so far (cos) or the section's
own entries (hayro, which now reads them before following /Prev) are asked
for that object first. If it holds a non-negative integer, that is the
offset; otherwise the object number still is, as the tiff2pdf files need.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NH3itd85jPuM6wNjgcGfEH
The cos reader learnt this in the previous commit, but the viewer and
the save's reopen check count pages with hayro, which still stopped at
the reference-valued /Prev: a four-page file showed one page, and the
correctly written full save was refused as a page-count change.
A number written with a doubled sign was read by the renderer's lexer as
two numbers. `--280` set the sign, found no digit before the second
`-`, and returned 0, leaving `-280` to be read next: a `cm` carrying it
got seven operands, kept the last six, and the matrix `0 0 1 0 -280 10`
hid everything drawn under it. The page rendered blank while text
extraction still reported the text, because the glyphs were there and
only the matrix was wrong.
Measured on the report's own file, at 72 dpi, counting pixels darker
than mid-grey:
double_minus.pdf before: 0 dark px (blank) after: 581, leftmost at x=121
control.pdf (-280) 581, leftmost at x=121 -- unchanged
so the two files now render identically, which is what the report asks
for and what MuPDF does.
One extra sign is absorbed and the value is negative if either sign was
a minus, matching the COS lexer since #3299 (`--280` and `+-280` are
both -280). `Number::skip` already consumed the whole token, so read and
skip had disagreed about where the number ended.
Fixes#3565. #3299 fixed the same input in the COS lexer only.
Review on #3239: a name that only numbers the glyph (C39, c251) says
nothing about the character. Reading it as the code's StandardEncoding
character invents text wherever the font's /Encoding puts the glyph
elsewhere: with /C1 at 233 it extracted O-slash for a glyph drawn as
e-acute. numbered_glyph_name and the fallback are removed; the AGL
specification's full mapping stays.
The test puts one glyph per line: the test font has none of these glyphs,
so they have no width, and on one line a later 'fi' sat on the first and
was dropped as its fake-bold copy once the guessed glyphs between them
were gone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R3niW7kVeT8mi7TpWc8eAJ
A font naming a Latvian comma-below letter by its code point (/uni0136) or by
the glyph list's name for it (/Kcedilla, /Gcedilla...) still drew nothing:
the name resolves to the cedilla name the glyph list prefers, which the
composite table does not hold. The thirteen comma-below letters are now
looked up by character, which also replaces the special case for U+021A and
U+021B. T and t with a cedilla (U+0162, U+0163) are not in that table and
keep their cedilla.
Raised by CodeRabbit on the PR.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stand-ins for the standard fonts (PDFium's Foxit fonts) have neither the
comma-below letters nor a commaaccent glyph, and the composite-letter patch
left comma accents out. A non-embedded font naming /scommaaccent therefore
drew nothing, which is the blank "si" left in Romanian XFA forms, and
/tcommaaccent drew a cedilla through the glyph list's alias to U+0163.
Romanian s/S/t/T and Latvian k/K/l/L/n/N/r/R/G with a comma below are now
built from the letter and the font's comma, scaled to three quarters and set
just under the baseline, moved back along the slant in italics. The uni021A and
uni021B names of T and t with a comma below, which the glyph list
does not name, resolve by code point. Lowercase g with its turned comma above
is still not built.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review follow-up on #3497: the editing map in crates/fonts and the vendored
renderer each read an identity /ToUnicode name as code = Unicode without
controls or lone surrogates; each now names the other, so a change to one
is made to both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LymhVaGZCWwB9HNEbLzeMM
Type0Font::new and char_code_to_unicode get doc comments, and
unicode_map's names the identity ToUnicode name. The comments and
vendor/README.md cite Acrobat's observed behaviour only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LymhVaGZCWwB9HNEbLzeMM
WSCAD (and other CAD exporters) write `/ToUnicode /Identity-H` — a name,
not a CMap stream — on composite fonts whose two-byte codes are the UTF-16
code units. Acrobat, MuPDF and pdf.js read each code as its Unicode value.
hayro resolved the name to the Identity CMap, which maps codes to CIDs only,
so such pages had no text at all: nothing could be selected, searched,
copied or edited (147 of 147 pages of a WSCAD schematic).
- vendored hayro-interpret patch 35: `Type0Font::identity_to_unicode`
- crates/fonts: the same reading for Edit text, find and redaction
(controls and lone surrogates map to nothing; simple fonts unaffected)
Tests fail without the fix: render `identity_to_unicode_names_give_text`,
fonts `identity_to_unicode_names_read_codes_as_unicode`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The end-of-data look-ahead rejected a valid EI candidate when any byte
above 0x7F lay between it and the next EI. Those bytes are the content
stream that follows the image, and content streams carry such bytes
inside strings, so no candidate qualified and the image data ran to the
end of the stream. The operator check that follows now decides alone,
bounded as before by MAX_EI_CANDIDATES.
Fixes#2456
Review on #2974: the interpreter decided with a full-resolution decode
(`decoded_raster(None)`) before the renderer decoded at its draw size, a
second decode of every downscaled image under another cache key; and an
image a preview skips for its size still cleared what was under it.
`Device::knockout_image` now lets the renderer decide, with the same
eligibility and decode size as `draw_image` (shared `image_draw_size`), so
the draw reuses that decode and a skipped image clears nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
Review on #3184: with no /BitsPerComponent the decoded image took the
codestream's precision, which for a palette image is that of the indices.
The resolved colours are decoded to 8 bits, so a 1-bit index rescaled them to
0 or 255 per channel ((64, 128, 192) became (0, 255, 255)). When the palette
is applied the samples are now reported as 8-bit; an image without a palette,
or one whose raw indices are kept for an /Indexed space, keeps its precision.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
Review on #2974. In the erase pass `set_paint` returned no clip, so a
shading fill cleared its whole path while the redraw is clipped to the
shading's /BBox (ISO 32000-2 Table 77): red outside the box stayed erased.
It now returns the shading's clip there too. And an unmasked image cleared
its placement before decoding, so an image that doesn't decode (over
MAX_IMAGE_PIXELS, damaged data) erased the group's earlier artwork and drew
nothing; the clear now happens only when the image decodes, asked only in a
knockout group (Device::in_knockout_group). The stencil case in the test now
also checks that the stencil draws.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
Since #1557 the page background was painted inside the page
transparency group, as its initial backdrop, so that a Multiply
highlight under reading colours blends with the dark background. With
document colours that made every object blend with the white paper:
a green /BM /Screen fill over an empty part of the page came out
white, and Difference or Exclusion magenta. Only Normal, Multiply,
Darken and HardLight, for which blending with white gives back the
source colour, looked right. 0.8.0 was correct.
The page's objects composite with an initially transparent backdrop
and the page group is then put on the paper (ISO 32000-2, page group).
Vendored hayro patch (18) now paints the background inside the group
only under reading colours, where #1548 needs it, and outside it
otherwise, as before #1557.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
After rebasing onto main: #3356's one-cell pattern renderer (merged) builds
a Renderer without the knockout fields this PR adds. Like the tiled cell's
renderer, a pattern cell is painted as its own artwork, so it starts with
no knockout group open.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
An image or a nested group is drawn into a layer of its own, which
covers the whole canvas (transparent where nothing was drawn), so it
can't be composited with Compose::Copy. What is under its shape is
now cleared first and the layer then composites over it:
- an unmasked image's shape is its placement rectangle: the
interpreter hands it to the device (`Device::knockout_shape`), which
clears it with a transparent Copy fill;
- a nested group's shape is the union of its elements' shapes: the
interpreter draws its content once in the device's erase mode
(`knockout_erase_wanted`, `begin_knockout_erase`,
`end_knockout_erase`), where every paint clears under its shape and
nothing is layered or drawn as an image, and then draws the group,
always in a layer of its own (drawn straight onto the knockout
group, its own elements knocked each other out).
A masked image's shape is its mask; it composites as before, which
matches MuPDF where the mask is opaque or transparent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
In a knockout group (/K true) each element composites with the group's
initial backdrop, not with the elements drawn before it (ISO 32000-2,
knockout groups). The renderer read /K (hayro-interpret patch 28) but
drew such a group like any other, so overlapping semi-transparent
shapes in it blended with each other.
hayro-interpret patch (39) tells the device a group is a knockout
group (`Device::set_knockout`, default no-op). hayro patch (19) then
draws that group's fills, strokes, glyph outlines and shadings with
Compose::Copy, which vello_cpu weights by coverage, so each replaces
what the group drew under its shape. Images and nested groups are
drawn into a group layer of their own and still composite normally:
copying a whole layer would erase everything outside it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
Review follow-up: num_components is unset both when the dictionary has no
/ColorSpace and when it has one that does not resolve. In the second case
the JP2 header's colours are the only ones available, so applying the
palette is intended; the comment and vendor/README.md now say so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
hayro-syntax decoded every JPX image with resolve_palette_indices off. That
is right when the image dictionary gives a /ColorSpace (an /Indexed space
needs the raw indices), but with no /ColorSpace the colours come from the
JP2 header (ISO 32000-2 §7.4.9) and nothing applied its palette: the
indices were drawn as a gray image. The palette is now resolved exactly
when the dictionary has no /ColorSpace.
Two unit tests decode a 4x2 palette JP2 written for the test: without a
colour space it is red/blue RGB, with /Indexed the raw indices stay.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
* M2: a JP2 image in a colour space the decoder refuses is drawn by its component count (#3306)
The veraPDF fixture's JPX header names CIEJab (enumerated colour space 19),
which hayro-jpeg2000 refuses, so the image failed and the page drew blank.
When the decoder refuses the image and its colr boxes name such a space, it is
decoded again from a copy naming greyscale, sRGB or CMYK by component count,
as MuPDF and Poppler draw it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* M2: copy the JP2 stream only when a colour enum needs replacing (#3306)
Review follow-up on #3363: a stream the decoder refused for another reason was
copied for nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* Renumber the JP2 colour-space patch (21): #3378 takes hayro-syntax (20)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* fix(parse): accept spaces between stream and its line break
Microsoft Reporting Services writes `stream \r\n`, a space before the line break.
hayro-syntax and pdfcraft-cos took the stream data to start at the space, so the
content stream failed to inflate: the page rendered blank (white, or black with
inverted colours) and an edited save corrupted the stream. Spaces and tabs are now
skipped, but only when a line break follows.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Number the stream end-of-line patch (20), as vendor/README does
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M2: a tiling pattern drawn once shows its cell at full scale (#3349)
PowerPoint writes a picture fill as a tiling pattern with /XStep 2147480100.
Fitting that step into the 3000 px cell pixmap shrank the picture below a
pixel. When what is painted can only show the first cell, the cell's /BBox is
drawn at full scale with a transparent border and padded instead of tiled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* M2: a tiling cell whose scale isn't finite keeps the old path (#3349)
Review follow-up on #3356: a /BBox 1e-40 wide passes the zero-area check and
gives an infinite cell scale, which would send non-finite geometry to the cell
renderer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* M2: a stroke's miter counts in the one-cell check (#3349)
Review follow-up on #3356: shows_one_cell took the line width as a stroke's
reach; a miter join reaches half the width times the miter limit. The reach
formula patch (9) uses is now one function, stroke_reach, used by both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* M2: a skewed stroke's reach uses the transform's largest stretch (#3349)
Review on #3356: under a skew, a direction between the axes stretches
further than either axis, so the larger axis stretch could place a miter
tip short of the next cell when it reached it, and the one-cell path left
that cell unpainted. The reach now uses the spectral norm.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Review on #3239: the Adobe Glyph List Specification's `uni` and `u` forms
take uppercase hexadecimal digits only (section 2), and its examples map
`uni20ac` to an empty string. glyph_name_to_text accepted lowercase too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
aiongg
mtlcl410andClaude Opus 5.5
narain
jonathanwxh-cell
PiohrunandClaude Opus 5.5
Hamza
NetterSilverandClaude Haiku 5.5
jpeskovandClaude
Ahmad Mujahid
wstockandClaude Opus 5.5
Tiago KielandClaude Opus 5.5
Earl Co
Lcoding15andClaude Opus 5.5
Luigi Pinna
