The end-of-data look-ahead rejected a valid EI candidate when any byte
above 0x7F lay between it and the next EI. Those bytes are the content
stream that follows the image, and content streams carry such bytes
inside strings, so no candidate qualified and the image data ran to the
end of the stream. The operator check that follows now decides alone,
bounded as before by MAX_EI_CANDIDATES.
Fixes#2456
Review on #2974: the interpreter decided with a full-resolution decode
(`decoded_raster(None)`) before the renderer decoded at its draw size, a
second decode of every downscaled image under another cache key; and an
image a preview skips for its size still cleared what was under it.
`Device::knockout_image` now lets the renderer decide, with the same
eligibility and decode size as `draw_image` (shared `image_draw_size`), so
the draw reuses that decode and a skipped image clears nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
Review on #3184: with no /BitsPerComponent the decoded image took the
codestream's precision, which for a palette image is that of the indices.
The resolved colours are decoded to 8 bits, so a 1-bit index rescaled them to
0 or 255 per channel ((64, 128, 192) became (0, 255, 255)). When the palette
is applied the samples are now reported as 8-bit; an image without a palette,
or one whose raw indices are kept for an /Indexed space, keeps its precision.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
Review on #2974. In the erase pass `set_paint` returned no clip, so a
shading fill cleared its whole path while the redraw is clipped to the
shading's /BBox (ISO 32000-2 Table 77): red outside the box stayed erased.
It now returns the shading's clip there too. And an unmasked image cleared
its placement before decoding, so an image that doesn't decode (over
MAX_IMAGE_PIXELS, damaged data) erased the group's earlier artwork and drew
nothing; the clear now happens only when the image decodes, asked only in a
knockout group (Device::in_knockout_group). The stencil case in the test now
also checks that the stencil draws.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
Since #1557 the page background was painted inside the page
transparency group, as its initial backdrop, so that a Multiply
highlight under reading colours blends with the dark background. With
document colours that made every object blend with the white paper:
a green /BM /Screen fill over an empty part of the page came out
white, and Difference or Exclusion magenta. Only Normal, Multiply,
Darken and HardLight, for which blending with white gives back the
source colour, looked right. 0.8.0 was correct.
The page's objects composite with an initially transparent backdrop
and the page group is then put on the paper (ISO 32000-2, page group).
Vendored hayro patch (18) now paints the background inside the group
only under reading colours, where #1548 needs it, and outside it
otherwise, as before #1557.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
An image or a nested group is drawn into a layer of its own, which
covers the whole canvas (transparent where nothing was drawn), so it
can't be composited with Compose::Copy. What is under its shape is
now cleared first and the layer then composites over it:
- an unmasked image's shape is its placement rectangle: the
interpreter hands it to the device (`Device::knockout_shape`), which
clears it with a transparent Copy fill;
- a nested group's shape is the union of its elements' shapes: the
interpreter draws its content once in the device's erase mode
(`knockout_erase_wanted`, `begin_knockout_erase`,
`end_knockout_erase`), where every paint clears under its shape and
nothing is layered or drawn as an image, and then draws the group,
always in a layer of its own (drawn straight onto the knockout
group, its own elements knocked each other out).
A masked image's shape is its mask; it composites as before, which
matches MuPDF where the mask is opaque or transparent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
In a knockout group (/K true) each element composites with the group's
initial backdrop, not with the elements drawn before it (ISO 32000-2,
knockout groups). The renderer read /K (hayro-interpret patch 28) but
drew such a group like any other, so overlapping semi-transparent
shapes in it blended with each other.
hayro-interpret patch (39) tells the device a group is a knockout
group (`Device::set_knockout`, default no-op). hayro patch (19) then
draws that group's fills, strokes, glyph outlines and shadings with
Compose::Copy, which vello_cpu weights by coverage, so each replaces
what the group drew under its shape. Images and nested groups are
drawn into a group layer of their own and still composite normally:
copying a whole layer would erase everything outside it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
* M2: a JP2 image in a colour space the decoder refuses is drawn by its component count (#3306)
The veraPDF fixture's JPX header names CIEJab (enumerated colour space 19),
which hayro-jpeg2000 refuses, so the image failed and the page drew blank.
When the decoder refuses the image and its colr boxes name such a space, it is
decoded again from a copy naming greyscale, sRGB or CMYK by component count,
as MuPDF and Poppler draw it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* M2: copy the JP2 stream only when a colour enum needs replacing (#3306)
Review follow-up on #3363: a stream the decoder refused for another reason was
copied for nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* Renumber the JP2 colour-space patch (21): #3378 takes hayro-syntax (20)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* fix(parse): accept spaces between stream and its line break
Microsoft Reporting Services writes `stream \r\n`, a space before the line break.
hayro-syntax and pdfcraft-cos took the stream data to start at the space, so the
content stream failed to inflate: the page rendered blank (white, or black with
inverted colours) and an edited save corrupted the stream. Spaces and tabs are now
skipped, but only when a line break follows.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Number the stream end-of-line patch (20), as vendor/README does
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M2: a tiling pattern drawn once shows its cell at full scale (#3349)
PowerPoint writes a picture fill as a tiling pattern with /XStep 2147480100.
Fitting that step into the 3000 px cell pixmap shrank the picture below a
pixel. When what is painted can only show the first cell, the cell's /BBox is
drawn at full scale with a transparent border and padded instead of tiled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* M2: a tiling cell whose scale isn't finite keeps the old path (#3349)
Review follow-up on #3356: a /BBox 1e-40 wide passes the zero-area check and
gives an infinite cell scale, which would send non-finite geometry to the cell
renderer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* M2: a stroke's miter counts in the one-cell check (#3349)
Review follow-up on #3356: shows_one_cell took the line width as a stroke's
reach; a miter join reaches half the width times the miter limit. The reach
formula patch (9) uses is now one function, stroke_reach, used by both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* M2: a skewed stroke's reach uses the transform's largest stretch (#3349)
Review on #3356: under a skew, a direction between the axes stretches
further than either axis, so the larger axis stretch could place a miter
tip short of the next cell when it reached it, and the one-cell path left
that cell unpainted. The reach now uses the spectral norm.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Review on #3239: the Adobe Glyph List Specification's `uni` and `u` forms
take uppercase hexadecimal digits only (section 2), and its examples map
`uni20ac` to an empty string. glyph_name_to_text accepted lowercase too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
Review follow-up: a red/blue regression for the centre-coverage pass. The
shared diagonal sits at x + y = 50.25 in device space (`cm` moves the mesh
down a quarter point), so pixel (20, 30) has its corner in the red triangle
and its centre in the blue one. It must stay red with either triangle
first; replacing `or_insert_with` with `insert` turns it blue when the red
triangle comes first, and the test fails.
Rebased onto main; the vendored hayro-interpret patch is renumbered 36
(41 before; 35 is taken by PR #2977).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
The mesh sampler took each pixel's colour at its corner, and only when the
corner was strictly inside a triangle. A corner on an edge two triangles
share belonged to neither, so a Type 4-7 shading drew a white seam along
every shared edge that ran through the sample grid, and dropped pixels
along the mesh's own sides (a 30 x 30 square cut along its diagonal:
812 of 900 pixels; MuPDF 900).
Such a pixel now goes to the triangle that covers its centre, edges
included (Triangle::covers), and still takes the colour at its corner, so
every pixel that was drawn keeps its colour. Vendored hayro-interpret
patch 41.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
Review follow-up. A pattern paint keeps the fill or stroke opacity it was
set with (`scn`), so resetting the graphics state's fill alpha did not stop
a pattern-filled stencil mask from being drawn at the opacity squared:
`Pattern::without_opacity` now drops it for the mask's paint (patch 35).
The renderer also painted a pattern-filled stencil mask with the stroke
paint, so an uncoloured tiling pattern drew in its stroke colour (black by
default); it now uses the fill paint.
The stencil test covers a colour fill, an uncoloured tiling pattern and a
shading pattern at ca 1, 0.5 and 0.25; each of the three changes, removed,
makes it fail. Rebased onto main; the vendored hayro-interpret patch is now
number 35 (36 was taken on main).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
hayro-interpret draws every image XObject inside a transparency group
that applies the fill opacity (ca). A stencil mask (/ImageMask true)
also takes the fill colour, with that same alpha, as its paint, so the
opacity was applied twice: at ca 0.5 a mask was drawn at 25 %, at
ca 0.25 at about 6 %. Rectangles and raster images were right.
Vendored hayro-interpret patch (36) resets the fill alpha to 1 once
the image's group is pushed, before the stencil's paint is taken; the
saved state is restored after the image. Type 3 glyph and pattern
stencils take their paint from the glyph or pattern and are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
An optional content membership dictionary may carry a visibility
expression (/VE, PDF 1.6): [/And e...], [/Or e...] or [/Not e], each e an
optional content group or a nested expression. It takes precedence
over /OCGs and /P (ISO 32000-2 8.11.2.2). hayro-interpret ignored it, so
content under such a dictionary stayed visible when its layers were
hidden in the Layers panel.
Vendored hayro-interpret patch (40) evaluates the expression, nested
up to 32 deep. A malformed expression (unknown operator, /Not without
exactly one operand, /And or /Or without any, an operand that is
neither a group nor an expression, too deep) falls back to /OCGs and
/P as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
`path_as_rect` decided a path was a rectangle from its element count and from
the four corners of its bounding box being covered. Its comment names the shape
it means to match -- one MoveTo, three LineTo, one ClosePath -- but `len() != 5`
also admits one MoveTo and four LineTo, a five-point open polyline, and
covering the corners says nothing about the points between them. Both call
sites then paint `device.draw_rect(bbox)`, so such a path was drawn as its own
bounding box.
Rendered through `pdfcraft-cli render`, an open zigzag whose ends reach the
corners of its bounding box
80 38 m 80 21.75 l 60.571 29.875 l 40 21.75 l 40 38 l S
lost its middle peak and gained a straight edge the file does not contain: no
dark pixel near the peak, and 36 along the bounding box's top. A six-point
zigzag and a real rectangle both drew correctly, which is why this shows on
some art and not the rest -- and `s` escapes it because its ClosePath makes six
elements.
A point that is not a corner now rejects the path. A rectangle still matches
whether written `m l l l` and closed or `m l l l l` back to its first corner,
since every one of its points is a corner. The same mistake also discarded a
clip of that shape, through `push_clip_path`, which this fixes too.
Found from craft-libs' ATTRIBUTION note that the shared reader centralizes the
patched Hayro trees from PdfCraft: the same flaw is in craft-libs, sent there
as storytold/craft-libs#175.
Layer options in the Layers panel header resets every layer to the state the
file's default configuration (/OCProperties /D) gives it, as Acrobat does.
Visibility is view state, so the reset doesn't mark the document changed and
doesn't enter Undo. doc_info now reports each layer's initial state.
usable() and is_catalog() checked the object number only, so a later stream
with /Root 1 1 R outranked the trailer naming 1 0 R; get_offset then found no
such object and Document Properties lost the catalog. cos resolves by number
alone, so its rank needs no change.
Test: inspect::tests::rebuilt_file_prefers_a_trailer_whose_root_is_the_catalog
The mirror of the cos change for render::inspect: a later cross-reference
stream whose /Root points at /Info replaced the earlier trailer, and Document
Properties lost the catalog. A candidate whose /Root is /Type /Catalog or has
/Pages now wins; another dictionary is still taken when nothing better is
there (FDF).
Test: inspect::tests::rebuilt_file_prefers_a_trailer_whose_root_is_the_catalog
* M4: custom document properties through the engine and the tools (#organize.custom-properties)
/Info entries beyond the ones the spec names are custom properties:
organize lists them (custom_info) and set_info accepts any such name
after Acrobat's checks (no standard name, no empty name, no spaces,
delimiters, / or #, at most 127 bytes); inspection lists them too
(DocInfo::custom); doc_info reports them under custom and doc_set_info
sets or clears them. The Custom tab in Document Properties and the XMP
mirror stay planned.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
* M4: custom document properties list the same before and after an edit
Review follow-up. The open-time reader (render/inspect.rs) now uses the
engine's rule (pdfcraft_organize::custom_info, trimmed, empty values
dropped): text strings only, NUL/whitespace trimmed, empty values left out,
and the 256 cap counting listed entries. Before, it also listed name
values (`/Dept /RnD`) and capped before filtering, so the list could change
after the first unrelated edit. An automation test opens a file with a name
value, blank, NUL-padded, empty and numeric entries and checks doc_info's
custom list is the same before and after setting the Title.
Drops the ROADMAP.md log line, which conflicted with main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
* M4: custom properties drop NUL/whitespace padding in one pass, before the cap
Review follow-up. custom_info now trims NULs and whitespace in any mix
(" \0Lab\0 " is "Lab"), leaves out values that are only padding, and does
both before taking 256, so blank entries can't crowd real ones out. It is
the one rule: the engine uses it as it is, and the open-time reader trims
the same way. Tests: 300 blank entries before a real one, and mixed
padding, in organize and through doc_info.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
ROADMAP.md, docs/roadmap.md: union of both sides' lines, each once.
parity/acrobat-features.toml: the XFA entry has both sides' evidence and notes (#2604 page areas, #2813 CLI layout).
Review follow-up: in a whole-XDP stream, a config/acrobat/acrobat7/
dynamicRender path inside xfa:datasets/xfa:data came before the real
config packet and decided the form type. The lookup now follows direct
children only: the config child of the xdp root (or the config packet
itself), then acrobat, acrobat7 and dynamicRender.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* Keep a subscript or superscript in its line when extracting text (#3174)
* Text extraction: only a glyph right beside the previous one gets the sub/superscript centre limit
A large negative gap (a small glyph drawn back under a large one) kept the wider limit.
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
ROADMAP.md, docs/roadmap.md: union of both sides' log lines and milestone rows, each line once.
parity/acrobat-features.toml, crates/xfdf/README.md: next's text, a superset of main's (#1917, #2686 plus #2688).
crates/edit/src/tests.rs: both sides' new tests (#2558 and #3084).
crates/ui-egui/src/editing.rs: the picked picture is cleared on every edit, and a new drawing commits its raster (#3149).
A simple font without a usable /ToUnicode reads its text from glyph
names, and a name missing from the glyph list dropped the glyph from the
extracted text while the page still drew. An 821-page book set in Type1C
fonts named C39, C40, ... extracted 4 KB of front matter and nothing else.
- glyph_name_to_text applies the Adobe Glyph List Specification's full
mapping: drop from the first '.', split on '_', map each component
through the list or as uniXXXX / uXXXX. f_i, f_l and hyphen.case now
read as fi, fl and '-'. Type 1, CFF, standard and embedded TrueType
simple fonts use it.
- A Type 1 / CFF / standard font name that only numbers the glyph (one
or two letters then digits, ZapfDingbats' aN excepted) reads as the
code's StandardEncoding character, so C174 is the fi ligature and C208
an em dash.
The book now extracts 1.9 MB (pdftotext: 1.85 MB), with fi and em dashes
where pdftotext writes Latin-1 mojibake.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
Document Properties are read through the inspector: the lazy cos
reader refuses a file that needed repair, and lopdf's rebuild only
looked for the trailer keyword, so a file whose only trailer is a
cross-reference stream failed to load there and every field showed
empty. lopdf's rebuild now takes a /Type /XRef stream dictionary as
the trailer, the latest in the file winning as a later trailer
keyword would.
Review follow-up. The config packet is decoded as UTF-16 (byte order mark,
or `<` written as two bytes) or UTF-8 before the scan, as crates/xfa reads
packets. The scan walks tags, skipping comments, CDATA sections, processing
instructions and declarations, and takes the value only at
config/acrobat/acrobat7/dynamicRender, so the same name in form data, in a
comment or elsewhere in the config no longer counts.
The engine test now adds the element to the fixture's own config packet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
An XFA form with no AcroForm fields was always classified as dynamic, so
opening it laid the template out into new pages and appended them as a
revision: the pages the user saw, and any later save, were PdfCraft's
layout rather than the file's own pages, with nothing saying so. Adobe
Designer writes <dynamicRender>forbidden</dynamicRender> in the config
packet of a static PDF form; such a form is now static and its pages are
kept. /NeedsRendering still makes a form dynamic, and with no word from
the config a form without fields stays dynamic as before.
The config packet is found in an /XFA array or a whole-XDP stream, read
with a 16 MiB decode cap, and scanned for the one element.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
Seven tests failed at random on slow or busy machines, main included
(FreeBSD run 38834256439 at 5fdd820b). They now count the work or wait
for the state they check; no assertion is loosened and nothing retries.
- spot_colour_images_run_their_tint_transform_once_per_sample_value
counts tint-transform runs (exactly 256, not one per pixel) through
vendored hayro-interpret patch (34), color::tint_transform_runs, a
thread-local counter, instead of a 10 s deadline.
- shadings_with_large_sampled_functions_paint_quickly: no counter tells
the flat table from the hash map (both read every sample per build),
so the build each paint does is timed against reading its samples on
the same thread, fastest of 8 rounds each, and must stay under 10x
(flat table 1.1x, hash map 109x), instead of a 10 s deadline.
- page_contents_and_nested_forms_share_one_budget: the budget is
checked by its warnings; the deadline is only a hang guard, now 600 s.
- The pen-stroke tests hold finished renders back from the view
(DocView::hold_rasters, for tests) while they look at the settling
stroke, then wait by state for its raster.
- files_and_quit_from_the_operating_system waits for the tab before
deleting the file the open worker reads.
- image_pixels_follow_the_frame_while_a_corner_is_dragged waits for the
page raster on screen before each drag.
Product fix (#1858): a released drawing stayed painted only until any
raster of its page arrived. In full screen its edit is applied a frame
later, so a raster already under way could end the preview first. A
settling drawing is now marked committed when its AddAnnotation lands,
only rasters after that end its preview, and a failed edit drops it.
Patch 38 now also replaces the /ColorSpace of a shading that is
DeviceGray, DeviceRGB or DeviceCMYK with the default for it, whether the
shading is painted by sh or as a shading pattern. Poppler and MuPDF draw
these in the default too. The shading's cache key includes the default,
as the image's does.
The forms test also covers a form whose /ColorSpace has no default: it
draws in its page's, as in Poppler and MuPDF.
Luigi Pinna
mtlcl410andClaude Opus 5.5
jonathanwxh-cell
Hamza
Marco Arnulfo
wstock
aiongg
MarekSpodBiedryandBrandon Thomas
Guillem Hermida
NetterSilverandClaude Haiku 5.5
Guillem Hermida