* perf(io): build JPEG/PNG tiles straight from the decoded rows, in parallel (#10259)
Surface::try_write_rows_with fills tiles row by row on all cores; image_surface
converts from the borrowed decoded rows into it (no per-band copy, no staging
Image). Opaque sources skip the prune scan. import_flat_with checks cancel after
the decode and the tile build and reports progress.
* fix(io): store an F16 target as F32 so its opaque alpha is 1.0
An image without alpha converted to an F16 target got the alpha 0.0: the
opaque-alpha path copied two bytes of 1.0f32. The surface keeps 16-bit float
as 32-bit float anyway, so image_surface now converts an F16 target to F32.
* Format fused tile import integration
* perf(io): decode PNG rows straight into the layer's tiles (#10259)
* test(codecs): PngSuite rows test skips files without a PNG signature (#10259)
* perf(io): stream PNG rows only for images of 32 MiB or more (#10259)
* Update crates/codecs/src/codecs/png.rs
Co-authored-by: aiongg <67543387+aiongg@users.noreply.github.com>
* fix(io): name the image, not the failed reservation, in PNG/JPEG allocation errors (#10259)
A failed surface reservation is one tile or row band, so reporting its bytes as
"a N MB image" was misleading (usually "0 MB"). Report the image's pixel size.
* refactor(codecs): the caller picks the PNG row-group height (#10259)
decode_png_rows takes the rows per group instead of a fixed 256, and io
passes TILE_SIZE, so the groups can't drift from the tile rows they fill.
A group of 0 counts as 1.
* Complete PNG streaming integration with upstream TIFF imports
* Preserve F16 conversion and route default imports through interrupt wrapper
* fix(io): check free memory before PNG tiles, and cancel TIFF imports after the decode (#10259)
import_png_streamed refuses an image whose tiles and one row group the machine
can't hold before the first tile exists, as tiff_to_document does; the
whole-image path (the base PNG path among it) checks the layer's tiles before
allocating them, for images above the streaming size, so a small image still
opens where the reserve alone would have refused it.
import_flat_with now honours the interrupt after a successful TIFF import, as
the whole-image path does; a decode error is still reported as itself.
* fix(io): pass lazy memory probe to PNG importer
* fix(io): handle scratch read errors during flat import
* test(io): respect early TIFF import cancellation
* fix(io): thread ImportOptions decode limits into PNG row streaming
After merging main, the whole-image open path respects
ImportOptions::flat_max_pixel_bytes via decode_limits, but the
streamed PNG path still used DecodeOptions::default(). Pass opts
through so both paths share the same decoder limits.
* fix(io): silence clippy needless_update on DecodeOptions; rustfmt
DecodeOptions only has limits + keep_orientation, so specifying both
and ..Default::default() trips -D clippy::needless_update. Also apply
rustfmt to the decode_png_rows call that gate's fmt check rejected.
---------
Co-authored-by: aiongg <67543387+aiongg@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Co-authored-by: FredApps <fred@apps.local>
Engine: droplet dependency capture skips disabled steps; a nested Batch
that was cancelled or could not write its error log fails the caller;
runDroplet forwards overrideOpen; actions.rename follows named Batch
steps; Batch steps inside a script count against MAX_SCRIPT_STEPS;
output claims compare destination identity; Save and Close never
replaces a read-only source. ATN: Md must be a BlnM enumeration,
adjustment-layer names must round-trip, Calculations exports a missing
blending as Multiply, Apply Image refuses unnamed-layer transparency.
Filters: Shape Blur records its resolved geometry, reserves its kernel
memory first, and reports a preparation failure as an error, not a
cancel; Crop keeps an artboard preset when the size is unchanged and
fits the board on a turned crop.
Codecs/cms/algo: zero-axis FITS image HDUs are empty; a GIF walk that
breaks off before the trailer fails closed; two-value .3dl shaper rows
are applied; fragmented custom kernels are refused and their work polls
cancellation.
UI: Conditional branches play through panel playback; paused actions
refuse to resume after being replaced; Playback Options OK validates
its fields; transform double-click ignores a hidden pivot; path boxes
snap to a stationary selection; modal blocking uses the effective tool;
a picker-consumed release clears snap lines and ends a Move drag; the
Crop preset control shows Selection and checks only it; Crop box undo
restores frame_from_selection; Bosnian fill labels.
Tests: timing asserts moved to an ignored test, plus the review's
coverage nitpicks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* perf(io): build JPEG/PNG tiles straight from the decoded rows, in parallel (#10259)
Surface::try_write_rows_with fills tiles row by row on all cores; image_surface
converts from the borrowed decoded rows into it (no per-band copy, no staging
Image). Opaque sources skip the prune scan. import_flat_with checks cancel after
the decode and the tile build and reports progress.
* fix(io): store an F16 target as F32 so its opaque alpha is 1.0
An image without alpha converted to an F16 target got the alpha 0.0: the
opaque-alpha path copied two bytes of 1.0f32. The surface keeps 16-bit float
as 32-bit float anyway, so image_surface now converts an F16 target to F32.
* Format fused tile import integration
* Route default flat imports through interrupt wrapper
* Remove unused flat import cancellation wrapper
* Exercise options import API in cancellation tests
* Stop flat image tile construction on cancellation
---------
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
* codecs: admit animated GIF screens, sparse RLE BMPs and icon PNG profiles
- Animated GIF decoding (decode_gif_frames) skipped the logical-screen
plausibility check the first-frame decode applies, although the
backend initializes the whole screen before reading a frame. Both now
share gif_plausible().
- RLE BMPs skipped the file-size plausibility check entirely, so a
header-only file could make the decoder allocate and clear a
multi-gigabyte canvas. Like a GIF's screen, 256 MiB of canvas is now
free of data and the rest must be backed by the file.
- The image crate's ICO decoder opens an embedded PNG with no limits,
and png inflates an iCCP profile while the decoder is built. Icon
PNG profiles are now inflated into a sink first and refused past
255 x 65519 bytes (or max_alloc), with a 64 MiB total across entries.
* address review: visit each ICO profile chunk offset once
* codecs: charge dropped PNG text against the metadata budget
read_text_metadata debited the shared text allowance only for text it
kept. A compressed iTXt chunk that inflated up to the whole remaining
allowance and then failed UTF-8 validation (or hit a corrupt or
truncated stream) was dropped without being charged, so every further
such chunk could repeat the same inflation. bounded_text now reports
the bytes it produced and the caller charges them whether or not the
text is kept.
* codecs: charge the output a failing text decompress wrote
* address review: charge Latin-1 zTXt error batch at UTF-8 length
* codecs: leave Extended XMP fragments out of zune-jpeg's header parse
zune-jpeg 0.5 keeps every Extended XMP APP1 fragment until its GUID's
set is complete, and re-sorts and re-checks the whole pending list
after every marker it parses. A file with many small, consistent but
incomplete fragments (or a few followed by many other markers) costs
quadratic time before any pixel is decoded.
Nothing in PhotoCraft reads zune's reassembled Extended XMP (metadata
comes from scan_metadata over the original bytes), so the decoder now
hands zune a copy without those segments when a file has any. Files
without Extended XMP are passed through unchanged.
* codecs: reserve only the bytes kept after dropping Extended XMP segments
Offer TIFF compression, image pyramid and transparency options on Save As and Save a Copy, with matching CLI and automation parameters.
Follow-up: preserve the unsaved state when a copy discards layers over the document's own file, and check the other export formats for the same issue.
Closes#8827.
The exr crate sizes a preview attribute's pixel buffer from its
declared dimensions (up to 4 MiB at once, zero-filled) before it finds
the attribute too short, and in the non-pedantic mode every reader here
uses, it then skips the attribute and continues. A header with
thousands of tiny malformed previews therefore cost thousands of 4 MiB
allocations before any size check ran.
check_previews walks the raw headers and refuses a preview whose
declared width x height x 4 exceeds its value size. It runs before
every header read: read_meta, read_layer, the Cryptomatte listing and
decode_deep_exr.
Resolve conflicts with #5331 (EXR streaming): the block-by-block EXR import now
reads from the same identity-verified handle as the rest of the open path, and
docs/scratch-disks.md keeps both the EXR streaming pointer and the RAM/scratch
admission notes.
Resolve against main's save restructure (Services::save_file/SaveCtl, scratch-integrity checks,
ZipWriter::with_entries, exact encoded_len):
- PSD/PSB saves stream the file model through atomic_write_stream_progress; the length used for
progress and reservations is main's exact encoded_len (the PR's size_bound under-counted lazy
data); export_with/encode_with share checked_stages, so main's scratch checks (previews
included) still refuse unreadable scratch.
- Drop ZipWriter::for_entries in favour of main's with_entries.
- TIFF: take_from_psd moves layer data out of the export's own model (never the document);
body() reserves its exact length, failures are OutOfMemory (not LimitExceeded, which would
trigger the PSB retry).
- Streaming writer: AlreadyExists from the writer is a failed save, not a name collision.
- Remove dead Vec put_widened_rle; channel_data_len is test-only.
- import_file_for_open_with decodes single-part flat scanline EXR block by block into
tiles; headers the band decoder refuses fall back to the whole-file decoder, so files
main opened still open. Band-time failures and cancellation publish no document.
- The RAM tile budget and scratch-disk checks live in exr_native (the PSD stream module
they came from is dropped); over-budget opens fail with IoError::OutOfMemory.
- The band decoder picks colour channels with exr_color_channels, like the byte decoder.
- Tests move into the per-crate it binaries and use import_file; an engine open-job test
covers the OpenSource::Path route. The ui-egui sync open test is dropped: main's
synchronous open reads bytes, and the io test covers the oversized header.
- docs/streaming-io.md, roadmap, scratch-disks and REL-212-1 describe EXR only.
Main already opens PSD/PSB from the file in bands (4915f0970, #5106) through
psd_to_document_owned, which keeps damaged-file flagging, failed-layer drops,
type-layer setup and the PSD pixel budget; PSD limits are handled in #10778.
This drops the second PSD/PSB stream reader (psd::stream, io::psd_stream),
the seekable PSB writer and atomic stream helpers, the authorized-workspace
seekable open plumbing and the working-memory pre-check that refused
ordinary exports. The flat scanline EXR band decoder and its tests stay and
are wired into main's path open after merging main.
Keep the PSD memory admission work and upstream's scratch storage
together. Conflicts resolved:
- Cargo.lock: both sysinfo versions are required. 0.40.1 serves
photocraft-io's adaptive import budget; 0.37.2 serves
photocraft-raster's scratch volume and memory supervision. Cargo
metadata resolved the lock with no other changes.
- crates/engine/src/lib.rs: Session keeps pending_preflights and
next_pending_preflight beside scratch_runtime and scratch_epoch.
Session derives Default, so neither set needs a manual initializer.
Upstream: 655295b2a2
Main now takes the manifest envelope from craft-pcraft-format, which has
no full_composite member. The save writes the member after the envelope
when it writes composite/full.png, and read_full_composite reads it back.
* TIFF: open YCbCr images as RGB (#10316)
Scanners often write JPEG-compressed YCbCr TIFFs with 2x2 chroma
subsampling. The tiff crate decodes and upsamples them but returns raw
YCbCr samples, which the decoder rejected with "colour type YCbCr(8)".
The new tiff/ycbcr.rs converts them to RGB after decoding: JPEG data
uses the JFIF full-range BT.601 conversion (as libtiff does), other
compressions honour ReferenceBlackWhite and YCbCrCoefficients with the
TIFF defaults as fallback. 8 and 16-bit. YCbCr with extra samples, or
in other sample formats, fails with a clear unsupported error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Contributors: add Yuki-init's display name
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* TIFF YCbCr: test that YCbCrCoefficients set the luma weights
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* TIFF: YCbCr tags with the wrong value count use the defaults
tag_f64s now returns None unless the tag holds exactly n values, so a
YCbCrCoefficients or ReferenceBlackWhite tag with extra values falls
back to the TIFF defaults instead of being truncated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lrgj5pnShYtsSnrsZq5iRn
* TIFF YCbCr: tests for tiled JPEG with ICC, big-endian 16-bit and planar
Covers the layouts the review asked for: a tiled 4:2:0 JPEG file with an
embedded profile (checked against an independent JPEG decoder), a
big-endian 16-bit file, and planar data in both byte orders.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lrgj5pnShYtsSnrsZq5iRn
* TIFF YCbCr: cite the formula sources and pin the green-channel choice
Documents where the conversion comes from: TIFF 6.0 Section 21
(YCbCrCoefficients, pp. 90-91) and libtiff's tif_color.c
(TIFFYCbCrToRGBInit, TIFFYCbCrtoRGB). Red and blue follow the spec;
green follows libtiff and adds Y unscaled, which matches the spec only
when the luma weights sum to 1. The module doc records this as a
deliberate compatibility choice, and a new test pins it: with weights
0.25/0.25/0.25, mid gray stays (128, 128, 128).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lrgj5pnShYtsSnrsZq5iRn
* Codecs README: YCbCr TIFFs with three 8- or 16-bit samples decode to RGB
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lrgj5pnShYtsSnrsZq5iRn
* Format tiff_ycbcr tests with rustfmt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(tiff): compare grouped YCbCr decoding and update support notes
* style(tiff): align grouped decode call with test formatting
* style(tiff): apply repository rustfmt to grouped YCbCr regression
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
A canvas over 2^26 pixels (256 MiB of RGBA8) is saved without composite/full.png. The PNG encoder's buffers are reserved fallibly, so an allocation failure reports an encode error, and a ZIP bundle that cannot be reserved with the composite is written without it.
Conflicts resolved as unions of both sides: codecs re-exports (PngRows + TiffBands), smart_cmds source imports
(main's source_surface_from_buffer and cache API with #8571's svg_source_image_in/convert_svg_source; main's
duplicate rendering import dropped), io flat.rs test modules (#8536's banded TIFF tests and main's PNG ones),
menus/invoke.rs (#8571's smart SVG intercept and main's layer.selectUnderCursor), state.rs (#8311's
type_box_overflow and main's type_insertion_style), the canvas_geom test keeps the is_finite guard, docs
status lines keep both change lists, i18n catalogs keep both sides' rows (hi.tsv: main's wording where both
added the same string).
UPDATE_APP (native builds; craft-update is a native-only dependency) carries our 128 px icon and accent (#2f7bf5, its colour in ArtCraft Launcher). The updater, the self-update
helper and craft-update's startup prompt all get them through crate::UPDATE_APP; the
prompt draws the icon, the accent and "by ArtCraft" (craft-update's default developer).
craft-update is pinned to storytold/craft-libs main at 1065653e7 (#434).
Clippy fixes for Rust 1.99 in untouched code (same logic):
- apps/photocraft/src/app_dirs.rs
- crates/codecs/src/codecs/via_image.rs
- crates/engine/src/canvas_geom/tests.rs
- crates/ui-egui/src/panels/blend_preview.rs
- The free-memory check ran for every flat TIFF on Linux with a 512 MiB
reserve, so small TIFFs that open on main were refused on a small VM or
in a container: it now runs only for sizes the whole-image decode
refused (over 2^30 px or its byte budget).
- In a cgroup, memory.current counts page cache the kernel reclaims
before memory.max is hit; its inactive_file part now counts as free.
A `docker --memory=2g` batch whose cache filled up refused every TIFF.
- A single-strip TIFF is one group, the whole image, and every converted
piece of it was held at once: conversion now runs in rounds of at most
32 MiB of source rows per core.
- TiffBands::open reported no ExtraSamplesDropped/OverLimit warnings on
32-bit targets (wasm); both decoders now share prepared_warnings, and
Truncated goes before MorePages by position.
- Drops the ar.tsv rows of "Fix CI after merging upstream": main added the
same two keys (#10173), so the merged catalogue had duplicates and
i18n::tests::bundled_catalogs_are_consistent failed.
- Doc comments: the swap cap is every cgroup's above too (the ancestor
walk FredApps asked for); a cut-off strip decodes as zeros.
Co-authored-by: Khawar Zamman Wani <28915582+wanikhawar@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Port behavior and regression coverage into current modules; preserve current upstream behavior.
Integrate lossless export-only JXL and independent decoder regression matrix, preserving current JPEG/TIFF save extensions and 12-bit/arithmetic JPEG decoder dependency.
The new plausibility check refused valid files that main opens:
- EXR DWAA/DWAB parts: they go past 2^16:1 (a flat 8192x4320 DWAB frame
with 12 alpha-scheme mattes is 29,948 bytes, 75,629:1; run-length
channels zlib-packed on top reach about 132,000:1). DWA parts now get
2^18:1, so a header of a few hundred bytes can ask for about 100 MB,
not gigabytes.
- BMP RLE8/RLE4: end-of-line, end-of-bitmap and delta escapes leave pixels
unwritten, so no ratio bounds a valid file (an EOB-only 4000x4000 RLE8 is
1,080 bytes). Skipped like HDR.
- GIF: the decoder's size is the logical screen, which needs no data (a
4000x4000 screen with a 1x1 frame is 35 bytes). The first frame must be
backed, and the screen only past 256 MiB, so a 35-byte GIF still can't
declare a 30000x30000 screen.
The declared-size products saturate: a Radiance header of 2^31 x 2^31
read without limits overflowed u64 in debug builds.
Co-authored-by: FinnNk <1424666+FinnNk@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zlib-rs maps level 1 to deflate_quick, which writes static Huffman blocks
only, so every Compression::fast() caller (PDF export and print, brush
presets, PNG Fast) wrote 5-32% bigger streams than miniz's level 1 did.
Level 3 is the lowest level that is never bigger than miniz level 1 on the
docs/images photos, raw and Up-filtered (level 2 was +0.9% on one); it is
0-16% smaller and takes 1.2-3.4 times as long. The new test exports a
noisy-gradient RGB document as PDF and checks the image stream against
miniz level 1 of the same samples: 603007 vs 547315 bytes with fast(),
passes with level 3. The PR's doc no longer says the levels are unchanged.
Main moved the io integration tests into crates/io/tests/it/, so the new
deflate_compatibility.rs moves there and uses crate::common.
Co-authored-by: Rene Leonhardt <65483435+reneleonhardt@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
David Mann
yoriswind
Dylan M. Taylor
Yuki-init
FredApps
dotCooCoo
aiongg