A compression-7 frame may declare up to four times its segment's samples
(ljpeg::decode is capped at 4x the tile), and read_plane returned the whole
frame, so each retained tile held up to 4x what retained_segment_bytes
charges it: tiles declared 256x256 holding 512x512 frames pass a 1 MiB
budget while keeping 2 MiB. The lossless-JPEG arm moves into
lossless_jpeg(), which now truncates to the segment's rows (all the scatter
reads) and releases the spare capacity.
The PR's test only called the helper; deleting the call in read_plane kept
it green. decode_charges_the_retained_tiles_together drives decode on an
18x18 CFA in four padded 16x16 LJ92 tiles: 1500 B -> LimitExceeded, 2048 B
-> Ok. lossless_jpeg_keeps_only_the_segment pins the truncation. The doc
comment now says the output plane is alive alongside the sum.
Co-authored-by: Muhammad Saad <188063027+saad-mughal435@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- gpu: the test modules share one GPU_LOCK, and timing and vector_mask
take it too, so one process drives one wgpu device at a time.
- ui-egui: the wgpu tests in pattern_preview_canvas, adjust_preview_gpu,
marquee_autoscroll and layout_perf take gpu_lock().
- affinity compound_limits: the child process selects its test by the
module path (compound_limits::<name>); with --exact, the bare name
matches no test and the child passes without running the check.
- xtask scorecard reads crates/io/tests/it/corpus.rs.
- Docs and comments name the new test paths and the --test it filters.
Every tests/*.rs file is its own executable that links the whole crate
graph. Move each crate's test files into tests/it/ as modules of one
tests/it/main.rs; helper modules under tests/common move with them.
These keep their own binary (own process):
- engine panic_hunt: a process-wide panic hook counts every panic.
- engine text_gamma: changes the process-wide text gamma.
- engine smart_filter_stages, mask_cache_purge: assert on global caches.
- codecs pnm_peak_memory: reads the process's peak RSS.
- photocraft mac_menu_appkit: harness = false.
crates/engine/src/file_cmds.rs: batch keeps the save-point closure (takes the output path) and also copies the session's authorize hook into each scratch session, as main now does.
crates/raw/src/sensor.rs: both sides cap the memory of retained tiles; kept the existing retained_segment_bytes check (checked multiplication, same limit) and dropped main's duplicate inline loop.
Resolutions: CLI usage, flag lists and flag parsing carry both --tiff-compression (t15) and
--jpeg-format (t16); file.saveACopy, batch, imageProcessor and layersToFiles param specs carry
both tiffCompression and jpegFormat; file_cmds keeps tiff_compression_param and
jpeg_format_param; atn_cmds keeps flag_set/applies_at_canvas_bounds (t15) and UNHOLDABLE (t16),
edit.transform uses the pixel_target guard (t15) with layer_ref(p)? (t16); ui-egui open paths
build ImportOptions (t16) and apply raw_defaults (t15); docs headers/rows unioned; atn tests keep
both sides; tr.tsv "Keep Using CPU" kept once.
Tiff::dump appended each entry's full ASCII value with no per-value or
total-output limit. Thousands of entries can reference one large aliased
ASCII value, expanding a small diagnostic input into a multi-gigabyte
String.
Cap each ASCII preview to 256 bytes (truncated at a char boundary) and
stop the listing once the accumulated output reaches 1 MiB, marking it
truncated.
Finding: d994ec7f-25dd-4ae7-9673-04adc0aa8a3c (CWE-400)
read_craw treats every segment as a run of full-width image rows and
grew the row-reference vector before checking coverage. A tiled layout
(which does not describe full-width rows) with repeated tile offsets
could push far more than height rows, allocating gigabytes before the
post-loop coverage check ran.
Reject tiled cRAW layouts up front (cRAW is only stored as strips) and
stop collecting rows once the image height is reached.
Finding: 3c2601a1-ad92-45b6-96f7-be5da87d97eb (CWE-400)
compressed_row positioned the bit cursor by replaying every preceding
byte within its page (skip % PAGE calls of get(8)). For narrow images
this replay runs per row, so a small file could consume disproportionate
CPU just setting up cursors.
Add Bits::new_at, which sets the page position and remaining-bit count
directly from the row offset in O(1), and use it in compressed_row. A
unit test proves it matches the old byte-by-byte replay.
Finding: 2a7d3d4b-90ab-4887-9dba-c3884e224990 (CWE-407)
GainMap::row materialized the entire horizontal gain grid (points_h
floats) once per applicable image row, even though each output pixel only
needs two neighboring points. File-controlled image height and horizontal
map size multiplied into processing and allocation churn far beyond the
input size.
Add GainMap::at_rv, a direct four-corner bilinear lookup, and use it per
output column in gain_row so no points_h-wide row is built per image row.
A test proves at_rv matches the former row() + at() path.
Finding: 7ebec8df-1a7a-4d33-915f-097f8e9be0e5 (CWE-400)
Each DHT definition for table id 0..3 calls Huffman::new, which
allocates and fills a ~48 KiB lookup pair. A stream repeating DHT
definitions (and DNG tiles referencing such a stream) could rebuild
tables thousands of times, multiplying header-processing cost and
memory churn while producing little output.
A real lossless JPEG uses a handful of Huffman tables; cap cumulative
table builds per stream at 64 and reject streams that exceed it.
Finding: 7a31b581-d1b7-4f9c-91e5-ce42b47db87f (CWE-400)
read_plane decodes every tile in parallel and retains all decoded tile
buffers in before assembly copies only the useful pixels. Each
tile is charged individually, but file-controlled padding and repeated
source ranges can make the cumulative retained samples far exceed both
the input and the useful image.
Sum the padded tile samples (two bytes each) and reject the plane before
decoding when that total exceeds max_alloc.
Finding: e66981b1-0d7a-4738-a64c-37b5e8bf28d1 (CWE-770)
Conflicts:
docs/gaps.md: header line is ours plus the PR's missing change item (later Last updated date); change-log rows kept both, newest first
docs/file-format-parity.md: same
The open-time dialog's raw curve is shared (Arc) by the proxy worker from main
(#5964) and the full-resolution preview.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The D4 profile's saturation level of 15520 was measured on 14-bit files but
applied to the 12-bit files the body also writes, whose samples are the
14-bit values shifted right by two. 12-bit D4 NEFs developed about 2.6 stops
darker than the same scene at 14 bits, with no warning.
The profile now records the depth its level was measured at, and
cameras::apply takes the stored sample depth: 14-bit files keep 15520, 12-bit
files get 3880. A synthetic D4-sized NEF at both depths must develop within
0.2 stop (it was 2.5 stops apart before); on the public 12-bit D4 files the
difference to the 14-bit twin is now within 0.02 stop.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rj8vs4QyUES3uwgX6SuGGh
The README's clean-room section names every maker-note tag the crate reads and
where it is documented; 0x001E is added, and the code comment now names
ExifTool's Nikon tag documentation as its source.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rj8vs4QyUES3uwgX6SuGGh
embedded_preview_color_space only read the EXIF ColorSpace, but Nikon bodies
write none: they say Adobe RGB only in the maker note (tag 0x001E: 1 sRGB,
2 Adobe RGB). A Nikon NEF with Adobe RGB selected had its JPEG preview read as
sRGB by the colour fit.
When the EXIF does not state a colour space, the Nikon maker note's is used
(the EXIF still wins, a DNG still states nothing, other makers' maker notes
are not read). The maker-note locator in tiffep.rs is shared. A packed Nikon
NEF with the tag joins the hostile-input seeds, and rawinfo prints the stated
colour space.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rj8vs4QyUES3uwgX6SuGGh
`develop::tone` interpolates the camera curve logarithmically between knots, and a curve whose first knot sits at black (0, 0) made the first segment divide by log₂ 0: every value in it came out NaN, and `retone` (the open-time Shadows/Highlights preview) propagated it into the pixels. From a knot at black the segment is now linear, where log₂ has no value; the other segments are unchanged.
Test: `retone_lifts_shadows_with_a_curve_from_black` (finite output, shadows lifted, highlights kept) fails without the fix.
Split out of #3816 at the reviewer's request.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
When a raw opens, Shadows, Highlights, Clarity, Texture and Dehaze act on the sensor data (`retone`, in linear light before the camera's tone curve) and Open develops the file that way, but the dialog's full-resolution preview ran the filter's RGB versions of those sliders: at 100 % the preview showed one thing and Open made another.
`DetailPreview` gets the raw's tone curve (`raw_curve`, set when the open-time dialog opens); with it, `develop` retones the whole area from the whole frame's maps exactly as Open does, then runs the rest of the pipeline with those five sliders at zero. Without a curve (the filter on a layer) nothing changes. `retone` runs its rows in parallel (`crate::par`) and recomputes the linear colours per band instead of keeping a second copy of the image, since the detail preview develops 16–45 MP frames.
Test: `a_raw_open_refines_raw_stage_sliders_from_the_whole_frame`.
Split out of #3816 at the reviewer's request.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Strict workspace Clippy stopped on collapsible_match in affinity, raw, plugins, engine, ui-egui and xtask. Pure conditions become guarded match arms (each falling through to an existing empty arm), and the ruler drag state and the measurement command result are bound to locals, preserving evaluation order, error messages and fallbacks.
The same lint fixes also appear as a side commit in #5157 (David2766), which can drop it on rebase.
read_plane decodes every segment through par::map and keeps the whole Vec of
decoded tiles until it scatters them into the destination, so the working set
is the sum of the tiles rather than the largest one. Each tile is charged
against Limits on its own just before it decodes (#2925), and max_alloc is a
per-buffer limit by contract, so a file that declares many segments which are
each admissible can still retain far more than any one of them allows.
Charge the sum before any segment decodes: retained_segment_bytes adds up the
decoded size of every segment from its declared geometry, refusing a segment
whose own size overflows rather than wrapping, and stops as soon as the running
total passes max_alloc.
A segment's geometry is declared independently of the image, which is what lets
the total run away. For a well-formed file the segments tile the image, so
their sum is about its own size and limits.check has already bounded it; what
this refuses is a file whose declared tiles are far larger than the image they
belong to.
Test: tiles that each sit far inside the budget are refused once their sum
passes it, more samples a pixel cost proportionally more, an overflowing
geometry is refused rather than wrapping, and an ordinary 6000 x 4000 frame in
512-pixel tiles still passes the default limits.
* TIFF: read and write CIE L*a*b* (CIELab, ICCLab); Lab documents stay Lab
Photoshop saves Lab documents as CIELab TIFFs (PhotometricInterpretation
8, a* and b* signed), which PhotoCraft could not open, and PhotoCraft
saved Lab documents as sRGB TIFFs.
- codecs: a Lab channel layout (Lab, LabA) holding a* and b* as
PhotoCraft's Lab documents do ((a + 128) / 255; 16 bits like the PSD
importer), with its own Lab <-> sRGB conversion (same formulas as
photocraft-color; codecs stays standalone) for formats without Lab,
which now warn LabConverted.
- TIFF read: CIELab and ICCLab (8 and 16 bits, alpha) are handed to the
tiff crate as RGB and labelled Lab; a* and b* are converted from the
file's encoding (signed a / 256a, or offset).
- TIFF write: Lab and LabA as CIELab, as Photoshop writes it; float Lab
is written with 16 bits (DepthReduced).
- io: a Lab TIFF opens as a Lab document; a Lab document with one plain
layer is written to TIFF in Lab (other formats still use sRGB).
Photoshop 2025's Lab TIFFs open with the colours Photoshop shows (its
own RGB conversion matches within one level at 8 bits and exactly at 16
bits), and Photoshop reads PhotoCraft's Lab TIFFs back with the same
L*a*b* values.
Fixes#4936
* DNG: decode lossy (JPEG-compressed) DNG tiles (FILE-215-11)
DNG 1.4 lossy JPEG (compression 34892) stores each tile or strip as a
baseline-DCT JPEG of 8-bit samples: a gray stream for CFA data, a YCbCr
stream for three-sample LinearRaw. The decoder arm from #2021 read the
streams but widened every sample by 257, so the values no longer matched
the file's levels: an 8-bit WhiteLevel (or its default, 255) clipped
everything to white and a 256-entry LinearizationTable was indexed past
its end. It also required the JPEG frame to match the clipped row count,
which rejected the padded edge tiles real tiled files carry.
The arm is now `lossy_jpeg()` in sensor.rs. The decoded 8-bit values are
the stored samples, and BlackLevel, WhiteLevel (default 255 for 8-bit
data) and the LinearizationTable apply to them exactly as for every other
compression. A tile may be coded at the full tile size (edge tiles padded,
as the spec says) or at the image's row count; a strip at its row count;
the width must be the segment's. BitsPerSample must be 8 and the JPEG's
component count must equal SamplesPerPixel (1 or 3); anything else is an
error, never a silent conversion. The frame header is checked before any
pixel is decoded and the JPEG decoder's own size cap is the segment the
IFD declares (already charged against the limits), so a hostile SOF cannot
size a buffer past the tile.
testgen writes true 8-bit lossy fixtures (`LossyJpegTiles`, and strips via
the new `LossyJpegStrips`), accepts pre-encoded `lossy_segments` for
hostile-input tests, exposes `baseline_jpeg()`, and leaves WhiteLevel out
when `white` is 0. The DNG tests cover CFA and LinearRaw RGB from tiles
and strips at 37x23 (partial edge tiles, a short last strip), the default
white level, black/white scaling and a LinearizationTable against the
uncompressed path, and hostile inputs: a truncated tile, frame-size and
component-count mismatches, a 65535x65535 SOF under default and tight
limits, a tile over the allocation budget, and 12/16-bit or 4-sample
files. The hostile mutation set gains a lossy CFA and a lossy LinearRaw
file. The parity doc's DNG row is updated; the scorecard row is left for
the coordinating branch.
* Raw: measure the Nikon D90 image area (4288x2848 at (10, 10))
Aligning the decoded D90 sensor plane with the camera's own JPEG puts the
image at (10, 10) in the 4352x2868 sensor plane, giving the camera's
4288x2848 frame (10-pixel margins; the right border reads the camera's flat
255 from column 4310). Add the measured ImageArea so a D90 NEF opens on the
camera's frame instead of the uncropped plane, and make a measured profile
leave a file that carries its own colour calibration (a DNG) untouched.
* Codecs: open Cineon (.cin) film frames read-only
Cineon is the 10-bit logarithmic film-scan format DPX grew from. Read it from
the same Kodak header family: the generic header (big-endian, orientation at
192, channel count at 193, the channel-1 bits/dimensions), one or three
channels, three 10-bit samples packed into each 32-bit word on a word
boundary. The 10-bit code values open as U16 (round(c x 65535 / 1023)); the
file's log black/white points are a display choice and are not applied. A
depth other than 10 bits or a channel count other than 1/3 is Unsupported;
truncation errors rather than panicking. Read-only (no Cineon writer), so
.cin joins the open filters and Cineon read closes part of #1049's G10 gap.
* Vector: bound flattening memory, keep over-budget dashes, cheaper edge build (#5195)
- flatten: a path's flattened points are capped at 1 << 20; past it every
curve keeps a proportional share of its pieces (at least one), so hostile
handles from files (vmsk, SVG, effect outlines) cannot exhaust memory.
fill_rasterizer flattens the whole path under one budget.
- stroke: past the dash step budget (or below coordinate precision),
dash_polyline keeps the dashes made so far and draws the rest solid
instead of dropping the stroke. Open polylines are no longer copied.
- raster: coordinates are clamped to +-1e15 and edges with a non-finite
slope (vanishingly short) are skipped, so huge finite coordinates no
longer produce NaN coverage. fill_rasterizer sorts edges once instead of
after every component (stable sort, identical order).
- Cleanups: dead miter `half`, dead loop-length check in trace_mask,
shared mask render body, map_knots helper, shared straight-segment test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* PSD: cap pattern memory across a parse, reject mixed-depth pattern channels (#5198)
- Patterns: one PatternBudget (decoded bytes + zero padding) is shared by
every pattern of a block, a .pat file and the pattern blocks of one
document; .abr patt sections are charged to the samples' budget.
- Patterns: a channel whose depth differs from the first is rejected (the
parsed pattern could not be written back).
- Slices: descriptor numbers no longer wrap; out-of-range ids are no id,
coordinates clamp to i32, and a v6 trailing entry without a usable id
attaches its outsets to no slice.
- LayerInfo::unpadded_len is computed from field sizes instead of
serializing every record into a scratch buffer.
- abr desc presets are moved out instead of cloned; UnFl and shmd counts
use the bounded preallocation; ImageData::decode_channel (decoded every
plane to return one, unused outside its test) is removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Vector: bound stroke outline points (#5211)
- stroke_polygons dashes all lines first, then counts round caps and
joins. When their full-resolution tessellation would exceed 1 << 20
outline points, every round cap and join gets the same smaller segment
count (down to 4 per circle, 1 per join; area-exact radius kept).
Strokes under the budget are unchanged; no piece is dropped.
- The dash step budget (100000) is now shared by all lines of a stroke
instead of per polyline, so many subpaths cannot multiply the dash
(and cap) count. Past it the rest is drawn solid, as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* PSD: write tiff blocks without copies, inflate merged ZIP once
- ImageSourceData::to_bytes writes the global mask and global blocks
straight from the model with 4-byte alignment instead of cloning each
block (data included) to set its padding. Same bytes.
- The tiff transcoder's nested regions and blocks write into the parent's
buffer (taken and handed back, error or not) instead of building their
own output and copying it up per nesting level. Bodies that fail or are
not transcoded are truncated before being copied as stored or dropped.
Same bytes.
- PsdFile::from_bytes_and_merged decodes the merged image with the parse
when asked: a ZIP stream is inflated once (the decode's stream check is
the parse's); the parse-only check runs only when the decode fails, to
keep from_bytes' errors. PSD import uses it with merged_needed.
- PsdBuilder::composite documents that Gray/CMYK composites stay straight.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Content-Aware Fill: measure a thin hole by its thickness, not its bounds (#1366)
Removing long, thin objects (coiled guitar strings, wires) left their
outline as darker and lighter blotches and copied pieces of other strings
into the fill. The pyramid shrank until the hole's bounding box was four
patches across: for a string across the image that is a level where the
hole has swallowed its surroundings, so the matches found there copy
whatever looks like the blur, and the finer levels (refine-only for wide
bounds) keep them.
A hole at most a quarter as thick as its bounds are long now counts as
thin: the pyramid stops while it is still about two patches thick, and
that level starts from a membrane across it (over a few pixels, the
surroundings themselves) instead of the onion peel, whose exhaustive
search is what made a shallow pyramid unaffordable. Compact holes are
unchanged bit for bit.
Refs #1366
* perf(compose,gpu): effect caches reserve tile addresses instead of pinning layers
Effect map cache entries kept a clone of their layer (a deep clone for groups) so the tile addresses the cache key hashes couldn't be reused. That kept an edited or deleted layer's pixels alive, uncounted, until the entry was evicted. Keep a Weak to each hashed tile instead: it reserves the address without keeping the pixels. The GPU compositor's styled-group pins and shape-source tile snapshots do the same. Part of #4530.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(history): count snapshot-only tiles against the history budget
History panel snapshots outlived every budget: a Merged Layers snapshot is a full flattened copy that neither the history budget, the GPU budget's document bytes nor Purge accounted for. Trim now counts the tiles only snapshots hold as fixed cost, the GPU budget includes them, and Purge > Histories no longer reports tiles a kept snapshot still holds as freed. Part of #4530.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Content-Aware Fill: every part of a thin selection keeps its surroundings (#1366)
Two more causes of ghosts when removing strings and the small knobs
selected with them:
* Auto sampling summarises the selection's surroundings by a few
clusters. Around a long selection those come from the strings'
surroundings, so a small part elsewhere (a tuning knob in a differently
coloured corner) lost its own surroundings and was filled from far away
in another colour. Around a thin selection the Auto area now always
reaches as far as the selection is thick.
* The membrane that starts a thin hole now starts a patch radius out, past
the anti-aliased fringe or halo that a selection drawn close to the
object leaves outside it.
Refs #1366
* Content-Aware Fill: peel a thin hole where that stays affordable (#1366)
Starting every thin hole from a membrane smoothed Spot Healing on film
grain (spot_healing_keeps_film_grain_through_overlapping_strokes: grain
9.0 against 12.1 around). The onion peel copies the grain; it was only
given up because its exhaustive search is too slow on the big level a
long thin hole stops at. It now starts the thin hole whenever hole cells
times sources stay within 2^28 comparisons, and the membrane only beyond.
Refs #1366
* Select Subject: keep subject parts coloured like the backdrop, leave enclosed gaps open
Users on #2788 reported Select Subject dropping clothes that are close in
colour to a plain wall and bridging the gap between an arm and the torso.
Both come from the same place: the saliency pass labels low-contrast parts
of the subject as background, the background colour model then learns
them and explains them away, and the final blind hole fill closes every
enclosed gap up to 5 % of the mask.
Two changes in crates/algo:
- A second GrabCut pass whose background model only knows the backdrop. A
colour model fitted to the image frame (minus a margin around the
salient blob, in case the subject runs out of the picture) marks every
pixel it explains as probable background; the blob and whatever the
backdrop cannot account for start as probable foreground. Only
components that touch the first pass's result survive.
- Hole filling that consults the colour models: specks (<= 0.2 % of the
mask) always fill, larger holes up to 5 % fill only when their pixels
fit the foreground model better than the background's.
Tests: two regressions (a cream-shouldered figure on a near-white wall
keeps its shoulders; a ring keeps its backdrop-coloured hole open), plus
ignored timing and picture-dump helpers. The 6 MP first run goes from
about 100 ms to about 200 ms on one core; the P29 repeat path is untouched.
* Keyboard shortcut sets: Set menu, New Set, Save, Delete, .kys Export and Summarize (K2)
Edit › Keyboard Shortcuts gets Photoshop's set management. A set is a saved copy of the
bindings (Preferences.shortcutSets, the one in use in shortcutSet; "" is PhotoCraft Defaults),
and the Set menu shows "(modified)" when the bindings differ from it. New Set…, Save and
Delete wait for OK like the rest of the dialog. Export… writes the dialog's bindings as a .kys
file (commands then tools, Photoshop's macOS key spelling), each row carrying our id as pc-id
so PhotoCraft imports its own sets back exactly, whatever the label; Summarize… writes
Photoshop's HTML list of every bound command by menu.
edit.keyboardShortcuts takes useSet (before the call's edits), saveSet, deleteSet, sets and
activeSet (after them), exportKys and summarize; every reply reports the sets, the set in use
and whether it is modified. Bad names, keys or counts are errors that change nothing.
* .kys import: tool keys and sets saved on Windows (K3)
Photoshop's <tool> rows now import: each matches our tool by name (our tool names are
Photoshop's) or by pc-id in sets PhotoCraft exported, and a letter other than the tool's own
becomes its override, while its own letter keeps the tool cycling with the others (#5036).
A set saved on Windows spells the primary modifier Ctrl where a Mac set writes Cmd and keeps
Control for the Control key. A set with no Cmd or Opt anywhere but some Ctrl is read as a
Windows set, its Ctrl as our Cmd: Ctrl+L is the Cmd+L default, not a Control-key override that
would also mean Control on a Mac. The import reports `windows`.
* cargo fmt the extension lists
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Gert Pellin <gert@pellin.be>
Co-authored-by: Vikram Bhaskaran <vikram.bhaskaran@outlook.com>
Co-authored-by: vangel76 <vangel1976@googlemail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Finn Newick <finnnk@gmail.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Diego Chagas <drochachagas@gmail.com>
Co-authored-by: Tanish Manku <tmanku.dev@gmail.com>
Aligning the decoded D90 sensor plane with the camera's own JPEG puts the
image at (10, 10) in the 4352x2868 sensor plane, giving the camera's
4288x2848 frame (10-pixel margins; the right border reads the camera's flat
255 from column 4310). Add the measured ImageArea so a D90 NEF opens on the
camera's frame instead of the uncropped plane, and make a measured profile
leave a file that carries its own colour calibration (a DNG) untouched.
Co-authored-by: Brandon Thomas <echelon@gmail.com>
The compressed RW2 layouts of earlier Lumix bodies (e.g. the DMC-GX7)
decode at full sensor resolution instead of falling back to the camera's
embedded JPEG preview. The layout was established by observation of
sample files (fixed 16 bytes per 14 pixels, predictor groups with
per-parity predictors, page-permuted downward bit order); the black
level of these files reads 15 counts low.
Adds round-trip tests through a mirrored synthetic encoder, cursor and
predictor-group unit vectors, and updates the support matrix.
Co-authored-by: Gotya-OFFCIAL <339410022+Gotya-OFFCIAL@users.noreply.github.com>
Some DNG writers state a white level the sensor never reaches (an iPhone
15 Pro Max Bayer DNG states 4095 and clips at 3567). Development clips
each channel at 1 after white balance, so red and blue reached 1 while
green stayed at 0.85 and blown highlights turned magenta. On decode, a
per-colour pile-up clearly below the white level is now raised to it, as
in storytold/lightcraft#586.
crates/raw/fuzz had no .gitignore, crates/psd/fuzz didn't ignore
Cargo.lock and crates/format/fuzz didn't ignore coverage/, so running
those fuzz targets left untracked files in the checkout. All five fuzz
crates now ignore target/, corpus/, artifacts/, coverage/ and Cargo.lock,
as codecs and affinity already did.
* Camera Raw: Whites, Blacks, Shadows and Highlights closer to Photoshop's
Measured as a black box on Photoshop 25.4's Camera Raw Filter.
Whites and Blacks are global curves of the grey level: the measured rows
per slider value, Whites first, after Shadows and Highlights. The
clipping directions adapt to the image's black and white points
(Whites up to 2.7 times stronger on a dark image, Blacks at least 1.2).
On ten public-domain photos the mean error against Photoshop drops from
4.2-36.8 to 0.9-8.9 levels of 255.
Shadows and Highlights are a local Laplacian filter (Paris et al. 2011,
fast form of Aubry et al. 2014) on the log luminance, the method Adobe
credits for process version 2012. Shadows raises pixels darker than
their surroundings by an amount set by their own and their
surroundings' tone, so big edges keep no halo, and keeps the 99th
percentile; Highlights pulls pixels brighter than their surroundings
toward them and keeps the 1st. The strengths are fitted to the photos
(mean error 2.8-9.4 levels, was 3.7-14.3); Photoshop's exact operator
was not recovered.
The tone stages decode light with the document's curve: filter.cameraRaw
stores encodingGamma (1.8 for ProPhoto, where raws open).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Camera Raw Shadows/Highlights: no contour lines on smooth gradients
Twelve reference values over a raw's whole range left more than a stop
between them, and the remapping changes within half a stop: coefficients
interpolated across that gap drew contour lines in out-of-focus areas
(Shadows +100 on a Nikon D4 raw). References are now an eighth of a stop
apart over the image's tones (the deepest 0.1 % share the lowest), on a
copy at most 1600 px across whose change comes back to full size by a
guided upsampling, so a 16-megapixel image takes about 2 s per slider and
the preview matches the result.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Camera Raw on raw opens: Shadows/Highlights in linear light, D4 look and black point as Photoshop
Measured black-box on 29 Nikon D4 NEFs opened through Camera Raw in Photoshop 25.4 (sliders at
+-100 and +50, exposure +-1) against our own linear development of the same files.
- Shadows and Highlights of the open-time dialog now act while developing, in linear light before
the tone curve, as Camera Raw does for raw files (exposure -1 gives the same gains). The log2
gain is slider/100 * C(B - anchor): B is a guided-filter base of the log2 luminance on a copy
1232 px on its long side, the anchor is image-adaptive (Shadows: log2 power mean p=1.5 of the
linear luminance; Highlights: mean log2 luminance). New module photocraft_raw::tone; the
preview undoes the camera curve to show the same (photocraft_raw::retone).
- The D4 profile copies Camera Raw's default rendering: look matrix folded into the
ForwardMatrix, per-channel curve, and an adaptive black point (0.33 x the 0.1th percentile of
the luminance, Sensor::black_point).
Against Photoshop on the 29 frames (1/4 size, log2 Y): default rendering median difference
0.29 -> 0.04 stop; correlation of the slider's effect Shadows +100 0.96, -100 0.91, Highlights
-100 0.87, +100 0.85 (the previous filter model: 0.68 and 0.55 on DSG_3464).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs: the open-time Camera Raw re-develop covers Shadows and Highlights
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Camera Raw raw-stage Shadows: keep shadow contrast as Photoshop does
The user saw Photoshop's shadows sink deeper with more contrast. Measured on the 29 D4 NEFs:
with Shadows +100 our shadow detail had 0.79 of Photoshop's local contrast and shadows sat 0.38
stop lighter. Two causes:
- The black point now comes before the Shadows / Highlights gain, as in Camera Raw (lifting
first let the lift wash out the black point's deep-shadow contrast).
- The guided upsampling's epsilon goes from 0.01 to 1 stop^2: the change follows strong edges
but stays smooth inside texture (with 0.01 it followed every detail and flattened it). Near
strong edges the effect still matches Photoshop as well as elsewhere.
Shadows +100: shadow detail 0.79 -> 0.91, shadow bias +0.38 -> +0.16 stop; effect correlation
Shadows -100 0.906 -> 0.946, Highlights -100 0.870 -> 0.886, others unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Camera Raw on raw opens: Clarity, Texture and Dehaze in linear light, as Photoshop
Measured black-box from Photoshop 25.4 opening 29 Nikon D4 NEFs through Camera Raw (descriptor
keys Cl12, CrTx, Dhze; each at ±100 and +50, and +100 at exposure −1, 348 opens). All three act
before the tone curve: the change at exposure −1 equals the one at 0 (correlation 0.98–0.99).
- Clarity: log2 luminance split into bands between guided-filter bases (radii 2, 8, 32, 128 on
the 1232 px work copy), each band scaled by a measured gain that follows its base's tone, plus
a tone term; carried to full size with the Shadows / Highlights change.
- Texture: Gaussian bands σ 0.5…16 on a half-size copy, each through its measured response
(small detail raised more than strong edges, flat areas unchanged), guided-upsampled.
- Dehaze: He, Sun and Tang's dark channel prior on linear ProPhoto (dark channel in Display P3,
airlight from the brightest 0.1 % of it, guided-smoothed transmission), w 0.8 at +100;
−100 a uniform veil. The black point stays the default rendering's.
Against Photoshop, PhotoCraft's develop vs Photoshop's open on all 29 frames (median effect
correlation, log2 luminance at 1/4 size, Texture at 1/2): Clarity +100 0.81, −100 0.85;
Texture +100 0.85, −100 0.88; Dehaze +100 0.61 (0.78 in gamma 2.2: the crushed shadows
differ), −100 0.96. Shadows / Highlights unchanged (0.964 / 0.886).
The open-time Camera Raw dialog re-develops for them and its preview shows them (retone).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
The per-file colour fit (`fit_preview_look`) converted the embedded JPEG to sRGB only when the JPEG
carried an ICC profile, and otherwise read it as sRGB. A Sony body set to Adobe RGB writes no profile
into its previews; it states the colour space only in the raw's EXIF (ColorSpace Uncalibrated with
InteropIndex "R03", the DCF option file for Adobe RGB). The fit then targeted a desaturated,
hue-shifted reference, and the developed raw came out that way.
`photocraft_raw::embedded_preview_color_space` reads that EXIF (sRGB, Adobe RGB or unstated; never for a
DNG, whose previews are the converter's), and `preview_profile` uses it when the preview has no ICC
profile of its own. A preview's own profile still wins. LightCraft reads the same tags for the same
reason (storytold/lightcraft#778).
Measured on 19 private Sony ILCE-1M2 ARWs, all shot in Adobe RGB, against the camera's JPEG read as
Adobe RGB (median CIE76 ΔE at 240 px): full-frame 400 mm shots against blue sky 5.6-7.0 → 1.5-2.4,
their mean a* offset +4.3…+4.9 → +0.7…+1.1; every file improved, mean of the medians 4.9 → 2.2.
Tests: `exif_colour_space_of_the_previews` (sRGB, Adobe RGB; R98, a missing interop IFD, other values,
a DNG and junk are unstated) and `a_preview_without_a_profile_takes_the_raws_exif_colour_space` (fails
without the EXIF fallback; the preview's own ICC profile wins). The hostile-input harness calls the new
function.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A one-sample LinearRaw DNG (VueScan film scans, Leica Monochrom) was
rejected as "LinearRaw data with 1 samples per pixel". The DNG decoder
now accepts it and develop renders it as neutral grey: the one sample
fills all three channels, white balance, colour matrices and camera
looks do not apply, and the output is R = G = B in 16-bit ProPhoto.
io skips the preview colour fit for such files.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* raw: fit non-DNG colour to the camera's embedded JPEG
* raw look: clippy (as_chunks) and rustfmt after merging main
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* DNG: lossy compression 34892 (baseline JPEG tiles)
Adobe's lossy DNG (compression 34892) decodes: each tile is a complete
baseline JPEG (8-bit luma at tile size), decoded by zune-jpeg into the
tile geometry (a mismatch is an error), widened 0xFF -> 0xFFFF like
Photoshop's read of 8-bit CFA data. Predictors and the rest of the
pipeline are unchanged.
testgen grows LossyJpegTiles (image-rs encodes the Q60 fixtures behind
the testgen feature), so CI runs a decode oracle: a smooth ramp stays
within a few quantisation steps of the 8-bit source, and a tile cut
mid-stream fails cleanly.
* Lossy DNG: check the tile JPEG header before decoding; zune deps go workspace
decode_headers() runs and the (width, height) == (s.w, rows) check fires
before decode(), so a hostile SOF cannot allocate the decoder's buffers
(up to 16384x16364) before the size mismatch is caught. The mismatch has
a regression test that swaps a 32x32 tile JPEG for a 16x16 one. zune-core
and zune-jpeg move to [workspace.dependencies] next to image and flate2.
* chore(ui): format prefs_ui (the workspace fmt gate is red on main until #2417 lands)
* Lossy DNG: 3-sample LinearRaw (YCbCr) decodes to RGB; tiles size-check
Real lossy DNGs are 3-sample YCbCr LinearRaw, not single-channel CFA:
the tile now decodes through zune-jpeg's YCbCr-to-RGB output when
SamplesPerPixel is 3 (gray stays for CFA), the decoded length is checked
against the tile's sample count, and other sample counts say unsupported.
The generator writes RGB tiles for 3-sample specs. A LinearizationTable
maps the decoded samples in the develop stage as before - nothing to add
in the sensor. Test: a 3-sample ramp with 64-step channel separation
(surviving Q60 chroma quantisation) decodes within the quality band.
* Web: leave the JPEG XL DNG decoder out of the wasm (0.88 MB, 23.1 MiB now)
The 0.6.0 web build went over packaging/web/package.sh's 24 MiB gate (26,980,503 bytes on
782bcb67). Today's landings added 1.34 MB; the biggest single item was jxl-oxide (#2047, JPEG
XL-compressed DNG): ~0.6 MB of jxl_* code plus brotli-decompressor and tables, 0.88 MB after
wasm-opt. 0c72d954 brought the wasm back to 25,144,413 bytes (21 KB under the gate) by deflating
catalogs and fonts; this takes it to 24,262,088 bytes (23.14 MiB, 904 KB of headroom).
jxl-oxide is now a native-only dependency of photocraft-raw. On wasm32 a JPEG XL DNG segment
returns RawError::Unsupported ("can't be opened in the browser version of PhotoCraft; open it
in the desktop app"), like the other cfg-gated web features; every other DNG compression and
raw format still opens in the browser. Native builds are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* xtask i18n-coverage: read the deflated catalog! registry entries
0c72d954 changed the LANGUAGES registry from include_str!("xx.tsv") to catalog!("xx"), so
`cargo xtask i18n-coverage` failed with "language `ja` has no include_str! catalog". Parse the
new form (the macro argument is the catalog's file stem).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* DNG: JPEG XL compression 52546 (DNG 1.7, Samsung Expert RAW)
Decode JPEG XL-compressed DNG strips and tiles with jxl-oxide (pinned
=0.12.6), with geometry and channel checks, Limits/AllocTracker budgets
and catch_unwind around the decoder. Warn when a DNG carries a
ProfileGainTableMap (tag 52525), which is not applied yet.
Tests encode fixtures in-process with zune-jpegxl (dev-dependency only).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* raw README: list Deflate DNG beside JPEG XL after rebasing onto #1971
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Integer DNGs with zlib-compressed strips or tiles (compression 8) now decode, with TIFF predictor 1
and 2 (wrapping at the sample's container). Output is capped at the tile size; floating-point
predictor 3 and lossy DNG keep their unsupported errors. flate2 was already a workspace dependency.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RAF's header, metadata records and Fuji IFD give size, packing (16-bit, 12-bit LSB-first,
14-bit in 32-bit words), layout, black, WB, crop and exposure bias; X-Trans and other
periodic non-Bayer CFAs go through a new edge-directed demosaic. Compressed and early
SuperCCD RAF stay unsupported (preview fallback). Tests use synthetic files only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sony cRAW files with no BlackLevel tag take the 512 their tone curve gives the black
code instead of 0 (the ILCE-7 no longer develops washed out; the RX0 corpus oracle moves
to 512 too). cameras.rs gains measured image areas without a full profile: the Nikon
D800E and Sony ILCE-7 crop like the camera's JPEG.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The five RAW readers cast the u32 Orientation to u16 before the 1..=8 check, so a LONG
65538 became 2 and rotated the image. A shared tiff::orientation checks the range on
the u32 first. Fixes#1817.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>