The decoder set the active area and the crop to the whole plane and never
read ActiveArea, so a file with masked borders and no DefaultCrop developed
with the borders in. Read ActiveArea like the DNG path, use it as the
initial crop, and place DefaultCrop relative to it as DNG does. Also give
3FR/FFF its own row in the camera raw table instead of "embedded preview
only".
Co-Authored-By: Gotya-OFFCIAL <Gotya-OFFCIAL@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The X2D II 100C's 3FR files develop at full sensor resolution instead of
falling back to the camera's embedded JPEG preview. A 3FR is a TIFF whose
IFD0 holds the camera's JPEG and whose CFA image sits in a SubIFD with
DNG-style tags but no DNGVersion and no CFA-pattern tag (neither the
TIFF/EP pair nor the EXIF one), which is why the generic TIFF/EP path
reported "CFA pattern not recorded". The new decoder reuses the shared
strip reader, takes BlackLevel, WhiteLevel, DefaultCropOrigin/Size and
AsShotNeutral from the file, and takes the Bayer phase from a measured
per-body table (the X2D II 100C: RGGB at the data origin, measured
against the camera's own embedded JPEG; unmeasured bodies warn).
Lossless-JPEG 3FR stays preview-only, like uncompressed-only ORF and RAF
before it.
Clean-room: the TIFF 6.0 and DNG specifications, the sample file's bytes
and its embedded JPEG; no code or tables from dcraw, LibRaw, rawspeed,
rawler or darktable.
Adds round-trip and hostile tests through a mirrored synthetic builder
(tests/it/hasselblad.rs, testgen::Hasselblad3frSpec) and updates the
support matrix (file-format-parity, gaps G14, target-app-parity).
Development applies every gain map to every pixel it covers, so a list
of thousands of maps multiplied the work of an otherwise admissible
image. Cap them like MapPolynomial; the rest are named as not applied.
jxl-oxide inflates Brotli-compressed auxiliary boxes (Exif, XMP) into
storage its allocation tracker doesn't see while it reads a container,
and samples spline geometry into an untracked vector with no bound on
the sample count. A raw tile needs neither: refuse a container holding a
brob box before reading it, and a codestream whose frames draw splines
before rendering.
read_plane decoded every segment separately, even when a file pointed
every tile at the same bytes. Each lossless JPEG decode rebuilds its
Huffman tables and each result is retained until assembly, so one small
range repeated across many tiles multiplied header work and memory by
the tile count.
Decode each distinct (offset, length, geometry) once and scatter the
shared samples into every tile that names it.
* raw: refuse lossy DNG tiles that carry Extended XMP
zune-jpeg keeps every Extended XMP fragment until its set is complete
and re-sorts and re-checks all pending fragments after each marker, so a
lossy (compression 34892) tile of many small incomplete fragments costs
quadratic time before a pixel is decoded. A raw tile has no metadata:
refuse one with an Extended XMP APP1 segment before parsing it.
* address review: bound Extended XMP identifier to APP1 payload
retone compared the buffer length with an unchecked w * h. In a release
build a product that wraps passed the check for a tiny buffer, and the
banded passes then ran over the original, huge height.
The end-of-strip rule accepts whole fixed-rate rows past the image (up to 4),
but no test covered a valid strip with such rows. Encode 1 to 5 extra rows at
12 and 14 bit: up to 4 decode with the image rows unchanged, the fifth is
refused (preview fallback).
Suggested by CodeRabbit on #10529.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Some lossy type 2 NEFs store a split row at byte 562 of maker note 0x0096.
They were refused ("lossy after split") and opened as the camera's preview.
The 11 such CC0 raw.pixls.us files come from the D3400, D5, D5000, D5200,
D5500, D5600, D7000, Nikon 1 J5 and Z 6_2.
Decode them following the rule LightCraft derived by black-box analysis of
those 11 files (storytold/lightcraft#633), the same route the Huffman tables
took. Rows above the split are Huffman coded as before. From the split row on,
every pixel is one MSB-first word of bits - 4 bits: a prefix code for the
difference category followed by the top bits of the T.81 additional bits,
dequantised to the centre of the kept bin (fixed_words, ported from
LightCraft's fixed_word_table), with the same predictor and an unclamped
running value. The section ends the strip: whole rows (up to 4 more than the
image), then 0-7 one-bits.
A split file that fails to decode for any reason (a section that does not end
the strip, the all-ones word, truncation, size limits, a split row past the
image, a curve long enough to cover byte 562) stays unsupported, so it keeps
the embedded-preview fallback it had before. Apart from the all-ones word and
the padding, the content of the fixed-rate section is not validated: a damaged
word inside a well-formed section decodes without an error, as in LightCraft.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Port behavior and regression coverage into current modules; preserve current upstream behavior.
Integrate bounded Sony SR2 metadata decoding and ILCE-6000 image area; preserve newer bit-depth-aware camera profile application and explicit file crop precedence. Keep current parity counts and original metadata/crop tests.
Canon's subsampled lossless JPEG (Y 2x2 or 2x1, Cb/Cr per MCU) with its
own prediction chain, de-sliced into YCbCr planes, bilinear chroma, a
BT.601 inverse and the per-shot sRAW gains from ColorData (version 15),
developed like a three-sample LinearRaw so white balance, the fitted
camera look, tone and the Camera Raw dialog all apply. Verified on EOS
6D Mark II mRAW files; other bodies decode with a warning when their
gain offset is unknown.
`for_bands_grouped` always split the bands into `min(workers, jobs)` static
chunks, so on the default develop (jobs == workers) a worker held up on the
shared pool kept a fixed share of the bands waiting, where main's
`chunks_mut_limited` runs one rayon task per band. Now, when every worker may
hold a scratch, each band is its own task and borrows a scratch from a pool
(`par::chunks_mut_pooled`, at most one per worker, dropped as soon as no
waiting band can use it); the grouping stays for memory-limited jobs.
The fused bilinear/MHC, X-Trans and LinearRaw paths ran one band per worker
with no memory cap: a 262144 x 455 Bayer frame with bilinear needed 2.92 GB
at 15+ workers and failed with LimitExceeded under the default 2 GiB.
`working_memory` now caps those bands like AHD's and the fused paths use the
returned jobs. The fused LinearRaw and X-Trans band buffers are `band.min(h)`
rows, as charged (CodeRabbit, develop.rs:414).
Tests: a_slow_band_does_not_hold_up_the_bands_after_it and
fused_bands_at_once_are_capped_by_the_memory_limit fail without the fix;
for_bands_keeps_at_most_jobs_scratches_alive, chunks_mut_pooled_makes_one_state_per_worker_at_most,
multi-band and short X-Trans/LinearRaw fused-vs-full cases.
Co-authored-by: Valentin Klein <271991477+vangel76@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main reverted the #9455 merge, so a plain merge would drop the band-by-band
develop; keep this PR's demosaic.rs, develop.rs and development_limits.rs and
take all other upstream changes.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Main reverted the earlier #9455 merge, which also removed the AHD changes
that came in with it; keep this PR's AHD code and add clampf plus the
NaN-plane test from main.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Keep the AHD scratch/band changes; take clampf and the NaN-plane test from main
(main reverted the earlier #9455 merge, so the AHD changes stay on this branch).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* chore: resolve SonarCloud maintainability issues
* fix: format imports and fix effect_entry argument type
* fix: restore test closure and wildcard import that broke compilation
* refactor: split composite_to_surface to reduce cognitive complexity
The bands were split into four groups per worker, clamped to the number
of bands. A 24 MP raw has 126 bands, so with 32 workers every group held
one band and every band allocated and zeroed a fresh 16-plane scratch,
which cost more than the reuse saved. The bands are now split into one
group per worker, so the scratch is allocated once per worker whatever
the thread count. Pixels are unchanged.
Adds a test that runs 35 bands in groups of 9 on a 4-thread pool and
compares the result with the reference bit for bit, so the band index of
later groups and a scratch reused for several bands are checked.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Each AHD worker allocated scratch for a full 38-row band while
develop's memory check budgets h.min(32) + 6 rows, so short images
allocated more than was approved: a 262144 x 1 strip was budgeted
~106 MB and allocated ~580 MB (5.4x). The scratch is now sized from
the same row count as the budget, and the budget also counts the two
u32 vertical-sum rows, so budget == allocation (~109 MB there).
Pixels are unchanged.
Tests: ahd_scratch_fits_its_budget (budget equals the real scratch for
h = 1..5000), ahd_matches_reference_with_reused_scratch (1100 rows,
jobs 1 and 2, all four phases: one scratch serves many bands).
Co-authored-by: vangel76 <271991477+vangel76@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 630e32c4238df452d9d5e70357bdb968045ab8c3)
Every lossless-JPEG round-trip wrote a single DHT (id 0) shared by all
components, so the new per-component table mapping (comp_table, the t0/t1
split of the two-component loop) went untested, though real CR2 files
declare one table per component. testgen gains lj92_encode_tables (one DHT
per table id, Td chosen per component, code lengths rotated per table so
the tables differ); lj92_encode delegates to it with output unchanged.
New tests cover 2- and 4-component frames with distinct and mixed tables,
predictors 1, 4, 6 and 7, restart intervals and 16-bit 0/65535 extremes.
They now catch swapping t0/t1, pushing 0 into comp_table, and resolving
every component to tables[0]; they pass on main's decoder as well.
Co-authored-by: vangel76 <271991477+vangel76@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
MHC evaluated every pixel through a Vec of (offset, weight) taps per
colour, and bilinear branched on the site's colour for every pixel.
A Bayer row has two kinds of pixel that alternate, so both methods now
get one function per row kind with the colours as constants, which
the compiler turns into straight-line code and vectorizes across the row.
The arithmetic is unchanged: MHC's taps keep their order and weights
(kernel / 8, exact in f32), so every sum is the one the tap loop made;
bilinear's expressions are as they were. The tap-loop code stays under
cfg(test) as the reference of a test that compares both methods on all
four Bayer phases, odd sizes, out-of-range values, signed zeros,
infinities and NaN (a NaN may differ in sign or payload only).
Part of #9435.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The 3-sample and monochrome fill called `Scale::value` for every sample:
two modulo operations, checked multiplications and a bounds-checked
lookup in `BlackLevels::at` each time. With no per-column black deltas
the black level and range depend on the row and the column phase only,
so they are now worked out once per row and phase, and the samples go
through the same arithmetic as before ((v - black) / range, the finite
check, the white-balance multiply, the clamp). Frames with deltas, or a
repeat pattern wider than 16 columns, keep the per-sample path.
A test compares the new fill with `Scale::value` bit for bit for one and
three samples, several black patterns, deltas, a linearization table and
offset starting columns.
Part of #9435.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
FredApps
FredApps
texjer
Dylan M. Taylor
lucas perssonandClaude Opus 5.5
aiongg
vangel76andClaude Sonnet 5.5
aiongg