* Camera Raw: Whites, Blacks, Shadows and Highlights closer to Photoshop's
Measured as a black box on Photoshop 25.4's Camera Raw Filter.
Whites and Blacks are global curves of the grey level: the measured rows
per slider value, Whites first, after Shadows and Highlights. The
clipping directions adapt to the image's black and white points
(Whites up to 2.7 times stronger on a dark image, Blacks at least 1.2).
On ten public-domain photos the mean error against Photoshop drops from
4.2-36.8 to 0.9-8.9 levels of 255.
Shadows and Highlights are a local Laplacian filter (Paris et al. 2011,
fast form of Aubry et al. 2014) on the log luminance, the method Adobe
credits for process version 2012. Shadows raises pixels darker than
their surroundings by an amount set by their own and their
surroundings' tone, so big edges keep no halo, and keeps the 99th
percentile; Highlights pulls pixels brighter than their surroundings
toward them and keeps the 1st. The strengths are fitted to the photos
(mean error 2.8-9.4 levels, was 3.7-14.3); Photoshop's exact operator
was not recovered.
The tone stages decode light with the document's curve: filter.cameraRaw
stores encodingGamma (1.8 for ProPhoto, where raws open).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Camera Raw Shadows/Highlights: no contour lines on smooth gradients
Twelve reference values over a raw's whole range left more than a stop
between them, and the remapping changes within half a stop: coefficients
interpolated across that gap drew contour lines in out-of-focus areas
(Shadows +100 on a Nikon D4 raw). References are now an eighth of a stop
apart over the image's tones (the deepest 0.1 % share the lowest), on a
copy at most 1600 px across whose change comes back to full size by a
guided upsampling, so a 16-megapixel image takes about 2 s per slider and
the preview matches the result.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Camera Raw on raw opens: Shadows/Highlights in linear light, D4 look and black point as Photoshop
Measured black-box on 29 Nikon D4 NEFs opened through Camera Raw in Photoshop 25.4 (sliders at
+-100 and +50, exposure +-1) against our own linear development of the same files.
- Shadows and Highlights of the open-time dialog now act while developing, in linear light before
the tone curve, as Camera Raw does for raw files (exposure -1 gives the same gains). The log2
gain is slider/100 * C(B - anchor): B is a guided-filter base of the log2 luminance on a copy
1232 px on its long side, the anchor is image-adaptive (Shadows: log2 power mean p=1.5 of the
linear luminance; Highlights: mean log2 luminance). New module photocraft_raw::tone; the
preview undoes the camera curve to show the same (photocraft_raw::retone).
- The D4 profile copies Camera Raw's default rendering: look matrix folded into the
ForwardMatrix, per-channel curve, and an adaptive black point (0.33 x the 0.1th percentile of
the luminance, Sensor::black_point).
Against Photoshop on the 29 frames (1/4 size, log2 Y): default rendering median difference
0.29 -> 0.04 stop; correlation of the slider's effect Shadows +100 0.96, -100 0.91, Highlights
-100 0.87, +100 0.85 (the previous filter model: 0.68 and 0.55 on DSG_3464).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs: the open-time Camera Raw re-develop covers Shadows and Highlights
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Camera Raw raw-stage Shadows: keep shadow contrast as Photoshop does
The user saw Photoshop's shadows sink deeper with more contrast. Measured on the 29 D4 NEFs:
with Shadows +100 our shadow detail had 0.79 of Photoshop's local contrast and shadows sat 0.38
stop lighter. Two causes:
- The black point now comes before the Shadows / Highlights gain, as in Camera Raw (lifting
first let the lift wash out the black point's deep-shadow contrast).
- The guided upsampling's epsilon goes from 0.01 to 1 stop^2: the change follows strong edges
but stays smooth inside texture (with 0.01 it followed every detail and flattened it). Near
strong edges the effect still matches Photoshop as well as elsewhere.
Shadows +100: shadow detail 0.79 -> 0.91, shadow bias +0.38 -> +0.16 stop; effect correlation
Shadows -100 0.906 -> 0.946, Highlights -100 0.870 -> 0.886, others unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Camera Raw on raw opens: Clarity, Texture and Dehaze in linear light, as Photoshop
Measured black-box from Photoshop 25.4 opening 29 Nikon D4 NEFs through Camera Raw (descriptor
keys Cl12, CrTx, Dhze; each at ±100 and +50, and +100 at exposure −1, 348 opens). All three act
before the tone curve: the change at exposure −1 equals the one at 0 (correlation 0.98–0.99).
- Clarity: log2 luminance split into bands between guided-filter bases (radii 2, 8, 32, 128 on
the 1232 px work copy), each band scaled by a measured gain that follows its base's tone, plus
a tone term; carried to full size with the Shadows / Highlights change.
- Texture: Gaussian bands σ 0.5…16 on a half-size copy, each through its measured response
(small detail raised more than strong edges, flat areas unchanged), guided-upsampled.
- Dehaze: He, Sun and Tang's dark channel prior on linear ProPhoto (dark channel in Display P3,
airlight from the brightest 0.1 % of it, guided-smoothed transmission), w 0.8 at +100;
−100 a uniform veil. The black point stays the default rendering's.
Against Photoshop, PhotoCraft's develop vs Photoshop's open on all 29 frames (median effect
correlation, log2 luminance at 1/4 size, Texture at 1/2): Clarity +100 0.81, −100 0.85;
Texture +100 0.85, −100 0.88; Dehaze +100 0.61 (0.78 in gamma 2.2: the crushed shadows
differ), −100 0.96. Shadows / Highlights unchanged (0.964 / 0.886).
The open-time Camera Raw dialog re-develops for them and its preview shows them (retone).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
The per-file colour fit (`fit_preview_look`) converted the embedded JPEG to sRGB only when the JPEG
carried an ICC profile, and otherwise read it as sRGB. A Sony body set to Adobe RGB writes no profile
into its previews; it states the colour space only in the raw's EXIF (ColorSpace Uncalibrated with
InteropIndex "R03", the DCF option file for Adobe RGB). The fit then targeted a desaturated,
hue-shifted reference, and the developed raw came out that way.
`photocraft_raw::embedded_preview_color_space` reads that EXIF (sRGB, Adobe RGB or unstated; never for a
DNG, whose previews are the converter's), and `preview_profile` uses it when the preview has no ICC
profile of its own. A preview's own profile still wins. LightCraft reads the same tags for the same
reason (storytold/lightcraft#778).
Measured on 19 private Sony ILCE-1M2 ARWs, all shot in Adobe RGB, against the camera's JPEG read as
Adobe RGB (median CIE76 ΔE at 240 px): full-frame 400 mm shots against blue sky 5.6-7.0 → 1.5-2.4,
their mean a* offset +4.3…+4.9 → +0.7…+1.1; every file improved, mean of the medians 4.9 → 2.2.
Tests: `exif_colour_space_of_the_previews` (sRGB, Adobe RGB; R98, a missing interop IFD, other values,
a DNG and junk are unstated) and `a_preview_without_a_profile_takes_the_raws_exif_colour_space` (fails
without the EXIF fallback; the preview's own ICC profile wins). The hostile-input harness calls the new
function.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A one-sample LinearRaw DNG (VueScan film scans, Leica Monochrom) was
rejected as "LinearRaw data with 1 samples per pixel". The DNG decoder
now accepts it and develop renders it as neutral grey: the one sample
fills all three channels, white balance, colour matrices and camera
looks do not apply, and the output is R = G = B in 16-bit ProPhoto.
io skips the preview colour fit for such files.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* raw: fit non-DNG colour to the camera's embedded JPEG
* raw look: clippy (as_chunks) and rustfmt after merging main
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* DNG: lossy compression 34892 (baseline JPEG tiles)
Adobe's lossy DNG (compression 34892) decodes: each tile is a complete
baseline JPEG (8-bit luma at tile size), decoded by zune-jpeg into the
tile geometry (a mismatch is an error), widened 0xFF -> 0xFFFF like
Photoshop's read of 8-bit CFA data. Predictors and the rest of the
pipeline are unchanged.
testgen grows LossyJpegTiles (image-rs encodes the Q60 fixtures behind
the testgen feature), so CI runs a decode oracle: a smooth ramp stays
within a few quantisation steps of the 8-bit source, and a tile cut
mid-stream fails cleanly.
* Lossy DNG: check the tile JPEG header before decoding; zune deps go workspace
decode_headers() runs and the (width, height) == (s.w, rows) check fires
before decode(), so a hostile SOF cannot allocate the decoder's buffers
(up to 16384x16364) before the size mismatch is caught. The mismatch has
a regression test that swaps a 32x32 tile JPEG for a 16x16 one. zune-core
and zune-jpeg move to [workspace.dependencies] next to image and flate2.
* chore(ui): format prefs_ui (the workspace fmt gate is red on main until #2417 lands)
* Lossy DNG: 3-sample LinearRaw (YCbCr) decodes to RGB; tiles size-check
Real lossy DNGs are 3-sample YCbCr LinearRaw, not single-channel CFA:
the tile now decodes through zune-jpeg's YCbCr-to-RGB output when
SamplesPerPixel is 3 (gray stays for CFA), the decoded length is checked
against the tile's sample count, and other sample counts say unsupported.
The generator writes RGB tiles for 3-sample specs. A LinearizationTable
maps the decoded samples in the develop stage as before - nothing to add
in the sensor. Test: a 3-sample ramp with 64-step channel separation
(surviving Q60 chroma quantisation) decodes within the quality band.
* Web: leave the JPEG XL DNG decoder out of the wasm (0.88 MB, 23.1 MiB now)
The 0.6.0 web build went over packaging/web/package.sh's 24 MiB gate (26,980,503 bytes on
782bcb67). Today's landings added 1.34 MB; the biggest single item was jxl-oxide (#2047, JPEG
XL-compressed DNG): ~0.6 MB of jxl_* code plus brotli-decompressor and tables, 0.88 MB after
wasm-opt. 0c72d954 brought the wasm back to 25,144,413 bytes (21 KB under the gate) by deflating
catalogs and fonts; this takes it to 24,262,088 bytes (23.14 MiB, 904 KB of headroom).
jxl-oxide is now a native-only dependency of photocraft-raw. On wasm32 a JPEG XL DNG segment
returns RawError::Unsupported ("can't be opened in the browser version of PhotoCraft; open it
in the desktop app"), like the other cfg-gated web features; every other DNG compression and
raw format still opens in the browser. Native builds are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* xtask i18n-coverage: read the deflated catalog! registry entries
0c72d954 changed the LANGUAGES registry from include_str!("xx.tsv") to catalog!("xx"), so
`cargo xtask i18n-coverage` failed with "language `ja` has no include_str! catalog". Parse the
new form (the macro argument is the catalog's file stem).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* DNG: JPEG XL compression 52546 (DNG 1.7, Samsung Expert RAW)
Decode JPEG XL-compressed DNG strips and tiles with jxl-oxide (pinned
=0.12.6), with geometry and channel checks, Limits/AllocTracker budgets
and catch_unwind around the decoder. Warn when a DNG carries a
ProfileGainTableMap (tag 52525), which is not applied yet.
Tests encode fixtures in-process with zune-jpegxl (dev-dependency only).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* raw README: list Deflate DNG beside JPEG XL after rebasing onto #1971
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Integer DNGs with zlib-compressed strips or tiles (compression 8) now decode, with TIFF predictor 1
and 2 (wrapping at the sample's container). Output is capped at the tile size; floating-point
predictor 3 and lossy DNG keep their unsupported errors. flate2 was already a workspace dependency.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RAF's header, metadata records and Fuji IFD give size, packing (16-bit, 12-bit LSB-first,
14-bit in 32-bit words), layout, black, WB, crop and exposure bias; X-Trans and other
periodic non-Bayer CFAs go through a new edge-directed demosaic. Compressed and early
SuperCCD RAF stay unsupported (preview fallback). Tests use synthetic files only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sony cRAW files with no BlackLevel tag take the 512 their tone curve gives the black
code instead of 0 (the ILCE-7 no longer develops washed out; the RX0 corpus oracle moves
to 512 too). cameras.rs gains measured image areas without a full profile: the Nikon
D800E and Sony ILCE-7 crop like the camera's JPEG.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The five RAW readers cast the u32 Orientation to u16 before the 1..=8 check, so a LONG
65538 became 2 and rotated the image. A shared tiff::orientation checks the range on
the u32 first. Fixes#1817.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* CR2: read ColorData's as-shot white balance as words
CANON_COLOR_DATA is an UNDEFINED entry: uints() hands its bytes back one
by one, but the WB_RGGBLevelsAsShot offsets count 16-bit words. Reading
word i at entry offset i*2 (file byte order, bounds-checked) finds the
quadruple at 0x47 in the corpus PowerShot SX50 HS: [1613, 951, 951, 1605]
-> [1.70, 1.0, 1.69], matching the white balance its own DNG conversion
neutralises (AsShotNeutral -> [1.58, 0.955, 1.666] through the color
matrix). With this the file's balance is used instead of the grey-world
estimate; the regression test pins the words and the missing estimate
warning. (The remaining brightness gap between the CR2 and the DNG of
the same camera is the CR2 path's exposure normalisation, to be looked
at next; the estimate was never what brightened the file.)
* cr2: drop the unused glob import in the corpus test
* CR2 WB: synthetic word-offset oracle, entry bounds, corpus test behind the feature
testgen writes ColorData as Canon does, UNDEFINED bytes that are 16-bit words
with the PowerShot's WB_RGGBLevelsAsShot at word offset 0x47, and the new
color_data_wb_as_words test checks the decoded balance in every CI run (the
corpus oracle stays behind the corpus feature and now panics when its file is
missing instead of silently skipping). word() reads stay inside the entry's
own byte count, and the shared inspect example moves back to the corpus PR.
The DNG default crop was applied only when both DefaultCropOrigin and
DefaultCropSize were present, so a file carrying just the size was
developed at the whole active area. Each tag now takes its own default
from DNG 1.7.1.0: the origin (0, 0), the size the whole image. The
existing bounds check is unchanged, so an origin-only crop that no
longer fits still falls back to the active area with a warning, and a
tag with the wrong number of values is still reported as incomplete.
Fixes#950.
NEF carries no colour description, so D4 frames developed with the camera channels taken as
sRGB primaries and no tone curve (dark, pale), the sensor's optical-black columns showed as a
purple stripe on the right, and clipped highlights turned pink (green saturates first).
crates/raw/src/cameras.rs holds camera profiles measured from the cameras' own output (no
third-party tables), applied to TIFF/EP raws by make, model and sensor size. Sensor gains
tone_curve (like DNG ProfileToneCurve, per channel in linear ProPhoto), folded into the gamma
LUT; other cameras develop exactly as before.
Nikon D4, from 148 uncompressed NEFs of one body (ISO 100–10000):
- image area 4928x3280 at (8, 6): where the camera's own full-size JPEG sits, found by aligning
the two (2 columns of optical black on the left, 50 + 8 on the right);
- black 0 (the data is black-subtracted in camera: the darkest image pixels reach 0);
- white 15520: green saturates at 15546–15672, red and blue at 15687–16383, so one level just
below green makes clipped highlights white without moving the white balance;
- ForwardMatrix (white-balanced camera → XYZ D50) and tone curve fitted to the embedded JPEGs
(Picture Control Standard, Active D-Lighting off; 52 frames, smooth areas). On 15 held-out
frames PhotoCraft's development is 7.0/255 per channel from the camera's JPEG on average,
against 34 before.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The "is this the undecoded compressed/packed ORF" guard summed the strip
byte counts and compared the sum with width × height × 2. Tiff::tag_uints
returns an empty list when the entry is missing or unreadable, so a file
whose IFD0 entry table is truncated (a partial download, a damaged card)
summed to zero and was reported as "Olympus compressed (or packed) ORF is
not decoded" — a format the user's camera does not produce — while the same
file cut a few bytes earlier or later was correctly reported as malformed.
Missing or unreadable strip byte counts are now RawError::Malformed before
the compression check. Regression test: a synthetic uncompressed ORF cut
right before its StripByteCounts entry is Malformed, and no prefix of the
file reads as the compressed variant.
Co-authored-by: wstock <1394687+wstock@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* raw: decode Nikon compressed NEF (lossless + lossy)
Nikon's default NEF (TIFF compression 34713) opened only as the embedded
8-bit JPEG preview. Decode the Huffman-coded strip: lossless and lossy
type 1 / 2, 12 and 14 bit, with the seeds and curve of maker note 0x0096
in the maker note's byte order. "Lossy after split" files stay
unsupported and keep the preview fallback.
Clean-room: format facts from prose write-ups (Clévy, Claff, ExifTool tag
names, T.81). The fixed Huffman tables are not in the files; they come
from LightCraft's black-box analysis of CC0 raw.pixls.us samples
(storytold/lightcraft#86, verified there on ~45 files from ~35 bodies).
No decoder code is shared: nefc.rs is written against this crate's TIFF
reader.
Also read Nikon BlackLevel (0x003d) in 14-bit units for 12-bit data, as
measured for LightCraft (D750 / D850 / Z 50).
Refs #50.
* raw: Nikon NEF without a BlackLevel tag is black-subtracted in camera
A D3200 NEF's darkest samples are 0-4 (noise around zero): assume 0
without the "black level not recorded" warning when the Nikon maker
note is present but has no 0x003d tag.
* raw: trim Nikon's right-edge padding columns
A D3200 NEF pads every row with 46 columns of 256 (exact in 97.5% of the
rows, lossy-coding noise elsewhere) after its 6034 image columns; they
developed as a pink stripe on the right. Trailing columns (at most 128)
that hold the last column's value in at least 90% of their rows, when
that value is dark, are left out of the active area and crop. A white-
clipped edge is never trimmed.
* raw: decode Sony cRAW, Panasonic RW2 (RawFormat 5) and uncompressed Olympus ORF
- sony.rs: compressed ARW (SonyRawFileType 2): 32-pixel runs as two
16-byte blocks of 11-bit max/min, 4-bit positions and 7-bit deltas,
then the SonyToneCurve (0x7010) to 14 bits. From Dietz (EI 2016) and
the RawDigger write-up; bit order and curve scale by observation.
- rw2.rs: RawFormat 5 12/14-bit packing (10/9 pixels per 16-byte LE
block), stored in 0x4000-byte pages split at 0x1FF8 (observed), with
PanasonicRaw CFA, black, linearity limit, WB and sensor borders.
RawFormat 4 and older stay unsupported (preview fallback).
- orf.rs: uncompressed 16-bit ORF (E-1, E-400): left-justified
samples, EXIF CFAPattern, Olympus ImageProcessing black / WB /
ValidBits / crop; compressed ORF stays unsupported but its maker-note
preview (CameraSettings 0x0101/0x0102) is now found.
- testgen: cRAW, RW2 and ORF encoders/builders; vendor_codecs tests
round-trip each codec; hostile tests fuzz/truncate all three.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* raw: README support matrix for cRAW, RW2, ORF and the clean-room gaps
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* raw: clean-room camera raw decoder and developer (photocraft-raw)
A new standalone crate (no workspace deps, wasm-safe, no unsafe) written
from TIFF 6.0, TIFF/EP, the DNG 1.7 spec, ITU-T T.81 Annex H and published
demosaicing papers:
- Lossless JPEG (LJ92) decoder: predictors 1-7, point transform, restart
intervals, combined Huffman+extra-bits fast table.
- DNG: uncompressed (8-16 bit, packed) and LJ92, strips and tiles, CFA and
LinearRaw, linearization, black level (repeat, deltas), white level,
active area, default crop, ColorMatrix/CameraCalibration/ForwardMatrix
interpolation by CCT, AnalogBalance, AsShotNeutral/WhiteXY,
BaselineExposure, orientation.
- CR2 (LJ92 slices, SensorInfo borders, ColorData as-shot WB, measured
black) and TIFF/EP raws (NEF/ARW/PEF uncompressed or LJ92, incl. Sony
lossless ARW tiles, Nikon/Sony documented black and WB tags).
- Develop: black/white scaling, WB with highlight clipping, bilinear /
Malvar-He-Cutler / AHD demosaic (rayon bands, sequential on wasm),
camera -> XYZ D50 -> ProPhoto, exposure, gamma 1.8, 16-bit, orientation.
- Embedded JPEG preview finder; clear Unsupported errors for CR3, RAF,
ORF, RW2, vendor-compressed NEF/ARW/PEF, lossy/JXL/float DNG, X-Trans.
- Limits checked before every allocation; testgen feature with an LJ92
encoder and DNG/CR2/TIFF-EP writers; round-trip, colour-reference,
truncation and corruption tests.
Registered as a standalone crate in xtask layers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* io: open camera raws through photocraft-raw (16-bit ProPhoto), preview fallback
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* raw: DNG GainMap opcode, fuzz targets, stress tools; open raws from the apps
- Apply DNG OpcodeList2 GainMap (lens shading, e.g. phone DNGs); report other opcodes.
- cargo-fuzz targets (develop, ljpeg) and a mutation stress example.
- Raw extensions in the desktop/web open dialogs and the batch folder list.
- file.open returns the import notes (how a raw was developed, preview fallback).
- codecs: the camera raw refusal points at photocraft-raw.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* raw: clippy 1.99 (is_multiple_of, clamp)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
DennisandClaude Opus 5.5
David MannandClaude Opus 5.5
Gert Pellin - GPEandswitch87
Elvis33LE
Kostis PapadakisandClaude Opus 5.5
Gotya-OFFCIALandClaude Opus 5.5
kayrockscreenprintingandClaude Opus 5.5
Books
Alex GorevskiandCopilot