mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-14 06:22:24 +00:00
craft-raw: Fujifilm RAF (#1016)
* craft-raw: the file-format layer of camera raw files, with DNG and CRW A new crate for #587 and #778: bytes in, the sensor data out as a neutral RawImage (samples, CFA, levels, active area, crop, orientation, colour tags, opcode lists, profile tables), and back for DNG. One Cargo feature per format: dng (read and write), jxl (JPEG XL DNG), crw (through craft-crw). The base is LightCraft's crates/raw at 5ff567a4: dng, dngwrite, tiffraw, unpack, ljpeg, jxl, semantic, preview, the parsing half of opcodes, profile and gaintable, the model and probe. Demosaicing and everything after it stay with the develop pipeline. Ported from PhotoCraft's and FilmCraft's DNG readers, each with a test: the default crop's per-tag defaults, an unusable crop falling back to the active area, non-finite white levels, the white default of a linearized image, the AsShotWhiteXY range, 8-bit-only lossy JPEG, lossy tiles that must cover their part of the image, and the CinemaDNG frame tags. * craft-raw: Hasselblad 3FR and FFF (feature hasselblad) RawFormat::CfaTiff decodes: the DNG-layout reader plus what these files add. Uncompressed data or Hasselblad's lossless JPEG (selection value 8), RGGB when no CFAPattern is stated, an FFF's colour matrix from the 3FR header it embeds, and the 8-bit RGB preview handed on as a baseline TIFF. Moved from LightCraft's vendor/hasselblad.rs at 5ff567a4 with its tests. Ported from PhotoCraft's reader: four BlackLevel values without a BlackLevelRepeatDim are one per CFA site in a file that is not a DNG. * craft-raw: Minolta MRW, Epson ERF, Mamiya MEF and Kodak DCS520C Four formats only LightCraft decodes, one feature each (mrw, erf, mef, kodak), moved from its vendor modules at 5ff567a4 with their tests; the MRW block container comes from lightcraft-meta. src/shared.rs holds the level helpers the maker decoders share. ERF and MEF state no white balance: the estimate LightCraft made from the preview is develop-side work, so wb_multipliers is None for them here. * craft-raw: shared scan budget for tagged previews, tile-sized lossy JPEG budget, bound on Deflate tile padding Ported from storytold/lightcraft#3695. Candidate preview ranges are deduplicated and charged to one byte budget and one header-walk budget; a lossy JPEG decoded for a tile may hold no more samples than the tile rounded up to whole MCUs (and never more than Limits::max_lossy_tile_samples); a Deflate chunk whose nominal size is mostly padding outside the image is refused before it is inflated. * craft-raw: lossless JPEG sample bound per referenced byte; previews larger than the file are passed over Ported from storytold/lightcraft#3910 (the parts whose code is in craft-raw today). check_image refuses a lossless JPEG image that declares more than 32 samples per referenced byte, counted once per chunk that points at the bytes so identical tiles stored once still decode. The Hasselblad RGB preview search passes over an image whose pixel bytes exceed the file and still takes a smaller complete one. * craft-raw: test for the Deflate padding bound * craft-raw: Pentax PEF and Samsung SRW Moved from LightCraft crates/raw (vendor/pef.rs, vendor/srw.rs) at 5ff567a4. The Huffman difference decoder and bit reader that PEF shares with the Nikon, Panasonic and Sony codings become the huffman module. * craft-raw: Canon CR2, with sRAW and mRAW Moved from LightCraft crates/raw (vendor/cr2.rs, vendor/cr2/sraw.rs) at 5ff567a4. The white-balance presets a CR2 lists come back as data in ColorData::kelvin_presets (the ordering from LightCraft's kelvin_presets.rs; the temperature and tint arithmetic stays with the developer). * craft-raw: an sRAW preview larger than the file is refused before its buffer is reserved Ported from storytold/lightcraft#3910 (cr2::sraw::read_preview). * WIP craft-raw: Nikon NEF/NRW (does not compile yet: limits threading, develop() calls in tests) * craft-raw: Nikon NEF/NRW, with PhotoCraft's maker-note compression routing and 12-bit D4 white level * craft-raw: Panasonic RW2 / RWL / RAW, every raw format * craft-raw: RW2 format 8 refuses strips stating fewer bits than samples Ported from storytold/lightcraft#3910. * craft-raw: RW2 format 8 refuses samples above the declared bit depth Ported from storytold/photocraft#11838. * craft-raw: the RW2 format 8 robustness seed carries a valid stream Its random payload decoded only while out-of-depth samples were clamped; mutations now start from a file that decodes. * craft-raw: Fujifilm RAF, uncompressed and compressed, with lens corrections * craft-raw: compressed RAF stripes cap their capacity hint at the stripe's bits Ported from storytold/lightcraft#3910. * craft-raw: test RAF orientation and camera from the embedded JPEG's Exif * craft-raw: RAF embedded preview, and its range from the header alone (bounded_preview) --------- Co-authored-by: radialmonster <radial@gmail.com> Co-authored-by: wnanbei <wnanbei@gmail.com>
This commit is contained in:
14 files changed
+2014
-8
No files matched your search
@@ -93,7 +93,7 @@ crates/raw (craft-raw) is moved from LightCraft, with behaviour ported from two
|
||||
- LightCraft, crates/raw (src/lib.rs, dng.rs, dngwrite.rs, tiffraw.rs, unpack.rs, ljpeg.rs, jxl.rs,
|
||||
opcodes.rs, profile.rs, gaintable.rs, semantic.rs, preview.rs, vendor/crw.rs, vendor/hasselblad.rs,
|
||||
vendor.rs (shared level helpers), vendor/mrw.rs, vendor/erf.rs, vendor/mef.rs, vendor/kodak.rs, vendor/pef.rs,
|
||||
vendor/srw.rs, vendor/cr2.rs, vendor/cr2/sraw.rs, vendor/nef.rs, vendor/nefc.rs, vendor/nef_lens.rs, docs/nikon-lens-corrections.md, vendor/rw2.rs, kelvin_presets.rs (the preset ordering), tests/it/vendor_robust.rs and their tests) and crates/meta/src/mrw.rs (the MRW block container),
|
||||
vendor/srw.rs, vendor/cr2.rs, vendor/cr2/sraw.rs, vendor/nef.rs, vendor/nefc.rs, vendor/nef_lens.rs, docs/nikon-lens-corrections.md, vendor/rw2.rs, vendor/raf.rs, vendor/rafc.rs, vendor/raf_lens.rs, docs/raf-compression.md, docs/raf-corpus.sha256, kelvin_presets.rs (the preset ordering), tests/it/vendor_robust.rs and their tests) and crates/meta/src/mrw.rs (the MRW block container),
|
||||
revision 5ff567a4a2e067bef1b6bce152c11e7f727ccbaf.
|
||||
https://github.com/storytold/lightcraft
|
||||
Copyright (c) 2026 ArtCraft Team and the LightCraft contributors.
|
||||
|
||||
@@ -149,6 +149,7 @@ craft-<name> = { git = "https://github.com/storytold/craft-libs", rev = "<commit
|
||||
what PhotoCraft's and FilmCraft's readers added: container parsing and decompression to a neutral
|
||||
raw image (samples, CFA, black and white levels, active area, crop, orientation, colour tags,
|
||||
opcode lists, embedded previews), one Cargo feature per format. DNG reading and writing,
|
||||
Hasselblad 3FR/FFF, Minolta MRW, Epson ERF, Mamiya MEF, Kodak DCS, Pentax PEF, Samsung SRW, Canon CR2, Nikon NEF/NRW, Panasonic RW2, Fujifilm RAF and Canon CRW (through
|
||||
Hasselblad 3FR/FFF, Minolta MRW, Epson ERF, Mamiya MEF, Kodak DCS, Pentax PEF, Samsung SRW, Canon CR2, Nikon NEF/NRW, Panasonic RW2 and Canon CRW (through
|
||||
Hasselblad 3FR/FFF, Minolta MRW, Epson ERF, Mamiya MEF, Kodak DCS, Pentax PEF, Samsung SRW, Canon CR2, Nikon NEF/NRW and Canon CRW (through
|
||||
`craft-crw`) so far; demosaicing and everything after stay with the develop
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "craft-raw"
|
||||
description = "Camera raw files, the file-format layer: container parsing, decompression and the sensor data out as a neutral raw image with its levels, crop, orientation, colour tags and embedded previews. DNG (read and write), Hasselblad 3FR/FFF, MRW, ERF, MEF, Kodak DCS, Pentax PEF, Samsung SRW, Canon CR2, Nikon NEF/NRW, Panasonic RW2 and Canon CRW so far."
|
||||
description = "Camera raw files, the file-format layer: container parsing, decompression and the sensor data out as a neutral raw image with its levels, crop, orientation, colour tags and embedded previews. DNG (read and write), Hasselblad 3FR/FFF, MRW, ERF, MEF, Kodak DCS, Pentax PEF, Samsung SRW, Canon CR2, Nikon NEF/NRW, Panasonic RW2, Fujifilm RAF and Canon CRW so far."
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
license.workspace = true
|
||||
@@ -11,7 +11,7 @@ repository.workspace = true
|
||||
workspace = true
|
||||
|
||||
[features]
|
||||
default = ["dng", "jxl", "crw", "hasselblad", "mrw", "erf", "mef", "kodak", "pef", "srw", "cr2", "nef", "rw2"]
|
||||
default = ["dng", "jxl", "crw", "hasselblad", "mrw", "erf", "mef", "kodak", "pef", "srw", "cr2", "nef", "rw2", "raf"]
|
||||
## Adobe DNG, reading and writing: uncompressed, lossless JPEG, lossy JPEG, Deflate.
|
||||
dng = ["dep:miniz_oxide", "dep:zune-jpeg", "dep:zune-core"]
|
||||
## JPEG XL compressed DNGs (DNG 1.7, compression 52546) via jxl-oxide (pure Rust, MIT OR Apache-2.0).
|
||||
@@ -36,6 +36,8 @@ cr2 = []
|
||||
nef = []
|
||||
## Panasonic RW2, Leica RWL and the older Panasonic / Leica RAW: every raw format (packed, 4, 6, 8 and 16-bit words).
|
||||
rw2 = []
|
||||
## Fujifilm RAF: uncompressed Bayer and X-Trans, lossless and lossy compressed, in-file lens corrections.
|
||||
raf = []
|
||||
## Canon CRW (CIFF) through craft-crw, the one CRW decoder.
|
||||
crw = ["dep:craft-crw", "dep:chrono"]
|
||||
|
||||
|
||||
@@ -56,6 +56,7 @@ One module and one Cargo feature per maker or container family. All are on by de
|
||||
| `cr2` | Canon CR2 | The lossless-JPEG frame of IFD 3 de-sliced (`0xc640`); the layout from `CR2CFAPattern` (`0xc5e0`); image area from `SensorInfo`; black measured on the masked columns; white from the saturation plateau; as-shot white balance from `ColorData` or `WBInfo`; the white-balance presets at their temperatures (`ColorData::kelvin_presets`). sRAW / mRAW (subsampled YCbCr, 4:2:2 and 4:2:0) come out as linear white-balanced RGB with three samples per pixel, the chroma gains measured against the file's own 16-bit RGB image in IFD 2 |
|
||||
| `nef` | Nikon NEF / NRW | Uncompressed (16-bit words, 12-bit packed, Coolpix word packing), the packed layouts labelled compression 34713, Nikon's Huffman coding (lossless and lossy types, 12 and 14 bits, "lossy after split") with the `0x0096` linearisation table, small raw (YCbCr 4:2:2) as linear white-balanced RGB; black from `0x003d` or the NRW block, as-shot white balance from `0x000c`, `CropArea` or the measured image area of bodies that record none, masked trailing columns left out of the active area, the D1X pixel aspect, and the in-file distortion correction as a `WarpRectilinear` opcode ([derivation](docs/nikon-lens-corrections.md)). High Efficiency (HE / HE★) is refused by name; the previews still read |
|
||||
| `rw2` | Panasonic RW2, Leica RWL, older Panasonic / Leica RAW | Packed blocks (formats 2, 5, 7), formats 4 and 6, format 8 prefix-coded strips and tiles (a sample outside the bit depth makes the strip corrupt), 16-bit words; black per colour from `0x001c`–`0x001e` (plus the 15 counts formats 4 and 6 store above it, 15 when no black is recorded); white from the saturation plateau; sensor borders as the active area and the in-camera aspect crop; mapped-out defect pixels filled; as-shot white balance from `0x0024`–`0x0026`; the in-camera distortion correction (`0x0119`) as a `WarpRectilinear` opcode. Raw formats 1 and 3 and compression 34826 are refused (no samples of them) |
|
||||
| `raf` | Fujifilm RAF | The RAF header, records and raw IFD; uncompressed Bayer and X-Trans (16-bit words, 12-bit LSB-first, 32-bit word packing); Fujifilm's lossless and lossy striped compression ([derivation](docs/raf-compression.md), [corpus](docs/raf-corpus.sha256)); black, as-shot white balance and, for bodies without colour matrices, the camera's neutral multipliers for Illuminant A and D65 (`ColorData::kelvin_presets`, a neutral pair); orientation, make and model from the embedded JPEG's Exif; the in-file distortion and vignetting corrections as opcodes; the embedded preview, also by range from the header alone (`bounded_preview`, `embedded_preview_from_range`). The pre-2010 FinePix layout without a raw IFD is refused |
|
||||
| `mrw` | Minolta / Konica Minolta MRW | The `\0MRM` block container (`PRD`, `WBG`, `TTW`), 12-bit packed and 16-bit word mosaics, as-shot white balance from `WBG`, white from the saturation plateau |
|
||||
| `erf` | Epson ERF (R-D1, R-D1s, R-D1x) | 12-bit samples in 16-byte blocks, black levels from the maker note, the measured picture origin. No white balance (the file states none) |
|
||||
| `mef` | Mamiya MEF (ZD) | Uncompressed 12-bit packed; default crop from the RGB preview's measured window. No white balance (the file states none) |
|
||||
@@ -150,6 +151,12 @@ Left as LightCraft has it, with the reason:
|
||||
limits (`0x000e`–`0x0010`); LightCraft measured those below the samples on extended-ISO shots and uses the
|
||||
saturation plateau. The ISO in the private IFD0 tags (`0x0017`, `0x0037`) is metadata, left to the
|
||||
metadata reader.
|
||||
- Fujifilm RAF is LightCraft's reader. PhotoCraft's compressed-RAF decoder is a port of the same `rafc.rs`
|
||||
(its own copy and photocraft#9066): a diff shows message wording, its parallel helper, and two overflow guards
|
||||
in the adaptive Rice state (`k < 31`, a saturating sum). Neither guard can trigger here: residuals are bounded
|
||||
by the sample depth and the count is halved at 64, so the sum stays below 2^23. The Kelvin scale of bodies that
|
||||
record only an A / D65 pair is kept as that pair (`KelvinPresets::from_neutral_pair`); LightCraft placed five
|
||||
points along it with CIE temperature maths, which stays with the developer.
|
||||
- Lossy JPEG is accepted for CFA data (one grey component) as in PhotoCraft; FilmCraft refuses it
|
||||
because DNG 1.4 allows the coding for LinearRaw only.
|
||||
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
# Fujifilm compressed RAF
|
||||
|
||||
The derivation record of `src/rafc.rs`, moved from LightCraft (`docs/raf-compression.md`) with the decoder. "Supplied" files were supplied to LightCraft and are not public; the CC0 ones are pinned in [raf-corpus.sha256](raf-corpus.sha256) (paths as in LightCraft's corpus).
|
||||
|
||||
## Format and implementation
|
||||
|
||||
`src/raf.rs` reads the RAF header, raw TIFF IFD, crop, CFA, black level and white balance. The big-endian compression field at byte 108 distinguishes uncompressed (0), lossless compressed (2) and lossy compressed (3); testing only the strip's first two bytes would misclassify an uncompressed 16-bit sample of `0x5349`.
|
||||
|
||||
`src/rafc.rs` handles the compression stream:
|
||||
|
||||
- A 16-byte `IS` header carries compression mode, sensor type (Bayer or X-Trans), bit depth, dimensions, padded width, 768-pixel stripe width, stripe count and six-row group count. It must agree with the TIFF dimensions and CFA.
|
||||
- A big-endian size table, padded to 16 bytes, locates independent entropy streams. Lossy files add a table of one quantizer byte per six-row group for each stripe, each table padded to 16 bytes.
|
||||
- Each six-row group is collated into three red, six green and three blue vectors. Bayer vectors have 384 samples; X-Trans vectors have 512 slots, including interpolation slots that consume no bits. The last stripe's padding is decoded for predictor state but excluded from the output.
|
||||
- Colour-vector pairs alternate R/G and G/B. Even columns lead odd columns by seven positions. Two preceding lines of each colour and the preceding line's edge values provide prediction context.
|
||||
- Signed prediction errors use adaptive Rice coding: a unary prefix followed by a context-dependent number of bits, with a bounded escape for large errors. Three pair phases have separate even/odd statistics. The 81 signed gradient contexts fold into 41 states.
|
||||
- Lossy quantizers change the error range and reconstruction step. Low-gradient sites retain finer precision (0, 1 or 2), with five folded contexts per precision and statistics that survive main-quantizer changes. The main context statistics reset when its quantizer changes. Their initial sum is `max(2, (range + 32) / 64)`; using 16 as the minimum breaks more heavily quantized files.
|
||||
|
||||
Stripes decode in parallel using the existing Rayon dependency. The entropy reader keeps up to 64 bits buffered and counts a unary prefix's zeros in one step instead of bit by bit; a property test checks it against reading bit by bit, errors included. Offsets, sizes, dimensions and entropy reads are checked; truncated data, impossible codes and inconsistent headers return errors. Unknown container compression modes, disagreements between container and stream modes, and compressed X-Trans layouts other than the supported CFA return errors. Header-only probing validates the tables and stripe ranges without decompressing the image.
|
||||
|
||||
## Verification
|
||||
|
||||
Complete sensor arrays were compared sample by sample with an external decoder used only as a black-box oracle (rawpy 0.27.1). No decoder source, Adobe data, camera matrices or profiles were used. This tool is not a product or test dependency.
|
||||
|
||||
| Coverage | Files |
|
||||
|---|---|
|
||||
| Supplied X-H2S | DSCF4192 uncompressed; DSCF4193 lossless; DSCF4194 lossy |
|
||||
| Supplied X-T4 | _DSF2355 / _DSF2356 uncompressed; _DSF2357 lossless; _DSF2358 lossy |
|
||||
| CC0 X-Trans | X-T2, X-T20, X-T4, X-H2, X-T5, X-M5, X-T50, X-E5 |
|
||||
| CC0 Bayer | GFX 50S (14-bit), GFX 100 (14/16-bit), GFX100S (16-bit lossless/lossy), GFX100RF (16-bit lossy) |
|
||||
|
||||
All 24 files matched the reference sensor arrays exactly. This covers 13 camera bodies and both older and current compression variants. Synthetic tests also cover 12-bit streams; a real compressed 12-bit camera file has not been verified. Support is selected from the file's layout, without a camera-name allowlist. Other bodies using this layout should decode, but this is not a claim that every Fujifilm model has been tested. Older FinePix layouts without the supported raw TIFF IFD remain unsupported.
|
||||
|
||||
The 17 CC0 files live in the gitignored corpus. Their published SHA-256 identities are pinned in [raf-corpus.sha256](raf-corpus.sha256); their source licence records were checked as CC0 1.0. `cargo xtask corpus --download` fetches them. `cargo test -p lightcraft-raw corpus_fujifilm_compressed_samples -- --nocapture` verifies each available file's checksum before checking its reference sensor sum and position-weighted sum. The test skips absent corpus files; the full-array comparisons were performed locally. No supplied media or large binary fixtures are committed.
|
||||
|
||||
Unconditional procedural tests cover Bayer/X-Trans, 12/14/16 bits, partial final stripes, quantizer changes, residual signs and escapes, malformed headers and truncated/random entropy streams. Regressions also check overflowing uncompressed row strides, unsupported modes and CFA layouts, and container/stream mode mismatches.
|
||||
|
||||
Local release decode measurements: about 0.16–0.24 seconds for 24–40 MP X-Trans files, 0.29 seconds for 50 MP Bayer, and 0.5–0.6 seconds for 100 MP Bayer. These measure sensor decoding, not the full develop/export pipeline. The buffered prefix count took 13–17 % off the decode time on an Apple M2 (8 cores, minimum of 5 runs, old and new build alternated): X-T2 178 → 152 ms, X-S10 lossless 265 → 226 ms, X-E5 442 → 385 ms, X-H2 448 → 378 ms, GFX 50S 522 → 435 ms, GFX 100 762 → 644 ms, with sensor arrays unchanged bit for bit.
|
||||
|
||||
## Sources
|
||||
|
||||
- [Fabian's original prose description of lossless RAF compression](https://capnfabs.net/posts/fuji-raf-compression-algorithm/): independent stripes, colour vectors, iteration order, predictors' neighbourhoods, Rice coding and gradient adaptation. Its linked implementations were not consulted.
|
||||
- [ExifTool Fujifilm tag-name documentation](https://exiftool.sourceforge.net/TagNames/FujiFilm.html): RAF header and raw-IFD metadata fields.
|
||||
- [Fujifilm's compression-mode explanation](https://digitalcamera-support-en.fujifilm.com/digitalcameraengpcdetail?aid=000008311&wd=compressed+raw).
|
||||
- [raw.pixls.us](https://raw.pixls.us/): CC0 camera samples, published checksums and licence metadata.
|
||||
- [rawpy's documented `raw_image` API](https://letmaik.github.io/rawpy/api/rawpy.RawPy.html): the complete sensor array includes margins, allowing comparisons before demosaic or colour processing.
|
||||
|
||||
Exact byte layouts, colour-vector masks, prediction formulas, adaptation parameters and lossy precision selection were established through sample analysis and black-box comparisons.
|
||||
@@ -0,0 +1,19 @@
|
||||
# Fujifilm RAF regression files; CC0 1.0, raw.pixls.us.
|
||||
# Published SHA-256 values retrieved 2026-10-07; see docs/raf-compression.md.
|
||||
018781e126842596fcf3876b3fb6cd8973dc3936ccc753ee8648618be8889287 corpus/raw/raf-fuji-gfx100-3773.raf
|
||||
0e7dad9819a748bb3d3be26bbf5770d4968e2d80b9cb41d97036bf7c27f7bfe7 corpus/raw/raf-fuji-gfx100-3775.raf
|
||||
14bed5ac8191d6f2a7c82984e3bf6470ad4740df329293eac42b8018cc6a668a corpus/raw/raf-fuji-gfx100rf-8091.raf
|
||||
a7bf8f061b603f7ef5fd29276eb6b2ac57355ca2f589a340117bfff0d03c065c corpus/raw/raf-fuji-gfx100s-4495.raf
|
||||
1bb4608b61861a3d9e85b0ca6c00689b8ba4620cbe8579374d958c21ed08edf0 corpus/raw/raf-fuji-gfx100s-4503.raf
|
||||
a7124d13910e0241a6b035154bc1fcdb7be4439371d043b1e1950a0251feb51e corpus/raw/raf-fuji-gfx50s-1435.raf
|
||||
28f56cc9784164209e92fae209bdcc78267bd692aa2fb964595485f7e93692ab corpus/raw/raf-fuji-xe5-8509.raf
|
||||
caeadb2666f6c5d9bbe32fbe692c2079b6a7e028f8514a13173f360656d08507 corpus/raw/raf-fuji-xh2-6001.raf
|
||||
109be38f59963c9b8ff289ebe1f5ec9168c52f609e5b601432bd194520b4605b corpus/raw/raf-fuji-xh2-6002.raf
|
||||
c76ce546e15727abb4ba201471010c554403176963a1a6b8cc017462606429e5 corpus/raw/raf-fuji-xm5-7748.raf
|
||||
f756ad5be551a8b3e1b503a8a6f7e8e637fba519c0b0f8c79dd58e94b0287602 corpus/raw/raf-fuji-xt2-865.raf
|
||||
a23045101b2912e64f154100d646c228c0d16585d21e95e0d1a006e537f7784c corpus/raw/raf-fuji-xt20-compressed.raf
|
||||
07ea642656074834ffe5c661870a612ffa2e80fc1218a05166ee966bbe5f6fae corpus/raw/raf-fuji-xt4-3914.raf
|
||||
871836938ffa695f4c41765ddb107c2703ce755f7f3ced7e10f9e04746f1734e corpus/raw/raf-fuji-xt4-3918.raf
|
||||
b2a067e7fda16338f15c2532caa12b238b2415dc1c75a96a6d24d3445cfd0ae3 corpus/raw/raf-fuji-xt5-6122.raf
|
||||
cddd7ae0c43f9280876e5fdcbdf8878718972ebd3d034d8affc8cfc6752d33dc corpus/raw/raf-fuji-xt5-6123.raf
|
||||
3a8a9ac6d92274969aaa1f55ea669191b98bb8f588d00ec8ec17afd8b3963dea corpus/raw/raf-fuji-xt50-7807.raf
|
||||
@@ -1,8 +1,10 @@
|
||||
//! A camera's white-balance presets at stated colour temperatures, as data.
|
||||
//!
|
||||
//! Some cameras record the multipliers they would apply for each of their white-balance presets, with the
|
||||
//! temperature the camera gives the preset (Canon CR2: the maker note's `ColorData`). For a raw without colour
|
||||
//! matrices they are the camera's own Kelvin scale. This crate reads and orders them; placing a white on the curve
|
||||
//! temperature the camera gives the preset (Canon CR2: the maker note's `ColorData`). Others record only the
|
||||
//! camera's neutral multipliers for CIE Illuminant A and D65 (Fujifilm RAF, raw IFD tag `0xf00c`):
|
||||
//! [`KelvinPresets::from_neutral_pair`] keeps that pair. For a raw without colour matrices they are the camera's own
|
||||
//! Kelvin scale. This crate reads and orders them; placing a white on the curve
|
||||
//! through them (temperature and tint) is the developer's arithmetic.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
@@ -14,6 +16,9 @@ const MAX_POINTS: usize = 32;
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub struct KelvinPresets {
|
||||
points: Vec<(f32, [f32; 3])>,
|
||||
/// Multipliers (green = 1) for Illuminant A and D65, when that pair is all the file records.
|
||||
#[serde(default)]
|
||||
pair: Option<[[f32; 3]; 2]>,
|
||||
}
|
||||
|
||||
fn usable(m: [f32; 3]) -> bool {
|
||||
@@ -35,7 +40,7 @@ impl KelvinPresets {
|
||||
let (a, b) = (log_point(w[0].1), log_point(w[1].1));
|
||||
w[0].0 < w[1].0 && b[0] > a[0] && b[1] < a[1]
|
||||
});
|
||||
ok.then(|| KelvinPresets { points: points.into_iter().map(|(k, m)| (k, [m[0] / m[1], 1.0, m[2] / m[1]])).collect() })
|
||||
ok.then(|| KelvinPresets { points: points.into_iter().map(|(k, m)| (k, [m[0] / m[1], 1.0, m[2] / m[1]])).collect(), pair: None })
|
||||
}
|
||||
|
||||
/// A curve from presets in any order, as a camera lists them by name (Canon's Daylight, Shade, Cloudy, …):
|
||||
@@ -57,10 +62,28 @@ impl KelvinPresets {
|
||||
KelvinPresets::new(kept)
|
||||
}
|
||||
|
||||
/// The presets as (Kelvin, multipliers with green = 1), ascending in Kelvin.
|
||||
/// The camera's neutral multipliers for Illuminant A and for D65; `None` unless both are positive and A is
|
||||
/// the warmer (more blue gain, less red: red rising and blue falling from A to D65, as on every preset curve).
|
||||
/// The temperatures belong to the illuminants, not the file, so placing points between them is the developer's
|
||||
/// colour arithmetic; [`Self::points`] is then empty.
|
||||
pub fn from_neutral_pair(a: [f32; 3], d65: [f32; 3]) -> Option<KelvinPresets> {
|
||||
if !usable(a) || !usable(d65) {
|
||||
return None;
|
||||
}
|
||||
let (pa, pd) = (log_point(a), log_point(d65));
|
||||
let unit = |m: [f32; 3]| [m[0] / m[1], 1.0, m[2] / m[1]];
|
||||
(pd[0] > pa[0] && pd[1] < pa[1]).then(|| KelvinPresets { points: Vec::new(), pair: Some([unit(a), unit(d65)]) })
|
||||
}
|
||||
|
||||
/// The presets as (Kelvin, multipliers with green = 1), ascending in Kelvin. Empty for a neutral pair.
|
||||
pub fn points(&self) -> &[(f32, [f32; 3])] {
|
||||
&self.points
|
||||
}
|
||||
|
||||
/// The Illuminant A and D65 multipliers (green = 1) of [`Self::from_neutral_pair`].
|
||||
pub fn neutral_pair(&self) -> Option<[[f32; 3]; 2]> {
|
||||
self.pair
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
@@ -110,4 +133,19 @@ mod tests {
|
||||
assert_eq!(KelvinPresets::from_unordered([(5000.0, g(1400.0, 1400.0)), (6000.0, g(1500.0, 1300.0))]), None);
|
||||
assert_eq!(KelvinPresets::from_unordered(std::iter::empty()), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_neutral_pair_is_kept_as_stated() {
|
||||
let p = KelvinPresets::from_neutral_pair([1.2, 0.8, 2.4], [2.0, 1.0, 1.5]).unwrap();
|
||||
assert_eq!(p.neutral_pair(), Some([[1.5, 1.0, 3.0], [2.0, 1.0, 1.5]]));
|
||||
assert!(p.points().is_empty());
|
||||
assert_eq!(KelvinPresets::from_neutral_pair([2.0, 1.0, 1.5], [1.5, 1.0, 3.0]), None, "D65 warmer than A");
|
||||
assert_eq!(KelvinPresets::from_neutral_pair([1.5, 1.0, 3.0], [1.5, 1.0, 3.0]), None, "one point twice");
|
||||
assert_eq!(KelvinPresets::from_neutral_pair([0.0, 1.0, 3.0], [2.0, 1.0, 1.5]), None);
|
||||
assert_eq!(KelvinPresets::from_neutral_pair([f32::NAN, 1.0, 3.0], [2.0, 1.0, 1.5]), None);
|
||||
assert_eq!(
|
||||
KelvinPresets::new([(2500.0, [1.0, 1.0, 3.2]), (4500.0, [1.8, 1.0, 1.9]), (8500.0, [2.8, 1.0, 1.2])]).unwrap().neutral_pair(),
|
||||
None
|
||||
);
|
||||
}
|
||||
}
|
||||
+14
-1
@@ -32,6 +32,7 @@
|
||||
//! | `kodak` | Kodak DCS520C TIFF: two-component lossless JPEG with a response curve |
|
||||
//! | `nef` | Nikon NEF and NRW: uncompressed, packed, Nikon's Huffman coding (lossless and lossy, 12 and 14 bits), small raw |
|
||||
//! | `rw2` | Panasonic RW2, Leica RWL and the older Panasonic / Leica RAW: packed, formats 4 and 6, format 8 prefix-coded strips, 16-bit words |
|
||||
//! | `raf` | Fujifilm RAF: uncompressed Bayer and X-Trans, Fujifilm's lossless and lossy compression, lens corrections |
|
||||
//! | `pef` | Pentax PEF: uncompressed (packed or 16-bit words) and Pentax's Huffman coding (compression 65535) |
|
||||
//! | `srw` | Samsung SRW: the uncompressed variants (compression 32769 words, 32770 at 12 bits packed) |
|
||||
//! | `hasselblad` | Hasselblad 3FR and FFF (and other CFA TIFFs in the DNG layout without a DNG version): uncompressed and Hasselblad's lossless JPEG; implies `dng` |
|
||||
@@ -89,6 +90,12 @@ pub mod opcodes;
|
||||
mod pef;
|
||||
mod preview;
|
||||
pub mod profile;
|
||||
#[cfg(feature = "raf")]
|
||||
mod raf;
|
||||
#[cfg(feature = "raf")]
|
||||
mod raf_lens;
|
||||
#[cfg(feature = "raf")]
|
||||
mod rafc;
|
||||
#[cfg(feature = "rw2")]
|
||||
mod rw2;
|
||||
#[cfg(feature = "dng")]
|
||||
@@ -106,7 +113,10 @@ mod unpack;
|
||||
#[cfg(feature = "dng")]
|
||||
pub use dngwrite::{DngCompression, DngWriteOptions, write_dng};
|
||||
pub use opcodes::{Opcode, OpcodeLists};
|
||||
pub use preview::{PreviewColorSpace, embedded_preview, embedded_preview_color_space, embedded_preview_covering};
|
||||
pub use preview::{
|
||||
PREVIEW_PREFIX, PreviewColorSpace, PreviewRange, bounded_preview, embedded_preview, embedded_preview_color_space, embedded_preview_covering,
|
||||
embedded_preview_from_range,
|
||||
};
|
||||
#[cfg(feature = "dng")]
|
||||
pub use semantic::{SemanticMask, semantic_masks};
|
||||
|
||||
@@ -340,6 +350,7 @@ impl RawFormat {
|
||||
|| (self == RawFormat::Cr2 && cfg!(feature = "cr2"))
|
||||
|| (matches!(self, RawFormat::Nef | RawFormat::Nrw) && cfg!(feature = "nef"))
|
||||
|| (self == RawFormat::Rw2 && cfg!(feature = "rw2"))
|
||||
|| (self == RawFormat::Raf && cfg!(feature = "raf"))
|
||||
|| (self == RawFormat::Pef && cfg!(feature = "pef"))
|
||||
|| (self == RawFormat::Srw && cfg!(feature = "srw"))
|
||||
}
|
||||
@@ -628,6 +639,8 @@ fn decode_in(bytes: &[u8], mode: Mode, limits: &Limits) -> Result<RawImage> {
|
||||
RawFormat::Nef | RawFormat::Nrw => nef::decode(bytes, mode, limits),
|
||||
#[cfg(feature = "rw2")]
|
||||
RawFormat::Rw2 => rw2::decode(bytes, mode, limits),
|
||||
#[cfg(feature = "raf")]
|
||||
RawFormat::Raf => raf::decode(bytes, mode, limits),
|
||||
#[cfg(feature = "pef")]
|
||||
RawFormat::Pef => pef::decode(bytes, mode, limits),
|
||||
#[cfg(feature = "srw")]
|
||||
|
||||
@@ -240,6 +240,9 @@ fn smallest_covering<'a>(found: &[&'a [u8]], largest: &[u8], min_edge: u32) -> O
|
||||
}
|
||||
|
||||
fn preview_covering(bytes: &[u8], min_edge: Option<u32>) -> Option<Vec<u8>> {
|
||||
if bytes.starts_with(b"FUJIFILMCCD-RAW") {
|
||||
return raf_jpeg(bytes).filter(|j| is_dct_jpeg(j)).map(|j| trim_eoi(j).to_vec());
|
||||
}
|
||||
let format = crate::probe(bytes);
|
||||
if matches!(format, Some(crate::RawFormat::Crw | crate::RawFormat::Mrw | crate::RawFormat::X3f)) {
|
||||
return scan_for_jpeg(bytes);
|
||||
@@ -299,6 +302,71 @@ fn bound_candidates(found: &mut Vec<&[u8]>, file_len: usize) {
|
||||
});
|
||||
}
|
||||
|
||||
/// Where a preview lives in a file, so a caller with a seekable file can read just that part of it.
|
||||
///
|
||||
/// Why it exists: a 56 MB Fuji RAF keeps its JPEG in the first 5.6 MB (measured: DSCF9492.RAF at
|
||||
/// 5.6 MB), so a grid thumbnail or a loupe preview has no reason to pull the remaining 50 MB —
|
||||
/// which on a network share is the difference between a moment and half a minute. A caller that has
|
||||
/// only bytes (`wasm`, tests) keeps using [`embedded_preview`].
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub struct PreviewRange {
|
||||
/// Byte offset of the preview in the file.
|
||||
pub offset: u64,
|
||||
/// Bytes to read at [`Self::offset`]: the container's declared preview extent, which may carry
|
||||
/// padding past the preview's end-of-image marker.
|
||||
pub window_len: u64,
|
||||
}
|
||||
|
||||
/// The preview's range in the file, when the file's header alone locates it.
|
||||
///
|
||||
/// `bytes` is the file's first [`PREVIEW_PREFIX`] bytes (more is fine). `None` means "read the file
|
||||
/// whole and call [`embedded_preview`]": the formats not covered here (a TIFF-based raw's pointer
|
||||
/// tag can sit in an IFD the prefix cuts short; CR3 scans box by box) and files whose pointer is
|
||||
/// damaged.
|
||||
///
|
||||
/// The caller reads `[range.offset, range.offset + range.window_len)` and passes those bytes to
|
||||
/// [`embedded_preview_from_range`], which returns exactly what [`embedded_preview`] returns for the
|
||||
/// whole file.
|
||||
///
|
||||
/// Pass a prefix that actually covers the pointer's range (read [`Self::offset`] +
|
||||
/// [`Self::window_len`] bytes, or start with [`PREVIEW_PREFIX`] and grow when it is not enough):
|
||||
/// `raf::header` clamps a slice to the buffer it was given, so a prefix that stops
|
||||
/// inside the declared extent yields a shorter `window_len` than the file really holds.
|
||||
pub fn bounded_preview(bytes: &[u8]) -> Option<PreviewRange> {
|
||||
let jpeg = raf_jpeg(bytes)?;
|
||||
let offset = (jpeg.as_ptr() as usize).checked_sub(bytes.as_ptr() as usize)?;
|
||||
if offset.checked_add(jpeg.len())? > bytes.len() {
|
||||
return None;
|
||||
}
|
||||
Some(PreviewRange { offset: offset as u64, window_len: jpeg.len() as u64 })
|
||||
}
|
||||
|
||||
/// The camera JPEG a RAF's header points to (clamped to `bytes`); `None` for any other file.
|
||||
#[cfg(feature = "raf")]
|
||||
fn raf_jpeg(bytes: &[u8]) -> Option<&[u8]> {
|
||||
bytes.starts_with(b"FUJIFILMCCD-RAW").then(|| crate::raf::header(bytes).ok()?.jpeg).flatten()
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "raf"))]
|
||||
fn raf_jpeg(_: &[u8]) -> Option<&[u8]> {
|
||||
None
|
||||
}
|
||||
|
||||
/// The preview inside `window`, which holds the file's bytes at `[range.offset, range.offset +
|
||||
/// range.window_len)`: the counterpart of [`bounded_preview`]. Padding past the end-of-image marker
|
||||
/// is dropped, exactly as [`embedded_preview`] drops it.
|
||||
pub fn embedded_preview_from_range(window: &[u8], range: PreviewRange) -> Option<Vec<u8>> {
|
||||
let preview = window.get(..usize::try_from(range.window_len).ok()?)?;
|
||||
if !is_dct_jpeg(preview) {
|
||||
return None;
|
||||
}
|
||||
Some(trim_eoi(preview).to_vec())
|
||||
}
|
||||
|
||||
/// How many bytes of a raw file are enough for [`bounded_preview`]: every covered format keeps its
|
||||
/// preview pointer in a fixed header, well inside this.
|
||||
pub const PREVIEW_PREFIX: usize = 4096;
|
||||
|
||||
/// The end of the DCT JPEG whose SOI is at `soi` (exclusive index after its EOI) and its component count,
|
||||
/// walking the marker segments and the entropy-coded data. `None` when it is not a complete, displayable
|
||||
/// (baseline or progressive) JPEG, so a stray `D8 FF` in other data is rejected. `*reach` is left at
|
||||
|
||||
@@ -406,3 +406,101 @@ fn picks_largest_dct_jpeg() {
|
||||
let _ = embedded_preview(&bytes[..n]);
|
||||
}
|
||||
}
|
||||
|
||||
/// A minimal RAF: the fixed header (JPEG offset/length at 84/88), the JPEG, then a raw tail the
|
||||
/// caller must not have to read. `jpeg_len` is written as declared, so a test can claim more than
|
||||
/// the JPEG holds the way a padded container does.
|
||||
#[cfg(feature = "raf")]
|
||||
fn raf_fixture(jpeg: &[u8], gap: usize, preamble: usize, declared_len: Option<usize>) -> Vec<u8> {
|
||||
let head = 160 + preamble;
|
||||
let jpeg_off = head;
|
||||
let mut f = vec![0u8; head];
|
||||
f[..16].copy_from_slice(b"FUJIFILMCCD-RAW ");
|
||||
f[84..88].copy_from_slice(&(jpeg_off as u32).to_be_bytes());
|
||||
f[88..92].copy_from_slice(&(declared_len.unwrap_or(jpeg.len()) as u32).to_be_bytes());
|
||||
// records directory and raw block are absent (offset 0), which `header` reads as "none"
|
||||
f.extend_from_slice(jpeg);
|
||||
f.extend(std::iter::repeat_n(0x33u8, gap));
|
||||
f
|
||||
}
|
||||
|
||||
/// A RAF's preview pointer sits at a fixed offset, so a header-sized prefix locates it and the
|
||||
/// caller reads just that range instead of the whole file — the whole point on a network share.
|
||||
#[cfg(feature = "raf")]
|
||||
#[test]
|
||||
fn bounded_preview_locates_a_rafs_jpeg_from_the_header_alone() {
|
||||
let jpeg = fake_jpeg(5000);
|
||||
let file = raf_fixture(&jpeg, 40_000, 0, None);
|
||||
// A prefix that covers the pointer's range: header + JPEG, and nothing of the tail.
|
||||
let prefix_len = 8192.min(file.len());
|
||||
let prefix = &file[..prefix_len];
|
||||
|
||||
let range = bounded_preview(prefix).expect("a RAF's preview is found from its header");
|
||||
assert_eq!(range.offset, 160);
|
||||
assert_eq!(range.window_len, jpeg.len() as u64);
|
||||
assert!(range.offset + range.window_len <= prefix_len as u64, "the prefix covers the whole range: {range:?} of {prefix_len}");
|
||||
assert!((range.offset + range.window_len) < file.len() as u64, "the range stops short of the tail: {range:?} of {}", file.len());
|
||||
|
||||
// What the caller reads, and what it holds afterwards: identical to the whole-file answer.
|
||||
let window = prefix.get(range.offset as usize..(range.offset + range.window_len) as usize).expect("the range lies inside the prefix");
|
||||
let bounded = embedded_preview_from_range(window, range).expect("the range holds the preview");
|
||||
assert_eq!(Some(bounded.clone()), embedded_preview(&file));
|
||||
assert!(bounded.ends_with(&[0xff, 0xd9]), "the preview ends at its EOI marker");
|
||||
|
||||
// A prefix that stops inside the declared extent is clamped by the container reader. The
|
||||
// caller then reads a short window; what comes back is the whole JPEG it holds (the EOI
|
||||
// marker is inside), never bytes past the window.
|
||||
let short = &file[..2000];
|
||||
let clipped = bounded_preview(short).expect("a pointer before the cut is still found");
|
||||
assert!(clipped.window_len < jpeg.len() as u64, "clamped: {clipped:?}");
|
||||
let window = short.get(clipped.offset as usize..).expect("inside");
|
||||
let preview = embedded_preview_from_range(window, clipped).expect("the bytes are a JPEG start");
|
||||
// No end-of-image marker inside the window: the bytes come back as they are, exactly as the
|
||||
// whole-file path returns an unclosed JPEG. The decoder refuses it later; nothing is invented.
|
||||
assert_eq!(preview, window, "an unclosed JPEG is handed on unchanged, never padded");
|
||||
assert!(preview.len() <= window.len(), "never past the window");
|
||||
}
|
||||
|
||||
/// The preview pointer may sit past the prefix (a long maker-note record before it), and a
|
||||
/// container may declare padding after the JPEG: both stay correct, and the padding is dropped.
|
||||
#[cfg(feature = "raf")]
|
||||
#[test]
|
||||
fn bounded_preview_handles_a_late_pointer_and_padding() {
|
||||
let jpeg = fake_jpeg(300);
|
||||
let late = raf_fixture(&jpeg, 0, 9000, None);
|
||||
let range = bounded_preview(&late).expect("the whole file is always a valid prefix");
|
||||
assert_eq!(range.offset, (160 + 9000) as u64);
|
||||
let padded = raf_fixture(&jpeg, 500, 0, Some(jpeg.len() + 500));
|
||||
let range = bounded_preview(&padded).expect("a padded extent is still located");
|
||||
assert_eq!(range.window_len, (jpeg.len() + 500) as u64);
|
||||
let window = padded.get(range.offset as usize..(range.offset + range.window_len) as usize).expect("the window is inside the file");
|
||||
let preview = embedded_preview_from_range(window, range).expect("the padding is dropped");
|
||||
assert_eq!(preview, embedded_preview(&padded).expect("the whole-file answer agrees"));
|
||||
assert!(preview.ends_with(&[0xff, 0xd9]));
|
||||
}
|
||||
|
||||
/// Without a usable header or pointer the caller must read the file whole: the answer is `None`,
|
||||
/// never a wrong range.
|
||||
#[cfg(feature = "raf")]
|
||||
#[test]
|
||||
fn bounded_preview_declines_what_the_header_cannot_answer() {
|
||||
let jpeg = fake_jpeg(64);
|
||||
let file = raf_fixture(&jpeg, 0, 0, None);
|
||||
assert_eq!(bounded_preview(&jpeg), None, "not a raw at all");
|
||||
assert_eq!(bounded_preview(b""), None);
|
||||
if let Some(short) = file.get(..64) {
|
||||
assert_eq!(bounded_preview(short), None, "a header cut short has not shown the pointer yet");
|
||||
}
|
||||
// a pointer that claims more than the file holds: `header` clamps the slice to the buffer,
|
||||
// so the range never runs past the end (no out-of-bounds read can follow from it)
|
||||
let mut lying = raf_fixture(&jpeg, 4096, 0, None);
|
||||
lying[88..92].copy_from_slice(&(u32::MAX / 2).to_be_bytes());
|
||||
let clamped = bounded_preview(&lying).expect("a pointer inside the file is still usable");
|
||||
assert!(clamped.offset + clamped.window_len <= lying.len() as u64, "clamped to the file: {clamped:?}");
|
||||
// a range the window cannot satisfy yields nothing
|
||||
assert_eq!(embedded_preview_from_range(&jpeg, PreviewRange { offset: 0, window_len: u64::MAX }), None);
|
||||
// a TIFF-based raw: its pointer tag lives in an IFD this prefix may cut short
|
||||
let mut tiff = b"II*\0\x08\0\0\0".to_vec();
|
||||
tiff.extend_from_slice(&[0u8; 64]);
|
||||
assert_eq!(bounded_preview(&tiff), None);
|
||||
}
|
||||
@@ -0,0 +1,536 @@
|
||||
//! Fujifilm RAF — uncompressed, lossless compressed and lossy compressed Bayer and X-Trans.
|
||||
//!
|
||||
//! Sources: the ExifTool FujiFilm tag-name documentation (RAF header record tags `0x0100` RawImageFullSize,
|
||||
//! `0x0110` RawImageCropTopLeft, `0x0111` RawImageCroppedSize, `0x0131` XTransLayout; raw-IFD tags `0xf001`
|
||||
//! width, `0xf002` height, `0xf003` bits, `0xf007` strip offset, `0xf008` strip byte count, `0xf00a` black
|
||||
//! levels, `0xf00e` WB_GRBLevels) and black-box analysis of CC0 samples from raw.pixls.us:
|
||||
//!
|
||||
//! - The file starts with a 16-byte magic, version and camera name; at byte 84 eight big-endian `u32`s give the
|
||||
//! embedded JPEG (offset, length), the header-record directory (offset, length) and the raw block (offset,
|
||||
//! length). The record directory is a `u32` count then `(u16 tag, u16 size, data)` records, big-endian.
|
||||
//! - In current bodies the raw block is a little-endian TIFF whose IFD0 has a `0xf000` IFD pointer to the raw
|
||||
//! IFD; offsets in it are relative to the raw block.
|
||||
//! - Sample packing, found by testing candidate bit orders for the smoothest image: 16-bit little-endian words;
|
||||
//! 12-bit LSB-first bit stream; 14-bit (and other depths) stored as little-endian 32-bit words read MSB-first.
|
||||
//! Raw-IFD flag `0xf006` = 1 selects the 32-bit-word packing at any depth: 12-bit files with the flag set
|
||||
//! (FinePix S1, X-A10, SL1000) are word-packed like the 14-bit ones, the 12-bit files without it are LSB-first.
|
||||
//! `0xf005`, when non-zero, is the row stride in 32-bit words.
|
||||
//! - CFA: the X-Trans layout record (36 bytes, 0 = R, 1 = G, 2 = B) is stored in reverse order and, reversed,
|
||||
//! is anchored at raw pixel (0, 0) — verified on X-Trans I and III samples by minimising the difference
|
||||
//! between horizontally / vertically adjacent "green" samples over all 72 orientations/phases, and by colour
|
||||
//! renders. Bayer bodies (X-A series) use RGGB at (0, 0), verified by colour renders.
|
||||
//! - Striped predictive compression is decoded by `rafc`; see `docs/raf-compression.md`.
|
||||
|
||||
use crate::shared::white_from_data;
|
||||
use crate::unpack::{unpack_lsb, unpack_words32_msb};
|
||||
use crate::{BlackLevel, Camera, Cfa, ColorData, Limits, Mode, OpcodeLists, Orientation, RawData, RawError, RawFormat, RawImage, Rect, Result};
|
||||
use craft_tiff::{ByteOrder, Ifd, Tiff};
|
||||
use rayon::prelude::*;
|
||||
|
||||
const CROP_TOP_LEFT: u16 = 0x0110;
|
||||
const CROPPED_SIZE: u16 = 0x0111;
|
||||
const XTRANS_LAYOUT: u16 = 0x0131;
|
||||
/// Raw-IFD flag `0xf006`: 1 in every 12- and 14-bit file here whose samples are packed into 32-bit words (the 12-bit
|
||||
/// ones are otherwise an LSB-first stream), 0 where they are an LSB-first stream or 16-bit words.
|
||||
const PACKED_WORDS: u16 = 0xf006;
|
||||
|
||||
/// The fixed-position header pointers.
|
||||
pub(crate) struct Header<'a> {
|
||||
pub jpeg: Option<&'a [u8]>,
|
||||
pub records: Vec<(u16, &'a [u8])>,
|
||||
pub raw: Option<&'a [u8]>,
|
||||
}
|
||||
|
||||
fn be32(b: &[u8], at: usize) -> Option<usize> {
|
||||
b.get(at..at + 4).map(|s| u32::from_be_bytes([s[0], s[1], s[2], s[3]]) as usize)
|
||||
}
|
||||
|
||||
fn slice(b: &[u8], off: Option<usize>, len: Option<usize>) -> Option<&[u8]> {
|
||||
let (o, l) = (off?, len?);
|
||||
if o == 0 || l == 0 {
|
||||
return None;
|
||||
}
|
||||
b.get(o..o.checked_add(l)?.min(b.len())).filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
pub(crate) fn header(b: &[u8]) -> Result<Header<'_>> {
|
||||
if !b.starts_with(b"FUJIFILMCCD-RAW") || b.len() < 108 {
|
||||
return Err(RawError::NotRaw);
|
||||
}
|
||||
let jpeg = slice(b, be32(b, 84), be32(b, 88));
|
||||
let raw = slice(b, be32(b, 100), be32(b, 104));
|
||||
let mut records = Vec::new();
|
||||
if let Some(dir) = slice(b, be32(b, 92), be32(b, 96)) {
|
||||
let n = be32(dir, 0).unwrap_or(0).min(4096);
|
||||
let mut p = 4;
|
||||
for _ in 0..n {
|
||||
let Some(h) = dir.get(p..p + 4) else { break };
|
||||
let (tag, size) = (u16::from_be_bytes([h[0], h[1]]), u16::from_be_bytes([h[2], h[3]]) as usize);
|
||||
let Some(d) = dir.get(p + 4..p + 4 + size) else { break };
|
||||
records.push((tag, d));
|
||||
p += 4 + size;
|
||||
}
|
||||
}
|
||||
Ok(Header { jpeg, records, raw })
|
||||
}
|
||||
|
||||
fn record<'a>(h: &Header<'a>, tag: u16) -> Option<&'a [u8]> {
|
||||
h.records.iter().find(|r| r.0 == tag).map(|r| r.1)
|
||||
}
|
||||
|
||||
/// A record holding two big-endian `u16`s.
|
||||
fn pair(h: &Header, tag: u16) -> Option<(usize, usize)> {
|
||||
let d = record(h, tag).filter(|d| d.len() >= 4)?;
|
||||
Some((u16::from_be_bytes([d[0], d[1]]) as usize, u16::from_be_bytes([d[2], d[3]]) as usize))
|
||||
}
|
||||
|
||||
pub(crate) fn raw_ifd(raw: &[u8]) -> Result<Ifd> {
|
||||
let tiff = Tiff::parse(raw).map_err(|_| RawError::Unsupported("RAF without a raw IFD (older FinePix layout)".into()))?;
|
||||
let off = tiff.ifds.first().and_then(|i| i.u64(0xf000)).ok_or_else(|| RawError::Unsupported("RAF without a raw IFD".into()))?;
|
||||
let (ifd, _) = craft_tiff::parse_ifd_at(raw, off, ByteOrder::Little, 0, false, &Default::default())?;
|
||||
Ok(ifd)
|
||||
}
|
||||
|
||||
/// IFD0 of the Exif block (APP1 `Exif\0\0`) of the embedded camera JPEG, where a RAF keeps its orientation, make
|
||||
/// and model. Walks the marker segments up to the start of scan; `None` when there is no readable Exif block.
|
||||
fn jpeg_exif_tiff(jpeg: &[u8]) -> Option<Tiff> {
|
||||
let mut at = 2usize;
|
||||
if !jpeg.starts_with(&[0xff, 0xd8]) {
|
||||
return None;
|
||||
}
|
||||
// every segment moves `at` forward by at least 4 bytes, so the walk ends within the JPEG's length
|
||||
while let (Some(&0xff), Some(&marker)) = (jpeg.get(at), jpeg.get(at + 1)) {
|
||||
if marker == 0xda || marker == 0xd9 {
|
||||
return None;
|
||||
}
|
||||
let len = usize::from(u16::from_be_bytes([*jpeg.get(at + 2)?, *jpeg.get(at + 3)?]));
|
||||
let segment = jpeg.get(at + 4..(at + 2).checked_add(len)?)?;
|
||||
if marker == 0xe1
|
||||
&& let Some(tiff) = segment.strip_prefix(b"Exif\0\0")
|
||||
{
|
||||
return Tiff::parse(tiff).ok();
|
||||
}
|
||||
at = at.checked_add(2 + len.max(2))?;
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// The camera name in the file header (bytes 28..60, NUL padded): the Exif `Model` of every sample.
|
||||
fn camera_name(b: &[u8]) -> String {
|
||||
let name = b.get(28..60).unwrap_or_default();
|
||||
let end = name.iter().position(|&c| c == 0).unwrap_or(name.len());
|
||||
String::from_utf8_lossy(name.get(..end).unwrap_or_default()).trim().to_string()
|
||||
}
|
||||
|
||||
/// The CFA: X-Trans from the (reversed) layout record, else RGGB.
|
||||
fn cfa(h: &Header) -> Cfa {
|
||||
match record(h, XTRANS_LAYOUT) {
|
||||
Some(l) if l.len() == 36 && l.iter().all(|&c| c <= 2) => Cfa { width: 6, height: 6, pattern: l.iter().rev().copied().collect() },
|
||||
_ => Cfa::bayer_static("RGGB"),
|
||||
}
|
||||
}
|
||||
|
||||
/// Unpack one row of `bits`-bit samples in Fujifilm's packing. `words32`: the raw IFD's `0xf006` flag is set, the
|
||||
/// samples are MSB-first in little-endian 32-bit words whatever the depth; otherwise 16-bit words, a 12-bit
|
||||
/// LSB-first bit stream, or (other depths) the same 32-bit words.
|
||||
pub(crate) fn unpack_row(src: &[u8], bits: u32, words32: bool, out: &mut [u16]) {
|
||||
match bits {
|
||||
16 => out.iter_mut().zip(src.as_chunks::<2>().0).for_each(|(o, c)| *o = u16::from_le_bytes([c[0], c[1]])),
|
||||
12 if !words32 => unpack_lsb(src, 12, out),
|
||||
_ => unpack_words32_msb(src, ByteOrder::Little, bits, out),
|
||||
}
|
||||
}
|
||||
|
||||
/// The Kelvin scale of a body without colour matrices, from tag `0xf00c` (`WB_GRBLevelsStandard`; tag number from
|
||||
/// `exiftool -v3` run as a black-box tool on the files): `G R B 17 G R B 21`, the camera's own neutral multipliers
|
||||
/// for Illuminant A and D65 (see [`crate::kelvin::KelvinPresets::from_neutral_pair`]).
|
||||
fn fuji_kelvin_presets(ifd: &Ifd) -> Option<crate::kelvin::KelvinPresets> {
|
||||
let v = ifd.f64s(0xf00c).filter(|v| v.len() >= 8)?;
|
||||
let grb = |at: usize, code: f64| {
|
||||
(v[at + 3] == code && v[at] > 0.0 && v[at + 1] > 0.0 && v[at + 2] > 0.0)
|
||||
.then(|| [(v[at + 1] / v[at]) as f32, 1.0, (v[at + 2] / v[at]) as f32])
|
||||
};
|
||||
let (a, d65) = (grb(0, 17.0)?, grb(4, 21.0)?);
|
||||
crate::kelvin::KelvinPresets::from_neutral_pair(a, d65)
|
||||
}
|
||||
|
||||
pub(crate) fn decode(bytes: &[u8], mode: Mode, limits: &Limits) -> Result<RawImage> {
|
||||
let h = header(bytes)?;
|
||||
let raw = h.raw.ok_or_else(|| RawError::Corrupt("RAF without raw data".into()))?;
|
||||
let ifd = raw_ifd(raw)?;
|
||||
let get = |t: u16| ifd.u64(t).map(|v| usize::try_from(v).map_err(|_| RawError::Limit("RAF IFD value too large"))).transpose();
|
||||
let (w, hgt) = (get(0xf001)?.unwrap_or(0), get(0xf002)?.unwrap_or(0));
|
||||
let bits = ifd.u64(0xf003).and_then(|v| u32::try_from(v).ok()).unwrap_or(0);
|
||||
let (off, len) = (get(0xf007)?.unwrap_or(0), get(0xf008)?.unwrap_or(0));
|
||||
if w == 0 || hgt == 0 || !(8..=16).contains(&bits) {
|
||||
return Err(RawError::Corrupt(format!("RAF raw IFD: {w}x{hgt}, {bits} bits")));
|
||||
}
|
||||
let n = w.checked_mul(hgt).filter(|n| *n <= limits.samples()).ok_or(RawError::Limit("image too large"))?;
|
||||
let src = raw
|
||||
.get(off..off.checked_add(len).ok_or_else(|| RawError::Corrupt("RAF strip size overflow".into()))?)
|
||||
.ok_or_else(|| RawError::Corrupt("RAF strip outside file".into()))?;
|
||||
let cfa = cfa(&h);
|
||||
let compression = be32(bytes, 108).unwrap_or(0);
|
||||
let compressed = match compression {
|
||||
0 => false,
|
||||
2 | 3 => true,
|
||||
_ => return Err(RawError::Unsupported(format!("Fujifilm RAF compression mode {compression}"))),
|
||||
};
|
||||
let mut data = Vec::new();
|
||||
if compressed {
|
||||
if cfa.width == 6 && cfa != Cfa::xtrans() {
|
||||
return Err(RawError::Unsupported("Fujifilm compressed RAF with an unknown X-Trans layout".into()));
|
||||
}
|
||||
let flag = if compression == 2 { 1 } else { 0 };
|
||||
if src.get(2) != Some(&flag) {
|
||||
return Err(RawError::Corrupt("RAF container and stream compression modes disagree".into()));
|
||||
}
|
||||
data = crate::rafc::decode(src, w, hgt, bits, cfa.width == 6, mode, limits)?;
|
||||
} else {
|
||||
let words32 = ifd.u64(PACKED_WORDS).is_some_and(|v| v != 0);
|
||||
// Samples may use 16-bit words (e.g. 14-bit X-T20); otherwise they are packed at `bits`.
|
||||
let store = if len as u64 >= (n as u64) * 2 { 16 } else { bits };
|
||||
let packed_row = w.checked_mul(store as usize).map(|n| n.div_ceil(8)).ok_or(RawError::Limit("RAF row too large"))?;
|
||||
let stride = match get(0xf005)?.unwrap_or(0) {
|
||||
0 => packed_row,
|
||||
words => words.checked_mul(4).ok_or(RawError::Limit("RAF stride too large"))?.max(packed_row),
|
||||
};
|
||||
let required = stride.checked_mul(hgt - 1).and_then(|n| n.checked_add(packed_row)).ok_or(RawError::Limit("RAF strip too large"))?;
|
||||
if len < required {
|
||||
return Err(RawError::Unsupported(format!("Fujifilm RAF packing ({len} bytes for {w}x{hgt} {bits}-bit)")));
|
||||
}
|
||||
if mode == Mode::Full {
|
||||
data = vec![0u16; n];
|
||||
data.par_chunks_mut(w).enumerate().for_each(|(y, row)| {
|
||||
// `required <= src.len()` proves these row offsets fit, including the final row.
|
||||
let s = src.get(y * stride..y * stride + packed_row).unwrap_or(&[]);
|
||||
unpack_row(s, store, words32, row);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
let active = match (pair(&h, CROP_TOP_LEFT), pair(&h, CROPPED_SIZE)) {
|
||||
(Some((top, left)), Some((ch, cw))) if ch > 0 && cw > 0 => Rect::new(left, top, cw, ch).clipped(w, hgt),
|
||||
_ => Rect::new(0, 0, w, hgt),
|
||||
};
|
||||
let active = if active.width == 0 || active.height == 0 { Rect::new(0, 0, w, hgt) } else { active };
|
||||
// black levels: one per CFA position, anchored at (0, 0) of the raw data; re-anchor at the active area
|
||||
let black = match ifd.f64s(0xf00a) {
|
||||
Some(v) if v.len() == cfa.pattern.len() && v.iter().any(|b| *b != v[0]) => {
|
||||
let (cw, chh) = (cfa.width, cfa.height);
|
||||
let values = (0..chh * cw).map(|i| v[((i / cw + active.y) % chh) * cw + (i % cw + active.x) % cw] as f32).collect();
|
||||
BlackLevel { repeat_rows: chh, repeat_cols: cw, values, ..Default::default() }
|
||||
}
|
||||
Some(v) if !v.is_empty() => BlackLevel::uniform(v[0] as f32),
|
||||
_ => BlackLevel::uniform(0.0),
|
||||
};
|
||||
let wb =
|
||||
ifd.f64s(0xf00e).filter(|v| v.len() >= 3 && v.iter().take(3).all(|x| *x > 0.0)).map(|v| [(v[1] / v[0]) as f32, 1.0, (v[2] / v[0]) as f32]);
|
||||
let white = white_from_data(&data, bits);
|
||||
let kelvin_presets = fuji_kelvin_presets(&ifd);
|
||||
let exif = h.jpeg.and_then(jpeg_exif_tiff);
|
||||
let ifd0 = exif.as_ref().and_then(|t| t.ifds.first());
|
||||
let orientation = Orientation::from_exif(ifd0.and_then(|i| i.u16(craft_tiff::tags::ORIENTATION)).unwrap_or(1));
|
||||
let img = RawImage {
|
||||
format: RawFormat::Raf,
|
||||
width: w,
|
||||
height: hgt,
|
||||
cpp: 1,
|
||||
data: RawData::U16(data),
|
||||
cfa: Some(cfa),
|
||||
bits,
|
||||
black,
|
||||
white: vec![white],
|
||||
active_area: active,
|
||||
crop: Rect::new(0, 0, active.width, active.height),
|
||||
orientation,
|
||||
color: ColorData { kelvin_presets, ..Default::default() },
|
||||
wb_multipliers: wb,
|
||||
linearized: false,
|
||||
pixel_aspect: 1.0,
|
||||
opcodes: OpcodeLists { list3: crate::raf_lens::corrections(&camera_name(bytes), &ifd, active), ..Default::default() },
|
||||
camera: ifd0
|
||||
.map(Camera::from_ifd0)
|
||||
.unwrap_or_else(|| Camera { model: Some(camera_name(bytes)).filter(|n| !n.is_empty()), ..Default::default() }),
|
||||
cinema: Default::default(),
|
||||
};
|
||||
img.validate_for(mode)?;
|
||||
Ok(img)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
pub(super) mod tests {
|
||||
use super::*;
|
||||
|
||||
/// Build a minimal RAF: header, records, raw block (LE TIFF whose IFD0 points to a 0xf000 raw IFD).
|
||||
pub(crate) fn raf(w: u32, h: u32, bits: u32, strip: Vec<u8>, layout: Option<[u8; 36]>, jpeg: &[u8]) -> Vec<u8> {
|
||||
raf_flagged(w, h, bits, strip, layout, jpeg, 0)
|
||||
}
|
||||
|
||||
/// [`raf`] with the raw IFD's `0xf006` flag set to `words32`.
|
||||
pub(crate) fn raf_flagged(w: u32, h: u32, bits: u32, strip: Vec<u8>, layout: Option<[u8; 36]>, jpeg: &[u8], words32: u32) -> Vec<u8> {
|
||||
raf_full(w, h, bits, strip, layout, jpeg, words32, "TEST", &[])
|
||||
}
|
||||
|
||||
/// [`raf_flagged`] with the camera name in the header and extra raw-IFD entries of signed rationals.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub(crate) fn raf_full(
|
||||
w: u32,
|
||||
h: u32,
|
||||
bits: u32,
|
||||
strip: Vec<u8>,
|
||||
layout: Option<[u8; 36]>,
|
||||
jpeg: &[u8],
|
||||
words32: u32,
|
||||
model: &str,
|
||||
extra: &[(u16, Vec<(i32, i32)>)],
|
||||
) -> Vec<u8> {
|
||||
let mut block = b"II*\0\x08\0\0\0".to_vec();
|
||||
let sub_off: u32 = 8 + 2 + 12 + 4;
|
||||
block.extend_from_slice(&1u16.to_le_bytes());
|
||||
block.extend_from_slice(&[0x00, 0xf0, 13, 0, 1, 0, 0, 0]);
|
||||
block.extend_from_slice(&sub_off.to_le_bytes());
|
||||
block.extend_from_slice(&0u32.to_le_bytes());
|
||||
let n = 8u32 + extra.len() as u32;
|
||||
let wb_off = sub_off + 2 + 12 * n + 4;
|
||||
// out-of-line data after the white balance: each extra table in turn, then the strip
|
||||
let mut next = wb_off + 12;
|
||||
let mut table_at = Vec::new();
|
||||
for (_, v) in extra {
|
||||
table_at.push(next);
|
||||
next += 8 * v.len() as u32;
|
||||
}
|
||||
let strip_off = next;
|
||||
// (tag, type, count, value or offset), in tag order
|
||||
let mut entries: Vec<(u16, u16, u32, u32)> = vec![
|
||||
(0xf001, 4, 1, w),
|
||||
(0xf002, 4, 1, h),
|
||||
(0xf003, 4, 1, bits),
|
||||
(0xf006, 4, 1, words32),
|
||||
(0xf007, 4, 1, strip_off),
|
||||
(0xf008, 4, 1, strip.len() as u32),
|
||||
(0xf00a, 4, 1, 64),
|
||||
(0xf00e, 4, 3, wb_off),
|
||||
];
|
||||
for ((tag, v), at) in extra.iter().zip(&table_at) {
|
||||
entries.push((*tag, 10, v.len() as u32, *at));
|
||||
}
|
||||
entries.sort_by_key(|e| e.0);
|
||||
block.extend_from_slice(&(n as u16).to_le_bytes());
|
||||
for (tag, ty, count, v) in entries {
|
||||
block.extend_from_slice(&tag.to_le_bytes());
|
||||
block.extend_from_slice(&ty.to_le_bytes());
|
||||
block.extend_from_slice(&count.to_le_bytes());
|
||||
block.extend_from_slice(&v.to_le_bytes());
|
||||
}
|
||||
block.extend_from_slice(&0u32.to_le_bytes());
|
||||
for v in [300u32, 600, 450] {
|
||||
block.extend_from_slice(&v.to_le_bytes());
|
||||
}
|
||||
for (_, v) in extra {
|
||||
for (num, den) in v {
|
||||
block.extend_from_slice(&num.to_le_bytes());
|
||||
block.extend_from_slice(&den.to_le_bytes());
|
||||
}
|
||||
}
|
||||
block.extend_from_slice(&strip);
|
||||
let mut recs: Vec<u8> = Vec::new();
|
||||
let mut nrec = 0u32;
|
||||
let mut add = |tag: u16, d: &[u8]| {
|
||||
recs.extend_from_slice(&tag.to_be_bytes());
|
||||
recs.extend_from_slice(&(d.len() as u16).to_be_bytes());
|
||||
recs.extend_from_slice(d);
|
||||
nrec += 1;
|
||||
};
|
||||
add(0x0100, &[(h >> 8) as u8, h as u8, (w >> 8) as u8, w as u8]);
|
||||
add(0x0110, &[0, 2, 0, 4]);
|
||||
add(0x0111, &[0, (h - 2) as u8, 0, (w - 4) as u8]);
|
||||
if let Some(l) = layout {
|
||||
add(0x0131, &l);
|
||||
}
|
||||
let mut dir = nrec.to_be_bytes().to_vec();
|
||||
dir.extend_from_slice(&recs);
|
||||
let mut out = b"FUJIFILMCCD-RAW 0201FF000000".to_vec();
|
||||
out.extend_from_slice(model.as_bytes());
|
||||
out.resize(108, 0);
|
||||
if strip.starts_with(b"IS") {
|
||||
let compression = if strip.get(2) == Some(&0) { 3u32 } else { 2u32 };
|
||||
out.extend_from_slice(&compression.to_be_bytes());
|
||||
}
|
||||
let jpeg_off = 160u32;
|
||||
let dir_off = jpeg_off + jpeg.len() as u32;
|
||||
let raw_off = dir_off + dir.len() as u32;
|
||||
for (i, v) in [jpeg_off, jpeg.len() as u32, dir_off, dir.len() as u32, raw_off, block.len() as u32].iter().enumerate() {
|
||||
out[84 + i * 4..88 + i * 4].copy_from_slice(&v.to_be_bytes());
|
||||
}
|
||||
out.resize(160, 0);
|
||||
out.extend_from_slice(jpeg);
|
||||
out.extend_from_slice(&dir);
|
||||
out.extend_from_slice(&block);
|
||||
out
|
||||
}
|
||||
|
||||
/// The embedded JPEG's Exif block gives orientation, make and model: found after other segments, and not
|
||||
/// past the start of scan, a truncated segment or a missing SOI.
|
||||
#[test]
|
||||
fn orientation_and_camera_come_from_the_jpeg_exif() {
|
||||
use craft_tiff::{IfdBuilder, TiffWriter, Value, tags as t};
|
||||
let mut ifd0 = IfdBuilder::new();
|
||||
ifd0.set(t::MAKE, Value::Ascii("FUJIFILM".into()));
|
||||
ifd0.set(t::MODEL, Value::Ascii("X-T4".into()));
|
||||
ifd0.set(t::ORIENTATION, Value::Short(vec![6]));
|
||||
let exif = [&b"Exif\0\0"[..], &TiffWriter::new(ByteOrder::Big, false).write(&[ifd0]).unwrap()].concat();
|
||||
let segment = |marker: u8, body: &[u8]| [&[0xff, marker][..], &((body.len() + 2) as u16).to_be_bytes(), body].concat();
|
||||
let jpeg = |parts: &[Vec<u8>]| [&[0xff, 0xd8][..], &parts.concat(), &[0xff, 0xd9]].concat();
|
||||
let app0 = segment(0xe0, b"JFIF\0\x01\x01\0\0\x01\0\x01\0\0");
|
||||
let decoded = |j: &[u8]| crate::decode(&raf(4, 4, 16, pack(&[1000; 16], 16), None, j)).unwrap();
|
||||
let r = decoded(&jpeg(&[app0.clone(), segment(0xe1, &exif)]));
|
||||
assert_eq!(r.orientation, Orientation::Rotate90);
|
||||
assert_eq!((r.camera.make.as_deref(), r.camera.model.as_deref()), (Some("FUJIFILM"), Some("X-T4")));
|
||||
// after the start of scan, or cut short: not read; the header's camera name stands in for the model
|
||||
for j in [jpeg(&[segment(0xda, &[0; 4]), segment(0xe1, &exif)]), jpeg(&[app0])[..8].to_vec(), exif.clone()] {
|
||||
let r = decoded(&j);
|
||||
assert_eq!(r.orientation, Orientation::Normal);
|
||||
assert_eq!((r.camera.make, r.camera.model.as_deref()), (None, Some("TEST")));
|
||||
}
|
||||
}
|
||||
|
||||
fn pack(px: &[u16], bits: u32) -> Vec<u8> {
|
||||
match bits {
|
||||
16 => px.iter().flat_map(|v| v.to_le_bytes()).collect(),
|
||||
12 => {
|
||||
let mut o = Vec::new();
|
||||
for p in px.chunks(2) {
|
||||
let v = p[0] as u32 | (p[1] as u32) << 12;
|
||||
o.extend_from_slice(&[v as u8, (v >> 8) as u8, (v >> 16) as u8]);
|
||||
}
|
||||
o
|
||||
}
|
||||
_ => {
|
||||
// MSB-first stream, then byte-swap every 32-bit word
|
||||
let mut bitsv: Vec<u8> = Vec::new();
|
||||
let (mut acc, mut n) = (0u64, 0u32);
|
||||
for &v in px {
|
||||
acc = (acc << bits) | v as u64;
|
||||
n += bits;
|
||||
while n >= 8 {
|
||||
bitsv.push((acc >> (n - 8)) as u8);
|
||||
n -= 8;
|
||||
}
|
||||
}
|
||||
if n > 0 {
|
||||
bitsv.push((acc << (8 - n)) as u8);
|
||||
}
|
||||
bitsv.resize(bitsv.len().div_ceil(4) * 4, 0);
|
||||
bitsv.chunks(4).flat_map(|c| [c[3], c[2], c[1], c[0]]).collect()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn decodes_all_packings() {
|
||||
let (w, h) = (16u32, 6u32);
|
||||
for (bits, store) in [(12u32, 12u32), (14, 14), (14, 16), (16, 16)] {
|
||||
let px: Vec<u16> = (0..w * h).map(|i| ((i * 977) % (1 << bits.min(14))) as u16).collect();
|
||||
let layout: [u8; 36] = std::array::from_fn(|i| Cfa::xtrans().pattern[35 - i]);
|
||||
let bytes = raf(w, h, bits, pack(&px, store), Some(layout), b"\xff\xd8\xff\xd9");
|
||||
assert_eq!(crate::probe(&bytes), Some(RawFormat::Raf));
|
||||
let r = crate::decode(&bytes).unwrap_or_else(|e| panic!("{bits}: {e}"));
|
||||
assert_eq!(crate::probe_info(&bytes).unwrap(), r.info());
|
||||
assert_eq!(r.data, RawData::U16(px), "{bits}-bit");
|
||||
assert_eq!(r.cfa, Some(Cfa::xtrans()));
|
||||
assert_eq!(r.active_area, Rect::new(4, 2, 12, 4));
|
||||
assert_eq!(r.black.mean(), 64.0);
|
||||
assert_eq!(r.wb_multipliers, Some([2.0, 1.0, 1.5]));
|
||||
}
|
||||
let bytes = raf(w, h, 16, pack(&[0; 96], 16), None, b"");
|
||||
assert_eq!(crate::decode(&bytes).unwrap().cfa.unwrap().name(), "RGGB");
|
||||
let mut px = vec![0; 96];
|
||||
px[0] = 0x5349;
|
||||
let mut bytes = raf(w, h, 16, pack(&px, 16), None, b"");
|
||||
bytes[108..112].copy_from_slice(&0u32.to_be_bytes());
|
||||
assert_eq!(crate::decode(&bytes).unwrap().data, RawData::U16(px));
|
||||
// too little data for the size: compressed
|
||||
let bytes = raf(w, h, 14, vec![0; 20], None, b"");
|
||||
assert!(matches!(crate::decode(&bytes), Err(RawError::Unsupported(_))));
|
||||
}
|
||||
|
||||
/// Raw IFD flag `0xf006` selects the 32-bit-word packing at 12 bits; without it 12 bits are an LSB-first stream.
|
||||
#[test]
|
||||
fn flag_selects_word_packed_12_bit() {
|
||||
let (w, h) = (16u32, 6u32);
|
||||
let px: Vec<u16> = (0..w * h).map(|i| ((i * 977) % 4096) as u16).collect();
|
||||
// the same 12-bit samples as little-endian 32-bit words (what the 14-bit branch of `pack` writes)
|
||||
let mut stream: Vec<u8> = Vec::new();
|
||||
let (mut acc, mut n) = (0u64, 0u32);
|
||||
for &v in &px {
|
||||
acc = (acc << 12) | v as u64;
|
||||
n += 12;
|
||||
while n >= 8 {
|
||||
stream.push((acc >> (n - 8)) as u8);
|
||||
n -= 8;
|
||||
}
|
||||
}
|
||||
let le: Vec<u8> = stream.chunks(4).flat_map(|c| [c[3], c[2], c[1], c[0]]).collect();
|
||||
let decode = |bytes: Vec<u8>| crate::decode(&bytes).unwrap().data;
|
||||
assert_eq!(decode(raf_flagged(w, h, 12, le.clone(), None, b"", 1)), RawData::U16(px.clone()));
|
||||
// flag clear: the same bytes are read as the LSB-first stream, which gives other samples
|
||||
assert_ne!(decode(raf_flagged(w, h, 12, le, None, b"", 0)), RawData::U16(px.clone()));
|
||||
// flag clear: the LSB-first stream still decodes
|
||||
assert_eq!(decode(raf_flagged(w, h, 12, pack(&px, 12), None, b"", 0)), RawData::U16(px));
|
||||
}
|
||||
|
||||
/// The lens tables of the raw IFD reach `Mode::Header` and `Mode::Full` alike, only for validated cameras,
|
||||
/// and survive a DNG round trip. The frame is 12 × 4 after the crop records (4, 2).
|
||||
#[test]
|
||||
fn lens_tables_become_opcodes_for_validated_cameras() {
|
||||
let (w, h) = (16u32, 6u32);
|
||||
let half = 12.0f64.hypot(4.0) / 2.0;
|
||||
let rat = |v: &[f64]| -> Vec<(i32, i32)> { v.iter().map(|x| ((x * 1.0e6).round() as i32, 1_000_000)).collect() };
|
||||
let knots = [0.3535211268, 0.5, 0.6126760563, 0.7070422535, 0.7908450704, 0.8661971831, 0.9352112676, 1.0, 1.06056338];
|
||||
let table = |values: &[f64]| {
|
||||
let mut v = vec![half / 9.0];
|
||||
v.extend_from_slice(&knots);
|
||||
v.extend_from_slice(values);
|
||||
rat(&v)
|
||||
};
|
||||
let dist = [-1.2, -2.4, -3.5, -4.4, -5.3, -5.9, -6.3, -6.4, -6.4];
|
||||
let vig = [98.5, 96.9, 94.9, 91.0, 84.7, 78.4, 73.0, 67.5, 67.5];
|
||||
// the colour-plane table is present in real files and ignored
|
||||
let ca = [-0.00003, -0.00009, -0.00015, -0.0002, -0.00027, -0.00034, -0.00046, -0.0006, -0.0006];
|
||||
let extra = vec![(0xf00b, table(&dist)), (0xf00f, table(&ca)), (0xf010, table(&vig))];
|
||||
for (model, expected) in [("X-T4", 2), ("GFX 100", 2), ("X-T6", 0), ("X-T4 II", 0), ("", 0)] {
|
||||
let bytes = raf_full(w, h, 16, pack(&[1000; 96], 16), None, b"", 0, model, &extra);
|
||||
let header = decode(&bytes, Mode::Header, &Limits::default()).unwrap();
|
||||
let full = crate::decode(&bytes).unwrap();
|
||||
assert_eq!(header.info(), full.info(), "{model}");
|
||||
assert_eq!(full.opcodes.list3.len(), expected, "{model}");
|
||||
let Some(crate::Opcode::WarpRectilinear { planes, center }) = full.opcodes.list3.first() else { continue };
|
||||
assert_eq!((planes.len(), *center), (1, [0.5, 0.5]), "{model}");
|
||||
assert!(matches!(full.opcodes.list3[1], crate::Opcode::FixVignetteRadial { .. }));
|
||||
#[cfg(feature = "dng")]
|
||||
{
|
||||
let dng = crate::write_dng(&full, &Default::default()).unwrap();
|
||||
assert_eq!(crate::decode(&dng).unwrap().opcodes.list3, full.opcodes.list3, "{model}");
|
||||
}
|
||||
}
|
||||
// a table in another scale is left alone, and so is a file without tables
|
||||
let wrong: Vec<(u16, Vec<(i32, i32)>)> =
|
||||
extra.iter().map(|(t, v)| (*t, v.iter().map(|&(n, d)| if n == v[0].0 { (n * 2, d) } else { (n, d) }).collect())).collect();
|
||||
let bytes = raf_full(w, h, 16, pack(&[1000; 96], 16), None, b"", 0, "X-T4", &wrong);
|
||||
assert!(crate::decode(&bytes).unwrap().opcodes.list3.iter().all(|o| !matches!(o, crate::Opcode::WarpRectilinear { .. })));
|
||||
let bytes = raf_full(w, h, 16, pack(&[1000; 96], 16), None, b"", 0, "X-T4", &[]);
|
||||
assert!(crate::decode(&bytes).unwrap().opcodes.list3.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn overflowing_row_stride_is_an_error() {
|
||||
let mut bytes = raf(16, 6, 16, vec![0; 192], None, &[]);
|
||||
let off = be32(&bytes, 100).unwrap();
|
||||
// Replace WB with a LONG8 stride, pointing to the existing out-of-line WB storage.
|
||||
bytes[off + 112..off + 116].copy_from_slice(&[0x05, 0xf0, 16, 0]);
|
||||
bytes[off + 116..off + 120].copy_from_slice(&1u32.to_le_bytes());
|
||||
bytes[off + 128..off + 136].copy_from_slice(&u64::MAX.to_le_bytes());
|
||||
assert!(matches!(crate::decode(&bytes), Err(RawError::Limit(_))));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,514 @@
|
||||
//! Fujifilm's per-shot lens correction tables from the RAF raw IFD, expressed as DNG opcodes for the shared optics
|
||||
//! pipeline (the same path as Sony `arw_lens` and Panasonic `rw2`).
|
||||
//!
|
||||
//! Tag identity: the ExifTool FujiFilm tag-name documentation names `0xf00b` GeometricDistortionParams and
|
||||
//! `0xf010` VignettingParams (signed rationals). Layout, units, radius normalisation and geometry were derived from
|
||||
//! the CC0 samples and the cameras' own JPEGs; see docs/fuji-lens-corrections.md. No lens database and no external
|
||||
//! decoder code. The lateral chromatic aberration table (`0xf00f`) is deliberately not applied: LightCraft's
|
||||
//! demosaic already registers the colour planes, so the stored shift over-corrects (see the document).
|
||||
//!
|
||||
//! Layout (`n` knots): `[scale, knot × n, value × n]`. `scale × n` is the half diagonal of the recorded image in
|
||||
//! pixels. Knots are radii in half diagonals, evenly spaced in `r` (`0, 0.1 … 1`) or in `r²` (`√(k/8)`) depending
|
||||
//! on the body; the corner is at 1. The nine-knot tables run one knot past it, to `√(9/8)` (on the X-T4 the ninth
|
||||
//! repeats the eighth's value).
|
||||
//!
|
||||
//! - distortion: percent, the bracket `1 + D/100` multiplies the radius; the camera's JPEG samples the raw at
|
||||
//! `source = z · r · (1 + D(source)/100)`, radii in half diagonals, with the zoom `z` that keeps the output
|
||||
//! rectangle inside the source (`1 / max` of the bracket over the frame's border radii).
|
||||
//! - vignette: relative illumination in percent at the source radius; the correction is the gain `100 / V`.
|
||||
|
||||
use crate::{Opcode, Rect};
|
||||
use craft_tiff::Ifd;
|
||||
|
||||
const DISTORTION: u16 = 0xf00b;
|
||||
const VIGNETTING: u16 = 0xf010;
|
||||
const SAMPLES: usize = 1024;
|
||||
/// Largest allowed deviation of the warp polynomial from the table, in half diagonals (median over the 118 sample
|
||||
/// tables 0.00012, largest 0.0007; about 1 px at 1440 px).
|
||||
const MAX_WARP_ERROR: f64 = 0.0008;
|
||||
/// Largest allowed relative deviation of the gain polynomial from the table (median 0.4 %, largest 1.9 %).
|
||||
const MAX_GAIN_ERROR: f64 = 0.025;
|
||||
/// Reweighting rounds of the minimax polynomial fit.
|
||||
const FIT_ROUNDS: usize = 40;
|
||||
/// `scale × n` must be the recorded image's half diagonal to this relative tolerance (other normalisations are not
|
||||
/// understood).
|
||||
const SCALE_TOLERANCE: f64 = 0.005;
|
||||
|
||||
/// Dimensions and half diagonals of the 26.1 MP (`6240 × 4160`) and 40 MP (`7728 × 5152`) active areas in pixels.
|
||||
/// The X-S20 writes `scale × n = 4643` (the 40 MP half diagonal) on its uncropped 26.1 MP frame (`3749` px
|
||||
/// half diagonal), while indexing the 9 knots `√(k/8)` over the 26.1 MP active area.
|
||||
const XTRANS4_26MP_WIDTH: usize = 6240;
|
||||
const XTRANS4_26MP_HEIGHT: usize = 4160;
|
||||
const XTRANS4_26MP_HALF_DIAGONAL: f64 = 3749.0;
|
||||
const XTRANS5_40MP_HALF_DIAGONAL: f64 = 4643.0;
|
||||
|
||||
/// Camera models whose tables were checked against the camera's own JPEG (distortion within 0.01 of the best free
|
||||
/// radial fit's correlation on at least one lens, see docs/fuji-lens-corrections.md). Other bodies stay uncorrected.
|
||||
/// The nine-knot bodies other than the X-T4 were checked with their tables read at `knot_last · r`; the plain radius
|
||||
/// they are now read at was measured on the X-T4 only ("Knot radius" in that doc).
|
||||
const VALIDATED: &[&str] = &[
|
||||
"FinePix F770EXR",
|
||||
"FinePix HS30EXR",
|
||||
"FinePix HS33EXR",
|
||||
"FinePix HS50EXR",
|
||||
"FinePix X100",
|
||||
"GFX 100",
|
||||
"GFX 50R",
|
||||
"GFX 50S",
|
||||
"GFX100 II",
|
||||
"GFX100RF",
|
||||
"GFX100S",
|
||||
"GFX100S II",
|
||||
"GFX50S II",
|
||||
"X-A1",
|
||||
"X-A10",
|
||||
"X-A2",
|
||||
"X-A3",
|
||||
"X-A5",
|
||||
"X-A7",
|
||||
"X-E1",
|
||||
"X-E2",
|
||||
"X-E2S",
|
||||
"X-E3",
|
||||
"X-E4",
|
||||
"X-E5",
|
||||
"X-H1",
|
||||
"X-H2",
|
||||
"X-H2S",
|
||||
"X-M1",
|
||||
"X-M5",
|
||||
"X-Pro1",
|
||||
"X-Pro2",
|
||||
"X-Pro3",
|
||||
"X-S1",
|
||||
"X-S10",
|
||||
"X-S20",
|
||||
"X-T1",
|
||||
"X-T10",
|
||||
"X-T100",
|
||||
"X-T2",
|
||||
"X-T20",
|
||||
"X-T200",
|
||||
"X-T3",
|
||||
"X-T30",
|
||||
"X-T30 II",
|
||||
"X-T30 III",
|
||||
"X-T4",
|
||||
"X-T5",
|
||||
"X-T50",
|
||||
"X100F",
|
||||
"X100S",
|
||||
"X100T",
|
||||
"X100V",
|
||||
"X100VI",
|
||||
"X20",
|
||||
"X30",
|
||||
"X70",
|
||||
"XF10",
|
||||
"XQ2",
|
||||
];
|
||||
|
||||
/// One parsed table: knot positions and the value array of the same length.
|
||||
struct Table {
|
||||
knots: Vec<f64>,
|
||||
values: Vec<f64>,
|
||||
}
|
||||
|
||||
/// Whether `scale × n` matches the recorded image's `half_diagonal` (or, on an uncropped 26.1 MP X-S20 frame,
|
||||
/// the 40 MP reference half diagonal that its firmware writes).
|
||||
fn scale_matches(model: &str, scale_times_n: f64, half_diagonal: f64, w: usize, h: usize) -> bool {
|
||||
if (scale_times_n / half_diagonal - 1.0).abs() <= SCALE_TOLERANCE {
|
||||
return true;
|
||||
}
|
||||
model == "X-S20"
|
||||
&& w == XTRANS4_26MP_WIDTH
|
||||
&& h == XTRANS4_26MP_HEIGHT
|
||||
&& (half_diagonal / XTRANS4_26MP_HALF_DIAGONAL - 1.0).abs() <= SCALE_TOLERANCE
|
||||
&& (scale_times_n / XTRANS5_40MP_HALF_DIAGONAL - 1.0).abs() <= SCALE_TOLERANCE
|
||||
}
|
||||
|
||||
/// Read and sanity-check a table. `half_diagonal` is the recorded image's half diagonal in pixels, which
|
||||
/// `scale × n` must reproduce.
|
||||
fn read(model: &str, ifd: &Ifd, tag: u16, half_diagonal: f64, w: usize, h: usize) -> Option<Table> {
|
||||
let v = ifd.f64s(tag)?;
|
||||
if v.is_empty() || v.iter().any(|x| !x.is_finite()) {
|
||||
return None;
|
||||
}
|
||||
let n = (v.len() - 1) / 2;
|
||||
if !(3..=40).contains(&n) || v.len() != 1 + 2 * n {
|
||||
return None;
|
||||
}
|
||||
let scale = v[0];
|
||||
if scale <= 0.0 || !scale_matches(model, scale * n as f64, half_diagonal, w, h) {
|
||||
return None;
|
||||
}
|
||||
let knots = v[1..1 + n].to_vec();
|
||||
if knots[0] < 0.0 || knots.windows(2).any(|w| w[1] <= w[0]) || !(0.5..=2.0).contains(&knots[n - 1]) {
|
||||
return None;
|
||||
}
|
||||
Some(Table { knots, values: v[1 + n..].to_vec() })
|
||||
}
|
||||
|
||||
impl Table {
|
||||
/// Value `which` at knot position `x`: linear between knots, flat outside, anchored at `x = 0` to `at_zero`
|
||||
/// when the first knot lies above it (distortion is 0 and illumination 100 % on the axis).
|
||||
fn at(&self, x: f64, at_zero: Option<f64>) -> f64 {
|
||||
let (k, y) = (&self.knots, &self.values);
|
||||
if let (Some(z), true) = (at_zero, k[0] > 0.0)
|
||||
&& x < k[0]
|
||||
{
|
||||
return z + (y[0] - z) * (x / k[0]).max(0.0);
|
||||
}
|
||||
if x <= k[0] {
|
||||
return y[0];
|
||||
}
|
||||
match k.iter().position(|&p| p >= x) {
|
||||
None => y[y.len() - 1],
|
||||
Some(i) => y[i - 1] + (y[i] - y[i - 1]) * (x - k[i - 1]) / (k[i] - k[i - 1]),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Source radius of the output radius `r` (both in half diagonals) with the distortion zoom.
|
||||
struct Geometry<'a> {
|
||||
table: Option<&'a Table>,
|
||||
zoom: f64,
|
||||
}
|
||||
|
||||
impl Geometry<'_> {
|
||||
/// The radius bracket `1 + D/100` at source radius `source` (half diagonals, the table's own unit); 1 without a table.
|
||||
fn bracket(&self, source: f64) -> f64 {
|
||||
self.table.map_or(1.0, |t| 1.0 + t.at(source, Some(0.0)) / 100.0)
|
||||
}
|
||||
fn source(&self, r: f64) -> f64 {
|
||||
let mut s = self.zoom * r;
|
||||
for _ in 0..24 {
|
||||
s = self.zoom * r * self.bracket(s);
|
||||
}
|
||||
s
|
||||
}
|
||||
}
|
||||
|
||||
/// The frame-filling zoom: the largest radial scale for which every border point of the `w × h` output samples
|
||||
/// inside the source (the bracket is largest at the nearest border radius or beyond).
|
||||
fn zoom(table: &Table, nearest: f64) -> Option<f64> {
|
||||
let mut g = Geometry { table: Some(table), zoom: 1.0 };
|
||||
for _ in 0..8 {
|
||||
let m = (0..=512).map(|i| nearest + (1.0 - nearest) * i as f64 / 512.0).map(|r| g.bracket(g.source(r))).fold(f64::MIN, f64::max);
|
||||
if !(0.5..2.0).contains(&m) {
|
||||
return None;
|
||||
}
|
||||
g.zoom = 1.0 / m;
|
||||
}
|
||||
Some(g.zoom)
|
||||
}
|
||||
|
||||
/// Fit `target(r) ≈ Σ c_i basis_i(r)` on `r ∈ [0, 1]` minimising the largest deviation (iteratively reweighted least
|
||||
/// squares), returning the coefficients and that deviation.
|
||||
fn minimax<const N: usize>(basis: impl Fn(f64) -> [f64; N], target: impl Fn(f64) -> f64) -> Option<([f64; N], f64)> {
|
||||
let rows: Vec<([f64; N], f64)> = (0..=SAMPLES).map(|j| j as f64 / SAMPLES as f64).map(|r| (basis(r), target(r))).collect();
|
||||
let mut weight = vec![1.0; rows.len()];
|
||||
let mut best: Option<([f64; N], f64)> = None;
|
||||
for _ in 0..FIT_ROUNDS {
|
||||
let mut matrix = [[0.0; N]; N];
|
||||
let mut rhs = [0.0; N];
|
||||
for ((terms, y), w) in rows.iter().zip(&weight) {
|
||||
for (i, &a) in terms.iter().enumerate() {
|
||||
rhs[i] += w * a * y;
|
||||
for (k, &b) in terms.iter().enumerate() {
|
||||
matrix[i][k] += w * a * b;
|
||||
}
|
||||
}
|
||||
}
|
||||
let c = solve(matrix, rhs)?;
|
||||
let errors: Vec<f64> = rows.iter().map(|(terms, y)| (terms.iter().zip(c).map(|(a, b)| a * b).sum::<f64>() - y).abs()).collect();
|
||||
let worst = errors.iter().copied().fold(0.0, f64::max);
|
||||
if best.as_ref().is_none_or(|b| worst < b.1) {
|
||||
best = Some((c, worst));
|
||||
}
|
||||
let mean = weight.iter().sum::<f64>() / weight.len() as f64;
|
||||
for (w, e) in weight.iter_mut().zip(&errors) {
|
||||
*w *= 0.3 + e / worst.max(1e-15);
|
||||
*w /= mean;
|
||||
}
|
||||
}
|
||||
best
|
||||
}
|
||||
|
||||
/// `source(r) = r (k0 + k1 r² + k2 r⁴ + k3 r⁶)`, rejected when it strays from the table or folds.
|
||||
fn polynomial(source: impl Fn(f64) -> f64) -> Option<[f64; 4]> {
|
||||
let mut previous = -1.0;
|
||||
for j in 0..=SAMPLES {
|
||||
let s = source(j as f64 / SAMPLES as f64);
|
||||
if !s.is_finite() || s <= previous {
|
||||
return None;
|
||||
}
|
||||
previous = s;
|
||||
}
|
||||
let basis = |r: f64| {
|
||||
let r2 = r * r;
|
||||
[r, r * r2, r * r2 * r2, r * r2 * r2 * r2]
|
||||
};
|
||||
let (k, error) = minimax(basis, &source)?;
|
||||
let folds = (0..=SAMPLES).any(|j| {
|
||||
let r2 = (j as f64 / SAMPLES as f64).powi(2);
|
||||
k[0] + r2 * (3.0 * k[1] + r2 * (5.0 * k[2] + r2 * 7.0 * k[3])) <= 0.0
|
||||
});
|
||||
(error <= MAX_WARP_ERROR && !folds).then_some(k)
|
||||
}
|
||||
|
||||
/// Gaussian elimination with partial pivoting for the small normal equations.
|
||||
fn solve<const N: usize>(mut a: [[f64; N]; N], mut b: [f64; N]) -> Option<[f64; N]> {
|
||||
for col in 0..N {
|
||||
let pivot = (col..N).max_by(|&i, &j| a[i][col].abs().total_cmp(&a[j][col].abs()))?;
|
||||
if !a[pivot][col].is_finite() || a[pivot][col].abs() < 1e-12 {
|
||||
return None;
|
||||
}
|
||||
a.swap(col, pivot);
|
||||
b.swap(col, pivot);
|
||||
for row in col + 1..N {
|
||||
let factor = a[row][col] / a[col][col];
|
||||
let (top, bottom) = a.split_at_mut(row);
|
||||
for (x, p) in bottom[0][col..].iter_mut().zip(&top[col][col..]) {
|
||||
*x -= factor * p;
|
||||
}
|
||||
b[row] -= factor * b[col];
|
||||
}
|
||||
}
|
||||
let mut x = [0.0; N];
|
||||
for i in (0..N).rev() {
|
||||
x[i] = (b[i] - (i + 1..N).map(|j| a[i][j] * x[j]).sum::<f64>()) / a[i][i];
|
||||
}
|
||||
x.iter().all(|v| v.is_finite()).then_some(x)
|
||||
}
|
||||
|
||||
/// The warp (distortion with the frame-filling zoom) for a recorded image of `w × h` pixels.
|
||||
fn warp(model: &str, ifd: &Ifd, w: usize, h: usize) -> Option<Opcode> {
|
||||
let half = (w as f64).hypot(h as f64) / 2.0;
|
||||
let table = read(model, ifd, DISTORTION, half, w, h)?;
|
||||
// Percent: anything beyond a third is not distortion; an all-zero table has nothing to correct.
|
||||
if table.values.iter().any(|d| d.abs() > 30.0) || table.values.iter().all(|d| *d == 0.0) {
|
||||
return None;
|
||||
}
|
||||
let (cw, ch) = (w as f64 - 1.0, h as f64 - 1.0);
|
||||
let nearest = cw.min(ch) / cw.hypot(ch);
|
||||
let g = Geometry { table: Some(&table), zoom: zoom(&table, nearest)? };
|
||||
let k = polynomial(|r| g.source(r))?;
|
||||
// The recorded image is the active area and the tables are centred on it: DNG normalises by the farthest
|
||||
// corner, which is the half diagonal used here.
|
||||
Some(Opcode::WarpRectilinear { planes: vec![[k[0], k[1], k[2], k[3], 0.0, 0.0]], center: [0.5, 0.5] })
|
||||
}
|
||||
|
||||
/// Radial illumination gain `100 / V` as DNG `FixVignetteRadial` (`1 + k0 r² + … + k4 r¹⁰`).
|
||||
fn vignette(model: &str, ifd: &Ifd, w: usize, h: usize) -> Option<Opcode> {
|
||||
let half = (w as f64).hypot(h as f64) / 2.0;
|
||||
let t = read(model, ifd, VIGNETTING, half, w, h)?;
|
||||
if t.values.iter().any(|v| !(10.0..=130.0).contains(v)) {
|
||||
return None;
|
||||
}
|
||||
let gain = |r: f64| 100.0 / t.at(r, Some(100.0));
|
||||
if (0..=SAMPLES).all(|j| (gain(j as f64 / SAMPLES as f64) - 1.0).abs() < 1e-4) {
|
||||
return None;
|
||||
}
|
||||
let basis = |r: f64| {
|
||||
let x = r * r;
|
||||
[x, x * x, x.powi(3), x.powi(4), x.powi(5)]
|
||||
};
|
||||
let (k, error) = minimax(basis, |r| gain(r) - 1.0)?;
|
||||
// the relative error at the worst point (gain is at least 1 / 1.3 here)
|
||||
if error > MAX_GAIN_ERROR || (0..=SAMPLES).any(|j| 1.0 + basis(j as f64 / SAMPLES as f64).iter().zip(k).map(|(a, b)| a * b).sum::<f64>() < 0.5) {
|
||||
return None;
|
||||
}
|
||||
Some(Opcode::FixVignetteRadial { k, center: [0.5, 0.5] })
|
||||
}
|
||||
|
||||
/// The lens corrections of a RAF whose camera `model` was validated (anything else, and any table that is not in
|
||||
/// the understood layout and scale, is left uncorrected). `active` is the recorded image (the raw crop).
|
||||
pub(super) fn corrections(model: &str, raw: &Ifd, active: Rect) -> Vec<Opcode> {
|
||||
let model = model.trim();
|
||||
if !VALIDATED.contains(&model) || active.width < 2 || active.height < 2 {
|
||||
return Vec::new();
|
||||
}
|
||||
warp(model, raw, active.width, active.height).into_iter().chain(vignette(model, raw, active.width, active.height)).collect()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use craft_tiff::{Entry, Value};
|
||||
|
||||
pub(crate) const KNOTS9: [f64; 9] = [0.3535211268, 0.5, 0.6126760563, 0.7070422535, 0.7908450704, 0.8661971831, 0.9352112676, 1.0, 1.06056338];
|
||||
/// X-T4 with the XF16-55 at 16 mm (3913): the tables as ExifTool prints them.
|
||||
pub(crate) const XT4_DIST: [f64; 9] =
|
||||
[-1.172515869, -2.354660034, -3.458114624, -4.434677124, -5.274719238, -5.918792725, -6.328369141, -6.426345825, -6.426345825];
|
||||
pub(crate) const XT4_VIG: [f64; 9] =
|
||||
[98.46142578, 96.87792969, 94.90234375, 90.98144531, 84.73291016, 78.43701172, 72.96044922, 67.45703125, 67.45703125];
|
||||
pub(crate) const XT4_AREA: Rect = Rect { x: 0, y: 0, width: 6240, height: 4160 };
|
||||
|
||||
pub(crate) fn rationals(v: &[f64]) -> Value {
|
||||
Value::SRational(v.iter().map(|x| ((x * 1.0e6).round() as i32, 1_000_000)).collect())
|
||||
}
|
||||
pub(crate) fn table(scale: f64, knots: &[f64], values: &[f64]) -> Value {
|
||||
let mut v = vec![scale];
|
||||
v.extend_from_slice(knots);
|
||||
v.extend_from_slice(values);
|
||||
rationals(&v)
|
||||
}
|
||||
pub(crate) fn xt4() -> Ifd {
|
||||
let scale = 3749.0 / 9.0;
|
||||
Ifd {
|
||||
entries: vec![
|
||||
Entry { tag: DISTORTION, value: table(scale, &KNOTS9, &XT4_DIST), offset: 0 },
|
||||
Entry { tag: VIGNETTING, value: table(scale, &KNOTS9, &XT4_VIG), offset: 0 },
|
||||
],
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
fn only(tag: u16, value: Value) -> Ifd {
|
||||
Ifd { entries: vec![Entry { tag, value, offset: 0 }], ..Default::default() }
|
||||
}
|
||||
fn evaluate(k: &[f64; 6], r: f64) -> f64 {
|
||||
r * (k[0] + r * r * (k[1] + r * r * (k[2] + r * r * k[3])))
|
||||
}
|
||||
|
||||
/// The X-T4's arrays become a one-plane warp with the frame-filling zoom and a vignette gain read at the plain radius.
|
||||
#[test]
|
||||
fn x_t4_tables_become_a_warp_and_a_vignette_gain() {
|
||||
let ops = corrections("X-T4", &xt4(), XT4_AREA);
|
||||
let [Opcode::WarpRectilinear { planes, center }, Opcode::FixVignetteRadial { k, center: vc }] = ops.as_slice() else { panic!("{ops:?}") };
|
||||
assert_eq!((*center, *vc), ([0.5, 0.5], [0.5, 0.5]));
|
||||
assert_eq!(planes.len(), 1, "no colour planes: lateral CA is not applied");
|
||||
// barrel distortion: zoomed to fill the frame, the corner samples inside the raw (-6.4 % bracket; the measured
|
||||
// free fit below gives 0.9635 there)
|
||||
let green = &planes[0];
|
||||
assert!((1.02..1.04).contains(&green[0]), "{}", green[0]);
|
||||
let corner = evaluate(green, 1.0);
|
||||
assert!((corner - 0.9641).abs() < 0.002, "{corner}");
|
||||
// vignette: 67.457 % at the corner, 100 % on the axis
|
||||
let gain = |r: f64| 1.0 + (0..5).map(|i| k[i] * r.powi(2 * i as i32 + 2)).sum::<f64>();
|
||||
assert!((gain(1.0) - 100.0 / 67.45703125).abs() < 0.01, "{}", gain(1.0));
|
||||
assert!((gain(0.0) - 1.0).abs() < 1e-12);
|
||||
// read at the plain radius: r = 0.5 is the second knot, r = 0.85 lies between the fifth and sixth
|
||||
assert!((gain(0.5) - 100.0 / 96.87792969).abs() < 0.01, "{}", gain(0.5));
|
||||
let v = 84.73291016 + (78.43701172 - 84.73291016) * (0.85 - 0.7908450704) / (0.8661971831 - 0.7908450704);
|
||||
assert!((gain(0.85) - 100.0 / v).abs() < 0.01, "{}", gain(0.85));
|
||||
}
|
||||
|
||||
/// The warp follows a free radial fit of the camera's own JPEG of pixls 3913 to within 0.0012 half diagonals.
|
||||
#[test]
|
||||
fn matches_the_measured_x_t4_geometry() {
|
||||
// Free radial fit (data/testing/raf-lens/tools/fit_model.py) of the camera JPEG against LightCraft's uncorrected
|
||||
// render of pixls 3913 at 600 px (maximum correlation 0.996 of the high-passed luminance), no table or zoom
|
||||
// model: source = s · r · (1 + k1 r² + k2 r⁴ + k3 r⁶), r in half diagonals.
|
||||
let measured = [1.0299688, -0.1024252, 0.0195257, 0.0183398];
|
||||
let ops = corrections("X-T4", &xt4(), XT4_AREA);
|
||||
let Opcode::WarpRectilinear { planes, .. } = &ops[0] else { panic!() };
|
||||
for i in 2..=20 {
|
||||
let r = i as f64 / 20.0;
|
||||
let r2 = r * r;
|
||||
let reference = measured[0] * r * (1.0 + measured[1] * r2 + measured[2] * r2 * r2 + measured[3] * r2 * r2 * r2);
|
||||
assert!((evaluate(&planes[0], r) - reference).abs() < 0.0012, "{r}: {} vs {reference}", evaluate(&planes[0], r));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn only_validated_models_are_corrected() {
|
||||
for model in ["", "X-T4 ", "X-T6", "X-Pro3", "FinePix S1", "FinePix F550EXR", "X10", "XF1", "XQ1", "GFX 100S", "x-t4"] {
|
||||
let n = corrections(model, &xt4(), XT4_AREA).len();
|
||||
assert_eq!(n, usize::from(VALIDATED.contains(&model.trim())) * 2, "{model:?}");
|
||||
}
|
||||
assert!(corrections("X-T4", &xt4(), Rect::new(0, 0, 1, 1)).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_unknown_scale_layout_and_values() {
|
||||
let scale = 3749.0 / 9.0;
|
||||
let bad = |ifd: Ifd| corrections("X-T4", &ifd, XT4_AREA);
|
||||
// another normalisation (ignoring camera exceptions like the X-S20 which writes the 40 MP half diagonal), non-increasing knots, wrong lengths, garbage
|
||||
assert!(bad(only(DISTORTION, table(4643.0 / 9.0, &KNOTS9, &XT4_DIST))).is_empty());
|
||||
// X-S20 accepts the 40 MP normalization on its uncropped 26 MP sensor, but rejects in-camera crops
|
||||
let xs20_ops = corrections("X-S20", &only(DISTORTION, table(4643.0 / 9.0, &KNOTS9, &XT4_DIST)), XT4_AREA);
|
||||
assert!(matches!(xs20_ops.as_slice(), [Opcode::WarpRectilinear { .. }]), "{xs20_ops:?}");
|
||||
// Nonzero active-frame offsets are permitted as long as dimensions are 6240 × 4160
|
||||
let offset_area = Rect::new(32, 20, 6240, 4160);
|
||||
let xs20_offset_ops = corrections("X-S20", &only(DISTORTION, table(4643.0 / 9.0, &KNOTS9, &XT4_DIST)), offset_area);
|
||||
assert!(matches!(xs20_offset_ops.as_slice(), [Opcode::WarpRectilinear { .. }]), "{xs20_offset_ops:?}");
|
||||
let cropped = Rect::new(0, 0, 4992, 3328);
|
||||
assert!(corrections("X-S20", &only(DISTORTION, table(4643.0 / 9.0, &KNOTS9, &XT4_DIST)), cropped).is_empty());
|
||||
// Differing active dimensions are rejected even if hypothetically close in half-diagonal
|
||||
let wrong_dim = Rect::new(0, 0, 6242, 4157);
|
||||
assert!(corrections("X-S20", &only(DISTORTION, table(4643.0 / 9.0, &KNOTS9, &XT4_DIST)), wrong_dim).is_empty());
|
||||
let mut flat = KNOTS9;
|
||||
flat[3] = flat[2];
|
||||
assert!(bad(only(DISTORTION, table(scale, &flat, &XT4_DIST))).is_empty());
|
||||
assert!(bad(only(DISTORTION, table(scale, &KNOTS9[..8], &XT4_DIST))).is_empty());
|
||||
assert!(bad(only(DISTORTION, table(scale, &KNOTS9, &[40.0; 9]))).is_empty());
|
||||
assert!(bad(only(DISTORTION, table(scale, &KNOTS9, &[f64::NAN; 9]))).is_empty());
|
||||
assert!(bad(only(DISTORTION, Value::Short(vec![19; 19]))).is_empty());
|
||||
assert!(bad(only(DISTORTION, table(0.0, &KNOTS9, &XT4_DIST))).is_empty());
|
||||
assert!(bad(only(VIGNETTING, table(scale, &KNOTS9, &[5.0; 9]))).is_empty());
|
||||
// flat tables (nothing to correct) produce nothing
|
||||
let flat_tables = Ifd {
|
||||
entries: vec![
|
||||
Entry { tag: DISTORTION, value: table(scale, &KNOTS9, &[0.0; 9]), offset: 0 },
|
||||
Entry { tag: VIGNETTING, value: table(scale, &KNOTS9, &[100.0; 9]), offset: 0 },
|
||||
],
|
||||
..Default::default()
|
||||
};
|
||||
assert!(bad(flat_tables).is_empty());
|
||||
assert!(bad(Ifd::default()).is_empty());
|
||||
// an unusable distortion table leaves the warp off but not the vignette
|
||||
let mut ifd = xt4();
|
||||
ifd.entries[0].value = table(1.0, &KNOTS9, &XT4_DIST);
|
||||
let ops = bad(ifd);
|
||||
assert!(matches!(ops.as_slice(), [Opcode::FixVignetteRadial { .. }]), "{ops:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn old_layout_with_eleven_knots_and_pincushion_zoom() {
|
||||
// X-T2-style: 0 … 1 in tenths, 11 knots, scale = half diagonal / 11; a 3:2 6000 × 4000 frame.
|
||||
let knots: Vec<f64> = (0..=10).map(|i| i as f64 / 10.0).collect();
|
||||
let dist = [0.0, 0.02, 0.05, 0.1, 0.17, 0.27, 0.39, 0.54, 0.69, 0.84, 0.98];
|
||||
let scale = 3605.0 / 11.0;
|
||||
let ifd = only(DISTORTION, table(scale, &knots, &dist));
|
||||
let area = Rect::new(0, 0, 6000, 4000);
|
||||
let ops = corrections("X-T2", &ifd, area);
|
||||
let [Opcode::WarpRectilinear { planes, .. }] = ops.as_slice() else { panic!("{ops:?}") };
|
||||
assert_eq!(planes.len(), 1);
|
||||
// pincushion bracket > 1 everywhere: the zoom shrinks the sampled radius; the corner samples exactly its edge
|
||||
assert!(planes[0][0] < 1.0 && planes[0][0] > 0.98);
|
||||
let nearest = 5999.0f64.min(3999.0) / 5999.0f64.hypot(3999.0);
|
||||
let at_edge = evaluate(&planes[0], nearest) / nearest;
|
||||
assert!((at_edge - planes[0][0]).abs() < 0.004);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn xs20_40mp_scale_table_produces_warp_and_vignette() {
|
||||
let scale = 4643.0 / 9.0;
|
||||
let ifd = Ifd {
|
||||
entries: vec![
|
||||
Entry { tag: DISTORTION, value: table(scale, &KNOTS9, &XT4_DIST), offset: 0 },
|
||||
Entry { tag: VIGNETTING, value: table(scale, &KNOTS9, &XT4_VIG), offset: 0 },
|
||||
],
|
||||
..Default::default()
|
||||
};
|
||||
// X-S20 on its uncropped 26 MP sensor emits both WarpRectilinear and FixVignetteRadial
|
||||
let ops = corrections("X-S20", &ifd, XT4_AREA);
|
||||
let [Opcode::WarpRectilinear { planes, center: wc }, Opcode::FixVignetteRadial { k, center: vc }] = ops.as_slice() else {
|
||||
panic!("expected warp and vignette for X-S20: {ops:?}");
|
||||
};
|
||||
assert_eq!((*wc, *vc), ([0.5, 0.5], [0.5, 0.5]));
|
||||
assert_eq!(planes.len(), 1);
|
||||
assert!((planes[0][0] - 1.03).abs() < 0.02);
|
||||
assert!((k[0]).abs() > 0.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_vignette_table_works_without_a_distortion_table() {
|
||||
let scale = 3749.0 / 9.0;
|
||||
let v = corrections("X-T4", &only(VIGNETTING, table(scale, &KNOTS9, &XT4_VIG)), XT4_AREA);
|
||||
assert!(matches!(v.as_slice(), [Opcode::FixVignetteRadial { .. }]));
|
||||
let d = corrections("X-T4", &only(DISTORTION, table(scale, &KNOTS9, &XT4_DIST)), XT4_AREA);
|
||||
assert!(matches!(d.as_slice(), [Opcode::WarpRectilinear { .. }]));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,592 @@
|
||||
//! Fujifilm's striped predictive RAW compression.
|
||||
//!
|
||||
//! Clean-room implementation: Fabian's prose description of lossless RAF compression
|
||||
//! (https://capnfabs.net/posts/fuji-raf-compression-algorithm/) supplied the broad structure;
|
||||
//! headers, colour-vector placement, predictors, contexts and adaptation were recovered
|
||||
//! from sample bytes and black-box comparisons of decoded sensor arrays. No other RAW
|
||||
//! decoder's source was consulted. See docs/raf-compression.md for verification details.
|
||||
|
||||
use crate::{Limits, Mode, RawError, Result};
|
||||
use rayon::prelude::*;
|
||||
|
||||
const BLOCK: usize = 768;
|
||||
const XTRANS: [[usize; 6]; 6] =
|
||||
[[1, 1, 0, 1, 1, 2], [1, 1, 2, 1, 1, 0], [2, 0, 1, 0, 2, 1], [1, 1, 2, 1, 1, 0], [1, 1, 0, 1, 1, 2], [0, 2, 1, 2, 0, 1]];
|
||||
const MASKS: [[u8; 2]; 6] = [[10, 15], [15, 10], [14, 10], [15, 11], [11, 15], [10, 14]];
|
||||
|
||||
fn corrupt(why: &str) -> RawError {
|
||||
RawError::Corrupt(format!("Fujifilm compressed RAF: {why}"))
|
||||
}
|
||||
|
||||
fn be16(src: &[u8], at: usize) -> Result<usize> {
|
||||
let s = src.get(at..at.saturating_add(2)).ok_or_else(|| corrupt("truncated header"))?;
|
||||
let [a, b] = s else { return Err(corrupt("truncated header")) };
|
||||
Ok(u16::from_be_bytes([*a, *b]) as usize)
|
||||
}
|
||||
|
||||
struct Stripe<'a> {
|
||||
bytes: &'a [u8],
|
||||
quantizers: Option<&'a [u8]>,
|
||||
width: usize,
|
||||
}
|
||||
|
||||
pub(crate) fn decode(src: &[u8], width: usize, height: usize, bits: u32, xtrans: bool, mode: Mode, limits: &Limits) -> Result<Vec<u16>> {
|
||||
let h = src.get(..16).ok_or_else(|| corrupt("truncated header"))?;
|
||||
if !h.starts_with(b"IS") {
|
||||
return Err(corrupt("missing compression signature"));
|
||||
}
|
||||
let flag = h.get(2).copied().ok_or_else(|| corrupt("truncated header"))?;
|
||||
if flag > 1 {
|
||||
return Err(corrupt("invalid compression flag"));
|
||||
}
|
||||
let sensor = h.get(3).copied().ok_or_else(|| corrupt("truncated header"))?;
|
||||
if sensor != if xtrans { 16 } else { 0 } {
|
||||
return Err(corrupt("compression sensor type disagrees with CFA"));
|
||||
}
|
||||
if !matches!(bits, 12 | 14 | 16) {
|
||||
return Err(RawError::Unsupported(format!("Fujifilm compressed RAF at {bits} bits")));
|
||||
}
|
||||
let count = h.get(13).copied().ok_or_else(|| corrupt("truncated header"))? as usize;
|
||||
if h.get(4).copied() != Some(bits as u8)
|
||||
|| be16(h, 5)? != height
|
||||
|| be16(h, 9)? != width
|
||||
|| count == 0
|
||||
|| count != width.div_ceil(BLOCK)
|
||||
|| be16(h, 7)? != count * BLOCK
|
||||
|| be16(h, 11)? != BLOCK
|
||||
|| height == 0
|
||||
|| !height.is_multiple_of(6)
|
||||
|| be16(h, 14)? != height / 6
|
||||
{
|
||||
return Err(corrupt("inconsistent compression dimensions"));
|
||||
}
|
||||
let n = width.checked_mul(height).filter(|&n| n <= limits.samples()).ok_or(RawError::Limit("image too large"))?;
|
||||
let table_end = 16 + count * 4;
|
||||
let mut offset = table_end.div_ceil(16) * 16;
|
||||
let quantizer_start = offset;
|
||||
let quantizer_stride = (height / 6).div_ceil(16) * 16;
|
||||
if flag == 0 {
|
||||
offset += quantizer_stride * count;
|
||||
if src.get(quantizer_start..offset).is_none() {
|
||||
return Err(corrupt("truncated quantizer tables"));
|
||||
}
|
||||
}
|
||||
let mut stripes = Vec::with_capacity(count);
|
||||
for i in 0..count {
|
||||
let entry = src.get(16 + i * 4..20 + i * 4).ok_or_else(|| corrupt("truncated stripe size table"))?;
|
||||
let [a, b, c, d] = entry else { return Err(corrupt("truncated stripe size table")) };
|
||||
let size = u32::from_be_bytes([*a, *b, *c, *d]) as usize;
|
||||
let end = offset.checked_add(size).ok_or_else(|| corrupt("stripe size overflow"))?;
|
||||
let bytes = src.get(offset..end).filter(|s| !s.is_empty()).ok_or_else(|| corrupt("stripe outside file"))?;
|
||||
let quantizers = if flag == 0 {
|
||||
Some(
|
||||
src.get(quantizer_start + i * quantizer_stride..quantizer_start + i * quantizer_stride + height / 6)
|
||||
.ok_or_else(|| corrupt("truncated quantizer table"))?,
|
||||
)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
stripes.push(Stripe { bytes, quantizers, width: (width - i * BLOCK).min(BLOCK) });
|
||||
offset = end;
|
||||
}
|
||||
if mode == Mode::Header {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
let decoded: Vec<Vec<u16>> = stripes.par_iter().map(|s| decode_stripe(s, height, bits, xtrans)).collect::<Result<_>>()?;
|
||||
let mut output = vec![0; n];
|
||||
for (i, (stripe, data)) in stripes.iter().zip(&decoded).enumerate() {
|
||||
for (row, samples) in output.chunks_exact_mut(width).zip(data.chunks_exact(stripe.width)) {
|
||||
row.get_mut(i * BLOCK..i * BLOCK + stripe.width).ok_or_else(|| corrupt("invalid stripe placement"))?.copy_from_slice(samples);
|
||||
}
|
||||
}
|
||||
Ok(output)
|
||||
}
|
||||
|
||||
struct Bits<'a> {
|
||||
src: &'a [u8],
|
||||
pos: usize,
|
||||
acc: u64,
|
||||
available: u32,
|
||||
}
|
||||
|
||||
impl<'a> Bits<'a> {
|
||||
fn new(src: &'a [u8]) -> Self {
|
||||
Self { src, pos: 0, acc: 0, available: 0 }
|
||||
}
|
||||
|
||||
/// Tops the accumulator up to at least 57 bits, or to every remaining byte near the end of the stream.
|
||||
#[inline]
|
||||
fn refill(&mut self) {
|
||||
while self.available <= 56 {
|
||||
let Some(&b) = self.src.get(self.pos) else { return };
|
||||
self.pos += 1;
|
||||
self.acc = (self.acc << 8) | u64::from(b);
|
||||
self.available += 8;
|
||||
}
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn get(&mut self, n: u32) -> Result<u32> {
|
||||
if self.available < n {
|
||||
self.refill();
|
||||
if self.available < n {
|
||||
return Err(corrupt("truncated entropy stream"));
|
||||
}
|
||||
}
|
||||
self.available -= n;
|
||||
Ok(((self.acc >> self.available) & ((1u64 << n) - 1)) as u32)
|
||||
}
|
||||
|
||||
/// Reads the unary prefix: the zeros before the next one bit, which is consumed. Errors exactly where reading
|
||||
/// bit by bit would: more than `limit` zeros, or a stream that ends first.
|
||||
#[inline]
|
||||
fn zeros(&mut self, limit: u32) -> Result<u32> {
|
||||
let mut zeros = 0;
|
||||
loop {
|
||||
if self.available == 0 {
|
||||
self.refill();
|
||||
if self.available == 0 {
|
||||
return Err(corrupt("truncated entropy stream"));
|
||||
}
|
||||
}
|
||||
// the unread bits, left-aligned; the zeros shifted in below them are cut off by `min`
|
||||
let run = (self.acc << (64 - self.available)).leading_zeros().min(self.available);
|
||||
if zeros + run > limit {
|
||||
return Err(corrupt("invalid residual escape"));
|
||||
}
|
||||
zeros += run;
|
||||
if run < self.available {
|
||||
self.available -= run + 1;
|
||||
return Ok(zeros);
|
||||
}
|
||||
self.available = 0;
|
||||
}
|
||||
}
|
||||
|
||||
fn residual(&mut self, k: u32, bits: u32, range: u32, limit: u32) -> Result<i32> {
|
||||
let zeros = self.zeros(limit)?;
|
||||
let code = if zeros == limit { self.get(bits)? + 1 } else { (zeros << k) | self.get(k)? };
|
||||
if code > range {
|
||||
return Err(corrupt(&format!(
|
||||
"residual {code} outside sample range {range} (k={k}, prefix={zeros}, bits={bits}, position={})",
|
||||
self.pos * 8 - self.available as usize
|
||||
)));
|
||||
}
|
||||
Ok(if code & 1 == 0 { (code / 2) as i32 } else { -((code / 2) as i32) - 1 })
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy)]
|
||||
struct State {
|
||||
sum: u32,
|
||||
count: u32,
|
||||
}
|
||||
|
||||
impl State {
|
||||
fn k(self) -> u32 {
|
||||
let mut k = 0;
|
||||
while self.count << k < self.sum {
|
||||
k += 1;
|
||||
}
|
||||
k
|
||||
}
|
||||
|
||||
fn update(&mut self, delta: i32) {
|
||||
self.sum += delta.unsigned_abs();
|
||||
if self.count == 64 {
|
||||
self.sum >>= 1;
|
||||
self.count >>= 1;
|
||||
}
|
||||
self.count += 1;
|
||||
}
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn quant(v: i32, near: u32) -> i32 {
|
||||
let t = [18 + 3 * near, 67 + 5 * near, 276 + 7 * near];
|
||||
let a = v.unsigned_abs();
|
||||
let bucket = if a <= near {
|
||||
0
|
||||
} else if a < t[0] {
|
||||
1
|
||||
} else if a < t[1] {
|
||||
2
|
||||
} else if a < t[2] {
|
||||
3
|
||||
} else {
|
||||
4
|
||||
};
|
||||
if v < 0 { -bucket } else { bucket }
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn even_prediction(nw: i32, n: i32, ne: i32, nn: i32) -> i32 {
|
||||
let (a, b, c) = ((nw - n).abs(), (ne - n).abs(), (nn - n).abs());
|
||||
let pair = if a > b && a > c {
|
||||
ne + nn
|
||||
} else if b > a && b > c {
|
||||
nw + nn
|
||||
} else {
|
||||
nw + ne
|
||||
};
|
||||
(pair + 2 * n) >> 2
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn sample(rows: &[Vec<i32>], y: usize, x: usize) -> Result<i32> {
|
||||
rows.get(y).and_then(|r| r.get(x)).copied().ok_or_else(|| corrupt("invalid predictor position"))
|
||||
}
|
||||
|
||||
fn decode_stripe(stripe: &Stripe<'_>, height: usize, bits: u32, xtrans: bool) -> Result<Vec<u16>> {
|
||||
let size = if xtrans { BLOCK * 2 / 3 } else { BLOCK / 2 };
|
||||
let mut previous = vec![vec![vec![0; size + 2]; 2]; 3];
|
||||
let max = (1u32 << bits) - 1;
|
||||
let range = |near: u32| (max + 2 * near) / (2 * near + 1) + 1;
|
||||
let seed = |near: u32| State { sum: ((range(near) + 32) / 64).max(2), count: 1 };
|
||||
let mut states = [[[seed(0); 41]; 2]; 3];
|
||||
// Smooth regions retain finer precision and independent state across block quantizer changes.
|
||||
let mut fine = [[[[seed(0); 5]; 2]; 3], [[[seed(1); 5]; 2]; 3], [[[seed(2); 5]; 2]; 3]];
|
||||
let mut last_near = None;
|
||||
let mut reader = Bits::new(stripe.bytes);
|
||||
let n = height.checked_mul(stripe.width).filter(|&n| n <= crate::MAX_SAMPLES).ok_or(RawError::Limit("image too large"))?;
|
||||
// a capacity hint only: the stripe's declared height is the file's word, so the reservation is capped at a sample
|
||||
// per bit of the stripe's real bytes (all stripes reserve in parallel; a one-byte stripe must not take 2 GiB)
|
||||
let mut output = Vec::with_capacity(n.min(stripe.bytes.len().saturating_mul(8)));
|
||||
for group in 0..height / 6 {
|
||||
let near = match stripe.quantizers {
|
||||
Some(q) => u32::from(*q.get(group).ok_or_else(|| corrupt("truncated quantizer table"))?),
|
||||
None => 0,
|
||||
};
|
||||
if last_near != Some(near) {
|
||||
states = [[[seed(near); 41]; 2]; 3];
|
||||
last_near = Some(near);
|
||||
}
|
||||
let mut rows = previous;
|
||||
for pair in 0..6 {
|
||||
let colours = if pair % 2 == 0 { [0, 1] } else { [1, 2] };
|
||||
for &colour in &colours {
|
||||
let lines = rows.get_mut(colour).ok_or_else(|| corrupt("invalid colour vector"))?;
|
||||
let mut row = vec![0; size + 2];
|
||||
let prev = lines.len() - 1;
|
||||
let left = sample(lines, prev, 1)?;
|
||||
let right = sample(lines, prev, size)?;
|
||||
if let Some(v) = row.first_mut() {
|
||||
*v = left;
|
||||
}
|
||||
if let Some(v) = row.last_mut() {
|
||||
*v = right;
|
||||
}
|
||||
lines.push(row);
|
||||
}
|
||||
for even in (0..size + 8).step_by(2) {
|
||||
for (odd, position) in [(0, Some(even)), (1, even.checked_sub(7))] {
|
||||
let Some(pos) = position.filter(|&x| x < size) else { continue };
|
||||
let x = pos + 1;
|
||||
for (which, &colour) in colours.iter().enumerate() {
|
||||
let lines = rows.get_mut(colour).ok_or_else(|| corrupt("invalid colour vector"))?;
|
||||
let y = lines.len() - 1;
|
||||
let (nw, north, ne, nn) =
|
||||
(sample(lines, y - 1, x - 1)?, sample(lines, y - 1, x)?, sample(lines, y - 1, x + 1)?, sample(lines, y - 2, x)?);
|
||||
let west = sample(lines, y, x - 1)?;
|
||||
let east = sample(lines, y, x + 1)?;
|
||||
let (predict, d1, d2) = if odd == 0 {
|
||||
(even_prediction(nw, north, ne, nn), north - nn, nw - north)
|
||||
} else {
|
||||
let p = if (nw.min(ne)..=nw.max(ne)).contains(&north) { (west + east) >> 1 } else { (west + east + 2 * north) >> 2 };
|
||||
(p, north - nw, nw - west)
|
||||
};
|
||||
let real = !xtrans || MASKS.get(pair).and_then(|m| m.get(which)).is_some_and(|m| (m >> (pos % 4)) & 1 != 0);
|
||||
let value = if real {
|
||||
let contrast = d1.unsigned_abs() + d2.unsigned_abs();
|
||||
let precision = match contrast {
|
||||
0..=5 => 0,
|
||||
6 => near.min(1),
|
||||
7 => near.min(2),
|
||||
_ => near,
|
||||
};
|
||||
let (context, state) = if precision < near {
|
||||
let sign = |v: i32| if v.unsigned_abs() <= precision { 0 } else { v.signum() };
|
||||
let context = 3 * sign(d1) + sign(d2);
|
||||
let state = fine
|
||||
.get_mut(precision as usize)
|
||||
.and_then(|f| f.get_mut(pair % 3))
|
||||
.and_then(|s| s.get_mut(odd))
|
||||
.and_then(|s| s.get_mut(context.unsigned_abs() as usize))
|
||||
.ok_or_else(|| corrupt("invalid fine coding context"))?;
|
||||
(context, state)
|
||||
} else {
|
||||
let context = 9 * quant(d1, near) + quant(d2, near);
|
||||
let state = states
|
||||
.get_mut(pair % 3)
|
||||
.and_then(|s| s.get_mut(odd))
|
||||
.and_then(|s| s.get_mut(context.unsigned_abs() as usize))
|
||||
.ok_or_else(|| corrupt("invalid coding context"))?;
|
||||
(context, state)
|
||||
};
|
||||
let coded_range = range(precision);
|
||||
let coded_bits = 32 - (coded_range - 1).leading_zeros();
|
||||
let mut delta = reader.residual(state.k(), coded_bits, coded_range, 4 * bits - coded_bits - 1)?;
|
||||
state.update(delta);
|
||||
if context < 0 {
|
||||
delta = -delta;
|
||||
}
|
||||
let step = (2 * precision + 1) as i32;
|
||||
let mut value = predict + delta * step;
|
||||
if value < -(precision as i32) {
|
||||
value += coded_range as i32 * step;
|
||||
} else if value > (max + precision) as i32 {
|
||||
value -= coded_range as i32 * step;
|
||||
}
|
||||
value.clamp(0, max as i32)
|
||||
} else {
|
||||
predict
|
||||
};
|
||||
*lines.get_mut(y).and_then(|r| r.get_mut(x)).ok_or_else(|| corrupt("invalid predictor position"))? = value;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for y in 0..6 {
|
||||
for x in 0..stripe.width {
|
||||
let colour = if xtrans {
|
||||
XTRANS.get(y).and_then(|r| r.get(x % 6)).copied().ok_or_else(|| corrupt("invalid CFA position"))?
|
||||
} else {
|
||||
match (y % 2, x % 2) {
|
||||
(0, 0) => 0,
|
||||
(1, 1) => 2,
|
||||
_ => 1,
|
||||
}
|
||||
};
|
||||
let row = 2 + if colour == 1 { y } else { y / 2 };
|
||||
let col = 1 + if xtrans { 2 * (x / 3) + (x % 3).min(1) } else { x / 2 };
|
||||
output.push(sample(rows.get(colour).ok_or_else(|| corrupt("invalid colour vector"))?, row, col)? as u16);
|
||||
}
|
||||
}
|
||||
previous = rows.into_iter().map(|mut r| r.split_off(r.len() - 2)).collect();
|
||||
}
|
||||
Ok(output)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::Cfa;
|
||||
use proptest::prelude::*;
|
||||
|
||||
/// A one-byte stripe declaring the tallest height the header can carry fails as corrupt (its capacity hint is
|
||||
/// capped at the stripe's bits, not the declared height × 768 samples).
|
||||
#[test]
|
||||
fn a_tiny_stripe_with_a_huge_height_is_corrupt() {
|
||||
let stripe = Stripe { bytes: &[0], quantizers: None, width: BLOCK };
|
||||
for xtrans in [false, true] {
|
||||
let got = decode_stripe(&stripe, 65532, 14, xtrans);
|
||||
assert!(matches!(got, Err(RawError::Corrupt(_))), "xtrans {xtrans}: {got:?}");
|
||||
}
|
||||
}
|
||||
|
||||
/// Procedural zero residuals: no camera media or encoder dependency.
|
||||
fn zero_stream(bits: u32, xtrans: bool, quantizers: Option<&[u8]>) -> Vec<u8> {
|
||||
let groups = quantizers.map_or(3, <[u8]>::len);
|
||||
let mut main = [[(1u32 << (bits - 6), 1u32); 2]; 3];
|
||||
let mut fine = main;
|
||||
let mut previous = None;
|
||||
let mut stream = Vec::new();
|
||||
for group in 0..groups {
|
||||
let q = quantizers.map_or(0, |q| q[group]);
|
||||
if previous != Some(q) {
|
||||
main = [[(1u32 << (bits - 6), 1u32); 2]; 3];
|
||||
previous = Some(q);
|
||||
}
|
||||
for (pair, masks) in MASKS.iter().enumerate() {
|
||||
let width = if xtrans { 512 } else { 384 };
|
||||
for even in (0usize..width + 8).step_by(2) {
|
||||
for (parity, column) in [(0, Some(even)), (1, even.checked_sub(7))] {
|
||||
let Some(column) = column.filter(|&x| x < width) else { continue };
|
||||
for mask in masks {
|
||||
if xtrans && (mask >> (column % 4)) & 1 == 0 {
|
||||
continue;
|
||||
}
|
||||
let (sum, count) = &mut if q == 0 { &mut main } else { &mut fine }[pair % 3][parity];
|
||||
let k = 32 - (sum.saturating_sub(1) / *count).leading_zeros();
|
||||
stream.push(true);
|
||||
stream.extend(std::iter::repeat_n(false, k as usize));
|
||||
if *count == 64 {
|
||||
*sum /= 2;
|
||||
*count /= 2;
|
||||
}
|
||||
*count += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
stream.chunks(8).map(|chunk| chunk.iter().enumerate().fold(0, |b, (i, bit)| b | (u8::from(*bit) << (7 - i)))).collect()
|
||||
}
|
||||
|
||||
fn compressed(bits: u32, xtrans: bool, quantizers: Option<&[u8]>) -> Vec<u8> {
|
||||
let groups = quantizers.map_or(3, <[u8]>::len);
|
||||
let stream = zero_stream(bits, xtrans, quantizers);
|
||||
// Two stripes; the second is mostly padding, which must never enter the output.
|
||||
let mut out = b"IS".to_vec();
|
||||
out.extend_from_slice(&[u8::from(quantizers.is_none()), if xtrans { 16 } else { 0 }, bits as u8]);
|
||||
for v in [groups as u16 * 6, 1536, 780, 768] {
|
||||
out.extend_from_slice(&v.to_be_bytes());
|
||||
}
|
||||
out.push(2);
|
||||
out.extend_from_slice(&(groups as u16).to_be_bytes());
|
||||
for _ in 0..2 {
|
||||
out.extend_from_slice(&(stream.len() as u32).to_be_bytes());
|
||||
}
|
||||
out.resize(32, 0);
|
||||
if let Some(q) = quantizers {
|
||||
for _ in 0..2 {
|
||||
out.extend_from_slice(q);
|
||||
out.resize(out.len() + q.len().div_ceil(16) * 16 - q.len(), 0);
|
||||
}
|
||||
}
|
||||
out.extend_from_slice(&stream);
|
||||
out.extend_from_slice(&stream);
|
||||
out
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lossless_and_lossy_bayer_and_xtrans_stripes() {
|
||||
for bits in [12, 14, 16] {
|
||||
for xtrans in [false, true] {
|
||||
for q in [None, Some([2, 0, 5].as_slice())] {
|
||||
let src = compressed(bits, xtrans, q);
|
||||
let out = decode(&src, 780, 18, bits, xtrans, Mode::Full, &Limits::default()).unwrap();
|
||||
assert_eq!(out, vec![0; 780 * 18]);
|
||||
assert!(decode(&src, 780, 18, bits, xtrans, Mode::Header, &Limits::default()).unwrap().is_empty());
|
||||
let layout = xtrans.then(|| {
|
||||
let mut l = [0; 36];
|
||||
for (to, from) in l.iter_mut().zip(XTRANS.iter().flatten().rev()) {
|
||||
*to = *from as u8;
|
||||
}
|
||||
l
|
||||
});
|
||||
let raf = crate::raf::tests::raf(780, 18, bits, src, layout, &[]);
|
||||
let image = crate::decode(&raf).unwrap();
|
||||
assert_eq!(crate::probe_info(&raf).unwrap(), image.info());
|
||||
assert_eq!(image.data, crate::RawData::U16(out));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn malformed_headers_and_truncated_entropy_are_errors() {
|
||||
let valid = compressed(14, true, Some(&[1, 2, 0]));
|
||||
for len in (0..96).chain([valid.len() / 2, valid.len() - 1]) {
|
||||
assert!(decode(&valid[..len], 780, 18, 14, true, Mode::Full, &Limits::default()).is_err(), "length {len}");
|
||||
}
|
||||
for at in [2, 3, 4, 5, 7, 9, 11, 13, 14, 16] {
|
||||
let mut src = valid.clone();
|
||||
src[at] ^= 0xff;
|
||||
assert!(decode(&src, 780, 18, 14, true, Mode::Full, &Limits::default()).is_err(), "header byte {at}");
|
||||
}
|
||||
assert!(decode(&valid, usize::MAX, usize::MAX, 14, true, Mode::Full, &Limits::default()).is_err());
|
||||
let mut reader = Bits::new(&[0; 64]);
|
||||
assert!(reader.residual(0, 14, 16384, 41).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unsupported_container_modes_and_cfa_are_not_silently_decoded() {
|
||||
let layout = std::array::from_fn(|i| Cfa::xtrans().pattern[35 - i]);
|
||||
let src = compressed(14, true, None);
|
||||
let raf = crate::raf::tests::raf(780, 18, 14, src, Some(layout), &[]);
|
||||
let mut mismatched = raf.clone();
|
||||
mismatched[108..112].copy_from_slice(&3u32.to_be_bytes());
|
||||
assert!(crate::decode(&mismatched).is_err(), "declared lossy mode disagrees with lossless stream");
|
||||
let bad_layout = layout.map(|c| {
|
||||
if c == 0 {
|
||||
2
|
||||
} else if c == 2 {
|
||||
0
|
||||
} else {
|
||||
c
|
||||
}
|
||||
});
|
||||
let raf = crate::raf::tests::raf(780, 18, 14, compressed(14, true, None), Some(bad_layout), &[]);
|
||||
assert!(matches!(crate::decode(&raf), Err(RawError::Unsupported(_))), "compression uses a fixed X-Trans arrangement");
|
||||
let mut raf = crate::raf::tests::raf(780, 18, 16, vec![0; 780 * 18 * 2], None, &[]);
|
||||
for mode in [1u32, 4, u32::MAX] {
|
||||
raf[108..112].copy_from_slice(&mode.to_be_bytes());
|
||||
assert!(matches!(crate::decode(&raf), Err(RawError::Unsupported(_))), "unknown mode {mode}");
|
||||
assert!(matches!(crate::probe_info(&raf), Err(RawError::Unsupported(_))), "header-only mode {mode}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn residual_sign_wrap_and_escape() {
|
||||
// Ordinary codes 0, 1, 2, 3 with k=2; then the escape for -8192.
|
||||
let mut stream = vec![true, false, false, true, false, true, true, true, false, true, true, true];
|
||||
stream.extend(std::iter::repeat_n(false, 41));
|
||||
stream.push(true);
|
||||
let escaped = 16382u32;
|
||||
stream.extend((0..14).rev().map(|i| (escaped >> i) & 1 != 0));
|
||||
let bytes: Vec<u8> = stream.chunks(8).map(|chunk| chunk.iter().enumerate().fold(0, |b, (i, bit)| b | (u8::from(*bit) << (7 - i)))).collect();
|
||||
let mut reader = Bits::new(&bytes);
|
||||
for delta in [0, -1, 1, -2, -8192] {
|
||||
assert_eq!(reader.residual(2, 14, 16384, 41).unwrap(), delta);
|
||||
}
|
||||
assert_eq!(even_prediction(1112, 858, 1104, 1152), 983);
|
||||
}
|
||||
|
||||
proptest! {
|
||||
#![proptest_config(ProptestConfig::with_cases(256))]
|
||||
#[test]
|
||||
fn malformed_streams_never_panic(data in prop::collection::vec(any::<u8>(), 0..2048), quantizer in any::<u8>(), bits in prop::sample::select(vec![12,14,16]), xtrans in any::<bool>()) {
|
||||
let mut src = compressed(bits, xtrans, Some(&[quantizer]));
|
||||
let start = 64;
|
||||
src.truncate(start);
|
||||
for i in [16, 20] { src[i..i+4].copy_from_slice(&(data.len() as u32).to_be_bytes()); }
|
||||
src.extend_from_slice(&data);
|
||||
src.extend_from_slice(&data);
|
||||
let _ = decode(&src, 780, 6, bits, xtrans, Mode::Full, &Limits::default());
|
||||
}
|
||||
|
||||
/// The word-at-a-time reader returns what reading bit by bit returns, and fails where it fails: long zero
|
||||
/// runs, the escape limit and the end of the stream included.
|
||||
#[test]
|
||||
fn bit_reader_matches_reading_bit_by_bit(
|
||||
data in prop::collection::vec(prop_oneof![3 => Just(0u8), 1 => any::<u8>()], 0..64),
|
||||
reads in prop::collection::vec((any::<bool>(), 0u32..=24, 0u32..=63), 1..200),
|
||||
) {
|
||||
let bit = |i: usize| data.get(i / 8).map(|b| (b >> (7 - i % 8)) & 1 == 1);
|
||||
let mut at = 0usize;
|
||||
let mut reader = Bits::new(&data);
|
||||
for (prefix, n, limit) in reads {
|
||||
let expected = if prefix {
|
||||
let mut zeros = 0;
|
||||
loop {
|
||||
let Some(b) = bit(at) else { break Err("truncated") };
|
||||
at += 1;
|
||||
if b {
|
||||
break Ok(zeros);
|
||||
}
|
||||
zeros += 1;
|
||||
if zeros > limit {
|
||||
break Err("escape");
|
||||
}
|
||||
}
|
||||
} else if at + n as usize > data.len() * 8 {
|
||||
Err("truncated")
|
||||
} else {
|
||||
let value = (0..n as usize).fold(0, |v, i| (v << 1) | u32::from(bit(at + i) == Some(true)));
|
||||
at += n as usize;
|
||||
Ok(value)
|
||||
};
|
||||
let actual = if prefix { reader.zeros(limit) } else { reader.get(n) };
|
||||
match (expected, actual) {
|
||||
(Ok(want), Ok(got)) => prop_assert_eq!(want, got),
|
||||
(Err(want), Err(got)) => {
|
||||
prop_assert!(got.to_string().contains(want), "{got}");
|
||||
break;
|
||||
}
|
||||
(want, got) => prop_assert!(false, "expected {want:?}, got {got:?}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -168,8 +168,79 @@ fn rw2_samples_decode_unmutated() {
|
||||
}
|
||||
}
|
||||
|
||||
/// A Fujifilm RAF: header, records, raw block (LE TIFF whose IFD0 points to a 0xf000 raw IFD).
|
||||
fn raf_file(w: u32, h: u32, bits: u32, strip: Vec<u8>, layout: Option<[u8; 36]>, jpeg: &[u8]) -> Vec<u8> {
|
||||
let mut block = b"II*\0\x08\0\0\0".to_vec();
|
||||
let sub_off: u32 = 8 + 2 + 12 + 4;
|
||||
block.extend_from_slice(&1u16.to_le_bytes());
|
||||
block.extend_from_slice(&[0x00, 0xf0, 13, 0, 1, 0, 0, 0]);
|
||||
block.extend_from_slice(&sub_off.to_le_bytes());
|
||||
block.extend_from_slice(&0u32.to_le_bytes());
|
||||
let n = 7u32;
|
||||
let wb_off = sub_off + 2 + 12 * n + 4;
|
||||
let strip_off = wb_off + 12;
|
||||
let entries: [(u16, u32, u32); 7] = [
|
||||
(0xf001, 1, w),
|
||||
(0xf002, 1, h),
|
||||
(0xf003, 1, bits),
|
||||
(0xf007, 1, strip_off),
|
||||
(0xf008, 1, strip.len() as u32),
|
||||
(0xf00a, 1, 64),
|
||||
(0xf00e, 3, wb_off),
|
||||
];
|
||||
block.extend_from_slice(&(n as u16).to_le_bytes());
|
||||
for (tag, count, v) in entries {
|
||||
block.extend_from_slice(&tag.to_le_bytes());
|
||||
block.extend_from_slice(&4u16.to_le_bytes());
|
||||
block.extend_from_slice(&count.to_le_bytes());
|
||||
block.extend_from_slice(&v.to_le_bytes());
|
||||
}
|
||||
block.extend_from_slice(&0u32.to_le_bytes());
|
||||
for v in [300u32, 600, 450] {
|
||||
block.extend_from_slice(&v.to_le_bytes());
|
||||
}
|
||||
block.extend_from_slice(&strip);
|
||||
let mut recs: Vec<u8> = Vec::new();
|
||||
let mut nrec = 0u32;
|
||||
let mut add = |tag: u16, d: &[u8]| {
|
||||
recs.extend_from_slice(&tag.to_be_bytes());
|
||||
recs.extend_from_slice(&(d.len() as u16).to_be_bytes());
|
||||
recs.extend_from_slice(d);
|
||||
nrec += 1;
|
||||
};
|
||||
add(0x0100, &[(h >> 8) as u8, h as u8, (w >> 8) as u8, w as u8]);
|
||||
add(0x0110, &[0, 2, 0, 4]);
|
||||
add(0x0111, &[0, (h - 2) as u8, 0, (w - 4) as u8]);
|
||||
if let Some(l) = layout {
|
||||
add(0x0131, &l);
|
||||
}
|
||||
let mut dir = nrec.to_be_bytes().to_vec();
|
||||
dir.extend_from_slice(&recs);
|
||||
let mut out = b"FUJIFILMCCD-RAW 0201FF000000TEST".to_vec();
|
||||
out.resize(108, 0);
|
||||
let jpeg_off = 160u32;
|
||||
let dir_off = jpeg_off + jpeg.len() as u32;
|
||||
let raw_off = dir_off + dir.len() as u32;
|
||||
for (i, v) in [jpeg_off, jpeg.len() as u32, dir_off, dir.len() as u32, raw_off, block.len() as u32].iter().enumerate() {
|
||||
out[84 + i * 4..88 + i * 4].copy_from_slice(&v.to_be_bytes());
|
||||
}
|
||||
out.resize(160, 0);
|
||||
out.extend_from_slice(jpeg);
|
||||
out.extend_from_slice(&dir);
|
||||
out.extend_from_slice(&block);
|
||||
out
|
||||
}
|
||||
|
||||
/// A RAF with an X-Trans layout and a stub JPEG.
|
||||
fn raf() -> Vec<u8> {
|
||||
let strip: Vec<u8> = (0..(W * H * 2) as usize).map(|i| (i * 11) as u8).collect();
|
||||
let layout: [u8; 36] = std::array::from_fn(|i| (i % 3) as u8);
|
||||
raf_file(W, H, 16, strip, Some(layout), b"\xff\xd8\xff\xd9")
|
||||
}
|
||||
|
||||
fn samples() -> Vec<Vec<u8>> {
|
||||
vec![
|
||||
raf(),
|
||||
rw2(),
|
||||
cr2(),
|
||||
cfa_tiff("NIKON CORPORATION", 1, 12, ByteOrder::Big),
|
||||
|
||||
Reference in new issue
Block a user