* craft-raw: the file-format layer of camera raw files, with DNG and CRW
A new crate for #587 and #778: bytes in, the sensor data out as a neutral
RawImage (samples, CFA, levels, active area, crop, orientation, colour tags,
opcode lists, profile tables), and back for DNG. One Cargo feature per format:
dng (read and write), jxl (JPEG XL DNG), crw (through craft-crw).
The base is LightCraft's crates/raw at 5ff567a4: dng, dngwrite, tiffraw,
unpack, ljpeg, jxl, semantic, preview, the parsing half of opcodes, profile
and gaintable, the model and probe. Demosaicing and everything after it stay
with the develop pipeline.
Ported from PhotoCraft's and FilmCraft's DNG readers, each with a test: the
default crop's per-tag defaults, an unusable crop falling back to the active
area, non-finite white levels, the white default of a linearized image, the
AsShotWhiteXY range, 8-bit-only lossy JPEG, lossy tiles that must cover their
part of the image, and the CinemaDNG frame tags.
* craft-raw: Hasselblad 3FR and FFF (feature hasselblad)
RawFormat::CfaTiff decodes: the DNG-layout reader plus what these files add.
Uncompressed data or Hasselblad's lossless JPEG (selection value 8), RGGB when
no CFAPattern is stated, an FFF's colour matrix from the 3FR header it embeds,
and the 8-bit RGB preview handed on as a baseline TIFF.
Moved from LightCraft's vendor/hasselblad.rs at 5ff567a4 with its tests.
Ported from PhotoCraft's reader: four BlackLevel values without a
BlackLevelRepeatDim are one per CFA site in a file that is not a DNG.
* craft-raw: Minolta MRW, Epson ERF, Mamiya MEF and Kodak DCS520C
Four formats only LightCraft decodes, one feature each (mrw, erf, mef, kodak),
moved from its vendor modules at 5ff567a4 with their tests; the MRW block
container comes from lightcraft-meta. src/shared.rs holds the level helpers
the maker decoders share.
ERF and MEF state no white balance: the estimate LightCraft made from the
preview is develop-side work, so wb_multipliers is None for them here.
* craft-raw: shared scan budget for tagged previews, tile-sized lossy JPEG budget, bound on Deflate tile padding
Ported from storytold/lightcraft#3695. Candidate preview ranges are deduplicated and charged to one byte
budget and one header-walk budget; a lossy JPEG decoded for a tile may hold no more samples than the tile
rounded up to whole MCUs (and never more than Limits::max_lossy_tile_samples); a Deflate chunk whose nominal
size is mostly padding outside the image is refused before it is inflated.
* craft-raw: lossless JPEG sample bound per referenced byte; previews larger than the file are passed over
Ported from storytold/lightcraft#3910 (the parts whose code is in craft-raw today). check_image refuses a
lossless JPEG image that declares more than 32 samples per referenced byte, counted once per chunk that points
at the bytes so identical tiles stored once still decode. The Hasselblad RGB preview search passes over an
image whose pixel bytes exceed the file and still takes a smaller complete one.
* craft-raw: test for the Deflate padding bound
* craft-raw: Pentax PEF and Samsung SRW
Moved from LightCraft crates/raw (vendor/pef.rs, vendor/srw.rs) at 5ff567a4. The Huffman difference decoder
and bit reader that PEF shares with the Nikon, Panasonic and Sony codings become the huffman module.
* craft-raw: Canon CR2, with sRAW and mRAW
Moved from LightCraft crates/raw (vendor/cr2.rs, vendor/cr2/sraw.rs) at 5ff567a4. The white-balance presets a
CR2 lists come back as data in ColorData::kelvin_presets (the ordering from LightCraft's kelvin_presets.rs;
the temperature and tint arithmetic stays with the developer).
* craft-raw: an sRAW preview larger than the file is refused before its buffer is reserved
Ported from storytold/lightcraft#3910 (cr2::sraw::read_preview).
* WIP craft-raw: Nikon NEF/NRW (does not compile yet: limits threading, develop() calls in tests)
* craft-raw: Nikon NEF/NRW, with PhotoCraft's maker-note compression routing and 12-bit D4 white level
* craft-raw: Panasonic RW2 / RWL / RAW, every raw format
* craft-raw: RW2 format 8 refuses strips stating fewer bits than samples
Ported from storytold/lightcraft#3910.
* craft-raw: RW2 format 8 refuses samples above the declared bit depth
Ported from storytold/photocraft#11838.
* craft-raw: the RW2 format 8 robustness seed carries a valid stream
Its random payload decoded only while out-of-depth samples were clamped; mutations now start from a file that decodes.
* craft-raw: Fujifilm RAF, uncompressed and compressed, with lens corrections
* craft-raw: compressed RAF stripes cap their capacity hint at the stripe's bits
Ported from storytold/lightcraft#3910.
* craft-raw: test RAF orientation and camera from the embedded JPEG's Exif
* WIP craft-raw: Olympus ORF
* craft-raw: RAF embedded preview, and its range from the header alone (bounded_preview)
* craft-raw: Olympus ORF, the uncompressed layouts, with its maker-note preview and WB_GLevel
---------
Co-authored-by: radialmonster <radial@gmail.com>
Co-authored-by: wnanbei <wnanbei@gmail.com>
description="Camera raw files, the file-format layer: container parsing, decompression and the sensor data out as a neutral raw image with its levels, crop, orientation, colour tags and embedded previews. DNG (read and write), Hasselblad 3FR/FFF, MRW, ERF, MEF, Kodak DCS, Pentax PEF, Samsung SRW, Canon CR2, Nikon NEF/NRW, Panasonic RW2, Fujifilm RAF and Canon CRW so far."
description="Camera raw files, the file-format layer: container parsing, decompression and the sensor data out as a neutral raw image with its levels, crop, orientation, colour tags and embedded previews. DNG (read and write), Hasselblad 3FR/FFF, MRW, ERF, MEF, Kodak DCS, Pentax PEF, Samsung SRW, Canon CR2, Nikon NEF/NRW, Panasonic RW2, Fujifilm RAF, Olympus ORF (uncompressed) and Canon CRW so far."
@@ -57,6 +57,7 @@ One module and one Cargo feature per maker or container family. All are on by de
| `nef` | Nikon NEF / NRW | Uncompressed (16-bit words, 12-bit packed, Coolpix word packing), the packed layouts labelled compression 34713, Nikon's Huffman coding (lossless and lossy types, 12 and 14 bits, "lossy after split") with the `0x0096` linearisation table, small raw (YCbCr 4:2:2) as linear white-balanced RGB; black from `0x003d` or the NRW block, as-shot white balance from `0x000c`, `CropArea` or the measured image area of bodies that record none, masked trailing columns left out of the active area, the D1X pixel aspect, and the in-file distortion correction as a `WarpRectilinear` opcode ([derivation](docs/nikon-lens-corrections.md)). High Efficiency (HE / HE★) is refused by name; the previews still read |
| `rw2` | Panasonic RW2, Leica RWL, older Panasonic / Leica RAW | Packed blocks (formats 2, 5, 7), formats 4 and 6, format 8 prefix-coded strips and tiles (a sample outside the bit depth makes the strip corrupt), 16-bit words; black per colour from `0x001c`–`0x001e` (plus the 15 counts formats 4 and 6 store above it, 15 when no black is recorded); white from the saturation plateau; sensor borders as the active area and the in-camera aspect crop; mapped-out defect pixels filled; as-shot white balance from `0x0024`–`0x0026`; the in-camera distortion correction (`0x0119`) as a `WarpRectilinear` opcode. Raw formats 1 and 3 and compression 34826 are refused (no samples of them) |
| `raf` | Fujifilm RAF | The RAF header, records and raw IFD; uncompressed Bayer and X-Trans (16-bit words, 12-bit LSB-first, 32-bit word packing); Fujifilm's lossless and lossy striped compression ([derivation](docs/raf-compression.md), [corpus](docs/raf-corpus.sha256)); black, as-shot white balance and, for bodies without colour matrices, the camera's neutral multipliers for Illuminant A and D65 (`ColorData::kelvin_presets`, a neutral pair); orientation, make and model from the embedded JPEG's Exif; the in-file distortion and vignetting corrections as opcodes; the embedded preview, also by range from the header alone (`bounded_preview`, `embedded_preview_from_range`). The pre-2010 FinePix layout without a raw IFD is refused |
| `orf` | Olympus / OM System ORF | Uncompressed: 16-bit words (12-bit values in the top bits shifted down), 12-bit packed in 32-bit words (XZ-2), 12-bit two-field (older compacts), 16-byte blocks of ten 12-bit samples (E-300, E-330, E-500, high-resolution modes); CFA from Exif `CFAPattern`; black, as-shot white balance (relative to `WB_GLevel`) and crop from the maker note's ImageProcessing, or from the main note on bodies without it; the CameraSettings preview. Compressed ORF is refused ([why](docs/orf-compression.md)); its preview still reads |
| `mrw` | Minolta / Konica Minolta MRW | The `\0MRM` block container (`PRD`, `WBG`, `TTW`), 12-bit packed and 16-bit word mosaics, as-shot white balance from `WBG`, white from the saturation plateau |
| `erf` | Epson ERF (R-D1, R-D1s, R-D1x) | 12-bit samples in 16-byte blocks, black levels from the maker note, the measured picture origin. No white balance (the file states none) |
| `mef` | Mamiya MEF (ZD) | Uncompressed 12-bit packed; default crop from the RGB preview's measured window. No white balance (the file states none) |
@@ -157,6 +158,10 @@ Left as LightCraft has it, with the reason:
by the sample depth and the count is halved at 64, so the sum stays below 2^23. The Kelvin scale of bodies that
record only an A / D65 pair is kept as that pair (`KelvinPresets::from_neutral_pair`); LightCraft placed five
points along it with CIE temperature maths, which stays with the developer.
- Olympus ORF is LightCraft's reader with one thing from PhotoCraft's (uncompressed 16-bit only): the as-shot
red and blue levels are divided by ImageProcessing `WB_GLevel` (`0x011f`) when it is recorded, not always by
256. PhotoCraft's `ValidBits` (`0x0611`) and crop handling are covered by LightCraft's own (the main note's
`0x102c`, and the top-bit test on the samples). Compressed ORF is not decoded: see `docs/orf-compression.md`.
- Lossy JPEG is accepted for CFA data (one grey component) as in PhotoCraft; FilmCraft refuses it
because DNG 1.4 allows the coding for LinearRaw only.
Moved from LightCraft (`docs/orf-compression.md`) with the ORF reader.
LightCraft once decoded compressed ORF sensor data (its PR #3775) and then removed that decoder while its provenance remained unresolved. The earlier clean-room and derivation claims are withdrawn; this note makes no legal determination and is not a format specification.
Compressed ORF files are unsupported for sensor decoding (`RawError::Unsupported`). Their embedded JPEG preview is still returned when there is one.
Uncompressed 16-bit, packed 12-bit, two-field 12-bit and 12.8 bpp ORF support remains in `src/orf.rs`. This includes supported uncompressed high-resolution modes; it does not imply compressed-mode support on the same camera.
Any future compressed ORF decoder requires independently established provenance and validation before support is claimed again. An open LightCraft pull request (storytold/lightcraft#2068) proposes one; its provenance has not been established, so it is not ported here.
//! | `nef` | Nikon NEF and NRW: uncompressed, packed, Nikon's Huffman coding (lossless and lossy, 12 and 14 bits), small raw |
//! | `rw2` | Panasonic RW2, Leica RWL and the older Panasonic / Leica RAW: packed, formats 4 and 6, format 8 prefix-coded strips, 16-bit words |
//! | `raf` | Fujifilm RAF: uncompressed Bayer and X-Trans, Fujifilm's lossless and lossy compression, lens corrections |
//! | `orf` | Olympus / OM System ORF: uncompressed 16-bit, 12-bit packed and two-field layouts, 16-byte blocks; compressed refused |
//! | `pef` | Pentax PEF: uncompressed (packed or 16-bit words) and Pentax's Huffman coding (compression 65535) |
//! | `srw` | Samsung SRW: the uncompressed variants (compression 32769 words, 32770 at 12 bits packed) |
//! | `hasselblad` | Hasselblad 3FR and FFF (and other CFA TIFFs in the DNG layout without a DNG version): uncompressed and Hasselblad's lossless JPEG; implies `dng` |
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
