mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-14 06:22:24 +00:00
craft-raw: Pentax PEF and Samsung SRW (#919)
* craft-raw: the file-format layer of camera raw files, with DNG and CRW A new crate for #587 and #778: bytes in, the sensor data out as a neutral RawImage (samples, CFA, levels, active area, crop, orientation, colour tags, opcode lists, profile tables), and back for DNG. One Cargo feature per format: dng (read and write), jxl (JPEG XL DNG), crw (through craft-crw). The base is LightCraft's crates/raw at 5ff567a4: dng, dngwrite, tiffraw, unpack, ljpeg, jxl, semantic, preview, the parsing half of opcodes, profile and gaintable, the model and probe. Demosaicing and everything after it stay with the develop pipeline. Ported from PhotoCraft's and FilmCraft's DNG readers, each with a test: the default crop's per-tag defaults, an unusable crop falling back to the active area, non-finite white levels, the white default of a linearized image, the AsShotWhiteXY range, 8-bit-only lossy JPEG, lossy tiles that must cover their part of the image, and the CinemaDNG frame tags. * craft-raw: Hasselblad 3FR and FFF (feature hasselblad) RawFormat::CfaTiff decodes: the DNG-layout reader plus what these files add. Uncompressed data or Hasselblad's lossless JPEG (selection value 8), RGGB when no CFAPattern is stated, an FFF's colour matrix from the 3FR header it embeds, and the 8-bit RGB preview handed on as a baseline TIFF. Moved from LightCraft's vendor/hasselblad.rs at 5ff567a4 with its tests. Ported from PhotoCraft's reader: four BlackLevel values without a BlackLevelRepeatDim are one per CFA site in a file that is not a DNG. * craft-raw: Minolta MRW, Epson ERF, Mamiya MEF and Kodak DCS520C Four formats only LightCraft decodes, one feature each (mrw, erf, mef, kodak), moved from its vendor modules at 5ff567a4 with their tests; the MRW block container comes from lightcraft-meta. src/shared.rs holds the level helpers the maker decoders share. ERF and MEF state no white balance: the estimate LightCraft made from the preview is develop-side work, so wb_multipliers is None for them here. * craft-raw: shared scan budget for tagged previews, tile-sized lossy JPEG budget, bound on Deflate tile padding Ported from storytold/lightcraft#3695. Candidate preview ranges are deduplicated and charged to one byte budget and one header-walk budget; a lossy JPEG decoded for a tile may hold no more samples than the tile rounded up to whole MCUs (and never more than Limits::max_lossy_tile_samples); a Deflate chunk whose nominal size is mostly padding outside the image is refused before it is inflated. * craft-raw: lossless JPEG sample bound per referenced byte; previews larger than the file are passed over Ported from storytold/lightcraft#3910 (the parts whose code is in craft-raw today). check_image refuses a lossless JPEG image that declares more than 32 samples per referenced byte, counted once per chunk that points at the bytes so identical tiles stored once still decode. The Hasselblad RGB preview search passes over an image whose pixel bytes exceed the file and still takes a smaller complete one. * craft-raw: test for the Deflate padding bound * craft-raw: Pentax PEF and Samsung SRW Moved from LightCraft crates/raw (vendor/pef.rs, vendor/srw.rs) at 5ff567a4. The Huffman difference decoder and bit reader that PEF shares with the Nikon, Panasonic and Sony codings become the huffman module. --------- Co-authored-by: radialmonster <radial@gmail.com>
This commit is contained in:
Dylan M. Taylorandradialmonster authored and 
GitHub committed
1 parent
afcc37cf20
commit
f86766e7d6
11 files changed
+1446
-8
No files matched your search
@@ -92,8 +92,8 @@ Source provenance
|
||||
crates/raw (craft-raw) is moved from LightCraft, with behaviour ported from two other readers:
|
||||
- LightCraft, crates/raw (src/lib.rs, dng.rs, dngwrite.rs, tiffraw.rs, unpack.rs, ljpeg.rs, jxl.rs,
|
||||
opcodes.rs, profile.rs, gaintable.rs, semantic.rs, preview.rs, vendor/crw.rs, vendor/hasselblad.rs,
|
||||
vendor.rs (shared level helpers), vendor/mrw.rs, vendor/erf.rs, vendor/mef.rs, vendor/kodak.rs and their
|
||||
tests) and crates/meta/src/mrw.rs (the MRW block container),
|
||||
vendor.rs (shared level helpers), vendor/mrw.rs, vendor/erf.rs, vendor/mef.rs, vendor/kodak.rs, vendor/pef.rs,
|
||||
vendor/srw.rs, tests/it/vendor_robust.rs and their tests) and crates/meta/src/mrw.rs (the MRW block container),
|
||||
revision 5ff567a4a2e067bef1b6bce152c11e7f727ccbaf.
|
||||
https://github.com/storytold/lightcraft
|
||||
Copyright (c) 2026 ArtCraft Team and the LightCraft contributors.
|
||||
|
||||
@@ -107,7 +107,7 @@ craft-<name> = { git = "https://github.com/storytold/craft-libs", rev = "<commit
|
||||
what PhotoCraft's and FilmCraft's readers added: container parsing and decompression to a neutral
|
||||
raw image (samples, CFA, black and white levels, active area, crop, orientation, colour tags,
|
||||
opcode lists, embedded previews), one Cargo feature per format. DNG reading and writing,
|
||||
Hasselblad 3FR/FFF, Minolta MRW, Epson ERF, Mamiya MEF, Kodak DCS and Canon CRW (through
|
||||
Hasselblad 3FR/FFF, Minolta MRW, Epson ERF, Mamiya MEF, Kodak DCS, Pentax PEF, Samsung SRW and Canon CRW (through
|
||||
`craft-crw`) so far; demosaicing and everything after stay with the develop
|
||||
pipeline.
|
||||
- [`craft-idml`](crates/idml/README.md): IDML (InDesign Markup Language) packages: the ZIP container read with bounded part sizes and lenient part names, `designmap.xml` with its includes in place, the `<Properties>` form on craft-xml elements, value codecs and the package writer; and InDesign Tagged Text, read and written.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "craft-raw"
|
||||
description = "Camera raw files, the file-format layer: container parsing, decompression and the sensor data out as a neutral raw image with its levels, crop, orientation, colour tags and embedded previews. DNG (read and write), Hasselblad 3FR/FFF, MRW, ERF, MEF, Kodak DCS and Canon CRW so far."
|
||||
description = "Camera raw files, the file-format layer: container parsing, decompression and the sensor data out as a neutral raw image with its levels, crop, orientation, colour tags and embedded previews. DNG (read and write), Hasselblad 3FR/FFF, MRW, ERF, MEF, Kodak DCS, Pentax PEF, Samsung SRW and Canon CRW so far."
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
license.workspace = true
|
||||
@@ -11,7 +11,7 @@ repository.workspace = true
|
||||
workspace = true
|
||||
|
||||
[features]
|
||||
default = ["dng", "jxl", "crw", "hasselblad", "mrw", "erf", "mef", "kodak"]
|
||||
default = ["dng", "jxl", "crw", "hasselblad", "mrw", "erf", "mef", "kodak", "pef", "srw"]
|
||||
## Adobe DNG, reading and writing: uncompressed, lossless JPEG, lossy JPEG, Deflate.
|
||||
dng = ["dep:miniz_oxide", "dep:zune-jpeg", "dep:zune-core"]
|
||||
## JPEG XL compressed DNGs (DNG 1.7, compression 52546) via jxl-oxide (pure Rust, MIT OR Apache-2.0).
|
||||
@@ -26,6 +26,10 @@ erf = []
|
||||
mef = []
|
||||
## Kodak DCS520C raw TIFF.
|
||||
kodak = []
|
||||
## Pentax PEF: uncompressed and Pentax's Huffman coding.
|
||||
pef = []
|
||||
## Samsung SRW: the uncompressed variants.
|
||||
srw = []
|
||||
## Canon CRW (CIFF) through craft-crw, the one CRW decoder.
|
||||
crw = ["dep:craft-crw", "dep:chrono"]
|
||||
|
||||
|
||||
@@ -57,11 +57,13 @@ One module and one Cargo feature per maker or container family. All are on by de
|
||||
| `erf` | Epson ERF (R-D1, R-D1s, R-D1x) | 12-bit samples in 16-byte blocks, black levels from the maker note, the measured picture origin. No white balance (the file states none) |
|
||||
| `mef` | Mamiya MEF (ZD) | Uncompressed 12-bit packed; default crop from the RGB preview's measured window. No white balance (the file states none) |
|
||||
| `kodak` | Kodak DCS520C TIFF | Two-component lossless JPEG CFA with the `GrayResponseCurve`, white balance from the `KodakIFD` |
|
||||
| `pef` | Pentax PEF | Uncompressed (packed 12-bit or 16-bit words, also under the PackBits tag some bodies write) and Pentax's Huffman coding (compression 65535, the table from the maker note); image area from the maker note's crop tags, else the dark borders trimmed from the samples; black from `BlackPoint` or the masked columns; as-shot white balance from the maker note; the layout from the Exif `CFAPattern` |
|
||||
| `srw` | Samsung SRW | The uncompressed variants: 16-bit words (compression 32769) and 12 bits packed (32770 with exactly the packed size; the bit order is decided from the samples); levels and white balance from the keyed maker-note vectors; framing from `SensorAreas`. The compressed codings (32770 with fewer bytes, 32772, 32773) are reported as unsupported |
|
||||
| `hasselblad` | Hasselblad 3FR and FFF, and other CFA TIFFs in the DNG layout without a DNG version tag (`RawFormat::CfaTiff`) | Uncompressed and Hasselblad's lossless JPEG (predictor selection value 8); RGGB from the sensor origin when the file states no `CFAPattern`; an FFF's colour matrix from the 3FR header it embeds; the 8-bit RGB preview handed on as a baseline TIFF. Implies `dng` |
|
||||
|
||||
Not decoded: GoPro VC-5 DNG (GPR), CFA with more than three colours, non-rectangular CFA layouts.
|
||||
`probe` names every container it can tell apart whatever the features (CR2, CR3, NEF, NRW, ARW,
|
||||
RAF, ORF, RW2, PEF, SRW, MRW, X3F, other TIFF-based raws);
|
||||
RAF, ORF, RW2, X3F, other TIFF-based raws);
|
||||
their decoders are still in the apps and move here one maker family at a time.
|
||||
`RawFormat::is_supported` says what this build decodes.
|
||||
|
||||
@@ -116,6 +118,9 @@ Left as LightCraft has it, with the reason:
|
||||
estimated one from the embedded preview's neutral pixels; that is an estimate from picture
|
||||
content, so it stays with the developer and `wb_multipliers` is `None` here. The inputs it needs
|
||||
(samples, crop, CFA, levels, preview) are all in `RawImage` and `embedded_preview`.
|
||||
- Pentax PEF and Samsung SRW are LightCraft's readers unchanged. PhotoCraft has no reader for either: it sends
|
||||
them down its generic TIFF/EP path, which reads an uncompressed CFA IFD and refuses Pentax's Huffman coding
|
||||
and Samsung's private compressions, so there was nothing of PhotoCraft's to fold in.
|
||||
- Lossy JPEG is accepted for CFA data (one grey component) as in PhotoCraft; FilmCraft refuses it
|
||||
because DNG 1.4 allows the coding for LinearRaw only.
|
||||
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
//! The table-driven Huffman decoder and MSB-first bit reader that the makers' difference codings share (Pentax PEF,
|
||||
//! Nikon NEF, Panasonic RW2, Sony ARW6): a code names a difference category, the category's additional bits follow
|
||||
//! as in ITU-T T.81 F.1.2.1.
|
||||
|
||||
use crate::{RawError, Result};
|
||||
|
||||
/// A canonical-by-table Huffman decoder with a 12-bit lookup.
|
||||
pub(crate) struct Huffman {
|
||||
/// Indexed by the next 12 bits: (symbol, length); length 0 = invalid.
|
||||
lut: Vec<(u8, u8)>,
|
||||
}
|
||||
|
||||
impl Huffman {
|
||||
/// From `(left-aligned 12-bit code, length)` per symbol.
|
||||
pub fn new(codes: &[(u16, u8)]) -> Result<Huffman> {
|
||||
let mut lut = vec![(0u8, 0u8); 4096];
|
||||
for (sym, &(code, len)) in codes.iter().enumerate() {
|
||||
if len == 0 {
|
||||
continue;
|
||||
}
|
||||
if len > 12 || sym > 16 {
|
||||
return Err(RawError::Unsupported(format!("PEF Huffman code of length {len}")));
|
||||
}
|
||||
let first = (code >> (12 - len) << (12 - len)) as usize;
|
||||
for e in &mut lut[first..first + (1 << (12 - len))] {
|
||||
if e.1 == 0 {
|
||||
*e = (sym as u8, len);
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(Huffman { lut })
|
||||
}
|
||||
}
|
||||
|
||||
/// MSB-first bit reader that yields zeros past the end.
|
||||
pub(crate) struct Bits<'a> {
|
||||
src: &'a [u8],
|
||||
pos: usize,
|
||||
acc: u64,
|
||||
n: u32,
|
||||
}
|
||||
|
||||
impl<'a> Bits<'a> {
|
||||
pub fn new(src: &'a [u8]) -> Self {
|
||||
Bits { src, pos: 0, acc: 0, n: 0 }
|
||||
}
|
||||
#[inline]
|
||||
fn fill(&mut self) {
|
||||
while self.n <= 56 {
|
||||
let b = self.src.get(self.pos).copied().unwrap_or(0);
|
||||
self.pos += 1;
|
||||
self.acc |= (b as u64) << (56 - self.n);
|
||||
self.n += 8;
|
||||
}
|
||||
}
|
||||
#[inline]
|
||||
pub fn peek(&mut self, k: u32) -> u32 {
|
||||
if self.n < k {
|
||||
self.fill();
|
||||
}
|
||||
(self.acc >> (64 - k)) as u32
|
||||
}
|
||||
#[inline]
|
||||
pub fn skip(&mut self, k: u32) {
|
||||
self.acc <<= k;
|
||||
self.n -= k;
|
||||
}
|
||||
#[inline]
|
||||
pub fn get(&mut self, k: u32) -> u32 {
|
||||
if k == 0 {
|
||||
return 0;
|
||||
}
|
||||
let v = self.peek(k);
|
||||
self.skip(k);
|
||||
v
|
||||
}
|
||||
/// Bits handed out so far (may exceed the source length: zeros are read past its end).
|
||||
pub fn consumed_bits(&self) -> usize {
|
||||
self.pos * 8 - self.n as usize
|
||||
}
|
||||
/// Whether more bits were handed out than the source holds (the zeros read past its end are lookahead padding,
|
||||
/// not data; using them would fabricate samples).
|
||||
pub fn overrun(&self) -> bool {
|
||||
self.consumed_bits() > self.src.len() * 8
|
||||
}
|
||||
}
|
||||
|
||||
/// Decode one difference: Huffman category then T.81 additional bits.
|
||||
#[inline]
|
||||
pub(crate) fn diff(bits: &mut Bits, h: &Huffman) -> Option<i32> {
|
||||
let (sym, len) = h.lut[bits.peek(12) as usize];
|
||||
if len == 0 {
|
||||
return None;
|
||||
}
|
||||
bits.skip(len as u32);
|
||||
let s = sym as u32;
|
||||
if s == 0 {
|
||||
return Some(0);
|
||||
}
|
||||
if s > 16 {
|
||||
return None;
|
||||
}
|
||||
let v = bits.get(s) as i32;
|
||||
Some(if v < 1 << (s - 1) { v - (1 << s) + 1 } else { v })
|
||||
}
|
||||
@@ -29,6 +29,8 @@
|
||||
//! | `erf` | Epson ERF (R-D1 family): 12-bit samples in 16-byte blocks |
|
||||
//! | `mef` | Mamiya MEF (ZD): uncompressed 12-bit packed |
|
||||
//! | `kodak` | Kodak DCS520C TIFF: two-component lossless JPEG with a response curve |
|
||||
//! | `pef` | Pentax PEF: uncompressed (packed or 16-bit words) and Pentax's Huffman coding (compression 65535) |
|
||||
//! | `srw` | Samsung SRW: the uncompressed variants (compression 32769 words, 32770 at 12 bits packed) |
|
||||
//! | `hasselblad` | Hasselblad 3FR and FFF (and other CFA TIFFs in the DNG layout without a DNG version): uncompressed and Hasselblad's lossless JPEG; implies `dng` |
|
||||
//!
|
||||
//! [`probe`] names every container it can tell apart whatever the features; [`RawFormat::is_supported`] says
|
||||
@@ -58,6 +60,9 @@ mod erf;
|
||||
pub mod gaintable;
|
||||
#[cfg(feature = "hasselblad")]
|
||||
mod hasselblad;
|
||||
// (the Huffman difference decoder Pentax, Nikon, Panasonic and Sony codings share)
|
||||
#[allow(dead_code)]
|
||||
mod huffman;
|
||||
#[cfg(feature = "jxl")]
|
||||
mod jxl;
|
||||
#[cfg(feature = "kodak")]
|
||||
@@ -68,6 +73,8 @@ mod mef;
|
||||
#[cfg(feature = "mrw")]
|
||||
mod mrw;
|
||||
pub mod opcodes;
|
||||
#[cfg(feature = "pef")]
|
||||
mod pef;
|
||||
mod preview;
|
||||
pub mod profile;
|
||||
#[cfg(feature = "dng")]
|
||||
@@ -75,6 +82,8 @@ pub mod semantic;
|
||||
// (the TIFF pixel plumbing every TIFF-based maker shares; DNG is its only user so far)
|
||||
#[allow(dead_code)] // each maker uses some of these helpers; which are live depends on the features
|
||||
mod shared;
|
||||
#[cfg(feature = "srw")]
|
||||
mod srw;
|
||||
#[cfg_attr(not(feature = "dng"), allow(dead_code))]
|
||||
mod tiffraw;
|
||||
#[cfg_attr(not(feature = "dng"), allow(dead_code))]
|
||||
@@ -314,6 +323,8 @@ impl RawFormat {
|
||||
|| (self == RawFormat::Mrw && cfg!(feature = "mrw"))
|
||||
|| (self == RawFormat::Erf && cfg!(feature = "erf"))
|
||||
|| (self == RawFormat::Mef && cfg!(feature = "mef"))
|
||||
|| (self == RawFormat::Pef && cfg!(feature = "pef"))
|
||||
|| (self == RawFormat::Srw && cfg!(feature = "srw"))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -594,6 +605,10 @@ fn decode_in(bytes: &[u8], mode: Mode, limits: &Limits) -> Result<RawImage> {
|
||||
RawFormat::Erf => erf::decode(bytes, mode, limits),
|
||||
#[cfg(feature = "mef")]
|
||||
RawFormat::Mef => mef::decode(bytes, mode, limits),
|
||||
#[cfg(feature = "pef")]
|
||||
RawFormat::Pef => pef::decode(bytes, mode, limits),
|
||||
#[cfg(feature = "srw")]
|
||||
RawFormat::Srw => srw::decode(bytes, mode, limits),
|
||||
RawFormat::OtherTiff => Err(RawError::Unsupported(other_tiff_reason(bytes))),
|
||||
other => Err(RawError::Unsupported(format!("{other:?} files are not decoded by this build"))),
|
||||
}
|
||||
|
||||
@@ -0,0 +1,619 @@
|
||||
//! Pentax PEF — uncompressed and Pentax Huffman-compressed (compression 65535).
|
||||
//!
|
||||
//! Sources: TIFF 6.0 (container), ITU-T T.81 (JPEG) Annex H / F.1.2.1 for the lossless "difference category +
|
||||
//! additional bits" coding, the ExifTool Pentax tag-name documentation (maker-note `0x0200` BlackPoint, `0x0201`
|
||||
//! WhitePoint i.e. WB levels, `0x0220` HuffmanTable) and black-box analysis of CC0 samples from
|
||||
//! raw.pixls.us (K10D, K-5 II s, K-3):
|
||||
//!
|
||||
//! - The Huffman table is stored in the file (maker note `0x0220`, in the maker note's byte order): a `u16` `d`,
|
||||
//! 12 further bytes, then `n = d + 12` `u16` codes left-aligned in 12 bits, then `n` code lengths (bytes).
|
||||
//! Symbol `i` is the difference category `i` (T.81 Table H.2: the number of additional bits).
|
||||
//! - The bit stream (MSB-first, no byte stuffing, continuous across rows) codes one difference per pixel; the
|
||||
//! additional bits follow T.81 F.1.2.1 (a leading 0 means a negative value).
|
||||
//! - Prediction: each pixel is predicted from the previous same-colour pixel in its row (two to the left); the
|
||||
//! first two pixels of a row are predicted from the first two pixels of the previous row of the same parity
|
||||
//! (starting at 0). Found by trying candidate predictors and checking the decoded rows are continuous.
|
||||
//! - Maker note `0x0038`/`0x0039`: the image area's left/top and width/height (verified against the data).
|
||||
|
||||
use crate::huffman::{Bits, Huffman, diff};
|
||||
use crate::shared::{black_from_columns, cfa_from_exif, white_from_data};
|
||||
|
||||
use crate::tiffraw::{Packing, read_image_in};
|
||||
use crate::{BlackLevel, Camera, Cfa, ColorData, Limits, Mode, OpcodeLists, Orientation, RawData, RawError, RawFormat, RawImage, Rect, Result};
|
||||
use craft_tiff::image::{ImageInfo, chunk_bytes};
|
||||
use craft_tiff::{ByteOrder, Tiff, makernote, tags as t};
|
||||
|
||||
const CROP_ORIGIN: u16 = 0x0038;
|
||||
const CROP_SIZE: u16 = 0x0039;
|
||||
const BLACK_POINT: u16 = 0x0200;
|
||||
const WB_LEVELS: u16 = 0x0201;
|
||||
const HUFFMAN: u16 = 0x0220;
|
||||
|
||||
/// Parse the maker-note Huffman table.
|
||||
pub(crate) fn table(b: &[u8], order: ByteOrder) -> Result<Huffman> {
|
||||
let u16_at = |i: usize| b.get(i..i + 2).map(|s| order.u16([s[0], s[1]]));
|
||||
let d = u16_at(0).ok_or_else(|| RawError::Corrupt("PEF Huffman table too short".into()))? as usize;
|
||||
let n = d + 12;
|
||||
if n > 17 || b.len() < 14 + 3 * n {
|
||||
return Err(RawError::Corrupt(format!("PEF Huffman table: {n} symbols in {} bytes", b.len())));
|
||||
}
|
||||
let codes: Vec<(u16, u8)> = (0..n).map(|i| (u16_at(14 + 2 * i).unwrap_or(0) & 0x0fff, b[14 + 2 * n + i])).collect();
|
||||
Huffman::new(&codes)
|
||||
}
|
||||
|
||||
/// Decode a `w × h` Huffman-coded image.
|
||||
pub(crate) fn decode_huffman(src: &[u8], h: &Huffman, w: usize, hgt: usize, bits_per: u32) -> Result<Vec<u16>> {
|
||||
// every sample takes at least one bit: a stream too short for the declared size is refused before its picture
|
||||
// is allocated, so a few bytes can't ask for gigabytes (storytold/lightcraft#1512)
|
||||
let samples = w.checked_mul(hgt).ok_or(RawError::Limit("image too large"))?;
|
||||
if (src.len() as u64).saturating_mul(8) < samples as u64 {
|
||||
return Err(RawError::Corrupt(format!("PEF: {w} x {hgt} samples need more than the {} bytes of data there", src.len())));
|
||||
}
|
||||
let mut out = vec![0u16; samples];
|
||||
let mut bits = Bits::new(src);
|
||||
let mut vpred = [[0i32; 2]; 2];
|
||||
let max = (1i32 << bits_per.min(16)) - 1;
|
||||
for y in 0..hgt {
|
||||
let mut hpred = [0i32; 2];
|
||||
let row = &mut out[y * w..(y + 1) * w];
|
||||
for (x, o) in row.iter_mut().enumerate() {
|
||||
let d = diff(&mut bits, h).ok_or_else(|| RawError::Corrupt(format!("PEF: invalid Huffman code at row {y}")))?;
|
||||
// a run of large differences in a corrupt stream can drive a predictor past i32 (storytold/lightcraft#1515)
|
||||
let overflow = || RawError::Corrupt(format!("PEF: predictor out of range at row {y}"));
|
||||
let p = if x < 2 {
|
||||
let v = vpred[y & 1][x].checked_add(d).ok_or_else(overflow)?;
|
||||
vpred[y & 1][x] = v;
|
||||
hpred[x] = v;
|
||||
v
|
||||
} else {
|
||||
let v = hpred[x & 1].checked_add(d).ok_or_else(overflow)?;
|
||||
hpred[x & 1] = v;
|
||||
v
|
||||
};
|
||||
*o = p.clamp(0, max) as u16;
|
||||
}
|
||||
if bits.overrun() {
|
||||
return Err(RawError::Corrupt(format!("PEF: data ends at row {y} of {hgt}")));
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// The image area without leading/trailing lines that hold no picture (masked or empty), kept at even offsets. With
|
||||
/// the maker-note `black` level, a line is dark when its mean is at most 1/256 of the remaining range (at least 16)
|
||||
/// above it, the tolerance `cr2::masked_columns` uses: masked lines sit at the black level plus noise, while dark
|
||||
/// real content (a night sky) is well above. Without it, below 5% of the interior's mean.
|
||||
fn dark_trimmed(d: &[u16], w: usize, h: usize, black: Option<f64>, white: f32) -> Rect {
|
||||
if w < 256 || h < 256 {
|
||||
return Rect::new(0, 0, w, h);
|
||||
}
|
||||
let (xstep, ystep) = (w / 256, h / 256);
|
||||
let at = |x: usize, y: usize| d.get(y * w + x).map_or(0.0, |&v| f64::from(v));
|
||||
let col_mean = |x: usize| (0..h).step_by(ystep).map(|y| at(x, y)).sum::<f64>() / h.div_ceil(ystep) as f64;
|
||||
let row_mean = |y: usize| (0..w).step_by(xstep).map(|x| at(x, y)).sum::<f64>() / w.div_ceil(xstep) as f64;
|
||||
let interior = (w / 4..w * 3 / 4).step_by(w / 64).map(col_mean).sum::<f64>() / 32.0;
|
||||
let (base, tol) = match black {
|
||||
Some(b) => (b, ((f64::from(white) - b) / 256.0).max(16.0)),
|
||||
None => (0.0, interior * 0.05),
|
||||
};
|
||||
// a frame no brighter than masked lines (a lens-cap shot) has nothing to tell them from picture
|
||||
if interior - base <= tol {
|
||||
return Rect::new(0, 0, w, h);
|
||||
}
|
||||
let range = f64::from(white) - base;
|
||||
// lines from one edge that hold no picture: within `tol` of the outermost line's level when that sits a little
|
||||
// above black (K-70, K-1: 65 over BlackPoint), and saturated lines when dark ones follow (645Z)
|
||||
let run = |mean: &dyn Fn(usize) -> f64| {
|
||||
let saturated = (0..64).take_while(|&k| mean(k) - base >= range * 0.98).count();
|
||||
let outer = mean(saturated) - base;
|
||||
let level = if outer > 0.0 && outer < range * 0.01 { outer } else { 0.0 };
|
||||
match (saturated..64).take_while(|&k| mean(k) - base < level + tol).count() {
|
||||
0 => 0,
|
||||
dark => saturated + dark,
|
||||
}
|
||||
};
|
||||
let edges = |n: usize, mean: &dyn Fn(usize) -> f64| {
|
||||
let lo = run(mean).next_multiple_of(2);
|
||||
let hi = n - run(&|k| mean(n - 1 - k));
|
||||
(lo, (hi - lo) & !1)
|
||||
};
|
||||
let (left, width) = edges(w, &col_mean);
|
||||
let (top, height) = edges(h, &row_mean);
|
||||
Rect::new(left, top, width, height)
|
||||
}
|
||||
|
||||
/// Whether every strip or tile of a one-sample-per-pixel image holds exactly the bytes that its rows need when
|
||||
/// the samples are packed MSB-first at the declared bit depth, rows starting on byte boundaries (TIFF 6.0
|
||||
/// section 7). A real PackBits stream of such an image has a different length (almost always shorter), so the
|
||||
/// exact match is what tells plain samples under a compression tag that says otherwise.
|
||||
fn is_exactly_packed(info: &ImageInfo, file_len: u64) -> bool {
|
||||
let bits = u64::from(info.bits());
|
||||
if info.samples_per_pixel != 1 || info.planar != 1 || !(1..=16).contains(&bits) || info.byte_counts.len() != info.offsets.len() {
|
||||
return false;
|
||||
}
|
||||
let chunks = info.chunks(file_len);
|
||||
!chunks.is_empty() && chunks.iter().all(|c| c.len == (u64::from(c.width) * bits).div_ceil(8) * u64::from(c.height))
|
||||
}
|
||||
|
||||
pub(crate) fn decode(bytes: &[u8], mode: Mode, limits: &Limits) -> Result<RawImage> {
|
||||
let tiff = Tiff::parse(bytes)?;
|
||||
let ifd0 = tiff.ifds.first().ok_or_else(|| RawError::Corrupt("TIFF without IFDs".into()))?;
|
||||
let info = ifd0.image()?;
|
||||
let (w, hgt) = (info.width as usize, info.height as usize);
|
||||
let bits = info.bits() as u32;
|
||||
if w == 0 || hgt == 0 || w.saturating_mul(hgt) > limits.samples() {
|
||||
return Err(RawError::Limit("image too large"));
|
||||
}
|
||||
let make = ifd0.string(t::MAKE).unwrap_or_default();
|
||||
let mn =
|
||||
tiff.exif().and_then(|e| e.get(t::MAKER_NOTE)).and_then(|e| makernote::parse_makernote(bytes, e.offset, e.count() as u64, tiff.order, &make));
|
||||
let pair = |tag: u16| mn.as_ref().and_then(|m| m.ifd.u64s(tag)).filter(|v| v.len() == 2).map(|v| (v[0] as usize, v[1] as usize));
|
||||
let tagged = match (pair(CROP_ORIGIN), pair(CROP_SIZE)) {
|
||||
// checked: the tags are file data; `x + cw` on a hostile value would wrap past the bounds
|
||||
(Some((x, y)), Some((cw, ch)))
|
||||
if cw > 0 && ch > 0 && x.checked_add(cw).is_some_and(|end| end <= w) && y.checked_add(ch).is_some_and(|end| end <= hgt) =>
|
||||
{
|
||||
Some(Rect::new(x, y, cw, ch))
|
||||
}
|
||||
_ => None,
|
||||
};
|
||||
// without crop tags the image area is found from the samples (dark borders)
|
||||
let read = if tagged.is_some() { mode } else { Mode::Full };
|
||||
let data = match info.compression {
|
||||
65535 => {
|
||||
let mn = mn.as_ref().ok_or_else(|| RawError::Corrupt("compressed PEF without maker note".into()))?;
|
||||
let tb = mn.ifd.bytes(HUFFMAN).ok_or_else(|| RawError::Unsupported("compressed PEF without a Huffman table".into()))?;
|
||||
let huff = table(tb, mn.order)?;
|
||||
let chunks = info.chunks(bytes.len() as u64);
|
||||
let first = chunks.first().ok_or_else(|| RawError::Corrupt("PEF without image data".into()))?;
|
||||
let mut c = *first;
|
||||
c.len = chunks.iter().fold(0u64, |a, c| a.saturating_add(c.len)).max(first.len);
|
||||
let src = chunk_bytes(bytes, &c)
|
||||
.or_else(|| bytes.get(first.offset as usize..))
|
||||
.ok_or_else(|| RawError::Corrupt("PEF data outside file".into()))?;
|
||||
if read == Mode::Full { RawData::U16(decode_huffman(src, &huff, w, hgt, bits)?) } else { RawData::U16(Vec::new()) }
|
||||
}
|
||||
1 => {
|
||||
let strip = info.chunks(bytes.len() as u64).iter().fold(0u64, |a, c| a.saturating_add(c.len));
|
||||
let packing = if strip >= (w * hgt * 2) as u64 { Packing::Word16 } else { Packing::Msb };
|
||||
read_image_in(read, bytes, &info, tiff.order, packing, limits)?
|
||||
}
|
||||
// Some bodies tag plain packed samples as PackBits (32773). Only a strip that holds exactly the packed
|
||||
// size of its rows is read, as the uncompressed MSB-first samples it is; any other size stays unsupported.
|
||||
32773 if is_exactly_packed(&info, bytes.len() as u64) => {
|
||||
let packed = ImageInfo { compression: 1, ..info.clone() };
|
||||
read_image_in(read, bytes, &packed, tiff.order, Packing::Msb, limits)?
|
||||
}
|
||||
c => return Err(RawError::Unsupported(format!("PEF compression {c}"))),
|
||||
};
|
||||
let RawData::U16(ref samples) = data else { return Err(RawError::Unsupported("float PEF".into())) };
|
||||
|
||||
let white = white_from_data(samples, bits);
|
||||
let black_point = mn.as_ref().and_then(|m| m.ifd.f64s(BLACK_POINT)).filter(|v| v.len() == 4);
|
||||
let active = match tagged {
|
||||
Some(r) => r,
|
||||
None => dark_trimmed(samples, w, hgt, black_point.as_deref().map(|v| v.iter().sum::<f64>() / 4.0), white),
|
||||
};
|
||||
let cfa = match (ifd0.u64s(t::CFA_REPEAT_PATTERN_DIM).as_deref(), ifd0.bytes(t::CFA_PATTERN_EP)) {
|
||||
(Some([2, 2]), Some(p)) if p.len() == 4 && p.iter().all(|&c| c <= 2) => Cfa { width: 2, height: 2, pattern: p.to_vec() },
|
||||
// Every body states its layout in the Exif `CFAPattern`, read from the first sample of the stored array, not
|
||||
// of the image area: the *ist D (origin 19, 11) and the 645D (80, 59) only fit unshifted. The 12-bit 10 MP
|
||||
// bodies and everything from the K-3 on say RGGB; the K-7 / K-5 / 645D generation says BGGR. Files without the
|
||||
// tag keep BGGR.
|
||||
_ => cfa_from_exif(&tiff).unwrap_or_else(|| Cfa::bayer_static("BGGR")),
|
||||
};
|
||||
let black = match black_point.as_deref() {
|
||||
Some(v @ [_, _, _, _]) => {
|
||||
// RGGB-ordered levels → per CFA position at the active area origin
|
||||
let a = cfa.shifted(active.x, active.y);
|
||||
let values = a.pattern.iter().map(|&c| [v[0], (v[1] + v[2]) / 2.0, v[3]][c as usize] as f32).collect();
|
||||
BlackLevel { repeat_rows: 2, repeat_cols: 2, values, ..Default::default() }
|
||||
}
|
||||
_ if active.x >= 4 => black_from_columns(samples, w, 0..active.x.saturating_sub(2), 0..hgt, active),
|
||||
_ => BlackLevel::uniform(0.0),
|
||||
};
|
||||
let wb = mn.as_ref().and_then(|m| m.ifd.f64s(WB_LEVELS)).filter(|v| v.len() == 4 && v.iter().all(|x| *x > 0.0)).map(|v| {
|
||||
let g = (v[1] + v[2]) / 2.0;
|
||||
[(v[0] / g) as f32, 1.0, (v[3] / g) as f32]
|
||||
});
|
||||
let img = RawImage {
|
||||
format: RawFormat::Pef,
|
||||
width: w,
|
||||
height: hgt,
|
||||
cpp: 1,
|
||||
data,
|
||||
cfa: Some(cfa),
|
||||
bits,
|
||||
black,
|
||||
white: vec![white],
|
||||
active_area: active,
|
||||
crop: Rect::new(0, 0, active.width, active.height),
|
||||
orientation: Orientation::from_exif(ifd0.u16(t::ORIENTATION).unwrap_or(1)),
|
||||
color: ColorData::default(),
|
||||
wb_multipliers: wb,
|
||||
linearized: false,
|
||||
pixel_aspect: 1.0,
|
||||
opcodes: OpcodeLists::default(),
|
||||
camera: tiff.ifds.first().map(Camera::from_ifd0).unwrap_or_default(),
|
||||
cinema: Default::default(),
|
||||
};
|
||||
img.validate_for(mode)?;
|
||||
Ok(img)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// Our own prefix code for categories 0..=14: lengths 2,2,3,3,4,5,..., codes assigned canonically.
|
||||
fn test_table() -> (Vec<u8>, Vec<(u16, u8)>) {
|
||||
let lens: [u8; 15] = [3, 2, 2, 3, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 12];
|
||||
let mut order: Vec<usize> = (0..15).collect();
|
||||
order.sort_by_key(|&i| (lens[i], i));
|
||||
let mut codes = vec![(0u16, 0u8); 15];
|
||||
let (mut code, mut prev) = (0u32, lens[order[0]]);
|
||||
for (k, &i) in order.iter().enumerate() {
|
||||
if k > 0 {
|
||||
code = (code + 1) << (lens[i] - prev);
|
||||
}
|
||||
prev = lens[i];
|
||||
codes[i] = ((code << (12 - lens[i])) as u16, lens[i]);
|
||||
}
|
||||
let mut b = vec![0u8, 3];
|
||||
b.extend_from_slice(&[0; 12]);
|
||||
for c in &codes {
|
||||
b.extend_from_slice(&c.0.to_be_bytes());
|
||||
}
|
||||
b.extend(codes.iter().map(|c| c.1));
|
||||
(b, codes)
|
||||
}
|
||||
|
||||
/// A compressed PEF declaring far more samples than its data could hold (one bit each at least) is refused
|
||||
/// before its picture is allocated (storytold/lightcraft#1512).
|
||||
#[test]
|
||||
fn a_stream_too_short_for_its_size_is_refused_before_allocating() {
|
||||
let (tb, _) = test_table();
|
||||
let h = table(&tb, ByteOrder::Big).unwrap();
|
||||
let t = std::time::Instant::now();
|
||||
let r = decode_huffman(&[0x55; 4096], &h, 30_000, 30_000, 12);
|
||||
assert!(matches!(r, Err(RawError::Corrupt(ref m)) if m.contains("need more")), "{r:?}");
|
||||
assert!(t.elapsed() < std::time::Duration::from_secs(1));
|
||||
}
|
||||
|
||||
fn encode(px: &[u16], w: usize, codes: &[(u16, u8)]) -> Vec<u8> {
|
||||
let mut bits: Vec<bool> = Vec::new();
|
||||
let put = |v: u32, n: u8, bits: &mut Vec<bool>| (0..n).rev().for_each(|i| bits.push(v >> i & 1 == 1));
|
||||
let mut vpred = [[0i32; 2]; 2];
|
||||
for (y, row) in px.chunks(w).enumerate() {
|
||||
let mut hpred = [0i32; 2];
|
||||
for (x, &v) in row.iter().enumerate() {
|
||||
let pred = if x < 2 { vpred[y & 1][x] } else { hpred[x & 1] };
|
||||
let d = v as i32 - pred;
|
||||
if x < 2 {
|
||||
vpred[y & 1][x] = v as i32;
|
||||
}
|
||||
hpred[x & 1] = v as i32;
|
||||
let s = if d == 0 { 0 } else { 32 - d.unsigned_abs().leading_zeros() } as u8;
|
||||
let (c, l) = codes[s as usize];
|
||||
put((c >> (12 - l)) as u32, l, &mut bits);
|
||||
if s > 0 {
|
||||
let extra = if d < 0 { (d - 1) as u32 & ((1 << s) - 1) } else { d as u32 };
|
||||
put(extra, s, &mut bits);
|
||||
}
|
||||
}
|
||||
}
|
||||
bits.chunks(8).map(|c| c.iter().enumerate().fold(0u8, |a, (i, &b)| a | (b as u8) << (7 - i))).collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn huffman_round_trip() {
|
||||
let (tb, codes) = test_table();
|
||||
let h = table(&tb, ByteOrder::Big).unwrap();
|
||||
let (w, hgt) = (37usize, 9usize);
|
||||
let px: Vec<u16> = (0..w * hgt).map(|i| ((i * 7919) % 16384) as u16).collect();
|
||||
let src = encode(&px, w, &codes);
|
||||
assert_eq!(decode_huffman(&src, &h, w, hgt, 14).unwrap(), px);
|
||||
// truncated data is reported, not a panic
|
||||
assert!(decode_huffman(&src[..src.len() / 3], &h, w, hgt, 14).is_err());
|
||||
assert!(table(&tb[..20], ByteOrder::Big).is_err());
|
||||
}
|
||||
|
||||
/// Zero-padded lookahead past the end must not be decoded as samples: with no entropy at all, the zero bits
|
||||
/// are a valid code (category 1) in this table, so a reader that only limits the refill position accepts them.
|
||||
#[test]
|
||||
fn symbols_decoded_from_padding_past_the_end_are_rejected() {
|
||||
let (tb, codes) = test_table();
|
||||
let h = table(&tb, ByteOrder::Big).unwrap();
|
||||
assert!(decode_huffman(&[], &h, 2, 1, 14).is_err());
|
||||
// the same image with its real entropy still decodes, including the last byte's padding bits
|
||||
let px = [5u16, 9];
|
||||
let src = encode(&px, 2, &codes);
|
||||
assert_eq!(decode_huffman(&src, &h, 2, 1, 14).unwrap(), px);
|
||||
// one byte short of the real entropy is rejected
|
||||
assert!(decode_huffman(&src[..src.len() - 1], &h, 2, 1, 14).is_err());
|
||||
}
|
||||
|
||||
/// A long run of the largest difference drove a predictor past i32: a panic in builds with overflow checks,
|
||||
/// wrapped values otherwise (storytold/lightcraft#1515). Both the row and the column predictors now stop with an error.
|
||||
#[test]
|
||||
fn predictor_overflow_is_an_error_not_a_panic() {
|
||||
let (tb, codes) = test_table();
|
||||
let h = table(&tb, ByteOrder::Big).unwrap();
|
||||
let (c, l) = codes[14];
|
||||
let d = (1u32 << 14) - 1;
|
||||
let stream = |n: usize| {
|
||||
let mut bits: Vec<bool> = Vec::with_capacity(n * (l as usize + 14));
|
||||
for _ in 0..n {
|
||||
(0..l).rev().for_each(|i| bits.push((c >> (12 - l)) >> i & 1 == 1));
|
||||
(0..14).rev().for_each(|i| bits.push(d >> i & 1 == 1));
|
||||
}
|
||||
bits.chunks(8).map(|c| c.iter().enumerate().fold(0u8, |a, (i, &b)| a | (b as u8) << (7 - i))).collect::<Vec<u8>>()
|
||||
};
|
||||
// differences one predictor needs to pass i32::MAX
|
||||
let steps = (i32::MAX as u32 / d) as usize + 2;
|
||||
// along a row: every other sample adds to the same predictor
|
||||
let w = 2 * steps + 2;
|
||||
let err = decode_huffman(&stream(w), &h, w, 1, 14).unwrap_err();
|
||||
assert!(matches!(err, RawError::Corrupt(ref m) if m.contains("predictor")), "{err}");
|
||||
// down two columns: each row adds to the predictor of its parity
|
||||
let rows = 2 * steps + 2;
|
||||
let err = decode_huffman(&stream(2 * rows), &h, 2, rows, 14).unwrap_err();
|
||||
assert!(matches!(err, RawError::Corrupt(ref m) if m.contains("predictor")), "{err}");
|
||||
}
|
||||
|
||||
/// A big-endian one-strip PEF-style TIFF of `w × h` 12-bit samples with the given compression tag and strip
|
||||
/// byte count, around `payload`.
|
||||
fn tiny_pef(w: u16, h: u16, compression: u16, strip_len: u32, payload: &[u8]) -> Vec<u8> {
|
||||
let entries: [(u16, u16, u32); 9] = [
|
||||
(0x0100, 3, w as u32 * 0x1_0000), // ImageWidth
|
||||
(0x0101, 3, h as u32 * 0x1_0000), // ImageLength
|
||||
(0x0102, 3, 12 * 0x1_0000), // BitsPerSample
|
||||
(0x0103, 3, compression as u32 * 0x1_0000), // Compression
|
||||
(0x0106, 3, 32803 * 0x1_0000), // Photometric: CFA
|
||||
(0x0111, 4, 8 + 2 + 9 * 12 + 4), // StripOffsets
|
||||
(0x0115, 3, 0x1_0000), // SamplesPerPixel
|
||||
(0x0116, 3, h as u32 * 0x1_0000), // RowsPerStrip
|
||||
(0x0117, 4, strip_len), // StripByteCounts
|
||||
];
|
||||
let mut f = b"MM\0*\0\0\0\x08".to_vec();
|
||||
f.extend_from_slice(&9u16.to_be_bytes());
|
||||
for (tag, ty, val) in entries {
|
||||
f.extend_from_slice(&tag.to_be_bytes());
|
||||
f.extend_from_slice(&ty.to_be_bytes());
|
||||
f.extend_from_slice(&1u32.to_be_bytes());
|
||||
f.extend_from_slice(&val.to_be_bytes());
|
||||
}
|
||||
f.extend_from_slice(&0u32.to_be_bytes()); // no next IFD
|
||||
f.extend_from_slice(payload);
|
||||
f
|
||||
}
|
||||
|
||||
/// Eight 12-bit samples, two rows of four, packed MSB-first: 0x123 0x456 0x789 0xabc / 0xdef 0x012 0x345 0x678.
|
||||
const PACKED: [u8; 12] = [0x12, 0x34, 0x56, 0x78, 0x9a, 0xbc, 0xde, 0xf0, 0x12, 0x34, 0x56, 0x78];
|
||||
|
||||
#[test]
|
||||
fn packed_samples_under_the_packbits_tag_are_read_when_the_strip_has_exactly_the_packed_size() {
|
||||
let file = tiny_pef(4, 2, 32773, 12, &PACKED);
|
||||
let img = decode(&file, Mode::Full, &Limits::default()).unwrap();
|
||||
let RawData::U16(v) = &img.data else { panic!("integer samples") };
|
||||
assert_eq!(v, &[0x123, 0x456, 0x789, 0xabc, 0xdef, 0x012, 0x345, 0x678]);
|
||||
assert_eq!((img.width, img.height, img.bits), (4, 2, 12));
|
||||
// the same bytes tagged as plain uncompressed give the same samples
|
||||
let plain = decode(&tiny_pef(4, 2, 1, 12, &PACKED), Mode::Full, &Limits::default()).unwrap();
|
||||
assert_eq!(plain.data, img.data);
|
||||
// header-only mode accepts it too
|
||||
assert!(decode(&file, Mode::Header, &Limits::default()).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_packbits_tagged_strip_of_any_other_size_stays_unsupported() {
|
||||
for len in [11u32, 13, 6, 24] {
|
||||
let mut payload = PACKED.to_vec();
|
||||
payload.resize(24, 0x55);
|
||||
let file = tiny_pef(4, 2, 32773, len, &payload);
|
||||
let err = decode(&file, Mode::Full, &Limits::default()).unwrap_err();
|
||||
assert!(matches!(&err, RawError::Unsupported(m) if m.contains("32773")), "strip of {len} bytes: {err:?}");
|
||||
}
|
||||
}
|
||||
|
||||
/// A 16 x 8 big-endian 12-bit PEF-style file with a 12 x 6 image area at the odd origin (3, 1), the Exif
|
||||
/// `CFAPattern` `pattern` and an AOC maker note with the area tags (and, for 65535, a Huffman table). The samples
|
||||
/// are plain 16-bit words (1), packed 12-bit (32773) or Huffman coded (65535).
|
||||
fn pef_with_pattern(compression: u16, pattern: [u8; 4]) -> Vec<u8> {
|
||||
use craft_tiff::{IfdBuilder, ImageData, TiffWriter, Value};
|
||||
let (w, h) = (16usize, 8usize);
|
||||
let px: Vec<u16> = (0..w * h).map(|i| ((i * 37) % 900 + 100) as u16).collect();
|
||||
let (tb, codes) = test_table();
|
||||
let strip: Vec<u8> = match compression {
|
||||
1 => px.iter().flat_map(|v| v.to_be_bytes()).collect(),
|
||||
32773 => px.chunks(2).flat_map(|p| [(p[0] >> 4) as u8, ((p[0] & 15) << 4 | p[1] >> 8) as u8, p[1] as u8]).collect(),
|
||||
_ => encode(&px, w, &codes),
|
||||
};
|
||||
let build = |table_at: u32| {
|
||||
let mut ifd = IfdBuilder::new();
|
||||
ifd.set(0x0100, Value::Long(vec![w as u32]));
|
||||
ifd.set(0x0101, Value::Long(vec![h as u32]));
|
||||
ifd.set(0x0102, Value::Short(vec![12]));
|
||||
ifd.set(0x0103, Value::Short(vec![compression]));
|
||||
ifd.set(0x0106, Value::Short(vec![32803]));
|
||||
ifd.set(0x010f, Value::Ascii("PENTAX Corporation".into()));
|
||||
ifd.set_image(ImageData::Strips { rows_per_strip: h as u32, strips: vec![strip.clone()] });
|
||||
// AOC + "MM", then the IFD (area origin, area size and, for Huffman data, the table), next-IFD 0, the table
|
||||
let mut note = b"AOC MM".to_vec();
|
||||
let mut entries = vec![(0x0038u16, 3u16, 2u32, [0u8, 3, 0, 1]), (0x0039, 3, 2, [0, 12, 0, 6])];
|
||||
if compression == 65535 {
|
||||
entries.push((0x0220, 7, tb.len() as u32, table_at.to_be_bytes()));
|
||||
}
|
||||
note.extend_from_slice(&(entries.len() as u16).to_be_bytes());
|
||||
for (tag, ty, cnt, val) in entries {
|
||||
note.extend_from_slice(&tag.to_be_bytes());
|
||||
note.extend_from_slice(&ty.to_be_bytes());
|
||||
note.extend_from_slice(&cnt.to_be_bytes());
|
||||
note.extend_from_slice(&val);
|
||||
}
|
||||
note.extend_from_slice(&0u32.to_be_bytes());
|
||||
if compression == 65535 {
|
||||
note.extend_from_slice(&tb);
|
||||
}
|
||||
let mut exif = IfdBuilder::new();
|
||||
exif.set(0xa302, Value::Undefined([&[0, 2, 0, 2][..], &pattern[..]].concat()));
|
||||
exif.set(0x927c, Value::Undefined(note));
|
||||
ifd.set_child(0x8769, exif);
|
||||
TiffWriter::new(ByteOrder::Big, false).write(&[ifd]).unwrap()
|
||||
};
|
||||
let first = build(0);
|
||||
if compression != 65535 {
|
||||
return first;
|
||||
}
|
||||
let at = Tiff::parse(&first).unwrap().exif().unwrap().get(0x927c).unwrap().offset;
|
||||
build((at + 6 + 2 + 3 * 12 + 4) as u32)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_pattern_is_the_exif_cfapattern_from_the_array_origin_for_every_compression() {
|
||||
for compression in [1u16, 32773, 65535] {
|
||||
for (p, name) in [([0u8, 1, 1, 2], "RGGB"), ([2, 1, 1, 0], "BGGR"), ([1, 0, 2, 1], "GRBG"), ([1, 2, 0, 1], "GBRG")] {
|
||||
let img = decode(&pef_with_pattern(compression, p), Mode::Full, &Limits::default()).unwrap();
|
||||
assert_eq!(img.active_area, Rect::new(3, 1, 12, 6), "compression {compression}");
|
||||
assert_eq!(img.cfa, Some(Cfa::bayer_static(name)), "compression {compression}, Exif {name}");
|
||||
}
|
||||
}
|
||||
// no tag: the assumed layout stays
|
||||
let img = decode(&tiny_pef(4, 2, 1, 16, &[0; 16]), Mode::Full, &Limits::default()).unwrap();
|
||||
assert_eq!(img.cfa, Some(Cfa::bayer_static("BGGR")));
|
||||
}
|
||||
|
||||
/// A `w x h` big-endian 14-bit PEF of plain 16-bit words without area tags, whose maker note has only a
|
||||
/// BlackPoint of `black`, with `px(x, y)` as the samples.
|
||||
fn pef_with_black_point(w: usize, h: usize, black: u16, px: impl Fn(usize, usize) -> u16) -> Vec<u8> {
|
||||
use craft_tiff::{IfdBuilder, ImageData, TiffWriter, Value};
|
||||
let strip: Vec<u8> = (0..w * h).flat_map(|i| px(i % w, i / w).to_be_bytes()).collect();
|
||||
let build = |levels_at: u32| {
|
||||
let mut ifd = IfdBuilder::new();
|
||||
ifd.set(0x0100, Value::Long(vec![w as u32]));
|
||||
ifd.set(0x0101, Value::Long(vec![h as u32]));
|
||||
ifd.set(0x0102, Value::Short(vec![14]));
|
||||
ifd.set(0x0103, Value::Short(vec![1]));
|
||||
ifd.set(0x0106, Value::Short(vec![32803]));
|
||||
ifd.set(0x010f, Value::Ascii("PENTAX Corporation".into()));
|
||||
ifd.set_image(ImageData::Strips { rows_per_strip: h as u32, strips: vec![strip.clone()] });
|
||||
// AOC\0 + "MM", one entry (BlackPoint, four shorts stored after the IFD), next-IFD 0, the four levels
|
||||
let mut note = b"AOC\0MM\0\x01".to_vec();
|
||||
note.extend_from_slice(&[0x02, 0x00, 0, 3, 0, 0, 0, 4]);
|
||||
note.extend_from_slice(&levels_at.to_be_bytes());
|
||||
note.extend_from_slice(&0u32.to_be_bytes());
|
||||
(0..4).for_each(|_| note.extend_from_slice(&black.to_be_bytes()));
|
||||
let mut exif = IfdBuilder::new();
|
||||
exif.set(0x927c, Value::Undefined(note));
|
||||
ifd.set_child(0x8769, exif);
|
||||
TiffWriter::new(ByteOrder::Big, false).write(&[ifd]).unwrap()
|
||||
};
|
||||
let at = Tiff::parse(&build(0)).unwrap().exif().unwrap().get(0x927c).unwrap().offset;
|
||||
build((at + 6 + 2 + 12 + 4) as u32)
|
||||
}
|
||||
|
||||
/// The K-3 Mark III / KP / KF layout (storytold/lightcraft#1271): masked columns and rows at the black level, an empty band
|
||||
/// at zero, around an interior less than 20 times as bright as the black level.
|
||||
#[test]
|
||||
fn masked_rows_and_columns_at_the_black_level_are_trimmed_without_crop_tags() {
|
||||
let (w, h) = (512, 384);
|
||||
let file = pef_with_black_point(w, h, 256, |x, y| {
|
||||
if x >= w - 8 {
|
||||
0
|
||||
} else if y >= h - 14 {
|
||||
15
|
||||
} else if x < 24 || x >= w - 16 || y < 28 {
|
||||
256
|
||||
} else {
|
||||
1256 + ((x * 7 + y * 13) % 64) as u16
|
||||
}
|
||||
});
|
||||
let img = decode(&file, Mode::Full, &Limits::default()).unwrap();
|
||||
assert_eq!(img.black.values, vec![256.0; 4]);
|
||||
assert_eq!(img.active_area, Rect::new(24, 28, w - 40, h - 42));
|
||||
}
|
||||
|
||||
/// A night frame (bright centre, dark sky): 20 lines of real content 100 DN above the black level inside the
|
||||
/// masked border stay in the picture, whatever the interior's brightness.
|
||||
#[test]
|
||||
fn dark_real_content_next_to_the_masked_border_is_kept() {
|
||||
let (w, h) = (512, 384);
|
||||
let file = pef_with_black_point(w, h, 256, |x, y| {
|
||||
if x >= w - 8 {
|
||||
0
|
||||
} else if y >= h - 14 {
|
||||
15
|
||||
} else if x < 24 || x >= w - 16 || y < 28 {
|
||||
256 + ((x + y) % 3) as u16
|
||||
} else if x < 44 || x >= w - 36 || y < 48 || y >= h - 34 {
|
||||
356 + ((x * 7 + y * 13) % 16) as u16
|
||||
} else {
|
||||
3256 + ((x * 7 + y * 13) % 64) as u16
|
||||
}
|
||||
});
|
||||
let img = decode(&file, Mode::Full, &Limits::default()).unwrap();
|
||||
assert_eq!(img.active_area, Rect::new(24, 28, w - 40, h - 42));
|
||||
}
|
||||
|
||||
/// The K-70 (and K-1) layout: masked lines 65 DN over BlackPoint 64, just above the tolerance over black, and
|
||||
/// empty rows at the bottom.
|
||||
#[test]
|
||||
fn masked_lines_a_little_above_the_black_level_are_trimmed() {
|
||||
let (w, h) = (320, 288);
|
||||
let file = pef_with_black_point(w, h, 64, |x, y| {
|
||||
if y >= h - 12 {
|
||||
0
|
||||
} else if x < 56 || y < 28 {
|
||||
128 + ((x + y) % 3) as u16
|
||||
} else {
|
||||
2064 + ((x * 7 + y * 13) % 64) as u16
|
||||
}
|
||||
});
|
||||
assert_eq!(decode(&file, Mode::Full, &Limits::default()).unwrap().active_area, Rect::new(56, 28, 264, 248));
|
||||
}
|
||||
|
||||
/// The 645Z layout: 12 saturated columns outside 46 masked ones on the right; they hold no picture either.
|
||||
#[test]
|
||||
fn saturated_lines_outside_masked_ones_are_trimmed() {
|
||||
let (w, h) = (400, 288);
|
||||
let file = pef_with_black_point(w, h, 64, |x, y| {
|
||||
if x >= w - 12 {
|
||||
16316
|
||||
} else if x >= w - 58 {
|
||||
64 + ((x + y) % 3) as u16
|
||||
} else {
|
||||
2064 + ((x * 7 + y * 13) % 64) as u16
|
||||
}
|
||||
});
|
||||
assert_eq!(decode(&file, Mode::Full, &Limits::default()).unwrap().active_area, Rect::new(0, 0, w - 58, h));
|
||||
}
|
||||
|
||||
/// A blown-out edge with the picture right behind it is picture: nothing is trimmed.
|
||||
#[test]
|
||||
fn a_saturated_edge_followed_by_picture_stays() {
|
||||
let (w, h) = (400, 288);
|
||||
let file = pef_with_black_point(w, h, 64, |x, y| if x >= w - 12 { 16316 } else { 2064 + ((x * 7 + y * 13) % 64) as u16 });
|
||||
assert_eq!(decode(&file, Mode::Full, &Limits::default()).unwrap().active_area, Rect::new(0, 0, w, h));
|
||||
}
|
||||
|
||||
/// A frame at the black level everywhere (a lens-cap shot) has no picture to tell masked lines from: it keeps
|
||||
/// its whole area.
|
||||
#[test]
|
||||
fn a_frame_at_the_black_level_keeps_its_whole_area() {
|
||||
let (w, h) = (320, 288);
|
||||
let file = pef_with_black_point(w, h, 64, |x, y| 64 + ((x * 7 + y * 13) % 5) as u16);
|
||||
assert_eq!(decode(&file, Mode::Full, &Limits::default()).unwrap().active_area, Rect::new(0, 0, w, h));
|
||||
}
|
||||
|
||||
/// Strip byte counts that add up past `u64::MAX` (LONG8 `[u64::MAX, 1]`) are an error or a picture, not an overflow,
|
||||
/// for the Huffman-coded and the uncompressed samples.
|
||||
#[test]
|
||||
fn strip_byte_counts_past_u64_max_do_not_overflow() {
|
||||
for compression in [65535u16, 1] {
|
||||
let file = crate::shared::tests::huge_strip_counts(pef_with_pattern(compression, [0, 1, 1, 2]));
|
||||
for mode in [Mode::Header, Mode::Full] {
|
||||
let _ = decode(&file, mode, &Limits::default());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -104,13 +104,13 @@ pub(crate) fn declare_strip_len(file: &mut [u8], len: u32, declared: u32) {
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
pub(crate) mod tests {
|
||||
use super::*;
|
||||
use craft_tiff::tags as t;
|
||||
|
||||
/// `file`, whose image IFD holds one strip, as two strips at the same offset whose StripByteCounts are LONG8 (which
|
||||
/// classic TIFF readers accept) `[u64::MAX, 1]`: their sum is past `u64::MAX`.
|
||||
pub(super) fn huge_strip_counts(mut file: Vec<u8>) -> Vec<u8> {
|
||||
pub(crate) fn huge_strip_counts(mut file: Vec<u8>) -> Vec<u8> {
|
||||
let tiff = Tiff::parse(&file).unwrap();
|
||||
let o = tiff.order;
|
||||
let ifd = tiff.all_ifds().into_iter().find(|i| i.contains(t::STRIP_OFFSETS)).unwrap();
|
||||
|
||||
@@ -0,0 +1,599 @@
|
||||
//! Samsung SRW: the variants that store the sensor samples uncompressed.
|
||||
//!
|
||||
//! Sources: TIFF 6.0 (container: a big- or little-endian TIFF whose IFD0 points at sub-IFDs, one of them the raw
|
||||
//! image), Exif 2.3 (`CFAPattern`, `PixelXDimension`, `PixelYDimension`; maker note `0x927c`), the ExifTool Samsung
|
||||
//! tag-name documentation (`SensorAreas` `0xa010`, `EncryptionKey` `0xa020`, `WB_RGGBLevelsUncorrected` `0xa021`,
|
||||
//! `WB_RGGBLevelsBlack` `0xa028`, `RawData` `0xa048`) and black-box analysis of CC0 samples from raw.pixls.us
|
||||
//! (NX5, NX10, NX11, NX20, NX200, NX210, NX1000, NX1100, EX1, WB2000; every number below was measured on them):
|
||||
//!
|
||||
//! - The raw image is the IFD whose `Compression` is a private value (32769..=32773); its `CFAPattern` (`0x828e`)
|
||||
//! names the 2x2 layout, anchored at the first sample of the stored array. Checked on the files above: the colours
|
||||
//! of the decoded mosaic follow the camera's JPEG best with exactly that layout (`corpus_samsung_srw`).
|
||||
//! - Compression 32769 (EX1, WB2000): one 16-bit little-endian word per sample (the TIFF is big-endian, the words are
|
||||
//! not); the strip holds exactly `width * height * 2` bytes.
|
||||
//! - Compression 32770 with exactly `width * height * 12 / 8` strip bytes (NX5, NX10, NX11, NX20, NX200, NX210, NX1000,
|
||||
//! NX1100): 12-bit samples, two in three bytes, rows without padding. The NX5/NX10/NX11 files read them MSB-first,
|
||||
//! the others LSB-first; the file doesn't say which, the image does: in the wrong order neighbouring same-colour
|
||||
//! samples differ 5 to 11 times more (10 files, every one decided by that margin), see [`lsb_is_smoother`].
|
||||
//! - Every other shape of those and of the other private values (32770 with fewer bytes: NX30, NX300, NX2000,
|
||||
//! EK-GN120; 32772: NX mini, NX3000, NX3300; 32773: NX1, NX500) is a compressed coding: reported as unsupported.
|
||||
//! - The maker note is a plain IFD whose value offsets count from the start of the note, not of the TIFF. Several
|
||||
//! vectors are stored offset by `EncryptionKey`, 11 integers in every file (element `i` of a vector uses key
|
||||
//! `i mod 11`): `WB_RGGBLevelsUncorrected` and `WB_RGGBLevelsBlack` hold `value + key`, `RawData` holds
|
||||
//! `value - key`, found by trying both signs and keeping the one that gives plausible numbers (green levels
|
||||
//! 4096 in all but one file, saturation levels 4095 or 16383, black levels 0 to 3, the dark-column level).
|
||||
//! - `SensorAreas`: the second rectangle (left, top, right, bottom) is the camera JPEG's framing (its size equals the
|
||||
//! JPEG's in every file but WB2000's), used as the active area. Registered against the JPEG its centre agrees to
|
||||
//! within 2 pixels (NX10 family; the NX20 family's JPEG is lens-corrected, edges differ).
|
||||
//! - The Exif picture size is smaller than the framing in one direction only on the WB2000 (16:9 from a 4:3 sensor):
|
||||
//! the JPEG is the centre of the framing (registration: top offset 366 = framing top 22 + (2736 - 2048) / 2).
|
||||
//! - `RawData`: four black levels, then saturation levels (R, G1, G2, B twice). The EX1 saturates at 15881, not at its
|
||||
//! stated 16383 (85786 samples sit on the plateau), so a plateau below the stated level wins.
|
||||
//! - Black: `WB_RGGBLevelsBlack` when it states one (NX5, NX10, NX11: 2 to 3, the right-hand dark columns measure
|
||||
//! 3.5 to 4.5); otherwise the dark rows or columns outside the framing (EX1: eight rows at 2.3); otherwise 0.
|
||||
//! - `WB_RGGBLevelsUncorrected` divided by its mean green is the as-shot gain: the renders of the 12 files match the
|
||||
//! camera JPEG at median ΔE76 2.2 to 7.6 (10.7 on the backlit WB2000 sample, 5.7 after registration).
|
||||
use crate::shared::{black_from_columns, white_from_data};
|
||||
use crate::unpack::{read_u16s, unpack_lsb, unpack_msb};
|
||||
use crate::{BlackLevel, Camera, Cfa, ColorData, Limits, Mode, OpcodeLists, Orientation, RawData, RawError, RawFormat, RawImage, Rect, Result};
|
||||
use craft_tiff::image::chunk_bytes;
|
||||
use craft_tiff::{ByteOrder, Ifd, Tiff, tags as t};
|
||||
use rayon::prelude::*;
|
||||
|
||||
const SENSOR_AREAS: u16 = 0xa010;
|
||||
const ENCRYPTION_KEY: u16 = 0xa020;
|
||||
const WB_UNCORRECTED: u16 = 0xa021;
|
||||
const WB_BLACK: u16 = 0xa028;
|
||||
const RAW_DATA: u16 = 0xa048;
|
||||
|
||||
/// The private `Compression` values of the raw image IFD.
|
||||
const COMPRESSIONS: std::ops::RangeInclusive<u16> = 32769..=32773;
|
||||
|
||||
/// The maker-note IFD (offsets relative to the note's own start).
|
||||
fn maker_note(bytes: &[u8], tiff: &Tiff) -> Option<Ifd> {
|
||||
let e = tiff.exif()?.get(t::MAKER_NOTE)?;
|
||||
let opts = craft_tiff::ParseOptions { max_ifds: 4, max_depth: 1, follow_children: false, ..Default::default() };
|
||||
craft_tiff::parse_ifd_at(bytes, e.offset, tiff.order, e.offset, false, &opts).ok().map(|(i, _)| i)
|
||||
}
|
||||
|
||||
/// A key-offset vector: the entries of `tag` shifted by the maker note's key (`sign` is +1 or -1 times the key).
|
||||
fn keyed(mn: &Ifd, tag: u16, sign: i64) -> Option<Vec<i64>> {
|
||||
let key = mn.u64s(ENCRYPTION_KEY).filter(|k| k.len() == 11)?;
|
||||
let vals = mn.f64s(tag)?;
|
||||
// 32-bit entries: the stored value wraps modulo 2^32 (a negative number stored as a LONG)
|
||||
// (a key value past i64::MAX must not overflow the multiply: only the low 32 bits are kept)
|
||||
Some(vals.iter().enumerate().map(|(i, &v)| i64::from((v as i64).wrapping_add((key[i % 11] as i64).wrapping_mul(sign)) as i32)).collect())
|
||||
}
|
||||
|
||||
/// The raw image's IFD: the one with a private compression value.
|
||||
fn raw_ifd(tiff: &Tiff) -> Option<&Ifd> {
|
||||
tiff.all_ifds().into_iter().find(|i| i.u16(t::COMPRESSION).is_some_and(|c| COMPRESSIONS.contains(&c)) && i.contains(t::IMAGE_WIDTH))
|
||||
}
|
||||
|
||||
/// How the samples of an IFD are stored, when uncompressed.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
enum Coding {
|
||||
/// One little-endian 16-bit word per sample.
|
||||
Words,
|
||||
/// 12-bit samples, two in three bytes: read MSB-first (the NX5, NX10 and NX11 files) or LSB-first (the later
|
||||
/// bodies' files); the file does not say which, the smoother image does ([`lsb_is_smoother`]).
|
||||
Packed12,
|
||||
}
|
||||
|
||||
fn coding(compression: u16, w: usize, h: usize, strip: u64) -> Option<Coding> {
|
||||
let n = (w as u64).checked_mul(h as u64)?;
|
||||
match compression {
|
||||
32769 if strip == n.checked_mul(2)? => Some(Coding::Words),
|
||||
32770 if w.is_multiple_of(2) && strip == n.checked_mul(12)? / 8 => Some(Coding::Packed12),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn decode(bytes: &[u8], mode: Mode, limits: &Limits) -> Result<RawImage> {
|
||||
let tiff = Tiff::parse(bytes)?;
|
||||
let ifd0 = tiff.ifds.first().ok_or_else(|| RawError::Corrupt("SRW without IFDs".into()))?;
|
||||
let raw = raw_ifd(&tiff).ok_or_else(|| RawError::Unsupported("Samsung raw image not found".into()))?;
|
||||
let info = raw.image()?;
|
||||
let (w, h) = (info.width as usize, info.height as usize);
|
||||
let n = w.checked_mul(h).filter(|n| *n > 0 && *n <= limits.samples()).ok_or(RawError::Limit("image too large"))?;
|
||||
let chunks = info.chunks(bytes.len() as u64);
|
||||
let strip = chunks.iter().fold(0u64, |a, c| a.saturating_add(c.len));
|
||||
let Some(coding) = coding(info.compression, w, h, strip).filter(|_| chunks.len() == 1) else {
|
||||
return Err(RawError::Unsupported(format!(
|
||||
"Samsung compressed raw (compression {}, {:.2} stored bits per sample)",
|
||||
info.compression,
|
||||
strip as f64 * 8.0 / n as f64
|
||||
)));
|
||||
};
|
||||
let data = if mode == Mode::Full {
|
||||
let src = chunk_bytes(bytes, &chunks[0]).ok_or_else(|| RawError::Corrupt("SRW strip outside file".into()))?;
|
||||
if src.len() as u64 != strip {
|
||||
return Err(RawError::Corrupt("SRW strip shorter than its rows".into()));
|
||||
}
|
||||
let mut d = vec![0u16; n];
|
||||
match coding {
|
||||
Coding::Words => read_u16s(src, ByteOrder::Little, &mut d),
|
||||
Coding::Packed12 => {
|
||||
let stride = w / 2 * 3;
|
||||
let unpack: fn(&[u8], u32, &mut [u16]) = if lsb_is_smoother(src, w, h) { unpack_lsb } else { unpack_msb };
|
||||
d.par_chunks_mut(w).zip(src.par_chunks(stride)).for_each(|(row, s)| unpack(s, 12, row));
|
||||
}
|
||||
}
|
||||
RawData::U16(d)
|
||||
} else {
|
||||
RawData::U16(Vec::new())
|
||||
};
|
||||
|
||||
let mn = maker_note(bytes, &tiff);
|
||||
// the camera JPEG's framing (left, top, right, bottom), when it lies inside the stored array
|
||||
let active = mn
|
||||
.as_ref()
|
||||
.and_then(|m| m.u64s(SENSOR_AREAS))
|
||||
.filter(|v| v.len() == 8)
|
||||
.map(|v| (v[4] as usize, v[5] as usize, v[6] as usize, v[7] as usize))
|
||||
.filter(|&(l, tp, r, b)| l < r && tp < b && r <= w && b <= h)
|
||||
.map_or(Rect::new(0, 0, w, h), |(l, tp, r, b)| Rect::new(l, tp, r - l, b - tp));
|
||||
let cfa = match (raw.u64s(t::CFA_REPEAT_PATTERN_DIM).as_deref(), raw.bytes(t::CFA_PATTERN_EP)) {
|
||||
(Some([2, 2]), Some(p)) if p.len() == 4 && p.iter().all(|&c| c <= 2) => Cfa { width: 2, height: 2, pattern: p.to_vec() },
|
||||
_ => return Err(RawError::Unsupported("Samsung raw without a 2x2 CFAPattern".into())),
|
||||
};
|
||||
// saturation levels and black levels (R, G1, G2, B), in the file's own words
|
||||
let levels = mn.as_ref().and_then(|m| keyed(m, RAW_DATA, 1)).filter(|v| v.len() == 12);
|
||||
let RawData::U16(samples) = &data else { return Err(RawError::Unsupported("float SRW".into())) };
|
||||
// saturation: the file's level, or lower where the samples pile up (EX1 clips at 15882 of its stated 16383)
|
||||
let stated = levels.as_ref().filter(|v| v[4..].iter().all(|&x| x == v[4]) && (255..=65535).contains(&v[4])).map(|v| v[4] as f32);
|
||||
let bits = stated.map_or((info.bits() as u32).clamp(8, 16), |s| 32 - (s as u32).leading_zeros());
|
||||
let plateau = (!samples.is_empty()).then(|| white_from_data(&active_rows(samples, w, active), bits));
|
||||
let white = match (stated, plateau) {
|
||||
(Some(s), Some(p)) => s.min(p),
|
||||
(Some(s), None) => s,
|
||||
(None, Some(p)) => p,
|
||||
(None, None) => 4095.0,
|
||||
};
|
||||
let tagged_black =
|
||||
mn.as_ref().and_then(|m| keyed(m, WB_BLACK, -1)).filter(|v| v.len() == 4 && v.iter().all(|&x| (0..=u16::MAX as i64).contains(&x)));
|
||||
let black = match tagged_black {
|
||||
Some(v) if v.iter().any(|&x| x > 0) => {
|
||||
// RGGB-ordered levels -> per CFA position at the active area origin
|
||||
let a = cfa.shifted(active.x, active.y);
|
||||
let values = a.pattern.iter().map(|&c| [v[0], (v[1] + v[2]) / 2, v[3]][c as usize] as f32).collect();
|
||||
BlackLevel { repeat_rows: 2, repeat_cols: 2, values, ..Default::default() }
|
||||
}
|
||||
// the file states no black level: measure the masked band around the active area, when there is one
|
||||
_ => masked_black(samples, w, h, active, white),
|
||||
};
|
||||
// as-shot white balance: gains with the greens near 4096 (unity)
|
||||
let wb = mn.as_ref().and_then(|m| keyed(m, WB_UNCORRECTED, -1)).filter(|v| v.len() == 4 && v.iter().all(|&x| x > 0)).map(|v| {
|
||||
let g = (v[1] + v[2]) as f64 / 2.0;
|
||||
[(v[0] as f64 / g) as f32, 1.0, (v[3] as f64 / g) as f32]
|
||||
});
|
||||
let crop = picture_crop(&tiff, active);
|
||||
let img = RawImage {
|
||||
format: RawFormat::Srw,
|
||||
width: w,
|
||||
height: h,
|
||||
cpp: 1,
|
||||
data,
|
||||
cfa: Some(cfa),
|
||||
bits,
|
||||
black,
|
||||
white: vec![white],
|
||||
active_area: active,
|
||||
crop,
|
||||
orientation: Orientation::from_exif(ifd0.u16(t::ORIENTATION).unwrap_or(1)),
|
||||
color: ColorData::default(),
|
||||
wb_multipliers: wb,
|
||||
linearized: false,
|
||||
pixel_aspect: 1.0,
|
||||
opcodes: OpcodeLists::default(),
|
||||
camera: tiff.ifds.first().map(Camera::from_ifd0).unwrap_or_default(),
|
||||
cinema: Default::default(),
|
||||
};
|
||||
img.validate_for(mode)?;
|
||||
Ok(img)
|
||||
}
|
||||
|
||||
/// Which bit order unpacks 12-bit samples of `w` x `h` (in `src`, `w / 2 * 3` bytes per row) into the image: in
|
||||
/// the right order neighbouring samples of one colour differ by little, in the wrong one by about the whole range.
|
||||
/// Compares the summed absolute differences of samples two apart over 64 rows from the middle of the frame.
|
||||
fn lsb_is_smoother(src: &[u8], w: usize, h: usize) -> bool {
|
||||
let stride = w / 2 * 3;
|
||||
let (mut msb, mut lsb) = (0u64, 0u64);
|
||||
let (mut a, mut b) = (vec![0u16; w], vec![0u16; w]);
|
||||
for y in (h / 5..h * 4 / 5).step_by((h * 3 / 5 / 64).max(1)) {
|
||||
let Some(row) = y.checked_mul(stride).and_then(|s| src.get(s..s + stride)) else { break };
|
||||
unpack_msb(row, 12, &mut a);
|
||||
unpack_lsb(row, 12, &mut b);
|
||||
let spread = |v: &[u16]| v.windows(3).map(|p| u64::from(p[0].abs_diff(p[2]))).sum::<u64>();
|
||||
msb += spread(&a);
|
||||
lsb += spread(&b);
|
||||
}
|
||||
lsb < msb
|
||||
}
|
||||
|
||||
/// The default crop: the whole framing, or, when the Exif picture size is the framing cut in one direction only
|
||||
/// (a 16:9 picture from a 4:3 sensor), the centred part of that size.
|
||||
fn picture_crop(tiff: &Tiff, active: Rect) -> Rect {
|
||||
let whole = Rect::new(0, 0, active.width, active.height);
|
||||
let Some(exif) = tiff.exif() else { return whole };
|
||||
let (Some(pw), Some(ph)) = (exif.u32(t::PIXEL_X_DIMENSION), exif.u32(t::PIXEL_Y_DIMENSION)) else { return whole };
|
||||
let (pw, ph) = (pw as usize, ph as usize);
|
||||
if pw == 0 || ph == 0 || pw > active.width || ph > active.height || (pw < active.width && ph < active.height) {
|
||||
return whole;
|
||||
}
|
||||
Rect::new((active.width - pw) / 2, (active.height - ph) / 2, pw, ph)
|
||||
}
|
||||
|
||||
/// Every third row of the active area (the saturation statistics need no more).
|
||||
fn active_rows(d: &[u16], w: usize, a: Rect) -> Vec<u16> {
|
||||
(a.y..a.y + a.height).step_by(3).filter_map(|y| d.get(y * w + a.x..y * w + a.x + a.width)).flatten().copied().collect()
|
||||
}
|
||||
|
||||
/// Black level from rows or columns outside the active area that are dark (mean below 1.5% of `white`): the
|
||||
/// longest run of dark lines on the right, else the left, below, above; at least four lines, the two outermost
|
||||
/// rows and columns of the array left out. Uniform 0 when there is no such run.
|
||||
fn masked_black(d: &[u16], w: usize, h: usize, a: Rect, white: f32) -> BlackLevel {
|
||||
if d.is_empty() {
|
||||
return BlackLevel::uniform(0.0);
|
||||
}
|
||||
let limit = f64::from(white) * 0.015;
|
||||
let col_mean = |x: usize| {
|
||||
(a.y..a.y + a.height).step_by(7).filter_map(|y| d.get(y * w + x)).map(|&v| f64::from(v)).sum::<f64>() / a.height.div_ceil(7) as f64
|
||||
};
|
||||
let row_mean =
|
||||
|y: usize| (a.x..a.x + a.width).step_by(7).filter_map(|x| d.get(y * w + x)).map(|&v| f64::from(v)).sum::<f64>() / a.width.div_ceil(7) as f64;
|
||||
// the longest run of consecutive dark lines within `lines`
|
||||
let longest = |lines: std::ops::Range<usize>, mean: &dyn Fn(usize) -> f64| {
|
||||
let (mut best, mut start) = (0..0, lines.start);
|
||||
for i in lines.clone() {
|
||||
if mean(i) >= limit {
|
||||
start = i + 1;
|
||||
} else if i + 1 - start > best.len() {
|
||||
best = start..i + 1;
|
||||
}
|
||||
}
|
||||
best
|
||||
};
|
||||
let cols = a.y..a.y + a.height;
|
||||
let rows = a.x..a.x + a.width;
|
||||
let right = longest(a.x + a.width..w.saturating_sub(2), &col_mean);
|
||||
let left = longest(2..a.x, &col_mean);
|
||||
let below = longest(a.y + a.height..h.saturating_sub(2), &row_mean);
|
||||
let above = longest(2..a.y, &row_mean);
|
||||
if right.len() >= 4 {
|
||||
black_from_columns(d, w, right, cols, a)
|
||||
} else if left.len() >= 4 {
|
||||
black_from_columns(d, w, left, cols, a)
|
||||
} else if below.len() >= 4 {
|
||||
black_from_columns(d, w, rows, below, a)
|
||||
} else if above.len() >= 4 {
|
||||
black_from_columns(d, w, rows, above, a)
|
||||
} else {
|
||||
BlackLevel::uniform(0.0)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::{decode, probe, probe_info};
|
||||
use craft_tiff::{IfdBuilder, ImageData, TiffWriter, Value};
|
||||
|
||||
const KEY: [u32; 11] = [305, 72, 737, 456, 282, 307, 519, 724, 13, 505, 193];
|
||||
|
||||
/// A maker note in the Samsung layout: a big-endian IFD of LONG vectors, offsets relative to the note's start.
|
||||
fn note(entries: &[(u16, Vec<u32>)]) -> Vec<u8> {
|
||||
let typed: Vec<_> = entries.iter().map(|(tag, v)| (*tag, 4, v.len() as u32, v.iter().flat_map(|x| x.to_be_bytes()).collect())).collect();
|
||||
note_typed(&typed)
|
||||
}
|
||||
|
||||
/// [`note`] with any entry types: `(tag, type, count, big-endian value bytes)`.
|
||||
fn note_typed(entries: &[(u16, u16, u32, Vec<u8>)]) -> Vec<u8> {
|
||||
let mut head = Vec::new();
|
||||
let mut tail = Vec::new();
|
||||
head.extend_from_slice(&(entries.len() as u16).to_be_bytes());
|
||||
let values_at = 2 + 12 * entries.len() + 4;
|
||||
for (tag, kind, count, v) in entries {
|
||||
head.extend_from_slice(&tag.to_be_bytes());
|
||||
head.extend_from_slice(&kind.to_be_bytes());
|
||||
head.extend_from_slice(&count.to_be_bytes());
|
||||
if v.len() <= 4 {
|
||||
head.extend(v.iter().copied().chain(std::iter::repeat(0)).take(4));
|
||||
} else {
|
||||
head.extend_from_slice(&((values_at + tail.len()) as u32).to_be_bytes());
|
||||
tail.extend_from_slice(v);
|
||||
}
|
||||
}
|
||||
head.extend_from_slice(&[0; 4]);
|
||||
head.extend(tail);
|
||||
head
|
||||
}
|
||||
|
||||
/// Stored form of a vector the way the file keeps it: `plain + key` (`sign` 1) or `plain - key` (`sign` -1).
|
||||
fn stored(plain: &[i64], sign: i64) -> Vec<u32> {
|
||||
plain.iter().enumerate().map(|(i, &p)| (p + sign * i64::from(KEY[i % 11])) as u32).collect()
|
||||
}
|
||||
|
||||
fn pack12(v: &[u16]) -> Vec<u8> {
|
||||
v.chunks(2).flat_map(|p| [(p[0] >> 4) as u8, ((p[0] & 15) << 4) as u8 | (p[1] >> 8) as u8, p[1] as u8]).collect()
|
||||
}
|
||||
|
||||
fn le_words(v: &[u16]) -> Vec<u8> {
|
||||
v.iter().flat_map(|x| x.to_le_bytes()).collect()
|
||||
}
|
||||
|
||||
struct Spec {
|
||||
w: u32,
|
||||
h: u32,
|
||||
compression: u16,
|
||||
strip: Vec<u8>,
|
||||
pattern: Option<[u8; 4]>,
|
||||
note: Option<Vec<u8>>,
|
||||
picture: Option<(u32, u32)>,
|
||||
}
|
||||
|
||||
fn srw(s: Spec) -> Vec<u8> {
|
||||
let mut raw = IfdBuilder::new();
|
||||
raw.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![0]));
|
||||
raw.set(t::IMAGE_WIDTH, Value::Long(vec![s.w]));
|
||||
raw.set(t::IMAGE_LENGTH, Value::Long(vec![s.h]));
|
||||
raw.set(t::BITS_PER_SAMPLE, Value::Short(vec![12]));
|
||||
raw.set(t::COMPRESSION, Value::Short(vec![s.compression]));
|
||||
raw.set(t::SAMPLES_PER_PIXEL, Value::Short(vec![1]));
|
||||
if let Some(p) = s.pattern {
|
||||
raw.set(t::CFA_REPEAT_PATTERN_DIM, Value::Short(vec![2, 2]));
|
||||
raw.set(t::CFA_PATTERN_EP, Value::Byte(p.to_vec()));
|
||||
}
|
||||
raw.set_image(ImageData::Strips { rows_per_strip: s.h, strips: vec![s.strip] });
|
||||
let mut ifd0 = IfdBuilder::new();
|
||||
ifd0.set(t::MAKE, Value::Ascii("SAMSUNG".into()));
|
||||
ifd0.set(t::MODEL, Value::Ascii("NX0".into()));
|
||||
let mut exif = IfdBuilder::new();
|
||||
if let Some(n) = s.note {
|
||||
exif.set(t::MAKER_NOTE, Value::Undefined(n));
|
||||
}
|
||||
if let Some((pw, ph)) = s.picture {
|
||||
exif.set(t::PIXEL_X_DIMENSION, Value::Long(vec![pw]));
|
||||
exif.set(t::PIXEL_Y_DIMENSION, Value::Long(vec![ph]));
|
||||
}
|
||||
ifd0.set_child(t::EXIF_IFD, exif);
|
||||
ifd0.add_sub_ifd(raw);
|
||||
TiffWriter::new(craft_tiff::ByteOrder::Big, false).write(&[ifd0]).unwrap()
|
||||
}
|
||||
|
||||
/// Strip byte counts that add up past `u64::MAX` (LONG8 `[u64::MAX, 1]`) are an error or a picture, not an overflow.
|
||||
#[test]
|
||||
fn strip_byte_counts_past_u64_max_do_not_overflow() {
|
||||
let file = crate::shared::tests::huge_strip_counts(srw(Spec {
|
||||
w: 4,
|
||||
h: 2,
|
||||
compression: 32769,
|
||||
strip: vec![0; 16],
|
||||
pattern: None,
|
||||
note: None,
|
||||
picture: None,
|
||||
}));
|
||||
for mode in [Mode::Header, Mode::Full] {
|
||||
let _ = super::decode(&file, mode, &Limits::default());
|
||||
}
|
||||
}
|
||||
|
||||
/// The camera-stated description every test file carries: JPEG framing, white balance, levels.
|
||||
fn full_note(area: [u32; 4], white: i64, black: i64) -> Vec<u8> {
|
||||
let mut levels = vec![0i64; 4];
|
||||
levels.extend([white; 8]);
|
||||
note(&[
|
||||
(SENSOR_AREAS, vec![0, 0, 64, 32, area[0], area[1], area[2], area[3]]),
|
||||
(ENCRYPTION_KEY, KEY.to_vec()),
|
||||
(WB_UNCORRECTED, stored(&[8192, 4096, 4096, 6144], 1)),
|
||||
(WB_BLACK, stored(&[black; 4], 1)),
|
||||
(RAW_DATA, stored(&levels, -1)),
|
||||
])
|
||||
}
|
||||
|
||||
/// A smooth gradient with a little texture, `max` the first value that is not reached.
|
||||
fn ramp(w: u32, h: u32, max: u32) -> Vec<u16> {
|
||||
(0..w * h).map(|i| ((1000 + (i % w) * 7 + (i / w) * 5 + (i * 13) % 9) % max) as u16).collect()
|
||||
}
|
||||
|
||||
fn pack12_lsb(v: &[u16]) -> Vec<u8> {
|
||||
v.chunks(2).flat_map(|p| [p[0] as u8, ((p[0] >> 8) as u8) | ((p[1] & 15) << 4) as u8, (p[1] >> 4) as u8]).collect()
|
||||
}
|
||||
|
||||
/// Known answer: the samples, layout, framing, levels and white balance all come from the file's own words.
|
||||
#[test]
|
||||
fn packed_12_bit_samples_decode_with_the_maker_note() {
|
||||
let v = ramp(64, 32, 4096);
|
||||
let bytes = srw(Spec {
|
||||
w: 64,
|
||||
h: 32,
|
||||
compression: 32770,
|
||||
strip: pack12(&v),
|
||||
pattern: Some([2, 1, 1, 0]),
|
||||
note: Some(full_note([6, 4, 58, 30], 4095, 3)),
|
||||
picture: None,
|
||||
});
|
||||
assert_eq!(probe(&bytes), Some(RawFormat::Srw));
|
||||
let r = decode(&bytes).unwrap();
|
||||
assert_eq!((r.format, r.width, r.height, r.bits), (RawFormat::Srw, 64, 32, 12));
|
||||
assert_eq!(r.data, RawData::U16(v));
|
||||
assert_eq!(r.cfa.as_ref().unwrap().name(), "BGGR");
|
||||
assert_eq!(r.active_area, Rect::new(6, 4, 52, 26));
|
||||
assert_eq!(r.crop, Rect::new(0, 0, 52, 26));
|
||||
assert_eq!((r.black.mean(), r.white.clone()), (3.0, vec![4095.0]));
|
||||
assert_eq!(r.wb_multipliers, Some([2.0, 1.0, 1.5]));
|
||||
// headers only: everything but the samples
|
||||
let i = probe_info(&bytes).unwrap();
|
||||
assert_eq!((i.active_area, i.wb_multipliers, i.bits), (Rect::new(6, 4, 52, 26), Some([2.0, 1.0, 1.5]), 12));
|
||||
}
|
||||
|
||||
/// The later bodies pack the same 12-bit samples LSB-first; which of the two orders a file uses is found from
|
||||
/// the samples (the wrong order makes a noisy image), not from the model.
|
||||
#[test]
|
||||
fn lsb_first_packing_is_found_from_the_samples() {
|
||||
let v = ramp(64, 32, 4096);
|
||||
for (strip, name) in [(pack12(&v), "msb"), (pack12_lsb(&v), "lsb")] {
|
||||
let bytes = srw(Spec { w: 64, h: 32, compression: 32770, strip, pattern: Some([1, 0, 2, 1]), note: None, picture: None });
|
||||
assert_eq!(decode(&bytes).unwrap().data, RawData::U16(v.clone()), "{name}");
|
||||
}
|
||||
}
|
||||
|
||||
/// A picture size narrower than the framing in one direction only (16:9 from a 4:3 sensor) is the framing's
|
||||
/// centre; any other size is ignored.
|
||||
#[test]
|
||||
fn the_exif_picture_size_centres_a_one_direction_crop() {
|
||||
let v = ramp(64, 32, 4096);
|
||||
let file = |picture| {
|
||||
srw(Spec {
|
||||
w: 64,
|
||||
h: 32,
|
||||
compression: 32770,
|
||||
strip: pack12(&v),
|
||||
pattern: Some([2, 1, 1, 0]),
|
||||
note: Some(full_note([6, 4, 58, 30], 4095, 0)),
|
||||
picture,
|
||||
})
|
||||
};
|
||||
// framing 52 x 26
|
||||
assert_eq!(decode(&file(Some((52, 18)))).unwrap().crop, Rect::new(0, 4, 52, 18));
|
||||
assert_eq!(decode(&file(Some((40, 26)))).unwrap().crop, Rect::new(6, 0, 40, 26));
|
||||
let r = decode(&file(Some((52, 18)))).unwrap();
|
||||
assert_eq!(r.info().developed_size(), (52, 18));
|
||||
for other in [None, Some((52, 26)), Some((40, 18)), Some((60, 18)), Some((52, 30)), Some((0, 18))] {
|
||||
assert_eq!(decode(&file(other)).unwrap().crop, Rect::new(0, 0, 52, 26), "{other:?}");
|
||||
}
|
||||
}
|
||||
|
||||
/// 16-bit words are little-endian although the file is big-endian; a saturation plateau below the stated
|
||||
/// level wins; dark rows below the framing give the black level when the file states none.
|
||||
#[test]
|
||||
fn word_samples_decode_and_measure_what_the_file_does_not_say() {
|
||||
let (w, h) = (64u32, 40u32);
|
||||
let mut v: Vec<u16> = (0..w * h).map(|i| 3000 + (i * 91 % 5000) as u16).collect();
|
||||
// saturation: a plateau at 15882 inside the framing (rows 4..28), two rows of picture beyond it, dark rows 30..40
|
||||
for i in 0..150usize {
|
||||
v[(4 + i % 24) * 64 + 8 + i / 24] = 15882;
|
||||
}
|
||||
for x in 0..64 * 10 {
|
||||
v[30 * 64 + x] = 80;
|
||||
}
|
||||
let bytes = srw(Spec {
|
||||
w,
|
||||
h,
|
||||
compression: 32769,
|
||||
strip: le_words(&v),
|
||||
pattern: Some([0, 1, 1, 2]),
|
||||
note: Some(full_note([0, 4, 64, 28], 16383, 0)),
|
||||
picture: None,
|
||||
});
|
||||
let r = decode(&bytes).unwrap();
|
||||
assert_eq!(r.data, RawData::U16(v));
|
||||
assert_eq!(r.bits, 14);
|
||||
assert!((15800.0..=15882.0).contains(&r.white[0]), "{:?}", r.white);
|
||||
assert_eq!(r.black.mean(), 80.0);
|
||||
assert_eq!(r.cfa.unwrap().name(), "RGGB");
|
||||
}
|
||||
|
||||
/// Files whose strip is not exactly the uncompressed size (or that use another private value) are compressed
|
||||
/// codings: reported as such, still recognised as SRW.
|
||||
#[test]
|
||||
fn compressed_codings_are_reported_not_misread() {
|
||||
let plain = |compression, strip: Vec<u8>| {
|
||||
srw(Spec { w: 64, h: 32, compression, strip, pattern: Some([2, 1, 1, 0]), note: Some(full_note([0, 0, 64, 32], 4095, 0)), picture: None })
|
||||
};
|
||||
for bytes in [
|
||||
plain(32770, vec![7; 64 * 32 * 12 / 8 - 3]),
|
||||
plain(32772, vec![7; 4000]),
|
||||
plain(32773, pack12(&ramp(64, 32, 4096))),
|
||||
plain(32769, vec![0; 64 * 32]),
|
||||
] {
|
||||
assert_eq!(probe(&bytes), Some(RawFormat::Srw));
|
||||
let Err(RawError::Unsupported(why)) = decode(&bytes) else { panic!("expected Unsupported") };
|
||||
assert!(why.contains("compressed"), "{why}");
|
||||
assert!(probe_info(&bytes).is_err());
|
||||
}
|
||||
}
|
||||
|
||||
/// Without a maker note the samples still decode: whole array, no white balance, depth from the TIFF tags.
|
||||
#[test]
|
||||
fn a_file_without_a_maker_note_decodes_with_defaults() {
|
||||
let v = ramp(64, 32, 4096);
|
||||
let bytes = srw(Spec { w: 64, h: 32, compression: 32770, strip: pack12(&v), pattern: Some([1, 0, 2, 1]), note: None, picture: None });
|
||||
let r = decode(&bytes).unwrap();
|
||||
assert_eq!(r.data, RawData::U16(v));
|
||||
assert_eq!((r.active_area, r.wb_multipliers, r.black.mean()), (Rect::new(0, 0, 64, 32), None, 0.0));
|
||||
assert_eq!(r.cfa.unwrap().name(), "GRBG");
|
||||
assert_eq!(r.bits, 12);
|
||||
}
|
||||
|
||||
/// A raw IFD that does not say how its mosaic is laid out is not guessed at.
|
||||
#[test]
|
||||
fn a_file_without_a_cfa_pattern_is_unsupported() {
|
||||
let bytes = srw(Spec { w: 64, h: 32, compression: 32770, strip: pack12(&ramp(64, 32, 4096)), pattern: None, note: None, picture: None });
|
||||
assert!(matches!(decode(&bytes), Err(RawError::Unsupported(w)) if w.contains("CFAPattern")));
|
||||
}
|
||||
|
||||
/// An encryption key value past i64::MAX (LONG8) overflowed `-1 * key` in debug builds.
|
||||
#[test]
|
||||
fn a_huge_encryption_key_does_not_panic() {
|
||||
let key: Vec<u8> = std::iter::repeat_n((1u64 << 63).to_be_bytes(), 11).flatten().collect();
|
||||
let note = note_typed(&[(ENCRYPTION_KEY, 16, 11, key), (WB_UNCORRECTED, 4, 4, [0u8; 16].to_vec()), (WB_BLACK, 4, 4, [0u8; 16].to_vec())]);
|
||||
let bytes = srw(Spec {
|
||||
w: 64,
|
||||
h: 32,
|
||||
compression: 32770,
|
||||
strip: pack12(&ramp(64, 32, 4096)),
|
||||
pattern: Some([2, 1, 1, 0]),
|
||||
note: Some(note),
|
||||
picture: None,
|
||||
});
|
||||
let mn = maker_note(&bytes, &Tiff::parse(&bytes).unwrap()).unwrap();
|
||||
assert_eq!(mn.u64s(ENCRYPTION_KEY).map(|k| k[0]), Some(1 << 63), "the key reaches keyed()");
|
||||
assert_eq!(keyed(&mn, WB_BLACK, -1), Some(vec![0; 4]), "(2^63 mod 2^32 is 0)");
|
||||
assert!(decode(&bytes).is_ok());
|
||||
assert!(probe_info(&bytes).is_ok());
|
||||
}
|
||||
|
||||
/// A framing rectangle outside the array, or reversed, is ignored; truncated files fail without panicking.
|
||||
#[test]
|
||||
fn hostile_framing_and_truncation_do_not_panic() {
|
||||
let v = ramp(64, 32, 4096);
|
||||
for area in [[0, 0, 65, 32], [10, 0, 10, 32], [0, 40, 64, 20]] {
|
||||
let bytes = srw(Spec {
|
||||
w: 64,
|
||||
h: 32,
|
||||
compression: 32770,
|
||||
strip: pack12(&v),
|
||||
pattern: Some([2, 1, 1, 0]),
|
||||
note: Some(full_note(area, 4095, 0)),
|
||||
picture: None,
|
||||
});
|
||||
assert_eq!(decode(&bytes).unwrap().active_area, Rect::new(0, 0, 64, 32));
|
||||
}
|
||||
let bytes = srw(Spec {
|
||||
w: 64,
|
||||
h: 32,
|
||||
compression: 32770,
|
||||
strip: pack12(&v),
|
||||
pattern: Some([2, 1, 1, 0]),
|
||||
note: Some(full_note([0, 0, 64, 32], 4095, 0)),
|
||||
picture: None,
|
||||
});
|
||||
for cut in [10, bytes.len() / 2, bytes.len() - 20] {
|
||||
let _ = decode(&bytes[..cut]);
|
||||
let _ = probe_info(&bytes[..cut]);
|
||||
}
|
||||
}
|
||||
|
||||
/// storytold/lightcraft#2561: a small file whose strip declares a large frame is refused before the full decode allocates the frame.
|
||||
#[test]
|
||||
fn a_strip_cut_short_is_refused_before_allocating() {
|
||||
let (w, h) = (3000u32, 2000u32);
|
||||
for (compression, declared) in [(32769, w * h * 2), (32770, w * h / 2 * 3)] {
|
||||
let mut f = srw(Spec { w, h, compression, strip: vec![0u8; 12], pattern: Some([0, 1, 1, 2]), note: None, picture: None });
|
||||
crate::shared::declare_strip_len(&mut f, 12, declared);
|
||||
assert_eq!(probe(&f), Some(crate::RawFormat::Srw));
|
||||
assert!(matches!(decode(&f), Err(RawError::Corrupt(_))), "compression {compression}");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -4,3 +4,4 @@
|
||||
mod dng;
|
||||
#[cfg(feature = "dng")]
|
||||
mod robust;
|
||||
mod vendor_robust;
|
||||
@@ -0,0 +1,90 @@
|
||||
//! Malformed maker raw files must decode to an error, never panic: every truncation and random mutations of one
|
||||
//! small synthetic file per maker layout.
|
||||
|
||||
use craft_tiff::tags as t;
|
||||
use craft_tiff::{ByteOrder, IfdBuilder, ImageData, TiffWriter, Value};
|
||||
use proptest::prelude::*;
|
||||
|
||||
const W: u32 = 32;
|
||||
const H: u32 = 8;
|
||||
|
||||
/// A CFA TIFF the way most makers lay it out: IFD0 (make) + a SubIFD holding the raw strip.
|
||||
fn cfa_tiff(make: &str, compression: u16, bits: u16, order: ByteOrder) -> Vec<u8> {
|
||||
let px = (W * H) as usize;
|
||||
let strip: Vec<u8> = (0..px * 2).map(|i| (i * 37 % 251) as u8).collect();
|
||||
let mut raw = IfdBuilder::new();
|
||||
raw.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![0]));
|
||||
raw.set(t::IMAGE_WIDTH, Value::Long(vec![W]));
|
||||
raw.set(t::IMAGE_LENGTH, Value::Long(vec![H]));
|
||||
raw.set(t::BITS_PER_SAMPLE, Value::Short(vec![bits]));
|
||||
raw.set(t::COMPRESSION, Value::Short(vec![compression]));
|
||||
raw.set(t::PHOTOMETRIC, Value::Short(vec![t::photometric::CFA]));
|
||||
raw.set(t::CFA_REPEAT_PATTERN_DIM, Value::Short(vec![2, 2]));
|
||||
raw.set(t::CFA_PATTERN_EP, Value::Byte(vec![0, 1, 1, 2]));
|
||||
raw.set_image(ImageData::Strips { rows_per_strip: H, strips: vec![strip] });
|
||||
let mut ifd0 = IfdBuilder::new();
|
||||
ifd0.set(t::MAKE, Value::Ascii(make.into()));
|
||||
ifd0.set(t::MODEL, Value::Ascii("TEST".into()));
|
||||
ifd0.add_sub_ifd(raw);
|
||||
TiffWriter::new(order, false).write(&[ifd0]).unwrap()
|
||||
}
|
||||
|
||||
/// A Pentax PEF: the raw image in IFD0, wide enough for the dark-border trim.
|
||||
fn pef(compression: u16) -> Vec<u8> {
|
||||
let (w, h) = (264u32, 4u32);
|
||||
let mut ifd = IfdBuilder::new();
|
||||
ifd.set(t::MAKE, Value::Ascii("PENTAX".into()));
|
||||
ifd.set(t::IMAGE_WIDTH, Value::Long(vec![w]));
|
||||
ifd.set(t::IMAGE_LENGTH, Value::Long(vec![h]));
|
||||
ifd.set(t::BITS_PER_SAMPLE, Value::Short(vec![16]));
|
||||
ifd.set(t::COMPRESSION, Value::Short(vec![compression]));
|
||||
ifd.set(t::PHOTOMETRIC, Value::Short(vec![t::photometric::CFA]));
|
||||
let strip: Vec<u8> = (0..(w * h * 2) as usize).map(|i| if i % 528 < 8 { 0 } else { (i * 7 % 200) as u8 }).collect();
|
||||
ifd.set_image(ImageData::Strips { rows_per_strip: h, strips: vec![strip] });
|
||||
TiffWriter::new(ByteOrder::Big, false).write(&[ifd]).unwrap()
|
||||
}
|
||||
|
||||
fn samples() -> Vec<Vec<u8>> {
|
||||
vec![
|
||||
pef(1),
|
||||
pef(65535),
|
||||
cfa_tiff("SAMSUNG", 1, 12, ByteOrder::Little),
|
||||
cfa_tiff("SAMSUNG", 32769, 14, ByteOrder::Big),
|
||||
cfa_tiff("ACME", 1, 16, ByteOrder::Little),
|
||||
]
|
||||
}
|
||||
|
||||
fn exercise(bytes: &[u8]) {
|
||||
let _ = craft_raw::probe(bytes);
|
||||
let _ = craft_raw::probe_info(bytes);
|
||||
let _ = craft_raw::embedded_preview(bytes);
|
||||
if let Ok(img) = craft_raw::decode(bytes) {
|
||||
assert!(img.validate().is_ok());
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn vendor_samples_truncated_at_every_length() {
|
||||
for s in samples() {
|
||||
exercise(&s);
|
||||
for n in (0..s.len()).step_by(3) {
|
||||
exercise(&s[..n]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
proptest! {
|
||||
#![proptest_config(ProptestConfig { cases: 2000, .. ProptestConfig::default() })]
|
||||
|
||||
#[test]
|
||||
fn mutated_vendor_files_never_panic(kind in any::<prop::sample::Index>(), flips in proptest::collection::vec((any::<usize>(), any::<u8>()), 1..16), cut in any::<usize>()) {
|
||||
let all = samples();
|
||||
let mut data = all[kind.index(all.len())].clone();
|
||||
let n = data.len();
|
||||
for (i, v) in flips {
|
||||
data[i % n] = v;
|
||||
}
|
||||
exercise(&data);
|
||||
exercise(&data[..cut % (n + 1)]);
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user