From a0c8a08020284110d4f7c5fc49c3bc6408b44283 Mon Sep 17 00:00:00 2001 From: RWoldberg <314159+rwoldberg@users.noreply.github.com> Date: Mon, 26 Oct 2026 08:20:01 -0600 Subject: [PATCH 01/10] craft-crw: Canon CRW (CIFF) raw decoder, moved from LightCraft with its clean-room record The clean-room Canon CRW decoder written in LightCraft (crates/raw/src/vendor/crw.rs, crw_tables.rs, branch clean-room-crw-format-decode, revision 70c755e4) becomes the shared crate craft-crw: the CIFF heap container, the entropy coding derived by black-box analysis (frozen code tables for selectors 0-2), sensor black and white levels, as-shot white balance, orientation and the make/model/time records. Public types are the crate's own (Error, Limits, Cfa, Rect, BlackLevel, Header, Image); the capture time is handed on as Unix seconds so no calendar code lives here; LightCraft's shared plateau estimate is copied as saturation_plateau. Algorithm, tables and thresholds are unchanged. Tests: the ported synthetic round-trip, table-structure and hostile-input tests; a proptest mutation sweep (tests/robust.rs); and tests/samples.rs, which pins the eleven CC0 raw.pixls.us sample files (tests/samples.json: URL, SHA-256, licence) to the SHA-256 of the verified decode plus levels, white balance and metadata, so the port is proven byte-identical to the LightCraft decode without redistributing the reference TIFFs. scripts/fetch-crw-samples.py downloads the files outside the checkout; CRAFT_CRW_SAMPLES and CRAFT_CRW_ORACLE point the tests at them. The crate builds for wasm32. The whole derivation record (starting assumptions, plan with the maintainer's decisions, source log, container and payload notes, derivation log, frozen format, verification, colour/levels notes, private-corpus and chart notes, scripts, logs and the frozen table JSON) moves verbatim to crates/crw/clean-room/ with a README that indexes it and maps the LightCraft paths it cites; tools/gen_tables.py regenerates src/tables.rs from the JSON, byte for byte. Repository: .github/workflows/crw.yml (fmt, clippy, tests on Linux, Windows and macOS; wasm check; the sample-file job with cached downloads), README and ATTRIBUTION rows, NOTICE source provenance, docs/clean-room.md (the clean-room standard a format crate follows here), and AGENTS.md expanded with the start-of-session steps, the new-crate checklist, clean-room, assets and test rules; CLAUDE.md points at it. Co-Authored-By: Claude Fable 5.1 --- .github/workflows/crw.yml | 79 + AGENTS.md | 49 +- ATTRIBUTION.md | 3 + CLAUDE.md | 7 +- Cargo.lock | 11 + Cargo.toml | 1 + NOTICE | 12 + README.md | 4 + crates/crw/Cargo.toml | 21 + crates/crw/README.md | 151 ++ .../crw/clean-room/00-starting-assumptions.md | 111 ++ crates/crw/clean-room/01-plan.md | 337 ++++ crates/crw/clean-room/02-container.md | 119 ++ crates/crw/clean-room/03-payload-layout.md | 106 ++ crates/crw/clean-room/04-derivation-log.md | 315 ++++ crates/crw/clean-room/06-crw-format.md | 115 ++ crates/crw/clean-room/07-verification.md | 79 + crates/crw/clean-room/08-color.md | 122 ++ crates/crw/clean-room/09-private-corpus.md | 58 + crates/crw/clean-room/10-chart-calibration.md | 97 + crates/crw/clean-room/README.md | 75 + crates/crw/clean-room/ciff_dump.py | 197 +++ crates/crw/clean-room/logs/.gitignore | 1 + crates/crw/clean-room/logs/chart-captures.tsv | 39 + .../crw/clean-room/logs/p1-payload-public.txt | 195 ++ .../clean-room/logs/p1b-markers-public.txt | 109 ++ .../crw/clean-room/logs/p1c-bounds-public.txt | 11 + crates/crw/clean-room/logs/private-corpus.tsv | 1564 +++++++++++++++++ .../clean-room/logs/structure-d60-public.txt | 73 + .../crw/clean-room/logs/structure-public.txt | 887 ++++++++++ crates/crw/clean-room/logs/table_sel0.json | 702 ++++++++ .../clean-room/logs/table_sel0_d60class.json | 574 ++++++ crates/crw/clean-room/logs/table_sel1.json | 706 ++++++++ crates/crw/clean-room/logs/table_sel2.json | 706 ++++++++ .../clean-room/logs/verification-sweep.txt | 22 + crates/crw/clean-room/p1_payload.py | 184 ++ crates/crw/clean-room/p1b_markers.py | 119 ++ crates/crw/clean-room/p1c_lowbits_bounds.py | 96 + crates/crw/clean-room/p2_oracle_lsb.py | 118 ++ crates/crw/clean-room/sources.md | 134 ++ crates/crw/clean-room/tools/gen_tables.py | 52 + crates/crw/src/lib.rs | 746 ++++++++ crates/crw/src/tables.rs | 128 ++ crates/crw/src/tests.rs | 525 ++++++ crates/crw/tests/robust.rs | 108 ++ crates/crw/tests/samples.json | 140 ++ crates/crw/tests/samples.rs | 229 +++ docs/clean-room.md | 85 + scripts/fetch-crw-samples.py | 93 + 49 files changed, 10409 insertions(+), 6 deletions(-) create mode 100644 .github/workflows/crw.yml create mode 100644 crates/crw/Cargo.toml create mode 100644 crates/crw/README.md create mode 100644 crates/crw/clean-room/00-starting-assumptions.md create mode 100644 crates/crw/clean-room/01-plan.md create mode 100644 crates/crw/clean-room/02-container.md create mode 100644 crates/crw/clean-room/03-payload-layout.md create mode 100644 crates/crw/clean-room/04-derivation-log.md create mode 100644 crates/crw/clean-room/06-crw-format.md create mode 100644 crates/crw/clean-room/07-verification.md create mode 100644 crates/crw/clean-room/08-color.md create mode 100644 crates/crw/clean-room/09-private-corpus.md create mode 100644 crates/crw/clean-room/10-chart-calibration.md create mode 100644 crates/crw/clean-room/README.md create mode 100644 crates/crw/clean-room/ciff_dump.py create mode 100644 crates/crw/clean-room/logs/.gitignore create mode 100644 crates/crw/clean-room/logs/chart-captures.tsv create mode 100644 crates/crw/clean-room/logs/p1-payload-public.txt create mode 100644 crates/crw/clean-room/logs/p1b-markers-public.txt create mode 100644 crates/crw/clean-room/logs/p1c-bounds-public.txt create mode 100644 crates/crw/clean-room/logs/private-corpus.tsv create mode 100644 crates/crw/clean-room/logs/structure-d60-public.txt create mode 100644 crates/crw/clean-room/logs/structure-public.txt create mode 100644 crates/crw/clean-room/logs/table_sel0.json create mode 100644 crates/crw/clean-room/logs/table_sel0_d60class.json create mode 100644 crates/crw/clean-room/logs/table_sel1.json create mode 100644 crates/crw/clean-room/logs/table_sel2.json create mode 100644 crates/crw/clean-room/logs/verification-sweep.txt create mode 100644 crates/crw/clean-room/p1_payload.py create mode 100644 crates/crw/clean-room/p1b_markers.py create mode 100644 crates/crw/clean-room/p1c_lowbits_bounds.py create mode 100644 crates/crw/clean-room/p2_oracle_lsb.py create mode 100644 crates/crw/clean-room/sources.md create mode 100644 crates/crw/clean-room/tools/gen_tables.py create mode 100644 crates/crw/src/lib.rs create mode 100644 crates/crw/src/tables.rs create mode 100644 crates/crw/src/tests.rs create mode 100644 crates/crw/tests/robust.rs create mode 100644 crates/crw/tests/samples.json create mode 100644 crates/crw/tests/samples.rs create mode 100644 docs/clean-room.md create mode 100644 scripts/fetch-crw-samples.py diff --git a/.github/workflows/crw.yml b/.github/workflows/crw.yml new file mode 100644 index 00000000..b55472f2 --- /dev/null +++ b/.github/workflows/crw.yml @@ -0,0 +1,79 @@ +name: Shared CRW + +on: + push: + branches: [main] + paths: ["crates/crw/**", "scripts/fetch-crw-samples.py", "Cargo.toml", "Cargo.lock", ".github/workflows/crw.yml"] + pull_request: + paths: ["crates/crw/**", "scripts/fetch-crw-samples.py", "Cargo.toml", "Cargo.lock", ".github/workflows/crw.yml"] + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: shared-crw-${{ github.ref }} + cancel-in-progress: true + +jobs: + native: + timeout-minutes: 20 + name: CRW lint and tests (${{ matrix.os }}) + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + runs-on: ${{ matrix.os }} + steps: + - uses: actions/checkout@v7 + with: + persist-credentials: false + - uses: dtolnay/rust-toolchain@b725eda525362e7cf34af9a7243cac1e3f01a7ec # stable + with: + components: rustfmt, clippy + - uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3 + with: + key: craft-crw-${{ matrix.os }} + - run: cargo fmt -p craft-crw --check + - run: cargo clippy -p craft-crw --all-targets --locked -- -D warnings + - run: cargo test -p craft-crw --locked + + wasm: + timeout-minutes: 10 + name: CRW WASM + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + with: + persist-credentials: false + - uses: dtolnay/rust-toolchain@b725eda525362e7cf34af9a7243cac1e3f01a7ec # stable + with: + targets: wasm32-unknown-unknown + - uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3 + with: + key: craft-crw-wasm + - run: cargo check -p craft-crw --locked --target wasm32-unknown-unknown + + samples: + timeout-minutes: 20 + name: CRW sample files decode exactly + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + with: + persist-credentials: false + - uses: dtolnay/rust-toolchain@b725eda525362e7cf34af9a7243cac1e3f01a7ec # stable + - uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3 + with: + key: craft-crw-samples + - name: Cache the CC0 sample files + uses: actions/cache@v4 + with: + path: ${{ runner.temp }}/crw-samples + key: crw-samples-${{ hashFiles('crates/crw/tests/samples.json') }} + - name: Fetch the pinned CC0 sample files (raw.pixls.us) + run: python scripts/fetch-crw-samples.py "$RUNNER_TEMP/crw-samples" + - run: cargo test -p craft-crw --release --locked --test samples -- --nocapture + env: + CRAFT_CRW_SAMPLES: ${{ runner.temp }}/crw-samples + CRAFT_CRW_REQUIRE_SAMPLES: "1" diff --git a/AGENTS.md b/AGENTS.md index e5adca3b..455051e3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,15 +1,54 @@ # AGENTS.md -Shared Rust libraries for the Craft Apps. The shared standards live in +Shared Rust libraries for the Craft Apps (LightCraft, PhotoCraft, PdfCraft, VectorCraft, FilmCraft, +DesignCraft, WordCraft and the others). The shared standards live in [craftrules](https://github.com/storytold/craftrules) (`../../craftrules` on a dev machine): read its `AGENTS.md`, then `standards/never-crash.md`, `standards/engineering.md` and `standards/licensing.md`. +## Start every session here + +1. Read this file, then the `README.md` of each crate you touch (its boundaries, origin and tests). +2. A crate that derives or reads a format (`jpeg`, `media`, `crw`, …) follows + [`docs/clean-room.md`](docs/clean-room.md); a crate with a `clean-room/` folder keeps that record + intact (its `README.md` says what is frozen and how a change is reopened). +3. Check `git status`, `git worktree list` and unmerged branches before starting anything new; commit + small and often. + +## Rules + - **Never crash.** Every crate's `lib.rs` starts with `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]`. - No `unsafe` (the workspace forbids it). + No `unsafe` (the workspace forbids it). Input-derived numbers are hostile: `get()` instead of + indexing for offsets that come from a file, checked or saturating arithmetic for lengths and + counts, bounded recursion (depth limits or budgets), allocations capped by a `Limits` the caller + can lower. Every crash fix lands with a regression test that panicked before it. - **New crates** go in `crates/`, named `craft-`, inherit the workspace package fields - and lints, and are listed in `[workspace.dependencies]`. + and lints, and are listed in `[workspace.dependencies]`, in the root `README.md` (one entry per + library) and in a workflow of their own (`.github/workflows/.yml`: fmt, clippy with + `-D warnings`, tests on Linux, Windows and macOS, and a `wasm32-unknown-unknown` check for a + crate that an app's web build can use). - **No app code.** A crate here must be useful to more than one app and must not depend on any app. -- **Time goes through `chrono`** (workspace dependency). Never hand-roll UTC/calendar conversion (days-to-date algorithms, leap-year math); a crate owns only its output format string. The one exception is range extension: past the years chrono can represent, shift the date by whole 400-year Gregorian cycles (146,097 days, after which the calendar repeats), let chrono compute the month and day, and add the cycles back to the year (splitting seconds into hours, minutes and seconds is plain arithmetic). + App policy (colour models, UI, settings, command handling) stays in the apps; the crate's README + states its boundaries and, when the code came from an app, its origin (repository, paths, + revision) and what changed in the move. `NOTICE` → *Source provenance* records the same. +- **Clean-room.** Format code is written only from published specifications, prose documentation, + usable sample files and black-box observation of reference binaries; never from GPL/LGPL/AGPL/CDDL + sources or Adobe material, in any form. The process and the record a crate keeps are in + [`docs/clean-room.md`](docs/clean-room.md). If you have seen something you should not have, stop + and say so. +- **Assets and samples.** Nothing enters the repository without a row in `ATTRIBUTION.md` (path, + author, source, licence) and a licence text where one is required (`licenses/`). Sample media is + never committed: tests fetch pinned, SHA-256-verified, licence-checked files outside the checkout + (`scripts/fetch-*.py`) and skip cleanly when they are absent. No Adobe assets of any kind, including + Adobe's open-licensed fonts; fonts come from craft-fonts. +- **Time goes through `chrono`** (workspace dependency). Never hand-roll UTC/calendar conversion (days-to-date algorithms, leap-year math); a crate owns only its output format string. The one exception is range extension: past the years chrono can represent, shift the date by whole 400-year Gregorian cycles (146,097 days, after which the calendar repeats), let chrono compute the month and day, and add the cycles back to the year (splitting seconds into hours, minutes and seconds is plain arithmetic). A library that reads a timestamp from a file hands it on as Unix seconds and lets the app convert. +- **Tests.** Unit and property tests next to the code; every feature has a test that would fail + without it; decoders have a hostile-input sweep (truncations, byte mutations, proptest) and, for + real files, a digest-pinned sample test. A test that cannot run (no sample files, no GPU) prints + why it skipped and passes, unless a `*_REQUIRE_*` variable (set in CI) turns the skip into a + failure. - **Before merging:** `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, - `cargo test --workspace`. + `cargo test --workspace`. Each crate's workflow runs the same for that crate on every PR. +- **Commits:** one change per commit, only green states, messages that say what and why. Apps + consume a crate by git revision (`craft- = { git = "https://github.com/storytold/craft-libs", rev = "" }`), + so a merged commit is an API contract: note breaking changes in the crate's README. diff --git a/ATTRIBUTION.md b/ATTRIBUTION.md index c6345e85..f3a6e899 100644 --- a/ATTRIBUTION.md +++ b/ATTRIBUTION.md @@ -22,6 +22,9 @@ author, source and license. Library code and original assets are under the repos | `crates/ui/tests/fonts/NotoSansArabic-screens.ttf` (instance at weight 400 and width 100, subset to the screen test strings) | The Noto Project Authors | [google/fonts@7085eb8 `ofl/notosansarabic`](https://github.com/google/fonts/tree/7085eb89a950e85db5b166b7a58d414544b4140c/ofl/notosansarabic), via craft-fonts | OFL-1.1 (`licenses/ui-test-fonts/OFL-NotoSansArabic.txt`) | | `crates/ui/tests/fonts/NotoSansCJKsc-screens.otf` (subset to printable ASCII and the screen test strings) | The Noto Project Authors (Adobe, Google) | [notofonts/noto-cjk@f8d1575 `Sans/OTF/SimplifiedChinese`](https://github.com/notofonts/noto-cjk/tree/f8d157532fbfaeda587e826d4cd5b21a49186f7c/Sans/OTF/SimplifiedChinese), via craft-fonts | OFL-1.1 (`licenses/ui-test-fonts/OFL-NotoSansCJK.txt`) | | `crates/ui/tests/fonts/NotoSansThai-screens.ttf` (instance at weight 400 and width 100, subset to the screen test strings) | The Noto Project Authors | [google/fonts@8b0a1d0 `ofl/notosansthai`](https://github.com/google/fonts/tree/8b0a1d0f5983c89bc2b93f1b5fb55f9e252744b5/ofl/notosansthai), via craft-fonts | OFL-1.1 (`licenses/ui-test-fonts/OFL-NotoSansThai.txt`) | +| `crates/crw/tests/samples.json` (manifest only; the 11 raw files are not bundled, `scripts/fetch-crw-samples.py` downloads them outside the checkout) | The photographers who contributed each file to raw.pixls.us (Canon EOS D30, D60, 10D ×3, 300D ×2, Digital Rebel, PowerShot G1, G5, G6) | [raw.pixls.us](https://raw.pixls.us/), each file's `getfile.php` URL and SHA-256 in the manifest | CC0-1.0 (as published by raw.pixls.us for each file) | +| `crates/crw/src/tables.rs`, `crates/crw/clean-room/logs/table_sel*.json` | craft-crw contributors (derived in LightCraft, 2026-10, by known-plaintext alignment of the sample files' entropy streams; `crates/crw/clean-room/06-crw-format.md`) | this repository | MIT OR Apache-2.0 | +| `crates/crw/clean-room/logs/*.txt`, `*.tsv` (statistics, file names, sizes and SHA-256 of sample files; no image content) | craft-crw contributors (LightCraft derivation record) | this repository | MIT OR Apache-2.0 | Original assets are covered by the repository licence. diff --git a/CLAUDE.md b/CLAUDE.md index 11908a75..2fe86b67 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1 +1,6 @@ -Read `AGENTS.md`. +# CLAUDE.md + +Read [`AGENTS.md`](AGENTS.md): it is the instruction file for every agent working in this repository +(start-of-session steps, never-crash, new-crate checklist, clean-room, assets, time, tests, merge +gates). Format work also follows [`docs/clean-room.md`](docs/clean-room.md). Suite-wide standards: +[craftrules](https://github.com/storytold/craftrules). diff --git a/Cargo.lock b/Cargo.lock index e2f00610..dba7c508 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -775,6 +775,17 @@ dependencies = [ "rayon", ] +[[package]] +name = "craft-crw" +version = "0.1.0" +dependencies = [ + "proptest", + "serde", + "serde_json", + "sha2", + "thiserror 2.0.21", +] + [[package]] name = "craft-docpath" version = "0.1.0" diff --git a/Cargo.toml b/Cargo.toml index a71f9fd6..2de3bcb9 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -25,6 +25,7 @@ kurbo = "0.13" thiserror = "2" craft-atomic = { path = "crates/atomic" } craft-cms = { path = "crates/cms" } +craft-crw = { path = "crates/crw" } craft-docpath = { path = "crates/docpath" } craft-font-catalog = { path = "crates/font-catalog" } craft-font-manager = { path = "crates/font-manager" } diff --git a/NOTICE b/NOTICE index a46d3754..59cb0f0a 100644 --- a/NOTICE +++ b/NOTICE @@ -200,3 +200,15 @@ https://github.com/storytold/craft-launcher, https://github.com/storytold/photoc https://github.com/storytold/filmcraft, https://github.com/storytold/effectcraft. Copyright (c) 2026 ArtCraft Team and the respective application and craft-libs contributors. All adapted source is licensed MIT OR Apache-2.0. + +The Canon CRW decoder in crates/crw (src/lib.rs, src/tables.rs, src/tests.rs, tests/) and its +derivation record (crates/crw/clean-room/) come from: +- LightCraft, crates/raw/src/vendor/crw.rs, crates/raw/src/vendor/crw_tables.rs, + crates/raw/tests/crw_oracle.rs, crates/raw/tests/it/vendor_robust.rs and clean-room/crw/, + branch clean-room-crw-format-decode, revision 70c755e42a65661401142819278d51b0c5977f7f. + https://github.com/storytold/lightcraft + Copyright (c) 2026 ArtCraft Team and the LightCraft contributors. +Its white-level plateau estimate (saturation_plateau) is LightCraft's vendor::white_from_data at the +same revision. The CRW compression was derived clean-room from CC0 sample files and the +maintainer's own photographs by black-box comparison with a reference decoder's command-line +output; no raw-decoder source was consulted (crates/crw/clean-room/sources.md). diff --git a/README.md b/README.md index 421c5aea..ab3ad9a5 100644 --- a/README.md +++ b/README.md @@ -34,6 +34,10 @@ craft- = { git = "https://github.com/storytold/craft-libs", rev = " bool`: the CIFF signature (`II`/`MM`, `HEAPCCDR` at offset 6). +- `header(bytes, &Limits) -> Result`: geometry, bit depth, CFA phase, selector, + make and model, capture time, orientation and white balance, without decoding the samples + (fast: it walks the heap and reads a few records). +- `decode(bytes, &Limits) -> Result`: the header plus the samples and the measured + black and white levels. + +### Output + +`Image { header, samples, black, white }`: + +- `samples`: `width × height` `u16`s, row-major, the whole sensor including the masked borders, + `bits` (10 or 12) deep, black not subtracted. Rows are in sensor order; apply + `header.exif_orientation` (1, 3, 6 or 8) when displaying. +- `header.active_area`: the picture (SensorInfo's inclusive crop); the rest is masked. The default + crop of these bodies is the active area itself. +- `header.cfa`: `Cfa::Rggb` or `Cfa::Gbrg`, the phase at sensor (0, 0); `pattern()` gives the + 2×2 cell as 0 = red, 1 = green, 2 = blue. +- `black`: `BlackLevel { values: [f32; 4] }`, one level per CFA site of the 2×2 cell counted + from the active area's origin (`at(active, x, y)`), measured on the darkest strips of the masked + borders. These bodies sit near 128 (the Digital Rebel sample near 261, the G1 near 29). +- `white`: the saturation plateau when the image has one, else the sensor's clip level when at + least 64 samples reach within 3 % of the full range (the D60, 10D and 300D clip near 4000 at + ISO 100–400), else the full range (4095 or 1023). +- `header.wb_multipliers`: `Some([r, 1.0, b])` from the ColorBalance record on the D60, 10D and + 300D, the slot chosen by the ShotInfo white-balance setting; `None` on the D30 and PowerShots, + whose records the derivation did not decipher. Apps fall back to their own estimate. +- `header.capture_unix`: the TimeStamp record, Unix seconds on the camera's local clock, no UTC + offset. Convert with chrono (`DateTime::from_timestamp`), never by hand. +- `header.make` / `header.model`: the MakeModel record, e.g. `Canon` / `Canon EOS D60`. CRW + files carry no Exif block, so this is where the camera name comes from. + +### Errors and limits + +`Error` is `NotCrw` (not a CIFF raw), `Corrupt(String)` (damaged structure or an undecodable +code), `Unsupported(String)` (an unknown decoder table, or the PowerShot G5) or +`LimitExceeded(&str)`. Every offset read from the file is bounds-checked, the heap walk is bounded +by depth (16) and by a per-file record budget (4,096), and `width × height` is checked against +`Limits::max_samples` (default 2³⁰ on 64-bit targets, 2²⁶ on wasm32; real frames hold at most +7.4 million) before anything is allocated. The decoder holds about three sample-sized `u16` +buffers at its peak. `Limits` is `#[non_exhaustive]`: start from `Limits::default()` and set the +fields you change. + +A stream that ends early is not an error: the remaining samples keep their predictor values (the +frozen stream-end rule; real files end 1–3 samples short). + +### Performance + +About 60 ms for a 6.5 MP EOS file and 70 ms for the 7.4 MP G6 in release mode on a 2024 laptop +(`cargo test -p craft-crw --release --test samples -- --nocapture` prints the times). + +## Tests + +- `cargo test -p craft-crw`: synthetic CIFF files round-tripped through a test-only encoder (the + model's inverse), the frozen tables' structure (prefix-free, all 162 symbols), hostile inputs + (every truncation and byte mutation of a valid file, self-referencing heaps), and a proptest + mutation sweep (`tests/robust.rs`). None needs sample files. +- `tests/samples.rs` pins the eleven CC0 sample files of [`tests/samples.json`](tests/samples.json) + (raw.pixls.us): each decodes to exactly the samples the derivation verified against the + reference, recorded as the SHA-256 of the sample buffer, with its levels, white balance and + metadata. The files (5–11 MB each) are not in the repository: + + ```sh + python scripts/fetch-crw-samples.py ~/crw-samples # downloads and verifies by SHA-256 + CRAFT_CRW_SAMPLES=~/crw-samples cargo test -p craft-crw --release --test samples -- --nocapture + ``` + + Unset, the test prints that it skipped; CI sets `CRAFT_CRW_REQUIRE_SAMPLES=1` and runs it on + Linux (`.github/workflows/crw.yml`; lint and the synthetic tests run on Linux, Windows and + macOS, plus a wasm32 check). +- `CRAFT_CRW_ORACLE=` additionally compares against `.crw.tiff` references next to the + files (the reference decoder's `unprocessed_raw -T` output), the check the derivation used. + Those references are not distributable and stay outside the repository. + +## Origin + +The decoder was written in LightCraft as `crates/raw/src/vendor/crw.rs` and `crw_tables.rs` +(branch `clean-room-crw-format-decode`, revision `70c755e42a65661401142819278d51b0c5977f7f`, +[storytold/lightcraft](https://github.com/storytold/lightcraft)) and moved here with its tests and +its derivation record. The move replaced LightCraft's types with this crate's own (`Error`, +`Limits`, `Cfa`, `Rect`, `BlackLevel`, `Header`, `Image`), exposed the capture time as Unix +seconds instead of converting it in the decoder, and copied LightCraft's shared plateau estimate +(`white_from_data`) as `saturation_plateau`. The algorithm, tables and thresholds are unchanged, +and the sample-file test proves the decode is byte-identical to the verified LightCraft decode. +Colour (camera matrices, profiles, the JPEG-fitted look) stayed in LightCraft. + +## Regenerating the tables + +`src/tables.rs` is generated data. Edit nothing by hand: the source of truth is +`clean-room/logs/table_sel{0,1,2}.json`, and `python crates/crw/clean-room/tools/gen_tables.py` +renders it (byte-identical to the committed file). A change to the tables is a change to the frozen +format and must follow the reopening rule of `clean-room/06-crw-format.md`. + +## Boundaries + +- In scope: the CIFF container records a raw developer needs, the entropy coding, sensor levels, + as-shot white balance, orientation and the records a catalog needs (make, model, time). +- Out of scope: colour (no CRW body carries matrices; apps own their profiles), demosaicing, the + embedded JPEG preview (apps find it with their JPEG scanner; the record is `JpgFromRaw`, 0x2007), + and the full CIFF metadata set (ExifTool's prose documents far more records than the decoder + reads). diff --git a/crates/crw/clean-room/00-starting-assumptions.md b/crates/crw/clean-room/00-starting-assumptions.md new file mode 100644 index 00000000..cfea605f --- /dev/null +++ b/crates/crw/clean-room/00-starting-assumptions.md @@ -0,0 +1,111 @@ +# CRW clean-room derivation — starting assumptions + +Written 2026-10-12, **before** reading any file in this repository (other than the standing +CLAUDE.md instructions loaded into the session), any web page, any specification, or any sample +file. Everything below is recalled background knowledge of the author (Claude Fable 5) and is the +baseline the derivation starts from. **Every item is: unverified, no source consulted.** + +This file is committed once and never edited. Corrections, confirmations and refutations go in +`01-plan.md` and the derivation documents, each citing the experiment or published document that +established them. A reviewer can diff this file against the final documentation to see which +facts were assumed and which were derived. + +Confidence labels are the author's own recollection strength, not evidence: *firm* = would be +surprised if wrong; *hazy* = plausibly misremembered or confused with another format (CR2/CR3, +other makers' raws). + +## A. Container + +- **A1** (firm, unverified, no source consulted): `.CRW` files are in CIFF, the "Camera Image + File Format", a Canon-proprietary heap-based container. It is not TIFF-based and not + Exif/JPEG-based (unlike the later `.CR2`, which is TIFF). +- **A2** (firm, unverified, no source consulted): the file begins with a 2-byte byte-order mark, + `II` for little-endian (as TIFF does); Canon cameras write little-endian. `MM`/big-endian is + allowed by the format but not expected in camera files. +- **A3** (hazy, unverified, no source consulted): a 32-bit integer at offset 2 gives the length + of the file header, i.e. the offset at which the heap data begins; the common value is 26 + (0x1A). +- **A4** (firm, unverified, no source consulted): the header contains the ASCII signature + `HEAPCCDR` (at offset 6, immediately after the header-length field). +- **A5** (firm, unverified, no source consulted): the body of the file is a "heap". A heap's + directory is located via the **last 4 bytes** of the heap region: a 32-bit offset, relative to + the start of the heap, pointing at the directory. Data blocks come first, directory last. +- **A6** (firm, unverified, no source consulted): a directory is a 16-bit record count followed + by that many fixed-size 10-byte records: tag (u16), size (u32), offset (u32), with offsets + relative to the start of the containing heap. There may be trailing bytes after the records. +- **A7** (hazy, unverified, no source consulted): the tag value's high bits encode storage class + and data type, the low bits the tag identity. In particular, one storage class means "value + stored in the heap" (size/offset meaningful) and another means "value stored in the record + itself" (the 8 bytes that would hold size+offset hold the data inline). Recalled masks: + storage class ≈ `0xC000`, data type ≈ `0x3800`, id ≈ low 11 bits — exact masks uncertain. +- **A8** (firm, unverified, no source consulted): some records are themselves heaps + (sub-directories), giving a tree: e.g. an image-properties heap, a camera-object heap, an + Exif-information heap. Parsing must recurse; recursion depth must be bounded against malformed + files. +- **A9** (hazy, unverified, no source consulted): recalled tag numbers — `0x2005` raw image + data; `0x1835` "decoder table" selector for the raw compression; `0x1031` sensor/image + dimensions (including the margins of masked pixels); `0x2007` embedded JPEG + ("JpgFromRaw"-like); `0x080a` make and model as NUL-separated strings; something like + `0x102a` holding shot info (ISO, etc.) and a white-balance record whose tag I do not recall. + All tag numbers here are suspect. +- **A10** (firm, unverified, no source consulted): cameras writing CRW also write a sibling + `.THM` file: a small Exif/JPEG thumbnail carrying the metadata. The CRW itself may embed a + preview JPEG as well. +- **A11** (hazy, unverified, no source consulted): CRW was written by early Canon EOS DSLRs + (D30, D60, 10D, 300D/Digital Rebel) and PowerShot models of the same era (G1–G6, S-series, + Pro1). Later bodies (20D on) switched to CR2. + +## B. Raw data storage and coding + +- **B1** (firm, unverified, no source consulted): the raw payload is a single plane of Bayer + CFA samples, pattern RGGB at the sensor origin, covering the full readout area including + masked/optically-black border pixels, which is slightly larger than the visible image. +- **B2** (hazy, unverified, no source consulted): sample depth is 10 or 12 bits depending on + the model; the EOS D60 is 12-bit. (10-bit is recalled for some PowerShots.) +- **B3** (firm, unverified, no source consulted): the raw data is losslessly compressed with a + Canon-proprietary scheme. It is **not** a JPEG stream: no JFIF/JPEG markers, no + entropy-coded-segment byte stuffing wrapper — a bare bitstream. +- **B4** (firm, unverified, no source consulted): the scheme is Huffman-coded DPCM in the style + of JPEG's AC coefficient coding: the stream is organised in blocks of 64 difference values; + each Huffman code decodes to a (run-of-zero-differences, magnitude-bit-count) pair; a + magnitude of n bits follows as n raw bits with JPEG-style sign extension (a leading 0 bit + means negative, value = read − (2^n − 1)). +- **B5** (hazy, unverified, no source consulted): there is a small fixed set of Huffman table + pairs (two or three), built into the camera/decoder rather than stored in the file; the + "decoder table" record (see A9) holds an index selecting which set. Within a set, one table + codes the first value of each 64-block and a second table codes the remaining 63 — the + first/rest split is the haziest part of this recollection. +- **B6** (hazy, unverified, no source consulted): a "low bits" mode exists: when enabled, the + 2 least-significant bits of every pixel are stored separately, uncompressed and packed 4 per + byte, at the start of the raw data section, and the Huffman stream codes only the high bits; + the final sample is (decoded << 2) | low_bits. Which cameras/flags enable it is unknown to + me; possibly all 10-bit → 12-bit? Uncertain. +- **B7** (hazy, unverified, no source consulted): decoded differences accumulate into samples + via two running predictors, one for even and one for odd columns (i.e. per CFA column + parity). I do not recall where the predictors reset (per row? per row pair? never after + initialisation to a mid-grey such as 512?) or how the first pixels of the image are formed. +- **B8** (hazy, unverified, no source consulted): bits are consumed MSB-first within each byte; + there is no marker-escaping; I vaguely recall some per-camera quirk where a count of spare + bits or a skipped prefix is involved, but cannot state it. +- **B9** (firm, unverified, no source consulted): the visible image is a crop of the decoded + area; a sensor-info record gives the full decoded width/height and the left/top borders of + the visible region. +- **B10** (hazy, unverified, no source consulted): black level is taken from the masked border + pixels (or a stored value); white-balance coefficients are in a metadata record as per-channel + levels or gains. Formats of those records not recalled. +- **B11** (hazy, unverified, no source consulted): EOS D60 visible size is 3072×2048 (6.3 MP); + the decoded raw area is slightly wider/taller (numbers not recalled). File size around 7–8 MB, + consistent with ~12 bits/pixel compressed losslessly below 2:1. +- **B12** (hazy, unverified, no source consulted): some PowerShot-era CRW variants may store + the raw data uncompressed or in a different packing; unsure whether any in-scope body does. + +## C. Process assumptions + +- **C1** (firm, unverified, no source consulted): ExifTool publishes prose documentation of the + CIFF/CRW structure and tag names (web pages, distinct from its Perl source, which is off + limits under the clean-room rules). +- **C2** (hazy, unverified, no source consulted): Canon once published a CIFF specification + document (circa 1997–98, "CIFF Specification on Image Data File", version ~1.0 r4) that + circulates publicly as a PDF. +- **C3** (hazy, unverified, no source consulted): raw.pixls.us hosts CRW samples for several of + the bodies in A11, some CC0; per-file licences vary, so each file must be checked. diff --git a/crates/crw/clean-room/01-plan.md b/crates/crw/clean-room/01-plan.md new file mode 100644 index 00000000..452467f8 --- /dev/null +++ b/crates/crw/clean-room/01-plan.md @@ -0,0 +1,337 @@ +# Canon CRW (CIFF) raw decoding — clean-room derivation plan + +Written 2026-10-12. **Status: awaiting the maintainer's approval. No decoder code and no table +data exist yet; nothing below §5 phase P1 starts until this plan is approved.** + +Companion documents in this directory: + +- [`00-starting-assumptions.md`](00-starting-assumptions.md) — what was assumed before any + source was consulted (committed first, immutable). Items are cited below as A1–A11 (container), + B1–B12 (raw coding), C1–C3 (process). +- [`sources.md`](sources.md) — every URL opened and search run, with what was taken. +- [`ciff_dump.py`](ciff_dump.py) + [`logs/`](logs/) — the planning-phase container survey. + +Rules in force (unchanged from the task): no raw-decoder source is ever opened, fetched, +searched inside or requested (dcraw, LibRaw, rawspeed, rawloader/rawler, dnglab, darktable, +RawTherapee, libopenraw, ExifTool's Perl code — including licence headers, comments and +mirrors); the earlier `crw-decode` branch is not checked out, diffed, logged or read; public +samples are CC0 files from raw.pixls.us only; the maintainer's private files never enter the +repository and no document ever includes their image content (SHA-256 and size only). + +--- + +## 1 What public documentation establishes (sources, kept separate from assumptions) + +### 1.1 Canon, *CIFF Specification on Image Data File*, v1.0 r4, 1997-12-24 + +Archived PDF (SHA-256 `416d4e49ea3f6ad2ba5ccc76b48ca100605f4a6127816c145e6dd5d6c3b51cf4`, +URL in `sources.md`; local copy in gitignored `plan/crw/`). Canon's +own published specification. It establishes, with exact definitions: + +- **Heap file header** (quoted struct): `byteOrder[2]` (`"II"` little-endian / `"MM"` + big-endian), `headerLength` u32 ("distance from the start of the file to the beginning of the + heap-space"), `type[4]` = `"HEAP"`, `subType[4]` (`"JPGM"` for JPEG-embedded; reserved values + include `"CCDR"`, `"TIFP"`, `"ARCH"`), `version` u32 = 0x00010002, two reserved u32s. +- **Heap** = heap space followed by the `offsetTbl`; a u32 `offsetTblOffset` is stored "in the + final position at the end of the heap", as an offset relative to the start of the heap. +- **offsetTbl** = `numRecords` u16 + that many `DC_RECORD_ENTRY {typeCode u16, length u32, + offset u32}`; offset is "offset value from the beginning of the heap"; `numRecords` may be 0. +- **typeCode bit fields**: bits 15–14 storage (`00` kStg_InHeapSpace, `01` kStg_InRecordEntry — + data held in the 8 bytes of the length+offset fields themselves — `10`/`11` reserved); bits + 13–11 dataType (`000` byte, `001` ASCII string, `010` 2-byte, `011` 4-byte, `100` structure, + `101`/`110` heap record, `111` reserved); bits 10–0 IDCode. Masks (quoted): + `kStgFormatMask 0xc000`, `kDataTypeMask 0x3800`, `kIDCodeMask 0x07ff`. +- Heap records (dataType 101/110) are themselves heaps → the tree structure. +- Null (`kTC_Null` 0x0000) and free (`kTC_Free` 0x0001) records; all records 2-byte aligned, + odd lengths padded. +- Common metadata record structs: ImageSpec (imageWidth, imageHeight, pixelAspectRatio f32, + rotationAngle s32, componentBitDepth, colorBitDepth, colorBW), ImageFormat, CapturedTime + (time_t + zone), ModelName ("two null-terminated strings … manufacturer name … model name"), + and the ImageProps/ShootingRecord/MeasuredInfo/Camera/CameraSpecification hierarchy. +- **The spec does not describe the raw image payload at all.** ImageFormat's fileFormat value 2 + is merely "Reserved (for CanonCRW format)". Everything about the raw coding must come from + sample analysis (and an approved oracle), not from documentation. + +### 1.2 ExifTool prose pages (documentation, not Perl source) + +`exiftool.org/canon_raw.html` (format description) and `TagNames/CanonRaw.html` (tag tables): + +- A `.CRW` file is a heap file: 26-byte header with signature `HEAPCCDR` (= type `HEAP` + + subType `CCDR`), CRWVersion 0x00010002; the root block runs from `headerLength` to EOF; a + subdirectory is a complete block inside its parent's ValueData whose block size is the + entry's Size; offsets are relative to the containing block's ValueData start. +- Tag IDs (TagID = dataType | IDCode, i.e. typeCode & 0x3fff): **0x2005 RawData** ("the bulk + of the CRW file"), **0x2007 JpgFromRaw** (embedded JPEG), 0x2008 ThumbnailImage, **0x1835 + DecoderTable** with sub-values named *DecoderTableNumber, CompressedDataOffset, + CompressedDataLength*, **0x1031 SensorInfo**, 0x1810 ImageInfo, 0x1030 WhiteSample (with + *BlackLevels*), 0x10a9 ColorBalance, 0x10ae ColorTemperature, 0x080a CanonRawMakeModel, + 0x102a CanonShotInfo, 0x1834 CanonModelID, 0x300a ImageProps, 0x2807 CameraObject, 0x300b + ExifInformation, 0x0000 NullRecord, 0x0001 FreeBytes, and others (full list in `sources.md`). +- Neither page describes the raw compression, bit depth or decoder tables' contents. + +### 1.3 Wikipedia, "Camera Image File Format" + +Spec identity and dates; the CRW camera list: EOS D30, D60, 10D, 300D; PowerShot Pro1, G1–G6, +S30–S70. (Matches A11.) + +## 2 Corpus + +### 2.1 Public (CC0, raw.pixls.us; downloaded into gitignored `corpus/crw/`, hashes verified) + +| File | Body | Size (bytes) | SHA-256 | +|---|---|---|---| +| `crw-canon-eos-d30.crw` | EOS D30 | 2 993 260 | `4ee482f2b6f9f1a8d370703ea04e9f2abaa49356973a7f56b229af93a78e8a2e` | +| `crw-canon-eos-d60.crw` | EOS D60 | 8 591 234 | `2b98659ecff170af1dfe44f89b493139e75e995b2979686f6ff0847cfb56b97f` | +| `crw-canon-eos-10d-a.crw` | EOS 10D | 7 787 330 | `b0730eb3a67cc2281a32e0b2ee104199246928d954a68b7e5b72366514c73d02` | +| `crw-canon-eos-10d-b.crw` | EOS 10D | 10 822 040 | `e879c94afdc64dabc88b01ce2d02628aa225259303ec71dd61f7fb0322bfea2d` | +| `crw-canon-eos-10d-c.crw` | EOS 10D | 10 583 702 | `58144118854657df5cd2185df40fecf6b665fc6754fef0416c2f222b68b025c6` | +| `crw-canon-eos-300d-a.crw` | EOS 300D | 9 056 348 | `47da16e1668af5500eea63cf42cf2992d9cb3d32180b8d786f17f1e2a57f39f4` | +| `crw-canon-eos-300d-b.crw` | EOS 300D | 8 389 842 | `9e7f62239fe728046af28d0d4a8a722d35d40146a79794a5bda1775306af4383` | +| `crw-canon-eos-digital-rebel.crw` | EOS 300D (US name) | 10 153 786 | `d6a3131bfac390fde01b96882ec378211f04570c3252708b58f0d05b33a9fdfc` | +| `crw-canon-powershot-g1.crw` | PowerShot G1 | 2 300 702 | `f39be5acf14057ce04527fd8d0574e57ec8ed5fed082d7bfaff763075a44ee8a` | +| `crw-canon-powershot-g5.crw` | PowerShot G5 | 4 894 440 | `bb084223d714871b99de4f549eaf00513e30a7080feeead700252c2008060196` | +| `crw-canon-powershot-g6.crw` | PowerShot G6 | 8 052 032 | `244378e1f5660b6cc6cefc93a75ed4928d0b28aa6304ff327b61c748538f4fef` | + +All 11 are listed CC0 by raw.pixls.us (licence per file verified in the site's data on +2026-10-12; `sources.md`). Each download was checked against the site's published SHA-256. + +### 2.2 Private (maintainer's EOS D60 files, `../RawImages/`, local only — never committed, no +image content in any document) + +18 files, `CRW_0476.crw` … `CRW_2219.crw`, 5 500 958 – 8 628 810 bytes. SHA-256 and sizes +recorded in `sources.md`-adjacent planning logs; full hash list: + +| File | Size | SHA-256 | +|---|---|---| +| CRW_0476.crw | 6 906 290 | `7966aeb5abf3abe80868b8cb1606d671112c42f44d877d3ff79305a52558cacc` | +| CRW_0478.crw | 5 708 018 | `6d07ef616f4b1c7c3025e7af315bb7fb4173ddefea45e2e52d5a4d28e9f2851f` | +| CRW_0486.crw | 8 055 488 | `bf0876f5ba55b2bd120f9588b91f9d4caad54ab73315e542114bf738459d8b3b` | +| CRW_0527.crw | 7 158 482 | `2fd42a69caa20e7c07fe022bd73dfe2626d60a3bacc4228073d9627a5dfa03c1` | +| CRW_0683.crw | 7 453 390 | `c4b320a158b1859fa0709050caddfe2bec0d0a566eafeb17df89cc6b64346a75` | +| CRW_0717.crw | 6 633 986 | `0d6b64935d928e2e0d378eb6efe2b46bed9026ed227111fd47b5fe8b911e22ad` | +| CRW_0765.crw | 5 636 168 | `e8da71d85e07d4157fb5753f850b331dfa6451d7382c4c0d4f9568144e6e8bc5` | +| CRW_0822.crw | 6 478 092 | `817acdc28dd1fe2cd0065b44c63722aaa1f800275e0b0b245ad3b3d964df7c9f` | +| CRW_0879.crw | 6 861 152 | `df66c000ac18b872a69b0fdc171245f89869bd0ad415e117c77e6c27ea93c0ca` | +| CRW_1011.crw | 6 822 000 | `9044f07f30064249cb5ea78bd6ba5d19ae3d2da278dbf7f10aa357872fbbd469` | +| CRW_1153.crw | 7 729 790 | `9cd504ed0991c4c6033d72a2676c778b84d198ddfad202437fcb7f014fa15a17` | +| CRW_1277.crw | 6 850 414 | `c4f0ce8c4ce85d3668b3726c46018d296c784b10a2f0f322acba8b1681fe86f4` | +| CRW_1514.crw | 6 531 360 | `9aaa960461f829b218eaa7f50cf49fa3cb80e92baf9f5aabc03038fddbefe06c` | +| CRW_1646.crw | 6 204 492 | `f9f3dbc2f9aed2cdc2ddccd9a9768a1e02501c337258a211ac9fb663ad78ff74` | +| CRW_1740.crw | 5 500 958 | `033f225fcc8bfdb07b013d971c594bda2adfa0a683c133acc22a171ddcdf95a6` | +| CRW_1776.crw | 6 746 924 | `b07921872c8cf7767841945818a6c8d59553415580d7ebe6a08115964ec6de44` | +| CRW_2035.crw | 6 595 632 | `0ffd9fec448a667b7850ea505b5b117f01c07a98aff5e0b7ec9da7fc5909b4e8` | +| CRW_2219.crw | 8 628 810 | `0460e9ba4884c7e0867570cf71be7a6d2d038fdcecf062fde5cc28422602dd07` | + +## 3 Established from sample files during planning (container level only) + +All from `ciff_dump.py` over the 29 files (logs: `logs/structure-public.txt`, +`logs/private/structure-private.txt`). No raw-data decoding was attempted. + +- **All 29 files parse fully** with the documented container model: `II`, headerLength 26, + `HEAP`+`CCDR`, version 0x00010002; every record resolves in bounds; heap tree ≤ 4 deep + (root → ImageProps → CameraObject/ExifInformation/… → CameraSpecification). Confirms + A1–A8 on real files (details per assumption in §4). +- **Root block layout** in every file: RawData (0x2005) first at heap offset 0, then + JpgFromRaw (0x2007, starts with JPEG SOI), then ImageProps (0x300a). +- **SensorInfo (0x1031)**, 17 u16s beginning with its own length (34): D60/10D/300D all + `(34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0)`; D30 + `(34, 2224, 1456, 1, 0, 56, 12, 2215, 1451, 8, 220, 40, 1244, …)`; G1 + `(34, 2144, 1560, 1, 1, 24, 15, 2071, 1550, 2108, 15, 2111, 1550, …)`; G5 2672×1968; G6 + 3160×2344. For the EOS bodies, fields 5–8 as (left, top, right, bottom) give exactly the + visible size: 3143−72+1 = 3072, 2063−16+1 = 2048, matching ImageInfo (0x1810) = 3072×2048. + (Bears on B1/B9/B11; the remaining fields are candidates for black-measurement windows — to + be established.) +- **ImageInfo (0x1810)** u32s `(3072, 2048, 1.0f, rot, 8, 24, 257)` — the 8/24 bit depths + describe the *full-view image* (per the spec's ImageSpec definition), **not** the raw depth; + rot = 270 on the portrait 300D shots. Corrects the hope that raw bit depth is read from here. +- **DecoderTable (0x1835)** = 4 u32s `(N, 0, 514, L)`: + - N ∈ {0, 1, 2} — all three values observed, varying per shot *on the same body* (private + D60 set has 15× N=1, 2× N=0, 1× N=2; public files: D30 1, D60 0, 10D 0/2/2, 300D 0/2, + Rebel 0, G1 1, G5 1, G6 0). Supports B5's "selector" reading; presumably exposure/ISO + dependent — to be established. + - Third u32 is 514 in all 29 files (meaning unknown; 512+2? — to be established). + - **L vs RawData size S**: on every EOS file (D30, D60, 10D, 300D, 23 files), + S − L = ⌈width×height/4⌉ + 2 or 3 bytes (e.g. D60: 6 694 146 − 5 064 559 = 1 629 587; + 3152×2068/4 = 1 629 584). On every PowerShot file (G1, G5, G6), S − L = 2 or 3 bytes. + Strong container-level evidence for B6 (a 2-bits-per-pixel region, pixels packed 4 per + byte, present on the EOS bodies and absent on these PowerShots), **but its placement, + packing order and meaning are still to be derived** — note the tension with B6's + recollection that low bits appear on PowerShots. +- **RawData byte statistics**: overall entropy 7.84–7.99 bits/byte (dense entropy coding); on + the D60-class files the first 2048 bytes are all zero; on D30 the data is high-entropy from + byte 0; on G1/G5 the first ~522–529 bytes are zero-dominated. No JPEG markers at the start; + JPEG SOI appears only in the separate 0x2007 record. +- The three 10D files and the three 300D/Rebel files are internally consistent per body; + the 18 private D60 files are uniform with the public D60 file. + +## 4 Starting assumptions → experiments + +Status: **spec** = established by §1 documents; **samples** = established by §3 observations; +**open** = still unverified — the listed experiment decides it during derivation. + +| # | Status | Correction / experiment, and what counts as confirmed | +|---|---|---| +| A1 CIFF heap container | spec+samples | Confirmed (header `HEAP`/`CCDR`; all 29 parse). | +| A2 `II` first, LE | spec+samples | Confirmed; `MM` legal per spec — decoder must accept or cleanly refuse it; no `MM` sample exists (§6). | +| A3 u32 header length @2 = 26 | spec+samples | Confirmed (26 in all 29; field is authoritative, not the constant). | +| A4 `HEAPCCDR` @6 | spec+samples | Confirmed, with correction: it is two fields, type `HEAP` + subType `CCDR`; other subTypes exist (`JPGM`…) and must be refused as unsupported. | +| A5 directory via trailing u32 | spec+samples | Confirmed (`offsetTblOffset` in the heap's final position, relative to heap start). | +| A6 u16 count + 10-byte entries | spec+samples | Confirmed (typeCode u16, length u32, offset u32 — in that order). | +| A7 tag bit masks | spec+samples | Confirmed exactly: 0xc000 storage (0x0000 in-heap, 0x4000 in-entry, rest reserved), 0x3800 dataType, 0x07ff IDCode. | +| A8 nested heaps, bounded recursion | spec+samples | Confirmed (dataType 5/6 = heap record); observed depth ≤ 4; parser caps at 8. | +| A9 tag numbers | samples | Mostly right: 0x2005 RawData, 0x1835 DecoderTable, 0x1031 SensorInfo, 0x2007 JpgFromRaw, 0x080a make/model, 0x102a ShotInfo all confirmed; ColorBalance is 0x10a9. DecoderTable is 4×u32, not "16 bytes of table data". | +| A10 .THM sibling + embedded JPEG | part | Embedded JpgFromRaw confirmed (SOI seen); ThumbnailFileName records name a `.THM`, but no `.THM` files are present in `../RawImages` — treat the THM as optional; not needed for decoding. | +| A11 camera list | spec-adjacent | Matches Wikipedia + raw.pixls.us. Later `.CRW`-named CHDK dumps are a different format: detect via header and refuse as unsupported. | +| B1 Bayer CFA, RGGB at origin, full readout incl. masked borders | open | Decode must yield exactly SensorInfo's width×height samples; CFA phase from (i) colour of uniform patches vs the embedded JPEG, (ii) per-channel black/white statistics, (iii) oracle's CFA output. Confirmed when all three agree on all 29 files. | +| B2 10- or 12-bit samples | open | From decoded histograms (max code, gaps), clipped-highlight files, and the low-bits arithmetic (wh/4 = 2 bits/pixel implies the Huffman stream codes depth−2 bits on EOS bodies). Confirmed when the white clip point is stable per body and matches the oracle's scale. | +| B3 not JPEG, bare bitstream | samples (part) + open | No JPEG structure observed; fully confirmed when the derived bitstream model consumes the payload exactly on all files. | +| B4 Huffman run/size DPCM in 64-blocks | open | The central derivation (P2, §5): a candidate code model must decode every file such that (i) the stream is consumed to within the final byte, (ii) every sample lands in [0, 2^depth), (iii) rows have natural image statistics (smoothness, CFA correlation). Confirmed only when one model does this for all 29 files and matches the oracle exactly. | +| B5 small fixed set of table pairs selected by file | samples (part) + open | DecoderTableNumber ∈ {0,1,2} observed varying per shot. Open: the tables' content, whether they live in camera or file (the constant `514` field and the RawData prefix are candidates to rule in/out), first-vs-rest table split. Confirmed when one fixed table set per N decodes every file with that N, across bodies. | +| B6 2 LSBs stored separately, packed 4/byte | samples (part) + open | wh/4-byte region established on EOS bodies — but B6 guessed PowerShots; observed on EOS instead, and placement unknown (D60-class zero prefix suggests the region leads, since masked top rows plausibly have zero LSBs — but D30's leading bytes are high-entropy, so order may differ or D30's masked rows are noisy). Experiments: (i) statistical — LSB region should be near-uniform in lit areas and structured in masked areas consistently across files; (ii) decisive — oracle output codes' 2 LSBs must equal the extracted region bit-for-bit with one fixed packing rule. | +| B7 two column-parity predictors | open | Derived in P2 by fitting predictor variants in the Python model; confirmed when interior and first-row/first-column pixels match the oracle on all files, incl. the masked border. | +| B8 MSB-first bits, no byte-stuffing | open | Both orders tried in P2; the wrong one fails the exact-consumption test almost immediately. Confirmed with B4. | +| B9 visible crop from sensor record | samples (part) + open | Crop rectangle (72,16)–(3143,2063) = 3072×2048 established; confirmed as *the* crop when our visible output aligns with the embedded JPEG (geometry check as in arw6) and the oracle. | +| B10 black from masked pixels / record; WB in metadata | open | Candidates observed: 0x1030 WhiteSample (BlackLevels), 0x10a9 ColorBalance, 0x10a8/0x10aa (u16 arrays that look like per-channel levels). Experiments: correlate record values with decoded masked-area means per channel and with the embedded JPEG's white balance; confirmed when black/WB applied to our decode reproduces the JPEG's grey axis on neutral patches. | +| B11 D60 3072×2048 / slightly larger raw | samples | Confirmed: 3152×2068 raw, 3072×2048 visible. | +| B12 uncompressed CRW variants | open | For the in-scope bodies, none observed (all payloads are high-entropy). CHDK `.CRW` dumps are a different format → refused as unsupported, not decoded. Confirmed by header discrimination tests. | +| C1 ExifTool prose docs exist | confirmed | §1.2. | +| C2 Canon CIFF spec circulates | confirmed | §1.1 (34-page archived PDF). | +| C3 raw.pixls.us CRW samples, mixed licences | confirmed | §2.1 — CC0 for 7 bodies; several classic bodies NC-only (§6). | + +## 5 Derivation plan (phases, freeze, verification) + +Modelled on [`docs/arw6-compression.md`](../../docs/arw6-compression.md): published docs + +samples → Python reference model → **freeze** → black-box verification, no tuning after the +freeze. All phase outputs are documents/scripts in `clean-room/crw/`; each derived fact cites +the experiment (script + log) or document that established it. + +- **P0 — container (mostly done during planning).** Finalize the CIFF parsing notes + (`02-container.md`): header, heap walk, record classes, the records we consume (0x2005, + 0x1835, 0x1031, 0x1810, 0x080a, 0x1834, 0x10a9/0x10a8/0x10aa, 0x1030, 0x180e, 0x5029 …), + endianness rules, malformed-file behaviour. Includes hostile-input rules for the eventual + Rust parser (offsets/lengths checked with `get`, bounded recursion, no panics). +- **P1 — payload partition.** Statistics-only scripts (`p1_*.py`): establish the split of + RawData into (candidate) low-bits region, entropy-coded region, and the 2–3 trailing bytes; + the meaning of DecoderTable fields (incl. the constant 514) by cross-file regression; + per-body low-bits presence. Output: `03-payload-layout.md` with per-file accounting tables. + Still no bitstream decoding. +- **P2 — entropy code derivation.** The core. Hypothesis-driven, in a Python reference model + (`model/`): bit order (B8), code alphabet and block structure (B4), table set per + DecoderTableNumber (B5), predictor/DPCM rules (B7), low-bits recombination (B6), bit depth + (B2). Method: propose a code model; the acceptance tests are *internal* — exact stream + consumption on every file, samples within range, natural-image statistics, cross-file + consistency (same tables for same N across bodies) — iterating over the hypothesis space. + If the oracle (§5a) is approved early, black-box probing is allowed to speed this up + (feeding CC0 files and byte-locally modified CC0 files to the oracle binary and observing + output deltas, as arw6 probed with synthetic tiles); every probe is logged in `sources.md` + style in `04-derivation-log.md`. **No decoder source is consulted regardless.** +- **P3 — geometry, black, white balance, colour.** Visible crop (B9) against the embedded + JPEG at pixel alignment; black level from masked regions + candidate records (B10); white + point per body (B2); WB coefficients from 0x10a9/0x10a8 against the JPEG's neutrals; CFA + phase (B1). Output: `05-geometry-colour.md`. +- **P4 — freeze + verification.** Write `06-crw-format.md`: the complete derived format + (container summary, payload layout, tables as data, algorithm in prose + the Python model's + final hash). **Freeze point: that document's commit.** Only then run the approved oracle + over all 11 public files (and, locally, the 18 private files) and compare decoded samples + exactly; record match/mismatch per file in `07-verification.md` with zero tuning. A + mismatch reopens derivation explicitly: the reopening, its cause and the new freeze commit + are recorded — never silently adjusted. +- **P5 — Rust implementation (separate task(s), after the maintainer approves the derived + format).** `crates/raw` CIFF container + `vendor/crw.rs` decoder; never-crash rules + (hostile offsets, bounded recursion, no unwrap/expect/panic, capped allocations); + unconditional synthetic tests (round-trip via a test-only encoder like `arw6_testenc`, + truncated/hostile input) + corpus-gated tests with pinned sums; `cargo xtask corpus` + entries for 2 pinned CC0 files; docs (`docs/crw-parity.md` or a section in + `docs/raw-parity.md`), parity-checklist/gaps/ROADMAP rows per CLAUDE.md. Perf budget: at + 6 MP these files should decode far inside the 1 s/24 MP export budget; measured in P5. + +### 5a Reference ("oracle") for what the decoded samples should be — **maintainer's decision** + +| Candidate | What it gives | Licence / repo-rule implications | +|---|---|---| +| **(1) LibRaw binary, run as a black box** (as `docs/arw6-compression.md` did: built binary only, source never read) | Exact 16-bit pre-processing codes; the gold standard for B4–B8 | The task forbids fetching LibRaw *source*, so building from source is out for me. LibRaw.org publishes pre-built Windows binaries; I am also forbidden to install/run downloaded programs, so the **maintainer** would download and run it (or approve my running it). Output used as data only; nothing enters the repo but comparison sums. LibRaw is LGPL/CDDL — running a binary copies no code; the house precedent treats this as clean. | +| **(2) Adobe DNG Converter** (free Adobe tool, CRW → DNG; mosaic DNG holds the raw codes losslessly) | Exact codes + Adobe's reading of black/white/WB/crop as DNG tags | Adobe software: CLAUDE.md forbids copying Adobe assets and reading app bundles; running their converter on our CC0/own files and reading the output DNG (an open, documented format) copies no Adobe asset. Residual concerns: Adobe's EULA, and that its interpretation (crop, black) is Adobe's, not the camera's. Same install/run constraint: maintainer runs it. | +| **(3) The embedded JpgFromRaw** | Geometry, orientation, CFA phase, WB neutrals, gross tone sanity | None — it is inside our samples. Lossy and processed, so never a pixel-exact reference. | +| **(4) No-outside-reference checks** | Exact stream consumption, range bounds, masked-area statistics, low-bits cross-checks, Python↔Rust equality, round-trip with our own test encoder | None. Necessary but not sufficient: they validate self-consistency, not correctness of interpretation. | + +**Recommendation:** (3) + (4) always; (1) as the primary oracle, obtained as an official +pre-built binary and run by the maintainer (or by me with explicit approval), used only after +the P4 freeze except for approved black-box probing in P2; (2) optionally as a second opinion +where (1) and our model disagree. **Flagged: the choice, and who runs what, is the +maintainer's.** + +## 6 What cannot be established from the available samples + +- **Bodies with no CC0 file** (raw.pixls.us has them only as CC-BY-NC-SA, which the rules + exclude): PowerShot G2, G3, Pro1, Pro70, S30, S40, S45, S50, S60, S70 (and S2 IS/S3 IS-class + CHDK-era bodies). They can at best be decoded speculatively: if their container metadata + (DecoderTable, SensorInfo, payload accounting) matches a verified body's pattern, the decoder + may accept them with an "unverified" note, as arw6 does for the ILCE-7M5; otherwise they + return unsupported. No G2/G3/S-series verification is possible without new samples. +- **Big-endian (`MM`) files**: none exist here; `MM` handling will be implemented from the spec + and tested only synthetically, or refused — decision in P5. +- **Reserved storage classes (0x8000/0xc000) and dataType 7**: unobserved; refused as + unsupported with a regression test. +- **Per-body table-number coverage**: observed N per body — D30 {1}, D60 {0,1,2}, 10D {0,2}, + 300D {0,2}, G1 {1}, G5 {1}, G6 {0}. Numbers a body never writes can't be verified for it; + cross-body consistency of a given N is testable and part of P2's acceptance. +- **Controlled-exposure pairs** (same scene raw+JPEG at known settings, dark frames, clipped + highlights): none; black/white derivation rests on masked areas, metadata records and the + embedded JPEGs, which is weaker than arw6's lossless/compressed pairs. If the maintainer can + shoot dark-frame and clipped D60 exposures, B2/B10 confirmation strengthens materially. +- **Colour calibration** (camera matrices/profiles): out of scope here, as for every body; + LightCraft's own calibration work, never copied. +- **The remaining SensorInfo fields** (e.g. the `24, 224, 40, 1856` tail on EOS bodies) may + resist interpretation without more bodies; unknowns will be documented as unknown. + +## 7 Open questions for the maintainer — **answered 2026-10-12, plan approved** + +Maintainer's decisions (recorded verbatim in intent, numbered as below): + +1. **Oracle:** no Adobe DNG Converter. LibRaw binary as the arw6 work did; the maintainer will + install it when it is needed. +2. **Black-box probing** before the freeze: allowed, arw6-style (every probe logged). +3. **libopenraw wiki page:** stays unread. +4. **Python reference model:** local-only, like arw6's Phase 0 model (not committed). The + committed evidence set stays as in this commit; the private-file name+size+hash table in + §2.2 is fine to keep. +5. **Scope order:** agreed — D60-class first, then D30, then G1/G5/G6. +6. **NC-licensed bodies:** accept "unverified, header-matched" decoding. The maintainer owns + none of those cameras and has no files from them. +7. **Extra captures:** the maintainer still has the D60 (and a D20) and will provide controlled + shots when the derivation needs them. + +Also noted by the maintainer: the earlier `crw-decode` branch has been deleted and no longer +exists in the repository; the do-not-read rule about it is therefore moot but remains honoured +for any surviving copies. + +### Original questions (for the record) + +1. **Oracle** (§5a): approve candidate (1) (official pre-built LibRaw binary, black box), and + who runs it — you, or me after your explicit approval of the specific download? Allow (2) + Adobe DNG Converter as a secondary reference? +2. **Black-box probing before the freeze** (P2): may the oracle be used interactively on CC0 + and synthetic/modified-CC0 inputs during derivation (arw6-style), with every probe logged — + or only after the freeze? +3. **Prose pages from decoder projects**: the libopenraw freedesktop wiki has a "Canon CRW" + page (surfaced in a search, not opened). It is prose documentation, but authored by a GPL + decoder project and may paraphrase its implementation. Default is to leave it unread; + confirm, or allow it as a documentation source logged in `sources.md`? +4. **Commit policy for `clean-room/crw/`**: plan, sources log, scripts and public-sample logs + are committed (this commit); `logs/private/` is gitignored (may hold personal metadata: + owner name, serial, timestamps). The Python reference model (P2): committed, or local-only + like arw6's Phase 0 model? Private-file hashes in §2.2: confirm you're happy with the + file-name+size+hash table being committed. +5. **Scope order**: derive on the D60-class first (richest corpus: 23 files, one sensor + geometry, all three table numbers), then D30, then G1/G5/G6 — agreed? +6. **NC-licensed bodies** (§6): accept "unverified, header-matched" decoding for G2/G3/Pro1/ + S-series, or return unsupported until samples exist? Do you own any of these cameras, or + CRW files from other bodies (10D/300D/G-series), that could serve as further private + samples? +7. **Extra captures**: if you still have the D60, controlled shots (lens cap dark frame; + clipped sky; grey card at two ISOs, each at all quality settings so all DecoderTableNumbers + appear) would directly confirm B2/B5/B10. Worth doing? + +--- + +*Approved 2026-10-12 (§7). P0 documentation and P1 are under way.* diff --git a/crates/crw/clean-room/02-container.md b/crates/crw/clean-room/02-container.md new file mode 100644 index 00000000..5b8af04c --- /dev/null +++ b/crates/crw/clean-room/02-container.md @@ -0,0 +1,119 @@ +# CRW container (CIFF heap file) — derived parsing notes (P0) + +Status: derived from published documentation and the 29-file corpus; every fact cites its +source. Sources: **[SPEC]** = Canon, *CIFF Specification on Image Data File* v1.0 r4 (see +`sources.md`, hash pinned); **[ET]** = ExifTool's prose pages `canon_raw.html` / +`TagNames/CanonRaw.html`; **[OBS]** = observation of the corpus via `ciff_dump.py` +(`logs/structure-public.txt`, `logs/private/structure-private.txt`). Assumption numbers refer +to `00-starting-assumptions.md`. + +## 1 File header (26 bytes) [SPEC §4.4.1, ET; OBS: all 29 files] + +| Offset | Size | Field | Observed | +|---|---|---|---| +| 0 | 2 | byteOrder: `II` little-endian, `MM` big-endian | `II` in all 29 | +| 2 | 4 | headerLength: file start → heap-space start | 26 in all 29 | +| 6 | 4 | type | `HEAP` in all 29 | +| 10 | 4 | subType | `CCDR` in all 29 (spec reserves `JPGM`, `TIFP`, `ARCH`) | +| 14 | 4 | version: hi16 major, lo16 minor | 0x00010002 in all 29 | +| 18 | 8 | reserved1, reserved2 | — | + +Multi-byte integers throughout the file follow byteOrder [SPEC]. Parser rules: refuse unknown +byteOrder; honour headerLength rather than assuming 26; refuse type ≠ `HEAP`; refuse +subType ≠ `CCDR` as unsupported (a CHDK sensor dump or other container — A11/B12). `MM` files +exist nowhere in the corpus (§6 of the plan); the Rust parser will implement both orders from +[SPEC] and test `MM` synthetically. + +## 2 Heap structure [SPEC §4.1–4.4; ET; OBS] + +- The **root block** spans headerLength..EOF. A **block** = ValueData (heap space), then the + offsetTbl, then optional other data, with a u32 `offsetTblOffset` stored in the final 4 bytes + of the block, relative to the block start [SPEC §4.1.2; ET "DirStart at BlockSize−4"]. +- **offsetTbl** = u16 `numRecords` + numRecords × 10-byte `DC_RECORD_ENTRY {typeCode u16, + length u32, offset u32}`; offsets are relative to the containing block's start [SPEC §4.4.2; + ET]. numRecords = 0 is legal [SPEC §4.1.4]. +- **typeCode bit fields** [SPEC §4.4.2.1, masks quoted in §8.2]: + - bits 15–14, mask 0xc000, storage: `00` in heap space (length/offset meaningful), `01` in + record entry (the 8 bytes of length+offset hold the value, zero-padded; treated as length 8 + unless self-delimiting), `10`/`11` reserved → refuse as unsupported. + - bits 13–11, mask 0x3800, dataType: 0 byte, 1 ASCII (NUL-terminated), 2 u16s, 3 u32s/f32s, + 4 structure (byte-aligned mix), 5/6 heap record (a complete sub-block), 7 reserved → + refuse. + - bits 10–0, mask 0x07ff, IDCode. ExifTool's "TagID" convention = typeCode & 0x3fff + (dataType | IDCode); this document uses that convention for tag numbers. +- **Sub-blocks** (dataType 5/6): the entry's length/offset give the sub-block's extent inside + the parent's ValueData; offsets inside it are relative to its own start [SPEC §4.2; ET; + OBS: all nested directories resolve correctly this way on all 29 files]. +- **Special records** [SPEC §4.4.4]: 0x0000 null (reserves an offsetTbl slot; length/offset + zero), 0x0001 free (reserves heap space). Skip both. +- **Alignment**: record lengths are even; odd-length values are padded [SPEC §4.4.5]. +- Records may appear in any order; data generally precedes its directory ("the natural way to + write information" [ET]). + +## 3 Observed file tree (CRW from cameras) [OBS: all 29 files] + +``` +root (block = headerLength..EOF) +├─ 0x2005 RawData struct, at heap offset 0, the bulk of the file +├─ 0x2007 JpgFromRaw struct, a complete JPEG (starts FF D8) +├─ (G1/G5/G6, D30: 0x2008 ThumbnailImage — JPEG) +└─ 0x300a ImageProps heap + ├─ 0x1803 ImageFormat dword, in-entry + ├─ 0x1810 ImageInfo dword[7] — the FULL-VIEW image: w, h, aspect f32, + │ rotation s32, componentBitDepth, colorBitDepth, colorBW + │ (per [SPEC §5.4.2] ImageSpec; NOT the raw bit depth) + ├─ 0x180e TimeStamp dword[3]: time_t, zone, zoneInfo [SPEC CapturedTime] + ├─ 0x0816/0x0817 file names ascii (e.g. CRW_0099.CRW / CRW_0099.THM) + ├─ 0x0805 description ascii + ├─ 0x3003 MeasuredInfo heap (0x1814 MeasuredEV …) + ├─ 0x2804 ImageDescription heap (0x0805 descr, 0x0815 CanonImageType, e.g. + │ "CRW:EOS D60 CMOS RAW IMAGE") + ├─ 0x2807 CameraObject heap + │ ├─ 0x0810 OwnerName ascii + │ ├─ 0x080a CanonRawMakeModel ascii: "Canon\0\0" [SPEC ModelName] + │ └─ 0x3004 CameraSpecification heap (0x180b SerialNumber, 0x101c BaseISO, + │ 0x080b FirmwareVersion …) + └─ 0x300b ExifInformation heap + ├─ 0x102a CanonShotInfo u16 array (first u16 = byte length) + ├─ 0x102d CanonCameraSettings u16 array + ├─ 0x1031 SensorInfo u16[17]: len(34), rawW, rawH, ?, ?, cropL, cropT, + │ cropR, cropB, then 4+ more u16 (black windows? TBD P3) + ├─ 0x1835 DecoderTable u32[4]: (tableNumber, 0, 514, compressedLength) — see + │ 03-payload-layout.md + ├─ 0x10a9 ColorBalance, 0x10a8, 0x10aa u16 arrays (WB candidates, P3) + ├─ 0x1030 WhiteSample (10D/300D; BlackLevels candidate [ET], P3) + ├─ 0x1834 CanonModelID dword, in-entry + └─ 0x1033 CustomFunctions, 0x10c0/0x10c1/0x10c2, 0x0036 … (not needed for decode) +``` + +Per-body constants observed (rawW×rawH; crop rect, inclusive): + +| Body | Raw | Crop (L,T)–(R,B) | Visible | +|---|---|---|---| +| EOS D30 | 2224×1456 | (56,12)–(2215,1451) | 2160×1440 | +| EOS D60 / 10D / 300D | 3152×2068 | (72,16)–(3143,2063) | 3072×2048 | +| PowerShot G1 | 2144×1560 | (24,15)–(2071,1550) | 2048×1536 | +| PowerShot G5 | 2672×1968 | — (fields differ; P3) | 2592×1944 (ImageInfo) | +| PowerShot G6 | 3160×2344 | — (P3) | 3072×2304 (ImageInfo) | + +(G-series SensorInfo tails differ in shape from the EOS ones — e.g. G1's fields 9–12 are +`2108, 15, 2111, 1550`, a second rectangle to the right of the crop: a plausible black/dark +measurement window. To be established in P3.) + +## 4 Records the decoder will consume + +0x2005 RawData (payload), 0x1835 DecoderTable, 0x1031 SensorInfo, 0x1810 ImageInfo (visible +size cross-check + rotation), 0x080a make/model + 0x1834 CanonModelID (dispatch), 0x10a9 / +0x10a8 / 0x10aa / 0x1030 (black & WB, P3), 0x180e TimeStamp, 0x2007 JpgFromRaw (preview + +geometry verification), 0x102a CanonShotInfo (ISO etc. for metadata display). + +## 5 Hostile-input rules for the eventual Rust parser (never-crash) + +All reads through checked `get`-style access; every offset/length validated against the +containing block before use; offsets/lengths are u32 → promote to u64/usize with checked math, +no wrapping; recursion over sub-blocks bounded (observed depth 4; cap at 16 with a clear +"malformed: heap nesting too deep" error); a record table whose 2 + 10·numRecords exceeds +offsetTblOffset→block-end space is malformed; in-entry values are exactly the 8 bytes of +length+offset; unknown storage class, dataType 7, unknown subType, or missing mandatory +records (0x2005, 0x1031, 0x1835) → typed unsupported/malformed errors, never a panic; +allocations sized by header fields capped against the actual file size. diff --git a/crates/crw/clean-room/03-payload-layout.md b/crates/crw/clean-room/03-payload-layout.md new file mode 100644 index 00000000..eafec02b --- /dev/null +++ b/crates/crw/clean-room/03-payload-layout.md @@ -0,0 +1,106 @@ +# CRW RawData payload layout (P1) — established by statistics, no decoding + +Status: derived 2026-10-12 from the 29-file corpus by `p1_payload.py`, `p1b_markers.py`, +`p1c_lowbits_bounds.py` (logs: `logs/p1-*.txt`, `logs/p1b-*.txt`, `logs/p1c-*.txt`, private +counterparts under `logs/private/`). **The entropy-coded bitstream itself has not been decoded;** +everything here comes from sizes, offsets, byte histograms and ff-digram statistics. +Assumption IDs refer to `00-starting-assumptions.md`; corrections are stated explicitly. + +## 1 The three layouts + +With S = RawData record size, (N, 0, 514, L) = the DecoderTable u32s, w×h = SensorInfo raw +dimensions, Q = ⌈w·h/4⌉ (one byte per four pixels — the "low-bits" region candidate): + +| Class (bodies) | RawData layout | Evidence | +|---|---|---| +| EOS: D30, D60, 10D, 300D (23 files) | `[low bits, Q][514 zero bytes][stuffed entropy, L−514−t][tail t]` | S−L = Q+2 or Q+3 on all 23; the section after Q starts with exactly 514 zero bytes in all 23; the entropy part contains **zero** non-stuffed ff (every 0xff is followed by 0x00) in all 23 | +| PowerShot 12-bit: G5, G6 | `[514 zero bytes][low bits, Q][stuffed entropy][ff d9 (+00)]` | S−L = 2; first 514 bytes zero; **all** non-stuffed ff (5002 of 5002 on G5, 7928 of 7928 on G6) fall inside [514, 514+Q); the rest of the stream is perfectly stuffed; payload ends `ff d9` | +| PowerShot 10-bit: G1 | `[514 zero bytes][stuffed entropy][ff d9 + 00]` | S−L = 3; first 514 bytes zero; zero non-stuffed ff anywhere; payload ends `ff d9 00` | + +Private corpus: all 18 D60 files match the EOS row exactly (`logs/private/p1c-bounds-private.txt`). + +## 2 DecoderTable (0x1835) semantics + +`(N, 0, 514, L)` where: + +- **N = decoder table selector**, 0–2, varies per shot on the same body (B5 supported; all + three values exercised by the corpus). Presumably chosen by the camera per image; the + mapping (ISO? content?) is not yet established and is not needed for decoding. +- Second u32: 0 in all 29 files. +- **514 = offset to the entropy data within the compressed section** (ExifTool names the field + *CompressedDataOffset*). Behaviourally: the compressed section begins with exactly 514 zero + bytes in all 29 files, EOS and PowerShot alike — a reserved/padding block the decoder skips. + (On G5/G6 the low-bits region sits between this zero block and the entropy data.) +- **L = CompressedDataLength**: the byte length of the compressed section — from the end of + the low-bits prefix on EOS, from 0 on PowerShots — excluding the small tail. + +## 3 Tails + +- EOS: 2 bytes `00 00` (D30, and files where S−L−Q = 2) or 3 bytes `ff 00 00` (D60/10D/Rebel + files with S−L−Q = 3). The `ff 00` there may be a final stuffed 0xff of the entropy stream + sitting just outside L — to be settled in P2 when the stream end is decodable. +- PowerShots: `ff d9` (the JPEG EOI marker used as a terminator) plus one `00` pad when needed + to keep the record length even (G1: S−L = 3 = `ff d9 00`; G5/G6: S−L = 2 = `ff d9`). + +## 4 The low-bits regions (still a hypothesis as to *content*, region *extent* established) + +- Extent Q = w·h/4 exactly, in all 25 files that have the region (B6's arithmetic holds). +- D60-class prefix: reshaped at w/4 = 788 bytes/row, the first **7 rows are exactly zero** + (5516 = 7×788 bytes) in the public D60 file, and the leading zero run is row-aligned — + strong evidence the region is row-major at 2 bits/pixel (`logs/p1-payload-public.txt`). + Below the crop-top band the bytes are near-uniform (entropy ≈ 7.99), as LSB noise should be. +- D30's region is high-entropy from byte 0 and ends in a run of `ff` bytes (saturated LSBs?); + no zero rows — the D30's masked rows evidently carry noise. Placement for D30 is supported + by the size arithmetic and by its entropy part being perfectly stuffed (0 non-stuffed ff). +- **Not yet established:** the bit packing order within each byte, and that the bits really + are the two LSBs of the final samples (vs. e.g. MSBs). Decisive test planned (P2, with the + oracle): the oracle's output codes' 2 LSBs must equal this region bit-for-bit under one + fixed packing rule, for every file. + +## 5 Corrections to the starting assumptions + +- **B3 (bare bitstream, "no JPEG wrapper")** — half right: there are no JPEG markers or + segment structure, but the entropy data **is byte-stuffed exactly like JPEG entropy data** + (0xff is always followed by 0x00; 2 576–28 275 stuffed ff bytes per file, zero exceptions + across 29 files), and the PowerShot payloads even end with a JPEG EOI. **B8's "no + byte-stuffing" is refuted.** The decoder must unstuff `ff 00 → ff`. +- **B6 (low bits)** — the region exists, but the recollection had its distribution backwards: + it is present on all EOS bodies (even the 10-bit-era D30 — see open question below) and on + the later 12-bit PowerShots (G5/G6), absent on the 10-bit G1; and its position differs by + class (prefix on EOS, after the 514-byte zero block on PowerShots). +- **B2 (bit depth)** — if the low-bits hypothesis holds, G1 is the only "no extra bits" body + here; whether EOS bodies are 10+2 or the region means something else is a P2 question. (The + D30 having the region argues the split is not simply "12-bit bodies".) +- **A9 DecoderTable** — layout now pinned: (selector, 0, entropy-offset 514, compressed length). + +## 6 Per-file accounting (public corpus) + +| File | w×h | S | N | L | S−L | Q | S−L−Q | zero run @comp | +|---|---|---|---|---|---|---|---|---| +| d30 | 2224×1456 | 2 824 648 | 1 | 2 015 110 | 809 538 | 809 536 | 2 | 514 | +| d60 | 3152×2068 | 6 694 146 | 0 | 5 064 559 | 1 629 587 | 1 629 584 | 3 | 514 | +| 10d-a | 3152×2068 | 7 134 454 | 0 | 5 504 868 | 1 629 586 | 1 629 584 | 2 | 514 | +| 10d-b | 3152×2068 | 7 244 858 | 2 | 5 615 272 | 1 629 586 | 1 629 584 | 2 | 514 | +| 10d-c | 3152×2068 | 7 455 524 | 2 | 5 825 938 | 1 629 586 | 1 629 584 | 2 | 514 | +| 300d-a | 3152×2068 | 6 929 638 | 0 | 5 300 052 | 1 629 586 | 1 629 584 | 2 | 514 | +| 300d-b | 3152×2068 | 7 206 250 | 2 | 5 576 664 | 1 629 586 | 1 629 584 | 2 | 514 | +| rebel | 3152×2068 | 6 406 164 | 0 | 4 776 577 | 1 629 587 | 1 629 584 | 3 | 514 | +| g1 | 2144×1560 | 2 257 354 | 1 | 2 257 351 | 3 | — | — | 514 | +| g5 | 2672×1968 | 4 861 806 | 1 | 4 861 804 | 2 | 1 314 624 (inside L) | — | 514 | +| g6 | 3160×2344 | 7 313 156 | 0 | 7 313 154 | 2 | 1 851 760 (inside L) | — | 514 | + +(S here is the RawData record size, not the file size. Private D60 table: +`logs/private/p1-payload-private.txt`.) + +## 7 Open for P2 (entropy derivation) + +1. The entropy code itself (B4): alphabet, tables per selector N, block structure, predictors + (B7), bit order within bytes (rest of B8), bit depth per body (B2). +2. Low-bits packing order and recombination (B6) — decisive oracle test in §4. +3. Why 514 (= 512 + 2?) — cosmetic; decoder just skips to the offset given by the record. +4. The D60-class "7 zero rows" and D30's trailing `ff` run in the low-bits region — expect an + explanation to fall out of the geometry work (P3). +5. EOS tail `ff 00 00` vs `00 00` — settle when the stream end is decodable. + +P2 requires the approved oracle (LibRaw pre-built binary, run black-box; plan §5a decision 1–2) +for probing and for the low-bits test. **Requested from the maintainer before P2 starts.** diff --git a/crates/crw/clean-room/04-derivation-log.md b/crates/crw/clean-room/04-derivation-log.md new file mode 100644 index 00000000..295b9e7a --- /dev/null +++ b/crates/crw/clean-room/04-derivation-log.md @@ -0,0 +1,315 @@ +# CRW entropy-coding derivation log (P2) + +Chronological record of experiments and oracle probes, arw6-style. The oracle is LibRaw +0.22.2, official pre-built Windows binaries installed by the maintainer in +a local `LibRaw-0.22.2\bin` folder (2026-10-12); **only `bin\unprocessed_raw.exe` (plus +`libraw.dll`) is executed, as a black box. The LibRaw source tree that ships alongside those +binaries is never opened, read or searched** — all structure below comes from feeding inputs +to the binary and observing outputs. The Python reference model lives in the gitignored +`plan/crw/model/` (maintainer decision #4); findings and probe campaigns are recorded here. + +## 2026-10-12 — oracle baseline runs + +`unprocessed_raw.exe -T ` over 7 public CC0 samples (D30, D60, 10D-a, 300D-a, G1, G5, +G6), each copied to a scratch directory first. Output: 16-bit TIFF of the full raw frame. +Oracle frame sizes equal SensorInfo raw sizes on all 7. Oracle "margins" (active area) differ +from SensorInfo's crop rectangle (e.g. D60: LibRaw top=12/left=64 vs SensorInfo (72,16)) — +LibRaw's own choice, noted, not adopted. + +## 2026-10-12 — low-bits region: CONFIRMED (B6 settled) + +`p2_oracle_lsb.py` (committed) compares the oracle's pixel LSBs against the payload's +candidate low-bits regions, both packing orders: + +| File | Region [0:Q) LSB-first packing | Notes | +|---|---|---| +| D60, 10D-a, 300D-a, D30 | **100.000000 %** (1 629 584/1 629 584 … 809 536/809 536 bytes) | EOS prefix confirmed | +| G6 | **100.000000 %** (1 851 760/1 851 760) | at [0:Q), **not** [514:514+Q) — P1's reading of the G6 layout corrected: G6 low bits precede the 514 block, same as EOS | +| G5 | 47.34 % at [0:Q), 0.39 % at [514:514+Q) | anomaly, parked (scope decision #5: D60-class first) | +| G1 | no region (0.38 % everywhere) | 10-bit body, no low bits ✓ | + +**Packing: LSB-first** — byte b holds pixels 4i..4i+3 with pixel 4i in bits 1:0 and pixel +4i+3 in bits 7:6, row-major over the full raw frame. MSB-first packing scores ~6 %. + +Oracle code ranges: D60 max 4000, 10D max 4003, 300D max 4008, D30 max 3206, G6 max 4095 +(12-bit class); G1 max 936 (10-bit). G5/G6 min ≈ 101/102 (no black subtraction in the oracle). + +Corrections this forces on `03-payload-layout.md`: G6 (and presumably the G5 pattern, once +understood) is `[low bits Q][514-byte block][stuffed entropy][ff d9]`; the G6 "514 zeros at +offset 0" measured in P1 were the low bits of its first 2056 pixels (top masked row), and the +block after the low bits is *not* all-zero on G6. The EOS picture is unchanged. + +## 2026-10-12 — entropy stream: structure findings from known-plaintext analysis + +Using the D60 public file and its oracle decode (scripts `plan/crw/model/align.py`, +`sweep.py`, local): + +- Budget check: unstuffed entropy = 5 049 856 bytes = 6.198 bits/pixel. Under the + two-predictor diff model (B7) E[category] = 3.23 and H(category) ≈ 2.9 bits — model + "code(k) + k magnitude bits" fits the budget almost exactly. Single-predictor E[k] = 5.9 + does not fit. **Two predictors by column parity supported.** +- **Bit order is LSB-first within each byte** (after ff00 unstuffing). Hand-verified at two + anchors: stream start decodes as code `100` (stream order) + 10 magnitude bits = 511 = + JPEG-style magnitude of −512 (first pixel 0 against predictor 512); the first token of row 7 + (byte-aligned at bit 20776) decodes as a 2-bit code + 6 magnitude bits = +55 against + predictor 0 carried from row 6. B8's MSB-first recollection is **refuted**; so is its + "no byte-stuffing" (every 0xff in the entropy data is followed by 0x00, which the decoder + must strip — `03-payload-layout.md`). +- Predictors persist across rows (row 7's first diff is +55 against the row-6 value 0), i.e. + no per-row reset; image start uses 512/512 (10-bit "hi" domain: pixel = hi·4 + lowbits). +- Zero diffs are **run-coded** (a plain per-value zero code is inconsistent with the bits in + the 7 all-zero top rows, which compress to a strictly periodic 7-bit token). +- JPEG-style magnitude convention (negative d stored as d + 2^k − 1) consistent at both anchors. + +## 2026-10-12 — black-box probe campaign 1 (D60 body, patch-in-place) + +Harness `plan/crw/model/probe.py` (local): takes the public D60 file, zeroes the low-bits +region (so TIFF pixel = hi·4), overwrites the entropy region with chosen bits (LSB-first +packed, ff restuffed), optionally patches the DecoderTable selector, runs the oracle, reads +decoded rows. First probes (filler = repeated bit): + +| Entropy bits | Row 0 decoded (hi) | Reading | +|---|---|---| +| all 0 | 497 505 490 498 483 491 476 484 … (parities fall by 7) | repeated all-zero token = k=3 code + mag `000` = −7 per parity; first token longer (−15, k=4) | +| all 1 | 512 512 512 … | all-ones token(s) = zero diffs | +| `100` + 511 + zeros | 16144 505 16137 498 … | no clamping in the oracle; convention probe | + +Then `plan/crw/model/treewalk.py`: adaptive prefix expansion. Three aborted attempts taught +the method its final shape: (1) a race — the first walker's process survived its kill and +fought a second one over a shared probe file (fixed with per-PID work files; both result +sets discarded as corrupt); (2) enumerating magnitude bits explodes 2^k-fold; (3) leaf +detection via an assumed magnitude convention fails because the convention is *not* plain +JPEG (single-bit probes behind a known codeword give non-binary-weight responses). + +## 2026-10-12/13 — truncation probing of the real stream (campaign 2) + +`plan/crw/model/boundary.py` / `boundary2.py`: feed the oracle the real stream truncated at +bit B plus a constant filler. `boundary2`'s agreement scan (decode under 0-filler AND +1-filler; count leading pixels where both agree and match the real decode) yields **exact +token boundaries**: agreement can only grow when a token completes inside [0,B). + +Zero-region result (D60, table 0, bits 0–120 ≈ pixels 0–321, all-zero rows; data in +`bounds2_zero.json`, local): + +| token bits (stream order, LSB-first in bytes) | len | emits pixels | +|---|---|---| +| `1001111111111111` | 16 | 1 (pix 0, d = −512) | +| `011111111111100111111111` | 24 | 2 (zeros) | +| `11111011` | 8 | 61 → ends exactly at pixel 64 (EOB) | +| `0000101111100000` | 16 | 1 (pix 64, first of block 1) | +| `1110000` | 7 | 1 | +| `1` | 1 | 62 → ends exactly at pixel 128 (**1-bit EOB**) | +| `11111011` | 8 | 2 | +| `11111101` | 8 | 63 (ends at pixel 193) | +| `0111111` | 7 | 1 | +| `010111111` | 9 | 63 | +| `11011111` | 8 | 63 | +| `11101111` | 8 | 1 | + +Established: **64-pixel block structure** (B4's "blocks of 64" holds); an EOB-style token +ends a block; the same bit string decodes differently in different blocks (`11111011` = EOB +in block 0 but a 2-pixel token later) → at least two tables/contexts in play (B5's two-table +recollection pointing the right way). First-of-block tokens are long (16 bits here, carrying +the −512 initial diffs). The value-token magnitude convention is still open — it is *not* +plain JPEG — and is being extracted from a second agreement scan over row 7's textured data +(`bounds2_row7.json`). + +## 2026-10-13 — flip probing (campaign 3): code/magnitude split read directly + +`plan/crw/model/flipscan.py`: flip one bit of the real stream at a time (real +continuation), diff the full decode against the real decode. A flipped magnitude bit +changes exactly one pixel by a power of two; a flipped code bit derails the parse. + +First result, the pixel-0 token `1001111111111111` (16 bits): bits 0–7 are the code +(`10011111`), bits 8–15 are an 8-bit magnitude, weights 1,2,4,…,128 in stream order +(LSB-first), and the decoded value responds **linearly** across the would-be JPEG sign +boundary (pixel0 = m − 255 over the whole probed range m = 127…255): the magnitude +convention is offset-style, not JPEG's two-branch rule, at least for this token. A full +flip scan over a textured region (row 11) is mapping code and magnitude bits for ~25 +consecutive tokens. + +## 2026-10-13 — bit order corrected: MSB-first; token structure pinned + +A systematic contradiction (magnitude fields appearing split across bytes, "token +boundaries" clustering at 8 bits) resolved when the flip map was re-plotted in MSB-first +bit coordinates: an early mis-step had locked the analysis into LSB-first reading, which +mirrors every byte. **The stream is MSB-first after ff00 unstuffing.** In MSB coordinates +every magnitude field is contiguous and the JPEG rule is confirmed by exact branch-flip +deltas (e.g. flipping the top magnitude bit of a k=5, m=2 token: −29 → +18, Δ+47 observed). + +Flip-verified token chain (D60 table 0, pixels 34994–34999, bits 20784–20819): + +| bits | table | meaning | pixels | +|---|---|---|---| +| `01` + `00010` | A | k=5, m=2 → −29 | 34994 | +| `11111100` + `11110` | B | run 1 + k=5, m=30 → +30 | 34995 (zero), 34996 | +| `1100` + `1` | A | k=1 → +1 | 34997 | +| `00` + `010` | B | k=3, m=2 → −5 | 34998 | +| `00` + `010` | A | k=3 → −5 | 34999 | + +Established: +- **Two Huffman tables, strictly alternating per token (A, B, A, B, …)** — forced by + prefix-freeness: the verified codes cannot coexist in one table or in parity-assigned + tables, and alternation renders both tables prefix-free. (This refines B5.) +- Tokens carry a **(zero-run r, size s)** pair: r zero-diff pixels plus one value of + size s; s = 0 codes are pure runs/EOB. Predictors: two, by pixel parity, init 512, + carried across rows (flip cascades propagate stride-2 to the image end). +- The 10-bit "hi" plane is nearly constant in dark/flat areas — the reason the low-bits + plane exists: the sensor noise lives in the 2 raw LSBs, keeping the entropy-coded + plane highly compressible. + +Open: exact placement of the bundled zeros (before vs after the value) in corner cases, +EOB/block semantics, the first-token special case at pixel 0, and the complete code +tables for DecoderTableNumber 0, 1, 2 (being extracted by continued flip scans plus the +known-plaintext aligner `plan/crw/model/aligner4.py`). + +## 2026-10-13 — complete token grammar established (D60, table 0) + +The MSB-faithful agreement scan (`bounds3_row11_msb.json`, 187 exact micro-boundaries over +bits 20776–21500) plus a merge analysis settled the remaining structure. Sub-boundary rises +occur inside tokens (fillers can reproduce zero pixels), and merging every zero segment into +the following value yields a token stream that is fully consistent — zero semantic conflicts +and zero prefix violations — under exactly one model: + +- **64-pixel blocks** (image order). The token that starts at a block's first pixel comes + from a **"first" Huffman table**; every other token from a **"second" table**. (The + apparent two-alternating-tables stage was a phantom: fitted as alternating tables, the two + halves come out *identical*, collapsing to one "second" table. B5's recollection — "one + table for the first element of a block, another for the rest" — was exactly right.) +- **Token = code → (run r, size s)**: r zero-diff pixels, then one value of size s coded as + s JPEG-rule magnitude bits (MSB-first). s = 0 codes are EOB (zeros to end of block). +- Two DPCM predictors by pixel parity, init 512, never reset. + +Second-table codes verified so far (D60, DecoderTableNumber 0): `00`=(0,3), `100`=(0,2), +`101`=(0,5), `1100`=(0,1), `111011`=(1,2), `111100`=(1,3), `111101`=(1,1), `11111100`=(1,5), +`1111111000`=(2,1), `1111111100`=(2,3), `11111110111100`=(16,1), and 16-bit codes +(3,1) (4,1) (4,2) (5,1) (5,4) (5,5) (6,3) (6,4) (8,1). First-table: `101`=(0,2), +`111110`=(0,10) at pixel 0. The code set has a canonical Huffman shape. + +A seeded known-plaintext aligner (`aligner6.py`) is now walking the whole 6.5-Mpixel stream +to learn the remaining codes of both tables and verify every pixel; the same harness then +runs on files with DecoderTableNumbers 1 and 2 and on the other bodies. + +## 2026-10-13 — the carry chain, and the first/second table split confirmed + +Synthetic sentinel probes plus the zero-region MSB scan established two more rules: + +- **Carry on block DC**: each 64-pixel block's first diff is coded relative to the + previous block's *combined* first diff (`diff0 += carry; carry = diff0`). This single + rule reconciles pixel 0 (coded −512), pixel 64 (coded +512 → effective 0) and pixel + 128 (coded 0 → effective 0) with plain parity chains init 512 — the earlier + "predictor anomalies" were all carry. +- **The first-of-block token really does use a separate Huffman table** (`101` = (0,2) + there vs (0,5) in the main table, verified by decode-iteration at pixel 35008), i.e. + the classic first/second table structure of B5. +- The S-EOB of the main table is 9 bits (`111111011`); agreement boundaries inside + all-zero blocks proved unreliable (fillers self-heal on zeros), so zero-region token + readings are being re-derived by decode-iteration rather than scans. + +Current state: the main table has ~20 verified codes; a decode-and-learn loop +(`plan/crw/model/aligner8.py`) walks the real stream, decoding deterministically with +known codes and learning first-table/main-table rows at each stall, each verified +against the oracle pixels. + +## 2026-10-13 (cont.) — row reset; full model assembled + +Two refinements completed the structural model, each validated against the stream: + +- **Chains reset to 512 at every row start** (not row padding — a padded-grid variant was + tried and refuted by block alignment at pixel 35008). With the reset, every all-zero row + needs one k=10 "−512" value token at its first pixel, which is exactly what the stream + shows: a 15-bit second-table code (15,10) at the first row crossing, mag 511 → −512. +- The verified second table gains `01` = (0,5) (flip-verified at pixel 34994); both `01` + and `101` map to (0,5) — duplicate leaves are legal and present. + +Model now complete: flat 64-sample blocks; first/second Huffman tables (first = the token +starting each block); (run ≤ 15, size) leaves with JPEG MSB magnitudes; (0,0) mid-block = +EOB; per-block DC carry; parity chains init 512, reset at row starts; low-bits recombined +as the 2 LSBs. Remaining: complete code tables per DecoderTableNumber, extracted by the +known-plaintext walker (`aligner11.py`), then full-corpus verification. + +## 2026-10-13 — SELECTOR-0 TABLES COMPLETE; four bodies decode end-to-end + +The greedy known-plaintext extractor (`plan/crw/model/aligner12.py`: deterministic decode +with deep trial-validation per unknown code, flip-scans resolving the few ambiguous sites) +completed the DecoderTableNumber-0 Huffman tables. Results: + +- **D60 (public): the full 6 518 336-sample frame decodes from bit 0 against the oracle, + pixel-exact,** in seconds, with only the final 3 samples open (stream tail handling). +- **10D, 300D and Digital Rebel (selector 0) decode end-to-end with the same table** — + per-selector tables are shared across bodies, as the plan hoped. Each file exercised a + few additional rare high-k codes (merged). +- Final selector-0 tables: 11 first-table codes, 131 second-table codes (incl. the 16-bit + `11111111xxxxxxxx` run/size family); saved as `logs/table_sel0_d60class.json`. +- The decisive last fix: a hand-guessed 15-bit seed blocked the true 11-bit (1,7) code by + prefix; a flip-scan at the stall site read the real code and magnitude (m = 83) directly. + +Selector 1 (from the D30) and selector 2 (from the second 10D file) extractions running +with the same tooling. Remaining after that: G1/G5/G6 PowerShot handling (10-bit G1; the +G5 low-bits anomaly), the stream-tail rule, then the frozen format document and the +Python reference decoder for the P4 verification pass. + +## 2026-10-13 — selectors 1 and 2; G1, G6, private files; G5 isolated + +- Anchor hunting (truncation-probe agreement at byte boundaries + bit-level pinning) + plus structural bootstrap (reading each file's opening k=10 tokens and the steady + zero-block unit directly off the bits) made extraction routine for new table sets. +- **Selector 1** (extracted from private CRW_0476, topped up from the public D30): + D30 **EXACT**, PowerShot **G1 EXACT** (10-bit, no low-bits plane — detected by the + frozen unstuffed-ff scan), CRW_0476 **EXACT**. Cross-body sharing confirmed again. +- **G6 EXACT** with the selector-0 table once the low-bits rule was frozen (presence + by the ff-scan; entropy always at low-bits-size + 514). +- **G5 remains anomalous**: its low-bits plane matches the oracle's LSBs ~97.5 % in the + top rows (not 100 %), degrading to 0 % in the lower half, and no constant bit-shift + explains it; its entropy stream also diverges from the selector-1 parse within the + first rows. G5 is documented as unresolved and will be refused (or preview-only) by + the decoder pending further work. +- Selector 2 (from the public 10D-b) extraction completing. + +## 2026-10-13 — P2 complete; tables frozen; 28 of 29 files verify EXACT + +A final systematic fix (first-table magnitudes reach s = 11+: the carry-coded block DC +spans ±2046) unlocked selector 2 (extracted from the 300D-b after the saturated 10D-b +proved hostile learning ground). Table top-ups over every private file completed the +three table sets (12 first + 154/161/162 second codes). The final sweep: **28 of 29 +corpus files decode with zero mismatching samples** against LibRaw 0.22.2 — every EOS +body and selector, the 10-bit G1, the G6, and all 18 private D60s. The PowerShot G5 +remains the open anomaly (stream decodes, values diverge from sample 2056; low-bits +plane fits no tested layout) and is excluded pending follow-up. + +Frozen at this commit: `06-crw-format.md` (the format), `logs/table_sel{0,1,2}.json` +(the tables), `07-verification.md` (the evidence). P3 (black/WB/colour records) and P5 +(the Rust decoder) proceed only after maintainer review, per the plan. + +## 2026-10-13 — P5: Rust decoder lands; P3 colour/geometry derived + +The maintainer approved the freeze. `crates/raw/src/vendor/crw.rs` implements the frozen +model (tables generated from `logs/table_sel{0,1,2}.json` into `crw_tables.rs`); the Rust +decoder matched the oracle **sample-exact on all 28 decodable corpus files on its first +run** (`crates/raw/tests/crw_oracle.rs`, ~65 ms per 8 MP file). The G5 is refused by model +string. Colour/geometry work (CFA phases by chroma correlation + green-diagonal tests, +border black levels, 0x10a9/0x10aa white balance, rotation and timestamps) is documented in +`08-color.md`; ExifTool prose pages and a raw.pixls.us repository-listing fetch (to recover +the 11 corpus files' stable URLs for `cargo xtask corpus`) are logged in `sources.md`. + +## 2026-10-13 — Freeze reopened (revision 2): 1,516 D60 files, two corrections + +The maintainer put all his D60 files in `../RawImages` (1,516). The revision-1 decoder +failed on 232 of them, which the 29-file corpus could not have shown: + +- **223 "undecodable entropy code".** The second tables were learned from exercised codes + only: selector 0 had 154 of 162 symbols, selector 1 had 161. Laying out the 16-bit code + range showed exactly eight unassigned slots inside selector 0's range and one inside + selector 1's, against exactly eight and one missing symbols: (7..14, 10) and (14, 10). + For selector 1 the assignment follows by elimination; the learner (`aligner12.py`, same + known-plaintext method, LibRaw as the black box) confirmed it and assigned selector 0's + eight from the failing files. +- **9 files decoded as 10-bit garbage.** Their low-bits plane holds no 0xff byte, because + the lowest bit of nearly every sample is 0, so the revision-1 presence rule said "no + plane". My first reading (an all-zero plane) was wrong: I had counted 0xff bytes, not + nonzero bytes; the plane has 1.5 M nonzero bytes. The rule now also accepts the zero + header found right after the plane. + +Result: 1,516 of 1,516 EXACT, and the first corpus unchanged (`07-verification.md`). +A lesson for the remaining unverified bodies: a table learned from a few files is complete +only when its symbol count is; selector 2 was already at 162. diff --git a/crates/crw/clean-room/06-crw-format.md b/crates/crw/clean-room/06-crw-format.md new file mode 100644 index 00000000..e3249f70 --- /dev/null +++ b/crates/crw/clean-room/06-crw-format.md @@ -0,0 +1,115 @@ +# Canon CRW raw compression — derived format (FROZEN) + +Status: **frozen, revision 2** (plan §5 P4). The derivation is not tuned; any correction +must reopen it explicitly with a recorded cause and a new freeze commit. + +**Revision 2 (2026-10-13), cause:** the maintainer supplied 1,516 D60 files, 50 times the +first corpus. 223 of them failed on code-table entries the first corpus never exercised, +and 9 were misread as having no low-bits plane. Two corrections, both recorded in +`04-derivation-log.md` and re-verified in `07-verification.md`: the low-bits presence rule +(§1) gained the header-position test, and the selector 0 and 1 second tables were completed +(§3). Nothing else in the model changed, and every file of the first corpus still verifies. Everything here was derived from the CC0/private sample files +and black-box probing of LibRaw 0.22.2 (`unprocessed_raw.exe`), per the approved plan; no +decoder source was consulted. The chronological evidence is `04-derivation-log.md`; the +container and payload groundwork are `02-container.md` and `03-payload-layout.md`. + +## 1 Payload layout + +RawData (CIFF record 0x2005), with w×h from SensorInfo (0x1031) and +(selector, 0, entropyOffset = 514, compressedLength) from DecoderTable (0x1835): + +``` +[ low-bits plane, ceil(w*h/4) bytes ] present on some bodies (rule below) +[ 514 zero bytes ] (the DecoderTable's entropyOffset) +[ entropy bitstream, ff00-stuffed ] +[ tail: 00 00 / ff 00 00 (EOS) or ff d9 (+00) (PowerShot) ] +``` + +- **Low-bits plane**: 2 bits per sensor sample, row-major, LSB-first within each byte + (sample 4i in bits 1:0 … sample 4i+3 in bits 7:6). Present ⇔ either + (a) the first w·h/4 + 514 bytes of the payload contain any 0xff byte NOT followed by + 0x00 (the entropy stream is always stuffed, the low-bits plane is raw), or + (b) the 514 bytes at offset w·h/4 are all zero (the zero header sits right after the + plane; without a plane that offset is deep in entropy data). + Rule (a) alone, the revision-1 rule, misses planes that hold no 0xff byte: some D60 + files have a lowest bit that is almost always 0 (low values 1 and 3 on about 0.04 % of + samples), so no four neighbouring samples are all 3. Verified: present on + D30/D60/10D/300D/G5/G6, absent on G1. +- On the EOS bodies the DecoderTable's compressedLength excludes the low-bits plane; + on G5/G6 it includes it. The decoder should not rely on compressedLength: entropy + data starts at (low-bits size, if present) + 514 and runs to the tail. + +## 2 Entropy bitstream + +After removing the stuffing (drop the 0x00 after every 0xff), bits are read +**MSB-first**. The stream codes the w×h sensor samples' 10-bit "high" plane +(the full sample is `(high << 2) | low_bits` where a low-bits plane exists, else +`high` itself): + +- Samples are produced in row-major order, grouped in **blocks of 64 samples** laid + flat over the frame (blocks cross row boundaries; w need not divide by 64). +- Two canonical Huffman code tables per DecoderTableNumber (0, 1, 2): the token that + begins each block is read from the **first table**, every other token from the + **second table**. +- A token decodes to a leaf **(r, s)** with r ≤ 15: skip r samples (each emits its + DPCM chain value unchanged, i.e. a zero diff), then read s magnitude bits and emit + one diff (s = 0 emits a zero diff). Magnitudes follow the JPEG rule, MSB-first: + m ≥ 2^(s−1) → d = +m, else d = m − 2^s + 1. Sample diffs fit s ≤ 10 (10-bit high + plane), but first-table leaves reach s = 11+: the carry-coded block DC difference + spans ±2046. +- The **(0,0) leaf read as a non-first token is EOB**: the rest of the block emits + zero diffs. ((0,0) as the block's first token is a plain zero diff.) +- **Carry**: the diff landing on each block's first sample has the previous block's + (combined) first diff added: `d0 += carry; carry = d0`. When the first sample is + covered by a token's skip (or EOB), its diff is exactly `carry` (and carry is + unchanged). +- **DPCM chains**: two running predictors by sample parity, both initialised to 512, + and both **reset to 512 at the start of every sensor row**. +- If the stream ends before the frame is complete, remaining samples emit their chain + values (observed: the final 2–3 samples of each file). + +## 3 Code tables + +Extracted from the samples by known-plaintext alignment and frozen as data: + +- Selector 0: `logs/table_sel0.json` — 12 first-table + 162 second-table codes. +- Selector 1: `logs/table_sel1.json` — 12 + 162. +- Selector 2: `logs/table_sel2.json` — 12 + 162. + +Each second table now holds all 162 symbols: (r, s) for r = 0..15, s = 1..10, plus (0, 0) +and (15, 0). Revision 1 lacked (7..14, 10) in selector 0 and (14, 10) in selector 1: runs +of 7 or more zero differences followed by a difference of 512 or more, which the first +corpus never contained. The nine codes were learned from the wider corpus by the same +known-plaintext method; each landed in one of the unassigned 16-bit slots inside the code +range, of which there were exactly eight and one. The slots still unassigned are the +all-ones tail of each table (5, 20 and 3 codes) and the first tables' longest code. + +Tables are shared across bodies per selector (verified across D60/10D/300D/Rebel for +selector 0; D30, G1 and the private D60s for selector 1; 10D, 300D and a private D60 +for selector 2). The tables have canonical Huffman structure; codes up to 16 bits, the +`11111111xxxxxxxx` region carrying the larger (run, size) pairs. The JSON rows are a +superset of each file's exercised codes: every row was learned from a real stream +position and validated by the decode reaching the end of that frame; rows never +exercised by two files may in principle be mergeable duplicates of canonical slots, +which the Rust implementation may normalise, provided the verification table of +`07-verification.md` still reproduces exactly. + +## 4 Not yet established / out of scope here + +- **G5 anomaly**: its entropy stream decodes end-to-end with the selector-1 tables, + but values diverge beginning at sample 2056 (= 4 × 514), and its low-bits plane fits + no tested layout (top rows mostly match, decaying to 0 %, no constant shift). The + decoder must detect the G5 (CanonModelID / make-model) and fall back to the embedded + JPEG until a follow-up derivation resolves it. +- The exact meaning of each file's tail bytes (handled by the stream-end rule). +- Big-endian (`MM`) CIFF files: no sample exists. +- PowerShot G2/G3/Pro1/Pro70/S30–S70: no CC0 samples (licence-excluded); decodable + speculatively via the same selectors once their geometry is read from SensorInfo, + to be marked unverified. +- Black level, white balance and colour (plan P3) are documented separately. + +## 5 Verification + +See `07-verification.md` (written after the freeze): full-frame exact-match decode +against LibRaw 0.22.2 for every sample file, public and private, with zero tuning +after the freeze commit. diff --git a/crates/crw/clean-room/07-verification.md b/crates/crw/clean-room/07-verification.md new file mode 100644 index 00000000..e379e833 --- /dev/null +++ b/crates/crw/clean-room/07-verification.md @@ -0,0 +1,79 @@ +# CRW decode verification (P4) — against LibRaw 0.22.2, zero tuning + +Method: the frozen model of `06-crw-format.md` plus the frozen tables +(`logs/table_sel{0,1,2}.json`) were run by the local Python reference decoder over every +sample file, comparing **every sensor sample** (full values, low bits merged) against +`unprocessed_raw.exe -T` output. "EXACT" means zero mismatching samples over the whole +frame. The table extraction itself consumed these same files (known-plaintext learning), +so this is a consistency proof against the oracle, not a blind holdout; the cross-body +sharing (tables learned on one body, exact on others) is the strongest independent check. + +Private files are the maintainer's own D60 photos (hashes in `01-plan.md` §2.2); only +their verdicts are recorded here, never content. + +## Results + +| File | Selector | Frame | Verdict | +|---|---|---|---| +| `crw-canon-eos-10d-a.crw` | 0 | 3152x2068 | **EXACT** | +| `crw-canon-eos-10d-b.crw` | 2 | 3152x2068 | **EXACT** | +| `crw-canon-eos-10d-c.crw` | 2 | 3152x2068 | **EXACT** | +| `crw-canon-eos-300d-a.crw` | 0 | 3152x2068 | **EXACT** | +| `crw-canon-eos-300d-b.crw` | 2 | 3152x2068 | **EXACT** | +| `crw-canon-eos-d30.crw` | 1 | 2224x1456 | **EXACT** | +| `crw-canon-eos-d60.crw` | 0 | 3152x2068 | **EXACT** | +| `crw-canon-eos-digital-rebel.crw` | 0 | 3152x2068 | **EXACT** | +| `crw-canon-powershot-g1.crw` | 1 | 2144x1560 | **EXACT** | +| `crw-canon-powershot-g5.crw` | 1 | 2672x1968 | 2685618 mismatches, first at 0 (row 0, col 0) | +| `crw-canon-powershot-g6.crw` | 0 | 3160x2344 | **EXACT** | +| `private/CRW_0476.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_0478.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_0486.crw` | 0 | 3152x2068 | **EXACT** | +| `private/CRW_0527.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_0683.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_0717.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_0765.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_0822.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_0879.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_1011.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_1153.crw` | 0 | 3152x2068 | **EXACT** | +| `private/CRW_1277.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_1514.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_1646.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_1740.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_1776.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_2035.crw` | 1 | 3152x2068 | **EXACT** | +| `private/CRW_2219.crw` | 2 | 3152x2068 | **EXACT** | + +**28 of 29 files decode EXACT** — all 10 usable public CC0 files and all 18 private +D60 files; the PowerShot G5 is the known exception (`06-crw-format.md` §4). + +## Notes + +- The final 1–3 samples of each frame follow the stream-end rule (chains persist); + every "EXACT" verdict includes them. +- PowerShot G5: see `06-crw-format.md` §4 — detected and excluded; embedded-JPEG + fallback until resolved. +- The G1 (10-bit, no low-bits plane) verifies the low-bits presence rule; the G6 + (12-bit, different geometry, w not divisible by 64) verifies the row-reset and + block rules on a second geometry. + +## Revision 2: the wider D60 sweep (2026-10-13) + +The maintainer supplied 1,516 Canon EOS D60 files (private; local use only, nothing +copied into the repository, verdicts only recorded here). With the revision-1 decoder, +223 failed on unseen code-table entries and 9 were misread as 10-bit (`06-crw-format.md`, +revision 2). After the two corrections every file was compared against LibRaw 0.22.2 +`unprocessed_raw.exe -T`, every sensor sample, with the Rust decoder +(`plan/crw/model/sweep_all.py`, local): + +| Selector | Files | Verdict | +|---|---:|---| +| 0 | 243 | all **EXACT** | +| 1 | 1,193 | all **EXACT** | +| 2 | 80 | all **EXACT** | +| **Total** | **1,516** | **1,516 EXACT** | + +The nine learned codes came from 30 of the failing files; the other 202 failing files and +the 1,284 that already decoded are a blind check of them. The public files were re-run +with the same build: 10 EXACT, the G5 still refused. diff --git a/crates/crw/clean-room/08-color.md b/crates/crw/clean-room/08-color.md new file mode 100644 index 00000000..8a04151c --- /dev/null +++ b/crates/crw/clean-room/08-color.md @@ -0,0 +1,122 @@ +# CRW colour, geometry and metadata records (P3/P5) + +Status: derived 2026-10-13, after the compression freeze. Everything here comes from the +approved sources: the corpus files themselves (public CC0 + the maintainer's private D60s), +the oracle's decoded sensor data, each file's own embedded JPEG, and ExifTool's prose tag +documentation (`TagNames/CanonRaw.html`, `TagNames/Canon.html` — tag names, table indices and +value enumerations only; fetches logged in `sources.md`). No decoder source was consulted. +Implemented in `crates/raw/src/vendor/crw.rs`. + +## 1 CFA phase + +Determined by two independent sample-derived tests on each body's oracle mosaic: + +- **Chroma correlation**: bin the mosaic 2×2 into RGB under each of the four Bayer phase + hypotheses, roughly gain-match to the body's own embedded JPEG (linearised ≈ γ2.2), and + correlate the R−B chroma plane with the JPEG's. The true phase correlates strongly + (+0.49…+0.94), its R/B swap strongly negatively (`cfa_check.py`, local model dir). +- **Green diagonal**: within 16×16 blocks the two green sites have near-equal sums; the + diagonal with the smaller |difference| carries the greens. + +Results: **RGGB at sensor (0, 0)** for EOS D30, D60, 10D, 300D (both tests decisive) and +PowerShot G6; **GBRG for the PowerShot G1** (greens on the main diagonal; chroma +0.77 GBRG +vs −0.67 GRBG). Unsampled PowerShots default to RGGB, unverified. + +## 2 Active area, black level, white level + +- **Active area**: SensorInfo (0x1031) values 5–8 are an inclusive crop (left, top, right, + bottom); valid on every corpus body (G5/G6 included). See `02-container.md` §3. +- **Black level**: measured from the masked borders. Observed (oracle data, per body class): + the left border is dark up to 8 columns before the active area on the EOS bodies + (≈127–129 DN on D60/10D, ≈128 on D30, ≈124–127 on 300D) but transitions bright earlier on + the G6 (dark only to col 40 of 64), and on the G1 only the top border is dark + (rows 0–7 ≈ 29 DN; its left border reads ≈98–160). The D60's first ~7 sensor rows are + clamped to 0 (not a black reference). Rule adopted: candidate strips = 8-column groups of + the left border and 4-row groups of the top border; black = per-CFA-site mean of the + darkest strip and all strips within 8 DN of it, ignoring strips with mean ≤ 1 (the clamped + rows). This lands on the true dark level on every corpus body. Cross-check: the trailing + quadruple of ColorBalance (0x10a9) is (126–127)×4 on D60/10D and (124–125)×4 on 300D — + matching the measured borders (ExifTool's prose notes this quadruple as "black levels"). +- **Correction (2026-10-13):** the first implementation also averaged in the clamped strips + it meant to ignore (mean ≤ 1 passes "within 8 DN of the darkest"), which put the D60's + level at 117 instead of 128 and lifted every shadow. Fixed and tested; on 20 private D60 + files the left border's mean is within 2 DN of the record's black quadruple, per site + (127–137, higher on the files with the larger calibration-row value). +- **White level**: saturation plateau of the decoded samples (shared `white_from_data`); + without a plateau, the level at least 64 samples reach when that is within 3 % of the + full range (`white_level`); else the full range, 4095 (12-bit) / 1023 (G1). +- **The sensor clips below 4095 at low ISO** (2026-10-13, 1,516 private D60 files by ISO + from ShotInfo value 2, ISO = 100·2^(v/32 − 5)): at ISO 100, 200 and 400 no sample exceeds + 4009 and the maxima cluster at 4000–4008; at ISO 800 and 1000 saturated files reach 4095. + The 10D and 300D samples stop at 4003–4008 as well. Before `white_level`, a photo with a + small blown area had no plateau and got 4095 (373 of 925 files at ISO 100, 283 of 287 at + ISO 400), leaving blown pixels at 0.977 of white, below the clip threshold. +- **Controlled frames from the maintainer** (lens cap and blown-out, ISO 100 and 1000; all + four decode sample-exact). Cap, ISO 100: active area 129.2–130.1 per site, left border + 128.2–129.6, our level 128.9, record 127–129. Cap, ISO 1000: active area 129.5–130.5, + but left border 134.5–135.7, our level 133.8, record 131–133: at high ISO the masked + border and the record read about 4 DN above the image's true dark level. One frame, so + no correction is applied; noise there is 15 DN. Blown-out, ISO 100: the common values + are 3997–4000 and the maximum 4002 (our level 3987). Blown-out, ISO 1000: every active + sample is 4095 (our level 4080). + +## 3 As-shot white balance + +- **0x10a9 ColorBalance** (D60/10D/300D only): a byte-length u16, then (R, G, G, B) level + quadruples per white-balance setting, the last quadruple being black levels. ExifTool's + prose gives the 10D/300D slot order — Auto, Daylight, Shade, Cloudy, Tungsten, Fluorescent, + Flash, Custom, Kelvin, Black — at indices 1, 5, …, 37, which the record sizes confirm + (82 bytes = 1 + 10×4 u16); the D60's 66-byte record holds 7 slots + black at index 29 + (= ExifTool's "D60 black levels" index), i.e. the same order without Shade and Kelvin. +- Multipliers are **level / green** (e.g. D60 slot 1: 1856, 784, 781, 945 → R 2.37, B 1.21). + Confirmed against the cameras' own JPEGs by a white-preserving matrix fit (rows sum to 1, + least squares from WB-corrected binned raw to the linearised JPEG): plausible slots score + far better than implausible ones; among near-daylight slots the fit cannot fully separate + (γ-model error), so the slot is chosen by metadata, not by fit. +- **Slot selection**: ShotInfo (0x102a) index 7 is the WhiteBalance setting (ExifTool prose: + 0 Auto, 1 Daylight, 2 Cloudy, 3 Tungsten, 4 Fluorescent, 5 Flash, 6 Custom, 8 Shade, + 9 Kelvin). Every corpus file reads 0 (Auto) → slot 0. +- **D60 preset mapping verified** (2026-10-14): the maintainer shot one overcast scene at + each setting, Auto, Daylight, Cloudy, Tungsten, Fluorescent, Flash and Custom, ISO 100, + within a minute (all seven decode sample-exact). Slots 1–6 hold the same multipliers in + every file; slot 0 (Auto) varies slightly per shot. On the pixels each camera JPEG + renders neutral, the multipliers the camera applied are nearest to slot 1 for Daylight, + 2 for Cloudy, 4 for Fluorescent and 5 for Flash, i.e. setting n uses slot n. Tungsten and + Custom leave too few neutral pixels for that test (5 and 0), so they were checked by + rendering: the overall tint of our render follows the camera JPEG for every setting + (Tungsten JPEG R/G 0.57 B/G 1.41, ours 0.66 / 1.52; Custom 0.62 / 2.93, ours 0.80 / + 3.02), which no other slot would give. Shade and Kelvin (10D/300D only) remain untested. +- **Auto slot confirmed on the private D60 files** (2026-10-13): on the pixels each camera + JPEG renders neutral (channel spread < 6 of 255, mid-tone, raw unclipped), the median raw + G/R and G/B give the multipliers the camera applied. On 17 files with enough neutral + pixels, slot 0 is within 0.005–0.036 (summed log error) of that on every one, flash + shots included, and is the nearest slot on 16 (the 17th prefers Daylight by 0.005). + Numbers only; no image content recorded. +- **0x10aa** (same three bodies): one (R, G, G, B) quadruple with G ≈ 1024 and R/B as + *levels* (multiplier = G/level); consistent with a measured-white-balance record (Canon + maker-note 0xaa MeasuredColor). On some files it fits the camera JPEG better than slot 0, + on others worse; not used, kept as a documented alternative. +- **Not derived yet**: D30 and PowerShot white balance (no 0x10a9/0x10aa; the 0x1030 + WhiteSample record holds an 8×8 packed sample patch — constant filler on G1/G6, varying on + D30 — undeciphered). These bodies render with the neutral fallback. +- **Colour matrices**: none of the CRW bodies is in `spectral_table.rs`. Rendering uses + the engine's per-file starting look fitted to the file's own embedded JPEG + (`camera_preview::file_local_look` lists CRW; `docs/camera-preview-colour.md`) — accepted + on all 10 decodable public corpus files, holdout RMS 0.009–0.043, including the D30 and + the PowerShots, whose missing white balance the fitted matrix absorbs. + **Update 2026-10-25:** the EOS D60 has dual-illuminant matrices of our own, fitted to + ColorChecker Classic shots under a tungsten bulb and direct sun (`10-chart-calibration.md`, + `crates/raw/src/chart_table.rs`). They give its white balance a Kelvin scale and are its + colour when the JPEG fit fails; the look stays the JPEG fit. The other CRW bodies have none. + +## 4 Orientation and capture time + +- ImageInfo (0x1810) value 3 is rotation in degrees. The two portrait 300D corpus files + carry 270, and their scene is upright after rotating the stored image 90° counter-clockwise + (checked visually against the embedded JPEG) → 270 maps to Exif orientation 8, hence + 90 → 6 and 180 → 3. All other corpus files carry 0. +- TimeStamp (0x180e) value 0 is a Unix time on the camera's local clock (cross-checked + against the files' Exif-era siblings and plausibility); decoded as a civil time with no + UTC offset. +- Make/model: 0x080a, two NUL-terminated strings ("Canon", "Canon EOS D60"); the G5 + refusal and the G1 CFA special-case key on the model string. diff --git a/crates/crw/clean-room/09-private-corpus.md b/crates/crw/clean-room/09-private-corpus.md new file mode 100644 index 00000000..e1b4566b --- /dev/null +++ b/crates/crw/clean-room/09-private-corpus.md @@ -0,0 +1,58 @@ +# Private EOS D60 corpus and controlled captures + +The maintainer's own Canon EOS D60 files in `../RawImages/` (local only, never copied into +the repository). Names, sizes and SHA-256 of every file are in +`logs/private-corpus.tsv`; no image content is recorded anywhere. All 1,561 files decode +sample-exact against LibRaw 0.22.2 `unprocessed_raw.exe -T`. + +| Set | Files | What it is | What it established | +|---|---:|---|---| +| `RawImages/` | 1,516 | Everyday photos, 2005–2014, all Auto white balance; ISO 100 (925), 200 (197), 400 (287), 800 (34), 1000 (73) | Freeze revision 2: two incomplete code tables and the plane-without-0xff case (`06-crw-format.md`); the low-ISO clip level (`08-color.md`); the D60 colour profile (`assets/camera-profiles/Canon_EOS_D60.json`, 1,424 usable photos) | +| `Cap&WhiteOut/` | 4 | Lens cap and blown-out frames, ISO 100 and 1000 (2026-10-13) | Black level at ISO 100 matches the image area; at ISO 1000 the border reads about 4 DN high. Clip level about 4000 at ISO 100, 4095 at ISO 1000 (`08-color.md`) | +| `WBModes/` | 7 | One overcast scene at Auto, Daylight, Cloudy, Tungsten, Fluorescent, Flash, Custom; ISO 100, within a minute (2026-10-14) | White-balance setting n uses ColorBalance slot n (`08-color.md`) | +| `TripodGrayCard/` | 14 | The same card filling most of the frame, tripod, f/8, ISO 100; two brackets (Auto 1/16–1/6 s, Cloudy 1/12–1/5 s), each starting and ending on the same speed; 14 s each (2026-10-14) | Black level confirmed; sensor linear; the spill strip (below) | +| `GrayCard/` | 20 | An 18 % grey card (film-era), handheld, overcast; 5 frames at f/4 with exposure compensation, 15 at f/1.8 stepping 1/640–1/1600 s under Cloudy, Daylight and Auto (2026-10-14) | See below | + +## Grey-card results (2026-10-14) + +Measured in the middle 28 % of each frame, which lies on the card in every frame (the card +is larger and was handheld; spread across the box about 3 %, sensor noise included). + +- **Tone matches the camera.** Our render's card lightness is within 0.3 L\* of the camera + JPEG's in all six frames checked (L\* 37.6–59.0 across exposures), so the profile's + per-photo tone fit reproduces Canon's curve at mid-grey. +- **Neutrals carry a slight magenta.** Our card is a\* +0.6 to +1.5 above the camera's, + b\* 0.4–1.0 below (about 1–1.5 ΔE, under the usual threshold of visibility). It is the + same small bias seen on the white-balance series' foliage, measured here on a true + neutral. The profile matrix's rows sum to 1.005, 0.998 and 1.008, which accounts for part + of it. Left for the chart refit, which can hold neutrals exactly. +- **Overcast neutral.** The card needs R × 2.68 and B × 1.13 (G = 1) to render neutral, the + same in all four groups to ±0.03. That lies between the camera's Daylight (2.54 / 1.22) + and Cloudy (2.80 / 1.13) presets; its Auto chose R 2.54–2.60, B 1.14–1.16, slightly cool, + which our As Shot reproduces, as it should. +- **Black level:** these handheld brackets could not pin it through exposure (shutter + accuracy at 1/640–1/1600 s, and the light dimmed about 10 % over the 90 s); the tripod + set below settles it. *Correction:* the "border measurement of 130.4–132.0" first written + here averaged every masked column, including the 8 next to the image, which catch spilled + light. The decoder skips that strip and reads 127.6–128.6 on these 20 frames; the + colour-ratio method's 124.5–127.5 is within 4 DN of it. +- **What it cannot do:** a single neutral says nothing about how colours map, so the card + does not change the colour profile. The chart will. + +## Tripod grey-card results (2026-10-14) + +- **The masked strip next to the image catches spilled light.** Columns 58–65 rise with the + scene: 157 at 1/16 s, 206 at 1/6 s, while the columns further out stay at 127–130. The + decoder's darkest-strip rule already excludes it: its black level stays at 127.2–128.7 + through both brackets, whatever the exposure. +- **Black level confirmed.** Each bracket was fitted with level = black + k · time · (1 + drift + · seconds), the drift taken from the repeated speed (the light fell 5–6 % within each 14 s + bracket; without correcting for it the fit puts black near 120). Red, the channel least + sensitive to any remaining exposure error, gives 128.0 and 128.2 against the decoder's + 127.9 and 128.3; green and blue give 129.9–133.1; the colour-ratio method gives + 125.5–126.2. All within 5 DN of the decoder, so no change. +- **The sensor is linear.** With the drift corrected, every frame lies within 1.3 DN of the + fitted line on the Auto bracket and within 4 DN on the Cloudy one. +- **Lens falloff at f/8:** the card's level varies about ±8 % across the frame (5 × 5 cells, + lightest at the bottom, darkest top right), part vignetting, part uneven light on the + card. A colour chart in the middle third will see well under that. diff --git a/crates/crw/clean-room/10-chart-calibration.md b/crates/crw/clean-room/10-chart-calibration.md new file mode 100644 index 00000000..beb24d0b --- /dev/null +++ b/crates/crw/clean-room/10-chart-calibration.md @@ -0,0 +1,97 @@ +# EOS D60 colour matrices from a ColorChecker (2026-10-25) + +The Canon EOS D60 has no published spectral sensitivities, so it is not in `spectral_table.rs`. +These are its DNG-style matrices (`ColorMatrix1/2`, `ForwardMatrix1/2`), fitted to our own +photographs of a ColorChecker Classic under two lights. They are in +`crates/raw/src/chart_table.rs`. Measurements come from the maintainer's own captures (names, +sizes and SHA-256 in `logs/chart-captures.tsv`; no image content recorded) and the chart's +published reference colours (`sources.md`). Scripts and measurements are local-only, in +`plan/crw/chart/`. + +## Captures + +| Set | Frames used | Light | Notes | +|---|---:|---|---| +| `SunlightColorCard/` | 20 of 22 | Direct afternoon sun, ISO 100 | Calibrite ColorChecker Classic (50th-anniversary edition, post-2014 formulation) leaning on a wall, filling most of the frame. Auto and Daylight white balance, a shutter bracket. Two frames dropped: a white-patch sample ≥ 3900 (clip ≈ 4000 at ISO 100, `08-color.md` §2). | +| `TungstonColorChart/` 2696–2706 | 11 | One household tungsten bulb, ISO 100 | Chart flat on a wall, a shutter bracket. Brightest white patch 2657 of ~4000. | +| `TungstonColorChart/` 2688–2691 | not used | Same bulb, 8 minutes earlier | Steady within the set, but its neutral reads ~3,040 K against ~2,900 K for the later set: the light changed (window light or bulb warm-up). The later, warmer and better-exposed set is used. | + +All 37 frames decode sample-exact against the black-box reference (`crw_oracle.rs`). + +An earlier set on a ColorChecker-layout clone (Pixel Perfect "24 Standard Color Calibration +Chart", 2026-10-15) was not used. The maker publishes no reference values, and its grey patches +are not neutral: against an 18 % grey board they read R/G 0.88–0.94 in shade and B/G up to +1.29 under warm LED light, the darker patches drifting most. That is the sign of greys printed +from coloured inks. + +## Measurement + +- Patches are located on the embedded JPEG (four corner patch centres, bilinear grid), tracked + per frame by phase correlation, and checked visually on drawn boxes. The tungsten-set chart + was upside down in the first four frames; patch order is mapped accordingly. +- Each patch is measured in a central box of about 40 % of its width in the linear raw (black + from the far masked border per CFA site; R, mean of both G, B per 2×2 site). Uniformity + across the box (3×3 sub-blocks): ≤ 4 % in sun, ≤ 7 % under the bulb (light falloff across + the closer chart). +- Frames are normalised by the four middle neutrals (patches 20–23), averaged, and white + balanced on those neutrals. White balance (R, B multipliers, G = 1): sun 2.362, 1.298; bulb + 1.353, 2.213. The neutrals agree on it to ±2 %. + +## Fit + +- The reference is X-Rite's published L\*a\*b\* (D50) for charts made after November 2014. +- The model is a 3×3 matrix from white-balanced camera RGB to XYZ (D50), with neutral (1, 1, 1) + held exactly at the engine's D50 white (`lightcraft_color::D50`): the third column is D50 + minus the first two. That is a DNG `ForwardMatrix`. +- **Chromaticity only.** Each prediction is scaled to the patch's reference luminance before + the CIEDE2000 residual. In the sun the chart warped as it heated: the grey row's brightness + slope swung from +4 % to −7 % across the 70 s of shooting, in time order, and the per-patch + brightness needed was not smooth across the chart. A fit including lightness would have + absorbed that into the matrix. Chromaticity still determines the whole 3×3 up to scale, + which the white constraint fixes. The cost is that luminance of saturated blues and purples + rests on the model, not the measurement. +- `ColorMatrix` per light = diag(1 / multipliers) · ForwardMatrix⁻¹ · Bradford(light → D50), + scaled so the light's white maps to camera values with maximum 1. By construction it maps + the filed illuminant's white to the measured camera neutral. + +| Light | CIEDE2000 on the 24 patches | Leaving each patch out | Worst | +|---|---|---|---| +| Sun (filed as D50) | mean 1.70, median 1.46 | mean 1.97 | white 4.5 | +| Bulb (filed as A) | mean 2.42, median 1.79 | mean 2.74 | bluish green 6.7 | + +- One matrix does not serve both lights: the sun's matrix on the bulb's chart gives mean 3.6 + (cyan 9.0), the bulb's on the sun's 3.4 (red 7.7). Hence the dual-illuminant entry. +- The neutral row reads a\* within ±0.4 for the middle greys; white and the darkest patches + are off by up to 2 (white patch sheen, flare). + +## Which illuminants + +The two lights' colour temperatures come from the camera's own white-balance presets +(ColorBalance slots, `08-color.md` §3) at the temperatures Canon's manuals give them (Daylight +5200 K, Tungsten 3200 K, Fluorescent 4000 K), interpolated in mired on ln(R/B): sun ~4,850 K, +bulb ~2,900 K. They are filed under the nearest DNG illuminants the engine knows, **D50 +(5003 K) and Standard Illuminant A (2856 K)**, so `Camera` now carries its own illuminant +pair (spectral rows: A and D65). + +## Checks in the app + +- `spectral::tests::the_chart_measured_eos_d60_reads_its_presets_in_kelvin`: the camera's + presets, which the fit never saw, read through the matrices as Tungsten 3,264 K, Daylight + 5,603 K, Cloudy 6,613 K, Flash 7,085 K, near the locus. Canon's nominal values are 3200, 5200, + 6000 and 6000 K. Order and spacing hold; the daylight presets read 8–15 % high, as Sony's + dial does against its spectral matrices (#730). +- Real files through the headless app (As Shot): WBModes Fluorescent 4,103 K, tint +26 (green + compensation); LED-lit room, Auto 3,761 K; open shade, Auto 6,596 K; sunlit chart, Auto + 4,762 K. +- `files::tests::crw_probes_name_their_camera_from_the_ciff_records`: CRW imports now take make, + model and capture time from the decoder's CIFF records (no Exif in a CRW; the catalog had an + empty camera, so a D60 stayed on the relative white-balance scale). + +## Limits + +- One body, one bulb, one afternoon. The chart is 24 patches; a 3×3 matrix cannot follow the + sensor's departures from the CIE observer (the bulb fit's worst patches are 5–7 ΔE). +- The lights are filed under standard illuminants they only approximate (~50 K for the bulb, + ~150 K for the sun). +- The D60's look is still the per-file JPEG fit and the pooled profile; these matrices give the + white-balance scale and the fallback colour only. diff --git a/crates/crw/clean-room/README.md b/crates/crw/clean-room/README.md new file mode 100644 index 00000000..e598f751 --- /dev/null +++ b/crates/crw/clean-room/README.md @@ -0,0 +1,75 @@ +# Canon CRW clean-room derivation record + +Everything that establishes how `craft-crw` decodes a CRW, in the order it happened. The record was +produced in LightCraft (`clean-room/crw/`, branch `clean-room-crw-format-decode`, revision +`70c755e42a65661401142819278d51b0c5977f7f`) between 2026-10-12 and 2026-10-25 and moved here +**verbatim** when the decoder became this shared crate. The documents are the evidence: they are +not rewritten after the fact, so paths they mention are LightCraft's paths of that time (the +mapping is below), and `00-starting-assumptions.md` is committed once and never edited. + +## Documents + +| File | What it is | +|---|---| +| [`00-starting-assumptions.md`](00-starting-assumptions.md) | What the author believed before consulting any source; every item marked unverified. Immutable. | +| [`01-plan.md`](01-plan.md) | The derivation plan, the corpus (public CC0 files with hashes; private files by name, size and hash), the maintainer's decisions (§7) and approval. | +| [`sources.md`](sources.md) | Every URL opened and search run, with the time, what was asked and what was taken. The rules in force are at its top. | +| [`02-container.md`](02-container.md) | CIFF heap parsing from Canon's specification and ExifTool's prose pages, each fact cited. | +| [`03-payload-layout.md`](03-payload-layout.md) | The RawData payload partition established by statistics alone (no decoding): low-bits plane, 514-byte header, ff00-stuffed stream, tail. | +| [`04-derivation-log.md`](04-derivation-log.md) | The entropy-coding derivation, experiment by experiment, against the black-box reference. | +| [`06-crw-format.md`](06-crw-format.md) | **The frozen format** (revision 2) and its reopening rule. The decoder implements this document. | +| [`07-verification.md`](07-verification.md) | Full-frame exact comparison against the reference, zero tuning: 28 of 29 files, then 1,516 of 1,516 D60 files. | +| [`08-color.md`](08-color.md) | CFA phase, active area, black and white levels, white-balance records, orientation and time: what the decoder reports beyond the samples. §3's colour matrices concern LightCraft (below). | +| [`09-private-corpus.md`](09-private-corpus.md) | The maintainer's EOS D60 files and controlled captures (grey card, lens cap, white-balance presets) and what each set established. | +| [`10-chart-calibration.md`](10-chart-calibration.md) | EOS D60 colour matrices from ColorChecker captures. This is LightCraft's colour work (`crates/raw/src/chart_table.rs` there), kept here because it is part of the same record and uses the same captures. | + +There is no `05-geometry-colour.md`: the plan named it, and the material went into `08-color.md`. + +## Scripts and logs + +- `ciff_dump.py`, `p1_payload.py`, `p1b_markers.py`, `p1c_lowbits_bounds.py`, `p2_oracle_lsb.py`: + the planning and payload-statistics scripts (standard library only; run them with `python -I`). + Their public outputs are in `logs/` (`structure-*.txt`, `p1*.txt`, `verification-sweep.txt`). +- `logs/table_sel{0,1,2}.json`: **the frozen code tables**, the source of truth that + `tools/gen_tables.py` renders into `../src/tables.rs`. `table_sel0_d60class.json` is the + revision-1 selector-0 table, kept for the record. +- `logs/private-corpus.tsv`, `logs/chart-captures.tsv`: names, sizes and SHA-256 of the + maintainer's own files. No image content, metadata or thumbnails are recorded anywhere here. +- `logs/private/` (gitignored): per-file dumps of the private files, which can hold personal + metadata. Local only. + +## What stayed local + +- The Python reference model and probe harness (`plan/crw/model/` in LightCraft, gitignored), + by the maintainer's decision #4. Its findings are all in `04-derivation-log.md`. +- The reference decoder: LibRaw 0.22.2, official pre-built Windows binaries installed by the + maintainer; only `unprocessed_raw.exe` was ever run, as a black box. The source tree shipped + alongside it was never opened, read or searched. +- Canon's CIFF specification PDF (copyrighted; archive URL and SHA-256 in `sources.md`). +- The reference TIFFs (`.crw.tiff`) the verification compared against; `CRAFT_CRW_ORACLE` + points the test at a local copy. + +## Path mapping (LightCraft at the time → craft-libs) + +| In the documents | Now | +|---|---| +| `clean-room/crw/…` | `crates/crw/clean-room/…` | +| `crates/raw/src/vendor/crw.rs` | `crates/crw/src/lib.rs` | +| `crates/raw/src/vendor/crw_tables.rs` | `crates/crw/src/tables.rs` | +| `crates/raw/tests/crw_oracle.rs` | `crates/crw/tests/samples.rs` (`CRAFT_CRW_ORACLE`, formerly `LIGHTCRAFT_CRW_ORACLE`) | +| `white_from_data` (LightCraft's shared estimate) | `saturation_plateau` in `crates/crw/src/lib.rs` | +| `xtask/src/main.rs` `RAW_SAMPLES` (corpus URLs) | `crates/crw/tests/samples.json` + `scripts/fetch-crw-samples.py` | +| `plan/crw/model/gen_tables.py` | `crates/crw/clean-room/tools/gen_tables.py` | +| `docs/arw6-compression.md` (the house model for a derivation) | LightCraft's document; the method is summarised in `docs/clean-room.md` of this repository | +| `crates/raw/src/chart_table.rs`, `spectral_table.rs`, `camera_preview`, `assets/camera-profiles/` | LightCraft (colour stays in the app) | + +## Rules that were in force + +From `sources.md` and `01-plan.md`: no raw-decoder source may be opened, fetched, searched inside +or requested (dcraw, LibRaw, rawspeed, rawloader/rawler, dnglab, darktable, RawTherapee, +libopenraw, ExifTool's Perl code, including licence headers, comments and mirrors). Allowed: +published format documentation in prose, published standards, the sample files, and a reference +decoder's binary run as a black box with every probe logged. No tuning after the freeze: a +mismatch reopens the derivation explicitly, with its cause and a new freeze commit recorded +(`06-crw-format.md`, revision 2, is the one reopening). The general standard this followed is +[`docs/clean-room.md`](../../../docs/clean-room.md). diff --git a/crates/crw/clean-room/ciff_dump.py b/crates/crw/clean-room/ciff_dump.py new file mode 100644 index 00000000..9a69a81f --- /dev/null +++ b/crates/crw/clean-room/ciff_dump.py @@ -0,0 +1,197 @@ +"""CIFF/CRW container structure dump (planning phase — no raw-data decoding). + +Prints the heap tree, record entries (tag, storage class, data type, size, offset), +a few interpreted metadata records (make/model, ImageSpec, SensorInfo-ish u16 arrays, +DecoderTable u32s), and byte statistics of the RawData record. It never decodes the +compressed raw bitstream. + +Container layout implemented from published documentation only: +- Canon, "CIFF Specification on Image Data File" v1.0 r4, 1997-12-24 (archived PDF): + heap file header {byteOrder[2], headerLength u32, type[4], subType[4], version u32, + reserved1/2 u32}; heap = heap space + offsetTbl; u32 offsetTblOffset stored in the + final position of the heap; offsetTbl = numRecords u16 + entries {typeCode u16, + length u32, offset u32}; typeCode bits 15-14 storage (00 in heap space, 01 in record + entry), bits 13-11 dataType (000 byte, 001 ASCII, 010 word, 011 dword, 100 struct, + 101/110 heap record), bits 10-0 IDCode. +- ExifTool "canon_raw.html" (prose page): offsets are relative to the start of the + containing block's ValueData; a subdirectory is a complete block inside ValueData + whose length is the entry's Size; the root block runs from headerLength to EOF; + tag IDs incl. 0x2005 RawData, 0x1835 DecoderTable, 0x1031 SensorInfo, 0x080a + make/model, 0x1810 ImageInfo. + +Usage: python -I ciff_dump.py [--no-strings] file.crw [more.crw ...] +--no-strings suppresses record contents (for logs of private files: structure, +sizes, offsets and statistics only; ASCII make/model still shown, pixel data never). +""" + +import math +import struct +import sys +from collections import Counter + +# Tag names from ExifTool's published tag table (documentation page, not source). +TAG_NAMES = { + 0x0000: "NullRecord", 0x0001: "FreeBytes", + 0x0805: "CanonFileDescription", 0x080A: "CanonRawMakeModel", + 0x080B: "CanonFirmwareVersion", 0x0810: "OwnerName", + 0x0815: "CanonImageType", 0x0816: "OriginalFileName", + 0x0817: "ThumbnailFileName", + 0x100A: "TargetImageType", 0x1010: "ShutterReleaseMethod", + 0x1011: "ShutterReleaseTiming", 0x1016: "ReleaseSetting", + 0x101C: "BaseISO", 0x1028: "CanonFlashInfo", + 0x1029: "CanonFocalLength", 0x102A: "CanonShotInfo", + 0x102C: "CanonColorInfo2", 0x102D: "CanonCameraSettings", + 0x1030: "WhiteSample", 0x1031: "SensorInfo", + 0x1033: "CustomFunctions", 0x1038: "CanonAFInfo", + 0x10A9: "ColorBalance", 0x10AE: "ColorTemperature", + 0x10B4: "ColorSpace", 0x10B5: "RawJpgInfo", + 0x1803: "ImageFormat", 0x1804: "RecordID", + 0x1806: "SelfTimerTime", 0x1807: "TargetDistanceSetting", + 0x180B: "SerialNumber", 0x180E: "TimeStamp", + 0x1810: "ImageInfo", 0x1813: "FlashInfo", + 0x1814: "MeasuredEV", 0x1817: "FileNumber", + 0x1818: "ExposureInfo", 0x1834: "CanonModelID", + 0x1835: "DecoderTable", 0x183B: "SerialNumberFormat", + 0x2005: "RawData", 0x2007: "JpgFromRaw", 0x2008: "ThumbnailImage", + 0x2804: "ImageDescription", 0x2807: "CameraObject", + 0x3002: "ShootingRecord", 0x3003: "MeasuredInfo", + 0x3004: "CameraSpecification", 0x300A: "ImageProps", + 0x300B: "ExifInformation", +} + +DATATYPES = {0: "byte", 1: "ascii", 2: "word", 3: "dword", + 4: "struct", 5: "heap1", 6: "heap2", 7: "rsvd7"} + +MAX_DEPTH = 8 + + +def entropy(data): + if not data: + return 0.0 + counts = Counter(data) + n = len(data) + return -sum(c / n * math.log2(c / n) for c in counts.values()) + + +def byte_stats(data, label, out): + n = len(data) + out.append(f" {label}: {n} bytes, entropy {entropy(data):.4f} bits/byte") + if n >= 4096: + for part, name in ((data[:2048], "first 2 KiB"), (data[n // 2:n // 2 + 2048], "middle 2 KiB"), + (data[-2048:], "last 2 KiB")): + c = Counter(part) + top = ", ".join(f"{b:02x}:{cnt}" for b, cnt in c.most_common(4)) + out.append(f" {name}: entropy {entropy(part):.3f}, top bytes {top}, " + f"zeros {c.get(0, 0)}, ff {c.get(255, 0)}") + + +def dump_heap(f, blk_start, blk_len, depth, endian, out, show, path="root"): + """blk_start/blk_len: the block (ValueData + offsetTbl + trailing u32).""" + pre = " " * depth + if depth > MAX_DEPTH: + out.append(f"{pre}!! depth limit at {path}") + return + if blk_len < 6: + out.append(f"{pre}!! block too small ({blk_len}) at {path}") + return + f.seek(blk_start + blk_len - 4) + (tbl_off,) = struct.unpack(endian + "I", f.read(4)) + if tbl_off + 2 > blk_len - 4: + out.append(f"{pre}!! offsetTblOffset {tbl_off:#x} out of range (block {blk_len:#x}) at {path}") + return + f.seek(blk_start + tbl_off) + (nrec,) = struct.unpack(endian + "H", f.read(2)) + out.append(f"{pre}heap {path}: block @{blk_start:#x} len {blk_len:#x}, offsetTbl @+{tbl_off:#x}, {nrec} records") + if 2 + nrec * 10 > blk_len - tbl_off: + out.append(f"{pre}!! record table overruns block at {path}") + return + entries = [] + for i in range(nrec): + tc, length, offset = struct.unpack(endian + "HII", f.read(10)) + entries.append((tc, length, offset)) + for i, (tc, length, offset) in enumerate(entries): + stg = (tc >> 14) & 3 + dt = (tc >> 11) & 7 + tag_id = tc & 0x3FFF # dataType | IDCode, ExifTool's "TagID" + name = TAG_NAMES.get(tag_id, "?") + loc = {0: "heap", 1: "inrec", 2: "stg2?", 3: "stg3?"}[stg] + out.append(f"{pre} [{i}] tag {tc:#06x} id {tag_id:#06x} {name:<22} {DATATYPES[dt]:<6} " + f"{loc:<5} len {length:#x} off {offset:#x}") + if stg == 1: + continue # value lives in the length/offset fields themselves + if offset + length > blk_len: + out.append(f"{pre} !! record overruns block") + continue + abs_off = blk_start + offset + if dt in (5, 6): + dump_heap(f, abs_off, length, depth + 1, endian, out, show, f"{path}/{name or hex(tag_id)}") + elif tag_id == 0x2005: + f.seek(abs_off) + data = f.read(length) + byte_stats(data, "RawData", out) + elif tag_id in (0x2007, 0x2008): + f.seek(abs_off) + head = f.read(4) + out.append(f" starts with {'JPEG SOI' if head[:2] == bytes([0xFF, 0xD8]) else head.hex()}") + elif show and dt == 1 and length < 256: + f.seek(abs_off) + s = f.read(length).rstrip(b"\x00") + out.append(f"{pre} = {s!r}") + elif show and dt == 2 and length <= 64: + f.seek(abs_off) + vals = struct.unpack(endian + "H" * (length // 2), f.read(length & ~1)) + out.append(f"{pre} = u16 {vals}") + elif show and dt == 3 and length <= 64: + f.seek(abs_off) + vals = struct.unpack(endian + "I" * (length // 4), f.read(length & ~3)) + out.append(f"{pre} = u32 {vals}") + elif show and dt == 4 and tag_id == 0x1835 and length <= 64: + f.seek(abs_off) + vals = struct.unpack(endian + "I" * (length // 4), f.read(length & ~3)) + out.append(f"{pre} = DecoderTable u32 {vals}") + + +def dump(fname, show): + out = [f"== {fname}"] + with open(fname, "rb") as f: + f.seek(0, 2) + fsize = f.tell() + f.seek(0) + hdr = f.read(26) + if len(hdr) < 26: + out.append("!! file shorter than header") + return out + bo = hdr[0:2] + endian = {b"II": "<", b"MM": ">"}.get(bo) + if endian is None: + out.append(f"!! unknown byte order {bo!r} — not CIFF? first 16: {hdr[:16].hex()}") + return out + (hdr_len,) = struct.unpack(endian + "I", hdr[2:6]) + sig_type, sig_sub = hdr[6:10], hdr[10:14] + (version,) = struct.unpack(endian + "I", hdr[14:18]) + out.append(f" size {fsize}, byteOrder {bo.decode()}, headerLength {hdr_len}, " + f"type {sig_type!r} subType {sig_sub!r}, version {version:#010x}") + if sig_type != b"HEAP": + out.append("!! not a HEAP file — stopping (CHDK dump or other format?)") + return out + dump_heap(f, hdr_len, fsize - hdr_len, 0, endian, out, show) + return out + + +def main(): + args = sys.argv[1:] + show = True + if args and args[0] == "--no-strings": + show = False + args = args[1:] + for fname in args: + try: + lines = dump(fname, show) + except Exception as e: # planning tool: report, don't die + lines = [f"== {fname}", f"!! exception: {e!r}"] + print("\n".join(lines)) + print() + + +if __name__ == "__main__": + main() diff --git a/crates/crw/clean-room/logs/.gitignore b/crates/crw/clean-room/logs/.gitignore new file mode 100644 index 00000000..1f6786d8 --- /dev/null +++ b/crates/crw/clean-room/logs/.gitignore @@ -0,0 +1 @@ +private/ diff --git a/crates/crw/clean-room/logs/chart-captures.tsv b/crates/crw/clean-room/logs/chart-captures.tsv new file mode 100644 index 00000000..c8030990 --- /dev/null +++ b/crates/crw/clean-room/logs/chart-captures.tsv @@ -0,0 +1,39 @@ +# EOS D60 ColorChecker Classic captures (maintainer, local only; never committed). Names, sizes, SHA-256; no image content. Generated 2026-10-25. +file bytes sha256 +SunlightColorCard/CRW_2666.CRW 7791628 371d8e18c12801c72754dd7b277163a866afc422c8edf6ad1ea822009e4170f8 +SunlightColorCard/CRW_2667.CRW 6638578 188d625c31ab44475a6cf54a5dae00fd861e26aafc90f4b5d18a1905c0a3cef5 +SunlightColorCard/CRW_2668.CRW 6832770 1189b8dc79642c57a8c9496b5629e36a1741c545d2b0b864d6e6e20d901510c1 +SunlightColorCard/CRW_2669.CRW 7079102 baf1686b27a9e0a04bf87c0b6cf55168f398d85247f8115a0af69b5e9ea1884c +SunlightColorCard/CRW_2670.CRW 7225298 067ba3f4d3026b8009cfe4babb5e4c3b28f988d95c48024a75c6715de32afccf +SunlightColorCard/CRW_2671.CRW 7504700 cf81cbb96389624318e890b5506221d49f5fba8e2d45447d87e0f3ff3e9bdc72 +SunlightColorCard/CRW_2672.CRW 7830852 7a59446bfb1f00d6eabd0115b8e5451c34f5d3d48f483cb27608ce9508e43ee5 +SunlightColorCard/CRW_2673.CRW 8053180 feeb8f818325d6ddb4e88b5eabe4a74f4cb5e1da9f70135ae3f85938c86695dc +SunlightColorCard/CRW_2674.CRW 8312060 45e7402c888582adadb607599eae80644d657d61c9607b426c411ee306e42daf +SunlightColorCard/CRW_2675.CRW 8565950 be28d2fa049aa4158ae9e3ec49dd28eb56fcd0ffa7af060235b8bda0d89dfd0a +SunlightColorCard/CRW_2676.CRW 7795508 76f7418a82e45e1034b3adacacf4bc756cb65a0d683e5bb3631992784e654f93 +SunlightColorCard/CRW_2677.CRW 7747450 a5e7bc618e6b80ea70f9bad6a1bf41ee63eadbc51df28b82d0864a73434f7964 +SunlightColorCard/CRW_2678.CRW 6815054 28824f0f64b087488d8dc5a66a8bdf717e83a4d4e1660676d07991bf32dffb96 +SunlightColorCard/CRW_2679.CRW 7050606 b5c915fa89542265deb4fa16da3ca7ffda01b1d961903836b0059d024316673a +SunlightColorCard/CRW_2680.CRW 7231410 a5780dfe99a2df2a3ed9de1888e91baa614b2e009a2b6e0a84d442b6bf1af9fd +SunlightColorCard/CRW_2681.CRW 7438088 eb4475919c2770f816d47787656ab7ecb5396f30d2c03628c3ab167ebbc47c65 +SunlightColorCard/CRW_2682.CRW 7584262 a6fe417b90ce41b03552963dee6b21e4f02b59a5c51148edc04af7224e8c59e9 +SunlightColorCard/CRW_2683.CRW 7811474 5402ad40e2689d78ade083d8855ecced5caea9368b9d77ecc4a61a16dc55aa36 +SunlightColorCard/CRW_2684.CRW 8043372 bbef5da457c0b6d6c45c74a99f3e318e6c80a7d3a0ad5c7cb7e13d24205f64fa +SunlightColorCard/CRW_2685.CRW 8242344 8d68d592a5c8933f0f35630114fe7017be808b007867beaf1a8d6d2bd1441f70 +SunlightColorCard/CRW_2686.CRW 8462380 79f7db32b18a834d5408500185c65d316387d1440aa61e26a11be85883fcc382 +SunlightColorCard/CRW_2687.CRW 7690732 0333d8d18390e77128f186239e6de71c0432ab6ee9669a71c210255e4d915d19 +TungstonColorChart/CRW_2688.CRW 6269390 d9c4334edfe3636a8a7e69f29c1d1e89f820b7b704697684d7996faee41f5d43 +TungstonColorChart/CRW_2689.CRW 6001348 1e8b47645d76a71fd9fb4111c4dce805d6dcd41ac82e6b4ce52debbf71565455 +TungstonColorChart/CRW_2690.CRW 6114056 f63fe49a9ed13af33ddb8d8ed8d75de4e304a3bc2c1f1d4275ba7edbc4f459f3 +TungstonColorChart/CRW_2691.CRW 6299730 ad8a2844c00965fb6a12e8fc85a6e8bfbdf1924e6d1a68b55472701cb8ffa115 +TungstonColorChart/CRW_2696.CRW 6169684 105ee515c94da78dedc1f8d777982c4a1e62781b160428524fbe922f62b3ef5c +TungstonColorChart/CRW_2697.CRW 5565982 327eebf702185127feb8a5f4bfac5d7ebad5532f7417b302997b19b97d7ff4a8 +TungstonColorChart/CRW_2698.CRW 5709092 f0fca20f8dcd51046e53bebf9fb9f624677ee674ad739622cdb2578977669705 +TungstonColorChart/CRW_2699.CRW 5873492 57724a1ff507cf4e78c80a34cd4e0b77eac397322228d32e5215fc3869853dd1 +TungstonColorChart/CRW_2700.CRW 5988146 6c97ae2c72114034e2fbe6aed0ef97f475696e43b56ed62273eccbfbba4cbaa0 +TungstonColorChart/CRW_2701.CRW 6202756 c4032e46b753a718d8748e269a9da5d124891d07912542368ea867529143aa62 +TungstonColorChart/CRW_2702.CRW 6409582 89b65c1c07e0685841046db85923b0bf80e747fed141ddf0335ba12dc75f3f3c +TungstonColorChart/CRW_2703.CRW 6407270 50179d0a330c967a74d76bc8342eea1052689c5c544aa1a913769b241ac6f736 +TungstonColorChart/CRW_2704.CRW 6483154 6bac4d8229a8d0fff1445d212ca63e53911dc7bda988743215009257b87787fa +TungstonColorChart/CRW_2705.CRW 7170488 629f94131e7f598964d2635e3039a20c441563201c4f3cbf177d83ad6b061f78 +TungstonColorChart/CRW_2706.CRW 6383026 67657e7a21f6fd8580012fccd2210afccf285e747bc4d39b8ea775811528ce83 diff --git a/crates/crw/clean-room/logs/p1-payload-public.txt b/crates/crw/clean-room/logs/p1-payload-public.txt new file mode 100644 index 00000000..a528740c --- /dev/null +++ b/crates/crw/clean-room/logs/p1-payload-public.txt @@ -0,0 +1,195 @@ +== ../../corpus/crw/crw-canon-eos-10d-a.crw + raw 3152x2068 S=7134454 tbl=0 f1=0 f2=514 L=5504868 S-L=1629586 wh/4=1629584 S-L-wh/4=2 + leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) + last 8 bytes: 95ad8fb002120000 + prefix hypothesis region [0 : wh/4) as 2-bit rows: + prefix: 2068 rows of 788 B, leading all-zero rows: 7 + rows 0..top: mean 85.37 ent 5.190 + top..top+32: mean 127.88 ent 7.993 + middle 32: mean 126.86 ent 7.992 + last 32: mean 127.02 ent 7.993 + suffix hypothesis region [L : L+wh/4) as 2-bit rows: + suffix: 2068 rows of 788 B, leading all-zero rows: 0 + rows 0..top: mean 132.23 ent 7.944 + top..top+32: mean 131.83 ent 7.953 + middle 32: mean 128.56 ent 7.959 + last 32: mean 126.95 ent 7.956 + compressed-candidate stats: + after-prefix [wh/4 : wh/4+L): entropy 7.9684 (first MiB), 7.9644 (last MiB) + from-0 [0 : L): entropy 7.9960 (first MiB), 7.9530 (last MiB) + stuffing check: ff count 13614 (0.2473%), ff00 digrams 13614, expected-if-independent 64.4 + +== ../../corpus/crw/crw-canon-eos-10d-b.crw + raw 3152x2068 S=7244858 tbl=2 f1=0 f2=514 L=5615272 S-L=1629586 wh/4=1629584 S-L-wh/4=2 + leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) + last 8 bytes: 0c849b8b12b70000 + prefix hypothesis region [0 : wh/4) as 2-bit rows: + prefix: 2068 rows of 788 B, leading all-zero rows: 7 + rows 0..top: mean 64.32 ent 5.152 + top..top+32: mean 121.98 ent 7.958 + middle 32: mean 126.50 ent 7.992 + last 32: mean 127.17 ent 7.993 + suffix hypothesis region [L : L+wh/4) as 2-bit rows: + suffix: 2068 rows of 788 B, leading all-zero rows: 0 + rows 0..top: mean 117.92 ent 7.937 + top..top+32: mean 117.33 ent 7.941 + middle 32: mean 114.53 ent 7.905 + last 32: mean 117.63 ent 7.904 + compressed-candidate stats: + after-prefix [wh/4 : wh/4+L): entropy 7.9877 (first MiB), 7.9150 (last MiB) + from-0 [0 : L): entropy 7.9898 (first MiB), 7.9220 (last MiB) + stuffing check: ff count 12542 (0.2234%), ff00 digrams 12542, expected-if-independent 97.9 + +== ../../corpus/crw/crw-canon-eos-10d-c.crw + raw 3152x2068 S=7455524 tbl=2 f1=0 f2=514 L=5825938 S-L=1629586 wh/4=1629584 S-L-wh/4=2 + leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) + last 8 bytes: 9c12ce7bde990000 + prefix hypothesis region [0 : wh/4) as 2-bit rows: + prefix: 2068 rows of 788 B, leading all-zero rows: 7 + rows 0..top: mean 78.12 ent 5.144 + top..top+32: mean 126.89 ent 7.992 + middle 32: mean 127.89 ent 7.993 + last 32: mean 127.04 ent 7.991 + suffix hypothesis region [L : L+wh/4) as 2-bit rows: + suffix: 2068 rows of 788 B, leading all-zero rows: 0 + rows 0..top: mean 119.60 ent 7.946 + top..top+32: mean 117.81 ent 7.946 + middle 32: mean 104.15 ent 7.770 + last 32: mean 114.88 ent 7.856 + compressed-candidate stats: + after-prefix [wh/4 : wh/4+L): entropy 7.8675 (first MiB), 7.8315 (last MiB) + from-0 [0 : L): entropy 7.9961 (first MiB), 7.9712 (last MiB) + stuffing check: ff count 6993 (0.1200%), ff00 digrams 6993, expected-if-independent 47.5 + +== ../../corpus/crw/crw-canon-eos-300d-a.crw + raw 3152x2068 S=6929638 tbl=0 f1=0 f2=514 L=5300052 S-L=1629586 wh/4=1629584 S-L-wh/4=2 + leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) + last 8 bytes: e438ce510ad90000 + prefix hypothesis region [0 : wh/4) as 2-bit rows: + prefix: 2068 rows of 788 B, leading all-zero rows: 7 + rows 0..top: mean 61.27 ent 5.230 + top..top+32: mean 127.52 ent 7.994 + middle 32: mean 127.63 ent 7.993 + last 32: mean 127.31 ent 7.992 + suffix hypothesis region [L : L+wh/4) as 2-bit rows: + suffix: 2068 rows of 788 B, leading all-zero rows: 0 + rows 0..top: mean 132.59 ent 7.950 + top..top+32: mean 130.96 ent 7.963 + middle 32: mean 131.10 ent 7.958 + last 32: mean 134.35 ent 7.947 + compressed-candidate stats: + after-prefix [wh/4 : wh/4+L): entropy 7.9405 (first MiB), 7.9600 (last MiB) + from-0 [0 : L): entropy 7.9961 (first MiB), 7.9634 (last MiB) + stuffing check: ff count 17631 (0.3327%), ff00 digrams 17631, expected-if-independent 99.9 + +== ../../corpus/crw/crw-canon-eos-300d-b.crw + raw 3152x2068 S=7206250 tbl=2 f1=0 f2=514 L=5576664 S-L=1629586 wh/4=1629584 S-L-wh/4=2 + leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) + last 8 bytes: f8ff00eefe7f0000 + prefix hypothesis region [0 : wh/4) as 2-bit rows: + prefix: 2068 rows of 788 B, leading all-zero rows: 7 + rows 0..top: mean 61.49 ent 5.196 + top..top+32: mean 127.65 ent 7.989 + middle 32: mean 112.85 ent 7.534 + last 32: mean 119.27 ent 7.834 + suffix hypothesis region [L : L+wh/4) as 2-bit rows: + suffix: 2068 rows of 788 B, leading all-zero rows: 0 + rows 0..top: mean 124.00 ent 7.968 + top..top+32: mean 127.27 ent 7.978 + middle 32: mean 127.36 ent 7.985 + last 32: mean 130.81 ent 7.973 + compressed-candidate stats: + after-prefix [wh/4 : wh/4+L): entropy 7.9774 (first MiB), 7.9887 (last MiB) + from-0 [0 : L): entropy 7.8118 (first MiB), 7.9812 (last MiB) + stuffing check: ff count 28275 (0.5070%), ff00 digrams 28275, expected-if-independent 265.2 + +== ../../corpus/crw/crw-canon-eos-d30.crw + raw 2224x1456 S=2824648 tbl=1 f1=0 f2=514 L=2015110 S-L=809538 wh/4=809536 S-L-wh/4=2 + leading zero run: 0 bytes (= 0.00 rows of 2-bit LSBs) + last 8 bytes: fadfd6feb7f50000 + prefix hypothesis region [0 : wh/4) as 2-bit rows: + prefix: 1456 rows of 556 B, leading all-zero rows: 0 + rows 0..top: mean 139.68 ent 7.911 + top..top+32: mean 127.19 ent 7.990 + middle 32: mean 127.59 ent 7.988 + last 32: mean 131.21 ent 7.924 + suffix hypothesis region [L : L+wh/4) as 2-bit rows: + suffix: 1456 rows of 556 B, leading all-zero rows: 0 + rows 0..top: mean 123.77 ent 7.922 + top..top+32: mean 123.76 ent 7.936 + middle 32: mean 116.78 ent 7.895 + last 32: mean 116.20 ent 7.890 + compressed-candidate stats: + after-prefix [wh/4 : wh/4+L): entropy 7.9343 (first MiB), 7.9341 (last MiB) + from-0 [0 : L): entropy 7.9949 (first MiB), 7.9410 (last MiB) + stuffing check: ff count 2576 (0.1278%), ff00 digrams 2576, expected-if-independent 14.1 + +== ../../corpus/crw/crw-canon-eos-d60.crw + raw 3152x2068 S=6694146 tbl=0 f1=0 f2=514 L=5064559 S-L=1629587 wh/4=1629584 S-L-wh/4=3 + leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) + last 8 bytes: 9385933317ff0000 + prefix hypothesis region [0 : wh/4) as 2-bit rows: + prefix: 2068 rows of 788 B, leading all-zero rows: 7 + rows 0..top: mean 55.84 ent 5.113 + top..top+32: mean 127.09 ent 7.992 + middle 32: mean 127.69 ent 7.993 + last 32: mean 127.98 ent 7.992 + suffix hypothesis region [L : L+wh/4) as 2-bit rows: + suffix: 2068 rows of 788 B, leading all-zero rows: 0 + rows 0..top: mean 130.50 ent 7.942 + top..top+32: mean 131.88 ent 7.949 + middle 32: mean 131.47 ent 7.951 + last 32: mean 126.89 ent 7.967 + compressed-candidate stats: + after-prefix [wh/4 : wh/4+L): entropy 7.9003 (first MiB), 7.9694 (last MiB) + from-0 [0 : L): entropy 7.9960 (first MiB), 7.9672 (last MiB) + stuffing check: ff count 14189 (0.2802%), ff00 digrams 14189, expected-if-independent 76.1 + +== ../../corpus/crw/crw-canon-eos-digital-rebel.crw + raw 3152x2068 S=6406164 tbl=0 f1=0 f2=514 L=4776577 S-L=1629587 wh/4=1629584 S-L-wh/4=3 + leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) + last 8 bytes: 191093af87ff0000 + prefix hypothesis region [0 : wh/4) as 2-bit rows: + prefix: 2068 rows of 788 B, leading all-zero rows: 7 + rows 0..top: mean 58.54 ent 3.603 + top..top+32: mean 144.80 ent 5.469 + middle 32: mean 163.93 ent 4.986 + last 32: mean 154.41 ent 5.106 + suffix hypothesis region [L : L+wh/4) as 2-bit rows: + suffix: 2068 rows of 788 B, leading all-zero rows: 0 + rows 0..top: mean 128.96 ent 7.960 + top..top+32: mean 129.35 ent 7.966 + middle 32: mean 127.02 ent 7.922 + last 32: mean 122.47 ent 7.926 + compressed-candidate stats: + after-prefix [wh/4 : wh/4+L): entropy 7.9244 (first MiB), 7.9355 (last MiB) + from-0 [0 : L): entropy 5.9926 (first MiB), 7.9542 (last MiB) + stuffing check: ff count 6718 (0.1406%), ff00 digrams 6718, expected-if-independent 36.3 + +== ../../corpus/crw/crw-canon-powershot-g1.crw + raw 2144x1560 S=2257354 tbl=1 f1=0 f2=514 L=2257351 S-L=3 wh/4=836160 S-L-wh/4=-836157 + leading zero run: 514 bytes (= 0.96 rows of 2-bit LSBs) + last 8 bytes: 34e0702c2fffd900 + no wh/4 region (PowerShot class): compressed assumed [0 : L) + entropy first MiB 7.9226, last MiB 7.9396 + stuffing check: ff count 2676 (0.1185%), ff00 digrams 2676, expected-if-independent 11.4 + tail region [L : S): 3 bytes, hex ffd900 + +== ../../corpus/crw/crw-canon-powershot-g5.crw + raw 2672x1968 S=4861806 tbl=1 f1=0 f2=514 L=4861804 S-L=2 wh/4=1314624 S-L-wh/4=-1314622 + leading zero run: 514 bytes (= 0.77 rows of 2-bit LSBs) + last 8 bytes: 71928822fd7fffd9 + no wh/4 region (PowerShot class): compressed assumed [0 : L) + entropy first MiB 7.9958, last MiB 7.9087 + stuffing check: ff count 9974 (0.2052%), ff00 digrams 4958, expected-if-independent 54.4 + tail region [L : S): 2 bytes, hex ffd9 + +== ../../corpus/crw/crw-canon-powershot-g6.crw + raw 3160x2344 S=7313156 tbl=0 f1=0 f2=514 L=7313154 S-L=2 wh/4=1851760 S-L-wh/4=-1851758 + leading zero run: 514 bytes (= 0.65 rows of 2-bit LSBs) + last 8 bytes: dfbf7efdfbf7ffd9 + no wh/4 region (PowerShot class): compressed assumed [0 : L) + entropy first MiB 7.9995, last MiB 7.9596 + stuffing check: ff count 18811 (0.2572%), ff00 digrams 10815, expected-if-independent 103.5 + tail region [L : S): 2 bytes, hex ffd9 + diff --git a/crates/crw/clean-room/logs/p1b-markers-public.txt b/crates/crw/clean-room/logs/p1b-markers-public.txt new file mode 100644 index 00000000..d0442666 --- /dev/null +++ b/crates/crw/clean-room/logs/p1b-markers-public.txt @@ -0,0 +1,109 @@ +== ../../corpus/crw/crw-canon-eos-10d-a.crw + raw 3152x2068 tbl=0 f2=514 L=5504868 lowbits=yes + ff followed by: 00:13614 + markers: none (pure stuffed entropy data) + comp[0:24] : 000000000000000000000000000000000000000000000000 + comp[-8:] : 9df595ad8fb00212 + raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 + raw[-8:] : 95ad8fb002120000 + low[-8:] : fc3590b2d228d3dd (end of low-bits region) + +== ../../corpus/crw/crw-canon-eos-10d-b.crw + raw 3152x2068 tbl=2 f2=514 L=5615272 lowbits=yes + ff followed by: 00:12542 + markers: none (pure stuffed entropy data) + comp[0:24] : 000000000000000000000000000000000000000000000000 + comp[-8:] : 38650c849b8b12b7 + raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 + raw[-8:] : 0c849b8b12b70000 + low[-8:] : c6371ccbb9d8dd2c (end of low-bits region) + +== ../../corpus/crw/crw-canon-eos-10d-c.crw + raw 3152x2068 tbl=2 f2=514 L=5825938 lowbits=yes + ff followed by: 00:6993 + markers: none (pure stuffed entropy data) + comp[0:24] : 000000000000000000000000000000000000000000000000 + comp[-8:] : e7389c12ce7bde99 + raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 + raw[-8:] : 9c12ce7bde990000 + low[-8:] : eb6d0fe72354f02c (end of low-bits region) + +== ../../corpus/crw/crw-canon-eos-300d-a.crw + raw 3152x2068 tbl=0 f2=514 L=5300052 lowbits=yes + ff followed by: 00:17631 + markers: none (pure stuffed entropy data) + comp[0:24] : 000000000000000000000000000000000000000000000000 + comp[-8:] : 22c9e438ce510ad9 + raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 + raw[-8:] : e438ce510ad90000 + low[-8:] : a93331aa85cdb788 (end of low-bits region) + +== ../../corpus/crw/crw-canon-eos-300d-b.crw + raw 3152x2068 tbl=2 f2=514 L=5576664 lowbits=yes + ff followed by: 00:28275 + markers: none (pure stuffed entropy data) + comp[0:24] : 000000000000000000000000000000000000000000000000 + comp[-8:] : bbe9f8ff00eefe7f + raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 + raw[-8:] : f8ff00eefe7f0000 + low[-8:] : bbb7b7b7bbb777c7 (end of low-bits region) + +== ../../corpus/crw/crw-canon-eos-d30.crw + raw 2224x1456 tbl=1 f2=514 L=2015110 lowbits=yes + ff followed by: 00:2576 + markers: none (pure stuffed entropy data) + comp[0:24] : 000000000000000000000000000000000000000000000000 + comp[-8:] : 7f5bfadfd6feb7f5 + raw[504:544] : a57fba0edb9db8e63afc7b4eaa3c6dae0efbabaf3fcc8bbee8f93db67aabbee9ce27ba68a9bfaa26 + raw[-8:] : fadfd6feb7f50000 + low[-8:] : ffffffffffffffff (end of low-bits region) + +== ../../corpus/crw/crw-canon-eos-d60.crw + raw 3152x2068 tbl=0 f2=514 L=5064559 lowbits=yes + ff followed by: 00:14189 + markers: none (pure stuffed entropy data) + comp[0:24] : 000000000000000000000000000000000000000000000000 + comp[-8:] : 51f18a9385933317 + raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 + raw[-8:] : 9385933317ff0000 + low[-8:] : 82961c457559780e (end of low-bits region) + +== ../../corpus/crw/crw-canon-eos-digital-rebel.crw + raw 3152x2068 tbl=0 f2=514 L=4776577 lowbits=yes + ff followed by: 00:6718 + markers: none (pure stuffed entropy data) + comp[0:24] : 000000000000000000000000000000000000000000000000 + comp[-8:] : fc86e6191093af87 + raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 + raw[-8:] : 191093af87ff0000 + low[-8:] : 66ee4e6c4cc6cece (end of low-bits region) + +== ../../corpus/crw/crw-canon-powershot-g1.crw + raw 2144x1560 tbl=1 f2=514 L=2257351 lowbits=no + ff followed by: 00:2676 + markers: none (pure stuffed entropy data) + comp[0:24] : 000000000000000000000000000000000000000000000000 + comp[-8:] : 3efec534e0702c2f + raw[504:544] : 00000000000000000000f83df43825313b62bf97ebdc24f476407e600ce3f30b9211928d5f8de276 + raw[-8:] : 34e0702c2fffd900 + +== ../../corpus/crw/crw-canon-powershot-g5.crw + raw 2672x1968 tbl=1 f2=514 L=4861804 lowbits=no + ff followed by: 00:4958, fa:40, e7:35, d8:35, 24:33, c0:32, d3:31, 34:30 + markers: ff01:20, ff02:16, ff03:24, ff04:23, ff05:16, ff06:23, ff07:23, ff08:14, ff09:16, ff0a:21, ff0b:22, ff0c:16, ff0d:22, ff0e:26, ff0f:24, ff10:27, ff11:25, ff12:17, ff13:30, ff14:21, ff15:13, ff16:15, ff17:22, ff18:19, ff19:22, ff1a:18, ff1b:24, ff1c:18, ff1d:9, ff1e:20, ff1f:20, ff20:22, ff21:18, ff22:20, ff23:19, ff24:33, ff25:12, ff26:15, ff27:24, ff28:22, ff29:27, ff2a:18, ff2b:21, ff2c:14, ff2d:23, ff2e:15, ff2f:17, ff30:16, ff31:18, ff32:22, ff33:17, ff34:30, ff35:15, ff36:20, ff37:19, ff38:21, ff39:18, ff3a:16, ff3b:19, ff3c:16, ff3d:27, ff3e:16, ff3f:19, ff40:22, ff41:20, ff42:23, ff43:16, ff44:13, ff45:15, ff46:25, ff47:12, ff48:10, ff49:19, ff4a:18, ff4b:17, ff4c:25, ff4d:17, ff4e:25, ff4f:17, ff50:27, ff51:24, ff52:20, ff53:17, ff54:17, ff55:20, ff56:21, ff57:19, ff58:18, ff59:17, ff5a:13, ff5b:14, ff5c:18, ff5d:27, ff5e:21, ff5f:22, ff60:18, ff61:15, ff62:15, ff63:15, ff64:24, ff65:25, ff66:15, ff67:13, ff68:19, ff69:16, ff6a:13, ff6b:17, ff6c:26, ff6d:24, ff6e:18, ff6f:12, ff70:22, ff71:13, ff72:17, ff73:18, ff74:16, ff75:13, ff76:24, ff77:24, ff78:19, ff79:13, ff7a:26, ff7b:23, ff7c:28, ff7d:14, ff7e:15, ff7f:19, ff80:22, ff81:20, ff82:17, ff83:23, ff84:15, ff85:22, ff86:16, ff87:24, ff88:24, ff89:25, ff8a:18, ff8b:13, ff8c:18, ff8d:18, ff8e:23, ff8f:15, ff90:20, ff91:19, ff92:17, ff93:17, ff94:20, ff95:10, ff96:14, ff97:18, ff98:15, ff99:14, ff9a:22, ff9b:20, ff9c:20, ff9d:21, ff9e:21, ff9f:18, ffa0:21, ffa1:14, ffa2:20, ffa3:18, ffa4:21, ffa5:22, ffa6:24, ffa7:10, ffa8:16, ffa9:22, ffaa:22, ffab:16, ffac:25, ffad:22, ffae:28, ffaf:10, ffb0:18, ffb1:20, ffb2:19, ffb3:20, ffb4:20, ffb5:16, ffb6:18, ffb7:18, ffb8:19, ffb9:25, ffba:24, ffbb:12, ffbc:21, ffbd:12, ffbe:28, ffbf:22, ffc0:32, ffc1:21, ffc2:20, ffc3:14, ffc4:25, ffc5:14, ffc6:24, ffc7:18, ffc8:19, ffc9:17, ffca:23, ffcb:16, ffcc:20, ffcd:18, ffce:21, ffcf:17, ffd0:18, ffd1:24, ffd2:12, ffd3:31, ffd4:23, ffd5:16, ffd6:24, ffd7:23, ffd8:35, ffd9:17, ffda:15, ffdb:30, ffdc:16, ffdd:21, ffde:17, ffdf:12, ffe0:24, ffe1:22, ffe2:16, ffe3:23, ffe4:25, ffe5:24, ffe6:13, ffe7:35, ffe8:19, ffe9:14, ffea:25, ffeb:13, ffec:24, ffed:26, ffee:20, ffef:17, fff0:24, fff1:17, fff2:20, fff3:17, fff4:19, fff5:23, fff6:19, fff7:18, fff8:21, fff9:17, fffa:40, fffb:12, fffc:18, fffd:30, fffe:25, ffff:14 + first markers at: [('0x2b9', '0x43'), ('0x36a', '0x9d'), ('0x669', '0x95'), ('0x6a3', '0x37'), ('0x909', '0x29'), ('0x93a', '0xad'), ('0x9f9', '0x7f'), ('0xa55', '0x68'), ('0xc5f', '0x47'), ('0xf97', '0x65'), ('0x1068', '0x3a'), ('0x1210', '0x51')] + comp[0:24] : 000000000000000000000000000000000000000000000000 + comp[-8:] : 426771928822fd7f + raw[504:544] : 000000000000000000009e16f4366ff2c547b59c5e132991784465d6f539fd37876356e9155982fa + raw[-8:] : 71928822fd7fffd9 + +== ../../corpus/crw/crw-canon-powershot-g6.crw + raw 3160x2344 tbl=0 f2=514 L=7313154 lowbits=no + ff followed by: 00:10815, ff:68, 37:61, 77:60, 7f:59, bb:58, ee:58, 33:56 + markers: ff01:24, ff02:24, ff03:31, ff04:17, ff05:39, ff06:29, ff07:31, ff08:23, ff09:31, ff0a:32, ff0b:26, ff0c:33, ff0d:27, ff0e:34, ff0f:29, ff10:27, ff11:22, ff12:33, ff13:28, ff14:27, ff15:35, ff16:24, ff17:31, ff18:18, ff19:26, ff1a:28, ff1b:28, ff1c:24, ff1d:21, ff1e:30, ff1f:29, ff20:35, ff21:38, ff22:31, ff23:37, ff24:32, ff25:34, ff26:18, ff27:22, ff28:30, ff29:29, ff2a:25, ff2b:23, ff2c:26, ff2d:36, ff2e:44, ff2f:27, ff30:38, ff31:38, ff32:27, ff33:56, ff34:19, ff35:27, ff36:31, ff37:61, ff38:27, ff39:26, ff3a:25, ff3b:41, ff3c:23, ff3d:32, ff3e:24, ff3f:51, ff40:39, ff41:34, ff42:21, ff43:21, ff44:26, ff45:35, ff46:25, ff47:33, ff48:27, ff49:35, ff4a:27, ff4b:31, ff4c:19, ff4d:25, ff4e:34, ff4f:31, ff50:22, ff51:23, ff52:17, ff53:35, ff54:29, ff55:28, ff56:29, ff57:30, ff58:21, ff59:30, ff5a:25, ff5b:35, ff5c:19, ff5d:33, ff5e:25, ff5f:26, ff60:32, ff61:35, ff62:31, ff63:26, ff64:29, ff65:20, ff66:32, ff67:23, ff68:38, ff69:27, ff6a:29, ff6b:27, ff6c:34, ff6d:32, ff6e:43, ff6f:24, ff70:22, ff71:24, ff72:34, ff73:50, ff74:30, ff75:26, ff76:28, ff77:60, ff78:36, ff79:21, ff7a:35, ff7b:47, ff7c:31, ff7d:27, ff7e:27, ff7f:59, ff80:35, ff81:35, ff82:33, ff83:22, ff84:27, ff85:20, ff86:23, ff87:24, ff88:21, ff89:24, ff8a:26, ff8b:31, ff8c:18, ff8d:23, ff8e:27, ff8f:41, ff90:22, ff91:21, ff92:26, ff93:28, ff94:32, ff95:27, ff96:27, ff97:29, ff98:33, ff99:29, ff9a:34, ff9b:32, ff9c:25, ff9d:33, ff9e:18, ff9f:28, ffa0:31, ffa1:25, ffa2:22, ffa3:29, ffa4:19, ffa5:20, ffa6:20, ffa7:32, ffa8:43, ffa9:33, ffaa:33, ffab:34, ffac:18, ffad:32, ffae:26, ffaf:16, ffb0:31, ffb1:23, ffb2:35, ffb3:50, ffb4:38, ffb5:29, ffb6:29, ffb7:43, ffb8:28, ffb9:22, ffba:34, ffbb:58, ffbc:32, ffbd:23, ffbe:34, ffbf:37, ffc0:24, ffc1:32, ffc2:32, ffc3:30, ffc4:28, ffc5:22, ffc6:28, ffc7:29, ffc8:31, ffc9:30, ffca:26, ffcb:32, ffcc:43, ffcd:45, ffce:56, ffcf:42, ffd0:29, ffd1:23, ffd2:24, ffd3:41, ffd4:22, ffd5:29, ffd6:31, ffd7:28, ffd8:27, ffd9:27, ffda:30, ffdb:25, ffdc:49, ffdd:42, ffde:47, ffdf:54, ffe0:25, ffe1:31, ffe2:32, ffe3:32, ffe4:37, ffe5:36, ffe6:36, ffe7:30, ffe8:31, ffe9:44, ffea:24, ffeb:31, ffec:47, ffed:49, ffee:58, ffef:45, fff0:24, fff1:24, fff2:30, fff3:45, fff4:36, fff5:27, fff6:36, fff7:48, fff8:29, fff9:25, fffa:38, fffb:49, fffc:48, fffd:42, fffe:37, ffff:68 + first markers at: [('0x307', '0xd2'), ('0x397', '0xb1'), ('0x428', '0xb0'), ('0x60f', '0xa0'), ('0x66c', '0x23'), ('0x691', '0x8d'), ('0x69e', '0x69'), ('0x79c', '0xd6'), ('0x827', '0x40'), ('0x8f4', '0x6a'), ('0x9fe', '0xa5'), ('0xa09', '0x88')] + comp[0:24] : 000000000000000000000000000000000000000000000000 + comp[-8:] : f7efdfbf7efdfbf7 + raw[504:544] : 00000000000000000000de521366dccd8cfe291a1c468992a578c2b0ad703f581c0b0667c07d2499 + raw[-8:] : dfbf7efdfbf7ffd9 + diff --git a/crates/crw/clean-room/logs/p1c-bounds-public.txt b/crates/crw/clean-room/logs/p1c-bounds-public.txt new file mode 100644 index 00000000..64aa1d69 --- /dev/null +++ b/crates/crw/clean-room/logs/p1c-bounds-public.txt @@ -0,0 +1,11 @@ +crw-canon-eos-10d-a.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 6241 total, 6241 in lowbits, 0 in entropy (expect 0) +crw-canon-eos-10d-b.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 6503 total, 6503 in lowbits, 0 in entropy (expect 0) +crw-canon-eos-10d-c.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 6139 total, 6139 in lowbits, 0 in entropy (expect 0) +crw-canon-eos-300d-a.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 6418 total, 6418 in lowbits, 0 in entropy (expect 0) +crw-canon-eos-300d-b.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 7180 total, 7180 in lowbits, 0 in entropy (expect 0) +crw-canon-eos-d30.crw EOS layout: lowbits [0:809536) + zeros(514) + entropy; non-stuffed ff: 3450 total, 3450 in lowbits, 0 in entropy (expect 0) +crw-canon-eos-d60.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 6419 total, 6419 in lowbits, 0 in entropy (expect 0) +crw-canon-eos-digital-rebel.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 23394 total, 23394 in lowbits, 0 in entropy (expect 0) +crw-canon-powershot-g1.crw PS 10-bit layout: zeros(514) + entropy; no non-stuffed ff (pure stuffed stream) +crw-canon-powershot-g5.crw PS 12-bit layout: zeros(514) + lowbits [514:1315138) + entropy; all 5002 non-stuffed ff inside lowbits; first 697, last 1315128 +crw-canon-powershot-g6.crw PS 12-bit layout: zeros(514) + lowbits [514:1852274) + entropy; all 7928 non-stuffed ff inside lowbits; first 775, last 1852264 diff --git a/crates/crw/clean-room/logs/private-corpus.tsv b/crates/crw/clean-room/logs/private-corpus.tsv new file mode 100644 index 00000000..9e64047d --- /dev/null +++ b/crates/crw/clean-room/logs/private-corpus.tsv @@ -0,0 +1,1564 @@ +# Private EOS D60 corpus (maintainer's files in ../RawImages, local only; never committed). +# Names, sizes and SHA-256 only: no image content. Generated 2026-10-14. +file bytes sha256 +CRW_0476.crw 6906290 7966aeb5abf3abe80868b8cb1606d671112c42f44d877d3ff79305a52558cacc +CRW_0477.crw 6755682 6c7789e91dff31f16bc1e1e8596212124c96d90363181096d3e486ee0aeed1a9 +CRW_0478.crw 5708018 6d07ef616f4b1c7c3025e7af315bb7fb4173ddefea45e2e52d5a4d28e9f2851f +CRW_0479.crw 6561756 a784769d3c1a95474ac314fa51c944f7eff928995af9badfacae95959419b667 +CRW_0480.crw 6001400 7362b296bee391b421c8b3cf353620fa5f355eff7542047333ea528a6e2b5571 +CRW_0481.crw 6273782 9a756b74f70494d2327981f1e8597da567f4d67b30b53763432c2a66d19269f7 +CRW_0482.crw 7817610 f1ea8769ec11e9476e13fbd31a9d32ca1a8f239974ef84e69fb6fc976407710a +CRW_0483.crw 8420474 bcdfda798c25924a8730f0c3eb2c7af931b12a3e88f64c52e8d26c45e43b3c58 +CRW_0484.crw 6964206 9bf387edfb25c577098b325c727feab22d17163d6e893b670813e81b3a450524 +CRW_0485.crw 6970050 f7dba6fed7d6c8c9b58c21e40c3955b5e8952da478351ea878c23d4c7a3c5428 +CRW_0486.crw 8055488 bf0876f5ba55b2bd120f9588b91f9d4caad54ab73315e542114bf738459d8b3b +CRW_0487.crw 7991184 d137c4bf56c78675f7bf7fd63faeea923ff6657de8ef8dfbc6de8c38f0df2f06 +CRW_0488.crw 7676308 8a8766ee5574cfa54c502e556f2ca90906173b8da958b0d781a7afe02cbf5b60 +CRW_0489.crw 6511330 532927313faf0ed691ce380b3eb6dab76fd7100a6a5c25b4b15e68859bb19bdc +CRW_0490.crw 6375618 e4b199c012fe3e5c8124dab66d6bf6aa0f83c965443d0f755d0c00c18a63751e +CRW_0491.crw 8501124 f729f4ab144a44c35d1a70df57297d2f6ba5238dadc280863451a479f3480954 +CRW_0492.crw 8121046 a7bf6f820cfbbc34abc685f41d4223ac50fc12b48f260595fc5569c0a95a57f6 +CRW_0493.crw 8405518 a8662e83f6fea7dc09fa8b29467988499bc2d49af87297888433ce747e7bb331 +CRW_0494.crw 8065722 f16b4bf927da07652f842220fce68edbf612d31452175a9031f7837079962829 +CRW_0495.crw 8812158 66d5a04560eaf262fe8ac4a6cfbe195c49e906a703f882f909d8f1dda2089ba2 +CRW_0496.crw 8669432 eeb26cd1257c6b01c1f885b7de85fe5d05d0740bede6e1d1761b8c3abbd98ffe +CRW_0497.crw 7228210 56677993eae948a3f1623ec151a7504de339ebcaf2d0f921e6152c5270a797ea +CRW_0498.crw 7220792 e600a48cd5519209e04dc419f8e6022b239adcbbf2bebad1097c25323a010a69 +CRW_0499.crw 6562642 bdd340851286f251147d500a43849f3ea545dac2a9fbb7cb4c7b7f17ce426e77 +CRW_0500.crw 6456426 2cbfa48e8280401b50dcacafd0d51f7288ef871e2866d4c5a81f1eacbd7a6a50 +CRW_0501.crw 6543378 49e235fcc63dfd19e7689184f6728eb73ba5431510bcc4f744d170432b16c023 +CRW_0502.crw 6892486 dd9c874f03b5e104ea451b748bf3b3e72358b8af6be9ad9ae5f75d3eb197df43 +CRW_0503.crw 7255972 92afd63740abfc09bd9d96c084b5b6c3d29600a15a50e5b69354af131af71d8c +CRW_0504.crw 6866524 2d95a81e2a5b9df3e1052806cafdf8d5818b377ddd2e68e547a1e752e95c986a +CRW_0505.crw 7343990 cecc557f84f5fe78b6ebe7ae5a54c92a000089b77abc9a0abcf7189a12ce7a50 +CRW_0506.crw 7422508 e921b4f6f78775cc86e16a9dc8f1fe1c343efa33d44770c6723daee8a4ec4b58 +CRW_0507.crw 6568918 a6572fe9eb89cbec0ccf74b0ec71713943ebc5e49c1a13972efef5616dc96ce6 +CRW_0508.crw 5531106 e02d406a24445b5e8723a528af8337416cbba469f646cde8c939564a04bf7164 +CRW_0509.crw 5537958 f35ac86b49a987101b0d1cde7438c2996e43a884eaf52205e10137f2fc16e740 +CRW_0510.crw 7194024 276b2eeb7cfd2622c21b16881e45688896f3b1f72a7d02d644e6e699902d1de4 +CRW_0511.crw 7716694 4d1fec6bb9b151c63f5c2ea70dedc6e786584d8e0831b24cd7fb7d8ba64d459b +CRW_0512.crw 7756364 60976d50b522f02ff8ffd44d0dac7c6bde53d0a354a4ce5f3b22301b0351831b +CRW_0513.crw 6348622 10175524ceb6210748bb3eda37103279adfa4b4819bb70517c090511f288997f +CRW_0514.crw 6601340 4315879141cd799d611bb561a0754cb420d48fcfd58d1a8d638a7689a8512e44 +CRW_0515.crw 6258296 e772083d3225cf6370b096bd278610e73ef359ed34169c4cb6ab36fd3cedd5ad +CRW_0516.crw 6815366 0f6b5d8340361273e495375093d0b584e1b6e221ea3ef2cc2624a674d4786c87 +CRW_0517.crw 6829426 b788c4741098dea13136310b28325b62d62d6b01e6060abc32ecd837b0602578 +CRW_0518.crw 6757106 162b21adc21bf5b508a2c2dd4233123ddaeb92b79c4424b6ad7d293f1587826c +CRW_0519.crw 7779258 21dbfdf8580b5cc8a1cba0f0e3e4150bff019ea7f17af08d77ca25ab60efef0a +CRW_0520.crw 6680694 3ed1ffe4010f9c6a6c4bbeea25b7542916b52601ce0aae3e88b91325ae8fe3cb +CRW_0521.crw 6891678 dd6c8262ad2533aecd825ab593ce2b679de8f98b368cc180e39c8dac2b0d6ecb +CRW_0522.crw 6523920 91a282829cd8e36dbe5d65652515f6f81b547f10d714623eda9abe266d785874 +CRW_0523.crw 7005042 9a27c0b8167b12f0ae703f7f23fb482b1cf1111fb1db46d017b603bf9496a7b2 +CRW_0524.crw 6430364 e5478c8828445043a802b21a6cc358e69bf55194c28e2b10fd276578fe8c82ec +CRW_0525.crw 6933852 b83662eb5d0fe750c0322397be2055cc7829d999749aa68bfd2a6264d6d962e7 +CRW_0526.crw 7125418 9b0ffab4cb3ebeb94cbcb8ab2e59ec850f240e77a7496e4939f4321d067304e1 +CRW_0527.crw 7158482 2fd42a69caa20e7c07fe022bd73dfe2626d60a3bacc4228073d9627a5dfa03c1 +CRW_0528.crw 6853512 61f6a1d786cc16b3149200b07502e489178ad138766e2d569d29b13428fcb8a5 +CRW_0529.crw 7165366 7c10e1595a18f8f3f4bea9753818d2978b1fe326900420568431b929927399ff +CRW_0530.crw 6860278 42d8b5e9649ec98a4df5abcf3b400c97bf8fd5ada9687681db8f0747ae4f28c8 +CRW_0531.crw 6937180 63d888d67460e2e990e4e616d39081b6113a92fc30be0791de3c6110e7f63385 +CRW_0532.crw 6976490 7cd69db23d6ed1abb7446fc7c3a85db2ce93ade9a619c78051f23482dbd74592 +CRW_0533.crw 6565874 d70acfdf3cc4d233988b17235a4e360d2e38da89cfaa4b68b7ff8811da3c1e90 +CRW_0534.crw 7733158 4e8ff6912cef9644b01b7b03c4d140e79886211588c13e373ee37ee015b9050c +CRW_0535.crw 6735024 b6c72b1da2e24308b1ac556b46340c46129909cdbb96286906ec070355e6283d +CRW_0536.crw 6739400 1237586de05327efbcc7830f120851edb93cda9014f5c8dd2ee5ee69d092bdad +CRW_0537.crw 6808064 cabe5dc770a5e5b7f60d1c8184c246760efeaefd0d2e549ba43552656e2fc78f +CRW_0538.crw 7332706 8e9f11b3c1f68d4993fe16e9c825d932530e71b5fa7a9f5c98189309eba4bec6 +CRW_0539.crw 6630390 052fe9cace956291ca5b944038f4e6fc413ec9ee17e4b13b2f49acae412f0c34 +CRW_0540.crw 7777214 2d4522d93f323295082cc1696b968966b1fd3c720c3de04c0980ae48663f0cfa +CRW_0541.crw 6919552 95770b751d94e38fef6c7a4ea3e06824a0b6141f146e752f63e6c88dad0918cc +CRW_0542.crw 6965128 36e9605fda601d271fe98cacaeb98da3dd3a3314b83d1ce9e16526ce490f6f07 +CRW_0543.crw 6697620 3c70c9c886d1e0d55a670fa2d950209e4230cdaa507f9ad4fce136587d895f38 +CRW_0544.crw 7480314 b99585609662b2b73b3c643962dd69470de345f5a3462b097501309b939c6cd6 +CRW_0545.crw 7744318 68d2d37549c1d00e803001d156ed890c26c6d783c1d7f4aba8a763aa978f120d +CRW_0546.crw 6981466 36d2dd70fe4ad5672d303bc74fe6b3b4ccc0dce53898e62c5d33695fd1c537b9 +CRW_0547.crw 7739746 83d740788e70314aadb498e8d8ad6afa5f40a29f16aa47426ebf64193b442e8a +CRW_0548.crw 7143664 0f535d3e6b70e749b3b5a8008b0f0ffc774d5f9cd5222b96a0792503afc25e54 +CRW_0549.crw 6700618 4732ad1023f737578449bff44e7b53083d30c408a76c4e6accc004b2f764aaaf +CRW_0550.crw 9461958 a3aa26948dda629aaafaaf085334a987426c3aca3f55d8599fcd3ac9e6832d11 +CRW_0551.crw 8335974 3360c67d2e2088054af6c1ae1ca266496bd6dabd4b5e89ab1fea1233b51d453f +CRW_0552.crw 8306140 58c06097c633481175e24ab9c14d0c5defd02e5c0a33543d71572f0c4975eac4 +CRW_0553.crw 7810646 2be97108bc71a4a73f16c38d532a7d62f9627929cd78e5fe9a738d2fc8b67cd6 +CRW_0554.crw 7011526 60795025278b0a4d32109086ca4de0fc24837eb1b8aa157999cb06eb2ff22de4 +CRW_0555.crw 8077816 2513875e4fa8ed300bb29f6a0b19ebdc917864714ea7c539c4b732bc6547a5fe +CRW_0556.crw 6585440 c65f342c8e7efe485d2e461887c4a8172dad45fddda31488fef709fbee347d18 +CRW_0557.crw 6722800 d56b58093b38d6cbb25b011c4647e03d412c9a04dc507c964893f6085bf95f4a +CRW_0558.crw 7763438 eea38446d2e01e3e03f5b05cbedbe3e40e9a94415bc2f8a06c888681cd6686f1 +CRW_0559.crw 7933942 956c74bfd0f7be606e5528f1385bfcd146f60c609514a805818d73c490ce39cc +CRW_0560.crw 7896310 e052bc93a520b90c9e2d50102a67e4c4ec083a033f5b1165ae82c8aa2ba14c17 +CRW_0561.crw 7197160 3f5664f041dbed882459a868e3fca0adb5d42292d3f9c358c269b3391b5a49d9 +CRW_0562.crw 7194854 ebe4145a73be1f4e778e8572c0cc88ae8d2aca176b2175e4801fd2da295fdf32 +CRW_0563.crw 7355972 898d19f877b23bc6ab3f1cf8ba66c3639de4a9643fafa94492ea926831acdc61 +CRW_0564.crw 7485124 c6c8afbd72047dc21565de5d62960de8fe9ae593d9ae136b138704ad7639c80a +CRW_0565.crw 7090998 acc54250bfe87a4ce910686bdc6e092d901ad9d5a0203f1248b5b4936c256e5c +CRW_0566.crw 7417276 23ef370aa4791d66af6a65b505bbd519ef625c5b15b5fbe0fa90b78b4c1298b8 +CRW_0567.crw 8122472 6aca3cbff9bab10b92c9945590741f76da4d5c8071dd7081f635504a97d9070c +CRW_0568.crw 6231582 43c32f58795fc1cfb2f4a0ea4a7b94f1095420bfa9ab2be24764220f48bac687 +CRW_0569.crw 6306434 2e6b3669b4401e1345f08eee28798d693d50e903d4169ca1deae1038680eed8c +CRW_0570.crw 7404880 ec6b1de97097a3c3785ce409cc02a3311144bb1de2e2fe796dfcd6229eed83d7 +CRW_0571.crw 7864670 fa617c478aa9fd99b3885ec9ae5fa365192dd12ac26eedc07b11f1b887b9b1a2 +CRW_0572.crw 8141078 8f11409f89189521955306e18b1099b322922b619fafda231fb2cfc4aa19ab8e +CRW_0573.crw 7227426 4b63edb8c6f2cc3da322abcb49591a36d42761ae7d6e36f45d2d188684e7ea17 +CRW_0574.crw 6329264 a1d3df08ea6f40eff62bd8aaa82dfccaf5e30776015b912f0c72970c567df254 +CRW_0575.crw 7142966 fa1014b21a3c6feb611b07483b117be51fc129c4f5fc1ed77a8fa8d0f790c341 +CRW_0576.crw 6365542 cdf574238af084721d38e9f646b5ec9bcde123207a093dc8369963a3ff207e6c +CRW_0577.crw 6512812 75ad91384ad7599b1dac9ae1c4367ec5b3cc8d2e8694806360847b5fc8bbab41 +CRW_0578.crw 5983780 a41bfc9569c768ca463eebf8be67293f3c08362815ba41cb239e9a36574ab364 +CRW_0579.crw 5826756 eb6ba40ed01875a24604479067e95c6805abb74c9cc759a3eb013871bbdc5cd5 +CRW_0580.crw 5839398 b136aac73f572d056389720f4253b4cc33547523a74ea898653b6a351b552813 +CRW_0581.crw 8104704 c6917b5e601e3bc92865c6372196b649cb0cfdeb6c0c2ee1514956db695bef61 +CRW_0582.crw 8050006 345698798402743960ea3ca3dbce991f1d9bb2031a3fa2f2bcbd1b2fd45e4d36 +CRW_0583.crw 5998420 d4f73656afe7f347fd3bb26b600342f63da1d5c8b7120f5fba0e6e18bef7519d +CRW_0584.crw 5837362 45d5b614815062ba80775783be127b67841acb898afce89690cfc12440a539b9 +CRW_0585.crw 7286206 f566edee08fa147918034faef8de86ad294c21972a3a44577f7734c154addc3c +CRW_0586.crw 6950724 5d57b3f5a29d44f563d84174a013f2a35dcee0999db28ae557d6b8f1fe755aa6 +CRW_0587.crw 7629588 f757064733c31b2d4dce7d29b6820e5b4dfc7ef82ef0c8fb10d92afe5622a072 +CRW_0588.crw 7228728 4e18179b4bb637b801afaa21ee6a5ea196532dd4be43f5929ce1fc6cee5fb0b9 +CRW_0589.crw 7147314 08778fb4ae6a5ecc332704947e4a1dfdf477188d460f9deed5b4d61d29b7c3f4 +CRW_0590.crw 7168332 760b6a146a41fef804e59569b45754adf5f0841e58cbb3a8f275c84f0fb0d3e4 +CRW_0591.crw 6025256 a7ee03fd1ff36006f6244adda555da3b79bc773c4b98c50ff76ba1ab891747fd +CRW_0592.crw 7394154 981cb5783041b74718d675fd99f78f94176fb9b59ebfae750e710d2c4597a5eb +CRW_0593.crw 7314796 55b76b712e9fc2702e979c2ecf61f47c0426da6f605415415a7f172d9541f7f7 +CRW_0594.crw 7293844 c3dee9d3dd3f1702e5408b84f2ac174929cd151c191042090c655506cb20b089 +CRW_0595.crw 7314844 3281efa86e8db4fa4f886755261c2763749f0696bd05d51e11522f456258f8b7 +CRW_0596.crw 7615176 d2154d7783eb9366b2f26618413f758cf7f34d004311443e784f41bc43d1a249 +CRW_0597.crw 8628574 a42212c63c4bf63e6ff9464c3380443c1de2aa90cd6c421cc896230e9b1795e9 +CRW_0598.crw 8014448 e4c484febe6b2ef7e4c4f4ffb0ccd61604d21516845c6914c35764e6cf9618b4 +CRW_0599.crw 7074164 cdda1aa0343c6a68355911f452f6ecaa4e94d1ed2cfc8fbf86c4296ce3d85045 +CRW_0600.crw 6400458 e049b679ea1848633bd98338a93570e5f753f00881153e118d3f444b09bd26a1 +CRW_0601.crw 5887082 4eb88a94b7a42dab69520ccfc39b5c2173cfb18982f68a6a4915d65dc61ed71c +CRW_0602.crw 6505276 16b7c05e4b70318618d2941544f46e991263088b41811379c288d4c171fe0bd2 +CRW_0603.crw 7546518 efb152058536d0e14722329f49b3857dae1f89f516ec25ae3a0e660d0eb711c9 +CRW_0604.crw 7853908 315bb409b9518c42ec9ed31342a379b44cbd2850896f50c576a13af10e06f573 +CRW_0605.crw 7003594 b44b9d289d7a265072b1aead1acfe452f0cfc1dd8cd3f2414f6b735ea7baa442 +CRW_0606.crw 6438426 498efdc783bfdeacc9c5063dd2b16822310a465a624ab2a3fba181616934e418 +CRW_0607.crw 6435124 fba3842a6b6444026bcba45d88b552240bc4a75c40f93e1b8b196c1c8029a8c8 +CRW_0608.crw 6144732 e6276d2a3995aabbd6252c6a77a2c392c315670c0fb1338865144d4536395151 +CRW_0609.crw 6723616 a6f36133bea5dda7dba839cf827db8e5108597b855b5a5bda16e5323a88f24d4 +CRW_0610.crw 6926122 902ae95683ca148af26dbf0b233a84a7d398b16d5e00be517b5e511cc236d528 +CRW_0611.crw 6548862 a2eec303ac42a208f6962ae506346a107eb8178b7f733db67c5a0eb9e473cfd9 +CRW_0612.crw 6538326 1b8a42b0aff87de4c896f56fab00d28333aee17dd072cd1ae869dd5e79324274 +CRW_0613.crw 6546670 e6ee1102bc3c59082b14a21e1a88f74bb3c9d415420b34e88c925655e1c4a1ca +CRW_0614.crw 7035508 effe697a5583b85d0366385fc64ae4d41245cadd6a729099cebd14abd5a165c6 +CRW_0615.crw 6798488 be366ccc5e78979c357b3618db7daaa5797eb0397a2a83836bc654977ab5077d +CRW_0616.crw 7227702 54962cacf749b4c8583db959ed0190233fed84ead5f894252bf69997f4791387 +CRW_0617.crw 8026802 c617be549b0e7e4ee3a9e46c14307330a8e2a72b5424dfe08bb07e3aa4d9c4d1 +CRW_0618.crw 8176100 08129858176dbc531f7fdf868a3b58a17bde304dde531ad3117fd1df56cf9959 +CRW_0619.crw 6994012 5d56ba06dec7f873abfb4f32f8657ac14e28904df0efc28d98abd3e568ff4784 +CRW_0620.crw 7691226 50926ac30f489291d7e1a72e49ef9563438f6477bbcd3b4a7a408cd30e1f84ff +CRW_0621.crw 7544558 3a30c6a31b0fb6430e6e45bd01e49123c7a872c57272a1ca3144fc313c5423fc +CRW_0622.crw 7850174 d9720a0cf2412cfbd2ecea897f609f88401ed05ffe703329a257d7606c19cdce +CRW_0623.crw 6299254 9949cf520b7ed15bb86d387b2cfbaf7515f8be87ff4927b8652e617d5ab1bc4f +CRW_0624.crw 6298902 14addf2167432a179e10473db284dc07d831233be6bab9d0ba6cecaef77246b4 +CRW_0625.crw 7045642 12760f39042280900018f3d7e5fcfe53a30b05974e1bd07cd270aebda7997041 +CRW_0626.crw 6188754 06b01e9d27503bc3e3ec3b6333a3df97921466e3454180e6645093a51cb3c022 +CRW_0627.crw 6210308 78e722c2f92b64d835e42a82c14a3812794c131e336cffd1bc6f724656fb8db9 +CRW_0628.crw 6231494 61cfebd84a64d55d4253599cf39a0fdaad9eef5cb92e1783f518ad4f9e1f5ecb +CRW_0629.crw 6817816 d1ad9fd9afb30b69860d2d9e999aec512c628d2153e092e08b9b341d66c29114 +CRW_0630.crw 7457666 c8f93700e258d1af932b373ce92429a693a3388c1458f778acef47804ceb25d0 +CRW_0631.crw 7135884 1e3b9ca0d68f7955f9f987025f16640ac651c117bf0418e114586467cb704a14 +CRW_0632.crw 7123000 516d59bee62b44d1c4465322f689794a89b296a553bfcc39c0ee6829306fe99c +CRW_0633.crw 6983896 12a057aadec4e33905ccdfacdda94596b03eb6b61a507f7ac7be5f4d0a4e4bfe +CRW_0634.crw 7287372 2876d1fa0e976106fa73890e4c036ce0f926249f1a318beb404fac828135fe43 +CRW_0635.crw 7466504 a3e82130aaaa829fa771e5d06f502255f8531648b97f461adcaaf6f2212d2170 +CRW_0636.crw 8028676 9245c19ceb73bc2d63ccf487aa5aaf6b651b4f3e94ad35528fadc4b5a4d300aa +CRW_0637.crw 7571282 e827641e1a43c1d5987b4a143d797024ca5f10e1ed28abdb9049a0f72e6c4fde +CRW_0638.crw 7713244 a994a0954bdc82bb104e0fb7785f0b91b92ab47287225896017acc93991c69d4 +CRW_0639.crw 6567520 b220468b3c4009cc626df16b9d6ab9f93ab744016b1fb0fa4606d9aad9a9e83d +CRW_0640.crw 7680446 8e9b9712c420b0159e3de8ad7ce32e0ee735ce2f67b1491c66a3144c78b6f403 +CRW_0641.crw 6426796 737d40ebfe7fafb0bc8091dbed7fab7b83d38a74088c644382fc48856b8978c8 +CRW_0642.crw 6165408 cbe343a61ce5485fb77f65652a6fb21310f072b4141ddb8e8423aa96002f2939 +CRW_0643.crw 6134020 0b8fe6beaf84c4dffe7731f61e2b42401576135d21890e6f18db15cae138ab61 +CRW_0644.crw 6434042 4bbbed2d215982de259f5ae32895c0c3e6eb87ad7845fa05a88349f0514ca936 +CRW_0645.crw 6190054 8563775efb176065d65cbf533140c61361eb84602200eed7f48dae89957656ee +CRW_0646.crw 6403278 17b44d07b707a2c20ffbfc50651451aa5e7edd72fd31caada3fbf8ad2b537511 +CRW_0647.crw 6928950 26a24e62d5fbe858b2d0a3e40d51f2800e1a46f1165ed0c13c84992c59892d0f +CRW_0648.crw 6458954 c0a5eeb1d0018f32cd691f2f228e5f13574b6b14942f95d45b4e04424551e47a +CRW_0649.crw 7405442 ac38af35e57ea91b8f871ce38884188478981d1b15f770cf2423bfbc67808311 +CRW_0650.crw 7190782 1c7ff250504c608904a44bb4524c5443a4649521d827b607267fbebf1dbd2240 +CRW_0651.crw 6757278 aea6626cd58441f2eaec88527a8eba76e369616acd646cfca8cee535c4999faf +CRW_0652.crw 6792788 1711588fb34d87ac2579d279e6f15ac7e45d79f04da730204d8bbac34297d150 +CRW_0653.crw 6512484 46c30dfcdf4edb39cd4b7b04fd62261fc1d895ac6a6a29cffd9865134891396c +CRW_0654.crw 6707780 b62962c05d3673c9d5d8fce281334b8d7fb8886bd377e624284745bf1c83232d +CRW_0655.crw 7749008 4d487a53be2db66675265a7fd29715f5bb7c7cbaf7df26b0ff2478ec2082fd63 +CRW_0656.crw 8343820 c9901d2d71f63a26ebdc8b91268da2217eacee0c49d1f4d5f64be62a2fc4ec1f +CRW_0657.crw 7991572 09b90f704b351fe5df8597ba56b07fb69d8f1c3d54d6cf5acc227b57027489ed +CRW_0658.crw 7807540 fe2f681d4e34193dd840ff9c191be7778b9ad4b3bb55701ea00919e5bed3e43b +CRW_0659.crw 8044158 6aa38c22d9e0009e65295bd23b8d287f16748430af98945da71177b059607534 +CRW_0660.crw 8227494 5c33f82523aa998e862cb74e27338f5505c94571c5b933cc80005d6d968744d3 +CRW_0661.crw 8245224 a046f43d8966dee03072489d63b63eb3d9f6f8ca9567b5f7b9c37fa07d36b3d9 +CRW_0662.crw 8329114 d84e37e3203649aad85eb8764cf9f4da5ec7dc6b13a542b58246eb2de3fea373 +CRW_0663.crw 7855000 a5e9a43f7971b1e460d57eb270512074790b644e665e6b3071a8baa2e10d3f56 +CRW_0664.crw 7994006 a718e069f8adaa13820de40ae984e56d6679c6dbcd62590984af8f342a903a84 +CRW_0665.crw 8016472 92dad0104ca3450bf227d11d75d8536be248e8fdd62069159fdac3970e504ece +CRW_0666.crw 8032376 769bdf813cadb9f336d2281e166e0aec7db88f00b93a77579cb3deb3e24604b6 +CRW_0667.crw 7994034 03655934bc8f4396a289c2edb55d1511417e3501b922467073efe5456ee4376d +CRW_0668.crw 7017106 a867a92ac489aac7dbff84ecb62dc47d07bf083090a08a7f80b798719dcc725a +CRW_0669.crw 7120012 3c417cd93cad9f78c3d50e2f947c69ea203cf7295118a799a32e8eae7736de52 +CRW_0670.crw 6462990 9ddf4eb49ceb7de144c9131f421a08452fc231af6286fb0359ade2d019154e4b +CRW_0671.crw 7217912 f44a5b789a32112643992cdc05fd7530fdf7c8d49b191eb562d2d71ec881b57e +CRW_0672.crw 7288824 df8c948ef223661b94ae50c9d49e0a5f8a8fe73ec026fb8c1ee66c89df78d6d6 +CRW_0673.crw 7485200 53ac229a674ba91edf86bfbf5ebab7671e01d67f33ade0f254f7fcc3d60faf5b +CRW_0674.crw 6469210 8ac3bbf7f37399aa3ce9c002efc237c719de7cf9a2af85b3c4e9fae4e455594e +CRW_0675.crw 6942490 65c79b4c722b9361abab096e1360b935d1e9933401c54c2630a30b50dd534335 +CRW_0676.crw 6928166 76289df918801505bb2204bba4411920f3339533c8bd665ece489bb0631fe5e9 +CRW_0677.crw 6748480 91a4fbf1000c2c2f14686ef172d53ffb10f9fc1fd34b230577c3dbd917c34062 +CRW_0678.crw 6498740 dc4b0a4124f4508e2a1613211a2231872bdeb1bd246e7b42543a3f53018a96bd +CRW_0679.crw 5740534 02654324f2bd8cddf89860defad4250111e2e82f6452b936946f2b47062724e4 +CRW_0680.crw 5996208 b7d6a776f617a6e50e99e48ad442fb6ed1ea14ec8087be6ca87fd08cd1dbe47f +CRW_0681.crw 5810736 c9386a7bd3b733d4b476e743d8c440dccbc89e3ff150f154245874d83fb80e84 +CRW_0682.crw 5868008 3be635d393b664aa32610b26ad17ee4b6ef8c59b2147727aaba86a2fae08dd2f +CRW_0683.crw 7453390 c4b320a158b1859fa0709050caddfe2bec0d0a566eafeb17df89cc6b64346a75 +CRW_0684.crw 7607918 a6e7365a264e7f799116fe40be209babd9c9840743224742a97acf47533817e3 +CRW_0685.crw 7567268 516e9eb1cd586173292121aaf3bc918d187a9a6e9152fe7742346475453fbfc9 +CRW_0686.crw 6996844 104e796e427977b83a65459fcb0eef9c84f98dcc92f75306952026cc0ef739bd +CRW_0687.crw 7584348 3495c8058ffdbdfeee8a27a2078697188554874add58c39a83682370db42df5a +CRW_0688.crw 6971628 6a5014a411245789f5c63acecfae516e54830404940fc6e568f6b8648b93d9fd +CRW_0689.crw 7216796 bca3d24463f3df4805099067ddb99af75a1982775e3dff8a4906dffa72f7ea27 +CRW_0690.crw 7445220 0207dde4d6b38cb4e72b5a4896139b7d3bfdc1d1af2a2ca7370e00ee96a20440 +CRW_0691.crw 7014684 d712b3f992b9de5c45f61d840522f19e66d52c7b0f1bf9966a02eac98e500b4a +CRW_0692.crw 6871636 3a915e8828c9b8eaf0d161662bbb2c5478e3b7ca9848d295b2c5e89832cf54d4 +CRW_0693.crw 6832392 4fc9124be677c8f31959c665d6c0301c024eb28c2c8d26313001c54f0e1123ba +CRW_0694.crw 7319104 135fc69166615f3dd7d36c6de688bc4502b1e3ac2e1abe3fd2be8d57ef276672 +CRW_0695.crw 7322988 98aff8c6259ed0ec053a91f4c703f3ab0dd818f4665460d1c03254260e5bc5f6 +CRW_0696.crw 6846970 54cbfc10a449b9c8746255f5ffd4115b90ddde41ec175339a6a44da2f96fc0f2 +CRW_0697.crw 7641328 40e2996a15a207b119dc01d176a25c32996a18b50f94cb66aa3b7b09863fde67 +CRW_0698.crw 7979918 b92ebead4fcb844d2b1bbfcedf82f1b82e38f9910d1343111b25a12259b1f9df +CRW_0699.crw 8016724 3d56c39bd998fab80166baab88ec1b635f3dde5ae4b383fdaea2babdc3978f5b +CRW_0700.crw 8131250 cd63a54e48073f371430fcead8be238adcad291e394f50bd93f19a3f30336961 +CRW_0701.crw 7865464 6ba8534543138927e1407424ab48d199ce66b2aef412ae68d39b192bce88e7c7 +CRW_0702.crw 7987094 a10e8f60ce2e79dc70ec506a1678aac487f0945a0ddcd838c1fa2879dfc68355 +CRW_0703.crw 7648530 d5139e927c43f70794e9b01e0c96fd56868e9b93efea6edff0a8f96fa6800971 +CRW_0704.crw 7328956 90c0dcbe08e61e2fe06d1b9360562e2cba0723332f398ceb63904d50fbd3f047 +CRW_0705.crw 7597760 28b47e53bc44fffa1debaf384af46532e953fdd8c1233343857ea90debaeea1f +CRW_0706.crw 7881800 f55c05fb34ef70fd7ea336a6b55d5326d04dea79e4d4a2812eb3715728a65f8f +CRW_0707.crw 7609704 f052eee15ad087f50d5b892c6dccf72b6dd9036b12e6ea206a6795eb7af1513e +CRW_0708.crw 6782660 7044e94919fa5f505cb6dc4d4a112e3c977bd8864dcb38d2a4906aa9e252a9fe +CRW_0709.crw 6767104 d6263fb1b58cfae40242324fc6458caabfc164703456c66b793fd1589258978f +CRW_0710.crw 7685486 b606869eaec1af4274af621b24c377a76b988ba4b0a3a8478253487f90a51b6e +CRW_0711.crw 7786332 1c3bbb5a807ff85e3eb2e973475b54e1c87f4127f8ef227c14613685fb77416a +CRW_0712.crw 7922654 508501bd84f2e9921a12ebc5e3ab627f74ae7bfc46a4c5336e1316959b6f059d +CRW_0713.crw 7982226 4e50d762b23a63f0f042d18645adfa8d2bda216f4dade8f95ded391a0b799954 +CRW_0714.crw 6949772 ae991c0a4d2ae4bf12081f928be0967be1aede9ef790118e59007d942e7a72c0 +CRW_0715.crw 7105136 db445bd7071fdb530cf4d3b5799faef32461ba065c60e261a5c03a4b78f14913 +CRW_0716.crw 7168414 8e9228055d1d6a887237a101c4262583d41d62bf8e64380aa923d46c07a54e29 +CRW_0717.crw 6633986 0d6b64935d928e2e0d378eb6efe2b46bed9026ed227111fd47b5fe8b911e22ad +CRW_0718.crw 6842620 cd63df8b6686eac356841c250df9a87382f880109bf27f36d076ea3aed7dfcc4 +CRW_0719.crw 9065828 0dee77d1ddc94001eb982db81ed7981f5c025d61dc16665f8bbd1fc96ee3c123 +CRW_0720.crw 8806736 8133069d375a58f74f2b56b18b38276b851f1445ea2e0901fc05597280c8f609 +CRW_0721.crw 9230712 2e0e220c164f0a273f5f9cb708c74a495fcdf4f7dd12905685ff870001013e86 +CRW_0722.crw 9200192 215c5cf0ebc3907240c1aa0a09590108ff4d7b9f7b59e1501775285e5e6f2453 +CRW_0723.crw 8974942 5def6dc28ff2874f0020a0058e4fb3c693e5ea14ba4431153c39363f53b8d8ae +CRW_0724.crw 9013410 f2c593f04db11a9aa092e02b915c2c9bb0e832aaed473d69dc08936ecf6cf653 +CRW_0725.crw 7577632 f9b63db9fd73268dd7adb13deba6449cef990ec6e4d765213d6ab097fc618682 +CRW_0726.crw 7692784 c67414b55465494c138fa1fc8c59079ef6322f730b0ab0e8b688624ff6beaaa3 +CRW_0727.crw 8309928 83cb1eb551e37c5df693759f57a197b435f9d5d83b2ff20f6812040f01e12edc +CRW_0728.crw 8620640 7d8346407c65d730a710fc02d5d201aa59f56ffd54ed7aec31bddc79373eac6d +CRW_0729.crw 8481254 b4b7c12f9cd0e790c9dda2be062f9c5a316ed32ed155a62355cfede74fe3d24f +CRW_0730.crw 8432946 5633f287ab91450006a8d80e2eb413a11a2d10ffa83ba7bb7d7e269f0f944bc3 +CRW_0731.crw 8389318 a6b2fa3d7ab3f71eea8e080b1b80f2d90a51bddab0e4dc95a393f63a7db12162 +CRW_0732.crw 7862228 cbbd06076ee28550114f1899113a903dfc2d3b979d1882de264dad8f507b62fe +CRW_0733.crw 6564388 4ff8989c28f7962acc73c7e5a4570580b0888c3ed19cb26778096752f8d55ce6 +CRW_0734.crw 6895910 463ff1d1d2fabbfecd35f9aecd446c35614f8a6be801fd48d3f96055c9e741df +CRW_0735.crw 6781248 3372a49dee3a0584e2acf165fbfcc3eb66a63a3a738bfdf5c4af2b2124591237 +CRW_0736.crw 6794102 0c5fcef75621208ff1ae50fd3a225d55ce54ae1a0081212052ef39b666469ab5 +CRW_0737.crw 6876624 1f7f1b8ed0f4617d6f1010cf75865703cf2c1e7ab39aeb12cc7dfbe9eb00512d +CRW_0738.crw 6703214 e34233e21c6f8fdf2d13ff83f3663334ef8daa9257ddab82ff2785b3fed7932e +CRW_0739.crw 6850310 144da4b92b9f023c90c9e07b27e4915d6f7946d08bfc512991ab0dd33c542838 +CRW_0740.crw 6859716 e77144eac506840f7f7d8c58b6d1bc3e37c9ad00347a48177e640292524c6ac5 +CRW_0741.crw 7160106 c79f628cf04b48ac8edf7f19d8d1818fa7b1da29b4f7c612c108d77a08685151 +CRW_0742.crw 5426594 d655284d3a803a0835f286a65a9c334fefbc67fcc206837eb319784eb50bcbc6 +CRW_0743.crw 5859522 df691be57c68476aa6caa1eb063b33377d7b2270de8664aa20076f767a80226b +CRW_0744.crw 6509026 d99f3158bfdb280843f77bbc0fbae77ed4a8ab10000156c54dd5445947a601d4 +CRW_0745.crw 7182896 54d51c3feb6e2f0fdd6274f3c8314fe4cae9f4394086fb843b531ff4c2a32905 +CRW_0746.crw 6801292 ee39baffae0c8c75b90d24d5e6f423e2d5bee6030f935c1e9b436a6534622b6c +CRW_0747.crw 6772278 a0f380b2f23f94ef33f10e42119b82d4c7e20ed79a3a90a45fa8e53d0051fbf3 +CRW_0748.crw 6291938 43c980b9c1850204d4336e93b9853ed56544e49e45ecd99eb1180b63ade7b1e7 +CRW_0749.crw 6106084 91857859fbc74eaff73c2a4aa7b38ad303d73322f9da158ae2bdf8b8812d7c06 +CRW_0750.crw 6302452 4cc52dabe6ef05d0c25fdffb9511ccc2128c8da5491e9adbd0811080e4537971 +CRW_0751.crw 6578946 c55b9f936f84ddcee77706462a482025ebc7a12237d721e38d9cc0137137452b +CRW_0752.crw 6133548 60a13ab4edc0ebedc62e245ae3372f798b1aca49a7acaa4a04b1b12762b9bfcd +CRW_0753.crw 6389262 97b5ec577105f4bd602e26484d9e1a0b1a4c1e10a89c68d23766d8fd1301c4dc +CRW_0754.crw 6174596 6a49f1a1e771ca265f7074e8f2b5cb0d86f2adb45ace220231dd6e0e96269f37 +CRW_0755.crw 5649396 ed86f52b0e4b6f9c30c6c476ef2f63562a03c9128661d7bbc7786f99c339cd78 +CRW_0756.crw 5753840 0094167fa6e1814229cf768240468067de74b322854b8093fc80b95f15f00693 +CRW_0757.crw 5768460 7c1bf370e54b72d9b7536e974294bc94b2b0c54a0a9236d04358c4e4fb20e9da +CRW_0758.crw 5791274 1a2eb759f0005a6bde79808d12e34f6c0a1aaf8fcfbc2d9451dad70326d94fed +CRW_0759.crw 5865692 6cfe61206ec5f7aa9999df607f857bae8ada56a5dcd2d0672deedc82c2464695 +CRW_0760.crw 6931812 ddf33ecbf19941405c493d0e3db3337c5277be31b655266af9d148a9eb789509 +CRW_0761.crw 7046128 38818f096e961ca9724354af9b78583f0d91e4beda5f4127717a99eaf4468650 +CRW_0762.crw 6972018 203d1f32d472f65152c8874dbfccd0c240df92cf465ec15ab83961e38a7875b1 +CRW_0763.crw 7412632 1d642d2e3cbb5f1e3c69cc77c440defd4eb1f57610a0f05813671459f825bd8f +CRW_0764.crw 7239628 0c57bf19ea16f9e6dc42b1fb30b29686975ef4d8000115c5367debab873a3238 +CRW_0765.crw 5636168 e8da71d85e07d4157fb5753f850b331dfa6451d7382c4c0d4f9568144e6e8bc5 +CRW_0766.crw 5737678 b96f30787a4a3785d6a6b9fb487341e3e249e4fb07ecce24ad41b3ff543fddab +CRW_0767.crw 5716082 80d13fd5e1e26f7a67164b9348479f3951fc4202fc2f445e9172cb3e62c0cdf0 +CRW_0768.crw 5652470 88ab468c9fe3c78280a39930d457f546385343f315a43e68064b9bc487e87cad +CRW_0769.crw 6309450 c64ae17623dfb78b41f59c0a1273a227b8f6106594775ab13d4fecbfc9dcca43 +CRW_0770.crw 5714726 de8093a9c07e67002b31a1f234b04f65790d89dfa09a8be0978916d7191fcb8a +CRW_0771.crw 5802618 c04d76152b3603f85e9cfece43e6d8069936045f655171b2d343647671045a92 +CRW_0772.crw 6678226 07a482d2d1c87411988ee900b4948a815fb25df18a29b8f4ee9c5921d5f54b03 +CRW_0773.crw 6001180 68db2c1b2f8c799935be5d3c2ffee62c6724ff7f8a16103f263b9a114919f38f +CRW_0774.crw 6032180 01c369ca439f68b16384902f759fd14bf984f4950756536a0ec8752184d5f174 +CRW_0775.crw 6426520 63cab0258da82696f2680d93c8094a0e04472d39e9f46be2c9614bfb748402c2 +CRW_0776.crw 6190866 c828c3c021d2a60f579ffd17b0deb7d90d25ce5c1acf71adc4295452d619a2a8 +CRW_0777.crw 6407186 e3ced542a059f6d1f2fdc7d85a83cd6b84b7a38620a9edea00eecc96b54e3f26 +CRW_0778.crw 5679716 dc009273e900fdfefa8789e470049d97fa56cfcffcaf044ff9d2a665947bb8a4 +CRW_0779.crw 7291208 d7cf5efbd7a83369354a7a224eb671a5d1eb671d6565ea6d5cb1ca7f7719dde6 +CRW_0780.crw 6733370 8ed5fb3b8341a15b97d75be32a660cc3c7651a8d481a1658c58156604e47526f +CRW_0781.crw 7300542 d7a1a6efb76c29d558476a87e3edd0ddd1494fb506e366529de877891d8ffdcf +CRW_0782.crw 6706848 fa0e639ec1f34f396234da0a45519858a0a751c4be60b6b0df324235f3880306 +CRW_0783.crw 6186472 6012996e3157304c4bd04d2ffff2ffefd2faf6c095dff9dc901f51d05b967d97 +CRW_0784.crw 6177780 632ded91f2ea24cbb5607bb806b4dc1b3d733478c5afc6c8f0601025a5777d01 +CRW_0785.crw 6185550 8a33759726999e36e895df084ee25a210cf8f3910b11c265a4640b3e5281a2c7 +CRW_0786.crw 6030298 04428e8cb1f8ae4756d84b8e5ed8ff28b1085f52dc481b69b166a741c2673928 +CRW_0787.crw 6818860 28f3ede7a94ee568ad374e7177b659fb3c6999f4cef5ef43052b47d42aee1102 +CRW_0788.crw 6738962 4d597e1c6c946952734d79593410519e0ba43a36f850ff5aebb086aad5f8f225 +CRW_0789.crw 6002886 c83a719b85b6a9b7ab858393a5301b0f368d467176a8e092085c87b10bdf3548 +CRW_0790.crw 6552724 96e40693a4c550bc6f4e688ba388be4bfca974a82ef0fc3f08258fe709f14938 +CRW_0791.crw 6403794 bb3d843c432ce9c07edf900074621e5d7f4e956bfb7d48635dd182bcf699c060 +CRW_0792.crw 6721816 54e0bc8e2721d602160dec293b0e41b70aee838fc04f175520623189af97a64d +CRW_0793.crw 6463932 e46e81ad4aab4c3442ec06fe99d52d7c252cdd3d469be50dc168372cf1ac0b5d +CRW_0794.crw 6047854 31f0357e003a0c5b54c9e13d7cf4c1a7ba69deed0148fae58443262944962f9a +CRW_0795.crw 7168692 b628f8780fd6a03acd867da0f3f9c947151bfc7ee7dde878dd839894bd651110 +CRW_0796.crw 5896170 bd9c8990bffc623e1bde2cee751ce9586ac470f90e02162dbb915e1499bb57e3 +CRW_0797.crw 7488738 5a0267e1be9b94d24524838ed907400aaadbe722ad7d11424b857427afe48277 +CRW_0798.crw 6996598 089fe2d3e11315246bb336f5e723b20e96aabcfa7d6092c1a18b76eba19f0f09 +CRW_0799.crw 5362534 5cb4a11f897c02c23b5161829302652c4290467e9f717bb3c82b8de08fa3c013 +CRW_0822.crw 6478092 817acdc28dd1fe2cd0065b44c63722aaa1f800275e0b0b245ad3b3d964df7c9f +CRW_0879.crw 6861152 df66c000ac18b872a69b0fdc171245f89869bd0ad415e117c77e6c27ea93c0ca +CRW_0896.crw 6622982 538f207223f106e129f15cb49fc26d0af028774a3541ff91f575da7fcf48d0c4 +CRW_0897.crw 6524356 c258a65b882b78a477d86322f155f14ed545805beb8a84af8a4188adc83d6727 +CRW_0898.crw 6523818 434be0337fcfbd23bc547f812f9e5141a5887ba730d6c6a488de680ee854876e +CRW_0899.crw 6439566 471ef1b0540c20f6ab990165453b20a969970d11bd5a719fc9145846d2dfcd86 +CRW_0900.crw 6445520 4a1ab49255d516d967f9f331d79dd489ce5e82f3a8d369921765552a436f5ff2 +CRW_0901.crw 5852310 183ec18bc4364cb6b366855225b710108623bd1132dc7609a7c12e9f67883fe8 +CRW_0902.crw 6283386 ef3c001f8977c4b89ad5f89053a7d64a40ae9a1d628db38230551a57161163e7 +CRW_0903.crw 6269188 2e188d26379efe071dd1770b849de1b9a6b0380544580e122b3c91b58264da46 +CRW_0904.crw 6035832 76c59b7756aedf2108b7e7371e6056f1aec2cd2f8ca706a94beb175aca448444 +CRW_0905.crw 6024576 b894e71394fb3a8c474e88f53fc9a5a594e70027e568c37c456c9479d9757c8f +CRW_0906.crw 5763780 670ff39b7b5923257ded9b2bef2877046ace2df71a1bcba677f480a108a4095e +CRW_0907.crw 5826910 baa4307515f681eead710eeec9ff61c5adde32bd51077dc347129e17a85689b0 +CRW_0908.crw 5854030 f06bea547d38937ec15f71f73dc8e7981e620d27350e05789248bef68d0ba77e +CRW_0909.crw 6429904 3886786f7be47e8f18c896056e771378370713eecbf935d5e618b35d1abccd63 +CRW_0910.crw 6214530 3158cf4a55dc7ac38e8b0ac1bd5ab16bd9250ac4e16c168c76d2ff5f7e914a9b +CRW_0911.crw 7276568 7e5e5d21b56c21a650749971da3d7a8aa7423afc0e81631ae26e38a96665f9b8 +CRW_0912.crw 7213432 688cc287897b0d3d82a8694cb8b03b750bc48846f3bf534cc1ff30ea5c1c87d7 +CRW_0913.crw 7681830 c975e0a09ee8f64f9f59ab59d4e9a7a956bcfff8d9a3303cfa47d09aa60b2fc0 +CRW_0914.crw 7598874 044e19e4f4a665e49e150588b3f7149b1f891167c45b1dd503b1ee567cf9f866 +CRW_0915.crw 6901528 ee1ecdbb06c3cb2b239e3d0bb8117e7d981eebe3194111cb125fcf30fb6e3eba +CRW_0916.crw 7256244 9528d4890d5e33183709ea1b8d7a38a3f812fbc810b574289803adac5cd428bf +CRW_0917.crw 7766576 6fc2c60754b35aade8becb34637c41353d94c5171063ff50276645a86b6a6756 +CRW_0918.crw 6725938 f0e105dd635628e06d80d438490a4c3729b71b55fa1a3d1614d2978623d8a2b7 +CRW_0919.crw 7150106 c904eef2d710e286d292ea6a77f48ec070b01de090655465e3127308c1016091 +CRW_0920.crw 6025970 289c484164e4eca58a52851200e6ca9db35600b6afe871a0ea65c8c361191f29 +CRW_0921.crw 6377198 31ed0d4b94b86f8f3e692e523df12329c49446a0e6090c7d24b9c7bfa108b5ce +CRW_0922.crw 6956430 e85b85cfd219724f2e9a8e796e8757b5f76a52c4fee9edc70377d50a8b3cbcde +CRW_0923.crw 6149030 360393f1cd5d85d1a1fbc51718f8e9726fedfa37d1d617d13066c611330fca68 +CRW_0924.crw 6480502 250c3006a2f39cc2b5b4183ccd9002e46a26eace5be49fa38d7d8a81c4642616 +CRW_0925.crw 6484968 e13d90f1db60abef49e3d08df96a20fd52a75267197ed89b839c07b3ef226360 +CRW_0926.crw 6391110 4408aa82115230416f2a599b87b02dc2e71791bdc943e1054458de8f75aeb354 +CRW_0927.crw 6295812 ea0f4566432b2500ebb9725281ebb5b8fa2104a427cbda9ac6425dbb95860c8d +CRW_0928.crw 6394232 f9c3c9b875b52e569cb1d619abce2cf9f474b0985cc5ac0039a85bc94046d1a8 +CRW_0929.crw 6885798 7f604e423ce9a3d7675706cb8741e20779b4f62e8a6476e49f352c841bf94298 +CRW_0930.crw 7515748 fd714aaa90a050e547cdcf74b8757ed7ed90157bb22dd5189a2476172ca52e23 +CRW_0931.crw 7565968 60b16c979407b0490e7829a74363f234cdaa5b129095ccc5a84a3ede1a1926a1 +CRW_0932.crw 7404174 a8fc047fe75fa795100191ece0b2fafbbbbc5f41a47cf96c56d84fe0e9d91e15 +CRW_0933.crw 8398062 8d9ceb01a078fd4cd7dc9bc75944fcc2afb5e8b93a3505f9c0d2f4bf48055453 +CRW_0934.crw 8424596 8f631c6c4efbdef20eb6c6fb59e044d6724f2607ef0450b7676ffbf7a3f4e544 +CRW_0935.crw 7848872 0c6b3138a1036b93a8da9a08688bbdb0bebb8a006cfcb4f90cc76ccea2b996a3 +CRW_0936.crw 7688882 a6eff3018089045a2d200418cc3c82c34192a34e4d66808b7fe98fc632c1197c +CRW_0937.crw 7310858 db385815047caf4c87705b6f1f226fd666c80e673965838939181816b5b6bdc0 +CRW_0938.crw 6814164 aac8db8e4d5c144bcaa478c64a02302061ced09c9102d51c0c504c67227fb76a +CRW_0939.crw 6551188 1f30c89ddc910200f6578a25c08e2ae1cba163394a640f9a03b97410d2f278df +CRW_0940.crw 5841744 a378e7e82803678a1e7dac05cf702fb405c1cc97d4c5a9fc0825317ac2799dd7 +CRW_0941.crw 5897192 6f5f2539268ed7e3d0972b3b20306150cb6247cac6ab774550f2f866e1d7f733 +CRW_0942.crw 5834046 38e95167031b94072faaa2a1e3e9a40b8857ef04bed13f4af9d77985953a39be +CRW_0943.crw 7220030 3ebc70b1ba65aa13e19d2755f6bb9a257291c44f708744bc2f26e183fa504067 +CRW_0944.crw 6478070 bd5837b720969bbe18bef33b7f88c19b885c0aec44f6e7cb93dde63b2c9395d7 +CRW_0945.crw 6421732 b62916ad4cdd11999d1dfe9f7f63532595af3fac289bd20f72e60ccd8f7a3370 +CRW_0946.crw 7003602 4088a6f5b75926167044fb47c834b4f98687c3a02b10e135c4a3a718f6a8e4c5 +CRW_0947.crw 6551224 57f551f81e4848187ee06c775fdee1a9780cf29d5a9d0765c1a600f1962c14b9 +CRW_0948.crw 6733812 19a728aebf83e054d3f4e017e13c4029441d0af27c88a65e2bddbc51a6cb28b9 +CRW_0949.crw 7366078 8bae20af6825a5f3617849c812780daaba85334b931c20711172099188f948e2 +CRW_0950.crw 7152750 46aa7f32362daaa3d8b8577447452d800d3efba41abfdff9092036fdb51fd7aa +CRW_0951.crw 5746550 539511e477f855df7ad497c2d4a5ee5cd7a3f8ad7c09173670c00d3b51df210f +CRW_0952.crw 5619942 27967a844c8d5cb0ad439da68b0c05b62105c0bfa9e597154eba25cfa617d889 +CRW_0953.crw 6007082 c2996a3b38bab99f867b8fe49365cc343db203d567b7c7e72a07443ebf6330e5 +CRW_0954.crw 6567470 3f94dc77cb60c395f81879cbcba403b20d70fd214cded814d5980c57450353c3 +CRW_0955.crw 5996572 3f59e5b66c48e3255ec737b377f6bc803537329093cb06cd22cfd56d0a189d04 +CRW_0956.crw 7652666 8e20d7379e87d293bbd9caa9bcb18ef2df306e3bf6bd85f30c651a47c196dec3 +CRW_0957.crw 5663504 aef7082544d5c4c32f2b25ede9b4df99599e43680cc89b11f42c79a85c3b17a3 +CRW_0958.crw 5839628 93ad9b338974709b3f8a18fec480e8709ff098f88154dc40030e11a304618ac5 +CRW_0959.crw 6909180 9318eaecb0bef2b3325826e0f86bc59e3bc13fb035fb6680e66780da4673d2ea +CRW_0960.crw 6446022 6fe010072a0675854ef71beb673fc7e2999251b912819e72f9e9e1232b7c0f45 +CRW_0961.crw 6979534 627a3d4a2d671ff10ed5ac6b286c3904d583da981cbb862a4251497b99df0249 +CRW_0962.crw 7293416 796308e54401187115a5aa0c2a54e9194db762698f541d9d58846f7cca3854ca +CRW_0963.crw 6367344 8541ed809a1ec5dcb30f9607349ab0b17e123076c52fa13b02d859623f9a5b6c +CRW_0964.crw 7014312 9c30c2be73af929d81d3a77121d0ef91f79957d7e47a35d01386bc35305a1fc4 +CRW_0965.crw 7003056 cfd84abbe0fdaa83ad501246996f79b04fd64c97c8be034927fa3ea7f0b6f457 +CRW_0966.crw 7646682 0d1021c100aa131b8e8cb4d6cef86de80ffb9f213de662641beb8e7fd05f3dca +CRW_0967.crw 8152336 582df254424c8921c28b75606bce1c546310d259ec48c5133723de00f26a38d8 +CRW_0968.crw 7717672 c70108cbf209ddc682cb93dc98142c2380e43c411a62c7eb872446dd4b9d59a3 +CRW_0969.crw 8070832 be8b5bcae8a806d412c68f396116f4325969146c1713d93f08e304aa315b8de8 +CRW_0970.crw 7369396 6581f786a3f8b96e9586f067747dd9403318b0b02ee4513c7d090c3d9bf5c618 +CRW_0971.crw 7305532 c423cc7fab0139a7cd8e3da32a7e17c8558b6865b40f980e92a1b3b6e9eb76ee +CRW_0972.crw 6256228 10a81471d234ef6bbcba7bb72d9d3c33b7db8adfa487cc7ff22430a87708001d +CRW_0973.crw 6465924 d9de74f1ff063853007296537f326915686da54577facbebfa720861823589bb +CRW_0974.crw 6429532 4bedc5dbfd845ef27099014d85b1b72d58003bee78728f5e0717ef80a573b7e5 +CRW_0975.crw 6533944 5dd867311356d665c6d6ccf6a4cd3c0bd098540a7b41bf687319d6615bd2d7dc +CRW_0976.crw 5471322 c9e5307b336102fb2ab7f6635ff10de45d174532e5c2fe95a80787cfd40bdc2f +CRW_0977.crw 6201926 073f5da1bea6187c6e3400b140d31078778ec9a64620f2da2d059d41bcf085c0 +CRW_0978.crw 5633880 ea8392410e4e5d9f10708e4b964e2a1f871d7b7ba199fb587aa5ebf437beba5e +CRW_0979.crw 5402192 2b602abe03433ed9e56615ee4dde41ae045366d5e94301a35672f53f6ba62752 +CRW_0980.crw 8316632 af4779847c296dca9ece788ab486d016ef513aed53396fc57cbd7d7b8f9d1cf3 +CRW_0981.crw 7752318 b3b08e51936c6f5fcf0e4d4842d8835aa83054d7f7e4f87dccc031fd6e5efaa9 +CRW_0982.crw 7340888 edbf8b186dd731ecb29a339b9abd0d2782bff08115740a40c02cdd5cf5c78646 +CRW_0983.crw 7472340 b315abd6f2316d81341dedc13a8edf2feb006ba5bb242c6837d2220ae2c9f022 +CRW_0984.crw 8168208 95c7e8e0ec9c2d6f4359f74816f5d1192f72edc558ef0972950e6e3e5765c62b +CRW_0985.crw 7092110 7df3e714a27e4b681264a671789d47b326e6e110358943dbf09e5015ab72bb40 +CRW_0986.crw 6634820 6c8b95eccb2ace2442e4b6e585b46d74cd062d7cb9da3386afa96ac770a646fc +CRW_0987.crw 6573080 c9d664d2aa92619983a29acd098112af6d42dc62d91a874918d1612cf6469e05 +CRW_0988.crw 6549478 e4b420d057ff35f99025f0a8c1995908f0ac2afd5acef0ecad635dc3a67a02f0 +CRW_0989.crw 5488948 ae8059ad2752f7cbff40f7c75d2865b4afe066594c30fa9fd42ac3aa7a5b112a +CRW_0990.crw 5895814 bbd8d56f8f6eec462ab621194ffc206dd2b7600144e47f15c9b85deac4ebc969 +CRW_0991.crw 7535746 c280c88356a22abe6f774e7aff9d0547c77dd4d3a9baff292eeb2b004ddb7b52 +CRW_0992.crw 7759654 ccaefc246b6bc6b4d4b0d5b8432d0be7ba528c3a5c726b28516d791528ba4dc9 +CRW_0993.crw 7966576 b03692d9c1599c7f420797e574c139fb9730237f97b2effd6874eb2bf66d7447 +CRW_0994.crw 7587298 d7d544c16ac9ec90443163e282ece863d787f49ed5e3986651df27013d90d932 +CRW_0995.crw 7582732 4b0b475e60967ac41c986d4455f36f4e1ceacdc50b7aba1ed6a437bf67e9e8cb +CRW_0996.crw 7708632 30a2c7c8c34047f3a46a1337c96f6452b7f3e9b14b770c2a3a872ca0ebca585f +CRW_0997.crw 7484298 571c8876fd45d69eecec5a695dd22c2e242a2154dd90aaa36fe958e683f9d100 +CRW_0998.crw 6246208 bfe5bb23b34e0fa165dfc66db12c814814e28645ae472c002acd42e3d1fa5898 +CRW_0999.crw 6072154 9038d48fcaf9b9092f049e94940ea99cdde07cb18e8599a3a84fbf62c1ff5c6f +CRW_1000.crw 6607706 2968313bba7907d1ef10292cf3e182d1b58719b2e878415b28d4719147ead6bd +CRW_1001.crw 5757454 d2df3d736d7163d94f449f4101ba6e0defc30f188aea37a7cd6c450ff905c836 +CRW_1002.crw 5020778 797471869b457118d8c35c9956843e24dcb70f78bf42f9d7713143a5ec6a3dab +CRW_1003.crw 7786118 ac99f4ccd192860e4fa818ca86f982f3af36f01366f03bcb2a1c7c2b55d9f0e0 +CRW_1004.crw 7602426 75ac3beae172cada4cf1ca96e4cc1642a9a38d8ba6da6f750f364a3e870ee8ce +CRW_1005.crw 6896862 f4777e2812b832fc8fac3752ba76bb7b4056f6da82d1cd99256a9648a2108e8a +CRW_1006.crw 6740704 e3c2abb7671c7b1361e2654bdf97091c4a0df07e91925a1ccff91a6f78bc8983 +CRW_1007.crw 6636722 a48f7e92d7aadbe961f51e0f9d29dc64fd40ad21ae11adea938df7e646e24ffc +CRW_1008.crw 6976510 ac07d7ab288aa3cae7cfc0e0e682adbf248ec92263ca54646c5ef34490853c6a +CRW_1009.crw 7057462 120cff168e283103a060d60cea5fd45105d1237a4970f440ab79e17903531244 +CRW_1010.crw 7449572 beb97f17bbbc1573f7d1c471477b24433c61124c6ecaa6fd450783236282eac5 +CRW_1011.crw 6822000 9044f07f30064249cb5ea78bd6ba5d19ae3d2da278dbf7f10aa357872fbbd469 +CRW_1012.crw 6384960 96ac76dd9b2a0eadbf3ad45a33c6658ebb7269f1dcc7f39040637b8531a986af +CRW_1013.crw 7382872 95edd80147253434cfe906dc2470f2d3faaa6e28e65827fffc647f43b1b32cc9 +CRW_1014.crw 7015924 785b1eabac358f4f27102348006f5951c045d68ba72cdfc767b3082ede9655d0 +CRW_1015.crw 6314748 6a383ad3635ef019a78b016bff3fc98eb10cf0d058625dacf554bb38114509cc +CRW_1016.crw 6236758 7a41929df60eb08ae3af19bd8979f995c5ddae6dfd8ff5bf4f9d6c96f08c7f19 +CRW_1017.crw 6224542 0a7f89787f99120046df232d2cb88b26cf4ced82b95322780e2b6bb74ceebd17 +CRW_1018.crw 6349782 db51dc5ae54f433e6f21a4d12cc73edbffccb7ec82b66215eb734f13465a4f5b +CRW_1019.crw 6072726 380873b6f70c9eb27c9b788b964c9b426b98131c0d284ef8fb28d5fef2744080 +CRW_1020.crw 6132318 da7deeb483ece8eaafb54ebea0869bfd779ecb8e20b020c1d44fbc5cb6006045 +CRW_1021.crw 5923438 262ad171948a9b821a84fd4ba7c75c7a5fbf47e66e61adb892335eafd96db62b +CRW_1022.crw 6283912 cbcfc988431713f5d1ba9cec7b05ee95387b6fbf2d503f129fc286eee612da45 +CRW_1023.crw 6339562 d130268fae767819a46a649bc3e0cd7ff3c2c13ae0446ce3ae5c9dbaa00a846f +CRW_1024.crw 6321582 8157a2405164f5cbf167c18c684fe6f66a8a83214aac442d17fc548da19e7907 +CRW_1025.crw 6329952 479c85e11814381a7f927e0d059fce9d340955c70ddccb0f30632c73f302c2a2 +CRW_1026.crw 6574584 359000a7e593c3c2308dbbcdc86c613c81a46f7796461f5b704ea94224ec59aa +CRW_1027.crw 5944588 0b23241a2222d20cbf088a9dd04def15b5c753c2e847ae2666736327a32c9ac4 +CRW_1028.crw 6766474 1f47fff8009d1f868c922991d47f59a41cb59f434fbbd0126742fa32add00e4b +CRW_1029.crw 6463304 b246fd11125ad649f916ab81c423b96778e054ea8366aa73aefcdf46c65c5850 +CRW_1030.crw 7506504 2a2384bcbc7e223c5ed71dd7287ebcb275601e5e83c61a7675daff531dc1b6d7 +CRW_1031.crw 7938810 238b5d0610771fe66cba5bff104c97f00c96105596e60effd72c095f1f532637 +CRW_1032.crw 7283682 f3ce98a3cac34a02597257f55762dd8685bb9283ff5b0f420c2ae61c28842ce6 +CRW_1033.crw 6373626 9b60ee55629c36c7a53abbaa692c8bc6016e6bb2fb36e2624f83fd51e6458b5b +CRW_1034.crw 7212078 fd0f21563ede8577450593d26735d80ebd19cea5c327247e3f28b0adb8b9eb06 +CRW_1035.crw 7514042 fe1371d757d51255bbef108fac95aba588e12a4f56380e936b36efad708ceea8 +CRW_1036.crw 6834220 6bc304f143a410f121053107340a375a569e02ad12dcb61f41df0ecfb8c604f9 +CRW_1037.crw 6602224 2c3d6fc44afccbced4b03d547058215560fd5615bf89355b016a41a56f5537a3 +CRW_1038.crw 6697636 63971e8e4ce309a4757a31ff18647766afcd10fc24435bdfe95690bde5d53f3d +CRW_1039.crw 6240538 6e0fd3a2c4c857222abb3a6e4a17544d9e1f672969cb9b295d39de3195a79d63 +CRW_1040.crw 6049872 c0dfb94771104c97f04bb6e5879d8e1c33fc560aa6fbcf63dfbd3c244536b71a +CRW_1041.crw 6673680 fcab8ba2ae2b5b1cc08289e6788196ccae0c6862c1268b7ba11b479ced7274db +CRW_1042.crw 6659260 bd9d2b8dceb27aae2f2d5f3b1247a1c5f2608ab26019b883dda81561fa1bf9a4 +CRW_1043.crw 5927970 70977e018334e958078de11bcb9ecfbd6e30776a54708d659b48f5fa8f80634d +CRW_1044.crw 6341884 b3ba86cc7f61dffa9fffba291d072207fc09a3f31998315e06094ca475572f2c +CRW_1045.crw 6472238 ae09b26f85dfccd666ce5261937c2b305e6923ede4a20bf2579e9af47501b4ca +CRW_1046.crw 6604644 d029a061b16e57f423b185675ba1f4e2a4d2b52cd162b377af450dadd204afa7 +CRW_1047.crw 5814138 c2951516d7d3ee3e161f56447351d68916b3c092cd4401423bd73ee2ae055c6b +CRW_1048.crw 6620120 6b277abe5348d56ff576d395d7f1b9ab42feccd01937f91fe8ffda087f40754b +CRW_1049.crw 7055198 8a7bd78fef7129f3ccbbe7e5e06eeddccf5c1a523f8a48d476e27696c6ad683a +CRW_1050.crw 5934524 0cd492921a53a919ee79c6269b55916e03c11fce4fde158e44fe05a48e9c8c1d +CRW_1051.crw 5849890 89d42df2ff2c5c4ece065a864715422f696b97120f18b4f68ec637f5406bec6b +CRW_1052.crw 6434760 89f2cafb020354096fe09b7b0b63d3f5d0914c448e02894c0a0d7fb3ad19db9e +CRW_1053.crw 5835494 b77a302f807273e65eace4aae6d0a47fe325ffa90d92f6df41bef1022d0d9869 +CRW_1054.crw 6356800 36b8a31fcb38565935cf7ddbd5465060a99c778cc8cfe5cf97d2f6ba4ef808f8 +CRW_1055.crw 6221326 d882f504c8a71917e6dd417b096e89dbf9f379ad5494a658f5cf7c05fef6b311 +CRW_1056.crw 7012412 0712abfae82ae30d5ec40a514cb195df88fb51205c13c2d106adb79632b4a972 +CRW_1057.crw 6171402 bbd107e3b61c16caf1e968fcff85f8ce076a9358bc4733029b6fcf4efd46262c +CRW_1058.crw 6097390 c689cc9b1eee92a72a2c8072d1821af330ea3dfead64a5b70f554ff0fcc1fad6 +CRW_1059.crw 5707598 5749f294c5e1a886df960887b77c8b3334924fda9cd1a0499c618158b3e7306d +CRW_1060.crw 5656070 326f23ca1171fc93b4e544049fc63f9eca7f3d67d83cc08649d8e5dd886668e5 +CRW_1061.crw 6569902 6a66253fcb6330c2f415143d42942a4d5575cc051876c3b243fe1a9c4176a3b3 +CRW_1062.crw 5966918 add204de2af67595135158f1db4375e110b48c2d91f226651e1729acdb1dc65c +CRW_1063.crw 8068470 bcd4167761c56a0f94050ade1df6bdf8b6fbd0c8ebc7fc16489ff55413572e45 +CRW_1064.crw 7107778 ffcde87b869d00891c0da921a0c695bb33bebd46c11a7f28f0370a938541de9a +CRW_1065.crw 6621602 c9ddad87c4973bbde244c06852b8d854c1f7462e8b536c13f6da388c509763ab +CRW_1066.crw 6889494 da08657a251b78fb7cdca06d99aecbeb34e545f33cf5551958dc7039f8552192 +CRW_1067.crw 7083346 921a16cf782ee751068c0fe9453151219f31a04b1fded1af71ae66a8f29c4af7 +CRW_1068.crw 6904428 7e0b7a51a0ef0e9a0b6758dd14e589d32aaa1f62a538bd7e24279d379cc87038 +CRW_1069.crw 6732366 30758b6bd5ee5a30792d9ba905fb7e120fb2e7b7636b3ebace95fb5ccedd3173 +CRW_1070.crw 7921380 62e9745e2950ad3840948c5d9519e0d9eabb8c4e9515bda53510f81abaa25561 +CRW_1071.crw 7045468 b7820d7cc02fbac5d153b6cc05e38e238bfab5563aa9a6a4b02c507efd7b75b7 +CRW_1072.crw 6769728 2dcf41af0b419614aaff5d57eef978727cafb4b0d4c00f16cdb4017eaac67b37 +CRW_1073.crw 7112908 13ea8d964cea4cc43bb2d8c76dc09150334c7df42e3063192aab8b9326e79ed1 +CRW_1074.crw 6618126 e9b76f2125d561993573e586d595b3f742fffff9d7fa329fc0f5e6ab61950a38 +CRW_1075.crw 7099328 b508fa087213b471808e7488bfc61470239642de981d8e7c940cc06523ee15c1 +CRW_1076.crw 6340522 7cc4e98db7c902e121914a6b634c875c30a5a3c6c8bfa95991b87edf68c7d97f +CRW_1077.crw 6787764 5648f252960d06a2c4688b8ac0ab74ab42d880fde1876ff83cf9b8513851f6e3 +CRW_1078.crw 5447356 a7fd5037923f65448965e16bdff0ba793a81f911adb55337c4324db7ae70ed82 +CRW_1079.crw 6809466 75282f602715c22315a6ab1a7191167780edc74b5fc4ded0d9290722af16b9ce +CRW_1080.crw 6466024 cb318b550e1e10b059e4de49091ae874892836175646e61e08d39d59e96b697f +CRW_1081.crw 6396352 9273affd797a847cc3e63d7358245c291d5e82b64b474464b875e5b86fddffbe +CRW_1082.crw 6040192 57e9db31b3f8b3f346c6a01f1cd7838a944de8fb84779b371c13d9acd720675e +CRW_1083.crw 6343538 10a39e66551b2cd04bd8cb6158effbcd78e60d2340d70031dce578bba3cdecbb +CRW_1084.crw 6280028 b1cb1990bf5740f0d42ae56c8cbc4b0d3bc8622786d9ca145de45535be14695a +CRW_1085.crw 6080572 b121872297204db5cfb302f3cc33e9e0c9468c7ce7dc8e26414f56bfc38c5711 +CRW_1086.crw 5668640 513c66737eefbdb140727a20ef19f41de4090ec57e87ef5fea9f9dd941e6463f +CRW_1087.crw 6083524 22aaa170d5fb67539d4eae44763db60dec1593e1d9656a08c34b57f493b633b2 +CRW_1088.crw 6100260 902c24019cb41a88059f47e209e21a7b47f6ed876869211c74db23ba25e5c323 +CRW_1089.crw 5730756 ee0c3ad87387661da09c801c5484be5553bf4d1cee82e11b6cb66bcc30b1feac +CRW_1090.crw 6914006 3896090d439d3788ea41cf160d06ad5cd16751b25e75d13ccfe6d132e5bdd2ef +CRW_1091.crw 7459352 7ca0635448b719f285befb5357433a66b0c5e915270acd12b9f92844953c1d9a +CRW_1092.crw 6114072 86e2c4e55a444bcd67d955a03fa8bf2c750a3d4bc1703b7ebe6343ef10c34af9 +CRW_1093.crw 5998716 0969d73b57678565a1d008c48c32f568c3235669aaec9e5256fcc8fc1357cf4f +CRW_1094.crw 6350954 a7733a8cedd38cfeaa903639d588d2e5387dfef22be1a63c6b39aefa7a9f1583 +CRW_1095.crw 6332160 40a9fe8210ea53881910e02d09e1de7181d74a478796777be76ede5d6ea3cc7d +CRW_1096.crw 6599700 24cb076e3267f59656beebb8bf4efe84e6853b35b35abad3591cf65b5becf9de +CRW_1097.crw 7194576 21e8a86e833528b854998f564af829b104f7beb9cf0277f4a3e741bac209dee4 +CRW_1098.crw 6860602 9ee6d07e344cd1ead6c7bff2ebae8d2962468172899b413e1f7ba7ee9134e888 +CRW_1099.crw 7198356 3e157ca8e3a6cbc8c3dcaf97c36cfe0ee01eda7a9d7f4b8162289a05089e00e1 +CRW_1100.crw 7902164 6d69fa1247092bc214fdd163d8948cd87776654edf121c998ad9d62b12dedbd4 +CRW_1101.crw 7941076 19c0e1474c3495706e414a84dc93946941350a053b7db599b8b1f76a0c07a08b +CRW_1102.crw 8137636 6077f5005793cd895519575f3559359bdae4639a4db6a6a5e2a0c0d9dbd5d4be +CRW_1103.crw 6613732 4ae7df22d33cfd09e38b9e5998a37de4c7e85058c60e26aeff3e52328cc6a053 +CRW_1104.crw 6644212 c72ecf2637921b55e22d167263c13e1a50079713da662c85ddf3f2d014a9fb8c +CRW_1105.crw 4886132 3c124b8af59b3ffdd02fb52b0161e16de9ef4225be5c253b877ef9e53e59ef42 +CRW_1106.crw 6338270 03e6c35b65ea41d5c812513be14859dccacb35ad18e206607cc33075dc1f449d +CRW_1107.crw 6112058 dcf2f5cd339089c462bf4cac0faf5e50fc82c1ca1eb870e47a4d5b4332a98c9f +CRW_1108.crw 7499154 e9d17ec0949c09852fa87a6c1d0bfa702097ca4b4dc88a2a027f7b62f799eacd +CRW_1109.crw 7284994 33a0960bba2180319c0cdf9f97000e5e180b86df75a32aab19e18443aed7b626 +CRW_1110.crw 7543374 74f1e973f8dbff526b1f04df5160ca44ab5af845e5f4622ba9099cfda4305ed1 +CRW_1111.crw 8454118 5d28e85b6cdd3c3050f2a6afcc32dda8ae5bd41bc68f80fb3a226176359d69fb +CRW_1112.crw 8675074 da016b8d3d9561ac9203d4cbafb9efb45b6299a027feb409b4d3f8b8c8577c4f +CRW_1113.crw 8827580 523ea7d4dbfc8b7cd3caef431a8d39607fabfd7fd476847342efbb58e4710c04 +CRW_1114.crw 9167540 de535ebbbcacf15c1e2c2019409abf3f9a3c4a661f7eeb5454e2551859032c7e +CRW_1115.crw 9153908 1cb60eae43c0e1a1566565639ce2d7c0383979c4c017747fd3437d7dd08fd0e2 +CRW_1116.crw 9106960 ae930c21fefb73559902f8c15768e8a3aec0a5ae208523ae19691fc650e1ff6b +CRW_1117.crw 8981110 e0a4abe726a80123314f57909a65f84728af488eb99f8ec31cc03f7baa2cac33 +CRW_1118.crw 8887628 f73775d5aac62c5f66c115510c62252a8858db1ec8965e72b8f40a3eae3c1467 +CRW_1119.crw 8964760 7bc481848dea92e5cf27ebc4e27b39ae7a95cc86ac8d0852053289118acd4c96 +CRW_1120.crw 8971278 f802ebbf59c3e9beacc47415a409ee0ce9bcbeed7892cebd4cef0991fe34ac27 +CRW_1121.crw 9172110 86bbcf0aa561e5251fee97d36ad1ee97e7124abcc077217efae83ea056d2d23d +CRW_1122.crw 9175174 dbea4fda750ba06742d78170a11fd4cc09b4f581e509f6a59697177d805cd050 +CRW_1123.crw 8969636 3196aaa3e0b94e95346925ef7ae60e73097a2eae267041e479b74c73e49943e6 +CRW_1124.crw 8095876 7fd0864d1c7db470ea0fa7d3a96b3a4257e8fe7c59922847446f3e2eb8319102 +CRW_1125.crw 8440126 ccdd2ae6a554b4cee4f87f9def9d58f1c0adcfe0d7ff3cb26ae4959ab652ddd1 +CRW_1126.crw 10378228 0b3cffb156e29f6ece2d298e074dc981afbabb111c995c12f1c084830734957b +CRW_1127.crw 8949062 54e30138c902e897707d37a333a392fbf5454f142e741edc4060b0fc2bd25237 +CRW_1128.crw 7594910 f5d1e69b994208bef70601de2854966952b62aef86989d60619163b281c19927 +CRW_1129.crw 8169200 b8e9c7f631098dab1c029d14f6f82da0feed7e97c360f0d827d0bd902d368ece +CRW_1130.crw 7594462 9022b0889e787c843cfa99d054db4355e01357b910ce2ab511f73622abdea2f1 +CRW_1131.crw 8070708 392ad382b77b9eb03381844ed9763c6da192112e1d21cd0666626971ff0ee596 +CRW_1132.crw 8818846 2dda72a97aec4f3b8bcdbfec7703d433a513039e0261cc4c81a7a2f46ee3615a +CRW_1133.crw 8834292 8c661dd09924fa564eec3fd1d156afac6d94e224ff46d87fafb160828b2caf5b +CRW_1134.crw 9266176 69f81cdbc5e678afcdf66318e03f2e8851caca79ca8b7cfa38d760cfa6c525ff +CRW_1135.crw 7004520 4f0596a4cf8c98432f455eb444a1d54e136c6a8104e5f390250b88f200df4890 +CRW_1136.crw 7047494 65d3639f756842960e92a6a166656168a0579b0f2b2a6ac4ecba932675eeb957 +CRW_1137.crw 7251826 9a9bba7a492917c164a8eb3ce1edb40037bb06a7a5c1167fa5c1a1296263f22d +CRW_1138.crw 7285118 fa98b791d5c775989c48ff513b7bfe67ed4fa8ddaab94bd4d927c97bb65aa4b4 +CRW_1139.crw 8153430 3c4d3b0ec717448239bc0c08c4a2dd7abeadedbeba99d63941d80b597c2ee6b1 +CRW_1140.crw 7001218 62f03f1c151eeb5a0edeae7c6ec2b2183c6f31144131b64a3c854de4d79057ef +CRW_1141.crw 7082946 e0e5c06ead3a6363ba623558ae381ad550e91c6460902aa627846583817a6500 +CRW_1142.crw 8560524 00566f5b2287644d9d1b0f532b51d87b4c14fa5143f357d938c82989f1772907 +CRW_1143.crw 9397440 18665c412db6451fabd254ad90f2586918006d09eec14895b9ee45fb4053711f +CRW_1144.crw 7504220 d25f18799902d82d3503314bd5aafa5e791cb08badabd80b7515ce559bbdc0a9 +CRW_1145.crw 7975248 d1f6de342e6a63a04241a09fe3a695387180e237c0c6c9ecf28a79b98e4e08b6 +CRW_1146.crw 7401952 4b1e8ac057682e1d446518419dca4b9ce02f69def289c52334ee9168f7a85b04 +CRW_1147.crw 8791278 c05ab1fb6fa25d63f297e1be8882f2751a982bc64260c84ff6975246254a8bd8 +CRW_1148.crw 8533322 799e7452ead57090ecbfd153473c11f6b6f89a493a5de1f1c57030eb03e1022e +CRW_1149.crw 8480430 a9f8e190fc77af7a5c5579f6d19285e49da6942dbbdabbd454faa9afa4c3f9ab +CRW_1150.crw 8394966 80c73bb045db7131936a913e02a554d487b4792237ce29ba8f426e79f94f7dc7 +CRW_1151.crw 8845948 5e0189ebfe66240d2862fdf2647ffdecb7efe2c69c65fdb14768c95116cab09e +CRW_1152.crw 7715120 78bdda7879d6eea13f5047af89e86d861d920455ef7075cee3394dbe466c46f5 +CRW_1153.crw 7729790 9cd504ed0991c4c6033d72a2676c778b84d198ddfad202437fcb7f014fa15a17 +CRW_1154.crw 7050252 8ca1837e5184eeb195efd82519ddddedf65a4d62538cd05f2e87d3e76049d74a +CRW_1155.crw 6825966 5ab5ce0f1f5d2beddd8222cdd81f7656f41edc52ea1934a12e83f58d3c579f7e +CRW_1156.crw 7034616 856e9500fea29d40240ba7e4899411835806021fd13a3a3aaa5b6cde11680fcc +CRW_1157.crw 5922612 bf71e444187768fb2c85523f6d8270a80325e7354914a032c2ead4761c9774f1 +CRW_1158.crw 6083628 9656a312ce8f3ba24d82d3aa23393995ea5292e0d7da67e14bd1bba51f759504 +CRW_1159.crw 6317264 b928371fd9dfd54d6e1d277bd5b950782d3d4fd6bc216622c61796e4bd1cc42f +CRW_1160.crw 6852356 0d2802358704963acb36ed6019ef181fe3c4620d6640e557496ced5057b7c58d +CRW_1161.crw 6847736 620148bc3e7043b6399e88dd26eef640b9512634a81c3db974a424b5c396b826 +CRW_1162.crw 7138460 dc7d3e97fc525c653d35e14278730bacd196b63768a67c9975cca27eeb0d1bd3 +CRW_1163.crw 5922892 73d2b0834f8975497560e0e2386b53489758178c8754c204556bc647ce3c8602 +CRW_1164.crw 6755376 c51dc1d162463a61e867dffaf31f4074a841454ea380bd71dbccc50e8aca574b +CRW_1165.crw 7050064 f5fd1c492f8cf045671589f0daf0bbd7f4659a9e1e36b72093f4ec665b1476e1 +CRW_1166.crw 6676644 0fd0f14c1956580c2c1ada3a94c63a626a15cd53c53bf65fd90be1e9841d7218 +CRW_1167.crw 7020086 059de3009739243ecd68ca7e2ec39b1a0b2fedfbf1bec5459a71456a4878bd11 +CRW_1168.crw 7454436 438fd14ac7c3f00aec3fca294438b52c3faad21c464b805d95e8d97e12b61edd +CRW_1169.crw 7566004 cb14750c6f0904201bf39e2397a1a9e5a5e82ccaca71b8476d4c8e5267b2f38a +CRW_1170.crw 7697734 6fa90a1cc2160852f25f6b541c92b338f23216da6e9e89f43e6d1abd30a72f6f +CRW_1171.crw 7807718 fcb964db7a1fbfa9b70101e82363b1f1c9084046b78b0040019b4b250b1b542b +CRW_1172.crw 7496624 4b8bbaad6aba14279c5114401b6145b5870b6ecb1cac346ef339d5671365a510 +CRW_1173.crw 8087388 daad4183174e5e27da7fc0dcdb54c91c58ebd64e84f964acf5d06f52aef50094 +CRW_1174.crw 7160200 73a89b9516d1b5dc53524789db2dff37f023a1d4af6582ae23b415d52378bacb +CRW_1175.crw 7208264 be064c7847b52403d2d9b038789ac0f727e9afbd993987b37d9053b2731f1693 +CRW_1176.crw 6889550 661a8a98550ad421b1124fd51575dc402fd215e6c5a800d578f1dbaf2e9da021 +CRW_1177.crw 6591382 153b157c942f74ab1b9b33202b79d09d84fc6704bfbbee8d4a5b859ee63f550c +CRW_1178.crw 7128576 767986e2cb6afedf011663fddfd055a2bb20b6bd6d5b3ef1f816ee645c3affa9 +CRW_1179.crw 8188078 3d8a082bdd460d2c1a130deb328187c3dd6687f9b0d61c1304f019f026d98835 +CRW_1180.crw 8217302 778ca213d95e6d601c1009da17ca7f5e7fedab324aef0ebf9d9131cf3b8a8137 +CRW_1181.crw 8342888 dc5b07e1b44abc3eae24f5293420b30a6828b5483b7db18e40e93b6d97d25fed +CRW_1182.crw 8228266 81ac20386f0736256c7c5e403ce5d692b89dd3aa7715457787dd5446fd05e99d +CRW_1183.crw 8230526 b2ef3b7fa694f6740b9f37149bb9a8086088857dd47d631882a1f638926adfe6 +CRW_1184.crw 8190786 cfacd6b5567eadea5784dde7989cdc13cd20116ca36fa5b18b8f6249f5657fa3 +CRW_1185.crw 5447806 d65a7037dc5c338fdd696f9c94e52ab5c5c96800a744ac3ca568ccedfb8fdb2f +CRW_1186.crw 6739154 bd36f49c17f769850efe66872d550b21d692d702e48f82b38abbd15023b70272 +CRW_1187.crw 6248272 d32e1723131d54bc7e2bff8fe2df34300e086f1e4522d5e78fa63967e1856a67 +CRW_1188.crw 6012138 0562fa8ab0283ad345ed74a63299a0da3bae1960cdf5afb8800688dc180ab185 +CRW_1189.crw 6299604 eeaabcfd76716112a02505c31dba94ffdab0981b7709844fbd86e695824950e4 +CRW_1190.crw 5945734 d15e698b7dc67bd31e95c8342590e8c4fa0cd0e64ed7833466b91d1d6d8bf99f +CRW_1191.crw 5661562 3fbd65409462ebb3dc3877a8e49c1217e47032cdac536e003ff0c02d5923eea1 +CRW_1192.crw 5917988 cb76f86751e844f5740ba32c1fd4bdaaa2746635727c4daf3d71b63c44067a43 +CRW_1193.crw 5915842 f5f2b990322b67a906bce6317bf9a19d04985884766e1d5fcb236a8e2b977510 +CRW_1194.crw 5775446 d8a3cf2a9e322aaf575e4097a0c27583886ff19b0a1d1e943a0f20dd57d9a927 +CRW_1195.crw 6543090 fc8ffcf5e15f9e8400980dc11e8532598ec0f877fdcb289d741a986c4eaa283a +CRW_1196.crw 5915808 3440b97a1275d310e46053d6f6f3ad653378cea98aea7e7dec2060d7c84d1024 +CRW_1197.crw 6087906 3ed85043936e31bb63a8ab0f23fce8fe8b3ad2642b37b77e26691a9c1e4183bf +CRW_1198.crw 5553708 ef083a44281abf7ab4732b61e7519f3dece45c96a8fafeb6d4e17eeb846e80e8 +CRW_1199.crw 5531220 8ac79ef8b2a0fa941f9baedde3e1f44e74b69bfc6c1de5a181ab83e6047755f4 +CRW_1200.crw 5541534 885dcede2e1fa7d8354599fec97dca5f2798c988e72481d04702ab279f1c161e +CRW_1201.crw 6504606 49585fd34fcdc0eff6c5084642b7a3ab20fd122a609caba98e17a10caf0100ff +CRW_1202.crw 6445992 188d5a63c9bd9a51b16873072ac6148cbbb9028d916fcfdef49d49c4cb71958b +CRW_1203.crw 6425150 4808666b4f1ba41b08acd6065709333636707092ce00c4b8b86a6cba1ebd4f8a +CRW_1204.crw 6161614 a1eed96d9116496a2cb4ffaa7660c436415bfee7f2c35adadb3d0187a15a36f3 +CRW_1205.crw 5931410 cc6f860c034270e5d2089d825322e70bf9ce22ecf5420db3948225e46e46e81a +CRW_1206.crw 5996318 4d36059dd11088675d9b2b30cfdf3f021cd0d24b3df3686eab5be310ba23f399 +CRW_1207.crw 6057826 e6b077b0a93256b17cbd8bc035c3ef85432f4e7b926e6118a031b32c10a65b99 +CRW_1208.crw 5776034 579199181fe7fa562ad52add2eb71d95aab07f06c420f043cd669ef6228a77f8 +CRW_1209.crw 6186290 3615c6e7d5b19716d7edcd3e3b16f9c4c6b35a2862962b13507fc36379a43c87 +CRW_1210.crw 5804094 a7b966934fd6031d9579510c1ec26e83cdae78e017825bfa016e77d01b0e0a86 +CRW_1211.crw 6058528 3cbe3223be28b0da4dba4b0e8c2022e7f39e773c2d6f08bcfd160a253c6b3439 +CRW_1212.crw 6034034 98f52adb76099c2fa30afbdf6252489c4abc6b60c661b704e6d0854d712fbd4b +CRW_1213.crw 4644198 76d746ddeaefadd792478f9235c78a9a715f9f4aa143bc4dbda7a55e2e5bbe0c +CRW_1214.crw 5122896 1b1973a8c4eff2d7950ba13ce5648cd44ed18688d5434ab7eaaf906ba01f7087 +CRW_1215.crw 6223472 9caf303d40b9e9e2e9b34e9ff696b16cdf8861ba8f6513aef595d16545aac82f +CRW_1216.crw 5469112 4a70f0910770d53dc252b6b6aa03bb16212cb1300a20bcf1b455621e0652523e +CRW_1217.crw 7470146 1d7ed12e681c922fb15f3cfd3edf2fd056e73a0be84895c44bad235744600614 +CRW_1218.crw 7792308 0a976bc4939619c478a9512a57046e9186f1db52be7775f0bc4ae027c08ef7fd +CRW_1219.crw 7234152 5b25ff207298ddb99eea4695a281a3a157adb68ff73027b3f6723b93166023c5 +CRW_1220.crw 8180202 8520b012d28fdcbed10ae38d5597a3f836762aa4ea7ce4cb2f177af6ef79e5b5 +CRW_1221.crw 7839496 fa30f4aa48e230d6c5fad3b949800c57844e657ef14e928d902e6a0f3ae67e55 +CRW_1222.crw 7130008 ea5997b019fede1bc2e0d9b36fa66e49ef706287b88704f6343d538a90b57d50 +CRW_1223.crw 8489060 046f68cd7bb94aafa65bde6f336757a9f4c1d6dc67728ba79b85f6f2bf6fdda2 +CRW_1224.crw 7117646 b756363512366d5b1dd8fae65a45a7bfe10dca7c39836b8e211a7965cbb34d7d +CRW_1225.crw 7733822 4b9704b34389860d6c0091e1a36290e4d95ac55a77626d94c40368ed432188a4 +CRW_1226.crw 7223810 288522afec2be97ec680cdcf389dbfd243a8b4b9e72414dbdf8ae24f7ea3a500 +CRW_1227.crw 7315694 c2be6316ab2ed2e24c8d350590dbb2a78e863e1d22495f43700cff496ec06d3e +CRW_1228.crw 8472046 3a3c4ad8cbda1d69fbca84dacbbc3effe8881cec0650798d4b3258c4a073d7c7 +CRW_1229.crw 9110078 5cf4f3cc0c49c9f01995a22c4827f1617652d0d6dacd357e7081cba64830802f +CRW_1230.crw 7297020 4ae969b9a3ecc05e7d16a42dcfc1c70c15239d8a050e5dc05ab26667c13ec5ce +CRW_1231.crw 7784706 0fb897d181bb148f581b3643c437901e0db7b106f32f03258976c6a97f610666 +CRW_1232.crw 7474926 05acb6fcdee01a54cdfcdcf77863ae932c8a2634205301c059bed47cf0cb7d94 +CRW_1233.crw 8273148 11b0f1fd0008ab26cf2a2262e035f63b651c03b0eb2b17e46306025311135398 +CRW_1234.crw 8676392 968616073f32f60026e6280b4f84dfd9e89740d1bbc7abbad4fc364bf12f1828 +CRW_1235.crw 6676060 c96f235e44404b71d8f1f315fca16fb0b9d01a9a2821f3f92246229aa8add510 +CRW_1236.crw 5729434 f4e5a3b6ad63ba97dec223207fffc37b8da31d41255754f824e6d3846c07bd10 +CRW_1237.crw 7192844 4ca41e3c3d316ae85235d073b0077ec88f6ba78529fc6c9484fc6635b402314d +CRW_1238.crw 6170872 67db92e504d93faa53be52f949eeb645b9dc8dec08329a247e4389080cf2f2ac +CRW_1239.crw 5919450 186809e974b40c45b1495a9ca2d66aad63ac5f8bfe191ea670c7ca16038b17db +CRW_1240.crw 8719542 95926f7da070ad8603e7ec3bde27d25506cae8fdaeb04d7fec6b8a71075797cd +CRW_1241.crw 6135512 884840af43a500b2eb9951b0d909617d5e101e5f1ff88ffc8f8ed7894539b0f5 +CRW_1242.crw 5703778 3b90f10dd942a8bdfeb23aab6ef27618b9430012dc275e6dac9e1ef47ee85628 +CRW_1243.crw 7056110 593438d6779b5f27275da84a3b6c3bd91bb5478df7deb8e3b3d0edb59483fd35 +CRW_1244.crw 5390116 20c502e7915556cadc5973948048b01900506527cb8376f7bb2b55743cf9b90e +CRW_1245.crw 6787804 876cf53591163de5209901da46fb2d715e242a26ffa47dc36e48c47f6706e679 +CRW_1246.crw 6335854 2d52eed355005cb7d4eefb4badfc4b5ef1fb9f2bea14778715547938aa7a3b20 +CRW_1247.crw 6819454 e2866105e43a365e11ecc62c2f6ac59c57f6dd72a721be31ddd1ef2f69d9a59e +CRW_1248.crw 6632016 034b3c9062e1b91479b3547ddc41fd1193c67c72bf8a7d4ccda7c4ed79fdb6c2 +CRW_1249.crw 7117434 9ed6974c08c8fa81e80a7e24a7dd134cd5a3b44b6314e35843be4a4c4c468ff3 +CRW_1250.crw 6036014 dee416e3a7c554874e04aa77b9f7cf2d972c6e459415b5f2e6d1f25b31d9212f +CRW_1251.crw 6991384 78033028d7a3b7c3b737014563ae4291019b19e171e8a7138ac802cbd7c3cd04 +CRW_1252.crw 6495458 cabd98f4d1f2fe715673cb9ff5850ee8d286d8f0f0c534a5772319d7ddd5a096 +CRW_1253.crw 6057288 5402c055c78fefbc08114022156ac6d6a2f811887824dc60927459737d228e96 +CRW_1254.crw 6160452 583ffb9ca6dd14ef1801a37290fcfc283cdf53da3a8432effaf190ef3ab029e2 +CRW_1255.crw 5654030 4b03077d9c3e2417d2264282327736a55ea450f2e5d9b1d7e8ec859c83bf8de2 +CRW_1256.crw 6332946 ce84bc0e78f20001635bec275325c3d1983fb5a751955a84256d45aab5693b6f +CRW_1257.crw 6221272 3c5d150da1a7ad3b497c9411e891b8333f2656adcd2e1e72193c6076de4738f7 +CRW_1258.crw 6112340 9b4f6cc4c640bda000f23db7c2b9906f6806dc1595c039ecf21adef4cf1b092c +CRW_1259.crw 5535828 b416f17b42e52e621e4516e07cc3ff4422662bd63b36f79d3f774eee3f7f884a +CRW_1260.crw 5718858 f23597ce665d633b7f66c49354054e7494f10448ba18a95c260f62ed50760ff1 +CRW_1261.crw 5727666 9ca3efe11067762cd64ecd47bbfc4efb5ce3db545ac91db6c0443eb5449b6f6c +CRW_1262.crw 6401556 00f3eeeaa586e9d15bba43613fd85a0c711eaf4f86bf051fea58d4166c08327b +CRW_1263.crw 6621930 d9ea71031f4d237e4c984ab797154007d10464ad12f18c2a2526713af6940dda +CRW_1264.crw 6500868 d88c163563fcf69d6f6e1dd1bb5b26f93f06d2cde0014c429edf79c435545239 +CRW_1265.crw 6973080 69350bb742b89569f86100f904198d284436346e47ab448559ffa9da8ab5b7fb +CRW_1266.crw 6288150 93e2cefbc8e547f724a86dce906254133ba1afe90d30a9a74a56f44bcd67af23 +CRW_1267.crw 6593326 b5dbca93022fdaea7104108f81fbb8232a6f26ecf5eb2572d26f0b6bd30d1ac3 +CRW_1268.crw 6147704 0400f4bc05cd64ba5f309cbab1629c11d007c35518c25e654c6c9fa4de3841ff +CRW_1269.crw 5983794 56731e7fd5042ddbd44b083bb5dd14a159f03e92272b1c058abcae93db161515 +CRW_1270.crw 7406970 dd3571af5dd4f75d840038e359b5a58890dcf52d9773600269a3ead10ad2b3af +CRW_1271.crw 7091586 6fb9709968f62ceabeef387a9c10d21e7caf3394b4d86e8e48096ec749f68fb3 +CRW_1272.crw 6068640 ecf138469830c51430696672d0f8efea52c1bc21ec292799a9f3631b7b80351f +CRW_1273.crw 6173290 2e44c04388d3145bd9aa018fb9e5e7536d2791650f89dd69d7ceba2ac8ef2b41 +CRW_1274.crw 6652380 cb051d3f880de0a9ab759e5775a0466279fd93b3f9e488ecbf46ae01a01dd94e +CRW_1275.crw 10363494 7aac42930c5b22390e152f5b967ad9ab4daf85cbf36ac12dcb319e2f367e374c +CRW_1276.crw 6189602 eb7ef1c39941c8fe22b2e6fc55abf261007a10a377bccf09876394e042ed16fa +CRW_1277.crw 6850414 c4f0ce8c4ce85d3668b3726c46018d296c784b10a2f0f322acba8b1681fe86f4 +CRW_1278.crw 7418696 19b41f70dc66419c107abe7d7761b6ba37e412a4a457f115d4bb996688f83f28 +CRW_1279.crw 6523864 bd3bb0d9c000858ece6a68d40c4d9537b8e834f20873025453adca8372a29d92 +CRW_1280.crw 7778354 501608ffe3c5899fd60f1e9d6e8526cd8b1d5b35758c378b6901d4ca011aa850 +CRW_1281.crw 8715152 736e34e192858ac758eda9ba7b6a92a5b2da3beee93eaa92e459a197df842775 +CRW_1282.crw 8003106 964747aab2f41c629a4eaf34c43e7f5a594b1b059579149040149b7db4d86415 +CRW_1283.crw 9687894 2e08c34e71d07f5f9a224fd9f3dd2e8a0ecaed60d505200967badd0b8bcf06d9 +CRW_1284.crw 7755078 e1ac4139b3609f0be36b327345172e160130098aea77dde5482483907c8b4396 +CRW_1285.crw 7532808 7d3556ec29a12e9b83df5cac7386979e59ec3aa3c496ff24ffbe4dd67b060f3d +CRW_1286.crw 5958184 9051d20f0d35b6a640f941fd96500334560c4ad46b62ca2c1682bcc194232adc +CRW_1287.crw 6677154 7e7c80be543d08d0d5ddb6b6a4928cbde2f649ee1bb42c6ea1c57b761e5d94dd +CRW_1288.crw 7539318 b9fcdb64b3bb33846d08e1600bc5fc972c3e94d7950d78528065125001402337 +CRW_1290.crw 8005344 db3c3da35f864398218ac0485b363b33421cb4ce1beff4bfb90987d08cb321d9 +CRW_1291.crw 6137302 2f9066af57e2a293e9f0a9905e115bc2cfb2bab2355a8ffe2cd68d28fe62c30b +CRW_1419.crw 6381388 4873f6a0b4656e17137a3a7aa7947fba5a7c8362580346674543fbec20674eb0 +CRW_1420.crw 6188476 6b3a70ba5c30478d915254511c2f1be173ac547e666adb8da3d58d635ab413f0 +CRW_1421.crw 6097350 84de2e68bd24a65be49b026357b735cf629c7bd868a0167814dda784e10feb62 +CRW_1422.crw 6119416 2e298c38b82bca240bfdebb1d882ad56c711ccbf8fb75b7877294e98fc70eb99 +CRW_1423.crw 6316574 e77b068f36b3ae010dc4d03ac7849cbac09cffd96ef5929194ee5edc9941d7f5 +CRW_1424.crw 6360256 9d6b592773e64c45bc338a1013b90ed55321bd46fdb19d5cd6152672ed8fb5c0 +CRW_1425.crw 6674950 df464954b960a85385950de964b5243ab9824ae3cd1db904478584bd16956578 +CRW_1426.crw 7031348 8ca8831086766f9a38799c75d8ce9a2f185e0e21a6fddabf371122050010a6dc +CRW_1427.crw 6654038 4620be24f95c68fbef7e7acd865e341767ee32a9fcc571ed6cf7622ecd83a50d +CRW_1428.crw 7016436 2f5588253c9ec5aa034ff234ebe5c4cc562c34ec9e5044e5e96082b801edaec2 +CRW_1429.crw 6677328 ea3f143f0b1be271f959651dfa42d385048a2113a1af493594f7e04cdc265ad6 +CRW_1430.crw 7172330 ec5c743fc71b842a8d7787bf4100a81fa42da33c1d4b4e8179088d9107bcd2e2 +CRW_1431.crw 6190592 8bcc9edba3cd901e69cacc22125211258afc1defee9d81c0cbf49a0e120b459e +CRW_1432.crw 7723112 7c0cf509580957bca38fc5c6bde36c107d8e7668b3aacfc470e64c046d223e51 +CRW_1433.crw 7804982 34037fa0a6f7c8219595262edf920e517c859badcf02a17a97edd8388e5cb774 +CRW_1434.crw 7012102 b96766d29bcd3068e910e2d9faaf6a208a4caf2623d8dabc2f556169984ce7d0 +CRW_1435.crw 6990698 c03496586933dde7416aa0691e52dd37b3d26708f8c8e20c3bd4891e40f06423 +CRW_1436.crw 7346052 912c83b5a4b67d9cffd31ae611a328ae8d410733e2abae058e03981df24412dc +CRW_1437.crw 7433126 33e0d309bb2ffbbe3f1232154acc6176d5578fd718ba741e9347be554f71f78a +CRW_1438.crw 7329026 a6ad6aeec9490985ddec72718c00147fba29d36cc7d53d8601a3d7fd246d5e65 +CRW_1439.crw 7567540 ea6b1872bf4e3f181b53be38f2d3cbd1ee02bd6760c6f6e6c7c4b0feafbd3f84 +CRW_1440.crw 7329634 27d0b253f1c0a3cffdaa5d2e8c13ef3f30141d88b509485ec88fd484d762ee2f +CRW_1441.crw 6428456 dd33b1640271c986088cbcac5ae033a8305d1b53a327e974e6a7295244de2d50 +CRW_1442.crw 6779548 a3f689ae44b0a799e531f1f911d5744dd6736cd4cd5125c11e7fbf1cb008278a +CRW_1443.crw 7635238 3b0faeb10c382842200f53158955bd01a93468346e8fadea732907fff12145d5 +CRW_1444.crw 7713768 81e361f8b333ce2cee8ea09ae0781d4145e3e5b8e8560776db56dd0f4b671cb1 +CRW_1445.crw 7270728 a43d7f55011fc63509b8f6c94e4df98506ab2272fa39357eccdd19323d0da3d2 +CRW_1446.crw 6995736 ab6c6bb37ac3652df6710c69e47a20c3b710218f8687a5cb5304be1b7c58a1d6 +CRW_1447.crw 7192394 aea0e64f14efa13ef114f42a7f1e34c09e85d965f0d0da6e530640e3893ea492 +CRW_1448.crw 7169042 a003aa9c7e3fb619610097c5cdaa45eb0733db350bc3ab00009e8e90330e3f00 +CRW_1449.crw 7190506 accffc1884aa3998ae2572f0e4c1bd323408c8ad5e5a3967d5e43c0053249664 +CRW_1450.crw 7288836 bfbbf60af8c8c7e83ca702da0fbd02f0ddc65130e6af0292cb6e2a7e46dffb62 +CRW_1451.crw 7435796 97fad026f8b9074c9173774defdc769d10186a86722a008512428779741b8a11 +CRW_1452.crw 7263308 d938c37fee35cb28927901d9295cde6b0626c451bad88cde65ebf2bc5dbc285e +CRW_1453.crw 7315528 644c34cfa9537816076335ba02c08230d2823d4695e46e156058e6d35a09a831 +CRW_1454.crw 7183244 020757713517e8388d505a4cc0ba73020b11a8c8348cb84892f9ccbf8bdf4ca3 +CRW_1455.crw 7060108 eaf281b260074379110d5e59ef03053cd642ec797740169e2a147084ecb11ecb +CRW_1456.crw 8189592 f86e110686647bdba61b47d8d7de420f6926d03445a977159ffad3131f6a77c2 +CRW_1457.crw 7909124 1afda093e9dc67485305d528fb0219da31aed568e159b38a392e7df2ddae8fd6 +CRW_1458.crw 7187038 0ba90b0cb59038209737cb3693a2de99fbf153e388a38c34315af7024b08baa6 +CRW_1459.crw 6910634 9f1d0e49b7afe904f292ab38848057e0846b09a1feef484d6241e7f54c645d29 +CRW_1460.crw 7056776 ebc9e6802aec00a0b755af6a5cca3d6036ea6cf22202e20dfd2c64d311fddc89 +CRW_1461.crw 7706206 e6563a24b53ef01981fe14eaa3f00e027c14c8fbaed3b16f555700b50ffa16aa +CRW_1462.crw 7167134 3e514eeb165c88cb8734773526b6f762068f298c2f019b8aad4150616034f79e +CRW_1463.crw 7026444 98d92ac5e2b6d7cae58a774e016ca0b40a204ae28079f0e8fceb66fc2fa150af +CRW_1464.crw 7001442 d7c21b20bcbf3efe8fb18e23dfd369dce555048a6e54ff2b9cacb620c12b3984 +CRW_1465.crw 7178660 231bbbeea90c103ee278d95c97dfe8953ff95f0f9672dbc5f9adfefb767c150d +CRW_1466.crw 5981438 00fb70ddbca6c124a9a42eeb84d24e37eb9267c9db000c5132bf83d55f7707df +CRW_1467.crw 7113724 e88b9d9b76f5b2c73b5ffdf9f7ac0bbfe856febe3992a76af4266c79f594ff27 +CRW_1468.crw 6386188 f34cc6f52f3dfbc81e4d82e8b660338247e31639305b3e4693f3b92f90efeada +CRW_1469.crw 7098782 ade55d4177553b307876962e2c0dd64bbd42eecf0f12d7b1911e4f1b767916c3 +CRW_1470.crw 6890172 3368c3219891ad118bdee11cb36c4f45bb9114f1bccb879ef6863771edeaf27f +CRW_1471.crw 6884584 f0d1a5e5ba438845beb18d83cfafb82063c9dbea3723c6e9a0259a84e3c15b50 +CRW_1472.crw 7519802 1ffa17e131d40ab2f498041650fbd5ae3ae62ddb2d2c8fa512316736449426ab +CRW_1473.crw 7231348 cbfc0d071b83cabb8e7b5a0574ed0a5fc27096a9560c516edbd0df419930cb49 +CRW_1474.crw 7317886 be9f580a5eefb5c9a9d81d6d91cc1d1fd221af63b3fe10e3fb23f7bb8a179a79 +CRW_1475.crw 7522416 10f28d00d6db9af9610ee6c0663c9c6dc150b5a5e6e20f59529f3934692dd994 +CRW_1476.crw 6447928 4cbf090d42b871e7eb9ded2310e69a1ab728911b4f0416666c3f2e357442ee2d +CRW_1477.crw 7289426 26a89e8c9fa02a7561648f18d017b034cc87656b7e67a96fe7e9bd4d835b7a99 +CRW_1478.crw 7462708 541de5069501e6b3df9d57ab97825468113744182608b4626b6f9d223b9ddbd7 +CRW_1479.crw 6846628 e6ac2dcae8533376c13c6a74c749cef7c57cd00dd576d54ade27cf6d2315cfdc +CRW_1480.crw 6922978 40bd099c71dbaeba70862cab1ceb0530ab8d2062b10cd36553db2b0e1fc4938b +CRW_1481.crw 7029724 53edde62af947d5f181316c7aeadb491a26ed7ab3569843c2801a3d2817716db +CRW_1482.crw 6989564 db6ee744ce4ccd4966eb87b3c7e285c35eca979e5d2f107e9c9b471a1a4f2300 +CRW_1483.crw 7130612 1b8ea49091266bce30dcbd3f1be69b8a616019ed45320a4d43d21e83dd52aea5 +CRW_1484.crw 7270830 9ad888fb8d61cd4ce51632af4ce99771a37ca6ed7bb2a65cac850f9aac55e2b7 +CRW_1485.crw 7376456 5baf4d51fd603af06b33008778ba9432b1a8d95018a7d58b59a56ad7306f0fce +CRW_1486.crw 7169188 7e6c598ac0a3d2371eca54d38f353c81a96a24ab30d0874cc8e0c2676432eebc +CRW_1487.crw 7294518 2a29584f6c56b875c1a53cd4c81157f569902ded2513258a82295e80239dca89 +CRW_1488.crw 7107118 997aae87d90a0aa1ac91df34f37ba584ea2ef546d0ccf08efc9048aae3534ce5 +CRW_1489.crw 7128806 fdfaad422cbeca0e0d8409b61ba9d16a3ed614fc83200e2755d47eac3c586a65 +CRW_1490.crw 7154858 2e3fdb3f91d0fb8229f336c33d6dca9da9af7c109e6346a960616e96fd5b218d +CRW_1500.crw 6459502 258f47584aa27cce69815b7bf68726efa41a91895b7781065585077893caf0db +CRW_1502.crw 7012354 7d99dbddfd264d6394ca8f21a101d272fd3a0ae38896045a9b9dd459ff1c7774 +CRW_1503.crw 7444712 8059eab08d3fd16e4f377a62d9d21c1823ee0d6194da2fc34bef4941fd606412 +CRW_1504.crw 6142982 7dcf263d92ef66e6659f16858a3cc38a5993379d4c0622ecb2fc773669e0a353 +CRW_1505.crw 7004246 cff250c929b7c1a29cac549e1dc05d4ba09d3f7670f925cc20c07f0c955bf300 +CRW_1506.crw 7118380 16290ca6e2094face6c6c02f634da91635eeacdc48f3d4874f1d36e0a5fc2cd1 +CRW_1507.crw 7022774 6c45fa238ee6815d981088e7cba7b8e7294f3224ec6997bf0a6d804c6bbbac80 +CRW_1508.crw 7040460 6052495114d6e54ff6fd22fd0643070b49fba1c5165bb637e9ca7d47f1f4f01a +CRW_1509.crw 6107724 0477f5a6eae95bee969ec1f85b107148ff12e20a30d3ce9b3e905168a45bb2cd +CRW_1510.crw 6378114 76bba363f45f54cdbe2f70b08dc80ca0349b28e38d54b9dfb17ab8a18f1e092f +CRW_1511.crw 7301454 382d5666f42c250a700497fc05ec3784230a3e491aceb9347e30ce046583d5d3 +CRW_1512.crw 6472066 1451b4bb5fd8f7aef208f72e758712fe629ebc1cb6c2e0e7cfeb59a933119efd +CRW_1513.crw 6434548 899294649f93490cd1104477b4c48afc45c0ba10864f0b3c8eef081be604512f +CRW_1514.crw 6531360 9aaa960461f829b218eaa7f50cf49fa3cb80e92baf9f5aabc03038fddbefe06c +CRW_1515.crw 6946094 60dfb72443bb255c2756fbfea8c1b7c23b93e089f444eb90558dbde68f55eae0 +CRW_1516.crw 7783240 3110817e87308ad3979ed7329206f67beaa97c0dfcd2877d34996ccd7d4b059f +CRW_1517.crw 6866584 34c80692cc73a6971dcffcc8b2bb4fd31fb7db348a21ee85b269e85c981dd0fe +CRW_1518.crw 6756612 cb44a60cf4a9a10479e583968dfdf3ab6be4c72d2af0584a40ae655ebbebe941 +CRW_1519.crw 6580992 3e176a9df4d96562c6596beeae69833ed4527141410e5c8fccc984462c40b2bd +CRW_1520.crw 6798986 0c6ae586928d5d01b8805759fc490fb635ad67abf07c034cda24c1c0adbf3e08 +CRW_1521.crw 6796416 fb5dd5b712d92ce2862bf2d28c2a840e9027c82c5e053b6fce678c136946fadf +CRW_1522.crw 6648394 09f8122210c88a328215b5374986fbe88c912e76ff87005e7ecc269205ebc8c1 +CRW_1523.crw 6976830 5edd66a7d53f5efeba3ae3a4f45cd99bd4e141ea99bf514d72f86edf305088b3 +CRW_1524.crw 5993060 f7068dc5499c29bf47cea0fedc6623e0509709ef0100c06821f1f0754a7c11fd +CRW_1525.crw 5976322 b9ea5718d347d721c52059ec1c4d5ce3b11f33eb5b0e3f465788dc596b711c10 +CRW_1526.crw 6008346 7e599c1399f6d05d53553e5fa56bba22e794190e4852c3e4ee46ed7c21fa7fc4 +CRW_1527.crw 6008956 b2ddc4ae51c38a64135b70ccbb8e67e18851e65c53c662d2ece9cf733cc20625 +CRW_1528.crw 5948078 b0ec06aa55a3509ae0e551e972e541e737ed35d523417ce17c171af6daab88e0 +CRW_1529.crw 5966588 f4b0af56ab0e8e1879432aac3b2eeae5fa741a813ee3c8e28ad03aa5fc708321 +CRW_1530.crw 5947686 b156a98ba34883978dbc7367ce1351df204fc9ad4b95c70a07fc85da9285a727 +CRW_1531.crw 5848072 08781f337771d51b2af012ae79eb2e1b81a729659209f0c059748eeed6f3e949 +CRW_1532.crw 5939082 ae3dca8f7f87dcc145cc8f95fb2377b18d089039d7ae97c3500849b6d0f68f01 +CRW_1533.crw 5927468 952eb1e35be64bed76d3a4143e0db4ee9637efb3a41d6328eee0c2ee4cef8aa9 +CRW_1534.crw 6501846 e8172873746637356484c3835e79c00fd9d8b83887ac87f0469639a2965b4144 +CRW_1535.crw 6104588 86b00d74209deeaef774b01c8da90fe4b4c8e19b98c214db164b873e928e14a3 +CRW_1536.crw 5901654 1f5fbc8bbc521ce583ebec4e13e1643a3d931e59e9f3a2acdd122dc111db861f +CRW_1537.crw 4415732 be6bf3a11f9083f629b2b5f11fd270338b89f1d4d1a3260fe2bdb9ffc76e9011 +CRW_1538.crw 6591470 fe68648fce9b99e376f5ca28c82275678cce185c5ee0826ad6b820f8de8944e2 +CRW_1539.crw 6586404 4fa52e47cc51cec27a8952cac1e1a5f3000ef590e07482e229512c75e8fe39f7 +CRW_1540.crw 6626982 c0eeb403bfbe896a65bde7441b96dba709e18e9b318bcfac693d93627534f9da +CRW_1541.crw 6609460 425328cee09c787d8f1422b5db539a8e6430cd2afc731574f46e4f08d81aa3e8 +CRW_1542.crw 6579132 30a4f65c348e92c2ad08e5e41a2651b36b2fe8449102cd3a0c97847e4fdce6d9 +CRW_1543.crw 6578928 b8ba3ce7cceedd561202ce6e549f99f068082f81696f6799a38749bf462aad40 +CRW_1544.crw 6576696 ddb72bc0b6554edeb4df731866c3622412cd0aa5e545f57a864b07e796522603 +CRW_1545.crw 6464288 0ecc578b2d95cb115076c540e72663de6ad8fbbefc1fb9c1fc6ab9d9db3246fc +CRW_1546.crw 6531618 a390b3807e546291e72c6e42abb72993d44c2a2f28b2a29fea61eed1e7b90ec2 +CRW_1547.crw 6706532 5ccb5146d8d6bcc388fbab67156f9f1260ecfdc9a57811b81a74034ae9793159 +CRW_1548.crw 6618142 d726b054171c266f25ed37884360409c59641b6971c06c51a42834ab72f7d9ea +CRW_1549.crw 6737760 5d2e78ae6850469c9fe90bd2a09e1ec8323b1e71598b8cba17543e5c03cb7504 +CRW_1550.crw 6929916 295d1fd4c2bf360d9d6175f46e6d376a26b8fe85a456e71c0b1568f46bf01860 +CRW_1551.crw 6773452 4acceecc22c9188103034265edb7bf8e1341c396596499b8af4fb2affa06a830 +CRW_1552.crw 6465790 dd8d70df30152c0b28e1ce69a7568ae30e037407c166a9ba703a76fd6ce87748 +CRW_1553.crw 6876636 9bef755b99238db8277319dc436cc42f8f97fea13c8bc2f01256df6b5c4ab7dc +CRW_1554.crw 6505118 430efc51f99fc10db0c7158f882b9ec03f46d5a4150c2e8499798834bfd74ffd +CRW_1555.crw 6838216 d5dff479f1d0938d9acf3dce244fde163d6e054e4de5ac955aa497d47ef84ee0 +CRW_1556.crw 6708550 8375948ee4f1f931342052381d916149ee32a5a0ac5ee91f16fd98b80e809a29 +CRW_1557.crw 6668258 8efca2c4c374ce84589a9511fc59cca8868004ed8a577645f61c25cec0fca578 +CRW_1558.crw 6571202 ee7cf7b692cf964984add597864feea32856de3899a091948e1d3d5ca835ae4a +CRW_1559.crw 5946686 e30fd8b55e55c41d43b160e8c061aae1f6f8e94baaab24316a335d5c2d5b0869 +CRW_1560.crw 6016046 268f7c2825777024c7bd131e5a273b2bc6a6593f276aebedd1770eb1c47d034c +CRW_1561.crw 5911556 a6b3fe3bb3f9d85af27fd66ce5ae9efadb0274985ba287832ae9046eb3bf5831 +CRW_1562.crw 5853688 7cfe94f586250e065432c209ca93e90809bb825f8e8842d1cfca8f0287abdfff +CRW_1563.crw 5709114 9947f25ec1368167715ea85bcec5a3761a38f236599835c21c0873d03127915a +CRW_1564.crw 5795220 f415187b15a5049a06551eac6b1ca4daecc297e4b7ff59ed5c2925b4b6a5698c +CRW_1565.crw 5802242 56145daae95169bbf96d5828a079e10a1180fb5a1ff4255e64fab8b5e2bdd0ec +CRW_1566.crw 5772274 3e1a6e311b36924d1f574ced676a4099ff20a96a2399857b527d035dc4de6e64 +CRW_1567.crw 5756038 65e38303db3dbdfb9b14675b800b60861f2e8e9e04e71180cf7d9e7186ce5d4d +CRW_1568.crw 5792258 1284981752c633a670d5a01c713bf6f111b0efd6ee3d0487757781c2836a560d +CRW_1569.crw 5828670 b11be0faaececc0538f2bee0c92d5be6bf40cd6cf97380c570d8b5542640b52b +CRW_1570.crw 5781904 222baf2cd0d1b14a1c254abccc75c8be31fbe303f1285b4a47ec3b85525265a0 +CRW_1571.crw 5788394 365cdad3083c47fe2909bce8a312c72adbd7f31bc63a982e4b7a11683e640c0f +CRW_1572.crw 5812588 5f3c795289aa2588b137508a6360f2820a90456dfadd80cdee36c66e69810d90 +CRW_1573.crw 5792076 852b632365db7fcba650014ad5cc2a5ad87e8165ce4830840ab3e02fa4b4bb28 +CRW_1574.crw 5771386 f90c992f24b0695b442595c6bd556678888332d13d5b8d1fba8aeae3bd38c6cb +CRW_1575.crw 5797504 2c354af121fbce9ee38528136a40ce1828232f68b04f0310e2f3a67518024a77 +CRW_1576.crw 5815138 3657022edc1e8f82714724d4cd66b81750907d13f7d3b2e7ed3ab6438edcc80c +CRW_1577.crw 5800184 6f916069e4cbd6b537e2efc6555d36a158463c0f875efb9f41278c9505469b69 +CRW_1578.crw 5828800 6928f66a75816320f1e2118c941458405c070c9392381a4b077b29d850114604 +CRW_1579.crw 5862056 4c4075a533ea62440887f31fc0375ea8595edb6c01a780559e92a0612a2a709e +CRW_1580.crw 5781694 18749ff82abb619e078aef18c116297458e79c4c0f4f22e97c051115703982cf +CRW_1581.crw 5814798 1f01c16f90e13a5e3d41e8cf978333882d882f3d89c1395b167b5e83c13ace1c +CRW_1582.crw 5803922 a41fbb8ac52e9c1cf92bbf8297432239aac625159d2322193db59b293756ef20 +CRW_1583.crw 5745218 49aaaae863ad410351245054a05ef2b1b5c05a778a0842fb1e67797acad63d23 +CRW_1584.crw 5774212 b9556bdf6b2badfc288c8dabb8e7598387edaad736ab2fc77da4ad61907ab8e1 +CRW_1585.crw 5785144 4342b89d9a155754976aaeb676462dc3dfee8c3411f746c2db8830b74fceafec +CRW_1586.crw 5764238 b64e28abd03a5e0dd84e71f92c02744d7881ce4bd47e186e854e7b44cb61ed98 +CRW_1587.crw 6242958 f17d5af2a1527957b0514e891af80510a1f79b7e8165668a858297b317dc0f1c +CRW_1588.crw 6239538 28d03c5c6c210cedce2ef054dd5750f8c6ff4d46ca69a94aaf3634822c19c993 +CRW_1589.crw 6160794 bd2ab706ba2764d301bc3579cc3d6882e69ba8af036aaa178128181d7dfbbc92 +CRW_1590.crw 6140614 051e2c7580c39fae0e8b567f716862446ff09451e13cf2154b592768d1a99487 +CRW_1591.crw 6168970 8c09c1eeb3eb066fe094c80cc50c0f63f72ad0c815fd5b1e2e9b8ffae6e35db0 +CRW_1592.crw 6126030 a6fcbe3dde83eaf91a0732e214facab05a288d01c1b4542c51bfe55bb1db860e +CRW_1593.crw 6086634 66614109e69841d301085301508d3d29297e76a201ed42bac174aa29929a1137 +CRW_1594.crw 6080500 9f15ce1e8ef55991d3c2e077f891184781b4e54613626e423d91917d564e5e06 +CRW_1595.crw 5917742 b6bb468e292828b0e13059c547c654d96bb8adf16bbfe5adc6ecd572c293c8f8 +CRW_1596.crw 5872418 e43a19726391edc531ce2dc5399920a37b35b66bf3c4443fea05acc320ccfca3 +CRW_1597.crw 5353852 5613b5d2d0aeb3e49b8334a051c891575b4d5e7145e8dd67cdc0b0e18b80bc42 +CRW_1598.crw 5535884 700254b35507633f03863247e0370b17e363a364bae6a67b0deafea6450711a7 +CRW_1599.crw 6419086 4ec819ce868334752561011edbfe1151130c4480b975cc7e92840349a5ca7770 +CRW_1600.crw 6321392 cefadbddae3a8f8234efed677e5d2aab546a3a189dedbc2ed9b024f82d17c55b +CRW_1601.crw 6566936 f2a554dfd3fde2625719f86ccb9fe5d4a43f78526d82b53606bdbfb450e03687 +CRW_1602.crw 6816052 f098d8825ff48cec100214439ba6c14632c8b72909b4fc8782baf105099f225c +CRW_1603.crw 6947654 1184427ccadc83fce9a4c15752024d4d16dac1c0a3cda45a355ee4064963594c +CRW_1604.crw 6093816 02747d1888b19558eddab904d51cec1b0820824f84b73a4d4c71aef12c509e00 +CRW_1605.crw 4554852 20cf102e54fe174a6ab0fb72ee000a9a0e5d90457da9dbb855dbd439a282b761 +CRW_1606.crw 6525244 12ebe06351582c5d774ff85a2e29cf9d09c5ca5a188e031c2f7082a16216b2c8 +CRW_1607.crw 6780112 941c85a6081d1e1529512a8e78bc72dfb8d81727d26d9aa5ec5b809543231598 +CRW_1608.crw 6101512 e35ccb9b0e3664525953192ab2c77c6ca5bf9cff9dca56abd5d8f2c6cea4b7eb +CRW_1609.crw 4933626 8c5d6935a2d80da05632fe714e30ec82dc379a6adf34a3b130274020f75deb39 +CRW_1610.crw 6413926 606d69c87a10bd07f17647366577db512bca81ed56cf7385004ee2440a4fd8d5 +CRW_1611.crw 5969004 3fc5075be26abdfb06d2d7b09c89b1d686c11d67d5b0d15f3525da8570210ce2 +CRW_1612.crw 6233490 dc074f1449fd471373af95eb4804e59c4fe7297f140535f6e9ee2fb59e65af5d +CRW_1613.crw 6252298 c3272d3ce2c30226adeca1d551af152b908ae923447f2f81d72b68e548f9a194 +CRW_1614.crw 5779080 12854d5e2e222e6a3e1327d8a6e472804d04b92bcda57c261fbce46de82712d7 +CRW_1615.crw 5639790 7655c9107d65dc6595a22b4e4649f9caf23abb0422dd492a281bc26b76ccad2f +CRW_1616.crw 5697474 1e43f989d03ecaf4f042def53e5006b1dce2c95240df6e0194a11adf955a662c +CRW_1617.crw 5235230 6ed12324cedf60c2aee8884618ddfb96003e1dfb20b3261a90b5fafadd1d5984 +CRW_1618.crw 6071314 d1350c02de8f23995276f2993a70539a76a7c98a16c63c93874dad7ed58b3a6c +CRW_1619.crw 7548702 dabc25590258349dad7bce45ad779a59755622835e7df2593d16d6d6752de62b +CRW_1620.crw 4951966 7f4354afcf7e308d3587c8f0d7170b28c57e0b3d8b0933b04fba7fc452a68f4c +CRW_1621.crw 8094694 9cc41df03738ea6fb2862ac068a9f088aad619294027c88e8d50c1718be936a0 +CRW_1622.crw 8032850 bb3d28d7f22987d5ac6cc99b75463600421a908347039694dc13432e342f4bb9 +CRW_1623.crw 6442782 88e513c9624ba27ed29059795e9aeeb841f2df209d6b724d852c032865f03c6e +CRW_1624.crw 6279200 2f529b53757a346ed6622505ee18b826d7015997557710237171d7c2e1bc6284 +CRW_1625.crw 6427534 fda070ea529f49838a2ce6b8fc806270bb8c43d49af3eabcd777f96877029eb3 +CRW_1626.crw 7858666 ebea849d8bd7995c4c8d90af6047f160bac2d3a9539fb306f723364a7a19c829 +CRW_1627.crw 7979082 9be2eb784ee866ba9534062fb06f0ba123bd3d9097195279526fa469337a1e66 +CRW_1628.crw 7692936 f4511a032d7dc52b14400ffea6a2c845a7d78728e84fce61b117c764aa738557 +CRW_1629.crw 6969578 6eb0421cfaff985795509b81007d7a3843cfb0d9ee1ddf6fb827d86fdf063211 +CRW_1630.crw 7334556 17da9d7409873e3a570e175f86aa119ffdfcf6782853abd5f2c7b4266e40622c +CRW_1631.crw 8798240 d15623028eb63a254588423e5a55f3dc6797dd47d7b5bf80881096e3b834c65b +CRW_1632.crw 8290906 30dfff7b858760eae660e6295a30f2dd56ca08a8969e9f662b92776b11530181 +CRW_1633.crw 7128720 b2437acfebf7796db9346000ed0825b5941924e93c61beac6ddf435f7c95ce36 +CRW_1634.crw 7138286 cbdf36ff86d3dc09007b32487a2afd96827d803658c8a945208bf1e5268970a6 +CRW_1635.crw 7295688 7c2fb2aee37e63e81c0749465bfb7f0056bc8768ad6f1e1096c74786495c9347 +CRW_1636.crw 7011766 b5128f14ce0e753d9821d0e26a7a574e07af878eff0043d9731893b08e34388c +CRW_1637.crw 7158020 287ac259a38c98e37f5e72fb4adef9f5df50082d6b159dc62fe7f43e56b015df +CRW_1638.crw 7438286 42dde10d56c53d7d7757c371b8072c44e6e6803a6a9fd74d6ad9ea00c6a0153c +CRW_1639.crw 7598264 84ca90247dd0b3dda4b3458073a756ac7c604cad42a69d521bcb63881efedc9b +CRW_1640.crw 7687218 59ada4c0842b61f65449add01208b42722e3f5ce1d8b57db8a91cef564b26cf1 +CRW_1641.crw 7767898 f26959b33d70cb2a3a2078606e74baa63a3bf43b7cfece327a28a00591cfef16 +CRW_1642.crw 6193004 ddd9e10bcf393af9272cf3ac56b24fc4af3b19ff289e31a63aa0609837087ad9 +CRW_1643.crw 5966756 2dad1430091ec4c24973119681dd6658f442c82d7c4ddbf1a14f10eb88ed1475 +CRW_1644.crw 6279614 bdf906933d0e1226b546506be62a2e5a75a123e488456348a0168f6aa545cea4 +CRW_1645.crw 6739406 72c845720ee120af30cbd962c4bb6a12a244049f6d0c4f14897eb4612339bbaf +CRW_1646.crw 6204492 f9f3dbc2f9aed2cdc2ddccd9a9768a1e02501c337258a211ac9fb663ad78ff74 +CRW_1647.crw 6352938 6acbd3b1276a38acac29e81318a4c0830b8c66995e04d3328e1a7d5d57fc86c3 +CRW_1648.crw 7078296 1726efd567e1c5cc111085a2f19bab6694377be078789813d7e6b89206481829 +CRW_1649.crw 7553936 a8bf3c14e1168bb431f35c848bbca63984d692536a8774e70a9ca1dacbc4136b +CRW_1650.crw 7018660 53b42e9f7c3d1c3b179faeb6af6b22c7b6a51997e0f51b71ac96a390f2ba5268 +CRW_1651.crw 7250710 e1aaab13f083d3312ee10dcf18ab241cd7e2e2726e71174818c9110d85c02e59 +CRW_1652.crw 7116824 ba8e39a5a074e33e73f74940abff96b1f3ecfbb86fb1d078a362da07524c05b7 +CRW_1653.crw 7925600 9942c2c49b2425de077294f1efb8433007547def948de0a20f94494e31083522 +CRW_1654.crw 8087454 df7e4daed216951937e322dc7f160750b52c1bc212ca4d660215408f03a92ca2 +CRW_1655.crw 7894788 bd9b7e97cdad6387d3b5f270662324d240be61e59a70afbf9f435d6d343ba07d +CRW_1656.crw 6835122 11328d91ad0a326ac07527f665daace5362769373dd71ed3413b25b9c5000157 +CRW_1657.crw 6325858 a4700475d40df854ffdcccedbe9392a3c4d91adbbeae65b49f0fc05a65e1bab6 +CRW_1658.crw 7452142 ed7a3ca18911462b010c1d0142612fd1665ac3525ea33bbd0cf509a646fe4303 +CRW_1659.crw 7314056 1a2298e0bf887a6f26e2d642ff7958e99a9bdc36e1d90284ec5c8355f036fa6e +CRW_1660.crw 7340436 d22fc91a8b232ac9da9c22b7ce9c5d33c0e6a4c9bbaf75221f0a20edf8ec8d3b +CRW_1661.crw 7329716 6e397f58312fe62337c41563beeae03426b31ca20abec0b70e3b786364a2452c +CRW_1662.crw 7548814 3d56d0da139e186249b2cb68587cc32bf05b4bfe816ee529faccca56f0e5b581 +CRW_1663.crw 5776758 69ad76174cefdc705c7dad80cfca18cb24ceb15c236672dd8483f7aea3d75dd1 +CRW_1664.crw 6642326 0237c2b4eeb56cc5895580775f0b516095adb722cdf7828402131c03708ce44e +CRW_1665.crw 7663654 369f05a943b434b5de66fd981569ef8ff00c1e1028fae6bcba3926897a483b93 +CRW_1666.crw 7062122 1348442e38b3f84ee558ba1267309340c01e3c86513efd62cec8734411edabd5 +CRW_1667.crw 5914932 6e4de38431ae14d9ce60a84c44d509c325cbc95342d3ef91cc600a47df19dfab +CRW_1668.crw 6397228 87a84baffceafdaf960975f2c0f056d4c87c60c9ef899b0bb4e5e466fdba3d82 +CRW_1669.crw 6576832 f388b958a4d2692d46f8ad594be83c8ef0159422fa542c3e8c3cc419c38420fa +CRW_1672.crw 7833750 c7947d06b89de293f380d4a1b615c493c44e5c1b2214010020f522d4d6a66075 +CRW_1673.crw 7322312 6fb0ea98cafd0570a20a0ce66940bffffb4398eff9b3598d401fd308eeb3c05d +CRW_1674.crw 5899388 14d72d8fe424444edd16b553ce04f234973ef2d7603da43d80bdba00795a6d7a +CRW_1675.crw 6903490 389340771ba0e5c98260c878985f74b01ce6bd4f98d2e416f7f886aebb46470e +CRW_1676.crw 7905504 22e6f1f1724b83c39c425bef1484491850fc37e4630f20f89dee01634c61400a +CRW_1677.crw 8224646 70547f21e57eb3b847f5665b2f2eab607554ec891e1a96cca9317a1bcfdff8a1 +CRW_1678.crw 7742632 697bfcf409f45cdeb5287a5f85f2b5b400973015b1fddc27250751ad1d1228c5 +CRW_1679.crw 5616114 0dde6313603f427fe9e657fda2decc32606ed28483104f0f950cb6da586b467c +CRW_1680.crw 5505994 52d2c0b03677700cc30b811568b87329cc8c4777db64526d4ef976f57ac0e1da +CRW_1681.crw 5502986 6cd9a8f82a3b08120a6f7c4c3647a2e5e1eb6e7fa1598ac951c68ee502c957c7 +CRW_1682.crw 5814460 164efb4560977f95c06bb540569b53a5698204498b1ebcd109316ad8423920d9 +CRW_1683.crw 5449152 449cdcc8e20af4cbab63c088e234d5ec17ed2eb019745a59ed259ef36b00708b +CRW_1684.crw 5161412 b97b948c46404737a0e0f3d74c23006a68bc9226a21b5e2478546e390cb0022f +CRW_1685.crw 5301348 a8433434e29667402e7c739367743a93e25159a93afb4862f238bfb8577d060b +CRW_1686.crw 5358672 3443d1bddb7436c8b69a16030131cb007bb5678209b88c5f4135177b7abd7aaf +CRW_1687.crw 5379530 ce823358c493335c605528a2b27b5f2fcaeeb4408cb4d440ce7ec54d529aeafb +CRW_1688.crw 5419276 0b90896f5b63025a6ee7917de2135c9ff9e7c7856f835cd8682efa6c583894c1 +CRW_1689.crw 5554824 ceb0a5ee11d681b50797ecf7e0b43b8ef335c06077022afc416c45ea56c7af8d +CRW_1690.crw 5674504 51490bd3e9be2fd39d6cb96a095fdbb3b4b8a06790d855e11e2bab08a9e31874 +CRW_1691.crw 5504548 e0dc990f4516ed8499a9cdfa8f3ada38da2f36a123d6448a6bf26ddc7f7139df +CRW_1692.crw 5446610 cb61971ff53a5f343371877fad2ea3ce93f7737a3d5968f2be0924b0103e01ba +CRW_1693.crw 5430604 693f9126403e3f491e5d8f0beea2dab0b90ca0ea20024f2cbdc09846102edf2f +CRW_1694.crw 5489700 1b95710c78a9c054fe0f17f9e025e1087105985f4a602460699b434692ba4759 +CRW_1695.crw 5938360 c314f82560e350af3a7dc5784229005c995bd6642aa173cef9b1fb97ec6aa19c +CRW_1696.crw 5808670 083bd920df88fd99eb2b194a4d62e07b74bf3a4d02686295393a7c828c02bc7f +CRW_1697.crw 7284078 66bf8fda4a12870d364fbb96b483de1ed5a3c3e439ba853f22cda5a3207794b7 +CRW_1698.crw 5507868 3a65e7373d2a43b9487a027b55e9b0e3b1ebb8f400c0881bf73d70d8da649388 +CRW_1699.crw 5569032 c612d7f0d36feeed513e06d57650ddcc11604eff2b052c36b362b623a9bf33e4 +CRW_1700.crw 5550308 55c9d38d76f466071f3bd17788d8f0de1084eb8a3f43c9529fcdda2199abc44a +CRW_1701.crw 5511124 fe1b3206cdd7eac511afe5b1133251740bc57de18a39987ddd01b3fc198cc325 +CRW_1702.crw 5549136 3a94fb97b442fc705425e555bb9da3647b8beb7af4f1a4844a6f7bc94b3356b2 +CRW_1703.crw 5434744 80ed0023e295ece6c5cf184682a0c0af7082246caad27ee2eb10326e5294afc5 +CRW_1704.crw 5516958 cb791d433d353860e0c0172678479ed77c828ff982244c8308429a517198748e +CRW_1705.crw 6469216 45705f5b505360e7ad55ac053aa37217c5aba0bee8bce050baa6f99af1080bc5 +CRW_1706.crw 5682902 91f520b9eaa56047bee1716f8323219de9158f7e1dff10d84b20e5449ffc0d28 +CRW_1707.crw 5949094 cf35c9486dc9794d54677e1d3445d81cce91be9e73b8c549f437b582658553e6 +CRW_1708.crw 6073218 5b4ae2f12946cbe59fb787ba844f43d6a3d28f875b420dcb45cd9bf7e54c21cd +CRW_1709.crw 6265560 272d365616216da48f4d12477055573af1d2155c0f07a0cfd0d9f8f3f49459ac +CRW_1710.crw 6791920 d024142d5d4bba71bf9f1a5f4f31ede0f7890af48366cdfcd92586002033223b +CRW_1711.crw 7884976 6b5c9663082a08dbf1a9e6861a80f5d4e0cc05c768adc7813666d0721cd90237 +CRW_1712.crw 7473578 d2c91f2903d9dfb2f574a3169130227e1aa088f5efc4e30a260f00f6dad4f57c +CRW_1713.crw 7374888 98202bb5d1b39e1ac51247d8df65de6e333eb87e2b6a86c70e30200e75a9de51 +CRW_1722.crw 6444464 9dce3caabda8e75a67c8e0fb4c8e6bd665de10ca75fc9be016f9535cc8e46b21 +CRW_1723.crw 6088126 ae0a827ca99e046f8db1b0c0df15c293405f13679bb82ec6daf5777806c877f3 +CRW_1724.crw 6648078 c2a6cb70c2404538b8731c6c6f500fd29194018ba8ad4035e6694442357bffbc +CRW_1725.crw 6293962 111aa9e1f8544ba9d927d93e51b443f65d8ee602ab463215786365ad1b2ed345 +CRW_1726.crw 6367158 bd00f7cb6faa248a8daf6a4f75f22acb8ad0a166ab8cc53fc01cf159e60c6f82 +CRW_1727.crw 6336626 c5921b9aadb6282e23a75fe10ba011b80ce20e03df74dab429cd4a2020e0eebe +CRW_1728.crw 6377674 8b2ff90b64150abe0ed1808298970810b21407336e177f0dba5751909faa74c5 +CRW_1729.crw 6407934 43662e3ceadf131ab1483859686f5585cbf613a33af39cf00c9a06b85e3c6df6 +CRW_1730.crw 5768792 f79aca99226408d1234d55d5a21de4c6b8f1155b8f807d64b43ce6c7cf59e88d +CRW_1731.crw 5599952 a9f2df474b134576beb611605e8376b4da63cd20fbc3a0d93d1d314f4c368a71 +CRW_1732.crw 5718744 f3a6100328f96f123a55617476599f06b6d9a6237d7c6a1551938d2f40532fc7 +CRW_1733.crw 6403562 90c477f92fc7df848d12b8885d48ac29e5a691a93417d040e0c9117ce6515513 +CRW_1734.crw 6214824 3f11fa3dcca59ec0ab5a9bccd3edb8ed7ccacf16de732951b3a134208ae1d175 +CRW_1735.crw 5957092 be00be533b7137c32909969bea392762a6a14baf9eaecf2ce59782cb6ecd63cc +CRW_1736.crw 6069320 e6bbf6ca88e9d1d35d85d9219fa693f080466cef4a29081c2671f8ca89cfe216 +CRW_1737.crw 4999180 d7ab5e2ed5fcc0df28cbd67d586881f8ae3585a13956c4435a3c2e9bdd023d81 +CRW_1738.crw 4458464 2c1da65c892869c18d60156c8110144f305c82ba02b5249dd6ff9bdd2adbe397 +CRW_1739.crw 5450770 60e877020767a70857c2c438ab0e32f4310854189ff564106fb42ee0df71e93e +CRW_1740.crw 5500958 033f225fcc8bfdb07b013d971c594bda2adfa0a683c133acc22a171ddcdf95a6 +CRW_1741.crw 7338748 ced0e2cb231821ecc5b131b64356ac9f7f1feba6517221774836e58f2ad5b5e4 +CRW_1742.crw 6682908 3fc3fc222a31ad19eba970868d4d6071252d120c593d386509a0d53d5ee003d3 +CRW_1743.crw 7774750 16bd364194eee3effd91d7f5b39831422f022128c9a486b5154fe525b9316022 +CRW_1744.crw 7622796 5c0a6ce61b020e6df89c2c533c94fcc64a7aaaefbdfdf7a1c62a5f3dca4ebac4 +CRW_1745.crw 7426196 833b0ceecd0e359b37867cf87e85126ed9b148372972ff95bc2afff64764edd8 +CRW_1746.crw 7447042 31590237e7c2f6974a42e408758b7e72b8ac9963f65c81f38554588a0101eb81 +CRW_1747.crw 6178924 462dc6b9d244d48a87e4ec4b3e4bd5c2c8202d5a29300a1043971c7f8c3a4c47 +CRW_1748.crw 6891386 d95f5f04eeaa28ed1f47bfb4fe1548d0f36c70d021fb9bbb4093c06409259abb +CRW_1749.crw 7078824 923e27c24b637b2a3556b59a5b3fad8394516ff47e1ce9526f27cb12db4e4de5 +CRW_1750.crw 7619354 055c7fbf5585dbe8c7f8b34bb17f0252626fd88c0a08cfbce71709f576a28121 +CRW_1751.crw 6458892 dd6b4ded5b18bbbf3adb33556fd34f8f5590971d06d365e7fd4bf54f44558b2c +CRW_1752.crw 7692916 8b2228645028a891c7b5901838917d92cdd070c24205136110ecb6e1e8b44e8f +CRW_1753.crw 6575960 2451a0424626aa7821ba360675e29bd301c84e382dc4cf684098c7601b42a851 +CRW_1754.crw 6727330 92343a4cb078311ead2b04c70b6df4f946ecdd6ac24f97738345e18eb38ec27d +CRW_1755.crw 6378734 e9fb9c9f907d80f9231f3dbef3a0561cf85d0910a94b44a87fadc74feef02550 +CRW_1756.crw 6869908 cfa2b2153549f6eea38cf673ed7a33adb7c7b3dc138973aa4a0293b12c952f0a +CRW_1757.crw 6632930 812448d4f4d6cbb60465e3976fad3301d091dcf61fd541b7d0b8614f0a771ba3 +CRW_1758.crw 7113164 29f237399e2341dcaece8d6c3de429e818b47310d13928ede2272b9e3e7d42fc +CRW_1759.crw 6754436 becc8ebeafcdc895afb9f76069ad9aa4acda845b958759e9c5dec36efef10aeb +CRW_1760.crw 6835162 143a453b8a2d4554772d24704812a8d1e6fc5c57281dff2d03906ebff518ddfb +CRW_1761.crw 6913246 78bcae708aeed992fb7a698e993820684ec3faeb33df64eee620e3feee73c757 +CRW_1762.crw 6527566 076f2b8fc8140dd8b06412359ca4f4be3e999bf7696da9f1f560997a4fd8aac5 +CRW_1763.crw 6539502 d56296da1a3dbd472ab9df4ffd6f0094d7bc73b1873f12e4683d488a0a851337 +CRW_1764.crw 6652080 6593cc1a321f93b02345d638a8861643416f4746ce26c68d62ef53dcdb47872e +CRW_1765.crw 8069160 ac1b21e78a44c98920ea775499c4264297f49ac567b7f07b6285a00af99064ae +CRW_1766.crw 5996886 3363e728fd807b1d2deeb0aa53d88f4b750c414bfea13c827f756cb636848808 +CRW_1767.crw 6159954 9700295494120fa3d5c1b54c6815ae44562cfc4f05148e8c140362a4894284cc +CRW_1768.crw 6128362 23cc28dd0edf8fe47e4aa70296506a337062cc00cd8042745b96a3f7e0ce732a +CRW_1769.crw 6751226 f333f6b5907fdb0863d09ae8e1443fd022296ca329d39a8ff490dca12594b649 +CRW_1770.crw 6785614 16ec3b8223b5fd6ca8ca69bd090c83ccd7a19c7f7d87d46a1f0c7ba2413a6de0 +CRW_1771.crw 6254564 1d84822f747d73b0c5b84983e944dd8f2c611c452a656f9a21a55fafebb660a2 +CRW_1772.crw 6559122 29fb3057f07da7a1e35bf8bd4fb6a1bbffc274fb04ab0bd54f210b531442dfd3 +CRW_1773.crw 5984462 6e72f14148a78025fa17ae7b8966312770c45a06dcb923d8320182783af2456c +CRW_1774.crw 6311000 c87d2e2fb470e943f5e36a5f78af62095fe42d98354d6f1a4f82969f89e38dd6 +CRW_1775.crw 6404128 c5f5a4379aecb2dd0a42bd9f48eaf25b0666f80642c9c553710dd0c53d14cb09 +CRW_1776.crw 6746924 b07921872c8cf7767841945818a6c8d59553415580d7ebe6a08115964ec6de44 +CRW_1777.crw 6441154 5ec5461d92dcbe55ceb4b4cf00a461e5ab841dc57c4dde2439b70275b6349a60 +CRW_1778.crw 6435368 bdae5c1c720398c1eed6bc74b9e57975261bd1fbeb993a49f96c6c5413323df0 +CRW_1782.crw 5926002 5972ec56eb18754f51a25604489211a351024d225826bd1e93bf8cf0998a3ed9 +CRW_1783.crw 7421532 8e257f2f8c4354217fe3ae596a6e7b809dda7d937b7ebcd4f324e6badfe85dbd +CRW_1784.crw 6716594 795f40ef399f70ebc30f83aa6420ded7659942a1495164a1b7cc6278cb890c99 +CRW_1785.crw 7316052 5d8e9cfd82541fe196840f075251661a6d1e8d5c10a11b8b9dcb471e6ccc2700 +CRW_1786.crw 6882236 2b278c99cc8cf1ece538cab83dff27f583ebb8aef34bb16b71b7a73d1f3a0db5 +CRW_1787.crw 6487362 ca2edf6188327c5be5875651c4413989a855c73c6916f12e636854012cc648c7 +CRW_1788.crw 7422082 7e5fbca7add373759d7cda6bce50bc2dffa0203f38ab3c7d9bcaa2af0b962c67 +CRW_1789.crw 6991776 fb8b68f5085d3283f5a38ba9a38b584d8af7441b56d0ce2f0210d60b3aba35c4 +CRW_1790.crw 7515744 3e672f0cd36543fd427c3973b498aa14ff44b9edd5e38df870389a0b43c639c8 +CRW_1791.crw 6092736 aacad56a82532c658ae3a3a72087a6fba823969b677a1b20aae0917a9afc4a3a +CRW_1792.crw 5795154 13effb6fb5151e8c01820e07e4980c3852c5b9d06dba51caae5eae6c48de6aa6 +CRW_1793.crw 7699098 e88a9c598fa7e9df7fcaef85beb5eced63ee903aaa729e44ad18db522d74ec0d +CRW_1794.crw 6217622 bf3595c5ec3e3c62a2a1584811f1f014b014295641a0fd07e310aa49ed950c10 +CRW_1795.crw 7220012 b5866c8aa04c4d166aac18c4d7a7bb95383dca0aa02e85b48deee15f14ab7848 +CRW_1796.crw 5926926 4b561d90b4ca69736922f69b6f168ba22c695628bbbfd252b9a7a035b7b9f470 +CRW_1797.crw 5704342 3d9e22199394719c18125e7caa45758c6609f0b12e2107d55e8c1b003d4806b9 +CRW_1798.crw 6223682 ea61a62628f0c540502343d651bb2b9dd1eb2a48156c4b192639f5daa1dac283 +CRW_1799.crw 7646872 d8650c3c25f5c2cedf4e8617d082d145be5a33a8fa6c151d751f012962ea2c43 +CRW_1800.crw 8404174 d66a15b3556e714c242bf7fcf0d85e5607bdfb353229a1341783516616658ac8 +CRW_1801.crw 6096246 b1a063b0e3d6610c6282adb9ac4c7a629e6b13cfd3646d2e7f7310707c94c5ff +CRW_1802.crw 7515590 cc99f99a17765633444d9bc1996a16726fcc7771152b79811ef4d8102dd6c18a +CRW_1803.crw 6852596 5468cb37642e62440989f8033125d0e36f8fa52d273520f1c7f741894ead675b +CRW_1804.crw 6612706 f23805ad48f8a4564dfb858c4e236a42adc2a5631250de80e62a74c53d3e0ac4 +CRW_1805.crw 6478108 df0366a7269a0fe56b2a413ee4a934ccf9e9eda33e47dddff7f6b5467c912347 +CRW_1806.crw 5367774 a585b30c20141f6b680b1834f5e1e74ad75addbb8ad8494d321f399b4097296a +CRW_1807.crw 6189884 654ed2c99324358c859839001687ba3486880ecfb7180c9825585a5b56e147e1 +CRW_1808.crw 5326850 f100138747e3d5b4c23fa51c19432ee42ec0e911a201ab715a0abad66328749a +CRW_1809.crw 5412098 a604042b9118d7becb4b524453ad1b8ac88c20f900f66df22d6d85e2f524ee38 +CRW_1810.crw 5772534 bab6ddb996d326f6c2ddca4c3439a1f1cc584e5d4de33d9117ec4076cfcd803a +CRW_1811.crw 5751488 9509dee61772e6abd1944f0cb7236231e9d3e6e82675da501b8279f7925ce852 +CRW_1812.crw 7076420 a17eeaa968292fdc3f5527c46114edf090f1822a9c2609d383c861e1888d23d5 +CRW_1813.crw 7587602 e674a7d097fe65f33f705e952dbee990011ede302a74c42b89358ad9f58f64e8 +CRW_1814.crw 7515516 f66f54860af67095a8a790e17365218bfbe820c0e88a457cdd48eaeaecaa8b4b +CRW_1815.crw 5800890 011cf6749b6faeb50c117b8de3d8356fbb1035b3a7300d5efa9f98c9cfc4b0af +CRW_1816.crw 5993026 8adde74ec4c7dc2ad04fffaf37ebec60c3645666ca0bb64e1a5bd2e8ed40810f +CRW_1817.crw 7304882 431a45ee50b175ef0a98246ccd06ccfe40655a5c643f08dcbad88e7bc87f1ffa +CRW_1818.crw 6896922 e28fb99b566d8361ec850ca930987f8b58133e504e9d33a0fd37196de2ef7201 +CRW_1819.crw 6797890 7bc7a64d09be3b33a2bc7a317f889361a16b207c6e27a7aaaeb0b77c8c2e768f +CRW_1820.crw 8274872 53b266c9fa6b858fc508a0d797c1446c20c67273c9e768c7ad4f922bacdece10 +CRW_1821.crw 7706804 f4a387557be2018af09d2cc97c988e67deebe17068708421b190023217d10459 +CRW_1822.crw 7337362 2e4b91d93ab807f3c81d8731f83c2999d71ea7f4fe14ecc5891456c7ca92b367 +CRW_1823.crw 6594010 44c45bac13e72b4609d78a88830d8630a9c90b7601e805f0151e4fadd55fa0fb +CRW_1824.crw 6835296 2d4343080aff3bcf142791bd05bf681584f528b143394cd005f6ecce76b4fec1 +CRW_1825.crw 7421192 2c08148eb218c780854e6b1fda1910f90d7eb5e5205aa7daa5de290b08812ee9 +CRW_1826.crw 6235294 a03452f2e680c7004c1a3466eafada3950293373e9e89f21145d7509328d5166 +CRW_1827.crw 6493354 1051b2d95a028a493ad8dbca9158525b2c76a90a5b57bda80e44606852cc460b +CRW_1828.crw 6738018 d336871488a0503d3affb3110c9eac17fee3689ed4363b9fe7134c4be1f5a5c4 +CRW_1829.crw 7056706 677a5144f653990b2593c5eb56dfbd90143e5e8da56ec45ff032ed2c7f48532e +CRW_1830.crw 7770270 03661f29b3432a7dee02b6c6e01ca2195fc72f1f4381ad1d8c003ae97a9b7e7b +CRW_1831.crw 6717212 f5dc0a7dcbcc84ed0bec9f3197418502757e45372697392074624b4aa0d55117 +CRW_1832.crw 7040260 68c25962d9a533e8d61ef91ca856d6ae709cd80b9e15d661adda99eb889cff87 +CRW_1833.crw 6438426 4595d54e45fbfd82cc21dd4a9a590957c4aa290387ef8f5c07f29acd6a421372 +CRW_1834.crw 6357544 f2d6858840a51e6459b18e5229ac4ca847a48eb8318d25a55969c6ace04e7a2a +CRW_1835.crw 6742106 9de8368224b5228d2f5ab2587ed6163963215f58858337ebc14a10b1895cbda6 +CRW_1836.crw 6845674 1a7833c8fe3ba80a5cb19c8ee6f8e36d2b38c484724123af7a2577465b4dd930 +CRW_1838.crw 6623396 9300ce381946effa449a614b9f815e1547be8e01d6a3ff7bb020c4f5d99d3ac5 +CRW_1839.crw 6210618 4387a84b75a08468768521601ecb67309e7c7eafe9a6114069c3249797dc2f47 +CRW_1840.crw 6000970 3717f75a2218bd2f38cac44c9d42f0702e256b0079c134ebc199637005d86c77 +CRW_1841.crw 6150098 d9412b3c73835442069f7f119fc7e043279dda745b1f5f18d56f36edea3dea44 +CRW_1842.crw 6757158 fdb60832789b94d4b5988d2092ff11edaf9e96d5daa1e94a0c10d9f888bbed7e +CRW_1843.crw 7281200 2bb466af8b85c2977d72e172fe30beba614a056892d9d9162c4549445456f59f +CRW_1844.crw 7531192 4573595ff53c04e55df934710c5420b6a145f9adbe4a9d6f9ec4030803ba4e4e +CRW_1845.crw 7588550 c7202765df09321dd74585442705e5abee7ea803703649da0f72c6d6e4c02fc1 +CRW_1846.crw 7813180 295dd7d3fd00cbb1d465651117e09f45d73d31d5cbe311ff6aa776eaace46cdf +CRW_1847.crw 7838154 4ca19581e4043fb70eab92e587dc150562b6f3ad691ad70322107748e5f43c75 +CRW_1848.crw 7874540 a06fd25c5dd0f58e5a59c3fbf089503dee649f9f2ddaccc45b353a3fc1243d28 +CRW_1849.crw 7804648 8aa988e46f094b69c106e74e5de86523f2c1d3d831449a6ad42d8b988f68509f +CRW_1850.crw 7734742 0349b97d061733407450bc1261645369d70d328fad688ef66a16fb3d0ff53949 +CRW_1851.crw 7722470 05f84a50c85babe5a817b4d0f216c05bcb234f9553da2bfafccd9069405f4c0f +CRW_1852.crw 7790844 09abf50846220af08633d6a264add7ddcd01bc8b9c6b765c420f945ae0df35e1 +CRW_1853.crw 7874586 e378e9765b84eeee608b589f989e9daa31136ca71c7089c959982084cda49f07 +CRW_1854.crw 7641846 7aba4090cfc1348fb8b9f09b825085688826f7539a09cc088cfbdd57bdade5a7 +CRW_1855.crw 7611922 1b3d5d433c0d2617dcc09039ab496d1e879879698191a7f917b593bc652b52ec +CRW_1856.crw 7624798 466a6233ca7b057c0437a51c0e5786db118f076eac495237637427c595c8819e +CRW_1857.crw 7661052 04d6d5912a20caebc334fa4cbbce881ff2f9e5f1a7034405d6488245935d2e78 +CRW_1858.crw 8882490 d28d37920a0b5d409f5dd1f0bc8c8e7818f9923b87b2df96b525751edabfb5b4 +CRW_1859.crw 7710860 537042b4c8fa1e7770aaf7437f750607fc57e563d2f3a88592316b0212b08e86 +CRW_1860.crw 8920668 1fb740aaaba7830435a5e3f8b8779e70dd5c2d03eb4e41c43939b254347d6d9e +CRW_1861.crw 8469054 b068c5fe21d046f329aa941f3c83f6b9875d06420c92c5f819a720effe8421a5 +CRW_1862.crw 8552100 8ee4d3f7bb974c2e54542b9d60aa6c2f6347aa669675a591970287679cfc0eba +CRW_1863.crw 7046456 118f1e3f1b48a33312ac673278a9338a4e3b79ec370101ee1d6d769c477d4330 +CRW_1864.crw 6666456 b680efe5010a944d3f0e285328ef18ced5d0470807614ce0d9f85970a37dbbd8 +CRW_1865.crw 6380878 a7830bfbfe7e0576d9ab30f69d19e0b9afafadd5814a00958b975fdc73325fa8 +CRW_1866.crw 6934998 4954c770e5e13cf3a243ebb5bd515bee54059c9d9ad323898576216dd9276293 +CRW_1867.crw 6459046 f13b7d4bcdb69c6fad476268f86fa1529764b5b48ea7cf92c17d896045607b52 +CRW_1868.crw 6432946 c6ffbb8fbd0c04d85864b437be3a348cd8b482f071b341dfbe085ea8b39bcbb1 +CRW_1869.crw 8775722 ecf6213b716dae168ab91c836259c620185d30d315a6274fbb203e0989c22b5b +CRW_1870.crw 6709944 96fce8e07ec3838b6a3a4e51b4b6063f9f74d098b05bdad97c0ff80420c9ce44 +CRW_1871.crw 6487410 2f5f555d8bc79ee58cafd6919f8fbe4c41f9ced770170fe01ec692295cdd6ea9 +CRW_1872.crw 7153722 7d0a8abd66a8b60aaddf6a7b0511b65abc72b3dd8892fbcea5ada633a709aede +CRW_1873.crw 7078802 463b785b7882b1e8e4db4321d2bc21393702b6b9302d451eb4ca74129fb44189 +CRW_1874.crw 6809648 77583dbee1a273309b619648d57a8d127b79d9ad1b4a4d2a9a1ac017533304b2 +CRW_1875.crw 6734074 4015cf95cd489a147fdbb60ef15373623b8e25efcaa3efea406734dec6df23d2 +CRW_1876.crw 6844734 239ec784c990b157f99be728c0baca0eff38ef0e441174235db4fa9a7046f9fc +CRW_1877.crw 6667958 92fa542c6fa614f4b7b256c08a33368be88622d19b436f0110c77dc6cbb9ba64 +CRW_1878.crw 6473626 bf423dfff4df6c6914d0b996335321788028279c1d3e4238d5c03622d8bf220c +CRW_1879.crw 6721972 8deaa85c32da6ed7a865080239788856523cba241639bbe4ba00fa0cad66fb45 +CRW_1880.crw 6551212 1bcdc27190fdf0c698ce29ae89ce21b346e53fd10aa8e90f466f467dfb7384c9 +CRW_1881.crw 6700468 7b2eb15a9615875eb69f745deaf2da8b1bd4ce700dc233ab5a1057fa968374cb +CRW_1882.crw 8351834 c812f0e74edeeeb772a90e9d739c431915fa8a57b36eeec52dda92167477e2fe +CRW_1883.crw 7066642 c3ac3df98375cd4c9dcbacd0702744212b5fd9b573cb0e100fb3e1e8492a3ee1 +CRW_1884.crw 7161296 187a0bb400969548551752047b51bec0fe8a0b8d6a66de04641f35666f5d5440 +CRW_1885.crw 6544628 e4bcd5a9679644869c68ec17a302041c44b336180650bad0fce5149b4dc0db16 +CRW_1886.crw 6594936 abc5dabd527bd40aa7af12322eabad71810bc3ad15e1fb841e0ac4ab1ce33d7c +CRW_1887.crw 7734812 0b9c7c217ee3a5886d3a2a94829b21c9c2369a6ef468d8799f6cc54573f888cc +CRW_1888.crw 7324806 6f8f49fc97f6c1ad18fdd41237fd653b5f18dd44b20bb533f49887cc308f3555 +CRW_1889.crw 7240236 3fe39ee9813bfc300f2365cdd21dcb01e3da90349ba2c0797374821253e4cfa9 +CRW_1890.crw 7522646 68f2b228bc7145b46117e261a20ec1d2cf94844b53fd3f27aee7407935d70d92 +CRW_1891.crw 7056912 a9e168ba62d984ac7081fb1bd2d6b71814a7f90f53dd23b07206e8bb4f018030 +CRW_1892.crw 7567512 d8c11ab78a3fbc15205b893d0e0cea31835feb56b514af54a080824e29a50e8c +CRW_1893.crw 7312970 c19562fde1fc59bdd6e5c4079de90a9389d8fa24b2df58d863f4c06700687a2c +CRW_1894.crw 6936298 2ffe951a5dd2888fe3152a8bdaee0782c1f0134d311fef00d890917073c4b5b4 +CRW_1895.crw 6582132 beff4dd0f3ac17d1f2a30b49f90eaa4e95de407819c12e06e7bcb53d60562ecd +CRW_1896.crw 7207014 6882df28f88ec1954e6d5e293b902b6802d375a9ca5fe7eca9b35b461a455acf +CRW_1897.crw 6583100 2dadee262372165c9be39ec716c37ff4342aacf8d2f9800d80c391dc8374ad44 +CRW_1898.crw 9220738 0441fec7025e6eba9e90c768ecba7838c5a4064c75344a1fe7fd861f3f681a8a +CRW_1899.crw 9822604 5a739ebcb453fcacd11e3ddd46f13c195160dabb8c09439b01ce8ac9672bf796 +CRW_1900.crw 9815218 284834c4d459fcf6a0c15145efb2add75d121cbafb6210ff1b5fc68dd7a161c3 +CRW_1901.crw 9046014 83fa5956b213a279338f89dda0e2ab3b597e85805aa71fd8b9b311bc5ad4e756 +CRW_1902.crw 8682030 d84ab8a77d8ee8d09d12375e54c2228ee6461e1ad77632022b35a7ca5c46b45c +CRW_1903.crw 8948740 1309c63f50535650ffd4ed455a2247ecd94066746a0475304f620b66f53cb4af +CRW_1904.crw 9489874 3681d9b46853d8707fc22ef997ab4ce941537311497211af751979953c59494a +CRW_1905.crw 8762786 04b21af33506546579d08e85dc8907ae97ef0ba2b4637b32e3cf57227881147a +CRW_1906.crw 8510982 945c0c32f4a0e4e2d16555f1c02272dafea5824b2700bbcea6bcccf0121ca3a3 +CRW_1907.crw 8521532 279f1941f8f8eaa84aacec8ab50f611875d5682848ed1344e63a939a9d1051cd +CRW_1908.crw 8692990 1296bfb1af1c070cd8b48a01bd3ed53b88f6c09987b7ab711983a7744c8e276a +CRW_1909.crw 8743466 59170d494f492fda18b6d3a7479e195b6c1572f4c49ac1e6d6f450ce86bcfb01 +CRW_1910.crw 9071458 e565c30fa82e8b3c0c96dfcdfdbd7f2938d1920c23957814f833417aac9e6d11 +CRW_1911.crw 8584086 6cf261f7e70abc385b442518cd8434e255b991b036de33891cbb355b9fd74f46 +CRW_1912.crw 9149894 cc5bb2c5e5913ff68ea1f5719bba44e977c141e8710bafdaddda79d63a851bb9 +CRW_1913.crw 7802254 fe1c311f1e809bd9423375f8c5baa577a32ee439aa8bd0610d3d9e2befab629b +CRW_1914.crw 8373886 639917afd69967bf114d0e092699556ab3a9f7d493049926c41ef919630b9ef8 +CRW_1915.crw 8830864 33b096d65ab76914ec95bf9bc73e9fa5c2d951aaf6744548014de50de0d75024 +CRW_1916.crw 8395094 6988674101dc7095810c67c6df995246798e7a4fce277ad6675d76b866d8ab37 +CRW_1917.crw 8068438 a27b948193893dcc9606343da61d8544b89ada2a2c56aed002073b6c2b879d1a +CRW_1918.crw 7167268 f00f563fa71654c4dbfca6890634805665d1b5f5b81140a99895f65c1afe672e +CRW_1919.crw 8763964 2375093ded635003f5a0f82ab94b86615f5d9e17388d2bc363ebf41e5051ebac +CRW_1920.crw 9125506 d731588e3b4f580cb23742d4b624514bde29a411da8c922d2fb917da27c683ed +CRW_1921.crw 8095690 f4ec95ad2888407e8350c7bbb523248821f401d81c7263a88a6d778330f554ba +CRW_1922.crw 7548310 256aee78a7fd02e38457263b1fb435daa8549029fa1bbe915c468dee11c41c36 +CRW_1923.crw 6744790 ac438fe2df968a9c39ac1ed89949a369be80a8bdda67d661c3084276264f3322 +CRW_1924.crw 7568184 bc1773945ff5cc773c08297f9cd80c7026ac0fb29211b67781b80dea826a798f +CRW_1925.crw 8491294 1726e9d69dc4e77ef48451ba56463004a395b7f5f7d2f323a517e2a843c8d903 +CRW_1926.crw 8295532 f48f7f67c6702847e1c38f01aaa41ac7114038c47d32410dad64ed61d8753423 +CRW_1927.crw 8612472 0e46a0f64e026484b11f4836cfe552312449ec45eae7bd271ae628adf69f52fc +CRW_1928.crw 7404886 56c5d3e4610fb6ae6edb358a7612e73b47681224ca4f9ff4d00f0664a27b6f6c +CRW_1929.crw 7428096 6a91c57390f40a497959b12cb874ba52c4907ce6740989087913e1c4723ac5e2 +CRW_1930.crw 7854508 5b7bdfa4ea18760f1305034e7f9c0bf425a3759812e3e38874b119a445bf4f40 +CRW_1931.crw 8187976 0957563fcc98f100ee54f4e0a27ed03a76e1168c942c07f5c1be85f73442911a +CRW_1932.crw 7765886 a0e03b58fe66f0236e6024210a4f503e7b7d7e18002e1203ec31ad04be9b414e +CRW_1933.crw 6509138 eee985839800473fd6bad03e80a88a93f8489dfeab74775094a2e5b6268bc981 +CRW_1934.crw 8379606 6623862fd93708b6b85ac4760f78ec228841427f5172d95213e010eee13365b0 +CRW_1935.crw 6855288 770d20bcf6dd37dea218f4e7cf0f2731dd7dfd49462e195681ceaeb2446855a9 +CRW_1936.crw 7069564 260505d0c1d7575faac85e206cdd6878074b6e56f5a649febd46ce9cc85ee595 +CRW_1937.crw 6916050 b22e80c28fa52fe6fe4d1d220e41d3d5ccb7c834fef0d17459fb58a62f1142de +CRW_1938.crw 7434024 2f84a40ae843c42cb66f4043ab8ba7756bd7b4f3e764f707eaa00a2f385be373 +CRW_1939.crw 7316674 02f6bba5a597bc59a6a84026d317cfe87f04b611aefdc5fd22f7a4977710ad29 +CRW_1940.crw 7210478 b131eef1975c448c42153688faed93d4668cc75f3514bad57b2f045f03e28d32 +CRW_1941.crw 6998358 24d9005c3c88a9d5a149b2bd496add43fe4933b3e5c359f3e5f7d3373957a256 +CRW_1942.crw 7815564 ddff1cc1f6b608bb77adc469dece35888834c8f4202167f58466fd43631d43f2 +CRW_1943.crw 7891206 a71d3d304b881f7fe66395bb122faa6dbf182dacf24f8580234be353ece8e2ca +CRW_1944.crw 7148422 917cde74e21b55a3e86b76d43a32d66f93f0b9a2715485595e4393e86adcba3d +CRW_1945.crw 8003210 99b5b7f58059041970d6408b79dc5edac3d0aecd87837078ff0b37497d5fb0cd +CRW_1946.crw 7115696 947a2958ae87fa36ace1d86e565dadebf5d91a4f82d647cfd43072216ed3aed4 +CRW_1947.crw 7134558 31d477b4a9a5ee44111d70d883f2ef3b850e980a6b985eb00836ffd57efbb6a1 +CRW_1949.crw 6547696 09649ea911c917d1caa4d423d1d2d839394531c69f0cdd1a1283920c758fde67 +CRW_1950.crw 6526862 f625992563349b84b207e4a42a25fa26f69fc94ae2b47381039edbd89b5dffb6 +CRW_1951.crw 7255114 a0c4888ec8b06d4a2da40afcb0264965a08f64439d287f9f7112e6af3d3b76cc +CRW_1952.crw 7035120 e4329ede9aa364416ae131c06b52733b37001ee53580a516afa4cc812a963a78 +CRW_1953.crw 6893708 af987082cef241552353b4dcb753c76440c27e2ef85cd248ef9ef12a7f14f696 +CRW_1954.crw 7894826 82335eb1c6a3a734d2eb62b90be4490992384147be962cf7331f7530c5e23be1 +CRW_1955.crw 7262856 007206725fab0ba7491d2045418c34e5494e13404b7e9fc5e87e9d9652c0ee41 +CRW_1956.crw 7839534 895680d34b00dfc419f1ca89019ce82940df63282804c1ed3b4dceb2330c59ab +CRW_1957.crw 7779418 a3ba4dbd97f4bc2da7a2e950679ab54f3bf6a1c5416c3fe71ff7a9feea0ecf5c +CRW_1958.crw 7770916 c30e4e055eb5054eeda124c474ce2e7f0c2a649172d800d754f3f696ed1c9375 +CRW_1959.crw 7883810 93db1c1a5c49ba1d3a9f884c9c00dd0b195233245cd19a9a8633efb557c59ad3 +CRW_1960.crw 7502752 11761d0234691f4edbd71f9fd56b9e11ed1977be1da242ad1adb9afa46208df4 +CRW_1961.crw 8424846 3ac9060ce676b601d017ab1d6fa547b8b72a4765b1c65e812eb19a5c344c1d2a +CRW_1962.crw 7999212 464c57294cceafa9ba94c42366ebf1059345e8bff718946507f0f239da0197a0 +CRW_1963.crw 7616780 b836c38cb0291537930523c2c8cf3d9cb816a717875903579755a2f3d2397047 +CRW_1964.crw 7585524 e3e9bada0f467bd5ea0586166280535bd0fcc0fdb2c1a09586f0e9bcd2a7b029 +CRW_1965.crw 7309912 5d4fa64e45a159513357c312f0e7c15d69c2fae99c51f0d59cc4699234dc37a9 +CRW_1966.crw 7278604 d16ce4cd7a6d0038fccbdcf23de3b5330a94cacf6bc5505fe486b3479f2c9e95 +CRW_1967.crw 7874362 9a5303bfd18c6b2ef69047cec94d7c405ab20ed72413463b1a86ec08ef5ab47d +CRW_1968.crw 7634396 50fc889df8f2f74c882e543ecf92221044dbb5d3215627260a0e8e081d681f18 +CRW_1969.crw 7105240 b023edb8b7f131d47fa7199a5f4cf0ab2ddc9f26430d963f8e5335baaf84fd7f +CRW_1970.crw 7212016 7db44de8fb8db890cb166d82a4ac1c6d067b96c58547e4a8d244d5d25c92f2ed +CRW_1971.crw 7071176 30ddeb00ebddd44764a6556e492d318b5711dae7b313fdc3b85e36e4ab7d8796 +CRW_1972.crw 6973390 77b9c7b3a2dfef8e1932a124d50a71bd36bbb87c288ffb26796010c7bfbb73a6 +CRW_1973.crw 7022276 9600f3e4b83455f79993fa43e84beafa6ed90b0067927a1494ad51bedde15008 +CRW_1974.crw 6799952 510c15ae63563038d4ad3735367a55d14a3b8817cc5c190a9b1a269d6487df61 +CRW_1975.crw 7226396 dad81d15491e5e9692e1e00a11c6eb5640661af53e56812c7f8cb8261319040a +CRW_1976.crw 7108394 272d023237a082bd15d2f32eb438818cc04a0fdaaf2f675f9b57b15be663753e +CRW_1977.crw 7176384 de9ab98e7a9adb53cc8bd9aa7d2928520082d9b1efff15bd8f6e8fb312b34f36 +CRW_1978.crw 7297998 2454771eeea0a9d3be16e2f0448739025eced20206d1975c220e5fe6e7ae83a9 +CRW_1979.crw 7346266 7ba9f3c7deade3d3e98c4e994cf6fbaad21384a5cd06dcffe4ae2e8761c87ec6 +CRW_1980.crw 7126154 d2aa0f22e6f9d543027928d9c7750fcda4ceaacd913d6746c4692a565c56d3b4 +CRW_1981.crw 6803080 c5e75a1c51e56ca9331638b6a5c2bceafd34ab6e394bdd32c1ac7ee4b9815679 +CRW_1982.crw 7010628 98ce0027b2fe9075534a5f7e0dc9f28232d9bf5ec65151974a88fd80290cd3b5 +CRW_1983.crw 7569218 34ee59fa5b54c7fb6f202b9a3d52ba51651c6b6b502464081824ba6c712e9846 +CRW_1984.crw 7767074 ea7412dd30cb5b0fb4764afc55ef9911754be74426e72f17b9c0a8ae80ce2b8d +CRW_1985.crw 7072242 3043b6df30d233abb2a54c16149a850ba6a1222542ae6343971e57309c4eddc1 +CRW_1986.crw 7195918 f58fff7404a707496ae5b0e9b5de06e7a0a8d7adbabd257ab89365735e8d0da7 +CRW_1987.crw 7272664 bd613e94ddeed674ba0ff17d7c81ef7d8a66dcd2f9686ad7183c8e72667e1c4b +CRW_1988.crw 6065700 0acd09e5e5ccd8305af3ccde49eaa7645704cb80b0ebfe8fb96415b3aa1099d4 +CRW_1989.crw 7227898 91b2fe2791753b87586dcf55f48b4b15f4cc0515a2f16b1526e88ca63896b56e +CRW_1990.crw 6534384 2ee8f3360d0cc1c663de20ec6e2c891808388af2bdf654ab656bd6d99820be39 +CRW_1991.crw 7491552 fa089e1aede7ad313997ce628221cdd2002a534f0d1e6b3574de9e5cb2aa0472 +CRW_1992.crw 7726060 d4616f1008ce4be9bdf5d6dcf0a0174acb405c654f02e9f91789b0fc2078fb75 +CRW_1993.crw 6431994 49a35c34812c2613fb46867e3bd42e3f1ecfa01ea2cde6948315a9f0f3070b46 +CRW_1994.crw 5928432 ff07d3af396dcec6a279947b66959eaaa29fee571ef74f90edb2a8f2bb3e1c6e +CRW_1995.crw 6067920 e5438a17566bb2cc68da5950e13b196f859f7b894e163f5ae9bdbb658b7c39f1 +CRW_1996.crw 7811994 2994ef6f76bff7792e9de28a38bfc6cf578dcaf903fb5f3626ac145ac1ad4a5c +CRW_1997.crw 7869464 783923efc4a0cc8da6baaa513dd6ed2241b18defee50a70a8948af76c706208d +CRW_1998.crw 8497136 2c8c1a30c637951412db742fe95d228297cc7565cc91802fa7b9a7a90875d989 +CRW_1999.crw 8483072 b03ad30d92921a9ff8a686bf2909cc3ff30519331a855010c2af8f571ba64e4a +CRW_2000.crw 7087206 c186fea29c86112f91dded64e24f958f13305c5198de7bddd07b8db61f04dfe0 +CRW_2001.crw 6949106 a52ddac475400ecdbe4648ea96e928274ff7f408d1cdde31d6bfcebfb2cf4f18 +CRW_2002.crw 7634410 df424058c4734114822e813de73f862d547f8001e1ef3f035e8816dfbf074749 +CRW_2003.crw 6284480 5e2854ff57e7cd04031620c5cd14e73be65311a42290de710654bd1b72ca094b +CRW_2004.crw 6232922 937681102389a5b4a6ca0c6b3c74fb07c199b1b6293f16df7a0c255a6ae8b4e2 +CRW_2005.crw 6131834 1020addf19827169aa4ab5e885c529e82715037697b8cb9471c7ea98b12f5dc3 +CRW_2006.crw 7870338 d723ed0504cea5b130fa231d477512cacfef6aadbd00a3c424663f7b40254396 +CRW_2007.crw 6910902 72c09adffa5cdb17a43af070f9165b98eb7ae9182780d6846a070980f424a071 +CRW_2008.crw 6374040 5a4d2faf38a4fc5f382cbb792c4355e2ee0fe05f9b74b17936d4a439e2dfc1e2 +CRW_2009.crw 6255108 6ad93fbbc045bfcba7c20c3ef7d4473d5321dae2590f2e7a7545fe9891ea11ff +CRW_2010.crw 6829458 819ab2c876b909c2884e1743e990c835e1ae68754271c4b1c3d51d233f6cd213 +CRW_2011.crw 6517124 53a26f69e5f7a72366ffb378d7e302337ae6e83c9fcc992fdcdc8f74921426cb +CRW_2012.crw 6477512 f02771256c72c86918b9451fc02e3475eee9ac7c1fa4a768486e385c4ad5dddd +CRW_2013.crw 6582452 f257082d330d2c86faceac1f4461c4de36a71b3a5e1e1307d19a0b820c012f35 +CRW_2014.crw 6106664 65028efad40a9281062f976b2348f5188f4c48b35ad410b3dbc3dfe46161762b +CRW_2015.crw 6168948 657559708f90506e354a8d6fd2819b850dafc5135bf69d77f37e109625059c7e +CRW_2016.crw 6329900 5d8d4d3ce0d62b1578c57d466bccfcde0719de2bf754deb634ac27ee9491d4c9 +CRW_2017.crw 6466830 780368bf91ea2ab0592356a14782bde42c008a7915271269729860470a0e7aa0 +CRW_2018.crw 7651726 079992f7d8e033f79437561e1bb37b23d633a18e3e80970f6de90a7d159aa43b +CRW_2019.crw 7313712 392de90e395e6e4443aff28326dace2ccac7c833b705767cb6fbeecfa5d60c76 +CRW_2020.crw 6663742 cd3f3189fd0fdce7af464a6c78493703886055a56a10880b5f30d2c9817ceefb +CRW_2021.crw 7185668 7aa1b7c5a1acdc517bb7183bf72887d6af9b89b6269b5605dda9345e203e82b3 +CRW_2022.crw 7244512 a021f1f3551730e77433bd91f171e76272834d9e99bab51762e65cbb0b98c19e +CRW_2023.crw 7474332 c8b06273358a643c455c2705364a827eb536699ff3724c49c6ee7dcfc7892633 +CRW_2024.crw 7113002 01908b56e8f5e7389e97a16726fd710fc90d34897e50fe75b4e891c9cafeb7ba +CRW_2025.crw 6952790 cb0316cae5bc3502ae3698142c2bfececd9c5a3fb07274e67b6cf25d8318f8c0 +CRW_2026.crw 6317272 74e19e6feac6d4fae59f1f45c5676f94fab2b00fb4e576dbad6989a0e54911ab +CRW_2027.crw 7480478 77ce7c94a95060b6d0e0bccd2976a8c43281301604b9255d168932169cdf7108 +CRW_2028.crw 7696802 f533100f16bcc943b1192dba07904c9034bbb985093bc7fa6c9b40af752449c9 +CRW_2029.crw 7597250 5e9d9959077c78d7cec57828187e46205ffece06d4a8bab21839d99bce025272 +CRW_2030.crw 8174750 466def4bc43d13152a8e5834ece5d842d65444c788a68dfee9f1da42a0e14e41 +CRW_2031.crw 6582724 8d9aad6ec402f8fca99075dd862167ce5c1659c80d971d542455b857c50a78a0 +CRW_2032.crw 6591226 d0d4aafa04263fa7ef060bbc1acdd5eb79dae213d0ce68c53af34062e3dbad45 +CRW_2033.crw 6583792 931e0236e612ead3e310b8090fd6f4937a7f7581f3af7e59c0ccff369c2d61b2 +CRW_2034.crw 6707660 4b982c4c602d7ffea50f967892c2be2bd33e73bada6bb2b9d6e523335cdf50cd +CRW_2035.crw 6595632 0ffd9fec448a667b7850ea505b5b117f01c07a98aff5e0b7ec9da7fc5909b4e8 +CRW_2036.crw 6058712 1b8a0aef4e767105cd69d9cfced3455c152c07ea49528fd9bae0cf5b09190109 +CRW_2037.crw 6026298 19571d1754546a2e56b37a88fc45d4da77fe3a7f61dc1fe4ce6082276d81a637 +CRW_2038.crw 6454452 f83058b5d9613ce529509fa3849f46aea2012073e6a86482a3f9999417e28ee5 +CRW_2039.crw 6631442 882f2c180df6c348c186d5f7d599c5e8f4c1bf9a4f3d530ffd8c72d48d13a5b3 +CRW_2040.crw 6317436 8fdc25b99bbd4de9bc7d5b7d3e5e75ff70a7bfd0221cbaf3374e916c0b581fed +CRW_2041.crw 5984790 6ee602ca6a793328aee3f49aa17546ce2e8f3d2f2941e809ed280af5b142437e +CRW_2042.crw 5626612 4f4e124c6fc79c07401c8701fb7708b64b3593408c4e861d004ef14e3b9dd4ff +CRW_2043.crw 6016270 8de431447104b407a5f288b14a2060e62154d0efddc8b4fb633b288e6d00decb +CRW_2044.crw 6118842 9aa095bea6f2b03d13183787300d81e072a28082501a88caf1f0efd0f8e51e9d +CRW_2045.crw 5991580 215084c318d03d4a3bb2f8090828b67d355bccf4b2b68cd76d39138b510dcdd0 +CRW_2046.crw 5960462 2ebb014863b499aed3ae67601628dd4589cb633dab29e8f90379e812a024830a +CRW_2047.crw 7304972 15b55a2e0ff443f3b3188b3c1a9a4a2b384442a1dd7f1089943b7b55afe3c4dd +CRW_2048.crw 5993386 9c42e173fbc9c8eda1e3bb9c8c4d0ab0d4a728c4193288706b56bc3bcf10084a +CRW_2049.crw 6309054 fb6980ef71dd2deac4ee8a85692828922140e2fbbfefa391d48fa05585875803 +CRW_2050.crw 7421828 fcbf6d0cc4d8c57b548633602e4e9486f81fd5a599aacf166457234fb3046574 +CRW_2051.crw 6391522 3bae2ae780b4505c95b2d13d0b2784ba077dc59c5d901646f5baae9ed93cf967 +CRW_2052.crw 6460164 49af00e5b28e20ef4d437700b9b218f70e2b04d3808557809d0df16806308e39 +CRW_2053.crw 8186946 5947cbe710e16023d0aeec33983c086f8a59be158911bc8de1ab48d90c564b3c +CRW_2054.crw 7613376 6648e828adf30ec42ba6f4525eba1871ad715204f5042216053fbf06fbcfbda0 +CRW_2055.crw 6719620 098317bb9b9e355e1a4497ff30cd18f9c0db93cbd16c779fc124d02d94b22a4b +CRW_2056.crw 7645968 0a2691511181deb22979436ea8564abfd1f8825e455b6e8b86eddb85cdb07c32 +CRW_2057.crw 8739480 b2d59a6ae40700b848a695f43c07080599ea743b05f8937d399914a3a0a46efb +CRW_2058.crw 8098416 f8bceadf165dd114ac90ace2de6b4e3c4a7e2bbd21bc0fd8ae3107547444b069 +CRW_2059.crw 7694810 cebc0b32b17c34e28bb04c185636076cee9184e7211a1598c7c19057ed5853cc +CRW_2060.crw 7740694 8b725ca9ea2e7447bd5778d4e2cdae7f429207925e86ce6371f57c8ac3e96f70 +CRW_2061.crw 8038048 855db483938d0e2d8fc718217a8d5a59d10c1bd6e61326c03fb304275a8959eb +CRW_2062.crw 8615728 981a4ade1011bec0570a51e9d67c742cc66f7d44ec1b286ed31208c2112991a4 +CRW_2063.crw 8639802 daf9cc9660272974d96d2de53a22f0f6c51325aa7d10aa836fccfd8f61767c6a +CRW_2064.crw 8238122 c5272e9aa3dd0c2308146dc19d987bfbf699cb2ce5f4f6cd82973df4bef54c89 +CRW_2065.crw 7718412 909f00009ac26c3c5dd3c5291daadde6d1771c5dfdb3a0c85a545012208e530f +CRW_2066.crw 7859958 b10b37a85f5de1136ba83e08693c07a1e99cde4772b30e55586c6be721c8d7be +CRW_2067.crw 7446352 dbf8846b375ea742aed14b95e29c10eeac896b5a0269810ed68a1d8515237267 +CRW_2068.crw 8101500 a016f16079915c561af30cf78a764c99ff38ae83c19a5e55c9d851493c65b4fb +CRW_2069.crw 8088446 c40f3bb672717de3673d68707a29e653d975508efc4edfa8112d2d625d930200 +CRW_2070.crw 8527114 96907f6cf73e1454839592e885bc24e8ea85d6bf3985783eb529da2b0f9fe48a +CRW_2071.crw 8526052 4311935159c66475dc971584d013a563c41e33c8adfd2c8ed014aa40f3df0a6d +CRW_2072.crw 7412466 03a5fed523aaf175ef0549e66a438094ac2666ca3a6330059628e03ccdc179c1 +CRW_2073.crw 7919440 ca79723364d26aed23e9e2fe8b95e37150904f27bf47c657fdfdd7a48215a033 +CRW_2074.crw 8603556 f025558304c0816f672360835eb3158ef3437c82e68943e73b0cba235f4fe3e3 +CRW_2075.crw 7631914 9d5c1bf0d7e1f28b5df7b07a841ba46ce4b03dd64deda54492480b7f8156b525 +CRW_2076.crw 7667072 733c6af494582c35441eac13a8f0ebd459979a483efa242a7fa1a3b2ae4b94a3 +CRW_2077.crw 5860736 916557d6ef755785fcf61256a8bb3c0c29b89f011374870e64bec9406c6b18f3 +CRW_2078.crw 5863278 3036c8adc934a8af7ff1d9e1671e5e652d4d88496bd8ef148a50c6253894985c +CRW_2079.crw 5848170 c248d3bf5a19a501880707c336e564a2e12011a2e26a9f20dbcc49fb13209ccd +CRW_2080.crw 6907548 1196fc9fb0f74743afda8561a89d23da757839fbaf40c5ffa0e5b42e7a8b3ded +CRW_2081.crw 6229322 54d91bd0b1610b7aa646bbfde6a8c904a59d259b1b36c1b467872660d4874ca4 +CRW_2082.crw 7688154 7dddc05edb5dec2079abd00fe74c49036f345a28207a96ab8e3db97d87718078 +CRW_2083.crw 7736758 245e4d0f4bf20af8569caa5dfd7ea040949cce1688d57edc4afc095500c98b94 +CRW_2084.crw 7858910 4ccc544979a9db9233db586062aa53e7b123926d236815459722293c94e13199 +CRW_2085.crw 7721298 039eedada121644469e72532bdee2355d0ab06f14230b185ce5fc61eb619e629 +CRW_2086.crw 7879040 e93586ef5fa06447233e6e6932cfddc4134809599dbf36ec679fd777795eace5 +CRW_2087.crw 7106792 9bcf719d8281be2072e0b5dbf2767541e61a937a86541edceb7dccd6a0f43cf4 +CRW_2088.crw 7284652 0309fda80184c0e4bb7873de6c800be4661fffa30463e4adce25a22d9f3f8b42 +CRW_2089.crw 6937282 1ec4242792892876988c9f941c26a1ad5b42853f62cb9f77c7bb37b5e539071c +CRW_2090.crw 7337554 96a125f77f47d5d9fbfac240babb17ebc6ba01457e92c31a38876cc9cacad3c1 +CRW_2091.crw 7092318 dfcd0a4e11e0544227312043d9e4dcb58a5346e2e5b857973c370bb9b1a410d0 +CRW_2092.crw 7202732 bc6d741d9c2d239fbe28338c3b0ae6a13ea59ffe0fb8832e3f73c8cae395a069 +CRW_2093.crw 5810360 be71e13d3f086184e5d67f192685cc18fb18f430d220cc1ae1cb572fcfc4bb64 +CRW_2094.crw 6909958 e5890aed56b78d24c5dbc7b658ebf6164fde00053098941220c1343d74f753f0 +CRW_2095.crw 7376448 4d1aba250f9183644c8fa8e9106e58128fc2694a249f1121826fe7826468c922 +CRW_2096.crw 6438020 e9980eaabed6830980d9251c96ef8e29eddaf41a0069cb9a4a278f97b0d29c55 +CRW_2097.crw 6474774 3526f41caf2d2c50432b37f067a50ccbaed03b2efe19e908206ccd32aeb61c45 +CRW_2098.crw 6800566 75d5f4ebad2deddb4547e9e814fab2b8d0c7d6883bc8510f2370b304555aabca +CRW_2099.crw 7627806 72676125042cfafc7d7e91de90b6186ac48359fc448fa6ae9917b81d52bea4f3 +CRW_2100.crw 7814644 05b88bcf21b210f2a4424aee402935c485a31d8557d804456e5020286c10aa0f +CRW_2101.crw 7564304 80b166956eef4baeda6702cba369075b45313bd4f84bf7c995009d57817a378b +CRW_2102.crw 7902652 2259f4d07665039753665d891f822005a452c1c1f44c5c8ca9766a2d3c5a6b57 +CRW_2103.crw 7699132 96b3e616dc9d396f41d4059d0b1b652019b54ef82dffe83def393efc642c0bb3 +CRW_2104.crw 7697256 c6fc20e4903da4cb4dec28a82cfa52b2efa0d39016749c7ddc71d8b99b4107c2 +CRW_2105.crw 7715884 d7dd1042208202400352ee6a485d1fd2c944b3834afee4a228672b2ab5f97a41 +CRW_2106.crw 7568190 3f20db53204e9ef8f7f3264ea1076fa268639d8569e648bc288278f3587861ce +CRW_2107.crw 7047034 26f254940c24d7ff3caa77a45b13235afefe96e5c8dd8315794dd0fb8f5ff6d7 +CRW_2108.crw 6452540 886e253ab825dafb24f6d33e88f64ed949cccf4b962ff598099766207e711314 +CRW_2109.crw 6449600 5039480dc9e6fd9a853627aef02a65018a6be6e1379ec1b4ce2636b656cf739b +CRW_2110.crw 6464620 d211ea9d20d25b2403f53f6d7bf1ce2cdd42a7372cff30fc602fb3525ccf8d41 +CRW_2111.crw 6831658 fdc2541a6adc79960cf7c3545a52db44adfccc7646b87b84ad7ec7b58006b7bd +CRW_2112.crw 7356466 56d9b331a2d2be6299401ce457f0db6cbe9c77f07fa9e1f1002fde6a21825040 +CRW_2113.crw 6788240 0a92c101b2b679bd910eb34702e2c7c23704c934cd2bbcd4b9836114b356c803 +CRW_2114.crw 6808600 c27cab7ff32e60c37d2aaca2a3aab1b5aa33b9185daefd490c4266008fbd0849 +CRW_2115.crw 7578280 69fa7d20fa9da56166f803b4714e5cdeb3600ae937e730f3c835a8c04936e5a6 +CRW_2116.crw 7477330 50e301d053b062ebc97fa2ac14ea0923f1fc9d00834d73ad8e0f69f022ace6d4 +CRW_2117.crw 6135702 9e16d78748d467e9dd8dca9b3d2c288866b4cbe724d1172b5fc913d05144e25b +CRW_2118.crw 6052144 cae959f609b60eac06f4a532a6f5f8e938e785deed46468e552d6892ed650f06 +CRW_2119.crw 6675992 130ea274662b0dfa90eb563966644448793e3e375f86a4ea09f8d4dfd98419b9 +CRW_2120.crw 7454448 c11376a68ecc313510479432b5b3b35bc2702cd261b759198fe43fbe3d3c3f95 +CRW_2121.crw 6692960 f5fae769a90b24ffd5f3aa7d86b099b2f09e32a887794c4015a92aeb0149c5b4 +CRW_2122.crw 5058256 21fef4ba6f2ee4c4c0a82810865fa4c799afc56cc15907ea0049a3e5b3bb6428 +CRW_2123.crw 5710502 7b60db51e8c96e842e009d6e1f8a67d0848ccd7fff83da9f915c885338442ecf +CRW_2124.crw 7090104 5c35ab0945f43918611b7fcd88e0eeac7263afe3e24e5b84dedb57d2ce1efcc8 +CRW_2125.crw 7042532 b9ee46e35cdf81f4ecad0e28cf0e95b47aa57f72d4353a126d37a0ca15785276 +CRW_2126.crw 5254678 a7a10208aa6b731c8612e32a9a699f6f03b4aafd09a9bea5f4304ce05f67e763 +CRW_2127.crw 6159368 3206fb94db176ee6afbd0005012027eb8e45a1c6ca15d079bd84c94700d5f7f9 +CRW_2128.crw 6686296 0064f082baead8d0cede16fd9e6fd6535d412fb19eba5cbb1de563d48bd8a34f +CRW_2129.crw 6553752 ec5a63909b4774ba1eedf445e512fa504b12880086a0a5fb9858c6d3f2095b79 +CRW_2130.crw 6528382 0f6a4b91eb73a03b8e838e0aa73f4e5f7cf587c8085993b3170daaad593e8e89 +CRW_2131.crw 6793818 c823d9fe8011116731a34c7a30fc5e701cf5b359d64482f0289003b68719b982 +CRW_2132.crw 7458850 129bf9b90cf94e740dea9b6eface4573d878f73bf54a55d7de814870262518fa +CRW_2133.crw 6328068 6ee2e8908bc38d1dd71cad6f4dcb4175960588efd3d46f36e34153d873be0e6d +CRW_2134.crw 6319346 9a4b503cd1228cb4ca8f8d9a3c6a923514795f44676fd36f28d3b61fd932856e +CRW_2135.crw 7572568 923456ce18555e9788402e8b917952d8485ead3baacd085e68b4e4965ed1ba59 +CRW_2136.crw 7024886 92b6c326703a21c37a99bbaa95ba724a9f517ccd3d7ff35afa9f405fc93e8155 +CRW_2137.crw 8676314 085e5c9ca11222fa9b875fac3c273b321a1a036b80139673c812ba380e9ae595 +CRW_2138.crw 8666568 6bd597a65144840e5c8b372f55096847d3fa963bba6b7c57372856d5f87b3290 +CRW_2139.crw 9043048 1860d0087f08fbfa51255e7316f650c0f868552ca1e4f1ae96b6273b585b96cc +CRW_2140.crw 8978738 85ca38b656f761f62ebd7fd8f3aaefbc91c31049bd90fc2aa229196b7f815757 +CRW_2141.crw 7877294 f56a2e47ce63d598bd13c7e8e6ac551787bfaf304743530dedc75f84f3981762 +CRW_2142.crw 7193534 5bde002096ddcac91190bb44a5107f19e153b3fe363755024df1062b928f6bf8 +CRW_2143.crw 8634760 70003b4a341f29449bf572cef714d685d7c474bccef51cf79b6f476ec83cadad +CRW_2144.crw 8672528 67bf553a5b5dc56371bbd7e519e1913ba8ece9df01f6e629011b05b8514ed553 +CRW_2145.crw 8758146 ae302d611bf4b1ba810763e3643b38f7d89d8f8e622ef04609e46964e9cb4ba2 +CRW_2146.crw 8611232 056f4fa40d0784572a580b14ae941945295b7fd9452369edd951912c63112159 +CRW_2147.crw 6861832 b1dfe18c2ae187dedde0e241ec6964c9416a6a9864371647a7e5d0757fef887f +CRW_2148.crw 7213382 f9568c6b7c8a92842664bc229ef2f69e224cfc77be9be408ae8057a6ab956ce5 +CRW_2149.crw 6313708 96b498061453f44b2bde701ac7d1a6e9f6f4312a55a9c9ac090e77f195f1018b +CRW_2150.crw 6289514 3ab1ec5b797650a8696283f91822d613642ce57efa928d802f6129b031d9cec3 +CRW_2151.crw 6829354 fa12a1931a299cf805c6c0c02f8ae0d0e2e7660a35d59b67bff13b75eeb80ecf +CRW_2152.crw 6663756 ea942d08be37919dd922286fbf7160b105acab7d7aee3e723605741dd844e0b9 +CRW_2153.crw 8823608 0b0704d5aa18f4fc4e9b6f98131b4612749fb08f921ff786300cb1e0989b0bcd +CRW_2154.crw 8057340 e71a34771e45fe442b70d89e0b3ece288db51bd8ad0a15bd563c7a6b4bbf7474 +CRW_2155.crw 8095012 298e4272ea806c10e3af6bad82fd22c9347da4d848f25b829bd914c5c85c4bf7 +CRW_2156.crw 8187074 8a06b7fd4b36b005c42af61e74763f5da0253c68670a33fe6b25cdca930c1d1f +CRW_2157.crw 8378766 2ca20e9510b8ed1c468c447593bc649deb3d754c2fead3fe36ad603f329cddda +CRW_2158.crw 8786152 d668f4a2bb2944e9c7262dce12bd49ee4c3d8d435406710d84de6345160d931f +CRW_2159.crw 8489468 f40d62006fdad5d5a7e9b266b8e03868d08349f938099e6fb115cd4b655ae4b9 +CRW_2160.crw 8804740 ab6d023066fab7cceee8a82744910d7e71ab81c193ab7db56e9be4bd6195bb52 +CRW_2161.crw 8098538 c3cbf6e43e697de7dbfc3edd53d28fc14483a47dc9da0a4346b358c72eae469b +CRW_2162.crw 7164774 cef168d43c0902dbcfc87a90364f76825eed461904202f67f4e567162decb8b2 +CRW_2163.crw 7052370 5ce66c85b99275ffdfe770a853d94150f90f90525c8b560f7f64e7f90ee73d49 +CRW_2164.crw 5367212 1d42656fe8929127dfe25592ebe049095bc56530a05a6e7ddf6a41b53b538c55 +CRW_2165.crw 7223556 51458905360cd73620631920669277086dad50252945aeb1c429ea60e566d991 +CRW_2166.crw 8912458 feb808df17de3a3fb67116cbc6aa911e64f05fdcb2706a25086c205b6abdca99 +CRW_2167.crw 8506678 9a0fea250d24be1b24013e5e634213e0e44f77a12290d60686853ff7667143c0 +CRW_2168.crw 7896620 1029c586ef25416ea9c6c8d5c8f9410b1ecae7280e905491623ca4f0a80be8ac +CRW_2169.crw 7673094 f5bef91d1044c73389a2109736abb1f73d6988fa51be3c4a4f507bef4d31b699 +CRW_2170.crw 6470336 c1c7336f47d5a1c989cc920e329c9c9f5e22f42fb0eaca05b71e20c1361adcd8 +CRW_2171.crw 6593248 07dbefca3159f70f461d35332c527c103c8a06259a725fc7f64cc79f5608ac56 +CRW_2172.crw 6038866 22fb2cf5a3dbad4854c7163f4556eb1b0664e8a8e5b22cf02fb25a39096fff94 +CRW_2173.crw 5512954 e09768bb93acd68244d489788a85cc3da0610a7bbeed90eb56b9d4ca9a5683c3 +CRW_2174.crw 5622738 9ea99474c7e284c533a37f6f369cd18ccdd6340a74b125b71eda405f6c72fc1d +CRW_2175.crw 5482982 9a0f21ed290342108cbc8480959839bac4c45937c5d27cd20c58817735a0dfd6 +CRW_2176.crw 6310936 abcb5390c599c29b1a71a5c90a49a23346477e16fc10232a85d8c970a36cc531 +CRW_2177.crw 6402068 afbfcbeeaa3a1fb1ed7c37cca309ee420a5109c15be8f8d1ddd2b9ad7d859fca +CRW_2178.crw 6230750 25df8b70d6ed0844495d069bc945d3510a5205a4db7dae3c78b59f43b5c3fcf8 +CRW_2179.crw 6436774 64d7e2020a56573304262b3063b8f93b94b3907e6cfc96c3eb05ce218a5d35b4 +CRW_2180.crw 6355602 89910bb625a93723321eeee26c0a44b9d751727f51221c3d7ca723b4f589bad3 +CRW_2181.crw 6024694 f7d8b5ae73a9111c9316920a87331a6969b4923c282e9e266cdd5340bc2adce2 +CRW_2182.crw 6332160 461ee2925528dfc8df30a6a5601d816476b098df76b5090d61bb51884e880bce +CRW_2183.crw 6184886 065e2f3ba2da865f98433ebad2a30375f7a60c78f9a7f961e063b27f10f7f155 +CRW_2184.crw 6299166 3ada8ddfa8ea2607b58c701690df8a43b73deacd5858b8a26d8e2afc7a6b6571 +CRW_2185.crw 5672828 c8078522903d47510bf8b1e47174ba49823924780fe0cd3e7dd4d433c326ac7d +CRW_2186.crw 8103078 fc15736bd0589604d39e0478774e17fd42a76a3578da96ab3da70442201ebb79 +CRW_2187.crw 6593504 5c3a2b6a77b739566d41d38517f8b9d46ef937707220417fd74130b40b5bbf20 +CRW_2188.crw 8724306 957eed19b1d2475adb40c8f233f551c79eca8de312d333646db7d11b529ec81e +CRW_2189.crw 6568186 e258540ade215b40163100cfe97dffde8041b96c7a26dc5ed1a944896dd78711 +CRW_2190.crw 6576826 95459e98d9c17dcfb3578efc2ab565d84e67c537cc82d55416cefa449137ec27 +CRW_2191.crw 7289096 cde0c83be7ecd12aaa9fca0a36542f5f65be476c8cf0a793ed5ea2941ae07d2d +CRW_2192.crw 6049924 0a035b07fa123a9ae1f1f721b000539e44088eac11a37acfb0e8f1c411dbe544 +CRW_2193.crw 5842892 9e9623f6121fcb3d40dcd82762b7843cc84108c2b00cbd19c1378ccdcd288084 +CRW_2194.crw 5931190 e694dcc2ebc450f53dfd9648ca2121f27ba5e7b17e6560436f2e65b0c5519644 +CRW_2195.crw 8757614 8f9c035ebfe6370c645d4b801b573153ad11c6ac8c7110422264a341d0e25fd2 +CRW_2196.crw 9010402 b70e5d2c0a6a403d28c9749eb6ccbb3f386d20159da419240cf8d4d7946647c1 +CRW_2197.crw 9937442 d41b64096a426d8a7a0121cfa6cd5c064ad7beee395c816916b80cb6542aafb6 +CRW_2198.crw 10059408 56f744778dfb7a14f586b60f079b48be5f2929139f2dad3e103c417d3540927d +CRW_2199.crw 9951406 7eb85bd5e707d9e37dffc48b8d303648798cb4d8730591ad8ab5aa3a936e738a +CRW_2200.crw 6655704 fdf1df36af9357900c3f977a8e184cf33c4899dcd9c28bd568f198d2b32bbe5c +CRW_2201.crw 6880712 c8f7bc101a05cc33b5d71d3f681bc554e636a827e5e144e0bce75138a42ab25e +CRW_2202.crw 6937766 099279687cab933ab6a58cc0d176f6f445a064eeca26e28e438dffa417899401 +CRW_2203.crw 6982294 956e5ec1cee8c4eb9ded3da7d41bd4d4d1d6796fb801a8ad3c5df1246757bf74 +CRW_2204.crw 10732754 de441821015a40e9b4494787a48521c682a42399c3ecc4a35ab7e101c14bfd53 +CRW_2205.crw 10652456 fc022620b7f3aee0a5314360ef76f05939e81bacda764d8a27a4959a0af87e8d +CRW_2206.crw 10204708 4f98c2a73dfccd9073dab3151e90752a2e27422b78964c7b77a67f1eb492d5f0 +CRW_2207.crw 8653354 80169c74949715b6f04ecf3248fcc7255d06f6af9eefc46797fbc1d5cd6d3ea4 +CRW_2208.crw 8712038 eb5892520a46b577da9a9df9080f6b79102835e059517a4482b2cdd809d2bd3d +CRW_2209.crw 7037472 838d0d421a3de2243087b54462876c77ad19ad3b2be00aac96e1f3a3cf9fd0a1 +CRW_2210.crw 6891164 84e3461b20809aaf8ea1f6445b712a4408ad0b42e33cb0c1b8fa779568da91ab +CRW_2211.crw 8846556 05e2ab097e9571808d42231bce5345ba281f532a6b1bbad4fac5cce9235d935c +CRW_2212.crw 6878158 298386a8a65fe2c4d7e91321a66e507f452c4d14de86c19b8e9cc507c616a86c +CRW_2213.crw 6797582 4ef34f38920d0cc4d24dfd7f0b04a3b2c518ba641c3d1d388d48e80f6fadabb8 +CRW_2214.crw 7335138 7fac6a552775dcb925d25aa0383b2958f975ebc175c6f348351604d7edc4a992 +CRW_2215.crw 7151570 4766df542bc26910eae7f55aa68d895eb4e61c86548a5a5c7b9ef903d12b526b +CRW_2216.crw 6858956 36bddaab288f98e2d214ac5cd84a22b92e424401bcd888262639ae9a1ca4c7d0 +CRW_2217.crw 7847394 fa18afeca61484eb41e6dd34a2eb9b3dc118bfdf068bc1acf1fd8c2a50e91cb6 +CRW_2218.crw 7900778 2740a987ed886bcdc083f18c7bc525b32bc2a31aa9f13f12cd6f29bde6cb1bc8 +CRW_2219.crw 8628810 0460e9ba4884c7e0867570cf71be7a6d2d038fdcecf062fde5cc28422602dd07 +CRW_2220.crw 8410374 a5163ae9ca28606b5f5424cab26f157d80ed97cc95eb26093af0cf9b26a4f06d +CRW_2221.crw 6207486 848a99c6e36376d1d94840da4ac2db547bde0a3b046fabb0375f6c20ea6ada41 +CRW_2222.crw 5569354 98497ed06c0a106217ccbd204fc8c32cf24cee7949ae789ce582a44f93a60c82 +CRW_2223.crw 5956552 3634405512bc061b14fbc9ae9d9d63188093256697f3c64a010e15310bce709e +CRW_2224.crw 8043476 b514e159b6b08f2b897d27c0bc1ba2c4a316bc96db80c6b9f24a8dee4bf28934 +CRW_2225.crw 9234034 ce597642cc1b3c6858ff7b0de4600128cb722fecd136b22cd14a831e451a6f73 +CRW_2226.crw 7091230 ba4849b0617b44dda66bc42c36b9d14f2df7484f9d6e65ce193265cb11bb05bc +CRW_2227.crw 7363902 456de393ca67a834c1e799d1d360d88369916662e5ad621dc32f0fda4d7665b4 +CRW_2228.crw 8603694 1a8bac1e5bbd2b386cedd5a44a32922807c58f24c01d5e4fa81178bf421f87d7 +CRW_2229.crw 8202850 8cfe40ba7f91c227b03dba3b5975dab3ca66f32e87865b1e984fb2101eed0aaa +CRW_2269.crw 5453378 bd2f03e57c961ca1bc89095646653bf5de62eccde9f6413f090076a0cc2d1489 +CRW_2270.crw 5614686 367c627c90b38a832a09622b9194e5d348fe89bff39a3f8012c9ab365b5f59d8 +CRW_2271.crw 5781634 80d08ba4b7cc036e3ba093882761522b4f7d8656d9a6c7ba06d574e236df6e68 +CRW_2272.crw 5615902 46e7229a8c0d376911e9b8f8e0341b8290ec69d6e1a82e0ba75fad20d5f4c8f9 +CRW_2273.crw 5751364 def2ab24c60441560e5e637950727458bddacd8cf1c3d0758f4ed7ac3c543fa0 +CRW_2274.crw 7654816 7466692c3f665fe0e886e8efeb0a83e00c603b30bbbf28e3c9e35e535a0d6706 +CRW_2275.crw 6399990 835191e1669a9c0d766a65c7b92601390e5e73e75992a1f552c7831ec516f27d +CRW_2276.crw 6134670 04744ba5965e86ea2d1067ee7ec4134615972245409fc3b34d439155f358baf5 +CRW_2277.crw 6053428 f520e296ea281e6faf0e52869f08cd5448fbd29cca6f47a3c08130c57056ed99 +Cap&WhiteOut/CRW_2532.CRW 4434692 c9c8f97994fe3be81584364db9a91ed60f909e2a97846d13c9d07dde082ec5ae +Cap&WhiteOut/CRW_2534.CRW 5722560 db2a90ccb3914de1c577c200c6c58eafd5067b1e17035430a5f1b456930c0bb2 +Cap&WhiteOut/CRW_2538.CRW 5828220 8278818fa9226000b4ec2aff314b65cebf29f04ef4551461a5ab120d8a0b76eb +Cap&WhiteOut/CRW_2539.CRW 1959714 6232061988044f21390ef1c9e63f1c95695238d0871bb5e450bdce02194e7cfc +GrayCard/CRW_2547.CRW 5799898 ef949dfc06afc91d9340fbd8352be55980afb6aee239f513e8bdff18568b5b43 +GrayCard/CRW_2548.CRW 5983342 940e86bc2385420b3fdef9881402acacdf8b5c953110617105fd6343c244230c +GrayCard/CRW_2549.CRW 6092382 a3da86a7e5ca2927df7d69119613187fcb70528944671607061f85dee6a2ece9 +GrayCard/CRW_2550.CRW 5718904 1ba8bb1e4e505e056980eaf6000db7e7a4a0a728c5b6cda586ed537356fd70b7 +GrayCard/CRW_2551.CRW 5587354 2a3618a0e2e8ac4459a57ec19e563fdda6b9d018a44b71702af1762324d61741 +GrayCard/CRW_2552.CRW 5621344 8b2a7b816a89212b32be4deab0546abc2601e464b7ccabf35b69ef200761aaa8 +GrayCard/CRW_2553.CRW 5759506 9938384d0e77ec6f713e7e9742b2ae3ea0bff0b4e1e3cfc708c1faaeb03d031e +GrayCard/CRW_2554.CRW 5870548 b1acf0dc7b234fbfcf8b6da2518b926b23e2cbc955717f4eb4b7f2064b612784 +GrayCard/CRW_2555.CRW 5537740 1c29b06b187f75cdb0dcd9fbf600fee0737be92f95d3081b1110c389c07a4b58 +GrayCard/CRW_2556.CRW 5441154 831543da6635bd13bf0297e6a611c2cca8c9b46cf81e68132aa5937add6300bc +GrayCard/CRW_2557.CRW 5612614 383c814c6c492647dc232c4d7611d3b0cd535cd0c57b9399f9052afc37c3dc05 +GrayCard/CRW_2558.CRW 5731810 24e6f26e23bed72bc419e789272b12317cf4baecc506566ca934805e53a0e300 +GrayCard/CRW_2559.CRW 5813986 93239cf334b26a52bee7b847bfb2f0b8ad03c2343f8eec5319fe82504c1ee9d7 +GrayCard/CRW_2560.CRW 5479622 1c6acf4ccae3e815f5955aa12354de604e59cb2906c91ca01909f2c8c0d9fd75 +GrayCard/CRW_2561.CRW 5428058 8affb7e1604a30e4a6ff31ac58cae2cea2582f4844d15d6b09e30ec8f8ee27b2 +GrayCard/CRW_2562.CRW 5615028 1be84b496491ee62246cdeaad2a6ca09a85c48eca13975888a441ebe0ee49eae +GrayCard/CRW_2563.CRW 5722106 97cfc65d530025d47334e207b0bc9e85b56bef4791f22c99b8d5067345569d74 +GrayCard/CRW_2564.CRW 5802660 7e18eedf945c798b74d8d5b7de2b0ce6c3dc222f931b77b0fa0098ca3f550bef +GrayCard/CRW_2565.CRW 5501692 84a336b568ec13f2978ce80c60340b2949dd05ca6a0ff13b03fa6022cb05a42c +GrayCard/CRW_2566.CRW 5404006 a69ab53b635565449243b6837b6e56bea26405ea7f6830b026c8d74e48d0f32f +WBModes/CRW_2540.CRW 7589400 9749143941406b451ec9b114c119da2f3335d03ffa0c7da24d01169a8a221cb3 +WBModes/CRW_2541.CRW 7812450 d5f517d45db1f3b8cfc70c478520492cddf929952c2a1ff77cfce4108bef3b08 +WBModes/CRW_2542.CRW 7427966 0c08805f0c39e837502a2804f2f7ef6da685b471881e4542a8ab1c65cc5952ed +WBModes/CRW_2543.CRW 7495942 695395e961c2c2927d8e7b7f2d9b240d83db4ad7cb30d44083dfb1ea74ad1082 +WBModes/CRW_2544.CRW 7393658 8c160fa09095854a3ef3bff31deb18f232cd14120a3982ef073cc4c734ab3616 +WBModes/CRW_2545.CRW 7382622 8f0c5489b6b4338eb92253605d40c11ce07e96102f5655c685fa290ccb1c95eb +WBModes/CRW_2546.CRW 7584734 070f2740c474cbd515b5a34a1a57fe2b502d114953c8b211ef7207e2497e2e3b +TripodGrayCard/CRW_2567.CRW 5994216 5581d68b7b77c1edb66cbf511a15267d3ba42ca160dd34c38964e982b3bba3f3 +TripodGrayCard/CRW_2568.CRW 5991366 d73fc7b364ac8d822a38725b1d59be10d3c74df6d33b150ccaeebafac34632a7 +TripodGrayCard/CRW_2569.CRW 5958374 48090cd06e3b2de0e069d482b8d16ac541e0948d5ad0c2a2024149cec213ebf5 +TripodGrayCard/CRW_2570.CRW 5900012 48ba16da5eac5509e323a55e5b464f9ca5d84b42c263e1d4dc01570696acd786 +TripodGrayCard/CRW_2571.CRW 5860480 ad1be55479a5dde9f321859a3ab131bbc54ecf8e32cc7b53a3f10835786abbde +TripodGrayCard/CRW_2572.CRW 5812496 9b0da84c0de9aaab7ec65dcc72560e89c200d99e9f59ba35c4d85f204cdf7ca9 +TripodGrayCard/CRW_2573.CRW 5812280 6bd50ad7e330a988bd1f00bbb4dd0699598a62cded8e65d4dc9b9f8dd49a413f +TripodGrayCard/CRW_2574.CRW 5796216 619fcbaa43c0aa8d8dec2e001b74b8c6c3a782a56f2bb2e34c03253c82f54499 +TripodGrayCard/CRW_2575.CRW 5804682 86fd0e96f3cd72551fe2e55188c311ae071a07eae662fb60e4ac130b1096f0c4 +TripodGrayCard/CRW_2576.CRW 5761378 a034e33efaab5a39e12771bb397ab6e0d14f5ff444d158d6f60d703cf5f1d399 +TripodGrayCard/CRW_2577.CRW 5809340 b1595ff813cf227d263f9ee6318f66112f25c10924e55328a8fb18be5157d3fe +TripodGrayCard/CRW_2578.CRW 5709476 b020d1a84ec121aea16319452e5aba7ab9e6c048c059c107898a1d28fce6ecf0 +TripodGrayCard/CRW_2579.CRW 5702676 6350f0e97b039247a1c9e9529bc7128e970b8b7570d7b1360a1ffd3239e14599 +TripodGrayCard/CRW_2580.CRW 5610414 f201fac8c69db558914010d0afd43e9b7976a1411147433f1483929678d29955 diff --git a/crates/crw/clean-room/logs/structure-d60-public.txt b/crates/crw/clean-room/logs/structure-d60-public.txt new file mode 100644 index 00000000..28337ee1 --- /dev/null +++ b/crates/crw/clean-room/logs/structure-d60-public.txt @@ -0,0 +1,73 @@ +== corpus/crw/crw-canon-eos-d60.crw + size 8591234, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 +heap root: block @0x1a len 0x831768, offsetTbl @+0x831744, 3 records + [0] tag 0x2005 id 0x2005 RawData struct heap len 0x662502 off 0x0 + RawData: 6694146 bytes, entropy 7.9868 bits/byte + first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 + middle 2 KiB: entropy 7.864, top bytes 8c:25, cb:21, 5d:18, 63:17, zeros 11, ff 4 + last 2 KiB: entropy 7.890, top bytes f1:16, 6d:16, 76:16, 55:15, zeros 11, ff 5 + [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x1cdc00 off 0x662502 + starts with JPEG SOI + [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x1642 off 0x830102 + heap root/ImageProps: block @0x83011c len 0x1642, offsetTbl @+0x15ba, 13 records + [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 + [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc + = u32 (3072, 2048, 1065353216, 0, 8, 24, 257) + [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x6d8f0000 off 0xcfba0d73 + [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xd8327aab + [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0xf42a3 off 0x9792ef1a + [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 + = u32 (1467566074, 0, 0) + [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 + = b'CRW_0099.CRW' + [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 + = b'CRW_0099.THM' + [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 + [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 + heap root/ImageProps/MeasuredInfo: block @0x830284 len 0x10, offsetTbl @+0x0, 1 records + [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x3fc00000 off 0xe2577c1e + [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 + heap root/ImageProps/ImageDescription: block @0x830294 len 0x5a, offsetTbl @+0x40, 2 records + [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 + = b'EOS D60 CMOS RAW IMAGE' + [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 + = b'CRW:EOS D60 CMOS RAW IMAGE' + [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 + heap root/ImageProps/CameraObject: block @0x8302ee len 0xb2, offsetTbl @+0x8e, 3 records + [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 + = b'' + [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 + = b'Canon\x00Canon EOS D60\x00\x0e{\xd7\xd8e\x18iG\x03,,c' + [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 + heap root/ImageProps/CameraObject/CameraSpecification: block @0x83032e len 0x4e, offsetTbl @+0x20, 4 records + [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x3645276f off 0x5982da24 + [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xa552febb + [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0xfedcfe5 + [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 + = b'Firmware Version 1.0.4' + [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x1336 off 0x284 + heap root/ImageProps/ExifInformation: block @0x8303a0 len 0x1336, offsetTbl @+0x129a, 15 records + [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 + [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x0 off 0x2620392 + [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x36 off 0x0 + = u16 (54, 0, 160, 52, 32768, 245, 65504, 0, 3, 0, 8, 8, 0, 0, 12288, 0, 0, 0, 1, 0, 0, 32768, 244, 99, 0, 0, 252) + [3] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x56 off 0x36 + [4] tag 0x0036 id 0x0036 ? byte heap len 0xbd6 off 0x8c + [5] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x1668000 off 0xed8d4a16 + [6] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xc62 + = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) + [7] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xc84 + = u32 (0, 0, 514, 5064559) + [8] tag 0x1033 id 0x1033 CustomFunctions word heap len 0x22 off 0xc94 + = u16 (32, 0, 515, 768, 1025, 1280, 1536, 1792, 2048, 2304, 2560, 2818, 3074, 3328, 3584, 3841, 18747) + [9] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xcb6 + = u16 (26, 279, 368, 371, 172, 219, 434, 436, 284, 162, 491, 494, 431) + [10] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xcd0 + = u16 (26, 281, 413, 415, 193, 223, 488, 490, 319, 150, 505, 507, 439) + [11] tag 0x10c2 id 0x10c2 ? word heap len 0x554 off 0xcea + [12] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x123e + = u16 (10, 440, 1021, 1027, 563) + [13] tag 0x10a8 id 0x10a8 ? word heap len 0x10 off 0x1248 + = u16 (16, 6000, 5312, 5989, 3161, 3717, 6503, 4118) + [14] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x42 off 0x1258 + diff --git a/crates/crw/clean-room/logs/structure-public.txt b/crates/crw/clean-room/logs/structure-public.txt new file mode 100644 index 00000000..e5674341 --- /dev/null +++ b/crates/crw/clean-room/logs/structure-public.txt @@ -0,0 +1,887 @@ +== corpus/crw/crw-canon-eos-10d-a.crw + size 7787330, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 +heap root: block @0x1a len 0x76d328, offsetTbl @+0x76d304, 3 records + [0] tag 0x2005 id 0x2005 RawData struct heap len 0x6cdcf6 off 0x0 + RawData: 7134454 bytes, entropy 7.9776 bits/byte + first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 + middle 2 KiB: entropy 7.877, top bytes db:20, d3:16, b6:16, de:16, zeros 6, ff 3 + last 2 KiB: entropy 7.882, top bytes ac:17, d4:17, 91:17, 6c:17, zeros 12, ff 4 + [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x9dde2 off 0x6cdcf6 + starts with JPEG SOI + [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x182c off 0x76bad8 + heap root/ImageProps: block @0x76baf2 len 0x182c, offsetTbl @+0x17a4, 13 records + [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 + [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc + = u32 (3072, 2048, 1065353216, 0, 8, 24, 257) + [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x3af0000 off 0x9b869aeb + [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xf5657a8 + [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x1af8f9 off 0xde23c13d + [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 + = u32 (1228574022, 0, 0) + [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 + = b'CRW_7673.CRW' + [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 + = b'CRW_7673.THM' + [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 + [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 + heap root/ImageProps/MeasuredInfo: block @0x76bc5a len 0x10, offsetTbl @+0x0, 1 records + [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x40cc0000 off 0x70138c52 + [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 + heap root/ImageProps/ImageDescription: block @0x76bc6a len 0x5a, offsetTbl @+0x40, 2 records + [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 + = b'EOS 10D CMOS RAW IMAGE' + [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 + = b'CRW:EOS 10D CMOS RAW IMAGE' + [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 + heap root/ImageProps/CameraObject: block @0x76bcc4 len 0xb2, offsetTbl @+0x8e, 3 records + [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 + = b'Horst Kakuschke' + [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 + = b'Canon\x00Canon EOS 10D\x00:g\x8fZ\xb5k\xa7\xea\xca\xda|\x1a' + [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 + heap root/ImageProps/CameraObject/CameraSpecification: block @0x76bd04 len 0x4e, offsetTbl @+0x20, 4 records + [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0xdb7145a off 0xde776b76 + [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0x8762cef2 + [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0xe63b7d8c + [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 + = b'Firmware Version 2.0.1' + [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x1520 off 0x284 + heap root/ImageProps/ExifInformation: block @0x76bd76 len 0x1520, offsetTbl @+0x142a, 24 records + [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 + [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x180000 off 0x2620392 + [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x42 off 0x0 + [3] tag 0x1093 id 0x1093 ? word heap len 0x12 off 0x42 + = u16 (18, 0, 0, 0, 0, 0, 65535, 65535, 65535) + [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x54 + [5] tag 0x0036 id 0x0036 ? byte heap len 0xe82 off 0xb0 + [6] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x80000168 off 0x4f2612e3 + [7] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xf32 + = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) + [8] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xf54 + = u32 (0, 0, 514, 5504868) + [9] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x30 off 0xf64 + = u16 (7, 7, 3072, 2048, 3072, 2048, 151, 151, 1014, 608, 0, 0, 0, 64928, 64522, 0, 0, 65030, 0, 506, 0, 0, 8, 65535) + [10] tag 0x1033 id 0x1033 CustomFunctions word heap len 0x24 off 0xf94 + = u16 (36, 256, 513, 768, 1024, 1280, 1536, 1792, 2048, 2304, 2560, 2818, 3072, 3328, 3584, 3841, 4097, 4352) + [11] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xfb8 + = u16 (26, 288, 343, 343, 160, 231, 433, 433, 275, 154, 444, 444, 384) + [12] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xfd2 + = u16 (26, 314, 364, 365, 165, 234, 428, 430, 262, 174, 484, 487, 412) + [13] tag 0x10c2 id 0x10c2 ? word heap len 0x374 off 0xfec + [14] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x1360 + = u16 (10, 418, 1021, 1027, 467) + [15] tag 0x10a8 id 0x10a8 ? word heap len 0x14 off 0x136a + = u16 (20, 4174, 5190, 7000, 6000, 3232, 3934, 6083, 5190, 5200) + [16] tag 0x10ad id 0x10ad ? word heap len 0x3e off 0x137e + = u16 (62, 21468, 8650, 7000, 6000, 5600, 5200, 4700, 4200, 3343, 1668, 64833, 65115, 65180, 65240, 65271, 65307, 65361, 65429, 64, 582, 65341, 65183, 65148, 65121, 65106, 65097, 65083, 65072, 65051, 65251) + [17] tag 0x10ae id 0x10ae ColorTemperature word heap len 0x2 off 0x140e + = u16 (5200,) + [18] tag 0x10af id 0x10af ? word heap len 0x2 off 0x1410 + = u16 (1024,) + [19] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x52 off 0x13bc + [20] tag 0x10b4 id 0x10b4 ColorSpace word heap len 0x2 off 0x1412 + = u16 (1,) + [21] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x1414 + = u16 (10, 2, 2, 1536, 1024) + [22] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x141e + = u16 (0, 159, 7, 112) + [23] tag 0x183b id 0x183b SerialNumberFormat dword heap len 0x4 off 0x1426 + = u32 (2415919104,) + +== corpus/crw/crw-canon-eos-10d-b.crw + size 10822040, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 +heap root: block @0x1a len 0xa5217e, offsetTbl @+0xa5215a, 3 records + [0] tag 0x2005 id 0x2005 RawData struct heap len 0x6e8c3a off 0x0 + RawData: 7244858 bytes, entropy 7.9718 bits/byte + first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 + middle 2 KiB: entropy 7.894, top bytes 8c:18, 23:17, 5b:16, 32:16, zeros 14, ff 2 + last 2 KiB: entropy 7.824, top bytes 52:20, 8c:19, 84:19, 92:18, zeros 17, ff 2 + [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x367cf4 off 0x6e8c3a + starts with JPEG SOI + [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x182c off 0xa5092e + heap root/ImageProps: block @0xa50948 len 0x182c, offsetTbl @+0x17a4, 13 records + [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 + [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc + = u32 (3072, 2048, 1065353216, 0, 8, 24, 257) + [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x0 off 0x1 + [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0x80002400 + [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x127edf off 0x1001000 + [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 + = u32 (1105175848, 0, 0) + [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 + = b'CRW_2127.CRW' + [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 + = b'CRW_2127.THM' + [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 + [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 + heap root/ImageProps/MeasuredInfo: block @0xa50ab0 len 0x10, offsetTbl @+0x0, 1 records + [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x40a80000 off 0x0 + [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 + heap root/ImageProps/ImageDescription: block @0xa50ac0 len 0x5a, offsetTbl @+0x40, 2 records + [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 + = b'EOS 10D CMOS RAW IMAGE' + [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 + = b'CRW:EOS 10D CMOS RAW IMAGE' + [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 + heap root/ImageProps/CameraObject: block @0xa50b1a len 0xb2, offsetTbl @+0x8e, 3 records + [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 + = b'Bill Ferguson' + [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 + = b'Canon\x00Canon EOS 10D\x00\x00\x00\x00\x06\x00\x04 \x80\x01\x04' + [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 + heap root/ImageProps/CameraObject/CameraSpecification: block @0xa50b5a len 0x4e, offsetTbl @+0x20, 4 records + [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x54ab324a off 0x200100 + [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0x1042 + [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0x1000000 + [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 + = b'Firmware Version 2.0.1' + [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x1520 off 0x284 + heap root/ImageProps/ExifInformation: block @0xa50bcc len 0x1520, offsetTbl @+0x142a, 24 records + [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 + [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x550000 off 0x2620392 + [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x42 off 0x0 + [3] tag 0x1093 id 0x1093 ? word heap len 0x12 off 0x42 + = u16 (18, 0, 0, 0, 0, 0, 65535, 65535, 65535) + [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x54 + [5] tag 0x0036 id 0x0036 ? byte heap len 0xe82 off 0xb0 + [6] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x80000168 off 0x20 + [7] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xf32 + = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) + [8] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xf54 + = u32 (2, 0, 514, 5615272) + [9] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x30 off 0xf64 + = u16 (7, 7, 3072, 2048, 3072, 2048, 151, 151, 1014, 608, 0, 0, 0, 64928, 64522, 0, 0, 65030, 0, 506, 0, 0, 31, 65535) + [10] tag 0x1033 id 0x1033 CustomFunctions word heap len 0x24 off 0xf94 + = u16 (36, 256, 513, 768, 1024, 1280, 1537, 1792, 2053, 2304, 2560, 2816, 3072, 3328, 3584, 3840, 4096, 4352) + [11] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xfb8 + = u16 (26, 353, 375, 375, 177, 268, 431, 432, 285, 190, 476, 477, 428) + [12] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xfd2 + = u16 (26, 326, 364, 365, 161, 254, 428, 430, 268, 186, 484, 487, 416) + [13] tag 0x10c2 id 0x10c2 ? word heap len 0x374 off 0xfec + [14] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x1360 + = u16 (10, 345, 1021, 1027, 1301) + [15] tag 0x10a8 id 0x10a8 ? word heap len 0x14 off 0x136a + = u16 (20, 4028, 5190, 7000, 6000, 3232, 3930, 6080, 2818, 5200) + [16] tag 0x10ad id 0x10ad ? word heap len 0x3e off 0x137e + = u16 (62, 21468, 8650, 7000, 6000, 5600, 5200, 4700, 4200, 3343, 1668, 64821, 65116, 65184, 65246, 65279, 65317, 65373, 65444, 83, 613, 65393, 65229, 65191, 65165, 65148, 65139, 65125, 65112, 65100, 65341) + [17] tag 0x10ae id 0x10ae ColorTemperature word heap len 0x2 off 0x140e + = u16 (5200,) + [18] tag 0x10af id 0x10af ? word heap len 0x2 off 0x1410 + = u16 (1024,) + [19] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x52 off 0x13bc + [20] tag 0x10b4 id 0x10b4 ColorSpace word heap len 0x2 off 0x1412 + = u16 (1,) + [21] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x1414 + = u16 (10, 3, 0, 3072, 2048) + [22] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x141e + = u16 (0, 159, 7, 112) + [23] tag 0x183b id 0x183b SerialNumberFormat dword heap len 0x4 off 0x1426 + = u32 (2415919104,) + +== corpus/crw/crw-canon-eos-10d-c.crw + size 10583702, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 +heap root: block @0x1a len 0xa17e7c, offsetTbl @+0xa17e58, 3 records + [0] tag 0x2005 id 0x2005 RawData struct heap len 0x71c324 off 0x0 + RawData: 7455524 bytes, entropy 7.9666 bits/byte + first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 + middle 2 KiB: entropy 7.847, top bytes 82:21, b2:18, 98:18, 84:18, zeros 15, ff 1 + last 2 KiB: entropy 7.785, top bytes 08:23, 9c:20, 38:20, cc:20, zeros 7, ff 0 + [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x2fa308 off 0x71c324 + starts with JPEG SOI + [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x182c off 0xa1662c + heap root/ImageProps: block @0xa16646 len 0x182c, offsetTbl @+0x17a4, 13 records + [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 + [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc + = u32 (3072, 2048, 1065353216, 0, 8, 24, 257) + [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x41020000 off 0xf2c31662 + [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xc3af7160 + [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x12f557 off 0x777d1c94 + [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 + = u32 (1109761444, 0, 0) + [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 + = b'CRW_2455.CRW' + [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 + = b'CRW_2455.THM' + [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 + [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 + heap root/ImageProps/MeasuredInfo: block @0xa167ae len 0x10, offsetTbl @+0x0, 1 records + [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x41160000 off 0x9a0c5382 + [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 + heap root/ImageProps/ImageDescription: block @0xa167be len 0x5a, offsetTbl @+0x40, 2 records + [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 + = b'EOS 10D CMOS RAW IMAGE' + [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 + = b'CRW:EOS 10D CMOS RAW IMAGE' + [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 + heap root/ImageProps/CameraObject: block @0xa16818 len 0xb2, offsetTbl @+0x8e, 3 records + [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 + = b'Bill Ferguson' + [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 + = b'Canon\x00Canon EOS 10D\x00\xfe\x19g\xba1\x94E\x02\x1cc(\xca' + [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 + heap root/ImageProps/CameraObject/CameraSpecification: block @0xa16858 len 0x4e, offsetTbl @+0x20, 4 records + [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x54ab324a off 0x18e17812 + [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xd734ef03 + [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0x70fc1863 + [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 + = b'Firmware Version 2.0.1' + [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x1520 off 0x284 + heap root/ImageProps/ExifInformation: block @0xa168ca len 0x1520, offsetTbl @+0x142a, 24 records + [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 + [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x180000 off 0x2620392 + [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x42 off 0x0 + [3] tag 0x1093 id 0x1093 ? word heap len 0x12 off 0x42 + = u16 (18, 0, 0, 0, 0, 0, 65535, 65535, 65535) + [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x54 + [5] tag 0x0036 id 0x0036 ? byte heap len 0xe82 off 0xb0 + [6] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x80000168 off 0x1ba7180e + [7] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xf32 + = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) + [8] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xf54 + = u32 (2, 0, 514, 5825938) + [9] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x30 off 0xf64 + = u16 (7, 7, 3072, 2048, 3072, 2048, 151, 151, 1014, 608, 0, 0, 0, 64928, 64522, 0, 0, 65030, 0, 506, 0, 0, 113, 65535) + [10] tag 0x1033 id 0x1033 CustomFunctions word heap len 0x24 off 0xf94 + = u16 (36, 256, 513, 768, 1024, 1280, 1537, 1792, 2053, 2304, 2560, 2816, 3072, 3328, 3584, 3840, 4096, 4352) + [11] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xfb8 + = u16 (26, 353, 375, 375, 177, 268, 431, 432, 285, 190, 476, 477, 428) + [12] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xfd2 + = u16 (26, 326, 364, 365, 161, 254, 428, 430, 268, 186, 484, 487, 416) + [13] tag 0x10c2 id 0x10c2 ? word heap len 0x374 off 0xfec + [14] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x1360 + = u16 (10, 508, 1021, 1027, 856) + [15] tag 0x10a8 id 0x10a8 ? word heap len 0x14 off 0x136a + = u16 (20, 5347, 5190, 7000, 6000, 3232, 3930, 6080, 2818, 5200) + [16] tag 0x10ad id 0x10ad ? word heap len 0x3e off 0x137e + = u16 (62, 21468, 8650, 7000, 6000, 5600, 5200, 4700, 4200, 3343, 1668, 64821, 65116, 65184, 65246, 65279, 65317, 65373, 65444, 83, 613, 65393, 65229, 65191, 65165, 65148, 65139, 65125, 65112, 65100, 65341) + [17] tag 0x10ae id 0x10ae ColorTemperature word heap len 0x2 off 0x140e + = u16 (5200,) + [18] tag 0x10af id 0x10af ? word heap len 0x2 off 0x1410 + = u16 (1024,) + [19] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x52 off 0x13bc + [20] tag 0x10b4 id 0x10b4 ColorSpace word heap len 0x2 off 0x1412 + = u16 (1,) + [21] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x1414 + = u16 (10, 3, 0, 3072, 2048) + [22] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x141e + = u16 (0, 159, 7, 112) + [23] tag 0x183b id 0x183b SerialNumberFormat dword heap len 0x4 off 0x1426 + = u32 (2415919104,) + +== corpus/crw/crw-canon-eos-300d-a.crw + size 9056348, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 +heap root: block @0x1a len 0x8a3042, offsetTbl @+0x8a301e, 3 records + [0] tag 0x2005 id 0x2005 RawData struct heap len 0x69bce6 off 0x0 + RawData: 6929638 bytes, entropy 7.9804 bits/byte + first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 + middle 2 KiB: entropy 7.886, top bytes bb:21, 9c:20, 8c:17, 18:14, zeros 13, ff 10 + last 2 KiB: entropy 7.850, top bytes ae:20, bb:18, e3:18, 6e:18, zeros 7, ff 4 + [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x205b64 off 0x69bce6 + starts with JPEG SOI + [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x17d4 off 0x8a184a + heap root/ImageProps: block @0x8a1864 len 0x17d4, offsetTbl @+0x174c, 13 records + [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 + [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0x0 + = u32 (3072, 2048, 1065353216, 270, 8, 24, 257) + [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x9ceb0000 off 0xc97abdef + [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xb1dab4b5 + [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0xfb8bd off 0x0 + [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x1c + = u32 (1083408238, 0, 0) + [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 + = b'CRW_0333.CRW' + [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 + = b'CRW_0333.THM' + [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 + [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 + heap root/ImageProps/MeasuredInfo: block @0x8a19cc len 0x10, offsetTbl @+0x0, 1 records + [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x41100000 off 0xf5e6df2f + [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x58 off 0x178 + heap root/ImageProps/ImageDescription: block @0x8a19dc len 0x58, offsetTbl @+0x3e, 2 records + [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 + = b'EOS 300D DIGITAL CMOS RAW' + [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x1e off 0x20 + = b'CRW:EOS 300D DIGITAL CMOS RAW' + [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0x8a off 0x1d0 + heap root/ImageProps/CameraObject: block @0x8a1a34 len 0x8a, offsetTbl @+0x66, 3 records + [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x1 off 0x0 + = b'' + [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x1d off 0x2 + = b'Canon\x00Canon EOS 300D DIGITAL' + [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x46 off 0x20 + heap root/ImageProps/CameraObject/CameraSpecification: block @0x8a1a54 len 0x46, offsetTbl @+0x18, 4 records + [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x3d6b01d9 off 0x8afd725d + [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xe53a2db9 + [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0x6efb17d7 + [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x17 off 0x0 + = b'Firmware Version 1.1.1' + [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x14f2 off 0x25a + heap root/ImageProps/ExifInformation: block @0x8a1abe len 0x14f2, offsetTbl @+0x1406, 23 records + [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 + [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x460000 off 0x2620392 + [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x42 off 0x0 + [3] tag 0x1093 id 0x1093 ? word heap len 0x12 off 0x42 + = u16 (18, 0, 0, 0, 0, 0, 65535, 65535, 65535) + [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x54 + [5] tag 0x0036 id 0x0036 ? byte heap len 0xe82 off 0xb0 + [6] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x80000170 off 0x79d599bf + [7] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xf32 + = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) + [8] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xf54 + = u32 (0, 0, 514, 5300052) + [9] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x30 off 0xf64 + = u16 (48, 7, 3072, 2048, 3072, 2048, 151, 151, 1014, 608, 0, 0, 0, 64928, 64522, 0, 0, 65030, 0, 506, 0, 0, 8, 65535) + [10] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xf94 + = u16 (26, 334, 372, 372, 175, 246, 427, 427, 279, 185, 488, 490, 443) + [11] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xfae + = u16 (26, 297, 375, 375, 168, 203, 394, 395, 244, 153, 453, 454, 388) + [12] tag 0x10c2 id 0x10c2 ? word heap len 0x374 off 0xfc8 + [13] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x133c + = u16 (10, 489, 1022, 1026, 668) + [14] tag 0x10a8 id 0x10a8 ? word heap len 0x14 off 0x1346 + = u16 (20, 4709, 5200, 6983, 5987, 3213, 3888, 6196, 2500, 5190) + [15] tag 0x10ad id 0x10ad ? word heap len 0x3e off 0x135a + = u16 (62, 20127, 8480, 7000, 6000, 5600, 5200, 4724, 4200, 3266, 1446, 64813, 65123, 65187, 65248, 65281, 65320, 65371, 65446, 114, 674, 65436, 65225, 65196, 65169, 65152, 65137, 65124, 65112, 65098, 65348) + [16] tag 0x10ae id 0x10ae ColorTemperature word heap len 0x2 off 0x1398 + = u16 (5200,) + [17] tag 0x10af id 0x10af ? word heap len 0x2 off 0x139a + = u16 (1024,) + [18] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x52 off 0x139c + [19] tag 0x10b4 id 0x10b4 ColorSpace word heap len 0x2 off 0x13ee + = u16 (1,) + [20] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x13f0 + = u16 (10, 3, 1, 2048, 1360) + [21] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x13fa + = u16 (0, 159, 7, 112) + [22] tag 0x183b id 0x183b SerialNumberFormat dword heap len 0x4 off 0x1402 + = u32 (2415919104,) + +== corpus/crw/crw-canon-eos-300d-b.crw + size 8389842, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 +heap root: block @0x1a len 0x8004b8, offsetTbl @+0x800494, 3 records + [0] tag 0x2005 id 0x2005 RawData struct heap len 0x6df56a off 0x0 + RawData: 7206250 bytes, entropy 7.9810 bits/byte + first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 + middle 2 KiB: entropy 7.903, top bytes d7:20, 7b:18, 32:17, 83:16, zeros 14, ff 6 + last 2 KiB: entropy 7.840, top bytes 00:29, ff:23, fc:21, af:20, zeros 29, ff 23 + [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x11f72c off 0x6df56a + starts with JPEG SOI + [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x17fe off 0x7fec96 + heap root/ImageProps: block @0x7fecb0 len 0x17fe, offsetTbl @+0x1776, 13 records + [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 + [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc + = u32 (3072, 2048, 1065353216, 270, 8, 24, 257) + [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x17890000 off 0x637f057b + [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0x9cae3253 + [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x2b4f7b off 0x7aaebd64 + [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 + = u32 (1182188238, 0, 0) + [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 + = b'CRW_8395.CRW' + [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 + = b'CRW_8395.THM' + [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 + [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 + heap root/ImageProps/MeasuredInfo: block @0x7fee18 len 0x10, offsetTbl @+0x0, 1 records + [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x41040000 off 0xde864d39 + [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 + heap root/ImageProps/ImageDescription: block @0x7fee28 len 0x5a, offsetTbl @+0x40, 2 records + [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 + = b'EOS 300D DIGITAL CMOS RAW' + [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 + = b'CRW:EOS 300D DIGITAL CMOS RAW' + [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 + heap root/ImageProps/CameraObject: block @0x7fee82 len 0xb2, offsetTbl @+0x8e, 3 records + [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 + = b'' + [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 + = b'Canon\x00Canon EOS 300D DIGITAL\x00\x97\xd4\xc4' + [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 + heap root/ImageProps/CameraObject/CameraSpecification: block @0x7feec2 len 0x4e, offsetTbl @+0x20, 4 records + [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x3d6aecf7 off 0x46ec30fc + [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0x5aa35384 + [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0xf1421364 + [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 + = b'Firmware Version 1.1.1' + [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x14f2 off 0x284 + heap root/ImageProps/ExifInformation: block @0x7fef34 len 0x14f2, offsetTbl @+0x1406, 23 records + [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 + [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x150000 off 0x2620392 + [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x42 off 0x0 + [3] tag 0x1093 id 0x1093 ? word heap len 0x12 off 0x42 + = u16 (18, 0, 0, 0, 0, 0, 65535, 65535, 65535) + [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x54 + [5] tag 0x0036 id 0x0036 ? byte heap len 0xe82 off 0xb0 + [6] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x80000170 off 0xedbad5b1 + [7] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xf32 + = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) + [8] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xf54 + = u32 (2, 0, 514, 5576664) + [9] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x30 off 0xf64 + = u16 (7, 7, 3072, 2048, 3072, 2048, 151, 151, 1014, 608, 0, 0, 0, 64928, 64522, 0, 0, 65030, 0, 506, 0, 0, 36, 65535) + [10] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xf94 + = u16 (26, 328, 374, 372, 163, 244, 430, 428, 262, 162, 436, 435, 366) + [11] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xfae + = u16 (26, 301, 375, 375, 166, 206, 394, 395, 242, 155, 453, 454, 383) + [12] tag 0x10c2 id 0x10c2 ? word heap len 0x374 off 0xfc8 + [13] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x133c + = u16 (10, 314, 1022, 1026, 1095) + [14] tag 0x10a8 id 0x10a8 ? word heap len 0x14 off 0x1346 + = u16 (20, 5781, 5200, 7000, 5975, 3211, 3882, 6203, 4166, 5190) + [15] tag 0x10ad id 0x10ad ? word heap len 0x3e off 0x135a + = u16 (62, 20127, 8480, 7000, 6000, 5600, 5200, 4724, 4200, 3266, 1446, 64849, 65150, 65212, 65271, 65304, 65342, 65392, 65464, 125, 664, 65367, 65170, 65143, 65117, 65102, 65088, 65078, 65068, 65055, 65302) + [16] tag 0x10ae id 0x10ae ColorTemperature word heap len 0x2 off 0x13ea + = u16 (5200,) + [17] tag 0x10af id 0x10af ? word heap len 0x2 off 0x13ec + = u16 (1024,) + [18] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x52 off 0x1398 + [19] tag 0x10b4 id 0x10b4 ColorSpace word heap len 0x2 off 0x13ee + = u16 (2,) + [20] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x13f0 + = u16 (10, 3, 2, 1536, 1024) + [21] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x13fa + = u16 (0, 159, 7, 112) + [22] tag 0x183b id 0x183b SerialNumberFormat dword heap len 0x4 off 0x1402 + = u32 (2415919104,) + +== corpus/crw/crw-canon-eos-d30.crw + size 2993260, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 +heap root: block @0x1a len 0x2dac52, offsetTbl @+0x2dac2e, 3 records + [0] tag 0x2005 id 0x2005 RawData struct heap len 0x2b19c8 off 0x0 + RawData: 2824648 bytes, entropy 7.9671 bits/byte + first 2 KiB: entropy 7.540, top bytes ef:35, fb:34, ae:33, bf:31, zeros 1, ff 24 + middle 2 KiB: entropy 7.835, top bytes ac:21, 95:19, b6:18, ab:18, zeros 11, ff 3 + last 2 KiB: entropy 7.792, top bytes 02:22, 4c:21, 00:20, 26:19, zeros 20, ff 2 + [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x28128 off 0x2b19c8 + starts with JPEG SOI + [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x113e off 0x2d9af0 + heap root/ImageProps: block @0x2d9b0a len 0x113e, offsetTbl @+0x10b6, 13 records + [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 + [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc + = u32 (2160, 1440, 1065353216, 0, 8, 24, 257) + [2] tag 0x500a id 0x100a TargetImageType word inrec len 0xc0200000 off 0x130000 + [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0x2108c00 + [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x22378c off 0x7400000 + [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 + = u32 (1482512730, 0, 0) + [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 + = b'CRW_2444.CRW\x00\x9a\x02]G\x18\xa6JG\xff\x80 H\x00@\x00\x02\x82' + [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 + = b'CRW_2444.THM\x00\x04\x04\x010\x02\x00\x80\x00\x00\x00@\x12\x14\x00\x00\x00 ' + [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 + [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 + heap root/ImageProps/MeasuredInfo: block @0x2d9c72 len 0x10, offsetTbl @+0x0, 1 records + [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0xbec00000 off 0xedfff6ff + [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 + heap root/ImageProps/ImageDescription: block @0x2d9c82 len 0x5a, offsetTbl @+0x40, 2 records + [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 + = b'EOS D30 CMOS RAW IMAGE\x00\xff\x9f\xff\xfd\xf7\xfd\xfd\xff\xff' + [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 + = b'CRW:EOS D30 CMOS RAW IMAGE\x00\xdf\xff\xde\xff\xff' + [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 + heap root/ImageProps/CameraObject: block @0x2d9cdc len 0xb2, offsetTbl @+0x8e, 3 records + [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 + = b'\x00\xdf\xfb\xff\xff\xff\xb5\xef\xff\xff_\xef\xff\xdf\xef\xfb\xff\xff\xff\xfb\xff\xff{\xff\xfe\xff\xff\x7f\xfd?\xdf\xff' + [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 + = b'Canon\x00Canon EOS D30\x00\x00\xff\xff\xff\xff\xff\x7f\xbf\xfd\xff\xffo' + [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 + heap root/ImageProps/CameraObject/CameraSpecification: block @0x2d9d1c len 0x4e, offsetTbl @+0x20, 4 records + [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x172700aa off 0xfffaffff + [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xfffff1ff + [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0xffffff5f + [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 + = b'Firmware Version 1.02\x00\xbf\xff\xff\xff\xe7\xff\xff\xdf\xff}' + [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0xe32 off 0x284 + heap root/ImageProps/ExifInformation: block @0x2d9d8e len 0xe32, offsetTbl @+0xdaa, 13 records + [0] tag 0x0001 id 0x0001 FreeBytes byte heap len 0xc off 0x0 + [1] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x400048 + [2] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x320000 off 0x2620392 + [3] tag 0x102a id 0x102a CanonShotInfo word heap len 0x32 off 0xc + = u16 (50, 0, 288, 65484, 31, 181, 0, 0, 3, 1, 8, 8, 0, 0, 12290, 0, 0, 0, 1, 0, 0, 32, 180, 75, 0) + [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x40 off 0x3e + = u16 (64, 2, 0, 4, 0, 1, 0, 0, 0, 2, 0, 1, 0, 0, 0, 0, 0, 5, 2, 12291, 4, 0, 198, 50, 50, 1, 31, 285, 255, 0, 0, 0) + [5] tag 0x102c id 0x102c CanonColorInfo2 word heap len 0x100 off 0x7e + [6] tag 0x0032 id 0x0032 ? byte heap len 0x300 off 0x17e + [7] tag 0x0036 id 0x0036 ? byte heap len 0x868 off 0x47e + [8] tag 0x1030 id 0x1030 WhiteSample word heap len 0x76 off 0xce6 + [9] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x1140000 off 0x400c + [10] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xd5c + = u16 (34, 2224, 1456, 1, 0, 56, 12, 2215, 1451, 8, 220, 40, 1244, 0, 0, 0, 0) + [11] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xd7e + = u32 (1, 0, 514, 2015110) + [12] tag 0x1033 id 0x1033 CustomFunctions word heap len 0x1c off 0xd8e + = u16 (28, 256, 514, 768, 1025, 1280, 1536, 1794, 2049, 2305, 2560, 2816, 3074, 3328) + +== corpus/crw/crw-canon-eos-d60.crw + size 8591234, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 +heap root: block @0x1a len 0x831768, offsetTbl @+0x831744, 3 records + [0] tag 0x2005 id 0x2005 RawData struct heap len 0x662502 off 0x0 + RawData: 6694146 bytes, entropy 7.9868 bits/byte + first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 + middle 2 KiB: entropy 7.864, top bytes 8c:25, cb:21, 5d:18, 63:17, zeros 11, ff 4 + last 2 KiB: entropy 7.890, top bytes f1:16, 6d:16, 76:16, 55:15, zeros 11, ff 5 + [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x1cdc00 off 0x662502 + starts with JPEG SOI + [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x1642 off 0x830102 + heap root/ImageProps: block @0x83011c len 0x1642, offsetTbl @+0x15ba, 13 records + [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 + [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc + = u32 (3072, 2048, 1065353216, 0, 8, 24, 257) + [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x6d8f0000 off 0xcfba0d73 + [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xd8327aab + [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0xf42a3 off 0x9792ef1a + [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 + = u32 (1467566074, 0, 0) + [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 + = b'CRW_0099.CRW' + [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 + = b'CRW_0099.THM' + [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 + [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 + heap root/ImageProps/MeasuredInfo: block @0x830284 len 0x10, offsetTbl @+0x0, 1 records + [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x3fc00000 off 0xe2577c1e + [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 + heap root/ImageProps/ImageDescription: block @0x830294 len 0x5a, offsetTbl @+0x40, 2 records + [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 + = b'EOS D60 CMOS RAW IMAGE' + [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 + = b'CRW:EOS D60 CMOS RAW IMAGE' + [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 + heap root/ImageProps/CameraObject: block @0x8302ee len 0xb2, offsetTbl @+0x8e, 3 records + [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 + = b'' + [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 + = b'Canon\x00Canon EOS D60\x00\x0e{\xd7\xd8e\x18iG\x03,,c' + [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 + heap root/ImageProps/CameraObject/CameraSpecification: block @0x83032e len 0x4e, offsetTbl @+0x20, 4 records + [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x3645276f off 0x5982da24 + [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xa552febb + [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0xfedcfe5 + [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 + = b'Firmware Version 1.0.4' + [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x1336 off 0x284 + heap root/ImageProps/ExifInformation: block @0x8303a0 len 0x1336, offsetTbl @+0x129a, 15 records + [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 + [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x0 off 0x2620392 + [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x36 off 0x0 + = u16 (54, 0, 160, 52, 32768, 245, 65504, 0, 3, 0, 8, 8, 0, 0, 12288, 0, 0, 0, 1, 0, 0, 32768, 244, 99, 0, 0, 252) + [3] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x56 off 0x36 + [4] tag 0x0036 id 0x0036 ? byte heap len 0xbd6 off 0x8c + [5] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x1668000 off 0xed8d4a16 + [6] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xc62 + = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) + [7] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xc84 + = u32 (0, 0, 514, 5064559) + [8] tag 0x1033 id 0x1033 CustomFunctions word heap len 0x22 off 0xc94 + = u16 (32, 0, 515, 768, 1025, 1280, 1536, 1792, 2048, 2304, 2560, 2818, 3074, 3328, 3584, 3841, 18747) + [9] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xcb6 + = u16 (26, 279, 368, 371, 172, 219, 434, 436, 284, 162, 491, 494, 431) + [10] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xcd0 + = u16 (26, 281, 413, 415, 193, 223, 488, 490, 319, 150, 505, 507, 439) + [11] tag 0x10c2 id 0x10c2 ? word heap len 0x554 off 0xcea + [12] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x123e + = u16 (10, 440, 1021, 1027, 563) + [13] tag 0x10a8 id 0x10a8 ? word heap len 0x10 off 0x1248 + = u16 (16, 6000, 5312, 5989, 3161, 3717, 6503, 4118) + [14] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x42 off 0x1258 + +== corpus/crw/crw-canon-eos-digital-rebel.crw + size 10153786, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 +heap root: block @0x1a len 0x9aef20, offsetTbl @+0x9aeefc, 3 records + [0] tag 0x2005 id 0x2005 RawData struct heap len 0x61c014 off 0x0 + RawData: 6406164 bytes, entropy 7.8363 bits/byte + first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 + middle 2 KiB: entropy 7.825, top bytes 18:24, 59:18, cd:17, 06:17, zeros 10, ff 6 + last 2 KiB: entropy 7.836, top bytes b5:19, 5a:17, 2c:17, 56:17, zeros 10, ff 2 + [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x3916ea off 0x61c014 + starts with JPEG SOI + [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x17fe off 0x9ad6fe + heap root/ImageProps: block @0x9ad718 len 0x17fe, offsetTbl @+0x1776, 13 records + [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 + [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc + = u32 (3072, 2048, 1065353216, 0, 8, 24, 257) + [2] tag 0x500a id 0x100a TargetImageType word inrec len 0xdfff0000 off 0x7ff757d7 + [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0x7d5d557f + [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x221046 off 0x7df757dd + [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 + = u32 (1480372394, 0, 0) + [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 + = b'CRW_2390.CRW' + [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 + = b'CRW_2390.THM' + [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 + [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 + heap root/ImageProps/MeasuredInfo: block @0x9ad880 len 0x10, offsetTbl @+0x0, 1 records + [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0xbec00000 off 0xd7dd55f5 + [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 + heap root/ImageProps/ImageDescription: block @0x9ad890 len 0x5a, offsetTbl @+0x40, 2 records + [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 + = b'EOS DIGITAL REBEL CMOS RAW' + [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 + = b'CRW:EOS DIGITAL REBEL CMOS RAW' + [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 + heap root/ImageProps/CameraObject: block @0x9ad8ea len 0xb2, offsetTbl @+0x8e, 3 records + [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 + = b'' + [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 + = b'Canon\x00Canon EOS DIGITAL REBEL\x00w\x7f' + [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 + heap root/ImageProps/CameraObject/CameraSpecification: block @0x9ad92a len 0x4e, offsetTbl @+0x20, 4 records + [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x5116c2ff off 0xf55d5ff7 + [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0x5ff555f5 + [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0x5ddd77ff + [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 + = b'Firmware Version 1.1.1' + [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x14f2 off 0x284 + heap root/ImageProps/ExifInformation: block @0x9ad99c len 0x14f2, offsetTbl @+0x1406, 23 records + [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 + [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x320000 off 0x2620392 + [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x42 off 0x0 + [3] tag 0x1093 id 0x1093 ? word heap len 0x12 off 0x42 + = u16 (18, 0, 0, 0, 0, 0, 65535, 65535, 65535) + [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x54 + [5] tag 0x0036 id 0x0036 ? byte heap len 0xe82 off 0xb0 + [6] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x80000170 off 0xdf777555 + [7] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xf32 + = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) + [8] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xf54 + = u32 (0, 0, 514, 4776577) + [9] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x30 off 0xf64 + = u16 (7, 7, 3072, 2048, 3072, 2048, 151, 151, 1014, 608, 0, 0, 0, 64928, 64522, 0, 0, 65030, 0, 506, 0, 0, 8, 65535) + [10] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xf94 + = u16 (26, 327, 378, 369, 173, 258, 466, 457, 290, 186, 508, 497, 420) + [11] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xfae + = u16 (26, 304, 375, 375, 168, 206, 394, 395, 235, 158, 453, 454, 363) + [12] tag 0x10c2 id 0x10c2 ? word heap len 0x374 off 0xfc8 + [13] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x133c + = u16 (10, 743, 1022, 1026, 483) + [14] tag 0x10a8 id 0x10a8 ? word heap len 0x14 off 0x1346 + = u16 (20, 2993, 5200, 6982, 5987, 3210, 3894, 6220, 2675, 5190) + [15] tag 0x10ad id 0x10ad ? word heap len 0x3e off 0x135a + = u16 (62, 20127, 8480, 7000, 6000, 5600, 5200, 4724, 4200, 3266, 1446, 64839, 65135, 65196, 65255, 65286, 65324, 65373, 65444, 110, 671, 65391, 65193, 65166, 65140, 65124, 65110, 65099, 65088, 65070, 65282) + [16] tag 0x10ae id 0x10ae ColorTemperature word heap len 0x2 off 0x13ea + = u16 (5200,) + [17] tag 0x10af id 0x10af ? word heap len 0x2 off 0x13ec + = u16 (1268,) + [18] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x52 off 0x1398 + [19] tag 0x10b4 id 0x10b4 ColorSpace word heap len 0x2 off 0x13ee + = u16 (1,) + [20] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x13f0 + = u16 (10, 3, 0, 3072, 2048) + [21] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x13fa + = u16 (0, 159, 7, 112) + [22] tag 0x183b id 0x183b SerialNumberFormat dword heap len 0x4 off 0x1402 + = u32 (2415919104,) + +== corpus/crw/crw-canon-powershot-g1.crw + size 2300702, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 +heap root: block @0x1a len 0x231b04, offsetTbl @+0x231ae0, 3 records + [0] tag 0x2005 id 0x2005 RawData struct heap len 0x2271ca off 0x0 + RawData: 2257354 bytes, entropy 7.9338 bits/byte + first 2 KiB: entropy 6.516, top bytes 00:522, 22:26, 24:21, 64:18, zeros 522, ff 4 + middle 2 KiB: entropy 7.827, top bytes 5a:23, 6d:23, 99:17, aa:17, zeros 8, ff 1 + last 2 KiB: entropy 7.669, top bytes 44:31, 4c:30, 88:26, 20:24, zeros 16, ff 9 + [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x9394 off 0x2271ca + starts with JPEG SOI + [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x1582 off 0x23055e + heap root/ImageProps: block @0x230578 len 0x1582, offsetTbl @+0x14fa, 13 records + [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 + [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0x0 + = u32 (2048, 1536, 1065353216, 0, 8, 24, 257) + [2] tag 0x500a id 0x100a TargetImageType word inrec len 0xc5600000 off 0xbee19e93 + [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0x51bd76d5 + [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x11b9dd off 0x0 + [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x1c + = u32 (1065594172, 0, 0) + [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 + = b'CRW_1693.CRW\x00\x8e&\xe1>4\xd2\xd2\xa1ko\xddv?\xa5\x0f\xa8C\x0eY' + [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 + = b'CRW_1693.THM\x00~,\x07\xef\x19\xef\x88\xe3\xef2t\x89\n\x82\x88\x84\xe97\xe8' + [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 + [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 + heap root/ImageProps/MeasuredInfo: block @0x2306e0 len 0x10, offsetTbl @+0x0, 1 records + [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x0 off 0xab71c23a + [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x58 off 0x178 + heap root/ImageProps/ImageDescription: block @0x2306f0 len 0x58, offsetTbl @+0x3e, 2 records + [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 + = b'High definition CCD image\x00\xfd\xa31\x18u\x0b' + [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x1e off 0x20 + = b'CRW:High definition CCD image' + [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0x8e off 0x1d0 + heap root/ImageProps/CameraObject: block @0x230748 len 0x8e, offsetTbl @+0x6a, 3 records + [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0xc off 0x0 + = b'Kelly Sauke' + [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x19 off 0xc + = b'Canon\x00Canon PowerShot G1' + [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x44 off 0x26 + heap root/ImageProps/CameraObject/CameraSpecification: block @0x23076e len 0x44, offsetTbl @+0x16, 4 records + [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x0 off 0x6eb7053 + [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xf13edcd9 + [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0xa9d94930 + [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x16 off 0x0 + = b'Firmware Version 1.02' + [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x129c off 0x25e + heap root/ImageProps/ExifInformation: block @0x2307d6 len 0x129c, offsetTbl @+0x121e, 12 records + [0] tag 0x0001 id 0x0001 FreeBytes byte heap len 0xc off 0x0 + [1] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x0 off 0x0 + [2] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x1d30002 off 0xbd00fc + [3] tag 0x102a id 0x102a CanonShotInfo word heap len 0x34 off 0xc + = u16 (52, 0, 128, 115, 73, 170, 0, 0, 0, 0, 24, 0, 0, 0, 0, 0, 0, 0, 1, 494, 0, 79, 164, 0, 0, 0) + [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x46 off 0x40 + [5] tag 0x102c id 0x102c CanonColorInfo2 word heap len 0x100 off 0x86 + [6] tag 0x0032 id 0x0032 ? byte heap len 0x800 off 0x186 + [7] tag 0x0036 id 0x0036 ? byte heap len 0x800 off 0x986 + [8] tag 0x1030 id 0x1030 WhiteSample word heap len 0x66 off 0x1186 + [9] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x4040000 off 0x2255b19d + [10] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0x11ec + = u16 (34, 2144, 1560, 1, 1, 24, 15, 2071, 1550, 2108, 15, 2111, 1550, 0, 0, 0, 0) + [11] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0x120e + = u32 (1, 0, 514, 2257351) + +== corpus/crw/crw-canon-powershot-g5.crw + size 4894440, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 +heap root: block @0x1a len 0x4aaece, offsetTbl @+0x4aaea0, 4 records + [0] tag 0x2005 id 0x2005 RawData struct heap len 0x4a2f6e off 0x0 + RawData: 4861806 bytes, entropy 7.9622 bits/byte + first 2 KiB: entropy 6.656, top bytes 00:529, bb:14, b8:13, 73:13, zeros 529, ff 4 + middle 2 KiB: entropy 7.819, top bytes b2:24, 55:22, 75:21, 57:21, zeros 8, ff 1 + last 2 KiB: entropy 7.680, top bytes 89:34, 44:34, 4c:28, 88:28, zeros 20, ff 6 + [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x530e off 0x4a2f6e + starts with JPEG SOI + [2] tag 0x2008 id 0x2008 ThumbnailImage struct heap len 0x9fe off 0x4a827c + starts with JPEG SOI + [3] tag 0x300a id 0x300a ImageProps heap2 heap len 0x2226 off 0x4a8c7a + heap root/ImageProps: block @0x4a8c94 len 0x2226, offsetTbl @+0x219e, 13 records + [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 + [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc + = u32 (2592, 1944, 1065353216, 0, 8, 24, 257) + [2] tag 0x500a id 0x100a TargetImageType word inrec len 0xc9e90000 off 0x1fba0082 + [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xfc48c47b + [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x10078a off 0xa0fcb31 + [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 + = u32 (1059363106, 0, 0) + [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 + = b'CRW_0506.CRW\x001Y)\x97\xab\xf9n8\xba)\xd7\x9d\xd9\xc2\x13\x8a 9\x01' + [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 + = b'CRW_0506.THM\x00\x05\xc3!\xe5\xbc\xb1<\xf83\xde\xe1E\x93\xb1e5\xab\x81\x92' + [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 + [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 + heap root/ImageProps/MeasuredInfo: block @0x4a8dfc len 0x10, offsetTbl @+0x0, 1 records + [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x0 off 0xff9410 + [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 + heap root/ImageProps/ImageDescription: block @0x4a8e0c len 0x5a, offsetTbl @+0x40, 2 records + [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 + = b'High definition CCD image\x00\xad\xc1<\xb1\x80\xc0' + [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 + = b'CRW:High definition CCD image\x00LF' + [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xa8 off 0x1d2 + heap root/ImageProps/CameraObject: block @0x4a8e66 len 0xa8, offsetTbl @+0x84, 3 records + [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 + = b'\x008\x9e!,v\xf7\xc7\x18\xc4c"K\xfe\xe8;\xf4=\x9f\x1e\'9\xe3\x19nq%\xa1\x92\xb3\x853' + [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 + = b'Canon\x00Canon PowerShot G5\x009\x1c\xed\x8f6\xac\xa2' + [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x44 off 0x40 + heap root/ImageProps/CameraObject/CameraSpecification: block @0x4a8ea6 len 0x44, offsetTbl @+0x20, 3 records + [0] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xad6e2c6d + [1] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0x5cc19942 + [2] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 + = b'Firmware Version 1.00\x00\x00\x98\xaa\xebP\xf9rhu\xd0' + [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x1f24 off 0x27a + heap root/ImageProps/ExifInformation: block @0x4a8f0e len 0x1f24, offsetTbl @+0x1e7e, 16 records + [0] tag 0x0001 id 0x0001 FreeBytes byte heap len 0xc off 0x0 + [1] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x400 off 0x0 + [2] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0xe60002 off 0xd7011e + [3] tag 0x102a id 0x102a CanonShotInfo word heap len 0x44 off 0xc + [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x50 + [5] tag 0x0032 id 0x0032 ? byte heap len 0x800 off 0xac + [6] tag 0x0036 id 0x0036 ? byte heap len 0x1400 off 0x8ac + [7] tag 0x1030 id 0x1030 WhiteSample word heap len 0x76 off 0x1cac + [8] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x1290000 off 0x725bca88 + [9] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0x1d22 + = u16 (34, 2672, 1968, 1, 1, 24, 14, 2615, 1957, 2640, 14, 2643, 1957, 0, 0, 0, 0) + [10] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0x1d44 + = u32 (1, 0, 514, 4861804) + [11] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x38 off 0x1d54 + = u16 (9, 1, 2592, 1944, 2592, 215, 467, 39, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0) + [12] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x1d8c + = u16 (0, 0, 0, 0) + [13] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x1d94 + = u16 (10, 2, 2, 640, 480) + [14] tag 0x103c id 0x103c ? word heap len 0x9c off 0x1d9e + [15] tag 0x107f id 0x107f ? word heap len 0x44 off 0x1e3a + +== corpus/crw/crw-canon-powershot-g6.crw + size 8052032, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 +heap root: block @0x1a len 0x7add26, offsetTbl @+0x7adcf8, 4 records + [0] tag 0x2005 id 0x2005 RawData struct heap len 0x6f9704 off 0x0 + RawData: 7313156 bytes, entropy 7.9774 bits/byte + first 2 KiB: entropy 6.684, top bytes 00:520, c1:14, 48:14, ae:13, zeros 520, ff 8 + middle 2 KiB: entropy 7.885, top bytes 12:18, 31:17, 93:16, 64:15, zeros 13, ff 5 + last 2 KiB: entropy 7.355, top bytes f7:42, 63:36, ef:35, 7e:35, zeros 14, ff 15 + [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0xae5f4 off 0x6f9704 + starts with JPEG SOI + [2] tag 0x2008 id 0x2008 ThumbnailImage struct heap len 0x17b8 off 0x7a7cf8 + starts with JPEG SOI + [3] tag 0x300a id 0x300a ImageProps heap2 heap len 0x4848 off 0x7a94b0 + heap root/ImageProps: block @0x7a94ca len 0x4848, offsetTbl @+0x47c0, 13 records + [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 + [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc + = u32 (3072, 2304, 1065353216, 0, 8, 24, 257) + [2] tag 0x500a id 0x100a TargetImageType word inrec len 0xffff0000 off 0xffffffff + [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xffffffff + [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x1257af off 0xffffffff + [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 + = u32 (1607955982, 0, 0) + [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 + = b'CRW_2095.CRW\x00\n\\U\x88JJ\x04\x7f\xff' + [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 + = b'CRW_2095.THM\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 ' + [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 + [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 + heap root/ImageProps/MeasuredInfo: block @0x7a9632 len 0x10, offsetTbl @+0x0, 1 records + [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x0 off 0x0 + [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 + heap root/ImageProps/ImageDescription: block @0x7a9642 len 0x5a, offsetTbl @+0x40, 2 records + [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 + = b'High definition CCD image' + [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 + = b'CRW:High definition CCD image' + [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xa8 off 0x1d2 + heap root/ImageProps/CameraObject: block @0x7a969c len 0xa8, offsetTbl @+0x84, 3 records + [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 + = b'\x00\x00\x00\x00\x00\x00\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x08' + [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 + = b'Canon\x00Canon PowerShot G6' + [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x44 off 0x40 + heap root/ImageProps/CameraObject/CameraSpecification: block @0x7a96dc len 0x44, offsetTbl @+0x20, 3 records + [0] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0x0 + [1] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0x0 + [2] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 + = b'Firmware Version 1.00' + [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x4546 off 0x27a + heap root/ImageProps/ExifInformation: block @0x7a9744 len 0x4546, offsetTbl @+0x4482, 19 records + [0] tag 0x0001 id 0x0001 FreeBytes byte heap len 0xc off 0x0 + [1] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x0 off 0x0 + [2] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x2960002 off 0xd7011e + [3] tag 0x102a id 0x102a CanonShotInfo word heap len 0x44 off 0xc + [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x50 + [5] tag 0x0032 id 0x0032 ? byte heap len 0xc00 off 0xac + [6] tag 0x0041 id 0x0041 ? byte heap len 0x100 off 0xcac + [7] tag 0x0036 id 0x0036 ? byte heap len 0x2000 off 0xdac + [8] tag 0x1030 id 0x1030 WhiteSample word heap len 0x76 off 0x42ac + [9] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x1400000 off 0xffffffff + [10] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0x4322 + = u16 (34, 3160, 2344, 1, 1, 64, 24, 3135, 2327, 36, 24, 39, 2327, 0, 0, 0, 0) + [11] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0x4344 + = u32 (0, 0, 514, 7313154) + [12] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x38 off 0x4354 + = u16 (9, 1, 3072, 2304, 3072, 230, 553, 41, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0) + [13] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x438c + = u16 (0, 0, 0, 0) + [14] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x4394 + = u16 (10, 2, 0, 3072, 2304) + [15] tag 0x0040 id 0x0040 ? byte heap len 0x100 off 0x2dac + [16] tag 0x003f id 0x003f ? byte heap len 0x1400 off 0x2eac + [17] tag 0x103c id 0x103c ? word heap len 0x9c off 0x439e + [18] tag 0x107f id 0x107f ? word heap len 0x48 off 0x443a + diff --git a/crates/crw/clean-room/logs/table_sel0.json b/crates/crw/clean-room/logs/table_sel0.json new file mode 100644 index 00000000..79f5aa77 --- /dev/null +++ b/crates/crw/clean-room/logs/table_sel0.json @@ -0,0 +1,702 @@ +{ + "first": { + "111110": [ + 0, + 10 + ], + "11110": [ + 0, + 0 + ], + "101": [ + 0, + 2 + ], + "00": [ + 0, + 4 + ], + "1101": [ + 0, + 1 + ], + "010": [ + 0, + 3 + ], + "011": [ + 0, + 5 + ], + "11101": [ + 0, + 9 + ], + "100": [ + 0, + 6 + ], + "11100": [ + 0, + 8 + ], + "1100": [ + 0, + 7 + ], + "1111110": [ + 0, + 11 + ] + }, + "main": { + "00": [ + 0, + 3 + ], + "01": [ + 0, + 4 + ], + "100": [ + 0, + 2 + ], + "101": [ + 0, + 5 + ], + "1100": [ + 0, + 1 + ], + "1101": [ + 0, + 6 + ], + "11100": [ + 0, + 7 + ], + "111011": [ + 1, + 2 + ], + "111100": [ + 1, + 3 + ], + "111101": [ + 1, + 1 + ], + "1111101": [ + 0, + 9 + ], + "11111100": [ + 1, + 5 + ], + "111111010": [ + 2, + 2 + ], + "111111011": [ + 0, + 0 + ], + "1111111010": [ + 0, + 10 + ], + "1111111011": [ + 15, + 0 + ], + "1111111101110100": [ + 3, + 3 + ], + "1111111101110101": [ + 2, + 5 + ], + "1111111101110111": [ + 3, + 4 + ], + "1111111110000111": [ + 5, + 2 + ], + "1111111110001010": [ + 4, + 3 + ], + "1111111110001111": [ + 6, + 1 + ], + "1111111110010001": [ + 7, + 1 + ], + "1111111110011000": [ + 5, + 3 + ], + "1111111110100010": [ + 9, + 1 + ], + "1111111110110110": [ + 11, + 1 + ], + "1111111111001010": [ + 10, + 1 + ], + "1111111111001100": [ + 13, + 1 + ], + "1111111111010101": [ + 12, + 1 + ], + "1111111111011001": [ + 14, + 1 + ], + "1111111111011110": [ + 15, + 1 + ], + "1111111101110000": [ + 3, + 1 + ], + "1111111101110110": [ + 4, + 1 + ], + "1111111101111000": [ + 4, + 2 + ], + "1111111101111010": [ + 5, + 1 + ], + "1111111110001000": [ + 5, + 5 + ], + "1111111110001110": [ + 5, + 4 + ], + "1111111110110001": [ + 8, + 1 + ], + "1111111111011000": [ + 6, + 4 + ], + "1111111111100111": [ + 6, + 3 + ], + "11111111010": [ + 1, + 7 + ], + "1111111000": [ + 2, + 1 + ], + "1111111100": [ + 2, + 3 + ], + "1111111110000000": [ + 2, + 6 + ], + "1111100": [ + 1, + 4 + ], + "1111111101110001": [ + 3, + 2 + ], + "111111110110": [ + 2, + 4 + ], + "111010": [ + 0, + 8 + ], + "1111111001": [ + 1, + 6 + ], + "1111111101110010": [ + 1, + 8 + ], + "1111111110000101": [ + 2, + 8 + ], + "1111111111100110": [ + 10, + 4 + ], + "1111111111000111": [ + 6, + 5 + ], + "1111111101111001": [ + 3, + 5 + ], + "1111111110111000": [ + 4, + 4 + ], + "1111111110001011": [ + 7, + 6 + ], + "1111111110101101": [ + 4, + 5 + ], + "1111111111001111": [ + 9, + 2 + ], + "1111111110111100": [ + 6, + 6 + ], + "1111111110111001": [ + 7, + 2 + ], + "1111111101111011": [ + 3, + 6 + ], + "1111111110010100": [ + 9, + 6 + ], + "1111111110101100": [ + 4, + 6 + ], + "1111111110010011": [ + 7, + 5 + ], + "1111111110000110": [ + 2, + 7 + ], + "1111111101111101": [ + 3, + 8 + ], + "1111111110011010": [ + 7, + 4 + ], + "1111111101111100": [ + 3, + 7 + ], + "1111111111011101": [ + 11, + 3 + ], + "1111111110000011": [ + 5, + 6 + ], + "1111111110001101": [ + 7, + 7 + ], + "1111111110011111": [ + 9, + 5 + ], + "1111111101110011": [ + 1, + 9 + ], + "1111111110001001": [ + 5, + 8 + ], + "1111111111100101": [ + 8, + 4 + ], + "1111111110111110": [ + 15, + 5 + ], + "1111111111000101": [ + 6, + 2 + ], + "1111111110010101": [ + 9, + 7 + ], + "1111111111010000": [ + 8, + 5 + ], + "1111111111101101": [ + 8, + 2 + ], + "1111111110011110": [ + 4, + 8 + ], + "1111111110011001": [ + 13, + 7 + ], + "1111111110101011": [ + 6, + 7 + ], + "1111111110101110": [ + 9, + 4 + ], + "1111111110011101": [ + 4, + 7 + ], + "1111111111101011": [ + 13, + 2 + ], + "1111111110110100": [ + 11, + 4 + ], + "1111111111010110": [ + 12, + 6 + ], + "1111111101111110": [ + 2, + 9 + ], + "1111111111100010": [ + 11, + 2 + ], + "1111111111110011": [ + 8, + 3 + ], + "1111111110000010": [ + 3, + 9 + ], + "1111111110100110": [ + 11, + 5 + ], + "1111111110001100": [ + 5, + 9 + ], + "1111111111110101": [ + 10, + 2 + ], + "1111111110100011": [ + 15, + 10 + ], + "1111111110010000": [ + 15, + 9 + ], + "1111111111010011": [ + 9, + 3 + ], + "1111111111111001": [ + 14, + 2 + ], + "1111111110100111": [ + 11, + 9 + ], + "1111111110111010": [ + 13, + 9 + ], + "1111111111101010": [ + 15, + 3 + ], + "1111111111000000": [ + 10, + 7 + ], + "1111111111000100": [ + 8, + 6 + ], + "1111111111111000": [ + 12, + 2 + ], + "1111111110000100": [ + 5, + 7 + ], + "1111111110100101": [ + 6, + 8 + ], + "1111111111001101": [ + 14, + 9 + ], + "1111111101111111": [ + 7, + 9 + ], + "1111111111000110": [ + 12, + 7 + ], + "1111111110111011": [ + 8, + 7 + ], + "1111111111101000": [ + 14, + 5 + ], + "1111111110011011": [ + 11, + 6 + ], + "1111111110011100": [ + 9, + 8 + ], + "1111111111100000": [ + 10, + 5 + ], + "1111111111010001": [ + 10, + 6 + ], + "1111111110010110": [ + 4, + 9 + ], + "1111111111011100": [ + 14, + 6 + ], + "1111111110101000": [ + 13, + 6 + ], + "1111111110110101": [ + 8, + 8 + ], + "1111111110101001": [ + 15, + 7 + ], + "1111111110101111": [ + 8, + 9 + ], + "1111111111000010": [ + 10, + 9 + ], + "1111111111010100": [ + 14, + 8 + ], + "1111111111101100": [ + 12, + 4 + ], + "1111111110110011": [ + 15, + 6 + ], + "1111111111101001": [ + 12, + 5 + ], + "1111111110101010": [ + 13, + 8 + ], + "1111111111000001": [ + 7, + 3 + ], + "1111111110110111": [ + 2, + 10 + ], + "1111111110000001": [ + 1, + 10 + ], + "1111111111110001": [ + 15, + 2 + ], + "1111111110100000": [ + 6, + 9 + ], + "1111111110010111": [ + 11, + 7 + ], + "1111111111111010": [ + 14, + 3 + ], + "1111111111110000": [ + 14, + 4 + ], + "1111111111000011": [ + 10, + 8 + ], + "1111111110010010": [ + 7, + 8 + ], + "1111111111011111": [ + 13, + 3 + ], + "1111111111001000": [ + 12, + 8 + ], + "1111111111010010": [ + 14, + 7 + ], + "1111111110110010": [ + 13, + 5 + ], + "1111111110111101": [ + 13, + 4 + ], + "1111111111001011": [ + 15, + 4 + ], + "1111111111001001": [ + 12, + 9 + ], + "1111111110100100": [ + 11, + 8 + ], + "1111111110110000": [ + 15, + 8 + ], + "1111111111110100": [ + 10, + 3 + ], + "1111111111011010": [ + 4, + 10 + ], + "1111111110111111": [ + 3, + 10 + ], + "1111111110100001": [ + 9, + 9 + ], + "1111111111110110": [ + 12, + 3 + ], + "1111111111011011": [ + 6, + 10 + ], + "1111111111001110": [ + 5, + 10 + ], + "1111111111101111": [ + 13, + 10 + ], + "1111111111010111": [ + 7, + 10 + ], + "1111111111110010": [ + 12, + 10 + ], + "1111111111100011": [ + 9, + 10 + ], + "1111111111101110": [ + 10, + 10 + ], + "1111111111100001": [ + 8, + 10 + ], + "1111111111110111": [ + 14, + 10 + ], + "1111111111100100": [ + 11, + 10 + ] + } +} \ No newline at end of file diff --git a/crates/crw/clean-room/logs/table_sel0_d60class.json b/crates/crw/clean-room/logs/table_sel0_d60class.json new file mode 100644 index 00000000..1dd35c6c --- /dev/null +++ b/crates/crw/clean-room/logs/table_sel0_d60class.json @@ -0,0 +1,574 @@ +{ + "first": { + "111110": [ + 0, + 10 + ], + "11110": [ + 0, + 0 + ], + "101": [ + 0, + 2 + ], + "00": [ + 0, + 4 + ], + "1101": [ + 0, + 1 + ], + "010": [ + 0, + 3 + ], + "011": [ + 0, + 5 + ], + "11101": [ + 0, + 9 + ], + "100": [ + 0, + 6 + ], + "11100": [ + 0, + 8 + ], + "1100": [ + 0, + 7 + ] + }, + "main": { + "00": [ + 0, + 3 + ], + "01": [ + 0, + 4 + ], + "100": [ + 0, + 2 + ], + "101": [ + 0, + 5 + ], + "1100": [ + 0, + 1 + ], + "1101": [ + 0, + 6 + ], + "11100": [ + 0, + 7 + ], + "111011": [ + 1, + 2 + ], + "111100": [ + 1, + 3 + ], + "111101": [ + 1, + 1 + ], + "1111101": [ + 0, + 9 + ], + "11111100": [ + 1, + 5 + ], + "111111010": [ + 2, + 2 + ], + "111111011": [ + 0, + 0 + ], + "1111111010": [ + 0, + 10 + ], + "1111111011": [ + 15, + 0 + ], + "1111111101110100": [ + 3, + 3 + ], + "1111111101110101": [ + 2, + 5 + ], + "1111111101110111": [ + 3, + 4 + ], + "1111111110000111": [ + 5, + 2 + ], + "1111111110001010": [ + 4, + 3 + ], + "1111111110001111": [ + 6, + 1 + ], + "1111111110010001": [ + 7, + 1 + ], + "1111111110011000": [ + 5, + 3 + ], + "1111111110100010": [ + 9, + 1 + ], + "1111111110110110": [ + 11, + 1 + ], + "1111111111001010": [ + 10, + 1 + ], + "1111111111001100": [ + 13, + 1 + ], + "1111111111010101": [ + 12, + 1 + ], + "1111111111011001": [ + 14, + 1 + ], + "1111111111011110": [ + 15, + 1 + ], + "1111111101110000": [ + 3, + 1 + ], + "1111111101110110": [ + 4, + 1 + ], + "1111111101111000": [ + 4, + 2 + ], + "1111111101111010": [ + 5, + 1 + ], + "1111111110001000": [ + 5, + 5 + ], + "1111111110001110": [ + 5, + 4 + ], + "1111111110110001": [ + 8, + 1 + ], + "1111111111011000": [ + 6, + 4 + ], + "1111111111100111": [ + 6, + 3 + ], + "11111111010": [ + 1, + 7 + ], + "1111111000": [ + 2, + 1 + ], + "1111111100": [ + 2, + 3 + ], + "1111111110000000": [ + 2, + 6 + ], + "1111100": [ + 1, + 4 + ], + "1111111101110001": [ + 3, + 2 + ], + "111111110110": [ + 2, + 4 + ], + "111010": [ + 0, + 8 + ], + "1111111001": [ + 1, + 6 + ], + "1111111101110010": [ + 1, + 8 + ], + "1111111110000101": [ + 2, + 8 + ], + "1111111111100110": [ + 10, + 4 + ], + "1111111111000111": [ + 6, + 5 + ], + "1111111101111001": [ + 3, + 5 + ], + "1111111110111000": [ + 4, + 4 + ], + "1111111110001011": [ + 7, + 6 + ], + "1111111110101101": [ + 4, + 5 + ], + "1111111111001111": [ + 9, + 2 + ], + "1111111110111100": [ + 6, + 6 + ], + "1111111110111001": [ + 7, + 2 + ], + "1111111101111011": [ + 3, + 6 + ], + "1111111110010100": [ + 9, + 6 + ], + "1111111110101100": [ + 4, + 6 + ], + "1111111110010011": [ + 7, + 5 + ], + "1111111110000110": [ + 2, + 7 + ], + "1111111101111101": [ + 3, + 8 + ], + "1111111110011010": [ + 7, + 4 + ], + "1111111101111100": [ + 3, + 7 + ], + "1111111111011101": [ + 11, + 3 + ], + "1111111110000011": [ + 5, + 6 + ], + "1111111110001101": [ + 7, + 7 + ], + "1111111110011111": [ + 9, + 5 + ], + "1111111101110011": [ + 1, + 9 + ], + "1111111110001001": [ + 5, + 8 + ], + "1111111111100101": [ + 8, + 4 + ], + "1111111110111110": [ + 15, + 5 + ], + "1111111111000101": [ + 6, + 2 + ], + "1111111110010101": [ + 9, + 7 + ], + "1111111111010000": [ + 8, + 5 + ], + "1111111111101101": [ + 8, + 2 + ], + "1111111110011110": [ + 4, + 8 + ], + "1111111110011001": [ + 13, + 7 + ], + "1111111110101011": [ + 6, + 7 + ], + "1111111110101110": [ + 9, + 4 + ], + "1111111110011101": [ + 4, + 7 + ], + "1111111111101011": [ + 13, + 2 + ], + "1111111110110100": [ + 11, + 4 + ], + "1111111111010110": [ + 12, + 6 + ], + "1111111101111110": [ + 2, + 9 + ], + "1111111111100010": [ + 11, + 2 + ], + "1111111111110011": [ + 8, + 3 + ], + "1111111110000010": [ + 3, + 9 + ], + "1111111110100110": [ + 11, + 5 + ], + "1111111110001100": [ + 5, + 9 + ], + "1111111111110101": [ + 10, + 2 + ], + "1111111110100011": [ + 15, + 10 + ], + "1111111110010000": [ + 15, + 9 + ], + "1111111111010011": [ + 9, + 3 + ], + "1111111111111001": [ + 14, + 2 + ], + "1111111110100111": [ + 11, + 9 + ], + "1111111110111010": [ + 13, + 9 + ], + "1111111111101010": [ + 15, + 3 + ], + "1111111111000000": [ + 10, + 7 + ], + "1111111111000100": [ + 8, + 6 + ], + "1111111111111000": [ + 12, + 2 + ], + "1111111110000100": [ + 5, + 7 + ], + "1111111110100101": [ + 6, + 8 + ], + "1111111111001101": [ + 14, + 9 + ], + "1111111101111111": [ + 7, + 9 + ], + "1111111111000110": [ + 12, + 7 + ], + "1111111110111011": [ + 8, + 7 + ], + "1111111111101000": [ + 14, + 5 + ], + "1111111110011011": [ + 11, + 6 + ], + "1111111110011100": [ + 9, + 8 + ], + "1111111111100000": [ + 10, + 5 + ], + "1111111111010001": [ + 10, + 6 + ], + "1111111110010110": [ + 4, + 9 + ], + "1111111111011100": [ + 14, + 6 + ], + "1111111110101000": [ + 13, + 6 + ], + "1111111110110101": [ + 8, + 8 + ], + "1111111110101001": [ + 15, + 7 + ], + "1111111110101111": [ + 8, + 9 + ], + "1111111111000010": [ + 10, + 9 + ], + "1111111111010100": [ + 14, + 8 + ], + "1111111111101100": [ + 12, + 4 + ], + "1111111110110011": [ + 15, + 6 + ], + "1111111111101001": [ + 12, + 5 + ], + "1111111110101010": [ + 13, + 8 + ], + "1111111111000001": [ + 7, + 3 + ], + "1111111110110111": [ + 2, + 10 + ], + "1111111110000001": [ + 1, + 10 + ] + } +} \ No newline at end of file diff --git a/crates/crw/clean-room/logs/table_sel1.json b/crates/crw/clean-room/logs/table_sel1.json new file mode 100644 index 00000000..d93a3356 --- /dev/null +++ b/crates/crw/clean-room/logs/table_sel1.json @@ -0,0 +1,706 @@ +{ + "first": { + "11111110": [ + 0, + 10 + ], + "1101": [ + 0, + 0 + ], + "00": [ + 0, + 3 + ], + "01": [ + 0, + 2 + ], + "100": [ + 0, + 4 + ], + "1100": [ + 0, + 5 + ], + "101": [ + 0, + 1 + ], + "1110": [ + 0, + 6 + ], + "11110": [ + 0, + 7 + ], + "111110": [ + 0, + 9 + ], + "1111110": [ + 0, + 8 + ], + "111111110": [ + 0, + 11 + ] + }, + "main": { + "1111111100": [ + 0, + 10 + ], + "111111010": [ + 0, + 0 + ], + "01": [ + 0, + 3 + ], + "101": [ + 0, + 4 + ], + "00": [ + 0, + 2 + ], + "100": [ + 0, + 1 + ], + "11010": [ + 1, + 2 + ], + "111111011": [ + 2, + 3 + ], + "11101": [ + 1, + 3 + ], + "11011": [ + 1, + 1 + ], + "1111111010": [ + 3, + 1 + ], + "11111100": [ + 2, + 1 + ], + "1111100": [ + 2, + 2 + ], + "1100": [ + 0, + 5 + ], + "11100": [ + 0, + 6 + ], + "1111011": [ + 1, + 4 + ], + "111100": [ + 0, + 7 + ], + "1111010": [ + 0, + 8 + ], + "111111100": [ + 1, + 5 + ], + "1111111101100001": [ + 2, + 4 + ], + "11111111010": [ + 1, + 6 + ], + "1111101": [ + 0, + 9 + ], + "1111111101110001": [ + 7, + 1 + ], + "1111111101100011": [ + 4, + 1 + ], + "1111111011": [ + 3, + 2 + ], + "1111111101101111": [ + 6, + 1 + ], + "1111111101100010": [ + 3, + 3 + ], + "1111111101101001": [ + 5, + 1 + ], + "1111111101101011": [ + 4, + 3 + ], + "1111111101100110": [ + 1, + 7 + ], + "1111111101111011": [ + 9, + 1 + ], + "1111111101100000": [ + 15, + 0 + ], + "1111111101111111": [ + 2, + 8 + ], + "1111111101100101": [ + 1, + 9 + ], + "1111111101101010": [ + 3, + 4 + ], + "1111111110001000": [ + 13, + 1 + ], + "1111111101101000": [ + 1, + 8 + ], + "1111111110000010": [ + 10, + 1 + ], + "1111111110000011": [ + 11, + 1 + ], + "1111111101100100": [ + 4, + 2 + ], + "1111111110000100": [ + 4, + 4 + ], + "1111111101111001": [ + 8, + 1 + ], + "1111111101110101": [ + 2, + 6 + ], + "1111111111001111": [ + 6, + 7 + ], + "1111111110100001": [ + 11, + 6 + ], + "1111111101101100": [ + 5, + 2 + ], + "1111111101100111": [ + 2, + 5 + ], + "1111111101111010": [ + 2, + 7 + ], + "1111111110000111": [ + 5, + 8 + ], + "1111111101111000": [ + 3, + 7 + ], + "1111111110001011": [ + 14, + 1 + ], + "1111111101110110": [ + 3, + 8 + ], + "1111111110011111": [ + 12, + 1 + ], + "1111111110110110": [ + 4, + 8 + ], + "1111111111000011": [ + 6, + 8 + ], + "1111111110010100": [ + 4, + 6 + ], + "1111111110000000": [ + 7, + 2 + ], + "1111111110001110": [ + 4, + 9 + ], + "1111111101101110": [ + 3, + 5 + ], + "1111111111100101": [ + 11, + 2 + ], + "1111111110001100": [ + 15, + 1 + ], + "1111111110001111": [ + 4, + 7 + ], + "1111111101110011": [ + 3, + 6 + ], + "1111111111000001": [ + 9, + 2 + ], + "1111111110110111": [ + 8, + 2 + ], + "1111111101101101": [ + 2, + 9 + ], + "1111111110001010": [ + 5, + 7 + ], + "1111111101110010": [ + 6, + 2 + ], + "1111111101111110": [ + 4, + 5 + ], + "1111111110011101": [ + 8, + 6 + ], + "1111111110100000": [ + 7, + 6 + ], + "1111111111000000": [ + 12, + 6 + ], + "1111111101110100": [ + 5, + 3 + ], + "1111111101111101": [ + 5, + 5 + ], + "1111111111011011": [ + 6, + 4 + ], + "1111111110011110": [ + 6, + 5 + ], + "1111111101110000": [ + 3, + 9 + ], + "1111111110000001": [ + 5, + 9 + ], + "1111111110010000": [ + 6, + 3 + ], + "1111111110111011": [ + 10, + 6 + ], + "1111111111001110": [ + 6, + 6 + ], + "1111111110101111": [ + 7, + 3 + ], + "1111111111010110": [ + 8, + 8 + ], + "1111111110011000": [ + 7, + 8 + ], + "1111111111001101": [ + 12, + 2 + ], + "1111111111001010": [ + 10, + 2 + ], + "1111111110110100": [ + 10, + 7 + ], + "1111111110000110": [ + 5, + 4 + ], + "1111111111011101": [ + 9, + 8 + ], + "1111111111011001": [ + 7, + 7 + ], + "1111111110101000": [ + 9, + 5 + ], + "1111111110100010": [ + 9, + 6 + ], + "1111111110101101": [ + 9, + 7 + ], + "1111111111011110": [ + 10, + 5 + ], + "1111111110100101": [ + 8, + 5 + ], + "1111111110111000": [ + 8, + 4 + ], + "1111111110011011": [ + 7, + 5 + ], + "1111111110010011": [ + 5, + 6 + ], + "1111111111010000": [ + 9, + 3 + ], + "1111111110000101": [ + 6, + 9 + ], + "1111111110001001": [ + 15, + 10 + ], + "1111111110010010": [ + 15, + 9 + ], + "1111111111101001": [ + 8, + 3 + ], + "1111111111001001": [ + 14, + 9 + ], + "1111111110100100": [ + 8, + 9 + ], + "1111111111011100": [ + 14, + 2 + ], + "1111111111101000": [ + 13, + 2 + ], + "1111111111101011": [ + 13, + 3 + ], + "1111111110110011": [ + 8, + 7 + ], + "1111111111100111": [ + 15, + 2 + ], + "1111111110001101": [ + 11, + 9 + ], + "1111111101111100": [ + 7, + 9 + ], + "1111111101110111": [ + 1, + 10 + ], + "1111111110110101": [ + 7, + 10 + ], + "1111111111010100": [ + 14, + 7 + ], + "1111111110110000": [ + 11, + 7 + ], + "1111111111011000": [ + 13, + 7 + ], + "1111111110110001": [ + 13, + 8 + ], + "1111111111010011": [ + 13, + 5 + ], + "1111111110100011": [ + 13, + 6 + ], + "1111111110110010": [ + 13, + 9 + ], + "1111111110010110": [ + 2, + 10 + ], + "1111111110011010": [ + 3, + 10 + ], + "1111111110111101": [ + 5, + 10 + ], + "1111111110010111": [ + 4, + 10 + ], + "1111111110101100": [ + 8, + 10 + ], + "1111111111000101": [ + 12, + 8 + ], + "1111111110010001": [ + 6, + 10 + ], + "1111111111101010": [ + 11, + 10 + ], + "1111111110011001": [ + 9, + 9 + ], + "1111111110011100": [ + 7, + 4 + ], + "1111111110101110": [ + 11, + 8 + ], + "1111111111010101": [ + 15, + 8 + ], + "1111111111010010": [ + 13, + 4 + ], + "1111111110010101": [ + 10, + 8 + ], + "1111111111000111": [ + 14, + 5 + ], + "1111111110101001": [ + 11, + 4 + ], + "1111111110111010": [ + 15, + 4 + ], + "1111111111100000": [ + 13, + 10 + ], + "1111111111001100": [ + 14, + 3 + ], + "1111111111001011": [ + 10, + 3 + ], + "1111111110111111": [ + 10, + 4 + ], + "1111111111000010": [ + 12, + 3 + ], + "1111111111000100": [ + 11, + 5 + ], + "1111111110101011": [ + 15, + 7 + ], + "1111111110100110": [ + 12, + 9 + ], + "1111111111000110": [ + 14, + 4 + ], + "1111111111011010": [ + 12, + 4 + ], + "1111111111100010": [ + 15, + 3 + ], + "1111111111100110": [ + 11, + 3 + ], + "1111111111001000": [ + 14, + 6 + ], + "1111111110111110": [ + 9, + 4 + ], + "1111111110111100": [ + 12, + 5 + ], + "1111111111100011": [ + 15, + 6 + ], + "1111111111010111": [ + 9, + 10 + ], + "1111111110100111": [ + 15, + 5 + ], + "1111111110101010": [ + 12, + 7 + ], + "1111111111100100": [ + 10, + 9 + ], + "1111111111010001": [ + 10, + 10 + ], + "1111111111100001": [ + 14, + 8 + ], + "1111111111011111": [ + 12, + 10 + ], + "1111111110111001": [ + 14, + 10 + ] + }, + "reached": 6518333, + "N": 6518336, + "file": "CRW_0476.crw", + "selector": 1 +} \ No newline at end of file diff --git a/crates/crw/clean-room/logs/table_sel2.json b/crates/crw/clean-room/logs/table_sel2.json new file mode 100644 index 00000000..27167b06 --- /dev/null +++ b/crates/crw/clean-room/logs/table_sel2.json @@ -0,0 +1,706 @@ +{ + "first": { + "011": [ + 0, + 4 + ], + "010": [ + 0, + 7 + ], + "101": [ + 0, + 3 + ], + "1100": [ + 0, + 9 + ], + "000": [ + 0, + 6 + ], + "100": [ + 0, + 8 + ], + "001": [ + 0, + 5 + ], + "111110": [ + 0, + 1 + ], + "1101": [ + 0, + 2 + ], + "1110": [ + 0, + 0 + ], + "11110": [ + 0, + 10 + ], + "1111110": [ + 0, + 11 + ] + }, + "main": { + "011": [ + 0, + 6 + ], + "001": [ + 0, + 5 + ], + "101": [ + 0, + 7 + ], + "1101": [ + 0, + 8 + ], + "100": [ + 0, + 2 + ], + "11100": [ + 0, + 9 + ], + "1111011": [ + 1, + 5 + ], + "000": [ + 0, + 4 + ], + "1100": [ + 0, + 1 + ], + "010": [ + 0, + 3 + ], + "1111100": [ + 0, + 10 + ], + "1111111100110": [ + 3, + 8 + ], + "1111111100111": [ + 3, + 7 + ], + "1111010": [ + 1, + 1 + ], + "111111100": [ + 1, + 8 + ], + "111100": [ + 1, + 4 + ], + "111011": [ + 1, + 3 + ], + "111111110001": [ + 3, + 1 + ], + "1111111111001011": [ + 6, + 1 + ], + "111111011": [ + 2, + 1 + ], + "111111001": [ + 0, + 0 + ], + "11111111100000": [ + 4, + 1 + ], + "111010": [ + 1, + 2 + ], + "11111010": [ + 1, + 6 + ], + "1111111010": [ + 2, + 3 + ], + "111111010": [ + 2, + 2 + ], + "11111110111": [ + 2, + 4 + ], + "11111011": [ + 1, + 7 + ], + "1111111110100111": [ + 5, + 1 + ], + "1111111111100101": [ + 8, + 1 + ], + "1111111100100": [ + 2, + 5 + ], + "1111111111010011": [ + 7, + 1 + ], + "1111111100101": [ + 3, + 3 + ], + "111111110000": [ + 3, + 2 + ], + "11111110110": [ + 1, + 9 + ], + "1111111110001001": [ + 2, + 6 + ], + "1111111111010100": [ + 9, + 1 + ], + "1111111110001011": [ + 7, + 6 + ], + "1111111110010000": [ + 9, + 7 + ], + "1111111111001111": [ + 6, + 5 + ], + "1111111111111000": [ + 10, + 1 + ], + "11111111011111": [ + 2, + 7 + ], + "11111111011101": [ + 5, + 6 + ], + "1111111110011011": [ + 6, + 8 + ], + "1111111110111110": [ + 6, + 6 + ], + "1111111110010010": [ + 4, + 8 + ], + "11111111100001": [ + 2, + 9 + ], + "11111111011100": [ + 2, + 8 + ], + "1111111101010": [ + 3, + 6 + ], + "1111111111101110": [ + 13, + 1 + ], + "1111111101001": [ + 3, + 5 + ], + "1111111111101100": [ + 11, + 1 + ], + "11111111011010": [ + 5, + 8 + ], + "1111111110100101": [ + 11, + 5 + ], + "1111111110110010": [ + 15, + 6 + ], + "1111111110100100": [ + 8, + 8 + ], + "1111111101000": [ + 3, + 4 + ], + "1111111111000001": [ + 4, + 4 + ], + "1111111110001111": [ + 9, + 8 + ], + "1111111110010110": [ + 4, + 7 + ], + "1111111110101101": [ + 8, + 7 + ], + "1111111110100010": [ + 5, + 2 + ], + "1111111110011010": [ + 7, + 5 + ], + "1111111111111001": [ + 12, + 1 + ], + "11111111011001": [ + 5, + 7 + ], + "1111111110001101": [ + 1, + 10 + ], + "11111111011011": [ + 5, + 9 + ], + "1111111101011": [ + 3, + 9 + ], + "11111111011000": [ + 7, + 9 + ], + "1111111110110101": [ + 10, + 9 + ], + "1111111110110111": [ + 5, + 3 + ], + "1111111110011101": [ + 6, + 7 + ], + "1111111110111010": [ + 4, + 3 + ], + "1111111111001110": [ + 12, + 6 + ], + "1111111110101000": [ + 4, + 6 + ], + "1111111110001110": [ + 5, + 5 + ], + "1111111110101110": [ + 4, + 5 + ], + "1111111110001010": [ + 4, + 2 + ], + "111111000": [ + 15, + 0 + ], + "1111111111000101": [ + 12, + 8 + ], + "1111111110010100": [ + 9, + 6 + ], + "1111111111011100": [ + 12, + 5 + ], + "1111111110101100": [ + 13, + 6 + ], + "1111111110011100": [ + 11, + 6 + ], + "1111111110111001": [ + 10, + 8 + ], + "1111111111010110": [ + 10, + 6 + ], + "1111111111011110": [ + 10, + 4 + ], + "1111111111110100": [ + 8, + 2 + ], + "1111111110110011": [ + 8, + 6 + ], + "1111111111010101": [ + 9, + 3 + ], + "1111111111101111": [ + 8, + 3 + ], + "11111111011110": [ + 7, + 8 + ], + "1111111110001100": [ + 9, + 9 + ], + "1111111110111111": [ + 10, + 7 + ], + "1111111111101101": [ + 14, + 4 + ], + "1111111111110000": [ + 6, + 3 + ], + "1111111111111100": [ + 14, + 1 + ], + "1111111110010001": [ + 15, + 9 + ], + "1111111110110100": [ + 11, + 4 + ], + "1111111110010011": [ + 5, + 4 + ], + "1111111111111011": [ + 10, + 2 + ], + "1111111111110101": [ + 15, + 1 + ], + "1111111111100001": [ + 6, + 4 + ], + "1111111111011111": [ + 8, + 4 + ], + "1111111110110000": [ + 9, + 5 + ], + "1111111110011000": [ + 4, + 9 + ], + "1111111110111100": [ + 15, + 7 + ], + "1111111111110010": [ + 12, + 3 + ], + "1111111111100011": [ + 11, + 3 + ], + "1111111111100110": [ + 14, + 5 + ], + "1111111111010010": [ + 14, + 6 + ], + "1111111111100100": [ + 13, + 2 + ], + "1111111111100010": [ + 10, + 5 + ], + "1111111110011110": [ + 6, + 9 + ], + "1111111110001000": [ + 7, + 7 + ], + "1111111111010001": [ + 7, + 2 + ], + "1111111111011010": [ + 6, + 2 + ], + "1111111111001001": [ + 7, + 3 + ], + "1111111110100011": [ + 13, + 7 + ], + "1111111111011001": [ + 8, + 5 + ], + "1111111111001010": [ + 4, + 10 + ], + "1111111110110001": [ + 13, + 5 + ], + "1111111110111101": [ + 13, + 4 + ], + "1111111110100110": [ + 7, + 4 + ], + "1111111110100000": [ + 11, + 8 + ], + "1111111110101011": [ + 3, + 10 + ], + "1111111110100001": [ + 13, + 8 + ], + "1111111110010101": [ + 8, + 9 + ], + "1111111110010111": [ + 11, + 7 + ], + "1111111110011111": [ + 11, + 9 + ], + "1111111111001100": [ + 12, + 7 + ], + "1111111111000110": [ + 14, + 7 + ], + "1111111110101010": [ + 15, + 8 + ], + "1111111110110110": [ + 9, + 4 + ], + "1111111111110011": [ + 14, + 2 + ], + "1111111110111011": [ + 15, + 5 + ], + "1111111111011101": [ + 11, + 2 + ], + "1111111111101001": [ + 12, + 4 + ], + "1111111110101001": [ + 13, + 9 + ], + "1111111111000100": [ + 14, + 8 + ], + "1111111111011011": [ + 15, + 3 + ], + "1111111111100111": [ + 13, + 3 + ], + "1111111111101011": [ + 15, + 2 + ], + "1111111111011000": [ + 9, + 2 + ], + "1111111111001101": [ + 15, + 4 + ], + "1111111111000000": [ + 5, + 10 + ], + "1111111111000011": [ + 12, + 9 + ], + "1111111110101111": [ + 7, + 10 + ], + "1111111111010111": [ + 13, + 10 + ], + "1111111111010000": [ + 14, + 9 + ], + "1111111111110111": [ + 12, + 2 + ], + "1111111111110110": [ + 10, + 3 + ], + "1111111111001000": [ + 6, + 10 + ], + "1111111111000111": [ + 9, + 10 + ], + "1111111110111000": [ + 2, + 10 + ], + "1111111111101010": [ + 12, + 10 + ], + "1111111111111010": [ + 14, + 3 + ], + "1111111111000010": [ + 8, + 10 + ], + "1111111111100000": [ + 11, + 10 + ], + "1111111111110001": [ + 14, + 10 + ], + "1111111110011001": [ + 15, + 10 + ], + "1111111111101000": [ + 10, + 10 + ] + }, + "reached": 6518335, + "N": 6518336, + "file": "crw-canon-eos-300d-b.crw", + "selector": 2 +} \ No newline at end of file diff --git a/crates/crw/clean-room/logs/verification-sweep.txt b/crates/crw/clean-room/logs/verification-sweep.txt new file mode 100644 index 00000000..18992d2d --- /dev/null +++ b/crates/crw/clean-room/logs/verification-sweep.txt @@ -0,0 +1,22 @@ +CRW_0476.crw: sel 1, 3152x2068: EXACT +CRW_0478.crw: sel 1, 3152x2068: EXACT +CRW_0486.crw: sel 0, 3152x2068: EXACT +CRW_0527.crw: sel 1, 3152x2068: EXACT +CRW_0683.crw: sel 1, 3152x2068: EXACT +CRW_0717.crw: sel 1, 3152x2068: EXACT +CRW_0765.crw: sel 1, 3152x2068: EXACT +CRW_0822.crw: sel 1, 3152x2068: EXACT +CRW_0879.crw: sel 1, 3152x2068: EXACT +CRW_1011.crw: sel 1, 3152x2068: EXACT +CRW_1153.crw: sel 0, 3152x2068: EXACT +crw-canon-eos-10d-a.crw: sel 0, 3152x2068: EXACT +crw-canon-eos-10d-b.crw: sel 2, 3152x2068: EXACT +crw-canon-eos-10d-c.crw: sel 2, 3152x2068: EXACT +crw-canon-eos-300d-a.crw: sel 0, 3152x2068: EXACT +crw-canon-eos-300d-b.crw: sel 2, 3152x2068: EXACT +crw-canon-eos-d30.crw: sel 1, 2224x1456: EXACT +crw-canon-eos-d60.crw: sel 0, 3152x2068: EXACT +crw-canon-eos-digital-rebel.crw: sel 0, 3152x2068: EXACT +crw-canon-powershot-g1.crw: sel 1, 2144x1560: EXACT +crw-canon-powershot-g5.crw: sel 1, 2672x1968: 2685618 mismatches, first at 0 (row 0, col 0) +crw-canon-powershot-g6.crw: sel 0, 3160x2344: EXACT diff --git a/crates/crw/clean-room/p1_payload.py b/crates/crw/clean-room/p1_payload.py new file mode 100644 index 00000000..d8faf5ad --- /dev/null +++ b/crates/crw/clean-room/p1_payload.py @@ -0,0 +1,184 @@ +"""P1 — RawData payload partition statistics (no bitstream decoding). + +Questions (plan 01-plan.md P1): + 1. Exact leading zero run of RawData per file. + 2. Does RawData split as [low-bits wh/4][compressed L][pad] (prefix hypothesis) or + [compressed L][low-bits wh/4][pad] (suffix hypothesis) on EOS bodies? Tested by + row-structure statistics of both candidate regions (2 bits/pixel, rawW/4 bytes/row): + masked sensor rows should show as structured/near-constant rows, the entropy-coded + stream should not align to 788-byte rows. + 3. The constant third DecoderTable u32 (514) and the 2-3 byte remainder. + 4. Byte-stuffing check in the compressed region (ff-digram statistics). + +Prints statistics only; no sample values of image content beyond aggregate counts. + +Usage: python -I p1_payload.py file.crw [more...] +""" + +import math +import struct +import sys +from collections import Counter + + +def read_block_records(buf, start, length, endian): + if length < 6: + return [] + (tbl,) = struct.unpack_from(endian + "I", buf, start + length - 4) + if tbl + 2 > length - 4: + return [] + (n,) = struct.unpack_from(endian + "H", buf, start + tbl) + recs = [] + pos = start + tbl + 2 + if tbl + 2 + n * 10 > length: + return [] + for _ in range(n): + tc, ln, off = struct.unpack_from(endian + "HII", buf, pos) + pos += 10 + recs.append((tc, ln, off)) + return recs + + +def find_records(buf, start, length, endian, depth=0): + """Yield (tag_id, abs_offset, length) for every in-heap record, recursively.""" + if depth > 8: + return + for tc, ln, off in read_block_records(buf, start, length, endian): + stg = (tc >> 14) & 3 + dt = (tc >> 11) & 7 + tag = tc & 0x3FFF + if stg != 0: + yield (tag, None, (ln, off)) # in-entry value + continue + if off + ln > length: + continue + if dt in (5, 6): + yield from find_records(buf, start + off, ln, endian, depth + 1) + else: + yield (tag, start + off, ln) + + +def entropy(data): + if not data: + return 0.0 + c = Counter(data) + n = len(data) + return -sum(v / n * math.log2(v / n) for v in c.values()) + + +def zero_run(data): + i = 0 + while i < len(data) and data[i] == 0: + i += 1 + return i + + +def row_structure(region, row_bytes, crop_top, label, out): + """Reshape region into rows of row_bytes; report zero/constant-row runs and + per-band mean of 2-bit values (both bit orders give the same mean-of-byte/4 scale, + so use byte means as the aggregate).""" + nrows = len(region) // row_bytes + if nrows == 0: + out.append(f" {label}: shorter than one row") + return + zrows = 0 + for r in range(nrows): + row = region[r * row_bytes : (r + 1) * row_bytes] + if any(row): + break + zrows += 1 + bands = [] + for name, lo, hi in (("rows 0..top", 0, min(crop_top, nrows)), + ("top..top+32", min(crop_top, nrows), min(crop_top + 32, nrows)), + ("middle 32", nrows // 2, min(nrows // 2 + 32, nrows)), + ("last 32", max(0, nrows - 32), nrows)): + seg = region[lo * row_bytes : hi * row_bytes] + if seg: + bands.append(f"{name}: mean {sum(seg) / len(seg):7.2f} ent {entropy(seg):5.3f}") + out.append(f" {label}: {nrows} rows of {row_bytes} B, leading all-zero rows: {zrows}") + for b in bands: + out.append(f" {b}") + + +def ff_digrams(data, out): + n = len(data) + c = Counter(data) + ffs = c.get(0xFF, 0) + ff00 = sum(1 for i in range(n - 1) if data[i] == 0xFF and data[i + 1] == 0) + exp = ffs * c.get(0, 0) / max(1, n) # expected if independent + out.append(f" stuffing check: ff count {ffs} ({ffs / max(1, n):.4%}), " + f"ff00 digrams {ff00}, expected-if-independent {exp:.1f}") + + +def analyse(fname, show_bytes): + out = [f"== {fname}"] + buf = open(fname, "rb").read() + endian = {b"II": "<", b"MM": ">"}.get(buf[0:2]) + (hlen,) = struct.unpack_from(endian + "I", buf, 2) + recs = dict() + raw = None + for tag, off, ln in find_records(buf, hlen, len(buf) - hlen, endian): + if tag == 0x2005: + raw = (off, ln) + recs.setdefault(tag, (off, ln)) + if raw is None or raw[0] is None: + out.append(" !! no RawData record") + return out + roff, rlen = raw + data = buf[roff : roff + rlen] + # SensorInfo + soff, sln = recs.get(0x1031, (None, 0)) + w = h = top = 0 + if soff: + vals = struct.unpack_from(endian + "H" * (sln // 2), buf, soff) + w, h, top = vals[1], vals[2], vals[6] + # DecoderTable + doff, dln = recs.get(0x1835, (None, 0)) + dt = struct.unpack_from(endian + "IIII", buf, doff) if doff and dln >= 16 else (None,) * 4 + n_tbl, d1, d2, comp_len = dt + low = (w * h) // 4 + rem = rlen - (comp_len or 0) + out.append(f" raw {w}x{h} S={rlen} tbl={n_tbl} f1={d1} f2={d2} L={comp_len}" + f" S-L={rem} wh/4={low} S-L-wh/4={rem - low}") + out.append(f" leading zero run: {zero_run(data)} bytes" + + (f" (= {zero_run(data) / (w / 4):.2f} rows of 2-bit LSBs)" if w else "")) + if show_bytes: + out.append(f" last 8 bytes: {data[-8:].hex()}") + has_low = rem - low in (0, 1, 2, 3, 4) and low > 4 * (w or 1) + if has_low and w: + out.append(" prefix hypothesis — region [0 : wh/4) as 2-bit rows:") + row_structure(data[:low], w // 4, top, "prefix", out) + out.append(" suffix hypothesis — region [L : L+wh/4) as 2-bit rows:") + row_structure(data[comp_len : comp_len + low], w // 4, top, "suffix", out) + out.append(" compressed-candidate stats:") + for name, seg in (("after-prefix [wh/4 : wh/4+L)", data[low : low + comp_len]), + ("from-0 [0 : L)", data[:comp_len])): + out.append(f" {name}: entropy {entropy(seg[:1 << 20]):.4f} (first MiB), " + f"{entropy(seg[-(1 << 20):]):.4f} (last MiB)") + ff_digrams(data[low : low + comp_len], out) + else: + out.append(" no wh/4 region (PowerShot class): compressed assumed [0 : L)") + out.append(f" entropy first MiB {entropy(data[:1 << 20]):.4f}, " + f"last MiB {entropy(data[max(0, comp_len - (1 << 20)) : comp_len]):.4f}") + ff_digrams(data[:comp_len], out) + out.append(f" tail region [L : S): {rem} bytes" + + (f", hex {data[comp_len:rlen].hex()}" if show_bytes and rem <= 8 else "")) + return out + + +def main(): + args = sys.argv[1:] + show = True + if args and args[0] == "--no-bytes": + show = False + args = args[1:] + for fname in args: + try: + print("\n".join(analyse(fname, show))) + except Exception as e: + print(f"== {fname}\n !! exception: {e!r}") + print() + + +if __name__ == "__main__": + main() diff --git a/crates/crw/clean-room/p1b_markers.py b/crates/crw/clean-room/p1b_markers.py new file mode 100644 index 00000000..65d9137e --- /dev/null +++ b/crates/crw/clean-room/p1b_markers.py @@ -0,0 +1,119 @@ +"""P1b — ff-successor and JPEG-marker statistics of the RawData payload regions. + +Follow-up to p1_payload.py: on EOS bodies every 0xff in the compressed region was followed +by 0x00 (JPEG-style stuffing); on the G5 the payload starts with exactly 514 zero bytes +(= the constant DecoderTable field) and ends in ff d9 (JPEG EOI). This script measures, +per file: the histogram of bytes following 0xff in the compressed region, any JPEG markers +(ff xx, xx not 00), and — for public files — hex of small structural windows (region +starts/ends, bytes around offset 514). Still no entropy decoding. + +Usage: python -I p1b_markers.py [--no-bytes] file.crw ... +""" + +import struct +import sys +from collections import Counter + + +def find_records(buf, start, length, endian, depth=0): + """Yield (tag_id, abs_offset, length) for in-heap records, recursively + (same container model as ciff_dump.py; duplicated so the script runs under python -I).""" + if depth > 8 or length < 6: + return + (tbl,) = struct.unpack_from(endian + "I", buf, start + length - 4) + if tbl + 2 > length - 4: + return + (n,) = struct.unpack_from(endian + "H", buf, start + tbl) + if tbl + 2 + n * 10 > length: + return + for k in range(n): + tc, ln, off = struct.unpack_from(endian + "HII", buf, start + tbl + 2 + k * 10) + stg = (tc >> 14) & 3 + dt = (tc >> 11) & 7 + tag = tc & 0x3FFF + if stg != 0: + yield (tag, None, (ln, off)) + continue + if off + ln > length: + continue + if dt in (5, 6): + yield from find_records(buf, start + off, ln, endian, depth + 1) + else: + yield (tag, start + off, ln) + + +def analyse(fname, show): + out = [f"== {fname}"] + buf = open(fname, "rb").read() + endian = {b"II": "<", b"MM": ">"}.get(buf[0:2]) + (hlen,) = struct.unpack_from(endian + "I", buf, 2) + recs = {} + for tag, off, ln in find_records(buf, hlen, len(buf) - hlen, endian): + if off is not None: + recs.setdefault(tag, (off, ln)) + roff, rlen = recs[0x2005] + data = buf[roff : roff + rlen] + soff, sln = recs.get(0x1031, (None, 0)) + vals = struct.unpack_from(endian + "H" * (sln // 2), buf, soff) if soff else () + w, h = (vals[1], vals[2]) if len(vals) > 2 else (0, 0) + doff, _ = recs.get(0x1835, (None, 0)) + n_tbl, _, f2, comp_len = struct.unpack_from(endian + "IIII", buf, doff) + low = w * h // 4 + has_low = 0 <= rlen - comp_len - low <= 4 + comp = data[low : low + comp_len] if has_low else data[:comp_len] + out.append(f" raw {w}x{h} tbl={n_tbl} f2={f2} L={comp_len} lowbits={'yes' if has_low else 'no'}") + + succ = Counter() + markers = Counter() + first_markers = [] + i = 0 + while True: + i = comp.find(0xFF, i) + if i < 0 or i + 1 >= len(comp): + break + nxt = comp[i + 1] + succ[nxt] += 1 + if nxt != 0: + markers[nxt] += 1 + if len(first_markers) < 12: + first_markers.append((i, nxt)) + i += 2 + else: + i += 2 + top = ", ".join(f"{b:02x}:{c}" for b, c in succ.most_common(8)) + out.append(f" ff followed by: {top}") + if markers: + mk = ", ".join(f"ff{b:02x}:{c}" for b, c in sorted(markers.items())) + out.append(f" markers: {mk}") + out.append(f" first markers at: {[(hex(p), hex(m)) for p, m in first_markers]}") + rst = [p for p, m in first_markers if 0xD0 <= m <= 0xD7] + if len(rst) > 2: + out.append(f" first restart gaps: {[rst[k + 1] - rst[k] for k in range(len(rst) - 1)]}") + else: + out.append(" markers: none (pure stuffed entropy data)") + if show: + out.append(f" comp[0:24] : {comp[:24].hex()}") + out.append(f" comp[-8:] : {comp[-8:].hex()}") + out.append(f" raw[504:544] : {data[504:544].hex()}") + out.append(f" raw[-8:] : {data[-8:].hex()}") + if has_low: + out.append(f" low[-8:] : {data[low - 8 : low].hex()} (end of low-bits region)") + return out + + +def main(): + args = sys.argv[1:] + show = True + if args and args[0] == "--no-bytes": + show = False + args = args[1:] + for fname in args: + try: + print("\n".join(analyse(fname, show))) + except Exception as e: + print(f"== {fname}\n !! exception: {e!r}") + print() + + +if __name__ == "__main__": + main() diff --git a/crates/crw/clean-room/p1c_lowbits_bounds.py b/crates/crw/clean-room/p1c_lowbits_bounds.py new file mode 100644 index 00000000..8ad04db5 --- /dev/null +++ b/crates/crw/clean-room/p1c_lowbits_bounds.py @@ -0,0 +1,96 @@ +"""P1c — verify the low-bits region bounds via non-stuffed ff statistics. + +On EOS bodies the entropy region contains no ff followed by non-00 at all (perfect +JPEG-style stuffing); on the 12-bit PowerShots (G5/G6) a region of unstuffed uniform +bytes exists. This script shows that every non-stuffed ff falls inside +[514, 514 + wh/4) of the RawData payload, establishing the layout +[514 zeros][low bits wh/4][stuffed entropy][terminator] for those bodies, and +reports the zero run + stuffing purity for every file class. No entropy decoding. + +Usage: python -I p1c_lowbits_bounds.py file.crw ... +""" + +import struct +import sys + + +def records(buf, start, length, endian, depth=0): + if depth > 8 or length < 6: + return + (tbl,) = struct.unpack_from(endian + "I", buf, start + length - 4) + if tbl + 2 > length - 4: + return + (n,) = struct.unpack_from(endian + "H", buf, start + tbl) + if tbl + 2 + n * 10 > length: + return + for k in range(n): + tc, ln, off = struct.unpack_from(endian + "HII", buf, start + tbl + 2 + k * 10) + if (tc >> 14) & 3: + continue + if off + ln > length: + continue + if ((tc >> 11) & 7) in (5, 6): + yield from records(buf, start + off, ln, endian, depth + 1) + else: + yield (tc & 0x3FFF, start + off, ln) + + +def analyse(fname): + buf = open(fname, "rb").read() + endian = {b"II": "<", b"MM": ">"}.get(buf[0:2]) + (hl,) = struct.unpack_from(endian + "I", buf, 2) + r = {} + for t, o, l in records(buf, hl, len(buf) - hl, endian): + r.setdefault(t, (o, l)) + ro, rl = r[0x2005] + so, sl = r[0x1031] + v = struct.unpack_from(endian + "H" * (sl // 2), buf, so) + w, h = v[1], v[2] + do, _ = r[0x1835] + n_tbl, _, f2, comp_len = struct.unpack_from(endian + "IIII", buf, do) + data = buf[ro : ro + rl] + low = w * h // 4 + eos = 0 <= rl - comp_len - low <= 4 + + # positions of non-stuffed ff (ff followed by non-00), excluding the terminator + pos = [] + i = 0 + while True: + i = data.find(0xFF, i) + if i < 0 or i + 1 >= len(data): + break + if data[i + 1] != 0: + pos.append(i) + i += 2 + pos = [p for p in pos if p < rl - 4] + + comp_start = low if eos else 0 + zr = 0 + while comp_start + zr < rl and data[comp_start + zr] == 0: + zr += 1 + name = fname.replace("\\", "/").rsplit("/", 1)[-1] + if eos: + inside = sum(1 for p in pos if p < low) + print(f"{name:36} EOS layout: lowbits [0:{low}) + zeros({zr}) + entropy; " + f"non-stuffed ff: {len(pos)} total, {inside} in lowbits, " + f"{len(pos) - inside} in entropy (expect 0)") + else: + lo, hi = 514, 514 + low + inside = sum(1 for p in pos if lo <= p < hi) + if pos and inside == len(pos): + print(f"{name:36} PS 12-bit layout: zeros({zr}) + lowbits [{lo}:{hi}) + entropy; " + f"all {len(pos)} non-stuffed ff inside lowbits; " + f"first {pos[0]}, last {pos[-1]}") + elif not pos: + print(f"{name:36} PS 10-bit layout: zeros({zr}) + entropy; " + f"no non-stuffed ff (pure stuffed stream)") + else: + print(f"{name:36} PS ?: {len(pos)} non-stuffed ff, {inside} inside " + f"[{lo}:{hi}); first {pos[0]}, last {pos[-1]} — hypothesis fails") + + +for fn in sys.argv[1:]: + try: + analyse(fn) + except Exception as e: + print(f"{fn}: !! {e!r}") diff --git a/crates/crw/clean-room/p2_oracle_lsb.py b/crates/crw/clean-room/p2_oracle_lsb.py new file mode 100644 index 00000000..e0e9dcfe --- /dev/null +++ b/crates/crw/clean-room/p2_oracle_lsb.py @@ -0,0 +1,118 @@ +"""P2 — the decisive low-bits test against the oracle (LibRaw unprocessed_raw, black box). + +Input: a CRW file and the TIFF written by `unprocessed_raw.exe -T` for it. +Tests, per 03-payload-layout.md: + - oracle frame size == SensorInfo raw size; + - oracle code histogram (min/max/percentiles) -> bit depth; + - the low-bits region equals the 2 LSBs of the oracle's codes, row-major, 4 pixels/byte, + under one of the two packing orders (pixel0 in bits 7-6 ... pixel3 in bits 1-0, or the + reverse); reports exact matching byte counts for both orders and both candidate + positions of the region (EOS prefix [0:Q) / PowerShot [514:514+Q)). + +Statistics and equality counts only. Usage: + python -I p2_oracle_lsb.py file.crw file.crw.tiff +""" + +import struct +import sys + + +def records(buf, start, length, endian, depth=0): + if depth > 8 or length < 6: + return + (tbl,) = struct.unpack_from(endian + "I", buf, start + length - 4) + if tbl + 2 > length - 4: + return + (n,) = struct.unpack_from(endian + "H", buf, start + tbl) + if tbl + 2 + n * 10 > length: + return + for k in range(n): + tc, ln, off = struct.unpack_from(endian + "HII", buf, start + tbl + 2 + k * 10) + if (tc >> 14) & 3: + continue + if off + ln > length: + continue + if ((tc >> 11) & 7) in (5, 6): + yield from records(buf, start + off, ln, endian, depth + 1) + else: + yield (tc & 0x3FFF, start + off, ln) + + +def read_tiff_gray16(path): + """Minimal classic-TIFF reader: single image, 16-bit, strips, no compression.""" + buf = open(path, "rb").read() + endian = {b"II": "<", b"MM": ">"}[buf[0:2]] + (magic, ifdoff) = struct.unpack_from(endian + "HI", buf, 2) + assert magic == 42 + (n,) = struct.unpack_from(endian + "H", buf, ifdoff) + tags = {} + for i in range(n): + tag, typ, cnt, val = struct.unpack_from(endian + "HHII", buf, ifdoff + 2 + i * 12) + tags[tag] = (typ, cnt, val, ifdoff + 2 + i * 12 + 8) + def val(tag): + typ, cnt, v, pos = tags[tag] + size = {1: 1, 3: 2, 4: 4}[typ] + if size * cnt <= 4: + if typ == 3 and cnt == 1: + return [struct.unpack_from(endian + "H", buf, pos)[0]] + return [v] + fmt = {1: "B", 3: "H", 4: "I"}[typ] + return list(struct.unpack_from(endian + fmt * cnt, buf, v)) + w = val(256)[0] + h = val(257)[0] + bps = val(258)[0] + comp = val(259)[0] if 259 in tags else 1 + assert bps == 16 and comp == 1, (bps, comp) + offs = val(273) + counts = val(279) + data = b"".join(buf[o : o + c] for o, c in zip(offs, counts)) + assert len(data) >= w * h * 2, (len(data), w * h * 2) + import array + a = array.array("H") + a.frombytes(data[: w * h * 2]) + if (endian == ">") != (sys.byteorder == "big"): + a.byteswap() + return w, h, a + + +def main(): + crw_path, tiff_path = sys.argv[1], sys.argv[2] + buf = open(crw_path, "rb").read() + endian = {b"II": "<", b"MM": ">"}[buf[0:2]] + (hl,) = struct.unpack_from(endian + "I", buf, 2) + r = {} + for t, o, l in records(buf, hl, len(buf) - hl, endian): + r.setdefault(t, (o, l)) + ro, rl = r[0x2005] + so, sl = r[0x1031] + v = struct.unpack_from(endian + "H" * (sl // 2), buf, so) + w, h = v[1], v[2] + q = w * h // 4 + data = buf[ro : ro + rl] + + tw, th, pix = read_tiff_gray16(tiff_path) + print(f"CRW raw {w}x{h}; oracle TIFF {tw}x{th} -> {'MATCH' if (tw, th) == (w, h) else 'MISMATCH'}") + s = sorted(pix[::997]) + print(f"oracle codes: min {min(pix)}, max {max(pix)}, p1 {s[len(s)//100]}, " + f"median {s[len(s)//2]}, p99 {s[-len(s)//100]}") + + # build expected low-bits bytes from oracle LSBs, both packing orders + import array + lsb = array.array("B", (p & 3 for p in pix)) + exp_msb = bytearray(q) + exp_lsb = bytearray(q) + for i in range(q): + a4 = lsb[4 * i : 4 * i + 4] + exp_msb[i] = (a4[0] << 6) | (a4[1] << 4) | (a4[2] << 2) | a4[3] + exp_lsb[i] = a4[0] | (a4[1] << 2) | (a4[2] << 4) | (a4[3] << 6) + for region_name, reg in (("prefix [0:Q)", data[:q]), ("ps [514:514+Q)", data[514 : 514 + q])): + if len(reg) < q: + continue + m1 = sum(1 for a, b in zip(reg, exp_msb) if a == b) + m2 = sum(1 for a, b in zip(reg, exp_lsb) if a == b) + print(f"{region_name}: match MSB-first {m1}/{q} ({m1 / q:.6%}), " + f"LSB-first {m2}/{q} ({m2 / q:.6%})") + + +if __name__ == "__main__": + main() diff --git a/crates/crw/clean-room/sources.md b/crates/crw/clean-room/sources.md new file mode 100644 index 00000000..44933c95 --- /dev/null +++ b/crates/crw/clean-room/sources.md @@ -0,0 +1,134 @@ +# CRW clean-room derivation — source log + +Every URL opened and every web search run for this work, in order, with the time (local, +Europe-style 24 h), what was asked, and what was taken. If a result or page exposed details of an +existing decoder's algorithm that were not asked for, that is recorded too, with what it said. + +Rules in force (from the task): no raw-decoder source may be opened, fetched, searched inside or +requested — dcraw, LibRaw, rawspeed, rawloader/rawler, dnglab, darktable, RawTherapee, +libopenraw, ExifTool's Perl code — including licence headers, comments and mirrors. Allowed: +published format documentation in prose (ExifTool's CIFF/CRW format and tag-name pages), +published standards, and the sample files. The branch `crw-decode` (an earlier attempt) is not +checked out, diffed, logged or read. + +## Local files consulted (non-web) + +- 2026-10-12 ~19:2x — `CLAUDE.md` (project instructions; in session context from the start) and + `docs/arw6-compression.md` (the house model for deriving and documenting a format). Taken: + process shape (assumptions → spec + samples → Python reference model → freeze → black-box + verification), documentation conventions, corpus table format. +- 2026-10-12 ~19:2x — directory listing and SHA-256 of the maintainer's private + `../RawImages/*.crw` (18 files, Canon EOS D60 era, 5.5–8.6 MB). Taken: file names, sizes, + hashes only. No .THM files present. Contents not yet parsed. + +## Web + +- 2026-10-12 19:26 — **Fetch** `https://exiftool.org/TagNames/CanonRaw.html` (ExifTool's + CanonRaw tag-name page; generated documentation, not Perl source). Asked: CIFF structure, tag + IDs/names, compression info, spec links. Taken: tag IDs — 0x2005 RawData, 0x2007 JpgFromRaw, + 0x2008 ThumbnailImage, 0x1835 DecoderTable (subdir: DecoderTableNumber, CompressedDataOffset, + CompressedDataLength), 0x1031 SensorInfo (subdir), 0x1810 ImageInfo (ImageWidth, ImageHeight, + ComponentBitDepth, ColorBitDepth…), 0x1030 WhiteSample (BlackLevels at index 55), 0x10a9 + ColorBalance, 0x10ae ColorTemperature, 0x080a CanonRawMakeModel, 0x102a CanonShotInfo, 0x1834 + CanonModelID, 0x300a ImageProps, 0x2807 CameraObject, 0x300b ExifInformation, 0x0000 + NullRecord, 0x0001 FreeBytes, and others. No compression algorithm described. Links to + `canon_raw.html` for structure. **No decoder algorithm details shown.** +- 2026-10-12 19:27 — **Fetch** `https://exiftool.org/canon_raw.html` (Phil Harvey's prose "Canon + CRW file format" page). Taken: 26-byte header (ByteOrder `II`/`MM` @0, HeaderLength u32 @2 = + 0x1a, signature `HEAPCCDR` @6, CRWVersion 0x00010002 @14, 8 reserved bytes @18); block = + ValueData, DirCount u16, N×10-byte entries (tag u16, size u32, offset u32), optional + OtherData, u32 DirStart at BlockSize−4; offsets relative to the block's ValueData start; + values 2-byte aligned; tag masks DataLocation 0xc000 (0x0000 = ValueData, 0x4000 = in-entry, + 0x8000/0xc000 unknown), DataFormat 0x3800 (0x0000 bytes, 0x0800 ASCII string, 0x1000 u16s, + 0x1800 u32s/floats, 0x2000 structure, 0x2800/0x3000 subdirectory), TagIndex 0x07ff; + subdirectory = a complete block inside ValueData, its BlockSize = entry Size; root block from + HeaderLength to EOF. RawData 0x2005 noted as "the bulk of the CRW file"; JpgFromRaw 0x2007; + no compression description. Links the original Canon CIFF spec at `http://xyrion.org/ciff/` + (dead) and lclevy's CR2/CR3 pages. **No decoder algorithm details shown.** +- 2026-10-12 19:28 — **Search** (WebSearch, standard): `Canon CIFF specification 1.0 rev 4 PDF + "CIFF" camera image file format specification download`. Results included Wikipedia + "Camera Image File Format", a freedesktop libopenraw wiki page "Canon CRW" (NOT opened — it is + a decoder project's page; left for the maintainer to approve, see plan §7), an unpkg mirror of + exiftool's canon_raw.html, PRONOM. Taken: pointer to Wikipedia for an archived spec URL. + **No decoder algorithm details shown in the result snippets.** +- 2026-10-12 19:28 — **Fetch attempt** `http://xyrion.org/ciff/` — failed (site dead). +- 2026-10-12 19:29 — **Fetch** `https://en.wikipedia.org/wiki/Camera_Image_File_Format`. Taken: + spec part names/dates (Image Data File v1.0 r4, 1997-12-24; File/Directory organization v1.0 + r3, 1998-02-09), archive URL for the PDF, camera list (EOS D30/D60/10D/300D, PowerShot Pro1, + G1–G6, S30–S70). +- 2026-10-12 19:29 — **Download** (curl) + `https://web.archive.org/web/20081230095207/http://xyrion.org/ciff/CIFFspecV1R04.pdf` → + scratchpad `ciff-spec/CIFFspecV1R04.pdf`, 99 422 bytes, SHA-256 + `416d4e49ea3f6ad2ba5ccc76b48ca100605f4a6127816c145e6dd5d6c3b51cf4`, 34 pages. Text extracted + with a local stdlib-Python script (`pdftext.py`, kept in scratchpad; extraction output + `extracted.txt`). This is Canon's own published "CIFF Specification on Image Data File" + v1.0 r4 (1997-12-24); local copies (PDF + text extraction) kept in the gitignored + `plan/crw/` (Canon-copyrighted document, not committed; re-fetchable from the archive URL, + hash above). Taken: the heap file header struct (byteOrder, headerLength, type + "HEAP", subType "JPGM"/"CCDR"/"TIFP"/"ARCH", version 0x00010002), heap = heap space + + offsetTbl, offsetTblOffset as last u32 of the heap, DC_RECORD_ENTRY {typeCode u16, length + u32, offset u32}, numRecords u16, typeCode bit fields (bits 15-14 storage: 00 kStg_InHeapSpace, + 01 kStg_InRecordEntry, 10/11 reserved; bits 13-11 dataType: 000 byte, 001 ASCII, 010 word, + 011 dword, 100 structure, 101/110 heap record, 111 reserved; bits 10-0 IDCode), masks + kStgFormatMask 0xc000 / kDataTypeMask 0x3800 / kIDCodeMask 0x07ff, null/free records, 2-byte + alignment rule, ImageSpec/ImageFormat/CapturedTime/ModelName/… structs, appendix type-code + defines. The spec documents the container and common metadata only — **it does not describe + the CRW raw-data compression** (ImageFormat fileFormat value 2 is merely "Reserved (for + CanonCRW format)"). **No decoder algorithm details shown.** +- 2026-10-12 19:31 — **Fetch** (curl) `https://raw.pixls.us/` and + `https://raw.pixls.us/json/getrepository.php?set=all` (the site's own data endpoint found in + its HTML; first tried `json/data.json`, 404). Taken: the full sample table (2016 rows); + filtered locally for `.crw` (51 rows) and classic CIFF-era bodies; per-file licence, URL and + SHA-256 as published by the site. CC0 CIFF-era bodies: EOS D30, D60, 10D (×3), 300D (×2), + Digital Rebel, PowerShot G1, G5, G6. CC-BY-NC-SA only (NOT downloaded, per task rules): + PowerShot G2, G3, Pro1, Pro70, S30, S40, S45, S50, S60, S70, S2 IS and others. Many other + `.CRW` rows are CHDK sensor dumps from much later PowerShots (A-, SX-, ELPH-series), a + different format. **No decoder algorithm details shown.** +- 2026-10-12 19:32 — **Download** (Python urllib, stdlib) the 11 CC0 files above from + `https://raw.pixls.us/getfile.php/...` into the gitignored `corpus/crw/`; every file verified + against the site's published SHA-256 (all OK; table with hashes in `01-plan.md`). + +## Local analysis (scripts in this directory) + +- 2026-10-12 19:35–19:45 — `ciff_dump.py` (written here, stdlib only; container layout + implemented from the two documented sources above) run over the 11 public files + (`logs/structure-public.txt`) and the 18 private files + (`logs/private/structure-private.txt`, gitignored — may contain personal metadata). Observed + facts are summarised in `01-plan.md` §3. No raw-data decoding was attempted; statistics only. +- 2026-10-12 ~20:0x — P0/P1 local analysis (post-approval): `02-container.md` written from the + already-logged sources + corpus observations. `p1_payload.py`, `p1b_markers.py`, + `p1c_lowbits_bounds.py` run over all 29 files (public logs `logs/p1*`, private under + `logs/private/`). Established statistically (no decoding): the three payload layouts, the + 514-byte zero block at the compressed section start of every file, JPEG-style ff00 stuffing + of every entropy stream (zero exceptions), the PowerShot `ff d9` terminator, and the exact + low-bits region bounds on G5/G6. Documented in `03-payload-layout.md`. No external sources + consulted in this step. + +## Oracle (P2, from 2026-10-12) + +- The maintainer installed LibRaw 0.22.2 (official pre-built Windows binaries) in + a local `LibRaw-0.22.2\bin` folder. That directory tree also contains LibRaw's source + code; **no source file there is ever opened, read or searched** — only + `bin\unprocessed_raw.exe` is executed, as a black box, per the approved plan (§5a/§7). + All oracle runs and probe campaigns are logged in `04-derivation-log.md`. + +- 2026-10-13 — **Fetch** `https://raw.pixls.us/json/getrepository.php?set=all` (and the site + root to find that endpoint): recovered the stable `getfile.php` URLs of the 11 CC0 corpus + files by matching their SHA-256s, and re-confirmed each row's CC0 licence; URLs pinned in + `xtask/src/main.rs` `RAW_SAMPLES`. No decoder content on these pages. +- 2026-10-13 — **Fetch** `https://exiftool.org/TagNames/CanonRaw.html` (again) and + `https://exiftool.org/TagNames/Canon.html` (prose tag tables only): 0x10a9 ColorBalance → + Canon ColorBalance table (Index2; WB_RGGBLevels slot order Auto/Daylight/Shade/Cloudy/ + Tungsten/Fluorescent/Flash/Custom/Kelvin, index-29 "black levels for the D60" note), + 0x1030 WhiteSample field names, CanonShotInfo index 7 = WhiteBalance and the Canon + WhiteBalance value enumeration, CanonModelID values. Prose documentation, no code. + +- 2026-10-25 — **Fetch** `https://www.xrite.com/service-support/new_color_specifications_for_colorchecker_sg_and_classic_charts` + (prose page): ColorChecker Classic charts made after November 2014 use new reference data, + published as CGATS files. The CGATS download itself was refused to scripts (HTTP 403) and + not worked around. +- 2026-10-25 — **Fetch** `https://raw.githubusercontent.com/colour-science/colour/v0.4.2/colour/characterisation/datasets/colour_checkers/chromaticity_coordinates.py` + (colour-science, BSD-3-Clause; also its rendered docs page): the 24 post-November-2014 + ColorChecker Classic L*a*b* values (D50), credited there to X-Rite's release. Reference + data only, the chart's measured colours; no camera or decoder content. Used in + `10-chart-calibration.md`. diff --git a/crates/crw/clean-room/tools/gen_tables.py b/crates/crw/clean-room/tools/gen_tables.py new file mode 100644 index 00000000..666f0b9e --- /dev/null +++ b/crates/crw/clean-room/tools/gen_tables.py @@ -0,0 +1,52 @@ +"""Regenerate crates/crw/src/tables.rs from the frozen tables in crates/crw/clean-room/logs/table_sel{0,1,2}.json. + +Run from anywhere: `python crates/crw/clean-room/tools/gen_tables.py`. The JSON files are the +derivation's frozen data (06-crw-format.md section 3); this script only renders them as Rust +constants, keeping the file's doc comment. It was a local tool during the derivation in LightCraft +(plan/crw/model/gen_tables.py) and moved here so the tables stay reproducible. +""" + +import json +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[4] +TABLES = ROOT / "crates/crw/src/tables.rs" +LOGS = ROOT / "crates/crw/clean-room/logs" + +TAIL = """ +/// The tables for DecoderTable selector 0, 1 or 2. +pub(crate) fn tables(selector: u32) -> Option { + match selector { + 0 => Some(SelTables { first: FIRST_0, main: MAIN_0 }), + 1 => Some(SelTables { first: FIRST_1, main: MAIN_1 }), + 2 => Some(SelTables { first: FIRST_2, main: MAIN_2 }), + _ => None, + } +} +""" + + +def main(): + head = TABLES.read_text(encoding="utf-8").split("#[rustfmt::skip]")[0] + out = [head.rstrip("\n") + "\n"] + for sel in (0, 1, 2): + spec = json.loads((LOGS / f"table_sel{sel}.json").read_text(encoding="utf-8")) + for part, name in (("first", "FIRST"), ("main", "MAIN")): + rows = sorted(spec[part].items(), key=lambda kv: (len(kv[0]), kv[0])) + out.append("#[rustfmt::skip]") + out.append(f"const {name}_{sel}: &[(u16, u8, u8, u8)] = &[") + line = [] + for code, (r, s) in rows: + line.append(f"(0b{code}, {len(code)}, {r}, {s}),") + if len(line) == 6: + out.append(" " + " ".join(line)) + line = [] + if line: + out.append(" " + " ".join(line)) + out.append("];") + TABLES.write_text("\n".join(out) + TAIL, encoding="utf-8", newline="\n") + print(f"generated {TABLES.relative_to(ROOT)}") + + +if __name__ == "__main__": + main() diff --git a/crates/crw/src/lib.rs b/crates/crw/src/lib.rs new file mode 100644 index 00000000..a115d5e0 --- /dev/null +++ b/crates/crw/src/lib.rs @@ -0,0 +1,746 @@ +#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] +//! Clean-room Canon CRW (CIFF heap) raw decoder: the bodies before the CR2 era (EOS D30, D60, +//! 10D, 300D / Digital Rebel, PowerShot G-series). +//! +//! The container rules come from Canon's published CIFF specification and ExifTool's prose tag +//! documentation; the compression was derived from CC0 and maintainer-owned sample files by +//! black-box analysis against a reference decoder's command-line output, frozen in +//! `clean-room/06-crw-format.md` and verified sample-exact on 28 of 29 corpus files plus 1,516 +//! further EOS D60 files (`clean-room/07-verification.md`). The code tables are frozen data +//! (`tables`, generated from `clean-room/logs/table_sel{0,1,2}.json`). No other raw decoder's +//! source was consulted; the whole derivation record is in the crate's `clean-room/` folder. +//! +//! - [`probe`] recognises the CIFF signature. +//! - [`header`] reads geometry, bit depth, CFA phase, make and model, capture time, orientation +//! and as-shot white balance without touching the entropy-coded data. +//! - [`decode`] also decodes the sensor samples and measures the black level (masked borders) +//! and the white level (saturation plateau or the sensor's clip level). +//! +//! The decoder never panics on input: every offset read from the file is bounds-checked, the heap +//! walk is bounded by depth and by a per-file record budget, and the sample count is capped by +//! [`Limits`]. Malformed files return [`Error::Corrupt`]; unknown decoder tables and the +//! PowerShot G5 (whose entropy coding diverges from the derived model) return +//! [`Error::Unsupported`]. +//! +//! Known gaps (documented in the freeze): the PowerShot G5 is refused (its embedded JPEG is the +//! app's fallback); PowerShot bodies without CC0 samples (G2, G3, Pro1, S-series) decode through +//! the same model unverified; the D30 and the PowerShots carry no white-balance record the +//! derivation deciphered, so `wb_multipliers` is `None` for them. +//! +//! Colour (camera matrices, profiles) is out of scope: the file carries none, and the app owns +//! its colour model. + +mod tables; +#[cfg(test)] +mod tests; + +use std::collections::BTreeMap; + +/// Why a file could not be decoded. +#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] +pub enum Error { + /// The bytes do not start with a CIFF `HEAPCCDR` header. + #[error("not a Canon CRW (CIFF) file")] + NotCrw, + /// The file is a CRW but its structure or data is damaged. + #[error("CRW: {0}")] + Corrupt(String), + /// A CRW the derived model does not cover (an unknown decoder table, the PowerShot G5). + #[error("unsupported CRW: {0}")] + Unsupported(String), + /// The file declares more than [`Limits`] allow. + #[error("CRW exceeds a limit: {0}")] + LimitExceeded(&'static str), +} + +/// Caps on what a file may ask the decoder to allocate, checked against the header before +/// anything is allocated. The struct is `#[non_exhaustive]`: start from [`Limits::default`] and +/// set the fields you change. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[non_exhaustive] +pub struct Limits { + /// Most sensor samples (`width × height`) a file may declare. The decoder holds about three + /// `u16` buffers of this size at its peak. Default: 2³⁰ on 64-bit targets, 2²⁶ on 32-bit + /// ones (wasm32). Real CRW frames hold at most 7.4 million samples. + pub max_samples: usize, +} + +impl Default for Limits { + fn default() -> Self { + #[cfg(target_pointer_width = "64")] + let max_samples = 1 << 30; + #[cfg(not(target_pointer_width = "64"))] + let max_samples = 1 << 26; + Self { max_samples } + } +} + +/// The 2×2 Bayer phase at sensor position (0, 0). +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub enum Cfa { + /// Red, green / green, blue: every verified body except the PowerShot G1. + Rggb, + /// Green, blue / red, green: the PowerShot G1. + Gbrg, +} + +impl Cfa { + /// The conventional four-letter name. + pub fn name(self) -> &'static str { + match self { + Cfa::Rggb => "RGGB", + Cfa::Gbrg => "GBRG", + } + } + + /// The colour at each site of the 2×2 cell, row-major: 0 = red, 1 = green, 2 = blue. + pub fn pattern(self) -> [u8; 4] { + match self { + Cfa::Rggb => [0, 1, 1, 2], + Cfa::Gbrg => [1, 2, 0, 1], + } + } +} + +/// A rectangle in sensor pixels. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub struct Rect { + pub x: usize, + pub y: usize, + pub width: usize, + pub height: usize, +} + +impl Rect { + pub const fn new(x: usize, y: usize, width: usize, height: usize) -> Self { + Self { x, y, width, height } + } +} + +/// Black level per 2×2 CFA site, row-major, anchored at the active area's origin. +#[derive(Debug, Clone, Copy, PartialEq)] +pub struct BlackLevel { + pub values: [f32; 4], +} + +impl BlackLevel { + /// The same level at every site. + pub const fn uniform(v: f32) -> Self { + Self { values: [v; 4] } + } + + /// The level at sensor position (`x`, `y`), whose CFA site is counted from `active`'s origin. + pub fn at(&self, active: Rect, x: usize, y: usize) -> f32 { + let px = (x as isize - active.x as isize).rem_euclid(2) as usize; + let py = (y as isize - active.y as isize).rem_euclid(2) as usize; + self.values.get(py * 2 + px).copied().unwrap_or(self.values[0]) + } +} + +/// Everything a CRW tells about its image short of the samples: what [`header`] returns, and +/// what [`Image::header`] carries. +#[derive(Debug, Clone, PartialEq)] +pub struct Header { + /// Sensor width in samples (SensorInfo), masked borders included. + pub width: usize, + /// Sensor height in samples. + pub height: usize, + /// Sample depth: 12 when the file carries a low-bits plane, else 10. + pub bits: u32, + /// The sensor area that holds the image (SensorInfo's inclusive crop); the rest is masked. + pub active_area: Rect, + /// Bayer phase at sensor (0, 0). + pub cfa: Cfa, + /// The DecoderTable number (0, 1 or 2): which frozen code tables the stream uses. + pub selector: u32, + /// Camera make from the MakeModel record (`Canon`). + pub make: Option, + /// Camera model from the MakeModel record (`Canon EOS D60`). + pub model: Option, + /// Capture time as Unix seconds on the camera's local clock (the TimeStamp record); the + /// file carries no UTC offset. Convert with a calendar library (chrono), not by hand. + pub capture_unix: Option, + /// Exif orientation (1, 3, 6 or 8) from the ImageInfo rotation. + pub exif_orientation: u8, + /// As-shot white balance as (R, G, B) multipliers with G = 1, from the ColorBalance record + /// slot the ShotInfo white-balance setting selects; `None` on bodies without the record + /// (D30, PowerShots). + pub wb_multipliers: Option<[f32; 3]>, +} + +/// A decoded CRW. +#[derive(Debug, Clone, PartialEq)] +pub struct Image { + pub header: Header, + /// `width × height` sensor samples, row-major, `bits` deep (10 or 12), black not subtracted. + pub samples: Vec, + /// Black level measured from the masked borders. + pub black: BlackLevel, + /// White level: the saturation plateau, the sensor's clip level, or the full range. + pub white: f32, +} + +/// Result alias for this crate. +pub type Result = std::result::Result; + +fn corrupt(why: &str) -> Error { + Error::Corrupt(why.to_string()) +} + +/// Heap recursion bound: the deepest directory observed in real files is 4. +const MAX_DEPTH: usize = 16; + +/// Records visited per file, across every heap level: real files hold a few dozen. Depth alone +/// doesn't bound the walk — a sub-heap record may point back at its own parent block, and N such +/// records per block multiply into N^MAX_DEPTH visits — so this shared budget does. +const MAX_RECORDS: usize = 4096; + +#[derive(Clone, Copy, PartialEq, Eq)] +enum Order { + Le, + Be, +} + +fn u16_at(b: &[u8], at: usize, o: Order) -> Option { + let s: [u8; 2] = b.get(at..at.checked_add(2)?)?.try_into().ok()?; + Some(match o { + Order::Le => u16::from_le_bytes(s), + Order::Be => u16::from_be_bytes(s), + }) +} + +fn u32_at(b: &[u8], at: usize, o: Order) -> Option { + let s: [u8; 4] = b.get(at..at.checked_add(4)?)?.try_into().ok()?; + Some(match o { + Order::Le => u32::from_le_bytes(s), + Order::Be => u32::from_be_bytes(s), + }) +} + +/// Walk a heap block (`start..start + len` of `bytes`) depth-first in record-table order, +/// recording each in-heap record's tag (typeCode & 0x3fff), absolute offset and length — first +/// occurrence wins. In-record values and reserved storage classes are skipped; dataType 5/6 +/// records are sub-heaps. `budget` is the number of records the whole file may still visit; +/// the walk stops wherever it runs out. +fn walk(bytes: &[u8], start: usize, len: usize, o: Order, depth: usize, budget: &mut usize, out: &mut BTreeMap) { + if depth > MAX_DEPTH || len < 6 { + return; + } + let Some(tbl) = start.checked_add(len).and_then(|e| e.checked_sub(4)).and_then(|at| u32_at(bytes, at, o)) else { return }; + let tbl = tbl as usize; + // the offset table must fit between the block start and its own trailing pointer + if tbl.checked_add(6).is_none_or(|e| e > len) { + return; + } + let Some(abs_tbl) = start.checked_add(tbl) else { return }; + let Some(n) = u16_at(bytes, abs_tbl, o) else { return }; + let Some(records_end) = (n as usize).checked_mul(10).and_then(|r| tbl.checked_add(2)?.checked_add(r)) else { return }; + if records_end > len { + return; + } + for k in 0..n as usize { + let Some(left) = budget.checked_sub(1) else { return }; + *budget = left; + let at = abs_tbl + 2 + k * 10; + let (Some(tc), Some(rl), Some(ro)) = (u16_at(bytes, at, o), u32_at(bytes, at + 2, o), u32_at(bytes, at + 6, o)) else { return }; + if tc & 0xc000 != 0 { + continue; // in-record value or reserved storage: nothing we need + } + let (rl, ro) = (rl as usize, ro as usize); + if ro.checked_add(rl).is_none_or(|e| e > len) { + continue; + } + let Some(abs) = start.checked_add(ro) else { continue }; + if matches!((tc >> 11) & 7, 5 | 6) { + walk(bytes, abs, rl, o, depth + 1, budget, out); + } else { + out.entry(tc & 0x3fff).or_insert((abs, rl)); + } + } +} + +/// CIFF records the decoder consumes. +const RAW_DATA: u16 = 0x2005; +const SENSOR_INFO: u16 = 0x1031; +const DECODER_TABLE: u16 = 0x1835; +const MAKE_MODEL: u16 = 0x080a; +const IMAGE_INFO: u16 = 0x1810; +const TIME_STAMP: u16 = 0x180e; +const COLOR_BALANCE: u16 = 0x10a9; +const SHOT_INFO: u16 = 0x102a; + +/// Whether `bytes` start with a CIFF raw header: `II` or `MM`, then `HEAPCCDR` at offset 6. +pub fn probe(bytes: &[u8]) -> bool { + matches!(bytes.get(0..2), Some(b"II") | Some(b"MM")) && bytes.get(6..14) == Some(b"HEAPCCDR") +} + +/// Read the header: geometry, bit depth, CFA phase, metadata and white balance, without +/// decoding the samples. Fails the same way [`decode`] would on the records it needs. +pub fn header(bytes: &[u8], limits: &Limits) -> Result
{ + Ok(parse(bytes, limits, false)?.header) +} + +/// Decode a CRW: the header plus every sensor sample, the black level and the white level. +pub fn decode(bytes: &[u8], limits: &Limits) -> Result { + parse(bytes, limits, true) +} + +fn parse(bytes: &[u8], limits: &Limits, full: bool) -> Result { + let o = match bytes.get(0..2) { + Some(b"II") => Order::Le, + Some(b"MM") => Order::Be, + _ => return Err(Error::NotCrw), + }; + if bytes.get(6..14) != Some(b"HEAPCCDR") { + return Err(Error::NotCrw); + } + let hl = u32_at(bytes, 2, o).ok_or_else(|| corrupt("truncated header"))? as usize; + let heap_len = bytes.len().checked_sub(hl).filter(|&l| l >= 6).ok_or_else(|| corrupt("header length beyond the file"))?; + let mut rec = BTreeMap::new(); + let mut budget = MAX_RECORDS; + walk(bytes, hl, heap_len, o, 0, &mut budget, &mut rec); + + let &(so, sl) = rec.get(&SENSOR_INFO).ok_or_else(|| corrupt("no SensorInfo record"))?; + let sensor: Vec = (0..sl / 2).map_while(|i| u16_at(bytes, so + i * 2, o)).collect(); + let (w, h) = match (sensor.get(1), sensor.get(2)) { + (Some(&w), Some(&h)) => (w as usize, h as usize), + _ => return Err(corrupt("SensorInfo too short")), + }; + let n = w.checked_mul(h).filter(|&n| n > 0 && n <= limits.max_samples).ok_or(Error::LimitExceeded("image too large"))?; + + let &(dto, dtl) = rec.get(&DECODER_TABLE).ok_or_else(|| corrupt("no DecoderTable record"))?; + if dtl < 16 { + return Err(corrupt("DecoderTable too short")); + } + let selector = u32_at(bytes, dto, o).ok_or_else(|| corrupt("truncated DecoderTable"))?; + let ent_off = u32_at(bytes, dto + 8, o).ok_or_else(|| corrupt("truncated DecoderTable"))? as usize; + let tables = tables::tables(selector).ok_or_else(|| Error::Unsupported(format!("decoder table {selector}")))?; + if ent_off > bytes.len() { + return Err(corrupt("entropy offset beyond the file")); + } + + // "Canon\0\0" + let (make, model) = rec + .get(&MAKE_MODEL) + .and_then(|&(mo, ml)| bytes.get(mo..mo.checked_add(ml)?)) + .map(|v| { + let mut parts = v.split(|&b| b == 0).map(|s| String::from_utf8_lossy(s).trim().to_string()); + (parts.next().unwrap_or_default(), parts.next().unwrap_or_default()) + }) + .unwrap_or_default(); + if model.contains("PowerShot G5") { + return Err(Error::Unsupported( + "Canon PowerShot G5 entropy coding (a documented anomaly of the derivation, clean-room/06-crw-format.md §4); use the embedded JPEG" + .into(), + )); + } + + // active area from the SensorInfo crop (inclusive left, top, right, bottom) + let active = match (sensor.get(5), sensor.get(6), sensor.get(7), sensor.get(8)) { + (Some(&l), Some(&t), Some(&r), Some(&b)) + if (l as usize) < (r as usize) && (r as usize) < w && (t as usize) < (b as usize) && (b as usize) < h => + { + Rect::new(l as usize, t as usize, r as usize - l as usize + 1, b as usize - t as usize + 1) + } + _ => Rect::new(0, 0, w, h), + }; + + let &(ro, rl) = rec.get(&RAW_DATA).ok_or_else(|| corrupt("no raw data record"))?; + let payload = + bytes.get(ro..ro.checked_add(rl).ok_or_else(|| corrupt("raw record size overflow"))?).ok_or_else(|| corrupt("raw data outside the file"))?; + + // Low-bits plane discriminator, two signs of a plane: (a) its raw bytes may hold an 0xff not + // followed by 0x00, which the stuffed entropy stream never does; (b) the zero header of + // `ent_off` bytes sits right after it, where a file without a plane is deep in entropy data. + // (b) catches the D60 files whose lowest bit is almost always 0, so no plane byte is 0xff. + let q = n.div_ceil(4); + let span = q.saturating_add(ent_off); + let head = payload.get(..span.min(payload.len())).unwrap_or(payload); + let header_after_plane = ent_off >= 64 && payload.len() > span && payload.get(q..span).is_some_and(|z| z.iter().all(|&b| b == 0)); + let has_low = header_after_plane || head.windows(2).any(|p| p[0] == 0xff && p[1] != 0x00); + let bits = if has_low { 12 } else { 10 }; + let low_len = if has_low { q } else { 0 }; + + let samples = if full { + let entropy = payload.get(low_len.saturating_add(ent_off)..).ok_or_else(|| corrupt("payload shorter than its header"))?; + let low = has_low.then(|| payload.get(..q)).flatten(); + if has_low && low.is_none() { + return Err(corrupt("payload shorter than its low-bits plane")); + } + let hi = decode_entropy(entropy, w, h, &tables)?; + let mut samples = vec![0u16; n]; + for (i, (dst, &hi)) in samples.iter_mut().zip(hi.iter()).enumerate() { + *dst = match low { + Some(low) => { + let lb = low.get(i >> 2).map_or(0, |&b| (b >> ((i & 3) * 2)) & 3); + ((((hi as u32) << 2) | lb as u32) & 0xffff) as u16 + } + None => hi, + }; + } + samples + } else { + Vec::new() + }; + + // CFA phase at sensor (0, 0): RGGB on every verified body except the PowerShot G1 (GBRG), + // established by chroma correlation against each body's embedded JPEG (derivation log). + let cfa = if model.contains("PowerShot G1") { Cfa::Gbrg } else { Cfa::Rggb }; + + let black = if full { black_from_borders(&samples, w, h, active) } else { BlackLevel::uniform(0.0) }; + let white = if full { white_level(&samples, bits) } else { ((1u32 << bits) - 1) as f32 }; + + // ImageInfo rotation: degrees; 270 observed on portrait files whose scene is upright after a + // 90° counter-clockwise turn (Exif 8), so 90 is its mirror (Exif 6) + let exif_orientation = match rec.get(&IMAGE_INFO).and_then(|&(io, il)| (il >= 16).then(|| u32_at(bytes, io + 12, o)).flatten()) { + Some(90) => 6, + Some(180) => 3, + Some(270) => 8, + _ => 1, + }; + + let capture_unix = rec.get(&TIME_STAMP).and_then(|&(to, tl)| (tl >= 4).then(|| u32_at(bytes, to, o)).flatten()).map(i64::from); + + let header = Header { + width: w, + height: h, + bits, + active_area: active, + cfa, + selector, + make: (!make.is_empty()).then_some(make), + model: (!model.is_empty()).then_some(model), + capture_unix, + exif_orientation, + wb_multipliers: wb_from_color_balance(bytes, &rec, o), + }; + Ok(Image { header, samples, black, white }) +} + +/// As-shot white-balance multipliers from the ColorBalance record (0x10a9, D60/10D/300D): a +/// byte-length word, then (R, G, G, B) level quadruples per white-balance setting, the last +/// quadruple being black levels (ExifTool's prose Canon/CanonRaw tag documentation names the +/// slots; the 10D/300D order is Auto, Daylight, Shade, Cloudy, Tungsten, Fluorescent, Flash, +/// Custom, Kelvin, and the D60's seven slots lack Shade and Kelvin). The slot is chosen by the +/// ShotInfo (0x102a) WhiteBalance value; multipliers are `level / green`, confirmed against the +/// cameras' own JPEG renderings (derivation log). Bodies without the record (D30, PowerShots) +/// return `None`. +fn wb_from_color_balance(bytes: &[u8], rec: &BTreeMap, o: Order) -> Option<[f32; 3]> { + let &(bo, bl) = rec.get(&COLOR_BALANCE)?; + let vals: Vec = (0..bl / 2).map_while(|i| u16_at(bytes, bo + i * 2, o)).collect(); + // quadruples after the length word; the last one is black levels + let slots = vals.len().checked_sub(1)? / 4; + let usable = slots.checked_sub(1)?; + if usable == 0 { + return None; + } + let setting = rec.get(&SHOT_INFO).and_then(|&(so, sl)| (sl / 2 > 7).then(|| u16_at(bytes, so + 14, o)).flatten()).unwrap_or(0); + // Canon WhiteBalance values: 0 Auto, 1 Daylight, 2 Cloudy, 3 Tungsten, 4 Fluorescent, + // 5 Flash, 6 Custom, 8 Shade, 9 Kelvin + let slot = if usable >= 9 { + match setting { + 1 => 1, + 8 => 2, + 2 => 3, + 3 => 4, + 4 => 5, + 5 => 6, + 6 => 7, + 9 => 8, + _ => 0, + } + } else { + match setting { + s @ 1..=6 if (s as usize) < usable => s as usize, + _ => 0, + } + }; + let q = vals.get(1 + slot.min(usable - 1) * 4..5 + slot.min(usable - 1) * 4)?; + let g = (q.get(1).copied()? as f32 + q.get(2).copied()? as f32) / 2.0; + let (r, b) = (q.first().copied()? as f32, q.get(3).copied()? as f32); + if g <= 0.0 || r <= 0.0 || b <= 0.0 { + return None; + } + let (mr, mb) = (r / g, b / g); + ((0.2..=8.0).contains(&mr) && (0.2..=8.0).contains(&mb)).then_some([mr, 1.0, mb]) +} + +/// Fewest samples at the top of the data that count as the sensor's clip level (hot pixels are +/// fewer). +const CLIP_SAMPLES: usize = 64; + +/// White level: the saturation plateau when [`saturation_plateau`] finds one, else the top of +/// the data when it sits just under the full range. +/// +/// These bodies clip below the full range at low ISO: on 1,409 D60 files at ISO 100–400 no +/// sample exceeds 4009 (most stop at 4000–4008), while ISO 800–1000 files reach 4095; the 10D +/// and 300D samples stop at 4003–4008 too. A photo with only a small blown area has no plateau +/// for [`saturation_plateau`], which then answered 4095 (373 of 925 ISO 100 files, 283 of 287 at +/// ISO 400): the blown pixels sat at 0.977 of white, under the clip threshold, and kept a +/// colour after white balance. So when at least [`CLIP_SAMPLES`] samples reach within 3 % of the +/// full range, the level they reach is the white level, lowered by the same 1/256 band the +/// plateau estimate uses. +fn white_level(data: &[u16], bits: u32) -> f32 { + let full = (1u32 << bits.clamp(1, 16)) - 1; + let plateau = saturation_plateau(data, bits); + if plateau < full as f32 { + return plateau; + } + let mut hist = vec![0usize; full as usize + 1]; + for &v in data { + if let Some(h) = hist.get_mut(v as usize) { + *h += 1; + } + } + let mut above = 0usize; + let top = hist.iter().enumerate().rev().find_map(|(v, &n)| { + above += n; + (above >= CLIP_SAMPLES).then_some(v as u32) + }); + match top { + Some(top) if top < full && top as f32 >= 0.97 * full as f32 => (top - (top / 256).max(1)) as f32, + _ => full as f32, + } +} + +/// White level estimate shared by the Craft raw decoders (LightCraft's `white_from_data`): the +/// saturation plateau if a noticeable number of samples sit at the maximum value, otherwise the +/// full `bits` range. A few stray samples above the plateau (hot pixels) don't hide it. +fn saturation_plateau(data: &[u16], bits: u32) -> f32 { + let full_level = (1u32 << bits.clamp(1, 16)) - 1; + let full = full_level as f32; + let step = (data.len() / 2_000_000).max(1); + // samples above the nominal range are padding / invalid, not saturation + let mut hist = vec![0usize; full_level as usize + 1]; + for &v in data.iter().step_by(step) { + if let Some(h) = hist.get_mut(v as usize) { + *h += 1; + } + } + let min_plateau = (data.len() / 20_000).max(16); + // the top of the data, skipping up to half a plateau's worth of strays so a real plateau can't be skipped + let strays = min_plateau / 2; + let mut above = 0usize; + let Some(mx) = hist.iter().enumerate().rev().find_map(|(v, &n)| { + above += n * step; + (above > strays).then_some(v as u32) + }) else { + return full; + }; + let band = (mx / 256).max(1); + let near = mx.saturating_sub(band); + let below = near.saturating_sub(band); + let count = |from: u32, to: u32| hist.get(from as usize..to as usize).map_or(0, |h| h.iter().sum::()); + // the strays count as saturated too + let top = count(near, full_level + 1); + let under = count(below, near); + // a saturation plateau is a spike: many more samples in the top band than in the band just below it + if mx as f32 > full * 0.5 && top * step >= min_plateau && top >= 4 * under + 4 { + // plateau: use its lower edge so everything at saturation maps to ≥ 1.0 + near as f32 + } else { + full + } +} + +/// Black level from the masked sensor borders: the darkest flat strip among 8-column groups of +/// the left border and 4-row groups of the top border, with the strips within 8 DN of it averaged +/// in. A strip whose mean is ≤ 1 is clamped padding (observed on the D60's first rows), not a +/// black reference, and bright transition strips next to the active area only raise means, so the +/// minimum stays on true dark pixels. +fn black_from_borders(data: &[u16], w: usize, h: usize, active: Rect) -> BlackLevel { + let mean_of = |xs: std::ops::Range, ys: std::ops::Range| -> Option { + let (mut sum, mut cnt) = (0f64, 0u64); + for y in ys { + let row = data.get(y.checked_mul(w)?..y.checked_mul(w)?.checked_add(w)?)?; + for &v in row.get(xs.clone())? { + sum += v as f64; + cnt += 1; + } + } + (cnt > 0).then(|| sum / cnt as f64) + }; + let mut strips: Vec<(f64, std::ops::Range, std::ops::Range)> = Vec::new(); + let rows = active.y..active.y.saturating_add(active.height).min(h); + let mut x = 2usize; + while x + 8 <= active.x { + if let Some(m) = mean_of(x..x + 8, rows.clone()) { + strips.push((m, x..x + 8, rows.clone())); + } + x += 8; + } + let cols = active.x..active.x.saturating_add(active.width).min(w); + let mut y = 2usize; + while y + 4 <= active.y { + if let Some(m) = mean_of(cols.clone(), y..y + 4) { + strips.push((m, cols.clone(), y..y + 4)); + } + y += 4; + } + let Some(darkest) = strips.iter().map(|s| s.0).filter(|&m| m > 1.0).reduce(f64::min) else { + return BlackLevel::uniform(0.0); + }; + // per-CFA-site means over every strip near the darkest + let (mut sum, mut cnt) = ([0f64; 4], [0u64; 4]); + for (m, xs, ys) in &strips { + // clamped padding (mean ≤ 1) is no black reference: counting it dragged the D60's level + // from 128 down to 117 and lifted every shadow + if *m <= 1.0 || *m > darkest + 8.0 { + continue; + } + for y in ys.clone() { + let py = (y as isize - active.y as isize).rem_euclid(2) as usize; + for x in xs.clone() { + let px = (x as isize - active.x as isize).rem_euclid(2) as usize; + if let Some(&v) = data.get(y * w + x) { + sum[py * 2 + px] += v as f64; + cnt[py * 2 + px] += 1; + } + } + } + } + if cnt.contains(&0) { + return BlackLevel::uniform(darkest as f32); + } + let mut values = [0f32; 4]; + for (v, (s, c)) in values.iter_mut().zip(sum.iter().zip(&cnt)) { + *v = (s / *c as f64) as f32; + } + BlackLevel { values } +} + +/// A code-table lookup built for 16-bit MSB-first peeks: entry = `len << 16 | run << 8 | size`, +/// zero where no code matches. +fn build_lut(codes: &[(u16, u8, u8, u8)]) -> Vec { + let mut lut = vec![0u32; 1 << 16]; + for &(code, len, r, s) in codes { + if len == 0 || len > 16 { + continue; + } + let span = 1usize << (16 - len); + let base = (code as usize) << (16 - len); + for e in lut.iter_mut().skip(base).take(span) { + *e = (len as u32) << 16 | (r as u32) << 8 | s as u32; + } + } + lut +} + +/// Decode the ff00-stuffed entropy stream into the `w × h` high-plane samples, per the frozen +/// model (`clean-room/06-crw-format.md` §2): 64-sample blocks, first/main code tables, +/// (run, size) leaves with JPEG-rule magnitudes, EOB, block-DC carry, and two parity DPCM chains +/// initialised to 512 and reset at every row start. A stream that ends early leaves the remaining +/// samples at their chain values (the frozen stream-end rule). +fn decode_entropy(entropy: &[u8], w: usize, h: usize, tables: &tables::SelTables) -> Result> { + // unstuff: an 0x00 after an 0xff is a stuffing byte + let mut d = Vec::with_capacity(entropy.len()); + let mut i = 0; + while let Some(&b) = entropy.get(i) { + d.push(b); + i += if b == 0xff && entropy.get(i + 1) == Some(&0x00) { 2 } else { 1 }; + } + let nbits = d.len().saturating_mul(8); + let peek16 = |pos: usize| -> u16 { + let (byte, sh) = (pos >> 3, pos & 7); + let b = |k: usize| d.get(byte + k).copied().unwrap_or(0) as u32; + let w24 = b(0) << 16 | b(1) << 8 | b(2); + ((w24 >> (8 - sh)) & 0xffff) as u16 + }; + + let luts = [build_lut(tables.first), build_lut(tables.main)]; + let n = w.checked_mul(h).ok_or(Error::LimitExceeded("image too large"))?; + let mut dec = vec![0u16; n]; + let (mut ce, mut co): (i32, i32) = (512, 512); + let mut carry: i32 = 0; + let mut pos = 0usize; + let mut qi = 0usize; + // emit one diff: row-start chain reset, parity chain update, 16-bit masked store + macro_rules! emit { + ($d:expr) => {{ + if qi % w == 0 { + ce = 512; + co = 512; + } + let c = if qi % 2 == 0 { &mut ce } else { &mut co }; + *c = c.wrapping_add($d); + if let Some(p) = dec.get_mut(qi) { + *p = (*c & 0xffff) as u16; + } + qi += 1; + }}; + } + 'frame: while qi < n { + let block_end = n.min((qi / 64 + 1) * 64); + let mut tok = 0u32; + while qi < block_end { + if pos >= nbits.saturating_sub(1) { + // stream exhausted: every remaining sample keeps its chain value + while qi < n { + emit!(0); + } + break 'frame; + } + let e = luts[usize::from(tok > 0)].get(peek16(pos) as usize).copied().unwrap_or(0); + if e == 0 { + return Err(corrupt("undecodable entropy code")); + } + let (len, rr, s) = ((e >> 16) as usize, ((e >> 8) & 0xff) as usize, e & 0xff); + if pos.saturating_add(len) > nbits { + while qi < n { + emit!(0); + } + break 'frame; + } + pos += len; + let mut rr = rr; + if s == 0 && rr == 0 && tok > 0 { + // end of block + while qi < block_end { + emit!(0); + } + tok += 1; + continue; + } + let diff: i32 = if s > 0 { + if pos.saturating_add(s as usize) > nbits { + while qi < n { + emit!(0); + } + break 'frame; + } + let m = (peek16(pos) >> (16 - s)) as i32; + pos += s as usize; + // JPEG magnitude rule + if m >= 1 << (s - 1) { m } else { m - (1 << s) + 1 } + } else { + 0 + }; + let mut first = qi.is_multiple_of(64); + if first && rr > 0 { + // the block's first sample covered by the skip: its diff is exactly the carry + emit!(carry); + rr -= 1; + first = false; + } + for _ in 0..rr { + if qi >= block_end { + break; + } + emit!(0); + } + if qi >= block_end { + tok += 1; + continue; + } + let dd = if first { + carry = carry.wrapping_add(diff); + carry + } else { + diff + }; + emit!(dd); + tok += 1; + } + } + Ok(dec) +} diff --git a/crates/crw/src/tables.rs b/crates/crw/src/tables.rs new file mode 100644 index 00000000..60ebb9f9 --- /dev/null +++ b/crates/crw/src/tables.rs @@ -0,0 +1,128 @@ +//! Canon CRW entropy code tables, frozen clean-room data. +//! +//! Generated by `clean-room/tools/gen_tables.py` from `clean-room/logs/table_sel{0,1,2}.json` (the +//! tables extracted by known-plaintext alignment and frozen at the CRW-FREEZE commit; derivation: +//! `clean-room/06-crw-format.md` §3). Do not edit by hand; regenerate from the JSON. +//! Each entry is `(code bits, code length, run, size)`; codes are MSB-first. + +/// One selector's pair of canonical Huffman tables: the code starting each 64-sample +/// block reads from `first`, every other code from `main`. +pub(crate) struct SelTables { + pub first: &'static [(u16, u8, u8, u8)], + pub main: &'static [(u16, u8, u8, u8)], +} + +#[rustfmt::skip] +const FIRST_0: &[(u16, u8, u8, u8)] = &[ + (0b00, 2, 0, 4), (0b010, 3, 0, 3), (0b011, 3, 0, 5), (0b100, 3, 0, 6), (0b101, 3, 0, 2), (0b1100, 4, 0, 7), + (0b1101, 4, 0, 1), (0b11100, 5, 0, 8), (0b11101, 5, 0, 9), (0b11110, 5, 0, 0), (0b111110, 6, 0, 10), (0b1111110, 7, 0, 11), +]; +#[rustfmt::skip] +const MAIN_0: &[(u16, u8, u8, u8)] = &[ + (0b00, 2, 0, 3), (0b01, 2, 0, 4), (0b100, 3, 0, 2), (0b101, 3, 0, 5), (0b1100, 4, 0, 1), (0b1101, 4, 0, 6), + (0b11100, 5, 0, 7), (0b111010, 6, 0, 8), (0b111011, 6, 1, 2), (0b111100, 6, 1, 3), (0b111101, 6, 1, 1), (0b1111100, 7, 1, 4), + (0b1111101, 7, 0, 9), (0b11111100, 8, 1, 5), (0b111111010, 9, 2, 2), (0b111111011, 9, 0, 0), (0b1111111000, 10, 2, 1), (0b1111111001, 10, 1, 6), + (0b1111111010, 10, 0, 10), (0b1111111011, 10, 15, 0), (0b1111111100, 10, 2, 3), (0b11111111010, 11, 1, 7), (0b111111110110, 12, 2, 4), (0b1111111101110000, 16, 3, 1), + (0b1111111101110001, 16, 3, 2), (0b1111111101110010, 16, 1, 8), (0b1111111101110011, 16, 1, 9), (0b1111111101110100, 16, 3, 3), (0b1111111101110101, 16, 2, 5), (0b1111111101110110, 16, 4, 1), + (0b1111111101110111, 16, 3, 4), (0b1111111101111000, 16, 4, 2), (0b1111111101111001, 16, 3, 5), (0b1111111101111010, 16, 5, 1), (0b1111111101111011, 16, 3, 6), (0b1111111101111100, 16, 3, 7), + (0b1111111101111101, 16, 3, 8), (0b1111111101111110, 16, 2, 9), (0b1111111101111111, 16, 7, 9), (0b1111111110000000, 16, 2, 6), (0b1111111110000001, 16, 1, 10), (0b1111111110000010, 16, 3, 9), + (0b1111111110000011, 16, 5, 6), (0b1111111110000100, 16, 5, 7), (0b1111111110000101, 16, 2, 8), (0b1111111110000110, 16, 2, 7), (0b1111111110000111, 16, 5, 2), (0b1111111110001000, 16, 5, 5), + (0b1111111110001001, 16, 5, 8), (0b1111111110001010, 16, 4, 3), (0b1111111110001011, 16, 7, 6), (0b1111111110001100, 16, 5, 9), (0b1111111110001101, 16, 7, 7), (0b1111111110001110, 16, 5, 4), + (0b1111111110001111, 16, 6, 1), (0b1111111110010000, 16, 15, 9), (0b1111111110010001, 16, 7, 1), (0b1111111110010010, 16, 7, 8), (0b1111111110010011, 16, 7, 5), (0b1111111110010100, 16, 9, 6), + (0b1111111110010101, 16, 9, 7), (0b1111111110010110, 16, 4, 9), (0b1111111110010111, 16, 11, 7), (0b1111111110011000, 16, 5, 3), (0b1111111110011001, 16, 13, 7), (0b1111111110011010, 16, 7, 4), + (0b1111111110011011, 16, 11, 6), (0b1111111110011100, 16, 9, 8), (0b1111111110011101, 16, 4, 7), (0b1111111110011110, 16, 4, 8), (0b1111111110011111, 16, 9, 5), (0b1111111110100000, 16, 6, 9), + (0b1111111110100001, 16, 9, 9), (0b1111111110100010, 16, 9, 1), (0b1111111110100011, 16, 15, 10), (0b1111111110100100, 16, 11, 8), (0b1111111110100101, 16, 6, 8), (0b1111111110100110, 16, 11, 5), + (0b1111111110100111, 16, 11, 9), (0b1111111110101000, 16, 13, 6), (0b1111111110101001, 16, 15, 7), (0b1111111110101010, 16, 13, 8), (0b1111111110101011, 16, 6, 7), (0b1111111110101100, 16, 4, 6), + (0b1111111110101101, 16, 4, 5), (0b1111111110101110, 16, 9, 4), (0b1111111110101111, 16, 8, 9), (0b1111111110110000, 16, 15, 8), (0b1111111110110001, 16, 8, 1), (0b1111111110110010, 16, 13, 5), + (0b1111111110110011, 16, 15, 6), (0b1111111110110100, 16, 11, 4), (0b1111111110110101, 16, 8, 8), (0b1111111110110110, 16, 11, 1), (0b1111111110110111, 16, 2, 10), (0b1111111110111000, 16, 4, 4), + (0b1111111110111001, 16, 7, 2), (0b1111111110111010, 16, 13, 9), (0b1111111110111011, 16, 8, 7), (0b1111111110111100, 16, 6, 6), (0b1111111110111101, 16, 13, 4), (0b1111111110111110, 16, 15, 5), + (0b1111111110111111, 16, 3, 10), (0b1111111111000000, 16, 10, 7), (0b1111111111000001, 16, 7, 3), (0b1111111111000010, 16, 10, 9), (0b1111111111000011, 16, 10, 8), (0b1111111111000100, 16, 8, 6), + (0b1111111111000101, 16, 6, 2), (0b1111111111000110, 16, 12, 7), (0b1111111111000111, 16, 6, 5), (0b1111111111001000, 16, 12, 8), (0b1111111111001001, 16, 12, 9), (0b1111111111001010, 16, 10, 1), + (0b1111111111001011, 16, 15, 4), (0b1111111111001100, 16, 13, 1), (0b1111111111001101, 16, 14, 9), (0b1111111111001110, 16, 5, 10), (0b1111111111001111, 16, 9, 2), (0b1111111111010000, 16, 8, 5), + (0b1111111111010001, 16, 10, 6), (0b1111111111010010, 16, 14, 7), (0b1111111111010011, 16, 9, 3), (0b1111111111010100, 16, 14, 8), (0b1111111111010101, 16, 12, 1), (0b1111111111010110, 16, 12, 6), + (0b1111111111010111, 16, 7, 10), (0b1111111111011000, 16, 6, 4), (0b1111111111011001, 16, 14, 1), (0b1111111111011010, 16, 4, 10), (0b1111111111011011, 16, 6, 10), (0b1111111111011100, 16, 14, 6), + (0b1111111111011101, 16, 11, 3), (0b1111111111011110, 16, 15, 1), (0b1111111111011111, 16, 13, 3), (0b1111111111100000, 16, 10, 5), (0b1111111111100001, 16, 8, 10), (0b1111111111100010, 16, 11, 2), + (0b1111111111100011, 16, 9, 10), (0b1111111111100100, 16, 11, 10), (0b1111111111100101, 16, 8, 4), (0b1111111111100110, 16, 10, 4), (0b1111111111100111, 16, 6, 3), (0b1111111111101000, 16, 14, 5), + (0b1111111111101001, 16, 12, 5), (0b1111111111101010, 16, 15, 3), (0b1111111111101011, 16, 13, 2), (0b1111111111101100, 16, 12, 4), (0b1111111111101101, 16, 8, 2), (0b1111111111101110, 16, 10, 10), + (0b1111111111101111, 16, 13, 10), (0b1111111111110000, 16, 14, 4), (0b1111111111110001, 16, 15, 2), (0b1111111111110010, 16, 12, 10), (0b1111111111110011, 16, 8, 3), (0b1111111111110100, 16, 10, 3), + (0b1111111111110101, 16, 10, 2), (0b1111111111110110, 16, 12, 3), (0b1111111111110111, 16, 14, 10), (0b1111111111111000, 16, 12, 2), (0b1111111111111001, 16, 14, 2), (0b1111111111111010, 16, 14, 3), +]; +#[rustfmt::skip] +const FIRST_1: &[(u16, u8, u8, u8)] = &[ + (0b00, 2, 0, 3), (0b01, 2, 0, 2), (0b100, 3, 0, 4), (0b101, 3, 0, 1), (0b1100, 4, 0, 5), (0b1101, 4, 0, 0), + (0b1110, 4, 0, 6), (0b11110, 5, 0, 7), (0b111110, 6, 0, 9), (0b1111110, 7, 0, 8), (0b11111110, 8, 0, 10), (0b111111110, 9, 0, 11), +]; +#[rustfmt::skip] +const MAIN_1: &[(u16, u8, u8, u8)] = &[ + (0b00, 2, 0, 2), (0b01, 2, 0, 3), (0b100, 3, 0, 1), (0b101, 3, 0, 4), (0b1100, 4, 0, 5), (0b11010, 5, 1, 2), + (0b11011, 5, 1, 1), (0b11100, 5, 0, 6), (0b11101, 5, 1, 3), (0b111100, 6, 0, 7), (0b1111010, 7, 0, 8), (0b1111011, 7, 1, 4), + (0b1111100, 7, 2, 2), (0b1111101, 7, 0, 9), (0b11111100, 8, 2, 1), (0b111111010, 9, 0, 0), (0b111111011, 9, 2, 3), (0b111111100, 9, 1, 5), + (0b1111111010, 10, 3, 1), (0b1111111011, 10, 3, 2), (0b1111111100, 10, 0, 10), (0b11111111010, 11, 1, 6), (0b1111111101100000, 16, 15, 0), (0b1111111101100001, 16, 2, 4), + (0b1111111101100010, 16, 3, 3), (0b1111111101100011, 16, 4, 1), (0b1111111101100100, 16, 4, 2), (0b1111111101100101, 16, 1, 9), (0b1111111101100110, 16, 1, 7), (0b1111111101100111, 16, 2, 5), + (0b1111111101101000, 16, 1, 8), (0b1111111101101001, 16, 5, 1), (0b1111111101101010, 16, 3, 4), (0b1111111101101011, 16, 4, 3), (0b1111111101101100, 16, 5, 2), (0b1111111101101101, 16, 2, 9), + (0b1111111101101110, 16, 3, 5), (0b1111111101101111, 16, 6, 1), (0b1111111101110000, 16, 3, 9), (0b1111111101110001, 16, 7, 1), (0b1111111101110010, 16, 6, 2), (0b1111111101110011, 16, 3, 6), + (0b1111111101110100, 16, 5, 3), (0b1111111101110101, 16, 2, 6), (0b1111111101110110, 16, 3, 8), (0b1111111101110111, 16, 1, 10), (0b1111111101111000, 16, 3, 7), (0b1111111101111001, 16, 8, 1), + (0b1111111101111010, 16, 2, 7), (0b1111111101111011, 16, 9, 1), (0b1111111101111100, 16, 7, 9), (0b1111111101111101, 16, 5, 5), (0b1111111101111110, 16, 4, 5), (0b1111111101111111, 16, 2, 8), + (0b1111111110000000, 16, 7, 2), (0b1111111110000001, 16, 5, 9), (0b1111111110000010, 16, 10, 1), (0b1111111110000011, 16, 11, 1), (0b1111111110000100, 16, 4, 4), (0b1111111110000101, 16, 6, 9), + (0b1111111110000110, 16, 5, 4), (0b1111111110000111, 16, 5, 8), (0b1111111110001000, 16, 13, 1), (0b1111111110001001, 16, 15, 10), (0b1111111110001010, 16, 5, 7), (0b1111111110001011, 16, 14, 1), + (0b1111111110001100, 16, 15, 1), (0b1111111110001101, 16, 11, 9), (0b1111111110001110, 16, 4, 9), (0b1111111110001111, 16, 4, 7), (0b1111111110010000, 16, 6, 3), (0b1111111110010001, 16, 6, 10), + (0b1111111110010010, 16, 15, 9), (0b1111111110010011, 16, 5, 6), (0b1111111110010100, 16, 4, 6), (0b1111111110010101, 16, 10, 8), (0b1111111110010110, 16, 2, 10), (0b1111111110010111, 16, 4, 10), + (0b1111111110011000, 16, 7, 8), (0b1111111110011001, 16, 9, 9), (0b1111111110011010, 16, 3, 10), (0b1111111110011011, 16, 7, 5), (0b1111111110011100, 16, 7, 4), (0b1111111110011101, 16, 8, 6), + (0b1111111110011110, 16, 6, 5), (0b1111111110011111, 16, 12, 1), (0b1111111110100000, 16, 7, 6), (0b1111111110100001, 16, 11, 6), (0b1111111110100010, 16, 9, 6), (0b1111111110100011, 16, 13, 6), + (0b1111111110100100, 16, 8, 9), (0b1111111110100101, 16, 8, 5), (0b1111111110100110, 16, 12, 9), (0b1111111110100111, 16, 15, 5), (0b1111111110101000, 16, 9, 5), (0b1111111110101001, 16, 11, 4), + (0b1111111110101010, 16, 12, 7), (0b1111111110101011, 16, 15, 7), (0b1111111110101100, 16, 8, 10), (0b1111111110101101, 16, 9, 7), (0b1111111110101110, 16, 11, 8), (0b1111111110101111, 16, 7, 3), + (0b1111111110110000, 16, 11, 7), (0b1111111110110001, 16, 13, 8), (0b1111111110110010, 16, 13, 9), (0b1111111110110011, 16, 8, 7), (0b1111111110110100, 16, 10, 7), (0b1111111110110101, 16, 7, 10), + (0b1111111110110110, 16, 4, 8), (0b1111111110110111, 16, 8, 2), (0b1111111110111000, 16, 8, 4), (0b1111111110111001, 16, 14, 10), (0b1111111110111010, 16, 15, 4), (0b1111111110111011, 16, 10, 6), + (0b1111111110111100, 16, 12, 5), (0b1111111110111101, 16, 5, 10), (0b1111111110111110, 16, 9, 4), (0b1111111110111111, 16, 10, 4), (0b1111111111000000, 16, 12, 6), (0b1111111111000001, 16, 9, 2), + (0b1111111111000010, 16, 12, 3), (0b1111111111000011, 16, 6, 8), (0b1111111111000100, 16, 11, 5), (0b1111111111000101, 16, 12, 8), (0b1111111111000110, 16, 14, 4), (0b1111111111000111, 16, 14, 5), + (0b1111111111001000, 16, 14, 6), (0b1111111111001001, 16, 14, 9), (0b1111111111001010, 16, 10, 2), (0b1111111111001011, 16, 10, 3), (0b1111111111001100, 16, 14, 3), (0b1111111111001101, 16, 12, 2), + (0b1111111111001110, 16, 6, 6), (0b1111111111001111, 16, 6, 7), (0b1111111111010000, 16, 9, 3), (0b1111111111010001, 16, 10, 10), (0b1111111111010010, 16, 13, 4), (0b1111111111010011, 16, 13, 5), + (0b1111111111010100, 16, 14, 7), (0b1111111111010101, 16, 15, 8), (0b1111111111010110, 16, 8, 8), (0b1111111111010111, 16, 9, 10), (0b1111111111011000, 16, 13, 7), (0b1111111111011001, 16, 7, 7), + (0b1111111111011010, 16, 12, 4), (0b1111111111011011, 16, 6, 4), (0b1111111111011100, 16, 14, 2), (0b1111111111011101, 16, 9, 8), (0b1111111111011110, 16, 10, 5), (0b1111111111011111, 16, 12, 10), + (0b1111111111100000, 16, 13, 10), (0b1111111111100001, 16, 14, 8), (0b1111111111100010, 16, 15, 3), (0b1111111111100011, 16, 15, 6), (0b1111111111100100, 16, 10, 9), (0b1111111111100101, 16, 11, 2), + (0b1111111111100110, 16, 11, 3), (0b1111111111100111, 16, 15, 2), (0b1111111111101000, 16, 13, 2), (0b1111111111101001, 16, 8, 3), (0b1111111111101010, 16, 11, 10), (0b1111111111101011, 16, 13, 3), +]; +#[rustfmt::skip] +const FIRST_2: &[(u16, u8, u8, u8)] = &[ + (0b000, 3, 0, 6), (0b001, 3, 0, 5), (0b010, 3, 0, 7), (0b011, 3, 0, 4), (0b100, 3, 0, 8), (0b101, 3, 0, 3), + (0b1100, 4, 0, 9), (0b1101, 4, 0, 2), (0b1110, 4, 0, 0), (0b11110, 5, 0, 10), (0b111110, 6, 0, 1), (0b1111110, 7, 0, 11), +]; +#[rustfmt::skip] +const MAIN_2: &[(u16, u8, u8, u8)] = &[ + (0b000, 3, 0, 4), (0b001, 3, 0, 5), (0b010, 3, 0, 3), (0b011, 3, 0, 6), (0b100, 3, 0, 2), (0b101, 3, 0, 7), + (0b1100, 4, 0, 1), (0b1101, 4, 0, 8), (0b11100, 5, 0, 9), (0b111010, 6, 1, 2), (0b111011, 6, 1, 3), (0b111100, 6, 1, 4), + (0b1111010, 7, 1, 1), (0b1111011, 7, 1, 5), (0b1111100, 7, 0, 10), (0b11111010, 8, 1, 6), (0b11111011, 8, 1, 7), (0b111111000, 9, 15, 0), + (0b111111001, 9, 0, 0), (0b111111010, 9, 2, 2), (0b111111011, 9, 2, 1), (0b111111100, 9, 1, 8), (0b1111111010, 10, 2, 3), (0b11111110110, 11, 1, 9), + (0b11111110111, 11, 2, 4), (0b111111110000, 12, 3, 2), (0b111111110001, 12, 3, 1), (0b1111111100100, 13, 2, 5), (0b1111111100101, 13, 3, 3), (0b1111111100110, 13, 3, 8), + (0b1111111100111, 13, 3, 7), (0b1111111101000, 13, 3, 4), (0b1111111101001, 13, 3, 5), (0b1111111101010, 13, 3, 6), (0b1111111101011, 13, 3, 9), (0b11111111011000, 14, 7, 9), + (0b11111111011001, 14, 5, 7), (0b11111111011010, 14, 5, 8), (0b11111111011011, 14, 5, 9), (0b11111111011100, 14, 2, 8), (0b11111111011101, 14, 5, 6), (0b11111111011110, 14, 7, 8), + (0b11111111011111, 14, 2, 7), (0b11111111100000, 14, 4, 1), (0b11111111100001, 14, 2, 9), (0b1111111110001000, 16, 7, 7), (0b1111111110001001, 16, 2, 6), (0b1111111110001010, 16, 4, 2), + (0b1111111110001011, 16, 7, 6), (0b1111111110001100, 16, 9, 9), (0b1111111110001101, 16, 1, 10), (0b1111111110001110, 16, 5, 5), (0b1111111110001111, 16, 9, 8), (0b1111111110010000, 16, 9, 7), + (0b1111111110010001, 16, 15, 9), (0b1111111110010010, 16, 4, 8), (0b1111111110010011, 16, 5, 4), (0b1111111110010100, 16, 9, 6), (0b1111111110010101, 16, 8, 9), (0b1111111110010110, 16, 4, 7), + (0b1111111110010111, 16, 11, 7), (0b1111111110011000, 16, 4, 9), (0b1111111110011001, 16, 15, 10), (0b1111111110011010, 16, 7, 5), (0b1111111110011011, 16, 6, 8), (0b1111111110011100, 16, 11, 6), + (0b1111111110011101, 16, 6, 7), (0b1111111110011110, 16, 6, 9), (0b1111111110011111, 16, 11, 9), (0b1111111110100000, 16, 11, 8), (0b1111111110100001, 16, 13, 8), (0b1111111110100010, 16, 5, 2), + (0b1111111110100011, 16, 13, 7), (0b1111111110100100, 16, 8, 8), (0b1111111110100101, 16, 11, 5), (0b1111111110100110, 16, 7, 4), (0b1111111110100111, 16, 5, 1), (0b1111111110101000, 16, 4, 6), + (0b1111111110101001, 16, 13, 9), (0b1111111110101010, 16, 15, 8), (0b1111111110101011, 16, 3, 10), (0b1111111110101100, 16, 13, 6), (0b1111111110101101, 16, 8, 7), (0b1111111110101110, 16, 4, 5), + (0b1111111110101111, 16, 7, 10), (0b1111111110110000, 16, 9, 5), (0b1111111110110001, 16, 13, 5), (0b1111111110110010, 16, 15, 6), (0b1111111110110011, 16, 8, 6), (0b1111111110110100, 16, 11, 4), + (0b1111111110110101, 16, 10, 9), (0b1111111110110110, 16, 9, 4), (0b1111111110110111, 16, 5, 3), (0b1111111110111000, 16, 2, 10), (0b1111111110111001, 16, 10, 8), (0b1111111110111010, 16, 4, 3), + (0b1111111110111011, 16, 15, 5), (0b1111111110111100, 16, 15, 7), (0b1111111110111101, 16, 13, 4), (0b1111111110111110, 16, 6, 6), (0b1111111110111111, 16, 10, 7), (0b1111111111000000, 16, 5, 10), + (0b1111111111000001, 16, 4, 4), (0b1111111111000010, 16, 8, 10), (0b1111111111000011, 16, 12, 9), (0b1111111111000100, 16, 14, 8), (0b1111111111000101, 16, 12, 8), (0b1111111111000110, 16, 14, 7), + (0b1111111111000111, 16, 9, 10), (0b1111111111001000, 16, 6, 10), (0b1111111111001001, 16, 7, 3), (0b1111111111001010, 16, 4, 10), (0b1111111111001011, 16, 6, 1), (0b1111111111001100, 16, 12, 7), + (0b1111111111001101, 16, 15, 4), (0b1111111111001110, 16, 12, 6), (0b1111111111001111, 16, 6, 5), (0b1111111111010000, 16, 14, 9), (0b1111111111010001, 16, 7, 2), (0b1111111111010010, 16, 14, 6), + (0b1111111111010011, 16, 7, 1), (0b1111111111010100, 16, 9, 1), (0b1111111111010101, 16, 9, 3), (0b1111111111010110, 16, 10, 6), (0b1111111111010111, 16, 13, 10), (0b1111111111011000, 16, 9, 2), + (0b1111111111011001, 16, 8, 5), (0b1111111111011010, 16, 6, 2), (0b1111111111011011, 16, 15, 3), (0b1111111111011100, 16, 12, 5), (0b1111111111011101, 16, 11, 2), (0b1111111111011110, 16, 10, 4), + (0b1111111111011111, 16, 8, 4), (0b1111111111100000, 16, 11, 10), (0b1111111111100001, 16, 6, 4), (0b1111111111100010, 16, 10, 5), (0b1111111111100011, 16, 11, 3), (0b1111111111100100, 16, 13, 2), + (0b1111111111100101, 16, 8, 1), (0b1111111111100110, 16, 14, 5), (0b1111111111100111, 16, 13, 3), (0b1111111111101000, 16, 10, 10), (0b1111111111101001, 16, 12, 4), (0b1111111111101010, 16, 12, 10), + (0b1111111111101011, 16, 15, 2), (0b1111111111101100, 16, 11, 1), (0b1111111111101101, 16, 14, 4), (0b1111111111101110, 16, 13, 1), (0b1111111111101111, 16, 8, 3), (0b1111111111110000, 16, 6, 3), + (0b1111111111110001, 16, 14, 10), (0b1111111111110010, 16, 12, 3), (0b1111111111110011, 16, 14, 2), (0b1111111111110100, 16, 8, 2), (0b1111111111110101, 16, 15, 1), (0b1111111111110110, 16, 10, 3), + (0b1111111111110111, 16, 12, 2), (0b1111111111111000, 16, 10, 1), (0b1111111111111001, 16, 12, 1), (0b1111111111111010, 16, 14, 3), (0b1111111111111011, 16, 10, 2), (0b1111111111111100, 16, 14, 1), +]; +/// The tables for DecoderTable selector 0, 1 or 2. +pub(crate) fn tables(selector: u32) -> Option { + match selector { + 0 => Some(SelTables { first: FIRST_0, main: MAIN_0 }), + 1 => Some(SelTables { first: FIRST_1, main: MAIN_1 }), + 2 => Some(SelTables { first: FIRST_2, main: MAIN_2 }), + _ => None, + } +} diff --git a/crates/crw/src/tests.rs b/crates/crw/src/tests.rs new file mode 100644 index 00000000..f78e19b8 --- /dev/null +++ b/crates/crw/src/tests.rs @@ -0,0 +1,525 @@ +//! Unit tests: synthetic CIFF files built with a test-only encoder that is the inverse of the +//! frozen model, plus the frozen tables' structural properties. The decoder's behaviour on real +//! files is pinned in `tests/samples.rs`. + +use super::*; + +fn put16(v: u16, o: Order) -> [u8; 2] { + match o { + Order::Le => v.to_le_bytes(), + Order::Be => v.to_be_bytes(), + } +} +fn put32(v: u32, o: Order) -> [u8; 4] { + match o { + Order::Le => v.to_le_bytes(), + Order::Be => v.to_be_bytes(), + } +} + +/// A heap block: values, record table, trailing offset-table pointer. +fn heap(records: &[(u16, Vec)], o: Order) -> Vec { + let mut b = Vec::new(); + let mut entries = Vec::new(); + for (tc, v) in records { + let off = b.len(); + b.extend_from_slice(v); + if b.len() % 2 == 1 { + b.push(0); + } + entries.push((*tc, v.len() as u32, off as u32)); + } + let tbl = b.len() as u32; + b.extend_from_slice(&put16(entries.len() as u16, o)); + for (tc, l, off) in entries { + b.extend_from_slice(&put16(tc, o)); + b.extend_from_slice(&put32(l, o)); + b.extend_from_slice(&put32(off, o)); + } + b.extend_from_slice(&put32(tbl, o)); + b +} + +/// A complete CIFF file: 26-byte header + the root heap. +fn ciff(records: &[(u16, Vec)], o: Order) -> Vec { + let mut f = Vec::new(); + f.extend_from_slice(match o { + Order::Le => b"II", + Order::Be => b"MM", + }); + f.extend_from_slice(&put32(26, o)); + f.extend_from_slice(b"HEAPCCDR"); + f.extend_from_slice(&put32(0x0001_0002, o)); + f.extend_from_slice(&[0; 8]); + f.extend_from_slice(&heap(records, o)); + f +} + +fn sensor_info(w: u16, h: u16, crop: Option<(u16, u16, u16, u16)>, o: Order) -> Vec { + let (l, t, r, b) = crop.unwrap_or((0, 0, 0, 0)); + [34, w, h, 0, 0, l, t, r, b, 0, 0, 0, 0, 0, 0, 0, 0].iter().flat_map(|&v| put16(v, o)).collect() +} + +fn decoder_table(sel: u32, o: Order) -> Vec { + [sel, 0, 514, 0].iter().flat_map(|&v| put32(v, o)).collect() +} + +/// Entropy-encode a 10-bit high plane with the model's inverse: per-block first/main tables, +/// (0, s) tokens only (the caller keeps mid-block diffs nonzero), JPEG magnitudes, carry, +/// parity chains reset at row starts; MSB-first packing, zero padding, ff00 stuffing. +fn encode_hi(hi: &[u16], w: usize, t: &tables::SelTables) -> Vec { + let mut bits: Vec = Vec::new(); + let push_code = |bits: &mut Vec, tab: &[(u16, u8, u8, u8)], s: u32| { + let &(c, l, ..) = tab.iter().find(|e| e.2 == 0 && e.3 as u32 == s).expect("(0, s) code exists"); + bits.extend((0..l).rev().map(|i| (c >> i) & 1 == 1)); + }; + let size_of = |d: i32| if d == 0 { 0 } else { 32 - d.unsigned_abs().leading_zeros() }; + let (mut ce, mut co) = (512i32, 512i32); + let mut carry = 0i32; + for (qi, &v) in hi.iter().enumerate() { + if qi % w == 0 { + ce = 512; + co = 512; + } + let c = if qi % 2 == 0 { &mut ce } else { &mut co }; + let dd = v as i32 - *c; + *c = v as i32; + let (tab, d) = if qi % 64 == 0 { + let d = dd - carry; + carry = dd; + (t.first, d) + } else { + assert_ne!(dd, 0, "test images avoid zero diffs mid-block"); + (t.main, dd) + }; + let s = size_of(d); + push_code(&mut bits, tab, s); + if s > 0 { + let m = if d > 0 { d } else { d + (1 << s) - 1 } as u32; + bits.extend((0..s).rev().map(|i| (m >> i) & 1 == 1)); + } + } + let mut bytes = Vec::new(); + for chunk in bits.chunks(8) { + let mut b = 0u8; + for (i, &bit) in chunk.iter().enumerate() { + b |= (bit as u8) << (7 - i); + } + bytes.push(b); + } + let mut stuffed = Vec::new(); + for b in bytes { + stuffed.push(b); + if b == 0xff { + stuffed.push(0x00); + } + } + stuffed +} + +/// A high plane whose diffs are all nonzero, bounded around 512: values follow +/// `(3 qi) mod 5 + 1`, so same-parity neighbours differ by 1 or −4 (never 0) and +/// row starts sit 1–5 above the reset value. +fn test_hi(n: usize) -> Vec { + (0..n).map(|qi| 512 + (qi as u16 * 3) % 5 + 1).collect() +} + +fn payload(low: Option<&[u8]>, entropy: &[u8]) -> Vec { + let mut p = low.map(|l| l.to_vec()).unwrap_or_default(); + p.extend_from_slice(&[0; 514]); + p.extend_from_slice(entropy); + p.extend_from_slice(&[0, 0]); + p +} + +fn make_model(model: &str) -> Vec { + let mut v = b"Canon\0".to_vec(); + v.extend_from_slice(model.as_bytes()); + v.push(0); + v +} + +fn roundtrip_file(w: u16, h: u16, model: &str, low: Option<&[u8]>, hi: &[u16], o: Order) -> Vec { + let tables = tables::tables(0).expect("selector 0"); + let entropy = encode_hi(hi, w as usize, &tables); + // the realistic shape: raw data in the root, properties in a nested heap + let props = heap(&[(SENSOR_INFO, sensor_info(w, h, None, o)), (DECODER_TABLE, decoder_table(0, o)), (MAKE_MODEL, make_model(model))], o); + ciff(&[(RAW_DATA, payload(low, &entropy)), (0x300a, props)], o) +} + +fn full(bytes: &[u8]) -> Result { + decode(bytes, &Limits::default()) +} + +#[test] +fn probe_needs_the_signature() { + assert!(probe(b"II\x1a\0\0\0HEAPCCDR\0\0")); + assert!(probe(b"MM\0\0\0\x1aHEAPCCDR")); + assert!(!probe(b"II\x1a\0\0\0HEAP")); + assert!(!probe(b"II*\0\x08\0\0\0"), "a TIFF"); + assert!(!probe(b"")); + assert_eq!(full(b"II*\0\x08\0\0\0"), Err(Error::NotCrw)); + // a signature with no heap behind it is corrupt, not a panic + assert!(matches!(full(b"II HEAPCCDR "), Err(Error::Corrupt(_)))); +} + +#[test] +fn roundtrips_a_10_bit_frame() { + let (w, h) = (16u16, 8u16); + let hi = test_hi(128); + let f = roundtrip_file(w, h, "Canon EOS D60", None, &hi, Order::Le); + let img = full(&f).expect("decodes"); + assert_eq!((img.header.width, img.header.height, img.header.bits), (16, 8, 10)); + assert_eq!(img.samples, hi); + assert_eq!(img.header.cfa, Cfa::Rggb); + assert_eq!(img.header.selector, 0); + assert_eq!(img.header.make.as_deref(), Some("Canon")); + assert_eq!(img.header.model.as_deref(), Some("Canon EOS D60")); + assert_eq!(img.header.exif_orientation, 1); + assert_eq!(img.header.capture_unix, None); + assert_eq!(img.header.wb_multipliers, None); +} + +#[test] +fn roundtrips_a_12_bit_frame_with_a_low_bits_plane() { + let (w, h) = (16u16, 8u16); + let hi = test_hi(128); + // low bits all 3: the plane is all 0xff bytes, which trips the presence rule + let low = vec![0xffu8; 32]; + let f = roundtrip_file(w, h, "Canon EOS D60", Some(&low), &hi, Order::Le); + let img = full(&f).expect("decodes"); + assert_eq!(img.header.bits, 12); + let expected: Vec = hi.iter().map(|&v| (v << 2) | 3).collect(); + assert_eq!(img.samples, expected); +} + +/// A low-bits plane without a single 0xff byte (real D60 files whose lowest bit is nearly +/// always 0) used to be taken for "no plane": the plane was read as entropy data and the +/// file came out as 10-bit garbage. The zero header after the plane gives it away. +#[test] +fn a_low_bits_plane_without_ff_bytes_is_still_a_plane() { + let hi = test_hi(128); + for byte in [0x22u8, 0x00] { + let low = vec![byte; 32]; + let f = roundtrip_file(16, 8, "Canon EOS D60", Some(&low), &hi, Order::Le); + let img = full(&f).expect("decodes"); + assert_eq!(img.header.bits, 12); + let expected: Vec = hi.iter().enumerate().map(|(i, &v)| (v << 2) | u16::from((byte >> ((i & 3) * 2)) & 3)).collect(); + assert_eq!(img.samples, expected); + } +} + +#[test] +fn big_endian_ciff_decodes_too() { + let hi = test_hi(128); + let f = roundtrip_file(16, 8, "Canon EOS D60", None, &hi, Order::Be); + let img = full(&f).expect("decodes"); + assert_eq!(img.samples, hi); +} + +#[test] +fn header_reads_geometry_without_samples() { + let hi = test_hi(128); + let f = roundtrip_file(16, 8, "Canon EOS D60", None, &hi, Order::Le); + let hdr = header(&f, &Limits::default()).expect("header decodes"); + assert_eq!((hdr.width, hdr.height, hdr.bits), (16, 8, 10)); + assert_eq!(hdr, full(&f).expect("decodes").header, "the header is the same either way"); +} + +#[test] +fn limits_cap_the_sample_count() { + let hi = test_hi(128); + let f = roundtrip_file(16, 8, "Canon EOS D60", None, &hi, Order::Le); + let mut limits = Limits { max_samples: 127 }; + assert_eq!(decode(&f, &limits), Err(Error::LimitExceeded("image too large"))); + assert!(header(&f, &limits).is_err()); + limits.max_samples = 128; + assert!(decode(&f, &limits).is_ok()); +} + +#[test] +fn g1_is_gbrg() { + let hi = test_hi(128); + let f = roundtrip_file(16, 8, "Canon PowerShot G1", None, &hi, Order::Le); + let img = full(&f).expect("decodes"); + assert_eq!(img.header.cfa, Cfa::Gbrg); + assert_eq!(img.header.cfa.pattern(), [1, 2, 0, 1]); + assert_eq!(Cfa::Rggb.pattern(), [0, 1, 1, 2]); +} + +#[test] +fn g5_is_refused_with_its_anomaly_note() { + let hi = test_hi(128); + let f = roundtrip_file(16, 8, "Canon PowerShot G5", None, &hi, Order::Le); + let Err(Error::Unsupported(why)) = full(&f) else { panic!("expected Unsupported") }; + assert!(why.contains("G5"), "{why}"); + assert!(matches!(header(&f, &Limits::default()), Err(Error::Unsupported(_)))); +} + +/// A D60 at ISO 100 clips near 4000, and a photo with a small blown area has no plateau: +/// the white level must still come down to where the data stops, not stay at 4095. A few +/// hot pixels, or a photo that never gets near the top, leave the full range. +#[test] +fn white_level_follows_a_small_clipped_area() { + let mut data: Vec = (0..200_000).map(|i| 300 + (i % 3000) as u16).collect(); + assert_eq!(white_level(&data, 12), 4095.0, "nothing near the top"); + for v in data.iter_mut().take(100) { + *v = 4002; + } + assert_eq!(white_level(&data, 12), (4002 - 15) as f32, "100 samples at the clip level"); + let mut hot: Vec = (0..200_000).map(|i| 300 + (i % 3000) as u16).collect(); + for v in hot.iter_mut().take(10) { + *v = 4002; + } + assert_eq!(white_level(&hot, 12), 4095.0, "ten hot pixels are not a clip level"); +} + +#[test] +fn saturation_plateau_is_found_with_strays_above_it() { + let mut d: Vec = (0..100_000).map(|i| (i % 10000) as u16).collect(); + assert_eq!(saturation_plateau(&d, 14), 16383.0); + d.extend(std::iter::repeat_n(15000u16, 1000)); + let w = saturation_plateau(&d, 14); + assert!((14900.0..=15000.0).contains(&w), "{w}"); + assert_eq!(saturation_plateau(&[], 12), 4095.0); + let mut d: Vec = (0..100_000).map(|i| (2048 + i % 10000) as u16).collect(); + d.extend(std::iter::repeat_n(13583u16, 5000)); + d.extend([13700, 13900, 14100, 14426]); + let w = saturation_plateau(&d, 14); + assert!((13500.0..=13583.0).contains(&w), "{w}"); +} + +/// The D60 layout: sensor rows clamped to 0 above the image, dark columns left of it. The +/// clamped rows must not count towards the black level. +#[test] +fn black_level_ignores_clamped_rows() { + let (w, h) = (64usize, 40usize); + let active = Rect::new(24, 16, 40, 24); + let mut data = vec![900u16; w * h]; + for y in 0..h { + for x in 0..w { + data[y * w + x] = if y < 7 { + 0 + } else if x < 24 { + 128 + } else { + 900 + }; + } + } + let black = black_from_borders(&data, w, h, active); + assert!(black.values.iter().all(|v| (v - 128.0).abs() < 0.5), "{:?}", black.values); + assert_eq!(black.at(active, 24, 16), black.values[0]); + assert_eq!(black.at(active, 25, 16), black.values[1]); + assert_eq!(black.at(active, 24, 17), black.values[2]); + assert_eq!(black.at(active, 23, 15), black.values[3], "sites are counted from the active origin, modulo 2"); + assert_eq!(black_from_borders(&data, w, h, Rect::new(0, 0, w, h)), BlackLevel::uniform(0.0), "no border, no level"); +} + +/// The frozen stream-end rule: a truncated entropy stream leaves the remaining samples at +/// their chain values instead of failing the file. +#[test] +fn truncated_stream_keeps_chain_values() { + let (w, h) = (16u16, 8u16); + let hi = test_hi(128); + let tables = tables::tables(0).expect("selector 0"); + let entropy = encode_hi(&hi, 16, &tables); + let cut = entropy.len() / 2; + let props = heap( + &[(SENSOR_INFO, sensor_info(w, h, None, Order::Le)), (DECODER_TABLE, decoder_table(0, Order::Le)), (MAKE_MODEL, make_model("Canon EOS D60"))], + Order::Le, + ); + let f = ciff(&[(RAW_DATA, payload(None, &entropy[..cut])), (0x300a, props)], Order::Le); + let img = full(&f).expect("truncated stream still decodes"); + assert_eq!(img.samples.len(), 128); + assert_eq!(&img.samples[..32], &hi[..32], "the leading samples decode exactly"); +} + +/// Every record the decoder needs missing, bogus selectors, dimensions and offsets: errors, +/// never panics; and every truncation of a valid file survives. +#[test] +fn hostile_inputs_error_cleanly() { + let o = Order::Le; + let hi = test_hi(128); + let good = roundtrip_file(16, 8, "Canon EOS D60", None, &hi, o); + for n in 0..good.len() { + let _ = full(&good[..n]); + let _ = header(&good[..n], &Limits::default()); + } + // single-byte mutations over the whole file + for i in 0..good.len() { + let mut bad = good.clone(); + bad[i] ^= 0xa5; + let _ = full(&bad); + } + let props_no_sensor = heap(&[(DECODER_TABLE, decoder_table(0, o))], o); + let f = ciff(&[(RAW_DATA, vec![0; 600]), (0x300a, props_no_sensor)], o); + assert!(matches!(full(&f), Err(Error::Corrupt(_)))); + let props_bad_sel = heap(&[(SENSOR_INFO, sensor_info(16, 8, None, o)), (DECODER_TABLE, decoder_table(7, o))], o); + let f = ciff(&[(RAW_DATA, vec![0; 600]), (0x300a, props_bad_sel)], o); + assert!(matches!(full(&f), Err(Error::Unsupported(_)))); + let props_zero = heap(&[(SENSOR_INFO, sensor_info(0, 0, None, o)), (DECODER_TABLE, decoder_table(0, o))], o); + let f = ciff(&[(RAW_DATA, vec![0; 600]), (0x300a, props_zero)], o); + assert!(full(&f).is_err()); + // a heap that nests into itself must hit the depth bound, not recurse forever + let mut self_heap = heap(&[(0x300a, vec![0; 10])], o); + let len = self_heap.len() as u32; + // point the nested record at the whole block + let tbl = u32::from_le_bytes(self_heap[len as usize - 4..].try_into().unwrap()) as usize; + self_heap[tbl + 4..tbl + 8].copy_from_slice(&len.to_le_bytes()); + self_heap[tbl + 8..tbl + 12].copy_from_slice(&0u32.to_le_bytes()); + let mut f = ciff(&[], o); + f.truncate(26); + f.extend_from_slice(&self_heap); + assert!(full(&f).is_err()); +} + +/// The frozen tables are prefix-free within each table, so the 16-bit lookup is unambiguous. +#[test] +fn frozen_tables_are_prefix_free() { + for sel in 0..3 { + let t = tables::tables(sel).expect("selector"); + for (name, tab) in [("first", t.first), ("main", t.main)] { + let mut lut = vec![false; 1 << 16]; + for &(code, len, ..) in tab { + assert!((1..=16).contains(&len), "sel {sel} {name}: code length {len}"); + let base = (code as usize) << (16 - len); + for cell in lut.iter_mut().skip(base).take(1 << (16 - len)) { + assert!(!*cell, "sel {sel} {name}: overlapping codes"); + *cell = true; + } + } + } + } + assert!(tables::tables(3).is_none()); +} + +/// Every second table codes all 162 symbols: (r, s) for r 0..=15 and s 1..=10, end of block +/// and (15, 0). Selectors 0 and 1 lacked (7..=14, 10) and (14, 10), learned from too few +/// files, and 223 of 1,516 real D60 files failed with "undecodable entropy code". +#[test] +fn second_tables_code_every_symbol() { + for sel in 0..3 { + let t = tables::tables(sel).expect("selector"); + let have: std::collections::BTreeSet<(u8, u8)> = t.main.iter().map(|e| (e.2, e.3)).collect(); + let mut want: std::collections::BTreeSet<(u8, u8)> = (0..16).flat_map(|r| (1..=10).map(move |s| (r, s))).collect(); + want.extend([(0, 0), (15, 0)]); + assert_eq!(have, want, "selector {sel}"); + assert_eq!(t.main.len(), 162, "selector {sel}: one code per symbol"); + } +} + +#[test] +fn white_balance_comes_from_the_shot_info_slot() { + let o = Order::Le; + let hi = test_hi(128); + // D60-shaped table: 7 slots + black levels; Daylight slot (1) = 1800, 800, 800, 960 + let mut cb: Vec = vec![66]; + cb.extend([2000u16, 760, 760, 830]); + cb.extend([1800u16, 800, 800, 960]); + for _ in 0..5 { + cb.extend([1500u16, 900, 900, 1500]); + } + cb.extend([126u16, 127, 126, 126]); + let cb: Vec = cb.iter().flat_map(|&v| put16(v, o)).collect(); + let mut shot: Vec = vec![54; 27]; + shot[7] = 1; // Daylight + let shot: Vec = shot.iter().flat_map(|&v| put16(v, o)).collect(); + let tables = tables::tables(0).expect("selector 0"); + let props = heap( + &[ + (SENSOR_INFO, sensor_info(16, 8, None, o)), + (DECODER_TABLE, decoder_table(0, o)), + (MAKE_MODEL, make_model("Canon EOS D60")), + (COLOR_BALANCE, cb), + (SHOT_INFO, shot), + ], + o, + ); + let f = ciff(&[(RAW_DATA, payload(None, &encode_hi(&hi, 16, &tables))), (0x300a, props)], o); + let img = full(&f).expect("decodes"); + let wb = img.header.wb_multipliers.expect("wb"); + assert!((wb[0] - 2.25).abs() < 1e-3 && wb[1] == 1.0 && (wb[2] - 1.2).abs() < 1e-3, "{wb:?}"); +} + +#[test] +fn rotation_maps_to_exif_orientation_and_the_time_stamp_is_unix_seconds() { + let o = Order::Le; + let hi = test_hi(128); + let image_info: Vec = [3072u32, 2048, 0, 270, 12, 12, 0].iter().flat_map(|&v| put32(v, o)).collect(); + let props = heap( + &[ + (SENSOR_INFO, sensor_info(16, 8, None, o)), + (DECODER_TABLE, decoder_table(0, o)), + (MAKE_MODEL, make_model("Canon EOS 300D DIGITAL")), + (IMAGE_INFO, image_info), + (TIME_STAMP, [1083410638u32, 0, 0].iter().flat_map(|&v| put32(v, o)).collect()), + ], + o, + ); + let tables = tables::tables(0).expect("selector 0"); + let f = ciff(&[(RAW_DATA, payload(None, &encode_hi(&hi, 16, &tables))), (0x300a, props)], o); + let img = full(&f).expect("decodes"); + assert_eq!(img.header.exif_orientation, 8); + assert_eq!(img.header.capture_unix, Some(1083410638)); +} + +#[test] +fn sensor_info_crop_is_the_active_area() { + let o = Order::Le; + let hi = test_hi(128); + let tables = tables::tables(0).expect("selector 0"); + let with = |crop| { + let props = + heap(&[(SENSOR_INFO, sensor_info(16, 8, crop, o)), (DECODER_TABLE, decoder_table(0, o)), (MAKE_MODEL, make_model("Canon EOS D60"))], o); + let f = ciff(&[(RAW_DATA, payload(None, &encode_hi(&hi, 16, &tables))), (0x300a, props)], o); + full(&f).expect("decodes").header.active_area + }; + assert_eq!(with(Some((4, 1, 15, 7))), Rect::new(4, 1, 12, 7), "inclusive left, top, right, bottom"); + assert_eq!(with(None), Rect::new(0, 0, 16, 8), "no crop: the whole sensor"); + assert_eq!(with(Some((4, 1, 16, 7))), Rect::new(0, 0, 16, 8), "a crop beyond the sensor is ignored"); + assert_eq!(with(Some((5, 1, 4, 7))), Rect::new(0, 0, 16, 8), "an inverted crop is ignored"); +} + +/// A block whose every record is a sub-heap pointing back at the block itself: depth alone +/// would allow N^MAX_DEPTH visits (here 64^17), the record budget ends the walk promptly. +#[test] +fn self_referencing_sub_heaps_terminate() { + const N: usize = 64; + for o in [Order::Le, Order::Be] { + let len = (2 + N * 10 + 4) as u32; + let mut f = Vec::new(); + f.extend_from_slice(match o { + Order::Le => b"II", + Order::Be => b"MM", + }); + f.extend_from_slice(&put32(26, o)); + f.extend_from_slice(b"HEAPCCDR"); + f.extend_from_slice(&put32(0x0001_0002, o)); + f.extend_from_slice(&[0; 8]); + f.extend_from_slice(&put16(N as u16, o)); // record table at block offset 0 + for _ in 0..N { + f.extend_from_slice(&put16(5 << 11 | 1, o)); // storage class 5: sub-heap + f.extend_from_slice(&put32(len, o)); // ... covering the whole parent block + f.extend_from_slice(&put32(0, o)); + } + f.extend_from_slice(&put32(0, o)); // trailing offset-table pointer + let started = std::time::Instant::now(); + assert!(matches!(header(&f, &Limits::default()), Err(Error::Corrupt(_)))); + assert!(matches!(full(&f), Err(Error::Corrupt(_)))); + assert!(started.elapsed() < std::time::Duration::from_secs(5), "walk did not stay within budget"); + let mut rec = BTreeMap::new(); + let mut budget = MAX_RECORDS; + walk(&f, 26, len as usize, o, 0, &mut budget, &mut rec); + assert_eq!(budget, 0); + assert!(rec.is_empty()); + } +} + +#[test] +fn errors_display_their_cause() { + assert_eq!(Error::NotCrw.to_string(), "not a Canon CRW (CIFF) file"); + assert_eq!(corrupt("no raw data record").to_string(), "CRW: no raw data record"); + assert_eq!(Error::Unsupported("decoder table 7".into()).to_string(), "unsupported CRW: decoder table 7"); + assert_eq!(Error::LimitExceeded("image too large").to_string(), "CRW exceeds a limit: image too large"); +} diff --git a/crates/crw/tests/robust.rs b/crates/crw/tests/robust.rs new file mode 100644 index 00000000..92b5355a --- /dev/null +++ b/crates/crw/tests/robust.rs @@ -0,0 +1,108 @@ +//! Malformed CRW files must decode to an error, never panic: a CIFF with the records the decoder +//! reads and a pseudo-random payload, mutated by bit flips and truncation. + +use craft_crw::{Limits, decode, header, probe}; +use proptest::prelude::*; + +const W: usize = 48; +const H: usize = 20; + +/// A Canon CRW: CIFF header, a raw-data payload of pseudo-random bytes, and a nested properties +/// heap with the records the decoder reads. +fn crw() -> Vec { + let le16 = |v: u16| v.to_le_bytes(); + let le32 = |v: u32| v.to_le_bytes(); + let heap = |records: &[(u16, Vec)]| -> Vec { + let mut b = Vec::new(); + let mut entries = Vec::new(); + for (tc, v) in records { + entries.push((*tc, v.len() as u32, b.len() as u32)); + b.extend_from_slice(v); + if b.len() % 2 == 1 { + b.push(0); + } + } + let tbl = b.len() as u32; + b.extend_from_slice(&le16(entries.len() as u16)); + for (tc, l, off) in entries { + b.extend_from_slice(&le16(tc)); + b.extend_from_slice(&le32(l)); + b.extend_from_slice(&le32(off)); + } + b.extend_from_slice(&le32(tbl)); + b + }; + let sensor: Vec = [34u16, W as u16, H as u16, 0, 0, 2, 2, W as u16 - 3, H as u16 - 3].iter().flat_map(|&v| le16(v)).collect(); + let dtable: Vec = [0u32, 0, 514, 0].iter().flat_map(|&v| le32(v)).collect(); + let image_info: Vec = [W as u32, H as u32, 0, 270, 12, 12, 0].iter().flat_map(|&v| le32(v)).collect(); + let mut color_balance: Vec = vec![66]; + color_balance.extend((0..8).flat_map(|i| [1800 + i * 10, 800, 800, 960 + i * 5])); + let color_balance: Vec = color_balance.iter().flat_map(|&v| le16(v)).collect(); + let shot: Vec = [54u16; 27].iter().flat_map(|&v| le16(v)).collect(); + let props = heap(&[ + (0x1031, sensor), + (0x1835, dtable), + (0x080a, b"Canon\0Canon EOS D60\0".to_vec()), + (0x1810, image_info), + (0x180e, [1083410638u32, 0, 0].iter().flat_map(|&v| le32(v)).collect()), + (0x10a9, color_balance), + (0x102a, shot), + ]); + let payload: Vec = (0..514 + 256).map(|i| (i * 41 % 253) as u8).collect(); + let root = heap(&[(0x2005, payload), (0x300a, props)]); + let mut f = b"II".to_vec(); + f.extend_from_slice(&le32(26)); + f.extend_from_slice(b"HEAPCCDR"); + f.extend_from_slice(&le32(0x0001_0002)); + f.extend_from_slice(&[0; 8]); + f.extend_from_slice(&root); + f +} + +#[test] +fn the_synthetic_file_is_recognised() { + let f = crw(); + assert!(probe(&f)); + let hdr = header(&f, &Limits::default()).expect("header"); + assert_eq!((hdr.width, hdr.height), (W, H)); + assert_eq!(hdr.exif_orientation, 8); + assert_eq!(hdr.capture_unix, Some(1083410638)); + assert!(hdr.wb_multipliers.is_some()); + // pseudo-random entropy data decodes or is refused, never panics + let _ = decode(&f, &Limits::default()); +} + +#[test] +fn every_truncation_and_single_byte_mutation_survives() { + let good = crw(); + for n in 0..good.len() { + let _ = header(&good[..n], &Limits::default()); + let _ = decode(&good[..n], &Limits::default()); + } + for i in 0..good.len() { + let mut bad = good.clone(); + bad[i] ^= 0xa5; + let _ = decode(&bad, &Limits::default()); + bad[i] = 0xff; + let _ = decode(&bad, &Limits::default()); + } +} + +proptest! { + #![proptest_config(ProptestConfig { cases: 2000, .. ProptestConfig::default() })] + + #[test] + fn mutated_files_never_panic(flips in proptest::collection::vec((any::(), any::()), 1..16), cut in any::(), max_samples in 1usize..4096) { + let mut data = crw(); + let n = data.len(); + for (i, v) in flips { + data[i % n] ^= v; + } + data.truncate(cut % (n + 1)); + let mut limits = Limits::default(); + limits.max_samples = max_samples; + let _ = header(&data, &limits); + let _ = decode(&data, &limits); + let _ = decode(&data, &Limits::default()); + } +} diff --git a/crates/crw/tests/samples.json b/crates/crw/tests/samples.json new file mode 100644 index 00000000..df90b371 --- /dev/null +++ b/crates/crw/tests/samples.json @@ -0,0 +1,140 @@ +[ + { + "name": "crw-canon-eos-d30.crw", + "body": "Canon EOS D30", + "source": "https://raw.pixls.us/getfile.php/1307/nice/Canon%20-%20EOS%20D30%20-%20RAW%20(3:2).CRW", + "sha256": "4ee482f2b6f9f1a8d370703ea04e9f2abaa49356973a7f56b229af93a78e8a2e", + "license": "CC0-1.0", + "expect": { + "width": 2224, "height": 1456, "bits": 12, "selector": 1, "cfa": "RGGB", + "active": [56, 12, 2160, 1440], "model": "Canon EOS D30", "exif_orientation": 1, "capture_unix": 1482512730, + "wb": null, "black": [127.9338, 127.8187, 128.2763, 128.4221], "white": 4095.0, + "samples_sha256": "8ac62f45a4ee04efebed7900eb46dd263dc4a03a96c4fe711dc41b6bec6078fe" + } + }, + { + "name": "crw-canon-eos-d60.crw", + "body": "Canon EOS D60", + "source": "https://raw.pixls.us/getfile.php/2010/nice/Canon%20-%20EOS%20D60%20-%20RAW%20(3:2).CRW", + "sha256": "2b98659ecff170af1dfe44f89b493139e75e995b2979686f6ff0847cfb56b97f", + "license": "CC0-1.0", + "expect": { + "width": 3152, "height": 2068, "bits": 12, "selector": 0, "cfa": "RGGB", + "active": [72, 16, 3072, 2048], "model": "Canon EOS D60", "exif_orientation": 1, "capture_unix": 1467566074, + "wb": [2.627477, 1.0, 1.0990753], "black": [127.4332, 127.0843, 127.3606, 126.9793], "white": 3984.0, + "samples_sha256": "e01a6bc5cb8f67cd066f68863a04050f183c41e45395e8d4bc3b5ba4393561bb" + } + }, + { + "name": "crw-canon-eos-10d-a.crw", + "body": "Canon EOS 10D", + "source": "https://raw.pixls.us/getfile.php/2027/nice/Canon%20-%20EOS%2010D%20-%20RAW%20(3:2).CRW", + "sha256": "b0730eb3a67cc2281a32e0b2ee104199246928d954a68b7e5b72366514c73d02", + "license": "CC0-1.0", + "expect": { + "width": 3152, "height": 2068, "bits": 12, "selector": 0, "cfa": "RGGB", + "active": [72, 16, 3072, 2048], "model": "Canon EOS 10D", "exif_orientation": 1, "capture_unix": 1228574022, + "wb": [2.09375, 1.0, 1.4915866], "black": [127.8010, 126.4857, 127.8322, 126.5070], "white": 3983.0, + "samples_sha256": "3aecb01756492569734885710dafd6ffc26421ac0b5a7fa1c151973d48cd389a" + } + }, + { + "name": "crw-canon-eos-10d-b.crw", + "body": "Canon EOS 10D", + "source": "https://raw.pixls.us/getfile.php/3164/nice/Canon%20-%20EOS%2010D%20-%20RAW%20(3:2).crw", + "sha256": "e879c94afdc64dabc88b01ce2d02628aa225259303ec71dd61f7fb0322bfea2d", + "license": "CC0-1.0", + "expect": { + "width": 3152, "height": 2068, "bits": 12, "selector": 2, "cfa": "RGGB", + "active": [72, 16, 3072, 2048], "model": "Canon EOS 10D", "exif_orientation": 1, "capture_unix": 1105175848, + "wb": [1.8184005, 1.0, 1.3529766], "black": [124.6726, 125.9032, 124.6995, 125.9359], "white": 3990.0, + "samples_sha256": "25d8c540bace7ae9b92ab631fae3c4effee9625e388af600cdf60894764d3f92" + } + }, + { + "name": "crw-canon-eos-10d-c.crw", + "body": "Canon EOS 10D", + "source": "https://raw.pixls.us/getfile.php/3165/nice/Canon%20-%20EOS%2010D%20-%20RAW%20(3:2).crw", + "sha256": "58144118854657df5cd2185df40fecf6b665fc6754fef0416c2f222b68b025c6", + "license": "CC0-1.0", + "expect": { + "width": 3152, "height": 2068, "bits": 12, "selector": 2, "cfa": "RGGB", + "active": [72, 16, 3072, 2048], "model": "Canon EOS 10D", "exif_orientation": 1, "capture_unix": 1109761444, + "wb": [1.9759471, 1.0, 1.1220685], "black": [127.7952, 128.2460, 127.8664, 128.3089], "white": 3988.0, + "samples_sha256": "c1bc8c958de399ecb5fbaeb3bd473baabbb534c5874508f9b51c3984a1319c7f" + } + }, + { + "name": "crw-canon-eos-300d-a.crw", + "body": "Canon EOS 300D", + "source": "https://raw.pixls.us/getfile.php/1983/nice/Canon%20-%20EOS%20300D%20-%20RAW%20(3:2).crw", + "sha256": "47da16e1668af5500eea63cf42cf2992d9cb3d32180b8d786f17f1e2a57f39f4", + "license": "CC0-1.0", + "expect": { + "width": 3152, "height": 2068, "bits": 12, "selector": 0, "cfa": "RGGB", + "active": [72, 16, 3072, 2048], "model": "Canon EOS 300D DIGITAL", "exif_orientation": 8, "capture_unix": 1083408238, + "wb": [1.9579327, 1.0, 1.2632211], "black": [125.1676, 124.4162, 125.1797, 124.3886], "white": 3987.0, + "samples_sha256": "87848353618212be07370d659ceaa2ab349ba79b29c73702368233be2371de65" + } + }, + { + "name": "crw-canon-eos-300d-b.crw", + "body": "Canon EOS 300D", + "source": "https://raw.pixls.us/getfile.php/3162/nice/Canon%20-%20EOS%20300D%20-%20RAW%20(3:2).CRW", + "sha256": "9e7f62239fe728046af28d0d4a8a722d35d40146a79794a5bda1775306af4383", + "license": "CC0-1.0", + "expect": { + "width": 3152, "height": 2068, "bits": 12, "selector": 2, "cfa": "RGGB", + "active": [72, 16, 3072, 2048], "model": "Canon EOS 300D DIGITAL", "exif_orientation": 8, "capture_unix": 1182188238, + "wb": [2.230908, 1.0, 1.1737823], "black": [124.8050, 124.1321, 125.1695, 124.2802], "white": 3992.0, + "samples_sha256": "b1b61526c67948b600a03a89df0a301bc220f303bf8b70c93ea657356875a715" + } + }, + { + "name": "crw-canon-eos-digital-rebel.crw", + "body": "Canon EOS Digital Rebel", + "source": "https://raw.pixls.us/getfile.php/1308/nice/Canon%20-%20EOS%20Digital%20Rebel%20-%20RAW%20(3:2).CRW", + "sha256": "d6a3131bfac390fde01b96882ec378211f04570c3252708b58f0d05b33a9fdfc", + "license": "CC0-1.0", + "expect": { + "width": 3152, "height": 2068, "bits": 12, "selector": 0, "cfa": "RGGB", + "active": [72, 16, 3072, 2048], "model": "Canon EOS DIGITAL REBEL", "exif_orientation": 1, "capture_unix": 1480372394, + "wb": [1.6094965, 1.0, 1.676627], "black": [262.0760, 260.0167, 261.7442, 260.0508], "white": 4095.0, + "samples_sha256": "ec487fc432a31a648fc8238452976fd96115ff058bc6989352bdebdf997d99aa" + } + }, + { + "name": "crw-canon-powershot-g1.crw", + "body": "Canon PowerShot G1", + "source": "https://raw.pixls.us/getfile.php/2073/nice/Canon%20-%20PowerShot%20G1%20-%20RAW%20(4:3).crw", + "sha256": "f39be5acf14057ce04527fd8d0574e57ec8ed5fed082d7bfaff763075a44ee8a", + "license": "CC0-1.0", + "expect": { + "width": 2144, "height": 1560, "bits": 10, "selector": 1, "cfa": "GBRG", + "active": [24, 15, 2048, 1536], "model": "Canon PowerShot G1", "exif_orientation": 1, "capture_unix": 1065594172, + "wb": null, "black": [28.7314, 28.7637, 28.7544, 28.6963], "white": 1023.0, + "samples_sha256": "2316b72fd8d84cff2abbac1e56d9a6ea76997ee9a4182a040d9e261735393ba2" + } + }, + { + "name": "crw-canon-powershot-g5.crw", + "body": "Canon PowerShot G5", + "source": "https://raw.pixls.us/getfile.php/1746/nice/Canon%20-%20PowerShot%20G5%20-%20RAW%20(4:3).CRW", + "sha256": "bb084223d714871b99de4f549eaf00513e30a7080feeead700252c2008060196", + "license": "CC0-1.0", + "expect": "unsupported" + }, + { + "name": "crw-canon-powershot-g6.crw", + "body": "Canon PowerShot G6", + "source": "https://raw.pixls.us/getfile.php/4226/nice/Canon%20-%20PowerShot%20G6%20-%20RAW%20(4:3).CRW", + "sha256": "244378e1f5660b6cc6cefc93a75ed4928d0b28aa6304ff327b61c748538f4fef", + "license": "CC0-1.0", + "expect": { + "width": 3160, "height": 2344, "bits": 12, "selector": 0, "cfa": "RGGB", + "active": [64, 24, 3072, 2304], "model": "Canon PowerShot G6", "exif_orientation": 1, "capture_unix": 1607955982, + "wb": null, "black": [128.9415, 128.9516, 129.0142, 129.0901], "white": 4080.0, + "samples_sha256": "17804ed2ff64873e7d24ca06c58a4c8d095aad1907affb938ebcd12b618fa7ac" + } + } +] diff --git a/crates/crw/tests/samples.rs b/crates/crw/tests/samples.rs new file mode 100644 index 00000000..de926af1 --- /dev/null +++ b/crates/crw/tests/samples.rs @@ -0,0 +1,229 @@ +//! The CC0 sample files (raw.pixls.us) pinned in `samples.json`: each decodes to exactly the +//! sensor samples the derivation verified against the black-box reference (`clean-room/ +//! 07-verification.md`), recorded as the SHA-256 of the little-endian sample buffer, with the +//! levels, white balance and metadata the decoder read then. +//! +//! The files are not in the repository (5–11 MB each): `python scripts/fetch-crw-samples.py +//! ` downloads and verifies them, and `CRAFT_CRW_SAMPLES=` points the test at them. +//! Without the variable the test prints that it skipped and passes; `CRAFT_CRW_REQUIRE_SAMPLES=1` +//! (CI) turns the skip into a failure. +//! +//! `CRAFT_CRW_ORACLE=` additionally compares every sample against `.crw.tiff` +//! references next to the files (16-bit grey TIFFs of the whole sensor, as written by the +//! reference decoder's `unprocessed_raw -T`), the check the derivation used. Those references +//! are not distributable and live outside the repository. + +use craft_crw::{Error, Limits, decode, probe}; +use serde::Deserialize; +use sha2::{Digest, Sha256}; +use std::path::{Path, PathBuf}; + +#[derive(Deserialize)] +struct Sample { + name: String, + body: String, + sha256: String, + expect: Expect, +} + +#[derive(Deserialize)] +#[serde(untagged)] +enum Expect { + Unsupported(String), + Decoded(Decoded), +} + +#[derive(Deserialize)] +struct Decoded { + width: usize, + height: usize, + bits: u32, + selector: u32, + cfa: String, + active: [usize; 4], + model: String, + exif_orientation: u8, + capture_unix: i64, + wb: Option<[f32; 3]>, + black: [f32; 4], + white: f32, + samples_sha256: String, +} + +fn manifest() -> Vec { + serde_json::from_str(include_str!("samples.json")).expect("samples.json parses") +} + +fn hex(digest: impl AsRef<[u8]>) -> String { + digest.as_ref().iter().map(|b| format!("{b:02x}")).collect() +} + +fn samples_dir() -> Option { + match std::env::var_os("CRAFT_CRW_SAMPLES") { + Some(dir) => Some(PathBuf::from(dir)), + None if std::env::var_os("CRAFT_CRW_REQUIRE_SAMPLES").is_some_and(|v| v == "1") => { + panic!("CRAFT_CRW_REQUIRE_SAMPLES=1 but CRAFT_CRW_SAMPLES is unset") + } + None => { + eprintln!("CRAFT_CRW_SAMPLES not set; skipping the sample-file tests (scripts/fetch-crw-samples.py)"); + None + } + } +} + +#[test] +fn manifest_is_well_formed() { + let m = manifest(); + assert_eq!(m.len(), 11); + for s in &m { + assert!(s.name.ends_with(".crw") && !s.name.contains('/') && !s.name.contains('\\'), "{}", s.name); + assert_eq!(s.sha256.len(), 64, "{}", s.name); + if let Expect::Decoded(d) = &s.expect { + assert_eq!(d.samples_sha256.len(), 64, "{}", s.name); + assert!(matches!(d.bits, 10 | 12), "{}", s.name); + } else { + assert!(s.body.contains("G5"), "only the PowerShot G5 is refused: {}", s.name); + } + } +} + +#[test] +fn sample_files_decode_to_the_verified_samples() { + let Some(dir) = samples_dir() else { return }; + let mut checked = 0; + for s in manifest() { + let path = dir.join(&s.name); + let bytes = std::fs::read(&path).unwrap_or_else(|e| panic!("{}: {e}", path.display())); + assert_eq!(hex(Sha256::digest(&bytes)), s.sha256, "{}: not the pinned CC0 file", s.name); + assert!(probe(&bytes), "{}: not recognised", s.name); + let started = std::time::Instant::now(); + let result = decode(&bytes, &Limits::default()); + let took = started.elapsed(); + match (&s.expect, result) { + (Expect::Unsupported(kind), Err(Error::Unsupported(why))) => { + assert_eq!(kind, "unsupported", "{}", s.name); + assert!(why.contains("G5"), "{}: {why}", s.name); + eprintln!("{}: refused as documented ({why})", s.name); + } + (Expect::Unsupported(_), other) => panic!("{}: expected Unsupported, got {other:?}", s.name.clone()), + (Expect::Decoded(d), Ok(img)) => { + let h = &img.header; + assert_eq!((h.width, h.height, h.bits, h.selector), (d.width, d.height, d.bits, d.selector), "{}", s.name); + assert_eq!(h.cfa.name(), d.cfa, "{}", s.name); + assert_eq!([h.active_area.x, h.active_area.y, h.active_area.width, h.active_area.height], d.active, "{}", s.name); + assert_eq!(h.model.as_deref(), Some(d.model.as_str()), "{}", s.name); + assert_eq!(h.exif_orientation, d.exif_orientation, "{}", s.name); + assert_eq!(h.capture_unix, Some(d.capture_unix), "{}", s.name); + match (h.wb_multipliers, d.wb) { + (None, None) => {} + (Some(got), Some(want)) => { + assert!(got.iter().zip(&want).all(|(a, b)| (a - b).abs() < 1e-4), "{}: wb {got:?} vs {want:?}", s.name) + } + (got, want) => panic!("{}: wb {got:?} vs {want:?}", s.name), + } + assert!( + img.black.values.iter().zip(&d.black).all(|(a, b)| (a - b).abs() < 0.01), + "{}: black {:?} vs {:?}", + s.name, + img.black.values, + d.black + ); + assert_eq!(img.white, d.white, "{}", s.name); + assert_eq!(img.samples.len(), d.width * d.height, "{}", s.name); + let mut hasher = Sha256::new(); + for v in &img.samples { + hasher.update(v.to_le_bytes()); + } + assert_eq!(hex(hasher.finalize()), d.samples_sha256, "{}: the decoded samples differ from the verified decode", s.name); + eprintln!("{}: {}x{} {}-bit selector {} EXACT in {took:.0?}", s.name, h.width, h.height, h.bits, h.selector); + } + (Expect::Decoded(_), Err(e)) => panic!("{}: {e}", s.name), + } + checked += 1; + } + assert_eq!(checked, 11); +} + +/// Minimal reader for the reference TIFFs: one 16-bit grey image, stripped, either byte order. +fn oracle_samples(bytes: &[u8]) -> (usize, usize, Vec) { + let be = match &bytes[..4] { + b"II*\0" => false, + b"MM\0*" => true, + other => panic!("not a TIFF: {other:?}"), + }; + let u16_at = |at: usize| -> u16 { + let b: [u8; 2] = bytes[at..at + 2].try_into().unwrap(); + if be { u16::from_be_bytes(b) } else { u16::from_le_bytes(b) } + }; + let u32_at = |at: usize| -> u32 { + let b: [u8; 4] = bytes[at..at + 4].try_into().unwrap(); + if be { u32::from_be_bytes(b) } else { u32::from_le_bytes(b) } + }; + let ifd = u32_at(4) as usize; + let n = u16_at(ifd) as usize; + let (mut w, mut h, mut bits, mut spp) = (0usize, 0usize, 0u16, 1u16); + let (mut offsets, mut counts) = (Vec::new(), Vec::new()); + for k in 0..n { + let e = ifd + 2 + k * 12; + let (tag, typ, cnt) = (u16_at(e), u16_at(e + 2), u32_at(e + 4) as usize); + let size = match typ { + 3 => 2, + 4 => 4, + _ => 1, + }; + let values_at = if cnt * size <= 4 { e + 8 } else { u32_at(e + 8) as usize }; + let value = |i: usize| -> usize { if typ == 3 { u16_at(values_at + i * 2) as usize } else { u32_at(values_at + i * 4) as usize } }; + match tag { + 256 => w = value(0), + 257 => h = value(0), + 258 => bits = value(0) as u16, + 277 => spp = value(0) as u16, + 273 => offsets = (0..cnt).map(value).collect(), + 279 => counts = (0..cnt).map(value).collect(), + _ => {} + } + } + assert_eq!((bits, spp), (16, 1), "oracle TIFF is 16-bit grey"); + let mut data = Vec::with_capacity(w * h); + for (&o, &c) in offsets.iter().zip(&counts) { + let strip = &bytes[o..o + c]; + data.extend(strip.as_chunks::<2>().0.iter().map(|p| if be { u16::from_be_bytes(*p) } else { u16::from_le_bytes(*p) })); + } + data.truncate(w * h); + (w, h, data) +} + +#[test] +fn oracle_tiffs_match_every_sample() { + let Some(dir) = std::env::var_os("CRAFT_CRW_ORACLE").map(PathBuf::from) else { + eprintln!("CRAFT_CRW_ORACLE not set; skipping the reference-TIFF comparison"); + return; + }; + let mut entries: Vec = std::fs::read_dir(&dir).expect("oracle directory").filter_map(|e| e.ok()).map(|e| e.path()).collect(); + entries.sort(); + let mut checked = 0; + for path in entries.iter().filter(|p| p.extension().and_then(|e| e.to_str()).is_some_and(|e| e.eq_ignore_ascii_case("crw"))) { + let name = path.file_name().and_then(|n| n.to_str()).unwrap_or_default(); + let reference: &Path = &path.with_extension("crw.tiff"); + if !reference.is_file() { + eprintln!("{name}: no reference TIFF, skipped"); + continue; + } + let bytes = std::fs::read(path).expect("read crw"); + let img = match decode(&bytes, &Limits::default()) { + Ok(img) => img, + Err(Error::Unsupported(why)) if why.contains("G5") => { + eprintln!("{name}: G5 refused as documented"); + continue; + } + Err(e) => panic!("{name}: {e}"), + }; + let (w, h, oracle) = oracle_samples(&std::fs::read(reference).expect("read oracle tiff")); + assert_eq!((img.header.width, img.header.height), (w, h), "{name}: dimensions"); + let mismatches = img.samples.iter().zip(&oracle).filter(|(a, b)| a != b).count(); + assert_eq!(mismatches, 0, "{name}: {mismatches} mismatching samples"); + eprintln!("{name}: EXACT ({w}x{h})"); + checked += 1; + } + assert!(checked > 0, "no .crw files with references found in {}", dir.display()); +} diff --git a/docs/clean-room.md b/docs/clean-room.md new file mode 100644 index 00000000..cc8c675f --- /dev/null +++ b/docs/clean-room.md @@ -0,0 +1,85 @@ +# Clean-room format work in craft-libs + +Several crates here read or write formats that other open-source projects also implement, often +under licences the Craft apps cannot take code from (GPL, LGPL, AGPL, CDDL) or under terms that +forbid it (Adobe's). This document is the standard a format crate follows so that every line of +it is ours, and so that a reviewer can check that claim from the repository alone. It is the +written form of what `craft-jpeg`, `craft-media` and `craft-crw` did; the suite-wide rules live in +[craftrules](https://github.com/storytold/craftrules) (`standards/licensing.md`, +`standards/never-crash.md`), which this does not replace. + +## What "clean-room" means here + +A clean-room crate is written only from: + +- **published specifications and standards** (ITU-T, ISO/IEC, IETF RFCs, a vendor's own published + format document, a registry's pages), read as documents; +- **prose documentation** of a format or its tags (ExifTool's tag-name and format pages, a wiki + page that describes a container), never a project's code; +- **sample files** we may use: CC0, or owned by a contributor who consents, or produced by us. + Not NC-licensed files, not files whose licence is unclear, never anything from inside a + proprietary application's bundle; +- **a reference implementation run as a black box**: an official pre-built binary of another + decoder, fed inputs and observed for outputs, with every run logged. Its source is never + opened, read, searched, diffed or quoted, including comments, licence headers, mirrors and + excerpts in forum posts or AI answers. + +Never consulted, in any form: GPL/LGPL/AGPL/CDDL decoder and encoder sources (dcraw, LibRaw, +rawspeed, rawloader/rawler, dnglab, darktable, RawTherapee, ART, libopenraw, lensfun, ExifTool's +Perl, FFmpeg/libav, x264/x265 and the like), Adobe's SDKs, profiles (DCP, LCP), presets, fonts, +icons or binaries, and any earlier attempt of ours that was itself not clean. + +## The process (what the record must show) + +Modelled on LightCraft's ARW compression derivation and repeated for CRW +(`crates/crw/clean-room/`). A crate that derives an undocumented format commits a `clean-room/` +folder with: + +1. **Starting assumptions**, written before any source is consulted, every item marked + unverified, committed once and never edited. A reviewer diffs them against the result to see + what was assumed and what was established. +2. **A plan** naming the sources that will be used, the sample corpus (public files with URL, + licence and SHA-256; private files by name, size and hash only, never content), the phases, + the freeze point, and the questions that are the maintainer's to decide. Work past the plan + waits for the maintainer's recorded approval. +3. **A source log**: every URL opened and every search run, in order, with the time, what was + asked and what was taken. If a page exposed implementation details that were not asked for, + the log says so and what it said. +4. **Derivation documents** in which every fact cites the experiment (script and log) or the + document that established it, and corrections are stated, not silently applied. Statistics + before decoding; hypotheses tested against internal consistency first; black-box probes + logged one by one. +5. **A freeze**: one document holding the complete derived format, with its tables as data. Its + commit is the freeze point. Nothing in it is tuned afterwards. A mismatch found later reopens + the derivation *explicitly*: the cause, the change and a new freeze revision are recorded in + that document. +6. **Verification** after the freeze, with zero tuning: every sample compared exactly against the + reference, every file's verdict recorded, failures recorded as failures. Cross-body checks + (tables learned on one body, exact on another) are the strongest independent evidence. +7. **The implementation** from the frozen document, with: never-crash rules (`get()` for every + file-derived offset, checked arithmetic, bounded recursion and allocation, no `unwrap`/`panic`), + synthetic tests that need no sample files (a test-only encoder that inverts the model is the + usual tool), hostile-input tests, and a sample-file test pinning the verified decode (a digest + of the output per file) so the crate can prove it still matches without redistributing the + references. +8. **Table data** lives in the record as JSON and is rendered into Rust by a committed script; the + generated file says so and is never edited by hand. + +Scripts in the record use the Python standard library only and run under `python -I`. Local-only +material (reference models, oracle binaries, copyrighted specification PDFs, dumps that may carry +personal metadata) is named in the record with how to obtain it, and is gitignored. + +## Assets and attribution + +Sample files are never committed (sizes aside, they are other people's photographs): tests +download them outside the checkout by pinned URL and SHA-256, licence checked (`scripts/ +fetch-*.py`), and skip cleanly without them. Each manifest, each table derived from samples and +each log set has a row in `ATTRIBUTION.md`; code and records that came from an app are listed in +`NOTICE` under *Source provenance* with the repository, paths and revision. + +## When something is not clean + +Stop, record what was seen and where (the source log), and tell the maintainer. Code written +afterwards by the same person for the same part of the format is suspect; the maintainer decides +whether the part is re-derived by someone else, documented as derived from that source under its +licence terms, or dropped. Never quietly continue. diff --git a/scripts/fetch-crw-samples.py b/scripts/fetch-crw-samples.py new file mode 100644 index 00000000..281d9509 --- /dev/null +++ b/scripts/fetch-crw-samples.py @@ -0,0 +1,93 @@ +#!/usr/bin/env python3 +"""Fetch the pinned CC0 Canon CRW sample files for craft-crw's tests, outside the checkout. + +The manifest is crates/crw/tests/samples.json (name, raw.pixls.us URL, SHA-256, licence). Files +already present with the right hash are kept. Then run the tests with +`CRAFT_CRW_SAMPLES= cargo test -p craft-crw --release --test samples`. +""" + +import argparse +import hashlib +import json +import os +from pathlib import Path +import tempfile +import time +import urllib.error +import urllib.parse +import urllib.request + +MAX_BYTES = 16 * 1024 * 1024 + + +class _HttpsOnlyRedirectHandler(urllib.request.HTTPRedirectHandler): + def redirect_request(self, req, fp, code, msg, headers, newurl): + if urllib.parse.urlsplit(newurl).scheme != "https": + raise ValueError("refusing to follow a non-HTTPS redirect") + return super().redirect_request(req, fp, code, msg, headers, newurl) + + +urllib.request.install_opener(urllib.request.build_opener(_HttpsOnlyRedirectHandler)) + + +def fetch(url): + if urllib.parse.urlsplit(url).scheme != "https": + raise ValueError("refusing to fetch a non-HTTPS URL") + for attempt in range(3): + try: + with urllib.request.urlopen(url, timeout=120) as response: + return response.read(MAX_BYTES + 1) + except urllib.error.HTTPError as error: + if error.code not in (408, 429) and not 500 <= error.code <= 599: + raise + if attempt == 2: + raise + error.close() + except (urllib.error.URLError, TimeoutError, ConnectionError): + if attempt == 2: + raise + time.sleep(attempt + 1) + raise RuntimeError("sample download retries exhausted") + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("destination", type=Path) + args = parser.parse_args() + root = Path(__file__).resolve().parent.parent + destination = args.destination.resolve() + if destination == root or root in destination.parents: + parser.error("sample files must be stored outside the checkout (they are never committed)") + samples = json.loads((root / "crates/crw/tests/samples.json").read_text(encoding="utf-8")) + destination.mkdir(parents=True, exist_ok=True) + for sample in samples: + name, url, digest, licence = sample["name"], sample["source"], sample["sha256"], sample["license"] + if Path(name).name != name or name in ("", ".", "..") or not url.startswith("https://"): + raise ValueError("invalid sample name or source") + if licence != "CC0-1.0": + raise ValueError(f"{name}: only CC0 samples are fetched (manifest says {licence})") + path = (destination / name).resolve() + if path.parent != destination: + raise ValueError(f"sample path escapes destination: {name}") + if path.is_file() and hashlib.sha256(path.read_bytes()).hexdigest() == digest: + print(f"Verified {name}") + continue + data = fetch(url) + if len(data) > MAX_BYTES or hashlib.sha256(data).hexdigest() != digest: + raise ValueError(f"sample size or SHA-256 mismatch: {name}") + fd, tmp = tempfile.mkstemp(dir=destination, prefix=f".{name}.", suffix=".tmp") + try: + with os.fdopen(fd, "wb") as handle: + handle.write(data) + os.replace(tmp, destination / name) + except BaseException: + try: + os.unlink(tmp) + except OSError: + pass + raise + print(f"Fetched and verified {name}") + + +if __name__ == "__main__": + main() From 4a4863fd9486ddf522187a12f70f71874660a0f5 Mon Sep 17 00:00:00 2001 From: RWoldberg <314159+rwoldberg@users.noreply.github.com> Date: Mon, 26 Oct 2026 08:21:27 -0600 Subject: [PATCH 02/10] craft-crw: cite the LightCraft branch tip in the provenance notes The branch gained review commits after the port (chrono for a date conversion that this crate does not contain, a test skip macro); the decoder and the clean-room record are the same at both revisions, so the notes name the tip and the ported revision. Co-Authored-By: Claude Fable 5.1 --- NOTICE | 3 ++- crates/crw/README.md | 6 ++++-- crates/crw/clean-room/README.md | 2 +- 3 files changed, 7 insertions(+), 4 deletions(-) diff --git a/NOTICE b/NOTICE index 59cb0f0a..0f7c4417 100644 --- a/NOTICE +++ b/NOTICE @@ -205,7 +205,8 @@ The Canon CRW decoder in crates/crw (src/lib.rs, src/tables.rs, src/tests.rs, te derivation record (crates/crw/clean-room/) come from: - LightCraft, crates/raw/src/vendor/crw.rs, crates/raw/src/vendor/crw_tables.rs, crates/raw/tests/crw_oracle.rs, crates/raw/tests/it/vendor_robust.rs and clean-room/crw/, - branch clean-room-crw-format-decode, revision 70c755e42a65661401142819278d51b0c5977f7f. + branch clean-room-crw-format-decode, revision 6ca439b5155820d2befeacd81ec0c6a80b2bd3bc + (decoder ported at 70c755e42a65661401142819278d51b0c5977f7f; the record is identical at both). https://github.com/storytold/lightcraft Copyright (c) 2026 ArtCraft Team and the LightCraft contributors. Its white-level plateau estimate (saturation_plateau) is LightCraft's vendor::white_from_data at the diff --git a/crates/crw/README.md b/crates/crw/README.md index c7c27bbe..51fee5c8 100644 --- a/crates/crw/README.md +++ b/crates/crw/README.md @@ -125,8 +125,10 @@ About 60 ms for a 6.5 MP EOS file and 70 ms for the 7.4 MP G6 in release mode on ## Origin The decoder was written in LightCraft as `crates/raw/src/vendor/crw.rs` and `crw_tables.rs` -(branch `clean-room-crw-format-decode`, revision `70c755e42a65661401142819278d51b0c5977f7f`, -[storytold/lightcraft](https://github.com/storytold/lightcraft)) and moved here with its tests and +(branch `clean-room-crw-format-decode`, revision `6ca439b5155820d2befeacd81ec0c6a80b2bd3bc`, +[storytold/lightcraft](https://github.com/storytold/lightcraft); the decoder itself was ported at +`70c755e4`, and the branch's later review commits changed only a date conversion that is not part +of this crate) and moved here with its tests and its derivation record. The move replaced LightCraft's types with this crate's own (`Error`, `Limits`, `Cfa`, `Rect`, `BlackLevel`, `Header`, `Image`), exposed the capture time as Unix seconds instead of converting it in the decoder, and copied LightCraft's shared plateau estimate diff --git a/crates/crw/clean-room/README.md b/crates/crw/clean-room/README.md index e598f751..e50f822e 100644 --- a/crates/crw/clean-room/README.md +++ b/crates/crw/clean-room/README.md @@ -2,7 +2,7 @@ Everything that establishes how `craft-crw` decodes a CRW, in the order it happened. The record was produced in LightCraft (`clean-room/crw/`, branch `clean-room-crw-format-decode`, revision -`70c755e42a65661401142819278d51b0c5977f7f`) between 2026-10-12 and 2026-10-25 and moved here +`6ca439b5155820d2befeacd81ec0c6a80b2bd3bc`; the record is identical at `70c755e4`, where the decoder was ported) between 2026-10-12 and 2026-10-25 and moved here **verbatim** when the decoder became this shared crate. The documents are the evidence: they are not rewritten after the fact, so paths they mention are LightCraft's paths of that time (the mapping is below), and `00-starting-assumptions.md` is committed once and never edited. From db0fe2045c02caf4e8146d0689e0b2538194c7b0 Mon Sep 17 00:00:00 2001 From: RWoldberg <314159+rwoldberg@users.noreply.github.com> Date: Mon, 26 Oct 2026 09:53:53 -0600 Subject: [PATCH 03/10] Review fixes --- crates/crw/README.md | 2 +- crates/crw/clean-room/06-crw-format.md | 4 ++-- crates/crw/src/lib.rs | 10 +++------- crates/crw/tests/samples.rs | 2 ++ 4 files changed, 8 insertions(+), 10 deletions(-) diff --git a/crates/crw/README.md b/crates/crw/README.md index 51fee5c8..bf987c9a 100644 --- a/crates/crw/README.md +++ b/crates/crw/README.md @@ -86,7 +86,7 @@ if craft_crw::probe(&bytes) { code), `Unsupported(String)` (an unknown decoder table, or the PowerShot G5) or `LimitExceeded(&str)`. Every offset read from the file is bounds-checked, the heap walk is bounded by depth (16) and by a per-file record budget (4,096), and `width × height` is checked against -`Limits::max_samples` (default 2³⁰ on 64-bit targets, 2²⁶ on wasm32; real frames hold at most +`Limits::max_samples` (default 2²³, about 8.4 million, on every target; real frames hold at most 7.4 million) before anything is allocated. The decoder holds about three sample-sized `u16` buffers at its peak. `Limits` is `#[non_exhaustive]`: start from `Limits::default()` and set the fields you change. diff --git a/crates/crw/clean-room/06-crw-format.md b/crates/crw/clean-room/06-crw-format.md index e3249f70..b61ecf94 100644 --- a/crates/crw/clean-room/06-crw-format.md +++ b/crates/crw/clean-room/06-crw-format.md @@ -27,9 +27,9 @@ RawData (CIFF record 0x2005), with w×h from SensorInfo (0x1031) and - **Low-bits plane**: 2 bits per sensor sample, row-major, LSB-first within each byte (sample 4i in bits 1:0 … sample 4i+3 in bits 7:6). Present ⇔ either - (a) the first w·h/4 + 514 bytes of the payload contain any 0xff byte NOT followed by + (a) the first ⌈w·h/4⌉ + 514 bytes of the payload contain any 0xff byte NOT followed by 0x00 (the entropy stream is always stuffed, the low-bits plane is raw), or - (b) the 514 bytes at offset w·h/4 are all zero (the zero header sits right after the + (b) the 514 bytes at offset ⌈w·h/4⌉ are all zero (the zero header sits right after the plane; without a plane that offset is deep in entropy data). Rule (a) alone, the revision-1 rule, misses planes that hold no 0xff byte: some D60 files have a lowest bit that is almost always 0 (low values 1 and 3 on about 0.04 % of diff --git a/crates/crw/src/lib.rs b/crates/crw/src/lib.rs index a115d5e0..4eb4d925 100644 --- a/crates/crw/src/lib.rs +++ b/crates/crw/src/lib.rs @@ -60,18 +60,14 @@ pub enum Error { #[non_exhaustive] pub struct Limits { /// Most sensor samples (`width × height`) a file may declare. The decoder holds about three - /// `u16` buffers of this size at its peak. Default: 2³⁰ on 64-bit targets, 2²⁶ on 32-bit - /// ones (wasm32). Real CRW frames hold at most 7.4 million samples. + /// `u16` buffers of this size at its peak. Default: 2²³ (8.4 million) on every target; real + /// CRW frames hold at most 7.4 million samples. pub max_samples: usize, } impl Default for Limits { fn default() -> Self { - #[cfg(target_pointer_width = "64")] - let max_samples = 1 << 30; - #[cfg(not(target_pointer_width = "64"))] - let max_samples = 1 << 26; - Self { max_samples } + Self { max_samples: 1 << 23 } } } diff --git a/crates/crw/tests/samples.rs b/crates/crw/tests/samples.rs index de926af1..71828438 100644 --- a/crates/crw/tests/samples.rs +++ b/crates/crw/tests/samples.rs @@ -220,6 +220,8 @@ fn oracle_tiffs_match_every_sample() { }; let (w, h, oracle) = oracle_samples(&std::fs::read(reference).expect("read oracle tiff")); assert_eq!((img.header.width, img.header.height), (w, h), "{name}: dimensions"); + assert_eq!(oracle.len(), w * h, "{name}: oracle sample count"); + assert_eq!(img.samples.len(), w * h, "{name}: decoded sample count"); let mismatches = img.samples.iter().zip(&oracle).filter(|(a, b)| a != b).count(); assert_eq!(mismatches, 0, "{name}: {mismatches} mismatching samples"); eprintln!("{name}: EXACT ({w}x{h})"); From 70decab20727da1c8310d8ec2fb8240c13cd92e6 Mon Sep 17 00:00:00 2001 From: RWoldberg <314159+rwoldberg@users.noreply.github.com> Date: Mon, 26 Oct 2026 10:24:43 -0600 Subject: [PATCH 04/10] Test update --- crates/crw/src/tests.rs | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/crates/crw/src/tests.rs b/crates/crw/src/tests.rs index f78e19b8..06bc9ffa 100644 --- a/crates/crw/src/tests.rs +++ b/crates/crw/src/tests.rs @@ -237,6 +237,18 @@ fn limits_cap_the_sample_count() { assert!(decode(&f, &limits).is_ok()); } +/// A header declaring 2²⁴ samples over a 128-sample stream is refused by the default limits +/// before any sample-sized buffer exists: the cap must stay above every real frame (7.4 million) +/// and below anything that would let a crafted file demand gigabytes. +#[test] +fn default_limits_refuse_a_crafted_giant_frame() { + let hi = test_hi(128); + let f = roundtrip_file(4096, 4096, "Canon EOS D60", None, &hi, Order::Le); + assert_eq!(decode(&f, &Limits::default()), Err(Error::LimitExceeded("image too large"))); + assert_eq!(header(&f, &Limits::default()), Err(Error::LimitExceeded("image too large"))); + assert!(Limits::default().max_samples >= 7_407_040, "the cap must admit the largest real frame"); +} + #[test] fn g1_is_gbrg() { let hi = test_hi(128); From 536fa7391158f0ed2095d6bddd9d4109209e0b43 Mon Sep 17 00:00:00 2001 From: RWoldberg <314159+rwoldberg@users.noreply.github.com> Date: Mon, 26 Oct 2026 10:28:38 -0600 Subject: [PATCH 05/10] Remove line --- crates/crw/src/lib.rs | 1 - 1 file changed, 1 deletion(-) diff --git a/crates/crw/src/lib.rs b/crates/crw/src/lib.rs index 4eb4d925..f09cd20b 100644 --- a/crates/crw/src/lib.rs +++ b/crates/crw/src/lib.rs @@ -356,7 +356,6 @@ fn parse(bytes: &[u8], limits: &Limits, full: bool) -> Result { let has_low = header_after_plane || head.windows(2).any(|p| p[0] == 0xff && p[1] != 0x00); let bits = if has_low { 12 } else { 10 }; let low_len = if has_low { q } else { 0 }; - let samples = if full { let entropy = payload.get(low_len.saturating_add(ent_off)..).ok_or_else(|| corrupt("payload shorter than its header"))?; let low = has_low.then(|| payload.get(..q)).flatten(); From 903b816bdfa22f2b415f9639aef85ddb19862edc Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Mon, 26 Oct 2026 17:40:25 -0400 Subject: [PATCH 06/10] crw: keep the format record, drop the derivation trail The crate's record should hold what the decoder is built from, not the process that produced it. The starting assumptions, the plan, the container notes, the entropy-derivation log, the chart calibration, the probe scripts and their observational logs are not part of it. The frozen-table JSONs and tools/gen_tables.py stay, so src/tables.rs can still be regenerated and audited. Renumber the surviving documents 01-05 in reading order and update every reference to them, so the record reads in order and cites nothing that is no longer there. AGENTS.md and CLAUDE.md go back to main: repository instructions are not part of this change. --- AGENTS.md | 49 +- ATTRIBUTION.md | 4 +- CLAUDE.md | 7 +- crates/crw/README.md | 6 +- .../crw/clean-room/00-starting-assumptions.md | 111 --- ...payload-layout.md => 01-payload-layout.md} | 21 +- crates/crw/clean-room/01-plan.md | 337 ------- crates/crw/clean-room/02-container.md | 119 --- .../{06-crw-format.md => 02-crw-format.md} | 12 +- ...{07-verification.md => 03-verification.md} | 12 +- .../clean-room/{08-color.md => 04-color.md} | 6 +- crates/crw/clean-room/04-derivation-log.md | 315 ------- ...private-corpus.md => 05-private-corpus.md} | 6 +- crates/crw/clean-room/10-chart-calibration.md | 97 -- crates/crw/clean-room/README.md | 47 +- crates/crw/clean-room/ciff_dump.py | 197 ---- crates/crw/clean-room/logs/chart-captures.tsv | 39 - .../crw/clean-room/logs/p1-payload-public.txt | 195 ---- .../clean-room/logs/p1b-markers-public.txt | 109 --- .../crw/clean-room/logs/p1c-bounds-public.txt | 11 - .../clean-room/logs/structure-d60-public.txt | 73 -- .../crw/clean-room/logs/structure-public.txt | 887 ------------------ .../clean-room/logs/verification-sweep.txt | 22 - crates/crw/clean-room/p1_payload.py | 184 ---- crates/crw/clean-room/p1b_markers.py | 119 --- crates/crw/clean-room/p1c_lowbits_bounds.py | 96 -- crates/crw/clean-room/p2_oracle_lsb.py | 118 --- crates/crw/clean-room/sources.md | 28 +- crates/crw/clean-room/tools/gen_tables.py | 2 +- crates/crw/src/lib.rs | 8 +- crates/crw/src/tables.rs | 2 +- crates/crw/tests/samples.rs | 2 +- 32 files changed, 85 insertions(+), 3156 deletions(-) delete mode 100644 crates/crw/clean-room/00-starting-assumptions.md rename crates/crw/clean-room/{03-payload-layout.md => 01-payload-layout.md} (88%) delete mode 100644 crates/crw/clean-room/01-plan.md delete mode 100644 crates/crw/clean-room/02-container.md rename crates/crw/clean-room/{06-crw-format.md => 02-crw-format.md} (94%) rename crates/crw/clean-room/{07-verification.md => 03-verification.md} (91%) rename crates/crw/clean-room/{08-color.md => 04-color.md} (97%) delete mode 100644 crates/crw/clean-room/04-derivation-log.md rename crates/crw/clean-room/{09-private-corpus.md => 05-private-corpus.md} (95%) delete mode 100644 crates/crw/clean-room/10-chart-calibration.md delete mode 100644 crates/crw/clean-room/ciff_dump.py delete mode 100644 crates/crw/clean-room/logs/chart-captures.tsv delete mode 100644 crates/crw/clean-room/logs/p1-payload-public.txt delete mode 100644 crates/crw/clean-room/logs/p1b-markers-public.txt delete mode 100644 crates/crw/clean-room/logs/p1c-bounds-public.txt delete mode 100644 crates/crw/clean-room/logs/structure-d60-public.txt delete mode 100644 crates/crw/clean-room/logs/structure-public.txt delete mode 100644 crates/crw/clean-room/logs/verification-sweep.txt delete mode 100644 crates/crw/clean-room/p1_payload.py delete mode 100644 crates/crw/clean-room/p1b_markers.py delete mode 100644 crates/crw/clean-room/p1c_lowbits_bounds.py delete mode 100644 crates/crw/clean-room/p2_oracle_lsb.py diff --git a/AGENTS.md b/AGENTS.md index 455051e3..e5adca3b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,54 +1,15 @@ # AGENTS.md -Shared Rust libraries for the Craft Apps (LightCraft, PhotoCraft, PdfCraft, VectorCraft, FilmCraft, -DesignCraft, WordCraft and the others). The shared standards live in +Shared Rust libraries for the Craft Apps. The shared standards live in [craftrules](https://github.com/storytold/craftrules) (`../../craftrules` on a dev machine): read its `AGENTS.md`, then `standards/never-crash.md`, `standards/engineering.md` and `standards/licensing.md`. -## Start every session here - -1. Read this file, then the `README.md` of each crate you touch (its boundaries, origin and tests). -2. A crate that derives or reads a format (`jpeg`, `media`, `crw`, …) follows - [`docs/clean-room.md`](docs/clean-room.md); a crate with a `clean-room/` folder keeps that record - intact (its `README.md` says what is frozen and how a change is reopened). -3. Check `git status`, `git worktree list` and unmerged branches before starting anything new; commit - small and often. - -## Rules - - **Never crash.** Every crate's `lib.rs` starts with `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]`. - No `unsafe` (the workspace forbids it). Input-derived numbers are hostile: `get()` instead of - indexing for offsets that come from a file, checked or saturating arithmetic for lengths and - counts, bounded recursion (depth limits or budgets), allocations capped by a `Limits` the caller - can lower. Every crash fix lands with a regression test that panicked before it. + No `unsafe` (the workspace forbids it). - **New crates** go in `crates/`, named `craft-`, inherit the workspace package fields - and lints, and are listed in `[workspace.dependencies]`, in the root `README.md` (one entry per - library) and in a workflow of their own (`.github/workflows/.yml`: fmt, clippy with - `-D warnings`, tests on Linux, Windows and macOS, and a `wasm32-unknown-unknown` check for a - crate that an app's web build can use). + and lints, and are listed in `[workspace.dependencies]`. - **No app code.** A crate here must be useful to more than one app and must not depend on any app. - App policy (colour models, UI, settings, command handling) stays in the apps; the crate's README - states its boundaries and, when the code came from an app, its origin (repository, paths, - revision) and what changed in the move. `NOTICE` → *Source provenance* records the same. -- **Clean-room.** Format code is written only from published specifications, prose documentation, - usable sample files and black-box observation of reference binaries; never from GPL/LGPL/AGPL/CDDL - sources or Adobe material, in any form. The process and the record a crate keeps are in - [`docs/clean-room.md`](docs/clean-room.md). If you have seen something you should not have, stop - and say so. -- **Assets and samples.** Nothing enters the repository without a row in `ATTRIBUTION.md` (path, - author, source, licence) and a licence text where one is required (`licenses/`). Sample media is - never committed: tests fetch pinned, SHA-256-verified, licence-checked files outside the checkout - (`scripts/fetch-*.py`) and skip cleanly when they are absent. No Adobe assets of any kind, including - Adobe's open-licensed fonts; fonts come from craft-fonts. -- **Time goes through `chrono`** (workspace dependency). Never hand-roll UTC/calendar conversion (days-to-date algorithms, leap-year math); a crate owns only its output format string. The one exception is range extension: past the years chrono can represent, shift the date by whole 400-year Gregorian cycles (146,097 days, after which the calendar repeats), let chrono compute the month and day, and add the cycles back to the year (splitting seconds into hours, minutes and seconds is plain arithmetic). A library that reads a timestamp from a file hands it on as Unix seconds and lets the app convert. -- **Tests.** Unit and property tests next to the code; every feature has a test that would fail - without it; decoders have a hostile-input sweep (truncations, byte mutations, proptest) and, for - real files, a digest-pinned sample test. A test that cannot run (no sample files, no GPU) prints - why it skipped and passes, unless a `*_REQUIRE_*` variable (set in CI) turns the skip into a - failure. +- **Time goes through `chrono`** (workspace dependency). Never hand-roll UTC/calendar conversion (days-to-date algorithms, leap-year math); a crate owns only its output format string. The one exception is range extension: past the years chrono can represent, shift the date by whole 400-year Gregorian cycles (146,097 days, after which the calendar repeats), let chrono compute the month and day, and add the cycles back to the year (splitting seconds into hours, minutes and seconds is plain arithmetic). - **Before merging:** `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, - `cargo test --workspace`. Each crate's workflow runs the same for that crate on every PR. -- **Commits:** one change per commit, only green states, messages that say what and why. Apps - consume a crate by git revision (`craft- = { git = "https://github.com/storytold/craft-libs", rev = "" }`), - so a merged commit is an API contract: note breaking changes in the crate's README. + `cargo test --workspace`. diff --git a/ATTRIBUTION.md b/ATTRIBUTION.md index f3a6e899..9585c992 100644 --- a/ATTRIBUTION.md +++ b/ATTRIBUTION.md @@ -23,8 +23,8 @@ author, source and license. Library code and original assets are under the repos | `crates/ui/tests/fonts/NotoSansCJKsc-screens.otf` (subset to printable ASCII and the screen test strings) | The Noto Project Authors (Adobe, Google) | [notofonts/noto-cjk@f8d1575 `Sans/OTF/SimplifiedChinese`](https://github.com/notofonts/noto-cjk/tree/f8d157532fbfaeda587e826d4cd5b21a49186f7c/Sans/OTF/SimplifiedChinese), via craft-fonts | OFL-1.1 (`licenses/ui-test-fonts/OFL-NotoSansCJK.txt`) | | `crates/ui/tests/fonts/NotoSansThai-screens.ttf` (instance at weight 400 and width 100, subset to the screen test strings) | The Noto Project Authors | [google/fonts@8b0a1d0 `ofl/notosansthai`](https://github.com/google/fonts/tree/8b0a1d0f5983c89bc2b93f1b5fb55f9e252744b5/ofl/notosansthai), via craft-fonts | OFL-1.1 (`licenses/ui-test-fonts/OFL-NotoSansThai.txt`) | | `crates/crw/tests/samples.json` (manifest only; the 11 raw files are not bundled, `scripts/fetch-crw-samples.py` downloads them outside the checkout) | The photographers who contributed each file to raw.pixls.us (Canon EOS D30, D60, 10D ×3, 300D ×2, Digital Rebel, PowerShot G1, G5, G6) | [raw.pixls.us](https://raw.pixls.us/), each file's `getfile.php` URL and SHA-256 in the manifest | CC0-1.0 (as published by raw.pixls.us for each file) | -| `crates/crw/src/tables.rs`, `crates/crw/clean-room/logs/table_sel*.json` | craft-crw contributors (derived in LightCraft, 2026-10, by known-plaintext alignment of the sample files' entropy streams; `crates/crw/clean-room/06-crw-format.md`) | this repository | MIT OR Apache-2.0 | -| `crates/crw/clean-room/logs/*.txt`, `*.tsv` (statistics, file names, sizes and SHA-256 of sample files; no image content) | craft-crw contributors (LightCraft derivation record) | this repository | MIT OR Apache-2.0 | +| `crates/crw/src/tables.rs`, `crates/crw/clean-room/logs/table_sel*.json` | craft-crw contributors (derived in LightCraft, 2026-10, by known-plaintext alignment of the sample files' entropy streams; `crates/crw/clean-room/02-crw-format.md`) | this repository | MIT OR Apache-2.0 | +| `crates/crw/clean-room/logs/private-corpus.tsv` (file names, sizes and SHA-256 of the maintainer's own EOS D60 files; no image content) | craft-crw contributors (LightCraft derivation record) | this repository | MIT OR Apache-2.0 | Original assets are covered by the repository licence. diff --git a/CLAUDE.md b/CLAUDE.md index 2fe86b67..11908a75 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,6 +1 @@ -# CLAUDE.md - -Read [`AGENTS.md`](AGENTS.md): it is the instruction file for every agent working in this repository -(start-of-session steps, never-crash, new-crate checklist, clean-room, assets, time, tests, merge -gates). Format work also follows [`docs/clean-room.md`](docs/clean-room.md). Suite-wide standards: -[craftrules](https://github.com/storytold/craftrules). +Read `AGENTS.md`. diff --git a/crates/crw/README.md b/crates/crw/README.md index bf987c9a..63b01b30 100644 --- a/crates/crw/README.md +++ b/crates/crw/README.md @@ -14,7 +14,7 @@ tag documentation. The compression was never documented by Canon: it was **deriv maintainer-owned sample files, frozen, and verified sample-exact against a reference decoder's command-line output, treated as a black box. No raw-decoder source was consulted, at any point. The whole record is in [`clean-room/`](clean-room/README.md); the frozen format is -[`clean-room/06-crw-format.md`](clean-room/06-crw-format.md). +[`clean-room/02-crw-format.md`](clean-room/02-crw-format.md). ## What it decodes @@ -29,7 +29,7 @@ The whole record is in [`clean-room/`](clean-room/README.md); the frozen format | PowerShot G5 | 1 | **refused** | its entropy stream diverges from the model from sample 2056 on (`Error::Unsupported`); apps show the embedded JPEG | | PowerShot G2, G3, Pro1, Pro70, S30–S70 | | unverified | no CC0 samples exist; they decode through the same model if their records match, with no guarantee | -"Exact" means every sensor sample equals the reference decode (`clean-room/07-verification.md`). +"Exact" means every sensor sample equals the reference decode (`clean-room/03-verification.md`). Big-endian (`MM`) files are handled from the specification and tested synthetically; none has been seen in the wild. CHDK sensor dumps named `.CRW` are a different format and are not CIFF files (`probe` says no). @@ -141,7 +141,7 @@ Colour (camera matrices, profiles, the JPEG-fitted look) stayed in LightCraft. `src/tables.rs` is generated data. Edit nothing by hand: the source of truth is `clean-room/logs/table_sel{0,1,2}.json`, and `python crates/crw/clean-room/tools/gen_tables.py` renders it (byte-identical to the committed file). A change to the tables is a change to the frozen -format and must follow the reopening rule of `clean-room/06-crw-format.md`. +format and must follow the reopening rule of `clean-room/02-crw-format.md`. ## Boundaries diff --git a/crates/crw/clean-room/00-starting-assumptions.md b/crates/crw/clean-room/00-starting-assumptions.md deleted file mode 100644 index cfea605f..00000000 --- a/crates/crw/clean-room/00-starting-assumptions.md +++ /dev/null @@ -1,111 +0,0 @@ -# CRW clean-room derivation — starting assumptions - -Written 2026-10-12, **before** reading any file in this repository (other than the standing -CLAUDE.md instructions loaded into the session), any web page, any specification, or any sample -file. Everything below is recalled background knowledge of the author (Claude Fable 5) and is the -baseline the derivation starts from. **Every item is: unverified, no source consulted.** - -This file is committed once and never edited. Corrections, confirmations and refutations go in -`01-plan.md` and the derivation documents, each citing the experiment or published document that -established them. A reviewer can diff this file against the final documentation to see which -facts were assumed and which were derived. - -Confidence labels are the author's own recollection strength, not evidence: *firm* = would be -surprised if wrong; *hazy* = plausibly misremembered or confused with another format (CR2/CR3, -other makers' raws). - -## A. Container - -- **A1** (firm, unverified, no source consulted): `.CRW` files are in CIFF, the "Camera Image - File Format", a Canon-proprietary heap-based container. It is not TIFF-based and not - Exif/JPEG-based (unlike the later `.CR2`, which is TIFF). -- **A2** (firm, unverified, no source consulted): the file begins with a 2-byte byte-order mark, - `II` for little-endian (as TIFF does); Canon cameras write little-endian. `MM`/big-endian is - allowed by the format but not expected in camera files. -- **A3** (hazy, unverified, no source consulted): a 32-bit integer at offset 2 gives the length - of the file header, i.e. the offset at which the heap data begins; the common value is 26 - (0x1A). -- **A4** (firm, unverified, no source consulted): the header contains the ASCII signature - `HEAPCCDR` (at offset 6, immediately after the header-length field). -- **A5** (firm, unverified, no source consulted): the body of the file is a "heap". A heap's - directory is located via the **last 4 bytes** of the heap region: a 32-bit offset, relative to - the start of the heap, pointing at the directory. Data blocks come first, directory last. -- **A6** (firm, unverified, no source consulted): a directory is a 16-bit record count followed - by that many fixed-size 10-byte records: tag (u16), size (u32), offset (u32), with offsets - relative to the start of the containing heap. There may be trailing bytes after the records. -- **A7** (hazy, unverified, no source consulted): the tag value's high bits encode storage class - and data type, the low bits the tag identity. In particular, one storage class means "value - stored in the heap" (size/offset meaningful) and another means "value stored in the record - itself" (the 8 bytes that would hold size+offset hold the data inline). Recalled masks: - storage class ≈ `0xC000`, data type ≈ `0x3800`, id ≈ low 11 bits — exact masks uncertain. -- **A8** (firm, unverified, no source consulted): some records are themselves heaps - (sub-directories), giving a tree: e.g. an image-properties heap, a camera-object heap, an - Exif-information heap. Parsing must recurse; recursion depth must be bounded against malformed - files. -- **A9** (hazy, unverified, no source consulted): recalled tag numbers — `0x2005` raw image - data; `0x1835` "decoder table" selector for the raw compression; `0x1031` sensor/image - dimensions (including the margins of masked pixels); `0x2007` embedded JPEG - ("JpgFromRaw"-like); `0x080a` make and model as NUL-separated strings; something like - `0x102a` holding shot info (ISO, etc.) and a white-balance record whose tag I do not recall. - All tag numbers here are suspect. -- **A10** (firm, unverified, no source consulted): cameras writing CRW also write a sibling - `.THM` file: a small Exif/JPEG thumbnail carrying the metadata. The CRW itself may embed a - preview JPEG as well. -- **A11** (hazy, unverified, no source consulted): CRW was written by early Canon EOS DSLRs - (D30, D60, 10D, 300D/Digital Rebel) and PowerShot models of the same era (G1–G6, S-series, - Pro1). Later bodies (20D on) switched to CR2. - -## B. Raw data storage and coding - -- **B1** (firm, unverified, no source consulted): the raw payload is a single plane of Bayer - CFA samples, pattern RGGB at the sensor origin, covering the full readout area including - masked/optically-black border pixels, which is slightly larger than the visible image. -- **B2** (hazy, unverified, no source consulted): sample depth is 10 or 12 bits depending on - the model; the EOS D60 is 12-bit. (10-bit is recalled for some PowerShots.) -- **B3** (firm, unverified, no source consulted): the raw data is losslessly compressed with a - Canon-proprietary scheme. It is **not** a JPEG stream: no JFIF/JPEG markers, no - entropy-coded-segment byte stuffing wrapper — a bare bitstream. -- **B4** (firm, unverified, no source consulted): the scheme is Huffman-coded DPCM in the style - of JPEG's AC coefficient coding: the stream is organised in blocks of 64 difference values; - each Huffman code decodes to a (run-of-zero-differences, magnitude-bit-count) pair; a - magnitude of n bits follows as n raw bits with JPEG-style sign extension (a leading 0 bit - means negative, value = read − (2^n − 1)). -- **B5** (hazy, unverified, no source consulted): there is a small fixed set of Huffman table - pairs (two or three), built into the camera/decoder rather than stored in the file; the - "decoder table" record (see A9) holds an index selecting which set. Within a set, one table - codes the first value of each 64-block and a second table codes the remaining 63 — the - first/rest split is the haziest part of this recollection. -- **B6** (hazy, unverified, no source consulted): a "low bits" mode exists: when enabled, the - 2 least-significant bits of every pixel are stored separately, uncompressed and packed 4 per - byte, at the start of the raw data section, and the Huffman stream codes only the high bits; - the final sample is (decoded << 2) | low_bits. Which cameras/flags enable it is unknown to - me; possibly all 10-bit → 12-bit? Uncertain. -- **B7** (hazy, unverified, no source consulted): decoded differences accumulate into samples - via two running predictors, one for even and one for odd columns (i.e. per CFA column - parity). I do not recall where the predictors reset (per row? per row pair? never after - initialisation to a mid-grey such as 512?) or how the first pixels of the image are formed. -- **B8** (hazy, unverified, no source consulted): bits are consumed MSB-first within each byte; - there is no marker-escaping; I vaguely recall some per-camera quirk where a count of spare - bits or a skipped prefix is involved, but cannot state it. -- **B9** (firm, unverified, no source consulted): the visible image is a crop of the decoded - area; a sensor-info record gives the full decoded width/height and the left/top borders of - the visible region. -- **B10** (hazy, unverified, no source consulted): black level is taken from the masked border - pixels (or a stored value); white-balance coefficients are in a metadata record as per-channel - levels or gains. Formats of those records not recalled. -- **B11** (hazy, unverified, no source consulted): EOS D60 visible size is 3072×2048 (6.3 MP); - the decoded raw area is slightly wider/taller (numbers not recalled). File size around 7–8 MB, - consistent with ~12 bits/pixel compressed losslessly below 2:1. -- **B12** (hazy, unverified, no source consulted): some PowerShot-era CRW variants may store - the raw data uncompressed or in a different packing; unsure whether any in-scope body does. - -## C. Process assumptions - -- **C1** (firm, unverified, no source consulted): ExifTool publishes prose documentation of the - CIFF/CRW structure and tag names (web pages, distinct from its Perl source, which is off - limits under the clean-room rules). -- **C2** (hazy, unverified, no source consulted): Canon once published a CIFF specification - document (circa 1997–98, "CIFF Specification on Image Data File", version ~1.0 r4) that - circulates publicly as a PDF. -- **C3** (hazy, unverified, no source consulted): raw.pixls.us hosts CRW samples for several of - the bodies in A11, some CC0; per-file licences vary, so each file must be checked. diff --git a/crates/crw/clean-room/03-payload-layout.md b/crates/crw/clean-room/01-payload-layout.md similarity index 88% rename from crates/crw/clean-room/03-payload-layout.md rename to crates/crw/clean-room/01-payload-layout.md index eafec02b..14adea06 100644 --- a/crates/crw/clean-room/03-payload-layout.md +++ b/crates/crw/clean-room/01-payload-layout.md @@ -1,10 +1,10 @@ # CRW RawData payload layout (P1) — established by statistics, no decoding -Status: derived 2026-10-12 from the 29-file corpus by `p1_payload.py`, `p1b_markers.py`, -`p1c_lowbits_bounds.py` (logs: `logs/p1-*.txt`, `logs/p1b-*.txt`, `logs/p1c-*.txt`, private -counterparts under `logs/private/`). **The entropy-coded bitstream itself has not been decoded;** -everything here comes from sizes, offsets, byte histograms and ff-digram statistics. -Assumption IDs refer to `00-starting-assumptions.md`; corrections are stated explicitly. +Status: derived 2026-10-12 from the 29-file corpus by statistics alone. The scripts that produced +it (`p1_payload.py`, `p1b_markers.py`, `p1c_lowbits_bounds.py`) and their logs are not part of +this record. **The entropy-coded bitstream itself has not been +decoded;** everything here comes from sizes, offsets, byte histograms and ff-digram statistics. +Corrections are stated against the starting assumptions, by their labels (B2, B3, …). ## 1 The three layouts @@ -17,7 +17,7 @@ dimensions, Q = ⌈w·h/4⌉ (one byte per four pixels — the "low-bits" region | PowerShot 12-bit: G5, G6 | `[514 zero bytes][low bits, Q][stuffed entropy][ff d9 (+00)]` | S−L = 2; first 514 bytes zero; **all** non-stuffed ff (5002 of 5002 on G5, 7928 of 7928 on G6) fall inside [514, 514+Q); the rest of the stream is perfectly stuffed; payload ends `ff d9` | | PowerShot 10-bit: G1 | `[514 zero bytes][stuffed entropy][ff d9 + 00]` | S−L = 3; first 514 bytes zero; zero non-stuffed ff anywhere; payload ends `ff d9 00` | -Private corpus: all 18 D60 files match the EOS row exactly (`logs/private/p1c-bounds-private.txt`). +Private corpus: all 18 D60 files match the EOS row exactly. ## 2 DecoderTable (0x1835) semantics @@ -47,7 +47,7 @@ Private corpus: all 18 D60 files match the EOS row exactly (`logs/private/p1c-bo - Extent Q = w·h/4 exactly, in all 25 files that have the region (B6's arithmetic holds). - D60-class prefix: reshaped at w/4 = 788 bytes/row, the first **7 rows are exactly zero** (5516 = 7×788 bytes) in the public D60 file, and the leading zero run is row-aligned — - strong evidence the region is row-major at 2 bits/pixel (`logs/p1-payload-public.txt`). + strong evidence the region is row-major at 2 bits/pixel. Below the crop-top band the bytes are near-uniform (entropy ≈ 7.99), as LSB noise should be. - D30's region is high-entropy from byte 0 and ends in a run of `ff` bytes (saturated LSBs?); no zero rows — the D30's masked rows evidently carry noise. Placement for D30 is supported @@ -89,8 +89,7 @@ Private corpus: all 18 D60 files match the EOS row exactly (`logs/private/p1c-bo | g5 | 2672×1968 | 4 861 806 | 1 | 4 861 804 | 2 | 1 314 624 (inside L) | — | 514 | | g6 | 3160×2344 | 7 313 156 | 0 | 7 313 154 | 2 | 1 851 760 (inside L) | — | 514 | -(S here is the RawData record size, not the file size. Private D60 table: -`logs/private/p1-payload-private.txt`.) +(S here is the RawData record size, not the file size.) ## 7 Open for P2 (entropy derivation) @@ -102,5 +101,5 @@ Private corpus: all 18 D60 files match the EOS row exactly (`logs/private/p1c-bo explanation to fall out of the geometry work (P3). 5. EOS tail `ff 00 00` vs `00 00` — settle when the stream end is decodable. -P2 requires the approved oracle (LibRaw pre-built binary, run black-box; plan §5a decision 1–2) -for probing and for the low-bits test. **Requested from the maintainer before P2 starts.** +P2 requires the approved oracle (LibRaw pre-built binary, run black-box, per the plan the +maintainer approved) for probing and for the low-bits test. **Requested from the maintainer before P2 starts.** diff --git a/crates/crw/clean-room/01-plan.md b/crates/crw/clean-room/01-plan.md deleted file mode 100644 index 452467f8..00000000 --- a/crates/crw/clean-room/01-plan.md +++ /dev/null @@ -1,337 +0,0 @@ -# Canon CRW (CIFF) raw decoding — clean-room derivation plan - -Written 2026-10-12. **Status: awaiting the maintainer's approval. No decoder code and no table -data exist yet; nothing below §5 phase P1 starts until this plan is approved.** - -Companion documents in this directory: - -- [`00-starting-assumptions.md`](00-starting-assumptions.md) — what was assumed before any - source was consulted (committed first, immutable). Items are cited below as A1–A11 (container), - B1–B12 (raw coding), C1–C3 (process). -- [`sources.md`](sources.md) — every URL opened and search run, with what was taken. -- [`ciff_dump.py`](ciff_dump.py) + [`logs/`](logs/) — the planning-phase container survey. - -Rules in force (unchanged from the task): no raw-decoder source is ever opened, fetched, -searched inside or requested (dcraw, LibRaw, rawspeed, rawloader/rawler, dnglab, darktable, -RawTherapee, libopenraw, ExifTool's Perl code — including licence headers, comments and -mirrors); the earlier `crw-decode` branch is not checked out, diffed, logged or read; public -samples are CC0 files from raw.pixls.us only; the maintainer's private files never enter the -repository and no document ever includes their image content (SHA-256 and size only). - ---- - -## 1 What public documentation establishes (sources, kept separate from assumptions) - -### 1.1 Canon, *CIFF Specification on Image Data File*, v1.0 r4, 1997-12-24 - -Archived PDF (SHA-256 `416d4e49ea3f6ad2ba5ccc76b48ca100605f4a6127816c145e6dd5d6c3b51cf4`, -URL in `sources.md`; local copy in gitignored `plan/crw/`). Canon's -own published specification. It establishes, with exact definitions: - -- **Heap file header** (quoted struct): `byteOrder[2]` (`"II"` little-endian / `"MM"` - big-endian), `headerLength` u32 ("distance from the start of the file to the beginning of the - heap-space"), `type[4]` = `"HEAP"`, `subType[4]` (`"JPGM"` for JPEG-embedded; reserved values - include `"CCDR"`, `"TIFP"`, `"ARCH"`), `version` u32 = 0x00010002, two reserved u32s. -- **Heap** = heap space followed by the `offsetTbl`; a u32 `offsetTblOffset` is stored "in the - final position at the end of the heap", as an offset relative to the start of the heap. -- **offsetTbl** = `numRecords` u16 + that many `DC_RECORD_ENTRY {typeCode u16, length u32, - offset u32}`; offset is "offset value from the beginning of the heap"; `numRecords` may be 0. -- **typeCode bit fields**: bits 15–14 storage (`00` kStg_InHeapSpace, `01` kStg_InRecordEntry — - data held in the 8 bytes of the length+offset fields themselves — `10`/`11` reserved); bits - 13–11 dataType (`000` byte, `001` ASCII string, `010` 2-byte, `011` 4-byte, `100` structure, - `101`/`110` heap record, `111` reserved); bits 10–0 IDCode. Masks (quoted): - `kStgFormatMask 0xc000`, `kDataTypeMask 0x3800`, `kIDCodeMask 0x07ff`. -- Heap records (dataType 101/110) are themselves heaps → the tree structure. -- Null (`kTC_Null` 0x0000) and free (`kTC_Free` 0x0001) records; all records 2-byte aligned, - odd lengths padded. -- Common metadata record structs: ImageSpec (imageWidth, imageHeight, pixelAspectRatio f32, - rotationAngle s32, componentBitDepth, colorBitDepth, colorBW), ImageFormat, CapturedTime - (time_t + zone), ModelName ("two null-terminated strings … manufacturer name … model name"), - and the ImageProps/ShootingRecord/MeasuredInfo/Camera/CameraSpecification hierarchy. -- **The spec does not describe the raw image payload at all.** ImageFormat's fileFormat value 2 - is merely "Reserved (for CanonCRW format)". Everything about the raw coding must come from - sample analysis (and an approved oracle), not from documentation. - -### 1.2 ExifTool prose pages (documentation, not Perl source) - -`exiftool.org/canon_raw.html` (format description) and `TagNames/CanonRaw.html` (tag tables): - -- A `.CRW` file is a heap file: 26-byte header with signature `HEAPCCDR` (= type `HEAP` + - subType `CCDR`), CRWVersion 0x00010002; the root block runs from `headerLength` to EOF; a - subdirectory is a complete block inside its parent's ValueData whose block size is the - entry's Size; offsets are relative to the containing block's ValueData start. -- Tag IDs (TagID = dataType | IDCode, i.e. typeCode & 0x3fff): **0x2005 RawData** ("the bulk - of the CRW file"), **0x2007 JpgFromRaw** (embedded JPEG), 0x2008 ThumbnailImage, **0x1835 - DecoderTable** with sub-values named *DecoderTableNumber, CompressedDataOffset, - CompressedDataLength*, **0x1031 SensorInfo**, 0x1810 ImageInfo, 0x1030 WhiteSample (with - *BlackLevels*), 0x10a9 ColorBalance, 0x10ae ColorTemperature, 0x080a CanonRawMakeModel, - 0x102a CanonShotInfo, 0x1834 CanonModelID, 0x300a ImageProps, 0x2807 CameraObject, 0x300b - ExifInformation, 0x0000 NullRecord, 0x0001 FreeBytes, and others (full list in `sources.md`). -- Neither page describes the raw compression, bit depth or decoder tables' contents. - -### 1.3 Wikipedia, "Camera Image File Format" - -Spec identity and dates; the CRW camera list: EOS D30, D60, 10D, 300D; PowerShot Pro1, G1–G6, -S30–S70. (Matches A11.) - -## 2 Corpus - -### 2.1 Public (CC0, raw.pixls.us; downloaded into gitignored `corpus/crw/`, hashes verified) - -| File | Body | Size (bytes) | SHA-256 | -|---|---|---|---| -| `crw-canon-eos-d30.crw` | EOS D30 | 2 993 260 | `4ee482f2b6f9f1a8d370703ea04e9f2abaa49356973a7f56b229af93a78e8a2e` | -| `crw-canon-eos-d60.crw` | EOS D60 | 8 591 234 | `2b98659ecff170af1dfe44f89b493139e75e995b2979686f6ff0847cfb56b97f` | -| `crw-canon-eos-10d-a.crw` | EOS 10D | 7 787 330 | `b0730eb3a67cc2281a32e0b2ee104199246928d954a68b7e5b72366514c73d02` | -| `crw-canon-eos-10d-b.crw` | EOS 10D | 10 822 040 | `e879c94afdc64dabc88b01ce2d02628aa225259303ec71dd61f7fb0322bfea2d` | -| `crw-canon-eos-10d-c.crw` | EOS 10D | 10 583 702 | `58144118854657df5cd2185df40fecf6b665fc6754fef0416c2f222b68b025c6` | -| `crw-canon-eos-300d-a.crw` | EOS 300D | 9 056 348 | `47da16e1668af5500eea63cf42cf2992d9cb3d32180b8d786f17f1e2a57f39f4` | -| `crw-canon-eos-300d-b.crw` | EOS 300D | 8 389 842 | `9e7f62239fe728046af28d0d4a8a722d35d40146a79794a5bda1775306af4383` | -| `crw-canon-eos-digital-rebel.crw` | EOS 300D (US name) | 10 153 786 | `d6a3131bfac390fde01b96882ec378211f04570c3252708b58f0d05b33a9fdfc` | -| `crw-canon-powershot-g1.crw` | PowerShot G1 | 2 300 702 | `f39be5acf14057ce04527fd8d0574e57ec8ed5fed082d7bfaff763075a44ee8a` | -| `crw-canon-powershot-g5.crw` | PowerShot G5 | 4 894 440 | `bb084223d714871b99de4f549eaf00513e30a7080feeead700252c2008060196` | -| `crw-canon-powershot-g6.crw` | PowerShot G6 | 8 052 032 | `244378e1f5660b6cc6cefc93a75ed4928d0b28aa6304ff327b61c748538f4fef` | - -All 11 are listed CC0 by raw.pixls.us (licence per file verified in the site's data on -2026-10-12; `sources.md`). Each download was checked against the site's published SHA-256. - -### 2.2 Private (maintainer's EOS D60 files, `../RawImages/`, local only — never committed, no -image content in any document) - -18 files, `CRW_0476.crw` … `CRW_2219.crw`, 5 500 958 – 8 628 810 bytes. SHA-256 and sizes -recorded in `sources.md`-adjacent planning logs; full hash list: - -| File | Size | SHA-256 | -|---|---|---| -| CRW_0476.crw | 6 906 290 | `7966aeb5abf3abe80868b8cb1606d671112c42f44d877d3ff79305a52558cacc` | -| CRW_0478.crw | 5 708 018 | `6d07ef616f4b1c7c3025e7af315bb7fb4173ddefea45e2e52d5a4d28e9f2851f` | -| CRW_0486.crw | 8 055 488 | `bf0876f5ba55b2bd120f9588b91f9d4caad54ab73315e542114bf738459d8b3b` | -| CRW_0527.crw | 7 158 482 | `2fd42a69caa20e7c07fe022bd73dfe2626d60a3bacc4228073d9627a5dfa03c1` | -| CRW_0683.crw | 7 453 390 | `c4b320a158b1859fa0709050caddfe2bec0d0a566eafeb17df89cc6b64346a75` | -| CRW_0717.crw | 6 633 986 | `0d6b64935d928e2e0d378eb6efe2b46bed9026ed227111fd47b5fe8b911e22ad` | -| CRW_0765.crw | 5 636 168 | `e8da71d85e07d4157fb5753f850b331dfa6451d7382c4c0d4f9568144e6e8bc5` | -| CRW_0822.crw | 6 478 092 | `817acdc28dd1fe2cd0065b44c63722aaa1f800275e0b0b245ad3b3d964df7c9f` | -| CRW_0879.crw | 6 861 152 | `df66c000ac18b872a69b0fdc171245f89869bd0ad415e117c77e6c27ea93c0ca` | -| CRW_1011.crw | 6 822 000 | `9044f07f30064249cb5ea78bd6ba5d19ae3d2da278dbf7f10aa357872fbbd469` | -| CRW_1153.crw | 7 729 790 | `9cd504ed0991c4c6033d72a2676c778b84d198ddfad202437fcb7f014fa15a17` | -| CRW_1277.crw | 6 850 414 | `c4f0ce8c4ce85d3668b3726c46018d296c784b10a2f0f322acba8b1681fe86f4` | -| CRW_1514.crw | 6 531 360 | `9aaa960461f829b218eaa7f50cf49fa3cb80e92baf9f5aabc03038fddbefe06c` | -| CRW_1646.crw | 6 204 492 | `f9f3dbc2f9aed2cdc2ddccd9a9768a1e02501c337258a211ac9fb663ad78ff74` | -| CRW_1740.crw | 5 500 958 | `033f225fcc8bfdb07b013d971c594bda2adfa0a683c133acc22a171ddcdf95a6` | -| CRW_1776.crw | 6 746 924 | `b07921872c8cf7767841945818a6c8d59553415580d7ebe6a08115964ec6de44` | -| CRW_2035.crw | 6 595 632 | `0ffd9fec448a667b7850ea505b5b117f01c07a98aff5e0b7ec9da7fc5909b4e8` | -| CRW_2219.crw | 8 628 810 | `0460e9ba4884c7e0867570cf71be7a6d2d038fdcecf062fde5cc28422602dd07` | - -## 3 Established from sample files during planning (container level only) - -All from `ciff_dump.py` over the 29 files (logs: `logs/structure-public.txt`, -`logs/private/structure-private.txt`). No raw-data decoding was attempted. - -- **All 29 files parse fully** with the documented container model: `II`, headerLength 26, - `HEAP`+`CCDR`, version 0x00010002; every record resolves in bounds; heap tree ≤ 4 deep - (root → ImageProps → CameraObject/ExifInformation/… → CameraSpecification). Confirms - A1–A8 on real files (details per assumption in §4). -- **Root block layout** in every file: RawData (0x2005) first at heap offset 0, then - JpgFromRaw (0x2007, starts with JPEG SOI), then ImageProps (0x300a). -- **SensorInfo (0x1031)**, 17 u16s beginning with its own length (34): D60/10D/300D all - `(34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0)`; D30 - `(34, 2224, 1456, 1, 0, 56, 12, 2215, 1451, 8, 220, 40, 1244, …)`; G1 - `(34, 2144, 1560, 1, 1, 24, 15, 2071, 1550, 2108, 15, 2111, 1550, …)`; G5 2672×1968; G6 - 3160×2344. For the EOS bodies, fields 5–8 as (left, top, right, bottom) give exactly the - visible size: 3143−72+1 = 3072, 2063−16+1 = 2048, matching ImageInfo (0x1810) = 3072×2048. - (Bears on B1/B9/B11; the remaining fields are candidates for black-measurement windows — to - be established.) -- **ImageInfo (0x1810)** u32s `(3072, 2048, 1.0f, rot, 8, 24, 257)` — the 8/24 bit depths - describe the *full-view image* (per the spec's ImageSpec definition), **not** the raw depth; - rot = 270 on the portrait 300D shots. Corrects the hope that raw bit depth is read from here. -- **DecoderTable (0x1835)** = 4 u32s `(N, 0, 514, L)`: - - N ∈ {0, 1, 2} — all three values observed, varying per shot *on the same body* (private - D60 set has 15× N=1, 2× N=0, 1× N=2; public files: D30 1, D60 0, 10D 0/2/2, 300D 0/2, - Rebel 0, G1 1, G5 1, G6 0). Supports B5's "selector" reading; presumably exposure/ISO - dependent — to be established. - - Third u32 is 514 in all 29 files (meaning unknown; 512+2? — to be established). - - **L vs RawData size S**: on every EOS file (D30, D60, 10D, 300D, 23 files), - S − L = ⌈width×height/4⌉ + 2 or 3 bytes (e.g. D60: 6 694 146 − 5 064 559 = 1 629 587; - 3152×2068/4 = 1 629 584). On every PowerShot file (G1, G5, G6), S − L = 2 or 3 bytes. - Strong container-level evidence for B6 (a 2-bits-per-pixel region, pixels packed 4 per - byte, present on the EOS bodies and absent on these PowerShots), **but its placement, - packing order and meaning are still to be derived** — note the tension with B6's - recollection that low bits appear on PowerShots. -- **RawData byte statistics**: overall entropy 7.84–7.99 bits/byte (dense entropy coding); on - the D60-class files the first 2048 bytes are all zero; on D30 the data is high-entropy from - byte 0; on G1/G5 the first ~522–529 bytes are zero-dominated. No JPEG markers at the start; - JPEG SOI appears only in the separate 0x2007 record. -- The three 10D files and the three 300D/Rebel files are internally consistent per body; - the 18 private D60 files are uniform with the public D60 file. - -## 4 Starting assumptions → experiments - -Status: **spec** = established by §1 documents; **samples** = established by §3 observations; -**open** = still unverified — the listed experiment decides it during derivation. - -| # | Status | Correction / experiment, and what counts as confirmed | -|---|---|---| -| A1 CIFF heap container | spec+samples | Confirmed (header `HEAP`/`CCDR`; all 29 parse). | -| A2 `II` first, LE | spec+samples | Confirmed; `MM` legal per spec — decoder must accept or cleanly refuse it; no `MM` sample exists (§6). | -| A3 u32 header length @2 = 26 | spec+samples | Confirmed (26 in all 29; field is authoritative, not the constant). | -| A4 `HEAPCCDR` @6 | spec+samples | Confirmed, with correction: it is two fields, type `HEAP` + subType `CCDR`; other subTypes exist (`JPGM`…) and must be refused as unsupported. | -| A5 directory via trailing u32 | spec+samples | Confirmed (`offsetTblOffset` in the heap's final position, relative to heap start). | -| A6 u16 count + 10-byte entries | spec+samples | Confirmed (typeCode u16, length u32, offset u32 — in that order). | -| A7 tag bit masks | spec+samples | Confirmed exactly: 0xc000 storage (0x0000 in-heap, 0x4000 in-entry, rest reserved), 0x3800 dataType, 0x07ff IDCode. | -| A8 nested heaps, bounded recursion | spec+samples | Confirmed (dataType 5/6 = heap record); observed depth ≤ 4; parser caps at 8. | -| A9 tag numbers | samples | Mostly right: 0x2005 RawData, 0x1835 DecoderTable, 0x1031 SensorInfo, 0x2007 JpgFromRaw, 0x080a make/model, 0x102a ShotInfo all confirmed; ColorBalance is 0x10a9. DecoderTable is 4×u32, not "16 bytes of table data". | -| A10 .THM sibling + embedded JPEG | part | Embedded JpgFromRaw confirmed (SOI seen); ThumbnailFileName records name a `.THM`, but no `.THM` files are present in `../RawImages` — treat the THM as optional; not needed for decoding. | -| A11 camera list | spec-adjacent | Matches Wikipedia + raw.pixls.us. Later `.CRW`-named CHDK dumps are a different format: detect via header and refuse as unsupported. | -| B1 Bayer CFA, RGGB at origin, full readout incl. masked borders | open | Decode must yield exactly SensorInfo's width×height samples; CFA phase from (i) colour of uniform patches vs the embedded JPEG, (ii) per-channel black/white statistics, (iii) oracle's CFA output. Confirmed when all three agree on all 29 files. | -| B2 10- or 12-bit samples | open | From decoded histograms (max code, gaps), clipped-highlight files, and the low-bits arithmetic (wh/4 = 2 bits/pixel implies the Huffman stream codes depth−2 bits on EOS bodies). Confirmed when the white clip point is stable per body and matches the oracle's scale. | -| B3 not JPEG, bare bitstream | samples (part) + open | No JPEG structure observed; fully confirmed when the derived bitstream model consumes the payload exactly on all files. | -| B4 Huffman run/size DPCM in 64-blocks | open | The central derivation (P2, §5): a candidate code model must decode every file such that (i) the stream is consumed to within the final byte, (ii) every sample lands in [0, 2^depth), (iii) rows have natural image statistics (smoothness, CFA correlation). Confirmed only when one model does this for all 29 files and matches the oracle exactly. | -| B5 small fixed set of table pairs selected by file | samples (part) + open | DecoderTableNumber ∈ {0,1,2} observed varying per shot. Open: the tables' content, whether they live in camera or file (the constant `514` field and the RawData prefix are candidates to rule in/out), first-vs-rest table split. Confirmed when one fixed table set per N decodes every file with that N, across bodies. | -| B6 2 LSBs stored separately, packed 4/byte | samples (part) + open | wh/4-byte region established on EOS bodies — but B6 guessed PowerShots; observed on EOS instead, and placement unknown (D60-class zero prefix suggests the region leads, since masked top rows plausibly have zero LSBs — but D30's leading bytes are high-entropy, so order may differ or D30's masked rows are noisy). Experiments: (i) statistical — LSB region should be near-uniform in lit areas and structured in masked areas consistently across files; (ii) decisive — oracle output codes' 2 LSBs must equal the extracted region bit-for-bit with one fixed packing rule. | -| B7 two column-parity predictors | open | Derived in P2 by fitting predictor variants in the Python model; confirmed when interior and first-row/first-column pixels match the oracle on all files, incl. the masked border. | -| B8 MSB-first bits, no byte-stuffing | open | Both orders tried in P2; the wrong one fails the exact-consumption test almost immediately. Confirmed with B4. | -| B9 visible crop from sensor record | samples (part) + open | Crop rectangle (72,16)–(3143,2063) = 3072×2048 established; confirmed as *the* crop when our visible output aligns with the embedded JPEG (geometry check as in arw6) and the oracle. | -| B10 black from masked pixels / record; WB in metadata | open | Candidates observed: 0x1030 WhiteSample (BlackLevels), 0x10a9 ColorBalance, 0x10a8/0x10aa (u16 arrays that look like per-channel levels). Experiments: correlate record values with decoded masked-area means per channel and with the embedded JPEG's white balance; confirmed when black/WB applied to our decode reproduces the JPEG's grey axis on neutral patches. | -| B11 D60 3072×2048 / slightly larger raw | samples | Confirmed: 3152×2068 raw, 3072×2048 visible. | -| B12 uncompressed CRW variants | open | For the in-scope bodies, none observed (all payloads are high-entropy). CHDK `.CRW` dumps are a different format → refused as unsupported, not decoded. Confirmed by header discrimination tests. | -| C1 ExifTool prose docs exist | confirmed | §1.2. | -| C2 Canon CIFF spec circulates | confirmed | §1.1 (34-page archived PDF). | -| C3 raw.pixls.us CRW samples, mixed licences | confirmed | §2.1 — CC0 for 7 bodies; several classic bodies NC-only (§6). | - -## 5 Derivation plan (phases, freeze, verification) - -Modelled on [`docs/arw6-compression.md`](../../docs/arw6-compression.md): published docs + -samples → Python reference model → **freeze** → black-box verification, no tuning after the -freeze. All phase outputs are documents/scripts in `clean-room/crw/`; each derived fact cites -the experiment (script + log) or document that established it. - -- **P0 — container (mostly done during planning).** Finalize the CIFF parsing notes - (`02-container.md`): header, heap walk, record classes, the records we consume (0x2005, - 0x1835, 0x1031, 0x1810, 0x080a, 0x1834, 0x10a9/0x10a8/0x10aa, 0x1030, 0x180e, 0x5029 …), - endianness rules, malformed-file behaviour. Includes hostile-input rules for the eventual - Rust parser (offsets/lengths checked with `get`, bounded recursion, no panics). -- **P1 — payload partition.** Statistics-only scripts (`p1_*.py`): establish the split of - RawData into (candidate) low-bits region, entropy-coded region, and the 2–3 trailing bytes; - the meaning of DecoderTable fields (incl. the constant 514) by cross-file regression; - per-body low-bits presence. Output: `03-payload-layout.md` with per-file accounting tables. - Still no bitstream decoding. -- **P2 — entropy code derivation.** The core. Hypothesis-driven, in a Python reference model - (`model/`): bit order (B8), code alphabet and block structure (B4), table set per - DecoderTableNumber (B5), predictor/DPCM rules (B7), low-bits recombination (B6), bit depth - (B2). Method: propose a code model; the acceptance tests are *internal* — exact stream - consumption on every file, samples within range, natural-image statistics, cross-file - consistency (same tables for same N across bodies) — iterating over the hypothesis space. - If the oracle (§5a) is approved early, black-box probing is allowed to speed this up - (feeding CC0 files and byte-locally modified CC0 files to the oracle binary and observing - output deltas, as arw6 probed with synthetic tiles); every probe is logged in `sources.md` - style in `04-derivation-log.md`. **No decoder source is consulted regardless.** -- **P3 — geometry, black, white balance, colour.** Visible crop (B9) against the embedded - JPEG at pixel alignment; black level from masked regions + candidate records (B10); white - point per body (B2); WB coefficients from 0x10a9/0x10a8 against the JPEG's neutrals; CFA - phase (B1). Output: `05-geometry-colour.md`. -- **P4 — freeze + verification.** Write `06-crw-format.md`: the complete derived format - (container summary, payload layout, tables as data, algorithm in prose + the Python model's - final hash). **Freeze point: that document's commit.** Only then run the approved oracle - over all 11 public files (and, locally, the 18 private files) and compare decoded samples - exactly; record match/mismatch per file in `07-verification.md` with zero tuning. A - mismatch reopens derivation explicitly: the reopening, its cause and the new freeze commit - are recorded — never silently adjusted. -- **P5 — Rust implementation (separate task(s), after the maintainer approves the derived - format).** `crates/raw` CIFF container + `vendor/crw.rs` decoder; never-crash rules - (hostile offsets, bounded recursion, no unwrap/expect/panic, capped allocations); - unconditional synthetic tests (round-trip via a test-only encoder like `arw6_testenc`, - truncated/hostile input) + corpus-gated tests with pinned sums; `cargo xtask corpus` - entries for 2 pinned CC0 files; docs (`docs/crw-parity.md` or a section in - `docs/raw-parity.md`), parity-checklist/gaps/ROADMAP rows per CLAUDE.md. Perf budget: at - 6 MP these files should decode far inside the 1 s/24 MP export budget; measured in P5. - -### 5a Reference ("oracle") for what the decoded samples should be — **maintainer's decision** - -| Candidate | What it gives | Licence / repo-rule implications | -|---|---|---| -| **(1) LibRaw binary, run as a black box** (as `docs/arw6-compression.md` did: built binary only, source never read) | Exact 16-bit pre-processing codes; the gold standard for B4–B8 | The task forbids fetching LibRaw *source*, so building from source is out for me. LibRaw.org publishes pre-built Windows binaries; I am also forbidden to install/run downloaded programs, so the **maintainer** would download and run it (or approve my running it). Output used as data only; nothing enters the repo but comparison sums. LibRaw is LGPL/CDDL — running a binary copies no code; the house precedent treats this as clean. | -| **(2) Adobe DNG Converter** (free Adobe tool, CRW → DNG; mosaic DNG holds the raw codes losslessly) | Exact codes + Adobe's reading of black/white/WB/crop as DNG tags | Adobe software: CLAUDE.md forbids copying Adobe assets and reading app bundles; running their converter on our CC0/own files and reading the output DNG (an open, documented format) copies no Adobe asset. Residual concerns: Adobe's EULA, and that its interpretation (crop, black) is Adobe's, not the camera's. Same install/run constraint: maintainer runs it. | -| **(3) The embedded JpgFromRaw** | Geometry, orientation, CFA phase, WB neutrals, gross tone sanity | None — it is inside our samples. Lossy and processed, so never a pixel-exact reference. | -| **(4) No-outside-reference checks** | Exact stream consumption, range bounds, masked-area statistics, low-bits cross-checks, Python↔Rust equality, round-trip with our own test encoder | None. Necessary but not sufficient: they validate self-consistency, not correctness of interpretation. | - -**Recommendation:** (3) + (4) always; (1) as the primary oracle, obtained as an official -pre-built binary and run by the maintainer (or by me with explicit approval), used only after -the P4 freeze except for approved black-box probing in P2; (2) optionally as a second opinion -where (1) and our model disagree. **Flagged: the choice, and who runs what, is the -maintainer's.** - -## 6 What cannot be established from the available samples - -- **Bodies with no CC0 file** (raw.pixls.us has them only as CC-BY-NC-SA, which the rules - exclude): PowerShot G2, G3, Pro1, Pro70, S30, S40, S45, S50, S60, S70 (and S2 IS/S3 IS-class - CHDK-era bodies). They can at best be decoded speculatively: if their container metadata - (DecoderTable, SensorInfo, payload accounting) matches a verified body's pattern, the decoder - may accept them with an "unverified" note, as arw6 does for the ILCE-7M5; otherwise they - return unsupported. No G2/G3/S-series verification is possible without new samples. -- **Big-endian (`MM`) files**: none exist here; `MM` handling will be implemented from the spec - and tested only synthetically, or refused — decision in P5. -- **Reserved storage classes (0x8000/0xc000) and dataType 7**: unobserved; refused as - unsupported with a regression test. -- **Per-body table-number coverage**: observed N per body — D30 {1}, D60 {0,1,2}, 10D {0,2}, - 300D {0,2}, G1 {1}, G5 {1}, G6 {0}. Numbers a body never writes can't be verified for it; - cross-body consistency of a given N is testable and part of P2's acceptance. -- **Controlled-exposure pairs** (same scene raw+JPEG at known settings, dark frames, clipped - highlights): none; black/white derivation rests on masked areas, metadata records and the - embedded JPEGs, which is weaker than arw6's lossless/compressed pairs. If the maintainer can - shoot dark-frame and clipped D60 exposures, B2/B10 confirmation strengthens materially. -- **Colour calibration** (camera matrices/profiles): out of scope here, as for every body; - LightCraft's own calibration work, never copied. -- **The remaining SensorInfo fields** (e.g. the `24, 224, 40, 1856` tail on EOS bodies) may - resist interpretation without more bodies; unknowns will be documented as unknown. - -## 7 Open questions for the maintainer — **answered 2026-10-12, plan approved** - -Maintainer's decisions (recorded verbatim in intent, numbered as below): - -1. **Oracle:** no Adobe DNG Converter. LibRaw binary as the arw6 work did; the maintainer will - install it when it is needed. -2. **Black-box probing** before the freeze: allowed, arw6-style (every probe logged). -3. **libopenraw wiki page:** stays unread. -4. **Python reference model:** local-only, like arw6's Phase 0 model (not committed). The - committed evidence set stays as in this commit; the private-file name+size+hash table in - §2.2 is fine to keep. -5. **Scope order:** agreed — D60-class first, then D30, then G1/G5/G6. -6. **NC-licensed bodies:** accept "unverified, header-matched" decoding. The maintainer owns - none of those cameras and has no files from them. -7. **Extra captures:** the maintainer still has the D60 (and a D20) and will provide controlled - shots when the derivation needs them. - -Also noted by the maintainer: the earlier `crw-decode` branch has been deleted and no longer -exists in the repository; the do-not-read rule about it is therefore moot but remains honoured -for any surviving copies. - -### Original questions (for the record) - -1. **Oracle** (§5a): approve candidate (1) (official pre-built LibRaw binary, black box), and - who runs it — you, or me after your explicit approval of the specific download? Allow (2) - Adobe DNG Converter as a secondary reference? -2. **Black-box probing before the freeze** (P2): may the oracle be used interactively on CC0 - and synthetic/modified-CC0 inputs during derivation (arw6-style), with every probe logged — - or only after the freeze? -3. **Prose pages from decoder projects**: the libopenraw freedesktop wiki has a "Canon CRW" - page (surfaced in a search, not opened). It is prose documentation, but authored by a GPL - decoder project and may paraphrase its implementation. Default is to leave it unread; - confirm, or allow it as a documentation source logged in `sources.md`? -4. **Commit policy for `clean-room/crw/`**: plan, sources log, scripts and public-sample logs - are committed (this commit); `logs/private/` is gitignored (may hold personal metadata: - owner name, serial, timestamps). The Python reference model (P2): committed, or local-only - like arw6's Phase 0 model? Private-file hashes in §2.2: confirm you're happy with the - file-name+size+hash table being committed. -5. **Scope order**: derive on the D60-class first (richest corpus: 23 files, one sensor - geometry, all three table numbers), then D30, then G1/G5/G6 — agreed? -6. **NC-licensed bodies** (§6): accept "unverified, header-matched" decoding for G2/G3/Pro1/ - S-series, or return unsupported until samples exist? Do you own any of these cameras, or - CRW files from other bodies (10D/300D/G-series), that could serve as further private - samples? -7. **Extra captures**: if you still have the D60, controlled shots (lens cap dark frame; - clipped sky; grey card at two ISOs, each at all quality settings so all DecoderTableNumbers - appear) would directly confirm B2/B5/B10. Worth doing? - ---- - -*Approved 2026-10-12 (§7). P0 documentation and P1 are under way.* diff --git a/crates/crw/clean-room/02-container.md b/crates/crw/clean-room/02-container.md deleted file mode 100644 index 5b8af04c..00000000 --- a/crates/crw/clean-room/02-container.md +++ /dev/null @@ -1,119 +0,0 @@ -# CRW container (CIFF heap file) — derived parsing notes (P0) - -Status: derived from published documentation and the 29-file corpus; every fact cites its -source. Sources: **[SPEC]** = Canon, *CIFF Specification on Image Data File* v1.0 r4 (see -`sources.md`, hash pinned); **[ET]** = ExifTool's prose pages `canon_raw.html` / -`TagNames/CanonRaw.html`; **[OBS]** = observation of the corpus via `ciff_dump.py` -(`logs/structure-public.txt`, `logs/private/structure-private.txt`). Assumption numbers refer -to `00-starting-assumptions.md`. - -## 1 File header (26 bytes) [SPEC §4.4.1, ET; OBS: all 29 files] - -| Offset | Size | Field | Observed | -|---|---|---|---| -| 0 | 2 | byteOrder: `II` little-endian, `MM` big-endian | `II` in all 29 | -| 2 | 4 | headerLength: file start → heap-space start | 26 in all 29 | -| 6 | 4 | type | `HEAP` in all 29 | -| 10 | 4 | subType | `CCDR` in all 29 (spec reserves `JPGM`, `TIFP`, `ARCH`) | -| 14 | 4 | version: hi16 major, lo16 minor | 0x00010002 in all 29 | -| 18 | 8 | reserved1, reserved2 | — | - -Multi-byte integers throughout the file follow byteOrder [SPEC]. Parser rules: refuse unknown -byteOrder; honour headerLength rather than assuming 26; refuse type ≠ `HEAP`; refuse -subType ≠ `CCDR` as unsupported (a CHDK sensor dump or other container — A11/B12). `MM` files -exist nowhere in the corpus (§6 of the plan); the Rust parser will implement both orders from -[SPEC] and test `MM` synthetically. - -## 2 Heap structure [SPEC §4.1–4.4; ET; OBS] - -- The **root block** spans headerLength..EOF. A **block** = ValueData (heap space), then the - offsetTbl, then optional other data, with a u32 `offsetTblOffset` stored in the final 4 bytes - of the block, relative to the block start [SPEC §4.1.2; ET "DirStart at BlockSize−4"]. -- **offsetTbl** = u16 `numRecords` + numRecords × 10-byte `DC_RECORD_ENTRY {typeCode u16, - length u32, offset u32}`; offsets are relative to the containing block's start [SPEC §4.4.2; - ET]. numRecords = 0 is legal [SPEC §4.1.4]. -- **typeCode bit fields** [SPEC §4.4.2.1, masks quoted in §8.2]: - - bits 15–14, mask 0xc000, storage: `00` in heap space (length/offset meaningful), `01` in - record entry (the 8 bytes of length+offset hold the value, zero-padded; treated as length 8 - unless self-delimiting), `10`/`11` reserved → refuse as unsupported. - - bits 13–11, mask 0x3800, dataType: 0 byte, 1 ASCII (NUL-terminated), 2 u16s, 3 u32s/f32s, - 4 structure (byte-aligned mix), 5/6 heap record (a complete sub-block), 7 reserved → - refuse. - - bits 10–0, mask 0x07ff, IDCode. ExifTool's "TagID" convention = typeCode & 0x3fff - (dataType | IDCode); this document uses that convention for tag numbers. -- **Sub-blocks** (dataType 5/6): the entry's length/offset give the sub-block's extent inside - the parent's ValueData; offsets inside it are relative to its own start [SPEC §4.2; ET; - OBS: all nested directories resolve correctly this way on all 29 files]. -- **Special records** [SPEC §4.4.4]: 0x0000 null (reserves an offsetTbl slot; length/offset - zero), 0x0001 free (reserves heap space). Skip both. -- **Alignment**: record lengths are even; odd-length values are padded [SPEC §4.4.5]. -- Records may appear in any order; data generally precedes its directory ("the natural way to - write information" [ET]). - -## 3 Observed file tree (CRW from cameras) [OBS: all 29 files] - -``` -root (block = headerLength..EOF) -├─ 0x2005 RawData struct, at heap offset 0, the bulk of the file -├─ 0x2007 JpgFromRaw struct, a complete JPEG (starts FF D8) -├─ (G1/G5/G6, D30: 0x2008 ThumbnailImage — JPEG) -└─ 0x300a ImageProps heap - ├─ 0x1803 ImageFormat dword, in-entry - ├─ 0x1810 ImageInfo dword[7] — the FULL-VIEW image: w, h, aspect f32, - │ rotation s32, componentBitDepth, colorBitDepth, colorBW - │ (per [SPEC §5.4.2] ImageSpec; NOT the raw bit depth) - ├─ 0x180e TimeStamp dword[3]: time_t, zone, zoneInfo [SPEC CapturedTime] - ├─ 0x0816/0x0817 file names ascii (e.g. CRW_0099.CRW / CRW_0099.THM) - ├─ 0x0805 description ascii - ├─ 0x3003 MeasuredInfo heap (0x1814 MeasuredEV …) - ├─ 0x2804 ImageDescription heap (0x0805 descr, 0x0815 CanonImageType, e.g. - │ "CRW:EOS D60 CMOS RAW IMAGE") - ├─ 0x2807 CameraObject heap - │ ├─ 0x0810 OwnerName ascii - │ ├─ 0x080a CanonRawMakeModel ascii: "Canon\0\0" [SPEC ModelName] - │ └─ 0x3004 CameraSpecification heap (0x180b SerialNumber, 0x101c BaseISO, - │ 0x080b FirmwareVersion …) - └─ 0x300b ExifInformation heap - ├─ 0x102a CanonShotInfo u16 array (first u16 = byte length) - ├─ 0x102d CanonCameraSettings u16 array - ├─ 0x1031 SensorInfo u16[17]: len(34), rawW, rawH, ?, ?, cropL, cropT, - │ cropR, cropB, then 4+ more u16 (black windows? TBD P3) - ├─ 0x1835 DecoderTable u32[4]: (tableNumber, 0, 514, compressedLength) — see - │ 03-payload-layout.md - ├─ 0x10a9 ColorBalance, 0x10a8, 0x10aa u16 arrays (WB candidates, P3) - ├─ 0x1030 WhiteSample (10D/300D; BlackLevels candidate [ET], P3) - ├─ 0x1834 CanonModelID dword, in-entry - └─ 0x1033 CustomFunctions, 0x10c0/0x10c1/0x10c2, 0x0036 … (not needed for decode) -``` - -Per-body constants observed (rawW×rawH; crop rect, inclusive): - -| Body | Raw | Crop (L,T)–(R,B) | Visible | -|---|---|---|---| -| EOS D30 | 2224×1456 | (56,12)–(2215,1451) | 2160×1440 | -| EOS D60 / 10D / 300D | 3152×2068 | (72,16)–(3143,2063) | 3072×2048 | -| PowerShot G1 | 2144×1560 | (24,15)–(2071,1550) | 2048×1536 | -| PowerShot G5 | 2672×1968 | — (fields differ; P3) | 2592×1944 (ImageInfo) | -| PowerShot G6 | 3160×2344 | — (P3) | 3072×2304 (ImageInfo) | - -(G-series SensorInfo tails differ in shape from the EOS ones — e.g. G1's fields 9–12 are -`2108, 15, 2111, 1550`, a second rectangle to the right of the crop: a plausible black/dark -measurement window. To be established in P3.) - -## 4 Records the decoder will consume - -0x2005 RawData (payload), 0x1835 DecoderTable, 0x1031 SensorInfo, 0x1810 ImageInfo (visible -size cross-check + rotation), 0x080a make/model + 0x1834 CanonModelID (dispatch), 0x10a9 / -0x10a8 / 0x10aa / 0x1030 (black & WB, P3), 0x180e TimeStamp, 0x2007 JpgFromRaw (preview + -geometry verification), 0x102a CanonShotInfo (ISO etc. for metadata display). - -## 5 Hostile-input rules for the eventual Rust parser (never-crash) - -All reads through checked `get`-style access; every offset/length validated against the -containing block before use; offsets/lengths are u32 → promote to u64/usize with checked math, -no wrapping; recursion over sub-blocks bounded (observed depth 4; cap at 16 with a clear -"malformed: heap nesting too deep" error); a record table whose 2 + 10·numRecords exceeds -offsetTblOffset→block-end space is malformed; in-entry values are exactly the 8 bytes of -length+offset; unknown storage class, dataType 7, unknown subType, or missing mandatory -records (0x2005, 0x1031, 0x1835) → typed unsupported/malformed errors, never a panic; -allocations sized by header fields capped against the actual file size. diff --git a/crates/crw/clean-room/06-crw-format.md b/crates/crw/clean-room/02-crw-format.md similarity index 94% rename from crates/crw/clean-room/06-crw-format.md rename to crates/crw/clean-room/02-crw-format.md index b61ecf94..7b5c855b 100644 --- a/crates/crw/clean-room/06-crw-format.md +++ b/crates/crw/clean-room/02-crw-format.md @@ -5,13 +5,13 @@ must reopen it explicitly with a recorded cause and a new freeze commit. **Revision 2 (2026-10-13), cause:** the maintainer supplied 1,516 D60 files, 50 times the first corpus. 223 of them failed on code-table entries the first corpus never exercised, -and 9 were misread as having no low-bits plane. Two corrections, both recorded in -`04-derivation-log.md` and re-verified in `07-verification.md`: the low-bits presence rule +and 9 were misread as having no low-bits plane. Two corrections, both recorded in the +derivation log (not part of this record) and re-verified in `03-verification.md`: the low-bits presence rule (§1) gained the header-position test, and the selector 0 and 1 second tables were completed (§3). Nothing else in the model changed, and every file of the first corpus still verifies. Everything here was derived from the CC0/private sample files and black-box probing of LibRaw 0.22.2 (`unprocessed_raw.exe`), per the approved plan; no -decoder source was consulted. The chronological evidence is `04-derivation-log.md`; the -container and payload groundwork are `02-container.md` and `03-payload-layout.md`. +decoder source was consulted. The chronological evidence is the derivation log, which is not part +of this record; the payload groundwork is `01-payload-layout.md`. ## 1 Payload layout @@ -92,7 +92,7 @@ superset of each file's exercised codes: every row was learned from a real strea position and validated by the decode reaching the end of that frame; rows never exercised by two files may in principle be mergeable duplicates of canonical slots, which the Rust implementation may normalise, provided the verification table of -`07-verification.md` still reproduces exactly. +`03-verification.md` still reproduces exactly. ## 4 Not yet established / out of scope here @@ -110,6 +110,6 @@ which the Rust implementation may normalise, provided the verification table of ## 5 Verification -See `07-verification.md` (written after the freeze): full-frame exact-match decode +See `03-verification.md` (written after the freeze): full-frame exact-match decode against LibRaw 0.22.2 for every sample file, public and private, with zero tuning after the freeze commit. diff --git a/crates/crw/clean-room/07-verification.md b/crates/crw/clean-room/03-verification.md similarity index 91% rename from crates/crw/clean-room/07-verification.md rename to crates/crw/clean-room/03-verification.md index e379e833..3afaeefd 100644 --- a/crates/crw/clean-room/07-verification.md +++ b/crates/crw/clean-room/03-verification.md @@ -1,6 +1,6 @@ # CRW decode verification (P4) — against LibRaw 0.22.2, zero tuning -Method: the frozen model of `06-crw-format.md` plus the frozen tables +Method: the frozen model of `02-crw-format.md` plus the frozen tables (`logs/table_sel{0,1,2}.json`) were run by the local Python reference decoder over every sample file, comparing **every sensor sample** (full values, low bits merged) against `unprocessed_raw.exe -T` output. "EXACT" means zero mismatching samples over the whole @@ -8,8 +8,8 @@ frame. The table extraction itself consumed these same files (known-plaintext le so this is a consistency proof against the oracle, not a blind holdout; the cross-body sharing (tables learned on one body, exact on others) is the strongest independent check. -Private files are the maintainer's own D60 photos (hashes in `01-plan.md` §2.2); only -their verdicts are recorded here, never content. +Private files are the maintainer's own D60 photos (hashes in `logs/private-corpus.tsv`); +only their verdicts are recorded here, never content. ## Results @@ -46,13 +46,13 @@ their verdicts are recorded here, never content. | `private/CRW_2219.crw` | 2 | 3152x2068 | **EXACT** | **28 of 29 files decode EXACT** — all 10 usable public CC0 files and all 18 private -D60 files; the PowerShot G5 is the known exception (`06-crw-format.md` §4). +D60 files; the PowerShot G5 is the known exception (`02-crw-format.md` §4). ## Notes - The final 1–3 samples of each frame follow the stream-end rule (chains persist); every "EXACT" verdict includes them. -- PowerShot G5: see `06-crw-format.md` §4 — detected and excluded; embedded-JPEG +- PowerShot G5: see `02-crw-format.md` §4 — detected and excluded; embedded-JPEG fallback until resolved. - The G1 (10-bit, no low-bits plane) verifies the low-bits presence rule; the G6 (12-bit, different geometry, w not divisible by 64) verifies the row-reset and @@ -62,7 +62,7 @@ D60 files; the PowerShot G5 is the known exception (`06-crw-format.md` §4). The maintainer supplied 1,516 Canon EOS D60 files (private; local use only, nothing copied into the repository, verdicts only recorded here). With the revision-1 decoder, -223 failed on unseen code-table entries and 9 were misread as 10-bit (`06-crw-format.md`, +223 failed on unseen code-table entries and 9 were misread as 10-bit (`02-crw-format.md`, revision 2). After the two corrections every file was compared against LibRaw 0.22.2 `unprocessed_raw.exe -T`, every sensor sample, with the Rust decoder (`plan/crw/model/sweep_all.py`, local): diff --git a/crates/crw/clean-room/08-color.md b/crates/crw/clean-room/04-color.md similarity index 97% rename from crates/crw/clean-room/08-color.md rename to crates/crw/clean-room/04-color.md index 8a04151c..39088859 100644 --- a/crates/crw/clean-room/08-color.md +++ b/crates/crw/clean-room/04-color.md @@ -25,7 +25,7 @@ vs −0.67 GRBG). Unsampled PowerShots default to RGGB, unverified. ## 2 Active area, black level, white level - **Active area**: SensorInfo (0x1031) values 5–8 are an inclusive crop (left, top, right, - bottom); valid on every corpus body (G5/G6 included). See `02-container.md` §3. + bottom); valid on every corpus body (G5/G6 included). - **Black level**: measured from the masked borders. Observed (oracle data, per body class): the left border is dark up to 8 columns before the active area on the EOS bodies (≈127–129 DN on D60/10D, ≈128 on D30, ≈124–127 on 300D) but transitions bright earlier on @@ -105,8 +105,8 @@ vs −0.67 GRBG). Unsampled PowerShots default to RGGB, unverified. on all 10 decodable public corpus files, holdout RMS 0.009–0.043, including the D30 and the PowerShots, whose missing white balance the fitted matrix absorbs. **Update 2026-10-25:** the EOS D60 has dual-illuminant matrices of our own, fitted to - ColorChecker Classic shots under a tungsten bulb and direct sun (`10-chart-calibration.md`, - `crates/raw/src/chart_table.rs`). They give its white balance a Kelvin scale and are its + ColorChecker Classic shots under a tungsten bulb and direct sun (the fitting is part of + LightCraft's colour work, `crates/raw/src/chart_table.rs`). They give its white balance a Kelvin scale and are its colour when the JPEG fit fails; the look stays the JPEG fit. The other CRW bodies have none. ## 4 Orientation and capture time diff --git a/crates/crw/clean-room/04-derivation-log.md b/crates/crw/clean-room/04-derivation-log.md deleted file mode 100644 index 295b9e7a..00000000 --- a/crates/crw/clean-room/04-derivation-log.md +++ /dev/null @@ -1,315 +0,0 @@ -# CRW entropy-coding derivation log (P2) - -Chronological record of experiments and oracle probes, arw6-style. The oracle is LibRaw -0.22.2, official pre-built Windows binaries installed by the maintainer in -a local `LibRaw-0.22.2\bin` folder (2026-10-12); **only `bin\unprocessed_raw.exe` (plus -`libraw.dll`) is executed, as a black box. The LibRaw source tree that ships alongside those -binaries is never opened, read or searched** — all structure below comes from feeding inputs -to the binary and observing outputs. The Python reference model lives in the gitignored -`plan/crw/model/` (maintainer decision #4); findings and probe campaigns are recorded here. - -## 2026-10-12 — oracle baseline runs - -`unprocessed_raw.exe -T ` over 7 public CC0 samples (D30, D60, 10D-a, 300D-a, G1, G5, -G6), each copied to a scratch directory first. Output: 16-bit TIFF of the full raw frame. -Oracle frame sizes equal SensorInfo raw sizes on all 7. Oracle "margins" (active area) differ -from SensorInfo's crop rectangle (e.g. D60: LibRaw top=12/left=64 vs SensorInfo (72,16)) — -LibRaw's own choice, noted, not adopted. - -## 2026-10-12 — low-bits region: CONFIRMED (B6 settled) - -`p2_oracle_lsb.py` (committed) compares the oracle's pixel LSBs against the payload's -candidate low-bits regions, both packing orders: - -| File | Region [0:Q) LSB-first packing | Notes | -|---|---|---| -| D60, 10D-a, 300D-a, D30 | **100.000000 %** (1 629 584/1 629 584 … 809 536/809 536 bytes) | EOS prefix confirmed | -| G6 | **100.000000 %** (1 851 760/1 851 760) | at [0:Q), **not** [514:514+Q) — P1's reading of the G6 layout corrected: G6 low bits precede the 514 block, same as EOS | -| G5 | 47.34 % at [0:Q), 0.39 % at [514:514+Q) | anomaly, parked (scope decision #5: D60-class first) | -| G1 | no region (0.38 % everywhere) | 10-bit body, no low bits ✓ | - -**Packing: LSB-first** — byte b holds pixels 4i..4i+3 with pixel 4i in bits 1:0 and pixel -4i+3 in bits 7:6, row-major over the full raw frame. MSB-first packing scores ~6 %. - -Oracle code ranges: D60 max 4000, 10D max 4003, 300D max 4008, D30 max 3206, G6 max 4095 -(12-bit class); G1 max 936 (10-bit). G5/G6 min ≈ 101/102 (no black subtraction in the oracle). - -Corrections this forces on `03-payload-layout.md`: G6 (and presumably the G5 pattern, once -understood) is `[low bits Q][514-byte block][stuffed entropy][ff d9]`; the G6 "514 zeros at -offset 0" measured in P1 were the low bits of its first 2056 pixels (top masked row), and the -block after the low bits is *not* all-zero on G6. The EOS picture is unchanged. - -## 2026-10-12 — entropy stream: structure findings from known-plaintext analysis - -Using the D60 public file and its oracle decode (scripts `plan/crw/model/align.py`, -`sweep.py`, local): - -- Budget check: unstuffed entropy = 5 049 856 bytes = 6.198 bits/pixel. Under the - two-predictor diff model (B7) E[category] = 3.23 and H(category) ≈ 2.9 bits — model - "code(k) + k magnitude bits" fits the budget almost exactly. Single-predictor E[k] = 5.9 - does not fit. **Two predictors by column parity supported.** -- **Bit order is LSB-first within each byte** (after ff00 unstuffing). Hand-verified at two - anchors: stream start decodes as code `100` (stream order) + 10 magnitude bits = 511 = - JPEG-style magnitude of −512 (first pixel 0 against predictor 512); the first token of row 7 - (byte-aligned at bit 20776) decodes as a 2-bit code + 6 magnitude bits = +55 against - predictor 0 carried from row 6. B8's MSB-first recollection is **refuted**; so is its - "no byte-stuffing" (every 0xff in the entropy data is followed by 0x00, which the decoder - must strip — `03-payload-layout.md`). -- Predictors persist across rows (row 7's first diff is +55 against the row-6 value 0), i.e. - no per-row reset; image start uses 512/512 (10-bit "hi" domain: pixel = hi·4 + lowbits). -- Zero diffs are **run-coded** (a plain per-value zero code is inconsistent with the bits in - the 7 all-zero top rows, which compress to a strictly periodic 7-bit token). -- JPEG-style magnitude convention (negative d stored as d + 2^k − 1) consistent at both anchors. - -## 2026-10-12 — black-box probe campaign 1 (D60 body, patch-in-place) - -Harness `plan/crw/model/probe.py` (local): takes the public D60 file, zeroes the low-bits -region (so TIFF pixel = hi·4), overwrites the entropy region with chosen bits (LSB-first -packed, ff restuffed), optionally patches the DecoderTable selector, runs the oracle, reads -decoded rows. First probes (filler = repeated bit): - -| Entropy bits | Row 0 decoded (hi) | Reading | -|---|---|---| -| all 0 | 497 505 490 498 483 491 476 484 … (parities fall by 7) | repeated all-zero token = k=3 code + mag `000` = −7 per parity; first token longer (−15, k=4) | -| all 1 | 512 512 512 … | all-ones token(s) = zero diffs | -| `100` + 511 + zeros | 16144 505 16137 498 … | no clamping in the oracle; convention probe | - -Then `plan/crw/model/treewalk.py`: adaptive prefix expansion. Three aborted attempts taught -the method its final shape: (1) a race — the first walker's process survived its kill and -fought a second one over a shared probe file (fixed with per-PID work files; both result -sets discarded as corrupt); (2) enumerating magnitude bits explodes 2^k-fold; (3) leaf -detection via an assumed magnitude convention fails because the convention is *not* plain -JPEG (single-bit probes behind a known codeword give non-binary-weight responses). - -## 2026-10-12/13 — truncation probing of the real stream (campaign 2) - -`plan/crw/model/boundary.py` / `boundary2.py`: feed the oracle the real stream truncated at -bit B plus a constant filler. `boundary2`'s agreement scan (decode under 0-filler AND -1-filler; count leading pixels where both agree and match the real decode) yields **exact -token boundaries**: agreement can only grow when a token completes inside [0,B). - -Zero-region result (D60, table 0, bits 0–120 ≈ pixels 0–321, all-zero rows; data in -`bounds2_zero.json`, local): - -| token bits (stream order, LSB-first in bytes) | len | emits pixels | -|---|---|---| -| `1001111111111111` | 16 | 1 (pix 0, d = −512) | -| `011111111111100111111111` | 24 | 2 (zeros) | -| `11111011` | 8 | 61 → ends exactly at pixel 64 (EOB) | -| `0000101111100000` | 16 | 1 (pix 64, first of block 1) | -| `1110000` | 7 | 1 | -| `1` | 1 | 62 → ends exactly at pixel 128 (**1-bit EOB**) | -| `11111011` | 8 | 2 | -| `11111101` | 8 | 63 (ends at pixel 193) | -| `0111111` | 7 | 1 | -| `010111111` | 9 | 63 | -| `11011111` | 8 | 63 | -| `11101111` | 8 | 1 | - -Established: **64-pixel block structure** (B4's "blocks of 64" holds); an EOB-style token -ends a block; the same bit string decodes differently in different blocks (`11111011` = EOB -in block 0 but a 2-pixel token later) → at least two tables/contexts in play (B5's two-table -recollection pointing the right way). First-of-block tokens are long (16 bits here, carrying -the −512 initial diffs). The value-token magnitude convention is still open — it is *not* -plain JPEG — and is being extracted from a second agreement scan over row 7's textured data -(`bounds2_row7.json`). - -## 2026-10-13 — flip probing (campaign 3): code/magnitude split read directly - -`plan/crw/model/flipscan.py`: flip one bit of the real stream at a time (real -continuation), diff the full decode against the real decode. A flipped magnitude bit -changes exactly one pixel by a power of two; a flipped code bit derails the parse. - -First result, the pixel-0 token `1001111111111111` (16 bits): bits 0–7 are the code -(`10011111`), bits 8–15 are an 8-bit magnitude, weights 1,2,4,…,128 in stream order -(LSB-first), and the decoded value responds **linearly** across the would-be JPEG sign -boundary (pixel0 = m − 255 over the whole probed range m = 127…255): the magnitude -convention is offset-style, not JPEG's two-branch rule, at least for this token. A full -flip scan over a textured region (row 11) is mapping code and magnitude bits for ~25 -consecutive tokens. - -## 2026-10-13 — bit order corrected: MSB-first; token structure pinned - -A systematic contradiction (magnitude fields appearing split across bytes, "token -boundaries" clustering at 8 bits) resolved when the flip map was re-plotted in MSB-first -bit coordinates: an early mis-step had locked the analysis into LSB-first reading, which -mirrors every byte. **The stream is MSB-first after ff00 unstuffing.** In MSB coordinates -every magnitude field is contiguous and the JPEG rule is confirmed by exact branch-flip -deltas (e.g. flipping the top magnitude bit of a k=5, m=2 token: −29 → +18, Δ+47 observed). - -Flip-verified token chain (D60 table 0, pixels 34994–34999, bits 20784–20819): - -| bits | table | meaning | pixels | -|---|---|---|---| -| `01` + `00010` | A | k=5, m=2 → −29 | 34994 | -| `11111100` + `11110` | B | run 1 + k=5, m=30 → +30 | 34995 (zero), 34996 | -| `1100` + `1` | A | k=1 → +1 | 34997 | -| `00` + `010` | B | k=3, m=2 → −5 | 34998 | -| `00` + `010` | A | k=3 → −5 | 34999 | - -Established: -- **Two Huffman tables, strictly alternating per token (A, B, A, B, …)** — forced by - prefix-freeness: the verified codes cannot coexist in one table or in parity-assigned - tables, and alternation renders both tables prefix-free. (This refines B5.) -- Tokens carry a **(zero-run r, size s)** pair: r zero-diff pixels plus one value of - size s; s = 0 codes are pure runs/EOB. Predictors: two, by pixel parity, init 512, - carried across rows (flip cascades propagate stride-2 to the image end). -- The 10-bit "hi" plane is nearly constant in dark/flat areas — the reason the low-bits - plane exists: the sensor noise lives in the 2 raw LSBs, keeping the entropy-coded - plane highly compressible. - -Open: exact placement of the bundled zeros (before vs after the value) in corner cases, -EOB/block semantics, the first-token special case at pixel 0, and the complete code -tables for DecoderTableNumber 0, 1, 2 (being extracted by continued flip scans plus the -known-plaintext aligner `plan/crw/model/aligner4.py`). - -## 2026-10-13 — complete token grammar established (D60, table 0) - -The MSB-faithful agreement scan (`bounds3_row11_msb.json`, 187 exact micro-boundaries over -bits 20776–21500) plus a merge analysis settled the remaining structure. Sub-boundary rises -occur inside tokens (fillers can reproduce zero pixels), and merging every zero segment into -the following value yields a token stream that is fully consistent — zero semantic conflicts -and zero prefix violations — under exactly one model: - -- **64-pixel blocks** (image order). The token that starts at a block's first pixel comes - from a **"first" Huffman table**; every other token from a **"second" table**. (The - apparent two-alternating-tables stage was a phantom: fitted as alternating tables, the two - halves come out *identical*, collapsing to one "second" table. B5's recollection — "one - table for the first element of a block, another for the rest" — was exactly right.) -- **Token = code → (run r, size s)**: r zero-diff pixels, then one value of size s coded as - s JPEG-rule magnitude bits (MSB-first). s = 0 codes are EOB (zeros to end of block). -- Two DPCM predictors by pixel parity, init 512, never reset. - -Second-table codes verified so far (D60, DecoderTableNumber 0): `00`=(0,3), `100`=(0,2), -`101`=(0,5), `1100`=(0,1), `111011`=(1,2), `111100`=(1,3), `111101`=(1,1), `11111100`=(1,5), -`1111111000`=(2,1), `1111111100`=(2,3), `11111110111100`=(16,1), and 16-bit codes -(3,1) (4,1) (4,2) (5,1) (5,4) (5,5) (6,3) (6,4) (8,1). First-table: `101`=(0,2), -`111110`=(0,10) at pixel 0. The code set has a canonical Huffman shape. - -A seeded known-plaintext aligner (`aligner6.py`) is now walking the whole 6.5-Mpixel stream -to learn the remaining codes of both tables and verify every pixel; the same harness then -runs on files with DecoderTableNumbers 1 and 2 and on the other bodies. - -## 2026-10-13 — the carry chain, and the first/second table split confirmed - -Synthetic sentinel probes plus the zero-region MSB scan established two more rules: - -- **Carry on block DC**: each 64-pixel block's first diff is coded relative to the - previous block's *combined* first diff (`diff0 += carry; carry = diff0`). This single - rule reconciles pixel 0 (coded −512), pixel 64 (coded +512 → effective 0) and pixel - 128 (coded 0 → effective 0) with plain parity chains init 512 — the earlier - "predictor anomalies" were all carry. -- **The first-of-block token really does use a separate Huffman table** (`101` = (0,2) - there vs (0,5) in the main table, verified by decode-iteration at pixel 35008), i.e. - the classic first/second table structure of B5. -- The S-EOB of the main table is 9 bits (`111111011`); agreement boundaries inside - all-zero blocks proved unreliable (fillers self-heal on zeros), so zero-region token - readings are being re-derived by decode-iteration rather than scans. - -Current state: the main table has ~20 verified codes; a decode-and-learn loop -(`plan/crw/model/aligner8.py`) walks the real stream, decoding deterministically with -known codes and learning first-table/main-table rows at each stall, each verified -against the oracle pixels. - -## 2026-10-13 (cont.) — row reset; full model assembled - -Two refinements completed the structural model, each validated against the stream: - -- **Chains reset to 512 at every row start** (not row padding — a padded-grid variant was - tried and refuted by block alignment at pixel 35008). With the reset, every all-zero row - needs one k=10 "−512" value token at its first pixel, which is exactly what the stream - shows: a 15-bit second-table code (15,10) at the first row crossing, mag 511 → −512. -- The verified second table gains `01` = (0,5) (flip-verified at pixel 34994); both `01` - and `101` map to (0,5) — duplicate leaves are legal and present. - -Model now complete: flat 64-sample blocks; first/second Huffman tables (first = the token -starting each block); (run ≤ 15, size) leaves with JPEG MSB magnitudes; (0,0) mid-block = -EOB; per-block DC carry; parity chains init 512, reset at row starts; low-bits recombined -as the 2 LSBs. Remaining: complete code tables per DecoderTableNumber, extracted by the -known-plaintext walker (`aligner11.py`), then full-corpus verification. - -## 2026-10-13 — SELECTOR-0 TABLES COMPLETE; four bodies decode end-to-end - -The greedy known-plaintext extractor (`plan/crw/model/aligner12.py`: deterministic decode -with deep trial-validation per unknown code, flip-scans resolving the few ambiguous sites) -completed the DecoderTableNumber-0 Huffman tables. Results: - -- **D60 (public): the full 6 518 336-sample frame decodes from bit 0 against the oracle, - pixel-exact,** in seconds, with only the final 3 samples open (stream tail handling). -- **10D, 300D and Digital Rebel (selector 0) decode end-to-end with the same table** — - per-selector tables are shared across bodies, as the plan hoped. Each file exercised a - few additional rare high-k codes (merged). -- Final selector-0 tables: 11 first-table codes, 131 second-table codes (incl. the 16-bit - `11111111xxxxxxxx` run/size family); saved as `logs/table_sel0_d60class.json`. -- The decisive last fix: a hand-guessed 15-bit seed blocked the true 11-bit (1,7) code by - prefix; a flip-scan at the stall site read the real code and magnitude (m = 83) directly. - -Selector 1 (from the D30) and selector 2 (from the second 10D file) extractions running -with the same tooling. Remaining after that: G1/G5/G6 PowerShot handling (10-bit G1; the -G5 low-bits anomaly), the stream-tail rule, then the frozen format document and the -Python reference decoder for the P4 verification pass. - -## 2026-10-13 — selectors 1 and 2; G1, G6, private files; G5 isolated - -- Anchor hunting (truncation-probe agreement at byte boundaries + bit-level pinning) - plus structural bootstrap (reading each file's opening k=10 tokens and the steady - zero-block unit directly off the bits) made extraction routine for new table sets. -- **Selector 1** (extracted from private CRW_0476, topped up from the public D30): - D30 **EXACT**, PowerShot **G1 EXACT** (10-bit, no low-bits plane — detected by the - frozen unstuffed-ff scan), CRW_0476 **EXACT**. Cross-body sharing confirmed again. -- **G6 EXACT** with the selector-0 table once the low-bits rule was frozen (presence - by the ff-scan; entropy always at low-bits-size + 514). -- **G5 remains anomalous**: its low-bits plane matches the oracle's LSBs ~97.5 % in the - top rows (not 100 %), degrading to 0 % in the lower half, and no constant bit-shift - explains it; its entropy stream also diverges from the selector-1 parse within the - first rows. G5 is documented as unresolved and will be refused (or preview-only) by - the decoder pending further work. -- Selector 2 (from the public 10D-b) extraction completing. - -## 2026-10-13 — P2 complete; tables frozen; 28 of 29 files verify EXACT - -A final systematic fix (first-table magnitudes reach s = 11+: the carry-coded block DC -spans ±2046) unlocked selector 2 (extracted from the 300D-b after the saturated 10D-b -proved hostile learning ground). Table top-ups over every private file completed the -three table sets (12 first + 154/161/162 second codes). The final sweep: **28 of 29 -corpus files decode with zero mismatching samples** against LibRaw 0.22.2 — every EOS -body and selector, the 10-bit G1, the G6, and all 18 private D60s. The PowerShot G5 -remains the open anomaly (stream decodes, values diverge from sample 2056; low-bits -plane fits no tested layout) and is excluded pending follow-up. - -Frozen at this commit: `06-crw-format.md` (the format), `logs/table_sel{0,1,2}.json` -(the tables), `07-verification.md` (the evidence). P3 (black/WB/colour records) and P5 -(the Rust decoder) proceed only after maintainer review, per the plan. - -## 2026-10-13 — P5: Rust decoder lands; P3 colour/geometry derived - -The maintainer approved the freeze. `crates/raw/src/vendor/crw.rs` implements the frozen -model (tables generated from `logs/table_sel{0,1,2}.json` into `crw_tables.rs`); the Rust -decoder matched the oracle **sample-exact on all 28 decodable corpus files on its first -run** (`crates/raw/tests/crw_oracle.rs`, ~65 ms per 8 MP file). The G5 is refused by model -string. Colour/geometry work (CFA phases by chroma correlation + green-diagonal tests, -border black levels, 0x10a9/0x10aa white balance, rotation and timestamps) is documented in -`08-color.md`; ExifTool prose pages and a raw.pixls.us repository-listing fetch (to recover -the 11 corpus files' stable URLs for `cargo xtask corpus`) are logged in `sources.md`. - -## 2026-10-13 — Freeze reopened (revision 2): 1,516 D60 files, two corrections - -The maintainer put all his D60 files in `../RawImages` (1,516). The revision-1 decoder -failed on 232 of them, which the 29-file corpus could not have shown: - -- **223 "undecodable entropy code".** The second tables were learned from exercised codes - only: selector 0 had 154 of 162 symbols, selector 1 had 161. Laying out the 16-bit code - range showed exactly eight unassigned slots inside selector 0's range and one inside - selector 1's, against exactly eight and one missing symbols: (7..14, 10) and (14, 10). - For selector 1 the assignment follows by elimination; the learner (`aligner12.py`, same - known-plaintext method, LibRaw as the black box) confirmed it and assigned selector 0's - eight from the failing files. -- **9 files decoded as 10-bit garbage.** Their low-bits plane holds no 0xff byte, because - the lowest bit of nearly every sample is 0, so the revision-1 presence rule said "no - plane". My first reading (an all-zero plane) was wrong: I had counted 0xff bytes, not - nonzero bytes; the plane has 1.5 M nonzero bytes. The rule now also accepts the zero - header found right after the plane. - -Result: 1,516 of 1,516 EXACT, and the first corpus unchanged (`07-verification.md`). -A lesson for the remaining unverified bodies: a table learned from a few files is complete -only when its symbol count is; selector 2 was already at 162. diff --git a/crates/crw/clean-room/09-private-corpus.md b/crates/crw/clean-room/05-private-corpus.md similarity index 95% rename from crates/crw/clean-room/09-private-corpus.md rename to crates/crw/clean-room/05-private-corpus.md index e1b4566b..69ca4c9f 100644 --- a/crates/crw/clean-room/09-private-corpus.md +++ b/crates/crw/clean-room/05-private-corpus.md @@ -7,9 +7,9 @@ sample-exact against LibRaw 0.22.2 `unprocessed_raw.exe -T`. | Set | Files | What it is | What it established | |---|---:|---|---| -| `RawImages/` | 1,516 | Everyday photos, 2005–2014, all Auto white balance; ISO 100 (925), 200 (197), 400 (287), 800 (34), 1000 (73) | Freeze revision 2: two incomplete code tables and the plane-without-0xff case (`06-crw-format.md`); the low-ISO clip level (`08-color.md`); the D60 colour profile (`assets/camera-profiles/Canon_EOS_D60.json`, 1,424 usable photos) | -| `Cap&WhiteOut/` | 4 | Lens cap and blown-out frames, ISO 100 and 1000 (2026-10-13) | Black level at ISO 100 matches the image area; at ISO 1000 the border reads about 4 DN high. Clip level about 4000 at ISO 100, 4095 at ISO 1000 (`08-color.md`) | -| `WBModes/` | 7 | One overcast scene at Auto, Daylight, Cloudy, Tungsten, Fluorescent, Flash, Custom; ISO 100, within a minute (2026-10-14) | White-balance setting n uses ColorBalance slot n (`08-color.md`) | +| `RawImages/` | 1,516 | Everyday photos, 2005–2014, all Auto white balance; ISO 100 (925), 200 (197), 400 (287), 800 (34), 1000 (73) | Freeze revision 2: two incomplete code tables and the plane-without-0xff case (`02-crw-format.md`); the low-ISO clip level (`04-color.md`); the D60 colour profile (`assets/camera-profiles/Canon_EOS_D60.json`, 1,424 usable photos) | +| `Cap&WhiteOut/` | 4 | Lens cap and blown-out frames, ISO 100 and 1000 (2026-10-13) | Black level at ISO 100 matches the image area; at ISO 1000 the border reads about 4 DN high. Clip level about 4000 at ISO 100, 4095 at ISO 1000 (`04-color.md`) | +| `WBModes/` | 7 | One overcast scene at Auto, Daylight, Cloudy, Tungsten, Fluorescent, Flash, Custom; ISO 100, within a minute (2026-10-14) | White-balance setting n uses ColorBalance slot n (`04-color.md`) | | `TripodGrayCard/` | 14 | The same card filling most of the frame, tripod, f/8, ISO 100; two brackets (Auto 1/16–1/6 s, Cloudy 1/12–1/5 s), each starting and ending on the same speed; 14 s each (2026-10-14) | Black level confirmed; sensor linear; the spill strip (below) | | `GrayCard/` | 20 | An 18 % grey card (film-era), handheld, overcast; 5 frames at f/4 with exposure compensation, 15 at f/1.8 stepping 1/640–1/1600 s under Cloudy, Daylight and Auto (2026-10-14) | See below | diff --git a/crates/crw/clean-room/10-chart-calibration.md b/crates/crw/clean-room/10-chart-calibration.md deleted file mode 100644 index beb24d0b..00000000 --- a/crates/crw/clean-room/10-chart-calibration.md +++ /dev/null @@ -1,97 +0,0 @@ -# EOS D60 colour matrices from a ColorChecker (2026-10-25) - -The Canon EOS D60 has no published spectral sensitivities, so it is not in `spectral_table.rs`. -These are its DNG-style matrices (`ColorMatrix1/2`, `ForwardMatrix1/2`), fitted to our own -photographs of a ColorChecker Classic under two lights. They are in -`crates/raw/src/chart_table.rs`. Measurements come from the maintainer's own captures (names, -sizes and SHA-256 in `logs/chart-captures.tsv`; no image content recorded) and the chart's -published reference colours (`sources.md`). Scripts and measurements are local-only, in -`plan/crw/chart/`. - -## Captures - -| Set | Frames used | Light | Notes | -|---|---:|---|---| -| `SunlightColorCard/` | 20 of 22 | Direct afternoon sun, ISO 100 | Calibrite ColorChecker Classic (50th-anniversary edition, post-2014 formulation) leaning on a wall, filling most of the frame. Auto and Daylight white balance, a shutter bracket. Two frames dropped: a white-patch sample ≥ 3900 (clip ≈ 4000 at ISO 100, `08-color.md` §2). | -| `TungstonColorChart/` 2696–2706 | 11 | One household tungsten bulb, ISO 100 | Chart flat on a wall, a shutter bracket. Brightest white patch 2657 of ~4000. | -| `TungstonColorChart/` 2688–2691 | not used | Same bulb, 8 minutes earlier | Steady within the set, but its neutral reads ~3,040 K against ~2,900 K for the later set: the light changed (window light or bulb warm-up). The later, warmer and better-exposed set is used. | - -All 37 frames decode sample-exact against the black-box reference (`crw_oracle.rs`). - -An earlier set on a ColorChecker-layout clone (Pixel Perfect "24 Standard Color Calibration -Chart", 2026-10-15) was not used. The maker publishes no reference values, and its grey patches -are not neutral: against an 18 % grey board they read R/G 0.88–0.94 in shade and B/G up to -1.29 under warm LED light, the darker patches drifting most. That is the sign of greys printed -from coloured inks. - -## Measurement - -- Patches are located on the embedded JPEG (four corner patch centres, bilinear grid), tracked - per frame by phase correlation, and checked visually on drawn boxes. The tungsten-set chart - was upside down in the first four frames; patch order is mapped accordingly. -- Each patch is measured in a central box of about 40 % of its width in the linear raw (black - from the far masked border per CFA site; R, mean of both G, B per 2×2 site). Uniformity - across the box (3×3 sub-blocks): ≤ 4 % in sun, ≤ 7 % under the bulb (light falloff across - the closer chart). -- Frames are normalised by the four middle neutrals (patches 20–23), averaged, and white - balanced on those neutrals. White balance (R, B multipliers, G = 1): sun 2.362, 1.298; bulb - 1.353, 2.213. The neutrals agree on it to ±2 %. - -## Fit - -- The reference is X-Rite's published L\*a\*b\* (D50) for charts made after November 2014. -- The model is a 3×3 matrix from white-balanced camera RGB to XYZ (D50), with neutral (1, 1, 1) - held exactly at the engine's D50 white (`lightcraft_color::D50`): the third column is D50 - minus the first two. That is a DNG `ForwardMatrix`. -- **Chromaticity only.** Each prediction is scaled to the patch's reference luminance before - the CIEDE2000 residual. In the sun the chart warped as it heated: the grey row's brightness - slope swung from +4 % to −7 % across the 70 s of shooting, in time order, and the per-patch - brightness needed was not smooth across the chart. A fit including lightness would have - absorbed that into the matrix. Chromaticity still determines the whole 3×3 up to scale, - which the white constraint fixes. The cost is that luminance of saturated blues and purples - rests on the model, not the measurement. -- `ColorMatrix` per light = diag(1 / multipliers) · ForwardMatrix⁻¹ · Bradford(light → D50), - scaled so the light's white maps to camera values with maximum 1. By construction it maps - the filed illuminant's white to the measured camera neutral. - -| Light | CIEDE2000 on the 24 patches | Leaving each patch out | Worst | -|---|---|---|---| -| Sun (filed as D50) | mean 1.70, median 1.46 | mean 1.97 | white 4.5 | -| Bulb (filed as A) | mean 2.42, median 1.79 | mean 2.74 | bluish green 6.7 | - -- One matrix does not serve both lights: the sun's matrix on the bulb's chart gives mean 3.6 - (cyan 9.0), the bulb's on the sun's 3.4 (red 7.7). Hence the dual-illuminant entry. -- The neutral row reads a\* within ±0.4 for the middle greys; white and the darkest patches - are off by up to 2 (white patch sheen, flare). - -## Which illuminants - -The two lights' colour temperatures come from the camera's own white-balance presets -(ColorBalance slots, `08-color.md` §3) at the temperatures Canon's manuals give them (Daylight -5200 K, Tungsten 3200 K, Fluorescent 4000 K), interpolated in mired on ln(R/B): sun ~4,850 K, -bulb ~2,900 K. They are filed under the nearest DNG illuminants the engine knows, **D50 -(5003 K) and Standard Illuminant A (2856 K)**, so `Camera` now carries its own illuminant -pair (spectral rows: A and D65). - -## Checks in the app - -- `spectral::tests::the_chart_measured_eos_d60_reads_its_presets_in_kelvin`: the camera's - presets, which the fit never saw, read through the matrices as Tungsten 3,264 K, Daylight - 5,603 K, Cloudy 6,613 K, Flash 7,085 K, near the locus. Canon's nominal values are 3200, 5200, - 6000 and 6000 K. Order and spacing hold; the daylight presets read 8–15 % high, as Sony's - dial does against its spectral matrices (#730). -- Real files through the headless app (As Shot): WBModes Fluorescent 4,103 K, tint +26 (green - compensation); LED-lit room, Auto 3,761 K; open shade, Auto 6,596 K; sunlit chart, Auto - 4,762 K. -- `files::tests::crw_probes_name_their_camera_from_the_ciff_records`: CRW imports now take make, - model and capture time from the decoder's CIFF records (no Exif in a CRW; the catalog had an - empty camera, so a D60 stayed on the relative white-balance scale). - -## Limits - -- One body, one bulb, one afternoon. The chart is 24 patches; a 3×3 matrix cannot follow the - sensor's departures from the CIE observer (the bulb fit's worst patches are 5–7 ΔE). -- The lights are filed under standard illuminants they only approximate (~50 K for the bulb, - ~150 K for the sun). -- The D60's look is still the per-file JPEG fit and the pooled profile; these matrices give the - white-balance scale and the fallback colour only. diff --git a/crates/crw/clean-room/README.md b/crates/crw/clean-room/README.md index e50f822e..1174e713 100644 --- a/crates/crw/clean-room/README.md +++ b/crates/crw/clean-room/README.md @@ -5,43 +5,44 @@ produced in LightCraft (`clean-room/crw/`, branch `clean-room-crw-format-decode` `6ca439b5155820d2befeacd81ec0c6a80b2bd3bc`; the record is identical at `70c755e4`, where the decoder was ported) between 2026-10-12 and 2026-10-25 and moved here **verbatim** when the decoder became this shared crate. The documents are the evidence: they are not rewritten after the fact, so paths they mention are LightCraft's paths of that time (the -mapping is below), and `00-starting-assumptions.md` is committed once and never edited. +mapping is below). + +The derivation trail — the starting assumptions, the plan, the container notes, the +entropy-derivation log, the chart calibration, the probe scripts and their observational logs — is +not part of this record. What this record holds is what the decoder is built from and what a +reviewer needs to check it: the frozen format, its verification, the payload-layout and colour +notes, the private-corpus record, the source log and the frozen code tables. Those documents are +renumbered `01`–`05` in reading order. ## Documents | File | What it is | |---|---| -| [`00-starting-assumptions.md`](00-starting-assumptions.md) | What the author believed before consulting any source; every item marked unverified. Immutable. | -| [`01-plan.md`](01-plan.md) | The derivation plan, the corpus (public CC0 files with hashes; private files by name, size and hash), the maintainer's decisions (§7) and approval. | | [`sources.md`](sources.md) | Every URL opened and search run, with the time, what was asked and what was taken. The rules in force are at its top. | -| [`02-container.md`](02-container.md) | CIFF heap parsing from Canon's specification and ExifTool's prose pages, each fact cited. | -| [`03-payload-layout.md`](03-payload-layout.md) | The RawData payload partition established by statistics alone (no decoding): low-bits plane, 514-byte header, ff00-stuffed stream, tail. | -| [`04-derivation-log.md`](04-derivation-log.md) | The entropy-coding derivation, experiment by experiment, against the black-box reference. | -| [`06-crw-format.md`](06-crw-format.md) | **The frozen format** (revision 2) and its reopening rule. The decoder implements this document. | -| [`07-verification.md`](07-verification.md) | Full-frame exact comparison against the reference, zero tuning: 28 of 29 files, then 1,516 of 1,516 D60 files. | -| [`08-color.md`](08-color.md) | CFA phase, active area, black and white levels, white-balance records, orientation and time: what the decoder reports beyond the samples. §3's colour matrices concern LightCraft (below). | -| [`09-private-corpus.md`](09-private-corpus.md) | The maintainer's EOS D60 files and controlled captures (grey card, lens cap, white-balance presets) and what each set established. | -| [`10-chart-calibration.md`](10-chart-calibration.md) | EOS D60 colour matrices from ColorChecker captures. This is LightCraft's colour work (`crates/raw/src/chart_table.rs` there), kept here because it is part of the same record and uses the same captures. | +| [`01-payload-layout.md`](01-payload-layout.md) | The RawData payload partition established by statistics alone (no decoding): low-bits plane, 514-byte header, ff00-stuffed stream, tail. | +| [`02-crw-format.md`](02-crw-format.md) | **The frozen format** (revision 2) and its reopening rule. The decoder implements this document. | +| [`03-verification.md`](03-verification.md) | Full-frame exact comparison against the reference, zero tuning: 28 of 29 files, then 1,516 of 1,516 D60 files. | +| [`04-color.md`](04-color.md) | CFA phase, active area, black and white levels, white-balance records, orientation and time: what the decoder reports beyond the samples. §3's colour matrices concern LightCraft (below). | +| [`05-private-corpus.md`](05-private-corpus.md) | The maintainer's EOS D60 files and controlled captures (grey card, lens cap, white-balance presets) and what each set established. | -There is no `05-geometry-colour.md`: the plan named it, and the material went into `08-color.md`. +## Tables and logs -## Scripts and logs - -- `ciff_dump.py`, `p1_payload.py`, `p1b_markers.py`, `p1c_lowbits_bounds.py`, `p2_oracle_lsb.py`: - the planning and payload-statistics scripts (standard library only; run them with `python -I`). - Their public outputs are in `logs/` (`structure-*.txt`, `p1*.txt`, `verification-sweep.txt`). - `logs/table_sel{0,1,2}.json`: **the frozen code tables**, the source of truth that `tools/gen_tables.py` renders into `../src/tables.rs`. `table_sel0_d60class.json` is the - revision-1 selector-0 table, kept for the record. -- `logs/private-corpus.tsv`, `logs/chart-captures.tsv`: names, sizes and SHA-256 of the - maintainer's own files. No image content, metadata or thumbnails are recorded anywhere here. + revision-1 selector-0 table, kept for the record. These are the only logs that ship with the + crate: without them the frozen tables cannot be regenerated or audited. +- `logs/private-corpus.tsv`: names, sizes and SHA-256 of the maintainer's own files. No image + content, metadata or thumbnails are recorded anywhere here. - `logs/private/` (gitignored): per-file dumps of the private files, which can hold personal metadata. Local only. +- The probe scripts and their observational logs (`structure-*.txt`, `p1*.txt`, + `verification-sweep.txt`, `chart-captures.tsv`) are not part of this record. ## What stayed local - The Python reference model and probe harness (`plan/crw/model/` in LightCraft, gitignored), - by the maintainer's decision #4. Its findings are all in `04-derivation-log.md`. + by the maintainer's decision #4. Its findings are the entropy-coding derivation, which is not + part of this record. - The reference decoder: LibRaw 0.22.2, official pre-built Windows binaries installed by the maintainer; only `unprocessed_raw.exe` was ever run, as a black box. The source tree shipped alongside it was never opened, read or searched. @@ -65,11 +66,11 @@ There is no `05-geometry-colour.md`: the plan named it, and the material went in ## Rules that were in force -From `sources.md` and `01-plan.md`: no raw-decoder source may be opened, fetched, searched inside +From `sources.md`: no raw-decoder source may be opened, fetched, searched inside or requested (dcraw, LibRaw, rawspeed, rawloader/rawler, dnglab, darktable, RawTherapee, libopenraw, ExifTool's Perl code, including licence headers, comments and mirrors). Allowed: published format documentation in prose, published standards, the sample files, and a reference decoder's binary run as a black box with every probe logged. No tuning after the freeze: a mismatch reopens the derivation explicitly, with its cause and a new freeze commit recorded -(`06-crw-format.md`, revision 2, is the one reopening). The general standard this followed is +(`02-crw-format.md`, revision 2, is the one reopening). The general standard this followed is [`docs/clean-room.md`](../../../docs/clean-room.md). diff --git a/crates/crw/clean-room/ciff_dump.py b/crates/crw/clean-room/ciff_dump.py deleted file mode 100644 index 9a69a81f..00000000 --- a/crates/crw/clean-room/ciff_dump.py +++ /dev/null @@ -1,197 +0,0 @@ -"""CIFF/CRW container structure dump (planning phase — no raw-data decoding). - -Prints the heap tree, record entries (tag, storage class, data type, size, offset), -a few interpreted metadata records (make/model, ImageSpec, SensorInfo-ish u16 arrays, -DecoderTable u32s), and byte statistics of the RawData record. It never decodes the -compressed raw bitstream. - -Container layout implemented from published documentation only: -- Canon, "CIFF Specification on Image Data File" v1.0 r4, 1997-12-24 (archived PDF): - heap file header {byteOrder[2], headerLength u32, type[4], subType[4], version u32, - reserved1/2 u32}; heap = heap space + offsetTbl; u32 offsetTblOffset stored in the - final position of the heap; offsetTbl = numRecords u16 + entries {typeCode u16, - length u32, offset u32}; typeCode bits 15-14 storage (00 in heap space, 01 in record - entry), bits 13-11 dataType (000 byte, 001 ASCII, 010 word, 011 dword, 100 struct, - 101/110 heap record), bits 10-0 IDCode. -- ExifTool "canon_raw.html" (prose page): offsets are relative to the start of the - containing block's ValueData; a subdirectory is a complete block inside ValueData - whose length is the entry's Size; the root block runs from headerLength to EOF; - tag IDs incl. 0x2005 RawData, 0x1835 DecoderTable, 0x1031 SensorInfo, 0x080a - make/model, 0x1810 ImageInfo. - -Usage: python -I ciff_dump.py [--no-strings] file.crw [more.crw ...] ---no-strings suppresses record contents (for logs of private files: structure, -sizes, offsets and statistics only; ASCII make/model still shown, pixel data never). -""" - -import math -import struct -import sys -from collections import Counter - -# Tag names from ExifTool's published tag table (documentation page, not source). -TAG_NAMES = { - 0x0000: "NullRecord", 0x0001: "FreeBytes", - 0x0805: "CanonFileDescription", 0x080A: "CanonRawMakeModel", - 0x080B: "CanonFirmwareVersion", 0x0810: "OwnerName", - 0x0815: "CanonImageType", 0x0816: "OriginalFileName", - 0x0817: "ThumbnailFileName", - 0x100A: "TargetImageType", 0x1010: "ShutterReleaseMethod", - 0x1011: "ShutterReleaseTiming", 0x1016: "ReleaseSetting", - 0x101C: "BaseISO", 0x1028: "CanonFlashInfo", - 0x1029: "CanonFocalLength", 0x102A: "CanonShotInfo", - 0x102C: "CanonColorInfo2", 0x102D: "CanonCameraSettings", - 0x1030: "WhiteSample", 0x1031: "SensorInfo", - 0x1033: "CustomFunctions", 0x1038: "CanonAFInfo", - 0x10A9: "ColorBalance", 0x10AE: "ColorTemperature", - 0x10B4: "ColorSpace", 0x10B5: "RawJpgInfo", - 0x1803: "ImageFormat", 0x1804: "RecordID", - 0x1806: "SelfTimerTime", 0x1807: "TargetDistanceSetting", - 0x180B: "SerialNumber", 0x180E: "TimeStamp", - 0x1810: "ImageInfo", 0x1813: "FlashInfo", - 0x1814: "MeasuredEV", 0x1817: "FileNumber", - 0x1818: "ExposureInfo", 0x1834: "CanonModelID", - 0x1835: "DecoderTable", 0x183B: "SerialNumberFormat", - 0x2005: "RawData", 0x2007: "JpgFromRaw", 0x2008: "ThumbnailImage", - 0x2804: "ImageDescription", 0x2807: "CameraObject", - 0x3002: "ShootingRecord", 0x3003: "MeasuredInfo", - 0x3004: "CameraSpecification", 0x300A: "ImageProps", - 0x300B: "ExifInformation", -} - -DATATYPES = {0: "byte", 1: "ascii", 2: "word", 3: "dword", - 4: "struct", 5: "heap1", 6: "heap2", 7: "rsvd7"} - -MAX_DEPTH = 8 - - -def entropy(data): - if not data: - return 0.0 - counts = Counter(data) - n = len(data) - return -sum(c / n * math.log2(c / n) for c in counts.values()) - - -def byte_stats(data, label, out): - n = len(data) - out.append(f" {label}: {n} bytes, entropy {entropy(data):.4f} bits/byte") - if n >= 4096: - for part, name in ((data[:2048], "first 2 KiB"), (data[n // 2:n // 2 + 2048], "middle 2 KiB"), - (data[-2048:], "last 2 KiB")): - c = Counter(part) - top = ", ".join(f"{b:02x}:{cnt}" for b, cnt in c.most_common(4)) - out.append(f" {name}: entropy {entropy(part):.3f}, top bytes {top}, " - f"zeros {c.get(0, 0)}, ff {c.get(255, 0)}") - - -def dump_heap(f, blk_start, blk_len, depth, endian, out, show, path="root"): - """blk_start/blk_len: the block (ValueData + offsetTbl + trailing u32).""" - pre = " " * depth - if depth > MAX_DEPTH: - out.append(f"{pre}!! depth limit at {path}") - return - if blk_len < 6: - out.append(f"{pre}!! block too small ({blk_len}) at {path}") - return - f.seek(blk_start + blk_len - 4) - (tbl_off,) = struct.unpack(endian + "I", f.read(4)) - if tbl_off + 2 > blk_len - 4: - out.append(f"{pre}!! offsetTblOffset {tbl_off:#x} out of range (block {blk_len:#x}) at {path}") - return - f.seek(blk_start + tbl_off) - (nrec,) = struct.unpack(endian + "H", f.read(2)) - out.append(f"{pre}heap {path}: block @{blk_start:#x} len {blk_len:#x}, offsetTbl @+{tbl_off:#x}, {nrec} records") - if 2 + nrec * 10 > blk_len - tbl_off: - out.append(f"{pre}!! record table overruns block at {path}") - return - entries = [] - for i in range(nrec): - tc, length, offset = struct.unpack(endian + "HII", f.read(10)) - entries.append((tc, length, offset)) - for i, (tc, length, offset) in enumerate(entries): - stg = (tc >> 14) & 3 - dt = (tc >> 11) & 7 - tag_id = tc & 0x3FFF # dataType | IDCode, ExifTool's "TagID" - name = TAG_NAMES.get(tag_id, "?") - loc = {0: "heap", 1: "inrec", 2: "stg2?", 3: "stg3?"}[stg] - out.append(f"{pre} [{i}] tag {tc:#06x} id {tag_id:#06x} {name:<22} {DATATYPES[dt]:<6} " - f"{loc:<5} len {length:#x} off {offset:#x}") - if stg == 1: - continue # value lives in the length/offset fields themselves - if offset + length > blk_len: - out.append(f"{pre} !! record overruns block") - continue - abs_off = blk_start + offset - if dt in (5, 6): - dump_heap(f, abs_off, length, depth + 1, endian, out, show, f"{path}/{name or hex(tag_id)}") - elif tag_id == 0x2005: - f.seek(abs_off) - data = f.read(length) - byte_stats(data, "RawData", out) - elif tag_id in (0x2007, 0x2008): - f.seek(abs_off) - head = f.read(4) - out.append(f" starts with {'JPEG SOI' if head[:2] == bytes([0xFF, 0xD8]) else head.hex()}") - elif show and dt == 1 and length < 256: - f.seek(abs_off) - s = f.read(length).rstrip(b"\x00") - out.append(f"{pre} = {s!r}") - elif show and dt == 2 and length <= 64: - f.seek(abs_off) - vals = struct.unpack(endian + "H" * (length // 2), f.read(length & ~1)) - out.append(f"{pre} = u16 {vals}") - elif show and dt == 3 and length <= 64: - f.seek(abs_off) - vals = struct.unpack(endian + "I" * (length // 4), f.read(length & ~3)) - out.append(f"{pre} = u32 {vals}") - elif show and dt == 4 and tag_id == 0x1835 and length <= 64: - f.seek(abs_off) - vals = struct.unpack(endian + "I" * (length // 4), f.read(length & ~3)) - out.append(f"{pre} = DecoderTable u32 {vals}") - - -def dump(fname, show): - out = [f"== {fname}"] - with open(fname, "rb") as f: - f.seek(0, 2) - fsize = f.tell() - f.seek(0) - hdr = f.read(26) - if len(hdr) < 26: - out.append("!! file shorter than header") - return out - bo = hdr[0:2] - endian = {b"II": "<", b"MM": ">"}.get(bo) - if endian is None: - out.append(f"!! unknown byte order {bo!r} — not CIFF? first 16: {hdr[:16].hex()}") - return out - (hdr_len,) = struct.unpack(endian + "I", hdr[2:6]) - sig_type, sig_sub = hdr[6:10], hdr[10:14] - (version,) = struct.unpack(endian + "I", hdr[14:18]) - out.append(f" size {fsize}, byteOrder {bo.decode()}, headerLength {hdr_len}, " - f"type {sig_type!r} subType {sig_sub!r}, version {version:#010x}") - if sig_type != b"HEAP": - out.append("!! not a HEAP file — stopping (CHDK dump or other format?)") - return out - dump_heap(f, hdr_len, fsize - hdr_len, 0, endian, out, show) - return out - - -def main(): - args = sys.argv[1:] - show = True - if args and args[0] == "--no-strings": - show = False - args = args[1:] - for fname in args: - try: - lines = dump(fname, show) - except Exception as e: # planning tool: report, don't die - lines = [f"== {fname}", f"!! exception: {e!r}"] - print("\n".join(lines)) - print() - - -if __name__ == "__main__": - main() diff --git a/crates/crw/clean-room/logs/chart-captures.tsv b/crates/crw/clean-room/logs/chart-captures.tsv deleted file mode 100644 index c8030990..00000000 --- a/crates/crw/clean-room/logs/chart-captures.tsv +++ /dev/null @@ -1,39 +0,0 @@ -# EOS D60 ColorChecker Classic captures (maintainer, local only; never committed). Names, sizes, SHA-256; no image content. Generated 2026-10-25. -file bytes sha256 -SunlightColorCard/CRW_2666.CRW 7791628 371d8e18c12801c72754dd7b277163a866afc422c8edf6ad1ea822009e4170f8 -SunlightColorCard/CRW_2667.CRW 6638578 188d625c31ab44475a6cf54a5dae00fd861e26aafc90f4b5d18a1905c0a3cef5 -SunlightColorCard/CRW_2668.CRW 6832770 1189b8dc79642c57a8c9496b5629e36a1741c545d2b0b864d6e6e20d901510c1 -SunlightColorCard/CRW_2669.CRW 7079102 baf1686b27a9e0a04bf87c0b6cf55168f398d85247f8115a0af69b5e9ea1884c -SunlightColorCard/CRW_2670.CRW 7225298 067ba3f4d3026b8009cfe4babb5e4c3b28f988d95c48024a75c6715de32afccf -SunlightColorCard/CRW_2671.CRW 7504700 cf81cbb96389624318e890b5506221d49f5fba8e2d45447d87e0f3ff3e9bdc72 -SunlightColorCard/CRW_2672.CRW 7830852 7a59446bfb1f00d6eabd0115b8e5451c34f5d3d48f483cb27608ce9508e43ee5 -SunlightColorCard/CRW_2673.CRW 8053180 feeb8f818325d6ddb4e88b5eabe4a74f4cb5e1da9f70135ae3f85938c86695dc -SunlightColorCard/CRW_2674.CRW 8312060 45e7402c888582adadb607599eae80644d657d61c9607b426c411ee306e42daf -SunlightColorCard/CRW_2675.CRW 8565950 be28d2fa049aa4158ae9e3ec49dd28eb56fcd0ffa7af060235b8bda0d89dfd0a -SunlightColorCard/CRW_2676.CRW 7795508 76f7418a82e45e1034b3adacacf4bc756cb65a0d683e5bb3631992784e654f93 -SunlightColorCard/CRW_2677.CRW 7747450 a5e7bc618e6b80ea70f9bad6a1bf41ee63eadbc51df28b82d0864a73434f7964 -SunlightColorCard/CRW_2678.CRW 6815054 28824f0f64b087488d8dc5a66a8bdf717e83a4d4e1660676d07991bf32dffb96 -SunlightColorCard/CRW_2679.CRW 7050606 b5c915fa89542265deb4fa16da3ca7ffda01b1d961903836b0059d024316673a -SunlightColorCard/CRW_2680.CRW 7231410 a5780dfe99a2df2a3ed9de1888e91baa614b2e009a2b6e0a84d442b6bf1af9fd -SunlightColorCard/CRW_2681.CRW 7438088 eb4475919c2770f816d47787656ab7ecb5396f30d2c03628c3ab167ebbc47c65 -SunlightColorCard/CRW_2682.CRW 7584262 a6fe417b90ce41b03552963dee6b21e4f02b59a5c51148edc04af7224e8c59e9 -SunlightColorCard/CRW_2683.CRW 7811474 5402ad40e2689d78ade083d8855ecced5caea9368b9d77ecc4a61a16dc55aa36 -SunlightColorCard/CRW_2684.CRW 8043372 bbef5da457c0b6d6c45c74a99f3e318e6c80a7d3a0ad5c7cb7e13d24205f64fa -SunlightColorCard/CRW_2685.CRW 8242344 8d68d592a5c8933f0f35630114fe7017be808b007867beaf1a8d6d2bd1441f70 -SunlightColorCard/CRW_2686.CRW 8462380 79f7db32b18a834d5408500185c65d316387d1440aa61e26a11be85883fcc382 -SunlightColorCard/CRW_2687.CRW 7690732 0333d8d18390e77128f186239e6de71c0432ab6ee9669a71c210255e4d915d19 -TungstonColorChart/CRW_2688.CRW 6269390 d9c4334edfe3636a8a7e69f29c1d1e89f820b7b704697684d7996faee41f5d43 -TungstonColorChart/CRW_2689.CRW 6001348 1e8b47645d76a71fd9fb4111c4dce805d6dcd41ac82e6b4ce52debbf71565455 -TungstonColorChart/CRW_2690.CRW 6114056 f63fe49a9ed13af33ddb8d8ed8d75de4e304a3bc2c1f1d4275ba7edbc4f459f3 -TungstonColorChart/CRW_2691.CRW 6299730 ad8a2844c00965fb6a12e8fc85a6e8bfbdf1924e6d1a68b55472701cb8ffa115 -TungstonColorChart/CRW_2696.CRW 6169684 105ee515c94da78dedc1f8d777982c4a1e62781b160428524fbe922f62b3ef5c -TungstonColorChart/CRW_2697.CRW 5565982 327eebf702185127feb8a5f4bfac5d7ebad5532f7417b302997b19b97d7ff4a8 -TungstonColorChart/CRW_2698.CRW 5709092 f0fca20f8dcd51046e53bebf9fb9f624677ee674ad739622cdb2578977669705 -TungstonColorChart/CRW_2699.CRW 5873492 57724a1ff507cf4e78c80a34cd4e0b77eac397322228d32e5215fc3869853dd1 -TungstonColorChart/CRW_2700.CRW 5988146 6c97ae2c72114034e2fbe6aed0ef97f475696e43b56ed62273eccbfbba4cbaa0 -TungstonColorChart/CRW_2701.CRW 6202756 c4032e46b753a718d8748e269a9da5d124891d07912542368ea867529143aa62 -TungstonColorChart/CRW_2702.CRW 6409582 89b65c1c07e0685841046db85923b0bf80e747fed141ddf0335ba12dc75f3f3c -TungstonColorChart/CRW_2703.CRW 6407270 50179d0a330c967a74d76bc8342eea1052689c5c544aa1a913769b241ac6f736 -TungstonColorChart/CRW_2704.CRW 6483154 6bac4d8229a8d0fff1445d212ca63e53911dc7bda988743215009257b87787fa -TungstonColorChart/CRW_2705.CRW 7170488 629f94131e7f598964d2635e3039a20c441563201c4f3cbf177d83ad6b061f78 -TungstonColorChart/CRW_2706.CRW 6383026 67657e7a21f6fd8580012fccd2210afccf285e747bc4d39b8ea775811528ce83 diff --git a/crates/crw/clean-room/logs/p1-payload-public.txt b/crates/crw/clean-room/logs/p1-payload-public.txt deleted file mode 100644 index a528740c..00000000 --- a/crates/crw/clean-room/logs/p1-payload-public.txt +++ /dev/null @@ -1,195 +0,0 @@ -== ../../corpus/crw/crw-canon-eos-10d-a.crw - raw 3152x2068 S=7134454 tbl=0 f1=0 f2=514 L=5504868 S-L=1629586 wh/4=1629584 S-L-wh/4=2 - leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) - last 8 bytes: 95ad8fb002120000 - prefix hypothesis region [0 : wh/4) as 2-bit rows: - prefix: 2068 rows of 788 B, leading all-zero rows: 7 - rows 0..top: mean 85.37 ent 5.190 - top..top+32: mean 127.88 ent 7.993 - middle 32: mean 126.86 ent 7.992 - last 32: mean 127.02 ent 7.993 - suffix hypothesis region [L : L+wh/4) as 2-bit rows: - suffix: 2068 rows of 788 B, leading all-zero rows: 0 - rows 0..top: mean 132.23 ent 7.944 - top..top+32: mean 131.83 ent 7.953 - middle 32: mean 128.56 ent 7.959 - last 32: mean 126.95 ent 7.956 - compressed-candidate stats: - after-prefix [wh/4 : wh/4+L): entropy 7.9684 (first MiB), 7.9644 (last MiB) - from-0 [0 : L): entropy 7.9960 (first MiB), 7.9530 (last MiB) - stuffing check: ff count 13614 (0.2473%), ff00 digrams 13614, expected-if-independent 64.4 - -== ../../corpus/crw/crw-canon-eos-10d-b.crw - raw 3152x2068 S=7244858 tbl=2 f1=0 f2=514 L=5615272 S-L=1629586 wh/4=1629584 S-L-wh/4=2 - leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) - last 8 bytes: 0c849b8b12b70000 - prefix hypothesis region [0 : wh/4) as 2-bit rows: - prefix: 2068 rows of 788 B, leading all-zero rows: 7 - rows 0..top: mean 64.32 ent 5.152 - top..top+32: mean 121.98 ent 7.958 - middle 32: mean 126.50 ent 7.992 - last 32: mean 127.17 ent 7.993 - suffix hypothesis region [L : L+wh/4) as 2-bit rows: - suffix: 2068 rows of 788 B, leading all-zero rows: 0 - rows 0..top: mean 117.92 ent 7.937 - top..top+32: mean 117.33 ent 7.941 - middle 32: mean 114.53 ent 7.905 - last 32: mean 117.63 ent 7.904 - compressed-candidate stats: - after-prefix [wh/4 : wh/4+L): entropy 7.9877 (first MiB), 7.9150 (last MiB) - from-0 [0 : L): entropy 7.9898 (first MiB), 7.9220 (last MiB) - stuffing check: ff count 12542 (0.2234%), ff00 digrams 12542, expected-if-independent 97.9 - -== ../../corpus/crw/crw-canon-eos-10d-c.crw - raw 3152x2068 S=7455524 tbl=2 f1=0 f2=514 L=5825938 S-L=1629586 wh/4=1629584 S-L-wh/4=2 - leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) - last 8 bytes: 9c12ce7bde990000 - prefix hypothesis region [0 : wh/4) as 2-bit rows: - prefix: 2068 rows of 788 B, leading all-zero rows: 7 - rows 0..top: mean 78.12 ent 5.144 - top..top+32: mean 126.89 ent 7.992 - middle 32: mean 127.89 ent 7.993 - last 32: mean 127.04 ent 7.991 - suffix hypothesis region [L : L+wh/4) as 2-bit rows: - suffix: 2068 rows of 788 B, leading all-zero rows: 0 - rows 0..top: mean 119.60 ent 7.946 - top..top+32: mean 117.81 ent 7.946 - middle 32: mean 104.15 ent 7.770 - last 32: mean 114.88 ent 7.856 - compressed-candidate stats: - after-prefix [wh/4 : wh/4+L): entropy 7.8675 (first MiB), 7.8315 (last MiB) - from-0 [0 : L): entropy 7.9961 (first MiB), 7.9712 (last MiB) - stuffing check: ff count 6993 (0.1200%), ff00 digrams 6993, expected-if-independent 47.5 - -== ../../corpus/crw/crw-canon-eos-300d-a.crw - raw 3152x2068 S=6929638 tbl=0 f1=0 f2=514 L=5300052 S-L=1629586 wh/4=1629584 S-L-wh/4=2 - leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) - last 8 bytes: e438ce510ad90000 - prefix hypothesis region [0 : wh/4) as 2-bit rows: - prefix: 2068 rows of 788 B, leading all-zero rows: 7 - rows 0..top: mean 61.27 ent 5.230 - top..top+32: mean 127.52 ent 7.994 - middle 32: mean 127.63 ent 7.993 - last 32: mean 127.31 ent 7.992 - suffix hypothesis region [L : L+wh/4) as 2-bit rows: - suffix: 2068 rows of 788 B, leading all-zero rows: 0 - rows 0..top: mean 132.59 ent 7.950 - top..top+32: mean 130.96 ent 7.963 - middle 32: mean 131.10 ent 7.958 - last 32: mean 134.35 ent 7.947 - compressed-candidate stats: - after-prefix [wh/4 : wh/4+L): entropy 7.9405 (first MiB), 7.9600 (last MiB) - from-0 [0 : L): entropy 7.9961 (first MiB), 7.9634 (last MiB) - stuffing check: ff count 17631 (0.3327%), ff00 digrams 17631, expected-if-independent 99.9 - -== ../../corpus/crw/crw-canon-eos-300d-b.crw - raw 3152x2068 S=7206250 tbl=2 f1=0 f2=514 L=5576664 S-L=1629586 wh/4=1629584 S-L-wh/4=2 - leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) - last 8 bytes: f8ff00eefe7f0000 - prefix hypothesis region [0 : wh/4) as 2-bit rows: - prefix: 2068 rows of 788 B, leading all-zero rows: 7 - rows 0..top: mean 61.49 ent 5.196 - top..top+32: mean 127.65 ent 7.989 - middle 32: mean 112.85 ent 7.534 - last 32: mean 119.27 ent 7.834 - suffix hypothesis region [L : L+wh/4) as 2-bit rows: - suffix: 2068 rows of 788 B, leading all-zero rows: 0 - rows 0..top: mean 124.00 ent 7.968 - top..top+32: mean 127.27 ent 7.978 - middle 32: mean 127.36 ent 7.985 - last 32: mean 130.81 ent 7.973 - compressed-candidate stats: - after-prefix [wh/4 : wh/4+L): entropy 7.9774 (first MiB), 7.9887 (last MiB) - from-0 [0 : L): entropy 7.8118 (first MiB), 7.9812 (last MiB) - stuffing check: ff count 28275 (0.5070%), ff00 digrams 28275, expected-if-independent 265.2 - -== ../../corpus/crw/crw-canon-eos-d30.crw - raw 2224x1456 S=2824648 tbl=1 f1=0 f2=514 L=2015110 S-L=809538 wh/4=809536 S-L-wh/4=2 - leading zero run: 0 bytes (= 0.00 rows of 2-bit LSBs) - last 8 bytes: fadfd6feb7f50000 - prefix hypothesis region [0 : wh/4) as 2-bit rows: - prefix: 1456 rows of 556 B, leading all-zero rows: 0 - rows 0..top: mean 139.68 ent 7.911 - top..top+32: mean 127.19 ent 7.990 - middle 32: mean 127.59 ent 7.988 - last 32: mean 131.21 ent 7.924 - suffix hypothesis region [L : L+wh/4) as 2-bit rows: - suffix: 1456 rows of 556 B, leading all-zero rows: 0 - rows 0..top: mean 123.77 ent 7.922 - top..top+32: mean 123.76 ent 7.936 - middle 32: mean 116.78 ent 7.895 - last 32: mean 116.20 ent 7.890 - compressed-candidate stats: - after-prefix [wh/4 : wh/4+L): entropy 7.9343 (first MiB), 7.9341 (last MiB) - from-0 [0 : L): entropy 7.9949 (first MiB), 7.9410 (last MiB) - stuffing check: ff count 2576 (0.1278%), ff00 digrams 2576, expected-if-independent 14.1 - -== ../../corpus/crw/crw-canon-eos-d60.crw - raw 3152x2068 S=6694146 tbl=0 f1=0 f2=514 L=5064559 S-L=1629587 wh/4=1629584 S-L-wh/4=3 - leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) - last 8 bytes: 9385933317ff0000 - prefix hypothesis region [0 : wh/4) as 2-bit rows: - prefix: 2068 rows of 788 B, leading all-zero rows: 7 - rows 0..top: mean 55.84 ent 5.113 - top..top+32: mean 127.09 ent 7.992 - middle 32: mean 127.69 ent 7.993 - last 32: mean 127.98 ent 7.992 - suffix hypothesis region [L : L+wh/4) as 2-bit rows: - suffix: 2068 rows of 788 B, leading all-zero rows: 0 - rows 0..top: mean 130.50 ent 7.942 - top..top+32: mean 131.88 ent 7.949 - middle 32: mean 131.47 ent 7.951 - last 32: mean 126.89 ent 7.967 - compressed-candidate stats: - after-prefix [wh/4 : wh/4+L): entropy 7.9003 (first MiB), 7.9694 (last MiB) - from-0 [0 : L): entropy 7.9960 (first MiB), 7.9672 (last MiB) - stuffing check: ff count 14189 (0.2802%), ff00 digrams 14189, expected-if-independent 76.1 - -== ../../corpus/crw/crw-canon-eos-digital-rebel.crw - raw 3152x2068 S=6406164 tbl=0 f1=0 f2=514 L=4776577 S-L=1629587 wh/4=1629584 S-L-wh/4=3 - leading zero run: 5516 bytes (= 7.00 rows of 2-bit LSBs) - last 8 bytes: 191093af87ff0000 - prefix hypothesis region [0 : wh/4) as 2-bit rows: - prefix: 2068 rows of 788 B, leading all-zero rows: 7 - rows 0..top: mean 58.54 ent 3.603 - top..top+32: mean 144.80 ent 5.469 - middle 32: mean 163.93 ent 4.986 - last 32: mean 154.41 ent 5.106 - suffix hypothesis region [L : L+wh/4) as 2-bit rows: - suffix: 2068 rows of 788 B, leading all-zero rows: 0 - rows 0..top: mean 128.96 ent 7.960 - top..top+32: mean 129.35 ent 7.966 - middle 32: mean 127.02 ent 7.922 - last 32: mean 122.47 ent 7.926 - compressed-candidate stats: - after-prefix [wh/4 : wh/4+L): entropy 7.9244 (first MiB), 7.9355 (last MiB) - from-0 [0 : L): entropy 5.9926 (first MiB), 7.9542 (last MiB) - stuffing check: ff count 6718 (0.1406%), ff00 digrams 6718, expected-if-independent 36.3 - -== ../../corpus/crw/crw-canon-powershot-g1.crw - raw 2144x1560 S=2257354 tbl=1 f1=0 f2=514 L=2257351 S-L=3 wh/4=836160 S-L-wh/4=-836157 - leading zero run: 514 bytes (= 0.96 rows of 2-bit LSBs) - last 8 bytes: 34e0702c2fffd900 - no wh/4 region (PowerShot class): compressed assumed [0 : L) - entropy first MiB 7.9226, last MiB 7.9396 - stuffing check: ff count 2676 (0.1185%), ff00 digrams 2676, expected-if-independent 11.4 - tail region [L : S): 3 bytes, hex ffd900 - -== ../../corpus/crw/crw-canon-powershot-g5.crw - raw 2672x1968 S=4861806 tbl=1 f1=0 f2=514 L=4861804 S-L=2 wh/4=1314624 S-L-wh/4=-1314622 - leading zero run: 514 bytes (= 0.77 rows of 2-bit LSBs) - last 8 bytes: 71928822fd7fffd9 - no wh/4 region (PowerShot class): compressed assumed [0 : L) - entropy first MiB 7.9958, last MiB 7.9087 - stuffing check: ff count 9974 (0.2052%), ff00 digrams 4958, expected-if-independent 54.4 - tail region [L : S): 2 bytes, hex ffd9 - -== ../../corpus/crw/crw-canon-powershot-g6.crw - raw 3160x2344 S=7313156 tbl=0 f1=0 f2=514 L=7313154 S-L=2 wh/4=1851760 S-L-wh/4=-1851758 - leading zero run: 514 bytes (= 0.65 rows of 2-bit LSBs) - last 8 bytes: dfbf7efdfbf7ffd9 - no wh/4 region (PowerShot class): compressed assumed [0 : L) - entropy first MiB 7.9995, last MiB 7.9596 - stuffing check: ff count 18811 (0.2572%), ff00 digrams 10815, expected-if-independent 103.5 - tail region [L : S): 2 bytes, hex ffd9 - diff --git a/crates/crw/clean-room/logs/p1b-markers-public.txt b/crates/crw/clean-room/logs/p1b-markers-public.txt deleted file mode 100644 index d0442666..00000000 --- a/crates/crw/clean-room/logs/p1b-markers-public.txt +++ /dev/null @@ -1,109 +0,0 @@ -== ../../corpus/crw/crw-canon-eos-10d-a.crw - raw 3152x2068 tbl=0 f2=514 L=5504868 lowbits=yes - ff followed by: 00:13614 - markers: none (pure stuffed entropy data) - comp[0:24] : 000000000000000000000000000000000000000000000000 - comp[-8:] : 9df595ad8fb00212 - raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 - raw[-8:] : 95ad8fb002120000 - low[-8:] : fc3590b2d228d3dd (end of low-bits region) - -== ../../corpus/crw/crw-canon-eos-10d-b.crw - raw 3152x2068 tbl=2 f2=514 L=5615272 lowbits=yes - ff followed by: 00:12542 - markers: none (pure stuffed entropy data) - comp[0:24] : 000000000000000000000000000000000000000000000000 - comp[-8:] : 38650c849b8b12b7 - raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 - raw[-8:] : 0c849b8b12b70000 - low[-8:] : c6371ccbb9d8dd2c (end of low-bits region) - -== ../../corpus/crw/crw-canon-eos-10d-c.crw - raw 3152x2068 tbl=2 f2=514 L=5825938 lowbits=yes - ff followed by: 00:6993 - markers: none (pure stuffed entropy data) - comp[0:24] : 000000000000000000000000000000000000000000000000 - comp[-8:] : e7389c12ce7bde99 - raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 - raw[-8:] : 9c12ce7bde990000 - low[-8:] : eb6d0fe72354f02c (end of low-bits region) - -== ../../corpus/crw/crw-canon-eos-300d-a.crw - raw 3152x2068 tbl=0 f2=514 L=5300052 lowbits=yes - ff followed by: 00:17631 - markers: none (pure stuffed entropy data) - comp[0:24] : 000000000000000000000000000000000000000000000000 - comp[-8:] : 22c9e438ce510ad9 - raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 - raw[-8:] : e438ce510ad90000 - low[-8:] : a93331aa85cdb788 (end of low-bits region) - -== ../../corpus/crw/crw-canon-eos-300d-b.crw - raw 3152x2068 tbl=2 f2=514 L=5576664 lowbits=yes - ff followed by: 00:28275 - markers: none (pure stuffed entropy data) - comp[0:24] : 000000000000000000000000000000000000000000000000 - comp[-8:] : bbe9f8ff00eefe7f - raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 - raw[-8:] : f8ff00eefe7f0000 - low[-8:] : bbb7b7b7bbb777c7 (end of low-bits region) - -== ../../corpus/crw/crw-canon-eos-d30.crw - raw 2224x1456 tbl=1 f2=514 L=2015110 lowbits=yes - ff followed by: 00:2576 - markers: none (pure stuffed entropy data) - comp[0:24] : 000000000000000000000000000000000000000000000000 - comp[-8:] : 7f5bfadfd6feb7f5 - raw[504:544] : a57fba0edb9db8e63afc7b4eaa3c6dae0efbabaf3fcc8bbee8f93db67aabbee9ce27ba68a9bfaa26 - raw[-8:] : fadfd6feb7f50000 - low[-8:] : ffffffffffffffff (end of low-bits region) - -== ../../corpus/crw/crw-canon-eos-d60.crw - raw 3152x2068 tbl=0 f2=514 L=5064559 lowbits=yes - ff followed by: 00:14189 - markers: none (pure stuffed entropy data) - comp[0:24] : 000000000000000000000000000000000000000000000000 - comp[-8:] : 51f18a9385933317 - raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 - raw[-8:] : 9385933317ff0000 - low[-8:] : 82961c457559780e (end of low-bits region) - -== ../../corpus/crw/crw-canon-eos-digital-rebel.crw - raw 3152x2068 tbl=0 f2=514 L=4776577 lowbits=yes - ff followed by: 00:6718 - markers: none (pure stuffed entropy data) - comp[0:24] : 000000000000000000000000000000000000000000000000 - comp[-8:] : fc86e6191093af87 - raw[504:544] : 00000000000000000000000000000000000000000000000000000000000000000000000000000000 - raw[-8:] : 191093af87ff0000 - low[-8:] : 66ee4e6c4cc6cece (end of low-bits region) - -== ../../corpus/crw/crw-canon-powershot-g1.crw - raw 2144x1560 tbl=1 f2=514 L=2257351 lowbits=no - ff followed by: 00:2676 - markers: none (pure stuffed entropy data) - comp[0:24] : 000000000000000000000000000000000000000000000000 - comp[-8:] : 3efec534e0702c2f - raw[504:544] : 00000000000000000000f83df43825313b62bf97ebdc24f476407e600ce3f30b9211928d5f8de276 - raw[-8:] : 34e0702c2fffd900 - -== ../../corpus/crw/crw-canon-powershot-g5.crw - raw 2672x1968 tbl=1 f2=514 L=4861804 lowbits=no - ff followed by: 00:4958, fa:40, e7:35, d8:35, 24:33, c0:32, d3:31, 34:30 - markers: ff01:20, ff02:16, ff03:24, ff04:23, ff05:16, ff06:23, ff07:23, ff08:14, ff09:16, ff0a:21, ff0b:22, ff0c:16, ff0d:22, ff0e:26, ff0f:24, ff10:27, ff11:25, ff12:17, ff13:30, ff14:21, ff15:13, ff16:15, ff17:22, ff18:19, ff19:22, ff1a:18, ff1b:24, ff1c:18, ff1d:9, ff1e:20, ff1f:20, ff20:22, ff21:18, ff22:20, ff23:19, ff24:33, ff25:12, ff26:15, ff27:24, ff28:22, ff29:27, ff2a:18, ff2b:21, ff2c:14, ff2d:23, ff2e:15, ff2f:17, ff30:16, ff31:18, ff32:22, ff33:17, ff34:30, ff35:15, ff36:20, ff37:19, ff38:21, ff39:18, ff3a:16, ff3b:19, ff3c:16, ff3d:27, ff3e:16, ff3f:19, ff40:22, ff41:20, ff42:23, ff43:16, ff44:13, ff45:15, ff46:25, ff47:12, ff48:10, ff49:19, ff4a:18, ff4b:17, ff4c:25, ff4d:17, ff4e:25, ff4f:17, ff50:27, ff51:24, ff52:20, ff53:17, ff54:17, ff55:20, ff56:21, ff57:19, ff58:18, ff59:17, ff5a:13, ff5b:14, ff5c:18, ff5d:27, ff5e:21, ff5f:22, ff60:18, ff61:15, ff62:15, ff63:15, ff64:24, ff65:25, ff66:15, ff67:13, ff68:19, ff69:16, ff6a:13, ff6b:17, ff6c:26, ff6d:24, ff6e:18, ff6f:12, ff70:22, ff71:13, ff72:17, ff73:18, ff74:16, ff75:13, ff76:24, ff77:24, ff78:19, ff79:13, ff7a:26, ff7b:23, ff7c:28, ff7d:14, ff7e:15, ff7f:19, ff80:22, ff81:20, ff82:17, ff83:23, ff84:15, ff85:22, ff86:16, ff87:24, ff88:24, ff89:25, ff8a:18, ff8b:13, ff8c:18, ff8d:18, ff8e:23, ff8f:15, ff90:20, ff91:19, ff92:17, ff93:17, ff94:20, ff95:10, ff96:14, ff97:18, ff98:15, ff99:14, ff9a:22, ff9b:20, ff9c:20, ff9d:21, ff9e:21, ff9f:18, ffa0:21, ffa1:14, ffa2:20, ffa3:18, ffa4:21, ffa5:22, ffa6:24, ffa7:10, ffa8:16, ffa9:22, ffaa:22, ffab:16, ffac:25, ffad:22, ffae:28, ffaf:10, ffb0:18, ffb1:20, ffb2:19, ffb3:20, ffb4:20, ffb5:16, ffb6:18, ffb7:18, ffb8:19, ffb9:25, ffba:24, ffbb:12, ffbc:21, ffbd:12, ffbe:28, ffbf:22, ffc0:32, ffc1:21, ffc2:20, ffc3:14, ffc4:25, ffc5:14, ffc6:24, ffc7:18, ffc8:19, ffc9:17, ffca:23, ffcb:16, ffcc:20, ffcd:18, ffce:21, ffcf:17, ffd0:18, ffd1:24, ffd2:12, ffd3:31, ffd4:23, ffd5:16, ffd6:24, ffd7:23, ffd8:35, ffd9:17, ffda:15, ffdb:30, ffdc:16, ffdd:21, ffde:17, ffdf:12, ffe0:24, ffe1:22, ffe2:16, ffe3:23, ffe4:25, ffe5:24, ffe6:13, ffe7:35, ffe8:19, ffe9:14, ffea:25, ffeb:13, ffec:24, ffed:26, ffee:20, ffef:17, fff0:24, fff1:17, fff2:20, fff3:17, fff4:19, fff5:23, fff6:19, fff7:18, fff8:21, fff9:17, fffa:40, fffb:12, fffc:18, fffd:30, fffe:25, ffff:14 - first markers at: [('0x2b9', '0x43'), ('0x36a', '0x9d'), ('0x669', '0x95'), ('0x6a3', '0x37'), ('0x909', '0x29'), ('0x93a', '0xad'), ('0x9f9', '0x7f'), ('0xa55', '0x68'), ('0xc5f', '0x47'), ('0xf97', '0x65'), ('0x1068', '0x3a'), ('0x1210', '0x51')] - comp[0:24] : 000000000000000000000000000000000000000000000000 - comp[-8:] : 426771928822fd7f - raw[504:544] : 000000000000000000009e16f4366ff2c547b59c5e132991784465d6f539fd37876356e9155982fa - raw[-8:] : 71928822fd7fffd9 - -== ../../corpus/crw/crw-canon-powershot-g6.crw - raw 3160x2344 tbl=0 f2=514 L=7313154 lowbits=no - ff followed by: 00:10815, ff:68, 37:61, 77:60, 7f:59, bb:58, ee:58, 33:56 - markers: ff01:24, ff02:24, ff03:31, ff04:17, ff05:39, ff06:29, ff07:31, ff08:23, ff09:31, ff0a:32, ff0b:26, ff0c:33, ff0d:27, ff0e:34, ff0f:29, ff10:27, ff11:22, ff12:33, ff13:28, ff14:27, ff15:35, ff16:24, ff17:31, ff18:18, ff19:26, ff1a:28, ff1b:28, ff1c:24, ff1d:21, ff1e:30, ff1f:29, ff20:35, ff21:38, ff22:31, ff23:37, ff24:32, ff25:34, ff26:18, ff27:22, ff28:30, ff29:29, ff2a:25, ff2b:23, ff2c:26, ff2d:36, ff2e:44, ff2f:27, ff30:38, ff31:38, ff32:27, ff33:56, ff34:19, ff35:27, ff36:31, ff37:61, ff38:27, ff39:26, ff3a:25, ff3b:41, ff3c:23, ff3d:32, ff3e:24, ff3f:51, ff40:39, ff41:34, ff42:21, ff43:21, ff44:26, ff45:35, ff46:25, ff47:33, ff48:27, ff49:35, ff4a:27, ff4b:31, ff4c:19, ff4d:25, ff4e:34, ff4f:31, ff50:22, ff51:23, ff52:17, ff53:35, ff54:29, ff55:28, ff56:29, ff57:30, ff58:21, ff59:30, ff5a:25, ff5b:35, ff5c:19, ff5d:33, ff5e:25, ff5f:26, ff60:32, ff61:35, ff62:31, ff63:26, ff64:29, ff65:20, ff66:32, ff67:23, ff68:38, ff69:27, ff6a:29, ff6b:27, ff6c:34, ff6d:32, ff6e:43, ff6f:24, ff70:22, ff71:24, ff72:34, ff73:50, ff74:30, ff75:26, ff76:28, ff77:60, ff78:36, ff79:21, ff7a:35, ff7b:47, ff7c:31, ff7d:27, ff7e:27, ff7f:59, ff80:35, ff81:35, ff82:33, ff83:22, ff84:27, ff85:20, ff86:23, ff87:24, ff88:21, ff89:24, ff8a:26, ff8b:31, ff8c:18, ff8d:23, ff8e:27, ff8f:41, ff90:22, ff91:21, ff92:26, ff93:28, ff94:32, ff95:27, ff96:27, ff97:29, ff98:33, ff99:29, ff9a:34, ff9b:32, ff9c:25, ff9d:33, ff9e:18, ff9f:28, ffa0:31, ffa1:25, ffa2:22, ffa3:29, ffa4:19, ffa5:20, ffa6:20, ffa7:32, ffa8:43, ffa9:33, ffaa:33, ffab:34, ffac:18, ffad:32, ffae:26, ffaf:16, ffb0:31, ffb1:23, ffb2:35, ffb3:50, ffb4:38, ffb5:29, ffb6:29, ffb7:43, ffb8:28, ffb9:22, ffba:34, ffbb:58, ffbc:32, ffbd:23, ffbe:34, ffbf:37, ffc0:24, ffc1:32, ffc2:32, ffc3:30, ffc4:28, ffc5:22, ffc6:28, ffc7:29, ffc8:31, ffc9:30, ffca:26, ffcb:32, ffcc:43, ffcd:45, ffce:56, ffcf:42, ffd0:29, ffd1:23, ffd2:24, ffd3:41, ffd4:22, ffd5:29, ffd6:31, ffd7:28, ffd8:27, ffd9:27, ffda:30, ffdb:25, ffdc:49, ffdd:42, ffde:47, ffdf:54, ffe0:25, ffe1:31, ffe2:32, ffe3:32, ffe4:37, ffe5:36, ffe6:36, ffe7:30, ffe8:31, ffe9:44, ffea:24, ffeb:31, ffec:47, ffed:49, ffee:58, ffef:45, fff0:24, fff1:24, fff2:30, fff3:45, fff4:36, fff5:27, fff6:36, fff7:48, fff8:29, fff9:25, fffa:38, fffb:49, fffc:48, fffd:42, fffe:37, ffff:68 - first markers at: [('0x307', '0xd2'), ('0x397', '0xb1'), ('0x428', '0xb0'), ('0x60f', '0xa0'), ('0x66c', '0x23'), ('0x691', '0x8d'), ('0x69e', '0x69'), ('0x79c', '0xd6'), ('0x827', '0x40'), ('0x8f4', '0x6a'), ('0x9fe', '0xa5'), ('0xa09', '0x88')] - comp[0:24] : 000000000000000000000000000000000000000000000000 - comp[-8:] : f7efdfbf7efdfbf7 - raw[504:544] : 00000000000000000000de521366dccd8cfe291a1c468992a578c2b0ad703f581c0b0667c07d2499 - raw[-8:] : dfbf7efdfbf7ffd9 - diff --git a/crates/crw/clean-room/logs/p1c-bounds-public.txt b/crates/crw/clean-room/logs/p1c-bounds-public.txt deleted file mode 100644 index 64aa1d69..00000000 --- a/crates/crw/clean-room/logs/p1c-bounds-public.txt +++ /dev/null @@ -1,11 +0,0 @@ -crw-canon-eos-10d-a.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 6241 total, 6241 in lowbits, 0 in entropy (expect 0) -crw-canon-eos-10d-b.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 6503 total, 6503 in lowbits, 0 in entropy (expect 0) -crw-canon-eos-10d-c.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 6139 total, 6139 in lowbits, 0 in entropy (expect 0) -crw-canon-eos-300d-a.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 6418 total, 6418 in lowbits, 0 in entropy (expect 0) -crw-canon-eos-300d-b.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 7180 total, 7180 in lowbits, 0 in entropy (expect 0) -crw-canon-eos-d30.crw EOS layout: lowbits [0:809536) + zeros(514) + entropy; non-stuffed ff: 3450 total, 3450 in lowbits, 0 in entropy (expect 0) -crw-canon-eos-d60.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 6419 total, 6419 in lowbits, 0 in entropy (expect 0) -crw-canon-eos-digital-rebel.crw EOS layout: lowbits [0:1629584) + zeros(514) + entropy; non-stuffed ff: 23394 total, 23394 in lowbits, 0 in entropy (expect 0) -crw-canon-powershot-g1.crw PS 10-bit layout: zeros(514) + entropy; no non-stuffed ff (pure stuffed stream) -crw-canon-powershot-g5.crw PS 12-bit layout: zeros(514) + lowbits [514:1315138) + entropy; all 5002 non-stuffed ff inside lowbits; first 697, last 1315128 -crw-canon-powershot-g6.crw PS 12-bit layout: zeros(514) + lowbits [514:1852274) + entropy; all 7928 non-stuffed ff inside lowbits; first 775, last 1852264 diff --git a/crates/crw/clean-room/logs/structure-d60-public.txt b/crates/crw/clean-room/logs/structure-d60-public.txt deleted file mode 100644 index 28337ee1..00000000 --- a/crates/crw/clean-room/logs/structure-d60-public.txt +++ /dev/null @@ -1,73 +0,0 @@ -== corpus/crw/crw-canon-eos-d60.crw - size 8591234, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 -heap root: block @0x1a len 0x831768, offsetTbl @+0x831744, 3 records - [0] tag 0x2005 id 0x2005 RawData struct heap len 0x662502 off 0x0 - RawData: 6694146 bytes, entropy 7.9868 bits/byte - first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 - middle 2 KiB: entropy 7.864, top bytes 8c:25, cb:21, 5d:18, 63:17, zeros 11, ff 4 - last 2 KiB: entropy 7.890, top bytes f1:16, 6d:16, 76:16, 55:15, zeros 11, ff 5 - [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x1cdc00 off 0x662502 - starts with JPEG SOI - [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x1642 off 0x830102 - heap root/ImageProps: block @0x83011c len 0x1642, offsetTbl @+0x15ba, 13 records - [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 - [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc - = u32 (3072, 2048, 1065353216, 0, 8, 24, 257) - [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x6d8f0000 off 0xcfba0d73 - [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xd8327aab - [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0xf42a3 off 0x9792ef1a - [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 - = u32 (1467566074, 0, 0) - [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 - = b'CRW_0099.CRW' - [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 - = b'CRW_0099.THM' - [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 - [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 - heap root/ImageProps/MeasuredInfo: block @0x830284 len 0x10, offsetTbl @+0x0, 1 records - [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x3fc00000 off 0xe2577c1e - [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 - heap root/ImageProps/ImageDescription: block @0x830294 len 0x5a, offsetTbl @+0x40, 2 records - [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 - = b'EOS D60 CMOS RAW IMAGE' - [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 - = b'CRW:EOS D60 CMOS RAW IMAGE' - [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 - heap root/ImageProps/CameraObject: block @0x8302ee len 0xb2, offsetTbl @+0x8e, 3 records - [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 - = b'' - [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 - = b'Canon\x00Canon EOS D60\x00\x0e{\xd7\xd8e\x18iG\x03,,c' - [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 - heap root/ImageProps/CameraObject/CameraSpecification: block @0x83032e len 0x4e, offsetTbl @+0x20, 4 records - [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x3645276f off 0x5982da24 - [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xa552febb - [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0xfedcfe5 - [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 - = b'Firmware Version 1.0.4' - [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x1336 off 0x284 - heap root/ImageProps/ExifInformation: block @0x8303a0 len 0x1336, offsetTbl @+0x129a, 15 records - [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 - [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x0 off 0x2620392 - [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x36 off 0x0 - = u16 (54, 0, 160, 52, 32768, 245, 65504, 0, 3, 0, 8, 8, 0, 0, 12288, 0, 0, 0, 1, 0, 0, 32768, 244, 99, 0, 0, 252) - [3] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x56 off 0x36 - [4] tag 0x0036 id 0x0036 ? byte heap len 0xbd6 off 0x8c - [5] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x1668000 off 0xed8d4a16 - [6] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xc62 - = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) - [7] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xc84 - = u32 (0, 0, 514, 5064559) - [8] tag 0x1033 id 0x1033 CustomFunctions word heap len 0x22 off 0xc94 - = u16 (32, 0, 515, 768, 1025, 1280, 1536, 1792, 2048, 2304, 2560, 2818, 3074, 3328, 3584, 3841, 18747) - [9] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xcb6 - = u16 (26, 279, 368, 371, 172, 219, 434, 436, 284, 162, 491, 494, 431) - [10] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xcd0 - = u16 (26, 281, 413, 415, 193, 223, 488, 490, 319, 150, 505, 507, 439) - [11] tag 0x10c2 id 0x10c2 ? word heap len 0x554 off 0xcea - [12] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x123e - = u16 (10, 440, 1021, 1027, 563) - [13] tag 0x10a8 id 0x10a8 ? word heap len 0x10 off 0x1248 - = u16 (16, 6000, 5312, 5989, 3161, 3717, 6503, 4118) - [14] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x42 off 0x1258 - diff --git a/crates/crw/clean-room/logs/structure-public.txt b/crates/crw/clean-room/logs/structure-public.txt deleted file mode 100644 index e5674341..00000000 --- a/crates/crw/clean-room/logs/structure-public.txt +++ /dev/null @@ -1,887 +0,0 @@ -== corpus/crw/crw-canon-eos-10d-a.crw - size 7787330, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 -heap root: block @0x1a len 0x76d328, offsetTbl @+0x76d304, 3 records - [0] tag 0x2005 id 0x2005 RawData struct heap len 0x6cdcf6 off 0x0 - RawData: 7134454 bytes, entropy 7.9776 bits/byte - first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 - middle 2 KiB: entropy 7.877, top bytes db:20, d3:16, b6:16, de:16, zeros 6, ff 3 - last 2 KiB: entropy 7.882, top bytes ac:17, d4:17, 91:17, 6c:17, zeros 12, ff 4 - [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x9dde2 off 0x6cdcf6 - starts with JPEG SOI - [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x182c off 0x76bad8 - heap root/ImageProps: block @0x76baf2 len 0x182c, offsetTbl @+0x17a4, 13 records - [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 - [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc - = u32 (3072, 2048, 1065353216, 0, 8, 24, 257) - [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x3af0000 off 0x9b869aeb - [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xf5657a8 - [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x1af8f9 off 0xde23c13d - [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 - = u32 (1228574022, 0, 0) - [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 - = b'CRW_7673.CRW' - [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 - = b'CRW_7673.THM' - [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 - [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 - heap root/ImageProps/MeasuredInfo: block @0x76bc5a len 0x10, offsetTbl @+0x0, 1 records - [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x40cc0000 off 0x70138c52 - [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 - heap root/ImageProps/ImageDescription: block @0x76bc6a len 0x5a, offsetTbl @+0x40, 2 records - [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 - = b'EOS 10D CMOS RAW IMAGE' - [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 - = b'CRW:EOS 10D CMOS RAW IMAGE' - [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 - heap root/ImageProps/CameraObject: block @0x76bcc4 len 0xb2, offsetTbl @+0x8e, 3 records - [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 - = b'Horst Kakuschke' - [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 - = b'Canon\x00Canon EOS 10D\x00:g\x8fZ\xb5k\xa7\xea\xca\xda|\x1a' - [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 - heap root/ImageProps/CameraObject/CameraSpecification: block @0x76bd04 len 0x4e, offsetTbl @+0x20, 4 records - [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0xdb7145a off 0xde776b76 - [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0x8762cef2 - [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0xe63b7d8c - [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 - = b'Firmware Version 2.0.1' - [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x1520 off 0x284 - heap root/ImageProps/ExifInformation: block @0x76bd76 len 0x1520, offsetTbl @+0x142a, 24 records - [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 - [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x180000 off 0x2620392 - [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x42 off 0x0 - [3] tag 0x1093 id 0x1093 ? word heap len 0x12 off 0x42 - = u16 (18, 0, 0, 0, 0, 0, 65535, 65535, 65535) - [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x54 - [5] tag 0x0036 id 0x0036 ? byte heap len 0xe82 off 0xb0 - [6] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x80000168 off 0x4f2612e3 - [7] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xf32 - = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) - [8] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xf54 - = u32 (0, 0, 514, 5504868) - [9] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x30 off 0xf64 - = u16 (7, 7, 3072, 2048, 3072, 2048, 151, 151, 1014, 608, 0, 0, 0, 64928, 64522, 0, 0, 65030, 0, 506, 0, 0, 8, 65535) - [10] tag 0x1033 id 0x1033 CustomFunctions word heap len 0x24 off 0xf94 - = u16 (36, 256, 513, 768, 1024, 1280, 1536, 1792, 2048, 2304, 2560, 2818, 3072, 3328, 3584, 3841, 4097, 4352) - [11] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xfb8 - = u16 (26, 288, 343, 343, 160, 231, 433, 433, 275, 154, 444, 444, 384) - [12] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xfd2 - = u16 (26, 314, 364, 365, 165, 234, 428, 430, 262, 174, 484, 487, 412) - [13] tag 0x10c2 id 0x10c2 ? word heap len 0x374 off 0xfec - [14] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x1360 - = u16 (10, 418, 1021, 1027, 467) - [15] tag 0x10a8 id 0x10a8 ? word heap len 0x14 off 0x136a - = u16 (20, 4174, 5190, 7000, 6000, 3232, 3934, 6083, 5190, 5200) - [16] tag 0x10ad id 0x10ad ? word heap len 0x3e off 0x137e - = u16 (62, 21468, 8650, 7000, 6000, 5600, 5200, 4700, 4200, 3343, 1668, 64833, 65115, 65180, 65240, 65271, 65307, 65361, 65429, 64, 582, 65341, 65183, 65148, 65121, 65106, 65097, 65083, 65072, 65051, 65251) - [17] tag 0x10ae id 0x10ae ColorTemperature word heap len 0x2 off 0x140e - = u16 (5200,) - [18] tag 0x10af id 0x10af ? word heap len 0x2 off 0x1410 - = u16 (1024,) - [19] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x52 off 0x13bc - [20] tag 0x10b4 id 0x10b4 ColorSpace word heap len 0x2 off 0x1412 - = u16 (1,) - [21] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x1414 - = u16 (10, 2, 2, 1536, 1024) - [22] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x141e - = u16 (0, 159, 7, 112) - [23] tag 0x183b id 0x183b SerialNumberFormat dword heap len 0x4 off 0x1426 - = u32 (2415919104,) - -== corpus/crw/crw-canon-eos-10d-b.crw - size 10822040, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 -heap root: block @0x1a len 0xa5217e, offsetTbl @+0xa5215a, 3 records - [0] tag 0x2005 id 0x2005 RawData struct heap len 0x6e8c3a off 0x0 - RawData: 7244858 bytes, entropy 7.9718 bits/byte - first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 - middle 2 KiB: entropy 7.894, top bytes 8c:18, 23:17, 5b:16, 32:16, zeros 14, ff 2 - last 2 KiB: entropy 7.824, top bytes 52:20, 8c:19, 84:19, 92:18, zeros 17, ff 2 - [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x367cf4 off 0x6e8c3a - starts with JPEG SOI - [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x182c off 0xa5092e - heap root/ImageProps: block @0xa50948 len 0x182c, offsetTbl @+0x17a4, 13 records - [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 - [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc - = u32 (3072, 2048, 1065353216, 0, 8, 24, 257) - [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x0 off 0x1 - [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0x80002400 - [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x127edf off 0x1001000 - [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 - = u32 (1105175848, 0, 0) - [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 - = b'CRW_2127.CRW' - [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 - = b'CRW_2127.THM' - [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 - [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 - heap root/ImageProps/MeasuredInfo: block @0xa50ab0 len 0x10, offsetTbl @+0x0, 1 records - [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x40a80000 off 0x0 - [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 - heap root/ImageProps/ImageDescription: block @0xa50ac0 len 0x5a, offsetTbl @+0x40, 2 records - [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 - = b'EOS 10D CMOS RAW IMAGE' - [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 - = b'CRW:EOS 10D CMOS RAW IMAGE' - [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 - heap root/ImageProps/CameraObject: block @0xa50b1a len 0xb2, offsetTbl @+0x8e, 3 records - [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 - = b'Bill Ferguson' - [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 - = b'Canon\x00Canon EOS 10D\x00\x00\x00\x00\x06\x00\x04 \x80\x01\x04' - [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 - heap root/ImageProps/CameraObject/CameraSpecification: block @0xa50b5a len 0x4e, offsetTbl @+0x20, 4 records - [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x54ab324a off 0x200100 - [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0x1042 - [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0x1000000 - [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 - = b'Firmware Version 2.0.1' - [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x1520 off 0x284 - heap root/ImageProps/ExifInformation: block @0xa50bcc len 0x1520, offsetTbl @+0x142a, 24 records - [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 - [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x550000 off 0x2620392 - [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x42 off 0x0 - [3] tag 0x1093 id 0x1093 ? word heap len 0x12 off 0x42 - = u16 (18, 0, 0, 0, 0, 0, 65535, 65535, 65535) - [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x54 - [5] tag 0x0036 id 0x0036 ? byte heap len 0xe82 off 0xb0 - [6] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x80000168 off 0x20 - [7] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xf32 - = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) - [8] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xf54 - = u32 (2, 0, 514, 5615272) - [9] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x30 off 0xf64 - = u16 (7, 7, 3072, 2048, 3072, 2048, 151, 151, 1014, 608, 0, 0, 0, 64928, 64522, 0, 0, 65030, 0, 506, 0, 0, 31, 65535) - [10] tag 0x1033 id 0x1033 CustomFunctions word heap len 0x24 off 0xf94 - = u16 (36, 256, 513, 768, 1024, 1280, 1537, 1792, 2053, 2304, 2560, 2816, 3072, 3328, 3584, 3840, 4096, 4352) - [11] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xfb8 - = u16 (26, 353, 375, 375, 177, 268, 431, 432, 285, 190, 476, 477, 428) - [12] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xfd2 - = u16 (26, 326, 364, 365, 161, 254, 428, 430, 268, 186, 484, 487, 416) - [13] tag 0x10c2 id 0x10c2 ? word heap len 0x374 off 0xfec - [14] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x1360 - = u16 (10, 345, 1021, 1027, 1301) - [15] tag 0x10a8 id 0x10a8 ? word heap len 0x14 off 0x136a - = u16 (20, 4028, 5190, 7000, 6000, 3232, 3930, 6080, 2818, 5200) - [16] tag 0x10ad id 0x10ad ? word heap len 0x3e off 0x137e - = u16 (62, 21468, 8650, 7000, 6000, 5600, 5200, 4700, 4200, 3343, 1668, 64821, 65116, 65184, 65246, 65279, 65317, 65373, 65444, 83, 613, 65393, 65229, 65191, 65165, 65148, 65139, 65125, 65112, 65100, 65341) - [17] tag 0x10ae id 0x10ae ColorTemperature word heap len 0x2 off 0x140e - = u16 (5200,) - [18] tag 0x10af id 0x10af ? word heap len 0x2 off 0x1410 - = u16 (1024,) - [19] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x52 off 0x13bc - [20] tag 0x10b4 id 0x10b4 ColorSpace word heap len 0x2 off 0x1412 - = u16 (1,) - [21] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x1414 - = u16 (10, 3, 0, 3072, 2048) - [22] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x141e - = u16 (0, 159, 7, 112) - [23] tag 0x183b id 0x183b SerialNumberFormat dword heap len 0x4 off 0x1426 - = u32 (2415919104,) - -== corpus/crw/crw-canon-eos-10d-c.crw - size 10583702, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 -heap root: block @0x1a len 0xa17e7c, offsetTbl @+0xa17e58, 3 records - [0] tag 0x2005 id 0x2005 RawData struct heap len 0x71c324 off 0x0 - RawData: 7455524 bytes, entropy 7.9666 bits/byte - first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 - middle 2 KiB: entropy 7.847, top bytes 82:21, b2:18, 98:18, 84:18, zeros 15, ff 1 - last 2 KiB: entropy 7.785, top bytes 08:23, 9c:20, 38:20, cc:20, zeros 7, ff 0 - [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x2fa308 off 0x71c324 - starts with JPEG SOI - [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x182c off 0xa1662c - heap root/ImageProps: block @0xa16646 len 0x182c, offsetTbl @+0x17a4, 13 records - [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 - [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc - = u32 (3072, 2048, 1065353216, 0, 8, 24, 257) - [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x41020000 off 0xf2c31662 - [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xc3af7160 - [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x12f557 off 0x777d1c94 - [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 - = u32 (1109761444, 0, 0) - [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 - = b'CRW_2455.CRW' - [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 - = b'CRW_2455.THM' - [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 - [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 - heap root/ImageProps/MeasuredInfo: block @0xa167ae len 0x10, offsetTbl @+0x0, 1 records - [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x41160000 off 0x9a0c5382 - [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 - heap root/ImageProps/ImageDescription: block @0xa167be len 0x5a, offsetTbl @+0x40, 2 records - [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 - = b'EOS 10D CMOS RAW IMAGE' - [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 - = b'CRW:EOS 10D CMOS RAW IMAGE' - [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 - heap root/ImageProps/CameraObject: block @0xa16818 len 0xb2, offsetTbl @+0x8e, 3 records - [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 - = b'Bill Ferguson' - [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 - = b'Canon\x00Canon EOS 10D\x00\xfe\x19g\xba1\x94E\x02\x1cc(\xca' - [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 - heap root/ImageProps/CameraObject/CameraSpecification: block @0xa16858 len 0x4e, offsetTbl @+0x20, 4 records - [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x54ab324a off 0x18e17812 - [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xd734ef03 - [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0x70fc1863 - [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 - = b'Firmware Version 2.0.1' - [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x1520 off 0x284 - heap root/ImageProps/ExifInformation: block @0xa168ca len 0x1520, offsetTbl @+0x142a, 24 records - [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 - [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x180000 off 0x2620392 - [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x42 off 0x0 - [3] tag 0x1093 id 0x1093 ? word heap len 0x12 off 0x42 - = u16 (18, 0, 0, 0, 0, 0, 65535, 65535, 65535) - [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x54 - [5] tag 0x0036 id 0x0036 ? byte heap len 0xe82 off 0xb0 - [6] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x80000168 off 0x1ba7180e - [7] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xf32 - = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) - [8] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xf54 - = u32 (2, 0, 514, 5825938) - [9] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x30 off 0xf64 - = u16 (7, 7, 3072, 2048, 3072, 2048, 151, 151, 1014, 608, 0, 0, 0, 64928, 64522, 0, 0, 65030, 0, 506, 0, 0, 113, 65535) - [10] tag 0x1033 id 0x1033 CustomFunctions word heap len 0x24 off 0xf94 - = u16 (36, 256, 513, 768, 1024, 1280, 1537, 1792, 2053, 2304, 2560, 2816, 3072, 3328, 3584, 3840, 4096, 4352) - [11] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xfb8 - = u16 (26, 353, 375, 375, 177, 268, 431, 432, 285, 190, 476, 477, 428) - [12] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xfd2 - = u16 (26, 326, 364, 365, 161, 254, 428, 430, 268, 186, 484, 487, 416) - [13] tag 0x10c2 id 0x10c2 ? word heap len 0x374 off 0xfec - [14] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x1360 - = u16 (10, 508, 1021, 1027, 856) - [15] tag 0x10a8 id 0x10a8 ? word heap len 0x14 off 0x136a - = u16 (20, 5347, 5190, 7000, 6000, 3232, 3930, 6080, 2818, 5200) - [16] tag 0x10ad id 0x10ad ? word heap len 0x3e off 0x137e - = u16 (62, 21468, 8650, 7000, 6000, 5600, 5200, 4700, 4200, 3343, 1668, 64821, 65116, 65184, 65246, 65279, 65317, 65373, 65444, 83, 613, 65393, 65229, 65191, 65165, 65148, 65139, 65125, 65112, 65100, 65341) - [17] tag 0x10ae id 0x10ae ColorTemperature word heap len 0x2 off 0x140e - = u16 (5200,) - [18] tag 0x10af id 0x10af ? word heap len 0x2 off 0x1410 - = u16 (1024,) - [19] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x52 off 0x13bc - [20] tag 0x10b4 id 0x10b4 ColorSpace word heap len 0x2 off 0x1412 - = u16 (1,) - [21] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x1414 - = u16 (10, 3, 0, 3072, 2048) - [22] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x141e - = u16 (0, 159, 7, 112) - [23] tag 0x183b id 0x183b SerialNumberFormat dword heap len 0x4 off 0x1426 - = u32 (2415919104,) - -== corpus/crw/crw-canon-eos-300d-a.crw - size 9056348, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 -heap root: block @0x1a len 0x8a3042, offsetTbl @+0x8a301e, 3 records - [0] tag 0x2005 id 0x2005 RawData struct heap len 0x69bce6 off 0x0 - RawData: 6929638 bytes, entropy 7.9804 bits/byte - first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 - middle 2 KiB: entropy 7.886, top bytes bb:21, 9c:20, 8c:17, 18:14, zeros 13, ff 10 - last 2 KiB: entropy 7.850, top bytes ae:20, bb:18, e3:18, 6e:18, zeros 7, ff 4 - [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x205b64 off 0x69bce6 - starts with JPEG SOI - [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x17d4 off 0x8a184a - heap root/ImageProps: block @0x8a1864 len 0x17d4, offsetTbl @+0x174c, 13 records - [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 - [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0x0 - = u32 (3072, 2048, 1065353216, 270, 8, 24, 257) - [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x9ceb0000 off 0xc97abdef - [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xb1dab4b5 - [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0xfb8bd off 0x0 - [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x1c - = u32 (1083408238, 0, 0) - [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 - = b'CRW_0333.CRW' - [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 - = b'CRW_0333.THM' - [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 - [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 - heap root/ImageProps/MeasuredInfo: block @0x8a19cc len 0x10, offsetTbl @+0x0, 1 records - [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x41100000 off 0xf5e6df2f - [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x58 off 0x178 - heap root/ImageProps/ImageDescription: block @0x8a19dc len 0x58, offsetTbl @+0x3e, 2 records - [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 - = b'EOS 300D DIGITAL CMOS RAW' - [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x1e off 0x20 - = b'CRW:EOS 300D DIGITAL CMOS RAW' - [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0x8a off 0x1d0 - heap root/ImageProps/CameraObject: block @0x8a1a34 len 0x8a, offsetTbl @+0x66, 3 records - [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x1 off 0x0 - = b'' - [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x1d off 0x2 - = b'Canon\x00Canon EOS 300D DIGITAL' - [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x46 off 0x20 - heap root/ImageProps/CameraObject/CameraSpecification: block @0x8a1a54 len 0x46, offsetTbl @+0x18, 4 records - [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x3d6b01d9 off 0x8afd725d - [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xe53a2db9 - [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0x6efb17d7 - [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x17 off 0x0 - = b'Firmware Version 1.1.1' - [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x14f2 off 0x25a - heap root/ImageProps/ExifInformation: block @0x8a1abe len 0x14f2, offsetTbl @+0x1406, 23 records - [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 - [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x460000 off 0x2620392 - [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x42 off 0x0 - [3] tag 0x1093 id 0x1093 ? word heap len 0x12 off 0x42 - = u16 (18, 0, 0, 0, 0, 0, 65535, 65535, 65535) - [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x54 - [5] tag 0x0036 id 0x0036 ? byte heap len 0xe82 off 0xb0 - [6] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x80000170 off 0x79d599bf - [7] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xf32 - = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) - [8] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xf54 - = u32 (0, 0, 514, 5300052) - [9] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x30 off 0xf64 - = u16 (48, 7, 3072, 2048, 3072, 2048, 151, 151, 1014, 608, 0, 0, 0, 64928, 64522, 0, 0, 65030, 0, 506, 0, 0, 8, 65535) - [10] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xf94 - = u16 (26, 334, 372, 372, 175, 246, 427, 427, 279, 185, 488, 490, 443) - [11] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xfae - = u16 (26, 297, 375, 375, 168, 203, 394, 395, 244, 153, 453, 454, 388) - [12] tag 0x10c2 id 0x10c2 ? word heap len 0x374 off 0xfc8 - [13] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x133c - = u16 (10, 489, 1022, 1026, 668) - [14] tag 0x10a8 id 0x10a8 ? word heap len 0x14 off 0x1346 - = u16 (20, 4709, 5200, 6983, 5987, 3213, 3888, 6196, 2500, 5190) - [15] tag 0x10ad id 0x10ad ? word heap len 0x3e off 0x135a - = u16 (62, 20127, 8480, 7000, 6000, 5600, 5200, 4724, 4200, 3266, 1446, 64813, 65123, 65187, 65248, 65281, 65320, 65371, 65446, 114, 674, 65436, 65225, 65196, 65169, 65152, 65137, 65124, 65112, 65098, 65348) - [16] tag 0x10ae id 0x10ae ColorTemperature word heap len 0x2 off 0x1398 - = u16 (5200,) - [17] tag 0x10af id 0x10af ? word heap len 0x2 off 0x139a - = u16 (1024,) - [18] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x52 off 0x139c - [19] tag 0x10b4 id 0x10b4 ColorSpace word heap len 0x2 off 0x13ee - = u16 (1,) - [20] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x13f0 - = u16 (10, 3, 1, 2048, 1360) - [21] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x13fa - = u16 (0, 159, 7, 112) - [22] tag 0x183b id 0x183b SerialNumberFormat dword heap len 0x4 off 0x1402 - = u32 (2415919104,) - -== corpus/crw/crw-canon-eos-300d-b.crw - size 8389842, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 -heap root: block @0x1a len 0x8004b8, offsetTbl @+0x800494, 3 records - [0] tag 0x2005 id 0x2005 RawData struct heap len 0x6df56a off 0x0 - RawData: 7206250 bytes, entropy 7.9810 bits/byte - first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 - middle 2 KiB: entropy 7.903, top bytes d7:20, 7b:18, 32:17, 83:16, zeros 14, ff 6 - last 2 KiB: entropy 7.840, top bytes 00:29, ff:23, fc:21, af:20, zeros 29, ff 23 - [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x11f72c off 0x6df56a - starts with JPEG SOI - [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x17fe off 0x7fec96 - heap root/ImageProps: block @0x7fecb0 len 0x17fe, offsetTbl @+0x1776, 13 records - [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 - [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc - = u32 (3072, 2048, 1065353216, 270, 8, 24, 257) - [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x17890000 off 0x637f057b - [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0x9cae3253 - [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x2b4f7b off 0x7aaebd64 - [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 - = u32 (1182188238, 0, 0) - [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 - = b'CRW_8395.CRW' - [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 - = b'CRW_8395.THM' - [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 - [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 - heap root/ImageProps/MeasuredInfo: block @0x7fee18 len 0x10, offsetTbl @+0x0, 1 records - [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x41040000 off 0xde864d39 - [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 - heap root/ImageProps/ImageDescription: block @0x7fee28 len 0x5a, offsetTbl @+0x40, 2 records - [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 - = b'EOS 300D DIGITAL CMOS RAW' - [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 - = b'CRW:EOS 300D DIGITAL CMOS RAW' - [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 - heap root/ImageProps/CameraObject: block @0x7fee82 len 0xb2, offsetTbl @+0x8e, 3 records - [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 - = b'' - [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 - = b'Canon\x00Canon EOS 300D DIGITAL\x00\x97\xd4\xc4' - [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 - heap root/ImageProps/CameraObject/CameraSpecification: block @0x7feec2 len 0x4e, offsetTbl @+0x20, 4 records - [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x3d6aecf7 off 0x46ec30fc - [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0x5aa35384 - [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0xf1421364 - [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 - = b'Firmware Version 1.1.1' - [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x14f2 off 0x284 - heap root/ImageProps/ExifInformation: block @0x7fef34 len 0x14f2, offsetTbl @+0x1406, 23 records - [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 - [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x150000 off 0x2620392 - [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x42 off 0x0 - [3] tag 0x1093 id 0x1093 ? word heap len 0x12 off 0x42 - = u16 (18, 0, 0, 0, 0, 0, 65535, 65535, 65535) - [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x54 - [5] tag 0x0036 id 0x0036 ? byte heap len 0xe82 off 0xb0 - [6] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x80000170 off 0xedbad5b1 - [7] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xf32 - = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) - [8] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xf54 - = u32 (2, 0, 514, 5576664) - [9] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x30 off 0xf64 - = u16 (7, 7, 3072, 2048, 3072, 2048, 151, 151, 1014, 608, 0, 0, 0, 64928, 64522, 0, 0, 65030, 0, 506, 0, 0, 36, 65535) - [10] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xf94 - = u16 (26, 328, 374, 372, 163, 244, 430, 428, 262, 162, 436, 435, 366) - [11] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xfae - = u16 (26, 301, 375, 375, 166, 206, 394, 395, 242, 155, 453, 454, 383) - [12] tag 0x10c2 id 0x10c2 ? word heap len 0x374 off 0xfc8 - [13] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x133c - = u16 (10, 314, 1022, 1026, 1095) - [14] tag 0x10a8 id 0x10a8 ? word heap len 0x14 off 0x1346 - = u16 (20, 5781, 5200, 7000, 5975, 3211, 3882, 6203, 4166, 5190) - [15] tag 0x10ad id 0x10ad ? word heap len 0x3e off 0x135a - = u16 (62, 20127, 8480, 7000, 6000, 5600, 5200, 4724, 4200, 3266, 1446, 64849, 65150, 65212, 65271, 65304, 65342, 65392, 65464, 125, 664, 65367, 65170, 65143, 65117, 65102, 65088, 65078, 65068, 65055, 65302) - [16] tag 0x10ae id 0x10ae ColorTemperature word heap len 0x2 off 0x13ea - = u16 (5200,) - [17] tag 0x10af id 0x10af ? word heap len 0x2 off 0x13ec - = u16 (1024,) - [18] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x52 off 0x1398 - [19] tag 0x10b4 id 0x10b4 ColorSpace word heap len 0x2 off 0x13ee - = u16 (2,) - [20] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x13f0 - = u16 (10, 3, 2, 1536, 1024) - [21] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x13fa - = u16 (0, 159, 7, 112) - [22] tag 0x183b id 0x183b SerialNumberFormat dword heap len 0x4 off 0x1402 - = u32 (2415919104,) - -== corpus/crw/crw-canon-eos-d30.crw - size 2993260, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 -heap root: block @0x1a len 0x2dac52, offsetTbl @+0x2dac2e, 3 records - [0] tag 0x2005 id 0x2005 RawData struct heap len 0x2b19c8 off 0x0 - RawData: 2824648 bytes, entropy 7.9671 bits/byte - first 2 KiB: entropy 7.540, top bytes ef:35, fb:34, ae:33, bf:31, zeros 1, ff 24 - middle 2 KiB: entropy 7.835, top bytes ac:21, 95:19, b6:18, ab:18, zeros 11, ff 3 - last 2 KiB: entropy 7.792, top bytes 02:22, 4c:21, 00:20, 26:19, zeros 20, ff 2 - [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x28128 off 0x2b19c8 - starts with JPEG SOI - [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x113e off 0x2d9af0 - heap root/ImageProps: block @0x2d9b0a len 0x113e, offsetTbl @+0x10b6, 13 records - [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 - [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc - = u32 (2160, 1440, 1065353216, 0, 8, 24, 257) - [2] tag 0x500a id 0x100a TargetImageType word inrec len 0xc0200000 off 0x130000 - [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0x2108c00 - [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x22378c off 0x7400000 - [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 - = u32 (1482512730, 0, 0) - [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 - = b'CRW_2444.CRW\x00\x9a\x02]G\x18\xa6JG\xff\x80 H\x00@\x00\x02\x82' - [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 - = b'CRW_2444.THM\x00\x04\x04\x010\x02\x00\x80\x00\x00\x00@\x12\x14\x00\x00\x00 ' - [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 - [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 - heap root/ImageProps/MeasuredInfo: block @0x2d9c72 len 0x10, offsetTbl @+0x0, 1 records - [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0xbec00000 off 0xedfff6ff - [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 - heap root/ImageProps/ImageDescription: block @0x2d9c82 len 0x5a, offsetTbl @+0x40, 2 records - [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 - = b'EOS D30 CMOS RAW IMAGE\x00\xff\x9f\xff\xfd\xf7\xfd\xfd\xff\xff' - [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 - = b'CRW:EOS D30 CMOS RAW IMAGE\x00\xdf\xff\xde\xff\xff' - [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 - heap root/ImageProps/CameraObject: block @0x2d9cdc len 0xb2, offsetTbl @+0x8e, 3 records - [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 - = b'\x00\xdf\xfb\xff\xff\xff\xb5\xef\xff\xff_\xef\xff\xdf\xef\xfb\xff\xff\xff\xfb\xff\xff{\xff\xfe\xff\xff\x7f\xfd?\xdf\xff' - [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 - = b'Canon\x00Canon EOS D30\x00\x00\xff\xff\xff\xff\xff\x7f\xbf\xfd\xff\xffo' - [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 - heap root/ImageProps/CameraObject/CameraSpecification: block @0x2d9d1c len 0x4e, offsetTbl @+0x20, 4 records - [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x172700aa off 0xfffaffff - [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xfffff1ff - [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0xffffff5f - [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 - = b'Firmware Version 1.02\x00\xbf\xff\xff\xff\xe7\xff\xff\xdf\xff}' - [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0xe32 off 0x284 - heap root/ImageProps/ExifInformation: block @0x2d9d8e len 0xe32, offsetTbl @+0xdaa, 13 records - [0] tag 0x0001 id 0x0001 FreeBytes byte heap len 0xc off 0x0 - [1] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x400048 - [2] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x320000 off 0x2620392 - [3] tag 0x102a id 0x102a CanonShotInfo word heap len 0x32 off 0xc - = u16 (50, 0, 288, 65484, 31, 181, 0, 0, 3, 1, 8, 8, 0, 0, 12290, 0, 0, 0, 1, 0, 0, 32, 180, 75, 0) - [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x40 off 0x3e - = u16 (64, 2, 0, 4, 0, 1, 0, 0, 0, 2, 0, 1, 0, 0, 0, 0, 0, 5, 2, 12291, 4, 0, 198, 50, 50, 1, 31, 285, 255, 0, 0, 0) - [5] tag 0x102c id 0x102c CanonColorInfo2 word heap len 0x100 off 0x7e - [6] tag 0x0032 id 0x0032 ? byte heap len 0x300 off 0x17e - [7] tag 0x0036 id 0x0036 ? byte heap len 0x868 off 0x47e - [8] tag 0x1030 id 0x1030 WhiteSample word heap len 0x76 off 0xce6 - [9] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x1140000 off 0x400c - [10] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xd5c - = u16 (34, 2224, 1456, 1, 0, 56, 12, 2215, 1451, 8, 220, 40, 1244, 0, 0, 0, 0) - [11] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xd7e - = u32 (1, 0, 514, 2015110) - [12] tag 0x1033 id 0x1033 CustomFunctions word heap len 0x1c off 0xd8e - = u16 (28, 256, 514, 768, 1025, 1280, 1536, 1794, 2049, 2305, 2560, 2816, 3074, 3328) - -== corpus/crw/crw-canon-eos-d60.crw - size 8591234, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 -heap root: block @0x1a len 0x831768, offsetTbl @+0x831744, 3 records - [0] tag 0x2005 id 0x2005 RawData struct heap len 0x662502 off 0x0 - RawData: 6694146 bytes, entropy 7.9868 bits/byte - first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 - middle 2 KiB: entropy 7.864, top bytes 8c:25, cb:21, 5d:18, 63:17, zeros 11, ff 4 - last 2 KiB: entropy 7.890, top bytes f1:16, 6d:16, 76:16, 55:15, zeros 11, ff 5 - [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x1cdc00 off 0x662502 - starts with JPEG SOI - [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x1642 off 0x830102 - heap root/ImageProps: block @0x83011c len 0x1642, offsetTbl @+0x15ba, 13 records - [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 - [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc - = u32 (3072, 2048, 1065353216, 0, 8, 24, 257) - [2] tag 0x500a id 0x100a TargetImageType word inrec len 0x6d8f0000 off 0xcfba0d73 - [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xd8327aab - [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0xf42a3 off 0x9792ef1a - [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 - = u32 (1467566074, 0, 0) - [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 - = b'CRW_0099.CRW' - [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 - = b'CRW_0099.THM' - [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 - [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 - heap root/ImageProps/MeasuredInfo: block @0x830284 len 0x10, offsetTbl @+0x0, 1 records - [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x3fc00000 off 0xe2577c1e - [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 - heap root/ImageProps/ImageDescription: block @0x830294 len 0x5a, offsetTbl @+0x40, 2 records - [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 - = b'EOS D60 CMOS RAW IMAGE' - [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 - = b'CRW:EOS D60 CMOS RAW IMAGE' - [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 - heap root/ImageProps/CameraObject: block @0x8302ee len 0xb2, offsetTbl @+0x8e, 3 records - [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 - = b'' - [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 - = b'Canon\x00Canon EOS D60\x00\x0e{\xd7\xd8e\x18iG\x03,,c' - [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 - heap root/ImageProps/CameraObject/CameraSpecification: block @0x83032e len 0x4e, offsetTbl @+0x20, 4 records - [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x3645276f off 0x5982da24 - [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xa552febb - [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0xfedcfe5 - [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 - = b'Firmware Version 1.0.4' - [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x1336 off 0x284 - heap root/ImageProps/ExifInformation: block @0x8303a0 len 0x1336, offsetTbl @+0x129a, 15 records - [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 - [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x0 off 0x2620392 - [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x36 off 0x0 - = u16 (54, 0, 160, 52, 32768, 245, 65504, 0, 3, 0, 8, 8, 0, 0, 12288, 0, 0, 0, 1, 0, 0, 32768, 244, 99, 0, 0, 252) - [3] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x56 off 0x36 - [4] tag 0x0036 id 0x0036 ? byte heap len 0xbd6 off 0x8c - [5] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x1668000 off 0xed8d4a16 - [6] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xc62 - = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) - [7] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xc84 - = u32 (0, 0, 514, 5064559) - [8] tag 0x1033 id 0x1033 CustomFunctions word heap len 0x22 off 0xc94 - = u16 (32, 0, 515, 768, 1025, 1280, 1536, 1792, 2048, 2304, 2560, 2818, 3074, 3328, 3584, 3841, 18747) - [9] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xcb6 - = u16 (26, 279, 368, 371, 172, 219, 434, 436, 284, 162, 491, 494, 431) - [10] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xcd0 - = u16 (26, 281, 413, 415, 193, 223, 488, 490, 319, 150, 505, 507, 439) - [11] tag 0x10c2 id 0x10c2 ? word heap len 0x554 off 0xcea - [12] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x123e - = u16 (10, 440, 1021, 1027, 563) - [13] tag 0x10a8 id 0x10a8 ? word heap len 0x10 off 0x1248 - = u16 (16, 6000, 5312, 5989, 3161, 3717, 6503, 4118) - [14] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x42 off 0x1258 - -== corpus/crw/crw-canon-eos-digital-rebel.crw - size 10153786, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 -heap root: block @0x1a len 0x9aef20, offsetTbl @+0x9aeefc, 3 records - [0] tag 0x2005 id 0x2005 RawData struct heap len 0x61c014 off 0x0 - RawData: 6406164 bytes, entropy 7.8363 bits/byte - first 2 KiB: entropy -0.000, top bytes 00:2048, zeros 2048, ff 0 - middle 2 KiB: entropy 7.825, top bytes 18:24, 59:18, cd:17, 06:17, zeros 10, ff 6 - last 2 KiB: entropy 7.836, top bytes b5:19, 5a:17, 2c:17, 56:17, zeros 10, ff 2 - [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x3916ea off 0x61c014 - starts with JPEG SOI - [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x17fe off 0x9ad6fe - heap root/ImageProps: block @0x9ad718 len 0x17fe, offsetTbl @+0x1776, 13 records - [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 - [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc - = u32 (3072, 2048, 1065353216, 0, 8, 24, 257) - [2] tag 0x500a id 0x100a TargetImageType word inrec len 0xdfff0000 off 0x7ff757d7 - [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0x7d5d557f - [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x221046 off 0x7df757dd - [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 - = u32 (1480372394, 0, 0) - [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 - = b'CRW_2390.CRW' - [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 - = b'CRW_2390.THM' - [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 - [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 - heap root/ImageProps/MeasuredInfo: block @0x9ad880 len 0x10, offsetTbl @+0x0, 1 records - [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0xbec00000 off 0xd7dd55f5 - [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 - heap root/ImageProps/ImageDescription: block @0x9ad890 len 0x5a, offsetTbl @+0x40, 2 records - [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 - = b'EOS DIGITAL REBEL CMOS RAW' - [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 - = b'CRW:EOS DIGITAL REBEL CMOS RAW' - [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xb2 off 0x1d2 - heap root/ImageProps/CameraObject: block @0x9ad8ea len 0xb2, offsetTbl @+0x8e, 3 records - [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 - = b'' - [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 - = b'Canon\x00Canon EOS DIGITAL REBEL\x00w\x7f' - [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x4e off 0x40 - heap root/ImageProps/CameraObject/CameraSpecification: block @0x9ad92a len 0x4e, offsetTbl @+0x20, 4 records - [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x5116c2ff off 0xf55d5ff7 - [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0x5ff555f5 - [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0x5ddd77ff - [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 - = b'Firmware Version 1.1.1' - [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x14f2 off 0x284 - heap root/ImageProps/ExifInformation: block @0x9ad99c len 0x14f2, offsetTbl @+0x1406, 23 records - [0] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x64 off 0x0 - [1] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x320000 off 0x2620392 - [2] tag 0x102a id 0x102a CanonShotInfo word heap len 0x42 off 0x0 - [3] tag 0x1093 id 0x1093 ? word heap len 0x12 off 0x42 - = u16 (18, 0, 0, 0, 0, 0, 65535, 65535, 65535) - [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x54 - [5] tag 0x0036 id 0x0036 ? byte heap len 0xe82 off 0xb0 - [6] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x80000170 off 0xdf777555 - [7] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0xf32 - = u16 (34, 3152, 2068, 1, 1, 72, 16, 3143, 2063, 24, 224, 40, 1856, 0, 0, 0, 0) - [8] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0xf54 - = u32 (0, 0, 514, 4776577) - [9] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x30 off 0xf64 - = u16 (7, 7, 3072, 2048, 3072, 2048, 151, 151, 1014, 608, 0, 0, 0, 64928, 64522, 0, 0, 65030, 0, 506, 0, 0, 8, 65535) - [10] tag 0x10c0 id 0x10c0 ? word heap len 0x1a off 0xf94 - = u16 (26, 327, 378, 369, 173, 258, 466, 457, 290, 186, 508, 497, 420) - [11] tag 0x10c1 id 0x10c1 ? word heap len 0x1a off 0xfae - = u16 (26, 304, 375, 375, 168, 206, 394, 395, 235, 158, 453, 454, 363) - [12] tag 0x10c2 id 0x10c2 ? word heap len 0x374 off 0xfc8 - [13] tag 0x10aa id 0x10aa ? word heap len 0xa off 0x133c - = u16 (10, 743, 1022, 1026, 483) - [14] tag 0x10a8 id 0x10a8 ? word heap len 0x14 off 0x1346 - = u16 (20, 2993, 5200, 6982, 5987, 3210, 3894, 6220, 2675, 5190) - [15] tag 0x10ad id 0x10ad ? word heap len 0x3e off 0x135a - = u16 (62, 20127, 8480, 7000, 6000, 5600, 5200, 4724, 4200, 3266, 1446, 64839, 65135, 65196, 65255, 65286, 65324, 65373, 65444, 110, 671, 65391, 65193, 65166, 65140, 65124, 65110, 65099, 65088, 65070, 65282) - [16] tag 0x10ae id 0x10ae ColorTemperature word heap len 0x2 off 0x13ea - = u16 (5200,) - [17] tag 0x10af id 0x10af ? word heap len 0x2 off 0x13ec - = u16 (1268,) - [18] tag 0x10a9 id 0x10a9 ColorBalance word heap len 0x52 off 0x1398 - [19] tag 0x10b4 id 0x10b4 ColorSpace word heap len 0x2 off 0x13ee - = u16 (1,) - [20] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x13f0 - = u16 (10, 3, 0, 3072, 2048) - [21] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x13fa - = u16 (0, 159, 7, 112) - [22] tag 0x183b id 0x183b SerialNumberFormat dword heap len 0x4 off 0x1402 - = u32 (2415919104,) - -== corpus/crw/crw-canon-powershot-g1.crw - size 2300702, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 -heap root: block @0x1a len 0x231b04, offsetTbl @+0x231ae0, 3 records - [0] tag 0x2005 id 0x2005 RawData struct heap len 0x2271ca off 0x0 - RawData: 2257354 bytes, entropy 7.9338 bits/byte - first 2 KiB: entropy 6.516, top bytes 00:522, 22:26, 24:21, 64:18, zeros 522, ff 4 - middle 2 KiB: entropy 7.827, top bytes 5a:23, 6d:23, 99:17, aa:17, zeros 8, ff 1 - last 2 KiB: entropy 7.669, top bytes 44:31, 4c:30, 88:26, 20:24, zeros 16, ff 9 - [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x9394 off 0x2271ca - starts with JPEG SOI - [2] tag 0x300a id 0x300a ImageProps heap2 heap len 0x1582 off 0x23055e - heap root/ImageProps: block @0x230578 len 0x1582, offsetTbl @+0x14fa, 13 records - [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 - [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0x0 - = u32 (2048, 1536, 1065353216, 0, 8, 24, 257) - [2] tag 0x500a id 0x100a TargetImageType word inrec len 0xc5600000 off 0xbee19e93 - [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0x51bd76d5 - [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x11b9dd off 0x0 - [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x1c - = u32 (1065594172, 0, 0) - [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 - = b'CRW_1693.CRW\x00\x8e&\xe1>4\xd2\xd2\xa1ko\xddv?\xa5\x0f\xa8C\x0eY' - [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 - = b'CRW_1693.THM\x00~,\x07\xef\x19\xef\x88\xe3\xef2t\x89\n\x82\x88\x84\xe97\xe8' - [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 - [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 - heap root/ImageProps/MeasuredInfo: block @0x2306e0 len 0x10, offsetTbl @+0x0, 1 records - [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x0 off 0xab71c23a - [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x58 off 0x178 - heap root/ImageProps/ImageDescription: block @0x2306f0 len 0x58, offsetTbl @+0x3e, 2 records - [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 - = b'High definition CCD image\x00\xfd\xa31\x18u\x0b' - [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x1e off 0x20 - = b'CRW:High definition CCD image' - [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0x8e off 0x1d0 - heap root/ImageProps/CameraObject: block @0x230748 len 0x8e, offsetTbl @+0x6a, 3 records - [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0xc off 0x0 - = b'Kelly Sauke' - [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x19 off 0xc - = b'Canon\x00Canon PowerShot G1' - [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x44 off 0x26 - heap root/ImageProps/CameraObject/CameraSpecification: block @0x23076e len 0x44, offsetTbl @+0x16, 4 records - [0] tag 0x580b id 0x180b SerialNumber dword inrec len 0x0 off 0x6eb7053 - [1] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xf13edcd9 - [2] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0xa9d94930 - [3] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x16 off 0x0 - = b'Firmware Version 1.02' - [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x129c off 0x25e - heap root/ImageProps/ExifInformation: block @0x2307d6 len 0x129c, offsetTbl @+0x121e, 12 records - [0] tag 0x0001 id 0x0001 FreeBytes byte heap len 0xc off 0x0 - [1] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x0 off 0x0 - [2] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x1d30002 off 0xbd00fc - [3] tag 0x102a id 0x102a CanonShotInfo word heap len 0x34 off 0xc - = u16 (52, 0, 128, 115, 73, 170, 0, 0, 0, 0, 24, 0, 0, 0, 0, 0, 0, 0, 1, 494, 0, 79, 164, 0, 0, 0) - [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x46 off 0x40 - [5] tag 0x102c id 0x102c CanonColorInfo2 word heap len 0x100 off 0x86 - [6] tag 0x0032 id 0x0032 ? byte heap len 0x800 off 0x186 - [7] tag 0x0036 id 0x0036 ? byte heap len 0x800 off 0x986 - [8] tag 0x1030 id 0x1030 WhiteSample word heap len 0x66 off 0x1186 - [9] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x4040000 off 0x2255b19d - [10] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0x11ec - = u16 (34, 2144, 1560, 1, 1, 24, 15, 2071, 1550, 2108, 15, 2111, 1550, 0, 0, 0, 0) - [11] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0x120e - = u32 (1, 0, 514, 2257351) - -== corpus/crw/crw-canon-powershot-g5.crw - size 4894440, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 -heap root: block @0x1a len 0x4aaece, offsetTbl @+0x4aaea0, 4 records - [0] tag 0x2005 id 0x2005 RawData struct heap len 0x4a2f6e off 0x0 - RawData: 4861806 bytes, entropy 7.9622 bits/byte - first 2 KiB: entropy 6.656, top bytes 00:529, bb:14, b8:13, 73:13, zeros 529, ff 4 - middle 2 KiB: entropy 7.819, top bytes b2:24, 55:22, 75:21, 57:21, zeros 8, ff 1 - last 2 KiB: entropy 7.680, top bytes 89:34, 44:34, 4c:28, 88:28, zeros 20, ff 6 - [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0x530e off 0x4a2f6e - starts with JPEG SOI - [2] tag 0x2008 id 0x2008 ThumbnailImage struct heap len 0x9fe off 0x4a827c - starts with JPEG SOI - [3] tag 0x300a id 0x300a ImageProps heap2 heap len 0x2226 off 0x4a8c7a - heap root/ImageProps: block @0x4a8c94 len 0x2226, offsetTbl @+0x219e, 13 records - [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 - [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc - = u32 (2592, 1944, 1065353216, 0, 8, 24, 257) - [2] tag 0x500a id 0x100a TargetImageType word inrec len 0xc9e90000 off 0x1fba0082 - [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xfc48c47b - [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x10078a off 0xa0fcb31 - [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 - = u32 (1059363106, 0, 0) - [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 - = b'CRW_0506.CRW\x001Y)\x97\xab\xf9n8\xba)\xd7\x9d\xd9\xc2\x13\x8a 9\x01' - [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 - = b'CRW_0506.THM\x00\x05\xc3!\xe5\xbc\xb1<\xf83\xde\xe1E\x93\xb1e5\xab\x81\x92' - [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 - [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 - heap root/ImageProps/MeasuredInfo: block @0x4a8dfc len 0x10, offsetTbl @+0x0, 1 records - [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x0 off 0xff9410 - [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 - heap root/ImageProps/ImageDescription: block @0x4a8e0c len 0x5a, offsetTbl @+0x40, 2 records - [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 - = b'High definition CCD image\x00\xad\xc1<\xb1\x80\xc0' - [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 - = b'CRW:High definition CCD image\x00LF' - [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xa8 off 0x1d2 - heap root/ImageProps/CameraObject: block @0x4a8e66 len 0xa8, offsetTbl @+0x84, 3 records - [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 - = b'\x008\x9e!,v\xf7\xc7\x18\xc4c"K\xfe\xe8;\xf4=\x9f\x1e\'9\xe3\x19nq%\xa1\x92\xb3\x853' - [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 - = b'Canon\x00Canon PowerShot G5\x009\x1c\xed\x8f6\xac\xa2' - [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x44 off 0x40 - heap root/ImageProps/CameraObject/CameraSpecification: block @0x4a8ea6 len 0x44, offsetTbl @+0x20, 3 records - [0] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0xad6e2c6d - [1] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0x5cc19942 - [2] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 - = b'Firmware Version 1.00\x00\x00\x98\xaa\xebP\xf9rhu\xd0' - [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x1f24 off 0x27a - heap root/ImageProps/ExifInformation: block @0x4a8f0e len 0x1f24, offsetTbl @+0x1e7e, 16 records - [0] tag 0x0001 id 0x0001 FreeBytes byte heap len 0xc off 0x0 - [1] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x400 off 0x0 - [2] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0xe60002 off 0xd7011e - [3] tag 0x102a id 0x102a CanonShotInfo word heap len 0x44 off 0xc - [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x50 - [5] tag 0x0032 id 0x0032 ? byte heap len 0x800 off 0xac - [6] tag 0x0036 id 0x0036 ? byte heap len 0x1400 off 0x8ac - [7] tag 0x1030 id 0x1030 WhiteSample word heap len 0x76 off 0x1cac - [8] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x1290000 off 0x725bca88 - [9] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0x1d22 - = u16 (34, 2672, 1968, 1, 1, 24, 14, 2615, 1957, 2640, 14, 2643, 1957, 0, 0, 0, 0) - [10] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0x1d44 - = u32 (1, 0, 514, 4861804) - [11] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x38 off 0x1d54 - = u16 (9, 1, 2592, 1944, 2592, 215, 467, 39, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0) - [12] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x1d8c - = u16 (0, 0, 0, 0) - [13] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x1d94 - = u16 (10, 2, 2, 640, 480) - [14] tag 0x103c id 0x103c ? word heap len 0x9c off 0x1d9e - [15] tag 0x107f id 0x107f ? word heap len 0x44 off 0x1e3a - -== corpus/crw/crw-canon-powershot-g6.crw - size 8052032, byteOrder II, headerLength 26, type b'HEAP' subType b'CCDR', version 0x00010002 -heap root: block @0x1a len 0x7add26, offsetTbl @+0x7adcf8, 4 records - [0] tag 0x2005 id 0x2005 RawData struct heap len 0x6f9704 off 0x0 - RawData: 7313156 bytes, entropy 7.9774 bits/byte - first 2 KiB: entropy 6.684, top bytes 00:520, c1:14, 48:14, ae:13, zeros 520, ff 8 - middle 2 KiB: entropy 7.885, top bytes 12:18, 31:17, 93:16, 64:15, zeros 13, ff 5 - last 2 KiB: entropy 7.355, top bytes f7:42, 63:36, ef:35, 7e:35, zeros 14, ff 15 - [1] tag 0x2007 id 0x2007 JpgFromRaw struct heap len 0xae5f4 off 0x6f9704 - starts with JPEG SOI - [2] tag 0x2008 id 0x2008 ThumbnailImage struct heap len 0x17b8 off 0x7a7cf8 - starts with JPEG SOI - [3] tag 0x300a id 0x300a ImageProps heap2 heap len 0x4848 off 0x7a94b0 - heap root/ImageProps: block @0x7a94ca len 0x4848, offsetTbl @+0x47c0, 13 records - [0] tag 0x5803 id 0x1803 ImageFormat dword inrec len 0x20001 off 0x41200000 - [1] tag 0x1810 id 0x1810 ImageInfo dword heap len 0x1c off 0xc - = u32 (3072, 2304, 1065353216, 0, 8, 24, 257) - [2] tag 0x500a id 0x100a TargetImageType word inrec len 0xffff0000 off 0xffffffff - [3] tag 0x5804 id 0x1804 RecordID dword inrec len 0x0 off 0xffffffff - [4] tag 0x5817 id 0x1817 FileNumber dword inrec len 0x1257af off 0xffffffff - [5] tag 0x180e id 0x180e TimeStamp dword heap len 0xc off 0x0 - = u32 (1607955982, 0, 0) - [6] tag 0x0816 id 0x0816 OriginalFileName ascii heap len 0x20 off 0x28 - = b'CRW_2095.CRW\x00\n\\U\x88JJ\x04\x7f\xff' - [7] tag 0x0817 id 0x0817 ThumbnailFileName ascii heap len 0x20 off 0x48 - = b'CRW_2095.THM\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 ' - [8] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x100 off 0x68 - [9] tag 0x3003 id 0x3003 MeasuredInfo heap2 heap len 0x10 off 0x168 - heap root/ImageProps/MeasuredInfo: block @0x7a9632 len 0x10, offsetTbl @+0x0, 1 records - [0] tag 0x5814 id 0x1814 MeasuredEV dword inrec len 0x0 off 0x0 - [10] tag 0x2804 id 0x2804 ImageDescription heap1 heap len 0x5a off 0x178 - heap root/ImageProps/ImageDescription: block @0x7a9642 len 0x5a, offsetTbl @+0x40, 2 records - [0] tag 0x0805 id 0x0805 CanonFileDescription ascii heap len 0x20 off 0x0 - = b'High definition CCD image' - [1] tag 0x0815 id 0x0815 CanonImageType ascii heap len 0x20 off 0x20 - = b'CRW:High definition CCD image' - [11] tag 0x2807 id 0x2807 CameraObject heap1 heap len 0xa8 off 0x1d2 - heap root/ImageProps/CameraObject: block @0x7a969c len 0xa8, offsetTbl @+0x84, 3 records - [0] tag 0x0810 id 0x0810 OwnerName ascii heap len 0x20 off 0x0 - = b'\x00\x00\x00\x00\x00\x00\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x08' - [1] tag 0x080a id 0x080a CanonRawMakeModel ascii heap len 0x20 off 0x20 - = b'Canon\x00Canon PowerShot G6' - [2] tag 0x3004 id 0x3004 CameraSpecification heap2 heap len 0x44 off 0x40 - heap root/ImageProps/CameraObject/CameraSpecification: block @0x7a96dc len 0x44, offsetTbl @+0x20, 3 records - [0] tag 0x501c id 0x101c BaseISO word inrec len 0x64 off 0x0 - [1] tag 0x480d id 0x080d ? ascii inrec len 0x415355 off 0x0 - [2] tag 0x080b id 0x080b CanonFirmwareVersion ascii heap len 0x20 off 0x0 - = b'Firmware Version 1.00' - [12] tag 0x300b id 0x300b ExifInformation heap2 heap len 0x4546 off 0x27a - heap root/ImageProps/ExifInformation: block @0x7a9744 len 0x4546, offsetTbl @+0x4482, 19 records - [0] tag 0x0001 id 0x0001 FreeBytes byte heap len 0xc off 0x0 - [1] tag 0x5028 id 0x1028 CanonFlashInfo word inrec len 0x0 off 0x0 - [2] tag 0x5029 id 0x1029 CanonFocalLength word inrec len 0x2960002 off 0xd7011e - [3] tag 0x102a id 0x102a CanonShotInfo word heap len 0x44 off 0xc - [4] tag 0x102d id 0x102d CanonCameraSettings word heap len 0x5c off 0x50 - [5] tag 0x0032 id 0x0032 ? byte heap len 0xc00 off 0xac - [6] tag 0x0041 id 0x0041 ? byte heap len 0x100 off 0xcac - [7] tag 0x0036 id 0x0036 ? byte heap len 0x2000 off 0xdac - [8] tag 0x1030 id 0x1030 WhiteSample word heap len 0x76 off 0x42ac - [9] tag 0x5834 id 0x1834 CanonModelID dword inrec len 0x1400000 off 0xffffffff - [10] tag 0x1031 id 0x1031 SensorInfo word heap len 0x22 off 0x4322 - = u16 (34, 3160, 2344, 1, 1, 64, 24, 3135, 2327, 36, 24, 39, 2327, 0, 0, 0, 0) - [11] tag 0x1835 id 0x1835 DecoderTable dword heap len 0x10 off 0x4344 - = u32 (0, 0, 514, 7313154) - [12] tag 0x1038 id 0x1038 CanonAFInfo word heap len 0x38 off 0x4354 - = u16 (9, 1, 3072, 2304, 3072, 230, 553, 41, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0) - [13] tag 0x1039 id 0x1039 ? word heap len 0x8 off 0x438c - = u16 (0, 0, 0, 0) - [14] tag 0x10b5 id 0x10b5 RawJpgInfo word heap len 0xa off 0x4394 - = u16 (10, 2, 0, 3072, 2304) - [15] tag 0x0040 id 0x0040 ? byte heap len 0x100 off 0x2dac - [16] tag 0x003f id 0x003f ? byte heap len 0x1400 off 0x2eac - [17] tag 0x103c id 0x103c ? word heap len 0x9c off 0x439e - [18] tag 0x107f id 0x107f ? word heap len 0x48 off 0x443a - diff --git a/crates/crw/clean-room/logs/verification-sweep.txt b/crates/crw/clean-room/logs/verification-sweep.txt deleted file mode 100644 index 18992d2d..00000000 --- a/crates/crw/clean-room/logs/verification-sweep.txt +++ /dev/null @@ -1,22 +0,0 @@ -CRW_0476.crw: sel 1, 3152x2068: EXACT -CRW_0478.crw: sel 1, 3152x2068: EXACT -CRW_0486.crw: sel 0, 3152x2068: EXACT -CRW_0527.crw: sel 1, 3152x2068: EXACT -CRW_0683.crw: sel 1, 3152x2068: EXACT -CRW_0717.crw: sel 1, 3152x2068: EXACT -CRW_0765.crw: sel 1, 3152x2068: EXACT -CRW_0822.crw: sel 1, 3152x2068: EXACT -CRW_0879.crw: sel 1, 3152x2068: EXACT -CRW_1011.crw: sel 1, 3152x2068: EXACT -CRW_1153.crw: sel 0, 3152x2068: EXACT -crw-canon-eos-10d-a.crw: sel 0, 3152x2068: EXACT -crw-canon-eos-10d-b.crw: sel 2, 3152x2068: EXACT -crw-canon-eos-10d-c.crw: sel 2, 3152x2068: EXACT -crw-canon-eos-300d-a.crw: sel 0, 3152x2068: EXACT -crw-canon-eos-300d-b.crw: sel 2, 3152x2068: EXACT -crw-canon-eos-d30.crw: sel 1, 2224x1456: EXACT -crw-canon-eos-d60.crw: sel 0, 3152x2068: EXACT -crw-canon-eos-digital-rebel.crw: sel 0, 3152x2068: EXACT -crw-canon-powershot-g1.crw: sel 1, 2144x1560: EXACT -crw-canon-powershot-g5.crw: sel 1, 2672x1968: 2685618 mismatches, first at 0 (row 0, col 0) -crw-canon-powershot-g6.crw: sel 0, 3160x2344: EXACT diff --git a/crates/crw/clean-room/p1_payload.py b/crates/crw/clean-room/p1_payload.py deleted file mode 100644 index d8faf5ad..00000000 --- a/crates/crw/clean-room/p1_payload.py +++ /dev/null @@ -1,184 +0,0 @@ -"""P1 — RawData payload partition statistics (no bitstream decoding). - -Questions (plan 01-plan.md P1): - 1. Exact leading zero run of RawData per file. - 2. Does RawData split as [low-bits wh/4][compressed L][pad] (prefix hypothesis) or - [compressed L][low-bits wh/4][pad] (suffix hypothesis) on EOS bodies? Tested by - row-structure statistics of both candidate regions (2 bits/pixel, rawW/4 bytes/row): - masked sensor rows should show as structured/near-constant rows, the entropy-coded - stream should not align to 788-byte rows. - 3. The constant third DecoderTable u32 (514) and the 2-3 byte remainder. - 4. Byte-stuffing check in the compressed region (ff-digram statistics). - -Prints statistics only; no sample values of image content beyond aggregate counts. - -Usage: python -I p1_payload.py file.crw [more...] -""" - -import math -import struct -import sys -from collections import Counter - - -def read_block_records(buf, start, length, endian): - if length < 6: - return [] - (tbl,) = struct.unpack_from(endian + "I", buf, start + length - 4) - if tbl + 2 > length - 4: - return [] - (n,) = struct.unpack_from(endian + "H", buf, start + tbl) - recs = [] - pos = start + tbl + 2 - if tbl + 2 + n * 10 > length: - return [] - for _ in range(n): - tc, ln, off = struct.unpack_from(endian + "HII", buf, pos) - pos += 10 - recs.append((tc, ln, off)) - return recs - - -def find_records(buf, start, length, endian, depth=0): - """Yield (tag_id, abs_offset, length) for every in-heap record, recursively.""" - if depth > 8: - return - for tc, ln, off in read_block_records(buf, start, length, endian): - stg = (tc >> 14) & 3 - dt = (tc >> 11) & 7 - tag = tc & 0x3FFF - if stg != 0: - yield (tag, None, (ln, off)) # in-entry value - continue - if off + ln > length: - continue - if dt in (5, 6): - yield from find_records(buf, start + off, ln, endian, depth + 1) - else: - yield (tag, start + off, ln) - - -def entropy(data): - if not data: - return 0.0 - c = Counter(data) - n = len(data) - return -sum(v / n * math.log2(v / n) for v in c.values()) - - -def zero_run(data): - i = 0 - while i < len(data) and data[i] == 0: - i += 1 - return i - - -def row_structure(region, row_bytes, crop_top, label, out): - """Reshape region into rows of row_bytes; report zero/constant-row runs and - per-band mean of 2-bit values (both bit orders give the same mean-of-byte/4 scale, - so use byte means as the aggregate).""" - nrows = len(region) // row_bytes - if nrows == 0: - out.append(f" {label}: shorter than one row") - return - zrows = 0 - for r in range(nrows): - row = region[r * row_bytes : (r + 1) * row_bytes] - if any(row): - break - zrows += 1 - bands = [] - for name, lo, hi in (("rows 0..top", 0, min(crop_top, nrows)), - ("top..top+32", min(crop_top, nrows), min(crop_top + 32, nrows)), - ("middle 32", nrows // 2, min(nrows // 2 + 32, nrows)), - ("last 32", max(0, nrows - 32), nrows)): - seg = region[lo * row_bytes : hi * row_bytes] - if seg: - bands.append(f"{name}: mean {sum(seg) / len(seg):7.2f} ent {entropy(seg):5.3f}") - out.append(f" {label}: {nrows} rows of {row_bytes} B, leading all-zero rows: {zrows}") - for b in bands: - out.append(f" {b}") - - -def ff_digrams(data, out): - n = len(data) - c = Counter(data) - ffs = c.get(0xFF, 0) - ff00 = sum(1 for i in range(n - 1) if data[i] == 0xFF and data[i + 1] == 0) - exp = ffs * c.get(0, 0) / max(1, n) # expected if independent - out.append(f" stuffing check: ff count {ffs} ({ffs / max(1, n):.4%}), " - f"ff00 digrams {ff00}, expected-if-independent {exp:.1f}") - - -def analyse(fname, show_bytes): - out = [f"== {fname}"] - buf = open(fname, "rb").read() - endian = {b"II": "<", b"MM": ">"}.get(buf[0:2]) - (hlen,) = struct.unpack_from(endian + "I", buf, 2) - recs = dict() - raw = None - for tag, off, ln in find_records(buf, hlen, len(buf) - hlen, endian): - if tag == 0x2005: - raw = (off, ln) - recs.setdefault(tag, (off, ln)) - if raw is None or raw[0] is None: - out.append(" !! no RawData record") - return out - roff, rlen = raw - data = buf[roff : roff + rlen] - # SensorInfo - soff, sln = recs.get(0x1031, (None, 0)) - w = h = top = 0 - if soff: - vals = struct.unpack_from(endian + "H" * (sln // 2), buf, soff) - w, h, top = vals[1], vals[2], vals[6] - # DecoderTable - doff, dln = recs.get(0x1835, (None, 0)) - dt = struct.unpack_from(endian + "IIII", buf, doff) if doff and dln >= 16 else (None,) * 4 - n_tbl, d1, d2, comp_len = dt - low = (w * h) // 4 - rem = rlen - (comp_len or 0) - out.append(f" raw {w}x{h} S={rlen} tbl={n_tbl} f1={d1} f2={d2} L={comp_len}" - f" S-L={rem} wh/4={low} S-L-wh/4={rem - low}") - out.append(f" leading zero run: {zero_run(data)} bytes" - + (f" (= {zero_run(data) / (w / 4):.2f} rows of 2-bit LSBs)" if w else "")) - if show_bytes: - out.append(f" last 8 bytes: {data[-8:].hex()}") - has_low = rem - low in (0, 1, 2, 3, 4) and low > 4 * (w or 1) - if has_low and w: - out.append(" prefix hypothesis — region [0 : wh/4) as 2-bit rows:") - row_structure(data[:low], w // 4, top, "prefix", out) - out.append(" suffix hypothesis — region [L : L+wh/4) as 2-bit rows:") - row_structure(data[comp_len : comp_len + low], w // 4, top, "suffix", out) - out.append(" compressed-candidate stats:") - for name, seg in (("after-prefix [wh/4 : wh/4+L)", data[low : low + comp_len]), - ("from-0 [0 : L)", data[:comp_len])): - out.append(f" {name}: entropy {entropy(seg[:1 << 20]):.4f} (first MiB), " - f"{entropy(seg[-(1 << 20):]):.4f} (last MiB)") - ff_digrams(data[low : low + comp_len], out) - else: - out.append(" no wh/4 region (PowerShot class): compressed assumed [0 : L)") - out.append(f" entropy first MiB {entropy(data[:1 << 20]):.4f}, " - f"last MiB {entropy(data[max(0, comp_len - (1 << 20)) : comp_len]):.4f}") - ff_digrams(data[:comp_len], out) - out.append(f" tail region [L : S): {rem} bytes" - + (f", hex {data[comp_len:rlen].hex()}" if show_bytes and rem <= 8 else "")) - return out - - -def main(): - args = sys.argv[1:] - show = True - if args and args[0] == "--no-bytes": - show = False - args = args[1:] - for fname in args: - try: - print("\n".join(analyse(fname, show))) - except Exception as e: - print(f"== {fname}\n !! exception: {e!r}") - print() - - -if __name__ == "__main__": - main() diff --git a/crates/crw/clean-room/p1b_markers.py b/crates/crw/clean-room/p1b_markers.py deleted file mode 100644 index 65d9137e..00000000 --- a/crates/crw/clean-room/p1b_markers.py +++ /dev/null @@ -1,119 +0,0 @@ -"""P1b — ff-successor and JPEG-marker statistics of the RawData payload regions. - -Follow-up to p1_payload.py: on EOS bodies every 0xff in the compressed region was followed -by 0x00 (JPEG-style stuffing); on the G5 the payload starts with exactly 514 zero bytes -(= the constant DecoderTable field) and ends in ff d9 (JPEG EOI). This script measures, -per file: the histogram of bytes following 0xff in the compressed region, any JPEG markers -(ff xx, xx not 00), and — for public files — hex of small structural windows (region -starts/ends, bytes around offset 514). Still no entropy decoding. - -Usage: python -I p1b_markers.py [--no-bytes] file.crw ... -""" - -import struct -import sys -from collections import Counter - - -def find_records(buf, start, length, endian, depth=0): - """Yield (tag_id, abs_offset, length) for in-heap records, recursively - (same container model as ciff_dump.py; duplicated so the script runs under python -I).""" - if depth > 8 or length < 6: - return - (tbl,) = struct.unpack_from(endian + "I", buf, start + length - 4) - if tbl + 2 > length - 4: - return - (n,) = struct.unpack_from(endian + "H", buf, start + tbl) - if tbl + 2 + n * 10 > length: - return - for k in range(n): - tc, ln, off = struct.unpack_from(endian + "HII", buf, start + tbl + 2 + k * 10) - stg = (tc >> 14) & 3 - dt = (tc >> 11) & 7 - tag = tc & 0x3FFF - if stg != 0: - yield (tag, None, (ln, off)) - continue - if off + ln > length: - continue - if dt in (5, 6): - yield from find_records(buf, start + off, ln, endian, depth + 1) - else: - yield (tag, start + off, ln) - - -def analyse(fname, show): - out = [f"== {fname}"] - buf = open(fname, "rb").read() - endian = {b"II": "<", b"MM": ">"}.get(buf[0:2]) - (hlen,) = struct.unpack_from(endian + "I", buf, 2) - recs = {} - for tag, off, ln in find_records(buf, hlen, len(buf) - hlen, endian): - if off is not None: - recs.setdefault(tag, (off, ln)) - roff, rlen = recs[0x2005] - data = buf[roff : roff + rlen] - soff, sln = recs.get(0x1031, (None, 0)) - vals = struct.unpack_from(endian + "H" * (sln // 2), buf, soff) if soff else () - w, h = (vals[1], vals[2]) if len(vals) > 2 else (0, 0) - doff, _ = recs.get(0x1835, (None, 0)) - n_tbl, _, f2, comp_len = struct.unpack_from(endian + "IIII", buf, doff) - low = w * h // 4 - has_low = 0 <= rlen - comp_len - low <= 4 - comp = data[low : low + comp_len] if has_low else data[:comp_len] - out.append(f" raw {w}x{h} tbl={n_tbl} f2={f2} L={comp_len} lowbits={'yes' if has_low else 'no'}") - - succ = Counter() - markers = Counter() - first_markers = [] - i = 0 - while True: - i = comp.find(0xFF, i) - if i < 0 or i + 1 >= len(comp): - break - nxt = comp[i + 1] - succ[nxt] += 1 - if nxt != 0: - markers[nxt] += 1 - if len(first_markers) < 12: - first_markers.append((i, nxt)) - i += 2 - else: - i += 2 - top = ", ".join(f"{b:02x}:{c}" for b, c in succ.most_common(8)) - out.append(f" ff followed by: {top}") - if markers: - mk = ", ".join(f"ff{b:02x}:{c}" for b, c in sorted(markers.items())) - out.append(f" markers: {mk}") - out.append(f" first markers at: {[(hex(p), hex(m)) for p, m in first_markers]}") - rst = [p for p, m in first_markers if 0xD0 <= m <= 0xD7] - if len(rst) > 2: - out.append(f" first restart gaps: {[rst[k + 1] - rst[k] for k in range(len(rst) - 1)]}") - else: - out.append(" markers: none (pure stuffed entropy data)") - if show: - out.append(f" comp[0:24] : {comp[:24].hex()}") - out.append(f" comp[-8:] : {comp[-8:].hex()}") - out.append(f" raw[504:544] : {data[504:544].hex()}") - out.append(f" raw[-8:] : {data[-8:].hex()}") - if has_low: - out.append(f" low[-8:] : {data[low - 8 : low].hex()} (end of low-bits region)") - return out - - -def main(): - args = sys.argv[1:] - show = True - if args and args[0] == "--no-bytes": - show = False - args = args[1:] - for fname in args: - try: - print("\n".join(analyse(fname, show))) - except Exception as e: - print(f"== {fname}\n !! exception: {e!r}") - print() - - -if __name__ == "__main__": - main() diff --git a/crates/crw/clean-room/p1c_lowbits_bounds.py b/crates/crw/clean-room/p1c_lowbits_bounds.py deleted file mode 100644 index 8ad04db5..00000000 --- a/crates/crw/clean-room/p1c_lowbits_bounds.py +++ /dev/null @@ -1,96 +0,0 @@ -"""P1c — verify the low-bits region bounds via non-stuffed ff statistics. - -On EOS bodies the entropy region contains no ff followed by non-00 at all (perfect -JPEG-style stuffing); on the 12-bit PowerShots (G5/G6) a region of unstuffed uniform -bytes exists. This script shows that every non-stuffed ff falls inside -[514, 514 + wh/4) of the RawData payload, establishing the layout -[514 zeros][low bits wh/4][stuffed entropy][terminator] for those bodies, and -reports the zero run + stuffing purity for every file class. No entropy decoding. - -Usage: python -I p1c_lowbits_bounds.py file.crw ... -""" - -import struct -import sys - - -def records(buf, start, length, endian, depth=0): - if depth > 8 or length < 6: - return - (tbl,) = struct.unpack_from(endian + "I", buf, start + length - 4) - if tbl + 2 > length - 4: - return - (n,) = struct.unpack_from(endian + "H", buf, start + tbl) - if tbl + 2 + n * 10 > length: - return - for k in range(n): - tc, ln, off = struct.unpack_from(endian + "HII", buf, start + tbl + 2 + k * 10) - if (tc >> 14) & 3: - continue - if off + ln > length: - continue - if ((tc >> 11) & 7) in (5, 6): - yield from records(buf, start + off, ln, endian, depth + 1) - else: - yield (tc & 0x3FFF, start + off, ln) - - -def analyse(fname): - buf = open(fname, "rb").read() - endian = {b"II": "<", b"MM": ">"}.get(buf[0:2]) - (hl,) = struct.unpack_from(endian + "I", buf, 2) - r = {} - for t, o, l in records(buf, hl, len(buf) - hl, endian): - r.setdefault(t, (o, l)) - ro, rl = r[0x2005] - so, sl = r[0x1031] - v = struct.unpack_from(endian + "H" * (sl // 2), buf, so) - w, h = v[1], v[2] - do, _ = r[0x1835] - n_tbl, _, f2, comp_len = struct.unpack_from(endian + "IIII", buf, do) - data = buf[ro : ro + rl] - low = w * h // 4 - eos = 0 <= rl - comp_len - low <= 4 - - # positions of non-stuffed ff (ff followed by non-00), excluding the terminator - pos = [] - i = 0 - while True: - i = data.find(0xFF, i) - if i < 0 or i + 1 >= len(data): - break - if data[i + 1] != 0: - pos.append(i) - i += 2 - pos = [p for p in pos if p < rl - 4] - - comp_start = low if eos else 0 - zr = 0 - while comp_start + zr < rl and data[comp_start + zr] == 0: - zr += 1 - name = fname.replace("\\", "/").rsplit("/", 1)[-1] - if eos: - inside = sum(1 for p in pos if p < low) - print(f"{name:36} EOS layout: lowbits [0:{low}) + zeros({zr}) + entropy; " - f"non-stuffed ff: {len(pos)} total, {inside} in lowbits, " - f"{len(pos) - inside} in entropy (expect 0)") - else: - lo, hi = 514, 514 + low - inside = sum(1 for p in pos if lo <= p < hi) - if pos and inside == len(pos): - print(f"{name:36} PS 12-bit layout: zeros({zr}) + lowbits [{lo}:{hi}) + entropy; " - f"all {len(pos)} non-stuffed ff inside lowbits; " - f"first {pos[0]}, last {pos[-1]}") - elif not pos: - print(f"{name:36} PS 10-bit layout: zeros({zr}) + entropy; " - f"no non-stuffed ff (pure stuffed stream)") - else: - print(f"{name:36} PS ?: {len(pos)} non-stuffed ff, {inside} inside " - f"[{lo}:{hi}); first {pos[0]}, last {pos[-1]} — hypothesis fails") - - -for fn in sys.argv[1:]: - try: - analyse(fn) - except Exception as e: - print(f"{fn}: !! {e!r}") diff --git a/crates/crw/clean-room/p2_oracle_lsb.py b/crates/crw/clean-room/p2_oracle_lsb.py deleted file mode 100644 index e0e9dcfe..00000000 --- a/crates/crw/clean-room/p2_oracle_lsb.py +++ /dev/null @@ -1,118 +0,0 @@ -"""P2 — the decisive low-bits test against the oracle (LibRaw unprocessed_raw, black box). - -Input: a CRW file and the TIFF written by `unprocessed_raw.exe -T` for it. -Tests, per 03-payload-layout.md: - - oracle frame size == SensorInfo raw size; - - oracle code histogram (min/max/percentiles) -> bit depth; - - the low-bits region equals the 2 LSBs of the oracle's codes, row-major, 4 pixels/byte, - under one of the two packing orders (pixel0 in bits 7-6 ... pixel3 in bits 1-0, or the - reverse); reports exact matching byte counts for both orders and both candidate - positions of the region (EOS prefix [0:Q) / PowerShot [514:514+Q)). - -Statistics and equality counts only. Usage: - python -I p2_oracle_lsb.py file.crw file.crw.tiff -""" - -import struct -import sys - - -def records(buf, start, length, endian, depth=0): - if depth > 8 or length < 6: - return - (tbl,) = struct.unpack_from(endian + "I", buf, start + length - 4) - if tbl + 2 > length - 4: - return - (n,) = struct.unpack_from(endian + "H", buf, start + tbl) - if tbl + 2 + n * 10 > length: - return - for k in range(n): - tc, ln, off = struct.unpack_from(endian + "HII", buf, start + tbl + 2 + k * 10) - if (tc >> 14) & 3: - continue - if off + ln > length: - continue - if ((tc >> 11) & 7) in (5, 6): - yield from records(buf, start + off, ln, endian, depth + 1) - else: - yield (tc & 0x3FFF, start + off, ln) - - -def read_tiff_gray16(path): - """Minimal classic-TIFF reader: single image, 16-bit, strips, no compression.""" - buf = open(path, "rb").read() - endian = {b"II": "<", b"MM": ">"}[buf[0:2]] - (magic, ifdoff) = struct.unpack_from(endian + "HI", buf, 2) - assert magic == 42 - (n,) = struct.unpack_from(endian + "H", buf, ifdoff) - tags = {} - for i in range(n): - tag, typ, cnt, val = struct.unpack_from(endian + "HHII", buf, ifdoff + 2 + i * 12) - tags[tag] = (typ, cnt, val, ifdoff + 2 + i * 12 + 8) - def val(tag): - typ, cnt, v, pos = tags[tag] - size = {1: 1, 3: 2, 4: 4}[typ] - if size * cnt <= 4: - if typ == 3 and cnt == 1: - return [struct.unpack_from(endian + "H", buf, pos)[0]] - return [v] - fmt = {1: "B", 3: "H", 4: "I"}[typ] - return list(struct.unpack_from(endian + fmt * cnt, buf, v)) - w = val(256)[0] - h = val(257)[0] - bps = val(258)[0] - comp = val(259)[0] if 259 in tags else 1 - assert bps == 16 and comp == 1, (bps, comp) - offs = val(273) - counts = val(279) - data = b"".join(buf[o : o + c] for o, c in zip(offs, counts)) - assert len(data) >= w * h * 2, (len(data), w * h * 2) - import array - a = array.array("H") - a.frombytes(data[: w * h * 2]) - if (endian == ">") != (sys.byteorder == "big"): - a.byteswap() - return w, h, a - - -def main(): - crw_path, tiff_path = sys.argv[1], sys.argv[2] - buf = open(crw_path, "rb").read() - endian = {b"II": "<", b"MM": ">"}[buf[0:2]] - (hl,) = struct.unpack_from(endian + "I", buf, 2) - r = {} - for t, o, l in records(buf, hl, len(buf) - hl, endian): - r.setdefault(t, (o, l)) - ro, rl = r[0x2005] - so, sl = r[0x1031] - v = struct.unpack_from(endian + "H" * (sl // 2), buf, so) - w, h = v[1], v[2] - q = w * h // 4 - data = buf[ro : ro + rl] - - tw, th, pix = read_tiff_gray16(tiff_path) - print(f"CRW raw {w}x{h}; oracle TIFF {tw}x{th} -> {'MATCH' if (tw, th) == (w, h) else 'MISMATCH'}") - s = sorted(pix[::997]) - print(f"oracle codes: min {min(pix)}, max {max(pix)}, p1 {s[len(s)//100]}, " - f"median {s[len(s)//2]}, p99 {s[-len(s)//100]}") - - # build expected low-bits bytes from oracle LSBs, both packing orders - import array - lsb = array.array("B", (p & 3 for p in pix)) - exp_msb = bytearray(q) - exp_lsb = bytearray(q) - for i in range(q): - a4 = lsb[4 * i : 4 * i + 4] - exp_msb[i] = (a4[0] << 6) | (a4[1] << 4) | (a4[2] << 2) | a4[3] - exp_lsb[i] = a4[0] | (a4[1] << 2) | (a4[2] << 4) | (a4[3] << 6) - for region_name, reg in (("prefix [0:Q)", data[:q]), ("ps [514:514+Q)", data[514 : 514 + q])): - if len(reg) < q: - continue - m1 = sum(1 for a, b in zip(reg, exp_msb) if a == b) - m2 = sum(1 for a, b in zip(reg, exp_lsb) if a == b) - print(f"{region_name}: match MSB-first {m1}/{q} ({m1 / q:.6%}), " - f"LSB-first {m2}/{q} ({m2 / q:.6%})") - - -if __name__ == "__main__": - main() diff --git a/crates/crw/clean-room/sources.md b/crates/crw/clean-room/sources.md index 44933c95..32b7e0a2 100644 --- a/crates/crw/clean-room/sources.md +++ b/crates/crw/clean-room/sources.md @@ -48,7 +48,7 @@ checked out, diffed, logged or read. - 2026-10-12 19:28 — **Search** (WebSearch, standard): `Canon CIFF specification 1.0 rev 4 PDF "CIFF" camera image file format specification download`. Results included Wikipedia "Camera Image File Format", a freedesktop libopenraw wiki page "Canon CRW" (NOT opened — it is - a decoder project's page; left for the maintainer to approve, see plan §7), an unpkg mirror of + a decoder project's page; left for the maintainer to approve), an unpkg mirror of exiftool's canon_raw.html, PRONOM. Taken: pointer to Wikipedia for an archived spec URL. **No decoder algorithm details shown in the result snippets.** - 2026-10-12 19:28 — **Fetch attempt** `http://xyrion.org/ciff/` — failed (site dead). @@ -86,22 +86,23 @@ checked out, diffed, logged or read. different format. **No decoder algorithm details shown.** - 2026-10-12 19:32 — **Download** (Python urllib, stdlib) the 11 CC0 files above from `https://raw.pixls.us/getfile.php/...` into the gitignored `corpus/crw/`; every file verified - against the site's published SHA-256 (all OK; table with hashes in `01-plan.md`). + against the site's published SHA-256 (all OK; the hashes are pinned in + `crates/crw/tests/samples.json`). ## Local analysis (scripts in this directory) - 2026-10-12 19:35–19:45 — `ciff_dump.py` (written here, stdlib only; container layout - implemented from the two documented sources above) run over the 11 public files - (`logs/structure-public.txt`) and the 18 private files - (`logs/private/structure-private.txt`, gitignored — may contain personal metadata). Observed - facts are summarised in `01-plan.md` §3. No raw-data decoding was attempted; statistics only. -- 2026-10-12 ~20:0x — P0/P1 local analysis (post-approval): `02-container.md` written from the + implemented from the two documented sources above) run over the 11 public files and the 18 + private files (whose dumps may contain personal metadata and are never committed). Observed + facts are summarised in `01-payload-layout.md`. No raw-data decoding was attempted; statistics + only. The script and its logs are not part of this record. +- 2026-10-12 ~20:0x — P0/P1 local analysis (post-approval): the container notes written from the already-logged sources + corpus observations. `p1_payload.py`, `p1b_markers.py`, - `p1c_lowbits_bounds.py` run over all 29 files (public logs `logs/p1*`, private under - `logs/private/`). Established statistically (no decoding): the three payload layouts, the + `p1c_lowbits_bounds.py` run over all 29 files. Established statistically (no decoding): the + three payload layouts, the 514-byte zero block at the compressed section start of every file, JPEG-style ff00 stuffing of every entropy stream (zero exceptions), the PowerShot `ff d9` terminator, and the exact - low-bits region bounds on G5/G6. Documented in `03-payload-layout.md`. No external sources + low-bits region bounds on G5/G6. Documented in `01-payload-layout.md`. No external sources consulted in this step. ## Oracle (P2, from 2026-10-12) @@ -110,7 +111,8 @@ checked out, diffed, logged or read. a local `LibRaw-0.22.2\bin` folder. That directory tree also contains LibRaw's source code; **no source file there is ever opened, read or searched** — only `bin\unprocessed_raw.exe` is executed, as a black box, per the approved plan (§5a/§7). - All oracle runs and probe campaigns are logged in `04-derivation-log.md`. + All oracle runs and probe campaigns are logged in the derivation log, which is not part of this + record. - 2026-10-13 — **Fetch** `https://raw.pixls.us/json/getrepository.php?set=all` (and the site root to find that endpoint): recovered the stable `getfile.php` URLs of the 11 CC0 corpus @@ -130,5 +132,5 @@ checked out, diffed, logged or read. - 2026-10-25 — **Fetch** `https://raw.githubusercontent.com/colour-science/colour/v0.4.2/colour/characterisation/datasets/colour_checkers/chromaticity_coordinates.py` (colour-science, BSD-3-Clause; also its rendered docs page): the 24 post-November-2014 ColorChecker Classic L*a*b* values (D50), credited there to X-Rite's release. Reference - data only, the chart's measured colours; no camera or decoder content. Used in - `10-chart-calibration.md`. + data only, the chart's measured colours; no camera or decoder content. Used for the EOS D60 + colour matrices. diff --git a/crates/crw/clean-room/tools/gen_tables.py b/crates/crw/clean-room/tools/gen_tables.py index 666f0b9e..dc8c0ca4 100644 --- a/crates/crw/clean-room/tools/gen_tables.py +++ b/crates/crw/clean-room/tools/gen_tables.py @@ -1,7 +1,7 @@ """Regenerate crates/crw/src/tables.rs from the frozen tables in crates/crw/clean-room/logs/table_sel{0,1,2}.json. Run from anywhere: `python crates/crw/clean-room/tools/gen_tables.py`. The JSON files are the -derivation's frozen data (06-crw-format.md section 3); this script only renders them as Rust +derivation's frozen data (02-crw-format.md section 3); this script only renders them as Rust constants, keeping the file's doc comment. It was a local tool during the derivation in LightCraft (plan/crw/model/gen_tables.py) and moved here so the tables stay reproducible. """ diff --git a/crates/crw/src/lib.rs b/crates/crw/src/lib.rs index f09cd20b..ce6cc633 100644 --- a/crates/crw/src/lib.rs +++ b/crates/crw/src/lib.rs @@ -5,8 +5,8 @@ //! The container rules come from Canon's published CIFF specification and ExifTool's prose tag //! documentation; the compression was derived from CC0 and maintainer-owned sample files by //! black-box analysis against a reference decoder's command-line output, frozen in -//! `clean-room/06-crw-format.md` and verified sample-exact on 28 of 29 corpus files plus 1,516 -//! further EOS D60 files (`clean-room/07-verification.md`). The code tables are frozen data +//! `clean-room/02-crw-format.md` and verified sample-exact on 28 of 29 corpus files plus 1,516 +//! further EOS D60 files (`clean-room/03-verification.md`). The code tables are frozen data //! (`tables`, generated from `clean-room/logs/table_sel{0,1,2}.json`). No other raw decoder's //! source was consulted; the whole derivation record is in the crate's `clean-room/` folder. //! @@ -326,7 +326,7 @@ fn parse(bytes: &[u8], limits: &Limits, full: bool) -> Result { .unwrap_or_default(); if model.contains("PowerShot G5") { return Err(Error::Unsupported( - "Canon PowerShot G5 entropy coding (a documented anomaly of the derivation, clean-room/06-crw-format.md §4); use the embedded JPEG" + "Canon PowerShot G5 entropy coding (a documented anomaly of the derivation, clean-room/02-crw-format.md §4); use the embedded JPEG" .into(), )); } @@ -622,7 +622,7 @@ fn build_lut(codes: &[(u16, u8, u8, u8)]) -> Vec { } /// Decode the ff00-stuffed entropy stream into the `w × h` high-plane samples, per the frozen -/// model (`clean-room/06-crw-format.md` §2): 64-sample blocks, first/main code tables, +/// model (`clean-room/02-crw-format.md` §2): 64-sample blocks, first/main code tables, /// (run, size) leaves with JPEG-rule magnitudes, EOB, block-DC carry, and two parity DPCM chains /// initialised to 512 and reset at every row start. A stream that ends early leaves the remaining /// samples at their chain values (the frozen stream-end rule). diff --git a/crates/crw/src/tables.rs b/crates/crw/src/tables.rs index 60ebb9f9..027329a5 100644 --- a/crates/crw/src/tables.rs +++ b/crates/crw/src/tables.rs @@ -2,7 +2,7 @@ //! //! Generated by `clean-room/tools/gen_tables.py` from `clean-room/logs/table_sel{0,1,2}.json` (the //! tables extracted by known-plaintext alignment and frozen at the CRW-FREEZE commit; derivation: -//! `clean-room/06-crw-format.md` §3). Do not edit by hand; regenerate from the JSON. +//! `clean-room/02-crw-format.md` §3). Do not edit by hand; regenerate from the JSON. //! Each entry is `(code bits, code length, run, size)`; codes are MSB-first. /// One selector's pair of canonical Huffman tables: the code starting each 64-sample diff --git a/crates/crw/tests/samples.rs b/crates/crw/tests/samples.rs index 71828438..1f729f02 100644 --- a/crates/crw/tests/samples.rs +++ b/crates/crw/tests/samples.rs @@ -1,6 +1,6 @@ //! The CC0 sample files (raw.pixls.us) pinned in `samples.json`: each decodes to exactly the //! sensor samples the derivation verified against the black-box reference (`clean-room/ -//! 07-verification.md`), recorded as the SHA-256 of the little-endian sample buffer, with the +//! 03-verification.md`), recorded as the SHA-256 of the little-endian sample buffer, with the //! levels, white balance and metadata the decoder read then. //! //! The files are not in the repository (5–11 MB each): `python scripts/fetch-crw-samples.py From df10ce00a5cebebeb178ea1d07c508a8cbd2d576 Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Mon, 26 Oct 2026 18:10:03 -0400 Subject: [PATCH 07/10] crw: fix the low-bits discriminator on payloads that end in the trailer A no-plane PowerShot frame whose payload is shorter than plane + header ends inside the scan window with the record's real `ff d9 00` trailer, so rule (a) read the `ff d9` as a plane byte, reported 12 bits and then refused a valid file. Both discriminator signs now require a payload long enough to hold a plane, so a flat G1 frame decodes instead of being rejected. Also from the review: clamp samples to the declared depth so a damaged stream cannot put more than `bits` into a consumer's `1 << bits` LUT, report zero sensor dimensions as corrupt rather than as a limit, take a signed ImageInfo rotation modulo 360 (so -90 is 270), warn against pairing BlackLevel::values with Header::cfa by index, and pin actions/cache in the crate's workflow. Tests cover the trailer payload, end-of-block chain values, the clamp, zero dimensions, both white-balance slot layouts with every Canon setting, and each rotation spelling. The 11 CC0 samples still decode exact with CRAFT_CRW_REQUIRE_SAMPLES=1. --- .github/workflows/crw.yml | 2 +- crates/crw/src/lib.rs | 32 +++++-- crates/crw/src/tests.rs | 171 +++++++++++++++++++++++++++++++++----- 3 files changed, 178 insertions(+), 27 deletions(-) diff --git a/.github/workflows/crw.yml b/.github/workflows/crw.yml index b55472f2..4c1da7fe 100644 --- a/.github/workflows/crw.yml +++ b/.github/workflows/crw.yml @@ -67,7 +67,7 @@ jobs: with: key: craft-crw-samples - name: Cache the CC0 sample files - uses: actions/cache@v4 + uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0 with: path: ${{ runner.temp }}/crw-samples key: crw-samples-${{ hashFiles('crates/crw/tests/samples.json') }} diff --git a/crates/crw/src/lib.rs b/crates/crw/src/lib.rs index ce6cc633..da12940e 100644 --- a/crates/crw/src/lib.rs +++ b/crates/crw/src/lib.rs @@ -114,6 +114,11 @@ impl Rect { } /// Black level per 2×2 CFA site, row-major, anchored at the active area's origin. +/// +/// The anchor differs from [`Header::cfa`]'s: `cfa` is the phase at sensor (0, 0), `values` is +/// indexed from the active area's origin. The active area can start at an odd sensor row (the +/// PowerShot G1 starts at y = 15), so `values[i]` does not line up with `cfa.pattern()[i]`: read a +/// level through [`BlackLevel::at`] rather than pairing the two by index. #[derive(Debug, Clone, Copy, PartialEq)] pub struct BlackLevel { pub values: [f32; 4], @@ -302,7 +307,10 @@ fn parse(bytes: &[u8], limits: &Limits, full: bool) -> Result { (Some(&w), Some(&h)) => (w as usize, h as usize), _ => return Err(corrupt("SensorInfo too short")), }; - let n = w.checked_mul(h).filter(|&n| n > 0 && n <= limits.max_samples).ok_or(Error::LimitExceeded("image too large"))?; + if w == 0 || h == 0 { + return Err(corrupt("zero sensor dimensions")); + } + let n = w.checked_mul(h).filter(|&n| n <= limits.max_samples).ok_or(Error::LimitExceeded("image too large"))?; let &(dto, dtl) = rec.get(&DECODER_TABLE).ok_or_else(|| corrupt("no DecoderTable record"))?; if dtl < 16 { @@ -352,8 +360,12 @@ fn parse(bytes: &[u8], limits: &Limits, full: bool) -> Result { let q = n.div_ceil(4); let span = q.saturating_add(ent_off); let head = payload.get(..span.min(payload.len())).unwrap_or(payload); - let header_after_plane = ent_off >= 64 && payload.len() > span && payload.get(q..span).is_some_and(|z| z.iter().all(|&b| b == 0)); - let has_low = header_after_plane || head.windows(2).any(|p| p[0] == 0xff && p[1] != 0x00); + // Both signs need a payload long enough to hold a plane and the header behind it. A shorter + // payload ends in the PowerShot `ff d9 00` trailer, and `ff d9` is a 0xff not followed by + // 0x00: without this gate a flat no-plane G1 frame read as 12-bit and was then refused. + let plane_fits = payload.len() > span; + let header_after_plane = plane_fits && ent_off >= 64 && payload.get(q..span).is_some_and(|z| z.iter().all(|&b| b == 0)); + let has_low = header_after_plane || (plane_fits && head.windows(2).any(|p| p[0] == 0xff && p[1] != 0x00)); let bits = if has_low { 12 } else { 10 }; let low_len = if has_low { q } else { 0 }; let samples = if full { @@ -364,13 +376,17 @@ fn parse(bytes: &[u8], limits: &Limits, full: bool) -> Result { } let hi = decode_entropy(entropy, w, h, &tables)?; let mut samples = vec![0u16; n]; + // `samples` is documented as `bits` deep, so a damaged stream whose chain value runs past + // the range is clamped here: a consumer indexing a `1 << bits` LUT must not see more. + let top = ((1u32 << bits) - 1) as u16; for (i, (dst, &hi)) in samples.iter_mut().zip(hi.iter()).enumerate() { *dst = match low { Some(low) => { let lb = low.get(i >> 2).map_or(0, |&b| (b >> ((i & 3) * 2)) & 3); - ((((hi as u32) << 2) | lb as u32) & 0xffff) as u16 + let v = (((hi as u32) << 2) | lb as u32) & 0xffff; + v.min(u32::from(top)) as u16 } - None => hi, + None => hi.min(top), }; } samples @@ -387,7 +403,11 @@ fn parse(bytes: &[u8], limits: &Limits, full: bool) -> Result { // ImageInfo rotation: degrees; 270 observed on portrait files whose scene is upright after a // 90° counter-clockwise turn (Exif 8), so 90 is its mirror (Exif 6) - let exif_orientation = match rec.get(&IMAGE_INFO).and_then(|&(io, il)| (il >= 16).then(|| u32_at(bytes, io + 12, o)).flatten()) { + // The record holds a signed degree count, so a file may write −90 for 270: take it modulo + // 360 before mapping, and leave anything off-axis (0, ±360) at the identity. + let rotation_deg = + rec.get(&IMAGE_INFO).and_then(|&(io, il)| (il >= 16).then(|| u32_at(bytes, io + 12, o)).flatten()).map(|r| (r as i32).rem_euclid(360)); + let exif_orientation = match rotation_deg { Some(90) => 6, Some(180) => 3, Some(270) => 8, diff --git a/crates/crw/src/tests.rs b/crates/crw/src/tests.rs index 06bc9ffa..a858cef9 100644 --- a/crates/crw/src/tests.rs +++ b/crates/crw/src/tests.rs @@ -99,16 +99,17 @@ fn encode_hi(hi: &[u16], w: usize, t: &tables::SelTables) -> Vec { bits.extend((0..s).rev().map(|i| (m >> i) & 1 == 1)); } } - let mut bytes = Vec::new(); + pack_entropy(&bits) +} + +/// MSB-first packing into bytes, zero padding in the last byte, then ff00 stuffing. +fn pack_entropy(bits: &[bool]) -> Vec { + let mut stuffed = Vec::new(); for chunk in bits.chunks(8) { let mut b = 0u8; for (i, &bit) in chunk.iter().enumerate() { b |= (bit as u8) << (7 - i); } - bytes.push(b); - } - let mut stuffed = Vec::new(); - for b in bytes { stuffed.push(b); if b == 0xff { stuffed.push(0x00); @@ -117,6 +118,25 @@ fn encode_hi(hi: &[u16], w: usize, t: &tables::SelTables) -> Vec { stuffed } +/// One end-of-block token pair per 64-sample block: the block's first sample takes the carry +/// (zero here), the other 63 keep their chain values. A flat frame like this compresses to a +/// small fraction of its low-bits plane, the shape a lens-cap or blown-out frame has. +fn encode_eob(n: usize, t: &tables::SelTables) -> Vec { + let code = |tab: &[(u16, u8, u8, u8)]| { + let &(c, l, ..) = tab.iter().find(|e| e.2 == 0 && e.3 == 0).expect("an end-of-block code"); + (c, l) + }; + let (fc, fl) = code(t.first); + let (mc, ml) = code(t.main); + let mut bits: Vec = Vec::new(); + for _ in 0..n.div_ceil(64) { + for (c, l) in [(fc, fl), (mc, ml)] { + bits.extend((0..l).rev().map(|i| (c >> i) & 1 == 1)); + } + } + pack_entropy(&bits) +} + /// A high plane whose diffs are all nonzero, bounded around 512: values follow /// `(3 qi) mod 5 + 1`, so same-parity neighbours differ by 1 or −4 (never 0) and /// row starts sit 1–5 above the reset value. @@ -132,6 +152,22 @@ fn payload(low: Option<&[u8]>, entropy: &[u8]) -> Vec { p } +/// The real PowerShot record shape: the payload closes with `ff d9` and a pad byte, not `00 00`. +fn payload_with_trailer(low: Option<&[u8]>, entropy: &[u8]) -> Vec { + let mut p = low.map(|l| l.to_vec()).unwrap_or_default(); + p.extend_from_slice(&[0; 514]); + p.extend_from_slice(entropy); + p.extend_from_slice(&[0xff, 0xd9, 0x00]); + p +} + +/// A file whose raw record is `payload` verbatim, with `extra` records beside the usual three. +fn file_with_payload(w: u16, h: u16, model: &str, payload: &[u8], extra: &[(u16, Vec)], o: Order) -> Vec { + let mut props = vec![(SENSOR_INFO, sensor_info(w, h, None, o)), (DECODER_TABLE, decoder_table(0, o)), (MAKE_MODEL, make_model(model))]; + props.extend_from_slice(extra); + ciff(&[(RAW_DATA, payload.to_vec()), (0x300a, heap(&props, o))], o) +} + fn make_model(model: &str) -> Vec { let mut v = b"Canon\0".to_vec(); v.extend_from_slice(model.as_bytes()); @@ -458,22 +494,17 @@ fn white_balance_comes_from_the_shot_info_slot() { fn rotation_maps_to_exif_orientation_and_the_time_stamp_is_unix_seconds() { let o = Order::Le; let hi = test_hi(128); - let image_info: Vec = [3072u32, 2048, 0, 270, 12, 12, 0].iter().flat_map(|&v| put32(v, o)).collect(); - let props = heap( - &[ - (SENSOR_INFO, sensor_info(16, 8, None, o)), - (DECODER_TABLE, decoder_table(0, o)), - (MAKE_MODEL, make_model("Canon EOS 300D DIGITAL")), - (IMAGE_INFO, image_info), - (TIME_STAMP, [1083410638u32, 0, 0].iter().flat_map(|&v| put32(v, o)).collect()), - ], - o, - ); let tables = tables::tables(0).expect("selector 0"); - let f = ciff(&[(RAW_DATA, payload(None, &encode_hi(&hi, 16, &tables))), (0x300a, props)], o); - let img = full(&f).expect("decodes"); - assert_eq!(img.header.exif_orientation, 8); - assert_eq!(img.header.capture_unix, Some(1083410638)); + // The record holds a signed degree count, so 270 may also be written as −90. + for (deg, want) in [(0u32, 1u8), (90, 6), (180, 3), (270, 8), (-90i32 as u32, 8), (-270i32 as u32, 6)] { + let image_info: Vec = [3072u32, 2048, 0, deg, 12, 12, 0].iter().flat_map(|&v| put32(v, o)).collect(); + let stamp: Vec = [1083410638u32, 0, 0].iter().flat_map(|&v| put32(v, o)).collect(); + let extra = [(IMAGE_INFO, image_info), (TIME_STAMP, stamp)]; + let f = file_with_payload(16, 8, "Canon EOS 300D DIGITAL", &payload(None, &encode_hi(&hi, 16, &tables)), &extra, o); + let img = full(&f).expect("decodes"); + assert_eq!(img.header.exif_orientation, want, "rotation {deg}"); + assert_eq!(img.header.capture_unix, Some(1083410638)); + } } #[test] @@ -528,6 +559,106 @@ fn self_referencing_sub_heaps_terminate() { } } +/// A no-plane G1 frame that compresses well closes with the record's real `ff d9 00` trailer and +/// is still shorter than plane + header — the shape of a lens-cap or blown-out frame. Rule (a) +/// read that `ff d9` as a plane byte, reported 12 bits and then refused the file. +#[test] +fn a_short_powershot_payload_closing_with_the_trailer_has_no_plane() { + let o = Order::Le; + let (w, h) = (64u16, 64u16); + let n = w as usize * h as usize; + let tables = tables::tables(0).expect("selector 0"); + let entropy = encode_eob(n, &tables); + let p = payload_with_trailer(None, &entropy); + let span = n.div_ceil(4) + 514; + assert!(p.len() < span, "the payload must be shorter than plane + header: {} vs {span}", p.len()); + assert!(p.windows(2).any(|q| q[0] == 0xff && q[1] != 0x00), "the trailer is the unstuffed ff pair"); + + let f = file_with_payload(w, h, "Canon PowerShot G1", &p, &[], o); + let hdr = header(&f, &Limits::default()).expect("header"); + assert_eq!(hdr.bits, 10, "the trailer is not a low-bits plane"); + // Every block is one end-of-block token, so the chain values stand (512 at each row start): + // the frame decodes instead of being refused for a plane it does not have. + let img = full(&f).expect("decodes"); + assert_eq!((img.header.bits, img.samples.iter().filter(|&&v| v == 512).count()), (10, n)); + + // The same frame without the trailer was already read as 10-bit. + let f = file_with_payload(w, h, "Canon PowerShot G1", &payload(None, &entropy), &[], o); + assert_eq!(header(&f, &Limits::default()).expect("header").bits, 10); +} + +/// An end-of-block token at the head of a block emits the carry and leaves the other 63 samples +/// at their chain values: the runs the sample hashes cover, on a file the unit tests can read. +#[test] +fn end_of_block_keeps_the_chain_values() { + let o = Order::Le; + let (w, h) = (64u16, 64u16); + let n = w as usize * h as usize; + let tables = tables::tables(0).expect("selector 0"); + let f = file_with_payload(w, h, "Canon EOS D30", &payload(None, &encode_eob(n, &tables)), &[], o); + let img = full(&f).expect("decodes"); + assert_eq!(img.samples.len(), n); + assert!(img.samples.iter().all(|&v| v == 512), "every sample keeps its row's chain value"); +} + +/// Zero sensor dimensions are damage, not a request for too much memory. +#[test] +fn zero_sensor_dimensions_are_corrupt_not_a_limit() { + let f = roundtrip_file(0, 0, "Canon EOS D60", None, &[], Order::Le); + assert!(matches!(full(&f), Err(Error::Corrupt(_)))); + assert!(matches!(header(&f, &Limits::default()), Err(Error::Corrupt(_)))); +} + +/// A hostile stream can run the chain past the bit depth (here a "10-bit" body writing 1535). +/// `samples` is documented as `bits` deep, so the surplus never reaches a `1 << bits` LUT. +#[test] +fn samples_are_clamped_to_the_declared_depth() { + let f = roundtrip_file(2, 1, "Canon PowerShot G1", None, &[512, 1535], Order::Le); + let img = full(&f).expect("decodes"); + assert_eq!((img.header.bits, img.samples.as_slice()), (10, &[512u16, 1023][..])); + // the 12-bit merge of the same chain cannot leave 0..=4095 either + let low = vec![0xffu8; 1]; + let f = roundtrip_file(2, 1, "Canon EOS D60", Some(&low), &[512, 1535], Order::Le); + let img = full(&f).expect("decodes"); + assert_eq!(img.header.bits, 12); + assert!(img.samples.iter().all(|&v| v <= 4095), "{:?}", img.samples); +} + +/// The two ColorBalance layouts and every Canon setting that picks a slot. Every corpus file is +/// set to Auto, so a wrong arm would still pass on the samples alone. +#[test] +fn white_balance_slot_arms_follow_the_camera_setting() { + let o = Order::Le; + // Quadruple k is (100+k, 200, 200, 300+k): the multipliers name the slot chosen, and the last + // quadruple is black levels, which is never a slot. + let wb = |usable: u16, setting: u16| { + let mut vals = vec![(usable + 1) * 8]; + for k in 0..=usable { + vals.extend([100 + k, 200, 200, 300 + k]); + } + let mut bytes: Vec = vals.iter().flat_map(|&v| put16(v, o)).collect(); + let cb_len = bytes.len(); + let si_off = bytes.len(); + let mut shot = [0u16; 8]; + shot[7] = setting; + bytes.extend(shot.iter().flat_map(|&v| put16(v, o))); + let mut rec = BTreeMap::new(); + rec.insert(COLOR_BALANCE, (0usize, cb_len)); + rec.insert(SHOT_INFO, (si_off, 16usize)); + wb_from_color_balance(&bytes, &rec, o) + }; + let expect = |k: u16| Some([(100 + k) as f32 / 200.0, 1.0, (300 + k) as f32 / 200.0]); + // 10D/300D: Auto, Daylight, Shade, Cloudy, Tungsten, Fluorescent, Flash, Custom, Kelvin + for (setting, slot) in [(0u16, 0u16), (1, 1), (8, 2), (2, 3), (3, 4), (4, 5), (5, 6), (6, 7), (9, 8), (7, 0)] { + assert_eq!(wb(9, setting), expect(slot), "nine-slot setting {setting}"); + } + // D60: Auto, Daylight, Cloudy, Tungsten, Fluorescent, Flash, Custom. Shade and Kelvin have no + // slot of their own and fall back to Auto. + for (setting, slot) in [(0u16, 0u16), (1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6), (8, 0), (9, 0)] { + assert_eq!(wb(7, setting), expect(slot), "seven-slot setting {setting}"); + } +} + #[test] fn errors_display_their_cause() { assert_eq!(Error::NotCrw.to_string(), "not a Canon CRW (CIFF) file"); From b600e6a5cd5f9ccab89377b3eb0f6bb8dbee0105 Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Wed, 28 Oct 2026 22:15:50 -0400 Subject: [PATCH 08/10] crw: split decode_entropy under the cognitive-complexity limit main's workspace lints now deny cognitive_complexity above 30, and decode_entropy scored 41. The emit macro becomes a Chains struct, unstuffing its own function, and the three stream-end checks one. Same algorithm: the pinned CC0 digests still match byte for byte. Also correct two record lines: the G5 is the one file that does not match, and the frozen code tables are not the only logs that ship. --- crates/crw/clean-room/02-crw-format.md | 4 +- crates/crw/clean-room/README.md | 4 +- crates/crw/src/lib.rs | 135 ++++++++++++------------- 3 files changed, 71 insertions(+), 72 deletions(-) diff --git a/crates/crw/clean-room/02-crw-format.md b/crates/crw/clean-room/02-crw-format.md index 7b5c855b..9d0304ad 100644 --- a/crates/crw/clean-room/02-crw-format.md +++ b/crates/crw/clean-room/02-crw-format.md @@ -111,5 +111,5 @@ which the Rust implementation may normalise, provided the verification table of ## 5 Verification See `03-verification.md` (written after the freeze): full-frame exact-match decode -against LibRaw 0.22.2 for every sample file, public and private, with zero tuning -after the freeze commit. +against LibRaw 0.22.2, with zero tuning after the freeze commit: every sample file, public +and private, matches except the PowerShot G5, which the decoder refuses (28 of 29). diff --git a/crates/crw/clean-room/README.md b/crates/crw/clean-room/README.md index 1174e713..4c2325ef 100644 --- a/crates/crw/clean-room/README.md +++ b/crates/crw/clean-room/README.md @@ -29,8 +29,8 @@ renumbered `01`–`05` in reading order. - `logs/table_sel{0,1,2}.json`: **the frozen code tables**, the source of truth that `tools/gen_tables.py` renders into `../src/tables.rs`. `table_sel0_d60class.json` is the - revision-1 selector-0 table, kept for the record. These are the only logs that ship with the - crate: without them the frozen tables cannot be regenerated or audited. + revision-1 selector-0 table, kept for the record. These are the frozen code-table logs that ship + with the crate: without them the frozen tables cannot be regenerated or audited. - `logs/private-corpus.tsv`: names, sizes and SHA-256 of the maintainer's own files. No image content, metadata or thumbnails are recorded anywhere here. - `logs/private/` (gitignored): per-file dumps of the private files, which can hold personal diff --git a/crates/crw/src/lib.rs b/crates/crw/src/lib.rs index da12940e..33ad8009 100644 --- a/crates/crw/src/lib.rs +++ b/crates/crw/src/lib.rs @@ -647,13 +647,7 @@ fn build_lut(codes: &[(u16, u8, u8, u8)]) -> Vec { /// initialised to 512 and reset at every row start. A stream that ends early leaves the remaining /// samples at their chain values (the frozen stream-end rule). fn decode_entropy(entropy: &[u8], w: usize, h: usize, tables: &tables::SelTables) -> Result> { - // unstuff: an 0x00 after an 0xff is a stuffing byte - let mut d = Vec::with_capacity(entropy.len()); - let mut i = 0; - while let Some(&b) = entropy.get(i) { - d.push(b); - i += if b == 0xff && entropy.get(i + 1) == Some(&0x00) { 2 } else { 1 }; - } + let d = unstuff(entropy); let nbits = d.len().saturating_mul(8); let peek16 = |pos: usize| -> u16 { let (byte, sh) = (pos >> 3, pos & 7); @@ -664,65 +658,35 @@ fn decode_entropy(entropy: &[u8], w: usize, h: usize, tables: &tables::SelTables let luts = [build_lut(tables.first), build_lut(tables.main)]; let n = w.checked_mul(h).ok_or(Error::LimitExceeded("image too large"))?; - let mut dec = vec![0u16; n]; - let (mut ce, mut co): (i32, i32) = (512, 512); + let mut out = Chains { dec: vec![0u16; n], w, qi: 0, ce: 512, co: 512 }; let mut carry: i32 = 0; let mut pos = 0usize; - let mut qi = 0usize; - // emit one diff: row-start chain reset, parity chain update, 16-bit masked store - macro_rules! emit { - ($d:expr) => {{ - if qi % w == 0 { - ce = 512; - co = 512; - } - let c = if qi % 2 == 0 { &mut ce } else { &mut co }; - *c = c.wrapping_add($d); - if let Some(p) = dec.get_mut(qi) { - *p = (*c & 0xffff) as u16; - } - qi += 1; - }}; - } - 'frame: while qi < n { - let block_end = n.min((qi / 64 + 1) * 64); + while out.qi < n { + let block_end = n.min((out.qi / 64 + 1) * 64); let mut tok = 0u32; - while qi < block_end { + while out.qi < block_end { + // stream exhausted: every remaining sample keeps its chain value if pos >= nbits.saturating_sub(1) { - // stream exhausted: every remaining sample keeps its chain value - while qi < n { - emit!(0); - } - break 'frame; + out.fill(n); + break; } let e = luts[usize::from(tok > 0)].get(peek16(pos) as usize).copied().unwrap_or(0); if e == 0 { return Err(corrupt("undecodable entropy code")); } - let (len, rr, s) = ((e >> 16) as usize, ((e >> 8) & 0xff) as usize, e & 0xff); - if pos.saturating_add(len) > nbits { - while qi < n { - emit!(0); - } - break 'frame; + let (len, mut rr, s) = ((e >> 16) as usize, ((e >> 8) & 0xff) as usize, e & 0xff); + if pos.saturating_add(len).saturating_add(s as usize) > nbits { + out.fill(n); + break; } pos += len; - let mut rr = rr; - if s == 0 && rr == 0 && tok > 0 { + tok += 1; + if s == 0 && rr == 0 && tok > 1 { // end of block - while qi < block_end { - emit!(0); - } - tok += 1; + out.fill(block_end); continue; } let diff: i32 = if s > 0 { - if pos.saturating_add(s as usize) > nbits { - while qi < n { - emit!(0); - } - break 'frame; - } let m = (peek16(pos) >> (16 - s)) as i32; pos += s as usize; // JPEG magnitude rule @@ -730,32 +694,67 @@ fn decode_entropy(entropy: &[u8], w: usize, h: usize, tables: &tables::SelTables } else { 0 }; - let mut first = qi.is_multiple_of(64); + let mut first = out.qi.is_multiple_of(64); if first && rr > 0 { // the block's first sample covered by the skip: its diff is exactly the carry - emit!(carry); + out.emit(carry); rr -= 1; first = false; } - for _ in 0..rr { - if qi >= block_end { - break; - } - emit!(0); - } - if qi >= block_end { - tok += 1; + out.fill(block_end.min(out.qi + rr)); + if out.qi >= block_end { continue; } - let dd = if first { + if first { carry = carry.wrapping_add(diff); - carry + out.emit(carry); } else { - diff - }; - emit!(dd); - tok += 1; + out.emit(diff); + } + } + } + Ok(out.dec) +} + +/// Drop the stuffing byte: an 0x00 after an 0xff. +fn unstuff(entropy: &[u8]) -> Vec { + let mut d = Vec::with_capacity(entropy.len()); + let mut i = 0; + while let Some(&b) = entropy.get(i) { + d.push(b); + i += if b == 0xff && entropy.get(i + 1) == Some(&0x00) { 2 } else { 1 }; + } + d +} + +/// The decoded samples and the two parity DPCM chains that produce them. +struct Chains { + dec: Vec, + w: usize, + qi: usize, + ce: i32, + co: i32, +} + +impl Chains { + /// Emit one diff: row-start chain reset, parity chain update, 16-bit masked store. + fn emit(&mut self, d: i32) { + if self.qi.is_multiple_of(self.w) { + self.ce = 512; + self.co = 512; + } + let c = if self.qi.is_multiple_of(2) { &mut self.ce } else { &mut self.co }; + *c = c.wrapping_add(d); + if let Some(p) = self.dec.get_mut(self.qi) { + *p = (*c & 0xffff) as u16; + } + self.qi += 1; + } + + /// Emit zero diffs up to sample `end`. + fn fill(&mut self, end: usize) { + while self.qi < end { + self.emit(0); } } - Ok(dec) } From 8829439d684e1404cf650cfbc857116156e2dc64 Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Wed, 28 Oct 2026 22:19:44 -0400 Subject: [PATCH 09/10] crw: address the remaining CodeRabbit record findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 02 §4: the decoder refuses the G5 with Error::Unsupported; the embedded-JPEG fallback is the caller's, not the decoder's. - 04: the Auto-slot claim holds outside the white-balance preset captures. - The record README no longer says it holds everything in the order it happened: the derivation trail stays at the pinned LightCraft revision. - docs/clean-room.md allows that split for a derivation done in another repository, so the standard and the CRW record agree. --- crates/crw/clean-room/02-crw-format.md | 4 ++-- crates/crw/clean-room/04-color.md | 2 +- crates/crw/clean-room/README.md | 6 +++--- docs/clean-room.md | 4 ++++ 4 files changed, 10 insertions(+), 6 deletions(-) diff --git a/crates/crw/clean-room/02-crw-format.md b/crates/crw/clean-room/02-crw-format.md index 9d0304ad..e25510c9 100644 --- a/crates/crw/clean-room/02-crw-format.md +++ b/crates/crw/clean-room/02-crw-format.md @@ -99,8 +99,8 @@ which the Rust implementation may normalise, provided the verification table of - **G5 anomaly**: its entropy stream decodes end-to-end with the selector-1 tables, but values diverge beginning at sample 2056 (= 4 × 514), and its low-bits plane fits no tested layout (top rows mostly match, decaying to 0 %, no constant shift). The - decoder must detect the G5 (CanonModelID / make-model) and fall back to the embedded - JPEG until a follow-up derivation resolves it. + decoder detects the G5 by its make-model and returns `Error::Unsupported` until a + follow-up derivation resolves it; a caller may fall back to the embedded JPEG. - The exact meaning of each file's tail bytes (handled by the stream-end rule). - Big-endian (`MM`) CIFF files: no sample exists. - PowerShot G2/G3/Pro1/Pro70/S30–S70: no CC0 samples (licence-excluded); decodable diff --git a/crates/crw/clean-room/04-color.md b/crates/crw/clean-room/04-color.md index 39088859..c5fa4c64 100644 --- a/crates/crw/clean-room/04-color.md +++ b/crates/crw/clean-room/04-color.md @@ -75,7 +75,7 @@ vs −0.67 GRBG). Unsampled PowerShots default to RGGB, unverified. (γ-model error), so the slot is chosen by metadata, not by fit. - **Slot selection**: ShotInfo (0x102a) index 7 is the WhiteBalance setting (ExifTool prose: 0 Auto, 1 Daylight, 2 Cloudy, 3 Tungsten, 4 Fluorescent, 5 Flash, 6 Custom, 8 Shade, - 9 Kelvin). Every corpus file reads 0 (Auto) → slot 0. + 9 Kelvin). Every corpus file outside the preset captures below reads 0 (Auto) → slot 0. - **D60 preset mapping verified** (2026-10-14): the maintainer shot one overcast scene at each setting, Auto, Daylight, Cloudy, Tungsten, Fluorescent, Flash and Custom, ISO 100, within a minute (all seven decode sample-exact). Slots 1–6 hold the same multipliers in diff --git a/crates/crw/clean-room/README.md b/crates/crw/clean-room/README.md index 4c2325ef..eb5d1620 100644 --- a/crates/crw/clean-room/README.md +++ b/crates/crw/clean-room/README.md @@ -1,9 +1,9 @@ # Canon CRW clean-room derivation record -Everything that establishes how `craft-crw` decodes a CRW, in the order it happened. The record was +What `craft-crw` decodes a CRW from, and the evidence that it is right. The full record was produced in LightCraft (`clean-room/crw/`, branch `clean-room-crw-format-decode`, revision -`6ca439b5155820d2befeacd81ec0c6a80b2bd3bc`; the record is identical at `70c755e4`, where the decoder was ported) between 2026-10-12 and 2026-10-25 and moved here -**verbatim** when the decoder became this shared crate. The documents are the evidence: they are +`6ca439b5155820d2befeacd81ec0c6a80b2bd3bc`; the record is identical at `70c755e4`, where the decoder was ported) between 2026-10-12 and 2026-10-25; the documents below moved here +**verbatim** when the decoder became this shared crate, and the derivation trail stays at that revision. The documents are the evidence: they are not rewritten after the fact, so paths they mention are LightCraft's paths of that time (the mapping is below). diff --git a/docs/clean-room.md b/docs/clean-room.md index cc8c675f..c2705cf8 100644 --- a/docs/clean-room.md +++ b/docs/clean-room.md @@ -65,6 +65,10 @@ folder with: 8. **Table data** lives in the record as JSON and is rendered into Rust by a committed script; the generated file says so and is never edited by hand. +When the derivation was done in another repository, the crate's `clean-room/` may hold only items +3, 5, 6 and 8, provided it pins that repository, path and revision (in its README and in `NOTICE`) +for the starting assumptions, the plan and the derivation documents, as `craft-crw` does. + Scripts in the record use the Python standard library only and run under `python -I`. Local-only material (reference models, oracle binaries, copyrighted specification PDFs, dumps that may carry personal metadata) is named in the record with how to obtain it, and is gitignored. From 0ad3952f5e8b24eba0aa3643134991e18fec04c6 Mon Sep 17 00:00:00 2001 From: "Dylan M. Taylor" Date: Fri, 30 Oct 2026 22:40:07 -0400 Subject: [PATCH 10/10] docs(crw): declare table generator Python version --- crates/crw/clean-room/tools/gen_tables.py | 1 + 1 file changed, 1 insertion(+) diff --git a/crates/crw/clean-room/tools/gen_tables.py b/crates/crw/clean-room/tools/gen_tables.py index dc8c0ca4..43bde633 100644 --- a/crates/crw/clean-room/tools/gen_tables.py +++ b/crates/crw/clean-room/tools/gen_tables.py @@ -1,3 +1,4 @@ +#!/usr/bin/env python3 """Regenerate crates/crw/src/tables.rs from the frozen tables in crates/crw/clean-room/logs/table_sel{0,1,2}.json. Run from anywhere: `python crates/crw/clean-room/tools/gen_tables.py`. The JSON files are the