Compare commits

..
Author SHA1 Message Date
Claude 1b5ba5eba7 Merge PR #397 2026-10-08 22:34:54 +00:00
Claude 8d79bf082e Batch fix: the header test's RLE PSD fixture uses PackBits runs of at most 128 bytes
The fixture wrote each row as one literal run of w bytes, which PackBits
can't express past 128 (a 160-pixel row's header byte read as a repeat
run). The old decoder clipped such rows silently; with the PSD fix in
this batch an overrun is an error, as it should be, and the fixture now
splits rows into valid runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 22:23:51 +00:00
Claude d316d97d80 Merge branch 'fix-psd-rle-alloc' into integ-b9
# Conflicts:
#	crates/codecs/src/psd.rs
2026-10-08 22:21:56 +00:00
Claude 2ab6e973c6 Merge branch 'fix-367-header-probe' into integ-b9 2026-10-08 22:21:37 +00:00
BFlat c395789026 Merge pull request #398 from storytold/claude/integrate-batch-8
Integrate #391 and #393 (undecodable raws import as preview only), with a work budget for the JPEG scan
2026-10-09 05:21:21 +07:00
Claude 6aa599f869 Fix: PSD RLE decode bounds its allocations by the input
A tiny hostile PSD could abort the process on out-of-memory in
`psd::decode` (allocation failure doesn't unwind, so catch_unwind
can't save it):

1. The RLE row byte-count table (channels x rows entries, from the
   header) was reserved with Vec::with_capacity before checking that
   the file holds it: 56 channels x 2^30 rows asked for ~480 GB from a
   ~40-byte file. The table is now taken as a slice of the input with
   checked size math and read lazily, so nothing is allocated from it.

2. RLE rows were zero-padded / run-expanded to the declared width with
   no tie to the input size, so all-zero row counts on a 1 Gpx (the
   DecodeOptions::max_pixels limit, already enforced by check_size)
   32-bit image expanded to 4 GiB per channel. A PackBits row expands
   at most 64x (2-byte repeat run -> 128 bytes), so each decoded row
   must now declare at least ceil(row_bytes / 128) * 2 encoded bytes,
   and the decoded channels' data must be present in the file, before
   any plane is allocated. Decoded planes are thus bounded by 64x the
   file size as well as by max_pixels. PackBits now decodes into a
   pre-sized, zeroed row and a run that would write past the declared
   width is an error instead of being silently clipped.

Also: size math (width x height x bpc, samples) is checked; image
buffers are allocated with try_reserve_exact (TooLarge on failure);
only the colour (+ alpha) planes are decoded, extra channels are
skipped; the PSB layer-section length no longer truncates on 32-bit
targets; interleaving uses iterators instead of indexing.

Valid PSDs (raw and RLE, 8/16/32-bit) decode identically; regression
tests cover a huge RLE header with a truncated table (aborted with a
481 GB allocation before), all-empty RLE rows, counts larger than the
file, a repeat run past the row width (was Ok before) and a 16-bit
gray RLE file with an extra channel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 22:19:42 +00:00
Claude f6f5e6f1b7 Part of #367: probes read image headers instead of decoding the pixels
The import probe of a non-raw image decoded it (at a 64 px fit, which for
JPEG still entropy-decodes every block and for PNG inflates everything)
only to learn its stored size and orientation: ~95 % of import CPU.

lightcraft-codecs gets `read_header(bytes) -> Header { format, width,
height, orientation }`: JPEG from its markers (EXIF orientation), PNG from
its chunks (IHDR, eXIf anywhere), TIFF from the first IFD (tiff crate,
no strips read), WebP from its RIFF chunks / bitstream header (image-webp),
PSD from its header and image resources. GIF, BMP and JPEG XL keep the
small decode. `files::probe_bytes` uses it; raw handling is unchanged.
The TIFF colour-layout check is shared with the decoder (`layout`).

Measured (release, 4 cores, throwaway benchmark, per file, bytes in
memory): 200 generated 4000x3000 JPEGs (3.6 MB): 72.7 ms -> 2.1 ms
(read_header alone 1.05 ms; the rest is the content hash and metadata);
20 PNGs (30.8 MB): 236-265 ms -> 9.0-9.6 ms.

Corrupt files (behaviour note): the header readers also refuse what a
decode refuses before the pixels (unsupported layout, compression, frame
type, bad IHDR, ...) and truncated files: a JPEG whose markers don't reach
its EOI, a PNG cut before the chunk after its IDAT run, TIFF strips/tiles
that start past the end, a WebP image chunk or PSD merged data past the
end. So truncated files (interrupted copies, partial downloads) are still
marked `error` in the import review, as before; tests check that whenever
the old decode accepted a (cut or byte-mutated) file the header agrees.
Changed: damage *inside* compressed data that is all present (flipped or
garbled bytes) was refused at scan and is now imported, and shows as
unreadable ("!" in the grid) when rendered. Also: tiny JPEGs (< 128 px)
were decoded leniently by the old probe, so a truncated one imported; it
is now refused. A TIFF truncated inside its last strip is not detected
(StripByteCounts are often overstated, so only strip starts are checked).

Tests: header readers vs decode on generated JPEG/PNG/TIFF/WebP/PSD/BMP
with and without EXIF/TIFF orientation; truncations and random mutations
agree with decode; hostile headers (zero/absurd sizes, bad lengths,
truncation, garbage) are errors; read_header added to the fuzz tests;
probe_bytes oriented dims match the decode-based values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 22:09:30 +00:00
radialmonster 7094ba1b6e LR-IMP-FORMATS: CR3 C-RAW samples that overshoot the bit depth are clamped
C-RAW is quantised wavelet coding: next to clipped highlights its
reconstruction overshoots the sensor range by a few quantisation steps.
On 17 CC0 CR3 files (SX70 HS, G5 X Mark II, M6 Mark II, EOS R, RP, 90D,
M200, R5 Mark II, R6 Mark II, R6 Mark III, R8) samples reached 16384 to
16419 at 14 bits, never below 0, and decoding failed with "decoded
sample outside its bit depth". Lossless files of the same bodies clip at
exactly 16383. Wavelet samples are now clamped to the CMP1 bit depth;
lossless samples outside it are still corrupt data.

The other 93 CR3 files have no sample outside the range and render
pixel-identically. Render cache version 13 -> 14.
2026-10-08 18:04:01 -04:00
Claude cb0e0cb2e5 Follow-up to #393: the embedded-JPEG scan shares a work budget
scan_for_jpeg walks every JPEG-like start in a container it can't parse;
each walk runs through the marker segments that follow (up to 4096) and
their entropy data. A crafted CRW/MRW/X3F (or unsupported TIFF raw) of
fake starts one after another makes every walk cover most of the file:
quadratic, about a minute for 40 MB, on the import's probe thread. The
walks now report how far they read and share a budget of eight times the
file's size (at least 1 MiB); real files' candidates fail within a few
bytes or are the preview itself. Test: 200,000 fake starts, 0.15 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 22:03:26 +00:00
Claude 0fb3351c91 Merge PR #393 2026-10-08 22:00:34 +00:00
Claude b9254242a2 Merge PR #391 2026-10-08 22:00:10 +00:00
BFlat ab9e8d372c Merge pull request #394 from storytold/claude/integrate-batch-7
Integrate part of #374 (bounded folder walks, launch imports off the UI thread) and #385, #386, #387
2026-10-09 04:43:24 +07:00
Claude b11e5399c6 Merge PR #387 2026-10-08 21:26:34 +00:00
Claude 83058f717e Merge PR #386 2026-10-08 21:26:34 +00:00
Claude 7962b66a06 Merge PR #385 2026-10-08 21:26:33 +00:00
Claude c140f63372 Merge branch 'fix-374-startup-drive-walk' into integ-b7 2026-10-08 21:19:42 +00:00
Claude 790a341776 Part of #374: launch imports run off the UI thread; folder walks are bounded and never take a drive root or home implicitly
Root cause (in 0.2.1 and still on main): files and folders named on the
command line were imported from the eframe app-creator closure with
`app.run("library.import", …)`, which runs the engine command synchronously:
`import::expand` walked the folder recursively (one `is_dir()` open per entry,
no depth or count bound, following links) and probed every file before the
first frame could be drawn. An argument that resolves to a drive root (`.` or
`..` with a launcher's working folder at `C:\`, `/` or `\` on Windows) or to
the home folder therefore walked the whole drive with the window stuck as a
black "not responding" box. The library-problem banner's pending import had
the same synchronous call. Every other recursive walk (Local's Include
Subfolders, Find Missing Photos, preset and LUT folders, MCP) was also
unbounded, which is the "Local folders with subfolders: not responding"
pattern of filmcraft#23, and a link loop recursed until the stack overflowed.

- engine `walk::files_in`: one bounded, iterative walker (32 folder levels,
  1,000,000 entries; `Walked::truncated` says it stopped), folder or file
  taken from the directory listing (no open per entry), links and junctions
  to a folder followed once. `import::expand` (+ `expand_within`), preset and
  LUT folder expansion and MCP `expand_paths` use it.
- engine `import::launch_path`: an empty launch argument is ignored; a
  relative one is resolved against the working folder; a drive or file-system
  root becomes its memory card's DCIM folder or is refused, and so is the home
  folder (a notice says why and to use File > Import Photos).
- desktop: launch arguments go through it and are imported with
  `import::start_paths` (the import worker), like dropped files; so does the
  library-problem banner's pending import.

Not verified on Windows (no machine here). The report says PhotoCraft 0.5.0
and `--memory` hang the same way with no arguments; nothing in LightCraft
walks the disk at startup without arguments (strace of a headless demo run
with HOME=/ and cwd /: no recursive listing), so that part may be outside
this code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 21:19:42 +00:00
BFlat 67b7bc8bc6 Merge pull request #389 from storytold/claude/integrate-batch-6
Integrate #373 (keymap editor), #218 (Lightroom catalog import), #367 (parallel import copies), #344 (import file types)
2026-10-09 04:19:01 +07:00
Claude 41e67e1edd Batch fix: the CLI snapshot test settles before its screenshots
Under load (the full workspace suite) the photo grid was captured before
its thumbnails rendered, darker than the export dialog dimmed over them,
and the dimming comparison failed; it passes alone. Each screenshot now
follows ui.settle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:56:35 +00:00
Claude 5d437489b9 Merge branch 'fix-344-import-file-types' into integ-b6 2026-10-08 20:53:11 +00:00
Claude 1e6f0e28e1 Batch fix: the Lightroom catalog dialog filter goes through add_filter_nocase
#218's catalog picker called rfd's add_filter, which #362's guard test
(no_dialog_bypasses_the_helper) forbids: on Linux the filter would hide
CATALOG.LRCAT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:53:11 +00:00
Claude 4af709995f Fix #344: the import review checks or unchecks a file type at once
With several file types among the candidates, the Import Photos review
shows a toggle per type (ARW · 120, JPG · 120): unchecking JPG leaves
the JPEGs kept beside the raws out of the import, checking a type checks
its importable files (duplicates stay unchecked). JPEG/JPG and TIFF/TIF
are one type; a file whose probe named no format goes by its extension.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:52:35 +00:00
Claude 9b0d0fd05f Parity summary regenerated (cargo xtask parity --write)
Main's summary block had a duplicated pair of total lines left from an
earlier merge; the regenerated one counts #373 and #218's rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:50:04 +00:00
Claude b1d8ba4284 Merge branch 'fix-367-import-parallel' into integ-b6 2026-10-08 20:48:31 +00:00
Claude d8efa6a147 Spanish for the keymap editor (#373) and the Lightroom catalog import (#218)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:48:31 +00:00
Claude d4c0a7a3d4 Merge remote-tracking branch 'origin/main' into integ-b6
# Conflicts:
#	crates/ui-egui/locales/de.json
#	crates/ui-egui/locales/ja.json
#	crates/ui-egui/locales/pt-br.json
#	crates/ui-egui/locales/ru.json
#	crates/ui-egui/locales/zh-hans.json
#	crates/ui-egui/locales/zh-hant.json
2026-10-08 20:48:15 +00:00
BFlat 68e0827633 Merge pull request #383 from storytold/claude/test-376-masking-layout
Part of #372: painted-text layout tests from #376
2026-10-09 03:47:22 +07:00
ClaudeandElliot Graham cad024fbff Part of #372: painted-text layout tests from #376
The tests from @QuirkyTurtle94's #376, a duplicate fix for #372 opened
while #379 was in review: they read the painted text shapes (not the
registered rects) at 1x and 2x scale, in English, German and Brazilian
Portuguese, at panel widths 250-520, and check that tile labels fit
unelided inside their tiles, that a long mask name is elided before the
eye, and that a long Describe prompt keeps Select inside the panel.
They pass unchanged against main's implementation.

Co-Authored-By: Elliot Graham <elliotgraham94@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:47:07 +00:00
Claude 86deb5b004 Fix #367: import copies and verifies files on several threads
Import -> Copy (and Move, and Copy as DNG) handled one file after another
on the import worker: copy, fsync, read back and hash, optionally decode
and re-encode the raw as DNG. Every other CPU-heavy step of an import was
already parallel (the probes, up to 8 threads), so with the probes cached
by the review dialog's scan the transfer phase ran on one core.

ImportJob::prepare_files now plans each file in order, as before
(duplicates by path and content, sidecars, rename sequence numbers,
folders, Kept notes) and collects the copies/moves, which then run on up
to 8 threads (std::thread::scope; sequential on wasm32). Outcomes keep
their places, so results, order, names and failure messages are those of
a one-by-one import:
- a file whose content matches a transfer still pending, or whose name
  could collide with one (same folder, same name family: IMG.jpg,
  IMG-1.jpg, img.dng), waits until that one has finished;
- cancel stops before the next file; transfers not started leave no item;
- a panic in one transfer fails that file instead of the batch;
- Copy as DNG holds a memory-gate permit (6x the file size) while it runs.
Copy verification and crash safety are unchanged (same copy_new / place).
The desktop's import worker prepares batches of 2 x workers files (8..16,
lightcraft_engine::import::batch_size) instead of 8, so a batch's slowest
file holds back fewer threads; the browser build keeps 8.

Measured (4-core shared VM, 200 procedural 3.7 MB JPEGs = 754 MB, warm
page cache, throwaway benchmark not committed):
- probe: ~80 ms CPU per file is the JPEG decode (DCT-scaled to 64 px),
  1.2 ms the content hash; the dialog's scan: 4.4 s, 3.6 cores busy.
- Copy, probes cached: copy+fsync+verify 1.59 s at 0.43 cores before,
  0.94 s at 0.81 cores after. Copy, not cached: 6.45 s -> 5.51 s.
- Add, probes cached: 2 ms prepare; nothing to parallelize.
- commit of a batch on the UI thread (journal append + fsync): ~4 ms,
  flat for libraries of 0, 5k, 20k and 50k photos (~0.7 s for 1300).
Copy as DNG could not be measured (no raw corpus here); it is the most
CPU-heavy per-file step and was sequential.

Test: copies_run_side_by_side_with_the_same_outcome (copies under the
test fault Slow run on more than one thread; names, order and in-batch
duplicates match a one-by-one copy; the outcome assertions also pass on
the old code, the thread assertion fails there).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:47:04 +00:00
BFlat a673b2bb7a Merge pull request #382 from storytold/claude/i18n-371-spanish
Fix #371: Spanish interface
2026-10-09 03:41:48 +07:00
Claude 305ff71474 Part of #371: Spanish for the SAM 3 install dialog (#366)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
(cherry picked from commit 73b22c8524ef17b209ff2fffbb16c248524e2e96)
2026-10-08 20:41:31 +00:00
Claude a0e49629d3 Merge remote-tracking branch 'origin/main' into i18n-371-spanish 2026-10-08 20:41:31 +00:00
BFlat c8f697a128 Merge pull request #381 from storytold/claude/fix-366-sam3-install-help
Fix #366: the SAM 3 dialog without a download location shows how to install by hand
2026-10-09 03:41:25 +07:00
BFlat 232cbace63 Merge pull request #379 from storytold/claude/fix-372-mask-labels
Fix #372: masking tile labels and long mask names stay inside their boxes
2026-10-09 03:41:22 +07:00
radialmonster 1fa3af8740 LRC-LIB-LIBFOLDERS: the sidebar rename test builds its folder ids with forward slashes on Windows
The folder tree writes paths with forward slashes and its widget ids
carry them; the test put the temp folder in the id with backslashes.
Found running cargo xtask ci on Windows.
2026-10-08 16:41:12 -04:00
radialmonster 8e4d7545ff M11.4: the X-T20 corpus RAF is fitted since the away-from-edges retry; update the test and the doc that said it was rejected
Found running cargo xtask ci with the corpus on Windows. The two changes
landed on the same day: the Fujifilm test (#282) expected the compressed
X-T20 file to keep the neutral fallback because its fit failed the gates
on all pixels (held-out RMS 0.114 against 0.10). The retry away from
edges (#263, issue #232) fits it (0.056), so the fit is accepted at both
sizes. Expect true.
2026-10-08 16:41:12 -04:00
radialmonster 1028ed6b88 LRC-LIB-LIBFOLDERS: a chosen folder that follows a move keeps its names' case and the platform's separators
follow_library_folder rebuilt the chosen subfolder from the lower-cased
folder_key on Windows and joined it with '/', so after a move the folder
shown in the sidebar came back as c:\archive\trip/day1 with its names in
lower case. Add folder_rest (compares by folder_key, returns the names as
spelled) and join them with the platform's separator. Tests compare
folders by identity where they only mean the same folder, and a new test
fails on Windows without the fix.
2026-10-08 16:41:11 -04:00
Claude efe7cad73c Follow-up to #218: Lightroom catalog import translated
The progress, result and menu strings of the Lightroom catalog import in
the ja, pt-br, zh-hans, zh-hant, de and ru catalogs. The formats catalogs
held English copies of the seven new format messages; they are
translated, and German and Russian (which the PR predates) get them too,
so the formats build check passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:36:22 +00:00
Claude a86defd1b9 Merge PR #218 2026-10-08 20:35:39 +00:00
radialmonster 881bc85dac LR-IMP-FORMATS: import CRW, MRW, X3F, KDC, MOS, ERF and IIQ files from folders as preview only
Folder import, drag and drop and MCP folder expansion skipped these extensions, so
the preview-only fallback for these containers was only reachable by naming a file.
Adds exactly the containers that now import as preview only to import::EXTENSIONS and
the headless PHOTO_EXTENSIONS. 3FR, FFF, DCR and STI still fail and stay out.
2026-10-08 16:32:17 -04:00
radialmonster dbe73f0f9c LR-IMP-FORMATS: raws we cannot decode import as preview only when any usable JPEG is embedded
Widens the preview-only fallback. A recognised raw whose decode fails now falls back
to its embedded JPEG for any decode error (unsupported, corrupt, limit, TIFF
structure), not only an unsupported variant, and the preview search covers more
containers:
- Canon CRW, Minolta MRW and Sigma X3F are recognised by their magic (RawFormat
  Crw, Mrw, X3f) instead of failing as "RawOther files are not supported yet".
- A TIFF whose IFD0 has no ImageWidth/ImageLength but a JPEG pointer or SubIFDs
  (Kodak KDC) is a raw container, not a malformed image. TIFF compression 99
  (Leaf MOS) counts as a raw compression.
- When no tag points to a preview, a recognised raw we cannot decode is scanned
  for the largest complete three-component DCT JPEG. A preview whose first byte
  was overwritten (Minolta, Epson) is restored. A one-component JPEG is skipped
  (a PowerShot CRW stores its mosaic that way).
Files with no JPEG at all still fail with "without an embedded preview".
2026-10-08 16:32:17 -04:00
radialmonster 0b8c7a237a LR-IMP-FORMATS: TIFF raws with a thumbnail-only IFD0 import as preview only, not as a tiny image
Phase One / Leaf IIQ, Canon EOS-1D / 1Ds and Kodak DCS TIFFs keep their raw data in
a private block. IFD0 is only a 160-640 px RGB copy and no IFD is marked as raw, so
they opened as an ordinary TIFF of that size (a 150 MP file rendered 296x220) with
no preview-only label.

probe() now recognises such a file from its structure (OtherTiff): a picture at
least 4x larger than IFD0 in both directions is declared by a SubIFDs child, or by
Exif PixelXDimension/PixelYDimension together with enough bytes outside every IFD's
strips and tiles to hold it at one bit per pixel. decode() reports the sizes as the
reason. A raw we cannot decode now falls back to its embedded JPEG, else to the TIFF
reader's view of its first image, so it imports preview-only with that reason.
RENDER_CACHE_VERSION 12: these files render differently.
2026-10-08 16:31:49 -04:00
Claude 43aa795539 Follow-up to #373: translated keymap editor, contextual keys noted
- The keymap editor's strings, the fixed-key labels and the two new
  command labels in the ja, pt-br, zh-hans, zh-hant, de and ru catalogs;
  'Restore {}' and 'Removed from {}' go through tr_format!.
- The Presets and Soft Proofing rows say what their key does in the
  Library grids (Flag as Pick and advance / expand or collapse the
  stack), which main's static Shortcuts list showed as a separate label.
- The macOS menu publishes no shortcuts while a key is being recorded
  (merged with main's owned_by_menu).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:24:13 +00:00
Claude 38cdd49274 Merge PR #373 2026-10-08 20:23:12 +00:00
Claude 0b587a0a8a Fix #371: Spanish interface
A Spanish (es) catalog and formats catalog covering the same messages as
the Brazilian Portuguese, Japanese and Simplified Chinese catalogs (1595
messages, 180 formats), an Edit > Language > Español command
(app.language.spanish), es tag tests (es, es-ES, es_MX.UTF-8, es-419),
docs/localization-es.md with the terminology used, and the language
lists in docs/localization.md, parity and What's New.

Instructions use infinitives or impersonal forms (no usted). Plural
endings follow the existing convention: the English "s" passes through
where Spanish adds one, else {:.0} swallows it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:20:17 +00:00
Claude 60a98354d9 Fix #366: the SAM 3 dialog without a download location shows how to install by hand
With no mirror configured the dialog was titled "Download the SAM 3
Model?", only said to put "the files" in the folder (see
docs/ai-masks.md) and offered Close. It is now titled "Install the SAM 3
Model", names the three files (model.safetensors, vocab.json,
merges.txt from facebook/sam3), and has a Show in Finder / Explorer /
File Manager button that creates and shows the model folder, plus a link
to the installation guide. Object and Describe work as soon as the files
are there.

The download location itself (a hosted mirror) is still a maintainer
decision: DEFAULT_MIRRORS stays empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:18:47 +00:00
Claude f16f413942 Fix #372: masking tile labels and long mask names stay inside their boxes
The Create New Mask tiles painted their (untranslated) labels with an
unconstrained Painter::text into tiles capped at 52 pt, so Background and
Luminance ran past their tiles, and translations would more so. The grid
now measures the translated labels: four 52 pt tiles when they fit (at
10.5 pt, or 9.5 pt before dropping a column), else three or two, a tile
growing past 52 pt for a long label; only in a panel too narrow for even
that is a label cut short (full text on hover).

A mask row's name is cut short with an ellipsis before the eye (full name
on hover). The Describe field places the translated Select button first
and gives the field what is left, instead of reserving a fixed 64 pt.
Status lines wrap explicitly.

The painted label and name rects are registered (maskNewLabel:*,
maskName:*), so the sidebar test checks the text, not just the widgets,
at four panel widths with a long name and a long prompt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:09:16 +00:00
BFlat 48cd9c5e3a Merge pull request #375 from storytold/claude/integrate-batch-4
Integrate #279 (Canon CR3 decoding) and #362 (file dialog filters)
2026-10-09 03:08:11 +07:00
parkerHurstandClaude Opus 5.5 e2b626b7f7 LR-PREF-SHORTCUTS: editable keyboard shortcuts in Help ▸ Keyboard Shortcuts (⌘/)
The ⌘/ dialog is now a keymap editor: click a shortcut and press the new
keys (Esc cancels), × removes it, ↺ restores the default, Reset All clears
every change. A key another command had moves to the new command, with a
toast. Changes go through the new app.setShortcut / app.resetShortcuts
commands into AppSettings.keymap (saved in ui.json) and show up in the
menus, the native macOS menu and the control channel's command list.

Modifier keys alone never record (holding ⌘ waits for the key it
modifies), ⌘C/⌘X/⌘V are recorded from the copy/cut/paste events, and
saved entries that don't parse fall back to the default shortcut.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:54:50 -05:00
Claude 28db7b200b Merge fixes for #279: CR3 joins RAF in the format lists, render cache version 13
Main already had Fujifilm RAF in relative WB, the file-local look, the
docs and the corpus lists; CR3 is added next to it. RENDER_CACHE_VERSION
was already 12 on main (from #255/#258/#282), so CR3 thumbnails rendered
by main from embedded previews would not be redone: 13.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 19:44:32 +00:00
Claude 3f63db0a0b Merge PR #279
# Conflicts:
#	README.md
#	ROADMAP.md
#	crates/catalog/src/model.rs
#	crates/engine/src/camera_preview.rs
#	crates/raw/src/lib.rs
#	crates/raw/tests/corpus.rs
#	docs/camera-preview-colour.md
#	docs/parity.md
#	docs/whats-new.md
2026-10-08 19:44:21 +00:00
Claude f611e76d0e Merge PR #362
# Conflicts:
#	apps/lightcraft/src/main.rs
#	docs/parity.md
2026-10-08 19:43:31 +00:00
BFlat eeb136b9ee Merge pull request #370 from storytold/claude/fix-260-startup-failure-dialog
Part of #260: a desktop start that fails says why, in a message box naming the log
2026-10-09 02:43:05 +07:00
BFlat 3f10ba4f3b Merge pull request #369 from storytold/claude/fix-171-unlocked-library-warning
Fix #171: a library that can't be locked opens with a warning the user sees
2026-10-09 02:43:01 +07:00
Claude 7fb24029f2 Part of #260: a desktop start that fails says why, in a message box naming the log
When eframe could not open the window (no usable graphics device, a
broken driver), main returned the error to standard error, which a
Windows release build (no console) or an app started from Finder never
shows: the app just vanished. The error now goes to the log (the file
#286 added) and into a native message box, in the user's language, that
names the log file. The saved UI language is applied before the window
starts so the message uses it. Scripts and tests (LIGHTCRAFT_NO_PREFS)
get no dialog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 19:30:26 +00:00
Claude 5601d7879a Fix #171: a library that can't be locked opens with a warning the user sees
When catalog.lock can't be locked (a file system without locks, such as
some network shares), the library opens unprotected, as documented, but
the promised warning went only to the log. It is now the first of the
library's warnings: library.info lists it under settingsWarnings, the
app shows it once in its notices, and lightcraft-cli run / mcp print the
library's warnings on stderr (stdout stays results / protocol only). A
normally locked library shows none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 19:27:32 +00:00
Adrian D'souza 095c7dce87 fix sqlite test payload extension 2026-10-09 00:57:19 +05:30
Иван Лапочкин f96f732baa M11.1: add CR3 decoding
Decode lossless CRX and version 0x100/0x200 C-RAW in portable Rust,
including sensor geometry, timed white balance and black/white levels.
Verify all sensor samples against six independent M50/R100/R8 references.

Any CR3 decode error falls back to the embedded JPEG, as CR3 files did
before this decoder. The full sensor buffer is allocated only after the
first plane decodes, and plane buffers grow row by row. Tile planes
decode in parallel with a windowed bit reader (24 MP: 424 -> 89 ms
lossless, 519 -> 138 ms C-RAW on an M4).

Add guarded CR3 camera-JPEG colour fitting with independent framing
validation, relative white balance and cache invalidation.

Validation: cargo xtask ci.
2026-10-08 22:22:13 +03:00
BFlat bf224c2a8d Merge pull request #361 from storytold/claude/fix-319-craft-fonts-pin
Fix #319: release builds pin craft-fonts with Noto Sans CJK SC (includes #203)
2026-10-09 02:19:59 +07:00
BFlat 16236ff779 Merge pull request #364 from storytold/claude/fix-293-undo-shows-photo
Fix #293: undo and redo show the photo they change
2026-10-09 02:19:56 +07:00
BFlat 2592ab47e9 Merge pull request #365 from storytold/claude/fix-sam-busy-after-reply
AI masks: the worker stops counting a job as pending before it replies
2026-10-09 02:19:49 +07:00
Claude 3e791905b9 Merge remote-tracking branch 'origin/main' into claude/fix-293-undo-shows-photo
# Conflicts:
#	crates/engine/src/tests.rs
2026-10-08 19:19:32 +00:00
Claude 2897f85861 Merge remote-tracking branch 'origin/main' into claude/fix-319-craft-fonts-pin 2026-10-08 19:19:27 +00:00
Claude 3517606ffc Merge remote-tracking branch 'origin/main' into claude/fix-sam-busy-after-reply 2026-10-08 19:19:27 +00:00
BFlat 9e51ca0da8 Merge pull request #354 from storytold/claude/fix-341-date-keyword-rows
Fix #341: By Date and Keywords rows show their photos from All Photos
2026-10-09 02:19:21 +07:00
Claude 85e38c7342 Merge remote-tracking branch 'origin/main' into claude/fix-341-date-keyword-rows
# Conflicts:
#	crates/ui-egui/src/tests_panels.rs
2026-10-08 19:19:15 +00:00
BFlat 4836143f9e Merge pull request #353 from storytold/claude/fix-316-section-eye
Fix #316: the section eye switches Light, Color and Detail off, and can be clicked
2026-10-09 02:19:02 +07:00
Claude 57dc811a6a Merge remote-tracking branch 'origin/main' into claude/fix-316-section-eye
# Conflicts:
#	crates/ui-egui/src/headless.rs
2026-10-08 19:18:55 +00:00
BFlat 9e327b4772 Merge pull request #352 from storytold/claude/fix-322-type-slider-values
Fix #322: type an exact value into a slider
2026-10-09 02:18:43 +07:00
Claude f6fcef4d5c Merge remote-tracking branch 'origin/main' into claude/fix-322-type-slider-values 2026-10-08 19:18:31 +00:00
BFlat 3895cf6411 Merge pull request #363: community PRs, batch 3 (German + Russian UI, Fujifilm compressed RAF, raw fixes)
Integrate community PRs, batch 3: German + Russian UI, Fujifilm compressed RAF, raw fixes (stacked on #350)
2026-10-09 02:18:21 +07:00
BFlat 5cb74e7313 Merge pull request #350: community PRs, batch 2 (14 PRs + follow-ups)
Integrate community PRs, batch 2 (14 PRs + follow-ups; stacked on #349)
2026-10-09 02:18:09 +07:00
BFlat 4165904bd1 Merge pull request #349: community fixes, batch 1 (18 PRs + follow-ups)
Integrate community fixes, batch 1 (18 PRs + follow-ups)
2026-10-09 02:17:58 +07:00
Claude 6ea9b979c8 AI masks: the worker stops counting a job as pending before it replies
The pending counter was decremented when the job's guard dropped at the
end of the loop iteration, after the reply was sent, so a waiting caller
(CLI / MCP mask.add) could see the segmenter still busy right after its
answer arrived. Under a loaded test run that made
tests_segment::waiting_requests_with_a_damaged_model_are_errors fail on
assert!(!busy()). The guard is now dropped just before each reply.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 19:02:44 +00:00
Claude e73d88a55f Merge PR #358
# Conflicts:
#	docs/parity.md
2026-10-08 18:57:38 +00:00
Claude d56d265044 Merge PR #282
# Conflicts:
#	ROADMAP.md
#	assets/ATTRIBUTION.md
#	crates/engine/src/camera_preview.rs
#	crates/raw/tests/corpus.rs
#	docs/camera-preview-colour.md
#	docs/parity.md
2026-10-08 18:55:07 +00:00
Adrian D'souza b4a1166f4b PR218: document optional-data and recovery warnings 2026-10-09 00:17:24 +05:30
Adrian D'souzaandCodex 5413c8bb3a PR218: make archive scale test exercise staging
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:13:49 +05:30
Claude c07395222f Merge PR #356
# Conflicts:
#	docs/parity.md
2026-10-08 18:43:38 +00:00
Claude 7b082660e7 Merge PR #339 2026-10-08 18:43:35 +00:00
Claude 3a9b6ccee4 Fix #293: undo and redo show the photo they change
After an undo or redo that changed one photo (its edit, rating, flag,
label, metadata or versions), that photo becomes the active one, as in
Lightroom Classic, so the change is on screen instead of happening to a
photo out of sight. It joins the selection if it isn't in it; steps that
touch several photos, albums or files, and photos outside the current
view, leave the selection alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:43:18 +00:00
Adrian D'souzaandCodex 07d9678209 PR218: cover optional payload cap warning
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:12:39 +05:30
Claude 8980c3bb0e Fix #341: clippy (no single-element loop in the test)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:42:11 +00:00
Adrian D'souzaandCodex 3c76902bbb PR218: cover unsupported schema entries
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:12:06 +05:30
Claude 1ebce934ea Fix #322: clippy (collapsible if)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:42:00 +00:00
Adrian D'souzaandCodex 7cf3719c63 PR218: localize complete import messages & scale prepared path
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:11:29 +05:30
Claude c3dce49c2e Format the gap-report test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:41:28 +00:00
Claude 3af89e6ed5 i18n: translate what #265 made translatable, and the formats gaps #264/#265 left
#265 wrapped about 460 more UI strings (command and control labels, rule
operators, built-in preset names, status messages) in tr()/tr_format!, and
filled the 54 new formats messages of the other catalogs with English
copies. These are now translated in ja, zh-hans, zh-hant and pt-br; the
Russian formats catalog gets the 58 messages it needs to build, and German
the strings batches 1-2 added. Release-note lines stay English outside
German.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:40:42 +00:00
Claude a7fcd81850 Fix #341: What's new
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:40:19 +00:00
Claude c3c30dae3f Fix #341: the test clicks the row's widget id
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:39:36 +00:00
Claude 47cf1ade6f Fix #341: the test window is tall enough to show the By Date rows
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:38:10 +00:00
Claude 00388f3fd2 Fix #341: test from Recently Deleted (the demo has picks)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:37:25 +00:00
Claude 1b4518c202 Fix #341: By Date and Keywords rows show their photos from All Photos
The rows count every photo in the library, but choosing one filtered
whatever source was open (an album, a folder, Picks), so it often showed
nothing. Choosing a row now switches to All Photos first; clearing it
leaves the source alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:36:25 +00:00
Adrian D'souzaandCodex d16ab1f238 PR218: check SQLite local payload span
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:52 +05:30
Adrian D'souzaandCodex bbcfdf085b PR218: harden catalog import review fixes
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex 1e6035d9db test: encode SQLite fixture serial types & overflow offsets correctly
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex 37cfc46e0d PR218: place prepared-import regression tests after production items
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex b256b8b635 PR218: bound egui worker message size & simplify cancellation
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex e8c817ddea PR218: satisfy current CI clippy without changing import behavior
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex d367d83ee6 LRC-LIB-CATALOG-IMPORT: exercise actual concurrent develop edit
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex 6a05f865ee LRC-LIB-CATALOG-IMPORT: refresh rebased parity summary
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex 866d02e577 LRC-LIB-CATALOG-IMPORT: address SQLite, worker & archive review
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Orthic LabsandCodex 4a427894ed LRC-LIB-CATALOG-IMPORT: import Lightroom Classic catalogs
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Claude aaa46003d5 Fix #316: the header keeps its hover shade while the pointer is on the eye
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:34:35 +00:00
Claude a9cd761d52 Fix #316: the section eye stays while the pointer is on it
The eye was drawn only while the header itself was hovered; with the
pointer on the eye, egui reports the eye (on top) as hovered and not the
header, so the eye vanished under the pointer and could not be clicked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:33:25 +00:00
Claude 68267810b3 Fix #316: headless test of the section eye
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:30:43 +00:00
Claude 9b4ba1aa80 Fix #316: the section eye switches off Light, Color and Detail too
The pipeline honoured a section's eye only where it applies Effects,
Optics, Geometry and Calibration, so turning the eye off on Light, Color
or Detail changed nothing. DevelopSettings::effective() resets the
switched-off sections to their defaults (Light with the tone curve; Color
with white balance, mixers, grading and point colour; Detail), and both the
CPU and GPU renderers render those. Borrowed, so free, when every section
is on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:29:29 +00:00
Claude 0634782fbc i18n: German and Russian follow-ups (#265, #264)
- #265's German coverage test failed whenever any catalog key, command or
  control label, or line of What's New lacked German, so every feature PR
  would have had to ship German. It reports those gaps for every language
  now (display_label_gaps_are_reported), as the project's other gap test
  does; docs/localization.md says how release-note lines are matched and
  that an English copy in a formats catalog marks an untranslated message.
- Russian: docs/localization-ru.md, the language lists in
  docs/localization.md and docs/parity.md, ru tag tests.
- What's new: German and Russian UI. Removes a status sentence #265 put
  in parity.md's Top gaps list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:26:56 +00:00
Claude 9b5e771fd8 Merge PR #264
# Conflicts:
#	crates/ui-egui/src/i18n.rs
#	crates/ui-egui/src/menus.rs
2026-10-08 18:25:10 +00:00
Claude 69384ad430 Merge PR #265
# Conflicts:
#	crates/ui-egui/locales/ja-formats.json
#	crates/ui-egui/locales/zh-hans-formats.json
#	crates/ui-egui/locales/zh-hant-formats.json
#	crates/ui-egui/src/lib.rs
2026-10-08 18:25:01 +00:00
Claude acc85c4147 Fix #322: the value field draws in a child UI and leaves the slider rows in place
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:22:43 +00:00
radialmonster e2d35de867 LR-IMP-ADD-DIALOG: file dialogs match extensions in any letter case on Linux (#342)
rfd turns each extension into a *.ext glob that GTK and the desktop portal
match case-sensitively, so DSC_1234.NEF was hidden. One helper
(dialog_filter) gives every dialog filter the lower, UPPER and bracket
forms on Linux; the Add Photos filter uses import::EXTENSIONS.
2026-10-08 14:21:58 -04:00
Claude 0fbf20db51 Fix #322: select the old value once the field has the focus
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:21:10 +00:00
Claude 2b66eb4800 Fix #322: the typed-value field takes the keyboard on the frames after the click
The click's own release took the focus straight back from a field created
in that frame, so typing went nowhere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:19:31 +00:00
Claude 92c7237c50 Fix #322: type an exact value into a slider
Click the value next to a slider's name to edit it: the old value starts
selected, Return or clicking away applies the typed number on the
control's steps and range as one undo step (like a keyboard nudge), and
Esc or text that isn't a number keeps the old value. A decimal comma,
a leading + and a % or K unit are accepted. The keys typed go to the
field, not to the shortcuts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:05:19 +00:00
radialmonster 34ecea4f24 LR-IMP-CAMERA-COVERAGE: read Canon ColorData stored as bytes, so PowerShot and EOS M CR2s get their as-shot white balance
Maker note tag 0x4001 is UNDEFINED bytes (ColorData versions -3 and -4) on
PowerShot and EOS M bodies. It was read as one number per byte, so no plausible
as-shot levels were found and the white balance was 1, 1, 1 (green cast, a* about
-13 on neutrals). Byte and undefined data is now paired into 16-bit words in the
maker note's byte order before the existing probe (word 71). No per-model table.

RENDER_CACHE_VERSION 11 -> 12.
2026-10-08 14:03:23 -04:00
radialmonster ff8d027766 LR-IMP-FORMATS: NEF lossy type 2 curve of version 0x44 0x40 spans a quarter of the sample range
Maker note 0x0096 of the D780, D6, Z 5, Z 6, Z 7 and Z 7_2 starts 0x44 0x40
and holds 257 points, but its first interval rises by 2^bits / 1024 where the
0x44 0x20 tables rise by 2^bits / 256: the curve is the identity in the dark
part and covers 2^bits / 4 codes, not 2^bits. The decoder spaced the points
2^bits / 256 codes apart for both, so these files decoded to a fraction of
their values, far below the black level.

parse_table now takes the span from the version (0x20: all codes, 0x40: a
quarter). Other versions decode to identical samples. A known-answer test
built from hand-coded table bytes and bit strings fails without the change.
Bumps RENDER_CACHE_VERSION to 12.
2026-10-08 14:02:01 -04:00
Claude 4fb8ad4dcd Format the #333 merge fix
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:01:00 +00:00
Claude dc67d348e6 Merge fixes: #333 hover label id, #346's test boxes the import dialog (#325), unused mut
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:00:16 +00:00
Claude 7f386aa258 Merge fixes: close the test functions the #307/#325 and #325/#346 merges ran together
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:58:41 +00:00
Claude 399aedd23a Docs: What's new for the import review size and Shift-click ranges, CLI logging (#346, #347, #340)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:57:54 +00:00
Claude b5fcf58e55 Merge PR #347
# Conflicts:
#	crates/ui-egui/src/import.rs
2026-10-08 17:57:23 +00:00
Claude 261994acbc Merge PR #346
# Conflicts:
#	crates/ui-egui/src/headless.rs
2026-10-08 17:57:02 +00:00
Claude b936c1a683 Merge PR #340 (CLI logging; the desktop keeps #286's file logger) 2026-10-08 17:56:31 +00:00
Claude cab6af8608 Fetch: say where the bearer token goes (#259)
The token_env comment said the token goes to the first mirror's host; it
goes to every configured mirror's own host (never to a redirect's).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:55:31 +00:00
Claude 4eb9b565e0 Profiles: imported LUT names shown as named, own ids for their folders (#333)
#333 passed imported LUT names and folder names through tr(), so a LUT
named like a message ("Vivid") was shown translated; they are now shown
as the file names them (builtin_label / new profile_label, as for user
presets). A LUT folder named like a built-in group ("Film") shared that
group's menu widget id and the browser section's open state; both are
keyed apart now. Restores the "Amount 0 = the photo as it was" assertion
#333 replaced, and adds a What's new line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:54:50 +00:00
Claude ccdd9e45d1 Merge PR #333 2026-10-08 17:52:46 +00:00
Claude 24c800f46f Merge PR #297 2026-10-08 17:52:46 +00:00
Claude e471e7c346 Merge PR #263 2026-10-08 17:52:46 +00:00
Claude aaec05dd11 Merge PR #262 2026-10-08 17:52:46 +00:00
Claude d808749792 Desktop: a --memory session writes no log file (#286 with #288)
#288 made --memory sessions write nothing (issue #164); #286's log file
would have been the exception. It is not opened there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:52:46 +00:00
Claude c03a67c59f Merge PR #286
# Conflicts:
#	apps/lightcraft/src/main.rs
#	docs/whats-new.md
2026-10-08 17:51:24 +00:00
Claude cd8d2b96a2 Merge PR #259 2026-10-08 17:50:50 +00:00
Claude a12fabd588 Library: the Photo Grid label footer is translucent; pt-br for "Color label cleared" (#308)
#308 painted an opaque label-coloured bar over the bottom 24 px of every
labelled photo in Photo Grid, hiding part of the photo, while its docs
said the photo pixels stay as they are. The footer is now translucent
over the photo and the docs say so. Also adds the one string pt-br lacked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:39:23 +00:00
Claude 99fa19099b Merge PR #308
# Conflicts:
#	apps/lightcraft/src/native_menu.rs
#	crates/ui-egui/src/panels/detail.rs
#	docs/parity.md
2026-10-08 17:38:20 +00:00
Claude bac4666e7f Docs: What's new for the crop lock, trackpad zoom/pan, Folders and Select All (#312, #313, #307, #325)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:37:27 +00:00
Claude b7f60a1fac i18n: translate the strings #312, #313 and #325 added (ja, pt-br, zh-hans, zh-hant)
Crop Apply / Lock / Locked, Set Image Zoom and Pan, the import review's
Recently Deleted choices and Empty Recently Deleted. The import's undo
labels and its "restored from Recently Deleted" summary used format!, so
they could not be translated; they go through tr_format! now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:37:06 +00:00
MarekSpodBiedry 4d506494d9 Issue #337: the Import Photos dialog is bigger and resizable; the photo grid fills it
The review dialog was a fixed-size window whose thumbnail grid was capped at 330 points (about 6 x 2.5 thumbnails). It now opens at 960 x 720, can be resized by its edges and corners, and the grid takes whatever height the options and buttons leave. The dialog has a window id of its own so its size does not carry over to the other dialogs.
2026-10-08 19:35:08 +02:00
Claude 90fe6d1616 Merge PR #325
# Conflicts:
#	crates/ui-egui/src/panels/detail.rs
#	crates/ui-egui/src/tests_panels.rs
2026-10-08 17:34:31 +00:00
Claude fa630a233a Merge PR #307
# Conflicts:
#	docs/parity.md
2026-10-08 17:34:04 +00:00
Claude 9e9e45b3c7 Merge PR #313
# Conflicts:
#	crates/ui-egui/src/panels/detail.rs
#	docs/parity.md
2026-10-08 17:33:58 +00:00
Claude 259de3d9df Merge PR #312 2026-10-08 17:33:04 +00:00
Claude 8b70d33d0f Fix #319: release builds pin craft-fonts with Noto Sans CJK SC
release.yml (five jobs) and freebsd.yml pinned craft-fonts abb83316, from
before Noto Sans CJK SC landed there, so v0.4.0 embedded only the Japanese
faces and most of the Simplified Chinese UI was tofu. The pin is now
8dcdacd5 (craft-fonts HEAD: Noto Sans CJK SC and Noto Sans Arabic), which
the Noto CJK exception merged from #203 allows. docs/releasing.md says when
to bump the pin and how to check it before a release.

flake.nix / flake.lock (community-maintained, not in CI) still pin the old
revision: the lock's narHash needs nix to recompute.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:32:09 +00:00
Claude 5e2ece0cf3 Merge PR #203 (Noto CJK allowed via craft-fonts) 2026-10-08 17:31:36 +00:00
ClaudeandHanmo 53584512f4 Issue #188: Tint sign documented as landed (DNG / Lightroom sign)
#258 left the Tint row at 🟡 and docs/tint-direction.md as a review note.
The sign now matches the slider's track, the DNG specification and
Lightroom XMP, so the row is back to ✅; the doc says what happens to
saved custom tints (they shift the other way; nothing is rewritten) and
What's new tells users.

Co-Authored-By: Hanmo <letsgo@88.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:29:47 +00:00
Claude e4737f68c7 Format: rustfmt the Show in Explorer button (#324)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:29:18 +00:00
Claude 21de86b3df UI: the loupe follows a texture's aspect only when it really differs (#280)
A texture's whole-pixel size is slightly off the frame's exact aspect (a
128 px thumbnail by up to ~1 %), so following it always made the loupe
rect jump a little as the thumbnail, preview and full render arrived, and
moved mask and crop overlays with it. Follow it only when it differs by
more than 2 % (an unsupported raw's embedded JPEG with another crop), and
never in the crop tool, whose overlays need the frame's own geometry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:29:00 +00:00
Claude 0b101be155 Issue #181: saved presets and Export with Previous drop keys this version doesn't know
#287 made app.export refuse unknown params. Stored settings (a user export
preset, prefs.json lastExport) written by another version are not the
caller's typo, and refusing them would break that preset or Export with
Previous for good after an update: their unknown keys (also inside
watermark) are dropped before the strict check. What the caller passes is
still checked strictly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:29:00 +00:00
ClaudeandDWAA1660 7d2bb5e5ec Issue #188: D65 reads as a small positive tint (test from #336)
Daylight sits above the Planckian locus, so with the DNG / Lightroom tint
sign D65 converts to about +10, as Lightroom's Daylight preset has it.
Taken from #336, which made the same sign fix as #258.

Co-Authored-By: DWAA1660 <80738438+DWAA1660@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:26:01 +00:00
MarekSpodBiedry 3a9fe13d3a Import: Shift-click checks or unchecks a range of photos in the review (#338) 2026-10-08 19:23:43 +02:00
Claude 2565f893c4 Merge PR #257
# Conflicts:
#	docs/parity.md
2026-10-08 17:21:02 +00:00
Claude 4e2584e559 Merge PR #277
# Conflicts:
#	docs/parity.md
2026-10-08 17:21:02 +00:00
Claude cb173c22dc Merge PR #311
# Conflicts:
#	docs/parity.md
2026-10-08 17:20:41 +00:00
Claude 98da2a0229 Merge PR #288
# Conflicts:
#	docs/whats-new.md
2026-10-08 17:19:31 +00:00
Claude 1ece9341d6 Merge PR #324 2026-10-08 17:18:59 +00:00
Claude 6d52e0744a Merge PR #280 2026-10-08 17:18:59 +00:00
Claude dd783b1d38 Merge PR #267 2026-10-08 17:18:59 +00:00
Claude 81e28f25c9 Merge PR #255 2026-10-08 17:18:59 +00:00
Claude b5b6f8caa8 Merge PR #300 2026-10-08 17:18:58 +00:00
Claude 7e21d185b8 Merge PR #258 2026-10-08 17:18:58 +00:00
Claude 70db2f2320 Merge PR #292 2026-10-08 17:18:58 +00:00
Claude 74b9129a05 Merge PR #287 2026-10-08 17:18:58 +00:00
Claude 36c6920f55 Merge PR #285 2026-10-08 17:18:58 +00:00
Claude 6d22d1235b Merge PR #284 2026-10-08 17:18:58 +00:00
Claude 5f9548a7ba Merge PR #269 2026-10-08 17:18:57 +00:00
Claude 0e188f6a40 Merge PR #268 2026-10-08 17:18:57 +00:00
Claude a66eebf778 Merge PR #261 2026-10-08 17:18:57 +00:00
Claude e3dc820d89 Merge PR #289 2026-10-08 17:18:57 +00:00
DWAA1660andClaude Opus 5.5 a6cedcc9e8 Issue #168: the desktop app and CLI log warnings to stderr; RUST_LOG works
lightcraft-cli installed no `log` backend, so every log::warn!/info! record
(the GPU-to-CPU fallback, an unknown LIGHTCRAFT_GPU_BACKEND, the adapter
chosen) was dropped and RUST_LOG, which docs/gpu-pipeline.md documents for
the fallback warning, did nothing. The desktop app had its own logger that
read only LIGHTCRAFT_LOG.

Both now install lightcraft_engine::logging: warnings and errors on stderr
by default; LIGHTCRAFT_LOG or RUST_LOG (off, error, warn, info, debug,
trace) picks another level, LIGHTCRAFT_LOG winning, and RUST_LOG's
per-target directives (`lightcraft=debug,wgpu=warn`) set LightCraft's
level. Below warnings only LightCraft's own records are shown, as the app's
logger did. The app's --help lists the variable.

Tests: level parsing and target filtering (engine logging::tests); the CLI
prints "LIGHTCRAFT_GPU_BACKEND=bogus names no known backend" with
RUST_LOG=warn and nothing with RUST_LOG=off (cli.rs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K3vVft4VXvvy8SpbbUceTW
2026-10-08 17:14:04 +00:00
MarekSpodBiedry f2bcc9219c Issue #234: Windows opens links (Map button) in the browser, not File Explorer
explorer <url> opened File Explorer for links with a query or fragment, such as the OpenStreetMap link behind the Map button. Windows now opens https links with rundll32 url.dll,FileProtocolHandler, which hands the URL to the default browser without a shell.
2026-10-08 18:59:04 +02:00
peiyan0 493600743d engine, ui: support imported LUT profiles in profile dropdown and browser (#328) 2026-10-09 00:43:19 +08:00
Xus BadiaandClaude Opus 5.5 e4ab8422b4 macOS DMG: branded window background and icon layout
The DMG opened as a plain Finder window. It now shows the app icon's figure on
the LightCraft colour, with LightCraft.app and the Applications link side by side.
Same approach as VectorCraft's (storytold/vectorcraft#493).

- packaging/macos/dmg/: background.svg (source; text outlined, the figure an
  <image> of assets/app-icon), background.tiff (1x + 2x HiDPI, Deflate, sRGB)
  and dmg-layout.DS_Store (Finder view settings: window, icon size 128,
  positions, background).
- generate.py writes the TIFF and the layout on any OS, byte-for-byte
  reproducible, with nothing from the machine that ran it (no paths, users,
  disk names or volume UUIDs).
- package.sh copies them into the staging folder; the image is still built with
  hdiutil makehybrid. Nothing extra runs in the signing job.
- The volume name drops the version: the layout finds the background through
  an alias that includes the volume name. The DMG file name keeps the version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 17:55:32 +02:00
Nathaniel LedfordandClaude Sonnet 5.5 0b5ef3fe11 Import review: choose what happens to files in Recently Deleted (#298)
Files whose photo is in the trash stay unchecked until Restore them or Import as
new is chosen; the toast and undo label say what was restored.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:39:28 -04:00
peiyan0 38d2ff07c9 desktop: fix Windows Explorer reveal path and platform file manager labels (#272) 2026-10-08 23:36:56 +08:00
Nathaniel LedfordandClaude Sonnet 5.5 6c19090fde Grid: selected photos whose thumbnail isn't drawn yet still show as selected (#298)
The grid and filmstrip now both ask Selection::state_of.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:29:05 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 5b80fc6bcf MCP import onDeleted; Empty Recently Deleted in the trash view's Photo menu; parity row
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:29:05 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 0a474cc06e Permanent delete of several photos is one consistent op list; restored photos join album/selection
Review of the onDeleted work found that purging photos one at a time built each
op list against the unchanged catalog, so photos sharing an album or a stack made
Empty Recently Deleted (and Fresh imports) fail with NoStack or leave dangling
album entries. Catalog::delete_photos_permanently_ops builds one list for all of
them; Empty, Delete Permanently and the Fresh purge use it. Photos an import
restores now take part in album/albumName/selection, and undo labels say Restore.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:29:05 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 2928880e81 Import: onDeleted policy (skip|restore|fresh) for files in Recently Deleted; library.emptyRecentlyDeleted
A file that is in Recently Deleted was always skipped as a duplicate with no way
forward (#298). The report now flags such duplicates (existingDeleted); onDeleted
restores the trashed photo with its edits or replaces it with a fresh import; and
Empty Recently Deleted removes every trashed photo in one undoable step.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:16:42 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 14a03052de Filmstrip: show every selected photo as selected, not only the active one (#298)
After Select All in the loupe only the active filmstrip cell was highlighted,
so the selection looked like a single photo. Cells now use Selection::state_of;
selected-but-not-active cells get the selected fill and a grey outline.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:01:50 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 7f6e4e48f7 Selection::state_of: one answer to active / selected / not selected for every view
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:01:50 -04:00
sebastian pfluegelmeier 912f56cac8 LR-VIEW-ZOOM: add trackpad zoom and pan with stable cursors 2026-10-08 16:38:56 +02:00
Nathaniel LedfordandClaude Sonnet 5.5 0e254bfbc6 Crop UI: lock toggle, custom ratio, edge-safe handle drags (#295)
The crop overlay now reports pointer positions to crop.drag instead of
computing rectangles itself. The Crop panel gets a visible Lock toggle, a
custom W × H entry, a dropdown chevron and a button label that names the
active ratio ("Original", a preset, or 1.37 : 1) instead of pixel sizes.
LR-CROP-RECT / LR-CROP-ASPECT parity rows updated.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 10:35:46 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 8d8178dc0a Engine: crop.drag command; validate custom crop ratios (#295)
crop.drag {handle, from, to, start?} runs drag_crop through the command
layer so the UI, CLI, control channel and MCP agree; a locked crop keeps the
shape of its frame. crop.aspect now rejects non-finite, non-positive and
beyond-20:1 ratios and non-numeric arrays instead of storing a degenerate
lock that produced an empty crop.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 10:35:46 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 90c63af951 Crop: drag_crop domain operation for handle drags (#295)
Dragging a crop handle past the image edge used to hand the raw rectangle to
CropGeometry::constrained, which slid the whole frame toward the centre and
moved the anchored corner. drag_crop owns the drag instead: the opposite
edges stay put, a locked ratio holds on all eight handles (edges grow about
their midline, sliding along a blocked side), the frame stops at the image
edge and slides along it, and it keeps a minimum size. Hostile numbers return
the start crop. Covered by Given/When/Then scenarios and property tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 10:35:46 -04:00
midasdfandCodex 69640df8a7 LR-EXP-WATERMARK: shape Japanese vertical watermark glyphs
Co-Authored-By: Codex <noreply@openai.com>
2026-10-08 23:34:43 +09:00
Hanmo a47afdc1f4 M1: merge upstream main and resolve documentation conflicts 2026-10-08 22:18:13 +08:00
Stano BustorandGPT-6 Luna b4a192a54e UI: preserve preview aspect ratio across views
Keep embedded and decoded preview proportions consistent in the justified
grid, photo views, alternate detail previews, and culling previews.

Co-Authored-By: GPT-6 Luna <noreply@openai.com>
2026-10-08 16:00:00 +02:00
Hanmo 4256865cf4 M1: tint labelled thumbnails and colour label feedback 2026-10-08 21:42:21 +08:00
Hanmo 57c9e252c1 M1: show bottom confirmation for colour label actions 2026-10-08 21:25:02 +08:00
Nathaniel LedfordandClaude Sonnet 5.5 46c4a2709d Sidebar: names end in an ellipsis at the count; bars end at the panel edge
A name that does not fit is cut with an ellipsis before the photo count (the
full name on hover) instead of running past it; scrolling the sidebar sideways
or widening it shows more. A row asks for room for at most 140 pt of its name,
so depth, not one long name, makes the sidebar scroll. Selection and hover
bars end at the panel's edge when the content is wider.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 09:06:18 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 198286e589 Folders: round-4 review fixes (counts at the edge, edge-space names, smart albums)
- ui: counts stay at the visible edge when the sidebar scrolls sideways
  (with the row's colour behind them where a long name runs under); the
  smart-album dialog's live count includes the album's folder
- catalog: a folder path is used as given (a name with a space at its edge is
  not its neighbour: Remove acted on the wrong folder); a plain POSIX path
  is checked against the folder without building a key (about half the cost
  on a big library, with a test that it agrees with the careful check)
- engine: editing a smart album's rules keeps the folder the editor cannot
  show; a shown folder that loses its last photo gives way to All Photos;
  library.state and view.json name the folder only while it is shown; a blank
  folder filter is no filter

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 09:06:18 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 6f33aacd22 Sidebar: scrolls sideways so deep folders keep their nesting
Every level indents one step more than its parent however deep it goes (the
five-level indent cap is gone). The whole sidebar scrolls in both directions:
rows report how wide they need to be, the widest one of the last frame sets
the content width (at least the panel's), and a bar appears only when it
overflows. Folder names are no longer cut in the tree; Create Album stays at
the visible edge.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 09:06:18 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 5d13d59520 Folders: review fixes (smart albums, disk guards, deep chains, long titles)
- engine: a smart album made from a folder view keeps the folder; only a
  folder that shows exactly its own photos can be the source (not the
  startup disk, not a folder above disks, not one with no photos); a
  folder source and a folder filter never coexist; a folder source with no
  folder shows nothing; a saved view whose folder is gone opens on All
  Photos; removing a folder above a disk needs `disk: true` even with one
  disk mounted; library.folders documents volume/selectable
- ui: deep chains stop indenting after five levels, the grid title gives
  way to the photo count (full name on hover), disclosure triangles have
  spoken names, the remove dialog says what is included and has a disk
  title; test temp folders are cleaned up on failure

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 09:06:18 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 0732edc64f Folders: every folder its own node; choosing one loads it like an album
- catalog: a row is exactly one folder (no folding of single-child levels);
  the UI starts a folder that holds nothing and leads to one folder open
- engine: a library folder is a source (LibrarySource::LibraryFolder with
  Session::library_folder, like Local's browse): library.source
  {kind: libraryFolder, path}, kept with the library's view, shown by
  library.state, followed through folder.rename / folder.move and undo,
  dropped back to All Photos when its folder is removed
- ui: clicking a folder replaces whatever the grid showed (Picks, Recently
  Deleted, a Local folder…) instead of narrowing it with a filter chip; the
  grid is titled after the folder

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 09:06:18 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 82e827b9d9 Sidebar: Folders section for the folders the library's photos were imported from
Another way to explore the library next to By Date and Keywords: a tree of
the folders that hold imported photos, with counts (subfolders included),
under the disk or network share each is on. Local stays the way to browse
any folder on disk.

- catalog: Catalog::folder_tree and Filter::library_folder. Each disk is a
  volume row (drive letter, Windows share, /Volumes/<share>, /media, /mnt,
  gvfs, /net…), found from the stored paths alone, so an offline share costs
  nothing. Levels that hold nothing and lead to one folder fold into one row;
  depth is bounded. The tree and the filter read paths through one splitter
  (separators, `.`, `..`, verbatim UNC prefix, repeated separators), and a
  row that would cover other disks' photos is not choosable
- engine: library.folders, library.removeFolder (to Recently Deleted, one
  undo step, no file touched; a whole disk or a folder spanning several
  needs `disk: true`, the startup disk never), libraryFolder in
  library.filter, chip; a chosen folder follows folder.rename / folder.move
  and their undo
- ui: collapsible Folders section; the chosen folder's row is always opened
  into view; right-click (row or triangle) Rename, Move, Show in Finder,
  Remove from Library…; disk rows Remove Disk from Library…; both ask first

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 09:06:18 -04:00
issuenateandClaude Opus 5.5 88c1e768f1 TIFF: decode 1-, 2- and 4-bit samples
Bilevel scans, CCITT fax images and 2- or 4-bit gray TIFFs were rejected with
"sample buffer too short". The `tiff` crate returns such samples packed, most
significant bits first, eight, four or two to a byte with each row padded to a
whole byte; the converter expects one byte per sample. They are now spread out
one per byte and scaled to 0–255 (255, 85 and 17 times the sample: exact) before
conversion. 3-, 5-, 6- and 7-bit samples, and planar images with more than one
sample narrower than a byte, are refused with an "unsupported" error.

The new test builds 1-bit (both photometric conventions, a width that leaves row
padding, set padding bits), 2-bit and 4-bit strips in code, plus a truncated
strip that must fail cleanly; it fails without the fix with the old error. A
600×400 bilevel image, uncompressed and CCITT G4, now decodes to the same pixels
PIL reads from it.

The module documentation no longer claims palette support: the `tiff` crate
refuses RGBPalette images on every read path, so the palette branch here has
never been reached. The parity row lists palette TIFF as missing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:47:51 +02:00
issuenateandClaude Opus 5.5 cc425d0cea TIFF: invert WhiteIsZero gray once, not twice
A WhiteIsZero (PhotometricInterpretation 0) gray TIFF opened as a negative at
every bit depth. The `tiff` crate (0.11) already inverts those samples as it
reads them, for every layout it accepts, and refuses the WhiteIsZero layouts it
can't invert; the decoder here then inverted them a second time. WhiteIsZero is
common in scanner and fax output, and CCITT fax images use it by definition.

The second inversion is removed. The new test builds 8- and 16-bit WhiteIsZero
and BlackIsZero strips in code and fails without the fix (the lowest sample
came out black instead of white).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:26:27 +02:00
HanmoandGardy 6277af55bf M1: restore library culling shortcuts and Classic shift actions
Keep macOS plain-key ratings, labels and flags with egui, adapting the
native menu ownership correction from PR #261 (4e012a5, Gardy).
Add Shift+6-9 and contextual grid Shift+P, recognize shifted digit
punctuation, and document the verified Lightroom Classic mapping.

Cover batch changes and undo, exactly-once advance, Compare targets,
text and crop context, physical number keys and native menu ownership.
Verified native demo/control screenshots and all seven cargo xtask ci gates.

Co-authored-by: Gardy <armandpro50@gmail.com>
2026-10-08 20:25:37 +08:00
radialmonster 30f9eb3674 LR-PROF-CAMERACOLOR: list the #232 entry after the other RAW decoding entries 2026-10-08 08:04:42 -04:00
wstockandClaude Opus 5.5 7177b9b9f3 Issue #187: the filmstrip highlights every selected photo, not only the active one
The filmstrip painted selection chrome only for the active photo, so after
Select All or a ⌘/Shift-click multi-selection the other selected photos
looked unselected, although the selection itself was right.

Paint it as the grid does: every selected photo gets the selected cell
fill; the active one keeps its white frame and the others get the grid's
grey one.

Test: tests_scroll::filmstrip_shows_every_selected_photo samples a cell's
background in the headless render after Select All, after selecting one
photo, and after a toggle (what a ⌘-click runs); before the fix a selected
photo that was not the active one was painted like an unselected one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:55:57 +00:00
wstockandClaude Opus 5.5 c60482f753 Issue #266: filmstrip labels cut file names on characters, not bytes
The detail view's filmstrip shortened a long file name with `&name[..13]`,
a byte slice. For a name with multi-byte characters byte 13 can fall
inside a character (`202407層三限定訂閱圖(4)`: inside '限'), and the
slice panicked the UI thread as soon as that photo scrolled into the
strip, so the app quit.

`film_label` now counts characters: more than 14 are cut to 13 and an
ellipsis, as before for ASCII names.

Tests: a unit test of `film_label` (CJK, exactly 14 characters, ASCII,
empty) and a headless filmstrip test with the name from the issue; both
panicked with the reported message before the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:39:52 +00:00
wstockandClaude Fable 5.1 c1861e5416 Issues #164, #169: --memory sessions write nothing — no ui.json, no settings folder
`--help`, the module doc and the README promise that a `--memory` session
"writes nothing", and the desktop app broke that in two places:

- `PrefsWriter` saved `ui.json` whenever the UI state changed (issue #164),
  so a throwaway session's `thumbSize`, window layout or language replaced
  the user's saved settings — the README showcase script does exactly that.
- The GPU crash sentinel from #136 armed itself in every session except
  `LIGHTCRAFT_NO_PREFS` (issue #169); `with_init_marker` creates the
  settings folder to hold `gpu-init.marker`, removes the marker and leaves
  the folder, so a `--memory` run on a fresh home left an empty
  `Library/Application Support/LightCraft` behind.

Now `PrefsWriter` carries its target path (`None` for `--memory`,
`LIGHTCRAFT_NO_PREFS` or no home folder) and writes nothing without one; it
also writes nothing while the session is the one offered when the library
can't be opened (the question window and Continue Without Saving, which
`docs/control-protocol.md` already describes as writing nothing) and
resumes when `app.openLibrary` ends it. `gpu_crash_check(in_memory)` neither
arms the sentinel nor removes a marker in a `--memory` session; it still
reads one (new `read_init_marker`), so GPU rendering starts off there too
and the next ordinary launch reports and clears it. A `--memory` session
still reads `ui.json`, but a damaged one is reported and left in place
rather than set aside. Normal launches are unchanged.

Tests: `prefs_writer_writes_nothing_in_a_memory_or_temporary_session`,
`memory_session_leaves_the_gpu_marker_and_folder_alone`,
`memory_session_leaves_a_damaged_ui_json_alone` (apps/lightcraft), and
`read_init_marker` coverage in the gpu crate's marker test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 11:17:38 +00:00
wstockandClaude Fable 5.1 88d9b7196d Issues #181, #183: app.export refuses unknown params and bad values; watermark keys documented and range-checked
`ExportOptions::from_json` reads what it recognises and defaults the rest, so
`longEdgee=400` exported at 3000 px, `longEdge=banana` at full size, and a
`watermark` of `5` or `{"text":"x","bogus":1}` passed with `ok:true`, while
`size: 3` was silently clamped to 0.5 of the short edge (#181, #183).

`from_json` stays lenient for stored prefs and dialog prefill. New
`ExportOptions::validate(cmd, params)` / `from_params(params)` check every key
against the option and target lists (unknown ones name the nearest known key),
every value's kind and range, and a `watermark` object's keys, kinds and ranges
(`size` 0.005..0.5 and `inset` 0..0.4 of the short edge, `imageWidth` 0.01..1 of
the width, `opacity` 0..1, `color` [r,g,b]), returning
`invalid parameters for \`app.export\`: …` like the other commands. The three
places that run an export use it — the desktop `app.export`, the headless
backend behind `lightcraft-cli run` and the MCP `export` tool, and
`lightcraft-cli render --opt` — and `export.savePreset` validates a preset's
params when it is saved. Built-in presets, the Export dialog's `to_json`,
Export with Previous and `render --opt` all pass unchanged.

The `app.export` param string (`lightcraft-cli commands`, `engine.commands`),
the MCP `export` tool schema and docs/parity.md LR-EXP-WATERMARK now list the
watermark keys, their units and ranges.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 11:06:59 +00:00
Gert Pellin 47f40eb2d5 Desktop: write log records to a rotating log file as well as standard error
The desktop app's logger (StderrLog in main.rs) wrote to standard error
only. A launch from a desktop menu or the Dock has no terminal, so its
warnings were lost, and the panic log in the temp folder is gone after a
reboot: there was nothing to attach to a bug report.

apps/lightcraft/src/logging.rs replaces it (no new dependency):
- standard error plus logs/lightcraft.log in the settings folder, next
  to ui.json (Linux $XDG_CONFIG_HOME/lightcraft/logs, never the library);
  each launch rotates the previous log to lightcraft.1.log and .2.log, so
  the log of a run that crashed survives the next start; the file is
  capped at 16 MiB with one note
- records logged before the settings folder is known are buffered
  (bounded) and written once the file is attached
- default levels: info for lightcraft*, warn for everything else;
  LIGHTCRAFT_LOG=info|debug keeps its old meaning (a test compares it
  with the old logger) and wins over RUST_LOG, which otherwise replaces
  the default with env_logger-style directives
- never panics: an unusable folder or a failed write leaves standard
  error as the only sink; a poisoned lock is taken back
- runs with LIGHTCRAFT_NO_PREFS (tests, scripts) log to standard error
  only, so they don't rotate away the user's own logs
- a panic hook chained after the engine guard's adds escaped panics to
  the file (try_lock, so a panic while logging can't hang the app);
  lightcraft-panics.log in the temp folder is kept

The logger is installed first in main and the file is opened after
argument parsing (--version, --help and usage errors leave no file).
The start-up and control-server messages in main.rs and
control_server.rs go through it instead of eprintln!, and a --control
or LIGHTCRAFT_CONTROL_PORT value that isn't a port is now reported.
Documented in README (Quick start, Logs), --help and docs/whats-new.md.
2026-10-08 13:00:29 +02:00
wstockandClaude Fable 5.1 39e04ae7ab Issue #157: colour-range mask tolerance narrowed so a sample stops at its own colour
`tol = 0.04 + 0.16·refine` fully selected anything within 0.06 OkLab of a
sample at the default Refine 50. Through `tonemap_for_select` the report's
teal mid-grey and neutral light grey are only ~0.063 apart (L 0.60 vs 0.72,
a/b ≤ 0.02), so the second patch was 99 % selected and the mask covered the
whole photo; Refine 85 and 100 kept most of it too. Now
`0.015 + 0.06·refine` (full ≤ 0.0225 / gone ≥ 0.045 at Refine 50, gone ≥
0.066 at 85, 0.075 at 100), computed once in
`lightcraft_pipeline::masks::color_range_tolerance` and used by both the CPU
evaluator and the GPU `mask.wgsl` kind-3 parameter so the two keep agreeing.

Test: two flat patches at the report's sRGB values, sampled as
`mask.sampleColor` does; the sampled patch stays fully selected and the other
one is out at Refine 0, 50 and 85 and mostly out at 100.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 10:52:40 +00:00
wstockandClaude Fable 5.1 ee0b0ef67e Issue #166: snapshot --script exits non-zero when a scripted request fails
`fn snapshot` wrote every reply but never looked at `ok`, so a script with a
bad `ui.set`, an unknown command or a failed `ui.screenshot` exited 0 while
the PNG showed the wrong view or was never written. The run now counts
replies with `"ok": false`, keeps going (every reply still reaches stdout and
the final `-o` screenshot is still written — continue-and-report, like
`run --keep-going`) and returns `N scripted request(s) failed` afterwards,
which `main` turns into exit 1. Usage text and docs/control-protocol.md
"Headless rendering" state the contract.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 10:46:29 +00:00
sebastian pfluegelmeier b4ca9f7439 M11.4: harden RAF validation and camera profile fallback 2026-10-08 11:23:55 +02:00
Marek HubařandClaude Opus 5.5 169a4ed059 LR-IMP-FORMATS: ORF colour-filter layout from the file's Exif CFAPattern
Split out of #240 without the compressed decoder (crates/raw/src/vendor/orfc.rs
is not part of this change; compressed ORF stays preview-only).

- The layout is read from Exif CFAPattern (0xa302, repeat counts in either
  byte order) when it describes a 2x2 Bayer cell. BGGR and GBRG were not
  reachable before.
- Without a usable tag the green diagonal is still found from the samples,
  now from totals of 32x32 blocks instead of single-pixel differences, so
  fine texture no longer outweighs a small red/blue difference. The old
  version indexed the samples directly; this one reads them with get().
- probe_info no longer unpacks 12-bit packed ORFs that state their layout
  (XZ-2: under 0.1 ms; it ran the same unpack as a decode before, which
  takes 11 ms). 16-bit files still read their samples to find the bit depth.

Tests: stated layouts in both byte orders, unusable tags falling back to the
samples, the texture case, hostile areas for the fallback, probe_info equal
to decode; a corpus test comparing the E-1, E-400 and XZ-2 mosaics with
their embedded JPEGs under every layout.

No user-visible change yet for the bodies whose layout the old guess got
wrong (E-M1 and E-620 are BGGR): they write compressed ORF.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:02:06 +02:00
sebastian pfluegelmeier cfcc169ff1 M11.4: calibrate Fujifilm colour with bundled X-H2S and X-T4 profiles 2026-10-08 10:49:55 +02:00
sebastian pfluegelmeier 5923be31d6 M11.1: decode Fujifilm lossless and lossy compressed RAF 2026-10-08 10:44:09 +02:00
wstockandClaude Fable 5.1 f7c0b9e49f Issue #182: library.select and id queries reject a photo id that is not in the library
`select_photos {"ids": [9999]}` answered `selected: 1` and left the session
with no usable active photo; `develop.get {"id": 9999}` answered too.
`library.select` now checks every id (and `active`) against the catalog
before touching the selection and returns `no such photo N`, so the
previous selection and active photo stay in place; `photo_arg` (develop.get,
photo.inspect, photo.allMetadata, history.list) does the same for an
explicit `id`. Engine and MCP tests cover every select mode.

Fixes #182

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 07:45:41 +00:00
wstockandClaude Fable 5.1 b7e37a388d Issue #180: ICC profiles carry a fixed header date, so exports are byte-identical
`moxcms` stamps the ICC header's creation date/time with the wall clock on
every `encode()` (the writer uses `ColorDateTime::now()` and ignores the
profile's `creation_date_time` field), so two exports of one photo with
the same settings differed in the embedded profile and nowhere else — the
pixels matched, the hashes never did. `write_matrix_trc`, which every
embedded profile goes through, now zeros the header's date bytes (24..36)
after encoding, as other deterministic profile writers do. A test asserts
two writes of every named space are identical, the date bytes are zero and
the profile ID is empty.

Fixes #180

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 07:45:39 +00:00
Mohammed Munzir 87dd1c201a Web: prune the thumbnail cache to the configured budget (#245)
The browser accepted and persisted library.preferences cacheMb, but stored thumbnails stayed over that limit: the worker index pruned with the native default (DISK_CACHE_BYTES), so the setting only reached the desktop cache. The worker group now carries the session cache_bytes(), the app refreshes it every frame, and the next stored thumbnail prunes to it.
2026-10-08 09:30:19 +03:00
MarcelandCodex 35dd973d91 M16: add German localization using the existing catalogs
Add German language selection and persisted locale support, complete German
label and format catalogs, and German release notes. Connect previously
untranslated UI text, preserve user names, and size translated buttons to
fit. Format cached import source facts in the active UI language.

Validate locale coverage, placeholder formats, language persistence,
German glyphs, button bounds and cached import language switching. Review
native and headless screenshots; run all seven cargo xtask ci gates.

Co-Authored-By: Codex (GPT-6) <noreply@openai.com>
2026-10-08 07:27:32 +02:00
Wondarkus 49868693d5 Add Russian language support in i18n.rs 2026-10-08 08:23:30 +04:00
Wondarkus 79fbb6b464 Add Russian language support in i18n.rs 2026-10-08 08:21:31 +04:00
Wondarkus 95b57d3786 Add Russian language option to menus 2026-10-08 08:20:17 +04:00
Wondarkus 782ade6c39 Add files via upload 2026-10-08 08:16:54 +04:00
Wondarkus 11d6676984 Delete crates/ui-egui/locales/ru-formats.json 2026-10-08 08:15:15 +04:00
Wondarkus 841850d516 Delete crates/ui-egui/locales/ru.json 2026-10-08 08:14:58 +04:00
radialmonster 15feb6e1fc LR-PROF-CAMERACOLOR: fit the camera look away from edges when a lens-corrected JPEG misaligns (#232)
Sony A7R II camera JPEGs are distortion-corrected by default while the raw is not. On the public
CC0 ILCE-7RM2 samples the JPEG is up to 15 px of 1440 off, and the misaligned window frames alone
pushed the held-out RMS to 0.108-0.115 (gate 0.10): the photo opened in the grey neutral fallback,
13 L* too dark. The same shot with distortion correction off fits at 0.059.

When the fit on all pixels fails the gates, try it once more, with the same gates, on pixels whose
3x3 neighbourhood in either proxy spans a luminance ratio of at most 3. Fits that already pass are
unchanged (29 other corpus ARW/NEF/RW2 files render pixel-identical). ILCE-7RM2 ΔE76 vs the camera
JPEG (registered) 16.6-16.9 -> 3.7-3.8; corpus GX80 19.1 -> 7.9. RENDER_CACHE_VERSION 11 -> 12.
2026-10-08 00:14:00 -04:00
radialmonster 03bc1469f9 LR-EDIT-OPTICS-PROFILE: apply the distortion correction Panasonic/Leica record in RW2/RWL (#256)
The RW2 reader reads PanasonicRaw tag 0x0119 (16 LE words) and expresses the
camera's own correction as the equivalent DNG WarpRectilinear in OpcodeList3,
so embedded_lens() turns it into the existing EmbeddedLens/EmbeddedWarp and
"Enable Profile Corrections" applies it like a DNG's warp (no new warp path;
Convert to DNG keeps it). Rule, from our own measurements against the
embedded camera JPEGs (clean-room; ExifTool tag names only):
corrected = rho*(1 + (w8 rho^2 + w4 rho^4 + w11 rho^6)/32768), rho = r/w12
(w12 = active-area half diagonal), output = corrected / (1 + w5/32768),
on when (w7 & 0xf) == 1. Chosen on 38 GX85 photos, held out on 35 photos
from 12 other bodies / 9 other lenses; corrected renders line up with the
camera JPEG within 0.75 px at 1440 px at the corners (up to 13 % of the
half diagonal before). Implausible values leave the photo uncorrected.

The loader now removes lens opcodes before the RW2 camera-look fit (its
binned proxy needs an empty OpcodeList3); Opcode::is_lens_correction()
replaces the duplicated matches. RENDER_CACHE_VERSION 11 -> 12.
2026-10-08 00:13:58 -04:00
Gardy 4e012a59a6 macOS menu: single-key shortcuts reach egui (E, C, ⇧P, ratings…)
On macOS every shortcut that appears in the menu bar was left to the native
menu, and the egui shortcut handler skipped those keys. AppKit only offers a
key press to the menu bar when it carries ⌘ or ⌃, or is a function key; a
plain key (E, C, H, M, I, K, D, 0–9, P, U, Y…) or one with only ⇧ / ⌥ (⇧P,
⇧V, ⌥Y…) goes to the window's winit view instead. So all of those did
nothing on macOS, while keys outside the menus (Space, G, X, ⌫) and ⌘
shortcuts worked.

`publish_shortcuts` now lists only the shortcuts the menu bar really receives
(`menu_delivers`: ⌘, ⌃ or a function key) in `native_shortcuts`. The others
keep their accelerator on the menu item, so the menu still shows the key, and
egui runs them.

Checked in a release build by pressing keys (not via the control channel,
which bypasses the native menu): C opens Crop & Rotate, E Edit, ⇧P Presets,
3 / 0 set and clear the rating; ⌘⇧H toggles the histogram once per press;
typing "c" in the search field types and leaves Crop closed. The same build
without this change ignored C, ⇧P and 3. ⌃ shortcuts (merges) are unchanged
and were not tried.

Regression test: native_menu::tests::single_key_shortcuts_stay_with_egui
(fails with the previous behaviour: "E must be left to egui").
2026-10-07 22:10:21 -06:00
Wondarkus 6c7405ed3e Fix Russian translations in ru.json 2026-10-08 07:06:23 +04:00
Daniel VoshartandClaude Opus 5.5 8dd3afce51 Fetch: validate caller-supplied names, sizes and hashes; FileSpec borrows its strings; download_url for an exact address
The fetch crate is about to serve downloads whose file names, sizes and addresses come from a user-editable list,
so it checks them before touching the disk: an ordinary, unique file name (no separators, drive or stream
syntax, control characters, trailing dots, Windows device names or .part suffix), a non-zero size within the
caller's bound, a well-formed SHA-256, bounded file, mirror and attempt counts, and URLs capped at 16 KiB.
FileSpec now borrows its strings (callers with static names are unchanged) so a name read from a file needs no
leaking. download_url fetches one file from an exact address, query string included, for signed links.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 22:59:22 -04:00
Daniel VoshartandClaude Sonnet 5.5 4b57d5dc5a Fetch: a bearer token is sent only when the caller names its variable
Options::token_env (default none) replaces the hard-coded LIGHTCRAFT_SAM3_TOKEN, so a download of one model can
never send another model's token to its host. SAM 3 names its variable through lightcraft_segment::fetch::options().
The decision (non-empty, https, the host the download began at) is a small function with its own test.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 22:59:01 -04:00
Wondarkus 6a34a1f8b8 Add files for further editing 2026-10-08 06:56:08 +04:00
radialmonsterandClaude Opus 5.5 fc4b054fa4 LR-IMP-FORMATS: RW2 format 8: invalidate cached thumbnails
The decoder change alters format-8 renders while the photo's content hash
and settings stay the same, so thumbnails rendered before it (edited
photos keep theirs on disk) would survive. Bump RENDER_CACHE_VERSION.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 22:52:45 -04:00
Hanmo d614dc72bd M3: isolate global Tint direction correction for compatibility review 2026-10-08 10:26:18 +08:00
Hanmo 024e983a7e M3: honour Nikon CropArea and embedded preview colour space 2026-10-08 10:16:44 +08:00
radialmonsterandClaude Opus 5.5 1ba0421a85 LR-IMP-FORMATS: RW2 format 8: decode each 2×2 cell column by column
Raw format 8 codes a cell top-left, bottom-left, top-right, bottom-right.
decode_strip placed the values in row order, which put the two green
samples of every RGGB cell on each other's sites: flat areas were right,
but every edge got a one-pixel colour zipper. Measured on a G9 II and the
corpus GH6, S5 II and S9: the Gb plane sat at (+0.5, -0.5) half-res cells
from Gr instead of the Bayer (-0.5, +0.5); with this change all four match.

The test encoder used the same order, so the round trip passed either way;
a new known-answer test hand-codes one cell and checks where it lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:58:09 -04:00
Brandon ThomasandClaude Opus 5.5 8148988eb8 Docs: allow Noto CJK (via craft-fonts) for Chinese text
Maintainer decision: Noto Sans/Serif CJK is Google-branded Source Han
(Adobe co-developed); nearly every OFL Chinese face derives from it, and the
Simplified/Traditional Chinese UI (#184, #198) uses Noto Sans CJK SC from
craft-fonts. Record it as a narrow, explicit exception to the Adobe-asset rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:16:43 -04:00
195 changed files with 32371 additions and 1405 deletions

No files matched your search

+1 -1
View File
@@ -31,7 +31,7 @@ jobs:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695
ref: 8dcdacd5153e64560d109541a47d806f26f048c0
path: craft-fonts
- name: Build and test in a FreeBSD VM
uses: vmactions/freebsd-vm@c46abacb49f09938ca4e1702d15d836285d694cc # v1
+5 -5
View File
@@ -77,7 +77,7 @@ jobs:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
@@ -135,7 +135,7 @@ jobs:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
@@ -192,7 +192,7 @@ jobs:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- name: Install build and packaging tools
run: |
@@ -287,7 +287,7 @@ jobs:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- name: Build and package in a FreeBSD VM (tar.gz)
# vmactions/freebsd-vm v1.5.9, pinned by commit (it runs our build).
@@ -336,7 +336,7 @@ jobs:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
+5 -2
View File
@@ -63,7 +63,10 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
- **Never use any iconography, image, artwork, font, sound or other asset from Adobe products** (no Lightroom/Creative Cloud icons, no screenshots, no presets/profiles/LUTs, no UI bitmaps — not even as a temporary placeholder or "reference copy"). Observing Adobe's UI to imitate *layout and behaviour* is allowed; copying or tracing its assets is not.
- **This includes Adobe's open-licensed assets**: no Source Sans/Serif/Code or Source Han fonts, no Adobe Fonts, no
Adobe-published icon sets, sample photos, colour profiles or LUTs — even when OFL/MIT. The UI font is Inter (OFL);
Japanese fonts come from craft-fonts (below).
Japanese and Chinese fonts come from craft-fonts (below). **One exception (maintainer decision, 2026-10-07): Noto
Sans/Serif CJK** (Google-branded, OFL, co-developed with Adobe as Source Han) is allowed via craft-fonts for Chinese
text, because nearly every OFL Chinese face derives from it. Use it unmodified under its OFL; this does not open the
door to Source Han under Adobe's name or to any other Adobe asset.
- Every asset in the repository must be one of: **our own original work** (e.g. icons drawn in code as vectors, procedurally generated demo photos), **public domain / CC0**, **Creative Commons** (CC-BY / CC-BY-SA with attribution honoured), **OFL** (fonts), or **permissive open-source** (MIT/Apache-2.0/BSD/ISC) — or contributed by a person who created the asset and licenses it openly.
- **Exception: `docs/brand/`.** The ArtCraft name, wordmark and logos there are ArtCraft Team trademarks, not open source and not covered by LightCraft's MIT OR Apache-2.0 licence (`LICENSE-MIT`, `LICENSE-APACHE`, `NOTICE`); their terms are in `docs/brand/LICENSE-brand.txt`. Use them only unmodified and never redraw, recolour or derive from them.
- **Every asset must have an entry in `assets/ATTRIBUTION.md`** (path, title, author/creator, source URL or "original work", licence, date added, modifications) and its licence text when required (e.g. `assets/fonts/OFL-*.txt`). Add the entry in the same commit as the asset. Assets without an attribution entry must not be committed.
@@ -72,7 +75,7 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
uses it as the optional build input `CRAFT_FONTS_DIR`: `git clone https://github.com/storytold/craft-fonts ../craft-fonts`
then `CRAFT_FONTS_DIR=../craft-fonts cargo run -p lightcraft` (or any cargo/xtask command). `crates/engine/build.rs`
embeds the manifest's fonts as `lightcraft_engine::CRAFT_FONTS` (wasm32: BIZ UDPGothic Regular only); the UI
(`theme::font_definitions`) and the export watermark renderer use its Japanese faces as fallbacks after Inter. Unset,
(`theme::font_definitions`) and the export watermark renderer use its CJK faces (picked by script) as fallbacks after Inter. Unset,
`CRAFT_FONTS` is empty: everything builds, tests and runs, but Japanese text has no glyphs. Releases always build
with it (`release.yml`, `CRAFT_FONTS_REQUIRED=1`) and ship the fonts' OFL licences. Tests that need these fonts skip
without it; the FreeBSD CI job runs them with it. Rules: craftrules
+5 -2
View File
@@ -63,7 +63,10 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
- **Never use any iconography, image, artwork, font, sound or other asset from Adobe products** (no Lightroom/Creative Cloud icons, no screenshots, no presets/profiles/LUTs, no UI bitmaps — not even as a temporary placeholder or "reference copy"). Observing Adobe's UI to imitate *layout and behaviour* is allowed; copying or tracing its assets is not.
- **This includes Adobe's open-licensed assets**: no Source Sans/Serif/Code or Source Han fonts, no Adobe Fonts, no
Adobe-published icon sets, sample photos, colour profiles or LUTs — even when OFL/MIT. The UI font is Inter (OFL);
Japanese fonts come from craft-fonts (below).
Japanese and Chinese fonts come from craft-fonts (below). **One exception (maintainer decision, 2026-10-07): Noto
Sans/Serif CJK** (Google-branded, OFL, co-developed with Adobe as Source Han) is allowed via craft-fonts for Chinese
text, because nearly every OFL Chinese face derives from it. Use it unmodified under its OFL; this does not open the
door to Source Han under Adobe's name or to any other Adobe asset.
- Every asset in the repository must be one of: **our own original work** (e.g. icons drawn in code as vectors, procedurally generated demo photos), **public domain / CC0**, **Creative Commons** (CC-BY / CC-BY-SA with attribution honoured), **OFL** (fonts), or **permissive open-source** (MIT/Apache-2.0/BSD/ISC) — or contributed by a person who created the asset and licenses it openly.
- **Exception: `docs/brand/`.** The ArtCraft name, wordmark and logos there are ArtCraft Team trademarks, not open source and not covered by LightCraft's MIT OR Apache-2.0 licence (`LICENSE-MIT`, `LICENSE-APACHE`, `NOTICE`); their terms are in `docs/brand/LICENSE-brand.txt`. Use them only unmodified and never redraw, recolour or derive from them.
- **Every asset must have an entry in `assets/ATTRIBUTION.md`** (path, title, author/creator, source URL or "original work", licence, date added, modifications) and its licence text when required (e.g. `assets/fonts/OFL-*.txt`). Add the entry in the same commit as the asset. Assets without an attribution entry must not be committed.
@@ -72,7 +75,7 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
uses it as the optional build input `CRAFT_FONTS_DIR`: `git clone https://github.com/storytold/craft-fonts ../craft-fonts`
then `CRAFT_FONTS_DIR=../craft-fonts cargo run -p lightcraft` (or any cargo/xtask command). `crates/engine/build.rs`
embeds the manifest's fonts as `lightcraft_engine::CRAFT_FONTS` (wasm32: BIZ UDPGothic Regular only); the UI
(`theme::font_definitions`) and the export watermark renderer use its Japanese faces as fallbacks after Inter. Unset,
(`theme::font_definitions`) and the export watermark renderer use its CJK faces (picked by script) as fallbacks after Inter. Unset,
`CRAFT_FONTS` is empty: everything builds, tests and runs, but Japanese text has no glyphs. Releases always build
with it (`release.yml`, `CRAFT_FONTS_REQUIRED=1`) and ship the fonts' OFL licences. Tests that need these fonts skip
without it; the FreeBSD CI job runs them with it. Rules: craftrules
Generated
+3
View File
@@ -2876,6 +2876,7 @@ name = "lightcraft-engine"
version = "0.4.0"
dependencies = [
"ab_glyph",
"harfrust",
"libc",
"lightcraft-catalog",
"lightcraft-codecs",
@@ -2891,6 +2892,7 @@ dependencies = [
"lightcraft-raw",
"lightcraft-scenes",
"lightcraft-segment",
"lightcraft-tiff",
"log",
"miniz_oxide 0.8.9",
"rayon",
@@ -3028,6 +3030,7 @@ dependencies = [
"proptest",
"rayon",
"serde",
"sha2",
"thiserror 2.0.21",
"zune-core",
"zune-jpeg",
+20 -7
View File
@@ -258,7 +258,7 @@ lightcraft --control 7980 ~/Pictures/trip
## Fast, native, private
- **Pure Rust, no C.** Our own RAW decoders (DNG, Canon CR2, Sony ARW, Nikon NEF, Fujifilm RAF incl. X-Trans,
- **Pure Rust, no C.** Our own RAW decoders (DNG, Canon CR2/CR3, Sony ARW, Nikon NEF, Fujifilm RAF incl. X-Trans,
Panasonic RW2 / Leica RWL, Pentax PEF, Olympus ORF), our own colour science, our own pipeline. JPEG, PNG, TIFF, WebP,
PSD composites and JPEG XL open today.
- **Scene-referred & wide-gamut.** Linear Rec.2020 float internally, Bradford-adapted white balance, gamut mapping
@@ -288,8 +288,8 @@ LightCraft is young and moving fast. **Where we honestly stand** (details in the
- **As a day-to-day Lightroom replacement we're nearer 60–70%.** It's great for JPEG/DNG and most Nikon / Sony /
older-Canon raws on one machine.
- **The biggest gaps:**
- **camera colour calibration:** raws other than DNG develop with a neutral colour matrix today, so colour is muted;
- **CR3 and compressed Fujifilm / Olympus raws:** these open as embedded previews only;
- **camera colour calibration:** Sony, Nikon, Panasonic, Fujifilm and Canon CR3 raws have guarded estimates from their camera JPEGs, with built-in ILCE-7M4, X-H2S and X-T4 profiles; measured calibration is missing, and other raws or rejected fits retain a neutral matrix;
- **compressed Olympus raws and unsupported CR3 variants:** these use embedded JPEG previews when present. Fujifilm lossless/lossy compressed RAF now decodes sensor data; [verification and existing-library reload instructions](docs/raf-compression.md);
- **AI masks and denoise:** subject and sky selection are classical heuristics;
- **HDR, video and the Classic Print / Book / Map modules.**
- **What's next:** see [where we're going](ROADMAP.md#where-were-going).
@@ -302,9 +302,9 @@ LightCraft is young and moving fast. **Where we honestly stand** (details in the
| Masking: brush, linear, radial, luminance/colour range, add/subtract/intersect | ✅ (AI subject/sky use classical heuristics for now) |
| Crop, straighten tool + auto straighten, flip, rotate, aspect ratios, overlays | ✅ |
| Profiles (Color, Neutral, Vivid, Landscape, Portrait, Monochrome: our own looks), presets, versions, history, copy/paste/sync settings | ✅ |
| Camera colour: DNG files use their own matrices | ✅ · our own calibration for other raws ⬜ (top priority; ARW, NEF and RW2 start from a look fitted to their own JPEG, other raws from a neutral fallback) |
| Camera colour: DNG files use their own matrices | ✅ DNG · 🟡 own Sony/Fujifilm profiles; measured calibration database missing |
| Native macOS menu bar (generated from the command registry), control channel + every widget addressable, headless UI snapshots | ✅ |
| RAW: DNG, CR2, ARW, NEF (uncompressed + lossless/lossy compressed), Fujifilm RAF (uncompressed, Bayer + X-Trans), Panasonic RW2 / Leica RWL / Panasonic RAW (every raw format, DMC-LX1 to DC-S1RM2), Pentax PEF, Olympus ORF (uncompressed); embedded previews for every format incl. CR3 | ✅ · CR3, compressed RAF/ORF decode ⬜ |
| RAW: DNG, CR2, CR3 (lossless CRX Bayer and version 0x100/0x200 C-RAW), ARW, NEF (uncompressed + lossless/lossy compressed), Fujifilm RAF (uncompressed + lossless/lossy compressed, Bayer + X-Trans), Panasonic RW2 / Leica RWL / Panasonic RAW (every raw format, DMC-LX1 to DC-S1RM2), Pentax PEF, Olympus ORF (uncompressed); embedded previews for every format incl. CR3 | 🟡 · CR3, compressed ORF decode ⬜ |
| Detail: sharpening, luminance + colour noise reduction | ✅ · AI Denoise, Super Resolution ⬜ |
| Remove / Heal / Clone spots (auto source), Visualize Spots, Red Eye and Pet Eye (auto pupil detection, catchlight) | ✅ · content-aware fill, spot pin editing 🚧 |
| Export: JPEG / PNG / TIFF / WebP / AVIF / DNG / original, sizing, file-size limit, output sharpening, naming templates, batch, metadata policy, text or image watermark | ✅ · HDR export ⬜ |
@@ -312,7 +312,7 @@ LightCraft is young and moving fast. **Where we honestly stand** (details in the
| Import: Add in place / Copy / Move, rename and folder templates, devices, duplicate detection, watched folders; Local folder browsing | ✅ |
| MCP server (headless or live app, persistent libraries), CLI, control channel | ✅ |
| XMP sidecars (read/write, auto-write), reading `crs:` develop settings, preset files (`.lcpreset`, XMP presets) | ✅ |
| Optics (distortion, vignetting, auto + manual CA, defringe, DNG-embedded lens corrections), Geometry (transforms, Constrain Crop), Upright (Auto/Level/Vertical/Full/Guided) | ✅ · camera lens profiles (our own) ⬜ |
| Optics (distortion, vignetting, auto + manual CA, defringe, lens corrections embedded in DNG files and Panasonic / Leica RW2 / RWL distortion data), Geometry (transforms, Constrain Crop), Upright (Auto/Level/Vertical/Full/Guided) | ✅ · camera lens profiles (our own) ⬜ |
| Photo Merge: HDR (auto-align, deghost), Panorama (spherical/cylindrical/perspective, boundary warp, auto crop), HDR Panorama → DNG | ✅ |
| GPU pipeline (wgpu compute, CPU-exact within 1/255), CPU fallback on device limits / errors | ✅ · WebGPU in the browser 🚧 |
| AI: segmentation masks, AI denoise, super resolution, faces; HDR editing; video | ⬜ (see [roadmap](ROADMAP.md#where-were-going)) |
@@ -353,9 +353,22 @@ CRAFT_FONTS_DIR=../craft-fonts cargo run --release -p lightcraft
Without it LightCraft builds and runs the same, but Chinese and Japanese text have no glyphs. Fonts are never committed to this
repo; see [craftrules `standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md).
**Interface language:** **Edit → Language** (English, 简体中文, 繁體中文(台灣), 日本語) or **Settings → General →
**Interface language:** **Edit → Language** (English, 简体中文, 繁體中文(台灣), 日本語, Português (Brasil), Deutsch) or **Settings → General →
Language**; the choice applies immediately and persists. See [docs/localization.md](docs/localization.md).
**Logs:** the desktop app writes its log to standard error and to `logs/lightcraft.log` in its settings folder
(Linux `$XDG_CONFIG_HOME/lightcraft/logs/`, by default `~/.config/lightcraft/logs/`; macOS
`~/Library/Application Support/LightCraft/logs/`; Windows `%APPDATA%\LightCraft\logs\`), never in the library. A
launch from a desktop menu or the Dock has no terminal, so attach this file to a bug report. Each start moves the
previous log to `lightcraft.1.log` and that one to `lightcraft.2.log`, so the log of a run that crashed survives the
next start; the file stops growing at 16 MiB, `--version` and `--help` write none, and runs with
`LIGHTCRAFT_NO_PREFS` log to standard error only. By default LightCraft's own crates log at `info` and everything else
at `warn`. `LIGHTCRAFT_LOG=info` or `debug` works as before (that level for LightCraft's own crates, warnings and
errors from the rest; any other value: warnings and errors only) and wins over `RUST_LOG`, which otherwise replaces
the default with env_logger-style directives such as `RUST_LOG=debug` or `RUST_LOG=warn,lightcraft_pipeline=trace` (a
directive ending in `*` covers every target starting with it, as in `lightcraft*=debug`). Panics are recorded there
too, and still in `lightcraft-panics.log` in the temp folder. The logger is `apps/lightcraft/src/logging.rs`.
The web build needs the `wasm32-unknown-unknown` target and the matching `wasm-bindgen` CLI
(`cargo xtask web` prints the exact install command); see [docs/web.md](docs/web.md).
+44 -24
View File
@@ -6,11 +6,11 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi
## Where we stand
*Honest assessment, 2026-10-05. Agents: read this before picking work. The checklist in
*Honest assessment, 2026-10-08. Agents: read this before picking work. The checklist in
[`docs/parity.md`](docs/parity.md) counts features that **exist**; this section is about whether a photographer can
**switch** from Lightroom. Update it when a gap below closes.*
**In one line:** the checklist says **79%** (P0 98.5%, P1 95.9%, P2 39%), but measured by whether a working
**In one line:** the checklist says **79.7%** (P0 98.2%, P1 95.6%, P2 41.6%), but measured by whether a working
photographer could replace Lightroom without noticing, we are at roughly **60–70%**. The remaining gap is mostly
**quality of results and camera coverage**, not missing buttons.
@@ -22,14 +22,14 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
| Dimension | Estimate | What's true today | Biggest gaps |
|---|---:|---|---|
| **Feature checklist** | 79% | P0 core and P1 nearly complete: import (Add / Copy / Move, templates, devices), library, grid/loupe/compare/survey, every Edit slider, curves, colour grading, masking tools, crop/Upright, heal/clone, presets/profiles, versions/history, sync, export, menus, shortcuts | P1: lens-profile database, content-aware fill (patch synthesis), video playback/trim |
| **RAW coverage** (formats people shoot) | ~55% | DNG (all kinds), CR2, ARW, NEF (uncompressed + Huffman lossless/lossy), uncompressed RAF/ORF, RW2 / RWL / Panasonic RAW (every raw format, checked on 178 files from 118 Panasonic and Leica bodies), PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed RAF/ORF, Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. NEFs labelled compressed but stored uncompressed (Z 6 packed 14-bit, D850 12-bit uncompressed). Per-model verification is thin (~55 corpus files vs >1,000 models) |
| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW, NEF and RW2 have a guarded per-file embedded-JPEG colour estimate (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
| **Feature checklist** | 79.7% | P0 core and P1 nearly complete: import (Add / Copy / Move, templates, devices), library, grid/loupe/compare/survey, every Edit slider, curves, colour grading, masking tools, crop/Upright, heal/clone, presets/profiles, versions/history, sync, export, menus, shortcuts | P0: full-resolution zoom detail; P1: lens-profile database, content-aware fill (patch synthesis), video playback/trim |
| **RAW coverage** (formats people shoot) | ~55% | DNG (all kinds), CR2, CR3 lossless Bayer and version 0x100/0x200 C-RAW (M50/R100/R8 pixel-exact corpus), ARW, NEF (uncompressed + Huffman lossless/lossy), RAF (uncompressed, lossless/lossy compressed Bayer and X-Trans), uncompressed ORF, RW2 / RWL / Panasonic RAW (every raw format, checked on 178 files from 118 Panasonic and Leica bodies), PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed ORF, Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. NEFs labelled compressed but stored uncompressed (Z 6 packed 14-bit, D850 12-bit uncompressed). Per-model verification is thin (~55 corpus files vs >1,000 models) |
| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW, NEF, RW2, RAF and CR3 have guarded camera-JPEG colour estimates, with bundled ILCE-7M4, X-H2S and X-T4 profiles (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
| **AI & computational** | ~15–20% | Assisted culling (focus, bursts), auto tone, HDR/panorama merge; subject/sky/background masks as classical heuristics | Real segmentation masks (subject, sky, people, objects, landscape, depth), AI denoise, super resolution, lens blur, generative remove, faces/people, natural-language search. **Blocked on a model strategy** (licensable weights or our own training; pure-Rust inference is feasible) |
| **Workflow & library** | ~85% (single machine) | Robust catalog (journal + snapshots, background compaction, crash-tested), 85k-photo libraries stay responsive, Local browsing with automatic cleanup, XMP interop, keywords, smart albums, Move import | Opening an 85k library takes 1.7–4.7 s; no cloud sync (out of scope), no tablet companion (#74, roadmap), shared albums, publish services, tethering |
| **Classic modules** | ~30% | Geotagging from GPX track logs, soft proofing (partial), slideshow (basic) | **Map view, Book, Print, Slideshow module, Web, publish services**: ~40 tracker rows ⬜ |
| **HDR & video** | 0% | | HDR edit/display/export; video play/trim/edit/export |
| **Platform & robustness** | ~70% | macOS native; Windows/Linux builds; web via WASM; no-panic lints workspace-wide, `unsafe` confined to `crates/sysmem`; failed saves are reported; GPU errors fall back to CPU | Windows installer UI unverified on Windows (PR #79); GPU path proven only on Apple + user reports; English, Simplified/Traditional Chinese and Japanese UI (see docs/localization.md); remaining technical errors; Traditional Chinese uses the Simplified Chinese font until craft-fonts has a TC face; accessibility partial; headless UI tests time out under machine load |
| **Platform & robustness** | ~70% | macOS native; Windows/Linux builds; web via WASM; no-panic lints workspace-wide, `unsafe` confined to `crates/sysmem`; failed saves are reported; GPU errors fall back to CPU | Windows installer UI unverified on Windows (PR #79); GPU path proven only on Apple + user reports; English, Simplified/Traditional Chinese, Japanese, Brazilian Portuguese and German UI (see docs/localization.md); remaining technical errors; Traditional Chinese uses the Simplified Chinese font until craft-fonts has a TC face; accessibility partial; headless UI tests time out under machine load |
### By kind of user
@@ -37,7 +37,8 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
|---|---:|---|
| JPEG / DNG shooter, single machine | ~85% | Fidelity polish, AI masks |
| Nikon / Sony / Panasonic / older-Canon raw shooter | ~65% | Camera colour fidelity and coverage (ARW, NEF and RW2 preview estimates are only a starting point); Panasonic compacts and kit zooms also miss their embedded distortion correction |
| Canon CR3 / Fujifilm / Olympus shooter | ~35% | Their raws open as embedded previews only (CR3: the full-size JPEG with full metadata) |
| Fujifilm raw shooter | ~65% | Uncompressed and lossless/lossy compressed RAF decoded; camera colour calibration remains missing. Verified on 13 bodies, including 14/16-bit GFX; see `docs/raf-compression.md` |
| Canon CR3 / Olympus compressed-raw shooter | ~45% | CR3 lossless Bayer and version 0x100/0x200 C-RAW develop from sensor data (M50, R100, R8 verified pixel for pixel); other CRX variants and compressed ORF open as embedded JPEG previews. Camera colour and broader model verification remain |
| Lightroom Classic power user | ~45% | Print, Book, Map, publish, tethering |
| Relies on AI (masks, denoise) | ~25% | Object / Describe masks via optional SAM 3 (download not yet hosted); no AI denoise |
@@ -46,9 +47,9 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
Priorities, in order. Each points at tracker rows in [`docs/parity.md`](docs/parity.md) → *Top gaps*.
1. **Camera colour calibration of our own** (LR-PROF-CAMERACOLOR, P0): fit each camera to its own embedded JPEG, use
matrices the files carry themselves, then chart shots. Sony ARW, Nikon NEF and Panasonic RW2 have the file-local fit
matrices the files carry themselves, then chart shots. Sony ARW, Nikon NEF, Panasonic RW2 and Fujifilm RAF have the file-local fit
(matrix + tone curve from their own JPEG); generalise it to the other makes' raws (PEF, ORF…), then validate fidelity.
2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): **CR3** first, then compressed RAF / ORF, NEF
2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): Remaining **unverified CR3 CRX** variants, compressed ORF, NEF
lossy-after-split, sRAW. Decided 2026-10-05: write our own decoders from prose descriptions (never decoder source,
no LGPL dependency); compressed NEF (#86) is the template.
3. **Verified camera coverage** (LR-IMP-CAMERA-COVERAGE, P0): a CC0 sample per model in the corpus, each decoded and
@@ -67,6 +68,10 @@ Already closed in the week of 2026-10-03: compressed NEF (#10), Move import (#29
Local roots / cleanup, grid and catalog performance at 85k photos (#35, #37), failed-save reporting, GPU export
hardening (#78), copyright metadata (#51), GPX geotagging (#60), import tag help and folder templates (#31, #32).
Image navigation now supports native pinch zoom and two-finger pan in Detail, Compare and Reference views.
The whole-image preview still has a configurable size cap (2,560 px default), so 100% on larger photos does not
yet guarantee original-pixel detail; visible-region rendering remains a gap (LR-VIEW-ZOOM).
## Milestones
**Status legend:** ✅ done · 🚧 in progress · ⬜ not started
@@ -84,24 +89,24 @@ hardening (#78), copyright metadata (#51), GPX geotagging (#60), import tag help
| M8 | Heal / Remove | content-aware remove (PatchMatch), heal, clone, brush spots, visualize spots, red/pet eye | 6–10 | 🚧 (heal, clone, auto source, visualize spots, red/pet eye ✅; PatchMatch remove ⬜) |
| M9 | Presets, profiles, versions, sync | preset browser + amount, create/import presets, profile browser, versions, history, copy/paste/sync settings | 5–8 | ✅ |
| M10 | Export & share | export dialog (JPEG/PNG/TIFF/DNG/AVIF/JXL/original), sizing, sharpening, metadata, watermark, naming, batch jobs, XMP sidecars, HDR export | 6–10 | 🚧 (all formats incl. DNG/original, sizing, presets, background jobs ✅; JXL encode, HDR export ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 (every raw format), PEF, ORF uncompressed ✅; **camera colour calibration** 🚧 (guarded ARW, NEF and RW2 preview fitting; measured database still missing), CR3, compressed ORF/RAF ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (CR3 lossless CRX and C-RAW, RAF uncompressed and lossless/lossy compressed, RW2 (every raw format), PEF, ORF uncompressed ✅; **camera colour calibration** 🚧 (guarded ARW, NEF, RW2 and RAF preview fitting; measured database still missing), CR3, compressed ORF ⬜) |
| M12 | AI & smart features | subject/sky/background/people/object masks, semantic search, faces/People (permissively licensed models, pure-Rust inference) | 20–40 | ⬜ |
| M13 | Merge | HDR merge (deghost), panorama (projections, boundary warp, fill edges), HDR panorama | 10–15 | ✅ |
| M14 | Video | import/playback/trim via FilmCraft crates, global edits + presets on video, video export | 6–10 | ⬜ |
| M15 | Classic modules | Map, Book, Slideshow, Print, Web; smart collections, stacks, virtual copies, publish services, tethering | 25–40 | 🚧 (smart albums, stacks, virtual copies, compare/survey ✅; Map/Book/Slideshow/Print/Web ⬜) |
| M16 | 1.0 polish | preferences, shortcut editor, accessibility, localization, packaging (dmg/msi/AppImage/web), hardening | 10–20 | 🚧 (settings, keyboard shortcuts sheet, packaging basics ✅; English/Chinese (Simplified, Traditional)/Japanese localisation 🟡; accessibility and further locales ⬜) |
| M16 | 1.0 polish | preferences, shortcut editor, accessibility, localization, packaging (dmg/msi/AppImage/web), hardening | 10–20 | 🚧 (settings, keyboard shortcuts sheet, packaging basics ✅; English/Chinese (Simplified, Traditional)/Japanese/Brazilian Portuguese/German localisation 🟡; accessibility and further locales ⬜) |
## Parity estimate (feature count updated 2026-10-05; effort estimate from 2026-10-02)
## Parity estimate (feature count updated 2026-10-08; effort estimate from 2026-10-02)
**By feature count** — from `docs/parity.md` (one row per Lightroom feature, menu item and shortcut; `cargo xtask
parity` prints this line on every run, so it stays current):
| Scope | Weighted completion | Rows | 2026-10-02 |
|---|---:|---:|---:|
| P0 (core) | **98.5%** | 200 | 98.7% |
| P1 (important parity) | **95.9%** | 147 | 94.4% |
| P2 (later / AI / niche, incl. Classic modules) | **39.2%** | 158 | 29.4% |
| **All in-scope rows** | **79.2%** | 505 | 75.6% |
| P0 (core) | **98.2%** | 200 | 98.7% |
| P1 (important parity) | **95.6%** | 149 | 94.4% |
| P2 (later / AI / niche, incl. Classic modules) | **41.6%** | 160 | 29.4% |
| **All in-scope rows** | **79.7%** | 509 | 75.6% |
The 2026-10-05 count includes three new 🟡 rows that make quality gaps visible (camera colour calibration, verified
camera coverage, render fidelity), which is why P0 dipped slightly.
@@ -118,7 +123,7 @@ trip, slideshow, auto import…) in ≈ 12 h on 2026-10-02; four to six parallel
| Work package | Tracker rows | Agent-hours | Risk |
|---|---|---:|---|
| Remaining P0/P1 UI and library features (folder rename/move, keyword painter, people view…) | ≈ 8 | 5–10 | low |
| Raw codecs: CR3 (CRX), compressed ORF / RAF, NEF lossy-after-split, HEIC/AVIF decode, JPEG XL DNG | LR-IMP-FORMATS | 40–80 | **high** — clean-room black-box analysis, no permissive specs |
| Raw codecs: CR3 (CRX), compressed ORF, NEF lossy-after-split, HEIC/AVIF decode, JPEG XL DNG | LR-IMP-FORMATS | 40–80 | **high** — clean-room black-box analysis, no permissive specs |
| Lens-profile database of our own (calibration targets, fitting, data) | LR-EDIT-OPTICS-PROFILE | 15–30 | data collection |
| Video: playback, trim, edits, export (pure-Rust decode, ideally shared with FilmCraft) | R. Video | 20–40 | medium |
| AI: subject / sky / background / people / object masks, object-aware remove, AI denoise, super resolution, lens blur, people & faces, natural-language search, culling | ≈ 30 | 80–150 | **high** — permissively licensed weights, pure-Rust inference, maybe training |
@@ -165,31 +170,45 @@ The milestone estimates in the table above were made before work started and are
Sony ILCE-7M4 downsized lossless ARW (subsampled YCbCr 4:2:0 / 4:2:2 tiles) now decodes to linear RGB;
private 3:2 & 4:3 examples verified through native rendering & full-resolution 16-bit export.
CR3 lossless Bayer and version 0x100/0x200 C-RAW sensor decoding is verified pixel-exactly on Canon M50, R100 and R8; any other CR3, or one that fails to decode, opens from its embedded JPEG as before. See [`docs/cr3.md`](docs/cr3.md) for coding limits, reference provenance and colour limitations.
Decoded (CC0 corpus from raw.pixls.us, `cargo xtask corpus --download`, `crates/raw/tests/corpus.rs`): DNG (uncompressed,
LJ92, lossy JPEG / Smart Previews, Deflate, float, linear), CR2, ARW (uncompressed, ARW2, LJ92; as-shot white balance and black level of pre-2017 bodies from the enciphered
maker-note `Tag2010` and the encrypted `SR2SubIFD`, both recovered by black-box analysis, `crates/raw/src/vendor/arw.rs`), NEF/NRW uncompressed and Huffman-compressed (lossless, lossy type 1/2, 12/14-bit), RAF uncompressed (Bayer and
X-Trans), RW2 / Leica RWL / Panasonic RAW in every raw format (compressed formats 4 and 6, the prefix-coded strips of format 8,
maker-note `Tag2010` and the encrypted `SR2SubIFD`, both recovered by black-box analysis, `crates/raw/src/vendor/arw.rs`), NEF/NRW uncompressed and Huffman-compressed (lossless, lossy type 1/2, 12/14-bit), RAF uncompressed and lossless/lossy compressed (Bayer and
X-Trans, 14/16-bit real-file verification), RW2 / Leica RWL / Panasonic RAW in every raw format (compressed formats 4 and 6, the prefix-coded strips of format 8,
packed formats 2, 5 and 7, the 16-bit words of the 2005–2007 bodies; all recovered by black-box analysis of 178 CC0 files from
118 bodies, `crates/raw/src/vendor/rw2.rs`), PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Every
supported container also yields its embedded JPEG preview (CR3 too), and the engine shows that preview for raw variants
it can't decode yet.
118 bodies, `crates/raw/src/vendor/rw2.rs`), PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Supported containers yield their embedded JPEG preview when present (including CR3). The engine uses it for RAW variants it cannot decode yet; HEVC-only CR3 previews are not decoded.
Not decoded yet — preview only (no permissively licensed description; black-box analysis incomplete):
- **Nikon "lossy after split" NEF** (non-zero split row in maker note `0x0096`, e.g. some D3400/D5000/D5200/D5500/D5600
files): rows above the split decode with the regular lossy table; from the split row on a different code is used
that our black-box analysis has not recovered yet (none of the four regular tables fits, also not with byte
alignment or reset predictors at the split row). The other Nikon Huffman variants are decoded (`crates/raw/src/vendor/nefc.rs` documents the analysis).
- **Olympus compressed ORF**, **Fujifilm compressed RAF**, **Canon CR3/CRX** (M11.1), **Canon sRAW/mRAW**, lossy DNG.
- **Olympus compressed ORF**, **Canon CR3 unverified CRX variants** (M11.1), **Canon sRAW/mRAW**, lossy DNG.
**Camera colour matrices:** ARW, NEF and RW2 files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
**Camera colour matrices:** ARW, NEF, RW2, RAF and CR3 files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
`matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer
matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax equivalents; Panasonic RW2 files carry none) and our
own chart-based calibration (M11.4). Adobe matrices are never used.
## Lightroom Classic catalog migration
File → Import Lightroom Catalog… reads `.lrcat` plus committed WAL pages directly through a pure-Rust
SQLite reader. Originals stay in place; ratings, flags, labels, XMP metadata, hierarchical keywords,
collections/sets, virtual copies and supported develop settings migrate into LightCraft. Existing edits
are preserved by default, reimport identities are persistent, and source settings/history/snapshots are
archived before catalog mutation. Missing originals remain available for relinking.
Rendering is approximate: unsupported Adobe profiles/AI/process settings are reported and archived;
history/snapshots remain source data, and smart collections import current membership. The Lightroom
database is read-only; LightCraft owns subsequent edits. See `docs/lightroom-catalog-import.md`.
## Log
- 2026-09-30: roadmap created; M0 in progress; research docs (Lightroom reference, Rust imaging ecosystem) complete.
- 2026-09-30 (later): app running with the full Lightroom-style UI; pipeline v0; DNG/CR2/ARW; README showcase. ≈ 8 h elapsed.
- 2026-10-02: parity estimate added (xtask parity prints weighted completion); milestone statuses refreshed.
- 2026-10-08: Fujifilm camera colour: guarded JPEG matching and relative WB, plus bundled X-H2S/X-T4 profiles from 201/545 photos. On 150 withheld files the pooled profiles improve mean ΔE slightly over per-file fitting (X-H2S 1.70 → 1.62, X-T4 2.43 → 2.34); dark X-H2S samples 8.74 → 2.25. Measured calibration remains open.
- 2026-10-08: Fujifilm lossless/lossy compressed RAF: pure-Rust striped decoder; full sensor arrays verified against a black-box reference on 17 CC0 files and 7 supplied X-H2S / X-T4 files (13 camera bodies, 14/16-bit Bayer and X-Trans). Camera colour calibration remains a separate gap.
- 2026-10-01: RAW II formats: RAF (uncompressed Bayer + X-Trans), RW2 (packed), PEF (incl. Huffman), ORF (uncompressed); embedded previews for every container incl. CR3; raw corpus test with 37 CC0 samples.
- 2026-10-03 – 10-05: community PRs (#42–#51, #60) and 25+ issues landed: compressed NEF, Move import, folder templates,
Local roots and cleanup, 85k-photo grid and catalog performance, background compaction, failed-save errors, GPU
@@ -198,6 +217,7 @@ own chart-based calibration (M11.4). Adobe matrices are never used.
- 2026-10-07: Panasonic RW2 / Leica RWL / Panasonic RAW decode in every raw format (compressed 4/6/8, packed 2/5/7,
16-bit words), black levels, in-camera aspect crops and mapped-out defects established on 178 CC0 files from 118
bodies; RW2 gets the file-local camera look (accepted on 140 of 174 files with a preview).
- 2026-10-08: independent Rust CR3 lossless and version 0x100/0x200 C-RAW decoding, six exact full-sensor regressions on M50/R100/R8; any CR3 decode error falls back to the embedded JPEG; guarded CR3 camera-JPEG colour fitting with independent framing validation (`docs/cr3.md`). Unverified CRX variants and broader model verification remain.
## Chinese and Japanese interfaces, and text watermarks
+26 -4
View File
@@ -71,6 +71,8 @@ USAGE:
per line: {\"method\": \"ui.set\", \"params\": {\"view\": \"detail\"}}.
Replies go to stdout. `ui.screenshot` without a path writes -o (then
OUT-2.png, OUT-3.png…); `ui.settle {timeoutMs?}` waits for renders.
A failed request (\"ok\": false) does not stop the script, but the exit
status is non-zero when any request failed.
-o, --output OUT PNG path (a final screenshot is written here if the script took none)
--size WxH window size in points (default 1600x1000)
--scale S pixels per point (default 1)
@@ -87,7 +89,7 @@ USAGE:
lightcraft-cli controls [--json] list every develop control id with its range
lightcraft-cli calibrate [--max N] [--out DIR] FOLDERS/FILES…
Fit a colour profile per camera model from raw files and their embedded camera JPEGs
(Sony ARW, Nikon NEF): up to N files spread over the folders (default 300; 0 = all), pooled per
(Sony ARW, Nikon NEF, Fujifilm RAF): up to N files spread over the folders (default 300; 0 = all), pooled per
model, written as <model>.json to DIR (default: the profiles folder LightCraft reads,
<config>/camera-profiles, or $LIGHTCRAFT_CAMERA_PROFILES). Raws of a profiled model then
take their colour from the profile and only their tone from their own JPEG.
@@ -100,6 +102,14 @@ static ALLOC: dhat::Alloc = dhat::Alloc;
/// Why `--library DIR` can't be opened; for a library open in another program, how to work with
/// that one instead (issue #99).
/// Print the open library's warnings (an unlockable library, damaged settings files) on stderr:
/// stdout carries command results / MCP protocol messages only.
fn library_warnings(session: &mut Session, who: &str) {
for w in session.take_library_warnings() {
eprintln!("{who}: warning: {w}");
}
}
fn library_error(dir: &str, e: lightcraft_engine::EngineError) -> String {
match e {
lightcraft_engine::EngineError::LibraryInUse(why) => format!(
@@ -122,6 +132,9 @@ fn main() -> ExitCode {
dhat::Profiler::builder().file_name(file).build()
};
alloc_release::install();
// Warnings (a GPU render redone on the CPU, an unknown backend name) on stderr; LIGHTCRAFT_LOG
// or RUST_LOG picks another level (#168).
lightcraft_engine::logging::install("lightcraft-cli");
let args: Vec<String> = std::env::args().skip(1).collect();
let r = match args.first().map(String::as_str) {
Some("run") => run(&args[1..]),
@@ -154,7 +167,8 @@ fn main() -> ExitCode {
/// Raw files below `path` (or `path` itself), skipping hidden and NAS metadata folders.
fn raw_files(path: &Path, out: &mut Vec<std::path::PathBuf>, depth: usize) {
let is_raw = |p: &Path| p.extension().and_then(|e| e.to_str()).is_some_and(|e| ["arw", "nef", "nrw"].iter().any(|x| e.eq_ignore_ascii_case(x)));
let is_raw =
|p: &Path| p.extension().and_then(|e| e.to_str()).is_some_and(|e| ["arw", "nef", "nrw", "raf"].iter().any(|x| e.eq_ignore_ascii_case(x)));
if path.is_file() {
if is_raw(path) {
out.push(path.to_path_buf());
@@ -311,6 +325,7 @@ fn mcp(args: &[String]) -> Result<(), String> {
let mut h = Headless::default();
let r = h.session.open_library(dir, demo).map_err(|e| library_error(dir, e))?;
eprintln!("lightcraft-cli mcp: opened library {dir} ({r:?})");
library_warnings(&mut h.session, "lightcraft-cli mcp");
h
}
None if demo => Headless::demo(),
@@ -547,6 +562,7 @@ fn run(args: &[String]) -> Result<(), String> {
Some(dir) => {
let mut h = Headless::default();
h.session.open_library(dir, demo).map_err(|e| library_error(dir, e))?;
library_warnings(&mut h.session, "lightcraft-cli");
h
}
None if demo => Headless::demo(),
@@ -659,7 +675,7 @@ fn render(args: &[String]) -> Result<(), String> {
for (k, v) in opts {
p[k] = v;
}
let mut o = ExportOptions::from_json(&p);
let mut o = ExportOptions::from_params(&p).map_err(|e| e.to_string())?;
if p.get("format").is_none() {
let ext = Path::new(&output).extension().map(|e| e.to_string_lossy().to_string()).unwrap_or_default();
o.format = ExportFormat::parse(&ext)
@@ -755,6 +771,9 @@ fn snapshot(args: &[String]) -> Result<(), String> {
Some(p.with_file_name(format!("{stem}-{shots}.{ext}")).to_string_lossy().to_string())
};
let mut wrote_output = false;
// Issue #166: a scripted request that fails still gets its reply printed and the run carries
// on (like `run --keep-going`), but the exit status reports it — CI judges by exit status.
let mut failed = 0usize;
if let Some(script) = &script {
let text = std::fs::read_to_string(script).map_err(|e| format!("{script}: {e}"))?;
let mut out = std::io::stdout().lock();
@@ -787,6 +806,9 @@ fn snapshot(args: &[String]) -> Result<(), String> {
}
_ => h.request(&method, params, timeout),
};
if reply["ok"] != true {
failed += 1;
}
if let Some(o) = reply.as_object_mut() {
o.insert("id".into(), id);
// wall time of the request (incl. the frames it ran), and since the start
@@ -823,7 +845,7 @@ fn snapshot(args: &[String]) -> Result<(), String> {
}
}
eprintln!("lightcraft-cli snapshot: done in {:.2} s ({} frames)", t0.elapsed().as_secs_f64(), h.frames());
Ok(())
if failed > 0 { Err(format!("{failed} scripted request(s) failed")) } else { Ok(()) }
}
fn commands(args: &[String]) -> Result<(), String> {
+78 -2
View File
@@ -19,6 +19,19 @@ fn gradient_png(path: &std::path::Path) {
std::fs::write(path, png).unwrap();
}
#[test]
fn calibrate_reads_raf_inputs_and_reports_bad_files() {
let input = tmp("invalid.RAF");
std::fs::write(&input, b"FUJIFILMCCD-RAW").unwrap();
let out = tmp("raf-profiles");
let result = Command::new(BIN).args(["calibrate", "--out"]).arg(&out).arg(&input).output().unwrap();
assert!(!result.status.success());
let stderr = String::from_utf8_lossy(&result.stderr);
assert!(stderr.contains("1 of 1 raw files"), "RAF was silently ignored: {stderr}");
assert!(stderr.contains("invalid.RAF") && stderr.contains("no profile written"), "{stderr}");
assert!(!out.join("X-T4.json").exists(), "bad input must not produce a profile");
}
fn mean(path: &std::path::Path) -> f64 {
let d = lightcraft_codecs::decode(&std::fs::read(path).unwrap(), Default::default()).unwrap();
let img = d.to_srgb8();
@@ -154,10 +167,14 @@ fn snapshot_subcommand_renders_the_ui_headlessly() {
std::fs::write(
&script,
format!(
"# comment\n{}\n{}\n{}\n{}\n",
"# comment\n{}\n{}\n{}\n{}\n{}\n{}\n",
json!({"method": "ui.set", "params": {"view": "photoGrid"}}),
// (settled: under load a grid shot taken before its thumbnails is darker than the
// dimmed export dialog over them, and the comparison below fails)
json!({"method": "ui.settle", "params": {"timeoutMs": 20000}}),
json!({"method": "ui.screenshot"}),
json!({"method": "engine.execute", "params": {"command": "dialog.export"}}),
json!({"method": "ui.settle", "params": {"timeoutMs": 20000}}),
json!({"method": "ui.screenshot", "params": {"path": b.to_str().unwrap()}}),
),
)
@@ -168,7 +185,7 @@ fn snapshot_subcommand_renders_the_ui_headlessly() {
.unwrap();
assert!(o.status.success(), "{}", String::from_utf8_lossy(&o.stderr));
let replies: Vec<Value> = String::from_utf8_lossy(&o.stdout).lines().map(|l| serde_json::from_str(l).unwrap()).collect();
assert_eq!(replies.len(), 4);
assert_eq!(replies.len(), 6);
assert!(replies.iter().all(|r| r["ok"] == true), "{replies:?}");
for (p, dimmed) in [(&a, false), (&b, true)] {
let d = lightcraft_codecs::decode(&std::fs::read(p).unwrap(), Default::default()).unwrap();
@@ -183,6 +200,39 @@ fn snapshot_subcommand_renders_the_ui_headlessly() {
assert_eq!((d.width, d.height), (640, 480));
}
/// Issue #166: a scripted request that fails makes `snapshot` exit non-zero. The failed reply is
/// still printed, the later lines still run, and the final `-o` screenshot is still written.
#[test]
fn snapshot_script_failure_exits_non_zero() {
let script = tmp("snap-fail.jsonl");
let out = tmp("snap-fail.png");
std::fs::write(
&script,
format!(
"{}\n{}\n",
json!({"method": "ui.set", "params": {"view": "grid"}}), // not a view: `photoGrid`, `detail`, …
json!({"method": "ui.set", "params": {"view": "photoGrid"}}),
),
)
.unwrap();
let _ = std::fs::remove_file(&out);
let o = Command::new(BIN)
.args(["snapshot", "--demo", "--script", script.to_str().unwrap(), "-o", out.to_str().unwrap(), "--size", "320x200"])
.output()
.unwrap();
let stderr = String::from_utf8_lossy(&o.stderr);
assert!(!o.status.success(), "a failed scripted request must fail the run\n{stderr}");
assert!(stderr.contains("1 scripted request(s) failed"), "{stderr}");
let replies: Vec<Value> = String::from_utf8_lossy(&o.stdout).lines().map(|l| serde_json::from_str(l).unwrap()).collect();
assert_eq!(replies.len(), 2, "{replies:?}");
assert_eq!(replies[0]["ok"], false, "{replies:?}");
assert!(replies[0]["error"].as_str().unwrap_or("").contains("grid"), "{replies:?}");
assert_eq!(replies[1]["ok"], true, "{replies:?}");
// continue-and-report: the final screenshot is still written
let d = lightcraft_codecs::decode(&std::fs::read(&out).unwrap(), Default::default()).unwrap();
assert_eq!((d.width, d.height), (320, 200));
}
/// Issue #136: with the GPU switched off from the environment the UI starts and renders on the CPU,
/// without creating a GPU device (no driver is loaded), and says why.
#[test]
@@ -336,3 +386,29 @@ fn connect_failure_hint_names_the_attempted_port() {
assert!(err.contains("--control 18437"), "the hint must name the port it tried: {err}");
assert!(!err.contains("--control 7980"), "{err}");
}
/// The CLI logs warnings on stderr (#168): with `RUST_LOG=warn` an unknown GPU backend name is
/// reported, and `RUST_LOG=off` silences it. Before, no logger was installed and nothing appeared.
#[test]
fn warnings_are_logged_on_stderr() {
let input = tmp("log-in.png");
gradient_png(&input);
let out = tmp("log-out.jpg");
let run = |level: &str| {
let o = Command::new(BIN)
.args(["render", input.to_str().unwrap(), "-o", out.to_str().unwrap()])
.env("LIGHTCRAFT_GPU_BACKEND", "bogus")
.env("RUST_LOG", level)
.env_remove("LIGHTCRAFT_LOG")
.env_remove("LIGHTCRAFT_GPU")
.output()
.unwrap();
let stderr = String::from_utf8_lossy(&o.stderr).into_owned();
assert!(o.status.success(), "{stderr}");
stderr
};
let warn = run("warn");
assert!(warn.contains("LIGHTCRAFT_GPU_BACKEND=bogus names no known backend"), "{warn}");
let off = run("off");
assert!(!off.contains("names no known backend"), "{off}");
}
+18 -3
View File
@@ -207,6 +207,7 @@ fn services(originals: Originals, backend: Option<Backend>, files: Files, frozen
// Preset files: browser pickers are asynchronous; not wired on the web yet.
pick_preset_files: None,
pick_tracklog: None,
pick_lightroom_catalog: None,
save_preset_file: None,
pick_curve_preset_files: None,
save_curve_preset_file: None,
@@ -429,8 +430,17 @@ impl WebApp {
cores.saturating_sub(1).clamp(1, 4)
});
let cache = app.session.media.rendered.clone();
let workers =
(n > 0).then(|| Workers::start(n, backend.as_ref().map_or("memory", |b| b.kind()), backend.clone(), index, &cache, cc.egui_ctx.clone()));
let workers = (n > 0).then(|| {
Workers::start(
n,
backend.as_ref().map_or("memory", |b| b.kind()),
backend.clone(),
index,
&cache,
app.session.cache_bytes(),
cc.egui_ctx.clone(),
)
});
if let Some(w) = &workers {
app.renderer.set_offload(Box::new(w.clone()));
}
@@ -601,11 +611,16 @@ impl eframe::App for WebApp {
fn logic(&mut self, ctx: &egui::Context, _frame: &mut eframe::Frame) {
for text in safety::take_notices() {
let now = ctx.input(|i| i.time);
self.app.ui.toast = Some((text, now + NOTICE_SECS));
self.app.ui.toast = Some((text, now + NOTICE_SECS, None));
}
self.run_inbox();
self.import_dropped(ctx);
self.load_originals(ctx);
if let Some(w) = &self.workers {
// `library.preferences ▸ cacheMb` is read by the session; the worker index prunes on
// its own thread, so it gets the budget every frame and applies it on the next store.
w.set_budget(self.app.session.cache_bytes());
}
self.app.logic(ctx);
self.save();
if let Some(b) = self.bench.as_mut()
+24 -3
View File
@@ -18,7 +18,7 @@ use std::sync::Arc;
use js_sys::{Array, Object, Reflect, Uint8Array, Uint32Array};
use lightcraft_engine::catalog::PhotoId;
use lightcraft_engine::media::{DISK_CACHE_BYTES, RenderJob, RenderResult, SourceLevel};
use lightcraft_engine::media::{RenderJob, RenderResult, SourceLevel};
use lightcraft_engine::pipeline::Rendered;
use lightcraft_preview::{Hash128, PreviewCache};
use lightcraft_raster::{Histogram, Rgba8};
@@ -183,6 +183,9 @@ struct Inner {
/// (photo, level) → worker that last decoded it.
affinity: HashMap<(PhotoId, SourceLevel), usize>,
index: ThumbIndex,
/// The session's thumbnail cache budget in bytes. Kept here because the prune happens on a
/// worker message, where the session is out of reach; the app refreshes it every frame.
budget: u64,
backend: Option<Backend>,
ctx: egui::Context,
/// Jobs finished by workers / inline (for diagnostics).
@@ -214,7 +217,15 @@ impl Workers {
/// Start `n` workers (none: every job runs inline). `store` is the backend kind for workers.
/// `index` lists the stored thumbnails of `cache`, the session's rendered-thumbnail cache
/// now: watching it from the start, a clear before the first render request still counts.
pub fn start(n: usize, store: &str, backend: Option<Backend>, index: ThumbIndex, cache: &Arc<PreviewCache>, ctx: egui::Context) -> Workers {
pub fn start(
n: usize,
store: &str,
backend: Option<Backend>,
index: ThumbIndex,
cache: &Arc<PreviewCache>,
budget: u64,
ctx: egui::Context,
) -> Workers {
let w = Workers(Rc::new(RefCell::new(Inner {
preview_generation: CacheWatch::new(cache),
workers: Vec::new(),
@@ -222,6 +233,7 @@ impl Workers {
next_id: 0,
affinity: HashMap::new(),
index,
budget,
backend,
ctx,
remote_done: 0,
@@ -328,7 +340,10 @@ impl Workers {
}
if stored > 0 {
g.index.insert(&hex, stored);
let gone = g.index.prune(DISK_CACHE_BYTES);
// The session's own budget (`library.preferences ▸ cacheMb`), not the native
// default: the browser accepted the setting, so it has to be the one enforced
// (#245).
let gone = g.index.prune(g.budget);
if !gone.is_empty()
&& let Some(be) = g.backend.clone()
{
@@ -386,6 +401,12 @@ impl Workers {
let ready = g.workers.iter().filter(|w| w.ready && !w.dead).count();
(alive, ready, g.remote_done, g.inline_done)
}
/// The budget the thumbnail index is pruned to. The app refreshes it from the session every
/// frame, so `library.preferences ▸ cacheMb` applies to the next stored thumbnail.
pub fn set_budget(&self, bytes: u64) {
self.0.borrow_mut().budget = bytes;
}
}
impl RenderOffload for Workers {
+2 -2
View File
@@ -28,11 +28,11 @@ pub fn start(port: u16, ctx: egui::Context) -> Receiver<ControlRequest> {
let listener = match TcpListener::bind(("127.0.0.1", port)) {
Ok(l) => l,
Err(e) => {
eprintln!("lightcraft: control server failed to bind 127.0.0.1:{port}: {e}");
log::error!("control server failed to bind 127.0.0.1:{port}: {e}");
return rx;
}
};
eprintln!("lightcraft: control server listening on 127.0.0.1:{port}");
log::info!("control server listening on 127.0.0.1:{port}");
std::thread::spawn(move || accept_loop(listener, tx, ctx));
rx
}
+145
View File
@@ -0,0 +1,145 @@
//! File-dialog filters that match extensions whatever their letter case.
//!
//! On Linux, `rfd` turns each extension into a `*.ext` glob for the GTK dialog and for the
//! desktop portal (`xdg-desktop-portal` FileChooser), and both match globs case-sensitively, so
//! a filter of `nef` hid `DSC_1234.NEF` (issue #342). Windows and macOS ignore case. Every
//! dialog filter of the app goes through [`FileDialogExt::add_filter_nocase`] (a test checks
//! that nothing else calls `add_filter`), so no extension can hit this.
use rfd::FileDialog;
/// The patterns to hand to `rfd` for `exts`, which are plain extensions without the dot.
///
/// Windows and macOS take the extension literally and ignore case: lower case, de-duplicated.
/// Linux (`linux`): `rfd` formats each entry as `*.{entry}`, which GTK 3 and the portal match
/// as a glob, so an entry may carry glob syntax. Each extension gives the lower-case form, the
/// UPPER-case form (kept in case a portal backend does not do `[...]` classes) and a bracket
/// form such as `[nN][eE][fF]` that matches every mix (`.Nef`, `.nEf`, ...).
pub fn dialog_extensions_for(exts: &[&str], linux: bool) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
let mut push = |s: String| {
if !s.is_empty() && !out.contains(&s) {
out.push(s);
}
};
for ext in exts {
let ext = ext.trim_start_matches('.');
let lower = ext.to_lowercase();
push(lower.clone());
if !linux {
continue;
}
push(ext.to_uppercase());
// `[` `]` `*` `?` in an extension would already be glob syntax: leave those alone
if lower.chars().any(|c| c.is_alphabetic()) && !lower.contains(['[', ']', '*', '?', '\\']) {
push(
lower
.chars()
.map(|c| {
let up: String = c.to_uppercase().collect();
if up.chars().count() == 1 && up != c.to_string() { format!("[{c}{up}]") } else { c.to_string() }
})
.collect(),
);
}
}
out
}
/// [`dialog_extensions_for`] for the platform this is built for.
pub fn dialog_extensions(exts: &[&str]) -> Vec<String> {
dialog_extensions_for(exts, cfg!(all(unix, not(target_os = "macos"))))
}
/// `add_filter` that matches the extensions in any letter case.
pub trait FileDialogExt {
fn add_filter_nocase(self, name: impl Into<String>, exts: &[&str]) -> Self;
}
impl FileDialogExt for FileDialog {
fn add_filter_nocase(self, name: impl Into<String>, exts: &[&str]) -> Self {
self.add_filter(name, &dialog_extensions(exts))
}
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::*;
#[test]
fn other_platforms_get_lower_case_once() {
assert_eq!(dialog_extensions_for(&["NEF", "nef", ".Jpg", "jpg", ""], false), ["nef", "jpg"]);
}
#[test]
fn linux_gets_lower_upper_and_any_mix() {
assert_eq!(dialog_extensions_for(&["nef"], true), ["nef", "NEF", "[nN][eE][fF]"]);
// digits have no case; a duplicate adds nothing
assert_eq!(dialog_extensions_for(&["mp4", "MP4"], true), ["mp4", "MP4", "[mM][pP]4"]);
assert_eq!(dialog_extensions_for(&["7z"], true), ["7z", "7Z", "7[zZ]"]);
assert_eq!(dialog_extensions_for(&["123"], true), ["123"]);
}
#[test]
fn linux_glob_syntax_is_not_wrapped_again() {
assert_eq!(dialog_extensions_for(&["[nN]ef"], true), ["[nn]ef", "[NN]EF"]);
}
/// The Linux patterns match like GTK's glob (`*.{entry}`): checked against a tiny matcher of
/// the subset used here (verified against real GTK 3.24 `GtkFileFilter` in the PR notes).
fn glob_ext(entry: &str, ext: &str) -> bool {
let mut pat = entry.chars().peekable();
let mut s = ext.chars();
while let Some(p) = pat.next() {
let Some(c) = s.next() else { return false };
if p == '[' {
let mut hit = false;
for q in pat.by_ref() {
if q == ']' {
break;
}
hit |= q == c;
}
if !hit {
return false;
}
} else if p != c {
return false;
}
}
s.next().is_none()
}
#[test]
fn linux_patterns_match_every_case_mix() {
let pats = dialog_extensions_for(&["nef", "cr2", "xmp"], true);
let matches = |name: &str| pats.iter().any(|p| glob_ext(p, name));
for ok in ["nef", "NEF", "Nef", "nEF", "CR2", "Cr2", "XMP", "xMp"] {
assert!(matches(ok), "{ok}");
}
for bad in ["nex", "ne", "nefs", "jpg", "cr3"] {
assert!(!matches(bad), "{bad}");
}
}
/// The app's own extension list (the one the folder scan uses) is covered whole.
#[test]
fn import_extensions_are_all_covered_in_both_cases() {
let pats = dialog_extensions_for(lightcraft_engine::import::EXTENSIONS, true);
for e in lightcraft_engine::import::EXTENSIONS {
assert!(pats.iter().any(|p| glob_ext(p, &e.to_uppercase())), "{e}");
assert!(pats.iter().any(|p| p == e), "{e}");
}
}
/// Every dialog filter goes through the helper: `main.rs` must not call rfd's own
/// `add_filter`.
#[test]
fn no_dialog_bypasses_the_helper() {
for (file, src) in [("main.rs", include_str!("main.rs")), ("control_server.rs", include_str!("control_server.rs"))] {
assert!(!src.contains(".add_filter("), "{file} calls rfd's add_filter; use add_filter_nocase");
}
assert!(include_str!("main.rs").contains(".add_filter_nocase("));
}
}
+689
View File
@@ -0,0 +1,689 @@
//! The desktop app's logger: `log::` records go to standard error and to a log file.
//!
//! It replaces the stderr-only `StderrLog` that `main.rs` used to install. A launch from a
//! desktop menu or the Dock has no terminal, so the file is what a bug report can attach:
//! `<settings folder>/logs/lightcraft.log`, next to `ui.json` (`config_dir` in `main.rs`; Linux
//! `$XDG_CONFIG_HOME/lightcraft/logs`, by default `~/.config/lightcraft/logs`). Never in the
//! library folder. Each start moves the previous log to `lightcraft.1.log` (and that one to `.2`),
//! so the log of a run that crashed survives the next launch. Runs with `LIGHTCRAFT_NO_PREFS`
//! (tests, scripts) log to standard error only, so they don't rotate away the user's own logs.
//!
//! Levels ([`filter_spec`]): `info` for LightCraft's own crates, `warn` for everything else (wgpu
//! and naga are chatty). `LIGHTCRAFT_LOG` keeps the meaning it had with the old logger: `info` or
//! `debug` lowers LightCraft's own crates to that level, any other value means warnings and errors
//! only. Without it, `RUST_LOG` replaces the default with env_logger-style directives: `debug`,
//! `warn,lightcraft_pipeline=trace`, `wgpu_core=info`. A directive ending in `*` matches every
//! target that starts with it (`lightcraft*=debug`).
//!
//! Records logged before the settings folder is known are kept (up to [`MAX_PENDING`]) and
//! written once the file is attached. Writing never panics: a file that can't be created or
//! written leaves standard error as the only sink. Panics that escape everything are added to the
//! file by [`AppLogger::record_panic`] (the default hook already printed them on standard error).
//!
//! This is the desktop app's logger only; `lightcraft-cli` doesn't share a process with it.
use std::fs::File;
use std::io::Write;
use std::path::{Path, PathBuf};
use std::sync::{Mutex, OnceLock, PoisonError, TryLockError};
use std::time::SystemTime;
use log::LevelFilter;
/// The current log file's name inside the log folder.
pub const LOG_FILE: &str = "lightcraft.log";
/// How many previous logs are kept (`lightcraft.1.log` … `lightcraft.<KEEP>.log`).
pub const KEEP: usize = 2;
/// The log file stops growing past this size (a runaway warning can't fill the disk).
pub const MAX_FILE_BYTES: u64 = 16 * 1024 * 1024;
/// Records kept in memory until the log file is attached.
pub const MAX_PENDING: usize = 512;
/// The built-in filter when neither `LIGHTCRAFT_LOG` nor `RUST_LOG` says otherwise.
pub const DEFAULT_FILTER: &str = "warn,lightcraft*=info";
/// The filter directives for this run from `LIGHTCRAFT_LOG` and `RUST_LOG` (blank = unset).
///
/// `LIGHTCRAFT_LOG` wins when set and means what it did for the old stderr logger: `info` or
/// `debug` for LightCraft's own crates (warnings and errors from everything else), any other value
/// warnings and errors only. Else `RUST_LOG` as env_logger directives, else [`DEFAULT_FILTER`].
pub fn filter_spec(lightcraft_log: Option<&str>, rust_log: Option<&str>) -> String {
let set = |v: Option<&str>| v.map(str::trim).filter(|v| !v.is_empty()).map(str::to_owned);
if let Some(level) = set(lightcraft_log) {
return match level.as_str() {
"debug" => "warn,lightcraft*=debug".to_owned(),
"info" => "warn,lightcraft*=info".to_owned(),
_ => "warn".to_owned(),
};
}
set(rust_log).unwrap_or_else(|| DEFAULT_FILTER.to_owned())
}
/// Per-target level filter parsed from env_logger-style directives.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Filter {
default: LevelFilter,
directives: Vec<(String, LevelFilter)>,
}
impl Filter {
/// Parse `spec`; unknown levels and empty parts are skipped. Without a bare level, targets no
/// directive names log errors only.
pub fn parse(spec: &str) -> Filter {
let mut f = Filter { default: LevelFilter::Error, directives: Vec::new() };
for part in spec.split(',').map(str::trim).filter(|p| !p.is_empty()) {
match part.split_once('=') {
Some((name, level)) => {
let name = name.trim();
if let (false, Some(level)) = (name.is_empty(), parse_level(level)) {
f.directives.push((name.to_owned(), level));
}
}
None => match parse_level(part) {
Some(level) => f.default = level,
// A bare target name: everything from it (env_logger does the same).
None => f.directives.push((part.to_owned(), LevelFilter::Trace)),
},
}
}
f
}
/// The level that applies to `target` (the most specific matching directive wins).
pub fn level_for(&self, target: &str) -> LevelFilter {
self.directives.iter().filter(|(name, _)| matches(name, target)).max_by_key(|(name, _)| name.len()).map_or(self.default, |(_, level)| *level)
}
/// The most verbose level any target can reach (for `log::set_max_level`).
pub fn max(&self) -> LevelFilter {
self.directives.iter().map(|(_, level)| *level).fold(self.default, Ord::max)
}
}
fn parse_level(s: &str) -> Option<LevelFilter> {
s.trim().parse().ok()
}
/// `name` matches `target` itself and its submodules (`a` matches `a` and `a::b`, not `ab`);
/// `name*` matches every target starting with `name`.
fn matches(name: &str, target: &str) -> bool {
match name.strip_suffix('*') {
Some(prefix) => target.starts_with(prefix),
None => target.strip_prefix(name).is_some_and(|rest| rest.is_empty() || rest.starts_with("::")),
}
}
/// `2026-10-08T07:59:17.728Z` (UTC, milliseconds). A clock before 1970 reads as the epoch.
pub fn timestamp(t: SystemTime) -> String {
let since = t.duration_since(std::time::UNIX_EPOCH).unwrap_or_default();
let secs = since.as_secs();
let (days, rem) = (secs / 86_400, secs % 86_400);
let (y, m, d) = civil_from_days(days);
format!("{y:04}-{m:02}-{d:02}T{:02}:{:02}:{:02}.{:03}Z", rem / 3600, rem % 3600 / 60, rem % 60, since.subsec_millis())
}
/// Days since 1970-01-01 to a proleptic Gregorian (year, month, day) (Howard Hinnant's
/// `civil_from_days`, unsigned because `timestamp` clamps at the epoch). Saturating, so no clock
/// value can overflow it.
fn civil_from_days(days: u64) -> (u64, u64, u64) {
let z = days.saturating_add(719_468);
let era = z / 146_097;
let doe = z % 146_097;
let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let d = doy - (153 * mp + 2) / 5 + 1;
let m = if mp < 10 { mp + 3 } else { mp - 9 };
let y = era.saturating_mul(400).saturating_add(yoe).saturating_add(u64::from(m <= 2));
(y, m, d)
}
fn numbered(dir: &Path, n: usize) -> PathBuf {
if n == 0 { dir.join(LOG_FILE) } else { dir.join(format!("lightcraft.{n}.log")) }
}
/// Shift the previous logs up one (`.log` → `.1.log` → … → `.<KEEP>.log`, the oldest dropped)
/// and create a fresh, empty log file in `dir` (created if missing).
pub fn rotate(dir: &Path) -> std::io::Result<(PathBuf, File)> {
std::fs::create_dir_all(dir)?;
for n in (1..=KEEP).rev() {
match std::fs::rename(numbered(dir, n - 1), numbered(dir, n)) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e),
_ => {}
}
}
let path = numbered(dir, 0);
let file = File::create(&path)?;
Ok((path, file))
}
/// One formatted log line: `<timestamp> <LEVEL> [<thread>] <target>: <message>\n`.
pub fn format_line(ts: &str, level: log::Level, thread: &str, target: &str, message: &str) -> String {
let message = message.strip_suffix('\n').unwrap_or(message);
format!("{ts} {level:<5} [{thread}] {target}: {message}\n")
}
/// Where formatted lines go besides standard error: kept in memory until a file is attached,
/// then appended to it until it reaches its size cap.
pub struct Sink {
file: Option<File>,
pending: Vec<String>,
dropped: usize,
written: u64,
max_bytes: u64,
capped: bool,
}
impl Sink {
pub fn new(max_bytes: u64) -> Sink {
Sink { file: None, pending: Vec::new(), dropped: 0, written: 0, max_bytes, capped: false }
}
/// Append `line` to the file, or keep it until [`Sink::attach`].
pub fn write(&mut self, line: &str) {
if self.file.is_none() {
if self.capped {
// No file is coming ([`Sink::no_file`]): standard error had the line.
} else if self.pending.len() < MAX_PENDING {
self.pending.push(line.to_owned());
} else {
self.dropped = self.dropped.saturating_add(1);
}
return;
}
self.append(line);
}
/// Start writing to `file` (a fresh one: the size cap starts over), first the lines kept so far.
pub fn attach(&mut self, file: File) {
self.file = Some(file);
self.written = 0;
self.capped = false;
for line in std::mem::take(&mut self.pending) {
self.append(&line);
}
if self.dropped > 0 {
let note = format!("{} earlier log lines were dropped before the log file was opened\n", self.dropped);
self.dropped = 0;
self.append(&note);
}
}
/// There will be no log file: forget the lines kept for it and keep no more.
pub fn no_file(&mut self) {
self.file = None;
self.pending = Vec::new();
self.dropped = 0;
self.capped = true;
}
fn append(&mut self, line: &str) {
if self.capped {
return;
}
let Some(file) = self.file.as_mut() else { return };
let len = u64::try_from(line.len()).unwrap_or(u64::MAX);
let text = if self.written.saturating_add(len) > self.max_bytes {
self.capped = true;
format!("log file reached {} bytes; later records go to standard error only\n", self.max_bytes)
} else {
self.written = self.written.saturating_add(len);
line.to_owned()
};
// A full disk or a vanished file must not take the app down: drop the file sink.
if file.write_all(text.as_bytes()).is_err() {
self.file = None;
self.capped = true;
}
}
}
/// The installed logger: a [`Filter`], standard error and a [`Sink`].
pub struct AppLogger {
filter: Filter,
stderr: bool,
sink: Mutex<Sink>,
}
impl AppLogger {
pub fn new(filter: Filter, stderr: bool) -> AppLogger {
AppLogger { filter, stderr, sink: Mutex::new(Sink::new(MAX_FILE_BYTES)) }
}
/// Rotate the logs in `dir` and send records to the fresh file; returns its path. On an error
/// standard error stays the only sink.
pub fn attach_dir(&self, dir: &Path) -> Result<PathBuf, String> {
// A thread that panicked while logging leaves the sink usable: take it back.
let mut sink = self.sink.lock().unwrap_or_else(PoisonError::into_inner);
match rotate(dir) {
Ok((path, file)) => {
sink.attach(file);
Ok(path)
}
Err(e) => {
sink.no_file();
Err(format!("{}: {e}", dir.display()))
}
}
}
/// No log file for this run (`LIGHTCRAFT_NO_PREFS`): log to standard error only.
pub fn no_file(&self) {
self.sink.lock().unwrap_or_else(PoisonError::into_inner).no_file();
}
/// Add a panic report to the log file only (the default panic hook has printed it on standard
/// error), whatever the filter. Skipped when the sink is busy: the panic may have happened on
/// this very thread while it held the lock, and waiting would hang the app.
pub fn record_panic(&self, thread: &str, report: &str) {
let line = format_line(&timestamp(SystemTime::now()), log::Level::Error, thread, "lightcraft", report);
let mut sink = match self.sink.try_lock() {
Ok(sink) => sink,
Err(TryLockError::Poisoned(p)) => p.into_inner(),
Err(TryLockError::WouldBlock) => return,
};
sink.write(&line);
}
}
impl log::Log for AppLogger {
fn enabled(&self, metadata: &log::Metadata<'_>) -> bool {
metadata.level() <= self.filter.level_for(metadata.target())
}
fn log(&self, record: &log::Record<'_>) {
if !self.enabled(record.metadata()) {
return;
}
// Formatted before the sink is locked, so a message that logs while it is formatted can't deadlock.
let thread = std::thread::current();
let line =
format_line(&timestamp(SystemTime::now()), record.level(), thread.name().unwrap_or("?"), record.target(), &record.args().to_string());
if self.stderr {
// No terminal (a Windows GUI build, a closed pipe) is not an error worth reporting.
let _ = std::io::stderr().write_all(line.as_bytes());
}
self.sink.lock().unwrap_or_else(PoisonError::into_inner).write(&line);
}
fn flush(&self) {}
}
/// Install the logger (filter from [`filter_spec`]); `None` when another logger was installed
/// first. Call [`AppLogger::attach_dir`] (or [`AppLogger::no_file`]) once the arguments are parsed.
pub fn install() -> Option<&'static AppLogger> {
static LOGGER: OnceLock<AppLogger> = OnceLock::new();
let var = |name| std::env::var(name).ok();
let filter = Filter::parse(&filter_spec(var("LIGHTCRAFT_LOG").as_deref(), var("RUST_LOG").as_deref()));
let max = filter.max();
let logger = LOGGER.get_or_init(|| AppLogger::new(filter, true));
log::set_logger(logger).ok()?;
log::set_max_level(max);
Some(logger)
}
/// Chain a panic hook after the current one (the engine guard's: standard error plus
/// `lightcraft-panics.log` in the temp folder) that also puts the panic in the log file.
pub fn record_panics(logger: &'static AppLogger) {
let previous = std::panic::take_hook();
std::panic::set_hook(Box::new(move |info| {
previous(info);
let thread = std::thread::current();
let at = info.location().map(|l| format!(" at {}:{}", l.file(), l.line())).unwrap_or_default();
let report = format!("panic{at}: {}", lightcraft_engine::guard::panic_message(info.payload()));
logger.record_panic(thread.name().unwrap_or("?"), &report);
}));
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::{Duration, UNIX_EPOCH};
fn temp_dir(tag: &str) -> PathBuf {
static N: AtomicUsize = AtomicUsize::new(0);
let d = std::env::temp_dir().join(format!("lightcraft-logging-{tag}-{}-{}", std::process::id(), N.fetch_add(1, Ordering::Relaxed)));
let _ = std::fs::remove_dir_all(&d);
d
}
fn read(path: &Path) -> String {
std::fs::read_to_string(path).expect("read log")
}
fn record(logger: &AppLogger, level: log::Level, target: &str, msg: &str) {
use log::Log;
logger.log(&log::Record::builder().level(level).target(target).args(format_args!("{msg}")).build());
}
/// The logger this replaced (`StderrLog`): warnings and errors from every target, `info` and
/// `debug` only from LightCraft's own crates, and only when `LIGHTCRAFT_LOG` asked for them.
fn old_stderr_log_enabled(lightcraft_log: &str, level: log::Level, target: &str) -> bool {
let max = match lightcraft_log {
"debug" => LevelFilter::Debug,
"info" => LevelFilter::Info,
_ => LevelFilter::Warn,
};
level <= max && (level <= log::Level::Warn || target.starts_with("lightcraft"))
}
#[test]
fn the_default_filter_shows_lightcraft_info_and_other_crates_warnings() {
let f = Filter::parse(DEFAULT_FILTER);
assert_eq!(f.level_for("lightcraft"), LevelFilter::Info);
assert_eq!(f.level_for("lightcraft::control_server"), LevelFilter::Info);
assert_eq!(f.level_for("lightcraft_engine::guard"), LevelFilter::Info);
assert_eq!(f.level_for("lightcraft_ui_egui::render"), LevelFilter::Info);
assert_eq!(f.level_for("wgpu_core::device"), LevelFilter::Warn);
assert_eq!(f.level_for("naga"), LevelFilter::Warn);
assert_eq!(f.max(), LevelFilter::Info);
assert_eq!(filter_spec(None, None), DEFAULT_FILTER);
// An empty variable is the same as an unset one.
assert_eq!(filter_spec(Some(""), Some(" ")), DEFAULT_FILTER);
}
/// `LIGHTCRAFT_LOG` keeps working exactly as it did with the old stderr-only logger.
#[test]
fn lightcraft_log_keeps_its_meaning() {
let targets =
["lightcraft", "lightcraft::control_server", "lightcraft_engine::segment", "lightcraft_gpu", "wgpu_core::device", "naga", "eframe"];
let levels = [log::Level::Error, log::Level::Warn, log::Level::Info, log::Level::Debug, log::Level::Trace];
for value in ["info", "debug", "warn", "error", "verbose", "INFO"] {
let f = Filter::parse(&filter_spec(Some(value), None));
for target in targets {
for level in levels {
assert_eq!(level <= f.level_for(target), old_stderr_log_enabled(value, level, target), "LIGHTCRAFT_LOG={value} {level} {target}");
}
}
}
assert_eq!(Filter::parse(&filter_spec(Some("debug"), None)).max(), LevelFilter::Debug);
assert_eq!(Filter::parse(&filter_spec(Some("warn"), None)).max(), LevelFilter::Warn);
}
#[test]
fn rust_log_replaces_the_default_and_lightcraft_log_wins_over_it() {
assert_eq!(filter_spec(None, Some("warn,wgpu_core=info")), "warn,wgpu_core=info");
let f = Filter::parse(&filter_spec(None, Some("debug")));
assert_eq!(f.level_for("eframe"), LevelFilter::Debug);
// The app's own variable is the more specific choice.
let f = Filter::parse(&filter_spec(Some("info"), Some("trace")));
assert_eq!(f.level_for("eframe"), LevelFilter::Warn);
assert_eq!(f.level_for("lightcraft_engine"), LevelFilter::Info);
}
#[test]
fn directives_follow_env_logger_and_the_most_specific_one_wins() {
let f = Filter::parse("info,wgpu_core=error,lightcraft_pipeline=trace,lightcraft_pipeline::tiles=off");
assert_eq!(f.level_for("eframe"), LevelFilter::Info);
assert_eq!(f.level_for("wgpu_core::instance"), LevelFilter::Error);
assert_eq!(f.level_for("lightcraft_pipeline::stage"), LevelFilter::Trace);
assert_eq!(f.level_for("lightcraft_pipeline::tiles"), LevelFilter::Off);
assert_eq!(f.max(), LevelFilter::Trace);
// A module name is matched at `::` boundaries, not as a bare prefix.
assert_eq!(f.level_for("wgpu_core_extra"), LevelFilter::Info);
// A trailing `*` is a prefix.
assert_eq!(Filter::parse("lightcraft*=debug").level_for("lightcraft_raw::cr2"), LevelFilter::Debug);
assert_eq!(Filter::parse("lightcraft*=debug").level_for("eframe"), LevelFilter::Error);
// A bare target name sets that target to the most verbose level, as env_logger does.
assert_eq!(Filter::parse("naga").level_for("naga::front"), LevelFilter::Trace);
assert_eq!(Filter::parse("naga").level_for("eframe"), LevelFilter::Error);
// Levels are case-insensitive and surrounding blanks are ignored.
assert_eq!(Filter::parse(" WARN , lightcraft = Debug ").level_for("lightcraft"), LevelFilter::Debug);
assert_eq!(Filter::parse(" WARN , lightcraft = Debug ").level_for("eframe"), LevelFilter::Warn);
}
#[test]
fn hostile_specs_never_panic_and_fall_back_sensibly() {
let long = "x".repeat(100_000);
let many = "a=info,".repeat(10_000);
for spec in [
"",
",,,",
"=",
"==",
"=debug",
"lightcraft=",
"nonsense=loud",
"🦀=info",
"*",
"*=",
"=*",
"a=b=c",
"warn,,lightcraft=DEBUG",
"\0",
"é::ü=trace",
&long,
&many,
] {
let f = Filter::parse(spec);
for target in ["", "lightcraft", "🦀", "a::b", "*"] {
let _ = f.level_for(target);
}
let _ = f.max();
let _ = filter_spec(Some(spec), Some(spec));
}
assert_eq!(Filter::parse("warn,,lightcraft=DEBUG").level_for("lightcraft"), LevelFilter::Debug);
// An unknown level is skipped: that target keeps the default.
assert_eq!(Filter::parse("nonsense=loud").level_for("nonsense"), LevelFilter::Error);
assert_eq!(Filter::parse("lightcraft=").level_for("lightcraft"), LevelFilter::Error);
assert_eq!(Filter::parse("=debug").max(), LevelFilter::Error);
assert_eq!(Filter::parse("🦀=info").level_for("🦀::claw"), LevelFilter::Info);
}
#[test]
fn timestamps_are_utc_with_milliseconds() {
assert_eq!(timestamp(UNIX_EPOCH), "1970-01-01T00:00:00.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_millis(1_791_446_357_728)), "2026-10-08T07:59:17.728Z");
// Leap day, and the day after it.
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(1_709_164_800)), "2024-02-29T00:00:00.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(1_709_251_199)), "2024-02-29T23:59:59.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(1_709_251_200)), "2024-03-01T00:00:00.000Z");
// 2000 is a leap year (divisible by 400), 2100 is not.
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(951_782_400)), "2000-02-29T00:00:00.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(4_107_542_400)), "2100-03-01T00:00:00.000Z");
// Before the epoch (a clock set wrong) clamps instead of panicking.
assert_eq!(timestamp(UNIX_EPOCH - Duration::from_secs(5)), "1970-01-01T00:00:00.000Z");
}
#[test]
fn a_line_carries_time_level_thread_target_and_message() {
assert_eq!(
format_line("2026-10-08T07:59:17.728Z", log::Level::Info, "main", "lightcraft", "library /photos: 12 photos"),
"2026-10-08T07:59:17.728Z INFO [main] lightcraft: library /photos: 12 photos\n"
);
// A multi-line message keeps its lines; the record still ends in one newline.
assert!(format_line("t", log::Level::Error, "w", "x", "a\nb\n").ends_with("x: a\nb\n"));
}
#[test]
fn rotation_keeps_the_previous_logs_and_starts_an_empty_file() {
let dir = temp_dir("rotate");
for run in 1..=4 {
let (path, _file) = rotate(&dir).expect("rotate");
assert_eq!(path, dir.join(LOG_FILE));
assert_eq!(read(&path), "");
std::fs::write(&path, format!("run {run}")).expect("write");
}
assert_eq!(read(&dir.join(LOG_FILE)), "run 4");
assert_eq!(read(&dir.join("lightcraft.1.log")), "run 3");
assert_eq!(read(&dir.join("lightcraft.2.log")), "run 2");
assert!(!dir.join("lightcraft.3.log").exists(), "only {KEEP} old logs are kept");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn rotation_into_an_unusable_directory_is_an_error_not_a_panic() {
let dir = temp_dir("blocked");
std::fs::create_dir_all(dir.parent().expect("parent")).expect("tmp");
std::fs::write(&dir, "a file where the directory should be").expect("block");
assert!(rotate(&dir).is_err());
let _ = std::fs::remove_file(&dir);
}
#[test]
fn lines_logged_before_the_file_exists_are_written_when_it_is_attached() {
let dir = temp_dir("pending");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(MAX_FILE_BYTES);
sink.write("early 1\n");
sink.write("early 2\n");
sink.attach(file);
sink.write("late\n");
assert_eq!(read(&path), "early 1\nearly 2\nlate\n");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_pending_buffer_is_bounded_and_says_how_much_it_dropped() {
let dir = temp_dir("bounded");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(MAX_FILE_BYTES);
for i in 0..MAX_PENDING + 5 {
sink.write(&format!("line {i}\n"));
}
sink.attach(file);
let text = read(&path);
assert_eq!(text.lines().filter(|l| l.starts_with("line ")).count(), MAX_PENDING);
assert!(text.contains("5 earlier log lines were dropped"), "{text}");
assert!(text.contains("line 0\n") && !text.contains(&format!("line {MAX_PENDING}\n")));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn exactly_a_full_pending_buffer_drops_nothing() {
let dir = temp_dir("full");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(MAX_FILE_BYTES);
for i in 0..MAX_PENDING {
sink.write(&format!("line {i}\n"));
}
sink.attach(file);
let text = read(&path);
assert_eq!(text.lines().count(), MAX_PENDING);
assert!(!text.contains("dropped"), "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn without_a_file_nothing_is_kept_in_memory() {
let mut sink = Sink::new(MAX_FILE_BYTES);
sink.write("early\n");
sink.no_file();
sink.write("late\n");
assert!(sink.pending.is_empty() && sink.dropped == 0);
}
#[test]
fn the_file_stops_at_its_size_cap_with_one_note() {
let dir = temp_dir("cap");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(100);
sink.attach(file);
for i in 0..50 {
sink.write(&format!("line {i:02} with some padding\n"));
}
let text = read(&path);
assert_eq!(text.lines().filter(|l| l.starts_with("line ")).count(), 3, "{text}");
assert_eq!(text.matches("log file reached").count(), 1, "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_logger_filters_records_and_writes_them_to_the_attached_file() {
use log::Log;
let dir = temp_dir("logger");
let logger = AppLogger::new(Filter::parse(DEFAULT_FILTER), false);
record(&logger, log::Level::Info, "lightcraft", "before the settings directory");
record(&logger, log::Level::Info, "wgpu_core::device", "too chatty");
let path = logger.attach_dir(&dir).expect("attach");
assert_eq!(path, dir.join(LOG_FILE));
record(&logger, log::Level::Warn, "wgpu_hal::vulkan", "a real warning");
record(&logger, log::Level::Error, "lightcraft_engine::guard", "`develop.reset` failed unexpectedly: boom");
record(&logger, log::Level::Debug, "lightcraft", "below info");
let text = read(&path);
let lines: Vec<&str> = text.lines().collect();
assert_eq!(lines.len(), 3, "{text}");
assert!(lines[0].contains(" INFO [") && lines[0].ends_with("lightcraft: before the settings directory"), "{text}");
assert!(lines[1].ends_with("wgpu_hal::vulkan: a real warning"), "{text}");
assert!(lines[2].contains(" ERROR [") && lines[2].ends_with("lightcraft_engine::guard: `develop.reset` failed unexpectedly: boom"), "{text}");
assert!(logger.enabled(&log::Metadata::builder().level(log::Level::Info).target("lightcraft_ui_egui").build()));
assert!(!logger.enabled(&log::Metadata::builder().level(log::Level::Info).target("naga").build()));
// Attaching again rotates: the first log becomes `.1`.
logger.attach_dir(&dir).expect("attach again");
assert_eq!(read(&dir.join("lightcraft.1.log")), text);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn records_from_many_threads_are_whole_lines() {
let dir = temp_dir("threads");
let logger = std::sync::Arc::new(AppLogger::new(Filter::parse("info"), false));
let path = logger.attach_dir(&dir).expect("attach");
let workers: Vec<_> = (0..8)
.map(|t| {
let l = std::sync::Arc::clone(&logger);
std::thread::spawn(move || {
for i in 0..100 {
record(&l, log::Level::Info, "lightcraft", &format!("thread {t} record {i}"));
}
})
})
.collect();
for w in workers {
w.join().expect("worker");
}
let text = read(&path);
assert_eq!(text.lines().count(), 800);
assert!(text.lines().all(|l| l.contains(" lightcraft: thread ")), "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn attaching_to_an_unusable_directory_reports_an_error_and_keeps_logging() {
let dir = temp_dir("unusable");
std::fs::create_dir_all(dir.parent().expect("parent")).expect("tmp");
std::fs::write(&dir, "not a directory").expect("block");
let logger = AppLogger::new(Filter::parse("info"), false);
record(&logger, log::Level::Info, "lightcraft", "early");
assert!(logger.attach_dir(&dir).is_err());
record(&logger, log::Level::Error, "x", "still fine");
// Nothing waits in memory for a file that won't come.
assert!(logger.sink.lock().expect("sink").pending.is_empty());
let _ = std::fs::remove_file(&dir);
}
#[test]
fn a_poisoned_sink_still_logs() {
let dir = temp_dir("poison");
let logger = std::sync::Arc::new(AppLogger::new(Filter::parse("info"), false));
let l = std::sync::Arc::clone(&logger);
let _ = std::thread::spawn(move || {
let _guard = l.sink.lock().expect("lock");
panic!("poison the sink");
})
.join();
assert!(logger.sink.is_poisoned());
let path = logger.attach_dir(&dir).expect("attach");
record(&logger, log::Level::Warn, "lightcraft", "after the poison");
assert!(read(&path).contains("lightcraft: after the poison"));
let _ = std::fs::remove_dir_all(&dir);
}
/// The panic hook's report lands in the file, and never deadlocks when the panic happened
/// while this thread held the sink.
#[test]
fn a_panic_report_goes_to_the_file_and_never_deadlocks() {
let dir = temp_dir("panic");
let logger = AppLogger::new(Filter::parse("off"), false);
let path = logger.attach_dir(&dir).expect("attach");
logger.record_panic("render", "panicked at crates/x/src/lib.rs:3:5:\nboom");
{
let _held = logger.sink.lock().expect("lock");
logger.record_panic("main", "while the sink is held");
}
let text = read(&path);
assert!(text.contains(" ERROR [render] lightcraft: panicked at crates/x/src/lib.rs:3:5:\nboom\n"), "{text}");
assert!(!text.contains("while the sink is held"), "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
}
+315 -72
View File
@@ -24,9 +24,12 @@
mod alloc_release;
mod control_server;
mod dialog_filter;
mod logging;
#[cfg(target_os = "macos")]
mod native_menu;
use dialog_filter::FileDialogExt as _;
use lightcraft_engine::Session;
use lightcraft_ui_egui::panels::library_problem::LibraryProblem;
use lightcraft_ui_egui::{LightcraftApp, Services, UiState};
@@ -63,10 +66,10 @@ impl eframe::App for App {
}
fn on_exit(&mut self) {
if let Err(e) = self.1.save(&self.0) {
eprintln!("lightcraft: {e}");
log::error!("{e}");
}
if let Err(e) = self.0.session.close_library() {
eprintln!("lightcraft: saving the library failed: {e}");
log::error!("saving the library failed: {e}");
}
}
}
@@ -93,10 +96,23 @@ fn gpu_marker_path() -> Option<std::path::PathBuf> {
/// Arm the GPU crash sentinel; if the previous launch left it behind (it died inside the GPU
/// driver while creating the device), the notice to show — GPU rendering then starts off.
fn gpu_crash_check() -> Option<String> {
let marker = gpu_marker_path();
let left = marker.as_deref().and_then(lightcraft_engine::gpu::backend::take_init_marker);
lightcraft_engine::gpu::backend::set_init_marker(marker);
///
/// A `--memory` session writes nothing (issue #169): it neither arms the sentinel — which would
/// create the settings folder — nor removes a marker it finds. It still honours one, so the GPU
/// starts off there too, and the next ordinary launch reports and clears it.
fn gpu_crash_check(in_memory: bool) -> Option<String> {
gpu_crash_check_at(gpu_marker_path(), in_memory)
}
fn gpu_crash_check_at(marker: Option<std::path::PathBuf>, in_memory: bool) -> Option<String> {
use lightcraft_engine::gpu::backend;
let left = if in_memory {
marker.as_deref().and_then(backend::read_init_marker)
} else {
let left = marker.as_deref().and_then(backend::take_init_marker);
backend::set_init_marker(marker);
left
};
left.map(|what| gpu_crash_notice(&what))
}
@@ -112,19 +128,28 @@ fn config_dir() -> Option<std::path::PathBuf> {
lightcraft_engine::camera_profiles::config_dir()
}
/// `<config>/ui.json`, the saved UI state and app settings — `None` when nothing is read or
/// written: `LIGHTCRAFT_NO_PREFS` (tests, scripts) or no home folder.
fn prefs_path() -> Option<std::path::PathBuf> {
if std::env::var_os("LIGHTCRAFT_NO_PREFS").is_some() {
return None;
}
config_dir().map(|d| d.join("ui.json"))
}
/// The saved UI state and app settings (`<config>/ui.json`), if any, and a warning for the user
/// when the file exists but can't be used (issue #103): a damaged file is kept as
/// `ui.json.corrupt-<unix time>` first, so the next save can't lose the library location in it;
/// one that can't be read at all is not written this session (`keep_file`).
fn load_prefs() -> (Option<UiState>, Option<String>, bool) {
if std::env::var_os("LIGHTCRAFT_NO_PREFS").is_some() {
return (None, None, false);
}
let Some(path) = config_dir().map(|d| d.join("ui.json")) else { return (None, None, false) };
load_prefs_at(&path)
///
/// A `--memory` session reads the settings (so it looks like the user's app) but writes nothing
/// (issue #164), so a damaged file is left as it is there, not set aside.
fn load_prefs(in_memory: bool) -> (Option<UiState>, Option<String>, bool) {
let Some(path) = prefs_path() else { return (None, None, false) };
load_prefs_at(&path, !in_memory)
}
fn load_prefs_at(path: &std::path::Path) -> (Option<UiState>, Option<String>, bool) {
fn load_prefs_at(path: &std::path::Path, set_aside_damaged: bool) -> (Option<UiState>, Option<String>, bool) {
let bytes = match std::fs::read(path) {
Ok(b) => b,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return (None, None, false),
@@ -135,6 +160,10 @@ fn load_prefs_at(path: &std::path::Path) -> (Option<UiState>, Option<String>, bo
};
match serde_json::from_slice::<UiState>(&bytes) {
Ok(ui) => (Some(ui.sanitized()), None, false),
Err(e) if !set_aside_damaged => {
let msg = format!("The app settings ({}) are damaged ({e}). Defaults are used; nothing is saved in this session.", path.display());
(None, Some(msg), true)
}
Err(e) => {
let secs = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0);
let keep = path.with_file_name(format!("ui.json.corrupt-{secs}"));
@@ -177,8 +206,15 @@ fn write_atomic(path: &std::path::Path, bytes: &[u8]) -> std::io::Result<()> {
/// Saves `ui.json` (app settings, incl. the library to open at launch): as soon as the library
/// changes, every few seconds when anything else changed, and at exit.
///
/// Nothing is written while the session is one that promises to save nothing: `--memory`
/// (`path` is `None`; issue #164), or the one running because the library couldn't be opened — the
/// question window and Continue Without Saving (issue #100; `app.openLibrary` ends it, and the
/// library it opened is saved then).
#[derive(Default)]
struct PrefsWriter {
/// Where to write; `None` writes nothing (`--memory`, `LIGHTCRAFT_NO_PREFS`, no home folder).
path: Option<std::path::PathBuf>,
written: Vec<u8>,
library: String,
checked: f64,
@@ -190,19 +226,26 @@ struct PrefsWriter {
impl PrefsWriter {
fn save(&mut self, app: &LightcraftApp) -> Result<(), String> {
if self.keep_file || std::env::var_os("LIGHTCRAFT_NO_PREFS").is_some() {
self.save_ui(&app.ui, app.library_problem.is_some())
}
/// Write `ui` if it changed since the last write; `temporary` (the library couldn't be
/// opened) saves nothing.
fn save_ui(&mut self, ui: &UiState, temporary: bool) -> Result<(), String> {
if self.keep_file || temporary {
return Ok(());
}
let Some(d) = config_dir() else { return Ok(()) };
let bytes = serde_json::to_vec_pretty(&app.ui).map_err(|e| e.to_string())?;
let Some(path) = self.path.clone() else { return Ok(()) };
let bytes = serde_json::to_vec_pretty(ui).map_err(|e| e.to_string())?;
if bytes == self.written {
return Ok(());
}
std::fs::create_dir_all(&d)
.and_then(|()| write_atomic(&d.join("ui.json"), &bytes))
path.parent()
.map_or(Ok(()), std::fs::create_dir_all)
.and_then(|()| write_atomic(&path, &bytes))
.map_err(|e| format!("saving the app settings failed: {e}"))?;
self.written = bytes;
self.library = app.ui.settings.library_path.clone();
self.library = ui.settings.library_path.clone();
Ok(())
}
@@ -216,8 +259,8 @@ impl PrefsWriter {
match self.save(app) {
Ok(()) => self.failing = false,
Err(e) => {
eprintln!("lightcraft: {e}");
if !self.failing {
log::warn!("{e}");
app.notices.push(format!("{e}. LightCraft keeps trying."));
}
self.failing = true;
@@ -228,8 +271,25 @@ impl PrefsWriter {
}
}
/// The Windows command that opens `url` in the default browser. `explorer <url>` opened File
/// Explorer instead for links with a query or fragment, like the Map button's OpenStreetMap link
/// (issue #234). The URL protocol handler hands the URL to the browser as is; no shell runs, so
/// `?`, `&` and `#` need no escaping.
fn windows_open_url_command(url: &str) -> std::process::Command {
let mut c = std::process::Command::new("rundll32");
c.args(["url.dll,FileProtocolHandler", url]);
c
}
fn services() -> Services {
Services {
pick_lightroom_catalog: Some(Box::new(|| {
rfd::FileDialog::new()
.set_title(lightcraft_ui_egui::i18n::tr("Import Lightroom Catalog"))
.add_filter_nocase(lightcraft_ui_egui::i18n::tr("Lightroom Classic Catalog"), &["lrcat"])
.pick_file()
.map(|p| p.to_string_lossy().to_string())
})),
pick_folder: Some(Box::new(|| {
rfd::FileDialog::new().set_title(lightcraft_ui_egui::i18n::tr("Open Library")).pick_folder().map(|p| p.to_string_lossy().to_string())
})),
@@ -262,7 +322,7 @@ fn services() -> Services {
let status = if cfg!(target_os = "macos") {
std::process::Command::new("open").arg(url).status()
} else if cfg!(target_os = "windows") {
std::process::Command::new("explorer").arg(url).status()
windows_open_url_command(url).status()
} else {
std::process::Command::new("xdg-open").arg(url).status()
};
@@ -272,7 +332,8 @@ fn services() -> Services {
let status = if cfg!(target_os = "macos") {
std::process::Command::new("open").args(["-R", path]).status()
} else if cfg!(target_os = "windows") {
std::process::Command::new("explorer").arg(format!("/select,{path}")).status()
let win_path = path.replace('/', "\\");
std::process::Command::new("explorer").arg(format!("/select,{win_path}")).status()
} else {
let dir = std::path::Path::new(path).parent().map(|d| d.to_string_lossy().to_string()).unwrap_or_else(|| ".".into());
std::process::Command::new("xdg-open").arg(dir).status()
@@ -283,13 +344,7 @@ fn services() -> Services {
})),
pick_files: Some(Box::new(|| {
rfd::FileDialog::new()
.add_filter(
lightcraft_ui_egui::i18n::tr("Photos"),
&[
"jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "nrw", "arw", "raf", "orf", "rw2", "rwl", "raw",
"pef", "psd", "jxl", "gif", "bmp",
],
)
.add_filter_nocase(lightcraft_ui_egui::i18n::tr("Photos"), lightcraft_engine::import::EXTENSIONS)
.pick_files()
.unwrap_or_default()
.into_iter()
@@ -299,7 +354,7 @@ fn services() -> Services {
pick_preset_files: Some(Box::new(|| {
rfd::FileDialog::new()
.set_title(lightcraft_ui_egui::i18n::tr("Import Presets"))
.add_filter(
.add_filter_nocase(
lightcraft_ui_egui::i18n::tr("Presets & Profiles"),
&["lcpreset", "xmp", "lrtemplate", "zip", "dng", "lmp", "mplumpack", "cube"],
)
@@ -312,7 +367,7 @@ fn services() -> Services {
pick_tracklog: Some(Box::new(|| {
rfd::FileDialog::new()
.set_title(lightcraft_ui_egui::i18n::tr("Auto-Tag from Tracklog"))
.add_filter(lightcraft_ui_egui::i18n::tr("GPS Track Log"), &["gpx"])
.add_filter_nocase(lightcraft_ui_egui::i18n::tr("GPS Track Log"), &["gpx"])
.pick_file()
.map(|p| vec![p.to_string_lossy().to_string()])
.unwrap_or_default()
@@ -320,7 +375,7 @@ fn services() -> Services {
save_preset_file: Some(Box::new(|name: &str| {
rfd::FileDialog::new()
.set_title(lightcraft_ui_egui::i18n::tr("Export Presets"))
.add_filter(lightcraft_ui_egui::i18n::tr("LightCraft Preset"), &["lcpreset"])
.add_filter_nocase(lightcraft_ui_egui::i18n::tr("LightCraft Preset"), &["lcpreset"])
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
@@ -328,7 +383,7 @@ fn services() -> Services {
pick_curve_preset_files: Some(Box::new(|| {
rfd::FileDialog::new()
.set_title(lightcraft_ui_egui::i18n::tr("Import Point Curve Presets"))
.add_filter(lightcraft_ui_egui::i18n::tr("Point Curve Presets"), &["lccurve", "json"])
.add_filter_nocase(lightcraft_ui_egui::i18n::tr("Point Curve Presets"), &["lccurve", "json"])
.pick_files()
.unwrap_or_default()
.into_iter()
@@ -338,7 +393,7 @@ fn services() -> Services {
save_curve_preset_file: Some(Box::new(|name: &str| {
rfd::FileDialog::new()
.set_title(lightcraft_ui_egui::i18n::tr("Export Point Curve Presets"))
.add_filter(lightcraft_ui_egui::i18n::tr("Point Curve Presets"), &["lccurve"])
.add_filter_nocase(lightcraft_ui_egui::i18n::tr("Point Curve Presets"), &["lccurve"])
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
@@ -366,7 +421,7 @@ fn open_session(in_memory: bool, dir: Option<std::path::PathBuf>, seed_demo: boo
}
let unopened = || Session::new().with_fs().with_system_clock();
let Some(dir) = dir else {
eprintln!("lightcraft: no library location (set --library or LIGHTCRAFT_LIBRARY)");
log::warn!("no library location (set --library or LIGHTCRAFT_LIBRARY)");
return (unopened(), Some(LibraryProblem::new("", "There is no home folder to keep the library in. Choose a folder for it.")));
};
let t0 = std::time::Instant::now();
@@ -374,8 +429,8 @@ fn open_session(in_memory: bool, dir: Option<std::path::PathBuf>, seed_demo: boo
match s.open_library(&dir, seed_demo) {
Ok(r) => {
let (replayed, torn) = (r.replayed, r.torn_bytes);
eprintln!(
"lightcraft: library {}: {} photos, {replayed} log records replayed{} ({:.1} ms)",
log::info!(
"library {}: {} photos, {replayed} log records replayed{} ({:.1} ms)",
dir.display(),
s.catalog.len(),
if torn > 0 { ", torn tail repaired" } else { "" },
@@ -384,7 +439,7 @@ fn open_session(in_memory: bool, dir: Option<std::path::PathBuf>, seed_demo: boo
(s, None)
}
Err(e) => {
eprintln!("lightcraft: can't open library {}: {e}", dir.display());
log::error!("can't open library {}: {e}", dir.display());
(unopened(), Some(LibraryProblem::new(dir.to_string_lossy(), e.to_string())))
}
}
@@ -409,41 +464,65 @@ ENVIRONMENT:
LIGHTCRAFT_SAM3_DIR=DIR the SAM 3 model for Object / Describe masks (default: <settings folder>/models/sam3;
optional: LightCraft offers to download it when first needed)
LIGHTCRAFT_SAM3_MIRRORS=URL,… where to download the SAM 3 model from (base URLs, tried in order)
LIGHTCRAFT_LOG=info|debug more log detail from LightCraft itself (default: info from LightCraft, warnings
from other crates); else RUST_LOG=<env_logger directives>. Records go to stderr and to
<settings folder>/logs/lightcraft.log (previous runs: lightcraft.1.log, lightcraft.2.log)
";
/// Warnings and errors (failed commands, AI mask analysis) on stderr; `LIGHTCRAFT_LOG=info`
/// (or `debug`) shows more.
struct StderrLog(log::LevelFilter);
impl log::Log for StderrLog {
fn enabled(&self, m: &log::Metadata) -> bool {
m.level() <= self.0 && (m.level() <= log::Level::Warn || m.target().starts_with("lightcraft"))
}
fn log(&self, r: &log::Record) {
if self.enabled(r.metadata()) {
eprintln!("lightcraft: {} {}: {}", r.level(), r.target(), r.args());
}
}
fn flush(&self) {}
/// The launch arguments' files and folders to import (absolute), and a notice for each one that
/// is not imported (see [`lightcraft_engine::import::launch_path`]; issue #374).
fn launch_imports(args: &[String]) -> (Vec<String>, Vec<String>) {
let cwd = std::env::current_dir().ok();
let home = std::env::var_os(if cfg!(windows) { "USERPROFILE" } else { "HOME" }).filter(|h| !h.is_empty()).map(std::path::PathBuf::from);
launch_imports_in(args, cwd.as_deref(), home.as_deref())
}
fn install_log() {
let level = match std::env::var("LIGHTCRAFT_LOG").unwrap_or_default().as_str() {
"debug" => log::LevelFilter::Debug,
"info" => log::LevelFilter::Info,
_ => log::LevelFilter::Warn,
};
static LOGGER: std::sync::OnceLock<StderrLog> = std::sync::OnceLock::new();
if log::set_logger(LOGGER.get_or_init(|| StderrLog(level))).is_ok() {
log::set_max_level(level);
fn launch_imports_in(args: &[String], cwd: Option<&std::path::Path>, home: Option<&std::path::Path>) -> (Vec<String>, Vec<String>) {
use lightcraft_engine::import::{LaunchPath, launch_path};
let (mut files, mut notices) = (Vec::new(), Vec::new());
for a in args {
match launch_path(a, cwd, home, lightcraft_engine::devices::dcim_in) {
Some(LaunchPath::Import(p)) => files.push(p),
Some(LaunchPath::Refused { path, why }) => {
log::warn!("not importing {path} given at launch: {why}");
notices
.push(format!("{path} was not imported: it is {why}. To import from it, choose a folder inside it with File ▸ Import Photos…"));
}
None => {}
}
}
(files, notices)
}
/// Where the log files live: `logs` in the settings folder, next to `ui.json` (see `logging`).
/// None with `LIGHTCRAFT_NO_PREFS` (tests, scripts), so those runs don't rotate away the user's logs.
fn log_dir() -> Option<std::path::PathBuf> {
if std::env::var_os("LIGHTCRAFT_NO_PREFS").is_some() {
return None;
}
config_dir().map(|d| d.join("logs"))
}
/// The control port `value` names; a value that isn't a port is logged and ignored (`what` is
/// the option or variable it came from). Logged before the log file opens, so it is kept for it.
fn control_port_from(what: &str, value: Option<String>) -> Option<u16> {
let value = value?;
let port = value.trim().parse().ok();
if port.is_none() {
log::warn!("{what} {value:?} is not a port number; no control server");
}
port
}
fn main() -> eframe::Result {
// First, so every start-up record is kept for the log file (`logging`).
let logger = logging::install();
lightcraft_engine::guard::install_hook(std::env::temp_dir().join("lightcraft-panics.log"));
install_log();
if let Some(logger) = logger {
logging::record_panics(logger);
}
alloc_release::install();
let mut control_port: Option<u16> = std::env::var("LIGHTCRAFT_CONTROL_PORT").ok().and_then(|p| p.parse().ok());
let mut control_port = control_port_from("LIGHTCRAFT_CONTROL_PORT", std::env::var("LIGHTCRAFT_CONTROL_PORT").ok().filter(|v| !v.is_empty()));
let mut files = Vec::new();
let mut seed_demo = true;
let mut in_memory = false;
@@ -451,7 +530,7 @@ fn main() -> eframe::Result {
let mut args = std::env::args().skip(1);
while let Some(a) = args.next() {
match a.as_str() {
"--control" => control_port = args.next().and_then(|p| p.parse().ok()),
"--control" => control_port = control_port_from("--control", Some(args.next().unwrap_or_default())),
"--library" => library_dir = args.next().map(std::path::PathBuf::from),
"--no-demo" => seed_demo = false,
"--memory" | "--demo" => in_memory = true,
@@ -470,7 +549,31 @@ fn main() -> eframe::Result {
_ => files.push(a),
}
}
let (prefs, prefs_warning, keep_prefs_file) = load_prefs();
// The log file: opened after the arguments, so `--version`, `--help` and a usage error leave
// no file behind. Records logged until now are written to it first.
let mut log_file = None;
if let Some(logger) = logger {
// a --memory session writes nothing (issue #164), a log file included
match log_dir().filter(|_| !in_memory) {
Some(dir) => match logger.attach_dir(&dir) {
Ok(path) => {
log::info!("LightCraft {}, log file {}", env!("CARGO_PKG_VERSION"), path.display());
log_file = Some(path);
}
// The file sink has given up by now, so this goes to standard error only.
Err(e) => log::warn!("no log file: {e}"),
},
None => logger.no_file(),
}
}
// what the arguments name, resolved now: an empty or relative argument never becomes a drive
// root, and a whole drive or home folder is not walked (issue #374)
let (files, launch_notices) = launch_imports(&files);
let (prefs, prefs_warning, keep_prefs_file) = load_prefs(in_memory);
// the saved language from the start, so a failed start is reported in it too
if let Some(ui) = &prefs {
lightcraft_ui_egui::i18n::set_language(ui.language);
}
// --library, else the library last opened from Settings, else the default location
let library_dir = library_dir.or_else(|| {
prefs
@@ -482,7 +585,7 @@ fn main() -> eframe::Result {
let library_dir = library_dir.or_else(lightcraft_engine::library::default_dir);
// GPU compute: off if the preference says so, or if the last launch died creating the device
// (issue #136) — before anything can create it
let gpu_crash = gpu_crash_check();
let gpu_crash = gpu_crash_check(in_memory);
let gpu_on = prefs.as_ref().is_none_or(|u| u.settings.gpu) && gpu_crash.is_none();
lightcraft_engine::gpu::set_enabled(gpu_on);
let options = eframe::NativeOptions {
@@ -500,7 +603,7 @@ fn main() -> eframe::Result {
wgpu_options: window_wgpu_options(),
..Default::default()
};
eframe::run_native(
let started = eframe::run_native(
"LightCraft",
options,
Box::new(move |cc| {
@@ -518,8 +621,10 @@ fn main() -> eframe::Result {
lightcraft_ui_egui::i18n::set_language(app.ui.language);
app.integrated_titlebar = cfg!(target_os = "macos");
app.notices.extend(prefs_warning);
app.notices.extend(launch_notices);
// what's on disk now: only changes are written
let writer = PrefsWriter {
path: if in_memory { None } else { prefs_path() },
written: serde_json::to_vec_pretty(&app.ui).unwrap_or_default(),
library: app.ui.settings.library_path.clone(),
keep_file: keep_prefs_file,
@@ -539,7 +644,12 @@ fn main() -> eframe::Result {
p.pending_import = files;
app.library_problem = Some(p);
} else if !files.is_empty() {
let _ = app.run("library.import", serde_json::json!({"paths": files}));
// listed and read on the import worker once the window shows (issue #374: never
// here, before the first frame, where a large folder kept the window from
// appearing)
if let Err(e) = lightcraft_ui_egui::import::start_paths(&mut app, files) {
log::warn!("import at launch: {e}");
}
app.ui.view = lightcraft_ui_egui::state::ViewMode::PhotoGrid;
}
// native menu bar generated from the command registry (macOS; elsewhere the menus are
@@ -553,7 +663,28 @@ fn main() -> eframe::Result {
menu,
)))
}),
)
);
if let Err(e) = &started {
startup_failed(&e.to_string(), log_file.as_deref());
}
started
}
/// The window could not start (e.g. no usable graphics device): say so in the log and in a native
/// message box naming the log file, instead of exiting without a trace — the release builds have
/// no console on Windows, and an app started from Finder shows no standard error (issue #260).
fn startup_failed(error: &str, log_file: Option<&std::path::Path>) {
log::error!("LightCraft could not open its window: {error}");
if std::env::var_os("LIGHTCRAFT_NO_PREFS").is_some() {
return; // tests and scripts: no dialog to click away
}
let (title, text) = lightcraft_ui_egui::i18n::startup_failed_message(error, log_file.map(|p| p.display().to_string()).as_deref());
let _ = rfd::MessageDialog::new()
.set_level(rfd::MessageLevel::Error)
.set_title(title)
.set_description(text)
.set_buttons(rfd::MessageButtons::Ok)
.show();
}
#[cfg(test)]
@@ -568,6 +699,24 @@ mod tests {
d
}
/// Issue #374: launch arguments that would walk a whole drive or home folder are reported,
/// not imported; empty ones are ignored; the rest are imported as absolute paths.
#[test]
fn launch_arguments_never_walk_a_drive_or_the_home_folder() {
let home = dir("launch");
std::fs::create_dir_all(home.join("Trip")).unwrap();
let args: Vec<String> = ["", " ", ".", "Trip", &home.to_string_lossy()].iter().map(|s| s.to_string()).collect();
let (files, notices) = launch_imports_in(&args, Some(&home), Some(&home));
assert_eq!(files, [home.join("Trip").to_string_lossy().to_string()]);
assert_eq!(notices.len(), 2, "{notices:?}");
assert!(notices.iter().all(|n| n.contains("was not imported")), "{notices:?}");
// the working folder is a drive root (a launcher's choice): `.` is never the whole drive
let root = home.ancestors().last().unwrap().to_path_buf();
let (files, notices) = launch_imports_in(&[".".to_string()], Some(&root), None);
assert!(files.iter().all(|f| f.ends_with("DCIM")) && files.len() + notices.len() == 1, "{files:?} {notices:?}");
let _ = std::fs::remove_dir_all(&home);
}
/// Issue #103: a damaged ui.json (which holds the library location) is kept aside and
/// reported, not silently replaced by the defaults.
#[test]
@@ -575,7 +724,7 @@ mod tests {
let d = dir("damaged");
let path = d.join("ui.json");
std::fs::write(&path, br#"{"settings": {"library_path": "/Volumes/Photos/Lib"#).unwrap();
let (ui, warning, keep) = load_prefs_at(&path);
let (ui, warning, keep) = load_prefs_at(&path, true);
assert!(ui.is_none() && !keep);
let warning = warning.unwrap();
assert!(warning.contains("damaged") && warning.contains("ui.json.corrupt-"), "{warning}");
@@ -583,7 +732,80 @@ mod tests {
let kept: Vec<_> = std::fs::read_dir(&d).unwrap().map(|e| e.unwrap().file_name().to_string_lossy().to_string()).collect();
assert!(kept.len() == 1 && kept[0].starts_with("ui.json.corrupt-"), "{kept:?}");
// a missing file is just "no settings yet"
assert_eq!(load_prefs_at(&path).1, None);
assert_eq!(load_prefs_at(&path, true).1, None);
let _ = std::fs::remove_dir_all(&d);
}
/// Issue #164: a `--memory` session reads the settings but writes nothing — a damaged file is
/// reported and left where it is, not set aside.
#[test]
fn memory_session_leaves_a_damaged_ui_json_alone() {
let d = dir("damaged-memory");
let path = d.join("ui.json");
let bytes = br#"{"settings": {"library_path": "/Volumes/Photos/Lib"#;
std::fs::write(&path, bytes).unwrap();
let (ui, warning, keep) = load_prefs_at(&path, false);
assert!(ui.is_none() && keep);
let warning = warning.unwrap();
assert!(warning.contains("damaged") && warning.contains("nothing is saved"), "{warning}");
assert_eq!(std::fs::read(&path).unwrap(), bytes);
assert_eq!(std::fs::read_dir(&d).unwrap().count(), 1);
let _ = std::fs::remove_dir_all(&d);
}
/// Issue #164: the writer saves changed settings to its file — and nothing at all without one
/// (`--memory`) or while the session is the temporary one offered when the library can't open.
#[test]
fn prefs_writer_writes_nothing_in_a_memory_or_temporary_session() {
let d = dir("writer");
let path = d.join("sub").join("ui.json");
let mut ui = UiState::default();
// an ordinary session: a change is written, the folder created on the way
let mut w = PrefsWriter { path: Some(path.clone()), ..Default::default() };
ui.thumb_size += 50.0;
w.save_ui(&ui, false).unwrap();
let saved = std::fs::read(&path).unwrap();
assert!(String::from_utf8_lossy(&saved).contains("thumbSize"), "{}", String::from_utf8_lossy(&saved));
// the temporary session (library problem): a further change is not written…
ui.thumb_size += 50.0;
w.save_ui(&ui, true).unwrap();
assert_eq!(std::fs::read(&path).unwrap(), saved);
// …until a library opens
w.save_ui(&ui, false).unwrap();
assert_ne!(std::fs::read(&path).unwrap(), saved);
// --memory: no path, so no file and no folder, whatever changes
let d2 = dir("writer-memory");
let _ = std::fs::remove_dir_all(&d2);
let mut w = PrefsWriter { path: None, ..Default::default() };
ui.thumb_size += 50.0;
w.save_ui(&ui, false).unwrap();
assert!(w.written.is_empty() && !d2.exists());
let _ = std::fs::remove_dir_all(&d);
}
/// Issue #169: a `--memory` session neither arms the GPU crash sentinel (which would create
/// the settings folder) nor removes a marker a crashed launch left; it still reports it, and
/// the next ordinary launch finds it, reports it and clears it.
#[test]
fn memory_session_leaves_the_gpu_marker_and_folder_alone() {
let d = dir("gpu-marker-memory");
let _ = std::fs::remove_dir_all(&d);
let m = d.join("gpu-init.marker");
// nothing left behind: no notice, and the folder is not created
assert_eq!(gpu_crash_check_at(Some(m.clone()), true), None);
assert!(!d.exists(), "a --memory session must not create the settings folder");
// a marker left by a crashed launch: reported, kept
std::fs::create_dir_all(&d).unwrap();
std::fs::write(&m, "GPU device creation started (backends METAL)").unwrap();
let notice = gpu_crash_check_at(Some(m.clone()), true).unwrap();
assert!(notice.contains("GPU rendering is now off"), "{notice}");
assert!(m.exists(), "the marker stays for the next ordinary launch");
// the next ordinary launch reports it once and removes it
assert!(gpu_crash_check_at(Some(m.clone()), false).is_some());
lightcraft_engine::gpu::backend::set_init_marker(None);
assert!(!m.exists());
assert_eq!(gpu_crash_check_at(Some(m.clone()), false), None);
lightcraft_engine::gpu::backend::set_init_marker(None);
let _ = std::fs::remove_dir_all(&d);
}
@@ -616,6 +838,27 @@ mod tests {
assert!(!b.is_empty());
}
/// Issue #234: Windows opens links with the URL protocol handler (the default browser), not
/// File Explorer, and passes the whole map link, `?`, `&` and `#` included, as one argument.
#[test]
fn windows_links_open_in_the_browser() {
let url = "https://www.openstreetmap.org/?mlat=48.856600&mlon=2.352200#map=15/48.856600/2.352200";
let c = windows_open_url_command(url);
assert_eq!(c.get_program(), "rundll32");
let args: Vec<&std::ffi::OsStr> = c.get_args().collect();
assert_eq!(args, ["url.dll,FileProtocolHandler", url]);
}
#[test]
fn a_control_port_that_is_not_a_number_is_ignored() {
assert_eq!(control_port_from("--control", Some("7980".into())), Some(7980));
assert_eq!(control_port_from("--control", Some(" 18001 ".into())), Some(18001));
assert_eq!(control_port_from("--control", Some("nope".into())), None);
assert_eq!(control_port_from("--control", Some(String::new())), None);
assert_eq!(control_port_from("--control", Some("70000".into())), None);
assert_eq!(control_port_from("LIGHTCRAFT_CONTROL_PORT", None), None);
}
#[test]
fn ui_json_is_written_atomically() {
let d = dir("atomic");
@@ -630,7 +873,7 @@ mod tests {
assert_eq!(std::fs::read(&path).unwrap(), b"{\"new\": true}");
let ui = UiState::default();
write_atomic(&path, &serde_json::to_vec_pretty(&ui).unwrap()).unwrap();
let (loaded, warning, _) = load_prefs_at(&path);
let (loaded, warning, _) = load_prefs_at(&path, true);
assert!(loaded.is_some() && warning.is_none());
let _ = std::fs::remove_dir_all(&d);
}
+66 -14
View File
@@ -2,9 +2,12 @@
//! the in-window menus ([`lightcraft_ui_egui::menubar`]): the registry's menu paths, shortcuts,
//! enabled and checked state, and live labels ("Undo Exposure").
//!
//! Key handling: a native key equivalent consumes the key press before egui sees it, so
//! - shortcuts installed natively are listed in `LightcraftApp::native_shortcuts` and skipped by
//! the egui shortcut handler (nothing fires twice);
//! Key handling: AppKit offers a key press to the menu bar only when it carries ⌘ or ⌃, or is a
//! function key (F1…). Every other key (`E`, `1`, `⇧P`, `⌥Y`…) goes straight to the window, whose
//! winit view always takes it, so a menu item never fires from it. Hence
//! - shortcuts the menu bar really receives ([`menu_delivers`]) are listed in
//! `LightcraftApp::native_shortcuts` and skipped by the egui shortcut handler (nothing fires
//! twice); the others stay on their menu items for display and egui runs them;
//! - while a text field has keyboard focus, accelerators without ⌘ (`G`, `1`, `Delete`…) and the
//! text-editing ones (⌘A/⌘C/⌘V/⌘X/⌘Z) are removed, so typing and text editing work; they come
//! back when the field loses focus.
@@ -51,6 +54,9 @@ pub struct NativeMenu {
/// Structure of the installed menu (ids, kinds, submenu names); a change rebuilds it.
structure: String,
text_focus: bool,
/// The keyboard shortcuts editor is recording a key: every accelerator is off so the key
/// press reaches it.
capturing: bool,
}
/// `Cmd+Shift+Z` → a muda accelerator (`None` for keys we leave to egui, e.g. Escape).
@@ -146,6 +152,18 @@ fn yields_to_text(sc: &str) -> bool {
!sc.contains("Cmd") && !sc.contains("Ctrl") || TEXT_EDIT.contains(&sc)
}
/// Whether AppKit hands this shortcut to the menu bar: only key presses with ⌘ or ⌃, and function
/// keys. A plain `C` or `⇧P` never reaches it (the window's view takes the key), so egui has to run
/// those even though their menu item shows the key.
fn menu_delivers(sc: &str) -> bool {
sc.split('+').any(|part| matches!(part, "Cmd" | "Ctrl") || part.strip_prefix('F').is_some_and(|n| n.parse::<u8>().is_ok()))
}
/// The shortcuts the menu bar runs itself, for `LightcraftApp::native_shortcuts`.
fn owned_by_menu<'a>(installed: impl Iterator<Item = &'a str>, text_focus: bool) -> HashSet<String> {
installed.filter(|sc| menu_delivers(sc) && !(text_focus && yields_to_text(sc))).map(str::to_string).collect()
}
/// `&` marks a mnemonic in muda labels.
fn label_text(s: &str) -> String {
lightcraft_ui_egui::i18n::tr(s).replace('&', "&&")
@@ -196,7 +214,7 @@ impl NativeMenu {
let _ = tx.send(e.id.0);
repaint.request_repaint();
}));
let mut m = NativeMenu { menu: Menu::new(), items: HashMap::new(), rx, structure: String::new(), text_focus: false };
let mut m = NativeMenu { menu: Menu::new(), items: HashMap::new(), rx, structure: String::new(), text_focus: false, capturing: false };
m.rebuild(app);
app.native_menu = true;
m
@@ -266,6 +284,7 @@ impl NativeMenu {
}
self.menu.init_for_nsapp();
self.text_focus = false;
self.capturing = false;
self.publish_shortcuts(app);
}
@@ -329,13 +348,9 @@ impl NativeMenu {
/// Tell the egui shortcut handler which shortcuts the menu bar currently owns.
fn publish_shortcuts(&self, app: &mut LightcraftApp) {
app.native_shortcuts = self
.items
.values()
.filter(|i| i.accel.is_some())
.filter_map(|i| i.shortcut.clone())
.filter(|sc| !(self.text_focus && yields_to_text(sc)))
.collect::<HashSet<_>>();
let installed = self.items.values().filter(|i| i.accel.is_some()).filter_map(|i| i.shortcut.as_deref());
// (none while the keymap editor records a shortcut: every key goes to it)
app.native_shortcuts = if self.capturing { HashSet::new() } else { owned_by_menu(installed, self.text_focus) };
app.native_shortcuts.insert(SETTINGS_KEY.to_string());
}
@@ -397,14 +412,16 @@ impl NativeMenu {
walk(&bar.iter().flat_map(|(_, v)| v.clone()).collect::<Vec<_>>(), &mut self.items);
let focus = ctx.egui_wants_keyboard_input();
if focus != self.text_focus {
let capturing = app.recording_shortcut.is_some();
if focus != self.text_focus || capturing != self.capturing {
self.text_focus = focus;
self.capturing = capturing;
for it in self.items.values() {
let Some(sc) = it.shortcut.as_deref() else { continue };
if it.accel.is_none() || !yields_to_text(sc) {
if it.accel.is_none() {
continue;
}
let accel = if focus { None } else { it.accel };
let accel = if capturing || (focus && yields_to_text(sc)) { None } else { it.accel };
let _ = match &it.handle {
Handle::Plain(h) => h.set_accelerator(accel),
Handle::Check(h) => h.set_accelerator(accel),
@@ -467,4 +484,39 @@ mod tests {
assert!(accelerator(sc).is_some(), "{sc}");
}
}
/// Single keys shown in the menu bar (E, C, ⇧P, ratings…) never reach it on macOS, so egui must
/// keep handling them; ⌘ / ⌃ combinations and function keys are the menu bar's own.
#[test]
fn single_key_shortcuts_stay_with_egui() {
assert!(menu_delivers("Cmd+Shift+H") && menu_delivers("Ctrl+H") && menu_delivers("F2") && menu_delivers("Cmd+F11"));
assert!(!menu_delivers("E") && !menu_delivers("Shift+P") && !menu_delivers("Alt+Y") && !menu_delivers("1") && !menu_delivers("F"));
let app = LightcraftApp::new(lightcraft_engine::Session::with_demo(), Default::default());
fn all(n: &[MenuNode], out: &mut Vec<String>) {
for x in n {
match x {
MenuNode::Item { shortcut: Some(s), .. } if accelerator(s).is_some() => out.push(s.clone()),
MenuNode::Submenu { children, .. } => all(children, out),
_ => {}
}
}
}
let mut installed = Vec::new();
for (_, v) in menu_bar(&app) {
all(&v, &mut installed);
}
for sc in ["E", "C", "Shift+P", "Cmd+Shift+H", "F2"] {
assert!(installed.iter().any(|s| s == sc), "{sc} is in the menu bar");
}
let owned = owned_by_menu(installed.iter().map(String::as_str), false);
for sc in ["E", "C", "Shift+P", "0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "P", "U"] {
assert!(installed.iter().any(|s| s == sc), "{sc} is displayed in the menu");
assert!(!owned.contains(sc), "{sc} must be left to egui");
}
for sc in ["Cmd+Shift+H", "F2"] {
assert!(owned.contains(sc), "{sc} is run by the menu bar");
}
// while typing, the menu bar gives up F2 too (the text field has it)
assert!(!owned_by_menu(installed.iter().map(String::as_str), true).contains("F2"));
}
}
+2
View File
@@ -11,6 +11,8 @@ covers a whole directory), or if a licence file referenced in the Licence column
| `assets/fonts/Inter-*.ttf` (Regular, Medium, SemiBold) | Inter | Rasmus Andersson | https://github.com/rsms/inter | SIL Open Font License 1.1 (`assets/fonts/OFL-Inter.txt`) | 2026-09-30 | none |
| `assets/app-icon/` (`lightcraft.svg` and the PNG/ICNS/ICO/SVG files derived from it) | LightCraft app icon (lynx) | LightCraft project owner (drawn in ArtCraft) | original work (vectorised from the owner's drawing; see `assets/app-icon/README.md`) | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) | 2026-10-02 | vectorised to SVG; rendered to PNG/ICNS/ICO by `packaging/icons.sh` |
| `assets/camera-profiles/*.json` | Camera colour profiles (matrix + hue/saturation/value table), built into LightCraft | ILCE-7M4: Harald Hoyer | original work: fitted by `lightcraft-cli calibrate` to the contributor's own raw photos and their in-camera JPEGs (597 ILCE-7M4 photos, 2026); aggregate colour statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 | 2026-10-07 | none |
| `packaging/macos/dmg/` (`background.svg`, the `background.tiff` rendered from it by `generate.py`, and the `dmg-layout.DS_Store` layout file) | LightCraft macOS DMG window background | @XusBadia | original work for this repository, derived from the LightCraft app icon (`assets/app-icon/lightcraft.svg`, referenced via `<image>`, not copied); text is outlined Inter / JetBrains Mono glyph paths | MIT OR Apache-2.0 | 2026-10-08 | none (original); used for the Finder window of the macOS DMG, `packaging/macos/dmg/generate.py` renders it to `background.tiff` |
| `assets/camera-profiles/X-H2S.json`, `assets/camera-profiles/X-T4.json` | Fujifilm camera colour profiles (matrix + hue/saturation/value table), built into LightCraft | LightCraft contributors; source photographs supplied by Sebastian Pfluegelmeier | original work: fitted by `lightcraft-cli calibrate` to 201 X-H2S and 545 X-T4 RAFs and their in-camera JPEGs; aggregate colour statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 | 2026-10-08 | none |
| `crates/scenes/` (generated images) | Procedural demo photographs | LightCraft contributors | original work (generated at runtime, no source imagery) | MIT OR Apache-2.0 | 2026-09-30 | n/a |
| `crates/ui-egui/src/icons.rs` | UI icons drawn as vector paths in code | LightCraft contributors | original work | MIT OR Apache-2.0 | 2026-09-30 | n/a |
| `docs/images/*-tetons.jpg`, `docs/images/grid-pd.jpg` (screenshots containing it) | "The Tetons and the Snake River" | Ansel Adams (1942), U.S. National Archives | https://commons.wikimedia.org/wiki/File:Adams_The_Tetons_and_the_Snake_River.jpg | Public domain (U.S. federal government work) | 2026-09-30 | developed in LightCraft; shown inside app screenshots |
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+226
View File
@@ -0,0 +1,226 @@
//! The library's folder tree: where on disk the imported photos live.
//!
//! A *folder* here is a folder holding at least one photo of the library (imported, not deleted,
//! not merely browsed in Local; demo scenes have no folder). The tree lists those folders and the
//! ones above them, never a folder that holds nothing of the library, with how many photos each
//! holds — its subfolders included, like a year in By Date counts its months.
//!
//! Every folder is a row of its own, the ones above the imported photos included
//! (`/Users` → `me` → `Pictures`), so a row is exactly the folder it names. A folder spelled two
//! ways (`/` or `\`, repeated or trailing separators, `.`) is one folder (see
//! [`crate::query::folder_key`]).
//!
//! Each disk is a *volume*, a top-level row of its own: a drive letter (`D:`), a Windows share
//! (`//server/share`), a mounted disk or share (macOS `/Volumes/tokyo`, Linux `/media/me/usb`,
//! `/mnt/disk`, `/run/user/1000/gvfs/smb-share:…`, `/net/host`) or, for everything else, the
//! startup disk (`/`). Only the spelling of the stored paths tells them apart: no disk is
//! touched, so a share that is asleep or gone lists and counts like one that is there, and the
//! sidebar never waits on the network.
//!
//! Limits of reading only the stored spelling of a path: file names are compared as written
//! (Windows folds case, other systems do not), so on a case-insensitive disk `/Volumes/tokyo` and
//! `/Volumes/TOKYO` are two volumes here; and one place reached by two spellings
//! (`/Volumes/Macintosh HD/Users/me` and `/Users/me`, `/media/me/usb` and `/run/media/me/usb`) is
//! two rows. Each row is still consistent: choosing it shows exactly the photos it counts.
//!
//! This is the library's own view of its photos; browsing any folder on disk without importing
//! is Local's job (see [`crate::local`]).
use std::collections::{BTreeMap, HashMap};
use serde::Serialize;
use crate::query::{folder_key, folder_within, split_path};
use crate::{Catalog, Source};
/// Folder levels kept per photo. No real path comes near it; a deeper one is counted at its
/// ancestor this many levels down, which bounds the tree's depth (and so every walk of it)
/// whatever a catalog holds.
pub const MAX_DEPTH: usize = 64;
/// One row of the library's folder tree.
#[derive(Clone, Debug, PartialEq, Serialize)]
pub struct FolderNode {
/// What the row says: the folder's name; for a volume, its name (`nas`, `D:`,
/// `//server/share`, `/` for the startup disk).
pub name: String,
/// The folder the row stands for, written out from the names of the photos' paths (forward
/// slashes, no verbatim prefix); hand it back to [`crate::Filter::library_folder`] to see its
/// photos.
pub path: String,
/// Photos in this folder or any folder inside it.
pub count: usize,
/// Photos directly in this folder.
pub own: usize,
/// A disk rather than a folder in one; its `path` is where it is mounted.
pub volume: bool,
/// Whether choosing the row (`Filter::library_folder`) shows exactly its `count` photos. Not
/// so for the startup disk and for folders that hold other disks (`/Volumes`, `/mnt`…): their
/// path covers those disks' photos too, so choosing them is not offered.
pub selectable: bool,
pub children: Vec<FolderNode>,
}
/// A folder while the tree is being built (children by identity key).
struct Level {
name: String,
path: String,
own: usize,
volume: bool,
children: Vec<String>,
}
/// Where a stored folder path lies: its volume and the folder names below it.
struct Placed {
mount: String,
volume: String,
names: Vec<String>,
}
/// A gvfs mount's name made readable: `smb-share:server=tokyo,share=photos` → `photos on tokyo`;
/// any other (`sftp:host=nas`) as it is.
fn mount_name(n: &str) -> String {
let field = |key: &str| {
let rest = n.strip_prefix("smb-share:")?;
rest.split(',').find_map(|kv| kv.strip_prefix(key)?.strip_prefix('=')).filter(|v| !v.is_empty())
};
match (field("share"), field("server")) {
(Some(share), Some(server)) => format!("{share} on {server}"),
_ => n.to_string(),
}
}
/// The volume `dir` lies on and the folder names below it, read the way [`folder_key`] reads a
/// path (so a row and the filter that chooses it never disagree). `None` for a path that is not
/// absolute or names no share.
fn place(dir: &str) -> Option<Placed> {
let sp = split_path(dir, false);
let names: Vec<&str> = sp.parts.iter().map(String::as_str).collect();
let named = |mount: String, volume: &str, used: usize| (mount, volume.to_string(), used);
let (mount, volume, used) = if sp.prefix.starts_with("//") {
// `//server/share`: a server alone (no share) is no place to keep photos
if sp.prefix.ends_with('/') {
return None;
}
named(sp.prefix.clone(), &sp.prefix, 0)
} else if !sp.prefix.is_empty() {
named(sp.prefix.clone(), &sp.prefix, 0)
} else if sp.absolute {
match names.as_slice() {
["Volumes", n, ..] => named(format!("/Volumes/{n}"), n, 2),
["run", "media", u, n, ..] => named(format!("/run/media/{u}/{n}"), n, 4),
["media", u, n, ..] => named(format!("/media/{u}/{n}"), n, 3),
["mnt", n, ..] => named(format!("/mnt/{n}"), n, 2),
// Linux desktops mount network places per user, naming server and share
["run", "user", uid, "gvfs", n, ..] => named(format!("/run/user/{uid}/gvfs/{n}"), &mount_name(n), 5),
// NFS automounts: Linux `/net/<host>`, macOS `/Network/Servers/<host>`
["net", host, ..] => named(format!("/net/{host}"), host, 2),
["Network", "Servers", host, ..] => named(format!("/Network/Servers/{host}"), host, 3),
_ => named("/".to_string(), "/", 0),
}
} else {
return None;
};
Some(Placed { mount, volume, names: names.iter().skip(used).take(MAX_DEPTH).map(|n| n.to_string()).collect() })
}
/// The disk or share (`/Volumes/nas`, `D:`, `/` for the startup disk) a file-backed photo is on,
/// from its stored path alone.
pub fn volume_of(p: &crate::Photo) -> Option<String> {
let Source::File { path } = &p.source else { return None };
let dir = path.rfind(['/', '\\']).and_then(|i| path.get(..i.max(1)))?;
place(dir).map(|pl| folder_key(&pl.mount))
}
/// Whether `path` is a whole disk or share (`/`, `C:\`, `\\srv\share`, `/Volumes/nas`), not a
/// folder in one.
pub fn is_disk_root(path: &str) -> bool {
place(path).is_some_and(|p| p.names.is_empty())
}
/// Whether `path` is the startup disk's root (`/`), whose path also covers every other disk.
pub fn is_startup_disk(path: &str) -> bool {
place(path).is_some_and(|p| p.names.is_empty() && p.mount == "/")
}
/// A folder named by its last two names (`photos/travel`), so two folders called `travel` are
/// told apart; the path itself when it has no names (`/`).
pub fn folder_label(path: &str) -> String {
let names: Vec<&str> = path.split(['/', '\\']).filter(|n| !n.is_empty()).collect();
if names.is_empty() { path.to_string() } else { names.iter().skip(names.len().saturating_sub(2)).copied().collect::<Vec<_>>().join("/") }
}
impl Catalog {
/// The library's volumes with their folders and photo counts: volumes by name
/// (case-insensitive), each folder with its subfolders the same way.
pub fn folder_tree(&self) -> Vec<FolderNode> {
// photos per containing folder, as spelled (sorted, so the spelling that names a folder
// never depends on hash order)
let mut dirs: BTreeMap<&str, usize> = BTreeMap::new();
for p in self.photos().filter(|p| p.in_library()) {
if let Source::File { path } = &p.source
&& let Some(i) = path.rfind(['/', '\\'])
// a photo straight in the root has an empty folder part: the root itself
&& let Some(dir) = path.get(..i.max(1))
{
*dirs.entry(dir).or_default() += 1;
}
}
let mut levels: HashMap<String, Level> = HashMap::new();
let mut roots: Vec<String> = Vec::new();
for (dir, n) in dirs {
let Some(placed) = place(dir) else { continue };
let vkey = folder_key(&placed.mount);
if !levels.contains_key(&vkey) {
levels.insert(
vkey.clone(),
Level { name: placed.volume.clone(), path: placed.mount.clone(), own: 0, volume: true, children: Vec::new() },
);
roots.push(vkey.clone());
}
let mut prefix = if placed.mount == "/" { String::new() } else { placed.mount.clone() };
let mut parent = vkey;
for name in &placed.names {
prefix.push('/');
prefix.push_str(name);
let key = folder_key(&prefix);
if !levels.contains_key(&key) {
levels.insert(key.clone(), Level { name: name.clone(), path: prefix.clone(), own: 0, volume: false, children: Vec::new() });
if let Some(l) = levels.get_mut(&parent) {
l.children.push(key.clone());
}
}
parent = key;
}
if let Some(l) = levels.get_mut(&parent) {
l.own += n;
}
}
let mut tree: Vec<FolderNode> = roots.iter().filter_map(|k| build(k, &levels)).collect();
sort(&mut tree);
// a folder that holds another disk's mount covers that disk's photos too
let mounts: Vec<String> = tree.iter().filter(|v| v.path != "/").map(|v| v.path.clone()).collect();
mark_selectable(&mut tree, &mounts);
tree
}
}
/// The node for `key` with everything below it (at most [`MAX_DEPTH`] deep).
fn build(key: &str, levels: &HashMap<String, Level>) -> Option<FolderNode> {
let l = levels.get(key)?;
let mut children: Vec<FolderNode> = l.children.iter().filter_map(|k| build(k, levels)).collect();
sort(&mut children);
let count = l.own + children.iter().map(|c| c.count).sum::<usize>();
Some(FolderNode { name: l.name.clone(), path: l.path.clone(), count, own: l.own, volume: l.volume, selectable: true, children })
}
fn mark_selectable(nodes: &mut [FolderNode], mounts: &[String]) {
for n in nodes {
n.selectable = !(n.volume && n.path == "/") && (n.volume || !mounts.iter().any(|m| folder_within(m, &n.path)));
mark_selectable(&mut n.children, mounts);
}
}
fn sort(v: &mut [FolderNode]) {
v.sort_by(|a, b| a.name.to_lowercase().cmp(&b.name.to_lowercase()).then_with(|| a.name.cmp(&b.name)));
}
+20
View File
@@ -14,6 +14,7 @@
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
pub mod dates;
pub mod folders;
pub mod journal;
pub mod keywords;
pub mod local;
@@ -29,6 +30,7 @@ use std::collections::BTreeMap;
use std::sync::Arc;
pub use dates::{DateRun, GroupBy};
pub use folders::FolderNode;
pub use journal::{Journal, LoadReport, PersistStats, SnapshotPolicy, SnapshotTiming};
pub use keywords::KeywordNode;
use lightcraft_develop::DevelopSettings;
@@ -612,6 +614,22 @@ impl Catalog {
Op::Batch { ops }
}
/// [`Self::delete_permanently_ops`] for several photos at once. One op list built against the
/// catalog as it is now: albums lose all of them in one write each, and stacks are shortened
/// (or dissolved) once, however many of their photos go.
pub fn delete_photos_permanently_ops(&self, ids: &[PhotoId]) -> Op {
let gone: std::collections::HashSet<PhotoId> = ids.iter().copied().collect();
let mut ops: Vec<Op> = self
.albums
.values()
.filter(|a| a.photos.iter().any(|p| gone.contains(p)))
.map(|a| Op::SetAlbumPhotos { id: a.id, photos: a.photos.iter().copied().filter(|p| !gone.contains(p)).collect() })
.collect();
ops.extend(self.remove_from_stacks_ops(ids));
ops.extend(gone.iter().map(|id| Op::RemovePhoto { id: *id }));
Op::Batch { ops }
}
// ---- persistence
/// Full snapshot as JSON.
@@ -655,6 +673,8 @@ mod tests;
#[cfg(test)]
mod tests_background;
#[cfg(test)]
mod tests_folders;
#[cfg(test)]
mod tests_format_version;
#[cfg(test)]
mod tests_journal;
+18 -5
View File
@@ -2,7 +2,7 @@
use std::sync::Arc;
use lightcraft_develop::DevelopSettings;
use lightcraft_develop::{DevelopSettings, WbMode};
use serde::{Deserialize, Serialize};
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
@@ -336,12 +336,12 @@ impl Photo {
pub fn develops_raw(&self) -> bool {
self.kind == MediaKind::Raw && self.preview_only.is_none()
}
/// The current ARW, NEF and RW2 readers have vendor WB multipliers but no measured camera
/// The current ARW, NEF, RW2, RAF and CR3 readers have vendor WB multipliers but no measured camera
/// illuminant. Use adjustments relative to the camera's as-shot look, as for rendered
/// photographs (the engine's `camera_preview::file_local_look` covers the same formats; RWL and
/// RAW are Leica's and the oldest Panasonic bodies' names for RW2 files).
pub fn relative_wb(&self) -> bool {
self.develops_raw() && ["ARW", "NEF", "NRW", "RW2", "RWL", "RAW"].iter().any(|f| self.format.eq_ignore_ascii_case(f))
self.develops_raw() && ["ARW", "NEF", "NRW", "RW2", "RWL", "RAW", "RAF", "CR3"].iter().any(|f| self.format.eq_ignore_ascii_case(f))
}
/// The develop settings import gave this photo: [`Photo::camera_defaults`], or the user's
/// default preset applied on top of them ([`Photo::import_look`]).
@@ -370,7 +370,12 @@ impl Photo {
}
/// Edited by the user: settings differ from what import gave the photo.
pub fn is_edited(&self) -> bool {
*self.develop != self.import_defaults() && !self.develop.is_unedited()
let mut defaults = self.import_defaults();
// As Shot reads the source's current WB; stored numbers can predate a decoder update.
if self.develop.wb.mode == WbMode::AsShot && defaults.wb.mode == WbMode::AsShot {
defaults.wb = self.develop.wb;
}
*self.develop != defaults && !self.develop.is_unedited()
}
/// Capture time if known, else import time (sort key).
pub fn date(&self) -> &str {
@@ -430,7 +435,7 @@ mod edited_tests {
}
#[test]
fn sony_and_nikon_raws_use_relative_white_balance() {
fn supported_raws_use_relative_white_balance() {
for (name, format, relative) in [
("a.arw", "ARW", true),
("a.nef", "NEF", true),
@@ -438,6 +443,8 @@ mod edited_tests {
("a.rw2", "RW2", true),
("a.rwl", "RWL", true),
("a.raw", "RAW", true),
("a.raf", "rAf", true),
("a.cr3", "cr3", true),
("a.dng", "DNG", false),
] {
let mut p = Photo::new(PhotoId(1), Source::Demo { scene: 0 }, name, format, 10, 10, "2026-10-01T00:00:00");
@@ -446,6 +453,12 @@ mod edited_tests {
assert_eq!(p.relative_wb(), relative, "{format}");
let wb = p.camera_defaults().wb;
assert_eq!((wb.temp, wb.tint), if relative { (6500.0, 0.0) } else { (5200.0, 4.0) }, "{format}");
p.develop = Arc::new(DevelopSettings::for_raw(5200.0, 4.0));
assert!(!p.is_edited(), "{format}: old As Shot values are not an edit");
let mut custom = (*p.develop).clone();
custom.wb.mode = WbMode::Custom;
p.develop = Arc::new(custom);
assert!(p.is_edited(), "{format}: Custom WB is still an edit");
// shown from the embedded preview: a rendered image, not a relative-WB raw
p.preview_only = Some("unsupported".into());
assert!(!p.relative_wb());
+104 -19
View File
@@ -37,6 +37,10 @@ pub struct Filter {
pub deleted: bool,
/// Capture date prefix (`2026`, `2026-04`, `2026-04-12`).
pub date: Option<String>,
/// A folder the library's photos were imported from: those photos and the ones imported from
/// the folders inside it (see [`crate::folders`]). Photos only browsed in Local are never part
/// of it, unlike [`Filter::folder`].
pub library_folder: Option<String>,
/// A keyword; hierarchical keywords match their children too (`travel` finds `travel|italy`).
pub keyword: Option<String>,
/// A person: photos with a named face region of this name (case-insensitive), as read from XMP.
@@ -186,6 +190,7 @@ impl Filter {
("camera", &self.camera),
("lens", &self.lens),
("date", &self.date),
("folder", &self.library_folder),
("imported", &self.imported),
] {
if let Some(x) = val {
@@ -213,6 +218,19 @@ impl Filter {
}
pub fn matches(&self, p: &Photo, cat: &Catalog) -> bool {
self.matches_in(p, cat, self.library_root().as_deref())
}
/// [`Filter::library_folder`] as a folder identity ([`folder_key`]); `None` without a choice.
fn library_root(&self) -> Option<String> {
// as given: a folder may be named with spaces at its edge, and trimming would make it
// another folder
self.library_folder.as_deref().filter(|d| !d.trim().is_empty()).map(folder_key)
}
/// [`Filter::matches`] with the library folder already turned into its identity, so a query
/// over a big library does that once, not once per photo.
fn matches_in(&self, p: &Photo, cat: &Catalog, root: Option<&str>) -> bool {
if p.deleted != self.deleted {
return false;
}
@@ -285,6 +303,11 @@ impl Filter {
{
return false;
}
if let Some(root) = root
&& !photo_in_root(p, root)
{
return false;
}
if let Some(d) = &self.imported
&& !p.imported.starts_with(d.as_str())
{
@@ -317,7 +340,8 @@ impl Filter {
impl Catalog {
/// Photos matching `filter`, in `sort` order (ties broken by id for stability).
pub fn query(&self, filter: &Filter, sort: &Sort) -> Vec<PhotoId> {
let mut v: Vec<&Photo> = self.photos().map(|p| p.as_ref()).filter(|p| filter.matches(p, self)).collect();
let root = filter.library_root();
let mut v: Vec<&Photo> = self.photos().map(|p| p.as_ref()).filter(|p| filter.matches_in(p, self, root.as_deref())).collect();
v.sort_by(|a, b| {
let o = match sort.key {
SortKey::CaptureDate => a.date().cmp(b.date()),
@@ -384,7 +408,8 @@ impl Catalog {
pub fn people_in(&self, filter: &Filter) -> Vec<Person> {
let filter = Filter { person: None, ..filter.clone() };
let mut m: std::collections::HashMap<String, (Person, f64)> = Default::default();
for p in self.photos().filter(|p| filter.matches(p, self)) {
let root = filter.library_root();
for p in self.photos().filter(|p| filter.matches_in(p, self, root.as_deref())) {
let mut seen: Vec<String> = Vec::new();
for r in p.meta.regions.iter().filter(|r| r.kind == lightcraft_meta::RegionKind::Face) {
let Some(name) = r.name.as_deref().map(str::trim).filter(|n| !n.is_empty()) else { continue };
@@ -438,12 +463,18 @@ pub fn in_folder(path: &str, dir: &str, deep: bool) -> bool {
!rest.is_empty() && (deep || !rest.contains(['/', '\\']))
}
/// A folder path's identity, for telling whether two spellings name the same folder: `/` and
/// `\\` are both separators, repeated separators, `.` and a trailing separator are dropped,
/// `..` is resolved lexically, a Windows verbatim prefix (`\\?\`) is removed and the drive
/// letter lower-cased; on Windows (case-insensitive file names) the whole path is lower-cased.
/// No file-system access: the paths compared should already be absolute.
pub fn folder_key(path: &str) -> String {
/// A path read the way [`folder_key`] reads it: the part `..` can't climb above (`//server/share`,
/// `c:`; empty for `/` and relative paths), the names below it, and whether it is absolute.
pub(crate) struct SplitPath {
pub prefix: String,
pub parts: Vec<String>,
pub absolute: bool,
}
/// Both separators, repeated separators (also between a server and its share), `.` and a
/// trailing separator dropped, `..` resolved lexically, a Windows verbatim prefix removed;
/// `lower_drive` lower-cases a drive letter (the identity does, a display name wants it as is).
pub(crate) fn split_path(path: &str, lower_drive: bool) -> SplitPath {
let mut s = path.replace('\\', "/");
if let Some(rest) = s.strip_prefix("//?/").or_else(|| s.strip_prefix("//./")) {
s = match rest.strip_prefix("UNC/") {
@@ -451,28 +482,39 @@ pub fn folder_key(path: &str) -> String {
None => rest.to_string(),
};
}
// the part `..` can't climb above: `//server/share`, `c:` or `/`
let (prefix, rest) = if let Some(unc) = s.strip_prefix("//").filter(|u| !u.is_empty() && !u.starts_with('/')) {
let mut it = unc.splitn(3, '/');
let (server, share, rest) = (it.next().unwrap_or(""), it.next().unwrap_or(""), it.next().unwrap_or(""));
(format!("//{server}/{share}"), rest.to_string())
} else if s.len() >= 2 && s.as_bytes()[1] == b':' && s.as_bytes()[0].is_ascii_alphabetic() {
(s[..2].to_ascii_lowercase(), s[2..].to_string())
let mut segs = unc.split('/').filter(|x| !x.is_empty());
let (server, share) = (segs.next().unwrap_or(""), segs.next().unwrap_or(""));
(format!("//{server}/{share}"), segs.collect::<Vec<_>>().join("/"))
} else if let (true, Some(drive), Some(rest)) = (matches!(s.as_bytes(), [a, b':', ..] if a.is_ascii_alphabetic()), s.get(..2), s.get(2..)) {
(if lower_drive { drive.to_ascii_lowercase() } else { drive.to_string() }, rest.to_string())
} else {
(String::new(), s.clone())
};
let absolute = rest.starts_with('/') || !prefix.is_empty();
let mut parts: Vec<&str> = Vec::new();
let mut parts: Vec<String> = Vec::new();
for c in rest.split('/') {
match c {
"" | "." => {}
".." if parts.last().is_some_and(|p| *p != "..") => {
".." if parts.last().is_some_and(|p| p != "..") => {
parts.pop();
}
".." if absolute => {}
c => parts.push(c),
c => parts.push(c.to_string()),
}
}
SplitPath { prefix, parts, absolute }
}
/// A folder path's identity, for telling whether two spellings name the same folder: `/` and
/// `\\` are both separators, repeated separators, `.` and a trailing separator are dropped,
/// `..` is resolved lexically, a Windows verbatim prefix (`\\?\`) is removed and the drive
/// letter lower-cased; on Windows (case-insensitive file names) the whole path is lower-cased.
/// File names are otherwise compared as written, so on a case-insensitive disk elsewhere
/// (macOS, a Samba share) two spellings that differ in case are two folders here.
/// No file-system access: the paths compared should already be absolute.
pub fn folder_key(path: &str) -> String {
let SplitPath { prefix, parts, absolute } = split_path(path, true);
let key = if absolute { format!("{prefix}/{}", parts.join("/")) } else { parts.join("/") };
if cfg!(windows) { key.to_lowercase() } else { key }
}
@@ -480,8 +522,51 @@ pub fn folder_key(path: &str) -> String {
/// Whether `path` is folder `root` or lies somewhere inside it, however either is spelled
/// (see [`folder_key`]).
pub fn folder_within(path: &str, root: &str) -> bool {
let (p, r) = (folder_key(path), folder_key(root));
p == r || (p.starts_with(&r) && (r.ends_with('/') || p[r.len()..].starts_with('/')))
key_within(&folder_key(path), &folder_key(root))
}
/// The names below folder `root` that lead to `path`, as the caller spelled them (`None` when
/// `path` is not at or inside `root`; empty when it is `root`). Compared by [`folder_key`], but
/// the names keep their case, which the key lowers on Windows.
pub fn folder_rest(path: &str, root: &str) -> Option<Vec<String>> {
if !key_within(&folder_key(path), &folder_key(root)) {
return None;
}
let depth = split_path(root, false).parts.len();
Some(split_path(path, false).parts.into_iter().skip(depth).collect())
}
/// Whether a library photo (not one only browsed in Local) lies in the folder whose
/// [`folder_key`] is `root`. A plain POSIX path (no `\`, `.`, `..`, repeated or trailing
/// separators) is its own identity, so it is read without building a key for it: the check runs
/// for every photo of the library on each refresh, and this keeps it cheap.
pub(crate) fn photo_in_root(p: &Photo, root: &str) -> bool {
if p.local {
return false;
}
let crate::Source::File { path } = &p.source else { return false };
let plain = path.starts_with('/')
&& !path.starts_with("//")
&& !path.ends_with('/')
&& !path.contains(['\\'])
&& !path.contains("//")
&& !path.contains("/.");
if !cfg!(windows) && plain && root.starts_with('/') && !root.starts_with("//") {
let r = root.trim_end_matches('/');
return path.strip_prefix(r).is_some_and(|rest| rest.starts_with('/'));
}
crate::local::folder_of(p).is_some_and(|f| key_within(&f, root))
}
/// [`folder_within`] for two paths already turned into their [`folder_key`].
pub(crate) fn key_within(p: &str, r: &str) -> bool {
// `.` or `a/..` name no folder: not "everything"
if r.is_empty() {
return false;
}
// a disk's root key ends in `/` (`c:/`, `//srv/share/`), a folder's does not: same folder
let (p, r) = (p.trim_end_matches('/'), r.trim_end_matches('/'));
p == r || p.strip_prefix(r).is_some_and(|rest| rest.starts_with('/'))
}
/// The kind of Photo Merge result a file is, from the name merges give it (`IMG_1-HDR.dng`,
+404
View File
@@ -0,0 +1,404 @@
//! The library's folder tree: where the imported photos live on disk (see [`crate::folders`]).
//!
//! Scenarios, in the words of someone browsing their library:
//!
//! * Given photos imported from several folders, the tree lists those folders and no others,
//! each with how many photos it holds, under the disk (volume) they are on.
//! * A folder's count includes the folders inside it.
//! * Every folder is a row of its own, the ones above the imported photos included.
//! * Photos only browsed in Local, deleted photos and demo scenes are not "imported from" anywhere.
//! * However a folder is spelled, it is one folder.
//! * Choosing a folder shows what was imported from it and from the folders inside it — exactly
//! as many photos as the row says.
use crate::*;
fn add(c: &mut Catalog, path: &str) -> PhotoId {
add_with(c, path, |_| {})
}
fn add_with(c: &mut Catalog, path: &str, tweak: impl FnOnce(&mut Photo)) -> PhotoId {
let id = c.alloc_photo_id();
let name = path.rsplit(['/', '\\']).next().unwrap_or(path);
let mut p = Photo::new(id, Source::File { path: path.into() }, name, "JPEG", 60, 40, "2026-01-01T10:00:00");
tweak(&mut p);
c.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
id
}
fn library(paths: &[&str]) -> Catalog {
let mut c = Catalog::new();
paths.iter().for_each(|p| {
add(&mut c, p);
});
c
}
/// `(name, count, own)` of each node, depth first.
fn outline(nodes: &[FolderNode]) -> Vec<(String, usize, usize)> {
fn walk(n: &FolderNode, out: &mut Vec<(String, usize, usize)>) {
out.push((n.name.clone(), n.count, n.own));
n.children.iter().for_each(|c| walk(c, out));
}
let mut out = Vec::new();
nodes.iter().for_each(|n| walk(n, &mut out));
out
}
fn row(name: &str, count: usize, own: usize) -> (String, usize, usize) {
(name.to_string(), count, own)
}
fn rows(paths: &[&str]) -> Vec<(String, usize, usize)> {
outline(&library(paths).folder_tree())
}
#[test]
fn an_empty_library_has_no_folders() {
assert!(Catalog::new().folder_tree().is_empty());
}
#[test]
fn only_folders_with_imported_photos_are_listed_with_their_counts() {
let c = library(&["/pics/trip/a.jpg", "/pics/trip/b.jpg", "/pics/home/c.jpg"]);
let tree = c.folder_tree();
assert_eq!(outline(&tree), vec![row("/", 3, 0), row("pics", 3, 0), row("home", 1, 1), row("trip", 2, 2)]);
assert!(tree[0].volume && !tree[0].children[0].volume);
assert_eq!(tree[0].children[0].children[1].path, "/pics/trip", "a row knows its folder, for choosing it");
}
#[test]
fn a_folders_count_includes_the_folders_inside_it() {
let r = rows(&["/pics/trip/a.jpg", "/pics/trip/day1/b.jpg", "/pics/trip/day1/c.jpg", "/pics/trip/day2/d.jpg"]);
assert_eq!(r, vec![row("/", 4, 0), row("pics", 4, 0), row("trip", 4, 1), row("day1", 2, 2), row("day2", 1, 1)]);
}
#[test]
fn every_folder_is_a_row_of_its_own_even_when_it_holds_nothing_itself() {
let c = library(&["/Users/me/Pictures/2024/a.jpg", "/Users/me/Pictures/2025/b.jpg", "/Volumes/nas/photos/c.jpg"]);
let tree = c.folder_tree();
assert_eq!(
outline(&tree),
vec![
row("/", 2, 0),
row("Users", 2, 0),
row("me", 2, 0),
row("Pictures", 2, 0),
row("2024", 1, 1),
row("2025", 1, 1),
row("nas", 1, 0),
row("photos", 1, 1)
]
);
assert_eq!(tree[0].children[0].children[0].children[0].path, "/Users/me/Pictures", "a row is exactly the folder it names");
}
#[test]
fn a_folder_with_photos_of_its_own_is_not_folded_into_its_child() {
assert_eq!(rows(&["/pics/a.jpg", "/pics/trip/b.jpg"]), vec![row("/", 2, 0), row("pics", 2, 1), row("trip", 1, 1)]);
}
#[test]
fn browsed_deleted_and_demo_photos_are_not_imported_from_anywhere() {
let mut c = Catalog::new();
add(&mut c, "/pics/kept/a.jpg");
add_with(&mut c, "/pics/browsed/b.jpg", |p| p.local = true);
add_with(&mut c, "/pics/gone/c.jpg", |p| p.deleted = true);
let id = c.alloc_photo_id();
let demo = Photo::new(id, Source::Demo { scene: 1 }, "demo", "JPEG", 60, 40, "2026-01-01T10:00:00");
c.apply(Op::AddPhoto { photo: Box::new(demo) }).unwrap();
assert_eq!(outline(&c.folder_tree()), vec![row("/", 1, 0), row("pics", 1, 0), row("kept", 1, 1)]);
}
#[test]
fn however_a_folder_is_spelled_it_is_one_folder() {
let r = rows(&["/pics/trip/a.jpg", "/pics//trip/./b.jpg", "/pics\\trip\\c.jpg", "/pics/x/../trip/d.jpg"]);
assert_eq!(r, vec![row("/", 4, 0), row("pics", 4, 0), row("trip", 4, 4)]);
}
#[test]
fn each_disk_is_a_volume_of_its_own() {
// macOS, Linux and Windows habits for where disks appear
let r = rows(&["/Volumes/nas/p/1.jpg", "/media/me/usb/2.jpg", "/run/media/me/card/3.jpg", "/mnt/disk/4.jpg"]);
assert_eq!(r, vec![row("card", 1, 1), row("disk", 1, 1), row("nas", 1, 0), row("p", 1, 1), row("usb", 1, 1)]);
let r = rows(&["D:\\Photos\\2024\\a.jpg", "D:\\Photos\\2025\\b.jpg", "C:\\x\\c.jpg"]);
assert_eq!(r, vec![row("C:", 1, 0), row("x", 1, 1), row("D:", 2, 0), row("Photos", 2, 0), row("2024", 1, 1), row("2025", 1, 1)]);
}
#[test]
fn a_network_share_is_a_volume_and_two_spellings_are_one() {
let r = rows(&["\\\\srv\\s1\\a\\1.jpg", "\\\\srv\\s2\\b\\2.jpg"]);
assert_eq!(r, vec![row("//srv/s1", 1, 0), row("a", 1, 1), row("//srv/s2", 1, 0), row("b", 1, 1)]);
// the verbatim form Windows APIs give is the same share
let r = rows(&["\\\\?\\UNC\\srv\\s1\\a\\1.jpg", "\\\\srv\\s1\\a\\2.jpg"]);
assert_eq!(r, vec![row("//srv/s1", 2, 0), row("a", 2, 2)]);
}
#[test]
fn a_samba_share_mounted_on_a_mac_is_a_volume_named_after_the_share() {
// Finder mounts every share under /Volumes, and a second mount of the same share gets a suffix
let r = rows(&["/Volumes/tokyo/Photos/2024/a.jpg", "/Volumes/tokyo/Photos/2025/b.jpg", "/Volumes/tokyo-1/old/c.jpg", "/Volumes/東京 NAS/d.jpg"]);
assert_eq!(
r,
vec![
row("tokyo", 2, 0),
row("Photos", 2, 0),
row("2024", 1, 1),
row("2025", 1, 1),
row("tokyo-1", 1, 0),
row("old", 1, 1),
row("東京 NAS", 1, 1)
]
);
let tree = library(&["/Volumes/tokyo/Photos/a.jpg"]).folder_tree();
assert_eq!((tree[0].path.as_str(), tree[0].volume), ("/Volumes/tokyo", true), "choosing the share shows everything on it");
}
#[test]
fn shares_the_other_systems_mount_are_volumes_too() {
// Linux desktops: gvfs mounts per user, with the server and share in the mount's name
let gvfs = "/run/user/1000/gvfs/smb-share:server=tokyo,share=photos";
let r = rows(&[&format!("{gvfs}/2024/a.jpg"), "/run/user/1000/gvfs/sftp:host=nas/b.jpg"]);
assert_eq!(r, vec![row("photos on tokyo", 1, 0), row("2024", 1, 1), row("sftp:host=nas", 1, 1)]);
// NFS automounts: Linux /net/<host>, macOS /Network/Servers/<host>
let r = rows(&["/net/tokyo/export/a.jpg", "/Network/Servers/osaka/vol/b.jpg"]);
assert_eq!(r, vec![row("osaka", 1, 0), row("vol", 1, 1), row("tokyo", 1, 0), row("export", 1, 1)]);
// Windows: a share by name or by address, the WSL file system, a mapped drive
let r = rows(&[r"\\tokyo.local\photos\a.jpg", r"\\192.168.1.5\scans\b.jpg", r"\\wsl$\Ubuntu\home\me\c.jpg", r"Z:\d.jpg"]);
assert_eq!(
r,
vec![
row("//192.168.1.5/scans", 1, 1),
row("//tokyo.local/photos", 1, 1),
row("//wsl$/Ubuntu", 1, 0),
row("home", 1, 0),
row("me", 1, 1),
row("Z:", 1, 1)
]
);
}
#[test]
fn a_share_that_is_not_mounted_costs_nothing() {
// the tree is made from the catalog's own records: it never looks at a disk, so a share that
// is asleep or gone lists (and counts) exactly like one that is there
let c = library(&["/Volumes/definitely-not-mounted/a/b.jpg", r"\\unreachable.invalid\share\c.jpg"]);
let tree = c.folder_tree();
assert_eq!(tree.iter().map(|n| n.count).sum::<usize>(), 2);
}
#[test]
fn a_photo_in_the_root_of_a_disk_counts_for_the_disk() {
assert_eq!(rows(&["/x.jpg", "/d/y.jpg"]), vec![row("/", 2, 1), row("d", 1, 1)]);
assert_eq!(rows(&["C:\\x.jpg"]), vec![row("C:", 1, 1)]);
}
#[test]
fn photos_with_no_folder_are_left_out() {
assert!(rows(&["x.jpg", ""]).is_empty());
}
#[test]
fn folders_are_listed_by_name_ignoring_case() {
let names: Vec<String> = rows(&["/p/b/1.jpg", "/p/A/2.jpg", "/p/c/3.jpg"]).into_iter().skip(2).map(|r| r.0).collect();
assert_eq!(names, ["A", "b", "c"]);
}
#[test]
fn absurdly_deep_paths_neither_crash_nor_lose_photos() {
let deep = format!("/{}/a.jpg", vec!["d"; 5000].join("/"));
let tree = library(&[&deep]).folder_tree();
assert_eq!(tree.iter().map(|n| n.count).sum::<usize>(), 1);
assert!(outline(&tree).len() <= crate::folders::MAX_DEPTH + 2);
}
#[test]
fn choosing_a_folder_shows_what_was_imported_from_it_and_below() {
let mut c = Catalog::new();
let trip = add(&mut c, "/pics/trip/a.jpg");
let day = add(&mut c, "/pics/trip/day1/b.jpg");
let sibling = add(&mut c, "/pics/trip2/c.jpg"); // shares the name's start, not the folder
let browsed = add_with(&mut c, "/pics/trip/d.jpg", |p| p.local = true);
let f = Filter { library_folder: Some("/pics//trip/".into()), ..Default::default() };
let shown = |id: PhotoId| f.matches(c.photo(id).unwrap(), &c);
assert!(shown(trip) && shown(day));
assert!(!shown(sibling), "/pics/trip2 is another folder");
assert!(!shown(browsed), "browsed photos are not part of the library's folders");
assert_eq!(f.describe(), "folder /pics//trip/");
}
#[test]
fn recently_deleted_shows_the_deleted_photos_of_the_chosen_folder() {
let mut c = Catalog::new();
let gone = add_with(&mut c, "/pics/trip/a.jpg", |p| p.deleted = true);
let kept = add(&mut c, "/pics/trip/b.jpg");
let f = Filter { library_folder: Some("/pics/trip".into()), deleted: true, ..Default::default() };
assert!(f.matches(c.photo(gone).unwrap(), &c));
assert!(!f.matches(c.photo(kept).unwrap(), &c));
}
#[test]
fn an_empty_folder_choice_is_no_choice() {
let c = library(&["/pics/a.jpg"]);
let f = Filter { library_folder: Some(" ".into()), ..Default::default() };
assert_eq!(c.query(&f, &Sort::default()).len(), 1);
}
#[test]
fn every_row_shows_exactly_as_many_photos_as_it_says() {
let c = library(&[
"/Users/me/Pictures/2024/a.jpg",
"/Users/me/Pictures/2024/b.jpg",
"/Users/me/Pictures/2025/c.jpg",
"/Users/me/Pictures/d.jpg",
"/Volumes/nas/photos/e.jpg",
"/Volumes/nas/f.jpg",
"/x.jpg",
"/a/../b/g.jpg",
"D:\\Photos\\2024\\h.jpg",
"d:/Photos/i.jpg",
"\\\\srv\\s1\\a\\j.jpg",
"\\\\?\\UNC\\srv\\s2\\k.jpg",
"/pics/trip/l.jpg",
"/pics/trip2/m.jpg",
"/Volumes/tokyo/Photos/2024/n.jpg",
"/Volumes/tokyo/o.jpg",
"/run/user/1000/gvfs/smb-share:server=tokyo,share=photos/p.jpg",
"/net/osaka/export/q.jpg",
r"\\wsl$\Ubuntu\home\me\r.jpg",
// odd spellings the tree and the filter must read the same way
"//server/share/../other/1.jpg",
"//server/share/ok/2.jpg",
"//a b//A/f.jpg",
r"\\wsl$\Ubuntu\..\a\1.jpg",
// a folder that holds other disks as well as photos of its own
"/Volumes/1.jpg",
"/mnt/2.jpg",
"/mnt/disk/3.jpg",
]);
fn check(c: &Catalog, nodes: &[FolderNode], seen: &mut usize) {
for n in nodes {
// the startup volume's row only opens: its path "/" would cover every other disk
if n.selectable {
let f = Filter { library_folder: Some(n.path.clone()), ..Default::default() };
assert_eq!(c.query(&f, &Sort::default()).len(), n.count, "{} ({})", n.name, n.path);
*seen += 1;
}
check(c, &n.children, seen);
}
}
let mut seen = 0;
check(&c, &c.folder_tree(), &mut seen);
assert!(seen > 10, "{seen} rows checked");
}
fn find<'a>(nodes: &'a [FolderNode], path: &str) -> Option<&'a FolderNode> {
nodes.iter().find_map(|n| if n.path == path { Some(n) } else { find(&n.children, path) })
}
#[test]
fn a_row_that_would_cover_other_disks_is_not_offered_for_choosing() {
let tree = library(&["/Users/me/a.jpg", "/Volumes/1.jpg", "/Volumes/tokyo/x.jpg", "/mnt/2.jpg"]).folder_tree();
let sel = |p: &str| find(&tree, p).map(|n| n.selectable);
assert_eq!(sel("/"), Some(false), "the startup disk's path covers every disk");
assert_eq!(sel("/Volumes"), Some(false), "it holds the tokyo disk");
assert_eq!(sel("/Volumes/tokyo"), Some(true));
assert_eq!(sel("/Users/me"), Some(true));
assert_eq!(sel("/mnt"), Some(true), "no disk is mounted below it in this library");
}
#[test]
fn the_tree_and_the_filter_read_odd_spellings_the_same_way() {
// `..` cannot climb above a share, and repeated separators between server and share are one
let tree = library(&["//server/share/../other/1.jpg", "//a b//A/f.jpg"]).folder_tree();
assert_eq!(outline(&tree), vec![row("//a b/A", 1, 1), row("//server/share", 1, 0), row("other", 1, 1)]);
}
#[test]
fn a_path_that_names_no_folder_chooses_nothing() {
let c = library(&["/pics/a.jpg", "C:\\b\\c.jpg"]);
for dot in [".", "./", "a/..", "../.."] {
let f = Filter { library_folder: Some(dot.into()), ..Default::default() };
assert_eq!(c.query(&f, &Sort::default()).len(), 0, "{dot:?} is no folder, not everything");
}
}
#[test]
fn a_folder_is_labelled_by_its_last_two_names() {
use crate::folders::folder_label;
assert_eq!(folder_label("/Volumes/tokyo/photos/travel"), "photos/travel");
assert_eq!(folder_label("/Volumes/tokyo/"), "Volumes/tokyo");
assert_eq!(folder_label(r"C:\Photos\2024"), "Photos/2024");
assert_eq!(folder_label("/"), "/");
assert_eq!(folder_label(""), "");
}
#[test]
fn disk_roots_are_told_from_folders() {
use crate::folders::{is_disk_root, is_startup_disk};
for p in ["/", "C:", "C:\\", r"\\?\C:\", r"\\srv\share", "/Volumes/nas", "/mnt/disk", "/net/host"] {
assert!(is_disk_root(p), "{p}");
}
for p in ["/Volumes", "/Users", "C:\\Photos", r"\\srv\share\x", "/Volumes/nas/p", "."] {
assert!(!is_disk_root(p), "{p}");
}
assert!(is_startup_disk("/") && is_startup_disk("//") && !is_startup_disk("/Volumes/nas") && !is_startup_disk("C:"));
}
#[test]
fn a_folder_name_with_spaces_at_its_edge_is_its_own_folder() {
// legal on macOS and Linux: `/p/shoot ` is not `/p/shoot`
let c = library(&["/p/shoot /a.jpg", "/p/shoot/b.jpg", "/p/other/c.jpg"]);
let tree = c.folder_tree();
let shoot_space = find(&tree, "/p/shoot ").expect("its own row");
let shoot = find(&tree, "/p/shoot").expect("and the other");
for (node, want) in [(shoot_space, "a.jpg"), (shoot, "b.jpg")] {
let f = Filter { library_folder: Some(node.path.clone()), ..Default::default() };
let ids = c.query(&f, &Sort::default());
let names: Vec<&str> = ids.iter().filter_map(|id| c.photo(*id)).map(|p| p.file_name.as_str()).collect();
assert_eq!(names, [want], "{:?} shows its own photo", node.path);
}
}
#[test]
fn a_local_photo_never_counts_as_imported_even_when_a_disk_folder_is_also_shown() {
let mut c = Catalog::new();
let browsed = add_with(&mut c, "/pics/trip/b.jpg", |p| p.local = true);
let f = Filter { folder: Some("/pics/trip".into()), library_folder: Some("/pics/trip".into()), ..Default::default() };
assert!(!f.matches(c.photo(browsed).unwrap(), &c));
}
#[test]
fn the_quick_check_agrees_with_the_careful_one() {
use crate::query::{folder_key, key_within, photo_in_root};
let paths = [
"/a/b/c.jpg",
"/a/b/d/e.jpg",
"/a/bc/x.jpg",
"/a/./b/x.jpg",
"/a/../a/b/x.jpg",
"/a//b/x.jpg",
"/a\\b\\x.jpg",
"/a/b ",
"/a/b /x.jpg",
"/.hidden/x.jpg",
"/a/.b/x.jpg",
"/x.jpg",
"//srv/share/x.jpg",
"C:\\a\\b\\x.jpg",
"relative/a/b/x.jpg",
"/Volumes/tokyo/a/x.jpg",
"/日本/写真/x.jpg",
];
let roots = ["/", "/a", "/a/b", "/a/b/", "/a/bc", "/a/b ", "/Volumes/tokyo", "//srv/share", "C:\\a", "/日本", "relative", "/a/../a/b"];
for p in paths {
let mut c = Catalog::new();
let id = add(&mut c, p);
let photo = c.photo(id).unwrap();
for r in roots {
let root = folder_key(r);
let slow = crate::local::folder_of(photo).is_some_and(|f| key_within(&f, &root));
assert_eq!(photo_in_root(photo, &root), slow, "{p:?} in {r:?}");
}
}
}
+5
View File
@@ -14,6 +14,7 @@ let d: Decoded = decode(&bytes, DecodeOptions::default())?;
let working: Rgb32f = to_working(&d); // linear Rec.2020 D65 (Bradford)
let fast = decode(&bytes, DecodeOptions::fit(2048, 2048))?; // JPEG: DCT-domain 1/2..1/8 scaling
let t: Thumbnail = decode_thumbnail(&bytes, 256)?; // embedded EXIF/MPF preview or scaled decode
let h: Header = read_header(&bytes)?; // stored width/height + orientation, no pixels decoded
let icc = icc::write_named(NamedSpace::DisplayP3); // or icc::write_matrix_trc(&space, &Trc::…)
let meta = EncodeMeta { icc: Some(&icc), exif: Some(&exif), xmp: Some(&xmp) };
@@ -56,6 +57,10 @@ or CMYK profile converted by the CMS straight to linear Rec.2020), `Container` (
EXIF `R03` Adobe RGB hint), `Untagged` (assumed sRGB; untagged float data assumed linear),
`IccUnsupported` (profile present but unusable → **sRGB fallback**), `Naive` (CMYK without profile).
`decode_jpeg_with_fallback(bytes, opts, space)` lets an enclosing RAW container supply the colour
space of a JPEG with no ICC or EXIF metadata. It applies the transfer curve before linear-light
resizing; a JPEG carrying its own metadata retains its existing interpretation.
ICC profiles are parsed with `moxcms` (v2/v4, `curv`/`para` TRCs, `chad`); sRGB, Display P3,
Adobe RGB (1998), ProPhoto (ROMM) and Rec.2020 are recognised by colorants. `icc::write_matrix_trc`
emits v4 display profiles (D50 PCS, Bradford `chad`) for export embedding.
+30 -1
View File
@@ -146,7 +146,19 @@ pub fn write_matrix_trc(space: &RgbSpace, trc: &Trc) -> Vec<u8> {
};
p.description = Some(ProfileText::Localizable(vec![LocalizableString::new("en".into(), "US".into(), desc)]));
p.copyright = Some(ProfileText::Localizable(vec![LocalizableString::new("en".into(), "US".into(), "No copyright, use freely".into())]));
align_tags(&p.encode().unwrap_or_default())
align_tags(&fixed_date(p.encode().unwrap_or_default()))
}
/// Zero the header's creation date/time (ICC.1 §7.2.8, bytes 24..36) so a profile's bytes depend
/// only on its colour space and curve. `moxcms` stamps the wall clock there on every `encode`
/// (ignoring `ColorProfile::creation_date_time`), which made two exports of one photo differ in
/// the embedded profile and nowhere else, so output hashes could not be compared (#180). Readers
/// do not interpret the field; all zeros is the conventional "unspecified" value.
fn fixed_date(mut b: Vec<u8>) -> Vec<u8> {
if let Some(date) = b.get_mut(24..36) {
date.fill(0);
}
b
}
/// Re-lay a profile's tag data on 4-byte boundaries (ICC.1 §7.3.1). The encoder packs tags back to
@@ -300,6 +312,23 @@ mod tests {
let _ = SRGB;
}
/// Two profiles for one space are byte-identical, and the header date is fixed: exports of the
/// same photo with the same settings hash the same (#180).
#[test]
fn written_profiles_are_deterministic() {
for n in NamedSpace::ALL {
let a = write_named(n);
let b = write_named(n);
assert_eq!(a, b, "{n:?}");
assert_eq!(&a[24..36], &[0u8; 12], "{n:?} header date/time");
// the profile ID is "not computed", not a hash of a timestamp
assert_eq!(&a[84..100], &[0u8; 16], "{n:?} profile ID");
}
let lin = write_matrix_trc(&PROPHOTO, &Trc::Linear);
assert_eq!(lin, write_matrix_trc(&PROPHOTO, &Trc::Linear));
assert_eq!(&lin[24..36], &[0u8; 12]);
}
#[test]
fn garbage_is_none() {
assert!(parse(&[]).is_none());
+126 -1
View File
@@ -105,6 +105,61 @@ pub(crate) fn parse_markers(b: &[u8]) -> Option<Markers> {
}
pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
decode_with_fallback(bytes, opts, None)
}
/// Stored dimensions and EXIF orientation from the markers, without decoding the scan data. The
/// file is refused, as a decode would refuse it, when it has no frame header, an unsupported frame
/// type or component count, or when its markers don't run to an end-of-image marker after the
/// image data (a truncated file). Damage *inside* the entropy-coded data can't be seen this way.
pub(crate) fn header(bytes: &[u8]) -> Result<(u32, u32, u16)> {
let m = parse_markers(bytes).ok_or_else(|| Error::Malformed(F, "missing SOI".into()))?;
if m.components == 0 {
return Err(Error::Malformed(F, "no frame header".into()));
}
check_size(F, m.width as u64, m.height as u64, &DecodeOptions::default())?;
if !matches!(m.sof, 0xC0..=0xC3) {
return Err(Error::Unsupported(F, "arithmetic-coded or hierarchical JPEG"));
}
if !matches!(m.components, 1 | 3 | 4) {
return Err(Error::Malformed(F, format!("{} colour components", m.components)));
}
if m.first_scan_components == 0 || !reaches_eoi(bytes) {
return Err(Error::Malformed(F, "truncated: the image data doesn't end with an end-of-image marker".into()));
}
let orientation = m.exif.as_deref().map(exif::summarize).unwrap_or_default().orientation.unwrap_or(1);
Ok((m.width, m.height, orientation))
}
/// Whether the markers after SOI, followed through every segment and scan, reach an EOI marker
/// (the first one: the primary image's, ahead of any MPF images appended to the file). Segments
/// are skipped by their length; in scan data `FF 00` (stuffing), `FF FF` (fill) and RSTn are not
/// markers that end it.
fn reaches_eoi(b: &[u8]) -> bool {
let mut p = 2usize;
loop {
// entropy-coded data (or garbage between segments, as `parse_markers` tolerates it)
let Some(skip) = b.get(p..).and_then(find_ff) else { return false };
p = p.saturating_add(skip);
let Some(&marker) = b.get(p.saturating_add(1)) else { return false };
p = match marker {
0xD9 => return true,
0xFF => p.saturating_add(1),
0x00 | 0x01 | 0xD0..=0xD8 => p.saturating_add(2),
_ => {
let Some(len) = b.get(p.saturating_add(2)..p.saturating_add(4)).map(|l| u16::from_be_bytes([l[0], l[1]]) as usize) else {
return false;
};
if len < 2 {
return false;
}
p.saturating_add(2).saturating_add(len)
}
};
}
}
pub(crate) fn decode_with_fallback(bytes: &[u8], opts: &DecodeOptions, fallback: Option<crate::NamedSpace>) -> Result<Decoded> {
let m = parse_markers(bytes).ok_or_else(|| Error::Malformed(F, "missing SOI".into()))?;
if m.components == 0 {
return Err(Error::Malformed(F, "no frame header".into()));
@@ -119,7 +174,10 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
} else {
decode_zune(bytes, &m)?
};
let meta = Meta { icc: m.icc, exif: m.exif, xmp: m.xmp, ..Default::default() };
// A raw container may carry the colour declaration of an otherwise untagged preview.
// Apply it before linearization/resizing; existing JPEG metadata always wins.
let hint = fallback.filter(|_| m.icc.is_none() && m.exif.is_none()).map(|space| crate::SourceSpace::named(space, crate::SpaceOrigin::Container));
let meta = Meta { icc: m.icc, exif: m.exif, xmp: m.xmp, hint, ..Default::default() };
finish(F, raw, meta, (m.width, m.height), opts)
}
@@ -168,6 +226,21 @@ fn decode_jpeg_decoder(bytes: &[u8], m: &Markers, scale_to: Option<(u32, u32)>)
Ok(Raw { width: w, height: h, model, alpha: false, premultiplied: false, buf, bit_depth: depth })
}
/// The first `0xFF` byte, eight bytes at a time (scan data has one every few hundred bytes: this
/// is most of [`header`]'s time).
fn find_ff(b: &[u8]) -> Option<usize> {
const ONES: u64 = 0x0101_0101_0101_0101;
let (words, tail) = b.as_chunks::<8>();
for (i, w) in words.iter().enumerate() {
// a zero byte in !w is an 0xFF byte in w
let x = !u64::from_ne_bytes(*w);
if x.wrapping_sub(ONES) & !x & (ONES << 7) != 0 {
return w.iter().position(|&v| v == 0xFF).map(|j| i * 8 + j);
}
}
tail.iter().position(|&v| v == 0xFF).map(|j| words.len() * 8 + j)
}
/// The box to request from `jpeg-decoder::scale` so the decoded image still covers `mw × mh` once
/// fitted with the source aspect ratio.
fn scaled_request(w: u32, h: u32, mw: u32, mh: u32) -> (u16, u16) {
@@ -246,12 +319,64 @@ fn mpf_images(m: &Markers) -> Vec<(usize, usize)> {
mod tests {
use super::*;
#[test]
fn container_fallback_matches_tagged_jpeg_before_resizing_and_keeps_own_metadata() {
use crate::{ChromaSubsampling, EncodeImage, EncodeMeta, NamedSpace, Samples, SpaceOrigin, encode_jpeg};
let pixels: Vec<u8> = (0..16 * 16).flat_map(|i| if i % 2 == 0 { [120, 180, 90] } else { [30, 100, 200] }).collect();
let encode = |icc, exif| {
encode_jpeg(
&EncodeImage::new(16, 16, 3, Samples::U8(&pixels)),
100,
ChromaSubsampling::S444,
&EncodeMeta { icc, exif, ..Default::default() },
)
.unwrap()
};
let untagged = encode(None, None);
let adobe = crate::icc::write_named(NamedSpace::AdobeRgb);
let tagged = encode(Some(&adobe), None);
for opts in [DecodeOptions::default(), DecodeOptions::fit(3, 3)] {
let with = crate::decode_jpeg_with_fallback(&untagged, opts, NamedSpace::AdobeRgb).unwrap();
let expected = crate::decode(&tagged, opts).unwrap();
assert_eq!(with.space.origin, SpaceOrigin::Container);
let (a, b) = (with.to_working(), expected.to_working());
for (p, q) in a.data.iter().zip(&b.data) {
assert!(p.iter().zip(q).all(|(a, b)| (a - b).abs() < 0.0001), "{p:?} != {q:?}");
}
}
let srgb = crate::icc::write_named(NamedSpace::Srgb);
let exif = crate::exif::minimal_exif(6);
for bytes in [encode(Some(&srgb), None), encode(None, Some(&exif)), encode(Some(b"bad ICC"), None)] {
let opts = DecodeOptions::fit(5, 5);
let with = crate::decode_jpeg_with_fallback(&bytes, opts, NamedSpace::AdobeRgb).unwrap();
let expected = crate::decode(&bytes, opts).unwrap();
assert_eq!(with.space, expected.space);
assert_eq!(with.image, expected.image);
assert_eq!(with.orientation, expected.orientation);
}
assert!(crate::decode_jpeg_with_fallback(b"bad JPEG", DecodeOptions::default(), NamedSpace::AdobeRgb).is_err());
}
#[test]
fn scaled_request_covers() {
assert_eq!(scaled_request(6000, 4000, 256, 256), (256, 171));
assert_eq!(scaled_request(100, 100, 256, 256), (100, 100));
}
#[test]
fn find_ff_matches_a_plain_search() {
let mut b: Vec<u8> = (0..300u32).map(|i| (i * 7 % 255) as u8).collect();
for at in [None, Some(0), Some(7), Some(8), Some(15), Some(203), Some(296), Some(299)] {
if let Some(i) = at {
b[i] = 0xFF;
}
for start in [0, 1, 5, 9, 290, 299, 300] {
let s = &b[start..];
assert_eq!(find_ff(s), s.iter().position(|&v| v == 0xFF), "{at:?} from {start}");
}
}
}
#[test]
fn markers_on_garbage() {
assert!(parse_markers(b"").is_none());
+56
View File
@@ -6,6 +6,8 @@
//! profiles (matrix/TRC exactly, LUT/CMYK via the `moxcms` CMS), 16-bit and float precision are
//! preserved. EXIF/XMP/ICC blobs are passed through untouched; orientation is reported, never applied.
//! - [`to_working`] converts a decode result to linear Rec.2020 D65 (the pipeline working space).
//! - [`read_header`] reads a file's stored dimensions and orientation from its headers, without
//! decoding pixels (import probes).
//! - [`decode_thumbnail`] is the fast path for grid thumbnails (EXIF thumbnail or DCT-scaled decode).
//! - [`encode`] writes JPEG, PNG, TIFF, lossless WebP and (native, feature `avif`) AVIF, embedding
//! ICC/EXIF/XMP. [`icc::write_matrix_trc`] builds profiles for export.
@@ -156,6 +158,60 @@ pub fn decode(bytes: &[u8], opts: DecodeOptions) -> Result<Decoded> {
std::panic::catch_unwind(|| decode_unguarded(bytes, format, &opts)).unwrap_or_else(|_| Err(Error::Malformed(format, "decoder panicked".into())))
}
/// What a file's headers say about its image: enough to catalogue it without decoding pixels.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Header {
pub format: Format,
/// Full-resolution stored dimensions, orientation **not** applied ([`Decoded::source_width`]).
pub width: u32,
pub height: u32,
/// EXIF/TIFF orientation 1..=8, 1 when absent ([`Decoded::orientation`]).
pub orientation: u16,
}
/// Read the stored dimensions and orientation of any format [`decode`] supports, agreeing with
/// what a decode reports, without decoding pixels: JPEG, PNG, TIFF, WebP and PSD are read from
/// their headers (markers, chunks, IFD, resources). GIF, BMP and JPEG XL fall back to a small
/// decode.
///
/// The header readers also refuse what a decode would refuse before reaching the pixels (an
/// unsupported layout, compression or frame type) and files whose image data is cut short
/// (truncated: the data runs past the end of the file, or a JPEG's markers don't reach its
/// end-of-image marker). Damage *inside* compressed data that is all present is only found by a
/// decode. Malformed input yields an error, never a panic (as for [`decode`], a panic inside a
/// third-party parser is caught on unwinding targets).
pub fn read_header(bytes: &[u8]) -> Result<Header> {
let format = sniff(bytes).ok_or(Error::UnknownFormat)?;
std::panic::catch_unwind(|| read_header_unguarded(bytes, format))
.unwrap_or_else(|_| Err(Error::Malformed(format, "header reader panicked".into())))
}
/// [`read_header`] without the panic guard (for fuzzing our own code paths).
#[doc(hidden)]
pub fn read_header_unguarded(bytes: &[u8], format: Format) -> Result<Header> {
let (width, height, orientation) = match format {
Format::Jpeg => jpeg::header(bytes)?,
Format::Png => png_codec::header(bytes)?,
Format::Tiff => tiff_codec::header(bytes)?,
Format::WebP => webp::header(bytes)?,
Format::Psd => psd::header(bytes)?,
// rare here (or, for JPEG XL, without a header reader yet): a small decode, as before
_ => {
let d = decode_unguarded(bytes, format, &DecodeOptions::fit(64, 64))?;
(d.source_width, d.source_height, d.orientation)
}
};
Ok(Header { format, width, height, orientation })
}
/// Decode a JPEG preview with its enclosing container's colour space as a fallback.
/// A JPEG carrying ICC or EXIF metadata keeps its own interpretation. The fallback is applied
/// before transfer decoding and linear-light resizing, never to already-resampled pixels.
pub fn decode_jpeg_with_fallback(bytes: &[u8], opts: DecodeOptions, fallback: NamedSpace) -> Result<Decoded> {
std::panic::catch_unwind(|| jpeg::decode_with_fallback(bytes, &opts, Some(fallback)))
.unwrap_or_else(|_| Err(Error::Malformed(Format::Jpeg, "decoder panicked".into())))
}
/// [`decode`] without the panic guard (for fuzzing our own code paths).
#[doc(hidden)]
pub fn decode_unguarded(bytes: &[u8], format: Format, opts: &DecodeOptions) -> Result<Decoded> {
+69
View File
@@ -57,6 +57,75 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
finish(F, raw, meta, (w, h), opts)
}
/// Stored dimensions and EXIF orientation (`eXIf`, wherever it is) from the chunk headers, without
/// inflating the image data. Refused, as a decode would refuse it: an invalid IHDR, an indexed
/// image without a palette, no IDAT, or a file that ends before the chunk after the last IDAT
/// begins (truncated image data). Chunk CRCs and the deflate stream are not checked.
pub(crate) fn header(b: &[u8]) -> Result<(u32, u32, u16)> {
if !b.starts_with(b"\x89PNG\r\n\x1a\n") {
return Err(err("bad signature"));
}
let be32 = |p: usize| b.get(p..p.checked_add(4)?).map(|v| u32::from_be_bytes([v[0], v[1], v[2], v[3]]));
if be32(8) != Some(13) || b.get(12..16) != Some(b"IHDR") {
return Err(err("missing IHDR"));
}
let ihdr = b.get(16..29).ok_or_else(|| err("truncated IHDR"))?;
let (w, h) = (u32::from_be_bytes([ihdr[0], ihdr[1], ihdr[2], ihdr[3]]), u32::from_be_bytes([ihdr[4], ihdr[5], ihdr[6], ihdr[7]]));
let (depth, color, compression, filter, interlace) = (ihdr[8], ihdr[9], ihdr[10], ihdr[11], ihdr[12]);
if w > i32::MAX as u32 || h > i32::MAX as u32 {
return Err(err("dimensions out of range"));
}
check_size(F, w as u64, h as u64, &DecodeOptions::default())?;
let depth_ok = match color {
0 => matches!(depth, 1 | 2 | 4 | 8 | 16),
3 => matches!(depth, 1 | 2 | 4 | 8),
2 | 4 | 6 => matches!(depth, 8 | 16),
_ => false,
};
if !depth_ok || compression != 0 || filter != 0 || interlace > 1 {
return Err(err("invalid IHDR"));
}
let (mut p, mut palette, mut idat, mut after_idat, mut exif) = (8usize, false, false, false, None);
while let Some(len) = be32(p) {
let Some(kind) = p.checked_add(4).and_then(|k| b.get(k..k.checked_add(4)?)) else { break };
if idat && kind != b"IDAT" {
// a chunk follows the image data: the IDAT run is complete
after_idat = true;
}
if kind == b"IEND" {
break;
}
let start = p.saturating_add(8);
let data = (len <= i32::MAX as u32).then(|| start.checked_add(len as usize).and_then(|end| b.get(start..end))).flatten();
let Some(data) = data else {
if after_idat {
break; // a damaged chunk after the image data: a decode ignores it too
}
return Err(err("truncated"));
};
match kind {
b"PLTE" => palette = true,
b"IDAT" if after_idat => {} // a stray IDAT after other chunks: the first run is the image
b"IDAT" => {
if color == 3 && !palette {
return Err(err("indexed image without a palette"));
}
idat = true;
}
b"eXIf" if exif.is_none() => exif = Some(data.strip_prefix(b"Exif\0\0").unwrap_or(data)),
_ => {}
}
p = start.saturating_add(data.len()).saturating_add(4);
}
if !idat {
return Err(err("no image data"));
}
if !after_idat {
return Err(err("truncated image data"));
}
Ok((w, h, exif.map(crate::exif::summarize).unwrap_or_default().orientation.unwrap_or(1)))
}
fn container_hint(info: &png::Info) -> Option<SourceSpace> {
if let Some(c) = &info.coding_independent_code_points {
let named = match c.color_primaries {
+265 -72
View File
@@ -69,8 +69,8 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let (icc, exif, xmp) = resources(res);
// Layer and mask information: only the layer count sign matters (merged transparency).
let lm_len = if psb { r.u64()? as usize } else { r.u32()? as usize };
let lm = r.take(lm_len)?;
let lm_len = if psb { r.u64()? } else { u64::from(r.u32()?) };
let lm = r.take(usize::try_from(lm_len).map_err(|_| err("truncated"))?)?;
let merged_transparency = if lm.len() >= if psb { 10 } else { 6 } {
let off = if psb { 8 } else { 4 };
let count = i16::from_be_bytes([lm[off], lm[off + 1]]);
@@ -88,33 +88,6 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
1 => return Err(Error::Unsupported(F, "1-bit bitmap PSD")),
_ => return Err(err("bad depth")),
};
let plane = width * height * bpc;
let row_bytes = width * bpc;
let mut planes: Vec<Vec<u8>> = Vec::with_capacity(channels);
match compression {
0 => {
for _ in 0..channels {
planes.push(r.take(plane)?.to_vec());
}
}
1 => {
let counts_len = channels * height;
let mut counts = Vec::with_capacity(counts_len);
for _ in 0..counts_len {
counts.push(if psb { r.u32()? as usize } else { r.u16()? as usize });
}
for c in 0..channels {
let mut out = Vec::with_capacity(plane);
for y in 0..height {
let src = r.take(counts[c * height + y])?;
unpackbits(src, row_bytes, &mut out);
}
planes.push(out);
}
}
_ => return Err(Error::Unsupported(F, "ZIP-compressed merged image data")),
}
let (model, n_color) = match mode {
1 | 8 => (Model::Gray, 1),
2 => (Model::Rgb, 1),
@@ -128,50 +101,106 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
return Err(err("too few channels"));
}
let alpha = merged_transparency && channels > n_color && mode != 2;
// Only the colour (+ alpha) planes are decoded; extra channels (spot colours, masks) are
// stored after them and never touched.
let used = n_color + alpha as usize;
let n = width * height;
let too_large = || Error::TooLarge(width as u64, height as u64);
let n = width.checked_mul(height).ok_or_else(too_large)?;
let row_bytes = width.checked_mul(bpc).ok_or_else(too_large)?;
let plane = n.checked_mul(bpc).ok_or_else(too_large)?;
let mut planes: Vec<Vec<u8>> = Vec::with_capacity(used);
match compression {
0 => {
// Raw planes are borrowed from the input, so their size is bounded by the file.
for _ in 0..used {
planes.push(r.take(plane)?.to_vec());
}
}
1 => {
// Row byte-count table: one entry per row per channel (all channels, since the data
// follows it). Taken from the input before anything is allocated from it.
let entry = if psb { 4 } else { 2 };
let table_len = channels.checked_mul(height).and_then(|c| c.checked_mul(entry)).ok_or_else(|| err("truncated"))?;
let table = r.take(table_len)?;
let count = |i: usize| -> usize {
let at = i.saturating_mul(entry);
match table.get(at..at.saturating_add(entry)) {
Some(&[a, b]) => usize::from(u16::from_be_bytes([a, b])),
Some(&[a, b, c, d]) => u32::from_be_bytes([a, b, c, d]) as usize,
_ => 0,
}
};
// PackBits expands at most 64× (a 2-byte repeat run yields 128 bytes), so a row needs
// at least this many encoded bytes. Requiring it (and the data being present) before
// allocating bounds the decoded planes by the file size: a tiny file can no longer
// declare gigabytes of zero-padded or repeat-run rows.
let min_row = row_bytes.div_ceil(128).saturating_mul(2);
let mut data_len = 0usize;
for i in 0..used.saturating_mul(height) {
let c = count(i);
if c < min_row {
return Err(err("RLE row too short for its width"));
}
data_len = data_len.saturating_add(c);
}
if data_len > bytes.len().saturating_sub(r.p) {
return Err(err("truncated"));
}
for c in 0..used {
let mut out = zeroed::<u8>(plane, too_large)?;
for (y, row) in out.chunks_exact_mut(row_bytes).enumerate() {
let src = r.take(count(c * height + y))?;
unpackbits(src, row)?;
}
planes.push(out);
}
}
_ => return Err(Error::Unsupported(F, "ZIP-compressed merged image data")),
}
let samples = n.checked_mul(used).ok_or_else(too_large)?;
// Interleave to device samples.
let buf = if mode == 2 {
// Indexed → 8-bit RGB via the 768-byte palette.
if color_mode_data.len() < 768 || depth != 8 {
let (Some(pal), true) = (color_mode_data.get(..768), depth == 8) else {
return Err(err("bad indexed palette"));
}
let mut v = Vec::with_capacity(n * 3);
for &i in &planes[0] {
};
let idx = planes.first().ok_or_else(|| err("missing plane"))?;
let mut v = zeroed::<u8>(n.checked_mul(3).ok_or_else(too_large)?, too_large)?;
for (o, &i) in v.chunks_exact_mut(3).zip(idx) {
let i = i as usize;
v.extend_from_slice(&[color_mode_data[i], color_mode_data[256 + i], color_mode_data[512 + i]]);
o.copy_from_slice(&[pal[i], pal[256 + i], pal[512 + i]]);
}
Buf::U8(v)
} else {
let cmyk = model == Model::Cmyk;
match bpc {
1 => {
let mut v = vec![0u8; n * used];
for (c, p) in planes.iter().take(used).enumerate() {
for i in 0..n {
let x = p[i];
let mut v = zeroed::<u8>(samples, too_large)?;
for (c, p) in planes.iter().enumerate() {
for (o, &x) in v.iter_mut().skip(c).step_by(used).zip(p) {
// PSD CMYK stores 255 = no ink; our CMYK convention is 0 = no ink.
v[i * used + c] = if model == Model::Cmyk && c < 4 { 255 - x } else { x };
*o = if cmyk && c < 4 { 255 - x } else { x };
}
}
Buf::U8(v)
}
2 => {
let mut v = vec![0u16; n * used];
for (c, p) in planes.iter().take(used).enumerate() {
for i in 0..n {
let x = u16::from_be_bytes([p[2 * i], p[2 * i + 1]]);
v[i * used + c] = if model == Model::Cmyk && c < 4 { 65535 - x } else { x };
let mut v = zeroed::<u16>(samples, too_large)?;
for (c, p) in planes.iter().enumerate() {
for (o, s) in v.iter_mut().skip(c).step_by(used).zip(p.chunks_exact(2)) {
let x = u16::from_be_bytes([s[0], s[1]]);
*o = if cmyk && c < 4 { 65535 - x } else { x };
}
}
Buf::U16(v)
}
_ => {
let mut v = vec![0f32; n * used];
for (c, p) in planes.iter().take(used).enumerate() {
for i in 0..n {
let x = f32::from_be_bytes([p[4 * i], p[4 * i + 1], p[4 * i + 2], p[4 * i + 3]]);
v[i * used + c] = if model == Model::Cmyk && c < 4 { 1.0 - x } else { x };
let mut v = zeroed::<f32>(samples, too_large)?;
for (c, p) in planes.iter().enumerate() {
for (o, s) in v.iter_mut().skip(c).step_by(used).zip(p.chunks_exact(4)) {
let x = f32::from_be_bytes([s[0], s[1], s[2], s[3]]);
*o = if cmyk && c < 4 { 1.0 - x } else { x };
}
}
Buf::F32(v)
@@ -182,25 +211,111 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
finish(F, raw, Meta { icc, exif, xmp, ..Default::default() }, (width as u32, height as u32), opts)
}
/// PackBits: append exactly `row_bytes` bytes (zero-padded if the run data is short).
fn unpackbits(mut src: &[u8], row_bytes: usize, out: &mut Vec<u8>) {
let target = out.len() + row_bytes;
while out.len() < target && !src.is_empty() {
let h = src[0] as i8;
src = &src[1..];
if h >= 0 {
let n = (h as usize + 1).min(src.len()).min(target - out.len());
out.extend_from_slice(&src[..n]);
src = &src[(h as usize + 1).min(src.len())..];
} else if h != -128 {
let n = ((1 - h as isize) as usize).min(target - out.len());
if let Some(&b) = src.first() {
out.extend(std::iter::repeat_n(b, n));
src = &src[1..];
/// Stored dimensions and EXIF orientation (resource 1058) from the header and image resources,
/// without reading the merged image data. Refused, as a decode would refuse it: an unsupported
/// depth, colour mode or compression, or merged image data that runs past the end of the file
/// (truncated). The run-length data itself is not unpacked.
pub(crate) fn header(bytes: &[u8]) -> Result<(u32, u32, u16)> {
let mut r = Rd { b: bytes, p: 0 };
if r.take(4)? != b"8BPS" {
return Err(err("bad signature"));
}
let psb = match r.u16()? {
1 => false,
2 => true,
_ => return Err(err("bad version")),
};
r.take(6)?;
let channels = r.u16()? as usize;
let height = r.u32()? as usize;
let width = r.u32()? as usize;
let depth = r.u16()?;
let mode = r.u16()?;
check_size(F, width as u64, height as u64, &DecodeOptions::default())?;
if channels == 0 || channels > 56 {
return Err(err("bad channel count"));
}
let cm_len = r.u32()? as usize;
let color_mode_data = r.take(cm_len)?;
let res_len = r.u32()? as usize;
let (_, exif, _) = resources(r.take(res_len)?);
let lm_len = if psb { r.u64()? as usize } else { r.u32()? as usize };
r.take(lm_len)?;
let compression = r.u16()?;
let bpc = match depth {
8 => 1usize,
16 => 2,
32 => 4,
1 => return Err(Error::Unsupported(F, "1-bit bitmap PSD")),
_ => return Err(err("bad depth")),
};
let n_color = match mode {
1 | 8 | 7 | 2 => 1,
3 => 3,
4 => 4,
9 => return Err(Error::Unsupported(F, "Lab PSD")),
_ => return Err(err("unknown colour mode")),
};
if channels < n_color {
return Err(err("too few channels"));
}
if mode == 2 && (color_mode_data.len() < 768 || depth != 8) {
return Err(err("bad indexed palette"));
}
let rest = bytes.len().saturating_sub(r.p);
let need = match compression {
0 => width.checked_mul(height).and_then(|n| n.checked_mul(bpc)).and_then(|n| n.checked_mul(channels)),
1 => {
// the row byte counts, then the rows they count
let rows = channels.checked_mul(height).ok_or_else(|| err("truncated"))?;
let table = rows.checked_mul(if psb { 4 } else { 2 }).filter(|&t| t <= rest).ok_or_else(|| err("truncated"))?;
let mut total = table;
for _ in 0..rows {
total = total.saturating_add(if psb { r.u32()? as usize } else { r.u16()? as usize });
}
Some(total)
}
_ => return Err(Error::Unsupported(F, "ZIP-compressed merged image data")),
};
if need.is_none_or(|n| n > rest) {
return Err(err("truncated"));
}
let orientation = exif.as_deref().map(crate::exif::summarize).unwrap_or_default().orientation.unwrap_or(1);
Ok((width as u32, height as u32, orientation))
}
/// A zero-filled buffer whose allocation failure is an error, not an abort.
fn zeroed<T: Clone + Default>(len: usize, too_large: impl Fn() -> Error) -> Result<Vec<T>> {
let mut v = Vec::new();
v.try_reserve_exact(len).map_err(|_| too_large())?;
v.resize(len, T::default());
Ok(v)
}
/// PackBits: decode one row into `row` (pre-zeroed, so short run data leaves zero padding).
/// A run that would write past the end of the row is an error; trailing bytes are ignored.
fn unpackbits(mut src: &[u8], row: &mut [u8]) -> Result<()> {
let mut at = 0;
while at < row.len() {
let Some((&h, rest)) = src.split_first() else { break };
src = rest;
let h = h as i8;
if h >= 0 {
let n = h as usize + 1;
let lit = src.get(..n).unwrap_or(src);
let dst = row.get_mut(at..at + lit.len()).ok_or_else(|| err("RLE row overruns its width"))?;
dst.copy_from_slice(lit);
at += lit.len();
src = src.get(n..).unwrap_or(&[]);
} else if h != -128 {
let n = (1 - h as isize) as usize;
let Some((&b, rest)) = src.split_first() else { break };
src = rest;
row.get_mut(at..at + n).ok_or_else(|| err("RLE row overruns its width"))?.fill(b);
at += n;
}
}
out.resize(target, 0);
Ok(())
}
fn resources(mut b: &[u8]) -> (Option<Vec<u8>>, Option<Vec<u8>>, Option<String>) {
@@ -234,11 +349,89 @@ mod tests {
#[test]
fn packbits() {
let mut out = Vec::new();
unpackbits(&[0xFE, 0xAA, 0x02, 0x80, 0x00, 0x2A], 6, &mut out);
assert_eq!(out, vec![0xAA, 0xAA, 0xAA, 0x80, 0x00, 0x2A]);
let mut out = Vec::new();
unpackbits(&[0x05, 1], 4, &mut out);
assert_eq!(out, vec![1, 0, 0, 0]);
let mut out = [0u8; 6];
unpackbits(&[0xFE, 0xAA, 0x02, 0x80, 0x00, 0x2A], &mut out).unwrap();
assert_eq!(out, [0xAA, 0xAA, 0xAA, 0x80, 0x00, 0x2A]);
// Short run data is zero-padded.
let mut out = [0u8; 4];
unpackbits(&[0x01, 1], &mut out).unwrap();
assert_eq!(out, [1, 0, 0, 0]);
// Truncated literal run: copies what is there.
let mut out = [0u8; 4];
unpackbits(&[0x03, 7, 8], &mut out).unwrap();
assert_eq!(out, [7, 8, 0, 0]);
}
#[test]
fn packbits_overrun_is_an_error() {
// Repeat run of 128 into a 4-byte row.
let mut out = [0u8; 4];
assert!(unpackbits(&[0x81, 0xAA], &mut out).is_err());
// Literal run of 6 into a 4-byte row.
assert!(unpackbits(&[0x05, 1, 2, 3, 4, 5, 6], &mut out).is_err());
}
/// PSD header + empty colour-mode/resources/layer sections + compression id.
fn header(channels: u16, w: u32, h: u32, depth: u16, mode: u16, compression: u16) -> Vec<u8> {
let mut b = b"8BPS".to_vec();
b.extend_from_slice(&1u16.to_be_bytes());
b.extend_from_slice(&[0; 6]);
b.extend_from_slice(&channels.to_be_bytes());
b.extend_from_slice(&h.to_be_bytes());
b.extend_from_slice(&w.to_be_bytes());
b.extend_from_slice(&depth.to_be_bytes());
b.extend_from_slice(&mode.to_be_bytes());
b.extend_from_slice(&[0; 12]); // colour mode data, resources, layer & mask: all empty
b.extend_from_slice(&compression.to_be_bytes());
b
}
#[test]
fn huge_rle_with_truncated_count_table_is_rejected_before_allocating() {
// 1 × 2^30 px, 56 channels: the row table alone would be 56 Gi entries; the file has 4 bytes.
let mut b = header(56, 1, 1 << 30, 32, 3, 1);
b.extend_from_slice(&[0, 0, 0, 0]);
let e = decode(&b, &DecodeOptions::default()).unwrap_err();
assert!(matches!(e, Error::Malformed(_, ref m) if m == "truncated"), "{e:?}");
}
#[test]
fn huge_rle_of_empty_rows_is_rejected_before_allocating() {
// 2^30 × 1 px RGB at 32 bits: every row count is 0, which used to zero-pad 3 × 4 GiB.
let mut b = header(3, 1 << 30, 1, 32, 3, 1);
b.extend_from_slice(&[0; 6]);
let e = decode(&b, &DecodeOptions::default()).unwrap_err();
assert!(matches!(e, Error::Malformed(_, ref m) if m.contains("too short")), "{e:?}");
// 1 Gpx RGB 8-bit whose (plausible) row counts claim more data than the file holds.
let mut b = header(3, 1 << 20, 1 << 10, 8, 3, 1);
b.extend_from_slice(&[0xFF; 3 * 1024 * 2]);
let e = decode(&b, &DecodeOptions::default()).unwrap_err();
assert!(matches!(e, Error::Malformed(_, ref m) if m == "truncated"), "{e:?}");
}
#[test]
fn rle_repeat_run_past_row_width_is_an_error() {
// 4 × 1 gray 8-bit; the single row is a 128-byte repeat run.
let mut b = header(1, 4, 1, 8, 1, 1);
b.extend_from_slice(&2u16.to_be_bytes());
b.extend_from_slice(&[0x81, 0x55]);
let e = decode(&b, &DecodeOptions::default()).unwrap_err();
assert!(matches!(e, Error::Malformed(_, ref m) if m.contains("overruns")), "{e:?}");
}
#[test]
fn rle_gray16_and_extra_channels_decode() {
// 2 × 2 gray 16-bit with an extra (unused) channel whose data is absent: only the used
// plane is decoded. Row: repeat run of 4 bytes 0x80 -> samples 0x8080.
let mut b = header(2, 2, 2, 16, 1, 1);
for _ in 0..4 {
b.extend_from_slice(&2u16.to_be_bytes());
}
b.extend_from_slice(&[0xFD, 0x80, 0xFD, 0x80]);
let d = decode(&b, &DecodeOptions::default()).unwrap();
assert_eq!((d.width, d.height), (2, 2));
let p = d.image.get(1, 1);
let q = d.image.get(0, 0);
assert!(p[0] > 0.0 && (p[0] - q[0]).abs() < 1e-6, "{p:?}");
}
}
+92 -29
View File
@@ -1,5 +1,6 @@
//! TIFF decode via the `tiff` crate: 8/16/32-bit integer, 16/32/64-bit float, gray/RGB/CMYK/palette,
//! alpha (associated or not), ICC (34675), XMP (700), orientation (274). First image only.
//! TIFF decode via the `tiff` crate: 1/2/4/8/16/32/64-bit integer, 16/32/64-bit float, gray/RGB/CMYK,
//! alpha (associated or not), ICC (34675), XMP (700), orientation (274). First image only. Palette
//! (indexed-colour) images are refused by the `tiff` crate itself, on every read path.
use crate::convert::{Buf, Meta, Model, Raw, check_size, finish};
use crate::{DecodeOptions, Decoded, Error, Format, Result};
@@ -14,10 +15,7 @@ fn err(e: impl std::fmt::Display) -> Error {
}
pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let mut limits = Limits::default();
limits.decoding_buffer_size = opts.max_pixels.saturating_mul(4 * 8).min(usize::MAX as u64) as usize;
limits.ifd_value_size = 32 << 20;
let mut d = Decoder::new(std::io::Cursor::new(bytes)).map_err(err)?.with_limits(limits);
let mut d = Decoder::new(std::io::Cursor::new(bytes)).map_err(err)?.with_limits(limits(opts));
let (w, h) = d.dimensions().map_err(err)?;
check_size(F, w as u64, h as u64, opts)?;
let ct = d.colortype().map_err(err)?;
@@ -25,26 +23,12 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let icc = tag_bytes(&mut d, Tag::IccProfile);
let xmp = tag_bytes(&mut d, Tag::from_u16_exhaustive(700)).map(|v| String::from_utf8_lossy(&v).trim_end_matches('\0').to_string());
let orientation = d.get_tag_u32(Tag::Orientation).ok().map(|v| v as u16);
let photometric = d.get_tag_u32(Tag::PhotometricInterpretation).ok();
let planar = d.get_tag_u32(Tag::PlanarConfiguration).ok() == Some(2);
let extra = d.get_tag_u16_vec(Tag::ExtraSamples).ok().unwrap_or_default();
let colormap = d.get_tag_u16_vec(Tag::ColorMap).ok();
let compression = d.get_tag_u32(Tag::Compression).ok();
let (model, n_color, n_samples, bits) = match ct {
ColorType::Gray(b) => (Model::Gray, 1, 1, b),
ColorType::GrayA(b) => (Model::Gray, 1, 2, b),
ColorType::RGB(b) => (Model::Rgb, 3, 3, b),
ColorType::RGBA(b) => (Model::Rgb, 3, 4, b),
ColorType::CMYK(b) => (Model::Cmyk, 4, 4, b),
ColorType::CMYKA(b) => (Model::Cmyk, 4, 5, b),
ColorType::Palette(b) => (Model::Rgb, 1, 1, b),
// JPEG-compressed YCbCr is converted to RGB by the JPEG decoder.
ColorType::YCbCr(b) if compression == Some(7) => (Model::Rgb, 3, 3, b),
ColorType::Multiband { bit_depth, num_samples } if num_samples >= 3 => (Model::Rgb, 3, num_samples as usize, bit_depth),
ColorType::Multiband { bit_depth, num_samples } if num_samples >= 1 => (Model::Gray, 1, num_samples as usize, bit_depth),
other => return Err(Error::Unsupported(F, unsupported_name(other))),
};
let (model, n_color, n_samples, bits) = layout(ct, compression, planar)?;
let has_alpha = n_samples > n_color && !matches!(ct, ColorType::Palette(_));
let premultiplied = has_alpha && extra.first() == Some(&1);
@@ -52,6 +36,7 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let (wu, hu) = (w as usize, h as usize);
let n = wu * hu;
let mut buf = match res {
DecodingResult::U8(v) if bits < 8 => Buf::U8(unpack(&v, wu, hu, n_samples, bits)?),
DecodingResult::U8(v) => Buf::U8(v),
DecodingResult::U16(v) => Buf::U16(v),
// Rare integer widths: keep 16 significant bits (plenty for display-referred data).
@@ -94,14 +79,8 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
return finish(F, raw, meta, (w, h), opts);
}
// WhiteIsZero → invert.
if photometric == Some(0) {
match &mut buf {
Buf::U8(v) => v.iter_mut().for_each(|x| *x = 255 - *x),
Buf::U16(v) => v.iter_mut().for_each(|x| *x = 65535 - *x),
Buf::F32(v) => v.iter_mut().for_each(|x| *x = 1.0 - *x),
}
}
// WhiteIsZero needs nothing here: the `tiff` crate inverts those samples as it reads them (and
// refuses WhiteIsZero layouts it can't invert), so inverting again would show a negative.
// Drop extra samples beyond colour + one alpha.
let keep = n_color + has_alpha as usize;
@@ -113,6 +92,67 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
finish(F, raw, meta, (w, h), opts)
}
fn limits(opts: &DecodeOptions) -> Limits {
let mut limits = Limits::default();
limits.decoding_buffer_size = opts.max_pixels.saturating_mul(4 * 8).min(usize::MAX as u64) as usize;
limits.ifd_value_size = 32 << 20;
limits
}
/// The sample layout of a colour type we can decode: (model, colour samples, samples per pixel,
/// bits per sample).
fn layout(ct: ColorType, compression: Option<u32>, planar: bool) -> Result<(Model, usize, usize, u8)> {
let l = match ct {
ColorType::Gray(b) => (Model::Gray, 1, 1, b),
ColorType::GrayA(b) => (Model::Gray, 1, 2, b),
ColorType::RGB(b) => (Model::Rgb, 3, 3, b),
ColorType::RGBA(b) => (Model::Rgb, 3, 4, b),
ColorType::CMYK(b) => (Model::Cmyk, 4, 4, b),
ColorType::CMYKA(b) => (Model::Cmyk, 4, 5, b),
ColorType::Palette(b) => (Model::Rgb, 1, 1, b),
// JPEG-compressed YCbCr is converted to RGB by the JPEG decoder.
ColorType::YCbCr(b) if compression == Some(7) => (Model::Rgb, 3, 3, b),
ColorType::Multiband { bit_depth, num_samples } if num_samples >= 3 => (Model::Rgb, 3, num_samples as usize, bit_depth),
ColorType::Multiband { bit_depth, num_samples } if num_samples >= 1 => (Model::Gray, 1, num_samples as usize, bit_depth),
other => return Err(Error::Unsupported(F, unsupported_name(other))),
};
if l.3 < 8 && planar && l.2 > 1 {
return Err(Error::Unsupported(F, "planar TIFF with samples narrower than a byte"));
}
Ok(l)
}
/// Stored dimensions and orientation from the first IFD, without reading the image data. Refused,
/// as a decode would refuse it: a layout or compression we can't decode, or strips/tiles that
/// start past the end of the file (truncated). The compressed data itself is not checked.
pub(crate) fn header(bytes: &[u8]) -> Result<(u32, u32, u16)> {
let mut d = Decoder::new(std::io::Cursor::new(bytes)).map_err(err)?.with_limits(limits(&DecodeOptions::default()));
let (w, h) = d.dimensions().map_err(err)?;
check_size(F, w as u64, h as u64, &DecodeOptions::default())?;
let ct = d.colortype().map_err(err)?;
let compression = d.get_tag_u32(Tag::Compression).ok();
let planar = d.get_tag_u32(Tag::PlanarConfiguration).ok() == Some(2);
layout(ct, compression, planar)?;
// what this build of the `tiff` crate decompresses: none, CCITT G4, LZW, JPEG, Deflate, PackBits
if !matches!(compression.unwrap_or(1), 1 | 4 | 5 | 7 | 8 | 32773 | 32946) {
return Err(Error::Unsupported(F, "compression method"));
}
let chunks = |offsets: Tag, counts: Tag, d: &mut Decoder<std::io::Cursor<&[u8]>>| {
Some((d.get_tag_u64_vec(offsets).ok()?, d.get_tag_u64_vec(counts).ok()?))
};
let (offsets, counts) = chunks(Tag::StripOffsets, Tag::StripByteCounts, &mut d)
.or_else(|| chunks(Tag::TileOffsets, Tag::TileByteCounts, &mut d))
.ok_or_else(|| err("no image data"))?;
// Only where each chunk starts: some writers overstate StripByteCounts, and the `tiff` crate
// decodes such files (it stops once it has the pixels), so a chunk's end proves nothing.
let len = bytes.len() as u64;
if offsets.is_empty() || offsets.iter().zip(&counts).any(|(&o, &c)| c > 0 && o >= len) {
return Err(err("truncated: image data past the end of the file"));
}
let orientation = d.get_tag_u32(Tag::Orientation).ok().map(|v| v as u16).filter(|o| (1..=8).contains(o)).unwrap_or(1);
Ok((w, h, orientation))
}
/// A BYTE/UNDEFINED tag as bytes (the decoder may surface unknown byte tags as unsigned values).
fn tag_bytes<R: std::io::Read + std::io::Seek>(d: &mut Decoder<R>, tag: Tag) -> Option<Vec<u8>> {
use tiff::decoder::ifd::Value;
@@ -140,6 +180,29 @@ fn unsupported_name(ct: ColorType) -> &'static str {
}
}
/// Samples narrower than a byte (1-bit bilevel and fax scans, 2- and 4-bit gray) arrive packed, most
/// significant bits first, each row padded to a whole byte. Spread them out one per byte, scaled to
/// 0–255 like any 8-bit sample.
fn unpack(packed: &[u8], width: usize, height: usize, samples: usize, bits: u8) -> Result<Vec<u8>> {
if !matches!(bits, 1 | 2 | 4) {
return Err(Error::Unsupported(F, "TIFF samples of 3, 5, 6 or 7 bits"));
}
let bits = bits as usize;
let per_row = width.checked_mul(samples).ok_or_else(|| err("row too wide"))?;
let row_bytes = per_row.checked_mul(bits).ok_or_else(|| err("row too wide"))?.div_ceil(8);
if row_bytes == 0 || packed.len() / row_bytes < height {
return Err(err("sample buffer too short"));
}
let max = (1u8 << bits) - 1;
let scale = 255 / max; // 255, 85, 17: exact
let mut out = Vec::with_capacity(per_row.saturating_mul(height));
for row in packed.chunks_exact(row_bytes).take(height) {
let values = row.iter().flat_map(|&b| (1..=8 / bits).map(move |i| (b >> (8 - bits * i)) & max));
out.extend(values.take(per_row).map(|v| v * scale));
}
Ok(out)
}
fn interleave(buf: Buf, n: usize, s: usize) -> Buf {
fn go<T: Copy + Default>(v: Vec<T>, n: usize, s: usize) -> Vec<T> {
if v.len() < n * s {
+27
View File
@@ -27,3 +27,30 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let raw = Raw { width: w as usize, height: h as usize, model: Model::Rgb, alpha, premultiplied: false, buf: Buf::U8(buf), bit_depth: 8 };
finish(F, raw, Meta { icc, exif, xmp, ..Default::default() }, (w, h), opts)
}
/// Stored dimensions and EXIF orientation from the RIFF chunks and the bitstream header, without
/// decoding the image. Refused, as a decode would refuse it: no image chunk (`VP8 `, `VP8L` or an
/// animation frame), or one that runs past the end of the file (truncated). The compressed data
/// itself is not checked.
pub(crate) fn header(bytes: &[u8]) -> Result<(u32, u32, u16)> {
let mut p = 12usize;
loop {
let Some(h) = bytes.get(p..p.saturating_add(8)) else { return Err(err("no image data")) };
let size = u32::from_le_bytes([h[4], h[5], h[6], h[7]]) as usize;
let end = p.saturating_add(8).saturating_add(size);
if matches!(&h[0..4], b"VP8 " | b"VP8L" | b"ANMF") {
if end > bytes.len() {
return Err(err("truncated image data"));
}
break;
}
p = end.saturating_add(size & 1);
}
let mut d = image_webp::WebPDecoder::new(std::io::Cursor::new(bytes)).map_err(err)?;
d.set_memory_limit(bytes.len());
let (w, h) = d.dimensions();
check_size(F, w as u64, h as u64, &DecodeOptions::default())?;
let exif = d.exif_metadata().ok().flatten();
let exif = exif.as_deref().map(|e| e.strip_prefix(b"Exif\0\0").unwrap_or(e));
Ok((w, h, exif.map(crate::exif::summarize).unwrap_or_default().orientation.unwrap_or(1)))
}
+2
View File
@@ -30,8 +30,10 @@ fn try_all(bytes: &[u8]) {
if let Some(f) = sniff(bytes) {
let _ = decode_unguarded(bytes, f, &small_opts());
let _ = decode_unguarded(bytes, f, &DecodeOptions { max_size: Some((8, 8)), max_pixels: 1 << 22 });
let _ = read_header_unguarded(bytes, f);
}
let _ = decode(bytes, small_opts());
let _ = read_header(bytes);
let _ = decode_thumbnail_with(bytes, &ThumbnailOptions { max_pixels: small_opts().max_pixels, ..ThumbnailOptions::new(16) });
let _ = lightcraft_codecs::icc::parse(bytes);
}
+366
View File
@@ -0,0 +1,366 @@
//! `read_header` (import probes) agrees with a decode on dimensions and orientation, refuses what a
//! decode refuses (truncated files included), and returns errors — never panics — on hostile input.
use lightcraft_codecs::exif::minimal_exif;
use lightcraft_codecs::*;
use lightcraft_raster::Rgba8;
fn pixels(w: usize, h: usize) -> Rgba8 {
Rgba8::from_fn(w, h, |x, y| [(x * 37 % 256) as u8, (y * 53 % 256) as u8, ((x ^ y) * 9 % 256) as u8, (128 + x % 100) as u8])
}
fn meta(exif: &Option<Vec<u8>>) -> EncodeMeta<'_> {
EncodeMeta { exif: exif.as_deref(), xmp: Some("<x/>"), ..Default::default() }
}
/// A TIFF with an Orientation tag (`encode_tiff` writes none), via the `tiff` crate's encoder.
fn tiff_oriented(img: &Rgba8, orientation: u16) -> Vec<u8> {
use tiff::encoder::{TiffEncoder, colortype};
let rgb: Vec<u8> = img.data.iter().flat_map(|p| [p[0], p[1], p[2]]).collect();
let mut cur = std::io::Cursor::new(Vec::new());
let mut enc = TiffEncoder::new(&mut cur).unwrap();
let mut im = enc.new_image::<colortype::RGB8>(img.width as u32, img.height as u32).unwrap();
im.encoder().write_tag(tiff::tags::Tag::Orientation, orientation).unwrap();
im.write_data(&rgb).unwrap();
cur.into_inner()
}
/// A minimal 8-bit RGB PSD (raw or PackBits-literal merged data) with an EXIF resource.
fn psd(w: usize, h: usize, orientation: Option<u16>, rle: bool) -> Vec<u8> {
let mut b = Vec::new();
b.extend_from_slice(b"8BPS");
b.extend_from_slice(&1u16.to_be_bytes());
b.extend_from_slice(&[0; 6]);
b.extend_from_slice(&3u16.to_be_bytes());
b.extend_from_slice(&(h as u32).to_be_bytes());
b.extend_from_slice(&(w as u32).to_be_bytes());
b.extend_from_slice(&8u16.to_be_bytes());
b.extend_from_slice(&3u16.to_be_bytes());
b.extend_from_slice(&0u32.to_be_bytes()); // colour mode data
let mut res = Vec::new();
if let Some(o) = orientation {
let exif = minimal_exif(o);
res.extend_from_slice(b"8BIM");
res.extend_from_slice(&1058u16.to_be_bytes());
res.extend_from_slice(&[0, 0]); // empty pascal name, padded
res.extend_from_slice(&(exif.len() as u32).to_be_bytes());
res.extend_from_slice(&exif);
if exif.len() % 2 == 1 {
res.push(0);
}
}
b.extend_from_slice(&(res.len() as u32).to_be_bytes());
b.extend_from_slice(&res);
b.extend_from_slice(&0u32.to_be_bytes()); // layer and mask info
b.extend_from_slice(&(rle as u16).to_be_bytes());
let plane = |c: usize| (0..w * h).map(move |i| ((i * (c + 3) * 11) % 256) as u8);
if rle {
// every row as literal runs of at most 128 bytes (PackBits' longest)
let row_len = w + w.div_ceil(128);
for _ in 0..3 * h {
b.extend_from_slice(&(row_len as u16).to_be_bytes());
}
for c in 0..3 {
let p: Vec<u8> = plane(c).collect();
for row in p.chunks(w) {
for run in row.chunks(128) {
b.push(run.len() as u8 - 1);
b.extend_from_slice(run);
}
}
}
} else {
for c in 0..3 {
b.extend(plane(c));
}
}
b
}
/// Generated files of every format with a header reader, plus a fallback one (BMP via `image`).
fn fixtures(w: usize, h: usize) -> Vec<(String, Vec<u8>)> {
let img = pixels(w, h);
let e = EncodeImage::rgba8(&img);
let rgb: Vec<u8> = img.data.iter().flat_map(|p| [p[0], p[1], p[2]]).collect();
let rgb_e = EncodeImage::new(w as u32, h as u32, 3, Samples::U8(&rgb));
let u16s: Vec<u16> = (0..w * h * 3).map(|i| (i * 331) as u16).collect();
let mut out = Vec::new();
for o in [None, Some(1), Some(6), Some(8)] {
let exif = o.map(minimal_exif);
let tag = format!("o{}", o.unwrap_or(0));
out.push((format!("jpeg 420 {tag}"), encode_jpeg(&rgb_e, 85, ChromaSubsampling::S420, &meta(&exif)).unwrap()));
out.push((format!("jpeg 444 {tag}"), encode_jpeg(&rgb_e, 85, ChromaSubsampling::S444, &meta(&exif)).unwrap()));
out.push((format!("png rgba {tag}"), encode_png(&e, &meta(&exif)).unwrap()));
out.push((format!("png 16-bit {tag}"), encode_png(&EncodeImage::new(w as u32, h as u32, 3, Samples::U16(&u16s)), &meta(&exif)).unwrap()));
out.push((format!("webp {tag}"), encode_webp_lossless(&e, &meta(&exif)).unwrap()));
out.push((format!("psd {tag}"), psd(w, h, o, false)));
out.push((format!("psd rle {tag}"), psd(w, h, o, true)));
out.push((format!("tiff {tag}"), tiff_oriented(&img, o.unwrap_or(1))));
}
out.push(("tiff lzw".into(), encode_tiff(&e, TiffCompression::Lzw, &EncodeMeta::default()).unwrap()));
out.push(("tiff packbits".into(), encode_tiff(&e, TiffCompression::PackBits, &EncodeMeta::default()).unwrap()));
let mut bmp = Vec::new();
image::codecs::bmp::BmpEncoder::new(&mut bmp).encode(&rgb, w as u32, h as u32, image::ExtendedColorType::Rgb8).unwrap();
out.push(("bmp".into(), bmp));
out
}
/// What the import probe learnt before it read headers: a small decode.
fn decoded(bytes: &[u8]) -> Result<Header> {
let d = decode(bytes, DecodeOptions::fit(64, 64))?;
Ok(Header { format: d.format, width: d.source_width, height: d.source_height, orientation: d.orientation })
}
#[test]
fn header_matches_decode() {
for (w, h) in [(160, 96), (96, 160), (7, 5)] {
for (name, bytes) in fixtures(w, h) {
let header = read_header(&bytes).unwrap_or_else(|e| panic!("{name} {w}×{h}: {e}"));
assert_eq!(header, decoded(&bytes).unwrap(), "{name} {w}×{h}");
assert_eq!((header.width, header.height), (w as u32, h as u32), "{name}");
}
}
let o = |name: &str| fixtures(16, 8).into_iter().find(|f| f.0 == name).map(|f| read_header(&f.1).unwrap().orientation).unwrap();
assert_eq!((o("jpeg 420 o6"), o("png rgba o8"), o("webp o6"), o("psd o8"), o("tiff o6"), o("jpeg 444 o0")), (6, 8, 6, 8, 6, 1));
}
/// Whenever the decode the probe used to run accepts a file (here: every cut of every fixture),
/// the header agrees with it: reading headers never refuses a file that decodes. The reverse
/// is checked for truncation, which the header readers detect.
#[test]
fn truncated_files_agree_with_decode() {
// 160 px wide: large JPEGs take the DCT-scaled (strict) decode path the probe took for photos
for (name, bytes) in fixtures(160, 96) {
let n = bytes.len();
let mut cuts: Vec<usize> = (0..100).map(|i| n * i / 100).collect();
cuts.extend(n.saturating_sub(64)..n);
for cut in cuts {
let part = &bytes[..cut];
if let Ok(d) = decoded(part) {
assert_eq!(read_header(part).ok(), Some(d), "{name} cut at {cut}/{n}");
}
}
for cut in [n / 4, n / 2, n * 9 / 10] {
assert!(read_header(&bytes[..cut]).is_err(), "{name}: cut at {cut}/{n} accepted");
}
}
}
#[test]
fn truncation_details() {
let f = fixtures(160, 96);
let get = |name: &str| f.iter().find(|x| x.0 == name).map(|x| x.1.clone()).unwrap();
let jpeg = get("jpeg 420 o6");
assert!(read_header(&jpeg[..jpeg.len() - 2]).is_err(), "a JPEG without its EOI is truncated");
let mut trailing = jpeg.clone();
trailing.extend_from_slice(&[0; 100]);
assert_eq!(read_header(&trailing).unwrap().orientation, 6, "bytes after the EOI are fine");
let png = get("png rgba o6");
assert!(read_header(&png[..png.len() - 2]).is_ok(), "a damaged IEND is ignored, as by a decode");
assert!(read_header(&png[..png.len() - 12]).is_err(), "nothing after the image data: truncated");
}
/// eXIf after the image data counts, as it does for a decode (which reads the whole file).
#[test]
fn png_exif_after_idat() {
let img = pixels(9, 4);
let plain = encode_png(&EncodeImage::rgba8(&img), &EncodeMeta::default()).unwrap();
let exif = minimal_exif(6);
let mut chunk = (exif.len() as u32).to_be_bytes().to_vec();
chunk.extend_from_slice(b"eXIf");
chunk.extend_from_slice(&exif);
chunk.extend_from_slice(&[0; 4]); // CRC: not checked by either reader
let iend = plain.len() - 12;
let mut b = plain[..iend].to_vec();
b.extend_from_slice(&chunk);
b.extend_from_slice(&plain[iend..]);
assert_eq!(read_header(&b).unwrap().orientation, 6);
}
fn jpeg_segments(sof: u8, w: u16, h: u16, comps: u8) -> Vec<u8> {
let mut b = vec![0xFF, 0xD8, 0xFF, sof, 0, 8 + 3 * comps as u16 as u8, 8];
b.extend_from_slice(&h.to_be_bytes());
b.extend_from_slice(&w.to_be_bytes());
b.push(comps);
for c in 0..comps {
b.extend_from_slice(&[c + 1, 0x11, 0]);
}
b.extend_from_slice(&[0xFF, 0xDA, 0, 8, 1, 1, 0, 0, 63, 0]);
b.extend_from_slice(&[0x12, 0xFF, 0x00, 0x34, 0xFF, 0xD0, 0x56]);
b.extend_from_slice(&[0xFF, 0xD9]);
b
}
fn png_ihdr(w: u32, h: u32, depth: u8, color: u8) -> Vec<u8> {
let mut b = b"\x89PNG\r\n\x1a\n\0\0\0\x0dIHDR".to_vec();
b.extend_from_slice(&w.to_be_bytes());
b.extend_from_slice(&h.to_be_bytes());
b.extend_from_slice(&[depth, color, 0, 0, 0, 0, 0, 0, 0]);
b
}
fn png_chunk(b: &mut Vec<u8>, kind: &[u8; 4], len: u32, data: &[u8]) {
b.extend_from_slice(&len.to_be_bytes());
b.extend_from_slice(kind);
b.extend_from_slice(data);
b.extend_from_slice(&[0; 4]);
}
#[test]
fn hostile_headers_are_errors() {
// a well-formed synthetic JPEG header is read
assert_eq!(read_header(&jpeg_segments(0xC0, 33, 21, 3)).unwrap(), Header { format: Format::Jpeg, width: 33, height: 21, orientation: 1 });
let jpeg_bad = [
jpeg_segments(0xC0, 0, 21, 3), // zero width
jpeg_segments(0xC0, 33, 0, 3), // zero height (DNL)
jpeg_segments(0xC0, 65535, 65535, 3), // 4.3 Gpx
jpeg_segments(0xC9, 33, 21, 3), // arithmetic coding
jpeg_segments(0xC5, 33, 21, 3), // hierarchical
jpeg_segments(0xC0, 33, 21, 2), // two components
jpeg_segments(0xC0, 33, 21, 3)[..30].to_vec(),
vec![0xFF, 0xD8, 0xFF, 0xE1, 0, 0], // segment length 0
vec![0xFF, 0xD8, 0xFF, 0xE1, 0xFF, 0xFF], // segment past the end
vec![0xFF, 0xD8, 0xFF, 0xD9],
];
for (i, b) in jpeg_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "JPEG case {i}");
}
let mut no_sos = jpeg_segments(0xC0, 33, 21, 3);
no_sos.truncate(2 + 2 + 17);
no_sos.extend_from_slice(&[0xFF, 0xD9]);
assert!(read_header(&no_sos).is_err(), "JPEG without a scan");
let mut ok = png_ihdr(5, 3, 8, 2);
png_chunk(&mut ok, b"IDAT", 1, &[0]);
png_chunk(&mut ok, b"IEND", 0, &[]);
assert_eq!(read_header(&ok).unwrap(), Header { format: Format::Png, width: 5, height: 3, orientation: 1 });
let png_case = |ihdr: Vec<u8>, chunks: &[(&[u8; 4], u32, &[u8])]| {
let mut b = ihdr;
for (k, l, d) in chunks {
png_chunk(&mut b, k, *l, d);
}
b
};
let idat: (&[u8; 4], u32, &[u8]) = (b"IDAT", 1, &[0]);
let iend: (&[u8; 4], u32, &[u8]) = (b"IEND", 0, &[]);
let png_bad = [
png_case(png_ihdr(0, 3, 8, 2), &[idat, iend]),
png_case(png_ihdr(5, 0, 8, 2), &[idat, iend]),
png_case(png_ihdr(0x8000_0000, 1, 8, 2), &[idat, iend]),
png_case(png_ihdr(65536, 65536, 8, 2), &[idat, iend]),
png_case(png_ihdr(5, 3, 4, 2), &[idat, iend]), // RGB at 4 bits
png_case(png_ihdr(5, 3, 8, 5), &[idat, iend]), // no colour type 5
png_case(png_ihdr(5, 3, 8, 3), &[idat, iend]), // indexed without PLTE
png_case(png_ihdr(5, 3, 8, 2), &[iend]), // no IDAT
png_case(png_ihdr(5, 3, 8, 2), &[idat]), // nothing after the image data
png_case(png_ihdr(5, 3, 8, 2), &[(b"IDAT", 0xFFFF_FFFF, &[0]), iend]),
png_case(png_ihdr(5, 3, 8, 2), &[(b"tEXt", 0x7FFF_FFFF, &[0]), idat, iend]),
png_ihdr(5, 3, 8, 2)[..20].to_vec(),
b"\x89PNG\r\n\x1a\n\0\0\0\x0dIHDX\0\0\0\x05\0\0\0\x03\x08\x02\0\0\0".to_vec(),
];
for (i, b) in png_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "PNG case {i}");
}
let tiff = encode_tiff(&EncodeImage::rgba8(&pixels(6, 4)), TiffCompression::None, &EncodeMeta::default()).unwrap();
let ifd = u32::from_le_bytes([tiff[4], tiff[5], tiff[6], tiff[7]]) as usize;
let mut tiff_bad = vec![b"II*\0\xff\xff\xff\x7f".to_vec(), b"II*\0\x08\0\0\0".to_vec(), b"II*\0\x04\0\0\0".to_vec()];
// cut short after the IFD
assert!(ifd < tiff.len() / 2, "the IFD comes before the image data");
tiff_bad.push(tiff[..tiff.len() - 10].to_vec());
for (i, b) in tiff_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "TIFF case {i}");
}
// a strip offset pointing past the end of the file
let t = |b: &[u8], p: usize| u16::from_le_bytes([b[p], b[p + 1]]);
let n = t(&tiff, ifd) as usize;
let mut far = tiff.clone();
for e in 0..n {
let p = ifd + 2 + e * 12;
if t(&far, p) == 273 && t(&far, p + 2) == 4 && u32::from_le_bytes([far[p + 4], far[p + 5], far[p + 6], far[p + 7]]) == 1 {
far[p + 8..p + 12].copy_from_slice(&0x7FFF_0000u32.to_le_bytes());
}
}
assert_ne!(far, tiff, "the strip offset was found");
assert!(read_header(&far).is_err(), "strip past the end");
let webp_bad = [
b"RIFF\0\0\0\0WEBP".to_vec(),
b"RIFF\0\0\0\0WEBPVP8L\xff\xff\xff\xff\x2f".to_vec(),
b"RIFF\0\0\0\0WEBPXXXX\xff\xff\xff\xff".to_vec(),
b"RIFF\0\0\0\0WEBPVP8 \x04\0\0\0\0\0\0\0".to_vec(),
];
for (i, b) in webp_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "WebP case {i}");
}
let good = psd(4, 3, Some(6), true);
assert_eq!(read_header(&good).unwrap().orientation, 6);
let set = |pos: usize, v: &[u8]| {
let mut b = good.clone();
b[pos..pos + v.len()].copy_from_slice(v);
b
};
let res_len = u32::from_be_bytes([good[30], good[31], good[32], good[33]]) as usize;
let compression = 34 + res_len + 4;
let mut psd_bad = vec![
set(12, &0u16.to_be_bytes()), // no channels
set(12, &57u16.to_be_bytes()), // too many
set(12, &2u16.to_be_bytes()), // too few for RGB
set(14, &0x7FFF_FFFFu32.to_be_bytes()), // absurd height
set(18, &0u32.to_be_bytes()), // zero width
set(22, &1u16.to_be_bytes()), // 1-bit
set(22, &7u16.to_be_bytes()), // bad depth
set(24, &9u16.to_be_bytes()), // Lab
set(24, &77u16.to_be_bytes()), // unknown mode
set(24, &2u16.to_be_bytes()), // indexed without a palette
set(compression, &2u16.to_be_bytes()), // ZIP
set(26, &0xFFFF_FFFFu32.to_be_bytes()), // colour mode data past the end
set(30, &0xFFFF_FFF0u32.to_be_bytes()), // resources past the end
];
// a 1 × 2^30 RLE image declared in a tiny file: refused without reading 2^30 row counts
let mut tall = good.clone();
tall[14..18].copy_from_slice(&(1u32 << 30).to_be_bytes());
tall[18..22].copy_from_slice(&1u32.to_be_bytes());
psd_bad.push(tall);
psd_bad.push(good[..good.len() - 1].to_vec());
for (i, b) in psd_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "PSD case {i}");
}
for b in [&b""[..], b"\xFF\xD8\xFF", b"\x89PNG\r\n\x1a\n", b"II*\0", b"MM\0*", b"8BPS\0\x01", b"GIF89a", b"not an image at all"] {
assert!(read_header(b).is_err(), "{b:?}");
}
}
/// The same with a few random bytes changed: whenever the decode accepts the damaged file, the
/// header agrees with it (deterministic seed).
#[test]
fn mutated_files_agree_with_decode() {
let mut s: u64 = 0x1234_5678_9abc_def1;
let mut rnd = || {
s ^= s << 13;
s ^= s >> 7;
s ^= s << 17;
s
};
let (mut decodable, mut disagreements) = (0, Vec::new());
for (name, bytes) in fixtures(160, 96) {
for _ in 0..300 {
let mut b = bytes.clone();
for _ in 0..1 + rnd() % 3 {
let i = rnd() as usize % b.len();
b[i] = rnd() as u8;
}
if let Ok(d) = decoded(&b) {
decodable += 1;
let h = read_header(&b);
if h.as_ref().ok() != Some(&d) {
disagreements.push(format!("{name}: decode {d:?}, header {h:?}"));
}
}
}
}
assert!(decodable > 1000, "{decodable}");
assert!(disagreements.is_empty(), "{disagreements:#?}");
}
+74
View File
@@ -245,6 +245,80 @@ fn tiff_cmyk_and_orientation() {
assert!(red[0] > 0.99 && red[1] < 0.01 && red[2] < 0.01, "{red:?}");
}
/// A one-strip, uncompressed, single-sample TIFF (little-endian) whose samples are `bits` wide and
/// already packed into `strip`, one padded row after another.
fn packed_tiff(width: u16, height: u16, bits: u16, photometric: u16, strip: &[u8]) -> Vec<u8> {
const STRIP_AT: u32 = 8 + 2 + 9 * 12 + 4;
let short = |tag: u16, v: u16| [&tag.to_le_bytes()[..], &3u16.to_le_bytes(), &1u32.to_le_bytes(), &v.to_le_bytes(), &[0, 0]].concat();
let long = |tag: u16, v: u32| [&tag.to_le_bytes()[..], &4u16.to_le_bytes(), &1u32.to_le_bytes(), &v.to_le_bytes()].concat();
let mut b = b"II*\0".to_vec();
b.extend(8u32.to_le_bytes());
b.extend(9u16.to_le_bytes());
b.extend(short(256, width));
b.extend(short(257, height));
b.extend(short(258, bits));
b.extend(short(259, 1));
b.extend(short(262, photometric));
b.extend(long(273, STRIP_AT));
b.extend(short(277, 1));
b.extend(short(278, height));
b.extend(long(279, strip.len() as u32));
b.extend(0u32.to_le_bytes());
assert_eq!(b.len(), STRIP_AT as usize);
b.extend_from_slice(strip);
b
}
/// WhiteIsZero (photometric 0) gray: the lowest sample is white. The `tiff` crate already inverts these
/// samples, and a second inversion here used to show every WhiteIsZero scan as a negative.
#[test]
fn tiff_white_is_zero_is_inverted_once() {
// Two pixels: the lowest sample, then the highest, at 8 and 16 bits.
for (bits, strip) in [(8, vec![0u8, 255]), (16, [0u16.to_le_bytes(), 65535u16.to_le_bytes()].concat())] {
for (photometric, first) in [(0, 1.0), (1, 0.0)] {
let d = decode(&packed_tiff(2, 1, bits, photometric, &strip), DecodeOptions::default()).unwrap();
let (a, b) = (d.image.get(0, 0)[1], d.image.get(1, 0)[1]);
assert!((a - first).abs() < 1e-6 && (b - (1.0 - first)).abs() < 1e-6, "{bits}-bit, photometric {photometric}: {a} {b}");
}
}
}
/// 1-, 2- and 4-bit samples (bilevel scans, fax images, low-depth gray) arrive packed eight, four or
/// two to a byte with each row padded to a byte; they used to be rejected as "sample buffer too short".
#[test]
fn tiff_samples_narrower_than_a_byte() {
let gray = |d: &Decoded, x: usize, y: usize| d.image.get(x, y)[1];
// 10 × 2 bilevel: rows are 2 bytes, the last 6 bits of each are padding (set, to prove they are ignored).
// Row 0 alternates set/clear bits; row 1 is clear but for its last pixel.
let rows = [0b1010_1010, 0b1011_1111, 0b0000_0000, 0b0111_1111];
// What a set bit decodes to: white for BlackIsZero (1), black for WhiteIsZero (0).
for (photometric, set_bit) in [(1, 1.0), (0, 0.0)] {
let d = decode(&packed_tiff(10, 2, 1, photometric, &rows), DecodeOptions::default()).unwrap();
assert_eq!((d.image.width, d.image.height, d.bit_depth), (10, 2, 1));
assert!(d.grayscale);
let clear_bit = 1.0 - set_bit;
for x in 0..10 {
let want = (if x % 2 == 0 { set_bit } else { clear_bit }, if x == 9 { set_bit } else { clear_bit });
assert!((gray(&d, x, 0) - want.0).abs() < 1e-6, "row 0, x {x}, photometric {photometric}");
assert!((gray(&d, x, 1) - want.1).abs() < 1e-6, "row 1, x {x}, photometric {photometric}");
}
}
// 2-bit: 0, 1, 2, 3 → 0, 85, 170, 255. 4-bit: 0, 15, 8 → 0, 255, 136 (the row's last nibble is padding).
let d = decode(&packed_tiff(4, 1, 2, 1, &[0b00_01_10_11]), DecodeOptions::default()).unwrap();
for (x, v) in [0, 85, 170, 255].into_iter().enumerate() {
assert!((gray(&d, x, 0) - srgb_to_linear(v as f32 / 255.0)).abs() < 1e-6, "2-bit x {x}");
}
let d = decode(&packed_tiff(3, 1, 4, 1, &[0x0F, 0x8F]), DecodeOptions::default()).unwrap();
for (x, v) in [0, 255, 136].into_iter().enumerate() {
assert!((gray(&d, x, 0) - srgb_to_linear(v as f32 / 255.0)).abs() < 1e-6, "4-bit x {x}");
}
// A strip shorter than its rows is an error, never a panic.
assert!(decode(&packed_tiff(10, 2, 1, 1, &rows[..3]), DecodeOptions::default()).is_err());
}
#[test]
fn webp_lossless_exact() {
let img = gradient(31, 19);
+17 -6
View File
@@ -2,7 +2,9 @@
//!
//! The Planckian locus uses the Kim et al. (2002) cubic approximation (1667–25000 K); outside that
//! range it is extended linearly in mired space. Tint is a signed offset perpendicular to the locus in
//! CIE 1960 (u, v): **positive tint = magenta** (below the locus), `Duv = −tint / TINT_SCALE`.
//! CIE 1960 (u, v). Tint describes the **correction**: positive adds magenta, negative adds green.
//! A positive correction neutralises a green illuminant above the locus, so its source white uses
//! `Duv = tint / TINT_SCALE`. Bradford adaptation removes that source cast.
use crate::{Mat3, RgbSpace, Xy, bradford};
@@ -65,11 +67,12 @@ fn locus_normal(t: f64) -> (f64, f64) {
if ny < 0.0 { (-nx, -ny) } else { (nx, ny) }
}
/// Chromaticity of the white point for temperature `t` (K) and `tint`.
/// Source illuminant chromaticity for temperature `t` (K) and correction `tint`.
/// Positive tint means a green source white, whose correction adds magenta to the image.
pub fn temp_tint_to_xy(t: f64, tint: f64) -> Xy {
let (u, v) = locus_uv(t);
let (nu, nv) = locus_normal(t);
let duv = -tint / TINT_SCALE;
let duv = tint / TINT_SCALE;
uv_to_xy(u + nu * duv, v + nv * duv)
}
@@ -104,7 +107,7 @@ pub fn xy_to_temp_tint(p: Xy) -> (f64, f64) {
let (lu, lv) = locus_uv(t);
let (nu, nv) = locus_normal(t);
let duv = (u - lu) * nu + (v - lv) * nv;
(t, -duv * TINT_SCALE)
(t, duv * TINT_SCALE)
}
/// Matrix (linear RGB in `space` → same space) that white-balances a scene lit by `src` so that it
@@ -141,10 +144,18 @@ mod tests {
let warm = temp_tint_to_xy(3000.0, 0.0);
let cool = temp_tint_to_xy(9000.0, 0.0);
assert!(warm.x > cool.x);
// positive tint is magenta: lower v
// A positive magenta correction specifies a greener source illuminant: higher v.
let (_, v0) = xy_to_uv(temp_tint_to_xy(5000.0, 0.0));
let (_, v1) = xy_to_uv(temp_tint_to_xy(5000.0, 50.0));
assert!(v1 < v0);
assert!(v1 > v0);
}
/// Daylight lies above the Planckian locus, so D65 reads as a small positive tint, as
/// Lightroom's Daylight preset (5500 K, +10) has it (#188).
#[test]
fn daylight_has_a_positive_tint() {
let (_, tint) = xy_to_temp_tint(crate::D65);
assert!((5.0..15.0).contains(&tint), "{tint}");
}
#[test]
+26
View File
@@ -68,6 +68,32 @@ impl DevelopSettings {
!self.disabled_sections.iter().any(|s| s == section)
}
/// The settings as rendered: an Edit-panel section whose eye is off contributes nothing (issue
/// #316). Light (with its tone curve), Color (white balance, presence, the mixers, colour grading
/// and point colour) and Detail are reset to their defaults here; Effects, Optics, Geometry and
/// Calibration the pipeline switches off where it applies them. Borrowed when every section is on.
pub fn effective(&self) -> std::borrow::Cow<'_, DevelopSettings> {
let off = |section: &str| !self.section_enabled(section);
if !(off("light") || off("color") || off("detail")) {
return std::borrow::Cow::Borrowed(self);
}
let mut d = self.clone();
if off("light") {
d.reset_section(Section::Light);
d.reset_section(Section::Curve);
}
if off("color") {
for section in [Section::Color, Section::Mixer, Section::BwMix, Section::Grading, Section::PointColor] {
d.reset_section(section);
}
d.point_colors.clear();
}
if off("detail") {
d.reset_section(Section::Detail);
}
std::borrow::Cow::Owned(d)
}
pub fn set_section_enabled(&mut self, section: &str, on: bool) {
self.disabled_sections.retain(|s| s != section);
if !on {
+4 -2
View File
@@ -158,6 +158,7 @@ impl WbMode {
pub struct WhiteBalance {
pub mode: WbMode,
pub temp: f64,
/// Correction direction: negative adds green, positive adds magenta.
pub tint: f64,
}
@@ -539,8 +540,9 @@ impl Default for Optics {
}
}
/// Lens corrections embedded in a DNG file (`OpcodeList3`: `WarpRectilinear`, `FixVignetteRadial`), converted
/// to the oriented, default-cropped image. This is camera/file data (stored on the photo record, not in the develop
/// Lens corrections embedded in the file, converted to the oriented, default-cropped image: a DNG's `OpcodeList3`
/// (`WarpRectilinear`, `FixVignetteRadial`), or a raw reader's equivalent of the camera's own correction (Panasonic /
/// Leica RW2 distortion, `lightcraft_raw`'s `vendor/rw2.rs`). This is camera/file data (stored on the photo record, not in the develop
/// settings); "Enable Profile Corrections" applies it, scaled by the profile distortion/vignetting amounts.
/// LightCraft never uses Adobe LCP lens profiles.
#[derive(Clone, Copy, Debug, Default, PartialEq, Serialize, Deserialize)]
+3
View File
@@ -30,6 +30,8 @@ serde_json = { workspace = true }
thiserror = { workspace = true }
log = { workspace = true }
ab_glyph = { workspace = true }
# Apply the font’s vertical substitutions and origins to Japanese watermarks.
harfrust = { version = "0.12", default-features = false, features = ["std"] }
lightcraft-segment = { workspace = true, optional = true }
[features]
@@ -38,3 +40,4 @@ sam = ["dep:lightcraft-segment"]
[dev-dependencies]
libc = { workspace = true }
lightcraft-tiff = { workspace = true }
+3 -1
View File
@@ -93,7 +93,9 @@ fn main() {
for path in &args {
let bytes = std::fs::read(path).expect("read");
println!("{path}");
let raw = lightcraft_raw::decode(&bytes).expect("decode");
let mut raw = lightcraft_raw::decode(&bytes).expect("decode");
// as the loader does: lens corrections are applied by the pipeline, not in the raw develop
raw.opcodes.list3.retain(|op| !op.is_lens_correction());
println!(" {}×{} cpp {} cfa {:?}", raw.width, raw.height, raw.cpp, raw.cfa.as_ref().map(|c| c.name()));
println!(" {:<36} {}", "raw decode:", best(n, || drop(lightcraft_raw::decode(&bytes).expect("decode"))));
println!(" {:<36} {}", "normalize:", best(n, || drop(raw.normalized().expect("norm"))));
+405 -22
View File
@@ -1,5 +1,5 @@
//! Estimate the starting look of a raw without a camera colour matrix (Sony ARW, Nikon NEF, Panasonic
//! RW2) from its own JPEG. Colour and luminance are fitted separately; the JPEG supplies correspondences only,
//! RW2, Fujifilm RAF, Canon CR3) from its own JPEG. Colour and luminance are fitted separately; the JPEG supplies correspondences only,
//! never output pixels or a replacement for RAW editing.
//! A global matrix can't follow the camera's hue-dependent rendering (the best matrix rendered a
//! lime shirt olive that the camera kept lime): a hue/saturation table fitted to the residuals
@@ -31,7 +31,7 @@ pub(crate) const PROFILE_PROXY: usize = 192;
/// as-shot look (`docs/camera-preview-colour.md`). The catalog's `Photo::relative_wb` matches the
/// same formats by file extension.
pub(crate) fn file_local_look(format: RawFormat) -> bool {
matches!(format, RawFormat::Arw | RawFormat::Nef | RawFormat::Nrw | RawFormat::Rw2)
matches!(format, RawFormat::Arw | RawFormat::Nef | RawFormat::Nrw | RawFormat::Rw2 | RawFormat::Raf | RawFormat::Cr3)
}
pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransform) -> Option<CameraLook> {
@@ -40,13 +40,13 @@ pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransf
}
let (sensor, reference) = proxies(raw, bytes, transform, PROXY)?;
// A camera profile pooled from many photos knows colours this photo shows too little of;
// only the tone and chroma curves are fitted per photo (DRO and picture styles vary).
// try its colour first and fit tone/chroma per photo (DRO and picture styles vary).
let profile = raw.metadata.model.as_deref().and_then(crate::camera_profiles::get);
let colour = profile.as_ref().and_then(|p| Some((p.matrix().mul(&transform.matrix.inverse()?), p.hue_sat.clone())));
let look = fit_pairs_with(&sensor, &reference, colour)?;
if lightcraft_pipeline::profiling() {
eprintln!(
"[profile] {:?} camera look: {:?}, {:?}, hue/sat table {}, camera profile {}",
"[profile] {:?} camera look: {:?}, {:?}, hue/sat table {}, camera profile available {}",
raw.format,
look.matrix.0,
look.tone,
@@ -60,9 +60,8 @@ pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransf
/// Same-size proxies of the sensor (white-balanced, baseline exposure, through `transform`'s
/// matrix: the generic camera ≈ sRGB model) and of the file's embedded camera JPEG (linear Rec.2020).
fn proxies(raw: &RawImage, bytes: &[u8], transform: &CameraTransform, size: usize) -> Option<(Rgb32f, Rgb32f)> {
let jpeg = lightcraft_raw::embedded_preview(bytes)?;
let edge = (2 * size).max(384) as u32;
let decoded = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions { max_size: Some((edge, edge)), max_pixels: 64_000_000 }).ok()?;
let decoded = crate::files::decode_raw_preview(bytes, lightcraft_codecs::DecodeOptions { max_size: Some((edge, edge)), max_pixels: 64_000_000 })?;
let mut reference = decoded.to_working();
let (a, crop) = (raw.active_area, raw.crop.clipped(raw.active_area.width, raw.active_area.height));
if crop.width == 0 || crop.height == 0 || reference.width == 0 || reference.height == 0 {
@@ -84,14 +83,161 @@ fn proxies(raw: &RawImage, bytes: &[u8], transform: &CameraTransform, size: usiz
}
// Fixed, bounded proxy: the selected look cannot depend on thumbnail/export resolution.
let k = (crop.width.max(crop.height).div_ceil(edge as usize).max(2)).div_ceil(2) * 2;
let sensor = sensor_proxy(raw, k, edge as usize)?;
let mut sensor = sensor_proxy(raw, k, edge as usize)?;
let gain = 2f32.powf(transform.baseline_exposure as f32);
let to_working = |p: [f32; 3]| transform.matrix.apply_f32(std::array::from_fn(|i| p[i] * transform.wb[i] * gain));
if raw.format == RawFormat::Cr3 {
// Orient the camera JPEG and the sensor identically before collecting pixel correspondences.
sensor = sensor.into_oriented(raw.orientation);
reference = reference.into_oriented(raw.orientation);
sensor.map_in_place(to_working);
// Canon's JPEG can be cropped differently from the sensor. Estimate only that common
// framing from edge directions, independently of the subsequent colour fit.
sensor = align_cr3_framing(sensor, &reference);
}
let mut sensor = fit(&sensor, size, size, Filter::Box);
let reference = fit(&reference, sensor.width, sensor.height, Filter::Box);
let gain = 2f32.powf(transform.baseline_exposure as f32);
sensor.map_in_place(|p| transform.matrix.apply_f32(std::array::from_fn(|i| p[i] * transform.wb[i] * gain)));
if raw.format != RawFormat::Cr3 {
sensor.map_in_place(to_working);
}
Some((sensor, reference))
}
#[derive(Clone, Copy, Debug)]
struct Cr3Framing {
scale: f32,
/// Translation as a fraction of the oriented frame's width and height.
offset: [f32; 2],
}
impl Cr3Framing {
const IDENTITY: Self = Self { scale: 1.0, offset: [0.0; 2] };
fn source(self, x: f32, y: f32, w: usize, h: usize) -> (f32, f32) {
let (w, h) = (w as f32, h as f32);
((x - w * 0.5) * self.scale + w * (0.5 + self.offset[0]), (y - h * 0.5) * self.scale + h * (0.5 + self.offset[1]))
}
}
/// Register the JPEG's remaining framing on a fixed proxy. Edge directions survive different
/// white balance and monotone camera tone; no target RGB values train this three-parameter fit.
/// A separate third of the edges must confirm a substantial improvement and a close match.
/// A weak, flat or unrelated preview leaves the sensor framing unchanged.
fn estimate_cr3_framing(sensor: &Rgb32f, reference: &Rgb32f) -> Option<Cr3Framing> {
if sensor.width.checked_mul(sensor.height)? != sensor.data.len() || reference.width.checked_mul(reference.height)? != reference.data.len() {
return None;
}
let sensor = fit(sensor, PROXY, PROXY, Filter::Box);
let (w, h) = (sensor.width, sensor.height);
if w < 16 || h < 16 {
return None;
}
let reference = fit(reference, w, h, Filter::Box);
if (reference.width, reference.height) != (w, h) {
return None;
}
let mut gradients = Rgb32f::new(w, h);
let mut edges = Vec::new();
for y in 1..h - 1 {
for x in 1..w - 1 {
let derivative = |image: &Rgb32f| {
[
luminance_2020(image.data[y * w + x + 1]) - luminance_2020(image.data[y * w + x - 1]),
luminance_2020(image.data[(y + 1) * w + x]) - luminance_2020(image.data[(y - 1) * w + x]),
]
};
let d = derivative(&sensor);
gradients.data[y * w + x] = [d[0], d[1], 0.0];
let d = derivative(&reference);
let magnitude = d[0].hypot(d[1]);
let value = luminance_2020(reference.data[y * w + x]);
if (0.02..0.9).contains(&value) && magnitude.is_finite() && magnitude > 0.03 {
edges.push((x as f32 + 0.5, y as f32 + 0.5, d, magnitude, (x * 17 + y * 11) % 3 == 0));
}
}
}
if edges.len() < 192 {
return None;
}
let score = |framing: Cr3Framing, held_out: bool| -> Option<f64> {
let (mut agreement, mut weight, mut valid, mut total) = (0.0, 0.0, 0usize, 0usize);
for &(x, y, target, magnitude, holdout) in &edges {
if holdout != held_out {
continue;
}
total += 1;
let (sx, sy) = framing.source(x, y, w, h);
if sx < 1.5 || sy < 1.5 || sx > w as f32 - 1.5 || sy > h as f32 - 1.5 {
continue;
}
let sample = gradients.sample_bilinear(sx, sy);
let length = sample[0].hypot(sample[1]);
if !length.is_finite() || length <= 0.01 {
continue;
}
let k = f64::from(magnitude.min(0.25));
agreement += k * f64::from((sample[0] * target[0] + sample[1] * target[1]) / (length * magnitude));
weight += k;
valid += 1;
}
(valid >= 64 && valid * 5 >= total * 4 && weight > 0.0).then_some(agreement / weight)
};
let before = score(Cr3Framing::IDENTITY, false)?;
let mut best = (before, Cr3Framing::IDENTITY);
// At most 10% zoom and two proxy pixels of translation; a coarse search followed by one
// local refinement prevents unbounded optimisation on the photograph's contents.
for s in -10..=10 {
for tx in -4..=4 {
for ty in -4..=4 {
let framing = Cr3Framing { scale: 1.0 + s as f32 * 0.01, offset: [tx as f32 * 0.5 / w as f32, ty as f32 * 0.5 / h as f32] };
if let Some(value) = score(framing, false)
&& value > best.0
{
best = (value, framing);
}
}
}
}
let coarse = best.1;
for s in -5..=5 {
for tx in -4..=4 {
for ty in -4..=4 {
let framing = Cr3Framing {
scale: coarse.scale + s as f32 * 0.002,
offset: [coarse.offset[0] + tx as f32 * 0.125 / w as f32, coarse.offset[1] + ty as f32 * 0.125 / h as f32],
};
if !(0.9..=1.1).contains(&framing.scale) || framing.offset[0].abs() > 2.0 / w as f32 || framing.offset[1].abs() > 2.0 / h as f32 {
continue;
}
if let Some(value) = score(framing, false)
&& value > best.0
{
best = (value, framing);
}
}
}
}
let (held_before, held_after) = (score(Cr3Framing::IDENTITY, true)?, score(best.1, true)?);
if best.0 < 0.9 || best.0 < before + 0.04 || held_after < 0.9 || held_after < held_before + 0.04 {
return None;
}
if lightcraft_pipeline::profiling() {
eprintln!("[profile] CR3 JPEG framing {:?}, held-out edge agreement {held_before:.4} -> {held_after:.4}", best.1);
}
Some(best.1)
}
fn align_cr3_framing(sensor: Rgb32f, reference: &Rgb32f) -> Rgb32f {
let Some(framing) = estimate_cr3_framing(&sensor, reference) else { return sensor };
let (w, h) = (sensor.width, sensor.height);
let mut aligned = Rgb32f::new(w, h);
for (i, pixel) in aligned.data.iter_mut().enumerate() {
let (x, y) = framing.source((i % w) as f32 + 0.5, (i / w) as f32 + 0.5, w, h);
*pixel = if x >= 0.5 && y >= 0.5 && x <= w as f32 - 0.5 && y <= h as f32 - 0.5 { sensor.sample_bilinear(x, y) } else { [f32::NAN; 3] };
}
aligned
}
/// Colour training pairs of one raw for a camera profile: white-balanced camera RGB (with the
/// baseline exposure) → its camera JPEG (linear Rec.2020). `None` for formats with their own
/// colour matrices, other files or unusable previews.
@@ -102,7 +248,7 @@ pub(crate) fn profile_pairs(raw: &RawImage, bytes: &[u8]) -> Option<Vec<([f64; 3
let transform = lightcraft_raw::color::camera_transform(raw, lightcraft_raw::color::as_shot_white_xy(raw));
let (sensor, reference) = proxies(raw, bytes, &transform, PROFILE_PROXY)?;
let to_camera = transform.matrix.inverse()?;
let (pairs, _) = collect_pairs(&sensor, &reference)?;
let (pairs, _) = collect_pairs(&sensor, &reference, 0.05, None)?;
Some(pairs.into_iter().map(|(x, y)| (to_camera.apply(x), y)).collect())
}
@@ -150,10 +296,31 @@ fn fit_pairs(sensor: &Rgb32f, reference: &Rgb32f) -> Option<CameraLook> {
fit_pairs_with(sensor, reference, None)
}
/// Largest luminance ratio within the 3×3 neighbourhood of pixel `i` (1 on flat areas, large on edges).
fn local_contrast(image: &Rgb32f, i: usize) -> f32 {
let (w, h) = (image.width, image.height);
if w == 0 {
return f32::INFINITY;
}
let (x, y) = (i % w, i / w);
let (mut lo, mut hi) = (f32::INFINITY, 0f32);
for row in y.saturating_sub(1)..(y + 2).min(h) {
for col in x.saturating_sub(1)..(x + 2).min(w) {
let l = image.data.get(row * w + col).map_or(0.0, |p| luminance_2020(*p));
lo = lo.min(l);
hi = hi.max(l);
}
}
if hi.is_finite() { hi / lo.max(1e-4) } else { f32::INFINITY }
}
/// Training pairs (sensor → JPEG, unclipped midtones) and the wider set including highlights
/// (for the chroma curve); `None` when too few, or the photo has too little colour.
/// (for the chroma curve); `None` when too few, or the photo has too little colour. With
/// `edge_limit`, pixels whose 3×3 neighbourhood in either proxy spans a larger luminance ratio
/// are left out: there a small geometric mismatch between the raw and its JPEG pairs unrelated
/// colours (see [`EDGE_CONTRAST`]).
type Pairs = Vec<([f64; 3], [f64; 3])>;
fn collect_pairs(sensor: &Rgb32f, reference: &Rgb32f) -> Option<(Pairs, Pairs)> {
fn collect_pairs(sensor: &Rgb32f, reference: &Rgb32f, min_chroma: f32, edge_limit: Option<f32>) -> Option<(Pairs, Pairs)> {
if (sensor.width, sensor.height) != (reference.width, reference.height) || sensor.data.len() != reference.data.len() {
return None;
}
@@ -161,7 +328,10 @@ fn collect_pairs(sensor: &Rgb32f, reference: &Rgb32f) -> Option<(Pairs, Pairs)>
// Highlights too (camera JPEGs bleach colours toward white there), for the chroma curve only.
let mut bright = Vec::new();
let mut colour = 0;
for (input, output) in sensor.data.iter().zip(&reference.data) {
for (i, (input, output)) in sensor.data.iter().zip(&reference.data).enumerate() {
if edge_limit.is_some_and(|limit| local_contrast(sensor, i) > limit || local_contrast(reference, i) > limit) {
continue;
}
let y = luminance_2020(*output);
if !input.iter().all(|v| v.is_finite() && *v > 0.001 && *v < 1.5) || !output.iter().all(|v| v.is_finite() && *v >= 0.0) {
continue;
@@ -174,7 +344,7 @@ fn collect_pairs(sensor: &Rgb32f, reference: &Rgb32f) -> Option<(Pairs, Pairs)>
}
let min = output.iter().copied().fold(f32::INFINITY, f32::min);
let max = output.iter().copied().fold(0.0, f32::max);
colour += usize::from(max - min > 0.05);
colour += usize::from(max - min > min_chroma);
pairs.push((input.map(f64::from), output.map(f64::from)));
}
(pairs.len() >= 256 && colour >= pairs.len() / 20).then_some((pairs, bright))
@@ -215,11 +385,36 @@ fn fit_matrix(pairs: &[([f64; 3], [f64; 3])]) -> Option<Mat3> {
matrix.0.iter().flatten().all(|v| v.is_finite() && v.abs() < 8.0).then_some(matrix)
}
/// Neighbourhood luminance ratio above which a pixel counts as an edge for the second attempt
/// of [`fit_pairs_with`]. Camera JPEGs are often lens-corrected (Sony "Distortion Comp.: Auto",
/// compacts and kit zooms): on a 51 mm ILCE-7RM2 shot of a glass façade the JPEG is up to 15 px
/// of 1440 (about one proxy pixel) off the raw, and the fit failed the gate on the mismatched
/// window frames alone (held-out RMS 0.111; 0.059 for the same shot with distortion correction
/// off). Away from edges: 0.049. Issue #232.
const EDGE_CONTRAST: f32 = 3.0;
/// The photo's look: its own matrix (with and without a hue/saturation table), or the given
/// colour model (a camera profile's, in the sensor proxy's space), each completed with a tone
/// and chroma curve fitted to this photo.
/// and chroma curve fitted to this photo. When the fit on all pixels fails the acceptance gates,
/// it is tried once more on the pixels away from edges (same gates): there colour pairs stay
/// valid when the camera JPEG's geometry differs slightly from the raw's.
fn fit_pairs_with(sensor: &Rgb32f, reference: &Rgb32f, colour: Option<(Mat3, Option<HsvTable>)>) -> Option<CameraLook> {
let (pairs, bright) = collect_pairs(sensor, reference)?;
let has_profile = colour.is_some();
// A different picture style can make a camera profile fail the gates. Preserve the
// photo's own colour fit before resorting to the neutral fallback.
fit_candidate(sensor, reference, colour).or_else(|| has_profile.then(|| fit_candidate(sensor, reference, None)).flatten())
}
/// One candidate colour model, fitted on all pixels and, when that fails the gates, once more on
/// the pixels away from edges (issue #232).
fn fit_candidate(sensor: &Rgb32f, reference: &Rgb32f, colour: Option<(Mat3, Option<HsvTable>)>) -> Option<CameraLook> {
fit_pairs_on(sensor, reference, colour.clone(), None).or_else(|| fit_pairs_on(sensor, reference, colour, Some(EDGE_CONTRAST)))
}
fn fit_pairs_on(sensor: &Rgb32f, reference: &Rgb32f, colour: Option<(Mat3, Option<HsvTable>)>, edge_limit: Option<f32>) -> Option<CameraLook> {
// A known colour model needs enough signal for tone fitting, not a scene rich enough to
// learn a new colour matrix. Still reject monochrome references.
let (pairs, bright) = collect_pairs(sensor, reference, if colour.is_some() { 0.005 } else { 0.05 }, edge_limit)?;
let candidates = match colour {
Some(given) => vec![given],
None => {
@@ -270,9 +465,10 @@ fn fit_pairs_with(sensor: &Rgb32f, reference: &Rgb32f, colour: Option<(Mat3, Opt
.sum();
if lightcraft_pipeline::profiling() {
eprintln!(
"[profile] camera look holdout RMS {:.5} -> {:.5} ({samples} channels)",
"[profile] camera look holdout RMS {:.5} -> {:.5} ({samples} channels{})",
(before / samples as f64).sqrt(),
(after / samples as f64).sqrt()
(after / samples as f64).sqrt(),
if edge_limit.is_some() { ", away from edges" } else { "" }
);
}
if samples == 0 || after >= before * MIN_IMPROVEMENT || after / samples as f64 > MAX_HOLDOUT_RMS.powi(2) {
@@ -740,7 +936,7 @@ mod tests {
let mut pool = Vec::new();
for seed in 0..4 {
let (sensor, reference) = photo(seed, 2.0 + seed as f32 * 0.5);
pool.extend(collect_pairs(&sensor, &reference).unwrap().0);
pool.extend(collect_pairs(&sensor, &reference, 0.05, None).unwrap().0);
}
let (matrix, _) = fit_profile(&pool).unwrap();
// the camera's colours (luminance-normalised: the tone curve sets brightness)
@@ -767,6 +963,48 @@ mod tests {
assert!(error.sqrt() < 0.02, "this photo's own tone is followed: RMS {}", error.sqrt());
}
#[test]
fn profile_fits_tone_without_learning_colour_from_a_dull_scene() {
let matrix = Mat3([[2.0, -0.02, 0.0], [0.0, 2.0, 0.0], [0.0, -0.03, 2.0]]);
let mut sensor = Rgb32f::new(64, 48);
let mut reference = sensor.clone();
for (i, (src, dst)) in sensor.data.iter_mut().zip(&mut reference.data).enumerate() {
*src = [0.02 + (i % 997) as f32 * 0.00045; 3];
let p = matrix.apply_f32(*src);
let y = luminance_2020(p);
*dst = p.map(|v| v * (1.0 - (-2.5 * y).exp()) / y);
}
assert!(fit_pairs(&sensor, &reference).is_none(), "not enough colour to learn a matrix");
let look = fit_pairs_with(&sensor, &reference, Some((matrix, None))).unwrap();
assert_eq!(look.matrix, matrix);
reference.map_in_place(|p| [luminance_2020(p); 3]);
assert!(fit_pairs_with(&sensor, &reference, Some((matrix, None))).is_none(), "monochrome JPEGs remain rejected");
}
#[test]
fn rejected_profile_preserves_a_usable_per_photo_fit() {
let matrix = Mat3([[1.8, -0.4, -0.1], [-0.2, 1.5, -0.1], [-0.05, -0.3, 1.7]]);
let wrong = Mat3([[0.0, 0.0, 1.0], [0.0, 1.0, 0.0], [1.0, 0.0, 0.0]]);
let mut sensor = Rgb32f::new(64, 64);
let mut reference = sensor.clone();
let mut seed = 0x2545_f491_u32;
for (src, dst) in sensor.data.iter_mut().zip(&mut reference.data) {
*src = std::array::from_fn(|_| {
seed ^= seed << 13;
seed ^= seed >> 17;
seed ^= seed << 5;
0.05 + (seed % 1000) as f32 * 0.0003
});
let p = matrix.apply_f32(*src);
let y = luminance_2020(p);
*dst = p.map(|v| (v * (1.0 - (-2.5 * y).exp()) / y).clamp(0.0, 0.999));
}
let per_photo = fit_pairs(&sensor, &reference).unwrap();
let look = fit_pairs_with(&sensor, &reference, Some((wrong, None))).unwrap();
assert_eq!(look.matrix, per_photo.matrix, "a rejected camera profile must not suppress the per-photo fit");
assert_eq!(look.tone, per_photo.tone);
}
#[test]
fn hue_sat_passes_black_and_non_finite_pixels_through() {
let table = HsvTable { hue_divisions: 4, sat_divisions: 2, val_divisions: 1, data: vec![[10.0, 1.5, 1.0]; 8], srgb_value: false };
@@ -778,9 +1016,70 @@ mod tests {
}
#[test]
fn sony_nikon_and_panasonic_raws_get_a_file_local_look() {
assert!([RawFormat::Arw, RawFormat::Nef, RawFormat::Nrw, RawFormat::Rw2].into_iter().all(file_local_look));
assert!(![RawFormat::Dng, RawFormat::Cr2, RawFormat::Raf].into_iter().any(file_local_look));
fn supported_raws_get_a_file_local_look() {
assert!([RawFormat::Arw, RawFormat::Nef, RawFormat::Nrw, RawFormat::Rw2, RawFormat::Raf, RawFormat::Cr3].into_iter().all(file_local_look));
assert!(![RawFormat::Dng, RawFormat::Cr2].into_iter().any(file_local_look));
}
#[test]
fn cr3_camera_framing_is_measured_independently_of_tone_and_rejects_unrelated_edges() {
let (w, h) = (192, 128);
let mut sensor = Rgb32f::new(w, h);
for (i, pixel) in sensor.data.iter_mut().enumerate() {
let (x, y) = ((i % w) as f32, (i / w) as f32);
let value = 0.35 + 0.13 * (x * 0.15 + y * 0.22).sin() + 0.1 * (x * 0.33 - y * 0.14).sin() + 0.07 * (x * 0.09 - y * 0.31).cos();
*pixel = [value; 3];
}
let expected = Cr3Framing { scale: 0.96, offset: [0.01, -0.006] };
let mut reference = Rgb32f::new(w, h);
for (i, pixel) in reference.data.iter_mut().enumerate() {
let (x, y) = expected.source((i % w) as f32 + 0.5, (i / w) as f32 + 0.5, w, h);
*pixel = sensor.sample_bilinear(x, y).map(|v| v.powf(0.7));
}
let measured = estimate_cr3_framing(&sensor, &reference).expect("coherent held-out edges locate camera framing despite nonlinear tone");
assert!((measured.scale - expected.scale).abs() < 0.006, "{measured:?}");
assert!((measured.offset[0] - expected.offset[0]).abs() < 0.003, "{measured:?}");
assert!((measured.offset[1] - expected.offset[1]).abs() < 0.003, "{measured:?}");
assert!(estimate_cr3_framing(&sensor, &sensor).is_none(), "already aligned pixels stay unchanged");
reference.data.fill([0.2; 3]);
assert!(estimate_cr3_framing(&sensor, &reference).is_none(), "flat previews cannot establish framing");
for (i, pixel) in reference.data.iter_mut().enumerate() {
let (x, y) = ((i % w) as f32, (i / w) as f32);
*pixel = [0.35 + 0.13 * (x * 0.41 - y * 0.07).sin() + 0.1 * (x * 0.04 + y * 0.39).cos(); 3];
}
assert!(estimate_cr3_framing(&sensor, &reference).is_none(), "unrelated edges cannot change the framing");
}
/// The guarded fit is assessed on real sensor data, never on the JPEG fallback. A corpus
/// download or an unfinished codec cannot make this test claim a successful RAW calibration.
#[test]
fn corpus_cr3_gets_a_framing_aligned_camera_look() {
let path = std::env::var_os("LIGHTCRAFT_CORPUS")
.map(std::path::PathBuf::from)
.unwrap_or_else(|| std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus"))
.join("raw/cr3-canon-r100-raw.cr3");
let Ok(bytes) = std::fs::read(&path) else {
eprintln!("skip: {} absent", path.display());
return;
};
let raw = match lightcraft_raw::decode(&bytes) {
Ok(raw) => raw,
Err(lightcraft_raw::RawError::Unsupported(why)) => {
eprintln!("skip CR3 sensor decoder: {why}");
return;
}
Err(error) => panic!("CR3 corpus failed to decode: {error}"),
};
assert_eq!(raw.format, RawFormat::Cr3);
assert_eq!(raw.info().developed_size(), (6000, 4000));
let transform = lightcraft_raw::color::camera_transform(&raw, lightcraft_raw::color::as_shot_white_xy(&raw));
let look = fit_preview(&raw, &bytes, &transform);
eprintln!("R100 guarded colour fit accepted: {}", look.is_some());
assert!(look.is_some(), "this R100 corpus photo has enough matching colour after camera-framing alignment");
let (_, info) = crate::files::load_bytes(&bytes, 400).unwrap();
assert!(info.raw && info.relative_wb);
assert_eq!((info.as_shot_temp, info.as_shot_tint), (6500.0, 0.0));
assert_eq!(info.camera_tone.is_some(), look.is_some());
}
/// A public D7500 NEF (skipped without the corpus): its look is fitted to its own JPEG and white
@@ -818,6 +1117,90 @@ mod tests {
assert!(info.relative_wb && info.as_shot_temp == 6500.0 && info.as_shot_tint == 0.0);
}
/// Issue #232 (skipped without the corpus): the public ILCE-7RM2 sample's camera JPEG is
/// distortion-corrected, and on its glass façade the fit on all pixels misses the gate
/// (held-out RMS 0.113): it used to open grey with the neutral fallback.
#[test]
fn corpus_arw_with_a_lens_corrected_preview_gets_a_camera_look() {
let path = std::env::var_os("LIGHTCRAFT_CORPUS")
.map(std::path::PathBuf::from)
.unwrap_or_else(|| std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus"))
.join("raw/arw-sony-a7rm2-12bit-uncompressed.arw");
let Ok(bytes) = std::fs::read(&path) else {
eprintln!("skip: {} absent", path.display());
return;
};
let (_, info) = crate::files::load_bytes(&bytes, 400).unwrap();
assert!(info.camera_tone.is_some(), "no camera look fitted");
}
/// Issue #232: an ILCE-7RM2 camera JPEG is lens-corrected ("Distortion Comp.: Auto"), so on a
/// façade of window frames the raw and its JPEG are a pixel or two apart. The colours agree
/// everywhere away from the edges, and the look must still be found there.
#[test]
fn fits_a_slightly_misaligned_camera_jpeg_away_from_edges() {
let known = Mat3([[1.8, -0.4, -0.1], [-0.2, 1.5, -0.1], [-0.05, -0.3, 1.7]]);
let (w, h, block, shift) = (96, 64, 8, 2);
// Patches of 8×8 pixels, alternately dark and bright (frames and panes), varied in colour.
let scene = |x: usize, y: usize| -> [f32; 3] {
let (bx, by) = (x / block, y / block);
let ev = if (bx + by) % 2 == 0 { 0.04 + (bx % 3) as f32 * 0.01 } else { 0.3 + (by % 4) as f32 * 0.08 };
[ev * (0.7 + ((bx * 7 + by * 3) % 11) as f32 * 0.05), ev, ev * (0.7 + ((bx * 5 + by) % 13) as f32 * 0.04)]
};
let camera = |p: [f32; 3]| {
let q = known.apply_f32(p);
let y = luminance_2020(q);
q.map(|v| v * (1.0 - (-2.5 * y).exp()) / y)
};
let mut sensor = Rgb32f::new(w, h);
let mut reference = sensor.clone();
for y in 0..h {
for x in 0..w {
sensor.data[y * w + x] = scene(x, y);
// The JPEG shows the scene `shift` pixels to the right of where the raw has it.
reference.data[y * w + x] = camera(scene(x.saturating_sub(shift), y));
}
}
let fit = fit_pairs(&sensor, &reference).expect("look fitted away from the misaligned edges");
// The look itself is right: on the aligned scene it reproduces the camera.
let tone = ToneMap::camera(&fit.tone, 0.0, 0.0, 0.0);
let (mut error, mut n) = (0.0, 0);
for y in 0..h {
for x in 0..w {
let p = displayed(fit.matrix.apply(scene(x, y).map(f64::from)), &tone);
let t = camera(scene(x, y));
error += (0..3).map(|c| (p[c] - f64::from(t[c])).powi(2)).sum::<f64>();
n += 3;
}
}
assert!((error / n as f64).sqrt() < 0.03, "{}", (error / n as f64).sqrt());
// Unrelated colours stay rejected away from edges too (covered below); a flat JPEG as well.
reference.data.fill([0.2; 3]);
assert!(fit_pairs(&sensor, &reference).is_none());
}
/// X-Trans and 16-bit Bayer RAFs use the same fixed fit at thumbnail and export sizes.
/// The X-T20 fails the gates on all pixels (held-out RMS 0.114 against 0.10) and passes away
/// from edges (0.056, issue #232), so it is fitted too. Skips cleanly without these CC0
/// corpus files.
#[test]
fn corpus_raf_colour_and_white_balance() {
let dir = std::env::var_os("LIGHTCRAFT_CORPUS")
.map(std::path::PathBuf::from)
.unwrap_or_else(|| std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus"))
.join("raw");
for (name, accepted) in [("raf-fuji-xt2-865.raf", true), ("raf-fuji-gfx100s-4503.raf", true), ("raf-fuji-xt20-compressed.raf", true)] {
let Ok(bytes) = std::fs::read(dir.join(name)) else { continue };
let header = crate::files::probe_bytes(name, &bytes).unwrap();
assert_eq!(header.as_shot_wb, Some((6500.0, 0.0)), "{name}");
let (_, small) = crate::files::load_bytes(&bytes, 400).unwrap();
let (_, large) = crate::files::load_bytes(&bytes, 1200).unwrap();
assert_eq!(small.camera_tone.is_some(), accepted, "{name}");
assert_eq!(small.camera_tone, large.camera_tone, "{name}: resolution changed the fit");
assert!(small.raw && small.relative_wb && small.as_shot_temp == 6500.0 && small.as_shot_tint == 0.0, "{name}");
}
}
#[test]
fn rejects_monochrome_invalid_and_unrelated_previews() {
let mut sensor = Rgb32f::new(32, 32);
+11 -5
View File
@@ -1,6 +1,6 @@
//! Camera colour profiles of our own, pooled from many photos of one camera model.
//!
//! A raw file without colour matrices (Sony ARW, Nikon NEF) gets its look fitted to its own embedded camera
//! A raw file without colour matrices (Sony ARW, Nikon NEF, Fujifilm RAF) gets its look fitted to its own embedded camera
//! JPEG (`camera_preview`), but one photo shows too little of some colours: a lime shirt covering a
//! few dozen proxy pixels next to a hillside of foliage at the same hue. `lightcraft-cli calibrate`
//! pools the colour pairs of many photos per model and fits one matrix and hue/saturation/value
@@ -24,7 +24,11 @@ const MAX_FILE: u64 = 4 << 20;
/// Profiles built into LightCraft (`assets/camera-profiles/`, see `assets/ATTRIBUTION.md`):
/// `(model, JSON)`.
pub const BUNDLED: &[(&str, &str)] = &[("ILCE-7M4", include_str!("../../../assets/camera-profiles/ILCE-7M4.json"))];
pub const BUNDLED: &[(&str, &str)] = &[
("ILCE-7M4", include_str!("../../../assets/camera-profiles/ILCE-7M4.json")),
("X-H2S", include_str!("../../../assets/camera-profiles/X-H2S.json")),
("X-T4", include_str!("../../../assets/camera-profiles/X-T4.json")),
];
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct CameraProfile {
@@ -196,10 +200,12 @@ pub struct Pool {
const PAIRS_PER_FILE: usize = 4000;
impl Pool {
/// Add one raw file's colour pairs. `Ok(None)` when the file can't contribute (not an ARW or
/// NEF without colour matrices, no usable camera JPEG, too little colour).
/// Add one raw file's colour pairs. `Ok(None)` when the file can't contribute (not an ARW,
/// NEF or RAF without colour matrices, no usable camera JPEG, too little colour).
pub fn add(&mut self, bytes: &[u8]) -> Result<Option<String>, String> {
let raw = lightcraft_raw::decode(bytes).map_err(|e| e.to_string())?;
let mut raw = lightcraft_raw::decode(bytes).map_err(|e| e.to_string())?;
// colour only: geometric lens corrections would stop the sensor proxy from binning
raw.opcodes.list3.retain(|op| !op.is_lens_correction());
let Some(model) = raw.metadata.model.as_deref().map(str::trim).filter(|m| !m.is_empty()) else { return Ok(None) };
let Some(pairs) = crate::camera_preview::profile_pairs(&raw, bytes) else { return Ok(None) };
let step = pairs.len().div_ceil(PAIRS_PER_FILE).max(1);
+14
View File
@@ -251,6 +251,9 @@ pub(crate) fn rename_folder_on_disk(from: &str, to: &str) -> std::result::Result
/// After a folder moved from `from` to `to`: a browsed folder at or below it follows.
pub(crate) fn follow_folder(s: &mut Session, from: &str, to: &str) {
// a library folder chosen at or below it follows too, as a source or as a filter
follow_library_folder(&mut s.library_folder, from, to);
follow_library_folder(&mut s.filter.library_folder, from, to);
if let Some(b) = &mut s.browse
&& let Ok(rel) = Path::new(&b.path).strip_prefix(from)
{
@@ -260,6 +263,17 @@ pub(crate) fn follow_folder(s: &mut Session, from: &str, to: &str) {
}
}
/// A chosen library folder at or below `from` becomes the same place below `to`. Compared the
/// way the filter does, by folder identity, so a differently spelled choice is not widened to
/// the folder itself.
fn follow_library_folder(chosen: &mut Option<String>, from: &str, to: &str) {
let Some(c) = chosen.as_deref() else { return };
if let Some(rest) = lightcraft_catalog::query::folder_rest(c, from) {
// joined by the platform, so the result is spelled like `to` and like a path read from disk
*chosen = Some(rest.iter().fold(std::path::PathBuf::from(to), |p, name| p.join(name)).to_string_lossy().to_string());
}
}
/// Rename or move a folder on disk (everything in it goes along, sidecars included) and relink
/// the photos inside, as one undo step: undo renames the folder back and restores the photos'
/// paths, redo repeats the move — each refused (and reported) when the destination is taken.
+72 -5
View File
@@ -2,7 +2,7 @@
use lightcraft_catalog::{Op, PhotoId, Version};
use lightcraft_develop::{DevelopSettings, Preset, Section, SettingsGroup, Treatment, Upright, WbMode, controls};
use lightcraft_geom::{CropGeometry, Point, Rect, crop_fit_angle};
use lightcraft_geom::{CropGeometry, CropHandle, Point, Rect, crop_fit_angle, drag_crop};
use serde_json::{Value, json};
use super::{CommandSpec, always, bad, bool_or, cmd, f64_or, f64_req, has_active, has_clipboard, has_selection, ok, str_param};
@@ -37,6 +37,32 @@ fn crop_dims(s: &Session, id: PhotoId) -> (f64, f64) {
if o.swaps_axes() { (h, w) } else { (w, h) }
}
/// `v` as a list of numbers; `None` unless it is an array of nothing but numbers.
fn numbers(v: Option<&Value>) -> Option<Vec<f64>> {
v?.as_array()?.iter().map(Value::as_f64).collect()
}
/// A width × height lock stored as hundredths. Rejects non-finite, non-positive and
/// extreme (beyond 20:1) shapes, which would give an empty or unusable crop.
fn aspect_tuple(c: &str, x: f64, y: f64) -> Result<(u32, u32)> {
let ratio = x / y;
if !(x.is_finite()
&& y.is_finite()
&& x > 0.0
&& y > 0.0
&& ratio.is_finite()
&& (1.0 / lightcraft_geom::MAX_RATIO..=lightcraft_geom::MAX_RATIO).contains(&ratio))
{
return Err(bad(c, "aspect needs two positive numbers with a ratio between 1:20 and 20:1"));
}
let to = |v: f64| (v * 100.0).round();
let (a, b) = (to(x), to(y));
if a < 1.0 || b < 1.0 || a > f64::from(u32::MAX) || b > f64::from(u32::MAX) {
return Err(bad(c, "aspect numbers are out of range"));
}
Ok((a as u32, b as u32))
}
fn set_aspect(d: &mut DevelopSettings, w: f64, h: f64, aspect: Option<(u32, u32)>) {
d.crop.aspect = aspect;
let a = aspect.map(|(aw, ah)| {
@@ -395,6 +421,46 @@ pub fn specs() -> Vec<CommandSpec> {
Ok(())
})
}),
cmd!(
"crop.drag",
"Drag Crop Handle",
[],
None,
"{handle: topLeft|topRight|bottomRight|bottomLeft|top|right|bottom|left|move | 0..8, from: [x,y], to: [x,y] (normalized, straightened frame), start?: [x0,y0,x1,y1] (default: the current crop)} — drags a crop handle; stops at the image edge, keeps the opposite edges, honours the aspect lock",
has_active,
|s, p| {
let id = active(s, "crop.drag")?;
let (w, h) = crop_dims(s, id);
let handle = match p.get("handle") {
Some(Value::String(n)) => CropHandle::from_name(n),
Some(v) => v.as_u64().and_then(|i| u8::try_from(i).ok()).and_then(CropHandle::from_index),
None => None,
}
.ok_or_else(|| bad("crop.drag", "unknown handle"))?;
let point = |k: &str| -> Result<Point> {
match numbers(p.get(k)).as_deref() {
Some([x, y]) if x.is_finite() && y.is_finite() => Ok(Point::new(*x, *y)),
_ => Err(bad("crop.drag", format!("`{k}` needs [x, y]"))),
}
};
let (from, to) = (point("from")?, point("to")?);
let given = match (p.get("start"), numbers(p.get("start")).as_deref()) {
(None | Some(Value::Null), _) => None,
(_, Some(&[x0, y0, x1, y1])) if [x0, y0, x1, y1].iter().all(|v| v.is_finite()) => {
Some(Rect::new(x0.min(x1), y0.min(y1), x0.max(x1), y0.max(y1)))
}
_ => return Err(bad("crop.drag", "`start` needs 4 numbers")),
};
edit(s, "crop.drag", "Crop", |d| {
let start = CropGeometry { rect: given.unwrap_or(d.crop.geometry.rect), angle: d.crop.geometry.angle };
// A locked crop keeps the shape it has: the lock stores a width:height tuple whose
// orientation is reinterpreted per image, so the frame itself is the truth.
let ratio = d.crop.aspect.and(Some(start.rect_px(w, h).aspect())).filter(|a| a.is_finite() && *a > 0.0);
d.crop.geometry = drag_crop(start, handle, from, to, w, h, ratio);
Ok(())
})
}
),
cmd!(
"crop.straighten",
"Straighten",
@@ -470,11 +536,12 @@ pub fn specs() -> Vec<CommandSpec> {
let (x, y) = a.split_once('x').ok_or_else(|| bad("crop.aspect", "use WxH"))?;
let (x, y): (f64, f64) =
(x.parse().map_err(|_| bad("crop.aspect", "bad number"))?, y.parse().map_err(|_| bad("crop.aspect", "bad number"))?);
Some(((x * 100.0) as u32, (y * 100.0) as u32))
}
Some(Value::Array(a)) if a.len() == 2 => {
Some(((a[0].as_f64().unwrap_or(1.0) * 100.0) as u32, (a[1].as_f64().unwrap_or(1.0) * 100.0) as u32))
Some(aspect_tuple("crop.aspect", x, y)?)
}
Some(v @ Value::Array(_)) => match numbers(Some(v)).as_deref() {
Some(&[x, y]) => Some(aspect_tuple("crop.aspect", x, y)?),
_ => return Err(bad("crop.aspect", "use [w, h] with two numbers")),
},
_ => return Err(bad("crop.aspect", "missing aspect")),
};
edit(s, "crop.aspect", "Crop Aspect", |d| {
+4 -1
View File
@@ -23,7 +23,8 @@ impl Session {
.into_iter()
.find(|(x, _)| x.name.eq_ignore_ascii_case(name.trim()))
.ok_or_else(|| format!("unknown export preset `{name}` (see export.presets)"))?;
let mut out = preset.params;
// a preset saved by another version may hold keys this one doesn't know: not the caller's typo
let mut out = crate::export::ExportOptions::known_keys_only(&preset.params);
if let (Some(o), Some(own)) = (out.as_object_mut(), p.as_object()) {
for (k, v) in own.iter().filter(|(k, _)| *k != "preset") {
o.insert(k.clone(), v.clone());
@@ -54,6 +55,8 @@ fn save(s: &mut Session, p: &Value) -> Result<Value> {
if let Some(o) = params.as_object_mut() {
o.retain(|k, _| !NOT_SETTINGS.contains(&k.as_str()));
}
// a preset that app.export would refuse is refused here, where the typo is on screen
crate::export::ExportOptions::validate(ID, &params)?;
let preset = ExportPreset { name: name.to_string(), params };
match s.export_presets.iter_mut().find(|x| x.name.eq_ignore_ascii_case(name)) {
Some(x) => *x = preset,
+199 -36
View File
@@ -1,7 +1,7 @@
//! Library commands: view source, filter/sort, selection, ratings/flags/labels, rotate, delete,
//! metadata, albums, import.
use lightcraft_catalog::{Album, AlbumId, ColorLabel, CopyrightStatus, Flag, GroupBy, Op, PhotoId, Sort, SortKey};
use lightcraft_catalog::{Album, AlbumId, ColorLabel, CopyrightStatus, Filter, Flag, GroupBy, Op, PhotoId, Sort, SortKey};
use serde_json::{Value, json};
use super::{CommandSpec, always, bad, bool_or, cmd, has_active, has_selection, ok, str_param};
@@ -66,7 +66,14 @@ pub fn import_params(s: &Session, p: &Value) -> Result<ImportRequest> {
{
return Err(bad("library.import", format!("unknown metadata preset `{n}`")));
}
let on_deleted = match str_param(p, "onDeleted") {
Some(v) => {
crate::import::OnDeleted::parse(v).ok_or_else(|| bad("library.import", format!("unknown onDeleted `{v}` (skip|restore|fresh)")))?
}
None => Default::default(),
};
let opts = crate::import::ImportOptions {
on_deleted,
mode,
preset,
keywords: strs(p, "keywords"),
@@ -87,7 +94,9 @@ pub fn import_params(s: &Session, p: &Value) -> Result<ImportRequest> {
/// `album`).
pub fn import_batch_done(s: &mut Session, report: crate::import::ImportReport, mut album: Option<u64>, album_name: Option<&str>) -> Result<Value> {
let mut report = serde_json::to_value(report).unwrap_or_default();
let imported: Vec<u64> = report["imported"].as_array().map(|a| a.iter().filter_map(Value::as_u64).collect()).unwrap_or_default();
// photos an import restored from Recently Deleted take part like new ones (album, selection)
let imported: Vec<u64> =
["imported", "restored"].iter().flat_map(|k| report[*k].as_array().into_iter().flatten().filter_map(Value::as_u64)).collect();
if album.is_none()
&& let Some(name) = album_name
&& !imported.is_empty()
@@ -251,7 +260,7 @@ pub fn specs() -> Vec<CommandSpec> {
"Show Source",
[],
None,
"{kind: all|recentlyAdded|album|recentlyDeleted|picks|missing, id?: albumId}",
"{kind: all|recentlyAdded|album|recentlyDeleted|picks|missing|libraryFolder, id?: albumId, path?: a path from library.folders (for libraryFolder)}",
always,
|s, p| {
let kind = str_param(p, "kind").unwrap_or("all");
@@ -268,6 +277,25 @@ pub fn specs() -> Vec<CommandSpec> {
}
LibrarySource::Album(a)
}
"libraryFolder" => {
let path = str_param(p, "path").filter(|d| !d.trim().is_empty() && !lightcraft_catalog::query::folder_key(d).is_empty());
let path = path.ok_or_else(|| bad("library.source", "libraryFolder needs the `path` of a folder from library.folders"))?;
if lightcraft_catalog::folders::is_startup_disk(path) {
return Err(bad("library.source", "the startup disk's path covers every disk: choose a folder in it"));
}
let f = Filter { library_folder: Some(path.to_string()), ..Default::default() };
let ids = s.catalog.query(&f, &Sort::default());
if ids.is_empty() {
return Err(bad("library.source", format!("{path}: no photo in the library was imported from it")));
}
if covers_other_disks(s, path, &ids) {
return Err(bad("library.source", format!("{path} holds other disks as well: choose a disk or a folder on one")));
}
s.library_folder = Some(path.to_string());
// one folder at a time: a folder filter left over would be ignored
s.filter.library_folder = None;
LibrarySource::LibraryFolder
}
other => return Err(bad("library.source", format!("unknown source `{other}`"))),
};
let vis = s.visible_cloned();
@@ -282,15 +310,75 @@ pub fn specs() -> Vec<CommandSpec> {
"Filter",
[],
None,
"partial Filter: {text?, rating?, ratingOp?: atLeast|exactly|atMost, flag?: pick|reject|none|null, label?, kind?, merged?: hdr|panorama|hdrPanorama|any, edited?, date?, keyword?, person?, camera?}",
"partial Filter: {text?, rating?, ratingOp?: atLeast|exactly|atMost, flag?: pick|reject|none|null, label?, kind?, merged?: hdr|panorama|hdrPanorama|any, edited?, date?, libraryFolder?: a path from library.folders, keyword?, person?, camera?}",
always,
|s, p| {
let mut v = serde_json::to_value(&s.filter).unwrap_or_default();
lightcraft_develop::presets::deep_merge(&mut v, p);
s.filter = serde_json::from_value(v).map_err(|e| bad("library.filter", e.to_string()))?;
let f: Filter = serde_json::from_value(v).map_err(|e| bad("library.filter", e.to_string()))?;
let mut f = f;
// a blank folder is no folder: no hidden "filters active" state
f.library_folder = f.library_folder.filter(|d| !d.trim().is_empty());
if f.library_folder.is_some() && s.source == LibrarySource::LibraryFolder {
return Err(bad("library.filter", "a folder is already shown (library.source): show another source first"));
}
s.filter = f;
Ok(json!({"count": s.visible().len()}))
}
),
cmd!(
query "library.folders",
"Library Folders",
[],
None,
"{} → [{name, path, count, own, volume, selectable, children}] the disks and the folders the library's photos were imported from, with photo counts (subfolders included in `count`); show one with library.source {kind: libraryFolder, path} (rows with selectable false only open)",
always,
|s, _| Ok(serde_json::to_value(s.catalog.folder_tree()).unwrap_or_default())
),
cmd!(
"library.removeFolder",
"Remove Folder from Library",
[],
None,
"{path, disk?: bool} — move the library photos imported from this folder (and the folders inside it) to Recently Deleted; one undo step, no file is touched. A whole disk or share (`/Volumes/nas`, `C:\\`, `\\\\srv\\share`) goes only with `disk: true`, the startup disk never → {removed}",
always,
|s, p| {
const C: &str = "library.removeFolder";
let path = str_param(p, "path").filter(|d| !d.trim().is_empty()).ok_or_else(|| bad(C, "missing `path`"))?;
if lightcraft_catalog::query::folder_key(path).is_empty() {
return Err(bad(C, format!("{path}: not a folder")));
}
if lightcraft_catalog::folders::is_startup_disk(path) {
return Err(bad(C, "choose a folder, not the whole startup disk"));
}
if lightcraft_catalog::folders::is_disk_root(path) && !bool_or(p, "disk", false) {
return Err(bad(C, format!("{path} is a whole disk or share: pass `disk: true` to remove everything on it")));
}
let f = Filter { library_folder: Some(path.to_string()), ..Default::default() };
let ids = s.catalog.query(&f, &Sort::default());
if ids.is_empty() {
return Err(bad(C, format!("{path}: no photo in the library was imported from it")));
}
if covers_other_disks(s, path, &ids) && !bool_or(p, "disk", false) {
return Err(bad(C, format!("{path} holds whole disks: pass `disk: true` to remove everything on them")));
}
let ops = ids.iter().map(|id| Op::SetDeleted { id: *id, deleted: true }).collect();
s.commit("Remove Folder from Library", Op::Batch { ops })?;
if s.filter.library_folder.as_deref().is_some_and(|c| lightcraft_catalog::query::folder_within(c, path)) {
s.filter.library_folder = None;
}
// the folder being shown is gone: back to everything, as for a deleted album
if s.library_folder.as_deref().is_some_and(|c| lightcraft_catalog::query::folder_within(c, path)) {
s.library_folder = None;
if s.source == LibrarySource::LibraryFolder {
s.source = LibrarySource::All;
}
}
let vis = s.visible_cloned();
s.selection = vis.first().map(|f| Selection::single(*f)).unwrap_or_default();
Ok(json!({"removed": ids.len()}))
}
),
cmd!("library.clearFilter", "Clear Filters", ["View"], None, "{}", always, |s, _| {
s.filter = Default::default();
Ok(json!({"count": s.visible().len()}))
@@ -356,36 +444,50 @@ pub fn specs() -> Vec<CommandSpec> {
}
),
// ---- selection
cmd!("library.select", "Select Photos", [], None, "{ids: [id], active?: id, mode?: replace|add|toggle|range}", always, |s, p| {
let ids = ids_param(p).unwrap_or_default();
let mode = str_param(p, "mode").unwrap_or("replace");
s.end_interaction()?;
match mode {
"replace" => {
s.selection =
Selection { ids: ids.clone(), active: p.get("active").and_then(Value::as_u64).map(PhotoId).or(ids.first().copied()) };
cmd!(
"library.select",
"Select Photos",
[],
None,
"{ids: [id], active?: id, mode?: replace|add|toggle|range} — every id must be a photo of the library",
always,
|s, p| {
let ids = ids_param(p).unwrap_or_default();
let active = p.get("active").and_then(Value::as_u64).map(PhotoId);
let mode = str_param(p, "mode").unwrap_or("replace");
// An id that is not in the library is an error, and the selection stays as it was:
// selecting it would otherwise report success and leave the session with no usable
// active photo (#182).
if let Some(id) = ids.iter().chain(active.as_ref()).find(|id| s.catalog.photo(**id).is_none()) {
return Err(bad("library.select", format!("no such photo {}", id.0)));
}
"add" => {
for id in ids {
if !s.selection.contains(id) {
s.selection.ids.push(id);
s.end_interaction()?;
match mode {
"replace" => {
s.selection = Selection { ids: ids.clone(), active: active.or(ids.first().copied()) };
}
"add" => {
for id in ids {
if !s.selection.contains(id) {
s.selection.ids.push(id);
}
s.selection.active = Some(id);
}
s.selection.active = Some(id);
}
}
"toggle" => ids.into_iter().for_each(|id| s.selection.toggle(id)),
"range" => {
let vis = s.visible_cloned();
if let Some(id) = ids.first() {
s.selection.extend_to(*id, &vis);
"toggle" => ids.into_iter().for_each(|id| s.selection.toggle(id)),
"range" => {
let vis = s.visible_cloned();
if let Some(id) = ids.first() {
s.selection.extend_to(*id, &vis);
}
}
other => return Err(bad("library.select", format!("unknown mode `{other}`"))),
}
other => return Err(bad("library.select", format!("unknown mode `{other}`"))),
s.active_mask = None;
s.active_spot = None;
Ok(json!({"selected": s.selection.ids.len()}))
}
s.active_mask = None;
s.active_spot = None;
Ok(json!({"selected": s.selection.ids.len()}))
}),
),
cmd!("library.selectAll", "Select All", ["Edit"], Some("Cmd+A"), "{}", always, |s, _| {
let vis = s.visible_cloned();
s.selection = Selection { active: s.selection.active.filter(|a| vis.contains(a)).or(vis.first().copied()), ids: vis };
@@ -560,10 +662,28 @@ pub fn specs() -> Vec<CommandSpec> {
id,
deleted: false
}))),
cmd!(
"library.emptyRecentlyDeleted",
"Empty Recently Deleted",
["Photo"],
None,
"{} → {deleted} — removes every photo in Recently Deleted from the library (files on disk stay)",
always,
|s, _| {
let ids: Vec<PhotoId> = s.catalog.photos().filter(|p| p.deleted).map(|p| p.id).collect();
if ids.is_empty() {
return Ok(json!({"deleted": 0}));
}
let op = s.catalog.delete_photos_permanently_ops(&ids);
s.commit("Empty Recently Deleted", op)?;
s.selection = Selection::default();
Ok(json!({"deleted": ids.len()}))
}
),
cmd!("photo.deletePermanently", "Delete Permanently", ["Photo"], None, "{ids?}", has_selection, |s, p| {
let t = s.targets(p);
let ops = t.iter().map(|id| s.catalog.delete_permanently_ops(*id)).collect::<Vec<_>>();
s.commit("Delete Permanently", Op::Batch { ops })?;
let op = s.catalog.delete_photos_permanently_ops(&t);
s.commit("Delete Permanently", op)?;
s.selection = Selection::default();
Ok(json!({"deleted": t.len()}))
}),
@@ -725,8 +845,16 @@ pub fn specs() -> Vec<CommandSpec> {
let rules = if bool_or(p, "fromView", false) {
view_rules(s)
} else {
let base = if bool_or(p, "replace", false) { Default::default() } else { cur };
merge_rules(&base, p.get("rules").unwrap_or(&Value::Null), "album.setRules")?
let replace = bool_or(p, "replace", false);
let folder = cur.library_folder.clone();
let base = if replace { Default::default() } else { cur };
let mut r = merge_rules(&base, p.get("rules").unwrap_or(&Value::Null), "album.setRules")?;
// the rules dialog has no folder field: replacing its rules keeps the folder
// unless the call says (`libraryFolder: null`) to drop it
if replace && p.get("rules").and_then(|r| r.get("libraryFolder")).is_none() {
r.library_folder = folder;
}
r
};
s.commit("Edit Smart Album", Op::SetAlbumRules { id, rules: Box::new(rules) })?;
Ok(json!({"count": s.catalog.album_count(id)}))
@@ -934,6 +1062,25 @@ pub fn specs() -> Vec<CommandSpec> {
ok()
}),
// ---- import
cmd!(query "library.inspectLightroom", "Inspect Lightroom Catalog", [], None,
"{path: .lrcat} → {photos, collections, missing, warnings}; read-only, no Lightroom required",
always, |_, p| {
let path = str_param(p,"path").ok_or_else(|| bad("library.inspectLightroom","missing path"))?;
let data = crate::lightroom_catalog::read(std::path::Path::new(path)).map_err(|e| bad("library.inspectLightroom",e))?;
Ok(json!({"photos":data.photos.len(),"collections":data.collections.iter().filter(|r|r.get("systemOnly").and_then(Value::as_f64).unwrap_or(0.0)==0.0).count(),"missing":data.photos.iter().filter(|p| !std::path::Path::new(&p.path).is_file()).map(|p|&p.path).collect::<Vec<_>>(),"warnings":data.warnings}))
}),
cmd!(
"library.importLightroom",
"Import Lightroom Catalog",
[],
None,
"{path: .lrcat, updateExisting?: false}; direct import of paths, ratings, flags, metadata, keywords, collections, virtual copies & mapped edits. Archives source settings/history first. Existing edited photos are preserved unless updateExisting=true; source catalog is read-only.",
always,
|s, p| {
let path = str_param(p, "path").ok_or_else(|| bad("library.importLightroom", "missing path"))?;
crate::lightroom_catalog::import(s, std::path::Path::new(path), bool_or(p, "updateExisting", false))
}
),
cmd!(
query "library.importPreview",
"Review Import",
@@ -956,7 +1103,7 @@ pub fn specs() -> Vec<CommandSpec> {
"Import Photos",
["File"],
Some("Cmd+Shift+I"),
"{paths: [file or folder (recursive)], mode?: add|copy|move (add = reference the files in place; copy = into the library's Originals/YYYY/YYYY-MM-DD/; move = as copy, then each original and its XMP sidecars are removed from the source — only after the copy is verified (a hard link on the same volume, else copied, synced and compared byte for byte) and its catalog record is saved; failed, duplicate and unchecked files keep their sources; a taken name gets -1, -2…; undo removes the photos from the library but leaves the files at the destination), destination?: folder for copies / moves, organize?: date (YYYY/YYYY-MM-DD) | month (YYYY/YYYY-MM) | flat | a folder template, e.g. `{date:%Y}/{date:%Y%m%d}` → 2026/20260114 (the template's `/` make the folders, each level expanded with the rename tokens and made a safe folder name: never outside the destination; must be relative, no `..`; a level with missing metadata is `unknown`) — dated by capture time, else the import time, rename?: file-name template for copies, original extension added (tokens: {name} {num} {seq} {seq:N} {date} {date:%Y%m%d} {folder} {camera} {lens} {iso} {rating} {title} {creator} {ext}; photo.renameTokens explains each; blank = keep names), renameStart?: 1, metadataPreset?: name, dng?: bool (copy raws as DNG; copy only), local?: bool (browsing: the photos stay out of the library, like library.browse; not with move), album?: albumId, albumName?: new album, preset?: presetId, keywords?: [..]} → {imported, duplicates, failed, moved?: [{from, to, sidecars?}], kept?: [{path, reason}] (move: sources left in place and why), album?}",
"{paths: [file or folder (recursive)], mode?: add|copy|move (add = reference the files in place; copy = into the library's Originals/YYYY/YYYY-MM-DD/; move = as copy, then each original and its XMP sidecars are removed from the source — only after the copy is verified (a hard link on the same volume, else copied, synced and compared byte for byte) and its catalog record is saved; failed, duplicate and unchecked files keep their sources; a taken name gets -1, -2…; undo removes the photos from the library but leaves the files at the destination), destination?: folder for copies / moves, organize?: date (YYYY/YYYY-MM-DD) | month (YYYY/YYYY-MM) | flat | a folder template, e.g. `{date:%Y}/{date:%Y%m%d}` → 2026/20260114 (the template's `/` make the folders, each level expanded with the rename tokens and made a safe folder name: never outside the destination; must be relative, no `..`; a level with missing metadata is `unknown`) — dated by capture time, else the import time, rename?: file-name template for copies, original extension added (tokens: {name} {num} {seq} {seq:N} {date} {date:%Y%m%d} {folder} {camera} {lens} {iso} {rating} {title} {creator} {ext}; photo.renameTokens explains each; blank = keep names), renameStart?: 1, metadataPreset?: name, dng?: bool (copy raws as DNG; copy only), local?: bool (browsing: the photos stay out of the library, like library.browse; not with move), album?: albumId, albumName?: new album, preset?: presetId, keywords?: [..], onDeleted?: skip|restore|fresh (a file that is in Recently Deleted: skip = leave it there and say so, restore = bring the photo back with its edits, fresh = delete the trashed record and import the file as new)} → {imported, duplicates: [{path, existing, reason, existingDeleted?}], restored?: [photoId], failed, moved?: [{from, to, sidecars?}], kept?: [{path, reason}] (move: sources left in place and why), album?}",
always,
|s, p| {
let req = import_params(s, p)?;
@@ -967,9 +1114,12 @@ pub fn specs() -> Vec<CommandSpec> {
let report = crate::import::import_with(s, &req.paths, &req.opts)?;
let report = import_batch_done(s, report, req.album, req.album_name.as_deref())?;
// one undo step for the whole import
let imported: Vec<u64> = report["imported"].as_array().map(|a| a.iter().filter_map(Value::as_u64).collect()).unwrap_or_default();
let ids_of = |k: &str| -> Vec<u64> { report[k].as_array().map(|a| a.iter().filter_map(Value::as_u64).collect()).unwrap_or_default() };
let (imported, restored) = (ids_of("imported"), ids_of("restored"));
let n = s.undo.len().saturating_sub(undo0);
s.merge_undo(n, &format!("Add {} Photo{}", imported.len(), if imported.len() == 1 { "" } else { "s" }));
let (verb, count) = if imported.is_empty() && !restored.is_empty() { ("Restore", restored.len()) } else { ("Add", imported.len()) };
s.merge_undo(n, &format!("{verb} {count} Photo{}", if count == 1 { "" } else { "s" }));
let imported = if imported.is_empty() { restored } else { imported };
if let Some(f) = imported.first() {
s.selection = Selection::single(PhotoId(*f));
}
@@ -1051,6 +1201,16 @@ fn merge_rules(base: &lightcraft_catalog::Filter, patch: &Value, c: &str) -> Res
Ok(f)
}
/// Whether `path` lies above a disk: some of its `ids` photos are on a disk (not the startup
/// disk) that the folder does not lie in. Such a folder (`/Volumes`, `/mnt`…) is not a folder of
/// one disk but a way to reach several.
fn covers_other_disks(s: &Session, path: &str, ids: &[PhotoId]) -> bool {
ids.iter()
.filter_map(|id| s.catalog.photo(*id))
.filter_map(|p| lightcraft_catalog::folders::volume_of(p))
.any(|v| v != "/" && !lightcraft_catalog::query::folder_within(path, &v))
}
/// The current view (source + filter) as smart-album rules. Viewing a smart album starts from
/// its rules with the filter bar's settings on top.
fn view_rules(s: &Session) -> lightcraft_catalog::Filter {
@@ -1077,6 +1237,9 @@ fn view_rules(s: &Session) -> lightcraft_catalog::Filter {
None => {
let mut f = s.source.to_filter(&s.filter, &s.catalog);
f.deleted = false;
if s.source == LibrarySource::LibraryFolder {
f.library_folder = s.library_folder.clone();
}
f
}
}
+5 -16
View File
@@ -41,22 +41,11 @@ fn cube_files(paths: &[String]) -> Vec<(String, String, Option<String>)> {
for p in paths {
let path = Path::new(p);
if path.is_dir() {
let mut stack = vec![path.to_path_buf()];
while let Some(d) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&d) else { continue };
for e in rd.flatten() {
let ep = e.path();
if ep.file_name().is_some_and(|n| n.to_string_lossy().starts_with('.')) {
continue;
}
if ep.is_dir() {
stack.push(ep);
} else if ep.extension().is_some_and(|x| x.eq_ignore_ascii_case("cube"))
&& let Ok(t) = std::fs::read_to_string(&ep)
{
let group = ep.parent().and_then(|d| d.file_name()).map(|n| n.to_string_lossy().to_string());
out.push((ep.file_stem().map(|n| n.to_string_lossy().to_string()).unwrap_or_default(), t, group));
}
let cube = |f: &Path| f.extension().is_some_and(|x| x.eq_ignore_ascii_case("cube"));
for ep in crate::walk::files_in(path, None, crate::walk::Limits::default(), cube).files {
if let Ok(t) = std::fs::read_to_string(&ep) {
let group = ep.parent().and_then(|d| d.file_name()).map(|n| n.to_string_lossy().to_string());
out.push((ep.file_stem().map(|n| n.to_string_lossy().to_string()).unwrap_or_default(), t, group));
}
}
} else if path.extension().is_some_and(|x| x.eq_ignore_ascii_case("zip")) {
+12 -3
View File
@@ -5,7 +5,7 @@ use lightcraft_develop::{CONTROLS, controls};
use serde_json::{Value, json};
use super::{CommandSpec, always, bad, cmd, has_active};
use crate::Session;
use crate::{LibrarySource, Session};
pub fn photo_summary(p: &Photo) -> Value {
json!({
@@ -48,8 +48,13 @@ fn album_json(a: &Album, all: &[Album], cat: &lightcraft_catalog::Catalog) -> Va
v
}
/// The photo a query is about: `id` when given (which must be in the library), else the active one.
fn photo_arg(s: &Session, p: &Value, c: &str) -> crate::Result<PhotoId> {
p.get("id").and_then(Value::as_u64).map(PhotoId).or(s.active()).ok_or_else(|| bad(c, "no photo"))
match p.get("id").and_then(Value::as_u64).map(PhotoId) {
Some(id) if s.catalog.photo(id).is_some() => Ok(id),
Some(id) => Err(bad(c, format!("no such photo {}", id.0))),
None => s.active().ok_or_else(|| bad(c, "no photo")),
}
}
pub fn specs() -> Vec<CommandSpec> {
@@ -81,10 +86,14 @@ pub fn specs() -> Vec<CommandSpec> {
}))
}),
cmd!(query "library.state", "Library State", [], None, "{}", always, |s, _| {
let label = s.source.label(&s.catalog);
let label = match (s.source, s.library_folder.as_deref()) {
(LibrarySource::LibraryFolder, Some(path)) => lightcraft_catalog::folders::folder_label(path),
_ => s.source.label(&s.catalog),
};
let n = s.visible().len();
Ok(json!({
"source": s.source,
"libraryFolder": s.library_folder.as_ref().filter(|_| s.source == LibrarySource::LibraryFolder),
"sourceLabel": label,
"filter": s.filter,
"sort": s.sort,
+1 -1
View File
@@ -191,7 +191,7 @@ pub fn devices_in(parents: &[std::path::PathBuf], extra: Vec<std::path::PathBuf>
/// The volume's `DCIM` folder: the usual spellings are checked directly, so a volume's root is
/// only listed (other spellings on a case-sensitive volume) off Windows, whose drives aren't.
fn dcim_in(root: &std::path::Path) -> Option<std::path::PathBuf> {
pub fn dcim_in(root: &std::path::Path) -> Option<std::path::PathBuf> {
if let Some(p) = ["DCIM", "dcim"].iter().map(|n| root.join(n)).find(|p| p.is_dir()) {
return Some(p);
}
+439
View File
@@ -258,6 +258,35 @@ fn watermark_fonts(craft: &'static [crate::fonts::CraftFont]) -> Vec<ab_glyph::F
.collect()
}
/// The font's vertical form, positioned from its vertical origin within the watermark cell.
/// Unlike Unicode presentation-form characters, GSUB works with BIZ UD fonts too.
fn vertical_watermark_glyph(
font: &ab_glyph::FontRef<'_>,
data: &harfrust::ShaperData,
ch: char,
px: f32,
left: f32,
top: f32,
) -> Option<ab_glyph::Glyph> {
use ab_glyph::{Font, ScaleFont};
let face = harfrust::FontRef::new(font.font_data()).ok()?;
let mut buf = harfrust::UnicodeBuffer::new();
buf.add(ch, 0);
buf.set_direction(harfrust::Direction::TopToBottom);
buf.guess_segment_properties();
let shaped = data.shaper(&face).build().shape(buf, harfrust::ShapeOptions::new());
if shaped.glyph_infos().len() != 1 {
return None;
}
let gid = u16::try_from(shaped.glyph_infos().first()?.glyph_id).ok().filter(|g| *g != 0)?;
let pos = shaped.glyph_positions().first()?;
let sf = font.as_scaled(px);
let (sx, sy) = (sf.h_scale_factor(), sf.v_scale_factor());
let height = -(pos.y_advance as f32) * sy;
let pen = ab_glyph::point(left + px / 2.0 + pos.x_offset as f32 * sx, top + (px - height) / 2.0 - pos.y_offset as f32 * sy);
Some(ab_glyph::GlyphId(gid).with_scale_and_position(px, pen))
}
/// Draw `wm` onto `img` (straight alpha blending of the encoded values).
pub fn draw_watermark(img: &mut Rgba8, wm: &Watermark) {
let width = img.width;
@@ -312,6 +341,11 @@ fn watermark_coverage(
}
let fonts = watermark_fonts(craft);
let Some(latin) = fonts.first() else { return };
let vertical_data: Vec<_> = if wm.vertical {
fonts.iter().map(|font| harfrust::FontRef::new(font.font_data()).ok().map(|face| harfrust::ShaperData::new(&face))).collect()
} else {
Vec::new()
};
let short = width.min(height) as f32;
let px = (wm.size.clamp(0.005, 0.5) * short).max(6.0);
let mut glyphs = Vec::new();
@@ -335,6 +369,21 @@ fn watermark_coverage(
prev = None;
continue;
}
if wm.vertical
&& matches!(original as u32, 0x3000..=0x30FF | 0x3400..=0x9FFF | 0xFF01..=0xFF60)
&& let Some(face) = fonts.iter().position(|font| font.glyph_id(original).0 != 0)
&& let (Some(font), Some(Some(data))) = (fonts.get(face), vertical_data.get(face))
{
let left = columns.saturating_sub(column.saturating_add(1)) as f32 * px;
if let Some(glyph) = vertical_watermark_glyph(font, data, original, px, left, y)
&& (glyph.id != font.glyph_id(original)
|| !matches!(original, '、' | '。' | 'ー' | '(' | ')' | '「' | '」' | '『' | '』' | '【' | '】'))
{
glyphs.push((face, glyph, None));
y += px;
continue;
}
}
let vertical_form = match original {
'、' => '︑',
'。' => '︒',
@@ -562,12 +611,264 @@ impl Default for ExportOptions {
}
}
/// The `app.export` params that are not export options: which photos, where to, a preset to start
/// from, and the desktop app's background flag.
pub const TARGET_PARAMS: &[&str] = &["id", "ids", "path", "dir", "preset", "background"];
/// The export option params [`ExportOptions::from_json`] reads (and [`ExportOptions::to_json`] writes).
pub const OPTION_PARAMS: &[&str] = &[
"format",
"quality",
"resize",
"longEdge",
"shortEdge",
"width",
"height",
"megapixels",
"percent",
"dontEnlarge",
"ppi",
"limitKb",
"sharpen",
"sharpenAmount",
"naming",
"startNumber",
"subfolder",
"conflict",
"tiffCompression",
"dngCompression",
"metadata",
"removeLocation",
"watermark",
"colorSpace",
"bitDepth",
];
/// The keys of a `watermark` object ([`Watermark`], camelCase).
pub const WATERMARK_PARAMS: &[&str] = &["text", "vertical", "size", "opacity", "anchor", "inset", "color", "shadow", "image", "imageWidth"];
/// Text height as a fraction of the short edge: the range the renderer draws without clamping.
pub const WATERMARK_SIZE_RANGE: (f32, f32) = (0.005, 0.5);
/// Margin as a fraction of the short edge.
pub const WATERMARK_INSET_RANGE: (f32, f32) = (0.0, 0.4);
/// A graphic's width as a fraction of the photo's width.
pub const WATERMARK_IMAGE_WIDTH_RANGE: (f32, f32) = (0.01, 1.0);
/// Levenshtein distance, for "did you mean" on a misspelled key (both inputs are short).
fn edit_distance(a: &str, b: &str) -> usize {
let (a, b): (Vec<char>, Vec<char>) = (a.chars().collect(), b.chars().collect());
let mut prev: Vec<usize> = (0..=b.len()).collect();
for (i, ca) in a.iter().enumerate() {
let mut cur = vec![i + 1; b.len() + 1];
for (j, cb) in b.iter().enumerate() {
let sub = prev.get(j).copied().unwrap_or(0) + usize::from(ca != cb);
let del = prev.get(j + 1).copied().unwrap_or(0) + 1;
let ins = cur.get(j).copied().unwrap_or(0) + 1;
if let Some(c) = cur.get_mut(j + 1) {
*c = sub.min(del).min(ins);
}
}
prev = cur;
}
prev.last().copied().unwrap_or(0)
}
/// `unknown parameter \`k\``, naming the closest known key when one is near.
fn unknown_key(what: &str, k: &str, known: &[&str]) -> String {
let near = known.iter().map(|n| (edit_distance(&k.to_ascii_lowercase(), &n.to_ascii_lowercase()), *n)).filter(|(d, _)| *d <= 2).min();
match near {
Some((_, n)) => format!("unknown {what} `{k}` (did you mean `{n}`?)"),
None => format!("unknown {what} `{k}` (one of {})", known.join(", ")),
}
}
/// A JSON integer (`92`; never `92.5` or `"92"`), as [`ExportOptions::from_json`] reads them.
fn whole(v: &serde_json::Value) -> Option<i64> {
v.as_i64()
}
impl ExportOptions {
/// Stored export settings (a saved preset, `prefs.json ▸ lastExport`) with the keys an older or
/// newer version may have written and this one doesn't know dropped, also inside `watermark`.
/// [`ExportOptions::validate`] is strict about what a caller types; settings saved on disk are
/// not the caller's typo, and refusing them would make a preset or Export with Previous fail
/// for good after an update.
pub fn known_keys_only(p: &serde_json::Value) -> serde_json::Value {
let mut p = p.clone();
if let Some(o) = p.as_object_mut() {
o.retain(|k, _| OPTION_PARAMS.contains(&k.as_str()) || TARGET_PARAMS.contains(&k.as_str()));
if let Some(w) = o.get_mut("watermark").and_then(serde_json::Value::as_object_mut) {
w.retain(|k, _| WATERMARK_PARAMS.contains(&k.as_str()));
}
}
p
}
/// Strict [`ExportOptions::from_json`] for a command that is about to export (issue #181): an
/// unknown parameter or a value that cannot be read as what it is for is an error, never a
/// silent default. `app.export` targets (`id`, `ids`, `path`, `dir`, `preset`, `background`)
/// are accepted and left to the caller.
pub fn from_params(p: &serde_json::Value) -> crate::Result<Self> {
Self::validate("app.export", p)?;
Ok(Self::from_json(p))
}
/// Check `app.export` params for `cmd` (the command named in the error): every key must be an
/// export option ([`OPTION_PARAMS`]) or a target ([`TARGET_PARAMS`]), every value must be the
/// kind of thing its key takes, and a `watermark` object must hold only [`WATERMARK_PARAMS`]
/// with its sizes in range (issue #183). `null` means "not given", as a dropped key does.
pub fn validate(cmd: &str, p: &serde_json::Value) -> crate::Result<()> {
use serde_json::Value;
let bad = |msg: String| crate::cmd::bad(cmd, msg);
let Some(obj) = p.as_object() else { return Err(bad("params must be an object".into())) };
let known: Vec<&str> = OPTION_PARAMS.iter().chain(TARGET_PARAMS).copied().collect();
let int = |k: &str, v: &Value, lo: i64, hi: i64| -> crate::Result<i64> {
whole(v).filter(|i| (lo..=hi).contains(i)).ok_or_else(|| bad(format!("`{k}` must be an integer {lo}..{hi}")))
};
let num = |k: &str, v: &Value, lo: f64, hi: f64| -> crate::Result<f64> {
v.as_f64().filter(|f| f.is_finite() && (lo..=hi).contains(f)).ok_or_else(|| bad(format!("`{k}` must be a number {lo}..{hi}")))
};
fn string<'a>(cmd: &str, k: &str, v: &'a Value) -> crate::Result<&'a str> {
v.as_str().ok_or_else(|| crate::cmd::bad(cmd, format!("`{k}` must be a string")))
}
let boolean = |k: &str, v: &Value| -> crate::Result<bool> { v.as_bool().ok_or_else(|| bad(format!("`{k}` must be true or false"))) };
let one_of = |k: &str, v: &Value, options: &[&str]| -> crate::Result<()> {
let s = string(cmd, k, v)?;
if options.iter().any(|o| o.eq_ignore_ascii_case(s)) { Ok(()) } else { Err(bad(format!("`{k}` must be one of {}", options.join("|")))) }
};
for (k, v) in obj {
if v.is_null() {
continue;
}
match k.as_str() {
"format" => {
if ExportFormat::parse(string(cmd, k, v)?).is_none() {
return Err(bad(format!("`{k}` must be one of jpeg|png|tiff|webp|avif|dng|original")));
}
}
"quality" => {
int(k, v, 1, 100)?;
}
"ppi" => {
int(k, v, 1, u16::MAX as i64)?;
}
"limitKb" => {
int(k, v, 0, u32::MAX as i64)?;
}
"startNumber" => {
int(k, v, 0, 999_999)?;
}
"bitDepth" => {
if !matches!(whole(v), Some(8 | 10 | 16 | 32)) {
return Err(bad(format!("`{k}` must be 8, 10, 16 or 32")));
}
}
"longEdge" | "shortEdge" | "width" | "height" | "megapixels" | "percent" => {
num(k, v, 0.0, 1.0e9)?;
}
"resize" => {
let Some(r) = v.as_object() else { return Err(bad(format!("`{k}` must be an object {{mode, value, height?, dontEnlarge?}}"))) };
if let Some(x) = r.keys().find(|x| !matches!(x.as_str(), "mode" | "value" | "height" | "dontEnlarge")) {
return Err(bad(unknown_key("resize key", x, &["mode", "value", "height", "dontEnlarge"])));
}
serde_json::from_value::<Resize>(v.clone()).map_err(|e| bad(format!("`{k}`: {e}")))?;
}
"dontEnlarge" | "removeLocation" | "background" => {
boolean(k, v)?;
}
"naming" | "subfolder" | "path" | "dir" | "preset" => {
string(cmd, k, v)?;
}
"sharpen" => one_of(k, v, &["none", "screen", "matte", "glossy"])?,
"sharpenAmount" => one_of(k, v, &["low", "standard", "high"])?,
"conflict" => one_of(k, v, &["unique", "overwrite", "skip"])?,
"tiffCompression" => one_of(k, v, &["none", "lzw", "zip", "deflate"])?,
"dngCompression" => {
serde_json::from_value::<DngCompression>(v.clone())
.map_err(|_| bad(format!("`{k}` must be one of lossless|deflate|uncompressed")))?;
}
"metadata" => one_of(k, v, &["all", "allExceptCamera", "copyright", "none"])?,
"colorSpace" => {
if OutputSpace::parse(string(cmd, k, v)?).is_none() {
return Err(bad(format!("`{k}` must be one of srgb|displayP3|adobeRgb|proPhoto|rec2020")));
}
}
"watermark" => Self::validate_watermark(cmd, v)?,
"id" => {
int(k, v, 0, i64::MAX)?;
}
"ids" => {
let ok = v.as_array().is_some_and(|a| a.iter().all(|x| x.is_u64()));
if !ok {
return Err(bad(format!("`{k}` must be an array of photo ids")));
}
}
_ => return Err(bad(unknown_key("parameter", k, &known))),
}
}
Ok(())
}
/// A `watermark` value: text, or an object of [`WATERMARK_PARAMS`] (issue #183).
fn validate_watermark(cmd: &str, v: &serde_json::Value) -> crate::Result<()> {
use serde_json::Value;
let bad = |msg: String| crate::cmd::bad(cmd, msg);
let obj = match v {
Value::String(_) => return Ok(()),
Value::Object(o) => o,
_ => {
return Err(bad(
"`watermark` must be text or an object {text, vertical, size, opacity, anchor, inset, color, shadow, image, imageWidth}".into(),
));
}
};
let frac = |k: &str, v: &Value, (lo, hi): (f32, f32), unit: &str| -> crate::Result<()> {
match v.as_f64() {
Some(f) if f.is_finite() && (f64::from(lo)..=f64::from(hi)).contains(&f) => Ok(()),
_ => Err(bad(format!("`watermark.{k}` must be a number {lo}..{hi} ({unit})"))),
}
};
for (k, v) in obj {
if v.is_null() {
continue;
}
match k.as_str() {
"text" | "image" => {
v.as_str().ok_or_else(|| bad(format!("`watermark.{k}` must be a string")))?;
}
"vertical" | "shadow" => {
v.as_bool().ok_or_else(|| bad(format!("`watermark.{k}` must be true or false")))?;
}
"size" => frac(k, v, WATERMARK_SIZE_RANGE, "fraction of the short edge")?,
"opacity" => frac(k, v, (0.0, 1.0), "0 = invisible, 1 = solid")?,
"inset" => frac(k, v, WATERMARK_INSET_RANGE, "fraction of the short edge")?,
"imageWidth" => frac(k, v, WATERMARK_IMAGE_WIDTH_RANGE, "fraction of the photo's width")?,
"anchor" => {
serde_json::from_value::<Anchor>(v.clone()).map_err(|_| {
bad("`watermark.anchor` must be one of topLeft|top|topRight|left|center|right|bottomLeft|bottom|bottomRight".into())
})?;
}
"color" => {
let ok = v.as_array().is_some_and(|a| a.len() == 3 && a.iter().all(|c| matches!(whole(c), Some(0..=255))));
if !ok {
return Err(bad("`watermark.color` must be [r, g, b] with 0..255 each (sRGB)".into()));
}
}
_ => return Err(bad(unknown_key("watermark key", k, WATERMARK_PARAMS))),
}
}
serde_json::from_value::<Watermark>(v.clone()).map(|_| ()).map_err(|e| bad(format!("`watermark`: {e}")))
}
/// Read options from command params (`format`, `quality`, `limitKb`, `sharpen`, `sharpenAmount`,
/// `naming`, `metadata`, `removeLocation`, `watermark`, `colorSpace`, `bitDepth`, `ppi`) and the
/// size: one of `longEdge`, `shortEdge`, `width`, `height` (both = fit inside W × H),
/// `megapixels`, `percent` (absent or 0 = full size), or a [`Resize`] object as `resize`; plus
/// `dontEnlarge` (default true).
///
/// Lenient: anything unknown or unreadable falls back to the default, which suits stored
/// prefs and dialog prefill. A command that is about to export uses
/// [`ExportOptions::from_params`], which validates first.
pub fn from_json(p: &serde_json::Value) -> Self {
use serde_json::Value;
let d = Self::default();
@@ -1392,6 +1693,49 @@ mod tests {
}
}
#[test]
fn biz_ud_vertical_watermarks_place_punctuation_at_the_top_right() {
use ab_glyph::Font;
let Some(entry) = crate::fonts::CRAFT_FONTS.iter().find(|f| f.family == "BIZ UDMincho") else {
eprintln!("skipped: built without BIZ UDMincho from craft-fonts");
return;
};
let font = ab_glyph::FontRef::try_from_slice(entry.bytes).unwrap();
// BIZ UDMincho has no Unicode presentation-form characters: the old fallback drew 、。.
assert_eq!(font.glyph_id('︑').0, 0);
assert_eq!(font.glyph_id('︒').0, 0);
let data = harfrust::ShaperData::new(&harfrust::FontRef::new(entry.bytes).unwrap());
for ch in ['、', '。'] {
let glyph = vertical_watermark_glyph(&font, &data, ch, 100.0, 0.0, 0.0).unwrap();
assert_ne!(glyph.id, font.glyph_id(ch), "the font's vertical alternate for {ch}");
let rect = font.outline_glyph(glyph).unwrap().px_bounds();
assert!(rect.min.x > 50.0 && rect.max.y < 50.0, "{ch} in the upper right: {rect:?}");
}
let craft = Box::leak(
vec![crate::fonts::CraftFont { family: entry.family, style: entry.style, scripts: entry.scripts, bytes: entry.bytes }].into_boxed_slice(),
);
let wm = Watermark {
text: "、。".into(),
vertical: true,
size: 0.1,
anchor: Anchor::TopLeft,
inset: 0.0,
shadow: false,
opacity: 1.0,
..Default::default()
};
let mut pixels = Vec::new();
watermark_coverage(400, 300, &wm, craft, |x, y, k, _| {
if k > 0.5 {
pixels.push((x, y));
}
});
assert!(!pixels.is_empty());
for (x, y) in pixels {
assert!(x > 15 && y % 30 < 15, "actual export coverage at the upper right of the 30 px cell: {x}, {y}");
}
}
#[test]
fn japanese_watermarks_support_vertical_columns() {
use ab_glyph::Font;
@@ -1720,4 +2064,99 @@ mod tests {
Err(e) => assert!(e.contains("not available"), "{e}"),
}
}
/// Issue #181: a command about to export refuses what `from_json` would silently default —
/// unknown keys (naming the nearest known one) and values of the wrong kind or range.
#[test]
fn export_params_are_validated() {
use serde_json::json;
let err = |p: serde_json::Value| ExportOptions::from_params(&p).err().map(|e| e.to_string()).unwrap_or_default();
let e = err(json!({"longEdgee": 400}));
assert!(e.starts_with("invalid parameters for `app.export`: unknown parameter `longEdgee` (did you mean `longEdge`?)"), "{e}");
assert!(err(json!({"qualty": 5})).contains("did you mean `quality`"));
assert!(err(json!({"longEdge": "banana"})).contains("`longEdge` must be a number"));
assert!(err(json!({"quality": 0})).contains("`quality` must be an integer 1..100"));
assert!(err(json!({"quality": "5"})).contains("`quality` must be an integer"));
assert!(err(json!({"quality": 92.5})).contains("`quality` must be an integer"));
assert!(err(json!({"sharpen": "lots"})).contains("`sharpen` must be one of none|screen|matte|glossy"));
assert!(err(json!({"format": "bmp"})).contains("`format` must be one of"));
assert!(err(json!({"bitDepth": 12})).contains("`bitDepth` must be 8, 10, 16 or 32"));
assert!(err(json!({"colorSpace": "cmyk"})).contains("`colorSpace` must be one of"));
assert!(err(json!({"dontEnlarge": "no"})).contains("`dontEnlarge` must be true or false"));
assert!(err(json!({"resize": {"mode": "longEdge", "valeu": 10}})).contains("did you mean `value`"));
assert!(err(json!({"resize": {"mode": "diagonal", "value": 10}})).contains("`resize`:"));
assert!(err(json!({"ids": [1, "x"]})).contains("`ids` must be an array of photo ids"));
assert!(err(json!("jpeg")).contains("params must be an object"));
assert!(
ExportOptions::validate("export.savePreset", &json!({"qualty": 5}))
.unwrap_err()
.to_string()
.starts_with("invalid parameters for `export.savePreset`")
);
// everything the presets, the dialog, Export with Previous and `render --opt` send passes
for p in builtin_presets() {
ExportOptions::from_params(&p.params).unwrap_or_else(|e| panic!("{}: {e}", p.name));
}
let full = ExportOptions {
resize: Some(Resize::default()),
limit_kb: Some(500),
watermark: Some(Watermark { text: "©".into(), ..Default::default() }),
bit_depth: Some(16),
..Default::default()
};
let mut j = full.to_json();
for (k, v) in [
("dir", json!("/tmp/out")),
("background", json!(true)),
("preset", json!("x")),
("ids", json!([1, 2])),
("id", json!(1)),
("path", json!("a.jpg")),
] {
j[k] = v;
}
assert_eq!(ExportOptions::from_params(&j).unwrap(), full);
assert_eq!(ExportOptions::from_params(&ExportOptions::default().to_json()).unwrap(), ExportOptions::default());
// null is "not given", as a dropped key is
ExportOptions::from_params(&json!({"quality": 92, "longEdge": null, "watermark": null})).unwrap();
}
/// Issue #183: the `watermark` object's keys, kinds and ranges are checked (and documented in
/// the `app.export` param string), instead of unknown keys passing and sizes clamping silently.
#[test]
fn watermark_params_are_validated() {
use serde_json::json;
let err = |w: serde_json::Value| ExportOptions::from_params(&json!({"watermark": w})).err().map(|e| e.to_string()).unwrap_or_default();
assert!(
err(json!(5))
.contains("`watermark` must be text or an object {text, vertical, size, opacity, anchor, inset, color, shadow, image, imageWidth}")
);
assert!(
err(json!({"text": "x", "bogus": 1}))
.contains("unknown watermark key `bogus` (one of text, vertical, size, opacity, anchor, inset, color, shadow, image, imageWidth)")
);
assert!(err(json!({"text": "x", "sizee": 0.1})).contains("did you mean `size`"));
for size in [json!(3), json!(0.5001), json!(0), json!("big")] {
let e = err(json!({"text": "x", "size": size}));
assert!(e.contains("`watermark.size` must be a number 0.005..0.5 (fraction of the short edge)"), "{size}: {e}");
}
assert!(err(json!({"text": "x", "opacity": 1.5})).contains("`watermark.opacity` must be a number 0..1"));
assert!(err(json!({"text": "x", "inset": -0.1})).contains("`watermark.inset` must be a number 0..0.4"));
assert!(err(json!({"image": "logo.png", "imageWidth": 2})).contains("`watermark.imageWidth` must be a number 0.01..1"));
assert!(err(json!({"text": "x", "anchor": "middle"})).contains("`watermark.anchor` must be one of"));
assert!(err(json!({"text": "x", "color": [255, 255]})).contains("`watermark.color` must be [r, g, b]"));
assert!(err(json!({"text": "x", "color": [255, 255, 256]})).contains("`watermark.color`"));
assert!(err(json!({"text": "x", "vertical": "yes"})).contains("`watermark.vertical` must be true or false"));
assert!(err(json!({"text": 7})).contains("`watermark.text` must be a string"));
// the documented shapes pass and read as before
let o = ExportOptions::from_params(&json!({"watermark": "© me"})).unwrap();
assert_eq!(o.watermark.map(|w| w.text).as_deref(), Some("© me"));
let w = json!({"text": "日本語", "vertical": true, "size": 0.1, "opacity": 0.5, "anchor": "topLeft", "inset": 0.05, "color": [0, 0, 0], "shadow": false, "image": "", "imageWidth": 0.3});
let wm = ExportOptions::from_params(&json!({"watermark": w})).unwrap().watermark.unwrap();
assert!((wm.size - 0.1).abs() < 1e-6 && wm.vertical && wm.anchor == Anchor::TopLeft && wm.color == [0, 0, 0] && !wm.shadow);
assert_eq!(ExportOptions::from_params(&json!({"watermark": {"text": "x", "size": 0.5}})).unwrap().watermark.map(|w| w.size), Some(0.5));
// the Export dialog's sliders (1–15 % size, 5–100 % opacity, 2–100 % width) stay inside the accepted ranges
assert!(WATERMARK_SIZE_RANGE.0 <= 0.01 && WATERMARK_SIZE_RANGE.1 >= 0.15);
assert!(WATERMARK_IMAGE_WIDTH_RANGE.0 <= 0.02 && WATERMARK_IMAGE_WIDTH_RANGE.1 >= 1.0);
}
}
+339 -25
View File
@@ -43,8 +43,9 @@ fn meta_of(m: &lightcraft_meta::Metadata) -> (Meta, Option<String>) {
(meta, m.capture_time.as_ref().map(|d| d.to_iso()))
}
/// Lens corrections embedded in a DNG's `OpcodeList3` (`WarpRectilinear`, `FixVignetteRadial`), re-expressed for
/// the default-cropped, EXIF-oriented image. These are the only "profile" corrections LightCraft applies.
/// Lens corrections embedded in a raw's `OpcodeList3` (`WarpRectilinear`, `FixVignetteRadial`: a DNG's own, or the
/// raw reader's equivalent of the camera's correction, e.g. Panasonic / Leica RW2 distortion), re-expressed for the
/// default-cropped, EXIF-oriented image. These are the only "profile" corrections LightCraft applies.
pub fn embedded_lens(raw: &lightcraft_raw::RawInfo) -> Option<lightcraft_develop::EmbeddedLens> {
use lightcraft_develop::{EmbeddedLens, EmbeddedVignette, EmbeddedWarp};
use lightcraft_geom::Point;
@@ -87,24 +88,22 @@ pub fn embedded_lens(raw: &lightcraft_raw::RawInfo) -> Option<lightcraft_develop
Some(lightcraft_pipeline::optics::reorient_lens(&lens, raw.orientation, cw, ch))
}
fn is_lens_opcode(op: &lightcraft_raw::Opcode) -> bool {
matches!(op, lightcraft_raw::Opcode::WarpRectilinear { .. } | lightcraft_raw::Opcode::FixVignetteRadial { .. })
}
fn ext_upper(name: &str) -> String {
std::path::Path::new(name).extension().map(|e| e.to_string_lossy().to_uppercase()).unwrap_or_default()
}
/// Probe a file's bytes: kind, dimensions (oriented), metadata. Raws are described from their
/// headers ([`lightcraft_raw::probe_info`]): no pixel data is decompressed.
/// Probe a file's bytes: kind, dimensions (oriented), metadata. No pixel data is decompressed:
/// raws are described from their headers ([`lightcraft_raw::probe_info`]), other images too
/// ([`lightcraft_codecs::read_header`], which refuses truncated files but can't see damage inside
/// compressed data that is all there: such a file imports and shows as unreadable when rendered).
pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result<ProbeInfo, String> {
let content_hash = Some(lightcraft_preview::hash_bytes(bytes).to_string());
let m = lightcraft_meta::extract(bytes);
let (meta, captured) = meta_of(&m);
if lightcraft_raw::probe(bytes).is_some() {
let raw = match lightcraft_raw::probe_info(bytes) {
let raw = match lightcraft_raw::probe_info(bytes).map_err(|e| preview_reason(bytes, e)) {
Ok(r) => r,
Err(lightcraft_raw::RawError::Unsupported(why)) => {
Err(Ok(why)) => {
// a raw variant we can't decode yet: describe it from its embedded preview
let (w, h) = embedded_preview_size(bytes).ok_or(format!("unsupported raw ({why}) without an embedded preview"))?;
return Ok(ProbeInfo {
@@ -122,7 +121,7 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result<ProbeInfo, String> {
..Default::default()
});
}
Err(e) => return Err(e.to_string()),
Err(Err(e)) => return Err(e),
};
let (mut w, mut h) = (raw.crop.width.max(1) as u32, raw.crop.height.max(1) as u32);
if w <= 1 || h <= 1 {
@@ -155,9 +154,10 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result<ProbeInfo, String> {
if !fmt.can_decode() {
return Err(format!("{fmt:?} files are not supported yet"));
}
let d = lightcraft_codecs::decode(bytes, lightcraft_codecs::DecodeOptions::fit(64, 64)).map_err(|e| e.to_string())?;
let o = Orientation::from_exif(d.orientation);
let (mut w, mut h) = (d.source_width, d.source_height);
// headers only (issue #367: decoding the pixels was nearly all of an import's CPU time)
let header = lightcraft_codecs::read_header(bytes).map_err(|e| e.to_string())?;
let o = Orientation::from_exif(header.orientation);
let (mut w, mut h) = (header.width, header.height);
if o.swaps_axes() {
std::mem::swap(&mut w, &mut h);
}
@@ -182,6 +182,21 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result<ProbeInfo, String> {
})
}
/// Why a raw that failed to decode should show its embedded preview instead (`Ok`), or the error
/// to report (`Err`). Variants we can't decode yet always fall back. So does any CR3 error: the CRX
/// decoder is verified on few bodies, and every CR3 opened from its embedded JPEG before it existed.
/// So does any other failure of a recognised raw (damaged or oversized raw data, unreadable TIFF
/// structure): its preview may still be intact. Only a file that is not a raw reports an error.
fn preview_reason(bytes: &[u8], e: lightcraft_raw::RawError) -> Result<String, String> {
use lightcraft_raw::RawError;
match e {
RawError::Unsupported(why) => Ok(why),
e if lightcraft_raw::probe(bytes) == Some(lightcraft_raw::RawFormat::Cr3) => Ok(format!("CR3 {e}")),
RawError::NotRaw => Err(RawError::NotRaw.to_string()),
e => Ok(e.to_string()),
}
}
/// Sensor clip level (normalised) for highlight reconstruction.
const HIGHLIGHT_CLIP: f32 = 0.99;
@@ -216,21 +231,22 @@ pub fn load_vec(bytes: Vec<u8>, max_edge: usize) -> Result<(Rgb32f, SourceInfo),
fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result<(Rgb32f, SourceInfo), String> {
if lightcraft_raw::probe(&bytes).is_some() {
let mut raw = match lightcraft_raw::decode(&bytes) {
let mut raw = match lightcraft_raw::decode(&bytes).map_err(|e| preview_reason(&bytes, e)) {
Ok(r) => r,
Err(lightcraft_raw::RawError::Unsupported(why)) => {
Err(Ok(why)) => {
// show the camera's embedded JPEG (rendered, not raw) until the variant is supported
return load_embedded_preview(&bytes, max_edge).ok_or(format!("unsupported raw ({why}) without an embedded preview"));
}
Err(e) => return Err(e.to_string()),
Err(Err(e)) => return Err(e),
};
// Embedded lens corrections are applied by the pipeline ("Enable Profile Corrections"), not baked in
// (removed before the camera look's binned sensor proxy too, which needs an empty `OpcodeList3`).
let lens = embedded_lens(&raw.info());
raw.opcodes.list3.retain(|op| !op.is_lens_correction());
let xy = lightcraft_raw::color::as_shot_white_xy(&raw);
let t = lightcraft_raw::color::camera_transform(&raw, xy);
let camera_look = crate::camera_preview::fit_preview(&raw, &bytes, &t);
drop(bytes);
// Embedded lens corrections are applied by the pipeline ("Enable Profile Corrections"), not baked in.
let lens = embedded_lens(&raw.info());
raw.opcodes.list3.retain(|op| !is_lens_opcode(op));
// Previews and thumbnails bin the mosaic straight to (about) the size they need; only
// larger levels (exports, 1:1) demosaic the whole sensor.
let t0 = web_time::Instant::now();
@@ -320,9 +336,38 @@ fn preview_orientation(raw_bytes: &[u8], jpeg_orientation: u16) -> Orientation {
lightcraft_meta::extract(raw_bytes).orientation.unwrap_or(Orientation::Normal)
}
/// What stands in for a raw file we can't decode: its largest embedded JPEG, else, for a TIFF-based
/// raw, the file's own first image when the TIFF reader can show it (a reduced RGB copy: the only
/// preview some containers carry). Returns image bytes for [`lightcraft_codecs::decode`].
fn stand_in_image(bytes: &[u8]) -> Option<std::borrow::Cow<'_, [u8]>> {
if let Some(jpeg) = lightcraft_raw::embedded_preview(bytes) {
return Some(std::borrow::Cow::Owned(jpeg));
}
let undecodable = lightcraft_raw::probe(bytes).is_some_and(|f| !f.is_supported());
(undecodable && lightcraft_codecs::sniff(bytes).is_some_and(|f| f == lightcraft_codecs::Format::Tiff))
.then_some(std::borrow::Cow::Borrowed(bytes))
}
/// Shared colour interpretation for quick previews, unsupported RAW fallback and camera-look fitting.
pub(crate) fn decode_raw_preview(bytes: &[u8], opts: lightcraft_codecs::DecodeOptions) -> Option<lightcraft_codecs::Decoded> {
let jpeg = stand_in_image(bytes)?;
// the colour space is a property of the JPEG; a TIFF stand-in is read by the TIFF reader
let is_jpeg = matches!(jpeg, std::borrow::Cow::Owned(_));
match lightcraft_raw::embedded_preview_color_space(bytes).filter(|_| is_jpeg) {
Some(space) => {
let space = match space {
lightcraft_raw::PreviewColorSpace::Srgb => lightcraft_codecs::NamedSpace::Srgb,
lightcraft_raw::PreviewColorSpace::AdobeRgb => lightcraft_codecs::NamedSpace::AdobeRgb,
};
lightcraft_codecs::decode_jpeg_with_fallback(&jpeg, opts, space).ok()
}
None => lightcraft_codecs::decode(&jpeg, opts).ok(),
}
}
/// Oriented size of the embedded preview of a raw file.
fn embedded_preview_size(bytes: &[u8]) -> Option<(u32, u32)> {
let jpeg = lightcraft_raw::embedded_preview(bytes)?;
let jpeg = stand_in_image(bytes)?;
let d = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions::fit(64, 64)).ok()?;
let (mut w, mut h) = (d.source_width, d.source_height);
if preview_orientation(bytes, d.orientation).swaps_axes() {
@@ -333,8 +378,7 @@ fn embedded_preview_size(bytes: &[u8]) -> Option<(u32, u32)> {
/// The embedded preview of a raw file as a working-space image no larger than `max_edge`, oriented.
pub fn load_embedded_preview(bytes: &[u8], max_edge: usize) -> Option<(Rgb32f, SourceInfo)> {
let jpeg = lightcraft_raw::embedded_preview(bytes)?;
let d = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions::fit(max_edge as u32, max_edge as u32)).ok()?;
let d = decode_raw_preview(bytes, lightcraft_codecs::DecodeOptions::fit(max_edge as u32, max_edge as u32))?;
let img = d.to_working();
let img = if img.width.max(img.height) > max_edge { fit(&img, max_edge, max_edge, Filter::Mitchell) } else { img };
Some((img.oriented(preview_orientation(bytes, d.orientation)), SourceInfo::default()))
@@ -343,8 +387,7 @@ pub fn load_embedded_preview(bytes: &[u8], max_edge: usize) -> Option<(Rgb32f, S
/// The embedded preview of a raw file for display (sRGB, oriented, no larger than `max_edge`): the
/// loupe and grid show it until the raw itself has been developed ([`crate::media::QuickJob`]).
pub fn embedded_preview_srgb(bytes: &[u8], max_edge: usize) -> Option<lightcraft_raster::Rgba8> {
let jpeg = lightcraft_raw::embedded_preview(bytes)?;
let mut d = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions::fit(max_edge as u32, max_edge as u32)).ok()?;
let mut d = decode_raw_preview(bytes, lightcraft_codecs::DecodeOptions::fit(max_edge as u32, max_edge as u32))?;
if d.image.width.max(d.image.height) > max_edge {
d.image = fit(&d.image, max_edge, max_edge, Filter::Box);
d.alpha = None;
@@ -402,6 +445,162 @@ mod tests {
use super::*;
use lightcraft_codecs::{ChromaSubsampling, EncodeImage, EncodeMeta, Samples, encode_jpeg};
/// Issue #367: probes read headers, not pixels, and report what the decode-based probe did:
/// oriented dimensions (EXIF orientation 6 swaps them), metadata, the error for a truncated
/// file or one that isn't an image.
#[test]
fn image_probe_reads_headers_like_the_decode_did() {
let (w, h) = (300u32, 200u32);
let rgb: Vec<u8> = (0..w * h * 3).map(|i| (i * 7 % 251) as u8).collect();
let img = EncodeImage::new(w, h, 3, Samples::U8(&rgb));
for o in [None, Some(1), Some(6), Some(3), Some(8)] {
let exif = o.map(lightcraft_codecs::exif::minimal_exif);
let meta = EncodeMeta { exif: exif.as_deref(), ..Default::default() };
let jpeg = encode_jpeg(&img, 90, ChromaSubsampling::S420, &meta).unwrap();
let png = lightcraft_codecs::encode_png(&img, &meta).unwrap();
for (name, bytes) in [("a.jpg", &jpeg), ("a.png", &png)] {
let p = probe_bytes(name, bytes).unwrap();
// what the probe computed from a decode before
let d = lightcraft_codecs::decode(bytes, lightcraft_codecs::DecodeOptions::fit(64, 64)).unwrap();
let swap = Orientation::from_exif(d.orientation).swaps_axes();
let want = if swap { (d.source_height, d.source_width) } else { (d.source_width, d.source_height) };
assert_eq!((p.width, p.height), want, "{name} {o:?}");
assert_eq!((p.width, p.height), if matches!(o, Some(6 | 8)) { (h, w) } else { (w, h) }, "{name} {o:?}");
assert_eq!(p.kind, MediaKind::Image);
assert_eq!(p.format, if name == "a.jpg" { "JPEG" } else { "PNG" });
assert_eq!(p.file_size, bytes.len() as u64);
assert!(p.content_hash.is_some());
assert!(probe_bytes(name, &bytes[..bytes.len() / 2]).is_err(), "{name}: truncated file accepted");
}
}
assert!(probe_bytes("x.jpg", b"not an image").is_err());
}
/// Luminance (from linear RGB) minus its 9 × 9 local mean: tone differences between a render and a camera JPEG
/// mostly cancel, edges remain.
fn detail(w: usize, h: usize, lum: impl Fn(usize, usize) -> f32) -> Vec<f32> {
let l: Vec<f32> = (0..w * h).map(|i| lum(i % w, i / w)).collect();
let mut out = vec![0.0; w * h];
for y in 0..h {
for x in 0..w {
let (x0, x1, y0, y1) = (x.saturating_sub(4), (x + 5).min(w), y.saturating_sub(4), (y + 5).min(h));
let m = (y0..y1).flat_map(|yy| (x0..x1).map(move |xx| (xx, yy))).map(|(xx, yy)| l[yy * w + xx]).sum::<f32>();
out[y * w + x] = l[y * w + x] - m / ((x1 - x0) * (y1 - y0)) as f32;
}
}
out
}
/// The integer shift (within ±`r` px) of the `p × p` patch of `a` at `(x, y)` that best matches `b`, with its
/// normalised cross-correlation, or `None` when the patch has too little detail to tell.
fn patch_shift(a: &[f32], b: &[f32], w: usize, (x, y): (usize, usize), p: usize, r: i64) -> Option<((i64, i64), f32)> {
let pa: Vec<f32> = (0..p * p).map(|i| a[(y + i / p) * w + x + i % p]).collect();
let na = pa.iter().map(|v| v * v).sum::<f32>().sqrt();
if na < 1e-3 * p as f32 {
return None;
}
let mut best = (f32::MIN, (0, 0));
for dy in -r..=r {
for dx in -r..=r {
let (bx, by) = (x as i64 + dx, y as i64 + dy);
let (mut dot, mut nb) = (0.0f32, 0.0f32);
for i in 0..p {
for j in 0..p {
let v = b[(by as usize + i) * w + bx as usize + j];
dot += pa[i * p + j] * v;
nb += v * v;
}
}
let c = dot / (na * nb.sqrt()).max(1e-12);
if c > best.0 {
best = (c, (dx, dy));
}
}
}
Some((best.1, best.0))
}
/// Issue #256: public Panasonic / Leica raws (skipped without the corpus) are corrected for distortion the way the
/// camera corrected its own JPEG (tag 0x0119): at the corners of the frame, the render with "Enable Profile
/// Corrections" lines up with the embedded JPEG within 2 px at 640 px, while the uncorrected render of the wide
/// lenses is far off. A file shot with the camera's correction off (DMC-GH1) carries no correction.
#[test]
fn corpus_rw2_distortion_matches_the_camera_jpeg() {
let dir = std::env::var_os("LIGHTCRAFT_CORPUS")
.map(std::path::PathBuf::from)
.unwrap_or_else(|| std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus"))
.join("raw");
let Ok(gh1) = std::fs::read(dir.join("rw2-panasonic-gh1.rw2")) else {
eprintln!("skip: {} absent", dir.display());
return;
};
assert_eq!(probe_bytes("gh1.rw2", &gh1).unwrap().embedded_lens, None, "correction off: nothing to apply");
assert_eq!(load_bytes(&gh1, 320).unwrap().1.lens, None);
// (file, how far off (px) the uncorrected corners are at the least, if they match within ±R at all)
let files = [
("rw2-panasonic-fz1000m2-4x3.rw2", 8), // 9.1 mm, ~13 % at the corners
("rwl-leica-dlux7.rwl", 8), // 10.9 mm, ~13 %
("rw2-panasonic-gx80.rw2", 8), // 14 mm, ~8 %
("rw2-panasonic-gh6.rw2", 3), // 20 mm, ~1.6 %
("rw2-panasonic-s9.rw2", 3), // 45 mm, 3 % zoom after the correction
("rw2-panasonic-g9.rw2", 0), // 40 mm, ~0.5 %
];
const EDGE: usize = 640;
const P: usize = 64;
const R: i64 = 12;
for (name, off_at_least) in files {
let bytes = std::fs::read(dir.join(name)).unwrap();
let probed = probe_bytes(name, &bytes).unwrap();
assert!(probed.embedded_lens.is_some_and(|l| l.warp.is_some()), "{name}: no distortion correction read");
let (src, info) = load_bytes(&bytes, 2 * EDGE).unwrap();
assert_eq!(info.lens, probed.embedded_lens, "{name}: probe and decode disagree");
// Convert to DNG keeps it: the same correction, as the DNG's own `WarpRectilinear`
let dng = lightcraft_raw::write_dng(&lightcraft_raw::decode(&bytes).unwrap(), &Default::default()).unwrap();
assert_eq!(probe_bytes("x.dng", &dng).unwrap().embedded_lens, probed.embedded_lens, "{name}: lost in the DNG");
let mut s = lightcraft_develop::DevelopSettings::default();
let render = |s: &lightcraft_develop::DevelopSettings| {
lightcraft_pipeline::render(&src, &info, s, &lightcraft_pipeline::RenderRequest::fit(EDGE, EDGE)).image
};
let before = render(&s);
s.optics.lens_profile = true;
let after = render(&s);
let (w, h) = (after.width, after.height);
assert_eq!((before.width, before.height), (w, h));
// the camera JPEG shows the whole sensor; a render in an in-camera aspect ratio is its centre
let (cam, _) = load_embedded_preview(&bytes, 4 * EDGE).unwrap();
let (cw, ch) = (cam.width as f64, cam.height as f64);
let (sw, sh) = if cw / ch > w as f64 / h as f64 { (ch * w as f64 / h as f64, ch) } else { (cw, cw * h as f64 / w as f64) };
let cam = cam.crop(((cw - sw) / 2.0).round() as usize, ((ch - sh) / 2.0).round() as usize, sw.round() as usize, sh.round() as usize);
let cam = lightcraft_raster::resample::resize(&cam, w, h, Filter::Mitchell);
let lin = |v: u8| (v as f32 / 255.0).powf(2.2);
let lum8 = |im: &lightcraft_raster::Rgba8, x: usize, y: usize| {
let p = im.data[y * im.width + x];
0.3 * lin(p[0]) + 0.6 * lin(p[1]) + 0.1 * lin(p[2])
};
let reference = detail(w, h, |x, y| {
let p = cam.data[y * w + x];
0.3 * p[0] + 0.6 * p[1] + 0.1 * p[2]
});
let (db, da) = (detail(w, h, |x, y| lum8(&before, x, y)), detail(w, h, |x, y| lum8(&after, x, y)));
let m = R as usize + 4;
let corners = [(m, m), (w - m - P, m), (m, h - m - P), (w - m - P, h - m - P)];
let mut measured = 0;
for c in corners {
let Some(((dx, dy), ca)) = patch_shift(&da, &reference, w, c, P, R) else { continue };
measured += 1;
assert!(dx.abs() <= 2 && dy.abs() <= 2 && ca >= 0.5, "{name}: corrected corner {c:?} off by ({dx}, {dy}) px, ncc {ca}");
if off_at_least > 0
&& let Some(((bx, by), cb)) = patch_shift(&db, &reference, w, c, P, R)
{
let off = bx.abs().max(by.abs()) >= off_at_least || cb < 0.3;
assert!(off, "{name}: uncorrected corner {c:?} already within ({bx}, {by}) px, ncc {cb}");
}
}
assert!(measured >= 2, "{name}: only {measured} corners with detail");
}
}
/// A CR3-shaped file (not decodable yet) whose only content is a `PRVW` preview box.
fn cr3_with_preview(w: u32, h: u32) -> Vec<u8> {
let px: Vec<u8> = (0..w * h).flat_map(|i| [(i % 251) as u8, 128, 200]).collect();
@@ -417,6 +616,43 @@ mod tests {
b
}
/// [`cr3_with_preview`] plus a full-size CRX raw track whose `CMP1` coding header is garbage:
/// the container parses, the raw data doesn't (`RawError::Corrupt`, not `Unsupported`).
fn cr3_with_corrupt_raw(w: u32, h: u32) -> Vec<u8> {
let bx = |kind: &[u8; 4], body: &[u8]| [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat();
let mut craw = vec![0u8; 82];
craw[24..26].copy_from_slice(&(w as u16).to_be_bytes());
craw[26..28].copy_from_slice(&(h as u16).to_be_bytes());
craw.extend(bx(b"CMP1", &[0xff; 4]));
let mut stsd = vec![0, 0, 0, 0, 0, 0, 0, 1];
stsd.extend(bx(b"CRAW", &craw));
let stsz = [0u32, 16, 1].map(u32::to_be_bytes).concat();
let co64 = [&[0u8, 0, 0, 0, 0, 0, 0, 1][..], &0u64.to_be_bytes()].concat();
let stbl = [bx(b"stsd", &stsd), bx(b"stsz", &stsz), bx(b"co64", &co64)].concat();
let trak = bx(b"trak", &bx(b"mdia", &bx(b"minf", &bx(b"stbl", &stbl))));
let preview = cr3_with_preview(w, h);
[&preview[..24], &bx(b"moov", &trak), &preview[24..]].concat()
}
/// A CR3 whose raw data fails to decode, for any reason, still opens from its embedded JPEG:
/// CR3s did before the CRX decoder, which is verified on few bodies (PR #279 review).
#[test]
fn corrupt_cr3_falls_back_to_embedded_preview() {
let b = cr3_with_corrupt_raw(48, 32);
let err = lightcraft_raw::decode(&b).expect_err("the fixture's raw data is corrupt");
assert!(!matches!(err, lightcraft_raw::RawError::Unsupported(_)), "{err}");
let p = probe_bytes("x.cr3", &b).unwrap();
assert_eq!((p.width, p.height, p.kind, p.format.as_str()), (48, 32, MediaKind::Raw, "CR3"));
assert!(p.preview_only.is_some_and(|why| why.contains("CR3")));
let (img, src) = load_bytes(&b, 24).unwrap();
assert_eq!((img.width, img.height), (24, 16));
assert!(!src.raw);
// other formats still report a corrupt file as an error
let mut dng = crate::tests_xmp::synthetic_dng_with(None, Default::default());
dng.truncate(dng.len() / 2);
assert!(load_bytes(&dng, 24).is_err());
}
/// Issue #138: a DNG's own profile look (hue/saturation map, look table, tone curve) is
/// applied when it's developed — Lightroom-converted DNGs rendered flat and muted without it.
#[test]
@@ -520,6 +756,84 @@ mod tests {
let _ = std::fs::remove_dir_all(&dir);
}
/// A raw whose TIFF shell holds only a small RGB image (IFD0) next to a private block, as Exif
/// declares a picture 25x its size.
fn tiff_shell(w: u32, h: u32, shell: bool) -> Vec<u8> {
use lightcraft_tiff::{IfdBuilder, ImageData, TiffWriter, Value, tags as t};
let mut ifd0 = IfdBuilder::new();
ifd0.set(t::IMAGE_WIDTH, Value::Long(vec![w]));
ifd0.set(t::IMAGE_LENGTH, Value::Long(vec![h]));
ifd0.set(t::BITS_PER_SAMPLE, Value::Short(vec![8, 8, 8]));
ifd0.set(t::SAMPLES_PER_PIXEL, Value::Short(vec![3]));
ifd0.set(t::PHOTOMETRIC, Value::Short(vec![2]));
ifd0.set(t::COMPRESSION, Value::Short(vec![1]));
let px: Vec<u8> = (0..w * h).flat_map(|i| [(i % 251) as u8, 128, 200]).collect();
ifd0.set_image(ImageData::Strips { rows_per_strip: h, strips: vec![px] });
ifd0.set_child(
t::EXIF_IFD,
IfdBuilder::new().with(t::PIXEL_X_DIMENSION, Value::Long(vec![w * 25])).with(t::PIXEL_Y_DIMENSION, Value::Long(vec![h * 25])),
);
let mut b = TiffWriter::default().write(&[ifd0]).unwrap();
if shell {
b.extend(std::iter::repeat_n(0x5au8, (w * h * 625 / 8) as usize + 100));
}
b
}
/// A TIFF raw whose only image is a small thumbnail used to open as a plain image of that size
/// ("a 150 MP file renders 296x220"), with nothing saying so. Now it is recognised as a raw we can't
/// decode: preview only, with the reason, still showing that image.
#[test]
fn thumbnail_only_tiff_raw_is_preview_only_not_a_plain_image() {
let b = tiff_shell(24, 16, true);
let p = probe_bytes("IMG_0001.IIQ", &b).unwrap();
assert_eq!((p.kind, p.format.as_str(), p.width, p.height), (MediaKind::Raw, "IIQ", 24, 16));
let why = p.preview_only.expect("marked preview only");
assert!(why.contains("600x400") && why.contains("24x16"), "{why}");
let (img, src) = load_bytes(&b, 12).unwrap();
assert_eq!((img.width, img.height, src.raw), (12, 8, false));
assert!(embedded_preview_srgb(&b, 12).is_some());
// the same pixels without the private block are a small ordinary image
let plain = probe_bytes("small.tif", &tiff_shell(24, 16, false)).unwrap();
assert_eq!((plain.kind, plain.preview_only, plain.width, plain.height), (MediaKind::Image, None, 24, 16));
assert!(lightcraft_raw::probe(&tiff_shell(24, 16, false)).is_none());
}
/// Recognised raw containers we don't decode (Minolta MRW here: the preview's first byte is
/// overwritten in the file) import as preview only with the JPEG's size, instead of failing.
#[test]
fn undecodable_container_with_a_preview_imports_as_preview_only() {
let px: Vec<u8> = (0..40 * 30).flat_map(|i| [(i % 251) as u8, 128, 200]).collect();
let mut jpeg = encode_jpeg(&EncodeImage::new(40, 30, 3, Samples::U8(&px)), 90, ChromaSubsampling::S444, &EncodeMeta::default()).unwrap();
jpeg[0] = 0x02;
let mut f = b"\0MRM\0\x01\0\0".to_vec();
f.extend(std::iter::repeat_n(0x11u8, 300));
f.extend_from_slice(&jpeg);
f.extend(std::iter::repeat_n(0x22u8, 300));
let p = probe_bytes("A.MRW", &f).unwrap();
assert_eq!((p.kind, p.format.as_str(), p.width, p.height), (MediaKind::Raw, "MRW", 40, 30));
assert!(p.preview_only.is_some_and(|w| w.contains("Mrw")));
let (img, src) = load_bytes(&f, 20).unwrap();
assert_eq!((img.width, img.height, src.raw), (20, 15, false));
// without a JPEG: a clear error
let e = probe_bytes("A.MRW", b"\0MRM\0\x01\0\0 nothing here").unwrap_err();
assert!(e.contains("without an embedded preview"), "{e}");
}
/// Any failure of a recognised raw (not only an unsupported variant) can fall back to its preview;
/// CR3 keeps its own wording.
#[test]
fn decode_failures_of_recognised_raws_may_fall_back_to_the_preview() {
use lightcraft_raw::RawError;
let cr3 = b"\0\0\0\x18ftypcrx \0\0\0\x01";
assert_eq!(preview_reason(b"x", RawError::Unsupported("x".into())), Ok("x".to_string()));
assert!(preview_reason(b"x", RawError::Corrupt("bad strip".into())).is_ok_and(|w| w.contains("bad strip")));
assert!(preview_reason(b"x", RawError::Limit("too big")).is_ok());
assert!(preview_reason(b"x", RawError::Tiff(lightcraft_tiff::TiffError::MissingTag(256))).is_ok());
assert!(preview_reason(b"x", RawError::NotRaw).is_err());
assert!(preview_reason(cr3, RawError::Corrupt("bad".into())).is_ok_and(|w| w.starts_with("CR3 ")));
}
#[test]
fn quick_jobs_show_embedded_previews_then_cached_renders() {
use crate::media::QuickSource;
+467 -75
View File
@@ -33,7 +33,8 @@ use crate::media::ProbeInfo;
/// File extensions LightCraft imports (lower case).
pub const EXTENSIONS: &[&str] = &[
"jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "nrw", "arw", "raf", "orf", "rw2", "rwl", "raw", "pef", "psd", "jxl",
"gif", "bmp", "heic", "avif",
"gif", "bmp", "heic", "avif", // containers LightCraft cannot decode but imports as preview only (their embedded JPEG)
"iiq", "crw", "mrw", "x3f", "kdc", "mos", "erf",
];
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
@@ -48,9 +49,34 @@ pub enum ImportMode {
Move,
}
/// What an import does with a file the library already has in Recently Deleted.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub enum OnDeleted {
/// Leave it there; the report says so ([`Duplicate::existing_deleted`]).
#[default]
Skip,
/// Bring the trashed photo back, with its edits ([`ImportReport::restored`]).
Restore,
/// Delete the trashed record permanently and import the file as a new photo.
Fresh,
}
impl OnDeleted {
pub fn parse(s: &str) -> Option<Self> {
match s {
"skip" => Some(Self::Skip),
"restore" => Some(Self::Restore),
"fresh" => Some(Self::Fresh),
_ => None,
}
}
}
/// What an import does besides adding the photos.
#[derive(Clone, Debug, Default)]
pub struct ImportOptions {
/// A file that is in Recently Deleted: skip (default), restore, or import afresh.
pub on_deleted: OnDeleted,
pub mode: ImportMode,
/// Applied to every imported photo (one History entry).
pub preset: Option<lightcraft_develop::Preset>,
@@ -147,12 +173,18 @@ pub struct Duplicate {
pub existing: Option<u64>,
/// `"path"` (already imported from there) or `"content"` (same bytes elsewhere).
pub reason: &'static str,
/// That photo is in Recently Deleted (see [`OnDeleted`]).
#[serde(rename = "existingDeleted", skip_serializing_if = "std::ops::Not::not")]
pub existing_deleted: bool,
}
#[derive(Clone, Debug, Default, Serialize)]
pub struct ImportReport {
pub imported: Vec<u64>,
pub duplicates: Vec<Duplicate>,
/// [`OnDeleted::Restore`]: photos brought back from Recently Deleted instead of imported.
#[serde(skip_serializing_if = "Vec::is_empty")]
pub restored: Vec<u64>,
/// Move: the originals that were moved (and removed from the source).
#[serde(skip_serializing_if = "Vec::is_empty")]
pub moved: Vec<Moved>,
@@ -286,32 +318,96 @@ pub fn is_supported(path: &Path) -> bool {
}
/// Expand files and folders (recursively) into supported files. `skip` (e.g. the library folder)
/// is never descended into.
/// is never descended into. Each folder's walk is bounded ([`crate::walk::Limits::default`]); a
/// walk that stops at a bound is logged.
pub fn expand(paths: &[String], skip: Option<&Path>) -> Vec<String> {
fn walk(p: &Path, skip: Option<&Path>, out: &mut Vec<String>, top: bool) {
let hidden = p.file_name().is_some_and(|n| n.to_string_lossy().starts_with('.'));
if (hidden && !top) || skip.is_some_and(|s| p == s) {
return;
expand_within(paths, skip, crate::walk::Limits::default()).0
}
/// [`expand`] with the walk bounded by `limits` → (files, whether a walk stopped at a bound).
pub fn expand_within(paths: &[String], skip: Option<&Path>, limits: crate::walk::Limits) -> (Vec<String>, bool) {
let mut out = Vec::new();
let mut truncated = false;
for p in paths {
let p = Path::new(p);
if skip.is_some_and(|s| p == s) {
continue;
}
if p.is_dir() {
let Ok(rd) = std::fs::read_dir(p) else { return };
let mut v: Vec<PathBuf> = rd.flatten().map(|e| e.path()).collect();
v.sort();
for c in v {
walk(&c, skip, out, false);
let w = crate::walk::files_in(p, skip, limits, is_supported);
if w.truncated {
log::warn!(
"import: stopped looking in {} after {} entries (limits: {} entries, {} folder levels)",
p.display(),
w.entries,
limits.max_entries,
limits.max_depth
);
}
} else if top || is_supported(p) {
truncated |= w.truncated;
out.extend(w.files.into_iter().map(|f| f.to_string_lossy().to_string()));
} else {
// explicitly named files are attempted even with an unknown extension (sniffed)
out.push(p.to_string_lossy().to_string());
}
}
let mut out = Vec::new();
for p in paths {
walk(Path::new(p), skip, &mut out, true);
}
let mut seen = std::collections::HashSet::new();
out.retain(|p| seen.insert(p.clone()));
out
(out, truncated)
}
/// What a path given on the command line at launch becomes ([`launch_path`]).
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum LaunchPath {
/// Import this (absolute) file or folder, in the background.
Import(String),
/// Not imported: `path` (absolute) is a whole drive or the whole home folder, which is never
/// walked without being chosen in the app (issue #374).
Refused { path: String, why: String },
}
/// Resolve a launch argument for importing (issue #374). An empty argument is ignored (`None`);
/// a relative one (`.`, `..`, `photos`; `/` or `\` on Windows, the current drive's root) is
/// resolved against `cwd` — the working folder a launcher picked, often a drive root or
/// `C:\Windows\System32` — and refused when that leaves no folder (`cwd` unknown). A drive
/// or file-system root is replaced by its camera folder (`DCIM`, from `dcim`: a memory card) or
/// refused, and so is the home folder `home`: either would walk most of the disk.
pub fn launch_path(arg: &str, cwd: Option<&Path>, home: Option<&Path>, dcim: impl Fn(&Path) -> Option<PathBuf>) -> Option<LaunchPath> {
if arg.trim().is_empty() {
return None;
}
let given = Path::new(arg);
let joined = if given.is_absolute() {
given.to_path_buf()
} else {
match cwd {
Some(c) => c.join(given),
None => return Some(LaunchPath::Refused { path: arg.to_string(), why: "a relative path, and the working folder is unknown".into() }),
}
};
// `.` and `..` resolved by name (no file-system access)
let mut abs = PathBuf::new();
for c in joined.components() {
match c {
std::path::Component::CurDir => {}
std::path::Component::ParentDir => {
abs.pop();
}
other => abs.push(other.as_os_str()),
}
}
let shown = abs.to_string_lossy().to_string();
if abs.parent().is_none() {
return Some(match dcim(&abs) {
Some(d) => LaunchPath::Import(d.to_string_lossy().to_string()),
None => LaunchPath::Refused { path: shown, why: "a whole drive".into() },
});
}
let key = lightcraft_catalog::query::folder_key;
if home.is_some_and(|h| key(&h.to_string_lossy()) == key(&shown)) {
return Some(LaunchPath::Refused { path: shown, why: "the whole home folder".into() });
}
Some(LaunchPath::Import(shown))
}
/// Probe files (headers + content hash) as an import would.
@@ -334,7 +430,7 @@ fn probe_all(probe: Option<&crate::media::FileProbe>, paths: &[String], progress
};
#[cfg(not(target_arch = "wasm32"))]
{
let n = std::thread::available_parallelism().map(|n| n.get()).unwrap_or(4).clamp(1, 8).min(paths.len().max(1));
let n = workers(paths.len());
if n > 1 {
let mut results: Vec<Option<Result<ProbeInfo, String>>> = vec![None; paths.len()];
let next = std::sync::atomic::AtomicUsize::new(0);
@@ -373,18 +469,16 @@ fn probe_all(probe: Option<&crate::media::FileProbe>, paths: &[String], progress
.collect()
}
/// Copy `src` into `root`/`folders` as `name` (default: its own name), made unique with -1, -2…;
/// returns the new path. The copy is verified ([`crate::import_move::copy_new`]: a new file,
/// synced and checked against `probe_hash`, the content hash the probe computed — or compared
/// byte for byte with the source when there is none); a bad one is removed and the photo
/// reported as failed.
fn copy_into(root: &Path, src: &str, folders: &[String], name: Option<&str>, probe_hash: Option<&str>) -> Result<String, String> {
let dir = folders.iter().fold(root.to_path_buf(), |d, f| d.join(f));
/// Copy `src` into `dir` as `name` (default: its own name), made unique with -1, -2…; returns the
/// new path. The copy is verified ([`crate::import_move::copy_new`]: a new file, synced and
/// checked against `probe_hash`, the content hash the probe computed — or compared byte for byte
/// with the source when there is none); a bad one is removed and the photo reported as failed.
fn copy_into(dir: &Path, src: &str, name: Option<&str>, probe_hash: Option<&str>) -> Result<String, String> {
let name = name
.map(str::to_string)
.unwrap_or_else(|| Path::new(src).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| "photo".into()));
let expect = probe_hash.and_then(lightcraft_preview::Hash128::parse);
crate::import_move::copy_new(Path::new(src), &dir, &name, expect).map(|p| p.to_string_lossy().to_string())
crate::import_move::copy_new(Path::new(src), dir, &name, expect).map(|p| p.to_string_lossy().to_string())
}
/// What a [`scan`] needs from the session, so it can run on another thread (a folder on a network
@@ -610,6 +704,50 @@ impl Prepared {
pub fn is_empty(&self) -> bool {
self.items.is_empty()
}
/// Recheck Add-only entries against the owner catalog immediately before commit.
/// Preparation may have run while another import added the same path or content.
pub fn revalidate_add(&mut self, s: &Session) {
let mut paths = HashMap::new();
let mut hashes = HashMap::new();
for photo in s.catalog.photos() {
if let Source::File { path } = &photo.source {
paths.insert(normalize_import_path(path), photo.id);
}
if let Some(hash) = &photo.content_hash {
hashes.insert(hash.clone(), photo.id);
}
}
let mut items = Vec::with_capacity(self.items.len());
for item in std::mem::take(&mut self.items) {
let PreparedItem::Ready(ready) = item else {
items.push(item);
continue;
};
let ReadyFile { path, stored, info, sidecar, placed } = *ready;
let existing_path = paths.get(&normalize_import_path(&stored)).copied().or_else(|| paths.get(&normalize_import_path(&path)).copied());
let existing_hash = info.content_hash.as_ref().and_then(|hash| hashes.get(hash).copied());
if let Some(existing) = existing_path.or(existing_hash) {
if let Some(placed) = &placed {
crate::import_move::rollback(placed);
}
items.push(PreparedItem::Duplicate {
path,
existing: Some(existing),
reason: if existing_path.is_some() { "path" } else { "content" },
hash: info.content_hash,
});
} else {
items.push(PreparedItem::Ready(Box::new(ReadyFile { path, stored, info, sidecar, placed })));
}
}
self.items = items;
}
}
fn normalize_import_path(path: &str) -> String {
let path = path.replace('\\', "/");
if cfg!(windows) { path.to_lowercase() } else { path }
}
impl ImportJob {
@@ -638,6 +776,10 @@ impl ImportJob {
let mut by_path = HashMap::new();
let mut by_hash = HashMap::new();
for p in s.catalog.photos() {
// a trashed photo is replaced by a fresh import, so it doesn't count as a duplicate
if p.deleted && opts.on_deleted == OnDeleted::Fresh {
continue;
}
if let Source::File { path } = &p.source {
by_path.insert(path.clone(), (p.id, p.local));
}
@@ -664,8 +806,9 @@ impl ImportJob {
}
/// The file-system half of importing `paths`: expand folders, skip known paths, probe, skip
/// duplicates by content, read sidecars, copy or place (Move) the files. Nothing in the
/// catalog changes. Stops before the next file once `cancel` is set.
/// duplicates by content, read sidecars, copy or place (Move) the files — several at once,
/// with the names and outcomes of one after another. Nothing in the catalog changes. Stops
/// before the next file once `cancel` is set.
pub fn prepare(&mut self, paths: &[String], cancel: &std::sync::atomic::AtomicBool) -> Prepared {
let files = self.expand(paths);
self.prepare_files(files, cancel)
@@ -715,24 +858,39 @@ impl ImportJob {
let probed: Vec<Result<ProbeInfo, String>> =
cached.into_iter().map(|c| c.map(Ok).unwrap_or_else(|| fresh.next().unwrap_or_else(|| Err("not probed".into())))).collect();
crate::memory::release();
// Files are planned one by one, in order (duplicates, sidecars, names, folders); their
// copies or moves — the slow part, reading and writing every byte and verifying it — are
// collected and run on several threads (issue #367), then their outcomes take their places.
let mut plan = Plan::default();
for (path, info) in todo.into_iter().zip(probed) {
if cancel.load(Relaxed) {
break;
}
self.done.fetch_add(1, Relaxed);
let mut info = match info {
Ok(i) => i,
Err(e) => {
log::warn!("import {path}: {e}");
out.items.push(PreparedItem::Failed(path, e));
self.done.fetch_add(1, Relaxed);
plan.slots.push(Slot::Item(PreparedItem::Failed(path, e)));
continue;
}
};
// a copy of a file whose transfer is still pending: a duplicate only if that one succeeds
if !opts.local
&& let Some(h) = &info.content_hash
&& plan.transfers.iter().any(|t| t.info.content_hash.as_ref() == Some(h))
{
self.run_plan(&mut plan, &mut out, cancel);
if cancel.load(Relaxed) {
break;
}
}
if let Some(h) = &info.content_hash
&& !opts.local
&& let Some(existing) = self.by_hash.get(h)
{
out.items.push(PreparedItem::Duplicate { path, existing: *existing, reason: "content", hash: Some(h.clone()) });
self.done.fetch_add(1, Relaxed);
plan.slots.push(Slot::Item(PreparedItem::Duplicate { path, existing: *existing, reason: "content", hash: Some(h.clone()) }));
continue;
}
// the XMP sidecar (or a raw's embedded XMP), read before copying: its capture time files
@@ -751,7 +909,6 @@ impl ImportJob {
if info.captured.is_none() {
info.captured = sidecar.as_ref().and_then(|sc| sc.captured.clone());
}
let mut placed = None;
let stored = match (mode, &self.copy_root) {
(ImportMode::Copy | ImportMode::Move, Some(root))
if !crate::import_move::inside(Path::new(&path), root)
@@ -768,37 +925,35 @@ impl ImportJob {
n
});
let folders = opts.organize.folders(&q);
if mode == ImportMode::Move && !crate::import_move::is_symlink(Path::new(&path)) {
let dir = folders.iter().fold(root.to_path_buf(), |d, f| d.join(f));
let name = name.unwrap_or(own);
match crate::import_move::place(Path::new(&path), &dir, &name) {
Ok(pl) => {
let dst = pl.dst.to_string_lossy().to_string();
placed = Some(pl);
dst
}
Err(e) => {
out.items.push(PreparedItem::Failed(path, e));
continue;
}
let dir = folders.iter().fold(root.to_path_buf(), |d, f| d.join(f));
// the name this file gets depends on what a pending transfer of a like-named file
// into the same folder ends up with: that one goes first
let as_named = name.as_deref().unwrap_or(if own.is_empty() { "photo" } else { own.as_str() });
let family = (dir.to_string_lossy().to_lowercase(), name_family(as_named));
if plan.transfers.iter().any(|t| t.family == family) {
self.run_plan(&mut plan, &mut out, cancel);
if cancel.load(Relaxed) {
break;
}
}
let how = if mode == ImportMode::Move && !crate::import_move::is_symlink(Path::new(&path)) {
How::Place { name: name.unwrap_or(own) }
} else {
if mode == ImportMode::Move {
let reason = "a symbolic link: the file it points to was copied, the link and its target stay";
out.items.push(PreparedItem::Kept(Kept { path: path.clone(), reason: reason.into() }));
plan.slots.push(Slot::Item(PreparedItem::Kept(Kept { path: path.clone(), reason: reason.into() })));
}
match copy_into(root, &path, &folders, name.as_deref(), info.content_hash.as_deref()) {
Ok(p) => p,
Err(e) => {
out.items.push(PreparedItem::Failed(path, e));
continue;
}
}
}
How::Copy { name }
};
// Copy as DNG: the copied raw becomes a DNG (the copy is ours to replace)
let dng = opts.convert_dng && mode == ImportMode::Copy && raw && !info.format.eq_ignore_ascii_case("DNG");
plan.slots.push(Slot::Transfer);
plan.transfers.push(Transfer { path, info, sidecar, dir, how, dng, family });
continue;
}
(ImportMode::Move, _) => {
let reason = "already inside the destination or the library: added where it is";
out.items.push(PreparedItem::Kept(Kept { path: path.clone(), reason: reason.into() }));
plan.slots.push(Slot::Item(PreparedItem::Kept(Kept { path: path.clone(), reason: reason.into() })));
path.clone()
}
_ => path.clone(),
@@ -806,27 +961,199 @@ impl ImportJob {
if let Some(h) = &info.content_hash {
self.by_hash.insert(h.clone(), None);
}
// Copy as DNG: the copied raw becomes a DNG (the copy is ours to replace)
let mut stored = stored;
if opts.convert_dng
&& mode == ImportMode::Copy
&& stored != path
&& info.kind == MediaKind::Raw
&& !info.format.eq_ignore_ascii_case("DNG")
{
match crate::cmd::convert::write_dng_with(self.file_bytes.as_ref(), &stored, String::new()) {
Ok(dng) => {
let _ = std::fs::remove_file(&stored);
stored = dng;
info.format = "DNG".into();
}
Err(e) => log::warn!("import {path}: copy as DNG: {e}"),
}
}
out.items.push(PreparedItem::Ready(Box::new(ReadyFile { path, stored, info, sidecar, placed })));
self.done.fetch_add(1, Relaxed);
plan.slots.push(Slot::Item(PreparedItem::Ready(Box::new(ReadyFile { path, stored, info, sidecar, placed: None }))));
}
self.run_plan(&mut plan, &mut out, cancel);
out
}
/// Run the transfers `plan` collected (on several threads) and append its items to `out` in
/// file order. Transfers not started because `cancel` was set leave no item.
fn run_plan(&mut self, plan: &mut Plan, out: &mut Prepared, cancel: &std::sync::atomic::AtomicBool) {
let mut done = run_transfers(std::mem::take(&mut plan.transfers), self.file_bytes.as_ref(), cancel, &self.done).into_iter();
for slot in std::mem::take(&mut plan.slots) {
let item = match slot {
Slot::Item(it) => it,
Slot::Transfer => match done.next().flatten() {
Some(it) => it,
None => continue,
},
};
if let PreparedItem::Ready(r) = &item
&& let Some(h) = &r.info.content_hash
{
self.by_hash.insert(h.clone(), None);
}
out.items.push(item);
}
}
}
/// A batch's items in file order while its transfers are pending ([`ImportJob::run_plan`]).
#[derive(Default)]
struct Plan {
slots: Vec<Slot>,
transfers: Vec<Transfer>,
}
enum Slot {
Item(PreparedItem),
/// The outcome of the next of [`Plan::transfers`].
Transfer,
}
/// A file to copy or move into `dir`, planned in order and run on any thread ([`run_transfer`]).
struct Transfer {
path: String,
info: ProbeInfo,
sidecar: Option<crate::sidecar::SidecarData>,
dir: PathBuf,
how: How,
/// Copy as DNG.
dng: bool,
/// The folder (lowercase) and [`name_family`]: transfers of one family can take each other's
/// names, so they never run at the same time.
family: (String, String),
}
enum How {
/// A verified copy, under `name` (default: the file's own).
Copy { name: Option<String> },
/// Move: placed (linked or copied) without touching the source.
Place { name: String },
}
/// The names a file called `name` could end up with, as one key: the transfers number a taken name
/// `IMG-1.JPG`, `IMG-2.JPG`…, and Copy as DNG names its DNG `IMG.dng`, `IMG-2.dng`…, so `IMG.JPG`,
/// `IMG-1.jpg` and `img.dng` are all `img` (case-insensitive: so are some file systems).
fn name_family(name: &str) -> String {
let mut stem = match name.rsplit_once('.') {
Some((s, _)) if !s.is_empty() => s,
_ => name,
};
while let Some((head, tail)) = stem.rsplit_once('-')
&& !head.is_empty()
&& !tail.is_empty()
&& tail.bytes().all(|b| b.is_ascii_digit())
{
stem = head;
}
stem.to_lowercase()
}
/// Threads for `jobs` file reads (probes, copies) at once.
#[cfg(not(target_arch = "wasm32"))]
fn workers(jobs: usize) -> usize {
std::thread::available_parallelism().map(|n| n.get()).unwrap_or(4).clamp(1, 8).min(jobs.max(1))
}
/// Files per batch for a caller that prepares an import in batches ([`ImportJob::prepare_files`]):
/// enough to keep every probe and copy thread busy (a batch waits for its slowest file), few
/// enough that each batch soon joins the catalog.
pub fn batch_size() -> usize {
#[cfg(not(target_arch = "wasm32"))]
{
workers(usize::MAX).saturating_mul(2).clamp(8, 16)
}
#[cfg(target_arch = "wasm32")]
{
8
}
}
/// Run `transfers` (several at once where threads exist); their outcomes in order, `None` for one
/// not started because `cancel` was set. `done` counts each one finished.
fn run_transfers(
transfers: Vec<Transfer>,
file_bytes: Option<&crate::merge::ByteReader>,
cancel: &std::sync::atomic::AtomicBool,
done: &std::sync::atomic::AtomicUsize,
) -> Vec<Option<PreparedItem>> {
use std::sync::atomic::Ordering::Relaxed;
let one = |t: Transfer| {
if cancel.load(Relaxed) {
return None;
}
let path = t.path.clone();
// last-resort guard: a panic is this file's failure, not the import's
let item = crate::guard::catch("import", || run_transfer(t, file_bytes)).unwrap_or_else(|e| PreparedItem::Failed(path, e));
done.fetch_add(1, Relaxed);
Some(item)
};
#[cfg(not(target_arch = "wasm32"))]
{
let n = workers(transfers.len());
if n > 1 {
let mut results: Vec<Option<PreparedItem>> = std::iter::repeat_with(|| None).take(transfers.len()).collect();
let queue = std::sync::Mutex::new(transfers.into_iter().enumerate());
let out = std::sync::Mutex::new(&mut results);
// (the tests' injected faults hold for the workers too)
#[cfg(test)]
let fault = crate::import_move::injected_fault();
let work = &|| {
#[cfg(test)]
crate::import_move::inject(fault);
loop {
let next = queue.lock().unwrap_or_else(std::sync::PoisonError::into_inner).next();
let Some((i, t)) = next else { break };
let r = one(t);
if let Some(slot) = out.lock().unwrap_or_else(std::sync::PoisonError::into_inner).get_mut(i) {
*slot = r;
}
}
};
std::thread::scope(|sc| {
let handles: Vec<_> =
(0..n).filter_map(|i| std::thread::Builder::new().name(format!("lc-import-copy-{i}")).spawn_scoped(sc, work).ok()).collect();
if handles.is_empty() {
// no thread to be had: copy here, one after another
work();
}
for h in handles {
if h.join().is_err() {
log::error!("import: a copy worker stopped unexpectedly");
}
}
});
return results;
}
}
transfers.into_iter().map(one).collect()
}
/// Copy or move one planned file (and Copy as DNG); its item.
fn run_transfer(t: Transfer, file_bytes: Option<&crate::merge::ByteReader>) -> PreparedItem {
let Transfer { path, mut info, sidecar, dir, how, dng, .. } = t;
let mut placed = None;
let mut stored = match how {
How::Place { name } => match crate::import_move::place(Path::new(&path), &dir, &name) {
Ok(pl) => {
let dst = pl.dst.to_string_lossy().to_string();
placed = Some(pl);
dst
}
Err(e) => return PreparedItem::Failed(path, e),
},
How::Copy { name } => match copy_into(&dir, &path, name.as_deref(), info.content_hash.as_deref()) {
Ok(p) => p,
Err(e) => return PreparedItem::Failed(path, e),
},
};
if dng {
// decoding the raw twice and encoding it takes several times the file's size in memory
let weight = usize::try_from(info.file_size).unwrap_or(usize::MAX).saturating_mul(6);
let _permit = crate::memory::work_gate().acquire(weight);
match crate::cmd::convert::write_dng_with(file_bytes, &stored, String::new()) {
Ok(dng) => {
let _ = std::fs::remove_file(&stored);
stored = dng;
info.format = "DNG".into();
}
Err(e) => log::warn!("import {path}: copy as DNG: {e}"),
}
}
PreparedItem::Ready(Box::new(ReadyFile { path, stored, info, sidecar, placed }))
}
/// The catalog half of an import: add the photos [`ImportJob::prepare`] readied (one undoable op;
@@ -840,6 +1167,20 @@ pub fn commit_prepared(s: &mut Session, opts: &ImportOptions, now: &str, prepare
let mut placed: Vec<crate::import_move::Placed> = Vec::new();
// photos added by this batch, by content (a duplicate of a file earlier in the import names it)
let mut new_hash: HashMap<String, PhotoId> = HashMap::new();
// Fresh: trashed photos a new import replaces (by content or by file), each removed once
let (mut trash_hash, mut trash_path): (HashMap<String, PhotoId>, HashMap<String, PhotoId>) = Default::default();
if opts.on_deleted == OnDeleted::Fresh {
for p in s.catalog.photos().filter(|p| p.deleted) {
if let Some(h) = &p.content_hash {
trash_hash.insert(h.clone(), p.id);
}
if let Source::File { path } = &p.source {
trash_path.insert(path.clone(), p.id);
}
}
}
let mut purge: Vec<PhotoId> = Vec::new();
let mut purged: std::collections::HashSet<PhotoId> = Default::default();
for it in prepared.items {
match it {
PreparedItem::Promote(id) => {
@@ -853,7 +1194,16 @@ pub fn commit_prepared(s: &mut Session, opts: &ImportOptions, now: &str, prepare
let h = hash?;
new_hash.get(&h).copied().or_else(|| s.catalog.photos().find(|p| p.content_hash.as_deref() == Some(h.as_str())).map(|p| p.id))
});
report.duplicates.push(Duplicate { path, existing: existing.map(|i| i.0), reason });
let trashed = existing.filter(|id| s.catalog.photo(*id).is_some_and(|p| p.deleted));
match trashed {
Some(id) if opts.on_deleted == OnDeleted::Restore => {
if !report.restored.contains(&id.0) {
ops.push(Op::SetDeleted { id, deleted: false });
report.restored.push(id.0);
}
}
_ => report.duplicates.push(Duplicate { path, existing: existing.map(|i| i.0), reason, existing_deleted: trashed.is_some() }),
}
}
PreparedItem::Failed(path, e) => report.failed.push((path, e)),
PreparedItem::Kept(k) => report.kept.push(k),
@@ -864,6 +1214,11 @@ pub fn commit_prepared(s: &mut Session, opts: &ImportOptions, now: &str, prepare
if let Some(h) = &info.content_hash {
new_hash.insert(h.clone(), id);
}
for old in info.content_hash.as_ref().and_then(|h| trash_hash.get(h)).into_iter().chain(trash_path.get(&stored)) {
if purged.insert(*old) {
purge.push(*old);
}
}
let sidecar = sidecar.map(|sc| sc.resolve_label(&s.catalog)).filter(|sc| *sc != crate::sidecar::SidecarData::default());
// a copy is catalogued under its new name
let name = Path::new(&stored).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| path.clone());
@@ -907,9 +1262,18 @@ pub fn commit_prepared(s: &mut Session, opts: &ImportOptions, now: &str, prepare
}
}
}
// (built against the catalog as it is: one op list for all of them, ahead of the additions)
if !purge.is_empty() {
ops.insert(0, s.catalog.delete_photos_permanently_ops(&purge));
}
let log0 = s.pending_log.len();
let (verb, n) = if report.imported.is_empty() && !report.restored.is_empty() {
("Restore", report.restored.len())
} else {
("Add", report.imported.len().max(1))
};
if !ops.is_empty()
&& let Err(e) = s.commit(&format!("Add {} Photo{}", ops.len(), if ops.len() == 1 { "" } else { "s" }), Op::Batch { ops })
&& let Err(e) = s.commit(&format!("{verb} {n} Photo{}", if n == 1 { "" } else { "s" }), Op::Batch { ops })
{
// nothing was catalogued: take the moves back (their sources were never touched)
placed.iter().for_each(crate::import_move::rollback);
@@ -1021,3 +1385,31 @@ impl Session {
self
}
}
#[cfg(test)]
mod prepared_tests {
use super::*;
#[test]
fn revalidate_add_only_converts_matching_ready_item() {
let mut s = Session::new();
let existing_path = std::env::temp_dir().join("lightcraft-revalidate-existing.jpg");
let other_path = std::env::temp_dir().join("lightcraft-revalidate-other.jpg");
let existing = Photo::new(PhotoId(1), Source::File { path: existing_path.to_string_lossy().into() }, "existing.jpg", "JPG", 1, 1, "now");
s.commit("existing", Op::AddPhoto { photo: Box::new(existing) }).unwrap();
let ready = |path: &Path| ReadyFile {
path: path.to_string_lossy().into(),
stored: path.to_string_lossy().into(),
info: ProbeInfo { format: "JPG".into(), ..ProbeInfo::default() },
sidecar: None,
placed: None,
};
let mut prepared = Prepared {
scanned: 2,
items: vec![PreparedItem::Ready(Box::new(ready(&existing_path))), PreparedItem::Ready(Box::new(ready(&other_path)))],
};
prepared.revalidate_add(&s);
assert!(matches!(prepared.items.first(), Some(PreparedItem::Duplicate { reason: "path", .. })));
assert!(matches!(prepared.items.get(1), Some(PreparedItem::Ready(_))));
}
}
+23
View File
@@ -56,6 +56,9 @@ pub(crate) enum Fault {
Corrupt,
/// Removing the source fails (as on a read-only card).
FailRemove,
/// Each copy takes a little longer and notes the thread it ran on ([`copy_threads`]), to see
/// an import's copies run side by side.
Slow,
}
#[cfg(test)]
@@ -69,6 +72,21 @@ pub(crate) fn inject(f: Fault) {
FAULT.with(|c| c.set(f));
}
/// The calling thread's injected fault (tests only), for the workers it starts to inherit.
#[cfg(test)]
pub(crate) fn injected_fault() -> Fault {
FAULT.with(|c| c.get())
}
/// The threads copies made under [`Fault::Slow`] ran on (tests only).
#[cfg(test)]
static COPY_THREADS: std::sync::Mutex<Vec<std::thread::ThreadId>> = std::sync::Mutex::new(Vec::new());
#[cfg(test)]
pub(crate) fn copy_threads() -> Vec<std::thread::ThreadId> {
COPY_THREADS.lock().unwrap_or_else(std::sync::PoisonError::into_inner).clone()
}
fn injected(f: Fault) -> bool {
#[cfg(test)]
{
@@ -284,6 +302,11 @@ pub(crate) fn copy_verified(src: &Path, dst: &Path, expect: Option<lightcraft_pr
return Err(io::Error::other("write failed (injected)"));
}
io::copy(&mut inp, &mut out)?;
#[cfg(test)]
if injected(Fault::Slow) {
COPY_THREADS.lock().unwrap_or_else(std::sync::PoisonError::into_inner).push(std::thread::current().id());
std::thread::sleep(std::time::Duration::from_millis(30));
}
if injected(Fault::Corrupt) {
out.write_all(b"!")?;
}
+81 -3
View File
@@ -25,6 +25,11 @@ pub mod guard;
pub mod import;
mod import_move;
pub mod library;
mod lightroom_archive;
pub mod lightroom_catalog;
pub mod lightroom_job;
mod lightroom_sqlite;
pub mod logging;
pub mod media;
pub mod memory;
pub mod merge;
@@ -37,6 +42,7 @@ pub mod segment;
pub mod sidecar;
pub mod smart;
mod view;
pub mod walk;
use std::sync::Arc;
@@ -46,7 +52,7 @@ use lightcraft_catalog::{Catalog, Filter, Op, PhotoId, Sort};
use lightcraft_develop::DevelopSettings;
pub use media::{RenderJob, SourceLevel};
use serde_json::Value;
pub use view::{Browse, FilterChip, LibrarySource, Selection, filter_chips};
pub use view::{Browse, FilterChip, LibrarySource, Selection, SelectionState, filter_chips};
pub use {lightcraft_catalog as catalog, lightcraft_develop as develop, lightcraft_gpu as gpu, lightcraft_pipeline as pipeline};
#[derive(Debug, thiserror::Error)]
@@ -82,6 +88,27 @@ pub struct UndoEntry {
pub folder: Option<FolderMove>,
}
/// The one photo an undo step changes, when it changes exactly one through its edit, rating, flag,
/// label, metadata or versions (`depth` bounds nested batches).
fn single_photo(op: &Op, depth: usize) -> Option<PhotoId> {
match op {
Op::SetDevelop { id, .. }
| Op::SetRating { id, .. }
| Op::SetFlag { id, .. }
| Op::SetLabel { id, .. }
| Op::SetMeta { id, .. }
| Op::SetVersions { id, .. }
| Op::SetHistory { id, .. }
| Op::PushHistory { id, .. } => Some(*id),
Op::Batch { ops } if depth < 8 => {
let mut ids = ops.iter().map(|o| single_photo(o, depth + 1));
let first = ids.next()??;
ids.all(|id| id == Some(first)).then_some(first)
}
_ => None,
}
}
/// A folder renamed or moved on disk as part of an undo step.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct FolderMove {
@@ -134,6 +161,9 @@ pub struct Session {
pub clipboard: Option<Value>,
/// The folder on disk the [`LibrarySource::Folder`] view browses.
pub browse: Option<Browse>,
/// The folder the [`LibrarySource::LibraryFolder`] view shows: the library's photos imported
/// from it and from the folders inside it.
pub library_folder: Option<String>,
/// Copied metadata (`photo.copyMetadata`): photo.setMeta params.
pub meta_clipboard: Option<Value>,
/// The photo that was active before the current one (Paste Settings from Previous).
@@ -161,6 +191,8 @@ pub struct Session {
pub active_spot: Option<usize>,
/// The persistent library this session writes to (`None` = in-memory only).
pub library: Option<library::Library>,
/// Identity of current library opening; refreshed only after a successful open.
pub(crate) library_identity: Arc<()>,
/// XMP sidecar preferences (persisted with the library).
pub xmp: sidecar::XmpPrefs,
/// Parameters of the last export (`app.export` params, minus targets), persisted in prefs.json.
@@ -234,6 +266,7 @@ impl Session {
clipboard: None,
meta_clipboard: None,
browse: None,
library_folder: None,
previous_active: None,
copy_groups: lightcraft_develop::SettingsGroup::default_copy(),
presets: presets::builtin(),
@@ -248,6 +281,7 @@ impl Session {
segmenter: segment::Segmenter::default(),
active_spot: None,
library: None,
library_identity: Arc::new(()),
xmp: sidecar::XmpPrefs::default(),
last_export: None,
export_presets: Vec::new(),
@@ -423,6 +457,7 @@ impl Session {
return Err(err);
}
};
self.show_undone(&e.op);
self.pending_log.push(e.op);
self.redo.push(UndoEntry { label: e.label.clone(), op: redo, folder: e.folder.as_ref().map(FolderMove::reversed) });
Ok(e.label)
@@ -437,11 +472,30 @@ impl Session {
return Err(err);
}
};
self.show_undone(&e.op);
self.pending_log.push(e.op);
self.undo.push(UndoEntry { label: e.label.clone(), op: undo, folder: e.folder.as_ref().map(FolderMove::reversed) });
Ok(e.label)
}
/// After an undo or redo that changed one photo (its edit, rating, flag, label, metadata or
/// versions), that photo becomes the active one, as in Lightroom Classic, so the change is on
/// screen (issue #293). It joins the selection if it isn't in it; nothing changes for steps
/// that touch several photos, albums or files, or for a photo not in the current view.
fn show_undone(&mut self, op: &Op) {
let Some(id) = single_photo(op, 0) else { return };
if self.selection.active == Some(id) || !self.visible().contains(&id) {
return;
}
if !self.selection.contains(id) {
self.selection = Selection { ids: vec![id], active: Some(id) };
} else {
self.selection.active = Some(id);
}
self.active_mask = None;
self.active_spot = None;
}
/// Apply an undo/redo op, first renaming the step's folder and moving the files its renames
/// imply (all or nothing). Files that can't be moved back after a failure are reported and the
/// library follows them (a logged change outside the undo history), so none goes missing; the
@@ -590,7 +644,8 @@ impl Session {
/// Photos shown in the grid/filmstrip for the current source, filter and sort.
pub fn visible(&mut self) -> &[PhotoId] {
let mut key = (self.catalog.revision, format!("{:?}|{:?}|{:?}|{:?}", self.source, self.filter, self.sort, self.browse));
let mut key =
(self.catalog.revision, format!("{:?}|{:?}|{:?}|{:?}|{:?}", self.source, self.filter, self.sort, self.browse, self.library_folder));
if self.source == LibrarySource::Missing && self.media.availability.is_background() {
// the view fills in as the background checks find files gone
key.1.push_str(&format!("|{}", self.media.availability.generation()));
@@ -605,7 +660,18 @@ impl Session {
f.folder = Some(b.path);
f.subfolders = b.subfolders;
}
if self.source == LibrarySource::LibraryFolder {
// no folder chosen: nothing (`.` names no folder)
f.library_folder = Some(self.library_folder.clone().unwrap_or_else(|| ".".into()));
}
let mut visible = self.catalog.query(&f, &self.sort);
if visible.is_empty() && self.source == LibrarySource::LibraryFolder && !self.folder_holds_photos() {
// the shown folder lost its last photo (deleted, moved, removed): everything, not
// an empty grid under the name of a folder that is gone from the sidebar
self.source = LibrarySource::All;
self.library_folder = None;
return self.visible();
}
if matches!(self.source, LibrarySource::Album(_))
&& self.sort.key == lightcraft_catalog::SortKey::CaptureDate
&& let LibrarySource::Album(a) = self.source
@@ -642,13 +708,20 @@ impl Session {
&self.visible
}
/// Whether the library still holds a photo imported from the folder a `LibraryFolder` source
/// shows.
fn folder_holds_photos(&self) -> bool {
let f = Filter { library_folder: self.library_folder.clone(), ..Default::default() };
self.library_folder.is_some() && !self.catalog.query(&f, &Sort::default()).is_empty()
}
/// Photos in the current source (folder, album, …) before the filter bar and search narrow
/// them; `None` where that is not a separate number (Missing Photos).
pub fn source_total(&mut self) -> Option<usize> {
if self.source == LibrarySource::Missing {
return None;
}
let key = (self.catalog.revision, format!("{:?}|{:?}", self.source, self.browse));
let key = (self.catalog.revision, format!("{:?}|{:?}|{:?}", self.source, self.browse, self.library_folder));
if self.total.as_ref().map(|t| &t.0) != Some(&key) {
let mut f = self.source.to_filter(&Filter::default(), &self.catalog);
if self.source == LibrarySource::Folder {
@@ -656,6 +729,9 @@ impl Session {
f.folder = Some(b.path);
f.subfolders = b.subfolders;
}
if self.source == LibrarySource::LibraryFolder {
f.library_folder = Some(self.library_folder.clone().unwrap_or_else(|| ".".into()));
}
let n = self.catalog.query(&f, &self.sort).len();
self.total = Some((key, n));
}
@@ -709,6 +785,8 @@ mod tests_color;
#[cfg(test)]
mod tests_export;
#[cfg(test)]
mod tests_folders;
#[cfg(test)]
mod tests_forget_local;
#[cfg(test)]
mod tests_import;
+56 -18
View File
@@ -102,6 +102,8 @@ struct PresetsFile {
struct ViewFile {
source: LibrarySource,
browse: Option<crate::Browse>,
/// The folder a `libraryFolder` source shows.
library_folder: Option<String>,
// No filter: a library opens unfiltered. A date, keyword or person left over from the last session
// would silently hide photos, with only a small badge to say so.
sort: lightcraft_catalog::Sort,
@@ -309,25 +311,12 @@ impl Session {
self.pending_log.clear();
self.selection = Selection::default();
self.source = LibrarySource::All;
self.library_folder = None;
if report.created && seed_demo {
crate::demo::load(self);
journal.snapshot(&self.catalog)?;
}
let mut settings = SettingsLoad::default();
// presets
if let Some(f) = settings.read::<PresetsFile>(files.as_mut(), "presets.json") {
for p in &mut self.presets {
p.favorite = p.builtin && f.favorites.contains(&p.id) || (!p.builtin && p.favorite);
}
for u in f.user {
if !self.presets.iter().any(|p| p.id == u.id) {
self.presets.push(u);
}
}
let known = |id: &String| crate::presets::profile(id).is_some();
self.profile_favorites = f.profile_favorites.into_iter().filter(known).collect();
self.profile_recent = f.profile_recent.into_iter().filter(known).take(crate::presets::RECENT_PROFILES).collect();
}
// preferences
let prefs = settings.read::<PrefsFile>(files.as_mut(), "prefs.json").unwrap_or_default();
self.xmp = prefs.xmp;
@@ -349,10 +338,34 @@ impl Session {
if let Some(d) = &self.smart_previews_dir {
self.media.smart_dir = Some(d.clone());
}
// presets
if let Some(f) = settings.read::<PresetsFile>(files.as_mut(), "presets.json") {
for p in &mut self.presets {
p.favorite = p.builtin && f.favorites.contains(&p.id) || (!p.builtin && p.favorite);
}
for u in f.user {
if !self.presets.iter().any(|p| p.id == u.id) {
self.presets.push(u);
}
}
let known = |id: &String| crate::presets::profile(id).is_some() || self.lut_profiles.iter().any(|p| &p.id == id);
self.profile_favorites = f.profile_favorites.into_iter().filter(known).collect();
self.profile_recent = f.profile_recent.into_iter().filter(known).take(crate::presets::RECENT_PROFILES).collect();
}
// view state
if let Some(v) = settings.read::<ViewFile>(files.as_mut(), "view.json") {
self.source = v.source;
self.browse = v.browse;
self.library_folder = v.library_folder.filter(|f| !f.trim().is_empty());
if self.source == LibrarySource::LibraryFolder {
// a folder that is gone (or none): everything, not an empty grid
let f = lightcraft_catalog::Filter { library_folder: self.library_folder.clone(), ..Default::default() };
let any = self.library_folder.is_some() && !self.catalog.query(&f, &lightcraft_catalog::Sort::default()).is_empty();
if !any {
self.source = LibrarySource::All;
self.library_folder = None;
}
}
self.sort = v.sort;
self.selection = v.selection;
self.selection.ids.retain(|id| self.catalog.photo(*id).is_some());
@@ -382,7 +395,7 @@ impl Session {
forgot_local: None,
presets_written,
view_written,
settings_warnings: settings.warnings,
settings_warnings: unlocked_warning(lock.as_ref()).into_iter().chain(settings.warnings).collect(),
warnings_reported: 0,
blocked: settings.blocked,
lock: lock.take(),
@@ -399,6 +412,7 @@ impl Session {
lib.forgot_local = Some(plan);
}
}
self.library_identity = std::sync::Arc::new(());
Ok(())
}
@@ -491,16 +505,27 @@ impl Session {
lib.retry_at = None;
lib.last_error = None;
}
match (persisted, snapshot) {
let result = match (persisted, snapshot) {
(_, Ok(())) => Ok(()),
// neither the log nor the snapshot took the queued ops
(Err(EngineError::NotSaved(e)), Err(s)) if lib.journal.seq() == before => Err(EngineError::NotSaved(format!("{e}; snapshot: {s}"))),
(_, Err(s)) => Err(s.into()),
};
if result.is_ok() {
self.library_identity = std::sync::Arc::new(());
}
result
}
fn view_json(&self) -> Vec<u8> {
let view = ViewFile { source: self.source, browse: self.browse.clone(), sort: self.sort, selection: self.selection.clone() };
let view = ViewFile {
source: self.source,
browse: self.browse.clone(),
// only while it is shown: a leftover would be a stale choice nobody made
library_folder: self.library_folder.clone().filter(|_| self.source == LibrarySource::LibraryFolder),
sort: self.sort,
selection: self.selection.clone(),
};
serde_json::to_vec_pretty(&view).unwrap_or_default()
}
@@ -550,7 +575,8 @@ impl Session {
lib.files.write_atomic("prefs.json", &v).map_err(|e| EngineError::Other(format!("prefs: {e}")))
}
/// Settings-file warnings of the open library not handed out yet (the UI shows each once).
/// Warnings about the open library not handed out yet: its settings files, and a lock that
/// couldn't be taken (the UI shows each once; the CLI and MCP print them on stderr).
pub fn take_library_warnings(&mut self) -> Vec<String> {
let Some(lib) = self.library.as_mut() else { return vec![] };
let new = lib.settings_warnings[lib.warnings_reported..].to_vec();
@@ -605,3 +631,15 @@ impl Session {
Ok(())
}
}
/// The warning for a library opened without its one-program-at-a-time lock (issue #171): its
/// file system can't lock `catalog.lock` (some network shares). It opens anyway, as documented —
/// refusing would lock the user out — but the user must learn that a second program could open it.
fn unlocked_warning(lock: Option<&LibraryLock>) -> Option<String> {
lock.filter(|l| !l.held()).map(|_| {
"This library could not be locked (its catalog.lock file can't be locked where it is stored, e.g. on some network \
shares), so it is open without protection against a second program: use it in one LightCraft app or command at \
a time, or changes made in one of them can be lost."
.to_string()
})
}
+685
View File
@@ -0,0 +1,685 @@
//! Bounded, deduplicated persistence for Lightroom import source data.
//!
//! Archives use a deterministic JSON representation wrapped in zlib. The JSON
//! writer streams through a bounded compressor, so an import cannot first build
//! an unbounded archive buffer. Blob columns are omitted by the catalog reader;
//! decoded XMP,
//! settings, history, snapshots, identities, paths, collections & membership
//! remain available for recovery.
use std::collections::BTreeMap;
use std::fs::{self, File};
use std::io::{self, Read, Write};
use std::path::{Path, PathBuf};
use miniz_oxide::deflate::{
core::{CompressorOxide, create_comp_flags_from_zip_params},
stream::deflate,
};
use miniz_oxide::{MZFlush, MZStatus};
use serde::ser::{Serialize, SerializeMap, SerializeSeq, SerializeStruct, Serializer};
use serde_json::Value;
use crate::lightroom_catalog::{CatalogImport, CatalogPhoto};
/// Maximum size of one complete archive on disk, including format header.
pub const MAX_ARCHIVE_BYTES: u64 = 32 * 1024 * 1024;
/// Maximum total size retained in managed archives.
pub const MAX_TOTAL_ARCHIVE_BYTES: u64 = 128 * 1024 * 1024;
/// Maximum number of managed archives retained.
pub const MAX_MANAGED_ARCHIVES: usize = 8;
const FORMAT_VERSION: u8 = 1;
const HEADER: &[u8] = b"LC-LRARCH\0\x01Z";
const NAME_PREFIX: &str = "lightroom-import-";
const NAME_SUFFIX: &str = ".lca";
const MAX_SCAN: usize = 1024;
const IO_BUFFER: usize = 16 * 1024;
/// Store compact source data below `directory` and return its managed archive.
///
/// The operation is safe to call before catalog mutation. It rejects a
/// symlink directory, writes only `lightroom-import-<number>.lca`, compares the
/// complete bounded byte stream with recent managed archives, and retains at
/// most [`MAX_MANAGED_ARCHIVES`] and [`MAX_TOTAL_ARCHIVE_BYTES`]. Legacy
/// `lightroom-import-*.json` files are deliberately ignored and preserved for
/// an explicit caller warning.
pub fn store(directory: &Path, data: &CatalogImport) -> Result<PathBuf, String> {
prepare_directory(directory)?;
let mut managed = managed_files(directory)?;
managed.sort_by_key(|entry| entry.index);
let temporary = temporary_path(directory)?;
write_archive(&temporary, data).inspect_err(|_| {
let _ = remove_managed_file(&temporary);
})?;
let recent = managed.iter().rev().take(MAX_MANAGED_ARCHIVES);
for entry in recent {
let same = match same_bounded(&temporary, &entry.path) {
Ok(same) => same,
Err(error) => {
let _ = remove_managed_file(&temporary);
return Err(error.to_string());
}
};
if same {
remove_managed_file(&temporary).map_err(|e| e.to_string())?;
let matched = entry.path.clone();
let mut all = managed.clone();
prune_existing(directory, &mut all, Some(&matched))?;
return Ok(matched);
}
}
let index = managed.last().map_or(0, |entry| entry.index.saturating_add(1));
if managed.last().is_some_and(|entry| entry.index == u64::MAX) {
let _ = remove_managed_file(&temporary);
return Err("managed Lightroom archive index is exhausted".into());
}
let destination = match next_destination(directory, index) {
Ok(path) => path,
Err(error) => {
let _ = remove_managed_file(&temporary);
return Err(error);
}
};
let size = match fs::metadata(&temporary) {
Ok(meta) => meta.len(),
Err(error) => {
let _ = remove_managed_file(&temporary);
return Err(error.to_string());
}
};
let new_entry = Managed { index, path: destination.clone(), size };
let mut after = managed;
after.push(new_entry);
after.sort_by_key(|entry| entry.index);
let retirees = match retirement_set(&after) {
Ok(retirees) => retirees,
Err(error) => {
let _ = remove_managed_file(&temporary);
return Err(error);
}
};
let quarantined = match quarantine(&retirees) {
Ok(quarantined) => quarantined,
Err(error) => {
let _ = remove_managed_file(&temporary);
return Err(error);
}
};
if let Err(error) = publish_new(&temporary, &destination) {
rollback_quarantine(&quarantined);
let _ = remove_managed_file(&temporary);
return Err(error.to_string());
}
if let Err(error) = remove_managed_file(&temporary) {
return Err(format!("archive published but temporary cleanup failed: {error}"));
}
for (old, held) in quarantined {
if let Err(error) = remove_managed_file(&held) {
return Err(format!("archive published but could not retire {}: {error}", old.display()));
}
}
Ok(destination)
}
/// Store source data when possible. An archive that reaches its bounded size is
/// omitted with a warning so catalog import can still complete.
pub fn store_best_effort(directory: &Path, data: &CatalogImport) -> Result<Option<PathBuf>, String> {
match store(directory, data) {
Ok(path) => Ok(Some(path)),
Err(error) if error.contains("Lightroom archive exceeds 32 MiB") => Ok(None),
Err(error) => Err(error),
}
}
#[derive(Clone, Debug)]
struct Managed {
index: u64,
path: PathBuf,
size: u64,
}
fn prepare_directory(directory: &Path) -> Result<(), String> {
match fs::symlink_metadata(directory) {
Ok(meta) => {
if meta.file_type().is_symlink() || !meta.is_dir() {
return Err(format!("archive directory is not a real directory: {}", directory.display()));
}
}
Err(error) if error.kind() == io::ErrorKind::NotFound => {
fs::create_dir_all(directory).map_err(|e| e.to_string())?;
let meta = fs::symlink_metadata(directory).map_err(|e| e.to_string())?;
if meta.file_type().is_symlink() || !meta.is_dir() {
return Err(format!("archive directory is not a real directory: {}", directory.display()));
}
}
Err(error) => return Err(error.to_string()),
}
Ok(())
}
fn managed_files(directory: &Path) -> Result<Vec<Managed>, String> {
let mut files = Vec::new();
for (seen, result) in fs::read_dir(directory).map_err(|e| e.to_string())?.enumerate() {
if seen >= MAX_SCAN {
return Err(format!("too many entries in archive directory (limit {MAX_SCAN})"));
}
let entry = result.map_err(|e| e.to_string())?;
let Some(index) = parse_managed_name(&entry.file_name()) else { continue };
let meta = fs::symlink_metadata(entry.path()).map_err(|e| e.to_string())?;
if meta.file_type().is_symlink() || !meta.is_file() {
continue;
}
files.push(Managed { index, path: entry.path(), size: meta.len() });
}
Ok(files)
}
fn parse_managed_name(name: &std::ffi::OsStr) -> Option<u64> {
let name = name.to_str()?;
let digits = name.strip_prefix(NAME_PREFIX)?.strip_suffix(NAME_SUFFIX)?;
if digits.is_empty() || !digits.bytes().all(|byte| byte.is_ascii_digit()) {
return None;
}
digits.parse().ok()
}
fn temporary_path(directory: &Path) -> Result<PathBuf, String> {
let pid = std::process::id();
for slot in 0..MAX_SCAN {
let path = directory.join(format!(".lightroom-import-{pid}-{slot}.tmp"));
match fs::symlink_metadata(&path) {
Ok(_) => continue,
Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(path),
Err(error) => return Err(error.to_string()),
}
}
Err("no safe temporary archive name available".into())
}
fn next_destination(directory: &Path, mut index: u64) -> Result<PathBuf, String> {
for _ in 0..MAX_SCAN {
let path = directory.join(format!("{NAME_PREFIX}{index}{NAME_SUFFIX}"));
match fs::symlink_metadata(&path) {
Ok(_) => index = index.checked_add(1).ok_or("managed archive index is exhausted")?,
Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(path),
Err(error) => return Err(error.to_string()),
}
}
Err("no safe managed archive name available".into())
}
fn write_archive(path: &Path, data: &CatalogImport) -> Result<(), String> {
lightcraft_catalog::safe_file::write_atomic_with(path, &mut |raw| {
let mut bounded = BoundedWriter { inner: raw, total: 0 };
bounded.write_all(HEADER)?;
let mut compressor = ZlibWriter::new(&mut bounded);
serde_json::to_writer(&mut compressor, &ArchiveRef(data)).map_err(json_io)?;
compressor.finish()?;
Ok(())
})
.map(|_| ())
.map_err(|error| format!("could not write Lightroom archive: {error}"))
}
fn json_io(error: serde_json::Error) -> io::Error {
io::Error::new(io::ErrorKind::InvalidData, error.to_string())
}
struct BoundedWriter<'a> {
inner: &'a mut dyn Write,
total: u64,
}
impl Write for BoundedWriter<'_> {
fn write(&mut self, bytes: &[u8]) -> io::Result<usize> {
let next = self.total.checked_add(bytes.len() as u64).ok_or_else(limit_error)?;
if next > MAX_ARCHIVE_BYTES {
return Err(limit_error());
}
self.inner.write_all(bytes)?;
self.total = next;
Ok(bytes.len())
}
fn flush(&mut self) -> io::Result<()> {
self.inner.flush()
}
}
fn limit_error() -> io::Error {
io::Error::new(io::ErrorKind::InvalidData, "Lightroom archive exceeds 32 MiB")
}
struct ZlibWriter<'a, 'b> {
sink: &'a mut BoundedWriter<'b>,
compressor: CompressorOxide,
plain: u64,
}
impl<'a, 'b> ZlibWriter<'a, 'b> {
fn new(sink: &'a mut BoundedWriter<'b>) -> ZlibWriter<'a, 'b> {
let flags = create_comp_flags_from_zip_params(6, 15, 0);
ZlibWriter { sink, compressor: CompressorOxide::new(flags), plain: 0 }
}
fn run(&mut self, input: &[u8], flush: MZFlush) -> io::Result<()> {
let mut rest = input;
loop {
let mut output = [0u8; IO_BUFFER];
let result = deflate(&mut self.compressor, rest, &mut output, flush);
self.sink.write_all(&output[..result.bytes_written])?;
match result.status {
Ok(MZStatus::StreamEnd) => return Ok(()),
Ok(MZStatus::Ok) => {
let consumed = result.bytes_consumed;
if consumed > rest.len() {
return Err(io::Error::new(io::ErrorKind::InvalidData, "invalid compressor progress"));
}
rest = &rest[consumed..];
if rest.is_empty() && flush == MZFlush::None {
return Ok(());
}
if consumed == 0 && result.bytes_written == 0 {
return Err(io::Error::new(io::ErrorKind::InvalidData, "compressor made no progress"));
}
}
Ok(status) => {
return Err(io::Error::new(io::ErrorKind::InvalidData, format!("unsupported compressor status: {status:?}")));
}
Err(error) => return Err(io::Error::new(io::ErrorKind::InvalidData, format!("compression failed: {error:?}"))),
}
}
}
fn finish(&mut self) -> io::Result<()> {
self.run(&[], MZFlush::Finish)
}
}
impl Write for ZlibWriter<'_, '_> {
fn write(&mut self, bytes: &[u8]) -> io::Result<usize> {
if bytes.is_empty() {
return Ok(0);
}
let next = self.plain.checked_add(bytes.len() as u64).ok_or_else(limit_error)?;
if next > MAX_ARCHIVE_BYTES {
return Err(limit_error());
}
self.run(bytes, MZFlush::None)?;
self.plain = next;
Ok(bytes.len())
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
fn same_bounded(left: &Path, right: &Path) -> io::Result<bool> {
let left_meta = fs::symlink_metadata(left)?;
let right_meta = fs::symlink_metadata(right)?;
if left_meta.file_type().is_symlink() || right_meta.file_type().is_symlink() || !left_meta.is_file() || !right_meta.is_file() {
return Ok(false);
}
let left_len = left_meta.len();
let right_len = right_meta.len();
if left_len != right_len || left_len > MAX_ARCHIVE_BYTES {
return Ok(false);
}
let mut left = File::open(left)?;
let mut right = File::open(right)?;
let mut left_buf = [0u8; IO_BUFFER];
let mut right_buf = [0u8; IO_BUFFER];
loop {
let left_n = left.read(&mut left_buf)?;
let right_n = right.read(&mut right_buf)?;
if left_n != right_n {
return Ok(false);
}
if left_n == 0 {
return Ok(true);
}
if left_buf[..left_n] != right_buf[..right_n] {
return Ok(false);
}
}
}
fn retirement_set(entries: &[Managed]) -> Result<Vec<Managed>, String> {
retirement_set_protected(entries, None)
}
fn retirement_set_protected(entries: &[Managed], protected: Option<&Path>) -> Result<Vec<Managed>, String> {
let mut ordered = entries.to_vec();
ordered.sort_by_key(|entry| entry.index);
let mut total = ordered.iter().try_fold(0u64, |sum, entry| sum.checked_add(entry.size)).ok_or("archive size overflow")?;
let mut retirees = Vec::new();
while ordered.len() > MAX_MANAGED_ARCHIVES || total > MAX_TOTAL_ARCHIVE_BYTES {
let Some(position) = ordered.iter().position(|entry| protected.is_none_or(|path| entry.path != path)) else {
return Err("cannot satisfy archive retention limit".into());
};
let oldest = ordered.remove(position);
total = total.saturating_sub(oldest.size);
retirees.push(oldest);
}
Ok(retirees)
}
fn prune_existing(directory: &Path, entries: &mut [Managed], protected: Option<&Path>) -> Result<(), String> {
let retirees = retirement_set_protected(entries, protected)?;
let quarantined = quarantine(&retirees)?;
for (old, held) in quarantined {
if let Err(error) = remove_managed_file(&held) {
return Err(format!("could not retire {} from {}: {error}", old.display(), directory.display()));
}
}
Ok(())
}
fn quarantine(entries: &[Managed]) -> Result<Vec<(PathBuf, PathBuf)>, String> {
let mut moved = Vec::new();
let pid = std::process::id();
for (slot, entry) in entries.iter().enumerate() {
let held = entry.path.with_file_name(format!(".lightroom-retire-{pid}-{slot}.tmp"));
match fs::symlink_metadata(&held) {
Ok(_) => {
rollback_quarantine(&moved);
return Err(format!("retention temporary path already exists: {}", held.display()));
}
Err(error) if error.kind() == io::ErrorKind::NotFound => {}
Err(error) => {
rollback_quarantine(&moved);
return Err(error.to_string());
}
}
let meta = fs::symlink_metadata(&entry.path).map_err(|error| {
rollback_quarantine(&moved);
error.to_string()
})?;
if meta.file_type().is_symlink() || !meta.is_file() {
rollback_quarantine(&moved);
return Err(format!("managed archive changed into non-file: {}", entry.path.display()));
}
if let Err(error) = fs::rename(&entry.path, &held) {
rollback_quarantine(&moved);
return Err(error.to_string());
}
moved.push((entry.path.clone(), held));
}
Ok(moved)
}
fn rollback_quarantine(moved: &[(PathBuf, PathBuf)]) {
for (original, held) in moved.iter().rev() {
let _ = fs::rename(held, original);
}
}
fn publish_new(temporary: &Path, destination: &Path) -> io::Result<()> {
match fs::hard_link(temporary, destination) {
Ok(()) => Ok(()),
Err(error) => Err(io::Error::new(error.kind(), format!("cannot atomically publish archive: {error}"))),
}
}
fn remove_managed_file(path: &Path) -> io::Result<()> {
let meta = fs::symlink_metadata(path)?;
if meta.file_type().is_symlink() || !meta.is_file() {
return Err(io::Error::new(io::ErrorKind::InvalidInput, "refusing to remove non-regular archive path"));
}
fs::remove_file(path)
}
struct ArchiveRef<'a>(&'a CatalogImport);
impl Serialize for ArchiveRef<'_> {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
let mut out = serializer.serialize_struct("CatalogImportArchive", 7)?;
out.serialize_field("version", &FORMAT_VERSION)?;
out.serialize_field("source", &self.0.source)?;
out.serialize_field("photos", &PhotoSeq(&self.0.photos))?;
out.serialize_field("collections", &RowSeq(&self.0.collections))?;
out.serialize_field("members", &RowSeq(&self.0.members))?;
out.serialize_field("collection_content", &RowSeq(&self.0.collection_content))?;
out.serialize_field("warnings", &self.0.warnings)?;
out.end()
}
}
struct PhotoSeq<'a>(&'a [CatalogPhoto]);
impl Serialize for PhotoSeq<'_> {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
let mut out = serializer.serialize_seq(Some(self.0.len()))?;
for photo in self.0 {
out.serialize_element(&PhotoRef(photo))?;
}
out.end()
}
}
struct PhotoRef<'a>(&'a CatalogPhoto);
impl Serialize for PhotoRef<'_> {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
let p = self.0;
let mut out = serializer.serialize_struct("CatalogPhotoArchive", 8)?;
out.serialize_field("source_id", &p.source_id)?;
out.serialize_field("uuid", &p.uuid)?;
out.serialize_field("path", &p.path)?;
out.serialize_field("image", &RowRef(&p.image))?;
out.serialize_field("settings", &p.settings)?;
out.serialize_field("xmp", &p.xmp)?;
out.serialize_field("keywords", &p.keywords)?;
out.serialize_field("history", &RowSeq(&p.history))?;
out.serialize_field("snapshots", &RowSeq(&p.snapshots))?;
out.end()
}
}
struct RowSeq<'a>(&'a [BTreeMap<String, Value>]);
impl Serialize for RowSeq<'_> {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
let mut out = serializer.serialize_seq(Some(self.0.len()))?;
for row in self.0 {
out.serialize_element(&RowRef(row))?;
}
out.end()
}
}
struct RowRef<'a>(&'a BTreeMap<String, Value>);
impl Serialize for RowRef<'_> {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
let mut out = serializer.serialize_map(None)?;
for (key, value) in self.0 {
out.serialize_entry(key, value)?;
}
out.end()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::lightroom_catalog::{CatalogImport, CatalogPhoto};
use std::sync::atomic::{AtomicU64, Ordering};
static NEXT: AtomicU64 = AtomicU64::new(0);
fn tempdir() -> PathBuf {
let path = std::env::temp_dir().join(format!("lightcraft-lr-archive-{}-{}", std::process::id(), NEXT.fetch_add(1, Ordering::Relaxed)));
fs::create_dir_all(&path).unwrap();
path
}
fn data(source: String) -> CatalogImport {
CatalogImport {
source,
photos: Vec::new(),
collections: Vec::new(),
members: Vec::new(),
collection_content: Vec::new(),
warnings: Vec::new(),
}
}
fn rich_data() -> CatalogImport {
let photo = CatalogPhoto {
source_id: 7,
uuid: "uuid-7".into(),
path: "/photos/one.raw".into(),
image: BTreeMap::from([("rating".into(), serde_json::json!(5)), ("opaque_blob".into(), serde_json::json!([1, 2, 3, 4]))]),
settings: "s = { Exposure2012 = 0.75 }".into(),
xmp: "<xmp>decoded</xmp>".into(),
keywords: vec!["Trips|Goa".into()],
history: vec![BTreeMap::from([("text".into(), serde_json::json!("step")), ("curve".into(), serde_json::json!([0, 128, 255]))])],
snapshots: vec![BTreeMap::from([("name".into(), serde_json::json!("before"))])],
};
CatalogImport {
source: "source.lrcat".into(),
photos: vec![photo],
collections: vec![BTreeMap::from([("name".into(), serde_json::json!("Keepers"))])],
members: vec![BTreeMap::from([("image".into(), serde_json::json!(7))])],
collection_content: vec![BTreeMap::from([("rule".into(), serde_json::json!("rating > 3"))])],
warnings: vec!["one warning".into()],
}
}
fn managed_count(path: &Path) -> usize {
fs::read_dir(path).unwrap().filter_map(Result::ok).filter(|entry| parse_managed_name(&entry.file_name()).is_some()).count()
}
#[test]
fn deduplicates_identical_deterministic_archives() {
let path = tempdir();
let input = data("catalog.lrcat".into());
let first = store(&path, &input).unwrap();
let second = store(&path, &input).unwrap();
assert_eq!(first, second);
assert_eq!(managed_count(&path), 1);
fs::remove_dir_all(path).unwrap();
}
#[test]
fn dedup_returns_archive_that_retention_keeps() {
let path = tempdir();
let input = data("catalog.lrcat".into());
let first = store(&path, &input).unwrap();
for index in 1..MAX_MANAGED_ARCHIVES {
let filler = path.join(format!("lightroom-import-{index}.lca"));
File::create(filler).unwrap().set_len(19 * 1024 * 1024).unwrap();
}
let reused = store(&path, &input).unwrap();
assert_eq!(reused, first);
assert!(reused.is_file());
assert!(managed_count(&path) <= MAX_MANAGED_ARCHIVES);
fs::remove_dir_all(path).unwrap();
}
#[test]
fn caps_managed_archive_count_and_preserves_unrelated_files() {
let path = tempdir();
let human = path.join("notes.txt");
let legacy = path.join("lightroom-import-0.json");
fs::write(&human, b"keep").unwrap();
fs::write(&legacy, b"legacy").unwrap();
for index in 0..(MAX_MANAGED_ARCHIVES + 3) {
let input = data(format!("catalog-{index}.lrcat"));
store(&path, &input).unwrap();
}
assert!(managed_count(&path) <= MAX_MANAGED_ARCHIVES);
assert_eq!(fs::read(&human).unwrap(), b"keep");
assert_eq!(fs::read(&legacy).unwrap(), b"legacy");
fs::remove_dir_all(path).unwrap();
}
#[test]
fn rejects_oversize_before_creating_archive() {
let path = tempdir();
let input = data("x".repeat((MAX_ARCHIVE_BYTES as usize) + 1));
assert!(store(&path, &input).is_err());
assert_eq!(managed_count(&path), 0);
fs::remove_dir_all(path).unwrap();
}
#[test]
fn large_catalog_history_is_best_effort_and_does_not_abort_import() {
let path = tempdir();
let mut input = data("catalog.lrcat".into());
input.photos = (0..5_000)
.map(|index| CatalogPhoto {
source_id: index,
uuid: format!("photo-{index}"),
path: format!("/photos/{index}.raw"),
image: BTreeMap::new(),
settings: String::new(),
xmp: String::new(),
keywords: Vec::new(),
history: vec![BTreeMap::from([("text".into(), serde_json::json!(format!("history-{index}-{}", "x".repeat(8 * 1024))))])],
snapshots: Vec::new(),
})
.collect();
assert_eq!(store_best_effort(&path, &input).unwrap(), None);
assert_eq!(managed_count(&path), 0);
fs::remove_dir_all(path).unwrap();
}
#[test]
fn archive_roundtrips_zlib_payload_and_preserves_numeric_arrays() {
let path = tempdir();
let archive = store(&path, &rich_data()).unwrap();
let bytes = fs::read(archive).unwrap();
assert!(bytes.starts_with(HEADER));
let json = miniz_oxide::inflate::decompress_to_vec_zlib_with_limit(&bytes[HEADER.len()..], MAX_ARCHIVE_BYTES as usize).unwrap();
let decoded: Value = serde_json::from_slice(&json).unwrap();
assert_eq!(decoded["version"], serde_json::json!(FORMAT_VERSION));
assert_eq!(decoded["source"], serde_json::json!("source.lrcat"));
assert_eq!(decoded["photos"].as_array().unwrap().len(), 1);
assert_eq!(decoded["photos"][0]["source_id"], serde_json::json!(7));
assert_eq!(decoded["photos"][0]["uuid"], serde_json::json!("uuid-7"));
assert_eq!(decoded["photos"][0]["path"], serde_json::json!("/photos/one.raw"));
assert_eq!(decoded["photos"][0]["settings"], serde_json::json!("s = { Exposure2012 = 0.75 }"));
assert_eq!(decoded["photos"][0]["xmp"], serde_json::json!("<xmp>decoded</xmp>"));
assert_eq!(decoded["photos"][0]["keywords"], serde_json::json!(["Trips|Goa"]));
assert_eq!(decoded["photos"][0]["image"]["rating"], serde_json::json!(5));
assert_eq!(decoded["photos"][0]["image"]["opaque_blob"], serde_json::json!([1, 2, 3, 4]));
assert_eq!(decoded["photos"][0]["history"].as_array().unwrap().len(), 1);
assert_eq!(decoded["photos"][0]["history"][0]["curve"], serde_json::json!([0, 128, 255]));
assert_eq!(decoded["photos"][0]["snapshots"].as_array().unwrap().len(), 1);
assert_eq!(decoded["collections"].as_array().unwrap().len(), 1);
assert_eq!(decoded["collections"][0]["name"], serde_json::json!("Keepers"));
assert_eq!(decoded["members"].as_array().unwrap().len(), 1);
assert_eq!(decoded["members"][0]["image"], serde_json::json!(7));
assert_eq!(decoded["collection_content"].as_array().unwrap().len(), 1);
assert_eq!(decoded["warnings"], serde_json::json!(["one warning"]));
fs::remove_dir_all(path).unwrap();
}
#[test]
fn retention_enforces_aggregate_byte_cap_with_small_synthetic_entries() {
let mut entries: Vec<_> = (0..MAX_MANAGED_ARCHIVES as u64)
.map(|index| Managed { index, path: PathBuf::from(format!("lightroom-import-{index}.lca")), size: 20 * 1024 * 1024 })
.collect();
entries.push(Managed { index: 8, path: PathBuf::from("lightroom-import-8.lca"), size: 20 * 1024 * 1024 });
let retirees = retirement_set(&entries).unwrap();
assert_eq!(retirees.len(), 3);
assert_eq!(retirees.iter().map(|entry| entry.index).collect::<Vec<_>>(), vec![0, 1, 2]);
}
#[test]
fn retention_failure_preserves_old_archive() {
let path = tempdir();
let old = path.join("lightroom-import-0.lca");
let held = path.join(format!(".lightroom-retire-{}-0.tmp", std::process::id()));
fs::write(&old, b"old archive").unwrap();
fs::write(&held, b"occupied").unwrap();
let entry = Managed { index: 0, path: old.clone(), size: 11 };
assert!(quarantine(&[entry]).is_err());
assert_eq!(fs::read(&old).unwrap(), b"old archive");
fs::remove_dir_all(path).unwrap();
}
}
File diff suppressed because it is too large. Load diff
+450
View File
@@ -0,0 +1,450 @@
//! Detached Lightroom catalog preparation and owner-thread commit.
//!
//! `LightroomJob::new` takes only an in-memory session snapshot. `prepare` owns all catalog
//! reading, probing and archive staging, while `commit_prepared` only applies catalog operations
//! to the session that created the job. Archive index bytes are returned for a worker to finish
//! after the owner has accepted the catalog operation.
use std::collections::HashSet;
use std::path::{Path, PathBuf};
use std::sync::{
Arc,
atomic::{AtomicBool, AtomicUsize, Ordering},
};
use serde_json::Value;
use crate::import::{ImportJob, ImportMode, ImportOptions, Prepared};
use crate::lightroom_catalog::{self, CatalogImport, ImportIndex};
const COMMAND: &str = "library.importLightroom";
const MAX_INDEX_BYTES: usize = 16 * 1024 * 1024;
fn error(message: impl Into<String>) -> crate::EngineError {
crate::EngineError::BadParams { cmd: COMMAND.into(), msg: message.into() }
}
fn stage_archive(archive_dir: Option<&Path>, data: &mut CatalogImport) -> crate::Result<Option<PathBuf>> {
let archive_path = archive_dir.map(|dir| crate::lightroom_archive::store_best_effort(dir, data).map_err(error)).transpose()?.flatten();
if archive_dir.is_some() && archive_path.is_none() {
data.warnings.push("Lightroom source archive exceeded 32 MiB; archive skipped".into());
}
Ok(archive_path)
}
/// Memory identity of the library which owns a prepared completion.
///
/// Each successful open/close refreshes session identity, including reopening the same path.
/// The owner check deliberately does not compare catalog revisions: ordinary edits while preparation
/// runs must survive and are merged by the owner commit.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct LightroomLibraryToken {
library_dir: Option<PathBuf>,
identity: Arc<()>,
}
impl LightroomLibraryToken {
fn capture(s: &crate::Session) -> Self {
Self {
library_dir: s.library.as_ref().filter(|library| library.on_disk).map(|library| library.dir.clone()),
identity: Arc::clone(&s.library_identity),
}
}
/// Whether `s` still refers to the library which started preparation.
pub fn matches(&self, s: &crate::Session) -> bool {
Arc::ptr_eq(&self.identity, &s.library_identity)
&& self.library_dir == s.library.as_ref().filter(|library| library.on_disk).map(|library| library.dir.clone())
}
pub fn library_dir(&self) -> Option<&Path> {
self.library_dir.as_deref()
}
}
/// Filesystem-free result of catalog preparation.
pub struct PreparedLightroom {
pub(crate) data: CatalogImport,
pub(crate) normal: Prepared,
pub(crate) index: ImportIndex,
pub(crate) archive_path: Option<PathBuf>,
pub(crate) token: LightroomLibraryToken,
pub(crate) missing_sources: HashSet<String>,
pub(crate) update_existing: bool,
pub(crate) now: String,
cancelled: bool,
}
impl PreparedLightroom {
pub fn source(&self) -> &str {
&self.data.source
}
pub fn photo_count(&self) -> usize {
self.data.photos.len()
}
pub fn token(&self) -> &LightroomLibraryToken {
&self.token
}
pub fn is_cancelled(&self) -> bool {
self.cancelled
}
/// Roll back any normal-import filesystem placement held by this prepared value.
pub fn rollback(&self) {
self.normal.rollback();
}
}
/// Index write returned by owner commit. `finish` is intended for a worker after the owner has
/// returned, and performs the only archive-index filesystem operation in this pipeline.
pub struct LightroomArchiveFinalization {
path: Option<PathBuf>,
bytes: Option<Vec<u8>>,
}
impl LightroomArchiveFinalization {
pub fn finish(self) -> crate::Result<()> {
let (Some(path), Some(bytes)) = (self.path, self.bytes) else { return Ok(()) };
if bytes.len() > MAX_INDEX_BYTES {
return Err(error("Lightroom import index exceeds 16 MiB"));
}
lightcraft_catalog::safe_file::write_atomic(&path, &bytes).map_err(|e| error(e.to_string()))
}
pub fn path(&self) -> Option<&Path> {
self.path.as_deref()
}
}
/// Owner-thread catalog result. The report is ready for the UI immediately; index finalization
/// can remain on the worker while its task stays alive.
pub struct LightroomCompletion {
pub report: Value,
pub finalization: LightroomArchiveFinalization,
}
/// Alias used by hosts that call the result a commit.
pub type LightroomCommit = LightroomCompletion;
/// A Lightroom import detached from the owner thread.
pub struct LightroomJob {
path: PathBuf,
update_existing: bool,
import: ImportJob,
token: LightroomLibraryToken,
archive_dir: Option<PathBuf>,
}
impl LightroomJob {
/// Snapshot session state only. This constructor performs no filesystem work.
pub fn new(s: &mut crate::Session, path: PathBuf, update_existing: bool) -> crate::Result<Self> {
let opts = ImportOptions { mode: ImportMode::Add, ..ImportOptions::default() };
let import = ImportJob::new(s, opts)?;
let archive_dir = s.library.as_ref().filter(|library| library.on_disk).map(|library| library.dir.join("Interop"));
Ok(Self { path, update_existing, import, token: LightroomLibraryToken::capture(s), archive_dir })
}
pub fn total(&self) -> usize {
self.import.total.load(Ordering::Relaxed)
}
pub fn done(&self) -> usize {
self.import.done.load(Ordering::Relaxed)
}
pub fn progress(&self) -> (usize, usize) {
(self.done(), self.total())
}
pub fn token(&self) -> &LightroomLibraryToken {
&self.token
}
pub fn total_atomic(&self) -> std::sync::Arc<AtomicUsize> {
std::sync::Arc::clone(&self.import.total)
}
pub fn done_atomic(&self) -> std::sync::Arc<AtomicUsize> {
std::sync::Arc::clone(&self.import.done)
}
/// Finish an owner commit's deferred archive-index write on a worker.
pub fn finish_archive(finalization: LightroomArchiveFinalization) -> crate::Result<()> {
finalization.finish()
}
/// Read/probe/stage all source data off the owner thread.
pub fn prepare(&mut self, cancel: &AtomicBool) -> crate::Result<PreparedLightroom> {
if cancel.load(Ordering::Relaxed) {
return Err(error("Lightroom import cancelled"));
}
self.import.total.store(1, Ordering::Relaxed);
self.import.done.store(0, Ordering::Relaxed);
let mut data = lightroom_catalog::read_with_progress(&self.path, cancel, &self.import.total, &self.import.done).map_err(error)?;
lightroom_catalog::validate_for_job(&data).map_err(error)?;
if cancel.load(Ordering::Relaxed) {
return Err(error("Lightroom import cancelled"));
}
self.import.done.fetch_add(1, Ordering::Relaxed);
let paths: Vec<String> = data
.photos
.iter()
.filter(|photo| lightroom_catalog::number_for_job(&photo.image, "masterImage") == 0)
.map(|photo| photo.path.clone())
.collect();
let missing_sources = data
.photos
.iter()
.filter(|photo| lightroom_catalog::number_for_job(&photo.image, "masterImage") == 0)
.filter(|photo| !Path::new(&photo.path).is_file())
.map(|photo| normalize(&photo.path))
.collect();
let normal = self.import.prepare(&paths, cancel);
if cancel.load(Ordering::Relaxed) {
normal.rollback();
return Err(error("Lightroom import cancelled"));
}
let archive_path = stage_archive(self.archive_dir.as_deref(), &mut data)?;
let index_path = self.archive_dir.as_ref().map(|dir| dir.join("lightroom-index.json"));
let index = lightroom_catalog::load_index(index_path.as_deref())?;
Ok(PreparedLightroom {
data,
normal,
index,
archive_path,
token: self.token.clone(),
missing_sources,
update_existing: self.update_existing,
now: self.import.now().to_string(),
cancelled: cancel.load(Ordering::Relaxed),
})
}
}
/// Apply prepared data on the owner thread. No source, archive or index filesystem work occurs.
pub fn commit_prepared(s: &mut crate::Session, mut prepared: PreparedLightroom) -> crate::Result<LightroomCompletion> {
if prepared.cancelled {
prepared.normal.rollback();
return Err(error("Lightroom import cancelled"));
}
if !prepared.token.matches(s) {
prepared.normal.rollback();
return Err(error("Lightroom import belongs to a different open library"));
}
// A command-scoped Lightroom import must never rewrite source XMP sidecars. Direct depth-0
// callers do not set this transient flag, so they cannot affect a later command.
if s.depth > 0 {
s.skip_auto_write = true;
}
let undo0 = s.undo.len();
prepared.normal.revalidate_add(s);
let options = ImportOptions { mode: ImportMode::Add, ..ImportOptions::default() };
let now = prepared.now.clone();
let files = crate::import::commit_prepared(s, &options, &now, prepared.normal)?;
let applied = lightroom_catalog::apply_prepared(
s,
prepared.data,
lightroom_catalog::ApplyPrepared {
update_existing: prepared.update_existing,
files,
index: &mut prepared.index,
archive_path: prepared.archive_path,
missing_sources: &prepared.missing_sources,
undo0,
},
)?;
let path = prepared.token.library_dir().map(|dir| dir.join("Interop/lightroom-index.json"));
let bytes = if path.is_some() { Some(serde_json::to_vec(&applied.index).map_err(|e| error(e.to_string()))?) } else { None };
Ok(LightroomCompletion { report: applied.report, finalization: LightroomArchiveFinalization { path, bytes } })
}
fn normalize(path: &str) -> String {
let path = path.replace('\\', "/");
if cfg!(windows) { path.to_lowercase() } else { path }
}
#[cfg(test)]
mod tests {
use super::*;
fn empty_data() -> CatalogImport {
CatalogImport {
source: "fixture.lrcat".into(),
photos: Vec::new(),
collections: Vec::new(),
members: Vec::new(),
collection_content: Vec::new(),
warnings: Vec::new(),
}
}
fn cancelled_prepared(s: &crate::Session) -> PreparedLightroom {
PreparedLightroom {
data: empty_data(),
normal: Prepared::default(),
index: ImportIndex::default(),
archive_path: None,
token: LightroomLibraryToken::capture(s),
missing_sources: HashSet::new(),
update_existing: false,
now: "2026-01-01T00:00:00".into(),
cancelled: true,
}
}
fn prepared_for(s: &crate::Session, path: &str, rating: i64) -> PreparedLightroom {
let mut image = std::collections::BTreeMap::new();
image.insert("id_local".into(), serde_json::json!(1));
image.insert("rating".into(), serde_json::json!(rating));
image.insert("pick".into(), serde_json::json!(0));
image.insert("fileFormat".into(), serde_json::json!("JPG"));
image.insert("fileWidth".into(), serde_json::json!(1));
image.insert("fileHeight".into(), serde_json::json!(1));
PreparedLightroom {
data: CatalogImport {
source: "fixture.lrcat".into(),
photos: vec![lightroom_catalog::CatalogPhoto {
source_id: 1,
uuid: "fixture-photo".into(),
path: path.into(),
image,
settings: String::new(),
xmp: String::new(),
keywords: Vec::new(),
history: Vec::new(),
snapshots: Vec::new(),
}],
collections: Vec::new(),
members: Vec::new(),
collection_content: Vec::new(),
warnings: Vec::new(),
},
normal: Prepared::default(),
index: ImportIndex::default(),
archive_path: None,
token: LightroomLibraryToken::capture(s),
missing_sources: [normalize(path)].into_iter().collect(),
update_existing: false,
now: "2026-01-01T00:00:00".into(),
cancelled: false,
}
}
#[test]
fn cancellation_before_commit_leaves_owner_catalog_unchanged() {
let mut session = crate::Session::new();
let before = session.catalog.to_snapshot();
let prepared = cancelled_prepared(&session);
assert!(commit_prepared(&mut session, prepared).is_err());
assert_eq!(session.catalog.to_snapshot(), before);
assert!(session.undo.is_empty());
}
#[test]
fn library_identity_rejects_late_completion_after_reopen() {
let mut session = crate::Session::new();
let token = LightroomLibraryToken::capture(&session);
assert!(token.matches(&session));
session.library_identity = Arc::new(());
assert!(!token.matches(&session));
}
#[test]
fn prepared_reimport_preserves_current_edit_and_uses_one_undo_per_import() {
let mut session = crate::Session::new();
let path = "/missing/lightroom-job-edit.jpg";
let prepared = prepared_for(&session, path, 4);
let first = commit_prepared(&mut session, prepared).unwrap();
assert_eq!(first.report["mapping"]["1"].as_u64(), Some(1));
assert_eq!(session.undo.len(), 1);
let id = lightcraft_catalog::PhotoId(1);
let mut personal = lightcraft_develop::DevelopSettings::default();
personal.light.exposure = 1.5;
session.set_develop(id, personal, "Personal").unwrap();
let before = session.catalog.photo(id).map(|photo| photo.develop.clone());
let prepared = prepared_for(&session, path, 5);
commit_prepared(&mut session, prepared).unwrap();
assert_eq!(session.catalog.photo(id).map(|photo| photo.develop.clone()), before);
assert_eq!(session.catalog.photo(id).map(|photo| photo.rating), Some(5));
assert_eq!(session.undo.len(), 3);
}
#[test]
fn oversized_history_archive_keeps_prepared_import_successful_with_warning() {
let archive_dir = std::env::temp_dir().join(format!("lightcraft-lr-prepared-scale-{}", std::process::id()));
let mut data = empty_data();
data.photos = (0..5_000)
.map(|source_id| {
let image = std::collections::BTreeMap::from([
("id_local".into(), serde_json::json!(source_id)),
("fileFormat".into(), serde_json::json!("JPG")),
("fileWidth".into(), serde_json::json!(1)),
("fileHeight".into(), serde_json::json!(1)),
]);
lightroom_catalog::CatalogPhoto {
source_id,
uuid: format!("scale-{source_id}"),
path: format!("/missing/scale-{source_id}.jpg"),
image,
settings: String::new(),
xmp: String::new(),
keywords: Vec::new(),
history: vec![std::collections::BTreeMap::from([(
"text".into(),
serde_json::json!(format!("history-{source_id}-{}", "x".repeat(8 * 1024))),
)])],
snapshots: Vec::new(),
}
})
.collect();
let archive_path = stage_archive(Some(&archive_dir), &mut data).unwrap();
assert!(archive_path.is_none());
let mut session = crate::Session::new();
let token = LightroomLibraryToken::capture(&session);
let prepared = PreparedLightroom {
data,
normal: Prepared::default(),
index: ImportIndex::default(),
archive_path,
token,
missing_sources: HashSet::new(),
update_existing: false,
now: "2026-01-01T00:00:00".into(),
cancelled: false,
};
let completion = commit_prepared(&mut session, prepared).unwrap();
assert_eq!(completion.report["photos"], 5_000);
assert!(
completion.report["warnings"]
.as_array()
.is_some_and(|warnings| warnings.iter().any(|warning| warning == "Lightroom source archive exceeded 32 MiB; archive skipped"))
);
assert_eq!(session.catalog.len(), 5_000);
std::fs::remove_dir_all(archive_dir).unwrap();
}
#[test]
fn execute_fn_import_does_not_rewrite_source_sidecar() {
let dir = std::env::temp_dir().join(format!("lightcraft-lr-job-sidecar-{}", std::process::id()));
let path = dir.join("source.jpg");
let sidecar = dir.join("source.xmp");
std::fs::create_dir_all(&dir).unwrap();
std::fs::write(&path, b"fixture").unwrap();
std::fs::write(&sidecar, b"original sidecar").unwrap();
let original = std::fs::read(&sidecar).unwrap();
let mut session = crate::Session::new();
session.xmp.auto_write = true;
let result = session.execute_fn("library.importLightroom", |s| {
let prepared = prepared_for(s, &path.to_string_lossy(), 5);
let completion = commit_prepared(s, prepared)?;
Ok(completion.report)
});
assert!(result.is_ok());
assert_eq!(std::fs::read(&sidecar).unwrap(), original);
std::fs::remove_dir_all(dir).unwrap();
}
}
File diff suppressed because it is too large. Load diff
+118
View File
@@ -0,0 +1,118 @@
//! `log` records on stderr for the native desktop app and CLI (#168). The web build installs
//! eframe's `WebLogger` instead.
//!
//! Warnings and errors are shown by default. `LIGHTCRAFT_LOG` or `RUST_LOG` picks another level
//! (`off`, `error`, `warn`, `info`, `debug`, `trace`); `LIGHTCRAFT_LOG` wins when both are set.
//! `RUST_LOG` also accepts per-target directives (`lightcraft=debug,wgpu=warn`): a bare level
//! applies to everything, and the most verbose `lightcraft…=level` directive sets LightCraft's.
//! Below warnings only LightCraft's own records are shown, so `info` and `debug` aren't drowned
//! out by the GPU and windowing libraries.
use log::{Level, LevelFilter, Metadata, Record};
/// Records shown at a level, written to stderr as `<prefix>: LEVEL target: message`.
pub struct StderrLog {
prefix: &'static str,
level: LevelFilter,
}
impl StderrLog {
pub fn new(prefix: &'static str, level: LevelFilter) -> Self {
Self { prefix, level }
}
}
impl log::Log for StderrLog {
fn enabled(&self, m: &Metadata) -> bool {
m.level() <= self.level && (m.level() <= Level::Warn || m.target().starts_with("lightcraft"))
}
fn log(&self, r: &Record) {
if self.enabled(r.metadata()) {
eprintln!("{}: {} {}: {}", self.prefix, r.level(), r.target(), r.args());
}
}
fn flush(&self) {}
}
fn parse_level(s: &str) -> Option<LevelFilter> {
match s.trim().to_ascii_lowercase().as_str() {
"off" | "none" => Some(LevelFilter::Off),
"error" => Some(LevelFilter::Error),
"warn" | "warning" => Some(LevelFilter::Warn),
"info" => Some(LevelFilter::Info),
"debug" => Some(LevelFilter::Debug),
"trace" => Some(LevelFilter::Trace),
_ => None,
}
}
/// The level that `LIGHTCRAFT_LOG` / `RUST_LOG` ask for; warnings when neither names one.
pub fn level_from(lightcraft_log: Option<&str>, rust_log: Option<&str>) -> LevelFilter {
if let Some(l) = lightcraft_log.and_then(parse_level) {
return l;
}
let Some(rust_log) = rust_log else { return LevelFilter::Warn };
let mut bare = None;
let mut ours: Option<LevelFilter> = None;
for directive in rust_log.split(',') {
match directive.split_once('=') {
None => bare = parse_level(directive).or(bare),
Some((target, level)) if target.trim().starts_with("lightcraft") => {
if let Some(l) = parse_level(level) {
ours = Some(ours.map_or(l, |o| o.max(l)));
}
}
Some(_) => {}
}
}
ours.or(bare).unwrap_or(LevelFilter::Warn)
}
/// Install the stderr logger at the level the environment asks for. Does nothing when a logger is
/// already installed.
pub fn install(prefix: &'static str) {
let level = level_from(std::env::var("LIGHTCRAFT_LOG").ok().as_deref(), std::env::var("RUST_LOG").ok().as_deref());
static LOGGER: std::sync::OnceLock<StderrLog> = std::sync::OnceLock::new();
if log::set_logger(LOGGER.get_or_init(|| StderrLog::new(prefix, level))).is_ok() {
log::set_max_level(level);
}
}
#[cfg(test)]
mod tests {
use super::*;
use log::Log;
#[test]
fn warnings_by_default_and_either_variable_picks_the_level() {
assert_eq!(level_from(None, None), LevelFilter::Warn);
assert_eq!(level_from(None, Some("")), LevelFilter::Warn);
assert_eq!(level_from(None, Some("warn")), LevelFilter::Warn);
assert_eq!(level_from(None, Some("TRACE")), LevelFilter::Trace);
assert_eq!(level_from(None, Some("off")), LevelFilter::Off);
assert_eq!(level_from(Some("info"), None), LevelFilter::Info);
assert_eq!(level_from(Some("error"), Some("debug")), LevelFilter::Error, "LIGHTCRAFT_LOG wins");
assert_eq!(level_from(Some("nonsense"), Some("debug")), LevelFilter::Debug, "an unknown LIGHTCRAFT_LOG is ignored");
}
#[test]
fn rust_log_directives_set_lightcraft_s_level() {
assert_eq!(level_from(None, Some("lightcraft=debug,wgpu=warn")), LevelFilter::Debug);
assert_eq!(level_from(None, Some("wgpu_core=trace")), LevelFilter::Warn, "other crates' directives don't apply");
assert_eq!(level_from(None, Some("error,lightcraft_gpu=info")), LevelFilter::Info);
assert_eq!(level_from(None, Some("lightcraft_gpu=info,lightcraft_engine=trace")), LevelFilter::Trace);
assert_eq!(level_from(None, Some("info,naga=off")), LevelFilter::Info);
}
#[test]
fn only_lightcraft_records_below_warnings() {
let log = StderrLog::new("test", LevelFilter::Debug);
let meta = |level, target| Metadata::builder().level(level).target(target).build();
assert!(log.enabled(&meta(Level::Warn, "wgpu_core::device")));
assert!(log.enabled(&meta(Level::Debug, "lightcraft_gpu::ctx")));
assert!(!log.enabled(&meta(Level::Info, "wgpu_core::device")));
assert!(!log.enabled(&meta(Level::Trace, "lightcraft_gpu::ctx")));
let quiet = StderrLog::new("test", LevelFilter::Off);
assert!(!quiet.enabled(&meta(Level::Error, "lightcraft_engine")));
}
}
+1 -1
View File
@@ -62,7 +62,7 @@ impl SettingsHashes {
}
/// Bump when the pipeline's output changes, to invalidate cached thumbnails.
pub const RENDER_CACHE_VERSION: u64 = 11;
pub const RENDER_CACHE_VERSION: u64 = 14;
/// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache).
pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512];
+21 -2
View File
@@ -100,6 +100,7 @@ impl Lua {
struct LuaParser<'a> {
s: &'a [u8],
i: usize,
depth: usize,
}
impl LuaParser<'_> {
@@ -210,6 +211,15 @@ impl LuaParser<'_> {
std::str::from_utf8(&self.s[start..self.i]).ok().and_then(|t| t.parse().ok()).map_or_else(|| self.err("bad number"), Ok)
}
fn value(&mut self) -> Result<Lua, String> {
if self.depth >= 64 {
return self.err("Lua settings exceed nesting limit");
}
self.depth += 1;
let result = self.value_inner();
self.depth -= 1;
result
}
fn value_inner(&mut self) -> Result<Lua, String> {
self.skip_ws();
let Some(&c) = self.s.get(self.i) else { return self.err("unexpected end") };
match c {
@@ -315,7 +325,10 @@ impl LuaParser<'_> {
/// Parse a Lua table literal, optionally preceded by `name =` or `return`.
pub fn parse_lua(text: &str) -> Result<Lua, String> {
let mut p = LuaParser { s: text.as_bytes(), i: 0 };
if text.len() > 16 << 20 {
return Err("Lua settings exceed 16 MiB".into());
}
let mut p = LuaParser { s: text.as_bytes(), i: 0, depth: 0 };
p.skip_ws();
let save = p.i;
match p.ident().as_deref() {
@@ -347,6 +360,12 @@ pub fn lrtemplate_props(text: &str) -> Result<(Option<String>, Props, crate::crs
let root = parse_lua(text)?;
let title = root.get("title").or_else(|| root.get("internalName")).and_then(Lua::str).map(delocalize).filter(|t| !t.trim().is_empty());
let settings = root.get("value").and_then(|v| v.get("settings")).ok_or("no develop settings in this template")?;
let (props, values) = lua_settings_props(settings)?;
Ok((title, props, values))
}
/// Data-only Lua settings shared by preset and native catalog import. Never executes Lua.
pub(crate) fn lua_settings_props(settings: &Lua) -> Result<(Props, crate::crs_masks::Values), String> {
let Lua::Table(_, fields) = settings else { return Err("no develop settings in this template".into()) };
let mut props = Props::new();
let mut values = crate::crs_masks::Values::new();
@@ -374,7 +393,7 @@ pub fn lrtemplate_props(text: &str) -> Result<(Option<String>, Props, crate::crs
};
props.insert(key, vals);
}
Ok((title, props, values))
Ok((props, values))
}
// ------------------------------------------------------------------------------- XMP in files
+21 -19
View File
@@ -83,28 +83,22 @@ pub fn parse_preset_file(name: &str, bytes: &[u8]) -> Result<Vec<Preset>, String
Ok(out)
}
/// Expand files/folders (recursively) into preset files (`.lcpreset`, `.xmp`).
/// Expand files/folders (recursively, bounded: see [`crate::walk`]) into preset files
/// (`.lcpreset`, `.xmp`).
pub fn expand_preset_paths(paths: &[String]) -> Vec<String> {
fn walk(p: &Path, out: &mut Vec<String>, top: bool) {
if p.is_dir() {
let Ok(rd) = std::fs::read_dir(p) else { return };
let mut v: Vec<_> = rd.flatten().map(|e| e.path()).collect();
v.sort();
for c in v {
if !c.file_name().is_some_and(|n| n.to_string_lossy().starts_with('.')) {
walk(&c, out, false);
}
}
} else {
let ext = p.extension().map(|e| e.to_string_lossy().to_ascii_lowercase()).unwrap_or_default();
if top || ext == LCPRESET_EXT || ["xmp", "lrtemplate", "zip", "lmp", "mplumpack"].contains(&ext.as_str()) {
out.push(p.to_string_lossy().to_string());
}
}
}
let preset_file = |p: &Path| {
let ext = p.extension().map(|e| e.to_string_lossy().to_ascii_lowercase()).unwrap_or_default();
ext == LCPRESET_EXT || ["xmp", "lrtemplate", "zip", "lmp", "mplumpack"].contains(&ext.as_str())
};
let mut out = Vec::new();
for p in paths {
walk(Path::new(p), &mut out, true);
let path = Path::new(p);
if path.is_dir() {
let w = crate::walk::files_in(path, None, crate::walk::Limits::default(), preset_file);
out.extend(w.files.into_iter().map(|f| f.to_string_lossy().to_string()));
} else {
out.push(p.clone());
}
}
out
}
@@ -446,6 +440,14 @@ pub fn profile(id: &str) -> Option<&'static ProfileInfo> {
pub const RECENT_PROFILES: usize = 5;
impl Session {
/// A profile by id, looking up built-in profiles first and imported LUT profiles second.
pub fn profile_info(&self, id: &str) -> Option<(&str, &str)> {
if let Some(p) = profile(id) {
return Some((p.name, p.group));
}
self.lut_profiles.iter().find(|p| p.id == id).map(|p| (p.name.as_str(), p.group.as_str()))
}
/// Remember `id` as the most recently applied profile.
pub fn note_profile_used(&mut self, id: &str) {
self.profile_recent.retain(|p| p != id);
+1 -1
View File
@@ -262,7 +262,7 @@ impl Segmenter {
if mirrors.is_empty() {
return Err(lightcraft_segment::fetch::no_mirrors_message());
}
self.download.start(lightcraft_segment::fetch::SAM3_FILES, mirrors, dir, lightcraft_segment::fetch::Options::default())
self.download.start(lightcraft_segment::fetch::SAM3_FILES, mirrors, dir, lightcraft_segment::fetch::options())
}
#[cfg(not(feature = "sam"))]
{
+5 -1
View File
@@ -210,8 +210,11 @@ fn run(rx: &mpsc::Receiver<Job>, shared: &Shared, idle: Duration) {
}
let Some(mut job) = queue.pop_front() else { continue };
let is_detail = matches!(job.kind, Kind::Detail { .. });
let _done = Done(if is_detail { &shared.detail } else { &shared.pending });
// counts the job as pending until just before its reply goes out: a caller woken by the
// reply must not still see the worker busy with it
let done = Done(if is_detail { &shared.detail } else { &shared.pending });
if queue.iter().any(|later| supersedes(later, &job)) {
drop(done);
let _ = job.reply.send(Outcome { tag: job.tag, result: Ok(None), superseded: true });
continue;
}
@@ -229,6 +232,7 @@ fn run(rx: &mpsc::Receiver<Job>, shared: &Shared, idle: Duration) {
state.cache = None;
}
shared.loaded.store(state.model.is_some(), Ordering::SeqCst);
drop(done);
let _ = job.reply.send(Outcome { tag: job.tag, result, superseded: false });
}
}
+152 -18
View File
@@ -14,24 +14,27 @@ fn active_dev(s: &Session) -> lightcraft_develop::DevelopSettings {
fn changing_one_wb_control_resolves_as_shot_without_stale_tint() {
use lightcraft_catalog::{Photo, PhotoId, Source};
use lightcraft_develop::{DevelopSettings, WbMode};
let mut s = demo();
let id = PhotoId(100);
let mut p = Photo::new(id, Source::File { path: "synthetic.arw".into() }, "synthetic.arw", "ARW", 16, 16, "");
// A catalog made by the old generic-matrix Kelvin inference.
p.as_shot_wb = Some((6829.0, -127.0));
p.develop = std::sync::Arc::new(DevelopSettings::for_raw(6829.0, -127.0));
s.catalog.apply(lightcraft_catalog::Op::AddPhoto { photo: Box::new(p) }).unwrap();
s.execute("library.select", &json!({"ids": [100], "active": 100})).unwrap();
s.execute("develop.set", &json!({"control": "wb.temp", "value": 8000})).unwrap();
let d = active_dev(&s);
assert_eq!(d.wb.mode, WbMode::Custom);
assert_eq!(d.wb.temp, 8000.0);
assert_eq!(d.wb.tint, 0.0, "untouched tint comes from the current as-shot reference");
s.execute("develop.wb", &json!({"mode": "asShot"})).unwrap();
s.execute("develop.set", &json!({"control": "wb.tint", "value": 10})).unwrap();
assert_eq!(active_dev(&s).wb.temp, 6500.0);
s.execute("develop.reset", &json!({})).unwrap();
assert_eq!((active_dev(&s).wb.temp, active_dev(&s).wb.tint), (6500.0, 0.0));
for format in ["ARW", "NEF", "NRW", "RAF"] {
let mut s = demo();
let id = PhotoId(100);
let name = format!("synthetic.{format}");
let mut p = Photo::new(id, Source::File { path: name.clone() }, &name, format, 16, 16, "");
// A catalog made by the old generic-matrix Kelvin inference.
p.as_shot_wb = Some((6829.0, -127.0));
p.develop = std::sync::Arc::new(DevelopSettings::for_raw(6829.0, -127.0));
s.catalog.apply(lightcraft_catalog::Op::AddPhoto { photo: Box::new(p) }).unwrap();
s.execute("library.select", &json!({"ids": [100], "active": 100})).unwrap();
s.execute("develop.set", &json!({"control": "wb.temp", "value": 8000})).unwrap();
let d = active_dev(&s);
assert_eq!(d.wb.mode, WbMode::Custom);
assert_eq!(d.wb.temp, 8000.0);
assert_eq!(d.wb.tint, 0.0, "{format}: untouched tint comes from the current as-shot reference");
s.execute("develop.wb", &json!({"mode": "asShot"})).unwrap();
s.execute("develop.set", &json!({"control": "wb.tint", "value": 10})).unwrap();
assert_eq!(active_dev(&s).wb.temp, 6500.0, "{format}");
s.execute("develop.reset", &json!({})).unwrap();
assert_eq!((active_dev(&s).wb.temp, active_dev(&s).wb.tint), (6500.0, 0.0), "{format}");
}
}
#[test]
@@ -382,6 +385,61 @@ fn crop_aspect_lock_current_and_toggle() {
assert!(active_dev(&s).crop.aspect.is_none(), "toggle unlocks");
}
// Feature: dragging crop handles (issue 295)
#[test]
fn crop_drag_stops_at_the_image_edge_and_keeps_the_anchor() {
let mut s = demo();
s.execute("crop.set", &json!({"rect": [0.2, 0.2, 0.8, 0.8]})).unwrap();
s.execute("crop.drag", &json!({"handle": "bottomRight", "from": [0.8, 0.8], "to": [1.4, 0.7]})).unwrap();
let r = active_dev(&s).crop.geometry.rect;
assert!((r.x0 - 0.2).abs() < 1e-6 && (r.y0 - 0.2).abs() < 1e-6, "anchor moved: {r:?}");
assert!((r.x1 - 1.0).abs() < 1e-6 && (r.y1 - 0.7).abs() < 1e-6, "{r:?}");
}
#[test]
fn crop_drag_keeps_a_locked_ratio_on_every_handle() {
let mut s = demo();
s.execute("crop.aspect", &json!({"aspect": "16x9"})).unwrap();
let d = active_dev(&s);
assert!(d.crop.aspect.is_some());
let want = d.crop.geometry.rect.aspect();
let r = d.crop.geometry.rect;
for (handle, to) in [("left", [r.x0 + 0.1, 0.5]), ("top", [0.5, r.y0 + 0.05]), ("bottomRight", [1.3, 1.3]), ("right", [0.9, 0.5])] {
s.execute("crop.drag", &json!({"handle": handle, "from": [0.5, 0.5], "to": to, "start": [r.x0, r.y0, r.x1, r.y1]})).unwrap();
let g = active_dev(&s).crop.geometry.rect;
assert!((g.aspect() - want).abs() < 1e-6, "{handle}: {} vs {want}", g.aspect());
}
}
#[test]
fn crop_drag_rejects_unknown_handles_and_bad_points() {
let mut s = demo();
assert!(s.execute("crop.drag", &json!({"handle": "sideways", "from": [0.5, 0.5], "to": [0.6, 0.6]})).is_err());
assert!(s.execute("crop.drag", &json!({"handle": "move", "from": [0.5], "to": [0.6, 0.6]})).is_err());
assert!(s.execute("crop.drag", &json!({"handle": "move", "to": [0.6, 0.6]})).is_err());
}
#[test]
fn crop_aspect_rejects_degenerate_custom_ratios() {
let mut s = demo();
let before = active_dev(&s).crop;
for bad in [json!([0.004, 3]), json!([3, 0]), json!([-1, 2]), json!([1000, 0.01]), json!(["a", 2]), json!([1, 2, 3])] {
assert!(s.execute("crop.aspect", &json!({"aspect": bad.clone()})).is_err(), "{bad}");
}
assert!(s.execute("crop.aspect", &json!({"aspect": "0x5"})).is_err());
let after = active_dev(&s).crop;
assert_eq!(before.geometry, after.geometry, "a rejected ratio leaves the crop alone");
s.execute("crop.aspect", &json!({"aspect": [2.5, 1]})).unwrap();
assert!(!active_dev(&s).crop.geometry.rect.is_empty());
}
#[test]
fn crop_drag_rejects_non_numeric_points() {
let mut s = demo();
assert!(s.execute("crop.drag", &json!({"handle": "move", "from": ["a", 0.5, 0.5], "to": [0.6, 0.6]})).is_err());
assert!(s.execute("crop.drag", &json!({"handle": "move", "from": [0.5, 0.5], "to": [0.6, 0.6], "start": [0.1, "x", 0.5, 0.5]})).is_err());
}
#[test]
fn paste_from_previous_and_copy_paste_metadata() {
let mut s = demo();
@@ -817,3 +875,79 @@ fn face_job_follows_rotate_right() {
let (pw, ph) = ((r.x1 - r.x0) * h, (r.y1 - r.y0) * w);
assert!((pw - ph).abs() < 1e-6 * w.max(h), "square in rotated pixels: {pw} × {ph}");
}
/// Selecting an id that is not in the library is an error, and the selection stays as it was
/// (#182: `select_photos {"ids": [9999]}` answered `selected: 1` and the session lost its photo).
#[test]
fn selecting_an_unknown_photo_is_an_error_and_keeps_the_selection() {
let mut s = demo();
let vis = s.visible_cloned();
s.execute("library.select", &json!({"ids": [vis[0].0]})).unwrap();
let before = s.selection.clone();
for p in [
json!({"ids": [9999]}),
json!({"ids": [9999], "active": 9999}),
json!({"ids": [vis[1].0], "active": 9999}),
json!({"ids": [vis[1].0, 9999]}),
json!({"ids": [9999], "mode": "add"}),
json!({"ids": [9999], "mode": "toggle"}),
json!({"ids": [9999], "mode": "range"}),
] {
let e = s.execute("library.select", &p).unwrap_err().to_string();
assert!(e.contains("no such photo 9999"), "{p}: {e}");
assert_eq!(s.selection, before, "{p}");
}
assert_eq!(s.active(), Some(vis[0]));
assert!(s.execute("develop.get", &json!({})).is_ok());
// queries about an explicit id check it too
for c in ["develop.get", "photo.inspect", "photo.allMetadata", "history.list"] {
let e = s.execute(c, &json!({"id": 9999})).unwrap_err().to_string();
assert!(e.contains("no such photo 9999"), "{c}: {e}");
assert!(s.execute(c, &json!({"id": vis[1].0})).is_ok(), "{c}");
}
// an empty selection is still allowed
assert_eq!(s.execute("library.select", &json!({"ids": []})).unwrap()["selected"], 0);
}
/// A preset saved by another version may hold keys this one doesn't know: exporting with it keeps
/// working (the stale keys are dropped), while a typo the caller makes is still refused (#181).
#[test]
fn saved_presets_with_unknown_keys_still_export() {
use crate::export::{ExportOptions, ExportPreset};
let mut s = demo();
s.export_presets.push(ExportPreset {
name: "Old".into(),
params: json!({"format": "png", "retiredOption": 3, "watermark": {"text": "©", "retiredKey": true}}),
});
let p = s.export_params(&json!({"preset": "old", "quality": 80})).unwrap();
let o = ExportOptions::from_params(&p).unwrap();
assert_eq!(o.format, crate::export::ExportFormat::Png);
assert_eq!(o.watermark.map(|w| w.text), Some("©".to_string()));
assert!(ExportOptions::from_params(&s.export_params(&json!({"preset": "old", "qualty": 80})).unwrap()).is_err());
}
/// Undo and redo bring the photo they change on screen (issue #293): after editing one photo and
/// moving on, Undo makes it the active photo again; a step that changes several photos leaves the
/// selection alone.
#[test]
fn undo_and_redo_show_the_photo_they_change() {
let mut s = demo();
let vis = s.visible_cloned();
let (a, b) = (vis[0], vis[1]);
s.execute("library.select", &json!({"ids": [a.0]})).unwrap();
s.execute("develop.set", &json!({"control": "light.exposure", "value": 1.0})).unwrap();
s.execute("library.select", &json!({"ids": [b.0]})).unwrap();
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.active(), Some(a), "undo shows the photo it changed");
assert_eq!(s.selection.ids, vec![a]);
s.execute("library.select", &json!({"ids": [b.0]})).unwrap();
s.execute("edit.redo", &json!({})).unwrap();
assert_eq!(s.active(), Some(a), "so does redo");
// rating both selected photos is one step over two photos: undoing it leaves the selection
s.execute("library.select", &json!({"ids": [a.0, b.0], "active": b.0})).unwrap();
s.execute("photo.rate", &json!({"rating": 3})).unwrap();
s.execute("library.select", &json!({"ids": [a.0, b.0], "active": a.0})).unwrap();
let before = s.selection.clone();
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.selection, before, "a step over several photos keeps the selection");
}
+465
View File
@@ -0,0 +1,465 @@
//! Exploring the library by folder: `library.folders` lists where the imported photos live and
//! `library.filter {libraryFolder}` shows one folder's photos (see `lightcraft_catalog::folders`);
//! `library.removeFolder` takes a folder's photos out of the library.
//!
//! * Given photos imported from two folders, an agent can list them with counts, then choose one
//! and see exactly its photos; the filter shows as a chip that clears just that choice.
//! * A path that is not a library folder shows nothing rather than everything.
//! * Removing a folder from the library moves its photos to Recently Deleted (one undo step) and
//! leaves every file where it is.
//! * Renaming or moving a folder on disk keeps the chosen folder chosen.
use lightcraft_catalog::{Op, Photo, Source};
use serde_json::json;
use crate::{LibrarySource, Session, filter_chips};
fn source_folder(s: &mut Session, path: &str) -> serde_json::Value {
s.execute("library.source", &json!({"kind": "libraryFolder", "path": path})).unwrap()
}
fn add(s: &mut Session, path: &str) {
let id = s.catalog.alloc_photo_id();
let p = Photo::new(id, Source::File { path: path.into() }, "x.jpg", "JPEG", 60, 40, "2026-01-01T10:00:00");
s.catalog.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
}
/// A scratch folder that goes away with the test, however it ends.
struct Scratch(std::path::PathBuf);
impl Scratch {
fn new(tag: &str) -> Scratch {
let dir = std::env::temp_dir().join(format!("lc-folders-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
Scratch(dir)
}
fn path(&self, rel: &str) -> String {
self.0.join(rel).to_string_lossy().to_string()
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
fn session() -> Session {
let mut s = Session::new();
for path in ["/pics/trip/a.jpg", "/pics/trip/b.jpg", "/pics/home/c.jpg"] {
add(&mut s, path);
}
s
}
#[test]
fn the_folders_command_lists_where_the_photos_were_imported_from() {
let mut s = session();
let r = s.execute("library.folders", &json!({})).unwrap();
assert_eq!((r[0]["name"].as_str(), r[0]["volume"].as_bool(), r[0]["count"].as_u64()), (Some("/"), Some(true), Some(3)));
let pics = &r[0]["children"][0];
assert_eq!(pics["name"], "pics");
let kids: Vec<(&str, u64)> =
pics["children"].as_array().unwrap().iter().map(|c| (c["name"].as_str().unwrap(), c["count"].as_u64().unwrap())).collect();
assert_eq!(kids, [("home", 1), ("trip", 2)]);
}
#[test]
fn choosing_a_folder_shows_its_photos_and_a_chip_that_clears_it() {
let mut s = session();
assert_eq!(s.visible().len(), 3);
let r = s.execute("library.filter", &json!({"libraryFolder": "/pics/trip"})).unwrap();
assert_eq!(r["count"], 2);
let chips = filter_chips(&s.filter, &s.catalog);
assert_eq!(chips.len(), 1);
assert_eq!(chips[0].label, "Folder: pics/trip", "two names, so two folders called trip are told apart");
s.execute("library.filter", &chips[0].clear).unwrap();
assert_eq!(s.visible().len(), 3, "clearing the chip shows everything again");
}
#[test]
fn a_path_that_is_not_a_library_folder_shows_nothing() {
let mut s = session();
let r = s.execute("library.filter", &json!({"libraryFolder": "/elsewhere"})).unwrap();
assert_eq!(r["count"], 0);
}
#[test]
fn removing_a_folder_moves_its_photos_to_recently_deleted_and_undo_brings_them_back() {
let mut s = session();
let r = s.execute("library.removeFolder", &json!({"path": "/pics//trip/"})).unwrap();
assert_eq!(r["removed"], 2);
assert_eq!(s.visible().len(), 1, "only /pics/home is left");
let tree = s.execute("library.folders", &json!({})).unwrap();
assert_eq!(tree[0]["count"], 1);
s.source = LibrarySource::RecentlyDeleted;
assert_eq!(s.visible().len(), 2, "they wait in Recently Deleted");
s.source = LibrarySource::All;
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.visible().len(), 3, "one undo step");
}
#[test]
fn removing_a_folder_that_holds_nothing_or_a_whole_disk_is_refused() {
let mut s = session();
assert!(s.execute("library.removeFolder", &json!({"path": "/elsewhere"})).is_err());
assert!(s.execute("library.removeFolder", &json!({"path": "/"})).is_err(), "choose a folder, not everything");
assert!(s.execute("library.removeFolder", &json!({"path": " "})).is_err());
assert!(s.execute("library.removeFolder", &json!({})).is_err());
assert_eq!(s.visible().len(), 3, "nothing changed");
}
#[test]
fn removing_a_folder_leaves_browsed_and_already_deleted_photos_alone() {
let mut s = session();
for (path, local, deleted) in [("/pics/trip/browsed.jpg", true, false), ("/pics/trip/gone.jpg", false, true)] {
let id = s.catalog.alloc_photo_id();
let mut p = Photo::new(id, Source::File { path: path.into() }, "x.jpg", "JPEG", 60, 40, "2026-01-01T10:00:00");
(p.local, p.deleted) = (local, deleted);
s.catalog.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
}
let r = s.execute("library.removeFolder", &json!({"path": "/pics/trip"})).unwrap();
assert_eq!(r["removed"], 2, "only the two library photos of the folder");
let state: Vec<(bool, bool)> = s.catalog.photos().filter(|p| p.local || p.deleted).map(|p| (p.local, p.deleted)).collect();
assert!(state.contains(&(true, false)) && state.contains(&(false, true)), "browsed stays browsed, the old deletion stays: {state:?}");
}
#[test]
fn removing_the_folder_that_is_chosen_clears_the_choice_but_a_parent_stays() {
let mut s = session();
s.execute("library.filter", &json!({"libraryFolder": "/pics/trip"})).unwrap();
s.execute("library.removeFolder", &json!({"path": "/pics/trip"})).unwrap();
assert_eq!(s.filter.library_folder, None, "no empty grid under a chip for a folder that is gone");
let mut s = session();
s.execute("library.filter", &json!({"libraryFolder": "/pics"})).unwrap();
s.execute("library.removeFolder", &json!({"path": "/pics/trip"})).unwrap();
assert_eq!(s.filter.library_folder.as_deref(), Some("/pics"));
}
#[test]
fn a_path_that_names_no_folder_removes_nothing() {
let mut s = session();
for path in [".", "./", "a/..", "../..", "x/../.."] {
assert!(s.execute("library.removeFolder", &json!({"path": path})).is_err(), "{path:?}");
}
assert_eq!(s.visible().len(), 3);
}
#[test]
fn a_whole_disk_goes_only_when_asked_for_by_name_and_never_the_startup_disk() {
let mut s = Session::new();
for p in ["/Volumes/nas/a/1.jpg", "/Volumes/nas/2.jpg", "/Users/me/3.jpg", r"C:\x\4.jpg", r"\\srv\share\5.jpg"] {
add(&mut s, p);
}
for path in ["/Volumes/nas", "C:", "C:\\", r"\\?\C:\", "C:\\..", r"\\srv\share"] {
assert!(s.execute("library.removeFolder", &json!({"path": path})).is_err(), "{path:?} is a whole disk");
}
for path in ["/", "//"] {
assert!(s.execute("library.removeFolder", &json!({"path": path, "disk": true})).is_err(), "{path:?}: never the startup disk");
}
assert_eq!(s.visible().len(), 5, "nothing changed");
let r = s.execute("library.removeFolder", &json!({"path": "/Volumes/nas", "disk": true})).unwrap();
assert_eq!(r["removed"], 2, "everything on the nas, nothing elsewhere");
assert_eq!(s.visible().len(), 3);
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.visible().len(), 5);
}
#[test]
fn a_folder_that_spans_several_disks_is_removed_only_when_asked_for_by_name() {
let mut s = Session::new();
for p in ["/Volumes/nas/a/1.jpg", "/Volumes/tokyo/2.jpg", "/Volumes/3.jpg"] {
add(&mut s, p);
}
assert!(s.execute("library.removeFolder", &json!({"path": "/Volumes"})).is_err(), "it would take two disks' photos at once");
assert_eq!(s.visible().len(), 3);
let r = s.execute("library.removeFolder", &json!({"path": "/Volumes", "disk": true})).unwrap();
assert_eq!(r["removed"], 3);
}
#[test]
fn the_folder_filter_takes_only_a_text_path() {
let mut s = session();
assert!(s.execute("library.filter", &json!({"libraryFolder": 5})).is_err());
assert_eq!(s.filter.library_folder, None, "refused, nothing changed");
s.execute("library.filter", &json!({"libraryFolder": " "})).unwrap();
assert_eq!(s.visible().len(), 3, "a blank path is no choice");
s.execute("library.filter", &json!({"libraryFolder": "."})).unwrap();
assert_eq!(s.visible().len(), 0, "a path that names no folder shows nothing, not everything");
}
/// Folders are compared by identity: a path read back may be spelled with the platform's separator.
fn same_folder(actual: Option<&str>, expected: &str, why: &str) {
let key = lightcraft_catalog::query::folder_key;
assert_eq!(actual.map(key), Some(key(expected)), "{why}: {actual:?} vs {expected}");
}
#[test]
fn a_renamed_folder_stays_the_chosen_one_and_undo_follows_it_back() {
let dir = Scratch::new("rename");
std::fs::create_dir_all(dir.0.join("trip/day1")).unwrap();
let (trip, renamed) = (dir.path("trip"), dir.path("holiday"));
let mut s = Session::new();
add(&mut s, &format!("{trip}/a.jpg"));
add(&mut s, &format!("{trip}/day1/b.jpg"));
s.execute("library.filter", &json!({"libraryFolder": trip})).unwrap();
assert_eq!(s.visible().len(), 2);
s.execute("folder.rename", &json!({"path": trip, "name": "holiday"})).unwrap();
same_folder(s.filter.library_folder.as_deref(), &renamed, "the choice follows the folder");
assert_eq!(s.visible().len(), 2, "and still shows its photos");
s.execute("edit.undo", &json!({})).unwrap();
same_folder(s.filter.library_folder.as_deref(), &trip, "the choice is the same folder");
assert_eq!(s.visible().len(), 2);
}
#[test]
fn a_moved_folder_keeps_a_chosen_subfolder_chosen() {
let dir = Scratch::new("move");
std::fs::create_dir_all(dir.0.join("trip/day1")).unwrap();
std::fs::create_dir_all(dir.0.join("archive")).unwrap();
let (day1, moved) = (dir.path("trip/day1"), dir.path("archive/trip/day1"));
let mut s = Session::new();
add(&mut s, &format!("{day1}/b.jpg"));
s.execute("library.filter", &json!({"libraryFolder": day1})).unwrap();
s.execute("folder.move", &json!({"path": dir.path("trip"), "into": dir.path("archive")})).unwrap();
same_folder(s.filter.library_folder.as_deref(), &moved, "the subfolder follows its parent");
assert_eq!(s.visible().len(), 1);
s.execute("edit.undo", &json!({})).unwrap();
same_folder(s.filter.library_folder.as_deref(), &day1, "the choice is the same folder");
}
#[test]
fn following_a_folder_reads_both_spellings_the_same_way() {
// the choice was typed with a `..` in it; the folder that moved was not
let mut s = Session::new();
s.filter.library_folder = Some("/a/x/../b/sub".into());
crate::cmd::browse::follow_folder(&mut s, "/a/b", "/a/c");
same_folder(s.filter.library_folder.as_deref(), "/a/c/sub", "a subfolder stays a subfolder, never widens to its parent");
s.filter.library_folder = Some("/elsewhere/b".into());
crate::cmd::browse::follow_folder(&mut s, "/a/b", "/a/c");
assert_eq!(s.filter.library_folder.as_deref(), Some("/elsewhere/b"), "other folders are left alone");
}
#[test]
fn following_a_folder_keeps_the_case_of_the_subfolder_names() {
// Windows compares folders in lower case; the name the user sees must not change with it
let mut s = Session::new();
s.filter.library_folder = Some("/A/Trip/Day1".into());
crate::cmd::browse::follow_folder(&mut s, "/A/Trip", "/B");
assert_eq!(s.filter.library_folder.as_deref(), Some(std::path::Path::new("/B").join("Day1").to_str().unwrap()));
}
#[test]
fn choosing_a_folder_replaces_the_source_the_way_an_album_does() {
let mut s = session();
s.execute("library.source", &json!({"kind": "picks"})).unwrap();
assert_eq!(s.visible().len(), 0, "no picks");
let r = source_folder(&mut s, "/pics/trip");
assert_eq!(r["count"], 2, "the folder's photos, whatever was shown before");
assert_eq!((s.source, s.library_folder.as_deref()), (LibrarySource::LibraryFolder, Some("/pics/trip")));
assert!(filter_chips(&s.filter, &s.catalog).is_empty(), "a source is not a filter chip");
let state = s.execute("library.state", &json!({})).unwrap();
assert_eq!((state["sourceLabel"].as_str(), state["libraryFolder"].as_str()), (Some("pics/trip"), Some("/pics/trip")), "agents see which folder");
// the filter bar narrows it further, as it does an album
s.execute("library.filter", &json!({"rating": 1})).unwrap();
assert_eq!(s.visible().len(), 0);
s.execute("library.clearFilter", &json!({})).unwrap();
assert_eq!(s.visible().len(), 2);
s.execute("library.source", &json!({"kind": "all"})).unwrap();
assert_eq!(s.visible().len(), 3, "back to everything");
}
#[test]
fn the_folder_source_takes_a_library_folder_path_only() {
let mut s = session();
for params in [
json!({"kind": "libraryFolder"}),
json!({"kind": "libraryFolder", "path": 5}),
json!({"kind": "libraryFolder", "path": " "}),
json!({"kind": "libraryFolder", "path": "."}),
] {
assert!(s.execute("library.source", &params).is_err(), "{params}");
}
assert_eq!(s.source, LibrarySource::All, "refused, nothing changed");
assert!(s.execute("library.source", &json!({"kind": "libraryFolder", "path": "/elsewhere"})).is_err(), "no photo was imported from it");
assert_eq!(s.source, LibrarySource::All);
}
#[test]
fn removing_the_folder_being_shown_returns_to_all_photos() {
let mut s = session();
source_folder(&mut s, "/pics/trip");
s.execute("library.removeFolder", &json!({"path": "/pics/trip"})).unwrap();
assert_eq!((s.source, s.library_folder.as_deref()), (LibrarySource::All, None));
assert_eq!(s.visible().len(), 1);
// removing another folder leaves the source alone
let mut s = session();
source_folder(&mut s, "/pics/home");
s.execute("library.removeFolder", &json!({"path": "/pics/trip"})).unwrap();
assert_eq!(s.source, LibrarySource::LibraryFolder);
assert_eq!(s.visible().len(), 1);
}
#[test]
fn a_renamed_folder_stays_the_one_shown_and_undo_follows_it_back() {
let dir = Scratch::new("source-rename");
std::fs::create_dir_all(dir.0.join("trip/day1")).unwrap();
let (trip, day1, renamed) = (dir.path("trip"), dir.path("trip/day1"), dir.path("holiday"));
let mut s = Session::new();
add(&mut s, &format!("{trip}/a.jpg"));
add(&mut s, &format!("{day1}/b.jpg"));
source_folder(&mut s, &day1);
s.execute("folder.rename", &json!({"path": trip, "name": "holiday"})).unwrap();
same_folder(s.library_folder.as_deref(), &format!("{renamed}/day1"), "the subfolder follows its parent");
assert_eq!(s.visible().len(), 1);
s.execute("edit.undo", &json!({})).unwrap();
same_folder(s.library_folder.as_deref(), &day1, "the choice is the same folder");
assert_eq!(s.visible().len(), 1);
}
#[test]
fn the_shown_folder_follows_reading_both_spellings_the_same_way() {
let mut s = Session::new();
s.library_folder = Some("/a/x/../b/sub".into());
crate::cmd::browse::follow_folder(&mut s, "/a/b", "/a/c");
same_folder(s.library_folder.as_deref(), "/a/c/sub", "the shown folder reads both spellings alike");
}
#[test]
fn switching_from_one_folder_to_another_changes_what_is_shown() {
let mut s = session();
assert_eq!(source_folder(&mut s, "/pics/trip")["count"], 2);
assert_eq!(s.source_total(), Some(2));
assert_eq!(source_folder(&mut s, "/pics/home")["count"], 1, "same source, another folder");
assert_eq!(s.visible().len(), 1);
assert_eq!(s.source_total(), Some(1));
}
#[test]
fn a_smart_album_made_from_a_folder_view_keeps_the_folder() {
let mut s = session();
source_folder(&mut s, "/pics/trip");
assert_eq!(s.visible().len(), 2);
let r = s.execute("album.createSmart", &json!({"name": "Trip"})).unwrap();
assert_eq!(r["count"], 2, "the album matches the folder's photos, not the whole library: {r}");
}
#[test]
fn a_folder_source_and_a_folder_filter_never_disagree() {
let mut s = session();
s.execute("library.filter", &json!({"libraryFolder": "/pics/home"})).unwrap();
source_folder(&mut s, "/pics/trip");
assert_eq!(s.filter.library_folder, None, "choosing a folder drops a folder filter left over");
assert!(s.execute("library.filter", &json!({"libraryFolder": "/pics/home"})).is_err(), "one folder at a time");
assert_eq!(s.visible().len(), 2);
s.execute("library.filter", &json!({"libraryFolder": null, "rating": 0})).unwrap();
s.execute("library.source", &json!({"kind": "all"})).unwrap();
assert!(s.execute("library.filter", &json!({"libraryFolder": "/pics/home"})).is_ok(), "fine while no folder is the source");
}
#[test]
fn a_folder_source_with_no_folder_falls_back_to_all_photos() {
let mut s = session();
s.source = LibrarySource::LibraryFolder;
s.library_folder = None;
assert_eq!(s.visible().len(), 3, "everything, not an empty grid under no name");
assert_eq!((s.source, s.library_folder.as_deref()), (LibrarySource::All, None));
assert_eq!(s.source_total(), Some(3));
}
#[test]
fn only_a_folder_that_shows_exactly_its_own_photos_can_be_the_source() {
let mut s = Session::new();
for p in ["/Volumes/nas/a/1.jpg", "/Users/me/2.jpg", "/Volumes/3.jpg"] {
add(&mut s, p);
}
for path in ["/", "//", "/elsewhere", "/Volumes"] {
assert!(s.execute("library.source", &json!({"kind": "libraryFolder", "path": path})).is_err(), "{path:?}");
}
assert_eq!(s.source, LibrarySource::All, "refused, nothing changed");
assert_eq!(source_folder(&mut s, "/Volumes/nas")["count"], 1, "a disk is a fine source");
assert_eq!(source_folder(&mut s, "/Users/me")["count"], 1);
}
#[test]
fn removing_a_folder_above_a_disk_needs_asking_by_name_even_with_one_disk() {
let mut s = Session::new();
for p in ["/Volumes/nas/a/1.jpg", "/Users/me/2.jpg"] {
add(&mut s, p);
}
assert!(s.execute("library.removeFolder", &json!({"path": "/Volumes"})).is_err(), "it would take the whole nas");
assert_eq!(s.visible().len(), 2);
assert_eq!(s.execute("library.removeFolder", &json!({"path": "/Volumes/nas/a"})).unwrap()["removed"], 1, "a folder on the disk is just a folder");
assert_eq!(s.visible().len(), 1);
}
#[test]
fn a_folder_name_with_spaces_at_its_edge_is_never_mistaken_for_its_neighbour() {
let mut s = Session::new();
for p in ["/p/shoot /a.jpg", "/p/shoot/b.jpg", "/p/other/c.jpg"] {
add(&mut s, p);
}
assert_eq!(source_folder(&mut s, "/p/shoot ")["count"], 1);
let shown: Vec<String> = s.visible().to_vec().iter().filter_map(|id| s.catalog.photo(*id)).map(|p| p.file_name.clone()).collect();
assert_eq!(s.library_folder.as_deref(), Some("/p/shoot "), "kept as the tree gave it");
assert_eq!(shown.len(), 1);
let before: Vec<_> = s.catalog.photos().filter(|p| p.deleted).map(|p| p.id).collect();
assert!(before.is_empty());
let r = s.execute("library.removeFolder", &json!({"path": "/p/shoot "})).unwrap();
assert_eq!(r["removed"], 1);
let gone: Vec<String> = s
.catalog
.photos()
.filter(|p| p.deleted)
.map(|p| match &p.source {
Source::File { path } => path.clone(),
Source::Demo { .. } => String::new(),
})
.collect();
assert_eq!(gone, ["/p/shoot /a.jpg"], "the folder named, not its neighbour");
}
#[test]
fn editing_a_smart_album_made_from_a_folder_view_keeps_the_folder() {
let mut s = session();
source_folder(&mut s, "/pics/trip");
let r = s.execute("album.createSmart", &json!({"name": "Trip"})).unwrap();
let id = r["id"].as_u64().unwrap();
// the rules dialog saves what it shows (no folder field) with `replace`
let r = s.execute("album.setRules", &json!({"id": id, "replace": true, "rules": {"ruleSet": {"rules": []}}})).unwrap();
assert_eq!(r["count"], 2, "the folder is not something the dialog can show, so it is kept: {r}");
let r = s.execute("album.setRules", &json!({"id": id, "replace": true, "rules": {"libraryFolder": null}})).unwrap();
assert_eq!(r["count"], 3, "and can be dropped on purpose");
}
#[test]
fn a_shown_folder_that_loses_its_last_photo_gives_way_to_all_photos() {
let mut s = session();
source_folder(&mut s, "/pics/home");
s.execute("photo.delete", &json!({})).unwrap();
assert_eq!(s.visible().len(), 2, "back to everything that is left");
assert_eq!((s.source, s.library_folder.as_deref()), (LibrarySource::All, None));
// a folder whose photos are only hidden by the filter bar stays
let mut s = session();
source_folder(&mut s, "/pics/trip");
s.execute("library.filter", &json!({"rating": 5})).unwrap();
assert_eq!(s.visible().len(), 0);
assert_eq!(s.source, LibrarySource::LibraryFolder, "the filters hide them, the folder is still there");
}
#[test]
fn the_state_names_a_folder_only_while_it_is_shown() {
let mut s = session();
source_folder(&mut s, "/pics/trip");
assert_eq!(s.execute("library.state", &json!({})).unwrap()["libraryFolder"], "/pics/trip");
s.execute("library.source", &json!({"kind": "all"})).unwrap();
assert!(s.execute("library.state", &json!({})).unwrap()["libraryFolder"].is_null(), "not a stale leftover");
}
#[test]
fn a_blank_folder_filter_is_no_filter() {
let mut s = session();
s.execute("library.filter", &json!({"libraryFolder": " "})).unwrap();
assert_eq!(s.filter, lightcraft_catalog::Filter::default(), "no hidden 'filters active' state");
}
+247
View File
@@ -545,12 +545,17 @@ fn cube_luts_become_profiles() {
);
s.execute("develop.profile", &json!({"id": id, "amount": 0})).unwrap();
assert_eq!(s.render_now(photo, 48, 32).unwrap().image.data, base.data, "Amount 0 = the photo as it was");
assert_eq!(s.profile_info(&id), Some(("Warm Test", "Film Looks")));
s.execute("profile.favorite", &json!({"id": id})).unwrap();
assert!(s.profile_favorites.contains(&id));
// the library keeps it
drop(s);
lightcraft_pipeline::lut::unregister(&id);
let mut s = Session::new().with_fs();
s.open_library(&lib, false).unwrap();
assert!(s.lut_profiles.iter().any(|p| p.id == id));
assert!(s.profile_favorites.contains(&id), "imported LUT favorite survives reopen");
assert_eq!(s.profile_info(&id), Some(("Warm Test", "Film Looks")));
assert!(lightcraft_pipeline::lut::get(&id).is_some(), "registered again on open");
s.execute("profile.deleteImported", &json!({"id": id})).unwrap();
assert!(lightcraft_pipeline::lut::get(&id).is_none());
@@ -858,3 +863,245 @@ fn missing_photos_skip_local_browse_records() {
s.execute("library.source", &json!({"kind": "missing"})).unwrap();
assert_eq!(s.visible_cloned().len(), 4, "the view lists what the count counts");
}
/// Import a file that is in Recently Deleted (issue #298): by default it is skipped but the
/// report says it is in the trash; `onDeleted: restore` brings it back with its edits;
/// `onDeleted: fresh` replaces the trashed record with a new one.
fn trashed_photo(tag: &str) -> (Session, std::path::PathBuf, u64) {
let src = temp_dir(tag);
write_png(&src.join("a.png"), 1);
let mut s = Session::new().with_fs();
let r = s.execute("library.import", &json!({"paths": [src.to_string_lossy()]})).unwrap();
let id = r["imported"][0].as_u64().unwrap();
s.execute("photo.rate", &json!({"ids": [id], "rating": 4})).unwrap();
s.execute("photo.delete", &json!({"ids": [id]})).unwrap();
(s, src, id)
}
#[test]
fn duplicate_in_recently_deleted_is_reported_as_such() {
let (mut s, src, id) = trashed_photo("trash-report");
let r = s.execute("library.import", &json!({"paths": [src.to_string_lossy()]})).unwrap();
assert_eq!(ids(&r, "imported"), 0, "{r}");
assert_eq!(r["duplicates"][0]["existing"], id, "{r}");
assert_eq!(r["duplicates"][0]["existingDeleted"], true, "{r}");
assert!(s.catalog.photo(lightcraft_catalog::PhotoId(id)).unwrap().deleted, "default: left in the trash");
let _ = std::fs::remove_dir_all(&src);
}
#[test]
fn import_can_restore_a_duplicate_from_recently_deleted() {
let (mut s, src, id) = trashed_photo("trash-restore");
let r = s.execute("library.import", &json!({"paths": [src.to_string_lossy()], "onDeleted": "restore"})).unwrap();
assert_eq!(r["restored"], json!([id]), "{r}");
assert_eq!(ids(&r, "duplicates"), 0, "{r}");
let p = s.catalog.photo(lightcraft_catalog::PhotoId(id)).unwrap();
assert!(!p.deleted && p.rating == 4, "restored with its edits");
let _ = std::fs::remove_dir_all(&src);
}
#[test]
fn import_can_replace_a_duplicate_from_recently_deleted() {
let (mut s, src, id) = trashed_photo("trash-fresh");
let r = s.execute("library.import", &json!({"paths": [src.to_string_lossy()], "onDeleted": "fresh"})).unwrap();
assert_eq!(ids(&r, "imported"), 1, "{r}");
let new = r["imported"][0].as_u64().unwrap();
assert_ne!(new, id);
assert!(s.catalog.photo(lightcraft_catalog::PhotoId(id)).is_none(), "the trashed record is gone");
let p = s.catalog.photo(lightcraft_catalog::PhotoId(new)).unwrap();
assert!(!p.deleted && p.rating == 0, "a new photo without the old edits");
assert_eq!(s.catalog.len(), 1);
let _ = std::fs::remove_dir_all(&src);
}
#[test]
fn unknown_on_deleted_is_an_error() {
let mut s = Session::new().with_fs();
let e = s.execute("library.import", &json!({"paths": [], "onDeleted": "bogus"})).unwrap_err().to_string();
assert!(e.contains("onDeleted"), "{e}");
}
#[test]
fn empty_recently_deleted_removes_only_trashed_photos() {
let (mut s, src, id) = trashed_photo("trash-empty");
write_png(&src.join("b.png"), 2);
let r = s.execute("library.import", &json!({"paths": [src.join("b.png").to_string_lossy()]})).unwrap();
let keep = r["imported"][0].as_u64().unwrap();
let r = s.execute("library.emptyRecentlyDeleted", &json!({})).unwrap();
assert_eq!(r["deleted"], 1, "{r}");
assert!(s.catalog.photo(lightcraft_catalog::PhotoId(id)).is_none());
assert!(s.catalog.photo(lightcraft_catalog::PhotoId(keep)).is_some());
// nothing to empty is not an error
assert_eq!(s.execute("library.emptyRecentlyDeleted", &json!({})).unwrap()["deleted"], 0);
s.execute("edit.undo", &json!({})).unwrap();
let _ = std::fs::remove_dir_all(&src);
}
/// Two trashed photos that share an album and a two-photo stack: emptying (or permanently
/// deleting) them together must not fail or leave dangling references.
fn two_trashed_in_album_and_stack(tag: &str) -> (Session, std::path::PathBuf, [u64; 2], u64) {
let src = temp_dir(tag);
write_png(&src.join("a.png"), 1);
write_png(&src.join("b.png"), 2);
let mut s = Session::new().with_fs();
let r = s.execute("library.import", &json!({"paths": [src.to_string_lossy()]})).unwrap();
let ids = [r["imported"][0].as_u64().unwrap(), r["imported"][1].as_u64().unwrap()];
let al = s.execute("album.create", &json!({"name": "Trip"})).unwrap()["id"].as_u64().unwrap();
s.execute("album.addPhotos", &json!({"id": al, "ids": ids})).unwrap();
s.execute("library.select", &json!({"ids": ids})).unwrap();
s.execute("stack.group", &json!({"ids": ids})).unwrap();
s.execute("photo.delete", &json!({"ids": ids})).unwrap();
(s, src, ids, al)
}
#[test]
fn emptying_trashed_photos_of_one_album_and_stack_leaves_nothing_dangling() {
let (mut s, src, ids, al) = two_trashed_in_album_and_stack("trash-empty-refs");
let r = s.execute("library.emptyRecentlyDeleted", &json!({})).unwrap();
assert_eq!(r["deleted"], 2, "{r}");
assert_eq!(s.catalog.len(), 0);
assert!(s.catalog.album(lightcraft_catalog::AlbumId(al)).unwrap().photos.is_empty(), "album still lists a removed photo");
assert!(ids.iter().all(|i| s.catalog.stack_of(lightcraft_catalog::PhotoId(*i)).is_none()));
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.catalog.len(), 2, "one undo step brings both back");
assert_eq!(s.catalog.album(lightcraft_catalog::AlbumId(al)).unwrap().photos.len(), 2);
assert!(s.catalog.stack_of(lightcraft_catalog::PhotoId(ids[0])).is_some());
assert!(s.catalog.photos().all(|p| p.deleted), "back in Recently Deleted");
let _ = std::fs::remove_dir_all(&src);
}
#[test]
fn deleting_several_photos_permanently_leaves_nothing_dangling() {
let (mut s, src, ids, al) = two_trashed_in_album_and_stack("trash-perm-refs");
s.execute("library.select", &json!({"ids": ids})).unwrap();
s.execute("photo.deletePermanently", &json!({"ids": ids})).unwrap();
assert!(s.catalog.album(lightcraft_catalog::AlbumId(al)).unwrap().photos.is_empty());
let _ = std::fs::remove_dir_all(&src);
}
#[test]
fn fresh_import_of_two_trashed_photos_in_one_album_and_stack_works() {
let (mut s, src, ids, al) = two_trashed_in_album_and_stack("trash-fresh-refs");
let r = s.execute("library.import", &json!({"paths": [src.to_string_lossy()], "onDeleted": "fresh"})).unwrap();
assert_eq!(ids_len(&r), 2, "{r}");
assert!(s.catalog.album(lightcraft_catalog::AlbumId(al)).unwrap().photos.is_empty());
assert!(ids.iter().all(|i| s.catalog.photo(lightcraft_catalog::PhotoId(*i)).is_none()));
let _ = std::fs::remove_dir_all(&src);
}
fn ids_len(r: &Value) -> usize {
ids(r, "imported")
}
#[test]
fn restore_import_applies_the_album_to_restored_photos() {
let (mut s, src, ids, _) = trashed_photo_ids("trash-restore-album");
let r = s.execute("library.import", &json!({"paths": [src.to_string_lossy()], "onDeleted": "restore", "albumName": "Back"})).unwrap();
assert_eq!(r["restored"], json!(ids), "{r}");
let album = r["album"].as_u64().expect("album created for the restored photos");
assert_eq!(s.catalog.album(lightcraft_catalog::AlbumId(album)).unwrap().photos.len(), 1);
let _ = std::fs::remove_dir_all(&src);
}
fn trashed_photo_ids(tag: &str) -> (Session, std::path::PathBuf, [u64; 1], u64) {
let (s, src, id) = trashed_photo(tag);
(s, src, [id], 0)
}
/// Issue #374: launch arguments never turn into a walk of a whole drive or home folder. An empty
/// argument is ignored; `.`, `..` and (on Windows) `/` are resolved against the working folder a
/// launcher picked, which is often a drive root; a root is replaced by its memory card's DCIM
/// folder or refused, and so is the home folder.
#[test]
fn launch_paths_never_become_a_drive_root_or_the_home_folder() {
use crate::import::{LaunchPath, launch_path};
let root = if cfg!(windows) { Path::new("C:\\") } else { Path::new("/") };
let home = if cfg!(windows) { Path::new("C:\\Users\\me") } else { Path::new("/home/me") };
let none = |_: &Path| None;
let refused = |r: Option<LaunchPath>| matches!(r, Some(LaunchPath::Refused { .. }));
for empty in ["", " "] {
assert_eq!(launch_path(empty, Some(root), Some(home), none), None, "{empty:?}");
}
// the working folder is the drive root: `.` (and `..`) name the whole drive
for arg in [".", "..", "./", ".\\..", "/"] {
let arg = if cfg!(windows) { arg.to_string() } else { arg.replace('\\', "/") };
assert!(refused(launch_path(&arg, Some(root), Some(home), none)), "{arg:?}");
}
assert!(refused(launch_path("..", Some(&home.join("Pictures")), Some(home), none)), "up to the home folder");
assert!(refused(launch_path(&home.to_string_lossy(), None, Some(home), none)));
assert!(refused(launch_path(".", None, Some(home), none)), "relative, no working folder");
// a memory card's root: its camera folder
let dcim = |r: &Path| Some(r.join("DCIM"));
assert_eq!(launch_path(".", Some(root), Some(home), dcim), Some(LaunchPath::Import(root.join("DCIM").to_string_lossy().to_string())));
// ordinary folders and files resolve to absolute paths
let pics = home.join("Pictures");
assert_eq!(launch_path("Pictures", Some(home), Some(home), none), Some(LaunchPath::Import(pics.to_string_lossy().to_string())));
assert_eq!(
launch_path(&pics.join("a.jpg").to_string_lossy(), None, Some(home), none),
Some(LaunchPath::Import(pics.join("a.jpg").to_string_lossy().to_string()))
);
}
/// Issue #374: a folder import is bounded — a tree deeper or larger than the limits is not walked
/// to the end (it stops and says so), and what it found is still returned.
#[test]
fn expand_stops_at_the_walk_limits() {
let d = temp_dir("bounded");
let mut deep = d.clone();
for i in 0..8 {
deep = deep.join(format!("d{i}"));
std::fs::create_dir_all(&deep).unwrap();
std::fs::write(deep.join(format!("p{i}.jpg")), b"x").unwrap();
}
let top = [d.to_string_lossy().to_string()];
let (all, cut) = crate::import::expand_within(&top, None, crate::walk::Limits::default());
assert_eq!((all.len(), cut), (8, false));
let (some, cut) = crate::import::expand_within(&top, None, crate::walk::Limits { max_depth: 3, max_entries: 1000 });
assert_eq!((some.len(), cut), (2, true), "{some:?}");
let (few, cut) = crate::import::expand_within(&top, None, crate::walk::Limits { max_depth: 32, max_entries: 5 });
assert!(cut && few.len() < 8, "{few:?}");
let _ = std::fs::remove_dir_all(&d);
}
/// Raw containers LightCraft cannot decode but shows by their embedded JPEG are picked up from a folder
/// (the extensions are in `import::EXTENSIONS`; the files differ so none is a duplicate) and import as preview only.
#[test]
fn folder_import_picks_up_undecodable_raw_containers_as_preview_only() {
use lightcraft_codecs::{ChromaSubsampling, EncodeImage, EncodeMeta, Samples, encode_jpeg};
let src = temp_dir("preview-only-exts");
let px: Vec<u8> = (0..40 * 30).flat_map(|i| [(i % 251) as u8, 128, 200]).collect();
let jpeg = encode_jpeg(&EncodeImage::new(40, 30, 3, Samples::U8(&px)), 90, ChromaSubsampling::S444, &EncodeMeta::default()).unwrap();
// CRW / MRW / X3F: their magic, then the JPEG. IIQ / KDC / MOS / ERF: a TIFF whose IFD0 only points at the JPEG.
let tiff = {
let off = 8 + 2 + 24 + 4;
let mut f = b"II*\0\x08\0\0\0\x02\0".to_vec();
for (tag, v) in [(513u16, off as u32), (514, jpeg.len() as u32)] {
f.extend_from_slice(&tag.to_le_bytes());
f.extend_from_slice(&[4, 0, 1, 0, 0, 0]);
f.extend_from_slice(&v.to_le_bytes());
}
f.extend_from_slice(&[0, 0, 0, 0]);
f.extend_from_slice(&jpeg);
f
};
let with_magic = |magic: &[u8]| [magic, &[0x11u8; 64][..], &jpeg].concat();
let files: [(&str, Vec<u8>); 7] = [
("a.CRW", with_magic(b"II\x1a\0\0\0HEAPCCDR")),
("b.mrw", with_magic(b"\0MRM\0\x01\0\0")),
("c.x3f", with_magic(b"FOVb\x02\0\x02\0")),
("d.iiq", [&tiff[..], b"d"].concat()),
("e.kdc", [&tiff[..], b"e"].concat()),
("f.mos", [&tiff[..], b"f"].concat()),
("g.erf", [&tiff[..], b"g"].concat()),
];
for (name, bytes) in &files {
std::fs::write(src.join(name), bytes).unwrap();
}
let mut s = Session::new().with_fs();
let r = s.execute("library.import", &json!({"paths": [src.to_string_lossy()]})).unwrap();
assert_eq!(r["scanned"], 7, "{r}");
assert_eq!(ids(&r, "imported"), 7, "{r}");
assert_eq!(ids(&r, "failed"), 0, "{r}");
assert!(s.catalog.photos().all(|p| p.preview_only.is_some() && p.kind == lightcraft_catalog::MediaKind::Raw && (p.width, p.height) == (40, 30)));
let _ = std::fs::remove_dir_all(&src);
}
+67
View File
@@ -433,3 +433,70 @@ fn write_policy_timing() {
println!("copy {n} × {} MB: byte compare with the source {bytewise:?}, probe hash {hashed:?}", size >> 20);
let _ = std::fs::remove_dir_all(&base);
}
/// Issue #367: Import → Copy wrote and verified one file after another on one thread. The copies
/// now run side by side, with the names, order, duplicates and reports a one-by-one copy gives:
/// like-named files take -1, -2… in file order (also against a file already there and a literal
/// `IMG_1-1`), and a file with the content of one copied earlier in the batch is its duplicate
/// (also while that copy is still under way).
#[test]
fn copies_run_side_by_side_with_the_same_outcome() {
let base = temp_dir("parallel-copy");
let (card, dest) = (base.join("card"), base.join("out"));
write_png(&card.join("a/IMG_1.png"), 1);
write_png(&card.join("a/IMG_2.png"), 2);
write_png(&card.join("b/IMG_1.png"), 3);
write_png(&card.join("b/IMG_1-1.png"), 4);
write_png(&card.join("c/IMG_1.png"), 5);
for i in 0..8u8 {
write_png(&card.join(format!("d/p{i}.png")), 10 + i);
}
// the content of a file whose copy is still under way
std::fs::create_dir_all(card.join("e")).unwrap();
std::fs::copy(card.join("d/p3.png"), card.join("e/dup.png")).unwrap();
std::fs::create_dir_all(&dest).unwrap();
std::fs::write(dest.join("IMG_2.png"), b"someone else's file").unwrap();
let mut s = session();
inject(Fault::Slow);
let r = s
.execute(
"library.import",
&json!({"paths": [card.to_string_lossy()], "mode": "copy", "destination": dest.to_string_lossy(), "organize": "flat"}),
)
.unwrap();
inject(Fault::None);
let threads: std::collections::HashSet<_> = crate::import_move::copy_threads().into_iter().collect();
if std::thread::available_parallelism().map_or(1, |n| n.get()) > 1 {
assert!(threads.len() > 1, "copies ran on {} thread(s)", threads.len());
}
assert_eq!(len(&r, "imported"), 13, "{r}");
assert_eq!(len(&r, "failed"), 0, "{r}");
assert_eq!(len(&r, "duplicates"), 1, "{r}");
assert_eq!(r["duplicates"][0]["reason"], "content");
assert!(r["duplicates"][0]["path"].as_str().unwrap().ends_with("dup.png"), "{r}");
// the order of a one-by-one copy (files in name order), each verified
let expect = [
("a/IMG_1.png", "IMG_1.png"),
("a/IMG_2.png", "IMG_2-1.png"),
("b/IMG_1-1.png", "IMG_1-1.png"),
("b/IMG_1.png", "IMG_1-2.png"),
("c/IMG_1.png", "IMG_1-3.png"),
];
for (src, dst) in expect {
assert_eq!(std::fs::read(dest.join(dst)).unwrap(), std::fs::read(card.join(src)).unwrap(), "{src} → {dst}");
}
for i in 0..8 {
assert_eq!(std::fs::read(dest.join(format!("p{i}.png"))).unwrap(), std::fs::read(card.join(format!("d/p{i}.png"))).unwrap());
}
assert_eq!(std::fs::read(dest.join("IMG_2.png")).unwrap(), b"someone else's file");
assert_eq!(files_under(&dest).len(), 14, "{:?}", files_under(&dest));
// the duplicate names the photo copied from the same content (d/p3); ids follow the file order
let ids: Vec<u64> = r["imported"].as_array().unwrap().iter().map(|v| v.as_u64().unwrap()).collect();
assert!(ids.windows(2).all(|w| w[0] < w[1]), "{ids:?}");
assert_eq!(r["duplicates"][0]["existing"], ids[8], "{r}");
let first = s.catalog.photo(lightcraft_catalog::PhotoId(ids[0])).unwrap();
assert_eq!(first.file_name, "IMG_1.png");
let _ = std::fs::remove_dir_all(&base);
}
+33
View File
@@ -212,6 +212,39 @@ fn profile_favorites_and_recent_survive_reopen() {
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_shown_library_folder_survives_reopen() {
let dir = temp_dir("libfolder");
let mut s = open(&dir, true);
let id = s.catalog.alloc_photo_id();
let p = lightcraft_catalog::Photo::new(
id,
lightcraft_catalog::Source::File { path: "/pics/trip/a.jpg".into() },
"a.jpg",
"JPEG",
60,
40,
"2026-01-01T10:00:00",
);
s.commit("Add", lightcraft_catalog::Op::AddPhoto { photo: Box::new(p) }).unwrap();
s.execute("library.source", &json!({"kind": "libraryFolder", "path": "/pics/trip"})).unwrap();
s.save_view();
drop(s);
let s = open(&dir, false);
assert_eq!((s.source, s.library_folder.as_deref()), (crate::LibrarySource::LibraryFolder, Some("/pics/trip")));
drop(s);
// a folder that holds none of the library's photos any more opens on everything too
std::fs::write(dir.join("view.json"), br#"{"source": {"kind": "libraryFolder"}, "library_folder": "/no/such/folder"}"#).unwrap();
let s = open(&dir, false);
assert_eq!((s.source, s.library_folder.clone()), (crate::LibrarySource::All, None));
drop(s);
// a view file that names the source but no folder opens on everything, never an empty grid
std::fs::write(dir.join("view.json"), br#"{"source": {"kind": "libraryFolder"}}"#).unwrap();
let s = open(&dir, false);
assert_eq!((s.source, s.library_folder), (crate::LibrarySource::All, None));
let _ = std::fs::remove_dir_all(&dir);
}
/// Smart albums with a rule set: all / any / none, nested groups, validation, live updates.
#[test]
fn smart_album_rule_sets() {
+25
View File
@@ -95,3 +95,28 @@ fn unreadable_settings_are_not_overwritten() {
assert_eq!(files.files.get("presets.json").unwrap(), presets);
assert_eq!(files.files.get("prefs.json").unwrap(), prefs);
}
/// Issue #171: a library whose `catalog.lock` can't be locked (stood in for here by a directory
/// of that name, which takes the same path as a file system without locks) still opens, and the
/// user is told it is unprotected — in `library.info` and once through the library warnings the
/// app's notices and the CLI show. A normally locked library says nothing.
#[test]
fn an_unlockable_library_opens_with_a_warning() {
let lib = temp_dir("unlocked");
std::fs::create_dir_all(lib.join("catalog.lock")).unwrap();
let mut s = Session::new().with_fs();
s.open_library(&lib, false).unwrap();
let info = s.execute("library.info", &json!({})).unwrap();
let listed = info["settingsWarnings"].as_array().cloned().unwrap_or_default();
assert!(listed.iter().any(|w| w.as_str().is_some_and(|w| w.contains("without protection against a second program"))), "{info}");
let warnings = s.take_library_warnings();
assert_eq!(warnings.len(), 1, "{warnings:?}");
drop(s);
// the same library with a lockable lock file: no warning
std::fs::remove_dir_all(lib.join("catalog.lock")).unwrap();
let mut s = Session::new().with_fs();
s.open_library(&lib, false).unwrap();
assert!(s.take_library_warnings().is_empty());
drop(s);
let _ = std::fs::remove_dir_all(&lib);
}
+45 -1
View File
@@ -19,6 +19,9 @@ pub enum LibrarySource {
Folder,
/// Photos whose original file can't be found (`library.missing`).
Missing,
/// A folder the library's photos were imported from ([`crate::Session::library_folder`]),
/// and the folders inside it (`library.folders`).
LibraryFolder,
}
/// The folder a [`LibrarySource::Folder`] view shows.
@@ -49,7 +52,7 @@ impl LibrarySource {
LibrarySource::RecentlyDeleted => f.deleted = true,
LibrarySource::Picks => f.flag = Some(lightcraft_catalog::Flag::Pick),
// the folder itself is filled in by the session (it holds the path)
LibrarySource::Folder | LibrarySource::Missing => {}
LibrarySource::Folder | LibrarySource::Missing | LibrarySource::LibraryFolder => {}
}
f
}
@@ -62,6 +65,7 @@ impl LibrarySource {
LibrarySource::RecentlyDeleted => "Recently Deleted".into(),
LibrarySource::Picks => "Picks".into(),
LibrarySource::Folder => "Folder".into(),
LibrarySource::LibraryFolder => "Folder".into(),
LibrarySource::Missing => "Missing Photos".into(),
}
}
@@ -74,7 +78,27 @@ pub struct Selection {
pub active: Option<PhotoId>,
}
/// How a photo relates to the selection: what every view draws (grid cell, filmstrip cell…).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum SelectionState {
NotSelected,
/// Selected, but not the photo the loupe and panels act on.
Selected,
/// The most-selected photo.
Active,
}
impl Selection {
/// What views draw for `id`: active, selected or neither (one answer for grid, filmstrip, …).
pub fn state_of(&self, id: PhotoId) -> SelectionState {
if self.active == Some(id) {
SelectionState::Active
} else if self.contains(id) {
SelectionState::Selected
} else {
SelectionState::NotSelected
}
}
pub fn single(id: PhotoId) -> Selection {
Selection { ids: vec![id], active: Some(id) }
}
@@ -173,6 +197,11 @@ pub fn filter_chips(f: &Filter, cat: &Catalog) -> Vec<FilterChip> {
if let Some(d) = &f.date {
add(format!("Date: {}", date_label(d)), json!({"date": Null}));
}
if let Some(d) = f.library_folder.as_deref().filter(|d| !d.trim().is_empty()) {
// the last two names, so two folders called "Pictures" are told apart
let name = lightcraft_catalog::folders::folder_label(d);
add(format!("Folder: {name}"), json!({"libraryFolder": Null}));
}
if let Some(d) = &f.imported {
add(format!("Imported: {}", date_label(d)), json!({"imported": Null}));
}
@@ -203,3 +232,18 @@ fn date_label(d: &str) -> String {
_ => d.to_string(),
}
}
#[cfg(test)]
mod selection_state_tests {
use super::*;
use lightcraft_catalog::PhotoId;
#[test]
fn select_all_marks_the_others_selected_and_one_active() {
let ids: Vec<PhotoId> = (1..=3).map(PhotoId).collect();
let s = Selection { ids: ids.clone(), active: Some(ids[0]) };
assert_eq!(s.state_of(ids[0]), SelectionState::Active);
assert_eq!(s.state_of(ids[1]), SelectionState::Selected);
assert_eq!(s.state_of(PhotoId(9)), SelectionState::NotSelected);
}
}
+183
View File
@@ -0,0 +1,183 @@
//! Bounded folder walks (issue #374): every recursive listing of the disk — import, Local's
//! Include Subfolders, Find Missing Photos, preset and LUT folders, MCP — goes through
//! [`files_in`], which stops at a depth and a number of directory entries instead of walking a
//! whole drive, follows a symbolic link or junction to a folder at most once (no loops), and
//! decides what is a folder from the directory listing itself (no extra open per entry, which
//! is what makes a large walk slow on Windows).
//!
//! It never chooses where to walk: callers pass a folder the user named. Nothing in the app walks
//! at startup or on the UI thread.
use std::collections::HashSet;
use std::path::{Path, PathBuf};
/// How far a walk may go.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Limits {
/// Folder levels below the starting folder that are listed (its own entries are level 1).
pub max_depth: usize,
/// Directory entries looked at in all (files and folders, hidden ones included).
pub max_entries: usize,
}
impl Default for Limits {
fn default() -> Self {
Limits { max_depth: MAX_DEPTH, max_entries: MAX_ENTRIES }
}
}
/// Folder levels a walk descends at most; photo trees are a handful of levels deep.
pub const MAX_DEPTH: usize = 32;
/// Directory entries a walk looks at at most: far more than any photo folder holds, far fewer
/// than a whole system drive.
pub const MAX_ENTRIES: usize = 1_000_000;
/// What a walk found.
#[derive(Debug, Default, PartialEq)]
pub struct Walked {
/// The files `keep` accepted, in path order, folder by folder (depth first).
pub files: Vec<PathBuf>,
/// The walk stopped at a [`Limits`] bound: there may be more files.
pub truncated: bool,
/// Directory entries looked at.
pub entries: usize,
}
fn hidden(p: &Path) -> bool {
p.file_name().is_some_and(|n| n.to_string_lossy().starts_with('.'))
}
/// The files below folder `root` that `keep` accepts. Hidden entries (names starting with `.`)
/// are left out, the folder `skip` (e.g. the library's own) is never entered, and the walk stops
/// at `limits` (see [`Walked::truncated`]). Unreadable folders are skipped.
pub fn files_in(root: &Path, skip: Option<&Path>, limits: Limits, mut keep: impl FnMut(&Path) -> bool) -> Walked {
let mut out = Walked::default();
// canonical folders reached through a link, so a link loop is walked at most once
let mut linked: HashSet<PathBuf> = HashSet::new();
if let Ok(c) = std::fs::canonicalize(root) {
linked.insert(c);
}
// (path, depth of the folder it is in, is a folder); popped in path order
let mut stack: Vec<(PathBuf, usize, bool)> = vec![(root.to_path_buf(), 0, true)];
while let Some((path, depth, is_dir)) = stack.pop() {
if !is_dir {
if keep(&path) {
out.files.push(path);
}
continue;
}
if skip.is_some_and(|s| path == s) {
continue;
}
if depth >= limits.max_depth {
out.truncated = true;
continue;
}
let Ok(rd) = std::fs::read_dir(&path) else { continue };
let mut children: Vec<(PathBuf, bool)> = Vec::new();
for e in rd.flatten() {
if out.entries >= limits.max_entries {
out.truncated = true;
break;
}
out.entries = out.entries.saturating_add(1);
let p = e.path();
if hidden(&p) {
continue;
}
let Ok(ft) = e.file_type() else { continue };
let dir = if ft.is_symlink() {
// a link (or a Windows junction): a folder is entered once, a file is a file
match std::fs::metadata(&p) {
Ok(m) if m.is_dir() => {
let Ok(c) = std::fs::canonicalize(&p) else { continue };
if !linked.insert(c) {
continue;
}
true
}
_ => false,
}
} else {
ft.is_dir()
};
children.push((p, dir));
}
children.sort();
let below = depth.saturating_add(1);
stack.extend(children.into_iter().rev().map(|(p, d)| (p, below, d)));
}
out
}
#[cfg(test)]
mod tests {
use super::*;
fn tmp(tag: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("lc-walk-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
#[test]
fn files_come_in_path_order_without_hidden_ones_or_the_skipped_folder() {
let d = tmp("order");
for f in ["b/2.jpg", "a/1.jpg", "a/x/0.jpg", "c.jpg", ".hidden/h.jpg", "a/.h.jpg", "lib/l.jpg", "a/n.txt"] {
let p = d.join(f);
std::fs::create_dir_all(p.parent().unwrap()).unwrap();
std::fs::write(p, b"x").unwrap();
}
let lib = d.join("lib");
let w = files_in(&d, Some(&lib), Limits::default(), |p| p.extension().is_some_and(|e| e == "jpg"));
let rel: Vec<String> = w.files.iter().map(|p| p.strip_prefix(&d).unwrap().to_string_lossy().replace('\\', "/")).collect();
assert_eq!(rel, ["a/1.jpg", "a/x/0.jpg", "b/2.jpg", "c.jpg"]);
assert!(!w.truncated);
let _ = std::fs::remove_dir_all(&d);
}
/// Issue #374: a walk never runs on without end — a tree deeper than the limit, or with more
/// entries than the limit, stops and says so.
#[test]
fn deep_and_wide_trees_stop_at_the_limits() {
let d = tmp("bounds");
let mut deep = d.join("deep");
for i in 0..12 {
deep = deep.join(format!("l{i}"));
}
std::fs::create_dir_all(&deep).unwrap();
std::fs::write(deep.join("bottom.jpg"), b"x").unwrap();
let wide = d.join("wide");
std::fs::create_dir_all(&wide).unwrap();
for i in 0..50 {
std::fs::write(wide.join(format!("{i:02}.jpg")), b"x").unwrap();
}
let all = |_: &Path| true;
let w = files_in(&d.join("deep"), None, Limits { max_depth: 5, max_entries: 1000 }, all);
assert!(w.files.is_empty() && w.truncated, "{w:?}");
let w = files_in(&d.join("deep"), None, Limits { max_depth: 13, max_entries: 1000 }, all);
assert_eq!((w.files.len(), w.truncated), (1, false), "{w:?}");
let w = files_in(&wide, None, Limits { max_depth: 5, max_entries: 20 }, all);
assert_eq!((w.files.len(), w.entries, w.truncated), (20, 20, true));
let w = files_in(&wide, None, Limits::default(), all);
assert_eq!((w.files.len(), w.truncated), (50, false));
let _ = std::fs::remove_dir_all(&d);
}
/// A link back up the tree (a loop) is followed at most once: the walk ends.
#[cfg(unix)]
#[test]
fn link_loops_end() {
let d = tmp("loop");
std::fs::create_dir_all(d.join("a/b")).unwrap();
std::fs::write(d.join("a/b/p.jpg"), b"x").unwrap();
std::os::unix::fs::symlink(&d, d.join("a/b/up")).unwrap();
std::os::unix::fs::symlink(d.join("a"), d.join("alias")).unwrap();
let w = files_in(&d, None, Limits::default(), |_| true);
assert!(!w.truncated, "{w:?}");
assert!(w.files.len() <= 2, "{w:?}");
assert!(w.files.iter().any(|p| p.ends_with("a/b/p.jpg")), "{w:?}");
let _ = std::fs::remove_dir_all(&d);
}
}
+7
View File
@@ -0,0 +1,7 @@
# lightcraft-fetch
Native model downloads in pure Rust (L0, Apache-2.0): HTTP/1.1 over std sockets and rustls with the RustCrypto provider, without ring, aws-lc, C or assembly build dependencies. The wasm32 crate is empty.
Call `download` with bounded file specifications and ordered mirrors, or `download_url` for an exact catalog URL (including its query). Each file has a local filename independent of the URL, a size cap and optional pinned size/SHA-256. Transfers resume pinned `.part` files, verify before renaming, report progress and check cancellation during reads. Run on a worker thread. No proxy support; DNS resolution uses the platform resolver. Tokens are off by default and never forwarded to a different redirect host.
Tests use local synthetic HTTP servers for redirects, resume, truncation, cancellation, stalls, hashes, size limits and hostile specifications; they make no internet requests.
+10 -1
View File
@@ -60,7 +60,10 @@ pub struct Url {
impl Url {
pub fn parse(s: &str) -> Result<Url, HttpError> {
let bad = |why: &str| HttpError::BadUrl(format!("{s}: {why}"));
let bad = |why: &str| HttpError::BadUrl(why.to_string());
if s.len() > 16 * 1024 {
return Err(bad("URL exceeds 16 KiB"));
}
let s = s.trim();
if s.chars().any(|c| c.is_control() || c == ' ') {
return Err(bad("contains spaces or control characters"));
@@ -509,6 +512,12 @@ pub fn get(url: &Url, headers: &[(&str, String)], limits: &Limits) -> Result<Res
mod tests {
use super::*;
#[test]
fn rejects_oversized_and_control_character_urls() {
assert!(Url::parse(&format!("https://example.com/{}", "x".repeat(16 * 1024))).is_err());
assert!(Url::parse("https://example.com/x?secret=abc\r\nInjected: 1").is_err());
}
#[test]
fn parses_urls() {
let u = Url::parse("https://cdn.example.com/models/sam3/model.safetensors?sig=1#x").unwrap();
+93 -20
View File
@@ -33,12 +33,12 @@ use http::{HttpError, Limits, Url};
/// One file of the model.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct FileSpec {
pub name: &'static str,
pub struct FileSpec<'a> {
pub name: &'a str,
/// Exact size in bytes, when pinned (then the download resumes and is checked against it).
pub size: Option<u64>,
/// SHA-256 (lower-case hex), when pinned.
pub sha256: Option<&'static str>,
pub sha256: Option<&'a str>,
/// Largest size accepted when `size` is not pinned.
pub max: u64,
}
@@ -79,11 +79,14 @@ pub struct Options {
pub stall_timeout: Duration,
/// Tries per mirror and file (each resumes where the last one stopped).
pub attempts: u32,
/// The environment variable holding a bearer token, sent to each configured mirror's own host (only over https,
/// never to a host a redirect leads to), so list only mirrors that may see it. `None`: nothing is ever sent, so one model's token cannot leak to another's server.
pub token_env: Option<&'static str>,
}
impl Default for Options {
fn default() -> Self {
Options { connect_timeout: Duration::from_secs(15), stall_timeout: Duration::from_secs(30), attempts: 3 }
Options { connect_timeout: Duration::from_secs(15), stall_timeout: Duration::from_secs(30), attempts: 3, token_env: None }
}
}
@@ -103,6 +106,8 @@ pub struct Progress {
pub enum DownloadError {
/// No mirror is configured.
NoMirrors,
/// Invalid filename, size, hash or aggregate bound supplied by the caller.
Invalid(String),
Cancelled,
/// A local file problem (folder not writable, disk full…).
Disk(String),
@@ -116,6 +121,7 @@ pub enum DownloadError {
impl std::fmt::Display for DownloadError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
DownloadError::Invalid(e) => write!(f, "invalid download: {e}"),
DownloadError::NoMirrors => write!(f, "no download location is configured for this model in this build"),
DownloadError::Cancelled => write!(f, "the download was cancelled"),
DownloadError::Disk(e) => write!(f, "{e}"),
@@ -132,16 +138,45 @@ enum Fail {
Disk(String),
}
/// Download `files` into `dir` from `mirrors` (in order; see [`mirrors`]). Files already there
/// and verified are kept. `progress` is called as bytes arrive (often: throttle in it).
/// Fetch a single file from its exact URL, preserving any query string. The local filename
/// remains `file.name`; it never comes from redirects or response headers.
pub fn download_url(
file: &FileSpec<'_>,
url: &str,
dir: &Path,
opts: &Options,
cancel: &AtomicBool,
progress: &mut dyn FnMut(&Progress),
) -> Result<(), DownloadError> {
download_source(std::slice::from_ref(file), &[url.to_string()], dir, opts, cancel, progress, true)
}
/// Download `files` into `dir` from `mirrors` (in order; see [`mirrors`]). Verified files
/// are kept. `progress` is called as bytes arrive (often: throttle in it).
pub fn download(
files: &[FileSpec],
files: &[FileSpec<'_>],
mirrors: &[String],
dir: &Path,
opts: &Options,
cancel: &AtomicBool,
progress: &mut dyn FnMut(&Progress),
) -> Result<(), DownloadError> {
download_source(files, mirrors, dir, opts, cancel, progress, false)
}
fn download_source(
files: &[FileSpec<'_>],
mirrors: &[String],
dir: &Path,
opts: &Options,
cancel: &AtomicBool,
progress: &mut dyn FnMut(&Progress),
exact_urls: bool,
) -> Result<(), DownloadError> {
let total = validate_files(files)?;
if mirrors.len() > MAX_MIRRORS {
return Err(DownloadError::Invalid("too many mirrors".into()));
}
std::fs::create_dir_all(dir).map_err(|e| DownloadError::Disk(format!("can't create {}: {e}", dir.display())))?;
// what's left to fetch
let mut todo = Vec::new();
@@ -156,14 +191,13 @@ pub fn download(
if mirrors.is_empty() {
return Err(DownloadError::NoMirrors);
}
let total: u64 = files.iter().map(|f| f.size.unwrap_or(0)).sum();
let mut before: u64 = files.iter().filter(|f| !todo.contains(f)).filter_map(|f| f.size).sum();
for f in todo {
let mut errors = Vec::new();
let mut ok = false;
'mirrors: for m in mirrors {
let url = format!("{m}/{}", f.name);
for _ in 0..opts.attempts.max(1) {
let url = if exact_urls { m.clone() } else { format!("{m}/{}", f.name) };
for _ in 0..opts.attempts.clamp(1, 16) {
if cancel.load(Ordering::Relaxed) {
return Err(DownloadError::Cancelled);
}
@@ -184,11 +218,11 @@ pub fn download(
Err(Fail::Cancelled) => return Err(DownloadError::Cancelled),
Err(Fail::Disk(e)) => return Err(DownloadError::Disk(e)),
Err(Fail::Retry(e)) => {
log::warn!("download of {url}: {e} (retrying)");
log::warn!("download of {}: {e} (retrying)", short(&url));
errors.push(format!("{}: {e}", short(m)));
}
Err(Fail::NextMirror(e)) => {
log::warn!("download of {url}: {e}");
log::warn!("download of {}: {e}", short(&url));
errors.push(format!("{}: {e}", short(m)));
continue 'mirrors;
}
@@ -203,13 +237,56 @@ pub fn download(
Ok(())
}
/// Validate before touching disk, including names supplied by a user model catalog.
fn validate_files(files: &[FileSpec<'_>]) -> Result<u64, DownloadError> {
if files.len() > 256 {
return Err(DownloadError::Invalid("too many files".into()));
}
let mut seen = std::collections::HashSet::new();
let mut total = 0u64;
for f in files {
let stem = f.name.split('.').next().unwrap_or_default().to_ascii_uppercase();
let reserved = matches!(stem.as_str(), "CON" | "PRN" | "AUX" | "NUL")
|| stem
.strip_prefix("COM")
.or_else(|| stem.strip_prefix("LPT"))
.is_some_and(|n| matches!(n, "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9"));
if reserved
|| f.name.ends_with(['.', ' '])
|| f.name.ends_with(".part")
|| f.name.is_empty()
|| f.name.len() > 255
|| f.name.starts_with('.')
|| f.name.contains(['/', '\\', ':'])
|| f.name.chars().any(char::is_control)
|| !seen.insert(f.name)
{
return Err(DownloadError::Invalid("a filename must be a unique, ordinary filename".into()));
}
if f.max == 0
|| f.size.is_some_and(|n| n == 0 || n > f.max)
|| f.sha256.is_some_and(|h| h.len() != 64 || !h.bytes().all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b)))
{
return Err(DownloadError::Invalid("invalid file size or SHA-256".into()));
}
total = total.checked_add(f.size.unwrap_or(0)).ok_or_else(|| DownloadError::Invalid("total size overflow".into()))?;
}
Ok(total)
}
/// The bearer token to send to `url`: only a non-empty one, only over https, and only to the host the download began
/// at (`origin`), never to the host a redirect leads to.
fn bearer<'a>(token: Option<&'a str>, url: &Url, origin: &str) -> Option<&'a str> {
token.map(str::trim).filter(|t| !t.is_empty() && url.tls && url.host == origin)
}
/// A mirror for messages (no query string: it may hold a token).
fn short(m: &str) -> String {
Url::parse(m).map(|u| u.display()).unwrap_or_else(|_| "mirror".into())
}
/// Whether `path` is already the right file (exact size and hash when pinned).
fn verified(f: &FileSpec, path: &Path) -> Result<bool, DownloadError> {
fn verified(f: &FileSpec<'_>, path: &Path) -> Result<bool, DownloadError> {
let Ok(meta) = std::fs::metadata(path) else { return Ok(false) };
if !meta.is_file() || meta.len() == 0 {
return Ok(false);
@@ -243,7 +320,7 @@ fn part_path(dir: &Path, name: &str) -> PathBuf {
/// Fetch one file from one mirror into `<name>.part` (resuming it when its size is pinned),
/// check it and move it into place.
fn fetch_file(f: &FileSpec, url: &str, dir: &Path, opts: &Options, cancel: &AtomicBool, report: &mut dyn FnMut(u64, u64)) -> Result<(), Fail> {
fn fetch_file(f: &FileSpec<'_>, url: &str, dir: &Path, opts: &Options, cancel: &AtomicBool, report: &mut dyn FnMut(u64, u64)) -> Result<(), Fail> {
let part = part_path(dir, f.name);
let limit = f.size.unwrap_or(f.max);
let disk = |e: std::io::Error| Fail::Disk(format!("{}: {e}", part.display()));
@@ -268,12 +345,8 @@ fn fetch_file(f: &FileSpec, url: &str, dir: &Path, opts: &Options, cancel: &Atom
headers.push(("Range", format!("bytes={have}-")));
}
// a token for the configured host only (never sent on to a redirect's host)
if let Ok(t) = std::env::var("LIGHTCRAFT_SAM3_TOKEN")
&& !t.trim().is_empty()
&& url.tls
&& url.host == origin
{
headers.push(("Authorization", format!("Bearer {}", t.trim())));
if let Some(t) = bearer(opts.token_env.and_then(|name| std::env::var(name).ok()).as_deref(), &url, &origin) {
headers.push(("Authorization", format!("Bearer {t}")));
}
let r = http::get(&url, &headers, &limits).map_err(http_fail)?;
match r.status {
+75 -3
View File
@@ -22,7 +22,7 @@ fn leak(s: String) -> &'static str {
Box::leak(s.into_boxed_str())
}
fn pinned() -> FileSpec {
fn pinned() -> FileSpec<'static> {
let p = payload();
FileSpec { name: "model.bin", size: Some(p.len() as u64), sha256: Some(leak(sha_hex(&p))), max: p.len() as u64 }
}
@@ -137,10 +137,10 @@ fn tmp(name: &str) -> PathBuf {
}
fn quick() -> Options {
Options { connect_timeout: Duration::from_secs(2), stall_timeout: Duration::from_millis(600), attempts: 2 }
Options { connect_timeout: Duration::from_secs(2), stall_timeout: Duration::from_millis(600), attempts: 2, token_env: None }
}
fn run(files: &[FileSpec], mirrors: &[String], dir: &Path, cancel: &AtomicBool) -> (Result<(), DownloadError>, Vec<Progress>) {
fn run(files: &[FileSpec<'_>], mirrors: &[String], dir: &Path, cancel: &AtomicBool) -> (Result<(), DownloadError>, Vec<Progress>) {
let mut seen = Vec::new();
let r = download(files, mirrors, dir, &quick(), cancel, &mut |p| seen.push(p.clone()));
(r, seen)
@@ -314,3 +314,75 @@ fn content_ranges() {
assert_eq!(parse_content_range("bytes 9-1/300"), None);
assert_eq!(parse_content_range("items 1-2/3"), None);
}
/// A bearer token goes only to https, only to the host the download began at, and only when there is one.
#[test]
fn a_token_is_sent_only_over_https_to_the_first_host() {
let u = |s: &str| Url::parse(s).unwrap();
let first = u("https://huggingface.co/x/model.bin");
assert_eq!(bearer(Some(" hf_abc \n"), &first, "huggingface.co"), Some("hf_abc"));
// the storage host a redirect leads to, plain http, no token, an empty one
assert_eq!(bearer(Some("hf_abc"), &u("https://cdn-lfs.example/x"), "huggingface.co"), None);
assert_eq!(bearer(Some("hf_abc"), &u("http://huggingface.co/x"), "huggingface.co"), None);
assert_eq!(bearer(None, &first, "huggingface.co"), None);
assert_eq!(bearer(Some(" "), &first, "huggingface.co"), None);
// a caller that names no variable sends nothing, whatever the environment holds
assert!(Options::default().token_env.is_none());
}
#[test]
fn exact_url_keeps_query_and_owned_catalog_values() {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let url = format!("http://{}/download?file=weights&sig=secret", listener.local_addr().unwrap());
let worker = std::thread::spawn(move || {
let (mut socket, _) = listener.accept().unwrap();
let mut reader = BufReader::new(socket.try_clone().unwrap());
let mut request = String::new();
reader.read_line(&mut request).unwrap();
assert_eq!(request.trim(), "GET /download?file=weights&sig=secret HTTP/1.1");
loop {
let mut line = String::new();
if reader.read_line(&mut line).unwrap() == 0 || line.trim().is_empty() {
break;
}
}
socket.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 3\r\nConnection: close\r\n\r\nabc").unwrap();
});
let name = String::from("catalog.onnx");
let hash = sha_hex(b"abc");
let spec = FileSpec { name: &name, size: Some(3), sha256: Some(&hash), max: 3 };
let dir = tmp("exact");
let mut progress = Vec::new();
download_url(&spec, &url, &dir, &quick(), &AtomicBool::new(false), &mut |p| progress.push(p.clone())).unwrap();
worker.join().unwrap();
assert_eq!(std::fs::read(dir.join(name)).unwrap(), b"abc");
assert!(progress.iter().all(|p| !p.mirror.contains("secret")));
let _ = std::fs::remove_dir_all(dir);
}
#[test]
fn hostile_file_specs_are_rejected_before_touching_disk() {
let dir = tmp("invalid");
let base = FileSpec { name: "valid.onnx", size: Some(1), sha256: None, max: 1 };
let cancel = AtomicBool::new(false);
for name in ["", ".hidden", "../escape", "a/b", "a\\b", "C:escape", "bad\nname", "trailing.", "CON.onnx", "a.part"] {
let spec = FileSpec { name, ..base.clone() };
assert!(matches!(run(&[spec], &[], &dir, &cancel).0, Err(DownloadError::Invalid(_))), "{name}");
assert!(!dir.exists());
}
for spec in [
FileSpec { max: 0, ..base.clone() },
FileSpec { size: Some(0), ..base.clone() },
FileSpec { size: Some(2), ..base.clone() },
FileSpec { sha256: Some("not a SHA"), ..base.clone() },
] {
assert!(matches!(run(&[spec], &[], &dir, &cancel).0, Err(DownloadError::Invalid(_))));
assert!(!dir.exists());
}
let huge = FileSpec { size: Some(u64::MAX), max: u64::MAX, ..base.clone() };
let other = FileSpec { name: "other.onnx", ..huge.clone() };
for specs in [vec![huge, other], vec![base.clone(); 257], vec![base.clone(), base]] {
assert!(matches!(run(&specs, &[], &dir, &cancel).0, Err(DownloadError::Invalid(_))));
assert!(!dir.exists());
}
}
+483
View File
@@ -0,0 +1,483 @@
//! Interactive crop editing: what dragging a crop handle does.
//!
//! The crop rectangle lives in the straightened (rotated) frame and must stay inside the rotated
//! image. A drag is a *domain operation* on the crop, not a UI detail: the handle being dragged
//! decides which edges are anchored, an optional aspect lock decides the shape, and the image
//! bounds decide how far the drag may go. [`drag_crop`] answers all three in one place so the app,
//! CLI, control channel and MCP agree.
use crate::{CropGeometry, Point, Rect};
/// Smallest crop side, as a fraction of the image's shorter side.
pub const MIN_CROP_FRACTION: f64 = 0.02;
/// Most elongated locked crop shape (width:height or its inverse) the drag supports.
pub const MAX_RATIO: f64 = 20.0;
/// A grab point on the crop frame.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum CropHandle {
TopLeft,
TopRight,
BottomRight,
BottomLeft,
Top,
Right,
Bottom,
Left,
/// Drag the whole frame.
Move,
}
impl CropHandle {
/// The overlay's numbering: 0..3 corners (clockwise from top-left), 4..7 edges (top, right,
/// bottom, left), 8 = move.
pub fn from_index(i: u8) -> Option<CropHandle> {
Some(match i {
0 => CropHandle::TopLeft,
1 => CropHandle::TopRight,
2 => CropHandle::BottomRight,
3 => CropHandle::BottomLeft,
4 => CropHandle::Top,
5 => CropHandle::Right,
6 => CropHandle::Bottom,
7 => CropHandle::Left,
8 => CropHandle::Move,
_ => return None,
})
}
/// Parse a command-parameter name (`"topLeft"`, `"top"`, `"move"`, …).
pub fn from_name(n: &str) -> Option<CropHandle> {
Some(match n {
"topLeft" => CropHandle::TopLeft,
"topRight" => CropHandle::TopRight,
"bottomRight" => CropHandle::BottomRight,
"bottomLeft" => CropHandle::BottomLeft,
"top" => CropHandle::Top,
"right" => CropHandle::Right,
"bottom" => CropHandle::Bottom,
"left" => CropHandle::Left,
"move" => CropHandle::Move,
_ => return None,
})
}
}
/// Drag `handle` of `start` from `grab` to `pointer` (both normalized, in the straightened frame)
/// on a `w × h` image. `aspect` is the locked width/height ratio in pixels, if any.
///
/// Guarantees (for any input, including hostile numbers):
/// - the result lies inside the rotated image;
/// - the edges opposite the handle stay where they were (a move keeps the size);
/// - with `aspect`, the frame keeps that shape on every handle (when `start` had it);
/// - the drag stops at the image edge instead of pushing the frame around, and slides along it.
pub fn drag_crop(start: CropGeometry, handle: CropHandle, grab: Point, pointer: Point, w: f64, h: f64, aspect: Option<f64>) -> CropGeometry {
let sane = |v: f64| v.is_finite();
let r0 = start.rect;
if ![w, h, grab.x, grab.y, pointer.x, pointer.y, start.angle, r0.x0, r0.y0, r0.x1, r0.y1].iter().copied().all(sane)
|| w <= 0.0
|| h <= 0.0
|| r0.is_empty()
|| aspect.is_some_and(|a| !(a.is_finite() && a > 0.0))
{
return start;
}
// Start from a valid frame, then work in pixels of the straightened frame.
let mut base = start.constrained(w, h);
if base.angle == 0.0 {
// absorb the 1e-6 tolerance of `is_within_image` so the exact test below accepts the start
let r = base.rect;
base.rect = Rect::new(r.x0.clamp(0.0, 1.0), r.y0.clamp(0.0, 1.0), r.x1.clamp(0.0, 1.0), r.y1.clamp(0.0, 1.0));
}
let sr = base.rect_px(w, h);
// Moving further than twice the image never matters; clamping keeps the searches well-conditioned.
let (dx, dy) = (((pointer.x - grab.x) * w).clamp(-2.0 * w, 2.0 * w), ((pointer.y - grab.y) * h).clamp(-2.0 * h, 2.0 * h));
// Ratios beyond 20:1 make even the minimum frame larger than the image; treat them as 20:1.
let aspect = aspect.map(|a| a.clamp(1.0 / MAX_RATIO, MAX_RATIO));
// Unrotated, the frame must be exactly inside the image (`is_within_image` tolerates 1e-6, which
// would leak out as a sliver past the edge and, after any clamp, a shifted ratio).
let feasible = |r: Rect| {
if base.angle == 0.0 {
r.x0 >= 0.0 && r.y0 >= 0.0 && r.x1 <= w && r.y1 <= h
} else {
CropGeometry { rect: r.scale(1.0 / w, 1.0 / h), angle: base.angle }.is_within_image(w, h)
}
};
let min_side = MIN_CROP_FRACTION * w.min(h);
let out = match (handle, aspect) {
(CropHandle::Move, _) => {
let slide = |cur: Rect, ax: f64, ay: f64| {
let t = bisect(0.0, 1.0, |t| feasible(cur.translate(crate::Vec2::new(ax * t, ay * t))));
cur.translate(crate::Vec2::new(ax * t, ay * t))
};
let cur = slide(sr, dx, 0.0);
slide(cur, 0.0, dy)
}
(hd, None) => {
let (mx0, mx1, my0, my1) = sides(hd);
let grow = |cur: Rect, tx: f64, ty: f64| {
// `tx`/`ty`: how far the moving edges would go; interpolate towards it
let target = Rect::new(
if mx0 { (cur.x0 + tx).min(cur.x1 - min_side) } else { cur.x0 },
if my0 { (cur.y0 + ty).min(cur.y1 - min_side) } else { cur.y0 },
if mx1 { (cur.x1 + tx).max(cur.x0 + min_side) } else { cur.x1 },
if my1 { (cur.y1 + ty).max(cur.y0 + min_side) } else { cur.y1 },
);
let at = |t: f64| lerp_rect(cur, target, t);
at(bisect(0.0, 1.0, |t| feasible(at(t))))
};
let cur = grow(sr, dx, 0.0);
grow(cur, 0.0, dy)
}
(hd, Some(a)) => {
let (mx0, _, my0, _) = sides(hd);
let (min_w, min_h) = (min_side.max(min_side * a), min_side.max(min_side / a));
let c = sr.center();
// Anchor: the opposite corner / edge midpoint. `build(n, o)` is the frame of width `n` that
// keeps it, shifted by `o` along the anchored edge (edge handles only).
let (target_w, build): (f64, Box<dyn Fn(f64, f64) -> Rect>) = match hd {
CropHandle::Top | CropHandle::Bottom => {
let (ay, sy, edge) = if my0 { (sr.y1, -1.0, sr.y0) } else { (sr.y0, 1.0, sr.y1) };
let nh = (sy * (edge + dy - ay)).max(min_h);
(nh * a, Box::new(move |nw, o| vband(c.x + o, ay, sy, nw, a)))
}
CropHandle::Left | CropHandle::Right => {
let (ax, sx, edge) = if mx0 { (sr.x1, -1.0, sr.x0) } else { (sr.x0, 1.0, sr.x1) };
let nw = (sx * (edge + dx - ax)).max(min_w);
(nw, Box::new(move |nw, o| hband(ax, sx, c.y + o, nw, a)))
}
_ => {
let (ax, sx, ex) = if mx0 { (sr.x1, -1.0, sr.x0) } else { (sr.x0, 1.0, sr.x1) };
let (ay, sy, ey) = if my0 { (sr.y1, -1.0, sr.y0) } else { (sr.y0, 1.0, sr.y1) };
let (wx, wy) = (sx * (ex + dx - ax), sy * (ey + dy - ay));
let nw = ((wx + wy * a) / 2.0).max(min_w);
(nw, Box::new(move |nw, _| corner_rect(ax, sx, ay, sy, nw, a)))
}
};
let target_w = target_w.min(2.0 * w.max(h * a));
let slides = matches!(hd, CropHandle::Top | CropHandle::Bottom | CropHandle::Left | CropHandle::Right);
let extent = if matches!(hd, CropHandle::Top | CropHandle::Bottom) { w } else { h };
// Frame of width `n`, centred on the anchored edge if it fits, else slid the least distance.
let place = |n: f64| -> Option<Rect> {
let at = |o: f64| build(n, o);
if feasible(at(0.0)) {
return Some(at(0.0));
}
if !slides {
return None;
}
const STEPS: u32 = 128;
let step = extent / STEPS as f64;
for k in 1..=STEPS {
for sign in [1.0, -1.0] {
let hi = sign * k as f64 * step;
if feasible(at(hi)) {
// closest feasible offset: bisect between the last infeasible and this one
let (mut bad, mut good) = (sign * (k - 1) as f64 * step, hi);
for _ in 0..40 {
let mid = (bad + good) / 2.0;
if feasible(at(mid)) {
good = mid;
} else {
bad = mid;
}
}
return Some(at(good));
}
}
}
None
};
if place(min_w).is_some() {
let n = bisect_range(min_w, target_w.max(min_w), |n| place(n).is_some());
place(n).unwrap_or(sr)
} else {
sr
}
}
};
let rect = out.scale(1.0 / w, 1.0 / h);
if [rect.x0, rect.y0, rect.x1, rect.y1].iter().all(|v| v.is_finite()) { CropGeometry { rect, angle: base.angle } } else { start }
}
/// Which sides a handle moves: (left, right, top, bottom).
fn sides(h: CropHandle) -> (bool, bool, bool, bool) {
use CropHandle::*;
(
matches!(h, TopLeft | BottomLeft | Left),
matches!(h, TopRight | BottomRight | Right),
matches!(h, TopLeft | TopRight | Top),
matches!(h, BottomLeft | BottomRight | Bottom),
)
}
fn lerp_rect(a: Rect, b: Rect, t: f64) -> Rect {
let l = |p: f64, q: f64| p + (q - p) * t;
Rect::new(l(a.x0, b.x0), l(a.y0, b.y0), l(a.x1, b.x1), l(a.y1, b.y1))
}
/// Frame of width `nw` and ratio `a` extending from corner (`ax`, `ay`) in directions `sx`, `sy`.
fn corner_rect(ax: f64, sx: f64, ay: f64, sy: f64, nw: f64, a: f64) -> Rect {
let nh = nw / a;
Rect::from_points(Point::new(ax, ay), Point::new(ax + sx * nw, ay + sy * nh))
}
/// Frame whose top/bottom edge (at `ay`, growing in direction `sy`) is anchored, centred on `cx`.
fn vband(cx: f64, ay: f64, sy: f64, nw: f64, a: f64) -> Rect {
let nh = nw / a;
Rect::from_points(Point::new(cx - nw / 2.0, ay), Point::new(cx + nw / 2.0, ay + sy * nh))
}
/// Frame whose left/right edge (at `ax`, growing in direction `sx`) is anchored, centred on `cy`.
fn hband(ax: f64, sx: f64, cy: f64, nw: f64, a: f64) -> Rect {
let nh = nw / a;
Rect::from_points(Point::new(ax, cy - nh / 2.0), Point::new(ax + sx * nw, cy + nh / 2.0))
}
/// Largest `t` in `[lo, hi]` for which `ok` holds, given that it holds at `lo` and the set of good
/// `t` is an interval (the feasible region is convex).
fn bisect_range(lo: f64, hi: f64, ok: impl Fn(f64) -> bool) -> f64 {
if ok(hi) {
return hi;
}
let (mut good, mut bad) = (lo, hi);
for _ in 0..48 {
let mid = (good + bad) / 2.0;
if ok(mid) {
good = mid;
} else {
bad = mid;
}
}
good
}
fn bisect(lo: f64, hi: f64, ok: impl Fn(f64) -> bool) -> f64 {
bisect_range(lo, hi, ok)
}
#[cfg(test)]
mod tests {
use super::*;
use CropHandle::*;
const W: f64 = 600.0;
const H: f64 = 400.0;
const EPS: f64 = 1e-6;
fn geo(x0: f64, y0: f64, x1: f64, y1: f64) -> CropGeometry {
CropGeometry { rect: Rect::new(x0, y0, x1, y1), angle: 0.0 }
}
fn p(x: f64, y: f64) -> Point {
Point::new(x, y)
}
fn drag(g: CropGeometry, hd: CropHandle, from: Point, to: Point, aspect: Option<f64>) -> CropGeometry {
drag_crop(g, hd, from, to, W, H, aspect)
}
fn px_aspect(g: &CropGeometry) -> f64 {
g.rect.width() * W / (g.rect.height() * H)
}
fn near(a: f64, b: f64) -> bool {
(a - b).abs() < EPS
}
// Feature: dragging a crop handle
// Scenario: a free corner drag inside the image
#[test]
fn given_free_crop_when_corner_dragged_inside_then_corner_follows_and_opposite_corner_stays() {
let g = drag(geo(0.2, 0.2, 0.8, 0.8), BottomRight, p(0.8, 0.8), p(0.9, 0.7), None);
assert!(near(g.rect.x0, 0.2) && near(g.rect.y0, 0.2));
assert!(near(g.rect.x1, 0.9) && near(g.rect.y1, 0.7));
}
// Scenario: dragging past the right edge (the weird behaviour of issue 295)
#[test]
fn given_corner_dragged_past_image_edge_then_it_stops_at_the_edge_and_anchor_stays() {
let g = drag(geo(0.2, 0.2, 0.8, 0.8), BottomRight, p(0.8, 0.8), p(1.3, 0.7), None);
assert!(near(g.rect.x0, 0.2) && near(g.rect.y0, 0.2), "anchor moved: {:?}", g.rect);
assert!(near(g.rect.x1, 1.0), "stops at the edge: {:?}", g.rect);
assert!(near(g.rect.y1, 0.7), "other axis still follows the pointer: {:?}", g.rect);
}
#[test]
fn given_locked_aspect_when_corner_dragged_past_edge_then_ratio_and_anchor_hold() {
let a = 1.5;
let g = drag(geo(0.2, 0.2, 0.8, 0.2 + 0.6 * W / H / a), BottomRight, p(0.8, 0.6), p(1.4, 0.9), Some(a));
assert!(near(g.rect.x0, 0.2) && near(g.rect.y0, 0.2), "{:?}", g.rect);
assert!(near(px_aspect(&g), a), "{}", px_aspect(&g));
assert!(g.rect.x1 <= 1.0 + EPS && g.rect.y1 <= 1.0 + EPS);
assert!(g.rect.x1 > 0.95 || g.rect.y1 > 0.95, "grows until it touches an edge: {:?}", g.rect);
}
#[test]
fn given_locked_aspect_when_edge_dragged_then_ratio_holds_and_frame_grows_about_its_midline() {
let a = 2.0;
let start = geo(0.3, 0.3, 0.7, 0.3 + 0.4 * W / H / a);
let g = drag(start, Bottom, p(0.5, start.rect.y1), p(0.5, start.rect.y1 + 0.1), Some(a));
assert!(near(g.rect.y0, start.rect.y0), "opposite edge stays");
assert!(near(g.rect.center().x, start.rect.center().x), "grows symmetrically sideways");
assert!(near(px_aspect(&g), a));
assert!(g.rect.height() > start.rect.height());
}
#[test]
fn given_free_crop_when_edge_dragged_then_only_that_edge_moves() {
let g = drag(geo(0.2, 0.2, 0.8, 0.8), Left, p(0.2, 0.5), p(0.1, 0.9), None);
assert!(near(g.rect.x0, 0.1) && near(g.rect.y0, 0.2) && near(g.rect.x1, 0.8) && near(g.rect.y1, 0.8));
}
#[test]
fn given_move_past_edge_then_size_is_kept_and_frame_slides_along_the_edge() {
let start = geo(0.5, 0.25, 0.9, 0.65);
let g = drag(start, Move, p(0.7, 0.45), p(1.3, 0.55), None);
assert!(near(g.rect.width(), 0.4) && near(g.rect.height(), 0.4));
assert!(near(g.rect.x1, 1.0), "flush with the right edge: {:?}", g.rect);
assert!(near(g.rect.y0, 0.35), "still follows the pointer vertically: {:?}", g.rect);
}
#[test]
fn given_edge_dragged_across_the_opposite_edge_then_crop_keeps_a_minimum_size() {
let g = drag(geo(0.2, 0.2, 0.8, 0.8), Right, p(0.8, 0.5), p(-0.5, 0.5), None);
assert!(g.rect.width() * W >= MIN_CROP_FRACTION * H - EPS && g.rect.x1 > g.rect.x0);
assert!(near(g.rect.x0, 0.2));
}
#[test]
fn given_straightened_image_when_dragged_outside_then_crop_stays_inside_the_rotated_image() {
let start = crate::crop_fit_angle(W, H, 12.0, Some(1.5));
let c = start.rect.center();
for hd in [TopLeft, TopRight, BottomRight, BottomLeft, Top, Right, Bottom, Left, Move] {
for aspect in [None, Some(1.5)] {
let g = drag(start, hd, c, p(3.0, -3.0), aspect);
assert!(g.is_within_image(W, H), "{hd:?} {aspect:?}: {:?}", g.rect);
assert_eq!(g.angle, 12.0);
}
}
}
#[test]
fn given_hostile_numbers_then_the_start_crop_is_returned_not_a_panic() {
let s = geo(0.2, 0.2, 0.8, 0.8);
for bad in [f64::NAN, f64::INFINITY, f64::NEG_INFINITY] {
assert_eq!(drag(s, BottomRight, p(0.8, 0.8), p(bad, 0.5), None), s);
}
assert_eq!(drag_crop(s, BottomRight, p(0.8, 0.8), p(0.9, 0.9), 0.0, 400.0, None), s);
assert_eq!(drag(s, BottomRight, p(0.8, 0.8), p(0.9, 0.9), Some(f64::NAN)), s);
assert_eq!(drag(s, BottomRight, p(0.8, 0.8), p(0.9, 0.9), Some(-1.0)), s);
}
// Scenario: a locked edge drag when the frame already touches the image edge (review finding)
#[test]
fn given_locked_frame_flush_with_the_top_when_right_edge_dragged_out_then_it_grows_sliding_down() {
let a = 1.5;
let start = geo(0.2, 0.0, 0.5, 0.3 * W / H * 0.0 + 0.3 * W / (a * H));
let g = drag(start, Right, p(0.5, 0.2), p(0.9, 0.2), Some(a));
assert!(near(g.rect.x0, 0.2), "anchor edge stays: {:?}", g.rect);
assert!(g.rect.width() > start.rect.width() + 0.05, "it must grow: {:?}", g.rect);
assert!(near(px_aspect(&g), a));
assert!(g.is_within_image(W, H));
}
#[test]
fn given_full_fit_straightened_crop_when_locked_edge_dragged_then_it_still_respects_the_image() {
let start = crate::crop_fit_angle(W, H, 20.0, Some(1.5));
let g = drag(start, Right, p(0.9, 0.5), p(1.5, 0.5), Some(1.5));
assert!(g.is_within_image(W, H) && near(px_aspect(&g), 1.5));
}
// Scenario: extreme or hostile magnitudes
#[test]
fn given_an_extreme_locked_ratio_then_the_frame_can_still_be_moved() {
let start = geo(0.1, 0.4, 0.9, 0.4 + 0.8 * W / (60.0 * H));
let g = drag(start, Move, p(0.5, 0.4), p(0.5, 0.7), Some(60.0));
assert!(g.rect.y0 > 0.5, "moved: {:?}", g.rect);
}
#[test]
fn given_a_huge_finite_pointer_then_a_locked_corner_still_grows_to_the_edge() {
let a = 1.5;
let start = geo(0.2, 0.2, 0.5, 0.2 + 0.3 * W / (a * H));
let g = drag(start, BottomRight, p(0.5, 0.4), p(1e300, 1e300), Some(a));
assert!(g.rect.x1 > 0.95 || g.rect.y1 > 0.95, "{:?}", g.rect);
let g = drag(start, BottomRight, p(0.5, 0.4), p(1e300, 1e300), None);
assert!(near(g.rect.x1, 1.0) && near(g.rect.y1, 1.0), "{:?}", g.rect);
}
#[test]
fn given_an_unrotated_image_then_results_never_leave_the_unit_square() {
let g = drag(geo(0.2, 0.2, 0.8, 0.8), BottomRight, p(0.8, 0.8), p(5.0, 5.0), None);
assert!(g.rect.x1 <= 1.0 && g.rect.y1 <= 1.0 && g.rect.x0 >= 0.0 && g.rect.y0 >= 0.0, "{:?}", g.rect);
}
#[test]
fn handle_numbering_matches_the_overlay() {
assert_eq!(CropHandle::from_index(0), Some(TopLeft));
assert_eq!(CropHandle::from_index(6), Some(Bottom));
assert_eq!(CropHandle::from_index(8), Some(Move));
assert_eq!(CropHandle::from_index(9), None);
assert_eq!(CropHandle::from_name("bottomLeft"), Some(BottomLeft));
assert_eq!(CropHandle::from_name("nope"), None);
}
mod props {
use super::*;
use proptest::prelude::*;
fn any_handle() -> impl Strategy<Value = CropHandle> {
(0u8..9).prop_map(|i| CropHandle::from_index(i).unwrap_or(Move))
}
proptest! {
/// Whatever the pointer does, the crop stays valid; the anchor and the ratio hold.
#[test]
fn drag_is_always_inside_anchored_and_shaped(
angle in -45.0f64..45.0,
ratio in prop::option::of(0.3f64..4.0),
hd in any_handle(),
tx in -4.0f64..5.0, ty in -4.0f64..5.0,
shrink in 0.2f64..1.0,
) {
let base = crate::crop_fit_angle(W, H, angle, ratio);
// a smaller start crop so there is room to move
let c = base.rect.center();
let start = CropGeometry { rect: Rect::from_center(c, base.rect.width() * shrink, base.rect.height() * shrink), angle };
prop_assume!(start.is_within_image(W, H));
let g = drag_crop(start, hd, c, p(tx, ty), W, H, ratio);
prop_assert!(g.is_within_image(W, H), "{:?} -> {:?}", start.rect, g.rect);
prop_assert!(g.rect.width() > 0.0 && g.rect.height() > 0.0);
let min = MIN_CROP_FRACTION * W.min(H) - 1e-6;
prop_assert!(g.rect.width() * W >= min.min(start.rect.width() * W) && g.rect.height() * H >= min.min(start.rect.height() * H), "min size {:?}", g.rect);
if let Some(a) = ratio {
prop_assert!((px_aspect(&g) - a).abs() < 1e-6 * a.max(1.0), "ratio {a}: {}", px_aspect(&g));
}
// anchors: edges opposite the handle do not move
let (s, r) = (start.rect, g.rect);
let keep_l = matches!(hd, TopRight | BottomRight | Right);
let keep_r = matches!(hd, TopLeft | BottomLeft | Left);
let keep_t = matches!(hd, BottomLeft | BottomRight | Bottom);
let keep_b = matches!(hd, TopLeft | TopRight | Top);
{
if keep_l { prop_assert!((r.x0 - s.x0).abs() < 1e-6, "x0"); }
if keep_r { prop_assert!((r.x1 - s.x1).abs() < 1e-6, "x1"); }
if keep_t { prop_assert!((r.y0 - s.y0).abs() < 1e-6, "y0"); }
if keep_b { prop_assert!((r.y1 - s.y1).abs() < 1e-6, "y1"); }
}
if hd == Move {
prop_assert!((r.width() - s.width()).abs() < 1e-9 && (r.height() - s.height()).abs() < 1e-9);
}
}
#[test]
fn drag_never_panics_on_arbitrary_numbers(
x0 in -2.0f64..3.0, y0 in -2.0f64..3.0, x1 in -2.0f64..3.0, y1 in -2.0f64..3.0,
angle in -90.0f64..90.0, tx in -1e9f64..1e9, ty in -1e9f64..1e9,
ratio in prop::option::of(-1.0f64..10.0), hd in any_handle(),
) {
let g = drag_crop(CropGeometry { rect: Rect::new(x0, y0, x1, y1), angle }, hd, p(0.5, 0.5), p(tx, ty), W, H, ratio);
prop_assert!(g.rect.x0.is_finite() && g.rect.y1.is_finite());
}
}
}
}
+2
View File
@@ -6,10 +6,12 @@
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
mod crop;
mod crop_drag;
mod homography;
mod real;
pub use crop::{CropGeometry, crop_fit_angle, max_inscribed_scale};
pub use crop_drag::{CropHandle, MAX_RATIO, MIN_CROP_FRACTION, drag_crop};
pub use homography::Homography;
pub use real::{Interval, Real};
+13 -2
View File
@@ -155,8 +155,16 @@ pub(crate) fn with_init_marker<T>(backends: Backends, f: impl FnOnce() -> T) ->
/// If the init marker `path` is present — the last process died while creating the GPU device —
/// remove it and return what it recorded.
pub fn take_init_marker(path: &std::path::Path) -> Option<String> {
let text = std::fs::read_to_string(path).ok()?;
let text = read_init_marker(path)?;
let _ = std::fs::remove_file(path);
Some(text)
}
/// What the init marker `path` recorded, if it is present — without removing it. For sessions
/// that must leave the disk as they found it (`--memory`, issues #164 and #169): GPU rendering
/// still starts off, and the marker stays for the next ordinary launch to report and clear.
pub fn read_init_marker(path: &std::path::Path) -> Option<String> {
let text = std::fs::read_to_string(path).ok()?;
Some(text.trim().to_string())
}
@@ -225,10 +233,13 @@ mod tests {
set_init_marker(None);
assert!(seen, "the marker exists while the device is created");
assert!(!m.exists(), "and is removed afterwards");
// a marker left behind by a crashed process is reported once
// a marker left behind by a crashed process is reported once; reading it leaves it in place
std::fs::write(&m, "GPU device creation started (backends DX12)").unwrap();
assert!(read_init_marker(&m).unwrap().contains("DX12"));
assert!(m.exists(), "read_init_marker leaves the marker for the next launch");
assert!(take_init_marker(&m).unwrap().contains("DX12"));
assert_eq!(take_init_marker(&m), None);
assert_eq!(read_init_marker(&m), None);
let _ = std::fs::remove_dir_all(&dir);
}
}
+3
View File
@@ -294,6 +294,9 @@ pub fn stage_bytes(stages: &StageCache) -> usize {
pub fn render(src: &Arc<Rgb32f>, info: &SourceInfo, s: &DevelopSettings, req: &RenderRequest, stages: Option<&StageCache>) -> Option<Rendered> {
#[cfg(not(target_arch = "wasm32"))]
{
// sections switched off with their eye render as if at their defaults (issue #316)
let effective = s.effective();
let s: &DevelopSettings = &effective;
// the kernel writes 8-bit output: high-bit-depth exports (and soft proofs) render on the CPU
if !enabled() || req.depth != lightcraft_pipeline::OutputDepth::U8 || req.proof.is_some() {
return None;
+1 -1
View File
@@ -905,7 +905,7 @@ fn masks(cx: &mut Cx<'_>, lin: &Buf, prep: &Prep, plan: &Plan<'_>, host: &mut Ho
Some(2)
}
MaskShape::ColorRange { samples, refine } => {
let tol = 0.04 + 0.16 * (*refine as f32 / 100.0);
let tol = lightcraft_pipeline::masks::color_range_tolerance(*refine);
p.extend([tol.to_bits(), ev.exp2().to_bits(), samples.len() as u32]);
aux.extend(samples.iter().flat_map(|s| s.map(|v| v as f32)));
Some(3)
+21
View File
@@ -639,3 +639,24 @@ fn output_spaces_match() {
}
}
}
#[test]
fn tint_directions_match_cpu_for_raw_and_rendered_sources() {
if !gpu() {
return;
}
let src = Arc::new(Rgb32f::filled(64, 64, [0.18; 3]));
for info in [
SourceInfo::default(),
SourceInfo { raw: true, relative_wb: true, ..Default::default() },
SourceInfo { raw: true, as_shot_temp: 4200.0, as_shot_tint: 15.0, ..Default::default() },
] {
for delta in [-50.0, 50.0] {
let mut s = DevelopSettings::default();
s.wb.mode = WbMode::Custom;
s.wb.temp = info.as_shot_temp;
s.wb.tint = info.as_shot_tint + delta;
check("tint direction", &src, &info, &s, &RenderRequest::fit(64, 64));
}
}
}
+11 -21
View File
@@ -13,7 +13,8 @@ use crate::backend::Backend;
/// File extensions recognised as photos when expanding folders.
pub const PHOTO_EXTENSIONS: &[&str] = &[
"jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "arw", "raf", "orf", "rw2", "rwl", "raw", "pef", "psd", "jxl", "gif",
"bmp", "avif",
"bmp", "avif", // containers LightCraft cannot decode but imports as preview only (their embedded JPEG)
"iiq", "crw", "mrw", "x3f", "kdc", "mos", "erf",
];
/// Headless backend: a [`Session`] with filesystem hooks.
@@ -61,7 +62,7 @@ impl Headless {
fn export(&mut self, p: &Value) -> Result<Value, String> {
use lightcraft_engine::export::{Destination, ExportFormat, ExportOptions, Resize, export_batch};
let p = &self.session.export_params(p)?;
let mut opts = ExportOptions::from_json(p);
let mut opts = ExportOptions::from_params(p).map_err(|e| e.to_string())?;
if !ExportOptions::has_size_param(p) {
opts.resize = Some(Resize::long_edge(3000));
}
@@ -107,7 +108,7 @@ impl Backend for Headless {
"engine.commands" => {
let mut v: Vec<Value> = self.session.commands().into_iter().map(|c| serde_json::to_value(c).unwrap_or_default()).collect();
v.push(json!({"id": "app.export", "label": "Export Now", "menu": [], "shortcut": null,
"params": "{path?: output file (.jpg/.png/.tif/.webp/.avif/.dng) | dir?, ids?, format?: jpeg|png|tiff|webp|avif|dng|original, longEdge?|shortEdge?|width?|height?|megapixels?|percent? (default longEdge 3000; longEdge 0 = full size), dontEnlarge?, ppi?, quality?, limitKb?, colorSpace?, bitDepth?, sharpen?, metadata?, watermark?, naming?}",
"params": "{path?: output file (.jpg/.png/.tif/.webp/.avif/.dng) | dir?, ids?, preset?, format?: jpeg|png|tiff|webp|avif|dng|original, longEdge?|shortEdge?|width?|height?|megapixels?|percent? (default longEdge 3000; longEdge 0 = full size), dontEnlarge?, ppi?, quality?: 1..100, limitKb?, colorSpace?: srgb|displayP3|adobeRgb|proPhoto|rec2020, bitDepth?: 8|10|16|32, sharpen?: none|screen|matte|glossy, sharpenAmount?: low|standard|high, metadata?: all|allExceptCamera|copyright|none, removeLocation?, naming?, startNumber?, subfolder?, conflict?: unique|overwrite|skip, tiffCompression?: none|lzw|zip, dngCompression?: lossless|deflate|uncompressed, watermark?: text | {text?, vertical? (upright columns, right to left), size? (text height, 0.005..0.5 of the short edge; default 0.035), opacity? (0..1; 0.7), anchor?: topLeft|top|topRight|left|center|right|bottomLeft|bottom|bottomRight, inset? (margin, 0..0.4 of the short edge; 0.025), color? [r,g,b] sRGB, shadow?, image? (graphic drawn instead of the text), imageWidth? (0.01..1 of the photo's width; 0.2)}} — an unknown parameter, an out-of-range watermark size or a value of the wrong kind is an error, not a default",
"enabled": self.session.active().is_some()}));
Ok(Value::Array(v))
}
@@ -139,31 +140,20 @@ impl Backend for Headless {
}
}
/// Expand folders (recursively, sorted) into photo files and make paths absolute.
/// Expand folders (recursively, sorted; bounded, see `lightcraft_engine::walk`) into photo files
/// and make paths absolute.
pub fn expand_paths(paths: &[String]) -> Vec<String> {
fn walk(p: &Path, out: &mut Vec<String>) {
if p.is_dir() {
if let Ok(rd) = std::fs::read_dir(p) {
let mut v: Vec<_> = rd.flatten().map(|e| e.path()).collect();
v.sort();
for c in v {
if c.file_name().is_some_and(|n| !n.to_string_lossy().starts_with('.')) {
walk(&c, out);
}
}
}
} else if p.extension().is_some_and(|e| PHOTO_EXTENSIONS.contains(&e.to_string_lossy().to_lowercase().as_str())) {
out.push(std::path::absolute(p).unwrap_or_else(|_| p.to_path_buf()).to_string_lossy().to_string());
}
}
let absolute = |p: &Path| std::path::absolute(p).unwrap_or_else(|_| p.to_path_buf()).to_string_lossy().to_string();
let photo = |p: &Path| p.extension().is_some_and(|e| PHOTO_EXTENSIONS.contains(&e.to_string_lossy().to_lowercase().as_str()));
let mut out = Vec::new();
for p in paths {
let path = Path::new(p);
if path.is_dir() {
walk(path, &mut out);
let w = lightcraft_engine::walk::files_in(path, None, lightcraft_engine::walk::Limits::default(), photo);
out.extend(w.files.iter().map(|f| absolute(f)));
} else {
// Explicit files are kept even with unknown extensions (the probe decides).
out.push(std::path::absolute(path).unwrap_or_else(|_| path.to_path_buf()).to_string_lossy().to_string());
out.push(absolute(path));
}
}
out
+40
View File
@@ -120,6 +120,25 @@ fn path_writes_never_replace_an_original() {
let _ = std::fs::remove_dir_all(&base);
}
/// Issue #181: a misspelled or unreadable `app.export` param is an error through the headless
/// backend (what `lightcraft-cli run` and the MCP `export` tool use), not a silent default.
#[test]
fn export_refuses_unknown_params_and_bad_values() {
let mut b = Headless::demo();
let out = "/nonexistent-lc-test/x.jpg";
let e = b.call("app.export", json!({"path": out, "longEdgee": 400})).unwrap_err();
assert!(e.contains("unknown parameter `longEdgee` (did you mean `longEdge`?)"), "{e}");
let e = b.call("app.export", json!({"path": out, "longEdge": "banana"})).unwrap_err();
assert!(e.contains("`longEdge` must be a number"), "{e}");
let e = b.call("app.export", json!({"path": out, "watermark": {"text": "x", "size": 3}})).unwrap_err();
assert!(e.contains("`watermark.size` must be a number 0.005..0.5"), "{e}");
// `export` tool and preset expansion go through the same check
let r = call_tool(&mut b, "export", &json!({"path": out, "quality": 101}));
assert!(r.is_error, "{r:?}");
let e = b.session.execute("export.savePreset", &json!({"name": "Typo", "params": {"qualty": 5}})).unwrap_err().to_string();
assert!(e.contains("did you mean `quality`"), "{e}");
}
/// The `import` helper moves: renamed into the folder template, the source removed.
#[test]
fn import_tool_moves_with_a_folder_template() {
@@ -205,3 +224,24 @@ fn resources() {
}
assert_eq!(rpc(&mut s, 99, "resources/read", json!({"uri": "lightcraft://nope"}))["error"]["code"], -32002);
}
/// `select_photos` with an id that is not in the library is a tool error, and the photo that was
/// active stays active (#182).
#[test]
fn select_photos_rejects_unknown_ids_and_keeps_the_active_photo() {
let mut b = Headless::demo();
let first = b.session.visible_cloned()[0];
let r = call_tool(&mut b, "select_photos", &json!({"ids": [first.0]}));
assert!(!r.is_error, "{r:?}");
let r = call_tool(&mut b, "select_photos", &json!({"ids": [9999]}));
assert!(r.is_error, "{r:?}");
assert!(r.content[0]["text"].as_str().unwrap().contains("no such photo 9999"), "{r:?}");
assert_eq!(b.session.active(), Some(first));
let r = call_tool(&mut b, "get_develop", &json!({}));
assert!(!r.is_error, "{r:?}");
let r = call_tool(&mut b, "get_develop", &json!({"id": 9999}));
assert!(r.is_error, "{r:?}");
let r = call_tool(&mut b, "set_develop", &json!({"id": 9999, "values": {"light.exposure": 1.0}}));
assert!(r.is_error, "{r:?}");
assert_eq!(b.session.active(), Some(first));
}
+3 -2
View File
@@ -102,6 +102,7 @@ pub fn helper_tools(has_ui: bool) -> Vec<Value> {
"mode": {"type": "string", "enum": ["add", "copy", "move"], "description": "add (default) = reference the files in place; copy = copy them into `destination` (default: the library's Originals/); move = as copy, then each original and its XMP sidecars are removed from the source once the copy is verified and catalogued (duplicates and failures keep their sources; `kept` lists sources left in place and why; undo leaves the moved files at the destination)"},
"destination": {"type": "string", "description": "Copy or move: destination folder"},
"organize": {"type": "string", "description": "Copy or move: folders inside the destination — date (YYYY/YYYY-MM-DD, default), month (YYYY/YYYY-MM), flat, or a folder template such as {date:%Y}/{date:%Y%m%d} (→ 2026/20260114; the template's / make the levels, tokens as for rename; relative, no ..). Dated by capture time, else the import time"},
"onDeleted": {"type": "string", "enum": ["skip", "restore", "fresh"], "description": "A file that is in Recently Deleted: skip (default) = leave it there (the duplicate is reported with existingDeleted); restore = bring the photo back with its edits (`restored` lists it); fresh = delete the trashed record and import the file as a new photo"},
"rename": {"type": "string", "description": "Copy or move: file-name template, e.g. {date:%Y%m%d_%H%M%S}_{seq:3} (run_command photo.renameTokens lists the tags); the extension is kept"}
}),
&["paths"],
@@ -205,7 +206,7 @@ pub fn helper_tools(has_ui: bool) -> Vec<Value> {
"removeLocation": {"type": "boolean"},
"colorSpace": {"type": "string", "enum": ["srgb", "displayP3", "adobeRgb", "proPhoto", "rec2020"], "description": "Output colour space (default sRGB; AVIF is always sRGB). adobeRgb = Adobe RGB (1998) compatible; the embedded ICC profile is generated from the published primaries"},
"bitDepth": {"type": "integer", "enum": [8, 10, 16, 32], "description": "Bits per channel: PNG 8|16 (default 8), TIFF 8|16|32 (default 16; 32 = linear float with a linear profile), AVIF 8|10; JPEG/WebP are 8-bit"},
"watermark": {"description": "Text, or {text, vertical (boolean; defaults to false), size (fraction of short edge), opacity, anchor (topLeft|top|topRight|left|center|right|bottomLeft|bottom|bottomRight), inset, color [r,g,b], shadow}"}
"watermark": {"description": "Text, or {text, vertical (boolean; defaults to false: upright columns right to left), size (text height as a fraction of the short edge, 0.005..0.5; default 0.035), opacity (0..1; default 0.7), anchor (topLeft|top|topRight|left|center|right|bottomLeft|bottom|bottomRight), inset (margin as a fraction of the short edge, 0..0.4; default 0.025), color [r,g,b] (sRGB 0..255), shadow (boolean), image (path of a graphic drawn instead of the text), imageWidth (fraction of the photo's width, 0.01..1; default 0.2)}. Unknown keys and out-of-range sizes are errors"}
}),
&[],
),
@@ -475,7 +476,7 @@ pub fn call_tool(b: &mut dyn Backend, name: &str, args: &Value) -> ToolResult {
return ToolResult::error(format!("no photos found in {paths:?}"));
}
let mut p = json!({"paths": files});
for k in ["album", "mode", "destination", "organize", "rename"] {
for k in ["album", "mode", "destination", "organize", "rename", "onDeleted"] {
if let Some(v) = args.get(k).filter(|v| !v.is_null()) {
p[k] = v.clone();
}
+1 -3
View File
@@ -111,9 +111,7 @@ pub fn load_frame(bytes: &[u8], max_edge: Option<usize>, orient: bool) -> Result
if lightcraft_raw::probe(bytes).is_some() {
let mut raw = lightcraft_raw::decode(bytes).map_err(|e| MergeError::Decode(e.to_string()))?;
// geometric lens corrections stay with the develop settings ("profile corrections")
raw.opcodes
.list3
.retain(|op| !matches!(op, lightcraft_raw::Opcode::WarpRectilinear { .. } | lightcraft_raw::Opcode::FixVignetteRadial { .. }));
raw.opcodes.list3.retain(|op| !op.is_lens_correction());
let small = max_edge.is_some_and(|m| m <= 1200);
let method = if small { lightcraft_raw::Method::Bilinear } else { lightcraft_raw::Method::Ahd };
let mut image = raw.develop(method).map_err(|e| MergeError::Decode(e.to_string()))?;
+405 -12
View File
@@ -17,7 +17,13 @@ pub enum Cr3TrackKind {
/// `CRAW` with a `JPEG` child: the full-size JPEG.
Jpeg,
/// `CRAW` raw image: dimensions and the `CMP1` coding header (byte range in the file).
Raw { width: u16, height: u16, cmp1: Option<(usize, usize)> },
Raw {
width: u16,
height: u16,
cmp1: Option<(usize, usize)>,
/// `CDI1/IAD1` image-area descriptor (byte range in the file).
iad1: Option<(usize, usize)>,
},
/// Anything else (`CTMD` timed metadata, unknown entries).
Other([u8; 4]),
}
@@ -30,6 +36,134 @@ pub struct Cr3Track {
pub data: Option<(usize, usize)>,
}
/// Canon's `CMP1` image coding descriptor, separate from the encoded track sample.
///
/// Parsed from the payload (after the eight-byte box header). Geometry is the full sensor mosaic,
/// including optically masked borders. Unsupported encoding modes are retained for the decoder to
/// report explicitly, rather than confused with the full-size embedded JPEG.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Cr3Compression {
pub version: u16,
pub width: u32,
pub height: u32,
pub tile_width: u32,
pub tile_height: u32,
pub bit_depth: u8,
pub planes: u8,
/// 0 = RGGB, 1 = GRBG, 2 = GBRG, 3 = BGGR.
pub cfa_pattern: u8,
pub encoding: u8,
pub levels: u8,
pub tile_flags: u8,
/// Number of bytes at the start of the sample before its entropy-coded data.
pub header_size: u32,
pub median_bit_depth: Option<u8>,
}
impl Cr3Compression {
/// Read the public `CMP1` field layout. Truncated or impossible descriptors return `None`.
pub fn parse(payload: &[u8]) -> Option<Self> {
// The first word is reserved in the public notes; real files use FF00, although older
// descriptions guessed FFFF. It is not a reliable format signature.
let header_len = usize::from(be16(payload, 2)?).checked_add(4)?;
if header_len < 52 || header_len > payload.len() {
return None;
}
let packed = *payload.get(25)?;
let coding = *payload.get(26)?;
let mut out = Self {
version: be16(payload, 4)?,
width: be32(payload, 8)?,
height: be32(payload, 12)?,
tile_width: be32(payload, 16)?,
tile_height: be32(payload, 20)?,
bit_depth: *payload.get(24)?,
planes: packed >> 4,
cfa_pattern: packed & 15,
encoding: coding >> 4,
levels: coding & 15,
tile_flags: *payload.get(27)?,
header_size: be32(payload, 28)?,
median_bit_depth: None,
};
if out.width == 0
|| out.height == 0
|| out.tile_width == 0
|| out.tile_height == 0
|| out.tile_width > out.width
|| out.tile_height > out.height
|| !(1..=16).contains(&out.bit_depth)
|| !(1..=4).contains(&out.planes)
|| (out.planes > 1 && out.cfa_pattern > 3)
|| out.header_size == 0
{
return None;
}
if payload.get(32).is_some_and(|flags| flags & 0x80 != 0) {
// Extended CMP1: a four-byte length, flags, reserved bytes and six words. The optional
// median precision is used by type-3 burst encodings, not ordinary Bayer raw/craw.
let extension_len = usize::try_from(be32(payload, 52)?).ok()?;
let extension_end = 52usize.checked_add(extension_len)?;
if extension_len < 32 || extension_end > payload.len() {
return None;
}
if payload.get(56).is_some_and(|flags| flags & 0x40 != 0) {
let precision = *payload.get(84).filter(|_| extension_end > 84)?;
if !(1..=16).contains(&precision) {
return None;
}
out.median_bit_depth = Some(precision);
}
}
Some(out)
}
}
/// `IAD1` geometry. Coordinates are inclusive sensor offsets: `[left, top, right, bottom]`.
/// A full raw descriptor distinguishes valid sensor pixels from the recommended final crop.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Cr3ImageArea {
pub width: u16,
pub height: u16,
pub crop: [u16; 4],
pub active: Option<[u16; 4]>,
pub masked_left: [u16; 4],
pub masked_top: Option<[u16; 4]>,
}
impl Cr3ImageArea {
pub fn parse(payload: &[u8]) -> Option<Self> {
let bounds = |at: usize| Some([be16(payload, at)?, be16(payload, at + 2)?, be16(payload, at + 4)?, be16(payload, at + 6)?]);
let width = be16(payload, 4)?;
let height = be16(payload, 6)?;
if width == 0 || height == 0 {
return None;
}
let full = be16(payload, 10)? == 2;
Some(Self {
width,
height,
crop: bounds(16)?,
masked_left: bounds(24)?,
masked_top: if full { Some(bounds(32)?) } else { None },
active: if full { Some(bounds(40)?) } else { None },
})
}
}
impl Cr3Track {
/// Decode this track's image coding descriptor without reading its pixel data.
pub fn compression(&self, bytes: &[u8]) -> Option<Cr3Compression> {
let Cr3TrackKind::Raw { cmp1: Some((at, len)), .. } = self.kind else { return None };
Cr3Compression::parse(bytes.get(at..at.checked_add(len)?)?)
}
pub fn image_area(&self, bytes: &[u8]) -> Option<Cr3ImageArea> {
let Cr3TrackKind::Raw { iad1: Some((at, len)), .. } = self.kind else { return None };
Cr3ImageArea::parse(bytes.get(at..at.checked_add(len)?)?)
}
}
/// The parts of a CR3 file LightCraft reads.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct Cr3<'a> {
@@ -40,6 +174,57 @@ pub struct Cr3<'a> {
pub tracks: Vec<Cr3Track>,
}
/// One independent TIFF block from Canon timed metadata. Its offsets remain relative to this block.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Cr3TimedExif<'a> {
/// CTMD record type (7, 8 or 9).
pub record_type: u16,
/// Exif IFD (`0x8769`) or Canon maker note (`0x927c`).
pub tag: u32,
pub data: &'a [u8],
}
impl Cr3<'_> {
/// Read TIFF blocks in the first samples of CTMD tracks. As-shot white balance and sensor
/// levels often live here rather than in the static `CMT3` maker note. Both record layers use
/// little-endian byte lengths; TIFF blocks retain their own byte order and offset origin.
pub fn timed_exif<'a>(&self, bytes: &'a [u8]) -> Vec<Cr3TimedExif<'a>> {
let mut out = Vec::new();
let mut budget = MAX_BOXES;
for track in &self.tracks {
if track.kind != Cr3TrackKind::Other(*b"CTMD") {
continue;
}
let Some(sample) = track.data.and_then(|(at, len)| at.checked_add(len).and_then(|end| bytes.get(at..end))) else { continue };
let mut at = 0usize;
while budget > 0 {
budget -= 1;
let Some(len) = le32(sample, at).and_then(|v| usize::try_from(v).ok()) else { break };
let Some(end) = at.checked_add(len).filter(|&end| len >= 12 && end <= sample.len()) else { break };
let Some(record) = sample.get(at..end) else { break };
let Some(kind) = le16(record, 4) else { break };
if matches!(kind, 7..=9) {
let mut item_at = 12usize;
while budget > 0 {
budget -= 1;
let Some(item_len) = le32(record, item_at).and_then(|v| usize::try_from(v).ok()) else { break };
let Some(item_end) = item_at.checked_add(item_len).filter(|&end| item_len >= 8 && end <= record.len()) else { break };
let Some(item) = record.get(item_at..item_end) else { break };
if let Some(tag @ (0x8769 | 0x927c)) = le32(item, 4)
&& let Some(data) = item.get(8..)
{
out.push(Cr3TimedExif { record_type: kind, tag, data });
}
item_at = item_end;
}
}
at = end;
}
}
out
}
}
const CANON_UUID: [u8; 16] = [0x85, 0xc0, 0xb6, 0x87, 0x82, 0x0f, 0x11, 0xe0, 0x81, 0x11, 0xf4, 0xce, 0x46, 0x2b, 0x6a, 0x48];
const XMP_UUID: [u8; 16] = [0xbe, 0x7a, 0xcf, 0xcb, 0x97, 0xa9, 0x42, 0xe8, 0x9c, 0x71, 0x99, 0x94, 0x91, 0xe3, 0xaf, 0xac];
const MAX_DEPTH: usize = 12;
@@ -158,19 +343,21 @@ fn trak(bytes: &[u8], t: &BoxRef, budget: &mut usize) -> Option<Cr3Track> {
let stbl = stbl?;
let (mut kind, mut size, mut offset) = (None, None, None);
for b in boxes(bytes, stbl.body, stbl.end, budget) {
let body = bytes.get(b.body..b.end)?;
match &b.kind {
// full box: version/flags, entry count, entries
b"stsd" => kind = boxes(bytes, b.body + 8, b.end, budget).first().map(|e| sample_entry(bytes, e, budget)),
b"stsd" if be32(body, 4)? > 0 => kind = boxes(bytes, b.body + 8, b.end, budget).first().map(|e| sample_entry(bytes, e, budget)),
// version/flags, sample_size (0 = per-sample table), count, sizes
b"stsz" => {
size = match be32(bytes, b.body + 4)? {
0 if be32(bytes, b.body + 8)? > 0 => be32(bytes, b.body + 12),
size = match be32(body, 4)? {
0 if be32(body, 8)? > 0 => be32(body, 12),
0 => None,
n => Some(n),
n if be32(body, 8)? > 0 => Some(n),
_ => None,
}
}
b"co64" if be32(bytes, b.body + 4)? > 0 => offset = be64(bytes, b.body + 8).and_then(|v| usize::try_from(v).ok()),
b"stco" if be32(bytes, b.body + 4)? > 0 => offset = be32(bytes, b.body + 8).map(|v| v as usize),
b"co64" if be32(body, 4)? > 0 => offset = be64(body, 8).and_then(|v| usize::try_from(v).ok()),
b"stco" if be32(body, 4)? > 0 => offset = be32(body, 8).map(|v| v as usize),
_ => {}
}
}
@@ -185,18 +372,37 @@ fn sample_entry(bytes: &[u8], e: &BoxRef, budget: &mut usize) -> Cr3TrackKind {
if &e.kind != b"CRAW" {
return Cr3TrackKind::Other(e.kind);
}
let (Some(width), Some(height)) = (be16(bytes, e.body + 24), be16(bytes, e.body + 26)) else {
let Some(body) = bytes.get(e.body..e.end) else { return Cr3TrackKind::Other(e.kind) };
let (Some(width), Some(height)) = (be16(body, 24), be16(body, 26)) else {
return Cr3TrackKind::Other(e.kind);
};
let mut cmp1 = None;
let (mut cmp1, mut iad1, mut rendered) = (None, None, None);
for c in boxes(bytes, e.body.saturating_add(CRAW_CHILDREN_AT), e.end, budget) {
match &c.kind {
b"JPEG" => return Cr3TrackKind::Jpeg,
b"JPEG" => rendered = Some(Cr3TrackKind::Jpeg),
// HDR-capable cameras also put an HEVC preview in a CRAW entry,
// often with the same displayed dimensions as the main raw track.
b"HEVC" => rendered = Some(Cr3TrackKind::Other(c.kind)),
b"CMP1" => cmp1 = Some((c.body, c.end - c.body)),
b"CDI1" => {
for area in boxes(bytes, c.body.saturating_add(4), c.end, budget) {
if &area.kind == b"IAD1" {
iad1 = Some((area.body, area.end - area.body));
}
}
}
_ => {}
}
}
Cr3TrackKind::Raw { width, height, cmp1 }
// A present, malformed CMP1 must remain a raw descriptor so its corruption
// is reported. Only an explicitly identified rendered codec can establish
// that a CRAW entry without CMP1 is a preview rather than a broken raw.
if cmp1.is_none()
&& let Some(kind) = rendered
{
return kind;
}
Cr3TrackKind::Raw { width, height, cmp1, iad1 }
}
fn be16(b: &[u8], at: usize) -> Option<u16> {
@@ -212,6 +418,16 @@ fn be64(b: &[u8], at: usize) -> Option<u64> {
Some(u64::from_be_bytes(a))
}
fn le16(b: &[u8], at: usize) -> Option<u16> {
let s = b.get(at..at.checked_add(2)?)?;
Some(u16::from_le_bytes([s[0], s[1]]))
}
fn le32(b: &[u8], at: usize) -> Option<u32> {
let s = b.get(at..at.checked_add(4)?)?;
Some(u32::from_le_bytes([s[0], s[1], s[2], s[3]]))
}
/// Merge `CMT1` (IFD0), `CMT2` (Exif) and `CMT4` (GPS) into one TIFF stream, so the Exif readers see a CR3
/// like any TIFF-based raw. The maker note (`CMT3`) is left out: its offsets are relative to its own block.
pub fn merged_exif(c: &Cr3<'_>) -> Option<Vec<u8>> {
@@ -308,12 +524,50 @@ mod tests {
assert_eq!(c.xmp, Some(b"<x:xmpmeta/>".as_slice()));
assert_eq!(c.tracks.len(), 2);
assert_eq!(c.tracks[0], Cr3Track { kind: Cr3TrackKind::Jpeg, data: Some((4096, 16)) });
let Cr3TrackKind::Raw { width, height, cmp1: Some((at, len)) } = c.tracks[1].kind else { panic!("{:?}", c.tracks[1]) };
let Cr3TrackKind::Raw { width, height, cmp1: Some((at, len)), .. } = c.tracks[1].kind else { panic!("{:?}", c.tracks[1]) };
assert_eq!((width, height, len), (60, 40, 4));
assert_eq!(&f[at..at + 2], &[0xff, 0x10]);
assert_eq!(c.tracks[1].data, Some((4112, 32)));
}
#[test]
fn hevc_preview_is_distinct_from_an_equal_size_raw_track() {
let preview = trak(craw(6000, 4000, &bx(b"HEVC", &[0; 4])), 4096, 16);
let reduced = trak(craw(1624, 1080, &bx(b"CMP1", &compression_payload(0))), 4112, 16);
let sensor = trak(craw(6000, 4000, &bx(b"CMP1", &compression_payload(0))), 4128, 16);
let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom");
file.extend(bx(b"moov", &[preview, reduced, sensor].concat()));
file.resize(4144, 0);
let container = parse_cr3(&file).unwrap();
assert_eq!(container.tracks[0].kind, Cr3TrackKind::Other(*b"HEVC"));
assert!(container.tracks[0].compression(&file).is_none());
assert!(matches!(container.tracks[2].kind, Cr3TrackKind::Raw { width: 6000, height: 4000, cmp1: Some(_), .. }));
assert_eq!(container.tracks[2].compression(&file).unwrap().width, 6888);
}
#[test]
fn rendered_child_does_not_hide_a_malformed_raw_descriptor() {
for codec in [b"JPEG", b"HEVC"] {
for first in [true, false] {
let rendered = bx(codec, &[0; 4]);
let corrupt = bx(b"CMP1", &[0xff, 0]);
let children = if first { [rendered, corrupt] } else { [corrupt, rendered] }.concat();
let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom");
file.extend(bx(b"moov", &trak(craw(6000, 4000, &children), 4096, 16)));
file.resize(4112, 0);
let container = parse_cr3(&file).unwrap();
assert!(matches!(container.tracks[0].kind, Cr3TrackKind::Raw { cmp1: Some(_), .. }));
assert!(container.tracks[0].compression(&file).is_none());
}
}
// Absence alone is not evidence of a preview: retain a broken raw so
// main-track selection cannot quietly fall back to a reduced image.
let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom");
file.extend(bx(b"moov", &trak(craw(6000, 4000, &bx(b"free", &[])), 4096, 16)));
file.resize(4112, 0);
assert!(matches!(parse_cr3(&file).unwrap().tracks[0].kind, Cr3TrackKind::Raw { cmp1: None, .. }));
}
#[test]
fn merged_exif_reads_like_a_tiff_raw() {
let f = sample();
@@ -322,6 +576,145 @@ mod tests {
assert_eq!(m.iso, Some(400));
}
fn compression_payload(levels: u8) -> Vec<u8> {
let mut b = vec![0u8; 52];
b[0..2].copy_from_slice(&0xff00u16.to_be_bytes());
b[2..4].copy_from_slice(&48u16.to_be_bytes());
b[4..6].copy_from_slice(&0x100u16.to_be_bytes());
for (at, value) in [(8, 6888u32), (12, 4546), (16, 3444), (20, 4546), (28, 216)] {
b[at..at + 4].copy_from_slice(&value.to_be_bytes());
}
b[24] = 14;
b[25] = 0x40;
b[26] = levels;
b
}
#[test]
fn compression_descriptor_retains_sensor_geometry_and_cfa() {
let b = compression_payload(0);
let h = Cr3Compression::parse(&b).unwrap();
assert_eq!((h.width, h.height, h.tile_width, h.tile_height), (6888, 4546, 3444, 4546));
assert_eq!((h.version, h.bit_depth, h.planes, h.cfa_pattern, h.encoding, h.levels, h.header_size), (0x100, 14, 4, 0, 0, 0, 216));
let mut craw = compression_payload(3);
craw[25] = 0x41;
craw[27] = 0x80;
let h = Cr3Compression::parse(&craw).unwrap();
assert_eq!((h.cfa_pattern, h.levels, h.tile_flags), (1, 3, 0x80));
for len in 0..b.len() {
assert!(Cr3Compression::parse(&b[..len]).is_none());
}
for (at, value) in [(24, 0), (24, 17), (25, 0), (25, 0x44)] {
let mut bad = b.clone();
bad[at] = value;
assert!(Cr3Compression::parse(&bad).is_none());
}
}
#[test]
fn extended_descriptor_checks_optional_median_precision() {
let mut b = compression_payload(0x30);
b[32] = 0x80;
b.extend_from_slice(&33u32.to_be_bytes());
b.resize(85, 0);
b[56] = 0xc0;
b[84] = 14;
assert_eq!(Cr3Compression::parse(&b).unwrap().median_bit_depth, Some(14));
assert!(Cr3Compression::parse(&b[..84]).is_none());
b[84] = 0;
assert!(Cr3Compression::parse(&b).is_none());
}
#[test]
fn raw_track_reads_distinct_active_area_and_default_crop() {
let mut area = vec![0u8; 48];
for (at, value) in [
(4, 6888u16),
(6, 4546),
(10, 2),
(16, 156),
(18, 158),
(20, 6875),
(22, 4537),
(24, 0),
(26, 0),
(28, 143),
(30, 4545),
(32, 144),
(34, 0),
(36, 6887),
(38, 45),
(40, 144),
(42, 46),
(44, 6887),
(46, 4545),
] {
area[at..at + 2].copy_from_slice(&value.to_be_bytes());
}
let cmp1 = bx(b"CMP1", &compression_payload(0));
let iad1 = full(b"CDI1", &bx(b"IAD1", &area));
let track = trak(craw(6888, 4546, &[cmp1, iad1].concat()), 4096, 32);
let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom");
file.extend(bx(b"moov", &track));
file.resize(4128, 0);
let c = parse_cr3(&file).unwrap();
assert_eq!(c.tracks[0].compression(&file).unwrap().width, 6888);
let geometry = c.tracks[0].image_area(&file).unwrap();
assert_eq!(geometry.crop, [156, 158, 6875, 4537]);
assert_eq!(geometry.active, Some([144, 46, 6887, 4545]));
assert_eq!(geometry.masked_left, [0, 0, 143, 4545]);
for len in 0..area.len() {
assert!(Cr3ImageArea::parse(&area[..len]).is_none());
}
}
#[test]
fn truncated_sample_table_does_not_read_the_next_box() {
let stbl = [
full(b"stsd", &[1u32.to_be_bytes().to_vec(), craw(16, 16, &bx(b"CMP1", &compression_payload(0)))].concat()),
bx(b"stsz", &[]),
full(b"co64", &[1u32.to_be_bytes().to_vec(), 256u64.to_be_bytes().to_vec()].concat()),
]
.concat();
let mut file = bx(b"ftyp", b"crx \0\0\0\x01crx isom");
file.extend(bx(b"moov", &bx(b"trak", &bx(b"mdia", &bx(b"minf", &bx(b"stbl", &stbl))))));
file.resize(512, 0);
assert!(parse_cr3(&file).unwrap().tracks.is_empty());
}
#[test]
fn timed_metadata_exposes_the_dynamic_maker_note() {
use lightcraft_tiff::{Value, tags as t};
let exif = tiff_with(t::ISO_SPEED, Value::Short(vec![800]));
let maker = tiff_with(0x4001, Value::Short(vec![48, 100, 200]));
let item = |tag: u32, data: &[u8]| {
let mut b = ((data.len() + 8) as u32).to_le_bytes().to_vec();
b.extend_from_slice(&tag.to_le_bytes());
b.extend_from_slice(data);
b
};
let items = [item(0x8769, &exif), item(0x927c, &maker)].concat();
let mut sample = vec![0u8; 12];
sample[0..4].copy_from_slice(&((12 + items.len()) as u32).to_le_bytes());
sample[4..6].copy_from_slice(&8u16.to_le_bytes());
sample.extend(items);
let container = Cr3 { tracks: vec![Cr3Track { kind: Cr3TrackKind::Other(*b"CTMD"), data: Some((0, sample.len())) }], ..Default::default() };
let parts = container.timed_exif(&sample);
assert_eq!(parts.len(), 2);
assert_eq!((parts[0].record_type, parts[0].tag, parts[0].data), (8, 0x8769, exif.as_slice()));
assert_eq!((parts[1].record_type, parts[1].tag, parts[1].data), (8, 0x927c, maker.as_slice()));
for len in 0..sample.len() {
assert!(container.timed_exif(&sample[..len]).is_empty());
}
for at in [0, 12] {
let mut corrupt = sample.clone();
corrupt[at..at + 4].copy_from_slice(&u32::MAX.to_le_bytes());
assert!(container.timed_exif(&corrupt).is_empty());
corrupt[at..at + 4].copy_from_slice(&0u32.to_le_bytes());
assert!(container.timed_exif(&corrupt).is_empty());
}
}
#[test]
fn hostile_input_never_panics() {
let f = sample();
+2 -2
View File
@@ -5,8 +5,8 @@ proptest! {
#![proptest_config(ProptestConfig { cases: 1500, .. ProptestConfig::default() })]
#[test]
fn random_containers_never_panic(mut data in proptest::collection::vec(any::<u8>(), 0..600), kind in 0usize..5) {
let heads: [&[u8]; 5] = [b"\xff\xd8\xff\xe1", b"\x89PNG\r\n\x1a\n", b"RIFF\0\0\0\0WEBP", b"II*\0", b"MM\0*"];
fn random_containers_never_panic(mut data in proptest::collection::vec(any::<u8>(), 0..600), kind in 0usize..6) {
let heads: [&[u8]; 6] = [b"\xff\xd8\xff\xe1", b"\x89PNG\r\n\x1a\n", b"RIFF\0\0\0\0WEBP", b"II*\0", b"MM\0*", b"\0\0\0\x18ftypcrx "];
let h = heads[kind];
if data.len() >= h.len() {
data[..h.len()].copy_from_slice(h);
+25
View File
@@ -191,3 +191,28 @@ mod bw_tests {
assert_eq!(auto_bw_mix(&grey, &SourceInfo::default(), &DevelopSettings::default()), [0.0; 8]);
}
}
#[cfg(test)]
mod tint_tests {
use super::*;
#[test]
fn auto_wb_and_picker_correct_green_with_positive_tint() {
// The picker delegates a sampled patch to this same auto_wb implementation.
for (rgb, sign) in [([0.18, 0.24, 0.18], 1.0), ([0.24, 0.18, 0.24], -1.0)] {
let img = Rgb32f::filled(16, 16, rgb);
let info = SourceInfo { raw: true, relative_wb: true, ..Default::default() };
let (temp, tint) = auto_wb(&img, &info);
assert!(tint * sign > 0.0, "{rgb:?}: temp {temp}, tint {tint}");
let mut s = DevelopSettings::default();
s.wb.mode = lightcraft_develop::WbMode::Custom;
s.wb.temp = temp;
s.wb.tint = tint;
let mut corrected = img;
crate::local::white_balance(&mut corrected, &info, &s);
let p = corrected.get(0, 0);
let spread = p[0].max(p[1]).max(p[2]) - p[0].min(p[1]).min(p[2]);
assert!(spread < 0.003, "{rgb:?} -> {p:?}");
}
}
}
+3
View File
@@ -365,6 +365,9 @@ enum Src<'a> {
}
fn render_impl(src: Src<'_>, info: &SourceInfo, s: &DevelopSettings, req: &RenderRequest, cache: Option<&StageCache>) -> Rendered {
// sections switched off with their eye render as if at their defaults (issue #316)
let effective = s.effective();
let s: &DevelopSettings = &effective;
// `Instant::now()` panics on wasm32-unknown-unknown: only read the clock when profiling.
let lap = |what: &str, t: &mut Option<std::time::Instant>| {
if let Some(t) = t {
+54 -1
View File
@@ -127,7 +127,7 @@ pub fn shape_alpha(shape: &MaskShape, frame: &Frame, w: usize, h: usize, img: &R
}
}
MaskShape::ColorRange { samples, refine } => {
let tol = 0.04 + 0.16 * (*refine as f32 / 100.0);
let tol = color_range_tolerance(*refine);
let samples: Vec<[f32; 3]> = samples.iter().map(|s| [s[0] as f32, s[1] as f32, s[2] as f32]).collect();
for (v, c) in out.data.iter_mut().zip(&img.data) {
let lab = lightcraft_color::perceptual::oklab_from_2020(tonemap_for_select(c.map(|v| v * gain)));
@@ -257,6 +257,21 @@ pub(crate) fn tonemap_for_select(c: [f32; 3]) -> [f32; 3] {
c.map(|v| v / (1.0 + v))
}
/// Colour range: the OkLab distance (a, b at full weight, L at a quarter) at which a pixel leaves
/// the selection for a Refine amount of 0..100. A pixel is fully selected up to half of it and
/// fades out over the other half (CPU here, `mask.wgsl` kind 3; both take this value).
///
/// Issue #157: `0.04 + 0.16·refine` selected the whole photo. Through [`tonemap_for_select`] a
/// slightly teal mid-grey (sRGB 120,150,155) and a neutral light grey (200,198,195) are only ~0.063
/// apart — L 0.60 vs 0.72, a/b ≤ 0.02 — so at the default Refine 50 (full selection below 0.06)
/// the grey patch was 99 % selected, and at Refine 85 / 100 still most of it. A JND is ~0.01 on
/// this scale, so Refine 50 now holds a surface up to ~2 JND from its sample and drops everything
/// beyond ~4.5 (full to 0.0225, gone at 0.045); Refine 85 is gone at 0.066, past that grey patch;
/// Refine 100 reaches 0.075 and Refine 0 is the sample's own shade (full to 0.0075, gone at 0.015).
pub fn color_range_tolerance(refine: f64) -> f32 {
0.015 + 0.06 * (refine / 100.0).clamp(0.0, 1.0) as f32
}
/// A brush stroke resolved to output pixels: dab centres (spacing r/4 along the path), radius,
/// hard-core radius, flow and density (0..1), and whether it erases.
pub struct BrushDabs {
@@ -598,4 +613,42 @@ mod tests {
let (plain, sharp) = (step(&soft), step(&refined));
assert!(sharp > plain * 1.5 && sharp > 0.1, "the edge in the mask follows the photo's: {plain} → {sharp}");
}
/// Issue #157: a colour-range sample of one flat patch must not select a clearly different
/// patch. The two patches are the ones from the report (sRGB 120,150,155 and 200,198,195),
/// sampled the way `mask.sampleColor` does (3×3 mean of the selection-space OkLab).
#[test]
fn color_range_keeps_to_the_sampled_colour() {
use lightcraft_color::{REC2020, SRGB, transfer::decode_srgb8};
let (w, h) = (120usize, 40usize);
let m = SRGB.to_space(&REC2020);
let patch = |rgb: [u8; 3]| m.apply_f32(rgb.map(decode_srgb8));
let (left, right) = (patch([120, 150, 155]), patch([200, 198, 195]));
let img = Rgb32f::from_fn(w, h, |x, _| if x < 60 { left } else { right });
let log_l = img.map(crate::local::log_lum);
let f = Frame::new(w, h, &Default::default(), true);
let mut sample = [0f64; 3];
for y in 19..=21 {
for x in 29..=31 {
let lab = lightcraft_color::perceptual::oklab_from_2020(tonemap_for_select(img.data[y * w + x]));
for k in 0..3 {
sample[k] += lab[k] as f64 / 9.0;
}
}
}
let alpha = |refine| {
let shape = MaskShape::ColorRange { samples: vec![sample], refine };
let a = shape_alpha(&shape, &f, w, h, &img, &log_l, 0.0);
(a.get(30, 20), a.get(90, 20))
};
for refine in [0.0, 50.0, 85.0] {
let (l, r) = alpha(refine);
assert!(l > 0.95, "refine {refine}: sampled patch {l}");
assert!(r < 0.05, "refine {refine}: other patch {r}");
}
// Refine 100 reaches furthest (the grey is ~0.063 away, the fade ends at 0.075), but the
// other patch stays mostly out
let (l, r) = alpha(100.0);
assert!(l > 0.95 && r < 0.3, "refine 100: {l} vs {r}");
}
}
+2 -1
View File
@@ -11,7 +11,8 @@
//! dimensionless scales), so a 400 px preview and a full-size export are warped identically.
//!
//! Also here: automatic lateral chromatic aberration estimation ([`estimate_lateral_ca`]) and [`defringe`].
//! Profile corrections use only lens data embedded in DNG files (`WarpRectilinear` / `FixVignetteRadial`).
//! Profile corrections use only lens data embedded in the files: DNG `WarpRectilinear` / `FixVignetteRadial`, and the
//! raw readers' equivalent of a camera's own correction (Panasonic / Leica RW2 distortion).
use std::sync::Mutex;
+54
View File
@@ -237,6 +237,35 @@ fn exposure_after_spatial_filters_equals_exposing_the_source() {
assert!(d <= 2, "max difference {d}");
}
/// A section whose eye is off renders as if it were at its defaults (issue #316): Light, Color
/// and Detail, which the pipeline doesn't switch off where it applies them.
#[test]
fn sections_switched_off_render_as_defaults() {
let src = Rgb32f::from_fn(32, 24, |x, y| [0.05 + x as f32 / 40.0, 0.1 + y as f32 / 30.0, 0.3]);
let info = SourceInfo::default();
let req = RenderRequest::fit(32, 24);
let plain = render(&src, &info, &DevelopSettings::default(), &req).image;
let mut s = DevelopSettings::default();
s.light.exposure = 1.5;
s.light.contrast = 40.0;
s.color.saturation = -60.0;
s.color.vibrance = 30.0;
s.detail.sharpen_amount = 120.0;
let edited = render(&src, &info, &s, &req).image;
assert_ne!(edited.data, plain.data);
for section in ["light", "color", "detail"] {
s.set_section_enabled(section, false);
}
assert_eq!(render(&src, &info, &s, &req).image.data, plain.data, "every edited section off = the unedited photo");
// one section back on brings back only its own adjustments
s.set_section_enabled("light", true);
let mut light_only = DevelopSettings::default();
light_only.light.exposure = 1.5;
light_only.light.contrast = 40.0;
assert_eq!(render(&src, &info, &s, &req).image.data, render(&src, &info, &light_only, &req).image.data);
assert!(matches!(DevelopSettings::default().effective(), std::borrow::Cow::Borrowed(_)), "nothing is copied when every section is on");
}
#[test]
fn calibration_shifts_colours_but_keeps_greys() {
let info = SourceInfo { raw: true, ..Default::default() };
@@ -330,3 +359,28 @@ fn soft_proof_maps_into_the_proof_gamut_and_flags_what_does_not_fit() {
assert!(blue(&pro) > 0);
assert!(red(&pro) < n, "ProPhoto holds more than sRGB");
}
#[test]
fn tint_negative_is_green_and_positive_is_magenta() {
use lightcraft_develop::WbMode;
let src = Rgb32f::filled(16, 16, [0.18; 3]);
// Rendered images, uncalibrated RAW and calibrated RAW with a nonzero As Shot tint.
for info in [
SourceInfo::default(),
SourceInfo { raw: true, relative_wb: true, ..Default::default() },
SourceInfo { raw: true, as_shot_temp: 4200.0, as_shot_tint: 15.0, ..Default::default() },
] {
let mut s = DevelopSettings::default();
let neutral = render(&src, &info, &s, &RenderRequest::fit(16, 16)).image.get(8, 8);
assert!((neutral[0] as i16 - neutral[1] as i16).abs() <= 1);
assert!((neutral[2] as i16 - neutral[1] as i16).abs() <= 1);
s.wb.mode = WbMode::Custom;
s.wb.temp = info.as_shot_temp;
for delta in [-50.0, 50.0] {
s.wb.tint = info.as_shot_tint + delta;
let p = render(&src, &info, &s, &RenderRequest::fit(16, 16)).image.get(8, 8);
let magenta = (p[0] as f64 + p[2] as f64) / 2.0 - p[1] as f64;
assert!(magenta * delta > 100.0, "delta {delta} must follow the green/magenta track, got {p:?}");
}
}
}
+2
View File
@@ -24,3 +24,5 @@ zune-core = { workspace = true }
[dev-dependencies]
jpeg-encoder = { workspace = true }
proptest = { workspace = true }
# Pinned CC0 corpus identities; not a product dependency.
sha2 = { version = "0.10", default-features = false, features = ["std"] }
+258 -15
View File
@@ -12,11 +12,11 @@
//! look tables and tone curve.
//!
//! Formats: DNG (uncompressed, lossless JPEG, lossy JPEG (Smart Previews), Deflate incl. floating point, tiled/stripped, CFA and LinearRaw),
//! Canon CR2, Nikon NEF/NRW (uncompressed, Huffman lossless / lossy compressed), Sony ARW (uncompressed, ARW2, lossless), Fujifilm RAF (uncompressed Bayer
//! and X-Trans), Panasonic RW2 / Leica RWL / Panasonic RAW (every raw format: compressed 4 and 6, the prefix-coded strips of 8,
//! Canon CR2 / CR3 (lossless CRX Bayer and version 0x100/0x200 C-RAW), Nikon NEF/NRW (uncompressed, Huffman lossless / lossy compressed), Sony ARW (uncompressed, ARW2, lossless), Fujifilm RAF (uncompressed Bayer
//! and X-Trans, lossless and lossy compressed), Panasonic RW2 / Leica RWL / Panasonic RAW (every raw format: compressed 4 and 6, the prefix-coded strips of 8,
//! packed 2/5/7, the 16-bit words of the oldest bodies), Pentax PEF (uncompressed, Huffman), Olympus ORF (uncompressed).
//! [`embedded_preview`] covers all of them plus CR3. Variants we can't decode yet (Nikon "lossy after split" NEF,
//! compressed ORF/RAF, CR3) return [`RawError::Unsupported`]; each vendor module documents its sources
//! [`embedded_preview`] covers these containers' JPEG previews. Variants we can't decode yet (Nikon "lossy after split" NEF,
//! compressed ORF, CR3 unverified marker families / C-RAW configurations) return [`RawError::Unsupported`]; each vendor module documents its sources
//! (public specifications, tag-name documentation, black-box analysis of CC0 samples) and gaps. Non-DNG files carry no
//! colour matrix: [`color`] falls back to a documented neutral model. The decoders never panic on malformed input.
#![forbid(unsafe_code)]
@@ -43,7 +43,7 @@ pub use lightcraft_geom::Orientation;
pub use lightcraft_meta::Metadata;
pub use lightcraft_raster::Rgb32f;
pub use opcodes::{Opcode, OpcodeLists};
pub use preview::embedded_preview;
pub use preview::{PreviewColorSpace, embedded_preview, embedded_preview_color_space};
use lightcraft_color::Xy;
use lightcraft_tiff::{Tiff, TiffError};
@@ -83,6 +83,12 @@ pub enum RawFormat {
Rw2,
Pef,
Srw,
/// Canon CRW (CIFF heap), the bodies before the CR2 era.
Crw,
/// Minolta MRW (`\0MRM` block container).
Mrw,
/// Sigma / Foveon X3F (`FOVb`).
X3f,
/// Another TIFF-based raw (3FR, IIQ, ERF, KDC, DCR, MOS, …).
OtherTiff,
}
@@ -92,7 +98,15 @@ impl RawFormat {
pub fn is_supported(self) -> bool {
matches!(
self,
RawFormat::Dng | RawFormat::Cr2 | RawFormat::Nef | RawFormat::Nrw | RawFormat::Arw | RawFormat::Raf | RawFormat::Rw2 | RawFormat::Pef
RawFormat::Dng
| RawFormat::Cr2
| RawFormat::Cr3
| RawFormat::Nef
| RawFormat::Nrw
| RawFormat::Arw
| RawFormat::Raf
| RawFormat::Rw2
| RawFormat::Pef
)
}
}
@@ -111,6 +125,15 @@ pub fn probe(bytes: &[u8]) -> Option<RawFormat> {
if bytes.starts_with(b"IIU\0") {
return Some(RawFormat::Rw2);
}
if bytes.starts_with(b"\0MRM") {
return Some(RawFormat::Mrw);
}
if bytes.starts_with(b"FOVb") {
return Some(RawFormat::X3f);
}
if bytes.len() >= 14 && bytes.starts_with(b"II") && &bytes[6..14] == b"HEAPCCDR" {
return Some(RawFormat::Crw);
}
if bytes.len() >= 10 && bytes.starts_with(b"II*\0") && &bytes[8..10] == b"CR" {
return Some(RawFormat::Cr2);
}
@@ -122,10 +145,7 @@ pub fn probe(bytes: &[u8]) -> Option<RawFormat> {
return Some(RawFormat::Dng);
}
let make = t.find(lightcraft_tiff::tags::MAKE).and_then(|e| e.value.as_str()).unwrap_or_default().to_ascii_uppercase();
let has_cfa = t.all_ifds().iter().any(|i| {
i.u16(lightcraft_tiff::tags::PHOTOMETRIC) == Some(lightcraft_tiff::tags::photometric::CFA)
|| i.u16(lightcraft_tiff::tags::COMPRESSION).is_some_and(|c| c == 34713 || c == 32767 || c == 32769 || c == 32770)
});
let has_cfa = has_raw_ifd(&t);
if make.starts_with("CANON") && t.ifds.len() >= 4 && t.ifds[3].u16(lightcraft_tiff::tags::COMPRESSION) == Some(6) {
return Some(RawFormat::Cr2);
}
@@ -141,12 +161,103 @@ pub fn probe(bytes: &[u8]) -> Option<RawFormat> {
if make.starts_with("SAMSUNG") {
return Some(RawFormat::Srw);
}
if has_cfa {
if has_cfa || thumbnail_shell(&t, bytes.len()).is_some() || is_preview_container(ifd0) {
return Some(RawFormat::OtherTiff);
}
None
}
/// Whether some IFD is marked as raw: CFA photometric, or a raw-only compression value (99 is not a
/// registered TIFF compression; Leaf MOS files use it for their tiled 16-bit lossless-JPEG raw).
fn has_raw_ifd(t: &Tiff) -> bool {
t.all_ifds().iter().any(|i| {
i.u16(lightcraft_tiff::tags::PHOTOMETRIC) == Some(lightcraft_tiff::tags::photometric::CFA)
|| i.u16(lightcraft_tiff::tags::COMPRESSION).is_some_and(|c| c == 34713 || c == 32767 || c == 32769 || c == 32770 || c == 99)
})
}
/// IFD0 that is not an image at all: no `ImageWidth` / `ImageLength`, but a JPEG preview pointer or
/// `SubIFDs` (the layout of Kodak KDC files, whose pictures sit in private tags). A plain TIFF reader
/// rejects such a file as malformed even though it carries a preview.
fn is_preview_container(ifd0: &lightcraft_tiff::Ifd) -> bool {
use lightcraft_tiff::tags::{IMAGE_LENGTH, IMAGE_WIDTH, JPEG_INTERCHANGE_FORMAT, SUB_IFDS};
!ifd0.contains(IMAGE_WIDTH) && !ifd0.contains(IMAGE_LENGTH) && (ifd0.contains(JPEG_INTERCHANGE_FORMAT) || ifd0.contains(SUB_IFDS))
}
/// A TIFF whose first image is only a reduced copy of a picture stored somewhere else.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) struct ThumbnailShell {
/// Size of the first image (IFD0).
pub thumb: (u32, u32),
/// Size of the picture it stands in for.
pub full: (u32, u32),
}
/// Some raw files are a TIFF "shell" around a private block: IFD0 holds a small RGB preview and nothing
/// in the TIFF structure marks the real image as raw (no CFA photometric, no raw compression). The
/// structure still gives them away, with a generic rule that names no maker or model: the file
/// declares a picture at least 4x larger than IFD0 in both directions, and either
/// - a `SubIFDs` child of IFD0 is that bigger image (an 8-bit grey mosaic in one such file), or
/// - the Exif `PixelXDimension` / `PixelYDimension` say so *and* the file holds enough bytes outside
/// every IFD's strips and tiles to carry that many pixels at one bit each (a downsized copy that kept
/// its old Exif size has no such block).
///
/// A multi-page TIFF is not caught (its pages are the IFD chain, not `SubIFDs`), nor a pyramid whose
/// IFD0 is the full image.
fn thumbnail_shell(t: &Tiff, len: usize) -> Option<ThumbnailShell> {
use lightcraft_tiff::tags::{IMAGE_LENGTH, IMAGE_WIDTH, PIXEL_X_DIMENSION, PIXEL_Y_DIMENSION};
let ifd0 = t.ifds.first()?;
let (w0, h0) = (ifd0.u32(IMAGE_WIDTH)?, ifd0.u32(IMAGE_LENGTH)?);
if w0 == 0 || h0 == 0 {
return None;
}
let much_bigger = |(w, h): (u32, u32)| u64::from(w) >= 4 * u64::from(w0) && u64::from(h) >= 4 * u64::from(h0);
let shell = |full| Some(ThumbnailShell { thumb: (w0, h0), full });
let mut best: Option<(u32, u32)> = None;
for s in &ifd0.sub_ifds {
if s.image().is_ok()
&& let (Some(w), Some(h)) = (s.u32(IMAGE_WIDTH), s.u32(IMAGE_LENGTH))
&& much_bigger((w, h))
&& best.is_none_or(|(bw, bh)| u64::from(w) * u64::from(h) > u64::from(bw) * u64::from(bh))
{
best = Some((w, h));
}
}
if let Some(full) = best {
return shell(full);
}
let exif = t.exif()?;
let full = (exif.u32(PIXEL_X_DIMENSION)?, exif.u32(PIXEL_Y_DIMENSION)?);
if !much_bigger(full) {
return None;
}
// bytes covered by image data (strips and tiles shared between IFDs, e.g. IFD0 and its thumbnail, count once)
let mut seen = std::collections::BTreeMap::new();
for ifd in t.all_ifds() {
if let Ok(info) = ifd.image() {
for c in info.chunks(len as u64) {
seen.insert(c.offset, c.len);
}
}
}
let covered = seen.values().fold(0u64, |a, &l| a.saturating_add(l));
let spare = (len as u64).saturating_sub(covered);
(spare.saturating_mul(8) >= u64::from(full.0) * u64::from(full.1)).then_some(())?;
shell(full)
}
/// Why [`decode`] gives up on a file [`probe`] called [`RawFormat::OtherTiff`].
fn other_tiff_reason(bytes: &[u8]) -> String {
let t = Tiff::parse_with(bytes, &lightcraft_tiff::ParseOptions { max_ifds: 256, ..Default::default() }).ok();
match t.as_ref().filter(|t| !has_raw_ifd(t)).and_then(|t| thumbnail_shell(t, bytes.len())) {
Some(s) => format!(
"{}x{} raw image in a private block, not decoded yet (the file's first image is a {}x{} reduced copy)",
s.full.0, s.full.1, s.thumb.0, s.thumb.1
),
None => "OtherTiff files are not decoded yet".to_string(),
}
}
/// Decode a raw file.
pub fn decode(bytes: &[u8]) -> Result<RawImage> {
decode_with(bytes, Mode::Full)
@@ -159,9 +270,9 @@ pub fn decode(bytes: &[u8]) -> Result<RawImage> {
/// measure them from the samples).
///
/// A few uncompressed vendor formats derive part of this from the samples themselves (Nikon
/// NEF: optically masked trailing columns; Olympus ORF: the CFA phase and bit depth; Pentax PEF
/// without crop tags: dark borders); for those the samples are read (unpacked, nothing to
/// decompress) and dropped.
/// NEF: optically masked trailing columns; Olympus ORF: the bit depth of 16-bit files, and the CFA
/// phase of files without an Exif `CFAPattern`; Pentax PEF without crop tags: dark borders); for
/// those the samples are read (unpacked, nothing to decompress) and dropped.
pub fn probe_info(bytes: &[u8]) -> Result<RawInfo> {
decode_with(bytes, Mode::Header).map(RawImage::into_info)
}
@@ -180,12 +291,14 @@ fn decode_with(bytes: &[u8], mode: Mode) -> Result<RawImage> {
match probe(bytes).ok_or(RawError::NotRaw)? {
RawFormat::Dng => dng::decode(bytes, mode),
RawFormat::Cr2 => vendor::cr2::decode(bytes, mode),
RawFormat::Cr3 => vendor::cr3::decode(bytes, mode),
RawFormat::Nef | RawFormat::Nrw => vendor::nef::decode(bytes),
RawFormat::Arw => vendor::arw::decode(bytes, mode),
RawFormat::Raf => vendor::raf::decode(bytes, mode),
RawFormat::Rw2 => vendor::rw2::decode(bytes, mode),
RawFormat::Pef => vendor::pef::decode(bytes, mode),
RawFormat::Orf => vendor::orf::decode(bytes),
RawFormat::Orf => vendor::orf::decode(bytes, mode),
RawFormat::OtherTiff => Err(RawError::Unsupported(other_tiff_reason(bytes))),
other => Err(RawError::Unsupported(format!("{other:?} files are not decoded yet"))),
}
}
@@ -614,6 +727,136 @@ mod tests {
assert_eq!(BlackLevel::uniform(7.0).at(9, 9, 0, 3), 7.0);
}
// --- TIFF shells around a private raw block (thumbnail-only IFD0) ---
use lightcraft_tiff::{IfdBuilder, ImageData, TiffWriter, Value, tags as t};
/// An uncompressed 8-bit RGB image IFD.
fn rgb_ifd(w: u32, h: u32) -> IfdBuilder {
let mut ifd = IfdBuilder::new();
ifd.set(t::IMAGE_WIDTH, Value::Long(vec![w]));
ifd.set(t::IMAGE_LENGTH, Value::Long(vec![h]));
ifd.set(t::BITS_PER_SAMPLE, Value::Short(vec![8, 8, 8]));
ifd.set(t::SAMPLES_PER_PIXEL, Value::Short(vec![3]));
ifd.set(t::PHOTOMETRIC, Value::Short(vec![2]));
ifd.set(t::COMPRESSION, Value::Short(vec![1]));
ifd.set_image(ImageData::Strips { rows_per_strip: h, strips: vec![vec![90u8; (w * h * 3) as usize]] });
ifd
}
fn exif_size(w: u32, h: u32) -> IfdBuilder {
IfdBuilder::new().with(t::PIXEL_X_DIMENSION, Value::Long(vec![w])).with(t::PIXEL_Y_DIMENSION, Value::Long(vec![h]))
}
fn write(chain: &[IfdBuilder]) -> Vec<u8> {
TiffWriter::default().write(chain).unwrap()
}
const SHELL_REASON: &str = "raw image in a private block";
/// A bigger image in a `SubIFDs` child of a small IFD0 (the layout of an 8-bit-mosaic TIFF raw).
#[test]
fn thumbnail_with_a_bigger_sub_ifd_is_an_undecodable_raw() {
let mut ifd0 = rgb_ifd(16, 12);
ifd0.add_sub_ifd(rgb_ifd(64, 48));
let bytes = write(&[ifd0]);
assert_eq!(probe(&bytes), Some(RawFormat::OtherTiff));
let Err(RawError::Unsupported(why)) = decode(&bytes) else { panic!("expected Unsupported") };
assert!(why.contains("64x48") && why.contains("16x12") && why.contains(SHELL_REASON), "{why}");
// a sub-IFD that is not much bigger (a pyramid level) does not count
let mut ifd0 = rgb_ifd(16, 12);
ifd0.add_sub_ifd(rgb_ifd(32, 24));
assert_eq!(probe(&write(&[ifd0])), None);
}
/// Exif says the picture is far bigger than IFD0 and the file holds a block that could carry it.
#[test]
fn thumbnail_with_a_private_block_behind_the_exif_size_is_an_undecodable_raw() {
let mut ifd0 = rgb_ifd(16, 12);
ifd0.set_child(t::EXIF_IFD, exif_size(400, 300));
let mut bytes = write(&[ifd0.clone()]);
// no private block: a downsized copy that kept its old Exif size is an ordinary image
assert_eq!(probe(&bytes), None);
// 400 x 300 pixels at one bit each is 15000 bytes: 14000 is not enough, 16000 is
bytes.extend(std::iter::repeat_n(7u8, 14_000));
assert_eq!(probe(&bytes), None);
bytes.extend(std::iter::repeat_n(7u8, 2_000));
assert_eq!(probe(&bytes), Some(RawFormat::OtherTiff));
let Err(RawError::Unsupported(why)) = decode(&bytes) else { panic!("expected Unsupported") };
assert!(why.contains("400x300") && why.contains("16x12") && why.contains(SHELL_REASON), "{why}");
// truncating it never panics
for n in (0..bytes.len()).step_by(37) {
let _ = probe(&bytes[..n]);
let _ = decode(&bytes[..n]);
}
}
/// Ordinary TIFFs stay images: the full picture first, equal Exif size, or later pages that are bigger.
#[test]
fn ordinary_tiffs_are_not_shells() {
let padded = |chain: &[IfdBuilder]| {
let mut b = write(chain);
b.extend(std::iter::repeat_n(0u8, 100_000));
b
};
let mut full = rgb_ifd(400, 300);
full.set_child(t::EXIF_IFD, exif_size(400, 300));
assert_eq!(probe(&padded(&[full])), None);
// the Exif size is much bigger than IFD0, with spare bytes, but IFD0 is not tiny next to it
let mut near = rgb_ifd(120, 90);
near.set_child(t::EXIF_IFD, exif_size(400, 300));
assert_eq!(probe(&padded(&[near])), None);
// a multi-page TIFF whose first page is small
assert_eq!(probe(&padded(&[rgb_ifd(16, 12), rgb_ifd(400, 300)])), None);
// no dimensions at all
assert_eq!(probe(&padded(&[IfdBuilder::new().with(t::MAKE, Value::Ascii("X".into()))])), None);
}
// --- containers that are recognised but not decoded, with a preview ---
#[test]
fn private_containers_are_recognised_by_their_magic() {
assert_eq!(probe(b"II\x1a\0\0\0HEAPCCDR\0\0"), Some(RawFormat::Crw));
assert_eq!(probe(b"\0MRM\0\x01\0\0"), Some(RawFormat::Mrw));
assert_eq!(probe(b"FOVb\x02\0\x02\0"), Some(RawFormat::X3f));
for f in [RawFormat::Crw, RawFormat::Mrw, RawFormat::X3f] {
assert!(!f.is_supported());
}
assert!(matches!(decode(b"\0MRM\0\x01\0\0"), Err(RawError::Unsupported(w)) if w.contains("Mrw")));
// too short for the CIFF signature
assert_eq!(probe(b"II\x1a\0\0\0HEAP"), None);
}
/// An IFD0 with a JPEG pointer but no `ImageWidth` / `ImageLength` (the Kodak KDC layout) is a raw
/// container with a preview; a TIFF reader would call it malformed.
#[test]
fn ifd0_without_dimensions_but_with_a_jpeg_is_a_preview_container() {
let jpeg: Vec<u8> = [&[0xffu8, 0xd8, 0xff, 0xc0, 0, 0x0b, 8, 0, 1, 0, 1, 1, 1, 0x11, 0][..], &[0x55; 40], &[0xff, 0xd9]].concat();
let off = 8 + 2 + 24 + 4;
let mut f = b"II*\0\x08\0\0\0\x02\0".to_vec();
for (tag, v) in [(513u16, off as u32), (514, jpeg.len() as u32)] {
f.extend_from_slice(&tag.to_le_bytes());
f.extend_from_slice(&[4, 0, 1, 0, 0, 0]);
f.extend_from_slice(&v.to_le_bytes());
}
f.extend_from_slice(&[0, 0, 0, 0]);
f.extend_from_slice(&jpeg);
assert_eq!(probe(&f), Some(RawFormat::OtherTiff));
assert!(matches!(decode(&f), Err(RawError::Unsupported(_))));
assert_eq!(embedded_preview(&f), Some(jpeg));
// no pointer and no sub-IFDs: just a broken TIFF
let g = write(&[IfdBuilder::new().with(t::MAKE, Value::Ascii("KODAK".into()))]);
assert_eq!(probe(&g), None);
}
/// Compression 99 (not a registered TIFF value; Leaf MOS tiles) marks a raw even without a CFA tag.
#[test]
fn private_compression_99_is_a_raw() {
let mut ifd = rgb_ifd(16, 12);
ifd.set(t::COMPRESSION, Value::Short(vec![99]));
assert_eq!(probe(&write(&[ifd])), Some(RawFormat::OtherTiff));
}
#[test]
fn probe_rejects_junk() {
assert_eq!(probe(b""), None);
+7
View File
@@ -98,6 +98,13 @@ impl Opcode {
pub fn is_applied(&self) -> bool {
!matches!(self, Opcode::Unknown { .. } | Opcode::TrimBounds { .. } | Opcode::WarpFisheye { .. })
}
/// Whether this is a lens correction (`WarpRectilinear`, `FixVignetteRadial`): from a DNG, or the equivalent of
/// a camera's own correction recorded in its raw (Panasonic RW2). Developers that apply these through the
/// "profile corrections" of the develop settings remove them before demosaicing.
pub fn is_lens_correction(&self) -> bool {
matches!(self, Opcode::WarpRectilinear { .. } | Opcode::FixVignetteRadial { .. })
}
}
/// The three opcode lists of a DNG raw IFD.
+197 -2
View File
@@ -5,6 +5,30 @@ use lightcraft_tiff::image::chunk_bytes;
use lightcraft_tiff::tags as t;
use lightcraft_tiff::{Ifd, Tiff, makernote};
/// Colour space of an embedded JPEG when the enclosing raw supplies it instead of the JPEG.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum PreviewColorSpace {
Srgb,
AdobeRgb,
}
/// Nikon maker-note ColorSpace (0x001e: 1 = sRGB, 2 = Adobe RGB), per Nikon tag documentation.
/// This is a fallback only: a JPEG's own ICC/EXIF colour declaration takes precedence.
pub fn embedded_preview_color_space(bytes: &[u8]) -> Option<PreviewColorSpace> {
let tiff = Tiff::parse(bytes).ok()?;
let make = tiff.ifds.first()?.string(t::MAKE)?;
if !make.to_ascii_uppercase().starts_with("NIKON") {
return None;
}
let e = tiff.exif()?.get(t::MAKER_NOTE)?;
let mn = makernote::parse_makernote(bytes, e.offset, e.count() as u64, tiff.order, &make)?;
match mn.ifd.u16(0x001e)? {
1 => Some(PreviewColorSpace::Srgb),
2 => Some(PreviewColorSpace::AdobeRgb),
_ => None,
}
}
/// Whether `b` looks like a displayable (DCT) JPEG: SOI, and the first SOF marker is not lossless.
fn is_dct_jpeg(b: &[u8]) -> bool {
if b.len() < 4 || b[0] != 0xff || b[1] != 0xd8 {
@@ -67,9 +91,13 @@ pub fn embedded_preview(bytes: &[u8]) -> Option<Vec<u8>> {
let j = crate::vendor::raf::header(bytes).ok()?.jpeg?;
return is_dct_jpeg(j).then(|| trim_eoi(j).to_vec());
}
if crate::probe(bytes) == Some(crate::RawFormat::Cr3) {
let format = crate::probe(bytes);
if format == Some(crate::RawFormat::Cr3) {
return cr3_preview(bytes).map(|j| trim_eoi(j).to_vec());
}
if matches!(format, Some(crate::RawFormat::Crw | crate::RawFormat::Mrw | crate::RawFormat::X3f)) {
return scan_for_jpeg(bytes);
}
let tiff = Tiff::parse(bytes).ok()?;
let mut found: Vec<&[u8]> = Vec::new();
for ifd in tiff.all_ifds() {
@@ -93,7 +121,112 @@ pub fn embedded_preview(bytes: &[u8]) -> Option<Vec<u8>> {
if let Some(p) = crate::vendor::orf::preview(bytes) {
found.push(p);
}
found.into_iter().filter(|s| is_dct_jpeg(s)).max_by_key(|s| s.len()).map(|s| trim_eoi(s).to_vec())
let best = found.into_iter().filter(|s| is_dct_jpeg(s)).max_by_key(|s| s.len()).map(|s| trim_eoi(s).to_vec());
// a TIFF raw we can't decode whose preview no tag points to (Leaf MOS, Epson ERF)
best.or_else(|| format.filter(|f| !f.is_supported()).and_then(|_| scan_for_jpeg(bytes)))
}
/// The end of the DCT JPEG whose SOI is at `soi` (exclusive index after its EOI) and its component count,
/// walking the marker segments and the entropy-coded data. `None` when it is not a complete, displayable
/// (baseline or progressive) JPEG, so a stray `D8 FF` in other data is rejected. `*reach` is left at
/// the furthest byte looked at (what the walk cost; see [`scan_for_jpeg`]).
fn jpeg_extent(b: &[u8], soi: usize, reach: &mut usize) -> Option<(usize, u8)> {
let mut i = soi + 2;
let r = walk_jpeg(b, &mut i);
*reach = i;
r
}
fn walk_jpeg(b: &[u8], i: &mut usize) -> Option<(usize, u8)> {
let mut components = None;
for _ in 0..4096 {
if *b.get(*i)? != 0xff {
return None;
}
while *b.get(*i)? == 0xff {
*i += 1;
}
let m = *b.get(*i)?;
*i += 1;
match m {
0xd9 => return components.map(|c| (*i, c)),
0x01 | 0xd0..=0xd8 => continue,
_ => {}
}
let len = usize::from(u16::from_be_bytes([*b.get(*i)?, *b.get(*i + 1)?]));
if len < 2 {
return None;
}
match m {
0xc0..=0xc2 => {
// precision, height, width, component count
let h = u16::from_be_bytes([*b.get(*i + 3)?, *b.get(*i + 4)?]);
let w = u16::from_be_bytes([*b.get(*i + 5)?, *b.get(*i + 6)?]);
if h == 0 || w == 0 || components.is_some() {
return None;
}
components = Some(*b.get(*i + 7)?);
}
0xc3 | 0xc5..=0xc7 | 0xc9..=0xcb | 0xcd..=0xcf => return None, // lossless, arithmetic, hierarchical
_ => {}
}
*i += len;
if m == 0xda {
// entropy-coded data up to the next marker that is not a stuffed 0xff or a restart
loop {
*i += b.get(*i..)?.iter().position(|&x| x == 0xff)?;
match *b.get(*i + 1)? {
0x00 | 0xd0..=0xd7 => *i += 2,
0xff => *i += 1,
_ => break,
}
}
}
}
None
}
/// Last resort for the containers whose structure is not walked (Canon CRW, Minolta MRW, Sigma X3F, TIFF raws
/// with private blocks such as Leaf MOS or Epson ERF): the largest complete colour DCT JPEG stored as a plain
/// byte run in the file. A one-component JPEG is skipped (a Canon PowerShot CRW stores its sensor mosaic as one).
///
/// Evidence: in the Minolta and Epson files the first byte of the stored preview is not `ff` (it is `00`, `02` or
/// `ee`) while the rest is a regular JPEG, so a run is found by its `d8 ff` and starts one byte earlier,
/// with that byte restored to `ff`.
///
/// A crafted file can hold millions of JPEG-like starts whose walks each run to its end: the walks
/// share a budget of eight times the file's size, so the scan stays linear (a real file's candidates
/// fail within a few bytes, or are the preview itself).
fn scan_for_jpeg(data: &[u8]) -> Option<Vec<u8>> {
let mut best: Option<(usize, usize)> = None;
let mut at = 1;
let mut budget = data.len().saturating_mul(8).max(1 << 20);
while let Some(off) = data.get(at..).and_then(|s| s.iter().position(|&b| b == 0xd8)) {
let d8 = at + off;
at = d8 + 1;
if data.get(d8 + 1) != Some(&0xff) || !matches!(data.get(d8 + 2), Some(0xc0..=0xc4 | 0xdb | 0xe0..=0xef | 0xfe)) {
continue;
}
let start = d8 - 1; // `at` starts at 1, so d8 >= 1
let mut reach = start;
let found = jpeg_extent(data, start, &mut reach);
budget = budget.saturating_sub(reach.saturating_sub(start));
if budget == 0 {
break;
}
if let Some((end, 3)) = found {
if best.is_none_or(|(s, e)| end - start > e - s) {
best = Some((start, end));
}
at = end;
}
}
let (start, end) = best?;
let mut jpeg = data.get(start..end)?.to_vec();
if let Some(first) = jpeg.first_mut() {
*first = 0xff;
}
Some(jpeg)
}
/// Canon CR3: the full-size JPEG track (see [`lightcraft_meta::cr3`]), else the `PRVW` / `THMB` boxes.
@@ -218,4 +351,66 @@ mod tests {
file.truncate(at as usize + 100);
assert_eq!(embedded_preview(&file).unwrap(), small);
}
/// A structurally complete DCT JPEG (no real image data): `nc` components, `n` entropy bytes.
fn scan_jpeg(nc: u8, n: usize) -> Vec<u8> {
let mut j = vec![0xff, 0xd8, 0xff, 0xdb, 0x00, 0x43, 0x00];
j.extend(std::iter::repeat_n(8u8, 64));
j.extend_from_slice(&[0xff, 0xc0, 0x00, 8 + 3 * nc, 8, 0, 16, 0, 24, nc]);
for c in 0..nc {
j.extend_from_slice(&[c + 1, 0x11, 0]);
}
j.extend_from_slice(&[0xff, 0xda, 0x00, 6 + 2 * nc, nc]);
for c in 0..nc {
j.extend_from_slice(&[c + 1, 0]);
}
j.extend_from_slice(&[0, 63, 0]);
j.extend(std::iter::repeat_n(0x55u8, n));
j.extend_from_slice(&[0xff, 0x00, 0x12, 0xff, 0xd0, 0x34, 0xff, 0xd9]);
j
}
/// A crafted container full of JPEG-like starts (each walk running on through the ones after it)
/// is scanned in linear time: the walks share a budget (without it this 40 MB file takes about a minute).
#[test]
fn a_scan_through_many_fake_jpegs_stays_linear() {
let mut unit = vec![0u8, 0xd8, 0xff, 0xdb, 0x00, 0x02, 0xff, 0xda, 0x00, 0x02];
unit.extend(std::iter::repeat_n(0x55u8, 190));
let mut f = b"\0MRM\0\x01\0\0".to_vec();
for _ in 0..200_000 {
f.extend_from_slice(&unit);
}
let t = std::time::Instant::now();
assert_eq!(scan_for_jpeg(&f), None);
assert!(t.elapsed() < std::time::Duration::from_secs(10), "{:?}", t.elapsed());
}
/// Containers whose structure we do not walk give up their largest colour JPEG, found by scanning.
#[test]
fn scanned_containers_give_the_largest_colour_jpeg() {
let (small, big, mosaic) = (scan_jpeg(3, 100), scan_jpeg(3, 900), scan_jpeg(1, 5000));
let heads: [&[u8]; 3] = [b"II\x1a\0\0\0HEAPCCDR", b"\0MRM\0\x01\0\0", b"FOVb\x02\0\x02\0"];
for head in heads {
let mut f = head.to_vec();
for part in [&small, &mosaic, &big] {
f.extend_from_slice(&[0xff, 0xd8, 0xff, 0x00, 0x00]); // a stray SOI-like run: not a JPEG
f.extend_from_slice(part);
f.extend_from_slice(&[0u8; 7]);
}
assert!(crate::probe(&f).is_some_and(|p| !p.is_supported()), "{head:?}");
assert_eq!(embedded_preview(&f).as_deref(), Some(&big[..]), "{head:?}");
for n in (0..f.len()).step_by(11) {
let _ = embedded_preview(&f[..n]);
}
}
// a preview whose first byte was overwritten (Minolta and Epson files store 00, 02 or ee there) is restored
let mut f = b"\0MRM\0\x01\0\0".to_vec();
f.push(0x02);
f.extend_from_slice(&big[1..]);
assert_eq!(embedded_preview(&f).as_deref(), Some(&big[..]));
// only a one-component JPEG, or none: nothing
let mut f = b"FOVb\x02\0\x02\0".to_vec();
f.extend_from_slice(&mosaic);
assert_eq!(embedded_preview(&f), None);
}
}
+81 -3
View File
@@ -15,14 +15,15 @@
//! - `SensorInfo` (maker note `0x00e0`): sensor width/height and the left/top/right/bottom borders of the image
//! area; the masked columns left of it give the black level.
//! - `ColorBalance` (maker note `0x4001`): as-shot `RGGB` levels at a model-dependent offset; we probe the known
//! offsets and accept the first plausible quadruple.
//! offsets and accept the first plausible quadruple. The array is 16-bit words; some models store it as UNDEFINED
//! bytes (ColorData versions -3 and -4), which are paired up in the maker note's byte order first.
//! - sRAW / mRAW (YCbCr, subsampled) are not supported yet.
use super::{black_from_columns, white_from_data};
use crate::{BlackLevel, Cfa, ColorData, Mode, OpcodeLists, RawData, RawError, RawFormat, RawImage, Rect, Result, ljpeg};
use lightcraft_geom::Orientation;
use lightcraft_tiff::image::chunk_bytes;
use lightcraft_tiff::{Ifd, Tiff, makernote, tags as t};
use lightcraft_tiff::{ByteOrder, Ifd, Tiff, Value, makernote, tags as t};
const CR2_SLICE: u16 = 0xc640;
const SRAW_TYPE: u16 = 0xc6c5;
@@ -46,6 +47,17 @@ fn cfa_from_tag(v: Option<u64>) -> Option<Cfa> {
Some(Cfa::bayer_static(name))
}
/// The 16-bit words of a maker-note array. Canon writes `ColorData` as SHORT in most models, but as UNDEFINED
/// bytes (count = byte length, e.g. 5120) in the PowerShot / EOS M models with ColorData versions -3 and -4;
/// those bytes are the same 16-bit words in the maker note's byte order, so they are paired up, not read one by
/// one. Other types are returned value by value.
fn words(v: &Value, order: ByteOrder) -> Vec<u64> {
match v {
Value::Byte(b) | Value::Undefined(b) => b.as_chunks::<2>().0.iter().map(|c| u64::from(order.u16(*c))).collect(),
_ => v.to_u64_vec(),
}
}
/// As-shot WB multipliers (R, G, B; G = 1) from the ColorBalance array.
fn wb_from_color_balance(v: &[u64]) -> Option<[f32; 3]> {
for off in [63usize, 25, 24, 34, 71, 85, 105, 69, 77] {
@@ -134,7 +146,7 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
active = Rect::new(l, tp, r - l + 1, b - tp + 1);
}
}
let wb = mn.as_ref().and_then(|m| m.ifd.u64s(COLOR_BALANCE)).and_then(|v| wb_from_color_balance(&v));
let wb = mn.as_ref().and_then(|m| m.ifd.value(COLOR_BALANCE).map(|v| words(v, m.order))).and_then(|v| wb_from_color_balance(&v));
let cfa = cfa_from_tag(raw.u64(CR2_CFA_PATTERN)).unwrap_or_else(|| Cfa::bayer_static("RGGB"));
let black = if active.x >= 8 {
black_from_columns(&data, width, 2..active.x - 2, active.y..active.y + active.height, active)
@@ -175,6 +187,11 @@ pub(crate) mod tests {
/// Build a synthetic CR2: IFD0..IFD2 placeholders + IFD3 holding a sliced 2-component LJ92 frame.
pub(crate) fn synthetic_cr2(w: usize, h: usize, slices: Option<[u16; 3]>) -> (Vec<u8>, Vec<u16>) {
synthetic_cr2_note(w, h, slices, None)
}
/// Like [`synthetic_cr2`], with `note` as the Exif maker note (bytes as stored).
fn synthetic_cr2_note(w: usize, h: usize, slices: Option<[u16; 3]>, note: Option<Vec<u8>>) -> (Vec<u8>, Vec<u16>) {
let img: Vec<u16> = (0..w * h).map(|i| 1024 + ((i * 7919) % 12000) as u16).collect();
let widths: Vec<usize> = match slices {
Some([n, sw, last]) => {
@@ -200,6 +217,9 @@ pub(crate) mod tests {
// maker note: SensorInfo + ColorBalance, plain IFD with offsets relative to the TIFF header
let mut exif = IfdBuilder::new();
exif.set(t::ISO_SPEED, Value::Short(vec![400]));
if let Some(n) = note {
exif.set(t::MAKER_NOTE, Value::Undefined(n));
}
ifd0.set_child(t::EXIF_IFD, exif);
let mut ifd3 = IfdBuilder::new();
ifd3.set(t::COMPRESSION, Value::Short(vec![6]));
@@ -251,4 +271,62 @@ pub(crate) mod tests {
assert!((wb[0] - 2000.0 / 1025.0).abs() < 1e-4 && (wb[2] - 1500.0 / 1025.0).abs() < 1e-4);
assert!(wb_from_color_balance(&[0; 10]).is_none());
}
#[test]
fn wb_from_undefined_bytes() {
// hand-coded: ColorData of an S110-like file, UNDEFINED bytes, little-endian words; word 71.. = 1988 745 745 1641
let mut b = vec![0u8; 2 * 120];
b[0..2].copy_from_slice(&[0xfd, 0xff]); // version -3
b[142..150].copy_from_slice(&[0xc4, 0x07, 0xe9, 0x02, 0xe9, 0x02, 0x69, 0x06]);
let le = words(&Value::Undefined(b.clone()), ByteOrder::Little);
assert_eq!(&le[71..75], &[1988, 745, 745, 1641]);
let wb = wb_from_color_balance(&le).unwrap();
assert!((wb[0] - 1988.0 / 745.0).abs() < 1e-5 && wb[1] == 1.0 && (wb[2] - 1641.0 / 745.0).abs() < 1e-5);
// read as single bytes (the old behaviour) nothing is plausible
assert!(wb_from_color_balance(&b.iter().map(|&x| u64::from(x)).collect::<Vec<_>>()).is_none());
// big-endian note: same words, bytes swapped
let mut be = b.clone();
be.as_chunks_mut::<2>().0.iter_mut().for_each(|c| c.swap(0, 1));
assert_eq!(words(&Value::Undefined(be), ByteOrder::Big), le);
// SHORT values pass through, an odd trailing byte is ignored
assert_eq!(words(&Value::Short(vec![1, 2, 3]), ByteOrder::Little), vec![1, 2, 3]);
assert_eq!(words(&Value::Undefined(vec![1, 0, 2]), ByteOrder::Little), vec![1]);
}
/// A maker note holding only `ColorData` (`0x4001`) as `ty` (3 = SHORT, 7 = UNDEFINED) with `n` values, a plain
/// IFD whose value offset is relative to the TIFF header (patched once the note's position is known).
fn note_with_color_data(ty: u16, data: &[u8]) -> Vec<u8> {
let count = if ty == 3 { data.len() / 2 } else { data.len() } as u32;
let mut note = Vec::new();
note.extend_from_slice(&1u16.to_le_bytes());
note.extend_from_slice(&COLOR_BALANCE.to_le_bytes());
note.extend_from_slice(&ty.to_le_bytes());
note.extend_from_slice(&count.to_le_bytes());
note.extend_from_slice(&0u32.to_le_bytes()); // value offset, patched below
note.extend_from_slice(&0u32.to_le_bytes()); // next IFD
note.extend_from_slice(data);
note
}
fn decode_with_color_data(ty: u16, data: &[u8]) -> RawImage {
let (mut bytes, _) = synthetic_cr2_note(64, 10, None, Some(note_with_color_data(ty, data)));
let tiff = Tiff::parse(&bytes).unwrap();
let e = tiff.exif().unwrap().get(t::MAKER_NOTE).unwrap();
let off = e.offset as usize;
bytes[off + 10..off + 14].copy_from_slice(&(e.offset as u32 + 18).to_le_bytes());
crate::decode(&bytes).unwrap()
}
/// `ColorData` stored as UNDEFINED bytes (versions -3 / -4: PowerShot, EOS M) gives the as-shot WB from word 71,
/// exactly as the same words stored as SHORT do.
#[test]
fn color_data_stored_as_bytes() {
let mut bytes = vec![0u8; 2 * 120];
bytes[0..2].copy_from_slice(&[0xfd, 0xff]); // version -3
bytes[142..150].copy_from_slice(&[0xc4, 0x07, 0xe9, 0x02, 0xe9, 0x02, 0x69, 0x06]); // 1988 745 745 1641
for ty in [7, 3] {
let wb = decode_with_color_data(ty, &bytes).wb_multipliers.unwrap_or_else(|| panic!("type {ty}: no WB"));
assert!((wb[0] - 1988.0 / 745.0).abs() < 1e-5 && wb[1] == 1.0 && (wb[2] - 1641.0 / 745.0).abs() < 1e-5, "type {ty}: {wb:?}");
}
}
}
+470
View File
@@ -0,0 +1,470 @@
//! Canon CR3 container, camera metadata and sensor geometry.
//!
//! Sources: Laurent Clévy's public CR3 format description (`CMP1`, `CDI1/IAD1`, `CMT3`),
//! <https://github.com/lclevy/canon_cr3/blob/master/readme.md>, and ExifTool's published Canon
//! tag-name tables (`SensorInfo`, `ColorData9`…`ColorData12`, `LensModel`),
//! <https://exiftool.org/TagNames/Canon.html>. No third-party decoder source was consulted.
//!
//! `IAD1` stores the valid sensor area separately from the recommended image crop. Both use inclusive
//! sensor offsets. The CFA is anchored at the full sensor's origin; cropping shifts its phase later.
//! Unknown camera metadata versions are ignored rather than assigned a guessed white balance.
use super::{black_from_columns, crx, white_from_data};
use crate::{BlackLevel, Cfa, ColorData, MAX_SAMPLES, Mode, OpcodeLists, RawData, RawError, RawFormat, RawImage, Rect, Result};
use lightcraft_meta::cr3::{Cr3ImageArea, Cr3Track, Cr3TrackKind, parse_cr3};
use lightcraft_tiff::{Ifd, Tiff};
const SENSOR_INFO: u16 = 0x00e0;
const COLOR_DATA: u16 = 0x4001;
const LENS_MODEL: u16 = 0x0095;
/// Pick the first largest raw track. Equal-sized later tracks can be the Dual Pixel delta, which
/// must never replace the main sensor image. An invalid full-size descriptor is reported, not
/// silently replaced with the reduced raw preview.
fn main_track(tracks: &[Cr3Track]) -> Option<&Cr3Track> {
let mut best = None;
let mut best_area = 0u64;
for track in tracks {
if let Cr3TrackKind::Raw { width, height, .. } = track.kind {
let area = u64::from(width) * u64::from(height);
if area > best_area {
best = Some(track);
best_area = area;
}
}
}
best
}
fn cfa(pattern: u8) -> Option<Cfa> {
let name = match pattern {
0 => "RGGB",
1 => "GRBG",
2 => "GBRG",
3 => "BGGR",
_ => return None,
};
Some(Cfa::bayer_static(name))
}
/// Inclusive offsets, clipped to the sample dimensions (some Canon crop-mode files record right
/// and bottom borders a few pixels beyond the encoded image).
fn inclusive_rect(bounds: [u16; 4], width: usize, height: usize) -> Option<Rect> {
let [left, top, right, bottom] = bounds.map(usize::from);
if left >= width || top >= height || right < left || bottom < top {
return None;
}
let end_x = right.checked_add(1)?.min(width);
let end_y = bottom.checked_add(1)?.min(height);
Some(Rect::new(left, top, end_x.checked_sub(left)?, end_y.checked_sub(top)?))
}
fn relative_crop(crop: Rect, active: Rect) -> Option<Rect> {
let x = crop.x.checked_sub(active.x)?;
let y = crop.y.checked_sub(active.y)?;
if crop.width < 2 || crop.height < 2 || x.checked_add(crop.width)? > active.width || y.checked_add(crop.height)? > active.height {
return None;
}
Some(Rect::new(x, y, crop.width, crop.height))
}
fn sensor_crop(maker: Option<&Ifd>, width: usize, height: usize) -> Option<Rect> {
let data = maker?.u64s(SENSOR_INFO)?;
let bounds = data.get(5..9)?;
inclusive_rect(
[
u16::try_from(*bounds.first()?).ok()?,
u16::try_from(*bounds.get(1)?).ok()?,
u16::try_from(*bounds.get(2)?).ok()?,
u16::try_from(*bounds.get(3)?).ok()?,
],
width,
height,
)
}
fn geometry(area: Option<&Cr3ImageArea>, maker: Option<&Ifd>, width: usize, height: usize) -> (Rect, Rect) {
let area = area.filter(|a| usize::from(a.width) == width && usize::from(a.height) == height);
let sensor = sensor_crop(maker, width, height);
let active = area.and_then(|a| a.active).and_then(|b| inclusive_rect(b, width, height)).or(sensor).unwrap_or(Rect::new(0, 0, width, height));
let crop = area.and_then(|a| inclusive_rect(a.crop, width, height)).or(sensor).and_then(|c| relative_crop(c, active)).unwrap_or(Rect::new(
0,
0,
active.width,
active.height,
));
(active, crop)
}
#[derive(Default)]
struct CameraLevels {
wb: Option<[f32; 3]>,
black: Option<[f32; 4]>,
white: Option<f32>,
}
/// Canon's versioned word arrays. Offsets are word indices from the published tag tables, not
/// byte offsets. The four as-shot WB values are R, G, G, B, independent of the CFA layout.
fn camera_levels(maker: Option<&Ifd>, bits: u8) -> CameraLevels {
let Some(values) = maker.and_then(|m| m.u64s(COLOR_DATA)) else { return CameraLevels::default() };
let offsets = match values.first().copied() {
Some(16..=19) => (71, 329, 797),
Some(32 | 33) => (85, 343, 811),
Some(34 | 48) => (105, 363, 641),
Some(64 | 65) => (105, 383, 661),
_ => return CameraLevels::default(),
};
let full = ((1u32 << u32::from(bits.clamp(1, 16))) - 1) as f32;
let quadruple = |at: usize| -> Option<[f32; 4]> {
let v = values.get(at..at + 4)?;
Some([*v.first()? as f32, *v.get(1)? as f32, *v.get(2)? as f32, *v.get(3)? as f32])
};
let wb = quadruple(offsets.0).and_then(|[r, g1, g2, b]| {
let green = (g1 + g2) * 0.5;
if !(64.0..=32767.0).contains(&green) || (g1 - g2).abs() > green * 0.1 {
return None;
}
let red = r / green;
let blue = b / green;
((0.1..8.0).contains(&red) && (0.1..8.0).contains(&blue)).then_some([red, 1.0, blue])
});
let black = quadruple(offsets.1).filter(|v| v.iter().any(|&b| b > 0.0) && v.iter().all(|&b| b >= 0.0 && b < full * 0.5));
let max_black = black.map(|v| v.into_iter().fold(0.0f32, f32::max)).unwrap_or(0.0);
let white = values.get(offsets.2).map(|&v| v as f32).filter(|&v| v > max_black && v >= full * 0.5 && v <= full);
CameraLevels { wb, black, white }
}
fn black_at_active(levels: [f32; 4], pattern: u8, active: Rect) -> BlackLevel {
// Convert Canon's channel order to full-sensor site order, then shift to the active origin.
let sites = match pattern {
1 => [1, 0, 3, 2],
2 => [2, 3, 0, 1],
3 => [3, 2, 1, 0],
_ => [0, 1, 2, 3],
};
let values = (0..4)
.map(|i| {
let x = (active.x + i % 2) % 2;
let y = (active.y + i / 2) % 2;
levels.get(*sites.get(y * 2 + x).unwrap_or(&0)).copied().unwrap_or(0.0)
})
.collect();
BlackLevel { repeat_rows: 2, repeat_cols: 2, values, delta_h: vec![], delta_v: vec![] }
}
pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
let container = parse_cr3(bytes).ok_or(RawError::NotRaw)?;
let track = main_track(&container.tracks).ok_or_else(|| RawError::Unsupported("CR3 without a Bayer raw track".into()))?;
let coding = track.compression(bytes).ok_or_else(|| RawError::Corrupt("CR3 raw track has an invalid CMP1 descriptor".into()))?;
if coding.planes != 4 || coding.encoding != 0 {
return Err(RawError::Unsupported(format!("Canon CR3 encoding {} with {} planes", coding.encoding, coding.planes)));
}
let (width, height) = (coding.width as usize, coding.height as usize);
let samples = width.checked_mul(height).filter(|&n| n > 0 && n <= MAX_SAMPLES).ok_or(RawError::Limit("CR3 sensor dimensions"))?;
let cfa = cfa(coding.cfa_pattern).ok_or_else(|| RawError::Unsupported(format!("Canon CR3 CFA {}", coding.cfa_pattern)))?;
let (offset, len) = track.data.ok_or_else(|| RawError::Corrupt("CR3 raw sample outside file".into()))?;
let sample =
offset.checked_add(len).and_then(|end| bytes.get(offset..end)).ok_or_else(|| RawError::Corrupt("CR3 raw sample outside file".into()))?;
if coding.header_size as usize > sample.len() {
return Err(RawError::Corrupt("CR3 coding header outside sample".into()));
}
crx::validate(sample, &coding)?;
let data = match mode {
Mode::Full => {
let data = crx::decode(sample, &coding)?;
if data.len() != samples {
return Err(RawError::Corrupt("CR3 decoded sample count does not match CMP1".into()));
}
data
}
Mode::Header => vec![],
};
let static_maker = container.cmt.get(2).and_then(|b| *b).and_then(|b| Tiff::parse(b).ok());
let maker = static_maker.as_ref().and_then(|m| m.ifds.first());
// Retain only the first useful timed maker note: repeated CTMD records must not accumulate
// independent copies of every bounded TIFF value in memory.
let timed_maker = container
.timed_exif(bytes)
.into_iter()
.filter(|b| b.tag == 0x927c)
.filter_map(|b| Tiff::parse(b.data).ok())
.find(|t| t.ifds.first().is_some_and(|m| m.contains(COLOR_DATA)));
let color_maker = timed_maker.as_ref().and_then(|t| t.ifds.first()).or(maker);
let area = track.image_area(bytes);
let (active, crop) = geometry(area.as_ref(), maker, width, height);
let levels = camera_levels(color_maker, coding.bit_depth);
let black = match levels.black {
Some(levels) => black_at_active(levels, coding.cfa_pattern, active),
None if active.x >= 8 => black_from_columns(&data, width, 2..active.x - 2, active.y..active.y + active.height, active),
None => BlackLevel::uniform(0.0),
};
let white = levels.white.unwrap_or_else(|| white_from_data(&data, u32::from(coding.bit_depth)));
let mut metadata = lightcraft_meta::extract(bytes);
if metadata.lens_model.is_none() {
metadata.lens_model = maker.and_then(|m| m.string(LENS_MODEL)).map(|s| s.trim().to_owned()).filter(|s| !s.is_empty());
}
metadata.width = Some(crop.width as u32);
metadata.height = Some(crop.height as u32);
let image = RawImage {
format: RawFormat::Cr3,
width,
height,
cpp: 1,
data: RawData::U16(data),
cfa: Some(cfa),
bits: u32::from(coding.bit_depth),
black,
white: vec![white],
active_area: active,
crop,
orientation: metadata.orientation.unwrap_or_default(),
color: ColorData::default(),
wb_multipliers: levels.wb,
linearized: false,
opcodes: OpcodeLists::default(),
metadata,
};
image.validate_for(mode)?;
Ok(image)
}
#[cfg(test)]
mod tests {
use super::*;
use lightcraft_tiff::{IfdBuilder, Value};
fn maker(version: u16, wb_at: usize, black_at: usize, white_at: usize) -> Ifd {
let mut words = vec![0u16; white_at + 1];
words[0] = version;
words[wb_at..wb_at + 4].copy_from_slice(&[2048, 1024, 1024, 1536]);
words[black_at..black_at + 4].copy_from_slice(&[100, 101, 102, 103]);
words[white_at] = 15000;
let b = IfdBuilder::new().with(COLOR_DATA, Value::Short(words));
let bytes = lightcraft_tiff::TiffWriter::new(lightcraft_tiff::ByteOrder::Little, false).write(&[b]).unwrap();
Tiff::parse(&bytes).unwrap().ifds.remove(0)
}
fn box_bytes(kind: &[u8; 4], body: &[u8]) -> Vec<u8> {
let mut b = ((body.len() + 8) as u32).to_be_bytes().to_vec();
b.extend_from_slice(kind);
b.extend_from_slice(body);
b
}
fn track(entry: &[u8], offset: u64, size: u32) -> Vec<u8> {
let mut stsd = vec![0u8; 4];
stsd.extend(1u32.to_be_bytes());
stsd.extend(entry);
let mut stsz = vec![0u8; 4];
stsz.extend(size.to_be_bytes());
stsz.extend(1u32.to_be_bytes());
let mut co64 = vec![0u8; 4];
co64.extend(1u32.to_be_bytes());
co64.extend(offset.to_be_bytes());
let tables = [box_bytes(b"stsd", &stsd), box_bytes(b"stsz", &stsz), box_bytes(b"co64", &co64)].concat();
box_bytes(b"trak", &box_bytes(b"mdia", &box_bytes(b"minf", &box_bytes(b"stbl", &tables))))
}
fn synthetic_header() -> Vec<u8> {
use lightcraft_tiff::{ByteOrder, TiffWriter, tags as t};
let tiff = |b: IfdBuilder| TiffWriter::new(ByteOrder::Little, false).write(&[b]).unwrap();
let cmt1 = tiff(
IfdBuilder::new()
.with(t::MAKE, Value::Ascii("Canon".into()))
.with(t::MODEL, Value::Ascii("EOS Test".into()))
.with(t::ORIENTATION, Value::Short(vec![6])),
);
let cmt2 = tiff(IfdBuilder::new().with(t::ISO_SPEED, Value::Short(vec![800])));
let cmt3 = tiff(
IfdBuilder::new()
.with(SENSOR_INFO, Value::Short(vec![34, 64, 48, 1, 1, 6, 4, 61, 45]))
.with(LENS_MODEL, Value::Ascii("RF-S18-45mm F4.5-6.3 IS STM".into())),
);
let mut canon = vec![0x85, 0xc0, 0xb6, 0x87, 0x82, 0x0f, 0x11, 0xe0, 0x81, 0x11, 0xf4, 0xce, 0x46, 0x2b, 0x6a, 0x48];
canon.extend([box_bytes(b"CMT1", &cmt1), box_bytes(b"CMT2", &cmt2), box_bytes(b"CMT3", &cmt3)].concat());
let mut coding = vec![0u8; 52];
coding[0..2].copy_from_slice(&0xff00u16.to_be_bytes());
coding[2..4].copy_from_slice(&48u16.to_be_bytes());
coding[4..6].copy_from_slice(&0x100u16.to_be_bytes());
for (at, value) in [(8, 64u32), (12, 48), (16, 32), (20, 48), (28, 216)] {
coding[at..at + 4].copy_from_slice(&value.to_be_bytes());
}
coding[24] = 14;
coding[25] = 0x40;
let mut area = vec![0u8; 48];
for (at, value) in [
(4, 64u16),
(6, 48),
(10, 2),
(16, 6),
(18, 4),
(20, 61),
(22, 45),
(28, 3),
(30, 47),
(32, 4),
(36, 63),
(38, 1),
(40, 4),
(42, 2),
(44, 63),
(46, 47),
] {
area[at..at + 2].copy_from_slice(&value.to_be_bytes());
}
let mut craw = vec![0u8; 82];
craw[24..26].copy_from_slice(&64u16.to_be_bytes());
craw[26..28].copy_from_slice(&48u16.to_be_bytes());
craw.extend(box_bytes(b"CMP1", &coding));
craw.extend(box_bytes(b"CDI1", &[vec![0u8; 4], box_bytes(b"IAD1", &area)].concat()));
let levels = maker(48, 105, 363, 641).value(COLOR_DATA).unwrap().clone();
let dynamic_maker = tiff(IfdBuilder::new().with(COLOR_DATA, levels));
let mut item = ((dynamic_maker.len() + 8) as u32).to_le_bytes().to_vec();
item.extend(0x927cu32.to_le_bytes());
item.extend(dynamic_maker);
let mut timed = vec![0u8; 12];
timed[0..4].copy_from_slice(&((12 + item.len()) as u32).to_le_bytes());
timed[4..6].copy_from_slice(&8u16.to_le_bytes());
timed.extend(item);
// Two tiles, each with four one-byte lossless plane streams. Header
// probing checks the complete marker layout without decoding entropy.
let mut sample = Vec::new();
let marker = |out: &mut Vec<u8>, code: u16, size: u32, flags: u32| {
out.extend(code.to_be_bytes());
out.extend(8u16.to_be_bytes());
out.extend(size.to_be_bytes());
out.extend(flags.to_be_bytes());
};
for tile in 0..2 {
marker(&mut sample, 0xff01, 4, tile << 16);
for plane in 0..4 {
marker(&mut sample, 0xff02, 1, (plane << 28) | 0x0800_0000);
marker(&mut sample, 0xff03, 1, 0x0020_0000);
}
}
assert_eq!(sample.len(), 216);
sample.extend([0x1e; 8]);
let moov =
[box_bytes(b"uuid", &canon), track(&box_bytes(b"CRAW", &craw), 4096, 224), track(&box_bytes(b"CTMD", &[]), 4320, timed.len() as u32)]
.concat();
let mut file = box_bytes(b"ftyp", b"crx \0\0\0\x01crx isom");
file.extend(box_bytes(b"moov", &moov));
assert!(file.len() < 4096);
file.resize(4096, 0);
file.extend(sample);
file.extend(timed);
file
}
#[test]
fn header_reads_timed_white_balance_and_distinct_geometry() {
let bytes = synthetic_header();
let image = decode(&bytes, Mode::Header).unwrap();
assert_eq!((image.width, image.height, image.bits), (64, 48, 14));
assert_eq!(image.active_area, Rect::new(4, 2, 60, 46));
assert_eq!(image.crop, Rect::new(2, 2, 56, 42));
assert_eq!(image.wb_multipliers, Some([2.0, 1.0, 1.5]));
assert_eq!(image.black.values, vec![100.0, 101.0, 102.0, 103.0]);
assert_eq!(image.white, vec![15000.0]);
assert_eq!(image.metadata.iso, Some(800));
assert_eq!(image.metadata.lens_model.as_deref(), Some("RF-S18-45mm F4.5-6.3 IS STM"));
assert_eq!((image.metadata.width, image.metadata.height), (Some(56), Some(42)));
assert_eq!(image.orientation, crate::Orientation::Rotate90);
assert!(image.data.is_empty());
assert!(decode(&bytes[..4200], Mode::Header).is_err());
}
fn prepend_tracks(bytes: &[u8], tracks: &[u8]) -> Vec<u8> {
let ftyp_end = u32::from_be_bytes(bytes[..4].try_into().unwrap()) as usize;
let moov_len = u32::from_be_bytes(bytes[ftyp_end..ftyp_end + 4].try_into().unwrap()) as usize;
assert_eq!(&bytes[ftyp_end + 4..ftyp_end + 8], b"moov");
let moov = [tracks, &bytes[ftyp_end + 8..ftyp_end + moov_len]].concat();
let mut out = [bytes[..ftyp_end].to_vec(), box_bytes(b"moov", &moov)].concat();
assert!(out.len() < 4096);
out.resize(4096, 0);
out.extend_from_slice(&bytes[4096..]);
out
}
#[test]
fn hevc_preview_with_equal_dimensions_does_not_replace_the_sensor_track() {
let bytes = synthetic_header();
let mut preview = vec![0u8; 82];
preview[24..26].copy_from_slice(&64u16.to_be_bytes());
preview[26..28].copy_from_slice(&48u16.to_be_bytes());
preview.extend(box_bytes(b"HEVC", &[0; 4]));
let bytes = prepend_tracks(&bytes, &track(&box_bytes(b"CRAW", &preview), 4096, 224));
let container = parse_cr3(&bytes).unwrap();
assert_eq!(container.tracks[0].kind, Cr3TrackKind::Other(*b"HEVC"));
let image = decode(&bytes, Mode::Header).unwrap();
assert_eq!((image.width, image.height, image.bits), (64, 48, 14));
assert_eq!(image.active_area, Rect::new(4, 2, 60, 46));
}
#[test]
fn corrupt_main_descriptor_is_not_replaced_by_the_reduced_raw() {
let bytes = synthetic_header();
let container = parse_cr3(&bytes).unwrap();
let Cr3TrackKind::Raw { cmp1: Some((at, len)), .. } = main_track(&container.tracks).unwrap().kind else { panic!("raw fixture") };
let mut reduced = vec![0u8; 82];
reduced[24..26].copy_from_slice(&32u16.to_be_bytes());
reduced[26..28].copy_from_slice(&24u16.to_be_bytes());
reduced.extend(box_bytes(b"CMP1", &bytes[at..at + len]));
let bytes = prepend_tracks(&bytes, &track(&box_bytes(b"CRAW", &reduced), 4096, 224));
let container = parse_cr3(&bytes).unwrap();
let Cr3TrackKind::Raw { cmp1: Some((at, len)), .. } = main_track(&container.tracks).unwrap().kind else { panic!("main raw fixture") };
let mut malformed = bytes.clone();
malformed[at..at + len].fill(0);
assert!(matches!(decode(&malformed, Mode::Header), Err(RawError::Corrupt(_))));
let mut missing = bytes;
missing[at - 4..at].copy_from_slice(b"free");
assert!(matches!(decode(&missing, Mode::Header), Err(RawError::Corrupt(_))));
}
#[test]
fn color_data_uses_versioned_as_shot_offsets() {
for (version, wb_at, black_at, white_at) in
[(16, 71, 329, 797), (19, 71, 329, 797), (33, 85, 343, 811), (48, 105, 363, 641), (64, 105, 383, 661)]
{
let m = maker(version, wb_at, black_at, white_at);
let levels = camera_levels(Some(&m), 14);
assert_eq!(levels.wb, Some([2.0, 1.0, 1.5]));
assert_eq!(levels.black, Some([100.0, 101.0, 102.0, 103.0]));
assert_eq!(levels.white, Some(15000.0));
}
assert!(camera_levels(Some(&maker(255, 71, 329, 797)), 14).wb.is_none());
assert!(camera_levels(None, 14).black.is_none());
}
#[test]
fn crop_is_relative_to_the_valid_sensor_area() {
let area = Cr3ImageArea {
width: 6888,
height: 4546,
crop: [156, 158, 6875, 4537],
active: Some([144, 46, 6887, 4545]),
masked_left: [0, 0, 143, 4545],
masked_top: None,
};
assert_eq!(geometry(Some(&area), None, 6888, 4546), (Rect::new(144, 46, 6744, 4500), Rect::new(12, 112, 6720, 4380)));
assert_eq!(inclusive_rect([152, 46, 4353, 2851], 4352, 2850), Some(Rect::new(152, 46, 4200, 2804)));
assert_eq!(inclusive_rect([100, 100, 50, 50], 200, 200), None);
assert_eq!(relative_crop(Rect::new(0, 0, 100, 100), Rect::new(4, 4, 92, 92)), None);
}
#[test]
fn first_full_size_track_wins_over_dual_pixel_delta() {
let raw = |w, h, offset| Cr3Track { kind: Cr3TrackKind::Raw { width: w, height: h, cmp1: None, iad1: None }, data: Some((offset, 100)) };
let tracks = [raw(1624, 1080, 1), raw(6888, 4546, 2), raw(6888, 4546, 3)];
assert_eq!(main_track(&tracks).unwrap().data, Some((2, 100)));
}
#[test]
fn cfa_and_black_follow_the_full_sensor_origin() {
for (code, name) in [(0, "RGGB"), (1, "GRBG"), (2, "GBRG"), (3, "BGGR")] {
assert_eq!(cfa(code).unwrap().name(), name);
}
let b = black_at_active([100.0, 101.0, 102.0, 103.0], 0, Rect::new(3, 1, 32, 20));
assert_eq!(b.values, vec![103.0, 102.0, 101.0, 100.0]);
}
}
Loaded 100 of 195 files, more files were not shown because too many files have changed in this diff. Show more