Compare commits

..
Author SHA1 Message Date
Brandon Thomas 53711eb1ef Merge PR #273
# Conflicts:
#	docs/parity.md
2026-10-09 12:04:03 -04:00
Brandon Thomas 1727477382 Merge PR #464 2026-10-09 12:03:42 -04:00
Brandon Thomas 03b3d606ce Merge pull request #546 from storytold/claude/integrate-batch-19
Integrate #474, #448, #447, #499, #459
2026-10-09 12:02:17 -04:00
Brandon Thomas de087c0c6f Merge remote-tracking branch 'origin/main' into claude/integrate-batch-19 2026-10-09 12:02:10 -04:00
Brandon Thomas da6cbcf968 Merge pull request #545 from storytold/claude/integrate-batch-18
Integrate #478, #368, #460, #449, #451, #468
2026-10-09 11:56:38 -04:00
Brandon ThomasandClaude Opus 5.5 95d8aed986 Merge origin/main into profile-gain-table-map (RENDER_CACHE_VERSION: main's 21 + 1 = 22)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:52:23 -04:00
Brandon Thomas 38431cb4db Merge PR #459
# Conflicts:
#	crates/engine/src/import.rs
#	crates/mcp/src/headless.rs
#	crates/raw/src/lib.rs
#	docs/parity.md
#	xtask/src/main.rs
2026-10-09 11:38:44 -04:00
Brandon Thomas abe4c5fe1b Merge PR #499 2026-10-09 11:37:31 -04:00
Brandon Thomas c6745bc6f7 Merge PR #447
# Conflicts:
#	docs/parity.md
2026-10-09 11:37:19 -04:00
Brandon Thomas 093c948f8b Merge PR #448 2026-10-09 11:36:52 -04:00
Brandon Thomas aa1daf5dc9 Merge PR #474 2026-10-09 11:36:51 -04:00
Brandon Thomas 38b8ebdfa5 Merge PR #468 2026-10-09 11:36:32 -04:00
Brandon Thomas 5328100722 Merge PR #451 2026-10-09 11:36:31 -04:00
Brandon ThomasandClaude Opus 5.5 6b671c9912 Merge origin/main into #499 (render cache version 22, as main is at 21)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:45:47 -04:00
Brandon ThomasandClaude Opus 5.5 b7e2f4eff4 Merge origin/main into #447 (render cache version 22, as main is at 21; parity row keeps main's CR3 AspectInfo note)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:39:22 -04:00
Brandon ThomasandClaude Opus 5.5 eae3df481d Merge origin/main into #459 (render cache version 22, as main is at 21; parity row keeps main's CR3 AspectInfo note)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:35:21 -04:00
Brandon ThomasandClaude Opus 5.5 a276febabe Merge origin/main into #448 (render cache version 22, as main is at 21; parity row keeps main's text plus the 3FR note)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:31:04 -04:00
Brandon Thomas 967b858129 Merge PR #449 2026-10-09 09:26:00 -04:00
Brandon Thomas d31f805945 Merge PR #460 2026-10-09 09:25:59 -04:00
Brandon Thomas 12a2a99f2a Merge PR #368 2026-10-09 09:25:58 -04:00
Brandon Thomas 5e6d33065a Merge PR #478 2026-10-09 09:25:57 -04:00
Brandon Thomas 65ad290e89 Merge pull request #512 from storytold/claude/integrate-batch-17
Integrate #489, #493, #495, #494, #444, #484, #477
2026-10-09 09:25:11 -04:00
Brandon Thomas e111734532 Merge remote-tracking branch 'origin/main' into claude/integrate-batch-17 2026-10-09 09:24:54 -04:00
Brandon Thomas 8f924a9e60 Merge pull request #511 from storytold/claude/integrate-batch-16
Integrate #453, #470, #482, #492, #433, #465
2026-10-09 09:08:47 -04:00
Brandon ThomasandClaude Opus 5.5 7a42f99cf0 MCP: the export tool forwards every option it advertises
preset, background, resize, subfolder, conflict, tiffCompression and
dngCompression were in the export tool's schema (and so passed the new
argument check) but were dropped before reaching app.export.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:05:04 -04:00
Brandon Thomas 081a619c51 Merge PR #477 2026-10-09 09:04:07 -04:00
Brandon Thomas 7e350eccb8 Merge PR #484 2026-10-09 09:04:06 -04:00
Brandon Thomas 8abcd2fc54 Merge PR #444 2026-10-09 09:04:05 -04:00
Brandon Thomas b7331d6c13 Merge PR #494 2026-10-09 09:04:04 -04:00
Brandon Thomas 5bd4302037 Merge PR #495 2026-10-09 09:04:03 -04:00
Brandon Thomas 1b4e646df5 Merge PR #493 2026-10-09 09:04:02 -04:00
Brandon Thomas 9fef5ac702 Merge PR #489 2026-10-09 09:04:01 -04:00
Brandon Thomas 0b6cfd89ec Merge remote-tracking branch 'origin/main' into fix-449 2026-10-09 08:57:20 -04:00
Brandon ThomasandClaude Opus 5.5 56b06426db docs/parity.md: drop the radial-drag note from Top gaps (the LR-MASK-RADIAL row carries it)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 08:57:20 -04:00
Brandon ThomasandClaude Opus 5.5 d418292fb0 Show in Explorer: drop quotes from the path so the raw command line stays one argument
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 08:56:53 -04:00
Brandon Thomas 11e739cd02 Merge PR #465 2026-10-09 08:34:42 -04:00
Brandon Thomas 7317e64d74 Merge PR #433
# Conflicts:
#	crates/engine/src/media.rs
2026-10-09 08:34:41 -04:00
Brandon Thomas ceb35048bb Merge PR #492
# Conflicts:
#	crates/engine/src/media.rs
2026-10-09 08:34:40 -04:00
Brandon Thomas f731922e16 Merge PR #482
# Conflicts:
#	crates/engine/src/media.rs
2026-10-09 08:34:38 -04:00
Brandon Thomas a18a6e6a5e Merge PR #470 2026-10-09 08:34:37 -04:00
Brandon Thomas ffd91332e9 Merge PR #453 2026-10-09 08:34:36 -04:00
Brandon ThomasandClaude Opus 5.5 8d96dab1af LR-IMP-FORMATS: checked sum of the sRAW slice widths (an overflowing or zero sum is an error, not a panic)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 08:24:34 -04:00
Brandon ThomasandClaude Opus 5.5 761631c56c Quit from the macOS menu: save settings and the library (and honour the unsaved-changes prompt) before exiting
std::process::exit skipped eframe's on_exit (ui.json, close_library) and the
may_close check, so changes still in memory could be lost on Cmd+Q.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 08:19:34 -04:00
Brandon ThomasandClaude Opus 5.5 0a4dc3f3ec Merge main into #433 (parity row: keep main's text plus the AspectInfo note; render cache version 17, as main is at 16 already)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 08:16:13 -04:00
Brandon Thomas 9eb8151984 Merge origin/main 2026-10-09 08:05:59 -04:00
radialmonster cdc2e57776 LR-PROF-CAMERACOLOR: docs: archive-wide numbers for the tone fit and its fallback
The earlier text quoted the 13 investigation files (held-out RMS within 0.005); the whole CC0 archive shows a worst increase of 0.013 and 7 files that need the conditional-median fallback.
2026-10-09 07:31:53 -04:00
rock-meisterandClaude Opus 5.5 369c56a6ed LRC-LIB-CATALOG-IMPORT: import straightened catalog crops as Lightroom shows them
With a CropAngle, Lightroom's CropLeft/CropTop and CropRight/CropBottom
are the top-left and bottom-right corners of the tilted crop on the
stored image, and its angle turns the other way to LightCraft's. The
import treated the edges as an axis-aligned rectangle in the rotated
frame and kept the angle's sign, so straightened crops came out the
wrong shape and size.

Carry the two corners through the orientation and then LightCraft's
straighten rotation (angle = -CropAngle, mirrored orientations flip it
back), using the stored image's aspect from fileWidth/fileHeight. On a
real catalog every one of 223 straightened crops now matches the frame
of Lightroom's export, and the content difference falls to that of
unstraightened crops.

Refs #483

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 06:26:47 -05:00
Elliot GrahamandCodex b9f951998f #491: Keep Local import headers within the centre pane
Co-Authored-By: Codex <noreply@openai.com>
2026-10-09 12:08:11 +01:00
radialmonster c691bcfa02 LR-PROF-CAMERACOLOR: fall back to the conditional-median tone fit when the quantile look misses the gates
Quantile matching fits the held-out pixels slightly worse than the conditional-median fit it replaced. On 7 of 1023 archive files (compacts, bridge cameras, early Micro Four Thirds) the look passed the acceptance gates only with the old curve, and the quantile curve lost it. The search now runs with the quantile fit first (all pixels, away from edges, with and without the camera profile) and, only when no look passes, runs again with the earlier conditional-median plus isotonic fit, restored as fit_tone_conditional. The gates are unchanged, so no file loses its look relative to the previous fit, and the quantile curve is used wherever it passes. Tests: the flat-step test stays; a synthetic photo whose JPEG carries per-pixel noise misses the gates with the quantile fit and passes with the fallback, and the order of the two is checked.
2026-10-09 07:08:01 -04:00
rock-meisterandClaude Opus 5.5 2ec2b5bf4a LRC-LIB-CATALOG-IMPORT: carry catalog crops through the photo orientation
crs crop edges are normalized to the stored (unrotated) image, while
LightCraft's crop rectangle lives in the oriented frame. Catalog import
applied them unchanged, so crops on portrait photos (EXIF 6 or 8) landed
along the wrong axes.

Read Lightroom's orientation code (Adobe_images.orientation: AB, BC, CD,
DA and the mirrored BA, CB, DC, AD) and map the rectangle into the
oriented frame with Orientation::map_norm_rect; a mirrored orientation
also mirrors the straighten angle. The code comes from the catalog, so
this works while the originals are still missing at import time.

Refs #483

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 05:49:59 -05:00
radialmonster e5ff5461e3 LR-IMP-FORMATS: render cache version 21 for the Canon sRAW/mRAW decode
Open PRs already use 17 to 20, so this one takes the next free number.
2026-10-09 06:33:11 -04:00
MarekSpodBiedry 44735cd5e7 Issue #272: Show in Explorer selects photos whose path has a space
Explorer only selects a file when the path after /select, is quoted. Rust's argument quoting wraps the whole "/select,C:\My Photos\a.jpg" switch in quotes as soon as the path has a space, so Explorer opened Documents. Build the switch as /select,"<path>" (backslashes, verbatim \\?\ and \\?\UNC\ prefixes stripped) and pass it with raw_arg on Windows. The library folder tree's context menu now shows the platform's reveal label too, and the Russian catalog gains "Show in Explorer" and "Show in File Manager".
2026-10-09 12:19:27 +02:00
rock-meisterandClaude Opus 5.5 c5449926b9 Contributors: add rock-meister (sync GitHub name)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 05:15:52 -05:00
rock-meisterandClaude Opus 5.5 fcef4c0fc9 LRC-LIB-CATALOG-IMPORT: import catalog crops that have no HasCrop
Lightroom Classic catalogs store the crop edges and straighten angle in
their develop settings but, unlike XMP, never a HasCrop flag. The shared
crs mapper only applies a crop when crs:HasCrop is True, so every crop in
an imported catalog was dropped and reported as unmapped.

Infer HasCrop in the catalog path when any crop field is present. The XMP
path is unchanged: there HasCrop="False" with crop fields means the crop
is switched off.

Refs #483

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 05:15:52 -05:00
radialmonster 0778874d13 LR-PROF-CAMERACOLOR: bump the render cache version for the new tone fit
Renders change for files with a fitted camera look. Open PRs already claim versions 17, 18 and 19.
2026-10-09 05:45:01 -04:00
radialmonster 24427912cb LR-PROF-CAMERACOLOR: fit the camera tone curve by quantile matching (no flat steps)
The per-photo look fitted its tone curve as the camera JPEG's median per
scene-luminance bin followed by isotonic pooling. On lens-corrected, sharpened
JPEGs of busy scenes neighbouring bins disagree by more than the gap between
them, so pooling produced flat steps: 13 of 31 segments on a CC0 Z 6II sample,
flat segments in 9 of 13 fitted Nikon Z / ILCE-7 samples. Each step maps a
range of scene tones to one output tone, which renders as posterised patches
with hard edges (issue #475, also #211, #454), and the conditional medians
flatten both ends of the curve.

Quantile matching (sorted scene quantiles against sorted JPEG quantiles) is
increasing by construction and insensitive to misregistration. On the same 13
files no fit has a flat segment, all still pass the gates, held-out RMS within
0.005 of before. Regression test: a misregistered synthetic JPEG.
2026-10-09 05:41:24 -04:00
hxwandClaude Opus 5.5 da62ea8c49 LR-BEHAV-GPU: compile DX12 shaders with FXC, never a stray dxcompiler.dll (issue #471)
wgpu's default DX12 compiler (Auto) loads whichever dxcompiler.dll the DLL
search path finds first. LightCraft ships none, so it picked up another
program's copy; one without dxil.dll beside it (e.g. DXC 1.7) warns that
the DXIL is unsigned, wgpu treats that as a compile error, its
indirect-validation pipelines fail, the device is lost and the window
never opens ("Parent device is lost", exit code 1).

Every wgpu instance (window and both compute devices) now uses
lightcraft_gpu::backend::backend_options(): wgpu's env options with the
DX12 compiler pinned to FXC (d3dcompiler_47.dll, part of Windows), unless
WGPU_DX12_COMPILER asks for another.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 04:34:51 -05:00
radialmonster cfca299945 LR-IMP-FORMATS: report Sony A200/A230/A350 packed ARW as unsupported (preview fallback)
The one-strip Sony ARW layout was accepted when the raw strip was 1.00-1.25x width x height bytes. The DSLR-A200, A230 and A350 sample files carry strips of 1.06x, 1.23x and 1.04x, passed that test and decoded as colour noise. Every file that decodes correctly (108 in the test archive) has a strip of exactly width x height bytes, so the layout is now accepted only then; the rest report "Sony ARW version 1 / packed compressed variant" and open from the embedded preview like the A290/A380. Part of #461 (not a decoder). RENDER_CACHE_VERSION 16 -> 19.
2026-10-09 05:29:23 -04:00
BuonBredaandClaude Opus 5.5 fa1565d855 LR-PROF-CAMERACOLOR: camera local tone mapping (ProfileGainTableMap) as a per-photo opt-in
Renders a raw's DNG ProfileGainTableMap (Apple ProRAW's local tone mapping)
when the photo's Profile option "Camera local tone mapping" is on
(`profile.camera_local_tone`, off by default: Lightroom Classic renders
ProRAW without it). Built on #427's parser; supersedes the decode-time
applier of the first version of #273.

- raw/gaintable: GainTableMap::gain / evaluator (GainEval) per DNG 1.7.1:
  bilinear between tables with edges replicated, input clamp(weights .
  (R,G,B,min,max))^gamma in linear ProPhoto after the baseline exposure,
  index input x MapPointsN clamped to the last point. Saturating cell
  math; maps whose sizes don't match their gains aren't evaluated.
  SourcePlacement: where a developed source sits in the active area.
- pipeline/local_tone: the gain per output pixel in the scene-linear
  stage, before the white balance, mapped back through the frame (crop,
  user orientation, warp) and the EXIF orientation. lin_needs_cpu routes
  the stage to the CPU when it's on (CPU and GPU match); lin_key includes
  it. SourceInfo::local_tone carries the map from the decoder.
- develop: Profile::camera_local_tone (omitted from JSON when off, so
  existing settings and their hashes are unchanged); Profile row checkbox
  when the source has a map; translations.
- RENDER_CACHE_VERSION 17. Docs: parity row and What's New as an opt-in.

Off renders pixel-identical to main on two iPhone 15 Pro ProRAWs (12 and
48 MP). On adds ~60-75 ms to the scene-linear stage at 1536x2048 on a
2-core VM, the same for both files (cost follows output pixels).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VFqow2P1EtDxt6nRjjQoTA
2026-10-09 11:11:52 +02:00
MarekSpodBiedry bbcd2f47e8 Web restore: reject originals that don't match their content hash (#227) 2026-10-09 10:46:29 +02:00
MarekSpodBiedry 5fc85ee034 Fix #226: a refused Web Locks request no longer leaves the web app loading 2026-10-09 10:45:45 +02:00
radialmonster cec18712d6 LR-IMP-CAMERA-COVERAGE: xtask corpus verifies the sha256 of every downloaded corpus file
Each RAW_SAMPLES entry now carries its SHA-256 (computed from the file as downloaded; the ProRAW value was only a comment). cargo xtask corpus --download hashes every file, new or already present, deletes any mismatch so the next run fetches it again, and fails listing the actual and expected hashes. A unit test keeps names unique and hashes well-formed.
2026-10-09 04:43:19 -04:00
radialmonster 3af7a0c0ae LR-IMP-FORMATS: drop the 40D luma floor from Canon sRAW decoding
The 512 floor and its ColorDataVersion 3 keying rested on one file.
Versions without a known black-level position now fall back to black 0
for the white level, so the 40D sRAW renders with slightly raised blacks.
2026-10-09 04:18:46 -04:00
peiyan0 e7da5d34a2 catalog: group photos without capture date under Unknown Date (#437) 2026-10-09 16:09:23 +08:00
radialmonster 1399c445ef LR-IMP-FORMATS: read the PEF colour-filter layout from the Exif CFAPattern for Huffman-compressed files too
The Huffman-compressed Pentax files (compression 65535) assumed BGGR from
the array origin. The K10D, K200D, K2000 and every body from the K-3 on are
RGGB, so red and blue were exchanged before demosaicing. They now use the
Exif CFAPattern from the first sample of the stored array, like the other
compressions; BGGR stays the fallback for files without the tag. Extends the
CFA test to compression 65535. RENDER_CACHE_VERSION 17 to 18.
2026-10-09 04:02:20 -04:00
Claude a45c2b661b M11.1: decode EOS R7 C-RAW (odd-height CRX QP map)
An odd-height adaptive QP map (EOS R7: 4732 sensor rows, 1183 QP rows)
was rejected as unsupported, so R7 C-RAW files opened from their embedded
JPEG. The map decodes to the exact end of its entropy stream; the last
row of the second-level high-frequency bands has no partner row and its
QP is used alone. Even-height maps never reach this rule, so the six
verified M50/R100/R8 fixtures are unchanged.

Checked on paired R7 RAW and C-RAW captures (ISO 100 and 6400): all
six C-RAW files decode, developed images match their RAW counterparts
visually, and ISO 6400 sensor means agree within 0.1%.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gs6roPby7THvteijZwptQt
2026-10-09 07:38:24 +00:00
radialmonster c7719ddcbf LR-IMP-FORMATS: read the PEF Exif CFAPattern from the array origin, not the image-area origin
The pattern was shifted by the image-area origin for compression 1 and
32773. The *ist D (area origin 19, 11, tag RGGB) fits RGGB from the array
origin in a mosaic-level fit against its camera JPEG, so the shifted reading
exchanged red and blue. The six 32773 files have an even origin and keep
their layout. Adds a hand-made-bytes test with an odd area origin for both
compressions.
2026-10-09 03:28:35 -04:00
supper_yijie bf35a3c6c5 fix #425: Restore Simplified Chinese font rendering
1.Added Simplified Chinese font support for web and desktop builds.
2.Kept Inter as the default font, with Chinese used only as a fallback.
3.Verified Chinese glyph tests and a macOS universal build.

Signed-off-by: supper_yijie <syj794844854@163.com>
2026-10-09 15:12:33 +08:00
radialmonster d9fd2f8ae0 LR-IMP-FORMATS: decode Samsung SRW files with uncompressed raw data
Samsung SRW raws whose raw IFD (private compression 32769 or 32770) stores the samples uncompressed are decoded from the sensor data: 12-bit samples packed two in three bytes (bit order told apart from the samples; NX5, NX10, NX11 and NX20, NX200, NX210, NX1000, NX1100) and 16-bit little-endian words (EX1, WB2000). CFA layout from the raw IFD's CFAPattern; framing, white balance, black and saturation levels from the maker note (value offsets relative to the note; the vectors are stored offset by its EncryptionKey); a one-direction Exif picture size crops to the centre of the framing; black from dark rows or columns when the file states none; saturation plateau below the stated level wins. Compressed codings (32770 with fewer bytes, 32772, 32773) stay preview only with that reason. srw is imported from folders; SRW takes the file-local camera look. RENDER_CACHE_VERSION 17.
2026-10-09 02:52:18 -04:00
radialmonster 1206d3fd52 LR-IMP-FORMATS: decode Canon sRAW and mRAW CR2 from their YCbCr data
sRAW and mRAW store subsampled YCbCr in a lossless-JPEG frame cut into
slices of whole MCUs. Reuse the subsampled lossless decoder added for
Sony M/S ARW with Canon's per-component sequential prediction, re-chop
the stream into image planes, and convert to linear white-balanced RGB
with chroma gains measured per file against the camera's own IFD2 RGB.
Render cache version 17.
2026-10-09 02:47:10 -04:00
midasdf 35718bc92c M1: keep deletion selection near the current photo 2026-10-09 15:04:40 +09:00
storybored379andOpenAI Codex dedc049692 MCP: add export progress and cancellation between photos
Answer ping at photo boundaries, preserve queued requests and EOF completion, and suppress cancelled responses. Keep completed exports and the existing atomic file writer; cover cancellation, output preservation and transport poison recovery.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-10-08 23:02:21 -07:00
storybored379andOpenAI Codex 5c16320f6b MCP: add shared core tools, hints and resources
Keep documented helpers listed, reject unknown helper keys, and recover escaped backend panics. Cover core tools, resource snapshots and modern result metadata with regression tests.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-10-08 23:02:13 -07:00
Brandon Thomas c435d143de Merge pull request #445 from storytold/claude/fix-freebsd-ci
CI: restore green FreeBSD builds
2026-10-09 01:38:58 -04:00
Brandon Thomas 82fc317922 Merge remote-tracking branch 'origin/main' into fix-445
# Conflicts:
#	crates/engine/src/media.rs
2026-10-09 01:36:13 -04:00
Hanmo 40652fc5f9 M6: move radial masks by dragging inside their ellipses 2026-10-09 13:35:02 +08:00
radialmonster 989422398c LR-IMP-FORMATS: read plain packed 12-bit PEF strips tagged as PackBits (older Pentax, Samsung GX-1L)
Compression 32773 is accepted in the PEF reader only when every strip has exactly the packed size of its rows at the declared bit depth, and is then read as uncompressed MSB-first samples. Any other 32773 size stays unsupported. Uncompressed PEFs take their colour filter from the Exif CFAPattern as seen from the image area origin. A Samsung-branded TIFF whose maker note has the Pentax layout is probed as PEF. RENDER_CACHE_VERSION 16 to 17.
2026-10-09 01:24:32 -04:00
Brandon Thomas 68e45585d0 Merge pull request #384 from voshart/pr/denoise-standalone
AI RAW denoise with opt-in models and pure-Rust CPU/GPU inference
2026-10-09 01:08:46 -04:00
Brandon ThomasandClaude Opus 5.5 7dcfbed05c LR-EDIT-DETAIL-DENOISE: list settings tabs from TABS; fix the SAM 3 worker's wasm check
The unknown-tab error now names every Settings tab (it missed AI Denoise after the
faces merge). lightcraft-sam3-worker (#359) only builds its server off wasm32, so
`cargo xtask wasm` passes again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 01:07:43 -04:00
Brandon ThomasandClaude Opus 5.5 d9457e6b23 CI: generate each procedural demo scene once per process (FreeBSD UI tests timing out)
Every demo-photo source load evaluated the procedural scene from scratch. A 24 MP demo
original costs ~200 CPU-seconds (AlpineLake, the photo the Detail tests open), and #323
added ~30 loupe tests that each load it at 1:1, Before and After separately. On the 4-vCPU
FreeBSD VM the ui-egui lib tests went from 1198 s to 3046 s, and the tests' 120 s/300 s
waits expired (loupe windows, fixture thumbnails starved of the shared rayon pool, the demo
detail snapshot).

Demo scene pixels are now generated once per (scene, size) and shared: concurrent loads
wait for one generation (Before/After at 1:1 no longer generate the original twice) and
the most recent results are kept up to half the memory budget. Locally, pinned to 4 CPUs,
the ui-egui lib suite now finishes in 155 s; without the change it had done 231 of 338
tests after 25 minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 01:03:40 -04:00
onemannoplan 5c1dcd4f97 fix(macos): prevent SIGILL crash on exit by replacing viewport close with clean process exit
### Summary
When selecting "Quit LightCraft" from the native macOS menu bar, the application occasionally crashes with an `EXC_BAD_INSTRUCTION` (SIGILL) error report during shutdown.

### Root Cause
1. The native menu action triggered `ctx.send_viewport_cmd(egui::ViewportCommand::Close)`.
2. Forcing an explicit viewport close via the command queue while macOS and AppKit are initiating their own termination sequence creates a race condition with `winit`.
3. This conflict during native window, view, and observer teardown leads to an illegal instruction fault.

### Solution
- Replaced the manual `ViewportCommand::Close` trigger on `QUIT` with a clean `std::process::exit(0)`.
- This bypasses the problematic `winit`/AppKit teardown race condition on macOS, allowing the application to shut down instantly and cleanly without generating crash reports.
2026-10-08 23:53:21 -05:00
Brandon ThomasandClaude Opus 5.5 1e1eb5fe16 Merge origin/main (batches 14 and 15) into AI RAW denoise (#384)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 00:30:17 -04:00
Brandon ThomasandClaude Opus 5.5 c65065de2f Merge the faces stack (#172/#173/#174, integrate-batch-14) into AI RAW denoise (#384)
One config-dir helper (engine::config::config_dir; camera_profiles re-exports it) with both
the face-models and denoise-models defaults; hosts set up both model folders; Settings has
Faces and AI Denoise tabs; both command sets registered; both model dialogs; locale entries
from both; one [features] table per app manifest; Cargo.lock regenerated by cargo; the
DecodedSource cache cost counts both segmentation mattes and the denoised twin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 00:29:59 -04:00
Brandon Thomas 3e76713c1d Merge pull request #440 from storytold/claude/integrate-batch-15
Integrate #423, #359
2026-10-09 00:24:40 -04:00
radialmonster 79b19fd886 LR-IMP-FORMATS: decode Hasselblad 3FR files with uncompressed raw data
TIFF raws that describe a full-resolution CFA IFD with DNG's tags (levels, crop, AsShotNeutral) but carry no DNG version tag are read by the DNG reader (RawFormat::CfaTiff, dng::decode_as). RGGB when the file states no CFAPattern. Lossless JPEG scans with a predictor selection value above 7 are rejected with that reason and stay preview-only. .3fr and .fff are imported from folders. RENDER_CACHE_VERSION 17.
2026-10-09 00:24:39 -04:00
radialmonster 82f904495f LR-IMP-FORMATS: decode lossy JPEG XL DNG tiles completely and without clipping
Decode every JPEG XL tile with wide (32-bit) modular buffers: the
modular_16bit_buffers header flag does not hold on real lossy files (68
of 108 tiles failed with narrow buffers).

Lossy (XYB) tiles are taken from the decoder as linear sRGB primaries
and restored to the declared primaries with a plain 3x3 matrix, instead
of the decoder's gamut-mapped, [0, 1]-clipped conversion. Lossy tiles
whose declared encoding cannot be restored faithfully (ICC profile,
non-linear transfer function, non-D65 white point, intensity target
above 255 nits, single plane) are unsupported. Lossless output is
unchanged.

A JPEG XL tile that fails to decode is now an error for the image
(the file falls back to its embedded preview) instead of a zero tile.

Render cache version 17.
2026-10-09 00:21:39 -04:00
Brandon Thomas ae2f06a035 Merge PR #359 (fmt) 2026-10-09 00:20:30 -04:00
Brandon ThomasandClaude Opus 5.5 b341101ab6 Remote SAM3 worker: cargo fmt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 00:20:28 -04:00
Brandon Thomas 7365d06c13 Merge PR #359 2026-10-09 00:17:59 -04:00
Brandon Thomas ab73d3d2e0 Merge PR #423 2026-10-09 00:17:58 -04:00
Brandon Thomas c4c96ecabb Merge pull request #439 from storytold/claude/integrate-batch-14
Integrate #429, #426, #422, #395, #172, #173, #174, #335, #427
2026-10-09 00:16:54 -04:00
Brandon Thomas cae7bea08e Merge remote-tracking branch 'origin/main' into fix-359
# Conflicts:
#	crates/engine/src/segment/worker.rs
2026-10-08 23:57:36 -04:00
Brandon Thomas 24615cc2d7 Merge PR #335 (clippy fix) 2026-10-08 23:57:23 -04:00
Brandon ThomasandClaude Opus 5.5 d0749ed7a9 HEIC: as_chunks for 16-bit samples (clippy 1.99, caught by the new heif CI step)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:57:22 -04:00
Brandon ThomasandClaude Opus 5.5 bb7c46ae3c Remote SAM3 worker: crate lint attributes; document the trust boundary
The worker binary gets the never-crash deny list and forbid(unsafe_code) like
every production crate root. docs/ai-masks.md spells out who can reach the
unauthenticated loopback listener.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:55:26 -04:00
Brandon ThomasandClaude Opus 5.5 7db3b8a1f9 Batch 14: one [features] table in apps/lightcraft (faces + heif), parity summary
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:53:54 -04:00
Brandon Thomas a628d5bc38 Merge PR #427
# Conflicts:
#	ROADMAP.md
#	docs/parity.md
2026-10-08 23:46:58 -04:00
Brandon Thomas 42f966eec2 Merge PR #335 2026-10-08 23:46:17 -04:00
Brandon Thomas ca97329ae1 Merge PR #174 2026-10-08 23:45:49 -04:00
Brandon Thomas 3368eb49ee Merge PR #173 2026-10-08 23:45:49 -04:00
Brandon Thomas cbf9935b6e Merge PR #172
# Conflicts:
#	docs/parity.md
2026-10-08 23:45:49 -04:00
Brandon ThomasandClaude Opus 5.5 4977c648f6 Merge main into formats/heif-heif-decode; CI gates the heif feature
Resolve the docs/parity.md conflict, keep the HEIC rows. `cargo xtask ci` gets
a heif step (clippy + tests for lightcraft-codecs with --features heif), since
the workspace build leaves the feature off. The Cargo.toml comments no longer
say official builds enable it; the web one notes that wasm32 aborts on panic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:24:28 -04:00
Brandon Thomas 91d317900f Merge PR #395 2026-10-08 23:22:56 -04:00
sanyamsingh001andCodex 56754e5703 LR-BEHAV-HEADLESS: keep CPU snapshots isolated from saved preferences
Co-authored-by: Codex <codex@openai.com>
2026-10-09 13:19:47 +10:00
radialmonster 04e8aea577 LR-IMP-FORMATS: CR3 photos render in the aspect ratio set in the camera
The Canon maker note's AspectInfo (tag 0x009a: aspect code, cropped width,
height, left, top) holds the rectangle of the shot, measured from the
recommended image crop. Use it as the CR3 default crop, so 3:2, 4:3, 16:9 and
1:1 shots develop in the shape the photographer chose. The tag is ignored when
it is missing, does not fit, or the recommended crop itself is inconsistent.

RENDER_CACHE_VERSION 15 -> 16. The M50 C-RAW corpus sample (raw.pixls.us 2663)
was shot at 1:1 and now develops at 4000 x 4000.
2026-10-08 23:06:57 -04:00
Brandon Thomas 1a2d2ab975 Merge PR #422 2026-10-08 22:43:10 -04:00
Brandon Thomas 1f30856774 Merge PR #426 2026-10-08 22:43:10 -04:00
Brandon Thomas 05769798f6 Merge PR #429 2026-10-08 22:43:10 -04:00
Daniel Voshart 59ab9b6e9b LR-EDIT-DETAIL-DENOISE: refresh the current parity summary
All seven gates pass with the hosted fonts and the earlier shared window-test correction.

Generated with Codex.
2026-10-08 22:31:04 -04:00
Daniel Voshart 111cc893ec LR-LIB-PEOPLE: preserve later person navigation on current main
Port the original Escape return and last-person shortcut behavior, keeping current translations and a no-wrap chip. The existing headless regressions and all seven quality gates pass with the hosted fonts on the refreshed recognition dependency chain.

Generated with Codex.
2026-10-08 22:29:31 -04:00
Daniel Voshart 24f87a91a0 LR-PREF-PEOPLE: update pure-Rust recognition and preserve valid model replacements
Keep current upstream UI behavior and the later detector contract checks. Validate candidate models before replacing working weights and records, with a synthetic regression that failed before the fix. Remove the unrelated GPU equivalence-test relaxation. All seven quality gates pass with the pinned hosted fonts.

Generated with Codex.
2026-10-08 22:29:31 -04:00
Daniel VoshartandClaude Opus 5.5 20a801d6d2 Faces: YuNet face detection and one-click model downloads, with nothing bundled
Photo > Detect Faces runs the YuNet detector (a small pure-Rust interpreter over the ONNX graph, no
runtime dependency) and adds the faces it finds as unnamed face regions in one undo step. The detector
and the recognition models are opt-in downloads: Settings > Faces shows each model's licence and
training-data note, Download stays disabled until the terms are accepted, and the file is fetched over
https with lightcraft-fetch from the model's own repository at a commit-pinned address, checked against
its recorded size and SHA-256 and then installed. Detect Faces offers the detector's download the first
time it is used. No model file is part of the repository (the real-model tests read LC_YUNET_MODEL; two
ignored tests download from the real hosts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 22:29:30 -04:00
Daniel Voshart 03b665e7a3 LR-PREF-PEOPLE: refresh the generated parity summary
Generated with Codex.
2026-10-08 22:29:29 -04:00
Daniel Voshart fd4019300a LR-EDIT-DETAIL-DENOISE: address installation and model-policy review
Generated with Codex.
2026-10-08 22:07:23 -04:00
7935999d94 LR-EDIT-DETAIL-DENOISE: non-destructive AI RAW denoise with pure-Rust CPU/GPU
Recover the unpublished denoise work against current upstream. Preserve later opt-in model setup, shared Rust HTTPS downloads, pending-photo resume and an Amount-independent switch. Keep original RAWs unchanged and share the current color finish between ordinary and denoised sources.

Bound public tiling inputs, refuse corrupt-cache exports, and retain synthetic/real-model coverage. Include two Windows path spelling fixes in existing tests. No model weights, face code, media or Tract runtime are included.

Validation: cargo xtask ci (all seven gates); installed RawNIND CPU/GPU agreement 1.38e-6; native CPU preview/export on a public 24 MP Sony RAW, unchanged original SHA-256.

Co-authored-by: Codex <noreply@openai.com>

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

Generated with Codex.
2026-10-08 22:07:20 -04:00
Daniel Voshart 6e8db91673 LR-PREF-PEOPLE: refresh model manager for current platform helpers
Keep upstream tests and the fixed SAM worker, refresh the workspace version and parity summary, and route the model picker through the case-insensitive native filter helper. Carry the shared translated font-symbol correction; validation includes the pinned hosted fonts and all seven CI gates.

Generated with Codex.
2026-10-08 21:59:02 -04:00
Daniel VoshartandClaude Sonnet 5.5 7845d515e8 Faces: nothing is bundled, so YuNet is not reported as installed
The model list marked the YuNet detector as bundled with LightCraft and installed, but no model ships
inside LightCraft (the weights are opt-in downloads or files the user adds). Drop the notion: a
model is installed when its folder is, it can be removed like any other, and the licence notice no
longer says it is bundled.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:59:02 -04:00
Daniel VoshartandClaude Sonnet 5.5 cb34bb4842 Faces: Settings > Faces tab and the add-a-model licence dialog
An off-by-default "Recognise faces" toggle and the face model list in Settings,
with a one-drop install: pick a .onnx (Add a model file…) or drop one on the
window. The engine inspects it (recognised by hash, or described from its
shape), a dialog shows its licence, whether commercial use is allowed, what it
was trained on and what LightCraft had to assume, and Install stays disabled
until "I accept" is ticked. "Get…" opens a model's own page in the browser:
LightCraft never fetches a model itself. A file it cannot use says why.

The models folder is shared by the app, the CLI and the MCP server
(`engine::config`, `$LIGHTCRAFT_FACE_MODELS` overrides `<config>/models`), so a
model installed from one is there for the others. A session has none until a host
asks (`with_default_face_models`), which keeps tests hermetic.

Nothing runs a model yet, and the tab says so. Headless tests cover the whole
flow (the dialog, refusing until accepted, installing, a file it cannot use).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:59:01 -04:00
Daniel VoshartandClaude Sonnet 5.5 7ca7113b32 Faces: model commands - list, inspect, install behind the licence, select, remove, on/off
`faces.models.list / inspect / install / remove / select` and `faces.enable`, all
reachable from the UI, CLI, control channel and MCP. Models live in a folder the
host provides (`Session::face_models_dir`): one subfolder each with the model, its
manifest and a record of what the user accepted, plus settings.json.

- install needs `acknowledged: true` (the user has been shown the licence), copies
  the file under a .part name, checks the copy's SHA-256 against the original, then
  renames; a failure leaves nothing behind.
- ids are validated before they become folder names (and a manifest that disagrees
  with its folder is ignored); remove only ever deletes a folder that has a model
  manifest. A manifest id can no longer be `.`, `..` or start with a dot.
- inspect describes any .onnx without writing anything; unusable files are an
  answer ("unsupported" with the reason), not a crash.
- the list says plainly that running a model is not built in yet.

`faces::synthetic` makes a minimal valid ONNX for tests (and the later self-check).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:59:01 -04:00
Daniel VoshartandClaude Sonnet 5.5 39de0e6a1c Faces: model manifests, known-model table and an ONNX probe (no runtime yet)
New L1 crate `lightcraft-faces`, the base of a one-drop model install:
- `ModelManifest` (role, licence with a commercial yes / no / unknown flag and a
  plain-language notice, provenance, input preprocessing, output, thresholds)
  with strict validation: sizes capped, numbers finite and sane, https-only
  addresses, bounded text. A manifest from any source is untrusted.
- `known`: models recognised by SHA-256 (YuNet, SFace, AuraFace v1), with what is
  and is not known about their licences and training data.
- `onnx`: a streaming, hostile-input-safe protobuf probe of a model's graph
  inputs and outputs. It reads a 248 MB model in about 4 ms and never loads the
  weights; truncation at every length and random corruption never panic.
- `suggest`: known hash -> its manifest; an unknown model shaped like a face
  embedder -> a draft manifest with the assumptions spelled out; anything else
  -> a plain reason it is not supported yet.
- `hash`: streamed SHA-256.

Checked against the real model files with an opt-in test (LC_FACE_MODELS). It
found an old exporter listing every weight as an input, now handled. Running a
model (tract) is not part of this crate yet.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:59:00 -04:00
Exalted5373andClaude Opus 5.5 3c3b527b7f Web: the window wire test allows the CPU/GPU difference
a_window_job_crosses_the_wire_and_renders_the_same_window compared the worker's
CPU render with the engine job's render bit for bit. On a machine with a GPU the
engine job renders there, so the test failed by 1 LSB (on main too); without a
GPU (CI) both run on the CPU. It now checks both sizes and a maximum channel
difference of 3, the bound docs/gpu-pipeline.md documents.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:33:43 -07:00
radialmonster 3d9f8492da LR-IMP-FORMATS: Coolpix NRW black level from the NRW data block
In NRW data version 0104 the u32 at byte 0x20 is the black level in sample units: 200 where the darkest samples of the B700, P950, P1000 and P1100 sit, 0 for the A1000 whose samples reach 0. Used when the maker note has no 0x003d.
2026-10-08 21:26:26 -04:00
radialmonster a12f1697a1 LR-IMP-FORMATS: decode 12-bit raws packed in 32-bit little-endian words (Fujifilm RAF flag 0xf006, Coolpix NRW data block)
Raw-IFD flag 0xf006 = 1 marks RAF samples packed MSB-first into 32-bit little-endian words at any depth; 12-bit files with it (FinePix S1, X-A10, SL1000) were read as an LSB-first stream. Nikon uncompressed 12-bit strips of exactly 1.5 bytes per pixel in a file whose maker note has an NRW data block (COOLPIX B700, P950, P1000, P1100) use the same words in file byte order; without the block (E5700, E8400, E8800) they stay an MSB-first byte stream. Shared unpack_words32_msb and Packing::Words32Msb. RENDER_CACHE_VERSION 15 to 16.
2026-10-08 21:26:26 -04:00
Exalted5373andClaude Opus 5.5 0ac5cf4430 LR-EDIT-COLOR-TEMP: camera-space white balance for raws with a colour matrix
A custom white balance on a raw with a colour matrix (DNGs, and other raws
without a file-local look) re-develops the pixels for the chosen white
through the file's own colour model: lightcraft_raw::color::rebalance
re-evaluates the camera transform at the new white and maps the as-shot
development onto it, so a neutral lit by that white renders neutral and
saturated colours move as the camera's matrices say (DNG spec ch. 6, as
Lightroom does). The decoder passes the colour tags and the white the
pixels were developed for in SourceInfo::camera_color; other photos, and
relative white balances, keep the Bradford adaptation. The eyedropper and
Auto invert the same model (color::neutral_white), so a picked grey card
renders neutral.

The Tint sign is main's (#349): unchanged here. Render cache version 15.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:18:54 -07:00
Exalted5373andClaude Opus 5.5 ce7b8f511e LR-IMP-FORMATS: read DNG BaselineSharpness and keep it in DNG exports
ColorData::baseline_sharpness holds the file's BaselineSharpness (raw IFD
or IFD 0; zero, negative, non-finite or above 16 is ignored) and the DNG
writer writes it back, so a conversion or smart preview keeps it. Merge
outputs leave it unset. Rendering doesn't use it yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:18:54 -07:00
Exalted5373andClaude Opus 5.5 ed59d78e83 LR-MASK-SKY: AI masks use DNG semantic mattes (iPhone ProRAW sky matte)
Read DNG 1.6 semantic masks (SemanticName, SemanticInstanceID,
MaskSubArea; any coding the tile/strip reader decodes, e.g. ProRAW's
lossy-JPEG matte) with lightcraft_raw::semantic_masks, map them onto the
developed image, and carry Apple's sky / portrait / skin / hair / teeth /
glasses mattes in SourceInfo::mattes (SourceInfo is no longer Copy).
Sky, Subject, Background and People components sample them through
orientation, warp (Warp::frame, as geometry.rs) and crop like the image,
so preview = export, instead of the heuristics; files without mattes
render as before. Malformed mask IFDs are skipped, and only the first
MAX_MASKS mask IFDs are decoded, so a file full of them costs bounded
work. The render cache version is bumped once, in the white-balance commit.

The corpus test reads corpus/raw/dng-apple-iphone12pro-proraw.dng, the
name `cargo xtask corpus` downloads it under.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:18:54 -07:00
Exalted5373andClaude Opus 5.5 a28dc4dbdf LR-PROF-CAMERACOLOR: read and keep DNG ProfileGainTableMap (not rendered)
Parse ProfileGainTableMap (DNG 1.6) and ProfileGainTableMap2 (DNG 1.7):
both versions, DNG 1.7 precedence (version 2 in IFD 0, then the raw IFD;
version 1 in the raw IFD, then IFD 0), u8/u16/f16/f32 gains, gamma.
Malformed tags (wrong size, zero or huge grids, non-finite or negative
gains, bad spacing or gamma) are ignored, never allocated for.

Byte order: the specification gives this tag no byte order of its own,
unlike opcode lists, so the payload is read in the file's byte order with
a big-endian fallback; a wrong-order read can't pass the size checks.
The module docs quote the DNG 1.6 "Byte Order" rule.

The map is kept in ColorData, written back by the DNG writer (version 1,
or version 2 when its gamma isn't 1) and dropped from merge outputs, whose
geometry no longer matches the capture's active area. It is not applied
to the render: Lightroom Classic renders Apple ProRAW without it.
Supersedes the parser of #273.

The CC0 iPhone 12 Pro ProRAW from raw.pixls.us joins the corpus.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:18:16 -07:00
Exalted5373andClaude Opus 5.5 1ed80c65bf LR-IMP-FORMATS: decode JPEG XL compressed DNG tiles (DNG 1.7)
Compression 52546 tiles and strips decode through jxl-oxide (new
default-on `jxl` feature of lightcraft-raw): bare codestreams and
ISO-BMFF containers, 1 or 3 planes, 8-16-bit integers scaled back
exactly from the codestream's own bit depth, 16-bit floats. Edge tiles
may be coded at their nominal or in-image size. Headers are checked
against the IFD in header-only probing too. A single-chunk JPEG XL
preview IFD (RGB or grey) is offered as the embedded preview.

The decoder's memory is bounded by the tile: 16 bytes per sample (three
times that for a lossy grey tile, which the decoder expands to three
channels) plus 64 KiB; jxl-oxide 0.12 was measured at <= 10.3 bytes per
sample for lossless and <= 13.5 per colour sample for lossy tiles. A
codestream needing more (e.g. extra channels) is an error.

Lossless tiles are checked bit-exact on synthetic CFA and LinearRaw
files (zune-jpegxl as a test-only encoder); lossy tiles are unverified
on a real file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:18:16 -07:00
BFlat b1883231bd Merge pull request #424 from storytold/claude/integrate-batch-13
Integrate #403, #411, #412, #414, #415, #416
2026-10-09 08:15:07 +07:00
Nathaniel LedfordandClaude Sonnet 5.5 733846b326 Docs: sidebar auto-scroll while dragging
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 8ff45f5237 Sidebar: dragging albums or photos near its top or bottom edge scrolls it
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 5fefe91640 Docs: top-level drop is on the Albums header
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 24b80b4ee3 Albums: the Albums header is the way back to the top level
It replaces the strip under the tree, which could be scrolled out of view
and shifted the rows when it appeared; the header is always visible and
nothing moves.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 922ea25d88 Docs: album tree, drag and hand order (parity, what's new)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 085c25b10b Albums: drop beside a row to place it by hand; Sort A–Z goes back to by name
Dropping on the edge of a row places the album there (album.reorder) with
a line showing where; a folder's middle still means inside. A closed folder
opens only when the drop would go inside it. Sort Contents A–Z and Sort
Albums A–Z (album.sort) appear once a folder is ordered by hand.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 8036f98aca Catalog: tidy a test assertion
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 34a4b61f7e Albums: one-pass children map, before-itself is a no-op, more order tests
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 2c5cb708bb Engine: album.reorder and album.sort
Place an album or folder before a sibling of its kind (also in another
folder), or last of its kind, in one undo step; album.sort goes back to
by name for one folder.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 39f5f114de Catalog: albums can be ordered by hand (format version 3)
Album.order and Op::SetAlbumOrder; album_children lists folders first, then
by place, then by name. Moving an album drops its place and undo restores it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 67aed0b4f4 Albums: a drag ends with the main button, Esc cancels it, hidden rows are no targets
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 63335e2b0a Parity: albums and folders move by dragging onto a folder
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 aaae39fae0 Albums: drag an album or folder onto a folder to move it
A folder under the pointer is outlined when the drop is allowed and opens
after resting there; a drop zone under the tree takes an album back to the
top level. The move is album.move, so it is one undo step.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 5443ae28f9 Albums: address review of folder creation
Show a dialog's error and keep it open instead of closing it, open the
folder menu from the triangle too, draw a triangle only on folders that
hold something, reveal even when the section is folded, reuse the
translated Create … labels, and fix a misplaced doc comment.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:06 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 4abf70e254 Parity: album folders fold like the other trees and create inside themselves
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:06 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 79403a5680 Albums: create albums, smart albums and folders inside a folder from its context menu
The dialog.new* commands take the folder to create in, the dialogs
remember it, and the Albums tree opens the folders down to the new row.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:06 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 96a9c6e04d Albums: folders get the same disclosure triangle as the other sidebar trees
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:06 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 e8b18ae636 Sidebar: one disclosure-triangle helper for the Local, By Date, Folders and Keywords trees
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:06 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 a46c0071a0 Albums: pin that albums are created only inside folders
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:06 -04:00
Claude 12188161f9 Merge remote-tracking branch 'origin/main' into integ-b13 2026-10-09 00:46:05 +00:00
BFlat de79310ae0 Merge pull request #419 from storytold/claude/integrate-batch-12
Integrate #351 (sharp zoomed Detail view, #323) with follow-ups
2026-10-09 07:45:11 +07:00
radialmonster caa102aabd MENU-BAR: pointing at another menu title switches the open menu in the in-window bar 2026-10-08 20:40:46 -04:00
Claude 373efc280e Merge PR #416 2026-10-09 00:20:27 +00:00
Claude c75389dda1 Merge PR #415 2026-10-09 00:20:27 +00:00
Claude 8ee6d63d41 Merge PR #414 2026-10-09 00:20:27 +00:00
Claude f578b27273 Merge PR #412 2026-10-09 00:20:26 +00:00
Claude 43419dcfb8 Merge PR #411 2026-10-09 00:20:26 +00:00
Claude 83ea5e14df Merge PR #403 2026-10-09 00:20:26 +00:00
Claude eab88ea30c Follow-up to #351: drag drafts without a window stay canvas-sized; de/es/ru hint
- With no window (a mask or point-colour overlay, Visualize Spots, soft
  proofing, a window refused for a far read) the loupe renders the whole
  frame at the size it is drawn. While a slider drags that meant the
  photo's every pixel each draft frame at 1:1 and beyond (24 MP per tick
  where 2.5 MP was rendered before #351; mask overlays are on by
  default). A drag now drafts at most the canvas size, as the windowed
  path does; at rest the zoomed frame is rendered as before.
- The Preview size hint in German, Spanish and Russian; the replaced
  'Largest long edge…' text removed from those catalogs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-09 00:16:01 +00:00
Claude 3e88a60dd6 Merge PR #351 2026-10-09 00:14:03 +00:00
ragtuxandClaude Fable 5.1 0cf3ceb438 Part of #260: Help ▸ Open Log Folder shows the log file in the file manager
The desktop app keeps logs/lightcraft.log in its settings folder (#286)
and a failed start names it (#370), but a user still had to find the
folder by hand. The host now hands the UI its log file (Services
.log_file) and Help ▸ Open Log Folder (app.openLogFolder) reveals it
through the existing reveal service: Finder and Explorer select the
file, Linux opens the folder. Off where no log is kept (the web,
--memory, LIGHTCRAFT_NO_PREFS). Translated in de, es, ja, pt-br, ru,
zh-hans and zh-hant; parity row MENU-HELP-LOGFOLDER; What's New and
README note it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 19:13:35 -05:00
BFlat bba4172f83 Merge pull request #413 from storytold/claude/integrate-batch-11
Integrate #407 (file dialogs off the UI thread) and #408 (macOS top-bar drag and zoom)
2026-10-09 07:06:20 +07:00
midasdf 93b900f8d7 M16: keep grapheme clusters intact in vertical watermarks 2026-10-09 09:02:24 +09:00
midasdf 0cb78071de M16: use paintable separators in localized UI text 2026-10-09 08:55:21 +09:00
Claude fec379db46 Russian for two strings the catalog lacked (colour label cleared, slideshow hint)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 23:47:41 +00:00
Claude 4d2f34fb35 Merge PR #408 2026-10-08 23:47:24 +00:00
Claude b306305472 Merge PR #407 2026-10-08 23:47:24 +00:00
BFlat 832b8ada51 Merge pull request #409 from storytold/claude/docs-374-nvidia-cache
Fix #374: troubleshooting for the NVIDIA shader-cache startup hang
2026-10-09 06:46:22 +07:00
Claude bf97105cf7 Fix #374: troubleshooting for the NVIDIA shader-cache startup hang
With Shader Cache Size disabled in NVIDIA Control Panel, the driver walks
the whole system drive inside D3D12CreateDevice (traced by the reporter
to NvMemMapStoragex.dll!TotalDiskUsage), before the window's first
frame; Driver Default fixes it (confirmed in #374, PhotoCraft too). The
launch-argument walk found on the way was fixed in #394.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 23:46:07 +00:00
Nathaniel LedfordandClaude Sonnet 5.5 eaaf859853 Parity: title-bar drag and zoom confirmed on a real macOS window
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:44:23 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 2bd13c1500 Parity: note the stale zoomed-state limitation of the title-bar toggle
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:42:52 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 cc8d617459 Top bar: a press that starts on a button or the search field never drags the window
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:42:52 -04:00
ragtuxandClaude Fable 5.1 d3ceaf0e79 Fix #191: commands' file dialogs run off the UI thread
The desktop showed rfd's synchronous dialogs on the UI thread, so the
window stopped answering the compositor while one was up: GNOME marked
LightCraft "not responding", the File menu stayed drawn and the control
channel went quiet. Services.picker lets the host start a dialog
elsewhere and answer through a channel; a command that needs a path
asks for one and returns, and runs again with the answer when the
dialog closes (pick.rs). Import Photos, Import from Folder, Open
Library, Import Lightroom Catalog, Find Missing Photos, the preset and
curve-preset imports and exports, Auto-Tag from Tracklog and Locate
Missing File use it; hosts without a picker (the web, tests) keep the
synchronous dialogs. ui.inspect lists the waiting commands as
fileDialogs. Same cause as storytold/photocraft#574.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 18:42:19 -05:00
BFlat 32cedb3153 Merge pull request #405 from storytold/claude/integrate-batch-10
Fix #310 (CR2/PEF starting look) and integrate #402 (menus stay below the menu bar)
2026-10-09 06:39:08 +07:00
Nathaniel LedfordandClaude Sonnet 5.5 86d88de2ee Parity: track title-bar drag and double-click zoom
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:37:55 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 d01ebd7953 Top bar: drag to move and double-click to zoom the window (integrated title bar)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:37:05 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 54b037a90a Headless host: record window commands and report the zoomed state back
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:37:05 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 1e27991cf1 Title bar: window-gesture policy (drag, double-click zoom, full-screen exempt)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:35:36 -04:00
Exalted5373andClaude Opus 5.5 7173242ab5 LR-IMP-FORMATS: no overflow deciding whether to scale a full-size JPEG decode
A full-size load asks the decoder for the largest box (`SourceLevel::Full` is
`usize::MAX`, cast to `u32::MAX`). `jpeg::decode` then computed `2 * u32::MAX` to
decide on DCT scaling: an overflow panic in debug builds (caught, so every
full-size JPEG load failed with "decoder panicked"), and a wrap that happened to
give the right answer in release. The comparison saturates now, and the engine
converts its `usize` edge with `u32::try_from` instead of a truncating `as`.

Found decoding camera JPEGs (no JFIF/EXIF header) at full size.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:26:29 -07:00
Exalted5373andClaude Opus 5.5 31aa638383 LR-IMP-PROFILES: red / green / blue curves only with the master curve; neutral test values
Lightroom Classic reads `ToneCurvePV2012Red/Green/Blue` only together with
`ToneCurvePV2012`: a packet with a channel curve and no master curve leaves the
photo unchanged there (rendered on a chart: the channel curve alone has no
effect, with an identity master curve it applies). Imports now do the same.

The preset-packet test helper uses made-up as-shot values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:25:25 -07:00
Exalted5373andClaude Opus 5.5 09d5284e5a LR-EDIT-OPTICS-PROFILE: a preset's LensProfileEnable=0 keeps embedded lens corrections
XMP presets usually say LensProfileEnable=0, which turned optics.lens_profile
off and with it the DNG-embedded WarpRectilinear / FixVignetteRadial
corrections (iPhone ProRAW and other files that carry them). Lightroom
applies a file's built-in corrections whatever that switch says (our
inference), and LightCraft's profile corrections are only those embedded
ones, so crs import now carries over only an enabling switch. The Enable
Lens Corrections checkbox and the amount sliders keep working as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:25:25 -07:00
Exalted5373andClaude Opus 5.5 7e0250e11e LR-IMP-PROFILES: don't report preset bookkeeping fields as unmapped
XMP presets carry fields that describe the preset or the photo rather than
an adjustment (Cluster, SupportsAmount2, RequiresRGBTables, SortName,
Description, CropConstrainToWarp, OverrideLookVignette, AsShotTemperature /
AsShotTint). preset.import listed them as settings that couldn't be carried
over; so did an HDREditMode of 0 and PointColors slots that are all -1.
They are now treated as bookkeeping; an HDR edit, a used Point Color slot and
unknown adjustments are still reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:25:25 -07:00
Exalted5373andClaude Opus 5.5 9087d3ad7f PSD: as_chunks for the palette and 16/32-bit plane loops (clippy 1.98)
clippy 1.98's manual_chunks lint fails `cargo xtask ci` on main at psd.rs:170/191/201
(chunks_exact with a constant size). The fixed-size chunks also drop the per-sample
indexing. No behaviour change; the codecs tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:25:17 -07:00
Claude a1e99fad9e Follow-up to #402: the menu scroll arrows' accessibility labels are translated
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 23:13:39 +00:00
Claude ff9d74f4e5 Merge PR #402 2026-10-08 23:13:27 +00:00
Claude 55ecd900ae Fix #310: CR2 and PEF raws get the starting look fitted to their camera JPEG
CR2 and PEF decode without a camera colour matrix, like ARW, NEF, RW2,
RAF and CR3, but camera_preview::file_local_look (and the catalog's
relative white balance) left them out, so they rendered with the neutral
fallback matrix: the camera preview showed first, then a flat,
desaturated render replaced it. They now use the same guarded fit and
relative WB; a fit that fails the gates keeps the fallback, so a file can
only stay as it was or move closer to its camera JPEG. Not measured on
the corpus here (raw.pixls.us is blocked from this environment).
RENDER_CACHE_VERSION 15 re-renders cached CR2/PEF thumbnails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 23:13:16 +00:00
ragtuxandClaude Fable 5.1 2d47ba3207 Fix #189: in-window menus and submenus stay below the menu bar
egui keeps a popup inside the window, so a menu taller than the room
under the bar (Photo, on a short window) was slid up over the bar: the
titles vanished under it, and the pointer left on the title rested on
the first row and opened its submenu by itself. Every level now gets
the room it has below the bar (a submenu: from its row down, or up to
the bar when egui opens it upward) and scrolls inside it, with wheel,
drag and ▲/▼ strips that scroll while the pointer rests on them, as in
PhotoCraft's menu bar (storytold/photocraft#337). Menu levels and
submenu rows are registered for the control channel and tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 17:50:18 -05:00
BFlat 7b47ba7894 Merge pull request #401 from storytold/claude/integrate-batch-9
Header-only import probes (part of #367), PSD RLE allocation bounds, and #397 (CR3 C-RAW clipped highlights)
2026-10-09 05:48:39 +07:00
Nathaniel LedfordandClaude Sonnet 5.5 826ba91075 #323: docs: measured drag times, the Before/After memory, what stays canvas-sized
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:48:20 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 22b7ba09b9 #323: stage budget: the open photo's views are cut only in a runaway; the shared source is reported
A review measured 1 GB of GPU stages for a Before/After at 1:1 on a 2800 px
canvas against a 400 MB budget, with nothing bounding it. The open photo's window
and before caches are now cut (the before side first) past four times the budget,
the whole-frame view's never; idle caches still go at the budget. The one shared
device copy of a big original is reported (sharedSourceBytes in the renderer's
memory report) and a misleading comment about it is corrected.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:48:19 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 057a8d308e #323: the Before window waits for the original the After is decoding
At first open of Before/After the two window jobs were asked for together, each
decoded its own 24 MP copy of the original (a review saw two decodes and three
uploads for one photo) and the shared device copy was swapped between them. The
Before's window now waits until the After's job has handed the original to the
session (region::defer_before_window, Session media has_source); later windows
of both share the one copy.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:38:39 -04:00
Claude 1b5ba5eba7 Merge PR #397 2026-10-08 22:34:54 +00:00
Nathaniel LedfordandClaude Sonnet 5.5 ddaea580a1 #323: test: holding \ in a wipe is tested with the key held, and the mutation fails it
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:34:02 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 03249f4619 #323: holding \ in a wipe shows the Before window everywhere
The window of the Before view alone was clipped to the wipe's left half while the
picture drawn was the Before on the whole canvas, leaving the right half the
low-resolution stand-in. The clip follows the effective mode.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:32:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 495b1ac9fb #323: tests: the wipe layering and pane clipping are checked on the painted shapes
A review showed the wipe test asserted on a hand-written list of labels (deleting
the stand-in draw calls still passed) and nothing tested the clip at the call
site (replacing it with the canvas still passed). The tests now read the meshes
the frame painted: the Before window comes after the stand-in and stays inside
its side of the line, and side by side each window is painted inside its pane.
Both mutations of the code now fail them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:31:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 f1a513ffe2 #323: docs: drag timings on two canvases, what a fit view decodes, the shared source
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 a5b9dd603f #323: the drag profile covers a 2800 x 1700 canvas
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 a94b647dfd #323: the open photo's views are never trimmed from the stage budget
A review measured the drag at 100 % on a 2800 px canvas taking 87 ms a tick
against 24 ms with a bigger budget: the open photo's caches were over the budget
(a 5 MP window alone is 230 MB) and a "hard" budget cleared them every frame, so
every tick rebuilt them. The whole-frame, zoom window and before caches are what a
drag reuses; only idle caches (hover, compare tiles, second window) are trimmed
now, the least useful first, and the open photo's are counted and reported. The
drag test no longer waits on a tick count.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 f68cd69b1e #323: the windows of a big photo share one device copy of its source
Keeping a 288 MB upload with each view's stages made them thrash against the stage
budget; not keeping it made every new window (a pan crossing a snap, the first
window of the Before side, any control that invalidates the sampled pixels) upload
and convert the source again, about 64 ms against 6-12 ms. A source too big for a
view's stages is now kept once for all views of the photo (shared_source), outside
the views' stages, and given back when the app idles (trim_pool(0)).
source_uploads and shared_source_bytes expose it for tests and diagnostics.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 e9fb4b08e1 #323: the render plan: drags keep the window, modes without windows render full size, limits count
Review findings on region::plan:
- a drag at fit on a 4K or 5K display switched the window off (the whole-frame
  draft shrinks, the preview-source cap does not): the window decision now
  compares against the undrafted whole-frame render;
- overlays, soft proofing and windows refused for reading too far (a spot or Auto
  Mask stroke) left the 1:1 view a canvas-sized upscale: those modes render the
  whole frame at the size it is drawn, as before windows (a refusal is remembered
  per photo, look and frame, and retried when any changes);
- the 3840 and 5120 preview sizes had become no-ops: a limit above the preview
  source level is honoured;
- windows were limited to 8192 px frames: they now reach the photo's own pixels
  (65536 px at most), so a medium-format photo is sharp at 1:1;
- a pinch out from a deep zoom asked for an ever wider window every frame: windows
  already there are kept, magnified, until the gesture has been quiet.
The unused region::needed is gone.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 8a5ecebc18 #323: wipe, pane clipping and identical-looks fixes for the Before/After windows
An adversarial review found that (1) in a wipe the Before window was drawn
before, so under, the low-resolution Before stand-in; (2) the windows of side by
side and stacked views were clipped to the canvas, so their margins spilled into
the neighbouring pane; (3) with identical Before and After looks (a fresh,
unedited photo) both windows had the same job key and the After tile replaced the
Before one in the tile table. Windows are now drawn over the stand-ins and under
the wipe line, clipped to their pane or their side of the line (region::window_clip,
pure and tested), and tiles are keyed by side and job key.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 eb06b65fdb #323: docs: the loupe's render plan, Before/After windows, drag timings
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 92196fddeb #323: tests: the drag keeps its original; a profile of slider drags (ignored)
profile_slider_drag prints the milliseconds from a slider tick to the loupe
showing it at fit, 100 % and 400 % of the demo library's 24 MP photos.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 ea33301e6e #323: a drag keeps the original its superseded drafts decoded; textures convert faster
A drag supersedes every request before it finishes, and the source a superseded
draft had decoded was thrown away with it, so a window cut from a full-size
original decoded it again for every tick. A view slot's superseded draft that is
newer than what the view shows now also hands its source to the session (late
results of other slots stay ignored, as before). Render results become textures
with a plain copy for the opaque case instead of egui's per-pixel premultiplying
conversion, on the UI thread.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 2f7d248d61 #323: a window does not keep the whole source uploaded with its stages
Profiling a slider drag at 100 % and 400 % found each tick of the window render
taking about 60 ms: the 24 MP source (288 MB) uploaded to the device was kept with
the view's stages, counted against the stage budget, and the stages cleared every
frame, so every tick uploaded and resampled again. A source over 96 MB is now
uploaded for the render only; the stages keep the window's own sampled pixels.
A tick costs 7-12 ms instead. equivalence.rs covers it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 35c9b67944 #323: test: no window over a picture that is not the frame (#280)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 6a07c35431 #323: hold the render sizes while a pinch or two-finger scroll runs
A continuous trackpad pinch changes the zoom every frame, and with it the sizes
the loupe plans, so it would have been a stream of re-renders of every size in
between. SizeHold (pure, tested) keeps the plan from before the gesture while it
runs and for a quarter of a second after; the view is drawn from the renders it
has, magnified by the GPU, and planned afresh when the gesture is quiet (a repaint
is asked for at that moment). Another photo resets the hold.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 72bab50c6e #323: Before/After at 1:1 compares like with like
The Before image was a stand-in capped at 2560 px beside an After at full
resolution, so a 1:1 comparison misled. Both sides now get the same treatment:
the same whole-frame size and, once zoomed, a window each (Slot::RegionBefore
beside Slot::Region) cut from the same frame, in side by side, top/bottom, the
\ original view and both wipes (the Before window is clipped to its side of the
line). Each side's windows and tiles are tracked apart, a side is freed when it
is no longer shown, and ui.inspect reports regionBefore.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 136b2851ee #323: engine: region_job_before renders a window of the photo without its edits
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 83d0304ce6 #323: the per-view stage caches are counted against the memory budget
The loupe's, the zoom window's and the before / hover renders' stage caches hold
decoded-image sized buffers that the engine's budget did not count. A quarter of
the memory budget is now theirs: each frame the renderer clears what exceeds it,
the least reusable first (hover, before, compare, second window, the zoom window)
and the open photo's own view last (stage_trim_order, pure and tested). The
inspector's memory report shows the budget and how often caches were trimmed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 3255cda67e #323: the loupe renders the canvas, plus the visible window at no more than 100 %
At 8:1 on a 24 MP photo every drag frame rendered the whole 24 MP frame, and a
Retina fit view (about 2800 px) decoded the original of every photo stepped to.
A new render plan (region::plan, pure and tested) makes the cost follow the
canvas, not the zoom:
- the whole-frame render is canvas-sized and never above the preview source
  level, the preview size limit or the texture side (a drag drafts that);
- once the view is zoomed past it (by more than 25 %) the visible window is
  rendered, cut from a frame that is at most the photo's own pixels, so beyond
  100 % the GPU magnifies it instead of a bigger render being made; this also
  ends the duplicated second render above 100 %;
- a window is drafted during a drag and a draft of an older value still draws;
- a user who capped the preview size keeps the whole-frame render at fit;
- no window where the picture on screen isn't the frame's own aspect (an
  unsupported raw's embedded JPEG cropped differently, #280).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 b7f632f5c0 #323: no loupe texture is larger than the GPU allows
The Automatic preview size let the loupe texture reach 8192 px on every
platform. The browser build draws with egui_glow, which panics on a texture
larger than the GPU takes, and many WebGL devices allow only 2048 or 4096.
Every render size the loupe asks for (loupe, neighbour prefetch, hover and
Before stand-ins, compare tiles, the second window) and every zoom window is now
clamped to the context's max_texture_side (2048 when the host reports none,
512 at the least). Headless drivers can pretend a small GPU for tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 63cfc6ab70 #323: parity, GPU and control docs: sharp zoomed views by window render
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 930d14aacd #323: fix the preview_limit doc comment (largest, not lowest)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 ff9c3fb2ec #323: windows: refuse what cannot fit, grow Auto Mask windows by the dabs near them
Review findings: window growth for spots was capped silently and then rendered
wrongly; Auto Mask growth took the whole stroke's bounding box; hidden masks grew
windows; the wasm window memory cap was not enforced. window_for_reads_checked
returns None when what a window reads would need more than 8192 px (4096 on
wasm) along an axis, Session::region_job returns None then (the loupe keeps the
whole-frame render), Auto Mask growth is per dab near the request (once, not
chained along the stroke) and ignores hidden masks. Docs say what the limits are.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 a1d907bf0f #323: a tight crop at its own pixels reads the source level its pixels need
An adversarial review found that sizing the loupe render from the cropped frame
(previous commit) left the source level chosen from the output edge alone: a
crop whose own long edge is under 2560 px was drawn from the 2560 px preview of
the whole photo, soft though sized right. The level now follows the pixels the
output needs along the source's long edge (output edge / crop fraction); grid
thumbnails keep their cheap level.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 05618b2ad4 #323: windows on the GPU; Auto Mask strokes grow the window; dab references at f32
The GPU renderer renders windows (no more CPU fallback): the vignette gets the
window's offset and the whole output's size (VIG_VIEW), the airlight and noise
reduction scale come from the whole frame, and a window grown for spots or
strokes is cut back after the readback. equivalence.rs renders every settings
case as a window plus a corner vignette and a spot reading from outside, within
the existing 1 LSB. Auto Mask strokes join spots in growing the window (they read
the pixel under each dab). An Auto Mask dab's reference pixel is chosen from its
position at f32 precision, like the GPU kernel: an f64 rounding error that
depended on the window's start could put a boundary dab on either pixel.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 4b83f8bb5e #323: windows cross the Web Worker boundary
WireJob carries the request's window (an old job without one has none), and the
browser build asks for window renders like the desktop; windows there are capped
at 3072 px a side to respect wasm memory.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 eebea486d5 #323: windows: spots render in a window; deep-zoom regression tests for the wide stages
Spot removal reads outside the spot and its automatic source depends on the
pixels at hand. A windowed render now fixes automatic sources from a fixed-size
render of the whole frame and grows the window it works on to hold every spot
that touches the requested one and the patch it reads (bounded at 8192 px),
cutting the result back to the request, so photos with spots get sharp windows
too. The wide stages (dehaze, clarity, highlights/shadows) are checked at depth
on hard-edged blocks: the cut kernels cost at most a few levels, so they are not
computed from a whole-frame proxy.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 dc5bff6947 #323: 100 % zoom is the photo's own pixels as shown (after crop and rotation)
fit_rect measured Percent zoom against the uncropped, unrotated photo, so a
cropped photo at 1:1 showed stretched and a rotated one used the wrong axis. The
loupe, its neighbour prefetch and the compare view now use the frame's
cropped-and-rotated size, which also stops a small crop being rendered above its
own pixels.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 f8f091436a #323: review fixes: no window renders during drags, current-look tiles only, margin by zoom
An adversarial review found that a slider drag at any zoom started a window
render (the 0.6-scale draft looked stretched), that a tile made for an earlier
look kept being drawn and that the tile table was cleared mid-drag, that the
fixed 256 px margin was far below the wide stages' kernels at depth, that spot
removal reads outside the window, and that the window texture stayed resident
after zooming out. Window renders now wait for the drag to end, a tile is drawn
only while photo, zoom and look match, the margin grows with the zoom (768 px
cap) and the window is capped at 6144 px, photos with spots keep the whole-frame
render, the texture and stages are freed when unneeded, and wasm never asks.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 244088498d #323: windows: noise reduction and mask edge refinement follow the whole frame's size
An adversarial review found local::denoise and Refine Edges scaled with the
window's own size, so their strength changed with pan and zoom. Frame::output_long
gives the whole output's long edge to both; tests on a noisy source cover them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 e0539db3f5 #323: the loupe renders the visible window at the zoom scale
When the whole-frame render is smaller than the zoomed view (a 1:1 or 8:1 view
of a large photo, or a capped Preview size) the loupe also requests a window
render (Slot::Region) of the pixels on screen plus a margin, and draws it over
the whole-frame render at its place of the frame. Stateless, grid-snapped
windows mean small pans reuse the render. Not used in the browser build, while
comparing, hovering a look, soft-proofing or showing an overlay. ui.inspect's
loupe reports the window.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 2079e8e08d #323: engine region_job and the window geometry for zoomed views
Session::region_job renders one PixelWindow of the loupe's frame from the source
level that frame size needs, with a key of its own and no cache writes.
ui-egui::region decides when a window is needed (the whole-frame render is
stretched) and which one: the visible pixels plus a margin, snapped to a 256 px
grid so small pans reuse the render, bounded in size, clamped into the frame.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 b358022f14 #323: pipeline: render a window of the output (RenderRequest::window)
A zoomed view needs the pixels it shows at the zoom scale, not a whole-frame
texture stretched over them. RenderRequest gains an optional PixelWindow of the
virtual full output that max_w x max_h describes; only the window is
materialized. It is Frame:🪟 the crop rectangle narrows to the window
(flips as drawn), so sampling, masks, spots, red eye and the lens warp need no
change. The post-crop vignette keeps the whole frame's geometry (Frame::view)
and dehaze uses the whole frame's airlight (estimated on a 384 px render), so a
window matches the same pixels of the full render. The GPU renderer declines
windowed requests (CPU for now).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 ca5c212d03 #323: review fixes: prefetch and stand-ins stay at preview level; render sizes move in 64 px steps
An adversarial review found that with Automatic the neighbour prefetch asked for
the full source level, whose single cache slot then evicted the open photo's
source, and that hover / Before renders and every window-resize pixel produced
full-size renders. Prefetch and stand-ins are now capped at the preview source
level and the loupe size is rounded up to a 64 px step.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 14570cfdf3 #323: Detail view renders at the size it is shown, not at a hidden 2560 px cap
The loupe capped every render at the "Preview size" setting (default 2560 px),
then stretched the texture over the on-screen rect, so 1:1 and Retina fit views
of larger photos looked soft or pixelated.

The setting is now a limit: Automatic (default) renders at the drawn size, up to
the photo's own pixels and an 8192 px ceiling; 1600–5120 px still cap it. The
domain rule is AppSettings::loupe_edge (NaN/zero/huge inputs are clamped). The
saved key changes from previewEdge to previewLimit so existing installs, whose
2560 was the old default, move to Automatic. Build Standard-Sized Previews keeps
a numeric edge via standard_preview_edge().

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Claude 8d79bf082e Batch fix: the header test's RLE PSD fixture uses PackBits runs of at most 128 bytes
The fixture wrote each row as one literal run of w bytes, which PackBits
can't express past 128 (a 160-pixel row's header byte read as a repeat
run). The old decoder clipped such rows silently; with the PSD fix in
this batch an overrun is an error, as it should be, and the fixture now
splits rows into valid runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 22:23:51 +00:00
Claude d316d97d80 Merge branch 'fix-psd-rle-alloc' into integ-b9
# Conflicts:
#	crates/codecs/src/psd.rs
2026-10-08 22:21:56 +00:00
Claude 2ab6e973c6 Merge branch 'fix-367-header-probe' into integ-b9 2026-10-08 22:21:37 +00:00
BFlat c395789026 Merge pull request #398 from storytold/claude/integrate-batch-8
Integrate #391 and #393 (undecodable raws import as preview only), with a work budget for the JPEG scan
2026-10-09 05:21:21 +07:00
Claude 6aa599f869 Fix: PSD RLE decode bounds its allocations by the input
A tiny hostile PSD could abort the process on out-of-memory in
`psd::decode` (allocation failure doesn't unwind, so catch_unwind
can't save it):

1. The RLE row byte-count table (channels x rows entries, from the
   header) was reserved with Vec::with_capacity before checking that
   the file holds it: 56 channels x 2^30 rows asked for ~480 GB from a
   ~40-byte file. The table is now taken as a slice of the input with
   checked size math and read lazily, so nothing is allocated from it.

2. RLE rows were zero-padded / run-expanded to the declared width with
   no tie to the input size, so all-zero row counts on a 1 Gpx (the
   DecodeOptions::max_pixels limit, already enforced by check_size)
   32-bit image expanded to 4 GiB per channel. A PackBits row expands
   at most 64x (2-byte repeat run -> 128 bytes), so each decoded row
   must now declare at least ceil(row_bytes / 128) * 2 encoded bytes,
   and the decoded channels' data must be present in the file, before
   any plane is allocated. Decoded planes are thus bounded by 64x the
   file size as well as by max_pixels. PackBits now decodes into a
   pre-sized, zeroed row and a run that would write past the declared
   width is an error instead of being silently clipped.

Also: size math (width x height x bpc, samples) is checked; image
buffers are allocated with try_reserve_exact (TooLarge on failure);
only the colour (+ alpha) planes are decoded, extra channels are
skipped; the PSB layer-section length no longer truncates on 32-bit
targets; interleaving uses iterators instead of indexing.

Valid PSDs (raw and RLE, 8/16/32-bit) decode identically; regression
tests cover a huge RLE header with a truncated table (aborted with a
481 GB allocation before), all-empty RLE rows, counts larger than the
file, a repeat run past the row width (was Ok before) and a 16-bit
gray RLE file with an extra channel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 22:19:42 +00:00
Claude f6f5e6f1b7 Part of #367: probes read image headers instead of decoding the pixels
The import probe of a non-raw image decoded it (at a 64 px fit, which for
JPEG still entropy-decodes every block and for PNG inflates everything)
only to learn its stored size and orientation: ~95 % of import CPU.

lightcraft-codecs gets `read_header(bytes) -> Header { format, width,
height, orientation }`: JPEG from its markers (EXIF orientation), PNG from
its chunks (IHDR, eXIf anywhere), TIFF from the first IFD (tiff crate,
no strips read), WebP from its RIFF chunks / bitstream header (image-webp),
PSD from its header and image resources. GIF, BMP and JPEG XL keep the
small decode. `files::probe_bytes` uses it; raw handling is unchanged.
The TIFF colour-layout check is shared with the decoder (`layout`).

Measured (release, 4 cores, throwaway benchmark, per file, bytes in
memory): 200 generated 4000x3000 JPEGs (3.6 MB): 72.7 ms -> 2.1 ms
(read_header alone 1.05 ms; the rest is the content hash and metadata);
20 PNGs (30.8 MB): 236-265 ms -> 9.0-9.6 ms.

Corrupt files (behaviour note): the header readers also refuse what a
decode refuses before the pixels (unsupported layout, compression, frame
type, bad IHDR, ...) and truncated files: a JPEG whose markers don't reach
its EOI, a PNG cut before the chunk after its IDAT run, TIFF strips/tiles
that start past the end, a WebP image chunk or PSD merged data past the
end. So truncated files (interrupted copies, partial downloads) are still
marked `error` in the import review, as before; tests check that whenever
the old decode accepted a (cut or byte-mutated) file the header agrees.
Changed: damage *inside* compressed data that is all present (flipped or
garbled bytes) was refused at scan and is now imported, and shows as
unreadable ("!" in the grid) when rendered. Also: tiny JPEGs (< 128 px)
were decoded leniently by the old probe, so a truncated one imported; it
is now refused. A TIFF truncated inside its last strip is not detected
(StripByteCounts are often overstated, so only strip starts are checked).

Tests: header readers vs decode on generated JPEG/PNG/TIFF/WebP/PSD/BMP
with and without EXIF/TIFF orientation; truncations and random mutations
agree with decode; hostile headers (zero/absurd sizes, bad lengths,
truncation, garbage) are errors; read_header added to the fuzz tests;
probe_bytes oriented dims match the decode-based values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 22:09:30 +00:00
radialmonster 7094ba1b6e LR-IMP-FORMATS: CR3 C-RAW samples that overshoot the bit depth are clamped
C-RAW is quantised wavelet coding: next to clipped highlights its
reconstruction overshoots the sensor range by a few quantisation steps.
On 17 CC0 CR3 files (SX70 HS, G5 X Mark II, M6 Mark II, EOS R, RP, 90D,
M200, R5 Mark II, R6 Mark II, R6 Mark III, R8) samples reached 16384 to
16419 at 14 bits, never below 0, and decoding failed with "decoded
sample outside its bit depth". Lossless files of the same bodies clip at
exactly 16383. Wavelet samples are now clamped to the CMP1 bit depth;
lossless samples outside it are still corrupt data.

The other 93 CR3 files have no sample outside the range and render
pixel-identically. Render cache version 13 -> 14.
2026-10-08 18:04:01 -04:00
Claude cb0e0cb2e5 Follow-up to #393: the embedded-JPEG scan shares a work budget
scan_for_jpeg walks every JPEG-like start in a container it can't parse;
each walk runs through the marker segments that follow (up to 4096) and
their entropy data. A crafted CRW/MRW/X3F (or unsupported TIFF raw) of
fake starts one after another makes every walk cover most of the file:
quadratic, about a minute for 40 MB, on the import's probe thread. The
walks now report how far they read and share a budget of eight times the
file's size (at least 1 MiB); real files' candidates fail within a few
bytes or are the preview itself. Test: 200,000 fake starts, 0.15 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 22:03:26 +00:00
Claude 0fb3351c91 Merge PR #393 2026-10-08 22:00:34 +00:00
Claude b9254242a2 Merge PR #391 2026-10-08 22:00:10 +00:00
BFlat ab9e8d372c Merge pull request #394 from storytold/claude/integrate-batch-7
Integrate part of #374 (bounded folder walks, launch imports off the UI thread) and #385, #386, #387
2026-10-09 04:43:24 +07:00
Claude b11e5399c6 Merge PR #387 2026-10-08 21:26:34 +00:00
Claude 83058f717e Merge PR #386 2026-10-08 21:26:34 +00:00
Claude 7962b66a06 Merge PR #385 2026-10-08 21:26:33 +00:00
Claude c140f63372 Merge branch 'fix-374-startup-drive-walk' into integ-b7 2026-10-08 21:19:42 +00:00
Claude 790a341776 Part of #374: launch imports run off the UI thread; folder walks are bounded and never take a drive root or home implicitly
Root cause (in 0.2.1 and still on main): files and folders named on the
command line were imported from the eframe app-creator closure with
`app.run("library.import", …)`, which runs the engine command synchronously:
`import::expand` walked the folder recursively (one `is_dir()` open per entry,
no depth or count bound, following links) and probed every file before the
first frame could be drawn. An argument that resolves to a drive root (`.` or
`..` with a launcher's working folder at `C:\`, `/` or `\` on Windows) or to
the home folder therefore walked the whole drive with the window stuck as a
black "not responding" box. The library-problem banner's pending import had
the same synchronous call. Every other recursive walk (Local's Include
Subfolders, Find Missing Photos, preset and LUT folders, MCP) was also
unbounded, which is the "Local folders with subfolders: not responding"
pattern of filmcraft#23, and a link loop recursed until the stack overflowed.

- engine `walk::files_in`: one bounded, iterative walker (32 folder levels,
  1,000,000 entries; `Walked::truncated` says it stopped), folder or file
  taken from the directory listing (no open per entry), links and junctions
  to a folder followed once. `import::expand` (+ `expand_within`), preset and
  LUT folder expansion and MCP `expand_paths` use it.
- engine `import::launch_path`: an empty launch argument is ignored; a
  relative one is resolved against the working folder; a drive or file-system
  root becomes its memory card's DCIM folder or is refused, and so is the home
  folder (a notice says why and to use File > Import Photos).
- desktop: launch arguments go through it and are imported with
  `import::start_paths` (the import worker), like dropped files; so does the
  library-problem banner's pending import.

Not verified on Windows (no machine here). The report says PhotoCraft 0.5.0
and `--memory` hang the same way with no arguments; nothing in LightCraft
walks the disk at startup without arguments (strace of a headless demo run
with HOME=/ and cwd /: no recursive listing), so that part may be outside
this code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 21:19:42 +00:00
BFlat 67b7bc8bc6 Merge pull request #389 from storytold/claude/integrate-batch-6
Integrate #373 (keymap editor), #218 (Lightroom catalog import), #367 (parallel import copies), #344 (import file types)
2026-10-09 04:19:01 +07:00
Claude 41e67e1edd Batch fix: the CLI snapshot test settles before its screenshots
Under load (the full workspace suite) the photo grid was captured before
its thumbnails rendered, darker than the export dialog dimmed over them,
and the dimming comparison failed; it passes alone. Each screenshot now
follows ui.settle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:56:35 +00:00
Claude 5d437489b9 Merge branch 'fix-344-import-file-types' into integ-b6 2026-10-08 20:53:11 +00:00
Claude 1e6f0e28e1 Batch fix: the Lightroom catalog dialog filter goes through add_filter_nocase
#218's catalog picker called rfd's add_filter, which #362's guard test
(no_dialog_bypasses_the_helper) forbids: on Linux the filter would hide
CATALOG.LRCAT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:53:11 +00:00
Claude 4af709995f Fix #344: the import review checks or unchecks a file type at once
With several file types among the candidates, the Import Photos review
shows a toggle per type (ARW · 120, JPG · 120): unchecking JPG leaves
the JPEGs kept beside the raws out of the import, checking a type checks
its importable files (duplicates stay unchecked). JPEG/JPG and TIFF/TIF
are one type; a file whose probe named no format goes by its extension.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:52:35 +00:00
Claude 9b0d0fd05f Parity summary regenerated (cargo xtask parity --write)
Main's summary block had a duplicated pair of total lines left from an
earlier merge; the regenerated one counts #373 and #218's rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:50:04 +00:00
Claude b1d8ba4284 Merge branch 'fix-367-import-parallel' into integ-b6 2026-10-08 20:48:31 +00:00
Claude d8efa6a147 Spanish for the keymap editor (#373) and the Lightroom catalog import (#218)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:48:31 +00:00
Claude d4c0a7a3d4 Merge remote-tracking branch 'origin/main' into integ-b6
# Conflicts:
#	crates/ui-egui/locales/de.json
#	crates/ui-egui/locales/ja.json
#	crates/ui-egui/locales/pt-br.json
#	crates/ui-egui/locales/ru.json
#	crates/ui-egui/locales/zh-hans.json
#	crates/ui-egui/locales/zh-hant.json
2026-10-08 20:48:15 +00:00
BFlat 68e0827633 Merge pull request #383 from storytold/claude/test-376-masking-layout
Part of #372: painted-text layout tests from #376
2026-10-09 03:47:22 +07:00
ClaudeandElliot Graham cad024fbff Part of #372: painted-text layout tests from #376
The tests from @QuirkyTurtle94's #376, a duplicate fix for #372 opened
while #379 was in review: they read the painted text shapes (not the
registered rects) at 1x and 2x scale, in English, German and Brazilian
Portuguese, at panel widths 250-520, and check that tile labels fit
unelided inside their tiles, that a long mask name is elided before the
eye, and that a long Describe prompt keeps Select inside the panel.
They pass unchanged against main's implementation.

Co-Authored-By: Elliot Graham <elliotgraham94@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:47:07 +00:00
Claude 86deb5b004 Fix #367: import copies and verifies files on several threads
Import -> Copy (and Move, and Copy as DNG) handled one file after another
on the import worker: copy, fsync, read back and hash, optionally decode
and re-encode the raw as DNG. Every other CPU-heavy step of an import was
already parallel (the probes, up to 8 threads), so with the probes cached
by the review dialog's scan the transfer phase ran on one core.

ImportJob::prepare_files now plans each file in order, as before
(duplicates by path and content, sidecars, rename sequence numbers,
folders, Kept notes) and collects the copies/moves, which then run on up
to 8 threads (std::thread::scope; sequential on wasm32). Outcomes keep
their places, so results, order, names and failure messages are those of
a one-by-one import:
- a file whose content matches a transfer still pending, or whose name
  could collide with one (same folder, same name family: IMG.jpg,
  IMG-1.jpg, img.dng), waits until that one has finished;
- cancel stops before the next file; transfers not started leave no item;
- a panic in one transfer fails that file instead of the batch;
- Copy as DNG holds a memory-gate permit (6x the file size) while it runs.
Copy verification and crash safety are unchanged (same copy_new / place).
The desktop's import worker prepares batches of 2 x workers files (8..16,
lightcraft_engine::import::batch_size) instead of 8, so a batch's slowest
file holds back fewer threads; the browser build keeps 8.

Measured (4-core shared VM, 200 procedural 3.7 MB JPEGs = 754 MB, warm
page cache, throwaway benchmark not committed):
- probe: ~80 ms CPU per file is the JPEG decode (DCT-scaled to 64 px),
  1.2 ms the content hash; the dialog's scan: 4.4 s, 3.6 cores busy.
- Copy, probes cached: copy+fsync+verify 1.59 s at 0.43 cores before,
  0.94 s at 0.81 cores after. Copy, not cached: 6.45 s -> 5.51 s.
- Add, probes cached: 2 ms prepare; nothing to parallelize.
- commit of a batch on the UI thread (journal append + fsync): ~4 ms,
  flat for libraries of 0, 5k, 20k and 50k photos (~0.7 s for 1300).
Copy as DNG could not be measured (no raw corpus here); it is the most
CPU-heavy per-file step and was sequential.

Test: copies_run_side_by_side_with_the_same_outcome (copies under the
test fault Slow run on more than one thread; names, order and in-batch
duplicates match a one-by-one copy; the outcome assertions also pass on
the old code, the thread assertion fails there).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:47:04 +00:00
BFlat a673b2bb7a Merge pull request #382 from storytold/claude/i18n-371-spanish
Fix #371: Spanish interface
2026-10-09 03:41:48 +07:00
Claude 305ff71474 Part of #371: Spanish for the SAM 3 install dialog (#366)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
(cherry picked from commit 73b22c8524ef17b209ff2fffbb16c248524e2e96)
2026-10-08 20:41:31 +00:00
Claude a0e49629d3 Merge remote-tracking branch 'origin/main' into i18n-371-spanish 2026-10-08 20:41:31 +00:00
BFlat c8f697a128 Merge pull request #381 from storytold/claude/fix-366-sam3-install-help
Fix #366: the SAM 3 dialog without a download location shows how to install by hand
2026-10-09 03:41:25 +07:00
BFlat 232cbace63 Merge pull request #379 from storytold/claude/fix-372-mask-labels
Fix #372: masking tile labels and long mask names stay inside their boxes
2026-10-09 03:41:22 +07:00
radialmonster 1fa3af8740 LRC-LIB-LIBFOLDERS: the sidebar rename test builds its folder ids with forward slashes on Windows
The folder tree writes paths with forward slashes and its widget ids
carry them; the test put the temp folder in the id with backslashes.
Found running cargo xtask ci on Windows.
2026-10-08 16:41:12 -04:00
radialmonster 8e4d7545ff M11.4: the X-T20 corpus RAF is fitted since the away-from-edges retry; update the test and the doc that said it was rejected
Found running cargo xtask ci with the corpus on Windows. The two changes
landed on the same day: the Fujifilm test (#282) expected the compressed
X-T20 file to keep the neutral fallback because its fit failed the gates
on all pixels (held-out RMS 0.114 against 0.10). The retry away from
edges (#263, issue #232) fits it (0.056), so the fit is accepted at both
sizes. Expect true.
2026-10-08 16:41:12 -04:00
radialmonster 1028ed6b88 LRC-LIB-LIBFOLDERS: a chosen folder that follows a move keeps its names' case and the platform's separators
follow_library_folder rebuilt the chosen subfolder from the lower-cased
folder_key on Windows and joined it with '/', so after a move the folder
shown in the sidebar came back as c:\archive\trip/day1 with its names in
lower case. Add folder_rest (compares by folder_key, returns the names as
spelled) and join them with the platform's separator. Tests compare
folders by identity where they only mean the same folder, and a new test
fails on Windows without the fix.
2026-10-08 16:41:11 -04:00
Claude efe7cad73c Follow-up to #218: Lightroom catalog import translated
The progress, result and menu strings of the Lightroom catalog import in
the ja, pt-br, zh-hans, zh-hant, de and ru catalogs. The formats catalogs
held English copies of the seven new format messages; they are
translated, and German and Russian (which the PR predates) get them too,
so the formats build check passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:36:22 +00:00
Claude a86defd1b9 Merge PR #218 2026-10-08 20:35:39 +00:00
radialmonster 881bc85dac LR-IMP-FORMATS: import CRW, MRW, X3F, KDC, MOS, ERF and IIQ files from folders as preview only
Folder import, drag and drop and MCP folder expansion skipped these extensions, so
the preview-only fallback for these containers was only reachable by naming a file.
Adds exactly the containers that now import as preview only to import::EXTENSIONS and
the headless PHOTO_EXTENSIONS. 3FR, FFF, DCR and STI still fail and stay out.
2026-10-08 16:32:17 -04:00
radialmonster dbe73f0f9c LR-IMP-FORMATS: raws we cannot decode import as preview only when any usable JPEG is embedded
Widens the preview-only fallback. A recognised raw whose decode fails now falls back
to its embedded JPEG for any decode error (unsupported, corrupt, limit, TIFF
structure), not only an unsupported variant, and the preview search covers more
containers:
- Canon CRW, Minolta MRW and Sigma X3F are recognised by their magic (RawFormat
  Crw, Mrw, X3f) instead of failing as "RawOther files are not supported yet".
- A TIFF whose IFD0 has no ImageWidth/ImageLength but a JPEG pointer or SubIFDs
  (Kodak KDC) is a raw container, not a malformed image. TIFF compression 99
  (Leaf MOS) counts as a raw compression.
- When no tag points to a preview, a recognised raw we cannot decode is scanned
  for the largest complete three-component DCT JPEG. A preview whose first byte
  was overwritten (Minolta, Epson) is restored. A one-component JPEG is skipped
  (a PowerShot CRW stores its mosaic that way).
Files with no JPEG at all still fail with "without an embedded preview".
2026-10-08 16:32:17 -04:00
radialmonster 0b8c7a237a LR-IMP-FORMATS: TIFF raws with a thumbnail-only IFD0 import as preview only, not as a tiny image
Phase One / Leaf IIQ, Canon EOS-1D / 1Ds and Kodak DCS TIFFs keep their raw data in
a private block. IFD0 is only a 160-640 px RGB copy and no IFD is marked as raw, so
they opened as an ordinary TIFF of that size (a 150 MP file rendered 296x220) with
no preview-only label.

probe() now recognises such a file from its structure (OtherTiff): a picture at
least 4x larger than IFD0 in both directions is declared by a SubIFDs child, or by
Exif PixelXDimension/PixelYDimension together with enough bytes outside every IFD's
strips and tiles to hold it at one bit per pixel. decode() reports the sizes as the
reason. A raw we cannot decode now falls back to its embedded JPEG, else to the TIFF
reader's view of its first image, so it imports preview-only with that reason.
RENDER_CACHE_VERSION 12: these files render differently.
2026-10-08 16:31:49 -04:00
Claude 43aa795539 Follow-up to #373: translated keymap editor, contextual keys noted
- The keymap editor's strings, the fixed-key labels and the two new
  command labels in the ja, pt-br, zh-hans, zh-hant, de and ru catalogs;
  'Restore {}' and 'Removed from {}' go through tr_format!.
- The Presets and Soft Proofing rows say what their key does in the
  Library grids (Flag as Pick and advance / expand or collapse the
  stack), which main's static Shortcuts list showed as a separate label.
- The macOS menu publishes no shortcuts while a key is being recorded
  (merged with main's owned_by_menu).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:24:13 +00:00
Claude 38cdd49274 Merge PR #373 2026-10-08 20:23:12 +00:00
Claude 0b587a0a8a Fix #371: Spanish interface
A Spanish (es) catalog and formats catalog covering the same messages as
the Brazilian Portuguese, Japanese and Simplified Chinese catalogs (1595
messages, 180 formats), an Edit > Language > Español command
(app.language.spanish), es tag tests (es, es-ES, es_MX.UTF-8, es-419),
docs/localization-es.md with the terminology used, and the language
lists in docs/localization.md, parity and What's New.

Instructions use infinitives or impersonal forms (no usted). Plural
endings follow the existing convention: the English "s" passes through
where Spanish adds one, else {:.0} swallows it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:20:17 +00:00
Claude 60a98354d9 Fix #366: the SAM 3 dialog without a download location shows how to install by hand
With no mirror configured the dialog was titled "Download the SAM 3
Model?", only said to put "the files" in the folder (see
docs/ai-masks.md) and offered Close. It is now titled "Install the SAM 3
Model", names the three files (model.safetensors, vocab.json,
merges.txt from facebook/sam3), and has a Show in Finder / Explorer /
File Manager button that creates and shows the model folder, plus a link
to the installation guide. Object and Describe work as soon as the files
are there.

The download location itself (a hosted mirror) is still a maintainer
decision: DEFAULT_MIRRORS stays empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:18:47 +00:00
Claude f16f413942 Fix #372: masking tile labels and long mask names stay inside their boxes
The Create New Mask tiles painted their (untranslated) labels with an
unconstrained Painter::text into tiles capped at 52 pt, so Background and
Luminance ran past their tiles, and translations would more so. The grid
now measures the translated labels: four 52 pt tiles when they fit (at
10.5 pt, or 9.5 pt before dropping a column), else three or two, a tile
growing past 52 pt for a long label; only in a panel too narrow for even
that is a label cut short (full text on hover).

A mask row's name is cut short with an ellipsis before the eye (full name
on hover). The Describe field places the translated Select button first
and gives the field what is left, instead of reserving a fixed 64 pt.
Status lines wrap explicitly.

The painted label and name rects are registered (maskNewLabel:*,
maskName:*), so the sidebar test checks the text, not just the widgets,
at four panel widths with a long name and a long prompt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:09:16 +00:00
BFlat 48cd9c5e3a Merge pull request #375 from storytold/claude/integrate-batch-4
Integrate #279 (Canon CR3 decoding) and #362 (file dialog filters)
2026-10-09 03:08:11 +07:00
parkerHurstandClaude Opus 5.5 e2b626b7f7 LR-PREF-SHORTCUTS: editable keyboard shortcuts in Help ▸ Keyboard Shortcuts (⌘/)
The ⌘/ dialog is now a keymap editor: click a shortcut and press the new
keys (Esc cancels), × removes it, ↺ restores the default, Reset All clears
every change. A key another command had moves to the new command, with a
toast. Changes go through the new app.setShortcut / app.resetShortcuts
commands into AppSettings.keymap (saved in ui.json) and show up in the
menus, the native macOS menu and the control channel's command list.

Modifier keys alone never record (holding ⌘ waits for the key it
modifies), ⌘C/⌘X/⌘V are recorded from the copy/cut/paste events, and
saved entries that don't parse fall back to the default shortcut.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:54:50 -05:00
Claude 28db7b200b Merge fixes for #279: CR3 joins RAF in the format lists, render cache version 13
Main already had Fujifilm RAF in relative WB, the file-local look, the
docs and the corpus lists; CR3 is added next to it. RENDER_CACHE_VERSION
was already 12 on main (from #255/#258/#282), so CR3 thumbnails rendered
by main from embedded previews would not be redone: 13.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 19:44:32 +00:00
Claude 3f63db0a0b Merge PR #279
# Conflicts:
#	README.md
#	ROADMAP.md
#	crates/catalog/src/model.rs
#	crates/engine/src/camera_preview.rs
#	crates/raw/src/lib.rs
#	crates/raw/tests/corpus.rs
#	docs/camera-preview-colour.md
#	docs/parity.md
#	docs/whats-new.md
2026-10-08 19:44:21 +00:00
Claude f611e76d0e Merge PR #362
# Conflicts:
#	apps/lightcraft/src/main.rs
#	docs/parity.md
2026-10-08 19:43:31 +00:00
BFlat eeb136b9ee Merge pull request #370 from storytold/claude/fix-260-startup-failure-dialog
Part of #260: a desktop start that fails says why, in a message box naming the log
2026-10-09 02:43:05 +07:00
BFlat 3f10ba4f3b Merge pull request #369 from storytold/claude/fix-171-unlocked-library-warning
Fix #171: a library that can't be locked opens with a warning the user sees
2026-10-09 02:43:01 +07:00
Claude 7fb24029f2 Part of #260: a desktop start that fails says why, in a message box naming the log
When eframe could not open the window (no usable graphics device, a
broken driver), main returned the error to standard error, which a
Windows release build (no console) or an app started from Finder never
shows: the app just vanished. The error now goes to the log (the file
#286 added) and into a native message box, in the user's language, that
names the log file. The saved UI language is applied before the window
starts so the message uses it. Scripts and tests (LIGHTCRAFT_NO_PREFS)
get no dialog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 19:30:26 +00:00
Claude 5601d7879a Fix #171: a library that can't be locked opens with a warning the user sees
When catalog.lock can't be locked (a file system without locks, such as
some network shares), the library opens unprotected, as documented, but
the promised warning went only to the log. It is now the first of the
library's warnings: library.info lists it under settingsWarnings, the
app shows it once in its notices, and lightcraft-cli run / mcp print the
library's warnings on stderr (stdout stays results / protocol only). A
normally locked library shows none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 19:27:32 +00:00
Adrian D'souza 095c7dce87 fix sqlite test payload extension 2026-10-09 00:57:19 +05:30
Иван Лапочкин f96f732baa M11.1: add CR3 decoding
Decode lossless CRX and version 0x100/0x200 C-RAW in portable Rust,
including sensor geometry, timed white balance and black/white levels.
Verify all sensor samples against six independent M50/R100/R8 references.

Any CR3 decode error falls back to the embedded JPEG, as CR3 files did
before this decoder. The full sensor buffer is allocated only after the
first plane decodes, and plane buffers grow row by row. Tile planes
decode in parallel with a windowed bit reader (24 MP: 424 -> 89 ms
lossless, 519 -> 138 ms C-RAW on an M4).

Add guarded CR3 camera-JPEG colour fitting with independent framing
validation, relative white balance and cache invalidation.

Validation: cargo xtask ci.
2026-10-08 22:22:13 +03:00
BFlat bf224c2a8d Merge pull request #361 from storytold/claude/fix-319-craft-fonts-pin
Fix #319: release builds pin craft-fonts with Noto Sans CJK SC (includes #203)
2026-10-09 02:19:59 +07:00
BFlat 16236ff779 Merge pull request #364 from storytold/claude/fix-293-undo-shows-photo
Fix #293: undo and redo show the photo they change
2026-10-09 02:19:56 +07:00
BFlat 2592ab47e9 Merge pull request #365 from storytold/claude/fix-sam-busy-after-reply
AI masks: the worker stops counting a job as pending before it replies
2026-10-09 02:19:49 +07:00
Claude 3e791905b9 Merge remote-tracking branch 'origin/main' into claude/fix-293-undo-shows-photo
# Conflicts:
#	crates/engine/src/tests.rs
2026-10-08 19:19:32 +00:00
Claude 2897f85861 Merge remote-tracking branch 'origin/main' into claude/fix-319-craft-fonts-pin 2026-10-08 19:19:27 +00:00
Claude 3517606ffc Merge remote-tracking branch 'origin/main' into claude/fix-sam-busy-after-reply 2026-10-08 19:19:27 +00:00
BFlat 9e51ca0da8 Merge pull request #354 from storytold/claude/fix-341-date-keyword-rows
Fix #341: By Date and Keywords rows show their photos from All Photos
2026-10-09 02:19:21 +07:00
Claude 85e38c7342 Merge remote-tracking branch 'origin/main' into claude/fix-341-date-keyword-rows
# Conflicts:
#	crates/ui-egui/src/tests_panels.rs
2026-10-08 19:19:15 +00:00
BFlat 4836143f9e Merge pull request #353 from storytold/claude/fix-316-section-eye
Fix #316: the section eye switches Light, Color and Detail off, and can be clicked
2026-10-09 02:19:02 +07:00
Claude 57dc811a6a Merge remote-tracking branch 'origin/main' into claude/fix-316-section-eye
# Conflicts:
#	crates/ui-egui/src/headless.rs
2026-10-08 19:18:55 +00:00
BFlat 9e327b4772 Merge pull request #352 from storytold/claude/fix-322-type-slider-values
Fix #322: type an exact value into a slider
2026-10-09 02:18:43 +07:00
Claude f6fcef4d5c Merge remote-tracking branch 'origin/main' into claude/fix-322-type-slider-values 2026-10-08 19:18:31 +00:00
BFlat 3895cf6411 Merge pull request #363: community PRs, batch 3 (German + Russian UI, Fujifilm compressed RAF, raw fixes)
Integrate community PRs, batch 3: German + Russian UI, Fujifilm compressed RAF, raw fixes (stacked on #350)
2026-10-09 02:18:21 +07:00
BFlat 5cb74e7313 Merge pull request #350: community PRs, batch 2 (14 PRs + follow-ups)
Integrate community PRs, batch 2 (14 PRs + follow-ups; stacked on #349)
2026-10-09 02:18:09 +07:00
BFlat 4165904bd1 Merge pull request #349: community fixes, batch 1 (18 PRs + follow-ups)
Integrate community fixes, batch 1 (18 PRs + follow-ups)
2026-10-09 02:17:58 +07:00
Claude 6ea9b979c8 AI masks: the worker stops counting a job as pending before it replies
The pending counter was decremented when the job's guard dropped at the
end of the loop iteration, after the reply was sent, so a waiting caller
(CLI / MCP mask.add) could see the segmenter still busy right after its
answer arrived. Under a loaded test run that made
tests_segment::waiting_requests_with_a_damaged_model_are_errors fail on
assert!(!busy()). The guard is now dropped just before each reply.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 19:02:44 +00:00
Claude e73d88a55f Merge PR #358
# Conflicts:
#	docs/parity.md
2026-10-08 18:57:38 +00:00
Claude d56d265044 Merge PR #282
# Conflicts:
#	ROADMAP.md
#	assets/ATTRIBUTION.md
#	crates/engine/src/camera_preview.rs
#	crates/raw/tests/corpus.rs
#	docs/camera-preview-colour.md
#	docs/parity.md
2026-10-08 18:55:07 +00:00
Adrian D'souza b4a1166f4b PR218: document optional-data and recovery warnings 2026-10-09 00:17:24 +05:30
Adrian D'souzaandCodex 5413c8bb3a PR218: make archive scale test exercise staging
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:13:49 +05:30
Claude c07395222f Merge PR #356
# Conflicts:
#	docs/parity.md
2026-10-08 18:43:38 +00:00
Claude 7b082660e7 Merge PR #339 2026-10-08 18:43:35 +00:00
Claude 3a9b6ccee4 Fix #293: undo and redo show the photo they change
After an undo or redo that changed one photo (its edit, rating, flag,
label, metadata or versions), that photo becomes the active one, as in
Lightroom Classic, so the change is on screen instead of happening to a
photo out of sight. It joins the selection if it isn't in it; steps that
touch several photos, albums or files, and photos outside the current
view, leave the selection alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:43:18 +00:00
Adrian D'souzaandCodex 07d9678209 PR218: cover optional payload cap warning
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:12:39 +05:30
Claude 8980c3bb0e Fix #341: clippy (no single-element loop in the test)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:42:11 +00:00
Adrian D'souzaandCodex 3c76902bbb PR218: cover unsupported schema entries
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:12:06 +05:30
Claude 1ebce934ea Fix #322: clippy (collapsible if)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:42:00 +00:00
Adrian D'souzaandCodex 7cf3719c63 PR218: localize complete import messages & scale prepared path
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:11:29 +05:30
Claude c3dce49c2e Format the gap-report test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:41:28 +00:00
Claude 3af89e6ed5 i18n: translate what #265 made translatable, and the formats gaps #264/#265 left
#265 wrapped about 460 more UI strings (command and control labels, rule
operators, built-in preset names, status messages) in tr()/tr_format!, and
filled the 54 new formats messages of the other catalogs with English
copies. These are now translated in ja, zh-hans, zh-hant and pt-br; the
Russian formats catalog gets the 58 messages it needs to build, and German
the strings batches 1-2 added. Release-note lines stay English outside
German.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:40:42 +00:00
Claude a7fcd81850 Fix #341: What's new
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:40:19 +00:00
Claude c3c30dae3f Fix #341: the test clicks the row's widget id
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:39:36 +00:00
Claude 47cf1ade6f Fix #341: the test window is tall enough to show the By Date rows
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:38:10 +00:00
Claude 00388f3fd2 Fix #341: test from Recently Deleted (the demo has picks)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:37:25 +00:00
Claude 1b4518c202 Fix #341: By Date and Keywords rows show their photos from All Photos
The rows count every photo in the library, but choosing one filtered
whatever source was open (an album, a folder, Picks), so it often showed
nothing. Choosing a row now switches to All Photos first; clearing it
leaves the source alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:36:25 +00:00
Adrian D'souzaandCodex d16ab1f238 PR218: check SQLite local payload span
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:52 +05:30
Adrian D'souzaandCodex bbcfdf085b PR218: harden catalog import review fixes
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex 1e6035d9db test: encode SQLite fixture serial types & overflow offsets correctly
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex 37cfc46e0d PR218: place prepared-import regression tests after production items
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex b256b8b635 PR218: bound egui worker message size & simplify cancellation
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex e8c817ddea PR218: satisfy current CI clippy without changing import behavior
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex d367d83ee6 LRC-LIB-CATALOG-IMPORT: exercise actual concurrent develop edit
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex 6a05f865ee LRC-LIB-CATALOG-IMPORT: refresh rebased parity summary
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex 866d02e577 LRC-LIB-CATALOG-IMPORT: address SQLite, worker & archive review
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Orthic LabsandCodex 4a427894ed LRC-LIB-CATALOG-IMPORT: import Lightroom Classic catalogs
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Claude aaa46003d5 Fix #316: the header keeps its hover shade while the pointer is on the eye
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:34:35 +00:00
Claude a9cd761d52 Fix #316: the section eye stays while the pointer is on it
The eye was drawn only while the header itself was hovered; with the
pointer on the eye, egui reports the eye (on top) as hovered and not the
header, so the eye vanished under the pointer and could not be clicked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:33:25 +00:00
Claude 68267810b3 Fix #316: headless test of the section eye
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:30:43 +00:00
Claude 9b4ba1aa80 Fix #316: the section eye switches off Light, Color and Detail too
The pipeline honoured a section's eye only where it applies Effects,
Optics, Geometry and Calibration, so turning the eye off on Light, Color
or Detail changed nothing. DevelopSettings::effective() resets the
switched-off sections to their defaults (Light with the tone curve; Color
with white balance, mixers, grading and point colour; Detail), and both the
CPU and GPU renderers render those. Borrowed, so free, when every section
is on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:29:29 +00:00
Claude 0634782fbc i18n: German and Russian follow-ups (#265, #264)
- #265's German coverage test failed whenever any catalog key, command or
  control label, or line of What's New lacked German, so every feature PR
  would have had to ship German. It reports those gaps for every language
  now (display_label_gaps_are_reported), as the project's other gap test
  does; docs/localization.md says how release-note lines are matched and
  that an English copy in a formats catalog marks an untranslated message.
- Russian: docs/localization-ru.md, the language lists in
  docs/localization.md and docs/parity.md, ru tag tests.
- What's new: German and Russian UI. Removes a status sentence #265 put
  in parity.md's Top gaps list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:26:56 +00:00
Claude 9b5e771fd8 Merge PR #264
# Conflicts:
#	crates/ui-egui/src/i18n.rs
#	crates/ui-egui/src/menus.rs
2026-10-08 18:25:10 +00:00
Claude 69384ad430 Merge PR #265
# Conflicts:
#	crates/ui-egui/locales/ja-formats.json
#	crates/ui-egui/locales/zh-hans-formats.json
#	crates/ui-egui/locales/zh-hant-formats.json
#	crates/ui-egui/src/lib.rs
2026-10-08 18:25:01 +00:00
Claude acc85c4147 Fix #322: the value field draws in a child UI and leaves the slider rows in place
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:22:43 +00:00
radialmonster e2d35de867 LR-IMP-ADD-DIALOG: file dialogs match extensions in any letter case on Linux (#342)
rfd turns each extension into a *.ext glob that GTK and the desktop portal
match case-sensitively, so DSC_1234.NEF was hidden. One helper
(dialog_filter) gives every dialog filter the lower, UPPER and bracket
forms on Linux; the Add Photos filter uses import::EXTENSIONS.
2026-10-08 14:21:58 -04:00
Claude 0fbf20db51 Fix #322: select the old value once the field has the focus
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:21:10 +00:00
Claude 2b66eb4800 Fix #322: the typed-value field takes the keyboard on the frames after the click
The click's own release took the focus straight back from a field created
in that frame, so typing went nowhere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:19:31 +00:00
Claude 92c7237c50 Fix #322: type an exact value into a slider
Click the value next to a slider's name to edit it: the old value starts
selected, Return or clicking away applies the typed number on the
control's steps and range as one undo step (like a keyboard nudge), and
Esc or text that isn't a number keeps the old value. A decimal comma,
a leading + and a % or K unit are accepted. The keys typed go to the
field, not to the shortcuts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:05:19 +00:00
radialmonster 34ecea4f24 LR-IMP-CAMERA-COVERAGE: read Canon ColorData stored as bytes, so PowerShot and EOS M CR2s get their as-shot white balance
Maker note tag 0x4001 is UNDEFINED bytes (ColorData versions -3 and -4) on
PowerShot and EOS M bodies. It was read as one number per byte, so no plausible
as-shot levels were found and the white balance was 1, 1, 1 (green cast, a* about
-13 on neutrals). Byte and undefined data is now paired into 16-bit words in the
maker note's byte order before the existing probe (word 71). No per-model table.

RENDER_CACHE_VERSION 11 -> 12.
2026-10-08 14:03:23 -04:00
radialmonster ff8d027766 LR-IMP-FORMATS: NEF lossy type 2 curve of version 0x44 0x40 spans a quarter of the sample range
Maker note 0x0096 of the D780, D6, Z 5, Z 6, Z 7 and Z 7_2 starts 0x44 0x40
and holds 257 points, but its first interval rises by 2^bits / 1024 where the
0x44 0x20 tables rise by 2^bits / 256: the curve is the identity in the dark
part and covers 2^bits / 4 codes, not 2^bits. The decoder spaced the points
2^bits / 256 codes apart for both, so these files decoded to a fraction of
their values, far below the black level.

parse_table now takes the span from the version (0x20: all codes, 0x40: a
quarter). Other versions decode to identical samples. A known-answer test
built from hand-coded table bytes and bit strings fails without the change.
Bumps RENDER_CACHE_VERSION to 12.
2026-10-08 14:02:01 -04:00
Claude 4fb8ad4dcd Format the #333 merge fix
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:01:00 +00:00
Claude dc67d348e6 Merge fixes: #333 hover label id, #346's test boxes the import dialog (#325), unused mut
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:00:16 +00:00
Claude 7f386aa258 Merge fixes: close the test functions the #307/#325 and #325/#346 merges ran together
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:58:41 +00:00
Claude 399aedd23a Docs: What's new for the import review size and Shift-click ranges, CLI logging (#346, #347, #340)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:57:54 +00:00
Claude b5fcf58e55 Merge PR #347
# Conflicts:
#	crates/ui-egui/src/import.rs
2026-10-08 17:57:23 +00:00
Claude 261994acbc Merge PR #346
# Conflicts:
#	crates/ui-egui/src/headless.rs
2026-10-08 17:57:02 +00:00
Claude b936c1a683 Merge PR #340 (CLI logging; the desktop keeps #286's file logger) 2026-10-08 17:56:31 +00:00
Claude cab6af8608 Fetch: say where the bearer token goes (#259)
The token_env comment said the token goes to the first mirror's host; it
goes to every configured mirror's own host (never to a redirect's).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:55:31 +00:00
Claude 4eb9b565e0 Profiles: imported LUT names shown as named, own ids for their folders (#333)
#333 passed imported LUT names and folder names through tr(), so a LUT
named like a message ("Vivid") was shown translated; they are now shown
as the file names them (builtin_label / new profile_label, as for user
presets). A LUT folder named like a built-in group ("Film") shared that
group's menu widget id and the browser section's open state; both are
keyed apart now. Restores the "Amount 0 = the photo as it was" assertion
#333 replaced, and adds a What's new line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:54:50 +00:00
Claude ccdd9e45d1 Merge PR #333 2026-10-08 17:52:46 +00:00
Claude 24c800f46f Merge PR #297 2026-10-08 17:52:46 +00:00
Claude e471e7c346 Merge PR #263 2026-10-08 17:52:46 +00:00
Claude aaec05dd11 Merge PR #262 2026-10-08 17:52:46 +00:00
Claude d808749792 Desktop: a --memory session writes no log file (#286 with #288)
#288 made --memory sessions write nothing (issue #164); #286's log file
would have been the exception. It is not opened there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:52:46 +00:00
Claude c03a67c59f Merge PR #286
# Conflicts:
#	apps/lightcraft/src/main.rs
#	docs/whats-new.md
2026-10-08 17:51:24 +00:00
Claude cd8d2b96a2 Merge PR #259 2026-10-08 17:50:50 +00:00
Claude a12fabd588 Library: the Photo Grid label footer is translucent; pt-br for "Color label cleared" (#308)
#308 painted an opaque label-coloured bar over the bottom 24 px of every
labelled photo in Photo Grid, hiding part of the photo, while its docs
said the photo pixels stay as they are. The footer is now translucent
over the photo and the docs say so. Also adds the one string pt-br lacked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:39:23 +00:00
Claude 99fa19099b Merge PR #308
# Conflicts:
#	apps/lightcraft/src/native_menu.rs
#	crates/ui-egui/src/panels/detail.rs
#	docs/parity.md
2026-10-08 17:38:20 +00:00
Claude bac4666e7f Docs: What's new for the crop lock, trackpad zoom/pan, Folders and Select All (#312, #313, #307, #325)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:37:27 +00:00
Claude b7f60a1fac i18n: translate the strings #312, #313 and #325 added (ja, pt-br, zh-hans, zh-hant)
Crop Apply / Lock / Locked, Set Image Zoom and Pan, the import review's
Recently Deleted choices and Empty Recently Deleted. The import's undo
labels and its "restored from Recently Deleted" summary used format!, so
they could not be translated; they go through tr_format! now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:37:06 +00:00
MarekSpodBiedry 4d506494d9 Issue #337: the Import Photos dialog is bigger and resizable; the photo grid fills it
The review dialog was a fixed-size window whose thumbnail grid was capped at 330 points (about 6 x 2.5 thumbnails). It now opens at 960 x 720, can be resized by its edges and corners, and the grid takes whatever height the options and buttons leave. The dialog has a window id of its own so its size does not carry over to the other dialogs.
2026-10-08 19:35:08 +02:00
Claude 90fe6d1616 Merge PR #325
# Conflicts:
#	crates/ui-egui/src/panels/detail.rs
#	crates/ui-egui/src/tests_panels.rs
2026-10-08 17:34:31 +00:00
Claude fa630a233a Merge PR #307
# Conflicts:
#	docs/parity.md
2026-10-08 17:34:04 +00:00
Claude 9e9e45b3c7 Merge PR #313
# Conflicts:
#	crates/ui-egui/src/panels/detail.rs
#	docs/parity.md
2026-10-08 17:33:58 +00:00
Claude 259de3d9df Merge PR #312 2026-10-08 17:33:04 +00:00
Claude 8b70d33d0f Fix #319: release builds pin craft-fonts with Noto Sans CJK SC
release.yml (five jobs) and freebsd.yml pinned craft-fonts abb83316, from
before Noto Sans CJK SC landed there, so v0.4.0 embedded only the Japanese
faces and most of the Simplified Chinese UI was tofu. The pin is now
8dcdacd5 (craft-fonts HEAD: Noto Sans CJK SC and Noto Sans Arabic), which
the Noto CJK exception merged from #203 allows. docs/releasing.md says when
to bump the pin and how to check it before a release.

flake.nix / flake.lock (community-maintained, not in CI) still pin the old
revision: the lock's narHash needs nix to recompute.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:32:09 +00:00
Claude 5e2ece0cf3 Merge PR #203 (Noto CJK allowed via craft-fonts) 2026-10-08 17:31:36 +00:00
ClaudeandHanmo 53584512f4 Issue #188: Tint sign documented as landed (DNG / Lightroom sign)
#258 left the Tint row at 🟡 and docs/tint-direction.md as a review note.
The sign now matches the slider's track, the DNG specification and
Lightroom XMP, so the row is back to ✅; the doc says what happens to
saved custom tints (they shift the other way; nothing is rewritten) and
What's new tells users.

Co-Authored-By: Hanmo <letsgo@88.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:29:47 +00:00
Claude e4737f68c7 Format: rustfmt the Show in Explorer button (#324)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:29:18 +00:00
Claude 21de86b3df UI: the loupe follows a texture's aspect only when it really differs (#280)
A texture's whole-pixel size is slightly off the frame's exact aspect (a
128 px thumbnail by up to ~1 %), so following it always made the loupe
rect jump a little as the thumbnail, preview and full render arrived, and
moved mask and crop overlays with it. Follow it only when it differs by
more than 2 % (an unsupported raw's embedded JPEG with another crop), and
never in the crop tool, whose overlays need the frame's own geometry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:29:00 +00:00
Claude 0b101be155 Issue #181: saved presets and Export with Previous drop keys this version doesn't know
#287 made app.export refuse unknown params. Stored settings (a user export
preset, prefs.json lastExport) written by another version are not the
caller's typo, and refusing them would break that preset or Export with
Previous for good after an update: their unknown keys (also inside
watermark) are dropped before the strict check. What the caller passes is
still checked strictly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:29:00 +00:00
ClaudeandDWAA1660 7d2bb5e5ec Issue #188: D65 reads as a small positive tint (test from #336)
Daylight sits above the Planckian locus, so with the DNG / Lightroom tint
sign D65 converts to about +10, as Lightroom's Daylight preset has it.
Taken from #336, which made the same sign fix as #258.

Co-Authored-By: DWAA1660 <80738438+DWAA1660@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:26:01 +00:00
MarekSpodBiedry 3a9fe13d3a Import: Shift-click checks or unchecks a range of photos in the review (#338) 2026-10-08 19:23:43 +02:00
Claude 2565f893c4 Merge PR #257
# Conflicts:
#	docs/parity.md
2026-10-08 17:21:02 +00:00
Claude 4e2584e559 Merge PR #277
# Conflicts:
#	docs/parity.md
2026-10-08 17:21:02 +00:00
Claude cb173c22dc Merge PR #311
# Conflicts:
#	docs/parity.md
2026-10-08 17:20:41 +00:00
Claude 98da2a0229 Merge PR #288
# Conflicts:
#	docs/whats-new.md
2026-10-08 17:19:31 +00:00
Claude 1ece9341d6 Merge PR #324 2026-10-08 17:18:59 +00:00
Claude 6d52e0744a Merge PR #280 2026-10-08 17:18:59 +00:00
Claude dd783b1d38 Merge PR #267 2026-10-08 17:18:59 +00:00
Claude 81e28f25c9 Merge PR #255 2026-10-08 17:18:59 +00:00
Claude b5b6f8caa8 Merge PR #300 2026-10-08 17:18:58 +00:00
Claude 7e21d185b8 Merge PR #258 2026-10-08 17:18:58 +00:00
Claude 70db2f2320 Merge PR #292 2026-10-08 17:18:58 +00:00
Claude 74b9129a05 Merge PR #287 2026-10-08 17:18:58 +00:00
Claude 36c6920f55 Merge PR #285 2026-10-08 17:18:58 +00:00
Claude 6d22d1235b Merge PR #284 2026-10-08 17:18:58 +00:00
Claude 5f9548a7ba Merge PR #269 2026-10-08 17:18:57 +00:00
Claude 0e188f6a40 Merge PR #268 2026-10-08 17:18:57 +00:00
Claude a66eebf778 Merge PR #261 2026-10-08 17:18:57 +00:00
Claude e3dc820d89 Merge PR #289 2026-10-08 17:18:57 +00:00
DWAA1660andClaude Opus 5.5 a6cedcc9e8 Issue #168: the desktop app and CLI log warnings to stderr; RUST_LOG works
lightcraft-cli installed no `log` backend, so every log::warn!/info! record
(the GPU-to-CPU fallback, an unknown LIGHTCRAFT_GPU_BACKEND, the adapter
chosen) was dropped and RUST_LOG, which docs/gpu-pipeline.md documents for
the fallback warning, did nothing. The desktop app had its own logger that
read only LIGHTCRAFT_LOG.

Both now install lightcraft_engine::logging: warnings and errors on stderr
by default; LIGHTCRAFT_LOG or RUST_LOG (off, error, warn, info, debug,
trace) picks another level, LIGHTCRAFT_LOG winning, and RUST_LOG's
per-target directives (`lightcraft=debug,wgpu=warn`) set LightCraft's
level. Below warnings only LightCraft's own records are shown, as the app's
logger did. The app's --help lists the variable.

Tests: level parsing and target filtering (engine logging::tests); the CLI
prints "LIGHTCRAFT_GPU_BACKEND=bogus names no known backend" with
RUST_LOG=warn and nothing with RUST_LOG=off (cli.rs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K3vVft4VXvvy8SpbbUceTW
2026-10-08 17:14:04 +00:00
MarekSpodBiedry f2bcc9219c Issue #234: Windows opens links (Map button) in the browser, not File Explorer
explorer <url> opened File Explorer for links with a query or fragment, such as the OpenStreetMap link behind the Map button. Windows now opens https links with rundll32 url.dll,FileProtocolHandler, which hands the URL to the default browser without a shell.
2026-10-08 18:59:04 +02:00
Joshua Warren 0613f925de Support remote SAM3 inference through a private Metal worker 2026-10-08 11:47:59 -05:00
Walter Boringandopencode 853ada7754 LR-IMP-FORMATS: HEIC/HEIF decode behind the optional heif feature
New crate `lightcraft-heif` (a port of PhotoCraft's photocraft-heif): a thin,
panic-guarded wrapper around heic-rs (pure Rust, pinned =0.1.1) - 8/10-bit,
alpha, grid tiles, container irot/imir/clap, ICC/EXIF/XMP. `lightcraft-codecs`
decodes HEIF behind its non-default `heif` feature (HEVC patents are the
distributor's call, the same policy as PhotoCraft); the apps forward it. HEIC
files stay recognised without the feature: opening one is a clear
"isn't included in this build" error. Fuzz-found heic-rs panics arrive as
regression fixtures (inline bytes, no media committed).

Verified end-to-end on real iPhone HEICs (lightcraft-cli render, rotation and
metadata correct). Note for reviewers: measured here, the engine currently
ignores the decoded image's source-space matrix on export for ANY format (a
true Display-P3 PNG round-trips unchanged); the ICC and space are carried for
all codecs, so this inherits - and does not worsen - existing behaviour.

Co-Authored-By: opencode <noreply@opencode.ai>
2026-10-08 12:45:33 -04:00
peiyan0 493600743d engine, ui: support imported LUT profiles in profile dropdown and browser (#328) 2026-10-09 00:43:19 +08:00
Xus BadiaandClaude Opus 5.5 e4ab8422b4 macOS DMG: branded window background and icon layout
The DMG opened as a plain Finder window. It now shows the app icon's figure on
the LightCraft colour, with LightCraft.app and the Applications link side by side.
Same approach as VectorCraft's (storytold/vectorcraft#493).

- packaging/macos/dmg/: background.svg (source; text outlined, the figure an
  <image> of assets/app-icon), background.tiff (1x + 2x HiDPI, Deflate, sRGB)
  and dmg-layout.DS_Store (Finder view settings: window, icon size 128,
  positions, background).
- generate.py writes the TIFF and the layout on any OS, byte-for-byte
  reproducible, with nothing from the machine that ran it (no paths, users,
  disk names or volume UUIDs).
- package.sh copies them into the staging folder; the image is still built with
  hdiutil makehybrid. Nothing extra runs in the signing job.
- The volume name drops the version: the layout finds the background through
  an alias that includes the volume name. The DMG file name keeps the version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 17:55:32 +02:00
Nathaniel LedfordandClaude Sonnet 5.5 0b5ef3fe11 Import review: choose what happens to files in Recently Deleted (#298)
Files whose photo is in the trash stay unchecked until Restore them or Import as
new is chosen; the toast and undo label say what was restored.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:39:28 -04:00
peiyan0 38d2ff07c9 desktop: fix Windows Explorer reveal path and platform file manager labels (#272) 2026-10-08 23:36:56 +08:00
Nathaniel LedfordandClaude Sonnet 5.5 6c19090fde Grid: selected photos whose thumbnail isn't drawn yet still show as selected (#298)
The grid and filmstrip now both ask Selection::state_of.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:29:05 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 5b80fc6bcf MCP import onDeleted; Empty Recently Deleted in the trash view's Photo menu; parity row
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:29:05 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 0a474cc06e Permanent delete of several photos is one consistent op list; restored photos join album/selection
Review of the onDeleted work found that purging photos one at a time built each
op list against the unchanged catalog, so photos sharing an album or a stack made
Empty Recently Deleted (and Fresh imports) fail with NoStack or leave dangling
album entries. Catalog::delete_photos_permanently_ops builds one list for all of
them; Empty, Delete Permanently and the Fresh purge use it. Photos an import
restores now take part in album/albumName/selection, and undo labels say Restore.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:29:05 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 2928880e81 Import: onDeleted policy (skip|restore|fresh) for files in Recently Deleted; library.emptyRecentlyDeleted
A file that is in Recently Deleted was always skipped as a duplicate with no way
forward (#298). The report now flags such duplicates (existingDeleted); onDeleted
restores the trashed photo with its edits or replaces it with a fresh import; and
Empty Recently Deleted removes every trashed photo in one undoable step.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:16:42 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 14a03052de Filmstrip: show every selected photo as selected, not only the active one (#298)
After Select All in the loupe only the active filmstrip cell was highlighted,
so the selection looked like a single photo. Cells now use Selection::state_of;
selected-but-not-active cells get the selected fill and a grey outline.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:01:50 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 7f6e4e48f7 Selection::state_of: one answer to active / selected / not selected for every view
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 11:01:50 -04:00
sebastian pfluegelmeier 912f56cac8 LR-VIEW-ZOOM: add trackpad zoom and pan with stable cursors 2026-10-08 16:38:56 +02:00
Nathaniel LedfordandClaude Sonnet 5.5 0e254bfbc6 Crop UI: lock toggle, custom ratio, edge-safe handle drags (#295)
The crop overlay now reports pointer positions to crop.drag instead of
computing rectangles itself. The Crop panel gets a visible Lock toggle, a
custom W × H entry, a dropdown chevron and a button label that names the
active ratio ("Original", a preset, or 1.37 : 1) instead of pixel sizes.
LR-CROP-RECT / LR-CROP-ASPECT parity rows updated.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 10:35:46 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 8d8178dc0a Engine: crop.drag command; validate custom crop ratios (#295)
crop.drag {handle, from, to, start?} runs drag_crop through the command
layer so the UI, CLI, control channel and MCP agree; a locked crop keeps the
shape of its frame. crop.aspect now rejects non-finite, non-positive and
beyond-20:1 ratios and non-numeric arrays instead of storing a degenerate
lock that produced an empty crop.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 10:35:46 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 90c63af951 Crop: drag_crop domain operation for handle drags (#295)
Dragging a crop handle past the image edge used to hand the raw rectangle to
CropGeometry::constrained, which slid the whole frame toward the centre and
moved the anchored corner. drag_crop owns the drag instead: the opposite
edges stay put, a locked ratio holds on all eight handles (edges grow about
their midline, sliding along a blocked side), the frame stops at the image
edge and slides along it, and it keeps a minimum size. Hostile numbers return
the start crop. Covered by Given/When/Then scenarios and property tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 10:35:46 -04:00
midasdfandCodex 69640df8a7 LR-EXP-WATERMARK: shape Japanese vertical watermark glyphs
Co-Authored-By: Codex <noreply@openai.com>
2026-10-08 23:34:43 +09:00
Hanmo a47afdc1f4 M1: merge upstream main and resolve documentation conflicts 2026-10-08 22:18:13 +08:00
Brandon ThomasandClaude Opus 5.5 2472021091 Bump version to 0.4.0 (from 0.3.0) for release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:01:45 -04:00
Stano BustorandGPT-6 Luna b4a192a54e UI: preserve preview aspect ratio across views
Keep embedded and decoded preview proportions consistent in the justified
grid, photo views, alternate detail previews, and culling previews.

Co-Authored-By: GPT-6 Luna <noreply@openai.com>
2026-10-08 16:00:00 +02:00
60828c3eaf README Downloads: where to get it first, then builds with Windows before macOS (#306)
Co-authored-by: Brandon Thomas <shared2@storyteller.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 09:50:53 -04:00
ebe54f934e Release: FreeBSD, Flatpak x86_64/aarch64, AppImage zsync updates; README Downloads (#302)
* Release: FreeBSD x86_64, Flatpak x86_64/aarch64, AppImage zsync updates; README Downloads

- linux: appimagetool -u embeds gh-releases-zsync update information; the .AppImage.zsync is
  published beside each AppImage (zsync installed; checked in the job).
- flatpak: new job per arch repackages the Linux tarball into a single-file .flatpak
  (packaging/linux/flatpak-bundle.sh + ai.storyteller.lightcraft.bundle.yml); the Flathub
  manifest gets the CLI and the same commented finish-args; packaging-lint checks they agree.
- freebsd: new job builds lightcraft-<ver>-freebsd-x86_64.tar.gz in a FreeBSD 14.3 VM with the
  freebsd.yml packages (packaging/freebsd/package.sh; layout dry run in packaging-lint).
- version, linux and web no longer use the release environment (they sign nothing), so a
  branch dispatch dry-runs every unsigned job; the release job still needs macOS and Windows.
- README: Downloads section listing every build; docs/releasing.md updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Linux packages: describe LightCraft, not PhotoCraft, in the deb/rpm description

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Brandon Thomas <shared2@storyteller.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 09:45:26 -04:00
Hanmo 4256865cf4 M1: tint labelled thumbnails and colour label feedback 2026-10-08 21:42:21 +08:00
Hanmo 57c9e252c1 M1: show bottom confirmation for colour label actions 2026-10-08 21:25:02 +08:00
Nathaniel LedfordandClaude Sonnet 5.5 46c4a2709d Sidebar: names end in an ellipsis at the count; bars end at the panel edge
A name that does not fit is cut with an ellipsis before the photo count (the
full name on hover) instead of running past it; scrolling the sidebar sideways
or widening it shows more. A row asks for room for at most 140 pt of its name,
so depth, not one long name, makes the sidebar scroll. Selection and hover
bars end at the panel's edge when the content is wider.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 09:06:18 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 198286e589 Folders: round-4 review fixes (counts at the edge, edge-space names, smart albums)
- ui: counts stay at the visible edge when the sidebar scrolls sideways
  (with the row's colour behind them where a long name runs under); the
  smart-album dialog's live count includes the album's folder
- catalog: a folder path is used as given (a name with a space at its edge is
  not its neighbour: Remove acted on the wrong folder); a plain POSIX path
  is checked against the folder without building a key (about half the cost
  on a big library, with a test that it agrees with the careful check)
- engine: editing a smart album's rules keeps the folder the editor cannot
  show; a shown folder that loses its last photo gives way to All Photos;
  library.state and view.json name the folder only while it is shown; a blank
  folder filter is no filter

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 09:06:18 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 6f33aacd22 Sidebar: scrolls sideways so deep folders keep their nesting
Every level indents one step more than its parent however deep it goes (the
five-level indent cap is gone). The whole sidebar scrolls in both directions:
rows report how wide they need to be, the widest one of the last frame sets
the content width (at least the panel's), and a bar appears only when it
overflows. Folder names are no longer cut in the tree; Create Album stays at
the visible edge.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 09:06:18 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 5d13d59520 Folders: review fixes (smart albums, disk guards, deep chains, long titles)
- engine: a smart album made from a folder view keeps the folder; only a
  folder that shows exactly its own photos can be the source (not the
  startup disk, not a folder above disks, not one with no photos); a
  folder source and a folder filter never coexist; a folder source with no
  folder shows nothing; a saved view whose folder is gone opens on All
  Photos; removing a folder above a disk needs `disk: true` even with one
  disk mounted; library.folders documents volume/selectable
- ui: deep chains stop indenting after five levels, the grid title gives
  way to the photo count (full name on hover), disclosure triangles have
  spoken names, the remove dialog says what is included and has a disk
  title; test temp folders are cleaned up on failure

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 09:06:18 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 0732edc64f Folders: every folder its own node; choosing one loads it like an album
- catalog: a row is exactly one folder (no folding of single-child levels);
  the UI starts a folder that holds nothing and leads to one folder open
- engine: a library folder is a source (LibrarySource::LibraryFolder with
  Session::library_folder, like Local's browse): library.source
  {kind: libraryFolder, path}, kept with the library's view, shown by
  library.state, followed through folder.rename / folder.move and undo,
  dropped back to All Photos when its folder is removed
- ui: clicking a folder replaces whatever the grid showed (Picks, Recently
  Deleted, a Local folder…) instead of narrowing it with a filter chip; the
  grid is titled after the folder

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 09:06:18 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 82e827b9d9 Sidebar: Folders section for the folders the library's photos were imported from
Another way to explore the library next to By Date and Keywords: a tree of
the folders that hold imported photos, with counts (subfolders included),
under the disk or network share each is on. Local stays the way to browse
any folder on disk.

- catalog: Catalog::folder_tree and Filter::library_folder. Each disk is a
  volume row (drive letter, Windows share, /Volumes/<share>, /media, /mnt,
  gvfs, /net…), found from the stored paths alone, so an offline share costs
  nothing. Levels that hold nothing and lead to one folder fold into one row;
  depth is bounded. The tree and the filter read paths through one splitter
  (separators, `.`, `..`, verbatim UNC prefix, repeated separators), and a
  row that would cover other disks' photos is not choosable
- engine: library.folders, library.removeFolder (to Recently Deleted, one
  undo step, no file touched; a whole disk or a folder spanning several
  needs `disk: true`, the startup disk never), libraryFolder in
  library.filter, chip; a chosen folder follows folder.rename / folder.move
  and their undo
- ui: collapsible Folders section; the chosen folder's row is always opened
  into view; right-click (row or triangle) Rename, Move, Show in Finder,
  Remove from Library…; disk rows Remove Disk from Library…; both ask first

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 09:06:18 -04:00
issuenateandClaude Opus 5.5 88c1e768f1 TIFF: decode 1-, 2- and 4-bit samples
Bilevel scans, CCITT fax images and 2- or 4-bit gray TIFFs were rejected with
"sample buffer too short". The `tiff` crate returns such samples packed, most
significant bits first, eight, four or two to a byte with each row padded to a
whole byte; the converter expects one byte per sample. They are now spread out
one per byte and scaled to 0–255 (255, 85 and 17 times the sample: exact) before
conversion. 3-, 5-, 6- and 7-bit samples, and planar images with more than one
sample narrower than a byte, are refused with an "unsupported" error.

The new test builds 1-bit (both photometric conventions, a width that leaves row
padding, set padding bits), 2-bit and 4-bit strips in code, plus a truncated
strip that must fail cleanly; it fails without the fix with the old error. A
600×400 bilevel image, uncompressed and CCITT G4, now decodes to the same pixels
PIL reads from it.

The module documentation no longer claims palette support: the `tiff` crate
refuses RGBPalette images on every read path, so the palette branch here has
never been reached. The parity row lists palette TIFF as missing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:47:51 +02:00
issuenateandClaude Opus 5.5 cc425d0cea TIFF: invert WhiteIsZero gray once, not twice
A WhiteIsZero (PhotometricInterpretation 0) gray TIFF opened as a negative at
every bit depth. The `tiff` crate (0.11) already inverts those samples as it
reads them, for every layout it accepts, and refuses the WhiteIsZero layouts it
can't invert; the decoder here then inverted them a second time. WhiteIsZero is
common in scanner and fax output, and CCITT fax images use it by definition.

The second inversion is removed. The new test builds 8- and 16-bit WhiteIsZero
and BlackIsZero strips in code and fails without the fix (the lowest sample
came out black instead of white).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:26:27 +02:00
HanmoandGardy 6277af55bf M1: restore library culling shortcuts and Classic shift actions
Keep macOS plain-key ratings, labels and flags with egui, adapting the
native menu ownership correction from PR #261 (4e012a5, Gardy).
Add Shift+6-9 and contextual grid Shift+P, recognize shifted digit
punctuation, and document the verified Lightroom Classic mapping.

Cover batch changes and undo, exactly-once advance, Compare targets,
text and crop context, physical number keys and native menu ownership.
Verified native demo/control screenshots and all seven cargo xtask ci gates.

Co-authored-by: Gardy <armandpro50@gmail.com>
2026-10-08 20:25:37 +08:00
Brandon ThomasandClaude Opus 5.5 b8df5f472c Bump version to 0.3.0 (from 0.2.1) for release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 08:12:53 -04:00
radialmonster 30f9eb3674 LR-PROF-CAMERACOLOR: list the #232 entry after the other RAW decoding entries 2026-10-08 08:04:42 -04:00
wstockandClaude Opus 5.5 7177b9b9f3 Issue #187: the filmstrip highlights every selected photo, not only the active one
The filmstrip painted selection chrome only for the active photo, so after
Select All or a ⌘/Shift-click multi-selection the other selected photos
looked unselected, although the selection itself was right.

Paint it as the grid does: every selected photo gets the selected cell
fill; the active one keeps its white frame and the others get the grid's
grey one.

Test: tests_scroll::filmstrip_shows_every_selected_photo samples a cell's
background in the headless render after Select All, after selecting one
photo, and after a toggle (what a ⌘-click runs); before the fix a selected
photo that was not the active one was painted like an unselected one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:55:57 +00:00
wstockandClaude Opus 5.5 c60482f753 Issue #266: filmstrip labels cut file names on characters, not bytes
The detail view's filmstrip shortened a long file name with `&name[..13]`,
a byte slice. For a name with multi-byte characters byte 13 can fall
inside a character (`202407層三限定訂閱圖(4)`: inside '限'), and the
slice panicked the UI thread as soon as that photo scrolled into the
strip, so the app quit.

`film_label` now counts characters: more than 14 are cut to 13 and an
ellipsis, as before for ASCII names.

Tests: a unit test of `film_label` (CJK, exactly 14 characters, ASCII,
empty) and a headless filmstrip test with the name from the issue; both
panicked with the reported message before the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:39:52 +00:00
wstockandClaude Fable 5.1 c1861e5416 Issues #164, #169: --memory sessions write nothing — no ui.json, no settings folder
`--help`, the module doc and the README promise that a `--memory` session
"writes nothing", and the desktop app broke that in two places:

- `PrefsWriter` saved `ui.json` whenever the UI state changed (issue #164),
  so a throwaway session's `thumbSize`, window layout or language replaced
  the user's saved settings — the README showcase script does exactly that.
- The GPU crash sentinel from #136 armed itself in every session except
  `LIGHTCRAFT_NO_PREFS` (issue #169); `with_init_marker` creates the
  settings folder to hold `gpu-init.marker`, removes the marker and leaves
  the folder, so a `--memory` run on a fresh home left an empty
  `Library/Application Support/LightCraft` behind.

Now `PrefsWriter` carries its target path (`None` for `--memory`,
`LIGHTCRAFT_NO_PREFS` or no home folder) and writes nothing without one; it
also writes nothing while the session is the one offered when the library
can't be opened (the question window and Continue Without Saving, which
`docs/control-protocol.md` already describes as writing nothing) and
resumes when `app.openLibrary` ends it. `gpu_crash_check(in_memory)` neither
arms the sentinel nor removes a marker in a `--memory` session; it still
reads one (new `read_init_marker`), so GPU rendering starts off there too
and the next ordinary launch reports and clears it. A `--memory` session
still reads `ui.json`, but a damaged one is reported and left in place
rather than set aside. Normal launches are unchanged.

Tests: `prefs_writer_writes_nothing_in_a_memory_or_temporary_session`,
`memory_session_leaves_the_gpu_marker_and_folder_alone`,
`memory_session_leaves_a_damaged_ui_json_alone` (apps/lightcraft), and
`read_init_marker` coverage in the gpu crate's marker test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 11:17:38 +00:00
wstockandClaude Fable 5.1 88d9b7196d Issues #181, #183: app.export refuses unknown params and bad values; watermark keys documented and range-checked
`ExportOptions::from_json` reads what it recognises and defaults the rest, so
`longEdgee=400` exported at 3000 px, `longEdge=banana` at full size, and a
`watermark` of `5` or `{"text":"x","bogus":1}` passed with `ok:true`, while
`size: 3` was silently clamped to 0.5 of the short edge (#181, #183).

`from_json` stays lenient for stored prefs and dialog prefill. New
`ExportOptions::validate(cmd, params)` / `from_params(params)` check every key
against the option and target lists (unknown ones name the nearest known key),
every value's kind and range, and a `watermark` object's keys, kinds and ranges
(`size` 0.005..0.5 and `inset` 0..0.4 of the short edge, `imageWidth` 0.01..1 of
the width, `opacity` 0..1, `color` [r,g,b]), returning
`invalid parameters for \`app.export\`: …` like the other commands. The three
places that run an export use it — the desktop `app.export`, the headless
backend behind `lightcraft-cli run` and the MCP `export` tool, and
`lightcraft-cli render --opt` — and `export.savePreset` validates a preset's
params when it is saved. Built-in presets, the Export dialog's `to_json`,
Export with Previous and `render --opt` all pass unchanged.

The `app.export` param string (`lightcraft-cli commands`, `engine.commands`),
the MCP `export` tool schema and docs/parity.md LR-EXP-WATERMARK now list the
watermark keys, their units and ranges.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 11:06:59 +00:00
Gert Pellin 47f40eb2d5 Desktop: write log records to a rotating log file as well as standard error
The desktop app's logger (StderrLog in main.rs) wrote to standard error
only. A launch from a desktop menu or the Dock has no terminal, so its
warnings were lost, and the panic log in the temp folder is gone after a
reboot: there was nothing to attach to a bug report.

apps/lightcraft/src/logging.rs replaces it (no new dependency):
- standard error plus logs/lightcraft.log in the settings folder, next
  to ui.json (Linux $XDG_CONFIG_HOME/lightcraft/logs, never the library);
  each launch rotates the previous log to lightcraft.1.log and .2.log, so
  the log of a run that crashed survives the next start; the file is
  capped at 16 MiB with one note
- records logged before the settings folder is known are buffered
  (bounded) and written once the file is attached
- default levels: info for lightcraft*, warn for everything else;
  LIGHTCRAFT_LOG=info|debug keeps its old meaning (a test compares it
  with the old logger) and wins over RUST_LOG, which otherwise replaces
  the default with env_logger-style directives
- never panics: an unusable folder or a failed write leaves standard
  error as the only sink; a poisoned lock is taken back
- runs with LIGHTCRAFT_NO_PREFS (tests, scripts) log to standard error
  only, so they don't rotate away the user's own logs
- a panic hook chained after the engine guard's adds escaped panics to
  the file (try_lock, so a panic while logging can't hang the app);
  lightcraft-panics.log in the temp folder is kept

The logger is installed first in main and the file is opened after
argument parsing (--version, --help and usage errors leave no file).
The start-up and control-server messages in main.rs and
control_server.rs go through it instead of eprintln!, and a --control
or LIGHTCRAFT_CONTROL_PORT value that isn't a port is now reported.
Documented in README (Quick start, Logs), --help and docs/whats-new.md.
2026-10-08 13:00:29 +02:00
wstockandClaude Fable 5.1 39e04ae7ab Issue #157: colour-range mask tolerance narrowed so a sample stops at its own colour
`tol = 0.04 + 0.16·refine` fully selected anything within 0.06 OkLab of a
sample at the default Refine 50. Through `tonemap_for_select` the report's
teal mid-grey and neutral light grey are only ~0.063 apart (L 0.60 vs 0.72,
a/b ≤ 0.02), so the second patch was 99 % selected and the mask covered the
whole photo; Refine 85 and 100 kept most of it too. Now
`0.015 + 0.06·refine` (full ≤ 0.0225 / gone ≥ 0.045 at Refine 50, gone ≥
0.066 at 85, 0.075 at 100), computed once in
`lightcraft_pipeline::masks::color_range_tolerance` and used by both the CPU
evaluator and the GPU `mask.wgsl` kind-3 parameter so the two keep agreeing.

Test: two flat patches at the report's sRGB values, sampled as
`mask.sampleColor` does; the sampled patch stays fully selected and the other
one is out at Refine 0, 50 and 85 and mostly out at 100.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 10:52:40 +00:00
wstockandClaude Fable 5.1 ee0b0ef67e Issue #166: snapshot --script exits non-zero when a scripted request fails
`fn snapshot` wrote every reply but never looked at `ok`, so a script with a
bad `ui.set`, an unknown command or a failed `ui.screenshot` exited 0 while
the PNG showed the wrong view or was never written. The run now counts
replies with `"ok": false`, keeps going (every reply still reaches stdout and
the final `-o` screenshot is still written — continue-and-report, like
`run --keep-going`) and returns `N scripted request(s) failed` afterwards,
which `main` turns into exit 1. Usage text and docs/control-protocol.md
"Headless rendering" state the contract.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 10:46:29 +00:00
sebastian pfluegelmeier b4ca9f7439 M11.4: harden RAF validation and camera profile fallback 2026-10-08 11:23:55 +02:00
Marek HubařandClaude Opus 5.5 169a4ed059 LR-IMP-FORMATS: ORF colour-filter layout from the file's Exif CFAPattern
Split out of #240 without the compressed decoder (crates/raw/src/vendor/orfc.rs
is not part of this change; compressed ORF stays preview-only).

- The layout is read from Exif CFAPattern (0xa302, repeat counts in either
  byte order) when it describes a 2x2 Bayer cell. BGGR and GBRG were not
  reachable before.
- Without a usable tag the green diagonal is still found from the samples,
  now from totals of 32x32 blocks instead of single-pixel differences, so
  fine texture no longer outweighs a small red/blue difference. The old
  version indexed the samples directly; this one reads them with get().
- probe_info no longer unpacks 12-bit packed ORFs that state their layout
  (XZ-2: under 0.1 ms; it ran the same unpack as a decode before, which
  takes 11 ms). 16-bit files still read their samples to find the bit depth.

Tests: stated layouts in both byte orders, unusable tags falling back to the
samples, the texture case, hostile areas for the fallback, probe_info equal
to decode; a corpus test comparing the E-1, E-400 and XZ-2 mosaics with
their embedded JPEGs under every layout.

No user-visible change yet for the bodies whose layout the old guess got
wrong (E-M1 and E-620 are BGGR): they write compressed ORF.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:02:06 +02:00
sebastian pfluegelmeier cfcc169ff1 M11.4: calibrate Fujifilm colour with bundled X-H2S and X-T4 profiles 2026-10-08 10:49:55 +02:00
sebastian pfluegelmeier 5923be31d6 M11.1: decode Fujifilm lossless and lossy compressed RAF 2026-10-08 10:44:09 +02:00
wstockandClaude Fable 5.1 f7c0b9e49f Issue #182: library.select and id queries reject a photo id that is not in the library
`select_photos {"ids": [9999]}` answered `selected: 1` and left the session
with no usable active photo; `develop.get {"id": 9999}` answered too.
`library.select` now checks every id (and `active`) against the catalog
before touching the selection and returns `no such photo N`, so the
previous selection and active photo stay in place; `photo_arg` (develop.get,
photo.inspect, photo.allMetadata, history.list) does the same for an
explicit `id`. Engine and MCP tests cover every select mode.

Fixes #182

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 07:45:41 +00:00
wstockandClaude Fable 5.1 b7e37a388d Issue #180: ICC profiles carry a fixed header date, so exports are byte-identical
`moxcms` stamps the ICC header's creation date/time with the wall clock on
every `encode()` (the writer uses `ColorDateTime::now()` and ignores the
profile's `creation_date_time` field), so two exports of one photo with
the same settings differed in the embedded profile and nowhere else — the
pixels matched, the hashes never did. `write_matrix_trc`, which every
embedded profile goes through, now zeros the header's date bytes (24..36)
after encoding, as other deterministic profile writers do. A test asserts
two writes of every named space are identical, the date bytes are zero and
the profile ID is empty.

Fixes #180

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 07:45:39 +00:00
Mohammed Munzir 87dd1c201a Web: prune the thumbnail cache to the configured budget (#245)
The browser accepted and persisted library.preferences cacheMb, but stored thumbnails stayed over that limit: the worker index pruned with the native default (DISK_CACHE_BYTES), so the setting only reached the desktop cache. The worker group now carries the session cache_bytes(), the app refreshes it every frame, and the next stored thumbnail prunes to it.
2026-10-08 09:30:19 +03:00
MarcelandCodex 35dd973d91 M16: add German localization using the existing catalogs
Add German language selection and persisted locale support, complete German
label and format catalogs, and German release notes. Connect previously
untranslated UI text, preserve user names, and size translated buttons to
fit. Format cached import source facts in the active UI language.

Validate locale coverage, placeholder formats, language persistence,
German glyphs, button bounds and cached import language switching. Review
native and headless screenshots; run all seven cargo xtask ci gates.

Co-Authored-By: Codex (GPT-6) <noreply@openai.com>
2026-10-08 07:27:32 +02:00
Wondarkus 49868693d5 Add Russian language support in i18n.rs 2026-10-08 08:23:30 +04:00
Wondarkus 79fbb6b464 Add Russian language support in i18n.rs 2026-10-08 08:21:31 +04:00
Wondarkus 95b57d3786 Add Russian language option to menus 2026-10-08 08:20:17 +04:00
Wondarkus 782ade6c39 Add files via upload 2026-10-08 08:16:54 +04:00
Wondarkus 11d6676984 Delete crates/ui-egui/locales/ru-formats.json 2026-10-08 08:15:15 +04:00
Wondarkus 841850d516 Delete crates/ui-egui/locales/ru.json 2026-10-08 08:14:58 +04:00
radialmonster 15feb6e1fc LR-PROF-CAMERACOLOR: fit the camera look away from edges when a lens-corrected JPEG misaligns (#232)
Sony A7R II camera JPEGs are distortion-corrected by default while the raw is not. On the public
CC0 ILCE-7RM2 samples the JPEG is up to 15 px of 1440 off, and the misaligned window frames alone
pushed the held-out RMS to 0.108-0.115 (gate 0.10): the photo opened in the grey neutral fallback,
13 L* too dark. The same shot with distortion correction off fits at 0.059.

When the fit on all pixels fails the gates, try it once more, with the same gates, on pixels whose
3x3 neighbourhood in either proxy spans a luminance ratio of at most 3. Fits that already pass are
unchanged (29 other corpus ARW/NEF/RW2 files render pixel-identical). ILCE-7RM2 ΔE76 vs the camera
JPEG (registered) 16.6-16.9 -> 3.7-3.8; corpus GX80 19.1 -> 7.9. RENDER_CACHE_VERSION 11 -> 12.
2026-10-08 00:14:00 -04:00
radialmonster 03bc1469f9 LR-EDIT-OPTICS-PROFILE: apply the distortion correction Panasonic/Leica record in RW2/RWL (#256)
The RW2 reader reads PanasonicRaw tag 0x0119 (16 LE words) and expresses the
camera's own correction as the equivalent DNG WarpRectilinear in OpcodeList3,
so embedded_lens() turns it into the existing EmbeddedLens/EmbeddedWarp and
"Enable Profile Corrections" applies it like a DNG's warp (no new warp path;
Convert to DNG keeps it). Rule, from our own measurements against the
embedded camera JPEGs (clean-room; ExifTool tag names only):
corrected = rho*(1 + (w8 rho^2 + w4 rho^4 + w11 rho^6)/32768), rho = r/w12
(w12 = active-area half diagonal), output = corrected / (1 + w5/32768),
on when (w7 & 0xf) == 1. Chosen on 38 GX85 photos, held out on 35 photos
from 12 other bodies / 9 other lenses; corrected renders line up with the
camera JPEG within 0.75 px at 1440 px at the corners (up to 13 % of the
half diagonal before). Implausible values leave the photo uncorrected.

The loader now removes lens opcodes before the RW2 camera-look fit (its
binned proxy needs an empty OpcodeList3); Opcode::is_lens_correction()
replaces the duplicated matches. RENDER_CACHE_VERSION 11 -> 12.
2026-10-08 00:13:58 -04:00
Gardy 4e012a59a6 macOS menu: single-key shortcuts reach egui (E, C, ⇧P, ratings…)
On macOS every shortcut that appears in the menu bar was left to the native
menu, and the egui shortcut handler skipped those keys. AppKit only offers a
key press to the menu bar when it carries ⌘ or ⌃, or is a function key; a
plain key (E, C, H, M, I, K, D, 0–9, P, U, Y…) or one with only ⇧ / ⌥ (⇧P,
⇧V, ⌥Y…) goes to the window's winit view instead. So all of those did
nothing on macOS, while keys outside the menus (Space, G, X, ⌫) and ⌘
shortcuts worked.

`publish_shortcuts` now lists only the shortcuts the menu bar really receives
(`menu_delivers`: ⌘, ⌃ or a function key) in `native_shortcuts`. The others
keep their accelerator on the menu item, so the menu still shows the key, and
egui runs them.

Checked in a release build by pressing keys (not via the control channel,
which bypasses the native menu): C opens Crop & Rotate, E Edit, ⇧P Presets,
3 / 0 set and clear the rating; ⌘⇧H toggles the histogram once per press;
typing "c" in the search field types and leaves Crop closed. The same build
without this change ignored C, ⇧P and 3. ⌃ shortcuts (merges) are unchanged
and were not tried.

Regression test: native_menu::tests::single_key_shortcuts_stay_with_egui
(fails with the previous behaviour: "E must be left to egui").
2026-10-07 22:10:21 -06:00
Wondarkus 6c7405ed3e Fix Russian translations in ru.json 2026-10-08 07:06:23 +04:00
Daniel VoshartandClaude Opus 5.5 8dd3afce51 Fetch: validate caller-supplied names, sizes and hashes; FileSpec borrows its strings; download_url for an exact address
The fetch crate is about to serve downloads whose file names, sizes and addresses come from a user-editable list,
so it checks them before touching the disk: an ordinary, unique file name (no separators, drive or stream
syntax, control characters, trailing dots, Windows device names or .part suffix), a non-zero size within the
caller's bound, a well-formed SHA-256, bounded file, mirror and attempt counts, and URLs capped at 16 KiB.
FileSpec now borrows its strings (callers with static names are unchanged) so a name read from a file needs no
leaking. download_url fetches one file from an exact address, query string included, for signed links.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 22:59:22 -04:00
Daniel VoshartandClaude Sonnet 5.5 4b57d5dc5a Fetch: a bearer token is sent only when the caller names its variable
Options::token_env (default none) replaces the hard-coded LIGHTCRAFT_SAM3_TOKEN, so a download of one model can
never send another model's token to its host. SAM 3 names its variable through lightcraft_segment::fetch::options().
The decision (non-empty, https, the host the download began at) is a small function with its own test.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 22:59:01 -04:00
Wondarkus 6a34a1f8b8 Add files for further editing 2026-10-08 06:56:08 +04:00
radialmonsterandClaude Opus 5.5 fc4b054fa4 LR-IMP-FORMATS: RW2 format 8: invalidate cached thumbnails
The decoder change alters format-8 renders while the photo's content hash
and settings stay the same, so thumbnails rendered before it (edited
photos keep theirs on disk) would survive. Bump RENDER_CACHE_VERSION.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 22:52:45 -04:00
Hanmo d614dc72bd M3: isolate global Tint direction correction for compatibility review 2026-10-08 10:26:18 +08:00
Hanmo 024e983a7e M3: honour Nikon CropArea and embedded preview colour space 2026-10-08 10:16:44 +08:00
Brandon ThomasandClaude Opus 5.5 629e39380e Release CI: macOS job on macos-15-xlarge
Custom macOS larger runners in a runner group do not pick up jobs; the
built-in M2 Pro label does. Only release.yml uses it, and the
actions_macos_xl budget caps the monthly spend.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 22:07:59 -04:00
radialmonsterandClaude Opus 5.5 1ba0421a85 LR-IMP-FORMATS: RW2 format 8: decode each 2×2 cell column by column
Raw format 8 codes a cell top-left, bottom-left, top-right, bottom-right.
decode_strip placed the values in row order, which put the two green
samples of every RGGB cell on each other's sites: flat areas were right,
but every edge got a one-pixel colour zipper. Measured on a G9 II and the
corpus GH6, S5 II and S9: the Gb plane sat at (+0.5, -0.5) half-res cells
from Gr instead of the Bayer (-0.5, +0.5); with this change all four match.

The test encoder used the same order, so the round trip passed either way;
a new known-answer test hand-codes one cell and checks where it lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:58:09 -04:00
Brandon ThomasandClaude Opus 5.5 8734e95dc8 Release CI: macOS job on the org macos-release runner
The standard macOS runners are shared with all CI and queue for an hour or
more. The macos-release runner (GitHub-hosted M2 Pro) is in a runner group
usable only by release.yml on the release branch, with a monthly spend cap
on the actions_macos_xl SKU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:54:56 -04:00
Brandon ThomasandClaude Opus 5.5 4159ceb1b0 About: contributor and model credits compiled in (grab bag + table, self-submitted names) (#252)
* About: contributors compiled in from contributors.json (Contributors + Models tabs, grab bag + table, sortable, name toggle)

Adds the craftrules standard (standards/contributors.md) to LightCraft: contributors/contributors.json
is generated by craftrules/scripts/contributors.py and baked into the binary by
crates/ui-egui/build.rs (merged next to the tr_format! generation) as static tables. The About
dialog gets tabs About · Contributors · Models. Each contributor line: @username, opt-in
display/real name, PRs, commits, lines +/−/Δ, binary assets +/−, first/last commit. Sortable by
every column; alphabetical ignores case and the leading @. Tabs and the list view are
control-channel widgets (button:aboutTab-*, button:creditsTable / button:creditsGrabBag).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors docs: craftrules is two levels up (../../craftrules)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors: self-submitted names in contributors/people.toml; AGENTS.md credit-yourself section

build.rs reads contributors/people.toml next to the generated contributors.json and shows a name only
for entries the generator verified as written by that person (craftrules standards/contributors.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors: regenerate (verified people.toml entries)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors: regenerate after rebase onto main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors: regenerate (verified people.toml entries)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:22:08 -04:00
Brandon Thomas 2694d7089f Merge pull request #249 from radialmonster/ci-available-ram 2026-10-07 21:06:13 -04:00
Brandon Thomas 88f38c7e2a Merge pull request #248 from andrewyao (web_time in raw preview decode timing) 2026-10-07 21:06:13 -04:00
Brandon Thomas df574a537a Merge PR #213 from Orthic-Labs/orthic/arw-lossless-ycbcr
# Conflicts:
#	crates/ui-egui/src/headless.rs
2026-10-07 20:53:51 -04:00
Brandon Thomas 5bc923b4f9 Merge PR #246 2026-10-07 20:53:29 -04:00
Brandon ThomasandClaude Opus 5.5 71c8382b10 Merge main into #213 (keep main's Windows test fixes from #207; parity row combines RW2 and the downsized ARW)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:53:25 -04:00
Andrew YaoandClaude Opus 5.5 0a952b3536 Web: time raw preview decodes with web_time so wasm doesn't panic
load_bytes_now timed the binned demosaic with std::time::Instant, which
panics on wasm32-unknown-unknown ("time not implemented on this
platform"). Opening a raw in the web build aborted the decode. Use
web_time::Instant, as library.rs already does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 17:53:19 -07:00
radialmonsterandClaude Opus 5.5 f9cfe15db6 CI: pick build jobs from available RAM, not installed RAM
Installed RAM overcommits a busy machine: on a 32 GB desktop with browsers,
a VM and editors open, about 7 GB was free and the installed-RAM rule chose
16 jobs. A fresh build then pushed free RAM below 1 GB.

`cargo xtask ci` now uses one build job per 1.5 GB of RAM available when it
starts (Linux MemAvailable, Windows AvailableMBytes, macOS free + inactive +
speculative pages), capped at the CPU count. With line-table debuginfo,
4 jobs measured about 3 GB in use, so this leaves about half the available
RAM spare. If available RAM can't be read, half the installed RAM is
assumed; if neither can, 4 jobs. Test threads follow, capped at 4.
Explicit CARGO_BUILD_JOBS / RUST_TEST_THREADS still win.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:46:10 -04:00
radialmonsterandClaude Opus 5.5 ed0a2a4f07 GPU tests: drop the edge-pixel tolerance; the coverage mask makes edges match
#207 let up to two edge pixels differ because the GPU's f32 framing could
fall on the other side of the lens-warp boundary. 'GPU: decide warped-geometry
edges with a CPU coverage mask' (this PR) makes the GPU use the CPU's f64
decision, so every case is back within the plain MAX_LSB check (embedded
lens + perspective: max 1 LSB, was 91 before the mask).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:40:47 -04:00
Brandon ThomasandClaude Opus 5.5 0801219b17 Merge #243 (with main) into #246
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:38:46 -04:00
Brandon Thomas e716ede8e4 Merge remote-tracking branch 'origin/main' into HEAD 2026-10-07 20:33:23 -04:00
Brandon ThomasandClaude Opus 5.5 2e48f25997 Merge main into #243 (combine with #207's Windows test fixes)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:33:23 -04:00
Brandon ThomasandClaude Opus 5.5 d716d74af5 Merge main into #243 (combine with #207's Windows test fixes: hide_home_above plus local.addRoot, FakeFs path normalising plus case folding)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:32:47 -04:00
Brandon ThomasandClaude Opus 5.5 a1c5e2dad1 Tests: three headless UI tests wait for the textures and dialog they check
settle() returns after a quiet spell, which on a loaded machine can come before the first grid
thumbnails, the profile browser's variant thumbnails or the async import review (and its
thumbnails) land. demo_grid_snapshot_has_ui_pixels, profile_browser_previews_on_hover_and_applies_on_click
and import_review_dialog failed that way when two test runs shared the machine; they (and the
two other import-review tests) now step until what they assert is there. 2 concurrent runs: 138/138 each.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:32:34 -04:00
radialmonsterandClaude Opus 5.5 c816ca8e35 UI tests: wait for sidebar rows with main's step_until
main gained Headless::step_until for the same job as this branch's
test-only wait_for_widget (run frames until a condition holds). Use the one
helper and drop the duplicate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:23:25 -04:00
radialmonsterandClaude Opus 5.5 62325bd45e Thumbnails: retire the replaced preview cache so its old jobs can't write
Reopening a library (or opening another one, or set_disk_cache_bytes
falling back to attaching) replaced MediaCache::rendered with a new
PreviewCache, but render jobs still in flight kept the old object.
Each PreviewCache checked only its own generation, so after a clear on
the new cache the old one's generation was still current: a finishing
thumbnail job wrote its pixels through it into the directory the clear
had just emptied. The UI rejected the result, but the JPEG stayed.

attach_disk_cache now retires the cache it replaces before swapping it:
PreviewCache::retire takes the generation write lock (so it waits for a
write in progress, like clear) and marks the object retired; every
get/put/put_deferred through it is then refused and its clear() is a
no-op, as its directory may now belong to the replacement. Files on
disk are left alone, so a plain reopen still finds its thumbnails, and
resizing the budget in place keeps the same cache object as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:23:17 -04:00
radialmonsterandClaude Opus 5.5 4d5504cf16 Web: watch the thumbnail cache from worker start so an early clear invalidates
The main thread's Workers tracked no rendered-thumbnail cache until the
first render request carried one. If Clear Preview Cache ran before that
(e.g. starting on an empty filtered view), the per-frame observation in
finished() had nothing to compare with, and the first request only
recorded the already-cleared generation as its baseline. The persisted
ThumbIndex was never invalidated, so once the filter was removed the
request touched the old entry and the worker read the pre-clear JPEG
back from browser storage. Replacing the cache (another library) before
the first request had the same gap.

Workers::start now takes the session's cache and records it, with its
current generation, as the baseline the persisted index belongs to. A
normal start with no clear observes the same cache and generation and
keeps the index. The comparison moves into wire::CacheWatch so it is
tested natively (cargo test --workspace) next to ThumbIndex.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:23:17 -04:00
Brandon Thomas aec820033b Merge pull request #206 from anarion80/add_nix_build
Packaging: add a Nix flake (nix build / nix develop)
2026-10-07 20:15:50 -04:00
Brandon Thomas b90c06b347 Merge pull request #210 from voshart/pr/fetch-crate
Fetch: the model downloader becomes its own crate (no behaviour change)
2026-10-07 20:15:46 -04:00
Brandon Thomas 52f64ce0b9 Merge pull request #215 from XRTce/rw2-panasonic-lumix
LR-IMP-FORMATS: Panasonic RW2 / Leica RWL / Panasonic RAW in every raw format, with the RW2 camera look
2026-10-07 20:15:41 -04:00
Brandon Thomas 767ed15adc Merge pull request #235 from Gremiger/fix/issue-212-nef-dng-white-balance
Issue #212: Raw: keep vendor white balance when writing DNG
2026-10-07 20:15:37 -04:00
Brandon Thomas 1c025fbb97 Merge pull request #207 from voshart/pr/tests-windows
Tests: pass on Windows (path separators, temp folder inside home, one GPU edge pixel)
2026-10-07 20:15:33 -04:00
Brandon ThomasandClaude Opus 5.5 9b41e33a24 Merge main into #213 (keep main's waits in the two flaky UI tests)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:14:41 -04:00
Brandon ThomasandClaude Opus 5.5 0143704c7d Merge #243 (with main) into #246
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:14:28 -04:00
Brandon ThomasandClaude Opus 5.5 9da8163ae2 Merge main into #243 (keep main's settle and step_until in the two flaky UI tests)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:12:10 -04:00
Brandon ThomasandClaude Opus 5.5 46f5d212e7 Merge main into #207 (keep main's step_until wait for the folder row)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:11:00 -04:00
Brandon Thomas eba0b9723b Merge pull request #223 from Ndsanes/l10n/zh-hans-catalog-gap
LR-BEHAV-LOCALIZE: fill the zh-hans catalog to match zh-hant
2026-10-07 20:09:36 -04:00
Brandon Thomas 5372309ad8 Merge pull request #228 from RicSchonfelder/i18n-pt-br
i18n: add Brazilian Portuguese (pt-br) UI translation
2026-10-07 20:09:32 -04:00
Brandon Thomas b63ea8f5c2 Merge pull request #222 from Ndsanes/fix/craft-fonts-relative-path
Build: resolve a relative CRAFT_FONTS_DIR against the workspace root
2026-10-07 20:09:28 -04:00
Brandon Thomas 2dda7cf842 Merge pull request #241 from agorevski/copilot/issue-cluster-docs-consistency
Docs: fix release, sibling, and command references
2026-10-07 20:09:24 -04:00
Brandon Thomas be87e1b6e7 Merge pull request #238 from DWAA1660/fix/export-missing-folder
Export: clear error when there's no export folder (instead of "Access is denied")
2026-10-07 20:09:19 -04:00
Brandon Thomas 97e34434d0 Merge pull request #217 from nledford/feat/random-sort
Library: Random sort with a stable seed and Reshuffle
2026-10-07 20:09:14 -04:00
Brandon Thomas 30192541ea Merge pull request #242 from nledford/feat/collapsible-sidebar-sections
Sidebar: collapsible Albums, Local, By Date and Keywords sections
2026-10-07 20:09:10 -04:00
Brandon Thomas c4d7c00e74 Merge pull request #225 from nledford/fix/stale-panels-after-library-switch
Library: switching libraries no longer shows the old library's sidebar
2026-10-07 20:09:06 -04:00
Brandon Thomas 3fe0bef364 Merge pull request #239 from nledford/feat/smart-album-file-path
Smart albums: File Path rule field
2026-10-07 20:09:03 -04:00
Brandon Thomas 590b387a53 Merge pull request #221 from MusseJusse/codex/navigation-arrows-pr
M16.1: complete navigation arrowheads
2026-10-07 20:08:59 -04:00
Brandon Thomas 51c878ddd8 Merge pull request #199 from storytold/fix/flaky-profile-hover-test
Tests: fix three FreeBSD-flaky headless UI tests
2026-10-07 20:08:55 -04:00
Brandon ThomasandClaude Opus 5.5 034bb6e324 Rename: drop a redundant from != to (clippy nonminimal_bool)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:08:52 -04:00
Brandon ThomasandClaude Opus 5.5 41f9ae0c2b Loupe: a draft that finishes after a newer request still replaces older pixels, so a long drag keeps updating
#243 dropped every result whose request id wasn't the newest for its slot. The loupe requests a new
draft every frame while a slider moves, so when a draft took longer than a frame nothing landed
until the drag paused. Main, Before and Hover now take a superseded draft that is newer than what
they show (never one from before a library or preview-cache reset, never an error or its source).
Also saturate the xtask RAM rounding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:06:58 -04:00
Brandon ThomasandClaude Opus 5.5 e2a941887e Nix: mark the package community-maintained in the agent guide; craft-fonts holds CJK fonts, not only Japanese
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:03:05 -04:00
Brandon ThomasandClaude Opus 5.5 e574839724 Merge main into #213 (camera_preview: main's proxy edge/size with the sensor_proxy fallback)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:01:23 -04:00
Brandon ThomasandClaude Opus 5.5 f0632e7022 Fetch: crate root forbids unsafe and denies panicking lints, like every other crate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:56:42 -04:00
Brandon ThomasandClaude Opus 5.5 d3c6dc0aa8 i18n zh-hans: "Add Remove Spot" means adding a spot removal
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:56:03 -04:00
Brandon ThomasandClaude Opus 5.5 ce165dfa4a Export: calmer wording for the missing-folder error
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:56:00 -04:00
radialmonsterandClaude Opus 5.5 1237e198d5 GPU: build the warp coverage mask per block with interval bounds
The coverage mask from 94fe300 evaluated the f64 warp for every output
pixel. coverage_mask now asks Warp::block_coverage for each 32 x 16 block
(the shared warp formulas evaluated with outward-rounded intervals, no
edge margin) and evaluates only undecided blocks near the edge per pixel
with Warp::covers; the bits are unchanged
(coverage_mask_matches_per_pixel_decision, incl. partial words and
padding bits; coverage_mask_with_the_horizon_in_view, a perspective
denominator exactly 0 inside the image, so the clamp and
undecided-division paths go through block_coverage and the mask).

Measured (release, synthetic source, embedded lens warp + CA +
perspective as in embedded_lens_perspective_edges_match):
6000x4000 ~30 ms -> 2.7 ms (32 threads, 11x), ~580 ms -> 24.5 ms
(1 thread, 24x); 2048x1365 3.6 ms -> 1.0 ms (3.5x), 67.5 ms -> ~5 ms
(12-14x). Rotate270 the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:29:29 -04:00
radialmonsterandClaude Opus 5.5 2b2ebfc7ca Geom: write the warp formulas once over a Real scalar (f64 or Interval)
Homography::apply, Affine::apply, Warp::to_corrected,
Warp::corrected_to_source and embedded_warp are now generic over
lightcraft_geom::Real (private *_real variants in pipeline, public
apply_real in geom); the f64 API is a thin wrapper, so callers are
unchanged. Real is implemented for f64 (plain IEEE operators) and for
Interval, which rounds every bound one ulp outward and turns division by
a range containing zero, the homography's 1e-300 denominator clamp and
non-finite bounds into "undecided" (NaN). Because the interval runs the
same operations as the f64 path, it bounds the rounded f64 results, not
only the exact ones.

The f64 path keeps the same operations in the same order (only commuted
operands, which IEEE + and x leave bit-identical; no fma). Proof:
warp_f64_results_are_bit_identical_to_the_pre_generic_formulas hashes the
bits of 2000 random warps x 40 points through every formula (plus the
clamp branch, including +-1e-300 exactly) and matches the digest computed
by the same test on thumb-split (419261c) before the change; reassociating
one expression, or `<=` for `<` in the clamp, changes it.

The framing chain (output px -> transformed -> lens-corrected -> green
source, then inside the edges) is written once: Warp::framing, generic.
Warp::frame (f64, returns the corrected and green source points) serves
both the CPU resample (sample_warped) and Warp::covers; block_coverage
evaluates the same chain with intervals over a whole block of output
pixels and compares the bounds against the same edges() inside uses, with
no margin: the bounds enclose every pixel's f64 result, so the decision
equals the per-pixel one; else None.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:29:29 -04:00
radialmonsterandClaude Opus 5.5 6b6e756830 CI: pick build jobs from installed RAM; cap test threads at 4
The fixed four jobs were safe on 8 GB but too many for 4 GB, and slower than
needed on large machines. `cargo xtask ci` now uses one build job per 2 GB of
installed RAM, capped at the CPU count, and 4 if the RAM can't be read
(/proc/meminfo, Win32_ComputerSystem, hw.memsize). With line-table debuginfo,
4 jobs measured about 3 GB of RAM in use and 2 jobs about 1.5 GB.

Test threads follow but stop at 4: more barely shortens CI, and at 16 threads
on a loaded machine the wall-clock frame-budget test
offline_originals_never_block_a_frame failed (one 303 ms frame). Explicit
CARGO_BUILD_JOBS / RUST_TEST_THREADS still win.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:27:20 -04:00
radialmonsterandClaude Opus 5.5 e9931d6871 UI tests: wait for the sidebar's file-system checks before clicking
keyword_list_filters_renames_and_suggests failed intermittently under
full-suite load (1 photo instead of 2 after clicking travel|italy, or no
travel|italy row at all). No metadata was lost: at the failure every
photo still had its keywords, but the filter was `travel|france`.

Local's rows (Downloads, Home, ...) are added only once their is_dir
checks come back from fs_cached worker threads. Those start when the left
panel is first drawn, which in this test is right before the clicks.
Headless::busy() did not count them. Under load a row could land between
the frame that read a widget's rect and the frame that delivered the
press. The keyword rows then moved down 29 px and the click hit the row
above (travel|france), or the toggle click missed travel. Diagnostics
showed the click aimed at y=1307 and travel|italy drawn at 1365 after
the two Local rows landed. The same failure reproduces on 5cee0c0 (2 of
20 runs), so the thumbnail work did not cause it.

fs_cached now keeps a per-context count of checks in flight
(fs_cached_running), and busy() includes it, so settle() waits for them.
The keyword test settles after opening the left panel. The new test
settle_waits_for_file_system_checks holds a check open and fails without
the busy() change.

Full ui-egui lib runs with 4 test threads: 2 failures in 25 before the
change, none in the runs after it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:04:03 -04:00
radialmonsterandClaude Opus 5.5 b4dc8c10ee Thumbnails: don't rebuild raw stand-in jobs every frame
An unedited raw first shows its embedded preview (ThumbQuick), then the
grid queues the real thumbnail at BACKGROUND_THUMB_PRIORITY. The grid asks
at its own priority (10 on screen), so thumb_current found no match while
the stand-in ran (nothing pending for the real slot) and while the real job
was queued (pending at 3, asked at 10): request_thumb rebuilt the job every
frame without changing anything. thumb_inputs now records whether the
request went through a stand-in; such a request is current while its
stand-in is pending or its real job is pending/rendered at the background
priority. Direct requests keep their priority check, so a higher priority
still promotes a queued job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:04:03 -04:00
radialmonsterandClaude Opus 5.5 73cf0d50bb Thumbnails: keep textures when the disk cache is resized
Changing the disk cache size (library.preferences cacheMb) re-attached a
new PreviewCache object, and the renderer treats a new cache object as a
library reset: every texture, the loupe included, was dropped and
re-rendered. Thumbnails are keyed by content, so a new budget leaves them
valid. The budget now changes in place on the attached cache (same object,
same generation); the next write trims to it, as before. A preview clear,
a generation bump and a new library still invalidate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:04:03 -04:00
radialmonsterandClaude Opus 5.5 c7f5664ce2 UI: don't reveal a hidden Local folder inside Home's tree
Hiding a kept Local folder beneath Home while browsing it (Remove from
Local, local.hide) expanded Home down to that folder and pushed the
"Show hidden locations" row out of view. local_places picked the
innermost listed folder containing the browsed one as the tree to
reveal; once the kept row was hidden that became Home, so folder_tree
opened Home, AppData, Local, Temp, ... to show the hidden folder again.

A tree no longer opens on the way down through a hidden folder: the
owner is chosen only among places whose path to the browsed folder
passes no hidden one. Hiding Home itself still lets Pictures reveal a
folder browsed inside it.

local_location_can_be_hidden_and_restored again hides a browsed
temporary folder (beneath Home on Windows), kept as Browse Folder...
does, while it is still browsed, and checks the restore row is inside
the viewport before clicking it. It failed before this fix. New unit
test: hidden_folder_is_not_revealed_in_an_outer_tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:04:03 -04:00
dbc05bedc7 Bench: thumbnail request example
thumb_request_bench measures UI-thread time and allocations for grid
thumbnail requests (with and without pending work), using the same counting
allocator as the other bench examples. REAL_PHOTOS=1 samples up to four
photos from PHOTO_ROOT, which must now be set explicitly; the original
defaulted to a local photo library path.

Split from a327b98 (T0 thumbnail work).

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:04:03 -04:00
f2f7a1cb26 T0: optimize thumbnail requests and reject obsolete preview work
Skip unchanged thumbnail jobs and reuse bounded settings hashes (both maps
capped at 1024 entries). Reject stale render results and cache writes after
edits, library changes and preview clears: render requests carry a request
id and cache generation, PreviewCache writes and reads hold the generation
lock that clear() takes, and library.clearPreviews also drops cached
sources. The web worker index follows the same generation rule.

The decoder guards, Windows path fixes and test fixture changes that were
bundled in a327b98 are separate commits; the benchmark follows this one.

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:04:03 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 7f3d9ce25f Sidebar: collapsible Albums, Local, By Date and Keywords sections
Each header folds its section on click, with a chevron after the title
showing the state. The choice is kept in the UI state across restarts.
The Albums "+" button keeps working inside its header.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 18:56:42 -04:00
f73aedcc94 UI tests: wait for folder listings and keep sidebar fixtures off Home
Folder listings arrive from background threads that Headless::busy() does
not track, so a fixed number of frames was not always enough under
full-suite load. Two tests failed intermittently and passed alone:
kept_local_root_stays_while_browsing_below_and_elsewhere and
local_folder_tree_expands_and_browses. They now wait for the expected
widget (wait_for_widget, test-only) and use their temporary folders as kept
roots. The kept-root test hides Home (in the test's in-memory session) so
the tree does not depend on the user's real folders.

local_location_can_be_hidden_and_restored now hides a kept root, reached
via "All Photographs". It no longer covers hiding a browsed temporary
folder: hiding a folder beneath Home expands the whole parent tree and
pushes the restore button out of view. That UI bug is not fixed here.

Fixture changes split from a327b98 (T0 thumbnail work), plus follow-ups.

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:50:00 -04:00
928c2c974f Windows: treat separator-only path differences as the same file
Catalog paths may mix `/` and `\` on Windows, so raw string comparison saw
`N:/photos/IMG.jpg` and `N:/photos\IMG.jpg` as different files:

- Rename: planning, the move list, move_file_with and file_moves now compare
  as paths. A separator-only "rename" is a no-op: no file move, no catalog
  change and no undo entry, alone or batched with a genuine rename.
  Previously it attempted a plain rename onto the existing file, failed, and
  rolled back the rest of the batch. Case-only renames still work (#95).
- Import: the already-in-library index is keyed by PathBuf.
- Tests build expected paths with Path::join, and the CLI test escapes the
  device path with serde_json instead of string formatting.

Split from a327b98 (T0 thumbnail work), with the follow-up rename fix that
the committed version needed.

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:50:00 -04:00
ee9a989834 Codecs: bound thumbnail fallback decodes and WebP metadata allocations
Non-JPEG thumbnail fallbacks decode at source resolution before resizing, so
the output size does not bound memory. ThumbnailOptions gains max_pixels
(default 64 MP, previously about 1 GP via DecodeOptions::fit); larger sources
return Error::TooLarge. Nothing in the workspace calls decode_thumbnail yet.

WebP RIFF chunk lengths are untrusted: set_memory_limit(bytes.len()) stops a
tiny truncated file requesting gigabytes for metadata. In image-webp 0.2.4
the limit applies only to ICC, EXIF and XMP chunks. Fuzz tests cover both;
the roundtrip test only switches to struct update syntax.

Split from a327b98 (T0 thumbnail work), where it was bundled.

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:49:59 -04:00
radialmonsterandClaude Opus 5.5 94fe30015c GPU: decide warped-geometry edges with a CPU coverage mask
The lens + perspective warp classified source-edge pixels in f32 on the GPU.
A position 0.000035 px outside the edge rounded inside, so the GPU sampled
photo pixels where the CPU reference draws blank canvas (geometry_variants,
"embedded lens + perspective", max 91 LSB). The CPU now evaluates the f64
framing decision into a one-bit-per-pixel mask; sampling and colour stay on
the GPU. The embedded-lens cases move to their own test,
embedded_lens_perspective_edges_match, with the same tolerances.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:49:59 -04:00
radialmonsterandClaude Opus 5.5 e99ce5a578 CI: default to line-table debuginfo, four jobs and four test threads
A full-debuginfo workspace test build with one rustc/link.exe per CPU peaked
at 14-20 GB commit on a 32-thread Windows machine before any test ran; the
test binaries themselves peak at about 3.3 GB each. `cargo xtask ci` now sets
CARGO_PROFILE_DEV_DEBUG=line-tables-only, CARGO_BUILD_JOBS=4 and
RUST_TEST_THREADS=4 unless they are already set. Ordinary cargo commands are
unchanged; GPU coverage is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:49:59 -04:00
Alex GorevskiandCopilot 6aebd48eec Docs: fix release and command references
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 73897efc-ca9e-48b7-8b5a-50be0a930b09
2026-10-07 15:18:44 -07:00
Nathaniel LedfordandClaude Sonnet 5.5 030ccf9782 Smart albums: File Path rule field
Adds a "File Path" text rule so a smart album can match on where a photo
lives, e.g. contains "/Aliah Ira Polanco-Grylls/". "contains" /
"doesn't contain" match the whole string (spaces included) rather than
word by word; the other text operators are the shared ones. `\` and `/`
both separate and matching ignores case, so catalogs moved between
platforms still match. Demo photos have no path.

The rule editor, CLI and MCP pick the field up from FIELDS. parity.md's
smart-collection rows now list it (and the stale field count is fixed).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 18:11:13 -04:00
DWAA1660andClaude Opus 5.5 6b0274120b Export: say so when there's no export folder instead of failing with Access denied
On Windows HOME is usually unset, so the default export folder came out
empty and photos were written by bare filename into the working directory,
failing with 'Access is denied. (os error 5)'.

- Default export folder uses %USERPROFILE%\Pictures\LightCraft Exports on
  Windows (matching the default library location).
- A blank Folder in the Export dialog, or no folder resolvable at all, now
  fails with 'You need to specify an export location!' before any write.
- The Export dialog stays open on an error (like Import) and shows it;
  the menu and keyboard export paths show it too instead of doing nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PNY4WojPYorWa6rQLxSYHa
2026-10-07 16:08:01 -06:00
Gremiger 855d3a7959 Issue #212: Raw: keep vendor white balance when writing DNG
NEF (and other vendor raws without a colour matrix) carry their as-shot
white balance as vendor multipliers, not as a DNG AsShotNeutral. The DNG
writer only copied AsShotNeutral/AsShotWhiteXY, so Convert to DNG and
Copy as DNG produced files with no white balance at all, which develop
dark and green. Write the reciprocal of the vendor multipliers as
AsShotNeutral when the source has no DNG white of its own.
2026-10-07 21:43:23 +00:00
RicSchonfelderandDeepSeek V4 Flash fb01966a00 i18n: add Brazilian Portuguese (pt-br) UI translation
Add pt-br as a fifth interface language beside English, Simplified
Chinese, Traditional Chinese (Taiwan) and Japanese.

`crates/ui-egui/locales/pt-br.json` and `pt-br-formats.json` carry the
messages and the format strings; the language table gains the entry
(`PtBr, "pt-br", "Português (Brasil)", "Latn", ...`), and Edit > Language
plus `app.language.portuguese` select it. Locale tags pt-BR, pt_BR.UTF-8
and pt all normalize to the same language.

Translation follows pt-BR UI conventions (Adobe terms: revelação,
balanço de branco, máscara, cópia virtual…); proper nouns and file
formats stay as written (LightCraft, ArtCraft, GitHub, DNG, TIFF, XMP).
Placeholders, format specs and punctuation match the English source; the
English plural suffix is consumed with `{:.0}` where Portuguese does not
need it.

Tests: every_language_has_a_catalog_covering_the_source,
catalogs_agree_on_placeholders_and_report_gaps, locale_tags_normalize
and the painted-window test now cover pt-br. Docs: localization.md, a
new localization-pt-br.md and the parity rows.

Co-Authored-By: DeepSeek V4 Flash (opencode) <noreply@opencode.ai>
2026-10-07 17:03:20 -03:00
Nathaniel LedfordandClaude Sonnet 5.5 520d7b8616 Library: switching libraries no longer shows the old library's sidebar
A catalog loaded from disk counts revisions from 0, like a new empty one, and the sidebar
counts, keyword tree, date groups and grid list are cached by revision alone. Opening an empty
library after one that was never edited this session therefore kept the old library's numbers.
Each catalog loaded in a process now gets its own revision range.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 15:40:18 -04:00
NDsans a15bd574f9 LR-BEHAV-LOCALIZE: fill the zh-hans catalog to match zh-hant
zh-hant carries 1246 messages; zh-hans had 1029, so 217 recently added
strings — mask component editing, point colour samples, spot removal,
colour-label sets, smart-preview status, memory budget, the CLI/MCP command
list and the eighteen stock presets — fell back to English in the
Simplified Chinese UI.

Translate them and order the file like zh-hant so the two catalogs can be
compared line by line.
2026-10-08 03:08:33 +08:00
NDsans b040f9bb8f Build: resolve a relative CRAFT_FONTS_DIR against the workspace root
A build script's cwd is its package root, so CRAFT_FONTS_DIR=../craft-fonts —
the spelling the README and craft-fonts' integration docs both use, written
relative to the workspace root — resolved to crates/craft-fonts and failed.

The failure is silent from the user's side: the app runs, the language menu
lists every language, and selecting Chinese or Japanese paints boxes. Warn
when no font ends up embedded, not only when the variable is set but wrong.
2026-10-08 03:08:28 +08:00
MusseJusse 036993f194 M16.1: complete navigation arrowheads
Made with gpt-6.1-sol using the Codex harness in T3 Code.
2026-10-07 19:55:15 +01:00
Brandon ThomasandClaude Opus 5.5 7aab3fd97f Rename PrintCraft to PdfCraft (storytold/pdfcraft) (#219)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:45:42 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 6de619ca9b Random sort: choosing Random starts a fresh shuffle
Switching to Random from another key (without an explicit seed) derives a new seed
instead of reusing the one left behind, which was the fixed seed-0 order on a new library.
Re-choosing Random or changing only the direction keeps the shuffle on screen.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 14:08:11 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 89446df6a8 Random sort: review fixes (null seed, 2^53 seed bound, real tests), README
- library.shuffle treats a null seed as none given
- seeds are limited to 0..2^53-1 so JSON-double clients read them back exactly
- album test now fails if manual order overrides Random; persistence test goes through
  close/reopen of the library instead of a serde round trip
- README lists random among the sort options

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 14:03:52 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 b310de1bd6 Menus: no direction check while sorting at random
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:46:24 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 17daca8a87 Menus: list Reshuffle once in the Sort submenu
library.shuffle's registry menu path made the menu builder append it a second time after
the hand-built entry. Hide it from the auto-listing; test that it appears once and is
enabled only while sorting at random.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:42:42 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 a72118ec6e Library: Random sort with a stable seed and Reshuffle
SortKey::Random orders photos by a splitmix64 rank of (Sort.seed, id): same seed gives
the same order, and added/edited photos never reshuffle the others. library.sort takes
an optional seed; library.shuffle picks a new one (below 2^53 for JSON clients). Menus
gain Random and Reshuffle; Ascending/Descending are disabled for Random. No effect in
Recently Added (always newest import first).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:33:38 -04:00
𝐀𝖓𝖆𝖗𝖎𝖔𝐍 c58128c36b Merge branch 'main' into add_nix_build 2026-10-07 19:32:10 +02:00
Orthic LabsandCodex 94780ed299 LR-IMP-FORMATS: decode downsized Sony lossless ARWs
Co-authored-by: Codex <noreply@openai.com>
2026-10-07 22:36:00 +05:30
Claude cabe5b7399 LR-PROF-CAMERACOLOR: Panasonic RW2 file-local camera look
RW2 / RWL / RAW join ARW and NEF: the starting look (matrix, hue/saturation table, tone and chroma curves) is
fitted to the file's own embedded JPEG behind the same gates, with white balance relative to the as-shot look.
Panasonic previews always show the whole active area while the default crop is the in-camera aspect ratio, so when
the preview matches the active area instead of the crop, the reference is cropped to the default crop before
fitting (corpus_rw2_with_an_in_camera_crop_gets_a_camera_look).

On 174 CC0 samples with a preview 140 fits are accepted (93 of the 114 bodies with a preview), median ΔE against
the camera JPEG 5.0 with matching lightness; most of the others are compacts and kit zooms whose camera JPEG is
distortion-corrected (tag 0x0119 isn't used yet). `lightcraft-cli calibrate` keeps collecting ARW and NEF only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QtQmZ6peZXGn3wiqeowV2v
2026-10-07 16:36:06 +00:00
Daniel VoshartandClaude Sonnet 5.5 7fa7506db4 Fetch: the model downloader is its own crate, lightcraft-fetch
The downloader in lightcraft-segment (mirrors, resume, pinned SHA-256, pure-Rust
HTTPS) was already generic apart from four SAM 3 constants. Move it to
crates/fetch (L0, native only) so other model features can use it without
depending on the SAM 3 inference code. lightcraft-segment keeps SAM3_FILES,
DEFAULT_MIRRORS, MIRRORS_ENV and wraps mirrors(); no behaviour change. The
downloader's 13 tests moved with it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 12:07:24 -04:00
Claude 6bd64e4ad4 LR-IMP-FORMATS: Panasonic RW2 / Leica RWL / Panasonic RAW in every raw format
Decode every raw encoding Panasonic and Leica bodies write, from the DMC-LX1 (2005) to the DC-S1RM2 (2025):
compressed formats 4 (most bodies, 2008-2022) and 6 (S1 / S1R / S1H / S5, GH5M2), the prefix-coded strips of
format 8 (GH6 and later), packed formats 2, 5 and 7, and the 16-bit words of the 2005-2007 .RAW files. Until now
only format 5 decoded; every other Panasonic file opened as preview only.

Clean-room: no raw-decoder source was read. Tag names come from tag documentation; every encoding was recovered
by black-box analysis of 178 CC0 files from raw.pixls.us (118 bodies), and the module docs note the measurement
that settled each rule. Format-8 tags of unknown meaning must carry the values every sample has, otherwise the file
is reported as unsupported rather than guessed at.

Levels and geometry: formats 4 and 6 store samples 15 above the recorded black level (files without one use 15);
white is the active area's saturation plateau (the linearity limits are wrong on extended low-ISO shots, the GF1
and the Digilux 3); the in-camera aspect ratio is the default crop; mapped-out defect pixels are filled from
same-colour neighbours; the CFA pattern applies at sample (0, 0).

.rwl and .raw files are imported (engine, MCP folder expansion, file dialog, Linux MIME types). One file per
encoding joins the never-panic proptest; 11 CC0 files join `cargo xtask corpus`, and corpus_panasonic_encodings
checks each encoding's bit depth, black level, crop and CFA layout against the camera's own JPEG.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QtQmZ6peZXGn3wiqeowV2v
2026-10-07 16:00:39 +00:00
Daniel VoshartandClaude Sonnet 5.5 9089092969 Tests: pass on Windows, and under load
- paths are built by component, not with a hard-coded '/' (cli, import, rename, find-missing)
- the rename tests' model volume treats both separators as one
- the Local-folder UI tests hide Home when the temp folder lies inside it, and wait for
  the worker-thread listing instead of three frames
- the GPU equivalence test tolerates up to two edge pixels where the CPU and the card
  fall on different sides of the lens-warp sampling boundary

cargo xtask ci: all 7 steps pass.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 11:28:33 -04:00
Brandon ThomasandClaude Opus 5.5 6408ccf244 Tests: the keyword-list test lets the list settle before clicking rows
keyword_list_filters_renames_and_suggests failed on FreeBSD CI (b35487a) and
locally (2 of 13 runs under load): ui.clickWidget clicks last frame's rect,
and right after opening the panel / expanding "travel" the rows could still
move, so the click hit "sunrise" instead of "travel|italy". Settling first:
30 of 30 runs pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:58:39 -04:00
Brandon ThomasandClaude Opus 5.5 517c745fad Tests: the folder-tree test waits for the subfolder row, not three frames
local_folder_tree_expands_and_browses failed on FreeBSD CI (707138f) and under
local load: the expanded folder's listing can land after the fixed three frames.
It now steps until the row is laid out (bounded by SETTLE).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:18:53 -04:00
Brandon Thomas 30225fb6a0 Merge remote-tracking branch 'origin/main' into fix/flaky-profile-hover-test 2026-10-07 10:15:03 -04:00
Brandon Thomas b35487add2 Merge pull request #162 from GuintherKovalski/feat/sam3-object-masks
M12: SAM 3 Object and Describe masks (pure Rust, Metal)
2026-10-07 10:14:20 -04:00
Brandon Thomas 65bc68fd33 Merge remote-tracking branch 'origin/main' into land-162 2026-10-07 10:07:54 -04:00
Brandon ThomasandClaude Opus 5.5 d5be201431 M12: SAM 3 download dialog: no Download button without a location, clearer wording
Looked at in the running app: without a configured mirror the dialog
offered Download / Try Again that could only fail; it now shows the
manual-install hint and Close (an agent confirming still gets the
reason). "A one-time download of about 3.4 GB, saved in <folder>"; a
cancelled download is reported as stopped (resumable), not failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:06:46 -04:00
Brandon ThomasandClaude Opus 5.5 d379e780cc M12: AI masks never require SAM 3 — background worker, consented download, review fixes
Engine (crates/engine/src/segment/):
- One worker thread owns the model and the encoded photo; the session
  sends jobs (with the render job for the model input) and gets results
  over channels, never locking what the model holds. In the app
  (Segmenter::background) clicks, descriptions, prepares and detail
  passes return at once and Session::segment_poll applies results (one
  undo step per settled selection, pending clicks shown on the photo);
  the CLI/MCP wait for them. Newer requests for the same selection
  supersede queued ones.
- Every job runs under catch_unwind: a candle panic becomes an error and
  the model is reloaded next time; busy counters are reset by drop
  guards; a dead worker is restarted. The model unloads after 10 min idle.
- segment.model.status / .download {acknowledged: true} / .cancel: the
  download runs on its own thread; status via atomics (try_lock only).
  Without the model every AI request fails at once with "The SAM 3 model
  is not installed…"; renders/exports never need it; `seg` params make
  AI masks from stored segmentations without it.
- Hostile documents: at most 4 detail patches per component (capped on
  deserialize and in the renderer), at most 64 Object clicks.

UI: Object / Describe without the model open "Download the SAM 3
Model?" (size, folder, SAM License link; Not Now / Download, progress,
cancel, Try Again with the reason, Continue once installed); progress in
the Masking panel when the dialog is closed. Reverted the PR's unrelated
default change (mask overlay off + hover-red) to main's behaviour.

tools/install-sam3.*: --fail so error pages never land in files, 416
restarts instead of looping, the weights are verified before they are
moved into place (a bad file is deleted, never resumed from).

Docs: docs/ai-masks.md (in-app download, mirrors, SAM License summary,
no weights in the repo), NOTICE, parity LR-MASK-OBJECTS, What's New,
ROADMAP.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:56:50 -04:00
Brandon ThomasandClaude Opus 5.5 b52f43d1d5 M12: lightcraft-segment: Apache-2.0, checked weights, pure-Rust model download
- The crate ports Apache-2.0-only Hugging Face transformers code: it is
  now `license = "Apache-2.0"`, and every ported file carries the
  modification notice Apache-2.0 §4(b) asks for.
- Weights::open checks every tensor's byte range (and the header)
  against the file's length: a truncated or damaged download is an
  error at load, never a short read later; regression test with
  hostile offsets.
- fetch/: the model download, only when asked for. An ordered mirror
  list (env LIGHTCRAFT_SAM3_MIRRORS, a mirrors file, DEFAULT_MIRRORS —
  empty until LightCraft's CDN exists, TODO for the maintainer),
  fallback to the next mirror, resume via Range, redirects, connect and
  stall timeouts, cancel, size caps, exact size + SHA-256 (pinned from
  tools/install-sam3.sh) before an atomic rename; damaged files are
  deleted, HTTP error bodies never written.
- HTTP/1.1 client over std sockets + rustls with the RustCrypto provider
  (rustls-rustcrypto) and webpki-roots: no ring / aws-lc-rs, nothing
  compiled from C (no new `cc` users in the app's build graph).
- Tests against a local TcpListener server (no internet): fallback,
  redirects, chunked bodies, resume, 416, hash mismatch, too much data,
  stalls, cancel.
- *.safetensors are gitignored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:56:49 -04:00
Brandon ThomasandClaude Opus 5.5 8148988eb8 Docs: allow Noto CJK (via craft-fonts) for Chinese text
Maintainer decision: Noto Sans/Serif CJK is Google-branded Source Han
(Adobe co-developed); nearly every OFL Chinese face derives from it, and the
Simplified/Traditional Chinese UI (#184, #198) uses Noto Sans CJK SC from
craft-fonts. Record it as a narrow, explicit exception to the Adobe-asset rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:16:43 -04:00
Brandon ThomasandClaude Opus 5.5 fefd61c967 M12: merge origin/main into the SAM 3 masks branch (PR #162)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:16:09 -04:00
Harald HoyerandClaude Opus 5.5 707138f3fe LR-PROF-CAMERACOLOR: built-in ILCE-7M4 camera profile (#202)
A built-in Sony ILCE-7M4 colour profile (assets/camera-profiles/ILCE-7M4.json, compact JSON, 39 KB), fitted by lightcraft-cli calibrate to the contributor's own 597 raw photos against their in-camera JPEGs; aggregate colour statistics only, MIT OR Apache-2.0. Compiled in via camera_profiles::BUNDLED; a local profile replaces it.

Co-authored-by: Harald Hoyer <harald@hoyer.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:16:08 -04:00
Brandon Thomas fb9fb7717e Merge pull request #201 from haraldh/fix/arw-saturated-hues
LR-PROF-CAMERACOLOR: camera-look hue table, chroma curve and per-model camera profiles
2026-10-07 08:36:09 -04:00
Anarion Dunedain a5966f9b2e ci: add nix build package 2026-10-07 13:54:51 +02:00
Harald HoyerandClaude Opus 5.5 623122c1a9 Merge origin/main into fix/arw-saturated-hues
Conflicts with issue #150 (NEF colour from the camera JPEG): the per-file
look, its hue/saturation/value table and chroma curve, and camera
profiles now apply to every `file_local_look` format (ARW, NEF, NRW);
`calibrate` also collects NEF/NRW. Render cache version 11 (main 9,
branch 10).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 13:51:08 +02:00
Harald HoyerandClaude Opus 5.5 a52c7448e4 LR-PROF-CAMERACOLOR: docs: describe the validation photos neutrally
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 13:46:42 +02:00
Harald HoyerandClaude Opus 5.5 27d6bc7790 LR-PROF-CAMERACOLOR: camera colour profiles pooled from many photos
One photo can show too little of a colour for its own fit: a lime shirt
covering a few dozen proxy pixels next to foliage at the same hue stayed
yellow (hue 68 deg vs the JPEG's 82 deg).

- `lightcraft-cli calibrate [--max N] [--out DIR] FOLDERS...` pools the
  colour pairs of many ARWs per model (each against its embedded JPEG,
  192 px proxies, <= 4000 pairs per photo) and fits one matrix from
  white-balanced camera RGB plus a hue/saturation/value table.
- Profiles live in $LIGHTCRAFT_CAMERA_PROFILES or
  <config>/camera-profiles/<model>.json; validated on load, damaged files
  ignored with a warning, part of the render cache keys.
- Photos of a profiled model take its colour and fit only their own tone
  and chroma curves; the acceptance gates still apply.
- The table fit buckets samples by hue and runs nodes in parallel; the
  app's config_dir moves to the engine.

Profile from 300 Feb-Aug ILCE-7M4 photos (2 min); 29 held-out September
photos: mean dE vs camera JPEG 3.54 -> 3.10, saturated 6.41 -> 5.20;
the shirt's hue 68 -> 82 deg (JPEG 81.5).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 13:38:46 +02:00
Brandon Thomas f2f449611d Merge pull request #198 from sakamoto824/codex/zh-tw-localization
M16: add Traditional Chinese UI, menus and stock preset localization
2026-10-07 07:09:14 -04:00
Brandon Thomas 288fd3f125 Merge sakamoto824's original PR #198 commit (superseded by the port above) 2026-10-07 07:06:54 -04:00
Brandon Thomas 1779216514 Merge remote-tracking branch 'origin/main' into land-198 2026-10-07 07:06:50 -04:00
Brandon Thomas f0bb7c768e Merge pull request #184 from lzjtugjc/i18n/zh-hans
LR-BEHAV-LOCALIZE: a language table, Simplified Chinese, and per-script font fallback
2026-10-07 07:06:42 -04:00
Brandon Thomas 2eff66dd1e Merge remote-tracking branch 'origin/main' into land-184
# Conflicts:
#	docs/parity.md
2026-10-07 07:04:00 -04:00
ca0c569863 LR-BEHAV-LOCALIZE: Traditional Chinese (Taiwan) UI, stock preset and date localisation
Port of PR #198 onto the language table:
- Traditional Chinese (Taiwan) catalogs as a table entry: code zh-hant, name
  繁體中文(台灣), script Hant; zh-TW / zh-HK / zh-Hant tags reach it; command
  app.language.traditionalChinese
- display-only helpers: builtin_label (stock preset/profile names), history_label
  (generated history steps, user preset names verbatim), source_label (library
  headings, album names verbatim), date_group_label / display_time
- menu, dialog and panel strings (profiles, presets, curves, crop, versions,
  history, grid, date sidebar, About links) go through the catalogs
- a language command and ui.set apply the language at once; a native-menu test
  checks that every language switch rebuilds the menu titles

Adapted while porting (Claude):
- date headings and capture times are localised for every language from its
  *-formats.json date patterns, not only for zh-TW; ja and zh-hans gain the date
  patterns, weekday names and "Unknown Date", and zh-hans the new menu messages
  (ja's come from the PR)
- plain catalogs may cover different messages (English fallback); the test
  reports the gaps instead of failing. The test requiring every engine command
  label to be translated is dropped
- preset groups stay grouped by name: a user preset in a stock-named group is
  not split into a separate group (the group header is translated when it holds
  stock presets); the PR's split was a side effect of labelling
- no Traditional Chinese face in craft-fonts yet: Hant prefers the Simplified
  Chinese face (which carries the Traditional characters) over the Japanese ones
- zh-hant translations for the messages main added since the PR branched

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:54:26 -04:00
Brandon Thomas 549cbe728c Merge pull request #170 from voshart/pr/faces-1-regions
Faces: read face regions from XMP, show them in the loupe, and a People view
2026-10-07 06:49:45 -04:00
Brandon ThomasandClaude Opus 5.5 4e0f8ced0c Tests: the hover-preview tests wait for the hover render, not just a quiet spell
profile_browser_previews_on_hover_and_applies_on_click failed on FreeBSD CI
(loupe showed "render", not "hover"): settle() returns after 12 idle frames,
which can come before the hover render lands on a loaded VM. A step_until
helper waits for the condition itself; the preset hover test had the same race.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:36:17 -04:00
Brandon ThomasandClaude Opus 5.5 862c1f253a LR-LIB-PEOPLE: document MWG face regions in xmp-interop and the parity row
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:21:27 -04:00
Brandon ThomasandClaude Opus 5.5 4ded49ecd9 LR-LIB-PEOPLE: face boxes and People close-ups follow Rotate Left/Right
Regions are stored on the upright (EXIF-oriented) photo, but the loupe's normalized frame and the
close-up crop include the user's orientation. The loupe overlay now maps each box into that frame
(and a resize drag back out of it), and face_job maps the face and swaps the photo's size for a
quarter turn. Flips need nothing: they are applied after the normalized frame. Adds
Orientation::inverse and Orientation::map_norm_rect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:21:04 -04:00
Brandon ThomasandClaude Opus 5.5 269934caca LR-BEHAV-LOCALIZE: reinstall fonts on a language switch; restore the painted-UI tests
- the CJK fallback order follows the UI language's script, but the fonts were
  only installed on the first frame: switching 日本語 ↔ 简体中文 at runtime kept
  the previous language's Han forms. The app remembers the language its fonts
  were built for and reinstalls them when it changes.
- the native menu bar rebuilds on any language change (keyed on the language
  code, not on "uses CJK fonts", which is true for both CJK languages)
- restore the tests the language-table change dropped, for every language:
  core catalog entries, user-named menu items stay untranslated, the window
  paints each language and menu ids don't change with it, CJK glyphs in every
  UI family, and the UI runs without craft-fonts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:19:50 -04:00
Brandon ThomasandClaude Opus 5.5 8c253b28ad LR-LIB-PEOPLE: a sidecar that states mwg-rs:Regions replaces the photo's regions, even when empty
Re-reading a sidecar whose region list was emptied by a region-aware app now clears the stale
regions. A sidecar without mwg-rs:Regions at all (an app that doesn't do regions) says nothing about
them and the photo's regions are kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:19:45 -04:00
Brandon ThomasandClaude Opus 5.5 87a58b4183 LR-BEHAV-LOCALIZE: one Language menu entry per language; parity reads the language table
- UI_COMMANDS still listed app.language.english/japanese next to the new
  LANGUAGE_COMMANDS table, so Edit > Language showed English and 日本語 twice
- LANGUAGE_COMMANDS is `pub const`, the form xtask parity scans for; a test
  reads the real menus.rs and checks every language id is found once
- the Simplified Chinese command is app.language.simplifiedChinese, in line
  with the other camelCase command ids (the settings code stays zh-hans)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:17:06 -04:00
Brandon ThomasandClaude Opus 5.5 60a8649779 LR-LIB-PEOPLE: leave empty face regions out of the catalog JSON
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:16:02 -04:00
Brandon ThomasandClaude Opus 5.5 313c555b50 LR-BEHAV-LOCALIZE: compile-checked tr_format! for every language catalog
Restore main's compile-checked formatting, generalised to N catalogs: build.rs
writes one tr_format! arm per English message that matches on the active
language and calls format! with that language's translation (English is the
fallback arm). The runtime substitution dropped format specs ({:.1}, {:+},
{:.0}%), e.g. the smart-preview size and the merge progress percentage.

- formats catalogs use main's convention again ({0}/{1} for reordering, {:.0}
  to consume an unused plural suffix); zh-hans converted, with the size and
  percentage placeholders fixed by hand
- build.rs: auto-discovers locales/*-formats.json, validates the code
  (^[a-z]+(-[a-z]+)*$), fails when the catalogs' keys differ, and reruns
  when the locales directory changes
- zh-hans: translate the strings main added since the PR branched
- test: precision, sign and reordered positional values survive in every
  language

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:13:44 -04:00
Brandon ThomasandClaude Opus 5.5 3793e8fc33 LR-LIB-PEOPLE: clip XMP face regions to the photo, drop off-photo ones, read Rotation π
A region parsed from mwg-rs:Regions is now intersected with the 0..1 frame and dropped when it
misses the photo, so a hostile or stale sidecar (x=1e5, a 1e5-wide box) can't park a box far
outside the image. Lightroom's mwg-rs:Rotation of ±π (Exif orientation 3) is mapped to the upright
frame like the quarter turns; mirrored orientations can't be expressed in MWG and are taken as
written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 06:12:49 -04:00
Brandon Thomas 1b22a0e53b Merge origin/main into land-170 2026-10-07 06:09:24 -04:00
Brandon Thomas d814a31497 Merge remote-tracking branch 'origin/main' into land-184
# Conflicts:
#	crates/ui-egui/locales/ja-formats.json
#	docs/parity.md
2026-10-07 06:03:46 -04:00
Brandon Thomas 889b610e9f Merge pull request #192 from itsmunzir/fix/167-connect-hint
CLI: the connect hint names the port it actually tried (#167)
2026-10-07 06:02:12 -04:00
Brandon Thomas eacce855b9 Merge pull request #179 from coygeek/docs/star-history
docs: add star history chart to README
2026-10-07 06:02:08 -04:00
Brandon Thomas 3757506f46 Merge pull request #178 from cruzer45/feat/cr3-preview-metadata
CR3: full-size preview and metadata from the container
2026-10-07 06:02:03 -04:00
Brandon Thomas 01c9b743fe Merge remote-tracking branch 'origin/main' into fix-178
# Conflicts:
#	ROADMAP.md
#	crates/ui-egui/locales/ja.json
#	docs/parity.md
#	docs/whats-new.md
2026-10-07 06:01:50 -04:00
Brandon Thomas db2d94a233 Merge pull request #177 from cruzer45/fix/recently-deleted-restore
Restore from Recently Deleted; re-adding a deleted photo shows where it is
2026-10-07 06:01:33 -04:00
Brandon Thomas 2c2694ff1e Merge pull request #165 from treyreynolds/feat/nef-camera-look
Issue #150: Raw: NEF colour from the camera JPEG; 12-bit NEF black level
2026-10-07 06:01:27 -04:00
Brandon Thomas 136c1a0d89 Merge pull request #155 from kywk/feat/radial-masking-tuning
feat(ui): improve radial mask resizing and rotation
2026-10-07 06:01:23 -04:00
Brandon Thomas 9ef83ae3ae Merge pull request #152 from storybored379/fix/symlink-original-guard
Export: protect originals imported through differently named symlinks
2026-10-07 06:01:18 -04:00
Brandon ThomasandClaude Opus 5.5 51f0ef25ae Export: look up the library's symlinked originals once per guard
The symlink fallback ran read_link on every library entry for each existing
target; a large library made every Overwrite export check do thousands of
syscalls. The guard now finds the links on first need and keeps them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 05:41:28 -04:00
Brandon Thomas 1ce28c3aea Merge main into fix/symlink-original-guard
# Conflicts:
#	crates/engine/src/tests_export.rs
2026-10-07 05:41:26 -04:00
sakamoto824andCodex db640ed1d3 M16: add Traditional Chinese UI, menus and stock preset localization
Co-authored-by: Codex <noreply@openai.com>
2026-10-07 17:32:12 +08:00
Harald HoyerandClaude Opus 5.5 987d14b6ad LR-PROF-CAMERACOLOR: ARW camera looks bleach highlights and shift hue by brightness
A sunny ILCE-7M4 waterfall shot rendered yellow: sunlit rock came out
cream, +4 b* in the brightest fifth of the picture. Camera JPEGs make
shadows up to 1.4x more colourful and bleach highlights to 0.1-0.3x
(a per-channel tone curve), which a luminance-only curve can't do.

- CameraTone gains an 8-node chroma-by-display-luminance curve, fitted
  per file (bright pixels included) and kept only when it lowers the
  held-out error; applied after the tone map on CPU and GPU, keeping
  luminance. Old smart-preview headers load with an identity curve.
- The hue/saturation table gets an sRGB-encoded value axis: dark yellows
  had to turn toward orange while bright yellow-greens turned toward
  green. It is applied after the baseline exposure, as it is fitted.
- Matrix-only and matrix+table looks are completed (tone, chroma) and
  compared in gamma-encoded values; acceptance gates stay linear.

29 local ARWs vs their camera JPEGs: mean dE 4.23 -> 3.54, every file
closer, saturated areas 7.65 -> 6.41. Render cache v10.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 11:04:39 +02:00
Harald HoyerandClaude Opus 5.5 d4b73b7f64 LR-PROF-CAMERACOLOR: fit a hue/saturation table to ARW camera looks
A global 3x3 matrix can't follow the camera's hue-dependent rendering:
on an ILCE-7M4 photo of a lime shirt among grey rocks the best matrix
rendered the shirt olive (16 deg hue error), since the few saturated
pixels carry little weight and a matrix fitted to them breaks the rest.

After the matrix, fit a 72x5 hue/saturation table (DNG ProfileHueSatMap
layout, linear ProPhoto) to the residual hue shifts and saturation
ratios: kernel-weighted per node, shrunk to identity where samples are
sparse, identity at saturation 0. It preserves luminance (the tone curve
owns it) and is kept only when it lowers the held-out error.

28 local ILCE-7M4 ARWs: mean dE vs camera JPEG 4.74 -> 4.27, none worse
overall; the shirt's hue error drops to 6-8 deg. Render cache v9.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 09:54:27 +02:00
Mohammed Munzir 489f78ba69 CLI: the connect hint names the port it actually tried (#167)
run --connect ADDR could not reach the app and always advised lightcraft --control 7980, whatever port ADDR named, so following the hint left the command connecting elsewhere. The hint now takes the port from ADDR itself, with an end-to-end test on a refused connection.
2026-10-07 10:41:45 +03:00
goujichengandDeepSeek V3.2 22eb4d3515 LR-BEHAV-LOCALIZE: a language table, Simplified Chinese, and per-script font fallback
The localization layer was hardcoded to English and Japanese: an enum with two
variants, an `is_japanese()` gate compiled into the format macro, and one
catalog pair. Adding a language meant editing the menus, the settings row, the
checkmark logic, the format macro and the font stack — each in a different place.

A language is now one entry in `i18n::language_table!`: BCP-47 code, endonym,
ISO 15924 script and catalog. The Language menu, the settings row, the shortcut
sheet and the control channel are all built from that table, and `build.rs`
picks up `locales/<code>-formats.json` by name, so a further language is a table
entry plus two catalogs (docs/localization.md).

Simplified Chinese (`zh-hans`) ships as the second translation: `zh-hans.json`
(1019 entries, key-set and placeholder-identical to `ja.json`) and
`zh-hans-formats.json` (102 entries). Its glyphs come from craft-fonts' Noto
Sans CJK SC, whose `Hans` faces the UI now installs *before* the Japanese ones
for a Chinese reader, so shared Han characters keep the language's forms
(`fonts::cjk_fallback`); the export watermark accepts every CJK face.

`tr_format!` keeps its compile-time check. It formats the English source at the
call site — a runtime template cannot be handed to `format!` — and names those
values so a translation can place them by name (`{n}`) or position (`{_1}`).
Every translation is still checked by `format!`, so a translation with an
unknown placeholder fails the build. The Japanese catalog was rewritten from the
old `{:.0}` plural-discard trick to that by-name form; behaviour is unchanged.

Also: settings persist the BCP-47 code (never the Rust variant name),
`LIGHTCRAFT_LANGUAGE` accepts system-style tags (`zh_CN`, `ja_JP.UTF-8`),
`xtask parity` reads the second command table, and the glyph tests cover every
catalog character plus the interface symbols (`▸`/`▾` turned out to have no
glyph in any installed font and are gone from the Chinese text).

Chinese and Japanese both need `CRAFT_FONTS_DIR` as before; the web build still
embeds only the Japanese face (Noto Sans CJK SC alone would push the wasm past
Cloudflare's 25 MiB per-file cap), documented in docs/localization-zh-hans.md.

Co-Authored-By: DeepSeek V3.2 <noreply@deepseek.com>
2026-10-07 12:22:15 +08:00
Maurice Rogers 9adb40908a LR-IMP-FORMATS: Reload from Disk in the Photo and context menus; fills missing camera metadata
photo.reload had no menu path, so the documented way to refresh CR3s imported
before their metadata was read (Photo > Reload from Disk) did not exist in the
app. It is now in the Photo menu and the photo context menu, and besides
size/preview-only it fills camera fields the catalog lacks (camera, lens,
exposure, GPS, capture time) from a fresh probe, never overwriting a set value.
2026-10-06 21:53:09 -06:00
Coy Geek c9fe78e178 docs: add star history chart to README
Show repository star growth at the bottom of the README. Link the date-based
chart with a top-left legend to the corresponding Star History page.
2026-10-06 19:27:15 -07:00
Maurice Rogers 13a3eff2e8 LR-IMP-FORMATS: CR3 full-size preview and metadata from the ISO-BMFF container
CR3 files imported as preview-only with the 1620x1080 PRVW JPEG and no metadata
(no camera, capture time, exposure, lens or GPS): nothing read the container.

- crates/meta/src/cr3.rs: bounded ISO base media box parser for CR3 (Canon uuid
  box with CMT1-4 and THMB, tracks with their sample entries, CMP1 coding header
  and sample location, XMP uuid box). Every size/offset checked; depth and box
  count bounded.
- Metadata: CMT1 (IFD0), CMT2 (Exif) and CMT4 (GPS) merged into one TIFF Exif
  block, so CR3 reads like any TIFF-based raw; XMP from its uuid box.
- Preview: the full-size JPEG track (6960x4640 on an EOS R6 Mark III) before
  PRVW/THMB; imports report the full image size.
- Tests: synthetic CR3 (boxes, tracks, merged Exif), truncation/corruption sweep,
  JPEG track preferred over PRVW with fallback.

Raw (CRX) data is still not decoded; CR3 stays preview-only.
2026-10-06 20:00:48 -06:00
Maurice Rogers 6bda431c4b LR-LIB-DELETE: restore from Recently Deleted; re-adding a deleted photo shows where it is
Photos in Recently Deleted could not be restored from the app: photo.restore and
photo.deletePermanently existed only as engine commands, and adding the same file
again just said "1 duplicate skipped" (the duplicate check counts deleted photos),
with the side panel that lists Recently Deleted collapsed by default.

- Restore and Delete Permanently in the Photo menu and the photo context menu,
  in place of Delete, when the selection is in Recently Deleted.
- The filmstrip gets the photo context menu (it had none).
- An import that only finds files the library already has selects those photos:
  in Recently Deleted (side panel opened, how to restore or delete permanently and
  import afresh) or in All Photos. Mixed imports note how many are in Recently
  Deleted. The message stays 6 s.
- Import review badges say "In Recently Deleted"; badges are translated.
- Japanese strings, parity rows, what's new; headless regression test.
2026-10-06 19:30:23 -06:00
Trey ReynoldsandClaude Opus 5.5 22c6aa6918 Issue #150: Raw: NEF colour from the camera JPEG; 12-bit NEF black level
LightCraft has no colour matrix for Nikon cameras. Because of this, NEF
files showed colours that were too gray and too green.

- Colour: LightCraft now calculates a colour matrix and a tone curve from
  the JPEG preview in each NEF file. Sony ARW files use the same
  procedure and the same quality checks. White balance is relative to
  the as-shot look, as for ARW files.
- One function, camera_preview::file_local_look, selects the formats for
  the fit and for the relative white balance. Photo::relative_wb in the
  catalog uses the same formats.
- Black level: Nikon stores the black level (maker note 0x003d) in
  14-bit units, also in 12-bit files. The decoder read it as a 12-bit
  value. Because of this, 12-bit files from the D750, D780, D850, D7500
  and Z 50 were too dark. Now the decoder divides the value by 4 for
  12-bit files.
- The corpus gets the D7500 12-bit and 14-bit lossless samples. A test
  compares their levels. The render cache version increases to 9.

Results on 13 NEF files that LightCraft can decode: LightCraft accepted
the fit for all 13 files. The mean colour difference from the camera
JPEG (CIE76 dE) decreased from 13-46 to 2.9-7.8. One file with mixed
light improved by a small quantity only (12.7 to 12.0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:26:45 -05:00
Daniel VoshartandClaude Sonnet 5.5 9bb06cc9dc Faces: resize a face box with eight drag handles
Hovering a box shows handles at its corners and edge midpoints; dragging one
previews the new box live and commits once on release, as one undo step, through
the new `photo.setRegion {id?, index, rect}`: catalog-only like removal, never
rewriting the sidecar, with the rectangle validated (finite numbers only,
clamped to the photo, at least 0.5 % each way) and a test with a real sidecar.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-06 19:44:15 -04:00
Daniel VoshartandClaude Sonnet 5.5 e5848aa1b2 Filters: open a library unfiltered, and show the active filters in the loupe
A date, keyword or person left over from the last session silently hid photos
behind a small badge, so view.json no longer stores or restores the filter (the
source, sort and selection still come back). In the loupe a small pill at the
canvas' top left names the active filters (the filmstrip and Next / Previous
follow them) and clears them on a click; it sits in the canvas margin, so the
photo does not move.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-06 19:44:15 -04:00
Daniel VoshartandClaude Sonnet 5.5 e2e6db0f14 Faces: switch face boxes on/off and remove one with the × shown on hover
View > Face Boxes (and a toolbar button in the loupe) hides or shows the boxes.
Hovering a box shows a × in its corner; clicking it runs the new undoable
`photo.removeRegion {id?, index}`, a catalog-only edit. The XMP sidecar is
never rewritten for it, even with auto-write on (a `skip_auto_write` flag
consumed when the command ends), because the sidecar writer builds a fresh
packet that does not carry the regions: reading the metadata from the file
brings a removed region back. A test with a real sidecar proves the file stays
byte for byte, that ordinary edits still auto-write, and that undo restores it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-06 19:44:14 -04:00
Daniel VoshartandClaude Sonnet 5.5 b9621eb5a3 People view: offer only people in the photos the active filters let through
The view listed everyone in the library, so with another filter active (a
date, a keyword…) picking a person could end in an empty grid. The list now
covers the photos matching every filter except `person` itself, counts are
within them, and the filter chips show above the cards (removable, Clear all).
`Catalog::people_in(&Filter)`; the cache is keyed on the catalog revision and
the filter.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-06 19:44:14 -04:00
Daniel VoshartandClaude Sonnet 5.5 8859f797d8 People view: a card per named person with a cropped face, replacing the sidebar list
A dedicated view mode (View > People, and a toolbar button) shows a card per
person named on faces: a close-up of their largest face, the name and how many
photos they are in. A click shows that person's photos in the grid. Only the
cards on screen render their face. The sidebar section is gone (it would get
crowded).

`Catalog::people()` now returns `Person { name, count, photo, face }`, the
photo and face being the person's largest. `Session::face_job` renders a
square close-up through the engine (the photo's own look, crop replaced, the
smallest source level that keeps it sharp, cached like variant thumbnails); it
clamps hostile rectangles, with a test for corners, oversized, degenerate and
non-finite faces.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-06 19:44:14 -04:00
Daniel VoshartandClaude Sonnet 5.5 7c27c3a920 People MVP: list named faces from XMP in the sidebar and filter by person
Read-only and names-only: the left panel gets a People section built from the
named Face regions read from XMP, with photo counts (once per photo, case
insensitive, pets and unnamed faces left out). A click filters the grid via a
new `person` field on `library.filter`, also reachable as a `person:` search
token and shown as a filter chip. The list is cached on the catalog revision.
No face detection, identity or writing regions back yet. Parity rows for the
People list, filter and view are 🟡 with that scope noted.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-06 19:44:00 -04:00
Daniel VoshartandClaude Sonnet 5.5 f5f8bc5d34 UI: draw face labels as a dark caret label above a white/black keyline box
The name sits in a dark rounded label centred above the box with a small
caret (below it when there is no room above), replacing the offset-shadow
text. The box is a 1px white outline with a 1px black keyline so it shows on
any background.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-06 19:42:32 -04:00
Daniel VoshartandClaude Sonnet 5.5 9715c90413 Meta: map Lightroom's quarter-turn region Rotation to the upright frame
Lightroom Classic stores the box of a photo with Exif orientation 6 / 8 in
the sensor's own frame and records the orientation as mwg-rs:Rotation
(-pi/2 for 6, +pi/2 for 8, 0 for 1) while AppliedToDimensions names the
upright size. Ignoring Rotation put every portrait photo's face box in the
wrong place. Convert such regions into the upright frame, with a synthetic
regression test for -pi/2, +pi/2 and 0.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-06 19:42:32 -04:00
Daniel VoshartandClaude Sonnet 5 f1b2d9af11 Catalog/UI: show XMP face/pet regions in the loupe view
Threads the Vec<Region> that lightcraft-meta already reads from XMP
(MWG-RS) through to the screen, so it's no longer inert:

- catalog::Meta gains a `regions` field (catalog now depends on
  lightcraft-meta, which doesn't depend back on catalog, so no cycle).
- crates/engine/src/files.rs (meta_of) and src/sidecar.rs (parse_sidecar
  / merge_into) populate it for both embedded-XMP and external-sidecar
  photos, mirroring how `keywords` is already handled.
- ui-egui's loupe view (panels/detail.rs) draws each region as a
  rectangle over the photo via the existing CanvasMap, color-coded by
  kind (Face/Pet/other) with its name label if present, right alongside
  the existing info overlay. Read-only, no new interaction.

New end-to-end test (regions_from_sidecar_are_read_on_import) covers a
region written to a sidecar reaching Photo.meta.regions through real
import, not just the parser. cargo xtask ci green (fmt, clippy -D warnings, test, parity, layers, assets, wasm).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-06 19:42:32 -04:00
Daniel VoshartandClaude Sonnet 5 f29436a1f6 Meta: read MWG-RS face/pet/focus regions from XMP (read-only)
Adds Metadata::regions: Vec<Region>, populated by parse_xmp from mwg-rs:Regions
(MWG Region Guidelines v2.0) — the format Lightroom, digiKam and Picasa write
for face/pet/focus/barcode regions. Read-only: LightCraft does not write
regions yet (blocked on #92, which would otherwise clobber existing sidecars).

- Canonicalises the mwg-rs/stArea/stDim namespaces so any prefix a writer
  picks resolves consistently, reusing the existing generic struct/array XMP
  machinery (handles both the element and attribute encodings transparently).
- Area center-coordinates are converted to lightcraft_geom::Rect via
  Rect::from_center. Normalized units (the default, and the only unit we've
  seen in practice) need no AppliedToDimensions cross-reference; the rare
  pixel-unit region is converted when AppliedToDimensions is present, and
  dropped (not stored as a nonsense fraction) when it isn't.
- Hostile/malformed regions (missing Area, NaN/absurd/zero/negative extents)
  are skipped individually rather than failing the whole list or panicking.
- regions merges like keywords in fill_missing/overlay_user_fields, since it
  only ever comes from XMP today.

Not included: nothing yet reads Metadata::regions (no catalog field, no UI,
no command) — this is plumbing for a later PR, with no user-visible effect
and no docs/parity.md row to update.

cargo xtask ci passes (fmt, clippy -D warnings, test, parity, layers, assets,
wasm).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-06 19:42:31 -04:00
Guinther KovalskiandClaude Opus 5.5 6468bfc06b M12: SAM 3 Object and Describe masks (pure Rust, Metal)
AI masks from Meta's SAM 3, run in LightCraft itself: a new
`lightcraft-segment` crate ports the Hugging Face transformers
implementation (Apache-2.0) to candle, on the GPU through Metal on macOS
(CPU elsewhere), with no C/C++ and no `unsafe`.

- Object: click to include, ⌥-click to leave a part out (SAM 3's
  interactive prompt encoder + two-way mask decoder).
- Describe: a text prompt selects every instance ("sky", "the red car";
  "car, road" merges several) via the CLIP text encoder and DETR detector.
- Both as new masks and as Add / Subtract / Intersect components; + / −
  buttons under the selected mask combine selections.
- The segmentation is stored with the mask (288² quantized logits, ~10-30
  KB), so renders, exports and the web build never need the model.
- A zoomed-in detail pass (background, ~5 s on an M4 Pro) re-segments the
  area around a selection from a sharper source: 5-10x finer edges on
  small objects.
- Per-selection Edge slider (hard ↔ soft); hovering a mask in the list
  shows it in red (the always-on overlay is now off by default, O toggles).
- The engine feature `sam` (on in the desktop app) gates the model; the
  image is encoded once per photo in the background, clicks take ~35 ms.
- Weights are not bundled (SAM License): tools/install-sam3.sh / .ps1
  download and verify facebook/sam3; docs/ai-masks.md.
- Clearer errors (missing file, missing model) and a stderr logger in the
  app (LIGHTCRAFT_LOG=info).

candle is pinned at 0.9.2: later releases make candle-core depend on
tokenizers with the Oniguruma C library. Every stage is checked against
the Python reference (tests/reference.rs, tools/sam3_reference.py): on
Metal the backbone matches to 7e-5 relative error, masks to ~5e-5.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C8BkS7JDqa1EMcjW6spDGA
2026-10-06 20:15:04 -03:00
Brandon Thomas 5cee0c08d0 Merge pull request #158 from storytold/fix/issue-148-rx100m3-green
Issue #148: Raw: read white balance, black level and crop of pre-2017 Sony ARWs
2026-10-06 18:25:50 -04:00
Yu Zhi Chen 6ad215e5b9 feat(ui): improve radial mask resizing and rotation
- Added 4 control handles on the edge of the radial gradient mask to resize rx and ry independently.
- Added a rotation handle to adjust the mask angle.
- Changed drag gesture handling to store the original mask shape to prevent drift from accumulated relative deltas.
- Holding the Option/Alt key while dragging a handle scales the mask proportionally.
- Snaps the mask to a perfect circle (1:1 aspect ratio) when the aspect ratio is within 5%.
2026-10-07 04:21:14 +08:00
Brandon ThomasandClaude Opus 5.5 33a84b8427 Issue #148: Raw: read white balance, black level and crop of pre-2017 Sony ARWs
Sony bodies from before ~2017 (RX100–RX100M5, RX10, NEX, SLT, ILCE-6000,
A7/A7II/A7RII, ...) write no plain 0x7313/0x7310/0x74c7 tags in the raw IFD,
so the decoder used unit WB multipliers (strong green cast, e.g. the CC0
DSC-RX100M3 sample), black 512 (800 on 1-inch sensors), and kept padding
columns at the right edge.

- White balance: WB_RGBLevels from the maker note's enciphered Tag2010
  block (layouts per ExifTool's Tag2010a..i tables; byte substitution
  p^3 mod 249 checked black-box on 15 CC0 files: SonyISO matches Exif ISO,
  ILCE-7M3 levels match its plain 0x7313).
- Black level: recovered from the encrypted SR2SubIFD by known-plaintext
  analysis (constant key in every file; RX100M6/ILCE-7M3 plain 0x7310 as
  plaintext; 8 keystream bytes at offset 2510, plausibility-checked).
- Crop: DNG DefaultCrop or the maker note's FullImageSize.
- "12-bit uncompressed" ARWs hold 14-bit-scale data: black/white follow it.
- Render cache version bumped; corpus manifest + corpus test, unit tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 15:27:33 -04:00
Brandon Thomas 265248c86f Merge pull request #151 from storytold/fix/snapshot-test-settle
Tests: snapshot-determinism test asserts both runs settled
2026-10-06 15:14:10 -04:00
Brandon ThomasandClaude Opus 5.5 1130e99095 Tests: the snapshot-determinism test asserts both runs settled before comparing pixels
snapshot() ignored settle()'s result, so on a loaded machine (seen on the FreeBSD VM and
under local load) a timed-out settle compared half-rendered frames and failed as a
misleading pixel diff. The test now settles explicitly with a 300 s budget, asserts it
settled, then paints.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 15:13:58 -04:00
Brandon Thomas 07a7316d8b Merge pull request #149 from storytold/pr/craft-fonts
Fonts: use craft-fonts as an optional build input; move the Japanese fonts out
2026-10-06 14:49:22 -04:00
Brandon ThomasandClaude Opus 5.5 2aa236537f Fonts: use craft-fonts as an optional build input; move the Japanese fonts out
The Japanese fonts (BIZ UDPGothic Regular/Bold for the UI, BIZ UDMincho for text
watermarks; #110, #114) now come from storytold/craft-fonts instead of assets/fonts/.

- crates/engine/build.rs: the craft-fonts recipe. With CRAFT_FONTS_DIR set it embeds
  the manifest's fonts as lightcraft_engine::CRAFT_FONTS (wasm32: BIZ UDPGothic Regular
  only, for the web size budget); unset, CRAFT_FONTS is empty. Local files only, no
  network; craft-fonts is not in any Cargo.toml.
- UI: theme::font_definitions adds the Japanese faces after Inter and egui's defaults
  in every family (BIZ UDPGothic first, Bold first for semibold). About box lists the
  embedded families.
- Watermarks: Inter, then the craft-fonts Japanese faces (Mincho first).
- Removed BIZUDPGothic-Regular/Bold.ttf, BIZUDMincho-Regular.ttf and their OFL files,
  and their ATTRIBUTION rows; added an attribution row for craft-fonts as an optional
  build input; NOTICE updated.
- Tests: Japanese glyph tests use CRAFT_FONTS and skip without it; new tests for
  Japanese in the UI with craft-fonts and for the UI and watermarks without it.
- CI: the FreeBSD job builds/tests with craft-fonts. Releases check out craft-fonts at
  a pinned commit, build with CRAFT_FONTS_DIR + CRAFT_FONTS_REQUIRED=1, and ship the
  fonts' OFL licences (plus NOTICE) in every package.
- Docs: AGENTS.md, CLAUDE.md, README, docs/web.md, docs/localization-ja.md, ROADMAP,
  parity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 11:35:02 -04:00
Brandon Thomas bd3185ef4d Merge pull request #145 from storytold/fix/issue-138-lightroom-dng
Issue #138: DNG: apply the file's own camera-profile look (HueSatMap, LookTable, ToneCurve)
2026-10-06 11:27:16 -04:00
Brandon Thomas 4cf883155a Merge pull request #144 from storytold/fix/issue-136-vulkan-startup-crash
Issue #136: Windows: use DX12 by default, honour WGPU_BACKEND, never let GPU init block startup
2026-10-06 11:27:03 -04:00
Brandon ThomasandClaude Opus 5.5 71eac70696 Issue #138: DNG: apply the file's own camera-profile look (HueSatMap, LookTable, ToneCurve)
Lightroom-converted DNGs carry their camera profile's ProfileHueSatMapData1/2
and ProfileLookTableData (and sometimes ProfileToneCurve). We used only the
colour matrices, so such DNGs rendered flat and muted.

- raw: new `profile` module reads the tags (DNG spec ch. 6: value-major,
  hue-middle, saturation-minor tables, wrap-around hue, tri-linear lookups,
  sRGB value encoding) from the raw IFD or IFD 0; malformed tags are ignored.
- engine: the raw loader applies the hue/sat map (interpolated by the same
  illuminant weight as the matrices) before BaselineExposure and the look
  table after it, in linear ProPhoto; a ProfileToneCurve becomes the source's
  camera tone. Without one the standard base tone is kept.
- dngwrite keeps the look with the data (conversions, smart previews, merges).
- Render cache version 7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 10:36:30 -04:00
Brandon ThomasandClaude Opus 5.5 b66cdb23a3 Issue #136: Windows: use DX12 by default, honour WGPU_BACKEND, never let GPU init block startup
wgpu loads the driver of every backend in an instance's set while it
enumerates adapters. The compute device (crates/gpu) hard-coded
Backends::PRIMARY (Vulkan + DX12 + Metal) and ignored WGPU_BACKEND, and
was created from LightcraftApp::new on a background thread at launch, so
a crashing Vulkan driver (Intel UHD 630, igvk64.dll) killed the process
before the window appeared; the window (egui-wgpu) also listed Vulkan.

- lightcraft_gpu::backend: DX12 only on Windows (Metal on macOS, as before
  on Linux) for both the compute device and the window;
  LIGHTCRAFT_GPU_BACKEND=dx12|vulkan|metal|gl|auto|off, else WGPU_BACKEND;
  backends missing from the build fall back to the default.
- The device is created only when GPU rendering is enabled, after the
  first frame; the preference is applied before the window opens.
- Crash sentinel: gpu-init.marker in the settings folder around device
  creation; if left behind, the next launch starts with GPU rendering off
  and a notice saying how to turn it back on.
- Tests: backend parsing/selection, sentinel, window defaults, headless
  snapshot with LIGHTCRAFT_GPU=0 / LIGHTCRAFT_GPU_BACKEND=off.
- Docs: gpu-pipeline.md (backends, env vars, troubleshooting), whats-new,
  parity LR-BEHAV-GPU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 10:16:05 -04:00
Brandon Thomas d2359ce095 Merge pull request #143 from storytold/fix/issue-134-trim-write-safety
Issue #134: Exports: atomic without fsync; Copy import verifies against the probe hash
2026-10-06 10:11:47 -04:00
Brandon ThomasandClaude Opus 5.5 7a2596cd43 Issue #134: Exports: atomic without fsync; Copy import verifies against the probe hash
- safe_file: write policy documented (durable for what can't be recreated, atomic-only for
  recreatable outputs); new write_atomic_nosync (temp + rename, temp removed on failure, no
  file/folder sync); per-thread sync counter for tests.
- export::write_file (exports, renders, screenshots, merge previews, CLI/MCP) uses the
  non-syncing writer; new write_file_durable keeps the sync for Edit a Copy (the TIFF becomes a
  library photo). Catalog, sidecars, settings, DNG conversion/merges and smart previews stay
  durable; the #93 original-overwrite guard is untouched.
- Import > Copy verifies the synced copy against the probe's content hash (SipHash-1-3/128 of the
  whole file) instead of re-reading the source; byte compare when there is no hash. A file changed
  since Review Import is now refused instead of imported with stale details. Move keeps its
  byte-for-byte check before deleting sources.
- check_write_target: skip building the library-wide guard when nothing is at the target.
- write_new_unique keeps its read-back (DNGs replace raws; cheap next to encoding).
- Tests: no sync on the export path, durable writers still sync, Copy caught by the probe hash
  (corrupt copy, stale hash, changed source); ignored timing test write_policy_timing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 10:08:29 -04:00
Brandon Thomas 8bef0b1ef3 Merge pull request #141 from storytold/pr/windows-arm64
Windows on ARM64: release builds, PE header check, install test on ARM64 hardware
2026-10-06 09:59:27 -04:00
Brandon ThomasandClaude Opus 5.5 a2dbefd1ee Windows ARM64 workflow: pin third-party actions by commit
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:51:08 -04:00
Brandon ThomasandClaude Opus 5.5 f5bd3cc2db Windows on ARM64: release builds, PE header check, install test on ARM64 hardware
Release builds add an arm64 MSI and portable zip, cross-compiled on the x64
runner (signing and WiX as for x64). package.ps1 checks each binary's PE
header before packaging: Machine must match -Arch, the app must be a GUI
program and the CLI a console program. It skips the CLI smoke run when an
ARM64 build can't run on the host. A new Windows ARM64 workflow packages the
ARM64 MSI, installs it on a Windows 11 ARM64 runner, checks both installed
exes are ARM64, runs the CLI, and uninstalls. Ported from printcraft#66.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:38:47 -04:00
Brandon Thomas 1969b311cc Merge pull request #137 from benlukka/fix-flickering-preview-hover
fix(ui): eliminate preview flickering when hovering presets and profiles
2026-10-06 09:24:12 -04:00
Brandon Thomas fc3fa1a71b Merge pull request #140 from storytold/pr/release-0.2.1
Release: LightCraft v0.2.1
2026-10-06 08:01:39 -04:00
Brandon ThomasandClaude Opus 5.5 d0612b0b21 Release: LightCraft v0.2.1
Fixes the Windows installer: the Start Menu shortcut and Add/Remove Programs icon were blank, and the shortcut description was wrong.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:01:29 -04:00
Brandon Thomas 66dc737ff5 Merge pull request #139 from storytold/pr/windows-packaging
Windows packaging: Start Menu/Add-Remove icon, no console window, correct description
2026-10-06 07:54:46 -04:00
Brandon ThomasandClaude Opus 5.5 fec3b15234 release.yml: explicit if for the wasm-bindgen install (shellcheck SC2015)
packaging-lint has failed on main since this step was added; same behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 07:53:44 -04:00
Brandon ThomasandClaude Opus 5.5 11ebb3bdb6 Windows packaging: Start Menu/Add-Remove icon, no console window, correct description
The MSI Icon id had no .ico/.exe extension, which Windows Installer needs for
advertised shortcuts and ARPPRODUCTICON, so both showed a blank icon. Release
builds of the desktop app are now GUI-subsystem (no console window; the CLI stays
a console program), the shortcut/summary description names what LightCraft does, and
packaging-lint rejects an Icon id without an extension. Found in printcraft#57.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 07:50:38 -04:00
benlukka ad2525ece2 fix(ui): eliminate preview flickering when hovering presets and profiles
- Retain active preset preview when hovering within the scroll or panel area across headers and gaps.
- Prevent premature dropping of hover preview textures during asynchronous rerendering in detail view.
- Clear cached hover state when clicking an item or when the pointer leaves the panel.
- Add headless regression tests covering preset hover persistence and transitions.

Signed-off-by: benlukka <68825026+benlukka@users.noreply.github.com>
2026-10-06 13:18:47 +02:00
storybored379andOpenAI Codex 2c94b7fa1f Export: protect originals imported through differently named symlinks
Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-10-05 23:51:43 -07:00
Brandon Thomas 65da36675f Merge pull request #115 from arata-1972/codex/sony-arw-initial-colour
Improve Sony ARW starting colour and tone, and preserve decoded source metadata
2026-10-05 19:47:57 -04:00
Brandon ThomasandClaude Opus 5.5 cbb6c3b8f7 Merge origin/main (#114) into land-115
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:41:46 -04:00
Brandon ThomasandClaude Opus 5.5 5b414ffb8e LR-PROF-CAMERACOLOR: record the ARW file-local fit's public-sample results
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:34:01 -04:00
Brandon ThomasandClaude Opus 5.5 f7a7831380 LR-PROF-CAMERACOLOR: keep the ARW camera tone curve with smart previews
Smart previews store the decoded source with the fitted colour matrix
baked in, but the file-local tone curve lives in the decoder's
SourceInfo, which a proxy did not keep: an offline ARW rendered and
exported with the generic filmic curve instead (darker, a different
look). The proxy header now carries the curve (validated on load,
ignored when invalid), DecodedSource::info_or merges it into the header
facts, and is_valid reads the longer header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:30:43 -04:00
Brandon Thomas 2b2ade7a23 Merge pull request #114 from arata-1972/codex/lightcraft-japanese
Extend Japanese localization across editing, import and export UI
2026-10-05 19:29:49 -04:00
Brandon ThomasandClaude Opus 5.5 d6c7228dd4 M16: merge origin/main into Japanese localisation (PR #114)
Keep main's data-safety work (ui.json recovery and atomic writes, library-problem
window, off-thread file checks, import Cancel) and wrap its new strings: import
cancel/progress messages, the library-problem window and banner, and the new
Back Up / Restore Library menu items now have Japanese translations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:28:34 -04:00
Brandon ThomasandClaude Opus 5.5 4b3048c48f LR-PROF-CAMERACOLOR: accept ARW camera-look fits up to 0.10 held-out RMS
The 0.055 cap rejected half of the public Sony samples tried (ILCE-6400,
ILCE-6700, ILCE-7M4) although their fits were 1.5-3x closer to the
camera JPEG than the fallback and visibly better (mean dE vs the embedded
preview 19-26 -> 5-10). Unrelated previews stay rejected (synthetic test
lands at 0.115).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:28:22 -04:00
Brandon ThomasandClaude Opus 5.5 bc4799ca42 Merge origin/main into land-115 (PR #115)
Resolve SourceRef::load_source against #104's offline fallback: a File
source keeps the decoder's SourceInfo; a smart-preview fallback has none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:25:37 -04:00
Brandon ThomasandClaude Opus 5.5 586479a309 M16: drop fork-only Japanese packaging from PR #114
Remove scripts/package-japanese-macos.py, the japanese-local feature and its
config/library/startup overrides, the settings.language migration, and the
local-build notes in the docs. Upstream library and config defaults (incl.
library::default_dir, which already honours LIGHTCRAFT_LIBRARY) are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:11:34 -04:00
Brandon Thomas 87f275b58e Merge pull request #132 from storytold/fix/issue-104-ui-fs-v2
Issue #104: keep file-system calls off the UI thread
2026-10-05 19:08:15 -04:00
Brandon Thomas 894759e6d8 Merge main into fix/issue-104-ui-fs-v2
# Conflicts:
#	docs/parity.md
#	docs/whats-new.md
2026-10-05 19:00:37 -04:00
Brandon Thomas 1952f7c960 Merge pull request #131 from storytold/fix/issue-107-web-safety
Issue #107: Web build: data-safety fixes for libraries kept in the browser
2026-10-05 18:59:14 -04:00
Brandon Thomas 7bbdbc03ef Merge main into fix/issue-107-web-safety
# Conflicts:
#	apps/lightcraft-web/src/web.rs
#	docs/parity.md
#	docs/whats-new.md
2026-10-05 18:58:53 -04:00
Brandon Thomas 0f34a65a8c Merge pull request #130 from storytold/fix/issue-97-folder-undo
Issue #97: Rename Folder / Move Folder are undo steps
2026-10-05 18:58:38 -04:00
Brandon Thomas 6c066b66f2 Merge main into fix/issue-97-folder-undo
# Conflicts:
#	crates/engine/src/lib.rs
#	docs/parity.md
#	docs/whats-new.md
2026-10-05 18:58:13 -04:00
Brandon ThomasandClaude Opus 5.5 7b59081d22 Issue #104: keep file-system calls off the UI thread
Re-applies fix/issue-104-ui-fs on top of the data-safety fixes (integrate/safety):
- availability cache: originals and smart previews are checked on a worker thread; the UI
  answers from the cache (grid, Info panel, photo menu, Missing Photos view)
- import split into ImportJob::prepare (worker) + commit_prepared, with Cancel; Copy keeps
  #96's verified copies
- Find Missing Photos: find_candidates (snapshot, no I/O) + plan_find_missing (all file I/O,
  on a worker, never the session) with #105's matching rules unchanged (name+size+hash;
  renamed by size+hash; single name+size; ambiguous look-alikes skipped and reported; files
  already in use excluded). The plan is applied with library.findMissing {found, missing,
  ambiguous}, re-checked against the catalog then: photos relinked meanwhile are left alone,
  files now in use are not taken. library.findMissing {folder} still runs synchronously.
- Build / Discard Smart Previews with background: true runs on a worker with progress and
  Stop, keeping #106's atomic writes, is_valid checks and `repaired` count (also in
  library.previewProgress); the render worker's smart-preview fallback uses is_valid
- Rename preview planned on a worker (plan_rename_photos) on the same planning core as
  Session::plan_rename_with (MoveFs, case-only respelling by file identity, #95)
- Local folder listing via fs_cached; tests_offline (slow injected probe, frames < 300 ms)

Tests: worker-planned Find Missing equals the synchronous command for a renamed + ambiguous
library; a plan is re-checked when applied; background smart previews repair a damaged proxy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 18:58:04 -04:00
Brandon Thomas e11f77d9ca Merge pull request #129 from storytold/fix/issue-94-control-close
Issue #94: Control: close the connection on malformed input
2026-10-05 18:57:59 -04:00
Brandon Thomas f383d1e736 Merge main into fix/issue-94-control-close
# Conflicts:
#	docs/whats-new.md
2026-10-05 18:57:50 -04:00
Brandon Thomas edf05789ed Merge pull request #128 from storytold/fix/issue-103-settings-files
Issue #103: Settings files: keep damaged ones, never overwrite unread ones, write ui.json atomically, confirm quit with unsaved changes
2026-10-05 18:57:30 -04:00
Brandon Thomas bccccfb92a Merge main into fix/issue-103-settings-files
# Conflicts:
#	crates/engine/src/library.rs
#	docs/whats-new.md
2026-10-05 18:56:50 -04:00
Brandon Thomas 4939d9b994 Merge pull request #127 from storytold/fix/issue-100-open-failure
Issue #100: Ask what to do when the library can't be opened, never a silent demo session
2026-10-05 18:56:35 -04:00
Brandon Thomas e0ab41ede0 Merge main into fix/issue-100-open-failure 2026-10-05 18:56:00 -04:00
Brandon Thomas 8690b51fcf Merge pull request #126 from storytold/fix/issue-99-library-lock
Issue #99: Library: exclusive lock so two processes can't open one library
2026-10-05 18:55:44 -04:00
Brandon Thomas e1d3b56bb6 Merge main into fix/issue-99-library-lock
# Conflicts:
#	crates/catalog/src/lib.rs
2026-10-05 18:54:56 -04:00
Brandon Thomas 6100237cdb Merge pull request #125 from storytold/fix/issue-102-format-version
Issue #102: Catalog format version 2; refuse newer libraries untouched
2026-10-05 18:54:41 -04:00
Brandon Thomas 4579599745 Merge main into fix/issue-102-format-version 2026-10-05 18:54:03 -04:00
Brandon Thomas 8506fecf5f Merge pull request #124 from storytold/fix/issue-101-torn-append
Issue #101: catalog.log: cut a partial append back before retrying
2026-10-05 18:53:49 -04:00
Brandon Thomas 0f49ebf8f0 Merge main into fix/issue-101-torn-append
# Conflicts:
#	docs/whats-new.md
2026-10-05 18:52:48 -04:00
Brandon Thomas ab86862011 Merge pull request #123 from storytold/fix/issue-92-xmp-merge
Issue #92: XMP: merge into an existing sidecar instead of replacing it
2026-10-05 18:52:25 -04:00
Brandon Thomas 577330a08b Merge main into fix/issue-92-xmp-merge
# Conflicts:
#	crates/engine/src/sidecar.rs
#	docs/whats-new.md
2026-10-05 18:50:42 -04:00
Brandon Thomas 30697cfc26 Merge pull request #122 from storytold/fix/issue-92-rename-shared-sidecar
Issue #92: Rename Photos copies a stem sidecar another file still shares
2026-10-05 18:50:26 -04:00
Brandon Thomas ddbf139e5e Merge main into fix/issue-92-rename-shared-sidecar 2026-10-05 18:49:29 -04:00
Brandon Thomas ee77490dd2 Merge pull request #121 from storytold/fix/issue-105-rename-rollback
Issue #105: Rename Photos reports failed rollbacks; Find Missing matches by content
2026-10-05 18:49:15 -04:00
Brandon Thomas f31a55c8a7 Merge main into fix/issue-105-rename-rollback
# Conflicts:
#	docs/parity.md
2026-10-05 18:48:27 -04:00
Brandon Thomas 7218b990f5 Merge pull request #120 from storytold/fix/issue-95-case-only-rename
Issue #95: Rename Photos: case-only renames never overwrite another file
2026-10-05 18:48:06 -04:00
Brandon Thomas d16d9aadeb Merge main into fix/issue-95-case-only-rename 2026-10-05 18:47:18 -04:00
Brandon Thomas 1caa04acbc Merge pull request #119 from storytold/fix/issue-98-thumb-cache-own-files
Issue #98: thumbnail cache only lists, counts and deletes its own files
2026-10-05 18:47:03 -04:00
Brandon Thomas bef2060b24 Merge main into fix/issue-98-thumb-cache-own-files 2026-10-05 18:46:59 -04:00
Brandon Thomas 40303b999c Merge pull request #118 from storytold/fix/issue-96-verified-copy-import
Issue #96: Import > Copy verifies each copy like Move
2026-10-05 18:46:42 -04:00
Brandon Thomas ce5488ad99 Merge main into fix/issue-96-verified-copy-import
# Conflicts:
#	docs/whats-new.md
2026-10-05 18:46:20 -04:00
Brandon Thomas f6bc1ae5ae Merge pull request #117 from storytold/fix/issue-106-dng-smart-previews
Issue #106: Convert to DNG and smart previews: verified, atomic writes
2026-10-05 18:46:06 -04:00
Brandon Thomas 6c8a65c337 Merge main into fix/issue-106-dng-smart-previews 2026-10-05 18:45:48 -04:00
Brandon Thomas fe386f4f15 Merge pull request #116 from storytold/fix/issue-93-export-overwrite
Issue #93: Export: never overwrite an original; atomic writes
2026-10-05 18:45:31 -04:00
Brandon Thomas a6f421aafb Merge main into fix/issue-93-export-overwrite 2026-10-05 18:44:02 -04:00
Brandon Thomas 50cd8dc6dc Merge branch 'fix/issue-97-folder-undo' into integrate/safety
# Conflicts:
#	crates/engine/src/lib.rs
#	docs/parity.md
#	docs/whats-new.md
2026-10-05 18:38:55 -04:00
Brandon Thomas cc8b3eb3bb Merge branch 'fix/issue-94-control-close' into integrate/safety
# Conflicts:
#	docs/whats-new.md
2026-10-05 18:38:13 -04:00
Brandon Thomas a6b402981b Merge branch 'fix/issue-103-settings-files' into integrate/safety
# Conflicts:
#	crates/engine/src/library.rs
#	docs/whats-new.md
2026-10-05 18:26:38 -04:00
Brandon Thomas 4deb5eff04 Merge branch 'fix/issue-100-open-failure' into integrate/safety 2026-10-05 18:26:38 -04:00
Brandon Thomas 6e2408ff7d Merge branch 'fix/issue-99-library-lock' into integrate/safety
# Conflicts:
#	crates/catalog/src/lib.rs
2026-10-05 18:26:38 -04:00
Brandon Thomas 66be31e3d8 Merge branch 'fix/issue-102-format-version' into integrate/safety 2026-10-05 18:26:19 -04:00
Brandon Thomas 332f37a5c7 Merge branch 'fix/issue-101-torn-append' into integrate/safety
# Conflicts:
#	docs/whats-new.md
2026-10-05 18:26:19 -04:00
Brandon Thomas aa1af6d5e9 Merge branch 'fix/issue-92-xmp-merge' into integrate/safety
# Conflicts:
#	crates/engine/src/sidecar.rs
#	docs/whats-new.md
2026-10-05 18:26:19 -04:00
Brandon Thomas 963912b231 Merge branch 'fix/issue-92-rename-shared-sidecar' into integrate/safety 2026-10-05 18:25:06 -04:00
Brandon Thomas 4174ce8c57 Merge branch 'fix/issue-105-rename-rollback' into integrate/safety
# Conflicts:
#	docs/parity.md
2026-10-05 18:24:59 -04:00
Brandon Thomas f28e1c362e Merge branch 'fix/issue-95-case-only-rename' into integrate/safety 2026-10-05 18:24:34 -04:00
Brandon Thomas 70d800ba68 Merge branch 'fix/issue-98-thumb-cache-own-files' into integrate/safety 2026-10-05 18:24:33 -04:00
Brandon Thomas 33ef91a123 Merge branch 'fix/issue-96-verified-copy-import' into integrate/safety
# Conflicts:
#	docs/whats-new.md
2026-10-05 18:24:33 -04:00
Brandon Thomas f1b29cfe4a Merge branch 'fix/issue-106-dng-smart-previews' into integrate/safety 2026-10-05 18:24:23 -04:00
Brandon Thomas 130b807e1b Merge branch 'fix/issue-93-export-overwrite' into integrate/safety 2026-10-05 18:24:22 -04:00
arata1972andCodex 43f4e24355 M11: separate Sony ARW colour and tone and preserve decoded source facts
Co-authored-by: Codex <noreply@openai.com>
2026-10-06 07:16:15 +09:00
Brandon ThomasandClaude Opus 5.5 5e618c553a Issue #100: Ask what to do when the library can't be opened, never a silent demo session
When open_library failed, the desktop app (and the web build) quietly ran
an in-memory session seeded with the demo photos: it looked like a reset
library and everything done in it was lost at quit.

- The session now starts empty and in memory (no demo photos) and
  LightcraftApp::library_problem shows a blocking window with the path and
  the error: Try Again, Choose Another Library… (app.openLibrary), Continue
  Without Saving, Quit. Command-line files are imported once the user has
  chosen.
- A temporary session shows a banner under the top bar (with Open
  Library…) for as long as it lasts; it has no library, so nothing is
  written anywhere. Opening a library ends it.
- ui.inspect → libraryProblem; widget ids for automation; documented in
  control-protocol.md.
- Web build: same window (Try Again disabled: reloading the page retries).
- Headless UI test: locked library → window → Continue Without Saving →
  banner, nothing written to the library → lock released → Try Again opens
  it. Checked in the real app over the control channel too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 18:13:00 -04:00
Brandon ThomasandClaude Opus 5.5 cbebabccad Issue #102: Catalog format version 2; refuse newer libraries untouched
main added Op::SetBrowsed, Catalog.browsed and Photo.local_baseline while
the snapshot still said version 1, so v0.2.0 read such a library in part
(unknown ops as log damage, unknown fields dropped on its next snapshot).

- journal::VERSION = 2 (now public), with a format table and rules in the
  journal docs; a test with an exhaustive match over Op makes adding a
  variant fail until it is assigned a version.
- Journal::open reads the snapshot header first and refuses a newer
  version with CatalogError::Newer ("written by a newer version of
  LightCraft"), even when its catalog wouldn't parse.
- A log record whose CRC matches but whose op doesn't parse is "newer",
  not damage or a torn tail: refused before any file is modified.
- Opening an older (or snapshot-less) library writes a snapshot in this
  format before anything is appended (LoadReport::upgraded_from); a new
  library gets one at once, so v0.2.0 refuses it instead of reading a bare
  log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 18:13:00 -04:00
Brandon ThomasandClaude Opus 5.5 08c2acda26 Issue #101: catalog.log: cut a partial append back before retrying
A failed append (disk full, dropped share) could leave part of its batch in
catalog.log; the retry (same seqs) then shared a line with the fragment and
the next load treated it as mid-log damage, setting later edits aside.

- Journal::append cuts the log back to the last whole record on error (or,
  if that fails too, before the next append).
- FsStore drops a failed append handle so the next attempt reopens the file;
  truncate of a never-created file is a no-op.
- The loader recognises a CRC-valid record glued after a fragment (logs
  written by older builds) and skips only the fragment.
- log_bytes now counts the newline added to an unterminated last record.
- close_library writes the closing snapshot even when appending fails; the
  snapshot absorbs the queued ops (Journal::snapshot_with_unlogged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 18:13:00 -04:00
Brandon ThomasandClaude Opus 5.5 1a05c3eed2 Issue #103: Settings files: keep damaged ones, never overwrite unread ones, write ui.json atomically, confirm quit with unsaved changes
- Library settings (prefs.json, presets.json, view.json): only a missing
  file means "defaults". A file that doesn't parse is kept as
  <name>.corrupt-<unix time> before the defaults are used; one that can't
  be read (locked, I/O error) is never written this session (save_prefs
  refuses, presets/view saves skip it). Warnings go to library.info →
  settingsWarnings and Session::take_library_warnings; the UI shows each
  once in a window (ui.inspect → notices, button:noticeOk).
- Desktop ui.json (holds the library location): written atomically (temp +
  fsync + rename + dir fsync), saved as soon as the library path changes and
  every few seconds when it changed, failures reported; a damaged file is
  renamed to ui.json.corrupt-<time> and reported, an unreadable one is not
  overwritten.
- Quit/close with changes only in memory: retry the save (log, then a
  snapshot of everything), and if it still fails cancel the close and ask:
  Try Saving Again / Quit Anyway / Cancel (ui.inspect → quitPrompt).
- Tests: engine (damaged prefs kept, unreadable presets/prefs untouched),
  desktop (damaged ui.json kept, atomic write), headless UI (quit prompt
  buttons, settings notice).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 18:12:26 -04:00
Brandon ThomasandClaude Opus 5.5 ee4b78f646 Issue #99: Library: exclusive lock so two processes can't open one library
Two processes on one library each kept their own catalog and seq counter;
the last snapshot silently dropped the other's edits.

- lightcraft_catalog:🔒 LibraryLock holds an exclusive OS lock
  (std File::try_lock: flock / LockFileEx) on <library>/catalog.lock for
  the session; who holds it goes to catalog.lock.owner (separate file: a
  Windows lock also blocks reads). Released on drop and by the OS when the
  process dies, so leftover files from a crash never lock anyone out. File
  systems without lock support open unprotected with a warning.
- Session::open_library takes the lock before reading anything and fails
  with EngineError::LibraryInUse ("this library is already open in
  LightCraft 0.x (process N on host)"); reopening the session's own
  library hands its lock over.
- lightcraft-cli (mcp/run/snapshot --library) adds how to drive the
  running app instead (--control + mcp --connect).
- Tests: in-process, leftover files, a killed child process, two engine
  sessions, CLI e2e (mcp --library holds it, run --library is refused).
  Existing tests that reopened a library while it was open now drop the
  first session.
- docs: parity row LRC-LIB-CATALOG-LOCK, mcp.md, web.md (no guard across
  browser tabs yet), whats-new.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:48:34 -04:00
Brandon ThomasandClaude Opus 5.5 d592b29d12 Issue #97: Rename Folder / Move Folder are undo steps
The folder move and the photos' relinks form one undo step: undo renames
the folder back on disk (files and sidecars along) and restores the
paths, redo repeats it. A destination taken in the meantime is never
overwritten; the step is refused, reported and stays on the stack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:46:30 -04:00
Brandon ThomasandClaude Opus 5.5 cddcb905e0 Issue #94: Control: close the connection on malformed input
A line that is not a JSON object with a string `method` (HTTP request
line, invalid UTF-8, non-object JSON, >4 MiB line) now gets one error
reply and the connection is closed, so a browser's cross-origin fetch
can't smuggle a command in its body. Connections are capped at 16.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:46:27 -04:00
Brandon ThomasandClaude Opus 5.5 ebc73bb7b4 Issue #107: Web build: data-safety fixes for libraries kept in the browser
- File > Back Up Library... (web): a zip of the catalog files and every
  stored original, built entry by entry; Restore Library from Backup...
  reads it into a new library folder and switches only when complete
  (the previous library stays in storage).
- Failed flushes to browser storage make the catalog Store refuse
  writes, so changes show as unsaved (top-bar warning) and are retried
  every 2 s instead of being lost silently.
- A picked/dropped photo whose bytes can't be stored is not added.
- One tab per library (Web Locks); a second tab shows a message.
- Notices: storage not persistent, library failed to open (temporary
  session), experimental build; ?reset asks first; a panic shows a
  message instead of a frozen page; long toasts wrap.
- Hosting: _headers/.htaccess no longer mark the unhashed .js/.wasm
  immutable; HOSTING.md rewritten for the actual build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:46:21 -04:00
Brandon ThomasandClaude Opus 5.5 e96a2750a1 Issue #96: Import > Copy verifies each copy like Move
Copy import used exists() then std::fs::copy: no sync, no comparison, and
a file appearing at the chosen name in between was replaced. It now goes
through import_move::copy_new, which reuses Move's copy_verified: a new
file (create_new, retrying -1, -2… on AlreadyExists), synced and compared
byte for byte with the source; a copy that fails or differs is removed and
reported as a failed import. Copy as DNG's copy step uses the same path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:22:35 -04:00
arata1972andCodex 509d5843dd M16: integrate expanded Japanese UI with upstream language settings
Co-authored-by: Codex <noreply@openai.com>
2026-10-06 06:20:23 +09:00
Brandon ThomasandClaude Opus 5.5 771b0b98d7 Issue #106: Convert to DNG and smart previews: verified, atomic writes
- Convert to DNG / Copy as DNG: the encoded DNG must decode to the same raw
  data, then goes through safe_file::write_new_unique (temp file, sync,
  read-back, hard-linked to the first free <stem>.dng / -2 / -3 name, never
  replacing a file). On failure no DNG is left, the photo is not relinked
  and the raw copy is kept.
- Photo Merge writes its DNG the same way.
- Smart previews are written atomically; smart::is_valid (header + complete
  JPEG) decides "built", has_smart_preview and the offline fallback, so a
  proxy cut short is rebuilt (reported as `repaired`); moving proxies to a
  new folder never deletes a good one in favour of a damaged copy and
  copies across drives atomically.

Stacked on the #93 branch (shared lightcraft_catalog::safe_file helper).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:17:41 -04:00
Brandon ThomasandClaude Opus 5.5 f4df914731 Issue #92: Rename Photos copies a stem sidecar another file still shares
Renaming IMG_1.CR3 alone took the shared IMG_1.xmp along and left
IMG_1.JPG without it. A Stem-named sidecar that a sibling photo file
still uses is now copied (independently, never linked) instead of moved;
the last file renamed takes it. A rollback removes the copy.

Stacked on fix/issue-105-rename-rollback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:12:25 -04:00
Brandon ThomasandClaude Opus 5.5 882a059fac Issue #92: XMP: merge into an existing sidecar instead of replacing it
Saving (Cmd+S or auto-write) now merges LightCraft's properties into an
existing sidecar (lightcraft_meta::merge_xmp): owned properties (rating,
label, dc/IPTC/rights fields, lc:*) are replaced or removed, capture time
and GPS only when LightCraft has a value; everything else (crs:,
xmpMM:History, unknown namespaces, packet wrapper) is kept byte for byte.
The result is re-parsed and checked before it is written (temp + fsync +
rename). A sidecar that isn't valid XMP is copied to .xmp.bak-<time>
(never overwriting) before it is replaced; one that can't be read at all
is left alone and the save fails.

Shared stems: when several catalogued files map to one stem sidecar
(IMG.CR3 + IMG.JPG), it belongs to the raw (else the first by name) and
the others write and read <file>.xmp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:08:18 -04:00
Brandon ThomasandClaude Opus 5.5 46732cb0fe Issue #93: Export: never overwrite an original; atomic writes
- lightcraft_catalog::safe_file: one tested helper for crash-safe writes
  (temp file in the same folder, sync, rename, folder sync; temp removed on
  failure), write_new/write_new_unique that never replace a file and read
  the result back, same_file, and a per-thread fault injection for tests.
- engine::originals::OriginalGuard: refuses a write whose target is a
  catalogued photo's original or XMP sidecar (by file identity), whatever
  the conflict policy or exact path. Used by run_batch (export, Export with
  Previous, control/MCP exact paths), ui.render/ui.screenshot paths, merge
  previewPath, MCP render_photo/screenshot paths (new query command
  export.checkTarget) and lightcraft-cli render.
- The conflict policy now covers an export's sidecars together with the
  file (Unique gives both the same free name, Skip skips when either is
  taken).
- export::write_file replaces the plain std::fs::write in the desktop app,
  CLI, MCP and engine writers; XMP sidecar writes use the shared helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:03:07 -04:00
arata1972andCodex 05f242c445 M11: estimate missing ARW colour from guarded camera-preview correspondences
Co-authored-by: Codex <noreply@openai.com>
2026-10-06 05:55:20 +09:00
Brandon Thomas 572b55a44a Merge pull request #110 from midasdf/feat/japanese-interface-and-watermarks
Add Japanese menus and vertical Japanese text watermarks
2026-10-05 16:49:18 -04:00
Brandon Thomas bd10f89d02 Merge main into PR #110 2026-10-05 16:48:57 -04:00
Brandon ThomasandClaude Opus 5.5 fb5b18843f Issue #105: Rename Photos reports failed rollbacks; Find Missing matches by content
Rollbacks in move_file, apply_rename and undo/redo no longer ignore
errors: every move that can't be undone is reported (old -> new), and the
catalog follows the files that kept their new names (an undoable partial
rename; a logged change for undo/redo), so nothing shows as missing.
A file that can't go back keeps the sidecars that moved with it.

library.findMissing: same name + size confirmed by the stored content
hash first, then renamed files by size + content hash, then a unique
name + size match; several look-alikes are skipped and reported in
'ambiguous' instead of guessed. Files already used by a photo are skipped.

Stacked on fix/issue-95-case-only-rename (same functions).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:47:50 -04:00
Brandon ThomasandClaude Opus 5.5 dfbb1ad935 Issue #95: Rename Photos: case-only renames never overwrite another file
Whether an existing target is the photo itself is decided by file identity
(dev+inode on Unix; an exact directory entry check elsewhere), not by a
lower-case name comparison, so on case-sensitive volumes img_1.JPG next to
IMG_1.JPG is a collision (-1 suffix). Moves use a no-replace rename (hard
link + unlink where supported) and case changes go through a unique temp
name. Tests model case-sensitive and case-insensitive volumes and check
the real disk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:36:35 -04:00
arata1972andCodex 87e0181690 M16: preserve upstream defaults with an opt-in Japanese local build
Co-authored-by: Codex <noreply@openai.com>
2026-10-06 05:36:19 +09:00
Brandon ThomasandClaude Opus 5.5 0fd0ffb783 Issue #98: thumbnail cache only lists, counts and deletes its own files
scan() now accepts only <2 hex>/<32 hex>.jpg (matching shard) and the
cache's own <32 hex>.tmp<id> temp files, so prune/clear never delete and
size() never counts foreign files in a shared thumbs/ folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:23:32 -04:00
Brandon Thomas e99aa61fc4 Merge pull request #111 from storybored379/fix/unicode-timezone
Metadata: gracefully ignore invalid Unicode timezone offsets
2026-10-05 16:22:21 -04:00
Brandon ThomasandClaude Opus 5.5 58626b982d Fonts: replace Shippori Mincho with BIZ UDMincho (no Adobe-derived glyphs)
Shippori Mincho's name table credits 'FONTDASU / Google Inc. / but / Adobe' — its kanji
come from Adobe's Source Han Serif, which our asset rules exclude even under the OFL
(CLAUDE.md: no Source Han fonts). BIZ UDMincho (Morisawa Inc., design TypeBank Co., Ltd.;
OFL-1.1, from google/fonts at 63833b7) is a Mincho of its own with no Adobe glyphs, and
smaller (6.2 MB vs 8.7 MB). Attribution, licence text and NOTICE updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:19:38 -04:00
arata1972andCodex 7c799a9acf M16: add Japanese UI localization and isolated local macOS packaging
Co-authored-by: Codex <noreply@openai.com>
2026-10-06 05:15:04 +09:00
midasdf e00025aaf6 i18n: add Japanese menus and vertical Japanese watermarks 2026-10-06 01:07:26 +09:00
Brandon Thomas 294012742e Merge pull request #108 from storytold/pr/freebsd-ci-sync
FreeBSD CI: match the shared template
2026-10-05 10:19:07 -04:00
Brandon ThomasandClaude Opus 5.5 4d13061580 FreeBSD CI: match the shared template (git + safe.directory, 4 vCPUs, pinned vmactions/freebsd-vm)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 09:45:24 -04:00
Brandon Thomas 63380829b1 Merge pull request #91 from storytold/pr/freebsd
FreeBSD: build and test the workspace in a FreeBSD VM
2026-10-05 09:30:12 -04:00
Brandon ThomasandClaude Opus 5.5 62e16d7adc FreeBSD CI: 4 vCPUs, git in the VM, pinned vmactions/freebsd-vm
The headless UI snapshot test settles for at most 120 s; on a 2-vCPU VM
under the full test load it timed out mid-render.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 08:58:05 -04:00
Brandon ThomasandClaude Opus 5.5 d47e941a0c docs: point agents to the shared test corpora (#90)
Real-file test oracles live in storytold/photocraft-corpus, explained in
craftrules standards/test-corpora.md. Never commit large binary fixtures
here; fetch them pinned and sha256-verified.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 08:24:23 -04:00
Brandon ThomasandClaude Opus 5.5 83fb206a29 FreeBSD: build and test the workspace in a FreeBSD VM
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 08:23:26 -04:00
Brandon Thomas 3567980442 Merge pull request #89 from storytold/chore/ignore-target-dirs
Ignore /target-*/ build dirs
2026-10-05 07:38:00 -04:00
Brandon ThomasandClaude Opus 5.5 8cd791100b Ignore /target-*/ build dirs
Agents sometimes build into target-<name>/ next to target/; ignore those too so a
'git add -A' can never commit build artifacts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 07:37:49 -04:00
Brandon Thomas ff1e51e4d4 Merge pull request #88 from storytold/fix/issue-85-cr2-cfa
Issue #85: Raw: CR2 colour pattern from the file's CR2CFAPattern tag, not a constant
2026-10-05 07:37:19 -04:00
Brandon Thomas c05f2de251 Merge main into fix/issue-85-cr2-cfa
# Conflicts:
#	docs/parity.md
2026-10-05 07:37:00 -04:00
Brandon ThomasandClaude Opus 5.5 e87ee8e36d Issue #85: Raw: CR2 colour pattern from the file's CR2CFAPattern tag, not a constant
The Bayer layout of Canon CR2 files differs by model (GBRG on the 7D, 50D,
60D, 550D, 600D, 1200D, 1300D, 5D Mark II, 1D Mark IV; RGGB on the 40D,
5D Mark III, 6D, 5DS R and others). Read it from raw IFD tag 0xc5e0
(CR2CFAPattern: 1 RGGB, 2 BGGR, 3 GBRG, 4 GRBG), anchored at the full
decoded sensor; fall back to RGGB when the tag is missing. No model table.

Checked on 35 decoded CC0 raw.pixls.us CR2s (35 models): the tag agrees
with the mosaic's green-diagonal statistic on every file; the SensorInfo
border parity does not predict the layout. Corpus manifest gains the
reporter's samples; a corpus test asserts each layout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 07:24:26 -04:00
Brandon Thomas efef3a5df1 Merge pull request #87 from storytold/docs/roadmap-assessment
Docs: honest parity assessment — where we stand, where we're going
2026-10-05 06:49:16 -04:00
Brandon ThomasandClaude Opus 5.5 b50fbf1285 Docs: honest parity assessment — where we stand, where we're going
ROADMAP.md: new 'Where we stand' (by dimension and by kind of user; checklist 79% vs
~60-70% as a day-to-day Lightroom replacement) and 'Where we're going' (ordered priorities:
camera colour calibration, CR3 and compressed raws, verified camera coverage, render fidelity
suite, lens profiles, AI model strategy, then HDR / Classic modules / video). Current
parity numbers, M11 status, risks (raw-source decision of 2026-10-05), log.

docs/parity.md: new rows LR-PROF-CAMERACOLOR, LR-IMP-CAMERA-COVERAGE (P0) and
LR-BEHAV-RENDER-FIDELITY (P1) so quality gaps are tracked; Top gaps reordered by user impact.

README.md: honest status summary above the feature table; stale rows fixed (compressed NEF,
DNG/original export, image watermark, Move import), camera colour and AI/HDR/video rows.

AGENTS.md / CLAUDE.md: agents read the assessment before picking work, and downgrade
✅ rows they find wrong or incomplete.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 06:48:54 -04:00
Brandon Thomas 92bd3aa688 Merge pull request #86 from storytold/fix/issue-10-nef-compressed
Issue #10: Raw: Nikon compressed NEF (lossless + lossy)
2026-10-05 06:08:28 -04:00
Brandon ThomasandClaude Opus 5.5 6592e62912 Issue #10: Raw: Nikon compressed NEF (lossless + lossy)
Decode Nikon Huffman-compressed NEF (compression 34713): lossless and lossy
(type 1 and 2) at 12 and 14 bits, with the maker-note 0x96 linearization
curve, predictor seeds and either byte order. Clean-room: 0x96 layout and
encoding-table sizes from prose write-ups (Clevy, Claff); the fixed Huffman
tables were recovered by black-box analysis of CC0 raw.pixls.us samples
(compressed vs uncompressed shots of the same scene) and verified on ~45
files from ~35 bodies. Lossy-after-split files stay preview only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 05:58:31 -04:00
Brandon Thomas 8532d4ea56 Merge pull request #84 from storytold/fix/issue-78-gpu-black-export
Issue #78: GPU export: respect device limits, fall back to CPU on GPU errors
2026-10-05 05:09:57 -04:00
Brandon ThomasandClaude Opus 5.5 2218ca5a4c Issue #78: GPU export: respect device limits, fall back to CPU on GPU errors
A GPU render no longer hands back an image it cannot vouch for:
- set the wgpu device-lost callback (wgpu never reports a lost device as an
  error, so a driver reset went unnoticed); readback waits with a timeout and
  checks the poll result and map callback instead of blocking forever
- each render runs in OutOfMemory/Validation/Internal error scopes; errors
  fail that render (OOM and limit failures for that render only, device
  errors for the process) and it is redone on the CPU
- every buffer is checked against the device limit when created (no
  validation error, no process-wide disable); LIGHTCRAFT_GPU_LIMITS=webgpu|<MiB>
  creates the device with lower limits to reproduce small adapters
- the output buffer is cleared and the per-pixel kernel writes alpha 255:
  any unwritten pixel (work silently dropped, the issue's all-black export)
  fails the render; an all-black result is re-rendered on the CPU
- renders submit per stage and every 16 M invocations, the per-pixel stage
  runs in ~4 MP bands: no long submission for a slow iGPU's watchdog to reset
- ui.inspect perf.gpuReason / perf.gpuFallback, app.gpu reason/lastFallback,
  System Info and Settings say why the GPU is off (e.g. llvmpipe skipped) and
  why the last render fell back

Tests: crates/gpu/tests/fallback.rs (limit, mid-render limit, validation
error, dropped work, black photo), engine export falls back to the CPU image,
banded 3000x2000 GPU/CPU equivalence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 05:06:24 -04:00
Brandon Thomas 14053b2e40 Merge pull request #83 from storytold/fix/local-record-eviction
Local: forget untouched browse records of folders not browsed for 30 days
2026-10-05 02:59:23 -04:00
Brandon Thomas 1f2f25356a Merge main into fix/local-record-eviction
# Conflicts:
#	crates/engine/src/cmd/library.rs
#	crates/engine/src/library.rs
#	docs/whats-new.md
2026-10-05 02:59:05 -04:00
Brandon Thomas 2ed6af155d Merge pull request #82 from storytold/fix/persist-errors
Persistence: a failed save fails the command (change kept in memory, queued, retried)
2026-10-05 02:58:49 -04:00
Brandon Thomas cd35ebdace Merge pull request #81 from storytold/pr/release-0.2.0
Release: LightCraft v0.2.0
2026-10-05 02:41:21 -04:00
Brandon ThomasandClaude Opus 5.5 65e638ce8a Release: LightCraft v0.2.0
Bump the workspace version 0.1.1 -> 0.2.0 (cargo xtask version set 0.2.0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 02:41:10 -04:00
Brandon ThomasandClaude Opus 5.5 b14cf58630 Local: forget untouched browse records of folders not browsed for 30 days
Local browse records can be most of a large catalog (85k photos, 90 %
Local: 313.5 MB snapshot). Untouched Local records of folders whose last
browse is older than library.preferences forgetLocalDays (default 30,
0 = never) are now removed from the catalog when the library opens (and by
library.forgetLocal {dryRun?, days?}), as journaled, non-undoable ops, then
compacted in the background. Files and XMP sidecars are never touched;
browsing the folder again catalogues them afresh.

Untouched = a Local file record, not deleted, not a virtual copy, whose
state still hashes to the fingerprint taken when the browse catalogued it
(any change keeps it); records from before fingerprints need every
user-editable field empty/default. Kept too: records in albums (or as
cover), stacks, sources of virtual copies, visible/selected/edited ones,
ones referenced by undo/redo, ones with a smart preview.

Per-folder last-browsed times live in the catalog (Op::SetBrowsed,
normalised with folder_key; re-stamped at most hourly). Old catalogs load
unchanged; folders without a time are stamped now, so nothing is forgotten
on the first open after the upgrade.

persist_bench (85k photos, 76.5k Local, every 7th folder recent, 1 in 50
rated): 63.8k forgotten, plan 162 ms, snapshot 313.5 MB -> 94.9 MB.
Settings -> Performance -> Local folders.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 02:26:49 -04:00
Brandon ThomasandClaude Opus 5.5 104444edcb Persistence: a failed save fails the command (change kept in memory, queued, retried)
A command whose journal append fails now returns EngineError::NotSaved
("saved in memory but not written to disk: <reason>; LightCraft will retry")
instead of OK. The change stays applied and undoable, its ops stay queued and
are written by the next save (any command, or the frame loop with a 2 s
backoff). Queries and commands that change nothing are unaffected; a failed
compaction is reported in lastError, not as a failed command (and a failed
synchronous snapshot now backs off like a background one).

Session::unsaved(), library.info unsavedOps/unsavedError, ui.inspect unsaved;
the desktop UI toasts the failure and shows a warning on the top bar's cloud
icon until a save succeeds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 02:26:49 -04:00
Brandon Thomas b312319c78 Merge pull request #80 from storytold/pr/unsafe-helper-crate
Never crash: move the one unsafe FFI call into crates/sysmem
2026-10-05 02:06:26 -04:00
Brandon ThomasandClaude Opus 5.5 1ef74a787d Never crash: move the one unsafe FFI call into crates/sysmem
The macOS malloc_zone_pressure_relief call (returns freed allocator pages
after raw decodes) lived in identical copies inside the desktop app and the
CLI. It now lives in lightcraft-sysmem, an isolated L0 helper crate and the
workspace's only crate allowed unsafe: one #[allow(unsafe_code)] module, a
SAFETY comment, a safe API that returns bytes released (0 off macOS), and
tests (repeat calls, release after large frees, concurrent calls). Both apps'
alloc_release.rs now just install the safe hook, and their crate roots gain
#![forbid(unsafe_code)], so every production crate except sysmem forbids it.
AGENTS.md/CLAUDE.md name the exception, per craftrules standards/never-crash.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 02:06:08 -04:00
storybored379andOpenAI Codex 35e38661a7 Metadata: gracefully reject invalid Unicode timezone offsets
Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-10-04 21:11:55 -07:00
Brandon Thomas 5525ae667a Merge pull request #77 from storytold/fix/issue-29-move-import
Issue #29: Import: Move mode with rename and template folders
2026-10-04 16:15:40 -04:00
Brandon Thomas f4316dcfb8 Merge main into fix/issue-29-move-import
# Conflicts:
#	docs/parity.md
2026-10-04 16:15:25 -04:00
Brandon Thomas 116c26d8ee Merge pull request #76 from storytold/fix/issue-35-grid-frames
Issue #35: Grid: unchanged frames reuse view data and visit only nearby rows
2026-10-04 16:14:14 -04:00
Brandon Thomas 8061e0019e Merge pull request #75 from storytold/fix/issue-37-persistence
Issue #37: Catalog: compact the log on a background thread with ordered journal boundaries
2026-10-04 16:14:02 -04:00
Brandon ThomasandClaude Opus 5.5 4788f3bfff Issue #37: Catalog: compact the log on a background thread with ordered journal boundaries
At the 2,000-record threshold the UI thread serialised and fsynced the whole catalog (1-2 s
at 85k photos after streaming). Now `persist()` starts `Journal::snapshot_in_background`:

- the worker writes `catalog.snap` for op N from a clone of the catalog (photos are Arc-shared,
  so the clone is cheap) through `Store::background_writer` (a second FsStore handle; MemStore
  opt-in; none on wasm/web, which stays synchronous);
- appends go on as before (fsynced before every command returns) and are also kept in memory
  as the "tail" (records > N);
- `Journal::poll` (from `persist` and the per-frame `persist_if_dirty`) sees the worker done and
  atomically replaces the log with exactly the tail.

Crash points: before the snapshot rename -> old snapshot + whole log; between the rename and the
log rewrite -> new snapshot + whole log (records <= N skipped as stale, existing rule); after ->
new snapshot + tail. A failed snapshot keeps the log whole and retries after another
max_records records (no retry storm). At most one snapshot in flight; a synchronous snapshot
(close, Optimize Library), reopening a library and dropping the Journal first wait for it, so an
older snapshot can never replace a newer one. On-disk format unchanged.

Tests (crates/catalog/src/tests_background.rs, a gated store holds the worker mid-flight and
"crashes" by copying the files): edits during a background snapshot survive crashes at each
point and the trimmed log holds exactly seq N+1.. in order; failure loses nothing and backs
off; sync snapshot waits (no regression); drop waits; real FsStore with a leftover partial
temp file; a proptest over interleavings of edits / start / land / finish / sync snapshot /
crash. Mutations checked: trimming the log to empty, or not waiting in `snapshot`, fail them.
Engine: `threshold_compaction_runs_in_the_background` (crash copy mid-run, reopen without close).
`library.info` -> persistence gains snapshotRunning, failedSnapshots, maxBlockingMs and
lastSnapshot.{background, blockingMs}.

persist_bench, same session (load avg ~160), UI-thread time per threshold compaction:
  85k photos: synchronous 0.85-1.7 s (median 1.25 / 1.34 s) -> background: blocking median
              45 / 74 ms (clone 1.3-154 ms + log rewrite 9-23 ms); worker 1.2-2.4 s; 230-380
              single-op appends during it at median 4-6 ms (same as without)
  10k photos: synchronous 72-233 ms -> blocking median 12.5 / 14 ms

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 16:01:28 -04:00
Brandon ThomasandClaude Opus 5.5 ae1cbbe87b Issue #37: Catalog: stream the snapshot into the temp file instead of building two strings
Journal::snapshot serialised the catalog into a String and then copied it into a second
`format!`-ed String before writing (2x the snapshot size in memory, ~620 MB at 85k photos).
It now streams `{"format":…,"seq":N,"catalog":` + serde_json::to_writer + `}\n` through a
1 MiB BufWriter into the store (new `Store::write_atomic_with`; FsStore writes the temp file
directly, then flush + sync_all + rename + dir fsync as before; other stores buffer, as before).

Durability and format are unchanged: same temp + fsync + atomic rename; a failed write leaves
the old snapshot (and now removes the temp file); the bytes are identical to the old format
(test `streamed_snapshot_keeps_the_on_disk_format`, which also loads a legacy snapshot), so
no version bump. Also tested: `failed_streamed_write_keeps_the_old_file`.

persist_bench, 85k photos (310 MB snapshot), compaction at the 2,000-record threshold,
before/after binaries alternated in the same session (load avg ~130-170), building+compacting
only (PERSIST_BENCH_NO_LOAD):
  compaction median: before 3.5 s / 4.6 s, after 1.9 s / 2.5 s
  peak RSS:          before 926 MB / 1263 MB, after 308 MB / 366 MB
  10k photos peak RSS: 149-235 MB -> 61-68 MB (times too noisy to compare)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:43:13 -04:00
Brandon ThomasandClaude Opus 5.5 db6242d2d3 Issue #37: Catalog: instrument persistence (append, snapshot stages) and whole-frame update time
- Journal::stats(): last/max append time (encode + write + sync_data), snapshot count, last
  snapshot by stage (serialise, write+sync+rename, log reset, bytes, records), slowest
  snapshot. Reported by `library.info` → `persistence`; printed per write to stderr under
  LIGHTCRAFT_PROFILE.
- UI: `perf.frame_ms` only covers layout (`LightcraftApp::ui`); `logic()` (control channel,
  shortcuts, render polling, `persist_if_dirty`) ran untimed. Add `logicMs`, `updateMs`
  (logic + layout) and `maxUpdateMs` to `ui.inspect` → perf and a "Frame update" row to
  Help ▸ System Info.
- `crates/catalog/examples/persist_bench.rs`: synthetic 1k/10k/85k-photo catalogs (90 % Local
  browse records) on FsStore: append latency, compaction at the 2,000-record threshold, load.

Baseline (release, load avg ~240, so absolute times are inflated; median of 3 compactions):
  1k photos  (3.6 MB snap): append 7 ms, compaction 31 ms (serialise 5, write+sync 12, reset 14)
  10k photos (36.5 MB):     append 15 ms, compaction 528 ms (serialise 463, write+sync 56)
  85k photos (310 MB):      append 12 ms, compaction 4.6 s (serialise 2.2-4.4 s, write+sync
                            0.25-0.8 s); peak RSS 963 MB; load 4.7 s

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:43:13 -04:00
Brandon ThomasandClaude Opus 5.5 73a2c01676 Issue #35: Grid: unchanged frames reuse view data and visit only nearby rows
An unchanged grid frame used to copy every visible id, hash them twice
(date-run and layout cache keys), clone the date runs, build a full
Vec<Cell>, rebuild the stack index, loop over every cell for the
viewport test and, in Local folder views, look up every photo to count
the browsed ones for "Add N to My Photos".

- engine: Session keeps the visible list as Arc<[PhotoId]> with a
  process-unique generation that changes whenever the list is
  recomputed; `visible_shared()` hands both out without copying.
- grid: date runs, layout, stack index and the Local count are cached
  by that generation (GridCache; replaces the id-hashing date_runs and
  grid_layout caches). The layout gains a row index (GridRow), so the
  visible + prefetch rows and the date headers are found by binary
  search; the sticky header too.
- GridStats counters (frames, cells visited, view/layout/stack-index
  builds, last frame time) for the bench and tests.

Measured with the new `cargo run --release -p lightcraft-ui-egui
--example grid_bench` (synthetic library, 1400x900, thumbs 160, missing
files so no decoding is timed, scrolled to the middle, 120 unchanged
frames; machine at load ~200-250, so medians of the whole app frame,
before/after interleaved in the same session):

  photos layout  source  median frame ms  allocs/frame   cells visited
                         before -> after  before -> after  before -> after
     200 photo   all      1.29 -> 1.28    4747 -> 4729     200 -> 93
     200 square  all      1.09 -> 1.09    4771 -> 4753     200 -> 74
  10 000 photo   all      1.52 -> 1.41    5425 -> 4911   10000 -> 100
  10 000 photo   local    1.90 -> 1.42    5475 -> 4948   10000 -> 100
  10 000 square  local    1.62 -> 1.07    5365 -> 4838   10000 -> 76
  85 000 photo   all      2.41 -> 1.29    8996 -> 4729   85000 -> 93
  85 000 square  all      2.29 -> 1.08    9020 -> 4753   85000 -> 74
  85 000 photo   local   17.6-34.0 -> 1.32  9049 -> 4766   85000 -> 93
  85 000 square  local   16.2-25.0 -> 1.10  9073 -> 4790   85000 -> 74

Frame cost no longer depends on the number of results; what is left
(~1 ms, ~4.7k allocs) is drawing the ~75-100 cells near the screen.
Headless snapshots of the demo grid (justified with a sticky header,
after keyboard navigation, square) are byte-identical before/after.

Tests: layout rows cover every cell in order and the binary search
matches a linear scan (square + justified, groups); a 3000-photo
headless library checks that unchanged frames build nothing and visit
< 300 cells, plus date headers, scroll-to-selection, stack badges,
keyboard navigation and a rotation re-laying the grid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:40:19 -04:00
Brandon ThomasandClaude Opus 5.5 589741b21e Issue #29: Import: Move mode with rename and template folders
library.import mode `move` (UI: Transfer ▸ Move, MCP import tool): reuses
Copy's destination, folder organisation (incl. custom templates) and rename
template. Each photo and its XMP sidecars (both namings) are placed without
touching the source (hard link on the same volume, else a new file that is
copied, synced and compared byte for byte; taken names get -1, -2…, never
overwritten). Sources are removed only after the catalog records are
committed and saved, and only when the destination is still intact; failed,
duplicate, unchecked and in-library files keep their sources; sources that
can't be removed are kept and reported (`kept`). Undo removes the catalog
records and leaves the moved files at the destination.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:39:30 -04:00
Brandon Thomas 840b294e23 Merge pull request #60 from storybored379/feat/gpx-tracklog
LRC-MAP-TRACKLOG: Auto-Tag from Tracklog (GPS from a GPX file by capture time)
2026-10-04 15:32:00 -04:00
Brandon Thomas f7361a7d01 Merge main into feat/gpx-tracklog
# Conflicts:
#	docs/parity.md
#	docs/whats-new.md
2026-10-04 15:31:45 -04:00
Brandon Thomas 8011396e91 Merge pull request #51 from storybored379/feat/copyright-status
LRC-LIB-METADATA: Copyright Status, Rights Usage Terms and Copyright Info URL
2026-10-04 15:30:24 -04:00
Brandon Thomas efdd8a4e10 Merge main into feat/copyright-status
# Conflicts:
#	crates/engine/src/tests_xmp.rs
#	docs/parity.md
2026-10-04 15:26:53 -04:00
Brandon Thomas ba5fff320b Merge pull request #59 from storytold/pr/never-crash-ui
Never crash: ui-egui and the web app
2026-10-04 14:24:24 -04:00
Brandon Thomas 528b4b46db Merge pull request #58 from storytold/pr/never-crash-guard
Never crash: last-resort guard around command execution
2026-10-04 14:24:15 -04:00
Brandon Thomas 815be4ef7f Merge pull request #57 from storytold/pr/never-crash-engine-mcp
Never crash: engine (outside export.rs) and mcp
2026-10-04 14:23:58 -04:00
Brandon Thomas 7b75cf4f91 Merge pull request #55 from storytold/pr/never-crash-gpu-merge
Never crash: gpu and merge
2026-10-04 14:23:40 -04:00
Brandon Thomas 3f20f98931 Merge pull request #56 from storytold/pr/never-crash-core
Never crash: color, geom, raster, develop, scenes, preview, pipeline
2026-10-04 14:23:22 -04:00
Brandon Thomas 2700c8a814 Merge pull request #54 from storytold/pr/never-crash-codecs-catalog
Never crash: codecs and catalog
2026-10-04 14:23:05 -04:00
Brandon Thomas 2e2f7ee5ac Merge pull request #53 from storytold/pr/never-crash-raw-tiff
Never crash: raw and tiff decoders
2026-10-04 14:22:47 -04:00
Brandon Thomas 302ed02d94 Merge pull request #52 from storytold/pr/never-crash-steering
Never crash: steering rule and clippy.toml
2026-10-04 14:22:29 -04:00
Brandon Thomas c02b39a15d Merge pull request #73 from storytold/fix/issue-30-import-entry-point
CI: clean under clippy 1.99 (as_chunks for constant chunk sizes, ? in export resize parsing)
2026-10-04 14:01:10 -04:00
Brandon Thomas b56e40e8d5 Merge main into fix/issue-30-import-entry-point
# Conflicts:
#	crates/ui-egui/src/import.rs
#	docs/parity.md
2026-10-04 14:00:55 -04:00
Brandon Thomas 42c19ddd1a Merge pull request #72 from storytold/fix/issue-32-folder-template
Issue #32: Import: custom date-based folder template for Copy
2026-10-04 14:00:34 -04:00
Brandon Thomas 6bf64277c4 Merge pull request #71 from storytold/fix/issue-31-rename-tag-help
Issue #31: Import: tag help beside the Rename field
2026-10-04 14:00:20 -04:00
Brandon Thomas fb47b35837 Merge pull request #70 from storytold/fix/issue-19-sidecar-capture-time
Issue #19: XMP sidecar capture time fills in when the file has none
2026-10-04 14:00:07 -04:00
Brandon Thomas 3b60b5bbc7 Merge pull request #69 from storytold/fix/issue-47-masking-fit
Issue #47: Masking: keep the panel's contents within its width when a mask is selected
2026-10-04 13:59:55 -04:00
Brandon Thomas 3165abf162 Merge pull request #68 from storytold/fix/issue-20-resizable-panels
Issue #20: Panels: resize the left sidebar and the right panel by dragging their edge
2026-10-04 13:59:41 -04:00
Brandon Thomas 00308b50f9 Merge pull request #67 from storytold/fix/issue-33-async-devices
Issue #33: Devices: serve the cached card list and rescan in the background
2026-10-04 13:59:27 -04:00
Brandon Thomas e7722296d3 Merge pull request #66 from storytold/fix/issue-27-quit-last
Issue #27: Menus: keep Quit last in the in-window File menu
2026-10-04 13:59:13 -04:00
Brandon Thomas f1dce86609 Merge pull request #65 from storytold/fix/issue-21-local-roots
Issue #21: Local: kept folders stay roots while browsing below them
2026-10-04 13:59:00 -04:00
Brandon Thomas 9d756b145d Merge pull request #64 from storytold/fix/issue-22-local-path-identity
Issue #22: Local: recognise equivalent folder paths
2026-10-04 13:58:47 -04:00
Brandon Thomas 9698defed6 Merge pull request #63 from storytold/fix/issue-36-smart-album-counts
Issue #36: cache smart-album counts in the sidebar
2026-10-04 13:58:33 -04:00
Brandon Thomas f940e0d03a Merge pull request #62 from storytold/fix/issue-34-missing-scope
Issue #34: Missing Photos checks library photos only
2026-10-04 13:58:21 -04:00
Brandon Thomas 40a599dac0 Merge pull request #61 from storytold/fix/clippy-1.99
CI: clean under clippy 1.99
2026-10-04 13:57:45 -04:00
Brandon ThomasandClaude Opus 5.5 ef57ae208b Never crash: the per-crate deny attribute is the end state, not workspace lints
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 13:57:00 -04:00
storybored379andClaude Opus 5.5 1901a756bb LRC-MAP-TRACKLOG: Auto-Tag from Tracklog — GPS from a GPX track log by capture time
GPX 1.0/1.1 reader in lightcraft-meta (timed track points per segment; interpolation only within a
segment, nearest point within maxGap otherwise). Engine command photo.autoTagTracklog (path or gpx
text, camera time-zone offset, maxGap, replace, dryRun; one undo step) and Photo ▸ Auto-Tag from
Tracklog… with a time-zone prompt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:52:12 -07:00
Brandon ThomasandClaude Opus 5.5 bd2e756b24 Never crash: gpu and merge free of panicking calls
gpu (its render already runs under catch_unwind and falls back to the CPU):
- a failed readback (device lost, out of memory) logs, marks the GPU
  broken and returns zeros, so crate::render discards the result and
  renders on the CPU, instead of panicking;
- an unknown kernel name does the same instead of panic!;
- finish-parameter fields are looked up as Option (debug_assert on an
  unknown name) instead of panic!;
- the warp sampler matches (mode, warp) like the CPU pipeline;
- Buf::raw keeps one documented expect (Some from construction to Drop).

merge:
- pyramid building (push_pull, Blender::finish), camera graph growth,
  bundle adjustment, focal median and alignment use let-else / filter_map
  instead of expect;
- synth::bracket_dngs / pano_views return Result (a missing demo scene
  or a DNG write error is an error); lightcraft-cli synth-merge and the
  engine merge tests handle it.

lightcraft-gpu, lightcraft-merge and lightcraft-cli now #![deny]
unwrap/expect/panic/unimplemented/todo/unreachable; clippy.toml as in the
steering PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 12:34:53 -04:00
Brandon ThomasandClaude Opus 5.5 fc13807f91 Never crash: color, geom, raster, develop, scenes, preview, pipeline
- color: RgbSpace::to_xyz/from_xyz and bradford() no longer expect a
  matrix inverse. Degenerate (collinear) primaries, which readers reject
  before building a space from a file, leave the primaries unscaled /
  give the identity; the constant Bradford and OKLab matrices fall back
  to the identity, which can't happen (round trips are tested).
- pipeline geometry: sample() matches (mode, warp) instead of
  expect("warp"); a warp plan without a warp (sample_plan never makes
  one) samples through the equivalent affine.
- color, geom, raster, develop, scenes, preview and pipeline now #![deny]
  unwrap/expect/panic/unimplemented/todo/unreachable; clippy.toml as in
  the steering PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 12:34:53 -04:00
Brandon ThomasandClaude Opus 5.5 685cf6f2e8 Never crash: ui-egui and the web app free of panicking calls
- import / merge polling: let-else on the task instead of expect("task").
- the merge worker catches a panic and reports "the merge failed
  unexpectedly" instead of dropping its channel, which left "Merging…"
  up forever.
- web: window() returns Option; its ten callers handle a missing window.
- lightcraft-ui-egui and lightcraft-web now #![deny] unwrap/expect/panic/
  unimplemented/todo/unreachable; clippy.toml as in the steering PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 12:33:45 -04:00
Brandon ThomasandClaude Opus 5.5 5218981c7b Never crash: last-resort guard around command execution
Session::execute (the entry point of the UI, CLI, control channel and
MCP) runs each command through lightcraft_engine::guard::catch: a panic
that escapes a command becomes that command's error (EngineError::Other,
shown like any failed command) and the session, catalog and undo history
stay usable. The desktop app installs guard::install_hook, which keeps
the default stderr report and appends the panic location and message to
<temp>/lightcraft-panics.log.

The desktop app crate (already clean) now #![deny]s unwrap/expect/panic/
unimplemented/todo/unreachable; clippy.toml as in the steering PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 12:25:22 -04:00
Brandon ThomasandClaude Opus 5.5 a74b1b32ff Never crash: engine panic sites outside export.rs; mcp denies panics
- merge output_path: a bounded search for a free name instead of
  `for k in 1..` + unreachable!().
- library open: Option::insert returns the new library, no expect.
- relink: a one-op batch unwraps through <[Op; 1]>::try_from, no expect.
- lightcraft-mcp (already clean) now #![deny] unwrap/expect/panic/
  unimplemented/todo/unreachable; clippy.toml as in the steering PR.

crates/engine/src/export.rs keeps one expect (open PR #51 edits that
file), so lightcraft-engine doesn't get the deny attribute yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 12:20:44 -04:00
Brandon ThomasandClaude Opus 5.5 eb378c491a Never crash: codecs and catalog free of panicking calls
- catalog store: the journal appender is a let-else with an io::Error,
  not expect("appender just opened").
- catalog dates: group_label slices with get(), so a non-ASCII key of a
  date's length no longer panics mid-character (regression test).
- codecs jpeg_par: rows come from map_while instead of next().unwrap().
- lightcraft-codecs and lightcraft-catalog now #![deny] unwrap/expect/
  panic/unimplemented/todo/unreachable (one test let-else uses panic!
  instead of unreachable!); clippy.toml as in the steering PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 12:18:11 -04:00
Brandon ThomasandClaude Opus 5.5 b42ca76777 Never crash: raw and tiff decoders free of panicking calls
- ljpeg: a scan naming an undefined Huffman table is an error, not an expect.
- ARW2/RW2 block reads use get() + try_from instead of slice + expect.
- Vendor fallback CFAs use Cfa::bayer_static (literal names, tested)
  instead of Cfa::bayer(..).expect("static").
- dngwrite matches (data, compression) instead of a let-else unreachable!.
- lightcraft-raw and lightcraft-tiff now #![deny] unwrap/expect/panic/
  unimplemented/todo/unreachable; clippy.toml (same as the steering PR)
  keeps them allowed in tests.
- tests/vendor_robust.rs: synthetic NEF, ARW, PEF, SRW, ORF, RW2, CR2 and
  RAF files, truncated at every length and randomly mutated, must decode
  to an error, never panic (no new crash found).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 12:00:26 -04:00
Brandon ThomasandClaude Opus 5.5 a8c3f01e1e CI: clean under clippy 1.99 (as_chunks for constant chunk sizes, ? in export resize parsing)
Rust 1.99 (stable since 2026-09-28) adds clippy::chunks_exact_to_as_chunks and widens
clippy::question_mark, which failed `cargo xtask ci` on main. Same behaviour: the
remainder that chunks_exact skipped is the remainder as_chunks returns separately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:53:43 -04:00
Brandon ThomasandClaude Opus 5.5 4f4426e498 Issue #30: Import Photos… as the clear import entry point
File now opens with Import Photos… (Cmd+Shift+I), Import from Folder… and
Import from Device (command ids file.addPhotos / file.addFolder /
file.addFromDevice unchanged; library.import's label is Import Photos).
The review dialog is titled Import Photos, names its source (a scanned
folder is not added to Local; Local -> Browse Folder... is the persistent
location), labels the transfer mode and explains Add in place vs Copy, and
its button reads Import N Photos.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:44:15 -04:00
Brandon ThomasandClaude Opus 5.5 0ad313d445 Issue #47: Masking: keep the panel's contents within its width when a mask is selected
Selecting a mask added the Add/Subtract/Intersect/Invert/Delete row in
one non-wrapping horizontal layout (~273 pt in a ~220 pt column), and the
Create New Mask grid needed 226 pt: the panel's contents grew wider than
the panel (a 250 pt panel came out 275 pt wide) and were drawn
shifted left, clipped by the photo area. The action row now wraps, the
tile grid sizes its tiles and columns to the available width (four tiles
of 48–52 pt, else three), and a component's name is truncated before its
options button. A headless test checks that every widget stays inside the
right panel at 250, 270, 330 and 520 pt, with and without a selected mask
and with the brush settings open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:41:42 -04:00
Brandon ThomasandClaude Opus 5.5 e95e9259ea Never crash: steering rule, clippy.toml for the no-panic lints
AGENTS.md/CLAUDE.md gain a "Never crash" section ahead of the
non-negotiables (no unwrap/expect/panic/unreachable/todo/unimplemented or
unsafe in non-test code, Result + ?, checked indexing and arithmetic on
input-derived values, bounded recursion, lock poisoning, the last-resort
guard, a regression test per crash fix), linking
craftrules/standards/never-crash.md. README's Contributing list points to it.

clippy.toml allows unwrap/expect/panic/indexing in tests only, so crates
can carry #![deny(clippy::unwrap_used, ...)] without touching their tests.

unsafe_code stays "deny" for now: apps/lightcraft and apps/lightcraft-cli
call malloc_zone_pressure_relief (libSystem) through a two-line FFI block
to return freed memory on macOS, and no safe wrapper crate exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:39:13 -04:00
Brandon ThomasandClaude Opus 5.5 24cf3abb62 Issue #32: Import: custom date-based folder template for Copy
library.import organize now also takes a folder template, e.g.
{date:%Y}/{date:%Y%m%d} -> 2026/20260114 (Organize::Template). The
template's own / (or \) make the folder levels; each level is expanded
with the rename tokens and sanitized, so metadata can never add a level or
climb out of the destination; absolute paths, drive letters, ~ and . / ..
levels are refused. Dated by capture time, else the import time; a level
whose metadata is missing becomes 'unknown'. The import dialog's Folders
menu gains 'Custom template...' with a template field, tag help, live
validation and an example full destination for every mode; the dialog
stays open (with a toast) when the import is refused. The MCP import
helper passes mode/destination/organize/rename through (docs/mcp.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:36:38 -04:00
Brandon ThomasandClaude Opus 5.5 d6476e7497 Issue #20: Panels: resize the left sidebar and the right panel by dragging their edge
Both side panels had an exact width. They are now resizable egui panels
whose width lives in UiState (leftWidth / rightWidth, serde, sanitized on
load) so it survives panel switches, view changes and restarts; egui's
own remembered size is dropped each frame and only a drag of the edge
writes back (measured from the fixed edge to the pointer, so content
wider than the panel can't creep it). Limits: left 200–480 pt, right
250–520 pt, and the photo area keeps at least 360 pt; a narrower window
shrinks the panels without forgetting the chosen width. The bottom bar,
which sits between the panels, now drops Copy/Paste Edit Settings and
then the rating pill when its centre has no room instead of overlapping
the zoom controls. The unused left_w theme token is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:18:04 -04:00
Brandon ThomasandClaude Opus 5.5 a16a81ad08 Issue #21: Local: kept folders stay roots while browsing below them
Browsing a folder inside a listed Local location (built-in or kept) no
longer adds it as a root of its own: it is highlighted inside that
location's tree, which opens down to it once per browsed folder, so its
ancestors and siblings stay reachable. A folder outside every listed
location gets a session-only row that stays while browsing below it and
offers Keep in Local; subfolder rows offer Add to Local. New UI command
local.addRoot keeps a folder in Local (saved with ui.json, so it
survives restarts; it un-hides it too); Browse Folder... uses it.
Removal stays sidebar-only (local.hide).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:15:12 -04:00
Brandon ThomasandClaude Opus 5.5 8be82cacf4 Issue #31: Import: tag help beside the Rename field
rename::TOKENS / DATE_DIRECTIVES / TEMPLATE_NOTES are now the one list of
file-name template tags (a test checks expand_tokens knows each, and that
the rename/import command descriptions mention them all). A Tags toggle
beside the import Rename field (also Rename Photos and export File name)
lists every tag with its meaning and a live example, the {date:...}
directives and the rules (extension appended, blank keeps names, missing
date/metadata, unknown tags stay literal); clicking a tag inserts it at
the text cursor. Unknown tags are flagged under the field. New query
command photo.renameTokens returns the same help as JSON. The import
preview now uses the probe's metadata and the import time for undated
photos, as the import itself does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:04:27 -04:00
Brandon ThomasandClaude Opus 5.5 ce3e2215fe Issue #22: Local: recognise equivalent folder paths
Local compared folder paths as raw strings, so a built-in location built
as "{home}/{sub}" and the same folder returned by library.browse with
native separators could be listed twice. Folder identity now goes
through catalog::query::folder_key (both separators, trailing/doubled
separators, ./.., verbatim and UNC prefixes, drive-letter case; whole
path case-folded on Windows): the sidebar's root list, the highlighted
and expanded-on-the-way folder, saved Browse Folder roots and
local.hide / local.restoreHidden all compare by it. Built-in paths are
joined with the platform separator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 11:04:21 -04:00
storybored379andClaude Opus 5.5 18153682f2 LRC-LIB-METADATA: Copyright Status, Rights Usage Terms and Copyright Info URL
Catalog Meta gets copyright_status (unknown / copyrighted / publicDomain), usage_terms and
copyright_url, read and written as XMP Rights Management fields (xmpRights:Marked, UsageTerms,
WebStatement) on import, sidecars and exports (kept by "copyright only"). photo.setMeta,
copy/paste metadata, metadata presets, a Copyright Status smart-album rule and the Info panel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 08:01:00 -07:00
Brandon ThomasandClaude Opus 5.5 889583433c Issue #36: cache smart-album counts in the sidebar
The album rows read their counts from Caches::album_counts, recomputed
only when the catalog revision changes (photos, metadata and album rules
all bump it) and, while a smart album uses an "in the last..." rule,
when the session clock enters a new minute. Catalog::album_count counts
smart-album matches in place instead of building the id list for its
length. RuleSet/Filter::depends_on_now tell clock-relative rules apart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:51:10 -04:00
Brandon ThomasandClaude Opus 5.5 1a6a9de7ee Issue #33: Devices: serve the cached card list and rescan in the background
Menus are rebuilt every frame, and File → Add from Device called
devices(), which scanned every mount (D:..Z: on Windows) while holding
the cache mutex once its 2 s cache expired. devices() now returns the
last scan at once and starts at most one background rescan when it is
stale; the scan runs without the lock. A changed result calls an
on_change hook (the UI requests a repaint), so hot-plugged cards still
appear. library.devices and Add from Device without a path scan
synchronously (devices_now). Volume roots are probed for DCIM directly
instead of listing them (Windows never lists a root). No threads or
clock on wasm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:50:29 -04:00
Brandon ThomasandClaude Opus 5.5 37ab41478d Issue #19: XMP sidecar capture time fills in when the file has none
A sidecar's exif:DateTimeOriginal / photoshop:DateCreated / xmp:CreateDate
now reaches Photo.captured on import and on Read Metadata from File when
the file has no embedded capture time (embedded wins). On import the
sidecar is read before copying, so the date also files and names copies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:40:54 -04:00
Brandon ThomasandClaude Opus 5.5 20d156c878 Issue #34: Missing Photos checks library photos only
Missing Photos (sidebar count, the view, library.missing, Find Missing)
now shares one scope rule, cmd::missing::checked_path: non-deleted
library photos backed by a file. Local browse records, which can
outnumber the library by orders of magnitude, are never stat'ed. The
Missing view checks only the photos its query kept instead of every
catalog record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:39:11 -04:00
Brandon ThomasandClaude Opus 5.5 7c662bc5d6 Issue #27: Menus: keep Quit last in the in-window File menu
Registered File commands the explicit layout doesn't list were appended
after Quit LightCraft. Items in LAST (app.quit) are now held back and
placed after the fallback entries, in their own separator group. The
macOS native menu filters Quit into the app menu and is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:38:44 -04:00
Brandon Thomas fb8e1e93aa Merge pull request #46 from GlemNL/fix/issue-24-hide-local-locations
Local: Remove from Local hides a sidebar location, with a way to restore
2026-10-04 10:24:27 -04:00
Brandon Thomas 5b73e4f08b Merge main into fix/issue-24-hide-local-locations
# Conflicts:
#	crates/ui-egui/src/panels/left.rs
2026-10-04 10:24:12 -04:00
Brandon Thomas 0467eb6c58 Merge pull request #43 from GlemNL/fix/issue-23-local-folder-async
Local: browse a folder in the background with progress
2026-10-04 10:24:03 -04:00
Brandon ThomasandClaude Opus 5.5 65bc8bcb42 Headless: a running folder scan counts as busy, so settle waits for a background browse
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:19:54 -04:00
Brandon ThomasandClaude Opus 5.5 4267a68b30 Local: use ? for the browse path (clippy)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:19:54 -04:00
Brandon ThomasandClaude Opus 5.5 3ac1e55a6c Local: drop the needless struct update in library.import (clippy)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:19:54 -04:00
Brandon Thomas 94fd0e6994 Merge main into fix/issue-23-local-folder-async
# Conflicts:
#	crates/ui-egui/src/panels/grid.rs
2026-10-04 10:19:54 -04:00
Brandon Thomas 4b863e9d47 Merge pull request #45 from GlemNL/fix/issue-25-active-filters
Show active filters as chips and count matching of total photos
2026-10-04 10:19:04 -04:00
Brandon Thomas cdaacc1e83 Merge pull request #48 from GlemNL/fix/issue-26-smart-previews-location
Smart Previews: choose the folder per library
2026-10-04 10:18:57 -04:00
Brandon Thomas e1857511af Merge pull request #44 from storybored379/feat/export-naming-tokens
LR-EXP-NAMING: export file names use the Rename Photos tokens (+ num, folder, lens, iso, rating, creator)
2026-10-04 10:18:50 -04:00
Brandon Thomas 1d7d1f66ab Merge pull request #49 from GlemNL/fix/issue-28-menu-spacing
UI: more spacing between in-window menu headings
2026-10-04 10:18:44 -04:00
Brandon Thomas cea57e2dc7 Merge pull request #42 from storybored379/pr/mcp-crop-requires-args
MCP: crop returns an error when given no rect, angle or reset instead of a silent no-op
2026-10-04 10:18:37 -04:00
Brandon Thomas 14594ee635 Merge pull request #50 from storytold/pr/license-standard
Licensing: MIT OR Apache-2.0 files, NOTICE, ArtCraft brand terms
2026-10-04 10:09:00 -04:00
Brandon ThomasandClaude Opus 5.5 320bcad19a Licensing: MIT OR Apache-2.0 files, NOTICE, ArtCraft brand terms
Add LICENSE-MIT and LICENSE-APACHE (Copyright (c) 2026 ArtCraft Team and the
LightCraft contributors) and docs/brand/LICENSE-brand.txt from the Crafting Apps
licensing standard. Rewrite NOTICE to the standard template and drop the stale
Source Sans 3 entry (only Inter is bundled). Attribute docs/brand/ in
assets/ATTRIBUTION.md so `cargo xtask assets` passes again. README gets the
standard "License and credits" section; AGENTS.md/CLAUDE.md note the brand
exception to the asset rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:05:02 -04:00
Clément LandClaude Sonnet 5.5 00766f8c21 UI: more spacing between in-window menu headings (#28)
The headings are frameless buttons, which egui renders without padding, so
the gap now comes from the layout's item spacing (24 px). The width check
that falls back to the single "Menu" button uses the same gap.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-04 11:42:39 +02:00
Clément LandClaude Sonnet 5.5 78f627228f Smart Previews: choose the folder per library, with a move/leave/delete choice (#26)
Smart previews always lived in <library>/Smart Previews, so a large set of proxies
filled the library's drive. The folder is now a per-library setting (prefs.json):

- library.smartPreviewsLocation shows the effective folder (path, count, size,
  availability) or changes it; changing it needs an explicit choice for the
  previews already built (move, leave or discard), and reset returns to the
  library's own folder.
- The new folder must accept writes before anything moves. A folder whose parent
  is missing (an unplugged drive) is an error and is never created on the current
  drive; building also refuses while a chosen folder is unavailable.
- Building, discarding, status and offline rendering all use the chosen folder.
- Settings -> Performance -> Smart previews shows the path before building, what
  it holds, the existing-previews choice, Choose Folder... and Use Library Folder.

The thumbnail cache stays in the library.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-04 11:21:25 +02:00
Clément LandClaude Sonnet 5.5 1c50d538c3 Local: Remove from Local hides a sidebar location; a row brings hidden ones back (#24)
Right-clicking a top-level Local location (built-in or the browsed folder) offers
Remove from Local. It only hides the shortcut: the folder, its photos and catalog
membership are untouched. The hidden list is saved with the UI state. While any
location is hidden, "Show N hidden locations" under Browse Folder… restores them,
and choosing a hidden folder with Browse Folder… puts it back. Both actions are
commands (local.hide, local.restoreHidden).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-04 11:10:24 +02:00
Clément LandClaude Sonnet 5.5 431186a24e Library: show active filters as removable chips and count matching of total photos (#25)
A filter set from the sidebar (By Date), the search field or the filter bar
narrows the current source, but nothing said so: a folder with a hidden date
filter looked empty.

- Chips under the grid header, one per active filter, each with an X that
  clears only that filter; "+N more" lists the ones that do not fit; Clear all.
- The header counts "N of M photos" (matching of the source's total) while a
  filter is on; the total is cached per source/catalog revision.
- A badge on the top-bar filter icon counts the active filters, filter bar
  open or not.
- An empty result says no photos match the active filters, and how many the
  source holds, instead of implying the folder is empty.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-04 10:53:25 +02:00
Clément LandClaude Sonnet 5.5 257a982678 Say when a filter hides the photos of an empty grid or filmstrip
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-04 10:42:34 +02:00
Clément LandClaude Sonnet 5.5 d6be9f6342 Local: clicking the folder being read again keeps the running read instead of restarting it
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-04 10:35:48 +02:00
Clément LandClaude Sonnet 5.5 e076f65df8 Sidebar: count missing files on a worker thread; read browsed folders in smaller batches
The Missing count stat()ed every catalog photo on the UI thread, which on a network share
(and with browsed NAS photos in the catalog) froze the window for seconds during a folder
browse. It now runs in the background and shows the last finished count.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-04 10:32:18 +02:00
storybored379andClaude Opus 5.5 309922eac9 LR-EXP-NAMING: export file names use the Rename Photos tokens
rename::expand_tokens is shared by Rename Photos, import renaming and export naming, so {seq:N},
{date:%…}, {camera}, {title}, {ext} work in exports too (a bare {seq} still pads to 3 digits there).
New shared tokens: {num} (number at the end of the name), {folder}, {lens}, {iso}, {rating},
{creator}. An export name that expands to nothing keeps the original name. The MCP export tool
passes startNumber through and documents the tokens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 01:26:57 -07:00
Clément LandClaude Sonnet 5.5 919e65efaf Local: browse a folder in the background with progress; keep picked folders in the sidebar
Browse Folder listed, probed and imported the whole folder on the UI thread, so a
network share froze the window with no feedback. It now lists and reads on a worker
(progress window with Cancel), then adds the photos in small batches reusing the
scan's probes. Folders picked with Browse Folder… stay in the Local list after
browsing elsewhere.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-04 10:22:47 +02:00
storybored379andClaude Opus 5.5 d53025cc34 MCP: crop returns an error when given no rect, angle or reset instead of a silent no-op
The crop tool fell through to crop.set with an empty object when none of
its parameters were given, so a call with a guessed parameter (e.g.
`aspect: "1:1"`) or `reset: false` answered "ok" and left the photo
unchanged. It now returns an error naming the parameters it needs, as
set_develop already does for an empty call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 01:17:34 -07:00
Brandon Thomas e3b61fc883 Merge pull request #41 from storytold/fix/issue-12-14-curve
Tone curve: drag points, reset curves, point-curve presets (fixes #12, #13, #14)
2026-10-04 03:05:47 -04:00
Brandon Thomas b80bec6e3c Merge pull request #40 from storytold/fix/issue-10-preview-only
Mark raws shown from their embedded preview as "preview only" (refs #10)
2026-10-04 03:05:18 -04:00
Brandon Thomas ad0b947f07 Merge pull request #39 from storytold/fix/issue-9-luminar
Import Luminar looks (.lmp, .mplumpack) (fixes #9)
2026-10-04 03:04:50 -04:00
Brandon ThomasandClaude Opus 5.5 966db71295 LR-EDIT-CURVE: point-curve presets (built-in, save, delete, import/export)
A Point Curve dropdown under the curve graph shows the preset the photo's
point curves match (or Custom) and lists our own built-ins (Linear, Medium
Contrast, Strong Contrast: symmetric S-curves, values chosen here) and the
user's presets; Save Point Curve... names and stores the current four
channels, right-click deletes, Import/Export Presets... read and write
.lccurve JSON files. Presets set only the point curves (parametric curve and
Refine Saturation stay) and are saved with the library preferences.

Engine commands: curve.presets, curve.applyPreset, curve.savePreset,
curve.deletePreset, curve.importPresets, curve.exportPresets; UI commands
file.importCurvePresets / file.exportCurvePresets (file dialogs on desktop).

Fixes #14

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 02:59:25 -04:00
Brandon ThomasandClaude Opus 5.5 1169a1beda LR-IMP-FORMATS: mark raws shown from their embedded preview as "preview only"
Raw variants the decoder can't read yet (e.g. Nikon Huffman-compressed NEF
from a D3200/D5100) silently fell back to the camera's embedded JPEG, so
users edited a JPEG with the camera's picture style (e.g. Monochrome) baked
in without being told.

- catalog: Photo.preview_only (serde default; old catalogs load), set /
  cleared by Op::SetContent with an exact inverse; Photo::develops_raw().
- engine: ProbeInfo.preview_only carries the decoder's Unsupported reason
  into import, the import review (previewOnly) and catalog.query
  (previewOnly); photo.inspect includes it. photo.reload and photo.relink
  re-probe and clear/set it when the file decodes (or not) now.
- source_info: a preview-only raw is a rendered source (relative WB scale,
  display tone curve) instead of raw:true/5500 K + filmic tone map on a
  display-referred JPEG; import uses the JPEG default preset for it; the
  Edit panel shows the rendered-file WB sliders.
- UI: "Preview" badge on grid thumbnails, a pill on the loupe, a notice in
  Edit and Info (amber caution token, Info icon drawn in code).
- Tests: catalog serde/undo, engine import -> reopen -> reload/relink,
  headless UI badge/notice widgets.

Refs #10

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 02:59:25 -04:00
Brandon ThomasandClaude Opus 5.5 4a70fcfe4f LR-PRE-IMPORT: bound plist nesting (a hostile look can't overflow the stack)
Nested <dict>/<array> recursed without limit; a crafted .lmp with deep
nesting would crash the app on import. Reading stops with an error past
64 levels. Test: hostile_nesting_is_an_error_not_a_crash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 02:59:25 -04:00
Brandon ThomasandClaude Opus 5.5 ae29c0569f LR-EDIT-CURVE: reset a curve channel or the whole tone curve
New engine command curve.reset {channel?: all|point|parametric|master|red|
green|blue} (default all = the whole Tone Curve), one undo step, usable from
CLI / MCP / control channel. In the editor: double-click a channel selector
resets that channel, a Reset button under the graph resets every curve
(point curves and parametric), and right-clicking the graph offers
Reset <channel> / Reset All Curves.

Fixes #13

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 02:59:25 -04:00
Brandon ThomasandClaude Opus 5.5 6c3c155c99 LR-PRE-IMPORT: import Luminar looks (.lmp, .mplumpack)
An .lmp look is an XML property list of adjustment layers with named
sliders (newer looks: a Name.lmp/Contents/preset.lmp bundle); an
.mplumpack is a zip of looks plus PresetsInfo.plist (GroupName).
preset.import, the file dialog and drag & drop now accept both. Sliders
with a clear counterpart (exposure, contrast, highlights/shadows,
whites/blacks, WB shift, saturation/vibrance, clarity/structure,
dehaze, HSL, point curves, vignette, grain) map onto develop settings;
the rest is reported in `unmapped`. Looks are named after the look and
grouped by the pack. Tests use synthetic looks and packs only.

Fixes #9

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 02:59:25 -04:00
Brandon ThomasandClaude Opus 5.5 e1acdb876d LR-EDIT-CURVE: drag point-curve points in both axes
The editor stored the dragged point index as Option<usize> but read it back
as usize, so the index was lost after the first frame: pressing on a point
did nothing and only click-to-add (overwrite) worked. Points now drag in X
and Y (X clamped between the neighbours), picked at the press origin, with a
grab cursor, a highlighted point and an input/output readout; one undo step
per drag via develop.beginInteraction / endInteraction.

Fixes #12

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 02:59:25 -04:00
Brandon Thomas 6eb5335a27 Merge pull request #38 from storytold/fix/issue-11-scroll
Grid and filmstrip keep your scroll position; mouse wheel scrolls the filmstrip (fixes #11)
2026-10-04 02:58:28 -04:00
Brandon ThomasandClaude Opus 5.5 103f53724f LR-FILMSTRIP: grid and filmstrip keep the user's scroll position; wheel scrolls the filmstrip
Fixes #11.

Root cause: the grid and the filmstrip remembered the last active photo in egui temp data
with mismatched types (inserted `Option<PhotoId>`, read back as `PhotoId` because of the
`let last: Option<PhotoId> = get_temp(..)` annotation), so the read always missed, "the
active photo changed" was true on every frame and both called `scroll_to_rect` on the
active photo every frame. Any wheel or scroll-bar scroll was undone as soon as the user
stopped (the animated scroll back read as a delayed snap).

- `grid::follow_active` (shared by grid and filmstrip): scroll to the active photo only
  when it changed since the strip was last drawn, or when the strip was not drawn on the
  previous pass (view switch). Time passing, finished renders, edits and a released scroll
  bar no longer move either strip.
- Filmstrip: a vertical mouse wheel scrolls it sideways (horizontal trackpad scrolling
  still works); it recentres on a new active photo only when that is off screen, so a
  click on a visible thumbnail leaves it alone.
- `ui.inspect` reports `scroll: {grid, filmstrip}`; headless tests in `tests_scroll.rs`.
- docs/parity.md (LR-VIEW-PHOTOGRID, LR-VIEW-FILMSTRIP), docs/control-protocol.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 02:56:32 -04:00
Brandon Thomas 296c67bf9b Merge pull request #17 from storytold/fix/issue-8-export-dialog-width
Export dialog no longer widens every frame (fixes #8)
2026-10-04 02:47:12 -04:00
Brandon Thomas 0a1e38db70 Merge pull request #16 from storytold/fix/issue-7-windows-console
Windows: the desktop app opens no console window (fixes #7)
2026-10-04 02:32:06 -04:00
Brandon ThomasandClaude Opus 5.5 b8c2d16a06 LR-EXP-DIALOG: export dialog no longer widens every frame
Rows with a trailing button (Folder / Choose…, watermark Graphic /
Choose…, Save preset / Save) sized their text field as the available
width minus a guessed button width. When the real button was wider
(e.g. Linux fonts, a folder picker present), the auto-sized dialog grew
a little every frame. These rows now lay out right to left: the button
first, then a field taking exactly the width that's left.

Test: export_dialog_keeps_its_width (grew 492 → 693 px in 60 frames
before; stable now).

Fixes #8

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 02:31:34 -04:00
Brandon ThomasandClaude Opus 5.5 3f8dbb41c0 Windows: the desktop app opens no console window
Release builds of lightcraft.exe are GUI-subsystem programs
(windows_subsystem = "windows"), so launching the app no longer opens
a terminal. Debug builds keep the console; lightcraft-cli.exe stays a
console program for terminal use (--help / --version / scripting).
Checked with cargo check --release --target x86_64-pc-windows-msvc.

Fixes #7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 02:31:34 -04:00
Brandon Thomas f5a4bb609d Merge pull request #1 from leipsfur/fix/arw-lossless-tiles
M3.7: Sony lossless-compressed ARW tiles (2×2 CFA cells per LJ92 sample)
2026-10-04 01:39:29 -04:00
Brandon Thomas c0794b95e8 Merge remote-tracking branch 'origin/main' into review/pr-1 2026-10-04 01:36:45 -04:00
Brandon Thomas c5c08c6af9 Merge pull request #15 from storybored379/pr/library-state-album-source
Library: an album source serializes (library.state no longer panics)
2026-10-04 01:26:26 -04:00
Brandon ThomasandClaude Opus 5.5 d6eac6685c M3.7: header probe agrees with decode on subsampled lossless JPEG; lossless ARW corpus samples
- tiffraw: Mode::Header parses the first LJ92 frame header (Compression 7),
  so probe_info reports Unsupported for subsampled frames (Sony lossless
  M/S) as the full decode does, instead of claiming the file decodes.
- corpus: CC0 ILCE-7M4 lossless L/M/S samples from raw.pixls.us in
  `cargo xtask corpus`; M/S listed as known unsupported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 01:21:33 -04:00
Brandon Thomas 2956f97352 Merge branch 'pr-1' into review/pr-1 2026-10-04 01:13:20 -04:00
storybored379andClaude Opus 5.5 6a83818192 Library: an album source serializes (library.state no longer panics)
LibrarySource was an internally tagged enum (tag = "kind") whose Album
variant holds an AlbumId, and serde cannot serialize an integer newtype
in an internally tagged enum. With an album open:

- library.state unwrapped the error and panicked; over the control
  channel this closed the desktop app.
- view.json (source, filter, sort, selection) was written empty
  (to_vec_pretty(...).unwrap_or_default()), so the view was lost on the
  next launch.

Adjacent tagging (content = "id") keeps the unit variants as
{"kind": "all"} and writes the album as {"kind": "album", "id": 8}: the
same shape library.source takes, so the reported source can be passed
back as is. Existing view.json files read unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-03 17:38:43 -07:00
Brandon ThomasandClaude Opus 5.5 8146a11cec LR-IMP-ADD-DIALOG: folder scan follow-ups (Cancel, busy guard, inspect)
- Cancel drops the scan at once (the worker stops at its next file) and
  has a widget id (button:scanCancel) for agents.
- Add from Device while a scan runs no longer flips the running scan to
  copy-into-library.
- ui.inspect reports the scan's {done, total}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 19:07:44 -04:00
Brandon Thomas fc66026ef0 Merge pull request #5 from GlemNL/feat/click-zoom-ratio
Loupe: animated click-zoom with selectable 2:1 / 3:1 / 5:1 ratio
2026-10-03 19:07:36 -04:00
Brandon ThomasandClaude Opus 5.5 a4b168550d Loupe click-zoom: 1:1 default, Z/Space use the ratio, render once at the final size
Review follow-ups on the contributor's click-zoom:
- Ratios 1:1 (default), 2:1, 3:1, 4:1, 8:1; set through the new
  view.clickZoom {ratio} command (bottom-bar dropdown, CLI, MCP).
- Z / Space zoom to the same ratio as a click, with the same ease.
- The loupe requests its render at the final size; the animation only
  changes how it is drawn (no render per animation frame). The test
  checks every animation frame asks for one render.
- docs/parity.md: LR-VIEW-ZOOM.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 19:07:18 -04:00
Brandon Thomas b49e32c094 Merge remote-tracking branch 'origin/main' into pr5-work 2026-10-03 18:36:14 -04:00
Brandon Thomas 542343099e Merge pull request #4 from GlemNL/feat/crop-double-click-apply
Crop: double-click inside the crop box applies the crop
2026-10-03 18:35:46 -04:00
Brandon Thomas ad567f6632 Merge pull request #6 from GlemNL/feat/zoomed-scrub-cursor
Loupe: zoomed Grab cursor only over the photo
2026-10-03 18:35:27 -04:00
Brandon Thomas d473a4d9bf Merge pull request #2 from GlemNL/fix/nas-folder-import
Import: scan folders in the background (fixes Add Folder on network shares)
2026-10-03 18:15:21 -04:00
Clément LandClaude Sonnet 5.5 97ba282044 Import: update copy-import test for the background folder scan
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 18:25:05 +02:00
Clément LandClaude Sonnet 5.5 3e70c0edfe Loupe: zoomed Grab cursor only over the photo, so slider cursor wins in side panels
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 18:12:35 +02:00
Clément LandClaude Sonnet 5.5 7b8df44947 Loupe: animated click-zoom with selectable 2:1 / 3:1 / 5:1 ratio
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 18:04:58 +02:00
Clément LandClaude Sonnet 5.5 ca1f4dbd8a Crop: double-click inside the crop box applies the crop
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 17:58:04 +02:00
Clément LandClaude Sonnet 5.5 5828a7d2d3 Import: scan folders on a background thread with progress and Cancel
Add Folder read every file on the UI thread before opening the review; on a
NAS (~26k files) the window froze and was dropped. The scan now runs in a
worker (ScanInput/scan_with) and the review opens when it finishes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 17:43:02 +02:00
Christian BretzandClaude Opus 5.5 5d143cf108 M3.7: Sony lossless-compressed ARW tiles (2×2 CFA cells per LJ92 sample)
Sony's lossless-compressed raw (Compression 7, e.g. ILCE-7M4) stores
512×512 tiles as LJ92 frames of 256×256 with four components: each
frame sample is one 2×2 CFA cell (top-left, top-right, bottom-left,
bottom-right). The generic TIFF path reads a frame as a row-major sample
stream (DNG's convention), which interleaved each tile's left and right
halves row by row: vertical bands, scrambled CFA, purple cast.

arw.rs now detects the layout from the first tile's frame header and
places the cells itself; other lossless-JPEG layouts keep the generic
path. Component order verified on a real ILCE-7M4 file (green
differences to the diagonal neighbour cell match those within a cell).
RENDER_CACHE_VERSION bumped so cached thumbnails of affected files are
re-rendered.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCLtGBctp1CxX2MdCuKCrm
2026-10-02 17:51:58 +02:00
496 changed files with 120205 additions and 4444 deletions

No files matched your search

+55
View File
@@ -0,0 +1,55 @@
# Builds and tests the workspace on FreeBSD, inside a FreeBSD VM on a Linux runner
# (GitHub has no FreeBSD runners). GPU tests skip themselves when the VM has no adapter.
name: FreeBSD
on:
push:
branches: [main]
pull_request:
paths:
- Cargo.toml
- Cargo.lock
- "**/Cargo.toml"
- .github/workflows/freebsd.yml
workflow_dispatch:
concurrency:
group: freebsd-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
freebsd:
name: FreeBSD 14 (build + test)
runs-on: ubuntu-latest
timeout-minutes: 300
steps:
- uses: actions/checkout@v4
# Optional font build input, so the Japanese font tests run (they skip without it).
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: 8dcdacd5153e64560d109541a47d806f26f048c0
path: craft-fonts
- name: Build and test in a FreeBSD VM
uses: vmactions/freebsd-vm@c46abacb49f09938ca4e1702d15d836285d694cc # v1
with:
release: "14.3"
usesh: true
cpu: 4
mem: 12288
# Nothing to bring back; copying the target dir out of the VM fails on large builds.
copyback: false
prepare: |
pkg install -y git curl pkgconf cmake python3 libxkbcommon wayland libX11 libXcursor libXrandr libXi libxcb mesa-libs vulkan-loader gtk3 fontconfig freetype2 alsa-lib
curl -fsSL https://sh.rustup.rs | sh -s -- -y --profile minimal
run: |
. "$HOME/.cargo/env"
# The checkout is synced in under another owner; let tests that call git read it.
git config --global --add safe.directory "*"
rustc --version
export CRAFT_FONTS_DIR="$PWD/craft-fonts" CRAFT_FONTS_REQUIRED=1
CARGO_PROFILE_DEV_DEBUG=0 cargo build --workspace --all-targets --locked
CARGO_PROFILE_DEV_DEBUG=0 cargo test --workspace --locked
+16 -1
View File
@@ -47,7 +47,22 @@ jobs:
- name: XML, desktop entry, AppStream, Flatpak manifest
run: |
xmllint --noout packaging/windows/lightcraft.wxs packaging/macos/*.plist* packaging/linux/*.xml*
# Windows Installer needs Icon ids ending in .ico or .exe, or shortcuts show a blank icon.
if grep -oE '<Icon Id="[^"]*"' packaging/windows/lightcraft.wxs | grep -vE '\.(ico|exe)"$'; then
echo "::error::Icon Id must end in .ico or .exe"; exit 1
fi
desktop-file-validate packaging/linux/ai.storyteller.lightcraft.desktop
sed -e 's/@VERSION@/0.0.0/' -e 's/@DATE@/2026-01-01/' packaging/linux/ai.storyteller.lightcraft.metainfo.xml.in >"$RUNNER_TEMP/ai.storyteller.lightcraft.metainfo.xml"
appstreamcli validate --no-net --explain "$RUNNER_TEMP/ai.storyteller.lightcraft.metainfo.xml"
python3 -c "import yaml; m = yaml.safe_load(open('packaging/linux/flatpak/ai.storyteller.lightcraft.yml')); assert m['id'] == 'ai.storyteller.lightcraft' and m['command'] == 'lightcraft'"
python3 - <<'EOF'
import yaml
src = yaml.safe_load(open("packaging/linux/flatpak/ai.storyteller.lightcraft.yml"))
bundle = yaml.safe_load(open("packaging/linux/flatpak/ai.storyteller.lightcraft.bundle.yml"))
for m in (src, bundle):
assert m["id"] == "ai.storyteller.lightcraft" and m["command"] == "lightcraft", m["id"]
for key in ("runtime", "runtime-version", "sdk", "finish-args"):
assert src[key] == bundle[key], f"Flatpak manifests disagree on {key}: {src[key]} vs {bundle[key]}"
print("Flatpak manifests ok")
EOF
- name: FreeBSD tarball layout (stub binaries)
run: bash packaging/freebsd/package.sh --dry-run
+143 -13
View File
@@ -1,9 +1,12 @@
# Release pipeline: every push to `release` builds signed installers for every platform and
# creates or updates the draft GitHub Release "LightCraft v<version>". See docs/releasing.md.
# Release pipeline: every push to `release` builds platform artifacts (signed where credentials
# are configured) and creates or updates the draft GitHub Release "LightCraft v<version>".
# See docs/releasing.md.
#
# All jobs run in the `release` environment (restricted to the `release` branch), which holds
# the signing secrets. Every secret is optional: missing ones produce unsigned artifacts and a
# warning, never a failed build.
# The signing jobs (macOS, Windows) and the draft-release job run in the `release` environment
# (restricted to the `release` branch), which holds the signing secrets. Every secret is optional:
# missing ones produce unsigned artifacts and a warning, never a failed build. The jobs that sign
# nothing (version, Linux, Flatpak, FreeBSD, web) run without it, so a dispatch on any branch is
# a dry run of them; the release job needs the signed jobs, so a branch run never drafts a release.
name: Release
on:
@@ -35,7 +38,7 @@ jobs:
version:
name: version
runs-on: ubuntu-latest
environment: release
# No `environment: release` here or on the other jobs that sign nothing: no release secrets.
outputs:
version: ${{ steps.v.outputs.version }}
date: ${{ steps.v.outputs.date }}
@@ -63,13 +66,19 @@ jobs:
macos:
name: macOS universal
needs: version
runs-on: macos-15
runs-on: macos-15-xlarge # M2 Pro larger runner; use only in release.yml (actions_macos_xl budget caps spend)
environment: release
env:
LIGHTCRAFT_VERSION: ${{ needs.version.outputs.version }}
LIGHTCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
# Fonts (optional build input; required for releases): see craftrules standards/fonts.md.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
targets: aarch64-apple-darwin, x86_64-apple-darwin
@@ -87,6 +96,8 @@ jobs:
run: packaging/macos/import-cert.sh
- name: Build, sign, notarize
env:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
APPLE_ID: ${{ secrets.APPLE_ID }}
APPLE_PASSWORD: ${{ secrets.APPLE_PASSWORD }}
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
@@ -112,11 +123,20 @@ jobs:
include:
- { arch: x64, target: x86_64-pc-windows-msvc }
- { arch: x86, target: i686-pc-windows-msvc }
# Windows on ARM: cross-compiled on the x64 runner, so signing and WiX run as above.
# .github/workflows/windows-arm64.yml installs and runs it on ARM64 hardware.
- { arch: arm64, target: aarch64-pc-windows-msvc }
env:
LIGHTCRAFT_VERSION: ${{ needs.version.outputs.version }}
LIGHTCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
# Fonts (optional build input; required for releases): see craftrules standards/fonts.md.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}
@@ -133,6 +153,8 @@ jobs:
- name: Build, sign, package
shell: pwsh
env:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
WINDOWS_CERTIFICATE: ${{ secrets.WINDOWS_CERTIFICATE }}
WINDOWS_CERTIFICATE_PASSWORD: ${{ secrets.WINDOWS_CERTIFICATE_PASSWORD }}
AZURE_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
@@ -154,7 +176,6 @@ jobs:
# The oldest GitHub-hosted image, so the binaries only need glibc >= 2.35 (Ubuntu 22.04+,
# Debian 12+, Fedora 36+, RHEL 10). See docs/releasing.md.
runs-on: ${{ matrix.runner }}
environment: release
strategy:
fail-fast: false
matrix:
@@ -167,12 +188,18 @@ jobs:
NFPM_VERSION: 2.47.0
steps:
- uses: actions/checkout@v4
# Fonts (optional build input; required for releases): see craftrules standards/fonts.md.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- name: Install build and packaging tools
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
libxkbcommon-dev libwayland-dev libx11-dev libxrandr-dev libxi-dev libgl1-mesa-dev \
desktop-file-utils appstream python3-yaml
desktop-file-utils appstream python3-yaml zsync
curl -fsSL -o "$RUNNER_TEMP/nfpm.deb" \
"https://github.com/goreleaser/nfpm/releases/download/v${NFPM_VERSION}/nfpm_${NFPM_VERSION}_${{ matrix.deb }}.deb"
sudo dpkg -i "$RUNNER_TEMP/nfpm.deb"
@@ -184,6 +211,9 @@ jobs:
if: inputs.version != ''
run: cargo xtask version set "$LIGHTCRAFT_VERSION"
- name: Build and package (AppImage, deb, rpm, tar.gz)
env:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
run: packaging/linux/package.sh
- name: Check the packages
run: |
@@ -194,6 +224,9 @@ jobs:
for appimage in dist/release/*.AppImage; do
chmod +x "$appimage"
APPIMAGE_EXTRACT_AND_RUN=1 "$appimage" --version
# Update information is embedded, and the .zsync for delta updates sits beside it.
"$appimage" --appimage-updateinformation | grep -F 'gh-releases-zsync|'
test -s "$appimage.zsync"
done
- uses: actions/upload-artifact@v4
with:
@@ -201,17 +234,110 @@ jobs:
path: dist/release/*
if-no-files-found: error
flatpak:
name: Flatpak ${{ matrix.arch }}
needs: [version, linux]
# Repackages the Linux job's tarball (the same binaries as the AppImage/deb/rpm); no Rust
# build here. 24.04 for a newer flatpak-builder (1.4) than 22.04 ships.
runs-on: ${{ matrix.runner }}
strategy:
fail-fast: false
matrix:
include:
- { arch: x86_64, runner: ubuntu-24.04 }
- { arch: aarch64, runner: ubuntu-24.04-arm }
env:
LIGHTCRAFT_VERSION: ${{ needs.version.outputs.version }}
LIGHTCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
name: release-linux-${{ matrix.arch }}
path: dist/linux
- name: Install flatpak-builder
run: |
sudo apt-get update
# librsvg2-common: the SVG pixbuf loader. flatpak-builder runs the host's
# `appstreamcli compose`, which renders the scalable icon and fails without it.
sudo apt-get install -y --no-install-recommends flatpak flatpak-builder librsvg2-common
# Ubuntu 24.04's AppArmor blocks the unprivileged user namespaces bubblewrap needs.
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
- name: Build the Flatpak bundle
run: packaging/linux/flatpak-bundle.sh "dist/linux/lightcraft-$LIGHTCRAFT_VERSION-linux-${{ matrix.arch }}.tar.gz"
- uses: actions/upload-artifact@v4
with:
name: release-flatpak-${{ matrix.arch }}
path: dist/release/*.flatpak
if-no-files-found: error
freebsd:
name: FreeBSD x86_64
needs: version
# GitHub has no FreeBSD runners: build in a FreeBSD 14.3 VM, the same image and packages as
# .github/workflows/freebsd.yml.
runs-on: ubuntu-latest
timeout-minutes: 180
env:
LIGHTCRAFT_VERSION: ${{ needs.version.outputs.version }}
LIGHTCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
# Fonts (optional build input; required for releases): see craftrules standards/fonts.md.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- name: Build and package in a FreeBSD VM (tar.gz)
# vmactions/freebsd-vm v1.5.9, pinned by commit (it runs our build).
uses: vmactions/freebsd-vm@c46abacb49f09938ca4e1702d15d836285d694cc
with:
release: "14.3"
usesh: true
cpu: 4
mem: 12288
# Passed into the VM by name; never templated into the script below.
envs: "LIGHTCRAFT_VERSION LIGHTCRAFT_BUILD_DATE LIGHTCRAFT_BUILD_SHA INPUT_VERSION"
# The FreeBSD CI packages, plus bash for the packaging script.
prepare: |
pkg install -y bash git curl pkgconf cmake python3 libxkbcommon wayland libX11 libXcursor libXrandr libXi libxcb mesa-libs vulkan-loader gtk3 fontconfig freetype2 alsa-lib
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain stable
run: |
set -e
. "$HOME/.cargo/env"
git config --global --add safe.directory "$PWD"
# Build outside the checkout so only dist/ is copied back out of the VM (copying the
# target dir back fails on large builds). 4 jobs: more runs the VM out of memory.
export CARGO_TARGET_DIR=/var/tmp/lightcraft-target CARGO_BUILD_JOBS=4
if [ -n "$INPUT_VERSION" ]; then cargo xtask version set "$LIGHTCRAFT_VERSION"; fi
# craft-fonts was checked out next to the sources and synced into the VM with them.
export CRAFT_FONTS_DIR="$PWD/craft-fonts" CRAFT_FONTS_REQUIRED=1
bash packaging/freebsd/package.sh
tar -tzf dist/release/lightcraft-*-freebsd-x86_64.tar.gz \
| grep -E '/bin/lightcraft$|/bin/lightcraft-cli$|/share/applications/ai\.storyteller\.lightcraft\.desktop$|/share/doc/lightcraft/LICENSE-MIT$'
- uses: actions/upload-artifact@v4
with:
name: release-freebsd-x86_64
path: dist/release/*
if-no-files-found: error
web:
name: Web (wasm)
needs: version
runs-on: ubuntu-latest
environment: release
env:
LIGHTCRAFT_VERSION: ${{ needs.version.outputs.version }}
LIGHTCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
WASM_BINDGEN_VERSION: 0.2.129
steps:
- uses: actions/checkout@v4
# Fonts (optional build input; required for releases): see craftrules standards/fonts.md.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-unknown-unknown
@@ -221,13 +347,17 @@ jobs:
- name: Install wasm-bindgen CLI
run: |
# Must match the wasm-bindgen version in Cargo.lock exactly (cargo xtask web checks it).
command -v wasm-bindgen >/dev/null && wasm-bindgen --version | grep -q "$WASM_BINDGEN_VERSION" \
|| cargo install wasm-bindgen-cli --version "$WASM_BINDGEN_VERSION" --locked
if ! { command -v wasm-bindgen >/dev/null && wasm-bindgen --version | grep -q "$WASM_BINDGEN_VERSION"; }; then
cargo install wasm-bindgen-cli --version "$WASM_BINDGEN_VERSION" --locked
fi
wasm-bindgen --version
- name: Apply version override
if: inputs.version != ''
run: cargo xtask version set "$LIGHTCRAFT_VERSION"
- name: Build and zip
env:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
run: packaging/web/package.sh
- uses: actions/upload-artifact@v4
with:
@@ -237,7 +367,7 @@ jobs:
release:
name: Draft GitHub Release
needs: [version, macos, windows, linux, web]
needs: [version, macos, windows, linux, flatpak, freebsd, web]
runs-on: ubuntu-latest
environment: release
permissions:
+95
View File
@@ -0,0 +1,95 @@
# Windows on ARM: package the ARM64 release the way release.yml does (cross-compiled on an x64
# runner, unsigned here), then install that MSI on a Windows 11 ARM64 runner, check the installed
# binaries are ARM64, run the CLI natively, and uninstall. (A native ARM64 `cargo test` job is left
# out for now: PdfCraft saw an intermittent access violation in egui UI tests there.)
name: Windows ARM64
on:
push:
branches: [main]
pull_request:
paths:
- Cargo.toml
- Cargo.lock
- "**/Cargo.toml"
- "**/build.rs"
- .github/workflows/windows-arm64.yml
- packaging/windows/**
schedule:
- cron: "41 4 * * 1" # weekly, so drift in the runner images shows up
workflow_dispatch:
concurrency:
group: windows-arm64-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
jobs:
package:
name: package (cross-compiled on x64)
runs-on: windows-latest
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@89b12181fb390509a0842a86cc55eeb8eb928c1d # stable
with:
targets: aarch64-pc-windows-msvc
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
key: windows-arm64-package
- name: Install WiX
shell: pwsh
run: dotnet tool install --global wix --version 5.0.2
- name: Build and package (unsigned)
shell: pwsh
env:
DIST: ${{ github.workspace }}\dist\arm64
run: ./packaging/windows/package.ps1 -Arch arm64
- uses: actions/upload-artifact@v4
with:
name: lightcraft-windows-arm64
path: dist/arm64/*
if-no-files-found: error
retention-days: 7
install:
name: install and run on ARM64
needs: package
runs-on: windows-11-arm
timeout-minutes: 30
steps:
- uses: actions/download-artifact@v4
with:
name: lightcraft-windows-arm64
path: dist
- name: Install the MSI, check it, run it, uninstall it
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
$msi = (Get-ChildItem dist/*.msi | Select-Object -First 1).FullName
$p = Start-Process msiexec.exe -ArgumentList '/i', "`"$msi`"", '/qn', '/l*v', 'install.log' -Wait -PassThru
if ($p.ExitCode -ne 0) { Get-Content install.log -Tail 60; throw "msiexec /i exited $($p.ExitCode)" }
$dir = Join-Path $env:ProgramFiles 'LightCraft'
foreach ($exe in 'lightcraft.exe', 'lightcraft-cli.exe') {
$path = Join-Path $dir $exe
if (-not (Test-Path $path)) { throw "$exe not installed in $dir" }
$bytes = [System.IO.File]::ReadAllBytes($path)
$machine = [BitConverter]::ToUInt16($bytes, [BitConverter]::ToInt32($bytes, 0x3C) + 4)
if ($machine -ne 0xAA64) { throw "$exe is for machine 0x$('{0:X}' -f $machine), not ARM64" }
Write-Output "ok $path (ARM64)"
}
# The installed CLI runs natively on ARM64.
$cli = Join-Path $dir 'lightcraft-cli.exe'
$out = & $cli --version 2>&1 | Out-String
if ($LASTEXITCODE -ne 0) { Write-Output $out; throw "lightcraft-cli --version exited $LASTEXITCODE" }
Write-Output "ok lightcraft-cli --version: $($out.Trim())"
$p = Start-Process msiexec.exe -ArgumentList '/x', "`"$msi`"", '/qn' -Wait -PassThru
if ($p.ExitCode -ne 0) { throw "msiexec /x exited $($p.ExitCode)" }
if (Test-Path (Join-Path $dir 'lightcraft.exe')) { throw 'uninstall left lightcraft.exe behind' }
Write-Output 'ok uninstalled'
+8
View File
@@ -1,8 +1,16 @@
*~
plan/
/target
/target-*/
/dist
/corpus
/log
.DS_Store
.claude/
/craft-fonts
# nix build / nix develop
/result
/result-*
# model weights are never committed (SAM 3: docs/ai-masks.md)
*.safetensors
*.safetensors.part
+76 -5
View File
@@ -1,12 +1,17 @@
# LightCraft — instructions for agents
LightCraft is a clean-room, open-source, pure-Rust photo library + non-destructive raw developer targeting Adobe Lightroom parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../printcraft` (Acrobat), `../photocraft` (Photoshop), `../drawcraft` (Illustrator) and `../filmcraft` (Premiere), with the same conventions.
LightCraft is a clean-room, open-source, pure-Rust photo library + non-destructive raw developer targeting Adobe Lightroom parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../pdfcraft` (Acrobat), `../photocraft` (Photoshop), `../vectorcraft` (Illustrator) and `../filmcraft` (Premiere), with the same conventions.
## Start every session here
1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers).
2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the README/docs of the crate you touch. Behaviour/visual reference: `plan/lightroom/` (incl. `10-observed-ui.md` + screenshots).
3. **Pick work** from [`docs/parity.md`](docs/parity.md) → *Top gaps* (Lightroom parity tracker: one row per feature, menu item and shortcut). When you land a feature, update its row(s) and the gap list in the same commit; `cargo xtask parity` (part of `ci`) checks that every `cmd:`/`ctl:` id and path the tracker cites still exists, and `cargo xtask parity --write` refreshes its summary.
4. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask unless §7 lists the decision as the user's.
3. **Know where we stand:** read [`ROADMAP.md`](ROADMAP.md) → *Where we stand* (honest assessment by dimension and
by kind of user) and *Where we're going* (ordered priorities). The checklist counts features that *exist*; the real
gaps are quality and coverage: camera colour calibration, CR3 / compressed raws, per-model verification, render
fidelity against Lightroom, AI models, HDR / video / Classic modules. A ✅ row is not proof of parity — if you find a
✅ feature that is wrong or incomplete, downgrade it to 🟡 with a note.
4. **Pick work** from [`docs/parity.md`](docs/parity.md) → *Top gaps* (ordered by user impact; the tracker has one row per feature, menu item and shortcut). When you land a feature, update its row(s) and the gap list in the same commit, and the *Where we stand* / *Where we're going* sections of `ROADMAP.md` when a listed gap closes; `cargo xtask parity` (part of `ci`) checks that every `cmd:`/`ctl:` id and path the tracker cites still exists, and `cargo xtask parity --write` refreshes its summary.
5. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask unless §7 lists the decision as the user's.
`plan/` is gitignored (local-only).
@@ -17,6 +22,34 @@ branches can still break each other (e.g. a new struct field vs. a new construct
`git log main..HEAD` + `git status` for unmerged commits or uncommitted work before starting anything new. Commit small
and often so a crash loses minutes, not hours.
## Never crash (outranks feature work)
People trust LightCraft with their photo libraries and edits; a crash loses their work. A malformed raw/JPEG/XMP, a bad
command, control or MCP argument, a corrupt catalog or settings file, or a full disk must produce an error the user (or
agent) can act on, never a panic. Don't ship a feature by adding a panic path, and fix a crash before building on top
of it. Full standard: `../craftrules/standards/never-crash.md`
([storytold/craftrules](https://github.com/storytold/craftrules/blob/main/standards/never-crash.md)).
- **Non-test code never panics:** no `unwrap()`, `expect()`, `panic!`, `unreachable!`, `todo!`, `unimplemented!`, and
no `unsafe`. Return the crate's error type through `Result` and `?`; use `ok_or(..)?`, `let … else`, `if let`, or a
fallback (`unwrap_or…`) only where it can't silently corrupt a document. An unfinished feature returns an
"unsupported" error or is disabled. Sole exception: a provably infallible literal, as `#[allow(clippy::expect_used)]`
+ `.expect("why it can't fail")`.
- **`unsafe` lives only in `crates/sysmem`** (one FFI call, `malloc_zone_pressure_relief`, that returns freed
allocator pages to macOS after raw decodes). Every other production crate root has `#![forbid(unsafe_code)]`. A new
unsafe need goes in an isolated, well-tested helper crate like it: `// SAFETY:` on every block, a safe API, a safe
fallback where possible, and a line here naming it.
- **Input-derived numbers are hostile:** `get()` instead of `[i]`/`[a..b]` for offsets from files, users, agents or
arithmetic on them; checked/saturating math for lengths, offsets and counts; no division by zero, NaN/inf or negative
casts to `usize`; cap allocations sized by input; slice strings only at char boundaries.
- **Bound recursion** (depth limits or seen-sets: IFD chains, nested metadata, collections). **Don't cascade:**
`lock().unwrap_or_else(PoisonError::into_inner)` or an error; worker-thread joins are `Result`s.
- **Last-resort guard:** a panic hook plus `catch_unwind` around command dispatch and import/export turns an escaped
panic into an error dialog and keeps the document. It's a safety net, not a licence; keep `panic = "unwind"` on native.
- **Prove it:** every crash fix lands with a small synthetic regression test that panicked before the fix.
- Enforced by clippy: root `clippy.toml` allows unwrap/expect/panic/indexing in tests only, and every production crate
root (`lib.rs`, each binary's `main.rs`) carries `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic,
clippy::unimplemented, clippy::todo, clippy::unreachable)]`. Not `[workspace.lints]`: those would also hit
integration tests, examples and benches. New crates start with the attribute.
## Non-negotiables
- **Clean-room.** Never read/disassemble anything inside Adobe app bundles (names/listings only). Never copy Adobe icons, presets, profiles (DCP), lens profiles (LCP), camera matrices, fonts. Observation of the installed Lightroom is read-only (it syncs the user's personal library: never import/edit/rate/delete there). Never copy GPL/LGPL/AGPL code (darktable, RawTherapee, ART, LibRaw, rawspeed, rawloader, rawler, lensfun, dcraw-derived GPL code…).
- **Pure Rust** in the product. No C/C++ dependencies.
@@ -29,9 +62,25 @@ and often so a crash loses minutes, not hours.
## Assets: icons, images, fonts (ABSOLUTE RULE — never violate)
- **Never use any iconography, image, artwork, font, sound or other asset from Adobe products** (no Lightroom/Creative Cloud icons, no screenshots, no presets/profiles/LUTs, no UI bitmaps — not even as a temporary placeholder or "reference copy"). Observing Adobe's UI to imitate *layout and behaviour* is allowed; copying or tracing its assets is not.
- **This includes Adobe's open-licensed assets**: no Source Sans/Serif/Code or Source Han fonts, no Adobe Fonts, no
Adobe-published icon sets, sample photos, colour profiles or LUTs — even when OFL/MIT. The UI font is Inter (OFL).
Adobe-published icon sets, sample photos, colour profiles or LUTs — even when OFL/MIT. The UI font is Inter (OFL);
Japanese and Chinese fonts come from craft-fonts (below). **One exception (maintainer decision, 2026-10-07): Noto
Sans/Serif CJK** (Google-branded, OFL, co-developed with Adobe as Source Han) is allowed via craft-fonts for Chinese
text, because nearly every OFL Chinese face derives from it. Use it unmodified under its OFL; this does not open the
door to Source Han under Adobe's name or to any other Adobe asset.
- Every asset in the repository must be one of: **our own original work** (e.g. icons drawn in code as vectors, procedurally generated demo photos), **public domain / CC0**, **Creative Commons** (CC-BY / CC-BY-SA with attribution honoured), **OFL** (fonts), or **permissive open-source** (MIT/Apache-2.0/BSD/ISC) — or contributed by a person who created the asset and licenses it openly.
- **Exception: `docs/brand/`.** The ArtCraft name, wordmark and logos there are ArtCraft Team trademarks, not open source and not covered by LightCraft's MIT OR Apache-2.0 licence (`LICENSE-MIT`, `LICENSE-APACHE`, `NOTICE`); their terms are in `docs/brand/LICENSE-brand.txt`. Use them only unmodified and never redraw, recolour or derive from them.
- **Every asset must have an entry in `assets/ATTRIBUTION.md`** (path, title, author/creator, source URL or "original work", licence, date added, modifications) and its licence text when required (e.g. `assets/fonts/OFL-*.txt`). Add the entry in the same commit as the asset. Assets without an attribution entry must not be committed.
- **Fonts live in [storytold/craft-fonts](https://github.com/storytold/craft-fonts), never in this repo.** Don't commit
font files here (Inter, already in `assets/fonts/`, is the one exception); add new fonts to craft-fonts. LightCraft
uses it as the optional build input `CRAFT_FONTS_DIR`: `git clone https://github.com/storytold/craft-fonts ../craft-fonts`
then `CRAFT_FONTS_DIR=../craft-fonts cargo run -p lightcraft` (or any cargo/xtask command). `crates/engine/build.rs`
embeds the manifest's fonts as `lightcraft_engine::CRAFT_FONTS` (wasm32: BIZ UDPGothic Regular only); `cargo xtask web`
ships Noto Sans CJK SC Regular as a separate asset loaded by the UI before startup. The UI
(`theme::font_definitions`) and the export watermark renderer use CJK faces (picked by script) as fallbacks after Inter. Unset,
`CRAFT_FONTS` is empty: everything builds, tests and runs, but Chinese and Japanese text lack glyphs. Releases always build
with it (`release.yml`, `CRAFT_FONTS_REQUIRED=1`) and ship the fonts' OFL licences. Tests that need these fonts skip
without it; the FreeBSD CI job runs them with it. Rules: craftrules
[`standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md).
- Icons drawn in code (e.g. `crates/ui-egui/src/icons.rs`) are original work and are recorded in `assets/ATTRIBUTION.md` as such; do not trace them from Adobe icons.
- Demo/test images: generated procedurally by `lightcraft-scenes`, or CC0 downloads kept in the gitignored `corpus/` with their source recorded. Screenshots of Adobe apps live only in the gitignored `plan/` and are never committed or published.
- **Enforced:** `cargo xtask assets` (in `ci`) fails when an image/icon/font/sound/video/raw/ICC/XMP file is not matched
@@ -57,6 +106,7 @@ and often so a crash loses minutes, not hours.
- Shell gotcha: `mv`/`cp` are aliased interactive here — use `/bin/mv -f` / `/bin/cp -f`.
- Parallel agents: separate git worktrees and `CARGO_TARGET_DIR=target/agent-<name>`; each agent uses its **own control port** (pick one in 18000–19999, never the default 7980) and its own scratch subfolder (`<scratch>/<agent-name>/`) — never `rm -rf` shared paths; delete your target dir when done (disk is shared); keep every `Cargo.toml` valid at all times (the `crates/*` glob means one broken manifest breaks everyone).
- Test corpora: `cargo xtask corpus --download` into `corpus/` (gitignored, CC0 only). Never commit media.
- Shared real-file test corpora (Photoshop-authored PSDs, etc.) live in [`storytold/photocraft-corpus`](https://github.com/storytold/photocraft-corpus), explained in [craftrules `standards/test-corpora.md`](https://github.com/storytold/craftrules/blob/main/standards/test-corpora.md). Never commit large binary fixtures to this repo; fetch them pinned by commit and sha256-verified, as PhotoCraft does with `cargo xtask corpus`.
## Testing & performance (do this often)
- Unit/property tests next to the code; end-to-end tests drive real binaries (`crates/mcp/tests/e2e.rs`,
@@ -73,4 +123,25 @@ and often so a crash loses minutes, not hours.
## Map of the code
`geom`, `color`, `raster`, `tiff` (L0) → `raw`, `codecs`, `meta`, `develop` (L1) → `pipeline` → `catalog` → `engine`
→ `ui-egui`, `mcp` (L5) → apps `lightcraft` (desktop), `lightcraft-cli` (render/commands/MCP). `scenes` generates demo
photos. `xtask` = tooling (`ci`, `layers`, `assets`, `parity`, `wasm`, `corpus`, `stats`).
photos. `xtask` = tooling (`ci`, `layers`, `assets`, `parity`, `wasm`, `corpus`, `stats`). `flake.nix` +
`nix/package.nix` = the Nix package (`nix build` builds both binaries with the craft-fonts input, installs the
desktop file/icons/AppStream metadata and runs `cargo test --workspace`; `nix develop` = dev shell). Community-maintained and not
in CI: it may lag behind the workspace; see README → Quick start.
## Contributor credits (About window)
- About ▸ Contributors/Models are compiled into the binary from `contributors/contributors.json`
(commit stats; generated, never hand-edit) and `contributors/people.toml` (names people chose for
themselves). See `docs/contributors.md`.
- **Agents working for a contributor:** when you prepare a PR, check whether your human's GitHub
username has a `[people.<username>]` entry in `contributors/people.toml`. If not, ask them once
whether they want to be credited by more than their username: a real name, a display name, and/or
their public GitHub profile name (`sync_github_name = true`). If yes, add **only their own** entry
(copy the template at the top of the file, or run
`python3 ../../craftrules/scripts/contributors.py --add-me . --real-name "…" --sync-github-name`)
and include it in their PR, committed as them. If no, change nothing: they are credited as
`@username` anyway.
- Never add, edit, guess or copy anyone else's entry or name (not from git config, commit authors or
GitHub profiles). Never hand-edit `contributors.json`.
- Maintainers refresh the stats with `python3 ../../craftrules/scripts/contributors.py .` (it also
re-verifies who wrote each `people.toml` entry; `--check` only verifies).
+58 -5
View File
@@ -1,12 +1,17 @@
# LightCraft — instructions for agents
LightCraft is a clean-room, open-source, pure-Rust photo library + non-destructive raw developer targeting Adobe Lightroom parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../printcraft` (Acrobat), `../photocraft` (Photoshop), `../drawcraft` (Illustrator) and `../filmcraft` (Premiere), with the same conventions.
LightCraft is a clean-room, open-source, pure-Rust photo library + non-destructive raw developer targeting Adobe Lightroom parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../pdfcraft` (Acrobat), `../photocraft` (Photoshop), `../vectorcraft` (Illustrator) and `../filmcraft` (Premiere), with the same conventions.
## Start every session here
1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers).
2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the README/docs of the crate you touch. Behaviour/visual reference: `plan/lightroom/` (incl. `10-observed-ui.md` + screenshots).
3. **Pick work** from [`docs/parity.md`](docs/parity.md) → *Top gaps* (Lightroom parity tracker: one row per feature, menu item and shortcut). When you land a feature, update its row(s) and the gap list in the same commit; `cargo xtask parity` (part of `ci`) checks that every `cmd:`/`ctl:` id and path the tracker cites still exists, and `cargo xtask parity --write` refreshes its summary.
4. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask unless §7 lists the decision as the user's.
3. **Know where we stand:** read [`ROADMAP.md`](ROADMAP.md) → *Where we stand* (honest assessment by dimension and
by kind of user) and *Where we're going* (ordered priorities). The checklist counts features that *exist*; the real
gaps are quality and coverage: camera colour calibration, CR3 / compressed raws, per-model verification, render
fidelity against Lightroom, AI models, HDR / video / Classic modules. A ✅ row is not proof of parity — if you find a
✅ feature that is wrong or incomplete, downgrade it to 🟡 with a note.
4. **Pick work** from [`docs/parity.md`](docs/parity.md) → *Top gaps* (ordered by user impact; the tracker has one row per feature, menu item and shortcut). When you land a feature, update its row(s) and the gap list in the same commit, and the *Where we stand* / *Where we're going* sections of `ROADMAP.md` when a listed gap closes; `cargo xtask parity` (part of `ci`) checks that every `cmd:`/`ctl:` id and path the tracker cites still exists, and `cargo xtask parity --write` refreshes its summary.
5. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask unless §7 lists the decision as the user's.
`plan/` is gitignored (local-only).
@@ -17,6 +22,34 @@ branches can still break each other (e.g. a new struct field vs. a new construct
`git log main..HEAD` + `git status` for unmerged commits or uncommitted work before starting anything new. Commit small
and often so a crash loses minutes, not hours.
## Never crash (outranks feature work)
People trust LightCraft with their photo libraries and edits; a crash loses their work. A malformed raw/JPEG/XMP, a bad
command, control or MCP argument, a corrupt catalog or settings file, or a full disk must produce an error the user (or
agent) can act on, never a panic. Don't ship a feature by adding a panic path, and fix a crash before building on top
of it. Full standard: `../craftrules/standards/never-crash.md`
([storytold/craftrules](https://github.com/storytold/craftrules/blob/main/standards/never-crash.md)).
- **Non-test code never panics:** no `unwrap()`, `expect()`, `panic!`, `unreachable!`, `todo!`, `unimplemented!`, and
no `unsafe`. Return the crate's error type through `Result` and `?`; use `ok_or(..)?`, `let … else`, `if let`, or a
fallback (`unwrap_or…`) only where it can't silently corrupt a document. An unfinished feature returns an
"unsupported" error or is disabled. Sole exception: a provably infallible literal, as `#[allow(clippy::expect_used)]`
+ `.expect("why it can't fail")`.
- **`unsafe` lives only in `crates/sysmem`** (one FFI call, `malloc_zone_pressure_relief`, that returns freed
allocator pages to macOS after raw decodes). Every other production crate root has `#![forbid(unsafe_code)]`. A new
unsafe need goes in an isolated, well-tested helper crate like it: `// SAFETY:` on every block, a safe API, a safe
fallback where possible, and a line here naming it.
- **Input-derived numbers are hostile:** `get()` instead of `[i]`/`[a..b]` for offsets from files, users, agents or
arithmetic on them; checked/saturating math for lengths, offsets and counts; no division by zero, NaN/inf or negative
casts to `usize`; cap allocations sized by input; slice strings only at char boundaries.
- **Bound recursion** (depth limits or seen-sets: IFD chains, nested metadata, collections). **Don't cascade:**
`lock().unwrap_or_else(PoisonError::into_inner)` or an error; worker-thread joins are `Result`s.
- **Last-resort guard:** a panic hook plus `catch_unwind` around command dispatch and import/export turns an escaped
panic into an error dialog and keeps the document. It's a safety net, not a licence; keep `panic = "unwind"` on native.
- **Prove it:** every crash fix lands with a small synthetic regression test that panicked before the fix.
- Enforced by clippy: root `clippy.toml` allows unwrap/expect/panic/indexing in tests only, and every production crate
root (`lib.rs`, each binary's `main.rs`) carries `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic,
clippy::unimplemented, clippy::todo, clippy::unreachable)]`. Not `[workspace.lints]`: those would also hit
integration tests, examples and benches. New crates start with the attribute.
## Non-negotiables
- **Clean-room.** Never read/disassemble anything inside Adobe app bundles (names/listings only). Never copy Adobe icons, presets, profiles (DCP), lens profiles (LCP), camera matrices, fonts. Observation of the installed Lightroom is read-only (it syncs the user's personal library: never import/edit/rate/delete there). Never copy GPL/LGPL/AGPL code (darktable, RawTherapee, ART, LibRaw, rawspeed, rawloader, rawler, lensfun, dcraw-derived GPL code…).
- **Pure Rust** in the product. No C/C++ dependencies.
@@ -29,9 +62,25 @@ and often so a crash loses minutes, not hours.
## Assets: icons, images, fonts (ABSOLUTE RULE — never violate)
- **Never use any iconography, image, artwork, font, sound or other asset from Adobe products** (no Lightroom/Creative Cloud icons, no screenshots, no presets/profiles/LUTs, no UI bitmaps — not even as a temporary placeholder or "reference copy"). Observing Adobe's UI to imitate *layout and behaviour* is allowed; copying or tracing its assets is not.
- **This includes Adobe's open-licensed assets**: no Source Sans/Serif/Code or Source Han fonts, no Adobe Fonts, no
Adobe-published icon sets, sample photos, colour profiles or LUTs — even when OFL/MIT. The UI font is Inter (OFL).
Adobe-published icon sets, sample photos, colour profiles or LUTs — even when OFL/MIT. The UI font is Inter (OFL);
Japanese and Chinese fonts come from craft-fonts (below). **One exception (maintainer decision, 2026-10-07): Noto
Sans/Serif CJK** (Google-branded, OFL, co-developed with Adobe as Source Han) is allowed via craft-fonts for Chinese
text, because nearly every OFL Chinese face derives from it. Use it unmodified under its OFL; this does not open the
door to Source Han under Adobe's name or to any other Adobe asset.
- Every asset in the repository must be one of: **our own original work** (e.g. icons drawn in code as vectors, procedurally generated demo photos), **public domain / CC0**, **Creative Commons** (CC-BY / CC-BY-SA with attribution honoured), **OFL** (fonts), or **permissive open-source** (MIT/Apache-2.0/BSD/ISC) — or contributed by a person who created the asset and licenses it openly.
- **Exception: `docs/brand/`.** The ArtCraft name, wordmark and logos there are ArtCraft Team trademarks, not open source and not covered by LightCraft's MIT OR Apache-2.0 licence (`LICENSE-MIT`, `LICENSE-APACHE`, `NOTICE`); their terms are in `docs/brand/LICENSE-brand.txt`. Use them only unmodified and never redraw, recolour or derive from them.
- **Every asset must have an entry in `assets/ATTRIBUTION.md`** (path, title, author/creator, source URL or "original work", licence, date added, modifications) and its licence text when required (e.g. `assets/fonts/OFL-*.txt`). Add the entry in the same commit as the asset. Assets without an attribution entry must not be committed.
- **Fonts live in [storytold/craft-fonts](https://github.com/storytold/craft-fonts), never in this repo.** Don't commit
font files here (Inter, already in `assets/fonts/`, is the one exception); add new fonts to craft-fonts. LightCraft
uses it as the optional build input `CRAFT_FONTS_DIR`: `git clone https://github.com/storytold/craft-fonts ../craft-fonts`
then `CRAFT_FONTS_DIR=../craft-fonts cargo run -p lightcraft` (or any cargo/xtask command). `crates/engine/build.rs`
embeds the manifest's fonts as `lightcraft_engine::CRAFT_FONTS` (wasm32: BIZ UDPGothic Regular only); `cargo xtask web`
ships Noto Sans CJK SC Regular as a separate asset loaded by the UI before startup. The UI
(`theme::font_definitions`) and the export watermark renderer use CJK faces (picked by script) as fallbacks after Inter. Unset,
`CRAFT_FONTS` is empty: everything builds, tests and runs, but Chinese and Japanese text lack glyphs. Releases always build
with it (`release.yml`, `CRAFT_FONTS_REQUIRED=1`) and ship the fonts' OFL licences. Tests that need these fonts skip
without it; the FreeBSD CI job runs them with it. Rules: craftrules
[`standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md).
- Icons drawn in code (e.g. `crates/ui-egui/src/icons.rs`) are original work and are recorded in `assets/ATTRIBUTION.md` as such; do not trace them from Adobe icons.
- Demo/test images: generated procedurally by `lightcraft-scenes`, or CC0 downloads kept in the gitignored `corpus/` with their source recorded. Screenshots of Adobe apps live only in the gitignored `plan/` and are never committed or published.
- **Enforced:** `cargo xtask assets` (in `ci`) fails when an image/icon/font/sound/video/raw/ICC/XMP file is not matched
@@ -57,6 +106,7 @@ and often so a crash loses minutes, not hours.
- Shell gotcha: `mv`/`cp` are aliased interactive here — use `/bin/mv -f` / `/bin/cp -f`.
- Parallel agents: separate git worktrees and `CARGO_TARGET_DIR=target/agent-<name>`; each agent uses its **own control port** (pick one in 18000–19999, never the default 7980) and its own scratch subfolder (`<scratch>/<agent-name>/`) — never `rm -rf` shared paths; delete your target dir when done (disk is shared); keep every `Cargo.toml` valid at all times (the `crates/*` glob means one broken manifest breaks everyone).
- Test corpora: `cargo xtask corpus --download` into `corpus/` (gitignored, CC0 only). Never commit media.
- Shared real-file test corpora (Photoshop-authored PSDs, etc.) live in [`storytold/photocraft-corpus`](https://github.com/storytold/photocraft-corpus), explained in [craftrules `standards/test-corpora.md`](https://github.com/storytold/craftrules/blob/main/standards/test-corpora.md). Never commit large binary fixtures to this repo; fetch them pinned by commit and sha256-verified, as PhotoCraft does with `cargo xtask corpus`.
## Testing & performance (do this often)
- Unit/property tests next to the code; end-to-end tests drive real binaries (`crates/mcp/tests/e2e.rs`,
@@ -73,4 +123,7 @@ and often so a crash loses minutes, not hours.
## Map of the code
`geom`, `color`, `raster`, `tiff` (L0) → `raw`, `codecs`, `meta`, `develop` (L1) → `pipeline` → `catalog` → `engine`
→ `ui-egui`, `mcp` (L5) → apps `lightcraft` (desktop), `lightcraft-cli` (render/commands/MCP). `scenes` generates demo
photos. `xtask` = tooling (`ci`, `layers`, `assets`, `parity`, `wasm`, `corpus`, `stats`).
photos. `xtask` = tooling (`ci`, `layers`, `assets`, `parity`, `wasm`, `corpus`, `stats`). `flake.nix` +
`nix/package.nix` = the Nix package (`nix build` builds both binaries with the craft-fonts input, installs the
desktop file/icons/AppStream metadata and runs `cargo test --workspace`; `nix develop` = dev shell). Community-maintained and not
in CI: it may lag behind the workspace; see README → Quick start.
Generated
+1841 -31
View File
File diff suppressed because it is too large. Load diff
+14 -1
View File
@@ -3,7 +3,7 @@ resolver = "3"
members = ["crates/*", "apps/*", "xtask"]
[workspace.package]
version = "0.1.1"
version = "0.4.0"
edition = "2024"
license = "MIT OR Apache-2.0"
rust-version = "1.90"
@@ -11,7 +11,11 @@ repository = "https://github.com/storytold/lightcraft"
[workspace.dependencies]
# internal
lightcraft-denoise-core = { path = "crates/denoise-core" }
lightcraft-denoise = { path = "crates/denoise", default-features = false }
lightcraft-geom = { path = "crates/geom" }
lightcraft-faces = { path = "crates/faces" }
lightcraft-sysmem = { path = "crates/sysmem" }
lightcraft-color = { path = "crates/color" }
lightcraft-raster = { path = "crates/raster" }
lightcraft-tiff = { path = "crates/tiff" }
@@ -28,9 +32,13 @@ lightcraft-merge = { path = "crates/merge" }
lightcraft-engine = { path = "crates/engine" }
lightcraft-ui-egui = { path = "crates/ui-egui" }
lightcraft-mcp = { path = "crates/mcp" }
lightcraft-fetch = { path = "crates/fetch" }
lightcraft-heif = { path = "crates/heif" }
lightcraft-segment = { path = "crates/segment" }
# foundation
thiserror = "2"
sha2 = "0.10"
serde = { version = "1", features = ["derive", "rc"] }
serde_json = { version = "1", features = ["float_roundtrip"] }
log = "0.4"
@@ -58,9 +66,14 @@ tiff = "0.11"
image-webp = "0.2"
moxcms = { version = "0.9", default-features = false, features = ["lut"] }
jxl-oxide = { version = "0.12", default-features = false }
zune-jpegxl = { version = "0.5", default-features = false, features = ["std"] } # MIT/Apache-2.0/Zlib lossless JXL encoder (test fixtures only)
ravif = { version = "0.13", default-features = false, features = ["threading"] } # no "asm": it needs nasm (non-Rust)
# gpu (crates/gpu; native backends only, no GL: everything stays pure Rust)
# SAM 3 inference (lightcraft-segment). 0.9.2: later releases pull `tokenizers` with the
# Oniguruma C library into candle-core, and the product is pure Rust.
candle-core = { version = "=0.9.2", default-features = false }
candle-nn = { version = "=0.9.2", default-features = false }
wgpu = { version = "30", default-features = false, features = ["std", "parking_lot", "metal", "vulkan", "dx12", "wgsl"] }
pollster = "1"
+177
View File
@@ -0,0 +1,177 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 ArtCraft Team and the LightCraft contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+38 -7
View File
@@ -1,12 +1,43 @@
LightCraft
Copyright the LightCraft contributors. Licensed under MIT OR Apache-2.0.
Copyright (c) 2026 ArtCraft Team and the LightCraft contributors.
Licensed under the MIT License or the Apache License, Version 2.0, at your option
(LICENSE-MIT, LICENSE-APACHE).
This product bundles third-party material under the following licences:
The ArtCraft name, wordmark and logos in docs/brand/ are trademarks of the ArtCraft Team. They are
not open source and are not covered by the license above; their terms are in
docs/brand/LICENSE-brand.txt.
- Source Sans 3 (assets/fonts/SourceSans3-*) — SIL Open Font License 1.1. See assets/fonts/OFL-SourceSans3.txt.
- Inter (assets/fonts/Inter-*.ttf) — SIL Open Font License 1.1. See assets/fonts/OFL-Inter.txt.
Third-party material
Every bundled asset (fonts, icons, images) and its licence is listed in assets/ATTRIBUTION.md.
This product bundles third-party material under its own licenses, listed below. The per-file
list (author, source, license) is in assets/ATTRIBUTION.md.
Algorithms are implemented from public specifications and papers (see crate READMEs). Rust dependencies
(egui, eframe, image, rayon, serde, …) are permissively licensed (MIT/Apache-2.0/BSD/ISC/Zlib) and fetched through crates.io.
Inter (assets/fonts/Inter-*.ttf) — SIL Open Font License 1.1. See assets/fonts/OFL-Inter.txt.
Showcase photographs by Ansel Adams, Dorothea Lange and NASA (docs/images/) — public domain. See assets/ATTRIBUTION.md.
crates/segment (lightcraft-segment) ports the SAM 3 model code of Hugging Face Transformers
(https://github.com/huggingface/transformers, src/transformers/models/sam3 and sam3_tracker),
Copyright The HuggingFace Team and Meta Platforms, Inc., licensed under the Apache License 2.0.
The port was modified by the LightCraft contributors (2026): translated from Python/PyTorch to
Rust on candle and restructured; each ported file says so. Because of this, crates/segment is
licensed under the Apache License 2.0 only (LICENSE-APACHE), not under the MIT option.
The SAM 3 weights are not part of LightCraft and are never distributed with it. A user downloads
the facebook/sam3 checkpoint only when they choose to (in the app, or by hand), under Meta's SAM
License (https://github.com/facebookresearch/sam3/blob/main/LICENSE), a separate, non-OSI
licence. See docs/ai-masks.md.
AI denoise weights are not part of LightCraft and are never distributed with it.
The default build offers user-supplied models only. The optional rawnind-model
feature offers a separately downloaded RawNIND UtNet2 Bayer checkpoint under
GPL-3.0 (https://www.gnu.org/licenses/gpl-3.0.html), after the user accepts its terms.
Commercial use is subject to those terms, including applicable source, licence and
notice obligations on redistribution. Enabling that offer in releases remains a
maintainer decision. The ONNX reader and CPU/GPU executor are our Rust implementation;
no darktable, RawSpeed, LibRaw or other GPL decoder/runtime code is included.
Builds made with craft-fonts (the optional CRAFT_FONTS_DIR build input; all official releases)
also embed the fonts listed in https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md
(the Japanese fonts BIZ UDPGothic, BIZ UDMincho and Shippori Mincho; the web build embeds only BIZ
UDPGothic Regular), under the SIL Open Font License 1.1. Release packages include each font's
licence as OFL-<family>.txt.
+161 -21
View File
@@ -59,6 +59,7 @@
<a href="#feature-status">Status</a> ·
<a href="#quick-start">Quick start</a> ·
<a href="ROADMAP.md">Roadmap</a> ·
<a href="#downloads">Downloads</a> ·
<a href="#the-crafting-apps">Crafting Apps</a>
</p>
@@ -200,7 +201,7 @@ A library that stays out of your way: **All Photos**, **Recently Added**, **Pick
**Folders**, and **Recently Deleted**. Rate with <kbd>0</kbd>–<kbd>5</kbd>, flag with <kbd>P</kbd> / <kbd>X</kbd> /
<kbd>U</kbd>, colour-label with <kbd>6</kbd>–<kbd>9</kbd>. Search understands fields:
`rating:>3 flag:pick iso:>800 camera:x2 date:2026-04 keyword:mountains`. Every view sorts by capture date,
import date, edit date, name, rating or size. The justified **Photo Grid** and **Square Grid** views are virtualized,
import date, edit date, name, rating, size or at random (a stable shuffle; View → Sort → Reshuffle for a new one). The justified **Photo Grid** and **Square Grid** views are virtualized,
so they stay smooth whether you have forty photos or forty thousand.
<table>
@@ -233,11 +234,11 @@ lightcraft --control 7980 ~/Pictures/trip
{"method": "ui.screenshot", "params": {"path": "after.png"}}
```
- **91 engine commands** and **48 UI commands.** List them all with `engine.commands`; read every slider's range,
default and current value with `develop.controls`.
- **MCP server.** `lightcraft-cli mcp` gives Claude (or any MCP client) ~100 tools: import, query, develop, mask,
render (returned as an image), export. It runs headless, or attached to the running app with screenshots, clicks
and gestures. See [docs/mcp.md](docs/mcp.md).
- **Command registry.** `engine.commands` lists the available commands; `develop.controls` lists every slider's
range, default and current value.
- **MCP server.** `lightcraft-cli mcp` exposes the command registry to Claude (or any MCP client), alongside
helpers for import, query, develop, mask, render (returned as an image) and export. It runs headless, or attached
to the running app with screenshots, clicks and gestures. See [docs/mcp.md](docs/mcp.md).
```sh
cargo build --release -p lightcraft-cli
@@ -257,9 +258,10 @@ lightcraft --control 7980 ~/Pictures/trip
## Fast, native, private
- **Pure Rust, no C.** Our own RAW decoders (DNG, Canon CR2, Sony ARW, Nikon NEF, Fujifilm RAF incl. X-Trans,
Panasonic RW2, Pentax PEF, Olympus ORF), our own colour science, our own pipeline. JPEG, PNG, TIFF, WebP, PSD
composites and JPEG XL open today.
- **Pure Rust, no C.** Our own RAW decoders (DNG, Canon CR2/CR3, Sony ARW, Nikon NEF, Fujifilm RAF incl. X-Trans,
Panasonic RW2 / Leica RWL, Pentax PEF, Olympus ORF), our own colour science, our own pipeline. JPEG, PNG, TIFF, WebP,
PSD composites and JPEG XL open today; HEIC/HEIF photos too with `--features heif` (HEVC is a build-time choice,
as in PhotoCraft).
- **Scene-referred & wide-gamut.** Linear Rec.2020 float internally, Bradford-adapted white balance, gamut mapping
instead of clipping, a filmic shoulder for raw and pixel-exact pass-through for JPEGs you haven't touched.
- **Resolution-independent edits.** Radii and brush sizes are relative to the image, so a 400 px preview, your
@@ -280,9 +282,18 @@ lightcraft --control 7980 ~/Pictures/trip
## Feature status
LightCraft is young and moving fast; see the [roadmap](ROADMAP.md) for estimates. The detailed, row-by-row
Lightroom parity tracker (every feature, menu item and shortcut with its status, evidence and the top gaps) is
[docs/parity.md](docs/parity.md).
LightCraft is young and moving fast. **Where we honestly stand** (details in the [roadmap](ROADMAP.md#where-we-stand)):
- **By feature count we're at ~79%** of Lightroom (core features 98%), tracked row by row in
[docs/parity.md](docs/parity.md).
- **As a day-to-day Lightroom replacement we're nearer 60–70%.** It's great for JPEG/DNG and most Nikon / Sony /
older-Canon raws on one machine.
- **The biggest gaps:**
- **camera colour calibration:** Sony, Nikon, Panasonic, Fujifilm and Canon CR3 raws have guarded estimates from their camera JPEGs, with built-in ILCE-7M4, X-H2S and X-T4 profiles; measured calibration is missing, and other raws or rejected fits retain a neutral matrix;
- **compressed Olympus raws and unsupported CR3 variants:** these use embedded JPEG previews when present. Fujifilm lossless/lossy compressed RAF now decodes sensor data; [verification and existing-library reload instructions](docs/raf-compression.md);
- **AI masks and denoise:** subject and sky selection are classical heuristics;
- **HDR, video and the Classic Print / Book / Map modules.**
- **What's next:** see [where we're going](ROADMAP.md#where-were-going).
| Area | Status |
|---|---|
@@ -292,17 +303,20 @@ Lightroom parity tracker (every feature, menu item and shortcut with its status,
| Masking: brush, linear, radial, luminance/colour range, add/subtract/intersect | ✅ (AI subject/sky use classical heuristics for now) |
| Crop, straighten tool + auto straighten, flip, rotate, aspect ratios, overlays | ✅ |
| Profiles (Color, Neutral, Vivid, Landscape, Portrait, Monochrome: our own looks), presets, versions, history, copy/paste/sync settings | ✅ |
| Camera colour: DNG files use their own matrices | ✅ DNG · 🟡 own Sony/Fujifilm profiles; measured calibration database missing |
| Native macOS menu bar (generated from the command registry), control channel + every widget addressable, headless UI snapshots | ✅ |
| RAW: DNG, CR2, ARW, NEF (uncompressed), Fujifilm RAF (Bayer + X-Trans), Panasonic RW2, Pentax PEF, Olympus ORF; embedded previews for every format incl. CR3 | ✅ · compressed NEF/RAF/ORF, CR3 decode 🚧 |
| RAW: DNG, CR2, CR3 (lossless CRX Bayer and version 0x100/0x200 C-RAW), ARW, NEF (uncompressed + lossless/lossy compressed), Fujifilm RAF (uncompressed + lossless/lossy compressed, Bayer + X-Trans), Panasonic RW2 / Leica RWL / Panasonic RAW (every raw format, DMC-LX1 to DC-S1RM2), Pentax PEF, Olympus ORF (uncompressed), Samsung SRW (uncompressed); embedded previews for every format incl. CR3 | 🟡 · CR3, compressed ORF and compressed SRW decode ⬜ |
| Detail: sharpening, luminance + colour noise reduction | ✅ · AI Denoise, Super Resolution ⬜ |
| Remove / Heal / Clone spots (auto source), Visualize Spots, Red Eye and Pet Eye (auto pupil detection, catchlight) | ✅ · content-aware fill, spot pin editing 🚧 |
| Export: JPEG / PNG / TIFF / WebP / AVIF, sizing, file-size limit, output sharpening, naming, batch, metadata policy, text watermark | ✅ · DNG export, image watermark ⬜ |
| Library persistence (crash-safe op log + snapshots), disk thumbnail cache, import with duplicate detection | ✅ |
| Export: JPEG / PNG / TIFF / WebP / AVIF / DNG / original, sizing, file-size limit, output sharpening, naming templates, batch, metadata policy, text or image watermark | ✅ · HDR export ⬜ |
| Library persistence (crash-safe op log + snapshots, background compaction, failed saves reported), disk thumbnail cache | ✅ |
| Import: Add in place / Copy / Move, rename and folder templates, devices, duplicate detection, watched folders; Local folder browsing | ✅ |
| MCP server (headless or live app, persistent libraries), CLI, control channel | ✅ |
| XMP sidecars (read/write, auto-write), reading `crs:` develop settings, preset files (`.lcpreset`, XMP presets) | ✅ |
| Optics (distortion, vignetting, auto + manual CA, defringe, DNG-embedded lens corrections), Geometry (transforms, Constrain Crop), Upright (Auto/Level/Vertical/Full/Guided) | ✅ · camera lens profiles (our own) ⬜ |
| Optics (distortion, vignetting, auto + manual CA, defringe, lens corrections embedded in DNG files and Panasonic / Leica RW2 / RWL distortion data), Geometry (transforms, Constrain Crop), Upright (Auto/Level/Vertical/Full/Guided) | ✅ · camera lens profiles (our own) ⬜ |
| Photo Merge: HDR (auto-align, deghost), Panorama (spherical/cylindrical/perspective, boundary warp, auto crop), HDR Panorama → DNG | ✅ |
| GPU pipeline (wgpu compute, CPU-exact within 1/255) with CPU fallback | ✅ · WebGPU in the browser 🚧 |
| GPU pipeline (wgpu compute, CPU-exact within 1/255), CPU fallback on device limits / errors | ✅ · WebGPU in the browser 🚧 |
| AI: segmentation masks, AI denoise, super resolution, faces; HDR editing; video | ⬜ (see [roadmap](ROADMAP.md#where-were-going)) |
| Web build (same UI in the browser via WASM): persistent library in OPFS/IndexedDB, Web Worker rendering, export downloads | ✅ · WebGPU, Safari/Firefox testing 🚧 |
<sub>✅ works today · 🚧 in progress · ⬜ not started</sub>
@@ -322,9 +336,69 @@ cargo run --release -p lightcraft-cli -- render photo.jpg -o out.jpg --set light
cargo xtask ci # fmt, clippy, tests, layering, wasm checks
```
CI defaults to line-table debug information and one build job per 1.5 GB of RAM
available when it starts (at most one per CPU; 4 if memory can't be read), so an
8 GB machine with 6 GB free builds with 4, and a busy machine with little free
builds with fewer. Test threads follow, capped at 4. Full-debuginfo linkers, one
per CPU, otherwise exhaust RAM before any test runs. GPU coverage is unchanged. Explicit
`CARGO_PROFILE_DEV_DEBUG`, `CARGO_BUILD_JOBS` and `RUST_TEST_THREADS` settings
override these defaults. Ordinary development commands keep their own settings.
**Chinese and Japanese text** need the shared font repo, an optional build input (official releases always include it):
```sh
git clone https://github.com/storytold/craft-fonts ../craft-fonts
CRAFT_FONTS_DIR=../craft-fonts cargo run --release -p lightcraft
```
Without it LightCraft builds and runs the same, but Chinese and Japanese text have no glyphs. Fonts are never committed to this
repo; see [craftrules `standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md).
**Interface language:** **Edit → Language** (English, 简体中文, 繁體中文(台灣), 日本語, Português (Brasil), Deutsch) or **Settings → General →
Language**; the choice applies immediately and persists. See [docs/localization.md](docs/localization.md).
**Logs:** the desktop app writes its log to standard error and to `logs/lightcraft.log` in its settings folder
(Linux `$XDG_CONFIG_HOME/lightcraft/logs/`, by default `~/.config/lightcraft/logs/`; macOS
`~/Library/Application Support/LightCraft/logs/`; Windows `%APPDATA%\LightCraft\logs\`), never in the library. A
launch from a desktop menu or the Dock has no terminal, so attach this file to a bug report (**Help ▸ Open Log
Folder** shows it in the file manager). Each start moves the
previous log to `lightcraft.1.log` and that one to `lightcraft.2.log`, so the log of a run that crashed survives the
next start; the file stops growing at 16 MiB, `--version` and `--help` write none, and runs with
`LIGHTCRAFT_NO_PREFS` log to standard error only. By default LightCraft's own crates log at `info` and everything else
at `warn`. `LIGHTCRAFT_LOG=info` or `debug` works as before (that level for LightCraft's own crates, warnings and
errors from the rest; any other value: warnings and errors only) and wins over `RUST_LOG`, which otherwise replaces
the default with env_logger-style directives such as `RUST_LOG=debug` or `RUST_LOG=warn,lightcraft_pipeline=trace` (a
directive ending in `*` covers every target starting with it, as in `lightcraft*=debug`). Panics are recorded there
too, and still in `lightcraft-panics.log` in the temp folder. The logger is `apps/lightcraft/src/logging.rs`.
The web build needs the `wasm32-unknown-unknown` target and the matching `wasm-bindgen` CLI
(`cargo xtask web` prints the exact install command); see [docs/web.md](docs/web.md).
**Nix** builds the desktop app and `lightcraft-cli` (the Nix build always includes the craft-fonts input, so Japanese
text has glyphs):
```sh
nix run github:storytold/lightcraft # the desktop app
nix build github:storytold/lightcraft # → ./result/bin/{lightcraft,lightcraft-cli}
nix develop github:storytold/lightcraft # rust toolchain + native deps + fonts
```
In a flake configuration (NixOS, home-manager, nix-darwin):
```nix
# flake.nix
inputs.lightcraft.url = "github:storytold/lightcraft";
# optional: build against your own nixpkgs instead of the one LightCraft pins
# inputs.lightcraft.inputs.nixpkgs.follows = "nixpkgs";
# then, in a NixOS or home-manager module (where `inputs` is in scope):
nixpkgs.overlays = [ inputs.lightcraft.overlays.default ]; # makes `pkgs.lightcraft` available
environment.systemPackages = [ pkgs.lightcraft ]; # home-manager: home.packages = [ pkgs.lightcraft ];
```
`nix build` installs the same desktop file, hicolor icons and AppStream metadata as the .deb/.rpm, and runs
`cargo test --workspace` as its check phase (skip it with `pkgs.lightcraft.overrideAttrs { doCheck = false; }`).
**Keyboard:** <kbd>G</kbd> grid · <kbd>D</kbd> detail · <kbd>E</kbd> edit · <kbd>C</kbd> crop · <kbd>M</kbd> masking ·
<kbd>Shift</kbd>+<kbd>P</kbd> presets · <kbd>\\</kbd> original · <kbd>Y</kbd> before/after · <kbd>Z</kbd> zoom ·
<kbd>J</kbd> clipping · <kbd>⌘Z</kbd> undo · <kbd>⌘/</kbd> all shortcuts.
@@ -344,14 +418,65 @@ Humans and agents follow the same rules, so read [AGENTS.md](AGENTS.md) first. T
work from public specs and black-box observation.
- **No Adobe assets, ever:** no icons, screenshots, presets, profiles, LUTs or fonts from Adobe products. Every
image, icon and font in the repo is original, public domain, Creative Commons, OFL or permissively licensed, and has
an entry in [assets/ATTRIBUTION.md](assets/ATTRIBUTION.md) added in the same commit.
an entry in [assets/ATTRIBUTION.md](assets/ATTRIBUTION.md) added in the same commit. New fonts go to
[storytold/craft-fonts](https://github.com/storytold/craft-fonts), not here.
- **Pure Rust**, enforced crate layering, everything is a command, and `cargo xtask ci` green before every commit
(one task id per commit).
- **Never crash.** Non-test code returns errors instead of panicking: no `unwrap()`, `expect()`, `panic!` or
`unsafe`, checked indexing on anything derived from input, and a regression test with every crash fix. Details in
[AGENTS.md](AGENTS.md#never-crash-outranks-feature-work).
Questions, ideas or a bug you'd like to talk through first? Bring them to [Discord](https://discord.gg/artcraft).
<br>
## Downloads
**New to LightCraft?** Download it from the [LightCraft page on getartcraft.com](https://getartcraft.com/apps/lightcraft). That's the easiest way to install it.
**Want a specific build or format?** On GitHub, the [latest release](https://github.com/storytold/lightcraft/releases/latest) has every build listed below, and [all releases](https://github.com/storytold/lightcraft/releases) has earlier versions and their notes. `<ver>` in the file names is the version number, and `SHA256SUMS.txt` lists a checksum for every file.
### Windows
| Build | Installer | Portable |
|---|---|---|
| x64 (64-bit Intel/AMD) | `lightcraft-<ver>-windows-x64.msi` | `lightcraft-<ver>-windows-x64-portable.zip` |
| arm64 (Snapdragon and other ARM PCs) | `lightcraft-<ver>-windows-arm64.msi` | `lightcraft-<ver>-windows-arm64-portable.zip` |
| x86 (32-bit) | `lightcraft-<ver>-windows-x86.msi` | `lightcraft-<ver>-windows-x86-portable.zip` |
Installers and executables are code-signed.
### macOS
| Build | File | Notes |
|---|---|---|
| App, universal (Apple silicon + Intel) | `lightcraft-<ver>-macos-universal.dmg` | Signed and notarized |
| Command-line tool, universal | `lightcraft-cli-<ver>-macos-universal.zip` | Signed and notarized |
### Linux
| Format | x86_64 | aarch64 (ARM64) | Notes |
|---|---|---|---|
| AppImage | `lightcraft-<ver>-linux-x86_64.AppImage` | `lightcraft-<ver>-linux-aarch64.AppImage` | Runs anywhere; updates itself with [AppImageUpdate](https://github.com/AppImageCommunity/AppImageUpdate) (`.zsync` files) |
| Flatpak | `lightcraft-<ver>-linux-x86_64.flatpak` | `lightcraft-<ver>-linux-aarch64.flatpak` | Sandboxed; `flatpak install --user <file>` |
| Debian/Ubuntu | `lightcraft-<ver>-linux-x86_64.deb` | `lightcraft-<ver>-linux-aarch64.deb` | |
| Fedora/RHEL/openSUSE | `lightcraft-<ver>-linux-x86_64.rpm` | `lightcraft-<ver>-linux-aarch64.rpm` | |
| Tarball | `lightcraft-<ver>-linux-x86_64.tar.gz` | `lightcraft-<ver>-linux-aarch64.tar.gz` | Unpack anywhere |
### FreeBSD
| Build | File |
|---|---|
| x86_64 | `lightcraft-<ver>-freebsd-x86_64.tar.gz` |
### Web (WebAssembly)
| Build | File | Notes |
|---|---|---|
| Static site | `lightcraft-web-<ver>.zip` | Runs in a modern browser; host it on any static server |
<br>
## The Crafting Apps
LightCraft is one of the **Crafting Apps**: free, open-source creative tools from the
@@ -364,7 +489,7 @@ stand on its own.
| <img src="https://raw.githubusercontent.com/storytold/vectorcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.vectorcraft.png" alt="" width="32" height="32"> | **VectorCraft** | Vector illustration | [GitHub](https://github.com/storytold/vectorcraft) | [Website](https://getartcraft.com/apps/vectorcraft) |
| <img src="https://raw.githubusercontent.com/storytold/filmcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.filmcraft.png" alt="" width="32" height="32"> | **FilmCraft** | Video editing, color and sound | [GitHub](https://github.com/storytold/filmcraft) | [Website](https://getartcraft.com/apps/filmcraft) |
| <img src="https://raw.githubusercontent.com/storytold/lightcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.lightcraft.png" alt="" width="32" height="32"> | **LightCraft** | **Photo library and raw development · you are here** | [GitHub](https://github.com/storytold/lightcraft) | [Website](https://getartcraft.com/apps/lightcraft) |
| <img src="https://raw.githubusercontent.com/storytold/printcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.printcraft.png" alt="" width="32" height="32"> | **PrintCraft** | Reading, organizing and protecting PDFs | [GitHub](https://github.com/storytold/printcraft) | [Website](https://getartcraft.com/apps/printcraft) |
| <img src="https://raw.githubusercontent.com/storytold/pdfcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.pdfcraft.png" alt="" width="32" height="32"> | **PdfCraft** | Reading, organizing and protecting PDFs | [GitHub](https://github.com/storytold/pdfcraft) | [Website](https://getartcraft.com/apps/pdfcraft) |
| <img src="https://raw.githubusercontent.com/storytold/effectcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.effectcraft.png" alt="" width="32" height="32"> | **EffectCraft** | Motion graphics and visual effects | [GitHub](https://github.com/storytold/effectcraft) | [Website](https://getartcraft.com/apps/effectcraft) |
| <img src="https://raw.githubusercontent.com/storytold/designcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.designcraft.png" alt="" width="32" height="32"> | **DesignCraft** | Page layout and publishing | [GitHub](https://github.com/storytold/designcraft) | [Website](https://getartcraft.com/apps/designcraft) |
@@ -396,12 +521,23 @@ And [**ArtCraft**](https://getartcraft.com/) itself, our AI image and video stud
## License and credits
LightCraft is licensed under **MIT OR Apache-2.0**.
LightCraft is dual-licensed under [MIT](LICENSE-MIT) or [Apache-2.0](LICENSE-APACHE), at your option.
Copyright (c) 2026 ArtCraft Team and the LightCraft contributors. Required notices are in [NOTICE](NOTICE).
Bundled fonts, icons, images and other assets keep their own open licenses; each one is listed
with its author, source and license in [assets/ATTRIBUTION.md](assets/ATTRIBUTION.md).
Showcase photographs are public-domain works, used via Wikimedia Commons: Ansel Adams, *The Tetons and the Snake River*
(1942, U.S. National Archives); Dorothea Lange, *Migrant Mother* (1936, Library of Congress); Bill Anders / NASA,
*Earthrise* (1968); NASA, *The Blue Marble* (1972). The demo library is procedurally generated by LightCraft. UI font:
Inter (SIL OFL). All icons are original. See [assets/ATTRIBUTION.md](assets/ATTRIBUTION.md).
Inter (SIL OFL 1.1). Builds made with [craft-fonts](https://github.com/storytold/craft-fonts) (all official releases)
also embed its Chinese and Japanese fonts (Noto Sans CJK SC, BIZ UDPGothic, BIZ UDMincho, Shippori Mincho; SIL OFL 1.1), listed in its
[ATTRIBUTION.md](https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md). All icons are original.
The ArtCraft name, wordmark and logos in [`docs/brand/`](docs/brand/) are trademarks of the
ArtCraft Team and are not covered by this license. They may be used only unmodified, and only as
part of this repository and LightCraft, under [`docs/brand/LICENSE-brand.txt`](docs/brand/LICENSE-brand.txt).
Forks and modified versions must remove them.
<sub>Adobe, Photoshop, Illustrator, Premiere Pro, Lightroom, Acrobat, After Effects and InDesign are trademarks or registered trademarks of Adobe Inc. in the United States and/or other countries. LightCraft is an independent, open-source project and is not affiliated with, sponsored by or endorsed by Adobe Inc.; these names are used only to describe the workflows it is compatible with.</sub>
@@ -409,3 +545,7 @@ Inter (SIL OFL). All icons are original. See [assets/ATTRIBUTION.md](assets/ATTR
<a href="https://getartcraft.com/"><img alt="ArtCraft" src="docs/brand/artcraft-mark.svg" width="28"></a><br>
<sub>Made by the <a href="https://getartcraft.com/">ArtCraft</a> team and community.</sub>
</p>
## Star history
[![Star History Chart](https://api.star-history.com/svg?repos=storytold/lightcraft&type=Date&legend=top-left)](https://www.star-history.com/?repos=storytold%2Flightcraft&type=date&legend=top-left)
+143 -29
View File
@@ -4,6 +4,77 @@ Milestones toward full Adobe Lightroom parity (cloud Lightroom first, then every
estimates for continuous (24/7) agent-driven development with 4–6 parallel agents. Estimates are calibrated on the sibling
projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revised as milestones land.
## Where we stand
*Honest assessment, 2026-10-08. Agents: read this before picking work. The checklist in
[`docs/parity.md`](docs/parity.md) counts features that **exist**; this section is about whether a photographer can
**switch** from Lightroom. Update it when a gap below closes.*
**In one line:** the checklist says **79.7%** (P0 98.2%, P1 95.6%, P2 41.6%), but measured by whether a working
photographer could replace Lightroom without noticing, we are at roughly **60–70%**. The remaining gap is mostly
**quality of results and camera coverage**, not missing buttons.
Caveat on the checklist: ✅ is set by whoever lands a feature, and nobody has systematically checked rows against
Lightroom's behaviour or output. Bugs keep turning up in ✅ areas (CR2 colour-filter phase on some Canon models #85,
duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
### By dimension
| Dimension | Estimate | What's true today | Biggest gaps |
|---|---:|---|---|
| **Feature checklist** | 79.7% | P0 core and P1 nearly complete: import (Add / Copy / Move, templates, devices), library, grid/loupe/compare/survey, every Edit slider, curves, colour grading, masking tools, crop/Upright, heal/clone, presets/profiles, versions/history, sync, export, menus, shortcuts | P0: full-resolution zoom detail; P1: lens-profile database, content-aware fill (patch synthesis), video playback/trim |
| **RAW coverage** (formats people shoot) | ~55% | DNG (all kinds), CR2, CR3 lossless Bayer and version 0x100/0x200 C-RAW (M50/R100/R8 pixel-exact corpus), ARW, NEF (uncompressed + Huffman lossless/lossy), RAF (uncompressed, lossless/lossy compressed Bayer and X-Trans), uncompressed ORF, uncompressed Samsung SRW (NX5 to NX1100, EX1, WB2000), RW2 / RWL / Panasonic RAW (every raw format, checked on 178 files from 118 Panasonic and Leica bodies), PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed ORF, compressed Samsung SRW (NX1, NX30, NX300, NX500, NX2000, NX3000, NX3300, NX mini), Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. NEFs labelled compressed but stored uncompressed (Z 6 packed 14-bit, D850 12-bit uncompressed). Per-model verification is thin (~55 corpus files vs >1,000 models) |
| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW, NEF, RW2, RAF, CR3, CR2 and PEF have guarded camera-JPEG colour estimates, with bundled ILCE-7M4, X-H2S and X-T4 profiles (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
| **AI & computational** | ~15–20% | Assisted culling (focus, bursts), auto tone, HDR/panorama merge; subject/sky/background masks as classical heuristics, or the photo's own segmentation mattes when it ships them (DNG semantic masks, e.g. iPhone ProRAW's sky matte); non-destructive Bayer RAW denoise with opt-in weights on pure-Rust CPU/GPU; **faces (MVP, [docs/faces.md](docs/faces.md))**: a pure-Rust detector (a 232 KB opt-in download), opt-in recognition models (one-click download or add a file; they run on the shared pure-Rust CPU interpreter; pinned SFace/AuraFace embeddings agree with an independent reference runtime on 40 public crops; one-thread medians 34–36 / 233–252 ms (AuraFace remains expensive)), a background scan that finds and embeds faces, name suggestions, a person's page with look-alike faces to confirm, an Unnamed faces section (look-alikes together, select a group and name it at once) | Real segmentation masks (subject, sky, people, objects, landscape, depth), denoise model policy / linear RGB / quality verification, super resolution, lens blur, generative remove, natural-language search; for faces: clusters with separators between groups, small faces in large photos (the detector looks at 640 px), thresholds calibrated on live faces, writing face regions back to XMP. **Blocked on a model strategy** (licensable weights or our own training; pure-Rust inference is feasible) |
| **Workflow & library** | ~85% (single machine) | Robust catalog (journal + snapshots, background compaction, crash-tested), 85k-photo libraries stay responsive, Local browsing with automatic cleanup, XMP interop, keywords, smart albums, Move import | Opening an 85k library takes 1.7–4.7 s; no cloud sync (out of scope), no tablet companion (#74, roadmap), shared albums, publish services, tethering |
| **Classic modules** | ~30% | Geotagging from GPX track logs, soft proofing (partial), slideshow (basic) | **Map view, Book, Print, Slideshow module, Web, publish services**: ~40 tracker rows ⬜ |
| **HDR & video** | 0% | | HDR edit/display/export; video play/trim/edit/export |
| **Platform & robustness** | ~70% | macOS native; Windows/Linux builds; web via WASM; no-panic lints workspace-wide, `unsafe` confined to `crates/sysmem`; failed saves are reported; GPU errors fall back to CPU | Windows installer UI unverified on Windows (PR #79); GPU path proven only on Apple + user reports; English, Simplified/Traditional Chinese, Japanese, Brazilian Portuguese and German UI (see docs/localization.md); remaining technical errors; Traditional Chinese uses the Simplified Chinese font until craft-fonts has a TC face; accessibility partial; headless UI tests time out under machine load |
### By kind of user
| User | Readiness | What blocks them |
|---|---:|---|
| JPEG / DNG shooter, single machine | ~85% | Fidelity polish, AI masks |
| Nikon / Sony / Panasonic / older-Canon raw shooter | ~65% | Camera colour fidelity and coverage (ARW, NEF and RW2 preview estimates are only a starting point); Panasonic compacts and kit zooms also miss their embedded distortion correction |
| Fujifilm raw shooter | ~65% | Uncompressed and lossless/lossy compressed RAF decoded; camera colour calibration remains missing. Verified on 13 bodies, including 14/16-bit GFX; see `docs/raf-compression.md` |
| Canon CR3 / Olympus compressed-raw shooter | ~45% | CR3 lossless Bayer and version 0x100/0x200 C-RAW develop from sensor data (M50, R100, R8 verified pixel for pixel); other CRX variants and compressed ORF open as embedded JPEG previews. Camera colour and broader model verification remain |
| Lightroom Classic power user | ~45% | Print, Book, Map, publish, tethering |
| Relies on AI (masks, denoise) | ~25% | Object / Describe masks via optional SAM 3 (download not yet hosted); Bayer AI denoise needs separately installed weights; X-Trans/linear RGB, model policy and quality parity remain open |
## Where we're going
Priorities, in order. Each points at tracker rows in [`docs/parity.md`](docs/parity.md) → *Top gaps*.
1. **Camera colour calibration of our own** (LR-PROF-CAMERACOLOR, P0): fit each camera to its own embedded JPEG, use
matrices the files carry themselves, then chart shots. Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3/CR2,
Pentax PEF and Samsung SRW have the file-local fit (matrix + tone curve from their own JPEG; CR2 and PEF since issue #310, not yet
checked on the corpus); then ORF, then validate fidelity.
2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): Remaining **unverified CR3 CRX** variants, compressed ORF, NEF
lossy-after-split, sRAW. Decided 2026-10-05: write our own decoders from prose descriptions (never decoder source,
no LGPL dependency); compressed NEF (#86) is the template.
3. **Verified camera coverage** (LR-IMP-CAMERA-COVERAGE, P0): a CC0 sample per model in the corpus, each decoded and
checked for plausible colour; fix per-model bugs (#85).
4. **Render fidelity suite** (LR-BEHAV-RENDER-FIDELITY, P1): measure our output against Lightroom on the same CC0 raws
(references stay in the local `plan/`), then tune against the numbers.
5. **Lens profiles of our own** (LR-EDIT-OPTICS-PROFILE, P1).
6. **AI model strategy** (maintainer decision): which permissively licensed models (or our own training) for
segmentation masks and denoise. Bayer denoise inference is implemented in pure Rust with an opt-in GPL model; release policy and independently licensed weights remain open. Unblocks M12 and Enhance. Face detection and recognition run on our own checked Rust engine. Object / Describe masks now
run Meta's SAM 3 in pure Rust (optional, user-consented download under the non-OSI SAM License; CDN mirrors still
to be set up, docs/ai-masks.md); Subject / Sky / People, Raw Details and Super Resolution are still open.
7. **Then:** HDR (Q), the Classic output modules (Print first, then Map view, Book, Slideshow), video (R), localisation
and accessibility.
Already closed in the week of 2026-10-03: compressed NEF (#10), Move import (#29), per-library smart-preview folder,
Local roots / cleanup, grid and catalog performance at 85k photos (#35, #37), failed-save reporting, GPU export
hardening (#78), copyright metadata (#51), GPX geotagging (#60), import tag help and folder templates (#31, #32).
Image navigation now supports native pinch zoom and two-finger pan in Detail, Compare and Reference views.
The whole-image preview still has a configurable size cap (2,560 px default), so 100% on larger photos does not
yet guarantee original-pixel detail; visible-region rendering remains a gap (LR-VIEW-ZOOM).
## Milestones
**Status legend:** ✅ done · 🚧 in progress · ⬜ not started
| # | Milestone | Scope (summary) | Estimate (h) | Status |
@@ -11,32 +82,35 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi
| M0 | Skeleton + visual shell | workspace, xtask CI + layering, geom/color/raster, develop model, pipeline v0, catalog v0, engine commands, Lightroom-look UI (grid, loupe, filmstrip, Edit panel), control channel, MCP, web build | 3–5 | ✅ |
| M1 | Library core | import (JPEG/PNG/TIFF/WebP), EXIF/XMP, persistent catalog (op log + snapshots), albums, ratings/flags/labels, filter/search/sort, thumbnail cache, 100k-photo grid | 6–10 | ✅ (100k-photo grid scale test pending) |
| M2 | Pipeline v1 (quality) | WB temp/tint, profiles, local tone mapping (highlights/shadows), curves, HSL, point colour, colour grading, texture/clarity/dehaze, vignette, grain, B&W, auto tone/WB, histogram, before/after | 10–15 | ✅ (look tuning vs our references ongoing) |
| M3 | RAW I | TIFF/DNG (LJ92, deflate, tiles, opcodes), demosaic (AHD/PPG/bilinear), highlight recovery, DNG colour model, CR2, NEF, ARW, embedded previews | 10–15 | 🚧 (DNG, CR2, ARW, NEF uncompressed, embedded previews ✅; NEF Huffman ⬜) |
| M3 | RAW I | TIFF/DNG (LJ92, deflate, tiles, opcodes), demosaic (AHD/PPG/bilinear), highlight recovery, DNG colour model, CR2, NEF, ARW, embedded previews | 10–15 | 🚧 (DNG, CR2, ARW, NEF uncompressed + Huffman lossless/lossy, embedded previews ✅; NEF lossy-after-split ⬜) |
| M4 | Crop, geometry, optics | crop tool + overlays, straighten, Upright (auto/level/vertical/full/guided), manual transforms, CA, defringe, manual lens corrections | 6–10 | ✅ |
| M5 | Performance | source pyramids, wgpu compute pipeline (CPU oracle), draft/full renders, prefetch, budgets (16 ms slider updates on 24 MP) | 10–15 | 🚧 (stage cache, source pyramid, wgpu pipeline, prefetch, memory budget ✅; colour NR at half resolution, GPU histogram ⬜) |
| M6 | Masking | brush, linear/radial gradients, colour/luminance/depth range, add/subtract/intersect/invert, all local adjustments, masks panel | 8–12 | 🚧 (brush/linear/radial/colour/luminance range, add/subtract/intersect, masks panel ✅; depth range, AI masks ⬜) |
| M7 | Detail | sharpening + masking preview, luminance/colour NR, Denoise, Raw Details, Super Resolution | 6–10 | 🚧 (sharpening, luminance/colour NR ✅; AI Denoise, Raw Details, Super Resolution ⬜) |
| M7 | Detail | sharpening + masking preview, luminance/colour NR, Denoise, Raw Details, Super Resolution | 6–10 | 🚧 (sharpening, luminance/colour NR ✅; AI Denoise 🟡 (Bayer, optional weights; policy/fidelity open); Raw Details, Super Resolution ⬜) |
| M8 | Heal / Remove | content-aware remove (PatchMatch), heal, clone, brush spots, visualize spots, red/pet eye | 6–10 | 🚧 (heal, clone, auto source, visualize spots, red/pet eye ✅; PatchMatch remove ⬜) |
| M9 | Presets, profiles, versions, sync | preset browser + amount, create/import presets, profile browser, versions, history, copy/paste/sync settings | 5–8 | ✅ |
| M10 | Export & share | export dialog (JPEG/PNG/TIFF/DNG/AVIF/JXL/original), sizing, sharpening, metadata, watermark, naming, batch jobs, XMP sidecars, HDR export | 6–10 | 🚧 (all formats incl. DNG/original, sizing, presets, background jobs ✅; JXL encode, HDR export ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 packed, PEF, ORF uncompressed ✅; CR3, compressed NEF/ORF/RAF ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (CR3 lossless CRX and C-RAW, RAF uncompressed and lossless/lossy compressed, RW2 (every raw format), PEF, ORF uncompressed ✅; **camera colour calibration** 🚧 (guarded ARW, NEF, RW2 and RAF preview fitting; measured database still missing), CR3, compressed ORF ⬜) |
| M12 | AI & smart features | subject/sky/background/people/object masks, semantic search, faces/People (permissively licensed models, pure-Rust inference) | 20–40 | ⬜ |
| M13 | Merge | HDR merge (deghost), panorama (projections, boundary warp, fill edges), HDR panorama | 10–15 | ✅ |
| M14 | Video | import/playback/trim via FilmCraft crates, global edits + presets on video, video export | 6–10 | ⬜ |
| M15 | Classic modules | Map, Book, Slideshow, Print, Web; smart collections, stacks, virtual copies, publish services, tethering | 25–40 | 🚧 (smart albums, stacks, virtual copies, compare/survey ✅; Map/Book/Slideshow/Print/Web ⬜) |
| M16 | 1.0 polish | preferences, shortcut editor, accessibility, localization, packaging (dmg/msi/AppImage/web), hardening | 10–20 | 🚧 (settings, keyboard shortcuts sheet, packaging basics ✅; accessibility, localisation ⬜) |
| M16 | 1.0 polish | preferences, shortcut editor, accessibility, localization, packaging (dmg/msi/AppImage/web), hardening | 10–20 | 🚧 (settings, keyboard shortcuts sheet, packaging basics ✅; English/Chinese (Simplified, Traditional)/Japanese/Brazilian Portuguese/German localisation 🟡; accessibility and further locales ⬜) |
## Parity estimate (2026-10-02, evening update)
## Parity estimate (feature count updated 2026-10-08; effort estimate from 2026-10-02)
**By feature count** — from `docs/parity.md` (one row per Lightroom feature, menu item and shortcut; `cargo xtask
parity` prints this line on every run, so it stays current):
| Scope | Weighted completion | Rows | Morning |
| Scope | Weighted completion | Rows | 2026-10-02 |
|---|---:|---:|---:|
| P0 (core) | **98.7%** | 198 | 98.5% |
| P1 (important parity) | **94.4%** | 144 | 79.9% |
| P2 (later / AI / niche, incl. Classic modules) | **29.4%** | 158 | 11.7% |
| **All in-scope rows** | **75.6%** | 500 | 65.7% |
| P0 (core) | **98.2%** | 200 | 98.7% |
| P1 (important parity) | **95.6%** | 149 | 94.4% |
| P2 (later / AI / niche, incl. Classic modules) | **41.6%** | 160 | 29.4% |
| **All in-scope rows** | **79.7%** | 509 | 75.6% |
The 2026-10-05 count includes three new 🟡 rows that make quality gaps visible (camera colour calibration, verified
camera coverage, render fidelity), which is why P0 dipped slightly.
✅ counts 1, 🟡 ½, ⬜ 0; out-of-scope rows (cloud sharing, Adobe accounts…) are left out.
@@ -50,7 +124,7 @@ trip, slideshow, auto import…) in ≈ 12 h on 2026-10-02; four to six parallel
| Work package | Tracker rows | Agent-hours | Risk |
|---|---|---:|---|
| Remaining P0/P1 UI and library features (folder rename/move, keyword painter, people view…) | ≈ 8 | 5–10 | low |
| Raw codecs: CR3 (CRX), compressed NEF / ORF / RAF, RW2 v4, HEIC/AVIF decode, JPEG XL DNG | LR-IMP-FORMATS | 40–80 | **high** — clean-room black-box analysis, no permissive specs |
| Raw codecs: CR3 (CRX), compressed ORF, NEF lossy-after-split, HEIC/AVIF decode | LR-IMP-FORMATS | 40–80 | **high** — clean-room black-box analysis, no permissive specs |
| Lens-profile database of our own (calibration targets, fitting, data) | LR-EDIT-OPTICS-PROFILE | 15–30 | data collection |
| Video: playback, trim, edits, export (pure-Rust decode, ideally shared with FilmCraft) | R. Video | 20–40 | medium |
| AI: subject / sky / background / people / object masks, object-aware remove, AI denoise, super resolution, lens blur, people & faces, natural-language search, culling | ≈ 30 | 80–150 | **high** — permissively licensed weights, pure-Rust inference, maybe training |
@@ -86,36 +160,76 @@ The milestone estimates in the table above were made before work started and are
fallbacks first. No permissively licensed sky-segmentation or raw-denoise model was found — we may need to train our own.
- **Camera colour and lens data** is a data problem: we never use Adobe's matrices, DCPs or LCPs. DNG-embedded data first,
then our own calibration; long-tail camera/lens coverage grows over time.
- **Legal decisions pending:** whether GPL-licensed *prose* format descriptions (e.g. the public CR3 write-up) may be read
by a designated engineer to produce an internal spec; freedom-to-operate review for local Laplacian filters, PatchMatch and
HEVC (HEIC).
- **Look parity** with Adobe's default rendering is subjective tuning against our own reference targets.
- **Raw-format sources:** decided 2026-10-05: decoders are written from *prose* format descriptions (even ones
published alongside GPL code); decoder source is never read. Still open: freedom-to-operate review for local
Laplacian filters, PatchMatch and HEVC (HEIC).
- **Look parity** with Adobe's default rendering is tuned by eye today; the planned fidelity suite (LR-BEHAV-RENDER-FIDELITY)
turns it into measured comparisons against local-only Lightroom references.
## Raw format coverage and known gaps
Sony ILCE-7M4 downsized lossless ARW (subsampled YCbCr 4:2:0 / 4:2:2 tiles) now decodes to linear RGB;
private 3:2 & 4:3 examples verified through native rendering & full-resolution 16-bit export.
CR3 lossless Bayer and version 0x100/0x200 C-RAW sensor decoding is verified pixel-exactly on Canon M50, R100 and R8; any other CR3, or one that fails to decode, opens from its embedded JPEG as before. See [`docs/cr3.md`](docs/cr3.md) for coding limits, reference provenance and colour limitations.
Decoded (CC0 corpus from raw.pixls.us, `cargo xtask corpus --download`, `crates/raw/tests/corpus.rs`): DNG (uncompressed,
LJ92, lossy JPEG / Smart Previews, Deflate, float, linear), CR2, ARW (uncompressed, ARW2, LJ92), NEF/NRW uncompressed, RAF uncompressed (Bayer and
X-Trans), RW2 packed 12/14-bit, PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Every
supported container also yields its embedded JPEG preview (CR3 too), and the engine shows that preview for raw variants
it can't decode yet.
LJ92, lossy JPEG / Smart Previews, Deflate, float, linear, DNG 1.7 JPEG XL tiles: lossless ones bit-exact on synthetic files,
lossy ones unclipped in the declared linear primaries, checked on one real file), CR2, ARW (uncompressed, ARW2, LJ92; as-shot white balance and black level of pre-2017 bodies from the enciphered
maker-note `Tag2010` and the encrypted `SR2SubIFD`, both recovered by black-box analysis, `crates/raw/src/vendor/arw.rs`), NEF/NRW uncompressed and Huffman-compressed (lossless, lossy type 1/2, 12/14-bit), RAF uncompressed and lossless/lossy compressed (Bayer and
X-Trans, 14/16-bit real-file verification), RW2 / Leica RWL / Panasonic RAW in every raw format (compressed formats 4 and 6, the prefix-coded strips of format 8,
packed formats 2, 5 and 7, the 16-bit words of the 2005–2007 bodies; all recovered by black-box analysis of 178 CC0 files from
118 bodies, `crates/raw/src/vendor/rw2.rs`), PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Supported containers yield their embedded JPEG preview when present (including CR3). The engine uses it for RAW variants it cannot decode yet; HEVC-only CR3 previews are not decoded.
Not decoded yet — preview only (no permissively licensed description; black-box analysis incomplete):
- **Nikon Huffman NEF** (lossless / lossy). Black-box findings so far: maker note `0x0096` holds per-parity predictor
seeds (2048 for 14-bit lossless); ≈ 9 bits per pixel; the category-0 code word is a 6-bit rotation of `011111`. The
Huffman tables are not stored in the files, and an automatic table search (beam search over prefix codes scored by
smoothness and bit rate, validated on Pentax files whose tables *are* stored) did not converge.
- **Panasonic RW2 raw format 4** (quantised): the block layout is known (0x4000-byte chunks rotated by 0x1ff8; 128-bit
blocks of two 12-bit seeds plus four groups of a 2-bit scale and three 8-bit codes; scales 0/1 are ×1/×2 differences
from the same-colour pixel two to the left), the reconstruction rule for scales 2/3 is not established.
- **Olympus compressed ORF**, **Fujifilm compressed RAF**, **Canon CR3/CRX** (M11.1), **Canon sRAW/mRAW**, lossy DNG.
- **Nikon "lossy after split" NEF** (non-zero split row in maker note `0x0096`, e.g. some D3400/D5000/D5200/D5500/D5600
files): rows above the split decode with the regular lossy table; from the split row on a different code is used
that our black-box analysis has not recovered yet (none of the four regular tables fits, also not with byte
alignment or reset predictors at the split row). The other Nikon Huffman variants are decoded (`crates/raw/src/vendor/nefc.rs` documents the analysis).
- **Olympus compressed ORF**, **Canon CR3 unverified CRX variants** (M11.1), **Canon sRAW/mRAW**.
**Camera colour matrices:** non-DNG raws use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
**Camera colour matrices:** ARW, NEF, RW2, RAF, CR3, CR2, PEF and Samsung SRW files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
`matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer
matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax/Panasonic equivalents) and our
matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax equivalents; Panasonic RW2 files carry none) and our
own chart-based calibration (M11.4). Adobe matrices are never used.
## Lightroom Classic catalog migration
File → Import Lightroom Catalog… reads `.lrcat` plus committed WAL pages directly through a pure-Rust
SQLite reader. Originals stay in place; ratings, flags, labels, XMP metadata, hierarchical keywords,
collections/sets, virtual copies and supported develop settings migrate into LightCraft. Existing edits
are preserved by default, reimport identities are persistent, and source settings/history/snapshots are
archived before catalog mutation. Missing originals remain available for relinking.
Rendering is approximate: unsupported Adobe profiles/AI/process settings are reported and archived;
history/snapshots remain source data, and smart collections import current membership. The Lightroom
database is read-only; LightCraft owns subsequent edits. See `docs/lightroom-catalog-import.md`.
## Log
- 2026-09-30: roadmap created; M0 in progress; research docs (Lightroom reference, Rust imaging ecosystem) complete.
- 2026-09-30 (later): app running with the full Lightroom-style UI; pipeline v0; DNG/CR2/ARW; README showcase. ≈ 8 h elapsed.
- 2026-10-02: parity estimate added (xtask parity prints weighted completion); milestone statuses refreshed.
- 2026-10-08: Fujifilm camera colour: guarded JPEG matching and relative WB, plus bundled X-H2S/X-T4 profiles from 201/545 photos. On 150 withheld files the pooled profiles improve mean ΔE slightly over per-file fitting (X-H2S 1.70 → 1.62, X-T4 2.43 → 2.34); dark X-H2S samples 8.74 → 2.25. Measured calibration remains open.
- 2026-10-08: Fujifilm lossless/lossy compressed RAF: pure-Rust striped decoder; full sensor arrays verified against a black-box reference on 17 CC0 files and 7 supplied X-H2S / X-T4 files (13 camera bodies, 14/16-bit Bayer and X-Trans). Camera colour calibration remains a separate gap.
- 2026-10-01: RAW II formats: RAF (uncompressed Bayer + X-Trans), RW2 (packed), PEF (incl. Huffman), ORF (uncompressed); embedded previews for every container incl. CR3; raw corpus test with 37 CC0 samples.
- 2026-10-03 – 10-05: community PRs (#42–#51, #60) and 25+ issues landed: compressed NEF, Move import, folder templates,
Local roots and cleanup, 85k-photo grid and catalog performance, background compaction, failed-save errors, GPU
export hardening, clippy 1.99. Added the honest *Where we stand* assessment and *Where we're going* priorities; added
tracker rows for camera colour, camera coverage and render fidelity.
- 2026-10-07: Panasonic RW2 / Leica RWL / Panasonic RAW decode in every raw format (compressed 4/6/8, packed 2/5/7,
16-bit words), black levels, in-camera aspect crops and mapped-out defects established on 178 CC0 files from 118
bodies; RW2 gets the file-local camera look (accepted on 140 of 174 files with a preview).
- 2026-10-08: independent Rust CR3 lossless and version 0x100/0x200 C-RAW decoding, six exact full-sensor regressions on M50/R100/R8; any CR3 decode error falls back to the embedded JPEG; guarded CR3 camera-JPEG colour fitting with independent framing validation (`docs/cr3.md`). Unverified CRX variants and broader model verification remain.
## Chinese and Japanese interfaces, and text watermarks
English, Simplified Chinese, Traditional Chinese (Taiwan) and Japanese interface languages are persisted in UI state, and the
language table (`crates/ui-egui/src/i18n.rs`) drives the menus, settings and fonts, so another
language is a table entry plus two catalogs (docs/localization.md). Core menus and panels are
translated; untranslated text retains English. CJK glyphs (Chinese: Noto Sans CJK SC; Japanese UI:
BIZ UDPGothic; watermarks: BIZ UDMincho) come from storytold/craft-fonts, embedded by builds made
with the optional `CRAFT_FONTS_DIR` input (all releases), so no system fonts are needed. Text watermarks now accept
`vertical: true` in export JSON/presets and expose an orientation selector. Japanese
characters stay upright in top-to-bottom columns, with newlines starting columns to the
left. This is basic lettering, without tate-chu-yoko, ruby, kinsoku, or general vertical
OpenType shaping. The same coverage renderer serves 8/16/32-bit exports.
+8
View File
@@ -15,6 +15,7 @@ workspace = true
[dependencies]
lightcraft-engine = { workspace = true }
lightcraft-sysmem = { workspace = true }
lightcraft-mcp = { workspace = true }
lightcraft-ui-egui = { workspace = true }
lightcraft-codecs = { workspace = true }
@@ -24,7 +25,14 @@ serde_json = { workspace = true }
dhat = { workspace = true, optional = true }
[features]
default = ["denoise"]
denoise = ["lightcraft-engine/denoise"]
rawnind-model = ["denoise", "lightcraft-engine/rawnind-model"]
# Heap profiling: `cargo run --release -p lightcraft-cli --features dhat-heap -- snapshot …` counts
# every allocation (live/peak bytes in `library.memory` / `ui.inspect`) and writes dhat-heap.json
# (open it in DHAT's viewer: allocation sites by bytes at the peak). Slows allocation-heavy work.
dhat-heap = ["dep:dhat"]
# Open HEIC/HEIF photos (the optional lightcraft-heif crate). Not default so distributors choose
# it (HEVC patents); build with `--features heif` to include it.
heif = ["lightcraft-codecs/heif"]
+6 -26
View File
@@ -1,32 +1,12 @@
//! Give memory the system allocator keeps after frees back to the system
//! ([`lightcraft_engine::memory::set_release_hook`]).
//!
//! macOS's allocator keeps freed large blocks mapped as "reusable": they no longer count in the
//! process's memory footprint but stay in its resident size until the system runs short. After a
//! raw decode (hundreds of MB of temporaries) that made the resident size climb towards the sum of
//! everything ever decoded. `malloc_zone_pressure_relief` (libSystem) returns those pages now.
//! Elsewhere (glibc, Windows) blocks this large are unmapped when freed: nothing to do.
//! ([`lightcraft_engine::memory::set_release_hook`]). The platform call lives in
//! `lightcraft-sysmem`, the workspace's only crate allowed `unsafe`.
/// Install the hook for this platform.
/// Install the hook.
pub fn install() {
#[cfg(target_os = "macos")]
lightcraft_engine::memory::set_release_hook(macos::release);
lightcraft_engine::memory::set_release_hook(release);
}
#[cfg(target_os = "macos")]
#[allow(unsafe_code)]
mod macos {
unsafe extern "C" {
/// libSystem malloc: release free memory of `zone` (all zones when null) back to the
/// system, up to `goal` bytes (0 = as much as possible). Returns the bytes released.
fn malloc_zone_pressure_relief(zone: *mut std::ffi::c_void, goal: usize) -> usize;
}
pub fn release() {
// SAFETY: a documented libSystem function taking a null zone (= every zone) and a size;
// it only returns free pages to the system and touches no memory we own.
unsafe {
malloc_zone_pressure_relief(std::ptr::null_mut(), 0);
}
}
fn release() {
let _ = lightcraft_sysmem::release_free_memory();
}
+225 -64
View File
@@ -9,7 +9,10 @@
//! lightcraft-cli synth-merge hdr|panorama -o DIR
//! lightcraft-cli commands [--json]
//! lightcraft-cli controls [--json]
//! lightcraft-cli calibrate [--max N] [--out DIR] FOLDERS/FILES…
//! ```
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
mod alloc_release;
@@ -68,6 +71,8 @@ USAGE:
per line: {\"method\": \"ui.set\", \"params\": {\"view\": \"detail\"}}.
Replies go to stdout. `ui.screenshot` without a path writes -o (then
OUT-2.png, OUT-3.png…); `ui.settle {timeoutMs?}` waits for renders.
A failed request (\"ok\": false) does not stop the script, but the exit
status is non-zero when any request failed.
-o, --output OUT PNG path (a final screenshot is written here if the script took none)
--size WxH window size in points (default 1600x1000)
--scale S pixels per point (default 1)
@@ -82,6 +87,12 @@ USAGE:
Write synthetic merge inputs (procedural scene; bracketed DNGs or overlapping PNG views).
lightcraft-cli commands [--json] list every command id with its parameters
lightcraft-cli controls [--json] list every develop control id with its range
lightcraft-cli calibrate [--max N] [--out DIR] FOLDERS/FILES…
Fit a colour profile per camera model from raw files and their embedded camera JPEGs
(Sony ARW, Nikon NEF, Fujifilm RAF): up to N files spread over the folders (default 300; 0 = all), pooled per
model, written as <model>.json to DIR (default: the profiles folder LightCraft reads,
<config>/camera-profiles, or $LIGHTCRAFT_CAMERA_PROFILES). Raws of a profiled model then
take their colour from the profile and only their tone from their own JPEG.
lightcraft-cli --version | --help
";
@@ -89,6 +100,25 @@ USAGE:
#[global_allocator]
static ALLOC: dhat::Alloc = dhat::Alloc;
/// Why `--library DIR` can't be opened; for a library open in another program, how to work with
/// that one instead (issue #99).
/// Print the open library's warnings (an unlockable library, damaged settings files) on stderr:
/// stdout carries command results / MCP protocol messages only.
fn library_warnings(session: &mut Session, who: &str) {
for w in session.take_library_warnings() {
eprintln!("{who}: warning: {w}");
}
}
fn library_error(dir: &str, e: lightcraft_engine::EngineError) -> String {
match e {
lightcraft_engine::EngineError::LibraryInUse(why) => format!(
"{dir}: {why}\nTo work with the library while the app has it open, start the app with `--control PORT` and use `lightcraft-cli mcp --connect 127.0.0.1:PORT`."
),
e => format!("{dir}: {e}"),
}
}
fn main() -> ExitCode {
// `--features dhat-heap`: count allocations; the profile is written when `_heap` drops
// (LIGHTCRAFT_DHAT_FILE, default dhat-heap.json).
@@ -102,6 +132,9 @@ fn main() -> ExitCode {
dhat::Profiler::builder().file_name(file).build()
};
alloc_release::install();
// Warnings (a GPU render redone on the CPU, an unknown backend name) on stderr; LIGHTCRAFT_LOG
// or RUST_LOG picks another level (#168).
lightcraft_engine::logging::install("lightcraft-cli");
let args: Vec<String> = std::env::args().skip(1).collect();
let r = match args.first().map(String::as_str) {
Some("run") => run(&args[1..]),
@@ -112,6 +145,7 @@ fn main() -> ExitCode {
Some("merge") => merge(&args[1..]),
Some("synth-merge") => synth_merge(&args[1..]),
Some("controls") => controls(&args[1..]),
Some("calibrate") => calibrate(&args[1..]),
Some("--version" | "-V" | "version") => {
println!("lightcraft-cli {}", env!("CARGO_PKG_VERSION"));
Ok(())
@@ -131,6 +165,115 @@ fn main() -> ExitCode {
}
}
/// Raw files below `path` (or `path` itself), skipping hidden and NAS metadata folders.
fn raw_files(path: &Path, out: &mut Vec<std::path::PathBuf>, depth: usize) {
let is_raw =
|p: &Path| p.extension().and_then(|e| e.to_str()).is_some_and(|e| ["arw", "nef", "nrw", "raf"].iter().any(|x| e.eq_ignore_ascii_case(x)));
if path.is_file() {
if is_raw(path) {
out.push(path.to_path_buf());
}
return;
}
let Ok(entries) = std::fs::read_dir(path) else { return };
if depth > 32 {
return;
}
for entry in entries.flatten() {
let name = entry.file_name();
if name.to_string_lossy().starts_with(['.', '@']) {
continue;
}
raw_files(&entry.path(), out, depth + 1);
}
}
fn calibrate(args: &[String]) -> Result<(), String> {
let mut max = 300usize;
let mut out: Option<std::path::PathBuf> = None;
let mut inputs = Vec::new();
let mut i = 0;
while i < args.len() {
match args[i].as_str() {
"--max" => max = take_value(args, &mut i, "--max")?.parse().map_err(|_| "--max needs a number".to_string())?,
"--out" => out = Some(take_value(args, &mut i, "--out")?.into()),
a if a.starts_with("--") => return Err(format!("unknown option `{a}`")),
f => inputs.push(f.to_string()),
}
i += 1;
}
if inputs.is_empty() {
return Err("calibrate needs folders or raw files".into());
}
let mut files = Vec::new();
for input in &inputs {
raw_files(Path::new(input), &mut files, 0);
}
files.sort();
files.dedup();
let found = files.len();
if max > 0 && files.len() > max {
// spread over all folders (dates, scenes) rather than the first N
files = (0..max).filter_map(|k| files.get(k * found / max).cloned()).collect();
}
eprintln!("calibrate: {} of {found} raw files", files.len());
let dir = out.or_else(lightcraft_engine::camera_profiles::dir).ok_or("no profiles folder: pass --out DIR")?;
let workers = std::thread::available_parallelism().map_or(4, |n| n.get()).clamp(1, 6);
let next = std::sync::atomic::AtomicUsize::new(0);
let done = std::sync::atomic::AtomicUsize::new(0);
let pools: Vec<lightcraft_engine::camera_profiles::Pool> = std::thread::scope(|scope| {
let handles: Vec<_> = (0..workers)
.map(|_| {
scope.spawn(|| {
let mut pool = lightcraft_engine::camera_profiles::Pool::default();
while let Some(path) = files.get(next.fetch_add(1, std::sync::atomic::Ordering::Relaxed)) {
let result = std::fs::read(path).map_err(|e| e.to_string()).and_then(|bytes| pool.add(&bytes));
let n = done.fetch_add(1, std::sync::atomic::Ordering::Relaxed) + 1;
match result {
Ok(Some(model)) => eprintln!("[{n}/{}] {} ({model})", files.len(), path.display()),
Ok(None) => eprintln!("[{n}/{}] {}: skipped (no usable camera JPEG or colour)", files.len(), path.display()),
Err(e) => eprintln!("[{n}/{}] {}: {e}", files.len(), path.display()),
}
}
pool
})
})
.collect();
handles.into_iter().filter_map(|h| h.join().ok()).collect()
});
let mut pool = lightcraft_engine::camera_profiles::Pool::default();
for p in pools {
pool.merge(p);
}
let mut written = 0;
for result in pool.fit(5) {
match result {
Ok(profile) => {
let path = lightcraft_engine::camera_profiles::save(&profile, &dir)?;
println!(
"{}: {} photos, {} colour pairs, hue/saturation table {} → {}",
profile.model,
profile.files,
profile.samples,
profile.hue_sat.is_some(),
path.display()
);
written += 1;
}
Err(e) => eprintln!("calibrate: {e}"),
}
}
for (model, n) in pool.files() {
if n < 5 {
eprintln!("calibrate: {model}: only {n} usable photo(s), at least 5 needed");
}
}
if written == 0 {
return Err("no profile written".into());
}
Ok(())
}
fn mcp(args: &[String]) -> Result<(), String> {
let mut connect: Option<String> = None;
let mut demo = false;
@@ -180,8 +323,9 @@ fn mcp(args: &[String]) -> Result<(), String> {
let mut h = match &library {
Some(dir) => {
let mut h = Headless::default();
let r = h.session.open_library(dir, demo).map_err(|e| e.to_string())?;
let r = h.session.open_library(dir, demo).map_err(|e| library_error(dir, e))?;
eprintln!("lightcraft-cli mcp: opened library {dir} ({r:?})");
library_warnings(&mut h.session, "lightcraft-cli mcp");
h
}
None if demo => Headless::demo(),
@@ -198,7 +342,7 @@ fn mcp(args: &[String]) -> Result<(), String> {
eprintln!("lightcraft-cli mcp: serving MCP on stdio ({})", backend.describe());
let mut server = Server::new(backend).with_command_tools(!compact);
let stdin = std::io::stdin();
server.serve(BufReader::new(stdin.lock()), std::io::stdout().lock()).map_err(|e| e.to_string())
server.serve(BufReader::new(stdin), std::io::stdout()).map_err(|e| e.to_string())
}
fn merge(args: &[String]) -> Result<(), String> {
@@ -247,7 +391,7 @@ fn merge(args: &[String]) -> Result<(), String> {
}
i += 1;
}
let mut s = Session::new().with_fs();
let mut s = Session::new().with_fs().with_default_denoise_models();
let paths = expand_paths(&files);
let r = s.execute("library.import", &json!({"paths": paths})).map_err(|e| e.to_string())?;
let mut ids: Vec<u64> = r["imported"].as_array().map(|a| a.iter().filter_map(Value::as_u64).collect()).unwrap_or_default();
@@ -283,14 +427,18 @@ fn synth_merge(args: &[String]) -> Result<(), String> {
let mut written = Vec::new();
match kind {
"hdr" => {
for (k, b) in lightcraft_merge::synth::bracket_dngs(1800, 1200, &[-2.0, 0.0, 2.0]).into_iter().enumerate() {
for (k, b) in lightcraft_merge::synth::bracket_dngs(1800, 1200, &[-2.0, 0.0, 2.0]).map_err(|e| e.to_string())?.into_iter().enumerate() {
let p = Path::new(&dir).join(format!("bracket-{k}.dng"));
std::fs::write(&p, b).map_err(|e| e.to_string())?;
written.push(p);
}
}
"panorama" | "pano" => {
for (k, v) in lightcraft_merge::synth::pano_views(1200, 900, 1000.0, &[-50.0, -25.0, 0.0, 25.0, 50.0]).into_iter().enumerate() {
for (k, v) in lightcraft_merge::synth::pano_views(1200, 900, 1000.0, &[-50.0, -25.0, 0.0, 25.0, 50.0])
.map_err(|e| e.to_string())?
.into_iter()
.enumerate()
{
let p = Path::new(&dir).join(format!("view-{k}.png"));
let img = v.to_srgb8();
let png = lightcraft_codecs::encode_png(&lightcraft_codecs::EncodeImage::rgba8(&img), &lightcraft_codecs::EncodeMeta::default())
@@ -399,39 +547,40 @@ fn run(args: &[String]) -> Result<(), String> {
if steps.is_empty() {
return Err("run: no command given (see `lightcraft-cli commands`)".into());
}
let mut backend: Box<dyn Backend> = match connect {
Some(addr) => {
if demo || library.is_some() || !imports.is_empty() {
return Err("--demo, --library and --import apply to headless mode only".into());
}
Box::new(
Remote::connect(&addr)
.map_err(|e| format!("LightCraft is not reachable at {addr} ({e}); start it with `lightcraft --control 7980`"))?,
)
}
None => {
let mut h = match &library {
Some(dir) => {
let mut h = Headless::default();
h.session.open_library(dir, demo).map_err(|e| e.to_string())?;
h
}
None if demo => Headless::demo(),
None => Headless::default(),
};
if !imports.is_empty() {
let paths = expand_paths(&imports);
let r = h.session.execute("library.import", &json!({"paths": paths})).map_err(|e| e.to_string())?;
// what follows acts on the imported photos (already-known files are reported as duplicates)
let mut ids: Vec<Value> = r["imported"].as_array().cloned().unwrap_or_default();
ids.extend(r["duplicates"].as_array().into_iter().flatten().filter_map(|d| d.get("existing").filter(|v| v.is_u64()).cloned()));
if let Some(first) = ids.first().cloned() {
h.session.execute("library.select", &json!({"ids": ids, "active": first})).map_err(|e| e.to_string())?;
let mut backend: Box<dyn Backend> =
match connect {
Some(addr) => {
if demo || library.is_some() || !imports.is_empty() {
return Err("--demo, --library and --import apply to headless mode only".into());
}
Box::new(Remote::connect(&addr).map_err(|e| {
format!("LightCraft is not reachable at {addr} ({e}); start it with `lightcraft --control {}`", connect_port(&addr))
})?)
}
Box::new(h)
}
};
None => {
let mut h = match &library {
Some(dir) => {
let mut h = Headless::default();
h.session.open_library(dir, demo).map_err(|e| library_error(dir, e))?;
library_warnings(&mut h.session, "lightcraft-cli");
h
}
None if demo => Headless::demo(),
None => Headless::default(),
};
if !imports.is_empty() {
let paths = expand_paths(&imports);
let r = h.session.execute("library.import", &json!({"paths": paths})).map_err(|e| e.to_string())?;
// what follows acts on the imported photos (already-known files are reported as duplicates)
let mut ids: Vec<Value> = r["imported"].as_array().cloned().unwrap_or_default();
ids.extend(r["duplicates"].as_array().into_iter().flatten().filter_map(|d| d.get("existing").filter(|v| v.is_u64()).cloned()));
if let Some(first) = ids.first().cloned() {
h.session.execute("library.select", &json!({"ids": ids, "active": first})).map_err(|e| e.to_string())?;
}
}
Box::new(h)
}
};
let mut out = std::io::stdout().lock();
let mut failed = 0;
for s in &steps {
@@ -457,6 +606,12 @@ fn take_value<'a>(args: &'a [String], i: &mut usize, flag: &str) -> Result<&'a s
args.get(*i).map(String::as_str).ok_or_else(|| format!("{flag} needs a value"))
}
/// The port an `--connect` address names, for the recovery hint. The whole address when it has
/// no port, so a hint is never built from a guess.
fn connect_port(addr: &str) -> &str {
addr.rsplit_once(':').map_or(addr, |(_, port)| port)
}
fn render(args: &[String]) -> Result<(), String> {
let mut input = None;
let mut output = None;
@@ -497,7 +652,7 @@ fn render(args: &[String]) -> Result<(), String> {
}
let input = input.ok_or("render: missing input file")?;
let output = output.ok_or("render: missing -o OUTPUT")?;
let mut s = Session::new().with_fs();
let mut s = Session::new().with_fs().with_default_denoise_models();
let abs = expand_paths(std::slice::from_ref(&input));
let r = s.execute("library.import", &json!({"paths": abs})).map_err(|e| e.to_string())?;
let id = r["imported"][0].as_u64().ok_or_else(|| format!("{input}: not a readable photo"))?;
@@ -520,18 +675,24 @@ fn render(args: &[String]) -> Result<(), String> {
for (k, v) in opts {
p[k] = v;
}
let mut o = ExportOptions::from_json(&p);
let mut o = ExportOptions::from_params(&p).map_err(|e| e.to_string())?;
if p.get("format").is_none() {
let ext = Path::new(&output).extension().map(|e| e.to_string_lossy().to_string()).unwrap_or_default();
o.format = ExportFormat::parse(&ext)
.ok_or_else(|| format!("{output}: unknown extension (use .jpg .png .tif .webp .avif .dng or --opt format=…)"))?;
}
let e = export_photo(&mut s, lightcraft_engine::catalog::PhotoId(id), &o, 1)?;
std::fs::write(&output, &e.bytes).map_err(|err| format!("{output}: {err}"))?;
for (ext, bytes) in &e.sidecars {
let sc = Path::new(&output).with_extension(ext);
std::fs::write(&sc, bytes).map_err(|err| format!("{}: {err}", sc.display()))?;
eprintln!("lightcraft-cli: wrote {}", sc.display());
// never over the input (or its sidecar), however it is spelled: `render IMG.jpg -o IMG.jpg`
let sidecars: Vec<(String, &Vec<u8>)> =
e.sidecars.iter().map(|(ext, bytes)| (Path::new(&output).with_extension(ext).to_string_lossy().to_string(), bytes)).collect();
let guard = s.original_guard();
for p in std::iter::once(&output).chain(sidecars.iter().map(|(p, _)| p)) {
guard.check(Path::new(p)).map_err(|err| format!("render: {err}"))?;
}
lightcraft_engine::export::write_file(&output, &e.bytes)?;
for (sc, bytes) in &sidecars {
lightcraft_engine::export::write_file(sc, bytes)?;
eprintln!("lightcraft-cli: wrote {sc}");
}
eprintln!("lightcraft-cli: wrote {output} ({}×{})", e.width, e.height);
Ok(())
@@ -567,19 +728,19 @@ fn snapshot(args: &[String]) -> Result<(), String> {
if script.is_none() && output.is_none() {
return Err("snapshot: give -o OUT.png and/or --script FILE".into());
}
if !(scale > 0.0 && size[0] >= 1.0 && size[1] >= 1.0) {
return Err("snapshot: bad --size/--scale".into());
}
lightcraft_ui_egui::headless::viewport_pixels(size, scale).map_err(|e| format!("snapshot: bad --size/--scale: {e}"))?;
let t0 = Instant::now();
let mut session = match &library {
Some(dir) => {
let mut s = Session::new().with_fs();
s.open_library(dir, false).map_err(|e| format!("{dir}: {e}"))?;
let mut s = Session::new().with_fs().with_default_denoise_models().with_default_face_models();
s.open_library(dir, false).map_err(|e| library_error(dir, e))?;
s
}
None if files.is_empty() => Session::with_demo().with_fs(),
None => Session::new().with_fs(),
None if files.is_empty() => Session::with_demo().with_fs().with_default_denoise_models().with_default_face_models(),
None => Session::new().with_fs().with_default_denoise_models().with_default_face_models(),
};
// Apply compute policy before imports or the first scripted query can discover an adapter.
session.execute("app.gpu", &json!({"enabled": false})).map_err(|e| e.to_string())?;
if !files.is_empty() {
let ti = Instant::now();
let r = session.execute("library.import", &json!({"paths": expand_paths(&files)})).map_err(|e| e.to_string())?;
@@ -587,24 +748,18 @@ fn snapshot(args: &[String]) -> Result<(), String> {
eprintln!("lightcraft-cli snapshot: imported {n} files in {:.0} ms", ti.elapsed().as_secs_f64() * 1e3);
}
let services = lightcraft_ui_egui::Services {
write_shared: Some(std::sync::Arc::new(|p: &str, b: &[u8]| {
if let Some(dir) = Path::new(p).parent().filter(|d| !d.as_os_str().is_empty()) {
std::fs::create_dir_all(dir).map_err(|e| e.to_string())?;
}
std::fs::write(p, b).map_err(|e| format!("{p}: {e}"))
})),
write: Some(Box::new(|p: &str, b: &[u8]| {
if let Some(dir) = Path::new(p).parent().filter(|d| !d.as_os_str().is_empty()) {
std::fs::create_dir_all(dir).map_err(|e| e.to_string())?;
}
std::fs::write(p, b).map_err(|e| format!("{p}: {e}"))
})),
write_shared: Some(std::sync::Arc::new(lightcraft_engine::export::write_file)),
write: Some(Box::new(lightcraft_engine::export::write_file)),
png: Some(Box::new(|img: &lightcraft_raster::Rgba8| {
lightcraft_codecs::encode_png(&lightcraft_codecs::EncodeImage::rgba8(img), &lightcraft_codecs::EncodeMeta::default()).unwrap_or_default()
})),
..Default::default()
};
let app = lightcraft_ui_egui::LightcraftApp::new(session, services);
let mut app = lightcraft_ui_egui::LightcraftApp::new(session, services);
// Snapshot sessions promise no GPU: disable photo compute as well as the compositor
// before the first frame can start background adapter discovery.
// This UI state is session-local: the CLI neither loads nor saves desktop ui.json.
app.ui.settings.gpu = false;
let mut h = Headless::new(app, size, scale);
let timeout = Duration::from_secs(60);
let mut shots = 0usize;
@@ -620,6 +775,9 @@ fn snapshot(args: &[String]) -> Result<(), String> {
Some(p.with_file_name(format!("{stem}-{shots}.{ext}")).to_string_lossy().to_string())
};
let mut wrote_output = false;
// Issue #166: a scripted request that fails still gets its reply printed and the run carries
// on (like `run --keep-going`), but the exit status reports it — CI judges by exit status.
let mut failed = 0usize;
if let Some(script) = &script {
let text = std::fs::read_to_string(script).map_err(|e| format!("{script}: {e}"))?;
let mut out = std::io::stdout().lock();
@@ -652,6 +810,9 @@ fn snapshot(args: &[String]) -> Result<(), String> {
}
_ => h.request(&method, params, timeout),
};
if reply["ok"] != true {
failed += 1;
}
if let Some(o) = reply.as_object_mut() {
o.insert("id".into(), id);
// wall time of the request (incl. the frames it ran), and since the start
@@ -688,7 +849,7 @@ fn snapshot(args: &[String]) -> Result<(), String> {
}
}
eprintln!("lightcraft-cli snapshot: done in {:.2} s ({} frames)", t0.elapsed().as_secs_f64(), h.frames());
Ok(())
if failed > 0 { Err(format!("{failed} scripted request(s) failed")) } else { Ok(()) }
}
fn commands(args: &[String]) -> Result<(), String> {
+301 -3
View File
@@ -19,6 +19,19 @@ fn gradient_png(path: &std::path::Path) {
std::fs::write(path, png).unwrap();
}
#[test]
fn calibrate_reads_raf_inputs_and_reports_bad_files() {
let input = tmp("invalid.RAF");
std::fs::write(&input, b"FUJIFILMCCD-RAW").unwrap();
let out = tmp("raf-profiles");
let result = Command::new(BIN).args(["calibrate", "--out"]).arg(&out).arg(&input).output().unwrap();
assert!(!result.status.success());
let stderr = String::from_utf8_lossy(&result.stderr);
assert!(stderr.contains("1 of 1 raw files"), "RAF was silently ignored: {stderr}");
assert!(stderr.contains("invalid.RAF") && stderr.contains("no profile written"), "{stderr}");
assert!(!out.join("X-T4.json").exists(), "bad input must not produce a profile");
}
fn mean(path: &std::path::Path) -> f64 {
let d = lightcraft_codecs::decode(&std::fs::read(path).unwrap(), Default::default()).unwrap();
let img = d.to_srgb8();
@@ -86,6 +99,24 @@ fn render_subcommand() {
assert!(String::from_utf8_lossy(&o.stderr).contains("unknown control"));
}
/// Issue #93: `render IMG -o IMG` replaced its own input with the render.
#[test]
fn render_refuses_to_overwrite_its_input() {
let input = tmp("self-in.png");
gradient_png(&input);
let before = std::fs::read(&input).unwrap();
let dir = input.parent().unwrap();
std::fs::create_dir_all(dir.join("sub")).unwrap();
let other_spelling = dir.join("sub/../self-in.png");
for out in [&input, &other_spelling] {
let o =
Command::new(BIN).args(["render", input.to_str().unwrap(), "-o", out.to_str().unwrap(), "--set", "light.exposure=1"]).output().unwrap();
assert!(!o.status.success());
assert!(String::from_utf8_lossy(&o.stderr).contains("never writes over an original"), "{}", String::from_utf8_lossy(&o.stderr));
}
assert_eq!(std::fs::read(&input).unwrap(), before, "the input is untouched");
}
#[test]
fn render_export_options() {
let input = tmp("o-in.png");
@@ -136,10 +167,14 @@ fn snapshot_subcommand_renders_the_ui_headlessly() {
std::fs::write(
&script,
format!(
"# comment\n{}\n{}\n{}\n{}\n",
"# comment\n{}\n{}\n{}\n{}\n{}\n{}\n",
json!({"method": "ui.set", "params": {"view": "photoGrid"}}),
// (settled: under load a grid shot taken before its thumbnails is darker than the
// dimmed export dialog over them, and the comparison below fails)
json!({"method": "ui.settle", "params": {"timeoutMs": 20000}}),
json!({"method": "ui.screenshot"}),
json!({"method": "engine.execute", "params": {"command": "dialog.export"}}),
json!({"method": "ui.settle", "params": {"timeoutMs": 20000}}),
json!({"method": "ui.screenshot", "params": {"path": b.to_str().unwrap()}}),
),
)
@@ -150,7 +185,7 @@ fn snapshot_subcommand_renders_the_ui_headlessly() {
.unwrap();
assert!(o.status.success(), "{}", String::from_utf8_lossy(&o.stderr));
let replies: Vec<Value> = String::from_utf8_lossy(&o.stdout).lines().map(|l| serde_json::from_str(l).unwrap()).collect();
assert_eq!(replies.len(), 4);
assert_eq!(replies.len(), 6);
assert!(replies.iter().all(|r| r["ok"] == true), "{replies:?}");
for (p, dimmed) in [(&a, false), (&b, true)] {
let d = lightcraft_codecs::decode(&std::fs::read(p).unwrap(), Default::default()).unwrap();
@@ -165,6 +200,127 @@ fn snapshot_subcommand_renders_the_ui_headlessly() {
assert_eq!((d.width, d.height), (640, 480));
}
/// Issue #166: a scripted request that fails makes `snapshot` exit non-zero. The failed reply is
/// still printed, the later lines still run, and the final `-o` screenshot is still written.
#[test]
fn snapshot_script_failure_exits_non_zero() {
let script = tmp("snap-fail.jsonl");
let out = tmp("snap-fail.png");
std::fs::write(
&script,
format!(
"{}\n{}\n",
json!({"method": "ui.set", "params": {"view": "grid"}}), // not a view: `photoGrid`, `detail`, …
json!({"method": "ui.set", "params": {"view": "photoGrid"}}),
),
)
.unwrap();
let _ = std::fs::remove_file(&out);
let o = Command::new(BIN)
.args(["snapshot", "--demo", "--script", script.to_str().unwrap(), "-o", out.to_str().unwrap(), "--size", "320x200"])
.output()
.unwrap();
let stderr = String::from_utf8_lossy(&o.stderr);
assert!(!o.status.success(), "a failed scripted request must fail the run\n{stderr}");
assert!(stderr.contains("1 scripted request(s) failed"), "{stderr}");
let replies: Vec<Value> = String::from_utf8_lossy(&o.stdout).lines().map(|l| serde_json::from_str(l).unwrap()).collect();
assert_eq!(replies.len(), 2, "{replies:?}");
assert_eq!(replies[0]["ok"], false, "{replies:?}");
assert!(replies[0]["error"].as_str().unwrap_or("").contains("grid"), "{replies:?}");
assert_eq!(replies[1]["ok"], true, "{replies:?}");
// continue-and-report: the final screenshot is still written
let d = lightcraft_codecs::decode(&std::fs::read(&out).unwrap(), Default::default()).unwrap();
assert_eq!((d.width, d.height), (320, 200));
}
#[test]
fn snapshot_ui_zoom_keeps_requested_pixel_dimensions() {
let script = tmp("snap-ui-zoom.jsonl");
std::fs::write(
&script,
format!("{}\n{}\n", json!({"method": "ui.key", "params": {"key": "Plus", "cmd": true}}), json!({"method": "ui.inspect"})),
)
.unwrap();
for (size, scale, expected) in [("400x240", "1", (400, 240)), ("345x200", "0.9", (311, 180))] {
let out = tmp(&format!("snap-ui-zoom-{scale}.png"));
let o = Command::new(BIN)
.args(["snapshot", "--script", script.to_str().unwrap(), "-o", out.to_str().unwrap(), "--size", size, "--scale", scale])
.output()
.unwrap();
assert!(o.status.success(), "{}", String::from_utf8_lossy(&o.stderr));
let replies: Vec<Value> = String::from_utf8_lossy(&o.stdout).lines().map(|line| serde_json::from_str(line).unwrap()).collect();
assert!(replies.iter().all(|reply| reply["ok"] == true), "{replies:?}");
assert!(replies[1]["result"]["pixelsPerPoint"].as_f64().unwrap() > scale.parse::<f64>().unwrap());
let decoded = lightcraft_codecs::decode(&std::fs::read(out).unwrap(), Default::default()).unwrap();
assert_eq!((decoded.width, decoded.height), expected);
}
}
/// Headless snapshots default to CPU photo rendering as well as CPU UI rasterization.
#[test]
fn snapshot_defaults_to_cpu_without_gpu_environment_overrides() {
let script = tmp("default-cpu.jsonl");
std::fs::write(
&script,
format!("{}\n{}\n", json!({"method": "engine.execute", "params": {"command": "app.gpu"}}), json!({"method": "ui.screenshot"})),
)
.unwrap();
let image = tmp("default-cpu.png");
let output = Command::new(BIN)
.env_remove("LIGHTCRAFT_GPU")
.env_remove("LIGHTCRAFT_GPU_BACKEND")
.env_remove("WGPU_BACKEND")
.args(["snapshot", "--demo", "--script", script.to_str().unwrap(), "-o", image.to_str().unwrap(), "--size", "480x320"])
.output()
.unwrap();
assert!(output.status.success(), "status {}: {}", output.status, String::from_utf8_lossy(&output.stderr));
let replies: Vec<Value> = String::from_utf8_lossy(&output.stdout).lines().map(|line| serde_json::from_str(line).unwrap()).collect();
assert!(replies.iter().all(|reply| reply["ok"] == true), "{replies:?}");
let gpu = &replies[0]["result"];
assert_eq!(gpu["enabled"], false, "{gpu}");
assert_eq!(gpu["available"], false, "{gpu}");
assert_eq!(gpu["adapter"], Value::Null, "no device is created: {gpu}");
assert!(gpu["reason"].as_str().is_some_and(|reason| reason.contains("preference")), "{gpu}");
let decoded = lightcraft_codecs::decode(&std::fs::read(image).unwrap(), Default::default()).unwrap();
assert_eq!((decoded.width, decoded.height), (480, 320));
}
/// Issue #136: with the GPU switched off from the environment the UI starts and renders on the CPU,
/// without creating a GPU device (no driver is loaded), and says why.
#[test]
fn snapshot_starts_without_a_gpu() {
let script = tmp("nogpu.jsonl");
std::fs::write(
&script,
format!(
"{}\n{}\n{}\n",
json!({"method": "ui.set", "params": {"view": "photoGrid"}}),
json!({"method": "engine.execute", "params": {"command": "app.gpu"}}),
json!({"method": "ui.screenshot"}),
),
)
.unwrap();
for (var, value, reason) in [("LIGHTCRAFT_GPU", "0", "LIGHTCRAFT_GPU=0"), ("LIGHTCRAFT_GPU_BACKEND", "off", "LIGHTCRAFT_GPU_BACKEND=off")] {
let out = tmp(&format!("nogpu-{var}.png"));
let o = Command::new(BIN)
.env_remove("LIGHTCRAFT_GPU")
.env_remove("LIGHTCRAFT_GPU_BACKEND")
.env(var, value)
.args(["snapshot", "--demo", "--script", script.to_str().unwrap(), "-o", out.to_str().unwrap(), "--size", "480x320"])
.output()
.unwrap();
assert!(o.status.success(), "{var}: {}", String::from_utf8_lossy(&o.stderr));
let replies: Vec<Value> = String::from_utf8_lossy(&o.stdout).lines().map(|l| serde_json::from_str(l).unwrap()).collect();
assert!(replies.iter().all(|r| r["ok"] == true), "{replies:?}");
let gpu = &replies[1]["result"];
assert_eq!(gpu["available"], false, "{var}: {gpu}");
assert_eq!(gpu["adapter"], Value::Null, "{var}: no device was created: {gpu}");
assert!(gpu["reason"].as_str().is_some_and(|r| r.contains(reason)), "{var}: {gpu}");
let d = lightcraft_codecs::decode(&std::fs::read(&out).unwrap(), Default::default()).unwrap();
assert_eq!((d.width, d.height), (480, 320));
}
}
fn run_cli(args: &[&str], stdin: Option<&str>) -> (bool, Vec<Value>, String) {
let mut child = Command::new(BIN).arg("run").args(args).stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::piped()).spawn().unwrap();
if let Some(text) = stdin {
@@ -238,7 +394,149 @@ fn devices_are_listed_and_imported_from() {
assert_eq!(dev["name"], "CARD", "{line}");
// the import review of its DCIM folder finds the photo
let dcim = dev["path"].as_str().unwrap().to_string();
let o = Command::new(BIN).args(["run", "library.importPreview", &format!("paths=[\"{dcim}\"]")]).output().unwrap();
// JSON-escaped: a Windows path has backslashes
let paths = serde_json::to_string(&[&dcim]).unwrap();
let o = Command::new(BIN).args(["run", "library.importPreview", &format!("paths={paths}")]).output().unwrap();
let line: Value = serde_json::from_slice(o.stdout.split(|b| *b == b'\n').next().unwrap()).unwrap();
assert_eq!(line["result"]["candidates"].as_array().map(Vec::len), Some(1), "{line}");
}
/// Issue #99: a library open in one process (here `mcp --library`) is refused by a second one,
/// with who has it and how to drive the running app instead; free again once the first exits.
#[test]
fn a_library_open_in_another_process_is_refused() {
let lib = tmp("locked-lib");
let _ = std::fs::remove_dir_all(&lib);
let lib_s = lib.to_str().unwrap();
let mut holder =
Command::new(BIN).args(["mcp", "--library", lib_s]).stdin(Stdio::piped()).stdout(Stdio::null()).stderr(Stdio::piped()).spawn().unwrap();
let mut err = BufReader::new(holder.stderr.take().unwrap());
let mut line = String::new();
while !line.contains("opened library") {
line.clear();
assert!(err.read_line(&mut line).unwrap() > 0, "mcp exited before opening the library");
}
let (ok, _, stderr) = run_cli(&["--library", lib_s, "library.info"], None);
assert!(!ok);
assert!(stderr.contains("already open in lightcraft-cli") && stderr.contains(&format!("process {}", holder.id())), "{stderr}");
assert!(stderr.contains("mcp --connect"), "{stderr}");
drop(holder.stdin.take()); // EOF: the server exits and lets go of the library
assert!(holder.wait().unwrap().success());
let (ok, lines, stderr) = run_cli(&["--library", lib_s, "library.info"], None);
assert!(ok, "{stderr}");
assert_eq!(lines[0]["ok"], true);
let _ = std::fs::remove_dir_all(&lib);
}
/// Issue #167: when `--connect ADDR` cannot reach the app, the recovery hint must name the port
/// ADDR actually names — following the old fixed "7980" left the retry connecting elsewhere.
#[test]
fn connect_failure_hint_names_the_attempted_port() {
let (ok, _lines, err) = run_cli(&["--connect", "127.0.0.1:18437", "ui.inspect"], None);
assert!(!ok);
assert!(err.contains("--control 18437"), "the hint must name the port it tried: {err}");
assert!(!err.contains("--control 7980"), "{err}");
}
/// The CLI logs warnings on stderr (#168): with `RUST_LOG=warn` an unknown GPU backend name is
/// reported, and `RUST_LOG=off` silences it. Before, no logger was installed and nothing appeared.
#[test]
fn warnings_are_logged_on_stderr() {
let input = tmp("log-in.png");
gradient_png(&input);
let out = tmp("log-out.jpg");
let run = |level: &str| {
let o = Command::new(BIN)
.args(["render", input.to_str().unwrap(), "-o", out.to_str().unwrap()])
.env("LIGHTCRAFT_GPU_BACKEND", "bogus")
.env("RUST_LOG", level)
.env_remove("LIGHTCRAFT_LOG")
.env_remove("LIGHTCRAFT_GPU")
.output()
.unwrap();
let stderr = String::from_utf8_lossy(&o.stderr).into_owned();
assert!(o.status.success(), "{stderr}");
stderr
};
let warn = run("warn");
assert!(warn.contains("LIGHTCRAFT_GPU_BACKEND=bogus names no known backend"), "{warn}");
let off = run("off");
assert!(!off.contains("names no known backend"), "{off}");
}
#[test]
fn snapshot_rejects_invalid_dimensions_before_creating_an_image() {
for (index, (size, scale)) in
[("NaNx100", "1"), ("100x100", "inf"), ("9000x9000", "1"), ("9000x100", "2"), ("100x100", "0.001")].into_iter().enumerate()
{
let image = tmp(&format!("invalid-viewport-{index}.png"));
let output = Command::new(BIN).args(["snapshot", "--size", size, "--scale", scale, "-o"]).arg(&image).output().unwrap();
assert!(!output.status.success());
assert!(String::from_utf8_lossy(&output.stderr).contains("snapshot: bad --size/--scale"), "{}", String::from_utf8_lossy(&output.stderr));
assert!(!image.exists());
}
}
#[test]
fn snapshot_preserves_fractional_scale_rounding() {
let image = tmp("fractional-scale-rounding.png");
let output =
Command::new(BIN).env("LIGHTCRAFT_GPU", "0").args(["snapshot", "--size", "345x200", "--scale", "0.9", "-o"]).arg(&image).output().unwrap();
assert!(output.status.success(), "{}", String::from_utf8_lossy(&output.stderr));
let decoded = lightcraft_codecs::decode(&std::fs::read(image).unwrap(), Default::default()).unwrap();
assert_eq!((decoded.width, decoded.height), (311, 180));
}
/// Snapshot's CPU setting must never replace the desktop preference, including with --library.
#[cfg(target_os = "linux")]
#[test]
fn snapshot_leaves_desktop_gpu_preferences_untouched() {
let dir = tmp("snapshot-desktop-preferences");
let library = dir.join("library");
let mut session = lightcraft_engine::Session::new().with_fs();
session.open_library(&library, true).unwrap();
drop(session);
let config = dir.join("config");
let app_config = config.join("lightcraft");
std::fs::create_dir_all(&app_config).unwrap();
let ui_path = app_config.join("ui.json");
let mut ui = lightcraft_ui_egui::UiState::default();
ui.settings.gpu = true;
let saved = serde_json::to_vec_pretty(&ui).unwrap();
std::fs::write(&ui_path, &saved).unwrap();
let script = dir.join("preferences.jsonl");
std::fs::write(
&script,
format!("{}\n{}\n", json!({"method": "engine.execute", "params": {"command": "app.gpu"}}), json!({"method": "ui.inspect"})),
)
.unwrap();
for gpu_environment in [None, Some("1")] {
let image = dir.join(format!("{}.png", gpu_environment.unwrap_or("default")));
let mut cmd = Command::new(BIN);
cmd.env("XDG_CONFIG_HOME", &config).env_remove("LIGHTCRAFT_GPU").env_remove("LIGHTCRAFT_GPU_BACKEND").env_remove("WGPU_BACKEND");
if let Some(value) = gpu_environment {
cmd.env("LIGHTCRAFT_GPU", value);
}
let output = cmd
.args(["snapshot", "--library"])
.arg(&library)
.args(["--script"])
.arg(&script)
.args(["-o"])
.arg(&image)
.args(["--size", "480x320"])
.output()
.unwrap();
assert!(output.status.success(), "{}", String::from_utf8_lossy(&output.stderr));
let replies: Vec<Value> = String::from_utf8_lossy(&output.stdout).lines().map(|line| serde_json::from_str(line).unwrap()).collect();
assert!(replies.iter().all(|reply| reply["ok"] == true), "{replies:?}");
assert_eq!(replies[0]["result"]["enabled"], false);
assert_eq!(replies[0]["result"]["available"], false);
assert_eq!(replies[0]["result"]["adapter"], Value::Null);
assert_eq!(replies[1]["result"]["ui"]["settings"]["gpu"], false);
assert_eq!(std::fs::read(&ui_path).unwrap(), saved, "snapshot changed the desktop preferences");
assert!(!app_config.join("ui.json.tmp").exists());
assert!(image.exists());
}
}
+7
View File
@@ -21,6 +21,7 @@ lightcraft-preview = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
log = { workspace = true }
crc32fast = { workspace = true }
[target.'cfg(target_arch = "wasm32")'.dependencies]
eframe = { workspace = true, features = ["default_fonts", "glow"] }
@@ -70,3 +71,9 @@ web-sys = { workspace = true, features = [
"WorkerType",
"WritableStream",
] }
[features]
# Open HEIC/HEIF photos (the optional lightcraft-heif crate). Not default so distributors choose
# it (HEVC patents); build with `--features heif` to include it. On wasm32 panics abort, so
# heic-rs's `catch_unwind` guard does not apply there.
heif = ["lightcraft-codecs/heif"]
+10 -2
View File
@@ -24,13 +24,21 @@
<script type="module">
import init, { start, command } from "./lightcraft_web.js";
(async () => {
// The Chinese font is a separate file: embedding it would exceed the host's WASM size limit.
// Font-free development builds can still run, with the same missing-glyph warning as before.
const chineseFont = fetch("./lightcraft_zh_hans.otf")
.then(async response => response.ok ? new Uint8Array(await response.arrayBuffer()) : new Uint8Array())
.catch(() => new Uint8Array());
// Compile once; the render workers (worker.js) instantiate the same compiled module.
const module = await WebAssembly.compileStreaming(fetch("./lightcraft_web_bg.wasm"));
const [module, font] = await Promise.all([
WebAssembly.compileStreaming(fetch("./lightcraft_web_bg.wasm")),
chineseFont,
]);
globalThis.lightcraftModule = module;
await init({ module_or_path: module });
// automation: `await lightcraft.command("library.info", "{}")` → JSON text
globalThis.lightcraft = { command };
start();
start(font);
})().catch((e) => {
console.error(e);
const el = document.getElementById("lightcraft_loading");
+33 -10
View File
@@ -232,6 +232,18 @@ impl Backend {
}
}
/// Is there a file at `path`?
pub async fn exists(&self, path: &str) -> Result<bool, String> {
match self {
Backend::Opfs(root) => Ok(Self::opfs_file(root, path, false).await?.is_some()),
Backend::Idb(db) => {
let (_tx, store) = Self::idb_store(db, false)?;
let n = idb_req(&store.count_with_key(&path.into()).map_err(err)?).await?;
Ok(n.as_f64().is_some_and(|n| n > 0.0))
}
}
}
pub async fn remove(&self, path: &str) -> Result<(), String> {
match self {
Backend::Opfs(root) => {
@@ -309,14 +321,25 @@ async fn dir_keys(dir: &FileSystemDirectoryHandle) -> Result<Vec<String>, String
Ok(out)
}
/// Ask the browser not to evict our storage under pressure (best effort; fire and forget).
pub fn request_persistence() {
if let Some(sm) = storage_manager()
&& let Ok(p) = sm.persist()
{
wasm_bindgen_futures::spawn_local(async move {
let granted = JsFuture::from(p).await.ok().is_some_and(|v| v.is_truthy());
log::info!("lightcraft: persistent storage {}", if granted { "granted" } else { "not granted (best effort)" });
});
}
/// Ask the browser not to evict our storage under pressure; `on_result(granted)` once known
/// (`false` also when the browser has no storage manager).
pub fn request_persistence(on_result: impl FnOnce(bool) + 'static) {
let Some(sm) = storage_manager() else {
on_result(false);
return;
};
wasm_bindgen_futures::spawn_local(async move {
// already persistent: nothing to ask
let already = match sm.persisted() {
Ok(p) => JsFuture::from(p).await.ok().is_some_and(|v| v.is_truthy()),
Err(_) => false,
};
let granted = already
|| match sm.persist() {
Ok(p) => JsFuture::from(p).await.ok().is_some_and(|v| v.is_truthy()),
Err(_) => false,
};
log::info!("lightcraft: persistent storage {}", if granted { "granted" } else { "not granted" });
on_result(granted);
});
}
+338
View File
@@ -0,0 +1,338 @@
//! Library backups for the browser build: a plain zip (entries stored, not compressed — photos
//! don't shrink) of the library files and every stored original, built one entry at a time so
//! only one file is in wasm memory at once, and read back the same way for a restore.
//!
//! Layout of a backup:
//! - `library/catalog.snap`, `library/catalog.log`, `library/presets.json`, … (see
//! [`crate::files::LIBRARY_FILES`]);
//! - `originals/<content hash>/<file name>`: each imported photo's original bytes, under its own
//! name so an unzipped backup is browsable;
//! - `README.txt`.
//!
//! Restoring never deletes anything: the library files go to a new library folder in browser
//! storage, originals that aren't stored yet are added (they're named by content, so existing
//! ones are identical; a restored one must match its content hash, see [`verify_original`]),
//! and only then does a small pointer file ([`ACTIVE_LIBRARY`]) switch to the restored library.
//! The library that was there before stays in storage.
//!
//! No zip64: a backup holds at most [`MAX_BYTES`] and 65 535 entries.
/// Storage key of the file naming the library folder in use (absent: `library`).
pub const ACTIVE_LIBRARY: &str = "active-library";
/// The default library folder.
pub const DEFAULT_LIBRARY_DIR: &str = "library";
/// Largest backup (offsets are 32-bit without zip64).
pub const MAX_BYTES: u64 = 0xFFFF_FFFF - (64 << 20);
const LOCAL_SIG: u32 = 0x0403_4b50;
const CENTRAL_SIG: u32 = 0x0201_4b50;
const END_SIG: u32 = 0x0605_4b50;
/// General-purpose flag bit 11: names are UTF-8.
const UTF8: u16 = 0x0800;
/// 1980-01-01 00:00 in MS-DOS format (date, time).
const DOS_DATE: u16 = (1 << 5) | 1;
/// The readme inside every backup.
pub const README: &str = "LightCraft library backup (browser version)\n\n\
library/ the catalog (catalog.snap + catalog.log), presets, view state and preferences\n\
originals/ every imported photo, as originals/<content hash>/<file name>\n\n\
Restore it in LightCraft for the web with File > Restore Library from Backup...\n\
The photos in originals/ are the untouched files you imported; edits live in the catalog.\n";
/// Is `name` a safe library folder name (`library` or `library-<letters, digits, ->`)?
pub fn valid_library_dir(name: &str) -> bool {
name == DEFAULT_LIBRARY_DIR
|| name.strip_prefix("library-").is_some_and(|s| !s.is_empty() && s.len() <= 64 && s.chars().all(|c| c.is_ascii_alphanumeric() || c == '-'))
}
/// Is `h` a plausible content hash (used as a storage key segment)?
pub fn valid_hash(h: &str) -> bool {
!h.is_empty() && h.len() <= 128 && h.chars().all(|c| c.is_ascii_alphanumeric())
}
/// Zip entry name for an original.
pub fn original_entry(hash: &str, file_name: &str) -> String {
let name: String = file_name.chars().map(|c| if c == '/' || c == '\\' || c.is_control() { '_' } else { c }).collect();
let name = if name.is_empty() || name == "." || name == ".." { "original".to_string() } else { name };
format!("originals/{hash}/{name}")
}
/// Where a backup entry goes when restoring into library folder `dir`: `Some(storage key)`, or
/// `None` to skip it (README, thumbnails, anything unknown or unsafe).
pub fn restore_key(entry: &str, dir: &str) -> Option<String> {
if let Some(name) = entry.strip_prefix("library/") {
return crate::files::LIBRARY_FILES.contains(&name).then(|| format!("{dir}/{name}"));
}
let rest = entry.strip_prefix("originals/")?;
let (hash, _) = rest.split_once('/').unwrap_or((rest, ""));
valid_hash(hash).then(|| format!("originals/{hash}"))
}
fn u16le(v: &mut Vec<u8>, x: u16) {
v.extend_from_slice(&x.to_le_bytes());
}
fn u32le(v: &mut Vec<u8>, x: u32) {
v.extend_from_slice(&x.to_le_bytes());
}
struct Central {
name: String,
crc: u32,
size: u32,
offset: u32,
}
/// Writes a stored (uncompressed) zip piece by piece: [`ZipWriter::entry`] returns the local
/// header to emit before the entry's bytes, [`ZipWriter::finish`] the central directory.
#[derive(Default)]
pub struct ZipWriter {
entries: Vec<Central>,
offset: u64,
}
impl ZipWriter {
/// The local header for `name` holding `data` (emit it, then `data`).
pub fn entry(&mut self, name: &str, data: &[u8]) -> Result<Vec<u8>, String> {
let size = u32::try_from(data.len()).map_err(|_| format!("{name}: larger than 4 GB"))?;
let name_len = u16::try_from(name.len()).map_err(|_| format!("{name}: name too long"))?;
if self.entries.len() >= usize::from(u16::MAX) {
return Err("too many files for one backup".into());
}
let header_len = 30 + u64::from(name_len);
if self.offset + header_len + u64::from(size) > MAX_BYTES {
return Err("the library is too large for one backup file (4 GB)".into());
}
let offset = u32::try_from(self.offset).map_err(|_| "backup too large".to_string())?;
let crc = crc32fast::hash(data);
let mut h = Vec::with_capacity(header_len as usize);
u32le(&mut h, LOCAL_SIG);
u16le(&mut h, 20); // version needed
u16le(&mut h, UTF8);
u16le(&mut h, 0); // stored
u16le(&mut h, 0); // time
u16le(&mut h, DOS_DATE);
u32le(&mut h, crc);
u32le(&mut h, size);
u32le(&mut h, size);
u16le(&mut h, name_len);
u16le(&mut h, 0); // extra
h.extend_from_slice(name.as_bytes());
self.offset += header_len + u64::from(size);
self.entries.push(Central { name: name.to_string(), crc, size, offset });
Ok(h)
}
/// The central directory and end record (emit last).
pub fn finish(self) -> Result<Vec<u8>, String> {
let mut cd = Vec::new();
for e in &self.entries {
u32le(&mut cd, CENTRAL_SIG);
u16le(&mut cd, 20); // made by
u16le(&mut cd, 20); // needed
u16le(&mut cd, UTF8);
u16le(&mut cd, 0);
u16le(&mut cd, 0);
u16le(&mut cd, DOS_DATE);
u32le(&mut cd, e.crc);
u32le(&mut cd, e.size);
u32le(&mut cd, e.size);
u16le(&mut cd, e.name.len() as u16);
u16le(&mut cd, 0); // extra
u16le(&mut cd, 0); // comment
u16le(&mut cd, 0); // disk
u16le(&mut cd, 0); // internal attrs
u32le(&mut cd, 0); // external attrs
u32le(&mut cd, e.offset);
cd.extend_from_slice(e.name.as_bytes());
}
let cd_offset = u32::try_from(self.offset).map_err(|_| "backup too large".to_string())?;
let cd_size = u32::try_from(cd.len()).map_err(|_| "backup too large".to_string())?;
let n = self.entries.len() as u16;
let mut out = cd;
u32le(&mut out, END_SIG);
u16le(&mut out, 0);
u16le(&mut out, 0);
u16le(&mut out, n);
u16le(&mut out, n);
u32le(&mut out, cd_size);
u32le(&mut out, cd_offset);
u16le(&mut out, 0); // comment
Ok(out)
}
}
/// One entry of a zip's central directory.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ZipEntry {
pub name: String,
pub crc: u32,
pub size: u64,
/// Offset of the entry's local header.
pub header: u64,
/// 0 = stored (the only method a restore reads).
pub method: u16,
}
fn rd16(b: &[u8], at: usize) -> Option<u16> {
Some(u16::from_le_bytes(b.get(at..at + 2)?.try_into().ok()?))
}
fn rd32(b: &[u8], at: usize) -> Option<u32> {
Some(u32::from_le_bytes(b.get(at..at + 4)?.try_into().ok()?))
}
/// How many bytes from the end of a zip to read to find its end record.
pub fn tail_len(file_len: u64) -> u64 {
file_len.min(22 + 65_535)
}
/// The central directory's (offset, size) from the last [`tail_len`] bytes of a zip.
pub fn find_central(tail: &[u8]) -> Result<(u64, u64), String> {
let at = (0..tail.len().saturating_sub(21)).rev().find(|&i| rd32(tail, i) == Some(END_SIG)).ok_or("not a zip file (no end record)")?;
let size = rd32(tail, at + 12).ok_or("truncated zip")?;
let offset = rd32(tail, at + 16).ok_or("truncated zip")?;
if size == u32::MAX || offset == u32::MAX {
return Err("zip64 backups are not supported".into());
}
Ok((u64::from(offset), u64::from(size)))
}
/// Parse a central directory.
pub fn parse_central(cd: &[u8]) -> Result<Vec<ZipEntry>, String> {
let mut out = Vec::new();
let mut i = 0;
while i + 46 <= cd.len() {
if rd32(cd, i) != Some(CENTRAL_SIG) {
return Err("damaged zip directory".into());
}
let bad = || "damaged zip directory".to_string();
let method = rd16(cd, i + 10).ok_or_else(bad)?;
let crc = rd32(cd, i + 16).ok_or_else(bad)?;
let size = rd32(cd, i + 24).ok_or_else(bad)?;
let n = usize::from(rd16(cd, i + 28).ok_or_else(bad)?);
let extra = usize::from(rd16(cd, i + 30).ok_or_else(bad)?);
let comment = usize::from(rd16(cd, i + 32).ok_or_else(bad)?);
let header = rd32(cd, i + 42).ok_or_else(bad)?;
let name = cd.get(i + 46..i + 46 + n).ok_or_else(bad)?;
out.push(ZipEntry { name: String::from_utf8_lossy(name).into_owned(), crc, size: u64::from(size), header: u64::from(header), method });
i += 46 + n + extra + comment;
}
Ok(out)
}
/// Offset of an entry's data from its local header's first 30 bytes.
pub fn data_offset(entry: &ZipEntry, local: &[u8]) -> Result<u64, String> {
if rd32(local, 0) != Some(LOCAL_SIG) {
return Err(format!("{}: damaged entry", entry.name));
}
let n = rd16(local, 26).ok_or("truncated entry")?;
let extra = rd16(local, 28).ok_or("truncated entry")?;
Ok(entry.header + 30 + u64::from(n) + u64::from(extra))
}
/// Check an entry's bytes.
pub fn verify(entry: &ZipEntry, data: &[u8]) -> Result<(), String> {
if entry.method != 0 {
return Err(format!("{}: compressed entries are not supported (not a LightCraft backup?)", entry.name));
}
if data.len() as u64 != entry.size || crc32fast::hash(data) != entry.crc {
return Err(format!("{}: damaged (checksum mismatch)", entry.name));
}
Ok(())
}
/// Check that an original's bytes are the photo its storage key names: originals are stored
/// under their content hash, and a restore that kept bytes under another photo's hash would
/// give that photo's catalog record a different image (the zip checksum can't tell).
pub fn verify_original(entry: &ZipEntry, hash: &str, data: &[u8]) -> Result<(), String> {
if crate::store::content_hash(data) != hash {
return Err(format!("{}: the file doesn't match its content hash (damaged or altered backup)", entry.name));
}
Ok(())
}
/// A name for the restored library's folder.
pub fn restored_dir_name(now_ms: f64) -> String {
format!("library-restored-{}", now_ms.max(0.0) as u64)
}
#[cfg(test)]
mod tests {
use super::*;
fn build(files: &[(&str, &[u8])]) -> Vec<u8> {
let mut w = ZipWriter::default();
let mut out = Vec::new();
for (n, d) in files {
out.extend(w.entry(n, d).unwrap());
out.extend_from_slice(d);
}
out.extend(w.finish().unwrap());
out
}
#[test]
fn written_backups_read_back() {
let files: [(&str, &[u8]); 3] =
[("README.txt", README.as_bytes()), ("library/catalog.log", b"{\"seq\":1}\n"), ("originals/abc123/IMG 1.jpg", &[0, 1, 2, 3, 255])];
let zip = build(&files);
let tail = &zip[zip.len() - tail_len(zip.len() as u64) as usize..];
let (off, size) = find_central(tail).unwrap();
let entries = parse_central(&zip[off as usize..(off + size) as usize]).unwrap();
assert_eq!(entries.len(), 3);
for (e, (name, data)) in entries.iter().zip(files) {
assert_eq!(e.name, name);
let start = data_offset(e, &zip[e.header as usize..e.header as usize + 30]).unwrap() as usize;
let got = &zip[start..start + e.size as usize];
assert_eq!(got, data);
verify(e, got).unwrap();
}
// damage is caught
let e = &entries[2];
assert!(verify(e, &[0, 1, 2, 3, 254]).is_err());
assert!(find_central(b"not a zip at all, just some text").is_err());
}
#[test]
fn originals_must_match_their_content_hash() {
let red: &[u8] = b"red corner photo";
let blue: &[u8] = b"blue corner photo";
let hash = crate::store::content_hash(red);
let name = original_entry(&hash, "IMG 1.png");
let key = restore_key(&name, "d").unwrap();
let key_hash = key.strip_prefix("originals/").unwrap();
assert_eq!(key_hash, hash);
// the entry's own bytes pass; other bytes under the same name (valid zip CRC) don't
let zip = build(&[(name.as_str(), blue)]);
let tail = &zip[zip.len() - tail_len(zip.len() as u64) as usize..];
let (off, size) = find_central(tail).unwrap();
let entries = parse_central(&zip[off as usize..(off + size) as usize]).unwrap();
let entry = &entries[0];
verify(entry, blue).unwrap();
let err = verify_original(entry, key_hash, blue).unwrap_err();
assert!(err.contains(&name), "{err}");
verify_original(entry, key_hash, red).unwrap();
}
#[test]
fn restore_keys_are_safe() {
assert_eq!(restore_key("library/catalog.snap", "library-restored-1").as_deref(), Some("library-restored-1/catalog.snap"));
assert_eq!(restore_key("library/../../x", "library-r"), None);
assert_eq!(restore_key("library/evil.bin", "library-r"), None);
assert_eq!(restore_key("originals/abc123/IMG.jpg", "d").as_deref(), Some("originals/abc123"));
assert_eq!(restore_key("originals/../library/catalog.log", "d"), None);
assert_eq!(restore_key("README.txt", "d"), None);
assert_eq!(restore_key("thumbs/x.jpg", "d"), None);
assert_eq!(original_entry("abc", "a/b\\c.jpg"), "originals/abc/a_b_c.jpg");
assert_eq!(original_entry("abc", ".."), "originals/abc/original");
assert!(valid_library_dir("library") && valid_library_dir("library-restored-17"));
assert!(!valid_library_dir("library-../x") && !valid_library_dir("originals") && !valid_library_dir("library-"));
}
#[test]
fn size_limits_are_errors_not_panics() {
let mut w = ZipWriter { offset: MAX_BYTES - 10, ..Default::default() };
assert!(w.entry("x", &[0; 64]).is_err());
}
}
+58
View File
@@ -49,6 +49,8 @@ struct Mirror {
files: BTreeMap<String, Vec<u8>>,
/// Dirty files, least recently modified first.
dirty: Vec<(String, Dirty)>,
/// Why the last flush to browser storage failed (quota, eviction…), until one succeeds.
error: Option<String>,
}
impl Mirror {
@@ -138,6 +140,18 @@ impl Files {
}
}
/// Record the outcome of a flush to browser storage: `Some(why)` when it failed, `None` once
/// one succeeds. While a flush is failing, the catalog [`Store`] refuses new writes, so the
/// engine keeps the changes queued and reports them unsaved (`library.info` → `unsavedOps`,
/// the top bar's warning) instead of pretending they were saved.
pub fn set_error(&self, e: Option<String>) {
self.lock().error = e;
}
pub fn error(&self) -> Option<String> {
self.lock().error.clone()
}
/// A catalog [`Store`] over these files.
pub fn store(&self) -> FilesStore {
FilesStore(self.clone())
@@ -147,19 +161,32 @@ impl Files {
/// [`Store`] implementation over [`Files`].
pub struct FilesStore(Files);
impl FilesStore {
/// Writes fail while browser storage does (see [`Files::set_error`]).
fn writable(&self) -> io::Result<()> {
match self.0.error() {
Some(e) => Err(io::Error::other(format!("browser storage: {e}"))),
None => Ok(()),
}
}
}
impl Store for FilesStore {
fn read(&mut self, name: &str) -> io::Result<Option<Vec<u8>>> {
Ok(self.0.get(name))
}
fn write_atomic(&mut self, name: &str, data: &[u8]) -> io::Result<()> {
self.writable()?;
self.0.write(name, data);
Ok(())
}
fn append(&mut self, name: &str, data: &[u8]) -> io::Result<()> {
self.writable()?;
self.0.append(name, data);
Ok(())
}
fn truncate(&mut self, name: &str, len: u64) -> io::Result<()> {
self.writable()?;
self.0.truncate(name, len as usize);
Ok(())
}
@@ -224,6 +251,37 @@ mod tests {
assert_eq!(names, ["catalog.snap", "catalog.log"]);
}
/// A failing flush (quota exceeded, storage evicted) is not silent: commands report the change
/// as unsaved and the engine keeps it queued; once storage works again it is written.
#[test]
fn flush_failures_show_as_unsaved_and_are_retried() {
let mut disk = BTreeMap::new();
let files = Files::default();
let mut s = Session::new();
let stores = LibraryStores { dir: "browser".into(), catalog: Box::new(files.store()), files: Box::new(files.store()), on_disk: false };
s.open_library_in(stores, true).unwrap();
flush(&files, &mut disk);
files.set_error(Some("QuotaExceededError".into()));
let r = s.execute("photo.rate", &json!({"rating": 5}));
assert!(matches!(r, Err(lightcraft_engine::EngineError::NotSaved(_))), "{r:?}");
let (ops, why) = s.unsaved().unwrap();
assert!(ops > 0 && why.contains("QuotaExceeded"), "{why}");
files.set_error(None);
s.persist_if_dirty();
s.persist().unwrap();
assert!(s.unsaved().is_none(), "written once storage works again");
flush(&files, &mut disk);
let id = s.selection.active.unwrap();
let files2 = Files::default();
for (k, v) in &disk {
files2.preload(k, v.clone());
}
let mut s2 = Session::new();
let stores = LibraryStores { dir: "browser".into(), catalog: Box::new(files2.store()), files: Box::new(files2.store()), on_disk: false };
s2.open_library_in(stores, true).unwrap();
assert_eq!(s2.catalog.photo(id).unwrap().rating, 5, "the rating survived");
}
#[test]
fn library_survives_reload_through_flushes() {
let mut disk = BTreeMap::new();
+4
View File
@@ -14,7 +14,9 @@
//! console (see [`bench`]); `?store=idb` forces the IndexedDB backend, `?store=memory` disables
//! persistence and `?workers=0` renders on the main thread.
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
pub mod backup;
pub mod bench;
pub mod files;
pub mod store;
@@ -23,6 +25,8 @@ pub mod wire;
#[cfg(target_arch = "wasm32")]
mod backend;
#[cfg(target_arch = "wasm32")]
mod safety;
#[cfg(target_arch = "wasm32")]
mod web;
#[cfg(target_arch = "wasm32")]
mod workers;
+275
View File
@@ -0,0 +1,275 @@
//! Keeping a library in the browser safe: notices the user must see (a failed save, storage the
//! browser may evict, a file that couldn't be stored), one tab at a time per library, a message
//! instead of a dead page after a panic, and backup / restore of the whole library
//! ([`crate::backup`]).
use std::cell::RefCell;
use std::collections::HashMap;
use js_sys::{Reflect, Uint8Array};
use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
use wasm_bindgen_futures::JsFuture;
use crate::backend::Backend;
use crate::backup::{
ACTIVE_LIBRARY, README, ZipWriter, data_offset, find_central, original_entry, parse_central, restore_key, restored_dir_name, tail_len,
valid_hash, verify, verify_original,
};
use crate::files::{Files, LIBRARY_FILES};
use crate::store::storage_key;
thread_local! {
/// Messages for the user, shown by the app on its next frame.
static NOTICES: RefCell<Vec<String>> = const { RefCell::new(Vec::new()) };
}
/// Queue a message for the user (shown for several seconds; also logged).
pub fn notice(text: impl Into<String>) {
let text = text.into();
log::warn!("{text}");
NOTICES.with(|n| n.borrow_mut().push(text));
crate::web::request_repaint();
}
/// Messages queued since the last call.
pub fn take_notices() -> Vec<String> {
NOTICES.with(|n| std::mem::take(&mut *n.borrow_mut()))
}
fn document() -> Option<web_sys::Document> {
web_sys::window()?.document()
}
/// Ask the user (the browser's own confirm box). `false` when it can't be shown.
pub fn confirm(text: &str) -> bool {
web_sys::window().and_then(|w| w.confirm_with_message(text).ok()).unwrap_or(false)
}
/// Cover the page with a message (the app can't continue, or must not start).
pub fn show_blocking(text: &str) {
let Some(doc) = document() else { return };
let el = match doc.get_element_by_id("lightcraft_blocking") {
Some(el) => el,
None => {
let Ok(el) = doc.create_element("div") else { return };
el.set_id("lightcraft_blocking");
let _ = el.set_attribute(
"style",
"position:fixed;inset:0;z-index:10;display:flex;align-items:center;justify-content:center;padding:32px;\
background:#1c1c1c;color:#e0e0e0;font:15px/1.5 system-ui,-apple-system,'Segoe UI',sans-serif;text-align:center;white-space:pre-line",
);
if let Some(body) = doc.body() {
let _ = body.append_child(&el);
}
el
}
};
el.set_text_content(Some(text));
if let Some(loading) = doc.get_element_by_id("lightcraft_loading") {
loading.remove();
}
}
/// `panic = "abort"` on wasm: a panic ends the module. Say so on the page instead of leaving a
/// frozen canvas, and say that the stored library is intact.
pub fn install_panic_hook() {
std::panic::set_hook(Box::new(|info| {
let msg = match (info.payload().downcast_ref::<&str>(), info.payload().downcast_ref::<String>()) {
(Some(s), _) => (*s).to_string(),
(_, Some(s)) => s.clone(),
_ => "unknown error".into(),
};
let at = info.location().map(|l| format!(" at {}:{}", l.file(), l.line())).unwrap_or_default();
log::error!("LightCraft panicked{at}: {msg}");
show_blocking(&format!(
"LightCraft stopped after an internal error:\n{msg}\n\nThe library saved in this browser is kept. Reload the page to continue.\n\
If it happens again right away, a photo may be the cause: reload with ?workers=0 or report the error."
));
}));
}
/// The name of the lock that keeps a library to one tab.
const TAB_LOCK: &str = "lightcraft-library";
/// Hold the library lock for as long as this page lives (Web Locks API). `Some(true)`: ours;
/// `Some(false)`: another tab of this site has it; `None`: the browser can't tell (no Web Locks, or
/// it refused the request).
pub async fn acquire_tab_lock() -> Option<bool> {
let nav = Reflect::get(&js_sys::global(), &"navigator".into()).ok()?;
let locks = Reflect::get(&nav, &"locks".into()).ok().filter(|l| !l.is_undefined() && !l.is_null())?;
let request: js_sys::Function = Reflect::get(&locks, &"request".into()).ok()?.dyn_into().ok()?;
let opts = js_sys::Object::new();
Reflect::set(&opts, &"ifAvailable".into(), &true.into()).ok()?;
let granted = js_sys::Promise::new(&mut |resolve, reject| {
let cb = Closure::once_into_js(move |lock: JsValue| -> JsValue {
let ours = !lock.is_null() && !lock.is_undefined();
let _ = resolve.call1(&JsValue::NULL, &ours.into());
// keep the lock until the page goes away: a promise that never settles
if ours { js_sys::Promise::new(&mut |_, _| {}).into() } else { JsValue::NULL }
});
// a refused request never calls `cb`: settle with its error instead of waiting forever
match request.call3(&locks, &TAB_LOCK.into(), &opts, &cb) {
Ok(pending) => {
let catch = Reflect::get(&pending, &"catch".into()).ok().and_then(|c| c.dyn_into::<js_sys::Function>().ok());
if let Some(catch) = catch {
let _ = catch.call1(&pending, &reject);
}
}
Err(e) => {
let _ = reject.call1(&JsValue::NULL, &e);
}
}
});
match JsFuture::from(granted).await {
Ok(v) => Some(v.is_truthy()),
Err(e) => {
log::warn!("Web Locks request failed, starting without the one-tab guard: {e:?}");
None
}
}
}
/// Offer a blob as a download named `name`.
pub fn download_blob(name: &str, blob: &web_sys::Blob) -> Result<(), String> {
let e = |e: JsValue| format!("download failed: {e:?}");
let url = web_sys::Url::create_object_url_with_blob(blob).map_err(e)?;
let doc = document().ok_or("no document")?;
let a: web_sys::HtmlAnchorElement = doc.create_element("a").map_err(e)?.unchecked_into();
a.set_href(&url);
a.set_download(name);
a.click();
// revoke once the download has started
let revoke = Closure::once_into_js(move || {
let _ = web_sys::Url::revoke_object_url(&url);
});
if let Some(w) = web_sys::window() {
let _ = w.set_timeout_with_callback_and_timeout_and_arguments_0(revoke.unchecked_ref(), 60_000);
}
Ok(())
}
/// File ▸ Back Up Library…: a zip of the library files (as the app has them, saved or not) and
/// every original in storage, built entry by entry into the browser's memory (outside the wasm
/// heap) and downloaded. `names`: content hash → the photo's file name.
pub async fn backup(files: Files, backend: Backend, names: HashMap<String, String>) -> Result<(usize, f64), String> {
let parts = js_sys::Array::new();
let mut zip = ZipWriter::default();
let mut bytes = 0f64;
let mut add = |zip: &mut ZipWriter, name: &str, data: &[u8]| -> Result<(), String> {
let header = zip.entry(name, data)?;
parts.push(&Uint8Array::from(&header[..]));
parts.push(&Uint8Array::from(data));
bytes += (header.len() + data.len()) as f64;
Ok(())
};
add(&mut zip, "README.txt", README.as_bytes())?;
for name in LIBRARY_FILES {
if let Some(d) = files.get(name) {
add(&mut zip, &format!("library/{name}"), &d)?;
}
}
let mut originals = 0;
for hash in backend.list("originals").await? {
if !valid_hash(&hash) {
continue;
}
match backend.read(&storage_key(&hash)).await {
Ok(Some(data)) => {
let name = names.get(&hash).map_or("original", String::as_str);
add(&mut zip, &original_entry(&hash, name), &data)?;
originals += 1;
}
Ok(None) => {}
Err(e) => return Err(format!("reading an original from browser storage: {e}")),
}
}
let tail = zip.finish()?;
parts.push(&Uint8Array::from(&tail[..]));
let opts = web_sys::BlobPropertyBag::new();
opts.set_type("application/zip");
let blob = web_sys::Blob::new_with_u8_array_sequence_and_options(&parts, &opts).map_err(|e| format!("{e:?}"))?;
let day: String = String::from(js_sys::Date::new_0().to_iso_string()).chars().take(10).collect();
download_blob(&format!("lightcraft-library-backup-{day}.zip"), &blob)?;
Ok((originals, bytes + tail.len() as f64))
}
/// Bytes `start..end` of a picked file.
async fn read_range(file: &web_sys::File, start: u64, end: u64) -> Result<Vec<u8>, String> {
let blob = file.slice_with_f64_and_f64(start as f64, end as f64).map_err(|e| format!("{e:?}"))?;
let buf = JsFuture::from(blob.array_buffer()).await.map_err(|e| format!("{e:?}"))?;
Ok(Uint8Array::new(&buf).to_vec())
}
/// Restore a backup into a new library folder (nothing in storage is deleted or overwritten),
/// then point [`ACTIVE_LIBRARY`] at it. Returns the folder's name; the caller reloads the page.
pub async fn restore(file: web_sys::File, backend: Backend) -> Result<String, String> {
let len = file.size() as u64;
let tail = read_range(&file, len - tail_len(len), len).await?;
let (cd_offset, cd_size) = find_central(&tail)?;
let cd = read_range(&file, cd_offset, cd_offset + cd_size).await?;
let entries = parse_central(&cd)?;
if !entries.iter().any(|e| e.name == "library/catalog.snap" || e.name == "library/catalog.log") {
return Err("this isn't a LightCraft library backup (no library/catalog files)".into());
}
let dir = restored_dir_name(js_sys::Date::now());
let mut written = 0usize;
for e in &entries {
let Some(key) = restore_key(&e.name, &dir) else { continue };
// originals are named by their content: one already stored is the same file
if key.starts_with("originals/") && backend.exists(&key).await? {
continue;
}
let local = read_range(&file, e.header, e.header + 30).await?;
let start = data_offset(e, &local)?;
let data = read_range(&file, start, start + e.size).await?;
verify(e, &data)?;
if let Some(hash) = key.strip_prefix("originals/") {
verify_original(e, hash, &data)?;
}
backend.write(&key, &data).await.map_err(|err| format!("storing {}: {err}", e.name))?;
written += 1;
}
backend.write(ACTIVE_LIBRARY, dir.as_bytes()).await?;
log::info!("lightcraft: restored {written} files into {dir}");
Ok(dir)
}
/// File ▸ Restore Library from Backup…: confirm, pick the zip, restore, reload.
pub fn pick_and_restore(backend: Backend, before_reload: impl FnOnce() + 'static) {
let ok = confirm(
"Restore a LightCraft library backup?\n\nThe backup becomes the library in this browser and the page reloads. \
The current library is not deleted: it stays in browser storage.",
);
if !ok {
return;
}
let Some(doc) = document() else { return };
let Ok(input) = doc.create_element("input").map(|e| e.unchecked_into::<web_sys::HtmlInputElement>()) else { return };
input.set_type("file");
input.set_accept(".zip,application/zip");
let inp = input.clone();
let mut before_reload = Some(before_reload);
let on_change = Closure::<dyn FnMut()>::new(move || {
let Some(file) = inp.files().and_then(|f| f.get(0)) else { return };
let backend = backend.clone();
let before = before_reload.take();
notice(format!("Restoring {}…", file.name()));
wasm_bindgen_futures::spawn_local(async move {
match restore(file, backend).await {
Ok(_) => {
if let Some(f) = before {
f();
}
if let Some(w) = web_sys::window() {
let _ = w.location().reload();
}
}
Err(e) => notice(format!("Restore failed: {e}. Nothing was changed: the current library is still in use.")),
}
});
});
input.set_onchange(Some(on_change.as_ref().unchecked_ref()));
on_change.forget();
input.click();
}
+205 -41
View File
@@ -15,37 +15,53 @@ use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
use crate::backend::Backend;
use crate::backup::{ACTIVE_LIBRARY, DEFAULT_LIBRARY_DIR, valid_library_dir};
use crate::bench::Bench;
use crate::files::{Files, FlushOp, LIBRARY_FILES};
use crate::safety::{self, notice};
use crate::store::{Originals, content_hash, download_name, storage_key};
use crate::wire::ThumbIndex;
use crate::workers::{THUMB_INDEX, Workers};
const ACCEPT: &str = ".jpg,.jpeg,.png,.tif,.tiff,.webp,.dng,.cr2,.nef,.arw,.psd,.jxl,.gif,.bmp";
/// Where the library files live in browser storage.
const LIBRARY_DIR: &str = "library";
/// After a failed save, wait this long before trying again (ms).
const SAVE_RETRY_MS: f64 = 2000.0;
/// How long a notice stays on screen (s).
const NOTICE_SECS: f64 = 8.0;
/// How often view state, UI prefs and the thumbnail index are saved (ms).
const SAVE_EVERY_MS: f64 = 1000.0;
fn window() -> web_sys::Window {
web_sys::window().expect("no window")
/// The page's window (`None` only outside a browser page, e.g. in a worker).
fn window() -> Option<web_sys::Window> {
web_sys::window()
}
/// Milliseconds since navigation start.
fn perf_now() -> f64 {
window().performance().map(|p| p.now()).unwrap_or(0.0)
window().and_then(|w| w.performance()).map(|p| p.now()).unwrap_or(0.0)
}
fn set_status(text: &str) {
if let Some(el) = window().document().and_then(|d| d.get_element_by_id("lightcraft_status")) {
if let Some(el) = window().and_then(|w| w.document()).and_then(|d| d.get_element_by_id("lightcraft_status")) {
el.set_text_content(Some(text));
}
}
fn library_key(name: &str) -> String {
format!("{LIBRARY_DIR}/{name}")
/// Storage key of library file `name` in library folder `dir`.
fn library_key(dir: &str, name: &str) -> String {
format!("{dir}/{name}")
}
/// Repaint the app (notices from async tasks).
pub(crate) fn request_repaint() {
CTX.with(|c| {
if let Some(c) = c.borrow().as_ref() {
c.request_repaint();
}
});
}
/// URL options (`?bench&store=idb&workers=2&reset`).
@@ -59,7 +75,7 @@ struct Options {
impl Options {
fn from_url() -> Options {
let q = window().location().search().unwrap_or_default();
let q = window().and_then(|w| w.location().search().ok()).unwrap_or_default();
let params: Vec<(String, String)> = q
.trim_start_matches('?')
.split('&')
@@ -85,8 +101,9 @@ async fn store_file(originals: Originals, backend: Option<Backend>, name: String
if let Some(b) = &backend
&& let Err(e) = b.write(&storage_key(&hash), &bytes).await
{
// still importable for this session; it won't survive a reload
log::error!("storing {name} in browser storage failed: {e}");
// a photo whose bytes aren't stored would be gone after a reload: don't add it
notice(format!("{name} was not added: browser storage refused it ({e}). Free up space or back up and clear old photos."));
return;
}
originals.added(&name, &hash, Arc::from(bytes));
ctx.request_repaint();
@@ -102,7 +119,7 @@ async fn read_file(originals: Originals, backend: Option<Backend>, file: web_sys
/// Show the browser's open dialog; picked files are imported asynchronously.
fn open_picker(originals: Originals, backend: Option<Backend>, ctx: egui::Context) {
let Some(doc) = window().document() else { return };
let Some(doc) = window().and_then(|w| w.document()) else { return };
let Ok(input) = doc.create_element("input").map(|e| e.unchecked_into::<web_sys::HtmlInputElement>()) else { return };
input.set_type("file");
input.set_multiple(true);
@@ -131,7 +148,7 @@ fn download(path: &str, bytes: &[u8]) -> Result<(), String> {
opts.set_type(crate::store::mime_for(name));
let blob = web_sys::Blob::new_with_u8_array_sequence_and_options(&arr, &opts).map_err(e)?;
let url = web_sys::Url::create_object_url_with_blob(&blob).map_err(e)?;
let doc = window().document().ok_or("no document")?;
let doc = window().and_then(|w| w.document()).ok_or("no document")?;
let a: web_sys::HtmlAnchorElement = doc.create_element("a").map_err(e)?.unchecked_into();
a.set_href(&url);
a.set_download(name);
@@ -140,20 +157,64 @@ fn download(path: &str, bytes: &[u8]) -> Result<(), String> {
let revoke = Closure::once_into_js(move || {
let _ = web_sys::Url::revoke_object_url(&url);
});
let _ = window().set_timeout_with_callback_and_timeout_and_arguments_0(revoke.unchecked_ref(), 5000);
if let Some(w) = window() {
let _ = w.set_timeout_with_callback_and_timeout_and_arguments_0(revoke.unchecked_ref(), 5000);
}
log::info!("exported {name} ({} bytes)", bytes.len());
Ok(())
}
fn services(originals: Originals, backend: Option<Backend>, ctx: egui::Context) -> Services {
fn services(originals: Originals, backend: Option<Backend>, files: Files, frozen: Rc<Cell<bool>>, ctx: egui::Context) -> Services {
let (backup_backend, backup_files) = (backend.clone(), files);
let restore_backend = backend.clone();
Services {
picker: None,
backup_library: Some(Box::new(move |session: &mut Session| {
let Some(b) = backup_backend.clone() else { return Err("nothing is stored in this browser session (?store=memory)".into()) };
// the photos' own names for the originals in the zip
let names: std::collections::HashMap<String, String> = session
.catalog
.photos()
.filter_map(|p| match &p.source {
Source::File { path } => crate::store::hash_of_path(path).map(|h| (h.to_string(), p.file_name.clone())),
_ => None,
})
.collect();
let files = backup_files.clone();
notice("Backing up the library… (the download starts when it's ready)");
wasm_bindgen_futures::spawn_local(async move {
match safety::backup(files, b, names).await {
Ok((n, bytes)) => notice(format!(
"Library backed up: {n} original{} ({:.1} MB). Keep the file somewhere safe.",
if n == 1 { "" } else { "s" },
bytes / 1e6
)),
Err(e) => notice(format!("Backup failed: {e}")),
}
});
Ok(json!({"started": true}))
})),
restore_library: Some(Box::new(move |_session: &mut Session| {
let Some(b) = restore_backend.clone() else { return Err("nothing is stored in this browser session (?store=memory)".into()) };
let frozen = frozen.clone();
// once the backup is in place, stop saving the old library (the page reloads)
safety::pick_and_restore(b, move || frozen.set(true));
Ok(json!({"started": true}))
})),
pick_files: Some(Box::new(move || {
open_picker(originals.clone(), backend.clone(), ctx.clone());
Vec::new() // files arrive asynchronously and are imported on a later frame
})),
pick_denoise_model: None,
// Preset files: browser pickers are asynchronous; not wired on the web yet.
pick_preset_files: None,
pick_tracklog: None,
pick_lightroom_catalog: None,
// Face models need a folder to live in; the web has none.
pick_model_file: None,
save_preset_file: None,
pick_curve_preset_files: None,
save_curve_preset_file: None,
write: Some(Box::new(download)),
// downloads happen on the main thread: exports run in the foreground on the web
write_shared: None,
@@ -161,6 +222,7 @@ fn services(originals: Originals, backend: Option<Backend>, ctx: egui::Context)
lightcraft_codecs::encode_png(&lightcraft_codecs::EncodeImage::rgba8(img), &lightcraft_codecs::EncodeMeta::default()).unwrap_or_default()
})),
reveal: None,
log_file: None,
open_with: None,
open_url: Some(Box::new(|url: &str| {
let w = web_sys::window().ok_or("no window")?;
@@ -170,21 +232,29 @@ fn services(originals: Originals, backend: Option<Backend>, ctx: egui::Context)
}
}
/// Write dirty library files to storage, in order, until none are left.
async fn flush(files: Files, backend: Backend, flushing: Rc<Cell<bool>>) {
/// Write dirty library files to storage, in order, until none are left. A failure (quota,
/// storage cleared) is recorded in `files` — the catalog then reports changes as unsaved instead
/// of accepting them silently — and retried after [`SAVE_RETRY_MS`].
async fn flush(files: Files, backend: Backend, dir: String, flushing: Rc<Cell<bool>>, failed_at: Rc<Cell<f64>>) {
'outer: loop {
let ops = files.take_dirty();
if ops.is_empty() {
if files.error().is_some() {
log::info!("lightcraft: saving works again");
}
files.set_error(None);
break;
}
for (i, op) in ops.iter().enumerate() {
let r = match op {
FlushOp::Write { name, data } => backend.write(&library_key(name), data).await,
FlushOp::Append { name, offset, data } => backend.write_at(&library_key(name), Some(*offset), data).await,
FlushOp::Write { name, data } => backend.write(&library_key(&dir, name), data).await,
FlushOp::Append { name, offset, data } => backend.write_at(&library_key(&dir, name), Some(*offset), data).await,
};
if let Err(e) = r {
log::error!("saving {} failed: {e}", op.name());
files.failed(&ops[i..]);
files.set_error(Some(e));
failed_at.set(perf_now());
break 'outer;
}
}
@@ -198,6 +268,8 @@ struct Boot {
backend: Option<Backend>,
files: Files,
index: ThumbIndex,
/// The library folder in storage (`library`, or a restored backup's).
lib_dir: String,
}
async fn boot(opts: Options) -> Boot {
@@ -215,15 +287,32 @@ async fn boot(opts: Options) -> Boot {
};
let files = Files::default();
let mut index = ThumbIndex::default();
let mut lib_dir = DEFAULT_LIBRARY_DIR.to_string();
if let Some(b) = &backend {
if opts.reset {
match b.clear().await {
Ok(()) => log::info!("lightcraft: storage cleared (?reset)"),
Err(e) => log::error!("clearing storage: {e}"),
// everything this site keeps in the browser, photos included: never without asking
let ok = safety::confirm(
"?reset deletes the LightCraft library stored in this browser, including every imported photo. \
This can't be undone. Delete it?",
);
if ok {
match b.clear().await {
Ok(()) => log::info!("lightcraft: storage cleared (?reset)"),
Err(e) => log::error!("clearing storage: {e}"),
}
} else {
log::info!("lightcraft: ?reset declined; storage kept");
}
}
// a restored backup lives in its own folder (see `backup`)
if let Ok(Some(d)) = b.read(ACTIVE_LIBRARY).await {
let d = String::from_utf8_lossy(&d).trim().to_string();
if valid_library_dir(&d) {
lib_dir = d;
}
}
for name in LIBRARY_FILES {
match b.read(&library_key(name)).await {
match b.read(&library_key(&lib_dir, name)).await {
Ok(Some(data)) => files.preload(name, data),
Ok(None) => {}
Err(e) => log::error!("reading {name}: {e}"),
@@ -232,7 +321,14 @@ async fn boot(opts: Options) -> Boot {
if let Ok(Some(j)) = b.read(THUMB_INDEX).await {
index = ThumbIndex::from_json(&j);
}
crate::backend::request_persistence();
crate::backend::request_persistence(|granted| {
if !granted {
notice(
"This browser may delete LightCraft's library when it runs short of space (persistent storage wasn't granted). \
Keep your original photos elsewhere and use File ▸ Back Up Library… now and then.",
);
}
});
// thumbnails stored without an index entry (index saved late, or lost): delete them
let b2 = b.clone();
let known: std::collections::HashSet<String> = b
@@ -260,7 +356,7 @@ async fn boot(opts: Options) -> Boot {
index.len(),
index.total() as f64 / 1e6
);
Boot { opts, backend, files, index }
Boot { opts, backend, files, index, lib_dir }
}
struct WebApp {
@@ -270,6 +366,11 @@ struct WebApp {
backend: Option<Backend>,
workers: Option<Workers>,
flushing: Rc<Cell<bool>>,
/// When the last save to storage failed (ms; 0 = never).
flush_failed_at: Rc<Cell<f64>>,
/// Saving stopped (a restored backup is about to replace this library: the page reloads).
frozen: Rc<Cell<bool>>,
lib_dir: String,
bench: Option<Bench>,
first_frame_logged: bool,
last_save: f64,
@@ -277,18 +378,19 @@ struct WebApp {
}
impl WebApp {
fn new(cc: &eframe::CreationContext<'_>, boot: Boot) -> Self {
let Boot { opts, backend, files, index } = boot;
fn new(cc: &eframe::CreationContext<'_>, boot: Boot, chinese_font: Vec<u8>) -> Self {
let Boot { opts, backend, files, index, lib_dir } = boot;
let originals = Originals::default();
let t = perf_now();
let mut session = Session::new();
originals.install(&mut session);
let stores = LibraryStores {
dir: format!("browser:{}/{LIBRARY_DIR}", backend.as_ref().map_or("memory", |b| b.kind())).into(),
dir: format!("browser:{}/{lib_dir}", backend.as_ref().map_or("memory", |b| b.kind())).into(),
catalog: Box::new(files.store()),
files: Box::new(files.store()),
on_disk: false,
};
let mut problem = None;
match session.open_library_in(stores, true).cloned() {
Ok(r) => log::info!(
"lightcraft: library {} in {:.0} ms ({} photos; snapshot seq {}, {} ops replayed)",
@@ -299,25 +401,52 @@ impl WebApp {
r.replayed
),
Err(e) => {
log::error!("opening the library failed: {e}; starting a temporary one");
session = Session::with_demo();
// never a silent demo session (issue #100): an empty one, and the window says why
log::error!("opening the library failed: {e}");
session = Session::new();
originals.install(&mut session);
problem = Some(lightcraft_ui_egui::panels::library_problem::LibraryProblem {
can_retry: false,
..lightcraft_ui_egui::panels::library_problem::LibraryProblem::new(
"the browser's storage for this page",
format!("{e}. Reload the page to try again."),
)
});
}
}
if backend.is_some() {
notice(
"LightCraft in the browser is experimental. Your library is kept in this browser's storage: keep your original photos \
elsewhere and back up with File ▸ Back Up Library….",
);
}
// previews are large (≤ 2560 px, f32): keep few in a 32-bit address space
session.media.preview_capacity = 3;
let mut app = LightcraftApp::new(session, services(originals.clone(), backend.clone(), cc.egui_ctx.clone()));
let frozen = Rc::new(Cell::new(false));
let mut app = LightcraftApp::new(session, services(originals.clone(), backend.clone(), files.clone(), frozen.clone(), cc.egui_ctx.clone()))
.with_chinese_font(chinese_font);
let ui_written = files.get("ui.json").unwrap_or_default();
if let Ok(ui) = serde_json::from_slice::<UiState>(&ui_written) {
app.ui = ui;
}
app.ui = app.ui.sanitized();
app.library_problem = problem;
let n = opts.workers.unwrap_or_else(|| {
let cores = window().navigator().hardware_concurrency() as usize;
let cores = window().map_or(1, |w| w.navigator().hardware_concurrency() as usize);
cores.saturating_sub(1).clamp(1, 4)
});
let workers =
(n > 0).then(|| Workers::start(n, backend.as_ref().map_or("memory", |b| b.kind()), backend.clone(), index, cc.egui_ctx.clone()));
let cache = app.session.media.rendered.clone();
let workers = (n > 0).then(|| {
Workers::start(
n,
backend.as_ref().map_or("memory", |b| b.kind()),
backend.clone(),
index,
&cache,
app.session.cache_bytes(),
cc.egui_ctx.clone(),
)
});
if let Some(w) = &workers {
app.renderer.set_offload(Box::new(w.clone()));
}
@@ -331,6 +460,9 @@ impl WebApp {
backend,
workers,
flushing: Rc::new(Cell::new(false)),
flush_failed_at: Rc::new(Cell::new(0.0)),
frozen,
lib_dir,
bench: opts.bench.then(|| Bench::new(origin)),
first_frame_logged: false,
last_save: 0.0,
@@ -392,6 +524,9 @@ impl WebApp {
/// Save view state, UI prefs and the thumbnail index now and then; flush dirty files.
fn save(&mut self) {
if self.frozen.get() {
return;
}
let now = perf_now();
if now - self.last_save >= SAVE_EVERY_MS {
self.last_save = now;
@@ -413,12 +548,16 @@ impl WebApp {
});
}
}
// after a failure, retry every couple of seconds rather than every frame
let backing_off = self.files.error().is_some() && now - self.flush_failed_at.get() < SAVE_RETRY_MS;
if let Some(b) = &self.backend
&& self.files.is_dirty()
&& !self.flushing.get()
&& !backing_off
{
self.flushing.set(true);
wasm_bindgen_futures::spawn_local(flush(self.files.clone(), b.clone(), self.flushing.clone()));
let (files, dir, flushing, failed_at) = (self.files.clone(), self.lib_dir.clone(), self.flushing.clone(), self.flush_failed_at.clone());
wasm_bindgen_futures::spawn_local(flush(files, b.clone(), dir, flushing, failed_at));
}
}
}
@@ -448,8 +587,10 @@ impl WebApp {
let (orig_n, orig_bytes) = self.originals.usage();
json!({
"storage": self.backend.as_ref().map_or("memory", |b| b.kind()),
"library": self.lib_dir,
"dirty": self.files.is_dirty(),
"flushing": self.flushing.get(),
"saveError": self.files.error(),
"workers": {"alive": alive, "ready": ready, "remoteJobs": remote, "inlineJobs": inline},
"originalsInMemory": orig_n,
"originalBytesInMemory": orig_bytes,
@@ -474,9 +615,18 @@ impl WebApp {
impl eframe::App for WebApp {
fn logic(&mut self, ctx: &egui::Context, _frame: &mut eframe::Frame) {
for text in safety::take_notices() {
let now = ctx.input(|i| i.time);
self.app.ui.toast = Some((text, now + NOTICE_SECS, None));
}
self.run_inbox();
self.import_dropped(ctx);
self.load_originals(ctx);
if let Some(w) = &self.workers {
// `library.preferences ▸ cacheMb` is read by the session; the worker index prunes on
// its own thread, so it gets the budget every frame and applies it on the next store.
w.set_budget(self.app.session.cache_bytes());
}
self.app.logic(ctx);
self.save();
if let Some(b) = self.bench.as_mut()
@@ -507,25 +657,39 @@ impl eframe::App for WebApp {
}
}
/// Start the app (called by `index.html` after instantiating the module).
/// Start the app with the separately fetched Simplified Chinese font (empty in font-free builds).
#[wasm_bindgen]
pub fn start() {
pub fn start(chinese_font: Vec<u8>) {
eframe::WebLogger::init(log::LevelFilter::Info).ok();
safety::install_panic_hook();
log::info!("lightcraft: wasm instantiated at {:.0} ms", perf_now());
if chinese_font.is_empty() {
log::warn!("lightcraft: no Simplified Chinese font supplied; Chinese-only glyphs will show as boxes");
}
let opts = Options::from_url();
wasm_bindgen_futures::spawn_local(async move {
let Some(canvas) =
window().document().and_then(|d| d.get_element_by_id("lightcraft_canvas")).and_then(|e| e.dyn_into::<web_sys::HtmlCanvasElement>().ok())
// one tab per library: two would each keep their own copy and overwrite each other's saves
if opts.store != "memory" && safety::acquire_tab_lock().await == Some(false) {
safety::show_blocking(
"LightCraft is already open in another tab or window of this browser.\n\n\
Switch to that tab, or close it and reload this page. (Two tabs would overwrite each other's changes.)",
);
return;
}
let Some(canvas) = window()
.and_then(|w| w.document())
.and_then(|d| d.get_element_by_id("lightcraft_canvas"))
.and_then(|e| e.dyn_into::<web_sys::HtmlCanvasElement>().ok())
else {
log::error!("missing <canvas id=\"lightcraft_canvas\">");
return;
};
let boot = boot(opts).await;
let runner = eframe::WebRunner::new();
let r = runner.start(canvas, eframe::WebOptions::default(), Box::new(move |cc| Ok(Box::new(WebApp::new(cc, boot))))).await;
let r = runner.start(canvas, eframe::WebOptions::default(), Box::new(move |cc| Ok(Box::new(WebApp::new(cc, boot, chinese_font))))).await;
match r {
Ok(()) => {
if let Some(el) = window().document().and_then(|d| d.get_element_by_id("lightcraft_loading")) {
if let Some(el) = window().and_then(|w| w.document()).and_then(|d| d.get_element_by_id("lightcraft_loading")) {
el.remove();
}
}
+202 -14
View File
@@ -11,14 +11,13 @@
//! ([`ThumbIndex`]).
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::{Arc, Weak};
use lightcraft_engine::catalog::Source;
use lightcraft_engine::develop::{DevelopSettings, EmbeddedLens};
use lightcraft_engine::media::{MediaCache, RenderJob, SourceLevel, SourceRef};
use lightcraft_engine::media::{DecodedSource, MediaCache, RenderJob, SourceLevel, SourceRef};
use lightcraft_engine::pipeline::{Quality, RenderRequest, Rendered, SourceInfo, StageCache};
use lightcraft_preview::Lru;
use lightcraft_raster::Rgb32f;
use lightcraft_preview::{Hash128, Lru, PreviewCache};
use serde::{Deserialize, Serialize};
use crate::store::hash_of_path;
@@ -26,11 +25,17 @@ use crate::store::hash_of_path;
/// One render request for a worker.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct WireJob {
#[serde(default)]
pub cache_generation: u64,
#[serde(default)]
pub source_identity: Option<String>,
pub level: SourceLevel,
pub origin: Source,
/// Decode the source at most this long.
pub max_edge: usize,
pub raw: bool,
#[serde(default)]
pub relative_wb: bool,
pub as_shot_temp: f64,
pub as_shot_tint: f64,
pub lens: Option<EmbeddedLens>,
@@ -48,6 +53,9 @@ pub struct WireJob {
/// The request's diagnostic overlay (`Overlay::to_parts`).
#[serde(default)]
pub overlay: (u8, f64),
/// Render only this window `[x, y, w, h]` of the output `max_w × max_h` describes (a zoomed view).
#[serde(default)]
pub window: Option<[usize; 4]>,
}
impl WireJob {
@@ -55,14 +63,17 @@ impl WireJob {
pub fn from_job(job: &RenderJob, thumb_cached: bool, view: &str) -> WireJob {
let max_edge = match &job.source {
SourceRef::Demo { max_edge, .. } | SourceRef::File { max_edge, .. } => *max_edge,
SourceRef::Loaded(img) => img.width.max(img.height),
SourceRef::Loaded(img) => img.image.width.max(img.image.height),
SourceRef::Smart { .. } => 2560,
};
WireJob {
cache_generation: job.cache_generation,
source_identity: job.source_key.map(|key| key.to_string()),
level: job.level,
origin: job.origin.clone(),
max_edge,
raw: job.info.raw,
relative_wb: job.info.relative_wb,
as_shot_temp: job.info.as_shot_temp,
as_shot_tint: job.info.as_shot_tint,
lens: job.info.lens,
@@ -75,11 +86,19 @@ impl WireJob {
thumb_cached,
stages: job.stages.is_some().then(|| view.to_string()),
overlay: job.request.overlay.to_parts(),
window: job.request.window.map(|w| [w.x, w.y, w.w, w.h]),
}
}
pub fn info(&self) -> SourceInfo {
SourceInfo { raw: self.raw, as_shot_temp: self.as_shot_temp, as_shot_tint: self.as_shot_tint, lens: self.lens }
SourceInfo {
raw: self.raw,
as_shot_temp: self.as_shot_temp,
as_shot_tint: self.as_shot_tint,
lens: self.lens,
relative_wb: self.relative_wb,
..Default::default()
}
}
pub fn request(&self) -> RenderRequest {
@@ -93,14 +112,15 @@ impl WireJob {
space: lightcraft_engine::pipeline::OutputSpace::Srgb,
depth: lightcraft_engine::pipeline::OutputDepth::U8,
proof: None,
window: self.window.map(|[x, y, w, h]| lightcraft_engine::pipeline::PixelWindow { x, y, w, h }),
}
}
/// Identifies the decoded source this job needs.
fn source_key(&self) -> String {
match &self.origin {
Source::File { path } => format!("{}:{path}", self.max_edge),
Source::Demo { scene } => format!("{}:demo:{scene}", self.max_edge),
Source::File { path } => format!("{}:{:?}:{}:{path}", self.cache_generation, self.source_identity, self.max_edge),
Source::Demo { scene } => format!("{}:{:?}:{}:demo:{scene}", self.cache_generation, self.source_identity, self.max_edge),
}
}
@@ -119,7 +139,7 @@ const WORKER_SOURCE_BYTES: usize = 192 << 20;
/// The synchronous part of a worker: decoded-source cache + rendering.
pub struct WorkerCore {
sources: Lru<String, Arc<Rgb32f>>,
sources: Lru<String, DecodedSource>,
media: MediaCache,
stages: HashMap<String, Arc<StageCache>>,
}
@@ -150,21 +170,25 @@ impl WorkerCore {
Some(s) => s,
None => {
let src = match (&job.origin, original) {
(Source::File { .. }, Some(bytes)) => Arc::new(lightcraft_engine::files::load_bytes(bytes, job.max_edge)?.0),
(Source::File { .. }, Some(bytes)) => {
let (image, info) = lightcraft_engine::files::load_bytes(bytes, job.max_edge)?;
DecodedSource::new(Arc::new(image), Some(info))
}
(Source::File { path }, None) => return Err(format!("{path}: original not found in browser storage")),
(origin @ Source::Demo { .. }, _) => self.media.origin_ref(origin, job.max_edge).load()?,
(origin @ Source::Demo { .. }, _) => self.media.origin_ref(origin, job.max_edge).load_source()?,
};
let cost = src.width * src.height * 12 + 64;
let cost = src.image.width * src.image.height * 12 + 64;
self.sources.insert(key, src.clone(), cost);
src
}
};
let info = src.info_or(job.info());
Ok(match &job.stages {
Some(view) => {
let st = self.stages.entry(view.clone()).or_default().clone();
lightcraft_engine::pipeline::render_cached(&src, &job.info(), &job.settings, &job.request(), &st)
lightcraft_engine::pipeline::render_cached(&src.image, &info, &job.settings, &job.request(), &st)
}
None => lightcraft_engine::pipeline::render(&src, &job.info(), &job.settings, &job.request()),
None => lightcraft_engine::pipeline::render(&src.image, &info, &job.settings, &job.request()),
})
}
}
@@ -173,6 +197,8 @@ impl WorkerCore {
/// pruned like the native disk cache (least recently used first, down to 80 % of the budget).
#[derive(Default, Serialize, Deserialize)]
pub struct ThumbIndex {
#[serde(default)]
namespace: u64,
/// hex key → (bytes, last use).
entries: HashMap<String, (u64, u64)>,
clock: u64,
@@ -183,6 +209,24 @@ pub struct ThumbIndex {
}
impl ThumbIndex {
pub fn namespace(&self) -> u64 {
self.namespace
}
/// A persistent new namespace prevents late old workers restoring an invalidated disk hit.
pub fn cache_key(&self, key: Hash128) -> String {
if self.namespace == 0 {
return key.to_string();
}
lightcraft_preview::Hasher128::new().str(&key.to_string()).str("cleared").u64(self.namespace).finish().to_string()
}
pub fn invalidate(&mut self) -> Vec<String> {
self.namespace = self.namespace.wrapping_add(1);
self.total = 0;
self.dirty = true;
self.entries.drain().map(|(key, _)| key).collect()
}
pub fn from_json(bytes: &[u8]) -> ThumbIndex {
let mut i: ThumbIndex = serde_json::from_slice(bytes).unwrap_or_default();
i.total = i.entries.values().map(|e| e.0).sum();
@@ -260,11 +304,89 @@ impl ThumbIndex {
}
}
/// Which rendered-thumbnail cache, at which generation, the stored thumbnails ([`ThumbIndex`])
/// belong to (main thread). Clearing the previews bumps the cache's generation, and opening
/// another library replaces the cache: either makes every stored thumbnail obsolete.
pub struct CacheWatch(Option<(Weak<PreviewCache>, u64)>);
impl CacheWatch {
/// Watch `active`, the session's cache when the workers start: the persisted index belongs
/// to it at its current generation. (Without this baseline, the first observation after a
/// clear would only record the cleared generation and keep the obsolete thumbnails.)
pub fn new(active: &Arc<PreviewCache>) -> CacheWatch {
CacheWatch(Some((Arc::downgrade(active), active.generation())))
}
/// The watched cache, while it's alive.
pub fn cache(&self) -> Option<Arc<PreviewCache>> {
self.0.as_ref().and_then(|(cache, _)| cache.upgrade())
}
/// `cache` at `generation` is the active one now. If it was cleared or replaced since the
/// last observation, invalidate `index` and return the keys whose files should be deleted.
pub fn observe(&mut self, cache: &Arc<PreviewCache>, generation: u64, index: &mut ThumbIndex) -> Vec<String> {
let changed = self.0.as_ref().is_some_and(|(old, g)| old.as_ptr() != Arc::as_ptr(cache) || *g != generation);
self.0 = Some((Arc::downgrade(cache), generation));
if changed { index.invalidate() } else { Vec::new() }
}
}
#[cfg(test)]
mod tests {
use super::*;
use lightcraft_engine::Session;
#[test]
fn clearing_index_changes_disk_namespace_and_survives_restart() {
let key = lightcraft_preview::hash_bytes(b"thumbnail");
let mut index = ThumbIndex::default();
let old = index.cache_key(key);
assert_eq!(old, key.to_string());
index.insert(&old, 100);
assert_eq!(index.invalidate(), vec![old.clone()]);
let new = index.cache_key(key);
assert_ne!(old, new);
index.insert(&new, 100);
let reopened = ThumbIndex::from_json(&index.to_json());
assert_eq!(reopened.cache_key(key), new);
assert!(!reopened.contains(&old));
assert!(reopened.contains(&new));
}
// Issue #323: a worker given a zoomed view's window renders that window, not the whole frame
#[test]
fn a_window_job_crosses_the_wire_and_renders_the_same_window() {
use lightcraft_engine::pipeline::PixelWindow;
let mut s = Session::with_demo();
let id = s.visible_cloned()[0];
let win = PixelWindow { x: 300, y: 200, w: 160, h: 120 };
let job = s.region_job(id, 1920, 1280, win, true).unwrap();
let wire = WireJob::from_job(&job, false, "Region");
let wire: WireJob = serde_json::from_str(&serde_json::to_string(&wire).unwrap()).unwrap();
assert_eq!(wire.request().window, Some(win));
let mut core = WorkerCore::default();
let a = core.render(&wire, None).unwrap();
let b = job.run().rendered.unwrap();
assert_eq!((a.image.width, a.image.height), (160, 120));
assert_eq!((b.image.width, b.image.height), (160, 120));
// the worker renders on the CPU; the engine job uses a GPU when the machine has one, which
// stays within docs/gpu-pipeline.md's bound (≤ 3 LSB per channel) rather than bit-exact
let max = a.image.data.iter().zip(&b.image.data).flat_map(|(p, q)| (0..3).map(move |c| p[c].abs_diff(q[c]))).max();
assert!(max.is_some_and(|m| m <= 3), "max channel difference {max:?}");
}
// a job from before windows existed has none
#[test]
fn a_job_without_a_window_field_has_no_window() {
let mut s = Session::with_demo();
let id = s.visible_cloned()[0];
let wire = WireJob::from_job(&s.thumb_job(id, 128).unwrap(), false, "Thumb");
let mut v = serde_json::to_value(&wire).unwrap();
v.as_object_mut().unwrap().remove("window");
let old: WireJob = serde_json::from_value(v).unwrap();
assert_eq!(old.request().window, None);
}
#[test]
fn wire_job_renders_like_the_engine() {
let mut s = Session::with_demo();
@@ -291,10 +413,13 @@ mod tests {
let hash = crate::store::content_hash(&bytes);
let origin = Source::File { path: crate::store::original_path(&hash, "a.png") };
let job = WireJob {
cache_generation: 0,
source_identity: None,
level: SourceLevel::Thumb,
origin,
max_edge: 512,
raw: false,
relative_wb: false,
as_shot_temp: 6500.0,
as_shot_tint: 0.0,
lens: None,
@@ -307,6 +432,7 @@ mod tests {
thumb_cached: false,
stages: Some("Main".into()),
overlay: (0, 0.0),
window: None,
};
let mut core = WorkerCore::default();
assert_eq!(core.needs_original(&job).as_deref(), Some(hash.as_str()));
@@ -314,6 +440,68 @@ mod tests {
let r = core.render(&job, Some(&bytes)).unwrap();
assert_eq!(r.image.width, 32);
assert_eq!(core.needs_original(&job), None, "decoded source is kept");
let mut reloaded = job.clone();
reloaded.source_identity = Some("changed content".into());
assert_eq!(core.needs_original(&reloaded).as_deref(), Some(hash.as_str()), "content reload must not reuse an old decoded source");
let mut refreshed = job.clone();
refreshed.cache_generation = 1;
assert_eq!(core.needs_original(&refreshed).as_deref(), Some(hash.as_str()), "explicit refresh must not reuse the old decoded source");
}
/// The persisted index as a restarted page loads it: one stored thumbnail.
fn persisted_index(key: Hash128) -> (ThumbIndex, String) {
let mut index = ThumbIndex::default();
let hex = index.cache_key(key);
index.insert(&hex, 100);
(ThumbIndex::from_json(&index.to_json()), hex)
}
#[test]
fn clearing_previews_before_the_first_request_invalidates_stored_thumbnails() {
let key = lightcraft_preview::hash_bytes(b"thumbnail");
let (mut index, old) = persisted_index(key);
let cache = Arc::new(PreviewCache::memory(1 << 20));
let mut watch = CacheWatch::new(&cache);
let mut gone = Vec::new();
// each frame (`Workers::finished`) observes the watched cache, even with no requests
let mut frame = |watch: &mut CacheWatch, index: &mut ThumbIndex| {
if let Some(c) = watch.cache() {
gone.extend(watch.observe(&c, c.generation(), index));
}
};
frame(&mut watch, &mut index); // an empty filtered view: no render request yet
cache.clear(); // File ▸ Clear Preview Cache
frame(&mut watch, &mut index);
// the filter is removed: the first thumbnail request (`Workers::try_start`)
gone.extend(watch.observe(&cache, cache.generation(), &mut index));
let disk_key = index.cache_key(key);
assert!(!index.touch(&disk_key), "the pre-clear thumbnail must not be read back");
assert_eq!(gone, vec![old.clone()], "the stored file is deleted");
assert_ne!(disk_key, old, "a fresh namespace: a late old write can't come back");
assert!(index.dirty, "the invalidated index is saved");
}
#[test]
fn replacing_the_cache_before_the_first_request_invalidates_stored_thumbnails() {
let key = lightcraft_preview::hash_bytes(b"thumbnail");
let (mut index, old) = persisted_index(key);
let first = Arc::new(PreviewCache::memory(1 << 20));
let mut watch = CacheWatch::new(&first);
let other = Arc::new(PreviewCache::memory(1 << 20)); // another library, generation 0 again
assert_eq!(watch.observe(&other, other.generation(), &mut index), vec![old]);
}
#[test]
fn a_normal_start_keeps_stored_thumbnails() {
let key = lightcraft_preview::hash_bytes(b"thumbnail");
let (mut index, old) = persisted_index(key);
let cache = Arc::new(PreviewCache::memory(1 << 20));
let mut watch = CacheWatch::new(&cache);
for _ in 0..3 {
assert!(watch.observe(&cache, cache.generation(), &mut index).is_empty());
}
assert_eq!(index.cache_key(key), old);
assert!(index.touch(&old), "persisted thumbnails survive a restart");
}
#[test]
+116 -12
View File
@@ -18,7 +18,7 @@ use std::sync::Arc;
use js_sys::{Array, Object, Reflect, Uint8Array, Uint32Array};
use lightcraft_engine::catalog::PhotoId;
use lightcraft_engine::media::{DISK_CACHE_BYTES, RenderJob, RenderResult, SourceLevel};
use lightcraft_engine::media::{RenderJob, RenderResult, SourceLevel};
use lightcraft_engine::pipeline::Rendered;
use lightcraft_preview::{Hash128, PreviewCache};
use lightcraft_raster::{Histogram, Rgba8};
@@ -27,7 +27,7 @@ use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
use crate::backend::Backend;
use crate::wire::{ThumbIndex, WireJob, WorkerCore, thumb_storage_key};
use crate::wire::{CacheWatch, ThumbIndex, WireJob, WorkerCore, thumb_storage_key};
/// Storage key of the thumbnail index.
pub const THUMB_INDEX: &str = "thumbs/index.json";
@@ -155,6 +155,10 @@ async fn handle(scope: &web_sys::DedicatedWorkerGlobalScope, backend: Option<Bac
// main side
struct Busy {
disk_key: Option<String>,
namespace: u64,
request_id: u64,
cache_generation: u64,
id: u32,
slot: Slot,
photo: PhotoId,
@@ -171,12 +175,17 @@ struct W {
}
struct Inner {
/// The rendered-thumbnail cache the stored thumbnails belong to.
preview_generation: CacheWatch,
workers: Vec<W>,
done: Vec<(Slot, RenderResult, f64)>,
next_id: u32,
/// (photo, level) → worker that last decoded it.
affinity: HashMap<(PhotoId, SourceLevel), usize>,
index: ThumbIndex,
/// The session's thumbnail cache budget in bytes. Kept here because the prune happens on a
/// worker message, where the session is out of reach; the app refreshes it every frame.
budget: u64,
backend: Option<Backend>,
ctx: egui::Context,
/// Jobs finished by workers / inline (for diagnostics).
@@ -185,19 +194,46 @@ struct Inner {
inline_this_frame: bool,
}
impl Inner {
fn observe_cache(&mut self, cache: &Arc<PreviewCache>, generation: u64) {
let gone = self.preview_generation.observe(cache, generation, &mut self.index);
if !gone.is_empty()
&& let Some(backend) = self.backend.clone()
{
wasm_bindgen_futures::spawn_local(async move {
for key in gone {
let _ = backend.remove(&thumb_storage_key(&key)).await;
}
});
}
}
}
/// The main thread's handle on the render workers.
#[derive(Clone)]
pub struct Workers(Rc<RefCell<Inner>>);
impl Workers {
/// Start `n` workers (none: every job runs inline). `store` is the backend kind for workers.
pub fn start(n: usize, store: &str, backend: Option<Backend>, index: ThumbIndex, ctx: egui::Context) -> Workers {
/// `index` lists the stored thumbnails of `cache`, the session's rendered-thumbnail cache
/// now: watching it from the start, a clear before the first render request still counts.
pub fn start(
n: usize,
store: &str,
backend: Option<Backend>,
index: ThumbIndex,
cache: &Arc<PreviewCache>,
budget: u64,
ctx: egui::Context,
) -> Workers {
let w = Workers(Rc::new(RefCell::new(Inner {
preview_generation: CacheWatch::new(cache),
workers: Vec::new(),
done: Vec::new(),
next_id: 0,
affinity: HashMap::new(),
index,
budget,
backend,
ctx,
remote_done: 0,
@@ -248,7 +284,16 @@ impl Workers {
w.dead = true;
w.worker.terminate();
if let Some(b) = w.busy.take() {
let r = RenderResult { photo: b.photo, level: b.level, key: b.key, rendered: Err(msg), loaded: None, quick: None };
let r = RenderResult {
request_id: b.request_id,
source_key: None,
photo: b.photo,
level: b.level,
key: b.key,
rendered: Err(msg),
loaded: None,
quick: None,
};
g.done.push((b.slot, r, 0.0));
}
}
@@ -284,16 +329,21 @@ impl Workers {
};
if let Ok(r) = &rendered
&& let Some((cache, key)) = &b.cache
&& cache.generation() == b.cache_generation
&& b.namespace == g.index.namespace()
{
cache.put(*key, Arc::new(r.image.clone()));
let hex = key.to_string();
cache.put_at(b.cache_generation, *key, Arc::new(r.image.clone()));
let hex = b.disk_key.clone().unwrap_or_else(|| key.to_string());
let stored = get(&data, "stored").as_f64().unwrap_or(0.0) as u64;
if get(&data, "cacheMiss").is_truthy() {
g.index.remove(&hex);
}
if stored > 0 {
g.index.insert(&hex, stored);
let gone = g.index.prune(DISK_CACHE_BYTES);
// The session's own budget (`library.preferences ▸ cacheMb`), not the native
// default: the browser accepted the setting, so it has to be the one enforced
// (#245).
let gone = g.index.prune(g.budget);
if !gone.is_empty()
&& let Some(be) = g.backend.clone()
{
@@ -307,8 +357,31 @@ impl Workers {
}
}
g.affinity.insert((b.photo, b.level), i);
// A cleared worker may finish writing after the initial deletion. Its old namespace
// is never reused, so removing this file cannot delete a replacement's cache entry.
if let Some((cache, _)) = &b.cache
&& (cache.generation() != b.cache_generation || b.namespace != g.index.namespace())
&& let (Some(backend), Some(key)) = (g.backend.clone(), b.disk_key.clone())
{
wasm_bindgen_futures::spawn_local(async move {
let _ = backend.remove(&thumb_storage_key(&key)).await;
});
}
g.remote_done += 1;
g.done.push((b.slot, RenderResult { photo: b.photo, level: b.level, key: b.key, rendered, loaded: None, quick: None }, ms));
g.done.push((
b.slot,
RenderResult {
request_id: b.request_id,
source_key: None,
photo: b.photo,
level: b.level,
key: b.key,
rendered,
loaded: None,
quick: None,
},
ms,
));
}
/// The thumbnail index as JSON if it changed since the last call (the host saves it).
@@ -328,19 +401,30 @@ impl Workers {
let ready = g.workers.iter().filter(|w| w.ready && !w.dead).count();
(alive, ready, g.remote_done, g.inline_done)
}
/// The budget the thumbnail index is pruned to. The app refreshes it from the session every
/// frame, so `library.preferences ▸ cacheMb` applies to the next stored thumbnail.
pub fn set_budget(&self, bytes: u64) {
self.0.borrow_mut().budget = bytes;
}
}
impl RenderOffload for Workers {
fn try_start(&mut self, slot: Slot, job: RenderJob) -> Option<RenderJob> {
let mut g = self.0.borrow_mut();
let g = &mut *g;
if let Some((cache, _)) = job.cache.as_ref().or(job.view_cache.as_ref()) {
g.observe_cache(cache, job.cache_generation);
}
// rendered thumbnail in memory: no worker needed
if let Some((cache, key)) = &job.cache
&& let Some(img) = cache.get(*key)
&& let Some(img) = cache.get_at(job.cache_generation, *key)
{
let image = Arc::unwrap_or_clone(img);
let histogram = Histogram::of_srgb8(&image);
let r = RenderResult {
request_id: job.request_id,
source_key: job.source_key,
photo: job.photo,
level: job.level,
key: job.key,
@@ -374,25 +458,45 @@ impl RenderOffload for Workers {
let Some(i) = pick else { return Some(job) };
// decided here, so the next job for this photo follows even before this one finishes
g.affinity.insert((job.photo, job.level), i);
let thumb_cached = job.cache.as_ref().is_some_and(|(_, k)| g.index.touch(&k.to_string()));
let wire = WireJob::from_job(&job, thumb_cached, &format!("{slot:?}"));
let disk_key = job.cache.as_ref().map(|(_, key)| g.index.cache_key(*key));
let thumb_cached = disk_key.as_ref().is_some_and(|key| g.index.touch(key));
let mut wire = WireJob::from_job(&job, thumb_cached, &format!("{slot:?}"));
// The namespace also changes when a new library's cache starts again at generation zero.
wire.cache_generation = g.index.namespace();
wire.thumb = disk_key.clone();
let Ok(text) = serde_json::to_string(&wire) else { return Some(job) };
g.next_id = g.next_id.wrapping_add(1);
let id = g.next_id;
let msg = Object::new();
set(&msg, "id", &id.into());
set(&msg, "job", &text.into());
let namespace = g.index.namespace();
let w = &mut g.workers[i];
if let Err(e) = w.worker.post_message(&msg) {
log::warn!("render worker {i}: {e:?}");
return Some(job);
}
w.busy = Some(Busy { id, slot, photo: job.photo, level: job.level, key: job.key, cache: job.cache });
w.busy = Some(Busy {
disk_key,
namespace,
request_id: job.request_id,
cache_generation: job.cache_generation,
id,
slot,
photo: job.photo,
level: job.level,
key: job.key,
cache: job.cache,
});
None
}
fn finished(&mut self) -> Vec<(Slot, RenderResult, f64)> {
let mut g = self.0.borrow_mut();
// Persist invalidation even when the cleared library has no visible render requests.
if let Some(cache) = g.preview_generation.cache() {
g.observe_cache(&cache, cache.generation());
}
g.inline_this_frame = false;
std::mem::take(&mut g.done)
}
+10 -1
View File
@@ -9,8 +9,17 @@ default-run = "lightcraft"
[lints]
workspace = true
[features]
default = ["denoise"]
denoise = ["lightcraft-engine/denoise"]
rawnind-model = ["denoise", "lightcraft-engine/rawnind-model"]
# Open HEIC/HEIF photos (the optional lightcraft-heif crate). Not default so distributors choose
# it (HEVC patents); build with `--features heif` to include it.
heif = ["lightcraft-codecs/heif"]
[dependencies]
lightcraft-engine = { workspace = true }
lightcraft-engine = { workspace = true, features = ["sam"] }
lightcraft-sysmem = { workspace = true }
lightcraft-ui-egui = { workspace = true }
lightcraft-codecs = { workspace = true }
lightcraft-raster = { workspace = true }
+6 -26
View File
@@ -1,32 +1,12 @@
//! Give memory the system allocator keeps after frees back to the system
//! ([`lightcraft_engine::memory::set_release_hook`]).
//!
//! macOS's allocator keeps freed large blocks mapped as "reusable": they no longer count in the
//! process's memory footprint but stay in its resident size until the system runs short. After a
//! raw decode (hundreds of MB of temporaries) that made the resident size climb towards the sum of
//! everything ever decoded. `malloc_zone_pressure_relief` (libSystem) returns those pages now.
//! Elsewhere (glibc, Windows) blocks this large are unmapped when freed: nothing to do.
//! ([`lightcraft_engine::memory::set_release_hook`]). The platform call lives in
//! `lightcraft-sysmem`, the workspace's only crate allowed `unsafe`.
/// Install the hook for this platform.
/// Install the hook.
pub fn install() {
#[cfg(target_os = "macos")]
lightcraft_engine::memory::set_release_hook(macos::release);
lightcraft_engine::memory::set_release_hook(release);
}
#[cfg(target_os = "macos")]
#[allow(unsafe_code)]
mod macos {
unsafe extern "C" {
/// libSystem malloc: release free memory of `zone` (all zones when null) back to the
/// system, up to `goal` bytes (0 = as much as possible). Returns the bytes released.
fn malloc_zone_pressure_relief(zone: *mut std::ffi::c_void, goal: usize) -> usize;
}
pub fn release() {
// SAFETY: a documented libSystem function taking a null zone (= every zone) and a size;
// it only returns free pages to the system and touches no memory we own.
unsafe {
malloc_zone_pressure_relief(std::ptr::null_mut(), 0);
}
}
fn release() {
let _ = lightcraft_sysmem::release_free_memory();
}
+239 -34
View File
@@ -1,62 +1,267 @@
//! Loopback JSON-lines control server: one request per line, one reply per line.
//! This is the transport the MCP server (`lightcraft-cli mcp --connect`) wraps.
//!
//! The server only ever reads requests: a line that is not a JSON object with a string `method`
//! (an HTTP request line from a browser's cross-origin `fetch`, a stray `nc`, binary junk) gets
//! one error reply and the connection is closed, so nothing after it is executed. Lines are
//! bounded ([`MAX_LINE`]) and so is the number of open connections ([`MAX_CONNECTIONS`]); junk
//! never reaches the UI thread. See `docs/control-protocol.md`.
use std::io::{BufRead, BufReader, Write};
use std::io::{BufRead, BufReader, Read, Write};
use std::net::{TcpListener, TcpStream};
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::mpsc::{Receiver, Sender, channel};
use std::time::Duration;
use lightcraft_ui_egui::ControlRequest;
use serde_json::{Value, json};
/// Longest accepted request line (bytes, without the newline). Requests are small JSON objects;
/// anything longer is refused and the connection closed.
pub const MAX_LINE: usize = 4 * 1024 * 1024;
/// Connections served at once; further ones get an error line and are closed.
pub const MAX_CONNECTIONS: usize = 16;
pub fn start(port: u16, ctx: egui::Context) -> Receiver<ControlRequest> {
let (tx, rx) = channel::<ControlRequest>();
let listener = match TcpListener::bind(("127.0.0.1", port)) {
Ok(l) => l,
Err(e) => {
eprintln!("lightcraft: control server failed to bind 127.0.0.1:{port}: {e}");
log::error!("control server failed to bind 127.0.0.1:{port}: {e}");
return rx;
}
};
eprintln!("lightcraft: control server listening on 127.0.0.1:{port}");
std::thread::spawn(move || {
for stream in listener.incoming().flatten() {
let tx = tx.clone();
let ctx = ctx.clone();
std::thread::spawn(move || serve(stream, tx, ctx));
}
});
log::info!("control server listening on 127.0.0.1:{port}");
std::thread::spawn(move || accept_loop(listener, tx, ctx));
rx
}
fn serve(stream: TcpStream, tx: Sender<ControlRequest>, ctx: egui::Context) {
let Ok(read) = stream.try_clone() else { return };
let mut out = stream;
for line in BufReader::new(read).lines() {
let Ok(line) = line else { break };
if line.trim().is_empty() {
fn accept_loop(listener: TcpListener, tx: Sender<ControlRequest>, ctx: egui::Context) {
let open = Arc::new(AtomicUsize::new(0));
for mut stream in listener.incoming().flatten() {
if open.fetch_add(1, Ordering::SeqCst) >= MAX_CONNECTIONS {
open.fetch_sub(1, Ordering::SeqCst);
let _ = writeln!(stream, "{}", json!({"ok": false, "error": "too many control connections"}));
continue;
}
let reply = match serde_json::from_str::<Value>(&line) {
Ok(msg) => {
let id = msg.get("id").cloned().unwrap_or(Value::Null);
let method = msg.get("method").and_then(Value::as_str).unwrap_or("").to_string();
let params = msg.get("params").cloned().unwrap_or(json!({}));
let (req, rrx) = ControlRequest::new(method, params);
if tx.send(req).is_err() {
break;
}
ctx.request_repaint();
let mut r = rrx.recv_timeout(Duration::from_secs(60)).unwrap_or_else(|_| json!({"ok": false, "error": "timeout"}));
if let Some(o) = r.as_object_mut() {
o.insert("id".into(), id);
let tx = tx.clone();
let ctx = ctx.clone();
let open = Arc::clone(&open);
std::thread::spawn(move || {
serve(stream, &tx, &ctx);
open.fetch_sub(1, Ordering::SeqCst);
});
}
}
fn serve(stream: TcpStream, tx: &Sender<ControlRequest>, ctx: &egui::Context) {
let Ok(read) = stream.try_clone() else { return };
serve_lines(read, stream, |method, params| {
let (req, rrx) = ControlRequest::new(method, params);
tx.send(req).ok()?;
ctx.request_repaint();
Some(rrx.recv_timeout(Duration::from_secs(60)).unwrap_or_else(|_| json!({"ok": false, "error": "timeout"})))
});
}
/// One parsed line.
enum Line {
Request {
id: Value,
method: String,
params: Value,
},
Blank,
/// Not a request: reply with this error and close the connection.
Reject(String),
}
fn parse_line(bytes: &[u8]) -> Line {
let Ok(text) = std::str::from_utf8(bytes) else {
return Line::Reject("not a control request (invalid UTF-8); closing the connection".into());
};
let text = text.trim();
if text.is_empty() {
return Line::Blank;
}
let msg = match serde_json::from_str::<Value>(text) {
Ok(v) => v,
Err(e) => return Line::Reject(format!("bad JSON ({e}); expected one JSON request object per line; closing the connection")),
};
let Some(obj) = msg.as_object() else {
return Line::Reject("not a control request (expected a JSON object); closing the connection".into());
};
let Some(method) = obj.get("method").and_then(Value::as_str) else {
return Line::Reject("not a control request (missing string `method`); closing the connection".into());
};
Line::Request {
id: obj.get("id").cloned().unwrap_or(Value::Null),
method: method.to_string(),
params: obj.get("params").cloned().unwrap_or(json!({})),
}
}
/// Read one `\n`-terminated line of at most `max` bytes into `buf` (newline stripped).
/// `Ok(false)` at end of stream; `Err` when the line is too long or the read fails.
fn read_bounded_line(reader: &mut impl BufRead, buf: &mut Vec<u8>, max: usize) -> Result<bool, String> {
buf.clear();
// `take` bounds the bytes buffered for one line: a peer can't make us allocate more than `max`.
let limit = u64::try_from(max).unwrap_or(u64::MAX).saturating_add(1);
let n = reader.take(limit).read_until(b'\n', buf).map_err(|e| e.to_string())?;
if n == 0 {
return Ok(false);
}
if buf.last() == Some(&b'\n') {
buf.pop();
if buf.last() == Some(&b'\r') {
buf.pop();
}
return Ok(true);
}
if buf.len() > max {
return Err(format!("request line longer than {max} bytes; closing the connection"));
}
// Last line without a newline, then end of stream.
Ok(true)
}
/// Serve requests from `read`, answering on `out`, until the peer closes, sends something that is
/// not a request, or `handle` returns `None` (app gone). `handle` gets `(method, params)` and
/// returns the reply object (the request's `id` is added here).
fn serve_lines<R: Read, W: Write>(read: R, mut out: W, mut handle: impl FnMut(String, Value) -> Option<Value>) {
let mut reader = BufReader::new(read);
let mut buf = Vec::new();
loop {
let reject = match read_bounded_line(&mut reader, &mut buf, MAX_LINE) {
Ok(false) => return,
Err(e) => e,
Ok(true) => match parse_line(&buf) {
Line::Blank => continue,
Line::Reject(e) => e,
Line::Request { id, method, params } => {
let Some(mut r) = handle(method, params) else { return };
if let Some(o) = r.as_object_mut() {
o.insert("id".into(), id);
}
if writeln!(out, "{r}").and_then(|()| out.flush()).is_err() {
return;
}
continue;
}
r
}
Err(e) => json!({"ok": false, "error": format!("bad JSON: {e}")}),
},
};
if writeln!(out, "{reply}").is_err() {
break;
let _ = writeln!(out, "{}", json!({"ok": false, "error": reject})).and_then(|()| out.flush());
return;
}
}
#[cfg(test)]
mod tests {
use super::*;
fn run(input: &[u8]) -> (Vec<Value>, Vec<String>) {
let mut out = Vec::new();
let mut calls = Vec::new();
serve_lines(input, &mut out, |m, _p| {
calls.push(m);
Some(json!({"ok": true, "result": null}))
});
let replies = String::from_utf8(out).unwrap().lines().map(|l| serde_json::from_str::<Value>(l).unwrap()).collect();
(replies, calls)
}
#[test]
fn requests_are_answered_in_order_with_ids() {
let (r, calls) = run(b"{\"id\":1,\"method\":\"a\"}\n\n{\"id\":2,\"method\":\"b\",\"params\":{}}\r\n{\"method\":\"c\"}");
assert_eq!(calls, ["a", "b", "c"]);
assert_eq!(r.len(), 3);
assert_eq!(r[0]["id"], 1);
assert_eq!(r[1]["id"], 2);
assert_eq!(r[2]["id"], Value::Null);
}
#[test]
fn http_request_closes_before_the_body_runs() {
// A browser's cross-origin `fetch` POST: the request line is rejected and the connection
// closed, so the JSON body is never executed.
let http = b"POST / HTTP/1.1\r\nHost: 127.0.0.1:7980\r\nContent-Type: text/plain\r\n\r\n\
{\"id\":1,\"method\":\"engine.execute\",\"params\":{\"command\":\"folder.move\"}}\n";
let (r, calls) = run(http);
assert!(calls.is_empty(), "nothing may run: {calls:?}");
assert_eq!(r.len(), 1);
assert_eq!(r[0]["ok"], false);
assert!(r[0]["error"].as_str().unwrap().contains("closing the connection"));
}
#[test]
fn non_request_json_closes_the_connection() {
for junk in [
&b"[1,2]\n{\"method\":\"a\"}\n"[..],
b"{\"id\":1}\n{\"method\":\"a\"}\n",
b"{\"method\":5}\n{\"method\":\"a\"}\n",
b"\xff\xfe\n{\"method\":\"a\"}\n",
] {
let (r, calls) = run(junk);
assert!(calls.is_empty());
assert_eq!(r.len(), 1);
assert_eq!(r[0]["ok"], false);
}
}
#[test]
fn a_request_before_junk_still_runs() {
let (r, calls) = run(b"{\"method\":\"a\"}\nGET / HTTP/1.1\n{\"method\":\"b\"}\n");
assert_eq!(calls, ["a"]);
assert_eq!(r.len(), 2);
assert_eq!(r[1]["ok"], false);
}
#[test]
fn overlong_line_is_refused_without_buffering_it_all() {
// An endless line without a newline: reading stops after MAX_LINE + 1 bytes.
let endless = std::io::repeat(b'x');
let mut out = Vec::new();
let mut ran = false;
serve_lines(endless, &mut out, |_, _| {
ran = true;
None
});
assert!(!ran);
let reply: Value = serde_json::from_slice(out.strip_suffix(b"\n").unwrap()).unwrap();
assert!(reply["error"].as_str().unwrap().contains("longer than"));
}
#[test]
fn line_at_the_limit_is_accepted() {
let mut buf = Vec::new();
let line = vec![b'a'; 10];
let mut input = line.clone();
input.push(b'\n');
assert_eq!(read_bounded_line(&mut &input[..], &mut buf, 10), Ok(true));
assert_eq!(buf, line);
assert!(read_bounded_line(&mut &[b'a'; 11][..], &mut buf, 10).is_err());
}
#[test]
fn tcp_connection_is_closed_after_junk() {
// End to end over a real loopback socket (the transport the app uses).
let listener = TcpListener::bind(("127.0.0.1", 0)).unwrap();
let addr = listener.local_addr().unwrap();
let server = std::thread::spawn(move || {
let (s, _) = listener.accept().unwrap();
let read = s.try_clone().unwrap();
serve_lines(read, s, |_, _| Some(json!({"ok": true})));
});
let mut c = TcpStream::connect(addr).unwrap();
c.set_read_timeout(Some(Duration::from_secs(10))).unwrap();
c.write_all(b"{\"id\":7,\"method\":\"x\"}\nhello\n{\"method\":\"y\"}\n").unwrap();
let mut text = String::new();
c.read_to_string(&mut text).unwrap(); // returns at EOF: the server closed the connection
server.join().unwrap();
let lines: Vec<&str> = text.lines().collect();
assert_eq!(lines.len(), 2, "{text}");
assert!(lines[0].contains("\"id\":7"));
assert!(lines[1].contains("closing the connection"));
}
}
+145
View File
@@ -0,0 +1,145 @@
//! File-dialog filters that match extensions whatever their letter case.
//!
//! On Linux, `rfd` turns each extension into a `*.ext` glob for the GTK dialog and for the
//! desktop portal (`xdg-desktop-portal` FileChooser), and both match globs case-sensitively, so
//! a filter of `nef` hid `DSC_1234.NEF` (issue #342). Windows and macOS ignore case. Every
//! dialog filter of the app goes through [`FileDialogExt::add_filter_nocase`] (a test checks
//! that nothing else calls `add_filter`), so no extension can hit this.
use rfd::FileDialog;
/// The patterns to hand to `rfd` for `exts`, which are plain extensions without the dot.
///
/// Windows and macOS take the extension literally and ignore case: lower case, de-duplicated.
/// Linux (`linux`): `rfd` formats each entry as `*.{entry}`, which GTK 3 and the portal match
/// as a glob, so an entry may carry glob syntax. Each extension gives the lower-case form, the
/// UPPER-case form (kept in case a portal backend does not do `[...]` classes) and a bracket
/// form such as `[nN][eE][fF]` that matches every mix (`.Nef`, `.nEf`, ...).
pub fn dialog_extensions_for(exts: &[&str], linux: bool) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
let mut push = |s: String| {
if !s.is_empty() && !out.contains(&s) {
out.push(s);
}
};
for ext in exts {
let ext = ext.trim_start_matches('.');
let lower = ext.to_lowercase();
push(lower.clone());
if !linux {
continue;
}
push(ext.to_uppercase());
// `[` `]` `*` `?` in an extension would already be glob syntax: leave those alone
if lower.chars().any(|c| c.is_alphabetic()) && !lower.contains(['[', ']', '*', '?', '\\']) {
push(
lower
.chars()
.map(|c| {
let up: String = c.to_uppercase().collect();
if up.chars().count() == 1 && up != c.to_string() { format!("[{c}{up}]") } else { c.to_string() }
})
.collect(),
);
}
}
out
}
/// [`dialog_extensions_for`] for the platform this is built for.
pub fn dialog_extensions(exts: &[&str]) -> Vec<String> {
dialog_extensions_for(exts, cfg!(all(unix, not(target_os = "macos"))))
}
/// `add_filter` that matches the extensions in any letter case.
pub trait FileDialogExt {
fn add_filter_nocase(self, name: impl Into<String>, exts: &[&str]) -> Self;
}
impl FileDialogExt for FileDialog {
fn add_filter_nocase(self, name: impl Into<String>, exts: &[&str]) -> Self {
self.add_filter(name, &dialog_extensions(exts))
}
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::*;
#[test]
fn other_platforms_get_lower_case_once() {
assert_eq!(dialog_extensions_for(&["NEF", "nef", ".Jpg", "jpg", ""], false), ["nef", "jpg"]);
}
#[test]
fn linux_gets_lower_upper_and_any_mix() {
assert_eq!(dialog_extensions_for(&["nef"], true), ["nef", "NEF", "[nN][eE][fF]"]);
// digits have no case; a duplicate adds nothing
assert_eq!(dialog_extensions_for(&["mp4", "MP4"], true), ["mp4", "MP4", "[mM][pP]4"]);
assert_eq!(dialog_extensions_for(&["7z"], true), ["7z", "7Z", "7[zZ]"]);
assert_eq!(dialog_extensions_for(&["123"], true), ["123"]);
}
#[test]
fn linux_glob_syntax_is_not_wrapped_again() {
assert_eq!(dialog_extensions_for(&["[nN]ef"], true), ["[nn]ef", "[NN]EF"]);
}
/// The Linux patterns match like GTK's glob (`*.{entry}`): checked against a tiny matcher of
/// the subset used here (verified against real GTK 3.24 `GtkFileFilter` in the PR notes).
fn glob_ext(entry: &str, ext: &str) -> bool {
let mut pat = entry.chars().peekable();
let mut s = ext.chars();
while let Some(p) = pat.next() {
let Some(c) = s.next() else { return false };
if p == '[' {
let mut hit = false;
for q in pat.by_ref() {
if q == ']' {
break;
}
hit |= q == c;
}
if !hit {
return false;
}
} else if p != c {
return false;
}
}
s.next().is_none()
}
#[test]
fn linux_patterns_match_every_case_mix() {
let pats = dialog_extensions_for(&["nef", "cr2", "xmp"], true);
let matches = |name: &str| pats.iter().any(|p| glob_ext(p, name));
for ok in ["nef", "NEF", "Nef", "nEF", "CR2", "Cr2", "XMP", "xMp"] {
assert!(matches(ok), "{ok}");
}
for bad in ["nex", "ne", "nefs", "jpg", "cr3"] {
assert!(!matches(bad), "{bad}");
}
}
/// The app's own extension list (the one the folder scan uses) is covered whole.
#[test]
fn import_extensions_are_all_covered_in_both_cases() {
let pats = dialog_extensions_for(lightcraft_engine::import::EXTENSIONS, true);
for e in lightcraft_engine::import::EXTENSIONS {
assert!(pats.iter().any(|p| glob_ext(p, &e.to_uppercase())), "{e}");
assert!(pats.iter().any(|p| p == e), "{e}");
}
}
/// Every dialog filter goes through the helper: `main.rs` must not call rfd's own
/// `add_filter`.
#[test]
fn no_dialog_bypasses_the_helper() {
for (file, src) in [("main.rs", include_str!("main.rs")), ("control_server.rs", include_str!("control_server.rs"))] {
assert!(!src.contains(".add_filter("), "{file} calls rfd's add_filter; use add_filter_nocase");
}
assert!(include_str!("main.rs").contains(".add_filter_nocase("));
}
}
+689
View File
@@ -0,0 +1,689 @@
//! The desktop app's logger: `log::` records go to standard error and to a log file.
//!
//! It replaces the stderr-only `StderrLog` that `main.rs` used to install. A launch from a
//! desktop menu or the Dock has no terminal, so the file is what a bug report can attach:
//! `<settings folder>/logs/lightcraft.log`, next to `ui.json` (`config_dir` in `main.rs`; Linux
//! `$XDG_CONFIG_HOME/lightcraft/logs`, by default `~/.config/lightcraft/logs`). Never in the
//! library folder. Each start moves the previous log to `lightcraft.1.log` (and that one to `.2`),
//! so the log of a run that crashed survives the next launch. Runs with `LIGHTCRAFT_NO_PREFS`
//! (tests, scripts) log to standard error only, so they don't rotate away the user's own logs.
//!
//! Levels ([`filter_spec`]): `info` for LightCraft's own crates, `warn` for everything else (wgpu
//! and naga are chatty). `LIGHTCRAFT_LOG` keeps the meaning it had with the old logger: `info` or
//! `debug` lowers LightCraft's own crates to that level, any other value means warnings and errors
//! only. Without it, `RUST_LOG` replaces the default with env_logger-style directives: `debug`,
//! `warn,lightcraft_pipeline=trace`, `wgpu_core=info`. A directive ending in `*` matches every
//! target that starts with it (`lightcraft*=debug`).
//!
//! Records logged before the settings folder is known are kept (up to [`MAX_PENDING`]) and
//! written once the file is attached. Writing never panics: a file that can't be created or
//! written leaves standard error as the only sink. Panics that escape everything are added to the
//! file by [`AppLogger::record_panic`] (the default hook already printed them on standard error).
//!
//! This is the desktop app's logger only; `lightcraft-cli` doesn't share a process with it.
use std::fs::File;
use std::io::Write;
use std::path::{Path, PathBuf};
use std::sync::{Mutex, OnceLock, PoisonError, TryLockError};
use std::time::SystemTime;
use log::LevelFilter;
/// The current log file's name inside the log folder.
pub const LOG_FILE: &str = "lightcraft.log";
/// How many previous logs are kept (`lightcraft.1.log` … `lightcraft.<KEEP>.log`).
pub const KEEP: usize = 2;
/// The log file stops growing past this size (a runaway warning can't fill the disk).
pub const MAX_FILE_BYTES: u64 = 16 * 1024 * 1024;
/// Records kept in memory until the log file is attached.
pub const MAX_PENDING: usize = 512;
/// The built-in filter when neither `LIGHTCRAFT_LOG` nor `RUST_LOG` says otherwise.
pub const DEFAULT_FILTER: &str = "warn,lightcraft*=info";
/// The filter directives for this run from `LIGHTCRAFT_LOG` and `RUST_LOG` (blank = unset).
///
/// `LIGHTCRAFT_LOG` wins when set and means what it did for the old stderr logger: `info` or
/// `debug` for LightCraft's own crates (warnings and errors from everything else), any other value
/// warnings and errors only. Else `RUST_LOG` as env_logger directives, else [`DEFAULT_FILTER`].
pub fn filter_spec(lightcraft_log: Option<&str>, rust_log: Option<&str>) -> String {
let set = |v: Option<&str>| v.map(str::trim).filter(|v| !v.is_empty()).map(str::to_owned);
if let Some(level) = set(lightcraft_log) {
return match level.as_str() {
"debug" => "warn,lightcraft*=debug".to_owned(),
"info" => "warn,lightcraft*=info".to_owned(),
_ => "warn".to_owned(),
};
}
set(rust_log).unwrap_or_else(|| DEFAULT_FILTER.to_owned())
}
/// Per-target level filter parsed from env_logger-style directives.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Filter {
default: LevelFilter,
directives: Vec<(String, LevelFilter)>,
}
impl Filter {
/// Parse `spec`; unknown levels and empty parts are skipped. Without a bare level, targets no
/// directive names log errors only.
pub fn parse(spec: &str) -> Filter {
let mut f = Filter { default: LevelFilter::Error, directives: Vec::new() };
for part in spec.split(',').map(str::trim).filter(|p| !p.is_empty()) {
match part.split_once('=') {
Some((name, level)) => {
let name = name.trim();
if let (false, Some(level)) = (name.is_empty(), parse_level(level)) {
f.directives.push((name.to_owned(), level));
}
}
None => match parse_level(part) {
Some(level) => f.default = level,
// A bare target name: everything from it (env_logger does the same).
None => f.directives.push((part.to_owned(), LevelFilter::Trace)),
},
}
}
f
}
/// The level that applies to `target` (the most specific matching directive wins).
pub fn level_for(&self, target: &str) -> LevelFilter {
self.directives.iter().filter(|(name, _)| matches(name, target)).max_by_key(|(name, _)| name.len()).map_or(self.default, |(_, level)| *level)
}
/// The most verbose level any target can reach (for `log::set_max_level`).
pub fn max(&self) -> LevelFilter {
self.directives.iter().map(|(_, level)| *level).fold(self.default, Ord::max)
}
}
fn parse_level(s: &str) -> Option<LevelFilter> {
s.trim().parse().ok()
}
/// `name` matches `target` itself and its submodules (`a` matches `a` and `a::b`, not `ab`);
/// `name*` matches every target starting with `name`.
fn matches(name: &str, target: &str) -> bool {
match name.strip_suffix('*') {
Some(prefix) => target.starts_with(prefix),
None => target.strip_prefix(name).is_some_and(|rest| rest.is_empty() || rest.starts_with("::")),
}
}
/// `2026-10-08T07:59:17.728Z` (UTC, milliseconds). A clock before 1970 reads as the epoch.
pub fn timestamp(t: SystemTime) -> String {
let since = t.duration_since(std::time::UNIX_EPOCH).unwrap_or_default();
let secs = since.as_secs();
let (days, rem) = (secs / 86_400, secs % 86_400);
let (y, m, d) = civil_from_days(days);
format!("{y:04}-{m:02}-{d:02}T{:02}:{:02}:{:02}.{:03}Z", rem / 3600, rem % 3600 / 60, rem % 60, since.subsec_millis())
}
/// Days since 1970-01-01 to a proleptic Gregorian (year, month, day) (Howard Hinnant's
/// `civil_from_days`, unsigned because `timestamp` clamps at the epoch). Saturating, so no clock
/// value can overflow it.
fn civil_from_days(days: u64) -> (u64, u64, u64) {
let z = days.saturating_add(719_468);
let era = z / 146_097;
let doe = z % 146_097;
let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let d = doy - (153 * mp + 2) / 5 + 1;
let m = if mp < 10 { mp + 3 } else { mp - 9 };
let y = era.saturating_mul(400).saturating_add(yoe).saturating_add(u64::from(m <= 2));
(y, m, d)
}
fn numbered(dir: &Path, n: usize) -> PathBuf {
if n == 0 { dir.join(LOG_FILE) } else { dir.join(format!("lightcraft.{n}.log")) }
}
/// Shift the previous logs up one (`.log` → `.1.log` → … → `.<KEEP>.log`, the oldest dropped)
/// and create a fresh, empty log file in `dir` (created if missing).
pub fn rotate(dir: &Path) -> std::io::Result<(PathBuf, File)> {
std::fs::create_dir_all(dir)?;
for n in (1..=KEEP).rev() {
match std::fs::rename(numbered(dir, n - 1), numbered(dir, n)) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e),
_ => {}
}
}
let path = numbered(dir, 0);
let file = File::create(&path)?;
Ok((path, file))
}
/// One formatted log line: `<timestamp> <LEVEL> [<thread>] <target>: <message>\n`.
pub fn format_line(ts: &str, level: log::Level, thread: &str, target: &str, message: &str) -> String {
let message = message.strip_suffix('\n').unwrap_or(message);
format!("{ts} {level:<5} [{thread}] {target}: {message}\n")
}
/// Where formatted lines go besides standard error: kept in memory until a file is attached,
/// then appended to it until it reaches its size cap.
pub struct Sink {
file: Option<File>,
pending: Vec<String>,
dropped: usize,
written: u64,
max_bytes: u64,
capped: bool,
}
impl Sink {
pub fn new(max_bytes: u64) -> Sink {
Sink { file: None, pending: Vec::new(), dropped: 0, written: 0, max_bytes, capped: false }
}
/// Append `line` to the file, or keep it until [`Sink::attach`].
pub fn write(&mut self, line: &str) {
if self.file.is_none() {
if self.capped {
// No file is coming ([`Sink::no_file`]): standard error had the line.
} else if self.pending.len() < MAX_PENDING {
self.pending.push(line.to_owned());
} else {
self.dropped = self.dropped.saturating_add(1);
}
return;
}
self.append(line);
}
/// Start writing to `file` (a fresh one: the size cap starts over), first the lines kept so far.
pub fn attach(&mut self, file: File) {
self.file = Some(file);
self.written = 0;
self.capped = false;
for line in std::mem::take(&mut self.pending) {
self.append(&line);
}
if self.dropped > 0 {
let note = format!("{} earlier log lines were dropped before the log file was opened\n", self.dropped);
self.dropped = 0;
self.append(&note);
}
}
/// There will be no log file: forget the lines kept for it and keep no more.
pub fn no_file(&mut self) {
self.file = None;
self.pending = Vec::new();
self.dropped = 0;
self.capped = true;
}
fn append(&mut self, line: &str) {
if self.capped {
return;
}
let Some(file) = self.file.as_mut() else { return };
let len = u64::try_from(line.len()).unwrap_or(u64::MAX);
let text = if self.written.saturating_add(len) > self.max_bytes {
self.capped = true;
format!("log file reached {} bytes; later records go to standard error only\n", self.max_bytes)
} else {
self.written = self.written.saturating_add(len);
line.to_owned()
};
// A full disk or a vanished file must not take the app down: drop the file sink.
if file.write_all(text.as_bytes()).is_err() {
self.file = None;
self.capped = true;
}
}
}
/// The installed logger: a [`Filter`], standard error and a [`Sink`].
pub struct AppLogger {
filter: Filter,
stderr: bool,
sink: Mutex<Sink>,
}
impl AppLogger {
pub fn new(filter: Filter, stderr: bool) -> AppLogger {
AppLogger { filter, stderr, sink: Mutex::new(Sink::new(MAX_FILE_BYTES)) }
}
/// Rotate the logs in `dir` and send records to the fresh file; returns its path. On an error
/// standard error stays the only sink.
pub fn attach_dir(&self, dir: &Path) -> Result<PathBuf, String> {
// A thread that panicked while logging leaves the sink usable: take it back.
let mut sink = self.sink.lock().unwrap_or_else(PoisonError::into_inner);
match rotate(dir) {
Ok((path, file)) => {
sink.attach(file);
Ok(path)
}
Err(e) => {
sink.no_file();
Err(format!("{}: {e}", dir.display()))
}
}
}
/// No log file for this run (`LIGHTCRAFT_NO_PREFS`): log to standard error only.
pub fn no_file(&self) {
self.sink.lock().unwrap_or_else(PoisonError::into_inner).no_file();
}
/// Add a panic report to the log file only (the default panic hook has printed it on standard
/// error), whatever the filter. Skipped when the sink is busy: the panic may have happened on
/// this very thread while it held the lock, and waiting would hang the app.
pub fn record_panic(&self, thread: &str, report: &str) {
let line = format_line(&timestamp(SystemTime::now()), log::Level::Error, thread, "lightcraft", report);
let mut sink = match self.sink.try_lock() {
Ok(sink) => sink,
Err(TryLockError::Poisoned(p)) => p.into_inner(),
Err(TryLockError::WouldBlock) => return,
};
sink.write(&line);
}
}
impl log::Log for AppLogger {
fn enabled(&self, metadata: &log::Metadata<'_>) -> bool {
metadata.level() <= self.filter.level_for(metadata.target())
}
fn log(&self, record: &log::Record<'_>) {
if !self.enabled(record.metadata()) {
return;
}
// Formatted before the sink is locked, so a message that logs while it is formatted can't deadlock.
let thread = std::thread::current();
let line =
format_line(&timestamp(SystemTime::now()), record.level(), thread.name().unwrap_or("?"), record.target(), &record.args().to_string());
if self.stderr {
// No terminal (a Windows GUI build, a closed pipe) is not an error worth reporting.
let _ = std::io::stderr().write_all(line.as_bytes());
}
self.sink.lock().unwrap_or_else(PoisonError::into_inner).write(&line);
}
fn flush(&self) {}
}
/// Install the logger (filter from [`filter_spec`]); `None` when another logger was installed
/// first. Call [`AppLogger::attach_dir`] (or [`AppLogger::no_file`]) once the arguments are parsed.
pub fn install() -> Option<&'static AppLogger> {
static LOGGER: OnceLock<AppLogger> = OnceLock::new();
let var = |name| std::env::var(name).ok();
let filter = Filter::parse(&filter_spec(var("LIGHTCRAFT_LOG").as_deref(), var("RUST_LOG").as_deref()));
let max = filter.max();
let logger = LOGGER.get_or_init(|| AppLogger::new(filter, true));
log::set_logger(logger).ok()?;
log::set_max_level(max);
Some(logger)
}
/// Chain a panic hook after the current one (the engine guard's: standard error plus
/// `lightcraft-panics.log` in the temp folder) that also puts the panic in the log file.
pub fn record_panics(logger: &'static AppLogger) {
let previous = std::panic::take_hook();
std::panic::set_hook(Box::new(move |info| {
previous(info);
let thread = std::thread::current();
let at = info.location().map(|l| format!(" at {}:{}", l.file(), l.line())).unwrap_or_default();
let report = format!("panic{at}: {}", lightcraft_engine::guard::panic_message(info.payload()));
logger.record_panic(thread.name().unwrap_or("?"), &report);
}));
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::{Duration, UNIX_EPOCH};
fn temp_dir(tag: &str) -> PathBuf {
static N: AtomicUsize = AtomicUsize::new(0);
let d = std::env::temp_dir().join(format!("lightcraft-logging-{tag}-{}-{}", std::process::id(), N.fetch_add(1, Ordering::Relaxed)));
let _ = std::fs::remove_dir_all(&d);
d
}
fn read(path: &Path) -> String {
std::fs::read_to_string(path).expect("read log")
}
fn record(logger: &AppLogger, level: log::Level, target: &str, msg: &str) {
use log::Log;
logger.log(&log::Record::builder().level(level).target(target).args(format_args!("{msg}")).build());
}
/// The logger this replaced (`StderrLog`): warnings and errors from every target, `info` and
/// `debug` only from LightCraft's own crates, and only when `LIGHTCRAFT_LOG` asked for them.
fn old_stderr_log_enabled(lightcraft_log: &str, level: log::Level, target: &str) -> bool {
let max = match lightcraft_log {
"debug" => LevelFilter::Debug,
"info" => LevelFilter::Info,
_ => LevelFilter::Warn,
};
level <= max && (level <= log::Level::Warn || target.starts_with("lightcraft"))
}
#[test]
fn the_default_filter_shows_lightcraft_info_and_other_crates_warnings() {
let f = Filter::parse(DEFAULT_FILTER);
assert_eq!(f.level_for("lightcraft"), LevelFilter::Info);
assert_eq!(f.level_for("lightcraft::control_server"), LevelFilter::Info);
assert_eq!(f.level_for("lightcraft_engine::guard"), LevelFilter::Info);
assert_eq!(f.level_for("lightcraft_ui_egui::render"), LevelFilter::Info);
assert_eq!(f.level_for("wgpu_core::device"), LevelFilter::Warn);
assert_eq!(f.level_for("naga"), LevelFilter::Warn);
assert_eq!(f.max(), LevelFilter::Info);
assert_eq!(filter_spec(None, None), DEFAULT_FILTER);
// An empty variable is the same as an unset one.
assert_eq!(filter_spec(Some(""), Some(" ")), DEFAULT_FILTER);
}
/// `LIGHTCRAFT_LOG` keeps working exactly as it did with the old stderr-only logger.
#[test]
fn lightcraft_log_keeps_its_meaning() {
let targets =
["lightcraft", "lightcraft::control_server", "lightcraft_engine::segment", "lightcraft_gpu", "wgpu_core::device", "naga", "eframe"];
let levels = [log::Level::Error, log::Level::Warn, log::Level::Info, log::Level::Debug, log::Level::Trace];
for value in ["info", "debug", "warn", "error", "verbose", "INFO"] {
let f = Filter::parse(&filter_spec(Some(value), None));
for target in targets {
for level in levels {
assert_eq!(level <= f.level_for(target), old_stderr_log_enabled(value, level, target), "LIGHTCRAFT_LOG={value} {level} {target}");
}
}
}
assert_eq!(Filter::parse(&filter_spec(Some("debug"), None)).max(), LevelFilter::Debug);
assert_eq!(Filter::parse(&filter_spec(Some("warn"), None)).max(), LevelFilter::Warn);
}
#[test]
fn rust_log_replaces_the_default_and_lightcraft_log_wins_over_it() {
assert_eq!(filter_spec(None, Some("warn,wgpu_core=info")), "warn,wgpu_core=info");
let f = Filter::parse(&filter_spec(None, Some("debug")));
assert_eq!(f.level_for("eframe"), LevelFilter::Debug);
// The app's own variable is the more specific choice.
let f = Filter::parse(&filter_spec(Some("info"), Some("trace")));
assert_eq!(f.level_for("eframe"), LevelFilter::Warn);
assert_eq!(f.level_for("lightcraft_engine"), LevelFilter::Info);
}
#[test]
fn directives_follow_env_logger_and_the_most_specific_one_wins() {
let f = Filter::parse("info,wgpu_core=error,lightcraft_pipeline=trace,lightcraft_pipeline::tiles=off");
assert_eq!(f.level_for("eframe"), LevelFilter::Info);
assert_eq!(f.level_for("wgpu_core::instance"), LevelFilter::Error);
assert_eq!(f.level_for("lightcraft_pipeline::stage"), LevelFilter::Trace);
assert_eq!(f.level_for("lightcraft_pipeline::tiles"), LevelFilter::Off);
assert_eq!(f.max(), LevelFilter::Trace);
// A module name is matched at `::` boundaries, not as a bare prefix.
assert_eq!(f.level_for("wgpu_core_extra"), LevelFilter::Info);
// A trailing `*` is a prefix.
assert_eq!(Filter::parse("lightcraft*=debug").level_for("lightcraft_raw::cr2"), LevelFilter::Debug);
assert_eq!(Filter::parse("lightcraft*=debug").level_for("eframe"), LevelFilter::Error);
// A bare target name sets that target to the most verbose level, as env_logger does.
assert_eq!(Filter::parse("naga").level_for("naga::front"), LevelFilter::Trace);
assert_eq!(Filter::parse("naga").level_for("eframe"), LevelFilter::Error);
// Levels are case-insensitive and surrounding blanks are ignored.
assert_eq!(Filter::parse(" WARN , lightcraft = Debug ").level_for("lightcraft"), LevelFilter::Debug);
assert_eq!(Filter::parse(" WARN , lightcraft = Debug ").level_for("eframe"), LevelFilter::Warn);
}
#[test]
fn hostile_specs_never_panic_and_fall_back_sensibly() {
let long = "x".repeat(100_000);
let many = "a=info,".repeat(10_000);
for spec in [
"",
",,,",
"=",
"==",
"=debug",
"lightcraft=",
"nonsense=loud",
"🦀=info",
"*",
"*=",
"=*",
"a=b=c",
"warn,,lightcraft=DEBUG",
"\0",
"é::ü=trace",
&long,
&many,
] {
let f = Filter::parse(spec);
for target in ["", "lightcraft", "🦀", "a::b", "*"] {
let _ = f.level_for(target);
}
let _ = f.max();
let _ = filter_spec(Some(spec), Some(spec));
}
assert_eq!(Filter::parse("warn,,lightcraft=DEBUG").level_for("lightcraft"), LevelFilter::Debug);
// An unknown level is skipped: that target keeps the default.
assert_eq!(Filter::parse("nonsense=loud").level_for("nonsense"), LevelFilter::Error);
assert_eq!(Filter::parse("lightcraft=").level_for("lightcraft"), LevelFilter::Error);
assert_eq!(Filter::parse("=debug").max(), LevelFilter::Error);
assert_eq!(Filter::parse("🦀=info").level_for("🦀::claw"), LevelFilter::Info);
}
#[test]
fn timestamps_are_utc_with_milliseconds() {
assert_eq!(timestamp(UNIX_EPOCH), "1970-01-01T00:00:00.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_millis(1_791_446_357_728)), "2026-10-08T07:59:17.728Z");
// Leap day, and the day after it.
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(1_709_164_800)), "2024-02-29T00:00:00.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(1_709_251_199)), "2024-02-29T23:59:59.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(1_709_251_200)), "2024-03-01T00:00:00.000Z");
// 2000 is a leap year (divisible by 400), 2100 is not.
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(951_782_400)), "2000-02-29T00:00:00.000Z");
assert_eq!(timestamp(UNIX_EPOCH + Duration::from_secs(4_107_542_400)), "2100-03-01T00:00:00.000Z");
// Before the epoch (a clock set wrong) clamps instead of panicking.
assert_eq!(timestamp(UNIX_EPOCH - Duration::from_secs(5)), "1970-01-01T00:00:00.000Z");
}
#[test]
fn a_line_carries_time_level_thread_target_and_message() {
assert_eq!(
format_line("2026-10-08T07:59:17.728Z", log::Level::Info, "main", "lightcraft", "library /photos: 12 photos"),
"2026-10-08T07:59:17.728Z INFO [main] lightcraft: library /photos: 12 photos\n"
);
// A multi-line message keeps its lines; the record still ends in one newline.
assert!(format_line("t", log::Level::Error, "w", "x", "a\nb\n").ends_with("x: a\nb\n"));
}
#[test]
fn rotation_keeps_the_previous_logs_and_starts_an_empty_file() {
let dir = temp_dir("rotate");
for run in 1..=4 {
let (path, _file) = rotate(&dir).expect("rotate");
assert_eq!(path, dir.join(LOG_FILE));
assert_eq!(read(&path), "");
std::fs::write(&path, format!("run {run}")).expect("write");
}
assert_eq!(read(&dir.join(LOG_FILE)), "run 4");
assert_eq!(read(&dir.join("lightcraft.1.log")), "run 3");
assert_eq!(read(&dir.join("lightcraft.2.log")), "run 2");
assert!(!dir.join("lightcraft.3.log").exists(), "only {KEEP} old logs are kept");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn rotation_into_an_unusable_directory_is_an_error_not_a_panic() {
let dir = temp_dir("blocked");
std::fs::create_dir_all(dir.parent().expect("parent")).expect("tmp");
std::fs::write(&dir, "a file where the directory should be").expect("block");
assert!(rotate(&dir).is_err());
let _ = std::fs::remove_file(&dir);
}
#[test]
fn lines_logged_before_the_file_exists_are_written_when_it_is_attached() {
let dir = temp_dir("pending");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(MAX_FILE_BYTES);
sink.write("early 1\n");
sink.write("early 2\n");
sink.attach(file);
sink.write("late\n");
assert_eq!(read(&path), "early 1\nearly 2\nlate\n");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_pending_buffer_is_bounded_and_says_how_much_it_dropped() {
let dir = temp_dir("bounded");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(MAX_FILE_BYTES);
for i in 0..MAX_PENDING + 5 {
sink.write(&format!("line {i}\n"));
}
sink.attach(file);
let text = read(&path);
assert_eq!(text.lines().filter(|l| l.starts_with("line ")).count(), MAX_PENDING);
assert!(text.contains("5 earlier log lines were dropped"), "{text}");
assert!(text.contains("line 0\n") && !text.contains(&format!("line {MAX_PENDING}\n")));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn exactly_a_full_pending_buffer_drops_nothing() {
let dir = temp_dir("full");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(MAX_FILE_BYTES);
for i in 0..MAX_PENDING {
sink.write(&format!("line {i}\n"));
}
sink.attach(file);
let text = read(&path);
assert_eq!(text.lines().count(), MAX_PENDING);
assert!(!text.contains("dropped"), "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn without_a_file_nothing_is_kept_in_memory() {
let mut sink = Sink::new(MAX_FILE_BYTES);
sink.write("early\n");
sink.no_file();
sink.write("late\n");
assert!(sink.pending.is_empty() && sink.dropped == 0);
}
#[test]
fn the_file_stops_at_its_size_cap_with_one_note() {
let dir = temp_dir("cap");
let (path, file) = rotate(&dir).expect("rotate");
let mut sink = Sink::new(100);
sink.attach(file);
for i in 0..50 {
sink.write(&format!("line {i:02} with some padding\n"));
}
let text = read(&path);
assert_eq!(text.lines().filter(|l| l.starts_with("line ")).count(), 3, "{text}");
assert_eq!(text.matches("log file reached").count(), 1, "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_logger_filters_records_and_writes_them_to_the_attached_file() {
use log::Log;
let dir = temp_dir("logger");
let logger = AppLogger::new(Filter::parse(DEFAULT_FILTER), false);
record(&logger, log::Level::Info, "lightcraft", "before the settings directory");
record(&logger, log::Level::Info, "wgpu_core::device", "too chatty");
let path = logger.attach_dir(&dir).expect("attach");
assert_eq!(path, dir.join(LOG_FILE));
record(&logger, log::Level::Warn, "wgpu_hal::vulkan", "a real warning");
record(&logger, log::Level::Error, "lightcraft_engine::guard", "`develop.reset` failed unexpectedly: boom");
record(&logger, log::Level::Debug, "lightcraft", "below info");
let text = read(&path);
let lines: Vec<&str> = text.lines().collect();
assert_eq!(lines.len(), 3, "{text}");
assert!(lines[0].contains(" INFO [") && lines[0].ends_with("lightcraft: before the settings directory"), "{text}");
assert!(lines[1].ends_with("wgpu_hal::vulkan: a real warning"), "{text}");
assert!(lines[2].contains(" ERROR [") && lines[2].ends_with("lightcraft_engine::guard: `develop.reset` failed unexpectedly: boom"), "{text}");
assert!(logger.enabled(&log::Metadata::builder().level(log::Level::Info).target("lightcraft_ui_egui").build()));
assert!(!logger.enabled(&log::Metadata::builder().level(log::Level::Info).target("naga").build()));
// Attaching again rotates: the first log becomes `.1`.
logger.attach_dir(&dir).expect("attach again");
assert_eq!(read(&dir.join("lightcraft.1.log")), text);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn records_from_many_threads_are_whole_lines() {
let dir = temp_dir("threads");
let logger = std::sync::Arc::new(AppLogger::new(Filter::parse("info"), false));
let path = logger.attach_dir(&dir).expect("attach");
let workers: Vec<_> = (0..8)
.map(|t| {
let l = std::sync::Arc::clone(&logger);
std::thread::spawn(move || {
for i in 0..100 {
record(&l, log::Level::Info, "lightcraft", &format!("thread {t} record {i}"));
}
})
})
.collect();
for w in workers {
w.join().expect("worker");
}
let text = read(&path);
assert_eq!(text.lines().count(), 800);
assert!(text.lines().all(|l| l.contains(" lightcraft: thread ")), "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn attaching_to_an_unusable_directory_reports_an_error_and_keeps_logging() {
let dir = temp_dir("unusable");
std::fs::create_dir_all(dir.parent().expect("parent")).expect("tmp");
std::fs::write(&dir, "not a directory").expect("block");
let logger = AppLogger::new(Filter::parse("info"), false);
record(&logger, log::Level::Info, "lightcraft", "early");
assert!(logger.attach_dir(&dir).is_err());
record(&logger, log::Level::Error, "x", "still fine");
// Nothing waits in memory for a file that won't come.
assert!(logger.sink.lock().expect("sink").pending.is_empty());
let _ = std::fs::remove_file(&dir);
}
#[test]
fn a_poisoned_sink_still_logs() {
let dir = temp_dir("poison");
let logger = std::sync::Arc::new(AppLogger::new(Filter::parse("info"), false));
let l = std::sync::Arc::clone(&logger);
let _ = std::thread::spawn(move || {
let _guard = l.sink.lock().expect("lock");
panic!("poison the sink");
})
.join();
assert!(logger.sink.is_poisoned());
let path = logger.attach_dir(&dir).expect("attach");
record(&logger, log::Level::Warn, "lightcraft", "after the poison");
assert!(read(&path).contains("lightcraft: after the poison"));
let _ = std::fs::remove_dir_all(&dir);
}
/// The panic hook's report lands in the file, and never deadlocks when the panic happened
/// while this thread held the sink.
#[test]
fn a_panic_report_goes_to_the_file_and_never_deadlocks() {
let dir = temp_dir("panic");
let logger = AppLogger::new(Filter::parse("off"), false);
let path = logger.attach_dir(&dir).expect("attach");
logger.record_panic("render", "panicked at crates/x/src/lib.rs:3:5:\nboom");
{
let _held = logger.sink.lock().expect("lock");
logger.record_panic("main", "while the sink is held");
}
let text = read(&path);
assert!(text.contains(" ERROR [render] lightcraft: panicked at crates/x/src/lib.rs:3:5:\nboom\n"), "{text}");
assert!(!text.contains("while the sink is held"), "{text}");
let _ = std::fs::remove_dir_all(&dir);
}
}
File diff suppressed because it is too large. Load diff
+131 -36
View File
@@ -2,9 +2,12 @@
//! the in-window menus ([`lightcraft_ui_egui::menubar`]): the registry's menu paths, shortcuts,
//! enabled and checked state, and live labels ("Undo Exposure").
//!
//! Key handling: a native key equivalent consumes the key press before egui sees it, so
//! - shortcuts installed natively are listed in `LightcraftApp::native_shortcuts` and skipped by
//! the egui shortcut handler (nothing fires twice);
//! Key handling: AppKit offers a key press to the menu bar only when it carries ⌘ or ⌃, or is a
//! function key (F1…). Every other key (`E`, `1`, `⇧P`, `⌥Y`…) goes straight to the window, whose
//! winit view always takes it, so a menu item never fires from it. Hence
//! - shortcuts the menu bar really receives ([`menu_delivers`]) are listed in
//! `LightcraftApp::native_shortcuts` and skipped by the egui shortcut handler (nothing fires
//! twice); the others stay on their menu items for display and egui runs them;
//! - while a text field has keyboard focus, accelerators without ⌘ (`G`, `1`, `Delete`…) and the
//! text-editing ones (⌘A/⌘C/⌘V/⌘X/⌘Z) are removed, so typing and text editing work; they come
//! back when the field loses focus.
@@ -51,6 +54,9 @@ pub struct NativeMenu {
/// Structure of the installed menu (ids, kinds, submenu names); a change rebuilds it.
structure: String,
text_focus: bool,
/// The keyboard shortcuts editor is recording a key: every accelerator is off so the key
/// press reaches it.
capturing: bool,
}
/// `Cmd+Shift+Z` → a muda accelerator (`None` for keys we leave to egui, e.g. Escape).
@@ -146,9 +152,21 @@ fn yields_to_text(sc: &str) -> bool {
!sc.contains("Cmd") && !sc.contains("Ctrl") || TEXT_EDIT.contains(&sc)
}
/// Whether AppKit hands this shortcut to the menu bar: only key presses with ⌘ or ⌃, and function
/// keys. A plain `C` or `⇧P` never reaches it (the window's view takes the key), so egui has to run
/// those even though their menu item shows the key.
fn menu_delivers(sc: &str) -> bool {
sc.split('+').any(|part| matches!(part, "Cmd" | "Ctrl") || part.strip_prefix('F').is_some_and(|n| n.parse::<u8>().is_ok()))
}
/// The shortcuts the menu bar runs itself, for `LightcraftApp::native_shortcuts`.
fn owned_by_menu<'a>(installed: impl Iterator<Item = &'a str>, text_focus: bool) -> HashSet<String> {
installed.filter(|sc| menu_delivers(sc) && !(text_focus && yields_to_text(sc))).map(str::to_string).collect()
}
/// `&` marks a mnemonic in muda labels.
fn label_text(s: &str) -> String {
s.replace('&', "&&")
lightcraft_ui_egui::i18n::tr(s).replace('&', "&&")
}
fn structure_of(bar: &[(String, Vec<MenuNode>)]) -> String {
@@ -171,7 +189,7 @@ fn structure_of(bar: &[(String, Vec<MenuNode>)]) -> String {
out.push(';');
}
}
let mut s = String::new();
let mut s = format!("{};", lightcraft_ui_egui::i18n::language().code());
for (t, items) in bar {
s.push_str(t);
walk(items, &mut s);
@@ -196,7 +214,7 @@ impl NativeMenu {
let _ = tx.send(e.id.0);
repaint.request_repaint();
}));
let mut m = NativeMenu { menu: Menu::new(), items: HashMap::new(), rx, structure: String::new(), text_focus: false };
let mut m = NativeMenu { menu: Menu::new(), items: HashMap::new(), rx, structure: String::new(), text_focus: false, capturing: false };
m.rebuild(app);
app.native_menu = true;
m
@@ -210,30 +228,30 @@ impl NativeMenu {
// the application menu
let app_menu = Submenu::new("LightCraft", true);
let about = MenuItem::with_id("app.about", "About LightCraft", true, None);
let settings = MenuItem::with_id(SETTINGS, "Settings…", true, accelerator(SETTINGS_KEY));
let quit = MenuItem::with_id(QUIT, "Quit LightCraft", true, accelerator("Cmd+Q"));
let about = MenuItem::with_id("app.about", lightcraft_ui_egui::i18n::tr("About LightCraft"), true, None);
let settings = MenuItem::with_id(SETTINGS, lightcraft_ui_egui::i18n::tr("Settings…"), true, accelerator(SETTINGS_KEY));
let quit = MenuItem::with_id(QUIT, lightcraft_ui_egui::i18n::tr("Quit LightCraft"), true, accelerator("Cmd+Q"));
let _ = app_menu.append_items(&[
&about,
&PredefinedMenuItem::separator(),
&settings,
&PredefinedMenuItem::separator(),
&PredefinedMenuItem::services(None),
&PredefinedMenuItem::services(Some(lightcraft_ui_egui::i18n::tr("Services"))),
&PredefinedMenuItem::separator(),
&PredefinedMenuItem::hide(Some("Hide LightCraft")),
&PredefinedMenuItem::hide_others(None),
&PredefinedMenuItem::show_all(None),
&PredefinedMenuItem::hide(Some(lightcraft_ui_egui::i18n::tr("Hide LightCraft"))),
&PredefinedMenuItem::hide_others(Some(lightcraft_ui_egui::i18n::tr("Hide Others"))),
&PredefinedMenuItem::show_all(Some(lightcraft_ui_egui::i18n::tr("Show All"))),
&PredefinedMenuItem::separator(),
&quit,
]);
let _ = self.menu.append(&app_menu);
for (title, nodes) in &bar {
let sub = Submenu::new(title, true);
let sub = Submenu::new(label_text(title), true);
if title == "Window" {
let _ = sub.append_items(&[
&PredefinedMenuItem::minimize(None),
&PredefinedMenuItem::maximize(Some("Zoom")),
&PredefinedMenuItem::minimize(Some(lightcraft_ui_egui::i18n::tr("Minimize"))),
&PredefinedMenuItem::maximize(Some(lightcraft_ui_egui::i18n::tr("Zoom"))),
&PredefinedMenuItem::separator(),
]);
}
@@ -247,10 +265,16 @@ impl NativeMenu {
self.append_nodes(&sub, &nodes);
if title == "File" {
// ⌘W, the system's own item
let _ = sub.append_items(&[&PredefinedMenuItem::separator(), &PredefinedMenuItem::close_window(None)]);
let _ = sub.append_items(&[
&PredefinedMenuItem::separator(),
&PredefinedMenuItem::close_window(Some(lightcraft_ui_egui::i18n::tr("Close Window"))),
]);
}
if title == "Window" {
let _ = sub.append_items(&[&PredefinedMenuItem::separator(), &PredefinedMenuItem::bring_all_to_front(None)]);
let _ = sub.append_items(&[
&PredefinedMenuItem::separator(),
&PredefinedMenuItem::bring_all_to_front(Some(lightcraft_ui_egui::i18n::tr("Bring All to Front"))),
]);
sub.set_as_windows_menu_for_nsapp();
}
if title == "Help" {
@@ -260,6 +284,7 @@ impl NativeMenu {
}
self.menu.init_for_nsapp();
self.text_focus = false;
self.capturing = false;
self.publish_shortcuts(app);
}
@@ -282,12 +307,23 @@ impl NativeMenu {
let accel = shortcut.as_deref().filter(|s| !CONTEXTUAL.contains(s)).and_then(accelerator);
let handle = match checked {
Some(c) => {
let it = CheckMenuItem::with_id(key.clone(), label_text(label), *enabled, *c, accel);
let it = CheckMenuItem::with_id(
key.clone(),
lightcraft_ui_egui::menubar::display_item_label(id, params, label).replace('&', "&&"),
*enabled,
*c,
accel,
);
let _ = sub.append(&it);
Handle::Check(it)
}
None => {
let it = MenuItem::with_id(key.clone(), label_text(label), *enabled, accel);
let it = MenuItem::with_id(
key.clone(),
lightcraft_ui_egui::menubar::display_item_label(id, params, label).replace('&', "&&"),
*enabled,
accel,
);
let _ = sub.append(&it);
Handle::Plain(it)
}
@@ -312,22 +348,22 @@ impl NativeMenu {
/// Tell the egui shortcut handler which shortcuts the menu bar currently owns.
fn publish_shortcuts(&self, app: &mut LightcraftApp) {
app.native_shortcuts = self
.items
.values()
.filter(|i| i.accel.is_some())
.filter_map(|i| i.shortcut.clone())
.filter(|sc| !(self.text_focus && yields_to_text(sc)))
.collect::<HashSet<_>>();
let installed = self.items.values().filter(|i| i.accel.is_some()).filter_map(|i| i.shortcut.as_deref());
// (none while the keymap editor records a shortcut: every key goes to it)
app.native_shortcuts = if self.capturing { HashSet::new() } else { owned_by_menu(installed, self.text_focus) };
app.native_shortcuts.insert(SETTINGS_KEY.to_string());
}
/// Per frame: run chosen items, then sync labels / enabled / checked and the text-focus
/// accelerators with the app state.
pub fn update(&mut self, app: &mut LightcraftApp, ctx: &egui::Context) {
/// accelerators with the app state. Returns whether Quit was chosen: the caller saves
/// everything and exits the process itself (closing the viewport while AppKit terminates the
/// app can crash in winit's teardown, PR #444).
pub fn update(&mut self, app: &mut LightcraftApp, ctx: &egui::Context) -> bool {
lightcraft_ui_egui::i18n::set_language(app.ui.language);
let mut quit = false;
while let Ok(key) = self.rx.try_recv() {
if key == QUIT {
ctx.send_viewport_cmd(egui::ViewportCommand::Close);
quit = true;
continue;
}
if key == "app.about" || key == SETTINGS {
@@ -342,7 +378,7 @@ impl NativeMenu {
let bar = menu_bar(app);
if structure_of(&bar) != self.structure {
self.rebuild(app);
return;
return quit;
}
fn walk(nodes: &[MenuNode], items: &mut HashMap<String, Item>) {
for n in nodes {
@@ -352,8 +388,8 @@ impl NativeMenu {
let Some(it) = items.get_mut(&MenuNode::key(id, params)) else { continue };
if it.label != *label {
match &it.handle {
Handle::Plain(h) => h.set_text(label_text(label)),
Handle::Check(h) => h.set_text(label_text(label)),
Handle::Plain(h) => h.set_text(lightcraft_ui_egui::menubar::display_item_label(id, params, label).replace('&', "&&")),
Handle::Check(h) => h.set_text(lightcraft_ui_egui::menubar::display_item_label(id, params, label).replace('&', "&&")),
}
it.label = label.clone();
}
@@ -379,14 +415,16 @@ impl NativeMenu {
walk(&bar.iter().flat_map(|(_, v)| v.clone()).collect::<Vec<_>>(), &mut self.items);
let focus = ctx.egui_wants_keyboard_input();
if focus != self.text_focus {
let capturing = app.recording_shortcut.is_some();
if focus != self.text_focus || capturing != self.capturing {
self.text_focus = focus;
self.capturing = capturing;
for it in self.items.values() {
let Some(sc) = it.shortcut.as_deref() else { continue };
if it.accel.is_none() || !yields_to_text(sc) {
if it.accel.is_none() {
continue;
}
let accel = if focus { None } else { it.accel };
let accel = if capturing || (focus && yields_to_text(sc)) { None } else { it.accel };
let _ = match &it.handle {
Handle::Plain(h) => h.set_accelerator(accel),
Handle::Check(h) => h.set_accelerator(accel),
@@ -394,6 +432,7 @@ impl NativeMenu {
}
self.publish_shortcuts(app);
}
quit
}
}
@@ -401,6 +440,27 @@ impl NativeMenu {
mod tests {
use super::*;
/// The menu model's titles stay English, so only the language can tell the native menu to
/// rebuild its titles (and the macOS app menu): every language switch — also between two CJK
/// languages — must change the structure key.
#[test]
fn switching_language_rebuilds_native_menu_structure() {
use lightcraft_ui_egui::i18n::{Locale, set_language};
let app = LightcraftApp::new(lightcraft_engine::Session::with_demo(), Default::default());
set_language(Locale::En);
let bar = menu_bar(&app);
let mut seen = std::collections::BTreeSet::new();
for language in Locale::ALL {
set_language(*language);
assert!(seen.insert(structure_of(&bar)), "{language:?} shares another language's menu structure key");
}
set_language(Locale::ZhHant);
assert_eq!(label_text("File"), "檔案");
assert_eq!(label_text("Settings…"), "設定…");
assert_eq!(label_text("Quit LightCraft"), "結束 LightCraft");
set_language(Locale::En);
}
#[test]
fn shortcuts_map_to_accelerators() {
assert_eq!(accelerator("Cmd+Shift+Z"), Some(Accelerator::new(Modifiers::META | Modifiers::SHIFT, Code::KeyZ)));
@@ -428,4 +488,39 @@ mod tests {
assert!(accelerator(sc).is_some(), "{sc}");
}
}
/// Single keys shown in the menu bar (E, C, ⇧P, ratings…) never reach it on macOS, so egui must
/// keep handling them; ⌘ / ⌃ combinations and function keys are the menu bar's own.
#[test]
fn single_key_shortcuts_stay_with_egui() {
assert!(menu_delivers("Cmd+Shift+H") && menu_delivers("Ctrl+H") && menu_delivers("F2") && menu_delivers("Cmd+F11"));
assert!(!menu_delivers("E") && !menu_delivers("Shift+P") && !menu_delivers("Alt+Y") && !menu_delivers("1") && !menu_delivers("F"));
let app = LightcraftApp::new(lightcraft_engine::Session::with_demo(), Default::default());
fn all(n: &[MenuNode], out: &mut Vec<String>) {
for x in n {
match x {
MenuNode::Item { shortcut: Some(s), .. } if accelerator(s).is_some() => out.push(s.clone()),
MenuNode::Submenu { children, .. } => all(children, out),
_ => {}
}
}
}
let mut installed = Vec::new();
for (_, v) in menu_bar(&app) {
all(&v, &mut installed);
}
for sc in ["E", "C", "Shift+P", "Cmd+Shift+H", "F2"] {
assert!(installed.iter().any(|s| s == sc), "{sc} is in the menu bar");
}
let owned = owned_by_menu(installed.iter().map(String::as_str), false);
for sc in ["E", "C", "Shift+P", "0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "P", "U"] {
assert!(installed.iter().any(|s| s == sc), "{sc} is displayed in the menu");
assert!(!owned.contains(sc), "{sc} must be left to egui");
}
for sc in ["Cmd+Shift+H", "F2"] {
assert!(owned.contains(sc), "{sc} is run by the menu bar");
}
// while typing, the menu bar gives up F2 too (the text field has it)
assert!(!owned_by_menu(installed.iter().map(String::as_str), true).contains("F2"));
}
}
+9
View File
@@ -10,6 +10,9 @@ covers a whole directory), or if a licence file referenced in the Licence column
|---|---|---|---|---|---|---|
| `assets/fonts/Inter-*.ttf` (Regular, Medium, SemiBold) | Inter | Rasmus Andersson | https://github.com/rsms/inter | SIL Open Font License 1.1 (`assets/fonts/OFL-Inter.txt`) | 2026-09-30 | none |
| `assets/app-icon/` (`lightcraft.svg` and the PNG/ICNS/ICO/SVG files derived from it) | LightCraft app icon (lynx) | LightCraft project owner (drawn in ArtCraft) | original work (vectorised from the owner's drawing; see `assets/app-icon/README.md`) | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) | 2026-10-02 | vectorised to SVG; rendered to PNG/ICNS/ICO by `packaging/icons.sh` |
| `assets/camera-profiles/*.json` | Camera colour profiles (matrix + hue/saturation/value table), built into LightCraft | ILCE-7M4: Harald Hoyer | original work: fitted by `lightcraft-cli calibrate` to the contributor's own raw photos and their in-camera JPEGs (597 ILCE-7M4 photos, 2026); aggregate colour statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 | 2026-10-07 | none |
| `packaging/macos/dmg/` (`background.svg`, the `background.tiff` rendered from it by `generate.py`, and the `dmg-layout.DS_Store` layout file) | LightCraft macOS DMG window background | @XusBadia | original work for this repository, derived from the LightCraft app icon (`assets/app-icon/lightcraft.svg`, referenced via `<image>`, not copied); text is outlined Inter / JetBrains Mono glyph paths | MIT OR Apache-2.0 | 2026-10-08 | none (original); used for the Finder window of the macOS DMG, `packaging/macos/dmg/generate.py` renders it to `background.tiff` |
| `assets/camera-profiles/X-H2S.json`, `assets/camera-profiles/X-T4.json` | Fujifilm camera colour profiles (matrix + hue/saturation/value table), built into LightCraft | LightCraft contributors; source photographs supplied by Sebastian Pfluegelmeier | original work: fitted by `lightcraft-cli calibrate` to 201 X-H2S and 545 X-T4 RAFs and their in-camera JPEGs; aggregate colour statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 | 2026-10-08 | none |
| `crates/scenes/` (generated images) | Procedural demo photographs | LightCraft contributors | original work (generated at runtime, no source imagery) | MIT OR Apache-2.0 | 2026-09-30 | n/a |
| `crates/ui-egui/src/icons.rs` | UI icons drawn as vector paths in code | LightCraft contributors | original work | MIT OR Apache-2.0 | 2026-09-30 | n/a |
| `docs/images/*-tetons.jpg`, `docs/images/grid-pd.jpg` (screenshots containing it) | "The Tetons and the Snake River" | Ansel Adams (1942), U.S. National Archives | https://commons.wikimedia.org/wiki/File:Adams_The_Tetons_and_the_Snake_River.jpg | Public domain (U.S. federal government work) | 2026-09-30 | developed in LightCraft; shown inside app screenshots |
@@ -17,7 +20,13 @@ covers a whole directory), or if a licence file referenced in the Licence column
| `docs/images/*-earthrise.jpg`, `docs/images/grid-pd.jpg` | "Earthrise" (AS08-14-2383) | NASA / Bill Anders, Apollo 8 (1968) | https://commons.wikimedia.org/wiki/File:NASA-Apollo8-Dec24-Earthrise.jpg | Public domain (NASA) | 2026-09-30 | developed in LightCraft; shown inside app screenshots |
| `docs/images/*-blue-marble.jpg`, `docs/images/grid-pd.jpg` | "The Blue Marble" (AS17-148-22727) | NASA, Apollo 17 crew (1972) | https://commons.wikimedia.org/wiki/File:The_Earth_seen_from_Apollo_17.jpg | Public domain (NASA) | 2026-09-30 | developed in LightCraft; shown inside app screenshots |
| `docs/images/*.jpg` (UI) | LightCraft application screenshots | LightCraft contributors | original work (captured with `docs/showcase/`) | MIT OR Apache-2.0 | 2026-09-30 | n/a |
| `docs/brand/` (artcraft-logo and artcraft-mark, SVG and PNG) | ArtCraft wordmark and mark | ArtCraft Team | original work (https://getartcraft.com/) | ArtCraft trademark, see `docs/brand/LICENSE-brand.txt` (not open source) | 2026-10-01 | none |
| craft-fonts: optional build input, not files in this repository | BIZ UDPGothic (Regular, Bold), BIZ UDMincho Regular, Shippori Mincho Regular, Noto Sans CJK SC Regular | Morisawa Inc. / The BIZ UDGothic and BIZ UDMincho Project Authors; The Shippori Mincho Project Authors; Google and Adobe (Noto Sans CJK, maintainer-approved exception; see AGENTS.md) (per craft-fonts' attribution) | https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md | SIL Open Font License 1.1 (each font's OFL.txt in craft-fonts; release packages ship it as OFL-<family>.txt) | 2026-10-06 | none; native builds made with `CRAFT_FONTS_DIR` embed the manifest fonts (all official releases); the web build embeds BIZ UDPGothic Regular and ships Noto Sans CJK SC Regular as a separate font file |
Notes
- Fonts authored or published by Adobe (Source Sans/Serif/Code, Source Han, …) are not used, even though some are
OFL-licensed: the asset rule excludes anything from Adobe. Source Sans 3 was removed on 2026-09-30 and replaced by Inter.
- CJK fonts (BIZ UDPGothic, BIZ UDMincho, Shippori Mincho, Noto Sans CJK SC) are not in this repository. They live in
[storytold/craft-fonts](https://github.com/storytold/craft-fonts), with their licences and attribution in its
`ATTRIBUTION.md`, and are embedded only in builds made with the optional `CRAFT_FONTS_DIR` build input (all official
releases). Never commit font files here; add new fonts to craft-fonts (craftrules `standards/fonts.md`).
+1
View File
@@ -0,0 +1 @@
{"version":1,"model":"ILCE-7M4","files":597,"samples":2182876,"matrix":[[1.06975,-0.0709886,-0.000770444],[-0.0169822,1.08012,-0.0651276],[-0.114835,-0.601525,1.74804]],"hue_sat":{"hue_divisions":72,"sat_divisions":5,"val_divisions":5,"data":[[0.0,1.0,1.0],[-0.819008,1.14375,1.0],[-6.07446,1.10974,1.0],[-4.0478,1.04285,1.0],[-0.546764,1.00329,1.0],[0.0,1.0,1.0],[-1.0412,1.10597,1.0],[-7.22645,1.05878,1.0],[-7.30635,0.989387,1.0],[-5.18589,0.974522,1.0],[0.0,1.0,1.0],[-1.02364,1.06303,1.0],[-6.94669,1.01579,1.0],[-7.47614,0.955435,1.0],[-7.02579,0.940854,1.0],[0.0,1.0,1.0],[-0.29852,1.04349,1.0],[-6.37202,1.00021,1.0],[-7.52098,0.935457,1.0],[-7.45702,0.922239,1.0],[0.0,1.0,1.0],[0.367464,1.03266,1.0],[-4.96951,0.996822,1.0],[-7.41706,0.948709,1.0],[-8.04735,0.91672,1.0],[0.0,1.0,1.0],[0.0388419,1.02587,1.0],[-4.47266,1.01282,1.0],[-7.26834,0.987279,1.0],[-9.02713,0.921453,1.0],[0.0,1.0,1.0],[-0.297476,1.02333,1.0],[-4.6323,1.00993,1.0],[-8.57248,0.962973,1.0],[-12.2375,0.872088,1.0],[0.0,1.0,1.0],[-0.785867,1.02786,1.0],[-4.73124,1.00876,1.0],[-9.09856,0.955343,1.0],[-12.4324,0.875472,1.0],[0.0,1.0,1.0],[-0.444072,1.03863,1.0],[-3.94773,1.01784,1.0],[-7.97996,0.975996,1.0],[-10.673,0.917401,1.0],[0.0,1.0,1.0],[1.22651,1.0559,1.0],[-1.89459,1.02934,1.0],[-5.54545,0.993046,1.0],[-8.48877,0.941628,1.0],[0.0,1.0,1.0],[3.24516,1.08292,1.0],[1.20522,1.0577,1.0],[-0.826224,1.01668,1.0],[-3.71884,0.946733,1.0],[0.0,1.0,1.0],[4.68802,1.12039,1.0],[3.23147,1.08619,1.0],[1.84476,1.04,1.0],[0.0970375,0.962298,1.0],[0.0,1.0,1.0],[5.36041,1.16313,1.0],[4.5937,1.10933,1.0],[3.83314,1.05056,1.0],[3.122,0.959876,1.0],[0.0,1.0,1.0],[5.22222,1.17824,1.0],[5.09129,1.1142,1.0],[4.99294,1.04813,1.0],[5.01485,0.957528,1.0],[0.0,1.0,1.0],[4.34382,1.18446,1.0],[4.9457,1.11351,1.0],[5.56715,1.04462,1.0],[6.27579,0.961892,1.0],[0.0,1.0,1.0],[3.5428,1.18985,1.0],[4.9116,1.11394,1.0],[6.11507,1.04098,1.0],[7.03638,0.964331,1.0],[0.0,1.0,1.0],[2.92641,1.20663,1.0],[4.57132,1.1281,1.0],[5.96964,1.04895,1.0],[6.98579,0.96261,1.0],[0.0,1.0,1.0],[2.07123,1.23257,1.0],[3.51736,1.15454,1.0],[4.85051,1.06995,1.0],[5.97805,0.971597,1.0],[0.0,1.0,1.0],[0.869122,1.26762,1.0],[2.22009,1.19021,1.0],[3.54297,1.10536,1.0],[4.31783,1.00254,1.0],[0.0,1.0,1.0],[-0.409355,1.31042,1.0],[0.778039,1.23845,1.0],[1.87845,1.15455,1.0],[1.53759,1.02961,1.0],[0.0,1.0,1.0],[-1.82067,1.35265,1.0],[-0.910751,1.28551,1.0],[-0.100378,1.18585,1.0],[0.0425375,1.01463,1.0],[0.0,1.0,1.0],[-3.26659,1.391,1.0],[-2.50117,1.33263,1.0],[-1.55789,1.18102,1.0],[-0.0853484,1.00594,1.0],[0.0,1.0,1.0],[-4.92446,1.43632,1.0],[-3.92066,1.35798,1.0],[-1.66274,1.11605,1.0],[-0.0358972,1.00178,1.0],[0.0,1.0,1.0],[-5.82474,1.46137,1.0],[-4.22466,1.34716,1.0],[-0.64525,1.04813,1.0],[-0.00357121,1.00023,1.0],[0.0,1.0,1.0],[-7.0672,1.47242,1.0],[-5.38667,1.32663,1.0],[-0.553542,1.02668,1.0],[-0.00188478,1.00008,1.0],[0.0,1.0,1.0],[-8.37522,1.43136,1.0],[-6.13631,1.28174,1.0],[-0.515332,1.01808,1.0],[-0.0015762,1.00005,1.0],[0.0,1.0,1.0],[-10.5196,1.39935,1.0],[-7.49316,1.24442,1.0],[-0.517621,1.01282,1.0],[-0.001445,1.00003,1.0],[0.0,1.0,1.0],[-10.5952,1.3534,1.0],[-7.81548,1.20414,1.0],[-0.63473,1.0111,1.0],[-0.00208267,1.00003,1.0],[0.0,1.0,1.0],[-10.879,1.32051,1.0],[-8.3374,1.18517,1.0],[-0.91817,1.01139,1.0],[-0.00474972,1.00003,1.0],[0.0,1.0,1.0],[-11.8094,1.29525,1.0],[-9.18818,1.17522,1.0],[-0.968552,1.01157,1.0],[-0.00411555,1.00003,1.0],[0.0,1.0,1.0],[-12.2038,1.27777,1.0],[-8.97374,1.15856,1.0],[-0.751018,1.00872,1.0],[-0.00250012,1.00002,1.0],[0.0,1.0,1.0],[-12.0564,1.2709,1.0],[-8.38288,1.15926,1.0],[-0.643806,1.00878,1.0],[-0.00202184,1.00002,1.0],[0.0,1.0,1.0],[-12.8403,1.25618,1.0],[-8.85664,1.14517,1.0],[-0.646111,1.00756,1.0],[-0.00195373,1.00001,1.0],[0.0,1.0,1.0],[-13.2604,1.25094,1.0],[-8.74013,1.11989,1.0],[-0.556009,1.00457,1.0],[-0.00173404,1.00001,1.0],[0.0,1.0,1.0],[-13.3939,1.24388,1.0],[-8.23916,1.11659,1.0],[-0.376536,1.00355,1.0],[-0.000916476,1.00001,1.0],[0.0,1.0,1.0],[-13.6578,1.24212,1.0],[-7.97883,1.11035,1.0],[-0.409859,1.00287,1.0],[-0.0018203,1.0,1.0],[0.0,1.0,1.0],[-15.7225,1.22414,1.0],[-10.2345,1.10217,1.0],[-0.968177,1.00427,1.0],[-0.00650675,1.00001,1.0],[0.0,1.0,1.0],[-16.1557,1.16423,1.0],[-12.721,1.06603,1.0],[-2.02337,1.00334,1.0],[-0.0134173,1.00001,1.0],[0.0,1.0,1.0],[-17.253,1.09714,1.0],[-15.785,1.02184,1.0],[-4.02264,0.996649,1.0],[-0.0279292,0.999956,1.0],[0.0,1.0,1.0],[-17.6206,1.04331,1.0],[-16.2039,0.99401,1.0],[-4.09728,0.990246,1.0],[-0.0269801,0.999896,1.0],[0.0,1.0,1.0],[-17.2302,0.987569,1.0],[-15.1656,0.964114,1.0],[-3.21398,0.983725,1.0],[-0.0202065,0.999807,1.0],[0.0,1.0,1.0],[-16.2184,0.943678,1.0],[-13.4633,0.938141,1.0],[-3.40639,0.971571,1.0],[-0.00923182,0.999563,1.0],[0.0,1.0,1.0],[-14.5653,0.909699,1.0],[-11.6873,0.915733,1.0],[-5.4654,0.942337,1.0],[-0.124331,0.998194,1.0],[0.0,1.0,1.0],[-11.5122,0.906891,1.0],[-9.31547,0.923874,1.0],[-6.12222,0.939832,1.0],[-0.337538,0.995963,1.0],[0.0,1.0,1.0],[-7.77619,0.92294,1.0]Line truncated
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+4
View File
@@ -0,0 +1,4 @@
allow-unwrap-in-tests = true
allow-expect-in-tests = true
allow-panic-in-tests = true
allow-indexing-slicing-in-tests = true
+466
View File
@@ -0,0 +1,466 @@
{
"schema": 1,
"repo": "storytold/lightcraft",
"branch": "main",
"head": "2694d7089fbf4ada62c6738a7c62fecdfd5711fb",
"head_date": "2026-10-08T00:53:19Z",
"commits": 424,
"unattributed_commits": 0,
"contributors": [
{
"login": "echelon",
"people_entry": true,
"github_name": null,
"prs": 71,
"commits": 330,
"lines_added": 125623,
"lines_deleted": 8643,
"binary_added": 38,
"binary_deleted": 8,
"first_commit": "2026-09-30T14:04:12Z",
"last_commit": "2026-10-08T00:32:34Z"
},
{
"login": "leipsfur",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 131,
"lines_deleted": 5,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-02T15:51:58Z",
"last_commit": "2026-10-02T15:51:58Z"
},
{
"login": "GlemNL",
"people_entry": false,
"github_name": null,
"prs": 9,
"commits": 13,
"lines_added": 1233,
"lines_deleted": 96,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-02T21:16:29Z",
"last_commit": "2026-10-04T09:42:39Z"
},
{
"login": "storybored379",
"people_entry": false,
"github_name": null,
"prs": 7,
"commits": 7,
"lines_added": 1194,
"lines_deleted": 82,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-04T00:28:49Z",
"last_commit": "2026-10-06T06:51:43Z"
},
{
"login": "midasdf",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 424,
"lines_deleted": 25,
"binary_added": 1,
"binary_deleted": 0,
"first_commit": "2026-10-05T16:07:26Z",
"last_commit": "2026-10-05T16:07:26Z"
},
{
"login": "voshart",
"people_entry": false,
"github_name": null,
"prs": 3,
"commits": 12,
"lines_added": 2481,
"lines_deleted": 1389,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-05T16:23:27Z",
"last_commit": "2026-10-07T16:07:24Z"
},
{
"login": "arata-1972",
"people_entry": false,
"github_name": null,
"prs": 2,
"commits": 4,
"lines_added": 2892,
"lines_deleted": 726,
"binary_added": 2,
"binary_deleted": 0,
"first_commit": "2026-10-05T20:14:35Z",
"last_commit": "2026-10-05T22:16:15Z"
},
{
"login": "benlukka",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 49,
"lines_deleted": 4,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-06T11:18:03Z",
"last_commit": "2026-10-06T11:18:03Z"
},
{
"login": "kywk",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 72,
"lines_deleted": 8,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-06T20:21:14Z",
"last_commit": "2026-10-06T20:21:14Z"
},
{
"login": "treyreynolds",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 159,
"lines_deleted": 27,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-06T22:21:46Z",
"last_commit": "2026-10-06T22:21:46Z"
},
{
"login": "kovalskiguinther",
"people_entry": false,
"github_name": null,
"prs": 0,
"commits": 1,
"lines_added": 4328,
"lines_deleted": 17,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-06T23:15:04Z",
"last_commit": "2026-10-06T23:15:04Z"
},
{
"login": "cruzer45",
"people_entry": false,
"github_name": null,
"prs": 2,
"commits": 3,
"lines_added": 658,
"lines_deleted": 26,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T01:30:23Z",
"last_commit": "2026-10-07T03:53:09Z"
},
{
"login": "coygeek",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 4,
"lines_deleted": 0,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T02:27:15Z",
"last_commit": "2026-10-07T02:27:15Z"
},
{
"login": "lzjtugjc",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 1895,
"lines_deleted": 294,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T03:34:33Z",
"last_commit": "2026-10-07T03:34:33Z"
},
{
"login": "itsmunzir",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 46,
"lines_deleted": 30,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T07:40:55Z",
"last_commit": "2026-10-07T07:40:55Z"
},
{
"login": "haraldh",
"people_entry": false,
"github_name": null,
"prs": 2,
"commits": 5,
"lines_added": 1163,
"lines_deleted": 141,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T07:54:27Z",
"last_commit": "2026-10-07T13:16:08Z"
},
{
"login": "sakamoto824",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 2,
"lines_added": 3649,
"lines_deleted": 167,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T09:32:12Z",
"last_commit": "2026-10-07T10:54:19Z"
},
{
"login": "anarion80",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 402,
"lines_deleted": 2,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T11:54:51Z",
"last_commit": "2026-10-07T11:54:51Z"
},
{
"login": "radialmonster",
"people_entry": false,
"github_name": null,
"prs": 3,
"commits": 19,
"lines_added": 2630,
"lines_deleted": 259,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T12:33:51Z",
"last_commit": "2026-10-08T00:46:10Z"
},
{
"login": "bogusyogi",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 434,
"lines_deleted": 29,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T17:06:00Z",
"last_commit": "2026-10-07T17:06:00Z"
},
{
"login": "nledford",
"people_entry": false,
"github_name": null,
"prs": 4,
"commits": 8,
"lines_added": 525,
"lines_deleted": 60,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T17:33:38Z",
"last_commit": "2026-10-07T22:49:00Z"
},
{
"login": "MusseJusse",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 34,
"lines_deleted": 3,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T18:38:23Z",
"last_commit": "2026-10-07T18:38:23Z"
},
{
"login": "Ndsanes",
"people_entry": false,
"github_name": null,
"prs": 2,
"commits": 2,
"lines_added": 332,
"lines_deleted": 86,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T19:08:28Z",
"last_commit": "2026-10-07T19:08:33Z"
},
{
"login": "RicSchonfelder",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 1439,
"lines_deleted": 6,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T20:03:20Z",
"last_commit": "2026-10-07T20:03:20Z"
},
{
"login": "Gremiger",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 19,
"lines_deleted": 0,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T21:43:23Z",
"last_commit": "2026-10-07T21:43:23Z"
},
{
"login": "DWAA1660",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 54,
"lines_deleted": 12,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T22:08:01Z",
"last_commit": "2026-10-07T22:08:01Z"
},
{
"login": "agorevski",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 68,
"lines_deleted": 11,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T22:18:44Z",
"last_commit": "2026-10-07T22:18:44Z"
},
{
"login": "andrewyao",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 1,
"lines_deleted": 1,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-08T00:53:19Z",
"last_commit": "2026-10-08T00:53:19Z"
},
{
"login": "GuintherKovalski",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 0,
"lines_added": 0,
"lines_deleted": 0,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": null,
"last_commit": null
},
{
"login": "XRTce",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 0,
"lines_added": 0,
"lines_deleted": 0,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": null,
"last_commit": null
}
],
"models": [
{
"id": "anthropic/claude-opus/5.5",
"company": "Anthropic",
"model": "Claude Opus",
"version": "5.5",
"commits": 364,
"lines_added": 131810,
"lines_deleted": 9367
},
{
"id": "anthropic/claude-sonnet/5.5",
"company": "Anthropic",
"model": "Claude Sonnet",
"version": "5.5",
"commits": 31,
"lines_added": 3941,
"lines_deleted": 1543
},
{
"id": "openai/codex",
"company": "OpenAI",
"model": "Codex",
"version": "",
"commits": 14,
"lines_added": 8341,
"lines_deleted": 995
},
{
"id": "anthropic/claude",
"company": "Anthropic",
"model": "Claude",
"version": "",
"commits": 2,
"lines_added": 1304,
"lines_deleted": 192
},
{
"id": "anthropic/claude-sonnet/5",
"company": "Anthropic",
"model": "Claude Sonnet",
"version": "5",
"commits": 2,
"lines_added": 298,
"lines_deleted": 2
},
{
"id": "anthropic/claude-opus/4.8",
"company": "Anthropic",
"model": "Claude Opus",
"version": "4.8",
"commits": 1,
"lines_added": 1633,
"lines_deleted": 3
},
{
"id": "deepseek/deepseek/v3.2",
"company": "DeepSeek",
"model": "DeepSeek",
"version": "V3.2",
"commits": 1,
"lines_added": 1895,
"lines_deleted": 294
}
]
}
+29
View File
@@ -0,0 +1,29 @@
# Contributor names for the About window (craftrules standards/contributors.md).
#
# Everyone who contributes is credited by GitHub username automatically. Anything more is up to
# you: add YOUR OWN entry below, in your own PR (commit it yourself). Never add someone else; a
# maintainer may add a person only when that person asked, recording where in consent_source.
#
# Easiest: python3 ../../craftrules/scripts/contributors.py --add-me . \
# [--real-name "Ada Lovelace"] [--display-name "Ada"] [--sync-github-name]
#
# Or copy this block, replacing your-github-username (every field is optional):
#
# [people.your-github-username]
# real_name = "Your Name" # shown when the viewer picks "Real name"
# display_name = "Nickname" # shown when the viewer picks "Display name"
# sync_github_name = true # or: use the name on your public GitHub profile as display name
#
# To remove your names, delete your entry (or set fields empty) in a PR. The next
# contributors.py run, and the next build, drop them.
[people.echelon]
real_name = "Brandon Thomas"
display_name = "Brandon Thomas"
sync_github_name = false
[people.rock-meister]
sync_github_name = true
[people.sanyamsingh001]
display_name = "sanyamsingh001"
+2
View File
@@ -10,11 +10,13 @@ workspace = true
[dependencies]
lightcraft-develop = { workspace = true }
lightcraft-meta = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
thiserror = { workspace = true }
crc32fast = { workspace = true }
log = { workspace = true }
web-time = { workspace = true }
[dev-dependencies]
proptest = { workspace = true }
+216
View File
@@ -0,0 +1,216 @@
//! Catalog persistence costs on large synthetic catalogs (issue #37): log append, compaction
//! (snapshot serialise / write+fsync / log reset), synchronous and on the background worker, load,
//! and a catalog clone.
//!
//! ```text
//! cargo run --release -p lightcraft-catalog --example persist_bench -- <dir> [photos…]
//! ```
//!
//! `<dir>` is a scratch directory (wiped per size). Default sizes: 1000 10000 85000. 90 % of the
//! photos are Local browse records, the rest library photos with a few edits and history steps.
//! Peak memory: run under `/usr/bin/time -l` (macOS) / `-v` (Linux) with one size;
//! `PERSIST_BENCH_NO_LOAD=1` skips the final load, so the peak is building + compacting.
use std::sync::Arc;
use std::time::Instant;
use lightcraft_catalog::*;
use lightcraft_develop::DevelopSettings;
fn ms(t: Instant) -> f64 {
t.elapsed().as_secs_f64() * 1e3
}
fn photo(c: &mut Catalog, i: u64) -> Photo {
let id = c.alloc_photo_id();
let mut p = Photo::new(
id,
Source::File { path: format!("/Users/someone/Pictures/2026/09/{:05}/IMG_{i:05}.CR3", i / 500) },
&format!("IMG_{i:05}.CR3"),
"CR3",
6000,
4000,
"2026-09-30T12:00:00",
);
p.kind = MediaKind::Raw;
p.file_size = 25_000_000 + i;
p.captured = Some(format!("2026-09-{:02}T10:{:02}:{:02}", 1 + i % 28, i % 60, (i / 60) % 60));
p.content_hash = Some(format!("{:032x}", i.wrapping_mul(0x9e37_79b9_7f4a_7c15)));
p.meta.camera = "Camera Model X".into();
p.meta.lens = "24-70mm f/2.8".into();
p.meta.focal_mm = Some(35.0);
p.meta.aperture = Some(4.0);
p.meta.shutter = "1/250".into();
p.meta.iso = Some(400);
p.as_shot_wb = Some((5200.0, 3.0));
p.local = !i.is_multiple_of(10);
if p.local {
// as a browse catalogues it: import defaults, a baseline fingerprint
p.develop = Arc::new(p.import_defaults());
p.set_local_baseline();
}
if !p.local {
let mut d = DevelopSettings::default();
d.light.exposure = (i % 7) as f64 * 0.1;
let s = Arc::new(d);
p.develop = s.clone();
p.edited = Some("2026-09-30T12:00:00".into());
p.history = vec![HistoryStep { label: "Exposure".into(), settings: s }];
p.rating = (i % 6) as u8;
}
p
}
fn stats(mut v: Vec<f64>) -> String {
v.sort_by(f64::total_cmp);
let pick = |q: f64| v.get(((v.len() as f64 - 1.0) * q).round() as usize).copied().unwrap_or(0.0);
format!("median {:.2} ms, p95 {:.2} ms, max {:.2} ms", pick(0.5), pick(0.95), pick(1.0))
}
fn main() -> std::result::Result<(), Box<dyn std::error::Error>> {
let mut args = std::env::args().skip(1);
let dir = std::path::PathBuf::from(args.next().ok_or("usage: persist_bench <scratch dir> [photos…]")?);
let mut sizes: Vec<u64> = args.map(|a| a.parse()).collect::<std::result::Result<_, _>>()?;
if sizes.is_empty() {
sizes = vec![1000, 10_000, 85_000];
}
for n in sizes {
let _ = std::fs::remove_dir_all(&dir);
let (mut j, _, _) = Journal::open(Box::new(FsStore::open(&dir)?))?;
let mut cat = Catalog::new();
// bulk-add (as one import would) and log it, so the first compaction is an import's
for chunk in (0..n).collect::<Vec<_>>().chunks(500) {
let ops: Vec<Op> = chunk.iter().map(|i| Op::AddPhoto { photo: Box::new(photo(&mut cat, *i)) }).collect();
for op in &ops {
cat.apply(op.clone())?;
}
j.append(&ops)?;
}
j.snapshot(&cat)?;
let first = j.stats().last_snapshot;
println!("== {n} photos ({} local): snapshot {:.1} MB", cat.photos().filter(|p| p.local).count(), first.bytes as f64 / 1e6);
let t = Instant::now();
let copy = cat.clone();
println!(" catalog clone: {:.2} ms", ms(t));
drop(copy);
// single-op appends (a rating change each), as commands do
let ids: Vec<PhotoId> = cat.photos().map(|p| p.id).take(1000).collect();
let mut appends = Vec::new();
for k in 0..300usize {
let id = ids.get(k % ids.len().max(1)).copied().unwrap_or(PhotoId(1));
let op = Op::SetRating { id, rating: (k % 6) as u8 };
cat.apply(op.clone())?;
j.append(std::slice::from_ref(&op))?;
appends.push(j.stats().last_append_ms);
}
println!(" append (1 op, fsync): {}", stats(appends));
// compactions at the default threshold crossing
let mut totals = Vec::new();
for _ in 0..3 {
while !j.wants_snapshot() {
let op = Op::SetRating { id: ids.first().copied().unwrap_or(PhotoId(1)), rating: (j.seq() % 6) as u8 };
cat.apply(op.clone())?;
j.append(&[op])?;
}
j.snapshot(&cat)?;
let s = j.stats().last_snapshot;
println!(
" compaction ({} records): serialize {:.1} ms, write+sync {:.1} ms, log reset {:.1} ms, total {:.1} ms",
s.records, s.serialize_ms, s.write_sync_ms, s.reset_ms, s.total_ms
);
totals.push(s.total_ms);
}
println!(" compaction total: {}", stats(totals));
// the same at the threshold, written by the worker while single-op appends go on
let mut blocking = Vec::new();
for _ in 0..3 {
while !j.wants_snapshot() {
let op = Op::SetRating { id: ids.first().copied().unwrap_or(PhotoId(1)), rating: (j.seq() % 6) as u8 };
cat.apply(op.clone())?;
j.append(&[op])?;
}
let t = Instant::now();
j.snapshot_in_background(&cat)?;
let start_ms = ms(t);
let mut during = Vec::new();
while !j.snapshot_written() {
let op = Op::SetRating { id: ids.get(1).copied().unwrap_or(PhotoId(1)), rating: (j.seq() % 6) as u8 };
cat.apply(op.clone())?;
j.append(&[op])?;
during.push(j.stats().last_append_ms);
}
let t = Instant::now();
j.poll()?;
let finish_ms = ms(t);
let s = j.stats().last_snapshot;
println!(
" background compaction ({} records): start {start_ms:.1} ms + finish {finish_ms:.1} ms blocking; worker {:.1} ms; {} appends during it: {}",
s.records,
s.total_ms,
during.len(),
stats(during)
);
blocking.push(s.blocking_ms);
}
println!(" background compaction blocking: {}", stats(blocking));
// forget the untouched Local records (folders last browsed long ago; every 7th folder
// recently; every 50th Local record rated), then compact
let folders: std::collections::BTreeSet<String> = cat.photos().filter(|p| p.local).filter_map(|p| folder_of(p)).collect();
let mut ops: Vec<Op> = folders
.iter()
.enumerate()
.map(|(k, f)| Op::SetBrowsed {
folder: f.clone(),
at: Some(if k % 7 == 0 { "2026-09-30T00:00:00" } else { "2026-06-01T00:00:00" }.into()),
})
.collect();
ops.extend(cat.photos().filter(|p| p.local && p.id.0.is_multiple_of(50)).map(|p| Op::SetRating { id: p.id, rating: 2 }).collect::<Vec<_>>());
for op in &ops {
cat.apply(op.clone())?;
}
j.append(&ops)?;
j.snapshot(&cat)?;
let before = j.stats().last_snapshot.bytes;
let t = Instant::now();
let plan = cat.forget_local_plan("2026-10-05T00:00:00", DEFAULT_FORGET_DAYS, &|_| false);
let plan_ms = ms(t);
let t = Instant::now();
let ops = plan.ops();
for op in &ops {
cat.apply(op.clone())?;
}
j.append(&ops)?;
let apply_ms = ms(t);
j.snapshot(&cat)?;
let after = j.stats().last_snapshot.bytes;
println!(
" forget Local: {} of {} local records forgotten (kept: {} recent, {} touched, {} in use); plan {plan_ms:.1} ms, apply+append {apply_ms:.1} ms; snapshot {:.1} MB -> {:.1} MB",
plan.evict.len(),
plan.local,
plan.kept_recent,
plan.kept_touched,
plan.kept_in_use,
before as f64 / 1e6,
after as f64 / 1e6
);
drop(j);
if std::env::var_os("PERSIST_BENCH_NO_LOAD").is_some() {
// peak memory of building + compacting only
continue;
}
let t = Instant::now();
let (_, loaded, _) = Journal::open(Box::new(FsStore::open(&dir)?))?;
println!(" open (load snapshot + replay): {:.1} ms", ms(t));
if loaded.len() != cat.len() {
return Err("reloaded catalog differs".into());
}
}
let _ = std::fs::remove_dir_all(&dir);
Ok(())
}
@@ -0,0 +1,8 @@
# Seeds for failure cases proptest has generated in the past. It is
# automatically read and these particular cases re-run before any
# novel cases are generated.
#
# It is recommended to check this file in to source control so that
# everyone who runs the test benefits from these saved cases.
cc 0227748e171237a13833c9175fe02c7fe6ff332266798961a6a43ec37b81d1b4 # shrinks to actions = [(3, 0), (0, 0), (4, 0), (0, 0)]
cc 0b0ab2531bd46fbe261b55ecffebea9d3e8949900ab7bc33fb27ca19f5cd7b8b # shrinks to actions = [(4, 443), (3, 474), (2, 27), (5, 858), (5, 466), (0, 901), (4, 907), (3, 839), (0, 630), (7, 578), (4, 13), (2, 361), (6, 136), (6, 12), (6, 223), (1, 394), (6, 78), (5, 837), (6, 765), (4, 207), (2, 185), (2, 583), (2, 56), (5, 430), (1, 702), (1, 547), (5, 552), (4, 762), (0, 443), (0, 851), (7, 240), (4, 438), (3, 599), (6, 70), (4, 694), (1, 327), (7, 859), (6, 814), (4, 336), (3, 438), (0, 131), (0, 831), (3, 18), (2, 963), (6, 10), (3, 114)]
+19 -10
View File
@@ -128,13 +128,15 @@ pub fn display_time(iso: &str) -> String {
/// Header text for a group key.
pub fn group_label(key: &str) -> String {
let month = |k: &str| k.get(5..7).and_then(|m| m.parse::<usize>().ok()).filter(|m| (1..=12).contains(m)).map(|m| MONTHS[m - 1]);
let month = |k: &str| k.get(5..7).and_then(|m| m.parse::<usize>().ok()).and_then(|m| MONTHS.get(m.wrapping_sub(1)));
// `get` rather than `[..]`: a non-ASCII key must not split a character
let year = key.get(..4).unwrap_or(key);
match key.len() {
10 => match (weekday(key), month(key), key[8..10].parse::<u32>().ok()) {
(Some(w), Some(m), Some(d)) => format!("{w}, {d} {m} {}", &key[..4]),
10 => match (weekday(key), month(key), key.get(8..10).and_then(|d| d.parse::<u32>().ok())) {
(Some(w), Some(m), Some(d)) => format!("{w}, {d} {m} {year}"),
_ => key.to_string(),
},
7 => month(key).map(|m| format!("{m} {}", &key[..4])).unwrap_or_else(|| key.to_string()),
7 => month(key).map(|m| format!("{m} {year}")).unwrap_or_else(|| key.to_string()),
4 => key.to_string(),
_ => "Unknown Date".into(),
}
@@ -153,7 +155,7 @@ impl Catalog {
let date_of = |id: &PhotoId| -> String {
let Some(p) = self.photo(*id) else { return String::new() };
let d = match key {
SortKey::CaptureDate => Some(p.date()),
SortKey::CaptureDate => p.captured.as_deref(),
SortKey::ImportDate => Some(p.imported.as_str()),
SortKey::EditDate => p.edited.as_deref(),
_ => None,
@@ -196,6 +198,9 @@ mod tests {
assert_eq!(group_label("2026-02"), "February 2026");
assert_eq!(group_label("2026"), "2026");
assert_eq!(group_label(""), "Unknown Date");
// non-ASCII keys of a date's length used to be sliced mid-character
assert_eq!(group_label("2024-01é0"), "2024-01é0");
assert_eq!(group_label("202é01"), "January 202é01");
assert_eq!(shift_iso("2026-12-31T23:30:00", 3600).as_deref(), Some("2027-01-01T00:30:00"));
assert_eq!(shift_iso("2026-03-01T00:00:00.25+02:00", -1).as_deref(), Some("2026-02-28T23:59:59.25+02:00"));
assert_eq!(shift_iso("nope", 5), None);
@@ -217,7 +222,7 @@ mod tests {
}
let shape = |r: &[DateRun]| r.iter().map(|r| (r.key.clone(), r.start, r.count)).collect::<Vec<_>>();
let day = c.date_runs(&ids, SortKey::CaptureDate, GroupBy::Day);
// the undated photo falls back to its import date
// the undated photo is grouped under Unknown Date
assert_eq!(
shape(&day),
vec![
@@ -225,16 +230,20 @@ mod tests {
("2026-09-29".into(), 2, 1),
("2026-08-01".into(), 3, 1),
("2025-12-31".into(), 4, 1),
("2026-10-01".into(), 5, 1)
(String::new(), 5, 1)
]
);
assert_eq!(day[0].label, "Wednesday, 30 September 2026");
assert_eq!(day[4].label, "Unknown Date");
assert_eq!(c.date_runs(&ids, SortKey::CaptureDate, GroupBy::Auto), day);
assert_eq!(
shape(&c.date_runs(&ids[..5], SortKey::CaptureDate, GroupBy::Month)),
vec![("2026-09".into(), 0, 3), ("2026-08".into(), 3, 1), ("2025-12".into(), 4, 1)]
shape(&c.date_runs(&ids, SortKey::CaptureDate, GroupBy::Month)),
vec![("2026-09".into(), 0, 3), ("2026-08".into(), 3, 1), ("2025-12".into(), 4, 1), (String::new(), 5, 1)]
);
assert_eq!(
shape(&c.date_runs(&ids, SortKey::CaptureDate, GroupBy::Year)),
vec![("2026".into(), 0, 4), ("2025".into(), 4, 1), (String::new(), 5, 1)]
);
assert_eq!(shape(&c.date_runs(&ids[..5], SortKey::CaptureDate, GroupBy::Year)), vec![("2026".into(), 0, 4), ("2025".into(), 4, 1)]);
assert!(c.date_runs(&ids, SortKey::FileName, GroupBy::Day).is_empty());
assert!(c.date_runs(&ids, SortKey::CaptureDate, GroupBy::None).is_empty());
// edit date: unedited photos have no date
+226
View File
@@ -0,0 +1,226 @@
//! The library's folder tree: where on disk the imported photos live.
//!
//! A *folder* here is a folder holding at least one photo of the library (imported, not deleted,
//! not merely browsed in Local; demo scenes have no folder). The tree lists those folders and the
//! ones above them, never a folder that holds nothing of the library, with how many photos each
//! holds — its subfolders included, like a year in By Date counts its months.
//!
//! Every folder is a row of its own, the ones above the imported photos included
//! (`/Users` → `me` → `Pictures`), so a row is exactly the folder it names. A folder spelled two
//! ways (`/` or `\`, repeated or trailing separators, `.`) is one folder (see
//! [`crate::query::folder_key`]).
//!
//! Each disk is a *volume*, a top-level row of its own: a drive letter (`D:`), a Windows share
//! (`//server/share`), a mounted disk or share (macOS `/Volumes/tokyo`, Linux `/media/me/usb`,
//! `/mnt/disk`, `/run/user/1000/gvfs/smb-share:…`, `/net/host`) or, for everything else, the
//! startup disk (`/`). Only the spelling of the stored paths tells them apart: no disk is
//! touched, so a share that is asleep or gone lists and counts like one that is there, and the
//! sidebar never waits on the network.
//!
//! Limits of reading only the stored spelling of a path: file names are compared as written
//! (Windows folds case, other systems do not), so on a case-insensitive disk `/Volumes/tokyo` and
//! `/Volumes/TOKYO` are two volumes here; and one place reached by two spellings
//! (`/Volumes/Macintosh HD/Users/me` and `/Users/me`, `/media/me/usb` and `/run/media/me/usb`) is
//! two rows. Each row is still consistent: choosing it shows exactly the photos it counts.
//!
//! This is the library's own view of its photos; browsing any folder on disk without importing
//! is Local's job (see [`crate::local`]).
use std::collections::{BTreeMap, HashMap};
use serde::Serialize;
use crate::query::{folder_key, folder_within, split_path};
use crate::{Catalog, Source};
/// Folder levels kept per photo. No real path comes near it; a deeper one is counted at its
/// ancestor this many levels down, which bounds the tree's depth (and so every walk of it)
/// whatever a catalog holds.
pub const MAX_DEPTH: usize = 64;
/// One row of the library's folder tree.
#[derive(Clone, Debug, PartialEq, Serialize)]
pub struct FolderNode {
/// What the row says: the folder's name; for a volume, its name (`nas`, `D:`,
/// `//server/share`, `/` for the startup disk).
pub name: String,
/// The folder the row stands for, written out from the names of the photos' paths (forward
/// slashes, no verbatim prefix); hand it back to [`crate::Filter::library_folder`] to see its
/// photos.
pub path: String,
/// Photos in this folder or any folder inside it.
pub count: usize,
/// Photos directly in this folder.
pub own: usize,
/// A disk rather than a folder in one; its `path` is where it is mounted.
pub volume: bool,
/// Whether choosing the row (`Filter::library_folder`) shows exactly its `count` photos. Not
/// so for the startup disk and for folders that hold other disks (`/Volumes`, `/mnt`…): their
/// path covers those disks' photos too, so choosing them is not offered.
pub selectable: bool,
pub children: Vec<FolderNode>,
}
/// A folder while the tree is being built (children by identity key).
struct Level {
name: String,
path: String,
own: usize,
volume: bool,
children: Vec<String>,
}
/// Where a stored folder path lies: its volume and the folder names below it.
struct Placed {
mount: String,
volume: String,
names: Vec<String>,
}
/// A gvfs mount's name made readable: `smb-share:server=tokyo,share=photos` → `photos on tokyo`;
/// any other (`sftp:host=nas`) as it is.
fn mount_name(n: &str) -> String {
let field = |key: &str| {
let rest = n.strip_prefix("smb-share:")?;
rest.split(',').find_map(|kv| kv.strip_prefix(key)?.strip_prefix('=')).filter(|v| !v.is_empty())
};
match (field("share"), field("server")) {
(Some(share), Some(server)) => format!("{share} on {server}"),
_ => n.to_string(),
}
}
/// The volume `dir` lies on and the folder names below it, read the way [`folder_key`] reads a
/// path (so a row and the filter that chooses it never disagree). `None` for a path that is not
/// absolute or names no share.
fn place(dir: &str) -> Option<Placed> {
let sp = split_path(dir, false);
let names: Vec<&str> = sp.parts.iter().map(String::as_str).collect();
let named = |mount: String, volume: &str, used: usize| (mount, volume.to_string(), used);
let (mount, volume, used) = if sp.prefix.starts_with("//") {
// `//server/share`: a server alone (no share) is no place to keep photos
if sp.prefix.ends_with('/') {
return None;
}
named(sp.prefix.clone(), &sp.prefix, 0)
} else if !sp.prefix.is_empty() {
named(sp.prefix.clone(), &sp.prefix, 0)
} else if sp.absolute {
match names.as_slice() {
["Volumes", n, ..] => named(format!("/Volumes/{n}"), n, 2),
["run", "media", u, n, ..] => named(format!("/run/media/{u}/{n}"), n, 4),
["media", u, n, ..] => named(format!("/media/{u}/{n}"), n, 3),
["mnt", n, ..] => named(format!("/mnt/{n}"), n, 2),
// Linux desktops mount network places per user, naming server and share
["run", "user", uid, "gvfs", n, ..] => named(format!("/run/user/{uid}/gvfs/{n}"), &mount_name(n), 5),
// NFS automounts: Linux `/net/<host>`, macOS `/Network/Servers/<host>`
["net", host, ..] => named(format!("/net/{host}"), host, 2),
["Network", "Servers", host, ..] => named(format!("/Network/Servers/{host}"), host, 3),
_ => named("/".to_string(), "/", 0),
}
} else {
return None;
};
Some(Placed { mount, volume, names: names.iter().skip(used).take(MAX_DEPTH).map(|n| n.to_string()).collect() })
}
/// The disk or share (`/Volumes/nas`, `D:`, `/` for the startup disk) a file-backed photo is on,
/// from its stored path alone.
pub fn volume_of(p: &crate::Photo) -> Option<String> {
let Source::File { path } = &p.source else { return None };
let dir = path.rfind(['/', '\\']).and_then(|i| path.get(..i.max(1)))?;
place(dir).map(|pl| folder_key(&pl.mount))
}
/// Whether `path` is a whole disk or share (`/`, `C:\`, `\\srv\share`, `/Volumes/nas`), not a
/// folder in one.
pub fn is_disk_root(path: &str) -> bool {
place(path).is_some_and(|p| p.names.is_empty())
}
/// Whether `path` is the startup disk's root (`/`), whose path also covers every other disk.
pub fn is_startup_disk(path: &str) -> bool {
place(path).is_some_and(|p| p.names.is_empty() && p.mount == "/")
}
/// A folder named by its last two names (`photos/travel`), so two folders called `travel` are
/// told apart; the path itself when it has no names (`/`).
pub fn folder_label(path: &str) -> String {
let names: Vec<&str> = path.split(['/', '\\']).filter(|n| !n.is_empty()).collect();
if names.is_empty() { path.to_string() } else { names.iter().skip(names.len().saturating_sub(2)).copied().collect::<Vec<_>>().join("/") }
}
impl Catalog {
/// The library's volumes with their folders and photo counts: volumes by name
/// (case-insensitive), each folder with its subfolders the same way.
pub fn folder_tree(&self) -> Vec<FolderNode> {
// photos per containing folder, as spelled (sorted, so the spelling that names a folder
// never depends on hash order)
let mut dirs: BTreeMap<&str, usize> = BTreeMap::new();
for p in self.photos().filter(|p| p.in_library()) {
if let Source::File { path } = &p.source
&& let Some(i) = path.rfind(['/', '\\'])
// a photo straight in the root has an empty folder part: the root itself
&& let Some(dir) = path.get(..i.max(1))
{
*dirs.entry(dir).or_default() += 1;
}
}
let mut levels: HashMap<String, Level> = HashMap::new();
let mut roots: Vec<String> = Vec::new();
for (dir, n) in dirs {
let Some(placed) = place(dir) else { continue };
let vkey = folder_key(&placed.mount);
if !levels.contains_key(&vkey) {
levels.insert(
vkey.clone(),
Level { name: placed.volume.clone(), path: placed.mount.clone(), own: 0, volume: true, children: Vec::new() },
);
roots.push(vkey.clone());
}
let mut prefix = if placed.mount == "/" { String::new() } else { placed.mount.clone() };
let mut parent = vkey;
for name in &placed.names {
prefix.push('/');
prefix.push_str(name);
let key = folder_key(&prefix);
if !levels.contains_key(&key) {
levels.insert(key.clone(), Level { name: name.clone(), path: prefix.clone(), own: 0, volume: false, children: Vec::new() });
if let Some(l) = levels.get_mut(&parent) {
l.children.push(key.clone());
}
}
parent = key;
}
if let Some(l) = levels.get_mut(&parent) {
l.own += n;
}
}
let mut tree: Vec<FolderNode> = roots.iter().filter_map(|k| build(k, &levels)).collect();
sort(&mut tree);
// a folder that holds another disk's mount covers that disk's photos too
let mounts: Vec<String> = tree.iter().filter(|v| v.path != "/").map(|v| v.path.clone()).collect();
mark_selectable(&mut tree, &mounts);
tree
}
}
/// The node for `key` with everything below it (at most [`MAX_DEPTH`] deep).
fn build(key: &str, levels: &HashMap<String, Level>) -> Option<FolderNode> {
let l = levels.get(key)?;
let mut children: Vec<FolderNode> = l.children.iter().filter_map(|k| build(k, levels)).collect();
sort(&mut children);
let count = l.own + children.iter().map(|c| c.count).sum::<usize>();
Some(FolderNode { name: l.name.clone(), path: l.path.clone(), count, own: l.own, volume: l.volume, selectable: true, children })
}
fn mark_selectable(nodes: &mut [FolderNode], mounts: &[String]) {
for n in nodes {
n.selectable = !(n.volume && n.path == "/") && (n.volume || !mounts.iter().any(|m| folder_within(m, &n.path)));
mark_selectable(&mut n.children, mounts);
}
}
fn sort(v: &mut [FolderNode]) {
v.sort_by(|a, b| a.name.to_lowercase().cmp(&b.name.to_lowercase()).then_with(|| a.name.cmp(&b.name)));
}
+486 -30
View File
@@ -1,8 +1,9 @@
//! Crash-safe catalog persistence: an append-only op log plus periodic snapshots.
//!
//! Files (in a [`Store`]):
//! - `catalog.snap` — `{"format":"lightcraft-catalog","version":1,"seq":N,"catalog":{…}}`, the
//! state after op `N`; replaced atomically (temp + fsync + rename).
//! - `catalog.snap` — `{"format":"lightcraft-catalog","version":V,"seq":N,"catalog":{…}}`, the
//! state after op `N` in catalog format `V` ([`VERSION`]); replaced atomically (temp + fsync +
//! rename).
//! - `catalog.log` — JSON lines, one per op applied after the snapshot:
//! `{"seq":N,"crc":C,"op":{…}}` where `C` is the CRC-32 of the op's JSON text. Appends are
//! fsynced before [`Journal::append`] returns.
@@ -10,10 +11,56 @@
//! Loading = snapshot + replay of the records with `seq > snapshot seq`. Recovery rules:
//! - a **torn final record** (crash mid-append: truncated line or bad CRC at the end) is dropped
//! and the file is cut back to the last good record, so later appends start on a clean line;
//! - a **failed append** (disk full, a dropped network share) may leave part of its batch in the
//! file: [`Journal::append`] cuts the log back to the last whole record before returning the
//! error (or, if that fails too, before the next append), so the retry — the same ops with the
//! same `seq`s — starts on a clean line. Logs written by builds without that cut-back hold the
//! fragment glued to the first retried record on one line; the loader recognises the whole,
//! CRC-valid record at the end of such a line and skips only the fragment;
//! - records already covered by the snapshot (crash between writing the snapshot and resetting
//! the log) are skipped;
//! - a bad record **followed by good ones** is real damage: replay stops there, the log is kept
//! as `catalog.log.damaged-<seq>`, and a fresh snapshot is written so the library stays usable.
//!
//! **Background compaction** ([`Journal::snapshot_in_background`], when the store offers a
//! [`Store::background_writer`]): a worker thread writes the snapshot of a copy of the catalog at
//! op `N` while appends go on (durably, as always) to the same log. Once the snapshot is durable,
//! [`Journal::poll`] atomically replaces the log with the records appended since the snapshot
//! started (`seq > N`, kept in memory). Every crash point is covered by the rules above:
//! - before the rename of `catalog.snap`: the old snapshot + the whole log (nothing was removed);
//! - between the rename and the log rewrite: the new snapshot + the whole log, whose records
//! `<= N` are skipped as stale;
//! - after the log rewrite: the new snapshot + exactly the records `> N`.
//!
//! At most one snapshot is in flight; a synchronous [`Journal::snapshot`] and dropping the journal
//! first wait for it, so an older snapshot can never replace a newer one.
//!
//! # Format versions
//!
//! The snapshot's `version` is the catalog format, and it covers the log beside it: the log has
//! no header of its own, and is only ever appended to under a snapshot of this build's format.
//!
//! | Version | Written by | Adds |
//! |---|---|---|
//! | (none) | early builds: a log without a snapshot | |
//! | 1 | up to v0.2.0 | |
//! | 2 | after v0.2.0 | `Op::SetBrowsed`, `Catalog.browsed`, `Photo.local_baseline` |
//! | 3 | albums by hand | `Op::SetAlbumOrder`, `Album.order` |
//!
//! Rules:
//! - **Bump [`VERSION`]** (and add a row above) in the change that adds an [`Op`] variant or a
//! serialized field to the catalog, a photo, an album, a stack or develop settings that an older
//! build would misread or silently drop. New fields must deserialize from older data
//! (`#[serde(default)]`), so newer builds keep reading every older format.
//! - **Opening an older format upgrades it**: right after loading, [`Journal::open`] writes a
//! snapshot in this build's format (also for a new library), before anything is appended — so a
//! library is never a mix, and older builds refuse it from then on instead of reading part of it.
//! [`LoadReport::upgraded_from`] says when that happened.
//! - **A newer format is refused, untouched**: a snapshot whose `version` is above [`VERSION`],
//! or a log record whose CRC matches (so it is exactly what was written) but whose op doesn't
//! parse, fails [`Journal::open`] with [`CatalogError::Newer`] before any file is modified — it
//! is never mistaken for a torn tail or damage. (Builds up to v0.2.0 refuse a v2 snapshot with
//! "unsupported format".)
use crate::store::Store;
use crate::{Catalog, CatalogError, Op, Result};
@@ -21,7 +68,9 @@ use crate::{Catalog, CatalogError, Op, Result};
pub const SNAPSHOT: &str = "catalog.snap";
pub const LOG: &str = "catalog.log";
const FORMAT: &str = "lightcraft-catalog";
const VERSION: u32 = 1;
/// The catalog format this build writes (and the newest it reads). See the module docs →
/// *Format versions*; bump it whenever an [`Op`] variant or a serialized field is added.
pub const VERSION: u32 = 3;
/// When [`Journal::wants_snapshot`] says it's time to compact the log.
#[derive(Clone, Copy, Debug)]
@@ -49,10 +98,75 @@ pub struct LoadReport {
pub failed: usize,
/// Bytes of a torn final record that were dropped.
pub torn_bytes: u64,
/// Partial records left mid-log by a failed append that was retried (written by builds that
/// didn't cut the log back after a failed write): skipped, the retried records were replayed.
pub torn_fragments: usize,
/// The damaged log was preserved under this name.
pub damaged: Option<String>,
/// No catalog files existed (a new library).
pub created: bool,
/// The library was in an older catalog format (`0` = before format versions: a log without a
/// snapshot) and was rewritten in this version's format ([`VERSION`]).
pub upgraded_from: Option<u32>,
}
/// Where persistence time goes (reported by `library.info` → `persistence`; printed to stderr
/// per write under `LIGHTCRAFT_PROFILE`). Times are wall-clock milliseconds on the calling
/// thread, i.e. how long the caller (the UI thread, for the app) was blocked.
#[derive(Clone, Copy, Debug, Default, PartialEq, serde::Serialize)]
#[serde(rename_all = "camelCase")]
pub struct PersistStats {
/// [`Journal::append`] calls that wrote something.
pub appends: u64,
/// Encode + write + `sync_data` of the last / slowest append.
pub last_append_ms: f64,
pub max_append_ms: f64,
/// Snapshots written (compactions, plus the ones on close / repair).
pub snapshots: u64,
/// The last snapshot, by stage.
pub last_snapshot: SnapshotTiming,
/// Total time of the slowest snapshot.
pub max_snapshot_ms: f64,
/// Longest time a snapshot blocked the caller (for background snapshots: copying the catalog
/// and starting the worker, plus rewriting the log when it is done).
pub max_blocking_ms: f64,
/// A background snapshot is being written.
pub snapshot_running: bool,
/// Background snapshots that failed (the log was kept whole; retried later).
pub failed_snapshots: u64,
}
/// One snapshot (compaction), by stage.
#[derive(Clone, Copy, Debug, Default, PartialEq, serde::Serialize)]
#[serde(rename_all = "camelCase")]
pub struct SnapshotTiming {
/// Serialising the catalog to JSON, streamed into the temp file (buffered writes included).
pub serialize_ms: f64,
/// Flushing, `sync_all` and the atomic rename of `catalog.snap`.
pub write_sync_ms: f64,
/// Resetting `catalog.log` (an atomic rewrite to empty, or to the records appended while a
/// background snapshot ran; fsynced).
pub reset_ms: f64,
/// Start to finish (for a background snapshot: until [`Journal::poll`] saw it done).
pub total_ms: f64,
/// How long the caller was blocked (equal to `total_ms` unless `background`).
pub blocking_ms: f64,
/// Written by a worker thread.
pub background: bool,
/// Size of `catalog.snap`.
pub bytes: u64,
/// Log records the snapshot compacted.
pub records: u64,
}
/// `LIGHTCRAFT_PROFILE` is set: print persistence timings to stderr.
fn profiling() -> bool {
static ON: std::sync::OnceLock<bool> = std::sync::OnceLock::new();
*ON.get_or_init(|| std::env::var_os("LIGHTCRAFT_PROFILE").is_some())
}
fn ms_since(t: web_time::Instant) -> f64 {
t.elapsed().as_secs_f64() * 1e3
}
pub struct Journal {
@@ -63,6 +177,29 @@ pub struct Journal {
log_records: u64,
log_bytes: u64,
pub policy: SnapshotPolicy,
stats: PersistStats,
/// The background snapshot in flight.
pending: Option<Pending>,
/// After a failed background snapshot: don't retry before the log has this many records.
retry_at_records: u64,
/// A failed append may have left part of its records after `log_bytes`, and cutting them off
/// failed too: the next append cuts the log back to `log_bytes` first.
tail_dirty: bool,
}
/// A background snapshot in flight.
struct Pending {
/// The snapshot holds the state after this op.
seq: u64,
/// Log bytes / records appended since it started (all `> seq`): the log once it's done.
tail: Vec<u8>,
tail_records: u64,
/// Log records it compacts.
records: u64,
started: web_time::Instant,
/// Time the caller spent starting it.
start_ms: f64,
worker: std::thread::JoinHandle<std::io::Result<SnapshotTiming>>,
}
fn io(e: std::io::Error) -> CatalogError {
@@ -76,24 +213,93 @@ pub fn encode_record(seq: u64, op: &Op) -> String {
format!("{{\"seq\":{seq},\"crc\":{crc},\"op\":{body}}}")
}
/// Parse one log line (without the newline). `None` if it's malformed or the CRC doesn't match.
/// Parse one log line (without the newline). `None` if it's malformed, the CRC doesn't match, or
/// the op is one this version doesn't know (see [`Journal::open`]).
pub fn decode_record(line: &str) -> Option<(u64, Op)> {
let rest = line.strip_prefix("{\"seq\":")?;
let (seq, rest) = rest.split_once(",\"crc\":")?;
let (crc, rest) = rest.split_once(",\"op\":")?;
let body = rest.strip_suffix('}')?;
let seq: u64 = seq.parse().ok()?;
let crc: u32 = crc.parse().ok()?;
if crc32fast::hash(body.as_bytes()) != crc {
return None;
match decode_line(line) {
Line::Record(seq, op) => Some((seq, op)),
Line::Newer(_) | Line::Bad => None,
}
serde_json::from_str(body).ok().map(|op| (seq, op))
}
/// One log line, decoded.
enum Line {
Record(u64, Op),
/// The CRC matches — the line is exactly what was written — but the op doesn't parse: it
/// was written by a newer LightCraft (an op or field this version doesn't know), not damaged.
Newer(u64),
/// Malformed or a CRC mismatch: torn or damaged.
Bad,
}
fn decode_line(line: &str) -> Line {
let parts = || {
let rest = line.strip_prefix("{\"seq\":")?;
let (seq, rest) = rest.split_once(",\"crc\":")?;
let (crc, rest) = rest.split_once(",\"op\":")?;
let body = rest.strip_suffix('}')?;
Some((seq.parse::<u64>().ok()?, crc.parse::<u32>().ok()?, body))
};
let Some((seq, crc, body)) = parts() else { return Line::Bad };
if crc32fast::hash(body.as_bytes()) != crc {
return Line::Bad;
}
match serde_json::from_str(body) {
Ok(op) => Line::Record(seq, op),
Err(_) => Line::Newer(seq),
}
}
/// The error for a library written by a newer LightCraft.
fn newer(what: String) -> CatalogError {
CatalogError::Newer(format!("{what}; this version reads catalog format v{VERSION} and older"))
}
/// A line that doesn't decode but ends with a whole record: a fragment of a failed append followed
/// by the record that retried it (the retry re-encodes the same ops with the same `seq`s). Returns
/// the fragment's length and the record, if that record continues the log (`seq <= last + 1`).
/// The record's CRC must match, so damage is never mistaken for it.
fn resync(line: &[u8], last: u64) -> Option<(usize, (u64, Op))> {
const START: &[u8] = b"{\"seq\":";
(1..line.len().saturating_sub(START.len() - 1)).filter(|&i| line[i..].starts_with(START)).find_map(|i| {
let rec = std::str::from_utf8(&line[i..]).ok().map(|l| l.trim_end_matches('\r')).and_then(decode_record)?;
(rec.0 <= last + 1).then_some((i, rec))
})
}
/// Write `catalog.snap` for the state after op `seq`, streaming the JSON into the store (no
/// whole-file string). The bytes are exactly
/// `{"format":"lightcraft-catalog","version":1,"seq":N,"catalog":<serde_json of the catalog>}\n`,
/// as before streaming. Fills in the serialise / write+sync times and the size.
fn write_snapshot(store: &mut dyn Store, seq: u64, catalog: &Catalog) -> std::io::Result<SnapshotTiming> {
let t0 = web_time::Instant::now();
let mut serialize_ms = 0.0;
let bytes = store.write_atomic_with(SNAPSHOT, &mut |w| {
use std::io::Write;
let t = web_time::Instant::now();
// serde_json emits many tiny writes: buffer them in a concrete writer (inlined), so only
// 1 MiB chunks go through the store's `dyn Write`
let mut b = std::io::BufWriter::with_capacity(1 << 20, w);
write!(b, "{{\"format\":\"{FORMAT}\",\"version\":{VERSION},\"seq\":{seq},\"catalog\":")?;
serde_json::to_writer(&mut b, catalog).map_err(std::io::Error::other)?;
b.write_all(b"}\n")?;
b.flush()?;
serialize_ms = ms_since(t);
Ok(())
})?;
Ok(SnapshotTiming { serialize_ms, write_sync_ms: (ms_since(t0) - serialize_ms).max(0.0), bytes, ..Default::default() })
}
/// Just the identification of `catalog.snap` (read before the catalog itself).
#[derive(serde::Deserialize)]
struct SnapFile {
struct SnapHeader {
format: String,
version: u32,
}
/// The whole `catalog.snap` (after [`SnapHeader`] was checked; `format` and `version` are ignored).
#[derive(serde::Deserialize)]
struct SnapFile {
seq: u64,
catalog: Catalog,
}
@@ -106,18 +312,34 @@ impl Journal {
let snap = store.read(SNAPSHOT).map_err(io)?;
let log = store.read(LOG).map_err(io)?;
report.created = snap.is_none() && log.is_none();
let (mut catalog, snapshot_seq) = match snap {
let (mut catalog, snapshot_seq, snapshot_version) = match snap {
Some(bytes) => {
let s: SnapFile = serde_json::from_slice(&bytes).map_err(|e| CatalogError::Corrupt(format!("{SNAPSHOT}: {e}")))?;
if s.format != FORMAT || s.version > VERSION {
return Err(CatalogError::Corrupt(format!("{SNAPSHOT}: unsupported format {} v{}", s.format, s.version)));
// the header first: a newer snapshot may not parse as this version's catalog
let h: SnapHeader = serde_json::from_slice(&bytes).map_err(|e| CatalogError::Corrupt(format!("{SNAPSHOT}: {e}")))?;
if h.format != FORMAT {
return Err(CatalogError::Corrupt(format!("{SNAPSHOT}: not a LightCraft catalog (format {:?})", h.format)));
}
(s.catalog, s.seq)
if h.version > VERSION {
return Err(newer(format!("{SNAPSHOT} is catalog format v{}", h.version)));
}
let s: SnapFile = serde_json::from_slice(&bytes).map_err(|e| CatalogError::Corrupt(format!("{SNAPSHOT}: {e}")))?;
(s.catalog, s.seq, Some(h.version))
}
None => (Catalog::new(), 0),
None => (Catalog::new(), 0, None),
};
report.snapshot_seq = snapshot_seq;
let mut j = Journal { store, seq: snapshot_seq, snapshot_seq, log_records: 0, log_bytes: 0, policy: SnapshotPolicy::default() };
let mut j = Journal {
store,
seq: snapshot_seq,
snapshot_seq,
log_records: 0,
log_bytes: 0,
policy: SnapshotPolicy::default(),
stats: PersistStats::default(),
pending: None,
retry_at_records: 0,
tail_dirty: false,
};
let log = log.unwrap_or_default();
// Scan records; `good_end` is the byte offset just past the last good record.
@@ -136,7 +358,22 @@ impl Journal {
good_end = next;
continue;
}
match line.and_then(decode_record) {
let mut record = line.and_then(decode_record);
if record.is_none()
&& let Some((at, r)) = resync(&log[pos..line_end], j.seq)
{
// the torn start of a failed append, then the retried records (older builds
// appended them right after the fragment)
log::warn!("catalog log: skipped a partial record ({at} bytes) left by a failed write");
report.torn_fragments += 1;
record = Some(r);
}
if record.is_none()
&& let Some(Line::Newer(seq)) = line.map(decode_line)
{
return Err(newer(format!("{LOG} record {seq} holds a change this version doesn't know")));
}
match record {
Some((seq, op)) if seq <= j.seq => {
// already in the snapshot
let _ = op;
@@ -152,7 +389,15 @@ impl Journal {
}
_ => {
// bad record (or a gap): torn tail if nothing good follows, else damage
let rest_has_good = log[next..].split(|b| *b == b'\n').any(|l| std::str::from_utf8(l).ok().and_then(decode_record).is_some());
let mut rest_has_good = false;
for l in log[next..].split(|b| *b == b'\n') {
match std::str::from_utf8(l).map(|l| decode_line(l.trim_end_matches('\r'))) {
Ok(Line::Record(..)) => rest_has_good = true,
// refused before anything is modified
Ok(Line::Newer(seq)) => return Err(newer(format!("{LOG} record {seq} holds a change this version doesn't know"))),
Ok(Line::Bad) | Err(_) => {}
}
}
if rest_has_good {
damaged_at = Some(pos);
}
@@ -176,10 +421,34 @@ impl Journal {
report.torn_bytes = (log.len() - good_end) as u64;
log::warn!("catalog log: dropped a torn final record ({} bytes)", report.torn_bytes);
j.store.truncate(LOG, good_end as u64).map_err(io)?;
j.log_bytes = good_end as u64;
} else if needs_newline {
j.store.append(LOG, b"\n").map_err(io)?;
j.log_bytes = good_end as u64 + 1;
} else {
j.log_bytes = good_end as u64;
}
}
// Bring an older (or new, or snapshot-less) library to this version's format before
// anything is appended: a snapshot in this format makes older builds refuse the library
// instead of misreading ops or dropping fields they don't know.
if report.created {
// tiny: written directly, not counted as a compaction
let empty = format!("{{\"format\":\"{FORMAT}\",\"version\":{VERSION},\"seq\":0,\"catalog\":{}}}\n", catalog.to_snapshot());
if let Err(e) = j.store.write_atomic(SNAPSHOT, empty.as_bytes()) {
log::warn!("catalog: can't write the new library's snapshot: {e}");
}
} else if report.damaged.is_none() && snapshot_version != Some(VERSION) {
report.upgraded_from = Some(snapshot_version.unwrap_or(0));
match j.snapshot(&catalog) {
Ok(()) => {
if let Some(v) = report.upgraded_from {
log::info!("catalog: upgraded from format v{v} to v{VERSION}");
}
}
// nothing lost: the old files are intact and still load; retried on next open
Err(e) => log::warn!("catalog: can't write the format v{VERSION} snapshot: {e}"),
}
j.log_bytes = good_end as u64;
}
catalog.revision = 0;
Ok((j, catalog, report))
@@ -190,6 +459,7 @@ impl Journal {
if ops.is_empty() {
return Ok(());
}
let t0 = web_time::Instant::now();
let mut buf = String::new();
let mut seq = self.seq;
for op in ops {
@@ -197,29 +467,202 @@ impl Journal {
buf.push_str(&encode_record(seq, op));
buf.push('\n');
}
self.store.append(LOG, buf.as_bytes()).map_err(io)?;
if self.tail_dirty {
self.store.truncate(LOG, self.log_bytes).map_err(io)?;
self.tail_dirty = false;
}
if let Err(e) = self.store.append(LOG, buf.as_bytes()) {
// Part of the batch may be in the file. Cut it back to the last whole record, so the
// retry (same seqs) starts on a clean line instead of after a fragment that would
// read back as damage.
if let Err(t) = self.store.truncate(LOG, self.log_bytes) {
log::error!("catalog: can't cut the log back after a failed append: {t}");
self.tail_dirty = true;
}
return Err(io(e));
}
if let Some(p) = self.pending.as_mut() {
p.tail.extend_from_slice(buf.as_bytes());
p.tail_records += ops.len() as u64;
}
self.seq = seq;
self.log_records += ops.len() as u64;
self.log_bytes += buf.len() as u64;
let ms = ms_since(t0);
self.stats.appends += 1;
self.stats.last_append_ms = ms;
self.stats.max_append_ms = self.stats.max_append_ms.max(ms);
if profiling() {
eprintln!("catalog: append {} op(s), {} B: {ms:.2} ms", ops.len(), buf.len());
}
Ok(())
}
/// Write a snapshot of `catalog` (which must reflect every appended op) and reset the log.
pub fn snapshot(&mut self, catalog: &Catalog) -> Result<()> {
let body = serde_json::to_string(catalog).map_err(|e| CatalogError::Invalid(e.to_string()))?;
let file = format!("{{\"format\":\"{FORMAT}\",\"version\":{VERSION},\"seq\":{},\"catalog\":{body}}}\n", self.seq);
self.store.write_atomic(SNAPSHOT, file.as_bytes()).map_err(io)?;
self.snapshot_with_unlogged(catalog, 0)
}
/// [`Journal::snapshot`] of a catalog that also reflects `unlogged` ops applied after the
/// last appended one but never written to the log (their append failed). The snapshot counts
/// them (it holds the state after op `seq + unlogged`), so they are saved and must not be
/// appended afterwards. Once the snapshot is durable [`Journal::seq`] includes them, even if
/// resetting the log then fails (the error is still returned).
pub fn snapshot_with_unlogged(&mut self, catalog: &Catalog, unlogged: u64) -> Result<()> {
// never two snapshot writers; the one in flight is older, so it must land first
if let Err(e) = self.wait() {
log::warn!("catalog: background snapshot failed ({e}); writing one now");
}
let t0 = web_time::Instant::now();
let seq = self.seq + unlogged;
let mut timing = match write_snapshot(self.store.as_mut(), seq, catalog) {
Ok(t) => t,
Err(e) => {
// nothing lost (the log is whole); don't retry on every append
self.stats.failed_snapshots += 1;
self.retry_at_records = self.log_records + self.policy.max_records;
return Err(io(e));
}
};
// the snapshot is durable: it holds every op up to `seq`
self.seq = seq;
self.snapshot_seq = seq;
let t2 = web_time::Instant::now();
// A crash here leaves old records in the log; they are skipped by seq on load.
self.store.write_atomic(LOG, b"").map_err(io)?;
self.snapshot_seq = self.seq;
timing.reset_ms = ms_since(t2);
timing.total_ms = ms_since(t0);
timing.blocking_ms = timing.total_ms;
timing.records = self.log_records;
self.record_snapshot(timing);
self.log_records = 0;
self.log_bytes = 0;
self.retry_at_records = 0;
self.tail_dirty = false;
Ok(())
}
/// Compact like [`Journal::snapshot`], but write the snapshot on a worker thread from a copy
/// of `catalog` (cheap: photos are shared) when the store supports it (else synchronously).
/// `catalog` must reflect every appended op. Appends go on meanwhile; [`Journal::poll`]
/// finishes the compaction. No-op while a snapshot is already in flight.
pub fn snapshot_in_background(&mut self, catalog: &Catalog) -> Result<()> {
if self.pending.is_some() {
return Ok(());
}
let t0 = web_time::Instant::now();
let Some(mut writer) = self.store.background_writer() else { return self.snapshot(catalog) };
let copy = catalog.clone();
let seq = self.seq;
let spawned = std::thread::Builder::new().name("catalog-snapshot".into()).spawn(move || write_snapshot(writer.as_mut(), seq, &copy));
let worker = match spawned {
Ok(w) => w,
Err(e) => {
log::warn!("catalog: can't start a snapshot thread ({e}); writing it now");
return self.snapshot(catalog);
}
};
self.pending =
Some(Pending { seq, tail: Vec::new(), tail_records: 0, records: self.log_records, started: t0, start_ms: ms_since(t0), worker });
Ok(())
}
/// Finish a background snapshot if its worker is done (cheap otherwise). Returns whether one
/// finished. An error means the snapshot failed: nothing was lost (the log is whole), and
/// compaction is retried once the log has grown by another [`SnapshotPolicy::max_records`].
pub fn poll(&mut self) -> Result<bool> {
if !self.pending.as_ref().is_some_and(|p| p.worker.is_finished()) {
return Ok(false);
}
self.finish().map(|()| true)
}
/// Wait for the background snapshot in flight, if any, and finish it.
pub fn wait(&mut self) -> Result<()> {
if self.pending.is_none() {
return Ok(());
}
self.finish()
}
/// A background snapshot is in flight (until [`Journal::poll`] or [`Journal::wait`] finish it).
pub fn snapshot_running(&self) -> bool {
self.pending.is_some()
}
/// The background worker is done (its result not yet applied by [`Journal::poll`]).
pub fn snapshot_written(&self) -> bool {
self.pending.as_ref().is_some_and(|p| p.worker.is_finished())
}
/// Join the worker; on success replace the log with the records appended meanwhile.
fn finish(&mut self) -> Result<()> {
let Some(p) = self.pending.take() else { return Ok(()) };
let t0 = web_time::Instant::now();
let written = p.worker.join().unwrap_or_else(|_| Err(std::io::Error::other("snapshot thread panicked")));
let mut timing = match written {
Ok(t) => t,
Err(e) => {
// the log still holds every record: nothing lost, retry later
self.stats.failed_snapshots += 1;
self.retry_at_records = self.log_records + self.policy.max_records;
log::error!("catalog: background snapshot failed: {e}");
return Err(io(e));
}
};
// `catalog.snap` (op `p.seq`) is durable; the log's records `<= p.seq` are now stale.
// Keep exactly the ones appended since (a crash before this rewrite skips the stale ones).
let t1 = web_time::Instant::now();
self.snapshot_seq = p.seq;
if let Err(e) = self.store.write_atomic(LOG, &p.tail) {
// the log stays whole (stale records are skipped on load); compact again later
self.retry_at_records = self.log_records + self.policy.max_records;
log::error!("catalog: can't trim the log after a snapshot: {e}");
return Err(io(e));
}
timing.reset_ms = ms_since(t1);
timing.total_ms = ms_since(p.started);
timing.blocking_ms = p.start_ms + ms_since(t0);
timing.background = true;
timing.records = p.records;
self.record_snapshot(timing);
self.log_records = p.tail_records;
self.log_bytes = p.tail.len() as u64;
self.retry_at_records = 0;
self.tail_dirty = false;
Ok(())
}
fn record_snapshot(&mut self, t: SnapshotTiming) {
self.stats.snapshots += 1;
self.stats.last_snapshot = t;
self.stats.max_snapshot_ms = self.stats.max_snapshot_ms.max(t.total_ms);
self.stats.max_blocking_ms = self.stats.max_blocking_ms.max(t.blocking_ms);
if profiling() {
eprintln!(
"catalog: {}snapshot of {} records, {} B: serialize {:.1} ms, write+sync {:.1} ms, log reset {:.1} ms, total {:.1} ms, blocking {:.1} ms",
if t.background { "background " } else { "" },
t.records,
t.bytes,
t.serialize_ms,
t.write_sync_ms,
t.reset_ms,
t.total_ms,
t.blocking_ms
);
}
}
/// Where persistence time went so far.
pub fn stats(&self) -> PersistStats {
PersistStats { snapshot_running: self.pending.is_some(), ..self.stats }
}
/// The log is long enough to be worth compacting.
pub fn wants_snapshot(&self) -> bool {
self.log_records >= self.policy.max_records || self.log_bytes >= self.policy.max_bytes
self.pending.is_none()
&& self.log_records >= self.retry_at_records
&& (self.log_records >= self.policy.max_records || self.log_bytes >= self.policy.max_bytes)
}
pub fn seq(&self) -> u64 {
@@ -238,3 +681,16 @@ impl Journal {
self.store.describe()
}
}
impl Drop for Journal {
/// Wait for a background snapshot (never leave a writer behind that could replace a newer
/// snapshot of a journal reopened on the same files). The log is left whole: the snapshot's
/// records in it are skipped as stale on the next load.
fn drop(&mut self) {
if let Some(p) = self.pending.take()
&& let Ok(Err(e)) = p.worker.join()
{
log::error!("catalog: background snapshot failed: {e}");
}
}
}
+121 -5
View File
@@ -5,14 +5,24 @@
//! snapshot — crash-safe and diff-friendly (see [`journal`]);
//! - **undo/redo**: the engine keeps inverse ops;
//! - **determinism**: replaying the log reproduces the state exactly (property-tested).
//!
//! **Catalog format version** ([`journal::VERSION`], see [`journal`] → *Format versions*): adding
//! an [`Op`] variant or a serialized field means bumping it. Newer builds read every older format
//! (and upgrade it on open); older builds refuse a newer library with [`CatalogError::Newer`]
//! instead of reading part of it.
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
pub mod dates;
pub mod folders;
pub mod journal;
pub mod keywords;
pub mod local;
pub mod lock;
pub mod model;
pub mod query;
pub mod rules;
pub mod safe_file;
pub mod stacks;
pub mod store;
@@ -20,11 +30,14 @@ use std::collections::BTreeMap;
use std::sync::Arc;
pub use dates::{DateRun, GroupBy};
pub use journal::{Journal, LoadReport, SnapshotPolicy};
pub use folders::FolderNode;
pub use journal::{Journal, LoadReport, PersistStats, SnapshotPolicy, SnapshotTiming};
pub use keywords::KeywordNode;
use lightcraft_develop::DevelopSettings;
pub use local::{DEFAULT_FORGET_DAYS, ForgetPlan, folder_of};
pub use lock::{LibraryLock, LockError, LockOwner};
pub use model::*;
pub use query::{DateGroup, Filter, RatingOp, Sort, SortKey};
pub use query::{DateGroup, Filter, Person, RatingOp, Sort, SortKey, mix64};
pub use rules::{Match, Rule, RuleSet};
use serde::{Deserialize, Serialize};
pub use store::{FsStore, MemStore, Store};
@@ -43,6 +56,10 @@ pub enum CatalogError {
Corrupt(String),
#[error("catalog storage: {0}")]
Io(String),
/// The library was written by a newer LightCraft (a newer catalog format, or a change this
/// version doesn't know). Nothing was read into the session and nothing was modified.
#[error("this library was written by a newer version of LightCraft ({0}); update LightCraft to open it. The library was left unchanged.")]
Newer(String),
}
pub type Result<T> = std::result::Result<T, CatalogError>;
@@ -119,6 +136,11 @@ pub enum Op {
id: AlbumId,
parent: Option<AlbumId>,
},
/// Where the album stands among its siblings (`None`: by name). Format version 3.
SetAlbumOrder {
id: AlbumId,
order: Option<u32>,
},
SetAlbumPhotos {
id: AlbumId,
photos: Vec<PhotoId>,
@@ -179,12 +201,22 @@ pub enum Op {
file_size: u64,
#[serde(default)]
content_hash: Option<String>,
/// Why the file can only be shown from its embedded preview (see [`Photo::preview_only`]);
/// `None` = its raw data decodes (or it isn't a raw).
#[serde(default, skip_serializing_if = "Option::is_none")]
preview_only: Option<String>,
},
/// The name shown for a colour label (`None` = its colour's name).
SetLabelName {
label: ColorLabel,
name: Option<String>,
},
/// When a Local folder was last browsed (ISO 8601; `None` = forget the time). Not an undo
/// step: it drives forgetting untouched Local records (see [`local`]).
SetBrowsed {
folder: String,
at: Option<String>,
},
/// Several ops as one step (undo applies the inverses in reverse).
Batch {
ops: Vec<Op>,
@@ -204,6 +236,9 @@ pub struct Catalog {
/// Custom colour label names.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
label_names: BTreeMap<ColorLabel, String>,
/// When each Local folder was last browsed (folder path → ISO 8601), see [`local`].
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
browsed: BTreeMap<String, String>,
/// Increments on every applied op.
#[serde(skip)]
pub revision: u64,
@@ -252,6 +287,28 @@ impl Catalog {
pub fn albums(&self) -> impl Iterator<Item = &Album> {
self.albums.values()
}
/// The albums and folders directly inside `parent` (`None`: the top level), in the order the
/// sidebar lists them: folders first, then by their place if the user ordered them (those
/// without one after, by name), else by name.
pub fn album_children(&self, parent: Option<AlbumId>) -> Vec<&Album> {
let mut kids: Vec<&Album> = self.albums.values().filter(|a| a.parent == parent).collect();
kids.sort_by_cached_key(|a| album_order_key(a));
kids
}
/// [`Self::album_children`] of every folder (and of the top level, under `None`) in one pass:
/// what a tree view that draws all of them wants each frame.
pub fn album_children_by_parent(&self) -> std::collections::HashMap<Option<AlbumId>, Vec<&Album>> {
let mut map: std::collections::HashMap<Option<AlbumId>, Vec<&Album>> = std::collections::HashMap::new();
for a in self.albums.values() {
map.entry(a.parent).or_default().push(a);
}
map.values_mut().for_each(|kids| kids.sort_by_cached_key(|a| album_order_key(a)));
map
}
/// Whether anything inside `parent` has a place of its own (the folder is ordered by hand).
pub fn album_children_are_ordered(&self, parent: Option<AlbumId>) -> bool {
self.albums.values().any(|a| a.parent == parent && a.order.is_some())
}
/// The Quick Collection, once something was added to it.
pub fn quick_collection(&self) -> Option<AlbumId> {
self.albums.values().find(|a| a.quick).map(|a| a.id)
@@ -285,7 +342,8 @@ impl Catalog {
/// Number of photos shown for an album in the sources list (excludes deleted photos).
pub fn album_count(&self, id: AlbumId) -> usize {
match self.albums.get(&id) {
Some(Album { smart: Some(_), .. }) => self.album_photos(id).len(),
// counted in place: no id list is built just for its length
Some(Album { smart: Some(_), .. }) => self.photos.values().filter(|p| self.album_contains(id, p)).count(),
Some(a) => a.photos.iter().filter(|p| self.photos.get(p).is_some_and(|p| !p.deleted)).count(),
None => 0,
}
@@ -448,7 +506,20 @@ impl Catalog {
}
}
let a = self.album_mut(id)?;
Op::MoveAlbum { id, parent: std::mem::replace(&mut a.parent, parent) }
if a.parent == parent {
return Ok(Op::MoveAlbum { id, parent });
}
// its place belonged to the old neighbours: the undo gives it back with the folder
let old_order = a.order.take();
let old_parent = std::mem::replace(&mut a.parent, parent);
match old_order {
Some(_) => Op::Batch { ops: vec![Op::MoveAlbum { id, parent: old_parent }, Op::SetAlbumOrder { id, order: old_order }] },
None => Op::MoveAlbum { id, parent: old_parent },
}
}
Op::SetAlbumOrder { id, order } => {
let a = self.album_mut(id)?;
Op::SetAlbumOrder { id, order: std::mem::replace(&mut a.order, order) }
}
Op::SetAlbumPhotos { id, photos } => {
let a = self.album_mut(id)?;
@@ -514,7 +585,7 @@ impl Catalog {
let old_format = format.map(|f| std::mem::replace(&mut p.format, f));
Op::Relink { id, file_name: old_name, source: std::mem::replace(&mut p.source, source), format: old_format }
}
Op::SetContent { id, width, height, file_size, content_hash } => {
Op::SetContent { id, width, height, file_size, content_hash, preview_only } => {
let p = self.photo_mut(id)?;
Op::SetContent {
id,
@@ -522,6 +593,7 @@ impl Catalog {
height: std::mem::replace(&mut p.height, height),
file_size: std::mem::replace(&mut p.file_size, file_size),
content_hash: std::mem::replace(&mut p.content_hash, content_hash),
preview_only: std::mem::replace(&mut p.preview_only, preview_only),
}
}
Op::SetFile { id, file_name, source } => {
@@ -540,6 +612,14 @@ impl Catalog {
};
Op::SetLabelName { label, name: old }
}
Op::SetBrowsed { folder, at } => {
let folder = crate::query::folder_key(&folder);
let old = match at {
Some(t) => self.browsed.insert(folder.clone(), t),
None => self.browsed.remove(&folder),
};
Op::SetBrowsed { folder, at: old }
}
Op::Batch { ops } => {
let mut inverses = Vec::with_capacity(ops.len());
for op in ops {
@@ -574,6 +654,22 @@ impl Catalog {
Op::Batch { ops }
}
/// [`Self::delete_permanently_ops`] for several photos at once. One op list built against the
/// catalog as it is now: albums lose all of them in one write each, and stacks are shortened
/// (or dissolved) once, however many of their photos go.
pub fn delete_photos_permanently_ops(&self, ids: &[PhotoId]) -> Op {
let gone: std::collections::HashSet<PhotoId> = ids.iter().copied().collect();
let mut ops: Vec<Op> = self
.albums
.values()
.filter(|a| a.photos.iter().any(|p| gone.contains(p)))
.map(|a| Op::SetAlbumPhotos { id: a.id, photos: a.photos.iter().copied().filter(|p| !gone.contains(p)).collect() })
.collect();
ops.extend(self.remove_from_stacks_ops(ids));
ops.extend(gone.iter().map(|id| Op::RemovePhoto { id: *id }));
Op::Batch { ops }
}
// ---- persistence
/// Full snapshot as JSON.
@@ -612,7 +708,27 @@ impl Catalog {
}
}
/// How siblings are listed: folders first, then the ones with a place of their own by it, then by
/// name; the id settles any tie.
fn album_order_key(a: &Album) -> (bool, bool, u32, String, AlbumId) {
(!a.folder, a.order.is_none(), a.order.unwrap_or(0), a.name.to_lowercase(), a.id)
}
#[cfg(test)]
mod tests;
#[cfg(test)]
mod tests_album_order;
#[cfg(test)]
mod tests_background;
#[cfg(test)]
mod tests_folders;
#[cfg(test)]
mod tests_format_version;
#[cfg(test)]
mod tests_journal;
#[cfg(test)]
mod tests_local;
#[cfg(test)]
mod tests_lock;
#[cfg(test)]
mod tests_torn_append;
+305
View File
@@ -0,0 +1,305 @@
//! Forgetting untouched Local browse records.
//!
//! Browsing a folder (Local) catalogues its photos as `local` records so they can be shown,
//! rated and edited in place. Most are only ever looked at, and on a large disk they can be most
//! of the catalog. A record is **forgotten** (removed from the catalog, never from the disk —
//! files and XMP sidecars are not touched) when all of these hold:
//!
//! - it is a Local record ([`Photo::local`]) of a file, not deleted, not a virtual copy;
//! - its folder was last browsed more than `days` ago ([`Catalog::last_browsed`]). A folder with
//! no recorded time (a catalog from before this existed) counts as browsed *now*: it is stamped
//! with the current time and nothing in it is forgotten this time;
//! - it is **untouched** ([`Photo::untouched_local`]): nothing about it changed since it was
//! browsed — see below;
//! - nothing refers to it: no album holds it or shows it as cover, no stack holds it, no virtual
//! copy was made of it; and the caller (the engine) says it isn't in use (visible, selected,
//! in the undo history, has a smart preview…).
//!
//! **Untouched.** A record catalogued by a browse carries a fingerprint of its whole state at
//! that moment ([`Photo::local_baseline`]). It is untouched while its state still hashes to that
//! fingerprint, i.e. *any* change since — develop settings or history, versions, rating, flag,
//! label, any metadata (title, caption, keywords, location, copyright…), capture time, crop,
//! rename or relink, culling scores, file content — keeps it. Records without a fingerprint
//! (browsed before fingerprints existed) are judged by their data alone and kept unless every
//! user-editable field is still empty / default: no rating, flag or label; develop settings equal
//! to the import defaults with no history, versions or edit time; no title, caption, alt text,
//! description, keywords, creator, copyright, rights, location or GPS; no culling scores.
//! When in doubt a record is kept.
//!
//! Browsing the folder again catalogues its files afresh (new records, new fingerprints).
use std::collections::{BTreeMap, BTreeSet, HashSet};
use serde::Serialize;
use crate::{Catalog, Op, Photo, PhotoId, Source};
/// Default for how long a Local folder may go unbrowsed before its untouched records are
/// forgotten (days).
pub const DEFAULT_FORGET_DAYS: u32 = 30;
/// Records removed per logged batch (keeps log lines a reasonable size).
const BATCH: usize = 2000;
/// 64-bit FNV-1a over everything written to it (stable across builds and platforms, unlike
/// `std`'s hasher), so a photo is hashed as it serialises, without a buffer.
struct Fnv(u64);
impl std::io::Write for Fnv {
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
for b in buf {
self.0 ^= u64::from(*b);
self.0 = self.0.wrapping_mul(0x0100_0000_01b3);
}
Ok(buf.len())
}
fn flush(&mut self) -> std::io::Result<()> {
Ok(())
}
}
/// The folder a file-backed photo is in (`None` for demo photos).
pub fn folder_of(p: &Photo) -> Option<String> {
match &p.source {
// `/` and `\` are both separators whatever the platform (a catalog can move between them)
Source::File { path } => {
let key = crate::query::folder_key(path);
key.rsplit_once('/').map(|(dir, _)| if dir.is_empty() { "/".to_string() } else { dir.to_string() })
}
Source::Demo { .. } => None,
}
}
impl Photo {
/// A hash of everything about the photo except [`Photo::local_baseline`] itself.
pub fn fingerprint(&self) -> u64 {
let mut h = Fnv(0xcbf2_9ce4_8422_2325);
let r = if self.local_baseline.is_some() {
let mut p = self.clone();
p.local_baseline = None;
serde_json::to_writer(&mut h, &p)
} else {
serde_json::to_writer(&mut h, self)
};
// (serialising a photo can't fail; if it ever did, no baseline would match: kept)
if r.is_err() { 0 } else { h.0 }
}
/// Record the current state as the browse baseline (done when a browse catalogues it).
pub fn set_local_baseline(&mut self) {
self.local_baseline = None;
self.local_baseline = Some(self.fingerprint());
}
/// A Local record nothing was changed on since it was browsed (see the module docs). Says
/// nothing about albums, stacks or virtual copies referring to it.
pub fn untouched_local(&self) -> bool {
if !self.local || self.deleted || self.copy_of.is_some() || self.copy_name.is_some() || folder_of(self).is_none() {
return false;
}
match self.local_baseline {
Some(b) => self.fingerprint() == b,
None => self.looks_unedited(),
}
}
/// Every user-editable field is still empty / default (records without a baseline).
fn looks_unedited(&self) -> bool {
let m = &self.meta;
self.rating == 0
&& self.flag == crate::Flag::None
&& self.label.is_none()
&& self.edited.is_none()
&& self.history.is_empty()
&& self.versions.is_empty()
&& *self.develop == self.import_defaults()
&& self.analysis.is_none()
&& m.title.is_empty()
&& m.caption.is_empty()
&& m.alt_text.is_empty()
&& m.extended_description.is_empty()
&& m.keywords.is_empty()
&& m.creator.is_empty()
&& m.copyright.is_empty()
&& m.copyright_status.is_unknown()
&& m.usage_terms.is_empty()
&& m.copyright_url.is_empty()
&& m.location.is_empty()
&& m.city.is_empty()
&& m.state.is_empty()
&& m.country.is_empty()
&& m.gps.is_none()
}
}
/// What forgetting Local records would do / did.
#[derive(Clone, Debug, Default, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct ForgetPlan {
/// Local records looked at.
pub local: usize,
/// Records to forget.
pub evict: Vec<PhotoId>,
/// Kept: their folder was browsed within the period (or has no time yet).
pub kept_recent: usize,
/// Kept: changed by the user since they were browsed.
pub kept_touched: usize,
/// Kept: in an album, a stack, the source of a virtual copy, or in use.
pub kept_in_use: usize,
/// Folders with Local records but no last-browsed time: stamped now.
pub stamp: Vec<String>,
/// Folders whose last Local record goes: their time is dropped.
pub unstamp: Vec<String>,
/// The time stamps are given.
#[serde(skip)]
pub now: String,
}
impl ForgetPlan {
/// The ops that carry the plan out (empty when there is nothing to do).
pub fn ops(&self) -> Vec<Op> {
let mut ops: Vec<Op> = self.stamp.iter().map(|f| Op::SetBrowsed { folder: f.clone(), at: Some(self.now.clone()) }).collect();
ops.extend(self.evict.chunks(BATCH).map(|c| Op::Batch { ops: c.iter().map(|id| Op::RemovePhoto { id: *id }).collect() }));
ops.extend(self.unstamp.iter().map(|f| Op::SetBrowsed { folder: f.clone(), at: None }));
ops
}
}
impl Catalog {
/// When `folder` was last browsed (ISO 8601), if known.
pub fn last_browsed(&self, folder: &str) -> Option<&str> {
self.browsed.get(&crate::query::folder_key(folder)).map(String::as_str)
}
/// Folders with a last-browsed time.
pub fn browsed_folders(&self) -> &BTreeMap<String, String> {
&self.browsed
}
/// Plan forgetting untouched Local records whose folder wasn't browsed for `days` days (as of
/// `now`, ISO 8601). `in_use` keeps records the caller still needs. `days == 0` (never) or an
/// unreadable `now` forgets nothing. See the module docs.
pub fn forget_local_plan(&self, now: &str, days: u32, in_use: &dyn Fn(&Photo) -> bool) -> ForgetPlan {
let mut plan = ForgetPlan { now: now.to_string(), ..Default::default() };
let now_s = crate::stacks::iso_seconds(now);
let max_age = i64::from(days) * 86_400;
let mut referenced: HashSet<PhotoId> = HashSet::new();
for a in self.albums.values() {
referenced.extend(a.photos.iter().copied());
referenced.extend(a.cover);
}
for s in self.stacks.values() {
referenced.extend(s.photos.iter().copied());
}
referenced.extend(self.photos.values().filter_map(|p| p.copy_of));
// the fingerprints are the costly part: work them out for the candidates in parallel
let candidates: Vec<&Photo> = self
.photos
.values()
.filter(|p| p.local && !referenced.contains(&p.id))
.filter(|p| folder_of(p).and_then(|f| self.browsed.get(&f)).is_some_and(|at| is_old(at, now_s, days, max_age)))
.map(|p| &**p)
.collect();
let untouched: HashSet<PhotoId> = untouched_of(&candidates);
let mut stamp = BTreeSet::new();
let mut remaining: BTreeSet<String> = BTreeSet::new();
for p in self.photos.values().filter(|p| p.local) {
plan.local += 1;
let Some(folder) = folder_of(p) else {
plan.kept_in_use += 1;
continue;
};
let recent = match (self.browsed.get(&folder), now_s) {
(None, _) => {
stamp.insert(folder.clone());
true
}
(Some(at), _) => !is_old(at, now_s, days, max_age),
};
let keep = if recent {
plan.kept_recent += 1;
true
} else if referenced.contains(&p.id) || in_use(p) {
plan.kept_in_use += 1;
true
} else if !untouched.contains(&p.id) {
plan.kept_touched += 1;
true
} else {
plan.evict.push(p.id);
false
};
if keep {
remaining.insert(folder);
}
}
plan.stamp = stamp.into_iter().collect();
plan.unstamp = self.browsed.keys().filter(|f| !remaining.contains(*f)).cloned().collect();
plan
}
}
/// A folder browsed at `at` is past the period (never when `days` is 0 or a time is unreadable).
fn is_old(at: &str, now_s: Option<i64>, days: u32, max_age: i64) -> bool {
days > 0 && matches!((now_s, crate::stacks::iso_seconds(at)), (Some(n), Some(a)) if n - a >= max_age)
}
/// The untouched ones among `photos` ([`Photo::untouched_local`]), checked on several threads
/// where there are many.
fn untouched_of(photos: &[&Photo]) -> HashSet<PhotoId> {
let check = |chunk: &[&Photo]| chunk.iter().filter(|p| p.untouched_local()).map(|p| p.id).collect::<Vec<_>>();
#[cfg(not(target_arch = "wasm32"))]
{
let threads = std::thread::available_parallelism().map_or(1, |n| n.get()).min(8);
if threads > 1 && photos.len() >= 256 {
let size = photos.len().div_ceil(threads);
return std::thread::scope(|sc| {
let jobs: Vec<_> = photos.chunks(size).map(|c| sc.spawn(move || check(c))).collect();
// a job that panicked (it can't) leaves its photos out: they are kept
jobs.into_iter().filter_map(|j| j.join().ok()).flatten().collect()
});
}
}
check(photos).into_iter().collect()
}
impl Op {
/// Every photo the op refers to.
pub fn photo_ids(&self, out: &mut impl FnMut(PhotoId)) {
match self {
Op::AddPhoto { photo } => out(photo.id),
Op::RemovePhoto { id }
| Op::SetRating { id, .. }
| Op::SetFlag { id, .. }
| Op::SetLabel { id, .. }
| Op::SetDevelop { id, .. }
| Op::SetMeta { id, .. }
| Op::SetDeleted { id, .. }
| Op::SetLocal { id, .. }
| Op::SetVersions { id, .. }
| Op::SetHistory { id, .. }
| Op::PushHistory { id, .. }
| Op::SetCaptured { id, .. }
| Op::SetAnalysis { id, .. }
| Op::SetFile { id, .. }
| Op::Relink { id, .. }
| Op::SetContent { id, .. } => out(*id),
Op::AddAlbum { album } => {
album.photos.iter().for_each(|p| out(*p));
album.cover.into_iter().for_each(&mut *out);
}
Op::SetAlbumPhotos { photos, .. } | Op::SetStack { photos, .. } => photos.iter().for_each(|p| out(*p)),
Op::SetAlbumCover { cover, .. } => cover.iter().for_each(|p| out(*p)),
Op::AddStack { stack } => stack.photos.iter().for_each(|p| out(*p)),
Op::Batch { ops } => ops.iter().for_each(|o| o.photo_ids(out)),
Op::RemoveAlbum { .. }
| Op::RenameAlbum { .. }
| Op::MoveAlbum { .. }
| Op::SetAlbumOrder { .. }
| Op::SetAlbumRules { .. }
| Op::RemoveStack { .. }
| Op::SetLabelName { .. }
| Op::SetBrowsed { .. } => {}
}
}
}
+140
View File
@@ -0,0 +1,140 @@
//! One process per library: an exclusive OS lock on `<library>/catalog.lock`, held for as long as
//! the library is open (issue #99).
//!
//! Two processes on the same library (the app and `lightcraft-cli --library`, two app instances,
//! two computers on a shared folder) would each keep their own catalog and `seq` counter, and
//! whichever wrote the last snapshot would silently drop the other's edits. So opening a library
//! takes the lock first, and a second opener gets [`LockError::InUse`] instead.
//!
//! - The lock is the OS's (`flock` on Unix, `LockFileEx` on Windows, via [`std::fs::File::try_lock`]):
//! it is released when the [`LibraryLock`] is dropped and, after a crash or a kill, by the OS when
//! the process ends. A `catalog.lock` file left behind is therefore never "stale": only a lock
//! that's actually held refuses the open.
//! - Who holds it (process id, computer, program, version) is written to `catalog.lock.owner`, for
//! the error message. It's a separate file because a Windows lock also blocks reading the
//! locked file. It's only a hint: it may be missing or left over from a crash.
//! - File systems without lock support (some network shares): the library opens unlocked, with a
//! warning — refusing it would lock the user out of their own library. Locks over SMB/NFS are
//! advisory and depend on the server; the owner file is then the only hint.
//! - Browser storage (OPFS) has no file locks: see the web host for its own guard.
use std::fs::{File, OpenOptions, TryLockError};
use std::io;
use std::path::{Path, PathBuf};
/// The lock file in the library directory.
pub const LOCK: &str = "catalog.lock";
/// Who holds the lock (JSON, see [`LockOwner`]).
pub const OWNER: &str = "catalog.lock.owner";
/// Who has the library open (as written in [`OWNER`]).
#[derive(Clone, Debug, Default, PartialEq, serde::Serialize, serde::Deserialize)]
#[serde(default)]
pub struct LockOwner {
pub pid: u32,
/// Computer name, if known.
pub host: String,
/// The program (`LightCraft`, `lightcraft-cli`…) and its version.
pub program: String,
pub version: String,
/// When it opened the library (seconds since 1970).
pub since: u64,
}
impl LockOwner {
/// This process, as `program`.
pub fn this_process(program: &str) -> LockOwner {
LockOwner {
pid: std::process::id(),
host: host_name(),
program: program.to_string(),
version: env!("CARGO_PKG_VERSION").to_string(),
since: web_time::SystemTime::now().duration_since(web_time::UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0),
}
}
/// "LightCraft 0.3.0 (process 123 on studio-mac)".
pub fn describe(&self) -> String {
let program = if self.program.is_empty() { "another LightCraft" } else { self.program.as_str() };
let version = if self.version.is_empty() { String::new() } else { format!(" {}", self.version) };
let host = if self.host.is_empty() {
String::new()
} else if self.host == host_name() {
" on this computer".to_string()
} else {
format!(" on {}", self.host)
};
format!("{program}{version} (process {}{host})", self.pid)
}
}
/// The computer's name, best effort (empty if unknown).
fn host_name() -> String {
let env = ["COMPUTERNAME", "HOSTNAME"].iter().find_map(|k| std::env::var(k).ok().filter(|h| !h.trim().is_empty()));
env.or_else(|| std::fs::read_to_string("/etc/hostname").ok()).map(|h| h.trim().to_string()).filter(|h| !h.is_empty()).unwrap_or_default()
}
#[derive(Debug, thiserror::Error)]
pub enum LockError {
/// Another process has the library open.
#[error("This library is already open in {}. Close it there first.", .0.as_ref().map(LockOwner::describe).unwrap_or_else(|| "another LightCraft program".into()))]
InUse(Option<LockOwner>),
}
/// The exclusive lock on a library directory; released on drop (and by the OS if the process
/// dies).
#[derive(Debug)]
pub struct LibraryLock {
dir: PathBuf,
/// `None` when the file system can't lock (the library is open unlocked).
_file: Option<File>,
held: bool,
}
impl LibraryLock {
/// Lock `dir` (which must exist) for this process. Fails only when another process holds it.
pub fn acquire(dir: &Path, program: &str) -> Result<LibraryLock, LockError> {
let unlocked = |why: io::Error| {
log::warn!("library {}: can't lock {LOCK} ({why}); opening it without protection against a second program", dir.display());
LibraryLock { dir: dir.to_path_buf(), _file: None, held: false }
};
let file = match OpenOptions::new().read(true).write(true).create(true).truncate(false).open(dir.join(LOCK)) {
Ok(f) => f,
Err(e) => return Ok(unlocked(e)),
};
match file.try_lock() {
Ok(()) => {}
Err(TryLockError::WouldBlock) => {
let owner = std::fs::read(dir.join(OWNER)).ok().and_then(|b| serde_json::from_slice::<LockOwner>(&b).ok());
return Err(LockError::InUse(owner));
}
Err(TryLockError::Error(e)) => return Ok(unlocked(e)),
}
let owner = LockOwner::this_process(program);
if let Ok(json) = serde_json::to_vec_pretty(&owner)
&& let Err(e) = std::fs::write(dir.join(OWNER), json)
{
log::warn!("library {}: can't write {OWNER}: {e}", dir.display());
}
Ok(LibraryLock { dir: dir.to_path_buf(), _file: Some(file), held: true })
}
/// The OS lock is held (`false`: the file system can't lock; opened unprotected).
pub fn held(&self) -> bool {
self.held
}
pub fn dir(&self) -> &Path {
&self.dir
}
}
impl Drop for LibraryLock {
/// Remove the owner note while still holding the lock (so it can't delete the next holder's),
/// then the file handle closes and the OS releases the lock.
fn drop(&mut self) {
if self.held {
let _ = std::fs::remove_file(self.dir.join(OWNER));
}
}
}
+142 -4
View File
@@ -2,7 +2,7 @@
use std::sync::Arc;
use lightcraft_develop::DevelopSettings;
use lightcraft_develop::{DevelopSettings, WbMode};
use serde::{Deserialize, Serialize};
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
@@ -93,6 +93,65 @@ pub enum Source {
Demo { scene: u32 },
}
/// Copyright status (IPTC / XMP Rights Management `xmpRights:Marked`).
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum CopyrightStatus {
/// Not stated (no `xmpRights:Marked`).
#[default]
Unknown,
/// `xmpRights:Marked` = True.
Copyrighted,
/// `xmpRights:Marked` = False.
PublicDomain,
}
impl CopyrightStatus {
pub const ALL: [CopyrightStatus; 3] = [CopyrightStatus::Unknown, CopyrightStatus::Copyrighted, CopyrightStatus::PublicDomain];
/// `unknown` / `copyrighted` / `publicDomain` (also `public domain`, `public-domain`).
pub fn parse(s: &str) -> Option<CopyrightStatus> {
match s.trim().to_ascii_lowercase().replace([' ', '-', '_'], "").as_str() {
"unknown" | "" => Some(CopyrightStatus::Unknown),
"copyrighted" => Some(CopyrightStatus::Copyrighted),
"publicdomain" => Some(CopyrightStatus::PublicDomain),
_ => None,
}
}
/// The id used by commands (`photo.setMeta`'s `copyrightStatus`).
pub fn id(self) -> &'static str {
match self {
CopyrightStatus::Unknown => "unknown",
CopyrightStatus::Copyrighted => "copyrighted",
CopyrightStatus::PublicDomain => "publicDomain",
}
}
pub fn label(self) -> &'static str {
match self {
CopyrightStatus::Unknown => "Unknown",
CopyrightStatus::Copyrighted => "Copyrighted",
CopyrightStatus::PublicDomain => "Public Domain",
}
}
/// As `xmpRights:Marked`: `Some(true)` copyrighted, `Some(false)` public domain.
pub fn marked(self) -> Option<bool> {
match self {
CopyrightStatus::Unknown => None,
CopyrightStatus::Copyrighted => Some(true),
CopyrightStatus::PublicDomain => Some(false),
}
}
pub fn from_marked(m: Option<bool>) -> CopyrightStatus {
match m {
None => CopyrightStatus::Unknown,
Some(true) => CopyrightStatus::Copyrighted,
Some(false) => CopyrightStatus::PublicDomain,
}
}
pub fn is_unknown(&self) -> bool {
*self == CopyrightStatus::Unknown
}
}
/// Descriptive + capture metadata.
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
#[serde(default)]
@@ -115,8 +174,20 @@ pub struct Meta {
pub alt_text: String,
pub extended_description: String,
pub copyright: String,
/// Copyright status, rights usage terms and copyright info URL (IPTC Core rights fields).
#[serde(skip_serializing_if = "CopyrightStatus::is_unknown")]
pub copyright_status: CopyrightStatus,
#[serde(skip_serializing_if = "String::is_empty")]
pub usage_terms: String,
#[serde(skip_serializing_if = "String::is_empty")]
pub copyright_url: String,
pub creator: String,
pub keywords: Vec<String>,
/// Face/pet/focus regions read from XMP (MWG-RS), on the upright (EXIF-oriented) photo.
/// Removing or resizing one edits the catalog only; LightCraft never writes regions to XMP.
/// Left out of the catalog JSON when empty (most photos), so older catalogs read unchanged.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub regions: Vec<lightcraft_meta::Region>,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
@@ -200,6 +271,16 @@ pub struct Photo {
/// Assisted culling scores (`None` until analysed).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub analysis: Option<Analysis>,
/// A raw file whose sensor data can't be decoded yet (an unsupported raw variant): why (the
/// raw decoder's reason, e.g. "Nikon Huffman-compressed NEF …"). The photo is shown and
/// edited from the camera's embedded JPEG preview — a rendered image with the camera's
/// picture style baked in — so it is treated as a rendered (non-raw) source.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub preview_only: Option<String>,
/// A Local record: the fingerprint of its state when the browse catalogued it, to tell
/// whether the user changed anything since (see [`crate::local`]).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub local_baseline: Option<u64>,
}
/// What assisted culling measured on a photo.
@@ -246,8 +327,23 @@ impl Photo {
copy_name: None,
import_look: None,
analysis: None,
preview_only: None,
local_baseline: None,
}
}
/// A raw file developed from its sensor data: not a rendered image, and not a raw shown from
/// its embedded preview ([`Photo::preview_only`]).
pub fn develops_raw(&self) -> bool {
self.kind == MediaKind::Raw && self.preview_only.is_none()
}
/// The current ARW, NEF, RW2, RAF, CR3, CR2, PEF and SRW readers have vendor WB multipliers but no measured camera
/// illuminant. Use adjustments relative to the camera's as-shot look, as for rendered
/// photographs (the engine's `camera_preview::file_local_look` covers the same formats; RWL and
/// RAW are Leica's and the oldest Panasonic bodies' names for RW2 files).
pub fn relative_wb(&self) -> bool {
self.develops_raw()
&& ["ARW", "NEF", "NRW", "RW2", "RWL", "RAW", "RAF", "CR3", "CR2", "PEF", "SRW"].iter().any(|f| self.format.eq_ignore_ascii_case(f))
}
/// The develop settings import gave this photo: [`Photo::camera_defaults`], or the user's
/// default preset applied on top of them ([`Photo::import_look`]).
pub fn import_defaults(&self) -> DevelopSettings {
@@ -259,7 +355,8 @@ impl Photo {
/// The built-in defaults for this photo, before any user default preset: raws start from
/// their as-shot white balance; embedded lens corrections on when the file has them.
pub fn camera_defaults(&self) -> DevelopSettings {
let mut d = match self.as_shot_wb {
let wb = if self.relative_wb() { Some((6500.0, 0.0)) } else { self.as_shot_wb };
let mut d = match wb {
Some((t, tint)) => DevelopSettings::for_raw(t, tint),
None => DevelopSettings::default(),
};
@@ -274,7 +371,12 @@ impl Photo {
}
/// Edited by the user: settings differ from what import gave the photo.
pub fn is_edited(&self) -> bool {
*self.develop != self.import_defaults() && !self.develop.is_unedited()
let mut defaults = self.import_defaults();
// As Shot reads the source's current WB; stored numbers can predate a decoder update.
if self.develop.wb.mode == WbMode::AsShot && defaults.wb.mode == WbMode::AsShot {
defaults.wb = self.develop.wb;
}
*self.develop != defaults && !self.develop.is_unedited()
}
/// Capture time if known, else import time (sort key).
pub fn date(&self) -> &str {
@@ -303,12 +405,16 @@ pub struct Album {
/// The Quick Collection: a regular album that B adds to (one per library).
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub quick: bool,
/// Its place among the albums of the same kind in its folder, once the user ordered them by
/// hand (see [`crate::Catalog::album_children`]); `None`: listed by name.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub order: Option<u32>,
}
impl Album {
/// A regular (manual) album.
pub fn new(id: AlbumId, name: impl Into<String>) -> Album {
Album { id, name: name.into(), parent: None, folder: false, photos: Vec::new(), cover: None, smart: None, quick: false }
Album { id, name: name.into(), parent: None, folder: false, photos: Vec::new(), cover: None, smart: None, quick: false, order: None }
}
pub fn is_smart(&self) -> bool {
self.smart.is_some()
@@ -332,4 +438,36 @@ mod edited_tests {
p.develop = Arc::new(DevelopSettings::default());
assert!(!p.is_edited(), "a full reset is unedited too");
}
#[test]
fn supported_raws_use_relative_white_balance() {
for (name, format, relative) in [
("a.arw", "ARW", true),
("a.nef", "NEF", true),
("a.nrw", "nrw", true),
("a.rw2", "RW2", true),
("a.rwl", "RWL", true),
("a.raw", "RAW", true),
("a.raf", "rAf", true),
("a.cr3", "cr3", true),
("a.srw", "SRW", true),
("a.dng", "DNG", false),
] {
let mut p = Photo::new(PhotoId(1), Source::Demo { scene: 0 }, name, format, 10, 10, "2026-10-01T00:00:00");
p.kind = MediaKind::Raw;
p.as_shot_wb = Some((5200.0, 4.0));
assert_eq!(p.relative_wb(), relative, "{format}");
let wb = p.camera_defaults().wb;
assert_eq!((wb.temp, wb.tint), if relative { (6500.0, 0.0) } else { (5200.0, 4.0) }, "{format}");
p.develop = Arc::new(DevelopSettings::for_raw(5200.0, 4.0));
assert!(!p.is_edited(), "{format}: old As Shot values are not an edit");
let mut custom = (*p.develop).clone();
custom.wb.mode = WbMode::Custom;
p.develop = Arc::new(custom);
assert!(p.is_edited(), "{format}: Custom WB is still an edit");
// shown from the embedded preview: a rendered image, not a relative-WB raw
p.preview_only = Some("unsupported".into());
assert!(!p.relative_wb());
}
}
}
+240 -11
View File
@@ -37,8 +37,14 @@ pub struct Filter {
pub deleted: bool,
/// Capture date prefix (`2026`, `2026-04`, `2026-04-12`).
pub date: Option<String>,
/// A folder the library's photos were imported from: those photos and the ones imported from
/// the folders inside it (see [`crate::folders`]). Photos only browsed in Local are never part
/// of it, unlike [`Filter::folder`].
pub library_folder: Option<String>,
/// A keyword; hierarchical keywords match their children too (`travel` finds `travel|italy`).
pub keyword: Option<String>,
/// A person: photos with a named face region of this name (case-insensitive), as read from XMP.
pub person: Option<String>,
pub camera: Option<String>,
/// Lens (case-insensitive substring).
pub lens: Option<String>,
@@ -70,6 +76,9 @@ pub enum SortKey {
FileName,
Rating,
FileSize,
/// A shuffle fixed by [`Sort::seed`]: the same seed always gives the same order, and photos
/// added or edited later never reshuffle the others. Pick a new seed to reshuffle.
Random,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
@@ -79,14 +88,30 @@ pub struct Sort {
pub ascending: bool,
/// Date headers in the grid (date sort keys only).
pub group: crate::GroupBy,
/// Which shuffle [`SortKey::Random`] gives (ignored by the other keys).
pub seed: u64,
}
impl Default for Sort {
fn default() -> Self {
Sort { key: SortKey::CaptureDate, ascending: false, group: crate::GroupBy::Auto }
Sort { key: SortKey::CaptureDate, ascending: false, group: crate::GroupBy::Auto, seed: 0 }
}
}
/// splitmix64 finaliser: a stateless, platform-independent mix (no RNG state, no `rand` version
/// to drift), so a seed names the same shuffle on every machine.
pub fn mix64(x: u64) -> u64 {
let mut z = x.wrapping_add(0x9E37_79B9_7F4A_7C15);
z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9);
z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB);
z ^ (z >> 31)
}
/// A photo's place in the shuffle for `seed`.
fn shuffle_rank(seed: u64, id: PhotoId) -> u64 {
mix64(mix64(seed) ^ id.0)
}
fn token_matches(p: &Photo, tok: &str) -> bool {
let t = tok.to_lowercase();
if let Some((field, val)) = t.split_once(':') {
@@ -113,9 +138,10 @@ fn token_matches(p: &Photo, tok: &str) -> bool {
"camera" => p.meta.camera.to_lowercase().contains(val),
"lens" => p.meta.lens.to_lowercase().contains(val),
"keyword" | "kw" => p.meta.keywords.iter().any(|k| crate::keywords::is_under(k, val)),
"person" | "who" => has_person(p, val),
"type" | "kind" => format!("{:?}", p.kind).eq_ignore_ascii_case(val),
"edited" => (val == "true" || val == "yes") == p.is_edited(),
"date" => p.date().starts_with(val),
"date" => p.captured.as_deref().is_some_and(|c| c.starts_with(val)),
"copy" | "virtual" => (val == "true" || val == "yes") == p.copy_of.is_some(),
"name" | "file" => p.file_name.to_lowercase().contains(val),
_ => false,
@@ -125,6 +151,12 @@ fn token_matches(p: &Photo, tok: &str) -> bool {
hay.iter().any(|h| h.to_lowercase().contains(&t)) || p.meta.keywords.iter().any(|k| k.to_lowercase().contains(&t))
}
/// Whether `p` has a named face region called `name` (case-insensitive).
fn has_person(p: &Photo, name: &str) -> bool {
let name = name.trim().to_lowercase();
p.meta.regions.iter().any(|r| r.kind == lightcraft_meta::RegionKind::Face && r.name.as_deref().is_some_and(|n| n.to_lowercase() == name))
}
impl Filter {
/// Human-readable summary of the active rules (smart album tooltips, `album.list`).
pub fn describe(&self) -> String {
@@ -152,9 +184,15 @@ impl Filter {
if let Some(e) = self.edited {
v.push(if e { "edited".into() } else { "unedited".into() });
}
for (name, val) in
[("keyword", &self.keyword), ("camera", &self.camera), ("lens", &self.lens), ("date", &self.date), ("imported", &self.imported)]
{
for (name, val) in [
("keyword", &self.keyword),
("person", &self.person),
("camera", &self.camera),
("lens", &self.lens),
("date", &self.date),
("folder", &self.library_folder),
("imported", &self.imported),
] {
if let Some(x) = val {
v.push(format!("{name} {x}"));
}
@@ -174,7 +212,25 @@ impl Filter {
if v.is_empty() { "all photos".into() } else { v.join(", ") }
}
/// Whether matches depend on the clock (only "in the last…" rules do).
pub fn depends_on_now(&self) -> bool {
self.rule_set.as_ref().is_some_and(crate::RuleSet::depends_on_now)
}
pub fn matches(&self, p: &Photo, cat: &Catalog) -> bool {
self.matches_in(p, cat, self.library_root().as_deref())
}
/// [`Filter::library_folder`] as a folder identity ([`folder_key`]); `None` without a choice.
fn library_root(&self) -> Option<String> {
// as given: a folder may be named with spaces at its edge, and trimming would make it
// another folder
self.library_folder.as_deref().filter(|d| !d.trim().is_empty()).map(folder_key)
}
/// [`Filter::matches`] with the library folder already turned into its identity, so a query
/// over a big library does that once, not once per photo.
fn matches_in(&self, p: &Photo, cat: &Catalog, root: Option<&str>) -> bool {
if p.deleted != self.deleted {
return false;
}
@@ -228,12 +284,18 @@ impl Filter {
return false;
}
if let Some(from) = &self.date_from
&& p.date() < from.as_str()
&& match p.captured.as_deref() {
Some(c) => c < from.as_str(),
None => true,
}
{
return false;
}
if let Some(to) = &self.date_to
&& p.date().get(..to.len()).unwrap_or(p.date()) > to.as_str()
&& match p.captured.as_deref() {
Some(c) => c.get(..to.len()).unwrap_or(c) > to.as_str(),
None => true,
}
{
return false;
}
@@ -243,7 +305,12 @@ impl Filter {
return false;
}
if let Some(d) = &self.date
&& !p.date().starts_with(d.as_str())
&& !p.captured.as_deref().is_some_and(|c| c.starts_with(d.as_str()))
{
return false;
}
if let Some(root) = root
&& !photo_in_root(p, root)
{
return false;
}
@@ -262,6 +329,11 @@ impl Filter {
{
return false;
}
if let Some(n) = &self.person
&& !has_person(p, n)
{
return false;
}
if let Some(c) = &self.camera
&& !p.meta.camera.eq_ignore_ascii_case(c)
{
@@ -274,15 +346,17 @@ impl Filter {
impl Catalog {
/// Photos matching `filter`, in `sort` order (ties broken by id for stability).
pub fn query(&self, filter: &Filter, sort: &Sort) -> Vec<PhotoId> {
let mut v: Vec<&Photo> = self.photos().map(|p| p.as_ref()).filter(|p| filter.matches(p, self)).collect();
let root = filter.library_root();
let mut v: Vec<&Photo> = self.photos().map(|p| p.as_ref()).filter(|p| filter.matches_in(p, self, root.as_deref())).collect();
v.sort_by(|a, b| {
let o = match sort.key {
SortKey::CaptureDate => a.date().cmp(b.date()),
SortKey::CaptureDate => a.captured.cmp(&b.captured).then_with(|| a.imported.cmp(&b.imported)),
SortKey::ImportDate => a.imported.cmp(&b.imported),
SortKey::EditDate => a.edited.cmp(&b.edited),
SortKey::FileName => a.file_name.to_lowercase().cmp(&b.file_name.to_lowercase()),
SortKey::Rating => a.rating.cmp(&b.rating),
SortKey::FileSize => a.file_size.cmp(&b.file_size),
SortKey::Random => shuffle_rank(sort.seed, a.id).cmp(&shuffle_rank(sort.seed, b.id)),
}
.then(a.id.cmp(&b.id));
if sort.ascending { o } else { o.reverse() }
@@ -295,7 +369,7 @@ impl Catalog {
use std::collections::BTreeMap;
let mut years: BTreeMap<&str, (BTreeMap<&str, usize>, BTreeMap<&str, usize>)> = BTreeMap::new();
for p in self.photos().filter(|p| p.in_library()) {
let d = p.date();
let Some(d) = p.captured.as_deref() else { continue };
if d.len() >= 10 {
let (months, days) = years.entry(&d[..4]).or_default();
*months.entry(&d[..7]).or_default() += 1;
@@ -325,6 +399,55 @@ impl Catalog {
}
m.into_iter().collect()
}
/// The people named on faces in the library (MWG regions read from XMP): how many photos each
/// appears in and the photo showing their largest face (for a card's picture). Most photos first,
/// then by name. Names that differ only in case are one person, shown as first seen; a person
/// twice in one photo counts once.
pub fn people(&self) -> Vec<Person> {
self.people_in(&Filter::default())
}
/// [`Self::people`] among the photos `filter` lets through (its `person` is ignored): what the
/// People view offers while other filters (a date, a rating, an album…) are active, so picking
/// a person never ends in an empty grid.
pub fn people_in(&self, filter: &Filter) -> Vec<Person> {
let filter = Filter { person: None, ..filter.clone() };
let mut m: std::collections::HashMap<String, (Person, f64)> = Default::default();
let root = filter.library_root();
for p in self.photos().filter(|p| filter.matches_in(p, self, root.as_deref())) {
let mut seen: Vec<String> = Vec::new();
for r in p.meta.regions.iter().filter(|r| r.kind == lightcraft_meta::RegionKind::Face) {
let Some(name) = r.name.as_deref().map(str::trim).filter(|n| !n.is_empty()) else { continue };
let key = name.to_lowercase();
// the face's size in pixels of the photo
let area = (r.rect.x1 - r.rect.x0) * (r.rect.y1 - r.rect.y0) * p.width as f64 * p.height as f64;
let (person, best) =
m.entry(key.clone()).or_insert_with(|| (Person { name: name.to_string(), count: 0, photo: p.id, face: r.rect }, area));
if !seen.contains(&key) {
person.count += 1;
seen.push(key);
}
if area > *best || (area == *best && p.id < person.photo) {
(person.photo, person.face, *best) = (p.id, r.rect, area);
}
}
}
let mut v: Vec<Person> = m.into_values().map(|(p, _)| p).collect();
v.sort_by(|a, b| b.count.cmp(&a.count).then_with(|| a.name.to_lowercase().cmp(&b.name.to_lowercase())));
v
}
}
/// A person named on faces in the library ([`Catalog::people`]).
#[derive(Clone, Debug, PartialEq)]
pub struct Person {
pub name: String,
/// Photos they appear in.
pub count: usize,
/// The photo with their largest face, and that face (normalized, in the photo's upright frame).
pub photo: PhotoId,
pub face: lightcraft_meta::Rect,
}
#[derive(Clone, Debug, PartialEq, Serialize)]
@@ -346,6 +469,112 @@ pub fn in_folder(path: &str, dir: &str, deep: bool) -> bool {
!rest.is_empty() && (deep || !rest.contains(['/', '\\']))
}
/// A path read the way [`folder_key`] reads it: the part `..` can't climb above (`//server/share`,
/// `c:`; empty for `/` and relative paths), the names below it, and whether it is absolute.
pub(crate) struct SplitPath {
pub prefix: String,
pub parts: Vec<String>,
pub absolute: bool,
}
/// Both separators, repeated separators (also between a server and its share), `.` and a
/// trailing separator dropped, `..` resolved lexically, a Windows verbatim prefix removed;
/// `lower_drive` lower-cases a drive letter (the identity does, a display name wants it as is).
pub(crate) fn split_path(path: &str, lower_drive: bool) -> SplitPath {
let mut s = path.replace('\\', "/");
if let Some(rest) = s.strip_prefix("//?/").or_else(|| s.strip_prefix("//./")) {
s = match rest.strip_prefix("UNC/") {
Some(unc) => format!("//{unc}"),
None => rest.to_string(),
};
}
let (prefix, rest) = if let Some(unc) = s.strip_prefix("//").filter(|u| !u.is_empty() && !u.starts_with('/')) {
let mut segs = unc.split('/').filter(|x| !x.is_empty());
let (server, share) = (segs.next().unwrap_or(""), segs.next().unwrap_or(""));
(format!("//{server}/{share}"), segs.collect::<Vec<_>>().join("/"))
} else if let (true, Some(drive), Some(rest)) = (matches!(s.as_bytes(), [a, b':', ..] if a.is_ascii_alphabetic()), s.get(..2), s.get(2..)) {
(if lower_drive { drive.to_ascii_lowercase() } else { drive.to_string() }, rest.to_string())
} else {
(String::new(), s.clone())
};
let absolute = rest.starts_with('/') || !prefix.is_empty();
let mut parts: Vec<String> = Vec::new();
for c in rest.split('/') {
match c {
"" | "." => {}
".." if parts.last().is_some_and(|p| p != "..") => {
parts.pop();
}
".." if absolute => {}
c => parts.push(c.to_string()),
}
}
SplitPath { prefix, parts, absolute }
}
/// A folder path's identity, for telling whether two spellings name the same folder: `/` and
/// `\\` are both separators, repeated separators, `.` and a trailing separator are dropped,
/// `..` is resolved lexically, a Windows verbatim prefix (`\\?\`) is removed and the drive
/// letter lower-cased; on Windows (case-insensitive file names) the whole path is lower-cased.
/// File names are otherwise compared as written, so on a case-insensitive disk elsewhere
/// (macOS, a Samba share) two spellings that differ in case are two folders here.
/// No file-system access: the paths compared should already be absolute.
pub fn folder_key(path: &str) -> String {
let SplitPath { prefix, parts, absolute } = split_path(path, true);
let key = if absolute { format!("{prefix}/{}", parts.join("/")) } else { parts.join("/") };
if cfg!(windows) { key.to_lowercase() } else { key }
}
/// Whether `path` is folder `root` or lies somewhere inside it, however either is spelled
/// (see [`folder_key`]).
pub fn folder_within(path: &str, root: &str) -> bool {
key_within(&folder_key(path), &folder_key(root))
}
/// The names below folder `root` that lead to `path`, as the caller spelled them (`None` when
/// `path` is not at or inside `root`; empty when it is `root`). Compared by [`folder_key`], but
/// the names keep their case, which the key lowers on Windows.
pub fn folder_rest(path: &str, root: &str) -> Option<Vec<String>> {
if !key_within(&folder_key(path), &folder_key(root)) {
return None;
}
let depth = split_path(root, false).parts.len();
Some(split_path(path, false).parts.into_iter().skip(depth).collect())
}
/// Whether a library photo (not one only browsed in Local) lies in the folder whose
/// [`folder_key`] is `root`. A plain POSIX path (no `\`, `.`, `..`, repeated or trailing
/// separators) is its own identity, so it is read without building a key for it: the check runs
/// for every photo of the library on each refresh, and this keeps it cheap.
pub(crate) fn photo_in_root(p: &Photo, root: &str) -> bool {
if p.local {
return false;
}
let crate::Source::File { path } = &p.source else { return false };
let plain = path.starts_with('/')
&& !path.starts_with("//")
&& !path.ends_with('/')
&& !path.contains(['\\'])
&& !path.contains("//")
&& !path.contains("/.");
if !cfg!(windows) && plain && root.starts_with('/') && !root.starts_with("//") {
let r = root.trim_end_matches('/');
return path.strip_prefix(r).is_some_and(|rest| rest.starts_with('/'));
}
crate::local::folder_of(p).is_some_and(|f| key_within(&f, root))
}
/// [`folder_within`] for two paths already turned into their [`folder_key`].
pub(crate) fn key_within(p: &str, r: &str) -> bool {
// `.` or `a/..` name no folder: not "everything"
if r.is_empty() {
return false;
}
// a disk's root key ends in `/` (`c:/`, `//srv/share/`), a folder's does not: same folder
let (p, r) = (p.trim_end_matches('/'), r.trim_end_matches('/'));
p == r || p.strip_prefix(r).is_some_and(|rest| rest.starts_with('/'))
}
/// The kind of Photo Merge result a file is, from the name merges give it (`IMG_1-HDR.dng`,
/// `IMG_1-Pano.dng`, `IMG_1-HDR-Pano.dng` — also the names other raw editors use):
/// `hdr`, `panorama` or `hdrPanorama`.
+48 -1
View File
@@ -7,7 +7,7 @@
use serde::{Deserialize, Serialize};
use serde_json::Value;
use crate::{Catalog, Photo};
use crate::{Catalog, Photo, Source};
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
@@ -62,6 +62,7 @@ pub const FIELDS: &[(&str, &str, Kind)] = &[
("keywords", "Keywords", Kind::Keywords),
("text", "Any Searchable Text", Kind::Text),
("fileName", "Filename", Kind::Text),
("filePath", "File Path", Kind::Text),
("format", "File Format", Kind::Text),
("title", "Title", Kind::Text),
("caption", "Caption", Kind::Text),
@@ -70,6 +71,7 @@ pub const FIELDS: &[(&str, &str, Kind)] = &[
("location", "Location", Kind::Text),
("creator", "Creator", Kind::Text),
("copyright", "Copyright", Kind::Text),
("copyrightStatus", "Copyright Status", Kind::Choice(&["copyrighted", "publicDomain", "unknown"])),
("captureDate", "Capture Date", Kind::Date),
("importDate", "Import Date", Kind::Date),
("editDate", "Edit Date", Kind::Date),
@@ -303,6 +305,20 @@ impl Rule {
text(&all)
}
"fileName" => text(&p.file_name),
"filePath" => {
// `/` and `\` both separate, whatever platform the catalog came from; demo photos have no path
let path = match &p.source {
Source::File { path } => path.replace('\\', "/"),
Source::Demo { .. } => String::new(),
};
let want = want.replace('\\', "/");
match op {
// the whole string, spaces included (not word by word like the other text fields)
"contains" => !want.trim().is_empty() && path.to_lowercase().contains(want.trim()),
"notContains" => want.trim().is_empty() || !path.to_lowercase().contains(want.trim()),
_ => text_op(op, &path, &want),
}
}
"format" => text(&p.format),
"title" => text(&m.title),
"caption" => text(&m.caption),
@@ -311,6 +327,10 @@ impl Rule {
"location" => text(&[m.location.as_str(), &m.city, &m.state, &m.country].join(" ")),
"creator" => text(&m.creator),
"copyright" => text(&m.copyright),
"copyrightStatus" => {
let want = crate::CopyrightStatus::parse(&want);
(want == Some(m.copyright_status)) == (op == "is")
}
"captureDate" => date_op(op, p.captured.as_deref(), value),
"importDate" => date_op(op, Some(&p.imported), value),
"editDate" => date_op(op, p.edited.as_deref(), value),
@@ -341,6 +361,15 @@ impl RuleSet {
}
}
/// Whether matches depend on the clock ("in the last…" rules, nested groups included): the
/// same photos can enter or leave the set without any catalog change.
pub fn depends_on_now(&self) -> bool {
self.rules.iter().any(|r| match r {
Rule::Group { group } => group.depends_on_now(),
Rule::Field { op, .. } => op == "inLast" || op == "notInLast",
})
}
/// Unknown fields or operators (for command validation), as readable messages.
pub fn problems(&self) -> Vec<String> {
let mut out = Vec::new();
@@ -431,6 +460,17 @@ mod tests {
);
no(json!({"rules": [{"field": "keywords", "op": "isEmpty"}]}));
yes(json!({"rules": [{"field": "fileName", "op": "startsWith", "value": "img_"}, {"field": "fileName", "op": "endsWith", "value": ".cr2"}]}));
// file path: the whole string (spaces included), either separator, case-insensitive
let mut fp = photo(3);
fp.source = Source::File { path: "D:\\Photos\\Aliah Ira Polanco-Grylls\\2026\\IMG_0042.CR2".into() };
let fpm = |v: serde_json::Value| rs(v).matches(&fp, &cat);
assert!(fpm(json!({"rules": [{"field": "filePath", "op": "contains", "value": "/Aliah Ira Polanco-Grylls/"}]})));
assert!(!fpm(json!({"rules": [{"field": "filePath", "op": "contains", "value": "/Ira Aliah/"}]})));
assert!(!fpm(json!({"rules": [{"field": "filePath", "op": "contains", "value": "Polanco Grylls"}]})));
assert!(fpm(json!({"rules": [{"field": "filePath", "op": "notContains", "value": "/Other/"}]})));
assert!(fpm(json!({"rules": [{"field": "filePath", "op": "startsWith", "value": "d:/photos/"}]})));
no(json!({"rules": [{"field": "filePath", "op": "contains", "value": "/Aliah/"}]})); // demo photo has no path
yes(json!({"rules": [{"field": "filePath", "op": "isEmpty"}]}));
yes(json!({"rules": [{"field": "camera", "op": "contains", "value": "x2"}, {"field": "title", "op": "isEmpty"}]}));
yes(json!({"rules": [{"field": "iso", "op": "between", "value": [800, 3200]}, {"field": "aperture", "op": "lte", "value": "f/4"}]}));
yes(json!({"rules": [{"field": "megapixels", "op": "gte", "value": 24}]}));
@@ -440,6 +480,13 @@ mod tests {
yes(json!({"rules": [{"field": "captureDate", "op": "between", "value": ["2026-08-01", "2026-08"]}]}));
no(json!({"rules": [{"field": "captureDate", "op": "after", "value": "2026-08"}]}));
yes(json!({"rules": [{"field": "kind", "op": "isNot", "value": "video"}, {"field": "edited", "op": "is", "value": false}]}));
// copyright status (unknown until set)
yes(json!({"rules": [{"field": "copyrightStatus", "op": "is", "value": "unknown"}]}));
no(json!({"rules": [{"field": "copyrightStatus", "op": "is", "value": "copyrighted"}]}));
let mut pd = photo(2);
pd.meta.copyright_status = crate::CopyrightStatus::PublicDomain;
assert!(rs(json!({"rules": [{"field": "copyrightStatus", "op": "is", "value": "publicDomain"}]})).matches(&pd, &cat));
assert!(rs(json!({"rules": [{"field": "copyrightStatus", "op": "isNot", "value": "copyrighted"}]})).matches(&pd, &cat));
// nested: rating ≥ 4 and (label is blue or keywords contain food)
yes(json!({"rules": [{"field": "rating", "op": "gte", "value": 4}, {"group": {"match": "any", "rules": [
{"field": "label", "op": "is", "value": "blue"}, {"field": "keywords", "op": "contains", "value": "food"}]}}]}));
+388
View File
@@ -0,0 +1,388 @@
//! Crash-safe writes of user-visible files: exports, renders, DNG conversions, smart previews and
//! XMP sidecars all go through here (the catalog's own files use [`crate::store::FsStore`]).
//!
//! **Write policy** (issue #134): what can't be recreated — originals, XMP sidecars, the catalog,
//! settings, DNGs that replace raws, smart previews (the only stand-in while an original is
//! offline), an edit copy that becomes a library photo — is written **durably**: synced to disk
//! before it counts as written. What can always be made again from the originals — exports,
//! renders, screenshots, merge previews — is written **atomically** only: a half-written file
//! never replaces a good one, but there is no per-file sync (on a USB drive or a NAS that sync
//! dominates a large export).
//!
//! - [`write_atomic`] replaces a file so that after a crash, a full disk or an unplugged drive
//! either the old or the complete new content is there — never a truncated mix. The content
//! goes to a new temp file in the same folder (unique name, `create_new`), is synced to disk and
//! renamed over the target; then the folder is synced (POSIX). Any failure removes the temp
//! file and leaves the target as it was.
//! - [`write_atomic_nosync`] is the same temp file + rename without the two syncs, for
//! recreatable outputs only: a failed write still removes the temp file and leaves the target as
//! it was, but after a power cut the new file may be lost (on some file systems left empty) —
//! it is then simply exported again.
//! - [`write_new`] / [`write_new_unique`] write a file that must not exist yet: the same temp file
//! and sync, plus a read-back check, and the file then appears under its final name **without
//! replacing anything** (a hard link, which fails on a taken name; a check-then-rename where the
//! volume has no hard links).
//! - [`same_file`] tells whether two paths name the same file on disk (device + inode on Unix,
//! resolved paths elsewhere), for "is this target an original?" checks.
use std::cell::Cell;
use std::fs::{self, File, OpenOptions};
use std::io::{self, Write};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicU64, Ordering};
thread_local! {
static FAIL_AFTER: Cell<Option<u64>> = const { Cell::new(None) };
static SYNCS: Cell<u64> = const { Cell::new(0) };
}
/// How many file and folder syncs this module made **on the calling thread** (for tests: a
/// durable write path syncs, a recreatable output doesn't).
#[doc(hidden)]
pub fn syncs_on_this_thread() -> u64 {
SYNCS.with(Cell::get)
}
fn sync_file(f: &File) -> io::Result<()> {
SYNCS.with(|c| c.set(c.get() + 1));
f.sync_all()
}
/// Fault injection for tests (in any crate): while the returned guard lives, every write made by
/// this module **on the calling thread** fails once `bytes` bytes of a file were written — as a
/// full disk or an unplugged drive would.
#[doc(hidden)]
pub fn fail_writes_after(bytes: u64) -> FaultGuard {
FAIL_AFTER.with(|c| c.set(Some(bytes)));
FaultGuard(())
}
/// Ends a [`fail_writes_after`] when dropped.
#[doc(hidden)]
pub struct FaultGuard(());
impl Drop for FaultGuard {
fn drop(&mut self) {
FAIL_AFTER.with(|c| c.set(None));
}
}
/// Counts what is written; fails past an injected limit.
struct Limited<W> {
inner: W,
n: u64,
limit: Option<u64>,
}
impl<W: Write> Write for Limited<W> {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
let buf = match self.limit {
Some(l) => {
let room = usize::try_from(l.saturating_sub(self.n)).unwrap_or(usize::MAX);
if room == 0 && !buf.is_empty() {
return Err(io::Error::other("write failed (injected fault)"));
}
buf.get(..room.min(buf.len())).unwrap_or(buf)
}
None => buf,
};
let k = self.inner.write(buf)?;
self.n += k as u64;
Ok(k)
}
fn flush(&mut self) -> io::Result<()> {
self.inner.flush()
}
}
fn parent_of(path: &Path) -> &Path {
path.parent().filter(|d| !d.as_os_str().is_empty()).unwrap_or(Path::new("."))
}
/// Make a rename or a new name in `dir` durable (POSIX). Opening a directory fails on Windows,
/// where NTFS journals the rename itself, so errors are ignored.
pub fn sync_dir(dir: &Path) {
if let Ok(d) = File::open(dir) {
let _ = sync_file(&d);
}
}
/// A new, empty temp file next to `path` (hidden, unique in this process and folder).
fn create_temp(path: &Path) -> io::Result<(PathBuf, File)> {
static N: AtomicU64 = AtomicU64::new(0);
if path.file_name().is_none() {
return Err(io::Error::new(io::ErrorKind::InvalidInput, format!("{} is not a file path", path.display())));
}
let dir = parent_of(path);
for _ in 0..1000 {
let tmp = dir.join(format!(".lightcraft-{}-{}.tmp", std::process::id(), N.fetch_add(1, Ordering::Relaxed)));
match OpenOptions::new().write(true).create_new(true).open(&tmp) {
Ok(f) => return Ok((tmp, f)),
Err(e) if e.kind() == io::ErrorKind::AlreadyExists => continue,
Err(e) => return Err(e),
}
}
Err(io::Error::new(io::ErrorKind::AlreadyExists, format!("no free temp file name in {}", dir.display())))
}
/// Write a temp file next to `path` with what `fill` writes, synced to disk when `sync`. On
/// failure the temp file is removed. Returns its path and length.
fn write_temp(path: &Path, sync: bool, fill: &mut dyn FnMut(&mut dyn Write) -> io::Result<()>) -> io::Result<(PathBuf, u64)> {
let (tmp, file) = create_temp(path)?;
let r = (|| {
let limit = FAIL_AFTER.with(Cell::get);
let mut w = Limited { inner: io::BufWriter::with_capacity(1 << 20, file), n: 0, limit };
fill(&mut w)?;
w.flush()?;
let n = w.n;
let f = w.inner.into_inner().map_err(|e| e.into_error())?;
if sync {
sync_file(&f)?;
}
Ok(n)
})();
match r {
Ok(n) => Ok((tmp, n)),
Err(e) => {
let _ = fs::remove_file(&tmp);
Err(e)
}
}
}
/// Replace (or create) `path` with `data` atomically and durably; see the module docs. The
/// folder must exist.
pub fn write_atomic(path: &Path, data: &[u8]) -> io::Result<()> {
write_atomic_with(path, &mut |w| w.write_all(data)).map(|_| ())
}
/// [`write_atomic`] with streamed content: `fill` writes it (an error aborts and leaves the old
/// file). Returns the length written.
pub fn write_atomic_with(path: &Path, fill: &mut dyn FnMut(&mut dyn Write) -> io::Result<()>) -> io::Result<u64> {
replace_with(path, true, fill)
}
/// Replace (or create) `path` with `data` atomically but **without syncing** it to disk — only for
/// outputs that can always be recreated (exports, renders, screenshots); see the module docs.
/// Anything that can't be made again goes through [`write_atomic`].
pub fn write_atomic_nosync(path: &Path, data: &[u8]) -> io::Result<()> {
replace_with(path, false, &mut |w| w.write_all(data)).map(|_| ())
}
/// Temp file (synced when `sync`) renamed over `path`, then the folder synced when `sync`.
fn replace_with(path: &Path, sync: bool, fill: &mut dyn FnMut(&mut dyn Write) -> io::Result<()>) -> io::Result<u64> {
let (tmp, n) = write_temp(path, sync, fill)?;
if let Err(e) = fs::rename(&tmp, path) {
let _ = fs::remove_file(&tmp);
return Err(e);
}
if sync {
sync_dir(parent_of(path));
}
Ok(n)
}
/// Write `data` as the new file `path`: never replaces an existing file (`AlreadyExists`), and
/// the file only appears once it is complete, synced and read back identical.
pub fn write_new(path: &Path, data: &[u8]) -> io::Result<()> {
write_new_unique(&mut std::iter::once(path.to_path_buf()), data).map(|_| ())
}
/// [`write_new`] under the first free name of `candidates` (all in the same folder, e.g.
/// `IMG_1.dng`, `IMG_1-2.dng`, …): the file is written once and then given the first name that
/// is not taken. Returns that name; `AlreadyExists` when every candidate is taken.
pub fn write_new_unique(candidates: &mut dyn Iterator<Item = PathBuf>, data: &[u8]) -> io::Result<PathBuf> {
let mut candidates = candidates.peekable();
let Some(first) = candidates.peek().cloned() else {
return Err(io::Error::new(io::ErrorKind::InvalidInput, "no file name to write"));
};
let (tmp, _) = write_temp(&first, true, &mut |w| w.write_all(data))?;
let r = (|| {
// Kept (issue #134 point 4): these files replace raws (Convert / Copy as DNG) or become
// library photos (merges). On a local disk the read usually comes from the cache, but on a
// network volume it can go back to the server, and next to encoding a DNG it costs little.
if !same_content(&tmp, data)? {
return Err(io::Error::other("the written file reads back different (failing drive or connection?)"));
}
for path in candidates.by_ref().take(100_000) {
match publish_new(&tmp, &path) {
Ok(()) => {
sync_dir(parent_of(&path));
return Ok(path);
}
Err(e) if e.kind() == io::ErrorKind::AlreadyExists => continue,
Err(e) => return Err(e),
}
}
Err(io::Error::new(io::ErrorKind::AlreadyExists, format!("{}: no free file name", first.display())))
})();
// after a hard link the temp name is a second name of the new file; after a rename it's gone
let _ = fs::remove_file(&tmp);
r
}
/// Give the complete temp file `tmp` the name `path`, unless that is taken.
fn publish_new(tmp: &Path, path: &Path) -> io::Result<()> {
match fs::hard_link(tmp, path) {
Ok(()) => Ok(()),
Err(e) if e.kind() == io::ErrorKind::AlreadyExists => Err(e),
// no hard links on this volume (FAT/exFAT, some shares): check, then rename
Err(_) => {
if fs::symlink_metadata(path).is_ok() {
return Err(io::Error::new(io::ErrorKind::AlreadyExists, format!("{} exists", path.display())));
}
fs::rename(tmp, path)
}
}
}
/// Does the file at `path` hold exactly `data`?
fn same_content(path: &Path, data: &[u8]) -> io::Result<bool> {
use std::io::Read;
let mut f = File::open(path)?;
if f.metadata()?.len() != data.len() as u64 {
return Ok(false);
}
let mut buf = vec![0u8; 1 << 20];
let mut at = 0usize;
loop {
let k = match f.read(&mut buf) {
Ok(0) => return Ok(at == data.len()),
Ok(k) => k,
Err(e) if e.kind() == io::ErrorKind::Interrupted => continue,
Err(e) => return Err(e),
};
if data.get(at..at + k) != buf.get(..k) {
return Ok(false);
}
at += k;
}
}
/// Do `a` and `b` name the same existing file (through links, `..`, case-insensitive names)?
pub fn same_file(a: &Path, b: &Path) -> bool {
let (Ok(ma), Ok(mb)) = (fs::metadata(a), fs::metadata(b)) else { return false };
#[cfg(unix)]
{
use std::os::unix::fs::MetadataExt;
ma.dev() == mb.dev() && ma.ino() == mb.ino()
}
#[cfg(not(unix))]
{
if ma.len() != mb.len() {
return false;
}
match (fs::canonicalize(a), fs::canonicalize(b)) {
(Ok(x), Ok(y)) => x == y || x.to_string_lossy().to_lowercase() == y.to_string_lossy().to_lowercase(),
_ => false,
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn temp(tag: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("lc-safe-file-{tag}-{}", std::process::id()));
let _ = fs::remove_dir_all(&d);
fs::create_dir_all(&d).unwrap();
d
}
fn names(dir: &Path) -> Vec<String> {
let mut v: Vec<String> = fs::read_dir(dir).unwrap().flatten().map(|e| e.file_name().to_string_lossy().to_string()).collect();
v.sort();
v
}
#[test]
fn atomic_write_replaces_and_leaves_no_temp() {
let d = temp("replace");
let p = d.join("out.jpg");
write_atomic(&p, b"one").unwrap();
write_atomic(&p, b"two two").unwrap();
assert_eq!(fs::read(&p).unwrap(), b"two two");
assert_eq!(names(&d), vec!["out.jpg"]);
let _ = fs::remove_dir_all(&d);
}
#[test]
fn a_failure_mid_write_keeps_the_old_file() {
let d = temp("midwrite");
let p = d.join("photo.jpg");
fs::write(&p, b"the original bytes").unwrap();
{
let _f = fail_writes_after(4);
let e = write_atomic(&p, &[7u8; 10_000]).unwrap_err();
assert!(e.to_string().contains("injected"), "{e}");
assert!(write_new(&d.join("new.dng"), &[1u8; 100]).is_err());
}
assert_eq!(fs::read(&p).unwrap(), b"the original bytes", "the target is untouched");
assert_eq!(names(&d), vec!["photo.jpg"], "no temp file and no partial new file");
// a fill error aborts the same way
let r = write_atomic_with(&p, &mut |w| {
w.write_all(b"half")?;
Err(io::Error::other("encoder failed"))
});
assert!(r.is_err());
assert_eq!(fs::read(&p).unwrap(), b"the original bytes");
assert_eq!(names(&d), vec!["photo.jpg"]);
let _ = fs::remove_dir_all(&d);
}
#[test]
fn recreatable_outputs_are_atomic_without_a_sync() {
let d = temp("nosync");
let p = d.join("export.jpg");
let before = syncs_on_this_thread();
write_atomic_nosync(&p, b"one").unwrap();
write_atomic_nosync(&p, b"two two").unwrap();
assert_eq!(syncs_on_this_thread(), before, "no file or folder sync");
assert_eq!(fs::read(&p).unwrap(), b"two two");
assert_eq!(names(&d), vec!["export.jpg"], "no temp file left");
// still atomic: a failure part-way leaves the previous file and no temp file
{
let _f = fail_writes_after(3);
assert!(write_atomic_nosync(&p, &[9u8; 10_000]).is_err());
}
assert_eq!(fs::read(&p).unwrap(), b"two two");
assert_eq!(names(&d), vec!["export.jpg"]);
// the durable writers do sync (the file; the folder too where it can be opened)
let before = syncs_on_this_thread();
write_atomic(&p, b"three").unwrap();
let mid = syncs_on_this_thread();
assert!(mid > before);
write_new(&d.join("new.dng"), b"dng").unwrap();
assert!(syncs_on_this_thread() > mid);
let _ = fs::remove_dir_all(&d);
}
#[test]
fn new_files_never_replace_anything() {
let d = temp("new");
let p = d.join("a.dng");
fs::write(&p, b"keep").unwrap();
assert_eq!(write_new(&p, b"other").unwrap_err().kind(), io::ErrorKind::AlreadyExists);
assert_eq!(fs::read(&p).unwrap(), b"keep");
let mut c = (1..).map(|k| if k == 1 { d.join("a.dng") } else { d.join(format!("a-{k}.dng")) });
let got = write_new_unique(&mut c, b"fresh").unwrap();
assert_eq!(got, d.join("a-2.dng"));
assert_eq!(fs::read(&got).unwrap(), b"fresh");
assert_eq!(names(&d), vec!["a-2.dng", "a.dng"], "the temp file is gone");
let _ = fs::remove_dir_all(&d);
}
#[test]
fn same_file_sees_through_other_spellings() {
let d = temp("same");
let p = d.join("x.jpg");
fs::write(&p, b"x").unwrap();
fs::write(d.join("y.jpg"), b"x").unwrap();
fs::create_dir_all(d.join("sub")).unwrap();
assert!(same_file(&p, &d.join("sub/../x.jpg")));
assert!(!same_file(&p, &d.join("y.jpg")));
assert!(!same_file(&p, &d.join("missing.jpg")));
let _ = fs::remove_dir_all(&d);
}
}
+1 -1
View File
@@ -277,7 +277,7 @@ mod tests {
#[test]
fn group_merge_set_top_remove_and_arrange() {
let (mut c, ids) = cat(6, &[]);
let [a, b, x, d, e, f] = ids[..] else { unreachable!() };
let [a, b, x, d, e, f] = ids[..] else { panic!("six ids") };
let op = c.group_ops(b, &[a, b, x], false).unwrap();
let inv = c.apply(op).unwrap();
let s = c.stack_of(a).unwrap().clone();
+91 -6
View File
@@ -16,12 +16,33 @@ pub trait Store: Send {
/// Replace a file atomically: after a crash either the old or the new content is visible,
/// never a mix (native: write temp + fsync + rename + fsync directory).
fn write_atomic(&mut self, name: &str, data: &[u8]) -> io::Result<()>;
/// Append and make durable (fsync) before returning.
/// [`Store::write_atomic`] with streamed content: `fill` writes the new content into the
/// given writer (an error aborts and leaves the old file). Same durability: the file is
/// replaced only once all of it is written (native: and fsynced). Returns its length.
///
/// The default buffers the content in memory; [`FsStore`] streams it into the temp file, so
/// a large snapshot is never held in memory as a whole.
fn write_atomic_with(&mut self, name: &str, fill: &mut dyn FnMut(&mut dyn io::Write) -> io::Result<()>) -> io::Result<u64> {
let mut buf = Vec::new();
fill(&mut buf)?;
self.write_atomic(name, &buf)?;
Ok(buf.len() as u64)
}
/// Append and make durable (fsync) before returning. On an error, part of `data` may have
/// been written (a full disk, a dropped network share): the journal cuts the file back to its
/// previous length with [`Store::truncate`] before appending again.
fn append(&mut self, name: &str, data: &[u8]) -> io::Result<()>;
/// Cut a file to `len` bytes (drop a torn tail before appending again).
/// Cut a file to `len` bytes (drop a torn tail before appending again). A missing file is
/// left missing.
fn truncate(&mut self, name: &str, len: u64) -> io::Result<()>;
/// Human-readable location (diagnostics).
fn describe(&self) -> String;
/// Another handle on the same files that a worker thread can write `catalog.snap` through
/// while this one keeps appending to the log (background compaction). `None` (the default):
/// snapshots are written synchronously.
fn background_writer(&self) -> Option<Box<dyn Store>> {
None
}
}
/// Files in a directory.
@@ -77,6 +98,32 @@ impl Store for FsStore {
Ok(())
}
fn write_atomic_with(&mut self, name: &str, fill: &mut dyn FnMut(&mut dyn io::Write) -> io::Result<()>) -> io::Result<u64> {
use std::io::Write;
if self.appender.as_ref().is_some_and(|(n, _)| n == name) {
self.appender = None;
}
let tmp = self.path(&format!("{name}.tmp"));
let written = (|| {
let mut w = Counter { inner: io::BufWriter::with_capacity(1 << 20, std::fs::File::create(&tmp)?), n: 0 };
fill(&mut w)?;
w.flush()?;
let f = w.inner.into_inner().map_err(|e| e.into_error())?;
f.sync_all()?;
Ok(w.n)
})();
let n = match written {
Ok(n) => n,
Err(e) => {
let _ = std::fs::remove_file(&tmp);
return Err(e);
}
};
std::fs::rename(&tmp, self.path(name))?;
self.sync_dir();
Ok(n)
}
fn append(&mut self, name: &str, data: &[u8]) -> io::Result<()> {
use std::io::Write;
if self.appender.as_ref().is_none_or(|(n, _)| n != name) {
@@ -87,16 +134,27 @@ impl Store for FsStore {
}
self.appender = Some((name.to_string(), f));
}
let (_, f) = self.appender.as_mut().expect("appender just opened");
f.write_all(data)?;
f.sync_data()
let Some((_, f)) = self.appender.as_mut() else { return Err(io::Error::other("appender not open")) };
let written = f.write_all(data).and_then(|()| f.sync_data());
if written.is_err() {
// The handle may be dead (a network share that reconnected, a removed volume): the next
// attempt reopens the file. Part of `data` may be in the file; the caller cuts it back
// ([`Store::truncate`]) before appending again.
self.appender = None;
}
written
}
fn truncate(&mut self, name: &str, len: u64) -> io::Result<()> {
if self.appender.as_ref().is_some_and(|(n, _)| n == name) {
self.appender = None;
}
let f = std::fs::OpenOptions::new().write(true).open(self.path(name))?;
let f = match std::fs::OpenOptions::new().write(true).open(self.path(name)) {
Ok(f) => f,
// nothing to cut (an append that failed before creating the file)
Err(e) if e.kind() == io::ErrorKind::NotFound => return Ok(()),
Err(e) => return Err(e),
};
f.set_len(len)?;
f.sync_all()
}
@@ -104,12 +162,36 @@ impl Store for FsStore {
fn describe(&self) -> String {
self.dir.display().to_string()
}
#[cfg(not(target_arch = "wasm32"))]
fn background_writer(&self) -> Option<Box<dyn Store>> {
Some(Box::new(FsStore { dir: self.dir.clone(), appender: None }))
}
}
/// Counts the bytes written through it.
struct Counter<W> {
inner: W,
n: u64,
}
impl<W: io::Write> io::Write for Counter<W> {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
let k = self.inner.write(buf)?;
self.n += k as u64;
Ok(k)
}
fn flush(&mut self) -> io::Result<()> {
self.inner.flush()
}
}
/// In-memory files. Clones share the same files (so a test can "reopen" or corrupt them).
#[derive(Clone, Default)]
pub struct MemStore {
pub files: Arc<Mutex<BTreeMap<String, Vec<u8>>>>,
/// Offer a [`Store::background_writer`] (off by default: snapshots stay synchronous).
pub background: bool,
}
impl MemStore {
@@ -145,4 +227,7 @@ impl Store for MemStore {
fn describe(&self) -> String {
"memory".into()
}
fn background_writer(&self) -> Option<Box<dyn Store>> {
self.background.then(|| Box::new(self.clone()) as Box<dyn Store>)
}
}
+272 -2
View File
@@ -58,12 +58,22 @@ fn albums_and_folders() {
let a = photo(&mut c, "a.jpg", "2026-04-01");
let f = c.alloc_album_id();
c.apply(Op::AddAlbum {
album: Album { id: f, name: "Trips".into(), parent: None, folder: true, photos: vec![], cover: None, smart: None, quick: false },
album: Album { id: f, name: "Trips".into(), parent: None, folder: true, photos: vec![], cover: None, smart: None, quick: false, order: None },
})
.unwrap();
let al = c.alloc_album_id();
c.apply(Op::AddAlbum {
album: Album { id: al, name: "Alps".into(), parent: Some(f), folder: false, photos: vec![a], cover: None, smart: None, quick: false },
album: Album {
id: al,
name: "Alps".into(),
parent: Some(f),
folder: false,
photos: vec![a],
cover: None,
smart: None,
quick: false,
order: None,
},
})
.unwrap();
assert_eq!(c.albums_of(a), vec![al]);
@@ -231,3 +241,263 @@ fn merge_results_are_recognised() {
assert_eq!(merged_kind("hdr.jpg"), None);
assert_eq!(merged_kind("panorama.jpg"), None);
}
/// Preview-only raws (an undecodable raw variant shown from its embedded JPEG): the reason is
/// stored with the photo, old catalogs without it still load, and Reload's op sets / clears it
/// with an exact inverse that survives serialisation (the journal).
#[test]
fn preview_only_survives_serde_and_set_content_undo() {
let mut c = Catalog::new();
let a = c.alloc_photo_id();
let mut p = Photo::new(a, Source::File { path: "/x/DSC_0001.NEF".into() }, "DSC_0001.NEF", "NEF", 6000, 4000, "2026-10-01T00:00:00");
p.kind = MediaKind::Raw;
p.preview_only = Some("Nikon Huffman-compressed NEF".into());
assert!(!p.develops_raw());
c.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
// the catalog round-trips it
let back: Catalog = serde_json::from_str(&serde_json::to_string(&c).unwrap()).unwrap();
assert_eq!(back.photo(a).unwrap().preview_only.as_deref(), Some("Nikon Huffman-compressed NEF"));
// an older catalog (no field) loads as decodable
let mut v = serde_json::to_value(&c).unwrap();
let photos = v["photos"].as_object_mut().unwrap();
let ph = photos.values_mut().next().unwrap();
ph.as_object_mut().unwrap().remove("preview_only").expect("serialised when set");
let old: Catalog = serde_json::from_value(v).unwrap();
assert_eq!(old.photo(a).unwrap().preview_only, None);
assert!(old.photo(a).unwrap().develops_raw());
// Reload: the file decodes now → cleared; undo (through the journal's JSON) restores it
let op = Op::SetContent { id: a, width: 6016, height: 4000, file_size: 1, content_hash: None, preview_only: None };
let inv = c.apply(op).unwrap();
assert!(c.photo(a).unwrap().develops_raw());
let inv: Op = serde_json::from_str(&serde_json::to_string(&inv).unwrap()).unwrap();
c.apply(inv).unwrap();
assert_eq!(c.photo(a).unwrap().preview_only.as_deref(), Some("Nikon Huffman-compressed NEF"));
assert_eq!(c.photo(a).unwrap().width, 6000);
}
/// Equivalent spellings of one folder (separators, trailing or doubled separators, `.`/`..`,
/// drive-letter case, verbatim and UNC prefixes) share an identity; different folders don't.
#[test]
fn folder_identity_ignores_spelling() {
use crate::query::{folder_key, folder_within};
let same = |a: &str, b: &str| assert_eq!(folder_key(a), folder_key(b), "{a} vs {b}");
let differ = |a: &str, b: &str| assert_ne!(folder_key(a), folder_key(b), "{a} vs {b}");
for (a, b) in [
("D:/Example/Photos", "D:\\Example\\Photos"),
("D:/Example/Photos", "d:\\Example\\Photos\\"),
("D:\\Example/Photos", "D:/Example//Photos/."),
("D:/Example/Photos", "D:/Example/Other/../Photos"),
("D:/Example/Photos", "\\\\?\\D:\\Example\\Photos"),
("\\\\server\\share\\Photos", "//server/share/Photos/"),
("\\\\?\\UNC\\server\\share\\Photos", "//server/share/Photos"),
("/home/me/Photos", "/home/me/Photos/"),
("/home/me/Photos", "/home//me/./Photos"),
("/", "//"),
("C:\\", "c:/"),
] {
same(a, b);
}
differ("D:/Example/Photos", "D:/Example/Photos2");
differ("D:/Example/Photos", "E:/Example/Photos");
differ("/home/me/Photos", "/home/me");
assert_eq!(folder_key("/a/../../b"), "/b", "`..` stops at the root");
assert!(folder_within("D:\\Example\\Photos\\2026", "D:/Example/Photos/"));
assert!(folder_within("D:/Example/Photos", "d:\\Example\\Photos"));
assert!(!folder_within("D:/Example/Photos2", "D:/Example/Photos"));
assert!(folder_within("/a/b", "/"));
assert!(folder_within("C:\\x", "c:\\"));
}
/// People come from named *face* regions: counted once per photo, case-insensitive, most photos
/// first; pets and unnamed faces are not people; the `person` filter and `person:` token match.
#[test]
fn people_from_named_face_regions() {
use lightcraft_meta::{Rect, Region, RegionKind};
let region = |name: Option<&str>, kind: RegionKind| Region {
rect: Rect { x0: 0.4, y0: 0.4, x1: 0.6, y1: 0.6 },
kind,
name: name.map(str::to_string),
description: None,
};
let mut c = Catalog::new();
let mut add = |name: &str, regions: Vec<Region>| {
let id = c.alloc_photo_id();
let mut p = Photo::new(id, Source::Demo { scene: 1 }, name, "JPEG", 6000, 4000, "2026-09-30T10:00:00");
p.meta.regions = regions;
c.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
id
};
let a = add(
"a.jpg",
vec![region(Some("Jane Doe"), RegionKind::Face), region(Some("jane doe"), RegionKind::Face), region(Some("Rex"), RegionKind::Pet)],
);
let b =
add("b.jpg", vec![region(Some("JANE DOE"), RegionKind::Face), region(Some("John Roe"), RegionKind::Face), region(None, RegionKind::Face)]);
let big = Rect { x0: 0.2, y0: 0.2, x1: 0.8, y1: 0.8 };
let d = add("d.jpg", vec![Region { rect: big, ..region(Some("John Roe"), RegionKind::Face) }]);
let e = add("e.jpg", vec![region(Some("Sam"), RegionKind::Face)]);
add("f.jpg", vec![]);
let people = c.people();
let summary: Vec<(&str, usize)> = people.iter().map(|p| (p.name.as_str(), p.count)).collect();
assert_eq!(summary, vec![("Jane Doe", 2), ("John Roe", 2), ("Sam", 1)], "once per photo, pets and unnamed faces left out");
// the picture is the person's largest face; ties go to the lower photo id
assert_eq!((people[0].photo, people[0].face), (a, Rect { x0: 0.4, y0: 0.4, x1: 0.6, y1: 0.6 }));
assert_eq!((people[1].photo, people[1].face), (d, big), "the larger face wins over an earlier, smaller one");
let q = |f: Filter| c.query(&f, &Sort::default());
assert_eq!(q(Filter { person: Some("jane doe".into()), ..Default::default() }).len(), 2);
let mut got = q(Filter { person: Some("John Roe".into()), ..Default::default() });
got.sort();
assert_eq!(got, vec![b, d]);
assert!(q(Filter { person: Some("Rex".into()), ..Default::default() }).is_empty(), "a pet is not a person");
assert_eq!(q(Filter { text: "person:SAM".into(), ..Default::default() }), vec![e], "the search token finds a person, any case");
assert!(q(Filter { person: Some("Jane Doe".into()), ..Default::default() }).contains(&a));
// other filters narrow who is offered (so picking a person never ends in an empty grid); the
// `person` filter itself does not
c.apply(Op::SetRating { id: b, rating: 3 }).unwrap();
let names = |f: Filter| c.people_in(&f).into_iter().map(|p| (p.name, p.count)).collect::<Vec<_>>();
let rated = Filter { rating: 1, ..Default::default() };
assert_eq!(
names(rated.clone()),
vec![("JANE DOE".to_string(), 1), ("John Roe".to_string(), 1)],
"only the rated photo's people, counted within it, spelled as first seen there"
);
assert_eq!(
names(Filter { person: Some("Sam".into()), ..rated }),
names(Filter { rating: 1, ..Default::default() }),
"the person filter is ignored"
);
assert_eq!(names(Filter { rating: 5, ..Default::default() }), vec![], "nobody in the filtered photos");
}
/// Most photos have no regions: the field is left out of the catalog JSON then, and a catalog written
/// before regions existed (no `regions` key) reads with none.
#[test]
fn empty_regions_are_not_serialized_and_default_when_missing() {
let m = Meta { keywords: vec!["k".into()], ..Default::default() };
let v = serde_json::to_value(&m).unwrap();
assert!(v.get("regions").is_none(), "{v}");
let back: Meta = serde_json::from_value(v).unwrap();
assert!(back.regions.is_empty());
let mut with = m.clone();
with.regions.push(lightcraft_meta::Region {
rect: lightcraft_meta::Rect { x0: 0.1, y0: 0.1, x1: 0.2, y1: 0.2 },
kind: lightcraft_meta::RegionKind::Face,
name: Some("A".into()),
description: None,
});
let back: Meta = serde_json::from_str(&serde_json::to_string(&with).unwrap()).unwrap();
assert_eq!(back, with);
}
// ---- random sort: Given a seed, the order is a stable, reproducible shuffle of the matching photos
fn many(c: &mut Catalog, n: usize) -> Vec<PhotoId> {
(0..n).map(|i| photo(c, &format!("p{i:03}.jpg"), &format!("2026-04-{:02}T10:00:00", i % 28 + 1))).collect()
}
fn random(seed: u64) -> Sort {
Sort { key: SortKey::Random, seed, ..Default::default() }
}
#[test]
fn random_sort_is_a_reproducible_permutation() {
let mut c = Catalog::new();
let ids = many(&mut c, 50);
let a = c.query(&Filter::default(), &random(7));
assert_eq!(a, c.query(&Filter::default(), &random(7)), "same seed, same order");
let mut sorted = a.clone();
sorted.sort();
let mut all = ids.clone();
all.sort();
assert_eq!(sorted, all, "every photo exactly once");
assert_ne!(a, c.query(&Filter::default(), &random(8)), "another seed, another order");
assert_ne!(a, c.query(&Filter::default(), &Sort::default()), "not just the date order");
}
#[test]
fn random_sort_keeps_the_relative_order_when_the_set_changes() {
let mut c = Catalog::new();
let ids = many(&mut c, 30);
let before = c.query(&Filter::default(), &random(3));
let added = photo(&mut c, "new.jpg", "2026-05-01T10:00:00");
c.apply(Op::SetRating { id: ids[0], rating: 5 }).unwrap();
let after: Vec<PhotoId> = c.query(&Filter::default(), &random(3)).into_iter().filter(|id| *id != added).collect();
assert_eq!(after, before, "adding a photo or editing metadata must not reshuffle the others");
}
#[test]
fn random_sort_respects_the_filter_and_handles_tiny_sets() {
let mut c = Catalog::new();
assert!(c.query(&Filter::default(), &random(1)).is_empty());
let ids = many(&mut c, 10);
c.apply(Op::SetRating { id: ids[4], rating: 5 }).unwrap();
assert_eq!(c.query(&Filter { rating: 5, ..Default::default() }, &random(1)), vec![ids[4]]);
}
#[test]
fn random_sort_direction_reverses_and_serde_defaults_hold() {
let mut c = Catalog::new();
many(&mut c, 20);
let desc = c.query(&Filter::default(), &Sort { ascending: false, ..random(5) });
let mut asc = c.query(&Filter::default(), &Sort { ascending: true, ..random(5) });
asc.reverse();
assert_eq!(desc, asc);
// saved sorts from before the seed existed still load
let old: Sort = serde_json::from_str(r#"{"key":"fileName","ascending":true}"#).unwrap();
assert_eq!(old.seed, 0);
assert_eq!(serde_json::from_str::<Sort>(r#"{"key":"random","seed":9}"#).unwrap(), Sort { key: SortKey::Random, seed: 9, ..Default::default() });
}
#[test]
fn random_sort_has_no_date_headers() {
let mut c = Catalog::new();
let ids = many(&mut c, 5);
assert!(c.date_runs(&ids, SortKey::Random, GroupBy::Day).is_empty());
}
#[test]
fn undated_photos_group_under_unknown_date_and_sort_together() {
let mut c = Catalog::new();
let add_photo = |c: &mut Catalog, name: &str, captured: Option<&str>, imported: &str| {
let id = c.alloc_photo_id();
let mut p = Photo::new(id, Source::Demo { scene: 0 }, name, "JPEG", 100, 100, imported);
p.captured = captured.map(str::to_string);
c.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
id
};
let p_dated1 = add_photo(&mut c, "dated1.jpg", Some("2026-05-10T12:00:00"), "2026-10-01T00:00:00");
let p_dated2 = add_photo(&mut c, "dated2.jpg", Some("2026-05-20T12:00:00"), "2026-10-01T00:00:00");
let p_undated1 = add_photo(&mut c, "undated1.jpg", None, "2026-10-05T00:00:00");
let p_undated2 = add_photo(&mut c, "undated2.jpg", None, "2026-10-02T00:00:00");
// Descending sort (default, newest first): dated photos first (newest to oldest), undated photos at the end
let desc = c.query(&Filter::default(), &Sort { key: SortKey::CaptureDate, ascending: false, ..Default::default() });
assert_eq!(desc, vec![p_dated2, p_dated1, p_undated1, p_undated2]);
// Ascending sort (oldest first): undated photos at the beginning, dated photos from oldest to newest
let asc = c.query(&Filter::default(), &Sort { key: SortKey::CaptureDate, ascending: true, ..Default::default() });
assert_eq!(asc, vec![p_undated2, p_undated1, p_dated1, p_dated2]);
// date_runs groups undated photos under Unknown Date
let runs = c.date_runs(&desc, SortKey::CaptureDate, GroupBy::Day);
assert_eq!(runs.len(), 3);
assert_eq!(runs[0].key, "2026-05-20");
assert_eq!(runs[1].key, "2026-05-10");
assert_eq!(runs[2].key, "");
assert_eq!(runs[2].label, "Unknown Date");
assert_eq!(runs[2].count, 2);
// date_groups sidebar tree only counts photos with capture dates
let groups = c.date_groups();
assert_eq!(groups.len(), 1);
assert_eq!(groups[0].year, "2026");
assert_eq!(groups[0].count, 2);
// Filter by capture date prefix does not match undated photos
let filter_day = Filter { date: Some("2026-10-05".into()), ..Default::default() };
assert_eq!(c.query(&filter_day, &Sort::default()), vec![]);
let filter_year = Filter { date: Some("2026".into()), ..Default::default() };
assert_eq!(c.query(&filter_year, &Sort { key: SortKey::CaptureDate, ascending: false, ..Default::default() }), vec![p_dated2, p_dated1]);
}
+122
View File
@@ -0,0 +1,122 @@
//! Album order: the children of a folder (or of the top level) are listed folders first, then
//! alphabetically, until some of them are given a place of their own (`Album.order`); moving an
//! album to another folder drops its place, and undoing the move brings it back.
use crate::*;
fn add(c: &mut Catalog, name: &str, parent: Option<AlbumId>, folder: bool) -> AlbumId {
let id = c.alloc_album_id();
c.apply(Op::AddAlbum { album: Album { parent, folder, ..Album::new(id, name) } }).unwrap();
id
}
fn names(c: &Catalog, parent: Option<AlbumId>) -> Vec<String> {
c.album_children(parent).iter().map(|a| a.name.clone()).collect()
}
fn place(c: &mut Catalog, id: AlbumId, order: Option<u32>) {
c.apply(Op::SetAlbumOrder { id, order }).unwrap();
}
/// Given albums that were never ordered, when they are listed, then folders come first and each
/// group is alphabetical regardless of case.
#[test]
fn children_are_folders_first_then_alphabetical() {
let mut c = Catalog::new();
add(&mut c, "beta", None, false);
add(&mut c, "Alpha", None, false);
add(&mut c, "zoo", None, true);
add(&mut c, "Archive", None, true);
assert_eq!(names(&c, None), ["Archive", "zoo", "Alpha", "beta"]);
assert!(!c.album_children_are_ordered(None));
}
/// Given some children with a place of their own, then they follow it; the others come after, by
/// name; folders stay ahead of albums.
#[test]
fn ordered_children_follow_their_order() {
let mut c = Catalog::new();
let a = add(&mut c, "A", None, false);
let b = add(&mut c, "B", None, false);
let cc = add(&mut c, "C", None, false);
add(&mut c, "D", None, false);
let f = add(&mut c, "F", None, true);
place(&mut c, cc, Some(0));
place(&mut c, a, Some(1));
assert_eq!(names(&c, None), ["F", "C", "A", "B", "D"]);
assert!(c.album_children_are_ordered(None));
// equal places fall back to the name, so a damaged catalog still lists deterministically
place(&mut c, b, Some(1));
assert_eq!(names(&c, None), ["F", "C", "A", "B", "D"]);
// another container is unaffected
let inner = add(&mut c, "Z", Some(f), false);
add(&mut c, "Y", Some(f), false);
assert_eq!(names(&c, Some(f)), ["Y", "Z"]);
assert!(!c.album_children_are_ordered(Some(f)));
place(&mut c, inner, Some(0));
assert_eq!(names(&c, Some(f)), ["Z", "Y"]);
}
/// The one-pass map lists every folder's children exactly as `album_children` does.
#[test]
fn the_children_map_matches_the_children_of_each_folder() {
let mut c = Catalog::new();
let f = add(&mut c, "F", None, true);
let g = add(&mut c, "G", Some(f), true);
for (name, parent) in [("b", None), ("a", None), ("y", Some(f)), ("x", Some(f)), ("q", Some(g))] {
add(&mut c, name, parent, false);
}
let x = c.albums().find(|a| a.name == "x").unwrap().id;
place(&mut c, x, Some(7));
let map = c.album_children_by_parent();
for parent in [None, Some(f), Some(g)] {
let from_map: Vec<AlbumId> = map.get(&parent).map(|v| v.iter().map(|a| a.id).collect()).unwrap_or_default();
let direct: Vec<AlbumId> = c.album_children(parent).iter().map(|a| a.id).collect();
assert_eq!(from_map, direct, "{parent:?}");
}
assert!(!map.contains_key(&Some(AlbumId(999))));
}
/// A place is one undo step, and `None` puts the album back among the alphabetical ones.
#[test]
fn a_place_can_be_set_cleared_and_undone() {
let mut c = Catalog::new();
let a = add(&mut c, "A", None, false);
add(&mut c, "B", None, false);
let inv = c.apply(Op::SetAlbumOrder { id: a, order: Some(5) }).unwrap();
assert_eq!(c.album(a).unwrap().order, Some(5));
c.apply(inv).unwrap();
assert_eq!(c.album(a).unwrap().order, None);
assert!(c.apply(Op::SetAlbumOrder { id: AlbumId(999), order: Some(1) }).is_err(), "no such album");
}
/// Moving to another folder drops the album's place there (it belonged to the old neighbours),
/// and undoing the move gives it back.
#[test]
fn moving_drops_the_place_and_undo_restores_it() {
let mut c = Catalog::new();
let from = add(&mut c, "From", None, true);
let to = add(&mut c, "To", None, true);
let a = add(&mut c, "A", Some(from), false);
place(&mut c, a, Some(3));
let inv = c.apply(Op::MoveAlbum { id: a, parent: Some(to) }).unwrap();
assert_eq!((c.album(a).unwrap().parent, c.album(a).unwrap().order), (Some(to), None));
c.apply(inv).unwrap();
assert_eq!((c.album(a).unwrap().parent, c.album(a).unwrap().order), (Some(from), Some(3)));
// "moving" to where it already is changes nothing
c.apply(Op::MoveAlbum { id: a, parent: Some(from) }).unwrap();
assert_eq!(c.album(a).unwrap().order, Some(3));
}
/// Catalogs written before albums had an order load as they were; an unset order is not written.
#[test]
fn albums_without_an_order_load_and_save_unchanged() {
let old = r#"{"id":4,"name":"Old","parent":null,"folder":false,"photos":[],"cover":null}"#;
let a: Album = serde_json::from_str(old).unwrap();
assert_eq!(a.order, None);
assert!(!serde_json::to_string(&a).unwrap().contains("order"));
let mut a = a;
a.order = Some(2);
let back: Album = serde_json::from_str(&serde_json::to_string(&a).unwrap()).unwrap();
assert_eq!(back.order, Some(2));
}
+339
View File
@@ -0,0 +1,339 @@
//! Background compaction (issue #37): a snapshot written by a worker while appends go on.
//! "Crashes" are simulated by copying the files at a given moment and loading the copy: every
//! acknowledged (appended) op must be there, in order, whatever the snapshot was doing.
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Condvar, Mutex};
use std::time::Duration;
use proptest::prelude::*;
use crate::journal::{LOG, SNAPSHOT};
use crate::*;
/// A MemStore whose snapshot writes wait for a gate (to hold a background snapshot mid-flight)
/// and can be made to fail.
#[derive(Clone)]
struct Gated {
files: MemStore,
gate: Arc<(Mutex<bool>, Condvar)>,
fail: Arc<AtomicBool>,
}
impl Gated {
fn new() -> Gated {
Gated { files: MemStore::new(), gate: Arc::new((Mutex::new(false), Condvar::new())), fail: Arc::new(AtomicBool::new(false)) }
}
fn set_gate(&self, open: bool) {
*self.gate.0.lock().unwrap() = open;
self.gate.1.notify_all();
}
/// The files as a crash at this moment would leave them.
fn crash(&self) -> MemStore {
let m = MemStore::new();
*m.files.lock().unwrap() = self.files.files.lock().unwrap().clone();
m
}
fn snapshot_seq_on_disk(&self) -> Option<u64> {
let v: serde_json::Value = serde_json::from_slice(&self.files.get(SNAPSHOT)?).ok()?;
v["seq"].as_u64()
}
}
impl Store for Gated {
fn read(&mut self, name: &str) -> std::io::Result<Option<Vec<u8>>> {
self.files.read(name)
}
fn write_atomic(&mut self, name: &str, data: &[u8]) -> std::io::Result<()> {
self.files.write_atomic(name, data)
}
fn write_atomic_with(&mut self, name: &str, fill: &mut dyn FnMut(&mut dyn std::io::Write) -> std::io::Result<()>) -> std::io::Result<u64> {
if name == SNAPSHOT {
let (lock, cv) = &*self.gate;
let (open, timeout) = cv.wait_timeout_while(lock.lock().unwrap(), Duration::from_secs(20), |open| !*open).unwrap();
if !*open || timeout.timed_out() {
return Err(std::io::Error::other("gate never opened"));
}
}
let mut buf = Vec::new();
fill(&mut buf)?;
if self.fail.load(Ordering::SeqCst) {
return Err(std::io::Error::other("disk full"));
}
self.files.write_atomic(name, &buf)?;
Ok(buf.len() as u64)
}
fn append(&mut self, name: &str, data: &[u8]) -> std::io::Result<()> {
self.files.append(name, data)
}
fn truncate(&mut self, name: &str, len: u64) -> std::io::Result<()> {
self.files.truncate(name, len)
}
fn describe(&self) -> String {
"gated".into()
}
fn background_writer(&self) -> Option<Box<dyn Store>> {
Some(Box::new(self.clone()))
}
}
/// Apply and durably append one edit (two ops in one append every fourth time).
fn edit(j: &mut Journal, live: &mut Catalog, k: u64) {
let one = |live: &mut Catalog, k: u64| {
let op = if live.len() < 4 || k.is_multiple_of(3) {
let id = live.alloc_photo_id();
Op::AddPhoto { photo: Box::new(Photo::new(id, Source::File { path: format!("/p/{}.jpg", id.0) }, "p.jpg", "JPEG", 4, 3, "t")) }
} else {
Op::SetRating { id: PhotoId(1 + k % live.len() as u64), rating: (k % 6) as u8 }
};
live.apply(op.clone()).unwrap();
op
};
let mut ops = vec![one(live, k)];
if k.is_multiple_of(4) {
ops.push(one(live, k / 4 + 1));
}
j.append(&ops).unwrap();
}
/// Load the files a crash now would leave: they must hold exactly the live state.
fn assert_recovers(g: &Gated, live: &Catalog) -> LoadReport {
let (_, c, r) = Journal::open(Box::new(g.crash())).unwrap();
assert_eq!(c.to_snapshot(), live.to_snapshot(), "{r:?}");
assert_eq!((r.failed, r.torn_bytes, r.damaged.clone()), (0, 0, None));
r
}
fn wait_written(j: &Journal) {
for _ in 0..20_000 {
if j.snapshot_written() {
return;
}
std::thread::sleep(Duration::from_millis(1));
}
panic!("background snapshot never finished");
}
#[test]
fn background_snapshot_keeps_edits_made_while_it_runs() {
let g = Gated::new();
let (mut j, mut live, _) = Journal::open(Box::new(g.clone())).unwrap();
for k in 0..30 {
edit(&mut j, &mut live, k);
}
let n = j.seq();
j.snapshot_in_background(&live).unwrap();
assert!(j.snapshot_running() && !j.wants_snapshot() && j.stats().snapshot_running);
// edits while the snapshot is being written are appended (durably) as usual
let mut during = 0;
for k in 30..50 {
edit(&mut j, &mut live, k);
during = j.seq() - n;
}
// crash before the snapshot landed: no snapshot yet, the whole log replays
let r = assert_recovers(&g, &live);
assert_eq!((r.snapshot_seq, r.replayed as u64), (0, j.seq()));
// the snapshot lands (op n); crash before the log was trimmed: its records are stale
g.set_gate(true);
wait_written(&j);
assert_eq!(g.snapshot_seq_on_disk(), Some(n));
for k in 50..55 {
edit(&mut j, &mut live, k);
during = j.seq() - n;
}
let r = assert_recovers(&g, &live);
assert_eq!((r.snapshot_seq, r.stale as u64, r.replayed as u64), (n, n, during));
// finishing trims the log to exactly the records after the snapshot, in order
assert!(j.poll().unwrap());
assert!(!j.snapshot_running());
assert_eq!((j.snapshot_seq(), j.log_records()), (n, during));
let log = String::from_utf8(g.files.get(LOG).unwrap()).unwrap();
let seqs: Vec<u64> = log.lines().map(|l| journal::decode_record(l).unwrap().0).collect();
assert_eq!(seqs, ((n + 1)..=j.seq()).collect::<Vec<_>>());
assert_eq!(j.log_bytes(), log.len() as u64);
let r = assert_recovers(&g, &live);
assert_eq!((r.snapshot_seq, r.stale, r.replayed as u64), (n, 0, during));
let s = j.stats();
assert!(s.last_snapshot.background && s.last_snapshot.records == n && !s.snapshot_running, "{s:?}");
// and appending goes on after it
edit(&mut j, &mut live, 99);
assert_recovers(&g, &live);
}
#[test]
fn failed_background_snapshot_loses_nothing_and_retries_later() {
let g = Gated::new();
let (mut j, mut live, _) = Journal::open(Box::new(g.clone())).unwrap();
j.policy = SnapshotPolicy { max_records: 10, max_bytes: u64::MAX };
for k in 0..12 {
edit(&mut j, &mut live, k);
}
assert!(j.wants_snapshot());
g.fail.store(true, Ordering::SeqCst);
g.set_gate(true);
j.snapshot_in_background(&live).unwrap();
edit(&mut j, &mut live, 12);
wait_written(&j);
assert!(j.poll().is_err());
assert_eq!((j.snapshot_seq(), j.stats().failed_snapshots), (0, 1));
// still the (empty) snapshot written when the library was created
assert_eq!(g.snapshot_seq_on_disk(), Some(0));
assert_recovers(&g, &live);
// no retry storm: the next attempt waits for another `max_records` records
assert!(!j.wants_snapshot());
g.fail.store(false, Ordering::SeqCst);
while !j.wants_snapshot() {
let k = j.seq();
edit(&mut j, &mut live, k);
}
j.snapshot_in_background(&live).unwrap();
j.wait().unwrap();
assert_eq!((j.snapshot_seq(), j.log_records()), (j.seq(), 0));
let r = assert_recovers(&g, &live);
assert_eq!((r.snapshot_seq, r.replayed), (j.seq(), 0));
}
/// A synchronous snapshot (close, "Optimize Library") waits for the background one, so the older
/// snapshot can't land after (and replace) the newer one.
#[test]
fn sync_snapshot_waits_for_the_background_one() {
let g = Gated::new();
let (mut j, mut live, _) = Journal::open(Box::new(g.clone())).unwrap();
for k in 0..10 {
edit(&mut j, &mut live, k);
}
j.snapshot_in_background(&live).unwrap();
for k in 10..20 {
edit(&mut j, &mut live, k);
}
let opener = {
let g = g.clone();
std::thread::spawn(move || {
std::thread::sleep(Duration::from_millis(50));
g.set_gate(true);
})
};
j.snapshot(&live).unwrap();
opener.join().unwrap();
assert!(!j.snapshot_running());
assert_eq!(g.snapshot_seq_on_disk(), Some(j.seq()));
assert_eq!((j.snapshot_seq(), j.log_records()), (j.seq(), 0));
assert!(g.files.get(LOG).unwrap().is_empty());
assert_eq!(j.stats().snapshots, 2);
assert_recovers(&g, &live);
}
/// Dropping the journal (e.g. reopening the library) waits for the snapshot in flight and leaves
/// the log whole: reopening finds every op.
#[test]
fn dropping_the_journal_waits_for_its_snapshot() {
let g = Gated::new();
let (mut j, mut live, _) = Journal::open(Box::new(g.clone())).unwrap();
for k in 0..10 {
edit(&mut j, &mut live, k);
}
let n = j.seq();
j.snapshot_in_background(&live).unwrap();
for k in 10..15 {
edit(&mut j, &mut live, k);
}
let opener = {
let g = g.clone();
std::thread::spawn(move || {
std::thread::sleep(Duration::from_millis(50));
g.set_gate(true);
})
};
let total = j.seq();
drop(j);
opener.join().unwrap();
assert_eq!(g.snapshot_seq_on_disk(), Some(n));
let (_, c, r) = Journal::open(Box::new(g.files.clone())).unwrap();
assert_eq!(c.to_snapshot(), live.to_snapshot());
assert_eq!((r.snapshot_seq, r.stale as u64, r.replayed as u64), (n, n, total - n));
}
/// On disk, with a real worker thread: a crash mid-write leaves only a temp file, which loading
/// ignores; edits during the snapshot survive; reopening resumes appends.
#[test]
fn fs_store_background_snapshot() {
let dir = std::env::temp_dir().join(format!("lc-journal-bg-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
let (mut j, mut live, _) = Journal::open(Box::new(FsStore::open(&dir).unwrap())).unwrap();
for k in 0..200 {
edit(&mut j, &mut live, k);
}
// a previous crash mid-snapshot left a partial temp file
std::fs::write(dir.join(format!("{SNAPSHOT}.tmp")), b"{\"format\":\"lightcraft-catal").unwrap();
let (mut j2, c2, r) = Journal::open(Box::new(FsStore::open(&dir).unwrap())).unwrap();
assert_eq!((c2.to_snapshot(), r.snapshot_seq), (live.to_snapshot(), 0));
drop(j);
let n = j2.seq();
j2.snapshot_in_background(&live).unwrap();
for k in 200..260 {
edit(&mut j2, &mut live, k);
}
j2.wait().unwrap();
assert_eq!((j2.snapshot_seq(), j2.log_records()), (n, j2.seq() - n));
assert!(!dir.join(format!("{SNAPSHOT}.tmp")).exists());
edit(&mut j2, &mut live, 999);
drop(j2);
let (_, c3, r) = Journal::open(Box::new(FsStore::open(&dir).unwrap())).unwrap();
assert_eq!(c3.to_snapshot(), live.to_snapshot());
assert_eq!((r.snapshot_seq, r.stale), (n, 0));
let _ = std::fs::remove_dir_all(&dir);
}
proptest! {
#![proptest_config(ProptestConfig::with_cases(48))]
/// Any interleaving of edits, background snapshots (started, landed, finished), synchronous
/// snapshots and crashes: a crash at any point recovers exactly the acknowledged ops.
#[test]
fn background_compaction_never_loses_acknowledged_ops(actions in proptest::collection::vec((0u8..8, 0u64..1000), 1..60)) {
let g = Gated::new();
let (mut j, mut live, _) = Journal::open(Box::new(g.clone())).unwrap();
for (ctl, k) in actions {
match ctl {
0..=2 => edit(&mut j, &mut live, k),
3 => j.snapshot_in_background(&live).unwrap(),
4 | 5 => {
// the worker writes its snapshot; finish it now or later
if j.snapshot_running() {
g.set_gate(true);
wait_written(&j);
g.set_gate(false);
if ctl == 4 {
prop_assert!(j.poll().unwrap());
}
}
}
6 => {
g.set_gate(true);
j.snapshot(&live).unwrap();
g.set_gate(false);
prop_assert_eq!(g.snapshot_seq_on_disk(), Some(j.seq()));
}
_ => {
// a crash now
let (_, c, r) = Journal::open(Box::new(g.crash())).unwrap();
prop_assert_eq!(c.to_snapshot(), live.to_snapshot());
prop_assert_eq!(r.failed, 0);
prop_assert!(r.damaged.is_none());
}
}
// the snapshot on disk never covers more than what was appended
prop_assert!(g.snapshot_seq_on_disk().unwrap_or(0) <= j.seq());
}
g.set_gate(true);
let (_, c, _) = Journal::open(Box::new(g.crash())).unwrap();
prop_assert_eq!(c.to_snapshot(), live.to_snapshot());
drop(j);
let (_, c, _) = Journal::open(Box::new(g.files.clone())).unwrap();
prop_assert_eq!(c.to_snapshot(), live.to_snapshot());
}
}
+404
View File
@@ -0,0 +1,404 @@
//! The library's folder tree: where the imported photos live on disk (see [`crate::folders`]).
//!
//! Scenarios, in the words of someone browsing their library:
//!
//! * Given photos imported from several folders, the tree lists those folders and no others,
//! each with how many photos it holds, under the disk (volume) they are on.
//! * A folder's count includes the folders inside it.
//! * Every folder is a row of its own, the ones above the imported photos included.
//! * Photos only browsed in Local, deleted photos and demo scenes are not "imported from" anywhere.
//! * However a folder is spelled, it is one folder.
//! * Choosing a folder shows what was imported from it and from the folders inside it — exactly
//! as many photos as the row says.
use crate::*;
fn add(c: &mut Catalog, path: &str) -> PhotoId {
add_with(c, path, |_| {})
}
fn add_with(c: &mut Catalog, path: &str, tweak: impl FnOnce(&mut Photo)) -> PhotoId {
let id = c.alloc_photo_id();
let name = path.rsplit(['/', '\\']).next().unwrap_or(path);
let mut p = Photo::new(id, Source::File { path: path.into() }, name, "JPEG", 60, 40, "2026-01-01T10:00:00");
tweak(&mut p);
c.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
id
}
fn library(paths: &[&str]) -> Catalog {
let mut c = Catalog::new();
paths.iter().for_each(|p| {
add(&mut c, p);
});
c
}
/// `(name, count, own)` of each node, depth first.
fn outline(nodes: &[FolderNode]) -> Vec<(String, usize, usize)> {
fn walk(n: &FolderNode, out: &mut Vec<(String, usize, usize)>) {
out.push((n.name.clone(), n.count, n.own));
n.children.iter().for_each(|c| walk(c, out));
}
let mut out = Vec::new();
nodes.iter().for_each(|n| walk(n, &mut out));
out
}
fn row(name: &str, count: usize, own: usize) -> (String, usize, usize) {
(name.to_string(), count, own)
}
fn rows(paths: &[&str]) -> Vec<(String, usize, usize)> {
outline(&library(paths).folder_tree())
}
#[test]
fn an_empty_library_has_no_folders() {
assert!(Catalog::new().folder_tree().is_empty());
}
#[test]
fn only_folders_with_imported_photos_are_listed_with_their_counts() {
let c = library(&["/pics/trip/a.jpg", "/pics/trip/b.jpg", "/pics/home/c.jpg"]);
let tree = c.folder_tree();
assert_eq!(outline(&tree), vec![row("/", 3, 0), row("pics", 3, 0), row("home", 1, 1), row("trip", 2, 2)]);
assert!(tree[0].volume && !tree[0].children[0].volume);
assert_eq!(tree[0].children[0].children[1].path, "/pics/trip", "a row knows its folder, for choosing it");
}
#[test]
fn a_folders_count_includes_the_folders_inside_it() {
let r = rows(&["/pics/trip/a.jpg", "/pics/trip/day1/b.jpg", "/pics/trip/day1/c.jpg", "/pics/trip/day2/d.jpg"]);
assert_eq!(r, vec![row("/", 4, 0), row("pics", 4, 0), row("trip", 4, 1), row("day1", 2, 2), row("day2", 1, 1)]);
}
#[test]
fn every_folder_is_a_row_of_its_own_even_when_it_holds_nothing_itself() {
let c = library(&["/Users/me/Pictures/2024/a.jpg", "/Users/me/Pictures/2025/b.jpg", "/Volumes/nas/photos/c.jpg"]);
let tree = c.folder_tree();
assert_eq!(
outline(&tree),
vec![
row("/", 2, 0),
row("Users", 2, 0),
row("me", 2, 0),
row("Pictures", 2, 0),
row("2024", 1, 1),
row("2025", 1, 1),
row("nas", 1, 0),
row("photos", 1, 1)
]
);
assert_eq!(tree[0].children[0].children[0].children[0].path, "/Users/me/Pictures", "a row is exactly the folder it names");
}
#[test]
fn a_folder_with_photos_of_its_own_is_not_folded_into_its_child() {
assert_eq!(rows(&["/pics/a.jpg", "/pics/trip/b.jpg"]), vec![row("/", 2, 0), row("pics", 2, 1), row("trip", 1, 1)]);
}
#[test]
fn browsed_deleted_and_demo_photos_are_not_imported_from_anywhere() {
let mut c = Catalog::new();
add(&mut c, "/pics/kept/a.jpg");
add_with(&mut c, "/pics/browsed/b.jpg", |p| p.local = true);
add_with(&mut c, "/pics/gone/c.jpg", |p| p.deleted = true);
let id = c.alloc_photo_id();
let demo = Photo::new(id, Source::Demo { scene: 1 }, "demo", "JPEG", 60, 40, "2026-01-01T10:00:00");
c.apply(Op::AddPhoto { photo: Box::new(demo) }).unwrap();
assert_eq!(outline(&c.folder_tree()), vec![row("/", 1, 0), row("pics", 1, 0), row("kept", 1, 1)]);
}
#[test]
fn however_a_folder_is_spelled_it_is_one_folder() {
let r = rows(&["/pics/trip/a.jpg", "/pics//trip/./b.jpg", "/pics\\trip\\c.jpg", "/pics/x/../trip/d.jpg"]);
assert_eq!(r, vec![row("/", 4, 0), row("pics", 4, 0), row("trip", 4, 4)]);
}
#[test]
fn each_disk_is_a_volume_of_its_own() {
// macOS, Linux and Windows habits for where disks appear
let r = rows(&["/Volumes/nas/p/1.jpg", "/media/me/usb/2.jpg", "/run/media/me/card/3.jpg", "/mnt/disk/4.jpg"]);
assert_eq!(r, vec![row("card", 1, 1), row("disk", 1, 1), row("nas", 1, 0), row("p", 1, 1), row("usb", 1, 1)]);
let r = rows(&["D:\\Photos\\2024\\a.jpg", "D:\\Photos\\2025\\b.jpg", "C:\\x\\c.jpg"]);
assert_eq!(r, vec![row("C:", 1, 0), row("x", 1, 1), row("D:", 2, 0), row("Photos", 2, 0), row("2024", 1, 1), row("2025", 1, 1)]);
}
#[test]
fn a_network_share_is_a_volume_and_two_spellings_are_one() {
let r = rows(&["\\\\srv\\s1\\a\\1.jpg", "\\\\srv\\s2\\b\\2.jpg"]);
assert_eq!(r, vec![row("//srv/s1", 1, 0), row("a", 1, 1), row("//srv/s2", 1, 0), row("b", 1, 1)]);
// the verbatim form Windows APIs give is the same share
let r = rows(&["\\\\?\\UNC\\srv\\s1\\a\\1.jpg", "\\\\srv\\s1\\a\\2.jpg"]);
assert_eq!(r, vec![row("//srv/s1", 2, 0), row("a", 2, 2)]);
}
#[test]
fn a_samba_share_mounted_on_a_mac_is_a_volume_named_after_the_share() {
// Finder mounts every share under /Volumes, and a second mount of the same share gets a suffix
let r = rows(&["/Volumes/tokyo/Photos/2024/a.jpg", "/Volumes/tokyo/Photos/2025/b.jpg", "/Volumes/tokyo-1/old/c.jpg", "/Volumes/東京 NAS/d.jpg"]);
assert_eq!(
r,
vec![
row("tokyo", 2, 0),
row("Photos", 2, 0),
row("2024", 1, 1),
row("2025", 1, 1),
row("tokyo-1", 1, 0),
row("old", 1, 1),
row("東京 NAS", 1, 1)
]
);
let tree = library(&["/Volumes/tokyo/Photos/a.jpg"]).folder_tree();
assert_eq!((tree[0].path.as_str(), tree[0].volume), ("/Volumes/tokyo", true), "choosing the share shows everything on it");
}
#[test]
fn shares_the_other_systems_mount_are_volumes_too() {
// Linux desktops: gvfs mounts per user, with the server and share in the mount's name
let gvfs = "/run/user/1000/gvfs/smb-share:server=tokyo,share=photos";
let r = rows(&[&format!("{gvfs}/2024/a.jpg"), "/run/user/1000/gvfs/sftp:host=nas/b.jpg"]);
assert_eq!(r, vec![row("photos on tokyo", 1, 0), row("2024", 1, 1), row("sftp:host=nas", 1, 1)]);
// NFS automounts: Linux /net/<host>, macOS /Network/Servers/<host>
let r = rows(&["/net/tokyo/export/a.jpg", "/Network/Servers/osaka/vol/b.jpg"]);
assert_eq!(r, vec![row("osaka", 1, 0), row("vol", 1, 1), row("tokyo", 1, 0), row("export", 1, 1)]);
// Windows: a share by name or by address, the WSL file system, a mapped drive
let r = rows(&[r"\\tokyo.local\photos\a.jpg", r"\\192.168.1.5\scans\b.jpg", r"\\wsl$\Ubuntu\home\me\c.jpg", r"Z:\d.jpg"]);
assert_eq!(
r,
vec![
row("//192.168.1.5/scans", 1, 1),
row("//tokyo.local/photos", 1, 1),
row("//wsl$/Ubuntu", 1, 0),
row("home", 1, 0),
row("me", 1, 1),
row("Z:", 1, 1)
]
);
}
#[test]
fn a_share_that_is_not_mounted_costs_nothing() {
// the tree is made from the catalog's own records: it never looks at a disk, so a share that
// is asleep or gone lists (and counts) exactly like one that is there
let c = library(&["/Volumes/definitely-not-mounted/a/b.jpg", r"\\unreachable.invalid\share\c.jpg"]);
let tree = c.folder_tree();
assert_eq!(tree.iter().map(|n| n.count).sum::<usize>(), 2);
}
#[test]
fn a_photo_in_the_root_of_a_disk_counts_for_the_disk() {
assert_eq!(rows(&["/x.jpg", "/d/y.jpg"]), vec![row("/", 2, 1), row("d", 1, 1)]);
assert_eq!(rows(&["C:\\x.jpg"]), vec![row("C:", 1, 1)]);
}
#[test]
fn photos_with_no_folder_are_left_out() {
assert!(rows(&["x.jpg", ""]).is_empty());
}
#[test]
fn folders_are_listed_by_name_ignoring_case() {
let names: Vec<String> = rows(&["/p/b/1.jpg", "/p/A/2.jpg", "/p/c/3.jpg"]).into_iter().skip(2).map(|r| r.0).collect();
assert_eq!(names, ["A", "b", "c"]);
}
#[test]
fn absurdly_deep_paths_neither_crash_nor_lose_photos() {
let deep = format!("/{}/a.jpg", vec!["d"; 5000].join("/"));
let tree = library(&[&deep]).folder_tree();
assert_eq!(tree.iter().map(|n| n.count).sum::<usize>(), 1);
assert!(outline(&tree).len() <= crate::folders::MAX_DEPTH + 2);
}
#[test]
fn choosing_a_folder_shows_what_was_imported_from_it_and_below() {
let mut c = Catalog::new();
let trip = add(&mut c, "/pics/trip/a.jpg");
let day = add(&mut c, "/pics/trip/day1/b.jpg");
let sibling = add(&mut c, "/pics/trip2/c.jpg"); // shares the name's start, not the folder
let browsed = add_with(&mut c, "/pics/trip/d.jpg", |p| p.local = true);
let f = Filter { library_folder: Some("/pics//trip/".into()), ..Default::default() };
let shown = |id: PhotoId| f.matches(c.photo(id).unwrap(), &c);
assert!(shown(trip) && shown(day));
assert!(!shown(sibling), "/pics/trip2 is another folder");
assert!(!shown(browsed), "browsed photos are not part of the library's folders");
assert_eq!(f.describe(), "folder /pics//trip/");
}
#[test]
fn recently_deleted_shows_the_deleted_photos_of_the_chosen_folder() {
let mut c = Catalog::new();
let gone = add_with(&mut c, "/pics/trip/a.jpg", |p| p.deleted = true);
let kept = add(&mut c, "/pics/trip/b.jpg");
let f = Filter { library_folder: Some("/pics/trip".into()), deleted: true, ..Default::default() };
assert!(f.matches(c.photo(gone).unwrap(), &c));
assert!(!f.matches(c.photo(kept).unwrap(), &c));
}
#[test]
fn an_empty_folder_choice_is_no_choice() {
let c = library(&["/pics/a.jpg"]);
let f = Filter { library_folder: Some(" ".into()), ..Default::default() };
assert_eq!(c.query(&f, &Sort::default()).len(), 1);
}
#[test]
fn every_row_shows_exactly_as_many_photos_as_it_says() {
let c = library(&[
"/Users/me/Pictures/2024/a.jpg",
"/Users/me/Pictures/2024/b.jpg",
"/Users/me/Pictures/2025/c.jpg",
"/Users/me/Pictures/d.jpg",
"/Volumes/nas/photos/e.jpg",
"/Volumes/nas/f.jpg",
"/x.jpg",
"/a/../b/g.jpg",
"D:\\Photos\\2024\\h.jpg",
"d:/Photos/i.jpg",
"\\\\srv\\s1\\a\\j.jpg",
"\\\\?\\UNC\\srv\\s2\\k.jpg",
"/pics/trip/l.jpg",
"/pics/trip2/m.jpg",
"/Volumes/tokyo/Photos/2024/n.jpg",
"/Volumes/tokyo/o.jpg",
"/run/user/1000/gvfs/smb-share:server=tokyo,share=photos/p.jpg",
"/net/osaka/export/q.jpg",
r"\\wsl$\Ubuntu\home\me\r.jpg",
// odd spellings the tree and the filter must read the same way
"//server/share/../other/1.jpg",
"//server/share/ok/2.jpg",
"//a b//A/f.jpg",
r"\\wsl$\Ubuntu\..\a\1.jpg",
// a folder that holds other disks as well as photos of its own
"/Volumes/1.jpg",
"/mnt/2.jpg",
"/mnt/disk/3.jpg",
]);
fn check(c: &Catalog, nodes: &[FolderNode], seen: &mut usize) {
for n in nodes {
// the startup volume's row only opens: its path "/" would cover every other disk
if n.selectable {
let f = Filter { library_folder: Some(n.path.clone()), ..Default::default() };
assert_eq!(c.query(&f, &Sort::default()).len(), n.count, "{} ({})", n.name, n.path);
*seen += 1;
}
check(c, &n.children, seen);
}
}
let mut seen = 0;
check(&c, &c.folder_tree(), &mut seen);
assert!(seen > 10, "{seen} rows checked");
}
fn find<'a>(nodes: &'a [FolderNode], path: &str) -> Option<&'a FolderNode> {
nodes.iter().find_map(|n| if n.path == path { Some(n) } else { find(&n.children, path) })
}
#[test]
fn a_row_that_would_cover_other_disks_is_not_offered_for_choosing() {
let tree = library(&["/Users/me/a.jpg", "/Volumes/1.jpg", "/Volumes/tokyo/x.jpg", "/mnt/2.jpg"]).folder_tree();
let sel = |p: &str| find(&tree, p).map(|n| n.selectable);
assert_eq!(sel("/"), Some(false), "the startup disk's path covers every disk");
assert_eq!(sel("/Volumes"), Some(false), "it holds the tokyo disk");
assert_eq!(sel("/Volumes/tokyo"), Some(true));
assert_eq!(sel("/Users/me"), Some(true));
assert_eq!(sel("/mnt"), Some(true), "no disk is mounted below it in this library");
}
#[test]
fn the_tree_and_the_filter_read_odd_spellings_the_same_way() {
// `..` cannot climb above a share, and repeated separators between server and share are one
let tree = library(&["//server/share/../other/1.jpg", "//a b//A/f.jpg"]).folder_tree();
assert_eq!(outline(&tree), vec![row("//a b/A", 1, 1), row("//server/share", 1, 0), row("other", 1, 1)]);
}
#[test]
fn a_path_that_names_no_folder_chooses_nothing() {
let c = library(&["/pics/a.jpg", "C:\\b\\c.jpg"]);
for dot in [".", "./", "a/..", "../.."] {
let f = Filter { library_folder: Some(dot.into()), ..Default::default() };
assert_eq!(c.query(&f, &Sort::default()).len(), 0, "{dot:?} is no folder, not everything");
}
}
#[test]
fn a_folder_is_labelled_by_its_last_two_names() {
use crate::folders::folder_label;
assert_eq!(folder_label("/Volumes/tokyo/photos/travel"), "photos/travel");
assert_eq!(folder_label("/Volumes/tokyo/"), "Volumes/tokyo");
assert_eq!(folder_label(r"C:\Photos\2024"), "Photos/2024");
assert_eq!(folder_label("/"), "/");
assert_eq!(folder_label(""), "");
}
#[test]
fn disk_roots_are_told_from_folders() {
use crate::folders::{is_disk_root, is_startup_disk};
for p in ["/", "C:", "C:\\", r"\\?\C:\", r"\\srv\share", "/Volumes/nas", "/mnt/disk", "/net/host"] {
assert!(is_disk_root(p), "{p}");
}
for p in ["/Volumes", "/Users", "C:\\Photos", r"\\srv\share\x", "/Volumes/nas/p", "."] {
assert!(!is_disk_root(p), "{p}");
}
assert!(is_startup_disk("/") && is_startup_disk("//") && !is_startup_disk("/Volumes/nas") && !is_startup_disk("C:"));
}
#[test]
fn a_folder_name_with_spaces_at_its_edge_is_its_own_folder() {
// legal on macOS and Linux: `/p/shoot ` is not `/p/shoot`
let c = library(&["/p/shoot /a.jpg", "/p/shoot/b.jpg", "/p/other/c.jpg"]);
let tree = c.folder_tree();
let shoot_space = find(&tree, "/p/shoot ").expect("its own row");
let shoot = find(&tree, "/p/shoot").expect("and the other");
for (node, want) in [(shoot_space, "a.jpg"), (shoot, "b.jpg")] {
let f = Filter { library_folder: Some(node.path.clone()), ..Default::default() };
let ids = c.query(&f, &Sort::default());
let names: Vec<&str> = ids.iter().filter_map(|id| c.photo(*id)).map(|p| p.file_name.as_str()).collect();
assert_eq!(names, [want], "{:?} shows its own photo", node.path);
}
}
#[test]
fn a_local_photo_never_counts_as_imported_even_when_a_disk_folder_is_also_shown() {
let mut c = Catalog::new();
let browsed = add_with(&mut c, "/pics/trip/b.jpg", |p| p.local = true);
let f = Filter { folder: Some("/pics/trip".into()), library_folder: Some("/pics/trip".into()), ..Default::default() };
assert!(!f.matches(c.photo(browsed).unwrap(), &c));
}
#[test]
fn the_quick_check_agrees_with_the_careful_one() {
use crate::query::{folder_key, key_within, photo_in_root};
let paths = [
"/a/b/c.jpg",
"/a/b/d/e.jpg",
"/a/bc/x.jpg",
"/a/./b/x.jpg",
"/a/../a/b/x.jpg",
"/a//b/x.jpg",
"/a\\b\\x.jpg",
"/a/b ",
"/a/b /x.jpg",
"/.hidden/x.jpg",
"/a/.b/x.jpg",
"/x.jpg",
"//srv/share/x.jpg",
"C:\\a\\b\\x.jpg",
"relative/a/b/x.jpg",
"/Volumes/tokyo/a/x.jpg",
"/日本/写真/x.jpg",
];
let roots = ["/", "/a", "/a/b", "/a/b/", "/a/bc", "/a/b ", "/Volumes/tokyo", "//srv/share", "C:\\a", "/日本", "relative", "/a/../a/b"];
for p in paths {
let mut c = Catalog::new();
let id = add(&mut c, p);
let photo = c.photo(id).unwrap();
for r in roots {
let root = folder_key(r);
let slow = crate::local::folder_of(photo).is_some_and(|f| key_within(&f, &root));
assert_eq!(photo_in_root(photo, &root), slow, "{p:?} in {r:?}");
}
}
}
+200
View File
@@ -0,0 +1,200 @@
//! Issue #102: catalog format versions. Older formats load and are upgraded; a newer format is
//! refused with a clear error and left untouched (never read in part, truncated or set aside).
use crate::journal::{LOG, SNAPSHOT, VERSION, encode_record};
use crate::*;
fn photo(c: &mut Catalog) -> Op {
let id = c.alloc_photo_id();
Op::AddPhoto { photo: Box::new(Photo::new(id, Source::File { path: format!("/p/{}.jpg", id.0) }, "p.jpg", "JPEG", 4, 3, "2026-01-01")) }
}
/// A catalog with two photos and the log line for a third op on top of it.
fn legacy_parts() -> (Catalog, String, Catalog) {
let mut c = Catalog::new();
for _ in 0..2 {
let op = photo(&mut c);
c.apply(op).unwrap();
}
let base = c.clone();
let id = c.photos().next().unwrap().id;
let op = Op::SetRating { id, rating: 4 };
c.apply(op.clone()).unwrap();
(base, format!("{}\n", encode_record(3, &op)), c)
}
fn snapshot_version(m: &MemStore) -> u64 {
let v: serde_json::Value = serde_json::from_slice(&m.get(SNAPSHOT).unwrap()).unwrap();
v["version"].as_u64().unwrap()
}
#[test]
fn v1_library_loads_and_is_upgraded() {
let (base, log, full) = legacy_parts();
let m = MemStore::new();
m.set(SNAPSHOT, format!("{{\"format\":\"lightcraft-catalog\",\"version\":1,\"seq\":2,\"catalog\":{}}}\n", base.to_snapshot()).into_bytes());
m.set(LOG, log.into_bytes());
let (mut j, c, r) = Journal::open(Box::new(m.clone())).unwrap();
assert_eq!(c.to_snapshot(), full.to_snapshot());
assert_eq!((r.replayed, r.upgraded_from), (1, Some(1)));
// rewritten in this format before anything is appended
assert_eq!(snapshot_version(&m), u64::from(VERSION));
assert!(m.get(LOG).unwrap().is_empty());
assert_eq!(j.seq(), 3);
j.append(&[Op::SetLabelName { label: ColorLabel::ALL[0], name: Some("x".into()) }]).unwrap();
drop(j);
let (_, _, r) = Journal::open(Box::new(m)).unwrap();
assert_eq!((r.replayed, r.upgraded_from), (1, None));
}
/// A library saved before albums could be ordered by hand opens as it was and is rewritten as v3.
#[test]
fn v2_library_loads_and_is_upgraded() {
let (base, log, full) = legacy_parts();
let m = MemStore::new();
m.set(SNAPSHOT, format!("{{\"format\":\"lightcraft-catalog\",\"version\":2,\"seq\":2,\"catalog\":{}}}\n", base.to_snapshot()).into_bytes());
m.set(LOG, log.into_bytes());
let (_, c, r) = Journal::open(Box::new(m.clone())).unwrap();
assert_eq!(c.to_snapshot(), full.to_snapshot());
assert_eq!((r.replayed, r.upgraded_from), (1, Some(2)));
assert_eq!(snapshot_version(&m), 3);
}
#[test]
fn versionless_log_only_library_is_upgraded() {
let mut c = Catalog::new();
let op = photo(&mut c);
c.apply(op.clone()).unwrap();
let m = MemStore::new();
m.set(LOG, format!("{}\n", encode_record(1, &op)).into_bytes());
let (_, loaded, r) = Journal::open(Box::new(m.clone())).unwrap();
assert_eq!(loaded.to_snapshot(), c.to_snapshot());
assert_eq!(r.upgraded_from, Some(0));
assert_eq!(snapshot_version(&m), u64::from(VERSION));
}
/// A new library gets a snapshot in this format at once, so an older build never sees a bare log
/// it would read in part.
#[test]
fn new_library_is_written_in_this_format() {
let m = MemStore::new();
let (_, _, r) = Journal::open(Box::new(m.clone())).unwrap();
assert!(r.created);
assert_eq!(r.upgraded_from, None);
assert_eq!(snapshot_version(&m), u64::from(VERSION));
}
fn assert_refused_untouched(m: &MemStore) {
let before = m.files.lock().unwrap().clone();
let e = Journal::open(Box::new(m.clone())).err().expect("refused");
assert!(matches!(e, CatalogError::Newer(_)), "{e:?}");
let msg = e.to_string();
assert!(msg.contains("newer version of LightCraft") && msg.contains("left unchanged"), "{msg}");
assert_eq!(*m.files.lock().unwrap(), before, "nothing modified, nothing renamed");
}
#[test]
fn newer_snapshot_is_refused_untouched() {
let (base, log, _) = legacy_parts();
let m = MemStore::new();
m.set(
SNAPSHOT,
format!("{{\"format\":\"lightcraft-catalog\",\"version\":{},\"seq\":2,\"catalog\":{}}}\n", VERSION + 1, base.to_snapshot()).into_bytes(),
);
m.set(LOG, log.into_bytes());
assert_refused_untouched(&m);
// even when its catalog doesn't parse as this version's
let m = MemStore::new();
m.set(
SNAPSHOT,
format!("{{\"format\":\"lightcraft-catalog\",\"version\":{},\"seq\":2,\"catalog\":{{\"photos\":7}}}}\n", VERSION + 1).into_bytes(),
);
assert_refused_untouched(&m);
let e = Journal::open(Box::new(m)).err().unwrap().to_string();
assert!(e.contains(&format!("v{}", VERSION + 1)), "{e}");
}
/// A log record with a valid CRC whose op this version doesn't know: refused, not taken for a
/// torn tail (truncated) or damage (renamed) — wherever it is in the log.
#[test]
fn unknown_op_in_the_log_is_refused_untouched() {
let (base, log, _) = legacy_parts();
let future = |seq: u64| {
let body = r#"{"op":"fromTheFuture","id":1,"what":"something new"}"#;
format!("{{\"seq\":{seq},\"crc\":{},\"op\":{body}}}\n", crc32fast::hash(body.as_bytes()))
};
let snap = format!("{{\"format\":\"lightcraft-catalog\",\"version\":{VERSION},\"seq\":2,\"catalog\":{}}}\n", base.to_snapshot());
let id = base.photos().next().unwrap().id;
let after = format!("{}\n", encode_record(5, &Op::SetRating { id, rating: 1 }));
for log in [format!("{log}{}", future(4)), format!("{log}{}{after}", future(4)), format!("{}{log}", future(3))] {
let m = MemStore::new();
m.set(SNAPSHOT, snap.clone().into_bytes());
m.set(LOG, log.into_bytes());
assert_refused_untouched(&m);
}
// a known op with a field of an unknown shape is "newer" too
let body = r#"{"op":"setRating","id":1,"rating":"five stars"}"#;
let m = MemStore::new();
m.set(SNAPSHOT, snap.into_bytes());
m.set(LOG, format!("{log}{{\"seq\":4,\"crc\":{},\"op\":{body}}}\n", crc32fast::hash(body.as_bytes())).into_bytes());
assert_refused_untouched(&m);
}
/// A CRC mismatch is still damage / a torn tail (not "newer").
#[test]
fn bad_crc_is_still_a_torn_tail() {
let (base, log, _) = legacy_parts();
let m = MemStore::new();
m.set(
SNAPSHOT,
format!("{{\"format\":\"lightcraft-catalog\",\"version\":{VERSION},\"seq\":2,\"catalog\":{}}}\n", base.to_snapshot()).into_bytes(),
);
let body = r#"{"op":"fromTheFuture"}"#;
m.set(LOG, format!("{log}{{\"seq\":4,\"crc\":1,\"op\":{body}}}\n").into_bytes());
let (_, _, r) = Journal::open(Box::new(m)).unwrap();
assert!(r.torn_bytes > 0 && r.damaged.is_none(), "{r:?}");
}
/// Guard for the version rule (see `journal` → *Format versions*): adding an [`Op`] variant breaks
/// this match. When it does, bump `journal::VERSION`, add a row to the format table, and add the
/// variant here under the new version.
#[test]
fn op_variants_are_versioned() {
fn since(op: &Op) -> u32 {
match op {
Op::AddPhoto { .. }
| Op::RemovePhoto { .. }
| Op::SetRating { .. }
| Op::SetFlag { .. }
| Op::SetLabel { .. }
| Op::SetDevelop { .. }
| Op::SetMeta { .. }
| Op::SetDeleted { .. }
| Op::SetLocal { .. }
| Op::SetVersions { .. }
| Op::SetHistory { .. }
| Op::PushHistory { .. }
| Op::AddAlbum { .. }
| Op::RemoveAlbum { .. }
| Op::RenameAlbum { .. }
| Op::MoveAlbum { .. }
| Op::SetAlbumPhotos { .. }
| Op::SetAlbumCover { .. }
| Op::SetAlbumRules { .. }
| Op::AddStack { .. }
| Op::RemoveStack { .. }
| Op::SetStack { .. }
| Op::SetCaptured { .. }
| Op::SetAnalysis { .. }
| Op::SetFile { .. }
| Op::Relink { .. }
| Op::SetContent { .. }
| Op::SetLabelName { .. }
| Op::Batch { .. } => 1,
Op::SetBrowsed { .. } => 2,
Op::SetAlbumOrder { .. } => 3,
}
}
let newest = since(&Op::SetAlbumOrder { id: AlbumId(0), order: None });
assert_eq!(newest, VERSION, "the newest op's version must be the current format version");
}
+52
View File
@@ -59,6 +59,7 @@ fn op_for(c: &mut Catalog, kind: u8, a: u8, b: u8) -> Op {
cover: None,
smart: None,
quick: false,
order: None,
},
}
}
@@ -77,6 +78,7 @@ fn op_for(c: &mut Catalog, kind: u8, a: u8, b: u8) -> Op {
13 => Op::Batch { ops: c.remove_from_stacks_ops(&[pid]) },
14 => c.set_top_ops(pid).unwrap_or(Op::Batch { ops: vec![] }),
9 if b.is_multiple_of(5) => Op::SetLabelName { label: ColorLabel::ALL[a as usize % 5], name: (a % 3 != 1).then(|| format!("Label {b}")) },
9 if b % 7 == 3 => Op::SetAlbumOrder { id: aid, order: (!b.is_multiple_of(3)).then_some(u32::from(b % 5)) },
9 if b.is_multiple_of(2) => {
Op::SetFile { id: pid, file_name: format!("Renamed_{b}.jpg"), source: Source::File { path: format!("/p/Renamed_{b}.jpg") } }
}
@@ -274,3 +276,53 @@ fn fs_store_roundtrip() {
assert_eq!(c3.to_snapshot(), c.to_snapshot());
let _ = std::fs::remove_dir_all(&dir);
}
/// The streamed snapshot is byte-for-byte the format written before streaming (issue #37), so
/// existing libraries load and older builds can read new snapshots; FsStore and MemStore agree.
#[test]
fn streamed_snapshot_keeps_the_on_disk_format() {
let (m, _, full) = sample_catalog_with_log();
let (mut j, c, _) = open(&m);
assert_eq!(c.to_snapshot(), full.to_snapshot());
j.snapshot(&c).unwrap();
let legacy = format!(
"{{\"format\":\"lightcraft-catalog\",\"version\":{},\"seq\":{},\"catalog\":{}}}\n",
crate::journal::VERSION,
j.seq(),
c.to_snapshot()
);
assert_eq!(String::from_utf8(m.get(SNAPSHOT).unwrap()).unwrap(), legacy);
assert_eq!(j.stats().last_snapshot.bytes, legacy.len() as u64);
let dir = std::env::temp_dir().join(format!("lc-journal-stream-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
// an old-style snapshot (written by a build before streaming) loads, and is rewritten the same
std::fs::write(dir.join(SNAPSHOT), legacy.as_bytes()).unwrap();
let (mut fj, fc, r) = Journal::open(Box::new(FsStore::open(&dir).unwrap())).unwrap();
assert_eq!((fc.to_snapshot(), r.snapshot_seq), (c.to_snapshot(), j.seq()));
fj.snapshot(&fc).unwrap();
assert_eq!(std::fs::read(dir.join(SNAPSHOT)).unwrap(), legacy.as_bytes());
drop(fj);
let _ = std::fs::remove_dir_all(&dir);
}
/// A snapshot that fails while streaming (serialisation or I/O error) leaves the old file in
/// place and removes its temp file.
#[test]
fn failed_streamed_write_keeps_the_old_file() {
let dir = std::env::temp_dir().join(format!("lc-journal-streamfail-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
let mut s = FsStore::open(&dir).unwrap();
s.write_atomic(SNAPSHOT, b"old").unwrap();
let r = s.write_atomic_with(SNAPSHOT, &mut |w| {
w.write_all(b"partial new content")?;
Err(std::io::Error::other("serialisation failed"))
});
assert!(r.is_err());
assert_eq!(std::fs::read(dir.join(SNAPSHOT)).unwrap(), b"old");
assert!(!dir.join(format!("{SNAPSHOT}.tmp")).exists());
assert_eq!(s.write_atomic_with(SNAPSHOT, &mut |w| w.write_all(b"new")).unwrap(), 3);
assert_eq!(std::fs::read(dir.join(SNAPSHOT)).unwrap(), b"new");
let _ = std::fs::remove_dir_all(&dir);
}
+267
View File
@@ -0,0 +1,267 @@
//! Forgetting untouched Local browse records (see [`crate::local`]).
use std::sync::Arc;
use lightcraft_develop::DevelopSettings;
use crate::*;
const OLD: &str = "2026-08-01T10:00:00";
const NOW: &str = "2026-10-05T10:00:00";
/// A Local record of `/pics/<folder>/IMG_<i>.JPG` as a browse catalogues it (with its baseline).
fn local(c: &mut Catalog, folder: &str, i: u32) -> PhotoId {
let id = c.alloc_photo_id();
let mut p = Photo::new(id, Source::File { path: format!("/pics/{folder}/IMG_{i}.JPG") }, &format!("IMG_{i}.JPG"), "JPEG", 60, 40, OLD);
p.meta.camera = "Cam".into();
p.meta.creator = "From EXIF".into(); // file metadata is part of the baseline, not a user change
p.focal_and_wb_for_test();
p.local = true;
p.set_local_baseline();
c.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
id
}
impl Photo {
fn focal_and_wb_for_test(&mut self) {
self.meta.focal_mm = Some(35.3);
self.as_shot_wb = Some((5123.4, 3.1));
}
}
fn browsed(c: &mut Catalog, folder: &str, at: &str) {
c.apply(Op::SetBrowsed { folder: format!("/pics/{folder}"), at: Some(at.into()) }).unwrap();
}
fn plan(c: &Catalog, days: u32) -> ForgetPlan {
c.forget_local_plan(NOW, days, &|_| false)
}
fn run(c: &mut Catalog, days: u32) -> ForgetPlan {
let p = plan(c, days);
for op in p.ops() {
c.apply(op).unwrap();
}
p
}
#[test]
fn untouched_records_of_old_folders_are_forgotten() {
let mut c = Catalog::new();
let old: Vec<_> = (0..5).map(|i| local(&mut c, "old", i)).collect();
let recent: Vec<_> = (0..3).map(|i| local(&mut c, "recent", i)).collect();
browsed(&mut c, "old", OLD);
browsed(&mut c, "recent", "2026-10-01T09:00:00");
// a library photo in the old folder is never a candidate
let lib = c.alloc_photo_id();
let mut p = Photo::new(lib, Source::File { path: "/pics/old/LIB.JPG".into() }, "LIB.JPG", "JPEG", 6, 4, OLD);
p.set_local_baseline();
c.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
let dry = plan(&c, 30);
assert_eq!((dry.local, dry.evict.len(), dry.kept_recent), (8, 5, 3), "{dry:?}");
assert_eq!(dry.evict, old);
assert_eq!(c.len(), 9, "a plan changes nothing");
let done = run(&mut c, 30);
assert_eq!(done.evict.len(), 5);
assert!(old.iter().all(|id| c.photo(*id).is_none()));
assert!(recent.iter().all(|id| c.photo(*id).is_some()));
assert!(c.photo(lib).is_some());
// the folder has no Local records left: its time goes
assert_eq!(done.unstamp, vec!["/pics/old".to_string()]);
assert_eq!(c.last_browsed("/pics/old"), None);
assert_eq!(c.last_browsed("/pics/recent/"), Some("2026-10-01T09:00:00"));
// 0 days = never
browsed(&mut c, "recent", OLD);
assert!(plan(&c, 0).evict.is_empty());
assert_eq!(plan(&c, 30).evict.len(), 3);
}
#[test]
fn any_user_change_keeps_a_record() {
type Change = Box<dyn Fn(&mut Catalog, PhotoId)>;
let set = |f: fn(&mut Photo)| -> Change {
Box::new(move |c: &mut Catalog, id: PhotoId| {
let mut p = (**c.photo(id).unwrap()).clone();
f(&mut p);
c.apply(Op::RemovePhoto { id }).unwrap();
c.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
})
};
let op = |f: fn(PhotoId) -> Op| -> Change { Box::new(move |c: &mut Catalog, id: PhotoId| drop(c.apply(f(id)).unwrap())) };
let edited = || {
let mut d = DevelopSettings::default();
d.light.exposure = 0.3;
Arc::new(d)
};
let changes: Vec<(&str, Change)> = vec![
("rating", op(|id| Op::SetRating { id, rating: 3 })),
("flag", op(|id| Op::SetFlag { id, flag: Flag::Reject })),
("label", op(|id| Op::SetLabel { id, label: Some(ColorLabel::Red) })),
("develop", Box::new(move |c, id| drop(c.apply(Op::SetDevelop { id, settings: edited(), label: "Exposure".into(), edited: None }).unwrap()))),
("crop", set(|p| Arc::make_mut(&mut p.develop).crop.flip_h = true)),
(
"history only",
Box::new(|c, id| {
let step = HistoryStep { label: "Reset".into(), settings: Arc::new(DevelopSettings::default()) };
drop(c.apply(Op::PushHistory { id, step }).unwrap())
}),
),
("versions", set(|p| p.versions.push(Version { name: "v".into(), created: NOW.into(), settings: p.develop.clone(), auto: false }))),
("title", set(|p| p.meta.title = "T".into())),
("caption", set(|p| p.meta.caption = "C".into())),
("keywords", set(|p| p.meta.keywords.push("k".into()))),
("location", set(|p| p.meta.city = "Oslo".into())),
("gps", set(|p| p.meta.gps = Some((1.0, 2.0)))),
("copyright", set(|p| p.meta.copyright = "© me".into())),
("creator edit", set(|p| p.meta.creator = "Me".into())),
("capture time", op(|id| Op::SetCaptured { id, captured: Some("2020-01-01T00:00:00".into()) })),
("rename", set(|p| p.file_name = "renamed.JPG".into())),
("culling", op(|id| Op::SetAnalysis { id, analysis: Some(Analysis::default()) })),
("deleted", op(|id| Op::SetDeleted { id, deleted: true })),
("pick", op(|id| Op::SetFlag { id, flag: Flag::Pick })),
(
"album",
Box::new(|c, id| {
let a = c.alloc_album_id();
let mut al = Album::new(a, "A");
al.photos.push(id);
drop(c.apply(Op::AddAlbum { album: al }).unwrap())
}),
),
(
"stack",
Box::new(|c, id| {
let other = local(c, "elsewhere", 99);
let s = c.alloc_stack_id();
drop(c.apply(Op::AddStack { stack: Stack { id: s, photos: vec![id, other], collapsed: false } }).unwrap())
}),
),
(
"virtual copy",
Box::new(|c, id| {
let mut v = (**c.photo(id).unwrap()).clone();
v.id = c.alloc_photo_id();
v.copy_of = Some(id);
v.copy_name = Some("Copy 1".into());
v.local = false;
drop(c.apply(Op::AddPhoto { photo: Box::new(v) }).unwrap())
}),
),
];
for (what, change) in changes {
let mut c = Catalog::new();
let id = local(&mut c, "old", 1);
let control = local(&mut c, "old", 2);
browsed(&mut c, "old", OLD);
change(&mut c, id);
let p = plan(&c, 30);
assert_eq!(p.evict, vec![control], "{what}: {p:?}");
assert_eq!(p.kept_touched + p.kept_in_use, 1, "{what}: {p:?}");
// the stamp stays: a record in the folder remains
assert!(p.unstamp.is_empty(), "{what}");
}
// a change undone (back to the browse state) is untouched again
let mut c = Catalog::new();
let id = local(&mut c, "old", 1);
browsed(&mut c, "old", OLD);
let inv = c.apply(Op::SetRating { id, rating: 2 }).unwrap();
assert!(plan(&c, 30).evict.is_empty());
c.apply(inv).unwrap();
assert_eq!(plan(&c, 30).evict, vec![id]);
}
/// Catalogs from before last-browsed times (and baselines): nothing is forgotten on the first
/// run — the folders are stamped now — and baseline-less records are judged by their data.
#[test]
fn upgrade_without_times_forgets_nothing_then_counts_from_then() {
let mut c = Catalog::new();
let a = local(&mut c, "a", 1);
// legacy records: no baseline
let legacy = |c: &mut Catalog, i: u32, f: fn(&mut Photo)| {
let id = c.alloc_photo_id();
let mut p = Photo::new(id, Source::File { path: format!("/pics/a/L{i}.JPG") }, "L.JPG", "JPEG", 6, 4, OLD);
p.meta.camera = "Cam".into();
p.local = true;
f(&mut p);
c.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
id
};
let clean = legacy(&mut c, 1, |_| {});
let rated = legacy(&mut c, 2, |p| p.rating = 1);
let with_creator = legacy(&mut c, 3, |p| p.meta.creator = "maybe the user".into());
let p = run(&mut c, 30);
assert!(p.evict.is_empty(), "{p:?}");
assert_eq!(p.stamp, vec!["/pics/a".to_string()]);
assert_eq!(c.last_browsed("/pics/a"), Some(NOW));
// 30+ days later
let later = c.forget_local_plan("2026-11-06T10:00:00", 30, &|_| false);
let mut evict = later.evict.clone();
evict.sort();
assert_eq!(evict, vec![a, clean]);
assert_eq!(later.kept_touched, 2, "{rated:?} {with_creator:?}");
}
#[test]
fn in_use_records_are_kept() {
let mut c = Catalog::new();
let a = local(&mut c, "old", 1);
let b = local(&mut c, "old", 2);
browsed(&mut c, "old", OLD);
let p = c.forget_local_plan(NOW, 30, &|p| p.id == a);
assert_eq!((p.evict.clone(), p.kept_in_use), (vec![b], 1));
}
/// The ops are journaled: a crash right after forgetting reloads the same catalog, and a
/// fingerprint survives the snapshot round trip (a reloaded record is still untouched).
#[test]
fn forgetting_is_journaled_and_replays() {
let m = MemStore::new();
let (mut j, mut c, _) = Journal::open(Box::new(m.clone())).unwrap();
let mut ops = Vec::new();
for i in 0..10 {
local(&mut c, if i % 2 == 0 { "old" } else { "new" }, i);
}
// log what was built so far: the ops are the records as added
for p in c.photos() {
ops.push(Op::AddPhoto { photo: Box::new((**p).clone()) });
}
ops.push(Op::SetBrowsed { folder: "/pics/old".into(), at: Some(OLD.into()) });
ops.push(Op::SetBrowsed { folder: "/pics/new".into(), at: Some(NOW.into()) });
browsed(&mut c, "old", OLD);
browsed(&mut c, "new", NOW);
j.append(&ops).unwrap();
j.snapshot(&c).unwrap();
drop(j);
// reload from the snapshot: baselines still match
let (mut j, mut c, _) = Journal::open(Box::new(m.clone())).unwrap();
let p = c.forget_local_plan(NOW, 30, &|_| false);
assert_eq!(p.evict.len(), 5, "{p:?}");
let ops = p.ops();
for op in &ops {
c.apply(op.clone()).unwrap();
}
j.append(&ops).unwrap();
let expect = c.to_snapshot();
drop(j); // crash: replay the log
let (_, c2, r) = Journal::open(Box::new(m)).unwrap();
assert_eq!(r.failed, 0);
assert_eq!(c2.to_snapshot(), expect);
assert_eq!(c2.len(), 5);
// nothing left to forget
assert!(c2.forget_local_plan(NOW, 30, &|_| false).evict.is_empty());
}
#[test]
fn op_photo_ids_cover_nested_ops() {
let op = Op::Batch {
ops: vec![
Op::SetRating { id: PhotoId(1), rating: 1 },
Op::SetAlbumPhotos { id: AlbumId(1), photos: vec![PhotoId(2), PhotoId(3)] },
Op::Batch { ops: vec![Op::RemovePhoto { id: PhotoId(4) }] },
],
};
let mut ids = Vec::new();
op.photo_ids(&mut |id| ids.push(id.0));
assert_eq!(ids, vec![1, 2, 3, 4]);
}
+93
View File
@@ -0,0 +1,93 @@
//! Issue #99: one process per library.
use std::path::PathBuf;
use crate::lock::{LOCK, OWNER};
use crate::*;
fn temp_dir(tag: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("lc-lock-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
#[test]
fn second_opener_is_refused_until_the_first_lets_go() {
let dir = temp_dir("twice");
let first = LibraryLock::acquire(&dir, "LightCraft").unwrap();
assert!(first.held());
let owner: LockOwner = serde_json::from_slice(&std::fs::read(dir.join(OWNER)).unwrap()).unwrap();
assert_eq!((owner.pid, owner.program.as_str()), (std::process::id(), "LightCraft"));
let e = LibraryLock::acquire(&dir, "lightcraft-cli").unwrap_err();
let LockError::InUse(Some(who)) = &e else { panic!("{e:?}") };
assert_eq!(who.pid, std::process::id());
let msg = e.to_string();
assert!(msg.contains("already open in LightCraft") && msg.contains(&format!("process {}", std::process::id())), "{msg}");
drop(first);
assert!(!dir.join(OWNER).exists(), "the owner note goes with the lock");
assert!(dir.join(LOCK).exists(), "the lock file itself may stay: only a held lock refuses");
let again = LibraryLock::acquire(&dir, "lightcraft-cli").unwrap();
assert!(again.held());
drop(again);
let _ = std::fs::remove_dir_all(&dir);
}
/// Files left behind by a crash (lock file + owner note) don't lock anyone out.
#[test]
fn leftover_lock_files_are_not_a_lock() {
let dir = temp_dir("stale");
std::fs::write(dir.join(LOCK), b"").unwrap();
std::fs::write(dir.join(OWNER), br#"{"pid":999999,"host":"elsewhere","program":"LightCraft","version":"0.2.0","since":1}"#).unwrap();
let l = LibraryLock::acquire(&dir, "LightCraft").unwrap();
assert!(l.held());
let owner: LockOwner = serde_json::from_slice(&std::fs::read(dir.join(OWNER)).unwrap()).unwrap();
assert_eq!(owner.pid, std::process::id(), "the note now names this process");
drop(l);
let _ = std::fs::remove_dir_all(&dir);
}
/// Another process holds the lock; when it is killed (a crash), the OS releases it.
#[test]
fn lock_held_by_another_process_is_released_when_it_dies() {
let dir = temp_dir("child");
let exe = std::env::current_exe().unwrap();
let mut child = std::process::Command::new(exe)
.args(["--exact", "tests_lock::lock_holder_child", "--ignored", "--test-threads=1"])
.env("LC_LOCK_CHILD_DIR", &dir)
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.spawn()
.unwrap();
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(60);
while !dir.join("child-locked").exists() {
assert!(child.try_wait().unwrap().is_none(), "child exited before locking");
assert!(std::time::Instant::now() < deadline, "child never locked");
std::thread::sleep(std::time::Duration::from_millis(20));
}
let e = LibraryLock::acquire(&dir, "LightCraft").unwrap_err();
let LockError::InUse(Some(who)) = &e else { panic!("{e:?}") };
assert_eq!(who.pid, child.id());
assert!(e.to_string().contains("on this computer") || who.host.is_empty(), "{e}");
child.kill().unwrap(); // SIGKILL: no clean-up runs, like a crash
child.wait().unwrap();
assert!(dir.join(OWNER).exists(), "the crashed holder's note is left behind");
let l = LibraryLock::acquire(&dir, "LightCraft").expect("the OS released the dead process's lock");
assert!(l.held());
drop(l);
let _ = std::fs::remove_dir_all(&dir);
}
/// Run by [`lock_held_by_another_process_is_released_when_it_dies`] in a child process: take the
/// lock, say so, and wait to be killed.
#[test]
#[ignore = "helper process for lock_held_by_another_process_is_released_when_it_dies"]
fn lock_holder_child() {
let Some(dir) = std::env::var_os("LC_LOCK_CHILD_DIR") else { return };
let _l = LibraryLock::acquire(std::path::Path::new(&dir), "lightcraft-cli").unwrap();
std::fs::write(std::path::Path::new(&dir).join("child-locked"), b"").unwrap();
std::thread::sleep(std::time::Duration::from_secs(120));
}
+223
View File
@@ -0,0 +1,223 @@
//! Issue #101: an append that fails part-way (disk full, a dropped network share) and is then
//! retried must not read back as log damage, and must lose nothing.
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use crate::journal::LOG;
use crate::*;
/// A store whose next append writes only a prefix of its data, then fails.
#[derive(Clone, Default)]
struct Torn {
files: MemStore,
/// The next append writes this many bytes, then returns an error.
torn_at: Arc<Mutex<Option<usize>>>,
/// `truncate` fails (the cut-back after a failed append can't be done right away).
truncate_fails: Arc<AtomicBool>,
}
impl Torn {
fn tear_next_append(&self, bytes: usize) {
*self.torn_at.lock().unwrap() = Some(bytes);
}
}
impl Store for Torn {
fn read(&mut self, name: &str) -> std::io::Result<Option<Vec<u8>>> {
self.files.read(name)
}
fn write_atomic(&mut self, name: &str, data: &[u8]) -> std::io::Result<()> {
self.files.write_atomic(name, data)
}
fn append(&mut self, name: &str, data: &[u8]) -> std::io::Result<()> {
if let Some(n) = self.torn_at.lock().unwrap().take() {
self.files.append(name, &data[..n.min(data.len())])?;
return Err(std::io::Error::other("disk full"));
}
self.files.append(name, data)
}
fn truncate(&mut self, name: &str, len: u64) -> std::io::Result<()> {
if self.truncate_fails.load(Ordering::SeqCst) {
return Err(std::io::Error::other("share unavailable"));
}
self.files.truncate(name, len)
}
fn describe(&self) -> String {
"torn".into()
}
}
fn add_photo(c: &mut Catalog) -> Op {
let id = c.alloc_photo_id();
Op::AddPhoto { photo: Box::new(Photo::new(id, Source::File { path: format!("/p/{}.jpg", id.0) }, "p.jpg", "JPEG", 4, 3, "2026-01-01")) }
}
/// Apply `ops` to the live catalog (like a command) and return them for the journal.
fn apply(c: &mut Catalog, ops: Vec<Op>) -> Vec<Op> {
for op in &ops {
c.apply(op.clone()).unwrap();
}
ops
}
/// Every tear position of a batch, then the retry (same ops, same seqs, like the engine's queue)
/// and more appends: reloading replays everything, reports no damage and no torn bytes.
#[test]
fn partial_append_then_retry_loads_cleanly() {
let probe = MemStore::new();
let (mut j, mut c, _) = Journal::open(Box::new(probe.clone())).unwrap();
let add = add_photo(&mut c);
let first = apply(&mut c, vec![add]);
j.append(&first).unwrap();
let id = c.photos().next().unwrap().id;
let batch = apply(&mut c, vec![Op::SetRating { id, rating: 3 }, Op::SetFlag { id, flag: Flag::Pick }, Op::SetRating { id, rating: 4 }]);
let before = probe.get(LOG).unwrap().len();
j.append(&batch).unwrap();
let batch_len = probe.get(LOG).unwrap().len() - before;
for tear in 1..batch_len {
let t = Torn::default();
let (mut j, mut c, _) = Journal::open(Box::new(t.clone())).unwrap();
let add = add_photo(&mut c);
let first = apply(&mut c, vec![add]);
j.append(&first).unwrap();
let good_len = t.files.get(LOG).unwrap().len();
let batch = apply(&mut c, vec![Op::SetRating { id, rating: 3 }, Op::SetFlag { id, flag: Flag::Pick }, Op::SetRating { id, rating: 4 }]);
t.tear_next_append(tear);
assert!(j.append(&batch).is_err(), "tear at {tear}");
// cut back to the last whole record
assert_eq!(t.files.get(LOG).unwrap().len(), good_len, "tear at {tear}");
j.append(&batch).unwrap();
let more = apply(&mut c, vec![Op::SetLabel { id, label: Some(ColorLabel::ALL[0]) }]);
j.append(&more).unwrap();
drop(j);
let (_, loaded, r) = Journal::open(Box::new(t.clone())).unwrap();
assert!(r.damaged.is_none(), "tear at {tear}: {r:?}");
assert_eq!((r.torn_bytes, r.torn_fragments, r.replayed), (0, 0, 5), "tear at {tear}");
assert_eq!(loaded.to_snapshot(), c.to_snapshot(), "tear at {tear}");
}
}
/// When cutting back fails too (the share is gone), the next append cuts back first; until that
/// works nothing more is appended after the fragment.
#[test]
fn failed_cut_back_is_done_before_the_next_append() {
let t = Torn::default();
let (mut j, mut c, _) = Journal::open(Box::new(t.clone())).unwrap();
let add = add_photo(&mut c);
let first = apply(&mut c, vec![add]);
j.append(&first).unwrap();
let good_len = t.files.get(LOG).unwrap().len();
let id = c.photos().next().unwrap().id;
let batch = apply(&mut c, vec![Op::SetRating { id, rating: 2 }, Op::SetRating { id, rating: 5 }]);
t.truncate_fails.store(true, Ordering::SeqCst);
t.tear_next_append(30);
assert!(j.append(&batch).is_err());
assert_eq!(t.files.get(LOG).unwrap().len(), good_len + 30, "the fragment is still there");
// still unreachable: the retry fails without writing anything after the fragment
assert!(j.append(&batch).is_err());
assert_eq!(t.files.get(LOG).unwrap().len(), good_len + 30);
t.truncate_fails.store(false, Ordering::SeqCst);
j.append(&batch).unwrap();
drop(j);
let (_, loaded, r) = Journal::open(Box::new(t.clone())).unwrap();
assert!(r.damaged.is_none(), "{r:?}");
assert_eq!(r.replayed, 3);
assert_eq!(loaded.to_snapshot(), c.to_snapshot());
}
/// Logs written before the fix: the fragment of the failed append is glued to the first retried
/// record on one line, followed by the rest of the retry. They load in full (the fragment is
/// skipped), instead of stopping at the fragment and setting the later records aside as damage.
#[test]
fn legacy_fragment_glued_to_the_retry_is_skipped() {
let m = MemStore::new();
let (mut j, mut c, _) = Journal::open(Box::new(m.clone())).unwrap();
let add = add_photo(&mut c);
let first = apply(&mut c, vec![add]);
j.append(&first).unwrap();
let good = m.get(LOG).unwrap();
let id = c.photos().next().unwrap().id;
let batch = apply(
&mut c,
vec![
Op::SetRating { id, rating: 3 },
Op::SetFlag { id, flag: Flag::Pick },
Op::SetLabelName { label: ColorLabel::ALL[1], name: Some("Größe ✓".into()) },
],
);
j.append(&batch).unwrap();
let full = m.get(LOG).unwrap();
let batch_bytes = &full[good.len()..];
let later = apply(&mut c, vec![Op::SetRating { id, rating: 1 }]);
j.append(&later).unwrap();
let later_bytes = m.get(LOG).unwrap()[full.len()..].to_vec();
drop(j);
// every tear (inside the first record, after it, inside a multi-byte character…)
for tear in 1..batch_bytes.len() {
if batch_bytes[tear - 1] == b'\n' {
continue; // a tear on a record boundary leaves no fragment
}
let mut log = good.clone();
log.extend_from_slice(&batch_bytes[..tear]);
log.extend_from_slice(batch_bytes);
log.extend_from_slice(&later_bytes);
let t = MemStore::new();
t.set(LOG, log);
let (_, loaded, r) = Journal::open(Box::new(t)).unwrap();
assert!(r.damaged.is_none(), "tear at {tear}: {r:?}");
assert_eq!(r.torn_fragments, 1, "tear at {tear}");
assert_eq!(loaded.to_snapshot(), c.to_snapshot(), "tear at {tear}");
}
// a damaged record whose tail happens to be a whole record from the future is still damage
let mut log = good.clone();
log.extend_from_slice(b"garbage");
log.extend_from_slice(&later_bytes);
log.extend_from_slice(&later_bytes);
let t = MemStore::new();
t.set(LOG, log);
let (_, _, r) = Journal::open(Box::new(t)).unwrap();
assert!(r.damaged.is_some(), "{r:?}");
}
/// A snapshot can absorb ops whose append failed: it records them as saved (`seq` moves past
/// them), and appending after it continues from there.
#[test]
fn snapshot_with_unlogged_ops_saves_them() {
let t = Torn::default();
let (mut j, mut c, _) = Journal::open(Box::new(t.clone())).unwrap();
let add = add_photo(&mut c);
let first = apply(&mut c, vec![add]);
j.append(&first).unwrap();
let id = c.photos().next().unwrap().id;
let queued = apply(&mut c, vec![Op::SetRating { id, rating: 2 }, Op::SetFlag { id, flag: Flag::Reject }]);
t.tear_next_append(10);
assert!(j.append(&queued).is_err());
j.snapshot_with_unlogged(&c, queued.len() as u64).unwrap();
assert_eq!(j.seq(), 3);
let after = apply(&mut c, vec![Op::SetRating { id, rating: 5 }]);
j.append(&after).unwrap();
drop(j);
let (j, loaded, r) = Journal::open(Box::new(t.clone())).unwrap();
assert_eq!((r.snapshot_seq, r.replayed), (3, 1));
assert_eq!(j.seq(), 4);
assert_eq!(loaded.to_snapshot(), c.to_snapshot());
}
/// `FsStore::truncate` on a file that was never created (an append that failed first) is a no-op.
#[test]
fn fs_store_truncates_a_missing_file_quietly() {
let dir = std::env::temp_dir().join(format!("lc-torn-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
let mut s = FsStore::open(&dir).unwrap();
s.truncate(LOG, 0).unwrap();
assert!(!dir.join(LOG).exists());
s.append(LOG, b"abc\n").unwrap();
s.truncate(LOG, 0).unwrap();
assert_eq!(std::fs::read(dir.join(LOG)).unwrap(), b"");
let _ = std::fs::remove_dir_all(&dir);
}
+5
View File
@@ -12,6 +12,7 @@ workspace = true
lightcraft-geom = { workspace = true }
lightcraft-color = { workspace = true }
lightcraft-raster = { workspace = true }
lightcraft-heif = { workspace = true, optional = true }
serde = { workspace = true }
bytemuck = { workspace = true }
thiserror = { workspace = true }
@@ -34,6 +35,10 @@ ravif = { workspace = true, optional = true }
default = ["parallel", "jxl", "avif"]
parallel = ["dep:rayon", "lightcraft-raster/parallel", "jpeg-decoder/rayon"]
jxl = ["dep:jxl-oxide"]
# HEIC/HEIF decode via the optional lightcraft-heif crate (heic-rs, pure Rust). Off by default so
# distributors choose it at build time (HEVC patents are a distributor's call, same policy as
# PhotoCraft); `cargo xtask ci` clippies and tests it (`--features heif`), releases don't ship it yet.
heif = ["dep:lightcraft-heif"]
## AVIF encoding via ravif/rav1e (pure Rust, BSD). Native targets only; a no-op on wasm32.
avif = ["dep:ravif"]
+11
View File
@@ -14,6 +14,7 @@ let d: Decoded = decode(&bytes, DecodeOptions::default())?;
let working: Rgb32f = to_working(&d); // linear Rec.2020 D65 (Bradford)
let fast = decode(&bytes, DecodeOptions::fit(2048, 2048))?; // JPEG: DCT-domain 1/2..1/8 scaling
let t: Thumbnail = decode_thumbnail(&bytes, 256)?; // embedded EXIF/MPF preview or scaled decode
let h: Header = read_header(&bytes)?; // stored width/height + orientation, no pixels decoded
let icc = icc::write_named(NamedSpace::DisplayP3); // or icc::write_matrix_trc(&space, &Trc::…)
let meta = EncodeMeta { icc: Some(&icc), exif: Some(&exif), xmp: Some(&xmp) };
@@ -24,6 +25,12 @@ encode_webp_lossless(&img, &meta)?;
encode_avif(&img, 70, 6, &meta)?; // native + feature `avif`
```
Thumbnail fallback decoding limits the source to 64 million pixels by default.
Non-JPEG decoders allocate at source resolution before resizing, even for a tiny
thumbnail. Use `decode_thumbnail_with` and `ThumbnailOptions::max_pixels` to set
a different source budget. Oversized sources return `Error::TooLarge`; embedded
JPEG previews can still be used without decoding the full source image.
`Format::RawTiffLike` (DNG, CR2, NEF, ARW, PEF, ORF, RW2, SRW, …) and `Format::RawOther` (CR3, RAF,
CRW, MRW, X3F) are detected so the engine can route them to `lightcraft-raw`; `decode` returns
`Error::Unsupported` for them.
@@ -50,6 +57,10 @@ or CMYK profile converted by the CMS straight to linear Rec.2020), `Container` (
EXIF `R03` Adobe RGB hint), `Untagged` (assumed sRGB; untagged float data assumed linear),
`IccUnsupported` (profile present but unusable → **sRGB fallback**), `Naive` (CMYK without profile).
`decode_jpeg_with_fallback(bytes, opts, space)` lets an enclosing RAW container supply the colour
space of a JPEG with no ICC or EXIF metadata. It applies the transfer curve before linear-light
resizing; a JPEG carrying its own metadata retains its existing interpretation.
ICC profiles are parsed with `moxcms` (v2/v4, `curv`/`para` TRCs, `chad`); sRGB, Display P3,
Adobe RGB (1998), ProPhoto (ROMM) and Rec.2020 are recognised by colorants. `icc::write_matrix_trc`
emits v4 display profiles (D50 PCS, Bradford `chad`) for export embedding.
+3 -1
View File
@@ -323,7 +323,9 @@ fn cmyk(raw: &Raw, ch: usize, icc_bytes: Option<&[u8]>, info: Option<&icc::IccIn
}
// Naive: RGB = (1 - C)(1 - K) … treated as sRGB-encoded.
let px = samples
.chunks_exact(4)
.as_chunks::<4>()
.0
.iter()
.map(|c| {
let k = 1.0 - c[3];
[(1.0 - c[0]) * k, (1.0 - c[1]) * k, (1.0 - c[2]) * k].map(lightcraft_color::transfer::srgb_to_linear)
+2 -2
View File
@@ -91,7 +91,7 @@ pub fn encode_jpeg(img: &EncodeImage, quality: u8, subsampling: ChromaSubsamplin
let n = img.width as usize * img.height as usize;
let (data, ct): (std::borrow::Cow<[u8]>, jpeg_encoder::ColorType) = match img.channels {
1 => (data[..n].into(), jpeg_encoder::ColorType::Luma),
2 => (data.chunks_exact(2).take(n).map(|c| c[0]).collect::<Vec<_>>().into(), jpeg_encoder::ColorType::Luma),
2 => (data.as_chunks::<2>().0.iter().take(n).map(|c| c[0]).collect::<Vec<_>>().into(), jpeg_encoder::ColorType::Luma),
3 => (data[..n * 3].into(), jpeg_encoder::ColorType::Rgb),
_ => (data[..n * 4].into(), jpeg_encoder::ColorType::Rgba),
};
@@ -254,7 +254,7 @@ pub fn encode_tiff(img: &EncodeImage, compression: TiffCompression, meta: &Encod
}};
}
fn expand_ga<T: Copy>(s: &[T], n: usize) -> Vec<T> {
s.chunks_exact(2).take(n).flat_map(|c| [c[0], c[0], c[0], c[1]]).collect()
s.as_chunks::<2>().0.iter().take(n).flat_map(|c| [c[0], c[0], c[0], c[1]]).collect()
}
match img.samples {
Samples::U8(s) => match ch {
+125
View File
@@ -0,0 +1,125 @@
//! HEIF / HEIC (the iPhone and Mac photo format), read-only.
//!
//! Decoding lives in the optional `lightcraft-heif` crate (heic-rs, a pure-Rust HEVC
//! still-picture decoder), enabled by this crate's `heif` feature — off by default because HEVC
//! is patent-encumbered and whether a build carries an HEVC decoder is the distributor's call
//! (the same policy as the sibling PhotoCraft). Without the feature, HEIF files are still
//! detected and opening one is an [`Error::Unsupported`] error, never a panic.
//!
//! HEIF records orientation in the container (`irot`/`imir`, plus a `clap` crop), not in EXIF,
//! so the decoder applies them and `orientation` is reported as 1: the EXIF Orientation tag only
//! mirrors what the container says, and applying it as well would turn the photo twice.
//! The samples come back in the container's own primaries with its ICC profile (if any); like
//! every decoder here, the colour interpretation happens in [`crate::convert::finish`].
#[cfg(feature = "heif")]
use crate::convert::{Buf, Meta, Model, Raw, finish};
use crate::{DecodeOptions, Decoded, Error, Format, Result};
const F: Format = Format::Heif;
/// The reason given when this build has no HEIF decoder.
#[cfg(not(feature = "heif"))]
pub(crate) const NOT_IN_BUILD: &str = "HEIC/HEIF support isn't included in this build of LightCraft";
/// The reason a HEIF file that no build can open is refused (this crate decodes HEVC stills
/// only; the wrapper names the exact construct in its own errors).
#[cfg(feature = "heif")]
const NOT_DECODED: &str = "the image is not a decodable HEVC still (image sequence, overlay, alpha-less grid or AVIF-coded item)";
#[cfg(not(feature = "heif"))]
pub(crate) fn decode(_bytes: &[u8], _opts: &DecodeOptions) -> Result<Decoded> {
Err(Error::Unsupported(F, NOT_IN_BUILD))
}
#[cfg(feature = "heif")]
fn err(e: lightcraft_heif::Error) -> Error {
match e {
// This crate's Unsupported takes a fixed reason; the dynamic wording lives in the
// lightcraft-heif error itself.
lightcraft_heif::Error::Unsupported(_) => Error::Unsupported(F, NOT_DECODED),
lightcraft_heif::Error::Limit(m) | lightcraft_heif::Error::Malformed(m) => Error::Malformed(F, m),
}
}
#[cfg(feature = "heif")]
pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
// The container alone: the declared size is checked before any pixel is decoded.
let info = lightcraft_heif::probe(bytes).map_err(err)?;
if (info.width as u64).saturating_mul(info.height as u64) > opts.max_pixels {
return Err(Error::TooLarge(info.width as u64, info.height as u64));
}
let options = lightcraft_heif::Options { max_pixels: opts.max_pixels, apply_transforms: true };
let decoded = lightcraft_heif::decode(bytes, &options).map_err(err)?;
// The decoder re-reads the container; trust its output's shape, not the probe's.
let buf = if decoded.sixteen_bit {
let mut v = Vec::with_capacity(decoded.data.len() / 2);
for pair in decoded.data.as_chunks::<2>().0 {
v.push(u16::from_ne_bytes(*pair));
}
Buf::U16(v)
} else {
Buf::U8(decoded.data)
};
let raw = Raw {
width: decoded.width as usize,
height: decoded.height as usize,
model: Model::Rgb,
alpha: decoded.has_alpha,
premultiplied: false,
buf,
bit_depth: if decoded.sixteen_bit { 16 } else { 8 },
};
let meta = Meta { icc: decoded.icc, exif: decoded.exif, xmp: decoded.xmp, orientation: Some(1), ..Default::default() };
finish(F, raw, meta, (decoded.width, decoded.height), opts)
}
#[cfg(test)]
mod tests {
use crate::{Format, sniff};
/// The container header both builds recognise, whatever the feature says.
const HEIC_HEADER: &[u8] = b"\0\0\0\x18ftypheic\0\0\0\0mif1heic";
#[test]
fn heif_is_always_recognised() {
assert_eq!(sniff(HEIC_HEADER), Some(Format::Heif));
}
#[test]
fn can_decode_follows_the_feature() {
assert_eq!(Format::Heif.can_decode(), cfg!(feature = "heif"));
}
#[cfg(not(feature = "heif"))]
#[test]
fn without_the_feature_heif_is_a_clear_unsupported_error() {
let r = crate::decode(HEIC_HEADER, crate::DecodeOptions::default());
assert!(matches!(&r, Err(crate::Error::Unsupported(Format::Heif, why)) if why.contains("isn't included in this build")), "{r:?}");
}
/// Found by PhotoCraft's `decode_heif` fuzz target: a malformed box makes heic-rs 0.1.1 slice
/// out of range (`boxes.rs:130`). It must be an error, not a crash.
#[cfg(feature = "heif")]
const HEIC_RS_BOX_PANIC: [u8; 72] = [
0x00, 0x00, 0x00, 0x24, 0x66, 0x74, 0x79, 0x70, 0x68, 0x65, 0x69, 0x63, 0x00, 0x00, 0x00, 0x00, 0x6d, 0x69, 0x66, 0x31, 0x4d, 0x69, 0x50,
0x72, 0x6d, 0x69, 0x61, 0x66, 0x4d, 0x69, 0x48, 0x42, 0x72, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x6d, 0x65, 0x74, 0x61, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x75, 0x75, 0x69, 0x64, 0x00, 0x00, 0x00, 0x00, 0x03, 0x08, 0x08, 0x08, 0x00, 0x03, 0x01, 0x03, 0x70,
0x00, 0xa8, 0x00,
];
#[cfg(feature = "heif")]
#[test]
fn a_heic_rs_panic_is_a_malformed_error() {
let r = crate::decode(&HEIC_RS_BOX_PANIC, crate::DecodeOptions::default());
assert!(matches!(&r, Err(crate::Error::Malformed(Format::Heif, _))), "{r:?}");
}
#[cfg(feature = "heif")]
#[test]
fn an_image_sequence_is_unsupported() {
// An ftyp with no meta box: a video track, not a still.
let r = crate::decode(b"\0\0\0\x18ftypmsf1\0\0\0\0msf1hevc", crate::DecodeOptions::default());
assert!(matches!(&r, Err(crate::Error::Unsupported(Format::Heif, _))), "{r:?}");
}
}
+31 -2
View File
@@ -146,7 +146,19 @@ pub fn write_matrix_trc(space: &RgbSpace, trc: &Trc) -> Vec<u8> {
};
p.description = Some(ProfileText::Localizable(vec![LocalizableString::new("en".into(), "US".into(), desc)]));
p.copyright = Some(ProfileText::Localizable(vec![LocalizableString::new("en".into(), "US".into(), "No copyright, use freely".into())]));
align_tags(&p.encode().unwrap_or_default())
align_tags(&fixed_date(p.encode().unwrap_or_default()))
}
/// Zero the header's creation date/time (ICC.1 §7.2.8, bytes 24..36) so a profile's bytes depend
/// only on its colour space and curve. `moxcms` stamps the wall clock there on every `encode`
/// (ignoring `ColorProfile::creation_date_time`), which made two exports of one photo differ in
/// the embedded profile and nowhere else, so output hashes could not be compared (#180). Readers
/// do not interpret the field; all zeros is the conventional "unspecified" value.
fn fixed_date(mut b: Vec<u8>) -> Vec<u8> {
if let Some(date) = b.get_mut(24..36) {
date.fill(0);
}
b
}
/// Re-lay a profile's tag data on 4-byte boundaries (ICC.1 §7.3.1). The encoder packs tags back to
@@ -236,7 +248,7 @@ pub(crate) fn cms_to_linear_rec2020(icc: &[u8], samples: &[f32], channels: usize
for (s, d) in src_chunks.zip(dst_chunks) {
t.transform(s, d).ok()?;
}
Some(out.chunks_exact(3).map(|c| [c[0], c[1], c[2]]).collect())
Some(out.as_chunks::<3>().0.iter().map(|c| [c[0], c[1], c[2]]).collect())
};
std::panic::catch_unwind(run).ok().flatten()
}
@@ -300,6 +312,23 @@ mod tests {
let _ = SRGB;
}
/// Two profiles for one space are byte-identical, and the header date is fixed: exports of the
/// same photo with the same settings hash the same (#180).
#[test]
fn written_profiles_are_deterministic() {
for n in NamedSpace::ALL {
let a = write_named(n);
let b = write_named(n);
assert_eq!(a, b, "{n:?}");
assert_eq!(&a[24..36], &[0u8; 12], "{n:?} header date/time");
// the profile ID is "not computed", not a hash of a timestamp
assert_eq!(&a[84..100], &[0u8; 16], "{n:?} profile ID");
}
let lin = write_matrix_trc(&PROPHOTO, &Trc::Linear);
assert_eq!(lin, write_matrix_trc(&PROPHOTO, &Trc::Linear));
assert_eq!(&lin[24..36], &[0u8; 12]);
}
#[test]
fn garbage_is_none() {
assert!(parse(&[]).is_none());
+140 -4
View File
@@ -105,12 +105,67 @@ pub(crate) fn parse_markers(b: &[u8]) -> Option<Markers> {
}
pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
decode_with_fallback(bytes, opts, None)
}
/// Stored dimensions and EXIF orientation from the markers, without decoding the scan data. The
/// file is refused, as a decode would refuse it, when it has no frame header, an unsupported frame
/// type or component count, or when its markers don't run to an end-of-image marker after the
/// image data (a truncated file). Damage *inside* the entropy-coded data can't be seen this way.
pub(crate) fn header(bytes: &[u8]) -> Result<(u32, u32, u16)> {
let m = parse_markers(bytes).ok_or_else(|| Error::Malformed(F, "missing SOI".into()))?;
if m.components == 0 {
return Err(Error::Malformed(F, "no frame header".into()));
}
check_size(F, m.width as u64, m.height as u64, &DecodeOptions::default())?;
if !matches!(m.sof, 0xC0..=0xC3) {
return Err(Error::Unsupported(F, "arithmetic-coded or hierarchical JPEG"));
}
if !matches!(m.components, 1 | 3 | 4) {
return Err(Error::Malformed(F, format!("{} colour components", m.components)));
}
if m.first_scan_components == 0 || !reaches_eoi(bytes) {
return Err(Error::Malformed(F, "truncated: the image data doesn't end with an end-of-image marker".into()));
}
let orientation = m.exif.as_deref().map(exif::summarize).unwrap_or_default().orientation.unwrap_or(1);
Ok((m.width, m.height, orientation))
}
/// Whether the markers after SOI, followed through every segment and scan, reach an EOI marker
/// (the first one: the primary image's, ahead of any MPF images appended to the file). Segments
/// are skipped by their length; in scan data `FF 00` (stuffing), `FF FF` (fill) and RSTn are not
/// markers that end it.
fn reaches_eoi(b: &[u8]) -> bool {
let mut p = 2usize;
loop {
// entropy-coded data (or garbage between segments, as `parse_markers` tolerates it)
let Some(skip) = b.get(p..).and_then(find_ff) else { return false };
p = p.saturating_add(skip);
let Some(&marker) = b.get(p.saturating_add(1)) else { return false };
p = match marker {
0xD9 => return true,
0xFF => p.saturating_add(1),
0x00 | 0x01 | 0xD0..=0xD8 => p.saturating_add(2),
_ => {
let Some(len) = b.get(p.saturating_add(2)..p.saturating_add(4)).map(|l| u16::from_be_bytes([l[0], l[1]]) as usize) else {
return false;
};
if len < 2 {
return false;
}
p.saturating_add(2).saturating_add(len)
}
};
}
}
pub(crate) fn decode_with_fallback(bytes: &[u8], opts: &DecodeOptions, fallback: Option<crate::NamedSpace>) -> Result<Decoded> {
let m = parse_markers(bytes).ok_or_else(|| Error::Malformed(F, "missing SOI".into()))?;
if m.components == 0 {
return Err(Error::Malformed(F, "no frame header".into()));
}
check_size(F, m.width as u64, m.height as u64, opts)?;
let scale_to = opts.max_size.filter(|&(mw, mh)| mw > 0 && mh > 0 && (m.width >= 2 * mw || m.height >= 2 * mh));
let scale_to = opts.max_size.filter(|&(mw, mh)| mw > 0 && mh > 0 && (m.width >= mw.saturating_mul(2) || m.height >= mh.saturating_mul(2)));
// zune-jpeg handles the common cases fastest; jpeg-decoder covers DCT scaling, CMYK/YCCK,
// 12-bit and non-interleaved sequential scans (which zune-jpeg 0.5 mis-decodes with subsampling).
let non_interleaved = matches!(m.sof, 0xC0 | 0xC1) && m.first_scan_components < m.components;
@@ -119,7 +174,10 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
} else {
decode_zune(bytes, &m)?
};
let meta = Meta { icc: m.icc, exif: m.exif, xmp: m.xmp, ..Default::default() };
// A raw container may carry the colour declaration of an otherwise untagged preview.
// Apply it before linearization/resizing; existing JPEG metadata always wins.
let hint = fallback.filter(|_| m.icc.is_none() && m.exif.is_none()).map(|space| crate::SourceSpace::named(space, crate::SpaceOrigin::Container));
let meta = Meta { icc: m.icc, exif: m.exif, xmp: m.xmp, hint, ..Default::default() };
finish(F, raw, meta, (m.width, m.height), opts)
}
@@ -159,7 +217,7 @@ fn decode_jpeg_decoder(bytes: &[u8], m: &Markers, scale_to: Option<(u32, u32)>)
let (model, buf, depth) = match info.pixel_format {
PixelFormat::L8 => (Model::Gray, Buf::U8(px), 8),
PixelFormat::L16 => {
let v = px.chunks_exact(2).map(|c| u16::from_be_bytes([c[0], c[1]])).collect();
let v = px.as_chunks::<2>().0.iter().map(|c| u16::from_be_bytes([c[0], c[1]])).collect();
(Model::Gray, Buf::U16(v), 16)
}
PixelFormat::RGB24 => (Model::Rgb, Buf::U8(px), 8),
@@ -168,6 +226,21 @@ fn decode_jpeg_decoder(bytes: &[u8], m: &Markers, scale_to: Option<(u32, u32)>)
Ok(Raw { width: w, height: h, model, alpha: false, premultiplied: false, buf, bit_depth: depth })
}
/// The first `0xFF` byte, eight bytes at a time (scan data has one every few hundred bytes: this
/// is most of [`header`]'s time).
fn find_ff(b: &[u8]) -> Option<usize> {
const ONES: u64 = 0x0101_0101_0101_0101;
let (words, tail) = b.as_chunks::<8>();
for (i, w) in words.iter().enumerate() {
// a zero byte in !w is an 0xFF byte in w
let x = !u64::from_ne_bytes(*w);
if x.wrapping_sub(ONES) & !x & (ONES << 7) != 0 {
return w.iter().position(|&v| v == 0xFF).map(|j| i * 8 + j);
}
}
tail.iter().position(|&v| v == 0xFF).map(|j| words.len() * 8 + j)
}
/// The box to request from `jpeg-decoder::scale` so the decoded image still covers `mw × mh` once
/// fitted with the source aspect ratio.
fn scaled_request(w: u32, h: u32, mw: u32, mh: u32) -> (u16, u16) {
@@ -226,7 +299,7 @@ fn mpf_images(m: &Markers) -> Vec<(usize, usize)> {
let Some(e) = entries.iter().find(|e| e.tag == 0xB002) else { return vec![] };
let Some(list) = t.bytes(e) else { return vec![] };
let mut out = Vec::new();
for (i, rec) in list.chunks_exact(16).enumerate().take(16) {
for (i, rec) in list.as_chunks::<16>().0.iter().enumerate().take(16) {
if i == 0 {
continue;
}
@@ -246,12 +319,75 @@ fn mpf_images(m: &Markers) -> Vec<(usize, usize)> {
mod tests {
use super::*;
#[test]
fn container_fallback_matches_tagged_jpeg_before_resizing_and_keeps_own_metadata() {
use crate::{ChromaSubsampling, EncodeImage, EncodeMeta, NamedSpace, Samples, SpaceOrigin, encode_jpeg};
let pixels: Vec<u8> = (0..16 * 16).flat_map(|i| if i % 2 == 0 { [120, 180, 90] } else { [30, 100, 200] }).collect();
let encode = |icc, exif| {
encode_jpeg(
&EncodeImage::new(16, 16, 3, Samples::U8(&pixels)),
100,
ChromaSubsampling::S444,
&EncodeMeta { icc, exif, ..Default::default() },
)
.unwrap()
};
let untagged = encode(None, None);
let adobe = crate::icc::write_named(NamedSpace::AdobeRgb);
let tagged = encode(Some(&adobe), None);
for opts in [DecodeOptions::default(), DecodeOptions::fit(3, 3)] {
let with = crate::decode_jpeg_with_fallback(&untagged, opts, NamedSpace::AdobeRgb).unwrap();
let expected = crate::decode(&tagged, opts).unwrap();
assert_eq!(with.space.origin, SpaceOrigin::Container);
let (a, b) = (with.to_working(), expected.to_working());
for (p, q) in a.data.iter().zip(&b.data) {
assert!(p.iter().zip(q).all(|(a, b)| (a - b).abs() < 0.0001), "{p:?} != {q:?}");
}
}
let srgb = crate::icc::write_named(NamedSpace::Srgb);
let exif = crate::exif::minimal_exif(6);
for bytes in [encode(Some(&srgb), None), encode(None, Some(&exif)), encode(Some(b"bad ICC"), None)] {
let opts = DecodeOptions::fit(5, 5);
let with = crate::decode_jpeg_with_fallback(&bytes, opts, NamedSpace::AdobeRgb).unwrap();
let expected = crate::decode(&bytes, opts).unwrap();
assert_eq!(with.space, expected.space);
assert_eq!(with.image, expected.image);
assert_eq!(with.orientation, expected.orientation);
}
assert!(crate::decode_jpeg_with_fallback(b"bad JPEG", DecodeOptions::default(), NamedSpace::AdobeRgb).is_err());
}
#[test]
fn scaled_request_covers() {
assert_eq!(scaled_request(6000, 4000, 256, 256), (256, 171));
assert_eq!(scaled_request(100, 100, 256, 256), (100, 100));
}
#[test]
fn find_ff_matches_a_plain_search() {
let mut b: Vec<u8> = (0..300u32).map(|i| (i * 7 % 255) as u8).collect();
for at in [None, Some(0), Some(7), Some(8), Some(15), Some(203), Some(296), Some(299)] {
if let Some(i) = at {
b[i] = 0xFF;
}
for start in [0, 1, 5, 9, 290, 299, 300] {
let s = &b[start..];
assert_eq!(find_ff(s), s.iter().position(|&v| v == 0xFF), "{at:?} from {start}");
}
}
}
#[test]
fn unbounded_fit_box_decodes_at_full_size() {
// a full-size load asks for the largest box (`usize::MAX` saturated to `u32::MAX`): no DCT scaling,
// and no overflow working out whether to scale
use crate::{ChromaSubsampling, EncodeImage, EncodeMeta, Samples, encode_jpeg};
let pixels = vec![128u8; 32 * 24 * 3];
let jpeg = encode_jpeg(&EncodeImage::new(32, 24, 3, Samples::U8(&pixels)), 90, ChromaSubsampling::S444, &EncodeMeta::default()).unwrap();
let d = crate::decode(&jpeg, DecodeOptions::fit(u32::MAX, u32::MAX)).unwrap();
assert_eq!((d.width, d.height), (32, 24));
}
#[test]
fn markers_on_garbage() {
assert!(parse_markers(b"").is_none());
+1 -1
View File
@@ -274,7 +274,7 @@ pub fn encode(
{
let rows: Vec<(usize, Vec<&mut [f32]>)> = {
let mut its: Vec<_> = full.iter_mut().map(|p| p.chunks_mut(width)).collect();
(0..height).map(|y| (y, its.iter_mut().map(|it| it.next().unwrap()).collect())).collect()
(0..height).map_while(|y| Some((y, its.iter_mut().map(|it| it.next()).collect::<Option<_>>()?))).collect()
};
let conv = |(y, mut out): (usize, Vec<&mut [f32]>)| {
let src = &data[y * width * channels..(y + 1) * width * channels];
+69 -3
View File
@@ -6,16 +6,20 @@
//! profiles (matrix/TRC exactly, LUT/CMYK via the `moxcms` CMS), 16-bit and float precision are
//! preserved. EXIF/XMP/ICC blobs are passed through untouched; orientation is reported, never applied.
//! - [`to_working`] converts a decode result to linear Rec.2020 D65 (the pipeline working space).
//! - [`read_header`] reads a file's stored dimensions and orientation from its headers, without
//! decoding pixels (import probes).
//! - [`decode_thumbnail`] is the fast path for grid thumbnails (EXIF thumbnail or DCT-scaled decode).
//! - [`encode`] writes JPEG, PNG, TIFF, lossless WebP and (native, feature `avif`) AVIF, embedding
//! ICC/EXIF/XMP. [`icc::write_matrix_trc`] builds profiles for export.
//!
//! No decoder panics on malformed input (property-tested).
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
mod convert;
pub mod encode;
pub mod exif;
mod heif;
pub mod icc;
mod jpeg;
pub mod jpeg_par;
@@ -155,6 +159,60 @@ pub fn decode(bytes: &[u8], opts: DecodeOptions) -> Result<Decoded> {
std::panic::catch_unwind(|| decode_unguarded(bytes, format, &opts)).unwrap_or_else(|_| Err(Error::Malformed(format, "decoder panicked".into())))
}
/// What a file's headers say about its image: enough to catalogue it without decoding pixels.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Header {
pub format: Format,
/// Full-resolution stored dimensions, orientation **not** applied ([`Decoded::source_width`]).
pub width: u32,
pub height: u32,
/// EXIF/TIFF orientation 1..=8, 1 when absent ([`Decoded::orientation`]).
pub orientation: u16,
}
/// Read the stored dimensions and orientation of any format [`decode`] supports, agreeing with
/// what a decode reports, without decoding pixels: JPEG, PNG, TIFF, WebP and PSD are read from
/// their headers (markers, chunks, IFD, resources). GIF, BMP and JPEG XL fall back to a small
/// decode.
///
/// The header readers also refuse what a decode would refuse before reaching the pixels (an
/// unsupported layout, compression or frame type) and files whose image data is cut short
/// (truncated: the data runs past the end of the file, or a JPEG's markers don't reach its
/// end-of-image marker). Damage *inside* compressed data that is all present is only found by a
/// decode. Malformed input yields an error, never a panic (as for [`decode`], a panic inside a
/// third-party parser is caught on unwinding targets).
pub fn read_header(bytes: &[u8]) -> Result<Header> {
let format = sniff(bytes).ok_or(Error::UnknownFormat)?;
std::panic::catch_unwind(|| read_header_unguarded(bytes, format))
.unwrap_or_else(|_| Err(Error::Malformed(format, "header reader panicked".into())))
}
/// [`read_header`] without the panic guard (for fuzzing our own code paths).
#[doc(hidden)]
pub fn read_header_unguarded(bytes: &[u8], format: Format) -> Result<Header> {
let (width, height, orientation) = match format {
Format::Jpeg => jpeg::header(bytes)?,
Format::Png => png_codec::header(bytes)?,
Format::Tiff => tiff_codec::header(bytes)?,
Format::WebP => webp::header(bytes)?,
Format::Psd => psd::header(bytes)?,
// rare here (or, for JPEG XL, without a header reader yet): a small decode, as before
_ => {
let d = decode_unguarded(bytes, format, &DecodeOptions::fit(64, 64))?;
(d.source_width, d.source_height, d.orientation)
}
};
Ok(Header { format, width, height, orientation })
}
/// Decode a JPEG preview with its enclosing container's colour space as a fallback.
/// A JPEG carrying ICC or EXIF metadata keeps its own interpretation. The fallback is applied
/// before transfer decoding and linear-light resizing, never to already-resampled pixels.
pub fn decode_jpeg_with_fallback(bytes: &[u8], opts: DecodeOptions, fallback: NamedSpace) -> Result<Decoded> {
std::panic::catch_unwind(|| jpeg::decode_with_fallback(bytes, &opts, Some(fallback)))
.unwrap_or_else(|_| Err(Error::Malformed(Format::Jpeg, "decoder panicked".into())))
}
/// [`decode`] without the panic guard (for fuzzing our own code paths).
#[doc(hidden)]
pub fn decode_unguarded(bytes: &[u8], format: Format, opts: &DecodeOptions) -> Result<Decoded> {
@@ -170,8 +228,8 @@ pub fn decode_unguarded(bytes: &[u8], format: Format, opts: &DecodeOptions) -> R
Format::Jxl => jxl::decode(bytes, &opts),
#[cfg(not(feature = "jxl"))]
Format::Jxl => Err(Error::Unsupported(format, "built without the `jxl` feature")),
Format::Heif => heif::decode(bytes, &opts),
Format::Avif => Err(Error::Unsupported(format, "no pure-Rust, permissively licensed AV1 decoder yet")),
Format::Heif => Err(Error::Unsupported(format, "no pure-Rust, permissively licensed HEVC decoder yet")),
Format::RawTiffLike | Format::RawOther => Err(Error::Unsupported(format, "camera raw: decode with lightcraft-raw")),
}
}
@@ -216,11 +274,19 @@ pub struct ThumbnailOptions {
/// smaller size). Defaults to `max_edge`, i.e. only previews that need no upscaling. Lower it
/// (e.g. 160 for typical EXIF thumbnails) for an instant placeholder.
pub min_embedded_edge: u32,
/// Maximum source pixels for fallback decoding (default 64 MP). Non-JPEG formats
/// decode at source resolution before resizing, so output size does not bound memory.
#[serde(default = "thumbnail_max_pixels")]
pub max_pixels: u64,
}
fn thumbnail_max_pixels() -> u64 {
64_000_000
}
impl ThumbnailOptions {
pub fn new(max_edge: u32) -> Self {
ThumbnailOptions { max_edge, min_embedded_edge: max_edge }
ThumbnailOptions { max_edge, min_embedded_edge: max_edge, max_pixels: thumbnail_max_pixels() }
}
}
@@ -239,7 +305,7 @@ pub fn decode_thumbnail_with(bytes: &[u8], opts: &ThumbnailOptions) -> Result<Th
{
return Ok(t);
}
let d = decode(bytes, DecodeOptions::fit(max_edge, max_edge))?;
let d = decode(bytes, DecodeOptions { max_size: Some((max_edge, max_edge)), max_pixels: opts.max_pixels })?;
Ok(Thumbnail {
image: d.to_srgb8(),
orientation: d.orientation,
+70 -1
View File
@@ -37,7 +37,7 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
};
buf.truncate(out.buffer_size());
let (samples, depth) = match out.bit_depth {
png::BitDepth::Sixteen => (Buf::U16(buf.chunks_exact(2).map(|c| u16::from_be_bytes([c[0], c[1]])).collect()), 16),
png::BitDepth::Sixteen => (Buf::U16(buf.as_chunks::<2>().0.iter().map(|c| u16::from_be_bytes([c[0], c[1]])).collect()), 16),
_ => (Buf::U8(buf), info.bit_depth as u8),
};
let raw = Raw { width: out.width as usize, height: out.height as usize, model, alpha, premultiplied: false, buf: samples, bit_depth: depth };
@@ -57,6 +57,75 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
finish(F, raw, meta, (w, h), opts)
}
/// Stored dimensions and EXIF orientation (`eXIf`, wherever it is) from the chunk headers, without
/// inflating the image data. Refused, as a decode would refuse it: an invalid IHDR, an indexed
/// image without a palette, no IDAT, or a file that ends before the chunk after the last IDAT
/// begins (truncated image data). Chunk CRCs and the deflate stream are not checked.
pub(crate) fn header(b: &[u8]) -> Result<(u32, u32, u16)> {
if !b.starts_with(b"\x89PNG\r\n\x1a\n") {
return Err(err("bad signature"));
}
let be32 = |p: usize| b.get(p..p.checked_add(4)?).map(|v| u32::from_be_bytes([v[0], v[1], v[2], v[3]]));
if be32(8) != Some(13) || b.get(12..16) != Some(b"IHDR") {
return Err(err("missing IHDR"));
}
let ihdr = b.get(16..29).ok_or_else(|| err("truncated IHDR"))?;
let (w, h) = (u32::from_be_bytes([ihdr[0], ihdr[1], ihdr[2], ihdr[3]]), u32::from_be_bytes([ihdr[4], ihdr[5], ihdr[6], ihdr[7]]));
let (depth, color, compression, filter, interlace) = (ihdr[8], ihdr[9], ihdr[10], ihdr[11], ihdr[12]);
if w > i32::MAX as u32 || h > i32::MAX as u32 {
return Err(err("dimensions out of range"));
}
check_size(F, w as u64, h as u64, &DecodeOptions::default())?;
let depth_ok = match color {
0 => matches!(depth, 1 | 2 | 4 | 8 | 16),
3 => matches!(depth, 1 | 2 | 4 | 8),
2 | 4 | 6 => matches!(depth, 8 | 16),
_ => false,
};
if !depth_ok || compression != 0 || filter != 0 || interlace > 1 {
return Err(err("invalid IHDR"));
}
let (mut p, mut palette, mut idat, mut after_idat, mut exif) = (8usize, false, false, false, None);
while let Some(len) = be32(p) {
let Some(kind) = p.checked_add(4).and_then(|k| b.get(k..k.checked_add(4)?)) else { break };
if idat && kind != b"IDAT" {
// a chunk follows the image data: the IDAT run is complete
after_idat = true;
}
if kind == b"IEND" {
break;
}
let start = p.saturating_add(8);
let data = (len <= i32::MAX as u32).then(|| start.checked_add(len as usize).and_then(|end| b.get(start..end))).flatten();
let Some(data) = data else {
if after_idat {
break; // a damaged chunk after the image data: a decode ignores it too
}
return Err(err("truncated"));
};
match kind {
b"PLTE" => palette = true,
b"IDAT" if after_idat => {} // a stray IDAT after other chunks: the first run is the image
b"IDAT" => {
if color == 3 && !palette {
return Err(err("indexed image without a palette"));
}
idat = true;
}
b"eXIf" if exif.is_none() => exif = Some(data.strip_prefix(b"Exif\0\0").unwrap_or(data)),
_ => {}
}
p = start.saturating_add(data.len()).saturating_add(4);
}
if !idat {
return Err(err("no image data"));
}
if !after_idat {
return Err(err("truncated image data"));
}
Ok((w, h, exif.map(crate::exif::summarize).unwrap_or_default().orientation.unwrap_or(1)))
}
fn container_hint(info: &png::Info) -> Option<SourceSpace> {
if let Some(c) = &info.coding_independent_code_points {
let named = match c.color_primaries {
+265 -72
View File
@@ -69,8 +69,8 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let (icc, exif, xmp) = resources(res);
// Layer and mask information: only the layer count sign matters (merged transparency).
let lm_len = if psb { r.u64()? as usize } else { r.u32()? as usize };
let lm = r.take(lm_len)?;
let lm_len = if psb { r.u64()? } else { u64::from(r.u32()?) };
let lm = r.take(usize::try_from(lm_len).map_err(|_| err("truncated"))?)?;
let merged_transparency = if lm.len() >= if psb { 10 } else { 6 } {
let off = if psb { 8 } else { 4 };
let count = i16::from_be_bytes([lm[off], lm[off + 1]]);
@@ -88,33 +88,6 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
1 => return Err(Error::Unsupported(F, "1-bit bitmap PSD")),
_ => return Err(err("bad depth")),
};
let plane = width * height * bpc;
let row_bytes = width * bpc;
let mut planes: Vec<Vec<u8>> = Vec::with_capacity(channels);
match compression {
0 => {
for _ in 0..channels {
planes.push(r.take(plane)?.to_vec());
}
}
1 => {
let counts_len = channels * height;
let mut counts = Vec::with_capacity(counts_len);
for _ in 0..counts_len {
counts.push(if psb { r.u32()? as usize } else { r.u16()? as usize });
}
for c in 0..channels {
let mut out = Vec::with_capacity(plane);
for y in 0..height {
let src = r.take(counts[c * height + y])?;
unpackbits(src, row_bytes, &mut out);
}
planes.push(out);
}
}
_ => return Err(Error::Unsupported(F, "ZIP-compressed merged image data")),
}
let (model, n_color) = match mode {
1 | 8 => (Model::Gray, 1),
2 => (Model::Rgb, 1),
@@ -128,50 +101,106 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
return Err(err("too few channels"));
}
let alpha = merged_transparency && channels > n_color && mode != 2;
// Only the colour (+ alpha) planes are decoded; extra channels (spot colours, masks) are
// stored after them and never touched.
let used = n_color + alpha as usize;
let n = width * height;
let too_large = || Error::TooLarge(width as u64, height as u64);
let n = width.checked_mul(height).ok_or_else(too_large)?;
let row_bytes = width.checked_mul(bpc).ok_or_else(too_large)?;
let plane = n.checked_mul(bpc).ok_or_else(too_large)?;
let mut planes: Vec<Vec<u8>> = Vec::with_capacity(used);
match compression {
0 => {
// Raw planes are borrowed from the input, so their size is bounded by the file.
for _ in 0..used {
planes.push(r.take(plane)?.to_vec());
}
}
1 => {
// Row byte-count table: one entry per row per channel (all channels, since the data
// follows it). Taken from the input before anything is allocated from it.
let entry = if psb { 4 } else { 2 };
let table_len = channels.checked_mul(height).and_then(|c| c.checked_mul(entry)).ok_or_else(|| err("truncated"))?;
let table = r.take(table_len)?;
let count = |i: usize| -> usize {
let at = i.saturating_mul(entry);
match table.get(at..at.saturating_add(entry)) {
Some(&[a, b]) => usize::from(u16::from_be_bytes([a, b])),
Some(&[a, b, c, d]) => u32::from_be_bytes([a, b, c, d]) as usize,
_ => 0,
}
};
// PackBits expands at most 64× (a 2-byte repeat run yields 128 bytes), so a row needs
// at least this many encoded bytes. Requiring it (and the data being present) before
// allocating bounds the decoded planes by the file size: a tiny file can no longer
// declare gigabytes of zero-padded or repeat-run rows.
let min_row = row_bytes.div_ceil(128).saturating_mul(2);
let mut data_len = 0usize;
for i in 0..used.saturating_mul(height) {
let c = count(i);
if c < min_row {
return Err(err("RLE row too short for its width"));
}
data_len = data_len.saturating_add(c);
}
if data_len > bytes.len().saturating_sub(r.p) {
return Err(err("truncated"));
}
for c in 0..used {
let mut out = zeroed::<u8>(plane, too_large)?;
for (y, row) in out.chunks_exact_mut(row_bytes).enumerate() {
let src = r.take(count(c * height + y))?;
unpackbits(src, row)?;
}
planes.push(out);
}
}
_ => return Err(Error::Unsupported(F, "ZIP-compressed merged image data")),
}
let samples = n.checked_mul(used).ok_or_else(too_large)?;
// Interleave to device samples.
let buf = if mode == 2 {
// Indexed → 8-bit RGB via the 768-byte palette.
if color_mode_data.len() < 768 || depth != 8 {
let (Some(pal), true) = (color_mode_data.get(..768), depth == 8) else {
return Err(err("bad indexed palette"));
}
let mut v = Vec::with_capacity(n * 3);
for &i in &planes[0] {
};
let idx = planes.first().ok_or_else(|| err("missing plane"))?;
let mut v = zeroed::<u8>(n.checked_mul(3).ok_or_else(too_large)?, too_large)?;
for (o, &i) in v.as_chunks_mut::<3>().0.iter_mut().zip(idx) {
let i = i as usize;
v.extend_from_slice(&[color_mode_data[i], color_mode_data[256 + i], color_mode_data[512 + i]]);
*o = [pal[i], pal[256 + i], pal[512 + i]];
}
Buf::U8(v)
} else {
let cmyk = model == Model::Cmyk;
match bpc {
1 => {
let mut v = vec![0u8; n * used];
for (c, p) in planes.iter().take(used).enumerate() {
for i in 0..n {
let x = p[i];
let mut v = zeroed::<u8>(samples, too_large)?;
for (c, p) in planes.iter().enumerate() {
for (o, &x) in v.iter_mut().skip(c).step_by(used).zip(p) {
// PSD CMYK stores 255 = no ink; our CMYK convention is 0 = no ink.
v[i * used + c] = if model == Model::Cmyk && c < 4 { 255 - x } else { x };
*o = if cmyk && c < 4 { 255 - x } else { x };
}
}
Buf::U8(v)
}
2 => {
let mut v = vec![0u16; n * used];
for (c, p) in planes.iter().take(used).enumerate() {
for i in 0..n {
let x = u16::from_be_bytes([p[2 * i], p[2 * i + 1]]);
v[i * used + c] = if model == Model::Cmyk && c < 4 { 65535 - x } else { x };
let mut v = zeroed::<u16>(samples, too_large)?;
for (c, p) in planes.iter().enumerate() {
for (o, s) in v.iter_mut().skip(c).step_by(used).zip(p.as_chunks::<2>().0) {
let x = u16::from_be_bytes(*s);
*o = if cmyk && c < 4 { 65535 - x } else { x };
}
}
Buf::U16(v)
}
_ => {
let mut v = vec![0f32; n * used];
for (c, p) in planes.iter().take(used).enumerate() {
for i in 0..n {
let x = f32::from_be_bytes([p[4 * i], p[4 * i + 1], p[4 * i + 2], p[4 * i + 3]]);
v[i * used + c] = if model == Model::Cmyk && c < 4 { 1.0 - x } else { x };
let mut v = zeroed::<f32>(samples, too_large)?;
for (c, p) in planes.iter().enumerate() {
for (o, s) in v.iter_mut().skip(c).step_by(used).zip(p.as_chunks::<4>().0) {
let x = f32::from_be_bytes(*s);
*o = if cmyk && c < 4 { 1.0 - x } else { x };
}
}
Buf::F32(v)
@@ -182,25 +211,111 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
finish(F, raw, Meta { icc, exif, xmp, ..Default::default() }, (width as u32, height as u32), opts)
}
/// PackBits: append exactly `row_bytes` bytes (zero-padded if the run data is short).
fn unpackbits(mut src: &[u8], row_bytes: usize, out: &mut Vec<u8>) {
let target = out.len() + row_bytes;
while out.len() < target && !src.is_empty() {
let h = src[0] as i8;
src = &src[1..];
if h >= 0 {
let n = (h as usize + 1).min(src.len()).min(target - out.len());
out.extend_from_slice(&src[..n]);
src = &src[(h as usize + 1).min(src.len())..];
} else if h != -128 {
let n = ((1 - h as isize) as usize).min(target - out.len());
if let Some(&b) = src.first() {
out.extend(std::iter::repeat_n(b, n));
src = &src[1..];
/// Stored dimensions and EXIF orientation (resource 1058) from the header and image resources,
/// without reading the merged image data. Refused, as a decode would refuse it: an unsupported
/// depth, colour mode or compression, or merged image data that runs past the end of the file
/// (truncated). The run-length data itself is not unpacked.
pub(crate) fn header(bytes: &[u8]) -> Result<(u32, u32, u16)> {
let mut r = Rd { b: bytes, p: 0 };
if r.take(4)? != b"8BPS" {
return Err(err("bad signature"));
}
let psb = match r.u16()? {
1 => false,
2 => true,
_ => return Err(err("bad version")),
};
r.take(6)?;
let channels = r.u16()? as usize;
let height = r.u32()? as usize;
let width = r.u32()? as usize;
let depth = r.u16()?;
let mode = r.u16()?;
check_size(F, width as u64, height as u64, &DecodeOptions::default())?;
if channels == 0 || channels > 56 {
return Err(err("bad channel count"));
}
let cm_len = r.u32()? as usize;
let color_mode_data = r.take(cm_len)?;
let res_len = r.u32()? as usize;
let (_, exif, _) = resources(r.take(res_len)?);
let lm_len = if psb { r.u64()? as usize } else { r.u32()? as usize };
r.take(lm_len)?;
let compression = r.u16()?;
let bpc = match depth {
8 => 1usize,
16 => 2,
32 => 4,
1 => return Err(Error::Unsupported(F, "1-bit bitmap PSD")),
_ => return Err(err("bad depth")),
};
let n_color = match mode {
1 | 8 | 7 | 2 => 1,
3 => 3,
4 => 4,
9 => return Err(Error::Unsupported(F, "Lab PSD")),
_ => return Err(err("unknown colour mode")),
};
if channels < n_color {
return Err(err("too few channels"));
}
if mode == 2 && (color_mode_data.len() < 768 || depth != 8) {
return Err(err("bad indexed palette"));
}
let rest = bytes.len().saturating_sub(r.p);
let need = match compression {
0 => width.checked_mul(height).and_then(|n| n.checked_mul(bpc)).and_then(|n| n.checked_mul(channels)),
1 => {
// the row byte counts, then the rows they count
let rows = channels.checked_mul(height).ok_or_else(|| err("truncated"))?;
let table = rows.checked_mul(if psb { 4 } else { 2 }).filter(|&t| t <= rest).ok_or_else(|| err("truncated"))?;
let mut total = table;
for _ in 0..rows {
total = total.saturating_add(if psb { r.u32()? as usize } else { r.u16()? as usize });
}
Some(total)
}
_ => return Err(Error::Unsupported(F, "ZIP-compressed merged image data")),
};
if need.is_none_or(|n| n > rest) {
return Err(err("truncated"));
}
let orientation = exif.as_deref().map(crate::exif::summarize).unwrap_or_default().orientation.unwrap_or(1);
Ok((width as u32, height as u32, orientation))
}
/// A zero-filled buffer whose allocation failure is an error, not an abort.
fn zeroed<T: Clone + Default>(len: usize, too_large: impl Fn() -> Error) -> Result<Vec<T>> {
let mut v = Vec::new();
v.try_reserve_exact(len).map_err(|_| too_large())?;
v.resize(len, T::default());
Ok(v)
}
/// PackBits: decode one row into `row` (pre-zeroed, so short run data leaves zero padding).
/// A run that would write past the end of the row is an error; trailing bytes are ignored.
fn unpackbits(mut src: &[u8], row: &mut [u8]) -> Result<()> {
let mut at = 0;
while at < row.len() {
let Some((&h, rest)) = src.split_first() else { break };
src = rest;
let h = h as i8;
if h >= 0 {
let n = h as usize + 1;
let lit = src.get(..n).unwrap_or(src);
let dst = row.get_mut(at..at + lit.len()).ok_or_else(|| err("RLE row overruns its width"))?;
dst.copy_from_slice(lit);
at += lit.len();
src = src.get(n..).unwrap_or(&[]);
} else if h != -128 {
let n = (1 - h as isize) as usize;
let Some((&b, rest)) = src.split_first() else { break };
src = rest;
row.get_mut(at..at + n).ok_or_else(|| err("RLE row overruns its width"))?.fill(b);
at += n;
}
}
out.resize(target, 0);
Ok(())
}
fn resources(mut b: &[u8]) -> (Option<Vec<u8>>, Option<Vec<u8>>, Option<String>) {
@@ -234,11 +349,89 @@ mod tests {
#[test]
fn packbits() {
let mut out = Vec::new();
unpackbits(&[0xFE, 0xAA, 0x02, 0x80, 0x00, 0x2A], 6, &mut out);
assert_eq!(out, vec![0xAA, 0xAA, 0xAA, 0x80, 0x00, 0x2A]);
let mut out = Vec::new();
unpackbits(&[0x05, 1], 4, &mut out);
assert_eq!(out, vec![1, 0, 0, 0]);
let mut out = [0u8; 6];
unpackbits(&[0xFE, 0xAA, 0x02, 0x80, 0x00, 0x2A], &mut out).unwrap();
assert_eq!(out, [0xAA, 0xAA, 0xAA, 0x80, 0x00, 0x2A]);
// Short run data is zero-padded.
let mut out = [0u8; 4];
unpackbits(&[0x01, 1], &mut out).unwrap();
assert_eq!(out, [1, 0, 0, 0]);
// Truncated literal run: copies what is there.
let mut out = [0u8; 4];
unpackbits(&[0x03, 7, 8], &mut out).unwrap();
assert_eq!(out, [7, 8, 0, 0]);
}
#[test]
fn packbits_overrun_is_an_error() {
// Repeat run of 128 into a 4-byte row.
let mut out = [0u8; 4];
assert!(unpackbits(&[0x81, 0xAA], &mut out).is_err());
// Literal run of 6 into a 4-byte row.
assert!(unpackbits(&[0x05, 1, 2, 3, 4, 5, 6], &mut out).is_err());
}
/// PSD header + empty colour-mode/resources/layer sections + compression id.
fn header(channels: u16, w: u32, h: u32, depth: u16, mode: u16, compression: u16) -> Vec<u8> {
let mut b = b"8BPS".to_vec();
b.extend_from_slice(&1u16.to_be_bytes());
b.extend_from_slice(&[0; 6]);
b.extend_from_slice(&channels.to_be_bytes());
b.extend_from_slice(&h.to_be_bytes());
b.extend_from_slice(&w.to_be_bytes());
b.extend_from_slice(&depth.to_be_bytes());
b.extend_from_slice(&mode.to_be_bytes());
b.extend_from_slice(&[0; 12]); // colour mode data, resources, layer & mask: all empty
b.extend_from_slice(&compression.to_be_bytes());
b
}
#[test]
fn huge_rle_with_truncated_count_table_is_rejected_before_allocating() {
// 1 × 2^30 px, 56 channels: the row table alone would be 56 Gi entries; the file has 4 bytes.
let mut b = header(56, 1, 1 << 30, 32, 3, 1);
b.extend_from_slice(&[0, 0, 0, 0]);
let e = decode(&b, &DecodeOptions::default()).unwrap_err();
assert!(matches!(e, Error::Malformed(_, ref m) if m == "truncated"), "{e:?}");
}
#[test]
fn huge_rle_of_empty_rows_is_rejected_before_allocating() {
// 2^30 × 1 px RGB at 32 bits: every row count is 0, which used to zero-pad 3 × 4 GiB.
let mut b = header(3, 1 << 30, 1, 32, 3, 1);
b.extend_from_slice(&[0; 6]);
let e = decode(&b, &DecodeOptions::default()).unwrap_err();
assert!(matches!(e, Error::Malformed(_, ref m) if m.contains("too short")), "{e:?}");
// 1 Gpx RGB 8-bit whose (plausible) row counts claim more data than the file holds.
let mut b = header(3, 1 << 20, 1 << 10, 8, 3, 1);
b.extend_from_slice(&[0xFF; 3 * 1024 * 2]);
let e = decode(&b, &DecodeOptions::default()).unwrap_err();
assert!(matches!(e, Error::Malformed(_, ref m) if m == "truncated"), "{e:?}");
}
#[test]
fn rle_repeat_run_past_row_width_is_an_error() {
// 4 × 1 gray 8-bit; the single row is a 128-byte repeat run.
let mut b = header(1, 4, 1, 8, 1, 1);
b.extend_from_slice(&2u16.to_be_bytes());
b.extend_from_slice(&[0x81, 0x55]);
let e = decode(&b, &DecodeOptions::default()).unwrap_err();
assert!(matches!(e, Error::Malformed(_, ref m) if m.contains("overruns")), "{e:?}");
}
#[test]
fn rle_gray16_and_extra_channels_decode() {
// 2 × 2 gray 16-bit with an extra (unused) channel whose data is absent: only the used
// plane is decoded. Row: repeat run of 4 bytes 0x80 -> samples 0x8080.
let mut b = header(2, 2, 2, 16, 1, 1);
for _ in 0..4 {
b.extend_from_slice(&2u16.to_be_bytes());
}
b.extend_from_slice(&[0xFD, 0x80, 0xFD, 0x80]);
let d = decode(&b, &DecodeOptions::default()).unwrap();
assert_eq!((d.width, d.height), (2, 2));
let p = d.image.get(1, 1);
let q = d.image.get(0, 0);
assert!(p[0] > 0.0 && (p[0] - q[0]).abs() < 1e-6, "{p:?}");
}
}
+45 -1
View File
@@ -66,7 +66,8 @@ impl Format {
pub fn can_decode(self) -> bool {
match self {
Format::Jxl => cfg!(feature = "jxl"),
Format::Avif | Format::Heif | Format::RawTiffLike | Format::RawOther => false,
Format::Heif => cfg!(feature = "heif"),
Format::Avif | Format::RawTiffLike | Format::RawOther => false,
_ => true,
}
}
@@ -163,6 +164,9 @@ fn tiff_is_raw(b: &[u8]) -> bool {
}
let Some(t) = Tiff::new(b) else { return false };
let Some(ifd0) = t.first_ifd() else { return false };
let sony = t.ifd(ifd0).is_some_and(|(entries, _)| {
entries.iter().any(|e| e.tag == 0x010f && t.bytes(e).and_then(|v| v.get(..4)).is_some_and(|v| v.eq_ignore_ascii_case(b"SONY")))
});
let mut stack = vec![ifd0];
let mut seen = 0;
while let Some(pos) = stack.pop() {
@@ -171,6 +175,12 @@ fn tiff_is_raw(b: &[u8]) -> bool {
break;
}
let Some((entries, _)) = t.ifd(pos) else { continue };
// Sony M/S lossless ARWs retain a dummy CFA pattern, but store linear YCbCr.
// Requiring the raw pattern plus this layout keeps ordinary Sony TIFFs as TIFFs.
let uint = |tag| entries.iter().find(|e| e.tag == tag).and_then(|e| t.uint(e));
if sony && entries.iter().any(|e| e.tag == 0x828e) && uint(0x0103) == Some(7) && uint(0x0106) == Some(6) && uint(0x0115) == Some(3) {
return true;
}
for e in &entries {
match e.tag {
0xC612 => return true, // DNGVersion
@@ -233,4 +243,38 @@ mod tests {
b.extend_from_slice(&0u32.to_le_bytes());
assert_eq!(sniff(&b), Some(Format::RawTiffLike));
}
#[test]
fn sony_linear_ycbcr_subifd_is_raw_but_ordinary_tiff_is_not() {
fn fixture(compression: u32, pattern: bool) -> Vec<u8> {
let mut b = b"II*\0\x08\0\0\0".to_vec();
let entry = |b: &mut Vec<u8>, tag: u16, typ: u16, count: u32, value: u32| {
b.extend_from_slice(&tag.to_le_bytes());
b.extend_from_slice(&typ.to_le_bytes());
b.extend_from_slice(&count.to_le_bytes());
b.extend_from_slice(&value.to_le_bytes());
};
b.extend_from_slice(&2u16.to_le_bytes());
entry(&mut b, 0x010f, 2, 5, 38);
entry(&mut b, 0x014a, 4, 1, 44);
b.extend_from_slice(&0u32.to_le_bytes());
b.extend_from_slice(b"SONY\0\0");
b.extend_from_slice(&(if pattern { 4u16 } else { 3 }).to_le_bytes());
entry(&mut b, 0x0103, 3, 1, compression);
entry(&mut b, 0x0106, 3, 1, 6);
entry(&mut b, 0x0115, 3, 1, 3);
if pattern {
entry(&mut b, 0x828e, 1, 4, u32::MAX);
}
b.extend_from_slice(&0u32.to_le_bytes());
b
}
let mut raw = fixture(7, true);
assert_eq!(sniff(&raw), Some(Format::RawTiffLike));
assert_eq!(sniff(&fixture(7, false)), Some(Format::Tiff));
assert_eq!(sniff(&fixture(1, true)), Some(Format::Tiff));
assert_eq!(sniff(&raw[..38]), Some(Format::Tiff));
raw[38..42].copy_from_slice(b"TEST");
assert_eq!(sniff(&raw), Some(Format::Tiff));
}
}
+92 -29
View File
@@ -1,5 +1,6 @@
//! TIFF decode via the `tiff` crate: 8/16/32-bit integer, 16/32/64-bit float, gray/RGB/CMYK/palette,
//! alpha (associated or not), ICC (34675), XMP (700), orientation (274). First image only.
//! TIFF decode via the `tiff` crate: 1/2/4/8/16/32/64-bit integer, 16/32/64-bit float, gray/RGB/CMYK,
//! alpha (associated or not), ICC (34675), XMP (700), orientation (274). First image only. Palette
//! (indexed-colour) images are refused by the `tiff` crate itself, on every read path.
use crate::convert::{Buf, Meta, Model, Raw, check_size, finish};
use crate::{DecodeOptions, Decoded, Error, Format, Result};
@@ -14,10 +15,7 @@ fn err(e: impl std::fmt::Display) -> Error {
}
pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let mut limits = Limits::default();
limits.decoding_buffer_size = opts.max_pixels.saturating_mul(4 * 8).min(usize::MAX as u64) as usize;
limits.ifd_value_size = 32 << 20;
let mut d = Decoder::new(std::io::Cursor::new(bytes)).map_err(err)?.with_limits(limits);
let mut d = Decoder::new(std::io::Cursor::new(bytes)).map_err(err)?.with_limits(limits(opts));
let (w, h) = d.dimensions().map_err(err)?;
check_size(F, w as u64, h as u64, opts)?;
let ct = d.colortype().map_err(err)?;
@@ -25,26 +23,12 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let icc = tag_bytes(&mut d, Tag::IccProfile);
let xmp = tag_bytes(&mut d, Tag::from_u16_exhaustive(700)).map(|v| String::from_utf8_lossy(&v).trim_end_matches('\0').to_string());
let orientation = d.get_tag_u32(Tag::Orientation).ok().map(|v| v as u16);
let photometric = d.get_tag_u32(Tag::PhotometricInterpretation).ok();
let planar = d.get_tag_u32(Tag::PlanarConfiguration).ok() == Some(2);
let extra = d.get_tag_u16_vec(Tag::ExtraSamples).ok().unwrap_or_default();
let colormap = d.get_tag_u16_vec(Tag::ColorMap).ok();
let compression = d.get_tag_u32(Tag::Compression).ok();
let (model, n_color, n_samples, bits) = match ct {
ColorType::Gray(b) => (Model::Gray, 1, 1, b),
ColorType::GrayA(b) => (Model::Gray, 1, 2, b),
ColorType::RGB(b) => (Model::Rgb, 3, 3, b),
ColorType::RGBA(b) => (Model::Rgb, 3, 4, b),
ColorType::CMYK(b) => (Model::Cmyk, 4, 4, b),
ColorType::CMYKA(b) => (Model::Cmyk, 4, 5, b),
ColorType::Palette(b) => (Model::Rgb, 1, 1, b),
// JPEG-compressed YCbCr is converted to RGB by the JPEG decoder.
ColorType::YCbCr(b) if compression == Some(7) => (Model::Rgb, 3, 3, b),
ColorType::Multiband { bit_depth, num_samples } if num_samples >= 3 => (Model::Rgb, 3, num_samples as usize, bit_depth),
ColorType::Multiband { bit_depth, num_samples } if num_samples >= 1 => (Model::Gray, 1, num_samples as usize, bit_depth),
other => return Err(Error::Unsupported(F, unsupported_name(other))),
};
let (model, n_color, n_samples, bits) = layout(ct, compression, planar)?;
let has_alpha = n_samples > n_color && !matches!(ct, ColorType::Palette(_));
let premultiplied = has_alpha && extra.first() == Some(&1);
@@ -52,6 +36,7 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let (wu, hu) = (w as usize, h as usize);
let n = wu * hu;
let mut buf = match res {
DecodingResult::U8(v) if bits < 8 => Buf::U8(unpack(&v, wu, hu, n_samples, bits)?),
DecodingResult::U8(v) => Buf::U8(v),
DecodingResult::U16(v) => Buf::U16(v),
// Rare integer widths: keep 16 significant bits (plenty for display-referred data).
@@ -94,14 +79,8 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
return finish(F, raw, meta, (w, h), opts);
}
// WhiteIsZero → invert.
if photometric == Some(0) {
match &mut buf {
Buf::U8(v) => v.iter_mut().for_each(|x| *x = 255 - *x),
Buf::U16(v) => v.iter_mut().for_each(|x| *x = 65535 - *x),
Buf::F32(v) => v.iter_mut().for_each(|x| *x = 1.0 - *x),
}
}
// WhiteIsZero needs nothing here: the `tiff` crate inverts those samples as it reads them (and
// refuses WhiteIsZero layouts it can't invert), so inverting again would show a negative.
// Drop extra samples beyond colour + one alpha.
let keep = n_color + has_alpha as usize;
@@ -113,6 +92,67 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
finish(F, raw, meta, (w, h), opts)
}
fn limits(opts: &DecodeOptions) -> Limits {
let mut limits = Limits::default();
limits.decoding_buffer_size = opts.max_pixels.saturating_mul(4 * 8).min(usize::MAX as u64) as usize;
limits.ifd_value_size = 32 << 20;
limits
}
/// The sample layout of a colour type we can decode: (model, colour samples, samples per pixel,
/// bits per sample).
fn layout(ct: ColorType, compression: Option<u32>, planar: bool) -> Result<(Model, usize, usize, u8)> {
let l = match ct {
ColorType::Gray(b) => (Model::Gray, 1, 1, b),
ColorType::GrayA(b) => (Model::Gray, 1, 2, b),
ColorType::RGB(b) => (Model::Rgb, 3, 3, b),
ColorType::RGBA(b) => (Model::Rgb, 3, 4, b),
ColorType::CMYK(b) => (Model::Cmyk, 4, 4, b),
ColorType::CMYKA(b) => (Model::Cmyk, 4, 5, b),
ColorType::Palette(b) => (Model::Rgb, 1, 1, b),
// JPEG-compressed YCbCr is converted to RGB by the JPEG decoder.
ColorType::YCbCr(b) if compression == Some(7) => (Model::Rgb, 3, 3, b),
ColorType::Multiband { bit_depth, num_samples } if num_samples >= 3 => (Model::Rgb, 3, num_samples as usize, bit_depth),
ColorType::Multiband { bit_depth, num_samples } if num_samples >= 1 => (Model::Gray, 1, num_samples as usize, bit_depth),
other => return Err(Error::Unsupported(F, unsupported_name(other))),
};
if l.3 < 8 && planar && l.2 > 1 {
return Err(Error::Unsupported(F, "planar TIFF with samples narrower than a byte"));
}
Ok(l)
}
/// Stored dimensions and orientation from the first IFD, without reading the image data. Refused,
/// as a decode would refuse it: a layout or compression we can't decode, or strips/tiles that
/// start past the end of the file (truncated). The compressed data itself is not checked.
pub(crate) fn header(bytes: &[u8]) -> Result<(u32, u32, u16)> {
let mut d = Decoder::new(std::io::Cursor::new(bytes)).map_err(err)?.with_limits(limits(&DecodeOptions::default()));
let (w, h) = d.dimensions().map_err(err)?;
check_size(F, w as u64, h as u64, &DecodeOptions::default())?;
let ct = d.colortype().map_err(err)?;
let compression = d.get_tag_u32(Tag::Compression).ok();
let planar = d.get_tag_u32(Tag::PlanarConfiguration).ok() == Some(2);
layout(ct, compression, planar)?;
// what this build of the `tiff` crate decompresses: none, CCITT G4, LZW, JPEG, Deflate, PackBits
if !matches!(compression.unwrap_or(1), 1 | 4 | 5 | 7 | 8 | 32773 | 32946) {
return Err(Error::Unsupported(F, "compression method"));
}
let chunks = |offsets: Tag, counts: Tag, d: &mut Decoder<std::io::Cursor<&[u8]>>| {
Some((d.get_tag_u64_vec(offsets).ok()?, d.get_tag_u64_vec(counts).ok()?))
};
let (offsets, counts) = chunks(Tag::StripOffsets, Tag::StripByteCounts, &mut d)
.or_else(|| chunks(Tag::TileOffsets, Tag::TileByteCounts, &mut d))
.ok_or_else(|| err("no image data"))?;
// Only where each chunk starts: some writers overstate StripByteCounts, and the `tiff` crate
// decodes such files (it stops once it has the pixels), so a chunk's end proves nothing.
let len = bytes.len() as u64;
if offsets.is_empty() || offsets.iter().zip(&counts).any(|(&o, &c)| c > 0 && o >= len) {
return Err(err("truncated: image data past the end of the file"));
}
let orientation = d.get_tag_u32(Tag::Orientation).ok().map(|v| v as u16).filter(|o| (1..=8).contains(o)).unwrap_or(1);
Ok((w, h, orientation))
}
/// A BYTE/UNDEFINED tag as bytes (the decoder may surface unknown byte tags as unsigned values).
fn tag_bytes<R: std::io::Read + std::io::Seek>(d: &mut Decoder<R>, tag: Tag) -> Option<Vec<u8>> {
use tiff::decoder::ifd::Value;
@@ -140,6 +180,29 @@ fn unsupported_name(ct: ColorType) -> &'static str {
}
}
/// Samples narrower than a byte (1-bit bilevel and fax scans, 2- and 4-bit gray) arrive packed, most
/// significant bits first, each row padded to a whole byte. Spread them out one per byte, scaled to
/// 0–255 like any 8-bit sample.
fn unpack(packed: &[u8], width: usize, height: usize, samples: usize, bits: u8) -> Result<Vec<u8>> {
if !matches!(bits, 1 | 2 | 4) {
return Err(Error::Unsupported(F, "TIFF samples of 3, 5, 6 or 7 bits"));
}
let bits = bits as usize;
let per_row = width.checked_mul(samples).ok_or_else(|| err("row too wide"))?;
let row_bytes = per_row.checked_mul(bits).ok_or_else(|| err("row too wide"))?.div_ceil(8);
if row_bytes == 0 || packed.len() / row_bytes < height {
return Err(err("sample buffer too short"));
}
let max = (1u8 << bits) - 1;
let scale = 255 / max; // 255, 85, 17: exact
let mut out = Vec::with_capacity(per_row.saturating_mul(height));
for row in packed.chunks_exact(row_bytes).take(height) {
let values = row.iter().flat_map(|&b| (1..=8 / bits).map(move |i| (b >> (8 - bits * i)) & max));
out.extend(values.take(per_row).map(|v| v * scale));
}
Ok(out)
}
fn interleave(buf: Buf, n: usize, s: usize) -> Buf {
fn go<T: Copy + Default>(v: Vec<T>, n: usize, s: usize) -> Vec<T> {
if v.len() < n * s {
+30
View File
@@ -12,6 +12,9 @@ fn err(e: impl std::fmt::Display) -> Error {
pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let mut d = image_webp::WebPDecoder::new(std::io::Cursor::new(bytes)).map_err(err)?;
// RIFF chunk lengths are untrusted. Metadata cannot exceed the input;
// otherwise a tiny truncated file can request gigabytes before read_exact fails.
d.set_memory_limit(bytes.len());
let (w, h) = d.dimensions();
check_size(F, w as u64, h as u64, opts)?;
let alpha = d.has_alpha();
@@ -24,3 +27,30 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let raw = Raw { width: w as usize, height: h as usize, model: Model::Rgb, alpha, premultiplied: false, buf: Buf::U8(buf), bit_depth: 8 };
finish(F, raw, Meta { icc, exif, xmp, ..Default::default() }, (w, h), opts)
}
/// Stored dimensions and EXIF orientation from the RIFF chunks and the bitstream header, without
/// decoding the image. Refused, as a decode would refuse it: no image chunk (`VP8 `, `VP8L` or an
/// animation frame), or one that runs past the end of the file (truncated). The compressed data
/// itself is not checked.
pub(crate) fn header(bytes: &[u8]) -> Result<(u32, u32, u16)> {
let mut p = 12usize;
loop {
let Some(h) = bytes.get(p..p.saturating_add(8)) else { return Err(err("no image data")) };
let size = u32::from_le_bytes([h[4], h[5], h[6], h[7]]) as usize;
let end = p.saturating_add(8).saturating_add(size);
if matches!(&h[0..4], b"VP8 " | b"VP8L" | b"ANMF") {
if end > bytes.len() {
return Err(err("truncated image data"));
}
break;
}
p = end.saturating_add(size & 1);
}
let mut d = image_webp::WebPDecoder::new(std::io::Cursor::new(bytes)).map_err(err)?;
d.set_memory_limit(bytes.len());
let (w, h) = d.dimensions();
check_size(F, w as u64, h as u64, &DecodeOptions::default())?;
let exif = d.exif_metadata().ok().flatten();
let exif = exif.as_deref().map(|e| e.strip_prefix(b"Exif\0\0").unwrap_or(e));
Ok((w, h, exif.map(crate::exif::summarize).unwrap_or_default().orientation.unwrap_or(1)))
}
+39 -1
View File
@@ -30,12 +30,50 @@ fn try_all(bytes: &[u8]) {
if let Some(f) = sniff(bytes) {
let _ = decode_unguarded(bytes, f, &small_opts());
let _ = decode_unguarded(bytes, f, &DecodeOptions { max_size: Some((8, 8)), max_pixels: 1 << 22 });
let _ = read_header_unguarded(bytes, f);
}
let _ = decode(bytes, small_opts());
let _ = decode_thumbnail(bytes, 16);
let _ = read_header(bytes);
let _ = decode_thumbnail_with(bytes, &ThumbnailOptions { max_pixels: small_opts().max_pixels, ..ThumbnailOptions::new(16) });
let _ = lightcraft_codecs::icc::parse(bytes);
}
#[test]
fn webp_truncated_metadata_cannot_allocate_declared_chunk_size() {
let mut bytes = seeds()[5].clone();
let tag = bytes.windows(4).rposition(|w| w == b"XMP ").unwrap();
bytes[tag + 4..tag + 8].copy_from_slice(&1_946_157_060u32.to_le_bytes());
let decoded = decode(&bytes, small_opts()).unwrap();
assert_eq!((decoded.image.width, decoded.image.height), (24, 16));
assert!(decoded.xmp.is_none(), "invalid optional metadata is ignored");
}
#[test]
fn thumbnail_rejects_huge_gif_before_allocating_source_pixels() {
// A logical screen header without a global palette; previously this reached a
// multi-gigabyte output allocation before discovering that no frame follows.
let bytes = [
0x47, 0x49, 0x46, 0x38, 0x39, 0x61, 0x8f, 0xa4, 0xb0, 0x3f, 0x57, 0x14, 0xe8, 0x8a, 0xf8, 0xe0, 0xc7, 0x41, 0x9d, 0x0c, 0x10, 0xde, 0x54,
0x24, 0xcb, 0xed, 0xff, 0x4a, 0xe7, 0xa0, 0x82, 0x10, 0xa0, 0xbc, 0xac, 0xe5, 0x33, 0x3f, 0xc9, 0x8d, 0x69, 0xf4, 0x24, 0xce, 0x99, 0x4f,
0x0d, 0x47, 0xa0, 0xb0, 0xf4, 0xa8, 0x59, 0xad, 0x29, 0xbd, 0x09, 0x48, 0x63, 0xea, 0x59, 0xb7, 0xb7, 0xf2, 0xa1, 0x90, 0xeb, 0x92, 0xc5,
0x9c, 0xde, 0xcc, 0x96, 0x67, 0xc4, 0xe5, 0x8c, 0x03, 0x50, 0xb2, 0x30, 0xf8, 0xf4, 0x69, 0x1d, 0xeb, 0xb9, 0xbd, 0x4e, 0xfb, 0xbd, 0xb6,
0x02,
];
let result = decode_thumbnail(&bytes, 16);
assert!(matches!(result, Err(Error::TooLarge(42_127, 16_304))), "{result:?}");
}
#[test]
fn thumbnail_respects_configured_source_pixel_limit() {
let image = Rgba8::from_fn(3, 2, |_, _| [40, 80, 120, 255]);
let bytes = encode_png(&EncodeImage::rgba8(&image), &EncodeMeta::default()).unwrap();
let mut opts = ThumbnailOptions::new(1);
opts.max_pixels = 5;
assert!(matches!(decode_thumbnail_with(&bytes, &opts), Err(Error::TooLarge(3, 2))));
opts.max_pixels = 6;
assert_eq!(decode_thumbnail_with(&bytes, &opts).unwrap().image.width, 1);
}
fn seeds() -> &'static Vec<Vec<u8>> {
static S: OnceLock<Vec<Vec<u8>>> = OnceLock::new();
S.get_or_init(|| {
+366
View File
@@ -0,0 +1,366 @@
//! `read_header` (import probes) agrees with a decode on dimensions and orientation, refuses what a
//! decode refuses (truncated files included), and returns errors — never panics — on hostile input.
use lightcraft_codecs::exif::minimal_exif;
use lightcraft_codecs::*;
use lightcraft_raster::Rgba8;
fn pixels(w: usize, h: usize) -> Rgba8 {
Rgba8::from_fn(w, h, |x, y| [(x * 37 % 256) as u8, (y * 53 % 256) as u8, ((x ^ y) * 9 % 256) as u8, (128 + x % 100) as u8])
}
fn meta(exif: &Option<Vec<u8>>) -> EncodeMeta<'_> {
EncodeMeta { exif: exif.as_deref(), xmp: Some("<x/>"), ..Default::default() }
}
/// A TIFF with an Orientation tag (`encode_tiff` writes none), via the `tiff` crate's encoder.
fn tiff_oriented(img: &Rgba8, orientation: u16) -> Vec<u8> {
use tiff::encoder::{TiffEncoder, colortype};
let rgb: Vec<u8> = img.data.iter().flat_map(|p| [p[0], p[1], p[2]]).collect();
let mut cur = std::io::Cursor::new(Vec::new());
let mut enc = TiffEncoder::new(&mut cur).unwrap();
let mut im = enc.new_image::<colortype::RGB8>(img.width as u32, img.height as u32).unwrap();
im.encoder().write_tag(tiff::tags::Tag::Orientation, orientation).unwrap();
im.write_data(&rgb).unwrap();
cur.into_inner()
}
/// A minimal 8-bit RGB PSD (raw or PackBits-literal merged data) with an EXIF resource.
fn psd(w: usize, h: usize, orientation: Option<u16>, rle: bool) -> Vec<u8> {
let mut b = Vec::new();
b.extend_from_slice(b"8BPS");
b.extend_from_slice(&1u16.to_be_bytes());
b.extend_from_slice(&[0; 6]);
b.extend_from_slice(&3u16.to_be_bytes());
b.extend_from_slice(&(h as u32).to_be_bytes());
b.extend_from_slice(&(w as u32).to_be_bytes());
b.extend_from_slice(&8u16.to_be_bytes());
b.extend_from_slice(&3u16.to_be_bytes());
b.extend_from_slice(&0u32.to_be_bytes()); // colour mode data
let mut res = Vec::new();
if let Some(o) = orientation {
let exif = minimal_exif(o);
res.extend_from_slice(b"8BIM");
res.extend_from_slice(&1058u16.to_be_bytes());
res.extend_from_slice(&[0, 0]); // empty pascal name, padded
res.extend_from_slice(&(exif.len() as u32).to_be_bytes());
res.extend_from_slice(&exif);
if exif.len() % 2 == 1 {
res.push(0);
}
}
b.extend_from_slice(&(res.len() as u32).to_be_bytes());
b.extend_from_slice(&res);
b.extend_from_slice(&0u32.to_be_bytes()); // layer and mask info
b.extend_from_slice(&(rle as u16).to_be_bytes());
let plane = |c: usize| (0..w * h).map(move |i| ((i * (c + 3) * 11) % 256) as u8);
if rle {
// every row as literal runs of at most 128 bytes (PackBits' longest)
let row_len = w + w.div_ceil(128);
for _ in 0..3 * h {
b.extend_from_slice(&(row_len as u16).to_be_bytes());
}
for c in 0..3 {
let p: Vec<u8> = plane(c).collect();
for row in p.chunks(w) {
for run in row.chunks(128) {
b.push(run.len() as u8 - 1);
b.extend_from_slice(run);
}
}
}
} else {
for c in 0..3 {
b.extend(plane(c));
}
}
b
}
/// Generated files of every format with a header reader, plus a fallback one (BMP via `image`).
fn fixtures(w: usize, h: usize) -> Vec<(String, Vec<u8>)> {
let img = pixels(w, h);
let e = EncodeImage::rgba8(&img);
let rgb: Vec<u8> = img.data.iter().flat_map(|p| [p[0], p[1], p[2]]).collect();
let rgb_e = EncodeImage::new(w as u32, h as u32, 3, Samples::U8(&rgb));
let u16s: Vec<u16> = (0..w * h * 3).map(|i| (i * 331) as u16).collect();
let mut out = Vec::new();
for o in [None, Some(1), Some(6), Some(8)] {
let exif = o.map(minimal_exif);
let tag = format!("o{}", o.unwrap_or(0));
out.push((format!("jpeg 420 {tag}"), encode_jpeg(&rgb_e, 85, ChromaSubsampling::S420, &meta(&exif)).unwrap()));
out.push((format!("jpeg 444 {tag}"), encode_jpeg(&rgb_e, 85, ChromaSubsampling::S444, &meta(&exif)).unwrap()));
out.push((format!("png rgba {tag}"), encode_png(&e, &meta(&exif)).unwrap()));
out.push((format!("png 16-bit {tag}"), encode_png(&EncodeImage::new(w as u32, h as u32, 3, Samples::U16(&u16s)), &meta(&exif)).unwrap()));
out.push((format!("webp {tag}"), encode_webp_lossless(&e, &meta(&exif)).unwrap()));
out.push((format!("psd {tag}"), psd(w, h, o, false)));
out.push((format!("psd rle {tag}"), psd(w, h, o, true)));
out.push((format!("tiff {tag}"), tiff_oriented(&img, o.unwrap_or(1))));
}
out.push(("tiff lzw".into(), encode_tiff(&e, TiffCompression::Lzw, &EncodeMeta::default()).unwrap()));
out.push(("tiff packbits".into(), encode_tiff(&e, TiffCompression::PackBits, &EncodeMeta::default()).unwrap()));
let mut bmp = Vec::new();
image::codecs::bmp::BmpEncoder::new(&mut bmp).encode(&rgb, w as u32, h as u32, image::ExtendedColorType::Rgb8).unwrap();
out.push(("bmp".into(), bmp));
out
}
/// What the import probe learnt before it read headers: a small decode.
fn decoded(bytes: &[u8]) -> Result<Header> {
let d = decode(bytes, DecodeOptions::fit(64, 64))?;
Ok(Header { format: d.format, width: d.source_width, height: d.source_height, orientation: d.orientation })
}
#[test]
fn header_matches_decode() {
for (w, h) in [(160, 96), (96, 160), (7, 5)] {
for (name, bytes) in fixtures(w, h) {
let header = read_header(&bytes).unwrap_or_else(|e| panic!("{name} {w}×{h}: {e}"));
assert_eq!(header, decoded(&bytes).unwrap(), "{name} {w}×{h}");
assert_eq!((header.width, header.height), (w as u32, h as u32), "{name}");
}
}
let o = |name: &str| fixtures(16, 8).into_iter().find(|f| f.0 == name).map(|f| read_header(&f.1).unwrap().orientation).unwrap();
assert_eq!((o("jpeg 420 o6"), o("png rgba o8"), o("webp o6"), o("psd o8"), o("tiff o6"), o("jpeg 444 o0")), (6, 8, 6, 8, 6, 1));
}
/// Whenever the decode the probe used to run accepts a file (here: every cut of every fixture),
/// the header agrees with it: reading headers never refuses a file that decodes. The reverse
/// is checked for truncation, which the header readers detect.
#[test]
fn truncated_files_agree_with_decode() {
// 160 px wide: large JPEGs take the DCT-scaled (strict) decode path the probe took for photos
for (name, bytes) in fixtures(160, 96) {
let n = bytes.len();
let mut cuts: Vec<usize> = (0..100).map(|i| n * i / 100).collect();
cuts.extend(n.saturating_sub(64)..n);
for cut in cuts {
let part = &bytes[..cut];
if let Ok(d) = decoded(part) {
assert_eq!(read_header(part).ok(), Some(d), "{name} cut at {cut}/{n}");
}
}
for cut in [n / 4, n / 2, n * 9 / 10] {
assert!(read_header(&bytes[..cut]).is_err(), "{name}: cut at {cut}/{n} accepted");
}
}
}
#[test]
fn truncation_details() {
let f = fixtures(160, 96);
let get = |name: &str| f.iter().find(|x| x.0 == name).map(|x| x.1.clone()).unwrap();
let jpeg = get("jpeg 420 o6");
assert!(read_header(&jpeg[..jpeg.len() - 2]).is_err(), "a JPEG without its EOI is truncated");
let mut trailing = jpeg.clone();
trailing.extend_from_slice(&[0; 100]);
assert_eq!(read_header(&trailing).unwrap().orientation, 6, "bytes after the EOI are fine");
let png = get("png rgba o6");
assert!(read_header(&png[..png.len() - 2]).is_ok(), "a damaged IEND is ignored, as by a decode");
assert!(read_header(&png[..png.len() - 12]).is_err(), "nothing after the image data: truncated");
}
/// eXIf after the image data counts, as it does for a decode (which reads the whole file).
#[test]
fn png_exif_after_idat() {
let img = pixels(9, 4);
let plain = encode_png(&EncodeImage::rgba8(&img), &EncodeMeta::default()).unwrap();
let exif = minimal_exif(6);
let mut chunk = (exif.len() as u32).to_be_bytes().to_vec();
chunk.extend_from_slice(b"eXIf");
chunk.extend_from_slice(&exif);
chunk.extend_from_slice(&[0; 4]); // CRC: not checked by either reader
let iend = plain.len() - 12;
let mut b = plain[..iend].to_vec();
b.extend_from_slice(&chunk);
b.extend_from_slice(&plain[iend..]);
assert_eq!(read_header(&b).unwrap().orientation, 6);
}
fn jpeg_segments(sof: u8, w: u16, h: u16, comps: u8) -> Vec<u8> {
let mut b = vec![0xFF, 0xD8, 0xFF, sof, 0, 8 + 3 * comps as u16 as u8, 8];
b.extend_from_slice(&h.to_be_bytes());
b.extend_from_slice(&w.to_be_bytes());
b.push(comps);
for c in 0..comps {
b.extend_from_slice(&[c + 1, 0x11, 0]);
}
b.extend_from_slice(&[0xFF, 0xDA, 0, 8, 1, 1, 0, 0, 63, 0]);
b.extend_from_slice(&[0x12, 0xFF, 0x00, 0x34, 0xFF, 0xD0, 0x56]);
b.extend_from_slice(&[0xFF, 0xD9]);
b
}
fn png_ihdr(w: u32, h: u32, depth: u8, color: u8) -> Vec<u8> {
let mut b = b"\x89PNG\r\n\x1a\n\0\0\0\x0dIHDR".to_vec();
b.extend_from_slice(&w.to_be_bytes());
b.extend_from_slice(&h.to_be_bytes());
b.extend_from_slice(&[depth, color, 0, 0, 0, 0, 0, 0, 0]);
b
}
fn png_chunk(b: &mut Vec<u8>, kind: &[u8; 4], len: u32, data: &[u8]) {
b.extend_from_slice(&len.to_be_bytes());
b.extend_from_slice(kind);
b.extend_from_slice(data);
b.extend_from_slice(&[0; 4]);
}
#[test]
fn hostile_headers_are_errors() {
// a well-formed synthetic JPEG header is read
assert_eq!(read_header(&jpeg_segments(0xC0, 33, 21, 3)).unwrap(), Header { format: Format::Jpeg, width: 33, height: 21, orientation: 1 });
let jpeg_bad = [
jpeg_segments(0xC0, 0, 21, 3), // zero width
jpeg_segments(0xC0, 33, 0, 3), // zero height (DNL)
jpeg_segments(0xC0, 65535, 65535, 3), // 4.3 Gpx
jpeg_segments(0xC9, 33, 21, 3), // arithmetic coding
jpeg_segments(0xC5, 33, 21, 3), // hierarchical
jpeg_segments(0xC0, 33, 21, 2), // two components
jpeg_segments(0xC0, 33, 21, 3)[..30].to_vec(),
vec![0xFF, 0xD8, 0xFF, 0xE1, 0, 0], // segment length 0
vec![0xFF, 0xD8, 0xFF, 0xE1, 0xFF, 0xFF], // segment past the end
vec![0xFF, 0xD8, 0xFF, 0xD9],
];
for (i, b) in jpeg_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "JPEG case {i}");
}
let mut no_sos = jpeg_segments(0xC0, 33, 21, 3);
no_sos.truncate(2 + 2 + 17);
no_sos.extend_from_slice(&[0xFF, 0xD9]);
assert!(read_header(&no_sos).is_err(), "JPEG without a scan");
let mut ok = png_ihdr(5, 3, 8, 2);
png_chunk(&mut ok, b"IDAT", 1, &[0]);
png_chunk(&mut ok, b"IEND", 0, &[]);
assert_eq!(read_header(&ok).unwrap(), Header { format: Format::Png, width: 5, height: 3, orientation: 1 });
let png_case = |ihdr: Vec<u8>, chunks: &[(&[u8; 4], u32, &[u8])]| {
let mut b = ihdr;
for (k, l, d) in chunks {
png_chunk(&mut b, k, *l, d);
}
b
};
let idat: (&[u8; 4], u32, &[u8]) = (b"IDAT", 1, &[0]);
let iend: (&[u8; 4], u32, &[u8]) = (b"IEND", 0, &[]);
let png_bad = [
png_case(png_ihdr(0, 3, 8, 2), &[idat, iend]),
png_case(png_ihdr(5, 0, 8, 2), &[idat, iend]),
png_case(png_ihdr(0x8000_0000, 1, 8, 2), &[idat, iend]),
png_case(png_ihdr(65536, 65536, 8, 2), &[idat, iend]),
png_case(png_ihdr(5, 3, 4, 2), &[idat, iend]), // RGB at 4 bits
png_case(png_ihdr(5, 3, 8, 5), &[idat, iend]), // no colour type 5
png_case(png_ihdr(5, 3, 8, 3), &[idat, iend]), // indexed without PLTE
png_case(png_ihdr(5, 3, 8, 2), &[iend]), // no IDAT
png_case(png_ihdr(5, 3, 8, 2), &[idat]), // nothing after the image data
png_case(png_ihdr(5, 3, 8, 2), &[(b"IDAT", 0xFFFF_FFFF, &[0]), iend]),
png_case(png_ihdr(5, 3, 8, 2), &[(b"tEXt", 0x7FFF_FFFF, &[0]), idat, iend]),
png_ihdr(5, 3, 8, 2)[..20].to_vec(),
b"\x89PNG\r\n\x1a\n\0\0\0\x0dIHDX\0\0\0\x05\0\0\0\x03\x08\x02\0\0\0".to_vec(),
];
for (i, b) in png_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "PNG case {i}");
}
let tiff = encode_tiff(&EncodeImage::rgba8(&pixels(6, 4)), TiffCompression::None, &EncodeMeta::default()).unwrap();
let ifd = u32::from_le_bytes([tiff[4], tiff[5], tiff[6], tiff[7]]) as usize;
let mut tiff_bad = vec![b"II*\0\xff\xff\xff\x7f".to_vec(), b"II*\0\x08\0\0\0".to_vec(), b"II*\0\x04\0\0\0".to_vec()];
// cut short after the IFD
assert!(ifd < tiff.len() / 2, "the IFD comes before the image data");
tiff_bad.push(tiff[..tiff.len() - 10].to_vec());
for (i, b) in tiff_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "TIFF case {i}");
}
// a strip offset pointing past the end of the file
let t = |b: &[u8], p: usize| u16::from_le_bytes([b[p], b[p + 1]]);
let n = t(&tiff, ifd) as usize;
let mut far = tiff.clone();
for e in 0..n {
let p = ifd + 2 + e * 12;
if t(&far, p) == 273 && t(&far, p + 2) == 4 && u32::from_le_bytes([far[p + 4], far[p + 5], far[p + 6], far[p + 7]]) == 1 {
far[p + 8..p + 12].copy_from_slice(&0x7FFF_0000u32.to_le_bytes());
}
}
assert_ne!(far, tiff, "the strip offset was found");
assert!(read_header(&far).is_err(), "strip past the end");
let webp_bad = [
b"RIFF\0\0\0\0WEBP".to_vec(),
b"RIFF\0\0\0\0WEBPVP8L\xff\xff\xff\xff\x2f".to_vec(),
b"RIFF\0\0\0\0WEBPXXXX\xff\xff\xff\xff".to_vec(),
b"RIFF\0\0\0\0WEBPVP8 \x04\0\0\0\0\0\0\0".to_vec(),
];
for (i, b) in webp_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "WebP case {i}");
}
let good = psd(4, 3, Some(6), true);
assert_eq!(read_header(&good).unwrap().orientation, 6);
let set = |pos: usize, v: &[u8]| {
let mut b = good.clone();
b[pos..pos + v.len()].copy_from_slice(v);
b
};
let res_len = u32::from_be_bytes([good[30], good[31], good[32], good[33]]) as usize;
let compression = 34 + res_len + 4;
let mut psd_bad = vec![
set(12, &0u16.to_be_bytes()), // no channels
set(12, &57u16.to_be_bytes()), // too many
set(12, &2u16.to_be_bytes()), // too few for RGB
set(14, &0x7FFF_FFFFu32.to_be_bytes()), // absurd height
set(18, &0u32.to_be_bytes()), // zero width
set(22, &1u16.to_be_bytes()), // 1-bit
set(22, &7u16.to_be_bytes()), // bad depth
set(24, &9u16.to_be_bytes()), // Lab
set(24, &77u16.to_be_bytes()), // unknown mode
set(24, &2u16.to_be_bytes()), // indexed without a palette
set(compression, &2u16.to_be_bytes()), // ZIP
set(26, &0xFFFF_FFFFu32.to_be_bytes()), // colour mode data past the end
set(30, &0xFFFF_FFF0u32.to_be_bytes()), // resources past the end
];
// a 1 × 2^30 RLE image declared in a tiny file: refused without reading 2^30 row counts
let mut tall = good.clone();
tall[14..18].copy_from_slice(&(1u32 << 30).to_be_bytes());
tall[18..22].copy_from_slice(&1u32.to_be_bytes());
psd_bad.push(tall);
psd_bad.push(good[..good.len() - 1].to_vec());
for (i, b) in psd_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "PSD case {i}");
}
for b in [&b""[..], b"\xFF\xD8\xFF", b"\x89PNG\r\n\x1a\n", b"II*\0", b"MM\0*", b"8BPS\0\x01", b"GIF89a", b"not an image at all"] {
assert!(read_header(b).is_err(), "{b:?}");
}
}
/// The same with a few random bytes changed: whenever the decode accepts the damaged file, the
/// header agrees with it (deterministic seed).
#[test]
fn mutated_files_agree_with_decode() {
let mut s: u64 = 0x1234_5678_9abc_def1;
let mut rnd = || {
s ^= s << 13;
s ^= s >> 7;
s ^= s << 17;
s
};
let (mut decodable, mut disagreements) = (0, Vec::new());
for (name, bytes) in fixtures(160, 96) {
for _ in 0..300 {
let mut b = bytes.clone();
for _ in 0..1 + rnd() % 3 {
let i = rnd() as usize % b.len();
b[i] = rnd() as u8;
}
if let Ok(d) = decoded(&b) {
decodable += 1;
let h = read_header(&b);
if h.as_ref().ok() != Some(&d) {
disagreements.push(format!("{name}: decode {d:?}, header {h:?}"));
}
}
}
}
assert!(decodable > 1000, "{decodable}");
assert!(disagreements.is_empty(), "{disagreements:#?}");
}
+75 -1
View File
@@ -245,6 +245,80 @@ fn tiff_cmyk_and_orientation() {
assert!(red[0] > 0.99 && red[1] < 0.01 && red[2] < 0.01, "{red:?}");
}
/// A one-strip, uncompressed, single-sample TIFF (little-endian) whose samples are `bits` wide and
/// already packed into `strip`, one padded row after another.
fn packed_tiff(width: u16, height: u16, bits: u16, photometric: u16, strip: &[u8]) -> Vec<u8> {
const STRIP_AT: u32 = 8 + 2 + 9 * 12 + 4;
let short = |tag: u16, v: u16| [&tag.to_le_bytes()[..], &3u16.to_le_bytes(), &1u32.to_le_bytes(), &v.to_le_bytes(), &[0, 0]].concat();
let long = |tag: u16, v: u32| [&tag.to_le_bytes()[..], &4u16.to_le_bytes(), &1u32.to_le_bytes(), &v.to_le_bytes()].concat();
let mut b = b"II*\0".to_vec();
b.extend(8u32.to_le_bytes());
b.extend(9u16.to_le_bytes());
b.extend(short(256, width));
b.extend(short(257, height));
b.extend(short(258, bits));
b.extend(short(259, 1));
b.extend(short(262, photometric));
b.extend(long(273, STRIP_AT));
b.extend(short(277, 1));
b.extend(short(278, height));
b.extend(long(279, strip.len() as u32));
b.extend(0u32.to_le_bytes());
assert_eq!(b.len(), STRIP_AT as usize);
b.extend_from_slice(strip);
b
}
/// WhiteIsZero (photometric 0) gray: the lowest sample is white. The `tiff` crate already inverts these
/// samples, and a second inversion here used to show every WhiteIsZero scan as a negative.
#[test]
fn tiff_white_is_zero_is_inverted_once() {
// Two pixels: the lowest sample, then the highest, at 8 and 16 bits.
for (bits, strip) in [(8, vec![0u8, 255]), (16, [0u16.to_le_bytes(), 65535u16.to_le_bytes()].concat())] {
for (photometric, first) in [(0, 1.0), (1, 0.0)] {
let d = decode(&packed_tiff(2, 1, bits, photometric, &strip), DecodeOptions::default()).unwrap();
let (a, b) = (d.image.get(0, 0)[1], d.image.get(1, 0)[1]);
assert!((a - first).abs() < 1e-6 && (b - (1.0 - first)).abs() < 1e-6, "{bits}-bit, photometric {photometric}: {a} {b}");
}
}
}
/// 1-, 2- and 4-bit samples (bilevel scans, fax images, low-depth gray) arrive packed eight, four or
/// two to a byte with each row padded to a byte; they used to be rejected as "sample buffer too short".
#[test]
fn tiff_samples_narrower_than_a_byte() {
let gray = |d: &Decoded, x: usize, y: usize| d.image.get(x, y)[1];
// 10 × 2 bilevel: rows are 2 bytes, the last 6 bits of each are padding (set, to prove they are ignored).
// Row 0 alternates set/clear bits; row 1 is clear but for its last pixel.
let rows = [0b1010_1010, 0b1011_1111, 0b0000_0000, 0b0111_1111];
// What a set bit decodes to: white for BlackIsZero (1), black for WhiteIsZero (0).
for (photometric, set_bit) in [(1, 1.0), (0, 0.0)] {
let d = decode(&packed_tiff(10, 2, 1, photometric, &rows), DecodeOptions::default()).unwrap();
assert_eq!((d.image.width, d.image.height, d.bit_depth), (10, 2, 1));
assert!(d.grayscale);
let clear_bit = 1.0 - set_bit;
for x in 0..10 {
let want = (if x % 2 == 0 { set_bit } else { clear_bit }, if x == 9 { set_bit } else { clear_bit });
assert!((gray(&d, x, 0) - want.0).abs() < 1e-6, "row 0, x {x}, photometric {photometric}");
assert!((gray(&d, x, 1) - want.1).abs() < 1e-6, "row 1, x {x}, photometric {photometric}");
}
}
// 2-bit: 0, 1, 2, 3 → 0, 85, 170, 255. 4-bit: 0, 15, 8 → 0, 255, 136 (the row's last nibble is padding).
let d = decode(&packed_tiff(4, 1, 2, 1, &[0b00_01_10_11]), DecodeOptions::default()).unwrap();
for (x, v) in [0, 85, 170, 255].into_iter().enumerate() {
assert!((gray(&d, x, 0) - srgb_to_linear(v as f32 / 255.0)).abs() < 1e-6, "2-bit x {x}");
}
let d = decode(&packed_tiff(3, 1, 4, 1, &[0x0F, 0x8F]), DecodeOptions::default()).unwrap();
for (x, v) in [0, 255, 136].into_iter().enumerate() {
assert!((gray(&d, x, 0) - srgb_to_linear(v as f32 / 255.0)).abs() < 1e-6, "4-bit x {x}");
}
// A strip shorter than its rows is an error, never a panic.
assert!(decode(&packed_tiff(10, 2, 1, 1, &rows[..3]), DecodeOptions::default()).is_err());
}
#[test]
fn webp_lossless_exact() {
let img = gradient(31, 19);
@@ -461,7 +535,7 @@ fn thumbnail_accepts_smaller_preview_when_asked() {
encode_jpeg(&EncodeImage::rgba8(&main), 80, ChromaSubsampling::S420, &EncodeMeta { exif: Some(&exif), ..Default::default() }).unwrap();
let t = decode_thumbnail(&bytes, 256).unwrap();
assert_eq!(t.source, ThumbnailSource::Scaled);
let t = decode_thumbnail_with(&bytes, &ThumbnailOptions { max_edge: 256, min_embedded_edge: 160 }).unwrap();
let t = decode_thumbnail_with(&bytes, &ThumbnailOptions { min_embedded_edge: 160, ..ThumbnailOptions::new(256) }).unwrap();
assert_eq!(t.source, ThumbnailSource::ExifThumbnail);
assert_eq!((t.image.width, t.image.height), (160, 120));
}
+17 -6
View File
@@ -2,7 +2,9 @@
//!
//! The Planckian locus uses the Kim et al. (2002) cubic approximation (1667–25000 K); outside that
//! range it is extended linearly in mired space. Tint is a signed offset perpendicular to the locus in
//! CIE 1960 (u, v): **positive tint = magenta** (below the locus), `Duv = −tint / TINT_SCALE`.
//! CIE 1960 (u, v). Tint describes the **correction**: positive adds magenta, negative adds green.
//! A positive correction neutralises a green illuminant above the locus, so its source white uses
//! `Duv = tint / TINT_SCALE`. Bradford adaptation removes that source cast.
use crate::{Mat3, RgbSpace, Xy, bradford};
@@ -65,11 +67,12 @@ fn locus_normal(t: f64) -> (f64, f64) {
if ny < 0.0 { (-nx, -ny) } else { (nx, ny) }
}
/// Chromaticity of the white point for temperature `t` (K) and `tint`.
/// Source illuminant chromaticity for temperature `t` (K) and correction `tint`.
/// Positive tint means a green source white, whose correction adds magenta to the image.
pub fn temp_tint_to_xy(t: f64, tint: f64) -> Xy {
let (u, v) = locus_uv(t);
let (nu, nv) = locus_normal(t);
let duv = -tint / TINT_SCALE;
let duv = tint / TINT_SCALE;
uv_to_xy(u + nu * duv, v + nv * duv)
}
@@ -104,7 +107,7 @@ pub fn xy_to_temp_tint(p: Xy) -> (f64, f64) {
let (lu, lv) = locus_uv(t);
let (nu, nv) = locus_normal(t);
let duv = (u - lu) * nu + (v - lv) * nv;
(t, -duv * TINT_SCALE)
(t, duv * TINT_SCALE)
}
/// Matrix (linear RGB in `space` → same space) that white-balances a scene lit by `src` so that it
@@ -141,10 +144,18 @@ mod tests {
let warm = temp_tint_to_xy(3000.0, 0.0);
let cool = temp_tint_to_xy(9000.0, 0.0);
assert!(warm.x > cool.x);
// positive tint is magenta: lower v
// A positive magenta correction specifies a greener source illuminant: higher v.
let (_, v0) = xy_to_uv(temp_tint_to_xy(5000.0, 0.0));
let (_, v1) = xy_to_uv(temp_tint_to_xy(5000.0, 50.0));
assert!(v1 < v0);
assert!(v1 > v0);
}
/// Daylight lies above the Planckian locus, so D65 reads as a small positive tint, as
/// Lightroom's Daylight preset (5500 K, +10) has it (#188).
#[test]
fn daylight_has_a_positive_tint() {
let (_, tint) = xy_to_temp_tint(crate::D65);
assert!((5.0..15.0).contains(&tint), "{tint}");
}
#[test]
+8 -3
View File
@@ -9,6 +9,7 @@
//!
//! The pipeline's working space is **linear Rec.2020, D65** ([`WORKING`]).
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
pub mod cct;
pub mod perceptual;
@@ -125,14 +126,17 @@ pub const WORKING: RgbSpace = REC2020;
impl RgbSpace {
/// Matrix taking linear RGB in this space to XYZ (white Y = 1) relative to its own white.
/// Degenerate (collinear) primaries, which readers reject before building a space from a
/// file, leave the primaries unscaled instead of panicking.
pub fn to_xyz(&self) -> Mat3 {
let p = [self.r.to_xyz(), self.g.to_xyz(), self.b.to_xyz()];
let m = Mat3([[p[0][0], p[1][0], p[2][0]], [p[0][1], p[1][1], p[2][1]], [p[0][2], p[1][2], p[2][2]]]);
let s = m.inverse().expect("primaries are independent").apply(self.white.to_xyz());
let s = m.inverse().map_or([1.0; 3], |inv| inv.apply(self.white.to_xyz()));
m.mul(&Mat3::diag(s[0], s[1], s[2]))
}
/// Inverse of [`to_xyz`](Self::to_xyz); the identity for degenerate primaries.
pub fn from_xyz(&self) -> Mat3 {
self.to_xyz().inverse().expect("invertible")
self.to_xyz().inverse().unwrap_or(Mat3::IDENTITY)
}
/// Luminance weights (the Y row of `to_xyz`).
pub fn luma(&self) -> [f64; 3] {
@@ -154,7 +158,8 @@ pub fn bradford(src: Xy, dst: Xy) -> Mat3 {
}
let s = BRADFORD.apply(src.to_xyz());
let d = BRADFORD.apply(dst.to_xyz());
let inv = BRADFORD.inverse().expect("invertible");
// a constant, invertible matrix: the fallback (no adaptation) can't happen
let Some(inv) = BRADFORD.inverse() else { return Mat3::IDENTITY };
inv.mul(&Mat3::diag(d[0] / s[0], d[1] / s[1], d[2] / s[2])).mul(&BRADFORD)
}
+3 -2
View File
@@ -28,9 +28,10 @@ fn mats() -> &'static OkMats {
let to_lms = XYZ_TO_LMS.mul(&REC2020.to_xyz());
OkMats {
to_lms: to_lms.to_f32(),
from_lms: to_lms.inverse().expect("invertible").to_f32(),
// constant, invertible matrices (round trips tested below): the identity fallback can't happen
from_lms: to_lms.inverse().unwrap_or(Mat3::IDENTITY).to_f32(),
to_lab: LMS_TO_LAB.to_f32(),
from_lab: LMS_TO_LAB.inverse().expect("invertible").to_f32(),
from_lab: LMS_TO_LAB.inverse().unwrap_or(Mat3::IDENTITY).to_f32(),
}
})
}
+26
View File
@@ -0,0 +1,26 @@
[package]
name = "lightcraft-denoise-core"
version.workspace = true
edition.workspace = true
license.workspace = true
description = "LightCraft denoise model metadata, Bayer tiling and cache formats (L1; no inference backend)"
[lints]
workspace = true
[features]
default = []
rawnind-model = []
[dependencies]
lightcraft-raster = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
sha2 = { workspace = true }
thiserror = { workspace = true }
miniz_oxide = { workspace = true }
rayon = { workspace = true }
web-time = { workspace = true }
[dev-dependencies]
proptest = { workspace = true }
+7
View File
@@ -0,0 +1,7 @@
# lightcraft-denoise-core
L1 model metadata, bounded archive/cache formats and Bayer packing/tiling helpers.
No weights or inference backend. See [the user workflow](../../docs/denoise.md).
`lightcraft-denoise` re-exports these types for CPU/GPU users; the engine uses this
crate directly when inference is disabled. The default download catalog is empty.
Run `cargo test -p lightcraft-denoise-core`; optional catalog checks use `--features rawnind-model`.
+279
View File
@@ -0,0 +1,279 @@
//! Taking one file out of a `.zip` (a `.dtmodel` is one), the little of the format a model download needs: stored and
//! deflated entries, found by name, checked against their CRC-32 and a size limit, written next to where they are
//! going and moved into place. It is a stranger's file, so every length and offset in it is checked, a name is only
//! ever compared (never used as a path), and zip64, encrypted and multi-disk archives are refused.
use std::io::{Read, Seek, SeekFrom, Write};
use std::path::Path;
/// How far from the end the end-of-central-directory record can start (a 22-byte record and a 64 KiB comment).
const EOCD_SEARCH: u64 = 22 + 65535;
/// Largest central directory we read.
const MAX_DIRECTORY: u64 = 1 << 20;
#[derive(Debug, PartialEq, thiserror::Error)]
pub enum ArchiveError {
#[error("the archive could not be read: {0}")]
Io(String),
#[error("not a usable archive: {0}")]
Format(String),
#[error("the archive has no file called `{0}`")]
Missing(String),
}
impl From<std::io::Error> for ArchiveError {
fn from(e: std::io::Error) -> Self {
ArchiveError::Io(e.to_string())
}
}
fn format<T>(why: &str) -> Result<T, ArchiveError> {
Err(ArchiveError::Format(why.into()))
}
fn u16_at(b: &[u8], at: usize) -> Option<u16> {
Some(u16::from_le_bytes(b.get(at..at.checked_add(2)?)?.try_into().ok()?))
}
fn u32_at(b: &[u8], at: usize) -> Option<u32> {
Some(u32::from_le_bytes(b.get(at..at.checked_add(4)?)?.try_into().ok()?))
}
/// CRC-32 (IEEE) of `bytes`.
fn crc32(bytes: &[u8]) -> u32 {
let mut table = [0u32; 256];
for (i, t) in table.iter_mut().enumerate() {
let mut c = i as u32;
for _ in 0..8 {
c = if c & 1 != 0 { 0xedb8_8320 ^ (c >> 1) } else { c >> 1 };
}
*t = c;
}
!bytes.iter().fold(!0u32, |c, &b| table[((c ^ u32::from(b)) & 0xff) as usize] ^ (c >> 8))
}
struct Entry {
method: u16,
crc: u32,
compressed: u64,
size: u64,
local_header: u64,
}
/// Find the entry called `name` (exactly, or as the last part of a longer name: `folder/name`) in the central directory.
fn find(f: &mut (impl Read + Seek), name: &str) -> Result<Entry, ArchiveError> {
let len = f.seek(SeekFrom::End(0))?;
let tail_len = len.min(EOCD_SEARCH);
f.seek(SeekFrom::Start(len - tail_len))?;
let mut tail = vec![0u8; tail_len as usize];
f.read_exact(&mut tail)?;
let eocd = (0..tail.len().saturating_sub(21)).rev().find(|&i| u32_at(&tail, i) == Some(0x0605_4b50));
let Some(e) = eocd else { return format("this is not a zip file") };
let (entries, size, offset) = (u16_at(&tail, e + 10), u32_at(&tail, e + 12), u32_at(&tail, e + 16));
let (Some(entries), Some(size), Some(offset)) = (entries, size, offset) else { return format("the end of the archive is cut short") };
if entries == 0xffff || size == u32::MAX || offset == u32::MAX {
return format("zip64 archives are not supported");
}
if u64::from(size) > MAX_DIRECTORY || u64::from(offset) + u64::from(size) > len {
return format("the list of files is not where it says");
}
f.seek(SeekFrom::Start(u64::from(offset)))?;
let mut dir = vec![0u8; size as usize];
f.read_exact(&mut dir)?;
let mut at = 0usize;
for _ in 0..entries {
if u32_at(&dir, at) != Some(0x0201_4b50) {
return format("the list of files is damaged");
}
let get = |o: usize| at.checked_add(o);
let (flags, method, crc) = (u16_at(&dir, at + 8), u16_at(&dir, at + 10), u32_at(&dir, at + 16));
let (compressed, uncompressed) = (u32_at(&dir, at + 20), u32_at(&dir, at + 24));
let (n, x, c) = (u16_at(&dir, at + 28), u16_at(&dir, at + 30), u16_at(&dir, at + 32));
let local = u32_at(&dir, at + 42);
let (Some(flags), Some(method), Some(crc), Some(compressed), Some(uncompressed), Some(n), Some(x), Some(c), Some(local), Some(start)) =
(flags, method, crc, compressed, uncompressed, n, x, c, local, get(46))
else {
return format("the list of files is cut short");
};
let Some(entry_name) = dir.get(start..start + usize::from(n)) else { return format("a file name is cut short") };
let matches = entry_name == name.as_bytes() || entry_name.strip_suffix(name.as_bytes()).is_some_and(|p| p.ends_with(b"/"));
if matches {
if flags & 1 != 0 {
return format("encrypted files are not supported");
}
if compressed == u32::MAX || uncompressed == u32::MAX {
return format("zip64 files are not supported");
}
return Ok(Entry { method, crc, compressed: u64::from(compressed), size: u64::from(uncompressed), local_header: u64::from(local) });
}
at = start + usize::from(n) + usize::from(x) + usize::from(c);
}
Err(ArchiveError::Missing(name.into()))
}
/// Take the file called `name` out of the archive at `archive` and write it to `dest` (through a temporary file next
/// to it); `max_bytes` bounds its size. Returns the number of bytes written.
pub fn extract(archive: &Path, name: &str, dest: &Path, max_bytes: u64) -> Result<u64, ArchiveError> {
let mut f = std::io::BufReader::new(std::fs::File::open(archive)?);
let e = find(&mut f, name)?;
if e.size > max_bytes || e.compressed > max_bytes {
return format("the file in the archive is larger than allowed");
}
f.seek(SeekFrom::Start(e.local_header))?;
let mut head = [0u8; 30];
f.read_exact(&mut head)?;
if u32_at(&head, 0) != Some(0x0403_4b50) {
return format("the file's header is not where it says");
}
let skip = u64::from(u16_at(&head, 26).unwrap_or(0)) + u64::from(u16_at(&head, 28).unwrap_or(0));
f.seek_relative(i64::try_from(skip).map_err(|_| ArchiveError::Format("a header is too long".into()))?)?;
let len = usize::try_from(e.compressed)
.ok()
.filter(|&n| n <= 256 * 1024 * 1024)
.ok_or_else(|| ArchiveError::Format("the compressed model is too large".into()))?;
if e.size > 256 * 1024 * 1024 {
return format("the extracted model is too large");
}
let mut packed = Vec::new();
packed.try_reserve_exact(len).map_err(|_| ArchiveError::Format("not enough memory for the model archive".into()))?;
packed.resize(len, 0u8);
f.read_exact(&mut packed)?;
let data = match e.method {
0 => packed,
8 => miniz_oxide::inflate::decompress_to_vec_with_limit(&packed, max_bytes.min(256 * 1024 * 1024) as usize)
.map_err(|err| ArchiveError::Format(format!("{err:?}")))?,
m => return format(&format!("compression method {m} is not supported")),
};
if data.len() as u64 != e.size || crc32(&data) != e.crc {
return format("the file in the archive is damaged");
}
let tmp = dest.with_extension("part");
let written = (|| -> std::io::Result<()> {
let mut out = std::fs::File::create(&tmp)?;
out.write_all(&data)?;
out.flush()?;
drop(out);
std::fs::rename(&tmp, dest)
})();
if let Err(err) = written {
let _ = std::fs::remove_file(&tmp);
return Err(err.into());
}
Ok(data.len() as u64)
}
#[cfg(test)]
mod tests {
use super::*;
/// A zip with the given (name, bytes, deflate) entries, written by hand.
pub(crate) fn make_zip(files: &[(&str, &[u8], bool)]) -> Vec<u8> {
let mut out = Vec::new();
let mut dir = Vec::new();
for (name, data, deflate) in files {
let packed = if *deflate { miniz_oxide::deflate::compress_to_vec(data, 6) } else { data.to_vec() };
let offset = out.len() as u32;
let method: u16 = if *deflate { 8 } else { 0 };
out.extend_from_slice(&0x0403_4b50u32.to_le_bytes());
out.extend_from_slice(&[20, 0, 0, 0]);
out.extend_from_slice(&method.to_le_bytes());
out.extend_from_slice(&[0, 0, 0, 0]);
out.extend_from_slice(&crc32(data).to_le_bytes());
out.extend_from_slice(&(packed.len() as u32).to_le_bytes());
out.extend_from_slice(&(data.len() as u32).to_le_bytes());
out.extend_from_slice(&(name.len() as u16).to_le_bytes());
out.extend_from_slice(&[0, 0]);
out.extend_from_slice(name.as_bytes());
out.extend_from_slice(&packed);
dir.extend_from_slice(&0x0201_4b50u32.to_le_bytes());
dir.extend_from_slice(&[20, 0, 20, 0, 0, 0]);
dir.extend_from_slice(&method.to_le_bytes());
dir.extend_from_slice(&[0, 0, 0, 0]);
dir.extend_from_slice(&crc32(data).to_le_bytes());
dir.extend_from_slice(&(packed.len() as u32).to_le_bytes());
dir.extend_from_slice(&(data.len() as u32).to_le_bytes());
dir.extend_from_slice(&(name.len() as u16).to_le_bytes());
dir.extend_from_slice(&[0; 12]);
dir.extend_from_slice(&offset.to_le_bytes());
dir.extend_from_slice(name.as_bytes());
}
let cd_offset = out.len() as u32;
out.extend_from_slice(&dir);
out.extend_from_slice(&0x0605_4b50u32.to_le_bytes());
out.extend_from_slice(&[0, 0, 0, 0]);
out.extend_from_slice(&(files.len() as u16).to_le_bytes());
out.extend_from_slice(&(files.len() as u16).to_le_bytes());
out.extend_from_slice(&(dir.len() as u32).to_le_bytes());
out.extend_from_slice(&cd_offset.to_le_bytes());
out.extend_from_slice(&[0, 0]);
out
}
fn temp(name: &str) -> std::path::PathBuf {
std::env::temp_dir().join(format!("lc-denoise-zip-{name}-{}", std::process::id()))
}
#[test]
fn a_stored_and_a_deflated_file_come_out_whole() {
let big: Vec<u8> = (0..200_000u32).map(|i| (i * 7 % 251) as u8).collect();
let zip = make_zip(&[("m/config.json", b"{}", false), ("m/model.onnx", &big, true), ("m/other.onnx", b"zz", true)]);
let (a, out) = (temp("a.zip"), temp("a.onnx"));
std::fs::write(&a, &zip).unwrap();
assert_eq!(extract(&a, "model.onnx", &out, 1 << 20).unwrap(), big.len() as u64);
assert_eq!(std::fs::read(&out).unwrap(), big);
assert_eq!(extract(&a, "m/config.json", &out, 1 << 20).unwrap(), 2);
assert_eq!(std::fs::read(&out).unwrap(), b"{}");
// a name must match whole path parts: `del.onnx` is not `model.onnx`
assert!(matches!(extract(&a, "del.onnx", &out, 1 << 20), Err(ArchiveError::Missing(_))));
assert!(matches!(extract(&a, "nothing", &out, 1 << 20), Err(ArchiveError::Missing(_))));
// the size limit applies
assert!(matches!(extract(&a, "model.onnx", &out, 1000), Err(ArchiveError::Format(_))));
for p in [a, out] {
let _ = std::fs::remove_file(p);
}
}
/// Opt-in: `LC_DENOISE_DTMODEL=<rawdenoise-nind.dtmodel> cargo test -p lightcraft-denoise -- --ignored archive`
#[test]
#[ignore = "needs the real download: set LC_DENOISE_DTMODEL"]
fn the_real_download_gives_the_model_it_should() {
let Some(path) = std::env::var_os("LC_DENOISE_DTMODEL") else { return };
let known = crate::known::find("rawnind-bayer").unwrap();
let (d, entry) = (known.download.unwrap(), known.entry.unwrap());
assert_eq!(crate::hash::sha256_file(Path::new(&path)).unwrap(), d.sha256);
let out = temp("real.onnx");
let n = extract(Path::new(&path), entry, &out, crate::manifest::MAX_MODEL_BYTES).unwrap();
assert_eq!(Some(n), known.manifest.size_bytes);
assert_eq!(Some(crate::hash::sha256_file(&out).unwrap()), known.manifest.sha256);
let _ = std::fs::remove_file(out);
}
#[test]
fn damaged_and_hostile_archives_are_errors_not_panics() {
let zip = make_zip(&[("m/model.onnx", &vec![9u8; 5000], true)]);
let (a, out) = (temp("b.zip"), temp("b.onnx"));
// cut anywhere
for n in (0..zip.len()).step_by(3) {
std::fs::write(&a, &zip[..n]).unwrap();
assert!(extract(&a, "model.onnx", &out, 1 << 20).is_err(), "cut at {n}");
}
// one byte changed anywhere: an error, or (when it falls in an unused field) the right file, never a panic
for at in 0..zip.len() {
let mut bad = zip.clone();
bad[at] ^= 0x5a;
std::fs::write(&a, &bad).unwrap();
if extract(&a, "model.onnx", &out, 1 << 20).is_ok() {
assert_eq!(std::fs::read(&out).unwrap(), vec![9u8; 5000], "a changed byte at {at} gave another file");
}
}
// not a zip at all, and an empty file
for junk in [Vec::new(), b"PK".to_vec(), vec![0u8; 100_000]] {
std::fs::write(&a, junk).unwrap();
assert!(extract(&a, "model.onnx", &out, 1 << 20).is_err());
}
assert!(extract(&temp("missing.zip"), "model.onnx", &out, 1 << 20).is_err());
for p in [a, out] {
let _ = std::fs::remove_file(p);
}
}
}
+191
View File
@@ -0,0 +1,191 @@
//! Bayer geometry: which way a mosaic is laid out, and how a patch of it is packed into the four planes
//! (red, green on the red row, green on the blue row, blue) a denoise model takes.
//!
//! Every Bayer layout is the same 2 × 2 cell shifted by one pixel at most, so a layout is just where its
//! first red sample is. The cell grid starts one pixel before the picture when that red sample is in
//! column or row 1, and the picture is read with its edge mirrored (without repeating the edge sample, so
//! the colour of a mirrored sample is the colour that belongs at that place).
/// Where the first red sample of a Bayer mosaic sits, `(dx, dy)` in `{0, 1}²`: RGGB is `(0, 0)`, GRBG `(1, 0)`,
/// GBRG `(0, 1)`, BGGR `(1, 1)`.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Layout {
pub dx: usize,
pub dy: usize,
}
impl Layout {
/// The layout of a 2 × 2 colour pattern, row-major, colours 0 = red, 1 = green, 2 = blue; `None` when
/// it is not a Bayer cell (one red, one blue, the greens on the other diagonal).
pub fn from_cell(cell: [u8; 4]) -> Option<Layout> {
let at = |c: u8| cell.iter().position(|&v| v == c);
let red = at(0)?;
let blue = at(2)?;
let greens = cell.iter().filter(|&&v| v == 1).count();
// red and blue sit on one diagonal: indices 0+3 or 1+2
if greens != 2 || red + blue != 3 {
return None;
}
Some(Layout { dx: red % 2, dy: red / 2 })
}
/// The cell grid's origin: `-1` along an axis whose first red sample is at 1, so that cell 0 starts on a red column or row.
fn origin(self) -> (isize, isize) {
(isize::try_from(self.dx).unwrap_or(isize::MAX).saturating_neg(), isize::try_from(self.dy).unwrap_or(isize::MAX).saturating_neg())
}
/// Cells across and down covering a `width × height` mosaic.
pub fn cells(self, width: usize, height: usize) -> (usize, usize) {
let cells = |n: usize, phase: usize| n / 2 + (n % 2 + phase.min(1)).div_ceil(2);
(cells(width, self.dx), cells(height, self.dy))
}
/// The mosaic position of cell `(cx, cy)`'s top-left sample (may lie outside the picture).
pub fn position(self, cx: isize, cy: isize) -> (isize, isize) {
let (ox, oy) = self.origin();
(ox.saturating_add(cx.saturating_mul(2)), oy.saturating_add(cy.saturating_mul(2)))
}
}
/// `i` mirrored into `0..n` (ping-pong, edge not repeated), so `-1` is `1` and `n` is `n - 2`. `n == 0` gives 0.
pub fn reflect(i: isize, n: usize) -> usize {
if n <= 1 {
return 0;
}
if let Some(period) = isize::try_from(n).ok().and_then(|n| (n - 1).checked_mul(2)) {
let m = i.rem_euclid(period);
return (if m < n as isize { m } else { period - m }) as usize;
}
let period = 2 * (n as i128 - 1);
let m = (i as i128).rem_euclid(period);
// The reflected value is in 0..n, so this conversion always fits usize.
(if m < n as i128 { m } else { period - m }) as usize
}
/// Pack the `tile × tile` cells whose top-left cell is `(cx0, cy0)` into `out` as four planes `[R, G1, G2, B]`
/// (`4 · tile²` values, NCHW). G1 is the green beside the red sample, G2 the green under it. Samples outside
/// the picture are mirrored. Returns `false` (and writes nothing) when the sizes do not fit.
pub fn pack_tile(mosaic: &[f32], width: usize, height: usize, layout: Layout, cx0: isize, cy0: isize, tile: usize, out: &mut [f32]) -> bool {
let Some(plane) = tile.checked_mul(tile) else { return false };
let Some(values) = plane.checked_mul(4) else { return false };
if width == 0
|| height == 0
|| tile == 0
|| layout.dx > 1
|| layout.dy > 1
|| width.checked_mul(height) != Some(mosaic.len())
|| out.len() != values
{
return false;
}
let Ok(last) = isize::try_from(tile - 1) else { return false };
let (ox, oy) = layout.origin();
for (start, origin) in [(cx0, ox), (cy0, oy)] {
if start.checked_mul(2).and_then(|v| v.checked_add(origin)).is_none()
|| start.checked_add(last).and_then(|v| v.checked_mul(2)).and_then(|v| v.checked_add(origin)).and_then(|v| v.checked_add(1)).is_none()
{
return false;
}
}
// the four samples of a cell, as (dx, dy) from its top-left sample
const SUB: [(isize, isize); 4] = [(0, 0), (1, 0), (0, 1), (1, 1)];
for (p, &(sx, sy)) in SUB.iter().enumerate() {
let Some(dst) = out.get_mut(p * plane..(p + 1) * plane) else { return false };
for (j, row) in dst.chunks_exact_mut(tile).enumerate() {
let (_, y) = layout.position(cx0, cy0 + j as isize);
let ry = reflect(y + sy, height);
let Some(src) = mosaic.get(ry * width..(ry + 1) * width) else { return false };
for (i, v) in row.iter_mut().enumerate() {
let (x, _) = layout.position(cx0 + i as isize, cy0);
*v = src.get(reflect(x + sx, width)).copied().unwrap_or(0.0);
}
}
}
true
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn hostile_sizes_and_positions_never_panic_or_write_output() {
let layout = Layout::from_cell([0, 1, 1, 2]).unwrap();
let mut out = [7.0; 4];
for (w, h, tile, x) in [(usize::MAX, 2, 1, 0), (1, 1, usize::MAX, 0), (1, 1, 0, 0), (1, 1, 1, isize::MAX), (1, 1, 1, isize::MIN)] {
assert!(!pack_tile(&[0.5], w, h, layout, x, 0, tile, &mut out));
assert_eq!(out, [7.0; 4]);
}
for n in [1, 2, isize::MAX as usize, usize::MAX] {
for i in [isize::MIN, -1, 0, isize::MAX] {
assert!(reflect(i, n) < n);
}
}
assert_eq!(layout.cells(usize::MAX, usize::MAX), (usize::MAX / 2 + 1, usize::MAX / 2 + 1));
}
#[test]
fn layouts_from_cells() {
assert_eq!(Layout::from_cell([0, 1, 1, 2]), Some(Layout { dx: 0, dy: 0 }));
assert_eq!(Layout::from_cell([1, 0, 2, 1]), Some(Layout { dx: 1, dy: 0 }));
assert_eq!(Layout::from_cell([1, 2, 0, 1]), Some(Layout { dx: 0, dy: 1 }));
assert_eq!(Layout::from_cell([2, 1, 1, 0]), Some(Layout { dx: 1, dy: 1 }));
// not Bayer: two reds, a missing blue, greens on the wrong diagonal
assert_eq!(Layout::from_cell([0, 0, 1, 2]), None);
assert_eq!(Layout::from_cell([0, 1, 2, 1]), None);
assert_eq!(Layout::from_cell([1, 1, 1, 1]), None);
}
#[test]
fn reflection_keeps_the_colour_of_a_site() {
assert_eq!(reflect(-1, 10), 1);
assert_eq!(reflect(-2, 10), 2);
assert_eq!(reflect(10, 10), 8);
assert_eq!(reflect(11, 10), 7);
assert_eq!(reflect(25, 10), 7);
assert_eq!(reflect(5, 0), 0);
assert_eq!(reflect(5, 1), 0);
for i in -40..60 {
assert_eq!(reflect(i, 11) % 2, (i.rem_euclid(2)) as usize, "parity at {i}");
}
}
#[test]
fn packing_puts_each_colour_in_its_plane_for_every_layout() {
// a mosaic whose sample value says which colour it is: R = 10, G = 20, B = 30
for cell in [[0u8, 1, 1, 2], [1, 0, 2, 1], [1, 2, 0, 1], [2, 1, 1, 0]] {
let layout = Layout::from_cell(cell).unwrap();
let (w, h) = (10usize, 8usize);
let mosaic: Vec<f32> = (0..w * h).map(|i| [10.0, 20.0, 30.0][cell[(i / w % 2) * 2 + i % w % 2] as usize]).collect();
let (nx, ny) = layout.cells(w, h);
let tile = nx.max(ny);
let mut out = vec![0.0; 4 * tile * tile];
assert!(pack_tile(&mosaic, w, h, layout, 0, 0, tile, &mut out));
let plane = tile * tile;
for (p, want) in [10.0, 20.0, 20.0, 30.0].into_iter().enumerate() {
assert!(out[p * plane..(p + 1) * plane].iter().all(|&v| v == want), "layout {cell:?} plane {p}");
}
}
}
#[test]
fn g1_is_beside_the_red_sample_and_g2_under_it() {
// RGGB with a distinct value at every site
let (w, h) = (4, 4);
let mosaic: Vec<f32> = (0..16).map(|i| i as f32).collect();
let mut out = vec![0.0; 4 * 4];
assert!(pack_tile(&mosaic, w, h, Layout { dx: 0, dy: 0 }, 0, 0, 2, &mut out));
// cell (0,0): R = site (0,0)=0, G1 = site (1,0)=1, G2 = site (0,1)=4, B = site (1,1)=5
assert_eq!([out[0], out[4], out[8], out[12]], [0.0, 1.0, 4.0, 5.0]);
// cell (1,0): sites (2,0)=2, (3,0)=3, (2,1)=6, (3,1)=7
assert_eq!([out[1], out[5], out[9], out[13]], [2.0, 3.0, 6.0, 7.0]);
}
#[test]
fn packing_rejects_wrong_sizes() {
let mut out = vec![0.0; 16];
assert!(!pack_tile(&[0.0; 15], 4, 4, Layout { dx: 0, dy: 0 }, 0, 0, 2, &mut out));
assert!(!pack_tile(&[0.0; 16], 4, 4, Layout { dx: 0, dy: 0 }, 0, 0, 3, &mut out));
assert!(!pack_tile(&[], 0, 0, Layout { dx: 0, dy: 0 }, 0, 0, 2, &mut out));
}
}
+46
View File
@@ -0,0 +1,46 @@
//! SHA-256 of a model file, streamed (a model can be hundreds of megabytes).
use std::io::{self, Read};
use std::path::Path;
use sha2::{Digest, Sha256};
/// Lowercase hex SHA-256 of everything `r` yields.
pub fn sha256_reader<R: Read>(mut r: R) -> io::Result<String> {
let mut h = Sha256::new();
let mut buf = vec![0u8; 1 << 20];
loop {
let n = r.read(&mut buf)?;
if n == 0 {
break;
}
h.update(buf.get(..n).unwrap_or(&[]));
}
Ok(hex(&h.finalize()))
}
/// Lowercase hex SHA-256 of a file.
pub fn sha256_file(path: &Path) -> io::Result<String> {
sha256_reader(io::BufReader::new(std::fs::File::open(path)?))
}
/// Lowercase hex SHA-256 of `bytes`.
pub fn sha256_hex(bytes: &[u8]) -> String {
hex(&Sha256::digest(bytes))
}
fn hex(bytes: &[u8]) -> String {
bytes.iter().map(|b| format!("{b:02x}")).collect()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn known_digests() {
assert_eq!(sha256_hex(b""), "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855");
assert_eq!(sha256_hex(b"abc"), "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad");
assert_eq!(sha256_reader(&b"abc"[..]).unwrap(), sha256_hex(b"abc"));
}
}
+128
View File
@@ -0,0 +1,128 @@
//! The denoise models LightCraft knows about: what each one is, who may use it, and where to fetch it.
//!
//! None is part of LightCraft. A model with an address here can be downloaded when the user asks and has accepted
//! its terms; anything else is brought by the user as an `.onnx` file with a `denoise-model.json` beside it.
#[cfg(any(feature = "rawnind-model", test))]
use crate::licence::{Commercial, Licence};
use crate::manifest::DenoiserManifest;
#[cfg(any(feature = "rawnind-model", test))]
use crate::manifest::{Domain, Gain};
/// A known model, and how to get it.
pub struct Known {
pub manifest: DenoiserManifest,
/// What to fetch (an archive or the `.onnx` itself); `None` when the user must bring the file.
pub download: Option<Download>,
/// The `.onnx` inside the downloaded archive (`None`: the download is the `.onnx`).
pub entry: Option<&'static str>,
}
/// RawNIND's UtNet2 for Bayer sensors, as published for darktable (`rawdenoise-nind.dtmodel`, a zip with a Bayer and a
/// linear model). Its weights are GPL-3.0; the training pictures are CC BY 4.0 and CC0.
#[cfg(any(feature = "rawnind-model", test))]
fn rawnind_bayer() -> Known {
let manifest = DenoiserManifest {
id: "rawnind-bayer".into(),
name: "RawNIND UtNet2 (Bayer)".into(),
version: "1.0".into(),
licence: Licence {
name: "GPL-3.0".into(),
commercial: Commercial::Yes,
url: Some("https://www.gnu.org/licenses/gpl-3.0.html".into()),
notice: "The weights are published under the GNU General Public License v3. LightCraft does not include them: they \
are downloaded to your computer at your request and used by it only. If you pass the model file on, the \
GPL applies to that copy. Commercial use is subject to the GPL terms, including applicable source, licence and notice obligations on redistribution."
.into(),
},
source: Some("https://github.com/darktable-org/darktable-ai/tree/master/models/rawdenoise-nind".into()),
sha256: Some("da27509dab6a2915da67e988acd86cf71f9d5bbc8d1aa0ed32933578a887b901".into()),
size_bytes: Some(31_056_425),
provenance: "Trained on RawNIND, real photographs of the same scenes taken noisy and clean (images CC BY 4.0 and CC0, \
published on Wikimedia Commons and by the University of Louvain). Paper: arXiv 2501.08924."
.into(),
domain: Domain::BayerToRgb,
tile: 512,
overlap: 64,
gain: Gain::MatchMean { nominal: 1.0e6, max_deviation: 0.05 },
};
let download = Download {
id: manifest.id.clone(),
url: "https://github.com/darktable-org/darktable-ai/releases/download/release-5.6.0/rawdenoise-nind.dtmodel".into(),
file_name: "rawdenoise-nind.dtmodel".into(),
size_bytes: 57_700_134,
sha256: "d71b5f1e727c85a359e6f74dca9e2016c9d8fc3e2f7ac3e9b347d80ceca969af".into(),
};
Known { manifest, download: Some(download), entry: Some("model_bayer.onnx") }
}
/// Every model LightCraft knows.
pub fn all() -> Vec<Known> {
#[cfg(feature = "rawnind-model")]
{
vec![rawnind_bayer()]
}
#[cfg(not(feature = "rawnind-model"))]
{
Vec::new()
}
}
/// The known model with this id.
pub fn find(id: &str) -> Option<Known> {
all().into_iter().find(|k| k.manifest.id == id)
}
/// The known model whose `.onnx` has this SHA-256.
pub fn by_sha256(sha256: &str) -> Option<DenoiserManifest> {
all().into_iter().map(|k| k.manifest).find(|m| m.sha256.as_deref() == Some(sha256))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::manifest::validate;
#[test]
fn known_models_are_valid_unique_and_downloaded_over_https() {
let all = all();
assert_eq!(all.is_empty(), !cfg!(feature = "rawnind-model"));
validate(&rawnind_bayer().manifest).unwrap();
for (i, k) in all.iter().enumerate() {
validate(&k.manifest).unwrap();
assert!(all.iter().skip(i + 1).all(|o| o.manifest.id != k.manifest.id), "unique ids");
assert!(!k.manifest.licence.name.is_empty() && !k.manifest.licence.notice.is_empty(), "terms are shown before use");
if let Some(d) = &k.download {
assert_eq!(d.id, k.manifest.id);
assert!(d.url.starts_with("https://") && d.sha256.len() == 64 && d.size_bytes > 0);
// a name that cannot leave the staging folder
assert!(!d.file_name.contains(['/', '\\']) && !d.file_name.starts_with('.'));
}
// a model that may not be used commercially is never offered for download
if k.manifest.licence.commercial == Commercial::No {
assert!(k.download.is_none(), "{}", k.manifest.id);
}
}
assert_eq!(find("rawnind-bayer").is_some(), cfg!(feature = "rawnind-model"));
assert!(find("nothing").is_none());
assert_eq!(
by_sha256("da27509dab6a2915da67e988acd86cf71f9d5bbc8d1aa0ed32933578a887b901").map(|m| m.id),
if cfg!(feature = "rawnind-model") { Some("rawnind-bayer".into()) } else { None }
);
}
}
#[derive(Clone, Debug, PartialEq)]
pub struct Download {
pub id: String,
pub url: String,
/// The name the file is kept under while it waits for the user to accept its terms.
pub file_name: String,
pub size_bytes: u64,
pub sha256: String,
}
pub fn host(url: &str) -> &str {
url.strip_prefix("https://").and_then(|r| r.split('/').next()).unwrap_or("")
}
+16
View File
@@ -0,0 +1,16 @@
//! Model metadata, bounded archive/cache formats and Bayer tiling, independent of inference.
//! Used by featureless engine and web builds; no neural-network or matrix backend is linked.
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
pub mod archive;
pub mod bayer;
pub mod hash;
pub mod known;
pub mod licence;
pub mod manifest;
pub mod product;
pub mod run;
pub mod tiles;
pub use manifest::{DenoiserManifest, Domain, Gain, ManifestError};
pub use run::{Control, Error, Params, TileRunner, denoise_bayer};
+34
View File
@@ -0,0 +1,34 @@
use serde::{Deserialize, Serialize};
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum Commercial {
Yes,
/// Research or personal use only. Never bundled or redistributed by LightCraft.
No,
/// The licence or the training data is unclear: the user is told so before installing.
Unknown,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct Licence {
/// SPDX id or a short name ("MIT", "Apache-2.0", "InsightFace non-commercial research").
pub name: String,
pub commercial: Commercial,
pub url: Option<String>,
/// Plain-language terms shown before the model is enabled.
pub notice: String,
}
impl Default for Licence {
fn default() -> Self {
Self { name: String::new(), commercial: Commercial::Unknown, url: None, notice: String::new() }
}
}
pub fn valid_id(id: &str) -> bool {
let mut bytes = id.bytes();
let first_ok = bytes.next().is_some_and(|b| b.is_ascii_lowercase() || b.is_ascii_digit());
first_ok && id.len() <= 64 && bytes.all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || matches!(b, b'.' | b'_' | b'-'))
}
+217
View File
@@ -0,0 +1,217 @@
//! A denoise model's manifest: what the model takes and gives, how big its tiles are and who may use it, as
//! plain data.
//!
//! It comes from a `denoise-model.json` next to the `.onnx` file or from the user's `catalog.json`. Both are
//! untrusted: [`validate`] must pass before a manifest is used, and [`parse`] applies it after reading JSON of
//! bounded size. The licence, id and address rules are the face models' (`crate::licence`).
use crate::licence::{Licence, valid_id};
use serde::{Deserialize, Serialize};
/// Largest `denoise-model.json` we read.
pub const MAX_MANIFEST_BYTES: usize = 64 * 1024;
/// Largest model file we accept (bytes).
pub const MAX_MODEL_BYTES: u64 = 2 * 1024 * 1024 * 1024;
/// What the model takes and what it gives.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum Domain {
/// The sensor's Bayer mosaic (black subtracted, white at 1, not white balanced) packed as four planes
/// `[R, G1, G2, B]` of its 2 × 2 cells, forced to RGGB; gives camera RGB at the mosaic's resolution, so
/// the picture is denoised and demosaiced in one step and must not be demosaiced again.
BayerToRgb,
}
/// How the model's output scale is brought back to the input's.
#[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "camelCase")]
pub enum Gain {
/// The output already has the input's scale.
None,
/// The output has a scale of its own, `nominal` times the input's (a million, for one model) and a little
/// different from tile to tile: each tile is scaled so that its channel means equal the input's. A tile too
/// dark to measure, or whose measured scale is further than `maxDeviation` (a fraction, 0.05 = 5 %) from
/// `nominal`, is scaled as if it were within that limit.
MatchMean { nominal: f32, max_deviation: f32 },
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct DenoiserManifest {
/// Lowercase letters, digits, `.` `_` `-`.
pub id: String,
pub name: String,
pub version: String,
#[serde(default)]
pub licence: Licence,
/// Where the weights come from (an `https://` page or file), for the user's information.
#[serde(default)]
pub source: Option<String>,
/// Lowercase hex SHA-256 of the `.onnx` file, when known.
#[serde(default)]
pub sha256: Option<String>,
#[serde(default)]
pub size_bytes: Option<u64>,
/// What the model was trained on, in plain words; says "undisclosed" when it is.
#[serde(default)]
pub provenance: String,
pub domain: Domain,
/// Side of the square the model runs on, in packed cells (the file's fixed input size).
pub tile: u32,
/// Cells neighbouring tiles share; the picture is blended across them.
pub overlap: u32,
pub gain: Gain,
}
#[derive(Debug, PartialEq, thiserror::Error)]
pub enum ManifestError {
#[error("`{0}` is not valid: {1}")]
Field(&'static str, String),
#[error("not a denoise-model manifest: {0}")]
Parse(String),
#[error("a manifest may be at most {MAX_MANIFEST_BYTES} bytes")]
TooLarge,
}
fn bad<T>(field: &'static str, why: impl Into<String>) -> Result<T, ManifestError> {
Err(ManifestError::Field(field, why.into()))
}
fn text(field: &'static str, s: &str, min: usize, max: usize) -> Result<(), ManifestError> {
let n = s.chars().count();
if n < min || n > max {
return bad(field, format!("must be {min} to {max} characters"));
}
if s.chars().any(|c| c.is_control() && c != '\n') {
return bad(field, "must not contain control characters");
}
Ok(())
}
fn url(field: &'static str, u: &Option<String>) -> Result<(), ManifestError> {
match u {
None => Ok(()),
Some(u) if u.len() <= 500 && u.starts_with("https://") && !u.chars().any(|c| c.is_control() || c.is_whitespace()) => Ok(()),
Some(_) => bad(field, "must be an https:// address of at most 500 characters"),
}
}
/// Check a manifest from outside before trusting it.
pub fn validate(m: &DenoiserManifest) -> Result<(), ManifestError> {
if !valid_id(&m.id) {
return bad("id", "1 to 64 lowercase letters, digits, `.`, `_` or `-`, starting with a letter or digit");
}
text("name", &m.name, 1, 80)?;
text("version", &m.version, 1, 40)?;
text("provenance", &m.provenance, 0, 1000)?;
text("licence.name", &m.licence.name, 0, 80)?;
text("licence.notice", &m.licence.notice, 0, 2000)?;
url("licence.url", &m.licence.url)?;
url("source", &m.source)?;
if let Some(h) = &m.sha256
&& !(h.len() == 64 && h.bytes().all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b)))
{
return bad("sha256", "must be 64 lowercase hex digits");
}
if let Some(n) = m.size_bytes
&& (n == 0 || n > MAX_MODEL_BYTES)
{
return bad("sizeBytes", format!("must be between 1 and {MAX_MODEL_BYTES}"));
}
// a U-Net with four poolings wants a multiple of 16; nothing smaller than 64 or larger than 2048 is a tile
if !(64..=2048).contains(&m.tile) || !m.tile.is_multiple_of(16) {
return bad("tile", "must be a multiple of 16 from 64 to 2048");
}
// tiles must overlap by less than half, or one tile's blend ramps would run into its other neighbour's
if !m.overlap.is_multiple_of(2) || m.overlap < 16 || m.overlap.saturating_mul(2) >= m.tile {
return bad("overlap", "must be even, at least 16 and less than half the tile");
}
if let Gain::MatchMean { nominal, max_deviation } = m.gain {
if !(nominal.is_finite() && (1e-6..=1e12).contains(&nominal)) {
return bad("gain.nominal", "must be between 0.000001 and 1000000000000");
}
if !(max_deviation.is_finite() && (0.0..=0.5).contains(&max_deviation)) {
return bad("gain.maxDeviation", "must be between 0 and 0.5");
}
}
Ok(())
}
/// Read and validate a `denoise-model.json`.
pub fn parse(bytes: &[u8]) -> Result<DenoiserManifest, ManifestError> {
if bytes.len() > MAX_MANIFEST_BYTES {
return Err(ManifestError::TooLarge);
}
let bytes = bytes.strip_prefix(&[0xEF, 0xBB, 0xBF]).unwrap_or(bytes);
let m: DenoiserManifest = serde_json::from_slice(bytes).map_err(|e| ManifestError::Parse(e.to_string()))?;
validate(&m)?;
Ok(m)
}
#[cfg(test)]
mod tests {
use super::*;
pub(crate) fn sample() -> DenoiserManifest {
DenoiserManifest {
id: "nind-bayer".into(),
name: "Bayer denoiser".into(),
version: "1".into(),
licence: Licence::default(),
source: Some("https://example.org/model.onnx".into()),
sha256: None,
size_bytes: Some(31_000_000),
provenance: "trained on real raw noise pairs".into(),
domain: Domain::BayerToRgb,
tile: 512,
overlap: 64,
gain: Gain::MatchMean { nominal: 1.0e6, max_deviation: 0.05 },
}
}
#[test]
fn a_good_manifest_round_trips() {
let m = sample();
let json = serde_json::to_vec(&m).unwrap();
assert_eq!(parse(&json).unwrap(), m);
}
#[test]
fn bad_fields_are_named() {
let cases: Vec<(&str, Box<dyn Fn(&mut DenoiserManifest)>)> = vec![
("id", Box::new(|m| m.id = "../evil".into())),
("id", Box::new(|m| m.id = String::new())),
("name", Box::new(|m| m.name = "x\u{7}".into())),
("source", Box::new(|m| m.source = Some("http://insecure".into()))),
("sha256", Box::new(|m| m.sha256 = Some("XYZ".into()))),
("sizeBytes", Box::new(|m| m.size_bytes = Some(0))),
("tile", Box::new(|m| m.tile = 500)),
("tile", Box::new(|m| m.tile = 4096)),
("overlap", Box::new(|m| m.overlap = 300)),
("overlap", Box::new(|m| m.overlap = 63)),
("gain.maxDeviation", Box::new(|m| m.gain = Gain::MatchMean { nominal: 1.0, max_deviation: f32::NAN })),
("gain.maxDeviation", Box::new(|m| m.gain = Gain::MatchMean { nominal: 1.0, max_deviation: 2.0 })),
("gain.nominal", Box::new(|m| m.gain = Gain::MatchMean { nominal: 0.0, max_deviation: 0.1 })),
("gain.nominal", Box::new(|m| m.gain = Gain::MatchMean { nominal: f32::INFINITY, max_deviation: 0.1 })),
];
for (field, edit) in cases {
let mut m = sample();
edit(&mut m);
match validate(&m) {
Err(ManifestError::Field(f, _)) => assert_eq!(f, field),
other => panic!("{field}: {other:?}"),
}
}
}
#[test]
fn parse_is_bounded_and_tolerant() {
assert_eq!(parse(&vec![b' '; MAX_MANIFEST_BYTES + 1]), Err(ManifestError::TooLarge));
assert!(matches!(parse(b"{ nope"), Err(ManifestError::Parse(_))));
assert!(matches!(parse(b""), Err(ManifestError::Parse(_))));
let mut with_bom = vec![0xEF, 0xBB, 0xBF];
with_bom.extend(serde_json::to_vec(&sample()).unwrap());
assert!(parse(&with_bom).is_ok());
}
}
+616
View File
@@ -0,0 +1,616 @@
//! The denoised picture as cached data, and the blend the Amount slider controls.
//!
//! A product is camera RGB at the sensor's resolution (what the model gave, before the raw's own opcodes and
//! crop), kept as half floats in strips of rows, each strip byte-shuffled and deflated with its own checksum.
//! It is derived data: any problem reading it (missing, truncated, corrupt, made for another photo or model) is
//! just a cache miss, never a panic, and the picture can be made again.
//!
//! File: `LCDN`, version, header length, header JSON (`width`, `height`, `strip`, `key`), then for each strip its
//! compressed length and CRC-32, then the strips' bytes. `key` says what the product was made from (the photo's
//! content, the model, this code's version, the settings it depends on); a file with another key is stale.
use std::io::{Read, Write};
use std::path::Path;
use lightcraft_raster::Rgb32f;
use rayon::prelude::*;
use serde::{Deserialize, Serialize};
const MAGIC: &[u8; 4] = b"LCDN";
const VERSION: u8 = 1;
const MAX_HEADER: usize = 4096;
const MAX_SIDE: usize = 1 << 17;
/// Most pixels a product may have (the denoiser's own limit, [`crate::run::MAX_PIXELS`]).
const MAX_PIXELS: usize = crate::run::MAX_PIXELS;
/// Rows per strip: about a megapixel of three planes at 24 MP widths.
const STRIP_ROWS: usize = 128;
/// Bumped when the meaning of a product changes (included in keys by the caller).
pub const ALGORITHM: u32 = 1;
#[derive(Debug, thiserror::Error)]
pub enum ProductError {
#[error("the denoised picture could not be read or written: {0}")]
Io(String),
#[error("not a usable denoised picture: {0}")]
Format(String),
/// A product made from something else (another photo state, model or version).
#[error("the denoised picture is out of date")]
Stale,
}
impl From<std::io::Error> for ProductError {
fn from(e: std::io::Error) -> Self {
ProductError::Io(e.to_string())
}
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct Header {
pub width: usize,
pub height: usize,
pub strip: usize,
pub key: String,
}
fn allocate<T: Clone>(n: usize, value: T) -> Result<Vec<T>, ProductError> {
let mut v = Vec::new();
v.try_reserve_exact(n).map_err(|_| ProductError::Format("not enough memory for the denoised picture".into()))?;
v.resize(n, value);
Ok(v)
}
/// `f32` as IEEE half-float bits, rounded to nearest even; beyond the range it saturates, NaN becomes 0.
pub fn to_f16(v: f32) -> u16 {
if !v.is_finite() {
return if v.is_nan() {
0
} else if v > 0.0 {
0x7bff
} else {
0xfbff
};
}
let b = v.to_bits();
let sign = ((b >> 16) & 0x8000) as u16;
let exp = ((b >> 23) & 0xff) as i32 - 127 + 15;
let man = b & 0x7f_ffff;
if exp >= 31 {
return sign | 0x7bff;
}
if exp <= 0 {
if exp < -10 {
return sign;
}
let man = man | 0x80_0000;
let shift = (14 - exp) as u32;
let (half, rem, halfway) = (man >> shift, man & ((1u32 << shift) - 1), 1u32 << (shift - 1));
let mut h = half as u16;
if rem > halfway || (rem == halfway && h & 1 == 1) {
h += 1;
}
return sign | h;
}
let mut h = (((exp as u32) << 10) | (man >> 13)) as u16;
let rem = man & 0x1fff;
if rem > 0x1000 || (rem == 0x1000 && h & 1 == 1) {
h += 1;
}
if h >= 0x7c00 {
h = 0x7bff;
}
sign | h
}
/// Half-float bits as `f32`.
pub fn from_f16(h: u16) -> f32 {
let sign = if h & 0x8000 != 0 { -1.0f32 } else { 1.0 };
let exp = i32::from((h >> 10) & 0x1f);
let man = f32::from(h & 0x3ff);
match exp {
0 => sign * man * 2f32.powi(-24),
31 => sign * 65504.0,
_ => sign * (1.0 + man / 1024.0) * 2f32.powi(exp - 15),
}
}
fn crc(bytes: &[u8]) -> u32 {
// CRC-32 (IEEE), bitwise: strips are small and this runs once per strip
let mut table = [0u32; 256];
for (i, t) in table.iter_mut().enumerate() {
let mut c = i as u32;
for _ in 0..8 {
c = if c & 1 != 0 { 0xedb8_8320 ^ (c >> 1) } else { c >> 1 };
}
*t = c;
}
!bytes.iter().fold(!0u32, |c, &b| table[((c ^ u32::from(b)) & 0xff) as usize] ^ (c >> 8))
}
/// One strip as bytes: for each plane (R, G, B) the low bytes of its half floats, then the high bytes.
fn encode_strip(img: &Rgb32f, y0: usize, y1: usize) -> Result<Vec<u8>, ProductError> {
let n = (y1 - y0) * img.width;
let mut raw = allocate(6 * n, 0u8)?;
let rows = img.data.get(y0 * img.width..y1 * img.width).unwrap_or(&[]);
for (i, px) in rows.iter().enumerate() {
for (c, &v) in px.iter().enumerate() {
let h = to_f16(v);
if let Some(lo) = raw.get_mut(2 * c * n + i) {
*lo = h as u8;
}
if let Some(hi) = raw.get_mut((2 * c + 1) * n + i) {
*hi = (h >> 8) as u8;
}
}
}
Ok(miniz_oxide::deflate::compress_to_vec(&raw, 3))
}
/// Write `img` to `path` (through a temporary file next to it, so a reader never sees half a product).
pub fn write(path: &Path, img: &Rgb32f, key: &str) -> Result<(), ProductError> {
let pixels = img.width.checked_mul(img.height).filter(|&n| n <= MAX_PIXELS);
if img.width == 0 || img.height == 0 || img.width > MAX_SIDE || img.height > MAX_SIDE || pixels != Some(img.data.len()) {
return Err(ProductError::Format("the picture has no usable size".into()));
}
let strips: Vec<(usize, usize)> = (0..img.height).step_by(STRIP_ROWS).map(|y| (y, (y + STRIP_ROWS).min(img.height))).collect();
let packed: Vec<Vec<u8>> = strips.par_iter().map(|&(a, b)| encode_strip(img, a, b)).collect::<Result<_, _>>()?;
let header = serde_json::to_vec(&Header { width: img.width, height: img.height, strip: STRIP_ROWS, key: key.to_string() })
.map_err(|e| ProductError::Format(e.to_string()))?;
let tmp = path.with_extension("tmp");
let result = (|| -> Result<(), ProductError> {
if let Some(dir) = path.parent() {
std::fs::create_dir_all(dir)?;
}
let mut f = std::io::BufWriter::new(std::fs::File::create(&tmp)?);
f.write_all(MAGIC)?;
f.write_all(&[VERSION, 0, 0, 0])?;
f.write_all(&(header.len() as u32).to_le_bytes())?;
f.write_all(&header)?;
for p in &packed {
f.write_all(&(p.len() as u32).to_le_bytes())?;
f.write_all(&crc(p).to_le_bytes())?;
}
for p in &packed {
f.write_all(p)?;
}
f.flush()?;
drop(f);
std::fs::rename(&tmp, path)?;
Ok(())
})();
if result.is_err() {
let _ = std::fs::remove_file(&tmp);
}
result
}
/// The parsed start of a product file: header, each strip's (compressed length, CRC-32), and the file length the
/// start says the whole file has.
struct Start {
header: Header,
table: Vec<(usize, u32)>,
expected_len: u64,
}
fn read_start(r: &mut impl Read) -> Result<Start, ProductError> {
let mut fixed = [0u8; 12];
r.read_exact(&mut fixed)?;
if fixed.get(0..4) != Some(&MAGIC[..]) || fixed.get(4) != Some(&VERSION) {
return Err(ProductError::Format("not a denoised picture of this version".into()));
}
let len = fixed.get(8..12).and_then(|b| <[u8; 4]>::try_from(b).ok()).map(u32::from_le_bytes).unwrap_or(u32::MAX) as usize;
if len > MAX_HEADER {
return Err(ProductError::Format("the header is too large".into()));
}
let mut hb = allocate(len, 0u8)?;
r.read_exact(&mut hb)?;
let h: Header = serde_json::from_slice(&hb).map_err(|e| ProductError::Format(e.to_string()))?;
let pixels = h.width.saturating_mul(h.height);
if h.width == 0 || h.height == 0 || h.width > MAX_SIDE || h.height > MAX_SIDE || pixels > MAX_PIXELS || h.strip != STRIP_ROWS {
return Err(ProductError::Format("the size is not usable".into()));
}
let n = h.height.div_ceil(h.strip);
let mut table = Vec::with_capacity(n);
let mut total = 12 + len as u64 + 8 * n as u64;
for _ in 0..n {
let mut e = [0u8; 8];
r.read_exact(&mut e)?;
let l = u32::from_le_bytes([e[0], e[1], e[2], e[3]]) as usize;
let c = u32::from_le_bytes([e[4], e[5], e[6], e[7]]);
// a strip never compresses to much more than its raw size
if l > 12 * h.strip * h.width + 4096 {
return Err(ProductError::Format("a strip is larger than it can be".into()));
}
total += l as u64;
table.push((l, c));
}
Ok(Start { header: h, table, expected_len: total })
}
/// The header of the product at `path`, when it is one.
pub fn read_header_at(path: &Path) -> Result<Header, ProductError> {
let mut f = std::io::BufReader::new(std::fs::File::open(path)?);
Ok(read_start(&mut f)?.header)
}
/// Whether `path` holds a complete-looking product made from `key` (the header and the file's length are checked;
/// strips are checked when read).
pub fn is_current(path: &Path, key: &str) -> bool {
let check = || -> Result<bool, ProductError> {
let mut f = std::io::BufReader::new(std::fs::File::open(path)?);
let s = read_start(&mut f)?;
Ok(s.header.key == key && std::fs::metadata(path)?.len() == s.expected_len)
};
check().unwrap_or(false)
}
/// A rectangle of a product, in the product's pixels.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Window {
pub x: usize,
pub y: usize,
pub width: usize,
pub height: usize,
}
/// Half floats as `f32`, for all 65536 bit patterns (a table is much faster than computing each).
fn f16_table() -> &'static [f32] {
static TABLE: std::sync::OnceLock<Vec<f32>> = std::sync::OnceLock::new();
TABLE.get_or_init(|| (0..=u16::MAX).map(from_f16).collect())
}
/// Strips decoded together (they are inflated in parallel, then folded in order).
const GROUP: usize = 8;
/// One strip, checked and inflated: for each plane the low bytes of its half floats, then the high bytes.
fn decode_strip(index: usize, packed: &[u8], expect_crc: u32, n: usize) -> Result<Vec<u8>, ProductError> {
if crc(packed) != expect_crc {
return Err(ProductError::Format(format!("strip {index} is damaged")));
}
let raw = miniz_oxide::inflate::decompress_to_vec_with_limit(packed, 6 * n).map_err(|e| ProductError::Format(format!("strip {index}: {e:?}")))?;
if raw.len() != 6 * n {
return Err(ProductError::Format(format!("strip {index} has the wrong size")));
}
Ok(raw)
}
/// Read the product at `path`, made from `key`, with every `factor × factor` block of pixels averaged into one
/// (`factor` 1 = full size). The result is camera RGB.
///
/// With a `window`, only that rectangle is read and the blocks start at its corner; only whole blocks are made, so
/// the result is `window.width / factor × window.height / factor` (the same blocks a binned development of the same
/// crop makes). Without one the whole product is read and blocks at the right and bottom edges may be smaller.
///
/// With `clip` (white is 1), a block in which any sample of a colour is at or above it takes that colour's
/// maximum instead of the mean, so a clipped highlight stays clipped for highlight reconstruction.
pub fn read_window(path: &Path, key: &str, window: Option<Window>, factor: usize, clip: Option<f32>) -> Result<Rgb32f, ProductError> {
let factor = factor.clamp(1, 1 << 12);
let mut f = std::io::BufReader::new(std::fs::File::open(path)?);
let Start { header: h, table, expected_len } = read_start(&mut f)?;
if h.key != key {
return Err(ProductError::Stale);
}
if std::fs::metadata(path)?.len() != expected_len {
return Err(ProductError::Format("the file is not the length its header says".into()));
}
let w = window.unwrap_or(Window { x: 0, y: 0, width: h.width, height: h.height });
let fits = |a: usize, b: usize, limit: usize| a.checked_add(b).is_some_and(|e| e <= limit);
if w.width == 0 || w.height == 0 || !fits(w.x, w.width, h.width) || !fits(w.y, w.height, h.height) {
return Err(ProductError::Format("the window does not lie inside the picture".into()));
}
// whole blocks for a window; partial ones at the edges when the whole picture is read
let whole = window.is_none();
let (ow, oh) = if whole { (w.width.div_ceil(factor), w.height.div_ceil(factor)) } else { (w.width / factor, w.height / factor) };
if ow == 0 || oh == 0 {
return Err(ProductError::Format("the window is smaller than one block".into()));
}
// the picture rows and columns the blocks cover
let (x_end, y_end) = if whole { (w.x + w.width, w.y + w.height) } else { (w.x + ow * factor, w.y + oh * factor) };
let mut out = Rgb32f { width: ow, height: oh, data: allocate(ow * oh, [0.0; 3])? };
let lut = f16_table();
let value = |raw: &[u8], c: usize, n: usize, i: usize| -> f32 {
let (lo, hi) = (raw.get(2 * c * n + i).copied().unwrap_or(0), raw.get((2 * c + 1) * n + i).copied().unwrap_or(0));
lut.get(usize::from(lo) | usize::from(hi) << 8).copied().unwrap_or(0.0)
};
// which strips are needed, and where each starts in the file (after the header and the strip table)
let wanted = |s: usize| s * h.strip < y_end && (s + 1) * h.strip > w.y;
let (mut sum, mut peak) = (allocate(ow, [0f32; 3])?, allocate(ow, [f32::MIN; 3])?);
let (mut rows_in, mut oy) = (0usize, 0usize);
let strips: Vec<usize> = (0..table.len()).filter(|&s| wanted(s)).collect();
// skip the strips before the first one wanted
let skipped: usize = table.iter().take(strips.first().copied().unwrap_or(0)).map(|e| e.0).sum();
f.seek_relative(i64::try_from(skipped).map_err(|_| ProductError::Format("the file is too large".into()))?)?;
for group in strips.chunks(GROUP) {
let mut packed = Vec::with_capacity(group.len());
for &s in group {
let Some(&(len, _)) = table.get(s) else { return Err(ProductError::Format("a strip is missing".into())) };
let mut p = allocate(len, 0u8)?;
f.read_exact(&mut p)?;
packed.push(p);
}
let decoded: Vec<Result<Vec<u8>, ProductError>> = group
.par_iter()
.zip(packed.par_iter())
.map(|(&s, p)| {
let rows = h.height.saturating_sub(s * h.strip).min(h.strip);
decode_strip(s, p, table.get(s).map_or(0, |e| e.1), rows * h.width)
})
.collect();
for (&s, raw) in group.iter().zip(decoded) {
let raw = raw?;
let y0 = s * h.strip;
let rows = h.height.saturating_sub(y0).min(h.strip);
let n = rows * h.width;
for r in 0..rows {
let y = y0 + r;
if y < w.y || y >= y_end {
continue;
}
let base = r * h.width;
for (ox, (sm, pk)) in sum.iter_mut().zip(peak.iter_mut()).enumerate() {
let x0 = w.x + ox * factor;
let x1 = (x0 + factor).min(x_end);
for x in x0..x1 {
for c in 0..3 {
let v = value(&raw, c, n, base + x);
if let (Some(s_), Some(p_)) = (sm.get_mut(c), pk.get_mut(c)) {
*s_ += v;
*p_ = p_.max(v);
}
}
}
}
rows_in += 1;
if rows_in == factor || y + 1 == y_end {
for (ox, (sm, pk)) in sum.iter_mut().zip(peak.iter_mut()).enumerate() {
// the block's size at the right and bottom edges may be smaller
let bw = (x_end - (w.x + ox * factor)).min(factor);
let k = 1.0 / (rows_in * bw).max(1) as f32;
if let Some(o) = out.data.get_mut(oy * ow + ox) {
for c in 0..3 {
let (s_, p_) = (sm.get(c).copied().unwrap_or(0.0), pk.get(c).copied().unwrap_or(0.0));
if let Some(slot) = o.get_mut(c) {
*slot = if factor > 1 && clip.is_some_and(|t| p_ >= t) { p_ } else { s_ * k };
}
}
}
*sm = [0.0; 3];
*pk = [f32::MIN; 3];
}
rows_in = 0;
oy += 1;
}
}
}
}
Ok(out)
}
/// [`read_window`] of the whole product: every `factor × factor` block averaged into one.
pub fn read(path: &Path, key: &str, factor: usize) -> Result<Rgb32f, ProductError> {
read_window(path, key, None, factor, None)
}
/// `base + (denoised − base) · amount` for every pixel (`amount` 0 to 1), in place in `base`. The two pictures must
/// be the same size; if they are not, `base` is left as it was and `false` is returned.
pub fn blend(base: &mut Rgb32f, denoised: &Rgb32f, amount: f32) -> bool {
if base.width != denoised.width || base.height != denoised.height {
return false;
}
let a = if amount.is_finite() { amount.clamp(0.0, 1.0) } else { 0.0 };
base.data.par_iter_mut().zip(denoised.data.par_iter()).for_each(|(b, d)| {
for c in 0..3 {
b[c] += (d[c] - b[c]) * a;
}
});
true
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn oversized_product_header_is_rejected_before_allocating_pixels() {
let p = temp("oversized-header");
let header = serde_json::to_vec(&Header { width: 10001, height: 10000, strip: STRIP_ROWS, key: "k".into() }).unwrap();
let mut bytes = Vec::from(MAGIC.as_slice());
bytes.extend_from_slice(&[VERSION, 0, 0, 0]);
bytes.extend_from_slice(&(header.len() as u32).to_le_bytes());
bytes.extend_from_slice(&header);
std::fs::write(&p, bytes).unwrap();
assert!(read_header_at(&p).is_err());
let _ = std::fs::remove_file(p);
}
fn temp(name: &str) -> std::path::PathBuf {
std::env::temp_dir().join(format!("lc-denoise-{name}-{}.lcdn", std::process::id()))
}
fn picture(w: usize, h: usize) -> Rgb32f {
Rgb32f {
width: w,
height: h,
data: (0..w * h).map(|i| [(i % w) as f32 / w as f32, (i / w) as f32 / h as f32, 0.001 + 0.5 * ((i % 7) as f32 / 7.0)]).collect(),
}
}
#[test]
fn half_floats_round_trip_closely() {
for v in [0.0f32, 1.0, -1.0, 0.5, 0.333_333, 1e-4, 6.1e-5, 3e-6, 123.456, 65000.0, -0.002] {
let back = from_f16(to_f16(v));
assert!((back - v).abs() <= v.abs() * 0.001 + 1e-7, "{v} -> {back}");
}
assert_eq!(from_f16(to_f16(1e10)), 65504.0);
assert_eq!(to_f16(f32::NAN), 0);
assert_eq!(from_f16(to_f16(f32::NEG_INFINITY)), -65504.0);
}
#[test]
fn a_product_survives_a_round_trip_at_full_and_reduced_size() {
let (w, h) = (301, 517);
let img = picture(w, h);
let p = temp("round");
write(&p, &img, "k1").unwrap();
assert!(is_current(&p, "k1") && !is_current(&p, "k2"));
assert_eq!(read_header_at(&p).unwrap().width, w);
let full = read(&p, "k1", 1).unwrap();
assert_eq!((full.width, full.height), (w, h));
for (a, b) in full.data.iter().zip(&img.data) {
for c in 0..3 {
assert!((a[c] - b[c]).abs() < 0.002, "{a:?} vs {b:?}");
}
}
// averaged into blocks of 4: the sizes round up, and the mean of the picture is kept
let small = read(&p, "k1", 4).unwrap();
assert_eq!((small.width, small.height), (w.div_ceil(4), h.div_ceil(4)));
let mean = |i: &Rgb32f| i.data.iter().map(|p| f64::from(p[1])).sum::<f64>() / i.data.len() as f64;
assert!((mean(&small) - mean(&img)).abs() < 0.01);
// a factor that does not divide the strip height is the same average
let odd = read(&p, "k1", 5).unwrap();
assert_eq!((odd.width, odd.height), (w.div_ceil(5), h.div_ceil(5)));
assert!((mean(&odd) - mean(&img)).abs() < 0.01);
assert!(matches!(read(&p, "other", 1), Err(ProductError::Stale)));
std::fs::remove_file(&p).unwrap();
}
/// What binning a window of `img` by `k` means, written the slow obvious way.
fn blocks(img: &Rgb32f, win: Window, k: usize, clip: Option<f32>) -> Rgb32f {
let (ow, oh) = (win.width / k, win.height / k);
let mut out = Rgb32f { width: ow, height: oh, data: vec![[0.0; 3]; ow * oh] };
for by in 0..oh {
for bx in 0..ow {
for c in 0..3 {
let vals: Vec<f32> = (0..k * k).map(|i| img.data[(win.y + by * k + i / k) * img.width + win.x + bx * k + i % k][c]).collect();
let max = vals.iter().copied().fold(f32::MIN, f32::max);
out.data[by * ow + bx][c] =
if k > 1 && clip.is_some_and(|t| max >= t) { max } else { vals.iter().sum::<f32>() / vals.len() as f32 };
}
}
}
out
}
#[test]
fn a_window_is_read_as_the_blocks_of_that_crop() {
let (w, h) = (301, 517);
let mut img = picture(w, h);
// a clipped sample in the middle of a block
img.data[(130 + 2) * w + 61 + 1] = [1.0, 0.4, 0.4];
let p = temp("window");
write(&p, &img, "k1").unwrap();
let near = |a: &Rgb32f, b: &Rgb32f| {
assert_eq!((a.width, a.height), (b.width, b.height));
for (x, y) in a.data.iter().zip(&b.data) {
for c in 0..3 {
assert!((x[c] - y[c]).abs() < 0.003, "{x:?} vs {y:?}");
}
}
};
// windows at odd places, across strip boundaries (128 rows), with factors that do and do not divide them
for (win, k) in [
(Window { x: 3, y: 5, width: 200, height: 300 }, 1),
(Window { x: 3, y: 5, width: 200, height: 300 }, 4),
(Window { x: 61, y: 130, width: 121, height: 250 }, 5),
(Window { x: 0, y: 0, width: w, height: h }, 2),
(Window { x: 1, y: 380, width: 90, height: 137 }, 3),
] {
let got = read_window(&p, "k1", Some(win), k, None).unwrap();
near(&got, &blocks(&img, win, k, None));
}
// with a clip level the block with the clipped sample keeps its highest value, the others stay means
let win = Window { x: 61, y: 130, width: 40, height: 20 };
let clipped = read_window(&p, "k1", Some(win), 4, Some(0.99)).unwrap();
near(&clipped, &blocks(&img, win, 4, Some(0.99)));
assert!(clipped.data[0][0] > 0.99, "the clipped block stays clipped: {:?}", clipped.data[0]);
let plain = read_window(&p, "k1", Some(win), 4, None).unwrap();
assert!(plain.data[0][0] < 0.99, "an average falls below the clip level");
// windows that do not fit, or are smaller than a block, are errors
for bad in [
Window { x: 0, y: 0, width: 0, height: 5 },
Window { x: 290, y: 0, width: 20, height: 5 },
Window { x: 0, y: 510, width: 5, height: 10 },
Window { x: usize::MAX, y: 0, width: 2, height: 2 },
Window { x: 0, y: 0, width: 3, height: 3 },
] {
assert!(read_window(&p, "k1", Some(bad), 4, None).is_err(), "{bad:?}");
}
std::fs::remove_file(&p).unwrap();
}
#[test]
fn a_smooth_picture_compresses_well() {
let img = Rgb32f {
width: 512,
height: 512,
data: (0..512 * 512)
.map(|i| {
let v = 0.2 + 0.0001 * (i % 512) as f32;
[v, v * 0.8, v * 0.6]
})
.collect(),
};
let p = temp("size");
write(&p, &img, "k").unwrap();
let bytes = std::fs::metadata(&p).unwrap().len() as usize;
assert!(bytes < 512 * 512 * 6 / 2, "{bytes} bytes for a smooth picture");
std::fs::remove_file(&p).unwrap();
}
#[test]
fn damaged_files_are_errors_not_panics() {
let img = picture(100, 300);
let p = temp("damage");
write(&p, &img, "k").unwrap();
let good = std::fs::read(&p).unwrap();
// truncated at every length
for cut in (0..good.len()).step_by(37) {
std::fs::write(&p, &good[..cut]).unwrap();
assert!(read(&p, "k", 1).is_err(), "cut at {cut}");
assert!(!is_current(&p, "k"));
}
// a flipped byte anywhere is caught by a checksum, the header parse, or the key
for at in (0..good.len()).step_by(53) {
let mut bad = good.clone();
bad[at] ^= 0xff;
std::fs::write(&p, &bad).unwrap();
let _ = read(&p, "k", 1); // must not panic; most are errors
}
// a header that claims an absurd size
let mut huge = b"LCDN\x01\0\0\0".to_vec();
let hdr = br#"{"width":4000000000,"height":4000000000,"strip":128,"key":"k"}"#;
huge.extend((hdr.len() as u32).to_le_bytes());
huge.extend(hdr);
std::fs::write(&p, &huge).unwrap();
assert!(matches!(read(&p, "k", 1), Err(ProductError::Format(_))));
std::fs::write(&p, b"").unwrap();
assert!(read(&p, "k", 1).is_err());
assert!(read(&temp("missing"), "k", 1).is_err());
std::fs::remove_file(&p).unwrap();
}
#[test]
fn writing_rejects_unusable_pictures() {
let p = temp("bad");
assert!(write(&p, &Rgb32f { width: 0, height: 0, data: vec![] }, "k").is_err());
assert!(write(&p, &Rgb32f { width: 3, height: 3, data: vec![[0.0; 3]; 8] }, "k").is_err());
assert!(!p.exists());
}
#[test]
fn blend_mixes_by_amount_and_rejects_other_sizes() {
let mut base = Rgb32f { width: 2, height: 1, data: vec![[0.0; 3], [1.0; 3]] };
let dn = Rgb32f { width: 2, height: 1, data: vec![[1.0; 3], [0.0; 3]] };
assert!(blend(&mut base, &dn, 0.25));
assert_eq!(base.data, vec![[0.25; 3], [0.75; 3]]);
assert!(blend(&mut base, &dn, f32::NAN));
assert_eq!(base.data, vec![[0.25; 3], [0.75; 3]]);
assert!(!blend(&mut base, &Rgb32f { width: 1, height: 1, data: vec![[0.0; 3]] }, 1.0));
assert!(blend(&mut base, &dn, 1.0));
assert_eq!(base.data, dn.data);
}
#[test]
fn crc_matches_the_standard_check_value() {
assert_eq!(crc(b"123456789"), 0xcbf4_3926);
}
}
+589
View File
@@ -0,0 +1,589 @@
//! Denoising a whole Bayer mosaic with a tile model: cut it into overlapping tiles, run the model on each, bring
//! each tile's scale back to the input's, keep clipped highlights as they were, and blend the tiles together.
//!
//! The model itself is a [`TileRunner`], so everything around it (packing, tiling, scale matching, highlight
//! protection, blending, cancelling, progress) is tested without one.
use std::sync::atomic::{AtomicBool, Ordering};
use lightcraft_raster::Rgb32f;
use rayon::prelude::*;
use crate::bayer::{Layout, pack_tile};
use crate::manifest::Gain;
use crate::tiles::{starts, weight};
/// Most pixels one picture may have (guards the allocations that follow from its size).
pub const MAX_PIXELS: usize = 100_000_000;
/// A tile whose mean signal is below this (white = 1) is too dark to measure its scale from.
const MIN_SIGNAL: f32 = 0.003;
#[derive(Debug, thiserror::Error)]
pub enum Error {
#[error("{0}")]
Input(String),
#[error("the model gave {0}")]
Model(String),
#[error("the model could not run: {0}")]
Runtime(String),
#[error("the model could not be loaded: {0}")]
Load(String),
#[error("cancelled")]
Cancelled,
}
fn allocate<T: Clone>(n: usize, value: T) -> Result<Vec<T>, Error> {
let mut v = Vec::new();
v.try_reserve_exact(n).map_err(|_| Error::Input("not enough memory for denoise".into()))?;
v.resize(n, value);
Ok(v)
}
/// Something that runs the model on one tile.
pub trait TileRunner: Sync {
/// `input` is the tile as four planes `[R, G1, G2, B]` of `tile × tile` cells (NCHW, batch 1); the answer is
/// camera RGB, three planes of `2·tile × 2·tile` pixels (CHW), in the model's own scale.
fn run(&self, input: &[f32]) -> Result<Vec<f32>, Error>;
}
pub struct Params {
/// Side of the model's square, in cells.
pub tile: usize,
/// Cells neighbouring tiles share.
pub overlap: usize,
pub gain: Gain,
/// White is 1; samples at or above this (minus a little) are clipped and keep their value.
pub clip: Option<f32>,
/// Tiles run at once (each model call may itself be single-threaded).
pub parallel: usize,
}
#[derive(Default)]
pub struct Control<'a> {
pub cancel: Option<&'a AtomicBool>,
/// `(tiles done, tiles in all)`.
pub progress: Option<&'a (dyn Fn(usize, usize) + Sync)>,
}
/// The tile grid of a picture: `(starts across, starts down)`.
pub fn grid(width: usize, height: usize, layout: Layout, tile: usize, overlap: usize) -> (Vec<usize>, Vec<usize>) {
let (nx, ny) = layout.cells(width, height);
(starts(nx, tile, overlap), starts(ny, tile, overlap))
}
/// How many model calls [`denoise_bayer`] makes for a picture.
pub fn tile_count(width: usize, height: usize, layout: Layout, tile: usize, overlap: usize) -> usize {
let (sx, sy) = grid(width, height, layout, tile, overlap);
sx.len().saturating_mul(sy.len())
}
/// Denoise (and demosaic) the normalised mosaic `mosaic` of `width × height` samples, laid out as `layout`.
/// Gives camera RGB at the mosaic's resolution.
pub fn denoise_bayer(
mosaic: &[f32],
width: usize,
height: usize,
layout: Layout,
runner: &dyn TileRunner,
p: &Params,
ctl: &Control<'_>,
) -> Result<Rgb32f, Error> {
let pixels = width
.checked_mul(height)
.filter(|&n| n > 0 && n <= MAX_PIXELS)
.ok_or_else(|| Error::Input("the picture has no size, or is too large".into()))?;
if mosaic.len() != pixels {
return Err(Error::Input("the mosaic does not have width × height samples".into()));
}
if !(16..=2048).contains(&p.tile) || p.overlap >= p.tile.div_ceil(2) || layout.dx > 1 || layout.dy > 1 {
return Err(Error::Input("the tile size and overlap do not fit".into()));
}
if let Gain::MatchMean { nominal, max_deviation } = p.gain
&& !(nominal.is_finite() && (1e-6..=1e12).contains(&nominal) && max_deviation.is_finite() && (0.0..=0.5).contains(&max_deviation))
{
return Err(Error::Input("the model gain is not finite or within its supported range".into()));
}
if p.clip.is_some_and(|v| !v.is_finite() || v <= 0.0) {
return Err(Error::Input("the highlight clip must be finite and positive".into()));
}
let (nx, ny) = layout.cells(width, height);
let (sx, sy) = (starts(nx, p.tile, p.overlap), starts(ny, p.tile, p.overlap));
let (cols, rows) = (sx.len(), sy.len());
if cols == 0 || rows == 0 {
return Err(Error::Input("the picture needs too many tiles".into()));
}
let tiles: Vec<(usize, usize, bool, bool, bool, bool)> = sy
.iter()
.enumerate()
.flat_map(|(j, &y0)| sx.iter().enumerate().map(move |(i, &x0)| (x0, y0, i > 0, i + 1 < cols, j > 0, j + 1 < rows)))
.collect();
let total = tiles.len();
let mut acc = allocate(pixels, [0f32; 3])?;
let mut wsum = allocate(nx * ny, 0f32)?;
let group = p.parallel.clamp(1, 4);
let mut done = 0;
// where the time goes, printed under `LIGHTCRAFT_PROFILE`
let (mut t_wait, mut t_total) = (std::time::Duration::ZERO, std::time::Duration::ZERO);
let began = web_time::Instant::now();
// One group of tiles at a time: pack, run the model, check and bring each to the input's scale, in parallel.
// Blending (sequential: the tiles add into one picture, in tile order so the result does not depend on timing) of
// the previous group runs while the next group is on the model, so neither the card nor the cores wait for it.
type Tile = (usize, usize, bool, bool, bool, bool);
let compute = |chunk: &[Tile]| -> Vec<Result<Vec<f32>, Error>> {
chunk
.par_iter()
.map(|&(x0, y0, bx, ax, by, ay)| {
let mut input = allocate(4 * p.tile * p.tile, 0f32)?;
if !pack_tile(mosaic, width, height, layout, x0 as isize, y0 as isize, p.tile, &mut input) {
return Err(Error::Input("a tile could not be packed".into()));
}
let mut out = runner.run(&input)?;
let side = 2 * p.tile;
if out.len() != 3 * side * side {
return Err(Error::Model(format!("{} numbers for a tile, not the {} expected", out.len(), 3 * side * side)));
}
if out.iter().any(|v| !v.is_finite()) {
return Err(Error::Model("numbers that are not finite".into()));
}
finish_tile(&input, &mut out, p, (bx, ax, by, ay));
Ok(out)
})
.collect()
};
let mut blend = |chunk: &[Tile], outs: Vec<Result<Vec<f32>, Error>>| -> Result<(), Error> {
for (&(x0, y0, bx, ax, by, ay), r) in chunk.iter().zip(outs) {
blend_tile(&r?, &mut acc, &mut wsum, (width, height), (nx, ny), layout, (x0, y0), p, (bx, ax, by, ay));
done += 1;
if let Some(f) = ctl.progress {
f(done, total);
}
}
Ok(())
};
let mut previous: Option<(&[Tile], Vec<Result<Vec<f32>, Error>>)> = None;
for chunk in tiles.chunks(group) {
if ctl.cancel.is_some_and(|c| c.load(Ordering::Relaxed)) {
return Err(Error::Cancelled);
}
let (outs, blended) = rayon::join(
|| compute(chunk),
|| match previous.take() {
Some((c, o)) => {
let started = web_time::Instant::now();
let r = blend(c, o);
(r, started.elapsed())
}
None => (Ok(()), std::time::Duration::ZERO),
},
);
blended.0?;
t_wait += blended.1;
previous = Some((chunk, outs));
}
if let Some((c, o)) = previous.take() {
let started = web_time::Instant::now();
blend(c, o)?;
t_wait += started.elapsed();
}
t_total += began.elapsed();
let started = web_time::Instant::now();
// normalise by the weights
let (ox, oy) = (layout.position(0, 0).0, layout.position(0, 0).1);
acc.par_chunks_mut(width).enumerate().for_each(|(y, row)| {
let cy = ((y as isize - oy) / 2) as usize;
for (x, px) in row.iter_mut().enumerate() {
let cx = ((x as isize - ox) / 2) as usize;
let w = wsum.get(cy * nx + cx).copied().unwrap_or(0.0);
if w > 0.0 {
*px = px.map(|v| v / w);
}
}
});
if std::env::var_os("LIGHTCRAFT_PROFILE").is_some() {
eprintln!(
"[denoise] {total} tiles, {} at once: tiles {:.0} ms (blending, {:.0} ms of it, overlaps the model), normalise {:.0} ms",
group,
t_total.as_secs_f64() * 1e3,
t_wait.as_secs_f64() * 1e3,
started.elapsed().as_secs_f64() * 1e3
);
}
Ok(Rgb32f { width, height, data: acc })
}
/// Bring a tile's output to the input's scale and keep clipped highlights as they were (in place).
fn finish_tile(input: &[f32], out: &mut [f32], p: &Params, edges: (bool, bool, bool, bool)) {
let t = p.tile;
let side = 2 * t;
if let Gain::MatchMean { nominal, max_deviation } = p.gain {
// the part of the tile whose scale is measured: away from the edges that blend into a neighbour
let m = p.overlap / 2;
let (i0, i1) = (if edges.0 { m } else { 0 }, t - if edges.1 { m } else { 0 });
let (j0, j1) = (if edges.2 { m } else { 0 }, t - if edges.3 { m } else { 0 });
let cells = ((i1 - i0) * (j1 - j0)) as f64;
let plane_sum = |pl: usize| -> f64 {
(j0..j1)
.map(|j| input.get(pl * t * t + j * t + i0..pl * t * t + j * t + i1).map_or(0.0, |r| r.iter().map(|&v| f64::from(v)).sum::<f64>()))
.sum()
};
let in_mean = [plane_sum(0) / cells, (plane_sum(1) + plane_sum(2)) / (2.0 * cells), plane_sum(3) / cells];
for (c, &mean_in) in in_mean.iter().enumerate() {
let sum_out: f64 = (2 * j0..2 * j1)
.map(|y| {
out.get(c * side * side + y * side + 2 * i0..c * side * side + y * side + 2 * i1)
.map_or(0.0, |r| r.iter().map(|&v| f64::from(v)).sum::<f64>())
})
.sum();
let mean_out = sum_out / (4.0 * cells);
let measured = if mean_in as f32 >= MIN_SIGNAL && mean_out > 0.0 { (mean_out / mean_in) as f32 } else { nominal };
let g = measured.clamp(nominal * (1.0 - max_deviation), nominal * (1.0 + max_deviation));
if g.is_finite() && g > 0.0 {
let inv = 1.0 / g;
if let Some(plane) = out.get_mut(c * side * side..(c + 1) * side * side) {
plane.iter_mut().for_each(|v| *v *= inv);
}
}
}
}
if let Some(clip) = p.clip {
protect_clipped(input, out, t, clip);
}
}
/// Where the mosaic is at (or within a cell of) white, the output becomes the plain cell values again, so a clipped
/// area stays exactly as clipped as the camera recorded it (highlight recovery later depends on that).
fn protect_clipped(input: &[f32], out: &mut [f32], t: usize, clip: f32) {
let side = 2 * t;
let plane = t * t;
let (Some(r), Some(g1), Some(g2), Some(b)) =
(input.get(0..plane), input.get(plane..2 * plane), input.get(2 * plane..3 * plane), input.get(3 * plane..4 * plane))
else {
return;
};
let mut m: Vec<f32> = (0..plane).map(|i| r[i].max(g1[i]).max(g2[i]).max(b[i])).collect();
if !m.iter().any(|&v| v >= clip) {
return;
}
// 3×3 maximum over cells (separable)
let dilate = |src: &[f32], horizontal: bool| -> Vec<f32> {
(0..plane)
.map(|i| {
let (x, y) = (i % t, i / t);
let mut best = src[i];
for d in [-1isize, 1] {
let (nx, ny) = if horizontal { (x as isize + d, y as isize) } else { (x as isize, y as isize + d) };
if nx >= 0 && ny >= 0 && (nx as usize) < t && (ny as usize) < t {
best = best.max(src[ny as usize * t + nx as usize]);
}
}
best
})
.collect()
};
m = dilate(&dilate(&m, true), false);
let lo = clip - 0.03;
for j in 0..t {
for i in 0..t {
let k = j * t + i;
let s = ((m[k] - lo) / (clip - lo)).clamp(0.0, 1.0);
if s <= 0.0 {
continue;
}
let s = s * s * (3.0 - 2.0 * s);
let plain = [r[k], 0.5 * (g1[k] + g2[k]), b[k]];
for (c, &pv) in plain.iter().enumerate() {
for (sy, sx) in [(0, 0), (0, 1), (1, 0), (1, 1)] {
if let Some(v) = out.get_mut(c * side * side + (2 * j + sy) * side + 2 * i + sx) {
*v += (pv - *v) * s;
}
}
}
}
}
}
/// Add a finished tile to the picture with its blend weights.
#[allow(clippy::too_many_arguments)]
fn blend_tile(
out: &[f32],
acc: &mut [[f32; 3]],
wsum: &mut [f32],
size: (usize, usize),
cells: (usize, usize),
layout: Layout,
at: (usize, usize),
p: &Params,
edges: (bool, bool, bool, bool),
) {
let (width, height) = size;
let (nx, ny) = cells;
let (t, side) = (p.tile, 2 * p.tile);
let fade = p.overlap;
let wx: Vec<f32> = (0..t).map(|i| weight(i, t, fade, edges.0, edges.1)).collect();
let wy: Vec<f32> = (0..t).map(|j| weight(j, t, fade, edges.2, edges.3)).collect();
for j in 0..t {
let cy = at.1 + j;
for i in 0..t {
let cx = at.0 + i;
// cells past the picture's edge (a picture smaller than one tile is padded) belong to no pixel
if cx >= nx || cy >= ny {
continue;
}
let (Some(&a), Some(&b)) = (wx.get(i), wy.get(j)) else { continue };
let w = a * b;
if let Some(s) = wsum.get_mut(cy * nx + cx) {
*s += w;
}
let (px0, py0) = layout.position(cx as isize, cy as isize);
for (sy, sx) in [(0isize, 0isize), (0, 1), (1, 0), (1, 1)] {
let (x, y) = (px0 + sx, py0 + sy);
if x < 0 || y < 0 || x as usize >= width || y as usize >= height {
continue;
}
let o = (2 * j + sy as usize) * side + 2 * i + sx as usize;
let (Some(&r), Some(&g), Some(&bl)) = (out.get(o), out.get(side * side + o), out.get(2 * side * side + o)) else { continue };
if let Some(px) = acc.get_mut(y as usize * width + x as usize) {
px[0] += w * r;
px[1] += w * g;
px[2] += w * bl;
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// A stand-in model: the cell's samples as an RGB picture at twice the resolution (a plain 2 × 2 "demosaic"),
/// times `scale`, so the output has a scale of its own like the real model's.
struct Mock {
tile: usize,
scale: f32,
}
impl TileRunner for Mock {
fn run(&self, input: &[f32]) -> Result<Vec<f32>, Error> {
let (t, side) = (self.tile, 2 * self.tile);
let plane = t * t;
let mut out = vec![0.0; 3 * side * side];
for j in 0..t {
for i in 0..t {
let k = j * t + i;
let rgb = [input[k], 0.5 * (input[plane + k] + input[2 * plane + k]), input[3 * plane + k]];
for (c, v) in rgb.iter().enumerate() {
for (sy, sx) in [(0, 0), (0, 1), (1, 0), (1, 1)] {
out[c * side * side + (2 * j + sy) * side + 2 * i + sx] = v * self.scale;
}
}
}
}
Ok(out)
}
}
fn params(tile: usize, overlap: usize, gain: Gain) -> Params {
Params { tile, overlap, gain, clip: None, parallel: 2 }
}
#[test]
fn overflowing_tiles_and_nan_gain_are_errors_before_the_model_runs() {
let layout = Layout::from_cell([0, 1, 1, 2]).unwrap();
let model = Mock { tile: 16, scale: 1.0 };
for (tile, overlap, gain) in [
(usize::MAX, 0, Gain::None),
(16, usize::MAX, Gain::None),
(16, 0, Gain::MatchMean { nominal: 1.0, max_deviation: f32::NAN }),
(16, 0, Gain::MatchMean { nominal: f32::INFINITY, max_deviation: 0.1 }),
] {
assert!(matches!(denoise_bayer(&[0.5], 1, 1, layout, &model, &params(tile, overlap, gain), &Control::default()), Err(Error::Input(_))));
}
}
/// A smooth picture whose colour at a sample is `base[colour] · (1 + shading)`.
fn mosaic(w: usize, h: usize, cell: [u8; 4]) -> Vec<f32> {
(0..w * h)
.map(|i| {
let (x, y) = (i % w, i / w);
let c = cell[(y % 2) * 2 + x % 2] as usize;
[0.5, 0.3, 0.2][c] * (0.6 + 0.4 * ((x as f32 / 23.0).sin() * (y as f32 / 31.0).cos()))
})
.collect()
}
fn reference(m: &[f32], w: usize, h: usize, layout: Layout) -> Rgb32f {
// what the mock gives for the whole picture in one tile
let (nx, ny) = layout.cells(w, h);
let t = nx.max(ny).div_ceil(16) * 16;
denoise_bayer(m, w, h, layout, &Mock { tile: t, scale: 1.0 }, &params(t, 16, Gain::None), &Control::default()).unwrap()
}
#[test]
fn tiles_blend_to_the_same_picture_as_one_tile_for_every_layout() {
for cell in [[0u8, 1, 1, 2], [1, 0, 2, 1], [1, 2, 0, 1], [2, 1, 1, 0]] {
let layout = Layout::from_cell(cell).unwrap();
let (w, h) = (150usize, 110usize);
let m = mosaic(w, h, cell);
let want = reference(&m, w, h, layout);
let got = denoise_bayer(&m, w, h, layout, &Mock { tile: 32, scale: 1.0 }, &params(32, 8, Gain::None), &Control::default()).unwrap();
assert_eq!((got.width, got.height), (w, h));
for (a, b) in got.data.iter().zip(&want.data) {
for c in 0..3 {
assert!((a[c] - b[c]).abs() < 1e-5, "layout {cell:?}: {a:?} vs {b:?}");
}
}
}
}
#[test]
fn the_picture_does_not_depend_on_how_many_tiles_run_at_once_or_how_long_each_takes() {
// tiles that finish in a different order every time: blending still adds them in tile order
struct Jittery(Mock);
impl TileRunner for Jittery {
fn run(&self, input: &[f32]) -> Result<Vec<f32>, Error> {
let wait = (input.iter().take(64).sum::<f32>() * 1000.0) as u64 % 7;
std::thread::sleep(std::time::Duration::from_millis(wait));
self.0.run(input)
}
}
let layout = Layout { dx: 0, dy: 1 };
let (w, h) = (150usize, 110usize);
let m = mosaic(w, h, [1, 0, 2, 1]);
let run = |parallel: usize| {
let mut p = params(32, 8, Gain::MatchMean { nominal: 1.0e6, max_deviation: 0.05 });
p.parallel = parallel;
denoise_bayer(&m, w, h, layout, &Jittery(Mock { tile: 32, scale: 1.01e6 }), &p, &Control::default()).unwrap()
};
let one = run(1);
for parallel in [2, 3, 8] {
assert_eq!(run(parallel).data, one.data, "{parallel} at once");
}
}
#[test]
fn a_failing_tile_or_a_cancel_ends_the_picture_without_a_panic() {
struct Fails(Mock, std::sync::atomic::AtomicUsize);
impl TileRunner for Fails {
fn run(&self, input: &[f32]) -> Result<Vec<f32>, Error> {
if self.1.fetch_add(1, Ordering::Relaxed) == 4 {
return Err(Error::Runtime("boom".into()));
}
self.0.run(input)
}
}
let layout = Layout { dx: 0, dy: 0 };
let (w, h) = (150usize, 110usize);
let m = mosaic(w, h, [0, 1, 1, 2]);
let r = denoise_bayer(
&m,
w,
h,
layout,
&Fails(Mock { tile: 32, scale: 1.0 }, Default::default()),
&params(32, 8, Gain::None),
&Control::default(),
);
assert!(matches!(r, Err(Error::Runtime(_))), "{r:?}");
let stop = AtomicBool::new(true);
let ctl = Control { cancel: Some(&stop), progress: None };
let r = denoise_bayer(&m, w, h, layout, &Mock { tile: 32, scale: 1.0 }, &params(32, 8, Gain::None), &ctl);
assert!(matches!(r, Err(Error::Cancelled)));
}
#[test]
fn a_scale_of_its_own_is_taken_out_tile_by_tile() {
let layout = Layout { dx: 0, dy: 0 };
let (w, h) = (150usize, 110usize);
let m = mosaic(w, h, [0, 1, 1, 2]);
let want = reference(&m, w, h, layout);
let gain = Gain::MatchMean { nominal: 1.0e6, max_deviation: 0.05 };
// the scale differs a little from nominal; each tile's own scale is measured and removed
let got = denoise_bayer(&m, w, h, layout, &Mock { tile: 32, scale: 1.02e6 }, &params(32, 8, gain), &Control::default()).unwrap();
let mean = |img: &Rgb32f, c: usize| img.data.iter().map(|p| f64::from(p[c])).sum::<f64>() / img.data.len() as f64;
for c in 0..3 {
let (a, b) = (mean(&got, c), mean(&want, c));
assert!((a - b).abs() / b < 0.01, "channel {c}: {a} vs {b}");
}
// a scale far from nominal is clamped rather than trusted
let wild = denoise_bayer(&m, w, h, layout, &Mock { tile: 32, scale: 3.0e6 }, &params(32, 8, gain), &Control::default()).unwrap();
assert!(mean(&wild, 1) > 2.0 * mean(&want, 1));
}
#[test]
fn a_picture_smaller_than_a_tile_is_padded_by_mirroring() {
let layout = Layout { dx: 1, dy: 1 };
let (w, h) = (20usize, 14usize);
let m = mosaic(w, h, [2, 1, 1, 0]);
let got = denoise_bayer(&m, w, h, layout, &Mock { tile: 64, scale: 1.0 }, &params(64, 16, Gain::None), &Control::default()).unwrap();
assert_eq!((got.width, got.height), (w, h));
assert!(got.data.iter().all(|p| p.iter().all(|v| v.is_finite() && *v > 0.0)));
}
#[test]
fn clipped_highlights_stay_clipped() {
// a model that darkens everything by 10 % would pull a clipped plateau below white
struct Dim(Mock);
impl TileRunner for Dim {
fn run(&self, input: &[f32]) -> Result<Vec<f32>, Error> {
Ok(self.0.run(input)?.into_iter().map(|v| v * 0.9).collect())
}
}
let layout = Layout { dx: 0, dy: 0 };
let (w, h) = (96usize, 96usize);
let m: Vec<f32> = (0..w * h).map(|i| if i % w > 40 && i % w < 70 { 1.0 } else { 0.4 }).collect();
let mut p = params(32, 8, Gain::None);
p.clip = Some(0.99);
let got = denoise_bayer(&m, w, h, layout, &Dim(Mock { tile: 32, scale: 1.0 }), &p, &Control::default()).unwrap();
let mid = got.data[48 * w + 55];
assert!(mid.iter().all(|&v| v > 0.995), "plateau {mid:?}");
let dark = got.data[48 * w + 10];
assert!(dark.iter().all(|&v| (v - 0.36).abs() < 0.01), "unclipped area still goes through the model: {dark:?}");
}
#[test]
fn cancelling_and_progress_work() {
let layout = Layout { dx: 0, dy: 0 };
let (w, h) = (150usize, 110usize);
let m = mosaic(w, h, [0, 1, 1, 2]);
let cancel = AtomicBool::new(true);
let r = denoise_bayer(
&m,
w,
h,
layout,
&Mock { tile: 32, scale: 1.0 },
&params(32, 8, Gain::None),
&Control { cancel: Some(&cancel), progress: None },
);
assert!(matches!(r, Err(Error::Cancelled)));
let seen = std::sync::Mutex::new(Vec::new());
let f = |d: usize, n: usize| seen.lock().unwrap().push((d, n));
denoise_bayer(&m, w, h, layout, &Mock { tile: 32, scale: 1.0 }, &params(32, 8, Gain::None), &Control { cancel: None, progress: Some(&f) })
.unwrap();
let seen = seen.into_inner().unwrap();
let total = tile_count(w, h, layout, 32, 8);
assert!(total > 1);
assert_eq!(seen.last(), Some(&(total, total)));
assert!(seen.windows(2).all(|p| p[1].0 == p[0].0 + 1));
}
#[test]
fn hostile_inputs_are_errors() {
let layout = Layout { dx: 0, dy: 0 };
let mock = Mock { tile: 32, scale: 1.0 };
let p = params(32, 8, Gain::None);
assert!(matches!(denoise_bayer(&[], 0, 0, layout, &mock, &p, &Control::default()), Err(Error::Input(_))));
assert!(matches!(denoise_bayer(&[0.0; 10], 4, 4, layout, &mock, &p, &Control::default()), Err(Error::Input(_))));
assert!(matches!(denoise_bayer(&[0.0; 16], 4, 4, layout, &mock, &params(32, 16, Gain::None), &Control::default()), Err(Error::Input(_))));
struct Bad(Vec<f32>);
impl TileRunner for Bad {
fn run(&self, _: &[f32]) -> Result<Vec<f32>, Error> {
Ok(self.0.clone())
}
}
assert!(matches!(denoise_bayer(&[0.5; 16], 4, 4, layout, &Bad(vec![0.0; 5]), &p, &Control::default()), Err(Error::Model(_))));
assert!(matches!(denoise_bayer(&[0.5; 16], 4, 4, layout, &Bad(vec![f32::NAN; 3 * 64 * 64]), &p, &Control::default()), Err(Error::Model(_))));
}
}
+100
View File
@@ -0,0 +1,100 @@
//! Cutting a picture into overlapping tiles and putting the results back together.
//!
//! A model runs on a fixed square, so a picture is covered by tiles spaced evenly along each axis, each sharing
//! at least `overlap` cells with its neighbours. Each tile's influence fades towards its own edges (except at the
//! picture's edge), and the result is the sum of tile × weight divided by the sum of weights, so any set of
//! overlapping tiles blends consistently and a seam between two tiles of slightly different scale is a smooth ramp.
/// Where the tiles start along one axis of `n` cells: `[0]` when the axis fits one tile (the tile is then padded by
/// mirroring), else evenly spaced starts, the first at 0 and the last at `n − tile`, each tile sharing at least
/// `overlap` cells with the next.
pub fn starts(n: usize, tile: usize, overlap: usize) -> Vec<usize> {
if n <= tile || tile == 0 {
return vec![0];
}
let step = tile.saturating_sub(overlap).max(1);
// tiles needed so that start k·spacing with spacing ≤ step reaches n − tile
let Some(k) = (n - tile).div_ceil(step).checked_add(1).filter(|&k| k <= 1_000_000) else { return Vec::new() };
let last = n - tile;
(0..k).map(|i| if k == 1 { 0 } else { ((i as u128 * last as u128 + (k - 1) as u128 / 2) / (k - 1) as u128) as usize }).collect()
}
/// Weight of the cell at offset `p` (0-based, inside a tile of `tile` cells) along one axis: 1 in the middle, easing
/// down to a sliver over `fade` cells at an edge that has a neighbour, 1 right up to an edge that is the picture's.
pub fn weight(p: usize, tile: usize, fade: usize, has_before: bool, has_after: bool) -> f32 {
let fade = fade.max(1) as f32;
let ramp = |d: usize| {
let t = ((d as f32 + 0.5) / fade).min(1.0);
// smoothstep, never exactly 0 so that a cell is always covered
(t * t * (3.0 - 2.0 * t)).max(1e-3)
};
let before = if has_before { ramp(p) } else { 1.0 };
let after = if has_after { ramp(tile.saturating_sub(1).saturating_sub(p)) } else { 1.0 };
before.min(after)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn hostile_axes_have_bounded_tile_counts_and_checked_spacing() {
assert!(starts(usize::MAX, 1, 0).is_empty());
assert!(starts(usize::MAX, 2, usize::MAX).is_empty());
let s = starts(usize::MAX, usize::MAX / 2, 0);
assert_eq!(s.first(), Some(&0));
assert_eq!(s.last(), Some(&(usize::MAX - usize::MAX / 2)));
assert!(s.windows(2).all(|s| s[0] < s[1]));
}
#[test]
fn one_tile_when_it_fits() {
assert_eq!(starts(100, 512, 64), vec![0]);
assert_eq!(starts(512, 512, 64), vec![0]);
assert_eq!(starts(0, 512, 64), vec![0]);
assert_eq!(starts(10, 0, 0), vec![0]);
}
#[test]
fn tiles_cover_the_axis_and_overlap_enough() {
for n in [513usize, 600, 900, 1000, 1024, 1500, 3024, 3600, 5000, 12345] {
let s = starts(n, 512, 64);
assert_eq!(s[0], 0, "n={n}");
assert_eq!(*s.last().unwrap(), n - 512, "n={n}");
for w in s.windows(2) {
assert!(w[1] > w[0], "starts increase, n={n}");
assert!(w[0] + 512 >= w[1] + 64, "at least 64 shared cells between {} and {}, n={n}", w[0], w[1]);
}
}
}
#[test]
fn weights_fade_only_towards_neighbours() {
let (t, f) = (512, 64);
// a lone tile: full weight everywhere
assert!((0..t).all(|p| weight(p, t, f, false, false) == 1.0));
// the first tile fades on its far side only
assert_eq!(weight(0, t, f, false, true), 1.0);
assert!(weight(t - 1, t, f, false, true) < 0.01);
// the last tile fades on its near side only
assert!(weight(0, t, f, true, false) < 0.01);
assert_eq!(weight(t - 1, t, f, true, false), 1.0);
// a middle tile is 1 in the middle and everywhere positive
assert_eq!(weight(t / 2, t, f, true, true), 1.0);
assert!((0..t).all(|p| weight(p, t, f, true, true) > 0.0));
}
#[test]
fn two_overlapping_tiles_blend_to_a_partition_of_unity() {
// at every cell of the shared region the normalised weights are in [0, 1] and sum to 1
let (t, f) = (512usize, 64usize);
let s = starts(900, t, 64);
assert_eq!(s.len(), 2);
for cell in s[1]..s[0] + t {
let a = weight(cell - s[0], t, f, false, true);
let b = weight(cell - s[1], t, f, true, false);
let (na, nb) = (a / (a + b), b / (a + b));
assert!((na + nb - 1.0).abs() < 1e-6 && na >= 0.0 && nb >= 0.0);
}
}
}
+32
View File
@@ -0,0 +1,32 @@
[package]
name = "lightcraft-denoise"
version.workspace = true
edition.workspace = true
license.workspace = true
description = "LightCraft AI denoise (L1): denoise-model manifests, Bayer packing, tiling and blending, the cache file, and pure-Rust CPU inference"
[lints]
workspace = true
[features]
default = []
runtime = ["dep:faer"]
# Product-policy switch: absent by default; users may install their own model.
rawnind-model = ["lightcraft-denoise-core/rawnind-model"]
[dependencies]
lightcraft-denoise-core = { workspace = true }
# Safe matrix views over the pure-Rust GEMM backend; no BLAS, cc/build scripts
# or inner thread pool. Its dependencies include architecture-specific assembly.
faer = { package = "faer-core", version = "=0.17.1", default-features = false, features = ["std"], optional = true }
lightcraft-raster = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
sha2 = { workspace = true }
thiserror = { workspace = true }
miniz_oxide = { workspace = true }
rayon = { workspace = true }
web-time = { workspace = true }
[dev-dependencies]
proptest = { workspace = true }
+18
View File
@@ -0,0 +1,18 @@
# lightcraft-denoise
L1 pure-Rust AI RAW denoise primitives and CPU inference. No weights are bundled.
See [user workflow and limitations](../../docs/denoise.md).
`lightcraft-denoise-core` provides metadata, Bayer helpers and cache formats,
re-exported here. `manifest` and `known` validate opt-in models; `onnx` reads a bounded operator subset
into `net`; `cpu` executes it and `reference` supplies scalar checks. `bayer`, `tiles`
and `run` pack sensor data and blend overlapping tiles while preserving clipped
highlights. `product` reads/writes the disposable `.lcdn` cache. `runtime` supplies
the installation self-test. Synthetic networks are generated in Rust for tests.
Run `cargo test -p lightcraft-denoise --features runtime`. The CPU backend is optional;
featureless and web builds keep the model metadata and cache format without faer/GEMM.
The `rawnind-model` feature enables the otherwise empty download catalog and requires
an explicit release-policy decision. Real-model tests are ignored by default and
require an explicitly supplied local model. The GPU twin is `lightcraft-gpu::nn`;
model management, background jobs and export integration live in `lightcraft-engine`.
Loaded 100 of 496 files, more files were not shown because too many files have changed in this diff. Show more