Compare commits

...
Author SHA1 Message Date
Brandon Thomas ae2f06a035 Merge PR #359 (fmt) 2026-10-09 00:20:30 -04:00
Brandon ThomasandClaude Opus 5.5 b341101ab6 Remote SAM3 worker: cargo fmt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 00:20:28 -04:00
Brandon Thomas 7365d06c13 Merge PR #359 2026-10-09 00:17:59 -04:00
Brandon Thomas ab73d3d2e0 Merge PR #423 2026-10-09 00:17:58 -04:00
Brandon Thomas c4c96ecabb Merge pull request #439 from storytold/claude/integrate-batch-14
Integrate #429, #426, #422, #395, #172, #173, #174, #335, #427
2026-10-09 00:16:54 -04:00
Brandon Thomas cae7bea08e Merge remote-tracking branch 'origin/main' into fix-359
# Conflicts:
#	crates/engine/src/segment/worker.rs
2026-10-08 23:57:36 -04:00
Brandon Thomas 24615cc2d7 Merge PR #335 (clippy fix) 2026-10-08 23:57:23 -04:00
Brandon ThomasandClaude Opus 5.5 d0749ed7a9 HEIC: as_chunks for 16-bit samples (clippy 1.99, caught by the new heif CI step)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:57:22 -04:00
Brandon ThomasandClaude Opus 5.5 bb7c46ae3c Remote SAM3 worker: crate lint attributes; document the trust boundary
The worker binary gets the never-crash deny list and forbid(unsafe_code) like
every production crate root. docs/ai-masks.md spells out who can reach the
unauthenticated loopback listener.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:55:26 -04:00
Brandon ThomasandClaude Opus 5.5 7db3b8a1f9 Batch 14: one [features] table in apps/lightcraft (faces + heif), parity summary
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:53:54 -04:00
Brandon Thomas a628d5bc38 Merge PR #427
# Conflicts:
#	ROADMAP.md
#	docs/parity.md
2026-10-08 23:46:58 -04:00
Brandon Thomas 42f966eec2 Merge PR #335 2026-10-08 23:46:17 -04:00
Brandon Thomas ca97329ae1 Merge PR #174 2026-10-08 23:45:49 -04:00
Brandon Thomas 3368eb49ee Merge PR #173 2026-10-08 23:45:49 -04:00
Brandon Thomas cbf9935b6e Merge PR #172
# Conflicts:
#	docs/parity.md
2026-10-08 23:45:49 -04:00
Brandon ThomasandClaude Opus 5.5 4977c648f6 Merge main into formats/heif-heif-decode; CI gates the heif feature
Resolve the docs/parity.md conflict, keep the HEIC rows. `cargo xtask ci` gets
a heif step (clippy + tests for lightcraft-codecs with --features heif), since
the workspace build leaves the feature off. The Cargo.toml comments no longer
say official builds enable it; the web one notes that wasm32 aborts on panic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:24:28 -04:00
Brandon Thomas 91d317900f Merge PR #395 2026-10-08 23:22:56 -04:00
sanyamsingh001andCodex 56754e5703 LR-BEHAV-HEADLESS: keep CPU snapshots isolated from saved preferences
Co-authored-by: Codex <codex@openai.com>
2026-10-09 13:19:47 +10:00
Brandon Thomas 1a2d2ab975 Merge PR #422 2026-10-08 22:43:10 -04:00
Brandon Thomas 1f30856774 Merge PR #426 2026-10-08 22:43:10 -04:00
Brandon Thomas 05769798f6 Merge PR #429 2026-10-08 22:43:10 -04:00
Daniel Voshart 111cc893ec LR-LIB-PEOPLE: preserve later person navigation on current main
Port the original Escape return and last-person shortcut behavior, keeping current translations and a no-wrap chip. The existing headless regressions and all seven quality gates pass with the hosted fonts on the refreshed recognition dependency chain.

Generated with Codex.
2026-10-08 22:29:31 -04:00
Daniel Voshart 24f87a91a0 LR-PREF-PEOPLE: update pure-Rust recognition and preserve valid model replacements
Keep current upstream UI behavior and the later detector contract checks. Validate candidate models before replacing working weights and records, with a synthetic regression that failed before the fix. Remove the unrelated GPU equivalence-test relaxation. All seven quality gates pass with the pinned hosted fonts.

Generated with Codex.
2026-10-08 22:29:31 -04:00
Daniel VoshartandClaude Opus 5.5 20a801d6d2 Faces: YuNet face detection and one-click model downloads, with nothing bundled
Photo > Detect Faces runs the YuNet detector (a small pure-Rust interpreter over the ONNX graph, no
runtime dependency) and adds the faces it finds as unnamed face regions in one undo step. The detector
and the recognition models are opt-in downloads: Settings > Faces shows each model's licence and
training-data note, Download stays disabled until the terms are accepted, and the file is fetched over
https with lightcraft-fetch from the model's own repository at a commit-pinned address, checked against
its recorded size and SHA-256 and then installed. Detect Faces offers the detector's download the first
time it is used. No model file is part of the repository (the real-model tests read LC_YUNET_MODEL; two
ignored tests download from the real hosts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 22:29:30 -04:00
Daniel Voshart 03b665e7a3 LR-PREF-PEOPLE: refresh the generated parity summary
Generated with Codex.
2026-10-08 22:29:29 -04:00
Daniel Voshart 6e8db91673 LR-PREF-PEOPLE: refresh model manager for current platform helpers
Keep upstream tests and the fixed SAM worker, refresh the workspace version and parity summary, and route the model picker through the case-insensitive native filter helper. Carry the shared translated font-symbol correction; validation includes the pinned hosted fonts and all seven CI gates.

Generated with Codex.
2026-10-08 21:59:02 -04:00
Daniel VoshartandClaude Sonnet 5.5 7845d515e8 Faces: nothing is bundled, so YuNet is not reported as installed
The model list marked the YuNet detector as bundled with LightCraft and installed, but no model ships
inside LightCraft (the weights are opt-in downloads or files the user adds). Drop the notion: a
model is installed when its folder is, it can be removed like any other, and the licence notice no
longer says it is bundled.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:59:02 -04:00
Daniel VoshartandClaude Sonnet 5.5 cb34bb4842 Faces: Settings > Faces tab and the add-a-model licence dialog
An off-by-default "Recognise faces" toggle and the face model list in Settings,
with a one-drop install: pick a .onnx (Add a model file…) or drop one on the
window. The engine inspects it (recognised by hash, or described from its
shape), a dialog shows its licence, whether commercial use is allowed, what it
was trained on and what LightCraft had to assume, and Install stays disabled
until "I accept" is ticked. "Get…" opens a model's own page in the browser:
LightCraft never fetches a model itself. A file it cannot use says why.

The models folder is shared by the app, the CLI and the MCP server
(`engine::config`, `$LIGHTCRAFT_FACE_MODELS` overrides `<config>/models`), so a
model installed from one is there for the others. A session has none until a host
asks (`with_default_face_models`), which keeps tests hermetic.

Nothing runs a model yet, and the tab says so. Headless tests cover the whole
flow (the dialog, refusing until accepted, installing, a file it cannot use).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:59:01 -04:00
Daniel VoshartandClaude Sonnet 5.5 7ca7113b32 Faces: model commands - list, inspect, install behind the licence, select, remove, on/off
`faces.models.list / inspect / install / remove / select` and `faces.enable`, all
reachable from the UI, CLI, control channel and MCP. Models live in a folder the
host provides (`Session::face_models_dir`): one subfolder each with the model, its
manifest and a record of what the user accepted, plus settings.json.

- install needs `acknowledged: true` (the user has been shown the licence), copies
  the file under a .part name, checks the copy's SHA-256 against the original, then
  renames; a failure leaves nothing behind.
- ids are validated before they become folder names (and a manifest that disagrees
  with its folder is ignored); remove only ever deletes a folder that has a model
  manifest. A manifest id can no longer be `.`, `..` or start with a dot.
- inspect describes any .onnx without writing anything; unusable files are an
  answer ("unsupported" with the reason), not a crash.
- the list says plainly that running a model is not built in yet.

`faces::synthetic` makes a minimal valid ONNX for tests (and the later self-check).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:59:01 -04:00
Daniel VoshartandClaude Sonnet 5.5 39de0e6a1c Faces: model manifests, known-model table and an ONNX probe (no runtime yet)
New L1 crate `lightcraft-faces`, the base of a one-drop model install:
- `ModelManifest` (role, licence with a commercial yes / no / unknown flag and a
  plain-language notice, provenance, input preprocessing, output, thresholds)
  with strict validation: sizes capped, numbers finite and sane, https-only
  addresses, bounded text. A manifest from any source is untrusted.
- `known`: models recognised by SHA-256 (YuNet, SFace, AuraFace v1), with what is
  and is not known about their licences and training data.
- `onnx`: a streaming, hostile-input-safe protobuf probe of a model's graph
  inputs and outputs. It reads a 248 MB model in about 4 ms and never loads the
  weights; truncation at every length and random corruption never panic.
- `suggest`: known hash -> its manifest; an unknown model shaped like a face
  embedder -> a draft manifest with the assumptions spelled out; anything else
  -> a plain reason it is not supported yet.
- `hash`: streamed SHA-256.

Checked against the real model files with an opt-in test (LC_FACE_MODELS). It
found an old exporter listing every weight as an input, now handled. Running a
model (tract) is not part of this crate yet.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:59:00 -04:00
Exalted5373andClaude Opus 5.5 3c3b527b7f Web: the window wire test allows the CPU/GPU difference
a_window_job_crosses_the_wire_and_renders_the_same_window compared the worker's
CPU render with the engine job's render bit for bit. On a machine with a GPU the
engine job renders there, so the test failed by 1 LSB (on main too); without a
GPU (CI) both run on the CPU. It now checks both sizes and a maximum channel
difference of 3, the bound docs/gpu-pipeline.md documents.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:33:43 -07:00
radialmonster 3d9f8492da LR-IMP-FORMATS: Coolpix NRW black level from the NRW data block
In NRW data version 0104 the u32 at byte 0x20 is the black level in sample units: 200 where the darkest samples of the B700, P950, P1000 and P1100 sit, 0 for the A1000 whose samples reach 0. Used when the maker note has no 0x003d.
2026-10-08 21:26:26 -04:00
radialmonster a12f1697a1 LR-IMP-FORMATS: decode 12-bit raws packed in 32-bit little-endian words (Fujifilm RAF flag 0xf006, Coolpix NRW data block)
Raw-IFD flag 0xf006 = 1 marks RAF samples packed MSB-first into 32-bit little-endian words at any depth; 12-bit files with it (FinePix S1, X-A10, SL1000) were read as an LSB-first stream. Nikon uncompressed 12-bit strips of exactly 1.5 bytes per pixel in a file whose maker note has an NRW data block (COOLPIX B700, P950, P1000, P1100) use the same words in file byte order; without the block (E5700, E8400, E8800) they stay an MSB-first byte stream. Shared unpack_words32_msb and Packing::Words32Msb. RENDER_CACHE_VERSION 15 to 16.
2026-10-08 21:26:26 -04:00
Exalted5373andClaude Opus 5.5 0ac5cf4430 LR-EDIT-COLOR-TEMP: camera-space white balance for raws with a colour matrix
A custom white balance on a raw with a colour matrix (DNGs, and other raws
without a file-local look) re-develops the pixels for the chosen white
through the file's own colour model: lightcraft_raw::color::rebalance
re-evaluates the camera transform at the new white and maps the as-shot
development onto it, so a neutral lit by that white renders neutral and
saturated colours move as the camera's matrices say (DNG spec ch. 6, as
Lightroom does). The decoder passes the colour tags and the white the
pixels were developed for in SourceInfo::camera_color; other photos, and
relative white balances, keep the Bradford adaptation. The eyedropper and
Auto invert the same model (color::neutral_white), so a picked grey card
renders neutral.

The Tint sign is main's (#349): unchanged here. Render cache version 15.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:18:54 -07:00
Exalted5373andClaude Opus 5.5 ce7b8f511e LR-IMP-FORMATS: read DNG BaselineSharpness and keep it in DNG exports
ColorData::baseline_sharpness holds the file's BaselineSharpness (raw IFD
or IFD 0; zero, negative, non-finite or above 16 is ignored) and the DNG
writer writes it back, so a conversion or smart preview keeps it. Merge
outputs leave it unset. Rendering doesn't use it yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:18:54 -07:00
Exalted5373andClaude Opus 5.5 ed59d78e83 LR-MASK-SKY: AI masks use DNG semantic mattes (iPhone ProRAW sky matte)
Read DNG 1.6 semantic masks (SemanticName, SemanticInstanceID,
MaskSubArea; any coding the tile/strip reader decodes, e.g. ProRAW's
lossy-JPEG matte) with lightcraft_raw::semantic_masks, map them onto the
developed image, and carry Apple's sky / portrait / skin / hair / teeth /
glasses mattes in SourceInfo::mattes (SourceInfo is no longer Copy).
Sky, Subject, Background and People components sample them through
orientation, warp (Warp::frame, as geometry.rs) and crop like the image,
so preview = export, instead of the heuristics; files without mattes
render as before. Malformed mask IFDs are skipped, and only the first
MAX_MASKS mask IFDs are decoded, so a file full of them costs bounded
work. The render cache version is bumped once, in the white-balance commit.

The corpus test reads corpus/raw/dng-apple-iphone12pro-proraw.dng, the
name `cargo xtask corpus` downloads it under.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:18:54 -07:00
Exalted5373andClaude Opus 5.5 a28dc4dbdf LR-PROF-CAMERACOLOR: read and keep DNG ProfileGainTableMap (not rendered)
Parse ProfileGainTableMap (DNG 1.6) and ProfileGainTableMap2 (DNG 1.7):
both versions, DNG 1.7 precedence (version 2 in IFD 0, then the raw IFD;
version 1 in the raw IFD, then IFD 0), u8/u16/f16/f32 gains, gamma.
Malformed tags (wrong size, zero or huge grids, non-finite or negative
gains, bad spacing or gamma) are ignored, never allocated for.

Byte order: the specification gives this tag no byte order of its own,
unlike opcode lists, so the payload is read in the file's byte order with
a big-endian fallback; a wrong-order read can't pass the size checks.
The module docs quote the DNG 1.6 "Byte Order" rule.

The map is kept in ColorData, written back by the DNG writer (version 1,
or version 2 when its gamma isn't 1) and dropped from merge outputs, whose
geometry no longer matches the capture's active area. It is not applied
to the render: Lightroom Classic renders Apple ProRAW without it.
Supersedes the parser of #273.

The CC0 iPhone 12 Pro ProRAW from raw.pixls.us joins the corpus.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:18:16 -07:00
Exalted5373andClaude Opus 5.5 1ed80c65bf LR-IMP-FORMATS: decode JPEG XL compressed DNG tiles (DNG 1.7)
Compression 52546 tiles and strips decode through jxl-oxide (new
default-on `jxl` feature of lightcraft-raw): bare codestreams and
ISO-BMFF containers, 1 or 3 planes, 8-16-bit integers scaled back
exactly from the codestream's own bit depth, 16-bit floats. Edge tiles
may be coded at their nominal or in-image size. Headers are checked
against the IFD in header-only probing too. A single-chunk JPEG XL
preview IFD (RGB or grey) is offered as the embedded preview.

The decoder's memory is bounded by the tile: 16 bytes per sample (three
times that for a lossy grey tile, which the decoder expands to three
channels) plus 64 KiB; jxl-oxide 0.12 was measured at <= 10.3 bytes per
sample for lossless and <= 13.5 per colour sample for lossy tiles. A
codestream needing more (e.g. extra channels) is an error.

Lossless tiles are checked bit-exact on synthetic CFA and LinearRaw
files (zune-jpegxl as a test-only encoder); lossy tiles are unverified
on a real file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:18:16 -07:00
BFlat b1883231bd Merge pull request #424 from storytold/claude/integrate-batch-13
Integrate #403, #411, #412, #414, #415, #416
2026-10-09 08:15:07 +07:00
Nathaniel LedfordandClaude Sonnet 5.5 733846b326 Docs: sidebar auto-scroll while dragging
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 8ff45f5237 Sidebar: dragging albums or photos near its top or bottom edge scrolls it
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 5fefe91640 Docs: top-level drop is on the Albums header
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 24b80b4ee3 Albums: the Albums header is the way back to the top level
It replaces the strip under the tree, which could be scrolled out of view
and shifted the rows when it appeared; the header is always visible and
nothing moves.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 922ea25d88 Docs: album tree, drag and hand order (parity, what's new)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 085c25b10b Albums: drop beside a row to place it by hand; Sort A–Z goes back to by name
Dropping on the edge of a row places the album there (album.reorder) with
a line showing where; a folder's middle still means inside. A closed folder
opens only when the drop would go inside it. Sort Contents A–Z and Sort
Albums A–Z (album.sort) appear once a folder is ordered by hand.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 8036f98aca Catalog: tidy a test assertion
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 34a4b61f7e Albums: one-pass children map, before-itself is a no-op, more order tests
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 2c5cb708bb Engine: album.reorder and album.sort
Place an album or folder before a sibling of its kind (also in another
folder), or last of its kind, in one undo step; album.sort goes back to
by name for one folder.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 39f5f114de Catalog: albums can be ordered by hand (format version 3)
Album.order and Op::SetAlbumOrder; album_children lists folders first, then
by place, then by name. Moving an album drops its place and undo restores it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 67aed0b4f4 Albums: a drag ends with the main button, Esc cancels it, hidden rows are no targets
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 63335e2b0a Parity: albums and folders move by dragging onto a folder
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 aaae39fae0 Albums: drag an album or folder onto a folder to move it
A folder under the pointer is outlined when the drop is allowed and opens
after resting there; a drop zone under the tree takes an album back to the
top level. The move is album.move, so it is one undo step.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:07 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 5443ae28f9 Albums: address review of folder creation
Show a dialog's error and keep it open instead of closing it, open the
folder menu from the triangle too, draw a triangle only on folders that
hold something, reveal even when the section is folded, reuse the
translated Create … labels, and fix a misplaced doc comment.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:06 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 4abf70e254 Parity: album folders fold like the other trees and create inside themselves
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:06 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 79403a5680 Albums: create albums, smart albums and folders inside a folder from its context menu
The dialog.new* commands take the folder to create in, the dialogs
remember it, and the Albums tree opens the folders down to the new row.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:06 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 96a9c6e04d Albums: folders get the same disclosure triangle as the other sidebar trees
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:06 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 e8b18ae636 Sidebar: one disclosure-triangle helper for the Local, By Date, Folders and Keywords trees
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:06 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 a46c0071a0 Albums: pin that albums are created only inside folders
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:02:06 -04:00
Claude 12188161f9 Merge remote-tracking branch 'origin/main' into integ-b13 2026-10-09 00:46:05 +00:00
BFlat de79310ae0 Merge pull request #419 from storytold/claude/integrate-batch-12
Integrate #351 (sharp zoomed Detail view, #323) with follow-ups
2026-10-09 07:45:11 +07:00
radialmonster caa102aabd MENU-BAR: pointing at another menu title switches the open menu in the in-window bar 2026-10-08 20:40:46 -04:00
Claude 373efc280e Merge PR #416 2026-10-09 00:20:27 +00:00
Claude c75389dda1 Merge PR #415 2026-10-09 00:20:27 +00:00
Claude 8ee6d63d41 Merge PR #414 2026-10-09 00:20:27 +00:00
Claude f578b27273 Merge PR #412 2026-10-09 00:20:26 +00:00
Claude 43419dcfb8 Merge PR #411 2026-10-09 00:20:26 +00:00
Claude 83ea5e14df Merge PR #403 2026-10-09 00:20:26 +00:00
Claude eab88ea30c Follow-up to #351: drag drafts without a window stay canvas-sized; de/es/ru hint
- With no window (a mask or point-colour overlay, Visualize Spots, soft
  proofing, a window refused for a far read) the loupe renders the whole
  frame at the size it is drawn. While a slider drags that meant the
  photo's every pixel each draft frame at 1:1 and beyond (24 MP per tick
  where 2.5 MP was rendered before #351; mask overlays are on by
  default). A drag now drafts at most the canvas size, as the windowed
  path does; at rest the zoomed frame is rendered as before.
- The Preview size hint in German, Spanish and Russian; the replaced
  'Largest long edge…' text removed from those catalogs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-09 00:16:01 +00:00
Claude 3e88a60dd6 Merge PR #351 2026-10-09 00:14:03 +00:00
ragtuxandClaude Fable 5.1 0cf3ceb438 Part of #260: Help ▸ Open Log Folder shows the log file in the file manager
The desktop app keeps logs/lightcraft.log in its settings folder (#286)
and a failed start names it (#370), but a user still had to find the
folder by hand. The host now hands the UI its log file (Services
.log_file) and Help ▸ Open Log Folder (app.openLogFolder) reveals it
through the existing reveal service: Finder and Explorer select the
file, Linux opens the folder. Off where no log is kept (the web,
--memory, LIGHTCRAFT_NO_PREFS). Translated in de, es, ja, pt-br, ru,
zh-hans and zh-hant; parity row MENU-HELP-LOGFOLDER; What's New and
README note it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 19:13:35 -05:00
BFlat bba4172f83 Merge pull request #413 from storytold/claude/integrate-batch-11
Integrate #407 (file dialogs off the UI thread) and #408 (macOS top-bar drag and zoom)
2026-10-09 07:06:20 +07:00
midasdf 93b900f8d7 M16: keep grapheme clusters intact in vertical watermarks 2026-10-09 09:02:24 +09:00
midasdf 0cb78071de M16: use paintable separators in localized UI text 2026-10-09 08:55:21 +09:00
Claude fec379db46 Russian for two strings the catalog lacked (colour label cleared, slideshow hint)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 23:47:41 +00:00
Claude 4d2f34fb35 Merge PR #408 2026-10-08 23:47:24 +00:00
Claude b306305472 Merge PR #407 2026-10-08 23:47:24 +00:00
BFlat 832b8ada51 Merge pull request #409 from storytold/claude/docs-374-nvidia-cache
Fix #374: troubleshooting for the NVIDIA shader-cache startup hang
2026-10-09 06:46:22 +07:00
Claude bf97105cf7 Fix #374: troubleshooting for the NVIDIA shader-cache startup hang
With Shader Cache Size disabled in NVIDIA Control Panel, the driver walks
the whole system drive inside D3D12CreateDevice (traced by the reporter
to NvMemMapStoragex.dll!TotalDiskUsage), before the window's first
frame; Driver Default fixes it (confirmed in #374, PhotoCraft too). The
launch-argument walk found on the way was fixed in #394.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 23:46:07 +00:00
Nathaniel LedfordandClaude Sonnet 5.5 eaaf859853 Parity: title-bar drag and zoom confirmed on a real macOS window
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:44:23 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 2bd13c1500 Parity: note the stale zoomed-state limitation of the title-bar toggle
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:42:52 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 cc8d617459 Top bar: a press that starts on a button or the search field never drags the window
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:42:52 -04:00
ragtuxandClaude Fable 5.1 d3ceaf0e79 Fix #191: commands' file dialogs run off the UI thread
The desktop showed rfd's synchronous dialogs on the UI thread, so the
window stopped answering the compositor while one was up: GNOME marked
LightCraft "not responding", the File menu stayed drawn and the control
channel went quiet. Services.picker lets the host start a dialog
elsewhere and answer through a channel; a command that needs a path
asks for one and returns, and runs again with the answer when the
dialog closes (pick.rs). Import Photos, Import from Folder, Open
Library, Import Lightroom Catalog, Find Missing Photos, the preset and
curve-preset imports and exports, Auto-Tag from Tracklog and Locate
Missing File use it; hosts without a picker (the web, tests) keep the
synchronous dialogs. ui.inspect lists the waiting commands as
fileDialogs. Same cause as storytold/photocraft#574.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 18:42:19 -05:00
BFlat 32cedb3153 Merge pull request #405 from storytold/claude/integrate-batch-10
Fix #310 (CR2/PEF starting look) and integrate #402 (menus stay below the menu bar)
2026-10-09 06:39:08 +07:00
Nathaniel LedfordandClaude Sonnet 5.5 86d88de2ee Parity: track title-bar drag and double-click zoom
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:37:55 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 d01ebd7953 Top bar: drag to move and double-click to zoom the window (integrated title bar)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:37:05 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 54b037a90a Headless host: record window commands and report the zoomed state back
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:37:05 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 1e27991cf1 Title bar: window-gesture policy (drag, double-click zoom, full-screen exempt)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 19:35:36 -04:00
Exalted5373andClaude Opus 5.5 7173242ab5 LR-IMP-FORMATS: no overflow deciding whether to scale a full-size JPEG decode
A full-size load asks the decoder for the largest box (`SourceLevel::Full` is
`usize::MAX`, cast to `u32::MAX`). `jpeg::decode` then computed `2 * u32::MAX` to
decide on DCT scaling: an overflow panic in debug builds (caught, so every
full-size JPEG load failed with "decoder panicked"), and a wrap that happened to
give the right answer in release. The comparison saturates now, and the engine
converts its `usize` edge with `u32::try_from` instead of a truncating `as`.

Found decoding camera JPEGs (no JFIF/EXIF header) at full size.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:26:29 -07:00
Exalted5373andClaude Opus 5.5 31aa638383 LR-IMP-PROFILES: red / green / blue curves only with the master curve; neutral test values
Lightroom Classic reads `ToneCurvePV2012Red/Green/Blue` only together with
`ToneCurvePV2012`: a packet with a channel curve and no master curve leaves the
photo unchanged there (rendered on a chart: the channel curve alone has no
effect, with an identity master curve it applies). Imports now do the same.

The preset-packet test helper uses made-up as-shot values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:25:25 -07:00
Exalted5373andClaude Opus 5.5 09d5284e5a LR-EDIT-OPTICS-PROFILE: a preset's LensProfileEnable=0 keeps embedded lens corrections
XMP presets usually say LensProfileEnable=0, which turned optics.lens_profile
off and with it the DNG-embedded WarpRectilinear / FixVignetteRadial
corrections (iPhone ProRAW and other files that carry them). Lightroom
applies a file's built-in corrections whatever that switch says (our
inference), and LightCraft's profile corrections are only those embedded
ones, so crs import now carries over only an enabling switch. The Enable
Lens Corrections checkbox and the amount sliders keep working as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:25:25 -07:00
Exalted5373andClaude Opus 5.5 7e0250e11e LR-IMP-PROFILES: don't report preset bookkeeping fields as unmapped
XMP presets carry fields that describe the preset or the photo rather than
an adjustment (Cluster, SupportsAmount2, RequiresRGBTables, SortName,
Description, CropConstrainToWarp, OverrideLookVignette, AsShotTemperature /
AsShotTint). preset.import listed them as settings that couldn't be carried
over; so did an HDREditMode of 0 and PointColors slots that are all -1.
They are now treated as bookkeeping; an HDR edit, a used Point Color slot and
unknown adjustments are still reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:25:25 -07:00
Exalted5373andClaude Opus 5.5 9087d3ad7f PSD: as_chunks for the palette and 16/32-bit plane loops (clippy 1.98)
clippy 1.98's manual_chunks lint fails `cargo xtask ci` on main at psd.rs:170/191/201
(chunks_exact with a constant size). The fixed-size chunks also drop the per-sample
indexing. No behaviour change; the codecs tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:25:17 -07:00
Claude a1e99fad9e Follow-up to #402: the menu scroll arrows' accessibility labels are translated
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 23:13:39 +00:00
Claude ff9d74f4e5 Merge PR #402 2026-10-08 23:13:27 +00:00
Claude 55ecd900ae Fix #310: CR2 and PEF raws get the starting look fitted to their camera JPEG
CR2 and PEF decode without a camera colour matrix, like ARW, NEF, RW2,
RAF and CR3, but camera_preview::file_local_look (and the catalog's
relative white balance) left them out, so they rendered with the neutral
fallback matrix: the camera preview showed first, then a flat,
desaturated render replaced it. They now use the same guarded fit and
relative WB; a fit that fails the gates keeps the fallback, so a file can
only stay as it was or move closer to its camera JPEG. Not measured on
the corpus here (raw.pixls.us is blocked from this environment).
RENDER_CACHE_VERSION 15 re-renders cached CR2/PEF thumbnails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 23:13:16 +00:00
ragtuxandClaude Fable 5.1 2d47ba3207 Fix #189: in-window menus and submenus stay below the menu bar
egui keeps a popup inside the window, so a menu taller than the room
under the bar (Photo, on a short window) was slid up over the bar: the
titles vanished under it, and the pointer left on the title rested on
the first row and opened its submenu by itself. Every level now gets
the room it has below the bar (a submenu: from its row down, or up to
the bar when egui opens it upward) and scrolls inside it, with wheel,
drag and ▲/▼ strips that scroll while the pointer rests on them, as in
PhotoCraft's menu bar (storytold/photocraft#337). Menu levels and
submenu rows are registered for the control channel and tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 17:50:18 -05:00
BFlat 7b47ba7894 Merge pull request #401 from storytold/claude/integrate-batch-9
Header-only import probes (part of #367), PSD RLE allocation bounds, and #397 (CR3 C-RAW clipped highlights)
2026-10-09 05:48:39 +07:00
Nathaniel LedfordandClaude Sonnet 5.5 826ba91075 #323: docs: measured drag times, the Before/After memory, what stays canvas-sized
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:48:20 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 22b7ba09b9 #323: stage budget: the open photo's views are cut only in a runaway; the shared source is reported
A review measured 1 GB of GPU stages for a Before/After at 1:1 on a 2800 px
canvas against a 400 MB budget, with nothing bounding it. The open photo's window
and before caches are now cut (the before side first) past four times the budget,
the whole-frame view's never; idle caches still go at the budget. The one shared
device copy of a big original is reported (sharedSourceBytes in the renderer's
memory report) and a misleading comment about it is corrected.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:48:19 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 057a8d308e #323: the Before window waits for the original the After is decoding
At first open of Before/After the two window jobs were asked for together, each
decoded its own 24 MP copy of the original (a review saw two decodes and three
uploads for one photo) and the shared device copy was swapped between them. The
Before's window now waits until the After's job has handed the original to the
session (region::defer_before_window, Session media has_source); later windows
of both share the one copy.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:38:39 -04:00
Claude 1b5ba5eba7 Merge PR #397 2026-10-08 22:34:54 +00:00
Nathaniel LedfordandClaude Sonnet 5.5 ddaea580a1 #323: test: holding \ in a wipe is tested with the key held, and the mutation fails it
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:34:02 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 03249f4619 #323: holding \ in a wipe shows the Before window everywhere
The window of the Before view alone was clipped to the wipe's left half while the
picture drawn was the Before on the whole canvas, leaving the right half the
low-resolution stand-in. The clip follows the effective mode.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:32:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 495b1ac9fb #323: tests: the wipe layering and pane clipping are checked on the painted shapes
A review showed the wipe test asserted on a hand-written list of labels (deleting
the stand-in draw calls still passed) and nothing tested the clip at the call
site (replacing it with the canvas still passed). The tests now read the meshes
the frame painted: the Before window comes after the stand-in and stays inside
its side of the line, and side by side each window is painted inside its pane.
Both mutations of the code now fail them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:31:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 f1a513ffe2 #323: docs: drag timings on two canvases, what a fit view decodes, the shared source
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 a5b9dd603f #323: the drag profile covers a 2800 x 1700 canvas
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 a94b647dfd #323: the open photo's views are never trimmed from the stage budget
A review measured the drag at 100 % on a 2800 px canvas taking 87 ms a tick
against 24 ms with a bigger budget: the open photo's caches were over the budget
(a 5 MP window alone is 230 MB) and a "hard" budget cleared them every frame, so
every tick rebuilt them. The whole-frame, zoom window and before caches are what a
drag reuses; only idle caches (hover, compare tiles, second window) are trimmed
now, the least useful first, and the open photo's are counted and reported. The
drag test no longer waits on a tick count.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 f68cd69b1e #323: the windows of a big photo share one device copy of its source
Keeping a 288 MB upload with each view's stages made them thrash against the stage
budget; not keeping it made every new window (a pan crossing a snap, the first
window of the Before side, any control that invalidates the sampled pixels) upload
and convert the source again, about 64 ms against 6-12 ms. A source too big for a
view's stages is now kept once for all views of the photo (shared_source), outside
the views' stages, and given back when the app idles (trim_pool(0)).
source_uploads and shared_source_bytes expose it for tests and diagnostics.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 e9fb4b08e1 #323: the render plan: drags keep the window, modes without windows render full size, limits count
Review findings on region::plan:
- a drag at fit on a 4K or 5K display switched the window off (the whole-frame
  draft shrinks, the preview-source cap does not): the window decision now
  compares against the undrafted whole-frame render;
- overlays, soft proofing and windows refused for reading too far (a spot or Auto
  Mask stroke) left the 1:1 view a canvas-sized upscale: those modes render the
  whole frame at the size it is drawn, as before windows (a refusal is remembered
  per photo, look and frame, and retried when any changes);
- the 3840 and 5120 preview sizes had become no-ops: a limit above the preview
  source level is honoured;
- windows were limited to 8192 px frames: they now reach the photo's own pixels
  (65536 px at most), so a medium-format photo is sharp at 1:1;
- a pinch out from a deep zoom asked for an ever wider window every frame: windows
  already there are kept, magnified, until the gesture has been quiet.
The unused region::needed is gone.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 8a5ecebc18 #323: wipe, pane clipping and identical-looks fixes for the Before/After windows
An adversarial review found that (1) in a wipe the Before window was drawn
before, so under, the low-resolution Before stand-in; (2) the windows of side by
side and stacked views were clipped to the canvas, so their margins spilled into
the neighbouring pane; (3) with identical Before and After looks (a fresh,
unedited photo) both windows had the same job key and the After tile replaced the
Before one in the tile table. Windows are now drawn over the stand-ins and under
the wipe line, clipped to their pane or their side of the line (region::window_clip,
pure and tested), and tiles are keyed by side and job key.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 eb06b65fdb #323: docs: the loupe's render plan, Before/After windows, drag timings
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 92196fddeb #323: tests: the drag keeps its original; a profile of slider drags (ignored)
profile_slider_drag prints the milliseconds from a slider tick to the loupe
showing it at fit, 100 % and 400 % of the demo library's 24 MP photos.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 ea33301e6e #323: a drag keeps the original its superseded drafts decoded; textures convert faster
A drag supersedes every request before it finishes, and the source a superseded
draft had decoded was thrown away with it, so a window cut from a full-size
original decoded it again for every tick. A view slot's superseded draft that is
newer than what the view shows now also hands its source to the session (late
results of other slots stay ignored, as before). Render results become textures
with a plain copy for the opaque case instead of egui's per-pixel premultiplying
conversion, on the UI thread.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 2f7d248d61 #323: a window does not keep the whole source uploaded with its stages
Profiling a slider drag at 100 % and 400 % found each tick of the window render
taking about 60 ms: the 24 MP source (288 MB) uploaded to the device was kept with
the view's stages, counted against the stage budget, and the stages cleared every
frame, so every tick uploaded and resampled again. A source over 96 MB is now
uploaded for the render only; the stages keep the window's own sampled pixels.
A tick costs 7-12 ms instead. equivalence.rs covers it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 35c9b67944 #323: test: no window over a picture that is not the frame (#280)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 6a07c35431 #323: hold the render sizes while a pinch or two-finger scroll runs
A continuous trackpad pinch changes the zoom every frame, and with it the sizes
the loupe plans, so it would have been a stream of re-renders of every size in
between. SizeHold (pure, tested) keeps the plan from before the gesture while it
runs and for a quarter of a second after; the view is drawn from the renders it
has, magnified by the GPU, and planned afresh when the gesture is quiet (a repaint
is asked for at that moment). Another photo resets the hold.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 72bab50c6e #323: Before/After at 1:1 compares like with like
The Before image was a stand-in capped at 2560 px beside an After at full
resolution, so a 1:1 comparison misled. Both sides now get the same treatment:
the same whole-frame size and, once zoomed, a window each (Slot::RegionBefore
beside Slot::Region) cut from the same frame, in side by side, top/bottom, the
\ original view and both wipes (the Before window is clipped to its side of the
line). Each side's windows and tiles are tracked apart, a side is freed when it
is no longer shown, and ui.inspect reports regionBefore.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 136b2851ee #323: engine: region_job_before renders a window of the photo without its edits
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 83d0304ce6 #323: the per-view stage caches are counted against the memory budget
The loupe's, the zoom window's and the before / hover renders' stage caches hold
decoded-image sized buffers that the engine's budget did not count. A quarter of
the memory budget is now theirs: each frame the renderer clears what exceeds it,
the least reusable first (hover, before, compare, second window, the zoom window)
and the open photo's own view last (stage_trim_order, pure and tested). The
inspector's memory report shows the budget and how often caches were trimmed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 3255cda67e #323: the loupe renders the canvas, plus the visible window at no more than 100 %
At 8:1 on a 24 MP photo every drag frame rendered the whole 24 MP frame, and a
Retina fit view (about 2800 px) decoded the original of every photo stepped to.
A new render plan (region::plan, pure and tested) makes the cost follow the
canvas, not the zoom:
- the whole-frame render is canvas-sized and never above the preview source
  level, the preview size limit or the texture side (a drag drafts that);
- once the view is zoomed past it (by more than 25 %) the visible window is
  rendered, cut from a frame that is at most the photo's own pixels, so beyond
  100 % the GPU magnifies it instead of a bigger render being made; this also
  ends the duplicated second render above 100 %;
- a window is drafted during a drag and a draft of an older value still draws;
- a user who capped the preview size keeps the whole-frame render at fit;
- no window where the picture on screen isn't the frame's own aspect (an
  unsupported raw's embedded JPEG cropped differently, #280).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 b7f632f5c0 #323: no loupe texture is larger than the GPU allows
The Automatic preview size let the loupe texture reach 8192 px on every
platform. The browser build draws with egui_glow, which panics on a texture
larger than the GPU takes, and many WebGL devices allow only 2048 or 4096.
Every render size the loupe asks for (loupe, neighbour prefetch, hover and
Before stand-ins, compare tiles, the second window) and every zoom window is now
clamped to the context's max_texture_side (2048 when the host reports none,
512 at the least). Headless drivers can pretend a small GPU for tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 63cfc6ab70 #323: parity, GPU and control docs: sharp zoomed views by window render
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 930d14aacd #323: fix the preview_limit doc comment (largest, not lowest)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 ff9c3fb2ec #323: windows: refuse what cannot fit, grow Auto Mask windows by the dabs near them
Review findings: window growth for spots was capped silently and then rendered
wrongly; Auto Mask growth took the whole stroke's bounding box; hidden masks grew
windows; the wasm window memory cap was not enforced. window_for_reads_checked
returns None when what a window reads would need more than 8192 px (4096 on
wasm) along an axis, Session::region_job returns None then (the loupe keeps the
whole-frame render), Auto Mask growth is per dab near the request (once, not
chained along the stroke) and ignores hidden masks. Docs say what the limits are.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 a1d907bf0f #323: a tight crop at its own pixels reads the source level its pixels need
An adversarial review found that sizing the loupe render from the cropped frame
(previous commit) left the source level chosen from the output edge alone: a
crop whose own long edge is under 2560 px was drawn from the 2560 px preview of
the whole photo, soft though sized right. The level now follows the pixels the
output needs along the source's long edge (output edge / crop fraction); grid
thumbnails keep their cheap level.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 05618b2ad4 #323: windows on the GPU; Auto Mask strokes grow the window; dab references at f32
The GPU renderer renders windows (no more CPU fallback): the vignette gets the
window's offset and the whole output's size (VIG_VIEW), the airlight and noise
reduction scale come from the whole frame, and a window grown for spots or
strokes is cut back after the readback. equivalence.rs renders every settings
case as a window plus a corner vignette and a spot reading from outside, within
the existing 1 LSB. Auto Mask strokes join spots in growing the window (they read
the pixel under each dab). An Auto Mask dab's reference pixel is chosen from its
position at f32 precision, like the GPU kernel: an f64 rounding error that
depended on the window's start could put a boundary dab on either pixel.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 4b83f8bb5e #323: windows cross the Web Worker boundary
WireJob carries the request's window (an old job without one has none), and the
browser build asks for window renders like the desktop; windows there are capped
at 3072 px a side to respect wasm memory.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 eebea486d5 #323: windows: spots render in a window; deep-zoom regression tests for the wide stages
Spot removal reads outside the spot and its automatic source depends on the
pixels at hand. A windowed render now fixes automatic sources from a fixed-size
render of the whole frame and grows the window it works on to hold every spot
that touches the requested one and the patch it reads (bounded at 8192 px),
cutting the result back to the request, so photos with spots get sharp windows
too. The wide stages (dehaze, clarity, highlights/shadows) are checked at depth
on hard-edged blocks: the cut kernels cost at most a few levels, so they are not
computed from a whole-frame proxy.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 dc5bff6947 #323: 100 % zoom is the photo's own pixels as shown (after crop and rotation)
fit_rect measured Percent zoom against the uncropped, unrotated photo, so a
cropped photo at 1:1 showed stretched and a rotated one used the wrong axis. The
loupe, its neighbour prefetch and the compare view now use the frame's
cropped-and-rotated size, which also stops a small crop being rendered above its
own pixels.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 f8f091436a #323: review fixes: no window renders during drags, current-look tiles only, margin by zoom
An adversarial review found that a slider drag at any zoom started a window
render (the 0.6-scale draft looked stretched), that a tile made for an earlier
look kept being drawn and that the tile table was cleared mid-drag, that the
fixed 256 px margin was far below the wide stages' kernels at depth, that spot
removal reads outside the window, and that the window texture stayed resident
after zooming out. Window renders now wait for the drag to end, a tile is drawn
only while photo, zoom and look match, the margin grows with the zoom (768 px
cap) and the window is capped at 6144 px, photos with spots keep the whole-frame
render, the texture and stages are freed when unneeded, and wasm never asks.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 244088498d #323: windows: noise reduction and mask edge refinement follow the whole frame's size
An adversarial review found local::denoise and Refine Edges scaled with the
window's own size, so their strength changed with pan and zoom. Frame::output_long
gives the whole output's long edge to both; tests on a noisy source cover them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 e0539db3f5 #323: the loupe renders the visible window at the zoom scale
When the whole-frame render is smaller than the zoomed view (a 1:1 or 8:1 view
of a large photo, or a capped Preview size) the loupe also requests a window
render (Slot::Region) of the pixels on screen plus a margin, and draws it over
the whole-frame render at its place of the frame. Stateless, grid-snapped
windows mean small pans reuse the render. Not used in the browser build, while
comparing, hovering a look, soft-proofing or showing an overlay. ui.inspect's
loupe reports the window.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 2079e8e08d #323: engine region_job and the window geometry for zoomed views
Session::region_job renders one PixelWindow of the loupe's frame from the source
level that frame size needs, with a key of its own and no cache writes.
ui-egui::region decides when a window is needed (the whole-frame render is
stretched) and which one: the visible pixels plus a margin, snapped to a 256 px
grid so small pans reuse the render, bounded in size, clamped into the frame.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 b358022f14 #323: pipeline: render a window of the output (RenderRequest::window)
A zoomed view needs the pixels it shows at the zoom scale, not a whole-frame
texture stretched over them. RenderRequest gains an optional PixelWindow of the
virtual full output that max_w x max_h describes; only the window is
materialized. It is Frame:🪟 the crop rectangle narrows to the window
(flips as drawn), so sampling, masks, spots, red eye and the lens warp need no
change. The post-crop vignette keeps the whole frame's geometry (Frame::view)
and dehaze uses the whole frame's airlight (estimated on a 384 px render), so a
window matches the same pixels of the full render. The GPU renderer declines
windowed requests (CPU for now).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 ca5c212d03 #323: review fixes: prefetch and stand-ins stay at preview level; render sizes move in 64 px steps
An adversarial review found that with Automatic the neighbour prefetch asked for
the full source level, whose single cache slot then evicted the open photo's
source, and that hover / Before renders and every window-resize pixel produced
full-size renders. Prefetch and stand-ins are now capped at the preview source
level and the loupe size is rounded up to a 64 px step.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 14570cfdf3 #323: Detail view renders at the size it is shown, not at a hidden 2560 px cap
The loupe capped every render at the "Preview size" setting (default 2560 px),
then stretched the texture over the on-screen rect, so 1:1 and Retina fit views
of larger photos looked soft or pixelated.

The setting is now a limit: Automatic (default) renders at the drawn size, up to
the photo's own pixels and an 8192 px ceiling; 1600–5120 px still cap it. The
domain rule is AppSettings::loupe_edge (NaN/zero/huge inputs are clamped). The
saved key changes from previewEdge to previewLimit so existing installs, whose
2560 was the old default, move to Automatic. Build Standard-Sized Previews keeps
a numeric edge via standard_preview_edge().

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 18:28:45 -04:00
Claude 8d79bf082e Batch fix: the header test's RLE PSD fixture uses PackBits runs of at most 128 bytes
The fixture wrote each row as one literal run of w bytes, which PackBits
can't express past 128 (a 160-pixel row's header byte read as a repeat
run). The old decoder clipped such rows silently; with the PSD fix in
this batch an overrun is an error, as it should be, and the fixture now
splits rows into valid runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 22:23:51 +00:00
Claude d316d97d80 Merge branch 'fix-psd-rle-alloc' into integ-b9
# Conflicts:
#	crates/codecs/src/psd.rs
2026-10-08 22:21:56 +00:00
Claude 2ab6e973c6 Merge branch 'fix-367-header-probe' into integ-b9 2026-10-08 22:21:37 +00:00
BFlat c395789026 Merge pull request #398 from storytold/claude/integrate-batch-8
Integrate #391 and #393 (undecodable raws import as preview only), with a work budget for the JPEG scan
2026-10-09 05:21:21 +07:00
Claude 6aa599f869 Fix: PSD RLE decode bounds its allocations by the input
A tiny hostile PSD could abort the process on out-of-memory in
`psd::decode` (allocation failure doesn't unwind, so catch_unwind
can't save it):

1. The RLE row byte-count table (channels x rows entries, from the
   header) was reserved with Vec::with_capacity before checking that
   the file holds it: 56 channels x 2^30 rows asked for ~480 GB from a
   ~40-byte file. The table is now taken as a slice of the input with
   checked size math and read lazily, so nothing is allocated from it.

2. RLE rows were zero-padded / run-expanded to the declared width with
   no tie to the input size, so all-zero row counts on a 1 Gpx (the
   DecodeOptions::max_pixels limit, already enforced by check_size)
   32-bit image expanded to 4 GiB per channel. A PackBits row expands
   at most 64x (2-byte repeat run -> 128 bytes), so each decoded row
   must now declare at least ceil(row_bytes / 128) * 2 encoded bytes,
   and the decoded channels' data must be present in the file, before
   any plane is allocated. Decoded planes are thus bounded by 64x the
   file size as well as by max_pixels. PackBits now decodes into a
   pre-sized, zeroed row and a run that would write past the declared
   width is an error instead of being silently clipped.

Also: size math (width x height x bpc, samples) is checked; image
buffers are allocated with try_reserve_exact (TooLarge on failure);
only the colour (+ alpha) planes are decoded, extra channels are
skipped; the PSB layer-section length no longer truncates on 32-bit
targets; interleaving uses iterators instead of indexing.

Valid PSDs (raw and RLE, 8/16/32-bit) decode identically; regression
tests cover a huge RLE header with a truncated table (aborted with a
481 GB allocation before), all-empty RLE rows, counts larger than the
file, a repeat run past the row width (was Ok before) and a 16-bit
gray RLE file with an extra channel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 22:19:42 +00:00
Claude f6f5e6f1b7 Part of #367: probes read image headers instead of decoding the pixels
The import probe of a non-raw image decoded it (at a 64 px fit, which for
JPEG still entropy-decodes every block and for PNG inflates everything)
only to learn its stored size and orientation: ~95 % of import CPU.

lightcraft-codecs gets `read_header(bytes) -> Header { format, width,
height, orientation }`: JPEG from its markers (EXIF orientation), PNG from
its chunks (IHDR, eXIf anywhere), TIFF from the first IFD (tiff crate,
no strips read), WebP from its RIFF chunks / bitstream header (image-webp),
PSD from its header and image resources. GIF, BMP and JPEG XL keep the
small decode. `files::probe_bytes` uses it; raw handling is unchanged.
The TIFF colour-layout check is shared with the decoder (`layout`).

Measured (release, 4 cores, throwaway benchmark, per file, bytes in
memory): 200 generated 4000x3000 JPEGs (3.6 MB): 72.7 ms -> 2.1 ms
(read_header alone 1.05 ms; the rest is the content hash and metadata);
20 PNGs (30.8 MB): 236-265 ms -> 9.0-9.6 ms.

Corrupt files (behaviour note): the header readers also refuse what a
decode refuses before the pixels (unsupported layout, compression, frame
type, bad IHDR, ...) and truncated files: a JPEG whose markers don't reach
its EOI, a PNG cut before the chunk after its IDAT run, TIFF strips/tiles
that start past the end, a WebP image chunk or PSD merged data past the
end. So truncated files (interrupted copies, partial downloads) are still
marked `error` in the import review, as before; tests check that whenever
the old decode accepted a (cut or byte-mutated) file the header agrees.
Changed: damage *inside* compressed data that is all present (flipped or
garbled bytes) was refused at scan and is now imported, and shows as
unreadable ("!" in the grid) when rendered. Also: tiny JPEGs (< 128 px)
were decoded leniently by the old probe, so a truncated one imported; it
is now refused. A TIFF truncated inside its last strip is not detected
(StripByteCounts are often overstated, so only strip starts are checked).

Tests: header readers vs decode on generated JPEG/PNG/TIFF/WebP/PSD/BMP
with and without EXIF/TIFF orientation; truncations and random mutations
agree with decode; hostile headers (zero/absurd sizes, bad lengths,
truncation, garbage) are errors; read_header added to the fuzz tests;
probe_bytes oriented dims match the decode-based values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 22:09:30 +00:00
radialmonster 7094ba1b6e LR-IMP-FORMATS: CR3 C-RAW samples that overshoot the bit depth are clamped
C-RAW is quantised wavelet coding: next to clipped highlights its
reconstruction overshoots the sensor range by a few quantisation steps.
On 17 CC0 CR3 files (SX70 HS, G5 X Mark II, M6 Mark II, EOS R, RP, 90D,
M200, R5 Mark II, R6 Mark II, R6 Mark III, R8) samples reached 16384 to
16419 at 14 bits, never below 0, and decoding failed with "decoded
sample outside its bit depth". Lossless files of the same bodies clip at
exactly 16383. Wavelet samples are now clamped to the CMP1 bit depth;
lossless samples outside it are still corrupt data.

The other 93 CR3 files have no sample outside the range and render
pixel-identically. Render cache version 13 -> 14.
2026-10-08 18:04:01 -04:00
Claude cb0e0cb2e5 Follow-up to #393: the embedded-JPEG scan shares a work budget
scan_for_jpeg walks every JPEG-like start in a container it can't parse;
each walk runs through the marker segments that follow (up to 4096) and
their entropy data. A crafted CRW/MRW/X3F (or unsupported TIFF raw) of
fake starts one after another makes every walk cover most of the file:
quadratic, about a minute for 40 MB, on the import's probe thread. The
walks now report how far they read and share a budget of eight times the
file's size (at least 1 MiB); real files' candidates fail within a few
bytes or are the preview itself. Test: 200,000 fake starts, 0.15 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 22:03:26 +00:00
Claude 0fb3351c91 Merge PR #393 2026-10-08 22:00:34 +00:00
Claude b9254242a2 Merge PR #391 2026-10-08 22:00:10 +00:00
BFlat ab9e8d372c Merge pull request #394 from storytold/claude/integrate-batch-7
Integrate part of #374 (bounded folder walks, launch imports off the UI thread) and #385, #386, #387
2026-10-09 04:43:24 +07:00
Claude b11e5399c6 Merge PR #387 2026-10-08 21:26:34 +00:00
Claude 83058f717e Merge PR #386 2026-10-08 21:26:34 +00:00
Claude 7962b66a06 Merge PR #385 2026-10-08 21:26:33 +00:00
Claude c140f63372 Merge branch 'fix-374-startup-drive-walk' into integ-b7 2026-10-08 21:19:42 +00:00
Claude 790a341776 Part of #374: launch imports run off the UI thread; folder walks are bounded and never take a drive root or home implicitly
Root cause (in 0.2.1 and still on main): files and folders named on the
command line were imported from the eframe app-creator closure with
`app.run("library.import", …)`, which runs the engine command synchronously:
`import::expand` walked the folder recursively (one `is_dir()` open per entry,
no depth or count bound, following links) and probed every file before the
first frame could be drawn. An argument that resolves to a drive root (`.` or
`..` with a launcher's working folder at `C:\`, `/` or `\` on Windows) or to
the home folder therefore walked the whole drive with the window stuck as a
black "not responding" box. The library-problem banner's pending import had
the same synchronous call. Every other recursive walk (Local's Include
Subfolders, Find Missing Photos, preset and LUT folders, MCP) was also
unbounded, which is the "Local folders with subfolders: not responding"
pattern of filmcraft#23, and a link loop recursed until the stack overflowed.

- engine `walk::files_in`: one bounded, iterative walker (32 folder levels,
  1,000,000 entries; `Walked::truncated` says it stopped), folder or file
  taken from the directory listing (no open per entry), links and junctions
  to a folder followed once. `import::expand` (+ `expand_within`), preset and
  LUT folder expansion and MCP `expand_paths` use it.
- engine `import::launch_path`: an empty launch argument is ignored; a
  relative one is resolved against the working folder; a drive or file-system
  root becomes its memory card's DCIM folder or is refused, and so is the home
  folder (a notice says why and to use File > Import Photos).
- desktop: launch arguments go through it and are imported with
  `import::start_paths` (the import worker), like dropped files; so does the
  library-problem banner's pending import.

Not verified on Windows (no machine here). The report says PhotoCraft 0.5.0
and `--memory` hang the same way with no arguments; nothing in LightCraft
walks the disk at startup without arguments (strace of a headless demo run
with HOME=/ and cwd /: no recursive listing), so that part may be outside
this code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 21:19:42 +00:00
BFlat 67b7bc8bc6 Merge pull request #389 from storytold/claude/integrate-batch-6
Integrate #373 (keymap editor), #218 (Lightroom catalog import), #367 (parallel import copies), #344 (import file types)
2026-10-09 04:19:01 +07:00
Claude 41e67e1edd Batch fix: the CLI snapshot test settles before its screenshots
Under load (the full workspace suite) the photo grid was captured before
its thumbnails rendered, darker than the export dialog dimmed over them,
and the dimming comparison failed; it passes alone. Each screenshot now
follows ui.settle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:56:35 +00:00
Claude 5d437489b9 Merge branch 'fix-344-import-file-types' into integ-b6 2026-10-08 20:53:11 +00:00
Claude 1e6f0e28e1 Batch fix: the Lightroom catalog dialog filter goes through add_filter_nocase
#218's catalog picker called rfd's add_filter, which #362's guard test
(no_dialog_bypasses_the_helper) forbids: on Linux the filter would hide
CATALOG.LRCAT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:53:11 +00:00
Claude 4af709995f Fix #344: the import review checks or unchecks a file type at once
With several file types among the candidates, the Import Photos review
shows a toggle per type (ARW · 120, JPG · 120): unchecking JPG leaves
the JPEGs kept beside the raws out of the import, checking a type checks
its importable files (duplicates stay unchecked). JPEG/JPG and TIFF/TIF
are one type; a file whose probe named no format goes by its extension.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:52:35 +00:00
Claude 9b0d0fd05f Parity summary regenerated (cargo xtask parity --write)
Main's summary block had a duplicated pair of total lines left from an
earlier merge; the regenerated one counts #373 and #218's rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:50:04 +00:00
Claude b1d8ba4284 Merge branch 'fix-367-import-parallel' into integ-b6 2026-10-08 20:48:31 +00:00
Claude d8efa6a147 Spanish for the keymap editor (#373) and the Lightroom catalog import (#218)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:48:31 +00:00
Claude d4c0a7a3d4 Merge remote-tracking branch 'origin/main' into integ-b6
# Conflicts:
#	crates/ui-egui/locales/de.json
#	crates/ui-egui/locales/ja.json
#	crates/ui-egui/locales/pt-br.json
#	crates/ui-egui/locales/ru.json
#	crates/ui-egui/locales/zh-hans.json
#	crates/ui-egui/locales/zh-hant.json
2026-10-08 20:48:15 +00:00
BFlat 68e0827633 Merge pull request #383 from storytold/claude/test-376-masking-layout
Part of #372: painted-text layout tests from #376
2026-10-09 03:47:22 +07:00
ClaudeandElliot Graham cad024fbff Part of #372: painted-text layout tests from #376
The tests from @QuirkyTurtle94's #376, a duplicate fix for #372 opened
while #379 was in review: they read the painted text shapes (not the
registered rects) at 1x and 2x scale, in English, German and Brazilian
Portuguese, at panel widths 250-520, and check that tile labels fit
unelided inside their tiles, that a long mask name is elided before the
eye, and that a long Describe prompt keeps Select inside the panel.
They pass unchanged against main's implementation.

Co-Authored-By: Elliot Graham <elliotgraham94@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:47:07 +00:00
Claude 86deb5b004 Fix #367: import copies and verifies files on several threads
Import -> Copy (and Move, and Copy as DNG) handled one file after another
on the import worker: copy, fsync, read back and hash, optionally decode
and re-encode the raw as DNG. Every other CPU-heavy step of an import was
already parallel (the probes, up to 8 threads), so with the probes cached
by the review dialog's scan the transfer phase ran on one core.

ImportJob::prepare_files now plans each file in order, as before
(duplicates by path and content, sidecars, rename sequence numbers,
folders, Kept notes) and collects the copies/moves, which then run on up
to 8 threads (std::thread::scope; sequential on wasm32). Outcomes keep
their places, so results, order, names and failure messages are those of
a one-by-one import:
- a file whose content matches a transfer still pending, or whose name
  could collide with one (same folder, same name family: IMG.jpg,
  IMG-1.jpg, img.dng), waits until that one has finished;
- cancel stops before the next file; transfers not started leave no item;
- a panic in one transfer fails that file instead of the batch;
- Copy as DNG holds a memory-gate permit (6x the file size) while it runs.
Copy verification and crash safety are unchanged (same copy_new / place).
The desktop's import worker prepares batches of 2 x workers files (8..16,
lightcraft_engine::import::batch_size) instead of 8, so a batch's slowest
file holds back fewer threads; the browser build keeps 8.

Measured (4-core shared VM, 200 procedural 3.7 MB JPEGs = 754 MB, warm
page cache, throwaway benchmark not committed):
- probe: ~80 ms CPU per file is the JPEG decode (DCT-scaled to 64 px),
  1.2 ms the content hash; the dialog's scan: 4.4 s, 3.6 cores busy.
- Copy, probes cached: copy+fsync+verify 1.59 s at 0.43 cores before,
  0.94 s at 0.81 cores after. Copy, not cached: 6.45 s -> 5.51 s.
- Add, probes cached: 2 ms prepare; nothing to parallelize.
- commit of a batch on the UI thread (journal append + fsync): ~4 ms,
  flat for libraries of 0, 5k, 20k and 50k photos (~0.7 s for 1300).
Copy as DNG could not be measured (no raw corpus here); it is the most
CPU-heavy per-file step and was sequential.

Test: copies_run_side_by_side_with_the_same_outcome (copies under the
test fault Slow run on more than one thread; names, order and in-batch
duplicates match a one-by-one copy; the outcome assertions also pass on
the old code, the thread assertion fails there).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:47:04 +00:00
BFlat a673b2bb7a Merge pull request #382 from storytold/claude/i18n-371-spanish
Fix #371: Spanish interface
2026-10-09 03:41:48 +07:00
Claude 305ff71474 Part of #371: Spanish for the SAM 3 install dialog (#366)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
(cherry picked from commit 73b22c8524ef17b209ff2fffbb16c248524e2e96)
2026-10-08 20:41:31 +00:00
Claude a0e49629d3 Merge remote-tracking branch 'origin/main' into i18n-371-spanish 2026-10-08 20:41:31 +00:00
BFlat c8f697a128 Merge pull request #381 from storytold/claude/fix-366-sam3-install-help
Fix #366: the SAM 3 dialog without a download location shows how to install by hand
2026-10-09 03:41:25 +07:00
BFlat 232cbace63 Merge pull request #379 from storytold/claude/fix-372-mask-labels
Fix #372: masking tile labels and long mask names stay inside their boxes
2026-10-09 03:41:22 +07:00
radialmonster 1fa3af8740 LRC-LIB-LIBFOLDERS: the sidebar rename test builds its folder ids with forward slashes on Windows
The folder tree writes paths with forward slashes and its widget ids
carry them; the test put the temp folder in the id with backslashes.
Found running cargo xtask ci on Windows.
2026-10-08 16:41:12 -04:00
radialmonster 8e4d7545ff M11.4: the X-T20 corpus RAF is fitted since the away-from-edges retry; update the test and the doc that said it was rejected
Found running cargo xtask ci with the corpus on Windows. The two changes
landed on the same day: the Fujifilm test (#282) expected the compressed
X-T20 file to keep the neutral fallback because its fit failed the gates
on all pixels (held-out RMS 0.114 against 0.10). The retry away from
edges (#263, issue #232) fits it (0.056), so the fit is accepted at both
sizes. Expect true.
2026-10-08 16:41:12 -04:00
radialmonster 1028ed6b88 LRC-LIB-LIBFOLDERS: a chosen folder that follows a move keeps its names' case and the platform's separators
follow_library_folder rebuilt the chosen subfolder from the lower-cased
folder_key on Windows and joined it with '/', so after a move the folder
shown in the sidebar came back as c:\archive\trip/day1 with its names in
lower case. Add folder_rest (compares by folder_key, returns the names as
spelled) and join them with the platform's separator. Tests compare
folders by identity where they only mean the same folder, and a new test
fails on Windows without the fix.
2026-10-08 16:41:11 -04:00
Claude efe7cad73c Follow-up to #218: Lightroom catalog import translated
The progress, result and menu strings of the Lightroom catalog import in
the ja, pt-br, zh-hans, zh-hant, de and ru catalogs. The formats catalogs
held English copies of the seven new format messages; they are
translated, and German and Russian (which the PR predates) get them too,
so the formats build check passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:36:22 +00:00
Claude a86defd1b9 Merge PR #218 2026-10-08 20:35:39 +00:00
radialmonster 881bc85dac LR-IMP-FORMATS: import CRW, MRW, X3F, KDC, MOS, ERF and IIQ files from folders as preview only
Folder import, drag and drop and MCP folder expansion skipped these extensions, so
the preview-only fallback for these containers was only reachable by naming a file.
Adds exactly the containers that now import as preview only to import::EXTENSIONS and
the headless PHOTO_EXTENSIONS. 3FR, FFF, DCR and STI still fail and stay out.
2026-10-08 16:32:17 -04:00
radialmonster dbe73f0f9c LR-IMP-FORMATS: raws we cannot decode import as preview only when any usable JPEG is embedded
Widens the preview-only fallback. A recognised raw whose decode fails now falls back
to its embedded JPEG for any decode error (unsupported, corrupt, limit, TIFF
structure), not only an unsupported variant, and the preview search covers more
containers:
- Canon CRW, Minolta MRW and Sigma X3F are recognised by their magic (RawFormat
  Crw, Mrw, X3f) instead of failing as "RawOther files are not supported yet".
- A TIFF whose IFD0 has no ImageWidth/ImageLength but a JPEG pointer or SubIFDs
  (Kodak KDC) is a raw container, not a malformed image. TIFF compression 99
  (Leaf MOS) counts as a raw compression.
- When no tag points to a preview, a recognised raw we cannot decode is scanned
  for the largest complete three-component DCT JPEG. A preview whose first byte
  was overwritten (Minolta, Epson) is restored. A one-component JPEG is skipped
  (a PowerShot CRW stores its mosaic that way).
Files with no JPEG at all still fail with "without an embedded preview".
2026-10-08 16:32:17 -04:00
radialmonster 0b8c7a237a LR-IMP-FORMATS: TIFF raws with a thumbnail-only IFD0 import as preview only, not as a tiny image
Phase One / Leaf IIQ, Canon EOS-1D / 1Ds and Kodak DCS TIFFs keep their raw data in
a private block. IFD0 is only a 160-640 px RGB copy and no IFD is marked as raw, so
they opened as an ordinary TIFF of that size (a 150 MP file rendered 296x220) with
no preview-only label.

probe() now recognises such a file from its structure (OtherTiff): a picture at
least 4x larger than IFD0 in both directions is declared by a SubIFDs child, or by
Exif PixelXDimension/PixelYDimension together with enough bytes outside every IFD's
strips and tiles to hold it at one bit per pixel. decode() reports the sizes as the
reason. A raw we cannot decode now falls back to its embedded JPEG, else to the TIFF
reader's view of its first image, so it imports preview-only with that reason.
RENDER_CACHE_VERSION 12: these files render differently.
2026-10-08 16:31:49 -04:00
Claude 43aa795539 Follow-up to #373: translated keymap editor, contextual keys noted
- The keymap editor's strings, the fixed-key labels and the two new
  command labels in the ja, pt-br, zh-hans, zh-hant, de and ru catalogs;
  'Restore {}' and 'Removed from {}' go through tr_format!.
- The Presets and Soft Proofing rows say what their key does in the
  Library grids (Flag as Pick and advance / expand or collapse the
  stack), which main's static Shortcuts list showed as a separate label.
- The macOS menu publishes no shortcuts while a key is being recorded
  (merged with main's owned_by_menu).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:24:13 +00:00
Claude 38cdd49274 Merge PR #373 2026-10-08 20:23:12 +00:00
Claude 0b587a0a8a Fix #371: Spanish interface
A Spanish (es) catalog and formats catalog covering the same messages as
the Brazilian Portuguese, Japanese and Simplified Chinese catalogs (1595
messages, 180 formats), an Edit > Language > Español command
(app.language.spanish), es tag tests (es, es-ES, es_MX.UTF-8, es-419),
docs/localization-es.md with the terminology used, and the language
lists in docs/localization.md, parity and What's New.

Instructions use infinitives or impersonal forms (no usted). Plural
endings follow the existing convention: the English "s" passes through
where Spanish adds one, else {:.0} swallows it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:20:17 +00:00
Claude 60a98354d9 Fix #366: the SAM 3 dialog without a download location shows how to install by hand
With no mirror configured the dialog was titled "Download the SAM 3
Model?", only said to put "the files" in the folder (see
docs/ai-masks.md) and offered Close. It is now titled "Install the SAM 3
Model", names the three files (model.safetensors, vocab.json,
merges.txt from facebook/sam3), and has a Show in Finder / Explorer /
File Manager button that creates and shows the model folder, plus a link
to the installation guide. Object and Describe work as soon as the files
are there.

The download location itself (a hosted mirror) is still a maintainer
decision: DEFAULT_MIRRORS stays empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:18:47 +00:00
Claude f16f413942 Fix #372: masking tile labels and long mask names stay inside their boxes
The Create New Mask tiles painted their (untranslated) labels with an
unconstrained Painter::text into tiles capped at 52 pt, so Background and
Luminance ran past their tiles, and translations would more so. The grid
now measures the translated labels: four 52 pt tiles when they fit (at
10.5 pt, or 9.5 pt before dropping a column), else three or two, a tile
growing past 52 pt for a long label; only in a panel too narrow for even
that is a label cut short (full text on hover).

A mask row's name is cut short with an ellipsis before the eye (full name
on hover). The Describe field places the translated Select button first
and gives the field what is left, instead of reserving a fixed 64 pt.
Status lines wrap explicitly.

The painted label and name rects are registered (maskNewLabel:*,
maskName:*), so the sidebar test checks the text, not just the widgets,
at four panel widths with a long name and a long prompt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 20:09:16 +00:00
BFlat 48cd9c5e3a Merge pull request #375 from storytold/claude/integrate-batch-4
Integrate #279 (Canon CR3 decoding) and #362 (file dialog filters)
2026-10-09 03:08:11 +07:00
parkerHurstandClaude Opus 5.5 e2b626b7f7 LR-PREF-SHORTCUTS: editable keyboard shortcuts in Help ▸ Keyboard Shortcuts (⌘/)
The ⌘/ dialog is now a keymap editor: click a shortcut and press the new
keys (Esc cancels), × removes it, ↺ restores the default, Reset All clears
every change. A key another command had moves to the new command, with a
toast. Changes go through the new app.setShortcut / app.resetShortcuts
commands into AppSettings.keymap (saved in ui.json) and show up in the
menus, the native macOS menu and the control channel's command list.

Modifier keys alone never record (holding ⌘ waits for the key it
modifies), ⌘C/⌘X/⌘V are recorded from the copy/cut/paste events, and
saved entries that don't parse fall back to the default shortcut.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:54:50 -05:00
Claude 28db7b200b Merge fixes for #279: CR3 joins RAF in the format lists, render cache version 13
Main already had Fujifilm RAF in relative WB, the file-local look, the
docs and the corpus lists; CR3 is added next to it. RENDER_CACHE_VERSION
was already 12 on main (from #255/#258/#282), so CR3 thumbnails rendered
by main from embedded previews would not be redone: 13.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 19:44:32 +00:00
Claude 3f63db0a0b Merge PR #279
# Conflicts:
#	README.md
#	ROADMAP.md
#	crates/catalog/src/model.rs
#	crates/engine/src/camera_preview.rs
#	crates/raw/src/lib.rs
#	crates/raw/tests/corpus.rs
#	docs/camera-preview-colour.md
#	docs/parity.md
#	docs/whats-new.md
2026-10-08 19:44:21 +00:00
Claude f611e76d0e Merge PR #362
# Conflicts:
#	apps/lightcraft/src/main.rs
#	docs/parity.md
2026-10-08 19:43:31 +00:00
BFlat eeb136b9ee Merge pull request #370 from storytold/claude/fix-260-startup-failure-dialog
Part of #260: a desktop start that fails says why, in a message box naming the log
2026-10-09 02:43:05 +07:00
BFlat 3f10ba4f3b Merge pull request #369 from storytold/claude/fix-171-unlocked-library-warning
Fix #171: a library that can't be locked opens with a warning the user sees
2026-10-09 02:43:01 +07:00
Claude 7fb24029f2 Part of #260: a desktop start that fails says why, in a message box naming the log
When eframe could not open the window (no usable graphics device, a
broken driver), main returned the error to standard error, which a
Windows release build (no console) or an app started from Finder never
shows: the app just vanished. The error now goes to the log (the file
#286 added) and into a native message box, in the user's language, that
names the log file. The saved UI language is applied before the window
starts so the message uses it. Scripts and tests (LIGHTCRAFT_NO_PREFS)
get no dialog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 19:30:26 +00:00
Claude 5601d7879a Fix #171: a library that can't be locked opens with a warning the user sees
When catalog.lock can't be locked (a file system without locks, such as
some network shares), the library opens unprotected, as documented, but
the promised warning went only to the log. It is now the first of the
library's warnings: library.info lists it under settingsWarnings, the
app shows it once in its notices, and lightcraft-cli run / mcp print the
library's warnings on stderr (stdout stays results / protocol only). A
normally locked library shows none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 19:27:32 +00:00
Adrian D'souza 095c7dce87 fix sqlite test payload extension 2026-10-09 00:57:19 +05:30
Иван Лапочкин f96f732baa M11.1: add CR3 decoding
Decode lossless CRX and version 0x100/0x200 C-RAW in portable Rust,
including sensor geometry, timed white balance and black/white levels.
Verify all sensor samples against six independent M50/R100/R8 references.

Any CR3 decode error falls back to the embedded JPEG, as CR3 files did
before this decoder. The full sensor buffer is allocated only after the
first plane decodes, and plane buffers grow row by row. Tile planes
decode in parallel with a windowed bit reader (24 MP: 424 -> 89 ms
lossless, 519 -> 138 ms C-RAW on an M4).

Add guarded CR3 camera-JPEG colour fitting with independent framing
validation, relative white balance and cache invalidation.

Validation: cargo xtask ci.
2026-10-08 22:22:13 +03:00
BFlat bf224c2a8d Merge pull request #361 from storytold/claude/fix-319-craft-fonts-pin
Fix #319: release builds pin craft-fonts with Noto Sans CJK SC (includes #203)
2026-10-09 02:19:59 +07:00
BFlat 16236ff779 Merge pull request #364 from storytold/claude/fix-293-undo-shows-photo
Fix #293: undo and redo show the photo they change
2026-10-09 02:19:56 +07:00
BFlat 2592ab47e9 Merge pull request #365 from storytold/claude/fix-sam-busy-after-reply
AI masks: the worker stops counting a job as pending before it replies
2026-10-09 02:19:49 +07:00
Claude 3e791905b9 Merge remote-tracking branch 'origin/main' into claude/fix-293-undo-shows-photo
# Conflicts:
#	crates/engine/src/tests.rs
2026-10-08 19:19:32 +00:00
Claude 2897f85861 Merge remote-tracking branch 'origin/main' into claude/fix-319-craft-fonts-pin 2026-10-08 19:19:27 +00:00
Claude 3517606ffc Merge remote-tracking branch 'origin/main' into claude/fix-sam-busy-after-reply 2026-10-08 19:19:27 +00:00
BFlat 9e51ca0da8 Merge pull request #354 from storytold/claude/fix-341-date-keyword-rows
Fix #341: By Date and Keywords rows show their photos from All Photos
2026-10-09 02:19:21 +07:00
Claude 85e38c7342 Merge remote-tracking branch 'origin/main' into claude/fix-341-date-keyword-rows
# Conflicts:
#	crates/ui-egui/src/tests_panels.rs
2026-10-08 19:19:15 +00:00
BFlat 4836143f9e Merge pull request #353 from storytold/claude/fix-316-section-eye
Fix #316: the section eye switches Light, Color and Detail off, and can be clicked
2026-10-09 02:19:02 +07:00
Claude 57dc811a6a Merge remote-tracking branch 'origin/main' into claude/fix-316-section-eye
# Conflicts:
#	crates/ui-egui/src/headless.rs
2026-10-08 19:18:55 +00:00
BFlat 9e327b4772 Merge pull request #352 from storytold/claude/fix-322-type-slider-values
Fix #322: type an exact value into a slider
2026-10-09 02:18:43 +07:00
Claude f6fcef4d5c Merge remote-tracking branch 'origin/main' into claude/fix-322-type-slider-values 2026-10-08 19:18:31 +00:00
BFlat 3895cf6411 Merge pull request #363: community PRs, batch 3 (German + Russian UI, Fujifilm compressed RAF, raw fixes)
Integrate community PRs, batch 3: German + Russian UI, Fujifilm compressed RAF, raw fixes (stacked on #350)
2026-10-09 02:18:21 +07:00
BFlat 5cb74e7313 Merge pull request #350: community PRs, batch 2 (14 PRs + follow-ups)
Integrate community PRs, batch 2 (14 PRs + follow-ups; stacked on #349)
2026-10-09 02:18:09 +07:00
BFlat 4165904bd1 Merge pull request #349: community fixes, batch 1 (18 PRs + follow-ups)
Integrate community fixes, batch 1 (18 PRs + follow-ups)
2026-10-09 02:17:58 +07:00
Claude 6ea9b979c8 AI masks: the worker stops counting a job as pending before it replies
The pending counter was decremented when the job's guard dropped at the
end of the loop iteration, after the reply was sent, so a waiting caller
(CLI / MCP mask.add) could see the segmenter still busy right after its
answer arrived. Under a loaded test run that made
tests_segment::waiting_requests_with_a_damaged_model_are_errors fail on
assert!(!busy()). The guard is now dropped just before each reply.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 19:02:44 +00:00
Claude e73d88a55f Merge PR #358
# Conflicts:
#	docs/parity.md
2026-10-08 18:57:38 +00:00
Claude d56d265044 Merge PR #282
# Conflicts:
#	ROADMAP.md
#	assets/ATTRIBUTION.md
#	crates/engine/src/camera_preview.rs
#	crates/raw/tests/corpus.rs
#	docs/camera-preview-colour.md
#	docs/parity.md
2026-10-08 18:55:07 +00:00
Adrian D'souza b4a1166f4b PR218: document optional-data and recovery warnings 2026-10-09 00:17:24 +05:30
Adrian D'souzaandCodex 5413c8bb3a PR218: make archive scale test exercise staging
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:13:49 +05:30
Claude c07395222f Merge PR #356
# Conflicts:
#	docs/parity.md
2026-10-08 18:43:38 +00:00
Claude 7b082660e7 Merge PR #339 2026-10-08 18:43:35 +00:00
Claude 3a9b6ccee4 Fix #293: undo and redo show the photo they change
After an undo or redo that changed one photo (its edit, rating, flag,
label, metadata or versions), that photo becomes the active one, as in
Lightroom Classic, so the change is on screen instead of happening to a
photo out of sight. It joins the selection if it isn't in it; steps that
touch several photos, albums or files, and photos outside the current
view, leave the selection alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:43:18 +00:00
Adrian D'souzaandCodex 07d9678209 PR218: cover optional payload cap warning
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:12:39 +05:30
Claude 8980c3bb0e Fix #341: clippy (no single-element loop in the test)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:42:11 +00:00
Adrian D'souzaandCodex 3c76902bbb PR218: cover unsupported schema entries
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:12:06 +05:30
Claude 1ebce934ea Fix #322: clippy (collapsible if)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:42:00 +00:00
Adrian D'souzaandCodex 7cf3719c63 PR218: localize complete import messages & scale prepared path
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:11:29 +05:30
Claude c3dce49c2e Format the gap-report test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:41:28 +00:00
Claude 3af89e6ed5 i18n: translate what #265 made translatable, and the formats gaps #264/#265 left
#265 wrapped about 460 more UI strings (command and control labels, rule
operators, built-in preset names, status messages) in tr()/tr_format!, and
filled the 54 new formats messages of the other catalogs with English
copies. These are now translated in ja, zh-hans, zh-hant and pt-br; the
Russian formats catalog gets the 58 messages it needs to build, and German
the strings batches 1-2 added. Release-note lines stay English outside
German.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:40:42 +00:00
Claude a7fcd81850 Fix #341: What's new
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:40:19 +00:00
Claude c3c30dae3f Fix #341: the test clicks the row's widget id
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:39:36 +00:00
Claude 47cf1ade6f Fix #341: the test window is tall enough to show the By Date rows
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:38:10 +00:00
Claude 00388f3fd2 Fix #341: test from Recently Deleted (the demo has picks)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:37:25 +00:00
Claude 1b4518c202 Fix #341: By Date and Keywords rows show their photos from All Photos
The rows count every photo in the library, but choosing one filtered
whatever source was open (an album, a folder, Picks), so it often showed
nothing. Choosing a row now switches to All Photos first; clearing it
leaves the source alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:36:25 +00:00
Adrian D'souzaandCodex d16ab1f238 PR218: check SQLite local payload span
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:52 +05:30
Adrian D'souzaandCodex bbcfdf085b PR218: harden catalog import review fixes
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex 1e6035d9db test: encode SQLite fixture serial types & overflow offsets correctly
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex 37cfc46e0d PR218: place prepared-import regression tests after production items
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex b256b8b635 PR218: bound egui worker message size & simplify cancellation
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex e8c817ddea PR218: satisfy current CI clippy without changing import behavior
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex d367d83ee6 LRC-LIB-CATALOG-IMPORT: exercise actual concurrent develop edit
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex 6a05f865ee LRC-LIB-CATALOG-IMPORT: refresh rebased parity summary
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Adrian D'souzaandCodex 866d02e577 LRC-LIB-CATALOG-IMPORT: address SQLite, worker & archive review
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Orthic LabsandCodex 4a427894ed LRC-LIB-CATALOG-IMPORT: import Lightroom Classic catalogs
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:05:35 +05:30
Claude aaa46003d5 Fix #316: the header keeps its hover shade while the pointer is on the eye
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:34:35 +00:00
Claude a9cd761d52 Fix #316: the section eye stays while the pointer is on it
The eye was drawn only while the header itself was hovered; with the
pointer on the eye, egui reports the eye (on top) as hovered and not the
header, so the eye vanished under the pointer and could not be clicked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:33:25 +00:00
Claude 68267810b3 Fix #316: headless test of the section eye
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:30:43 +00:00
Claude 9b4ba1aa80 Fix #316: the section eye switches off Light, Color and Detail too
The pipeline honoured a section's eye only where it applies Effects,
Optics, Geometry and Calibration, so turning the eye off on Light, Color
or Detail changed nothing. DevelopSettings::effective() resets the
switched-off sections to their defaults (Light with the tone curve; Color
with white balance, mixers, grading and point colour; Detail), and both the
CPU and GPU renderers render those. Borrowed, so free, when every section
is on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:29:29 +00:00
Claude 0634782fbc i18n: German and Russian follow-ups (#265, #264)
- #265's German coverage test failed whenever any catalog key, command or
  control label, or line of What's New lacked German, so every feature PR
  would have had to ship German. It reports those gaps for every language
  now (display_label_gaps_are_reported), as the project's other gap test
  does; docs/localization.md says how release-note lines are matched and
  that an English copy in a formats catalog marks an untranslated message.
- Russian: docs/localization-ru.md, the language lists in
  docs/localization.md and docs/parity.md, ru tag tests.
- What's new: German and Russian UI. Removes a status sentence #265 put
  in parity.md's Top gaps list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:26:56 +00:00
Claude 9b5e771fd8 Merge PR #264
# Conflicts:
#	crates/ui-egui/src/i18n.rs
#	crates/ui-egui/src/menus.rs
2026-10-08 18:25:10 +00:00
Claude 69384ad430 Merge PR #265
# Conflicts:
#	crates/ui-egui/locales/ja-formats.json
#	crates/ui-egui/locales/zh-hans-formats.json
#	crates/ui-egui/locales/zh-hant-formats.json
#	crates/ui-egui/src/lib.rs
2026-10-08 18:25:01 +00:00
Claude acc85c4147 Fix #322: the value field draws in a child UI and leaves the slider rows in place
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:22:43 +00:00
radialmonster e2d35de867 LR-IMP-ADD-DIALOG: file dialogs match extensions in any letter case on Linux (#342)
rfd turns each extension into a *.ext glob that GTK and the desktop portal
match case-sensitively, so DSC_1234.NEF was hidden. One helper
(dialog_filter) gives every dialog filter the lower, UPPER and bracket
forms on Linux; the Add Photos filter uses import::EXTENSIONS.
2026-10-08 14:21:58 -04:00
Claude 0fbf20db51 Fix #322: select the old value once the field has the focus
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:21:10 +00:00
Claude 2b66eb4800 Fix #322: the typed-value field takes the keyboard on the frames after the click
The click's own release took the focus straight back from a field created
in that frame, so typing went nowhere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:19:31 +00:00
Claude 92c7237c50 Fix #322: type an exact value into a slider
Click the value next to a slider's name to edit it: the old value starts
selected, Return or clicking away applies the typed number on the
control's steps and range as one undo step (like a keyboard nudge), and
Esc or text that isn't a number keeps the old value. A decimal comma,
a leading + and a % or K unit are accepted. The keys typed go to the
field, not to the shortcuts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 18:05:19 +00:00
radialmonster 34ecea4f24 LR-IMP-CAMERA-COVERAGE: read Canon ColorData stored as bytes, so PowerShot and EOS M CR2s get their as-shot white balance
Maker note tag 0x4001 is UNDEFINED bytes (ColorData versions -3 and -4) on
PowerShot and EOS M bodies. It was read as one number per byte, so no plausible
as-shot levels were found and the white balance was 1, 1, 1 (green cast, a* about
-13 on neutrals). Byte and undefined data is now paired into 16-bit words in the
maker note's byte order before the existing probe (word 71). No per-model table.

RENDER_CACHE_VERSION 11 -> 12.
2026-10-08 14:03:23 -04:00
radialmonster ff8d027766 LR-IMP-FORMATS: NEF lossy type 2 curve of version 0x44 0x40 spans a quarter of the sample range
Maker note 0x0096 of the D780, D6, Z 5, Z 6, Z 7 and Z 7_2 starts 0x44 0x40
and holds 257 points, but its first interval rises by 2^bits / 1024 where the
0x44 0x20 tables rise by 2^bits / 256: the curve is the identity in the dark
part and covers 2^bits / 4 codes, not 2^bits. The decoder spaced the points
2^bits / 256 codes apart for both, so these files decoded to a fraction of
their values, far below the black level.

parse_table now takes the span from the version (0x20: all codes, 0x40: a
quarter). Other versions decode to identical samples. A known-answer test
built from hand-coded table bytes and bit strings fails without the change.
Bumps RENDER_CACHE_VERSION to 12.
2026-10-08 14:02:01 -04:00
Claude 8b70d33d0f Fix #319: release builds pin craft-fonts with Noto Sans CJK SC
release.yml (five jobs) and freebsd.yml pinned craft-fonts abb83316, from
before Noto Sans CJK SC landed there, so v0.4.0 embedded only the Japanese
faces and most of the Simplified Chinese UI was tofu. The pin is now
8dcdacd5 (craft-fonts HEAD: Noto Sans CJK SC and Noto Sans Arabic), which
the Noto CJK exception merged from #203 allows. docs/releasing.md says when
to bump the pin and how to check it before a release.

flake.nix / flake.lock (community-maintained, not in CI) still pin the old
revision: the lock's narHash needs nix to recompute.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
2026-10-08 17:32:09 +00:00
Claude 5e2ece0cf3 Merge PR #203 (Noto CJK allowed via craft-fonts) 2026-10-08 17:31:36 +00:00
MarekSpodBiedry f2bcc9219c Issue #234: Windows opens links (Map button) in the browser, not File Explorer
explorer <url> opened File Explorer for links with a query or fragment, such as the OpenStreetMap link behind the Map button. Windows now opens https links with rundll32 url.dll,FileProtocolHandler, which hands the URL to the default browser without a shell.
2026-10-08 18:59:04 +02:00
Joshua Warren 0613f925de Support remote SAM3 inference through a private Metal worker 2026-10-08 11:47:59 -05:00
Walter Boringandopencode 853ada7754 LR-IMP-FORMATS: HEIC/HEIF decode behind the optional heif feature
New crate `lightcraft-heif` (a port of PhotoCraft's photocraft-heif): a thin,
panic-guarded wrapper around heic-rs (pure Rust, pinned =0.1.1) - 8/10-bit,
alpha, grid tiles, container irot/imir/clap, ICC/EXIF/XMP. `lightcraft-codecs`
decodes HEIF behind its non-default `heif` feature (HEVC patents are the
distributor's call, the same policy as PhotoCraft); the apps forward it. HEIC
files stay recognised without the feature: opening one is a clear
"isn't included in this build" error. Fuzz-found heic-rs panics arrive as
regression fixtures (inline bytes, no media committed).

Verified end-to-end on real iPhone HEICs (lightcraft-cli render, rotation and
metadata correct). Note for reviewers: measured here, the engine currently
ignores the decoded image's source-space matrix on export for ANY format (a
true Display-P3 PNG round-trips unchanged); the ICC and space are carried for
all codecs, so this inherits - and does not worsen - existing behaviour.

Co-Authored-By: opencode <noreply@opencode.ai>
2026-10-08 12:45:33 -04:00
sebastian pfluegelmeier b4ca9f7439 M11.4: harden RAF validation and camera profile fallback 2026-10-08 11:23:55 +02:00
sebastian pfluegelmeier cfcc169ff1 M11.4: calibrate Fujifilm colour with bundled X-H2S and X-T4 profiles 2026-10-08 10:49:55 +02:00
sebastian pfluegelmeier 5923be31d6 M11.1: decode Fujifilm lossless and lossy compressed RAF 2026-10-08 10:44:09 +02:00
MarcelandCodex 35dd973d91 M16: add German localization using the existing catalogs
Add German language selection and persisted locale support, complete German
label and format catalogs, and German release notes. Connect previously
untranslated UI text, preserve user names, and size translated buttons to
fit. Format cached import source facts in the active UI language.

Validate locale coverage, placeholder formats, language persistence,
German glyphs, button bounds and cached import language switching. Review
native and headless screenshots; run all seven cargo xtask ci gates.

Co-Authored-By: Codex (GPT-6) <noreply@openai.com>
2026-10-08 07:27:32 +02:00
Wondarkus 49868693d5 Add Russian language support in i18n.rs 2026-10-08 08:23:30 +04:00
Wondarkus 79fbb6b464 Add Russian language support in i18n.rs 2026-10-08 08:21:31 +04:00
Wondarkus 95b57d3786 Add Russian language option to menus 2026-10-08 08:20:17 +04:00
Wondarkus 782ade6c39 Add files via upload 2026-10-08 08:16:54 +04:00
Wondarkus 11d6676984 Delete crates/ui-egui/locales/ru-formats.json 2026-10-08 08:15:15 +04:00
Wondarkus 841850d516 Delete crates/ui-egui/locales/ru.json 2026-10-08 08:14:58 +04:00
Wondarkus 6c7405ed3e Fix Russian translations in ru.json 2026-10-08 07:06:23 +04:00
Wondarkus 6a34a1f8b8 Add files for further editing 2026-10-08 06:56:08 +04:00
Brandon ThomasandClaude Opus 5.5 8148988eb8 Docs: allow Noto CJK (via craft-fonts) for Chinese text
Maintainer decision: Noto Sans/Serif CJK is Google-branded Source Han
(Adobe co-developed); nearly every OFL Chinese face derives from it, and the
Simplified/Traditional Chinese UI (#184, #198) uses Noto Sans CJK SC from
craft-fonts. Record it as a narrow, explicit exception to the Adobe-asset rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:16:43 -04:00
244 changed files with 45604 additions and 1327 deletions

No files matched your search

+1 -1
View File
@@ -31,7 +31,7 @@ jobs:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695
ref: 8dcdacd5153e64560d109541a47d806f26f048c0
path: craft-fonts
- name: Build and test in a FreeBSD VM
uses: vmactions/freebsd-vm@c46abacb49f09938ca4e1702d15d836285d694cc # v1
+5 -5
View File
@@ -77,7 +77,7 @@ jobs:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
@@ -135,7 +135,7 @@ jobs:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
@@ -192,7 +192,7 @@ jobs:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- name: Install build and packaging tools
run: |
@@ -287,7 +287,7 @@ jobs:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- name: Build and package in a FreeBSD VM (tar.gz)
# vmactions/freebsd-vm v1.5.9, pinned by commit (it runs our build).
@@ -336,7 +336,7 @@ jobs:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
ref: 8dcdacd5153e64560d109541a47d806f26f048c0 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
+5 -2
View File
@@ -63,7 +63,10 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
- **Never use any iconography, image, artwork, font, sound or other asset from Adobe products** (no Lightroom/Creative Cloud icons, no screenshots, no presets/profiles/LUTs, no UI bitmaps — not even as a temporary placeholder or "reference copy"). Observing Adobe's UI to imitate *layout and behaviour* is allowed; copying or tracing its assets is not.
- **This includes Adobe's open-licensed assets**: no Source Sans/Serif/Code or Source Han fonts, no Adobe Fonts, no
Adobe-published icon sets, sample photos, colour profiles or LUTs — even when OFL/MIT. The UI font is Inter (OFL);
Japanese fonts come from craft-fonts (below).
Japanese and Chinese fonts come from craft-fonts (below). **One exception (maintainer decision, 2026-10-07): Noto
Sans/Serif CJK** (Google-branded, OFL, co-developed with Adobe as Source Han) is allowed via craft-fonts for Chinese
text, because nearly every OFL Chinese face derives from it. Use it unmodified under its OFL; this does not open the
door to Source Han under Adobe's name or to any other Adobe asset.
- Every asset in the repository must be one of: **our own original work** (e.g. icons drawn in code as vectors, procedurally generated demo photos), **public domain / CC0**, **Creative Commons** (CC-BY / CC-BY-SA with attribution honoured), **OFL** (fonts), or **permissive open-source** (MIT/Apache-2.0/BSD/ISC) — or contributed by a person who created the asset and licenses it openly.
- **Exception: `docs/brand/`.** The ArtCraft name, wordmark and logos there are ArtCraft Team trademarks, not open source and not covered by LightCraft's MIT OR Apache-2.0 licence (`LICENSE-MIT`, `LICENSE-APACHE`, `NOTICE`); their terms are in `docs/brand/LICENSE-brand.txt`. Use them only unmodified and never redraw, recolour or derive from them.
- **Every asset must have an entry in `assets/ATTRIBUTION.md`** (path, title, author/creator, source URL or "original work", licence, date added, modifications) and its licence text when required (e.g. `assets/fonts/OFL-*.txt`). Add the entry in the same commit as the asset. Assets without an attribution entry must not be committed.
@@ -72,7 +75,7 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
uses it as the optional build input `CRAFT_FONTS_DIR`: `git clone https://github.com/storytold/craft-fonts ../craft-fonts`
then `CRAFT_FONTS_DIR=../craft-fonts cargo run -p lightcraft` (or any cargo/xtask command). `crates/engine/build.rs`
embeds the manifest's fonts as `lightcraft_engine::CRAFT_FONTS` (wasm32: BIZ UDPGothic Regular only); the UI
(`theme::font_definitions`) and the export watermark renderer use its Japanese faces as fallbacks after Inter. Unset,
(`theme::font_definitions`) and the export watermark renderer use its CJK faces (picked by script) as fallbacks after Inter. Unset,
`CRAFT_FONTS` is empty: everything builds, tests and runs, but Japanese text has no glyphs. Releases always build
with it (`release.yml`, `CRAFT_FONTS_REQUIRED=1`) and ship the fonts' OFL licences. Tests that need these fonts skip
without it; the FreeBSD CI job runs them with it. Rules: craftrules
+5 -2
View File
@@ -63,7 +63,10 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
- **Never use any iconography, image, artwork, font, sound or other asset from Adobe products** (no Lightroom/Creative Cloud icons, no screenshots, no presets/profiles/LUTs, no UI bitmaps — not even as a temporary placeholder or "reference copy"). Observing Adobe's UI to imitate *layout and behaviour* is allowed; copying or tracing its assets is not.
- **This includes Adobe's open-licensed assets**: no Source Sans/Serif/Code or Source Han fonts, no Adobe Fonts, no
Adobe-published icon sets, sample photos, colour profiles or LUTs — even when OFL/MIT. The UI font is Inter (OFL);
Japanese fonts come from craft-fonts (below).
Japanese and Chinese fonts come from craft-fonts (below). **One exception (maintainer decision, 2026-10-07): Noto
Sans/Serif CJK** (Google-branded, OFL, co-developed with Adobe as Source Han) is allowed via craft-fonts for Chinese
text, because nearly every OFL Chinese face derives from it. Use it unmodified under its OFL; this does not open the
door to Source Han under Adobe's name or to any other Adobe asset.
- Every asset in the repository must be one of: **our own original work** (e.g. icons drawn in code as vectors, procedurally generated demo photos), **public domain / CC0**, **Creative Commons** (CC-BY / CC-BY-SA with attribution honoured), **OFL** (fonts), or **permissive open-source** (MIT/Apache-2.0/BSD/ISC) — or contributed by a person who created the asset and licenses it openly.
- **Exception: `docs/brand/`.** The ArtCraft name, wordmark and logos there are ArtCraft Team trademarks, not open source and not covered by LightCraft's MIT OR Apache-2.0 licence (`LICENSE-MIT`, `LICENSE-APACHE`, `NOTICE`); their terms are in `docs/brand/LICENSE-brand.txt`. Use them only unmodified and never redraw, recolour or derive from them.
- **Every asset must have an entry in `assets/ATTRIBUTION.md`** (path, title, author/creator, source URL or "original work", licence, date added, modifications) and its licence text when required (e.g. `assets/fonts/OFL-*.txt`). Add the entry in the same commit as the asset. Assets without an attribution entry must not be committed.
@@ -72,7 +75,7 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
uses it as the optional build input `CRAFT_FONTS_DIR`: `git clone https://github.com/storytold/craft-fonts ../craft-fonts`
then `CRAFT_FONTS_DIR=../craft-fonts cargo run -p lightcraft` (or any cargo/xtask command). `crates/engine/build.rs`
embeds the manifest's fonts as `lightcraft_engine::CRAFT_FONTS` (wasm32: BIZ UDPGothic Regular only); the UI
(`theme::font_definitions`) and the export watermark renderer use its Japanese faces as fallbacks after Inter. Unset,
(`theme::font_definitions`) and the export watermark renderer use its CJK faces (picked by script) as fallbacks after Inter. Unset,
`CRAFT_FONTS` is empty: everything builds, tests and runs, but Japanese text has no glyphs. Releases always build
with it (`release.yml`, `CRAFT_FONTS_REQUIRED=1`) and ship the fonts' OFL licences. Tests that need these fonts skip
without it; the FreeBSD CI job runs them with it. Rules: craftrules
Generated
+80
View File
@@ -2296,6 +2296,15 @@ version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "heic-rs"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "adf9dde634bb6d5b718bacd01bef80192f2334306b2f785519a0136cca7b9618"
dependencies = [
"rayon",
]
[[package]]
name = "hermit-abi"
version = "0.5.3"
@@ -2837,6 +2846,7 @@ dependencies = [
"jxl-oxide",
"lightcraft-color",
"lightcraft-geom",
"lightcraft-heif",
"lightcraft-raster",
"moxcms 0.9.1",
"png",
@@ -2882,6 +2892,8 @@ dependencies = [
"lightcraft-codecs",
"lightcraft-color",
"lightcraft-develop",
"lightcraft-faces",
"lightcraft-fetch",
"lightcraft-geom",
"lightcraft-gpu",
"lightcraft-merge",
@@ -2892,12 +2904,28 @@ dependencies = [
"lightcraft-raw",
"lightcraft-scenes",
"lightcraft-segment",
"lightcraft-tiff",
"log",
"miniz_oxide 0.8.9",
"rayon",
"serde",
"serde_json",
"thiserror 2.0.21",
"unicode-segmentation",
"web-time",
]
[[package]]
name = "lightcraft-faces"
version = "0.4.0"
dependencies = [
"matrixmultiply",
"ndarray",
"proptest",
"serde",
"serde_json",
"sha2",
"thiserror 2.0.21",
"web-time",
]
@@ -2936,6 +2964,13 @@ dependencies = [
"wgpu",
]
[[package]]
name = "lightcraft-heif"
version = "0.4.0"
dependencies = [
"heic-rs",
]
[[package]]
name = "lightcraft-mcp"
version = "0.4.0"
@@ -2984,6 +3019,7 @@ dependencies = [
"lightcraft-develop",
"lightcraft-geom",
"lightcraft-raster",
"lightcraft-raw",
"lightcraft-scenes",
"proptest",
"rayon",
@@ -3020,6 +3056,7 @@ name = "lightcraft-raw"
version = "0.4.0"
dependencies = [
"jpeg-encoder",
"jxl-oxide",
"lightcraft-color",
"lightcraft-geom",
"lightcraft-meta",
@@ -3029,9 +3066,11 @@ dependencies = [
"proptest",
"rayon",
"serde",
"sha2",
"thiserror 2.0.21",
"zune-core",
"zune-jpeg",
"zune-jpegxl",
]
[[package]]
@@ -3081,6 +3120,7 @@ dependencies = [
"lightcraft-color",
"lightcraft-develop",
"lightcraft-engine",
"lightcraft-faces",
"lightcraft-geom",
"lightcraft-meta",
"lightcraft-pipeline",
@@ -3171,6 +3211,16 @@ dependencies = [
"libc",
]
[[package]]
name = "matrixmultiply"
version = "0.3.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f607c237553f086e7043417a51df26b2eb899d3caff94e6a67592ff992fedc7"
dependencies = [
"autocfg",
"rawpointer",
]
[[package]]
name = "maybe-rayon"
version = "0.1.1"
@@ -3324,6 +3374,21 @@ dependencies = [
"thiserror 2.0.21",
]
[[package]]
name = "ndarray"
version = "0.17.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "520080814a7a6b4a6e9070823bb24b4531daac8c4627e08ba5de8c5ef2f2752d"
dependencies = [
"matrixmultiply",
"num-complex",
"num-integer",
"num-traits",
"portable-atomic",
"portable-atomic-util",
"rawpointer",
]
[[package]]
name = "ndk"
version = "0.9.0"
@@ -4542,6 +4607,12 @@ dependencies = [
"objc2-quartz-core 0.3.2",
]
[[package]]
name = "rawpointer"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "60a357793950651c4ed0f3f52338f53b2f809f32d83a07f72909fa13e4c6c1e3"
[[package]]
name = "rayon"
version = "1.12.0"
@@ -6749,6 +6820,15 @@ dependencies = [
"zune-core",
]
[[package]]
name = "zune-jpegxl"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cba11ecd07cc23351500e840ae03841b7e4997923ec8e4832ee3ae199ea0b6b4"
dependencies = [
"zune-core",
]
[[package]]
name = "zvariant"
version = "5.15.0"
+4
View File
@@ -12,6 +12,7 @@ repository = "https://github.com/storytold/lightcraft"
[workspace.dependencies]
# internal
lightcraft-geom = { path = "crates/geom" }
lightcraft-faces = { path = "crates/faces" }
lightcraft-sysmem = { path = "crates/sysmem" }
lightcraft-color = { path = "crates/color" }
lightcraft-raster = { path = "crates/raster" }
@@ -30,10 +31,12 @@ lightcraft-engine = { path = "crates/engine" }
lightcraft-ui-egui = { path = "crates/ui-egui" }
lightcraft-mcp = { path = "crates/mcp" }
lightcraft-fetch = { path = "crates/fetch" }
lightcraft-heif = { path = "crates/heif" }
lightcraft-segment = { path = "crates/segment" }
# foundation
thiserror = "2"
sha2 = "0.10"
serde = { version = "1", features = ["derive", "rc"] }
serde_json = { version = "1", features = ["float_roundtrip"] }
log = "0.4"
@@ -61,6 +64,7 @@ tiff = "0.11"
image-webp = "0.2"
moxcms = { version = "0.9", default-features = false, features = ["lut"] }
jxl-oxide = { version = "0.12", default-features = false }
zune-jpegxl = { version = "0.5", default-features = false, features = ["std"] } # MIT/Apache-2.0/Zlib lossless JXL encoder (test fixtures only)
ravif = { version = "0.13", default-features = false, features = ["threading"] } # no "asm": it needs nasm (non-Rust)
# gpu (crates/gpu; native backends only, no GL: everything stays pure Rust)
+10 -8
View File
@@ -258,9 +258,10 @@ lightcraft --control 7980 ~/Pictures/trip
## Fast, native, private
- **Pure Rust, no C.** Our own RAW decoders (DNG, Canon CR2, Sony ARW, Nikon NEF, Fujifilm RAF incl. X-Trans,
- **Pure Rust, no C.** Our own RAW decoders (DNG, Canon CR2/CR3, Sony ARW, Nikon NEF, Fujifilm RAF incl. X-Trans,
Panasonic RW2 / Leica RWL, Pentax PEF, Olympus ORF), our own colour science, our own pipeline. JPEG, PNG, TIFF, WebP,
PSD composites and JPEG XL open today.
PSD composites and JPEG XL open today; HEIC/HEIF photos too with `--features heif` (HEVC is a build-time choice,
as in PhotoCraft).
- **Scene-referred & wide-gamut.** Linear Rec.2020 float internally, Bradford-adapted white balance, gamut mapping
instead of clipping, a filmic shoulder for raw and pixel-exact pass-through for JPEGs you haven't touched.
- **Resolution-independent edits.** Radii and brush sizes are relative to the image, so a 400 px preview, your
@@ -288,8 +289,8 @@ LightCraft is young and moving fast. **Where we honestly stand** (details in the
- **As a day-to-day Lightroom replacement we're nearer 60–70%.** It's great for JPEG/DNG and most Nikon / Sony /
older-Canon raws on one machine.
- **The biggest gaps:**
- **camera colour calibration:** raws other than DNG develop with a neutral colour matrix today, so colour is muted;
- **CR3 and compressed Fujifilm / Olympus raws:** these open as embedded previews only;
- **camera colour calibration:** Sony, Nikon, Panasonic, Fujifilm and Canon CR3 raws have guarded estimates from their camera JPEGs, with built-in ILCE-7M4, X-H2S and X-T4 profiles; measured calibration is missing, and other raws or rejected fits retain a neutral matrix;
- **compressed Olympus raws and unsupported CR3 variants:** these use embedded JPEG previews when present. Fujifilm lossless/lossy compressed RAF now decodes sensor data; [verification and existing-library reload instructions](docs/raf-compression.md);
- **AI masks and denoise:** subject and sky selection are classical heuristics;
- **HDR, video and the Classic Print / Book / Map modules.**
- **What's next:** see [where we're going](ROADMAP.md#where-were-going).
@@ -302,9 +303,9 @@ LightCraft is young and moving fast. **Where we honestly stand** (details in the
| Masking: brush, linear, radial, luminance/colour range, add/subtract/intersect | ✅ (AI subject/sky use classical heuristics for now) |
| Crop, straighten tool + auto straighten, flip, rotate, aspect ratios, overlays | ✅ |
| Profiles (Color, Neutral, Vivid, Landscape, Portrait, Monochrome: our own looks), presets, versions, history, copy/paste/sync settings | ✅ |
| Camera colour: DNG files use their own matrices | ✅ · our own calibration for other raws ⬜ (top priority; ARW, NEF and RW2 start from a look fitted to their own JPEG, other raws from a neutral fallback) |
| Camera colour: DNG files use their own matrices | ✅ DNG · 🟡 own Sony/Fujifilm profiles; measured calibration database missing |
| Native macOS menu bar (generated from the command registry), control channel + every widget addressable, headless UI snapshots | ✅ |
| RAW: DNG, CR2, ARW, NEF (uncompressed + lossless/lossy compressed), Fujifilm RAF (uncompressed, Bayer + X-Trans), Panasonic RW2 / Leica RWL / Panasonic RAW (every raw format, DMC-LX1 to DC-S1RM2), Pentax PEF, Olympus ORF (uncompressed); embedded previews for every format incl. CR3 | ✅ · CR3, compressed RAF/ORF decode ⬜ |
| RAW: DNG, CR2, CR3 (lossless CRX Bayer and version 0x100/0x200 C-RAW), ARW, NEF (uncompressed + lossless/lossy compressed), Fujifilm RAF (uncompressed + lossless/lossy compressed, Bayer + X-Trans), Panasonic RW2 / Leica RWL / Panasonic RAW (every raw format, DMC-LX1 to DC-S1RM2), Pentax PEF, Olympus ORF (uncompressed); embedded previews for every format incl. CR3 | 🟡 · CR3, compressed ORF decode ⬜ |
| Detail: sharpening, luminance + colour noise reduction | ✅ · AI Denoise, Super Resolution ⬜ |
| Remove / Heal / Clone spots (auto source), Visualize Spots, Red Eye and Pet Eye (auto pupil detection, catchlight) | ✅ · content-aware fill, spot pin editing 🚧 |
| Export: JPEG / PNG / TIFF / WebP / AVIF / DNG / original, sizing, file-size limit, output sharpening, naming templates, batch, metadata policy, text or image watermark | ✅ · HDR export ⬜ |
@@ -353,13 +354,14 @@ CRAFT_FONTS_DIR=../craft-fonts cargo run --release -p lightcraft
Without it LightCraft builds and runs the same, but Chinese and Japanese text have no glyphs. Fonts are never committed to this
repo; see [craftrules `standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md).
**Interface language:** **Edit → Language** (English, 简体中文, 繁體中文(台灣), 日本語) or **Settings → General →
**Interface language:** **Edit → Language** (English, 简体中文, 繁體中文(台灣), 日本語, Português (Brasil), Deutsch) or **Settings → General →
Language**; the choice applies immediately and persists. See [docs/localization.md](docs/localization.md).
**Logs:** the desktop app writes its log to standard error and to `logs/lightcraft.log` in its settings folder
(Linux `$XDG_CONFIG_HOME/lightcraft/logs/`, by default `~/.config/lightcraft/logs/`; macOS
`~/Library/Application Support/LightCraft/logs/`; Windows `%APPDATA%\LightCraft\logs\`), never in the library. A
launch from a desktop menu or the Dock has no terminal, so attach this file to a bug report. Each start moves the
launch from a desktop menu or the Dock has no terminal, so attach this file to a bug report (**Help ▸ Open Log
Folder** shows it in the file manager). Each start moves the
previous log to `lightcraft.1.log` and that one to `lightcraft.2.log`, so the log of a run that crashed survives the
next start; the file stops growing at 16 MiB, `--version` and `--help` write none, and runs with
`LIGHTCRAFT_NO_PREFS` log to standard error only. By default LightCraft's own crates log at `info` and everything else
+40 -22
View File
@@ -6,7 +6,7 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi
## Where we stand
*Honest assessment, 2026-10-05. Agents: read this before picking work. The checklist in
*Honest assessment, 2026-10-08. Agents: read this before picking work. The checklist in
[`docs/parity.md`](docs/parity.md) counts features that **exist**; this section is about whether a photographer can
**switch** from Lightroom. Update it when a gap below closes.*
@@ -23,13 +23,13 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
| Dimension | Estimate | What's true today | Biggest gaps |
|---|---:|---|---|
| **Feature checklist** | 79.7% | P0 core and P1 nearly complete: import (Add / Copy / Move, templates, devices), library, grid/loupe/compare/survey, every Edit slider, curves, colour grading, masking tools, crop/Upright, heal/clone, presets/profiles, versions/history, sync, export, menus, shortcuts | P0: full-resolution zoom detail; P1: lens-profile database, content-aware fill (patch synthesis), video playback/trim |
| **RAW coverage** (formats people shoot) | ~55% | DNG (all kinds), CR2, ARW, NEF (uncompressed + Huffman lossless/lossy), uncompressed RAF/ORF, RW2 / RWL / Panasonic RAW (every raw format, checked on 178 files from 118 Panasonic and Leica bodies), PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed RAF/ORF, Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. NEFs labelled compressed but stored uncompressed (Z 6 packed 14-bit, D850 12-bit uncompressed). Per-model verification is thin (~55 corpus files vs >1,000 models) |
| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW, NEF and RW2 have a guarded per-file embedded-JPEG colour estimate (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
| **AI & computational** | ~15–20% | Assisted culling (focus, bursts), auto tone, HDR/panorama merge; subject/sky/background masks as classical heuristics | Real segmentation masks (subject, sky, people, objects, landscape, depth), AI denoise, super resolution, lens blur, generative remove, faces/people, natural-language search. **Blocked on a model strategy** (licensable weights or our own training; pure-Rust inference is feasible) |
| **RAW coverage** (formats people shoot) | ~55% | DNG (all kinds), CR2, CR3 lossless Bayer and version 0x100/0x200 C-RAW (M50/R100/R8 pixel-exact corpus), ARW, NEF (uncompressed + Huffman lossless/lossy), RAF (uncompressed, lossless/lossy compressed Bayer and X-Trans), uncompressed ORF, RW2 / RWL / Panasonic RAW (every raw format, checked on 178 files from 118 Panasonic and Leica bodies), PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed ORF, Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. NEFs labelled compressed but stored uncompressed (Z 6 packed 14-bit, D850 12-bit uncompressed). Per-model verification is thin (~55 corpus files vs >1,000 models) |
| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW, NEF, RW2, RAF, CR3, CR2 and PEF have guarded camera-JPEG colour estimates, with bundled ILCE-7M4, X-H2S and X-T4 profiles (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
| **AI & computational** | ~15–20% | Assisted culling (focus, bursts), auto tone, HDR/panorama merge; subject/sky/background masks as classical heuristics, or the photo's own segmentation mattes when it ships them (DNG semantic masks, e.g. iPhone ProRAW's sky matte); **faces (MVP, [docs/faces.md](docs/faces.md))**: a pure-Rust detector (a 232 KB opt-in download), opt-in recognition models (one-click download or add a file; they run on the shared pure-Rust CPU interpreter; pinned SFace/AuraFace embeddings agree with an independent reference runtime on 40 public crops; one-thread medians 34–36 / 233–252 ms (AuraFace remains expensive)), a background scan that finds and embeds faces, name suggestions, a person's page with look-alike faces to confirm, an Unnamed faces section (look-alikes together, select a group and name it at once) | Real segmentation masks (subject, sky, people, objects, landscape, depth), AI denoise, super resolution, lens blur, generative remove, natural-language search; for faces: clusters with separators between groups, small faces in large photos (the detector looks at 640 px), thresholds calibrated on live faces, writing face regions back to XMP. **Blocked on a model strategy** (licensable weights or our own training; pure-Rust inference is feasible) |
| **Workflow & library** | ~85% (single machine) | Robust catalog (journal + snapshots, background compaction, crash-tested), 85k-photo libraries stay responsive, Local browsing with automatic cleanup, XMP interop, keywords, smart albums, Move import | Opening an 85k library takes 1.7–4.7 s; no cloud sync (out of scope), no tablet companion (#74, roadmap), shared albums, publish services, tethering |
| **Classic modules** | ~30% | Geotagging from GPX track logs, soft proofing (partial), slideshow (basic) | **Map view, Book, Print, Slideshow module, Web, publish services**: ~40 tracker rows ⬜ |
| **HDR & video** | 0% | | HDR edit/display/export; video play/trim/edit/export |
| **Platform & robustness** | ~70% | macOS native; Windows/Linux builds; web via WASM; no-panic lints workspace-wide, `unsafe` confined to `crates/sysmem`; failed saves are reported; GPU errors fall back to CPU | Windows installer UI unverified on Windows (PR #79); GPU path proven only on Apple + user reports; English, Simplified/Traditional Chinese and Japanese UI (see docs/localization.md); remaining technical errors; Traditional Chinese uses the Simplified Chinese font until craft-fonts has a TC face; accessibility partial; headless UI tests time out under machine load |
| **Platform & robustness** | ~70% | macOS native; Windows/Linux builds; web via WASM; no-panic lints workspace-wide, `unsafe` confined to `crates/sysmem`; failed saves are reported; GPU errors fall back to CPU | Windows installer UI unverified on Windows (PR #79); GPU path proven only on Apple + user reports; English, Simplified/Traditional Chinese, Japanese, Brazilian Portuguese and German UI (see docs/localization.md); remaining technical errors; Traditional Chinese uses the Simplified Chinese font until craft-fonts has a TC face; accessibility partial; headless UI tests time out under machine load |
### By kind of user
@@ -37,7 +37,8 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
|---|---:|---|
| JPEG / DNG shooter, single machine | ~85% | Fidelity polish, AI masks |
| Nikon / Sony / Panasonic / older-Canon raw shooter | ~65% | Camera colour fidelity and coverage (ARW, NEF and RW2 preview estimates are only a starting point); Panasonic compacts and kit zooms also miss their embedded distortion correction |
| Canon CR3 / Fujifilm / Olympus shooter | ~35% | Their raws open as embedded previews only (CR3: the full-size JPEG with full metadata) |
| Fujifilm raw shooter | ~65% | Uncompressed and lossless/lossy compressed RAF decoded; camera colour calibration remains missing. Verified on 13 bodies, including 14/16-bit GFX; see `docs/raf-compression.md` |
| Canon CR3 / Olympus compressed-raw shooter | ~45% | CR3 lossless Bayer and version 0x100/0x200 C-RAW develop from sensor data (M50, R100, R8 verified pixel for pixel); other CRX variants and compressed ORF open as embedded JPEG previews. Camera colour and broader model verification remain |
| Lightroom Classic power user | ~45% | Print, Book, Map, publish, tethering |
| Relies on AI (masks, denoise) | ~25% | Object / Describe masks via optional SAM 3 (download not yet hosted); no AI denoise |
@@ -46,9 +47,10 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
Priorities, in order. Each points at tracker rows in [`docs/parity.md`](docs/parity.md) → *Top gaps*.
1. **Camera colour calibration of our own** (LR-PROF-CAMERACOLOR, P0): fit each camera to its own embedded JPEG, use
matrices the files carry themselves, then chart shots. Sony ARW, Nikon NEF and Panasonic RW2 have the file-local fit
(matrix + tone curve from their own JPEG); generalise it to the other makes' raws (PEF, ORF…), then validate fidelity.
2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): **CR3** first, then compressed RAF / ORF, NEF
matrices the files carry themselves, then chart shots. Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3/CR2
and Pentax PEF have the file-local fit (matrix + tone curve from their own JPEG; CR2 and PEF since issue #310, not yet
checked on the corpus); then ORF, then validate fidelity.
2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): Remaining **unverified CR3 CRX** variants, compressed ORF, NEF
lossy-after-split, sRAW. Decided 2026-10-05: write our own decoders from prose descriptions (never decoder source,
no LGPL dependency); compressed NEF (#86) is the template.
3. **Verified camera coverage** (LR-IMP-CAMERA-COVERAGE, P0): a CC0 sample per model in the corpus, each decoded and
@@ -57,7 +59,7 @@ Priorities, in order. Each points at tracker rows in [`docs/parity.md`](docs/par
(references stay in the local `plan/`), then tune against the numbers.
5. **Lens profiles of our own** (LR-EDIT-OPTICS-PROFILE, P1).
6. **AI model strategy** (maintainer decision): which permissively licensed models (or our own training) for
segmentation masks and denoise; then pure-Rust inference. Unblocks M12 and Enhance. Object / Describe masks now
segmentation masks and denoise; then pure-Rust inference. Unblocks M12 and Enhance. Face detection and recognition run on our own checked Rust engine. Object / Describe masks now
run Meta's SAM 3 in pure Rust (optional, user-consented download under the non-OSI SAM License; CDN mirrors still
to be set up, docs/ai-masks.md); Subject / Sky / People and Enhance are still open.
7. **Then:** HDR (Q), the Classic output modules (Print first, then Map view, Book, Slideshow), video (R), localisation
@@ -88,14 +90,14 @@ yet guarantee original-pixel detail; visible-region rendering remains a gap (LR-
| M8 | Heal / Remove | content-aware remove (PatchMatch), heal, clone, brush spots, visualize spots, red/pet eye | 6–10 | 🚧 (heal, clone, auto source, visualize spots, red/pet eye ✅; PatchMatch remove ⬜) |
| M9 | Presets, profiles, versions, sync | preset browser + amount, create/import presets, profile browser, versions, history, copy/paste/sync settings | 5–8 | ✅ |
| M10 | Export & share | export dialog (JPEG/PNG/TIFF/DNG/AVIF/JXL/original), sizing, sharpening, metadata, watermark, naming, batch jobs, XMP sidecars, HDR export | 6–10 | 🚧 (all formats incl. DNG/original, sizing, presets, background jobs ✅; JXL encode, HDR export ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 (every raw format), PEF, ORF uncompressed ✅; **camera colour calibration** 🚧 (guarded ARW, NEF and RW2 preview fitting; measured database still missing), CR3, compressed ORF/RAF ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (CR3 lossless CRX and C-RAW, RAF uncompressed and lossless/lossy compressed, RW2 (every raw format), PEF, ORF uncompressed ✅; **camera colour calibration** 🚧 (guarded ARW, NEF, RW2 and RAF preview fitting; measured database still missing), CR3, compressed ORF ⬜) |
| M12 | AI & smart features | subject/sky/background/people/object masks, semantic search, faces/People (permissively licensed models, pure-Rust inference) | 20–40 | ⬜ |
| M13 | Merge | HDR merge (deghost), panorama (projections, boundary warp, fill edges), HDR panorama | 10–15 | ✅ |
| M14 | Video | import/playback/trim via FilmCraft crates, global edits + presets on video, video export | 6–10 | ⬜ |
| M15 | Classic modules | Map, Book, Slideshow, Print, Web; smart collections, stacks, virtual copies, publish services, tethering | 25–40 | 🚧 (smart albums, stacks, virtual copies, compare/survey ✅; Map/Book/Slideshow/Print/Web ⬜) |
| M16 | 1.0 polish | preferences, shortcut editor, accessibility, localization, packaging (dmg/msi/AppImage/web), hardening | 10–20 | 🚧 (settings, keyboard shortcuts sheet, packaging basics ✅; English/Chinese (Simplified, Traditional)/Japanese localisation 🟡; accessibility and further locales ⬜) |
| M16 | 1.0 polish | preferences, shortcut editor, accessibility, localization, packaging (dmg/msi/AppImage/web), hardening | 10–20 | 🚧 (settings, keyboard shortcuts sheet, packaging basics ✅; English/Chinese (Simplified, Traditional)/Japanese/Brazilian Portuguese/German localisation 🟡; accessibility and further locales ⬜) |
## Parity estimate (feature count updated 2026-10-05; effort estimate from 2026-10-02)
## Parity estimate (feature count updated 2026-10-08; effort estimate from 2026-10-02)
**By feature count** — from `docs/parity.md` (one row per Lightroom feature, menu item and shortcut; `cargo xtask
parity` prints this line on every run, so it stays current):
@@ -122,7 +124,7 @@ trip, slideshow, auto import…) in ≈ 12 h on 2026-10-02; four to six parallel
| Work package | Tracker rows | Agent-hours | Risk |
|---|---|---:|---|
| Remaining P0/P1 UI and library features (folder rename/move, keyword painter, people view…) | ≈ 8 | 5–10 | low |
| Raw codecs: CR3 (CRX), compressed ORF / RAF, NEF lossy-after-split, HEIC/AVIF decode, JPEG XL DNG | LR-IMP-FORMATS | 40–80 | **high** — clean-room black-box analysis, no permissive specs |
| Raw codecs: CR3 (CRX), compressed ORF, NEF lossy-after-split, HEIC/AVIF decode | LR-IMP-FORMATS | 40–80 | **high** — clean-room black-box analysis, no permissive specs |
| Lens-profile database of our own (calibration targets, fitting, data) | LR-EDIT-OPTICS-PROFILE | 15–30 | data collection |
| Video: playback, trim, edits, export (pure-Rust decode, ideally shared with FilmCraft) | R. Video | 20–40 | medium |
| AI: subject / sky / background / people / object masks, object-aware remove, AI denoise, super resolution, lens blur, people & faces, natural-language search, culling | ≈ 30 | 80–150 | **high** — permissively licensed weights, pure-Rust inference, maybe training |
@@ -169,31 +171,46 @@ The milestone estimates in the table above were made before work started and are
Sony ILCE-7M4 downsized lossless ARW (subsampled YCbCr 4:2:0 / 4:2:2 tiles) now decodes to linear RGB;
private 3:2 & 4:3 examples verified through native rendering & full-resolution 16-bit export.
CR3 lossless Bayer and version 0x100/0x200 C-RAW sensor decoding is verified pixel-exactly on Canon M50, R100 and R8; any other CR3, or one that fails to decode, opens from its embedded JPEG as before. See [`docs/cr3.md`](docs/cr3.md) for coding limits, reference provenance and colour limitations.
Decoded (CC0 corpus from raw.pixls.us, `cargo xtask corpus --download`, `crates/raw/tests/corpus.rs`): DNG (uncompressed,
LJ92, lossy JPEG / Smart Previews, Deflate, float, linear), CR2, ARW (uncompressed, ARW2, LJ92; as-shot white balance and black level of pre-2017 bodies from the enciphered
maker-note `Tag2010` and the encrypted `SR2SubIFD`, both recovered by black-box analysis, `crates/raw/src/vendor/arw.rs`), NEF/NRW uncompressed and Huffman-compressed (lossless, lossy type 1/2, 12/14-bit), RAF uncompressed (Bayer and
X-Trans), RW2 / Leica RWL / Panasonic RAW in every raw format (compressed formats 4 and 6, the prefix-coded strips of format 8,
LJ92, lossy JPEG / Smart Previews, Deflate, float, linear, DNG 1.7 JPEG XL tiles: lossless ones bit-exact on synthetic files,
lossy ones decode but no real file has verified them), CR2, ARW (uncompressed, ARW2, LJ92; as-shot white balance and black level of pre-2017 bodies from the enciphered
maker-note `Tag2010` and the encrypted `SR2SubIFD`, both recovered by black-box analysis, `crates/raw/src/vendor/arw.rs`), NEF/NRW uncompressed and Huffman-compressed (lossless, lossy type 1/2, 12/14-bit), RAF uncompressed and lossless/lossy compressed (Bayer and
X-Trans, 14/16-bit real-file verification), RW2 / Leica RWL / Panasonic RAW in every raw format (compressed formats 4 and 6, the prefix-coded strips of format 8,
packed formats 2, 5 and 7, the 16-bit words of the 2005–2007 bodies; all recovered by black-box analysis of 178 CC0 files from
118 bodies, `crates/raw/src/vendor/rw2.rs`), PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Every
supported container also yields its embedded JPEG preview (CR3 too), and the engine shows that preview for raw variants
it can't decode yet.
118 bodies, `crates/raw/src/vendor/rw2.rs`), PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Supported containers yield their embedded JPEG preview when present (including CR3). The engine uses it for RAW variants it cannot decode yet; HEVC-only CR3 previews are not decoded.
Not decoded yet — preview only (no permissively licensed description; black-box analysis incomplete):
- **Nikon "lossy after split" NEF** (non-zero split row in maker note `0x0096`, e.g. some D3400/D5000/D5200/D5500/D5600
files): rows above the split decode with the regular lossy table; from the split row on a different code is used
that our black-box analysis has not recovered yet (none of the four regular tables fits, also not with byte
alignment or reset predictors at the split row). The other Nikon Huffman variants are decoded (`crates/raw/src/vendor/nefc.rs` documents the analysis).
- **Olympus compressed ORF**, **Fujifilm compressed RAF**, **Canon CR3/CRX** (M11.1), **Canon sRAW/mRAW**, lossy DNG.
- **Olympus compressed ORF**, **Canon CR3 unverified CRX variants** (M11.1), **Canon sRAW/mRAW**.
**Camera colour matrices:** ARW, NEF and RW2 files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
**Camera colour matrices:** ARW, NEF, RW2, RAF, CR3, CR2 and PEF files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
`matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer
matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax equivalents; Panasonic RW2 files carry none) and our
own chart-based calibration (M11.4). Adobe matrices are never used.
## Lightroom Classic catalog migration
File → Import Lightroom Catalog… reads `.lrcat` plus committed WAL pages directly through a pure-Rust
SQLite reader. Originals stay in place; ratings, flags, labels, XMP metadata, hierarchical keywords,
collections/sets, virtual copies and supported develop settings migrate into LightCraft. Existing edits
are preserved by default, reimport identities are persistent, and source settings/history/snapshots are
archived before catalog mutation. Missing originals remain available for relinking.
Rendering is approximate: unsupported Adobe profiles/AI/process settings are reported and archived;
history/snapshots remain source data, and smart collections import current membership. The Lightroom
database is read-only; LightCraft owns subsequent edits. See `docs/lightroom-catalog-import.md`.
## Log
- 2026-09-30: roadmap created; M0 in progress; research docs (Lightroom reference, Rust imaging ecosystem) complete.
- 2026-09-30 (later): app running with the full Lightroom-style UI; pipeline v0; DNG/CR2/ARW; README showcase. ≈ 8 h elapsed.
- 2026-10-02: parity estimate added (xtask parity prints weighted completion); milestone statuses refreshed.
- 2026-10-08: Fujifilm camera colour: guarded JPEG matching and relative WB, plus bundled X-H2S/X-T4 profiles from 201/545 photos. On 150 withheld files the pooled profiles improve mean ΔE slightly over per-file fitting (X-H2S 1.70 → 1.62, X-T4 2.43 → 2.34); dark X-H2S samples 8.74 → 2.25. Measured calibration remains open.
- 2026-10-08: Fujifilm lossless/lossy compressed RAF: pure-Rust striped decoder; full sensor arrays verified against a black-box reference on 17 CC0 files and 7 supplied X-H2S / X-T4 files (13 camera bodies, 14/16-bit Bayer and X-Trans). Camera colour calibration remains a separate gap.
- 2026-10-01: RAW II formats: RAF (uncompressed Bayer + X-Trans), RW2 (packed), PEF (incl. Huffman), ORF (uncompressed); embedded previews for every container incl. CR3; raw corpus test with 37 CC0 samples.
- 2026-10-03 – 10-05: community PRs (#42–#51, #60) and 25+ issues landed: compressed NEF, Move import, folder templates,
Local roots and cleanup, 85k-photo grid and catalog performance, background compaction, failed-save errors, GPU
@@ -202,6 +219,7 @@ own chart-based calibration (M11.4). Adobe matrices are never used.
- 2026-10-07: Panasonic RW2 / Leica RWL / Panasonic RAW decode in every raw format (compressed 4/6/8, packed 2/5/7,
16-bit words), black levels, in-camera aspect crops and mapped-out defects established on 178 CC0 files from 118
bodies; RW2 gets the file-local camera look (accepted on 140 of 174 files with a preview).
- 2026-10-08: independent Rust CR3 lossless and version 0x100/0x200 C-RAW decoding, six exact full-sensor regressions on M50/R100/R8; any CR3 decode error falls back to the embedded JPEG; guarded CR3 camera-JPEG colour fitting with independent framing validation (`docs/cr3.md`). Unverified CRX variants and broader model verification remain.
## Chinese and Japanese interfaces, and text watermarks
+5
View File
@@ -25,7 +25,12 @@ serde_json = { workspace = true }
dhat = { workspace = true, optional = true }
[features]
default = []
# Heap profiling: `cargo run --release -p lightcraft-cli --features dhat-heap -- snapshot …` counts
# every allocation (live/peak bytes in `library.memory` / `ui.inspect`) and writes dhat-heap.json
# (open it in DHAT's viewer: allocation sites by bytes at the peak). Slows allocation-heavy work.
dhat-heap = ["dep:dhat"]
# Open HEIC/HEIF photos (the optional lightcraft-heif crate). Not default so distributors choose
# it (HEVC patents); build with `--features heif` to include it.
heif = ["lightcraft-codecs/heif"]
+24 -9
View File
@@ -89,7 +89,7 @@ USAGE:
lightcraft-cli controls [--json] list every develop control id with its range
lightcraft-cli calibrate [--max N] [--out DIR] FOLDERS/FILES…
Fit a colour profile per camera model from raw files and their embedded camera JPEGs
(Sony ARW, Nikon NEF): up to N files spread over the folders (default 300; 0 = all), pooled per
(Sony ARW, Nikon NEF, Fujifilm RAF): up to N files spread over the folders (default 300; 0 = all), pooled per
model, written as <model>.json to DIR (default: the profiles folder LightCraft reads,
<config>/camera-profiles, or $LIGHTCRAFT_CAMERA_PROFILES). Raws of a profiled model then
take their colour from the profile and only their tone from their own JPEG.
@@ -102,6 +102,14 @@ static ALLOC: dhat::Alloc = dhat::Alloc;
/// Why `--library DIR` can't be opened; for a library open in another program, how to work with
/// that one instead (issue #99).
/// Print the open library's warnings (an unlockable library, damaged settings files) on stderr:
/// stdout carries command results / MCP protocol messages only.
fn library_warnings(session: &mut Session, who: &str) {
for w in session.take_library_warnings() {
eprintln!("{who}: warning: {w}");
}
}
fn library_error(dir: &str, e: lightcraft_engine::EngineError) -> String {
match e {
lightcraft_engine::EngineError::LibraryInUse(why) => format!(
@@ -159,7 +167,8 @@ fn main() -> ExitCode {
/// Raw files below `path` (or `path` itself), skipping hidden and NAS metadata folders.
fn raw_files(path: &Path, out: &mut Vec<std::path::PathBuf>, depth: usize) {
let is_raw = |p: &Path| p.extension().and_then(|e| e.to_str()).is_some_and(|e| ["arw", "nef", "nrw"].iter().any(|x| e.eq_ignore_ascii_case(x)));
let is_raw =
|p: &Path| p.extension().and_then(|e| e.to_str()).is_some_and(|e| ["arw", "nef", "nrw", "raf"].iter().any(|x| e.eq_ignore_ascii_case(x)));
if path.is_file() {
if is_raw(path) {
out.push(path.to_path_buf());
@@ -316,6 +325,7 @@ fn mcp(args: &[String]) -> Result<(), String> {
let mut h = Headless::default();
let r = h.session.open_library(dir, demo).map_err(|e| library_error(dir, e))?;
eprintln!("lightcraft-cli mcp: opened library {dir} ({r:?})");
library_warnings(&mut h.session, "lightcraft-cli mcp");
h
}
None if demo => Headless::demo(),
@@ -552,6 +562,7 @@ fn run(args: &[String]) -> Result<(), String> {
Some(dir) => {
let mut h = Headless::default();
h.session.open_library(dir, demo).map_err(|e| library_error(dir, e))?;
library_warnings(&mut h.session, "lightcraft-cli");
h
}
None if demo => Headless::demo(),
@@ -717,19 +728,19 @@ fn snapshot(args: &[String]) -> Result<(), String> {
if script.is_none() && output.is_none() {
return Err("snapshot: give -o OUT.png and/or --script FILE".into());
}
if !(scale > 0.0 && size[0] >= 1.0 && size[1] >= 1.0) {
return Err("snapshot: bad --size/--scale".into());
}
lightcraft_ui_egui::headless::viewport_pixels(size, scale).map_err(|e| format!("snapshot: bad --size/--scale: {e}"))?;
let t0 = Instant::now();
let mut session = match &library {
Some(dir) => {
let mut s = Session::new().with_fs();
let mut s = Session::new().with_fs().with_default_face_models();
s.open_library(dir, false).map_err(|e| library_error(dir, e))?;
s
}
None if files.is_empty() => Session::with_demo().with_fs(),
None => Session::new().with_fs(),
None if files.is_empty() => Session::with_demo().with_fs().with_default_face_models(),
None => Session::new().with_fs().with_default_face_models(),
};
// Apply compute policy before imports or the first scripted query can discover an adapter.
session.execute("app.gpu", &json!({"enabled": false})).map_err(|e| e.to_string())?;
if !files.is_empty() {
let ti = Instant::now();
let r = session.execute("library.import", &json!({"paths": expand_paths(&files)})).map_err(|e| e.to_string())?;
@@ -744,7 +755,11 @@ fn snapshot(args: &[String]) -> Result<(), String> {
})),
..Default::default()
};
let app = lightcraft_ui_egui::LightcraftApp::new(session, services);
let mut app = lightcraft_ui_egui::LightcraftApp::new(session, services);
// Snapshot sessions promise no GPU: disable photo compute as well as the compositor
// before the first frame can start background adapter discovery.
// This UI state is session-local: the CLI neither loads nor saves desktop ui.json.
app.ui.settings.gpu = false;
let mut h = Headless::new(app, size, scale);
let timeout = Duration::from_secs(60);
let mut shots = 0usize;
+147 -2
View File
@@ -19,6 +19,19 @@ fn gradient_png(path: &std::path::Path) {
std::fs::write(path, png).unwrap();
}
#[test]
fn calibrate_reads_raf_inputs_and_reports_bad_files() {
let input = tmp("invalid.RAF");
std::fs::write(&input, b"FUJIFILMCCD-RAW").unwrap();
let out = tmp("raf-profiles");
let result = Command::new(BIN).args(["calibrate", "--out"]).arg(&out).arg(&input).output().unwrap();
assert!(!result.status.success());
let stderr = String::from_utf8_lossy(&result.stderr);
assert!(stderr.contains("1 of 1 raw files"), "RAF was silently ignored: {stderr}");
assert!(stderr.contains("invalid.RAF") && stderr.contains("no profile written"), "{stderr}");
assert!(!out.join("X-T4.json").exists(), "bad input must not produce a profile");
}
fn mean(path: &std::path::Path) -> f64 {
let d = lightcraft_codecs::decode(&std::fs::read(path).unwrap(), Default::default()).unwrap();
let img = d.to_srgb8();
@@ -154,10 +167,14 @@ fn snapshot_subcommand_renders_the_ui_headlessly() {
std::fs::write(
&script,
format!(
"# comment\n{}\n{}\n{}\n{}\n",
"# comment\n{}\n{}\n{}\n{}\n{}\n{}\n",
json!({"method": "ui.set", "params": {"view": "photoGrid"}}),
// (settled: under load a grid shot taken before its thumbnails is darker than the
// dimmed export dialog over them, and the comparison below fails)
json!({"method": "ui.settle", "params": {"timeoutMs": 20000}}),
json!({"method": "ui.screenshot"}),
json!({"method": "engine.execute", "params": {"command": "dialog.export"}}),
json!({"method": "ui.settle", "params": {"timeoutMs": 20000}}),
json!({"method": "ui.screenshot", "params": {"path": b.to_str().unwrap()}}),
),
)
@@ -168,7 +185,7 @@ fn snapshot_subcommand_renders_the_ui_headlessly() {
.unwrap();
assert!(o.status.success(), "{}", String::from_utf8_lossy(&o.stderr));
let replies: Vec<Value> = String::from_utf8_lossy(&o.stdout).lines().map(|l| serde_json::from_str(l).unwrap()).collect();
assert_eq!(replies.len(), 4);
assert_eq!(replies.len(), 6);
assert!(replies.iter().all(|r| r["ok"] == true), "{replies:?}");
for (p, dimmed) in [(&a, false), (&b, true)] {
let d = lightcraft_codecs::decode(&std::fs::read(p).unwrap(), Default::default()).unwrap();
@@ -216,6 +233,58 @@ fn snapshot_script_failure_exits_non_zero() {
assert_eq!((d.width, d.height), (320, 200));
}
#[test]
fn snapshot_ui_zoom_keeps_requested_pixel_dimensions() {
let script = tmp("snap-ui-zoom.jsonl");
std::fs::write(
&script,
format!("{}\n{}\n", json!({"method": "ui.key", "params": {"key": "Plus", "cmd": true}}), json!({"method": "ui.inspect"})),
)
.unwrap();
for (size, scale, expected) in [("400x240", "1", (400, 240)), ("345x200", "0.9", (311, 180))] {
let out = tmp(&format!("snap-ui-zoom-{scale}.png"));
let o = Command::new(BIN)
.args(["snapshot", "--script", script.to_str().unwrap(), "-o", out.to_str().unwrap(), "--size", size, "--scale", scale])
.output()
.unwrap();
assert!(o.status.success(), "{}", String::from_utf8_lossy(&o.stderr));
let replies: Vec<Value> = String::from_utf8_lossy(&o.stdout).lines().map(|line| serde_json::from_str(line).unwrap()).collect();
assert!(replies.iter().all(|reply| reply["ok"] == true), "{replies:?}");
assert!(replies[1]["result"]["pixelsPerPoint"].as_f64().unwrap() > scale.parse::<f64>().unwrap());
let decoded = lightcraft_codecs::decode(&std::fs::read(out).unwrap(), Default::default()).unwrap();
assert_eq!((decoded.width, decoded.height), expected);
}
}
/// Headless snapshots default to CPU photo rendering as well as CPU UI rasterization.
#[test]
fn snapshot_defaults_to_cpu_without_gpu_environment_overrides() {
let script = tmp("default-cpu.jsonl");
std::fs::write(
&script,
format!("{}\n{}\n", json!({"method": "engine.execute", "params": {"command": "app.gpu"}}), json!({"method": "ui.screenshot"})),
)
.unwrap();
let image = tmp("default-cpu.png");
let output = Command::new(BIN)
.env_remove("LIGHTCRAFT_GPU")
.env_remove("LIGHTCRAFT_GPU_BACKEND")
.env_remove("WGPU_BACKEND")
.args(["snapshot", "--demo", "--script", script.to_str().unwrap(), "-o", image.to_str().unwrap(), "--size", "480x320"])
.output()
.unwrap();
assert!(output.status.success(), "status {}: {}", output.status, String::from_utf8_lossy(&output.stderr));
let replies: Vec<Value> = String::from_utf8_lossy(&output.stdout).lines().map(|line| serde_json::from_str(line).unwrap()).collect();
assert!(replies.iter().all(|reply| reply["ok"] == true), "{replies:?}");
let gpu = &replies[0]["result"];
assert_eq!(gpu["enabled"], false, "{gpu}");
assert_eq!(gpu["available"], false, "{gpu}");
assert_eq!(gpu["adapter"], Value::Null, "no device is created: {gpu}");
assert!(gpu["reason"].as_str().is_some_and(|reason| reason.contains("preference")), "{gpu}");
let decoded = lightcraft_codecs::decode(&std::fs::read(image).unwrap(), Default::default()).unwrap();
assert_eq!((decoded.width, decoded.height), (480, 320));
}
/// Issue #136: with the GPU switched off from the environment the UI starts and renders on the CPU,
/// without creating a GPU device (no driver is loaded), and says why.
#[test]
@@ -395,3 +464,79 @@ fn warnings_are_logged_on_stderr() {
let off = run("off");
assert!(!off.contains("names no known backend"), "{off}");
}
#[test]
fn snapshot_rejects_invalid_dimensions_before_creating_an_image() {
for (index, (size, scale)) in
[("NaNx100", "1"), ("100x100", "inf"), ("9000x9000", "1"), ("9000x100", "2"), ("100x100", "0.001")].into_iter().enumerate()
{
let image = tmp(&format!("invalid-viewport-{index}.png"));
let output = Command::new(BIN).args(["snapshot", "--size", size, "--scale", scale, "-o"]).arg(&image).output().unwrap();
assert!(!output.status.success());
assert!(String::from_utf8_lossy(&output.stderr).contains("snapshot: bad --size/--scale"), "{}", String::from_utf8_lossy(&output.stderr));
assert!(!image.exists());
}
}
#[test]
fn snapshot_preserves_fractional_scale_rounding() {
let image = tmp("fractional-scale-rounding.png");
let output =
Command::new(BIN).env("LIGHTCRAFT_GPU", "0").args(["snapshot", "--size", "345x200", "--scale", "0.9", "-o"]).arg(&image).output().unwrap();
assert!(output.status.success(), "{}", String::from_utf8_lossy(&output.stderr));
let decoded = lightcraft_codecs::decode(&std::fs::read(image).unwrap(), Default::default()).unwrap();
assert_eq!((decoded.width, decoded.height), (311, 180));
}
/// Snapshot's CPU setting must never replace the desktop preference, including with --library.
#[cfg(target_os = "linux")]
#[test]
fn snapshot_leaves_desktop_gpu_preferences_untouched() {
let dir = tmp("snapshot-desktop-preferences");
let library = dir.join("library");
let mut session = lightcraft_engine::Session::new().with_fs();
session.open_library(&library, true).unwrap();
drop(session);
let config = dir.join("config");
let app_config = config.join("lightcraft");
std::fs::create_dir_all(&app_config).unwrap();
let ui_path = app_config.join("ui.json");
let mut ui = lightcraft_ui_egui::UiState::default();
ui.settings.gpu = true;
let saved = serde_json::to_vec_pretty(&ui).unwrap();
std::fs::write(&ui_path, &saved).unwrap();
let script = dir.join("preferences.jsonl");
std::fs::write(
&script,
format!("{}\n{}\n", json!({"method": "engine.execute", "params": {"command": "app.gpu"}}), json!({"method": "ui.inspect"})),
)
.unwrap();
for gpu_environment in [None, Some("1")] {
let image = dir.join(format!("{}.png", gpu_environment.unwrap_or("default")));
let mut cmd = Command::new(BIN);
cmd.env("XDG_CONFIG_HOME", &config).env_remove("LIGHTCRAFT_GPU").env_remove("LIGHTCRAFT_GPU_BACKEND").env_remove("WGPU_BACKEND");
if let Some(value) = gpu_environment {
cmd.env("LIGHTCRAFT_GPU", value);
}
let output = cmd
.args(["snapshot", "--library"])
.arg(&library)
.args(["--script"])
.arg(&script)
.args(["-o"])
.arg(&image)
.args(["--size", "480x320"])
.output()
.unwrap();
assert!(output.status.success(), "{}", String::from_utf8_lossy(&output.stderr));
let replies: Vec<Value> = String::from_utf8_lossy(&output.stdout).lines().map(|line| serde_json::from_str(line).unwrap()).collect();
assert!(replies.iter().all(|reply| reply["ok"] == true), "{replies:?}");
assert_eq!(replies[0]["result"]["enabled"], false);
assert_eq!(replies[0]["result"]["available"], false);
assert_eq!(replies[0]["result"]["adapter"], Value::Null);
assert_eq!(replies[1]["result"]["ui"]["settings"]["gpu"], false);
assert_eq!(std::fs::read(&ui_path).unwrap(), saved, "snapshot changed the desktop preferences");
assert!(!app_config.join("ui.json.tmp").exists());
assert!(image.exists());
}
}
+6
View File
@@ -71,3 +71,9 @@ web-sys = { workspace = true, features = [
"WorkerType",
"WritableStream",
] }
[features]
# Open HEIC/HEIF photos (the optional lightcraft-heif crate). Not default so distributors choose
# it (HEVC patents); build with `--features heif` to include it. On wasm32 panics abort, so
# heic-rs's `catch_unwind` guard does not apply there.
heif = ["lightcraft-codecs/heif"]
+5
View File
@@ -168,6 +168,7 @@ fn services(originals: Originals, backend: Option<Backend>, files: Files, frozen
let (backup_backend, backup_files) = (backend.clone(), files);
let restore_backend = backend.clone();
Services {
picker: None,
backup_library: Some(Box::new(move |session: &mut Session| {
let Some(b) = backup_backend.clone() else { return Err("nothing is stored in this browser session (?store=memory)".into()) };
// the photos' own names for the originals in the zip
@@ -207,6 +208,9 @@ fn services(originals: Originals, backend: Option<Backend>, files: Files, frozen
// Preset files: browser pickers are asynchronous; not wired on the web yet.
pick_preset_files: None,
pick_tracklog: None,
pick_lightroom_catalog: None,
// Face models need a folder to live in; the web has none.
pick_model_file: None,
save_preset_file: None,
pick_curve_preset_files: None,
save_curve_preset_file: None,
@@ -217,6 +221,7 @@ fn services(originals: Originals, backend: Option<Backend>, files: Files, frozen
lightcraft_codecs::encode_png(&lightcraft_codecs::EncodeImage::rgba8(img), &lightcraft_codecs::EncodeMeta::default()).unwrap_or_default()
})),
reveal: None,
log_file: None,
open_with: None,
open_url: Some(Box::new(|url: &str| {
let w = web_sys::window().ok_or("no window")?;
+40
View File
@@ -53,6 +53,9 @@ pub struct WireJob {
/// The request's diagnostic overlay (`Overlay::to_parts`).
#[serde(default)]
pub overlay: (u8, f64),
/// Render only this window `[x, y, w, h]` of the output `max_w × max_h` describes (a zoomed view).
#[serde(default)]
pub window: Option<[usize; 4]>,
}
impl WireJob {
@@ -83,6 +86,7 @@ impl WireJob {
thumb_cached,
stages: job.stages.is_some().then(|| view.to_string()),
overlay: job.request.overlay.to_parts(),
window: job.request.window.map(|w| [w.x, w.y, w.w, w.h]),
}
}
@@ -108,6 +112,7 @@ impl WireJob {
space: lightcraft_engine::pipeline::OutputSpace::Srgb,
depth: lightcraft_engine::pipeline::OutputDepth::U8,
proof: None,
window: self.window.map(|[x, y, w, h]| lightcraft_engine::pipeline::PixelWindow { x, y, w, h }),
}
}
@@ -348,6 +353,40 @@ mod tests {
assert!(reopened.contains(&new));
}
// Issue #323: a worker given a zoomed view's window renders that window, not the whole frame
#[test]
fn a_window_job_crosses_the_wire_and_renders_the_same_window() {
use lightcraft_engine::pipeline::PixelWindow;
let mut s = Session::with_demo();
let id = s.visible_cloned()[0];
let win = PixelWindow { x: 300, y: 200, w: 160, h: 120 };
let job = s.region_job(id, 1920, 1280, win, true).unwrap();
let wire = WireJob::from_job(&job, false, "Region");
let wire: WireJob = serde_json::from_str(&serde_json::to_string(&wire).unwrap()).unwrap();
assert_eq!(wire.request().window, Some(win));
let mut core = WorkerCore::default();
let a = core.render(&wire, None).unwrap();
let b = job.run().rendered.unwrap();
assert_eq!((a.image.width, a.image.height), (160, 120));
assert_eq!((b.image.width, b.image.height), (160, 120));
// the worker renders on the CPU; the engine job uses a GPU when the machine has one, which
// stays within docs/gpu-pipeline.md's bound (≤ 3 LSB per channel) rather than bit-exact
let max = a.image.data.iter().zip(&b.image.data).flat_map(|(p, q)| (0..3).map(move |c| p[c].abs_diff(q[c]))).max();
assert!(max.is_some_and(|m| m <= 3), "max channel difference {max:?}");
}
// a job from before windows existed has none
#[test]
fn a_job_without_a_window_field_has_no_window() {
let mut s = Session::with_demo();
let id = s.visible_cloned()[0];
let wire = WireJob::from_job(&s.thumb_job(id, 128).unwrap(), false, "Thumb");
let mut v = serde_json::to_value(&wire).unwrap();
v.as_object_mut().unwrap().remove("window");
let old: WireJob = serde_json::from_value(v).unwrap();
assert_eq!(old.request().window, None);
}
#[test]
fn wire_job_renders_like_the_engine() {
let mut s = Session::with_demo();
@@ -393,6 +432,7 @@ mod tests {
thumb_cached: false,
stages: Some("Main".into()),
overlay: (0, 0.0),
window: None,
};
let mut core = WorkerCore::default();
assert_eq!(core.needs_original(&job).as_deref(), Some(hash.as_str()));
+6
View File
@@ -9,6 +9,12 @@ default-run = "lightcraft"
[lints]
workspace = true
[features]
default = []
# Open HEIC/HEIF photos (the optional lightcraft-heif crate). Not default so distributors choose
# it (HEVC patents); build with `--features heif` to include it.
heif = ["lightcraft-codecs/heif"]
[dependencies]
lightcraft-engine = { workspace = true, features = ["sam"] }
lightcraft-sysmem = { workspace = true }
+145
View File
@@ -0,0 +1,145 @@
//! File-dialog filters that match extensions whatever their letter case.
//!
//! On Linux, `rfd` turns each extension into a `*.ext` glob for the GTK dialog and for the
//! desktop portal (`xdg-desktop-portal` FileChooser), and both match globs case-sensitively, so
//! a filter of `nef` hid `DSC_1234.NEF` (issue #342). Windows and macOS ignore case. Every
//! dialog filter of the app goes through [`FileDialogExt::add_filter_nocase`] (a test checks
//! that nothing else calls `add_filter`), so no extension can hit this.
use rfd::FileDialog;
/// The patterns to hand to `rfd` for `exts`, which are plain extensions without the dot.
///
/// Windows and macOS take the extension literally and ignore case: lower case, de-duplicated.
/// Linux (`linux`): `rfd` formats each entry as `*.{entry}`, which GTK 3 and the portal match
/// as a glob, so an entry may carry glob syntax. Each extension gives the lower-case form, the
/// UPPER-case form (kept in case a portal backend does not do `[...]` classes) and a bracket
/// form such as `[nN][eE][fF]` that matches every mix (`.Nef`, `.nEf`, ...).
pub fn dialog_extensions_for(exts: &[&str], linux: bool) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
let mut push = |s: String| {
if !s.is_empty() && !out.contains(&s) {
out.push(s);
}
};
for ext in exts {
let ext = ext.trim_start_matches('.');
let lower = ext.to_lowercase();
push(lower.clone());
if !linux {
continue;
}
push(ext.to_uppercase());
// `[` `]` `*` `?` in an extension would already be glob syntax: leave those alone
if lower.chars().any(|c| c.is_alphabetic()) && !lower.contains(['[', ']', '*', '?', '\\']) {
push(
lower
.chars()
.map(|c| {
let up: String = c.to_uppercase().collect();
if up.chars().count() == 1 && up != c.to_string() { format!("[{c}{up}]") } else { c.to_string() }
})
.collect(),
);
}
}
out
}
/// [`dialog_extensions_for`] for the platform this is built for.
pub fn dialog_extensions(exts: &[&str]) -> Vec<String> {
dialog_extensions_for(exts, cfg!(all(unix, not(target_os = "macos"))))
}
/// `add_filter` that matches the extensions in any letter case.
pub trait FileDialogExt {
fn add_filter_nocase(self, name: impl Into<String>, exts: &[&str]) -> Self;
}
impl FileDialogExt for FileDialog {
fn add_filter_nocase(self, name: impl Into<String>, exts: &[&str]) -> Self {
self.add_filter(name, &dialog_extensions(exts))
}
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::*;
#[test]
fn other_platforms_get_lower_case_once() {
assert_eq!(dialog_extensions_for(&["NEF", "nef", ".Jpg", "jpg", ""], false), ["nef", "jpg"]);
}
#[test]
fn linux_gets_lower_upper_and_any_mix() {
assert_eq!(dialog_extensions_for(&["nef"], true), ["nef", "NEF", "[nN][eE][fF]"]);
// digits have no case; a duplicate adds nothing
assert_eq!(dialog_extensions_for(&["mp4", "MP4"], true), ["mp4", "MP4", "[mM][pP]4"]);
assert_eq!(dialog_extensions_for(&["7z"], true), ["7z", "7Z", "7[zZ]"]);
assert_eq!(dialog_extensions_for(&["123"], true), ["123"]);
}
#[test]
fn linux_glob_syntax_is_not_wrapped_again() {
assert_eq!(dialog_extensions_for(&["[nN]ef"], true), ["[nn]ef", "[NN]EF"]);
}
/// The Linux patterns match like GTK's glob (`*.{entry}`): checked against a tiny matcher of
/// the subset used here (verified against real GTK 3.24 `GtkFileFilter` in the PR notes).
fn glob_ext(entry: &str, ext: &str) -> bool {
let mut pat = entry.chars().peekable();
let mut s = ext.chars();
while let Some(p) = pat.next() {
let Some(c) = s.next() else { return false };
if p == '[' {
let mut hit = false;
for q in pat.by_ref() {
if q == ']' {
break;
}
hit |= q == c;
}
if !hit {
return false;
}
} else if p != c {
return false;
}
}
s.next().is_none()
}
#[test]
fn linux_patterns_match_every_case_mix() {
let pats = dialog_extensions_for(&["nef", "cr2", "xmp"], true);
let matches = |name: &str| pats.iter().any(|p| glob_ext(p, name));
for ok in ["nef", "NEF", "Nef", "nEF", "CR2", "Cr2", "XMP", "xMp"] {
assert!(matches(ok), "{ok}");
}
for bad in ["nex", "ne", "nefs", "jpg", "cr3"] {
assert!(!matches(bad), "{bad}");
}
}
/// The app's own extension list (the one the folder scan uses) is covered whole.
#[test]
fn import_extensions_are_all_covered_in_both_cases() {
let pats = dialog_extensions_for(lightcraft_engine::import::EXTENSIONS, true);
for e in lightcraft_engine::import::EXTENSIONS {
assert!(pats.iter().any(|p| glob_ext(p, &e.to_uppercase())), "{e}");
assert!(pats.iter().any(|p| p == e), "{e}");
}
}
/// Every dialog filter goes through the helper: `main.rs` must not call rfd's own
/// `add_filter`.
#[test]
fn no_dialog_bypasses_the_helper() {
for (file, src) in [("main.rs", include_str!("main.rs")), ("control_server.rs", include_str!("control_server.rs"))] {
assert!(!src.contains(".add_filter("), "{file} calls rfd's add_filter; use add_filter_nocase");
}
assert!(include_str!("main.rs").contains(".add_filter_nocase("));
}
}
+178 -20
View File
@@ -24,10 +24,12 @@
mod alloc_release;
mod control_server;
mod dialog_filter;
mod logging;
#[cfg(target_os = "macos")]
mod native_menu;
use dialog_filter::FileDialogExt as _;
use lightcraft_engine::Session;
use lightcraft_ui_egui::panels::library_problem::LibraryProblem;
use lightcraft_ui_egui::{LightcraftApp, Services, UiState};
@@ -269,8 +271,61 @@ impl PrefsWriter {
}
}
fn services() -> Services {
/// The Windows command that opens `url` in the default browser. `explorer <url>` opened File
/// Explorer instead for links with a query or fragment, like the Map button's OpenStreetMap link
/// (issue #234). The URL protocol handler hands the URL to the browser as is; no shell runs, so
/// `?`, `&` and `#` need no escaping.
fn windows_open_url_command(url: &str) -> std::process::Command {
let mut c = std::process::Command::new("rundll32");
c.args(["url.dll,FileProtocolHandler", url]);
c
}
/// The platform services; `ctx` repaints when a file dialog closes, and `log_file` is
/// for Help ▸ Open Log Folder.
fn services(ctx: egui::Context, log_file: Option<&std::path::Path>) -> Services {
Services {
// Commands' file dialogs run on their own thread (#191): a dialog on the UI thread
// stopped the window answering the compositor, which then reported the app as hung.
picker: Some(Box::new(move |req: lightcraft_ui_egui::pick::PickRequest| {
use lightcraft_ui_egui::pick::PickKind;
let (tx, rx) = std::sync::mpsc::channel();
let repaint = ctx.clone();
std::thread::Builder::new()
.name("file-dialog".into())
.spawn(move || {
let mut d = rfd::FileDialog::new();
if let Some(t) = &req.title {
d = d.set_title(t);
}
if let Some((name, exts)) = &req.filter {
d = d.add_filter_nocase(name.clone(), exts);
}
if let Some(n) = &req.file_name {
d = d.set_file_name(n);
}
let s = |p: std::path::PathBuf| p.to_string_lossy().to_string();
let paths: Vec<String> = match req.kind {
PickKind::Files => d.pick_files().unwrap_or_default().into_iter().map(s).collect(),
PickKind::File => d.pick_file().map(s).into_iter().collect(),
PickKind::Folder => d.pick_folder().map(s).into_iter().collect(),
PickKind::Save => d.save_file().map(s).into_iter().collect(),
};
// the app may have quit meanwhile: nobody to tell
let _ = tx.send(paths);
repaint.request_repaint();
})
.map_err(|e| format!("could not start the file dialog: {e}"))?;
Ok(rx)
})),
log_file: log_file.map(|p| p.display().to_string()),
pick_lightroom_catalog: Some(Box::new(|| {
rfd::FileDialog::new()
.set_title(lightcraft_ui_egui::i18n::tr("Import Lightroom Catalog"))
.add_filter_nocase(lightcraft_ui_egui::i18n::tr("Lightroom Classic Catalog"), &["lrcat"])
.pick_file()
.map(|p| p.to_string_lossy().to_string())
})),
pick_folder: Some(Box::new(|| {
rfd::FileDialog::new().set_title(lightcraft_ui_egui::i18n::tr("Open Library")).pick_folder().map(|p| p.to_string_lossy().to_string())
})),
@@ -303,7 +358,7 @@ fn services() -> Services {
let status = if cfg!(target_os = "macos") {
std::process::Command::new("open").arg(url).status()
} else if cfg!(target_os = "windows") {
std::process::Command::new("explorer").arg(url).status()
windows_open_url_command(url).status()
} else {
std::process::Command::new("xdg-open").arg(url).status()
};
@@ -325,13 +380,7 @@ fn services() -> Services {
})),
pick_files: Some(Box::new(|| {
rfd::FileDialog::new()
.add_filter(
lightcraft_ui_egui::i18n::tr("Photos"),
&[
"jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "nrw", "arw", "raf", "orf", "rw2", "rwl", "raw",
"pef", "psd", "jxl", "gif", "bmp",
],
)
.add_filter_nocase(lightcraft_ui_egui::i18n::tr("Photos"), lightcraft_engine::import::EXTENSIONS)
.pick_files()
.unwrap_or_default()
.into_iter()
@@ -341,7 +390,7 @@ fn services() -> Services {
pick_preset_files: Some(Box::new(|| {
rfd::FileDialog::new()
.set_title(lightcraft_ui_egui::i18n::tr("Import Presets"))
.add_filter(
.add_filter_nocase(
lightcraft_ui_egui::i18n::tr("Presets & Profiles"),
&["lcpreset", "xmp", "lrtemplate", "zip", "dng", "lmp", "mplumpack", "cube"],
)
@@ -351,10 +400,18 @@ fn services() -> Services {
.map(|p| p.to_string_lossy().to_string())
.collect()
})),
pick_model_file: Some(Box::new(|| {
rfd::FileDialog::new()
.set_title(lightcraft_ui_egui::i18n::tr("Add a Face Recognition Model"))
.add_filter_nocase(lightcraft_ui_egui::i18n::tr("Face models (ONNX)"), &["onnx"])
.pick_file()
.map(|p| vec![p.to_string_lossy().to_string()])
.unwrap_or_default()
})),
pick_tracklog: Some(Box::new(|| {
rfd::FileDialog::new()
.set_title(lightcraft_ui_egui::i18n::tr("Auto-Tag from Tracklog"))
.add_filter(lightcraft_ui_egui::i18n::tr("GPS Track Log"), &["gpx"])
.add_filter_nocase(lightcraft_ui_egui::i18n::tr("GPS Track Log"), &["gpx"])
.pick_file()
.map(|p| vec![p.to_string_lossy().to_string()])
.unwrap_or_default()
@@ -362,7 +419,7 @@ fn services() -> Services {
save_preset_file: Some(Box::new(|name: &str| {
rfd::FileDialog::new()
.set_title(lightcraft_ui_egui::i18n::tr("Export Presets"))
.add_filter(lightcraft_ui_egui::i18n::tr("LightCraft Preset"), &["lcpreset"])
.add_filter_nocase(lightcraft_ui_egui::i18n::tr("LightCraft Preset"), &["lcpreset"])
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
@@ -370,7 +427,7 @@ fn services() -> Services {
pick_curve_preset_files: Some(Box::new(|| {
rfd::FileDialog::new()
.set_title(lightcraft_ui_egui::i18n::tr("Import Point Curve Presets"))
.add_filter(lightcraft_ui_egui::i18n::tr("Point Curve Presets"), &["lccurve", "json"])
.add_filter_nocase(lightcraft_ui_egui::i18n::tr("Point Curve Presets"), &["lccurve", "json"])
.pick_files()
.unwrap_or_default()
.into_iter()
@@ -380,7 +437,7 @@ fn services() -> Services {
save_curve_preset_file: Some(Box::new(|name: &str| {
rfd::FileDialog::new()
.set_title(lightcraft_ui_egui::i18n::tr("Export Point Curve Presets"))
.add_filter(lightcraft_ui_egui::i18n::tr("Point Curve Presets"), &["lccurve"])
.add_filter_nocase(lightcraft_ui_egui::i18n::tr("Point Curve Presets"), &["lccurve"])
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
@@ -397,12 +454,19 @@ fn services() -> Services {
}
}
/// The library session with the folder face models are kept in (`<config>/models`).
fn open_session(in_memory: bool, dir: Option<std::path::PathBuf>, seed_demo: bool) -> (Session, Option<LibraryProblem>) {
let (mut s, problem) = open_library_session(in_memory, dir, seed_demo);
s.face_models_dir = lightcraft_engine::config::default_face_models_dir();
(s, problem)
}
/// The persistent library session, or with `--memory` an in-memory one (demo photos).
///
/// If the library can't be opened the session is empty and in memory — never seeded with the
/// demo photos, never written anywhere — and the problem is returned: the window then says so
/// and offers Try Again / Choose Another Library… / Continue Without Saving / Quit (issue #100).
fn open_session(in_memory: bool, dir: Option<std::path::PathBuf>, seed_demo: bool) -> (Session, Option<LibraryProblem>) {
fn open_library_session(in_memory: bool, dir: Option<std::path::PathBuf>, seed_demo: bool) -> (Session, Option<LibraryProblem>) {
if in_memory {
return (if seed_demo { Session::with_demo() } else { Session::new() }.with_fs().with_system_clock(), None);
}
@@ -456,6 +520,31 @@ ENVIRONMENT:
<settings folder>/logs/lightcraft.log (previous runs: lightcraft.1.log, lightcraft.2.log)
";
/// The launch arguments' files and folders to import (absolute), and a notice for each one that
/// is not imported (see [`lightcraft_engine::import::launch_path`]; issue #374).
fn launch_imports(args: &[String]) -> (Vec<String>, Vec<String>) {
let cwd = std::env::current_dir().ok();
let home = std::env::var_os(if cfg!(windows) { "USERPROFILE" } else { "HOME" }).filter(|h| !h.is_empty()).map(std::path::PathBuf::from);
launch_imports_in(args, cwd.as_deref(), home.as_deref())
}
fn launch_imports_in(args: &[String], cwd: Option<&std::path::Path>, home: Option<&std::path::Path>) -> (Vec<String>, Vec<String>) {
use lightcraft_engine::import::{LaunchPath, launch_path};
let (mut files, mut notices) = (Vec::new(), Vec::new());
for a in args {
match launch_path(a, cwd, home, lightcraft_engine::devices::dcim_in) {
Some(LaunchPath::Import(p)) => files.push(p),
Some(LaunchPath::Refused { path, why }) => {
log::warn!("not importing {path} given at launch: {why}");
notices
.push(format!("{path} was not imported: it is {why}. To import from it, choose a folder inside it with File ▸ Import Photos…"));
}
None => {}
}
}
(files, notices)
}
/// Where the log files live: `logs` in the settings folder, next to `ui.json` (see `logging`).
/// None with `LIGHTCRAFT_NO_PREFS` (tests, scripts), so those runs don't rotate away the user's logs.
fn log_dir() -> Option<std::path::PathBuf> {
@@ -513,18 +602,29 @@ fn main() -> eframe::Result {
}
// The log file: opened after the arguments, so `--version`, `--help` and a usage error leave
// no file behind. Records logged until now are written to it first.
let mut log_file = None;
if let Some(logger) = logger {
// a --memory session writes nothing (issue #164), a log file included
match log_dir().filter(|_| !in_memory) {
Some(dir) => match logger.attach_dir(&dir) {
Ok(path) => log::info!("LightCraft {}, log file {}", env!("CARGO_PKG_VERSION"), path.display()),
Ok(path) => {
log::info!("LightCraft {}, log file {}", env!("CARGO_PKG_VERSION"), path.display());
log_file = Some(path);
}
// The file sink has given up by now, so this goes to standard error only.
Err(e) => log::warn!("no log file: {e}"),
},
None => logger.no_file(),
}
}
// what the arguments name, resolved now: an empty or relative argument never becomes a drive
// root, and a whole drive or home folder is not walked (issue #374)
let (files, launch_notices) = launch_imports(&files);
let (prefs, prefs_warning, keep_prefs_file) = load_prefs(in_memory);
// the saved language from the start, so a failed start is reported in it too
if let Some(ui) = &prefs {
lightcraft_ui_egui::i18n::set_language(ui.language);
}
// --library, else the library last opened from Settings, else the default location
let library_dir = library_dir.or_else(|| {
prefs
@@ -554,7 +654,9 @@ fn main() -> eframe::Result {
wgpu_options: window_wgpu_options(),
..Default::default()
};
eframe::run_native(
// the app's copy: `log_file` is still named in the message of a start that fails
let app_log_file = log_file.clone();
let started = eframe::run_native(
"LightCraft",
options,
Box::new(move |cc| {
@@ -565,13 +667,14 @@ fn main() -> eframe::Result {
std::env::var_os("LIGHTCRAFT_SAM3_DIR").map(std::path::PathBuf::from).or_else(|| config_dir().map(|d| d.join("models").join("sam3")));
// the user's own download locations, one base URL per line (LIGHTCRAFT_SAM3_MIRRORS too)
session.segmenter.mirrors_file = config_dir().map(|d| d.join("models").join("sam3-mirrors.txt"));
let mut app = LightcraftApp::new(session, services());
let mut app = LightcraftApp::new(session, services(cc.egui_ctx.clone(), app_log_file.as_deref()));
if let Some(ui) = prefs {
app.ui = ui;
}
lightcraft_ui_egui::i18n::set_language(app.ui.language);
app.integrated_titlebar = cfg!(target_os = "macos");
app.notices.extend(prefs_warning);
app.notices.extend(launch_notices);
// what's on disk now: only changes are written
let writer = PrefsWriter {
path: if in_memory { None } else { prefs_path() },
@@ -594,7 +697,12 @@ fn main() -> eframe::Result {
p.pending_import = files;
app.library_problem = Some(p);
} else if !files.is_empty() {
let _ = app.run("library.import", serde_json::json!({"paths": files}));
// listed and read on the import worker once the window shows (issue #374: never
// here, before the first frame, where a large folder kept the window from
// appearing)
if let Err(e) = lightcraft_ui_egui::import::start_paths(&mut app, files) {
log::warn!("import at launch: {e}");
}
app.ui.view = lightcraft_ui_egui::state::ViewMode::PhotoGrid;
}
// native menu bar generated from the command registry (macOS; elsewhere the menus are
@@ -608,7 +716,28 @@ fn main() -> eframe::Result {
menu,
)))
}),
)
);
if let Err(e) = &started {
startup_failed(&e.to_string(), log_file.as_deref());
}
started
}
/// The window could not start (e.g. no usable graphics device): say so in the log and in a native
/// message box naming the log file, instead of exiting without a trace — the release builds have
/// no console on Windows, and an app started from Finder shows no standard error (issue #260).
fn startup_failed(error: &str, log_file: Option<&std::path::Path>) {
log::error!("LightCraft could not open its window: {error}");
if std::env::var_os("LIGHTCRAFT_NO_PREFS").is_some() {
return; // tests and scripts: no dialog to click away
}
let (title, text) = lightcraft_ui_egui::i18n::startup_failed_message(error, log_file.map(|p| p.display().to_string()).as_deref());
let _ = rfd::MessageDialog::new()
.set_level(rfd::MessageLevel::Error)
.set_title(title)
.set_description(text)
.set_buttons(rfd::MessageButtons::Ok)
.show();
}
#[cfg(test)]
@@ -623,6 +752,24 @@ mod tests {
d
}
/// Issue #374: launch arguments that would walk a whole drive or home folder are reported,
/// not imported; empty ones are ignored; the rest are imported as absolute paths.
#[test]
fn launch_arguments_never_walk_a_drive_or_the_home_folder() {
let home = dir("launch");
std::fs::create_dir_all(home.join("Trip")).unwrap();
let args: Vec<String> = ["", " ", ".", "Trip", &home.to_string_lossy()].iter().map(|s| s.to_string()).collect();
let (files, notices) = launch_imports_in(&args, Some(&home), Some(&home));
assert_eq!(files, [home.join("Trip").to_string_lossy().to_string()]);
assert_eq!(notices.len(), 2, "{notices:?}");
assert!(notices.iter().all(|n| n.contains("was not imported")), "{notices:?}");
// the working folder is a drive root (a launcher's choice): `.` is never the whole drive
let root = home.ancestors().last().unwrap().to_path_buf();
let (files, notices) = launch_imports_in(&[".".to_string()], Some(&root), None);
assert!(files.iter().all(|f| f.ends_with("DCIM")) && files.len() + notices.len() == 1, "{files:?} {notices:?}");
let _ = std::fs::remove_dir_all(&home);
}
/// Issue #103: a damaged ui.json (which holds the library location) is kept aside and
/// reported, not silently replaced by the defaults.
#[test]
@@ -744,6 +891,17 @@ mod tests {
assert!(!b.is_empty());
}
/// Issue #234: Windows opens links with the URL protocol handler (the default browser), not
/// File Explorer, and passes the whole map link, `?`, `&` and `#` included, as one argument.
#[test]
fn windows_links_open_in_the_browser() {
let url = "https://www.openstreetmap.org/?mlat=48.856600&mlon=2.352200#map=15/48.856600/2.352200";
let c = windows_open_url_command(url);
assert_eq!(c.get_program(), "rundll32");
let args: Vec<&std::ffi::OsStr> = c.get_args().collect();
assert_eq!(args, ["url.dll,FileProtocolHandler", url]);
}
#[test]
fn a_control_port_that_is_not_a_number_is_ignored() {
assert_eq!(control_port_from("--control", Some("7980".into())), Some(7980));
+12 -5
View File
@@ -54,6 +54,9 @@ pub struct NativeMenu {
/// Structure of the installed menu (ids, kinds, submenu names); a change rebuilds it.
structure: String,
text_focus: bool,
/// The keyboard shortcuts editor is recording a key: every accelerator is off so the key
/// press reaches it.
capturing: bool,
}
/// `Cmd+Shift+Z` → a muda accelerator (`None` for keys we leave to egui, e.g. Escape).
@@ -211,7 +214,7 @@ impl NativeMenu {
let _ = tx.send(e.id.0);
repaint.request_repaint();
}));
let mut m = NativeMenu { menu: Menu::new(), items: HashMap::new(), rx, structure: String::new(), text_focus: false };
let mut m = NativeMenu { menu: Menu::new(), items: HashMap::new(), rx, structure: String::new(), text_focus: false, capturing: false };
m.rebuild(app);
app.native_menu = true;
m
@@ -281,6 +284,7 @@ impl NativeMenu {
}
self.menu.init_for_nsapp();
self.text_focus = false;
self.capturing = false;
self.publish_shortcuts(app);
}
@@ -345,7 +349,8 @@ impl NativeMenu {
/// Tell the egui shortcut handler which shortcuts the menu bar currently owns.
fn publish_shortcuts(&self, app: &mut LightcraftApp) {
let installed = self.items.values().filter(|i| i.accel.is_some()).filter_map(|i| i.shortcut.as_deref());
app.native_shortcuts = owned_by_menu(installed, self.text_focus);
// (none while the keymap editor records a shortcut: every key goes to it)
app.native_shortcuts = if self.capturing { HashSet::new() } else { owned_by_menu(installed, self.text_focus) };
app.native_shortcuts.insert(SETTINGS_KEY.to_string());
}
@@ -407,14 +412,16 @@ impl NativeMenu {
walk(&bar.iter().flat_map(|(_, v)| v.clone()).collect::<Vec<_>>(), &mut self.items);
let focus = ctx.egui_wants_keyboard_input();
if focus != self.text_focus {
let capturing = app.recording_shortcut.is_some();
if focus != self.text_focus || capturing != self.capturing {
self.text_focus = focus;
self.capturing = capturing;
for it in self.items.values() {
let Some(sc) = it.shortcut.as_deref() else { continue };
if it.accel.is_none() || !yields_to_text(sc) {
if it.accel.is_none() {
continue;
}
let accel = if focus { None } else { it.accel };
let accel = if capturing || (focus && yields_to_text(sc)) { None } else { it.accel };
let _ = match &it.handle {
Handle::Plain(h) => h.set_accelerator(accel),
Handle::Check(h) => h.set_accelerator(accel),
+1
View File
@@ -12,6 +12,7 @@ covers a whole directory), or if a licence file referenced in the Licence column
| `assets/app-icon/` (`lightcraft.svg` and the PNG/ICNS/ICO/SVG files derived from it) | LightCraft app icon (lynx) | LightCraft project owner (drawn in ArtCraft) | original work (vectorised from the owner's drawing; see `assets/app-icon/README.md`) | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) | 2026-10-02 | vectorised to SVG; rendered to PNG/ICNS/ICO by `packaging/icons.sh` |
| `assets/camera-profiles/*.json` | Camera colour profiles (matrix + hue/saturation/value table), built into LightCraft | ILCE-7M4: Harald Hoyer | original work: fitted by `lightcraft-cli calibrate` to the contributor's own raw photos and their in-camera JPEGs (597 ILCE-7M4 photos, 2026); aggregate colour statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 | 2026-10-07 | none |
| `packaging/macos/dmg/` (`background.svg`, the `background.tiff` rendered from it by `generate.py`, and the `dmg-layout.DS_Store` layout file) | LightCraft macOS DMG window background | @XusBadia | original work for this repository, derived from the LightCraft app icon (`assets/app-icon/lightcraft.svg`, referenced via `<image>`, not copied); text is outlined Inter / JetBrains Mono glyph paths | MIT OR Apache-2.0 | 2026-10-08 | none (original); used for the Finder window of the macOS DMG, `packaging/macos/dmg/generate.py` renders it to `background.tiff` |
| `assets/camera-profiles/X-H2S.json`, `assets/camera-profiles/X-T4.json` | Fujifilm camera colour profiles (matrix + hue/saturation/value table), built into LightCraft | LightCraft contributors; source photographs supplied by Sebastian Pfluegelmeier | original work: fitted by `lightcraft-cli calibrate` to 201 X-H2S and 545 X-T4 RAFs and their in-camera JPEGs; aggregate colour statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 | 2026-10-08 | none |
| `crates/scenes/` (generated images) | Procedural demo photographs | LightCraft contributors | original work (generated at runtime, no source imagery) | MIT OR Apache-2.0 | 2026-09-30 | n/a |
| `crates/ui-egui/src/icons.rs` | UI icons drawn as vector paths in code | LightCraft contributors | original work | MIT OR Apache-2.0 | 2026-09-30 | n/a |
| `docs/images/*-tetons.jpg`, `docs/images/grid-pd.jpg` (screenshots containing it) | "The Tetons and the Snake River" | Ansel Adams (1942), U.S. National Archives | https://commons.wikimedia.org/wiki/File:Adams_The_Tetons_and_the_Snake_River.jpg | Public domain (U.S. federal government work) | 2026-09-30 | developed in LightCraft; shown inside app screenshots |
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+3
View File
@@ -21,3 +21,6 @@
real_name = "Brandon Thomas"
display_name = "Brandon Thomas"
sync_github_name = false
[people.sanyamsingh001]
display_name = "sanyamsingh001"
+2 -1
View File
@@ -45,6 +45,7 @@
//! | (none) | early builds: a log without a snapshot | |
//! | 1 | up to v0.2.0 | |
//! | 2 | after v0.2.0 | `Op::SetBrowsed`, `Catalog.browsed`, `Photo.local_baseline` |
//! | 3 | albums by hand | `Op::SetAlbumOrder`, `Album.order` |
//!
//! Rules:
//! - **Bump [`VERSION`]** (and add a row above) in the change that adds an [`Op`] variant or a
@@ -69,7 +70,7 @@ pub const LOG: &str = "catalog.log";
const FORMAT: &str = "lightcraft-catalog";
/// The catalog format this build writes (and the newest it reads). See the module docs →
/// *Format versions*; bump it whenever an [`Op`] variant or a serialized field is added.
pub const VERSION: u32 = 2;
pub const VERSION: u32 = 3;
/// When [`Journal::wants_snapshot`] says it's time to compact the log.
#[derive(Clone, Copy, Debug)]
+49 -1
View File
@@ -136,6 +136,11 @@ pub enum Op {
id: AlbumId,
parent: Option<AlbumId>,
},
/// Where the album stands among its siblings (`None`: by name). Format version 3.
SetAlbumOrder {
id: AlbumId,
order: Option<u32>,
},
SetAlbumPhotos {
id: AlbumId,
photos: Vec<PhotoId>,
@@ -282,6 +287,28 @@ impl Catalog {
pub fn albums(&self) -> impl Iterator<Item = &Album> {
self.albums.values()
}
/// The albums and folders directly inside `parent` (`None`: the top level), in the order the
/// sidebar lists them: folders first, then by their place if the user ordered them (those
/// without one after, by name), else by name.
pub fn album_children(&self, parent: Option<AlbumId>) -> Vec<&Album> {
let mut kids: Vec<&Album> = self.albums.values().filter(|a| a.parent == parent).collect();
kids.sort_by_cached_key(|a| album_order_key(a));
kids
}
/// [`Self::album_children`] of every folder (and of the top level, under `None`) in one pass:
/// what a tree view that draws all of them wants each frame.
pub fn album_children_by_parent(&self) -> std::collections::HashMap<Option<AlbumId>, Vec<&Album>> {
let mut map: std::collections::HashMap<Option<AlbumId>, Vec<&Album>> = std::collections::HashMap::new();
for a in self.albums.values() {
map.entry(a.parent).or_default().push(a);
}
map.values_mut().for_each(|kids| kids.sort_by_cached_key(|a| album_order_key(a)));
map
}
/// Whether anything inside `parent` has a place of its own (the folder is ordered by hand).
pub fn album_children_are_ordered(&self, parent: Option<AlbumId>) -> bool {
self.albums.values().any(|a| a.parent == parent && a.order.is_some())
}
/// The Quick Collection, once something was added to it.
pub fn quick_collection(&self) -> Option<AlbumId> {
self.albums.values().find(|a| a.quick).map(|a| a.id)
@@ -479,7 +506,20 @@ impl Catalog {
}
}
let a = self.album_mut(id)?;
Op::MoveAlbum { id, parent: std::mem::replace(&mut a.parent, parent) }
if a.parent == parent {
return Ok(Op::MoveAlbum { id, parent });
}
// its place belonged to the old neighbours: the undo gives it back with the folder
let old_order = a.order.take();
let old_parent = std::mem::replace(&mut a.parent, parent);
match old_order {
Some(_) => Op::Batch { ops: vec![Op::MoveAlbum { id, parent: old_parent }, Op::SetAlbumOrder { id, order: old_order }] },
None => Op::MoveAlbum { id, parent: old_parent },
}
}
Op::SetAlbumOrder { id, order } => {
let a = self.album_mut(id)?;
Op::SetAlbumOrder { id, order: std::mem::replace(&mut a.order, order) }
}
Op::SetAlbumPhotos { id, photos } => {
let a = self.album_mut(id)?;
@@ -668,9 +708,17 @@ impl Catalog {
}
}
/// How siblings are listed: folders first, then the ones with a place of their own by it, then by
/// name; the id settles any tie.
fn album_order_key(a: &Album) -> (bool, bool, u32, String, AlbumId) {
(!a.folder, a.order.is_none(), a.order.unwrap_or(0), a.name.to_lowercase(), a.id)
}
#[cfg(test)]
mod tests;
#[cfg(test)]
mod tests_album_order;
#[cfg(test)]
mod tests_background;
#[cfg(test)]
mod tests_folders;
+1
View File
@@ -295,6 +295,7 @@ impl Op {
Op::RemoveAlbum { .. }
| Op::RenameAlbum { .. }
| Op::MoveAlbum { .. }
| Op::SetAlbumOrder { .. }
| Op::SetAlbumRules { .. }
| Op::RemoveStack { .. }
| Op::SetLabelName { .. }
+24 -6
View File
@@ -2,7 +2,7 @@
use std::sync::Arc;
use lightcraft_develop::DevelopSettings;
use lightcraft_develop::{DevelopSettings, WbMode};
use serde::{Deserialize, Serialize};
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
@@ -336,12 +336,13 @@ impl Photo {
pub fn develops_raw(&self) -> bool {
self.kind == MediaKind::Raw && self.preview_only.is_none()
}
/// The current ARW, NEF and RW2 readers have vendor WB multipliers but no measured camera
/// The current ARW, NEF, RW2, RAF, CR3, CR2 and PEF readers have vendor WB multipliers but no measured camera
/// illuminant. Use adjustments relative to the camera's as-shot look, as for rendered
/// photographs (the engine's `camera_preview::file_local_look` covers the same formats; RWL and
/// RAW are Leica's and the oldest Panasonic bodies' names for RW2 files).
pub fn relative_wb(&self) -> bool {
self.develops_raw() && ["ARW", "NEF", "NRW", "RW2", "RWL", "RAW"].iter().any(|f| self.format.eq_ignore_ascii_case(f))
self.develops_raw()
&& ["ARW", "NEF", "NRW", "RW2", "RWL", "RAW", "RAF", "CR3", "CR2", "PEF"].iter().any(|f| self.format.eq_ignore_ascii_case(f))
}
/// The develop settings import gave this photo: [`Photo::camera_defaults`], or the user's
/// default preset applied on top of them ([`Photo::import_look`]).
@@ -370,7 +371,12 @@ impl Photo {
}
/// Edited by the user: settings differ from what import gave the photo.
pub fn is_edited(&self) -> bool {
*self.develop != self.import_defaults() && !self.develop.is_unedited()
let mut defaults = self.import_defaults();
// As Shot reads the source's current WB; stored numbers can predate a decoder update.
if self.develop.wb.mode == WbMode::AsShot && defaults.wb.mode == WbMode::AsShot {
defaults.wb = self.develop.wb;
}
*self.develop != defaults && !self.develop.is_unedited()
}
/// Capture time if known, else import time (sort key).
pub fn date(&self) -> &str {
@@ -399,12 +405,16 @@ pub struct Album {
/// The Quick Collection: a regular album that B adds to (one per library).
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub quick: bool,
/// Its place among the albums of the same kind in its folder, once the user ordered them by
/// hand (see [`crate::Catalog::album_children`]); `None`: listed by name.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub order: Option<u32>,
}
impl Album {
/// A regular (manual) album.
pub fn new(id: AlbumId, name: impl Into<String>) -> Album {
Album { id, name: name.into(), parent: None, folder: false, photos: Vec::new(), cover: None, smart: None, quick: false }
Album { id, name: name.into(), parent: None, folder: false, photos: Vec::new(), cover: None, smart: None, quick: false, order: None }
}
pub fn is_smart(&self) -> bool {
self.smart.is_some()
@@ -430,7 +440,7 @@ mod edited_tests {
}
#[test]
fn sony_and_nikon_raws_use_relative_white_balance() {
fn supported_raws_use_relative_white_balance() {
for (name, format, relative) in [
("a.arw", "ARW", true),
("a.nef", "NEF", true),
@@ -438,6 +448,8 @@ mod edited_tests {
("a.rw2", "RW2", true),
("a.rwl", "RWL", true),
("a.raw", "RAW", true),
("a.raf", "rAf", true),
("a.cr3", "cr3", true),
("a.dng", "DNG", false),
] {
let mut p = Photo::new(PhotoId(1), Source::Demo { scene: 0 }, name, format, 10, 10, "2026-10-01T00:00:00");
@@ -446,6 +458,12 @@ mod edited_tests {
assert_eq!(p.relative_wb(), relative, "{format}");
let wb = p.camera_defaults().wb;
assert_eq!((wb.temp, wb.tint), if relative { (6500.0, 0.0) } else { (5200.0, 4.0) }, "{format}");
p.develop = Arc::new(DevelopSettings::for_raw(5200.0, 4.0));
assert!(!p.is_edited(), "{format}: old As Shot values are not an edit");
let mut custom = (*p.develop).clone();
custom.wb.mode = WbMode::Custom;
p.develop = Arc::new(custom);
assert!(p.is_edited(), "{format}: Custom WB is still an edit");
// shown from the embedded preview: a rendered image, not a relative-WB raw
p.preview_only = Some("unsupported".into());
assert!(!p.relative_wb());
+11
View File
@@ -525,6 +525,17 @@ pub fn folder_within(path: &str, root: &str) -> bool {
key_within(&folder_key(path), &folder_key(root))
}
/// The names below folder `root` that lead to `path`, as the caller spelled them (`None` when
/// `path` is not at or inside `root`; empty when it is `root`). Compared by [`folder_key`], but
/// the names keep their case, which the key lowers on Windows.
pub fn folder_rest(path: &str, root: &str) -> Option<Vec<String>> {
if !key_within(&folder_key(path), &folder_key(root)) {
return None;
}
let depth = split_path(root, false).parts.len();
Some(split_path(path, false).parts.into_iter().skip(depth).collect())
}
/// Whether a library photo (not one only browsed in Local) lies in the folder whose
/// [`folder_key`] is `root`. A plain POSIX path (no `\`, `.`, `..`, repeated or trailing
/// separators) is its own identity, so it is read without building a key for it: the check runs
+12 -2
View File
@@ -58,12 +58,22 @@ fn albums_and_folders() {
let a = photo(&mut c, "a.jpg", "2026-04-01");
let f = c.alloc_album_id();
c.apply(Op::AddAlbum {
album: Album { id: f, name: "Trips".into(), parent: None, folder: true, photos: vec![], cover: None, smart: None, quick: false },
album: Album { id: f, name: "Trips".into(), parent: None, folder: true, photos: vec![], cover: None, smart: None, quick: false, order: None },
})
.unwrap();
let al = c.alloc_album_id();
c.apply(Op::AddAlbum {
album: Album { id: al, name: "Alps".into(), parent: Some(f), folder: false, photos: vec![a], cover: None, smart: None, quick: false },
album: Album {
id: al,
name: "Alps".into(),
parent: Some(f),
folder: false,
photos: vec![a],
cover: None,
smart: None,
quick: false,
order: None,
},
})
.unwrap();
assert_eq!(c.albums_of(a), vec![al]);
+122
View File
@@ -0,0 +1,122 @@
//! Album order: the children of a folder (or of the top level) are listed folders first, then
//! alphabetically, until some of them are given a place of their own (`Album.order`); moving an
//! album to another folder drops its place, and undoing the move brings it back.
use crate::*;
fn add(c: &mut Catalog, name: &str, parent: Option<AlbumId>, folder: bool) -> AlbumId {
let id = c.alloc_album_id();
c.apply(Op::AddAlbum { album: Album { parent, folder, ..Album::new(id, name) } }).unwrap();
id
}
fn names(c: &Catalog, parent: Option<AlbumId>) -> Vec<String> {
c.album_children(parent).iter().map(|a| a.name.clone()).collect()
}
fn place(c: &mut Catalog, id: AlbumId, order: Option<u32>) {
c.apply(Op::SetAlbumOrder { id, order }).unwrap();
}
/// Given albums that were never ordered, when they are listed, then folders come first and each
/// group is alphabetical regardless of case.
#[test]
fn children_are_folders_first_then_alphabetical() {
let mut c = Catalog::new();
add(&mut c, "beta", None, false);
add(&mut c, "Alpha", None, false);
add(&mut c, "zoo", None, true);
add(&mut c, "Archive", None, true);
assert_eq!(names(&c, None), ["Archive", "zoo", "Alpha", "beta"]);
assert!(!c.album_children_are_ordered(None));
}
/// Given some children with a place of their own, then they follow it; the others come after, by
/// name; folders stay ahead of albums.
#[test]
fn ordered_children_follow_their_order() {
let mut c = Catalog::new();
let a = add(&mut c, "A", None, false);
let b = add(&mut c, "B", None, false);
let cc = add(&mut c, "C", None, false);
add(&mut c, "D", None, false);
let f = add(&mut c, "F", None, true);
place(&mut c, cc, Some(0));
place(&mut c, a, Some(1));
assert_eq!(names(&c, None), ["F", "C", "A", "B", "D"]);
assert!(c.album_children_are_ordered(None));
// equal places fall back to the name, so a damaged catalog still lists deterministically
place(&mut c, b, Some(1));
assert_eq!(names(&c, None), ["F", "C", "A", "B", "D"]);
// another container is unaffected
let inner = add(&mut c, "Z", Some(f), false);
add(&mut c, "Y", Some(f), false);
assert_eq!(names(&c, Some(f)), ["Y", "Z"]);
assert!(!c.album_children_are_ordered(Some(f)));
place(&mut c, inner, Some(0));
assert_eq!(names(&c, Some(f)), ["Z", "Y"]);
}
/// The one-pass map lists every folder's children exactly as `album_children` does.
#[test]
fn the_children_map_matches_the_children_of_each_folder() {
let mut c = Catalog::new();
let f = add(&mut c, "F", None, true);
let g = add(&mut c, "G", Some(f), true);
for (name, parent) in [("b", None), ("a", None), ("y", Some(f)), ("x", Some(f)), ("q", Some(g))] {
add(&mut c, name, parent, false);
}
let x = c.albums().find(|a| a.name == "x").unwrap().id;
place(&mut c, x, Some(7));
let map = c.album_children_by_parent();
for parent in [None, Some(f), Some(g)] {
let from_map: Vec<AlbumId> = map.get(&parent).map(|v| v.iter().map(|a| a.id).collect()).unwrap_or_default();
let direct: Vec<AlbumId> = c.album_children(parent).iter().map(|a| a.id).collect();
assert_eq!(from_map, direct, "{parent:?}");
}
assert!(!map.contains_key(&Some(AlbumId(999))));
}
/// A place is one undo step, and `None` puts the album back among the alphabetical ones.
#[test]
fn a_place_can_be_set_cleared_and_undone() {
let mut c = Catalog::new();
let a = add(&mut c, "A", None, false);
add(&mut c, "B", None, false);
let inv = c.apply(Op::SetAlbumOrder { id: a, order: Some(5) }).unwrap();
assert_eq!(c.album(a).unwrap().order, Some(5));
c.apply(inv).unwrap();
assert_eq!(c.album(a).unwrap().order, None);
assert!(c.apply(Op::SetAlbumOrder { id: AlbumId(999), order: Some(1) }).is_err(), "no such album");
}
/// Moving to another folder drops the album's place there (it belonged to the old neighbours),
/// and undoing the move gives it back.
#[test]
fn moving_drops_the_place_and_undo_restores_it() {
let mut c = Catalog::new();
let from = add(&mut c, "From", None, true);
let to = add(&mut c, "To", None, true);
let a = add(&mut c, "A", Some(from), false);
place(&mut c, a, Some(3));
let inv = c.apply(Op::MoveAlbum { id: a, parent: Some(to) }).unwrap();
assert_eq!((c.album(a).unwrap().parent, c.album(a).unwrap().order), (Some(to), None));
c.apply(inv).unwrap();
assert_eq!((c.album(a).unwrap().parent, c.album(a).unwrap().order), (Some(from), Some(3)));
// "moving" to where it already is changes nothing
c.apply(Op::MoveAlbum { id: a, parent: Some(from) }).unwrap();
assert_eq!(c.album(a).unwrap().order, Some(3));
}
/// Catalogs written before albums had an order load as they were; an unset order is not written.
#[test]
fn albums_without_an_order_load_and_save_unchanged() {
let old = r#"{"id":4,"name":"Old","parent":null,"folder":false,"photos":[],"cover":null}"#;
let a: Album = serde_json::from_str(old).unwrap();
assert_eq!(a.order, None);
assert!(!serde_json::to_string(&a).unwrap().contains("order"));
let mut a = a;
a.order = Some(2);
let back: Album = serde_json::from_str(&serde_json::to_string(&a).unwrap()).unwrap();
assert_eq!(back.order, Some(2));
}
+15 -1
View File
@@ -47,6 +47,19 @@ fn v1_library_loads_and_is_upgraded() {
assert_eq!((r.replayed, r.upgraded_from), (1, None));
}
/// A library saved before albums could be ordered by hand opens as it was and is rewritten as v3.
#[test]
fn v2_library_loads_and_is_upgraded() {
let (base, log, full) = legacy_parts();
let m = MemStore::new();
m.set(SNAPSHOT, format!("{{\"format\":\"lightcraft-catalog\",\"version\":2,\"seq\":2,\"catalog\":{}}}\n", base.to_snapshot()).into_bytes());
m.set(LOG, log.into_bytes());
let (_, c, r) = Journal::open(Box::new(m.clone())).unwrap();
assert_eq!(c.to_snapshot(), full.to_snapshot());
assert_eq!((r.replayed, r.upgraded_from), (1, Some(2)));
assert_eq!(snapshot_version(&m), 3);
}
#[test]
fn versionless_log_only_library_is_upgraded() {
let mut c = Catalog::new();
@@ -179,8 +192,9 @@ fn op_variants_are_versioned() {
| Op::SetLabelName { .. }
| Op::Batch { .. } => 1,
Op::SetBrowsed { .. } => 2,
Op::SetAlbumOrder { .. } => 3,
}
}
let newest = since(&Op::SetBrowsed { folder: String::new(), at: None });
let newest = since(&Op::SetAlbumOrder { id: AlbumId(0), order: None });
assert_eq!(newest, VERSION, "the newest op's version must be the current format version");
}
+2
View File
@@ -59,6 +59,7 @@ fn op_for(c: &mut Catalog, kind: u8, a: u8, b: u8) -> Op {
cover: None,
smart: None,
quick: false,
order: None,
},
}
}
@@ -77,6 +78,7 @@ fn op_for(c: &mut Catalog, kind: u8, a: u8, b: u8) -> Op {
13 => Op::Batch { ops: c.remove_from_stacks_ops(&[pid]) },
14 => c.set_top_ops(pid).unwrap_or(Op::Batch { ops: vec![] }),
9 if b.is_multiple_of(5) => Op::SetLabelName { label: ColorLabel::ALL[a as usize % 5], name: (a % 3 != 1).then(|| format!("Label {b}")) },
9 if b % 7 == 3 => Op::SetAlbumOrder { id: aid, order: (!b.is_multiple_of(3)).then_some(u32::from(b % 5)) },
9 if b.is_multiple_of(2) => {
Op::SetFile { id: pid, file_name: format!("Renamed_{b}.jpg"), source: Source::File { path: format!("/p/Renamed_{b}.jpg") } }
}
+5
View File
@@ -12,6 +12,7 @@ workspace = true
lightcraft-geom = { workspace = true }
lightcraft-color = { workspace = true }
lightcraft-raster = { workspace = true }
lightcraft-heif = { workspace = true, optional = true }
serde = { workspace = true }
bytemuck = { workspace = true }
thiserror = { workspace = true }
@@ -34,6 +35,10 @@ ravif = { workspace = true, optional = true }
default = ["parallel", "jxl", "avif"]
parallel = ["dep:rayon", "lightcraft-raster/parallel", "jpeg-decoder/rayon"]
jxl = ["dep:jxl-oxide"]
# HEIC/HEIF decode via the optional lightcraft-heif crate (heic-rs, pure Rust). Off by default so
# distributors choose it at build time (HEVC patents are a distributor's call, same policy as
# PhotoCraft); `cargo xtask ci` clippies and tests it (`--features heif`), releases don't ship it yet.
heif = ["dep:lightcraft-heif"]
## AVIF encoding via ravif/rav1e (pure Rust, BSD). Native targets only; a no-op on wasm32.
avif = ["dep:ravif"]
+1
View File
@@ -14,6 +14,7 @@ let d: Decoded = decode(&bytes, DecodeOptions::default())?;
let working: Rgb32f = to_working(&d); // linear Rec.2020 D65 (Bradford)
let fast = decode(&bytes, DecodeOptions::fit(2048, 2048))?; // JPEG: DCT-domain 1/2..1/8 scaling
let t: Thumbnail = decode_thumbnail(&bytes, 256)?; // embedded EXIF/MPF preview or scaled decode
let h: Header = read_header(&bytes)?; // stored width/height + orientation, no pixels decoded
let icc = icc::write_named(NamedSpace::DisplayP3); // or icc::write_matrix_trc(&space, &Trc::…)
let meta = EncodeMeta { icc: Some(&icc), exif: Some(&exif), xmp: Some(&xmp) };
+125
View File
@@ -0,0 +1,125 @@
//! HEIF / HEIC (the iPhone and Mac photo format), read-only.
//!
//! Decoding lives in the optional `lightcraft-heif` crate (heic-rs, a pure-Rust HEVC
//! still-picture decoder), enabled by this crate's `heif` feature — off by default because HEVC
//! is patent-encumbered and whether a build carries an HEVC decoder is the distributor's call
//! (the same policy as the sibling PhotoCraft). Without the feature, HEIF files are still
//! detected and opening one is an [`Error::Unsupported`] error, never a panic.
//!
//! HEIF records orientation in the container (`irot`/`imir`, plus a `clap` crop), not in EXIF,
//! so the decoder applies them and `orientation` is reported as 1: the EXIF Orientation tag only
//! mirrors what the container says, and applying it as well would turn the photo twice.
//! The samples come back in the container's own primaries with its ICC profile (if any); like
//! every decoder here, the colour interpretation happens in [`crate::convert::finish`].
#[cfg(feature = "heif")]
use crate::convert::{Buf, Meta, Model, Raw, finish};
use crate::{DecodeOptions, Decoded, Error, Format, Result};
const F: Format = Format::Heif;
/// The reason given when this build has no HEIF decoder.
#[cfg(not(feature = "heif"))]
pub(crate) const NOT_IN_BUILD: &str = "HEIC/HEIF support isn't included in this build of LightCraft";
/// The reason a HEIF file that no build can open is refused (this crate decodes HEVC stills
/// only; the wrapper names the exact construct in its own errors).
#[cfg(feature = "heif")]
const NOT_DECODED: &str = "the image is not a decodable HEVC still (image sequence, overlay, alpha-less grid or AVIF-coded item)";
#[cfg(not(feature = "heif"))]
pub(crate) fn decode(_bytes: &[u8], _opts: &DecodeOptions) -> Result<Decoded> {
Err(Error::Unsupported(F, NOT_IN_BUILD))
}
#[cfg(feature = "heif")]
fn err(e: lightcraft_heif::Error) -> Error {
match e {
// This crate's Unsupported takes a fixed reason; the dynamic wording lives in the
// lightcraft-heif error itself.
lightcraft_heif::Error::Unsupported(_) => Error::Unsupported(F, NOT_DECODED),
lightcraft_heif::Error::Limit(m) | lightcraft_heif::Error::Malformed(m) => Error::Malformed(F, m),
}
}
#[cfg(feature = "heif")]
pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
// The container alone: the declared size is checked before any pixel is decoded.
let info = lightcraft_heif::probe(bytes).map_err(err)?;
if (info.width as u64).saturating_mul(info.height as u64) > opts.max_pixels {
return Err(Error::TooLarge(info.width as u64, info.height as u64));
}
let options = lightcraft_heif::Options { max_pixels: opts.max_pixels, apply_transforms: true };
let decoded = lightcraft_heif::decode(bytes, &options).map_err(err)?;
// The decoder re-reads the container; trust its output's shape, not the probe's.
let buf = if decoded.sixteen_bit {
let mut v = Vec::with_capacity(decoded.data.len() / 2);
for pair in decoded.data.as_chunks::<2>().0 {
v.push(u16::from_ne_bytes(*pair));
}
Buf::U16(v)
} else {
Buf::U8(decoded.data)
};
let raw = Raw {
width: decoded.width as usize,
height: decoded.height as usize,
model: Model::Rgb,
alpha: decoded.has_alpha,
premultiplied: false,
buf,
bit_depth: if decoded.sixteen_bit { 16 } else { 8 },
};
let meta = Meta { icc: decoded.icc, exif: decoded.exif, xmp: decoded.xmp, orientation: Some(1), ..Default::default() };
finish(F, raw, meta, (decoded.width, decoded.height), opts)
}
#[cfg(test)]
mod tests {
use crate::{Format, sniff};
/// The container header both builds recognise, whatever the feature says.
const HEIC_HEADER: &[u8] = b"\0\0\0\x18ftypheic\0\0\0\0mif1heic";
#[test]
fn heif_is_always_recognised() {
assert_eq!(sniff(HEIC_HEADER), Some(Format::Heif));
}
#[test]
fn can_decode_follows_the_feature() {
assert_eq!(Format::Heif.can_decode(), cfg!(feature = "heif"));
}
#[cfg(not(feature = "heif"))]
#[test]
fn without_the_feature_heif_is_a_clear_unsupported_error() {
let r = crate::decode(HEIC_HEADER, crate::DecodeOptions::default());
assert!(matches!(&r, Err(crate::Error::Unsupported(Format::Heif, why)) if why.contains("isn't included in this build")), "{r:?}");
}
/// Found by PhotoCraft's `decode_heif` fuzz target: a malformed box makes heic-rs 0.1.1 slice
/// out of range (`boxes.rs:130`). It must be an error, not a crash.
#[cfg(feature = "heif")]
const HEIC_RS_BOX_PANIC: [u8; 72] = [
0x00, 0x00, 0x00, 0x24, 0x66, 0x74, 0x79, 0x70, 0x68, 0x65, 0x69, 0x63, 0x00, 0x00, 0x00, 0x00, 0x6d, 0x69, 0x66, 0x31, 0x4d, 0x69, 0x50,
0x72, 0x6d, 0x69, 0x61, 0x66, 0x4d, 0x69, 0x48, 0x42, 0x72, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x6d, 0x65, 0x74, 0x61, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x75, 0x75, 0x69, 0x64, 0x00, 0x00, 0x00, 0x00, 0x03, 0x08, 0x08, 0x08, 0x00, 0x03, 0x01, 0x03, 0x70,
0x00, 0xa8, 0x00,
];
#[cfg(feature = "heif")]
#[test]
fn a_heic_rs_panic_is_a_malformed_error() {
let r = crate::decode(&HEIC_RS_BOX_PANIC, crate::DecodeOptions::default());
assert!(matches!(&r, Err(crate::Error::Malformed(Format::Heif, _))), "{r:?}");
}
#[cfg(feature = "heif")]
#[test]
fn an_image_sequence_is_unsupported() {
// An ftyp with no meta box: a video track, not a still.
let r = crate::decode(b"\0\0\0\x18ftypmsf1\0\0\0\0msf1hevc", crate::DecodeOptions::default());
assert!(matches!(&r, Err(crate::Error::Unsupported(Format::Heif, _))), "{r:?}");
}
}
+92 -1
View File
@@ -108,13 +108,64 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
decode_with_fallback(bytes, opts, None)
}
/// Stored dimensions and EXIF orientation from the markers, without decoding the scan data. The
/// file is refused, as a decode would refuse it, when it has no frame header, an unsupported frame
/// type or component count, or when its markers don't run to an end-of-image marker after the
/// image data (a truncated file). Damage *inside* the entropy-coded data can't be seen this way.
pub(crate) fn header(bytes: &[u8]) -> Result<(u32, u32, u16)> {
let m = parse_markers(bytes).ok_or_else(|| Error::Malformed(F, "missing SOI".into()))?;
if m.components == 0 {
return Err(Error::Malformed(F, "no frame header".into()));
}
check_size(F, m.width as u64, m.height as u64, &DecodeOptions::default())?;
if !matches!(m.sof, 0xC0..=0xC3) {
return Err(Error::Unsupported(F, "arithmetic-coded or hierarchical JPEG"));
}
if !matches!(m.components, 1 | 3 | 4) {
return Err(Error::Malformed(F, format!("{} colour components", m.components)));
}
if m.first_scan_components == 0 || !reaches_eoi(bytes) {
return Err(Error::Malformed(F, "truncated: the image data doesn't end with an end-of-image marker".into()));
}
let orientation = m.exif.as_deref().map(exif::summarize).unwrap_or_default().orientation.unwrap_or(1);
Ok((m.width, m.height, orientation))
}
/// Whether the markers after SOI, followed through every segment and scan, reach an EOI marker
/// (the first one: the primary image's, ahead of any MPF images appended to the file). Segments
/// are skipped by their length; in scan data `FF 00` (stuffing), `FF FF` (fill) and RSTn are not
/// markers that end it.
fn reaches_eoi(b: &[u8]) -> bool {
let mut p = 2usize;
loop {
// entropy-coded data (or garbage between segments, as `parse_markers` tolerates it)
let Some(skip) = b.get(p..).and_then(find_ff) else { return false };
p = p.saturating_add(skip);
let Some(&marker) = b.get(p.saturating_add(1)) else { return false };
p = match marker {
0xD9 => return true,
0xFF => p.saturating_add(1),
0x00 | 0x01 | 0xD0..=0xD8 => p.saturating_add(2),
_ => {
let Some(len) = b.get(p.saturating_add(2)..p.saturating_add(4)).map(|l| u16::from_be_bytes([l[0], l[1]]) as usize) else {
return false;
};
if len < 2 {
return false;
}
p.saturating_add(2).saturating_add(len)
}
};
}
}
pub(crate) fn decode_with_fallback(bytes: &[u8], opts: &DecodeOptions, fallback: Option<crate::NamedSpace>) -> Result<Decoded> {
let m = parse_markers(bytes).ok_or_else(|| Error::Malformed(F, "missing SOI".into()))?;
if m.components == 0 {
return Err(Error::Malformed(F, "no frame header".into()));
}
check_size(F, m.width as u64, m.height as u64, opts)?;
let scale_to = opts.max_size.filter(|&(mw, mh)| mw > 0 && mh > 0 && (m.width >= 2 * mw || m.height >= 2 * mh));
let scale_to = opts.max_size.filter(|&(mw, mh)| mw > 0 && mh > 0 && (m.width >= mw.saturating_mul(2) || m.height >= mh.saturating_mul(2)));
// zune-jpeg handles the common cases fastest; jpeg-decoder covers DCT scaling, CMYK/YCCK,
// 12-bit and non-interleaved sequential scans (which zune-jpeg 0.5 mis-decodes with subsampling).
let non_interleaved = matches!(m.sof, 0xC0 | 0xC1) && m.first_scan_components < m.components;
@@ -175,6 +226,21 @@ fn decode_jpeg_decoder(bytes: &[u8], m: &Markers, scale_to: Option<(u32, u32)>)
Ok(Raw { width: w, height: h, model, alpha: false, premultiplied: false, buf, bit_depth: depth })
}
/// The first `0xFF` byte, eight bytes at a time (scan data has one every few hundred bytes: this
/// is most of [`header`]'s time).
fn find_ff(b: &[u8]) -> Option<usize> {
const ONES: u64 = 0x0101_0101_0101_0101;
let (words, tail) = b.as_chunks::<8>();
for (i, w) in words.iter().enumerate() {
// a zero byte in !w is an 0xFF byte in w
let x = !u64::from_ne_bytes(*w);
if x.wrapping_sub(ONES) & !x & (ONES << 7) != 0 {
return w.iter().position(|&v| v == 0xFF).map(|j| i * 8 + j);
}
}
tail.iter().position(|&v| v == 0xFF).map(|j| words.len() * 8 + j)
}
/// The box to request from `jpeg-decoder::scale` so the decoded image still covers `mw × mh` once
/// fitted with the source aspect ratio.
fn scaled_request(w: u32, h: u32, mw: u32, mh: u32) -> (u16, u16) {
@@ -297,6 +363,31 @@ mod tests {
assert_eq!(scaled_request(100, 100, 256, 256), (100, 100));
}
#[test]
fn find_ff_matches_a_plain_search() {
let mut b: Vec<u8> = (0..300u32).map(|i| (i * 7 % 255) as u8).collect();
for at in [None, Some(0), Some(7), Some(8), Some(15), Some(203), Some(296), Some(299)] {
if let Some(i) = at {
b[i] = 0xFF;
}
for start in [0, 1, 5, 9, 290, 299, 300] {
let s = &b[start..];
assert_eq!(find_ff(s), s.iter().position(|&v| v == 0xFF), "{at:?} from {start}");
}
}
}
#[test]
fn unbounded_fit_box_decodes_at_full_size() {
// a full-size load asks for the largest box (`usize::MAX` saturated to `u32::MAX`): no DCT scaling,
// and no overflow working out whether to scale
use crate::{ChromaSubsampling, EncodeImage, EncodeMeta, Samples, encode_jpeg};
let pixels = vec![128u8; 32 * 24 * 3];
let jpeg = encode_jpeg(&EncodeImage::new(32, 24, 3, Samples::U8(&pixels)), 90, ChromaSubsampling::S444, &EncodeMeta::default()).unwrap();
let d = crate::decode(&jpeg, DecodeOptions::fit(u32::MAX, u32::MAX)).unwrap();
assert_eq!((d.width, d.height), (32, 24));
}
#[test]
fn markers_on_garbage() {
assert!(parse_markers(b"").is_none());
+50 -1
View File
@@ -6,6 +6,8 @@
//! profiles (matrix/TRC exactly, LUT/CMYK via the `moxcms` CMS), 16-bit and float precision are
//! preserved. EXIF/XMP/ICC blobs are passed through untouched; orientation is reported, never applied.
//! - [`to_working`] converts a decode result to linear Rec.2020 D65 (the pipeline working space).
//! - [`read_header`] reads a file's stored dimensions and orientation from its headers, without
//! decoding pixels (import probes).
//! - [`decode_thumbnail`] is the fast path for grid thumbnails (EXIF thumbnail or DCT-scaled decode).
//! - [`encode`] writes JPEG, PNG, TIFF, lossless WebP and (native, feature `avif`) AVIF, embedding
//! ICC/EXIF/XMP. [`icc::write_matrix_trc`] builds profiles for export.
@@ -17,6 +19,7 @@
mod convert;
pub mod encode;
pub mod exif;
mod heif;
pub mod icc;
mod jpeg;
pub mod jpeg_par;
@@ -156,6 +159,52 @@ pub fn decode(bytes: &[u8], opts: DecodeOptions) -> Result<Decoded> {
std::panic::catch_unwind(|| decode_unguarded(bytes, format, &opts)).unwrap_or_else(|_| Err(Error::Malformed(format, "decoder panicked".into())))
}
/// What a file's headers say about its image: enough to catalogue it without decoding pixels.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Header {
pub format: Format,
/// Full-resolution stored dimensions, orientation **not** applied ([`Decoded::source_width`]).
pub width: u32,
pub height: u32,
/// EXIF/TIFF orientation 1..=8, 1 when absent ([`Decoded::orientation`]).
pub orientation: u16,
}
/// Read the stored dimensions and orientation of any format [`decode`] supports, agreeing with
/// what a decode reports, without decoding pixels: JPEG, PNG, TIFF, WebP and PSD are read from
/// their headers (markers, chunks, IFD, resources). GIF, BMP and JPEG XL fall back to a small
/// decode.
///
/// The header readers also refuse what a decode would refuse before reaching the pixels (an
/// unsupported layout, compression or frame type) and files whose image data is cut short
/// (truncated: the data runs past the end of the file, or a JPEG's markers don't reach its
/// end-of-image marker). Damage *inside* compressed data that is all present is only found by a
/// decode. Malformed input yields an error, never a panic (as for [`decode`], a panic inside a
/// third-party parser is caught on unwinding targets).
pub fn read_header(bytes: &[u8]) -> Result<Header> {
let format = sniff(bytes).ok_or(Error::UnknownFormat)?;
std::panic::catch_unwind(|| read_header_unguarded(bytes, format))
.unwrap_or_else(|_| Err(Error::Malformed(format, "header reader panicked".into())))
}
/// [`read_header`] without the panic guard (for fuzzing our own code paths).
#[doc(hidden)]
pub fn read_header_unguarded(bytes: &[u8], format: Format) -> Result<Header> {
let (width, height, orientation) = match format {
Format::Jpeg => jpeg::header(bytes)?,
Format::Png => png_codec::header(bytes)?,
Format::Tiff => tiff_codec::header(bytes)?,
Format::WebP => webp::header(bytes)?,
Format::Psd => psd::header(bytes)?,
// rare here (or, for JPEG XL, without a header reader yet): a small decode, as before
_ => {
let d = decode_unguarded(bytes, format, &DecodeOptions::fit(64, 64))?;
(d.source_width, d.source_height, d.orientation)
}
};
Ok(Header { format, width, height, orientation })
}
/// Decode a JPEG preview with its enclosing container's colour space as a fallback.
/// A JPEG carrying ICC or EXIF metadata keeps its own interpretation. The fallback is applied
/// before transfer decoding and linear-light resizing, never to already-resampled pixels.
@@ -179,8 +228,8 @@ pub fn decode_unguarded(bytes: &[u8], format: Format, opts: &DecodeOptions) -> R
Format::Jxl => jxl::decode(bytes, &opts),
#[cfg(not(feature = "jxl"))]
Format::Jxl => Err(Error::Unsupported(format, "built without the `jxl` feature")),
Format::Heif => heif::decode(bytes, &opts),
Format::Avif => Err(Error::Unsupported(format, "no pure-Rust, permissively licensed AV1 decoder yet")),
Format::Heif => Err(Error::Unsupported(format, "no pure-Rust, permissively licensed HEVC decoder yet")),
Format::RawTiffLike | Format::RawOther => Err(Error::Unsupported(format, "camera raw: decode with lightcraft-raw")),
}
}
+69
View File
@@ -57,6 +57,75 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
finish(F, raw, meta, (w, h), opts)
}
/// Stored dimensions and EXIF orientation (`eXIf`, wherever it is) from the chunk headers, without
/// inflating the image data. Refused, as a decode would refuse it: an invalid IHDR, an indexed
/// image without a palette, no IDAT, or a file that ends before the chunk after the last IDAT
/// begins (truncated image data). Chunk CRCs and the deflate stream are not checked.
pub(crate) fn header(b: &[u8]) -> Result<(u32, u32, u16)> {
if !b.starts_with(b"\x89PNG\r\n\x1a\n") {
return Err(err("bad signature"));
}
let be32 = |p: usize| b.get(p..p.checked_add(4)?).map(|v| u32::from_be_bytes([v[0], v[1], v[2], v[3]]));
if be32(8) != Some(13) || b.get(12..16) != Some(b"IHDR") {
return Err(err("missing IHDR"));
}
let ihdr = b.get(16..29).ok_or_else(|| err("truncated IHDR"))?;
let (w, h) = (u32::from_be_bytes([ihdr[0], ihdr[1], ihdr[2], ihdr[3]]), u32::from_be_bytes([ihdr[4], ihdr[5], ihdr[6], ihdr[7]]));
let (depth, color, compression, filter, interlace) = (ihdr[8], ihdr[9], ihdr[10], ihdr[11], ihdr[12]);
if w > i32::MAX as u32 || h > i32::MAX as u32 {
return Err(err("dimensions out of range"));
}
check_size(F, w as u64, h as u64, &DecodeOptions::default())?;
let depth_ok = match color {
0 => matches!(depth, 1 | 2 | 4 | 8 | 16),
3 => matches!(depth, 1 | 2 | 4 | 8),
2 | 4 | 6 => matches!(depth, 8 | 16),
_ => false,
};
if !depth_ok || compression != 0 || filter != 0 || interlace > 1 {
return Err(err("invalid IHDR"));
}
let (mut p, mut palette, mut idat, mut after_idat, mut exif) = (8usize, false, false, false, None);
while let Some(len) = be32(p) {
let Some(kind) = p.checked_add(4).and_then(|k| b.get(k..k.checked_add(4)?)) else { break };
if idat && kind != b"IDAT" {
// a chunk follows the image data: the IDAT run is complete
after_idat = true;
}
if kind == b"IEND" {
break;
}
let start = p.saturating_add(8);
let data = (len <= i32::MAX as u32).then(|| start.checked_add(len as usize).and_then(|end| b.get(start..end))).flatten();
let Some(data) = data else {
if after_idat {
break; // a damaged chunk after the image data: a decode ignores it too
}
return Err(err("truncated"));
};
match kind {
b"PLTE" => palette = true,
b"IDAT" if after_idat => {} // a stray IDAT after other chunks: the first run is the image
b"IDAT" => {
if color == 3 && !palette {
return Err(err("indexed image without a palette"));
}
idat = true;
}
b"eXIf" if exif.is_none() => exif = Some(data.strip_prefix(b"Exif\0\0").unwrap_or(data)),
_ => {}
}
p = start.saturating_add(data.len()).saturating_add(4);
}
if !idat {
return Err(err("no image data"));
}
if !after_idat {
return Err(err("truncated image data"));
}
Ok((w, h, exif.map(crate::exif::summarize).unwrap_or_default().orientation.unwrap_or(1)))
}
fn container_hint(info: &png::Info) -> Option<SourceSpace> {
if let Some(c) = &info.coding_independent_code_points {
let named = match c.color_primaries {
+265 -72
View File
@@ -69,8 +69,8 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let (icc, exif, xmp) = resources(res);
// Layer and mask information: only the layer count sign matters (merged transparency).
let lm_len = if psb { r.u64()? as usize } else { r.u32()? as usize };
let lm = r.take(lm_len)?;
let lm_len = if psb { r.u64()? } else { u64::from(r.u32()?) };
let lm = r.take(usize::try_from(lm_len).map_err(|_| err("truncated"))?)?;
let merged_transparency = if lm.len() >= if psb { 10 } else { 6 } {
let off = if psb { 8 } else { 4 };
let count = i16::from_be_bytes([lm[off], lm[off + 1]]);
@@ -88,33 +88,6 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
1 => return Err(Error::Unsupported(F, "1-bit bitmap PSD")),
_ => return Err(err("bad depth")),
};
let plane = width * height * bpc;
let row_bytes = width * bpc;
let mut planes: Vec<Vec<u8>> = Vec::with_capacity(channels);
match compression {
0 => {
for _ in 0..channels {
planes.push(r.take(plane)?.to_vec());
}
}
1 => {
let counts_len = channels * height;
let mut counts = Vec::with_capacity(counts_len);
for _ in 0..counts_len {
counts.push(if psb { r.u32()? as usize } else { r.u16()? as usize });
}
for c in 0..channels {
let mut out = Vec::with_capacity(plane);
for y in 0..height {
let src = r.take(counts[c * height + y])?;
unpackbits(src, row_bytes, &mut out);
}
planes.push(out);
}
}
_ => return Err(Error::Unsupported(F, "ZIP-compressed merged image data")),
}
let (model, n_color) = match mode {
1 | 8 => (Model::Gray, 1),
2 => (Model::Rgb, 1),
@@ -128,50 +101,106 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
return Err(err("too few channels"));
}
let alpha = merged_transparency && channels > n_color && mode != 2;
// Only the colour (+ alpha) planes are decoded; extra channels (spot colours, masks) are
// stored after them and never touched.
let used = n_color + alpha as usize;
let n = width * height;
let too_large = || Error::TooLarge(width as u64, height as u64);
let n = width.checked_mul(height).ok_or_else(too_large)?;
let row_bytes = width.checked_mul(bpc).ok_or_else(too_large)?;
let plane = n.checked_mul(bpc).ok_or_else(too_large)?;
let mut planes: Vec<Vec<u8>> = Vec::with_capacity(used);
match compression {
0 => {
// Raw planes are borrowed from the input, so their size is bounded by the file.
for _ in 0..used {
planes.push(r.take(plane)?.to_vec());
}
}
1 => {
// Row byte-count table: one entry per row per channel (all channels, since the data
// follows it). Taken from the input before anything is allocated from it.
let entry = if psb { 4 } else { 2 };
let table_len = channels.checked_mul(height).and_then(|c| c.checked_mul(entry)).ok_or_else(|| err("truncated"))?;
let table = r.take(table_len)?;
let count = |i: usize| -> usize {
let at = i.saturating_mul(entry);
match table.get(at..at.saturating_add(entry)) {
Some(&[a, b]) => usize::from(u16::from_be_bytes([a, b])),
Some(&[a, b, c, d]) => u32::from_be_bytes([a, b, c, d]) as usize,
_ => 0,
}
};
// PackBits expands at most 64× (a 2-byte repeat run yields 128 bytes), so a row needs
// at least this many encoded bytes. Requiring it (and the data being present) before
// allocating bounds the decoded planes by the file size: a tiny file can no longer
// declare gigabytes of zero-padded or repeat-run rows.
let min_row = row_bytes.div_ceil(128).saturating_mul(2);
let mut data_len = 0usize;
for i in 0..used.saturating_mul(height) {
let c = count(i);
if c < min_row {
return Err(err("RLE row too short for its width"));
}
data_len = data_len.saturating_add(c);
}
if data_len > bytes.len().saturating_sub(r.p) {
return Err(err("truncated"));
}
for c in 0..used {
let mut out = zeroed::<u8>(plane, too_large)?;
for (y, row) in out.chunks_exact_mut(row_bytes).enumerate() {
let src = r.take(count(c * height + y))?;
unpackbits(src, row)?;
}
planes.push(out);
}
}
_ => return Err(Error::Unsupported(F, "ZIP-compressed merged image data")),
}
let samples = n.checked_mul(used).ok_or_else(too_large)?;
// Interleave to device samples.
let buf = if mode == 2 {
// Indexed → 8-bit RGB via the 768-byte palette.
if color_mode_data.len() < 768 || depth != 8 {
let (Some(pal), true) = (color_mode_data.get(..768), depth == 8) else {
return Err(err("bad indexed palette"));
}
let mut v = Vec::with_capacity(n * 3);
for &i in &planes[0] {
};
let idx = planes.first().ok_or_else(|| err("missing plane"))?;
let mut v = zeroed::<u8>(n.checked_mul(3).ok_or_else(too_large)?, too_large)?;
for (o, &i) in v.as_chunks_mut::<3>().0.iter_mut().zip(idx) {
let i = i as usize;
v.extend_from_slice(&[color_mode_data[i], color_mode_data[256 + i], color_mode_data[512 + i]]);
*o = [pal[i], pal[256 + i], pal[512 + i]];
}
Buf::U8(v)
} else {
let cmyk = model == Model::Cmyk;
match bpc {
1 => {
let mut v = vec![0u8; n * used];
for (c, p) in planes.iter().take(used).enumerate() {
for i in 0..n {
let x = p[i];
let mut v = zeroed::<u8>(samples, too_large)?;
for (c, p) in planes.iter().enumerate() {
for (o, &x) in v.iter_mut().skip(c).step_by(used).zip(p) {
// PSD CMYK stores 255 = no ink; our CMYK convention is 0 = no ink.
v[i * used + c] = if model == Model::Cmyk && c < 4 { 255 - x } else { x };
*o = if cmyk && c < 4 { 255 - x } else { x };
}
}
Buf::U8(v)
}
2 => {
let mut v = vec![0u16; n * used];
for (c, p) in planes.iter().take(used).enumerate() {
for i in 0..n {
let x = u16::from_be_bytes([p[2 * i], p[2 * i + 1]]);
v[i * used + c] = if model == Model::Cmyk && c < 4 { 65535 - x } else { x };
let mut v = zeroed::<u16>(samples, too_large)?;
for (c, p) in planes.iter().enumerate() {
for (o, s) in v.iter_mut().skip(c).step_by(used).zip(p.as_chunks::<2>().0) {
let x = u16::from_be_bytes(*s);
*o = if cmyk && c < 4 { 65535 - x } else { x };
}
}
Buf::U16(v)
}
_ => {
let mut v = vec![0f32; n * used];
for (c, p) in planes.iter().take(used).enumerate() {
for i in 0..n {
let x = f32::from_be_bytes([p[4 * i], p[4 * i + 1], p[4 * i + 2], p[4 * i + 3]]);
v[i * used + c] = if model == Model::Cmyk && c < 4 { 1.0 - x } else { x };
let mut v = zeroed::<f32>(samples, too_large)?;
for (c, p) in planes.iter().enumerate() {
for (o, s) in v.iter_mut().skip(c).step_by(used).zip(p.as_chunks::<4>().0) {
let x = f32::from_be_bytes(*s);
*o = if cmyk && c < 4 { 1.0 - x } else { x };
}
}
Buf::F32(v)
@@ -182,25 +211,111 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
finish(F, raw, Meta { icc, exif, xmp, ..Default::default() }, (width as u32, height as u32), opts)
}
/// PackBits: append exactly `row_bytes` bytes (zero-padded if the run data is short).
fn unpackbits(mut src: &[u8], row_bytes: usize, out: &mut Vec<u8>) {
let target = out.len() + row_bytes;
while out.len() < target && !src.is_empty() {
let h = src[0] as i8;
src = &src[1..];
if h >= 0 {
let n = (h as usize + 1).min(src.len()).min(target - out.len());
out.extend_from_slice(&src[..n]);
src = &src[(h as usize + 1).min(src.len())..];
} else if h != -128 {
let n = ((1 - h as isize) as usize).min(target - out.len());
if let Some(&b) = src.first() {
out.extend(std::iter::repeat_n(b, n));
src = &src[1..];
/// Stored dimensions and EXIF orientation (resource 1058) from the header and image resources,
/// without reading the merged image data. Refused, as a decode would refuse it: an unsupported
/// depth, colour mode or compression, or merged image data that runs past the end of the file
/// (truncated). The run-length data itself is not unpacked.
pub(crate) fn header(bytes: &[u8]) -> Result<(u32, u32, u16)> {
let mut r = Rd { b: bytes, p: 0 };
if r.take(4)? != b"8BPS" {
return Err(err("bad signature"));
}
let psb = match r.u16()? {
1 => false,
2 => true,
_ => return Err(err("bad version")),
};
r.take(6)?;
let channels = r.u16()? as usize;
let height = r.u32()? as usize;
let width = r.u32()? as usize;
let depth = r.u16()?;
let mode = r.u16()?;
check_size(F, width as u64, height as u64, &DecodeOptions::default())?;
if channels == 0 || channels > 56 {
return Err(err("bad channel count"));
}
let cm_len = r.u32()? as usize;
let color_mode_data = r.take(cm_len)?;
let res_len = r.u32()? as usize;
let (_, exif, _) = resources(r.take(res_len)?);
let lm_len = if psb { r.u64()? as usize } else { r.u32()? as usize };
r.take(lm_len)?;
let compression = r.u16()?;
let bpc = match depth {
8 => 1usize,
16 => 2,
32 => 4,
1 => return Err(Error::Unsupported(F, "1-bit bitmap PSD")),
_ => return Err(err("bad depth")),
};
let n_color = match mode {
1 | 8 | 7 | 2 => 1,
3 => 3,
4 => 4,
9 => return Err(Error::Unsupported(F, "Lab PSD")),
_ => return Err(err("unknown colour mode")),
};
if channels < n_color {
return Err(err("too few channels"));
}
if mode == 2 && (color_mode_data.len() < 768 || depth != 8) {
return Err(err("bad indexed palette"));
}
let rest = bytes.len().saturating_sub(r.p);
let need = match compression {
0 => width.checked_mul(height).and_then(|n| n.checked_mul(bpc)).and_then(|n| n.checked_mul(channels)),
1 => {
// the row byte counts, then the rows they count
let rows = channels.checked_mul(height).ok_or_else(|| err("truncated"))?;
let table = rows.checked_mul(if psb { 4 } else { 2 }).filter(|&t| t <= rest).ok_or_else(|| err("truncated"))?;
let mut total = table;
for _ in 0..rows {
total = total.saturating_add(if psb { r.u32()? as usize } else { r.u16()? as usize });
}
Some(total)
}
_ => return Err(Error::Unsupported(F, "ZIP-compressed merged image data")),
};
if need.is_none_or(|n| n > rest) {
return Err(err("truncated"));
}
let orientation = exif.as_deref().map(crate::exif::summarize).unwrap_or_default().orientation.unwrap_or(1);
Ok((width as u32, height as u32, orientation))
}
/// A zero-filled buffer whose allocation failure is an error, not an abort.
fn zeroed<T: Clone + Default>(len: usize, too_large: impl Fn() -> Error) -> Result<Vec<T>> {
let mut v = Vec::new();
v.try_reserve_exact(len).map_err(|_| too_large())?;
v.resize(len, T::default());
Ok(v)
}
/// PackBits: decode one row into `row` (pre-zeroed, so short run data leaves zero padding).
/// A run that would write past the end of the row is an error; trailing bytes are ignored.
fn unpackbits(mut src: &[u8], row: &mut [u8]) -> Result<()> {
let mut at = 0;
while at < row.len() {
let Some((&h, rest)) = src.split_first() else { break };
src = rest;
let h = h as i8;
if h >= 0 {
let n = h as usize + 1;
let lit = src.get(..n).unwrap_or(src);
let dst = row.get_mut(at..at + lit.len()).ok_or_else(|| err("RLE row overruns its width"))?;
dst.copy_from_slice(lit);
at += lit.len();
src = src.get(n..).unwrap_or(&[]);
} else if h != -128 {
let n = (1 - h as isize) as usize;
let Some((&b, rest)) = src.split_first() else { break };
src = rest;
row.get_mut(at..at + n).ok_or_else(|| err("RLE row overruns its width"))?.fill(b);
at += n;
}
}
out.resize(target, 0);
Ok(())
}
fn resources(mut b: &[u8]) -> (Option<Vec<u8>>, Option<Vec<u8>>, Option<String>) {
@@ -234,11 +349,89 @@ mod tests {
#[test]
fn packbits() {
let mut out = Vec::new();
unpackbits(&[0xFE, 0xAA, 0x02, 0x80, 0x00, 0x2A], 6, &mut out);
assert_eq!(out, vec![0xAA, 0xAA, 0xAA, 0x80, 0x00, 0x2A]);
let mut out = Vec::new();
unpackbits(&[0x05, 1], 4, &mut out);
assert_eq!(out, vec![1, 0, 0, 0]);
let mut out = [0u8; 6];
unpackbits(&[0xFE, 0xAA, 0x02, 0x80, 0x00, 0x2A], &mut out).unwrap();
assert_eq!(out, [0xAA, 0xAA, 0xAA, 0x80, 0x00, 0x2A]);
// Short run data is zero-padded.
let mut out = [0u8; 4];
unpackbits(&[0x01, 1], &mut out).unwrap();
assert_eq!(out, [1, 0, 0, 0]);
// Truncated literal run: copies what is there.
let mut out = [0u8; 4];
unpackbits(&[0x03, 7, 8], &mut out).unwrap();
assert_eq!(out, [7, 8, 0, 0]);
}
#[test]
fn packbits_overrun_is_an_error() {
// Repeat run of 128 into a 4-byte row.
let mut out = [0u8; 4];
assert!(unpackbits(&[0x81, 0xAA], &mut out).is_err());
// Literal run of 6 into a 4-byte row.
assert!(unpackbits(&[0x05, 1, 2, 3, 4, 5, 6], &mut out).is_err());
}
/// PSD header + empty colour-mode/resources/layer sections + compression id.
fn header(channels: u16, w: u32, h: u32, depth: u16, mode: u16, compression: u16) -> Vec<u8> {
let mut b = b"8BPS".to_vec();
b.extend_from_slice(&1u16.to_be_bytes());
b.extend_from_slice(&[0; 6]);
b.extend_from_slice(&channels.to_be_bytes());
b.extend_from_slice(&h.to_be_bytes());
b.extend_from_slice(&w.to_be_bytes());
b.extend_from_slice(&depth.to_be_bytes());
b.extend_from_slice(&mode.to_be_bytes());
b.extend_from_slice(&[0; 12]); // colour mode data, resources, layer & mask: all empty
b.extend_from_slice(&compression.to_be_bytes());
b
}
#[test]
fn huge_rle_with_truncated_count_table_is_rejected_before_allocating() {
// 1 × 2^30 px, 56 channels: the row table alone would be 56 Gi entries; the file has 4 bytes.
let mut b = header(56, 1, 1 << 30, 32, 3, 1);
b.extend_from_slice(&[0, 0, 0, 0]);
let e = decode(&b, &DecodeOptions::default()).unwrap_err();
assert!(matches!(e, Error::Malformed(_, ref m) if m == "truncated"), "{e:?}");
}
#[test]
fn huge_rle_of_empty_rows_is_rejected_before_allocating() {
// 2^30 × 1 px RGB at 32 bits: every row count is 0, which used to zero-pad 3 × 4 GiB.
let mut b = header(3, 1 << 30, 1, 32, 3, 1);
b.extend_from_slice(&[0; 6]);
let e = decode(&b, &DecodeOptions::default()).unwrap_err();
assert!(matches!(e, Error::Malformed(_, ref m) if m.contains("too short")), "{e:?}");
// 1 Gpx RGB 8-bit whose (plausible) row counts claim more data than the file holds.
let mut b = header(3, 1 << 20, 1 << 10, 8, 3, 1);
b.extend_from_slice(&[0xFF; 3 * 1024 * 2]);
let e = decode(&b, &DecodeOptions::default()).unwrap_err();
assert!(matches!(e, Error::Malformed(_, ref m) if m == "truncated"), "{e:?}");
}
#[test]
fn rle_repeat_run_past_row_width_is_an_error() {
// 4 × 1 gray 8-bit; the single row is a 128-byte repeat run.
let mut b = header(1, 4, 1, 8, 1, 1);
b.extend_from_slice(&2u16.to_be_bytes());
b.extend_from_slice(&[0x81, 0x55]);
let e = decode(&b, &DecodeOptions::default()).unwrap_err();
assert!(matches!(e, Error::Malformed(_, ref m) if m.contains("overruns")), "{e:?}");
}
#[test]
fn rle_gray16_and_extra_channels_decode() {
// 2 × 2 gray 16-bit with an extra (unused) channel whose data is absent: only the used
// plane is decoded. Row: repeat run of 4 bytes 0x80 -> samples 0x8080.
let mut b = header(2, 2, 2, 16, 1, 1);
for _ in 0..4 {
b.extend_from_slice(&2u16.to_be_bytes());
}
b.extend_from_slice(&[0xFD, 0x80, 0xFD, 0x80]);
let d = decode(&b, &DecodeOptions::default()).unwrap();
assert_eq!((d.width, d.height), (2, 2));
let p = d.image.get(1, 1);
let q = d.image.get(0, 0);
assert!(p[0] > 0.0 && (p[0] - q[0]).abs() < 1e-6, "{p:?}");
}
}
+2 -1
View File
@@ -66,7 +66,8 @@ impl Format {
pub fn can_decode(self) -> bool {
match self {
Format::Jxl => cfg!(feature = "jxl"),
Format::Avif | Format::Heif | Format::RawTiffLike | Format::RawOther => false,
Format::Heif => cfg!(feature = "heif"),
Format::Avif | Format::RawTiffLike | Format::RawOther => false,
_ => true,
}
}
+63 -21
View File
@@ -15,10 +15,7 @@ fn err(e: impl std::fmt::Display) -> Error {
}
pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let mut limits = Limits::default();
limits.decoding_buffer_size = opts.max_pixels.saturating_mul(4 * 8).min(usize::MAX as u64) as usize;
limits.ifd_value_size = 32 << 20;
let mut d = Decoder::new(std::io::Cursor::new(bytes)).map_err(err)?.with_limits(limits);
let mut d = Decoder::new(std::io::Cursor::new(bytes)).map_err(err)?.with_limits(limits(opts));
let (w, h) = d.dimensions().map_err(err)?;
check_size(F, w as u64, h as u64, opts)?;
let ct = d.colortype().map_err(err)?;
@@ -31,23 +28,7 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let colormap = d.get_tag_u16_vec(Tag::ColorMap).ok();
let compression = d.get_tag_u32(Tag::Compression).ok();
let (model, n_color, n_samples, bits) = match ct {
ColorType::Gray(b) => (Model::Gray, 1, 1, b),
ColorType::GrayA(b) => (Model::Gray, 1, 2, b),
ColorType::RGB(b) => (Model::Rgb, 3, 3, b),
ColorType::RGBA(b) => (Model::Rgb, 3, 4, b),
ColorType::CMYK(b) => (Model::Cmyk, 4, 4, b),
ColorType::CMYKA(b) => (Model::Cmyk, 4, 5, b),
ColorType::Palette(b) => (Model::Rgb, 1, 1, b),
// JPEG-compressed YCbCr is converted to RGB by the JPEG decoder.
ColorType::YCbCr(b) if compression == Some(7) => (Model::Rgb, 3, 3, b),
ColorType::Multiband { bit_depth, num_samples } if num_samples >= 3 => (Model::Rgb, 3, num_samples as usize, bit_depth),
ColorType::Multiband { bit_depth, num_samples } if num_samples >= 1 => (Model::Gray, 1, num_samples as usize, bit_depth),
other => return Err(Error::Unsupported(F, unsupported_name(other))),
};
if bits < 8 && planar && n_samples > 1 {
return Err(Error::Unsupported(F, "planar TIFF with samples narrower than a byte"));
}
let (model, n_color, n_samples, bits) = layout(ct, compression, planar)?;
let has_alpha = n_samples > n_color && !matches!(ct, ColorType::Palette(_));
let premultiplied = has_alpha && extra.first() == Some(&1);
@@ -111,6 +92,67 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
finish(F, raw, meta, (w, h), opts)
}
fn limits(opts: &DecodeOptions) -> Limits {
let mut limits = Limits::default();
limits.decoding_buffer_size = opts.max_pixels.saturating_mul(4 * 8).min(usize::MAX as u64) as usize;
limits.ifd_value_size = 32 << 20;
limits
}
/// The sample layout of a colour type we can decode: (model, colour samples, samples per pixel,
/// bits per sample).
fn layout(ct: ColorType, compression: Option<u32>, planar: bool) -> Result<(Model, usize, usize, u8)> {
let l = match ct {
ColorType::Gray(b) => (Model::Gray, 1, 1, b),
ColorType::GrayA(b) => (Model::Gray, 1, 2, b),
ColorType::RGB(b) => (Model::Rgb, 3, 3, b),
ColorType::RGBA(b) => (Model::Rgb, 3, 4, b),
ColorType::CMYK(b) => (Model::Cmyk, 4, 4, b),
ColorType::CMYKA(b) => (Model::Cmyk, 4, 5, b),
ColorType::Palette(b) => (Model::Rgb, 1, 1, b),
// JPEG-compressed YCbCr is converted to RGB by the JPEG decoder.
ColorType::YCbCr(b) if compression == Some(7) => (Model::Rgb, 3, 3, b),
ColorType::Multiband { bit_depth, num_samples } if num_samples >= 3 => (Model::Rgb, 3, num_samples as usize, bit_depth),
ColorType::Multiband { bit_depth, num_samples } if num_samples >= 1 => (Model::Gray, 1, num_samples as usize, bit_depth),
other => return Err(Error::Unsupported(F, unsupported_name(other))),
};
if l.3 < 8 && planar && l.2 > 1 {
return Err(Error::Unsupported(F, "planar TIFF with samples narrower than a byte"));
}
Ok(l)
}
/// Stored dimensions and orientation from the first IFD, without reading the image data. Refused,
/// as a decode would refuse it: a layout or compression we can't decode, or strips/tiles that
/// start past the end of the file (truncated). The compressed data itself is not checked.
pub(crate) fn header(bytes: &[u8]) -> Result<(u32, u32, u16)> {
let mut d = Decoder::new(std::io::Cursor::new(bytes)).map_err(err)?.with_limits(limits(&DecodeOptions::default()));
let (w, h) = d.dimensions().map_err(err)?;
check_size(F, w as u64, h as u64, &DecodeOptions::default())?;
let ct = d.colortype().map_err(err)?;
let compression = d.get_tag_u32(Tag::Compression).ok();
let planar = d.get_tag_u32(Tag::PlanarConfiguration).ok() == Some(2);
layout(ct, compression, planar)?;
// what this build of the `tiff` crate decompresses: none, CCITT G4, LZW, JPEG, Deflate, PackBits
if !matches!(compression.unwrap_or(1), 1 | 4 | 5 | 7 | 8 | 32773 | 32946) {
return Err(Error::Unsupported(F, "compression method"));
}
let chunks = |offsets: Tag, counts: Tag, d: &mut Decoder<std::io::Cursor<&[u8]>>| {
Some((d.get_tag_u64_vec(offsets).ok()?, d.get_tag_u64_vec(counts).ok()?))
};
let (offsets, counts) = chunks(Tag::StripOffsets, Tag::StripByteCounts, &mut d)
.or_else(|| chunks(Tag::TileOffsets, Tag::TileByteCounts, &mut d))
.ok_or_else(|| err("no image data"))?;
// Only where each chunk starts: some writers overstate StripByteCounts, and the `tiff` crate
// decodes such files (it stops once it has the pixels), so a chunk's end proves nothing.
let len = bytes.len() as u64;
if offsets.is_empty() || offsets.iter().zip(&counts).any(|(&o, &c)| c > 0 && o >= len) {
return Err(err("truncated: image data past the end of the file"));
}
let orientation = d.get_tag_u32(Tag::Orientation).ok().map(|v| v as u16).filter(|o| (1..=8).contains(o)).unwrap_or(1);
Ok((w, h, orientation))
}
/// A BYTE/UNDEFINED tag as bytes (the decoder may surface unknown byte tags as unsigned values).
fn tag_bytes<R: std::io::Read + std::io::Seek>(d: &mut Decoder<R>, tag: Tag) -> Option<Vec<u8>> {
use tiff::decoder::ifd::Value;
+27
View File
@@ -27,3 +27,30 @@ pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let raw = Raw { width: w as usize, height: h as usize, model: Model::Rgb, alpha, premultiplied: false, buf: Buf::U8(buf), bit_depth: 8 };
finish(F, raw, Meta { icc, exif, xmp, ..Default::default() }, (w, h), opts)
}
/// Stored dimensions and EXIF orientation from the RIFF chunks and the bitstream header, without
/// decoding the image. Refused, as a decode would refuse it: no image chunk (`VP8 `, `VP8L` or an
/// animation frame), or one that runs past the end of the file (truncated). The compressed data
/// itself is not checked.
pub(crate) fn header(bytes: &[u8]) -> Result<(u32, u32, u16)> {
let mut p = 12usize;
loop {
let Some(h) = bytes.get(p..p.saturating_add(8)) else { return Err(err("no image data")) };
let size = u32::from_le_bytes([h[4], h[5], h[6], h[7]]) as usize;
let end = p.saturating_add(8).saturating_add(size);
if matches!(&h[0..4], b"VP8 " | b"VP8L" | b"ANMF") {
if end > bytes.len() {
return Err(err("truncated image data"));
}
break;
}
p = end.saturating_add(size & 1);
}
let mut d = image_webp::WebPDecoder::new(std::io::Cursor::new(bytes)).map_err(err)?;
d.set_memory_limit(bytes.len());
let (w, h) = d.dimensions();
check_size(F, w as u64, h as u64, &DecodeOptions::default())?;
let exif = d.exif_metadata().ok().flatten();
let exif = exif.as_deref().map(|e| e.strip_prefix(b"Exif\0\0").unwrap_or(e));
Ok((w, h, exif.map(crate::exif::summarize).unwrap_or_default().orientation.unwrap_or(1)))
}
+2
View File
@@ -30,8 +30,10 @@ fn try_all(bytes: &[u8]) {
if let Some(f) = sniff(bytes) {
let _ = decode_unguarded(bytes, f, &small_opts());
let _ = decode_unguarded(bytes, f, &DecodeOptions { max_size: Some((8, 8)), max_pixels: 1 << 22 });
let _ = read_header_unguarded(bytes, f);
}
let _ = decode(bytes, small_opts());
let _ = read_header(bytes);
let _ = decode_thumbnail_with(bytes, &ThumbnailOptions { max_pixels: small_opts().max_pixels, ..ThumbnailOptions::new(16) });
let _ = lightcraft_codecs::icc::parse(bytes);
}
+366
View File
@@ -0,0 +1,366 @@
//! `read_header` (import probes) agrees with a decode on dimensions and orientation, refuses what a
//! decode refuses (truncated files included), and returns errors — never panics — on hostile input.
use lightcraft_codecs::exif::minimal_exif;
use lightcraft_codecs::*;
use lightcraft_raster::Rgba8;
fn pixels(w: usize, h: usize) -> Rgba8 {
Rgba8::from_fn(w, h, |x, y| [(x * 37 % 256) as u8, (y * 53 % 256) as u8, ((x ^ y) * 9 % 256) as u8, (128 + x % 100) as u8])
}
fn meta(exif: &Option<Vec<u8>>) -> EncodeMeta<'_> {
EncodeMeta { exif: exif.as_deref(), xmp: Some("<x/>"), ..Default::default() }
}
/// A TIFF with an Orientation tag (`encode_tiff` writes none), via the `tiff` crate's encoder.
fn tiff_oriented(img: &Rgba8, orientation: u16) -> Vec<u8> {
use tiff::encoder::{TiffEncoder, colortype};
let rgb: Vec<u8> = img.data.iter().flat_map(|p| [p[0], p[1], p[2]]).collect();
let mut cur = std::io::Cursor::new(Vec::new());
let mut enc = TiffEncoder::new(&mut cur).unwrap();
let mut im = enc.new_image::<colortype::RGB8>(img.width as u32, img.height as u32).unwrap();
im.encoder().write_tag(tiff::tags::Tag::Orientation, orientation).unwrap();
im.write_data(&rgb).unwrap();
cur.into_inner()
}
/// A minimal 8-bit RGB PSD (raw or PackBits-literal merged data) with an EXIF resource.
fn psd(w: usize, h: usize, orientation: Option<u16>, rle: bool) -> Vec<u8> {
let mut b = Vec::new();
b.extend_from_slice(b"8BPS");
b.extend_from_slice(&1u16.to_be_bytes());
b.extend_from_slice(&[0; 6]);
b.extend_from_slice(&3u16.to_be_bytes());
b.extend_from_slice(&(h as u32).to_be_bytes());
b.extend_from_slice(&(w as u32).to_be_bytes());
b.extend_from_slice(&8u16.to_be_bytes());
b.extend_from_slice(&3u16.to_be_bytes());
b.extend_from_slice(&0u32.to_be_bytes()); // colour mode data
let mut res = Vec::new();
if let Some(o) = orientation {
let exif = minimal_exif(o);
res.extend_from_slice(b"8BIM");
res.extend_from_slice(&1058u16.to_be_bytes());
res.extend_from_slice(&[0, 0]); // empty pascal name, padded
res.extend_from_slice(&(exif.len() as u32).to_be_bytes());
res.extend_from_slice(&exif);
if exif.len() % 2 == 1 {
res.push(0);
}
}
b.extend_from_slice(&(res.len() as u32).to_be_bytes());
b.extend_from_slice(&res);
b.extend_from_slice(&0u32.to_be_bytes()); // layer and mask info
b.extend_from_slice(&(rle as u16).to_be_bytes());
let plane = |c: usize| (0..w * h).map(move |i| ((i * (c + 3) * 11) % 256) as u8);
if rle {
// every row as literal runs of at most 128 bytes (PackBits' longest)
let row_len = w + w.div_ceil(128);
for _ in 0..3 * h {
b.extend_from_slice(&(row_len as u16).to_be_bytes());
}
for c in 0..3 {
let p: Vec<u8> = plane(c).collect();
for row in p.chunks(w) {
for run in row.chunks(128) {
b.push(run.len() as u8 - 1);
b.extend_from_slice(run);
}
}
}
} else {
for c in 0..3 {
b.extend(plane(c));
}
}
b
}
/// Generated files of every format with a header reader, plus a fallback one (BMP via `image`).
fn fixtures(w: usize, h: usize) -> Vec<(String, Vec<u8>)> {
let img = pixels(w, h);
let e = EncodeImage::rgba8(&img);
let rgb: Vec<u8> = img.data.iter().flat_map(|p| [p[0], p[1], p[2]]).collect();
let rgb_e = EncodeImage::new(w as u32, h as u32, 3, Samples::U8(&rgb));
let u16s: Vec<u16> = (0..w * h * 3).map(|i| (i * 331) as u16).collect();
let mut out = Vec::new();
for o in [None, Some(1), Some(6), Some(8)] {
let exif = o.map(minimal_exif);
let tag = format!("o{}", o.unwrap_or(0));
out.push((format!("jpeg 420 {tag}"), encode_jpeg(&rgb_e, 85, ChromaSubsampling::S420, &meta(&exif)).unwrap()));
out.push((format!("jpeg 444 {tag}"), encode_jpeg(&rgb_e, 85, ChromaSubsampling::S444, &meta(&exif)).unwrap()));
out.push((format!("png rgba {tag}"), encode_png(&e, &meta(&exif)).unwrap()));
out.push((format!("png 16-bit {tag}"), encode_png(&EncodeImage::new(w as u32, h as u32, 3, Samples::U16(&u16s)), &meta(&exif)).unwrap()));
out.push((format!("webp {tag}"), encode_webp_lossless(&e, &meta(&exif)).unwrap()));
out.push((format!("psd {tag}"), psd(w, h, o, false)));
out.push((format!("psd rle {tag}"), psd(w, h, o, true)));
out.push((format!("tiff {tag}"), tiff_oriented(&img, o.unwrap_or(1))));
}
out.push(("tiff lzw".into(), encode_tiff(&e, TiffCompression::Lzw, &EncodeMeta::default()).unwrap()));
out.push(("tiff packbits".into(), encode_tiff(&e, TiffCompression::PackBits, &EncodeMeta::default()).unwrap()));
let mut bmp = Vec::new();
image::codecs::bmp::BmpEncoder::new(&mut bmp).encode(&rgb, w as u32, h as u32, image::ExtendedColorType::Rgb8).unwrap();
out.push(("bmp".into(), bmp));
out
}
/// What the import probe learnt before it read headers: a small decode.
fn decoded(bytes: &[u8]) -> Result<Header> {
let d = decode(bytes, DecodeOptions::fit(64, 64))?;
Ok(Header { format: d.format, width: d.source_width, height: d.source_height, orientation: d.orientation })
}
#[test]
fn header_matches_decode() {
for (w, h) in [(160, 96), (96, 160), (7, 5)] {
for (name, bytes) in fixtures(w, h) {
let header = read_header(&bytes).unwrap_or_else(|e| panic!("{name} {w}×{h}: {e}"));
assert_eq!(header, decoded(&bytes).unwrap(), "{name} {w}×{h}");
assert_eq!((header.width, header.height), (w as u32, h as u32), "{name}");
}
}
let o = |name: &str| fixtures(16, 8).into_iter().find(|f| f.0 == name).map(|f| read_header(&f.1).unwrap().orientation).unwrap();
assert_eq!((o("jpeg 420 o6"), o("png rgba o8"), o("webp o6"), o("psd o8"), o("tiff o6"), o("jpeg 444 o0")), (6, 8, 6, 8, 6, 1));
}
/// Whenever the decode the probe used to run accepts a file (here: every cut of every fixture),
/// the header agrees with it: reading headers never refuses a file that decodes. The reverse
/// is checked for truncation, which the header readers detect.
#[test]
fn truncated_files_agree_with_decode() {
// 160 px wide: large JPEGs take the DCT-scaled (strict) decode path the probe took for photos
for (name, bytes) in fixtures(160, 96) {
let n = bytes.len();
let mut cuts: Vec<usize> = (0..100).map(|i| n * i / 100).collect();
cuts.extend(n.saturating_sub(64)..n);
for cut in cuts {
let part = &bytes[..cut];
if let Ok(d) = decoded(part) {
assert_eq!(read_header(part).ok(), Some(d), "{name} cut at {cut}/{n}");
}
}
for cut in [n / 4, n / 2, n * 9 / 10] {
assert!(read_header(&bytes[..cut]).is_err(), "{name}: cut at {cut}/{n} accepted");
}
}
}
#[test]
fn truncation_details() {
let f = fixtures(160, 96);
let get = |name: &str| f.iter().find(|x| x.0 == name).map(|x| x.1.clone()).unwrap();
let jpeg = get("jpeg 420 o6");
assert!(read_header(&jpeg[..jpeg.len() - 2]).is_err(), "a JPEG without its EOI is truncated");
let mut trailing = jpeg.clone();
trailing.extend_from_slice(&[0; 100]);
assert_eq!(read_header(&trailing).unwrap().orientation, 6, "bytes after the EOI are fine");
let png = get("png rgba o6");
assert!(read_header(&png[..png.len() - 2]).is_ok(), "a damaged IEND is ignored, as by a decode");
assert!(read_header(&png[..png.len() - 12]).is_err(), "nothing after the image data: truncated");
}
/// eXIf after the image data counts, as it does for a decode (which reads the whole file).
#[test]
fn png_exif_after_idat() {
let img = pixels(9, 4);
let plain = encode_png(&EncodeImage::rgba8(&img), &EncodeMeta::default()).unwrap();
let exif = minimal_exif(6);
let mut chunk = (exif.len() as u32).to_be_bytes().to_vec();
chunk.extend_from_slice(b"eXIf");
chunk.extend_from_slice(&exif);
chunk.extend_from_slice(&[0; 4]); // CRC: not checked by either reader
let iend = plain.len() - 12;
let mut b = plain[..iend].to_vec();
b.extend_from_slice(&chunk);
b.extend_from_slice(&plain[iend..]);
assert_eq!(read_header(&b).unwrap().orientation, 6);
}
fn jpeg_segments(sof: u8, w: u16, h: u16, comps: u8) -> Vec<u8> {
let mut b = vec![0xFF, 0xD8, 0xFF, sof, 0, 8 + 3 * comps as u16 as u8, 8];
b.extend_from_slice(&h.to_be_bytes());
b.extend_from_slice(&w.to_be_bytes());
b.push(comps);
for c in 0..comps {
b.extend_from_slice(&[c + 1, 0x11, 0]);
}
b.extend_from_slice(&[0xFF, 0xDA, 0, 8, 1, 1, 0, 0, 63, 0]);
b.extend_from_slice(&[0x12, 0xFF, 0x00, 0x34, 0xFF, 0xD0, 0x56]);
b.extend_from_slice(&[0xFF, 0xD9]);
b
}
fn png_ihdr(w: u32, h: u32, depth: u8, color: u8) -> Vec<u8> {
let mut b = b"\x89PNG\r\n\x1a\n\0\0\0\x0dIHDR".to_vec();
b.extend_from_slice(&w.to_be_bytes());
b.extend_from_slice(&h.to_be_bytes());
b.extend_from_slice(&[depth, color, 0, 0, 0, 0, 0, 0, 0]);
b
}
fn png_chunk(b: &mut Vec<u8>, kind: &[u8; 4], len: u32, data: &[u8]) {
b.extend_from_slice(&len.to_be_bytes());
b.extend_from_slice(kind);
b.extend_from_slice(data);
b.extend_from_slice(&[0; 4]);
}
#[test]
fn hostile_headers_are_errors() {
// a well-formed synthetic JPEG header is read
assert_eq!(read_header(&jpeg_segments(0xC0, 33, 21, 3)).unwrap(), Header { format: Format::Jpeg, width: 33, height: 21, orientation: 1 });
let jpeg_bad = [
jpeg_segments(0xC0, 0, 21, 3), // zero width
jpeg_segments(0xC0, 33, 0, 3), // zero height (DNL)
jpeg_segments(0xC0, 65535, 65535, 3), // 4.3 Gpx
jpeg_segments(0xC9, 33, 21, 3), // arithmetic coding
jpeg_segments(0xC5, 33, 21, 3), // hierarchical
jpeg_segments(0xC0, 33, 21, 2), // two components
jpeg_segments(0xC0, 33, 21, 3)[..30].to_vec(),
vec![0xFF, 0xD8, 0xFF, 0xE1, 0, 0], // segment length 0
vec![0xFF, 0xD8, 0xFF, 0xE1, 0xFF, 0xFF], // segment past the end
vec![0xFF, 0xD8, 0xFF, 0xD9],
];
for (i, b) in jpeg_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "JPEG case {i}");
}
let mut no_sos = jpeg_segments(0xC0, 33, 21, 3);
no_sos.truncate(2 + 2 + 17);
no_sos.extend_from_slice(&[0xFF, 0xD9]);
assert!(read_header(&no_sos).is_err(), "JPEG without a scan");
let mut ok = png_ihdr(5, 3, 8, 2);
png_chunk(&mut ok, b"IDAT", 1, &[0]);
png_chunk(&mut ok, b"IEND", 0, &[]);
assert_eq!(read_header(&ok).unwrap(), Header { format: Format::Png, width: 5, height: 3, orientation: 1 });
let png_case = |ihdr: Vec<u8>, chunks: &[(&[u8; 4], u32, &[u8])]| {
let mut b = ihdr;
for (k, l, d) in chunks {
png_chunk(&mut b, k, *l, d);
}
b
};
let idat: (&[u8; 4], u32, &[u8]) = (b"IDAT", 1, &[0]);
let iend: (&[u8; 4], u32, &[u8]) = (b"IEND", 0, &[]);
let png_bad = [
png_case(png_ihdr(0, 3, 8, 2), &[idat, iend]),
png_case(png_ihdr(5, 0, 8, 2), &[idat, iend]),
png_case(png_ihdr(0x8000_0000, 1, 8, 2), &[idat, iend]),
png_case(png_ihdr(65536, 65536, 8, 2), &[idat, iend]),
png_case(png_ihdr(5, 3, 4, 2), &[idat, iend]), // RGB at 4 bits
png_case(png_ihdr(5, 3, 8, 5), &[idat, iend]), // no colour type 5
png_case(png_ihdr(5, 3, 8, 3), &[idat, iend]), // indexed without PLTE
png_case(png_ihdr(5, 3, 8, 2), &[iend]), // no IDAT
png_case(png_ihdr(5, 3, 8, 2), &[idat]), // nothing after the image data
png_case(png_ihdr(5, 3, 8, 2), &[(b"IDAT", 0xFFFF_FFFF, &[0]), iend]),
png_case(png_ihdr(5, 3, 8, 2), &[(b"tEXt", 0x7FFF_FFFF, &[0]), idat, iend]),
png_ihdr(5, 3, 8, 2)[..20].to_vec(),
b"\x89PNG\r\n\x1a\n\0\0\0\x0dIHDX\0\0\0\x05\0\0\0\x03\x08\x02\0\0\0".to_vec(),
];
for (i, b) in png_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "PNG case {i}");
}
let tiff = encode_tiff(&EncodeImage::rgba8(&pixels(6, 4)), TiffCompression::None, &EncodeMeta::default()).unwrap();
let ifd = u32::from_le_bytes([tiff[4], tiff[5], tiff[6], tiff[7]]) as usize;
let mut tiff_bad = vec![b"II*\0\xff\xff\xff\x7f".to_vec(), b"II*\0\x08\0\0\0".to_vec(), b"II*\0\x04\0\0\0".to_vec()];
// cut short after the IFD
assert!(ifd < tiff.len() / 2, "the IFD comes before the image data");
tiff_bad.push(tiff[..tiff.len() - 10].to_vec());
for (i, b) in tiff_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "TIFF case {i}");
}
// a strip offset pointing past the end of the file
let t = |b: &[u8], p: usize| u16::from_le_bytes([b[p], b[p + 1]]);
let n = t(&tiff, ifd) as usize;
let mut far = tiff.clone();
for e in 0..n {
let p = ifd + 2 + e * 12;
if t(&far, p) == 273 && t(&far, p + 2) == 4 && u32::from_le_bytes([far[p + 4], far[p + 5], far[p + 6], far[p + 7]]) == 1 {
far[p + 8..p + 12].copy_from_slice(&0x7FFF_0000u32.to_le_bytes());
}
}
assert_ne!(far, tiff, "the strip offset was found");
assert!(read_header(&far).is_err(), "strip past the end");
let webp_bad = [
b"RIFF\0\0\0\0WEBP".to_vec(),
b"RIFF\0\0\0\0WEBPVP8L\xff\xff\xff\xff\x2f".to_vec(),
b"RIFF\0\0\0\0WEBPXXXX\xff\xff\xff\xff".to_vec(),
b"RIFF\0\0\0\0WEBPVP8 \x04\0\0\0\0\0\0\0".to_vec(),
];
for (i, b) in webp_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "WebP case {i}");
}
let good = psd(4, 3, Some(6), true);
assert_eq!(read_header(&good).unwrap().orientation, 6);
let set = |pos: usize, v: &[u8]| {
let mut b = good.clone();
b[pos..pos + v.len()].copy_from_slice(v);
b
};
let res_len = u32::from_be_bytes([good[30], good[31], good[32], good[33]]) as usize;
let compression = 34 + res_len + 4;
let mut psd_bad = vec![
set(12, &0u16.to_be_bytes()), // no channels
set(12, &57u16.to_be_bytes()), // too many
set(12, &2u16.to_be_bytes()), // too few for RGB
set(14, &0x7FFF_FFFFu32.to_be_bytes()), // absurd height
set(18, &0u32.to_be_bytes()), // zero width
set(22, &1u16.to_be_bytes()), // 1-bit
set(22, &7u16.to_be_bytes()), // bad depth
set(24, &9u16.to_be_bytes()), // Lab
set(24, &77u16.to_be_bytes()), // unknown mode
set(24, &2u16.to_be_bytes()), // indexed without a palette
set(compression, &2u16.to_be_bytes()), // ZIP
set(26, &0xFFFF_FFFFu32.to_be_bytes()), // colour mode data past the end
set(30, &0xFFFF_FFF0u32.to_be_bytes()), // resources past the end
];
// a 1 × 2^30 RLE image declared in a tiny file: refused without reading 2^30 row counts
let mut tall = good.clone();
tall[14..18].copy_from_slice(&(1u32 << 30).to_be_bytes());
tall[18..22].copy_from_slice(&1u32.to_be_bytes());
psd_bad.push(tall);
psd_bad.push(good[..good.len() - 1].to_vec());
for (i, b) in psd_bad.iter().enumerate() {
assert!(read_header(b).is_err(), "PSD case {i}");
}
for b in [&b""[..], b"\xFF\xD8\xFF", b"\x89PNG\r\n\x1a\n", b"II*\0", b"MM\0*", b"8BPS\0\x01", b"GIF89a", b"not an image at all"] {
assert!(read_header(b).is_err(), "{b:?}");
}
}
/// The same with a few random bytes changed: whenever the decode accepts the damaged file, the
/// header agrees with it (deterministic seed).
#[test]
fn mutated_files_agree_with_decode() {
let mut s: u64 = 0x1234_5678_9abc_def1;
let mut rnd = || {
s ^= s << 13;
s ^= s >> 7;
s ^= s << 17;
s
};
let (mut decodable, mut disagreements) = (0, Vec::new());
for (name, bytes) in fixtures(160, 96) {
for _ in 0..300 {
let mut b = bytes.clone();
for _ in 0..1 + rnd() % 3 {
let i = rnd() as usize % b.len();
b[i] = rnd() as u8;
}
if let Ok(d) = decoded(&b) {
decodable += 1;
let h = read_header(&b);
if h.as_ref().ok() != Some(&d) {
disagreements.push(format!("{name}: decode {d:?}, header {h:?}"));
}
}
}
}
assert!(decodable > 1000, "{decodable}");
assert!(disagreements.is_empty(), "{disagreements:#?}");
}
+26
View File
@@ -68,6 +68,32 @@ impl DevelopSettings {
!self.disabled_sections.iter().any(|s| s == section)
}
/// The settings as rendered: an Edit-panel section whose eye is off contributes nothing (issue
/// #316). Light (with its tone curve), Color (white balance, presence, the mixers, colour grading
/// and point colour) and Detail are reset to their defaults here; Effects, Optics, Geometry and
/// Calibration the pipeline switches off where it applies them. Borrowed when every section is on.
pub fn effective(&self) -> std::borrow::Cow<'_, DevelopSettings> {
let off = |section: &str| !self.section_enabled(section);
if !(off("light") || off("color") || off("detail")) {
return std::borrow::Cow::Borrowed(self);
}
let mut d = self.clone();
if off("light") {
d.reset_section(Section::Light);
d.reset_section(Section::Curve);
}
if off("color") {
for section in [Section::Color, Section::Mixer, Section::BwMix, Section::Grading, Section::PointColor] {
d.reset_section(section);
}
d.point_colors.clear();
}
if off("detail") {
d.reset_section(Section::Detail);
}
std::borrow::Cow::Owned(d)
}
pub fn set_section_enabled(&mut self, section: &str, on: bool) {
self.disabled_sections.retain(|s| s != section);
if !on {
+7
View File
@@ -22,6 +22,7 @@ lightcraft-preview = { workspace = true }
lightcraft-codecs = { workspace = true }
lightcraft-raw = { workspace = true }
lightcraft-meta = { workspace = true }
lightcraft-faces = { workspace = true }
miniz_oxide = { workspace = true }
lightcraft-merge = { workspace = true }
rayon = { workspace = true }
@@ -32,11 +33,17 @@ log = { workspace = true }
ab_glyph = { workspace = true }
# Apply the font’s vertical substitutions and origins to Japanese watermarks.
harfrust = { version = "0.12", default-features = false, features = ["std"] }
unicode-segmentation = "1.13"
lightcraft-segment = { workspace = true, optional = true }
# model downloads run natively only (the web build fetches no models)
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
lightcraft-fetch = { workspace = true }
[features]
# AI masks with SAM 3 (Object / Describe). Off for the web and CLI builds.
sam = ["dep:lightcraft-segment"]
[dev-dependencies]
libc = { workspace = true }
lightcraft-tiff = { workspace = true }
+304 -15
View File
@@ -1,5 +1,5 @@
//! Estimate the starting look of a raw without a camera colour matrix (Sony ARW, Nikon NEF, Panasonic
//! RW2) from its own JPEG. Colour and luminance are fitted separately; the JPEG supplies correspondences only,
//! RW2, Fujifilm RAF, Canon CR3) from its own JPEG. Colour and luminance are fitted separately; the JPEG supplies correspondences only,
//! never output pixels or a replacement for RAW editing.
//! A global matrix can't follow the camera's hue-dependent rendering (the best matrix rendered a
//! lime shirt olive that the camera kept lime): a hue/saturation table fitted to the residuals
@@ -31,7 +31,10 @@ pub(crate) const PROFILE_PROXY: usize = 192;
/// as-shot look (`docs/camera-preview-colour.md`). The catalog's `Photo::relative_wb` matches the
/// same formats by file extension.
pub(crate) fn file_local_look(format: RawFormat) -> bool {
matches!(format, RawFormat::Arw | RawFormat::Nef | RawFormat::Nrw | RawFormat::Rw2)
matches!(
format,
RawFormat::Arw | RawFormat::Nef | RawFormat::Nrw | RawFormat::Rw2 | RawFormat::Raf | RawFormat::Cr3 | RawFormat::Cr2 | RawFormat::Pef
)
}
pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransform) -> Option<CameraLook> {
@@ -40,13 +43,13 @@ pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransf
}
let (sensor, reference) = proxies(raw, bytes, transform, PROXY)?;
// A camera profile pooled from many photos knows colours this photo shows too little of;
// only the tone and chroma curves are fitted per photo (DRO and picture styles vary).
// try its colour first and fit tone/chroma per photo (DRO and picture styles vary).
let profile = raw.metadata.model.as_deref().and_then(crate::camera_profiles::get);
let colour = profile.as_ref().and_then(|p| Some((p.matrix().mul(&transform.matrix.inverse()?), p.hue_sat.clone())));
let look = fit_pairs_with(&sensor, &reference, colour)?;
if lightcraft_pipeline::profiling() {
eprintln!(
"[profile] {:?} camera look: {:?}, {:?}, hue/sat table {}, camera profile {}",
"[profile] {:?} camera look: {:?}, {:?}, hue/sat table {}, camera profile available {}",
raw.format,
look.matrix.0,
look.tone,
@@ -83,14 +86,161 @@ fn proxies(raw: &RawImage, bytes: &[u8], transform: &CameraTransform, size: usiz
}
// Fixed, bounded proxy: the selected look cannot depend on thumbnail/export resolution.
let k = (crop.width.max(crop.height).div_ceil(edge as usize).max(2)).div_ceil(2) * 2;
let sensor = sensor_proxy(raw, k, edge as usize)?;
let mut sensor = sensor_proxy(raw, k, edge as usize)?;
let gain = 2f32.powf(transform.baseline_exposure as f32);
let to_working = |p: [f32; 3]| transform.matrix.apply_f32(std::array::from_fn(|i| p[i] * transform.wb[i] * gain));
if raw.format == RawFormat::Cr3 {
// Orient the camera JPEG and the sensor identically before collecting pixel correspondences.
sensor = sensor.into_oriented(raw.orientation);
reference = reference.into_oriented(raw.orientation);
sensor.map_in_place(to_working);
// Canon's JPEG can be cropped differently from the sensor. Estimate only that common
// framing from edge directions, independently of the subsequent colour fit.
sensor = align_cr3_framing(sensor, &reference);
}
let mut sensor = fit(&sensor, size, size, Filter::Box);
let reference = fit(&reference, sensor.width, sensor.height, Filter::Box);
let gain = 2f32.powf(transform.baseline_exposure as f32);
sensor.map_in_place(|p| transform.matrix.apply_f32(std::array::from_fn(|i| p[i] * transform.wb[i] * gain)));
if raw.format != RawFormat::Cr3 {
sensor.map_in_place(to_working);
}
Some((sensor, reference))
}
#[derive(Clone, Copy, Debug)]
struct Cr3Framing {
scale: f32,
/// Translation as a fraction of the oriented frame's width and height.
offset: [f32; 2],
}
impl Cr3Framing {
const IDENTITY: Self = Self { scale: 1.0, offset: [0.0; 2] };
fn source(self, x: f32, y: f32, w: usize, h: usize) -> (f32, f32) {
let (w, h) = (w as f32, h as f32);
((x - w * 0.5) * self.scale + w * (0.5 + self.offset[0]), (y - h * 0.5) * self.scale + h * (0.5 + self.offset[1]))
}
}
/// Register the JPEG's remaining framing on a fixed proxy. Edge directions survive different
/// white balance and monotone camera tone; no target RGB values train this three-parameter fit.
/// A separate third of the edges must confirm a substantial improvement and a close match.
/// A weak, flat or unrelated preview leaves the sensor framing unchanged.
fn estimate_cr3_framing(sensor: &Rgb32f, reference: &Rgb32f) -> Option<Cr3Framing> {
if sensor.width.checked_mul(sensor.height)? != sensor.data.len() || reference.width.checked_mul(reference.height)? != reference.data.len() {
return None;
}
let sensor = fit(sensor, PROXY, PROXY, Filter::Box);
let (w, h) = (sensor.width, sensor.height);
if w < 16 || h < 16 {
return None;
}
let reference = fit(reference, w, h, Filter::Box);
if (reference.width, reference.height) != (w, h) {
return None;
}
let mut gradients = Rgb32f::new(w, h);
let mut edges = Vec::new();
for y in 1..h - 1 {
for x in 1..w - 1 {
let derivative = |image: &Rgb32f| {
[
luminance_2020(image.data[y * w + x + 1]) - luminance_2020(image.data[y * w + x - 1]),
luminance_2020(image.data[(y + 1) * w + x]) - luminance_2020(image.data[(y - 1) * w + x]),
]
};
let d = derivative(&sensor);
gradients.data[y * w + x] = [d[0], d[1], 0.0];
let d = derivative(&reference);
let magnitude = d[0].hypot(d[1]);
let value = luminance_2020(reference.data[y * w + x]);
if (0.02..0.9).contains(&value) && magnitude.is_finite() && magnitude > 0.03 {
edges.push((x as f32 + 0.5, y as f32 + 0.5, d, magnitude, (x * 17 + y * 11) % 3 == 0));
}
}
}
if edges.len() < 192 {
return None;
}
let score = |framing: Cr3Framing, held_out: bool| -> Option<f64> {
let (mut agreement, mut weight, mut valid, mut total) = (0.0, 0.0, 0usize, 0usize);
for &(x, y, target, magnitude, holdout) in &edges {
if holdout != held_out {
continue;
}
total += 1;
let (sx, sy) = framing.source(x, y, w, h);
if sx < 1.5 || sy < 1.5 || sx > w as f32 - 1.5 || sy > h as f32 - 1.5 {
continue;
}
let sample = gradients.sample_bilinear(sx, sy);
let length = sample[0].hypot(sample[1]);
if !length.is_finite() || length <= 0.01 {
continue;
}
let k = f64::from(magnitude.min(0.25));
agreement += k * f64::from((sample[0] * target[0] + sample[1] * target[1]) / (length * magnitude));
weight += k;
valid += 1;
}
(valid >= 64 && valid * 5 >= total * 4 && weight > 0.0).then_some(agreement / weight)
};
let before = score(Cr3Framing::IDENTITY, false)?;
let mut best = (before, Cr3Framing::IDENTITY);
// At most 10% zoom and two proxy pixels of translation; a coarse search followed by one
// local refinement prevents unbounded optimisation on the photograph's contents.
for s in -10..=10 {
for tx in -4..=4 {
for ty in -4..=4 {
let framing = Cr3Framing { scale: 1.0 + s as f32 * 0.01, offset: [tx as f32 * 0.5 / w as f32, ty as f32 * 0.5 / h as f32] };
if let Some(value) = score(framing, false)
&& value > best.0
{
best = (value, framing);
}
}
}
}
let coarse = best.1;
for s in -5..=5 {
for tx in -4..=4 {
for ty in -4..=4 {
let framing = Cr3Framing {
scale: coarse.scale + s as f32 * 0.002,
offset: [coarse.offset[0] + tx as f32 * 0.125 / w as f32, coarse.offset[1] + ty as f32 * 0.125 / h as f32],
};
if !(0.9..=1.1).contains(&framing.scale) || framing.offset[0].abs() > 2.0 / w as f32 || framing.offset[1].abs() > 2.0 / h as f32 {
continue;
}
if let Some(value) = score(framing, false)
&& value > best.0
{
best = (value, framing);
}
}
}
}
let (held_before, held_after) = (score(Cr3Framing::IDENTITY, true)?, score(best.1, true)?);
if best.0 < 0.9 || best.0 < before + 0.04 || held_after < 0.9 || held_after < held_before + 0.04 {
return None;
}
if lightcraft_pipeline::profiling() {
eprintln!("[profile] CR3 JPEG framing {:?}, held-out edge agreement {held_before:.4} -> {held_after:.4}", best.1);
}
Some(best.1)
}
fn align_cr3_framing(sensor: Rgb32f, reference: &Rgb32f) -> Rgb32f {
let Some(framing) = estimate_cr3_framing(&sensor, reference) else { return sensor };
let (w, h) = (sensor.width, sensor.height);
let mut aligned = Rgb32f::new(w, h);
for (i, pixel) in aligned.data.iter_mut().enumerate() {
let (x, y) = framing.source((i % w) as f32 + 0.5, (i / w) as f32 + 0.5, w, h);
*pixel = if x >= 0.5 && y >= 0.5 && x <= w as f32 - 0.5 && y <= h as f32 - 0.5 { sensor.sample_bilinear(x, y) } else { [f32::NAN; 3] };
}
aligned
}
/// Colour training pairs of one raw for a camera profile: white-balanced camera RGB (with the
/// baseline exposure) → its camera JPEG (linear Rec.2020). `None` for formats with their own
/// colour matrices, other files or unusable previews.
@@ -101,7 +251,7 @@ pub(crate) fn profile_pairs(raw: &RawImage, bytes: &[u8]) -> Option<Vec<([f64; 3
let transform = lightcraft_raw::color::camera_transform(raw, lightcraft_raw::color::as_shot_white_xy(raw));
let (sensor, reference) = proxies(raw, bytes, &transform, PROFILE_PROXY)?;
let to_camera = transform.matrix.inverse()?;
let (pairs, _) = collect_pairs(&sensor, &reference, None)?;
let (pairs, _) = collect_pairs(&sensor, &reference, 0.05, None)?;
Some(pairs.into_iter().map(|(x, y)| (to_camera.apply(x), y)).collect())
}
@@ -173,7 +323,7 @@ fn local_contrast(image: &Rgb32f, i: usize) -> f32 {
/// are left out: there a small geometric mismatch between the raw and its JPEG pairs unrelated
/// colours (see [`EDGE_CONTRAST`]).
type Pairs = Vec<([f64; 3], [f64; 3])>;
fn collect_pairs(sensor: &Rgb32f, reference: &Rgb32f, edge_limit: Option<f32>) -> Option<(Pairs, Pairs)> {
fn collect_pairs(sensor: &Rgb32f, reference: &Rgb32f, min_chroma: f32, edge_limit: Option<f32>) -> Option<(Pairs, Pairs)> {
if (sensor.width, sensor.height) != (reference.width, reference.height) || sensor.data.len() != reference.data.len() {
return None;
}
@@ -197,7 +347,7 @@ fn collect_pairs(sensor: &Rgb32f, reference: &Rgb32f, edge_limit: Option<f32>) -
}
let min = output.iter().copied().fold(f32::INFINITY, f32::min);
let max = output.iter().copied().fold(0.0, f32::max);
colour += usize::from(max - min > 0.05);
colour += usize::from(max - min > min_chroma);
pairs.push((input.map(f64::from), output.map(f64::from)));
}
(pairs.len() >= 256 && colour >= pairs.len() / 20).then_some((pairs, bright))
@@ -252,11 +402,22 @@ const EDGE_CONTRAST: f32 = 3.0;
/// it is tried once more on the pixels away from edges (same gates): there colour pairs stay
/// valid when the camera JPEG's geometry differs slightly from the raw's.
fn fit_pairs_with(sensor: &Rgb32f, reference: &Rgb32f, colour: Option<(Mat3, Option<HsvTable>)>) -> Option<CameraLook> {
let has_profile = colour.is_some();
// A different picture style can make a camera profile fail the gates. Preserve the
// photo's own colour fit before resorting to the neutral fallback.
fit_candidate(sensor, reference, colour).or_else(|| has_profile.then(|| fit_candidate(sensor, reference, None)).flatten())
}
/// One candidate colour model, fitted on all pixels and, when that fails the gates, once more on
/// the pixels away from edges (issue #232).
fn fit_candidate(sensor: &Rgb32f, reference: &Rgb32f, colour: Option<(Mat3, Option<HsvTable>)>) -> Option<CameraLook> {
fit_pairs_on(sensor, reference, colour.clone(), None).or_else(|| fit_pairs_on(sensor, reference, colour, Some(EDGE_CONTRAST)))
}
fn fit_pairs_on(sensor: &Rgb32f, reference: &Rgb32f, colour: Option<(Mat3, Option<HsvTable>)>, edge_limit: Option<f32>) -> Option<CameraLook> {
let (pairs, bright) = collect_pairs(sensor, reference, edge_limit)?;
// A known colour model needs enough signal for tone fitting, not a scene rich enough to
// learn a new colour matrix. Still reject monochrome references.
let (pairs, bright) = collect_pairs(sensor, reference, if colour.is_some() { 0.005 } else { 0.05 }, edge_limit)?;
let candidates = match colour {
Some(given) => vec![given],
None => {
@@ -778,7 +939,7 @@ mod tests {
let mut pool = Vec::new();
for seed in 0..4 {
let (sensor, reference) = photo(seed, 2.0 + seed as f32 * 0.5);
pool.extend(collect_pairs(&sensor, &reference, None).unwrap().0);
pool.extend(collect_pairs(&sensor, &reference, 0.05, None).unwrap().0);
}
let (matrix, _) = fit_profile(&pool).unwrap();
// the camera's colours (luminance-normalised: the tone curve sets brightness)
@@ -805,6 +966,48 @@ mod tests {
assert!(error.sqrt() < 0.02, "this photo's own tone is followed: RMS {}", error.sqrt());
}
#[test]
fn profile_fits_tone_without_learning_colour_from_a_dull_scene() {
let matrix = Mat3([[2.0, -0.02, 0.0], [0.0, 2.0, 0.0], [0.0, -0.03, 2.0]]);
let mut sensor = Rgb32f::new(64, 48);
let mut reference = sensor.clone();
for (i, (src, dst)) in sensor.data.iter_mut().zip(&mut reference.data).enumerate() {
*src = [0.02 + (i % 997) as f32 * 0.00045; 3];
let p = matrix.apply_f32(*src);
let y = luminance_2020(p);
*dst = p.map(|v| v * (1.0 - (-2.5 * y).exp()) / y);
}
assert!(fit_pairs(&sensor, &reference).is_none(), "not enough colour to learn a matrix");
let look = fit_pairs_with(&sensor, &reference, Some((matrix, None))).unwrap();
assert_eq!(look.matrix, matrix);
reference.map_in_place(|p| [luminance_2020(p); 3]);
assert!(fit_pairs_with(&sensor, &reference, Some((matrix, None))).is_none(), "monochrome JPEGs remain rejected");
}
#[test]
fn rejected_profile_preserves_a_usable_per_photo_fit() {
let matrix = Mat3([[1.8, -0.4, -0.1], [-0.2, 1.5, -0.1], [-0.05, -0.3, 1.7]]);
let wrong = Mat3([[0.0, 0.0, 1.0], [0.0, 1.0, 0.0], [1.0, 0.0, 0.0]]);
let mut sensor = Rgb32f::new(64, 64);
let mut reference = sensor.clone();
let mut seed = 0x2545_f491_u32;
for (src, dst) in sensor.data.iter_mut().zip(&mut reference.data) {
*src = std::array::from_fn(|_| {
seed ^= seed << 13;
seed ^= seed >> 17;
seed ^= seed << 5;
0.05 + (seed % 1000) as f32 * 0.0003
});
let p = matrix.apply_f32(*src);
let y = luminance_2020(p);
*dst = p.map(|v| (v * (1.0 - (-2.5 * y).exp()) / y).clamp(0.0, 0.999));
}
let per_photo = fit_pairs(&sensor, &reference).unwrap();
let look = fit_pairs_with(&sensor, &reference, Some((wrong, None))).unwrap();
assert_eq!(look.matrix, per_photo.matrix, "a rejected camera profile must not suppress the per-photo fit");
assert_eq!(look.tone, per_photo.tone);
}
#[test]
fn hue_sat_passes_black_and_non_finite_pixels_through() {
let table = HsvTable { hue_divisions: 4, sat_divisions: 2, val_divisions: 1, data: vec![[10.0, 1.5, 1.0]; 8], srgb_value: false };
@@ -816,9 +1019,73 @@ mod tests {
}
#[test]
fn sony_nikon_and_panasonic_raws_get_a_file_local_look() {
assert!([RawFormat::Arw, RawFormat::Nef, RawFormat::Nrw, RawFormat::Rw2].into_iter().all(file_local_look));
assert!(![RawFormat::Dng, RawFormat::Cr2, RawFormat::Raf].into_iter().any(file_local_look));
fn supported_raws_get_a_file_local_look() {
// every decoded raw without a colour matrix of its own (issue #310: CR2 rendered flat with the fallback)
let local = [RawFormat::Arw, RawFormat::Nef, RawFormat::Nrw, RawFormat::Rw2, RawFormat::Raf, RawFormat::Cr3, RawFormat::Cr2, RawFormat::Pef];
assert!(local.into_iter().all(file_local_look));
// DNG carries its own colour model
assert!(!file_local_look(RawFormat::Dng));
}
#[test]
fn cr3_camera_framing_is_measured_independently_of_tone_and_rejects_unrelated_edges() {
let (w, h) = (192, 128);
let mut sensor = Rgb32f::new(w, h);
for (i, pixel) in sensor.data.iter_mut().enumerate() {
let (x, y) = ((i % w) as f32, (i / w) as f32);
let value = 0.35 + 0.13 * (x * 0.15 + y * 0.22).sin() + 0.1 * (x * 0.33 - y * 0.14).sin() + 0.07 * (x * 0.09 - y * 0.31).cos();
*pixel = [value; 3];
}
let expected = Cr3Framing { scale: 0.96, offset: [0.01, -0.006] };
let mut reference = Rgb32f::new(w, h);
for (i, pixel) in reference.data.iter_mut().enumerate() {
let (x, y) = expected.source((i % w) as f32 + 0.5, (i / w) as f32 + 0.5, w, h);
*pixel = sensor.sample_bilinear(x, y).map(|v| v.powf(0.7));
}
let measured = estimate_cr3_framing(&sensor, &reference).expect("coherent held-out edges locate camera framing despite nonlinear tone");
assert!((measured.scale - expected.scale).abs() < 0.006, "{measured:?}");
assert!((measured.offset[0] - expected.offset[0]).abs() < 0.003, "{measured:?}");
assert!((measured.offset[1] - expected.offset[1]).abs() < 0.003, "{measured:?}");
assert!(estimate_cr3_framing(&sensor, &sensor).is_none(), "already aligned pixels stay unchanged");
reference.data.fill([0.2; 3]);
assert!(estimate_cr3_framing(&sensor, &reference).is_none(), "flat previews cannot establish framing");
for (i, pixel) in reference.data.iter_mut().enumerate() {
let (x, y) = ((i % w) as f32, (i / w) as f32);
*pixel = [0.35 + 0.13 * (x * 0.41 - y * 0.07).sin() + 0.1 * (x * 0.04 + y * 0.39).cos(); 3];
}
assert!(estimate_cr3_framing(&sensor, &reference).is_none(), "unrelated edges cannot change the framing");
}
/// The guarded fit is assessed on real sensor data, never on the JPEG fallback. A corpus
/// download or an unfinished codec cannot make this test claim a successful RAW calibration.
#[test]
fn corpus_cr3_gets_a_framing_aligned_camera_look() {
let path = std::env::var_os("LIGHTCRAFT_CORPUS")
.map(std::path::PathBuf::from)
.unwrap_or_else(|| std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus"))
.join("raw/cr3-canon-r100-raw.cr3");
let Ok(bytes) = std::fs::read(&path) else {
eprintln!("skip: {} absent", path.display());
return;
};
let raw = match lightcraft_raw::decode(&bytes) {
Ok(raw) => raw,
Err(lightcraft_raw::RawError::Unsupported(why)) => {
eprintln!("skip CR3 sensor decoder: {why}");
return;
}
Err(error) => panic!("CR3 corpus failed to decode: {error}"),
};
assert_eq!(raw.format, RawFormat::Cr3);
assert_eq!(raw.info().developed_size(), (6000, 4000));
let transform = lightcraft_raw::color::camera_transform(&raw, lightcraft_raw::color::as_shot_white_xy(&raw));
let look = fit_preview(&raw, &bytes, &transform);
eprintln!("R100 guarded colour fit accepted: {}", look.is_some());
assert!(look.is_some(), "this R100 corpus photo has enough matching colour after camera-framing alignment");
let (_, info) = crate::files::load_bytes(&bytes, 400).unwrap();
assert!(info.raw && info.relative_wb);
assert_eq!((info.as_shot_temp, info.as_shot_tint), (6500.0, 0.0));
assert_eq!(info.camera_tone.is_some(), look.is_some());
}
/// A public D7500 NEF (skipped without the corpus): its look is fitted to its own JPEG and white
@@ -918,6 +1185,28 @@ mod tests {
assert!(fit_pairs(&sensor, &reference).is_none());
}
/// X-Trans and 16-bit Bayer RAFs use the same fixed fit at thumbnail and export sizes.
/// The X-T20 fails the gates on all pixels (held-out RMS 0.114 against 0.10) and passes away
/// from edges (0.056, issue #232), so it is fitted too. Skips cleanly without these CC0
/// corpus files.
#[test]
fn corpus_raf_colour_and_white_balance() {
let dir = std::env::var_os("LIGHTCRAFT_CORPUS")
.map(std::path::PathBuf::from)
.unwrap_or_else(|| std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus"))
.join("raw");
for (name, accepted) in [("raf-fuji-xt2-865.raf", true), ("raf-fuji-gfx100s-4503.raf", true), ("raf-fuji-xt20-compressed.raf", true)] {
let Ok(bytes) = std::fs::read(dir.join(name)) else { continue };
let header = crate::files::probe_bytes(name, &bytes).unwrap();
assert_eq!(header.as_shot_wb, Some((6500.0, 0.0)), "{name}");
let (_, small) = crate::files::load_bytes(&bytes, 400).unwrap();
let (_, large) = crate::files::load_bytes(&bytes, 1200).unwrap();
assert_eq!(small.camera_tone.is_some(), accepted, "{name}");
assert_eq!(small.camera_tone, large.camera_tone, "{name}: resolution changed the fit");
assert!(small.raw && small.relative_wb && small.as_shot_temp == 6500.0 && small.as_shot_tint == 0.0, "{name}");
}
}
#[test]
fn rejects_monochrome_invalid_and_unrelated_previews() {
let mut sensor = Rgb32f::new(32, 32);
+8 -4
View File
@@ -1,6 +1,6 @@
//! Camera colour profiles of our own, pooled from many photos of one camera model.
//!
//! A raw file without colour matrices (Sony ARW, Nikon NEF) gets its look fitted to its own embedded camera
//! A raw file without colour matrices (Sony ARW, Nikon NEF, Fujifilm RAF) gets its look fitted to its own embedded camera
//! JPEG (`camera_preview`), but one photo shows too little of some colours: a lime shirt covering a
//! few dozen proxy pixels next to a hillside of foliage at the same hue. `lightcraft-cli calibrate`
//! pools the colour pairs of many photos per model and fits one matrix and hue/saturation/value
@@ -24,7 +24,11 @@ const MAX_FILE: u64 = 4 << 20;
/// Profiles built into LightCraft (`assets/camera-profiles/`, see `assets/ATTRIBUTION.md`):
/// `(model, JSON)`.
pub const BUNDLED: &[(&str, &str)] = &[("ILCE-7M4", include_str!("../../../assets/camera-profiles/ILCE-7M4.json"))];
pub const BUNDLED: &[(&str, &str)] = &[
("ILCE-7M4", include_str!("../../../assets/camera-profiles/ILCE-7M4.json")),
("X-H2S", include_str!("../../../assets/camera-profiles/X-H2S.json")),
("X-T4", include_str!("../../../assets/camera-profiles/X-T4.json")),
];
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct CameraProfile {
@@ -196,8 +200,8 @@ pub struct Pool {
const PAIRS_PER_FILE: usize = 4000;
impl Pool {
/// Add one raw file's colour pairs. `Ok(None)` when the file can't contribute (not an ARW or
/// NEF without colour matrices, no usable camera JPEG, too little colour).
/// Add one raw file's colour pairs. `Ok(None)` when the file can't contribute (not an ARW,
/// NEF or RAF without colour matrices, no usable camera JPEG, too little colour).
pub fn add(&mut self, bytes: &[u8]) -> Result<Option<String>, String> {
let mut raw = lightcraft_raw::decode(bytes).map_err(|e| e.to_string())?;
// colour only: geometric lens corrections would stop the sensor proxy from binning
+3 -7
View File
@@ -268,13 +268,9 @@ pub(crate) fn follow_folder(s: &mut Session, from: &str, to: &str) {
/// the folder itself.
fn follow_library_folder(chosen: &mut Option<String>, from: &str, to: &str) {
let Some(c) = chosen.as_deref() else { return };
let (ck, fk) = (lightcraft_catalog::query::folder_key(c), lightcraft_catalog::query::folder_key(from));
if fk.is_empty() {
return;
}
if let Some(rest) = ck.strip_prefix(fk.as_str()).filter(|r| r.is_empty() || r.starts_with('/') || fk.ends_with('/')) {
let rest = rest.trim_start_matches('/');
*chosen = Some(if rest.is_empty() { to.to_string() } else { format!("{}/{rest}", to.trim_end_matches(['/', '\\'])) });
if let Some(rest) = lightcraft_catalog::query::folder_rest(c, from) {
// joined by the platform, so the result is spelled like `to` and like a path read from disk
*chosen = Some(rest.iter().fold(std::path::PathBuf::from(to), |p, name| p.join(name)).to_string_lossy().to_string());
}
}
+184
View File
@@ -0,0 +1,184 @@
//! `faces.detect`: find faces with the YuNet detector the user has downloaded (Settings > Faces).
//!
//! The photo is rendered upright and uncropped with default settings (so edits, crops and rotations you have
//! made do not matter), the detector runs on it, and the faces come back as fractions of the photo's width
//! and height, the same frame face regions from XMP use. Unless `apply` is false, they are added to the photo as
//! unnamed face regions in one undoable step; earlier detections are replaced, and regions that came from XMP (or
//! that you drew or named) are never touched. Like every region edit it is catalog-only: no sidecar is written.
use std::path::PathBuf;
use std::sync::{Arc, Mutex, PoisonError};
use std::time::Instant;
use lightcraft_catalog::Op;
use lightcraft_develop::DevelopSettings;
use lightcraft_faces::known::YUNET_ID;
use lightcraft_faces::yunet::{Detector, Face, Options};
use lightcraft_meta::{Region, RegionKind};
use serde_json::{Value, json};
use super::{CommandSpec, bad, cmd, has_selection};
use crate::{Result, Session};
const C: &str = "faces.detect";
/// Regions made by this command say so in their description; only those are replaced on a new run.
pub(crate) const MARK: &str = "Detected by ";
pub(crate) const LABEL: &str = "YuNet 2023mar";
/// Long edge of the picture the detector looks at (it shrinks it to its own 640 anyway).
const LOOK_EDGE: usize = 1280;
/// YuNet is 232 KB; a file this much larger in its folder is not it.
const MAX_MODEL_BYTES: u64 = 8 << 20;
/// How much a detection must overlap a region the photo already has to count as the same face.
const EXISTING_IOU: f64 = 0.3;
/// What to tell someone who has not downloaded the detector yet.
const NOT_INSTALLED: &str = "finding faces needs the YuNet face detector (232 KB, MIT): download it in Settings > Faces";
/// The installed YuNet, loaded once and kept (a new one is loaded when the model folder changes).
/// (also used by the background scan, which aligns faces by the detector's landmarks and finds faces in photos that have none)
pub(crate) fn detector(s: &Session) -> std::result::Result<Arc<Detector>, String> {
static LOADED: Mutex<Option<(PathBuf, Arc<Detector>)>> = Mutex::new(None);
let home = s.face_models_dir.as_deref().ok_or("this build has nowhere to keep face models (the desktop app does)")?.join(YUNET_ID);
let path = home.join("model.onnx");
// installing writes the manifest last, so a folder without one is not (yet) a model
if !path.is_file() || !home.join("face-model.json").is_file() {
return Err(NOT_INSTALLED.into());
}
let mut loaded = LOADED.lock().unwrap_or_else(PoisonError::into_inner);
if let Some((p, d)) = loaded.as_ref()
&& *p == path
{
return Ok(d.clone());
}
let len = std::fs::metadata(&path).map_err(|e| format!("could not read the detector: {e}"))?.len();
if len > MAX_MODEL_BYTES {
return Err("the detector's file is far bigger than YuNet: remove it in Settings > Faces and download it again".into());
}
let bytes = std::fs::read(&path).map_err(|e| format!("could not read the detector: {e}"))?;
let d = Arc::new(Detector::new(&bytes).map_err(|e| format!("{e}: remove the detector in Settings > Faces and download it again"))?);
*loaded = Some((path, d.clone()));
Ok(d)
}
fn face_json(f: &Face) -> Value {
json!({
"rect": {"x0": f.x0, "y0": f.y0, "x1": f.x1, "y1": f.y1},
"score": f.score,
"landmarks": f.landmarks.iter().map(|(x, y)| json!([x, y])).collect::<Vec<_>>(),
})
}
/// A detected face that lies on a region the photo already has (from XMP, drawn or named) is that region, not a new face.
fn overlaps_existing(f: &Face, regions: &[Region]) -> bool {
regions.iter().any(|r| {
let (w, h) = (
(r.rect.x1.min(f64::from(f.x1)) - r.rect.x0.max(f64::from(f.x0))).max(0.0),
(r.rect.y1.min(f64::from(f.y1)) - r.rect.y0.max(f64::from(f.y0))).max(0.0),
);
let inter = w * h;
let union = (r.rect.x1 - r.rect.x0) * (r.rect.y1 - r.rect.y0) + f64::from((f.x1 - f.x0) * (f.y1 - f.y0)) - inter;
union > 0.0 && inter / union >= EXISTING_IOU
})
}
pub(crate) fn is_detected(r: &Region) -> bool {
r.description.as_deref().is_some_and(|d| d.starts_with(MARK))
}
/// The photo as the faces see it: upright, uncropped, default settings, as 8-bit RGB with its size, its long edge at
/// most `edge`.
pub(crate) fn render_rgb(s: &mut Session, id: lightcraft_catalog::PhotoId, edge: usize) -> std::result::Result<(Vec<u8>, usize, usize), String> {
let settings = DevelopSettings::default();
let job = s.preview_job(id, edge, edge, false, &settings).ok_or("no such photo")?;
let image = job.run().rendered?.image;
let rgb: Vec<u8> = image.data.iter().flat_map(|px| [px[0], px[1], px[2]]).collect();
Ok((rgb, image.width, image.height))
}
fn detect(s: &mut Session, p: &Value) -> Result<Value> {
let apply = p.get("apply").and_then(Value::as_bool).unwrap_or(true);
let defaults = Options::default();
let opts = Options {
score: p.get("score").and_then(Value::as_f64).map_or(defaults.score, |v| v as f32),
nms_iou: p.get("nmsIou").and_then(Value::as_f64).map_or(defaults.nms_iou, |v| v as f32),
max_faces: p.get("maxFaces").and_then(Value::as_u64).map_or(defaults.max_faces, |v| usize::try_from(v).unwrap_or(defaults.max_faces)),
};
let detector = detector(s).map_err(|e| bad(C, e))?;
let started = Instant::now();
let (mut results, mut ops) = (Vec::new(), Vec::new());
for id in s.targets(p) {
let (rgb, width, height) = match render_rgb(s, id, LOOK_EDGE) {
Ok(r) => r,
Err(e) => {
results.push(json!({"id": id.0, "error": e}));
continue;
}
};
let faces = match detector.detect(&rgb, width, height, &opts) {
Ok(f) => f,
Err(e) => {
results.push(json!({"id": id.0, "error": e.to_string()}));
continue;
}
};
if apply && let Some(photo) = s.catalog.photo(id) {
let mut meta = photo.meta.clone();
meta.regions.retain(|r| !is_detected(r));
let fresh: Vec<&Face> = faces.iter().filter(|f| !overlaps_existing(f, &meta.regions)).collect();
for f in fresh {
meta.regions.push(Region {
rect: lightcraft_geom::Rect { x0: f64::from(f.x0), y0: f64::from(f.y0), x1: f64::from(f.x1), y1: f64::from(f.y1) },
kind: RegionKind::Face,
name: None,
description: Some(format!("{MARK}{LABEL}")),
});
}
ops.push(Op::SetMeta { id, meta: Box::new(meta) });
}
results.push(json!({"id": id.0, "faces": faces.iter().map(face_json).collect::<Vec<_>>()}));
}
let applied = !ops.is_empty();
if applied {
s.commit("Detect Faces", Op::Batch { ops })?;
// regions are not written to XMP, so a sidecar must not be rewritten for this
s.skip_auto_write = true;
}
Ok(json!({"detector": LABEL, "applied": applied, "photos": results, "ms": started.elapsed().as_secs_f64() * 1000.0}))
}
pub fn specs() -> Vec<CommandSpec> {
vec![cmd!(
"faces.detect",
"Detect Faces",
["Photo"],
None,
"{ids?, apply?: bool (default true), score?: 0.6, nmsIou?: 0.3, maxFaces?} → {photos: [{id, faces: [{rect, score, landmarks}]}]}. Finds faces with the YuNet detector (download it first: `faces.models.download {id: \"yunet-2023mar\"}`) and, unless `apply` is false, adds them as unnamed face regions (one undo step), replacing earlier detections but never regions from XMP or ones you named; the sidecar is not touched",
has_selection,
detect
)]
}
#[cfg(test)]
mod tests {
use super::*;
fn face(x0: f32, y0: f32, x1: f32, y1: f32) -> Face {
Face { x0, y0, x1, y1, score: 0.9, landmarks: [(0.0, 0.0); 5] }
}
fn region(x0: f64, y0: f64, x1: f64, y1: f64) -> Region {
Region { rect: lightcraft_geom::Rect { x0, y0, x1, y1 }, kind: RegionKind::Face, name: Some("Ann".into()), description: None }
}
#[test]
fn a_detection_on_an_existing_region_is_not_a_new_face() {
let existing = [region(0.40, 0.20, 0.60, 0.50), region(0.10, 0.10, 0.20, 0.20)];
// the same face, a bit larger or shifted (a Lightroom box is not YuNet's box)
assert!(overlaps_existing(&face(0.38, 0.18, 0.64, 0.54), &existing));
assert!(overlaps_existing(&face(0.42, 0.22, 0.58, 0.48), &existing));
// another face, a sliver of overlap, and no regions at all
assert!(!overlaps_existing(&face(0.70, 0.20, 0.90, 0.50), &existing));
assert!(!overlaps_existing(&face(0.59, 0.49, 0.80, 0.80), &existing));
assert!(!overlaps_existing(&face(0.0, 0.0, 0.0, 0.0), &existing), "an empty box overlaps nothing");
assert!(!overlaps_existing(&face(0.4, 0.2, 0.6, 0.5), &[]));
}
}
+511
View File
@@ -0,0 +1,511 @@
//! Face model commands (`faces.*`): which models exist, inspecting a file the user dropped in, installing
//! it once its licence is accepted, removing it, and the faces on/off setting.
//!
//! Models live in the folder the host provides (`Session::face_models_dir`): one subfolder per model with
//! `model.onnx`, `face-model.json` (its manifest) and `installed.json` (what the user accepted), plus
//! `settings.json` beside them. Model files and manifests are hostile input: sizes are capped, ids are
//! validated before they become folder names, and nothing outside a model's own folder is ever touched.
use std::io::Read;
use std::path::{Path, PathBuf};
use lightcraft_faces::catalog::{self, Catalog};
use lightcraft_faces::hash::sha256_file;
use lightcraft_faces::manifest::{self, MAX_MANIFEST_BYTES, MAX_MODEL_BYTES, ModelManifest, Role};
use lightcraft_faces::suggest::{Suggestion, suggest_with};
use lightcraft_faces::{known, onnx};
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use super::{CommandSpec, always, bad, cmd, str_param};
use crate::face_download::{self, State};
use crate::{EngineError, Result, Session};
#[derive(Default, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub(super) struct FaceSettings {
pub(super) enabled: bool,
/// The installed recogniser in use.
pub(super) embedder: Option<String>,
}
fn fail(what: &str, e: impl std::fmt::Display) -> EngineError {
EngineError::Other(format!("{what}: {e}"))
}
fn models_dir(s: &Session, cmd: &str) -> Result<PathBuf> {
s.face_models_dir.clone().ok_or_else(|| bad(cmd, "this build has nowhere to keep face models (the desktop app does)"))
}
/// Read at most `max` bytes of a small file.
fn read_capped(path: &Path, max: usize) -> Option<Vec<u8>> {
let mut buf = Vec::new();
std::fs::File::open(path).ok()?.take(max as u64 + 1).read_to_end(&mut buf).ok()?;
(buf.len() <= max).then_some(buf)
}
pub(super) fn read_settings(dir: &Path) -> FaceSettings {
read_capped(&dir.join("settings.json"), 4096).and_then(|b| serde_json::from_slice(&b).ok()).unwrap_or_default()
}
fn write_atomic(path: &Path, bytes: &[u8]) -> Result<()> {
let part = path.with_extension("part");
std::fs::write(&part, bytes).map_err(|e| fail(&format!("could not write {}", path.display()), e))?;
std::fs::rename(&part, path).map_err(|e| {
let _ = std::fs::remove_file(&part);
fail(&format!("could not write {}", path.display()), e)
})
}
fn write_settings(dir: &Path, st: &FaceSettings) -> Result<()> {
std::fs::create_dir_all(dir).map_err(|e| fail("could not create the face models folder", e))?;
write_atomic(&dir.join("settings.json"), &serde_json::to_vec_pretty(st).map_err(|e| fail("settings", e))?)
}
pub(super) struct Installed {
pub(super) manifest: ModelManifest,
pub(super) accepted: Value,
}
/// The models in the folder. A folder that is not a valid model (wrong name, no model file, a manifest that
/// does not validate or does not match its folder) is ignored, never trusted.
pub(super) fn installed_models(dir: &Path) -> Vec<Installed> {
let Ok(entries) = std::fs::read_dir(dir) else { return Vec::new() };
let mut out = Vec::new();
for e in entries.flatten() {
let p = e.path();
let Some(name) = p.file_name().and_then(|n| n.to_str()).map(str::to_string) else { continue };
if !manifest::valid_id(&name) || !p.is_dir() || !p.join("model.onnx").is_file() {
continue;
}
let Some(m) = read_capped(&p.join("face-model.json"), MAX_MANIFEST_BYTES).and_then(|b| manifest::parse(&b).ok()) else { continue };
if m.id != name {
continue;
}
let accepted = read_capped(&p.join("installed.json"), 4096).and_then(|b| serde_json::from_slice(&b).ok()).unwrap_or(Value::Null);
out.push(Installed { manifest: m, accepted });
}
out.sort_by(|a, b| a.manifest.id.cmp(&b.manifest.id));
out
}
/// Read the user's catalog (`catalog.json` in the models folder) anew: a few kilobytes, so it is always current.
fn refresh_catalog(s: &mut Session) {
let Some(dir) = s.face_models_dir.clone() else { return };
let file = dir.join("catalog.json");
s.face_catalog = match read_capped(&file, catalog::MAX_CATALOG_BYTES) {
Some(bytes) => catalog::parse(&bytes),
None if file.is_file() => catalog::parse(&vec![b' '; catalog::MAX_CATALOG_BYTES + 1]),
None => Catalog::default(),
};
}
/// The manifests of the catalog's models (what a file's hash is matched against, besides the built-in ones).
fn catalog_manifests(s: &Session) -> Vec<ModelManifest> {
s.face_catalog.entries.iter().map(|e| e.manifest.clone()).collect()
}
/// How to fetch a model: the built-in address, or the one in the user's catalog.
fn download_spec(s: &Session, id: &str) -> Option<known::Download> {
known::download(id).or_else(|| s.face_catalog.entries.iter().find(|e| e.manifest.id == id).and_then(catalog::Entry::download))
}
fn row(s: &Session, m: &ModelManifest, installed: bool, selected: bool, accepted: &Value) -> Value {
let from_catalog = s.face_catalog.entries.iter().any(|e| e.manifest.id == m.id);
json!({
"id": m.id,
"name": m.name,
"role": m.role,
"licence": m.licence,
"provenance": m.provenance,
"source": m.source,
"sizeBytes": m.size_bytes,
"sha256": m.sha256,
// how fast it is as a multiple of a ResNet-100 model (computers differ, ratios between models hardly do), and
// that in words; null when nobody has measured it
"speed": m.speed,
"speedText": m.speed.map(manifest::describe_speed).filter(|t| !t.is_empty()),
"known": from_catalog || known::all().iter().any(|k| k.id == m.id),
// from the user's own catalog file
"fromCatalog": from_catalog,
// a pinned address LightCraft can fetch it from (the user presses Download), and which site that is
"downloadHost": download_spec(s, &m.id).map(|d| known::host(&d.url).to_string()),
"installed": installed,
"selected": selected,
"accepted": accepted,
})
}
struct Inspection {
sha256: String,
size: u64,
file_name: String,
suggestion: Suggestion,
}
/// Look at a model file. `known_sha` is its SHA-256 when something already checked it (a finished download), so a
/// file of hundreds of megabytes is not read again just to be hashed.
fn inspect_file(path: &str, cmd: &str, known_sha: Option<&str>, extra: &[ModelManifest]) -> Result<Inspection> {
let p = Path::new(path);
let meta = std::fs::metadata(p).map_err(|e| bad(cmd, format!("cannot read `{path}`: {e}")))?;
if !meta.is_file() {
return Err(bad(cmd, format!("`{path}` is not a file")));
}
if meta.len() == 0 || meta.len() > MAX_MODEL_BYTES {
return Err(bad(cmd, "a model file must be between 1 byte and 512 MiB"));
}
let sha256 = match known_sha {
Some(h) => h.to_string(),
None => sha256_file(p).map_err(|e| fail(&format!("could not read {path}"), e))?,
};
let file_name = p.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
let suggestion = match onnx::probe_path(p) {
Ok(info) => suggest_with(&info, &sha256, meta.len(), &file_name, extra),
Err(e) => Suggestion::Unsupported(e.to_string()),
};
Ok(Inspection { sha256, size: meta.len(), file_name, suggestion })
}
fn list(s: &mut Session, _: &Value) -> Result<Value> {
refresh_catalog(s);
let dir = s.face_models_dir.clone();
let (settings, installed) = match &dir {
Some(d) => (read_settings(d), installed_models(d)),
None => (FaceSettings::default(), Vec::new()),
};
let on_disk = |id: &str| installed.iter().find(|i| i.manifest.id == id);
let mut models: Vec<Value> = Vec::new();
for k in known::all() {
let found = on_disk(&k.id);
models.push(row(s, &k, found.is_some(), settings.embedder.as_deref() == Some(k.id.as_str()), found.map_or(&Value::Null, |f| &f.accepted)));
}
// the user's own catalog
for e in &s.face_catalog.entries {
let found = on_disk(&e.manifest.id);
let chosen = settings.embedder.as_deref() == Some(e.manifest.id.as_str());
models.push(row(s, &e.manifest, found.is_some(), chosen, found.map_or(&Value::Null, |f| &f.accepted)));
}
for i in &installed {
let listed = known::all().iter().any(|k| k.id == i.manifest.id) || s.face_catalog.entries.iter().any(|e| e.manifest.id == i.manifest.id);
if !listed {
models.push(row(s, &i.manifest, true, settings.embedder.as_deref() == Some(i.manifest.id.as_str()), &i.accepted));
}
}
Ok(json!({
"dir": dir.as_ref().map(|d| d.display().to_string()),
"enabled": settings.enabled,
"embedder": settings.embedder,
// Native builds always have the pure-Rust runner; model files/background scanning remain native-only.
"runtime": cfg!(not(target_arch = "wasm32")),
"models": models,
// the user's own list of models to download: where it goes, how many it holds, and what was wrong with the rest
"catalog": {
"path": dir.as_ref().map(|d| d.join("catalog.json").display().to_string()),
"models": s.face_catalog.entries.len(),
"errors": s.face_catalog.errors,
},
}))
}
fn inspect(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "faces.models.inspect";
let path = str_param(p, "path").ok_or_else(|| bad(C, "missing `path`"))?;
refresh_catalog(s);
let ins = inspect_file(path, C, None, &catalog_manifests(s))?;
let installed = s.face_models_dir.as_deref().map(installed_models).unwrap_or_default();
let (kind, model, assumptions, reason) = match &ins.suggestion {
Suggestion::Known(m) => ("known", Some(m), Vec::new(), None),
Suggestion::Draft { manifest, assumptions } => ("draft", Some(manifest), assumptions.clone(), None),
Suggestion::Unsupported(why) => ("unsupported", None, Vec::new(), Some(why.clone())),
};
let already = model.is_some_and(|m| installed.iter().any(|i| i.manifest.sha256 == m.sha256 && i.manifest.id == m.id));
Ok(json!({
"path": path,
"fileName": ins.file_name,
"sizeBytes": ins.size,
"sha256": ins.sha256,
"kind": kind,
"model": model.map(|m| row(s, m, already, false, &Value::Null)),
"assumptions": assumptions,
"reason": reason,
"alreadyInstalled": already,
}))
}
fn install(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "faces.models.install";
let path = str_param(p, "path").ok_or_else(|| bad(C, "missing `path`"))?;
let acknowledged = p.get("acknowledged").and_then(Value::as_bool).unwrap_or(false);
let activate = p.get("activate").and_then(Value::as_bool).unwrap_or(true);
refresh_catalog(s);
let ins = inspect_file(path, C, None, &catalog_manifests(s))?;
install_file(s, C, Path::new(path), ins, acknowledged, activate, false)
}
/// Install the inspected model file `src`. A recognition model that passes its self-test becomes the one in use and
/// recognition is switched on (`activate`, the default): installing a model is how someone says they want it.
/// `verified` is for a download that was already checked against its recorded hash: it is moved into place instead
/// of being copied and read again.
fn install_file(s: &mut Session, c: &str, src: &Path, ins: Inspection, acknowledged: bool, activate: bool, verified: bool) -> Result<Value> {
let dir = models_dir(s, c)?;
let m = match ins.suggestion {
Suggestion::Known(m) | Suggestion::Draft { manifest: m, .. } => m,
Suggestion::Unsupported(why) => return Err(bad(c, format!("this model cannot be used yet: {why}"))),
};
if !matches!(m.role, Role::Embedder | Role::Detector) {
return Err(bad(c, "only face detection and recognition models can be installed"));
}
manifest::validate(&m).map_err(|e| bad(c, e.to_string()))?;
if !acknowledged {
return Err(bad(c, "the licence has not been accepted: show the user its terms and pass `acknowledged: true` once they agree"));
}
let home = dir.join(&m.id);
std::fs::create_dir_all(&home).map_err(|e| fail("could not create the model's folder", e))?;
let (part, final_path) = (home.join("model.onnx.part"), home.join("model.onnx"));
let moved = verified && std::fs::rename(src, &part).is_ok();
let placed = if moved {
Ok(())
} else {
std::fs::copy(src, &part).map_err(|e| fail("could not copy the model (is the disk full?)", e)).and_then(|_| match sha256_file(&part) {
Ok(h) if h == ins.sha256 => Ok(()),
Ok(_) => Err(EngineError::Other("the copy does not match the original (the file changed while copying?)".into())),
Err(e) => Err(fail("could not check the copy", e)),
})
};
if let Err(e) = placed {
let _ = std::fs::remove_file(&part);
return Err(e);
}
// Validate the candidate before replacing a working installation.
let test = match self_test(&part, &m) {
Ok(t) => t,
Err(e) => {
let _ = std::fs::remove_file(&part);
let _ = std::fs::remove_dir(&home);
return Err(bad(c, e));
}
};
if let Err(e) = std::fs::rename(&part, &final_path) {
let _ = std::fs::remove_file(&part);
return Err(fail("could not finish the copy", e));
}
write_atomic(&home.join("face-model.json"), &serde_json::to_vec_pretty(&m).map_err(|e| fail("manifest", e))?)?;
let accepted = json!({"acceptedAt": (s.clock)(), "licence": m.licence.name, "commercial": m.licence.commercial, "fileName": ins.file_name, "selfTest": test});
write_atomic(&home.join("installed.json"), &serde_json::to_vec_pretty(&accepted).map_err(|e| fail("record", e))?)?;
// a file that was downloaded and waited in the staging folder has served: the installed copy is the model now
if src.parent() == Some(dir.join(face_download::STAGING).as_path()) {
let _ = std::fs::remove_file(src);
s.face_downloads.staged_file_used(src);
}
// the newest model is the one in use (the earlier one stays installed, and `faces.models.select` switches back)
if activate && m.role == Role::Embedder {
let mut st = read_settings(&dir);
st.embedder = Some(m.id.clone());
st.enabled = true;
write_settings(&dir, &st)?;
}
let now = read_settings(&dir);
let selected = now.embedder.as_deref() == Some(m.id.as_str());
Ok(json!({"installed": row(s, &m, true, selected, &accepted), "active": {"embedder": now.embedder, "enabled": now.enabled}}))
}
/// What an error says, without the command it came from (for a line shown to the user).
fn plain(e: &EngineError) -> String {
match e {
EngineError::BadParams { msg, .. } => msg.clone(),
other => other.to_string(),
}
}
/// Install every download that has arrived: each was accepted when it was started, so it is installed, chosen and
/// switched on without another question. Called by `faces.models.downloads` and by every frame's `faces.pump`.
pub(super) fn finish_downloads(s: &mut Session) {
const C: &str = "faces.models.download";
let extra = catalog_manifests(s);
for (id, path, sha) in s.face_downloads.finished() {
let outcome = inspect_file(&path.to_string_lossy(), C, Some(&sha), &extra).and_then(|ins| install_file(s, C, &path, ins, true, true, true));
match outcome {
Ok(_) => s.face_downloads.set_outcome(&id, State::Installed),
Err(e) => {
// a file that cannot be installed is of no use to anyone: throw it away, say why
let _ = std::fs::remove_file(&path);
s.face_downloads.set_outcome(&id, State::Failed(plain(&e)));
}
}
}
}
/// `faces.models.download {id, acknowledged: true}`: fetch a model LightCraft knows an address for, in the background;
/// when it has arrived and checked out it is installed.
fn download(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "faces.models.download";
let id = str_param(p, "id").ok_or_else(|| bad(C, "missing `id`"))?;
let dir = models_dir(s, C)?;
refresh_catalog(s);
let spec = download_spec(s, id).ok_or_else(|| bad(C, "LightCraft has no download for that model: get the file from its page, then add it"))?;
if cfg!(target_arch = "wasm32") {
return Err(bad(C, "the web build downloads no models"));
}
if p.get("acknowledged").and_then(Value::as_bool) != Some(true) {
return Err(bad(
C,
"the licence has not been accepted: show the user the model's terms (see `faces.models.list`) and pass `acknowledged: true` once they agree",
));
}
if installed_models(&dir).iter().any(|i| i.manifest.id == spec.id) {
return Err(bad(C, "that model is already installed"));
}
let host = spec.host().to_string();
s.face_downloads.start(spec, &dir).map_err(|e| bad(C, e))?;
Ok(json!({"started": id, "from": host}))
}
fn download_row(s: &Session, id: &str, state: &State) -> Value {
let from = download_spec(s, id).map(|d| d.host().to_string());
match state {
State::Running { bytes, total } => json!({"id": id, "state": "running", "bytes": bytes, "total": total, "from": from}),
State::Done { path, .. } => json!({"id": id, "state": "done", "path": path.display().to_string(), "from": from}),
State::Installed => json!({"id": id, "state": "installed", "from": from}),
State::Failed(why) => json!({"id": id, "state": "failed", "error": why, "from": from}),
State::Cancelled => json!({"id": id, "state": "cancelled", "from": from}),
}
}
/// `faces.models.downloads`: where each download stands (a finished one is installed first).
fn downloads(s: &mut Session, _: &Value) -> Result<Value> {
finish_downloads(s);
let all: Vec<Value> = s.face_downloads.snapshot().iter().map(|(id, st)| download_row(s, id, st)).collect();
Ok(json!({"running": s.face_downloads.running(), "downloads": all}))
}
/// `faces.models.downloadCancel {id}`: stop a download, or clear a finished one (a staged file is deleted).
fn download_cancel(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "faces.models.downloadCancel";
let id = str_param(p, "id").ok_or_else(|| bad(C, "missing `id`"))?;
Ok(json!({"discarded": s.face_downloads.discard(id)}))
}
/// Load the model and run its self-test: `Ok(null)` in a build without the recognition runtime (nothing can
/// be checked), `Ok(result)` when it passes, `Err(reason)` when it does not load or fails a check.
#[cfg(not(target_arch = "wasm32"))]
fn self_test(path: &Path, m: &ModelManifest) -> std::result::Result<Value, String> {
if m.role == Role::Detector {
let bytes = read_capped(path, 8 << 20).ok_or("the detector cannot be read within its 8 MiB limit")?;
let detector = lightcraft_faces::yunet::Detector::new(&bytes).map_err(|e| e.to_string())?;
if Some(detector.input_side()) != usize::try_from(m.input.width).ok()
|| m.input.width != m.input.height
|| !matches!(&m.output, manifest::OutputSpec::Detector { decoder } if decoder == "yunet-v2")
{
return Err("unsupported detector input or output decoder".into());
}
detector.detect(&[0u8; 12], 2, 2, &lightcraft_faces::yunet::Options::default()).map_err(|e| e.to_string())?;
return Ok(json!({"ok": true}));
}
let embedder = lightcraft_faces::runtime::Embedder::load(path, m).map_err(|e| e.to_string())?;
let t = embedder.self_test();
if !t.ok {
let failed: Vec<&str> = t.checks.iter().filter(|(_, ok)| !*ok).map(|(c, _)| c.as_str()).collect();
return Err(format!("the model failed its self-test: it does not {}", failed.join(", and does not ")));
}
serde_json::to_value(&t).map_err(|e| e.to_string())
}
#[cfg(target_arch = "wasm32")]
fn self_test(_: &Path, _: &ModelManifest) -> std::result::Result<Value, String> {
Ok(Value::Null)
}
/// `faces.models.test {id}`: run an installed model's self-test again.
fn test(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "faces.models.test";
let id = str_param(p, "id").ok_or_else(|| bad(C, "missing `id`"))?;
let dir = models_dir(s, C)?;
if !cfg!(not(target_arch = "wasm32")) {
return Err(bad(C, "this build cannot run face recognition models"));
}
let found = installed_models(&dir).into_iter().find(|i| i.manifest.id == id).ok_or_else(|| bad(C, "that model is not installed"))?;
if found.manifest.role != Role::Embedder {
return Err(bad(C, "only recognition models have a self-test"));
}
let result = self_test(&dir.join(id).join("model.onnx"), &found.manifest);
let (ok, detail) = match &result {
Ok(v) => (true, v.clone()),
Err(e) => (false, json!(e)),
};
// keep the latest outcome with the model's record
let mut record = found.accepted;
if let Some(o) = record.as_object_mut() {
o.insert("selfTest".into(), if ok { detail.clone() } else { json!({"ok": false, "error": detail}) });
write_atomic(&dir.join(id).join("installed.json"), &serde_json::to_vec_pretty(&record).map_err(|e| fail("record", e))?)?;
}
Ok(json!({"id": id, "ok": ok, "result": detail}))
}
fn remove(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "faces.models.remove";
let id = str_param(p, "id").ok_or_else(|| bad(C, "missing `id`"))?;
let dir = models_dir(s, C)?;
if !manifest::valid_id(id) {
return Err(bad(C, "not a model id"));
}
let home = dir.join(id);
// only a folder that is a model (it has a manifest) is ever deleted
if !home.join("face-model.json").is_file() {
return Err(bad(C, "that model is not installed"));
}
std::fs::remove_dir_all(&home).map_err(|e| fail("could not remove the model", e))?;
let mut st = read_settings(&dir);
if st.embedder.as_deref() == Some(id) {
// another installed recogniser takes over, so recognition keeps working; with none left nothing is chosen
st.embedder = installed_models(&dir).into_iter().find(|i| i.manifest.id != id && i.manifest.role == Role::Embedder).map(|i| i.manifest.id);
write_settings(&dir, &st)?;
}
Ok(json!({"removed": id, "embedder": st.embedder}))
}
fn select(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "faces.models.select";
let dir = models_dir(s, C)?;
let id = match p.get("id") {
None | Some(Value::Null) => None,
Some(v) => Some(v.as_str().ok_or_else(|| bad(C, "`id` must be a model id or null"))?.to_string()),
};
if let Some(id) = &id
&& !installed_models(&dir).iter().any(|i| &i.manifest.id == id && i.manifest.role == Role::Embedder)
{
return Err(bad(C, "that recognition model is not installed"));
}
let mut st = read_settings(&dir);
st.embedder = id;
write_settings(&dir, &st)?;
Ok(json!({"embedder": st.embedder}))
}
fn enable(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "faces.enable";
let dir = models_dir(s, C)?;
let mut st = read_settings(&dir);
st.enabled = p.get("enabled").and_then(Value::as_bool).unwrap_or(!st.enabled);
write_settings(&dir, &st)?;
// the background scan notices at once, not within the second it trusts its last look
#[cfg(not(target_arch = "wasm32"))]
{
s.faces.enabled_seen = None;
}
Ok(json!({"enabled": st.enabled}))
}
pub fn specs() -> Vec<CommandSpec> {
vec![
cmd!(query "faces.models.list", "Face Models", [], None, "{} → {dir, enabled, embedder, runtime, models: [{id, name, role, licence{name, commercial, url, notice}, provenance, source, sizeBytes, known, installed, selected}]}", always, list),
cmd!(query "faces.models.inspect", "Inspect Face Model File", [], None, "{path} → what a .onnx file is: {kind: known | draft | unsupported, model, assumptions, reason, alreadyInstalled}; installs nothing", always, inspect),
cmd!(query "faces.models.install", "Install Face Model", [], None, "{path, acknowledged: true, activate?: true} → {installed, active: {embedder, enabled}} — copy a .onnx face recognition model into the models folder. `acknowledged` must be true: the user has been shown its licence (see inspect) and accepted it. Unless `activate` is false the model becomes the one in use and recognition is switched on; an earlier model stays installed", always, install),
cmd!(query "faces.models.download", "Download Face Model", [], None, "{id, acknowledged: true} → {started, from} — fetch a recognition model LightCraft has a pinned address for (see `downloadHost` in the list), in the background over pure-Rust https (lightcraft-fetch). `acknowledged` must be true: the user has been shown the model's terms and accepted them. It is checked against its size and SHA-256, then installed, chosen and switched on by itself; `faces.models.downloads` shows how far it is", always, download),
cmd!(query "faces.models.downloads", "Face Model Downloads", [], None, "{} → {running, downloads: [{id, state: running | done | installed | failed | cancelled, bytes, total, error, from}]} — also installs any download that has arrived; `installed` stays listed until `faces.models.downloadCancel` clears it", always, downloads),
cmd!(query "faces.models.downloadCancel", "Cancel Face Model Download", [], None, "{id} → {discarded} — stop a download, or delete a finished one that was not installed", always, download_cancel),
cmd!(query "faces.models.test", "Test Face Model", [], None, "{id} → {ok, result: {loadMs, embedMs, dimension, checks}} — load an installed recognition model and check it gives sensible faces; needs the recognition runtime", always, test),
cmd!(query "faces.models.remove", "Remove Face Model", [], None, "{id} — delete an installed model", always, remove),
cmd!(query "faces.models.select", "Choose Face Recognition Model", [], None, "{id: installed recogniser | null}", always, select),
cmd!(query "faces.enable", "Face Recognition On/Off", [], None, "{enabled?: bool} (toggles when omitted)", always, enable),
]
}
File diff suppressed because it is too large. Load diff
+102 -1
View File
@@ -119,6 +119,14 @@ fn album_param(p: &Value, key: &str, c: &str) -> Result<AlbumId> {
p.get(key).and_then(Value::as_u64).map(AlbumId).ok_or_else(|| bad(c, format!("missing album `{key}`")))
}
/// An optional album id: absent or `null` is `None`, anything but a number is an error.
fn opt_album_param(p: &Value, key: &str, c: &str) -> Result<Option<AlbumId>> {
match p.get(key) {
None | Some(Value::Null) => Ok(None),
Some(v) => v.as_u64().map(|n| Some(AlbumId(n))).ok_or_else(|| bad(c, format!("`{key}` must be an album id or null"))),
}
}
fn strs(p: &Value, key: &str) -> Vec<String> {
p.get(key).and_then(Value::as_array).map(|a| a.iter().filter_map(Value::as_str).map(str::to_string).collect()).unwrap_or_default()
}
@@ -784,7 +792,7 @@ pub fn specs() -> Vec<CommandSpec> {
let cover = photos.first().copied();
s.commit(
if folder { "New Folder" } else { "New Album" },
Op::AddAlbum { album: Album { id, name, parent, folder, photos, cover, smart: None, quick: false } },
Op::AddAlbum { album: Album { id, name, parent, folder, photos, cover, smart: None, quick: false, order: None } },
)?;
Ok(json!({"id": id.0}))
}),
@@ -880,6 +888,80 @@ pub fn specs() -> Vec<CommandSpec> {
s.commit("Move Album", Op::MoveAlbum { id, parent })?;
ok()
}),
cmd!(
"album.reorder",
"Reorder Album",
[],
None,
"{id, parent?: folderId|null, before?: albumId|null} — places the album (or folder) in `parent` (default: where it is; null: the top level) before the sibling `before` of the same kind, or last of its kind; the folder is then ordered by hand. One undo step. → {changed}: how many edits it took (0: already there)",
always,
|s, p| {
let id = album_param(p, "id", "album.reorder")?;
let me = s.catalog.album(id).ok_or_else(|| bad("album.reorder", "no such album"))?;
let parent = if p.get("parent").is_some() { opt_album_param(p, "parent", "album.reorder")? } else { me.parent };
let before = opt_album_param(p, "before", "album.reorder")?;
if before == Some(id) {
// before itself: where it is
return Ok(json!({"changed": 0}));
}
let mut order: Vec<AlbumId> = s.catalog.album_children(parent).into_iter().filter(|a| a.id != id).map(|a| a.id).collect();
let at = match before {
Some(b) => {
let at = order.iter().position(|x| *x == b).ok_or_else(|| bad("album.reorder", "`before` is not inside that folder"))?;
if s.catalog.album(b).is_none_or(|x| x.folder != me.folder) {
return Err(bad("album.reorder", "folders are placed among folders and albums among albums"));
}
at
}
None if me.folder => order.iter().take_while(|x| s.catalog.album(**x).is_some_and(|a| a.folder)).count(),
None => order.len(),
};
order.insert(at.min(order.len()), id);
let moved = me.parent != parent;
let mut ops = Vec::new();
if moved {
ops.push(Op::MoveAlbum { id, parent });
}
for (i, aid) in order.iter().enumerate() {
let want = u32::try_from(i).ok();
// a moved album starts without a place in its new folder
let have = if moved && *aid == id { None } else { s.catalog.album(*aid).and_then(|a| a.order) };
if have != want {
ops.push(Op::SetAlbumOrder { id: *aid, order: want });
}
}
if ops.is_empty() {
return Ok(json!({"changed": 0}));
}
let changed = ops.len();
s.commit("Reorder Album", Op::Batch { ops })?;
Ok(json!({"changed": changed}))
}
),
cmd!(
"album.sort",
"Sort Albums by Name",
[],
None,
"{parent?: folderId|null} — drops the hand order of the albums inside the folder (default: the top level), so they are listed by name → {changed}: how many albums lost their place",
always,
|s, p| {
let parent = opt_album_param(p, "parent", "album.sort")?;
let ops: Vec<Op> = s
.catalog
.album_children(parent)
.into_iter()
.filter(|a| a.order.is_some())
.map(|a| Op::SetAlbumOrder { id: a.id, order: None })
.collect();
if ops.is_empty() {
return Ok(json!({"changed": 0}));
}
let changed = ops.len();
s.commit("Sort Albums by Name", Op::Batch { ops })?;
Ok(json!({"changed": changed}))
}
),
cmd!("album.addPhotos", "Add to Album", ["Photo"], None, "{id: albumId, ids?: [photoIds]} (default: selection)", has_selection, |s, p| {
let id = album_param(p, "id", "album.addPhotos")?;
let targets = ids_param(p).unwrap_or_else(|| s.targets(&Value::Null));
@@ -1062,6 +1144,25 @@ pub fn specs() -> Vec<CommandSpec> {
ok()
}),
// ---- import
cmd!(query "library.inspectLightroom", "Inspect Lightroom Catalog", [], None,
"{path: .lrcat} → {photos, collections, missing, warnings}; read-only, no Lightroom required",
always, |_, p| {
let path = str_param(p,"path").ok_or_else(|| bad("library.inspectLightroom","missing path"))?;
let data = crate::lightroom_catalog::read(std::path::Path::new(path)).map_err(|e| bad("library.inspectLightroom",e))?;
Ok(json!({"photos":data.photos.len(),"collections":data.collections.iter().filter(|r|r.get("systemOnly").and_then(Value::as_f64).unwrap_or(0.0)==0.0).count(),"missing":data.photos.iter().filter(|p| !std::path::Path::new(&p.path).is_file()).map(|p|&p.path).collect::<Vec<_>>(),"warnings":data.warnings}))
}),
cmd!(
"library.importLightroom",
"Import Lightroom Catalog",
[],
None,
"{path: .lrcat, updateExisting?: false}; direct import of paths, ratings, flags, metadata, keywords, collections, virtual copies & mapped edits. Archives source settings/history first. Existing edited photos are preserved unless updateExisting=true; source catalog is read-only.",
always,
|s, p| {
let path = str_param(p, "path").ok_or_else(|| bad("library.importLightroom", "missing path"))?;
crate::lightroom_catalog::import(s, std::path::Path::new(path), bool_or(p, "updateExisting", false))
}
),
cmd!(
query "library.importPreview",
"Review Import",
+5 -16
View File
@@ -41,22 +41,11 @@ fn cube_files(paths: &[String]) -> Vec<(String, String, Option<String>)> {
for p in paths {
let path = Path::new(p);
if path.is_dir() {
let mut stack = vec![path.to_path_buf()];
while let Some(d) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&d) else { continue };
for e in rd.flatten() {
let ep = e.path();
if ep.file_name().is_some_and(|n| n.to_string_lossy().starts_with('.')) {
continue;
}
if ep.is_dir() {
stack.push(ep);
} else if ep.extension().is_some_and(|x| x.eq_ignore_ascii_case("cube"))
&& let Ok(t) = std::fs::read_to_string(&ep)
{
let group = ep.parent().and_then(|d| d.file_name()).map(|n| n.to_string_lossy().to_string());
out.push((ep.file_stem().map(|n| n.to_string_lossy().to_string()).unwrap_or_default(), t, group));
}
let cube = |f: &Path| f.extension().is_some_and(|x| x.eq_ignore_ascii_case("cube"));
for ep in crate::walk::files_in(path, None, crate::walk::Limits::default(), cube).files {
if let Ok(t) = std::fs::read_to_string(&ep) {
let group = ep.parent().and_then(|d| d.file_name()).map(|n| n.to_string_lossy().to_string());
out.push((ep.file_stem().map(|n| n.to_string_lossy().to_string()).unwrap_or_default(), t, group));
}
}
} else if path.extension().is_some_and(|x| x.eq_ignore_ascii_case("zip")) {
+1
View File
@@ -432,6 +432,7 @@ pub fn specs() -> Vec<CommandSpec> {
"available": crate::segment::Segmenter::AVAILABLE,
"installed": g.installed(),
"dir": g.dir.as_ref().map(|d| d.display().to_string()),
"remote": g.remote_endpoint(),
"loaded": g.loaded(),
"busy": g.busy(),
"analyzing": g.analyzing(),
+6
View File
@@ -13,6 +13,9 @@ pub mod curves;
mod develop;
mod edit;
mod export;
pub(crate) mod face_detect;
mod face_models;
mod face_recognize;
pub mod filters;
pub mod keywords;
pub mod library;
@@ -140,6 +143,9 @@ pub fn command_specs() -> &'static [CommandSpec] {
v.extend(missing::specs());
v.extend(metadata::specs());
v.extend(filters::specs());
v.extend(face_models::specs());
v.extend(face_detect::specs());
v.extend(face_recognize::specs());
v
})
}
+37
View File
@@ -0,0 +1,37 @@
//! Where LightCraft keeps per-user data on this machine: the config folder (app settings, face models).
//!
//! Hosts that have a file system (the desktop app, the CLI, the MCP server) share these defaults so a model
//! installed from one is there for the others. Nothing here is applied automatically: a [`Session`] has no
//! face-models folder until a host asks for one ([`Session::with_default_face_models`]), so tests stay
//! hermetic.
use std::path::PathBuf;
use crate::Session;
/// LightCraft's per-user config folder (`…/LightCraft`), if the platform tells us where.
pub fn config_dir() -> Option<PathBuf> {
if cfg!(target_os = "macos") {
std::env::var_os("HOME").map(|h| PathBuf::from(h).join("Library/Application Support/LightCraft"))
} else if cfg!(windows) {
std::env::var_os("APPDATA").map(|a| PathBuf::from(a).join("LightCraft"))
} else {
std::env::var_os("XDG_CONFIG_HOME")
.map(PathBuf::from)
.or_else(|| std::env::var_os("HOME").map(|h| PathBuf::from(h).join(".config")))
.map(|c| c.join("lightcraft"))
}
}
/// Where face models are kept: `$LIGHTCRAFT_FACE_MODELS` if set, else `<config>/models`.
pub fn default_face_models_dir() -> Option<PathBuf> {
std::env::var_os("LIGHTCRAFT_FACE_MODELS").filter(|v| !v.is_empty()).map(PathBuf::from).or_else(|| config_dir().map(|d| d.join("models")))
}
impl Session {
/// Keep face models in the shared default folder ([`default_face_models_dir`]).
pub fn with_default_face_models(mut self) -> Self {
self.face_models_dir = default_face_models_dir();
self
}
}
+97 -10
View File
@@ -44,6 +44,17 @@ const NON_ADJUSTMENT: &[&str] = &[
"crs:AlreadyApplied",
"crs:RawFileName",
"crs:HasCrop",
// preset/library bookkeeping and tool options that don't change the rendering
"crs:Cluster",
"crs:SupportsAmount2",
"crs:RequiresRGBTables",
"crs:SortName",
"crs:Description",
"crs:CropConstrainToWarp",
"crs:OverrideLookVignette",
// the file's as-shot white, recorded next to the edit (ours comes from the file itself)
"crs:AsShotTemperature",
"crs:AsShotTint",
];
/// True if the packet carries any `crs:` adjustment (not just bookkeeping fields).
@@ -233,11 +244,14 @@ pub fn to_partial(props: &Props, raw: Option<bool>) -> Value {
put(o, "curve.master", c);
}
}
for (crs, ch) in
[("ToneCurvePV2012", "master"), ("ToneCurvePV2012Red", "red"), ("ToneCurvePV2012Green", "green"), ("ToneCurvePV2012Blue", "blue")]
{
if let Some(c) = curve(props, &format!("crs:{crs}")) {
put(o, &format!("curve.{ch}"), c);
// Lightroom Classic reads the red / green / blue curves only together with the master 2012
// curve: a channel curve without `ToneCurvePV2012` leaves the photo unchanged
if let Some(c) = curve(props, "crs:ToneCurvePV2012") {
put(o, "curve.master", c);
for (crs, ch) in [("ToneCurvePV2012Red", "red"), ("ToneCurvePV2012Green", "green"), ("ToneCurvePV2012Blue", "blue")] {
if let Some(c) = curve(props, &format!("crs:{crs}")) {
put(o, &format!("curve.{ch}"), c);
}
}
}
@@ -287,9 +301,12 @@ pub fn to_partial(props: &Props, raw: Option<bool>) -> Value {
n(o, "GrainSize", "grain.size");
n(o, "GrainFrequency", "grain.roughness");
// ---- Optics (manual corrections; lens profiles are ours, only the switch carries over)
if let Some(b) = boolean(props, "crs:LensProfileEnable") {
put(o, "optics.lens_profile", json!(b));
// ---- Optics (manual corrections; lens profiles are ours, only the switch carries over).
// Our profile corrections are the file's own (DNG-embedded) ones, which Lightroom applies
// whatever its "Enable Profile Corrections" box says [inferred]: so only an enabling switch
// carries over, and `LensProfileEnable=0` (the default in most presets) leaves ours as it is.
if boolean(props, "crs:LensProfileEnable") == Some(true) {
put(o, "optics.lens_profile", json!(true));
}
if let Some(b) = boolean(props, "crs:AutoLateralCA") {
put(o, "optics.remove_ca", json!(b));
@@ -373,8 +390,19 @@ pub fn to_partial_report(props: &Props, values: Option<&crate::crs_masks::Values
mask_skips = skipped.into_iter().map(|k| format!("Mask: {k}")).collect();
}
}
// fields that only switch a panel on/off or name things: not adjustments by themselves
let quiet = |k: &str| k.starts_with("Enable") || k.starts_with("ToneCurveName") || k == "AutoTone" || k == "AutoGrayscaleMix";
// fields that only switch a panel on/off or name things: not adjustments by themselves; an
// HDR edit mode that is off, and Point Color slots that are all empty (-1)
let value_is = |k: &str, f: &dyn Fn(&[String]) -> bool| props.get(&format!("crs:{k}")).is_some_and(|v| f(v));
let off = |v: &[String]| v.iter().all(|s| s.trim() == "0");
let empty_points = |v: &[String]| v.iter().all(|s| s.split(',').all(|n| n.trim().parse::<f64>().is_ok_and(|x| x == -1.0) || n.trim().is_empty()));
let quiet = |k: &str| {
k.starts_with("Enable")
|| k.starts_with("ToneCurveName")
|| k == "AutoTone"
|| k == "AutoGrayscaleMix"
|| (k == "HDREditMode" && value_is(k, &off))
|| (k == "PointColors" && value_is(k, &empty_points))
};
let mut unmapped: Vec<String> = props
.keys()
.filter(|k| k.starts_with("crs:"))
@@ -539,4 +567,63 @@ mod tests {
assert!(!p.builtin);
assert!(preset_from_xmp("<x/>", "f").is_none());
}
/// A preset packet (written for this test) with the bookkeeping fields Lightroom presets carry.
fn preset_packet(extra_attrs: &str, extra_elems: &str) -> String {
format!(
r#"<x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<rdf:Description rdf:about="" xmlns:crs="http://ns.adobe.com/camera-raw-settings/1.0/"
crs:PresetType="Normal" crs:Cluster="" crs:UUID="0123ABCD" crs:SupportsAmount2="True" crs:SupportsAmount="True"
crs:RequiresRGBTables="False" crs:Contrast2012="+25" crs:LensProfileEnable="0" crs:OverrideLookVignette="False"
crs:CropConstrainToWarp="0" crs:AsShotTemperature="5500" crs:AsShotTint="10" {extra_attrs}>
<crs:SortName><rdf:Alt><rdf:li xml:lang="x-default"/></rdf:Alt></crs:SortName>
<crs:Description><rdf:Alt><rdf:li xml:lang="x-default"/></rdf:Alt></crs:Description>
{extra_elems}
</rdf:Description></rdf:RDF></x:xmpmeta>"#
)
}
#[test]
fn channel_curves_need_the_master_curve() {
// Lightroom ignores a red / green / blue curve without `ToneCurvePV2012`
let packet = |master: &str| {
format!(
r#"<x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<rdf:Description rdf:about="" xmlns:crs="http://ns.adobe.com/camera-raw-settings/1.0/" crs:ToneCurveName2012="Custom">{master}
<crs:ToneCurvePV2012Green><rdf:Seq><rdf:li>0, 0</rdf:li><rdf:li>64, 40</rdf:li><rdf:li>255, 255</rdf:li></rdf:Seq></crs:ToneCurvePV2012Green>
</rdf:Description></rdf:RDF></x:xmpmeta>"#
)
};
let alone = to_partial(&props(&packet("")), Some(true));
assert!(alone.pointer("/curve/green").is_none(), "{alone}");
let master = "<crs:ToneCurvePV2012><rdf:Seq><rdf:li>0, 0</rdf:li><rdf:li>255, 255</rdf:li></rdf:Seq></crs:ToneCurvePV2012>";
let with = to_partial(&props(&packet(master)), Some(true));
assert!(with.pointer("/curve/green").is_some() && with.pointer("/curve/master").is_some(), "{with}");
}
fn unmapped(x: &str) -> Vec<String> {
to_partial_report(&props(x), None, None, 1.5).1
}
#[test]
fn preset_bookkeeping_is_not_reported_but_real_gaps_are() {
let empty_points = "<crs:PointColors><rdf:Seq><rdf:li>-1.000000, -1.000000, -1.000000, -1.000000</rdf:li></rdf:Seq></crs:PointColors>";
assert_eq!(unmapped(&preset_packet(r#"crs:HDREditMode="0""#, empty_points)), Vec::<String>::new());
// an HDR edit, a used Point Color slot and an unknown adjustment still are
let used_points = "<crs:PointColors><rdf:Seq><rdf:li>0.5, 0.2, 0.1, 10, 0, 0, 0, 0</rdf:li></rdf:Seq></crs:PointColors>";
let got = unmapped(&preset_packet(r#"crs:HDREditMode="1" crs:FutureSlider="12""#, used_points));
assert_eq!(got, ["FutureSlider", "HDREditMode", "PointColors"]);
}
#[test]
fn disabled_lens_profile_switch_keeps_embedded_corrections() {
// Lightroom applies a file's built-in lens corrections regardless of its profile switch,
// and ours are only those: a preset's `LensProfileEnable=0` must not turn them off
let p = props(&preset_packet("", ""));
let mut d = DevelopSettings::default();
d.optics.lens_profile = true;
assert!(apply_partial(&d, &to_partial(&p, None), 1.0).optics.lens_profile);
let on = props(&preset_packet("", "").replace(r#"crs:LensProfileEnable="0""#, r#"crs:LensProfileEnable="1""#));
assert!(apply_partial(&DevelopSettings::default(), &to_partial(&on, None), 1.0).optics.lens_profile);
}
}
+14 -2
View File
@@ -62,7 +62,17 @@ pub fn load(s: &mut Session) {
}
let folder = s.catalog.alloc_album_id();
ops.push(Op::AddAlbum {
album: Album { id: folder, name: "Travel 2026".into(), parent: None, folder: true, photos: vec![], cover: None, smart: None, quick: false },
album: Album {
id: folder,
name: "Travel 2026".into(),
parent: None,
folder: true,
photos: vec![],
cover: None,
smart: None,
quick: false,
order: None,
},
});
let by_kw = |kw: &str| -> Vec<_> { scenes.iter().zip(&ids).filter(|(sc, _)| sc.meta.keywords.contains(&kw)).map(|(_, id)| *id).collect() };
for (name, parent, photos) in [
@@ -75,7 +85,9 @@ pub fn load(s: &mut Session) {
] {
let id = s.catalog.alloc_album_id();
let cover = photos.first().copied();
ops.push(Op::AddAlbum { album: Album { id, name: name.into(), parent, folder: false, photos, cover, smart: None, quick: false } });
ops.push(Op::AddAlbum {
album: Album { id, name: name.into(), parent, folder: false, photos, cover, smart: None, quick: false, order: None },
});
}
for op in ops {
let _ = s.catalog.apply(op);
+1 -1
View File
@@ -191,7 +191,7 @@ pub fn devices_in(parents: &[std::path::PathBuf], extra: Vec<std::path::PathBuf>
/// The volume's `DCIM` folder: the usual spellings are checked directly, so a volume's root is
/// only listed (other spellings on a case-sensitive volume) off Windows, whose drives aren't.
fn dcim_in(root: &std::path::Path) -> Option<std::path::PathBuf> {
pub fn dcim_in(root: &std::path::Path) -> Option<std::path::PathBuf> {
if let Some(p) = ["DCIM", "dcim"].iter().map(|n| root.join(n)).find(|p| p.is_dir()) {
return Some(p);
}
+91 -25
View File
@@ -258,33 +258,41 @@ fn watermark_fonts(craft: &'static [crate::fonts::CraftFont]) -> Vec<ab_glyph::F
.collect()
}
/// The font's vertical form, positioned from its vertical origin within the watermark cell.
/// Shape a whole grapheme in one cell. CJK uses vertical origins and forms; Latin stays upright.
/// Unlike Unicode presentation-form characters, GSUB works with BIZ UD fonts too.
fn vertical_watermark_glyph(
fn watermark_cell_glyphs(
font: &ab_glyph::FontRef<'_>,
data: &harfrust::ShaperData,
ch: char,
text: &str,
vertical: bool,
px: f32,
left: f32,
top: f32,
) -> Option<ab_glyph::Glyph> {
) -> Option<Vec<ab_glyph::Glyph>> {
use ab_glyph::{Font, ScaleFont};
let face = harfrust::FontRef::new(font.font_data()).ok()?;
let mut buf = harfrust::UnicodeBuffer::new();
buf.add(ch, 0);
buf.set_direction(harfrust::Direction::TopToBottom);
buf.push_str(text);
buf.set_direction(if vertical { harfrust::Direction::TopToBottom } else { harfrust::Direction::LeftToRight });
buf.guess_segment_properties();
let shaped = data.shaper(&face).build().shape(buf, harfrust::ShapeOptions::new());
if shaped.glyph_infos().len() != 1 {
if shaped.glyph_infos().is_empty() {
return None;
}
let gid = u16::try_from(shaped.glyph_infos().first()?.glyph_id).ok().filter(|g| *g != 0)?;
let pos = shaped.glyph_positions().first()?;
let sf = font.as_scaled(px);
let (sx, sy) = (sf.h_scale_factor(), sf.v_scale_factor());
let height = -(pos.y_advance as f32) * sy;
let pen = ab_glyph::point(left + px / 2.0 + pos.x_offset as f32 * sx, top + (px - height) / 2.0 - pos.y_offset as f32 * sy);
Some(ab_glyph::GlyphId(gid).with_scale_and_position(px, pen))
let advance_x = shaped.glyph_positions().iter().map(|p| p.x_advance as f32 * sx).sum::<f32>();
let advance_y = shaped.glyph_positions().iter().map(|p| -(p.y_advance as f32) * sy).sum::<f32>();
let (mut x, mut y) = if vertical { (left + px / 2.0, top + (px - advance_y) / 2.0) } else { (left + (px - advance_x) / 2.0, top + sf.ascent()) };
let mut glyphs = Vec::new();
for (info, pos) in shaped.glyph_infos().iter().zip(shaped.glyph_positions()) {
let gid = u16::try_from(info.glyph_id).ok().filter(|g| *g != 0)?;
let pen = ab_glyph::point(x + pos.x_offset as f32 * sx, y - pos.y_offset as f32 * sy);
glyphs.push(ab_glyph::GlyphId(gid).with_scale_and_position(px, pen));
x += pos.x_advance as f32 * sx;
y -= pos.y_advance as f32 * sy;
}
Some(glyphs)
}
/// Draw `wm` onto `img` (straight alpha blending of the encoded values).
@@ -355,8 +363,13 @@ fn watermark_coverage(
let mut prev = None;
let columns = text.split('\n').count();
let mut column = 0usize;
for original in text.chars() {
if original == '\n' {
use unicode_segmentation::UnicodeSegmentation;
// Horizontal watermarks retain scalar layout and kerning; only vertical cells use graphemes.
let cells: Box<dyn Iterator<Item = &str>> =
if wm.vertical { Box::new(text.graphemes(true)) } else { Box::new(text.char_indices().filter_map(|(i, c)| text.get(i..i + c.len_utf8()))) };
for cell in cells {
let Some(original) = cell.chars().next() else { continue };
if cell == "\n" || (wm.vertical && cell == "\r\n") {
if wm.vertical {
column = column.saturating_add(1);
tw = tw.max(y);
@@ -369,17 +382,20 @@ fn watermark_coverage(
prev = None;
continue;
}
if wm.vertical
&& matches!(original as u32, 0x3000..=0x30FF | 0x3400..=0x9FFF | 0xFF01..=0xFF60)
&& let Some(face) = fonts.iter().position(|font| font.glyph_id(original).0 != 0)
&& let (Some(font), Some(Some(data))) = (fonts.get(face), vertical_data.get(face))
{
let cjk = matches!(original as u32, 0x3000..=0x30FF | 0x3400..=0x9FFF | 0xFF01..=0xFF60);
if wm.vertical && (cjk || cell.chars().count() > 1) {
let left = columns.saturating_sub(column.saturating_add(1)) as f32 * px;
if let Some(glyph) = vertical_watermark_glyph(font, data, original, px, left, y)
&& (glyph.id != font.glyph_id(original)
|| !matches!(original, '、' | '。' | 'ー' | '(' | ')' | '「' | '」' | '『' | '』' | '【' | '】'))
{
glyphs.push((face, glyph, None));
let shaped = fonts.iter().zip(&vertical_data).enumerate().find_map(|(face, (font, data))| {
let data = data.as_ref()?;
let cluster = watermark_cell_glyphs(font, data, cell, cjk, px, left, y)?;
let substituted = cluster.len() != 1 || cluster.first().is_some_and(|g| g.id != font.glyph_id(original));
if cjk && !substituted && matches!(original, '、' | '。' | 'ー' | '(' | ')' | '「' | '」' | '『' | '』' | '【' | '】') {
return None;
}
Some((face, cluster))
});
if let Some((face, cluster)) = shaped {
glyphs.extend(cluster.into_iter().map(|g| (face, g, None)));
y += px;
continue;
}
@@ -1706,7 +1722,7 @@ mod tests {
assert_eq!(font.glyph_id('︒').0, 0);
let data = harfrust::ShaperData::new(&harfrust::FontRef::new(entry.bytes).unwrap());
for ch in ['、', '。'] {
let glyph = vertical_watermark_glyph(&font, &data, ch, 100.0, 0.0, 0.0).unwrap();
let glyph = watermark_cell_glyphs(&font, &data, &ch.to_string(), true, 100.0, 0.0, 0.0).unwrap().remove(0);
assert_ne!(glyph.id, font.glyph_id(ch), "the font's vertical alternate for {ch}");
let rect = font.outline_glyph(glyph).unwrap().px_bounds();
assert!(rect.min.x > 50.0 && rect.max.y < 50.0, "{ch} in the upper right: {rect:?}");
@@ -1736,6 +1752,56 @@ mod tests {
}
}
#[test]
fn vertical_watermarks_keep_combining_marks_in_one_cell() {
let coverage = |text: &str, craft| {
let wm = Watermark {
text: text.into(),
vertical: true,
size: 0.1,
anchor: Anchor::Center,
inset: 0.0,
shadow: false,
opacity: 1.0,
..Default::default()
};
let mut pixels = vec![0.0f32; 400 * 300];
watermark_coverage(400, 300, &wm, craft, |x, y, k, _| pixels[y * 400 + x] = k);
assert!(pixels.iter().any(|&k| k > 0.0));
pixels
};
let compare = |a: &str, b: &str, craft| {
let (a_pixels, b_pixels) = (coverage(a, craft), coverage(b, craft));
let difference = a_pixels.iter().zip(&b_pixels).position(|(a, b)| a != b);
assert!(difference.is_none(), "{a:?} vs {b:?}: first differing pixel {difference:?}");
};
for (composed, decomposed) in [("éA", "e\u{301}A"), ("Å\nA", "A\u{30a}\nA"), ("A\nB", "A\r\nB"), ("日A", "日\u{e0100}A")] {
compare(composed, decomposed, &[]);
}
if crate::fonts::CRAFT_FONTS.iter().any(|f| f.family == "BIZ UDMincho") {
for (composed, decomposed) in [("が日", "か\u{3099}日"), ("ぱ\n日", "は\u{309a}\n日")] {
compare(composed, decomposed, crate::fonts::CRAFT_FONTS);
}
} else {
eprintln!("skipped Japanese comparisons: built without BIZ UDMincho from craft-fonts");
}
}
#[test]
fn upright_watermark_cell_retains_multiple_glyphs() {
use ab_glyph::{Font, ScaleFont};
let font = ab_glyph::FontRef::try_from_slice(WATERMARK_FONT).unwrap();
let data = harfrust::ShaperData::new(&harfrust::FontRef::new(WATERMARK_FONT).unwrap());
// There is no precomposed A with a combining long solidus overlay.
let glyphs = watermark_cell_glyphs(&font, &data, "A\u{338}", false, 30.0, 0.0, 0.0).unwrap();
assert_eq!(glyphs.len(), 2, "retain the base and the separately positioned combining glyph");
let baseline = font.as_scaled(30.0).ascent();
for glyph in glyphs {
assert!((glyph.position.y - baseline).abs() < 1.0, "no extra vertical cell for the mark");
assert!(font.outline_glyph(glyph).is_some(), "both glyphs have ink");
}
}
#[test]
fn japanese_watermarks_support_vertical_columns() {
use ab_glyph::Font;
+372
View File
@@ -0,0 +1,372 @@
//! Downloading a face model the user asked for (Settings ▸ Faces ▸ Download).
//!
//! The transfer is [`lightcraft_fetch`] (pure-Rust HTTPS with resume, size and SHA-256 checks), on a background thread,
//! so the UI never waits for the network. Only a model listed in [`lightcraft_faces::known::download`] can be fetched:
//! the address is a pinned constant, never something typed or passed in, and no token is ever sent with it. What arrives
//! matches the model's recorded size and SHA-256 or it is thrown away; it then waits in a staging folder until the model
//! is installed (`faces.models.install`, or by itself once the user has accepted the model's terms in the download).
//!
//! Failure is a message, never a panic: no network, a stalled or truncated transfer, a full disk, a file that is not the
//! one expected, and a panic inside the thread itself all end as [`State::Failed`].
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex, PoisonError};
use std::time::Duration;
use lightcraft_faces::known::Download;
/// The folder inside the models folder where a downloaded file waits to be installed. Its name starts with a dot, so it
/// can never be a model's own folder (ids start with a letter or digit).
pub const STAGING: &str = ".downloads";
/// Where a download stands.
#[derive(Clone, Debug, PartialEq)]
pub enum State {
/// Fetching (`bytes` of `total` so far); the file is checked as the last step.
Running {
bytes: u64,
total: u64,
},
/// Fetched and verified (its SHA-256 is `sha256`), waiting in the staging folder at `path` to be installed.
Done {
path: PathBuf,
sha256: String,
},
/// Installed and in use: nothing more to do but tell the user.
Installed,
Failed(String),
Cancelled,
}
struct Shared {
state: Mutex<State>,
cancel: AtomicBool,
}
impl Shared {
fn new(total: u64) -> Self {
Shared { state: Mutex::new(State::Running { bytes: 0, total }), cancel: AtomicBool::new(false) }
}
fn set(&self, s: State) {
*self.state.lock().unwrap_or_else(PoisonError::into_inner) = s;
}
fn get(&self) -> State {
self.state.lock().unwrap_or_else(PoisonError::into_inner).clone()
}
}
/// The downloads of one session, by model id.
#[derive(Default)]
pub struct Downloads {
jobs: HashMap<String, Arc<Shared>>,
}
impl Downloads {
/// Start fetching `spec` into `<models dir>/.downloads/`. Different models can download side by side; the same
/// one is started once.
pub fn start(&mut self, spec: Download, models_dir: &Path) -> Result<(), String> {
if self.jobs.get(spec.id.as_str()).is_some_and(|j| matches!(j.get(), State::Running { .. })) {
return Err("that model is already downloading".into());
}
let staging = models_dir.join(STAGING);
std::fs::create_dir_all(&staging).map_err(|e| format!("could not create the download folder: {e}"))?;
let shared = Arc::new(Shared::new(spec.size_bytes));
let worker = shared.clone();
let id = spec.id.clone();
std::thread::Builder::new()
.name("face-model-download".into())
.spawn(move || {
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| run(&spec, spec.base(), &staging, &worker)));
if outcome.is_err() {
worker.set(State::Failed("the download stopped unexpectedly".into()));
}
})
.map_err(|e| format!("could not start the download: {e}"))?;
self.jobs.insert(id, shared);
Ok(())
}
/// A download that has arrived and been checked, as if the transfer had just ended (tests of installing).
#[cfg(test)]
pub(crate) fn arrived(&mut self, id: &str, path: PathBuf, sha256: &str) {
let j = Shared::new(0);
j.set(State::Done { path, sha256: sha256.to_string() });
self.jobs.insert(id.to_string(), Arc::new(j));
}
/// Every download this session has started and not discarded, by model id.
pub fn snapshot(&self) -> Vec<(String, State)> {
let mut all: Vec<_> = self.jobs.iter().map(|(id, j)| (id.clone(), j.get())).filter(|(_, s)| *s != State::Cancelled).collect();
all.sort_by(|a, b| a.0.cmp(&b.0));
all
}
pub fn running(&self) -> bool {
self.jobs.values().any(|j| matches!(j.get(), State::Running { .. }))
}
/// Stop a running download, or discard a finished one (its staged file is deleted). `true` if there was one.
pub fn discard(&mut self, id: &str) -> bool {
let Some(job) = self.jobs.get(id) else { return false };
job.cancel.store(true, Ordering::Relaxed);
match job.get() {
// the thread notices, deletes the partial file and marks itself cancelled
State::Running { .. } => {}
State::Done { path, .. } => {
let _ = std::fs::remove_file(path);
self.jobs.remove(id);
}
State::Installed | State::Failed(_) | State::Cancelled => {
self.jobs.remove(id);
}
}
true
}
/// The staged file at `path` has been installed (its copy is the model now) or is no longer wanted.
pub fn staged_file_used(&mut self, path: &Path) {
self.jobs.retain(|_, j| !matches!(j.get(), State::Done { path: p, .. } if p == path));
}
/// Downloads that have arrived and been checked, ready to install: (model id, staged file, its SHA-256).
pub fn finished(&self) -> Vec<(String, PathBuf, String)> {
let mut out: Vec<_> = self
.jobs
.iter()
.filter_map(|(id, j)| match j.get() {
State::Done { path, sha256 } => Some((id.clone(), path, sha256)),
_ => None,
})
.collect();
out.sort_by(|a, b| a.0.cmp(&b.0));
out
}
/// Record how installing a finished download went (`Installed` or `Failed`).
pub fn set_outcome(&mut self, id: &str, state: State) {
match self.jobs.get(id) {
Some(j) => j.set(state),
None => {
let j = Shared::new(0);
j.set(state);
self.jobs.insert(id.to_string(), Arc::new(j));
}
}
}
}
impl Drop for Downloads {
/// Quitting while a download runs stops it instead of leaving it behind.
fn drop(&mut self) {
for job in self.jobs.values() {
job.cancel.store(true, Ordering::Relaxed);
}
let started = std::time::Instant::now();
while self.running() && started.elapsed() < Duration::from_secs(1) {
std::thread::sleep(Duration::from_millis(10));
}
}
}
/// What went wrong, in words the user can act on.
#[cfg(not(target_arch = "wasm32"))]
fn explain(e: &lightcraft_fetch::DownloadError) -> String {
use lightcraft_fetch::DownloadError;
match e {
DownloadError::AllMirrorsFailed { errors, .. } => {
let detail: String = errors.join("; ").chars().take(300).collect();
format!("could not download the model ({detail}): check the internet connection, or get the file from its page and add it yourself")
}
other => other.to_string(),
}
}
/// Fetch `spec` from `mirror` (the address without the file name) into `staging`, check it and leave it there.
#[cfg(not(target_arch = "wasm32"))]
fn fetch(spec: &Download, mirror: &str, staging: &Path, shared: &Shared) -> Result<(), lightcraft_fetch::DownloadError> {
let file = lightcraft_fetch::FileSpec { name: &spec.file_name, size: Some(spec.size_bytes), sha256: Some(&spec.sha256), max: spec.size_bytes };
lightcraft_fetch::download(&[file], &[mirror.to_string()], staging, &lightcraft_fetch::Options::default(), &shared.cancel, &mut |p| {
shared.set(State::Running { bytes: p.done.min(spec.size_bytes), total: spec.size_bytes });
})
}
#[cfg(target_arch = "wasm32")]
fn fetch(_: &Download, _: &str, _: &Path, _: &Shared) -> Result<(), String> {
Err("the web build downloads no models".into())
}
/// The whole job: reuse a verified earlier copy, else fetch, check and stage. Always ends in a final [`State`].
fn run(spec: &Download, mirror: &str, staging: &Path, shared: &Shared) {
let target = staging.join(&spec.file_name);
let part = staging.join(format!("{}.part", spec.file_name));
match fetch(spec, mirror, staging, shared) {
Ok(()) => shared.set(State::Done { path: target, sha256: spec.sha256.to_string() }),
#[cfg(not(target_arch = "wasm32"))]
Err(lightcraft_fetch::DownloadError::Cancelled) => {
let _ = std::fs::remove_file(&part);
shared.set(State::Cancelled);
}
#[cfg(not(target_arch = "wasm32"))]
Err(e) => {
let _ = std::fs::remove_file(&part);
shared.set(State::Failed(explain(&e)));
}
#[cfg(target_arch = "wasm32")]
Err(e) => {
let _ = std::fs::remove_file(&part);
shared.set(State::Failed(e));
}
}
}
#[cfg(all(test, not(target_arch = "wasm32")))]
mod tests {
use super::*;
use lightcraft_faces::hash::sha256_hex;
use std::io::{BufRead, BufReader, Write};
use std::net::TcpListener;
fn scratch(name: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("lc-face-dl-{name}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
fn spec_for(bytes: &[u8]) -> Download {
Download {
id: "test-model".into(),
url: "https://example.invalid/m/test-model.onnx".into(),
file_name: "test-model.onnx".into(),
size_bytes: bytes.len() as u64,
sha256: sha256_hex(bytes),
}
}
/// A local server that answers every request with `status` and `body` (the whole file, no ranges).
fn serve(status: &'static str, body: Vec<u8>) -> String {
let l = TcpListener::bind("127.0.0.1:0").unwrap();
let base = format!("http://{}/m", l.local_addr().unwrap());
std::thread::spawn(move || {
for mut s in l.incoming().flatten() {
let mut r = BufReader::new(s.try_clone().unwrap());
loop {
let mut line = String::new();
if r.read_line(&mut line).unwrap_or(0) == 0 || line.trim().is_empty() {
break;
}
}
let _ = s.write_all(format!("HTTP/1.1 {status}\r\nContent-Length: {}\r\nConnection: close\r\n\r\n", body.len()).as_bytes());
let _ = s.write_all(&body);
}
});
base
}
#[test]
fn fetches_checks_and_stages_a_file() {
let dir = scratch("ok");
let bytes: Vec<u8> = (0..200_000u32).map(|i| (i % 251) as u8).collect();
let (staging, spec) = (dir.join(STAGING), spec_for(&bytes));
std::fs::create_dir_all(&staging).unwrap();
let mirror = serve("200 OK", bytes.clone());
let shared = Shared::new(spec.size_bytes);
run(&spec, &mirror, &staging, &shared);
let State::Done { path, sha256 } = shared.get() else { panic!("{:?}", shared.get()) };
assert_eq!((std::fs::read(&path).unwrap(), sha256), (bytes, spec.sha256.to_string()));
assert!(!staging.join("test-model.onnx.part").exists());
// asked again with the copy still staged: no fetch needed (nothing listens there)
let again = Shared::new(spec.size_bytes);
run(&spec, "http://127.0.0.1:1/m", &staging, &again);
assert!(matches!(again.get(), State::Done { .. }), "{:?}", again.get());
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn a_file_that_does_not_match_is_thrown_away() {
let dir = scratch("bad");
let staging = dir.join(STAGING);
std::fs::create_dir_all(&staging).unwrap();
// the right size and the wrong content
let spec = spec_for(&[7u8; 5000]);
let shared = Shared::new(spec.size_bytes);
run(&spec, &serve("200 OK", vec![8u8; 5000]), &staging, &shared);
assert!(matches!(shared.get(), State::Failed(_)), "{:?}", shared.get());
// nothing is left behind: not the file, not a partial one
assert_eq!(std::fs::read_dir(&staging).unwrap().count(), 0);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn an_unreachable_or_refusing_server_is_a_message() {
let dir = scratch("down");
let staging = dir.join(STAGING);
std::fs::create_dir_all(&staging).unwrap();
let spec = spec_for(b"abc");
for mirror in ["http://127.0.0.1:1/m".to_string(), serve("404 Not Found", b"not found".to_vec())] {
let shared = Shared::new(3);
run(&spec, &mirror, &staging, &shared);
let State::Failed(why) = shared.get() else { panic!("{:?}", shared.get()) };
assert!(why.contains("internet connection"), "{why}");
}
assert_eq!(std::fs::read_dir(&staging).unwrap().count(), 0);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn a_cancelled_download_leaves_nothing() {
let dir = scratch("cancel");
let staging = dir.join(STAGING);
std::fs::create_dir_all(&staging).unwrap();
let spec = spec_for(b"abc");
let shared = Shared::new(3);
shared.cancel.store(true, Ordering::Relaxed);
run(&spec, &serve("200 OK", b"abc".to_vec()), &staging, &shared);
assert_eq!(shared.get(), State::Cancelled);
assert_eq!(std::fs::read_dir(&staging).unwrap().count(), 0);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn dropping_the_session_stops_a_running_download() {
let shared = Arc::new(Shared::new(10));
// a stand-in for the download thread: notices the cancel flag the way the downloader does
let worker = shared.clone();
let handle = std::thread::spawn(move || {
while !worker.cancel.load(Ordering::Relaxed) {
std::thread::sleep(Duration::from_millis(5));
}
worker.set(State::Cancelled);
});
let mut d = Downloads::default();
d.jobs.insert("m".into(), shared.clone());
assert!(d.running());
let started = std::time::Instant::now();
drop(d);
assert!(started.elapsed() < Duration::from_millis(900), "quitting must not wait for a transfer to finish");
assert_eq!(shared.get(), State::Cancelled);
handle.join().unwrap();
}
#[test]
fn discarding_a_finished_download_deletes_its_file() {
let dir = scratch("discard");
let file = dir.join("staged.onnx");
std::fs::write(&file, b"x").unwrap();
let mut d = Downloads::default();
let shared = Arc::new(Shared::new(1));
shared.set(State::Done { path: file.clone(), sha256: String::new() });
d.jobs.insert("m".into(), shared);
assert_eq!(d.snapshot().len(), 1);
assert!(!d.running());
assert!(d.discard("m"));
assert!(!file.exists());
assert!(d.snapshot().is_empty());
assert!(!d.discard("m"), "nothing left to discard");
let _ = std::fs::remove_dir_all(&dir);
}
}
+455
View File
@@ -0,0 +1,455 @@
//! The face index: one embedding per face region, kept in memory and cached beside the library.
//!
//! Embedding a face costs a model run (tens to hundreds of milliseconds), so each region is embedded once and
//! remembered, keyed by its photo and its box. The cache file (`face-embeddings-<model id>.bin` in the library folder,
//! one per model, so switching back to an earlier model does not start over) belongs to one model: its header names the
//! model and its file hash, and a file made by another model, an older format, or a damaged one is simply ignored and
//! rebuilt. Embeddings are kept out of the catalog on purpose: they are large, rebuildable, and not comparable between
//! models.
//!
//! Each face also keeps the box to cut its picture from (its "view"): the detector's box when the detector found the same
//! face, so every face is shown equally close however loosely or tightly its own region was drawn (a Lightroom box, an
//! older tool's box), else the region's own box.
//!
//! Layout (little-endian): `"LCFE"`, version `2`, tag length `u16`, the tag (`<model id>@<sha256>`), dimension `u32`,
//! then records of `photo u64`, four `i32` (the region's box in ten-thousandths), four `i32` (the view box likewise), and
//! `dim` `f32`.
use std::collections::HashMap;
use std::io::Write;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use lightcraft_geom::Rect;
const MAGIC: &[u8; 4] = b"LCFE";
const VERSION: u8 = 2;
/// Largest cache file we read.
const MAX_FILE: u64 = 1 << 30;
pub(crate) type Key = (u64, [i32; 4]);
/// One face's record: its embedding and its view box (quantized like a key).
type Entry = (Arc<[f32]>, [i32; 4]);
/// A quantized box as a rectangle again.
fn rect_of(k: [i32; 4]) -> Rect {
let f = |v: i32| f64::from(v) / 10_000.0;
Rect { x0: f(k[0]), y0: f(k[1]), x1: f(k[2]), y1: f(k[3]) }
}
/// A region's box as the integer key it is cached under.
pub(crate) fn region_key(r: &Rect) -> [i32; 4] {
let q = |v: f64| if v.is_finite() { (v.clamp(-1e3, 1e3) * 10_000.0).round() as i32 } else { 0 };
[q(r.x0), q(r.y0), q(r.x1), q(r.y1)]
}
#[derive(Default)]
pub(crate) struct Index {
/// `<model id>@<sha256>` of the model the embeddings belong to.
pub tag: String,
pub dim: usize,
entries: HashMap<Key, Entry>,
unsaved: Vec<(Key, Entry)>,
/// Faces that could not be embedded this session (so they are not retried on every call).
skipped: std::collections::HashSet<Key>,
/// The cache file on disk belongs to this model, so new records can be appended to it.
file_matches: bool,
pub path: Option<PathBuf>,
}
impl Index {
pub fn len(&self) -> usize {
self.entries.len()
}
pub fn get(&self, photo: u64, rect: &Rect) -> Option<&Arc<[f32]>> {
self.entries.get(&(photo, region_key(rect))).map(|e| &e.0)
}
/// The box to cut this face's picture from, once the scan has looked at it.
pub fn view(&self, photo: u64, rect: &Rect) -> Option<Rect> {
self.entries.get(&(photo, region_key(rect))).map(|e| rect_of(e.1))
}
#[cfg(test)]
pub fn contains(&self, photo: u64, rect: &Rect) -> bool {
self.entries.contains_key(&(photo, region_key(rect)))
}
/// Whether this face still has to be embedded.
pub fn needs(&self, photo: u64, rect: &Rect) -> bool {
let key = (photo, region_key(rect));
!self.entries.contains_key(&key) && !self.skipped.contains(&key)
}
pub fn skip(&mut self, photo: u64, rect: &Rect) {
self.skipped.insert((photo, region_key(rect)));
}
#[cfg(test)]
pub fn insert(&mut self, photo: u64, rect: &Rect, embedding: Vec<f32>) {
self.insert_with_view(photo, rect, embedding, rect);
}
/// Remember a face's embedding and the box to show it by.
pub fn insert_with_view(&mut self, photo: u64, rect: &Rect, embedding: Vec<f32>, view: &Rect) {
let entry: Entry = (embedding.into(), region_key(view));
let key = (photo, region_key(rect));
self.entries.insert(key, entry.clone());
self.unsaved.push((key, entry));
}
/// Whether there are embeddings not yet written to the cache file.
pub fn has_unsaved(&self) -> bool {
!self.unsaved.is_empty()
}
/// Start over for `tag` (`dim` numbers per face), loading what the cache file at `path` holds for it. What the
/// previous model's index had not written yet is written first, so switching models loses nothing.
pub fn reset(&mut self, tag: &str, dim: usize, path: Option<PathBuf>) {
let _ = self.save();
self.tag = tag.to_string();
self.dim = dim;
self.path = path;
self.entries.clear();
self.unsaved.clear();
self.skipped.clear();
self.file_matches = false;
if let Some(p) = self.path.clone() {
self.load(&p);
}
}
fn load(&mut self, path: &Path) {
let Ok(meta) = std::fs::metadata(path) else { return };
if meta.len() > MAX_FILE {
return;
}
let Ok(bytes) = std::fs::read(path) else { return };
let Some((tag_len, rest)) = bytes
.strip_prefix(MAGIC)
.and_then(|r| r.split_first())
.filter(|(v, _)| **v == VERSION)
.and_then(|(_, r)| r.split_at_checked(2))
.map(|(l, r)| (usize::from(u16::from_le_bytes([l[0], l[1]])), r))
else {
return;
};
let Some((tag, rest)) = rest.split_at_checked(tag_len) else { return };
let Some((dim, mut rest)) = rest.split_at_checked(4).map(|(d, r)| (u32::from_le_bytes([d[0], d[1], d[2], d[3]]) as usize, r)) else { return };
if tag != self.tag.as_bytes() || dim != self.dim || dim == 0 {
return;
}
self.file_matches = true;
let record = 8 + 16 + 16 + dim * 4;
while let Some((rec, tail)) = rest.split_at_checked(record) {
rest = tail;
let photo = u64::from_le_bytes([rec[0], rec[1], rec[2], rec[3], rec[4], rec[5], rec[6], rec[7]]);
let int = |at: usize| i32::from_le_bytes([rec[at], rec[at + 1], rec[at + 2], rec[at + 3]]);
let key = [int(8), int(12), int(16), int(20)];
let view = [int(24), int(28), int(32), int(36)];
let emb: Vec<f32> = rec[40..].as_chunks::<4>().0.iter().map(|c| f32::from_le_bytes(*c)).collect();
if emb.iter().all(|v| v.is_finite()) {
self.entries.insert((photo, key), (emb.into(), view));
}
}
}
fn header(&self) -> Vec<u8> {
let tag = self.tag.as_bytes();
let mut h = Vec::with_capacity(11 + tag.len());
h.extend_from_slice(MAGIC);
h.push(VERSION);
h.extend_from_slice(&u16::try_from(tag.len()).unwrap_or(0).to_le_bytes());
h.extend_from_slice(tag);
h.extend_from_slice(&(self.dim as u32).to_le_bytes());
h
}
fn record(&self, key: &Key, entry: &Entry) -> Vec<u8> {
let mut r = Vec::with_capacity(40 + entry.0.len() * 4);
r.extend_from_slice(&key.0.to_le_bytes());
key.1.iter().chain(entry.1.iter()).for_each(|k| r.extend_from_slice(&k.to_le_bytes()));
entry.0.iter().for_each(|v| r.extend_from_slice(&v.to_le_bytes()));
r
}
/// Write what was added since the last save: appended when the file already belongs to this model, else the
/// whole index is written fresh (atomically). A failure is returned, the records stay pending.
pub fn save(&mut self) -> std::io::Result<()> {
let Some(path) = self.path.clone() else {
self.unsaved.clear();
return Ok(());
};
if self.unsaved.is_empty() {
return Ok(());
}
if self.file_matches {
let mut f = std::fs::OpenOptions::new().append(true).open(&path)?;
let mut buf = Vec::new();
for (k, e) in &self.unsaved {
buf.extend(self.record(k, e));
}
f.write_all(&buf)?;
} else {
let mut buf = self.header();
for (k, e) in &self.entries {
buf.extend(self.record(k, e));
}
let part = path.with_extension("part");
std::fs::write(&part, &buf)?;
std::fs::rename(&part, &path)?;
self.file_matches = true;
}
self.unsaved.clear();
Ok(())
}
}
impl Drop for Index {
fn drop(&mut self) {
let _ = self.save();
}
}
const SCAN_MAGIC: &[u8; 4] = b"LCFS";
const SCAN_VERSION: u8 = 1;
/// The detector a search was made with: a record made by another one is ignored (a better detector may find more).
pub(crate) const SCAN_TAG: &str = "yunet-2023mar";
/// Largest record file we read.
const SCAN_MAX_FILE: u64 = 64 << 20;
/// The photos the detector has already searched for faces, so each is searched once, however many times LightCraft
/// starts. Kept beside the library (`face-scanned.bin`: a header naming the detector, then photo ids, appended to).
#[derive(Default)]
pub(crate) struct Scanned {
set: std::collections::HashSet<u64>,
unsaved: Vec<u64>,
path: Option<PathBuf>,
file_matches: bool,
}
impl Scanned {
pub fn path(&self) -> Option<&Path> {
self.path.as_deref()
}
pub fn contains(&self, photo: u64) -> bool {
self.set.contains(&photo)
}
pub fn len(&self) -> usize {
self.set.len()
}
pub fn insert(&mut self, photo: u64) {
if self.set.insert(photo) {
self.unsaved.push(photo);
}
}
pub fn has_unsaved(&self) -> bool {
!self.unsaved.is_empty()
}
/// Start over with the record at `path` (written first: what was not yet saved is not lost).
pub fn reset(&mut self, path: Option<PathBuf>) {
let _ = self.save();
self.set.clear();
self.unsaved.clear();
self.file_matches = false;
self.path = path;
if let Some(p) = self.path.clone() {
self.load(&p);
}
}
fn header() -> Vec<u8> {
let mut h = Vec::new();
h.extend_from_slice(SCAN_MAGIC);
h.push(SCAN_VERSION);
h.extend_from_slice(&(SCAN_TAG.len() as u16).to_le_bytes());
h.extend_from_slice(SCAN_TAG.as_bytes());
h
}
fn load(&mut self, path: &Path) {
let Ok(meta) = std::fs::metadata(path) else { return };
if meta.len() > SCAN_MAX_FILE {
return;
}
let Ok(bytes) = std::fs::read(path) else { return };
let head = Self::header();
let Some(rest) = bytes.strip_prefix(head.as_slice()) else { return };
self.file_matches = true;
// a half-written last id (a crash while appending) is ignored
self.set.extend(rest.as_chunks::<8>().0.iter().map(|c| u64::from_le_bytes(*c)));
}
/// Write what was added since the last save: appended when the file is this detector's, else the whole record
/// is written fresh (atomically). A failure is returned and the ids stay pending.
pub fn save(&mut self) -> std::io::Result<()> {
let Some(path) = self.path.clone() else {
self.unsaved.clear();
return Ok(());
};
if self.unsaved.is_empty() {
return Ok(());
}
if self.file_matches {
let mut f = std::fs::OpenOptions::new().append(true).open(&path)?;
let buf: Vec<u8> = self.unsaved.iter().flat_map(|id| id.to_le_bytes()).collect();
f.write_all(&buf)?;
} else {
let mut buf = Self::header();
buf.extend(self.set.iter().flat_map(|id| id.to_le_bytes()));
let part = path.with_extension("part");
std::fs::write(&part, &buf)?;
std::fs::rename(&part, &path)?;
self.file_matches = true;
}
self.unsaved.clear();
Ok(())
}
}
impl Drop for Scanned {
fn drop(&mut self) {
let _ = self.save();
}
}
#[cfg(test)]
mod tests {
use super::*;
fn rect(x: f64) -> Rect {
Rect { x0: x, y0: 0.1, x1: x + 0.2, y1: 0.4 }
}
fn temp(name: &str) -> PathBuf {
std::env::temp_dir().join(format!("lc-faceindex-{name}-{}.bin", std::process::id()))
}
#[test]
fn embeddings_survive_a_save_and_load_and_append() {
let path = temp("roundtrip");
let _ = std::fs::remove_file(&path);
let mut a = Index::default();
a.reset("m@abc", 3, Some(path.clone()));
a.insert(7, &rect(0.1), vec![1.0, 0.0, 0.5]);
a.insert(8, &rect(0.3), vec![0.0, 1.0, 0.25]);
a.save().unwrap();
a.insert(9, &rect(0.5), vec![0.5, 0.5, 0.5]);
a.save().unwrap(); // appended
let mut b = Index::default();
b.reset("m@abc", 3, Some(path.clone()));
assert_eq!(b.len(), 3);
assert_eq!(b.get(8, &rect(0.3)).map(|e| e.to_vec()), Some(vec![0.0, 1.0, 0.25]));
assert!(b.contains(9, &rect(0.5)) && !b.contains(9, &rect(0.6)) && !b.contains(1, &rect(0.5)));
let _ = std::fs::remove_file(path);
}
#[test]
fn another_model_a_damaged_file_or_a_torn_tail_are_handled() {
let path = temp("other");
let _ = std::fs::remove_file(&path);
let mut a = Index::default();
a.reset("m@abc", 3, Some(path.clone()));
a.insert(1, &rect(0.1), vec![1.0, 0.0, 0.0]);
a.insert(2, &rect(0.2), vec![0.0, 1.0, 0.0]);
a.save().unwrap();
// another model: nothing is loaded, and saving replaces the file for the new model
let mut other = Index::default();
other.reset("n@def", 4, Some(path.clone()));
assert_eq!(other.len(), 0);
other.insert(5, &rect(0.1), vec![1.0, 0.0, 0.0, 0.0]);
other.save().unwrap();
let mut again = Index::default();
again.reset("n@def", 4, Some(path.clone()));
assert_eq!(again.len(), 1);
let mut old = Index::default();
old.reset("m@abc", 3, Some(path.clone()));
assert_eq!(old.len(), 0, "the old model's records are gone");
// a torn last record loses only that record
let bytes = std::fs::read(&path).unwrap();
std::fs::write(&path, &bytes[..bytes.len() - 5]).unwrap();
let mut torn = Index::default();
torn.reset("n@def", 4, Some(path.clone()));
assert_eq!(torn.len(), 0);
// garbage of every kind is ignored
for junk in [Vec::new(), b"LCFE".to_vec(), b"LCFE\x01\xff\xff".to_vec(), vec![0xff; 200], b"LCFE\x02\x00\x00".to_vec()] {
std::fs::write(&path, &junk).unwrap();
let mut j = Index::default();
j.reset("n@def", 4, Some(path.clone()));
assert_eq!(j.len(), 0);
}
// and a missing file is fine
let _ = std::fs::remove_file(&path);
let mut m = Index::default();
m.reset("n@def", 4, Some(path.clone()));
assert_eq!(m.len(), 0);
}
#[test]
fn non_finite_boxes_and_vectors_do_not_break_the_keys() {
assert_eq!(region_key(&Rect { x0: f64::NAN, y0: f64::INFINITY, x1: 1e300, y1: -1e300 }), [0, 0, 10_000_000, -10_000_000]);
let path = temp("nan");
let _ = std::fs::remove_file(&path);
let mut a = Index::default();
a.reset("m@abc", 2, Some(path.clone()));
a.insert(1, &rect(0.1), vec![f32::NAN, 1.0]);
a.save().unwrap();
let mut b = Index::default();
b.reset("m@abc", 2, Some(path.clone()));
assert_eq!(b.len(), 0, "a non-finite vector is not loaded back");
let _ = std::fs::remove_file(path);
}
}
/// What the recognition commands keep between calls: the loaded model and the embeddings made with it.
#[derive(Default)]
pub(crate) struct FacesState {
/// `<model id>@<sha256>` and the loaded model.
pub embedder: Option<(String, Arc<lightcraft_faces::runtime::Embedder>)>,
pub index: Index,
/// When the models folder was last looked at (the per-frame pump trusts a look for a second).
pub checked: Option<std::time::Instant>,
/// Whether recognition was switched on when last looked up, and when.
pub enabled_seen: Option<(std::time::Instant, bool)>,
pub worker: Option<crate::faces_worker::Worker>,
/// After a failure to load the chosen model, the background pump leaves it alone until then.
pub retry_at: Option<std::time::Instant>,
/// Photos the worker is busy with.
pub in_flight: std::collections::HashSet<lightcraft_catalog::PhotoId>,
/// Photos still to hand to the worker, and the catalog revision this list was made at.
pub queue: Vec<lightcraft_catalog::PhotoId>,
pub queue_stamp: Option<u64>,
/// When the index was last written to its cache file by the background pump.
pub saved_at: Option<std::time::Instant>,
/// Bumped when another library is opened: what a worker was making for the old one is dropped.
pub epoch: u64,
/// Photos already searched for faces.
pub scanned: Scanned,
/// Photos that could not be searched this session (unreadable files): left alone until the next launch.
pub scan_failed: std::collections::HashSet<u64>,
}
impl FacesState {
/// Another library is being opened: what was learned about the old one is saved, and nothing of it is kept (photo
/// ids are only unique within a library). The loaded model stays; the index is reloaded for the new library.
pub fn library_changed(&mut self) {
self.index.reset("", 0, None);
self.scanned.reset(None);
self.scan_failed.clear();
self.embedder = None;
self.checked = None;
self.queue.clear();
self.queue_stamp = None;
self.in_flight.clear();
self.retry_at = None;
self.saved_at = None;
self.epoch = self.epoch.wrapping_add(1);
}
}
+483
View File
@@ -0,0 +1,483 @@
//! The heavy part of scanning faces, run off the session thread, on a few threads at once.
//!
//! For one photo: render it (upright, uncropped, default settings), find its faces with the YuNet detector (when installed)
//! so each region can be aligned by landmarks, cut each region out and run the recognition model. A photo that has
//! no face regions at all and has not been searched yet also gets its detections back, with their embeddings, so the
//! scan decodes each photo once. The session thread only prepares the job (which needs the catalog) and later ingests
//! the result, so the window never waits for a model.
use std::panic::{AssertUnwindSafe, catch_unwind};
use std::sync::atomic::{AtomicU8, AtomicUsize, Ordering};
use std::sync::mpsc::{Receiver, Sender, channel};
use std::sync::{Arc, Condvar, Mutex, PoisonError};
use std::time::Duration;
use lightcraft_catalog::PhotoId;
use lightcraft_faces::align::{Rgb, align_to_template, crop_box};
use lightcraft_faces::runtime::Embedder;
use lightcraft_faces::yunet::{Detector, Face, Options};
use lightcraft_geom::Rect;
use crate::media::{PreviewLoader, RenderJob};
/// Long edge of the picture faces are cut from.
pub(crate) const EDGE: usize = 2048;
/// A box from an XMP sidecar is cut with this much room around it (there are no landmarks to align by).
pub(crate) const BOX_PAD: f32 = 1.5;
/// How much a detected face must overlap a region to lend it its landmarks.
pub(crate) const MATCH_IOU: f32 = 0.3;
/// A detection must be this sure to become a face region (the bar `faces.detect` uses).
pub(crate) const REGION_SCORE: f32 = 0.6;
/// A raw's embedded preview is used for the scan only when its long edge is at least this (a thumbnail is too small to
/// find faces in; the raw itself is decoded then).
const MIN_PREVIEW_EDGE: usize = 1000;
/// Most photos ever worked on at once.
const MAX_WORKERS: usize = 24;
/// What one photo in progress holds: the file, its decoded picture and the copies the detector and recogniser read.
/// Measured: peak memory grew by about 180 MB for each extra worker.
const WORKER_BYTES: usize = 180 << 20;
/// Photos queued behind the ones running, so a worker that finishes never waits for the next frame to be fed.
pub(crate) const PREFETCH: usize = 2;
/// How hard the background scan may work right now, as the app judges from what the user is doing.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum Pace {
/// Start nothing new: the user is dragging, typing or scrolling.
Pause = 0,
/// One photo at a time: the user is around, or the window is minimized.
Light = 1,
/// Half the machine: the user is idle.
Normal = 2,
/// Most of the machine: the user is watching the progress.
Full = 3,
}
impl Pace {
fn from_u8(n: u8) -> Pace {
match n {
0 => Pace::Pause,
1 => Pace::Light,
3 => Pace::Full,
_ => Pace::Normal,
}
}
/// `pause`, `light`, `normal` or `full`; anything else (or nothing) is `normal`.
pub fn parse(name: Option<&str>) -> Pace {
match name {
Some("pause") => Pace::Pause,
Some("light") => Pace::Light,
Some("full") => Pace::Full,
_ => Pace::Normal,
}
}
}
/// How many photos are worked on at once at `pace` on a machine with `cores` threads (as the system reports them, which
/// respects container limits and affinity). `cap` is a limit the user set (`LIGHTCRAFT_FACE_THREADS`). Memory is not
/// decided here: decodes wait at the process-wide memory gate ([`crate::memory::work_gate`]), sized from the RAM.
pub(crate) fn workers_for(cores: usize, pace: Pace, cap: Option<usize>) -> usize {
let n = match pace {
Pace::Pause => return 0,
Pace::Light => 1,
Pace::Normal => cores / 2,
Pace::Full => cores * 4 / 5,
}
.clamp(1, MAX_WORKERS);
cap.map_or(n, |c| n.min(c.clamp(1, MAX_WORKERS)))
}
fn cores() -> usize {
std::thread::available_parallelism().map_or(1, |n| n.get())
}
fn cap_from_env() -> Option<usize> {
std::env::var("LIGHTCRAFT_FACE_THREADS").ok().and_then(|v| v.trim().parse::<usize>().ok())
}
/// The most photos the memory budget can hold in progress at once: half of it (the rest is the window's, the caches',
/// the renders'), a worker at a time. Raising the budget (`LIGHTCRAFT_MEMORY_MB`) raises this.
fn memory_workers() -> usize {
(crate::memory::budget() / 2 / WORKER_BYTES).max(1)
}
/// Photos to work on at once at `pace` on this machine: by the processor's threads and the pace, and never more than
/// the memory allows. A limit the user sets (`LIGHTCRAFT_FACE_THREADS`) replaces the memory one.
pub(crate) fn target_workers(pace: Pace) -> usize {
workers_for(cores(), pace, Some(cap_from_env().unwrap_or_else(memory_workers)))
}
/// Threads the worker pool starts with: enough for the fullest pace (the idle ones cost nothing).
pub(crate) fn max_workers() -> usize {
target_workers(Pace::Full)
}
/// Threads for the scan's own parallel work (decoding a picture, developing it) at `pace` on a machine with `cores` threads:
/// two when light, half the machine when normal, four fifths when watched. A photo's parallel parts are most of its cost,
/// so this, more than the number of photos at once, is how much of the processor the scan takes. Measured on a 32-thread
/// desktop (4 photos at once): 8 threads scanned 2.2 photos a second, 16 threads 5.8, 25 threads 6.4.
fn pool_threads_for(cores: usize, pace: Pace, cap: Option<usize>) -> usize {
let n = match pace {
Pace::Pause | Pace::Light => 2,
Pace::Normal => cores / 2,
Pace::Full => cores * 4 / 5,
}
.clamp(2, 32);
cap.map_or(n, |c| n.min(c.max(1)))
}
/// One photo's faces to embed (and, if `detect`, to find first).
pub(crate) struct Prepared {
/// `<model id>@<sha256>` the embeddings are for.
pub tag: String,
/// The session's face epoch when this was prepared: another library has been opened since if it differs.
pub epoch: u64,
pub id: PhotoId,
pub job: RenderJob,
/// A raw's embedded preview, tried before the render job (much faster than decoding the raw).
pub preview: Option<(String, PreviewLoader)>,
pub todo: Vec<Rect>,
/// The photo has no face regions and has not been searched: look for faces too.
pub detect: bool,
/// The YuNet detector, when the user has installed it: faces are aligned by its landmarks, and it finds the faces of
/// photos that have none. Without it a scan only embeds the faces the photos already have, cut by their boxes.
pub detector: Option<Arc<Detector>>,
}
/// The embeddings made for a [`Prepared`]; `None` for a face that could not be embedded.
pub(crate) struct Done {
pub tag: String,
pub epoch: u64,
pub id: PhotoId,
/// Each face's region, its embedding (`None` when it could not be made) and the box to show it by.
pub results: Vec<(Rect, Option<Vec<f32>>, Rect)>,
/// Faces were to be looked for in this photo.
pub detect_wanted: bool,
/// What the detector found (their embeddings are in `results`); `None` when the photo could not be searched.
pub found: Option<Vec<Rect>>,
}
fn overlap(a: &Rect, f: &Face) -> f32 {
let (w, h) = ((a.x1.min(f64::from(f.x1)) - a.x0.max(f64::from(f.x0))).max(0.0), (a.y1.min(f64::from(f.y1)) - a.y0.max(f64::from(f.y0))).max(0.0));
let inter = w * h;
let union = (a.x1 - a.x0) * (a.y1 - a.y0) + f64::from((f.x1 - f.x0) * (f.y1 - f.y0)) - inter;
if union > 0.0 { (inter / union) as f32 } else { 0.0 }
}
/// A detection as a region box: inside the photo and not a sliver; `None` for anything else (it is only ever data).
fn face_rect(f: &Face) -> Option<Rect> {
if ![f.x0, f.y0, f.x1, f.y1].iter().all(|v| v.is_finite()) {
return None;
}
let c = |v: f32| f64::from(v.clamp(0.0, 1.0));
let r = Rect { x0: c(f.x0), y0: c(f.y0), x1: c(f.x1), y1: c(f.y1) };
(r.x1 - r.x0 > 0.005 && r.y1 - r.y0 > 0.005).then_some(r)
}
/// Do the work for one photo. Never panics: whatever goes wrong, its faces come back as `None`.
pub(crate) fn process(p: Prepared, embedder: &Embedder) -> Done {
let Prepared { tag, epoch, id, job, preview, todo, detect, detector } = p;
let fallback = |todo: Vec<Rect>| todo.into_iter().map(|r| (r, None, r)).collect::<Vec<_>>();
let rects = todo.clone();
let run = catch_unwind(AssertUnwindSafe(|| {
let from_preview = preview.and_then(|(path, load)| load(&path, EDGE)).filter(|img| img.width.max(img.height) >= MIN_PREVIEW_EDGE);
let image = match from_preview {
Some(img) => img,
None => match job.run().rendered {
Ok(rendered) => rendered.image,
Err(_) => return (fallback(todo), None),
},
};
let (w, h) = (image.width, image.height);
let rgb: Vec<u8> = image.data.iter().flat_map(|px| [px[0], px[1], px[2]]).collect();
// the detector's landmarks, where it finds the same face: faces are aligned by them, not cut by a box
let detected = detector.as_deref().and_then(|d| d.detect(&rgb, w, h, &Options { score: 0.5, ..Options::default() }).ok());
let found = detected.clone().unwrap_or_default();
let img = Rgb { data: &rgb, width: w, height: h };
let (iw, ih) = embedder.input_size();
let scaled = |f: &Face| f.landmarks.map(|(x, y)| (x * w as f32, y * h as f32));
let mut results: Vec<(Rect, Option<Vec<f32>>, Rect)> = todo
.into_iter()
.map(|rect| {
let best = found.iter().map(|f| (overlap(&rect, f), f)).filter(|(o, _)| *o >= MATCH_IOU).max_by(|a, b| a.0.total_cmp(&b.0));
let crop = match best {
Some((_, f)) => align_to_template(&img, &scaled(f), iw, ih),
None => crop_box(
&img,
rect.x0 as f32 * w as f32,
rect.y0 as f32 * h as f32,
rect.x1 as f32 * w as f32,
rect.y1 as f32 * h as f32,
BOX_PAD,
iw,
ih,
),
};
let v = crop.and_then(|c| embedder.embed(&c).ok());
// shown by the detector's box where it found this face: every face then has the same tightness
let view = best.and_then(|(_, f)| face_rect(f)).unwrap_or(rect);
(rect, v, view)
})
.collect();
// a photo with no face regions: what the detector is sure of becomes its faces, embedded in the same pass
let new_faces = if detect {
detected.map(|faces| {
let mut rects = Vec::new();
for f in faces.iter().filter(|f| f.score >= REGION_SCORE) {
let Some(rect) = face_rect(f) else { continue };
let v = align_to_template(&img, &scaled(f), iw, ih).and_then(|c| embedder.embed(&c).ok());
results.push((rect, v, rect));
rects.push(rect);
}
rects
})
} else {
None
};
(results, new_faces)
}));
let (results, found) = run.unwrap_or_else(|_| (fallback(rects), None));
Done { tag, epoch, id, results, detect_wanted: detect, found }
}
type Job = (Arc<Embedder>, Prepared);
/// How many photos may be worked on at once, changed at any moment by the pace: a thread that has a job waits here until
/// fewer than that are running. A pace of zero holds every job where it is, with nothing decoded.
struct Limit {
allowed: AtomicUsize,
/// The pace the limit was last set for (a [`Pace`] as a number): the pool a photo's parallel work uses.
pace: AtomicU8,
running: Mutex<usize>,
wake: Condvar,
}
impl Limit {
fn new() -> Limit {
Limit { allowed: AtomicUsize::new(0), pace: AtomicU8::new(Pace::Normal as u8), running: Mutex::new(0), wake: Condvar::new() }
}
fn set(&self, n: usize, pace: Pace) {
self.pace.store(pace as u8, Ordering::Relaxed);
self.allowed.store(n, Ordering::Relaxed);
self.wake.notify_all();
}
fn pace(&self) -> Pace {
Pace::from_u8(self.pace.load(Ordering::Relaxed))
}
/// Wait for a place; the limit is looked at again whenever it changes and at least every 100 ms.
fn enter(&self) {
let mut running = self.running.lock().unwrap_or_else(PoisonError::into_inner);
while *running >= self.allowed.load(Ordering::Relaxed) {
running = self.wake.wait_timeout(running, Duration::from_millis(100)).unwrap_or_else(PoisonError::into_inner).0;
}
*running += 1;
}
fn leave(&self) {
let mut running = self.running.lock().unwrap_or_else(PoisonError::into_inner);
*running = running.saturating_sub(1);
drop(running);
self.wake.notify_one();
}
}
/// The threads a photo's parallel work (decoding, developing a picture) runs on at `pace`: one pool for each of three sizes
/// ([`pool_threads_for`]), made when first needed, whose idle threads cost nothing. rayon's global pool serves the
/// interactive work (the loupe, exports) and has no priorities, so scan work queued there would make a slider drag wait
/// behind it; here a light scan can use only two threads however large the machine is.
fn scan_pool(pace: Pace) -> Option<&'static rayon::ThreadPool> {
static LIGHT: std::sync::OnceLock<Option<rayon::ThreadPool>> = std::sync::OnceLock::new();
static NORMAL: std::sync::OnceLock<Option<rayon::ThreadPool>> = std::sync::OnceLock::new();
static FULL: std::sync::OnceLock<Option<rayon::ThreadPool>> = std::sync::OnceLock::new();
let (cell, pace) = match pace {
Pace::Pause | Pace::Light => (&LIGHT, Pace::Light),
Pace::Normal => (&NORMAL, Pace::Normal),
Pace::Full => (&FULL, Pace::Full),
};
cell.get_or_init(|| {
rayon::ThreadPoolBuilder::new()
.num_threads(pool_threads_for(cores(), pace, cap_from_env()))
.thread_name(move |i| format!("lightcraft-faces-{}-{i}", pace as u8))
.build()
.ok()
})
.as_ref()
}
/// Run one photo on the pool for the pace, so its parallel parts stay there.
///
/// It must NOT wait at the memory gate ([`crate::memory::work_gate`], which background decodes do): a job running on a
/// pool thread that waits inside a nested rayon call can pick up another job to run on top of itself, and if that one waits
/// for memory the first holds, neither can finish (seen as a few photos that never come back). The memory a scan may use is
/// instead limited by how many photos run at once ([`memory_workers`]); its decodes count as interactive ones at the gate,
/// which never wait.
fn process_in_background(prepared: Prepared, embedder: &Embedder, pace: Pace) -> Done {
let work = || process(prepared, embedder);
match scan_pool(pace) {
Some(pool) => pool.install(work),
None => work(),
}
}
/// Threads that run [`process`] for the jobs they are given, each on one photo at a time.
pub(crate) struct Worker {
jobs: Sender<Job>,
done: Receiver<Done>,
limit: Arc<Limit>,
}
impl Worker {
pub fn start(threads: usize) -> Option<Worker> {
let (jobs, job_rx) = channel::<Job>();
let (done_tx, done) = channel::<Done>();
let job_rx = Arc::new(Mutex::new(job_rx));
let limit = Arc::new(Limit::new());
let mut started = 0;
for i in 0..threads.clamp(1, MAX_WORKERS) {
let (rx, tx, limit) = (job_rx.clone(), done_tx.clone(), limit.clone());
let spawned = std::thread::Builder::new().name(format!("lightcraft-faces-{i}")).spawn(move || {
loop {
// the lock is held only while waiting for a job, never while working on one; the threads end when
// the session (the only sender) is dropped
let next = rx.lock().unwrap_or_else(PoisonError::into_inner).recv();
let Ok((embedder, prepared)) = next else { break };
limit.enter();
let done = process_in_background(prepared, &embedder, limit.pace());
limit.leave();
if tx.send(done).is_err() {
break;
}
}
});
if spawned.is_ok() {
started += 1;
}
}
(started > 0).then_some(Worker { jobs, done, limit })
}
/// How many photos may be worked on at once from now on (what is running is not interrupted).
pub fn allow(&self, n: usize, pace: Pace) {
self.limit.set(n, pace);
}
pub fn submit(&self, embedder: Arc<Embedder>, prepared: Prepared) -> bool {
self.jobs.send((embedder, prepared)).is_ok()
}
/// Everything finished since the last call.
pub fn finished(&self) -> Vec<Done> {
self.done.try_iter().collect()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_pace_sets_how_much_of_the_machine_the_scan_may_use() {
use Pace::*;
// a 32-thread desktop: nothing while the user works, one photo around them, half when idle, most when watched
assert_eq!([Pause, Light, Normal, Full].map(|p| workers_for(32, p, None)), [0, 1, 16, 24]);
assert_eq!([Pause, Light, Normal, Full].map(|p| workers_for(8, p, None)), [0, 1, 4, 6]);
assert_eq!([Pause, Light, Normal, Full].map(|p| workers_for(4, p, None)), [0, 1, 2, 3]);
// small machines still get one worker (except while paused)
assert_eq!([Pause, Light, Normal, Full].map(|p| workers_for(1, p, None)), [0, 1, 1, 1]);
assert_eq!(workers_for(0, Full, None), 1);
assert_eq!(workers_for(10_000, Full, None), MAX_WORKERS);
// a limit set by the user caps every pace, and is itself kept sane
assert_eq!([Light, Normal, Full].map(|p| workers_for(32, p, Some(3))), [1, 3, 3]);
assert_eq!((workers_for(32, Full, Some(0)), workers_for(32, Pause, Some(5))), (1, 0));
assert_eq!(workers_for(32, Full, Some(usize::MAX)), 24);
}
#[test]
fn the_limit_holds_threads_to_the_pace_and_can_change_at_any_moment() {
let limit = Arc::new(Limit::new());
let (now, peak) = (Arc::new(AtomicUsize::new(0)), Arc::new(AtomicUsize::new(0)));
let started = Arc::new(AtomicUsize::new(0));
let threads: Vec<_> = (0..6)
.map(|_| {
let (limit, now, peak, started) = (limit.clone(), now.clone(), peak.clone(), started.clone());
std::thread::spawn(move || {
limit.enter();
started.fetch_add(1, Ordering::SeqCst);
let n = now.fetch_add(1, Ordering::SeqCst) + 1;
peak.fetch_max(n, Ordering::SeqCst);
std::thread::sleep(Duration::from_millis(40));
now.fetch_sub(1, Ordering::SeqCst);
limit.leave();
})
})
.collect();
// paused: nothing starts
std::thread::sleep(Duration::from_millis(150));
assert_eq!(started.load(Ordering::SeqCst), 0);
// two at a time, never more
limit.set(2, Pace::Normal);
std::thread::sleep(Duration::from_millis(100));
assert!(started.load(Ordering::SeqCst) >= 2);
// raised, the rest go; every thread finishes
limit.set(6, Pace::Full);
for t in threads {
t.join().unwrap();
}
assert_eq!(started.load(Ordering::SeqCst), 6);
assert!(peak.load(Ordering::SeqCst) >= 2, "it did run in parallel");
assert_eq!(*limit.running.lock().unwrap(), 0, "every place was given back");
}
#[test]
fn memory_limits_the_workers_and_a_users_limit_replaces_it() {
// the default budget (a quarter of the RAM, at most 1.5 GiB) holds a few photos in progress, not dozens
assert!((1..=8).contains(&memory_workers()), "{}", memory_workers());
assert!(target_workers(Pace::Full) >= 1);
assert_eq!(target_workers(Pace::Pause), 0);
}
#[test]
fn pace_names_are_forgiving() {
assert_eq!(["pause", "light", "normal", "full"].map(|n| Pace::parse(Some(n))), [Pace::Pause, Pace::Light, Pace::Normal, Pace::Full]);
assert_eq!(
(Pace::parse(None), Pace::parse(Some("")), Pace::parse(Some("FULL")), Pace::parse(Some("turbo"))),
(Pace::Normal, Pace::Normal, Pace::Normal, Pace::Normal)
);
}
#[test]
fn the_scans_parallel_work_gets_a_pool_the_size_of_the_pace() {
use Pace::*;
assert_eq!([Pause, Light, Normal, Full].map(|p| pool_threads_for(32, p, None)), [2, 2, 16, 25]);
assert_eq!([Light, Normal, Full].map(|p| pool_threads_for(8, p, None)), [2, 4, 6]);
assert_eq!([Light, Normal, Full].map(|p| pool_threads_for(2, p, None)), [2, 2, 2]);
assert_eq!(pool_threads_for(10_000, Full, None), 32);
assert_eq!([Light, Normal, Full].map(|p| pool_threads_for(32, p, Some(3))), [2, 3, 3]);
// the pools exist and are the sizes asked for
for pace in [Light, Normal, Full] {
let pool = scan_pool(pace).expect("a pool can be built");
assert_eq!(pool.install(rayon::current_num_threads), pool_threads_for(cores(), pace, cap_from_env()));
}
// a paused scan has the light pool; the pace is remembered as a number and read back
assert_eq!(scan_pool(Pause).map(|p| p as *const _), scan_pool(Light).map(|p| p as *const _));
assert_eq!([0, 1, 2, 3, 9].map(Pace::from_u8), [Pause, Light, Normal, Full, Normal]);
}
#[test]
fn detections_become_boxes_only_when_they_make_sense() {
let f = |x0, y0, x1, y1| Face { x0, y0, x1, y1, score: 0.9, landmarks: [(0.0, 0.0); 5] };
assert!(face_rect(&f(0.2, 0.2, 0.4, 0.5)).is_some());
// a box that spills over the edge is brought back inside
let r = face_rect(&f(-0.1, 0.1, 0.3, 1.2)).unwrap();
assert_eq!((r.x0, r.y1), (0.0, 1.0));
// slivers, inverted boxes and non-numbers are dropped, never turned into regions
for bad in [f(0.2, 0.2, 0.2, 0.5), f(0.4, 0.2, 0.2, 0.5), f(f32::NAN, 0.2, 0.4, 0.5), f(0.2, 0.2, f32::INFINITY, 0.5), f(1.5, 0.2, 2.0, 0.5)]
{
assert!(face_rect(&bad).is_none(), "{bad:?}");
}
}
}
+380 -24
View File
@@ -92,16 +92,18 @@ fn ext_upper(name: &str) -> String {
std::path::Path::new(name).extension().map(|e| e.to_string_lossy().to_uppercase()).unwrap_or_default()
}
/// Probe a file's bytes: kind, dimensions (oriented), metadata. Raws are described from their
/// headers ([`lightcraft_raw::probe_info`]): no pixel data is decompressed.
/// Probe a file's bytes: kind, dimensions (oriented), metadata. No pixel data is decompressed:
/// raws are described from their headers ([`lightcraft_raw::probe_info`]), other images too
/// ([`lightcraft_codecs::read_header`], which refuses truncated files but can't see damage inside
/// compressed data that is all there: such a file imports and shows as unreadable when rendered).
pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result<ProbeInfo, String> {
let content_hash = Some(lightcraft_preview::hash_bytes(bytes).to_string());
let m = lightcraft_meta::extract(bytes);
let (meta, captured) = meta_of(&m);
if lightcraft_raw::probe(bytes).is_some() {
let raw = match lightcraft_raw::probe_info(bytes) {
let raw = match lightcraft_raw::probe_info(bytes).map_err(|e| preview_reason(bytes, e)) {
Ok(r) => r,
Err(lightcraft_raw::RawError::Unsupported(why)) => {
Err(Ok(why)) => {
// a raw variant we can't decode yet: describe it from its embedded preview
let (w, h) = embedded_preview_size(bytes).ok_or(format!("unsupported raw ({why}) without an embedded preview"))?;
return Ok(ProbeInfo {
@@ -119,7 +121,7 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result<ProbeInfo, String> {
..Default::default()
});
}
Err(e) => return Err(e.to_string()),
Err(Err(e)) => return Err(e),
};
let (mut w, mut h) = (raw.crop.width.max(1) as u32, raw.crop.height.max(1) as u32);
if w <= 1 || h <= 1 {
@@ -152,9 +154,10 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result<ProbeInfo, String> {
if !fmt.can_decode() {
return Err(format!("{fmt:?} files are not supported yet"));
}
let d = lightcraft_codecs::decode(bytes, lightcraft_codecs::DecodeOptions::fit(64, 64)).map_err(|e| e.to_string())?;
let o = Orientation::from_exif(d.orientation);
let (mut w, mut h) = (d.source_width, d.source_height);
// headers only (issue #367: decoding the pixels was nearly all of an import's CPU time)
let header = lightcraft_codecs::read_header(bytes).map_err(|e| e.to_string())?;
let o = Orientation::from_exif(header.orientation);
let (mut w, mut h) = (header.width, header.height);
if o.swaps_axes() {
std::mem::swap(&mut w, &mut h);
}
@@ -179,6 +182,21 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result<ProbeInfo, String> {
})
}
/// Why a raw that failed to decode should show its embedded preview instead (`Ok`), or the error
/// to report (`Err`). Variants we can't decode yet always fall back. So does any CR3 error: the CRX
/// decoder is verified on few bodies, and every CR3 opened from its embedded JPEG before it existed.
/// So does any other failure of a recognised raw (damaged or oversized raw data, unreadable TIFF
/// structure): its preview may still be intact. Only a file that is not a raw reports an error.
fn preview_reason(bytes: &[u8], e: lightcraft_raw::RawError) -> Result<String, String> {
use lightcraft_raw::RawError;
match e {
RawError::Unsupported(why) => Ok(why),
e if lightcraft_raw::probe(bytes) == Some(lightcraft_raw::RawFormat::Cr3) => Ok(format!("CR3 {e}")),
RawError::NotRaw => Err(RawError::NotRaw.to_string()),
e => Ok(e.to_string()),
}
}
/// Sensor clip level (normalised) for highlight reconstruction.
const HIGHLIGHT_CLIP: f32 = 0.99;
@@ -213,13 +231,13 @@ pub fn load_vec(bytes: Vec<u8>, max_edge: usize) -> Result<(Rgb32f, SourceInfo),
fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result<(Rgb32f, SourceInfo), String> {
if lightcraft_raw::probe(&bytes).is_some() {
let mut raw = match lightcraft_raw::decode(&bytes) {
let mut raw = match lightcraft_raw::decode(&bytes).map_err(|e| preview_reason(&bytes, e)) {
Ok(r) => r,
Err(lightcraft_raw::RawError::Unsupported(why)) => {
Err(Ok(why)) => {
// show the camera's embedded JPEG (rendered, not raw) until the variant is supported
return load_embedded_preview(&bytes, max_edge).ok_or(format!("unsupported raw ({why}) without an embedded preview"));
}
Err(e) => return Err(e.to_string()),
Err(Err(e)) => return Err(e),
};
// Embedded lens corrections are applied by the pipeline ("Enable Profile Corrections"), not baked in
// (removed before the camera look's binned sensor proxy too, which needs an empty `OpcodeList3`).
@@ -227,7 +245,8 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result<
raw.opcodes.list3.retain(|op| !op.is_lens_correction());
let xy = lightcraft_raw::color::as_shot_white_xy(&raw);
let t = lightcraft_raw::color::camera_transform(&raw, xy);
let camera_look = crate::camera_preview::fit_preview(&raw, &bytes, &t);
// the source's segmentation mattes (DNG semantic masks), read while the preview is fitted
let (camera_look, mattes) = rayon::join(|| crate::camera_preview::fit_preview(&raw, &bytes, &t), || dng_mattes(&bytes, &raw));
drop(bytes);
// Previews and thumbnails bin the mosaic straight to (about) the size they need; only
// larger levels (exports, 1:1) demosaic the whole sensor.
@@ -288,17 +307,62 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result<
}
let (temp, tint) = xy_to_temp_tint(xy);
let relative = crate::camera_preview::file_local_look(raw.format) && t.matrix_is_fallback;
// White balance re-evaluates the file's own colour model (when it has one and no
// file-local look matrix sits on top of it)
let camera_color = (!t.matrix_is_fallback && camera_look.is_none()).then(|| {
let tags = lightcraft_raw::ColorData { profile: Default::default(), ..raw.color.clone() };
Arc::new(lightcraft_pipeline::CameraColor { tags, developed_for: xy })
});
let camera_tone = camera_look.as_ref().map(|p| p.tone).or_else(|| raw.color.profile.tone_curve.as_ref().and_then(dng_tone_curve));
let (temp, tint) = if relative { (6500.0, 0.0) } else { (temp.round(), tint.round()) };
return Ok((img, SourceInfo { raw: true, as_shot_temp: temp, as_shot_tint: tint, lens, relative_wb: relative, camera_tone }));
return Ok((
img,
SourceInfo { raw: true, as_shot_temp: temp, as_shot_tint: tint, lens, relative_wb: relative, camera_color, camera_tone, mattes },
));
}
let d = lightcraft_codecs::decode(&bytes, lightcraft_codecs::DecodeOptions::fit(max_edge as u32, max_edge as u32)).map_err(|e| e.to_string())?;
let d = lightcraft_codecs::decode(&bytes, fit_box(max_edge)).map_err(|e| e.to_string())?;
drop(bytes);
let img = d.to_working();
let img = if img.width.max(img.height) > max_edge { fit(&img, max_edge, max_edge, Filter::Mitchell) } else { img };
Ok((img.into_oriented(Orientation::from_exif(d.orientation)), SourceInfo::default()))
}
/// The semantic masks of a DNG that AI masks understand, over the developed image (default crop,
/// oriented like it). Person mattes without a confidently selected pixel are left out (the
/// Subject mask then falls back to its heuristic); a sky matte counts even when empty (no sky).
fn dng_mattes(bytes: &[u8], raw: &lightcraft_raw::RawImage) -> Option<Arc<lightcraft_pipeline::masks::Mattes>> {
use lightcraft_pipeline::masks::{MatteKind, Mattes};
if raw.format != lightcraft_raw::RawFormat::Dng {
return None;
}
let mut mattes = Mattes::default();
for m in lightcraft_raw::semantic_masks(bytes) {
let Some(kind) = matte_kind(&m.name) else { continue };
let Some(img) = m.developed(raw.active_area, raw.crop) else { continue };
if kind != MatteKind::Sky && !img.data.iter().any(|&v| v >= 128) {
continue;
}
mattes.push(kind, img.into_oriented(raw.orientation));
}
(!mattes.is_empty()).then(|| Arc::new(mattes))
}
/// What a semantic mask selects, by its `SemanticName`: Apple's (iPhone ProRAW) are named after
/// their auxiliary image types, `urn:com:apple:photo:<year>:aux:<type>`. Other names: `None`.
fn matte_kind(name: &str) -> Option<lightcraft_pipeline::masks::MatteKind> {
use lightcraft_pipeline::masks::MatteKind::*;
let kind = name.strip_prefix("urn:com:apple:photo:")?.rsplit(':').next()?;
Some(match kind {
"semanticskymatte" => Sky,
"semanticskinmatte" => Skin,
"semantichairmatte" => Hair,
"semanticteethmatte" => Teeth,
"semanticglassesmatte" => Glasses,
"portraiteffectsmatte" => Person,
_ => return None,
})
}
/// A DNG `ProfileToneCurve` (linear in, linear out, 1.0 = white after exposure compensation) as the
/// finish stage's camera tone curve: 32 knots, log-spaced over 12 stops below white; above white
/// the camera tone's shoulder continues it.
@@ -318,10 +382,24 @@ fn preview_orientation(raw_bytes: &[u8], jpeg_orientation: u16) -> Orientation {
lightcraft_meta::extract(raw_bytes).orientation.unwrap_or(Orientation::Normal)
}
/// What stands in for a raw file we can't decode: its largest embedded JPEG, else, for a TIFF-based
/// raw, the file's own first image when the TIFF reader can show it (a reduced RGB copy: the only
/// preview some containers carry). Returns image bytes for [`lightcraft_codecs::decode`].
fn stand_in_image(bytes: &[u8]) -> Option<std::borrow::Cow<'_, [u8]>> {
if let Some(jpeg) = lightcraft_raw::embedded_preview(bytes) {
return Some(std::borrow::Cow::Owned(jpeg));
}
let undecodable = lightcraft_raw::probe(bytes).is_some_and(|f| !f.is_supported());
(undecodable && lightcraft_codecs::sniff(bytes).is_some_and(|f| f == lightcraft_codecs::Format::Tiff))
.then_some(std::borrow::Cow::Borrowed(bytes))
}
/// Shared colour interpretation for quick previews, unsupported RAW fallback and camera-look fitting.
pub(crate) fn decode_raw_preview(bytes: &[u8], opts: lightcraft_codecs::DecodeOptions) -> Option<lightcraft_codecs::Decoded> {
let jpeg = lightcraft_raw::embedded_preview(bytes)?;
match lightcraft_raw::embedded_preview_color_space(bytes) {
let jpeg = stand_in_image(bytes)?;
// the colour space is a property of the JPEG; a TIFF stand-in is read by the TIFF reader
let is_jpeg = matches!(jpeg, std::borrow::Cow::Owned(_));
match lightcraft_raw::embedded_preview_color_space(bytes).filter(|_| is_jpeg) {
Some(space) => {
let space = match space {
lightcraft_raw::PreviewColorSpace::Srgb => lightcraft_codecs::NamedSpace::Srgb,
@@ -335,7 +413,7 @@ pub(crate) fn decode_raw_preview(bytes: &[u8], opts: lightcraft_codecs::DecodeOp
/// Oriented size of the embedded preview of a raw file.
fn embedded_preview_size(bytes: &[u8]) -> Option<(u32, u32)> {
let jpeg = lightcraft_raw::embedded_preview(bytes)?;
let jpeg = stand_in_image(bytes)?;
let d = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions::fit(64, 64)).ok()?;
let (mut w, mut h) = (d.source_width, d.source_height);
if preview_orientation(bytes, d.orientation).swaps_axes() {
@@ -346,7 +424,7 @@ fn embedded_preview_size(bytes: &[u8]) -> Option<(u32, u32)> {
/// The embedded preview of a raw file as a working-space image no larger than `max_edge`, oriented.
pub fn load_embedded_preview(bytes: &[u8], max_edge: usize) -> Option<(Rgb32f, SourceInfo)> {
let d = decode_raw_preview(bytes, lightcraft_codecs::DecodeOptions::fit(max_edge as u32, max_edge as u32))?;
let d = decode_raw_preview(bytes, fit_box(max_edge))?;
let img = d.to_working();
let img = if img.width.max(img.height) > max_edge { fit(&img, max_edge, max_edge, Filter::Mitchell) } else { img };
Some((img.oriented(preview_orientation(bytes, d.orientation)), SourceInfo::default()))
@@ -355,7 +433,7 @@ pub fn load_embedded_preview(bytes: &[u8], max_edge: usize) -> Option<(Rgb32f, S
/// The embedded preview of a raw file for display (sRGB, oriented, no larger than `max_edge`): the
/// loupe and grid show it until the raw itself has been developed ([`crate::media::QuickJob`]).
pub fn embedded_preview_srgb(bytes: &[u8], max_edge: usize) -> Option<lightcraft_raster::Rgba8> {
let mut d = decode_raw_preview(bytes, lightcraft_codecs::DecodeOptions::fit(max_edge as u32, max_edge as u32))?;
let mut d = decode_raw_preview(bytes, fit_box(max_edge))?;
if d.image.width.max(d.image.height) > max_edge {
d.image = fit(&d.image, max_edge, max_edge, Filter::Box);
d.alpha = None;
@@ -364,9 +442,24 @@ pub fn embedded_preview_srgb(bytes: &[u8], max_edge: usize) -> Option<lightcraft
Some(d.to_srgb8().oriented(o))
}
/// Decode options fitting into a `max_edge` square; `usize::MAX` (a full-size load) or any edge past
/// `u32::MAX` asks for the largest box rather than wrapping to a small one.
fn fit_box(max_edge: usize) -> lightcraft_codecs::DecodeOptions {
let e = u32::try_from(max_edge).unwrap_or(u32::MAX);
lightcraft_codecs::DecodeOptions::fit(e, e)
}
/// Filesystem-backed embedded-preview hook (native).
///
/// Reads the whole file to find the preview, so it holds the memory gate for twice the file's size (the file and the
/// decoded preview): background work (the face scan) waits for room, interactive work is counted and never waits.
pub fn fs_preview_loader() -> PreviewLoader {
Arc::new(|path: &str, max_edge: usize| embedded_preview_srgb(&std::fs::read(path).ok()?, max_edge))
Arc::new(|path: &str, max_edge: usize| {
let len = std::fs::metadata(path).map(|m| usize::try_from(m.len()).unwrap_or(usize::MAX)).unwrap_or(0);
let gate = crate::memory::work_gate();
let _permit = if crate::memory::is_background() { gate.acquire(len.saturating_mul(2)) } else { gate.acquire_urgent(len.saturating_mul(2)) };
embedded_preview_srgb(&std::fs::read(path).ok()?, max_edge)
})
}
/// Filesystem-backed hooks (native). On the web the host installs bytes-based hooks instead.
@@ -378,7 +471,7 @@ pub fn fs_preview_loader() -> PreviewLoader {
/// yields to them.
pub fn fs_hooks() -> (FileLoader, FileProbe) {
let loader: FileLoader = Arc::new(|path: &str, max_edge: usize| {
let len = std::fs::metadata(path).map(|m| m.len() as usize).unwrap_or(0);
let len = std::fs::metadata(path).map(|m| usize::try_from(m.len()).unwrap_or(usize::MAX)).unwrap_or(0);
let weight = len * if max_edge <= crate::media::SourceLevel::Thumb.max_edge() { 3 } else { 6 };
let gate = crate::memory::work_gate();
let _permit = if crate::memory::is_background() { gate.acquire(weight) } else { gate.acquire_urgent(weight) };
@@ -389,7 +482,7 @@ pub fn fs_hooks() -> (FileLoader, FileProbe) {
r
});
let probe: FileProbe = Arc::new(|path: &str| {
let len = std::fs::metadata(path).map(|m| m.len() as usize).unwrap_or(0);
let len = std::fs::metadata(path).map(|m| usize::try_from(m.len()).unwrap_or(usize::MAX)).unwrap_or(0);
let _permit = crate::memory::work_gate().acquire(len);
let bytes = std::fs::read(path).map_err(|e| format!("{path}: {e}"))?;
probe_bytes(path, &bytes)
@@ -413,6 +506,37 @@ mod tests {
use super::*;
use lightcraft_codecs::{ChromaSubsampling, EncodeImage, EncodeMeta, Samples, encode_jpeg};
/// Issue #367: probes read headers, not pixels, and report what the decode-based probe did:
/// oriented dimensions (EXIF orientation 6 swaps them), metadata, the error for a truncated
/// file or one that isn't an image.
#[test]
fn image_probe_reads_headers_like_the_decode_did() {
let (w, h) = (300u32, 200u32);
let rgb: Vec<u8> = (0..w * h * 3).map(|i| (i * 7 % 251) as u8).collect();
let img = EncodeImage::new(w, h, 3, Samples::U8(&rgb));
for o in [None, Some(1), Some(6), Some(3), Some(8)] {
let exif = o.map(lightcraft_codecs::exif::minimal_exif);
let meta = EncodeMeta { exif: exif.as_deref(), ..Default::default() };
let jpeg = encode_jpeg(&img, 90, ChromaSubsampling::S420, &meta).unwrap();
let png = lightcraft_codecs::encode_png(&img, &meta).unwrap();
for (name, bytes) in [("a.jpg", &jpeg), ("a.png", &png)] {
let p = probe_bytes(name, bytes).unwrap();
// what the probe computed from a decode before
let d = lightcraft_codecs::decode(bytes, lightcraft_codecs::DecodeOptions::fit(64, 64)).unwrap();
let swap = Orientation::from_exif(d.orientation).swaps_axes();
let want = if swap { (d.source_height, d.source_width) } else { (d.source_width, d.source_height) };
assert_eq!((p.width, p.height), want, "{name} {o:?}");
assert_eq!((p.width, p.height), if matches!(o, Some(6 | 8)) { (h, w) } else { (w, h) }, "{name} {o:?}");
assert_eq!(p.kind, MediaKind::Image);
assert_eq!(p.format, if name == "a.jpg" { "JPEG" } else { "PNG" });
assert_eq!(p.file_size, bytes.len() as u64);
assert!(p.content_hash.is_some());
assert!(probe_bytes(name, &bytes[..bytes.len() / 2]).is_err(), "{name}: truncated file accepted");
}
}
assert!(probe_bytes("x.jpg", b"not an image").is_err());
}
/// Luminance (from linear RGB) minus its 9 × 9 local mean: tone differences between a render and a camera JPEG
/// mostly cancel, edges remain.
fn detail(w: usize, h: usize, lum: impl Fn(usize, usize) -> f32) -> Vec<f32> {
@@ -553,6 +677,43 @@ mod tests {
b
}
/// [`cr3_with_preview`] plus a full-size CRX raw track whose `CMP1` coding header is garbage:
/// the container parses, the raw data doesn't (`RawError::Corrupt`, not `Unsupported`).
fn cr3_with_corrupt_raw(w: u32, h: u32) -> Vec<u8> {
let bx = |kind: &[u8; 4], body: &[u8]| [&((body.len() + 8) as u32).to_be_bytes()[..], kind, body].concat();
let mut craw = vec![0u8; 82];
craw[24..26].copy_from_slice(&(w as u16).to_be_bytes());
craw[26..28].copy_from_slice(&(h as u16).to_be_bytes());
craw.extend(bx(b"CMP1", &[0xff; 4]));
let mut stsd = vec![0, 0, 0, 0, 0, 0, 0, 1];
stsd.extend(bx(b"CRAW", &craw));
let stsz = [0u32, 16, 1].map(u32::to_be_bytes).concat();
let co64 = [&[0u8, 0, 0, 0, 0, 0, 0, 1][..], &0u64.to_be_bytes()].concat();
let stbl = [bx(b"stsd", &stsd), bx(b"stsz", &stsz), bx(b"co64", &co64)].concat();
let trak = bx(b"trak", &bx(b"mdia", &bx(b"minf", &bx(b"stbl", &stbl))));
let preview = cr3_with_preview(w, h);
[&preview[..24], &bx(b"moov", &trak), &preview[24..]].concat()
}
/// A CR3 whose raw data fails to decode, for any reason, still opens from its embedded JPEG:
/// CR3s did before the CRX decoder, which is verified on few bodies (PR #279 review).
#[test]
fn corrupt_cr3_falls_back_to_embedded_preview() {
let b = cr3_with_corrupt_raw(48, 32);
let err = lightcraft_raw::decode(&b).expect_err("the fixture's raw data is corrupt");
assert!(!matches!(err, lightcraft_raw::RawError::Unsupported(_)), "{err}");
let p = probe_bytes("x.cr3", &b).unwrap();
assert_eq!((p.width, p.height, p.kind, p.format.as_str()), (48, 32, MediaKind::Raw, "CR3"));
assert!(p.preview_only.is_some_and(|why| why.contains("CR3")));
let (img, src) = load_bytes(&b, 24).unwrap();
assert_eq!((img.width, img.height), (24, 16));
assert!(!src.raw);
// other formats still report a corrupt file as an error
let mut dng = crate::tests_xmp::synthetic_dng_with(None, Default::default());
dng.truncate(dng.len() / 2);
assert!(load_bytes(&dng, 24).is_err());
}
/// Issue #138: a DNG's own profile look (hue/saturation map, look table, tone curve) is
/// applied when it's developed — Lightroom-converted DNGs rendered flat and muted without it.
#[test]
@@ -569,8 +730,12 @@ mod tests {
assert!(info.camera_tone.is_none());
// a map that removes all saturation, and a tone curve
let grey = HsvTable { hue_divisions: 4, sat_divisions: 2, val_divisions: 1, data: vec![[0.0, 0.0, 1.0]; 8], srgb_value: false };
raw.color.profile =
ProfileLook { hue_sat_map: [Some(grey), None], look_table: None, tone_curve: ToneCurve::from_tag(&[0.0, 0.0, 0.18, 0.3, 1.0, 1.0]) };
raw.color.profile = ProfileLook {
hue_sat_map: [Some(grey), None],
look_table: None,
tone_curve: ToneCurve::from_tag(&[0.0, 0.0, 0.18, 0.3, 1.0, 1.0]),
gain_table_map: None,
};
let with = lightcraft_raw::write_dng(&raw, &Default::default()).unwrap();
let (after, info) = load_bytes(&with, 64).unwrap();
assert!(sat(&after) < 1e-3, "saturation {} → {}", sat(&before), sat(&after));
@@ -588,6 +753,119 @@ mod tests {
assert!((mean(&dim) / mean(&before) - 0.5).abs() < 0.02, "{} vs {}", mean(&dim), mean(&before));
}
/// iPhone ProRAW-style semantic masks drive the Sky mask: Apple's sky matte (here the right
/// half of the sensor, i.e. the bottom of the photo once turned 90° clockwise — where the sky
/// heuristic would never look) is used; a matte with an unknown name is ignored.
#[test]
fn dng_sky_matte_drives_the_sky_mask() {
use lightcraft_develop::{DevelopSettings, LocalAdjustments, Mask, MaskComponent, MaskOp, MaskShape};
use lightcraft_pipeline::{RenderRequest, render};
use lightcraft_tiff::tags::{self as t, photometric};
use lightcraft_tiff::{ByteOrder, IfdBuilder, ImageData, TiffWriter, Value};
let (w, h) = (16u32, 8u32);
let image = |ifd: &mut IfdBuilder, w: u32, h: u32, cpp: u16, bits: u16, data: Vec<u8>| {
ifd.set(t::IMAGE_WIDTH, Value::Long(vec![w]));
ifd.set(t::IMAGE_LENGTH, Value::Long(vec![h]));
ifd.set(t::BITS_PER_SAMPLE, Value::Short(vec![bits; cpp as usize]));
ifd.set(t::SAMPLES_PER_PIXEL, Value::Short(vec![cpp]));
ifd.set(t::COMPRESSION, Value::Short(vec![1]));
ifd.set_image(ImageData::Strips { rows_per_strip: h, strips: vec![data] });
};
let mut raw = IfdBuilder::new();
raw.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![0]));
raw.set(t::PHOTOMETRIC, Value::Short(vec![photometric::LINEAR_RAW]));
image(&mut raw, w, h, 3, 16, 12000u16.to_le_bytes().repeat((w * h * 3) as usize));
let matte = |name: &str, right: bool| {
let mut m = IfdBuilder::new();
m.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![t::SUBFILE_SEMANTIC_MASK]));
m.set(t::PHOTOMETRIC, Value::Short(vec![photometric::MASK]));
m.set(t::SEMANTIC_NAME, Value::Ascii(name.into()));
image(&mut m, w / 2, h / 2, 1, 8, (0..w / 2 * h / 2).map(|i| if (i % (w / 2) >= w / 4) == right { 255 } else { 0 }).collect());
m
};
let mut ifd0 = IfdBuilder::new();
ifd0.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![1]));
ifd0.set(t::DNG_VERSION, Value::Byte(vec![1, 6, 0, 0]));
ifd0.set(t::MAKE, Value::Ascii("Apple".into()));
ifd0.set(t::ORIENTATION, Value::Short(vec![6]));
ifd0.set(t::COLOR_MATRIX_1, Value::SRational(vec![(1, 1), (0, 1), (0, 1), (0, 1), (1, 1), (0, 1), (0, 1), (0, 1), (1, 1)]));
ifd0.set(t::CALIBRATION_ILLUMINANT_1, Value::Short(vec![21]));
ifd0.set(t::PHOTOMETRIC, Value::Short(vec![photometric::RGB]));
image(&mut ifd0, 4, 2, 3, 8, vec![128; 24]);
ifd0.add_sub_ifd(raw);
ifd0.add_sub_ifd(matte("urn:com:apple:photo:2020:aux:semanticskymatte", true));
ifd0.add_sub_ifd(matte("urn:com:apple:photo:2020:aux:semanticsomethingelse", false));
let bytes = TiffWriter::new(ByteOrder::Little, false).write(&[ifd0]).unwrap();
let (img, info) = load_bytes(&bytes, 64).unwrap();
assert_eq!((img.width, img.height), (8, 16));
assert!(info.mattes.is_some());
let sky = Mask {
components: vec![MaskComponent { name: None, op: MaskOp::Add, invert: false, shape: MaskShape::Sky }],
adjust: LocalAdjustments { exposure: -3.0, ..Default::default() },
..Default::default()
};
let s = DevelopSettings { masks: vec![sky], ..Default::default() };
let req = RenderRequest::fit(8, 16);
let green = |s: &DevelopSettings, info: &SourceInfo, y: usize| render(&img, info, s, &req).image.get(4, y)[1] as i32;
let plain = DevelopSettings::default();
assert_eq!(green(&s, &info, 2), green(&plain, &info, 2), "no sky at the top");
assert!(green(&s, &info, 13) + 40 < green(&plain, &info, 13), "the sky at the bottom is darkened");
// without the matte the heuristic looks at the top of the frame instead
let heuristic = SourceInfo { mattes: None, ..info.clone() };
assert!(green(&s, &heuristic, 2) < green(&plain, &heuristic, 2));
assert_eq!(green(&s, &heuristic, 13), green(&plain, &heuristic, 13));
}
/// A DNG's white balance is re-evaluated through its own colour model: the decoder hands the
/// pipeline the file's colour tags (without the profile look, applied at load) and the white
/// the pixels were developed for, and a custom white balance renders a neutral differently
/// from the Bradford adaptation it replaces.
#[test]
fn dng_white_balance_uses_the_camera_colour_model() {
use lightcraft_pipeline::{RenderRequest, render};
let dng = crate::tests_xmp::synthetic_dng_with(None, Default::default());
let raw = lightcraft_raw::decode(&dng).unwrap();
let (img, info) = load_bytes(&dng, 64).unwrap();
let cc = info.camera_color.as_ref().expect("a DNG with a colour matrix");
assert_eq!(cc.developed_for, lightcraft_raw::color::as_shot_white_xy(&raw));
assert!(cc.tags.profile.is_empty());
assert_eq!(cc.tags.color_matrix, raw.color.color_matrix);
let mut s = lightcraft_develop::DevelopSettings::default();
(s.wb.mode, s.wb.temp, s.wb.tint) = (lightcraft_develop::WbMode::Custom, 3000.0, 20.0);
let req = RenderRequest::fit(32, 32);
let camera = render(&img, &info, &s, &req).image;
let adapted = render(&img, &SourceInfo { camera_color: None, ..info.clone() }, &s, &req).image;
assert_ne!(camera.data, adapted.data);
}
/// Apple ProRAW: like Lightroom Classic, the default render ignores the file's
/// `ProfileGainTableMap` (Apple's local tone mapping); a map that would double every pixel
/// changes nothing.
#[test]
fn dng_gain_table_map_is_not_rendered() {
use lightcraft_raw::gaintable::GainTableMap;
let plain = crate::tests_xmp::synthetic_dng_with(None, Default::default());
let mut raw = lightcraft_raw::decode(&plain).unwrap();
let (before, _) = load_bytes(&lightcraft_raw::write_dng(&raw, &Default::default()).unwrap(), 64).unwrap();
raw.color.profile.gain_table_map = Some(GainTableMap {
points_v: 1,
points_h: 1,
points_n: 1,
spacing_v: 1.0,
spacing_h: 1.0,
origin_v: 0.0,
origin_h: 0.0,
weights: [0.2, 0.2, 0.2, 0.2, 0.2],
gamma: 1.0,
gains: vec![2.0],
});
let with_map = lightcraft_raw::write_dng(&raw, &Default::default()).unwrap();
assert!(lightcraft_raw::decode(&with_map).unwrap().color.profile.gain_table_map.is_some(), "the map is kept");
let (after, _) = load_bytes(&with_map, 64).unwrap();
assert_eq!(before.data, after.data);
}
#[test]
fn unsupported_raw_falls_back_to_embedded_preview() {
let b = cr3_with_preview(48, 32);
@@ -656,6 +934,84 @@ mod tests {
let _ = std::fs::remove_dir_all(&dir);
}
/// A raw whose TIFF shell holds only a small RGB image (IFD0) next to a private block, as Exif
/// declares a picture 25x its size.
fn tiff_shell(w: u32, h: u32, shell: bool) -> Vec<u8> {
use lightcraft_tiff::{IfdBuilder, ImageData, TiffWriter, Value, tags as t};
let mut ifd0 = IfdBuilder::new();
ifd0.set(t::IMAGE_WIDTH, Value::Long(vec![w]));
ifd0.set(t::IMAGE_LENGTH, Value::Long(vec![h]));
ifd0.set(t::BITS_PER_SAMPLE, Value::Short(vec![8, 8, 8]));
ifd0.set(t::SAMPLES_PER_PIXEL, Value::Short(vec![3]));
ifd0.set(t::PHOTOMETRIC, Value::Short(vec![2]));
ifd0.set(t::COMPRESSION, Value::Short(vec![1]));
let px: Vec<u8> = (0..w * h).flat_map(|i| [(i % 251) as u8, 128, 200]).collect();
ifd0.set_image(ImageData::Strips { rows_per_strip: h, strips: vec![px] });
ifd0.set_child(
t::EXIF_IFD,
IfdBuilder::new().with(t::PIXEL_X_DIMENSION, Value::Long(vec![w * 25])).with(t::PIXEL_Y_DIMENSION, Value::Long(vec![h * 25])),
);
let mut b = TiffWriter::default().write(&[ifd0]).unwrap();
if shell {
b.extend(std::iter::repeat_n(0x5au8, (w * h * 625 / 8) as usize + 100));
}
b
}
/// A TIFF raw whose only image is a small thumbnail used to open as a plain image of that size
/// ("a 150 MP file renders 296x220"), with nothing saying so. Now it is recognised as a raw we can't
/// decode: preview only, with the reason, still showing that image.
#[test]
fn thumbnail_only_tiff_raw_is_preview_only_not_a_plain_image() {
let b = tiff_shell(24, 16, true);
let p = probe_bytes("IMG_0001.IIQ", &b).unwrap();
assert_eq!((p.kind, p.format.as_str(), p.width, p.height), (MediaKind::Raw, "IIQ", 24, 16));
let why = p.preview_only.expect("marked preview only");
assert!(why.contains("600x400") && why.contains("24x16"), "{why}");
let (img, src) = load_bytes(&b, 12).unwrap();
assert_eq!((img.width, img.height, src.raw), (12, 8, false));
assert!(embedded_preview_srgb(&b, 12).is_some());
// the same pixels without the private block are a small ordinary image
let plain = probe_bytes("small.tif", &tiff_shell(24, 16, false)).unwrap();
assert_eq!((plain.kind, plain.preview_only, plain.width, plain.height), (MediaKind::Image, None, 24, 16));
assert!(lightcraft_raw::probe(&tiff_shell(24, 16, false)).is_none());
}
/// Recognised raw containers we don't decode (Minolta MRW here: the preview's first byte is
/// overwritten in the file) import as preview only with the JPEG's size, instead of failing.
#[test]
fn undecodable_container_with_a_preview_imports_as_preview_only() {
let px: Vec<u8> = (0..40 * 30).flat_map(|i| [(i % 251) as u8, 128, 200]).collect();
let mut jpeg = encode_jpeg(&EncodeImage::new(40, 30, 3, Samples::U8(&px)), 90, ChromaSubsampling::S444, &EncodeMeta::default()).unwrap();
jpeg[0] = 0x02;
let mut f = b"\0MRM\0\x01\0\0".to_vec();
f.extend(std::iter::repeat_n(0x11u8, 300));
f.extend_from_slice(&jpeg);
f.extend(std::iter::repeat_n(0x22u8, 300));
let p = probe_bytes("A.MRW", &f).unwrap();
assert_eq!((p.kind, p.format.as_str(), p.width, p.height), (MediaKind::Raw, "MRW", 40, 30));
assert!(p.preview_only.is_some_and(|w| w.contains("Mrw")));
let (img, src) = load_bytes(&f, 20).unwrap();
assert_eq!((img.width, img.height, src.raw), (20, 15, false));
// without a JPEG: a clear error
let e = probe_bytes("A.MRW", b"\0MRM\0\x01\0\0 nothing here").unwrap_err();
assert!(e.contains("without an embedded preview"), "{e}");
}
/// Any failure of a recognised raw (not only an unsupported variant) can fall back to its preview;
/// CR3 keeps its own wording.
#[test]
fn decode_failures_of_recognised_raws_may_fall_back_to_the_preview() {
use lightcraft_raw::RawError;
let cr3 = b"\0\0\0\x18ftypcrx \0\0\0\x01";
assert_eq!(preview_reason(b"x", RawError::Unsupported("x".into())), Ok("x".to_string()));
assert!(preview_reason(b"x", RawError::Corrupt("bad strip".into())).is_ok_and(|w| w.contains("bad strip")));
assert!(preview_reason(b"x", RawError::Limit("too big")).is_ok());
assert!(preview_reason(b"x", RawError::Tiff(lightcraft_tiff::TiffError::MissingTag(256))).is_ok());
assert!(preview_reason(b"x", RawError::NotRaw).is_err());
assert!(preview_reason(cr3, RawError::Corrupt("bad".into())).is_ok_and(|w| w.starts_with("CR3 ")));
}
#[test]
fn quick_jobs_show_embedded_previews_then_cached_renders() {
use crate::media::QuickSource;
+393 -73
View File
@@ -33,7 +33,8 @@ use crate::media::ProbeInfo;
/// File extensions LightCraft imports (lower case).
pub const EXTENSIONS: &[&str] = &[
"jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "nrw", "arw", "raf", "orf", "rw2", "rwl", "raw", "pef", "psd", "jxl",
"gif", "bmp", "heic", "avif",
"gif", "bmp", "heic", "avif", // containers LightCraft cannot decode but imports as preview only (their embedded JPEG)
"iiq", "crw", "mrw", "x3f", "kdc", "mos", "erf",
];
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
@@ -317,32 +318,96 @@ pub fn is_supported(path: &Path) -> bool {
}
/// Expand files and folders (recursively) into supported files. `skip` (e.g. the library folder)
/// is never descended into.
/// is never descended into. Each folder's walk is bounded ([`crate::walk::Limits::default`]); a
/// walk that stops at a bound is logged.
pub fn expand(paths: &[String], skip: Option<&Path>) -> Vec<String> {
fn walk(p: &Path, skip: Option<&Path>, out: &mut Vec<String>, top: bool) {
let hidden = p.file_name().is_some_and(|n| n.to_string_lossy().starts_with('.'));
if (hidden && !top) || skip.is_some_and(|s| p == s) {
return;
expand_within(paths, skip, crate::walk::Limits::default()).0
}
/// [`expand`] with the walk bounded by `limits` → (files, whether a walk stopped at a bound).
pub fn expand_within(paths: &[String], skip: Option<&Path>, limits: crate::walk::Limits) -> (Vec<String>, bool) {
let mut out = Vec::new();
let mut truncated = false;
for p in paths {
let p = Path::new(p);
if skip.is_some_and(|s| p == s) {
continue;
}
if p.is_dir() {
let Ok(rd) = std::fs::read_dir(p) else { return };
let mut v: Vec<PathBuf> = rd.flatten().map(|e| e.path()).collect();
v.sort();
for c in v {
walk(&c, skip, out, false);
let w = crate::walk::files_in(p, skip, limits, is_supported);
if w.truncated {
log::warn!(
"import: stopped looking in {} after {} entries (limits: {} entries, {} folder levels)",
p.display(),
w.entries,
limits.max_entries,
limits.max_depth
);
}
} else if top || is_supported(p) {
truncated |= w.truncated;
out.extend(w.files.into_iter().map(|f| f.to_string_lossy().to_string()));
} else {
// explicitly named files are attempted even with an unknown extension (sniffed)
out.push(p.to_string_lossy().to_string());
}
}
let mut out = Vec::new();
for p in paths {
walk(Path::new(p), skip, &mut out, true);
}
let mut seen = std::collections::HashSet::new();
out.retain(|p| seen.insert(p.clone()));
out
(out, truncated)
}
/// What a path given on the command line at launch becomes ([`launch_path`]).
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum LaunchPath {
/// Import this (absolute) file or folder, in the background.
Import(String),
/// Not imported: `path` (absolute) is a whole drive or the whole home folder, which is never
/// walked without being chosen in the app (issue #374).
Refused { path: String, why: String },
}
/// Resolve a launch argument for importing (issue #374). An empty argument is ignored (`None`);
/// a relative one (`.`, `..`, `photos`; `/` or `\` on Windows, the current drive's root) is
/// resolved against `cwd` — the working folder a launcher picked, often a drive root or
/// `C:\Windows\System32` — and refused when that leaves no folder (`cwd` unknown). A drive
/// or file-system root is replaced by its camera folder (`DCIM`, from `dcim`: a memory card) or
/// refused, and so is the home folder `home`: either would walk most of the disk.
pub fn launch_path(arg: &str, cwd: Option<&Path>, home: Option<&Path>, dcim: impl Fn(&Path) -> Option<PathBuf>) -> Option<LaunchPath> {
if arg.trim().is_empty() {
return None;
}
let given = Path::new(arg);
let joined = if given.is_absolute() {
given.to_path_buf()
} else {
match cwd {
Some(c) => c.join(given),
None => return Some(LaunchPath::Refused { path: arg.to_string(), why: "a relative path, and the working folder is unknown".into() }),
}
};
// `.` and `..` resolved by name (no file-system access)
let mut abs = PathBuf::new();
for c in joined.components() {
match c {
std::path::Component::CurDir => {}
std::path::Component::ParentDir => {
abs.pop();
}
other => abs.push(other.as_os_str()),
}
}
let shown = abs.to_string_lossy().to_string();
if abs.parent().is_none() {
return Some(match dcim(&abs) {
Some(d) => LaunchPath::Import(d.to_string_lossy().to_string()),
None => LaunchPath::Refused { path: shown, why: "a whole drive".into() },
});
}
let key = lightcraft_catalog::query::folder_key;
if home.is_some_and(|h| key(&h.to_string_lossy()) == key(&shown)) {
return Some(LaunchPath::Refused { path: shown, why: "the whole home folder".into() });
}
Some(LaunchPath::Import(shown))
}
/// Probe files (headers + content hash) as an import would.
@@ -365,7 +430,7 @@ fn probe_all(probe: Option<&crate::media::FileProbe>, paths: &[String], progress
};
#[cfg(not(target_arch = "wasm32"))]
{
let n = std::thread::available_parallelism().map(|n| n.get()).unwrap_or(4).clamp(1, 8).min(paths.len().max(1));
let n = workers(paths.len());
if n > 1 {
let mut results: Vec<Option<Result<ProbeInfo, String>>> = vec![None; paths.len()];
let next = std::sync::atomic::AtomicUsize::new(0);
@@ -404,18 +469,16 @@ fn probe_all(probe: Option<&crate::media::FileProbe>, paths: &[String], progress
.collect()
}
/// Copy `src` into `root`/`folders` as `name` (default: its own name), made unique with -1, -2…;
/// returns the new path. The copy is verified ([`crate::import_move::copy_new`]: a new file,
/// synced and checked against `probe_hash`, the content hash the probe computed — or compared
/// byte for byte with the source when there is none); a bad one is removed and the photo
/// reported as failed.
fn copy_into(root: &Path, src: &str, folders: &[String], name: Option<&str>, probe_hash: Option<&str>) -> Result<String, String> {
let dir = folders.iter().fold(root.to_path_buf(), |d, f| d.join(f));
/// Copy `src` into `dir` as `name` (default: its own name), made unique with -1, -2…; returns the
/// new path. The copy is verified ([`crate::import_move::copy_new`]: a new file, synced and
/// checked against `probe_hash`, the content hash the probe computed — or compared byte for byte
/// with the source when there is none); a bad one is removed and the photo reported as failed.
fn copy_into(dir: &Path, src: &str, name: Option<&str>, probe_hash: Option<&str>) -> Result<String, String> {
let name = name
.map(str::to_string)
.unwrap_or_else(|| Path::new(src).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| "photo".into()));
let expect = probe_hash.and_then(lightcraft_preview::Hash128::parse);
crate::import_move::copy_new(Path::new(src), &dir, &name, expect).map(|p| p.to_string_lossy().to_string())
crate::import_move::copy_new(Path::new(src), dir, &name, expect).map(|p| p.to_string_lossy().to_string())
}
/// What a [`scan`] needs from the session, so it can run on another thread (a folder on a network
@@ -641,6 +704,50 @@ impl Prepared {
pub fn is_empty(&self) -> bool {
self.items.is_empty()
}
/// Recheck Add-only entries against the owner catalog immediately before commit.
/// Preparation may have run while another import added the same path or content.
pub fn revalidate_add(&mut self, s: &Session) {
let mut paths = HashMap::new();
let mut hashes = HashMap::new();
for photo in s.catalog.photos() {
if let Source::File { path } = &photo.source {
paths.insert(normalize_import_path(path), photo.id);
}
if let Some(hash) = &photo.content_hash {
hashes.insert(hash.clone(), photo.id);
}
}
let mut items = Vec::with_capacity(self.items.len());
for item in std::mem::take(&mut self.items) {
let PreparedItem::Ready(ready) = item else {
items.push(item);
continue;
};
let ReadyFile { path, stored, info, sidecar, placed } = *ready;
let existing_path = paths.get(&normalize_import_path(&stored)).copied().or_else(|| paths.get(&normalize_import_path(&path)).copied());
let existing_hash = info.content_hash.as_ref().and_then(|hash| hashes.get(hash).copied());
if let Some(existing) = existing_path.or(existing_hash) {
if let Some(placed) = &placed {
crate::import_move::rollback(placed);
}
items.push(PreparedItem::Duplicate {
path,
existing: Some(existing),
reason: if existing_path.is_some() { "path" } else { "content" },
hash: info.content_hash,
});
} else {
items.push(PreparedItem::Ready(Box::new(ReadyFile { path, stored, info, sidecar, placed })));
}
}
self.items = items;
}
}
fn normalize_import_path(path: &str) -> String {
let path = path.replace('\\', "/");
if cfg!(windows) { path.to_lowercase() } else { path }
}
impl ImportJob {
@@ -699,8 +806,9 @@ impl ImportJob {
}
/// The file-system half of importing `paths`: expand folders, skip known paths, probe, skip
/// duplicates by content, read sidecars, copy or place (Move) the files. Nothing in the
/// catalog changes. Stops before the next file once `cancel` is set.
/// duplicates by content, read sidecars, copy or place (Move) the files — several at once,
/// with the names and outcomes of one after another. Nothing in the catalog changes. Stops
/// before the next file once `cancel` is set.
pub fn prepare(&mut self, paths: &[String], cancel: &std::sync::atomic::AtomicBool) -> Prepared {
let files = self.expand(paths);
self.prepare_files(files, cancel)
@@ -750,24 +858,39 @@ impl ImportJob {
let probed: Vec<Result<ProbeInfo, String>> =
cached.into_iter().map(|c| c.map(Ok).unwrap_or_else(|| fresh.next().unwrap_or_else(|| Err("not probed".into())))).collect();
crate::memory::release();
// Files are planned one by one, in order (duplicates, sidecars, names, folders); their
// copies or moves — the slow part, reading and writing every byte and verifying it — are
// collected and run on several threads (issue #367), then their outcomes take their places.
let mut plan = Plan::default();
for (path, info) in todo.into_iter().zip(probed) {
if cancel.load(Relaxed) {
break;
}
self.done.fetch_add(1, Relaxed);
let mut info = match info {
Ok(i) => i,
Err(e) => {
log::warn!("import {path}: {e}");
out.items.push(PreparedItem::Failed(path, e));
self.done.fetch_add(1, Relaxed);
plan.slots.push(Slot::Item(PreparedItem::Failed(path, e)));
continue;
}
};
// a copy of a file whose transfer is still pending: a duplicate only if that one succeeds
if !opts.local
&& let Some(h) = &info.content_hash
&& plan.transfers.iter().any(|t| t.info.content_hash.as_ref() == Some(h))
{
self.run_plan(&mut plan, &mut out, cancel);
if cancel.load(Relaxed) {
break;
}
}
if let Some(h) = &info.content_hash
&& !opts.local
&& let Some(existing) = self.by_hash.get(h)
{
out.items.push(PreparedItem::Duplicate { path, existing: *existing, reason: "content", hash: Some(h.clone()) });
self.done.fetch_add(1, Relaxed);
plan.slots.push(Slot::Item(PreparedItem::Duplicate { path, existing: *existing, reason: "content", hash: Some(h.clone()) }));
continue;
}
// the XMP sidecar (or a raw's embedded XMP), read before copying: its capture time files
@@ -786,7 +909,6 @@ impl ImportJob {
if info.captured.is_none() {
info.captured = sidecar.as_ref().and_then(|sc| sc.captured.clone());
}
let mut placed = None;
let stored = match (mode, &self.copy_root) {
(ImportMode::Copy | ImportMode::Move, Some(root))
if !crate::import_move::inside(Path::new(&path), root)
@@ -803,37 +925,35 @@ impl ImportJob {
n
});
let folders = opts.organize.folders(&q);
if mode == ImportMode::Move && !crate::import_move::is_symlink(Path::new(&path)) {
let dir = folders.iter().fold(root.to_path_buf(), |d, f| d.join(f));
let name = name.unwrap_or(own);
match crate::import_move::place(Path::new(&path), &dir, &name) {
Ok(pl) => {
let dst = pl.dst.to_string_lossy().to_string();
placed = Some(pl);
dst
}
Err(e) => {
out.items.push(PreparedItem::Failed(path, e));
continue;
}
let dir = folders.iter().fold(root.to_path_buf(), |d, f| d.join(f));
// the name this file gets depends on what a pending transfer of a like-named file
// into the same folder ends up with: that one goes first
let as_named = name.as_deref().unwrap_or(if own.is_empty() { "photo" } else { own.as_str() });
let family = (dir.to_string_lossy().to_lowercase(), name_family(as_named));
if plan.transfers.iter().any(|t| t.family == family) {
self.run_plan(&mut plan, &mut out, cancel);
if cancel.load(Relaxed) {
break;
}
}
let how = if mode == ImportMode::Move && !crate::import_move::is_symlink(Path::new(&path)) {
How::Place { name: name.unwrap_or(own) }
} else {
if mode == ImportMode::Move {
let reason = "a symbolic link: the file it points to was copied, the link and its target stay";
out.items.push(PreparedItem::Kept(Kept { path: path.clone(), reason: reason.into() }));
plan.slots.push(Slot::Item(PreparedItem::Kept(Kept { path: path.clone(), reason: reason.into() })));
}
match copy_into(root, &path, &folders, name.as_deref(), info.content_hash.as_deref()) {
Ok(p) => p,
Err(e) => {
out.items.push(PreparedItem::Failed(path, e));
continue;
}
}
}
How::Copy { name }
};
// Copy as DNG: the copied raw becomes a DNG (the copy is ours to replace)
let dng = opts.convert_dng && mode == ImportMode::Copy && raw && !info.format.eq_ignore_ascii_case("DNG");
plan.slots.push(Slot::Transfer);
plan.transfers.push(Transfer { path, info, sidecar, dir, how, dng, family });
continue;
}
(ImportMode::Move, _) => {
let reason = "already inside the destination or the library: added where it is";
out.items.push(PreparedItem::Kept(Kept { path: path.clone(), reason: reason.into() }));
plan.slots.push(Slot::Item(PreparedItem::Kept(Kept { path: path.clone(), reason: reason.into() })));
path.clone()
}
_ => path.clone(),
@@ -841,27 +961,199 @@ impl ImportJob {
if let Some(h) = &info.content_hash {
self.by_hash.insert(h.clone(), None);
}
// Copy as DNG: the copied raw becomes a DNG (the copy is ours to replace)
let mut stored = stored;
if opts.convert_dng
&& mode == ImportMode::Copy
&& stored != path
&& info.kind == MediaKind::Raw
&& !info.format.eq_ignore_ascii_case("DNG")
{
match crate::cmd::convert::write_dng_with(self.file_bytes.as_ref(), &stored, String::new()) {
Ok(dng) => {
let _ = std::fs::remove_file(&stored);
stored = dng;
info.format = "DNG".into();
}
Err(e) => log::warn!("import {path}: copy as DNG: {e}"),
}
}
out.items.push(PreparedItem::Ready(Box::new(ReadyFile { path, stored, info, sidecar, placed })));
self.done.fetch_add(1, Relaxed);
plan.slots.push(Slot::Item(PreparedItem::Ready(Box::new(ReadyFile { path, stored, info, sidecar, placed: None }))));
}
self.run_plan(&mut plan, &mut out, cancel);
out
}
/// Run the transfers `plan` collected (on several threads) and append its items to `out` in
/// file order. Transfers not started because `cancel` was set leave no item.
fn run_plan(&mut self, plan: &mut Plan, out: &mut Prepared, cancel: &std::sync::atomic::AtomicBool) {
let mut done = run_transfers(std::mem::take(&mut plan.transfers), self.file_bytes.as_ref(), cancel, &self.done).into_iter();
for slot in std::mem::take(&mut plan.slots) {
let item = match slot {
Slot::Item(it) => it,
Slot::Transfer => match done.next().flatten() {
Some(it) => it,
None => continue,
},
};
if let PreparedItem::Ready(r) = &item
&& let Some(h) = &r.info.content_hash
{
self.by_hash.insert(h.clone(), None);
}
out.items.push(item);
}
}
}
/// A batch's items in file order while its transfers are pending ([`ImportJob::run_plan`]).
#[derive(Default)]
struct Plan {
slots: Vec<Slot>,
transfers: Vec<Transfer>,
}
enum Slot {
Item(PreparedItem),
/// The outcome of the next of [`Plan::transfers`].
Transfer,
}
/// A file to copy or move into `dir`, planned in order and run on any thread ([`run_transfer`]).
struct Transfer {
path: String,
info: ProbeInfo,
sidecar: Option<crate::sidecar::SidecarData>,
dir: PathBuf,
how: How,
/// Copy as DNG.
dng: bool,
/// The folder (lowercase) and [`name_family`]: transfers of one family can take each other's
/// names, so they never run at the same time.
family: (String, String),
}
enum How {
/// A verified copy, under `name` (default: the file's own).
Copy { name: Option<String> },
/// Move: placed (linked or copied) without touching the source.
Place { name: String },
}
/// The names a file called `name` could end up with, as one key: the transfers number a taken name
/// `IMG-1.JPG`, `IMG-2.JPG`…, and Copy as DNG names its DNG `IMG.dng`, `IMG-2.dng`…, so `IMG.JPG`,
/// `IMG-1.jpg` and `img.dng` are all `img` (case-insensitive: so are some file systems).
fn name_family(name: &str) -> String {
let mut stem = match name.rsplit_once('.') {
Some((s, _)) if !s.is_empty() => s,
_ => name,
};
while let Some((head, tail)) = stem.rsplit_once('-')
&& !head.is_empty()
&& !tail.is_empty()
&& tail.bytes().all(|b| b.is_ascii_digit())
{
stem = head;
}
stem.to_lowercase()
}
/// Threads for `jobs` file reads (probes, copies) at once.
#[cfg(not(target_arch = "wasm32"))]
fn workers(jobs: usize) -> usize {
std::thread::available_parallelism().map(|n| n.get()).unwrap_or(4).clamp(1, 8).min(jobs.max(1))
}
/// Files per batch for a caller that prepares an import in batches ([`ImportJob::prepare_files`]):
/// enough to keep every probe and copy thread busy (a batch waits for its slowest file), few
/// enough that each batch soon joins the catalog.
pub fn batch_size() -> usize {
#[cfg(not(target_arch = "wasm32"))]
{
workers(usize::MAX).saturating_mul(2).clamp(8, 16)
}
#[cfg(target_arch = "wasm32")]
{
8
}
}
/// Run `transfers` (several at once where threads exist); their outcomes in order, `None` for one
/// not started because `cancel` was set. `done` counts each one finished.
fn run_transfers(
transfers: Vec<Transfer>,
file_bytes: Option<&crate::merge::ByteReader>,
cancel: &std::sync::atomic::AtomicBool,
done: &std::sync::atomic::AtomicUsize,
) -> Vec<Option<PreparedItem>> {
use std::sync::atomic::Ordering::Relaxed;
let one = |t: Transfer| {
if cancel.load(Relaxed) {
return None;
}
let path = t.path.clone();
// last-resort guard: a panic is this file's failure, not the import's
let item = crate::guard::catch("import", || run_transfer(t, file_bytes)).unwrap_or_else(|e| PreparedItem::Failed(path, e));
done.fetch_add(1, Relaxed);
Some(item)
};
#[cfg(not(target_arch = "wasm32"))]
{
let n = workers(transfers.len());
if n > 1 {
let mut results: Vec<Option<PreparedItem>> = std::iter::repeat_with(|| None).take(transfers.len()).collect();
let queue = std::sync::Mutex::new(transfers.into_iter().enumerate());
let out = std::sync::Mutex::new(&mut results);
// (the tests' injected faults hold for the workers too)
#[cfg(test)]
let fault = crate::import_move::injected_fault();
let work = &|| {
#[cfg(test)]
crate::import_move::inject(fault);
loop {
let next = queue.lock().unwrap_or_else(std::sync::PoisonError::into_inner).next();
let Some((i, t)) = next else { break };
let r = one(t);
if let Some(slot) = out.lock().unwrap_or_else(std::sync::PoisonError::into_inner).get_mut(i) {
*slot = r;
}
}
};
std::thread::scope(|sc| {
let handles: Vec<_> =
(0..n).filter_map(|i| std::thread::Builder::new().name(format!("lc-import-copy-{i}")).spawn_scoped(sc, work).ok()).collect();
if handles.is_empty() {
// no thread to be had: copy here, one after another
work();
}
for h in handles {
if h.join().is_err() {
log::error!("import: a copy worker stopped unexpectedly");
}
}
});
return results;
}
}
transfers.into_iter().map(one).collect()
}
/// Copy or move one planned file (and Copy as DNG); its item.
fn run_transfer(t: Transfer, file_bytes: Option<&crate::merge::ByteReader>) -> PreparedItem {
let Transfer { path, mut info, sidecar, dir, how, dng, .. } = t;
let mut placed = None;
let mut stored = match how {
How::Place { name } => match crate::import_move::place(Path::new(&path), &dir, &name) {
Ok(pl) => {
let dst = pl.dst.to_string_lossy().to_string();
placed = Some(pl);
dst
}
Err(e) => return PreparedItem::Failed(path, e),
},
How::Copy { name } => match copy_into(&dir, &path, name.as_deref(), info.content_hash.as_deref()) {
Ok(p) => p,
Err(e) => return PreparedItem::Failed(path, e),
},
};
if dng {
// decoding the raw twice and encoding it takes several times the file's size in memory
let weight = usize::try_from(info.file_size).unwrap_or(usize::MAX).saturating_mul(6);
let _permit = crate::memory::work_gate().acquire(weight);
match crate::cmd::convert::write_dng_with(file_bytes, &stored, String::new()) {
Ok(dng) => {
let _ = std::fs::remove_file(&stored);
stored = dng;
info.format = "DNG".into();
}
Err(e) => log::warn!("import {path}: copy as DNG: {e}"),
}
}
PreparedItem::Ready(Box::new(ReadyFile { path, stored, info, sidecar, placed }))
}
/// The catalog half of an import: add the photos [`ImportJob::prepare`] readied (one undoable op;
@@ -1093,3 +1385,31 @@ impl Session {
self
}
}
#[cfg(test)]
mod prepared_tests {
use super::*;
#[test]
fn revalidate_add_only_converts_matching_ready_item() {
let mut s = Session::new();
let existing_path = std::env::temp_dir().join("lightcraft-revalidate-existing.jpg");
let other_path = std::env::temp_dir().join("lightcraft-revalidate-other.jpg");
let existing = Photo::new(PhotoId(1), Source::File { path: existing_path.to_string_lossy().into() }, "existing.jpg", "JPG", 1, 1, "now");
s.commit("existing", Op::AddPhoto { photo: Box::new(existing) }).unwrap();
let ready = |path: &Path| ReadyFile {
path: path.to_string_lossy().into(),
stored: path.to_string_lossy().into(),
info: ProbeInfo { format: "JPG".into(), ..ProbeInfo::default() },
sidecar: None,
placed: None,
};
let mut prepared = Prepared {
scanned: 2,
items: vec![PreparedItem::Ready(Box::new(ready(&existing_path))), PreparedItem::Ready(Box::new(ready(&other_path)))],
};
prepared.revalidate_add(&s);
assert!(matches!(prepared.items.first(), Some(PreparedItem::Duplicate { reason: "path", .. })));
assert!(matches!(prepared.items.get(1), Some(PreparedItem::Ready(_))));
}
}
+23
View File
@@ -56,6 +56,9 @@ pub(crate) enum Fault {
Corrupt,
/// Removing the source fails (as on a read-only card).
FailRemove,
/// Each copy takes a little longer and notes the thread it ran on ([`copy_threads`]), to see
/// an import's copies run side by side.
Slow,
}
#[cfg(test)]
@@ -69,6 +72,21 @@ pub(crate) fn inject(f: Fault) {
FAULT.with(|c| c.set(f));
}
/// The calling thread's injected fault (tests only), for the workers it starts to inherit.
#[cfg(test)]
pub(crate) fn injected_fault() -> Fault {
FAULT.with(|c| c.get())
}
/// The threads copies made under [`Fault::Slow`] ran on (tests only).
#[cfg(test)]
static COPY_THREADS: std::sync::Mutex<Vec<std::thread::ThreadId>> = std::sync::Mutex::new(Vec::new());
#[cfg(test)]
pub(crate) fn copy_threads() -> Vec<std::thread::ThreadId> {
COPY_THREADS.lock().unwrap_or_else(std::sync::PoisonError::into_inner).clone()
}
fn injected(f: Fault) -> bool {
#[cfg(test)]
{
@@ -284,6 +302,11 @@ pub(crate) fn copy_verified(src: &Path, dst: &Path, expect: Option<lightcraft_pr
return Err(io::Error::other("write failed (injected)"));
}
io::copy(&mut inp, &mut out)?;
#[cfg(test)]
if injected(Fault::Slow) {
COPY_THREADS.lock().unwrap_or_else(std::sync::PoisonError::into_inner).push(std::thread::current().id());
std::thread::sleep(std::time::Duration::from_millis(30));
}
if injected(Fault::Corrupt) {
out.write_all(b"!")?;
}
+97
View File
@@ -14,17 +14,27 @@ pub mod availability;
mod camera_preview;
pub mod camera_profiles;
pub mod cmd;
pub mod config;
pub mod crs;
pub mod crs_masks;
pub mod demo;
pub mod devices;
pub mod export;
pub mod face_download;
#[cfg(not(target_arch = "wasm32"))]
mod faces_index;
#[cfg(not(target_arch = "wasm32"))]
mod faces_worker;
pub mod files;
pub mod fonts;
pub mod guard;
pub mod import;
mod import_move;
pub mod library;
mod lightroom_archive;
pub mod lightroom_catalog;
pub mod lightroom_job;
mod lightroom_sqlite;
pub mod logging;
pub mod media;
pub mod memory;
@@ -38,6 +48,7 @@ pub mod segment;
pub mod sidecar;
pub mod smart;
mod view;
pub mod walk;
use std::sync::Arc;
@@ -83,6 +94,27 @@ pub struct UndoEntry {
pub folder: Option<FolderMove>,
}
/// The one photo an undo step changes, when it changes exactly one through its edit, rating, flag,
/// label, metadata or versions (`depth` bounds nested batches).
fn single_photo(op: &Op, depth: usize) -> Option<PhotoId> {
match op {
Op::SetDevelop { id, .. }
| Op::SetRating { id, .. }
| Op::SetFlag { id, .. }
| Op::SetLabel { id, .. }
| Op::SetMeta { id, .. }
| Op::SetVersions { id, .. }
| Op::SetHistory { id, .. }
| Op::PushHistory { id, .. } => Some(*id),
Op::Batch { ops } if depth < 8 => {
let mut ids = ops.iter().map(|o| single_photo(o, depth + 1));
let first = ids.next()??;
ids.all(|id| id == Some(first)).then_some(first)
}
_ => None,
}
}
/// A folder renamed or moved on disk as part of an undo step.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct FolderMove {
@@ -131,6 +163,15 @@ pub struct Session {
/// Set by a command whose change must not rewrite the photo's XMP sidecar even with auto-write on
/// (a catalog-only edit of data the sidecar writer does not emit); consumed when the command ends.
pub(crate) skip_auto_write: bool,
/// Where the host keeps face models (one folder each); `None` where there is no file system (the web).
pub face_models_dir: Option<std::path::PathBuf>,
/// Face model downloads started this session (the staged files wait in `<face_models_dir>/.downloads`).
pub face_downloads: face_download::Downloads,
/// The user's own list of models to download (`catalog.json` in the models folder), as last read.
pub(crate) face_catalog: lightcraft_faces::catalog::Catalog,
/// The loaded recognition model and the face embeddings made with it.
#[cfg(not(target_arch = "wasm32"))]
pub(crate) faces: faces_index::FacesState,
/// Copied develop settings (partial JSON) for Paste.
pub clipboard: Option<Value>,
/// The folder on disk the [`LibrarySource::Folder`] view browses.
@@ -165,6 +206,8 @@ pub struct Session {
pub active_spot: Option<usize>,
/// The persistent library this session writes to (`None` = in-memory only).
pub library: Option<library::Library>,
/// Identity of current library opening; refreshed only after a successful open.
pub(crate) library_identity: Arc<()>,
/// XMP sidecar preferences (persisted with the library).
pub xmp: sidecar::XmpPrefs,
/// Parameters of the last export (`app.export` params, minus targets), persisted in prefs.json.
@@ -235,6 +278,11 @@ impl Session {
redo: Vec::new(),
interaction: None,
skip_auto_write: false,
face_models_dir: None,
face_downloads: Default::default(),
face_catalog: Default::default(),
#[cfg(not(target_arch = "wasm32"))]
faces: Default::default(),
clipboard: None,
meta_clipboard: None,
browse: None,
@@ -253,6 +301,7 @@ impl Session {
segmenter: segment::Segmenter::default(),
active_spot: None,
library: None,
library_identity: Arc::new(()),
xmp: sidecar::XmpPrefs::default(),
last_export: None,
export_presets: Vec::new(),
@@ -374,6 +423,27 @@ impl Session {
Ok(())
}
/// The box to cut a face's picture from: the detector's, when the scan has looked at the face (every face is then shown
/// equally close, however loosely or tightly its own region was drawn), else the region's own box.
pub fn face_view(&self, id: PhotoId, rect: lightcraft_geom::Rect) -> lightcraft_geom::Rect {
#[cfg(not(target_arch = "wasm32"))]
if let Some(v) = self.faces.index.view(id.0, &rect) {
return v;
}
let _ = id;
rect
}
/// Apply an op that is LightCraft's own bookkeeping rather than something the user did (faces found by the background
/// scan): journaled like any op, but not an undo step, and it leaves the redo stack alone.
#[cfg(not(target_arch = "wasm32"))]
pub(crate) fn apply_system(&mut self, op: Op) -> Result<()> {
let fwd = op.clone();
self.catalog.apply(op)?;
self.pending_log.push(fwd);
Ok(())
}
/// Leaving photo `id` after editing it: keep its settings as an automatic version (when they
/// differ from its latest version; at most [`AUTO_VERSIONS`] auto versions, oldest dropped).
/// Saved with the library but not an undo step.
@@ -428,6 +498,7 @@ impl Session {
return Err(err);
}
};
self.show_undone(&e.op);
self.pending_log.push(e.op);
self.redo.push(UndoEntry { label: e.label.clone(), op: redo, folder: e.folder.as_ref().map(FolderMove::reversed) });
Ok(e.label)
@@ -442,11 +513,30 @@ impl Session {
return Err(err);
}
};
self.show_undone(&e.op);
self.pending_log.push(e.op);
self.undo.push(UndoEntry { label: e.label.clone(), op: undo, folder: e.folder.as_ref().map(FolderMove::reversed) });
Ok(e.label)
}
/// After an undo or redo that changed one photo (its edit, rating, flag, label, metadata or
/// versions), that photo becomes the active one, as in Lightroom Classic, so the change is on
/// screen (issue #293). It joins the selection if it isn't in it; nothing changes for steps
/// that touch several photos, albums or files, or for a photo not in the current view.
fn show_undone(&mut self, op: &Op) {
let Some(id) = single_photo(op, 0) else { return };
if self.selection.active == Some(id) || !self.visible().contains(&id) {
return;
}
if !self.selection.contains(id) {
self.selection = Selection { ids: vec![id], active: Some(id) };
} else {
self.selection.active = Some(id);
}
self.active_mask = None;
self.active_spot = None;
}
/// Apply an undo/redo op, first renaming the step's folder and moving the files its renames
/// imply (all or nothing). Files that can't be moved back after a failure are reported and the
/// library follows them (a logged change outside the undo history), so none goes missing; the
@@ -732,10 +822,17 @@ pub fn json_delta(old: &Value, new: &Value) -> Option<Value> {
#[cfg(test)]
mod tests;
#[cfg(test)]
mod tests_album_order;
#[cfg(test)]
mod tests_color;
#[cfg(test)]
mod tests_export;
#[cfg(test)]
mod tests_face_models;
#[cfg(test)]
#[cfg(not(target_arch = "wasm32"))]
mod tests_face_recognize;
#[cfg(test)]
mod tests_folders;
#[cfg(test)]
mod tests_forget_local;
+24 -3
View File
@@ -297,6 +297,9 @@ impl Session {
{
log::error!("library: {e}");
}
// what was learned about the old library's faces is saved, and none of it carries over (photo ids are per library)
#[cfg(not(target_arch = "wasm32"))]
self.faces.library_changed();
let (mut journal, mut catalog, report) = Journal::open(catalog)?;
// A loaded catalog counts revisions from 0, like every other one. Caches (sidebar counts,
// keyword tree, the grid's list…) are keyed on the revision, so give each library loaded
@@ -395,7 +398,7 @@ impl Session {
forgot_local: None,
presets_written,
view_written,
settings_warnings: settings.warnings,
settings_warnings: unlocked_warning(lock.as_ref()).into_iter().chain(settings.warnings).collect(),
warnings_reported: 0,
blocked: settings.blocked,
lock: lock.take(),
@@ -412,6 +415,7 @@ impl Session {
lib.forgot_local = Some(plan);
}
}
self.library_identity = std::sync::Arc::new(());
Ok(())
}
@@ -504,12 +508,16 @@ impl Session {
lib.retry_at = None;
lib.last_error = None;
}
match (persisted, snapshot) {
let result = match (persisted, snapshot) {
(_, Ok(())) => Ok(()),
// neither the log nor the snapshot took the queued ops
(Err(EngineError::NotSaved(e)), Err(s)) if lib.journal.seq() == before => Err(EngineError::NotSaved(format!("{e}; snapshot: {s}"))),
(_, Err(s)) => Err(s.into()),
};
if result.is_ok() {
self.library_identity = std::sync::Arc::new(());
}
result
}
fn view_json(&self) -> Vec<u8> {
@@ -570,7 +578,8 @@ impl Session {
lib.files.write_atomic("prefs.json", &v).map_err(|e| EngineError::Other(format!("prefs: {e}")))
}
/// Settings-file warnings of the open library not handed out yet (the UI shows each once).
/// Warnings about the open library not handed out yet: its settings files, and a lock that
/// couldn't be taken (the UI shows each once; the CLI and MCP print them on stderr).
pub fn take_library_warnings(&mut self) -> Vec<String> {
let Some(lib) = self.library.as_mut() else { return vec![] };
let new = lib.settings_warnings[lib.warnings_reported..].to_vec();
@@ -625,3 +634,15 @@ impl Session {
Ok(())
}
}
/// The warning for a library opened without its one-program-at-a-time lock (issue #171): its
/// file system can't lock `catalog.lock` (some network shares). It opens anyway, as documented —
/// refusing would lock the user out — but the user must learn that a second program could open it.
fn unlocked_warning(lock: Option<&LibraryLock>) -> Option<String> {
lock.filter(|l| !l.held()).map(|_| {
"This library could not be locked (its catalog.lock file can't be locked where it is stored, e.g. on some network \
shares), so it is open without protection against a second program: use it in one LightCraft app or command at \
a time, or changes made in one of them can be lost."
.to_string()
})
}
+685
View File
@@ -0,0 +1,685 @@
//! Bounded, deduplicated persistence for Lightroom import source data.
//!
//! Archives use a deterministic JSON representation wrapped in zlib. The JSON
//! writer streams through a bounded compressor, so an import cannot first build
//! an unbounded archive buffer. Blob columns are omitted by the catalog reader;
//! decoded XMP,
//! settings, history, snapshots, identities, paths, collections & membership
//! remain available for recovery.
use std::collections::BTreeMap;
use std::fs::{self, File};
use std::io::{self, Read, Write};
use std::path::{Path, PathBuf};
use miniz_oxide::deflate::{
core::{CompressorOxide, create_comp_flags_from_zip_params},
stream::deflate,
};
use miniz_oxide::{MZFlush, MZStatus};
use serde::ser::{Serialize, SerializeMap, SerializeSeq, SerializeStruct, Serializer};
use serde_json::Value;
use crate::lightroom_catalog::{CatalogImport, CatalogPhoto};
/// Maximum size of one complete archive on disk, including format header.
pub const MAX_ARCHIVE_BYTES: u64 = 32 * 1024 * 1024;
/// Maximum total size retained in managed archives.
pub const MAX_TOTAL_ARCHIVE_BYTES: u64 = 128 * 1024 * 1024;
/// Maximum number of managed archives retained.
pub const MAX_MANAGED_ARCHIVES: usize = 8;
const FORMAT_VERSION: u8 = 1;
const HEADER: &[u8] = b"LC-LRARCH\0\x01Z";
const NAME_PREFIX: &str = "lightroom-import-";
const NAME_SUFFIX: &str = ".lca";
const MAX_SCAN: usize = 1024;
const IO_BUFFER: usize = 16 * 1024;
/// Store compact source data below `directory` and return its managed archive.
///
/// The operation is safe to call before catalog mutation. It rejects a
/// symlink directory, writes only `lightroom-import-<number>.lca`, compares the
/// complete bounded byte stream with recent managed archives, and retains at
/// most [`MAX_MANAGED_ARCHIVES`] and [`MAX_TOTAL_ARCHIVE_BYTES`]. Legacy
/// `lightroom-import-*.json` files are deliberately ignored and preserved for
/// an explicit caller warning.
pub fn store(directory: &Path, data: &CatalogImport) -> Result<PathBuf, String> {
prepare_directory(directory)?;
let mut managed = managed_files(directory)?;
managed.sort_by_key(|entry| entry.index);
let temporary = temporary_path(directory)?;
write_archive(&temporary, data).inspect_err(|_| {
let _ = remove_managed_file(&temporary);
})?;
let recent = managed.iter().rev().take(MAX_MANAGED_ARCHIVES);
for entry in recent {
let same = match same_bounded(&temporary, &entry.path) {
Ok(same) => same,
Err(error) => {
let _ = remove_managed_file(&temporary);
return Err(error.to_string());
}
};
if same {
remove_managed_file(&temporary).map_err(|e| e.to_string())?;
let matched = entry.path.clone();
let mut all = managed.clone();
prune_existing(directory, &mut all, Some(&matched))?;
return Ok(matched);
}
}
let index = managed.last().map_or(0, |entry| entry.index.saturating_add(1));
if managed.last().is_some_and(|entry| entry.index == u64::MAX) {
let _ = remove_managed_file(&temporary);
return Err("managed Lightroom archive index is exhausted".into());
}
let destination = match next_destination(directory, index) {
Ok(path) => path,
Err(error) => {
let _ = remove_managed_file(&temporary);
return Err(error);
}
};
let size = match fs::metadata(&temporary) {
Ok(meta) => meta.len(),
Err(error) => {
let _ = remove_managed_file(&temporary);
return Err(error.to_string());
}
};
let new_entry = Managed { index, path: destination.clone(), size };
let mut after = managed;
after.push(new_entry);
after.sort_by_key(|entry| entry.index);
let retirees = match retirement_set(&after) {
Ok(retirees) => retirees,
Err(error) => {
let _ = remove_managed_file(&temporary);
return Err(error);
}
};
let quarantined = match quarantine(&retirees) {
Ok(quarantined) => quarantined,
Err(error) => {
let _ = remove_managed_file(&temporary);
return Err(error);
}
};
if let Err(error) = publish_new(&temporary, &destination) {
rollback_quarantine(&quarantined);
let _ = remove_managed_file(&temporary);
return Err(error.to_string());
}
if let Err(error) = remove_managed_file(&temporary) {
return Err(format!("archive published but temporary cleanup failed: {error}"));
}
for (old, held) in quarantined {
if let Err(error) = remove_managed_file(&held) {
return Err(format!("archive published but could not retire {}: {error}", old.display()));
}
}
Ok(destination)
}
/// Store source data when possible. An archive that reaches its bounded size is
/// omitted with a warning so catalog import can still complete.
pub fn store_best_effort(directory: &Path, data: &CatalogImport) -> Result<Option<PathBuf>, String> {
match store(directory, data) {
Ok(path) => Ok(Some(path)),
Err(error) if error.contains("Lightroom archive exceeds 32 MiB") => Ok(None),
Err(error) => Err(error),
}
}
#[derive(Clone, Debug)]
struct Managed {
index: u64,
path: PathBuf,
size: u64,
}
fn prepare_directory(directory: &Path) -> Result<(), String> {
match fs::symlink_metadata(directory) {
Ok(meta) => {
if meta.file_type().is_symlink() || !meta.is_dir() {
return Err(format!("archive directory is not a real directory: {}", directory.display()));
}
}
Err(error) if error.kind() == io::ErrorKind::NotFound => {
fs::create_dir_all(directory).map_err(|e| e.to_string())?;
let meta = fs::symlink_metadata(directory).map_err(|e| e.to_string())?;
if meta.file_type().is_symlink() || !meta.is_dir() {
return Err(format!("archive directory is not a real directory: {}", directory.display()));
}
}
Err(error) => return Err(error.to_string()),
}
Ok(())
}
fn managed_files(directory: &Path) -> Result<Vec<Managed>, String> {
let mut files = Vec::new();
for (seen, result) in fs::read_dir(directory).map_err(|e| e.to_string())?.enumerate() {
if seen >= MAX_SCAN {
return Err(format!("too many entries in archive directory (limit {MAX_SCAN})"));
}
let entry = result.map_err(|e| e.to_string())?;
let Some(index) = parse_managed_name(&entry.file_name()) else { continue };
let meta = fs::symlink_metadata(entry.path()).map_err(|e| e.to_string())?;
if meta.file_type().is_symlink() || !meta.is_file() {
continue;
}
files.push(Managed { index, path: entry.path(), size: meta.len() });
}
Ok(files)
}
fn parse_managed_name(name: &std::ffi::OsStr) -> Option<u64> {
let name = name.to_str()?;
let digits = name.strip_prefix(NAME_PREFIX)?.strip_suffix(NAME_SUFFIX)?;
if digits.is_empty() || !digits.bytes().all(|byte| byte.is_ascii_digit()) {
return None;
}
digits.parse().ok()
}
fn temporary_path(directory: &Path) -> Result<PathBuf, String> {
let pid = std::process::id();
for slot in 0..MAX_SCAN {
let path = directory.join(format!(".lightroom-import-{pid}-{slot}.tmp"));
match fs::symlink_metadata(&path) {
Ok(_) => continue,
Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(path),
Err(error) => return Err(error.to_string()),
}
}
Err("no safe temporary archive name available".into())
}
fn next_destination(directory: &Path, mut index: u64) -> Result<PathBuf, String> {
for _ in 0..MAX_SCAN {
let path = directory.join(format!("{NAME_PREFIX}{index}{NAME_SUFFIX}"));
match fs::symlink_metadata(&path) {
Ok(_) => index = index.checked_add(1).ok_or("managed archive index is exhausted")?,
Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(path),
Err(error) => return Err(error.to_string()),
}
}
Err("no safe managed archive name available".into())
}
fn write_archive(path: &Path, data: &CatalogImport) -> Result<(), String> {
lightcraft_catalog::safe_file::write_atomic_with(path, &mut |raw| {
let mut bounded = BoundedWriter { inner: raw, total: 0 };
bounded.write_all(HEADER)?;
let mut compressor = ZlibWriter::new(&mut bounded);
serde_json::to_writer(&mut compressor, &ArchiveRef(data)).map_err(json_io)?;
compressor.finish()?;
Ok(())
})
.map(|_| ())
.map_err(|error| format!("could not write Lightroom archive: {error}"))
}
fn json_io(error: serde_json::Error) -> io::Error {
io::Error::new(io::ErrorKind::InvalidData, error.to_string())
}
struct BoundedWriter<'a> {
inner: &'a mut dyn Write,
total: u64,
}
impl Write for BoundedWriter<'_> {
fn write(&mut self, bytes: &[u8]) -> io::Result<usize> {
let next = self.total.checked_add(bytes.len() as u64).ok_or_else(limit_error)?;
if next > MAX_ARCHIVE_BYTES {
return Err(limit_error());
}
self.inner.write_all(bytes)?;
self.total = next;
Ok(bytes.len())
}
fn flush(&mut self) -> io::Result<()> {
self.inner.flush()
}
}
fn limit_error() -> io::Error {
io::Error::new(io::ErrorKind::InvalidData, "Lightroom archive exceeds 32 MiB")
}
struct ZlibWriter<'a, 'b> {
sink: &'a mut BoundedWriter<'b>,
compressor: CompressorOxide,
plain: u64,
}
impl<'a, 'b> ZlibWriter<'a, 'b> {
fn new(sink: &'a mut BoundedWriter<'b>) -> ZlibWriter<'a, 'b> {
let flags = create_comp_flags_from_zip_params(6, 15, 0);
ZlibWriter { sink, compressor: CompressorOxide::new(flags), plain: 0 }
}
fn run(&mut self, input: &[u8], flush: MZFlush) -> io::Result<()> {
let mut rest = input;
loop {
let mut output = [0u8; IO_BUFFER];
let result = deflate(&mut self.compressor, rest, &mut output, flush);
self.sink.write_all(&output[..result.bytes_written])?;
match result.status {
Ok(MZStatus::StreamEnd) => return Ok(()),
Ok(MZStatus::Ok) => {
let consumed = result.bytes_consumed;
if consumed > rest.len() {
return Err(io::Error::new(io::ErrorKind::InvalidData, "invalid compressor progress"));
}
rest = &rest[consumed..];
if rest.is_empty() && flush == MZFlush::None {
return Ok(());
}
if consumed == 0 && result.bytes_written == 0 {
return Err(io::Error::new(io::ErrorKind::InvalidData, "compressor made no progress"));
}
}
Ok(status) => {
return Err(io::Error::new(io::ErrorKind::InvalidData, format!("unsupported compressor status: {status:?}")));
}
Err(error) => return Err(io::Error::new(io::ErrorKind::InvalidData, format!("compression failed: {error:?}"))),
}
}
}
fn finish(&mut self) -> io::Result<()> {
self.run(&[], MZFlush::Finish)
}
}
impl Write for ZlibWriter<'_, '_> {
fn write(&mut self, bytes: &[u8]) -> io::Result<usize> {
if bytes.is_empty() {
return Ok(0);
}
let next = self.plain.checked_add(bytes.len() as u64).ok_or_else(limit_error)?;
if next > MAX_ARCHIVE_BYTES {
return Err(limit_error());
}
self.run(bytes, MZFlush::None)?;
self.plain = next;
Ok(bytes.len())
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
fn same_bounded(left: &Path, right: &Path) -> io::Result<bool> {
let left_meta = fs::symlink_metadata(left)?;
let right_meta = fs::symlink_metadata(right)?;
if left_meta.file_type().is_symlink() || right_meta.file_type().is_symlink() || !left_meta.is_file() || !right_meta.is_file() {
return Ok(false);
}
let left_len = left_meta.len();
let right_len = right_meta.len();
if left_len != right_len || left_len > MAX_ARCHIVE_BYTES {
return Ok(false);
}
let mut left = File::open(left)?;
let mut right = File::open(right)?;
let mut left_buf = [0u8; IO_BUFFER];
let mut right_buf = [0u8; IO_BUFFER];
loop {
let left_n = left.read(&mut left_buf)?;
let right_n = right.read(&mut right_buf)?;
if left_n != right_n {
return Ok(false);
}
if left_n == 0 {
return Ok(true);
}
if left_buf[..left_n] != right_buf[..right_n] {
return Ok(false);
}
}
}
fn retirement_set(entries: &[Managed]) -> Result<Vec<Managed>, String> {
retirement_set_protected(entries, None)
}
fn retirement_set_protected(entries: &[Managed], protected: Option<&Path>) -> Result<Vec<Managed>, String> {
let mut ordered = entries.to_vec();
ordered.sort_by_key(|entry| entry.index);
let mut total = ordered.iter().try_fold(0u64, |sum, entry| sum.checked_add(entry.size)).ok_or("archive size overflow")?;
let mut retirees = Vec::new();
while ordered.len() > MAX_MANAGED_ARCHIVES || total > MAX_TOTAL_ARCHIVE_BYTES {
let Some(position) = ordered.iter().position(|entry| protected.is_none_or(|path| entry.path != path)) else {
return Err("cannot satisfy archive retention limit".into());
};
let oldest = ordered.remove(position);
total = total.saturating_sub(oldest.size);
retirees.push(oldest);
}
Ok(retirees)
}
fn prune_existing(directory: &Path, entries: &mut [Managed], protected: Option<&Path>) -> Result<(), String> {
let retirees = retirement_set_protected(entries, protected)?;
let quarantined = quarantine(&retirees)?;
for (old, held) in quarantined {
if let Err(error) = remove_managed_file(&held) {
return Err(format!("could not retire {} from {}: {error}", old.display(), directory.display()));
}
}
Ok(())
}
fn quarantine(entries: &[Managed]) -> Result<Vec<(PathBuf, PathBuf)>, String> {
let mut moved = Vec::new();
let pid = std::process::id();
for (slot, entry) in entries.iter().enumerate() {
let held = entry.path.with_file_name(format!(".lightroom-retire-{pid}-{slot}.tmp"));
match fs::symlink_metadata(&held) {
Ok(_) => {
rollback_quarantine(&moved);
return Err(format!("retention temporary path already exists: {}", held.display()));
}
Err(error) if error.kind() == io::ErrorKind::NotFound => {}
Err(error) => {
rollback_quarantine(&moved);
return Err(error.to_string());
}
}
let meta = fs::symlink_metadata(&entry.path).map_err(|error| {
rollback_quarantine(&moved);
error.to_string()
})?;
if meta.file_type().is_symlink() || !meta.is_file() {
rollback_quarantine(&moved);
return Err(format!("managed archive changed into non-file: {}", entry.path.display()));
}
if let Err(error) = fs::rename(&entry.path, &held) {
rollback_quarantine(&moved);
return Err(error.to_string());
}
moved.push((entry.path.clone(), held));
}
Ok(moved)
}
fn rollback_quarantine(moved: &[(PathBuf, PathBuf)]) {
for (original, held) in moved.iter().rev() {
let _ = fs::rename(held, original);
}
}
fn publish_new(temporary: &Path, destination: &Path) -> io::Result<()> {
match fs::hard_link(temporary, destination) {
Ok(()) => Ok(()),
Err(error) => Err(io::Error::new(error.kind(), format!("cannot atomically publish archive: {error}"))),
}
}
fn remove_managed_file(path: &Path) -> io::Result<()> {
let meta = fs::symlink_metadata(path)?;
if meta.file_type().is_symlink() || !meta.is_file() {
return Err(io::Error::new(io::ErrorKind::InvalidInput, "refusing to remove non-regular archive path"));
}
fs::remove_file(path)
}
struct ArchiveRef<'a>(&'a CatalogImport);
impl Serialize for ArchiveRef<'_> {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
let mut out = serializer.serialize_struct("CatalogImportArchive", 7)?;
out.serialize_field("version", &FORMAT_VERSION)?;
out.serialize_field("source", &self.0.source)?;
out.serialize_field("photos", &PhotoSeq(&self.0.photos))?;
out.serialize_field("collections", &RowSeq(&self.0.collections))?;
out.serialize_field("members", &RowSeq(&self.0.members))?;
out.serialize_field("collection_content", &RowSeq(&self.0.collection_content))?;
out.serialize_field("warnings", &self.0.warnings)?;
out.end()
}
}
struct PhotoSeq<'a>(&'a [CatalogPhoto]);
impl Serialize for PhotoSeq<'_> {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
let mut out = serializer.serialize_seq(Some(self.0.len()))?;
for photo in self.0 {
out.serialize_element(&PhotoRef(photo))?;
}
out.end()
}
}
struct PhotoRef<'a>(&'a CatalogPhoto);
impl Serialize for PhotoRef<'_> {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
let p = self.0;
let mut out = serializer.serialize_struct("CatalogPhotoArchive", 8)?;
out.serialize_field("source_id", &p.source_id)?;
out.serialize_field("uuid", &p.uuid)?;
out.serialize_field("path", &p.path)?;
out.serialize_field("image", &RowRef(&p.image))?;
out.serialize_field("settings", &p.settings)?;
out.serialize_field("xmp", &p.xmp)?;
out.serialize_field("keywords", &p.keywords)?;
out.serialize_field("history", &RowSeq(&p.history))?;
out.serialize_field("snapshots", &RowSeq(&p.snapshots))?;
out.end()
}
}
struct RowSeq<'a>(&'a [BTreeMap<String, Value>]);
impl Serialize for RowSeq<'_> {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
let mut out = serializer.serialize_seq(Some(self.0.len()))?;
for row in self.0 {
out.serialize_element(&RowRef(row))?;
}
out.end()
}
}
struct RowRef<'a>(&'a BTreeMap<String, Value>);
impl Serialize for RowRef<'_> {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
let mut out = serializer.serialize_map(None)?;
for (key, value) in self.0 {
out.serialize_entry(key, value)?;
}
out.end()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::lightroom_catalog::{CatalogImport, CatalogPhoto};
use std::sync::atomic::{AtomicU64, Ordering};
static NEXT: AtomicU64 = AtomicU64::new(0);
fn tempdir() -> PathBuf {
let path = std::env::temp_dir().join(format!("lightcraft-lr-archive-{}-{}", std::process::id(), NEXT.fetch_add(1, Ordering::Relaxed)));
fs::create_dir_all(&path).unwrap();
path
}
fn data(source: String) -> CatalogImport {
CatalogImport {
source,
photos: Vec::new(),
collections: Vec::new(),
members: Vec::new(),
collection_content: Vec::new(),
warnings: Vec::new(),
}
}
fn rich_data() -> CatalogImport {
let photo = CatalogPhoto {
source_id: 7,
uuid: "uuid-7".into(),
path: "/photos/one.raw".into(),
image: BTreeMap::from([("rating".into(), serde_json::json!(5)), ("opaque_blob".into(), serde_json::json!([1, 2, 3, 4]))]),
settings: "s = { Exposure2012 = 0.75 }".into(),
xmp: "<xmp>decoded</xmp>".into(),
keywords: vec!["Trips|Goa".into()],
history: vec![BTreeMap::from([("text".into(), serde_json::json!("step")), ("curve".into(), serde_json::json!([0, 128, 255]))])],
snapshots: vec![BTreeMap::from([("name".into(), serde_json::json!("before"))])],
};
CatalogImport {
source: "source.lrcat".into(),
photos: vec![photo],
collections: vec![BTreeMap::from([("name".into(), serde_json::json!("Keepers"))])],
members: vec![BTreeMap::from([("image".into(), serde_json::json!(7))])],
collection_content: vec![BTreeMap::from([("rule".into(), serde_json::json!("rating > 3"))])],
warnings: vec!["one warning".into()],
}
}
fn managed_count(path: &Path) -> usize {
fs::read_dir(path).unwrap().filter_map(Result::ok).filter(|entry| parse_managed_name(&entry.file_name()).is_some()).count()
}
#[test]
fn deduplicates_identical_deterministic_archives() {
let path = tempdir();
let input = data("catalog.lrcat".into());
let first = store(&path, &input).unwrap();
let second = store(&path, &input).unwrap();
assert_eq!(first, second);
assert_eq!(managed_count(&path), 1);
fs::remove_dir_all(path).unwrap();
}
#[test]
fn dedup_returns_archive_that_retention_keeps() {
let path = tempdir();
let input = data("catalog.lrcat".into());
let first = store(&path, &input).unwrap();
for index in 1..MAX_MANAGED_ARCHIVES {
let filler = path.join(format!("lightroom-import-{index}.lca"));
File::create(filler).unwrap().set_len(19 * 1024 * 1024).unwrap();
}
let reused = store(&path, &input).unwrap();
assert_eq!(reused, first);
assert!(reused.is_file());
assert!(managed_count(&path) <= MAX_MANAGED_ARCHIVES);
fs::remove_dir_all(path).unwrap();
}
#[test]
fn caps_managed_archive_count_and_preserves_unrelated_files() {
let path = tempdir();
let human = path.join("notes.txt");
let legacy = path.join("lightroom-import-0.json");
fs::write(&human, b"keep").unwrap();
fs::write(&legacy, b"legacy").unwrap();
for index in 0..(MAX_MANAGED_ARCHIVES + 3) {
let input = data(format!("catalog-{index}.lrcat"));
store(&path, &input).unwrap();
}
assert!(managed_count(&path) <= MAX_MANAGED_ARCHIVES);
assert_eq!(fs::read(&human).unwrap(), b"keep");
assert_eq!(fs::read(&legacy).unwrap(), b"legacy");
fs::remove_dir_all(path).unwrap();
}
#[test]
fn rejects_oversize_before_creating_archive() {
let path = tempdir();
let input = data("x".repeat((MAX_ARCHIVE_BYTES as usize) + 1));
assert!(store(&path, &input).is_err());
assert_eq!(managed_count(&path), 0);
fs::remove_dir_all(path).unwrap();
}
#[test]
fn large_catalog_history_is_best_effort_and_does_not_abort_import() {
let path = tempdir();
let mut input = data("catalog.lrcat".into());
input.photos = (0..5_000)
.map(|index| CatalogPhoto {
source_id: index,
uuid: format!("photo-{index}"),
path: format!("/photos/{index}.raw"),
image: BTreeMap::new(),
settings: String::new(),
xmp: String::new(),
keywords: Vec::new(),
history: vec![BTreeMap::from([("text".into(), serde_json::json!(format!("history-{index}-{}", "x".repeat(8 * 1024))))])],
snapshots: Vec::new(),
})
.collect();
assert_eq!(store_best_effort(&path, &input).unwrap(), None);
assert_eq!(managed_count(&path), 0);
fs::remove_dir_all(path).unwrap();
}
#[test]
fn archive_roundtrips_zlib_payload_and_preserves_numeric_arrays() {
let path = tempdir();
let archive = store(&path, &rich_data()).unwrap();
let bytes = fs::read(archive).unwrap();
assert!(bytes.starts_with(HEADER));
let json = miniz_oxide::inflate::decompress_to_vec_zlib_with_limit(&bytes[HEADER.len()..], MAX_ARCHIVE_BYTES as usize).unwrap();
let decoded: Value = serde_json::from_slice(&json).unwrap();
assert_eq!(decoded["version"], serde_json::json!(FORMAT_VERSION));
assert_eq!(decoded["source"], serde_json::json!("source.lrcat"));
assert_eq!(decoded["photos"].as_array().unwrap().len(), 1);
assert_eq!(decoded["photos"][0]["source_id"], serde_json::json!(7));
assert_eq!(decoded["photos"][0]["uuid"], serde_json::json!("uuid-7"));
assert_eq!(decoded["photos"][0]["path"], serde_json::json!("/photos/one.raw"));
assert_eq!(decoded["photos"][0]["settings"], serde_json::json!("s = { Exposure2012 = 0.75 }"));
assert_eq!(decoded["photos"][0]["xmp"], serde_json::json!("<xmp>decoded</xmp>"));
assert_eq!(decoded["photos"][0]["keywords"], serde_json::json!(["Trips|Goa"]));
assert_eq!(decoded["photos"][0]["image"]["rating"], serde_json::json!(5));
assert_eq!(decoded["photos"][0]["image"]["opaque_blob"], serde_json::json!([1, 2, 3, 4]));
assert_eq!(decoded["photos"][0]["history"].as_array().unwrap().len(), 1);
assert_eq!(decoded["photos"][0]["history"][0]["curve"], serde_json::json!([0, 128, 255]));
assert_eq!(decoded["photos"][0]["snapshots"].as_array().unwrap().len(), 1);
assert_eq!(decoded["collections"].as_array().unwrap().len(), 1);
assert_eq!(decoded["collections"][0]["name"], serde_json::json!("Keepers"));
assert_eq!(decoded["members"].as_array().unwrap().len(), 1);
assert_eq!(decoded["members"][0]["image"], serde_json::json!(7));
assert_eq!(decoded["collection_content"].as_array().unwrap().len(), 1);
assert_eq!(decoded["warnings"], serde_json::json!(["one warning"]));
fs::remove_dir_all(path).unwrap();
}
#[test]
fn retention_enforces_aggregate_byte_cap_with_small_synthetic_entries() {
let mut entries: Vec<_> = (0..MAX_MANAGED_ARCHIVES as u64)
.map(|index| Managed { index, path: PathBuf::from(format!("lightroom-import-{index}.lca")), size: 20 * 1024 * 1024 })
.collect();
entries.push(Managed { index: 8, path: PathBuf::from("lightroom-import-8.lca"), size: 20 * 1024 * 1024 });
let retirees = retirement_set(&entries).unwrap();
assert_eq!(retirees.len(), 3);
assert_eq!(retirees.iter().map(|entry| entry.index).collect::<Vec<_>>(), vec![0, 1, 2]);
}
#[test]
fn retention_failure_preserves_old_archive() {
let path = tempdir();
let old = path.join("lightroom-import-0.lca");
let held = path.join(format!(".lightroom-retire-{}-0.tmp", std::process::id()));
fs::write(&old, b"old archive").unwrap();
fs::write(&held, b"occupied").unwrap();
let entry = Managed { index: 0, path: old.clone(), size: 11 };
assert!(quarantine(&[entry]).is_err());
assert_eq!(fs::read(&old).unwrap(), b"old archive");
fs::remove_dir_all(path).unwrap();
}
}
File diff suppressed because it is too large. Load diff
+450
View File
@@ -0,0 +1,450 @@
//! Detached Lightroom catalog preparation and owner-thread commit.
//!
//! `LightroomJob::new` takes only an in-memory session snapshot. `prepare` owns all catalog
//! reading, probing and archive staging, while `commit_prepared` only applies catalog operations
//! to the session that created the job. Archive index bytes are returned for a worker to finish
//! after the owner has accepted the catalog operation.
use std::collections::HashSet;
use std::path::{Path, PathBuf};
use std::sync::{
Arc,
atomic::{AtomicBool, AtomicUsize, Ordering},
};
use serde_json::Value;
use crate::import::{ImportJob, ImportMode, ImportOptions, Prepared};
use crate::lightroom_catalog::{self, CatalogImport, ImportIndex};
const COMMAND: &str = "library.importLightroom";
const MAX_INDEX_BYTES: usize = 16 * 1024 * 1024;
fn error(message: impl Into<String>) -> crate::EngineError {
crate::EngineError::BadParams { cmd: COMMAND.into(), msg: message.into() }
}
fn stage_archive(archive_dir: Option<&Path>, data: &mut CatalogImport) -> crate::Result<Option<PathBuf>> {
let archive_path = archive_dir.map(|dir| crate::lightroom_archive::store_best_effort(dir, data).map_err(error)).transpose()?.flatten();
if archive_dir.is_some() && archive_path.is_none() {
data.warnings.push("Lightroom source archive exceeded 32 MiB; archive skipped".into());
}
Ok(archive_path)
}
/// Memory identity of the library which owns a prepared completion.
///
/// Each successful open/close refreshes session identity, including reopening the same path.
/// The owner check deliberately does not compare catalog revisions: ordinary edits while preparation
/// runs must survive and are merged by the owner commit.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct LightroomLibraryToken {
library_dir: Option<PathBuf>,
identity: Arc<()>,
}
impl LightroomLibraryToken {
fn capture(s: &crate::Session) -> Self {
Self {
library_dir: s.library.as_ref().filter(|library| library.on_disk).map(|library| library.dir.clone()),
identity: Arc::clone(&s.library_identity),
}
}
/// Whether `s` still refers to the library which started preparation.
pub fn matches(&self, s: &crate::Session) -> bool {
Arc::ptr_eq(&self.identity, &s.library_identity)
&& self.library_dir == s.library.as_ref().filter(|library| library.on_disk).map(|library| library.dir.clone())
}
pub fn library_dir(&self) -> Option<&Path> {
self.library_dir.as_deref()
}
}
/// Filesystem-free result of catalog preparation.
pub struct PreparedLightroom {
pub(crate) data: CatalogImport,
pub(crate) normal: Prepared,
pub(crate) index: ImportIndex,
pub(crate) archive_path: Option<PathBuf>,
pub(crate) token: LightroomLibraryToken,
pub(crate) missing_sources: HashSet<String>,
pub(crate) update_existing: bool,
pub(crate) now: String,
cancelled: bool,
}
impl PreparedLightroom {
pub fn source(&self) -> &str {
&self.data.source
}
pub fn photo_count(&self) -> usize {
self.data.photos.len()
}
pub fn token(&self) -> &LightroomLibraryToken {
&self.token
}
pub fn is_cancelled(&self) -> bool {
self.cancelled
}
/// Roll back any normal-import filesystem placement held by this prepared value.
pub fn rollback(&self) {
self.normal.rollback();
}
}
/// Index write returned by owner commit. `finish` is intended for a worker after the owner has
/// returned, and performs the only archive-index filesystem operation in this pipeline.
pub struct LightroomArchiveFinalization {
path: Option<PathBuf>,
bytes: Option<Vec<u8>>,
}
impl LightroomArchiveFinalization {
pub fn finish(self) -> crate::Result<()> {
let (Some(path), Some(bytes)) = (self.path, self.bytes) else { return Ok(()) };
if bytes.len() > MAX_INDEX_BYTES {
return Err(error("Lightroom import index exceeds 16 MiB"));
}
lightcraft_catalog::safe_file::write_atomic(&path, &bytes).map_err(|e| error(e.to_string()))
}
pub fn path(&self) -> Option<&Path> {
self.path.as_deref()
}
}
/// Owner-thread catalog result. The report is ready for the UI immediately; index finalization
/// can remain on the worker while its task stays alive.
pub struct LightroomCompletion {
pub report: Value,
pub finalization: LightroomArchiveFinalization,
}
/// Alias used by hosts that call the result a commit.
pub type LightroomCommit = LightroomCompletion;
/// A Lightroom import detached from the owner thread.
pub struct LightroomJob {
path: PathBuf,
update_existing: bool,
import: ImportJob,
token: LightroomLibraryToken,
archive_dir: Option<PathBuf>,
}
impl LightroomJob {
/// Snapshot session state only. This constructor performs no filesystem work.
pub fn new(s: &mut crate::Session, path: PathBuf, update_existing: bool) -> crate::Result<Self> {
let opts = ImportOptions { mode: ImportMode::Add, ..ImportOptions::default() };
let import = ImportJob::new(s, opts)?;
let archive_dir = s.library.as_ref().filter(|library| library.on_disk).map(|library| library.dir.join("Interop"));
Ok(Self { path, update_existing, import, token: LightroomLibraryToken::capture(s), archive_dir })
}
pub fn total(&self) -> usize {
self.import.total.load(Ordering::Relaxed)
}
pub fn done(&self) -> usize {
self.import.done.load(Ordering::Relaxed)
}
pub fn progress(&self) -> (usize, usize) {
(self.done(), self.total())
}
pub fn token(&self) -> &LightroomLibraryToken {
&self.token
}
pub fn total_atomic(&self) -> std::sync::Arc<AtomicUsize> {
std::sync::Arc::clone(&self.import.total)
}
pub fn done_atomic(&self) -> std::sync::Arc<AtomicUsize> {
std::sync::Arc::clone(&self.import.done)
}
/// Finish an owner commit's deferred archive-index write on a worker.
pub fn finish_archive(finalization: LightroomArchiveFinalization) -> crate::Result<()> {
finalization.finish()
}
/// Read/probe/stage all source data off the owner thread.
pub fn prepare(&mut self, cancel: &AtomicBool) -> crate::Result<PreparedLightroom> {
if cancel.load(Ordering::Relaxed) {
return Err(error("Lightroom import cancelled"));
}
self.import.total.store(1, Ordering::Relaxed);
self.import.done.store(0, Ordering::Relaxed);
let mut data = lightroom_catalog::read_with_progress(&self.path, cancel, &self.import.total, &self.import.done).map_err(error)?;
lightroom_catalog::validate_for_job(&data).map_err(error)?;
if cancel.load(Ordering::Relaxed) {
return Err(error("Lightroom import cancelled"));
}
self.import.done.fetch_add(1, Ordering::Relaxed);
let paths: Vec<String> = data
.photos
.iter()
.filter(|photo| lightroom_catalog::number_for_job(&photo.image, "masterImage") == 0)
.map(|photo| photo.path.clone())
.collect();
let missing_sources = data
.photos
.iter()
.filter(|photo| lightroom_catalog::number_for_job(&photo.image, "masterImage") == 0)
.filter(|photo| !Path::new(&photo.path).is_file())
.map(|photo| normalize(&photo.path))
.collect();
let normal = self.import.prepare(&paths, cancel);
if cancel.load(Ordering::Relaxed) {
normal.rollback();
return Err(error("Lightroom import cancelled"));
}
let archive_path = stage_archive(self.archive_dir.as_deref(), &mut data)?;
let index_path = self.archive_dir.as_ref().map(|dir| dir.join("lightroom-index.json"));
let index = lightroom_catalog::load_index(index_path.as_deref())?;
Ok(PreparedLightroom {
data,
normal,
index,
archive_path,
token: self.token.clone(),
missing_sources,
update_existing: self.update_existing,
now: self.import.now().to_string(),
cancelled: cancel.load(Ordering::Relaxed),
})
}
}
/// Apply prepared data on the owner thread. No source, archive or index filesystem work occurs.
pub fn commit_prepared(s: &mut crate::Session, mut prepared: PreparedLightroom) -> crate::Result<LightroomCompletion> {
if prepared.cancelled {
prepared.normal.rollback();
return Err(error("Lightroom import cancelled"));
}
if !prepared.token.matches(s) {
prepared.normal.rollback();
return Err(error("Lightroom import belongs to a different open library"));
}
// A command-scoped Lightroom import must never rewrite source XMP sidecars. Direct depth-0
// callers do not set this transient flag, so they cannot affect a later command.
if s.depth > 0 {
s.skip_auto_write = true;
}
let undo0 = s.undo.len();
prepared.normal.revalidate_add(s);
let options = ImportOptions { mode: ImportMode::Add, ..ImportOptions::default() };
let now = prepared.now.clone();
let files = crate::import::commit_prepared(s, &options, &now, prepared.normal)?;
let applied = lightroom_catalog::apply_prepared(
s,
prepared.data,
lightroom_catalog::ApplyPrepared {
update_existing: prepared.update_existing,
files,
index: &mut prepared.index,
archive_path: prepared.archive_path,
missing_sources: &prepared.missing_sources,
undo0,
},
)?;
let path = prepared.token.library_dir().map(|dir| dir.join("Interop/lightroom-index.json"));
let bytes = if path.is_some() { Some(serde_json::to_vec(&applied.index).map_err(|e| error(e.to_string()))?) } else { None };
Ok(LightroomCompletion { report: applied.report, finalization: LightroomArchiveFinalization { path, bytes } })
}
fn normalize(path: &str) -> String {
let path = path.replace('\\', "/");
if cfg!(windows) { path.to_lowercase() } else { path }
}
#[cfg(test)]
mod tests {
use super::*;
fn empty_data() -> CatalogImport {
CatalogImport {
source: "fixture.lrcat".into(),
photos: Vec::new(),
collections: Vec::new(),
members: Vec::new(),
collection_content: Vec::new(),
warnings: Vec::new(),
}
}
fn cancelled_prepared(s: &crate::Session) -> PreparedLightroom {
PreparedLightroom {
data: empty_data(),
normal: Prepared::default(),
index: ImportIndex::default(),
archive_path: None,
token: LightroomLibraryToken::capture(s),
missing_sources: HashSet::new(),
update_existing: false,
now: "2026-01-01T00:00:00".into(),
cancelled: true,
}
}
fn prepared_for(s: &crate::Session, path: &str, rating: i64) -> PreparedLightroom {
let mut image = std::collections::BTreeMap::new();
image.insert("id_local".into(), serde_json::json!(1));
image.insert("rating".into(), serde_json::json!(rating));
image.insert("pick".into(), serde_json::json!(0));
image.insert("fileFormat".into(), serde_json::json!("JPG"));
image.insert("fileWidth".into(), serde_json::json!(1));
image.insert("fileHeight".into(), serde_json::json!(1));
PreparedLightroom {
data: CatalogImport {
source: "fixture.lrcat".into(),
photos: vec![lightroom_catalog::CatalogPhoto {
source_id: 1,
uuid: "fixture-photo".into(),
path: path.into(),
image,
settings: String::new(),
xmp: String::new(),
keywords: Vec::new(),
history: Vec::new(),
snapshots: Vec::new(),
}],
collections: Vec::new(),
members: Vec::new(),
collection_content: Vec::new(),
warnings: Vec::new(),
},
normal: Prepared::default(),
index: ImportIndex::default(),
archive_path: None,
token: LightroomLibraryToken::capture(s),
missing_sources: [normalize(path)].into_iter().collect(),
update_existing: false,
now: "2026-01-01T00:00:00".into(),
cancelled: false,
}
}
#[test]
fn cancellation_before_commit_leaves_owner_catalog_unchanged() {
let mut session = crate::Session::new();
let before = session.catalog.to_snapshot();
let prepared = cancelled_prepared(&session);
assert!(commit_prepared(&mut session, prepared).is_err());
assert_eq!(session.catalog.to_snapshot(), before);
assert!(session.undo.is_empty());
}
#[test]
fn library_identity_rejects_late_completion_after_reopen() {
let mut session = crate::Session::new();
let token = LightroomLibraryToken::capture(&session);
assert!(token.matches(&session));
session.library_identity = Arc::new(());
assert!(!token.matches(&session));
}
#[test]
fn prepared_reimport_preserves_current_edit_and_uses_one_undo_per_import() {
let mut session = crate::Session::new();
let path = "/missing/lightroom-job-edit.jpg";
let prepared = prepared_for(&session, path, 4);
let first = commit_prepared(&mut session, prepared).unwrap();
assert_eq!(first.report["mapping"]["1"].as_u64(), Some(1));
assert_eq!(session.undo.len(), 1);
let id = lightcraft_catalog::PhotoId(1);
let mut personal = lightcraft_develop::DevelopSettings::default();
personal.light.exposure = 1.5;
session.set_develop(id, personal, "Personal").unwrap();
let before = session.catalog.photo(id).map(|photo| photo.develop.clone());
let prepared = prepared_for(&session, path, 5);
commit_prepared(&mut session, prepared).unwrap();
assert_eq!(session.catalog.photo(id).map(|photo| photo.develop.clone()), before);
assert_eq!(session.catalog.photo(id).map(|photo| photo.rating), Some(5));
assert_eq!(session.undo.len(), 3);
}
#[test]
fn oversized_history_archive_keeps_prepared_import_successful_with_warning() {
let archive_dir = std::env::temp_dir().join(format!("lightcraft-lr-prepared-scale-{}", std::process::id()));
let mut data = empty_data();
data.photos = (0..5_000)
.map(|source_id| {
let image = std::collections::BTreeMap::from([
("id_local".into(), serde_json::json!(source_id)),
("fileFormat".into(), serde_json::json!("JPG")),
("fileWidth".into(), serde_json::json!(1)),
("fileHeight".into(), serde_json::json!(1)),
]);
lightroom_catalog::CatalogPhoto {
source_id,
uuid: format!("scale-{source_id}"),
path: format!("/missing/scale-{source_id}.jpg"),
image,
settings: String::new(),
xmp: String::new(),
keywords: Vec::new(),
history: vec![std::collections::BTreeMap::from([(
"text".into(),
serde_json::json!(format!("history-{source_id}-{}", "x".repeat(8 * 1024))),
)])],
snapshots: Vec::new(),
}
})
.collect();
let archive_path = stage_archive(Some(&archive_dir), &mut data).unwrap();
assert!(archive_path.is_none());
let mut session = crate::Session::new();
let token = LightroomLibraryToken::capture(&session);
let prepared = PreparedLightroom {
data,
normal: Prepared::default(),
index: ImportIndex::default(),
archive_path,
token,
missing_sources: HashSet::new(),
update_existing: false,
now: "2026-01-01T00:00:00".into(),
cancelled: false,
};
let completion = commit_prepared(&mut session, prepared).unwrap();
assert_eq!(completion.report["photos"], 5_000);
assert!(
completion.report["warnings"]
.as_array()
.is_some_and(|warnings| warnings.iter().any(|warning| warning == "Lightroom source archive exceeded 32 MiB; archive skipped"))
);
assert_eq!(session.catalog.len(), 5_000);
std::fs::remove_dir_all(archive_dir).unwrap();
}
#[test]
fn execute_fn_import_does_not_rewrite_source_sidecar() {
let dir = std::env::temp_dir().join(format!("lightcraft-lr-job-sidecar-{}", std::process::id()));
let path = dir.join("source.jpg");
let sidecar = dir.join("source.xmp");
std::fs::create_dir_all(&dir).unwrap();
std::fs::write(&path, b"fixture").unwrap();
std::fs::write(&sidecar, b"original sidecar").unwrap();
let original = std::fs::read(&sidecar).unwrap();
let mut session = crate::Session::new();
session.xmp.auto_write = true;
let result = session.execute_fn("library.importLightroom", |s| {
let prepared = prepared_for(s, &path.to_string_lossy(), 5);
let completion = commit_prepared(s, prepared)?;
Ok(completion.report)
});
assert!(result.is_ok());
assert_eq!(std::fs::read(&sidecar).unwrap(), original);
std::fs::remove_dir_all(dir).unwrap();
}
}
File diff suppressed because it is too large. Load diff
+220 -13
View File
@@ -62,7 +62,7 @@ impl SettingsHashes {
}
/// Bump when the pipeline's output changes, to invalidate cached thumbnails.
pub const RENDER_CACHE_VERSION: u64 = 12;
pub const RENDER_CACHE_VERSION: u64 = 16;
/// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache).
pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512];
@@ -104,6 +104,20 @@ impl SourceLevel {
}
}
/// Pixels along the source's long edge that an output `out_long` pixels long needs: a crop shows
/// only part of the photo, so its pixels come from a source that many times larger (the source
/// level is chosen from this, or a tight crop shown at its own pixels would be a stretched preview).
fn source_edge_needed(p: &Photo, settings: &DevelopSettings, apply_crop: bool, out_long: usize) -> usize {
let (w, h) = (f64::from(p.width.max(1)), f64::from(p.height.max(1)));
let r = settings.crop.geometry.rect;
let shown = if apply_crop { (r.width() * w).max(r.height() * h) } else { w.max(h) };
if !shown.is_finite() || shown < 1.0 {
return out_long;
}
let needed = out_long as f64 * w.max(h) / shown;
if needed.is_finite() { needed.min(usize::MAX as f64 / 2.0) as usize } else { out_long }
}
/// Decodes a file into a linear Rec.2020 image no larger than `max_edge` (set by the app).
pub type FileLoader = Arc<dyn Fn(&str, usize) -> Result<(Rgb32f, SourceInfo), String> + Send + Sync>;
@@ -129,7 +143,7 @@ impl DecodedSource {
/// What to render these pixels against: the decoder's facts, else `header` (the catalog's)
/// with any stored camera tone curve.
pub fn info_or(&self, header: SourceInfo) -> SourceInfo {
self.info.unwrap_or(SourceInfo { camera_tone: self.camera_tone.or(header.camera_tone), ..header })
self.info.clone().unwrap_or_else(|| SourceInfo { camera_tone: self.camera_tone.or(header.camera_tone), ..header })
}
}
@@ -239,8 +253,9 @@ fn rendered_budget(share: usize) -> usize {
RENDERED_MEM_BYTES.min(share / 4)
}
fn source_bytes(img: &Rgb32f) -> usize {
img.data.len() * 12 + 64 + std::mem::size_of::<SourceInfo>()
fn source_bytes(s: &DecodedSource) -> usize {
let mattes = s.info.as_ref().and_then(|i| i.mattes.as_ref()).map_or(0, |m| m.bytes());
s.image.data.len() * 12 + 64 + std::mem::size_of::<SourceInfo>() + mattes
}
impl MediaCache {
@@ -305,8 +320,8 @@ impl MediaCache {
let Some((tick, which)) = candidates.into_iter().flatten().min() else { break };
let freed = match which {
0 => self.thumbs.pop_oldest(),
1 => self.previews.iter().position(|e| e.2 == tick).map(|i| source_bytes(&self.previews.remove(i).1.image)),
2 => self.full.take().map(|e| source_bytes(&e.1.image)),
1 => self.previews.iter().position(|e| e.2 == tick).map(|i| source_bytes(&self.previews.remove(i).1)),
2 => self.full.take().map(|e| source_bytes(&e.1)),
_ => self.rendered.evict_oldest(),
};
match freed {
@@ -358,7 +373,7 @@ impl MediaCache {
let tick = lightcraft_preview::next_tick();
match level {
SourceLevel::Thumb => {
let cost = source_bytes(&img.image);
let cost = source_bytes(&img);
self.thumbs.insert(id, img, cost);
}
SourceLevel::Preview => {
@@ -389,11 +404,20 @@ impl MediaCache {
(self.thumbs.len(), self.thumbs.cost())
}
/// Whether the decoded source of `id` at `level` is held.
pub fn has_source(&self, id: PhotoId, level: SourceLevel) -> bool {
match level {
SourceLevel::Thumb => self.thumbs.contains(&id),
SourceLevel::Preview => self.previews.iter().any(|e| e.0 == id),
SourceLevel::Full => self.full.as_ref().is_some_and(|e| e.0 == id),
}
}
/// Decoded sources held: (thumbnail level, preview level, full size).
pub fn usage(&self) -> (crate::memory::Usage, crate::memory::Usage, crate::memory::Usage) {
use crate::memory::Usage;
let previews = Usage::new(self.previews.len(), self.previews.iter().map(|e| source_bytes(&e.1.image)).sum());
let full = self.full.as_ref().map(|e| Usage::new(1, source_bytes(&e.1.image))).unwrap_or_default();
let previews = Usage::new(self.previews.len(), self.previews.iter().map(|e| source_bytes(&e.1)).sum());
let full = self.full.as_ref().map(|e| Usage::new(1, source_bytes(&e.1))).unwrap_or_default();
(Usage::new(self.thumbs.len(), self.thumbs.cost()), previews, full)
}
@@ -718,9 +742,13 @@ impl crate::Session {
thumb_bucket: Option<usize>,
) -> Option<RenderJob> {
let p = self.catalog.photo(id)?.clone();
let level = SourceLevel::for_size(max_w.max(max_h));
let source = self.media.source_ref(&p, level);
let settings = if before { Arc::new(self.before_settings(&p)) } else { p.develop.clone() };
let level = SourceLevel::for_size(if thumb_bucket.is_some() {
max_w.max(max_h) // (grid thumbnails of tight crops stay cheap)
} else {
source_edge_needed(&p, &settings, apply_crop, max_w.max(max_h))
});
let source = self.media.source_ref(&p, level);
let request = RenderRequest { apply_crop, ..RenderRequest::fit(max_w, max_h) };
// the photo id is part of the key: two photos with the same settings and size must not
// share a result (a view slot showing photo A would otherwise look current for photo B)
@@ -881,6 +909,69 @@ impl crate::Session {
Some(job)
}
/// A render of one window of the loupe's frame, for a view zoomed past what one whole-frame
/// render can hold: the frame is `full_w × full_h` (the size the loupe would draw it at) and
/// the result is `window`'s pixels of it, at that scale, from the source level that size needs.
/// Nothing is cached here (a window is only worth keeping while it is on screen). `None`: no
/// such photo, or the window reads (spots, Auto Mask strokes) more than one render can hold.
pub fn region_job(
&mut self,
id: PhotoId,
full_w: usize,
full_h: usize,
window: lightcraft_pipeline::PixelWindow,
apply_crop: bool,
) -> Option<RenderJob> {
self.region_job_of(id, full_w, full_h, window, apply_crop, false)
}
/// [`Self::region_job`] of the photo without its edits (its crop kept): the Before side.
pub fn region_job_before(
&mut self,
id: PhotoId,
full_w: usize,
full_h: usize,
window: lightcraft_pipeline::PixelWindow,
apply_crop: bool,
) -> Option<RenderJob> {
self.region_job_of(id, full_w, full_h, window, apply_crop, true)
}
fn region_job_of(
&mut self,
id: PhotoId,
full_w: usize,
full_h: usize,
window: lightcraft_pipeline::PixelWindow,
apply_crop: bool,
before: bool,
) -> Option<RenderJob> {
let mut job = self.render_job(id, full_w, full_h, before, apply_crop)?;
// what the window's spots and Auto Mask strokes read must fit in one render: else the
// caller keeps the whole-frame render (a window alone would come out wrong)
let p = self.catalog.photo(id)?;
let frame = lightcraft_pipeline::geometry::Frame::with_lens(
p.width.max(1) as usize,
p.height.max(1) as usize,
&job.settings,
apply_crop,
p.embedded_lens.as_ref(),
);
let ppl = frame.px_per_long(full_w);
lightcraft_pipeline::spots::window_for_reads_checked(&job.settings, &frame, full_w, full_h, ppl, window.clamped(full_w, full_h))?;
job.request.window = Some(window);
job.key = Hasher128::new()
.u64(job.key)
.str("window")
.u64(window.x as u64)
.u64(window.y as u64)
.u64(window.w as u64)
.u64(window.h as u64)
.finish()
.0 as u64;
Some(job)
}
/// The embedded preview of an unedited raw (path, loader), when the app installed a loader.
fn embedded_of(&self, p: &Photo) -> Option<(String, PreviewLoader)> {
match (&p.source, &self.media.preview_loader) {
@@ -889,6 +980,17 @@ impl crate::Session {
}
}
/// The camera's embedded preview of a raw file, for the face scan. Faces are read from the unedited picture, so the
/// photo's edits do not matter, and the camera's own JPEG (oriented, display-ready) costs a fraction of decoding the
/// raw. `None` for anything else, and where the host installed no loader.
#[cfg(not(target_arch = "wasm32"))]
pub(crate) fn scan_preview_of(&self, p: &Photo) -> Option<(String, PreviewLoader)> {
match (&p.source, &self.media.preview_loader) {
(Source::File { path }, Some(l)) if p.kind == MediaKind::Raw => Some((path.clone(), l.clone())),
_ => None,
}
}
/// Something to show in the loupe right away for `id` (see [`QuickJob`]): its cached view
/// render, else the embedded preview of an unedited raw, else a cached or fresh thumbnail.
pub fn quick_view_job(&mut self, id: PhotoId, max_edge: usize, apply_crop: bool) -> Option<QuickJob> {
@@ -1120,20 +1222,21 @@ mod tests {
let mut s = crate::Session::with_demo();
let id = s.active().unwrap();
let mut job = s.render_job(id, 64, 64, false, true).unwrap();
let loaded_info = info.clone();
job.source = SourceRef::File {
path: "synthetic.arw".into(),
max_edge: 64,
loader: Some(Arc::new(move |_, _| {
let mut image = Rgb32f::new(64, 64);
image.data.fill([0.1; 3]);
Ok((image, info))
Ok((image, loaded_info.clone()))
})),
fallback: None,
};
job.info = SourceInfo::default(); // Header facts cannot override decoder facts.
job.settings = Arc::new(DevelopSettings::default());
let r = job.clone().run();
assert_eq!(r.loaded.as_ref().unwrap().info, Some(info));
assert_eq!(r.loaded.as_ref().unwrap().info.as_ref(), Some(&info));
let expected = lightcraft_pipeline::render(&r.loaded.as_ref().unwrap().image, &info, &job.settings, &job.request);
assert_eq!(r.rendered.as_ref().unwrap().image.data, expected.image.data);
s.accept(&r);
@@ -1218,6 +1321,110 @@ mod tests {
assert_ne!(a.key, b.key);
}
// Issue #323: a window of a 1:1 view is its own job: the window's pixels, at the zoom scale,
// from the original, with a key of its own
#[test]
fn region_jobs_render_a_window_of_the_zoomed_frame() {
use lightcraft_pipeline::PixelWindow;
let mut s = crate::Session::with_demo();
let p = s.catalog.photos().next().unwrap().clone();
let (w, h) = (p.width as usize, p.height as usize);
assert!(w.max(h) > 2560, "demo photos are camera-sized");
let win = PixelWindow { x: w / 3, y: h / 3, w: 300, h: 200 };
let job = s.region_job(p.id, w, h, win, true).unwrap();
assert_eq!(job.level, SourceLevel::Full, "a 1:1 window reads the original");
let img = job.run().rendered.unwrap().image;
assert_eq!((img.width, img.height), (300, 200));
// other windows, and the whole-frame job, never share a key
let other = s.region_job(p.id, w, h, PixelWindow { x: win.x + 64, ..win }, true).unwrap();
let whole = s.render_job(p.id, w, h, false, true).unwrap();
let again = s.region_job(p.id, w, h, win, true).unwrap();
assert_ne!(other.key, again.key);
assert_ne!(whole.key, again.key);
assert_eq!(s.region_job(p.id, w, h, win, true).unwrap().key, again.key, "the same window keeps its key");
// a window never writes the photo's view preview or the thumbnail cache
assert!(again.view_cache.is_none() && again.cache.is_none());
}
// Issue #323: a tight crop shown at its own pixels needs the original, not the 2560 px preview
#[test]
fn a_tight_crop_reads_the_source_level_its_pixels_need() {
let mut s = crate::Session::with_demo();
let p = s.catalog.photos().next().unwrap().clone();
let long = p.width.max(p.height) as usize;
assert!(long > 2560);
let id = p.id;
let mut d = (*s.develop_of(id).unwrap()).clone();
// the uncropped photo at 1600 px: the preview is plenty
assert_eq!(s.render_job(id, 1600, 1600, false, true).unwrap().level, SourceLevel::Preview);
// a crop to 30 % of each side shown 1600 px long needs 1600 / 0.3 px of the source
d.crop.geometry.rect = lightcraft_geom::Rect { x0: 0.2, y0: 0.2, x1: 0.5, y1: 0.5 };
s.set_develop(id, d, "Crop").unwrap();
assert_eq!(s.render_job(id, 1600, 1600, false, true).unwrap().level, SourceLevel::Full);
// …but with the crop tool open (the whole photo shown) it does not
assert_eq!(s.render_job(id, 1600, 1600, false, false).unwrap().level, SourceLevel::Preview);
// and a crop shown small enough still reads the preview
assert_eq!(s.render_job(id, 600, 600, false, true).unwrap().level, SourceLevel::Preview);
}
// a window whose spot reads from further away than a render can hold is refused
#[test]
fn a_region_job_is_refused_when_a_spot_reads_beyond_what_fits() {
use lightcraft_pipeline::PixelWindow;
let mut s = crate::Session::with_demo();
let id = s.active().unwrap();
let p = s.catalog.photo(id).unwrap().clone();
let (w, h) = (p.width as usize * 8, p.height as usize * 8);
let mut d = (*s.develop_of(id).unwrap()).clone();
d.spots.push(lightcraft_develop::Spot {
points: vec![lightcraft_geom::Point::new(0.2, 0.5)],
size: 0.01,
source_offset: Some(lightcraft_geom::Point::new(0.6, 0.0)),
..Default::default()
});
s.set_develop(id, d, "Spot").unwrap();
let win = PixelWindow { x: (w as f64 * 0.2) as usize - 100, y: h / 2 - 100, w: 400, h: 300 };
assert!(s.region_job(id, w, h, win, true).is_none());
// a window elsewhere is fine
assert!(s.region_job(id, w, h, PixelWindow { x: 5000, y: 5000, w: 400, h: 300 }, true).is_some());
}
// Issue #323: the Before side of a Before/After view at 1:1 is a window too, of the photo
// without its edits (the crop is kept), with a key of its own
#[test]
fn a_before_window_shows_the_unedited_look() {
use lightcraft_pipeline::PixelWindow;
let mut s = crate::Session::with_demo();
let id = s.active().unwrap();
let p = s.catalog.photo(id).unwrap().clone();
let (w, h) = (p.width as usize, p.height as usize);
let mut d = (*s.develop_of(id).unwrap()).clone();
d.light.exposure = 2.0;
s.set_develop(id, d, "Exposure").unwrap();
let win = PixelWindow { x: w / 3, y: h / 3, w: 200, h: 150 };
let after = s.region_job(id, w, h, win, true).unwrap();
let before = s.region_job_before(id, w, h, win, true).unwrap();
assert_ne!(after.key, before.key);
let (a, b) = (after.run().rendered.unwrap().image, before.run().rendered.unwrap().image);
assert_eq!((a.width, a.height), (b.width, b.height));
let mean = |i: &Rgba8| i.data.iter().map(|p| p[1] as f64).sum::<f64>() / i.data.len() as f64;
assert!(mean(&a) > mean(&b) + 10.0, "two stops brighter after: {} vs {}", mean(&a), mean(&b));
}
#[test]
fn has_source_follows_what_the_session_accepted() {
let mut s = crate::Session::with_demo();
let id = s.active().unwrap();
let p = s.catalog.photo(id).unwrap().clone();
let long = p.width.max(p.height) as usize;
assert!(!s.media.has_source(id, SourceLevel::Full));
let r = s.render_job(id, long, long, false, true).unwrap().run();
s.accept(&r);
assert!(s.media.has_source(id, SourceLevel::Full));
assert!(!s.media.has_source(id, SourceLevel::Preview));
assert!(!s.media.has_source(PhotoId(id.0 + 1), SourceLevel::Full));
}
#[test]
fn variant_jobs_have_their_own_keys_and_hit_the_cache() {
let mut s = crate::Session::with_demo();
+21 -2
View File
@@ -100,6 +100,7 @@ impl Lua {
struct LuaParser<'a> {
s: &'a [u8],
i: usize,
depth: usize,
}
impl LuaParser<'_> {
@@ -210,6 +211,15 @@ impl LuaParser<'_> {
std::str::from_utf8(&self.s[start..self.i]).ok().and_then(|t| t.parse().ok()).map_or_else(|| self.err("bad number"), Ok)
}
fn value(&mut self) -> Result<Lua, String> {
if self.depth >= 64 {
return self.err("Lua settings exceed nesting limit");
}
self.depth += 1;
let result = self.value_inner();
self.depth -= 1;
result
}
fn value_inner(&mut self) -> Result<Lua, String> {
self.skip_ws();
let Some(&c) = self.s.get(self.i) else { return self.err("unexpected end") };
match c {
@@ -315,7 +325,10 @@ impl LuaParser<'_> {
/// Parse a Lua table literal, optionally preceded by `name =` or `return`.
pub fn parse_lua(text: &str) -> Result<Lua, String> {
let mut p = LuaParser { s: text.as_bytes(), i: 0 };
if text.len() > 16 << 20 {
return Err("Lua settings exceed 16 MiB".into());
}
let mut p = LuaParser { s: text.as_bytes(), i: 0, depth: 0 };
p.skip_ws();
let save = p.i;
match p.ident().as_deref() {
@@ -347,6 +360,12 @@ pub fn lrtemplate_props(text: &str) -> Result<(Option<String>, Props, crate::crs
let root = parse_lua(text)?;
let title = root.get("title").or_else(|| root.get("internalName")).and_then(Lua::str).map(delocalize).filter(|t| !t.trim().is_empty());
let settings = root.get("value").and_then(|v| v.get("settings")).ok_or("no develop settings in this template")?;
let (props, values) = lua_settings_props(settings)?;
Ok((title, props, values))
}
/// Data-only Lua settings shared by preset and native catalog import. Never executes Lua.
pub(crate) fn lua_settings_props(settings: &Lua) -> Result<(Props, crate::crs_masks::Values), String> {
let Lua::Table(_, fields) = settings else { return Err("no develop settings in this template".into()) };
let mut props = Props::new();
let mut values = crate::crs_masks::Values::new();
@@ -374,7 +393,7 @@ pub fn lrtemplate_props(text: &str) -> Result<(Option<String>, Props, crate::crs
};
props.insert(key, vals);
}
Ok((title, props, values))
Ok((props, values))
}
// ------------------------------------------------------------------------------- XMP in files
+13 -19
View File
@@ -83,28 +83,22 @@ pub fn parse_preset_file(name: &str, bytes: &[u8]) -> Result<Vec<Preset>, String
Ok(out)
}
/// Expand files/folders (recursively) into preset files (`.lcpreset`, `.xmp`).
/// Expand files/folders (recursively, bounded: see [`crate::walk`]) into preset files
/// (`.lcpreset`, `.xmp`).
pub fn expand_preset_paths(paths: &[String]) -> Vec<String> {
fn walk(p: &Path, out: &mut Vec<String>, top: bool) {
if p.is_dir() {
let Ok(rd) = std::fs::read_dir(p) else { return };
let mut v: Vec<_> = rd.flatten().map(|e| e.path()).collect();
v.sort();
for c in v {
if !c.file_name().is_some_and(|n| n.to_string_lossy().starts_with('.')) {
walk(&c, out, false);
}
}
} else {
let ext = p.extension().map(|e| e.to_string_lossy().to_ascii_lowercase()).unwrap_or_default();
if top || ext == LCPRESET_EXT || ["xmp", "lrtemplate", "zip", "lmp", "mplumpack"].contains(&ext.as_str()) {
out.push(p.to_string_lossy().to_string());
}
}
}
let preset_file = |p: &Path| {
let ext = p.extension().map(|e| e.to_string_lossy().to_ascii_lowercase()).unwrap_or_default();
ext == LCPRESET_EXT || ["xmp", "lrtemplate", "zip", "lmp", "mplumpack"].contains(&ext.as_str())
};
let mut out = Vec::new();
for p in paths {
walk(Path::new(p), &mut out, true);
let path = Path::new(p);
if path.is_dir() {
let w = crate::walk::files_in(path, None, crate::walk::Limits::default(), preset_file);
out.extend(w.files.into_iter().map(|f| f.to_string_lossy().to_string()));
} else {
out.push(p.clone());
}
}
out
}
+12 -1
View File
@@ -157,7 +157,14 @@ impl Segmenter {
/// Whether this build can compute AI masks at all.
pub const AVAILABLE: bool = cfg!(feature = "sam");
/// Whether the model's files are in place.
pub fn remote_endpoint(&self) -> Option<String> {
#[cfg(feature = "sam")]
return lightcraft_segment::remote::configured();
#[cfg(not(feature = "sam"))]
None
}
/// Whether the local model files are in place.
pub fn installed(&self) -> bool {
#[cfg(feature = "sam")]
if let Some(dir) = &self.dir {
@@ -230,6 +237,10 @@ impl Segmenter {
if !Self::AVAILABLE {
return Err("AI masks are not available in this build".into());
}
#[cfg(feature = "sam")]
if lightcraft_segment::remote::configured().is_some() {
return Ok(PathBuf::new());
}
let dir = self.dir.clone().ok_or("no folder is set for the SAM 3 model")?;
if self.installed() {
return Ok(dir);
+64 -17
View File
@@ -17,7 +17,10 @@ use std::sync::mpsc::{self, RecvTimeoutError};
use std::time::Duration;
use lightcraft_develop::SegMask;
use lightcraft_segment::{Encoded, Sam3};
use lightcraft_segment::{
Encoded, Sam3,
remote::{self, Operation, Request, text_logits},
};
use super::{Input, Prompt, Tag};
use crate::media::RenderJob;
@@ -175,6 +178,7 @@ impl Worker {
struct State {
model: Option<(PathBuf, Sam3)>,
cache: Option<(u64, Encoded)>,
remote: Option<remote::Client>,
}
/// Whether `later` makes `job` pointless (it asks for the same selection again).
@@ -195,7 +199,7 @@ fn run(rx: &mpsc::Receiver<Job>, shared: &Shared, idle: Duration) {
match rx.recv_timeout(idle) {
Ok(j) => queue.push_back(j),
Err(RecvTimeoutError::Timeout) => {
if state.model.is_some() {
if state.model.is_some() || state.remote.is_some() {
state = State::default();
shared.loaded.store(false, Ordering::SeqCst);
log::info!("SAM 3 unloaded after {} minutes without use", idle.as_secs() / 60);
@@ -210,8 +214,11 @@ fn run(rx: &mpsc::Receiver<Job>, shared: &Shared, idle: Duration) {
}
let Some(mut job) = queue.pop_front() else { continue };
let is_detail = matches!(job.kind, Kind::Detail { .. });
let _done = Done(if is_detail { &shared.detail } else { &shared.pending });
// counts the job as pending until just before its reply goes out: a caller woken by the
// reply must not still see the worker busy with it
let done = Done(if is_detail { &shared.detail } else { &shared.pending });
if queue.iter().any(|later| supersedes(later, &job)) {
drop(done);
let _ = job.reply.send(Outcome { tag: job.tag, result: Ok(None), superseded: true });
continue;
}
@@ -228,7 +235,8 @@ fn run(rx: &mpsc::Receiver<Job>, shared: &Shared, idle: Duration) {
if result.is_err() && !is_detail {
state.cache = None;
}
shared.loaded.store(state.model.is_some(), Ordering::SeqCst);
shared.loaded.store(state.model.is_some() || state.remote.is_some(), Ordering::SeqCst);
drop(done);
let _ = job.reply.send(Outcome { tag: job.tag, result, superseded: false });
}
}
@@ -280,29 +288,68 @@ fn encoded<'a>(state: &'a mut State, shared: &Shared, job: &mut Job) -> Result<(
log::info!("SAM 3 encoded {}×{} in {:?}", input.w, input.h, t.elapsed());
state.cache = Some((job.key, enc));
}
let State { model, cache } = state;
let State { model, cache, .. } = state;
match (model.as_mut(), cache.as_mut()) {
(Some((_, m)), Some((_, e))) => Ok((m, e)),
_ => Err("the model is not ready".into()),
}
}
/// Every phrase of `text` ("car, road"), merged (the per-pixel maximum); `None` when nothing
/// matches.
fn text_logits(model: &mut Sam3, enc: &mut Encoded, text: &str) -> Result<Option<Vec<f32>>, String> {
let mut merged: Option<Vec<f32>> = None;
for phrase in text.split([',', ';']).map(str::trim).filter(|p| !p.is_empty()) {
if let Some(p) = model.segment_text(enc, phrase, 0.5).map_err(|e| e.to_string())? {
merged = Some(match merged {
None => p.logits,
Some(m) => m.iter().zip(&p.logits).map(|(a, b)| a.max(*b)).collect(),
});
}
fn remote_job(state: &mut State, shared: &Shared, job: &mut Job, endpoint: &str) -> Result<Option<SegMask>, String> {
let _busy = Flag::raise(&shared.encoding);
if state.remote.is_none() {
state.remote = Some(remote::Client::connect(endpoint)?);
}
Ok(merged)
let client = state.remote.as_mut().ok_or("remote SAM unavailable")?;
let mut region = None;
let operation = match std::mem::replace(&mut job.kind, Kind::Prepare) {
Kind::Prepare => Operation::Prepare,
Kind::Text(text) => Operation::Text(text),
Kind::Clicks(clicks) => Operation::Clicks(clicks.iter().map(|c| [c.x, c.y, if c.positive { 1.0 } else { 0.0 }]).collect()),
Kind::Detail { prompt, region: r } => {
region = Some(r);
let (rw, rh) = (r[2] - r[0], r[3] - r[1]);
if !(rw > 0.0 && rh > 0.0) {
return Ok(None);
}
match prompt {
Prompt::Text(text) => Operation::Text(text),
Prompt::Clicks(clicks) => {
let inside: Vec<_> = clicks
.iter()
.map(|c| [((c.at.x - r[0]) / rw) as f32, ((c.at.y - r[1]) / rh) as f32, if c.include { 1.0 } else { 0.0 }])
.filter(|p| (0.0..=1.0).contains(&p[0]) && (0.0..=1.0).contains(&p[1]))
.collect();
if !inside.iter().any(|p| p[2] == 1.0) {
return Ok(None);
}
Operation::Clicks(inside)
}
}
}
#[cfg(test)]
Kind::Panic => return Err("test panic request".into()),
};
let input = if region.is_some() || client.key != Some(job.key) { render(job)? } else { Input { rgb: Vec::new(), w: 0, h: 0 } };
let request = Request { width: input.w, height: input.h, temporary: region.is_some(), operation };
let logits = client.request(&request, &input.rgb)?;
if region.is_none() {
client.key = Some(job.key);
}
Ok(logits.map(|l| match region {
Some(r) => SegMask::from_logits_in(lightcraft_segment::MASK_SIDE, &l, r),
None => SegMask::from_logits(lightcraft_segment::MASK_SIDE, &l),
}))
}
fn run_job(state: &mut State, shared: &Shared, job: &mut Job) -> Result<Option<SegMask>, String> {
if let Some(endpoint) = remote::configured() {
let result = remote_job(state, shared, job, &endpoint);
if result.is_err() {
state.remote = None;
}
return result.map_err(|e| format!("Remote SAM 3 failed: {e}. Check the worker and SSH tunnel, then retry."));
}
let side = lightcraft_segment::MASK_SIDE;
match std::mem::replace(&mut job.kind, Kind::Prepare) {
Kind::Prepare => {
+70 -18
View File
@@ -14,24 +14,27 @@ fn active_dev(s: &Session) -> lightcraft_develop::DevelopSettings {
fn changing_one_wb_control_resolves_as_shot_without_stale_tint() {
use lightcraft_catalog::{Photo, PhotoId, Source};
use lightcraft_develop::{DevelopSettings, WbMode};
let mut s = demo();
let id = PhotoId(100);
let mut p = Photo::new(id, Source::File { path: "synthetic.arw".into() }, "synthetic.arw", "ARW", 16, 16, "");
// A catalog made by the old generic-matrix Kelvin inference.
p.as_shot_wb = Some((6829.0, -127.0));
p.develop = std::sync::Arc::new(DevelopSettings::for_raw(6829.0, -127.0));
s.catalog.apply(lightcraft_catalog::Op::AddPhoto { photo: Box::new(p) }).unwrap();
s.execute("library.select", &json!({"ids": [100], "active": 100})).unwrap();
s.execute("develop.set", &json!({"control": "wb.temp", "value": 8000})).unwrap();
let d = active_dev(&s);
assert_eq!(d.wb.mode, WbMode::Custom);
assert_eq!(d.wb.temp, 8000.0);
assert_eq!(d.wb.tint, 0.0, "untouched tint comes from the current as-shot reference");
s.execute("develop.wb", &json!({"mode": "asShot"})).unwrap();
s.execute("develop.set", &json!({"control": "wb.tint", "value": 10})).unwrap();
assert_eq!(active_dev(&s).wb.temp, 6500.0);
s.execute("develop.reset", &json!({})).unwrap();
assert_eq!((active_dev(&s).wb.temp, active_dev(&s).wb.tint), (6500.0, 0.0));
for format in ["ARW", "NEF", "NRW", "RAF", "CR2", "PEF"] {
let mut s = demo();
let id = PhotoId(100);
let name = format!("synthetic.{format}");
let mut p = Photo::new(id, Source::File { path: name.clone() }, &name, format, 16, 16, "");
// A catalog made by the old generic-matrix Kelvin inference.
p.as_shot_wb = Some((6829.0, -127.0));
p.develop = std::sync::Arc::new(DevelopSettings::for_raw(6829.0, -127.0));
s.catalog.apply(lightcraft_catalog::Op::AddPhoto { photo: Box::new(p) }).unwrap();
s.execute("library.select", &json!({"ids": [100], "active": 100})).unwrap();
s.execute("develop.set", &json!({"control": "wb.temp", "value": 8000})).unwrap();
let d = active_dev(&s);
assert_eq!(d.wb.mode, WbMode::Custom);
assert_eq!(d.wb.temp, 8000.0);
assert_eq!(d.wb.tint, 0.0, "{format}: untouched tint comes from the current as-shot reference");
s.execute("develop.wb", &json!({"mode": "asShot"})).unwrap();
s.execute("develop.set", &json!({"control": "wb.tint", "value": 10})).unwrap();
assert_eq!(active_dev(&s).wb.temp, 6500.0, "{format}");
s.execute("develop.reset", &json!({})).unwrap();
assert_eq!((active_dev(&s).wb.temp, active_dev(&s).wb.tint), (6500.0, 0.0), "{format}");
}
}
#[test]
@@ -115,6 +118,29 @@ fn albums_crud() {
s.execute("album.delete", &json!({"id": f})).unwrap();
}
/// Given an album and a folder, when something is created inside each, then only the folder
/// accepts it: the sidebar only lists children of folders, so a child of a plain album would
/// be invisible.
#[test]
fn albums_and_smart_albums_are_created_only_inside_folders() {
let mut s = demo();
let folder = s.execute("album.create", &json!({"name": "Trips", "folder": true})).unwrap()["id"].as_u64().unwrap();
let album = s.execute("album.create", &json!({"name": "Best", "parent": folder})).unwrap()["id"].as_u64().unwrap();
for (cmd, params) in [
("album.create", json!({"name": "Nested", "parent": album})),
("album.create", json!({"name": "Nested", "folder": true, "parent": album})),
("album.createSmart", json!({"name": "Nested", "parent": album})),
("album.create", json!({"name": "Nested", "parent": 987_654})),
] {
assert!(s.execute(cmd, &params).is_err(), "{cmd} {params} must be refused");
}
let n = s.catalog.albums().count();
let sub = s.execute("album.create", &json!({"name": "Sub", "folder": true, "parent": folder})).unwrap()["id"].as_u64().unwrap();
let smart = s.execute("album.createSmart", &json!({"name": "Rated", "parent": sub})).unwrap()["id"].as_u64().unwrap();
assert_eq!(s.catalog.albums().count(), n + 2);
assert_eq!(s.catalog.album(lightcraft_catalog::AlbumId(smart)).unwrap().parent, Some(lightcraft_catalog::AlbumId(sub)));
}
#[test]
fn filters_and_delete_restore() {
let mut s = demo();
@@ -922,3 +948,29 @@ fn saved_presets_with_unknown_keys_still_export() {
assert_eq!(o.watermark.map(|w| w.text), Some("©".to_string()));
assert!(ExportOptions::from_params(&s.export_params(&json!({"preset": "old", "qualty": 80})).unwrap()).is_err());
}
/// Undo and redo bring the photo they change on screen (issue #293): after editing one photo and
/// moving on, Undo makes it the active photo again; a step that changes several photos leaves the
/// selection alone.
#[test]
fn undo_and_redo_show_the_photo_they_change() {
let mut s = demo();
let vis = s.visible_cloned();
let (a, b) = (vis[0], vis[1]);
s.execute("library.select", &json!({"ids": [a.0]})).unwrap();
s.execute("develop.set", &json!({"control": "light.exposure", "value": 1.0})).unwrap();
s.execute("library.select", &json!({"ids": [b.0]})).unwrap();
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.active(), Some(a), "undo shows the photo it changed");
assert_eq!(s.selection.ids, vec![a]);
s.execute("library.select", &json!({"ids": [b.0]})).unwrap();
s.execute("edit.redo", &json!({})).unwrap();
assert_eq!(s.active(), Some(a), "so does redo");
// rating both selected photos is one step over two photos: undoing it leaves the selection
s.execute("library.select", &json!({"ids": [a.0, b.0], "active": b.0})).unwrap();
s.execute("photo.rate", &json!({"rating": 3})).unwrap();
s.execute("library.select", &json!({"ids": [a.0, b.0], "active": a.0})).unwrap();
let before = s.selection.clone();
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.selection, before, "a step over several photos keeps the selection");
}
+159
View File
@@ -0,0 +1,159 @@
//! Ordering albums by hand: `album.reorder` puts an album (or folder) before a sibling, or last of
//! its kind, in one undo step, also when it changes folder; `album.sort` goes back to by name.
use lightcraft_catalog::AlbumId;
use serde_json::{Value, json};
use crate::Session;
fn make(s: &mut Session, name: &str, parent: Option<u64>, folder: bool) -> u64 {
let mut p = json!({"name": name, "folder": folder});
if let Some(parent) = parent {
p["parent"] = json!(parent);
}
s.execute("album.create", &p).unwrap()["id"].as_u64().unwrap()
}
fn names(s: &Session, parent: Option<u64>) -> Vec<String> {
s.catalog.album_children(parent.map(AlbumId)).iter().map(|a| a.name.clone()).collect()
}
fn reorder(s: &mut Session, id: u64, params: Value) -> Result<Value, String> {
let mut p = params;
p["id"] = json!(id);
s.execute("album.reorder", &p).map_err(|e| e.to_string())
}
/// Given three albums listed by name, when one is placed before another, then the list follows;
/// "no sibling" puts it last (among albums).
#[test]
fn an_album_is_placed_before_a_sibling_or_last() {
let mut s = Session::new();
let (a, b, c) = (make(&mut s, "A", None, false), make(&mut s, "B", None, false), make(&mut s, "C", None, false));
reorder(&mut s, c, json!({"before": a})).unwrap();
assert_eq!(names(&s, None), ["C", "A", "B"]);
assert!(s.catalog.album_children_are_ordered(None));
reorder(&mut s, c, json!({"before": null})).unwrap();
assert_eq!(names(&s, None), ["A", "B", "C"]);
reorder(&mut s, a, json!({})).unwrap();
assert_eq!(names(&s, None), ["B", "C", "A"], "no `before`: last");
assert_eq!(reorder(&mut s, b, json!({"before": b})).unwrap()["changed"], 0, "before itself is where it is");
assert_eq!(names(&s, None), ["B", "C", "A"]);
}
/// Folders stay ahead of albums: a folder is placed among folders, an album among albums.
#[test]
fn folders_and_albums_keep_to_their_own_group() {
let mut s = Session::new();
let (f1, f2) = (make(&mut s, "F1", None, true), make(&mut s, "F2", None, true));
let (a1, a2) = (make(&mut s, "A1", None, false), make(&mut s, "A2", None, false));
reorder(&mut s, f2, json!({"before": f1})).unwrap();
reorder(&mut s, a2, json!({"before": a1})).unwrap();
assert_eq!(names(&s, None), ["F2", "F1", "A2", "A1"]);
reorder(&mut s, f1, json!({"before": a1})).expect_err("a folder is not placed among albums");
reorder(&mut s, a1, json!({"before": f1})).expect_err("an album is not placed among folders");
reorder(&mut s, f2, json!({"before": null})).unwrap();
assert_eq!(names(&s, None), ["F1", "F2", "A2", "A1"], "last of the folders");
}
/// Placing an album in another folder moves it there, and one undo undoes both.
#[test]
fn placing_across_folders_moves_it_and_undoes_in_one_step() {
let mut s = Session::new();
let folder = make(&mut s, "Trips", None, true);
let (x, y) = (make(&mut s, "X", Some(folder), false), make(&mut s, "Y", Some(folder), false));
let loose = make(&mut s, "Loose", None, false);
reorder(&mut s, y, json!({"before": x})).unwrap();
assert_eq!(names(&s, Some(folder)), ["Y", "X"]);
reorder(&mut s, loose, json!({"parent": folder, "before": x})).unwrap();
assert_eq!(names(&s, Some(folder)), ["Y", "Loose", "X"]);
assert_eq!(s.catalog.album(AlbumId(loose)).unwrap().parent, Some(AlbumId(folder)));
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(names(&s, Some(folder)), ["Y", "X"]);
assert_eq!(s.catalog.album(AlbumId(loose)).unwrap().parent, None);
assert_eq!(s.catalog.album(AlbumId(loose)).unwrap().order, None);
}
/// Undo and redo of placing across folders, of sorting, and of moving a folder that has children.
#[test]
fn placing_and_sorting_undo_and_redo() {
let mut s = Session::new();
let top = make(&mut s, "Top", None, true);
let child = make(&mut s, "Child", Some(top), false);
let (a, b) = (make(&mut s, "A", None, false), make(&mut s, "B", None, false));
reorder(&mut s, b, json!({"before": a})).unwrap();
// a folder with children, placed inside another folder
let other = make(&mut s, "Other", None, true);
reorder(&mut s, top, json!({"parent": other})).unwrap();
assert_eq!(names(&s, Some(other)), ["Top"]);
assert_eq!(names(&s, Some(top)), ["Child"], "its children came along");
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.catalog.album(AlbumId(top)).unwrap().parent, None);
s.execute("edit.redo", &json!({})).unwrap();
assert_eq!(s.catalog.album(AlbumId(top)).unwrap().parent, Some(AlbumId(other)));
// sort, undo, redo
s.execute("album.sort", &json!({})).unwrap();
assert_eq!(names(&s, None), ["Other", "A", "B"]);
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(names(&s, None), ["Other", "B", "A"], "the hand order is back");
s.execute("edit.redo", &json!({})).unwrap();
assert_eq!(names(&s, None), ["Other", "A", "B"]);
assert_eq!(s.catalog.album(AlbumId(child)).unwrap().parent, Some(AlbumId(top)));
}
/// Bad requests change nothing: a `before` that is not in the target folder, a target that is not
/// a folder or lies inside the album, an unknown album.
#[test]
fn bad_placements_are_refused_and_change_nothing() {
let mut s = Session::new();
let f = make(&mut s, "F", None, true);
let inner = make(&mut s, "Inner", Some(f), true);
let (a, b) = (make(&mut s, "A", None, false), make(&mut s, "B", Some(f), false));
let before = s.catalog.to_snapshot();
reorder(&mut s, a, json!({"before": b})).expect_err("B is in F, not at the top level");
reorder(&mut s, a, json!({"parent": b})).expect_err("a plain album is no folder");
reorder(&mut s, f, json!({"parent": inner})).expect_err("a folder into itself");
reorder(&mut s, 987_654, json!({})).expect_err("no such album");
reorder(&mut s, a, json!({"before": 987_654})).expect_err("no such sibling");
assert_eq!(s.catalog.to_snapshot(), before, "nothing changed");
}
/// `album.sort` drops the hand order of one folder (and only that one): by name again.
#[test]
fn sorting_goes_back_to_by_name_for_one_folder() {
let mut s = Session::new();
let f = make(&mut s, "F", None, true);
let (x, y) = (make(&mut s, "X", Some(f), false), make(&mut s, "Y", Some(f), false));
let (a, b) = (make(&mut s, "A", None, false), make(&mut s, "B", None, false));
reorder(&mut s, y, json!({"before": x})).unwrap();
reorder(&mut s, b, json!({"before": a})).unwrap();
assert_eq!(names(&s, Some(f)), ["Y", "X"]);
assert_eq!(names(&s, None), ["F", "B", "A"]);
let r = s.execute("album.sort", &json!({"parent": f})).unwrap();
assert_eq!(r["changed"], 2);
assert_eq!(names(&s, Some(f)), ["X", "Y"]);
assert_eq!(names(&s, None), ["F", "B", "A"], "the top level keeps its order");
assert!(!s.catalog.album_children_are_ordered(Some(AlbumId(f))));
// nothing to do: no undo step is made
let r = s.execute("album.sort", &json!({"parent": f})).unwrap();
assert_eq!(r["changed"], 0);
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(names(&s, Some(f)), ["Y", "X"], "the one undo step was the sort");
let r = s.execute("album.sort", &json!({})).unwrap();
assert_eq!(r["changed"], 3, "every child of the top level had a place: F, B and A");
assert_eq!(names(&s, None), ["F", "A", "B"]);
}
/// Moving an album with `album.move` out of an ordered folder leaves the old neighbours' order.
#[test]
fn moving_out_of_an_ordered_folder_keeps_the_others_in_place() {
let mut s = Session::new();
let f = make(&mut s, "F", None, true);
let (x, y, z) = (make(&mut s, "X", Some(f), false), make(&mut s, "Y", Some(f), false), make(&mut s, "Z", Some(f), false));
reorder(&mut s, z, json!({"before": x})).unwrap();
assert_eq!(names(&s, Some(f)), ["Z", "X", "Y"]);
s.execute("album.move", &json!({"id": x, "parent": null})).unwrap();
assert_eq!(names(&s, Some(f)), ["Z", "Y"]);
assert_eq!(names(&s, None), ["F", "X"]);
let _ = y;
}
+586
View File
@@ -0,0 +1,586 @@
//! The `faces.*` model commands: list, inspect, install behind the licence, select, remove, the on/off
//! setting, and the refusals (no folder, licence not accepted, unsupported or hostile files and ids).
use std::path::PathBuf;
use serde_json::{Value, json};
use crate::Session;
fn temp(name: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("lc-facemodels-{name}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
fn session(dir: &std::path::Path) -> Session {
let mut s = Session::new();
s.face_models_dir = Some(dir.join("models"));
s
}
fn model_file(dir: &std::path::Path, name: &str, dim: u64) -> String {
let p = dir.join(name);
std::fs::write(&p, lightcraft_faces::synthetic::tiny_embedder_model(dim)).unwrap();
p.to_string_lossy().into_owned()
}
fn find<'a>(list: &'a Value, id: &str) -> &'a Value {
list["models"].as_array().unwrap().iter().find(|m| m["id"] == id).unwrap_or(&Value::Null)
}
#[test]
fn the_list_knows_the_known_models_and_is_honest_about_the_runtime() {
let d = temp("list");
let mut s = session(&d);
let l = s.execute("faces.models.list", &json!({})).unwrap();
assert_eq!(l["enabled"], false);
assert_eq!(l["runtime"], cfg!(not(target_arch = "wasm32")));
assert_eq!(find(&l, "yunet-2023mar")["installed"], false, "nothing ships with LightCraft");
assert_eq!(find(&l, "auraface-v1")["installed"], false);
assert_eq!(find(&l, "auraface-v1")["licence"]["commercial"], "yes");
assert_eq!(find(&l, "sface-2021dec")["licence"]["commercial"], "unknown");
// speed is a ratio to a ResNet-100 model, in words, for models that are not installed yet too; never milliseconds
assert_eq!(find(&l, "sface-2021dec")["speedText"], "6.9× faster than a ResNet-100 model");
assert_eq!(find(&l, "auraface-v1")["speedText"], "Same speed as a ResNet-100 model");
assert_eq!(find(&l, "yunet-2023mar")["speedText"], Value::Null, "nobody timed the detector against it");
// a build with no folder lists too, and cannot install
let mut web = Session::new();
assert!(web.execute("faces.models.list", &json!({})).is_ok());
let f = model_file(&d, "m.onnx", 512);
assert!(web.execute("faces.models.install", &json!({"path": f, "acknowledged": true})).is_err());
assert!(web.execute("faces.enable", &json!({"enabled": true})).is_err());
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn inspect_describes_a_file_and_installs_nothing() {
let d = temp("inspect");
let mut s = session(&d);
let f = model_file(&d, "My Model.onnx", 512);
let r = s.execute("faces.models.inspect", &json!({"path": f})).unwrap();
assert_eq!(r["kind"], "draft");
assert_eq!(r["model"]["licence"]["commercial"], "unknown");
assert_eq!(r["model"]["known"], false);
assert!(r["assumptions"].as_array().unwrap().iter().any(|a| a.as_str().unwrap().contains("127.5")));
assert_eq!(r["alreadyInstalled"], false);
assert!(!d.join("models").exists(), "inspecting writes nothing");
// not a model: a plain answer, not an error
let junk = d.join("junk.onnx");
std::fs::write(&junk, b"this is not a model").unwrap();
let r = s.execute("faces.models.inspect", &json!({"path": junk.to_string_lossy()})).unwrap();
assert_eq!(r["kind"], "unsupported");
assert!(r["reason"].as_str().unwrap().len() > 5);
// missing and empty files and folders are errors
assert!(s.execute("faces.models.inspect", &json!({"path": d.join("nope.onnx").to_string_lossy()})).is_err());
assert!(s.execute("faces.models.inspect", &json!({"path": d.to_string_lossy()})).is_err());
let empty = d.join("empty.onnx");
std::fs::write(&empty, b"").unwrap();
assert!(s.execute("faces.models.inspect", &json!({"path": empty.to_string_lossy()})).is_err());
assert!(s.execute("faces.models.inspect", &json!({})).is_err());
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn install_needs_the_licence_accepted_then_select_enable_and_remove_work() {
let d = temp("flow");
let mut s = session(&d);
let f = model_file(&d, "Mine.onnx", 512);
assert!(s.execute("faces.models.install", &json!({"path": f})).is_err(), "no acknowledgement");
assert!(s.execute("faces.models.install", &json!({"path": f, "acknowledged": false})).is_err());
assert!(s.execute("faces.models.install", &json!({"path": f, "acknowledged": "yes"})).is_err(), "must be the boolean true");
assert!(!d.join("models").exists(), "refused installs leave nothing behind");
let r = s.execute("faces.models.install", &json!({"path": f, "acknowledged": true})).unwrap();
let id = r["installed"]["id"].as_str().unwrap().to_string();
assert!(id.starts_with("custom-mine-"));
let home = d.join("models").join(&id);
assert!(home.join("model.onnx").is_file() && home.join("face-model.json").is_file() && home.join("installed.json").is_file());
assert!(!home.join("model.onnx.part").exists());
assert_eq!(std::fs::read(home.join("model.onnx")).unwrap(), std::fs::read(&f).unwrap());
let rec: Value = serde_json::from_slice(&std::fs::read(home.join("installed.json")).unwrap()).unwrap();
assert_eq!(rec["commercial"], "unknown");
assert_eq!(rec["fileName"], "Mine.onnx");
// listed as installed and custom; installing again is harmless
let l = s.execute("faces.models.list", &json!({})).unwrap();
assert_eq!(find(&l, &id)["installed"], true);
assert_eq!(find(&l, &id)["known"], false);
s.execute("faces.models.install", &json!({"path": f, "acknowledged": true})).unwrap();
assert_eq!(s.execute("faces.models.inspect", &json!({"path": f})).unwrap()["alreadyInstalled"], true);
// choose it; the choice and the on/off setting survive a new session on the same folder
assert!(s.execute("faces.models.select", &json!({"id": "auraface-v1"})).is_err(), "not installed");
assert!(s.execute("faces.models.select", &json!({"id": "yunet-2023mar"})).is_err(), "a detector is not a recogniser");
s.execute("faces.models.select", &json!({"id": id})).unwrap();
assert_eq!(s.execute("faces.enable", &json!({"enabled": true})).unwrap()["enabled"], true);
let mut again = session(&d);
let l = again.execute("faces.models.list", &json!({})).unwrap();
assert_eq!((l["enabled"].clone(), l["embedder"].clone()), (json!(true), json!(id)));
assert_eq!(again.execute("faces.enable", &json!({})).unwrap()["enabled"], false, "toggles when omitted");
// removing it clears the choice
assert_eq!(s.execute("faces.models.remove", &json!({"id": id})).unwrap()["removed"], id);
assert!(!home.exists());
assert_eq!(s.execute("faces.models.list", &json!({})).unwrap()["embedder"], Value::Null);
assert!(s.execute("faces.models.remove", &json!({"id": id})).is_err(), "already gone");
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn hostile_ids_unsupported_models_and_odd_folders_are_handled() {
let d = temp("hostile");
let mut s = session(&d);
let models = d.join("models");
std::fs::create_dir_all(&models).unwrap();
// something that is not a model sits in the folder: it is never listed and never deleted
let precious = models.join("precious");
std::fs::create_dir_all(&precious).unwrap();
std::fs::write(precious.join("keep.txt"), b"mine").unwrap();
for id in ["precious", "..", ".", "", "../x", "a/b", r"a\b", ".hidden", "yunet-2023mar", "UPPER"] {
assert!(s.execute("faces.models.remove", &json!({"id": id})).is_err(), "{id:?}");
}
assert!(precious.join("keep.txt").is_file());
assert!(d.join("models").is_dir());
// a folder whose manifest lies about its id is ignored
let liar = models.join("liar");
std::fs::create_dir_all(&liar).unwrap();
std::fs::write(liar.join("model.onnx"), lightcraft_faces::synthetic::embedder_model(64)).unwrap();
let mut m = lightcraft_faces::known::auraface();
m.id = "someone-else".into();
std::fs::write(liar.join("face-model.json"), serde_json::to_vec(&m).unwrap()).unwrap();
std::fs::write(models.join("settings.json"), b"{ not json").unwrap();
let l = s.execute("faces.models.list", &json!({})).unwrap();
assert!(find(&l, "someone-else").is_null() && find(&l, "liar").is_null());
assert_eq!(l["enabled"], false, "a corrupt settings file is ignored");
// not-a-model and wrong-shaped files cannot be installed
let junk = d.join("junk.onnx");
std::fs::write(&junk, b"nonsense").unwrap();
let e = s.execute("faces.models.install", &json!({"path": junk.to_string_lossy(), "acknowledged": true})).unwrap_err().to_string();
assert!(e.contains("cannot be used yet"), "{e}");
let tokens = d.join("tokens.onnx");
{
use lightcraft_faces::synthetic::{len_field, value_info_bytes};
let mut graph = Vec::new();
len_field(11, &value_info_bytes("pixels", 1, &[Err("N"), Ok(3), Ok(224), Ok(224)]), &mut graph);
len_field(12, &value_info_bytes("tokens", 1, &[Err("N"), Ok(257), Ok(384)]), &mut graph);
let mut model = Vec::new();
len_field(7, &graph, &mut model);
std::fs::write(&tokens, model).unwrap();
}
assert!(s.execute("faces.models.install", &json!({"path": tokens.to_string_lossy(), "acknowledged": true})).is_err());
assert!(s.execute("faces.models.select", &json!({"id": 5})).is_err());
assert!(s.execute("faces.models.select", &json!({"id": ""})).is_err());
let _ = std::fs::remove_dir_all(&d);
}
/// A demo session with the real YuNet installed, for the tests that need it: `LC_YUNET_MODEL=<the .onnx file>` (what
/// Settings > Faces downloads; nothing is committed). Without it those tests say so and pass.
fn demo_with_yunet() -> Option<Session> {
let Some(file) = std::env::var_os("LC_YUNET_MODEL") else {
eprintln!("LC_YUNET_MODEL is not set: skipping");
return None;
};
let dir = temp("yunet");
let mut s = Session::with_demo();
s.face_models_dir = Some(dir.join("models"));
let r = s.execute("faces.models.install", &json!({"path": file.to_string_lossy(), "acknowledged": true}));
assert!(r.is_ok(), "{r:?}");
Some(s)
}
#[test]
fn installing_the_detector_does_not_choose_it_as_the_recogniser_or_switch_recognition_on() {
let Some(mut s) = demo_with_yunet() else { return };
let l = s.execute("faces.models.list", &json!({})).unwrap();
assert_eq!(find(&l, "yunet-2023mar")["installed"], true);
assert_eq!(l["enabled"], false, "{l}");
assert!(l["embedder"].is_null(), "{l}");
assert!(s.execute("faces.models.test", &json!({"id": "yunet-2023mar"})).is_err(), "a detector has no recognition self-test");
}
#[test]
fn detect_without_the_model_says_where_to_get_it() {
let d = temp("nodetector");
let mut s = Session::with_demo();
// no folder for models at all, then a folder that has none yet
assert!(s.execute("faces.detect", &json!({})).is_err());
s.face_models_dir = Some(d.join("models"));
let e = s.execute("faces.detect", &json!({})).unwrap_err().to_string();
assert!(e.contains("Settings > Faces"), "{e}");
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn the_detector_and_the_recognisers_can_be_downloaded_and_the_download_is_refused_otherwise() {
let d = temp("dl-refusals");
let mut s = session(&d);
let l = s.execute("faces.models.list", &json!({})).unwrap();
for (id, host) in [("yunet-2023mar", "github.com"), ("sface-2021dec", "github.com"), ("auraface-v1", "huggingface.co")] {
assert_eq!(find(&l, id)["downloadHost"], host, "{id}");
}
// without the user's acceptance, for an id nobody knows, and without a folder: nothing starts
assert!(s.execute("faces.models.download", &json!({"id": "yunet-2023mar"})).is_err());
assert!(s.execute("faces.models.download", &json!({"id": "yunet-2023mar", "acknowledged": false})).is_err());
for id in ["nope", "", "../yunet-2023mar"] {
assert!(s.execute("faces.models.download", &json!({"id": id, "acknowledged": true})).is_err(), "{id}");
}
assert!(Session::new().execute("faces.models.download", &json!({"id": "yunet-2023mar", "acknowledged": true})).is_err());
assert_eq!(s.execute("faces.models.downloads", &json!({})).unwrap()["downloads"], json!([]));
assert!(!d.join("models").join(".downloads").exists(), "a refused download touches nothing");
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn a_download_that_has_arrived_is_installed_by_itself_and_its_staged_file_goes() {
let d = temp("dl-arrived");
let mut s = session(&d);
let staging = d.join("models").join(".downloads");
std::fs::create_dir_all(&staging).unwrap();
let staged = staging.join("model.onnx");
std::fs::write(&staged, lightcraft_faces::synthetic::tiny_embedder_model(512)).unwrap();
let sha = lightcraft_faces::hash::sha256_file(&staged).unwrap();
s.face_downloads.arrived("my-model", staged.clone(), &sha);
let r = s.execute("faces.models.downloads", &json!({})).unwrap();
let row = &r["downloads"][0];
assert_eq!((row["id"].as_str(), row["state"].as_str()), (Some("my-model"), Some("installed")), "{r}");
assert!(!staged.exists(), "the staged file was moved into the model's folder");
let l = s.execute("faces.models.list", &json!({})).unwrap();
assert_eq!(l["models"].as_array().unwrap().iter().filter(|m| m["installed"] == true).count(), 1);
// announced once, cleared on request
assert_eq!(s.execute("faces.models.downloadCancel", &json!({"id": "my-model"})).unwrap()["discarded"], true);
assert_eq!(s.execute("faces.models.downloads", &json!({})).unwrap()["downloads"], json!([]));
assert_eq!(s.execute("faces.models.downloadCancel", &json!({"id": "my-model"})).unwrap()["discarded"], false);
// a file that cannot be installed is thrown away with a reason
std::fs::write(&staged, b"this is not a model").unwrap();
let sha = lightcraft_faces::hash::sha256_file(&staged).unwrap();
s.face_downloads.arrived("bad", staged.clone(), &sha);
let r = s.execute("faces.models.downloads", &json!({})).unwrap();
assert_eq!(r["downloads"][0]["state"], "failed", "{r}");
assert!(!staged.exists());
let _ = std::fs::remove_dir_all(&d);
}
/// `faces.detect` on photos without faces: it reports nothing, and a new run replaces earlier detections but
/// never regions that came from XMP or were named; one undo step restores everything.
#[test]
fn detect_replaces_only_earlier_detections_and_is_one_undo_step() {
use lightcraft_catalog::Op;
use lightcraft_meta::{Region, RegionKind};
let region = |name: Option<&str>, description: Option<&str>, x: f64| Region {
rect: lightcraft_geom::Rect { x0: x, y0: 0.2, x1: x + 0.2, y1: 0.5 },
kind: RegionKind::Face,
name: name.map(str::to_string),
description: description.map(str::to_string),
};
let Some(mut s) = demo_with_yunet() else { return };
let id = s.active().unwrap();
let mut meta = s.catalog.photo(id).unwrap().meta.clone();
meta.regions =
vec![region(Some("Jane Doe"), None, 0.1), region(None, Some("Detected by YuNet 2023mar"), 0.5), region(None, Some("Drawn by hand"), 0.7)];
s.commit("setup", Op::SetMeta { id, meta: Box::new(meta) }).unwrap();
let undo_before = s.undo.len();
// a dry run reports and changes nothing
let r = s.execute("faces.detect", &json!({"apply": false})).unwrap();
assert_eq!(r["applied"], false);
assert_eq!(s.catalog.photo(id).unwrap().meta.regions.len(), 3);
assert_eq!(s.undo.len(), undo_before);
// a real run (the default) drops the earlier detection (the demo has no faces) and keeps the others
let r = s.execute("faces.detect", &json!({})).unwrap();
assert_eq!(r["applied"], true);
assert_eq!(r["detector"], "YuNet 2023mar");
let photo = r["photos"].as_array().unwrap().iter().find(|p| p["id"] == id.0).unwrap();
assert_eq!(photo["faces"].as_array().unwrap().len(), 0, "the procedural demo photos have no faces");
let names: Vec<_> = s.catalog.photo(id).unwrap().meta.regions.iter().map(|r| (r.name.clone(), r.description.clone())).collect();
assert_eq!(names, vec![(Some("Jane Doe".to_string()), None), (None, Some("Drawn by hand".to_string()))]);
assert_eq!(s.undo.len(), undo_before + 1, "one undo step for the whole run");
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.catalog.photo(id).unwrap().meta.regions.len(), 3, "undo brings the detection back");
// odd parameters are tolerated, and nothing is detected without a selection
for p in [json!({"score": 5}), json!({"score": -1}), json!({"nmsIou": 99}), json!({"maxFaces": 0}), json!({"score": "x"})] {
assert!(s.execute("faces.detect", &p).is_ok(), "{p}");
}
let mut empty = Session::new();
assert!(empty.execute("faces.detect", &json!({})).is_err());
}
/// Opt-in, needs the internet: `cargo test -p lightcraft-engine real_download -- --ignored --nocapture`. Downloads YuNet
/// from its pinned address the way the app does, waits for it to be installed, then finds faces with it.
#[test]
#[ignore = "downloads from github.com"]
fn real_download_of_yunet_installs_it_and_detects() {
let d = temp("real-download");
let mut s = Session::with_demo();
s.face_models_dir = Some(d.join("models"));
let started = s.execute("faces.models.download", &json!({"id": "yunet-2023mar", "acknowledged": true})).unwrap();
assert_eq!(started["from"], "github.com");
let t0 = std::time::Instant::now();
loop {
let r = s.execute("faces.models.downloads", &json!({})).unwrap();
let row = r["downloads"][0].clone();
match row["state"].as_str() {
Some("installed") => break,
Some("running") | Some("done") => {}
_ => panic!("{r}"),
}
assert!(t0.elapsed() < std::time::Duration::from_secs(120), "{r}");
std::thread::sleep(std::time::Duration::from_millis(100));
}
println!("downloaded and installed in {:?}", t0.elapsed());
let l = s.execute("faces.models.list", &json!({})).unwrap();
assert_eq!(find(&l, "yunet-2023mar")["installed"], true);
let r = s.execute("faces.detect", &json!({"apply": false})).unwrap();
assert_eq!(r["detector"], "YuNet 2023mar");
// asking again once installed is refused
assert!(s.execute("faces.models.download", &json!({"id": "yunet-2023mar", "acknowledged": true})).is_err());
let _ = std::fs::remove_dir_all(&d);
}
/// Opt-in, needs the internet: the recognisers are big, so this only checks that bytes start to arrive (through the
/// hosts' redirects) and that cancelling leaves nothing behind.
#[test]
#[ignore = "downloads from github.com and huggingface.co"]
fn real_downloads_of_the_recognisers_start_and_can_be_cancelled() {
let d = temp("real-start");
let mut s = Session::with_demo();
s.face_models_dir = Some(d.join("models"));
for id in ["sface-2021dec", "auraface-v1"] {
s.execute("faces.models.download", &json!({"id": id, "acknowledged": true})).unwrap();
let t0 = std::time::Instant::now();
loop {
let r = s.execute("faces.models.downloads", &json!({})).unwrap();
let row = r["downloads"].as_array().unwrap().iter().find(|x| x["id"] == id).unwrap().clone();
if row["state"] == "running" && row["bytes"].as_u64().unwrap_or(0) > 100_000 {
println!("{id}: {} of {} bytes after {:?}", row["bytes"], row["total"], t0.elapsed());
break;
}
assert!(row["state"] == "running", "{r}");
assert!(t0.elapsed() < std::time::Duration::from_secs(60), "{r}");
std::thread::sleep(std::time::Duration::from_millis(50));
}
s.execute("faces.models.downloadCancel", &json!({"id": id})).unwrap();
}
drop(s);
let staging = d.join("models").join(".downloads");
let left: Vec<_> = std::fs::read_dir(&staging).map(|r| r.flatten().map(|e| e.file_name()).collect()).unwrap_or_default();
println!("left in staging: {left:?}");
let _ = std::fs::remove_dir_all(&d);
}
/// With the recognition runtime: installing runs the model's self-test, a model that does not work is refused
/// and leaves nothing behind, and `faces.models.test` runs it again.
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn installing_runs_the_self_test_and_refuses_models_that_do_not_work() {
let d = temp("selftest");
let mut s = session(&d);
// a working model passes; its record keeps the result
let f = model_file(&d, "Works.onnx", 64);
let r = s.execute("faces.models.install", &json!({"path": f, "acknowledged": true})).unwrap();
let id = r["installed"]["id"].as_str().unwrap().to_string();
assert_eq!(r["installed"]["accepted"]["selfTest"]["ok"], true, "{r}");
assert_eq!(r["installed"]["accepted"]["selfTest"]["dimension"], 64);
let t = s.execute("faces.models.test", &json!({"id": id})).unwrap();
assert_eq!(t["ok"], true, "{t}");
assert!(t["result"]["embedMs"].as_f64().unwrap() >= 0.0);
assert!(s.execute("faces.models.test", &json!({"id": "nope"})).is_err());
assert!(s.execute("faces.models.test", &json!({})).is_err());
// a graph with no layers passes the shape check but cannot run: refused, and nothing is left on disk
let broken = d.join("Broken.onnx");
std::fs::write(&broken, lightcraft_faces::synthetic::embedder_model(64)).unwrap();
let e = s.execute("faces.models.install", &json!({"path": broken.to_string_lossy(), "acknowledged": true})).unwrap_err().to_string();
assert!(e.contains("could not be loaded") || e.contains("self-test"), "{e}");
let leftovers: Vec<_> = std::fs::read_dir(d.join("models"))
.unwrap()
.flatten()
.map(|e| e.file_name().to_string_lossy().into_owned())
.filter(|n| n != "settings.json")
.collect();
assert!(leftovers.len() == 1 && leftovers[0].starts_with("custom-works-"), "only the working model remains: {leftovers:?}");
let _ = std::fs::remove_dir_all(&d);
}
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn failed_replacement_keeps_the_installed_model_and_records() {
let d = temp("replacement");
let models = d.join("models");
std::fs::create_dir_all(&models).unwrap();
let good = model_file(&d, "Good.onnx", 64);
let write_catalog = |path: &std::path::Path| {
let catalog = json!({"models": [{
"id": "replacement", "name": "Replacement", "version": "1", "role": "embedder",
"url": "https://models.example.org/weights.onnx",
"sha256": lightcraft_faces::hash::sha256_file(path).unwrap(),
"sizeBytes": std::fs::metadata(path).unwrap().len(),
"licence": {"name": "MIT", "commercial": "yes"},
"output": {"kind": "embedding", "dim": 64}
}]});
std::fs::write(models.join("catalog.json"), catalog.to_string()).unwrap();
};
write_catalog(std::path::Path::new(&good));
let mut s = session(&d);
s.execute("faces.models.install", &json!({"path": good, "acknowledged": true})).unwrap();
let home = models.join("replacement");
let files = ["model.onnx", "face-model.json", "installed.json"];
let before: Vec<_> = files.iter().map(|name| std::fs::read(home.join(name)).unwrap()).collect();
let settings = std::fs::read(models.join("settings.json")).unwrap();
let bad = d.join("Bad.onnx");
std::fs::write(&bad, lightcraft_faces::synthetic::embedder_model(64)).unwrap();
write_catalog(&bad);
let mut s = session(&d);
assert!(s.execute("faces.models.install", &json!({"path": bad, "acknowledged": true})).is_err());
for (name, bytes) in files.iter().zip(before) {
assert_eq!(std::fs::read(home.join(name)).unwrap(), bytes, "{name} was replaced");
}
assert_eq!(std::fs::read(models.join("settings.json")).unwrap(), settings);
assert!(!home.join("model.onnx.part").exists());
let _ = std::fs::remove_dir_all(d);
}
fn settings_of(d: &std::path::Path) -> Value {
serde_json::from_slice(&std::fs::read(d.join("models").join("settings.json")).unwrap()).unwrap()
}
#[test]
fn installing_a_model_chooses_it_and_switches_recognition_on() {
let d = temp("activate");
let mut s = session(&d);
let a = s.execute("faces.models.install", &json!({"path": model_file(&d, "First.onnx", 64), "acknowledged": true})).unwrap();
let a_id = a["installed"]["id"].as_str().unwrap().to_string();
assert_eq!(a["active"], json!({"embedder": a_id, "enabled": true}));
assert_eq!(a["installed"]["selected"], true);
assert_eq!(settings_of(&d), json!({"enabled": true, "embedder": a_id}));
// a second model takes over (the newest is the one the user just asked for); the first stays installed
let b = s.execute("faces.models.install", &json!({"path": model_file(&d, "Second.onnx", 96), "acknowledged": true})).unwrap();
let b_id = b["installed"]["id"].as_str().unwrap().to_string();
assert_ne!(a_id, b_id);
assert_eq!(b["active"]["embedder"], json!(b_id));
let l = s.execute("faces.models.list", &json!({})).unwrap();
assert_eq!((find(&l, &a_id)["installed"].clone(), find(&l, &a_id)["selected"].clone()), (json!(true), json!(false)));
assert_eq!(find(&l, &b_id)["selected"], true);
// switching back is a choice, and a model added with `activate: false` leaves the choice alone
s.execute("faces.models.select", &json!({"id": a_id})).unwrap();
let c = s.execute("faces.models.install", &json!({"path": model_file(&d, "Third.onnx", 128), "acknowledged": true, "activate": false})).unwrap();
let c_id = c["installed"]["id"].as_str().unwrap().to_string();
assert_eq!(c["active"]["embedder"], json!(a_id));
assert_eq!(c["installed"]["selected"], false);
// a switched-off recognition is switched back on by the next install, not by a refused one
s.execute("faces.enable", &json!({"enabled": false})).unwrap();
assert!(s.execute("faces.models.install", &json!({"path": model_file(&d, "Nope.onnx", 64)})).is_err());
assert_eq!(settings_of(&d)["enabled"], false);
s.execute("faces.models.install", &json!({"path": model_file(&d, "Fourth.onnx", 64), "acknowledged": true})).unwrap();
assert_eq!(settings_of(&d)["enabled"], true);
// removing the model in use hands over to another installed one instead of leaving nothing chosen
let in_use = settings_of(&d)["embedder"].as_str().unwrap().to_string();
let r = s.execute("faces.models.remove", &json!({"id": in_use})).unwrap();
let next = r["embedder"].as_str().unwrap().to_string();
assert_ne!(next, in_use);
assert!([&a_id, &b_id, &c_id].contains(&&next) || next.starts_with("custom-"), "{next}");
assert_eq!(settings_of(&d)["embedder"], json!(next));
// removing a model that is not in use changes nothing
let other = [&a_id, &b_id, &c_id].into_iter().find(|i| **i != next && **i != in_use).unwrap().clone();
s.execute("faces.models.remove", &json!({"id": other})).unwrap();
assert_eq!(settings_of(&d)["embedder"], json!(next));
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn a_download_that_arrives_is_installed_and_switched_on_by_itself() {
use crate::face_download::{STAGING, State};
let d = temp("finish");
let mut s = session(&d);
let staging = d.join("models").join(STAGING);
std::fs::create_dir_all(&staging).unwrap();
// what the download thread leaves when it succeeds: a verified file in the staging folder and a finished job
let staged = model_file(&staging, "arrived.onnx", 64);
let sha = lightcraft_faces::hash::sha256_file(std::path::Path::new(&staged)).unwrap();
s.face_downloads.set_outcome("fake-model", State::Done { path: staged.clone().into(), sha256: sha });
let r = s.execute("faces.models.downloads", &json!({})).unwrap();
assert_eq!(r["downloads"][0]["id"], "fake-model");
assert_eq!(r["downloads"][0]["state"], "installed", "{r}");
let st = settings_of(&d);
assert_eq!(st["enabled"], true);
assert!(st["embedder"].as_str().unwrap().starts_with("custom-arrived-"), "{st}");
assert!(!std::path::Path::new(&staged).exists(), "the staged file was moved into place");
// the record stays until it is cleared, and is cleared once
assert_eq!(s.execute("faces.models.downloads", &json!({})).unwrap()["downloads"][0]["state"], "installed");
assert_eq!(s.execute("faces.models.downloadCancel", &json!({"id": "fake-model"})).unwrap()["discarded"], true);
assert_eq!(s.execute("faces.models.downloads", &json!({})).unwrap()["downloads"], json!([]));
// a download that turns out not to be a usable model is thrown away, and the reason is shown
let junk = staging.join("junk.onnx");
std::fs::write(&junk, b"definitely not a model").unwrap();
let sha = lightcraft_faces::hash::sha256_file(&junk).unwrap();
s.face_downloads.set_outcome("fake-junk", State::Done { path: junk.clone(), sha256: sha });
let r = s.execute("faces.models.downloads", &json!({})).unwrap();
assert_eq!(r["downloads"][0]["state"], "failed", "{r}");
assert!(r["downloads"][0]["error"].as_str().unwrap().contains("cannot be used"), "{r}");
assert!(!junk.exists());
assert_eq!(settings_of(&d)["embedder"], st["embedder"], "a failed download leaves the choice alone");
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn the_users_own_catalog_adds_models_with_the_same_download_and_install() {
use crate::face_download::State;
let d = temp("catalog");
let mut s = session(&d);
let models = d.join("models");
std::fs::create_dir_all(&models).unwrap();
// a model file, and a catalog that lists it by its hash (plus an entry that cannot be used)
let file = model_file(&d, "weights.onnx", 64);
let (sha, size) = (lightcraft_faces::hash::sha256_file(std::path::Path::new(&file)).unwrap(), std::fs::metadata(&file).unwrap().len());
let catalog = json!({"models": [
{"id": "my-research-model", "name": "My research model", "version": "1", "role": "embedder", "url": "https://models.example.org/w/weights.onnx",
"sha256": sha, "sizeBytes": size, "licence": {"name": "Research only", "commercial": "no", "notice": "Not for commercial use."},
"provenance": "Some dataset", "output": {"kind": "embedding", "dim": 64}, "thresholds": {"matchCosine": 0.42}},
{"id": "broken", "name": "No hash", "version": "1", "role": "embedder", "url": "https://models.example.org/x.onnx", "output": {"kind": "embedding", "dim": 64}},
]});
std::fs::write(models.join("catalog.json"), catalog.to_string()).unwrap();
// it is listed like a built-in model, with its own terms, a download address, and the problem with the other entry
let l = s.execute("faces.models.list", &json!({})).unwrap();
let m = find(&l, "my-research-model");
assert_eq!(
(m["fromCatalog"].clone(), m["downloadHost"].clone(), m["licence"]["commercial"].clone()),
(json!(true), json!("models.example.org"), json!("no"))
);
assert_eq!(m["installed"], false);
assert_eq!(find(&l, "broken"), &Value::Null);
assert_eq!(l["catalog"]["models"], 1);
assert!(l["catalog"]["errors"][0].as_str().unwrap().contains("broken"), "{}", l["catalog"]);
// nothing is fetched without the terms being accepted
assert!(s.execute("faces.models.download", &json!({"id": "my-research-model"})).is_err());
// a file that matches is recognised as that model (not as a draft), so its terms and settings are used
let seen = s.execute("faces.models.inspect", &json!({"path": file})).unwrap();
assert_eq!((seen["kind"].clone(), seen["model"]["id"].clone()), (json!("known"), json!("my-research-model")));
// a finished download of it installs under the catalog's id, with the catalog's licence record
let staging = models.join(crate::face_download::STAGING);
std::fs::create_dir_all(&staging).unwrap();
let staged = staging.join("weights.onnx");
std::fs::copy(&file, &staged).unwrap();
s.face_downloads.set_outcome("my-research-model", State::Done { path: staged, sha256: sha });
let r = s.execute("faces.models.downloads", &json!({})).unwrap();
assert_eq!(r["downloads"][0]["state"], "installed", "{r}");
let st = settings_of(&d);
assert_eq!((st["embedder"].clone(), st["enabled"].clone()), (json!("my-research-model"), json!(true)));
let l = s.execute("faces.models.list", &json!({})).unwrap();
assert_eq!(
(find(&l, "my-research-model")["installed"].clone(), find(&l, "my-research-model")["accepted"]["commercial"].clone()),
(json!(true), json!("no"))
);
// a broken or hostile catalog never stops the list
for bytes in [&b"{"[..], b"[]", &vec![b'x'; 400_000]] {
std::fs::write(models.join("catalog.json"), bytes).unwrap();
let l = s.execute("faces.models.list", &json!({})).unwrap();
assert!(l["catalog"]["errors"].as_array().is_some_and(|e| !e.is_empty()), "{}", l["catalog"]);
assert!(find(&l, "sface-2021dec")["id"] == "sface-2021dec");
}
let _ = std::fs::remove_dir_all(&d);
}
+489
View File
@@ -0,0 +1,489 @@
//! Recognition: indexing faces, suggesting names, naming, with a tiny working network standing in for a real
//! model (its embedding depends on the picture's colours, which is enough to test the machinery; how well real
//! models recognise people is measured separately, on real photos).
use std::path::PathBuf;
use lightcraft_catalog::{Op, PhotoId};
use lightcraft_geom::Rect;
use lightcraft_meta::{Region, RegionKind};
use serde_json::{Value, json};
use crate::Session;
pub(crate) fn temp(name: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("lc-facerec-{name}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
/// A demo session with a tiny model installed and chosen.
pub(crate) fn setup(d: &std::path::Path, dim: u64) -> (Session, String) {
let mut s = Session::with_demo();
s.face_models_dir = Some(d.join("models"));
let model = d.join(format!("tiny{dim}.onnx"));
std::fs::write(&model, lightcraft_faces::synthetic::tiny_embedder_model(dim)).unwrap();
let r = s.execute("faces.models.install", &json!({"path": model.to_string_lossy(), "acknowledged": true})).unwrap();
let id = r["installed"]["id"].as_str().unwrap().to_string();
s.execute("faces.models.select", &json!({"id": id})).unwrap();
(s, id)
}
/// Installs the real YuNet from `LC_YUNET_MODEL=<the .onnx file>` (what Settings > Faces downloads; nothing is
/// committed) and says whether it did: the parts of a test that need to find faces run only then.
pub(crate) fn install_yunet(s: &mut Session) -> bool {
let Some(file) = std::env::var_os("LC_YUNET_MODEL") else {
eprintln!("LC_YUNET_MODEL is not set: skipping the face-finding part");
return false;
};
let r = s.execute("faces.models.install", &json!({"path": file.to_string_lossy(), "acknowledged": true}));
assert!(r.is_ok(), "{r:?}");
true
}
pub(crate) fn region(x: f64, name: Option<&str>) -> Region {
Region { rect: Rect { x0: x, y0: 0.2, x1: x + 0.25, y1: 0.6 }, kind: RegionKind::Face, name: name.map(str::to_string), description: None }
}
pub(crate) fn set_regions(s: &mut Session, id: PhotoId, regions: Vec<Region>) {
let mut meta = s.catalog.photo(id).unwrap().meta.clone();
meta.regions = regions;
s.commit("setup", Op::SetMeta { id, meta: Box::new(meta) }).unwrap();
}
pub(crate) fn two_photos(s: &Session) -> (PhotoId, PhotoId) {
let mut ids = s.catalog.photos().map(|p| p.id).collect::<Vec<_>>();
ids.sort();
(ids[0], ids[1])
}
#[test]
fn indexing_embeds_each_face_once_and_reports_progress() {
let d = temp("index");
let (mut s, _) = setup(&d, 64);
let (a, b) = two_photos(&s);
set_regions(&mut s, a, vec![region(0.1, Some("Ann")), region(0.5, None)]);
set_regions(&mut s, b, vec![region(0.3, None)]);
// only a report
let r = s.execute("faces.index", &json!({"budgetMs": 0})).unwrap();
assert_eq!((r["embedded"].clone(), r["photosDone"].clone(), r["pendingPhotos"].clone()), (json!(0), json!(0), json!(2)), "{r}");
// a generous budget does it all, the named photo first
let r = s.execute("faces.index", &json!({"budgetMs": 60_000})).unwrap();
assert_eq!((r["embedded"].clone(), r["pendingPhotos"].clone(), r["indexedFaces"].clone()), (json!(3), json!(0), json!(3)), "{r}");
// nothing is embedded twice
let r = s.execute("faces.index", &json!({"budgetMs": 60_000})).unwrap();
assert_eq!((r["embedded"].clone(), r["photosDone"].clone()), (json!(0), json!(0)));
// a small budget still makes progress (at least one photo per call), and `ids` go first
let c = s.catalog.photos().map(|p| p.id).find(|id| *id != a && *id != b).unwrap();
set_regions(&mut s, c, vec![region(0.2, None)]);
set_regions(&mut s, b, vec![region(0.3, None), region(0.6, None)]);
let r = s.execute("faces.index", &json!({"budgetMs": 1, "ids": [c.0]})).unwrap();
assert_eq!(r["photosDone"], 1, "{r}");
assert_eq!(r["pendingPhotos"], 1, "{r}");
assert!(s.faces.index.contains(c.0, &region(0.2, None).rect), "the photo asked for first was done first");
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn suggestions_come_from_the_named_faces_and_are_never_applied() {
let d = temp("suggest");
let (mut s, _) = setup(&d, 64);
let (a, b) = two_photos(&s);
set_regions(&mut s, a, vec![region(0.1, Some("Ann"))]);
set_regions(&mut s, b, vec![region(0.1, None), region(0.5, Some("Pet?")), region(0.7, None)]);
// thresholds that accept anything: Ann is the only person with a named face except "Pet?"
let r = s.execute("faces.suggest", &json!({"ids": [b.0], "threshold": -1.0, "margin": -2.0, "budgetMs": 60_000})).unwrap();
assert_eq!(r["galleryFaces"], 2, "{r}");
let faces = r["photos"][0]["faces"].as_array().unwrap();
assert_eq!(faces.len(), 2, "only the unnamed faces get suggestions: {r}");
assert!(faces.iter().all(|f| f["suggestion"]["name"].is_string()), "{r}");
assert_eq!(faces[0]["index"], 0);
assert_eq!(faces[1]["index"], 2);
assert!(faces[0]["candidates"].as_array().unwrap().len() == 2);
// an impossible threshold suggests nobody but still lists the candidates
let r = s.execute("faces.suggest", &json!({"ids": [b.0], "threshold": 2.0, "budgetMs": 60_000})).unwrap();
assert!(r["photos"][0]["faces"][0]["suggestion"].is_null());
assert!(!r["photos"][0]["faces"][0]["candidates"].as_array().unwrap().is_empty());
// nothing was named by suggesting
assert!(s.catalog.photo(b).unwrap().meta.regions.iter().filter(|r| r.name.is_none()).count() == 2);
// pets and other kinds are not people
let mut meta = s.catalog.photo(b).unwrap().meta.clone();
meta.regions[1].kind = RegionKind::Pet;
s.commit("pet", Op::SetMeta { id: b, meta: Box::new(meta) }).unwrap();
let r = s.execute("faces.suggest", &json!({"ids": [b.0], "threshold": -1.0, "margin": -2.0, "budgetMs": 60_000})).unwrap();
assert_eq!(r["galleryFaces"], 1, "{r}");
// odd parameters are tolerated
for p in [json!({"threshold": "x"}), json!({"margin": null}), json!({"budgetMs": 0}), json!({"ids": []}), json!({"ids": [999999]})] {
assert!(s.execute("faces.suggest", &p).is_ok(), "{p}");
}
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn naming_a_face_is_undoable_and_makes_it_yours() {
let d = temp("name");
let (mut s, _) = setup(&d, 64);
let (a, _) = two_photos(&s);
let mut detected = region(0.1, None);
detected.description = Some("Detected by YuNet 2023mar".into());
set_regions(&mut s, a, vec![detected, region(0.5, None)]);
let undo = s.undo.len();
let r = s.execute("faces.setName", &json!({"id": a.0, "index": 0, "name": " Ann "})).unwrap();
assert_eq!(r["name"], "Ann");
let regions = s.catalog.photo(a).unwrap().meta.regions.clone();
assert_eq!((regions[0].name.as_deref(), regions[0].description.as_deref()), (Some("Ann"), None), "a named detection loses its detected marker");
assert_eq!(s.undo.len(), undo + 1);
// a new detection run now leaves it alone
if install_yunet(&mut s) {
s.execute("faces.detect", &json!({"id": a.0})).unwrap();
assert_eq!(s.catalog.photo(a).unwrap().meta.regions.iter().filter(|r| r.name.as_deref() == Some("Ann")).count(), 1);
}
// clearing, undo
s.execute("faces.setName", &json!({"id": a.0, "index": 0, "name": null})).unwrap();
assert_eq!(s.catalog.photo(a).unwrap().meta.regions[0].name, None);
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.catalog.photo(a).unwrap().meta.regions[0].name.as_deref(), Some("Ann"));
// bad input
for p in [
json!({"id": a.0}),
json!({"id": a.0, "index": 9, "name": "x"}),
json!({"id": a.0, "index": 0, "name": 5}),
json!({"id": a.0, "index": 0, "name": "a\u{7}b"}),
json!({"id": a.0, "index": 0, "name": "x".repeat(201)}),
json!({"id": 999999, "index": 0, "name": "x"}),
] {
assert!(s.execute("faces.setName", &p).is_err(), "{p}");
}
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn embeddings_are_cached_beside_the_library_and_reset_for_another_model() {
let d = temp("persist");
let lib = d.join("lib");
let models = d.join("models");
let install = |s: &mut Session, dim: u64| -> String {
let model = d.join(format!("tiny{dim}.onnx"));
std::fs::write(&model, lightcraft_faces::synthetic::tiny_embedder_model(dim)).unwrap();
let r = s.execute("faces.models.install", &json!({"path": model.to_string_lossy(), "acknowledged": true})).unwrap();
let id = r["installed"]["id"].as_str().unwrap().to_string();
s.execute("faces.models.select", &json!({"id": id})).unwrap();
id
};
let mut s = Session::new().with_fs();
s.face_models_dir = Some(models.clone());
s.open_library(&lib, true).unwrap();
let (a, _) = two_photos(&s);
set_regions(&mut s, a, vec![region(0.1, Some("Ann")), region(0.5, None)]);
install(&mut s, 64);
s.execute("faces.index", &json!({"budgetMs": 60_000})).unwrap();
assert!(lib.join(format!("face-embeddings-{}.bin", s.faces.index.tag.split('@').next().unwrap())).is_file());
s.close_library().unwrap();
drop(s);
// a new session finds the embeddings on disk and has nothing left to do
let mut s2 = Session::new().with_fs();
s2.face_models_dir = Some(models);
s2.open_library(&lib, true).unwrap();
let r = s2.execute("faces.index", &json!({"budgetMs": 60_000})).unwrap();
assert_eq!((r["embedded"].clone(), r["indexedFaces"].clone(), r["pendingPhotos"].clone()), (json!(0), json!(2), json!(0)), "{r}");
// choosing another model starts over: its faces are different numbers, the old ones are not used
install(&mut s2, 32);
let r = s2.execute("faces.index", &json!({"budgetMs": 0})).unwrap();
assert_eq!((r["indexedFaces"].clone(), r["pendingPhotos"].clone()), (json!(0), json!(1)), "{r}");
let r = s2.execute("faces.index", &json!({"budgetMs": 60_000})).unwrap();
assert_eq!(r["embedded"], 2);
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn without_a_chosen_model_recognition_says_so() {
let d = temp("nomodel");
let mut s = Session::with_demo();
assert!(s.execute("faces.index", &json!({})).is_err(), "no models folder");
s.face_models_dir = Some(d.join("models"));
for cmd in ["faces.index", "faces.suggest"] {
let e = s.execute(cmd, &json!({})).unwrap_err().to_string();
assert!(e.contains("no recognition model is chosen"), "{cmd}: {e}");
}
let _: Value = json!(null);
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn the_background_pump_indexes_by_itself_once_recognition_is_on() {
let d = temp("pump");
let (mut s, _) = setup(&d, 64);
let (a, b) = two_photos(&s);
set_regions(&mut s, a, vec![region(0.1, Some("Ann")), region(0.5, None)]);
set_regions(&mut s, b, vec![region(0.3, None)]);
// installing the model switched recognition on; switched off, the pump does nothing, and says so
assert_eq!(s.execute("faces.enable", &json!({"enabled": false})).unwrap()["enabled"], false);
assert_eq!(s.execute("faces.pump", &json!({})).unwrap()["active"], false);
assert_eq!(s.faces.index.len(), 0);
s.execute("faces.enable", &json!({"enabled": true})).unwrap();
// `enabled` is looked up at most once a second, so the first pump may still see "off"
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(60);
let mut last = json!(null);
while std::time::Instant::now() < deadline {
last = s.execute("faces.pump", &json!({})).unwrap();
if last["active"] == true && last["pendingPhotos"] == 0 && last["indexedFaces"] == 3 {
break;
}
std::thread::sleep(std::time::Duration::from_millis(50));
}
assert_eq!((last["active"].clone(), last["pendingPhotos"].clone(), last["indexedFaces"].clone()), (json!(true), json!(0), json!(3)), "{last}");
// a new face is picked up (the catalog changed), and suggestions use what the pump embedded
set_regions(&mut s, b, vec![region(0.3, None), region(0.6, None)]);
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(60);
while std::time::Instant::now() < deadline && s.faces.index.len() < 4 {
s.execute("faces.pump", &json!({})).unwrap();
std::thread::sleep(std::time::Duration::from_millis(50));
}
assert_eq!(s.faces.index.len(), 4);
let r = s.execute("faces.suggest", &json!({"ids": [b.0], "threshold": -1.0, "margin": -2.0, "budgetMs": 0})).unwrap();
assert_eq!(r["photos"][0]["faces"].as_array().unwrap().len(), 2);
assert!(r["photos"][0]["faces"][0]["suggestion"]["name"].is_string(), "{r}");
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn the_background_pump_saves_what_it_embeds_for_the_next_launch() {
let d = temp("pumpsave");
let (lib, models) = (d.join("lib"), d.join("models"));
let mut s = Session::new().with_fs();
s.face_models_dir = Some(models);
s.open_library(&lib, true).unwrap();
let model = d.join("tiny64.onnx");
std::fs::write(&model, lightcraft_faces::synthetic::tiny_embedder_model(64)).unwrap();
// installing switches recognition on: nothing else is asked for
s.execute("faces.models.install", &json!({"path": model.to_string_lossy(), "acknowledged": true})).unwrap();
// without the detector (not bundled) photos without faces are left unsearched
let searching = install_yunet(&mut s);
let (a, b) = two_photos(&s);
set_regions(&mut s, a, vec![region(0.1, Some("Ann")), region(0.5, None)]);
set_regions(&mut s, b, vec![region(0.3, None)]);
let photos = s.catalog.photos().count();
let searched_photos = if searching { photos - 2 } else { 0 };
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(120);
let mut last = json!(null);
while std::time::Instant::now() < deadline {
last = s.execute("faces.pump", &json!({})).unwrap();
if last["pendingPhotos"] == 0 && s.faces.index.len() == 3 && s.faces.scanned.len() == searched_photos {
break;
}
std::thread::sleep(std::time::Duration::from_millis(50));
}
assert_eq!((s.faces.index.len(), s.faces.scanned.len()), (3, searched_photos), "{last}");
let (tag, path) = (s.faces.index.tag.clone(), s.faces.index.path.clone().unwrap());
assert!(path.file_name().unwrap().to_string_lossy().starts_with("face-embeddings-custom-"), "one cache file per model: {}", path.display());
// the first batch is written at once; whatever is left is written when the session ends, so a launch never redoes
// work: neither the embeddings nor the list of photos already searched for faces
s.close_library().unwrap();
drop(s);
let mut again = crate::faces_index::Index::default();
again.reset(&tag, 64, Some(path));
assert_eq!(again.len(), 3);
let mut searched = crate::faces_index::Scanned::default();
searched.reset(Some(lib.join("face-scanned.bin")));
assert_eq!(searched.len(), searched_photos, "every photo without faces is remembered as searched");
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn faces_person_lists_a_persons_faces_and_looks_for_more_only_while_recognition_runs() {
let d = temp("person");
let (mut s, _) = setup(&d, 64);
s.execute("faces.index", &json!({"budgetMs": 0})).unwrap();
let (a, b) = two_photos(&s);
set_regions(&mut s, a, vec![region(0.1, Some("Ann")), region(0.5, Some("Bob"))]);
set_regions(&mut s, b, vec![region(0.1, None), region(0.5, Some("ann"))]);
let unit = |parts: &[(usize, f32)]| {
let mut v = vec![0.0f32; 64];
for (i, x) in parts {
v[*i] = *x;
}
let n = v.iter().map(|x| x * x).sum::<f32>().sqrt();
v.iter().map(|x| x / n).collect::<Vec<f32>>()
};
for (id, x, v) in [(a, 0.1, unit(&[(0, 1.0)])), (a, 0.5, unit(&[(1, 1.0)])), (b, 0.1, unit(&[(0, 0.9), (1, 0.1)])), (b, 0.5, unit(&[(0, 1.0)]))] {
s.faces.index.insert(id.0, &region(x, None).rect, v);
}
// the name is matched without regard to case or surrounding spaces; every face is listed, wherever it is
let r = s.execute("faces.person", &json!({"name": " ANN "})).unwrap();
assert_eq!((r["name"].clone(), r["total"].clone()), (json!("Ann"), json!(2)));
let mut at: Vec<(u64, u64)> =
r["confirmed"].as_array().unwrap().iter().map(|f| (f["photo"].as_u64().unwrap(), f["index"].as_u64().unwrap())).collect();
at.sort();
let mut want = vec![(a.0, 0), (b.0, 1)];
want.sort();
assert_eq!(at, want);
assert!(r["confirmed"].as_array().unwrap().iter().any(|f| f["rect"]["x0"] == 0.1 && f["rect"]["y1"] == 0.6));
// recognition is on (installing the model switched it on): the unnamed face that looks like Ann is offered
assert_eq!(r["ready"], true);
let more = r["more"].as_array().unwrap();
assert_eq!((more.len(), more[0]["photo"].clone(), more[0]["index"].clone()), (1, json!(b.0), json!(0)), "{r}");
assert!(more[0]["score"].as_f64().unwrap() > 0.9);
// confirming it moves it from one list to the other (and is one undo step)
s.execute("faces.setName", &json!({"id": b.0, "index": 0, "name": "Ann"})).unwrap();
let r = s.execute("faces.person", &json!({"name": "Ann"})).unwrap();
assert_eq!((r["total"].clone(), r["more"].as_array().unwrap().len()), (json!(3), 0));
// switched off, the faces are still listed but nothing is searched for
s.execute("faces.enable", &json!({"enabled": false})).unwrap();
let r = s.execute("faces.person", &json!({"name": "Ann"})).unwrap();
assert_eq!((r["total"].clone(), r["ready"].clone(), r["more"].as_array().unwrap().len()), (json!(3), json!(false), 0));
// nobody by that name, and bad input, are plain answers and errors
let r = s.execute("faces.person", &json!({"name": "Nobody"})).unwrap();
assert_eq!((r["total"].clone(), r["confirmed"].as_array().unwrap().len(), r["ready"].clone()), (json!(0), 0, json!(false)));
for bad in [json!({}), json!({"name": ""}), json!({"name": " "}), json!({"name": 7})] {
assert!(s.execute("faces.person", &bad).is_err(), "{bad}");
}
assert!(s.execute("faces.person", &json!({"name": "Ann", "more": u64::MAX})).is_ok());
let _ = std::fs::remove_dir_all(&d);
}
fn unit64(parts: &[(usize, f32)]) -> Vec<f32> {
let mut v = vec![0.0f32; 64];
for (i, x) in parts {
v[*i] = *x;
}
let n = v.iter().map(|x| x * x).sum::<f32>().sqrt();
v.iter().map(|x| x / n).collect()
}
#[test]
fn many_faces_are_named_in_one_undo_step() {
let d = temp("namemany");
let (mut s, _) = setup(&d, 64);
let (a, b) = two_photos(&s);
let detected = |x: f64| Region { description: Some("Detected by YuNet 2023mar".into()), ..region(x, None) };
set_regions(&mut s, a, vec![detected(0.1), region(0.5, None)]);
set_regions(&mut s, b, vec![detected(0.3)]);
let undo = s.undo.len();
let r = s
.execute(
"faces.nameFaces",
&json!({"faces": [{"photo": a.0, "index": 0}, {"photo": a.0, "index": 1}, {"photo": b.0, "index": 0}], "name": " Jane Doe "}),
)
.unwrap();
assert_eq!((r["named"].clone(), r["photos"].clone()), (json!(3), json!(2)));
assert_eq!(s.undo.len(), undo + 1, "one step for the whole group");
for (id, i) in [(a, 0), (a, 1), (b, 0)] {
let reg = &s.catalog.photo(id).unwrap().meta.regions[i];
assert_eq!(reg.name.as_deref(), Some("Jane Doe"));
assert!(reg.description.is_none() || !reg.description.as_deref().unwrap().starts_with("Detected by"), "a named face is the user's now");
}
// undo restores every one of them together
s.execute("edit.undo", &json!({})).unwrap();
assert!(s.catalog.photo(a).unwrap().meta.regions.iter().chain(s.catalog.photo(b).unwrap().meta.regions.iter()).all(|r| r.name.is_none()));
// an empty name clears; faces that do not exist are skipped, and with none left it is an error
s.execute("faces.nameFaces", &json!({"faces": [{"photo": a.0, "index": 0}], "name": "Ann"})).unwrap();
let r = s
.execute(
"faces.nameFaces",
&json!({"faces": [{"photo": a.0, "index": 0}, {"photo": a.0, "index": 99}, {"photo": 987_654, "index": 0}], "name": null}),
)
.unwrap();
assert_eq!(r["named"], 1);
assert!(s.catalog.photo(a).unwrap().meta.regions[0].name.is_none());
for bad in [
json!({"name": "x"}),
json!({"faces": "x", "name": "x"}),
json!({"faces": [{"photo": a.0}], "name": "x"}),
json!({"faces": [{"photo": 987_654, "index": 0}], "name": "x"}),
json!({"faces": [{"photo": a.0, "index": 0}], "name": 7}),
json!({"faces": [{"photo": a.0, "index": 0}], "name": "bad\u{7}name"}),
json!({"faces": vec![json!({"photo": a.0, "index": 0}); 5001], "name": "x"}),
] {
assert!(s.execute("faces.nameFaces", &bad).is_err(), "{bad}");
}
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn unnamed_faces_come_with_look_alikes_together_and_the_name_the_named_ones_suggest() {
let d = temp("unnamed");
let (mut s, _) = setup(&d, 64);
s.execute("faces.index", &json!({"budgetMs": 0})).unwrap();
let (a, b) = two_photos(&s);
let c = s.catalog.photos().map(|p| p.id).find(|id| *id != a && *id != b).unwrap();
set_regions(&mut s, a, vec![region(0.1, Some("Ann")), region(0.5, None), region(0.7, None)]);
set_regions(&mut s, b, vec![region(0.1, None), region(0.5, None)]);
set_regions(&mut s, c, vec![region(0.3, None)]);
// three faces like Ann, two like Bob (unnamed), one unembedded
let put = |s: &mut Session, id: lightcraft_catalog::PhotoId, x: f64, v: Vec<f32>| s.faces.index.insert(id.0, &region(x, None).rect, v);
put(&mut s, a, 0.1, unit64(&[(0, 1.0)])); // Ann, named
put(&mut s, a, 0.5, unit64(&[(1, 1.0)])); // like Bob
put(&mut s, b, 0.1, unit64(&[(0, 0.97), (2, 0.2)])); // like Ann
put(&mut s, b, 0.5, unit64(&[(1, 0.98), (2, 0.2)])); // like Bob
put(&mut s, c, 0.3, unit64(&[(0, 0.95), (3, 0.3)])); // like Ann
// (a's last face, at 0.7, is not embedded yet)
let r = s.execute("faces.unnamed", &json!({})).unwrap();
assert_eq!((r["total"].clone(), r["ordered"].clone(), r["ready"].clone()), (json!(5), json!(true), json!(true)), "{r}");
let faces = r["faces"].as_array().unwrap();
let at = |f: &Value| (f["photo"].as_u64().unwrap(), f["index"].as_u64().unwrap());
let order: Vec<(u64, u64)> = faces.iter().map(at).collect();
// look-alikes are neighbours: the two like Bob are adjacent, and the three like Ann are adjacent; the unembedded one is last
let pos = |x: (u64, u64)| order.iter().position(|o| *o == x).unwrap();
assert_eq!(pos((a.0, 1)).abs_diff(pos((b.0, 1))), 1, "{order:?}");
assert_eq!(order.last(), Some(&(a.0, 2)), "{order:?}");
// a face the named ones recognise carries the name they suggest
let like_ann = faces.iter().find(|f| at(f) == (b.0, 0)).unwrap();
assert_eq!(like_ann["suggestion"]["name"], "Ann", "{like_ann}");
assert!(faces.iter().find(|f| at(f) == (b.0, 1)).unwrap()["suggestion"].is_null(), "Bob is nobody the named faces know");
// each face says which box to show it by; with no scan view it is its own box
for k in ["x0", "y0", "x1", "y1"] {
assert!((like_ann["view"][k].as_f64().unwrap() - like_ann["rect"][k].as_f64().unwrap()).abs() < 1e-3, "{like_ann}");
}
// a limit cuts the list, not the count
let r = s.execute("faces.unnamed", &json!({"limit": 2})).unwrap();
assert_eq!((r["total"].clone(), r["faces"].as_array().unwrap().len()), (json!(5), 2));
// switched off, the faces are still listed, in photo order, without suggestions
s.execute("faces.enable", &json!({"enabled": false})).unwrap();
let r = s.execute("faces.unnamed", &json!({})).unwrap();
assert_eq!((r["total"].clone(), r["ordered"].clone(), r["ready"].clone()), (json!(5), json!(false), json!(false)));
assert!(r["faces"].as_array().unwrap().iter().all(|f| f["suggestion"].is_null()));
for odd in [json!({"limit": 0}), json!({"limit": "x"}), json!({"limit": u64::MAX})] {
assert!(s.execute("faces.unnamed", &odd).is_ok(), "{odd}");
}
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn every_scanned_face_is_shown_by_the_same_kind_of_box() {
let d = temp("view");
let (mut s, _) = setup(&d, 64);
s.execute("faces.index", &json!({"budgetMs": 0})).unwrap();
let (a, _) = two_photos(&s);
// a loosely drawn box (from some other tool) and the detector's tighter box of the same face
let loose = lightcraft_geom::Rect { x0: 0.2, y0: 0.1, x1: 0.8, y1: 0.9 };
let tight = lightcraft_geom::Rect { x0: 0.35, y0: 0.25, x1: 0.65, y1: 0.65 };
let mut meta = s.catalog.photo(a).unwrap().meta.clone();
meta.regions = vec![Region { rect: loose, kind: RegionKind::Face, name: Some("Ann".into()), description: None }];
s.commit("setup", Op::SetMeta { id: a, meta: Box::new(meta) }).unwrap();
// before the scan has looked at it, the face is shown by its own box
assert_eq!(s.face_view(a, loose), loose);
s.faces.index.insert_with_view(a.0, &loose, unit64(&[(0, 1.0)]), &tight);
let shown = s.face_view(a, loose);
assert!((shown.x0 - tight.x0).abs() < 1e-3 && (shown.y1 - tight.y1).abs() < 1e-3, "shown by the detector's box once known: {shown:?}");
// a face nobody has looked at, or in another photo, is shown by its own box
let other = lightcraft_geom::Rect { x0: 0.1, y0: 0.1, x1: 0.2, y1: 0.2 };
assert_eq!(s.face_view(a, other), other);
// the page of the person uses it too
let r = s.execute("faces.person", &json!({"name": "Ann"})).unwrap();
let view = &r["confirmed"][0]["view"];
assert!((view["x0"].as_f64().unwrap() - tight.x0).abs() < 1e-3, "{r}");
// and it survives a restart: the index file keeps it beside the embedding
let (tag, path) = (s.faces.index.tag.clone(), d.join("view-cache.bin"));
let mut written = crate::faces_index::Index::default();
written.reset(&tag, 64, Some(path.clone()));
written.insert_with_view(a.0, &loose, unit64(&[(0, 1.0)]), &tight);
written.save().unwrap();
drop(written);
let mut again = crate::faces_index::Index::default();
again.reset(&tag, 64, Some(path));
let v = again.view(a.0, &loose).expect("the face is remembered");
assert!((v.x0 - tight.x0).abs() < 1e-3 && (v.y0 - tight.y0).abs() < 1e-3, "{v:?}");
let _ = std::fs::remove_dir_all(&d);
}
+23 -8
View File
@@ -189,6 +189,12 @@ fn the_folder_filter_takes_only_a_text_path() {
assert_eq!(s.visible().len(), 0, "a path that names no folder shows nothing, not everything");
}
/// Folders are compared by identity: a path read back may be spelled with the platform's separator.
fn same_folder(actual: Option<&str>, expected: &str, why: &str) {
let key = lightcraft_catalog::query::folder_key;
assert_eq!(actual.map(key), Some(key(expected)), "{why}: {actual:?} vs {expected}");
}
#[test]
fn a_renamed_folder_stays_the_chosen_one_and_undo_follows_it_back() {
let dir = Scratch::new("rename");
@@ -200,10 +206,10 @@ fn a_renamed_folder_stays_the_chosen_one_and_undo_follows_it_back() {
s.execute("library.filter", &json!({"libraryFolder": trip})).unwrap();
assert_eq!(s.visible().len(), 2);
s.execute("folder.rename", &json!({"path": trip, "name": "holiday"})).unwrap();
assert_eq!(s.filter.library_folder.as_deref(), Some(renamed.as_str()), "the choice follows the folder");
same_folder(s.filter.library_folder.as_deref(), &renamed, "the choice follows the folder");
assert_eq!(s.visible().len(), 2, "and still shows its photos");
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.filter.library_folder.as_deref(), Some(trip.as_str()));
same_folder(s.filter.library_folder.as_deref(), &trip, "the choice is the same folder");
assert_eq!(s.visible().len(), 2);
}
@@ -217,10 +223,10 @@ fn a_moved_folder_keeps_a_chosen_subfolder_chosen() {
add(&mut s, &format!("{day1}/b.jpg"));
s.execute("library.filter", &json!({"libraryFolder": day1})).unwrap();
s.execute("folder.move", &json!({"path": dir.path("trip"), "into": dir.path("archive")})).unwrap();
assert_eq!(s.filter.library_folder.as_deref(), Some(moved.as_str()), "the subfolder follows its parent");
same_folder(s.filter.library_folder.as_deref(), &moved, "the subfolder follows its parent");
assert_eq!(s.visible().len(), 1);
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.filter.library_folder.as_deref(), Some(day1.as_str()));
same_folder(s.filter.library_folder.as_deref(), &day1, "the choice is the same folder");
}
#[test]
@@ -229,12 +235,21 @@ fn following_a_folder_reads_both_spellings_the_same_way() {
let mut s = Session::new();
s.filter.library_folder = Some("/a/x/../b/sub".into());
crate::cmd::browse::follow_folder(&mut s, "/a/b", "/a/c");
assert_eq!(s.filter.library_folder.as_deref(), Some("/a/c/sub"), "a subfolder stays a subfolder, never widens to its parent");
same_folder(s.filter.library_folder.as_deref(), "/a/c/sub", "a subfolder stays a subfolder, never widens to its parent");
s.filter.library_folder = Some("/elsewhere/b".into());
crate::cmd::browse::follow_folder(&mut s, "/a/b", "/a/c");
assert_eq!(s.filter.library_folder.as_deref(), Some("/elsewhere/b"), "other folders are left alone");
}
#[test]
fn following_a_folder_keeps_the_case_of_the_subfolder_names() {
// Windows compares folders in lower case; the name the user sees must not change with it
let mut s = Session::new();
s.filter.library_folder = Some("/A/Trip/Day1".into());
crate::cmd::browse::follow_folder(&mut s, "/A/Trip", "/B");
assert_eq!(s.filter.library_folder.as_deref(), Some(std::path::Path::new("/B").join("Day1").to_str().unwrap()));
}
#[test]
fn choosing_a_folder_replaces_the_source_the_way_an_album_does() {
let mut s = session();
@@ -296,10 +311,10 @@ fn a_renamed_folder_stays_the_one_shown_and_undo_follows_it_back() {
add(&mut s, &format!("{day1}/b.jpg"));
source_folder(&mut s, &day1);
s.execute("folder.rename", &json!({"path": trip, "name": "holiday"})).unwrap();
assert_eq!(s.library_folder.as_deref(), Some(format!("{renamed}/day1").as_str()), "the subfolder follows its parent");
same_folder(s.library_folder.as_deref(), &format!("{renamed}/day1"), "the subfolder follows its parent");
assert_eq!(s.visible().len(), 1);
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.library_folder.as_deref(), Some(day1.as_str()));
same_folder(s.library_folder.as_deref(), &day1, "the choice is the same folder");
assert_eq!(s.visible().len(), 1);
}
@@ -308,7 +323,7 @@ fn the_shown_folder_follows_reading_both_spellings_the_same_way() {
let mut s = Session::new();
s.library_folder = Some("/a/x/../b/sub".into());
crate::cmd::browse::follow_folder(&mut s, "/a/b", "/a/c");
assert_eq!(s.library_folder.as_deref(), Some("/a/c/sub"));
same_folder(s.library_folder.as_deref(), "/a/c/sub", "the shown folder reads both spellings alike");
}
#[test]
+98
View File
@@ -1007,3 +1007,101 @@ fn trashed_photo_ids(tag: &str) -> (Session, std::path::PathBuf, [u64; 1], u64)
let (s, src, id) = trashed_photo(tag);
(s, src, [id], 0)
}
/// Issue #374: launch arguments never turn into a walk of a whole drive or home folder. An empty
/// argument is ignored; `.`, `..` and (on Windows) `/` are resolved against the working folder a
/// launcher picked, which is often a drive root; a root is replaced by its memory card's DCIM
/// folder or refused, and so is the home folder.
#[test]
fn launch_paths_never_become_a_drive_root_or_the_home_folder() {
use crate::import::{LaunchPath, launch_path};
let root = if cfg!(windows) { Path::new("C:\\") } else { Path::new("/") };
let home = if cfg!(windows) { Path::new("C:\\Users\\me") } else { Path::new("/home/me") };
let none = |_: &Path| None;
let refused = |r: Option<LaunchPath>| matches!(r, Some(LaunchPath::Refused { .. }));
for empty in ["", " "] {
assert_eq!(launch_path(empty, Some(root), Some(home), none), None, "{empty:?}");
}
// the working folder is the drive root: `.` (and `..`) name the whole drive
for arg in [".", "..", "./", ".\\..", "/"] {
let arg = if cfg!(windows) { arg.to_string() } else { arg.replace('\\', "/") };
assert!(refused(launch_path(&arg, Some(root), Some(home), none)), "{arg:?}");
}
assert!(refused(launch_path("..", Some(&home.join("Pictures")), Some(home), none)), "up to the home folder");
assert!(refused(launch_path(&home.to_string_lossy(), None, Some(home), none)));
assert!(refused(launch_path(".", None, Some(home), none)), "relative, no working folder");
// a memory card's root: its camera folder
let dcim = |r: &Path| Some(r.join("DCIM"));
assert_eq!(launch_path(".", Some(root), Some(home), dcim), Some(LaunchPath::Import(root.join("DCIM").to_string_lossy().to_string())));
// ordinary folders and files resolve to absolute paths
let pics = home.join("Pictures");
assert_eq!(launch_path("Pictures", Some(home), Some(home), none), Some(LaunchPath::Import(pics.to_string_lossy().to_string())));
assert_eq!(
launch_path(&pics.join("a.jpg").to_string_lossy(), None, Some(home), none),
Some(LaunchPath::Import(pics.join("a.jpg").to_string_lossy().to_string()))
);
}
/// Issue #374: a folder import is bounded — a tree deeper or larger than the limits is not walked
/// to the end (it stops and says so), and what it found is still returned.
#[test]
fn expand_stops_at_the_walk_limits() {
let d = temp_dir("bounded");
let mut deep = d.clone();
for i in 0..8 {
deep = deep.join(format!("d{i}"));
std::fs::create_dir_all(&deep).unwrap();
std::fs::write(deep.join(format!("p{i}.jpg")), b"x").unwrap();
}
let top = [d.to_string_lossy().to_string()];
let (all, cut) = crate::import::expand_within(&top, None, crate::walk::Limits::default());
assert_eq!((all.len(), cut), (8, false));
let (some, cut) = crate::import::expand_within(&top, None, crate::walk::Limits { max_depth: 3, max_entries: 1000 });
assert_eq!((some.len(), cut), (2, true), "{some:?}");
let (few, cut) = crate::import::expand_within(&top, None, crate::walk::Limits { max_depth: 32, max_entries: 5 });
assert!(cut && few.len() < 8, "{few:?}");
let _ = std::fs::remove_dir_all(&d);
}
/// Raw containers LightCraft cannot decode but shows by their embedded JPEG are picked up from a folder
/// (the extensions are in `import::EXTENSIONS`; the files differ so none is a duplicate) and import as preview only.
#[test]
fn folder_import_picks_up_undecodable_raw_containers_as_preview_only() {
use lightcraft_codecs::{ChromaSubsampling, EncodeImage, EncodeMeta, Samples, encode_jpeg};
let src = temp_dir("preview-only-exts");
let px: Vec<u8> = (0..40 * 30).flat_map(|i| [(i % 251) as u8, 128, 200]).collect();
let jpeg = encode_jpeg(&EncodeImage::new(40, 30, 3, Samples::U8(&px)), 90, ChromaSubsampling::S444, &EncodeMeta::default()).unwrap();
// CRW / MRW / X3F: their magic, then the JPEG. IIQ / KDC / MOS / ERF: a TIFF whose IFD0 only points at the JPEG.
let tiff = {
let off = 8 + 2 + 24 + 4;
let mut f = b"II*\0\x08\0\0\0\x02\0".to_vec();
for (tag, v) in [(513u16, off as u32), (514, jpeg.len() as u32)] {
f.extend_from_slice(&tag.to_le_bytes());
f.extend_from_slice(&[4, 0, 1, 0, 0, 0]);
f.extend_from_slice(&v.to_le_bytes());
}
f.extend_from_slice(&[0, 0, 0, 0]);
f.extend_from_slice(&jpeg);
f
};
let with_magic = |magic: &[u8]| [magic, &[0x11u8; 64][..], &jpeg].concat();
let files: [(&str, Vec<u8>); 7] = [
("a.CRW", with_magic(b"II\x1a\0\0\0HEAPCCDR")),
("b.mrw", with_magic(b"\0MRM\0\x01\0\0")),
("c.x3f", with_magic(b"FOVb\x02\0\x02\0")),
("d.iiq", [&tiff[..], b"d"].concat()),
("e.kdc", [&tiff[..], b"e"].concat()),
("f.mos", [&tiff[..], b"f"].concat()),
("g.erf", [&tiff[..], b"g"].concat()),
];
for (name, bytes) in &files {
std::fs::write(src.join(name), bytes).unwrap();
}
let mut s = Session::new().with_fs();
let r = s.execute("library.import", &json!({"paths": [src.to_string_lossy()]})).unwrap();
assert_eq!(r["scanned"], 7, "{r}");
assert_eq!(ids(&r, "imported"), 7, "{r}");
assert_eq!(ids(&r, "failed"), 0, "{r}");
assert!(s.catalog.photos().all(|p| p.preview_only.is_some() && p.kind == lightcraft_catalog::MediaKind::Raw && (p.width, p.height) == (40, 30)));
let _ = std::fs::remove_dir_all(&src);
}
+67
View File
@@ -433,3 +433,70 @@ fn write_policy_timing() {
println!("copy {n} × {} MB: byte compare with the source {bytewise:?}, probe hash {hashed:?}", size >> 20);
let _ = std::fs::remove_dir_all(&base);
}
/// Issue #367: Import → Copy wrote and verified one file after another on one thread. The copies
/// now run side by side, with the names, order, duplicates and reports a one-by-one copy gives:
/// like-named files take -1, -2… in file order (also against a file already there and a literal
/// `IMG_1-1`), and a file with the content of one copied earlier in the batch is its duplicate
/// (also while that copy is still under way).
#[test]
fn copies_run_side_by_side_with_the_same_outcome() {
let base = temp_dir("parallel-copy");
let (card, dest) = (base.join("card"), base.join("out"));
write_png(&card.join("a/IMG_1.png"), 1);
write_png(&card.join("a/IMG_2.png"), 2);
write_png(&card.join("b/IMG_1.png"), 3);
write_png(&card.join("b/IMG_1-1.png"), 4);
write_png(&card.join("c/IMG_1.png"), 5);
for i in 0..8u8 {
write_png(&card.join(format!("d/p{i}.png")), 10 + i);
}
// the content of a file whose copy is still under way
std::fs::create_dir_all(card.join("e")).unwrap();
std::fs::copy(card.join("d/p3.png"), card.join("e/dup.png")).unwrap();
std::fs::create_dir_all(&dest).unwrap();
std::fs::write(dest.join("IMG_2.png"), b"someone else's file").unwrap();
let mut s = session();
inject(Fault::Slow);
let r = s
.execute(
"library.import",
&json!({"paths": [card.to_string_lossy()], "mode": "copy", "destination": dest.to_string_lossy(), "organize": "flat"}),
)
.unwrap();
inject(Fault::None);
let threads: std::collections::HashSet<_> = crate::import_move::copy_threads().into_iter().collect();
if std::thread::available_parallelism().map_or(1, |n| n.get()) > 1 {
assert!(threads.len() > 1, "copies ran on {} thread(s)", threads.len());
}
assert_eq!(len(&r, "imported"), 13, "{r}");
assert_eq!(len(&r, "failed"), 0, "{r}");
assert_eq!(len(&r, "duplicates"), 1, "{r}");
assert_eq!(r["duplicates"][0]["reason"], "content");
assert!(r["duplicates"][0]["path"].as_str().unwrap().ends_with("dup.png"), "{r}");
// the order of a one-by-one copy (files in name order), each verified
let expect = [
("a/IMG_1.png", "IMG_1.png"),
("a/IMG_2.png", "IMG_2-1.png"),
("b/IMG_1-1.png", "IMG_1-1.png"),
("b/IMG_1.png", "IMG_1-2.png"),
("c/IMG_1.png", "IMG_1-3.png"),
];
for (src, dst) in expect {
assert_eq!(std::fs::read(dest.join(dst)).unwrap(), std::fs::read(card.join(src)).unwrap(), "{src} → {dst}");
}
for i in 0..8 {
assert_eq!(std::fs::read(dest.join(format!("p{i}.png"))).unwrap(), std::fs::read(card.join(format!("d/p{i}.png"))).unwrap());
}
assert_eq!(std::fs::read(dest.join("IMG_2.png")).unwrap(), b"someone else's file");
assert_eq!(files_under(&dest).len(), 14, "{:?}", files_under(&dest));
// the duplicate names the photo copied from the same content (d/p3); ids follow the file order
let ids: Vec<u64> = r["imported"].as_array().unwrap().iter().map(|v| v.as_u64().unwrap()).collect();
assert!(ids.windows(2).all(|w| w[0] < w[1]), "{ids:?}");
assert_eq!(r["duplicates"][0]["existing"], ids[8], "{r}");
let first = s.catalog.photo(lightcraft_catalog::PhotoId(ids[0])).unwrap();
assert_eq!(first.file_name, "IMG_1.png");
let _ = std::fs::remove_dir_all(&base);
}
+25
View File
@@ -95,3 +95,28 @@ fn unreadable_settings_are_not_overwritten() {
assert_eq!(files.files.get("presets.json").unwrap(), presets);
assert_eq!(files.files.get("prefs.json").unwrap(), prefs);
}
/// Issue #171: a library whose `catalog.lock` can't be locked (stood in for here by a directory
/// of that name, which takes the same path as a file system without locks) still opens, and the
/// user is told it is unprotected — in `library.info` and once through the library warnings the
/// app's notices and the CLI show. A normally locked library says nothing.
#[test]
fn an_unlockable_library_opens_with_a_warning() {
let lib = temp_dir("unlocked");
std::fs::create_dir_all(lib.join("catalog.lock")).unwrap();
let mut s = Session::new().with_fs();
s.open_library(&lib, false).unwrap();
let info = s.execute("library.info", &json!({})).unwrap();
let listed = info["settingsWarnings"].as_array().cloned().unwrap_or_default();
assert!(listed.iter().any(|w| w.as_str().is_some_and(|w| w.contains("without protection against a second program"))), "{info}");
let warnings = s.take_library_warnings();
assert_eq!(warnings.len(), 1, "{warnings:?}");
drop(s);
// the same library with a lockable lock file: no warning
std::fs::remove_dir_all(lib.join("catalog.lock")).unwrap();
let mut s = Session::new().with_fs();
s.open_library(&lib, false).unwrap();
assert!(s.take_library_warnings().is_empty());
drop(s);
let _ = std::fs::remove_dir_all(&lib);
}
+183
View File
@@ -0,0 +1,183 @@
//! Bounded folder walks (issue #374): every recursive listing of the disk — import, Local's
//! Include Subfolders, Find Missing Photos, preset and LUT folders, MCP — goes through
//! [`files_in`], which stops at a depth and a number of directory entries instead of walking a
//! whole drive, follows a symbolic link or junction to a folder at most once (no loops), and
//! decides what is a folder from the directory listing itself (no extra open per entry, which
//! is what makes a large walk slow on Windows).
//!
//! It never chooses where to walk: callers pass a folder the user named. Nothing in the app walks
//! at startup or on the UI thread.
use std::collections::HashSet;
use std::path::{Path, PathBuf};
/// How far a walk may go.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Limits {
/// Folder levels below the starting folder that are listed (its own entries are level 1).
pub max_depth: usize,
/// Directory entries looked at in all (files and folders, hidden ones included).
pub max_entries: usize,
}
impl Default for Limits {
fn default() -> Self {
Limits { max_depth: MAX_DEPTH, max_entries: MAX_ENTRIES }
}
}
/// Folder levels a walk descends at most; photo trees are a handful of levels deep.
pub const MAX_DEPTH: usize = 32;
/// Directory entries a walk looks at at most: far more than any photo folder holds, far fewer
/// than a whole system drive.
pub const MAX_ENTRIES: usize = 1_000_000;
/// What a walk found.
#[derive(Debug, Default, PartialEq)]
pub struct Walked {
/// The files `keep` accepted, in path order, folder by folder (depth first).
pub files: Vec<PathBuf>,
/// The walk stopped at a [`Limits`] bound: there may be more files.
pub truncated: bool,
/// Directory entries looked at.
pub entries: usize,
}
fn hidden(p: &Path) -> bool {
p.file_name().is_some_and(|n| n.to_string_lossy().starts_with('.'))
}
/// The files below folder `root` that `keep` accepts. Hidden entries (names starting with `.`)
/// are left out, the folder `skip` (e.g. the library's own) is never entered, and the walk stops
/// at `limits` (see [`Walked::truncated`]). Unreadable folders are skipped.
pub fn files_in(root: &Path, skip: Option<&Path>, limits: Limits, mut keep: impl FnMut(&Path) -> bool) -> Walked {
let mut out = Walked::default();
// canonical folders reached through a link, so a link loop is walked at most once
let mut linked: HashSet<PathBuf> = HashSet::new();
if let Ok(c) = std::fs::canonicalize(root) {
linked.insert(c);
}
// (path, depth of the folder it is in, is a folder); popped in path order
let mut stack: Vec<(PathBuf, usize, bool)> = vec![(root.to_path_buf(), 0, true)];
while let Some((path, depth, is_dir)) = stack.pop() {
if !is_dir {
if keep(&path) {
out.files.push(path);
}
continue;
}
if skip.is_some_and(|s| path == s) {
continue;
}
if depth >= limits.max_depth {
out.truncated = true;
continue;
}
let Ok(rd) = std::fs::read_dir(&path) else { continue };
let mut children: Vec<(PathBuf, bool)> = Vec::new();
for e in rd.flatten() {
if out.entries >= limits.max_entries {
out.truncated = true;
break;
}
out.entries = out.entries.saturating_add(1);
let p = e.path();
if hidden(&p) {
continue;
}
let Ok(ft) = e.file_type() else { continue };
let dir = if ft.is_symlink() {
// a link (or a Windows junction): a folder is entered once, a file is a file
match std::fs::metadata(&p) {
Ok(m) if m.is_dir() => {
let Ok(c) = std::fs::canonicalize(&p) else { continue };
if !linked.insert(c) {
continue;
}
true
}
_ => false,
}
} else {
ft.is_dir()
};
children.push((p, dir));
}
children.sort();
let below = depth.saturating_add(1);
stack.extend(children.into_iter().rev().map(|(p, d)| (p, below, d)));
}
out
}
#[cfg(test)]
mod tests {
use super::*;
fn tmp(tag: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("lc-walk-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
#[test]
fn files_come_in_path_order_without_hidden_ones_or_the_skipped_folder() {
let d = tmp("order");
for f in ["b/2.jpg", "a/1.jpg", "a/x/0.jpg", "c.jpg", ".hidden/h.jpg", "a/.h.jpg", "lib/l.jpg", "a/n.txt"] {
let p = d.join(f);
std::fs::create_dir_all(p.parent().unwrap()).unwrap();
std::fs::write(p, b"x").unwrap();
}
let lib = d.join("lib");
let w = files_in(&d, Some(&lib), Limits::default(), |p| p.extension().is_some_and(|e| e == "jpg"));
let rel: Vec<String> = w.files.iter().map(|p| p.strip_prefix(&d).unwrap().to_string_lossy().replace('\\', "/")).collect();
assert_eq!(rel, ["a/1.jpg", "a/x/0.jpg", "b/2.jpg", "c.jpg"]);
assert!(!w.truncated);
let _ = std::fs::remove_dir_all(&d);
}
/// Issue #374: a walk never runs on without end — a tree deeper than the limit, or with more
/// entries than the limit, stops and says so.
#[test]
fn deep_and_wide_trees_stop_at_the_limits() {
let d = tmp("bounds");
let mut deep = d.join("deep");
for i in 0..12 {
deep = deep.join(format!("l{i}"));
}
std::fs::create_dir_all(&deep).unwrap();
std::fs::write(deep.join("bottom.jpg"), b"x").unwrap();
let wide = d.join("wide");
std::fs::create_dir_all(&wide).unwrap();
for i in 0..50 {
std::fs::write(wide.join(format!("{i:02}.jpg")), b"x").unwrap();
}
let all = |_: &Path| true;
let w = files_in(&d.join("deep"), None, Limits { max_depth: 5, max_entries: 1000 }, all);
assert!(w.files.is_empty() && w.truncated, "{w:?}");
let w = files_in(&d.join("deep"), None, Limits { max_depth: 13, max_entries: 1000 }, all);
assert_eq!((w.files.len(), w.truncated), (1, false), "{w:?}");
let w = files_in(&wide, None, Limits { max_depth: 5, max_entries: 20 }, all);
assert_eq!((w.files.len(), w.entries, w.truncated), (20, 20, true));
let w = files_in(&wide, None, Limits::default(), all);
assert_eq!((w.files.len(), w.truncated), (50, false));
let _ = std::fs::remove_dir_all(&d);
}
/// A link back up the tree (a loop) is followed at most once: the walk ends.
#[cfg(unix)]
#[test]
fn link_loops_end() {
let d = tmp("loop");
std::fs::create_dir_all(d.join("a/b")).unwrap();
std::fs::write(d.join("a/b/p.jpg"), b"x").unwrap();
std::os::unix::fs::symlink(&d, d.join("a/b/up")).unwrap();
std::os::unix::fs::symlink(d.join("a"), d.join("alias")).unwrap();
let w = files_in(&d, None, Limits::default(), |_| true);
assert!(!w.truncated, "{w:?}");
assert!(w.files.len() <= 2, "{w:?}");
assert!(w.files.iter().any(|p| p.ends_with("a/b/p.jpg")), "{w:?}");
let _ = std::fs::remove_dir_all(&d);
}
}
+25
View File
@@ -0,0 +1,25 @@
[package]
name = "lightcraft-faces"
version.workspace = true
edition.workspace = true
license.workspace = true
description = "LightCraft faces (L1): bounded ONNX CPU inference, face-model manifests, validation and detection"
[lints]
workspace = true
[features]
default = []
[dependencies]
ndarray = { version = "=0.17.2", default-features = false, features = ["std"] }
# Rust intrinsics only, CPU-dispatched; no BLAS, cc/assembly or nested threading.
matrixmultiply = { version = "=0.3.11", default-features = false, features = ["std", "avx512"] }
web-time = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
sha2 = { workspace = true }
thiserror = { workspace = true }
[dev-dependencies]
proptest = { workspace = true }
+273
View File
@@ -0,0 +1,273 @@
//! Cutting a face out of a photo the way recognition models expect it.
//!
//! With the five landmarks the detector finds (eyes, nose tip, mouth corners), the face is turned and scaled
//! (a similarity transform, found by least squares) so those points land on the standard 112 × 112
//! ArcFace template. Without landmarks (a box from an XMP sidecar) a padded square around the box is used
//! instead. Both warp with supersampling, so shrinking a large face does not alias.
//!
//! Pure geometry on 8-bit RGB: no image types, no panics (every access is checked).
/// The five template points of a 112 × 112 aligned face, image-left eye first: eye, eye, nose tip, mouth
/// corner, mouth corner. (The standard ArcFace template.)
pub const TEMPLATE_112: [(f32, f32); 5] = [(38.2946, 51.6963), (73.5318, 51.5014), (56.0252, 71.7366), (41.5493, 92.3655), (70.7299, 92.2041)];
/// A borrowed 8-bit RGB picture.
#[derive(Clone, Copy)]
pub struct Rgb<'a> {
pub data: &'a [u8],
pub width: usize,
pub height: usize,
}
impl Rgb<'_> {
fn valid(&self) -> bool {
self.width > 0 && self.height > 0 && self.width.checked_mul(self.height).and_then(|n| n.checked_mul(3)) == Some(self.data.len())
}
/// The pixel at integer coordinates, clamped to the picture.
fn at(&self, x: isize, y: isize) -> [f32; 3] {
let (x, y) = (x.clamp(0, self.width as isize - 1) as usize, y.clamp(0, self.height as isize - 1) as usize);
let i = (y * self.width + x) * 3;
match self.data.get(i..i + 3) {
Some(p) => [f32::from(p[0]), f32::from(p[1]), f32::from(p[2])],
None => [0.0; 3],
}
}
/// Bilinear sample at a fractional position (pixel centres at integers).
fn sample(&self, x: f32, y: f32) -> [f32; 3] {
let (x0, y0) = (x.floor(), y.floor());
let (fx, fy) = (x - x0, y - y0);
let (x0, y0) = (x0 as isize, y0 as isize);
let (a, b, c, d) = (self.at(x0, y0), self.at(x0 + 1, y0), self.at(x0, y0 + 1), self.at(x0 + 1, y0 + 1));
let mut out = [0.0; 3];
for k in 0..3 {
let top = a[k] * (1.0 - fx) + b[k] * fx;
let bottom = c[k] * (1.0 - fx) + d[k] * fx;
out[k] = top * (1.0 - fy) + bottom * fy;
}
out
}
}
/// `q = a · p + t` with complex `a` (scale and rotation) and `t`, mapping source positions to output positions.
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct Similarity {
pub a: (f32, f32),
pub t: (f32, f32),
}
impl Similarity {
pub fn scale(&self) -> f32 {
self.a.0.hypot(self.a.1)
}
/// Where an output position `q` comes from in the source.
fn invert(&self, q: (f32, f32)) -> Option<(f32, f32)> {
let n = self.a.0 * self.a.0 + self.a.1 * self.a.1;
if !(n.is_finite() && n > 1e-12) {
return None;
}
let (dx, dy) = (q.0 - self.t.0, q.1 - self.t.1);
Some(((dx * self.a.0 + dy * self.a.1) / n, (dy * self.a.0 - dx * self.a.1) / n))
}
}
/// The least-squares similarity transform taking `from[i]` to `to[i]` (no reflection). `None` when the points
/// do not determine one (all the same point, or non-finite).
pub fn fit_similarity(from: &[(f32, f32)], to: &[(f32, f32)]) -> Option<Similarity> {
if from.len() != to.len() || from.len() < 2 {
return None;
}
let n = from.len() as f32;
let mean = |pts: &[(f32, f32)]| (pts.iter().map(|p| p.0).sum::<f32>() / n, pts.iter().map(|p| p.1).sum::<f32>() / n);
let (mp, mq) = (mean(from), mean(to));
// a = Σ conj(p') q' / Σ |p'|² (complex numbers)
let (mut re, mut im, mut norm) = (0.0f32, 0.0f32, 0.0f32);
for (p, q) in from.iter().zip(to) {
let (px, py, qx, qy) = (p.0 - mp.0, p.1 - mp.1, q.0 - mq.0, q.1 - mq.1);
re += px * qx + py * qy;
im += px * qy - py * qx;
norm += px * px + py * py;
}
if !(norm.is_finite() && norm > 1e-9) {
return None;
}
let a = (re / norm, im / norm);
let t = (mq.0 - (a.0 * mp.0 - a.1 * mp.1), mq.1 - (a.0 * mp.1 + a.1 * mp.0));
(a.0.is_finite() && a.1.is_finite() && t.0.is_finite() && t.1.is_finite()).then_some(Similarity { a, t })
}
/// Fill an `out_w` × `out_h` picture by taking, for each output pixel, the source under it, averaging
/// enough sub-samples that shrinking does not alias.
fn warp(img: &Rgb, m: &Similarity, out_w: usize, out_h: usize) -> Option<Vec<u8>> {
if !img.valid() || out_w == 0 || out_h == 0 || out_w > 4096 || out_h > 4096 {
return None;
}
if ![m.a.0, m.a.1, m.t.0, m.t.1].iter().all(|v| v.is_finite()) {
return None;
}
// how many source pixels one output pixel spans, per side
let span = 1.0 / m.scale();
let n = if span.is_finite() { (span.ceil() as usize).clamp(1, 4) } else { 1 };
let mut out = vec![0u8; out_w * out_h * 3];
for (i, px) in out.as_chunks_mut::<3>().0.iter_mut().enumerate() {
let (ox, oy) = ((i % out_w) as f32, (i / out_w) as f32);
let mut acc = [0.0f32; 3];
for sy in 0..n {
for sx in 0..n {
let q = (ox + (sx as f32 + 0.5) / n as f32 - 0.5, oy + (sy as f32 + 0.5) / n as f32 - 0.5);
let (px_, py_) = m.invert(q)?;
let s = img.sample(px_, py_);
acc.iter_mut().zip(s).for_each(|(a, v)| *a += v);
}
}
let k = (n * n) as f32;
for (o, a) in px.iter_mut().zip(acc) {
*o = (a / k).round().clamp(0.0, 255.0) as u8;
}
}
Some(out)
}
/// The face aligned to the ArcFace template, `out_w` × `out_h` (the template is for 112 × 112 and is scaled for
/// other sizes). `landmarks` are in pixels of `img`. `None` when the landmarks cannot define a transform.
pub fn align_to_template(img: &Rgb, landmarks: &[(f32, f32); 5], out_w: usize, out_h: usize) -> Option<Vec<u8>> {
let (sx, sy) = (out_w as f32 / 112.0, out_h as f32 / 112.0);
let template: Vec<(f32, f32)> = TEMPLATE_112.iter().map(|(x, y)| (x * sx, y * sy)).collect();
let m = fit_similarity(landmarks, &template)?;
// a face smaller than a few pixels, or absurdly large, is not a face we can use
let scale = m.scale();
if !(scale.is_finite() && (0.01..=100.0).contains(&scale)) {
return None;
}
warp(img, &m, out_w, out_h)
}
/// A square around a face box (pixels of `img`), `pad` times its larger side, as `out_w` × `out_h`.
pub fn crop_box(img: &Rgb, x0: f32, y0: f32, x1: f32, y1: f32, pad: f32, out_w: usize, out_h: usize) -> Option<Vec<u8>> {
if ![x0, y0, x1, y1, pad].iter().all(|v| v.is_finite()) {
return None;
}
let side = (x1 - x0).abs().max((y1 - y0).abs()) * pad;
if !(side.is_finite() && side >= 2.0) {
return None;
}
let (cx, cy) = ((x0 + x1) / 2.0, (y0 + y1) / 2.0);
let s = out_w.min(out_h) as f32 / side;
let m = Similarity { a: (s, 0.0), t: (out_w as f32 / 2.0 - s * cx, out_h as f32 / 2.0 - s * cy) };
warp(img, &m, out_w, out_h)
}
#[cfg(test)]
mod tests {
use super::*;
fn close(a: (f32, f32), b: (f32, f32), tol: f32) -> bool {
(a.0 - b.0).abs() <= tol && (a.1 - b.1).abs() <= tol
}
fn apply(m: &Similarity, p: (f32, f32)) -> (f32, f32) {
(m.a.0 * p.0 - m.a.1 * p.1 + m.t.0, m.a.0 * p.1 + m.a.1 * p.0 + m.t.1)
}
#[test]
fn a_known_similarity_is_recovered() {
// rotate 20 degrees, scale 0.7, translate
let (s, th) = (0.7f32, 20f32.to_radians());
let truth = Similarity { a: (s * th.cos(), s * th.sin()), t: (13.0, -5.0) };
let from: Vec<(f32, f32)> = TEMPLATE_112.to_vec();
let to: Vec<(f32, f32)> = from.iter().map(|p| apply(&truth, *p)).collect();
let m = fit_similarity(&from, &to).unwrap();
assert!(close(m.a, truth.a, 1e-4) && close(m.t, truth.t, 1e-3), "{m:?} vs {truth:?}");
// noisy points still give the best fit (the residual is small)
let noisy: Vec<(f32, f32)> = to.iter().enumerate().map(|(i, p)| (p.0 + (i as f32 - 2.0) * 0.2, p.1 - (i as f32 - 2.0) * 0.1)).collect();
let m2 = fit_similarity(&from, &noisy).unwrap();
assert!(from.iter().zip(&noisy).all(|(p, q)| close(apply(&m2, *p), *q, 1.0)));
}
#[test]
fn degenerate_points_give_nothing() {
let same = vec![(5.0, 5.0); 5];
assert!(fit_similarity(&same, &TEMPLATE_112).is_none());
assert!(fit_similarity(&TEMPLATE_112, &TEMPLATE_112[..3]).is_none());
assert!(fit_similarity(&[(0.0, 0.0)], &[(1.0, 1.0)]).is_none());
let nan = vec![(f32::NAN, 0.0); 5];
assert!(fit_similarity(&nan, &TEMPLATE_112).is_none());
let inf = vec![(f32::INFINITY, 0.0), (1.0, 1.0), (2.0, 2.0), (3.0, 3.0), (4.0, 4.0)];
assert!(fit_similarity(&inf, &TEMPLATE_112).is_none());
}
/// A picture with a bright dot at each of five landmarks, the landmarks being the template moved by a known
/// similarity: aligning must put the dots back on the template.
#[test]
fn alignment_puts_the_landmarks_on_the_template() {
let (w, h) = (400usize, 300usize);
let (s, th) = (1.7f32, -15f32.to_radians());
let place = Similarity { a: (s * th.cos(), s * th.sin()), t: (150.0, 60.0) };
let landmarks: Vec<(f32, f32)> = TEMPLATE_112.iter().map(|p| apply(&place, *p)).collect();
let mut data = vec![0u8; w * h * 3];
for l in &landmarks {
for dy in -3i32..=3 {
for dx in -3i32..=3 {
let (x, y) = ((l.0 as i32 + dx) as usize, (l.1 as i32 + dy) as usize);
if x < w && y < h {
data[(y * w + x) * 3] = 255;
}
}
}
}
let img = Rgb { data: &data, width: w, height: h };
let lm: [(f32, f32); 5] = [landmarks[0], landmarks[1], landmarks[2], landmarks[3], landmarks[4]];
let out = align_to_template(&img, &lm, 112, 112).unwrap();
assert_eq!(out.len(), 112 * 112 * 3);
// the red channel's centre of mass near each template point is on the point
for t in TEMPLATE_112 {
let (mut sx, mut sy, mut sw) = (0.0f32, 0.0f32, 0.0f32);
for y in (t.1 as usize).saturating_sub(8)..(t.1 as usize + 9).min(112) {
for x in (t.0 as usize).saturating_sub(8)..(t.0 as usize + 9).min(112) {
let v = f32::from(out[(y * 112 + x) * 3]);
sx += v * x as f32;
sy += v * y as f32;
sw += v;
}
}
assert!(sw > 100.0, "no dot near {t:?}");
assert!(close((sx / sw, sy / sw), t, 1.5), "dot at {:?}, template {t:?}", (sx / sw, sy / sw));
}
}
#[test]
fn box_crops_are_centred_and_bad_inputs_are_refused() {
let (w, h) = (200usize, 100usize);
let mut data = vec![0u8; w * h * 3];
for y in 40..60 {
for x in 90..110 {
data[(y * w + x) * 3 + 1] = 255;
}
}
let img = Rgb { data: &data, width: w, height: h };
let out = crop_box(&img, 90.0, 40.0, 110.0, 60.0, 2.0, 64, 64).unwrap();
// the 20 px block fills the middle half of the 64 px output
let at = |x: usize, y: usize| out[(y * 64 + x) * 3 + 1];
assert!(at(32, 32) > 250 && at(20, 20) > 250 && at(44, 44) > 250 && at(5, 5) < 5 && at(60, 32) < 5);
assert!(crop_box(&img, 5.0, 5.0, 5.0, 5.0, 2.0, 64, 64).is_none(), "an empty box");
assert!(crop_box(&img, f32::NAN, 0.0, 10.0, 10.0, 2.0, 64, 64).is_none());
assert!(crop_box(&img, 0.0, 0.0, 10.0, 10.0, 2.0, 0, 64).is_none());
assert!(crop_box(&img, 0.0, 0.0, 10.0, 10.0, 2.0, 100_000, 64).is_none());
let bad = Rgb { data: &data[..10], width: w, height: h };
assert!(crop_box(&bad, 0.0, 0.0, 10.0, 10.0, 2.0, 64, 64).is_none(), "wrong data length");
}
#[test]
fn faces_off_the_edge_and_tiny_or_huge_landmarks_do_not_panic() {
let data = vec![80u8; 50 * 50 * 3];
let img = Rgb { data: &data, width: 50, height: 50 };
// a box mostly outside the picture is padded by clamping, not an error
assert!(crop_box(&img, -30.0, -30.0, 20.0, 20.0, 1.5, 32, 32).is_some());
let far: [(f32, f32); 5] = [(1e6, 1e6), (1e6 + 5.0, 1e6), (1e6 + 2.0, 1e6 + 3.0), (1e6, 1e6 + 6.0), (1e6 + 5.0, 1e6 + 6.0)];
let _ = align_to_template(&img, &far, 112, 112);
let tiny: [(f32, f32); 5] = [(1.0, 1.0), (1.01, 1.0), (1.0, 1.01), (1.01, 1.01), (1.005, 1.005)];
assert!(align_to_template(&img, &tiny, 112, 112).is_none(), "landmarks a hundredth of a pixel apart");
}
}
+228
View File
@@ -0,0 +1,228 @@
//! Your own list of models to download: `catalog.json` in the models folder.
//!
//! LightCraft's built-in list only holds models whose terms let the project point at them. Anything else (a stronger
//! recogniser whose weights are for research use, a model you trained, a mirror you trust) can be added by you: each entry
//! is a model manifest (the same fields as a `face-model.json`) plus the address to fetch it from, and it then gets the same
//! Download button as a built-in one, with its licence notice shown before anything is fetched. LightCraft ships and links to
//! none of them; what the file says, and whether you may use the weights, is yours to check.
//!
//! ```json
//! { "models": [ {
//! "id": "my-recogniser", "name": "My recogniser", "version": "1", "role": "embedder",
//! "url": "https://example.org/weights/recogniser.onnx",
//! "sha256": "…64 lowercase hex digits…", "sizeBytes": 166000000,
//! "licence": { "name": "Research use only", "commercial": "no", "notice": "Not for commercial use." },
//! "provenance": "Trained on …",
//! "output": { "kind": "embedding", "dim": 512 },
//! "thresholds": { "matchCosine": 0.4 },
//! "speed": 1.7
//! } ] }
//! ```
//!
//! `speed` (optional) is how many times faster the model is than a ResNet-100, shown in Settings instead of timings.
//! `input` may be left out (112 × 112 RGB, `(x − 127.5) / 127.5`, the ArcFace convention). The file is untrusted: it is read
//! with a size limit, every entry is validated, and a bad entry is reported and skipped, never a reason to refuse the rest.
//! The `sha256` and `sizeBytes` are required, since they are what a downloaded file is checked against.
use serde::Deserialize;
use serde_json::Value;
use crate::known::{self, Download};
use crate::manifest::{self, ModelManifest, Role};
/// Largest `catalog.json` we read.
pub const MAX_CATALOG_BYTES: usize = 256 * 1024;
/// Most entries we take from it.
pub const MAX_ENTRIES: usize = 50;
/// One model from the catalog: its manifest, where to fetch it and what to call the file meanwhile.
#[derive(Clone, Debug, PartialEq)]
pub struct Entry {
pub manifest: ModelManifest,
pub url: String,
pub file_name: String,
}
impl Entry {
/// What the download machinery needs. (`sha256` and `size_bytes` are present: [`parse`] requires them.)
pub fn download(&self) -> Option<Download> {
Some(Download {
id: self.manifest.id.clone(),
url: self.url.clone(),
file_name: self.file_name.clone(),
size_bytes: self.manifest.size_bytes?,
sha256: self.manifest.sha256.clone()?,
})
}
}
/// What a catalog file holds.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct Catalog {
pub entries: Vec<Entry>,
/// One plain sentence for each entry (or the file) that could not be used.
pub errors: Vec<String>,
}
#[derive(Deserialize)]
struct Raw {
url: String,
#[serde(flatten)]
manifest: ModelManifest,
}
/// A file name for the download, from the address: letters, digits, `.`, `_`, `-`, ending in `.onnx`.
fn file_name_of(url: &str) -> String {
let last = url.split(['?', '#']).next().unwrap_or("").rsplit('/').next().unwrap_or("");
let clean: String = last.chars().filter(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-')).take(80).collect();
if clean.ends_with(".onnx") && clean.len() > 5 && !clean.starts_with('.') { clean } else { "model.onnx".to_string() }
}
fn entry(v: &Value) -> Result<Entry, String> {
let raw: Raw = serde_json::from_value(v.clone()).map_err(|e| e.to_string())?;
manifest::validate(&raw.manifest).map_err(|e| e.to_string())?;
let m = &raw.manifest;
if m.role != Role::Embedder {
return Err("only face recognition models (role \"embedder\") can be downloaded".into());
}
if m.sha256.is_none() || m.size_bytes.is_none() {
return Err("needs `sha256` and `sizeBytes`: a download is checked against them".into());
}
if !(raw.url.len() <= 500 && raw.url.starts_with("https://") && !raw.url.chars().any(|c| c.is_control() || c.is_whitespace())) {
return Err("`url` must be an https:// address of at most 500 characters, without spaces".into());
}
if known::all().iter().any(|k| k.id == m.id) {
return Err("that id belongs to a model LightCraft already knows".into());
}
if m.sha256.as_deref().is_some_and(|h| known::lookup(h).is_some()) {
return Err("LightCraft already knows a model with that file".into());
}
Ok(Entry { file_name: file_name_of(&raw.url), url: raw.url, manifest: raw.manifest })
}
/// Read a catalog file's bytes: the usable entries, and why the others were left out.
pub fn parse(bytes: &[u8]) -> Catalog {
let mut out = Catalog::default();
if bytes.len() > MAX_CATALOG_BYTES {
out.errors.push(format!("catalog.json is larger than {} KB: not read", MAX_CATALOG_BYTES / 1024));
return out;
}
// Notepad and older PowerShell begin a UTF-8 file with a byte-order mark
let bytes = bytes.strip_prefix(&[0xEF, 0xBB, 0xBF]).unwrap_or(bytes);
let value: Value = match serde_json::from_slice(bytes) {
Ok(v) => v,
Err(e) => {
out.errors.push(format!("catalog.json is not valid JSON: {e}"));
return out;
}
};
let Some(models) = value.get("models").and_then(Value::as_array) else {
out.errors.push("catalog.json needs a \"models\" list".into());
return out;
};
if models.len() > MAX_ENTRIES {
out.errors.push(format!("only the first {MAX_ENTRIES} models of the catalog are read"));
}
for (i, v) in models.iter().take(MAX_ENTRIES).enumerate() {
let label = v
.get("id")
.and_then(Value::as_str)
.map_or_else(|| format!("model {}", i + 1), |id| format!("`{}`", id.chars().take(40).collect::<String>()));
match entry(v) {
Ok(e) if out.entries.iter().any(|x| x.manifest.id == e.manifest.id) => {
out.errors.push(format!("{label}: listed twice, the first is used"))
}
Ok(e) => out.entries.push(e),
Err(why) => out.errors.push(format!("{label}: {why}")),
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
fn good(id: &str) -> Value {
json!({
"id": id, "name": "Test model", "version": "1", "role": "embedder",
"url": "https://example.org/w/recogniser-r50.onnx?download=1",
"sha256": "ab".repeat(32), "sizeBytes": 1234,
"licence": {"name": "Research only", "commercial": "no", "notice": "Not for commercial use."},
"output": {"kind": "embedding", "dim": 512}, "thresholds": {"matchCosine": 0.4}, "speed": 1.7,
})
}
fn cat(models: Vec<Value>) -> Catalog {
parse(json!({"models": models}).to_string().as_bytes())
}
#[test]
fn a_manifest_with_an_address_is_a_downloadable_model() {
let c = cat(vec![good("my-model")]);
assert_eq!(c.errors, Vec::<String>::new());
let e = &c.entries[0];
assert_eq!((e.manifest.id.as_str(), e.file_name.as_str()), ("my-model", "recogniser-r50.onnx"), "the file name comes from the address");
assert_eq!(e.manifest.licence.commercial, crate::Commercial::No, "a non-commercial model is the user's to add");
let d = e.download().unwrap();
assert_eq!((d.size_bytes, d.sha256.len(), d.url.as_str()), (1234, 64, "https://example.org/w/recogniser-r50.onnx?download=1"));
assert_eq!(e.manifest.input, crate::InputSpec::default(), "the input defaults to the ArcFace convention");
assert_eq!(e.manifest.speed, Some(1.7), "the speed ratio is the user's to state");
let mut nonsense = good("nonsense-speed");
nonsense["speed"] = json!(-3);
let c = cat(vec![nonsense]);
assert!(c.entries.is_empty() && c.errors.iter().any(|e| e.contains("speed")), "{:?}", c.errors);
}
#[test]
fn a_file_saved_with_a_byte_order_mark_still_reads() {
let mut bytes = vec![0xEF, 0xBB, 0xBF];
bytes.extend_from_slice(json!({"models": [good("with-bom")]}).to_string().as_bytes());
let c = parse(&bytes);
assert_eq!((c.entries.len(), c.errors.len()), (1, 0), "{:?}", c.errors);
}
#[test]
fn bad_entries_are_reported_and_the_rest_still_load() {
let mut no_hash = good("no-hash");
no_hash.as_object_mut().unwrap().remove("sha256");
let mut http = good("plain-http");
http["url"] = json!("http://example.org/m.onnx");
let mut spaces = good("spaces");
spaces["url"] = json!("https://example.org/a b.onnx");
let mut detector = good("a-detector");
detector["role"] = json!("detector");
detector["output"] = json!({"kind": "detector", "decoder": "yunet-v2"});
let mut bad_id = good("Bad/Id");
bad_id["id"] = json!("../escape");
let builtin = good("sface-2021dec");
let mut known_file = good("known-file");
known_file["sha256"] = json!(known::SFACE_SHA256);
let c = cat(vec![good("fine"), no_hash, http, spaces, detector, bad_id, builtin, known_file, good("fine"), json!(7), json!({"id": "x"})]);
assert_eq!(c.entries.iter().map(|e| e.manifest.id.as_str()).collect::<Vec<_>>(), ["fine"]);
assert_eq!(c.errors.len(), 10, "{:#?}", c.errors);
assert!(c.errors.iter().any(|e| e.contains("`no-hash`") && e.contains("sha256")));
assert!(c.errors.iter().any(|e| e.contains("https")));
assert!(c.errors.iter().any(|e| e.contains("already knows")));
assert!(c.errors.iter().any(|e| e.contains("listed twice")));
}
#[test]
fn a_hostile_or_broken_file_is_an_error_message_not_a_crash() {
for bytes in [&b""[..], b"not json", b"[]", b"{}", b"{\"models\": 5}", b"\xff\xfe\x00", &vec![b' '; MAX_CATALOG_BYTES + 1]] {
let c = parse(bytes);
assert!(c.entries.is_empty() && !c.errors.is_empty(), "{:?}", String::from_utf8_lossy(&bytes[..bytes.len().min(20)]));
}
// more entries than allowed: the extra are ignored, with a note
let many: Vec<Value> = (0..MAX_ENTRIES + 20).map(|i| good(&format!("m{i}"))).collect();
let c = cat(many);
assert_eq!(c.entries.len(), MAX_ENTRIES);
assert!(c.errors.iter().any(|e| e.contains("first")));
// file names from odd addresses are safe
for url in ["https://x.org/a/b/c.onnx", "https://x.org/", "https://x.org/..%2f..%2fevil.onnx", "https://x.org/q.bin", "https://x.org/.onnx"] {
let got = file_name_of(url);
assert!(!got.contains(['/', '\\']) && got.ends_with(".onnx") && !got.starts_with('.'), "{url} -> {got}");
}
}
}
+442
View File
@@ -0,0 +1,442 @@
//! A full, bounded read of a small ONNX model: its nodes, attributes and weights.
//!
//! The detector and installed recognisers share this reader and [`crate::net`]. Files are capped at
//! 512 MiB, with at most 64 million weight elements. The bytes are still treated as hostile: the
//! number of nodes, weight tensors and elements is capped, every length is checked against its parent,
//! and a weight blob whose size disagrees with its shape is an error.
use std::collections::HashMap;
use std::io::Cursor;
use crate::onnx::{ProbeError, Walker, Wire};
type Result<T> = std::result::Result<T, ProbeError>;
const MAX_NODES: usize = 20_000;
const MAX_TENSORS: usize = 20_000;
/// Elements across all weight tensors (4 bytes each as float32).
const MAX_ELEMENTS: usize = 64 * 1024 * 1024;
const MAX_NAME: usize = 4096;
const MAX_LIST: usize = 1_000_000;
#[derive(Clone, Debug, PartialEq)]
pub enum Attr {
Int(i64),
Float(f32),
Str(String),
Ints(Vec<i64>),
Floats(Vec<f32>),
/// A kind we do not read (a tensor, a sub-graph…).
Other,
}
#[derive(Clone, Debug)]
pub struct Node {
pub op: String,
pub inputs: Vec<String>,
pub outputs: Vec<String>,
pub attrs: HashMap<String, Attr>,
}
impl Node {
pub fn int(&self, name: &str) -> Option<i64> {
match self.attrs.get(name) {
Some(Attr::Int(v)) => Some(*v),
_ => None,
}
}
pub fn ints(&self, name: &str) -> Option<&[i64]> {
match self.attrs.get(name) {
Some(Attr::Ints(v)) => Some(v),
_ => None,
}
}
pub fn string(&self, name: &str) -> Option<&str> {
match self.attrs.get(name) {
Some(Attr::Str(v)) => Some(v),
_ => None,
}
}
}
#[derive(Clone, Debug, PartialEq)]
pub enum Data {
F32(Vec<f32>),
I64(Vec<i64>),
}
/// A weight tensor.
#[derive(Clone, Debug, PartialEq)]
pub struct Weight {
pub dims: Vec<usize>,
pub data: Data,
}
#[derive(Clone, Debug)]
pub struct Graph {
pub nodes: Vec<Node>,
pub weights: HashMap<String, Weight>,
/// The inputs the caller feeds (weights some exporters also list as inputs are left out), and their shapes
/// (0 for a dynamic size).
pub inputs: Vec<(String, Vec<usize>)>,
pub outputs: Vec<String>,
}
fn malformed<T>(why: &'static str) -> Result<T> {
Err(ProbeError::Malformed(why))
}
fn f32s(raw: &[u8]) -> Vec<f32> {
raw.as_chunks::<4>().0.iter().map(|b| f32::from_le_bytes(*b)).collect()
}
fn i64s(raw: &[u8]) -> Vec<i64> {
raw.as_chunks::<8>().0.iter().map(|b| i64::from_le_bytes(*b)).collect()
}
/// A list of varints in a packed field (`start..end`), appended to `out`.
fn packed_varints(w: &mut Walker<Cursor<&[u8]>>, end: u64, out: &mut Vec<u64>) -> Result<()> {
while w.pos()? < end {
if out.len() >= MAX_LIST {
return malformed("a list is too long");
}
out.push(w.varint()?);
if w.pos()? > end {
return malformed("a packed number runs past its field");
}
}
Ok(())
}
fn attribute(w: &mut Walker<Cursor<&[u8]>>, end: u64) -> Result<(String, Attr)> {
let (mut name, mut kind) = (String::new(), 0u64);
let (mut i, mut f, mut s) = (0i64, 0f32, String::new());
let (mut ints, mut floats): (Vec<u64>, Vec<f32>) = (Vec::new(), Vec::new());
while let Some((n, f_)) = w.next(end)? {
match (n, f_) {
(1, Wire::Len { start, end: e }) => {
name = w.string(start, e)?;
w.seek_to(e)?;
}
(2, Wire::Fixed32(v)) => f = f32::from_bits(v),
(3, Wire::Varint(v)) => i = v as i64,
(4, Wire::Len { start, end: e }) => {
s = w.string(start, e)?;
w.seek_to(e)?;
}
(7, Wire::Len { end: e, .. }) => {
// packed floats
let at = w.pos()?;
let raw = w.bytes(at, e, (MAX_LIST * 4) as u64)?;
if !raw.len().is_multiple_of(4) || floats.len().saturating_add(raw.len() / 4) > MAX_LIST {
return malformed("an attribute float list is invalid or too long");
}
floats.extend(f32s(&raw));
}
(7, Wire::Fixed32(v)) => {
if floats.len() >= MAX_LIST {
return malformed("an attribute float list is too long");
}
floats.push(f32::from_bits(v));
}
(8, Wire::Len { end: e, .. }) => {
packed_varints(w, e, &mut ints)?;
w.seek_to(e)?;
}
(8, Wire::Varint(v)) => {
if ints.len() >= MAX_LIST {
return malformed("a list is too long");
}
ints.push(v);
}
(20, Wire::Varint(v)) => kind = v,
(_, Wire::Len { end: e, .. }) => w.seek_to(e)?,
_ => {}
}
}
let ints = ints.into_iter().map(|v| v as i64).collect();
let value = match kind {
1 => Attr::Float(f),
2 => Attr::Int(i),
3 => Attr::Str(s),
6 => Attr::Floats(floats),
7 => Attr::Ints(ints),
_ => Attr::Other,
};
Ok((name, value))
}
fn node(w: &mut Walker<Cursor<&[u8]>>, end: u64) -> Result<Node> {
let mut n = Node { op: String::new(), inputs: Vec::new(), outputs: Vec::new(), attrs: HashMap::new() };
while let Some((num, f)) = w.next(end)? {
match (num, f) {
(1, Wire::Len { start, end: e }) => {
n.inputs.push(w.string(start, e)?);
w.seek_to(e)?;
}
(2, Wire::Len { start, end: e }) => {
n.outputs.push(w.string(start, e)?);
w.seek_to(e)?;
}
(4, Wire::Len { start, end: e }) => {
n.op = w.string(start, e)?;
w.seek_to(e)?;
}
(5, Wire::Len { end: e, .. }) => {
let (k, v) = attribute(w, e)?;
if k.is_empty() || n.attrs.insert(k, v).is_some() {
return malformed("an attribute name is empty or repeated");
}
w.seek_to(e)?;
}
(7, Wire::Len { start, end: e }) => {
let domain = w.string(start, e)?;
if !domain.is_empty() && domain != "ai.onnx" {
return malformed("custom operator domains are unsupported");
}
w.seek_to(e)?;
}
(_, Wire::Len { end: e, .. }) => w.seek_to(e)?,
_ => {}
}
if n.inputs.len() > 64 || n.outputs.len() > 64 || n.attrs.len() > 64 {
return malformed("a node is too large");
}
}
Ok(n)
}
/// A weight tensor (`TensorProto`): its name, shape and float32 or int64 values.
fn tensor(w: &mut Walker<Cursor<&[u8]>>, end: u64, budget: &mut usize) -> Result<(String, Option<Weight>)> {
let (mut name, mut dtype) = (String::new(), 0u64);
let mut dims: Vec<u64> = Vec::new();
let mut raw: Option<&[u8]> = None;
let (mut float_data, mut int_data): (Vec<f32>, Vec<u64>) = (Vec::new(), Vec::new());
let max_bytes = (*budget as u64).saturating_mul(8);
while let Some((n, f)) = w.next(end)? {
match (n, f) {
(1, Wire::Len { end: e, .. }) => {
packed_varints(w, e, &mut dims)?;
w.seek_to(e)?;
}
(1, Wire::Varint(v)) => dims.push(v),
(2, Wire::Varint(v)) => dtype = v,
(4, Wire::Len { end: e, .. }) => {
let at = w.pos()?;
let bytes = w.bytes(at, e, max_bytes)?;
if !bytes.len().is_multiple_of(4) || float_data.len().saturating_add(bytes.len() / 4) > *budget {
return malformed("invalid or excessive packed float weights");
}
float_data.extend(f32s(&bytes));
}
(4, Wire::Fixed32(v)) => {
if float_data.len() >= *budget {
return malformed("too many float weights");
}
float_data.push(f32::from_bits(v));
}
(7, Wire::Varint(v)) => {
if int_data.len() >= *budget {
return malformed("too many integer weights");
}
int_data.push(v);
}
(7, Wire::Len { end: e, .. }) => {
packed_varints(w, e, &mut int_data)?;
w.seek_to(e)?;
}
(8, Wire::Len { start, end: e }) => {
name = w.string(start, e)?;
w.seek_to(e)?;
}
(9, Wire::Len { start, end: e }) => {
if raw.is_some() || e.saturating_sub(start) > max_bytes {
return malformed("repeated or excessive raw weights");
}
let from = usize::try_from(start).map_err(|_| ProbeError::Malformed("weight offset overflow"))?;
let to = usize::try_from(e).map_err(|_| ProbeError::Malformed("weight offset overflow"))?;
raw = Some((*w.r.get_ref()).get(from..to).ok_or(ProbeError::Malformed("short raw weights"))?);
w.seek_to(e)?;
}
(13, Wire::Len { .. }) | (14, Wire::Varint(1..)) => return malformed("external tensor data is unsupported"),
(_, Wire::Len { end: e, .. }) => w.seek_to(e)?,
_ => {}
}
if dims.len() > 8 {
return malformed("a tensor has too many dimensions");
}
}
let mut elements: usize = 1;
let mut shape = Vec::new();
for d in &dims {
let d = usize::try_from(*d).map_err(|_| ProbeError::Malformed("a tensor dimension is too large"))?;
elements = elements.checked_mul(d).ok_or(ProbeError::Malformed("a tensor is too large"))?;
shape.push(d);
}
if elements > *budget {
return malformed("the weights are too large");
}
*budget -= elements;
if raw.is_some() && (!float_data.is_empty() || !int_data.is_empty()) {
return malformed("ambiguous raw and typed weights");
}
let data = match dtype {
1 => {
let v = if let Some(raw) = raw {
if elements.checked_mul(4) != Some(raw.len()) {
return malformed("float weight bytes disagree with shape");
}
f32s(raw)
} else {
float_data
};
if v.len() != elements {
return malformed("a weight tensor's size disagrees with its shape");
}
Some(Data::F32(v))
}
7 => {
let v = if let Some(raw) = raw {
if elements.checked_mul(8) != Some(raw.len()) {
return malformed("integer weight bytes disagree with shape");
}
i64s(raw)
} else {
int_data.into_iter().map(|v| v as i64).collect()
};
if v.len() != elements {
return malformed("a weight tensor's size disagrees with its shape");
}
Some(Data::I64(v))
}
_ => return malformed("only float32 and int64 initializers are supported"),
};
if name.is_empty() {
return malformed("a weight name is empty");
}
Ok((name, data.map(|data| Weight { dims: shape, data })))
}
fn value_info(w: &mut Walker<Cursor<&[u8]>>, end: u64) -> Result<(String, Vec<usize>)> {
let (mut name, mut shape) = (String::new(), Vec::new());
while let Some((n, f)) = w.next(end)? {
match (n, f) {
(1, Wire::Len { start, end: e }) => {
name = w.string(start, e)?;
w.seek_to(e)?;
}
(2, Wire::Len { end: e, .. }) => {
// TypeProto > tensor_type (1) > shape (2) > dim (1) > dim_value (1)
let mut info = crate::onnx::TensorInfo { name: String::new(), elem_type: 0, shape: Vec::new() };
crate::onnx::type_proto(w, e, &mut info)?;
shape = info
.shape
.iter()
.map(|d| match d {
crate::onnx::Dim::Fixed(v) => {
usize::try_from(*v).map_err(|_| ProbeError::Malformed("a declared dimension exceeds this platform"))
}
crate::onnx::Dim::Dynamic(_) => Ok(0),
})
.collect::<Result<_>>()?;
w.seek_to(e)?;
}
(_, Wire::Len { end: e, .. }) => w.seek_to(e)?,
_ => {}
}
}
if name.len() > MAX_NAME {
return malformed("a name is too long");
}
Ok((name, shape))
}
/// Read a bounded float32 ONNX inference model.
pub fn load(bytes: &[u8]) -> Result<Graph> {
let total = bytes.len() as u64;
if total > 512 * 1024 * 1024 {
return malformed("the model exceeds 512 MiB");
}
let info = crate::onnx::probe_reader(&mut Cursor::new(bytes))?;
if info.opset.is_none_or(|v| !(10..=17).contains(&v)) {
return malformed("only ONNX default-domain opsets 10 through 17 are supported");
}
if info.inputs.iter().chain(&info.outputs).any(|v| v.shape.iter().any(|d| matches!(d, crate::onnx::Dim::Fixed(0)))) {
return malformed("a declared input/output has a zero dimension");
}
if info.inputs.iter().chain(&info.outputs).any(|v| v.elem_type != 1) {
return malformed("model inputs and outputs must be float32");
}
if total == 0 {
return malformed("the file is empty");
}
let mut cursor = Cursor::new(bytes);
let mut w = Walker { r: &mut cursor, total, fields: 0 };
let (mut nodes, mut weights, mut inputs, mut outputs) = (Vec::new(), HashMap::new(), Vec::new(), Vec::new());
let mut budget = MAX_ELEMENTS;
let mut saw_graph = false;
while let Some((n, f)) = w.next(total)? {
match (n, f) {
(7, Wire::Len { end, .. }) => {
if saw_graph {
return malformed("the model contains multiple graphs");
}
saw_graph = true;
while let Some((gn, gf)) = w.next(end)? {
match (gn, gf) {
(1, Wire::Len { end: e, .. }) => {
if nodes.len() >= MAX_NODES {
return malformed("too many nodes");
}
nodes.push(node(&mut w, e)?);
w.seek_to(e)?;
}
(5, Wire::Len { end: e, .. }) => {
if weights.len() >= MAX_TENSORS {
return malformed("too many weight tensors");
}
let (name, wt) = tensor(&mut w, e, &mut budget)?;
if let Some(wt) = wt
&& weights.insert(name, wt).is_some()
{
return malformed("a weight is defined twice");
}
w.seek_to(e)?;
}
(11, Wire::Len { end: e, .. }) => {
if inputs.len() >= MAX_TENSORS {
return malformed("too many graph inputs");
}
inputs.push(value_info(&mut w, e)?);
w.seek_to(e)?;
}
(12, Wire::Len { end: e, .. }) => {
if outputs.len() >= 64 {
return malformed("too many graph outputs");
}
outputs.push(value_info(&mut w, e)?.0);
w.seek_to(e)?;
}
(15, Wire::Len { .. }) => return malformed("sparse initializers are unsupported"),
(_, Wire::Len { end: e, .. }) => w.seek_to(e)?,
_ => {}
}
}
w.seek_to(end)?;
}
(_, Wire::Len { end, .. }) => w.seek_to(end)?,
_ => {}
}
}
if !saw_graph || nodes.is_empty() {
return malformed("no graph");
}
inputs.retain(|(name, _)| !weights.contains_key(name));
if inputs.is_empty() || outputs.is_empty() {
return malformed("the graph has no inputs or no outputs");
}
Ok(Graph { nodes, weights, inputs, outputs })
}
#[cfg(test)]
mod tests;
+117
View File
@@ -0,0 +1,117 @@
use super::*;
use crate::synthetic::{len_field, value_info_bytes, varint, varint_field};
fn node() -> Vec<u8> {
let mut n = Vec::new();
for (id, v) in [(1, "x"), (2, "y"), (4, "Identity")] {
len_field(id, v.as_bytes(), &mut n);
}
n
}
fn weight(raw: &[u8]) -> Vec<u8> {
let mut t = Vec::new();
varint_field(1, 1, &mut t);
varint_field(2, 1, &mut t);
len_field(8, b"w", &mut t);
len_field(9, raw, &mut t);
t
}
fn model(node: &[u8], weights: &[Vec<u8>]) -> Vec<u8> {
let mut g = Vec::new();
len_field(1, node, &mut g);
for t in weights {
len_field(5, t, &mut g);
}
for (id, name) in [(11, "x"), (12, "y")] {
len_field(id, &value_info_bytes(name, 1, &[Ok(1), Ok(3), Ok(112), Ok(112)]), &mut g);
}
let mut ops = Vec::new();
varint_field(2, 11, &mut ops);
let mut out = Vec::new();
len_field(7, &g, &mut out);
len_field(8, &ops, &mut out);
out
}
#[test]
fn raw_and_typed_weights_are_exact_not_truncated_or_ambiguous() {
assert!(load(&model(&node(), &[weight(&1f32.to_le_bytes())])).is_ok());
for raw in [vec![0; 3], vec![0; 5], vec![0; 8]] {
assert!(load(&model(&node(), &[weight(&raw)])).is_err());
}
let mut t = weight(&1f32.to_le_bytes());
varint(4 << 3 | 5, &mut t);
t.extend_from_slice(&1f32.to_le_bytes());
assert!(load(&model(&node(), &[t])).is_err());
let mut t = weight(&1f32.to_le_bytes());
len_field(9, &1f32.to_le_bytes(), &mut t);
assert!(load(&model(&node(), &[t])).is_err());
// The protobuf float_data form must work as well as raw_data.
let mut t = Vec::new();
varint_field(1, 1, &mut t);
varint_field(2, 1, &mut t);
len_field(8, b"w", &mut t);
varint(4 << 3 | 5, &mut t);
t.extend_from_slice(&1f32.to_le_bytes());
assert!(load(&model(&node(), &[t])).is_ok());
}
#[test]
fn external_sparse_duplicate_custom_domain_and_wrong_dtype_are_errors() {
let t = weight(&1f32.to_le_bytes());
assert!(load(&model(&node(), &[t.clone(), t.clone()])).is_err());
for field in [13, 14] {
let mut bad = t.clone();
if field == 13 {
len_field(field, b"external", &mut bad);
} else {
varint_field(field, 1, &mut bad);
}
assert!(load(&model(&node(), &[bad])).is_err());
}
let mut bad = t.clone();
varint_field(2, 10, &mut bad);
assert!(load(&model(&node(), &[bad])).is_err());
let mut n = node();
len_field(7, b"vendor.example", &mut n);
assert!(load(&model(&n, &[])).is_err());
let mut attr = Vec::new();
len_field(1, b"ratio", &mut attr);
varint_field(20, 1, &mut attr);
let mut n = node();
len_field(5, &attr, &mut n);
len_field(5, &attr, &mut n);
assert!(load(&model(&n, &[])).is_err());
let mut g = Vec::new();
len_field(15, b"sparse", &mut g);
let mut bad = model(&node(), &[]);
len_field(7, &g, &mut bad);
assert!(load(&bad).is_err());
}
#[test]
fn overflowing_and_cross_boundary_varints_long_strings_and_bad_shapes_are_errors() {
assert!(load(&[0x08, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x02]).is_err());
let mut t = Vec::new();
len_field(1, &[0x80], &mut t);
varint_field(2, 1, &mut t);
len_field(8, b"w", &mut t);
assert!(load(&model(&node(), &[t])).is_err());
let mut n = node();
len_field(1, &vec![b'x'; 4097], &mut n);
assert!(load(&model(&n, &[])).is_err());
let mut n = node();
len_field(1, &[0xff], &mut n);
assert!(load(&model(&n, &[])).is_err());
let mut t = Vec::new();
varint_field(1, u64::MAX, &mut t);
varint_field(1, 8, &mut t);
varint_field(2, 1, &mut t);
len_field(8, b"w", &mut t);
assert!(load(&model(&node(), &[t])).is_err());
let mut bad = model(&node(), &[]);
let mut ops = Vec::new();
varint_field(2, 999, &mut ops);
len_field(8, &ops, &mut bad);
assert!(load(&bad).is_err());
}
proptest::proptest! {
#[test]
fn arbitrary_graph_bytes_do_not_panic(bytes in proptest::collection::vec(proptest::num::u8::ANY,0..4096)) { let _=load(&bytes); }
}
+46
View File
@@ -0,0 +1,46 @@
//! SHA-256 of a model file, streamed (a model can be hundreds of megabytes).
use std::io::{self, Read};
use std::path::Path;
use sha2::{Digest, Sha256};
/// Lowercase hex SHA-256 of everything `r` yields.
pub fn sha256_reader<R: Read>(mut r: R) -> io::Result<String> {
let mut h = Sha256::new();
let mut buf = vec![0u8; 1 << 20];
loop {
let n = r.read(&mut buf)?;
if n == 0 {
break;
}
h.update(buf.get(..n).unwrap_or(&[]));
}
Ok(hex(&h.finalize()))
}
/// Lowercase hex SHA-256 of a file.
pub fn sha256_file(path: &Path) -> io::Result<String> {
sha256_reader(io::BufReader::new(std::fs::File::open(path)?))
}
/// Lowercase hex SHA-256 of `bytes`.
pub fn sha256_hex(bytes: &[u8]) -> String {
hex(&Sha256::digest(bytes))
}
fn hex(bytes: &[u8]) -> String {
bytes.iter().map(|b| format!("{b:02x}")).collect()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn known_digests() {
assert_eq!(sha256_hex(b""), "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855");
assert_eq!(sha256_hex(b"abc"), "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad");
assert_eq!(sha256_reader(&b"abc"[..]).unwrap(), sha256_hex(b"abc"));
}
}
+223
View File
@@ -0,0 +1,223 @@
//! The models LightCraft recognises by their SHA-256, with what is honestly known about each.
//!
//! The licence and provenance texts here are what the user reads before enabling a model, so they say what
//! is *not* known too. Weights are never part of LightCraft: every model is an opt-in the user installs.
use crate::manifest::{Colour, Commercial, InputSpec, Licence, ModelManifest, OutputSpec, Resize, Role, Thresholds};
/// Detector output decoders built into LightCraft.
pub const DECODERS: &[&str] = &["yunet-v2"];
/// The id of the face detector LightCraft runs (see [`yunet`]).
pub const YUNET_ID: &str = "yunet-2023mar";
pub const YUNET_SHA256: &str = "8f2383e4dd3cfbb4553ea8718107fc0423210dc964f9f4280604804ed2552fa4";
pub const SFACE_SHA256: &str = "0ba9fbfa01b5270c96627c4ef784da859931e02f04419c829e83484087c34e79";
pub const AURAFACE_SHA256: &str = "a7933ea5330113b01c9b60351d8f4c33003f145d8470ac5f0e52ee2effe25c60";
fn licence(name: &str, commercial: Commercial, url: &str, notice: &str) -> Licence {
Licence { name: name.into(), commercial, url: Some(url.into()), notice: notice.into() }
}
/// YuNet 2023mar (OpenCV Zoo): a small face detector.
pub fn yunet() -> ModelManifest {
ModelManifest {
id: YUNET_ID.into(),
name: "YuNet (face detector)".into(),
version: "2023mar".into(),
role: Role::Detector,
licence: licence(
"MIT",
Commercial::Yes,
"https://github.com/opencv/opencv_zoo/blob/main/models/face_detection_yunet/LICENSE",
"MIT licence, copyright Shiqi Yu. A separate opt-in download.",
),
source: Some("https://github.com/opencv/opencv_zoo/tree/main/models/face_detection_yunet".into()),
sha256: Some(YUNET_SHA256.into()),
size_bytes: Some(232_589),
provenance:
"Trained on WIDER FACE, whose terms are not settled by the weights' MIT licence. A detector: it finds faces, it does not identify anyone."
.into(),
input: InputSpec { width: 640, height: 640, colour: Colour::Bgr, mean: [0.0; 3], std: [1.0; 3], resize: Resize::Letterbox },
output: OutputSpec::Detector { decoder: "yunet-v2".into() },
thresholds: Thresholds { match_cosine: None, score: Some(0.6), nms_iou: Some(0.3) },
speed: None,
}
}
/// SFace 2021dec (OpenCV Zoo). Its match threshold is a starting point: on 755 named faces (busts, matched across
/// shots more than five seconds apart) 0.55 gave 97.5% right suggestions for 88% of faces; `faces.evaluate` checks it on
/// your own photos.
pub fn sface() -> ModelManifest {
ModelManifest {
id: "sface-2021dec".into(),
name: "SFace (face recogniser)".into(),
version: "2021dec".into(),
role: Role::Embedder,
licence: licence(
"Apache-2.0 (as labelled by OpenCV Zoo)",
Commercial::Unknown,
"https://github.com/opencv/opencv_zoo/tree/main/models/face_recognition_sface",
"Labelled Apache-2.0, but what it was trained on is not documented, and two questions about commercial use (opencv_zoo issues 313 and 318) are unanswered. Fine to try for yourself; do not redistribute.",
),
source: Some("https://github.com/opencv/opencv_zoo/tree/main/models/face_recognition_sface".into()),
sha256: Some(SFACE_SHA256.into()),
size_bytes: Some(38_696_353),
provenance: "Undocumented. The original SFace repository mentions CASIA-WebFace, VGGFace2 and MS1MV2.".into(),
input: InputSpec { width: 112, height: 112, colour: Colour::Rgb, mean: [0.0; 3], std: [1.0; 3], resize: Resize::Stretch },
output: OutputSpec::Embedding { dim: 128 },
thresholds: Thresholds { match_cosine: Some(0.55), ..Thresholds::default() },
speed: Some(6.9),
}
}
/// AuraFace v1 `glintr100` (fal.ai). Starting-point threshold: on the same 755 faces 0.40 gave 96.6% right suggestions
/// for 56% of faces (0.30: 97.3% for 85%).
pub fn auraface() -> ModelManifest {
ModelManifest {
id: "auraface-v1".into(),
name: "AuraFace v1 (face recogniser)".into(),
version: "1".into(),
role: Role::Embedder,
licence: licence(
"Apache-2.0",
Commercial::Yes,
"https://huggingface.co/fal/AuraFace-v1",
"Apache-2.0. Its training data is described only as a commercial dataset, with no dataset named and no consent statement, and it covers some ethnicities less well. About 261 MB, and slow on a CPU.",
),
source: Some("https://huggingface.co/fal/AuraFace-v1".into()),
sha256: Some(AURAFACE_SHA256.into()),
size_bytes: Some(260_694_151),
provenance: "Undisclosed: \"a commercial dataset comprising face images from various sources\".".into(),
input: InputSpec::default(),
output: OutputSpec::Embedding { dim: 512 },
thresholds: Thresholds { match_cosine: Some(0.40), ..Thresholds::default() },
speed: Some(1.0),
}
}
/// Every model we know by hash.
pub fn all() -> Vec<ModelManifest> {
vec![yunet(), sface(), auraface()]
}
/// The known model whose file has this SHA-256 (lowercase hex).
pub fn lookup(sha256: &str) -> Option<ModelManifest> {
all().into_iter().find(|m| m.sha256.as_deref() == Some(sha256))
}
/// Where the user's "Download" button fetches a model from. The URL is pinned to a commit of the model's own
/// repository, and what arrives is checked against the manifest's size and SHA-256 before anything uses it.
#[derive(Clone, Debug, PartialEq)]
pub struct Download {
pub id: String,
pub url: String,
/// The name the file is kept under while it waits for the user to accept its terms.
pub file_name: String,
pub size_bytes: u64,
pub sha256: String,
}
/// Pinned download addresses by model id. Only models whose terms allow us to point at them are here: a model
/// that is marked non-commercial is never offered for download (the user brings the file themselves).
const SOURCES: &[(&str, &str, &str)] = &[
(
YUNET_ID,
"https://github.com/opencv/opencv_zoo/raw/25f423d0e04c31a17254620e58febd7386da523b/models/face_detection_yunet/face_detection_yunet_2023mar.onnx",
"face_detection_yunet_2023mar.onnx",
),
(
"sface-2021dec",
"https://github.com/opencv/opencv_zoo/raw/25f423d0e04c31a17254620e58febd7386da523b/models/face_recognition_sface/face_recognition_sface_2021dec.onnx",
"face_recognition_sface_2021dec.onnx",
),
("auraface-v1", "https://huggingface.co/fal/AuraFace-v1/resolve/af6d057c9b0ec4071d4c49c80e3539258798b609/glintr100.onnx", "glintr100.onnx"),
];
/// How to download the model with this id, if LightCraft offers to.
pub fn download(id: &str) -> Option<Download> {
let (_, url, file_name) = SOURCES.iter().find(|(i, _, _)| *i == id)?;
let m = all().into_iter().find(|m| m.id == id)?;
if m.licence.commercial == Commercial::No {
return None;
}
Some(Download { id: m.id, url: (*url).into(), file_name: (*file_name).into(), size_bytes: m.size_bytes?, sha256: m.sha256? })
}
/// "github.com" for a download address: what the user is told the file comes from.
pub fn host(url: &str) -> &str {
url.strip_prefix("https://").and_then(|r| r.split('/').next()).unwrap_or("")
}
impl Download {
/// The address without the file name (the downloader's "mirror": it fetches `<mirror>/<file name>`).
pub fn base(&self) -> &str {
self.url.strip_suffix(self.file_name.as_str()).map_or(self.url.as_str(), |b| b.trim_end_matches('/'))
}
/// The site the file comes from, as the user is told.
pub fn host(&self) -> &str {
host(&self.url)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::manifest::validate;
#[test]
fn known_models_are_valid_and_unique() {
let all = all();
for m in &all {
assert_eq!(validate(m), Ok(()), "{}", m.id);
}
let mut ids: Vec<_> = all.iter().map(|m| &m.id).collect();
ids.sort();
ids.dedup();
assert_eq!(ids.len(), all.len(), "unique ids");
let mut hashes: Vec<_> = all.iter().filter_map(|m| m.sha256.as_deref()).collect();
hashes.sort();
hashes.dedup();
assert_eq!(hashes.len(), all.len(), "unique hashes");
}
#[test]
fn lookup_finds_by_hash() {
assert_eq!(lookup(AURAFACE_SHA256).map(|m| m.id), Some("auraface-v1".to_string()));
assert_eq!(lookup(YUNET_SHA256).map(|m| m.role), Some(Role::Detector));
assert!(lookup(&"0".repeat(64)).is_none());
assert!(lookup("").is_none());
}
#[test]
fn downloads_are_pinned_checked_and_only_for_models_we_may_point_at() {
for m in all() {
let Some(d) = download(&m.id) else { continue };
assert!(d.url.starts_with("https://"), "{}: https only", d.id);
assert!(!d.url.contains("/main/") && !d.url.contains("/master/"), "{}: pinned to a commit, not a branch", d.id);
assert!(d.file_name.ends_with(".onnx") && !d.file_name.contains(['/', '\\']), "{}", d.file_name);
assert!(d.url.ends_with(&format!("/{}", d.file_name)), "{}: the address ends in the file's name", d.id);
assert!(d.base().starts_with("https://") && !d.base().ends_with('/') && !d.base().ends_with(".onnx"), "{}", d.base());
assert_eq!((Some(d.size_bytes), Some(d.sha256.as_str())), (m.size_bytes, m.sha256.as_deref()), "{}", d.id);
assert!(m.licence.commercial != Commercial::No, "{} is non-commercial and must not be offered", d.id);
}
assert_eq!(download("yunet-2023mar").map(|d| d.host().to_string()).as_deref(), Some("github.com"));
assert_eq!(download("sface-2021dec").map(|d| host(&d.url).to_string()).as_deref(), Some("github.com"));
assert_eq!(download("auraface-v1").map(|d| host(&d.url).to_string()).as_deref(), Some("huggingface.co"));
// unknown ids and path tricks have nothing to download
for id in ["nope", "", "../sface-2021dec", "SFACE-2021DEC"] {
assert!(download(id).is_none(), "{id}");
}
// every address in the table belongs to a known model
for (id, _, _) in SOURCES {
assert!(all().iter().any(|m| m.id == *id), "{id}");
}
}
#[test]
fn unresolved_models_say_so() {
assert_eq!(sface().licence.commercial, Commercial::Unknown);
assert!(sface().provenance.to_lowercase().contains("undocumented"));
assert!(auraface().provenance.to_lowercase().contains("undisclosed"));
}
}
+26
View File
@@ -0,0 +1,26 @@
//! Face models for LightCraft: what a model is ([`ModelManifest`]), whether to trust a description of one
//! ([`manifest::validate`]), the models we know by hash ([`known`]), and a probe that reads an unknown
//! `.onnx` file's input and output shapes without running it ([`onnx`]), so installing a model of your
//! own can be one drop and one licence prompt.
//!
//! Everything here treats model files and manifests as hostile input: sizes are capped, numbers must be
//! finite and sane, and a malformed file is an error, never a panic. Detection and recognition run
//! on the same checked, single-thread pure-Rust CPU interpreter.
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
pub mod align;
pub mod catalog;
pub mod graph;
pub mod hash;
pub mod known;
pub mod manifest;
pub mod matching;
pub mod net;
pub mod onnx;
pub mod runtime;
pub mod suggest;
pub mod synthetic;
pub mod yunet;
pub use manifest::{Colour, Commercial, InputSpec, Licence, ManifestError, ModelManifest, OutputSpec, Resize, Role, Thresholds};
+386
View File
@@ -0,0 +1,386 @@
//! A face model's manifest: what the model is, what it needs and who may use it, as plain data.
//!
//! A manifest comes from the known-model table, from a suggestion made for a file the user dropped in, or
//! from a `face-model.json` next to a model. All of those are untrusted: [`validate`] must pass before a
//! manifest is used, and [`parse`] applies it after reading JSON of bounded size.
use serde::{Deserialize, Serialize};
/// Largest `face-model.json` we read.
pub const MAX_MANIFEST_BYTES: usize = 64 * 1024;
/// Largest model file we accept (bytes).
pub const MAX_MODEL_BYTES: u64 = 512 * 1024 * 1024;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum Role {
/// Finds faces (boxes, and landmarks when the model has them).
Detector,
/// Turns an aligned face into a vector whose distance says whether two faces are the same person.
Embedder,
}
/// Whether the weights may be used commercially.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum Commercial {
Yes,
/// Research or personal use only. Never bundled or redistributed by LightCraft.
No,
/// The licence or the training data is unclear: the user is told so before installing.
Unknown,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct Licence {
/// SPDX id or a short name ("MIT", "Apache-2.0", "InsightFace non-commercial research").
pub name: String,
pub commercial: Commercial,
pub url: Option<String>,
/// Plain-language terms shown before the model is enabled.
pub notice: String,
}
impl Default for Licence {
fn default() -> Self {
Self { name: String::new(), commercial: Commercial::Unknown, url: None, notice: String::new() }
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum Colour {
Rgb,
Bgr,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum Resize {
/// Stretch the crop to the input size (aligned face crops).
Stretch,
/// Keep the aspect ratio and pad (whole photos for a detector).
Letterbox,
}
/// How an image becomes the model's input: NCHW float, `(pixel − mean) / std` per channel, pixel in 0..255.
#[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct InputSpec {
pub width: u32,
pub height: u32,
pub colour: Colour,
pub mean: [f32; 3],
pub std: [f32; 3],
pub resize: Resize,
}
impl Default for InputSpec {
/// ArcFace-style: 112 × 112, RGB, `(x − 127.5) / 127.5`.
fn default() -> Self {
Self { width: 112, height: 112, colour: Colour::Rgb, mean: [127.5; 3], std: [127.5; 3], resize: Resize::Stretch }
}
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "camelCase")]
pub enum OutputSpec {
/// One vector of `dim` numbers per face.
Embedding { dim: u32 },
/// A detector whose outputs a built-in decoder understands (`"yunet-v2"`).
Detector { decoder: String },
}
#[derive(Clone, Copy, Debug, Default, PartialEq, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct Thresholds {
/// Embedders: cosine similarity at or above which two faces are suggested as a match.
pub match_cosine: Option<f32>,
/// Detectors: lowest detection score kept.
pub score: Option<f32>,
/// Detectors: overlap above which two boxes are one face.
pub nms_iou: Option<f32>,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct ModelManifest {
/// Lowercase letters, digits, `.` `_` `-`.
pub id: String,
pub name: String,
pub version: String,
pub role: Role,
#[serde(default)]
pub licence: Licence,
/// Where the weights come from (an `https://` page or file), for the user's information.
#[serde(default)]
pub source: Option<String>,
/// Lowercase hex SHA-256 of the `.onnx` file, when known.
#[serde(default)]
pub sha256: Option<String>,
#[serde(default)]
pub size_bytes: Option<u64>,
/// What the model was trained on, in plain words; says "undisclosed" when it is.
#[serde(default)]
pub provenance: String,
#[serde(default)]
pub input: InputSpec,
pub output: OutputSpec,
#[serde(default)]
pub thresholds: Thresholds,
/// How fast it is as a multiple of a ResNet-100 model's speed (`1.0`): `2.0` is twice as fast. Shown instead of
/// timings, which depend on the computer; leave it out when you have not measured it.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub speed: Option<f32>,
}
/// The reference model the `speed` ratios are measured against.
pub const SPEED_REFERENCE: &str = "a ResNet-100 model";
/// A speed ratio in plain words, as times faster than the reference ("4.6× faster than a ResNet-100 model").
pub fn describe_speed(speed: f32) -> String {
if !(speed.is_finite() && speed > 0.0) {
return String::new();
}
let times = |x: f32| {
let s = format!("{x:.1}");
s.strip_suffix(".0").map_or(s.clone(), str::to_string)
};
if speed >= 1.15 {
format!("{}× faster than {SPEED_REFERENCE}", times(speed))
} else if speed > 0.87 {
format!("Same speed as {SPEED_REFERENCE}")
} else {
format!("{}× slower than {SPEED_REFERENCE}", times(1.0 / speed))
}
}
#[derive(Debug, PartialEq, thiserror::Error)]
pub enum ManifestError {
#[error("`{0}` is not valid: {1}")]
Field(&'static str, String),
#[error("not a face-model manifest: {0}")]
Parse(String),
#[error("a manifest may be at most {MAX_MANIFEST_BYTES} bytes")]
TooLarge,
}
fn bad<T>(field: &'static str, why: impl Into<String>) -> Result<T, ManifestError> {
Err(ManifestError::Field(field, why.into()))
}
/// A model id is a folder name: 1 to 64 characters, starting with a lowercase letter or digit, then
/// lowercase letters, digits, `.`, `_` or `-`. So it is never `.`, `..`, hidden, or a path.
pub fn valid_id(id: &str) -> bool {
let mut bytes = id.bytes();
let first_ok = bytes.next().is_some_and(|b| b.is_ascii_lowercase() || b.is_ascii_digit());
first_ok && id.len() <= 64 && bytes.all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || matches!(b, b'.' | b'_' | b'-'))
}
fn text(field: &'static str, s: &str, min: usize, max: usize) -> Result<(), ManifestError> {
let n = s.chars().count();
if n < min || n > max {
return bad(field, format!("must be {min} to {max} characters"));
}
if s.chars().any(|c| c.is_control() && c != '\n') {
return bad(field, "must not contain control characters");
}
Ok(())
}
fn url(field: &'static str, u: &Option<String>) -> Result<(), ManifestError> {
match u {
None => Ok(()),
Some(u) if u.len() <= 500 && u.starts_with("https://") && !u.chars().any(|c| c.is_control() || c.is_whitespace()) => Ok(()),
Some(_) => bad(field, "must be an https:// address of at most 500 characters"),
}
}
fn ratio(field: &'static str, v: Option<f32>, lo: f32, hi: f32) -> Result<(), ManifestError> {
match v {
Some(v) if !(v.is_finite() && (lo..=hi).contains(&v)) => bad(field, format!("must be a number from {lo} to {hi}")),
_ => Ok(()),
}
}
/// Check a manifest from any source. Everything the rest of LightCraft relies on (sizes it will allocate,
/// numbers it will divide by, text it will show) is bounded here.
pub fn validate(m: &ModelManifest) -> Result<(), ManifestError> {
if !valid_id(&m.id) {
return bad("id", "must be 1 to 64 characters: lowercase letters and digits, then also '.', '_' or '-'");
}
text("name", &m.name, 1, 120)?;
text("version", &m.version, 1, 40)?;
text("licence.name", &m.licence.name, 0, 80)?;
text("licence.notice", &m.licence.notice, 0, 800)?;
text("provenance", &m.provenance, 0, 800)?;
url("licence.url", &m.licence.url)?;
url("source", &m.source)?;
if let Some(h) = &m.sha256
&& !(h.len() == 64 && h.bytes().all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b)))
{
return bad("sha256", "must be 64 lowercase hexadecimal digits");
}
if m.size_bytes.is_some_and(|s| s == 0 || s > MAX_MODEL_BYTES) {
return bad("sizeBytes", format!("must be 1 to {MAX_MODEL_BYTES}"));
}
let i = &m.input;
if !(16..=4096).contains(&i.width) || !(16..=4096).contains(&i.height) {
return bad("input.width/height", "must be 16 to 4096");
}
if i.mean.iter().any(|v| !v.is_finite() || v.abs() > 1000.0) {
return bad("input.mean", "must be finite numbers within ±1000");
}
if i.std.iter().any(|v| !v.is_finite() || v.abs() < 1e-3 || v.abs() > 1000.0) {
return bad("input.std", "must be finite, not zero, within ±1000");
}
match &m.output {
OutputSpec::Embedding { dim } => {
if m.role != Role::Embedder {
return bad("output", "an embedding output needs the embedder role");
}
if !(1..=4096).contains(dim) {
return bad("output.dim", "must be 1 to 4096");
}
}
OutputSpec::Detector { decoder } => {
if m.role != Role::Detector {
return bad("output", "a detector output needs the detector role");
}
if !crate::known::DECODERS.contains(&decoder.as_str()) {
return bad("output.decoder", format!("unknown decoder `{}`", decoder.chars().take(40).collect::<String>()));
}
}
}
ratio("thresholds.matchCosine", m.thresholds.match_cosine, -1.0, 1.0)?;
ratio("thresholds.score", m.thresholds.score, 0.0, 1.0)?;
ratio("thresholds.nmsIou", m.thresholds.nms_iou, 0.0, 1.0)?;
ratio("speed", m.speed, 0.01, 1000.0)?;
Ok(())
}
/// Read a `face-model.json` (at most [`MAX_MANIFEST_BYTES`]) and validate it.
pub fn parse(bytes: &[u8]) -> Result<ModelManifest, ManifestError> {
if bytes.len() > MAX_MANIFEST_BYTES {
return Err(ManifestError::TooLarge);
}
let m: ModelManifest = serde_json::from_slice(bytes).map_err(|e| ManifestError::Parse(e.to_string()))?;
validate(&m)?;
Ok(m)
}
#[cfg(test)]
mod tests {
use super::*;
fn good() -> ModelManifest {
ModelManifest {
id: "example-r50".into(),
name: "Example R50".into(),
version: "1".into(),
role: Role::Embedder,
licence: Licence {
name: "MIT".into(),
commercial: Commercial::Yes,
url: Some("https://example.org/licence".into()),
notice: String::new(),
},
source: Some("https://example.org/model.onnx".into()),
sha256: Some("a".repeat(64)),
size_bytes: Some(1234),
provenance: "synthetic".into(),
input: InputSpec::default(),
output: OutputSpec::Embedding { dim: 512 },
thresholds: Thresholds { match_cosine: Some(0.4), ..Default::default() },
speed: Some(1.7),
}
}
#[test]
fn speed_reads_as_times_faster_than_the_reference() {
assert_eq!(describe_speed(7.0), "7× faster than a ResNet-100 model");
assert_eq!(describe_speed(4.6), "4.6× faster than a ResNet-100 model");
assert_eq!(describe_speed(1.7), "1.7× faster than a ResNet-100 model");
assert_eq!(describe_speed(1.0), "Same speed as a ResNet-100 model");
assert_eq!(describe_speed(0.5), "2× slower than a ResNet-100 model");
for odd in [0.0, -1.0, f32::NAN, f32::INFINITY] {
assert_eq!(describe_speed(odd), "", "{odd}");
}
}
#[test]
fn a_good_manifest_passes_and_round_trips() {
let m = good();
assert_eq!(validate(&m), Ok(()));
let json = serde_json::to_vec(&m).unwrap();
assert_eq!(parse(&json).unwrap(), m);
}
#[test]
fn every_field_is_bounded() {
let cases: Vec<(&str, Box<dyn Fn(&mut ModelManifest)>)> = vec![
("empty id", Box::new(|m| m.id = String::new())),
("uppercase id", Box::new(|m| m.id = "Bad".into())),
("path in id", Box::new(|m| m.id = "../etc".into())),
("dot id", Box::new(|m| m.id = ".".into())),
("dotdot id", Box::new(|m| m.id = "..".into())),
("hidden id", Box::new(|m| m.id = ".hidden".into())),
("slash id", Box::new(|m| m.id = "a/b".into())),
("backslash id", Box::new(|m| m.id = r"a\b".into())),
("long id", Box::new(|m| m.id = "a".repeat(65))),
("empty name", Box::new(|m| m.name = String::new())),
("control in name", Box::new(|m| m.name = "a\u{7}b".into())),
("huge notice", Box::new(|m| m.licence.notice = "x".repeat(801))),
("http source", Box::new(|m| m.source = Some("http://example.org/m.onnx".into()))),
("file source", Box::new(|m| m.source = Some("file:///etc/passwd".into()))),
("space in url", Box::new(|m| m.source = Some("https://a b".into()))),
("short sha", Box::new(|m| m.sha256 = Some("abc".into()))),
("uppercase sha", Box::new(|m| m.sha256 = Some("A".repeat(64)))),
("zero size", Box::new(|m| m.size_bytes = Some(0))),
("huge size", Box::new(|m| m.size_bytes = Some(u64::MAX))),
("tiny input", Box::new(|m| m.input.width = 4)),
("huge input", Box::new(|m| m.input.height = 1 << 20)),
("nan mean", Box::new(|m| m.input.mean[1] = f32::NAN)),
("inf mean", Box::new(|m| m.input.mean[0] = f32::INFINITY)),
("zero std", Box::new(|m| m.input.std[2] = 0.0)),
("nan std", Box::new(|m| m.input.std[0] = f32::NAN)),
("zero dim", Box::new(|m| m.output = OutputSpec::Embedding { dim: 0 })),
("huge dim", Box::new(|m| m.output = OutputSpec::Embedding { dim: 10_000_000 })),
("role mismatch", Box::new(|m| m.role = Role::Detector)),
(
"unknown decoder",
Box::new(|m| {
m.role = Role::Detector;
m.output = OutputSpec::Detector { decoder: "made-up".into() };
}),
),
("nan threshold", Box::new(|m| m.thresholds.match_cosine = Some(f32::NAN))),
("threshold range", Box::new(|m| m.thresholds.score = Some(1.5))),
("nan speed", Box::new(|m| m.speed = Some(f32::NAN))),
("zero speed", Box::new(|m| m.speed = Some(0.0))),
("huge speed", Box::new(|m| m.speed = Some(1e9))),
];
for (what, f) in cases {
let mut m = good();
f(&mut m);
assert!(validate(&m).is_err(), "{what} must be rejected");
}
}
#[test]
fn hostile_json_is_an_error() {
assert!(matches!(parse(&vec![b' '; MAX_MANIFEST_BYTES + 1]), Err(ManifestError::TooLarge)));
for bytes in [
&b""[..],
b"null",
b"[]",
b"{",
b"{\"id\": 1}",
b"\xff\xfe\x00",
b"{\"id\":\"x\",\"name\":\"n\",\"version\":\"1\",\"role\":\"embedder\"}",
] {
assert!(parse(bytes).is_err(), "{:?}", String::from_utf8_lossy(bytes));
}
}
}
+128
View File
@@ -0,0 +1,128 @@
//! Who does this face look like? Rank the people in a gallery of named faces by how close their faces are,
//! and suggest one only when the match is strong *and* clearly ahead of the runner-up.
//!
//! Embeddings are unit vectors, so closeness is the dot product (cosine similarity). A person's score is the
//! mean of their best few faces (one stray, wrongly named face cannot make a stranger look like them). The
//! result is only ever a suggestion for the user to confirm: it is better to leave a face unnamed than to
//! name it wrongly.
use std::cmp::Ordering;
/// How many of a person's best matching faces are averaged into their score.
pub const TOP_FACES: usize = 2;
#[derive(Clone, Debug, PartialEq)]
pub struct Ranked {
pub name: String,
pub score: f32,
/// How many of their faces the score is based on.
pub faces: usize,
}
#[derive(Clone, Debug, PartialEq)]
pub struct Suggestion {
pub name: String,
pub score: f32,
/// The next best person, and how far behind.
pub runner_up: Option<(String, f32)>,
}
/// Cosine similarity of two unit vectors. `None` when their lengths differ or either has a non-finite number.
pub fn cosine(a: &[f32], b: &[f32]) -> Option<f32> {
if a.len() != b.len() || a.is_empty() {
return None;
}
let d: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
d.is_finite().then_some(d)
}
/// Rank the people in `gallery` (a name and one face's embedding per entry; a person appears once per face),
/// best first. Names are compared case-insensitively and shown as first seen.
pub fn rank(query: &[f32], gallery: &[(&str, &[f32])]) -> Vec<Ranked> {
let mut people: Vec<(String, String, Vec<f32>)> = Vec::new(); // (key, shown name, similarities)
for (name, emb) in gallery {
let Some(c) = cosine(query, emb) else { continue };
let key = name.to_lowercase();
match people.iter_mut().find(|p| p.0 == key) {
Some(p) => p.2.push(c),
None => people.push((key, (*name).to_string(), vec![c])),
}
}
let mut out: Vec<Ranked> = people
.into_iter()
.map(|(_, name, mut sims)| {
sims.sort_by(|a, b| b.partial_cmp(a).unwrap_or(Ordering::Equal));
let top = sims.get(..sims.len().min(TOP_FACES)).unwrap_or(&[]);
Ranked { name, score: top.iter().sum::<f32>() / top.len().max(1) as f32, faces: sims.len() }
})
.collect();
out.sort_by(|a, b| b.score.partial_cmp(&a.score).unwrap_or(Ordering::Equal).then_with(|| a.name.cmp(&b.name)));
out
}
/// The best person if they pass both bars: a score of at least `threshold`, and at least `margin` ahead of the
/// next person (when there is one).
pub fn suggest(ranked: &[Ranked], threshold: f32, margin: f32) -> Option<Suggestion> {
let best = ranked.first()?;
let next = ranked.get(1);
if best.score.is_nan() || best.score < threshold || next.is_some_and(|n| best.score - n.score < margin) {
return None;
}
Some(Suggestion { name: best.name.clone(), score: best.score, runner_up: next.map(|n| (n.name.clone(), n.score)) })
}
#[cfg(test)]
mod tests {
use super::*;
fn unit(v: &[f32]) -> Vec<f32> {
let n = v.iter().map(|x| x * x).sum::<f32>().sqrt();
v.iter().map(|x| x / n).collect()
}
#[test]
fn people_are_ranked_by_their_closest_faces() {
let (a1, a2, b1) = (unit(&[1.0, 0.1, 0.0]), unit(&[0.9, 0.2, 0.0]), unit(&[0.0, 1.0, 0.2]));
let gallery: Vec<(&str, &[f32])> = vec![("Ann", &a1), ("ann", &a2), ("Bob", &b1)];
let ranked = rank(&unit(&[1.0, 0.0, 0.0]), &gallery);
assert_eq!(ranked.iter().map(|r| r.name.as_str()).collect::<Vec<_>>(), vec!["Ann", "Bob"], "case-insensitive, shown as first seen");
assert_eq!(ranked[0].faces, 2);
assert!(ranked[0].score > 0.95 && ranked[1].score < 0.2);
// a stray, wrongly named face cannot carry a person: Cy has one far face and one good one
let (c_good, c_bad) = (unit(&[1.0, 0.0, 0.0]), unit(&[0.0, 0.0, 1.0]));
let g2: Vec<(&str, &[f32])> = vec![("Cy", &c_good), ("Cy", &c_bad), ("Di", &a1), ("Di", &a2)];
let r2 = rank(&unit(&[1.0, 0.0, 0.0]), &g2);
assert_eq!(r2[0].name, "Di", "{r2:?}");
}
#[test]
fn a_suggestion_needs_a_strong_match_clearly_ahead() {
let r = |name: &str, score: f32| Ranked { name: name.into(), score, faces: 1 };
// strong and clear
let s = suggest(&[r("Ann", 0.62), r("Bob", 0.20)], 0.45, 0.08).unwrap();
assert_eq!((s.name.as_str(), s.runner_up.as_ref().map(|x| x.0.as_str())), ("Ann", Some("Bob")));
// strong but too close to the runner-up: nobody
assert!(suggest(&[r("Ann", 0.62), r("Bob", 0.58)], 0.45, 0.08).is_none());
// clear but not strong enough
assert!(suggest(&[r("Ann", 0.40), r("Bob", 0.10)], 0.45, 0.08).is_none());
// a single person has no runner-up to be ahead of
assert!(suggest(&[r("Ann", 0.5)], 0.45, 0.08).is_some());
assert!(suggest(&[], 0.45, 0.08).is_none());
assert!(suggest(&[r("Ann", f32::NAN)], 0.45, 0.08).is_none());
}
#[test]
fn mismatched_or_non_finite_embeddings_are_skipped_not_panics() {
assert!(cosine(&[1.0], &[1.0, 0.0]).is_none());
assert!(cosine(&[], &[]).is_none());
assert!(cosine(&[f32::NAN], &[1.0]).is_none());
let good = unit(&[1.0, 0.0]);
let bad = [f32::NAN, 0.0];
let short = [1.0f32];
let gallery: Vec<(&str, &[f32])> = vec![("Ann", &good), ("Nan", &bad), ("Short", &short)];
let ranked = rank(&good, &gallery);
assert_eq!(ranked.len(), 1);
assert!(rank(&[], &gallery).is_empty());
assert!(rank(&good, &[]).is_empty());
}
}
+570
View File
@@ -0,0 +1,570 @@
//! A small CPU interpreter for the convolutional networks LightCraft bundles (the YuNet face detector).
//!
//! It runs checked convolution, affine/activation, broadcast, pooling, matrix and shape operators on
//! float32 tensors. Unsupported semantics fail at load time. Dense convolutions use bounded im2col and
//! single-thread GEMM; depthwise convolutions use row-wise Rust loops. Every file-derived shape is checked.
//!
//! The same checked float32 engine runs YuNet and installed SFace/AuraFace recognisers.
//! Unknown operators, attributes and shapes are rejected before recognition starts.
use std::collections::HashMap;
use std::rc::Rc;
mod kernels;
mod shape;
use crate::graph::{Data, Graph, Node};
/// Elements of any one tensor the interpreter will allocate.
const MAX_TENSOR: usize = 64 * 1024 * 1024;
#[derive(Debug, thiserror::Error, PartialEq)]
pub enum NetError {
#[error("this model uses something the built-in runtime does not support: {0}")]
Unsupported(String),
#[error("the model does not fit together: {0}")]
Shape(String),
}
type Result<T> = std::result::Result<T, NetError>;
fn shape_err<T>(why: impl Into<String>) -> Result<T> {
Err(NetError::Shape(why.into()))
}
fn unsupported<T>(why: impl Into<String>) -> Result<T> {
Err(NetError::Unsupported(why.into()))
}
#[derive(Clone, Debug, PartialEq)]
pub struct Tensor {
pub shape: Vec<usize>,
pub data: Vec<f32>,
}
impl Tensor {
pub fn new(shape: Vec<usize>, data: Vec<f32>) -> Result<Tensor> {
let n = shape.iter().try_fold(1usize, |a, d| a.checked_mul(*d));
if shape.len() > 8 || shape.contains(&0) || n.is_none_or(|n| n > MAX_TENSOR) || n != Some(data.len()) || data.iter().any(|v| !v.is_finite()) {
return shape_err(format!("{} values do not fill the shape {shape:?}", data.len()));
}
Ok(Tensor { shape, data })
}
fn zeros(shape: Vec<usize>) -> Result<Tensor> {
match shape.iter().try_fold(1usize, |a, d| a.checked_mul(*d)) {
Some(n) if n > 0 && n <= MAX_TENSOR && shape.len() <= 8 && !shape.contains(&0) => {
let mut data = Vec::new();
data.try_reserve_exact(n).map_err(|_| NetError::Shape("not enough memory for a model tensor".into()))?;
data.resize(n, 0.0);
Ok(Tensor { shape, data })
}
_ => shape_err("a tensor would be too large"),
}
}
}
/// A loaded network, ready to run.
pub struct Net {
graph: Graph,
}
/// Validate semantics rather than silently ignoring unknown or mistyped attributes.
fn check(node: &Node, g: &Graph) -> Result<()> {
shape::check(node, g)
}
impl Net {
pub fn new(graph: Graph) -> Result<Net> {
shape::check_graph(&graph)?;
for n in &graph.nodes {
check(n, &graph)?;
}
Ok(Net { graph })
}
/// The shape the model declares for its input (0 for a dynamic size).
pub fn input_shape(&self) -> Option<&[usize]> {
self.graph.inputs.first().map(|(_, s)| s.as_slice())
}
/// Infer and check every intermediate without allocating its pixel data.
/// Recognition calls this at load time with the manifest's fixed input.
pub fn validate_input(&self, shape: &[usize]) -> Result<Vec<Vec<usize>>> {
shape::infer(&self.graph, shape)
}
/// Run the network on `input` (its single input); returns each output by name.
pub fn run(&self, input: Tensor) -> Result<Vec<(String, Tensor)>> {
shape::tensor(&input.shape, input.data.len(), &input.data)?;
self.validate_input(&input.shape)?;
let mut scratch = kernels::Scratch::default();
let profile = std::env::var_os("LIGHTCRAFT_PROFILE").is_some();
let g = &self.graph;
let Some((input_name, _)) = g.inputs.first() else { return shape_err("the model has no input") };
// when each value is last read, so big intermediates are dropped as soon as nothing needs them
let mut last_use: HashMap<&str, usize> = HashMap::new();
for (i, n) in g.nodes.iter().enumerate() {
for name in &n.inputs {
last_use.insert(name.as_str(), i);
}
}
for o in &g.outputs {
last_use.insert(o.as_str(), usize::MAX);
}
let mut values: HashMap<String, Rc<Tensor>> = HashMap::new();
values.insert(input_name.clone(), Rc::new(input));
for (i, node) in g.nodes.iter().enumerate() {
let get = |idx: usize| -> Result<Rc<Tensor>> {
let name = node.inputs.get(idx).ok_or_else(|| NetError::Shape(format!("{} is missing an input", node.op)))?;
if let Some(value) = values.get(name) {
return Ok(value.clone());
}
if let Some(crate::graph::Weight { dims, data: Data::F32(data) }) = g.weights.get(name) {
return Ok(Rc::new(Tensor::new(dims.clone(), data.clone())?));
}
values.get(name).cloned().ok_or_else(|| NetError::Shape(format!("`{name}` is read before it is computed")))
};
let started = web_time::Instant::now();
let out = match node.op.as_str() {
"Conv" => kernels::conv(node, &*get(0)?, g, &mut scratch)?,
"Relu" => map(&*get(0)?, |v| v.max(0.0)),
"Sigmoid" => map(&*get(0)?, |v| 1.0 / (1.0 + (-v).exp())),
"Add" | "Sub" | "Mul" => kernels::binary(node.op.as_str(), &*get(0)?, &*get(1)?)?,
"PRelu" => kernels::prelu(node, &*get(0)?, g)?,
"BatchNormalization" => kernels::batch_norm(node, &*get(0)?, g)?,
"Gemm" => kernels::gemm(node, &*get(0)?, g)?,
"Flatten" => kernels::flatten(node, &*get(0)?)?,
"Dropout" | "Identity" => (*get(0)?).clone(),
"GlobalAveragePool" => kernels::global_average(&*get(0)?)?,
"MaxPool" => max_pool(node, &*get(0)?)?,
"Resize" => resize(node, &*get(0)?, g)?,
"Transpose" => transpose(node, &*get(0)?)?,
"Reshape" => reshape(node, &*get(0)?, g)?,
other => return unsupported(format!("the operator `{other}`")),
};
if out.data.iter().any(|v| !v.is_finite()) {
return shape_err(format!("{} produced non-finite values", node.op));
}
if profile {
eprintln!("[faces-cpu] layer {i} {} {:?}: {:.3} ms", node.op, out.shape, started.elapsed().as_secs_f64() * 1000.0);
}
let Some(name) = node.outputs.first() else { return shape_err("a node has no output") };
values.insert(name.clone(), Rc::new(out));
for name in &node.inputs {
if last_use.get(name.as_str()) == Some(&i) {
values.remove(name);
}
}
}
g.outputs
.iter()
.map(|name| {
let t = values.remove(name).ok_or_else(|| NetError::Shape(format!("output `{name}` was never computed")))?;
let t = Rc::try_unwrap(t).map_err(|_| NetError::Shape(format!("output `{name}` is still shared")))?;
Ok((name.clone(), t))
})
.collect()
}
}
fn map(x: &Tensor, f: impl Fn(f32) -> f32) -> Tensor {
Tensor { shape: x.shape.clone(), data: x.data.iter().map(|v| f(*v)).collect() }
}
/// `[1, c, h, w]` → `(c, h, w)`.
fn chw(x: &Tensor) -> Result<(usize, usize, usize)> {
match x.shape.as_slice() {
[1, c, h, w] => Ok((*c, *h, *w)),
s => unsupported(format!("a tensor of shape {s:?} (only [1, channels, height, width])")),
}
}
/// `pads` of an ONNX node as (top, left, bottom, right).
fn pads(node: &Node) -> Result<(usize, usize, usize, usize)> {
match node.ints("pads") {
None => Ok((0, 0, 0, 0)),
Some([t, l, b, r]) => {
let u = |v: &i64| usize::try_from(*v).map_err(|_| NetError::Shape("a negative pad".into()));
Ok((u(t)?, u(l)?, u(b)?, u(r)?))
}
Some(_) => unsupported("pads that are not [top, left, bottom, right]"),
}
}
fn pair(node: &Node, name: &str, default: usize) -> Result<(usize, usize)> {
match node.ints(name) {
None => Ok((default, default)),
Some([a, b]) => match (usize::try_from(*a), usize::try_from(*b)) {
(Ok(a), Ok(b)) if a >= 1 && b >= 1 => Ok((a, b)),
_ => shape_err(format!("`{name}` must be positive")),
},
Some(_) => unsupported(format!("`{name}` that is not two numbers")),
}
}
fn conv_direct(node: &Node, x: &Tensor, g: &Graph) -> Result<Tensor> {
let (cin, h, w) = chw(x)?;
let weight = node.inputs.get(1).and_then(|n| g.weights.get(n)).ok_or_else(|| NetError::Shape("a Conv lost its weights".into()))?;
let Data::F32(wd) = &weight.data else { return unsupported("non-float Conv weights") };
let [cout, cin_g, kh, kw] = weight.dims.as_slice() else { return unsupported("a Conv that is not 2-D") };
let (cout, cin_g, kh, kw) = (*cout, *cin_g, *kh, *kw);
let groups =
usize::try_from(node.int("group").unwrap_or(1)).ok().filter(|g| *g >= 1).ok_or_else(|| NetError::Shape("a bad Conv group".into()))?;
if groups.checked_mul(cin_g) != Some(cin) || cout % groups != 0 || kh == 0 || kw == 0 {
return shape_err("the Conv weights do not match its input");
}
let bias: Option<&[f32]> = match node.inputs.get(2).filter(|b| !b.is_empty()).and_then(|b| g.weights.get(b)) {
Some(b) => match &b.data {
Data::F32(v) if v.len() == cout => Some(v),
_ => return shape_err("the Conv bias does not match"),
},
None => None,
};
let (sh, sw) = pair(node, "strides", 1)?;
let (pt, pl, pb, pr) = pads(node)?;
let ph = h.checked_add(pt).and_then(|v| v.checked_add(pb)).ok_or_else(|| NetError::Shape("Conv padding overflow".into()))?;
let pw = w.checked_add(pl).and_then(|v| v.checked_add(pr)).ok_or_else(|| NetError::Shape("Conv padding overflow".into()))?;
if ph < kh || pw < kw {
return shape_err("the Conv kernel is larger than its input");
}
let (oh, ow) = ((ph - kh) / sh + 1, (pw - kw) / sw + 1);
let mut out = Tensor::zeros(vec![1, cout, oh, ow])?;
let plane = oh * ow;
let in_planes: Vec<&[f32]> = x.data.chunks_exact((h * w).max(1)).collect();
let cout_g = cout / groups;
for (co, out_plane) in out.data.chunks_exact_mut(plane.max(1)).enumerate() {
if let Some(b) = bias.and_then(|b| b.get(co)) {
out_plane.iter_mut().for_each(|o| *o = *b);
}
let group = co / cout_g;
for ci_l in 0..cin_g {
let Some(in_plane) = in_planes.get(group * cin_g + ci_l) else { return shape_err("a Conv input channel is missing") };
let base = (co * cin_g + ci_l) * kh * kw;
let Some(kernel) = wd.get(base..base + kh * kw) else { return shape_err("the Conv weights are too short") };
for (k, wv) in kernel.iter().enumerate() {
if *wv == 0.0 {
continue;
}
let (ky, kx) = (k / kw, k % kw);
for oy in 0..oh {
// input row for this output row and kernel row (in padded coordinates minus the pad)
let Some(iy) = (oy * sh + ky).checked_sub(pt).filter(|iy| *iy < h) else { continue };
let in_row = in_plane.get(iy * w..(iy + 1) * w).unwrap_or(&[]);
let out_row = out_plane.get_mut(oy * ow..(oy + 1) * ow).unwrap_or(&mut []);
if sw == 1 {
// ox runs over outputs whose input column ox + kx - pl lies inside the row
let lo = pl.saturating_sub(kx);
let hi = ow.min((w + pl).saturating_sub(kx));
if lo < hi {
let src = in_row.get(lo + kx - pl..hi + kx - pl).unwrap_or(&[]);
let dst = out_row.get_mut(lo..hi).unwrap_or(&mut []);
dst.iter_mut().zip(src).for_each(|(o, i)| *o += wv * i);
}
} else {
for (ox, o) in out_row.iter_mut().enumerate() {
if let Some(ix) = (ox * sw + kx).checked_sub(pl).filter(|ix| *ix < w)
&& let Some(i) = in_row.get(ix)
{
*o += wv * i;
}
}
}
}
}
}
}
Ok(out)
}
fn max_pool(node: &Node, x: &Tensor) -> Result<Tensor> {
let (c, h, w) = chw(x)?;
let (kh, kw) = pair(node, "kernel_shape", 1)?;
let (sh, sw) = pair(node, "strides", 1)?;
let (pt, pl, pb, pr) = pads(node)?;
if (pt, pl, pb, pr) != (0, 0, 0, 0) {
return unsupported("a padded MaxPool");
}
if h < kh || w < kw {
return shape_err("the MaxPool window is larger than its input");
}
let (oh, ow) = ((h - kh) / sh + 1, (w - kw) / sw + 1);
let mut out = Tensor::zeros(vec![1, c, oh, ow])?;
for (plane_in, plane_out) in x.data.chunks_exact((h * w).max(1)).zip(out.data.chunks_exact_mut((oh * ow).max(1))) {
for oy in 0..oh {
for ox in 0..ow {
let mut m = f32::NEG_INFINITY;
for ky in 0..kh {
let row = plane_in.get((oy * sh + ky) * w..(oy * sh + ky + 1) * w).unwrap_or(&[]);
for kx in 0..kw {
if let Some(v) = row.get(ox * sw + kx) {
m = m.max(*v);
}
}
}
if let Some(o) = plane_out.get_mut(oy * ow + ox) {
*o = m;
}
}
}
}
Ok(out)
}
/// Nearest-neighbour upscale by whole numbers (YuNet's feature pyramid doubles each map).
fn resize(node: &Node, x: &Tensor, g: &Graph) -> Result<Tensor> {
let (c, h, w) = chw(x)?;
let scales = match node.inputs.get(2).and_then(|n| g.weights.get(n)).map(|w| &w.data) {
Some(Data::F32(s)) if s.len() == 4 => s,
_ => return unsupported("a Resize whose scales are not four floats"),
};
let whole = |v: f32| ((1.0..=16.0).contains(&v) && v.fract() == 0.0).then_some(v as usize);
let (Some(sh), Some(sw)) = (whole(scales.get(2).copied().unwrap_or(0.0)), whole(scales.get(3).copied().unwrap_or(0.0))) else {
return unsupported("a Resize by a scale that is not a whole number from 1 to 16");
};
if scales.first() != Some(&1.0) || scales.get(1) != Some(&1.0) {
return unsupported("a Resize that scales channels");
}
let (oh, ow) = (h * sh, w * sw);
let mut out = Tensor::zeros(vec![1, c, oh, ow])?;
for (plane_in, plane_out) in x.data.chunks_exact((h * w).max(1)).zip(out.data.chunks_exact_mut((oh * ow).max(1))) {
for (oy, row_out) in plane_out.chunks_exact_mut(ow.max(1)).enumerate() {
let row_in = plane_in.get((oy / sh) * w..(oy / sh + 1) * w).unwrap_or(&[]);
for (ox, o) in row_out.iter_mut().enumerate() {
*o = row_in.get(ox / sw).copied().unwrap_or(0.0);
}
}
}
Ok(out)
}
fn transpose(node: &Node, x: &Tensor) -> Result<Tensor> {
let default_perm: Vec<i64> = (0..x.shape.len() as i64).rev().collect();
let perm: Vec<usize> = node
.ints("perm")
.unwrap_or(&default_perm)
.iter()
.map(|p| usize::try_from(*p).map_err(|_| NetError::Shape("a negative permutation".into())))
.collect::<Result<_>>()?;
let rank = x.shape.len();
let mut seen = vec![false; rank];
if perm.len() != rank || perm.iter().any(|p| *p >= rank || std::mem::replace(seen.get_mut(*p).unwrap_or(&mut true), true)) {
return shape_err("the Transpose permutation does not match the tensor");
}
let out_shape: Vec<usize> = perm.iter().filter_map(|p| x.shape.get(*p).copied()).collect();
// strides of the input, in elements
let mut in_strides = vec![1usize; rank];
for i in (0..rank.saturating_sub(1)).rev() {
let v = in_strides.get(i + 1).copied().unwrap_or(1) * x.shape.get(i + 1).copied().unwrap_or(1);
*in_strides.get_mut(i).ok_or_else(|| NetError::Shape("missing transpose stride".into()))? = v;
}
let strides_for_out: Vec<usize> = perm.iter().filter_map(|p| in_strides.get(*p).copied()).collect();
let mut out = Tensor::zeros(out_shape.clone())?;
let mut counter = vec![0usize; rank];
let mut src = 0usize;
for o in out.data.iter_mut() {
*o = x.data.get(src).copied().unwrap_or(0.0);
// advance the multi-index of the output and the matching input offset
for d in (0..rank).rev() {
let (Some(c), Some(len), Some(stride)) = (counter.get_mut(d), out_shape.get(d), strides_for_out.get(d)) else { break };
*c += 1;
src += stride;
if *c < *len {
break;
}
src -= stride * *len;
*c = 0;
}
}
Ok(out)
}
fn reshape(node: &Node, x: &Tensor, g: &Graph) -> Result<Tensor> {
let Some(Data::I64(dims)) = node.inputs.get(1).and_then(|n| g.weights.get(n)).map(|w| &w.data) else {
return unsupported("a Reshape with a computed shape");
};
let total = x.data.len();
let mut shape: Vec<usize> = Vec::with_capacity(dims.len());
let mut infer = None;
for (i, d) in dims.iter().enumerate() {
match *d {
-1 if infer.is_none() => {
infer = Some(i);
shape.push(1);
}
0 => shape.push(x.shape.get(i).copied().ok_or_else(|| NetError::Shape("a Reshape copies a missing dimension".into()))?),
d if d > 0 => shape.push(usize::try_from(d).map_err(|_| NetError::Shape("a Reshape dimension is too large".into()))?),
_ => return shape_err("a bad Reshape dimension"),
}
}
let known = shape.iter().try_fold(1usize, |a, d| a.checked_mul(*d)).filter(|k| *k > 0);
if let (Some(i), Some(known)) = (infer, known) {
if !total.is_multiple_of(known) {
return shape_err("the Reshape does not divide evenly");
}
if let Some(s) = shape.get_mut(i) {
*s = total / known;
}
}
Tensor::new(shape, x.data.clone())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::graph::{Attr, Weight};
fn node(op: &str, inputs: &[&str], attrs: &[(&str, Attr)]) -> Node {
Node {
op: op.into(),
inputs: inputs.iter().map(|s| s.to_string()).collect(),
outputs: vec!["y".into()],
attrs: attrs.iter().map(|(k, v)| (k.to_string(), v.clone())).collect(),
}
}
fn graph(nodes: Vec<Node>, weights: Vec<(&str, Weight)>) -> Graph {
Graph {
nodes,
weights: weights.into_iter().map(|(k, v)| (k.to_string(), v)).collect(),
inputs: vec![("x".into(), vec![1, 1, 3, 3])],
outputs: vec!["y".into()],
}
}
fn w(dims: &[usize], v: Vec<f32>) -> Weight {
Weight { dims: dims.to_vec(), data: Data::F32(v) }
}
fn input(c: usize, h: usize, wd: usize, f: impl Fn(usize) -> f32) -> Tensor {
Tensor::new(vec![1, c, h, wd], (0..c * h * wd).map(f).collect()).unwrap()
}
fn run(g: Graph, x: Tensor) -> Result<Tensor> {
Ok(Net::new(g)?.run(x)?.remove(0).1)
}
/// A direct, slow reference convolution to check the fast one against.
fn reference_conv(x: &Tensor, wd: &[f32], dims: [usize; 4], bias: &[f32], s: usize, p: usize, groups: usize) -> Tensor {
let (c, h, wi) = (x.shape[1], x.shape[2], x.shape[3]);
let [co_n, cin_g, kh, kw] = dims;
let (oh, ow) = ((h + 2 * p - kh) / s + 1, (wi + 2 * p - kw) / s + 1);
let mut out = vec![0.0f32; co_n * oh * ow];
for co in 0..co_n {
let g = co / (co_n / groups);
for oy in 0..oh {
for ox in 0..ow {
let mut acc = bias[co];
for cl in 0..cin_g {
for ky in 0..kh {
for kx in 0..kw {
let (iy, ix) = ((oy * s + ky) as isize - p as isize, (ox * s + kx) as isize - p as isize);
if iy >= 0 && ix >= 0 && (iy as usize) < h && (ix as usize) < wi {
acc +=
x.data[((g * cin_g + cl) * h + iy as usize) * wi + ix as usize] * wd[((co * cin_g + cl) * kh + ky) * kw + kx];
}
}
}
}
out[(co * oh + oy) * ow + ox] = acc;
}
}
}
let _ = c;
Tensor::new(vec![1, co_n, oh, ow], out).unwrap()
}
#[test]
fn convolutions_match_a_slow_reference() {
// (cin, cout, groups, kernel, stride, pad, h, w)
for (cin, cout, groups, k, s, p, h, wi) in [
(3, 4, 1, 3, 1, 1, 7, 9),
(3, 4, 1, 3, 2, 1, 8, 8),
(4, 4, 4, 3, 1, 1, 6, 5),
(2, 6, 1, 1, 1, 0, 5, 5),
(4, 8, 2, 3, 1, 1, 5, 6),
(1, 1, 1, 3, 1, 0, 3, 3),
] {
let cin_g = cin / groups;
let wd: Vec<f32> = (0..cout * cin_g * k * k).map(|i| ((i * 37 % 11) as f32 - 5.0) * 0.1).collect();
let bias: Vec<f32> = (0..cout).map(|i| i as f32 * 0.25 - 0.5).collect();
let x = input(cin, h, wi, |i| ((i * 13 % 17) as f32 - 8.0) * 0.2);
let attrs = [("group", Attr::Int(groups as i64)), ("strides", Attr::Ints(vec![s as i64; 2])), ("pads", Attr::Ints(vec![p as i64; 4]))];
let g = graph(
vec![node("Conv", &["x", "w", "b"], &attrs)],
vec![("w", w(&[cout, cin_g, k, k], wd.clone())), ("b", w(&[cout], bias.clone()))],
);
let got = run(g, x.clone()).unwrap();
let want = reference_conv(&x, &wd, [cout, cin_g, k, k], &bias, s, p, groups);
assert_eq!(got.shape, want.shape, "{cin} {cout} {groups} {k} {s} {p}");
for (a, b) in got.data.iter().zip(&want.data) {
assert!((a - b).abs() < 1e-4, "{cin} {cout} {groups} {k} {s} {p}: {a} vs {b}");
}
}
}
#[test]
fn the_small_operators() {
let x = input(1, 4, 4, |i| i as f32 - 6.0);
// Relu, Sigmoid
let r = run(graph(vec![node("Relu", &["x"], &[])], vec![]), x.clone()).unwrap();
assert!(r.data.iter().all(|v| *v >= 0.0) && r.data.get(7) == Some(&1.0));
let s = run(graph(vec![node("Sigmoid", &["x"], &[])], vec![]), x.clone()).unwrap();
assert!((s.data[6] - 0.5).abs() < 1e-6 && s.data.iter().all(|v| (0.0..=1.0).contains(v)));
// MaxPool 2x2/2
let m = run(
graph(vec![node("MaxPool", &["x"], &[("kernel_shape", Attr::Ints(vec![2, 2])), ("strides", Attr::Ints(vec![2, 2]))])], vec![]),
x.clone(),
)
.unwrap();
assert_eq!((m.shape.clone(), m.data.clone()), (vec![1, 1, 2, 2], vec![-1.0, 1.0, 7.0, 9.0]));
// Resize x2 nearest
let z = run(graph(vec![node("Resize", &["x", "", "s"], &[])], vec![("s", w(&[4], vec![1.0, 1.0, 2.0, 2.0]))]), input(1, 2, 2, |i| i as f32))
.unwrap();
assert_eq!(z.shape, vec![1, 1, 4, 4]);
assert_eq!(&z.data[..8], &[0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 1.0, 1.0]);
// Add
let a = run(graph(vec![node("Add", &["x", "x"], &[])], vec![]), x.clone()).unwrap();
assert_eq!(a.data[5], 2.0 * x.data[5]);
// Transpose [0,2,3,1] then Reshape [1,-1,2]
let t = run(graph(vec![node("Transpose", &["x"], &[("perm", Attr::Ints(vec![0, 2, 3, 1]))])], vec![]), input(2, 2, 3, |i| i as f32)).unwrap();
assert_eq!(t.shape, vec![1, 2, 3, 2]);
assert_eq!(t.data, vec![0.0, 6.0, 1.0, 7.0, 2.0, 8.0, 3.0, 9.0, 4.0, 10.0, 5.0, 11.0]);
let rs = run(
graph(vec![node("Reshape", &["x", "s"], &[])], vec![("s", Weight { dims: vec![3], data: Data::I64(vec![1, -1, 2]) })]),
input(2, 2, 3, |i| i as f32),
)
.unwrap();
assert_eq!(rs.shape, vec![1, 6, 2]);
}
#[test]
fn unsupported_and_inconsistent_models_are_errors() {
let x = input(1, 3, 3, |i| i as f32);
for (what, g) in [
("unknown op", graph(vec![node("Softmax", &["x"], &[])], vec![])),
(
"dilated conv",
graph(vec![node("Conv", &["x", "w"], &[("dilations", Attr::Ints(vec![2, 2]))])], vec![("w", w(&[1, 1, 1, 1], vec![1.0]))]),
),
("conv without weights", graph(vec![node("Conv", &["x", "w"], &[])], vec![])),
("channels do not match", graph(vec![node("Conv", &["x", "w"], &[])], vec![("w", w(&[1, 5, 1, 1], vec![1.0; 5]))])),
("kernel too big", graph(vec![node("Conv", &["x", "w"], &[])], vec![("w", w(&[1, 1, 5, 5], vec![1.0; 25]))])),
("zero group", graph(vec![node("Conv", &["x", "w"], &[("group", Attr::Int(0))])], vec![("w", w(&[1, 1, 1, 1], vec![1.0]))])),
("zero stride", graph(vec![node("Conv", &["x", "w"], &[("strides", Attr::Ints(vec![0, 0]))])], vec![("w", w(&[1, 1, 1, 1], vec![1.0]))])),
(
"negative pad",
graph(vec![node("Conv", &["x", "w"], &[("pads", Attr::Ints(vec![-1, 0, 0, 0]))])], vec![("w", w(&[1, 1, 1, 1], vec![1.0]))]),
),
("bad perm", graph(vec![node("Transpose", &["x"], &[("perm", Attr::Ints(vec![0, 0, 1, 2]))])], vec![])),
(
"huge reshape",
graph(vec![node("Reshape", &["x", "s"], &[])], vec![("s", Weight { dims: vec![2], data: Data::I64(vec![i64::MAX, 1]) })]),
),
("reshape mismatch", graph(vec![node("Reshape", &["x", "s"], &[])], vec![("s", Weight { dims: vec![2], data: Data::I64(vec![4, 4]) })])),
("non-integer resize", graph(vec![node("Resize", &["x", "", "s"], &[])], vec![("s", w(&[4], vec![1.0, 1.0, 1.5, 1.5]))])),
("add that broadcasts", graph(vec![node("Add", &["x", "c"], &[])], vec![])),
] {
assert!(run(g, x.clone()).is_err(), "{what}");
}
}
}
#[cfg(test)]
mod recognition_tests;
+306
View File
@@ -0,0 +1,306 @@
//! Safe CPU kernels shared by detection and recognition. No inner thread pool.
use super::*;
use ndarray::{ArrayView2, ArrayViewMut2, linalg::general_mat_mul};
const BLOCK: usize = 256;
#[derive(Default)]
pub(super) struct Scratch {
lower: Vec<f32>,
product: Vec<f32>,
}
fn storage(v: &mut Vec<f32>, n: usize) -> Result<&mut [f32]> {
if n > MAX_TENSOR {
return shape_err("convolution scratch exceeds the limit");
}
if v.len() < n {
v.try_reserve_exact(n - v.len()).map_err(|_| NetError::Shape("not enough memory for convolution scratch".into()))?;
v.resize(n, 0.0);
}
v.get_mut(..n).ok_or_else(|| NetError::Shape("missing convolution scratch".into()))
}
/// The safe matrix views assert lengths, so validate exact slices before creating any view.
fn multiply(w: &[f32], x: &[f32], y: &mut [f32], m: usize, k: usize, n: usize) -> Result<()> {
if m == 0 || k == 0 || n == 0 || m.checked_mul(k) != Some(w.len()) || k.checked_mul(n) != Some(x.len()) || m.checked_mul(n) != Some(y.len()) {
return shape_err("invalid GEMM dimensions");
}
let a = ArrayView2::from_shape((m, k), w).map_err(|e| NetError::Shape(e.to_string()))?;
let b = ArrayView2::from_shape((k, n), x).map_err(|e| NetError::Shape(e.to_string()))?;
let mut c = ArrayViewMut2::from_shape((m, n), y).map_err(|e| NetError::Shape(e.to_string()))?;
general_mat_mul(1.0, &a, &b, 0.0, &mut c);
Ok(())
}
/// For equal-width, unit-stride maps a shifted plane is one contiguous copy. Only horizontal
/// border stripes need masking; clipping the flat source range also removes vertical padding.
fn shifted_plane(plane: &[f32], width: usize, ky: usize, kx: usize, pt: usize, pl: usize, start: usize, row: &mut [f32]) -> Result<()> {
let end = start + row.len();
let delta = (ky as i128 - pt as i128) * width as i128 + kx as i128 - pl as i128;
let lo = (-delta).clamp(start as i128, end as i128);
let hi = (plane.len() as i128 - delta).clamp(start as i128, end as i128);
row.fill(0.0);
if lo < hi {
let from = usize::try_from(lo + delta).map_err(|_| NetError::Shape("shifted source overflow".into()))?;
let at = usize::try_from(lo - start as i128).map_err(|_| NetError::Shape("shifted destination overflow".into()))?;
let len = usize::try_from(hi - lo).map_err(|_| NetError::Shape("shifted length overflow".into()))?;
row.get_mut(at..at + len)
.ok_or_else(|| NetError::Shape("short shifted destination".into()))?
.copy_from_slice(plane.get(from..from + len).ok_or_else(|| NetError::Shape("short shifted source".into()))?);
}
let left = pl.saturating_sub(kx).min(width);
let right = (width + pl).saturating_sub(kx).min(width);
if left > 0 || right < width {
for y in start / width..=(end - 1) / width {
for (a, b) in [(y * width, y * width + left), (y * width + right, (y + 1) * width)] {
let lo = a.max(start);
let hi = b.min(end);
if lo < hi {
row.get_mut(lo - start..hi - start).ok_or_else(|| NetError::Shape("short border stripe".into()))?.fill(0.0);
}
}
}
}
Ok(())
}
pub(super) fn conv(n: &Node, x: &Tensor, g: &Graph, scratch: &mut Scratch) -> Result<Tensor> {
let (cin, h, w) = chw(x)?;
let wt = shape::weight(n, g, 1)?;
let wd = shape::floats(n, g, 1)?;
let [cout, cg, kh, kw] = wt.dims.as_slice() else {
return shape_err("invalid convolution weights");
};
let (cout, cg, kh, kw) = (*cout, *cg, *kh, *kw);
let groups = usize::try_from(n.int("group").unwrap_or(1)).map_err(|_| NetError::Shape("invalid group".into()))?;
if cg == 1 {
return conv_direct(n, x, g);
}
let (sh, sw) = pair(n, "strides", 1)?;
let (pt, pl, pb, pr) = pads(n)?;
let ph = h.checked_add(pt).and_then(|v| v.checked_add(pb)).ok_or_else(|| NetError::Shape("padding overflow".into()))?;
let pw = w.checked_add(pl).and_then(|v| v.checked_add(pr)).ok_or_else(|| NetError::Shape("padding overflow".into()))?;
if ph < kh || pw < kw || groups == 0 || cin != groups * cg {
return shape_err("invalid convolution shape");
}
let (oh, ow) = ((ph - kh) / sh + 1, (pw - kw) / sw + 1);
let mut out = Tensor::zeros(vec![1, cout, oh, ow])?;
let pixels = oh * ow;
let k = cg * kh * kw;
let m = cout / groups;
let bias = if n.inputs.get(2).is_some_and(|s| !s.is_empty()) { Some(shape::floats(n, g, 2)?) } else { None };
for group in 0..groups {
let weights = wd.get(group * m * k..(group + 1) * m * k).ok_or_else(|| NetError::Shape("short convolution weights".into()))?;
// A pointwise convolution is already a matrix: no lowering, copy or spatial blocking needed.
if kh == 1 && kw == 1 && sh == 1 && sw == 1 && (pt, pl, pb, pr) == (0, 0, 0, 0) {
let input = x.data.get(group * cg * pixels..(group + 1) * cg * pixels).ok_or_else(|| NetError::Shape("short pointwise input".into()))?;
let output =
out.data.get_mut(group * m * pixels..(group + 1) * m * pixels).ok_or_else(|| NetError::Shape("short pointwise output".into()))?;
multiply(weights, input, output, m, cg, pixels)?;
for (co, plane) in output.chunks_exact_mut(pixels).enumerate() {
let b = bias.and_then(|v| v.get(group * m + co)).copied().unwrap_or(0.0);
if b != 0.0 {
for v in plane {
*v += b;
}
}
}
continue;
}
for start in (0..pixels).step_by(BLOCK) {
let len = BLOCK.min(pixels - start);
let lower = storage(&mut scratch.lower, k * len)?;
for cl in 0..cg {
let plane = x
.data
.get((group * cg + cl) * h * w..(group * cg + cl + 1) * h * w)
.ok_or_else(|| NetError::Shape("short convolution input".into()))?;
for ky in 0..kh {
for kx in 0..kw {
let row = lower
.get_mut((cl * kh * kw + ky * kw + kx) * len..(cl * kh * kw + ky * kw + kx + 1) * len)
.ok_or_else(|| NetError::Shape("short im2col row".into()))?;
if sh == 1 && sw == 1 && ow == w {
shifted_plane(
plane,
w,
ky,
kx,
pt,
pl,
start,
row.get_mut(..len).ok_or_else(|| NetError::Shape("short lower panel".into()))?,
)?;
continue;
}
let mut at = 0;
while at < len {
let p = start + at;
let (oy, ox) = (p / ow, p % ow);
let span = (ow - ox).min(len - at);
if let Some(yy) = (oy * sh + ky).checked_sub(pt).filter(|&v| v < h) {
let lo = ox.max(pl.saturating_sub(kx).div_ceil(sw));
let hi = (ox + span).min((w + pl).saturating_sub(kx).div_ceil(sw));
if lo < hi {
let from = yy * w + lo * sw + kx - pl;
let to = at + lo - ox;
row.get_mut(at..to).ok_or_else(|| NetError::Shape("short im2col padding".into()))?.fill(0.0);
let src = plane
.get(from..from + (hi - lo - 1) * sw + 1)
.ok_or_else(|| NetError::Shape("short im2col source row".into()))?;
let dst = row.get_mut(to..to + hi - lo).ok_or_else(|| NetError::Shape("short im2col row".into()))?;
if sw == 1 {
dst.copy_from_slice(src);
} else {
for (o, &v) in dst.iter_mut().zip(src.iter().step_by(sw)) {
*o = v;
}
}
row.get_mut(to + hi - lo..at + span).ok_or_else(|| NetError::Shape("short im2col padding".into()))?.fill(0.0);
} else {
row.get_mut(at..at + span).ok_or_else(|| NetError::Shape("short im2col row".into()))?.fill(0.0);
}
} else {
row.get_mut(at..at + span).ok_or_else(|| NetError::Shape("short im2col row".into()))?.fill(0.0);
}
at += span;
}
}
}
}
let product = storage(&mut scratch.product, m * len)?;
multiply(weights, lower, product, m, k, len)?;
for (co, row) in product.chunks_exact(len).enumerate() {
let channel = group * m + co;
let b = bias.and_then(|v| v.get(channel)).copied().unwrap_or(0.0);
let dest = out
.data
.get_mut(channel * pixels + start..channel * pixels + start + len)
.ok_or_else(|| NetError::Shape("short convolution output".into()))?;
for (to, &value) in dest.iter_mut().zip(row) {
*to = value + b;
}
}
}
}
Ok(out)
}
/// Flat offset under ONNX's trailing-dimension broadcasting; shapes are checked before execution.
fn broadcast_offset(mut at: usize, out: &[usize], input: &[usize]) -> usize {
let mut offset = 0;
let mut stride = 1;
for (axis, &size) in out.iter().enumerate().rev() {
let coordinate = at % size;
at /= size;
if let Some(&d) = axis.checked_sub(out.len() - input.len()).and_then(|i| input.get(i)) {
if d != 1 {
offset += coordinate * stride;
}
stride *= d;
}
}
offset
}
pub(super) fn binary(op: &str, a: &Tensor, b: &Tensor) -> Result<Tensor> {
let dims = shape::broadcast(&a.shape, &b.shape)?;
let mut out = Tensor::zeros(dims)?;
let f = |x: f32, y: f32| match op {
"Add" => x + y,
"Sub" => x - y,
_ => x * y,
};
if a.shape == b.shape {
for ((o, &x), &y) in out.data.iter_mut().zip(&a.data).zip(&b.data) {
*o = f(x, y);
}
} else if b.data.len() == 1 && a.shape == out.shape {
let y = *b.data.first().ok_or_else(|| NetError::Shape("empty scalar".into()))?;
for (o, &x) in out.data.iter_mut().zip(&a.data) {
*o = f(x, y);
}
} else {
for (i, o) in out.data.iter_mut().enumerate() {
let x = a.data.get(broadcast_offset(i, &out.shape, &a.shape)).ok_or_else(|| NetError::Shape("bad broadcast A".into()))?;
let y = b.data.get(broadcast_offset(i, &out.shape, &b.shape)).ok_or_else(|| NetError::Shape("bad broadcast B".into()))?;
*o = f(*x, *y);
}
}
Ok(out)
}
pub(super) fn prelu(n: &Node, x: &Tensor, g: &Graph) -> Result<Tensor> {
let wt = shape::weight(n, g, 1)?;
let slope = shape::floats(n, g, 1)?;
let mut out = Tensor::zeros(x.shape.clone())?;
let channels = x.shape.get(1).copied().unwrap_or(1);
let mut expanded = vec![1; x.shape.len().saturating_sub(wt.dims.len())];
expanded.extend_from_slice(&wt.dims);
let channel_slope = expanded.iter().enumerate().all(|(i, &d)| d == 1 || i == 1) && slope.len() == channels;
if channel_slope || slope.len() == 1 {
let plane = shape::elements(x.shape.get(2..).unwrap_or(&[]))?;
for (c, (src, dst)) in x.data.chunks_exact(plane).zip(out.data.chunks_exact_mut(plane)).enumerate() {
let alpha = *slope.get(if slope.len() == 1 { 0 } else { c % channels }).ok_or_else(|| NetError::Shape("short PRelu slope".into()))?;
for (to, &v) in dst.iter_mut().zip(src) {
*to = if v < 0.0 { v * alpha } else { v };
}
}
} else {
for (i, (o, &v)) in out.data.iter_mut().zip(&x.data).enumerate() {
let alpha = *slope.get(broadcast_offset(i, &x.shape, &wt.dims)).ok_or_else(|| NetError::Shape("bad PRelu broadcast".into()))?;
*o = if v < 0.0 { v * alpha } else { v };
}
}
Ok(out)
}
pub(super) fn batch_norm(n: &Node, x: &Tensor, g: &Graph) -> Result<Tensor> {
let channels = *x.shape.get(1).ok_or_else(|| NetError::Shape("BatchNormalization rank".into()))?;
let (scale, bias, mean, var) = (shape::floats(n, g, 1)?, shape::floats(n, g, 2)?, shape::floats(n, g, 3)?, shape::floats(n, g, 4)?);
let eps = shape::float(n, "epsilon", 1e-5);
let plane = shape::elements(x.shape.get(2..).unwrap_or(&[]))?;
let mut out = Tensor::zeros(x.shape.clone())?;
for (j, (src, dst)) in x.data.chunks_exact(plane).zip(out.data.chunks_exact_mut(plane)).enumerate() {
let c = j % channels;
let coefficient = *scale.get(c).ok_or_else(|| NetError::Shape("short BN scale".into()))?
/ (var.get(c).copied().ok_or_else(|| NetError::Shape("short BN variance".into()))? + eps).sqrt();
let offset = bias.get(c).copied().ok_or_else(|| NetError::Shape("short BN bias".into()))?
- mean.get(c).copied().ok_or_else(|| NetError::Shape("short BN mean".into()))? * coefficient;
for (to, &value) in dst.iter_mut().zip(src) {
*to = value * coefficient + offset;
}
}
Ok(out)
}
pub(super) fn gemm(n: &Node, a: &Tensor, g: &Graph) -> Result<Tensor> {
let wt = shape::weight(n, g, 1)?;
let b = shape::floats(n, g, 1)?;
let [ar, ac] = a.shape.as_slice() else {
return shape_err("Gemm A is not a matrix");
};
let [br, bc] = wt.dims.as_slice() else {
return shape_err("Gemm B is not a matrix");
};
if ar.checked_mul(*ac) != Some(a.data.len()) || br.checked_mul(*bc) != Some(b.len()) {
return shape_err("invalid matrix buffers");
}
let av = ArrayView2::from_shape((*ar, *ac), a.data.as_slice()).map_err(|e| NetError::Shape(e.to_string()))?;
let bv = ArrayView2::from_shape((*br, *bc), b).map_err(|e| NetError::Shape(e.to_string()))?;
let av = if n.int("transA").unwrap_or(0) != 0 { av.reversed_axes() } else { av };
let bv = if n.int("transB").unwrap_or(0) != 0 { bv.reversed_axes() } else { bv };
if av.ncols() != bv.nrows() {
return shape_err("incompatible matrices");
}
let mut out = Tensor::zeros(vec![av.nrows(), bv.ncols()])?;
let mut view = ArrayViewMut2::from_shape((av.nrows(), bv.ncols()), out.data.as_mut_slice()).map_err(|e| NetError::Shape(e.to_string()))?;
general_mat_mul(shape::float(n, "alpha", 1.0), &av, &bv, 0.0, &mut view);
if n.inputs.get(2).is_some_and(|s| !s.is_empty()) {
let wt = shape::weight(n, g, 2)?;
let bias = shape::floats(n, g, 2)?;
let beta = shape::float(n, "beta", 1.0);
for (i, to) in out.data.iter_mut().enumerate() {
*to += beta * bias.get(broadcast_offset(i, &out.shape, &wt.dims)).copied().ok_or_else(|| NetError::Shape("short matrix bias".into()))?;
}
}
Ok(out)
}
pub(super) fn flatten(n: &Node, x: &Tensor) -> Result<Tensor> {
let axis = shape::axis(n, x.shape.len())?;
Tensor::new(vec![shape::elements(x.shape.get(..axis).unwrap_or(&[]))?, shape::elements(x.shape.get(axis..).unwrap_or(&[]))?], x.data.clone())
}
pub(super) fn global_average(x: &Tensor) -> Result<Tensor> {
let (c, h, w) = chw(x)?;
Tensor::new(vec![1, c, 1, 1], x.data.chunks_exact(h * w).map(|v| v.iter().sum::<f32>() / (h * w) as f32).collect())
}
+337
View File
@@ -0,0 +1,337 @@
use super::*;
use crate::graph::{Attr, Weight};
fn weight(shape: &[usize], values: Vec<f32>) -> Weight {
Weight { dims: shape.to_vec(), data: Data::F32(values) }
}
fn graph(op: &str, extra: &[&str], weights: Vec<(&str, Weight)>, attrs: Vec<(&str, Attr)>) -> Graph {
Graph {
nodes: vec![Node {
op: op.into(),
inputs: std::iter::once("x").chain(extra.iter().copied()).map(str::to_string).collect(),
outputs: vec!["y".into()],
attrs: attrs.into_iter().map(|(s, a)| (s.into(), a)).collect(),
}],
weights: weights.into_iter().map(|(s, w)| (s.into(), w)).collect(),
inputs: vec![("x".into(), vec![1, 3, 2, 2])],
outputs: vec!["y".into()],
}
}
fn execute(g: Graph, shape: &[usize], values: Vec<f32>) -> Tensor {
Net::new(g).unwrap().run(Tensor::new(shape.to_vec(), values).unwrap()).unwrap().remove(0).1
}
#[test]
fn batch_norm_matches_the_inference_formula_for_images_and_matrices() {
for shape in [vec![1, 3, 2, 5], vec![2, 3]] {
let values: Vec<_> = (0..shape.iter().product()).map(|i| i as f32 * 0.07 - 0.8).collect();
let g = graph(
"BatchNormalization",
&["scale", "bias", "mean", "var"],
vec![
("scale", weight(&[3], vec![0.7, -1.1, 2.3])),
("bias", weight(&[3], vec![-0.4, 0.6, 0.2])),
("mean", weight(&[3], vec![0.5, -0.3, 0.9])),
("var", weight(&[3], vec![0.2, 0.0, 1.4])),
],
vec![("epsilon", Attr::Float(1e-3))],
);
let out = execute(g.clone(), &shape, values.clone());
let spatial: usize = shape.get(2..).unwrap_or(&[]).iter().product();
for (i, (&value, &got)) in values.iter().zip(&out.data).enumerate() {
let c = (i / spatial) % 3;
let [s, b, m, v] = ["scale", "bias", "mean", "var"].map(|name| {
let Data::F32(v) = &g.weights[name].data else { panic!() };
v[c]
});
let want = (value - m) * s / (v + 1e-3).sqrt() + b;
assert!((got - want).abs() < 1e-5);
}
}
}
#[test]
fn prelu_obeys_trailing_broadcasting_including_spatial_slopes() {
for (shape, slopes) in
[(vec![3, 1, 1], vec![0.1, 0.2, 0.3]), (vec![1, 3, 1, 1], vec![0.1, 0.2, 0.3]), (vec![1], vec![0.3]), (vec![2, 2], vec![0.1, 0.2, 0.3, 0.4])]
{
let values: Vec<_> = (0..12).map(|i| i as f32 - 6.0).collect();
let out = execute(graph("PRelu", &["s"], vec![("s", weight(&shape, slopes.clone()))], vec![]), &[1, 3, 2, 2], values.clone());
for (i, (&v, &got)) in values.iter().zip(&out.data).enumerate() {
let alpha = if slopes.len() == 1 {
slopes[0]
} else if slopes.len() == 3 {
slopes[i / 4]
} else {
slopes[i % 4]
};
assert_eq!(got, if v < 0.0 { v * alpha } else { v });
}
}
}
#[test]
fn gemm_transpositions_scalars_vectors_matrices_and_optional_bias() {
for ta in [0, 1] {
for tb in [0, 1] {
for bias_shape in [vec![], vec![2], vec![1, 2], vec![2, 1], vec![2, 2]] {
let (ash, bsh) = (if ta == 0 { vec![2, 3] } else { vec![3, 2] }, if tb == 0 { vec![3, 2] } else { vec![2, 3] });
let a: Vec<_> = (0..6).map(|i| i as f32 * 0.4 - 0.8).collect();
let b: Vec<_> = (0..6).map(|i| i as f32 * 0.1 - 0.3).collect();
let bias: Vec<_> = (0..bias_shape.iter().product()).map(|i| i as f32 * 0.25 + 0.3).collect();
let g = graph(
"Gemm",
&["w", "b"],
vec![("w", weight(&bsh, b.clone())), ("b", weight(&bias_shape, bias.clone()))],
vec![("transA", Attr::Int(ta)), ("transB", Attr::Int(tb)), ("alpha", Attr::Float(0.7)), ("beta", Attr::Float(-0.4))],
);
let out = execute(g, &ash, a.clone());
assert_eq!(out.shape, vec![2, 2]);
for i in 0..2 {
for j in 0..2 {
let sum = (0..3)
.map(|k| a[if ta == 0 { i * 3 + k } else { k * 2 + i }] * b[if tb == 0 { k * 2 + j } else { j * 3 + k }])
.sum::<f32>();
let off = match bias_shape.as_slice() {
[] => 0,
[2] | [1, 2] => j,
[2, 1] => i,
_ => i * 2 + j,
};
assert!((out.data[i * 2 + j] - (0.7 * sum - 0.4 * bias[off])).abs() < 1e-6);
}
}
}
}
}
let out = execute(graph("Gemm", &["w"], vec![("w", weight(&[2, 1], vec![2.0, 3.0]))], vec![]), &[1, 2], vec![4.0, 5.0]);
assert_eq!(out.data, vec![23.0]);
}
#[test]
fn arithmetic_broadcasts_and_keeps_operand_order() {
for op in ["Add", "Sub", "Mul"] {
let out =
execute(graph(op, &["w"], vec![("w", weight(&[3, 1], vec![0.5, 1.5, 2.5]))], vec![]), &[1, 3, 2], vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0]);
for (i, &got) in out.data.iter().enumerate() {
let x = (i + 1) as f32;
let y = (i / 2) as f32 + 0.5;
let want = match op {
"Add" => x + y,
"Sub" => x - y,
_ => x * y,
};
assert_eq!(got, want);
}
}
}
#[test]
fn flatten_negative_axis_and_default_transpose_are_correct() {
let out = execute(graph("Flatten", &[], vec![], vec![("axis", Attr::Int(-1))]), &[1, 3, 2, 5], (0..30).map(|i| i as f32).collect());
assert_eq!(out.shape, vec![6, 5]);
let out = execute(graph("Transpose", &[], vec![], vec![]), &[2, 3], (0..6).map(|i| i as f32).collect());
assert_eq!(out.shape, vec![3, 2]);
assert_eq!(out.data, vec![0.0, 3.0, 1.0, 4.0, 2.0, 5.0]);
}
#[test]
fn all_recognition_operators_run_on_seeded_synthetic_networks() {
let input = Tensor::new(vec![1, 3, 112, 112], (0..3 * 112 * 112).map(|i| (i * 37 % 256) as f32).collect()).unwrap();
for seed in 0..20 {
let net = Net::new(crate::graph::load(&crate::synthetic::recognition_model(seed)).unwrap()).unwrap();
let a = net.run(input.clone()).unwrap();
let b = net.run(input.clone()).unwrap();
assert_eq!(a, b);
assert_eq!(a[0].1.shape, vec![1, 8]);
}
}
#[test]
fn hostile_attributes_weights_shapes_and_public_tensors_return_errors() {
for (name, attr) in [
("strides", Attr::Ints(vec![0, 1])),
("pads", Attr::Ints(vec![i64::MAX; 4])),
("group", Attr::Int(0)),
("kernel_shape", Attr::Ints(vec![7, 7])),
("strides", Attr::Float(1.0)),
("new_attribute", Attr::Int(1)),
] {
let g = graph("Conv", &["w"], vec![("w", weight(&[3, 3, 1, 1], vec![0.1; 9]))], vec![(name, attr)]);
let result = Net::new(g).and_then(|n| n.validate_input(&[1, 3, 3, 3]));
assert!(result.is_err(), "{name}");
}
for op in ["Loop", "MatMul", "If", "Concat", "Unknown"] {
assert!(Net::new(graph(op, &[], vec![], vec![])).is_err());
}
for value in [f32::NAN, f32::INFINITY] {
assert!(Net::new(graph("PRelu", &["w"], vec![("w", weight(&[1], vec![value]))], vec![])).is_err());
}
let net = Net::new(graph("Relu", &[], vec![], vec![])).unwrap();
for input in [
Tensor { shape: vec![usize::MAX, 2], data: vec![] },
Tensor { shape: vec![0, 3], data: vec![] },
Tensor { shape: vec![1], data: vec![f32::NAN] },
Tensor { shape: vec![1; 9], data: vec![1.0] },
Tensor { shape: vec![2], data: vec![1.0] },
] {
assert!(net.run(input).is_err());
}
let mut g = graph("Relu", &[], vec![], vec![]);
g.nodes.push(g.nodes[0].clone());
assert!(Net::new(g).is_err());
let mut g = graph("Relu", &[], vec![], vec![]);
g.nodes[0].inputs[0] = "future".into();
assert!(Net::new(g).is_err());
}
#[test]
fn malformed_normalization_prelu_matrix_dropout_and_axes_are_errors() {
let bn = || {
graph(
"BatchNormalization",
&["s", "b", "m", "v"],
vec![
("s", weight(&[3], vec![1.0; 3])),
("b", weight(&[3], vec![0.0; 3])),
("m", weight(&[3], vec![0.0; 3])),
("v", weight(&[3], vec![1.0; 3])),
],
vec![],
)
};
for mutation in 0..4 {
let mut g = bn();
match mutation {
0 => {
g.nodes[0].attrs.insert("training_mode".into(), Attr::Int(1));
}
1 => {
g.nodes[0].attrs.insert("epsilon".into(), Attr::Float(0.0));
}
2 => {
g.weights.insert("v".into(), weight(&[3], vec![-1.0; 3]));
}
_ => {
g.weights.insert("b".into(), weight(&[1, 3], vec![0.0; 3]));
}
};
assert!(Net::new(g).and_then(|n| n.validate_input(&[1, 3, 2, 2])).is_err());
}
for g in [
graph("PRelu", &["s"], vec![("s", weight(&[3], vec![0.1; 3]))], vec![]),
graph("Gemm", &["w"], vec![("w", weight(&[4, 2], vec![0.1; 8]))], vec![]),
graph("Flatten", &[], vec![], vec![("axis", Attr::Int(i64::MIN))]),
graph("Dropout", &[], vec![], vec![("ratio", Attr::Float(1.0))]),
graph("Transpose", &[], vec![], vec![("perm", Attr::Ints(vec![0, 1, 1, 3]))]),
] {
assert!(Net::new(g).and_then(|n| n.validate_input(&[1, 3, 2, 2])).is_err());
}
}
#[test]
fn arithmetic_overflow_is_reported_and_next_run_succeeds() {
let net = Net::new(graph("Mul", &["w"], vec![("w", weight(&[1], vec![f32::MAX]))], vec![])).unwrap();
assert!(net.run(Tensor::new(vec![1], vec![2.0]).unwrap()).is_err());
assert!(net.run(Tensor::new(vec![1], vec![0.0]).unwrap()).is_ok());
}
#[test]
fn multiple_batches_prelu_keeps_channels_independent() {
let out = execute(graph("PRelu", &["s"], vec![("s", weight(&[3, 1, 1], vec![0.1, 0.2, 0.3]))], vec![]), &[2, 3, 2, 2], vec![-1.0; 24]);
for (i, &v) in out.data.iter().enumerate() {
assert_eq!(v, -[0.1, 0.2, 0.3][i / 4 % 3]);
}
}
#[test]
fn integer_activations_and_constant_outputs_fail_at_load() {
let integer = Weight { dims: vec![1], data: Data::I64(vec![1]) };
assert!(Net::new(graph("Add", &["w"], vec![("w", integer.clone())], vec![])).is_err());
let mut g = graph("Relu", &[], vec![("w", integer)], vec![]);
g.outputs = vec!["w".into()];
assert!(Net::new(g).is_err());
}
#[test]
fn normalisation_coefficient_and_working_memory_overflows_fail_before_running() {
let g = graph(
"BatchNormalization",
&["s", "b", "m", "v"],
vec![
("s", weight(&[3], vec![f32::MAX; 3])),
("b", weight(&[3], vec![0.0; 3])),
("m", weight(&[3], vec![f32::MAX; 3])),
("v", weight(&[3], vec![0.0; 3])),
],
vec![],
);
assert!(Net::new(g).unwrap().validate_input(&[1, 3, 2, 2]).is_err());
let g = graph("Conv", &["w"], vec![("w", weight(&[4096, 3, 1, 1], vec![0.1; 4096 * 3]))], vec![]);
assert!(Net::new(g).unwrap().validate_input(&[1, 3, 4096, 4096]).is_err());
}
#[test]
fn empty_broadcast_operands_are_rejected_before_allocation() {
for op in ["Add", "Sub", "Mul", "PRelu"] {
let g = graph(op, &["w"], vec![("w", weight(&[0, 3, 1, 1], vec![]))], vec![]);
assert!(Net::new(g).and_then(|n| n.validate_input(&[1, 3, 2, 2])).is_err());
}
}
#[test]
fn odd_maps_cross_panel_boundaries_and_reused_padding_is_zero() {
for (c, co, groups) in [(3, 11, 1), (6, 10, 2)] {
let weights: Vec<_> = (0..co * (c / groups) * 9).map(|i| (i as f32 % 19.0 - 9.0) * 0.013).collect();
let g = graph(
"Conv",
&["w", "b"],
vec![("w", weight(&[co, c / groups, 3, 3], weights)), ("b", weight(&[co], vec![0.1; co]))],
vec![("group", Attr::Int(groups as i64)), ("strides", Attr::Ints(vec![1, 1])), ("pads", Attr::Ints(vec![2, 1, 0, 1]))],
);
let mut scratch = kernels::Scratch::default();
for zero in [false, true, false] {
let input = Tensor::new(vec![1, c, 23, 29], (0..c * 23 * 29).map(|i| if zero { 0.0 } else { (i as f32 % 29.0 - 14.0) * 0.12 }).collect())
.unwrap();
let got = kernels::conv(&g.nodes[0], &input, &g, &mut scratch).unwrap();
let want = conv_direct(&g.nodes[0], &input, &g).unwrap();
assert_eq!(got.shape, want.shape);
for (&a, &b) in got.data.iter().zip(&want.data) {
assert!((a - b).abs() < 1e-5, "{a} vs {b}");
}
}
}
}
#[test]
fn retained_convolution_scratch_counts_towards_the_working_limit() {
let network = |output_channels: usize| {
let pads = vec![("pads", Attr::Ints(vec![1, 0, 0, 0]))];
let mut g = graph("Conv", &["w"], vec![("w", weight(&[2, 130_000, 2, 1], vec![0.0; 2 * 130_000 * 2]))], pads.clone());
g.inputs[0].1 = vec![1, 130_000, 2, 256];
g.weights.insert("v".into(), weight(&[output_channels, 2, 2, 1], vec![0.0; output_channels * 4]));
g.nodes.push(Node {
op: "Conv".into(),
inputs: vec!["y".into(), "v".into()],
outputs: vec!["z".into()],
attrs: pads.into_iter().map(|(s, a)| (s.into(), a)).collect(),
});
g.outputs = vec!["z".into()];
Net::new(g).unwrap()
};
// The first layer retains a large im2col buffer; the second grows the product buffer.
// Both layers individually fit, but keeping both scratch peaks would exceed 512 MiB.
let shape = [1, 130_000, 2, 256];
assert!(network(131_072).validate_input(&shape).is_err());
assert!(network(65_536).validate_input(&shape).is_ok());
}
#[test]
fn constant_input_copies_count_towards_the_working_limit() {
let network = |retained_outputs: usize| {
let dims = [1, 1, 2048, 2048];
let mut g = graph("Identity", &[], vec![("w", weight(&dims, vec![0.0; 2048 * 2048]))], vec![]);
g.inputs[0].1 = dims.to_vec();
g.nodes.clear();
g.outputs.clear();
for i in 0..retained_outputs {
let name = format!("retained-{i}");
g.nodes.push(Node { op: "Identity".into(), inputs: vec!["x".into()], outputs: vec![name.clone()], attrs: HashMap::new() });
g.outputs.push(name);
}
g.nodes.push(Node { op: "Add".into(), inputs: vec!["x".into(), "w".into()], outputs: vec!["z".into()], attrs: HashMap::new() });
g.outputs.push("z".into());
Net::new(g).unwrap()
};
// Infer only: no pixel tensors are allocated. The constant's temporary copy must be counted.
assert!(network(30).validate_input(&[1, 1, 2048, 2048]).is_err());
assert!(network(29).validate_input(&[1, 1, 2048, 2048]).is_ok());
}
+496
View File
@@ -0,0 +1,496 @@
//! Bounded, allocation-free (apart from small shape lists) graph checking.
use super::*;
use crate::graph::{Attr, Weight};
pub(super) fn elements(s: &[usize]) -> Result<usize> {
if s.len() > 8 || s.contains(&0) {
return shape_err("a tensor has a zero dimension or excessive rank");
}
s.iter()
.try_fold(1usize, |n, &d| n.checked_mul(d))
.filter(|&n| n <= MAX_TENSOR)
.ok_or_else(|| NetError::Shape("a tensor exceeds the element limit".into()))
}
pub(super) fn tensor(s: &[usize], n: usize, data: &[f32]) -> Result<()> {
if elements(s)? != n || data.iter().any(|v| !v.is_finite()) {
return shape_err("invalid tensor size or non-finite values");
}
Ok(())
}
pub(super) fn float(n: &Node, name: &str, default: f32) -> f32 {
match n.attrs.get(name) {
Some(Attr::Float(v)) => *v,
_ => default,
}
}
pub(super) fn weight<'a>(n: &Node, g: &'a Graph, i: usize) -> Result<&'a Weight> {
n.inputs.get(i).and_then(|s| g.weights.get(s)).ok_or_else(|| NetError::Unsupported(format!("{} needs constant input {i}", n.op)))
}
pub(super) fn floats<'a>(n: &Node, g: &'a Graph, i: usize) -> Result<&'a [f32]> {
match &weight(n, g, i)?.data {
Data::F32(v) => Ok(v),
_ => unsupported(format!("{} needs float32 weights", n.op)),
}
}
pub(super) fn check_graph(g: &Graph) -> Result<()> {
if g.inputs.len() != 1 || g.outputs.is_empty() || g.outputs.len() > 64 || g.nodes.len() > 20_000 || g.weights.len() > 20_000 {
return unsupported("one input and at most 64 outputs are required");
}
let mut known = std::collections::HashSet::new();
let mut total_weights = 0usize;
for (name, w) in &g.weights {
let count = w.dims.iter().try_fold(1usize, |n, &d| n.checked_mul(d));
let len = match &w.data {
Data::F32(v) => {
if v.iter().any(|v| !v.is_finite()) {
return shape_err("non-finite weights");
}
v.len()
}
Data::I64(v) => v.len(),
};
if name.is_empty()
|| name.len() > 4096
|| w.dims.len() > 8
|| w.dims.iter().any(|&d| d > MAX_TENSOR)
|| count != Some(len)
|| len > MAX_TENSOR
{
return shape_err("invalid weight name, shape or size");
}
total_weights = total_weights
.checked_add(len)
.filter(|&v| v <= MAX_TENSOR)
.ok_or_else(|| NetError::Shape("combined weights exceed 64 Mi elements".into()))?;
known.insert(name.as_str());
}
let input = g.inputs.first().ok_or_else(|| NetError::Shape("missing input".into()))?;
if input.0.is_empty() || !known.insert(&input.0) {
return shape_err("invalid or repeated input name");
}
for n in &g.nodes {
if n.inputs.len() > 64 || n.outputs.len() != 1 || n.outputs.first().is_none_or(|s| s.is_empty() || s.len() > 4096) {
return unsupported(format!("{} requires one named output", n.op));
}
if n.inputs.iter().any(|s| !s.is_empty() && !known.contains(s.as_str())) {
return shape_err(format!("{} reads an unknown or forward value", n.op));
}
if !known.insert(n.outputs.first().ok_or_else(|| NetError::Shape("missing output".into()))?) {
return shape_err("a graph value is defined twice");
}
}
if g.outputs.iter().any(|s| g.weights.contains_key(s)) {
return unsupported("constant graph outputs");
}
let mut output_names = std::collections::HashSet::new();
if g.outputs.iter().any(|s| !known.contains(s.as_str()) || !output_names.insert(s)) {
return shape_err("unknown or repeated graph output");
}
Ok(())
}
pub(super) fn check(n: &Node, g: &Graph) -> Result<()> {
let (min, max, attrs): (usize, usize, &[(&str, u8)]) = match n.op.as_str() {
"Conv" => (2, 3, &[("group", 0), ("kernel_shape", 2), ("strides", 2), ("pads", 2), ("dilations", 2), ("auto_pad", 3)]),
"BatchNormalization" => (5, 5, &[("epsilon", 1), ("momentum", 1), ("training_mode", 0)]),
"PRelu" | "Add" | "Sub" | "Mul" => (2, 2, &[]),
"Relu" | "Sigmoid" | "GlobalAveragePool" | "Identity" => (1, 1, &[]),
"Dropout" => (1, 1, &[("ratio", 1), ("seed", 0)]),
"Flatten" => (1, 1, &[("axis", 0)]),
"Gemm" => (2, 3, &[("alpha", 1), ("beta", 1), ("transA", 0), ("transB", 0)]),
"MaxPool" => {
(1, 1, &[("kernel_shape", 2), ("strides", 2), ("pads", 2), ("dilations", 2), ("ceil_mode", 0), ("auto_pad", 3), ("storage_order", 0)])
}
"Resize" => (
3,
4,
&[
("mode", 3),
("coordinate_transformation_mode", 3),
("nearest_mode", 3),
("cubic_coeff_a", 1),
("exclude_outside", 0),
("extrapolation_value", 1),
],
),
"Transpose" => (1, 1, &[("perm", 2)]),
"Reshape" => (2, 2, &[("allowzero", 0)]),
op => return unsupported(format!("the operator `{op}`")),
};
if !(min..=max).contains(&n.inputs.len()) || n.inputs.first().is_none_or(String::is_empty) {
return shape_err(format!("{} has invalid inputs", n.op));
}
for (name, value) in &n.attrs {
let actual = match value {
Attr::Int(_) => 0,
Attr::Float(v) if v.is_finite() => 1,
Attr::Ints(_) => 2,
Attr::Str(_) => 3,
_ => 255,
};
if attrs.iter().find(|(key, _)| *key == name).is_none_or(|(_, kind)| *kind != actual) {
return unsupported(format!("{} attribute `{name}` (or its type)", n.op));
}
}
for (i, name) in n.inputs.iter().enumerate() {
if let Some(w) = g.weights.get(name)
&& matches!(w.data, Data::I64(_))
&& !(n.op == "Reshape" && i == 1)
{
return unsupported(format!("{} needs float32 input {i}", n.op));
}
}
match n.op.as_str() {
"Conv" => {
let w = weight(n, g, 1)?;
floats(n, g, 1)?;
if w.dims.len() != 4 || w.dims.contains(&0) {
return shape_err("Conv needs nonempty 2-D weights");
}
if n.ints("kernel_shape").is_some_and(|v| {
v.iter().map(|&d| usize::try_from(d).ok()).collect::<Option<Vec<_>>>().as_deref() != Some(w.dims.get(2..).unwrap_or(&[]))
}) {
return shape_err("Conv kernel_shape differs from its weights");
}
if n.ints("dilations").is_some_and(|d| d != [1, 1]) || n.string("auto_pad").is_some_and(|s| s != "NOTSET") {
return unsupported("Conv dilation or automatic padding");
}
let (sh, sw) = pair(n, "strides", 1)?;
let (pt, pl, pb, pr) = pads(n)?;
if [sh, sw, pt, pl, pb, pr].iter().any(|&v| v > 65536) {
return shape_err("Conv stride or padding exceeds 65536");
}
if n.int("group").unwrap_or(1) < 1 {
return shape_err("Conv group must be positive");
}
if n.inputs.get(2).is_some_and(|s| !s.is_empty()) {
floats(n, g, 2)?;
}
}
"BatchNormalization" => {
if n.int("training_mode").unwrap_or(0) != 0 {
return unsupported("training BatchNormalization");
}
if float(n, "epsilon", 1e-5) <= 0.0 {
return shape_err("BatchNormalization epsilon must be positive");
}
for i in 1..5 {
floats(n, g, i)?;
}
}
"PRelu" => {
floats(n, g, 1)?;
}
"Gemm" => {
floats(n, g, 1)?;
if n.inputs.get(2).is_some_and(|s| !s.is_empty()) {
floats(n, g, 2)?;
}
}
"Dropout" => {
if !(0.0..1.0).contains(&float(n, "ratio", 0.5)) {
return shape_err("Dropout ratio is invalid");
}
}
"MaxPool" => {
pair(n, "kernel_shape", 1)?;
pair(n, "strides", 1)?;
if n.ints("kernel_shape").is_none()
|| n.int("ceil_mode").unwrap_or(0) != 0
|| n.int("storage_order").unwrap_or(0) != 0
|| n.ints("dilations").is_some_and(|v| v != [1, 1])
|| pads(n)? != (0, 0, 0, 0)
|| n.string("auto_pad").is_some_and(|s| s != "NOTSET")
{
return unsupported("this MaxPool padding, dilation or ceil mode");
}
}
"Resize" => {
floats(n, g, 2)?;
if n.string("mode").unwrap_or("nearest") != "nearest" || n.inputs.get(3).is_some_and(|s| !s.is_empty()) {
return unsupported("Resize requires nearest constant scales");
}
if n.int("exclude_outside").unwrap_or(0) != 0 || float(n, "extrapolation_value", 0.0) != 0.0 {
return unsupported("Resize excludes or extrapolates outside values");
}
let coord = n.string("coordinate_transformation_mode").unwrap_or("half_pixel");
let nearest = n.string("nearest_mode").unwrap_or("round_prefer_floor");
if !matches!((coord, nearest), ("asymmetric", "floor") | ("half_pixel", "round_prefer_floor")) {
return unsupported("this Resize coordinate/nearest mode");
}
if n.inputs.get(1).is_some_and(|s| !s.is_empty()) && !floats(n, g, 1)?.is_empty() {
return unsupported("Resize with an ROI");
}
}
"Reshape" => {
if n.int("allowzero").unwrap_or(0) != 0 {
return unsupported("Reshape allowzero");
}
if !matches!(weight(n, g, 1)?.data, Data::I64(_)) {
return unsupported("Reshape needs a constant int64 shape");
}
}
_ => {}
}
Ok(())
}
pub(super) fn broadcast(a: &[usize], b: &[usize]) -> Result<Vec<usize>> {
elements(a)?;
elements(b)?;
let rank = a.len().max(b.len());
let mut out = vec![1; rank];
for (i, o) in out.iter_mut().enumerate() {
let aa = i.checked_sub(rank - a.len()).and_then(|j| a.get(j)).copied().unwrap_or(1);
let bb = i.checked_sub(rank - b.len()).and_then(|j| b.get(j)).copied().unwrap_or(1);
if aa != bb && aa != 1 && bb != 1 {
return shape_err("incompatible broadcast shapes");
}
*o = aa.max(bb);
}
elements(&out)?;
Ok(out)
}
pub(super) fn axis(n: &Node, rank: usize) -> Result<usize> {
let raw = n.int("axis").unwrap_or(1);
let raw = if raw < 0 { raw.checked_add(rank as i64).ok_or_else(|| NetError::Shape("axis overflow".into()))? } else { raw };
usize::try_from(raw).ok().filter(|&a| a <= rank).ok_or_else(|| NetError::Shape("Flatten axis is outside its rank".into()))
}
fn image(s: &[usize]) -> Result<(usize, usize, usize)> {
match s {
[1, c, h, w] => Ok((*c, *h, *w)),
_ => unsupported("a spatial operator needs [1,C,H,W]"),
}
}
pub(super) fn infer(g: &Graph, input: &[usize]) -> Result<Vec<Vec<usize>>> {
elements(input)?;
let mut shapes: HashMap<&str, Vec<usize>> = g.weights.iter().map(|(n, w)| (n.as_str(), w.dims.clone())).collect();
shapes.insert(g.inputs.first().ok_or_else(|| NetError::Shape("no input".into()))?.0.as_str(), input.to_vec());
let mut live = elements(input)?;
// Scratch grows independently and remains allocated across all subsequent layers.
let mut scratch_lower = 0;
let mut scratch_product = 0;
let mut last = HashMap::new();
for (i, n) in g.nodes.iter().enumerate() {
for s in &n.inputs {
last.insert(s.as_str(), i);
}
}
for s in &g.outputs {
last.insert(s.as_str(), usize::MAX);
}
for (i, n) in g.nodes.iter().enumerate() {
let a = n.inputs.first().and_then(|s| shapes.get(s.as_str())).ok_or_else(|| NetError::Shape("missing input shape".into()))?;
elements(a)?;
let out = match n.op.as_str() {
"Conv" => {
let (cin, h, w) = image(a)?;
let dims = &weight(n, g, 1)?.dims;
let [co, cg, kh, kw] = dims.as_slice() else {
return shape_err("Conv dimensions");
};
let groups = usize::try_from(n.int("group").unwrap_or(1)).map_err(|_| NetError::Shape("Conv group".into()))?;
if groups.checked_mul(*cg) != Some(cin) || !co.is_multiple_of(groups) {
return shape_err("Conv channels do not match weights/groups");
}
if n.inputs.get(2).is_some_and(|s| !s.is_empty()) && weight(n, g, 2)?.dims != [*co] {
return shape_err("Conv bias is not a channel vector");
}
let (sh, sw) = pair(n, "strides", 1)?;
let (pt, pl, pb, pr) = pads(n)?;
let padded = |size: usize, p: usize, q: usize, k: usize, s: usize| {
size.checked_add(p)
.and_then(|v| v.checked_add(q))
.and_then(|v| v.checked_sub(k))
.and_then(|v| (v / s).checked_add(1))
.ok_or_else(|| NetError::Shape("Conv padding overflow or oversized kernel".into()))
};
vec![1, *co, padded(h, pt, pb, *kh, sh)?, padded(w, pl, pr, *kw, sw)?]
}
"BatchNormalization" => {
let channels = *a.get(1).ok_or_else(|| NetError::Shape("BatchNormalization rank must be at least two".into()))?;
for j in 1..5 {
if weight(n, g, j)?.dims != [channels] {
return shape_err("BatchNormalization parameters must be channel vectors");
}
}
if floats(n, g, 4)?.iter().any(|v| *v < 0.0) {
return shape_err("BatchNormalization has negative variance");
}
let eps = float(n, "epsilon", 1e-5);
let (s, b, m, v) = (floats(n, g, 1)?, floats(n, g, 2)?, floats(n, g, 3)?, floats(n, g, 4)?);
for (((&s, &b), &m), &v) in s.iter().zip(b).zip(m).zip(v) {
let variance = v + eps;
let coefficient = s / variance.sqrt();
let offset = b - m * coefficient;
if !variance.is_finite() || !coefficient.is_finite() || !offset.is_finite() {
return shape_err("BatchNormalization coefficients overflow");
}
}
a.clone()
}
"PRelu" => {
if broadcast(a, &weight(n, g, 1)?.dims)? != *a {
return shape_err("PRelu slope would enlarge its input");
}
a.clone()
}
"Add" | "Sub" | "Mul" => {
let b = n.inputs.get(1).and_then(|s| shapes.get(s.as_str())).ok_or_else(|| NetError::Shape("missing second input".into()))?;
broadcast(a, b)?
}
"Gemm" => {
let b = &weight(n, g, 1)?.dims;
let [ar, ac] = a.as_slice() else {
return shape_err("Gemm needs matrix A");
};
let [br, bc] = b.as_slice() else {
return shape_err("Gemm needs matrix B");
};
let (m, k) = if n.int("transA").unwrap_or(0) != 0 { (*ac, *ar) } else { (*ar, *ac) };
let (bk, cols) = if n.int("transB").unwrap_or(0) != 0 { (*bc, *br) } else { (*br, *bc) };
if k != bk {
return shape_err("Gemm inner dimensions differ");
}
let s = vec![m, cols];
if n.inputs.get(2).is_some_and(|s| !s.is_empty()) && broadcast(&s, &weight(n, g, 2)?.dims)? != s {
return shape_err("Gemm bias would enlarge the output");
}
s
}
"Flatten" => {
let at = axis(n, a.len())?;
vec![
elements(a.get(..at).ok_or_else(|| NetError::Shape("bad axis".into()))?)?,
elements(a.get(at..).ok_or_else(|| NetError::Shape("bad axis".into()))?)?,
]
}
"GlobalAveragePool" => {
let (c, _, _) = image(a)?;
vec![1, c, 1, 1]
}
"MaxPool" => {
let (c, h, w) = image(a)?;
let (kh, kw) = pair(n, "kernel_shape", 1)?;
let (sh, sw) = pair(n, "strides", 1)?;
if h < kh || w < kw {
return shape_err("MaxPool kernel exceeds input");
}
vec![1, c, (h - kh) / sh + 1, (w - kw) / sw + 1]
}
"Resize" => {
let (c, h, w) = image(a)?;
let s = floats(n, g, 2)?;
if s.len() != 4
|| s.first() != Some(&1.0)
|| s.get(1) != Some(&1.0)
|| s.iter().any(|&v| !(1.0..=16.0).contains(&v) || v.fract() != 0.0)
{
return unsupported("Resize supports whole spatial scales from 1 to 16");
}
vec![
1,
c,
h.checked_mul(s.get(2).copied().unwrap_or(0.0) as usize).ok_or_else(|| NetError::Shape("Resize overflow".into()))?,
w.checked_mul(s.get(3).copied().unwrap_or(0.0) as usize).ok_or_else(|| NetError::Shape("Resize overflow".into()))?,
]
}
"Transpose" => {
let perm: Vec<i64> = n.ints("perm").map_or_else(|| (0..a.len() as i64).rev().collect(), Vec::from);
let mut seen = std::collections::HashSet::new();
let mut s = Vec::new();
if perm.len() != a.len() {
return shape_err("Transpose rank mismatch");
}
for d in perm {
let d = usize::try_from(d).map_err(|_| NetError::Shape("negative permutation".into()))?;
if !seen.insert(d) {
return shape_err("repeated permutation axis");
}
s.push(*a.get(d).ok_or_else(|| NetError::Shape("bad permutation axis".into()))?);
}
s
}
"Reshape" => {
let Data::I64(dims) = &weight(n, g, 1)?.data else {
return shape_err("Reshape datatype");
};
if dims.len() > 8 {
return shape_err("Reshape rank exceeds eight");
}
let total = elements(a)?;
let mut s = Vec::new();
let mut infer = None;
for (j, &v) in dims.iter().enumerate() {
s.push(match v {
-1 if infer.is_none() => {
infer = Some(j);
1
}
0 => *a.get(j).ok_or_else(|| NetError::Shape("Reshape copies missing dimension".into()))?,
v if v > 0 => usize::try_from(v).map_err(|_| NetError::Shape("Reshape dimension overflow".into()))?,
_ => return shape_err("invalid Reshape dimension"),
});
}
let known = elements(&s)?;
if let Some(j) = infer {
if !total.is_multiple_of(known) {
return shape_err("Reshape does not divide evenly");
}
*s.get_mut(j).ok_or_else(|| NetError::Shape("missing inferred dimension".into()))? = total / known;
}
if elements(&s)? != total {
return shape_err("Reshape changes element count");
}
s
}
_ => a.clone(),
};
let out_count = elements(&out)?;
let mut copied_constants = 0usize;
for (j, name) in n.inputs.iter().enumerate() {
// Execution borrows operator weights, but get(0) and binary get(1) materialize constants.
if (j == 0 || (j == 1 && matches!(n.op.as_str(), "Add" | "Sub" | "Mul")))
&& let Some(w) = g.weights.get(name)
{
copied_constants =
copied_constants.checked_add(elements(&w.dims)?).ok_or_else(|| NetError::Shape("constant copies overflow".into()))?;
}
}
if n.op == "Conv" {
let wd = &weight(n, g, 1)?.dims;
let [co, cg, kh, kw] = wd.as_slice() else {
return shape_err("Conv rank");
};
let block = 256.min(out.get(2).copied().unwrap_or(1) * out.get(3).copied().unwrap_or(1));
let lower = cg
.checked_mul(*kh)
.and_then(|v| v.checked_mul(*kw))
.and_then(|v| v.checked_mul(block))
.filter(|&v| v <= MAX_TENSOR)
.ok_or_else(|| NetError::Shape("convolution scratch exceeds the limit".into()))?;
scratch_lower = scratch_lower.max(lower);
scratch_product = scratch_product.max(co * block);
}
let scratch = scratch_lower + scratch_product;
if live
.checked_add(out_count)
.and_then(|v| v.checked_add(scratch))
.and_then(|v| v.checked_add(copied_constants))
.is_none_or(|v| v > 128 * 1024 * 1024)
{
return shape_err("tensors and scratch exceed the 512 MiB working limit");
}
live = live
.checked_add(out_count)
.filter(|&n| n <= 128 * 1024 * 1024)
.ok_or_else(|| NetError::Shape("live tensors exceed the 512 MiB working limit".into()))?;
shapes.insert(n.outputs.first().ok_or_else(|| NetError::Shape("no output".into()))?, out);
let mut removed = std::collections::HashSet::new();
for name in &n.inputs {
if last.get(name.as_str()) == Some(&i) && !g.weights.contains_key(name) && removed.insert(name) {
live = live.saturating_sub(shapes.get(name.as_str()).map_or(0, |s| elements(s).unwrap_or(0)));
}
}
}
g.outputs.iter().map(|s| shapes.get(s.as_str()).cloned().ok_or_else(|| NetError::Shape("missing output".into()))).collect()
}
Loaded 100 of 244 files, more files were not shown because too many files have changed in this diff. Show more