Compare commits

..
Author SHA1 Message Date
Brandon ThomasandClaude Opus 5.5 67ee9348c3 WIP Issue #147: Sony M/S-size lossless ARW (subsampled YCbCr) — interrupted, unverified, do not land
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 18:22:50 -04:00
Brandon ThomasandClaude Opus 5.5 33a84b8427 Issue #148: Raw: read white balance, black level and crop of pre-2017 Sony ARWs
Sony bodies from before ~2017 (RX100–RX100M5, RX10, NEX, SLT, ILCE-6000,
A7/A7II/A7RII, ...) write no plain 0x7313/0x7310/0x74c7 tags in the raw IFD,
so the decoder used unit WB multipliers (strong green cast, e.g. the CC0
DSC-RX100M3 sample), black 512 (800 on 1-inch sensors), and kept padding
columns at the right edge.

- White balance: WB_RGBLevels from the maker note's enciphered Tag2010
  block (layouts per ExifTool's Tag2010a..i tables; byte substitution
  p^3 mod 249 checked black-box on 15 CC0 files: SonyISO matches Exif ISO,
  ILCE-7M3 levels match its plain 0x7313).
- Black level: recovered from the encrypted SR2SubIFD by known-plaintext
  analysis (constant key in every file; RX100M6/ILCE-7M3 plain 0x7310 as
  plaintext; 8 keystream bytes at offset 2510, plausibility-checked).
- Crop: DNG DefaultCrop or the maker note's FullImageSize.
- "12-bit uncompressed" ARWs hold 14-bit-scale data: black/white follow it.
- Render cache version bumped; corpus manifest + corpus test, unit tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 15:27:33 -04:00
Brandon Thomas 265248c86f Merge pull request #151 from storytold/fix/snapshot-test-settle
Tests: snapshot-determinism test asserts both runs settled
2026-10-06 15:14:10 -04:00
Brandon ThomasandClaude Opus 5.5 1130e99095 Tests: the snapshot-determinism test asserts both runs settled before comparing pixels
snapshot() ignored settle()'s result, so on a loaded machine (seen on the FreeBSD VM and
under local load) a timed-out settle compared half-rendered frames and failed as a
misleading pixel diff. The test now settles explicitly with a 300 s budget, asserts it
settled, then paints.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 15:13:58 -04:00
Brandon Thomas 07a7316d8b Merge pull request #149 from storytold/pr/craft-fonts
Fonts: use craft-fonts as an optional build input; move the Japanese fonts out
2026-10-06 14:49:22 -04:00
Brandon ThomasandClaude Opus 5.5 2aa236537f Fonts: use craft-fonts as an optional build input; move the Japanese fonts out
The Japanese fonts (BIZ UDPGothic Regular/Bold for the UI, BIZ UDMincho for text
watermarks; #110, #114) now come from storytold/craft-fonts instead of assets/fonts/.

- crates/engine/build.rs: the craft-fonts recipe. With CRAFT_FONTS_DIR set it embeds
  the manifest's fonts as lightcraft_engine::CRAFT_FONTS (wasm32: BIZ UDPGothic Regular
  only, for the web size budget); unset, CRAFT_FONTS is empty. Local files only, no
  network; craft-fonts is not in any Cargo.toml.
- UI: theme::font_definitions adds the Japanese faces after Inter and egui's defaults
  in every family (BIZ UDPGothic first, Bold first for semibold). About box lists the
  embedded families.
- Watermarks: Inter, then the craft-fonts Japanese faces (Mincho first).
- Removed BIZUDPGothic-Regular/Bold.ttf, BIZUDMincho-Regular.ttf and their OFL files,
  and their ATTRIBUTION rows; added an attribution row for craft-fonts as an optional
  build input; NOTICE updated.
- Tests: Japanese glyph tests use CRAFT_FONTS and skip without it; new tests for
  Japanese in the UI with craft-fonts and for the UI and watermarks without it.
- CI: the FreeBSD job builds/tests with craft-fonts. Releases check out craft-fonts at
  a pinned commit, build with CRAFT_FONTS_DIR + CRAFT_FONTS_REQUIRED=1, and ship the
  fonts' OFL licences (plus NOTICE) in every package.
- Docs: AGENTS.md, CLAUDE.md, README, docs/web.md, docs/localization-ja.md, ROADMAP,
  parity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 11:35:02 -04:00
Brandon Thomas bd3185ef4d Merge pull request #145 from storytold/fix/issue-138-lightroom-dng
Issue #138: DNG: apply the file's own camera-profile look (HueSatMap, LookTable, ToneCurve)
2026-10-06 11:27:16 -04:00
Brandon Thomas 4cf883155a Merge pull request #144 from storytold/fix/issue-136-vulkan-startup-crash
Issue #136: Windows: use DX12 by default, honour WGPU_BACKEND, never let GPU init block startup
2026-10-06 11:27:03 -04:00
Brandon ThomasandClaude Opus 5.5 71eac70696 Issue #138: DNG: apply the file's own camera-profile look (HueSatMap, LookTable, ToneCurve)
Lightroom-converted DNGs carry their camera profile's ProfileHueSatMapData1/2
and ProfileLookTableData (and sometimes ProfileToneCurve). We used only the
colour matrices, so such DNGs rendered flat and muted.

- raw: new `profile` module reads the tags (DNG spec ch. 6: value-major,
  hue-middle, saturation-minor tables, wrap-around hue, tri-linear lookups,
  sRGB value encoding) from the raw IFD or IFD 0; malformed tags are ignored.
- engine: the raw loader applies the hue/sat map (interpolated by the same
  illuminant weight as the matrices) before BaselineExposure and the look
  table after it, in linear ProPhoto; a ProfileToneCurve becomes the source's
  camera tone. Without one the standard base tone is kept.
- dngwrite keeps the look with the data (conversions, smart previews, merges).
- Render cache version 7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 10:36:30 -04:00
Brandon ThomasandClaude Opus 5.5 b66cdb23a3 Issue #136: Windows: use DX12 by default, honour WGPU_BACKEND, never let GPU init block startup
wgpu loads the driver of every backend in an instance's set while it
enumerates adapters. The compute device (crates/gpu) hard-coded
Backends::PRIMARY (Vulkan + DX12 + Metal) and ignored WGPU_BACKEND, and
was created from LightcraftApp::new on a background thread at launch, so
a crashing Vulkan driver (Intel UHD 630, igvk64.dll) killed the process
before the window appeared; the window (egui-wgpu) also listed Vulkan.

- lightcraft_gpu::backend: DX12 only on Windows (Metal on macOS, as before
  on Linux) for both the compute device and the window;
  LIGHTCRAFT_GPU_BACKEND=dx12|vulkan|metal|gl|auto|off, else WGPU_BACKEND;
  backends missing from the build fall back to the default.
- The device is created only when GPU rendering is enabled, after the
  first frame; the preference is applied before the window opens.
- Crash sentinel: gpu-init.marker in the settings folder around device
  creation; if left behind, the next launch starts with GPU rendering off
  and a notice saying how to turn it back on.
- Tests: backend parsing/selection, sentinel, window defaults, headless
  snapshot with LIGHTCRAFT_GPU=0 / LIGHTCRAFT_GPU_BACKEND=off.
- Docs: gpu-pipeline.md (backends, env vars, troubleshooting), whats-new,
  parity LR-BEHAV-GPU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 10:16:05 -04:00
Brandon Thomas d2359ce095 Merge pull request #143 from storytold/fix/issue-134-trim-write-safety
Issue #134: Exports: atomic without fsync; Copy import verifies against the probe hash
2026-10-06 10:11:47 -04:00
Brandon ThomasandClaude Opus 5.5 7a2596cd43 Issue #134: Exports: atomic without fsync; Copy import verifies against the probe hash
- safe_file: write policy documented (durable for what can't be recreated, atomic-only for
  recreatable outputs); new write_atomic_nosync (temp + rename, temp removed on failure, no
  file/folder sync); per-thread sync counter for tests.
- export::write_file (exports, renders, screenshots, merge previews, CLI/MCP) uses the
  non-syncing writer; new write_file_durable keeps the sync for Edit a Copy (the TIFF becomes a
  library photo). Catalog, sidecars, settings, DNG conversion/merges and smart previews stay
  durable; the #93 original-overwrite guard is untouched.
- Import > Copy verifies the synced copy against the probe's content hash (SipHash-1-3/128 of the
  whole file) instead of re-reading the source; byte compare when there is no hash. A file changed
  since Review Import is now refused instead of imported with stale details. Move keeps its
  byte-for-byte check before deleting sources.
- check_write_target: skip building the library-wide guard when nothing is at the target.
- write_new_unique keeps its read-back (DNGs replace raws; cheap next to encoding).
- Tests: no sync on the export path, durable writers still sync, Copy caught by the probe hash
  (corrupt copy, stale hash, changed source); ignored timing test write_policy_timing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 10:08:29 -04:00
Brandon Thomas 8bef0b1ef3 Merge pull request #141 from storytold/pr/windows-arm64
Windows on ARM64: release builds, PE header check, install test on ARM64 hardware
2026-10-06 09:59:27 -04:00
Brandon ThomasandClaude Opus 5.5 a2dbefd1ee Windows ARM64 workflow: pin third-party actions by commit
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:51:08 -04:00
Brandon ThomasandClaude Opus 5.5 f5bd3cc2db Windows on ARM64: release builds, PE header check, install test on ARM64 hardware
Release builds add an arm64 MSI and portable zip, cross-compiled on the x64
runner (signing and WiX as for x64). package.ps1 checks each binary's PE
header before packaging: Machine must match -Arch, the app must be a GUI
program and the CLI a console program. It skips the CLI smoke run when an
ARM64 build can't run on the host. A new Windows ARM64 workflow packages the
ARM64 MSI, installs it on a Windows 11 ARM64 runner, checks both installed
exes are ARM64, runs the CLI, and uninstalls. Ported from printcraft#66.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:38:47 -04:00
Brandon Thomas 1969b311cc Merge pull request #137 from benlukka/fix-flickering-preview-hover
fix(ui): eliminate preview flickering when hovering presets and profiles
2026-10-06 09:24:12 -04:00
Brandon Thomas fc3fa1a71b Merge pull request #140 from storytold/pr/release-0.2.1
Release: LightCraft v0.2.1
2026-10-06 08:01:39 -04:00
Brandon ThomasandClaude Opus 5.5 d0612b0b21 Release: LightCraft v0.2.1
Fixes the Windows installer: the Start Menu shortcut and Add/Remove Programs icon were blank, and the shortcut description was wrong.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:01:29 -04:00
Brandon Thomas 66dc737ff5 Merge pull request #139 from storytold/pr/windows-packaging
Windows packaging: Start Menu/Add-Remove icon, no console window, correct description
2026-10-06 07:54:46 -04:00
Brandon ThomasandClaude Opus 5.5 fec3b15234 release.yml: explicit if for the wasm-bindgen install (shellcheck SC2015)
packaging-lint has failed on main since this step was added; same behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 07:53:44 -04:00
Brandon ThomasandClaude Opus 5.5 11ebb3bdb6 Windows packaging: Start Menu/Add-Remove icon, no console window, correct description
The MSI Icon id had no .ico/.exe extension, which Windows Installer needs for
advertised shortcuts and ARPPRODUCTICON, so both showed a blank icon. Release
builds of the desktop app are now GUI-subsystem (no console window; the CLI stays
a console program), the shortcut/summary description names what LightCraft does, and
packaging-lint rejects an Icon id without an extension. Found in printcraft#57.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 07:50:38 -04:00
benlukka ad2525ece2 fix(ui): eliminate preview flickering when hovering presets and profiles
- Retain active preset preview when hovering within the scroll or panel area across headers and gaps.
- Prevent premature dropping of hover preview textures during asynchronous rerendering in detail view.
- Clear cached hover state when clicking an item or when the pointer leaves the panel.
- Add headless regression tests covering preset hover persistence and transitions.

Signed-off-by: benlukka <68825026+benlukka@users.noreply.github.com>
2026-10-06 13:18:47 +02:00
Brandon Thomas 65da36675f Merge pull request #115 from arata-1972/codex/sony-arw-initial-colour
Improve Sony ARW starting colour and tone, and preserve decoded source metadata
2026-10-05 19:47:57 -04:00
Brandon ThomasandClaude Opus 5.5 cbb6c3b8f7 Merge origin/main (#114) into land-115
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:41:46 -04:00
Brandon ThomasandClaude Opus 5.5 5b414ffb8e LR-PROF-CAMERACOLOR: record the ARW file-local fit's public-sample results
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:34:01 -04:00
Brandon ThomasandClaude Opus 5.5 f7a7831380 LR-PROF-CAMERACOLOR: keep the ARW camera tone curve with smart previews
Smart previews store the decoded source with the fitted colour matrix
baked in, but the file-local tone curve lives in the decoder's
SourceInfo, which a proxy did not keep: an offline ARW rendered and
exported with the generic filmic curve instead (darker, a different
look). The proxy header now carries the curve (validated on load,
ignored when invalid), DecodedSource::info_or merges it into the header
facts, and is_valid reads the longer header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:30:43 -04:00
Brandon Thomas 2b2ade7a23 Merge pull request #114 from arata-1972/codex/lightcraft-japanese
Extend Japanese localization across editing, import and export UI
2026-10-05 19:29:49 -04:00
Brandon ThomasandClaude Opus 5.5 d6c7228dd4 M16: merge origin/main into Japanese localisation (PR #114)
Keep main's data-safety work (ui.json recovery and atomic writes, library-problem
window, off-thread file checks, import Cancel) and wrap its new strings: import
cancel/progress messages, the library-problem window and banner, and the new
Back Up / Restore Library menu items now have Japanese translations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:28:34 -04:00
Brandon ThomasandClaude Opus 5.5 4b3048c48f LR-PROF-CAMERACOLOR: accept ARW camera-look fits up to 0.10 held-out RMS
The 0.055 cap rejected half of the public Sony samples tried (ILCE-6400,
ILCE-6700, ILCE-7M4) although their fits were 1.5-3x closer to the
camera JPEG than the fallback and visibly better (mean dE vs the embedded
preview 19-26 -> 5-10). Unrelated previews stay rejected (synthetic test
lands at 0.115).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:28:22 -04:00
Brandon ThomasandClaude Opus 5.5 bc4799ca42 Merge origin/main into land-115 (PR #115)
Resolve SourceRef::load_source against #104's offline fallback: a File
source keeps the decoder's SourceInfo; a smart-preview fallback has none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:25:37 -04:00
Brandon ThomasandClaude Opus 5.5 586479a309 M16: drop fork-only Japanese packaging from PR #114
Remove scripts/package-japanese-macos.py, the japanese-local feature and its
config/library/startup overrides, the settings.language migration, and the
local-build notes in the docs. Upstream library and config defaults (incl.
library::default_dir, which already honours LIGHTCRAFT_LIBRARY) are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:11:34 -04:00
Brandon Thomas 87f275b58e Merge pull request #132 from storytold/fix/issue-104-ui-fs-v2
Issue #104: keep file-system calls off the UI thread
2026-10-05 19:08:15 -04:00
Brandon Thomas 894759e6d8 Merge main into fix/issue-104-ui-fs-v2
# Conflicts:
#	docs/parity.md
#	docs/whats-new.md
2026-10-05 19:00:37 -04:00
Brandon Thomas 1952f7c960 Merge pull request #131 from storytold/fix/issue-107-web-safety
Issue #107: Web build: data-safety fixes for libraries kept in the browser
2026-10-05 18:59:14 -04:00
Brandon Thomas 7bbdbc03ef Merge main into fix/issue-107-web-safety
# Conflicts:
#	apps/lightcraft-web/src/web.rs
#	docs/parity.md
#	docs/whats-new.md
2026-10-05 18:58:53 -04:00
Brandon Thomas 0f34a65a8c Merge pull request #130 from storytold/fix/issue-97-folder-undo
Issue #97: Rename Folder / Move Folder are undo steps
2026-10-05 18:58:38 -04:00
Brandon Thomas 6c066b66f2 Merge main into fix/issue-97-folder-undo
# Conflicts:
#	crates/engine/src/lib.rs
#	docs/parity.md
#	docs/whats-new.md
2026-10-05 18:58:13 -04:00
Brandon ThomasandClaude Opus 5.5 7b59081d22 Issue #104: keep file-system calls off the UI thread
Re-applies fix/issue-104-ui-fs on top of the data-safety fixes (integrate/safety):
- availability cache: originals and smart previews are checked on a worker thread; the UI
  answers from the cache (grid, Info panel, photo menu, Missing Photos view)
- import split into ImportJob::prepare (worker) + commit_prepared, with Cancel; Copy keeps
  #96's verified copies
- Find Missing Photos: find_candidates (snapshot, no I/O) + plan_find_missing (all file I/O,
  on a worker, never the session) with #105's matching rules unchanged (name+size+hash;
  renamed by size+hash; single name+size; ambiguous look-alikes skipped and reported; files
  already in use excluded). The plan is applied with library.findMissing {found, missing,
  ambiguous}, re-checked against the catalog then: photos relinked meanwhile are left alone,
  files now in use are not taken. library.findMissing {folder} still runs synchronously.
- Build / Discard Smart Previews with background: true runs on a worker with progress and
  Stop, keeping #106's atomic writes, is_valid checks and `repaired` count (also in
  library.previewProgress); the render worker's smart-preview fallback uses is_valid
- Rename preview planned on a worker (plan_rename_photos) on the same planning core as
  Session::plan_rename_with (MoveFs, case-only respelling by file identity, #95)
- Local folder listing via fs_cached; tests_offline (slow injected probe, frames < 300 ms)

Tests: worker-planned Find Missing equals the synchronous command for a renamed + ambiguous
library; a plan is re-checked when applied; background smart previews repair a damaged proxy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 18:58:04 -04:00
Brandon Thomas e11f77d9ca Merge pull request #129 from storytold/fix/issue-94-control-close
Issue #94: Control: close the connection on malformed input
2026-10-05 18:57:59 -04:00
Brandon Thomas f383d1e736 Merge main into fix/issue-94-control-close
# Conflicts:
#	docs/whats-new.md
2026-10-05 18:57:50 -04:00
Brandon Thomas edf05789ed Merge pull request #128 from storytold/fix/issue-103-settings-files
Issue #103: Settings files: keep damaged ones, never overwrite unread ones, write ui.json atomically, confirm quit with unsaved changes
2026-10-05 18:57:30 -04:00
Brandon Thomas bccccfb92a Merge main into fix/issue-103-settings-files
# Conflicts:
#	crates/engine/src/library.rs
#	docs/whats-new.md
2026-10-05 18:56:50 -04:00
Brandon Thomas 4939d9b994 Merge pull request #127 from storytold/fix/issue-100-open-failure
Issue #100: Ask what to do when the library can't be opened, never a silent demo session
2026-10-05 18:56:35 -04:00
Brandon Thomas e0ab41ede0 Merge main into fix/issue-100-open-failure 2026-10-05 18:56:00 -04:00
Brandon Thomas 8690b51fcf Merge pull request #126 from storytold/fix/issue-99-library-lock
Issue #99: Library: exclusive lock so two processes can't open one library
2026-10-05 18:55:44 -04:00
Brandon Thomas e1d3b56bb6 Merge main into fix/issue-99-library-lock
# Conflicts:
#	crates/catalog/src/lib.rs
2026-10-05 18:54:56 -04:00
Brandon Thomas 6100237cdb Merge pull request #125 from storytold/fix/issue-102-format-version
Issue #102: Catalog format version 2; refuse newer libraries untouched
2026-10-05 18:54:41 -04:00
Brandon Thomas 4579599745 Merge main into fix/issue-102-format-version 2026-10-05 18:54:03 -04:00
Brandon Thomas 8506fecf5f Merge pull request #124 from storytold/fix/issue-101-torn-append
Issue #101: catalog.log: cut a partial append back before retrying
2026-10-05 18:53:49 -04:00
Brandon Thomas 50cd8dc6dc Merge branch 'fix/issue-97-folder-undo' into integrate/safety
# Conflicts:
#	crates/engine/src/lib.rs
#	docs/parity.md
#	docs/whats-new.md
2026-10-05 18:38:55 -04:00
Brandon Thomas cc8b3eb3bb Merge branch 'fix/issue-94-control-close' into integrate/safety
# Conflicts:
#	docs/whats-new.md
2026-10-05 18:38:13 -04:00
Brandon Thomas a6b402981b Merge branch 'fix/issue-103-settings-files' into integrate/safety
# Conflicts:
#	crates/engine/src/library.rs
#	docs/whats-new.md
2026-10-05 18:26:38 -04:00
Brandon Thomas 4deb5eff04 Merge branch 'fix/issue-100-open-failure' into integrate/safety 2026-10-05 18:26:38 -04:00
Brandon Thomas 6e2408ff7d Merge branch 'fix/issue-99-library-lock' into integrate/safety
# Conflicts:
#	crates/catalog/src/lib.rs
2026-10-05 18:26:38 -04:00
Brandon Thomas 66be31e3d8 Merge branch 'fix/issue-102-format-version' into integrate/safety 2026-10-05 18:26:19 -04:00
Brandon Thomas 332f37a5c7 Merge branch 'fix/issue-101-torn-append' into integrate/safety
# Conflicts:
#	docs/whats-new.md
2026-10-05 18:26:19 -04:00
Brandon Thomas aa1af6d5e9 Merge branch 'fix/issue-92-xmp-merge' into integrate/safety
# Conflicts:
#	crates/engine/src/sidecar.rs
#	docs/whats-new.md
2026-10-05 18:26:19 -04:00
Brandon Thomas 963912b231 Merge branch 'fix/issue-92-rename-shared-sidecar' into integrate/safety 2026-10-05 18:25:06 -04:00
Brandon Thomas 4174ce8c57 Merge branch 'fix/issue-105-rename-rollback' into integrate/safety
# Conflicts:
#	docs/parity.md
2026-10-05 18:24:59 -04:00
Brandon Thomas f28e1c362e Merge branch 'fix/issue-95-case-only-rename' into integrate/safety 2026-10-05 18:24:34 -04:00
Brandon Thomas 70d800ba68 Merge branch 'fix/issue-98-thumb-cache-own-files' into integrate/safety 2026-10-05 18:24:33 -04:00
Brandon Thomas 33ef91a123 Merge branch 'fix/issue-96-verified-copy-import' into integrate/safety
# Conflicts:
#	docs/whats-new.md
2026-10-05 18:24:33 -04:00
Brandon Thomas f1b29cfe4a Merge branch 'fix/issue-106-dng-smart-previews' into integrate/safety 2026-10-05 18:24:23 -04:00
Brandon Thomas 130b807e1b Merge branch 'fix/issue-93-export-overwrite' into integrate/safety 2026-10-05 18:24:22 -04:00
arata1972andCodex 43f4e24355 M11: separate Sony ARW colour and tone and preserve decoded source facts
Co-authored-by: Codex <noreply@openai.com>
2026-10-06 07:16:15 +09:00
Brandon ThomasandClaude Opus 5.5 5e618c553a Issue #100: Ask what to do when the library can't be opened, never a silent demo session
When open_library failed, the desktop app (and the web build) quietly ran
an in-memory session seeded with the demo photos: it looked like a reset
library and everything done in it was lost at quit.

- The session now starts empty and in memory (no demo photos) and
  LightcraftApp::library_problem shows a blocking window with the path and
  the error: Try Again, Choose Another Library… (app.openLibrary), Continue
  Without Saving, Quit. Command-line files are imported once the user has
  chosen.
- A temporary session shows a banner under the top bar (with Open
  Library…) for as long as it lasts; it has no library, so nothing is
  written anywhere. Opening a library ends it.
- ui.inspect → libraryProblem; widget ids for automation; documented in
  control-protocol.md.
- Web build: same window (Try Again disabled: reloading the page retries).
- Headless UI test: locked library → window → Continue Without Saving →
  banner, nothing written to the library → lock released → Try Again opens
  it. Checked in the real app over the control channel too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 18:13:00 -04:00
Brandon ThomasandClaude Opus 5.5 cbebabccad Issue #102: Catalog format version 2; refuse newer libraries untouched
main added Op::SetBrowsed, Catalog.browsed and Photo.local_baseline while
the snapshot still said version 1, so v0.2.0 read such a library in part
(unknown ops as log damage, unknown fields dropped on its next snapshot).

- journal::VERSION = 2 (now public), with a format table and rules in the
  journal docs; a test with an exhaustive match over Op makes adding a
  variant fail until it is assigned a version.
- Journal::open reads the snapshot header first and refuses a newer
  version with CatalogError::Newer ("written by a newer version of
  LightCraft"), even when its catalog wouldn't parse.
- A log record whose CRC matches but whose op doesn't parse is "newer",
  not damage or a torn tail: refused before any file is modified.
- Opening an older (or snapshot-less) library writes a snapshot in this
  format before anything is appended (LoadReport::upgraded_from); a new
  library gets one at once, so v0.2.0 refuses it instead of reading a bare
  log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 18:13:00 -04:00
Brandon ThomasandClaude Opus 5.5 1a05c3eed2 Issue #103: Settings files: keep damaged ones, never overwrite unread ones, write ui.json atomically, confirm quit with unsaved changes
- Library settings (prefs.json, presets.json, view.json): only a missing
  file means "defaults". A file that doesn't parse is kept as
  <name>.corrupt-<unix time> before the defaults are used; one that can't
  be read (locked, I/O error) is never written this session (save_prefs
  refuses, presets/view saves skip it). Warnings go to library.info →
  settingsWarnings and Session::take_library_warnings; the UI shows each
  once in a window (ui.inspect → notices, button:noticeOk).
- Desktop ui.json (holds the library location): written atomically (temp +
  fsync + rename + dir fsync), saved as soon as the library path changes and
  every few seconds when it changed, failures reported; a damaged file is
  renamed to ui.json.corrupt-<time> and reported, an unreadable one is not
  overwritten.
- Quit/close with changes only in memory: retry the save (log, then a
  snapshot of everything), and if it still fails cancel the close and ask:
  Try Saving Again / Quit Anyway / Cancel (ui.inspect → quitPrompt).
- Tests: engine (damaged prefs kept, unreadable presets/prefs untouched),
  desktop (damaged ui.json kept, atomic write), headless UI (quit prompt
  buttons, settings notice).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 18:12:26 -04:00
Brandon ThomasandClaude Opus 5.5 ee4b78f646 Issue #99: Library: exclusive lock so two processes can't open one library
Two processes on one library each kept their own catalog and seq counter;
the last snapshot silently dropped the other's edits.

- lightcraft_catalog:🔒 LibraryLock holds an exclusive OS lock
  (std File::try_lock: flock / LockFileEx) on <library>/catalog.lock for
  the session; who holds it goes to catalog.lock.owner (separate file: a
  Windows lock also blocks reads). Released on drop and by the OS when the
  process dies, so leftover files from a crash never lock anyone out. File
  systems without lock support open unprotected with a warning.
- Session::open_library takes the lock before reading anything and fails
  with EngineError::LibraryInUse ("this library is already open in
  LightCraft 0.x (process N on host)"); reopening the session's own
  library hands its lock over.
- lightcraft-cli (mcp/run/snapshot --library) adds how to drive the
  running app instead (--control + mcp --connect).
- Tests: in-process, leftover files, a killed child process, two engine
  sessions, CLI e2e (mcp --library holds it, run --library is refused).
  Existing tests that reopened a library while it was open now drop the
  first session.
- docs: parity row LRC-LIB-CATALOG-LOCK, mcp.md, web.md (no guard across
  browser tabs yet), whats-new.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:48:34 -04:00
Brandon ThomasandClaude Opus 5.5 d592b29d12 Issue #97: Rename Folder / Move Folder are undo steps
The folder move and the photos' relinks form one undo step: undo renames
the folder back on disk (files and sidecars along) and restores the
paths, redo repeats it. A destination taken in the meantime is never
overwritten; the step is refused, reported and stays on the stack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:46:30 -04:00
Brandon ThomasandClaude Opus 5.5 cddcb905e0 Issue #94: Control: close the connection on malformed input
A line that is not a JSON object with a string `method` (HTTP request
line, invalid UTF-8, non-object JSON, >4 MiB line) now gets one error
reply and the connection is closed, so a browser's cross-origin fetch
can't smuggle a command in its body. Connections are capped at 16.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:46:27 -04:00
Brandon ThomasandClaude Opus 5.5 ebc73bb7b4 Issue #107: Web build: data-safety fixes for libraries kept in the browser
- File > Back Up Library... (web): a zip of the catalog files and every
  stored original, built entry by entry; Restore Library from Backup...
  reads it into a new library folder and switches only when complete
  (the previous library stays in storage).
- Failed flushes to browser storage make the catalog Store refuse
  writes, so changes show as unsaved (top-bar warning) and are retried
  every 2 s instead of being lost silently.
- A picked/dropped photo whose bytes can't be stored is not added.
- One tab per library (Web Locks); a second tab shows a message.
- Notices: storage not persistent, library failed to open (temporary
  session), experimental build; ?reset asks first; a panic shows a
  message instead of a frozen page; long toasts wrap.
- Hosting: _headers/.htaccess no longer mark the unhashed .js/.wasm
  immutable; HOSTING.md rewritten for the actual build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:46:21 -04:00
arata1972andCodex 509d5843dd M16: integrate expanded Japanese UI with upstream language settings
Co-authored-by: Codex <noreply@openai.com>
2026-10-06 06:20:23 +09:00
arata1972andCodex 05f242c445 M11: estimate missing ARW colour from guarded camera-preview correspondences
Co-authored-by: Codex <noreply@openai.com>
2026-10-06 05:55:20 +09:00
arata1972andCodex 87e0181690 M16: preserve upstream defaults with an opt-in Japanese local build
Co-authored-by: Codex <noreply@openai.com>
2026-10-06 05:36:19 +09:00
arata1972andCodex 7c799a9acf M16: add Japanese UI localization and isolated local macOS packaging
Co-authored-by: Codex <noreply@openai.com>
2026-10-06 05:15:04 +09:00
143 changed files with 10971 additions and 1784 deletions

No files matched your search

+7
View File
@@ -27,6 +27,12 @@ jobs:
timeout-minutes: 300
steps:
- uses: actions/checkout@v4
# Optional font build input, so the Japanese font tests run (they skip without it).
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695
path: craft-fonts
- name: Build and test in a FreeBSD VM
uses: vmactions/freebsd-vm@c46abacb49f09938ca4e1702d15d836285d694cc # v1
with:
@@ -44,5 +50,6 @@ jobs:
# The checkout is synced in under another owner; let tests that call git read it.
git config --global --add safe.directory "*"
rustc --version
export CRAFT_FONTS_DIR="$PWD/craft-fonts" CRAFT_FONTS_REQUIRED=1
CARGO_PROFILE_DEV_DEBUG=0 cargo build --workspace --all-targets --locked
CARGO_PROFILE_DEV_DEBUG=0 cargo test --workspace --locked
+4
View File
@@ -47,6 +47,10 @@ jobs:
- name: XML, desktop entry, AppStream, Flatpak manifest
run: |
xmllint --noout packaging/windows/lightcraft.wxs packaging/macos/*.plist* packaging/linux/*.xml*
# Windows Installer needs Icon ids ending in .ico or .exe, or shortcuts show a blank icon.
if grep -oE '<Icon Id="[^"]*"' packaging/windows/lightcraft.wxs | grep -vE '\.(ico|exe)"$'; then
echo "::error::Icon Id must end in .ico or .exe"; exit 1
fi
desktop-file-validate packaging/linux/ai.storyteller.lightcraft.desktop
sed -e 's/@VERSION@/0.0.0/' -e 's/@DATE@/2026-01-01/' packaging/linux/ai.storyteller.lightcraft.metainfo.xml.in >"$RUNNER_TEMP/ai.storyteller.lightcraft.metainfo.xml"
appstreamcli validate --no-net --explain "$RUNNER_TEMP/ai.storyteller.lightcraft.metainfo.xml"
+40 -2
View File
@@ -70,6 +70,12 @@ jobs:
LIGHTCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
# Fonts (optional build input; required for releases): see craftrules standards/fonts.md.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
targets: aarch64-apple-darwin, x86_64-apple-darwin
@@ -87,6 +93,8 @@ jobs:
run: packaging/macos/import-cert.sh
- name: Build, sign, notarize
env:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
APPLE_ID: ${{ secrets.APPLE_ID }}
APPLE_PASSWORD: ${{ secrets.APPLE_PASSWORD }}
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
@@ -112,11 +120,20 @@ jobs:
include:
- { arch: x64, target: x86_64-pc-windows-msvc }
- { arch: x86, target: i686-pc-windows-msvc }
# Windows on ARM: cross-compiled on the x64 runner, so signing and WiX run as above.
# .github/workflows/windows-arm64.yml installs and runs it on ARM64 hardware.
- { arch: arm64, target: aarch64-pc-windows-msvc }
env:
LIGHTCRAFT_VERSION: ${{ needs.version.outputs.version }}
LIGHTCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
# Fonts (optional build input; required for releases): see craftrules standards/fonts.md.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}
@@ -133,6 +150,8 @@ jobs:
- name: Build, sign, package
shell: pwsh
env:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
WINDOWS_CERTIFICATE: ${{ secrets.WINDOWS_CERTIFICATE }}
WINDOWS_CERTIFICATE_PASSWORD: ${{ secrets.WINDOWS_CERTIFICATE_PASSWORD }}
AZURE_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
@@ -167,6 +186,12 @@ jobs:
NFPM_VERSION: 2.47.0
steps:
- uses: actions/checkout@v4
# Fonts (optional build input; required for releases): see craftrules standards/fonts.md.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- name: Install build and packaging tools
run: |
sudo apt-get update
@@ -184,6 +209,9 @@ jobs:
if: inputs.version != ''
run: cargo xtask version set "$LIGHTCRAFT_VERSION"
- name: Build and package (AppImage, deb, rpm, tar.gz)
env:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
run: packaging/linux/package.sh
- name: Check the packages
run: |
@@ -212,6 +240,12 @@ jobs:
WASM_BINDGEN_VERSION: 0.2.129
steps:
- uses: actions/checkout@v4
# Fonts (optional build input; required for releases): see craftrules standards/fonts.md.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-unknown-unknown
@@ -221,13 +255,17 @@ jobs:
- name: Install wasm-bindgen CLI
run: |
# Must match the wasm-bindgen version in Cargo.lock exactly (cargo xtask web checks it).
command -v wasm-bindgen >/dev/null && wasm-bindgen --version | grep -q "$WASM_BINDGEN_VERSION" \
|| cargo install wasm-bindgen-cli --version "$WASM_BINDGEN_VERSION" --locked
if ! { command -v wasm-bindgen >/dev/null && wasm-bindgen --version | grep -q "$WASM_BINDGEN_VERSION"; }; then
cargo install wasm-bindgen-cli --version "$WASM_BINDGEN_VERSION" --locked
fi
wasm-bindgen --version
- name: Apply version override
if: inputs.version != ''
run: cargo xtask version set "$LIGHTCRAFT_VERSION"
- name: Build and zip
env:
CRAFT_FONTS_DIR: ${{ github.workspace }}/craft-fonts
CRAFT_FONTS_REQUIRED: "1"
run: packaging/web/package.sh
- uses: actions/upload-artifact@v4
with:
+95
View File
@@ -0,0 +1,95 @@
# Windows on ARM: package the ARM64 release the way release.yml does (cross-compiled on an x64
# runner, unsigned here), then install that MSI on a Windows 11 ARM64 runner, check the installed
# binaries are ARM64, run the CLI natively, and uninstall. (A native ARM64 `cargo test` job is left
# out for now: PrintCraft saw an intermittent access violation in egui UI tests there.)
name: Windows ARM64
on:
push:
branches: [main]
pull_request:
paths:
- Cargo.toml
- Cargo.lock
- "**/Cargo.toml"
- "**/build.rs"
- .github/workflows/windows-arm64.yml
- packaging/windows/**
schedule:
- cron: "41 4 * * 1" # weekly, so drift in the runner images shows up
workflow_dispatch:
concurrency:
group: windows-arm64-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
jobs:
package:
name: package (cross-compiled on x64)
runs-on: windows-latest
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@89b12181fb390509a0842a86cc55eeb8eb928c1d # stable
with:
targets: aarch64-pc-windows-msvc
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
key: windows-arm64-package
- name: Install WiX
shell: pwsh
run: dotnet tool install --global wix --version 5.0.2
- name: Build and package (unsigned)
shell: pwsh
env:
DIST: ${{ github.workspace }}\dist\arm64
run: ./packaging/windows/package.ps1 -Arch arm64
- uses: actions/upload-artifact@v4
with:
name: lightcraft-windows-arm64
path: dist/arm64/*
if-no-files-found: error
retention-days: 7
install:
name: install and run on ARM64
needs: package
runs-on: windows-11-arm
timeout-minutes: 30
steps:
- uses: actions/download-artifact@v4
with:
name: lightcraft-windows-arm64
path: dist
- name: Install the MSI, check it, run it, uninstall it
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
$msi = (Get-ChildItem dist/*.msi | Select-Object -First 1).FullName
$p = Start-Process msiexec.exe -ArgumentList '/i', "`"$msi`"", '/qn', '/l*v', 'install.log' -Wait -PassThru
if ($p.ExitCode -ne 0) { Get-Content install.log -Tail 60; throw "msiexec /i exited $($p.ExitCode)" }
$dir = Join-Path $env:ProgramFiles 'LightCraft'
foreach ($exe in 'lightcraft.exe', 'lightcraft-cli.exe') {
$path = Join-Path $dir $exe
if (-not (Test-Path $path)) { throw "$exe not installed in $dir" }
$bytes = [System.IO.File]::ReadAllBytes($path)
$machine = [BitConverter]::ToUInt16($bytes, [BitConverter]::ToInt32($bytes, 0x3C) + 4)
if ($machine -ne 0xAA64) { throw "$exe is for machine 0x$('{0:X}' -f $machine), not ARM64" }
Write-Output "ok $path (ARM64)"
}
# The installed CLI runs natively on ARM64.
$cli = Join-Path $dir 'lightcraft-cli.exe'
$out = & $cli --version 2>&1 | Out-String
if ($LASTEXITCODE -ne 0) { Write-Output $out; throw "lightcraft-cli --version exited $LASTEXITCODE" }
Write-Output "ok lightcraft-cli --version: $($out.Trim())"
$p = Start-Process msiexec.exe -ArgumentList '/x', "`"$msi`"", '/qn' -Wait -PassThru
if ($p.ExitCode -ne 0) { throw "msiexec /x exited $($p.ExitCode)" }
if (Test-Path (Join-Path $dir 'lightcraft.exe')) { throw 'uninstall left lightcraft.exe behind' }
Write-Output 'ok uninstalled'
+1
View File
@@ -7,3 +7,4 @@ plan/
/log
.DS_Store
.claude/
/craft-fonts
+12 -1
View File
@@ -62,10 +62,21 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
## Assets: icons, images, fonts (ABSOLUTE RULE — never violate)
- **Never use any iconography, image, artwork, font, sound or other asset from Adobe products** (no Lightroom/Creative Cloud icons, no screenshots, no presets/profiles/LUTs, no UI bitmaps — not even as a temporary placeholder or "reference copy"). Observing Adobe's UI to imitate *layout and behaviour* is allowed; copying or tracing its assets is not.
- **This includes Adobe's open-licensed assets**: no Source Sans/Serif/Code or Source Han fonts, no Adobe Fonts, no
Adobe-published icon sets, sample photos, colour profiles or LUTs — even when OFL/MIT. The UI font is Inter (OFL).
Adobe-published icon sets, sample photos, colour profiles or LUTs — even when OFL/MIT. The UI font is Inter (OFL);
Japanese fonts come from craft-fonts (below).
- Every asset in the repository must be one of: **our own original work** (e.g. icons drawn in code as vectors, procedurally generated demo photos), **public domain / CC0**, **Creative Commons** (CC-BY / CC-BY-SA with attribution honoured), **OFL** (fonts), or **permissive open-source** (MIT/Apache-2.0/BSD/ISC) — or contributed by a person who created the asset and licenses it openly.
- **Exception: `docs/brand/`.** The ArtCraft name, wordmark and logos there are ArtCraft Team trademarks, not open source and not covered by LightCraft's MIT OR Apache-2.0 licence (`LICENSE-MIT`, `LICENSE-APACHE`, `NOTICE`); their terms are in `docs/brand/LICENSE-brand.txt`. Use them only unmodified and never redraw, recolour or derive from them.
- **Every asset must have an entry in `assets/ATTRIBUTION.md`** (path, title, author/creator, source URL or "original work", licence, date added, modifications) and its licence text when required (e.g. `assets/fonts/OFL-*.txt`). Add the entry in the same commit as the asset. Assets without an attribution entry must not be committed.
- **Fonts live in [storytold/craft-fonts](https://github.com/storytold/craft-fonts), never in this repo.** Don't commit
font files here (Inter, already in `assets/fonts/`, is the one exception); add new fonts to craft-fonts. LightCraft
uses it as the optional build input `CRAFT_FONTS_DIR`: `git clone https://github.com/storytold/craft-fonts ../craft-fonts`
then `CRAFT_FONTS_DIR=../craft-fonts cargo run -p lightcraft` (or any cargo/xtask command). `crates/engine/build.rs`
embeds the manifest's fonts as `lightcraft_engine::CRAFT_FONTS` (wasm32: BIZ UDPGothic Regular only); the UI
(`theme::font_definitions`) and the export watermark renderer use its Japanese faces as fallbacks after Inter. Unset,
`CRAFT_FONTS` is empty: everything builds, tests and runs, but Japanese text has no glyphs. Releases always build
with it (`release.yml`, `CRAFT_FONTS_REQUIRED=1`) and ship the fonts' OFL licences. Tests that need these fonts skip
without it; the FreeBSD CI job runs them with it. Rules: craftrules
[`standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md).
- Icons drawn in code (e.g. `crates/ui-egui/src/icons.rs`) are original work and are recorded in `assets/ATTRIBUTION.md` as such; do not trace them from Adobe icons.
- Demo/test images: generated procedurally by `lightcraft-scenes`, or CC0 downloads kept in the gitignored `corpus/` with their source recorded. Screenshots of Adobe apps live only in the gitignored `plan/` and are never committed or published.
- **Enforced:** `cargo xtask assets` (in `ci`) fails when an image/icon/font/sound/video/raw/ICC/XMP file is not matched
+12 -1
View File
@@ -62,10 +62,21 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
## Assets: icons, images, fonts (ABSOLUTE RULE — never violate)
- **Never use any iconography, image, artwork, font, sound or other asset from Adobe products** (no Lightroom/Creative Cloud icons, no screenshots, no presets/profiles/LUTs, no UI bitmaps — not even as a temporary placeholder or "reference copy"). Observing Adobe's UI to imitate *layout and behaviour* is allowed; copying or tracing its assets is not.
- **This includes Adobe's open-licensed assets**: no Source Sans/Serif/Code or Source Han fonts, no Adobe Fonts, no
Adobe-published icon sets, sample photos, colour profiles or LUTs — even when OFL/MIT. The UI font is Inter (OFL).
Adobe-published icon sets, sample photos, colour profiles or LUTs — even when OFL/MIT. The UI font is Inter (OFL);
Japanese fonts come from craft-fonts (below).
- Every asset in the repository must be one of: **our own original work** (e.g. icons drawn in code as vectors, procedurally generated demo photos), **public domain / CC0**, **Creative Commons** (CC-BY / CC-BY-SA with attribution honoured), **OFL** (fonts), or **permissive open-source** (MIT/Apache-2.0/BSD/ISC) — or contributed by a person who created the asset and licenses it openly.
- **Exception: `docs/brand/`.** The ArtCraft name, wordmark and logos there are ArtCraft Team trademarks, not open source and not covered by LightCraft's MIT OR Apache-2.0 licence (`LICENSE-MIT`, `LICENSE-APACHE`, `NOTICE`); their terms are in `docs/brand/LICENSE-brand.txt`. Use them only unmodified and never redraw, recolour or derive from them.
- **Every asset must have an entry in `assets/ATTRIBUTION.md`** (path, title, author/creator, source URL or "original work", licence, date added, modifications) and its licence text when required (e.g. `assets/fonts/OFL-*.txt`). Add the entry in the same commit as the asset. Assets without an attribution entry must not be committed.
- **Fonts live in [storytold/craft-fonts](https://github.com/storytold/craft-fonts), never in this repo.** Don't commit
font files here (Inter, already in `assets/fonts/`, is the one exception); add new fonts to craft-fonts. LightCraft
uses it as the optional build input `CRAFT_FONTS_DIR`: `git clone https://github.com/storytold/craft-fonts ../craft-fonts`
then `CRAFT_FONTS_DIR=../craft-fonts cargo run -p lightcraft` (or any cargo/xtask command). `crates/engine/build.rs`
embeds the manifest's fonts as `lightcraft_engine::CRAFT_FONTS` (wasm32: BIZ UDPGothic Regular only); the UI
(`theme::font_definitions`) and the export watermark renderer use its Japanese faces as fallbacks after Inter. Unset,
`CRAFT_FONTS` is empty: everything builds, tests and runs, but Japanese text has no glyphs. Releases always build
with it (`release.yml`, `CRAFT_FONTS_REQUIRED=1`) and ship the fonts' OFL licences. Tests that need these fonts skip
without it; the FreeBSD CI job runs them with it. Rules: craftrules
[`standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md).
- Icons drawn in code (e.g. `crates/ui-egui/src/icons.rs`) are original work and are recorded in `assets/ATTRIBUTION.md` as such; do not trace them from Adobe icons.
- Demo/test images: generated procedurally by `lightcraft-scenes`, or CC0 downloads kept in the gitignored `corpus/` with their source recorded. Screenshots of Adobe apps live only in the gitignored `plan/` and are never committed or published.
- **Enforced:** `cargo xtask assets` (in `ci`) fails when an image/icon/font/sound/video/raw/ICC/XMP file is not matched
Generated
+23 -22
View File
@@ -2073,7 +2073,7 @@ dependencies = [
[[package]]
name = "lightcraft"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"eframe",
"egui",
@@ -2091,7 +2091,7 @@ dependencies = [
[[package]]
name = "lightcraft-catalog"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"crc32fast",
"lightcraft-develop",
@@ -2105,7 +2105,7 @@ dependencies = [
[[package]]
name = "lightcraft-cli"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"dhat",
"lightcraft-codecs",
@@ -2120,7 +2120,7 @@ dependencies = [
[[package]]
name = "lightcraft-codecs"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"bytemuck",
"image",
@@ -2145,7 +2145,7 @@ dependencies = [
[[package]]
name = "lightcraft-color"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"lightcraft-geom",
"proptest",
@@ -2154,7 +2154,7 @@ dependencies = [
[[package]]
name = "lightcraft-develop"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"lightcraft-color",
"lightcraft-geom",
@@ -2165,7 +2165,7 @@ dependencies = [
[[package]]
name = "lightcraft-engine"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"ab_glyph",
"libc",
@@ -2193,7 +2193,7 @@ dependencies = [
[[package]]
name = "lightcraft-geom"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"proptest",
"serde",
@@ -2201,7 +2201,7 @@ dependencies = [
[[package]]
name = "lightcraft-gpu"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"bytemuck",
"lightcraft-color",
@@ -2217,7 +2217,7 @@ dependencies = [
[[package]]
name = "lightcraft-mcp"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"lightcraft-codecs",
"lightcraft-engine",
@@ -2228,7 +2228,7 @@ dependencies = [
[[package]]
name = "lightcraft-merge"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"lightcraft-codecs",
"lightcraft-color",
@@ -2244,7 +2244,7 @@ dependencies = [
[[package]]
name = "lightcraft-meta"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"lightcraft-geom",
"lightcraft-tiff",
@@ -2256,7 +2256,7 @@ dependencies = [
[[package]]
name = "lightcraft-pipeline"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"image",
"lightcraft-color",
@@ -2272,7 +2272,7 @@ dependencies = [
[[package]]
name = "lightcraft-preview"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"jpeg-encoder",
"lightcraft-raster",
@@ -2284,7 +2284,7 @@ dependencies = [
[[package]]
name = "lightcraft-raster"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"libc",
"lightcraft-color",
@@ -2296,7 +2296,7 @@ dependencies = [
[[package]]
name = "lightcraft-raw"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"jpeg-encoder",
"lightcraft-color",
@@ -2315,7 +2315,7 @@ dependencies = [
[[package]]
name = "lightcraft-scenes"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"image",
"lightcraft-color",
@@ -2326,11 +2326,11 @@ dependencies = [
[[package]]
name = "lightcraft-sysmem"
version = "0.2.0"
version = "0.2.1"
[[package]]
name = "lightcraft-tiff"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"proptest",
"serde",
@@ -2339,7 +2339,7 @@ dependencies = [
[[package]]
name = "lightcraft-ui-egui"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"egui",
"lightcraft-catalog",
@@ -2359,8 +2359,9 @@ dependencies = [
[[package]]
name = "lightcraft-web"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"crc32fast",
"eframe",
"egui",
"js-sys",
@@ -5127,7 +5128,7 @@ checksum = "e450f9b2ed1dff33c94c12589a87338689467b9c4f5d8a5710bd09a847d2c8a7"
[[package]]
name = "xtask"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"brotli",
"flate2",
+1 -1
View File
@@ -3,7 +3,7 @@ resolver = "3"
members = ["crates/*", "apps/*", "xtask"]
[workspace.package]
version = "0.2.0"
version = "0.2.1"
edition = "2024"
license = "MIT OR Apache-2.0"
rust-version = "1.90"
+5 -3
View File
@@ -15,6 +15,8 @@ list (author, source, license) is in assets/ATTRIBUTION.md.
Inter (assets/fonts/Inter-*.ttf) — SIL Open Font License 1.1. See assets/fonts/OFL-Inter.txt.
Showcase photographs by Ansel Adams, Dorothea Lange and NASA (docs/images/) — public domain. See assets/ATTRIBUTION.md.
BIZ UDMincho: Copyright 2022 The BIZ UDMincho Project Authors
(https://github.com/googlefonts/morisawa-biz-ud-mincho). SIL Open Font License 1.1;
see assets/fonts/OFL-BIZUDMincho.txt.
Builds made with craft-fonts (the optional CRAFT_FONTS_DIR build input; all official releases)
also embed the fonts listed in https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md
(the Japanese fonts BIZ UDPGothic, BIZ UDMincho and Shippori Mincho; the web build embeds only BIZ
UDPGothic Regular), under the SIL Open Font License 1.1. Release packages include each font's
licence as OFL-<family>.txt.
+15 -2
View File
@@ -334,6 +334,16 @@ cargo run --release -p lightcraft-cli -- render photo.jpg -o out.jpg --set light
cargo xtask ci # fmt, clippy, tests, layering, wasm checks
```
**Japanese text** needs the shared font repo, an optional build input (official releases always include it):
```sh
git clone https://github.com/storytold/craft-fonts ../craft-fonts
CRAFT_FONTS_DIR=../craft-fonts cargo run --release -p lightcraft
```
Without it LightCraft builds and runs the same, but Japanese text has no glyphs. Fonts are never committed to this
repo; see [craftrules `standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md).
The web build needs the `wasm32-unknown-unknown` target and the matching `wasm-bindgen` CLI
(`cargo xtask web` prints the exact install command); see [docs/web.md](docs/web.md).
@@ -356,7 +366,8 @@ Humans and agents follow the same rules, so read [AGENTS.md](AGENTS.md) first. T
work from public specs and black-box observation.
- **No Adobe assets, ever:** no icons, screenshots, presets, profiles, LUTs or fonts from Adobe products. Every
image, icon and font in the repo is original, public domain, Creative Commons, OFL or permissively licensed, and has
an entry in [assets/ATTRIBUTION.md](assets/ATTRIBUTION.md) added in the same commit.
an entry in [assets/ATTRIBUTION.md](assets/ATTRIBUTION.md) added in the same commit. New fonts go to
[storytold/craft-fonts](https://github.com/storytold/craft-fonts), not here.
- **Pure Rust**, enforced crate layering, everything is a command, and `cargo xtask ci` green before every commit
(one task id per commit).
- **Never crash.** Non-test code returns errors instead of panicking: no `unwrap()`, `expect()`, `panic!` or
@@ -420,7 +431,9 @@ with its author, source and license in [assets/ATTRIBUTION.md](assets/ATTRIBUTIO
Showcase photographs are public-domain works, used via Wikimedia Commons: Ansel Adams, *The Tetons and the Snake River*
(1942, U.S. National Archives); Dorothea Lange, *Migrant Mother* (1936, Library of Congress); Bill Anders / NASA,
*Earthrise* (1968); NASA, *The Blue Marble* (1972). The demo library is procedurally generated by LightCraft. UI font:
Inter (SIL OFL 1.1). All icons are original.
Inter (SIL OFL 1.1). Builds made with [craft-fonts](https://github.com/storytold/craft-fonts) (all official releases)
also embed its Japanese fonts (BIZ UDPGothic, BIZ UDMincho, Shippori Mincho; SIL OFL 1.1), listed in its
[ATTRIBUTION.md](https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md). All icons are original.
The ArtCraft name, wordmark and logos in [`docs/brand/`](docs/brand/) are trademarks of the
ArtCraft Team and are not covered by this license. They may be used only unmodified, and only as
+13 -10
View File
@@ -24,19 +24,19 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
|---|---:|---|---|
| **Feature checklist** | 79% | P0 core and P1 nearly complete: import (Add / Copy / Move, templates, devices), library, grid/loupe/compare/survey, every Edit slider, curves, colour grading, masking tools, crop/Upright, heal/clone, presets/profiles, versions/history, sync, export, menus, shortcuts | P1: lens-profile database, content-aware fill (patch synthesis), video playback/trim |
| **RAW coverage** (formats people shoot) | ~50% | DNG (all kinds), CR2, ARW, NEF (uncompressed + Huffman lossless/lossy), uncompressed RAF/ORF, packed RW2, PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed RAF/ORF, RW2 v4, Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. Per-model verification is thin (~40 corpus files vs >1,000 models) |
| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No camera colour calibration**: non-DNG raws use a neutral matrix, so colour is muted/inaccurate. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW has a guarded per-file embedded-JPEG colour estimate (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
| **AI & computational** | ~15–20% | Assisted culling (focus, bursts), auto tone, HDR/panorama merge; subject/sky/background masks as classical heuristics | Real segmentation masks (subject, sky, people, objects, landscape, depth), AI denoise, super resolution, lens blur, generative remove, faces/people, natural-language search. **Blocked on a model strategy** (licensable weights or our own training; pure-Rust inference is feasible) |
| **Workflow & library** | ~85% (single machine) | Robust catalog (journal + snapshots, background compaction, crash-tested), 85k-photo libraries stay responsive, Local browsing with automatic cleanup, XMP interop, keywords, smart albums, Move import | Opening an 85k library takes 1.7–4.7 s; no cloud sync (out of scope), no tablet companion (#74, roadmap), shared albums, publish services, tethering |
| **Classic modules** | ~30% | Geotagging from GPX track logs, soft proofing (partial), slideshow (basic) | **Map view, Book, Print, Slideshow module, Web, publish services**: ~40 tracker rows ⬜ |
| **HDR & video** | 0% | | HDR edit/display/export; video play/trim/edit/export |
| **Platform & robustness** | ~70% | macOS native; Windows/Linux builds; web via WASM; no-panic lints workspace-wide, `unsafe` confined to `crates/sysmem`; failed saves are reported; GPU errors fall back to CPU | Windows installer UI unverified on Windows (PR #79); GPU path proven only on Apple + user reports; no localisation; accessibility partial; headless UI tests time out under machine load |
| **Platform & robustness** | ~70% | macOS native; Windows/Linux builds; web via WASM; no-panic lints workspace-wide, `unsafe` confined to `crates/sysmem`; failed saves are reported; GPU errors fall back to CPU | Windows installer UI unverified on Windows (PR #79); GPU path proven only on Apple + user reports; Japanese/English UI (see docs/localization-ja.md); remaining technical errors and other languages; accessibility partial; headless UI tests time out under machine load |
### By kind of user
| User | Readiness | What blocks them |
|---|---:|---|
| JPEG / DNG shooter, single machine | ~85% | Fidelity polish, AI masks |
| Nikon / Sony / older-Canon raw shooter | ~65% | Camera colour (muted colour on every photo) |
| Nikon / Sony / older-Canon raw shooter | ~65% | Camera colour fidelity and coverage (ARW preview estimates are only a starting point) |
| Canon CR3 / Fujifilm / Olympus shooter | ~35% | Their raws open as embedded previews only |
| Lightroom Classic power user | ~45% | Print, Book, Map, publish, tethering |
| Relies on AI (masks, denoise) | ~25% | No segmentation or AI denoise models |
@@ -46,7 +46,8 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
Priorities, in order. Each points at tracker rows in [`docs/parity.md`](docs/parity.md) → *Top gaps*.
1. **Camera colour calibration of our own** (LR-PROF-CAMERACOLOR, P0): fit each camera to its own embedded JPEG, use
matrices the files carry themselves, then chart shots. Raises colour quality for every non-DNG raw at once.
matrices the files carry themselves, then chart shots. Sony ARW's file-local fit (matrix + tone curve from its own
JPEG) is the first step; generalise it to the other makes' raws (NEF, RW2, PEF, ORF…), then validate fidelity.
2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): **CR3** first, then compressed RAF / ORF, RW2 v4, NEF
lossy-after-split, sRAW. Decided 2026-10-05: write our own decoders from prose descriptions (never decoder source,
no LGPL dependency); compressed NEF (#86) is the template.
@@ -81,12 +82,12 @@ hardening (#78), copyright metadata (#51), GPX geotagging (#60), import tag help
| M8 | Heal / Remove | content-aware remove (PatchMatch), heal, clone, brush spots, visualize spots, red/pet eye | 6–10 | 🚧 (heal, clone, auto source, visualize spots, red/pet eye ✅; PatchMatch remove ⬜) |
| M9 | Presets, profiles, versions, sync | preset browser + amount, create/import presets, profile browser, versions, history, copy/paste/sync settings | 5–8 | ✅ |
| M10 | Export & share | export dialog (JPEG/PNG/TIFF/DNG/AVIF/JXL/original), sizing, sharpening, metadata, watermark, naming, batch jobs, XMP sidecars, HDR export | 6–10 | 🚧 (all formats incl. DNG/original, sizing, presets, background jobs ✅; JXL encode, HDR export ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 packed, PEF, ORF uncompressed ✅; **camera colour calibration** ⬜ (top priority), CR3, compressed ORF/RAF ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 packed, PEF, ORF uncompressed ✅; **camera colour calibration** 🚧 (guarded ARW preview fitting; measured database still missing), CR3, compressed ORF/RAF ⬜) |
| M12 | AI & smart features | subject/sky/background/people/object masks, semantic search, faces/People (permissively licensed models, pure-Rust inference) | 20–40 | ⬜ |
| M13 | Merge | HDR merge (deghost), panorama (projections, boundary warp, fill edges), HDR panorama | 10–15 | ✅ |
| M14 | Video | import/playback/trim via FilmCraft crates, global edits + presets on video, video export | 6–10 | ⬜ |
| M15 | Classic modules | Map, Book, Slideshow, Print, Web; smart collections, stacks, virtual copies, publish services, tethering | 25–40 | 🚧 (smart albums, stacks, virtual copies, compare/survey ✅; Map/Book/Slideshow/Print/Web ⬜) |
| M16 | 1.0 polish | preferences, shortcut editor, accessibility, localization, packaging (dmg/msi/AppImage/web), hardening | 10–20 | 🚧 (settings, keyboard shortcuts sheet, packaging basics ✅; accessibility, localisation ⬜) |
| M16 | 1.0 polish | preferences, shortcut editor, accessibility, localization, packaging (dmg/msi/AppImage/web), hardening | 10–20 | 🚧 (settings, keyboard shortcuts sheet, packaging basics ✅; Japanese/English localisation 🟡; accessibility and other locales ⬜) |
## Parity estimate (feature count updated 2026-10-05; effort estimate from 2026-10-02)
@@ -160,7 +161,8 @@ The milestone estimates in the table above were made before work started and are
## Raw format coverage and known gaps
Decoded (CC0 corpus from raw.pixls.us, `cargo xtask corpus --download`, `crates/raw/tests/corpus.rs`): DNG (uncompressed,
LJ92, lossy JPEG / Smart Previews, Deflate, float, linear), CR2, ARW (uncompressed, ARW2, LJ92), NEF/NRW uncompressed and Huffman-compressed (lossless, lossy type 1/2, 12/14-bit), RAF uncompressed (Bayer and
LJ92, lossy JPEG / Smart Previews, Deflate, float, linear), CR2, ARW (uncompressed, ARW2, LJ92; as-shot white balance and black level of pre-2017 bodies from the enciphered
maker-note `Tag2010` and the encrypted `SR2SubIFD`, both recovered by black-box analysis, `crates/raw/src/vendor/arw.rs`), NEF/NRW uncompressed and Huffman-compressed (lossless, lossy type 1/2, 12/14-bit), RAF uncompressed (Bayer and
X-Trans), RW2 packed 12/14-bit, PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Every
supported container also yields its embedded JPEG preview (CR3 too), and the engine shows that preview for raw variants
it can't decode yet.
@@ -175,7 +177,7 @@ Not decoded yet — preview only (no permissively licensed description; black-bo
from the same-colour pixel two to the left), the reconstruction rule for scales 2/3 is not established.
- **Olympus compressed ORF**, **Fujifilm compressed RAF**, **Canon CR3/CRX** (M11.1), **Canon sRAW/mRAW**, lossy DNG.
**Camera colour matrices:** non-DNG raws use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
**Camera colour matrices:** ARW files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
`matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer
matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax/Panasonic equivalents) and our
own chart-based calibration (M11.4). Adobe matrices are never used.
@@ -193,8 +195,9 @@ own chart-based calibration (M11.4). Adobe matrices are never used.
## Japanese interface and text watermarks
English/Japanese interface language is persisted in UI state. Core menus have Japanese
translations; untranslated panels and dialogs retain English. BIZ UDMincho is bundled
under OFL for Japanese glyph coverage without system fonts. Text watermarks now accept
translations; untranslated panels and dialogs retain English. Japanese glyphs (UI: BIZ UDPGothic;
watermarks: BIZ UDMincho) come from storytold/craft-fonts, embedded by builds made with the
optional `CRAFT_FONTS_DIR` input (all releases), so no system fonts are needed. Text watermarks now accept
`vertical: true` in export JSON/presets and expose an orientation selector. Japanese
characters stay upright in top-to-bottom columns, with newlines starting columns to the
left. This is basic lettering, without tate-chu-yoko, ruby, kinsoku, or general vertical
+14 -3
View File
@@ -91,6 +91,17 @@ USAGE:
#[global_allocator]
static ALLOC: dhat::Alloc = dhat::Alloc;
/// Why `--library DIR` can't be opened; for a library open in another program, how to work with
/// that one instead (issue #99).
fn library_error(dir: &str, e: lightcraft_engine::EngineError) -> String {
match e {
lightcraft_engine::EngineError::LibraryInUse(why) => format!(
"{dir}: {why}\nTo work with the library while the app has it open, start the app with `--control PORT` and use `lightcraft-cli mcp --connect 127.0.0.1:PORT`."
),
e => format!("{dir}: {e}"),
}
}
fn main() -> ExitCode {
// `--features dhat-heap`: count allocations; the profile is written when `_heap` drops
// (LIGHTCRAFT_DHAT_FILE, default dhat-heap.json).
@@ -182,7 +193,7 @@ fn mcp(args: &[String]) -> Result<(), String> {
let mut h = match &library {
Some(dir) => {
let mut h = Headless::default();
let r = h.session.open_library(dir, demo).map_err(|e| e.to_string())?;
let r = h.session.open_library(dir, demo).map_err(|e| library_error(dir, e))?;
eprintln!("lightcraft-cli mcp: opened library {dir} ({r:?})");
h
}
@@ -419,7 +430,7 @@ fn run(args: &[String]) -> Result<(), String> {
let mut h = match &library {
Some(dir) => {
let mut h = Headless::default();
h.session.open_library(dir, demo).map_err(|e| e.to_string())?;
h.session.open_library(dir, demo).map_err(|e| library_error(dir, e))?;
h
}
None if demo => Headless::demo(),
@@ -586,7 +597,7 @@ fn snapshot(args: &[String]) -> Result<(), String> {
let mut session = match &library {
Some(dir) => {
let mut s = Session::new().with_fs();
s.open_library(dir, false).map_err(|e| format!("{dir}: {e}"))?;
s.open_library(dir, false).map_err(|e| library_error(dir, e))?;
s
}
None if files.is_empty() => Session::with_demo().with_fs(),
+64
View File
@@ -183,6 +183,42 @@ fn snapshot_subcommand_renders_the_ui_headlessly() {
assert_eq!((d.width, d.height), (640, 480));
}
/// Issue #136: with the GPU switched off from the environment the UI starts and renders on the CPU,
/// without creating a GPU device (no driver is loaded), and says why.
#[test]
fn snapshot_starts_without_a_gpu() {
let script = tmp("nogpu.jsonl");
std::fs::write(
&script,
format!(
"{}\n{}\n{}\n",
json!({"method": "ui.set", "params": {"view": "photoGrid"}}),
json!({"method": "engine.execute", "params": {"command": "app.gpu"}}),
json!({"method": "ui.screenshot"}),
),
)
.unwrap();
for (var, value, reason) in [("LIGHTCRAFT_GPU", "0", "LIGHTCRAFT_GPU=0"), ("LIGHTCRAFT_GPU_BACKEND", "off", "LIGHTCRAFT_GPU_BACKEND=off")] {
let out = tmp(&format!("nogpu-{var}.png"));
let o = Command::new(BIN)
.env_remove("LIGHTCRAFT_GPU")
.env_remove("LIGHTCRAFT_GPU_BACKEND")
.env(var, value)
.args(["snapshot", "--demo", "--script", script.to_str().unwrap(), "-o", out.to_str().unwrap(), "--size", "480x320"])
.output()
.unwrap();
assert!(o.status.success(), "{var}: {}", String::from_utf8_lossy(&o.stderr));
let replies: Vec<Value> = String::from_utf8_lossy(&o.stdout).lines().map(|l| serde_json::from_str(l).unwrap()).collect();
assert!(replies.iter().all(|r| r["ok"] == true), "{replies:?}");
let gpu = &replies[1]["result"];
assert_eq!(gpu["available"], false, "{var}: {gpu}");
assert_eq!(gpu["adapter"], Value::Null, "{var}: no device was created: {gpu}");
assert!(gpu["reason"].as_str().is_some_and(|r| r.contains(reason)), "{var}: {gpu}");
let d = lightcraft_codecs::decode(&std::fs::read(&out).unwrap(), Default::default()).unwrap();
assert_eq!((d.width, d.height), (480, 320));
}
}
fn run_cli(args: &[&str], stdin: Option<&str>) -> (bool, Vec<Value>, String) {
let mut child = Command::new(BIN).arg("run").args(args).stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::piped()).spawn().unwrap();
if let Some(text) = stdin {
@@ -260,3 +296,31 @@ fn devices_are_listed_and_imported_from() {
let line: Value = serde_json::from_slice(o.stdout.split(|b| *b == b'\n').next().unwrap()).unwrap();
assert_eq!(line["result"]["candidates"].as_array().map(Vec::len), Some(1), "{line}");
}
/// Issue #99: a library open in one process (here `mcp --library`) is refused by a second one,
/// with who has it and how to drive the running app instead; free again once the first exits.
#[test]
fn a_library_open_in_another_process_is_refused() {
let lib = tmp("locked-lib");
let _ = std::fs::remove_dir_all(&lib);
let lib_s = lib.to_str().unwrap();
let mut holder =
Command::new(BIN).args(["mcp", "--library", lib_s]).stdin(Stdio::piped()).stdout(Stdio::null()).stderr(Stdio::piped()).spawn().unwrap();
let mut err = BufReader::new(holder.stderr.take().unwrap());
let mut line = String::new();
while !line.contains("opened library") {
line.clear();
assert!(err.read_line(&mut line).unwrap() > 0, "mcp exited before opening the library");
}
let (ok, _, stderr) = run_cli(&["--library", lib_s, "library.info"], None);
assert!(!ok);
assert!(stderr.contains("already open in lightcraft-cli") && stderr.contains(&format!("process {}", holder.id())), "{stderr}");
assert!(stderr.contains("mcp --connect"), "{stderr}");
drop(holder.stdin.take()); // EOF: the server exits and lets go of the library
assert!(holder.wait().unwrap().success());
let (ok, lines, stderr) = run_cli(&["--library", lib_s, "library.info"], None);
assert!(ok, "{stderr}");
assert_eq!(lines[0]["ok"], true);
let _ = std::fs::remove_dir_all(&lib);
}
+1
View File
@@ -21,6 +21,7 @@ lightcraft-preview = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
log = { workspace = true }
crc32fast = { workspace = true }
[target.'cfg(target_arch = "wasm32")'.dependencies]
eframe = { workspace = true, features = ["default_fonts", "glow"] }
+33 -10
View File
@@ -232,6 +232,18 @@ impl Backend {
}
}
/// Is there a file at `path`?
pub async fn exists(&self, path: &str) -> Result<bool, String> {
match self {
Backend::Opfs(root) => Ok(Self::opfs_file(root, path, false).await?.is_some()),
Backend::Idb(db) => {
let (_tx, store) = Self::idb_store(db, false)?;
let n = idb_req(&store.count_with_key(&path.into()).map_err(err)?).await?;
Ok(n.as_f64().is_some_and(|n| n > 0.0))
}
}
}
pub async fn remove(&self, path: &str) -> Result<(), String> {
match self {
Backend::Opfs(root) => {
@@ -309,14 +321,25 @@ async fn dir_keys(dir: &FileSystemDirectoryHandle) -> Result<Vec<String>, String
Ok(out)
}
/// Ask the browser not to evict our storage under pressure (best effort; fire and forget).
pub fn request_persistence() {
if let Some(sm) = storage_manager()
&& let Ok(p) = sm.persist()
{
wasm_bindgen_futures::spawn_local(async move {
let granted = JsFuture::from(p).await.ok().is_some_and(|v| v.is_truthy());
log::info!("lightcraft: persistent storage {}", if granted { "granted" } else { "not granted (best effort)" });
});
}
/// Ask the browser not to evict our storage under pressure; `on_result(granted)` once known
/// (`false` also when the browser has no storage manager).
pub fn request_persistence(on_result: impl FnOnce(bool) + 'static) {
let Some(sm) = storage_manager() else {
on_result(false);
return;
};
wasm_bindgen_futures::spawn_local(async move {
// already persistent: nothing to ask
let already = match sm.persisted() {
Ok(p) => JsFuture::from(p).await.ok().is_some_and(|v| v.is_truthy()),
Err(_) => false,
};
let granted = already
|| match sm.persist() {
Ok(p) => JsFuture::from(p).await.ok().is_some_and(|v| v.is_truthy()),
Err(_) => false,
};
log::info!("lightcraft: persistent storage {}", if granted { "granted" } else { "not granted" });
on_result(granted);
});
}
+306
View File
@@ -0,0 +1,306 @@
//! Library backups for the browser build: a plain zip (entries stored, not compressed — photos
//! don't shrink) of the library files and every stored original, built one entry at a time so
//! only one file is in wasm memory at once, and read back the same way for a restore.
//!
//! Layout of a backup:
//! - `library/catalog.snap`, `library/catalog.log`, `library/presets.json`, … (see
//! [`crate::files::LIBRARY_FILES`]);
//! - `originals/<content hash>/<file name>`: each imported photo's original bytes, under its own
//! name so an unzipped backup is browsable;
//! - `README.txt`.
//!
//! Restoring never deletes anything: the library files go to a new library folder in browser
//! storage, originals that aren't stored yet are added (they're named by content, so existing
//! ones are identical), and only then does a small pointer file ([`ACTIVE_LIBRARY`]) switch to
//! the restored library. The library that was there before stays in storage.
//!
//! No zip64: a backup holds at most [`MAX_BYTES`] and 65 535 entries.
/// Storage key of the file naming the library folder in use (absent: `library`).
pub const ACTIVE_LIBRARY: &str = "active-library";
/// The default library folder.
pub const DEFAULT_LIBRARY_DIR: &str = "library";
/// Largest backup (offsets are 32-bit without zip64).
pub const MAX_BYTES: u64 = 0xFFFF_FFFF - (64 << 20);
const LOCAL_SIG: u32 = 0x0403_4b50;
const CENTRAL_SIG: u32 = 0x0201_4b50;
const END_SIG: u32 = 0x0605_4b50;
/// General-purpose flag bit 11: names are UTF-8.
const UTF8: u16 = 0x0800;
/// 1980-01-01 00:00 in MS-DOS format (date, time).
const DOS_DATE: u16 = (1 << 5) | 1;
/// The readme inside every backup.
pub const README: &str = "LightCraft library backup (browser version)\n\n\
library/ the catalog (catalog.snap + catalog.log), presets, view state and preferences\n\
originals/ every imported photo, as originals/<content hash>/<file name>\n\n\
Restore it in LightCraft for the web with File > Restore Library from Backup...\n\
The photos in originals/ are the untouched files you imported; edits live in the catalog.\n";
/// Is `name` a safe library folder name (`library` or `library-<letters, digits, ->`)?
pub fn valid_library_dir(name: &str) -> bool {
name == DEFAULT_LIBRARY_DIR
|| name.strip_prefix("library-").is_some_and(|s| !s.is_empty() && s.len() <= 64 && s.chars().all(|c| c.is_ascii_alphanumeric() || c == '-'))
}
/// Is `h` a plausible content hash (used as a storage key segment)?
pub fn valid_hash(h: &str) -> bool {
!h.is_empty() && h.len() <= 128 && h.chars().all(|c| c.is_ascii_alphanumeric())
}
/// Zip entry name for an original.
pub fn original_entry(hash: &str, file_name: &str) -> String {
let name: String = file_name.chars().map(|c| if c == '/' || c == '\\' || c.is_control() { '_' } else { c }).collect();
let name = if name.is_empty() || name == "." || name == ".." { "original".to_string() } else { name };
format!("originals/{hash}/{name}")
}
/// Where a backup entry goes when restoring into library folder `dir`: `Some(storage key)`, or
/// `None` to skip it (README, thumbnails, anything unknown or unsafe).
pub fn restore_key(entry: &str, dir: &str) -> Option<String> {
if let Some(name) = entry.strip_prefix("library/") {
return crate::files::LIBRARY_FILES.contains(&name).then(|| format!("{dir}/{name}"));
}
let rest = entry.strip_prefix("originals/")?;
let (hash, _) = rest.split_once('/').unwrap_or((rest, ""));
valid_hash(hash).then(|| format!("originals/{hash}"))
}
fn u16le(v: &mut Vec<u8>, x: u16) {
v.extend_from_slice(&x.to_le_bytes());
}
fn u32le(v: &mut Vec<u8>, x: u32) {
v.extend_from_slice(&x.to_le_bytes());
}
struct Central {
name: String,
crc: u32,
size: u32,
offset: u32,
}
/// Writes a stored (uncompressed) zip piece by piece: [`ZipWriter::entry`] returns the local
/// header to emit before the entry's bytes, [`ZipWriter::finish`] the central directory.
#[derive(Default)]
pub struct ZipWriter {
entries: Vec<Central>,
offset: u64,
}
impl ZipWriter {
/// The local header for `name` holding `data` (emit it, then `data`).
pub fn entry(&mut self, name: &str, data: &[u8]) -> Result<Vec<u8>, String> {
let size = u32::try_from(data.len()).map_err(|_| format!("{name}: larger than 4 GB"))?;
let name_len = u16::try_from(name.len()).map_err(|_| format!("{name}: name too long"))?;
if self.entries.len() >= usize::from(u16::MAX) {
return Err("too many files for one backup".into());
}
let header_len = 30 + u64::from(name_len);
if self.offset + header_len + u64::from(size) > MAX_BYTES {
return Err("the library is too large for one backup file (4 GB)".into());
}
let offset = u32::try_from(self.offset).map_err(|_| "backup too large".to_string())?;
let crc = crc32fast::hash(data);
let mut h = Vec::with_capacity(header_len as usize);
u32le(&mut h, LOCAL_SIG);
u16le(&mut h, 20); // version needed
u16le(&mut h, UTF8);
u16le(&mut h, 0); // stored
u16le(&mut h, 0); // time
u16le(&mut h, DOS_DATE);
u32le(&mut h, crc);
u32le(&mut h, size);
u32le(&mut h, size);
u16le(&mut h, name_len);
u16le(&mut h, 0); // extra
h.extend_from_slice(name.as_bytes());
self.offset += header_len + u64::from(size);
self.entries.push(Central { name: name.to_string(), crc, size, offset });
Ok(h)
}
/// The central directory and end record (emit last).
pub fn finish(self) -> Result<Vec<u8>, String> {
let mut cd = Vec::new();
for e in &self.entries {
u32le(&mut cd, CENTRAL_SIG);
u16le(&mut cd, 20); // made by
u16le(&mut cd, 20); // needed
u16le(&mut cd, UTF8);
u16le(&mut cd, 0);
u16le(&mut cd, 0);
u16le(&mut cd, DOS_DATE);
u32le(&mut cd, e.crc);
u32le(&mut cd, e.size);
u32le(&mut cd, e.size);
u16le(&mut cd, e.name.len() as u16);
u16le(&mut cd, 0); // extra
u16le(&mut cd, 0); // comment
u16le(&mut cd, 0); // disk
u16le(&mut cd, 0); // internal attrs
u32le(&mut cd, 0); // external attrs
u32le(&mut cd, e.offset);
cd.extend_from_slice(e.name.as_bytes());
}
let cd_offset = u32::try_from(self.offset).map_err(|_| "backup too large".to_string())?;
let cd_size = u32::try_from(cd.len()).map_err(|_| "backup too large".to_string())?;
let n = self.entries.len() as u16;
let mut out = cd;
u32le(&mut out, END_SIG);
u16le(&mut out, 0);
u16le(&mut out, 0);
u16le(&mut out, n);
u16le(&mut out, n);
u32le(&mut out, cd_size);
u32le(&mut out, cd_offset);
u16le(&mut out, 0); // comment
Ok(out)
}
}
/// One entry of a zip's central directory.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ZipEntry {
pub name: String,
pub crc: u32,
pub size: u64,
/// Offset of the entry's local header.
pub header: u64,
/// 0 = stored (the only method a restore reads).
pub method: u16,
}
fn rd16(b: &[u8], at: usize) -> Option<u16> {
Some(u16::from_le_bytes(b.get(at..at + 2)?.try_into().ok()?))
}
fn rd32(b: &[u8], at: usize) -> Option<u32> {
Some(u32::from_le_bytes(b.get(at..at + 4)?.try_into().ok()?))
}
/// How many bytes from the end of a zip to read to find its end record.
pub fn tail_len(file_len: u64) -> u64 {
file_len.min(22 + 65_535)
}
/// The central directory's (offset, size) from the last [`tail_len`] bytes of a zip.
pub fn find_central(tail: &[u8]) -> Result<(u64, u64), String> {
let at = (0..tail.len().saturating_sub(21)).rev().find(|&i| rd32(tail, i) == Some(END_SIG)).ok_or("not a zip file (no end record)")?;
let size = rd32(tail, at + 12).ok_or("truncated zip")?;
let offset = rd32(tail, at + 16).ok_or("truncated zip")?;
if size == u32::MAX || offset == u32::MAX {
return Err("zip64 backups are not supported".into());
}
Ok((u64::from(offset), u64::from(size)))
}
/// Parse a central directory.
pub fn parse_central(cd: &[u8]) -> Result<Vec<ZipEntry>, String> {
let mut out = Vec::new();
let mut i = 0;
while i + 46 <= cd.len() {
if rd32(cd, i) != Some(CENTRAL_SIG) {
return Err("damaged zip directory".into());
}
let bad = || "damaged zip directory".to_string();
let method = rd16(cd, i + 10).ok_or_else(bad)?;
let crc = rd32(cd, i + 16).ok_or_else(bad)?;
let size = rd32(cd, i + 24).ok_or_else(bad)?;
let n = usize::from(rd16(cd, i + 28).ok_or_else(bad)?);
let extra = usize::from(rd16(cd, i + 30).ok_or_else(bad)?);
let comment = usize::from(rd16(cd, i + 32).ok_or_else(bad)?);
let header = rd32(cd, i + 42).ok_or_else(bad)?;
let name = cd.get(i + 46..i + 46 + n).ok_or_else(bad)?;
out.push(ZipEntry { name: String::from_utf8_lossy(name).into_owned(), crc, size: u64::from(size), header: u64::from(header), method });
i += 46 + n + extra + comment;
}
Ok(out)
}
/// Offset of an entry's data from its local header's first 30 bytes.
pub fn data_offset(entry: &ZipEntry, local: &[u8]) -> Result<u64, String> {
if rd32(local, 0) != Some(LOCAL_SIG) {
return Err(format!("{}: damaged entry", entry.name));
}
let n = rd16(local, 26).ok_or("truncated entry")?;
let extra = rd16(local, 28).ok_or("truncated entry")?;
Ok(entry.header + 30 + u64::from(n) + u64::from(extra))
}
/// Check an entry's bytes.
pub fn verify(entry: &ZipEntry, data: &[u8]) -> Result<(), String> {
if entry.method != 0 {
return Err(format!("{}: compressed entries are not supported (not a LightCraft backup?)", entry.name));
}
if data.len() as u64 != entry.size || crc32fast::hash(data) != entry.crc {
return Err(format!("{}: damaged (checksum mismatch)", entry.name));
}
Ok(())
}
/// A name for the restored library's folder.
pub fn restored_dir_name(now_ms: f64) -> String {
format!("library-restored-{}", now_ms.max(0.0) as u64)
}
#[cfg(test)]
mod tests {
use super::*;
fn build(files: &[(&str, &[u8])]) -> Vec<u8> {
let mut w = ZipWriter::default();
let mut out = Vec::new();
for (n, d) in files {
out.extend(w.entry(n, d).unwrap());
out.extend_from_slice(d);
}
out.extend(w.finish().unwrap());
out
}
#[test]
fn written_backups_read_back() {
let files: [(&str, &[u8]); 3] =
[("README.txt", README.as_bytes()), ("library/catalog.log", b"{\"seq\":1}\n"), ("originals/abc123/IMG 1.jpg", &[0, 1, 2, 3, 255])];
let zip = build(&files);
let tail = &zip[zip.len() - tail_len(zip.len() as u64) as usize..];
let (off, size) = find_central(tail).unwrap();
let entries = parse_central(&zip[off as usize..(off + size) as usize]).unwrap();
assert_eq!(entries.len(), 3);
for (e, (name, data)) in entries.iter().zip(files) {
assert_eq!(e.name, name);
let start = data_offset(e, &zip[e.header as usize..e.header as usize + 30]).unwrap() as usize;
let got = &zip[start..start + e.size as usize];
assert_eq!(got, data);
verify(e, got).unwrap();
}
// damage is caught
let e = &entries[2];
assert!(verify(e, &[0, 1, 2, 3, 254]).is_err());
assert!(find_central(b"not a zip at all, just some text").is_err());
}
#[test]
fn restore_keys_are_safe() {
assert_eq!(restore_key("library/catalog.snap", "library-restored-1").as_deref(), Some("library-restored-1/catalog.snap"));
assert_eq!(restore_key("library/../../x", "library-r"), None);
assert_eq!(restore_key("library/evil.bin", "library-r"), None);
assert_eq!(restore_key("originals/abc123/IMG.jpg", "d").as_deref(), Some("originals/abc123"));
assert_eq!(restore_key("originals/../library/catalog.log", "d"), None);
assert_eq!(restore_key("README.txt", "d"), None);
assert_eq!(restore_key("thumbs/x.jpg", "d"), None);
assert_eq!(original_entry("abc", "a/b\\c.jpg"), "originals/abc/a_b_c.jpg");
assert_eq!(original_entry("abc", ".."), "originals/abc/original");
assert!(valid_library_dir("library") && valid_library_dir("library-restored-17"));
assert!(!valid_library_dir("library-../x") && !valid_library_dir("originals") && !valid_library_dir("library-"));
}
#[test]
fn size_limits_are_errors_not_panics() {
let mut w = ZipWriter { offset: MAX_BYTES - 10, ..Default::default() };
assert!(w.entry("x", &[0; 64]).is_err());
}
}
+58
View File
@@ -49,6 +49,8 @@ struct Mirror {
files: BTreeMap<String, Vec<u8>>,
/// Dirty files, least recently modified first.
dirty: Vec<(String, Dirty)>,
/// Why the last flush to browser storage failed (quota, eviction…), until one succeeds.
error: Option<String>,
}
impl Mirror {
@@ -138,6 +140,18 @@ impl Files {
}
}
/// Record the outcome of a flush to browser storage: `Some(why)` when it failed, `None` once
/// one succeeds. While a flush is failing, the catalog [`Store`] refuses new writes, so the
/// engine keeps the changes queued and reports them unsaved (`library.info` → `unsavedOps`,
/// the top bar's warning) instead of pretending they were saved.
pub fn set_error(&self, e: Option<String>) {
self.lock().error = e;
}
pub fn error(&self) -> Option<String> {
self.lock().error.clone()
}
/// A catalog [`Store`] over these files.
pub fn store(&self) -> FilesStore {
FilesStore(self.clone())
@@ -147,19 +161,32 @@ impl Files {
/// [`Store`] implementation over [`Files`].
pub struct FilesStore(Files);
impl FilesStore {
/// Writes fail while browser storage does (see [`Files::set_error`]).
fn writable(&self) -> io::Result<()> {
match self.0.error() {
Some(e) => Err(io::Error::other(format!("browser storage: {e}"))),
None => Ok(()),
}
}
}
impl Store for FilesStore {
fn read(&mut self, name: &str) -> io::Result<Option<Vec<u8>>> {
Ok(self.0.get(name))
}
fn write_atomic(&mut self, name: &str, data: &[u8]) -> io::Result<()> {
self.writable()?;
self.0.write(name, data);
Ok(())
}
fn append(&mut self, name: &str, data: &[u8]) -> io::Result<()> {
self.writable()?;
self.0.append(name, data);
Ok(())
}
fn truncate(&mut self, name: &str, len: u64) -> io::Result<()> {
self.writable()?;
self.0.truncate(name, len as usize);
Ok(())
}
@@ -224,6 +251,37 @@ mod tests {
assert_eq!(names, ["catalog.snap", "catalog.log"]);
}
/// A failing flush (quota exceeded, storage evicted) is not silent: commands report the change
/// as unsaved and the engine keeps it queued; once storage works again it is written.
#[test]
fn flush_failures_show_as_unsaved_and_are_retried() {
let mut disk = BTreeMap::new();
let files = Files::default();
let mut s = Session::new();
let stores = LibraryStores { dir: "browser".into(), catalog: Box::new(files.store()), files: Box::new(files.store()), on_disk: false };
s.open_library_in(stores, true).unwrap();
flush(&files, &mut disk);
files.set_error(Some("QuotaExceededError".into()));
let r = s.execute("photo.rate", &json!({"rating": 5}));
assert!(matches!(r, Err(lightcraft_engine::EngineError::NotSaved(_))), "{r:?}");
let (ops, why) = s.unsaved().unwrap();
assert!(ops > 0 && why.contains("QuotaExceeded"), "{why}");
files.set_error(None);
s.persist_if_dirty();
s.persist().unwrap();
assert!(s.unsaved().is_none(), "written once storage works again");
flush(&files, &mut disk);
let id = s.selection.active.unwrap();
let files2 = Files::default();
for (k, v) in &disk {
files2.preload(k, v.clone());
}
let mut s2 = Session::new();
let stores = LibraryStores { dir: "browser".into(), catalog: Box::new(files2.store()), files: Box::new(files2.store()), on_disk: false };
s2.open_library_in(stores, true).unwrap();
assert_eq!(s2.catalog.photo(id).unwrap().rating, 5, "the rating survived");
}
#[test]
fn library_survives_reload_through_flushes() {
let mut disk = BTreeMap::new();
+3
View File
@@ -16,6 +16,7 @@
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
pub mod backup;
pub mod bench;
pub mod files;
pub mod store;
@@ -24,6 +25,8 @@ pub mod wire;
#[cfg(target_arch = "wasm32")]
mod backend;
#[cfg(target_arch = "wasm32")]
mod safety;
#[cfg(target_arch = "wasm32")]
mod web;
#[cfg(target_arch = "wasm32")]
mod workers;
+256
View File
@@ -0,0 +1,256 @@
//! Keeping a library in the browser safe: notices the user must see (a failed save, storage the
//! browser may evict, a file that couldn't be stored), one tab at a time per library, a message
//! instead of a dead page after a panic, and backup / restore of the whole library
//! ([`crate::backup`]).
use std::cell::RefCell;
use std::collections::HashMap;
use js_sys::{Reflect, Uint8Array};
use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
use wasm_bindgen_futures::JsFuture;
use crate::backend::Backend;
use crate::backup::{
ACTIVE_LIBRARY, README, ZipWriter, data_offset, find_central, original_entry, parse_central, restore_key, restored_dir_name, tail_len,
valid_hash, verify,
};
use crate::files::{Files, LIBRARY_FILES};
use crate::store::storage_key;
thread_local! {
/// Messages for the user, shown by the app on its next frame.
static NOTICES: RefCell<Vec<String>> = const { RefCell::new(Vec::new()) };
}
/// Queue a message for the user (shown for several seconds; also logged).
pub fn notice(text: impl Into<String>) {
let text = text.into();
log::warn!("{text}");
NOTICES.with(|n| n.borrow_mut().push(text));
crate::web::request_repaint();
}
/// Messages queued since the last call.
pub fn take_notices() -> Vec<String> {
NOTICES.with(|n| std::mem::take(&mut *n.borrow_mut()))
}
fn document() -> Option<web_sys::Document> {
web_sys::window()?.document()
}
/// Ask the user (the browser's own confirm box). `false` when it can't be shown.
pub fn confirm(text: &str) -> bool {
web_sys::window().and_then(|w| w.confirm_with_message(text).ok()).unwrap_or(false)
}
/// Cover the page with a message (the app can't continue, or must not start).
pub fn show_blocking(text: &str) {
let Some(doc) = document() else { return };
let el = match doc.get_element_by_id("lightcraft_blocking") {
Some(el) => el,
None => {
let Ok(el) = doc.create_element("div") else { return };
el.set_id("lightcraft_blocking");
let _ = el.set_attribute(
"style",
"position:fixed;inset:0;z-index:10;display:flex;align-items:center;justify-content:center;padding:32px;\
background:#1c1c1c;color:#e0e0e0;font:15px/1.5 system-ui,-apple-system,'Segoe UI',sans-serif;text-align:center;white-space:pre-line",
);
if let Some(body) = doc.body() {
let _ = body.append_child(&el);
}
el
}
};
el.set_text_content(Some(text));
if let Some(loading) = doc.get_element_by_id("lightcraft_loading") {
loading.remove();
}
}
/// `panic = "abort"` on wasm: a panic ends the module. Say so on the page instead of leaving a
/// frozen canvas, and say that the stored library is intact.
pub fn install_panic_hook() {
std::panic::set_hook(Box::new(|info| {
let msg = match (info.payload().downcast_ref::<&str>(), info.payload().downcast_ref::<String>()) {
(Some(s), _) => (*s).to_string(),
(_, Some(s)) => s.clone(),
_ => "unknown error".into(),
};
let at = info.location().map(|l| format!(" at {}:{}", l.file(), l.line())).unwrap_or_default();
log::error!("LightCraft panicked{at}: {msg}");
show_blocking(&format!(
"LightCraft stopped after an internal error:\n{msg}\n\nThe library saved in this browser is kept. Reload the page to continue.\n\
If it happens again right away, a photo may be the cause: reload with ?workers=0 or report the error."
));
}));
}
/// The name of the lock that keeps a library to one tab.
const TAB_LOCK: &str = "lightcraft-library";
/// Hold the library lock for as long as this page lives (Web Locks API). `Some(true)`: ours;
/// `Some(false)`: another tab of this site has it; `None`: the browser can't tell (no Web Locks).
pub async fn acquire_tab_lock() -> Option<bool> {
let nav = Reflect::get(&js_sys::global(), &"navigator".into()).ok()?;
let locks = Reflect::get(&nav, &"locks".into()).ok().filter(|l| !l.is_undefined() && !l.is_null())?;
let request: js_sys::Function = Reflect::get(&locks, &"request".into()).ok()?.dyn_into().ok()?;
let opts = js_sys::Object::new();
Reflect::set(&opts, &"ifAvailable".into(), &true.into()).ok()?;
let granted = js_sys::Promise::new(&mut |resolve, _reject| {
let cb = Closure::once_into_js(move |lock: JsValue| -> JsValue {
let ours = !lock.is_null() && !lock.is_undefined();
let _ = resolve.call1(&JsValue::NULL, &ours.into());
// keep the lock until the page goes away: a promise that never settles
if ours { js_sys::Promise::new(&mut |_, _| {}).into() } else { JsValue::NULL }
});
if let Err(e) = request.call3(&locks, &TAB_LOCK.into(), &opts, &cb) {
log::warn!("Web Locks request failed: {e:?}");
}
});
JsFuture::from(granted).await.ok().map(|v| v.is_truthy())
}
/// Offer a blob as a download named `name`.
pub fn download_blob(name: &str, blob: &web_sys::Blob) -> Result<(), String> {
let e = |e: JsValue| format!("download failed: {e:?}");
let url = web_sys::Url::create_object_url_with_blob(blob).map_err(e)?;
let doc = document().ok_or("no document")?;
let a: web_sys::HtmlAnchorElement = doc.create_element("a").map_err(e)?.unchecked_into();
a.set_href(&url);
a.set_download(name);
a.click();
// revoke once the download has started
let revoke = Closure::once_into_js(move || {
let _ = web_sys::Url::revoke_object_url(&url);
});
if let Some(w) = web_sys::window() {
let _ = w.set_timeout_with_callback_and_timeout_and_arguments_0(revoke.unchecked_ref(), 60_000);
}
Ok(())
}
/// File ▸ Back Up Library…: a zip of the library files (as the app has them, saved or not) and
/// every original in storage, built entry by entry into the browser's memory (outside the wasm
/// heap) and downloaded. `names`: content hash → the photo's file name.
pub async fn backup(files: Files, backend: Backend, names: HashMap<String, String>) -> Result<(usize, f64), String> {
let parts = js_sys::Array::new();
let mut zip = ZipWriter::default();
let mut bytes = 0f64;
let mut add = |zip: &mut ZipWriter, name: &str, data: &[u8]| -> Result<(), String> {
let header = zip.entry(name, data)?;
parts.push(&Uint8Array::from(&header[..]));
parts.push(&Uint8Array::from(data));
bytes += (header.len() + data.len()) as f64;
Ok(())
};
add(&mut zip, "README.txt", README.as_bytes())?;
for name in LIBRARY_FILES {
if let Some(d) = files.get(name) {
add(&mut zip, &format!("library/{name}"), &d)?;
}
}
let mut originals = 0;
for hash in backend.list("originals").await? {
if !valid_hash(&hash) {
continue;
}
match backend.read(&storage_key(&hash)).await {
Ok(Some(data)) => {
let name = names.get(&hash).map_or("original", String::as_str);
add(&mut zip, &original_entry(&hash, name), &data)?;
originals += 1;
}
Ok(None) => {}
Err(e) => return Err(format!("reading an original from browser storage: {e}")),
}
}
let tail = zip.finish()?;
parts.push(&Uint8Array::from(&tail[..]));
let opts = web_sys::BlobPropertyBag::new();
opts.set_type("application/zip");
let blob = web_sys::Blob::new_with_u8_array_sequence_and_options(&parts, &opts).map_err(|e| format!("{e:?}"))?;
let day: String = String::from(js_sys::Date::new_0().to_iso_string()).chars().take(10).collect();
download_blob(&format!("lightcraft-library-backup-{day}.zip"), &blob)?;
Ok((originals, bytes + tail.len() as f64))
}
/// Bytes `start..end` of a picked file.
async fn read_range(file: &web_sys::File, start: u64, end: u64) -> Result<Vec<u8>, String> {
let blob = file.slice_with_f64_and_f64(start as f64, end as f64).map_err(|e| format!("{e:?}"))?;
let buf = JsFuture::from(blob.array_buffer()).await.map_err(|e| format!("{e:?}"))?;
Ok(Uint8Array::new(&buf).to_vec())
}
/// Restore a backup into a new library folder (nothing in storage is deleted or overwritten),
/// then point [`ACTIVE_LIBRARY`] at it. Returns the folder's name; the caller reloads the page.
pub async fn restore(file: web_sys::File, backend: Backend) -> Result<String, String> {
let len = file.size() as u64;
let tail = read_range(&file, len - tail_len(len), len).await?;
let (cd_offset, cd_size) = find_central(&tail)?;
let cd = read_range(&file, cd_offset, cd_offset + cd_size).await?;
let entries = parse_central(&cd)?;
if !entries.iter().any(|e| e.name == "library/catalog.snap" || e.name == "library/catalog.log") {
return Err("this isn't a LightCraft library backup (no library/catalog files)".into());
}
let dir = restored_dir_name(js_sys::Date::now());
let mut written = 0usize;
for e in &entries {
let Some(key) = restore_key(&e.name, &dir) else { continue };
// originals are named by their content: one already stored is the same file
if key.starts_with("originals/") && backend.exists(&key).await? {
continue;
}
let local = read_range(&file, e.header, e.header + 30).await?;
let start = data_offset(e, &local)?;
let data = read_range(&file, start, start + e.size).await?;
verify(e, &data)?;
backend.write(&key, &data).await.map_err(|err| format!("storing {}: {err}", e.name))?;
written += 1;
}
backend.write(ACTIVE_LIBRARY, dir.as_bytes()).await?;
log::info!("lightcraft: restored {written} files into {dir}");
Ok(dir)
}
/// File ▸ Restore Library from Backup…: confirm, pick the zip, restore, reload.
pub fn pick_and_restore(backend: Backend, before_reload: impl FnOnce() + 'static) {
let ok = confirm(
"Restore a LightCraft library backup?\n\nThe backup becomes the library in this browser and the page reloads. \
The current library is not deleted: it stays in browser storage.",
);
if !ok {
return;
}
let Some(doc) = document() else { return };
let Ok(input) = doc.create_element("input").map(|e| e.unchecked_into::<web_sys::HtmlInputElement>()) else { return };
input.set_type("file");
input.set_accept(".zip,application/zip");
let inp = input.clone();
let mut before_reload = Some(before_reload);
let on_change = Closure::<dyn FnMut()>::new(move || {
let Some(file) = inp.files().and_then(|f| f.get(0)) else { return };
let backend = backend.clone();
let before = before_reload.take();
notice(format!("Restoring {}…", file.name()));
wasm_bindgen_futures::spawn_local(async move {
match restore(file, backend).await {
Ok(_) => {
if let Some(f) = before {
f();
}
if let Some(w) = web_sys::window() {
let _ = w.location().reload();
}
}
Err(e) => notice(format!("Restore failed: {e}. Nothing was changed: the current library is still in use.")),
}
});
});
input.set_onchange(Some(on_change.as_ref().unchecked_ref()));
on_change.forget();
input.click();
}
+154 -23
View File
@@ -15,16 +15,21 @@ use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
use crate::backend::Backend;
use crate::backup::{ACTIVE_LIBRARY, DEFAULT_LIBRARY_DIR, valid_library_dir};
use crate::bench::Bench;
use crate::files::{Files, FlushOp, LIBRARY_FILES};
use crate::safety::{self, notice};
use crate::store::{Originals, content_hash, download_name, storage_key};
use crate::wire::ThumbIndex;
use crate::workers::{THUMB_INDEX, Workers};
const ACCEPT: &str = ".jpg,.jpeg,.png,.tif,.tiff,.webp,.dng,.cr2,.nef,.arw,.psd,.jxl,.gif,.bmp";
/// Where the library files live in browser storage.
const LIBRARY_DIR: &str = "library";
/// After a failed save, wait this long before trying again (ms).
const SAVE_RETRY_MS: f64 = 2000.0;
/// How long a notice stays on screen (s).
const NOTICE_SECS: f64 = 8.0;
/// How often view state, UI prefs and the thumbnail index are saved (ms).
const SAVE_EVERY_MS: f64 = 1000.0;
@@ -45,8 +50,18 @@ fn set_status(text: &str) {
}
}
fn library_key(name: &str) -> String {
format!("{LIBRARY_DIR}/{name}")
/// Storage key of library file `name` in library folder `dir`.
fn library_key(dir: &str, name: &str) -> String {
format!("{dir}/{name}")
}
/// Repaint the app (notices from async tasks).
pub(crate) fn request_repaint() {
CTX.with(|c| {
if let Some(c) = c.borrow().as_ref() {
c.request_repaint();
}
});
}
/// URL options (`?bench&store=idb&workers=2&reset`).
@@ -86,8 +101,9 @@ async fn store_file(originals: Originals, backend: Option<Backend>, name: String
if let Some(b) = &backend
&& let Err(e) = b.write(&storage_key(&hash), &bytes).await
{
// still importable for this session; it won't survive a reload
log::error!("storing {name} in browser storage failed: {e}");
// a photo whose bytes aren't stored would be gone after a reload: don't add it
notice(format!("{name} was not added: browser storage refused it ({e}). Free up space or back up and clear old photos."));
return;
}
originals.added(&name, &hash, Arc::from(bytes));
ctx.request_repaint();
@@ -148,8 +164,42 @@ fn download(path: &str, bytes: &[u8]) -> Result<(), String> {
Ok(())
}
fn services(originals: Originals, backend: Option<Backend>, ctx: egui::Context) -> Services {
fn services(originals: Originals, backend: Option<Backend>, files: Files, frozen: Rc<Cell<bool>>, ctx: egui::Context) -> Services {
let (backup_backend, backup_files) = (backend.clone(), files);
let restore_backend = backend.clone();
Services {
backup_library: Some(Box::new(move |session: &mut Session| {
let Some(b) = backup_backend.clone() else { return Err("nothing is stored in this browser session (?store=memory)".into()) };
// the photos' own names for the originals in the zip
let names: std::collections::HashMap<String, String> = session
.catalog
.photos()
.filter_map(|p| match &p.source {
Source::File { path } => crate::store::hash_of_path(path).map(|h| (h.to_string(), p.file_name.clone())),
_ => None,
})
.collect();
let files = backup_files.clone();
notice("Backing up the library… (the download starts when it's ready)");
wasm_bindgen_futures::spawn_local(async move {
match safety::backup(files, b, names).await {
Ok((n, bytes)) => notice(format!(
"Library backed up: {n} original{} ({:.1} MB). Keep the file somewhere safe.",
if n == 1 { "" } else { "s" },
bytes / 1e6
)),
Err(e) => notice(format!("Backup failed: {e}")),
}
});
Ok(json!({"started": true}))
})),
restore_library: Some(Box::new(move |_session: &mut Session| {
let Some(b) = restore_backend.clone() else { return Err("nothing is stored in this browser session (?store=memory)".into()) };
let frozen = frozen.clone();
// once the backup is in place, stop saving the old library (the page reloads)
safety::pick_and_restore(b, move || frozen.set(true));
Ok(json!({"started": true}))
})),
pick_files: Some(Box::new(move || {
open_picker(originals.clone(), backend.clone(), ctx.clone());
Vec::new() // files arrive asynchronously and are imported on a later frame
@@ -176,21 +226,29 @@ fn services(originals: Originals, backend: Option<Backend>, ctx: egui::Context)
}
}
/// Write dirty library files to storage, in order, until none are left.
async fn flush(files: Files, backend: Backend, flushing: Rc<Cell<bool>>) {
/// Write dirty library files to storage, in order, until none are left. A failure (quota,
/// storage cleared) is recorded in `files` — the catalog then reports changes as unsaved instead
/// of accepting them silently — and retried after [`SAVE_RETRY_MS`].
async fn flush(files: Files, backend: Backend, dir: String, flushing: Rc<Cell<bool>>, failed_at: Rc<Cell<f64>>) {
'outer: loop {
let ops = files.take_dirty();
if ops.is_empty() {
if files.error().is_some() {
log::info!("lightcraft: saving works again");
}
files.set_error(None);
break;
}
for (i, op) in ops.iter().enumerate() {
let r = match op {
FlushOp::Write { name, data } => backend.write(&library_key(name), data).await,
FlushOp::Append { name, offset, data } => backend.write_at(&library_key(name), Some(*offset), data).await,
FlushOp::Write { name, data } => backend.write(&library_key(&dir, name), data).await,
FlushOp::Append { name, offset, data } => backend.write_at(&library_key(&dir, name), Some(*offset), data).await,
};
if let Err(e) = r {
log::error!("saving {} failed: {e}", op.name());
files.failed(&ops[i..]);
files.set_error(Some(e));
failed_at.set(perf_now());
break 'outer;
}
}
@@ -204,6 +262,8 @@ struct Boot {
backend: Option<Backend>,
files: Files,
index: ThumbIndex,
/// The library folder in storage (`library`, or a restored backup's).
lib_dir: String,
}
async fn boot(opts: Options) -> Boot {
@@ -221,15 +281,32 @@ async fn boot(opts: Options) -> Boot {
};
let files = Files::default();
let mut index = ThumbIndex::default();
let mut lib_dir = DEFAULT_LIBRARY_DIR.to_string();
if let Some(b) = &backend {
if opts.reset {
match b.clear().await {
Ok(()) => log::info!("lightcraft: storage cleared (?reset)"),
Err(e) => log::error!("clearing storage: {e}"),
// everything this site keeps in the browser, photos included: never without asking
let ok = safety::confirm(
"?reset deletes the LightCraft library stored in this browser, including every imported photo. \
This can't be undone. Delete it?",
);
if ok {
match b.clear().await {
Ok(()) => log::info!("lightcraft: storage cleared (?reset)"),
Err(e) => log::error!("clearing storage: {e}"),
}
} else {
log::info!("lightcraft: ?reset declined; storage kept");
}
}
// a restored backup lives in its own folder (see `backup`)
if let Ok(Some(d)) = b.read(ACTIVE_LIBRARY).await {
let d = String::from_utf8_lossy(&d).trim().to_string();
if valid_library_dir(&d) {
lib_dir = d;
}
}
for name in LIBRARY_FILES {
match b.read(&library_key(name)).await {
match b.read(&library_key(&lib_dir, name)).await {
Ok(Some(data)) => files.preload(name, data),
Ok(None) => {}
Err(e) => log::error!("reading {name}: {e}"),
@@ -238,7 +315,14 @@ async fn boot(opts: Options) -> Boot {
if let Ok(Some(j)) = b.read(THUMB_INDEX).await {
index = ThumbIndex::from_json(&j);
}
crate::backend::request_persistence();
crate::backend::request_persistence(|granted| {
if !granted {
notice(
"This browser may delete LightCraft's library when it runs short of space (persistent storage wasn't granted). \
Keep your original photos elsewhere and use File ▸ Back Up Library… now and then.",
);
}
});
// thumbnails stored without an index entry (index saved late, or lost): delete them
let b2 = b.clone();
let known: std::collections::HashSet<String> = b
@@ -266,7 +350,7 @@ async fn boot(opts: Options) -> Boot {
index.len(),
index.total() as f64 / 1e6
);
Boot { opts, backend, files, index }
Boot { opts, backend, files, index, lib_dir }
}
struct WebApp {
@@ -276,6 +360,11 @@ struct WebApp {
backend: Option<Backend>,
workers: Option<Workers>,
flushing: Rc<Cell<bool>>,
/// When the last save to storage failed (ms; 0 = never).
flush_failed_at: Rc<Cell<f64>>,
/// Saving stopped (a restored backup is about to replace this library: the page reloads).
frozen: Rc<Cell<bool>>,
lib_dir: String,
bench: Option<Bench>,
first_frame_logged: bool,
last_save: f64,
@@ -284,17 +373,18 @@ struct WebApp {
impl WebApp {
fn new(cc: &eframe::CreationContext<'_>, boot: Boot) -> Self {
let Boot { opts, backend, files, index } = boot;
let Boot { opts, backend, files, index, lib_dir } = boot;
let originals = Originals::default();
let t = perf_now();
let mut session = Session::new();
originals.install(&mut session);
let stores = LibraryStores {
dir: format!("browser:{}/{LIBRARY_DIR}", backend.as_ref().map_or("memory", |b| b.kind())).into(),
dir: format!("browser:{}/{lib_dir}", backend.as_ref().map_or("memory", |b| b.kind())).into(),
catalog: Box::new(files.store()),
files: Box::new(files.store()),
on_disk: false,
};
let mut problem = None;
match session.open_library_in(stores, true).cloned() {
Ok(r) => log::info!(
"lightcraft: library {} in {:.0} ms ({} photos; snapshot seq {}, {} ops replayed)",
@@ -305,19 +395,35 @@ impl WebApp {
r.replayed
),
Err(e) => {
log::error!("opening the library failed: {e}; starting a temporary one");
session = Session::with_demo();
// never a silent demo session (issue #100): an empty one, and the window says why
log::error!("opening the library failed: {e}");
session = Session::new();
originals.install(&mut session);
problem = Some(lightcraft_ui_egui::panels::library_problem::LibraryProblem {
can_retry: false,
..lightcraft_ui_egui::panels::library_problem::LibraryProblem::new(
"the browser's storage for this page",
format!("{e}. Reload the page to try again."),
)
});
}
}
if backend.is_some() {
notice(
"LightCraft in the browser is experimental. Your library is kept in this browser's storage: keep your original photos \
elsewhere and back up with File ▸ Back Up Library….",
);
}
// previews are large (≤ 2560 px, f32): keep few in a 32-bit address space
session.media.preview_capacity = 3;
let mut app = LightcraftApp::new(session, services(originals.clone(), backend.clone(), cc.egui_ctx.clone()));
let frozen = Rc::new(Cell::new(false));
let mut app = LightcraftApp::new(session, services(originals.clone(), backend.clone(), files.clone(), frozen.clone(), cc.egui_ctx.clone()));
let ui_written = files.get("ui.json").unwrap_or_default();
if let Ok(ui) = serde_json::from_slice::<UiState>(&ui_written) {
app.ui = ui;
}
app.ui = app.ui.sanitized();
app.library_problem = problem;
let n = opts.workers.unwrap_or_else(|| {
let cores = window().map_or(1, |w| w.navigator().hardware_concurrency() as usize);
cores.saturating_sub(1).clamp(1, 4)
@@ -337,6 +443,9 @@ impl WebApp {
backend,
workers,
flushing: Rc::new(Cell::new(false)),
flush_failed_at: Rc::new(Cell::new(0.0)),
frozen,
lib_dir,
bench: opts.bench.then(|| Bench::new(origin)),
first_frame_logged: false,
last_save: 0.0,
@@ -398,6 +507,9 @@ impl WebApp {
/// Save view state, UI prefs and the thumbnail index now and then; flush dirty files.
fn save(&mut self) {
if self.frozen.get() {
return;
}
let now = perf_now();
if now - self.last_save >= SAVE_EVERY_MS {
self.last_save = now;
@@ -419,12 +531,16 @@ impl WebApp {
});
}
}
// after a failure, retry every couple of seconds rather than every frame
let backing_off = self.files.error().is_some() && now - self.flush_failed_at.get() < SAVE_RETRY_MS;
if let Some(b) = &self.backend
&& self.files.is_dirty()
&& !self.flushing.get()
&& !backing_off
{
self.flushing.set(true);
wasm_bindgen_futures::spawn_local(flush(self.files.clone(), b.clone(), self.flushing.clone()));
let (files, dir, flushing, failed_at) = (self.files.clone(), self.lib_dir.clone(), self.flushing.clone(), self.flush_failed_at.clone());
wasm_bindgen_futures::spawn_local(flush(files, b.clone(), dir, flushing, failed_at));
}
}
}
@@ -454,8 +570,10 @@ impl WebApp {
let (orig_n, orig_bytes) = self.originals.usage();
json!({
"storage": self.backend.as_ref().map_or("memory", |b| b.kind()),
"library": self.lib_dir,
"dirty": self.files.is_dirty(),
"flushing": self.flushing.get(),
"saveError": self.files.error(),
"workers": {"alive": alive, "ready": ready, "remoteJobs": remote, "inlineJobs": inline},
"originalsInMemory": orig_n,
"originalBytesInMemory": orig_bytes,
@@ -480,6 +598,10 @@ impl WebApp {
impl eframe::App for WebApp {
fn logic(&mut self, ctx: &egui::Context, _frame: &mut eframe::Frame) {
for text in safety::take_notices() {
let now = ctx.input(|i| i.time);
self.app.ui.toast = Some((text, now + NOTICE_SECS));
}
self.run_inbox();
self.import_dropped(ctx);
self.load_originals(ctx);
@@ -517,9 +639,18 @@ impl eframe::App for WebApp {
#[wasm_bindgen]
pub fn start() {
eframe::WebLogger::init(log::LevelFilter::Info).ok();
safety::install_panic_hook();
log::info!("lightcraft: wasm instantiated at {:.0} ms", perf_now());
let opts = Options::from_url();
wasm_bindgen_futures::spawn_local(async move {
// one tab per library: two would each keep their own copy and overwrite each other's saves
if opts.store != "memory" && safety::acquire_tab_lock().await == Some(false) {
safety::show_blocking(
"LightCraft is already open in another tab or window of this browser.\n\n\
Switch to that tab, or close it and reload this page. (Two tabs would overwrite each other's changes.)",
);
return;
}
let Some(canvas) = window()
.and_then(|w| w.document())
.and_then(|d| d.get_element_by_id("lightcraft_canvas"))
+24 -10
View File
@@ -15,10 +15,9 @@ use std::sync::Arc;
use lightcraft_engine::catalog::Source;
use lightcraft_engine::develop::{DevelopSettings, EmbeddedLens};
use lightcraft_engine::media::{MediaCache, RenderJob, SourceLevel, SourceRef};
use lightcraft_engine::media::{DecodedSource, MediaCache, RenderJob, SourceLevel, SourceRef};
use lightcraft_engine::pipeline::{Quality, RenderRequest, Rendered, SourceInfo, StageCache};
use lightcraft_preview::Lru;
use lightcraft_raster::Rgb32f;
use serde::{Deserialize, Serialize};
use crate::store::hash_of_path;
@@ -31,6 +30,8 @@ pub struct WireJob {
/// Decode the source at most this long.
pub max_edge: usize,
pub raw: bool,
#[serde(default)]
pub relative_wb: bool,
pub as_shot_temp: f64,
pub as_shot_tint: f64,
pub lens: Option<EmbeddedLens>,
@@ -55,7 +56,7 @@ impl WireJob {
pub fn from_job(job: &RenderJob, thumb_cached: bool, view: &str) -> WireJob {
let max_edge = match &job.source {
SourceRef::Demo { max_edge, .. } | SourceRef::File { max_edge, .. } => *max_edge,
SourceRef::Loaded(img) => img.width.max(img.height),
SourceRef::Loaded(img) => img.image.width.max(img.image.height),
SourceRef::Smart { .. } => 2560,
};
WireJob {
@@ -63,6 +64,7 @@ impl WireJob {
origin: job.origin.clone(),
max_edge,
raw: job.info.raw,
relative_wb: job.info.relative_wb,
as_shot_temp: job.info.as_shot_temp,
as_shot_tint: job.info.as_shot_tint,
lens: job.info.lens,
@@ -79,7 +81,14 @@ impl WireJob {
}
pub fn info(&self) -> SourceInfo {
SourceInfo { raw: self.raw, as_shot_temp: self.as_shot_temp, as_shot_tint: self.as_shot_tint, lens: self.lens }
SourceInfo {
raw: self.raw,
as_shot_temp: self.as_shot_temp,
as_shot_tint: self.as_shot_tint,
lens: self.lens,
relative_wb: self.relative_wb,
..Default::default()
}
}
pub fn request(&self) -> RenderRequest {
@@ -119,7 +128,7 @@ const WORKER_SOURCE_BYTES: usize = 192 << 20;
/// The synchronous part of a worker: decoded-source cache + rendering.
pub struct WorkerCore {
sources: Lru<String, Arc<Rgb32f>>,
sources: Lru<String, DecodedSource>,
media: MediaCache,
stages: HashMap<String, Arc<StageCache>>,
}
@@ -150,21 +159,25 @@ impl WorkerCore {
Some(s) => s,
None => {
let src = match (&job.origin, original) {
(Source::File { .. }, Some(bytes)) => Arc::new(lightcraft_engine::files::load_bytes(bytes, job.max_edge)?.0),
(Source::File { .. }, Some(bytes)) => {
let (image, info) = lightcraft_engine::files::load_bytes(bytes, job.max_edge)?;
DecodedSource::new(Arc::new(image), Some(info))
}
(Source::File { path }, None) => return Err(format!("{path}: original not found in browser storage")),
(origin @ Source::Demo { .. }, _) => self.media.origin_ref(origin, job.max_edge).load()?,
(origin @ Source::Demo { .. }, _) => self.media.origin_ref(origin, job.max_edge).load_source()?,
};
let cost = src.width * src.height * 12 + 64;
let cost = src.image.width * src.image.height * 12 + 64;
self.sources.insert(key, src.clone(), cost);
src
}
};
let info = src.info_or(job.info());
Ok(match &job.stages {
Some(view) => {
let st = self.stages.entry(view.clone()).or_default().clone();
lightcraft_engine::pipeline::render_cached(&src, &job.info(), &job.settings, &job.request(), &st)
lightcraft_engine::pipeline::render_cached(&src.image, &info, &job.settings, &job.request(), &st)
}
None => lightcraft_engine::pipeline::render(&src, &job.info(), &job.settings, &job.request()),
None => lightcraft_engine::pipeline::render(&src.image, &info, &job.settings, &job.request()),
})
}
}
@@ -295,6 +308,7 @@ mod tests {
origin,
max_edge: 512,
raw: false,
relative_wb: false,
as_shot_temp: 6500.0,
as_shot_tint: 0.0,
lens: None,
+237 -32
View File
@@ -1,14 +1,28 @@
//! Loopback JSON-lines control server: one request per line, one reply per line.
//! This is the transport the MCP server (`lightcraft-cli mcp --connect`) wraps.
//!
//! The server only ever reads requests: a line that is not a JSON object with a string `method`
//! (an HTTP request line from a browser's cross-origin `fetch`, a stray `nc`, binary junk) gets
//! one error reply and the connection is closed, so nothing after it is executed. Lines are
//! bounded ([`MAX_LINE`]) and so is the number of open connections ([`MAX_CONNECTIONS`]); junk
//! never reaches the UI thread. See `docs/control-protocol.md`.
use std::io::{BufRead, BufReader, Write};
use std::io::{BufRead, BufReader, Read, Write};
use std::net::{TcpListener, TcpStream};
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::mpsc::{Receiver, Sender, channel};
use std::time::Duration;
use lightcraft_ui_egui::ControlRequest;
use serde_json::{Value, json};
/// Longest accepted request line (bytes, without the newline). Requests are small JSON objects;
/// anything longer is refused and the connection closed.
pub const MAX_LINE: usize = 4 * 1024 * 1024;
/// Connections served at once; further ones get an error line and are closed.
pub const MAX_CONNECTIONS: usize = 16;
pub fn start(port: u16, ctx: egui::Context) -> Receiver<ControlRequest> {
let (tx, rx) = channel::<ControlRequest>();
let listener = match TcpListener::bind(("127.0.0.1", port)) {
@@ -19,44 +33,235 @@ pub fn start(port: u16, ctx: egui::Context) -> Receiver<ControlRequest> {
}
};
eprintln!("lightcraft: control server listening on 127.0.0.1:{port}");
std::thread::spawn(move || {
for stream in listener.incoming().flatten() {
let tx = tx.clone();
let ctx = ctx.clone();
std::thread::spawn(move || serve(stream, tx, ctx));
}
});
std::thread::spawn(move || accept_loop(listener, tx, ctx));
rx
}
fn serve(stream: TcpStream, tx: Sender<ControlRequest>, ctx: egui::Context) {
let Ok(read) = stream.try_clone() else { return };
let mut out = stream;
for line in BufReader::new(read).lines() {
let Ok(line) = line else { break };
if line.trim().is_empty() {
fn accept_loop(listener: TcpListener, tx: Sender<ControlRequest>, ctx: egui::Context) {
let open = Arc::new(AtomicUsize::new(0));
for mut stream in listener.incoming().flatten() {
if open.fetch_add(1, Ordering::SeqCst) >= MAX_CONNECTIONS {
open.fetch_sub(1, Ordering::SeqCst);
let _ = writeln!(stream, "{}", json!({"ok": false, "error": "too many control connections"}));
continue;
}
let reply = match serde_json::from_str::<Value>(&line) {
Ok(msg) => {
let id = msg.get("id").cloned().unwrap_or(Value::Null);
let method = msg.get("method").and_then(Value::as_str).unwrap_or("").to_string();
let params = msg.get("params").cloned().unwrap_or(json!({}));
let (req, rrx) = ControlRequest::new(method, params);
if tx.send(req).is_err() {
break;
}
ctx.request_repaint();
let mut r = rrx.recv_timeout(Duration::from_secs(60)).unwrap_or_else(|_| json!({"ok": false, "error": "timeout"}));
if let Some(o) = r.as_object_mut() {
o.insert("id".into(), id);
let tx = tx.clone();
let ctx = ctx.clone();
let open = Arc::clone(&open);
std::thread::spawn(move || {
serve(stream, &tx, &ctx);
open.fetch_sub(1, Ordering::SeqCst);
});
}
}
fn serve(stream: TcpStream, tx: &Sender<ControlRequest>, ctx: &egui::Context) {
let Ok(read) = stream.try_clone() else { return };
serve_lines(read, stream, |method, params| {
let (req, rrx) = ControlRequest::new(method, params);
tx.send(req).ok()?;
ctx.request_repaint();
Some(rrx.recv_timeout(Duration::from_secs(60)).unwrap_or_else(|_| json!({"ok": false, "error": "timeout"})))
});
}
/// One parsed line.
enum Line {
Request {
id: Value,
method: String,
params: Value,
},
Blank,
/// Not a request: reply with this error and close the connection.
Reject(String),
}
fn parse_line(bytes: &[u8]) -> Line {
let Ok(text) = std::str::from_utf8(bytes) else {
return Line::Reject("not a control request (invalid UTF-8); closing the connection".into());
};
let text = text.trim();
if text.is_empty() {
return Line::Blank;
}
let msg = match serde_json::from_str::<Value>(text) {
Ok(v) => v,
Err(e) => return Line::Reject(format!("bad JSON ({e}); expected one JSON request object per line; closing the connection")),
};
let Some(obj) = msg.as_object() else {
return Line::Reject("not a control request (expected a JSON object); closing the connection".into());
};
let Some(method) = obj.get("method").and_then(Value::as_str) else {
return Line::Reject("not a control request (missing string `method`); closing the connection".into());
};
Line::Request {
id: obj.get("id").cloned().unwrap_or(Value::Null),
method: method.to_string(),
params: obj.get("params").cloned().unwrap_or(json!({})),
}
}
/// Read one `\n`-terminated line of at most `max` bytes into `buf` (newline stripped).
/// `Ok(false)` at end of stream; `Err` when the line is too long or the read fails.
fn read_bounded_line(reader: &mut impl BufRead, buf: &mut Vec<u8>, max: usize) -> Result<bool, String> {
buf.clear();
// `take` bounds the bytes buffered for one line: a peer can't make us allocate more than `max`.
let limit = u64::try_from(max).unwrap_or(u64::MAX).saturating_add(1);
let n = reader.take(limit).read_until(b'\n', buf).map_err(|e| e.to_string())?;
if n == 0 {
return Ok(false);
}
if buf.last() == Some(&b'\n') {
buf.pop();
if buf.last() == Some(&b'\r') {
buf.pop();
}
return Ok(true);
}
if buf.len() > max {
return Err(format!("request line longer than {max} bytes; closing the connection"));
}
// Last line without a newline, then end of stream.
Ok(true)
}
/// Serve requests from `read`, answering on `out`, until the peer closes, sends something that is
/// not a request, or `handle` returns `None` (app gone). `handle` gets `(method, params)` and
/// returns the reply object (the request's `id` is added here).
fn serve_lines<R: Read, W: Write>(read: R, mut out: W, mut handle: impl FnMut(String, Value) -> Option<Value>) {
let mut reader = BufReader::new(read);
let mut buf = Vec::new();
loop {
let reject = match read_bounded_line(&mut reader, &mut buf, MAX_LINE) {
Ok(false) => return,
Err(e) => e,
Ok(true) => match parse_line(&buf) {
Line::Blank => continue,
Line::Reject(e) => e,
Line::Request { id, method, params } => {
let Some(mut r) = handle(method, params) else { return };
if let Some(o) = r.as_object_mut() {
o.insert("id".into(), id);
}
if writeln!(out, "{r}").and_then(|()| out.flush()).is_err() {
return;
}
continue;
}
r
}
Err(e) => json!({"ok": false, "error": format!("bad JSON: {e}")}),
},
};
if writeln!(out, "{reply}").is_err() {
break;
let _ = writeln!(out, "{}", json!({"ok": false, "error": reject})).and_then(|()| out.flush());
return;
}
}
#[cfg(test)]
mod tests {
use super::*;
fn run(input: &[u8]) -> (Vec<Value>, Vec<String>) {
let mut out = Vec::new();
let mut calls = Vec::new();
serve_lines(input, &mut out, |m, _p| {
calls.push(m);
Some(json!({"ok": true, "result": null}))
});
let replies = String::from_utf8(out).unwrap().lines().map(|l| serde_json::from_str::<Value>(l).unwrap()).collect();
(replies, calls)
}
#[test]
fn requests_are_answered_in_order_with_ids() {
let (r, calls) = run(b"{\"id\":1,\"method\":\"a\"}\n\n{\"id\":2,\"method\":\"b\",\"params\":{}}\r\n{\"method\":\"c\"}");
assert_eq!(calls, ["a", "b", "c"]);
assert_eq!(r.len(), 3);
assert_eq!(r[0]["id"], 1);
assert_eq!(r[1]["id"], 2);
assert_eq!(r[2]["id"], Value::Null);
}
#[test]
fn http_request_closes_before_the_body_runs() {
// A browser's cross-origin `fetch` POST: the request line is rejected and the connection
// closed, so the JSON body is never executed.
let http = b"POST / HTTP/1.1\r\nHost: 127.0.0.1:7980\r\nContent-Type: text/plain\r\n\r\n\
{\"id\":1,\"method\":\"engine.execute\",\"params\":{\"command\":\"folder.move\"}}\n";
let (r, calls) = run(http);
assert!(calls.is_empty(), "nothing may run: {calls:?}");
assert_eq!(r.len(), 1);
assert_eq!(r[0]["ok"], false);
assert!(r[0]["error"].as_str().unwrap().contains("closing the connection"));
}
#[test]
fn non_request_json_closes_the_connection() {
for junk in [
&b"[1,2]\n{\"method\":\"a\"}\n"[..],
b"{\"id\":1}\n{\"method\":\"a\"}\n",
b"{\"method\":5}\n{\"method\":\"a\"}\n",
b"\xff\xfe\n{\"method\":\"a\"}\n",
] {
let (r, calls) = run(junk);
assert!(calls.is_empty());
assert_eq!(r.len(), 1);
assert_eq!(r[0]["ok"], false);
}
}
#[test]
fn a_request_before_junk_still_runs() {
let (r, calls) = run(b"{\"method\":\"a\"}\nGET / HTTP/1.1\n{\"method\":\"b\"}\n");
assert_eq!(calls, ["a"]);
assert_eq!(r.len(), 2);
assert_eq!(r[1]["ok"], false);
}
#[test]
fn overlong_line_is_refused_without_buffering_it_all() {
// An endless line without a newline: reading stops after MAX_LINE + 1 bytes.
let endless = std::io::repeat(b'x');
let mut out = Vec::new();
let mut ran = false;
serve_lines(endless, &mut out, |_, _| {
ran = true;
None
});
assert!(!ran);
let reply: Value = serde_json::from_slice(out.strip_suffix(b"\n").unwrap()).unwrap();
assert!(reply["error"].as_str().unwrap().contains("longer than"));
}
#[test]
fn line_at_the_limit_is_accepted() {
let mut buf = Vec::new();
let line = vec![b'a'; 10];
let mut input = line.clone();
input.push(b'\n');
assert_eq!(read_bounded_line(&mut &input[..], &mut buf, 10), Ok(true));
assert_eq!(buf, line);
assert!(read_bounded_line(&mut &[b'a'; 11][..], &mut buf, 10).is_err());
}
#[test]
fn tcp_connection_is_closed_after_junk() {
// End to end over a real loopback socket (the transport the app uses).
let listener = TcpListener::bind(("127.0.0.1", 0)).unwrap();
let addr = listener.local_addr().unwrap();
let server = std::thread::spawn(move || {
let (s, _) = listener.accept().unwrap();
let read = s.try_clone().unwrap();
serve_lines(read, s, |_, _| Some(json!({"ok": true})));
});
let mut c = TcpStream::connect(addr).unwrap();
c.set_read_timeout(Some(Duration::from_secs(10))).unwrap();
c.write_all(b"{\"id\":7,\"method\":\"x\"}\nhello\n{\"method\":\"y\"}\n").unwrap();
let mut text = String::new();
c.read_to_string(&mut text).unwrap(); // returns at EOF: the server closed the connection
server.join().unwrap();
let lines: Vec<&str> = text.lines().collect();
assert_eq!(lines.len(), 2, "{text}");
assert!(lines[0].contains("\"id\":7"));
assert!(lines[1].contains("closing the connection"));
}
}
+299 -40
View File
@@ -7,7 +7,9 @@
//! `~/Pictures/LightCraft Library`; a new library starts with the
//! procedural demo photos unless `--no-demo` or files are given. Files and folders on the command
//! line are imported (duplicates are skipped). `--memory` runs an in-memory session that writes
//! nothing (demo photos unless files are given; used by the README showcase scripts).
//! nothing (demo photos unless files are given; used by the README showcase scripts). A library
//! that can't be opened is never replaced silently: the window says why and asks what to do
//! (`lightcraft_ui_egui::panels::library_problem`).
//!
//! `--control <port>` (or `LIGHTCRAFT_CONTROL_PORT`) starts a localhost JSON-lines control server:
//! `{"id":1,"method":"ui.inspect","params":{}}` → `{"id":1,"ok":true,"result":…}`.
@@ -26,6 +28,7 @@ mod control_server;
mod native_menu;
use lightcraft_engine::Session;
use lightcraft_ui_egui::panels::library_problem::LibraryProblem;
use lightcraft_ui_egui::{LightcraftApp, Services, UiState};
/// Reverse-DNS app id: Wayland app id, `.desktop` file and hicolor icon name.
@@ -41,15 +44,16 @@ fn app_icon() -> egui::IconData {
eframe::icon_data::from_png_bytes(png).unwrap_or_default()
}
struct App(LightcraftApp, #[cfg(target_os = "macos")] Option<native_menu::NativeMenu>);
struct App(LightcraftApp, PrefsWriter, #[cfg(target_os = "macos")] Option<native_menu::NativeMenu>);
impl eframe::App for App {
fn logic(&mut self, ctx: &egui::Context, _frame: &mut eframe::Frame) {
#[cfg(target_os = "macos")]
if let Some(m) = self.1.as_mut() {
if let Some(m) = self.2.as_mut() {
m.update(&mut self.0, ctx);
}
self.0.logic(ctx);
self.1.tick(&mut self.0, ctx);
}
fn raw_input_hook(&mut self, _ctx: &egui::Context, raw: &mut egui::RawInput) {
self.0.raw_input_hook(raw);
@@ -58,13 +62,52 @@ impl eframe::App for App {
self.0.ui(ui);
}
fn on_exit(&mut self) {
save_prefs(&self.0);
if let Err(e) = self.1.save(&self.0) {
eprintln!("lightcraft: {e}");
}
if let Err(e) = self.0.session.close_library() {
eprintln!("lightcraft: saving the library failed: {e}");
}
}
}
/// The window's renderer (egui-wgpu): eframe's defaults, with LightCraft's backend choice — DX12
/// alone on Windows unless `LIGHTCRAFT_GPU_BACKEND` / `WGPU_BACKEND` say otherwise (issue #136:
/// with Vulkan in the set, wgpu loads the Vulkan driver even when it then picks DX12).
fn window_wgpu_options() -> eframe::egui_wgpu::WgpuConfiguration {
let mut c = eframe::egui_wgpu::WgpuConfiguration::default();
if let eframe::egui_wgpu::WgpuSetup::CreateNew(n) = &mut c.wgpu_setup {
n.instance_descriptor.backends = lightcraft_engine::gpu::backend::window_backends();
}
c
}
/// Written while the GPU compute device is created, removed once that returned
/// (`lightcraft_gpu::backend::set_init_marker`). Not with `LIGHTCRAFT_NO_PREFS` (tests, scripts).
fn gpu_marker_path() -> Option<std::path::PathBuf> {
if std::env::var_os("LIGHTCRAFT_NO_PREFS").is_some() {
return None;
}
config_dir().map(|d| d.join("gpu-init.marker"))
}
/// Arm the GPU crash sentinel; if the previous launch left it behind (it died inside the GPU
/// driver while creating the device), the notice to show — GPU rendering then starts off.
fn gpu_crash_check() -> Option<String> {
let marker = gpu_marker_path();
let left = marker.as_deref().and_then(lightcraft_engine::gpu::backend::take_init_marker);
lightcraft_engine::gpu::backend::set_init_marker(marker);
left.map(|what| gpu_crash_notice(&what))
}
fn gpu_crash_notice(what: &str) -> String {
format!(
"LightCraft closed unexpectedly while starting the GPU last time ({what}), so GPU rendering is now off and photos render on the CPU. \
To try the GPU again, turn on Settings ▸ Performance ▸ Use the GPU for rendering; to try another graphics backend, start LightCraft with \
LIGHTCRAFT_GPU_BACKEND=dx12, vulkan or off (see docs/gpu-pipeline.md → Troubleshooting)."
)
}
fn config_dir() -> Option<std::path::PathBuf> {
if cfg!(target_os = "macos") {
std::env::var_os("HOME").map(|h| std::path::PathBuf::from(h).join("Library/Application Support/LightCraft"))
@@ -78,30 +121,127 @@ fn config_dir() -> Option<std::path::PathBuf> {
}
}
/// The saved UI state and app settings (`<config>/ui.json`), if any.
fn load_prefs() -> Option<UiState> {
/// The saved UI state and app settings (`<config>/ui.json`), if any, and a warning for the user
/// when the file exists but can't be used (issue #103): a damaged file is kept as
/// `ui.json.corrupt-<unix time>` first, so the next save can't lose the library location in it;
/// one that can't be read at all is not written this session (`keep_file`).
fn load_prefs() -> (Option<UiState>, Option<String>, bool) {
if std::env::var_os("LIGHTCRAFT_NO_PREFS").is_some() {
return None;
return (None, None, false);
}
let bytes = std::fs::read(config_dir()?.join("ui.json")).ok()?;
serde_json::from_slice::<UiState>(&bytes).ok().map(UiState::sanitized)
let Some(path) = config_dir().map(|d| d.join("ui.json")) else { return (None, None, false) };
load_prefs_at(&path)
}
fn save_prefs(app: &LightcraftApp) {
if std::env::var_os("LIGHTCRAFT_NO_PREFS").is_some() {
return;
fn load_prefs_at(path: &std::path::Path) -> (Option<UiState>, Option<String>, bool) {
let bytes = match std::fs::read(path) {
Ok(b) => b,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return (None, None, false),
Err(e) => {
let msg = format!("The app settings ({}) couldn't be read: {e}. Defaults are used, and the file isn't overwritten.", path.display());
return (None, Some(msg), true);
}
};
match serde_json::from_slice::<UiState>(&bytes) {
Ok(ui) => (Some(ui.sanitized()), None, false),
Err(e) => {
let secs = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0);
let keep = path.with_file_name(format!("ui.json.corrupt-{secs}"));
let kept = std::fs::rename(path, &keep);
let what = match &kept {
Ok(()) => format!("it was kept as {}", keep.display()),
Err(r) => format!("it couldn't be set aside ({r})"),
};
let msg = format!(
"The app settings ({}) are damaged ({e}); {what}. Defaults are used — reopen your library with Settings → Open Library… if it isn't shown.",
path.display()
);
(None, Some(msg), kept.is_err())
}
}
if let Some(d) = config_dir() {
let _ = std::fs::create_dir_all(&d);
if let Ok(bytes) = serde_json::to_vec_pretty(&app.ui) {
let _ = std::fs::write(d.join("ui.json"), bytes);
}
/// Write `bytes` to `path` atomically: a temp file, fsynced, renamed over it (a crash or a full
/// disk leaves the old file or the new one, never a truncated one).
fn write_atomic(path: &std::path::Path, bytes: &[u8]) -> std::io::Result<()> {
use std::io::Write;
let tmp = path.with_extension("json.tmp");
let written = (|| {
let mut f = std::fs::File::create(&tmp)?;
f.write_all(bytes)?;
f.sync_all()
})();
if let Err(e) = written {
let _ = std::fs::remove_file(&tmp);
return Err(e);
}
std::fs::rename(&tmp, path)?;
if let Some(dir) = path.parent()
&& let Ok(d) = std::fs::File::open(dir)
{
let _ = d.sync_all();
}
Ok(())
}
/// Saves `ui.json` (app settings, incl. the library to open at launch): as soon as the library
/// changes, every few seconds when anything else changed, and at exit.
#[derive(Default)]
struct PrefsWriter {
written: Vec<u8>,
library: String,
checked: f64,
/// The last write failed (reported once until a write works).
failing: bool,
/// `ui.json` couldn't be read at launch: never overwrite it this session.
keep_file: bool,
}
impl PrefsWriter {
fn save(&mut self, app: &LightcraftApp) -> Result<(), String> {
if self.keep_file || std::env::var_os("LIGHTCRAFT_NO_PREFS").is_some() {
return Ok(());
}
let Some(d) = config_dir() else { return Ok(()) };
let bytes = serde_json::to_vec_pretty(&app.ui).map_err(|e| e.to_string())?;
if bytes == self.written {
return Ok(());
}
std::fs::create_dir_all(&d)
.and_then(|()| write_atomic(&d.join("ui.json"), &bytes))
.map_err(|e| format!("saving the app settings failed: {e}"))?;
self.written = bytes;
self.library = app.ui.settings.library_path.clone();
Ok(())
}
fn tick(&mut self, app: &mut LightcraftApp, ctx: &egui::Context) {
let now = ctx.input(|i| i.time);
let moved = app.ui.settings.library_path != self.library;
if !moved && now - self.checked < 3.0 {
return;
}
self.checked = now;
match self.save(app) {
Ok(()) => self.failing = false,
Err(e) => {
eprintln!("lightcraft: {e}");
if !self.failing {
app.notices.push(format!("{e}. LightCraft keeps trying."));
}
self.failing = true;
// don't retry every frame
self.library = app.ui.settings.library_path.clone();
}
}
}
}
fn services() -> Services {
Services {
pick_folder: Some(Box::new(|| rfd::FileDialog::new().set_title("Open Library").pick_folder().map(|p| p.to_string_lossy().to_string()))),
pick_folder: Some(Box::new(|| {
rfd::FileDialog::new().set_title(lightcraft_ui_egui::i18n::tr("Open Library")).pick_folder().map(|p| p.to_string_lossy().to_string())
})),
open_with: Some(Box::new(|path: &str, app: &str| {
// spawned, never waited for: the editor runs alongside
let app = app.trim();
@@ -153,7 +293,7 @@ fn services() -> Services {
pick_files: Some(Box::new(|| {
rfd::FileDialog::new()
.add_filter(
"Photos",
lightcraft_ui_egui::i18n::tr("Photos"),
&[
"jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "nrw", "arw", "raf", "orf", "rw2", "pef", "psd",
"jxl", "gif", "bmp",
@@ -167,8 +307,11 @@ fn services() -> Services {
})),
pick_preset_files: Some(Box::new(|| {
rfd::FileDialog::new()
.set_title("Import Presets")
.add_filter("Presets & Profiles", &["lcpreset", "xmp", "lrtemplate", "zip", "dng", "lmp", "mplumpack", "cube"])
.set_title(lightcraft_ui_egui::i18n::tr("Import Presets"))
.add_filter(
lightcraft_ui_egui::i18n::tr("Presets & Profiles"),
&["lcpreset", "xmp", "lrtemplate", "zip", "dng", "lmp", "mplumpack", "cube"],
)
.pick_files()
.unwrap_or_default()
.into_iter()
@@ -177,24 +320,24 @@ fn services() -> Services {
})),
pick_tracklog: Some(Box::new(|| {
rfd::FileDialog::new()
.set_title("Auto-Tag from Tracklog")
.add_filter("GPS Track Log", &["gpx"])
.set_title(lightcraft_ui_egui::i18n::tr("Auto-Tag from Tracklog"))
.add_filter(lightcraft_ui_egui::i18n::tr("GPS Track Log"), &["gpx"])
.pick_file()
.map(|p| vec![p.to_string_lossy().to_string()])
.unwrap_or_default()
})),
save_preset_file: Some(Box::new(|name: &str| {
rfd::FileDialog::new()
.set_title("Export Presets")
.add_filter("LightCraft Preset", &["lcpreset"])
.set_title(lightcraft_ui_egui::i18n::tr("Export Presets"))
.add_filter(lightcraft_ui_egui::i18n::tr("LightCraft Preset"), &["lcpreset"])
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
})),
pick_curve_preset_files: Some(Box::new(|| {
rfd::FileDialog::new()
.set_title("Import Point Curve Presets")
.add_filter("Point Curve Presets", &["lccurve", "json"])
.set_title(lightcraft_ui_egui::i18n::tr("Import Point Curve Presets"))
.add_filter(lightcraft_ui_egui::i18n::tr("Point Curve Presets"), &["lccurve", "json"])
.pick_files()
.unwrap_or_default()
.into_iter()
@@ -203,8 +346,8 @@ fn services() -> Services {
})),
save_curve_preset_file: Some(Box::new(|name: &str| {
rfd::FileDialog::new()
.set_title("Export Point Curve Presets")
.add_filter("Point Curve Presets", &["lccurve"])
.set_title(lightcraft_ui_egui::i18n::tr("Export Point Curve Presets"))
.add_filter(lightcraft_ui_egui::i18n::tr("Point Curve Presets"), &["lccurve"])
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
@@ -215,18 +358,25 @@ fn services() -> Services {
png: Some(Box::new(|img: &lightcraft_raster::Rgba8| {
lightcraft_codecs::encode_png(&lightcraft_codecs::EncodeImage::rgba8(img), &lightcraft_codecs::EncodeMeta::default()).unwrap_or_default()
})),
// the library is a folder on disk: backed up with the user's other files
backup_library: None,
restore_library: None,
}
}
/// The persistent library session (or an in-memory one with `--memory` / if the library can't open).
fn open_session(in_memory: bool, dir: Option<std::path::PathBuf>, seed_demo: bool) -> Session {
let fallback = || if seed_demo { Session::with_demo() } else { Session::new() }.with_fs().with_system_clock();
/// The persistent library session, or with `--memory` an in-memory one (demo photos).
///
/// If the library can't be opened the session is empty and in memory — never seeded with the
/// demo photos, never written anywhere — and the problem is returned: the window then says so
/// and offers Try Again / Choose Another Library… / Continue Without Saving / Quit (issue #100).
fn open_session(in_memory: bool, dir: Option<std::path::PathBuf>, seed_demo: bool) -> (Session, Option<LibraryProblem>) {
if in_memory {
return fallback();
return (if seed_demo { Session::with_demo() } else { Session::new() }.with_fs().with_system_clock(), None);
}
let unopened = || Session::new().with_fs().with_system_clock();
let Some(dir) = dir else {
eprintln!("lightcraft: no library location (set --library or LIGHTCRAFT_LIBRARY); running in memory");
return fallback();
eprintln!("lightcraft: no library location (set --library or LIGHTCRAFT_LIBRARY)");
return (unopened(), Some(LibraryProblem::new("", "There is no home folder to keep the library in. Choose a folder for it.")));
};
let t0 = std::time::Instant::now();
let mut s = Session::new().with_fs().with_system_clock();
@@ -240,11 +390,11 @@ fn open_session(in_memory: bool, dir: Option<std::path::PathBuf>, seed_demo: boo
if torn > 0 { ", torn tail repaired" } else { "" },
t0.elapsed().as_secs_f64() * 1000.0
);
s
(s, None)
}
Err(e) => {
eprintln!("lightcraft: can't open library {}: {e}; running in memory", dir.display());
fallback()
eprintln!("lightcraft: can't open library {}: {e}", dir.display());
(unopened(), Some(LibraryProblem::new(dir.to_string_lossy(), e.to_string())))
}
}
}
@@ -261,6 +411,10 @@ OPTIONS:
--control PORT serve the JSON-lines control channel on 127.0.0.1:PORT (env LIGHTCRAFT_CONTROL_PORT;
see docs/control-protocol.md)
--version, --help
ENVIRONMENT:
LIGHTCRAFT_GPU_BACKEND=dx12|vulkan|metal|auto|off graphics backend (default: DX12 on Windows, Metal on macOS,
Vulkan on Linux; off = render on the CPU); else WGPU_BACKEND. LIGHTCRAFT_GPU=0: CPU rendering.
";
fn main() -> eframe::Result {
@@ -293,7 +447,7 @@ fn main() -> eframe::Result {
_ => files.push(a),
}
}
let prefs = load_prefs();
let (prefs, prefs_warning, keep_prefs_file) = load_prefs();
// --library, else the library last opened from Settings, else the default location
let library_dir = library_dir.or_else(|| {
prefs
@@ -303,6 +457,11 @@ fn main() -> eframe::Result {
.map(Into::into)
});
let library_dir = library_dir.or_else(lightcraft_engine::library::default_dir);
// GPU compute: off if the preference says so, or if the last launch died creating the device
// (issue #136) — before anything can create it
let gpu_crash = gpu_crash_check();
let gpu_on = prefs.as_ref().is_none_or(|u| u.settings.gpu) && gpu_crash.is_none();
lightcraft_engine::gpu::set_enabled(gpu_on);
let options = eframe::NativeOptions {
viewport: egui::ViewportBuilder::default()
.with_title("LightCraft")
@@ -315,23 +474,42 @@ fn main() -> eframe::Result {
.with_icon(app_icon())
// Wayland matches the window to packaging/linux/ai.storyteller.lightcraft.desktop by this id
.with_app_id(APP_ID),
wgpu_options: window_wgpu_options(),
..Default::default()
};
eframe::run_native(
"LightCraft",
options,
Box::new(move |cc| {
let session = open_session(in_memory, library_dir, seed_demo && files.is_empty());
let (session, problem) = open_session(in_memory, library_dir, seed_demo && files.is_empty());
let mut app = LightcraftApp::new(session, services());
if let Some(ui) = prefs {
app.ui = ui;
}
lightcraft_ui_egui::i18n::set_language(app.ui.language);
app.integrated_titlebar = cfg!(target_os = "macos");
app.notices.extend(prefs_warning);
// what's on disk now: only changes are written
let writer = PrefsWriter {
written: serde_json::to_vec_pretty(&app.ui).unwrap_or_default(),
library: app.ui.settings.library_path.clone(),
keep_file: keep_prefs_file,
..Default::default()
};
// after the snapshot above, so the writer saves the switched-off preference
if let Some(notice) = gpu_crash {
app.ui.settings.gpu = false;
app.notices.push(notice);
}
if let Some(port) = control_port {
let rx = control_server::start(port, cc.egui_ctx.clone());
app = app.with_control(rx);
}
if !files.is_empty() {
if let Some(mut p) = problem {
// imported once the user has chosen where (into the library, or the temporary session)
p.pending_import = files;
app.library_problem = Some(p);
} else if !files.is_empty() {
let _ = app.run("library.import", serde_json::json!({"paths": files}));
app.ui.view = lightcraft_ui_egui::state::ViewMode::PhotoGrid;
}
@@ -341,9 +519,90 @@ fn main() -> eframe::Result {
let menu = (std::env::var_os("LIGHTCRAFT_NO_NATIVE_MENU").is_none()).then(|| native_menu::NativeMenu::install(&mut app, &cc.egui_ctx));
Ok(Box::new(App(
app,
writer,
#[cfg(target_os = "macos")]
menu,
)))
}),
)
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::*;
fn dir(tag: &str) -> std::path::PathBuf {
let d = std::env::temp_dir().join(format!("lc-app-prefs-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
/// Issue #103: a damaged ui.json (which holds the library location) is kept aside and
/// reported, not silently replaced by the defaults.
#[test]
fn damaged_ui_json_is_kept_and_reported() {
let d = dir("damaged");
let path = d.join("ui.json");
std::fs::write(&path, br#"{"settings": {"library_path": "/Volumes/Photos/Lib"#).unwrap();
let (ui, warning, keep) = load_prefs_at(&path);
assert!(ui.is_none() && !keep);
let warning = warning.unwrap();
assert!(warning.contains("damaged") && warning.contains("ui.json.corrupt-"), "{warning}");
assert!(!path.exists());
let kept: Vec<_> = std::fs::read_dir(&d).unwrap().map(|e| e.unwrap().file_name().to_string_lossy().to_string()).collect();
assert!(kept.len() == 1 && kept[0].starts_with("ui.json.corrupt-"), "{kept:?}");
// a missing file is just "no settings yet"
assert_eq!(load_prefs_at(&path).1, None);
let _ = std::fs::remove_dir_all(&d);
}
/// Issue #136: a GPU init marker left by a crashed launch is reported once and removed.
#[test]
fn gpu_crash_marker_is_reported_once() {
let d = dir("gpu-marker");
let m = d.join("gpu-init.marker");
assert_eq!(lightcraft_engine::gpu::backend::take_init_marker(&m), None);
std::fs::write(&m, "GPU device creation started (backends VULKAN | DX12)").unwrap();
let what = lightcraft_engine::gpu::backend::take_init_marker(&m).unwrap();
let notice = gpu_crash_notice(&what);
assert!(notice.contains("GPU rendering is now off") && notice.contains("Use the GPU for rendering"), "{notice}");
assert!(!m.exists());
let _ = std::fs::remove_dir_all(&d);
}
/// Issue #136: Windows windows render with DX12 alone (no Vulkan driver loaded) by default.
#[test]
fn window_backends_follow_the_platform_default() {
let eframe::egui_wgpu::WgpuSetup::CreateNew(n) = window_wgpu_options().wgpu_setup else { panic!("expected CreateNew") };
let b = n.instance_descriptor.backends;
if std::env::var_os("LIGHTCRAFT_GPU_BACKEND").is_none() && std::env::var_os("WGPU_BACKEND").is_none() {
if cfg!(windows) {
assert_eq!(b, eframe::wgpu::Backends::DX12);
} else if cfg!(target_os = "macos") {
assert_eq!(b, eframe::wgpu::Backends::METAL);
}
}
assert!(!b.is_empty());
}
#[test]
fn ui_json_is_written_atomically() {
let d = dir("atomic");
let path = d.join("ui.json");
std::fs::write(&path, b"old").unwrap();
write_atomic(&path, b"{\"new\": true}").unwrap();
assert_eq!(std::fs::read(&path).unwrap(), b"{\"new\": true}");
assert!(!d.join("ui.json.tmp").exists());
// a failed write leaves the old file
let missing = d.join("gone").join("ui.json");
assert!(write_atomic(&missing, b"x").is_err());
assert_eq!(std::fs::read(&path).unwrap(), b"{\"new\": true}");
let ui = UiState::default();
write_atomic(&path, &serde_json::to_vec_pretty(&ui).unwrap()).unwrap();
let (loaded, warning, _) = load_prefs_at(&path);
assert!(loaded.is_some() && warning.is_none());
let _ = std::fs::remove_dir_all(&d);
}
}
+40 -22
View File
@@ -148,7 +148,7 @@ fn yields_to_text(sc: &str) -> bool {
/// `&` marks a mnemonic in muda labels.
fn label_text(s: &str) -> String {
s.replace('&', "&&")
lightcraft_ui_egui::i18n::tr(s).replace('&', "&&")
}
fn structure_of(bar: &[(String, Vec<MenuNode>)]) -> String {
@@ -171,7 +171,7 @@ fn structure_of(bar: &[(String, Vec<MenuNode>)]) -> String {
out.push(';');
}
}
let mut s = String::new();
let mut s = format!("{};", lightcraft_ui_egui::i18n::is_japanese());
for (t, items) in bar {
s.push_str(t);
walk(items, &mut s);
@@ -210,30 +210,30 @@ impl NativeMenu {
// the application menu
let app_menu = Submenu::new("LightCraft", true);
let about = MenuItem::with_id("app.about", "About LightCraft", true, None);
let settings = MenuItem::with_id(SETTINGS, app.ui.language.tr("Settings…"), true, accelerator(SETTINGS_KEY));
let quit = MenuItem::with_id(QUIT, "Quit LightCraft", true, accelerator("Cmd+Q"));
let about = MenuItem::with_id("app.about", lightcraft_ui_egui::i18n::tr("About LightCraft"), true, None);
let settings = MenuItem::with_id(SETTINGS, lightcraft_ui_egui::i18n::tr("Settings…"), true, accelerator(SETTINGS_KEY));
let quit = MenuItem::with_id(QUIT, lightcraft_ui_egui::i18n::tr("Quit LightCraft"), true, accelerator("Cmd+Q"));
let _ = app_menu.append_items(&[
&about,
&PredefinedMenuItem::separator(),
&settings,
&PredefinedMenuItem::separator(),
&PredefinedMenuItem::services(None),
&PredefinedMenuItem::services(Some(lightcraft_ui_egui::i18n::tr("Services"))),
&PredefinedMenuItem::separator(),
&PredefinedMenuItem::hide(Some("Hide LightCraft")),
&PredefinedMenuItem::hide_others(None),
&PredefinedMenuItem::show_all(None),
&PredefinedMenuItem::hide(Some(lightcraft_ui_egui::i18n::tr("Hide LightCraft"))),
&PredefinedMenuItem::hide_others(Some(lightcraft_ui_egui::i18n::tr("Hide Others"))),
&PredefinedMenuItem::show_all(Some(lightcraft_ui_egui::i18n::tr("Show All"))),
&PredefinedMenuItem::separator(),
&quit,
]);
let _ = self.menu.append(&app_menu);
for (title, nodes) in &bar {
let sub = Submenu::new(title, true);
if title == app.ui.language.tr("Window") {
let sub = Submenu::new(label_text(title), true);
if title == "Window" {
let _ = sub.append_items(&[
&PredefinedMenuItem::minimize(None),
&PredefinedMenuItem::maximize(Some("Zoom")),
&PredefinedMenuItem::minimize(Some(lightcraft_ui_egui::i18n::tr("Minimize"))),
&PredefinedMenuItem::maximize(Some(lightcraft_ui_egui::i18n::tr("Zoom"))),
&PredefinedMenuItem::separator(),
]);
}
@@ -245,15 +245,21 @@ impl NativeMenu {
.collect();
let nodes = tidy_separators(nodes);
self.append_nodes(&sub, &nodes);
if title == app.ui.language.tr("File") {
if title == "File" {
// ⌘W, the system's own item
let _ = sub.append_items(&[&PredefinedMenuItem::separator(), &PredefinedMenuItem::close_window(None)]);
let _ = sub.append_items(&[
&PredefinedMenuItem::separator(),
&PredefinedMenuItem::close_window(Some(lightcraft_ui_egui::i18n::tr("Close Window"))),
]);
}
if title == app.ui.language.tr("Window") {
let _ = sub.append_items(&[&PredefinedMenuItem::separator(), &PredefinedMenuItem::bring_all_to_front(None)]);
if title == "Window" {
let _ = sub.append_items(&[
&PredefinedMenuItem::separator(),
&PredefinedMenuItem::bring_all_to_front(Some(lightcraft_ui_egui::i18n::tr("Bring All to Front"))),
]);
sub.set_as_windows_menu_for_nsapp();
}
if title == app.ui.language.tr("Help") {
if title == "Help" {
sub.set_as_help_menu_for_nsapp();
}
let _ = self.menu.append(&sub);
@@ -282,12 +288,23 @@ impl NativeMenu {
let accel = shortcut.as_deref().filter(|s| !CONTEXTUAL.contains(s)).and_then(accelerator);
let handle = match checked {
Some(c) => {
let it = CheckMenuItem::with_id(key.clone(), label_text(label), *enabled, *c, accel);
let it = CheckMenuItem::with_id(
key.clone(),
lightcraft_ui_egui::menubar::display_item_label(id, params, label).replace('&', "&&"),
*enabled,
*c,
accel,
);
let _ = sub.append(&it);
Handle::Check(it)
}
None => {
let it = MenuItem::with_id(key.clone(), label_text(label), *enabled, accel);
let it = MenuItem::with_id(
key.clone(),
lightcraft_ui_egui::menubar::display_item_label(id, params, label).replace('&', "&&"),
*enabled,
accel,
);
let _ = sub.append(&it);
Handle::Plain(it)
}
@@ -325,6 +342,7 @@ impl NativeMenu {
/// Per frame: run chosen items, then sync labels / enabled / checked and the text-focus
/// accelerators with the app state.
pub fn update(&mut self, app: &mut LightcraftApp, ctx: &egui::Context) {
lightcraft_ui_egui::i18n::set_language(app.ui.language);
while let Ok(key) = self.rx.try_recv() {
if key == QUIT {
ctx.send_viewport_cmd(egui::ViewportCommand::Close);
@@ -352,8 +370,8 @@ impl NativeMenu {
let Some(it) = items.get_mut(&MenuNode::key(id, params)) else { continue };
if it.label != *label {
match &it.handle {
Handle::Plain(h) => h.set_text(label_text(label)),
Handle::Check(h) => h.set_text(label_text(label)),
Handle::Plain(h) => h.set_text(lightcraft_ui_egui::menubar::display_item_label(id, params, label).replace('&', "&&")),
Handle::Check(h) => h.set_text(lightcraft_ui_egui::menubar::display_item_label(id, params, label).replace('&', "&&")),
}
it.label = label.clone();
}
+5 -2
View File
@@ -9,8 +9,6 @@ covers a whole directory), or if a licence file referenced in the Licence column
| Path | Asset | Creator | Source | Licence | Added | Modifications |
|---|---|---|---|---|---|---|
| `assets/fonts/Inter-*.ttf` (Regular, Medium, SemiBold) | Inter | Rasmus Andersson | https://github.com/rsms/inter | SIL Open Font License 1.1 (`assets/fonts/OFL-Inter.txt`) | 2026-09-30 | none |
| `assets/fonts/BIZUDMincho-Regular.ttf` | BIZ UDMincho Regular | The BIZ UDMincho Project Authors (Morisawa Inc., design TypeBank Co., Ltd.) | https://github.com/google/fonts/tree/63833b7d10bb1f86a8f0b91cba2d3ae1f68d1aa3/ofl/bizudmincho | OFL-1.1 (`assets/fonts/OFL-BIZUDMincho.txt`) | 2026-10-05 | none |
| `assets/fonts/OFL-BIZUDMincho.txt` | BIZ UDMincho licence | The BIZ UDMincho Project Authors | https://github.com/google/fonts/blob/63833b7d10bb1f86a8f0b91cba2d3ae1f68d1aa3/ofl/bizudmincho/OFL.txt | OFL-1.1 | 2026-10-05 | none |
| `assets/app-icon/` (`lightcraft.svg` and the PNG/ICNS/ICO/SVG files derived from it) | LightCraft app icon (lynx) | LightCraft project owner (drawn in ArtCraft) | original work (vectorised from the owner's drawing; see `assets/app-icon/README.md`) | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) | 2026-10-02 | vectorised to SVG; rendered to PNG/ICNS/ICO by `packaging/icons.sh` |
| `crates/scenes/` (generated images) | Procedural demo photographs | LightCraft contributors | original work (generated at runtime, no source imagery) | MIT OR Apache-2.0 | 2026-09-30 | n/a |
| `crates/ui-egui/src/icons.rs` | UI icons drawn as vector paths in code | LightCraft contributors | original work | MIT OR Apache-2.0 | 2026-09-30 | n/a |
@@ -20,7 +18,12 @@ covers a whole directory), or if a licence file referenced in the Licence column
| `docs/images/*-blue-marble.jpg`, `docs/images/grid-pd.jpg` | "The Blue Marble" (AS17-148-22727) | NASA, Apollo 17 crew (1972) | https://commons.wikimedia.org/wiki/File:The_Earth_seen_from_Apollo_17.jpg | Public domain (NASA) | 2026-09-30 | developed in LightCraft; shown inside app screenshots |
| `docs/images/*.jpg` (UI) | LightCraft application screenshots | LightCraft contributors | original work (captured with `docs/showcase/`) | MIT OR Apache-2.0 | 2026-09-30 | n/a |
| `docs/brand/` (artcraft-logo and artcraft-mark, SVG and PNG) | ArtCraft wordmark and mark | ArtCraft Team | original work (https://getartcraft.com/) | ArtCraft trademark, see `docs/brand/LICENSE-brand.txt` (not open source) | 2026-10-01 | none |
| craft-fonts: optional build input, not files in this repository | BIZ UDPGothic (Regular, Bold), BIZ UDMincho Regular, Shippori Mincho Regular | Morisawa Inc. / The BIZ UDGothic and BIZ UDMincho Project Authors; The Shippori Mincho Project Authors (per craft-fonts' attribution) | https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md | SIL Open Font License 1.1 (each font's OFL.txt in craft-fonts; release packages ship it as OFL-<family>.txt) | 2026-10-06 | none; embedded only in builds made with `CRAFT_FONTS_DIR` (all official releases); the web (wasm32) build embeds BIZ UDPGothic Regular only |
Notes
- Fonts authored or published by Adobe (Source Sans/Serif/Code, Source Han, …) are not used, even though some are
OFL-licensed: the asset rule excludes anything from Adobe. Source Sans 3 was removed on 2026-09-30 and replaced by Inter.
- Japanese fonts (BIZ UDPGothic, BIZ UDMincho, Shippori Mincho) are not in this repository. They live in
[storytold/craft-fonts](https://github.com/storytold/craft-fonts), with their licences and attribution in its
`ATTRIBUTION.md`, and are embedded only in builds made with the optional `CRAFT_FONTS_DIR` build input (all official
releases). Never commit font files here; add new fonts to craft-fonts (craftrules `standards/fonts.md`).
Binary file not shown.
-93
View File
@@ -1,93 +0,0 @@
Copyright 2022 The BIZ UDMincho Project Authors (https://github.com/googlefonts/morisawa-biz-ud-mincho)
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
https://scripts.sil.org/OFL
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
+126 -21
View File
@@ -1,8 +1,9 @@
//! Crash-safe catalog persistence: an append-only op log plus periodic snapshots.
//!
//! Files (in a [`Store`]):
//! - `catalog.snap` — `{"format":"lightcraft-catalog","version":1,"seq":N,"catalog":{…}}`, the
//! state after op `N`; replaced atomically (temp + fsync + rename).
//! - `catalog.snap` — `{"format":"lightcraft-catalog","version":V,"seq":N,"catalog":{…}}`, the
//! state after op `N` in catalog format `V` ([`VERSION`]); replaced atomically (temp + fsync +
//! rename).
//! - `catalog.log` — JSON lines, one per op applied after the snapshot:
//! `{"seq":N,"crc":C,"op":{…}}` where `C` is the CRC-32 of the op's JSON text. Appends are
//! fsynced before [`Journal::append`] returns.
@@ -33,6 +34,32 @@
//!
//! At most one snapshot is in flight; a synchronous [`Journal::snapshot`] and dropping the journal
//! first wait for it, so an older snapshot can never replace a newer one.
//!
//! # Format versions
//!
//! The snapshot's `version` is the catalog format, and it covers the log beside it: the log has
//! no header of its own, and is only ever appended to under a snapshot of this build's format.
//!
//! | Version | Written by | Adds |
//! |---|---|---|
//! | (none) | early builds: a log without a snapshot | |
//! | 1 | up to v0.2.0 | |
//! | 2 | after v0.2.0 | `Op::SetBrowsed`, `Catalog.browsed`, `Photo.local_baseline` |
//!
//! Rules:
//! - **Bump [`VERSION`]** (and add a row above) in the change that adds an [`Op`] variant or a
//! serialized field to the catalog, a photo, an album, a stack or develop settings that an older
//! build would misread or silently drop. New fields must deserialize from older data
//! (`#[serde(default)]`), so newer builds keep reading every older format.
//! - **Opening an older format upgrades it**: right after loading, [`Journal::open`] writes a
//! snapshot in this build's format (also for a new library), before anything is appended — so a
//! library is never a mix, and older builds refuse it from then on instead of reading part of it.
//! [`LoadReport::upgraded_from`] says when that happened.
//! - **A newer format is refused, untouched**: a snapshot whose `version` is above [`VERSION`],
//! or a log record whose CRC matches (so it is exactly what was written) but whose op doesn't
//! parse, fails [`Journal::open`] with [`CatalogError::Newer`] before any file is modified — it
//! is never mistaken for a torn tail or damage. (Builds up to v0.2.0 refuse a v2 snapshot with
//! "unsupported format".)
use crate::store::Store;
use crate::{Catalog, CatalogError, Op, Result};
@@ -40,7 +67,9 @@ use crate::{Catalog, CatalogError, Op, Result};
pub const SNAPSHOT: &str = "catalog.snap";
pub const LOG: &str = "catalog.log";
const FORMAT: &str = "lightcraft-catalog";
const VERSION: u32 = 1;
/// The catalog format this build writes (and the newest it reads). See the module docs →
/// *Format versions*; bump it whenever an [`Op`] variant or a serialized field is added.
pub const VERSION: u32 = 2;
/// When [`Journal::wants_snapshot`] says it's time to compact the log.
#[derive(Clone, Copy, Debug)]
@@ -75,6 +104,9 @@ pub struct LoadReport {
pub damaged: Option<String>,
/// No catalog files existed (a new library).
pub created: bool,
/// The library was in an older catalog format (`0` = before format versions: a log without a
/// snapshot) and was rewritten in this version's format ([`VERSION`]).
pub upgraded_from: Option<u32>,
}
/// Where persistence time goes (reported by `library.info` → `persistence`; printed to stderr
@@ -180,18 +212,46 @@ pub fn encode_record(seq: u64, op: &Op) -> String {
format!("{{\"seq\":{seq},\"crc\":{crc},\"op\":{body}}}")
}
/// Parse one log line (without the newline). `None` if it's malformed or the CRC doesn't match.
/// Parse one log line (without the newline). `None` if it's malformed, the CRC doesn't match, or
/// the op is one this version doesn't know (see [`Journal::open`]).
pub fn decode_record(line: &str) -> Option<(u64, Op)> {
let rest = line.strip_prefix("{\"seq\":")?;
let (seq, rest) = rest.split_once(",\"crc\":")?;
let (crc, rest) = rest.split_once(",\"op\":")?;
let body = rest.strip_suffix('}')?;
let seq: u64 = seq.parse().ok()?;
let crc: u32 = crc.parse().ok()?;
if crc32fast::hash(body.as_bytes()) != crc {
return None;
match decode_line(line) {
Line::Record(seq, op) => Some((seq, op)),
Line::Newer(_) | Line::Bad => None,
}
serde_json::from_str(body).ok().map(|op| (seq, op))
}
/// One log line, decoded.
enum Line {
Record(u64, Op),
/// The CRC matches — the line is exactly what was written — but the op doesn't parse: it
/// was written by a newer LightCraft (an op or field this version doesn't know), not damaged.
Newer(u64),
/// Malformed or a CRC mismatch: torn or damaged.
Bad,
}
fn decode_line(line: &str) -> Line {
let parts = || {
let rest = line.strip_prefix("{\"seq\":")?;
let (seq, rest) = rest.split_once(",\"crc\":")?;
let (crc, rest) = rest.split_once(",\"op\":")?;
let body = rest.strip_suffix('}')?;
Some((seq.parse::<u64>().ok()?, crc.parse::<u32>().ok()?, body))
};
let Some((seq, crc, body)) = parts() else { return Line::Bad };
if crc32fast::hash(body.as_bytes()) != crc {
return Line::Bad;
}
match serde_json::from_str(body) {
Ok(op) => Line::Record(seq, op),
Err(_) => Line::Newer(seq),
}
}
/// The error for a library written by a newer LightCraft.
fn newer(what: String) -> CatalogError {
CatalogError::Newer(format!("{what}; this version reads catalog format v{VERSION} and older"))
}
/// A line that doesn't decode but ends with a whole record: a fragment of a failed append followed
@@ -229,10 +289,16 @@ fn write_snapshot(store: &mut dyn Store, seq: u64, catalog: &Catalog) -> std::io
Ok(SnapshotTiming { serialize_ms, write_sync_ms: (ms_since(t0) - serialize_ms).max(0.0), bytes, ..Default::default() })
}
/// Just the identification of `catalog.snap` (read before the catalog itself).
#[derive(serde::Deserialize)]
struct SnapFile {
struct SnapHeader {
format: String,
version: u32,
}
/// The whole `catalog.snap` (after [`SnapHeader`] was checked; `format` and `version` are ignored).
#[derive(serde::Deserialize)]
struct SnapFile {
seq: u64,
catalog: Catalog,
}
@@ -245,15 +311,20 @@ impl Journal {
let snap = store.read(SNAPSHOT).map_err(io)?;
let log = store.read(LOG).map_err(io)?;
report.created = snap.is_none() && log.is_none();
let (mut catalog, snapshot_seq) = match snap {
let (mut catalog, snapshot_seq, snapshot_version) = match snap {
Some(bytes) => {
let s: SnapFile = serde_json::from_slice(&bytes).map_err(|e| CatalogError::Corrupt(format!("{SNAPSHOT}: {e}")))?;
if s.format != FORMAT || s.version > VERSION {
return Err(CatalogError::Corrupt(format!("{SNAPSHOT}: unsupported format {} v{}", s.format, s.version)));
// the header first: a newer snapshot may not parse as this version's catalog
let h: SnapHeader = serde_json::from_slice(&bytes).map_err(|e| CatalogError::Corrupt(format!("{SNAPSHOT}: {e}")))?;
if h.format != FORMAT {
return Err(CatalogError::Corrupt(format!("{SNAPSHOT}: not a LightCraft catalog (format {:?})", h.format)));
}
(s.catalog, s.seq)
if h.version > VERSION {
return Err(newer(format!("{SNAPSHOT} is catalog format v{}", h.version)));
}
let s: SnapFile = serde_json::from_slice(&bytes).map_err(|e| CatalogError::Corrupt(format!("{SNAPSHOT}: {e}")))?;
(s.catalog, s.seq, Some(h.version))
}
None => (Catalog::new(), 0),
None => (Catalog::new(), 0, None),
};
report.snapshot_seq = snapshot_seq;
let mut j = Journal {
@@ -296,6 +367,11 @@ impl Journal {
report.torn_fragments += 1;
record = Some(r);
}
if record.is_none()
&& let Some(Line::Newer(seq)) = line.map(decode_line)
{
return Err(newer(format!("{LOG} record {seq} holds a change this version doesn't know")));
}
match record {
Some((seq, op)) if seq <= j.seq => {
// already in the snapshot
@@ -312,7 +388,15 @@ impl Journal {
}
_ => {
// bad record (or a gap): torn tail if nothing good follows, else damage
let rest_has_good = log[next..].split(|b| *b == b'\n').any(|l| std::str::from_utf8(l).ok().and_then(decode_record).is_some());
let mut rest_has_good = false;
for l in log[next..].split(|b| *b == b'\n') {
match std::str::from_utf8(l).map(|l| decode_line(l.trim_end_matches('\r'))) {
Ok(Line::Record(..)) => rest_has_good = true,
// refused before anything is modified
Ok(Line::Newer(seq)) => return Err(newer(format!("{LOG} record {seq} holds a change this version doesn't know"))),
Ok(Line::Bad) | Err(_) => {}
}
}
if rest_has_good {
damaged_at = Some(pos);
}
@@ -344,6 +428,27 @@ impl Journal {
j.log_bytes = good_end as u64;
}
}
// Bring an older (or new, or snapshot-less) library to this version's format before
// anything is appended: a snapshot in this format makes older builds refuse the library
// instead of misreading ops or dropping fields they don't know.
if report.created {
// tiny: written directly, not counted as a compaction
let empty = format!("{{\"format\":\"{FORMAT}\",\"version\":{VERSION},\"seq\":0,\"catalog\":{}}}\n", catalog.to_snapshot());
if let Err(e) = j.store.write_atomic(SNAPSHOT, empty.as_bytes()) {
log::warn!("catalog: can't write the new library's snapshot: {e}");
}
} else if report.damaged.is_none() && snapshot_version != Some(VERSION) {
report.upgraded_from = Some(snapshot_version.unwrap_or(0));
match j.snapshot(&catalog) {
Ok(()) => {
if let Some(v) = report.upgraded_from {
log::info!("catalog: upgraded from format v{v} to v{VERSION}");
}
}
// nothing lost: the old files are intact and still load; retried on next open
Err(e) => log::warn!("catalog: can't write the format v{VERSION} snapshot: {e}"),
}
}
catalog.revision = 0;
Ok((j, catalog, report))
}
+15
View File
@@ -5,6 +5,11 @@
//! snapshot — crash-safe and diff-friendly (see [`journal`]);
//! - **undo/redo**: the engine keeps inverse ops;
//! - **determinism**: replaying the log reproduces the state exactly (property-tested).
//!
//! **Catalog format version** ([`journal::VERSION`], see [`journal`] → *Format versions*): adding
//! an [`Op`] variant or a serialized field means bumping it. Newer builds read every older format
//! (and upgrade it on open); older builds refuse a newer library with [`CatalogError::Newer`]
//! instead of reading part of it.
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
@@ -12,6 +17,7 @@ pub mod dates;
pub mod journal;
pub mod keywords;
pub mod local;
pub mod lock;
pub mod model;
pub mod query;
pub mod rules;
@@ -27,6 +33,7 @@ pub use journal::{Journal, LoadReport, PersistStats, SnapshotPolicy, SnapshotTim
pub use keywords::KeywordNode;
use lightcraft_develop::DevelopSettings;
pub use local::{DEFAULT_FORGET_DAYS, ForgetPlan, folder_of};
pub use lock::{LibraryLock, LockError, LockOwner};
pub use model::*;
pub use query::{DateGroup, Filter, RatingOp, Sort, SortKey};
pub use rules::{Match, Rule, RuleSet};
@@ -47,6 +54,10 @@ pub enum CatalogError {
Corrupt(String),
#[error("catalog storage: {0}")]
Io(String),
/// The library was written by a newer LightCraft (a newer catalog format, or a change this
/// version doesn't know). Nothing was read into the session and nothing was modified.
#[error("this library was written by a newer version of LightCraft ({0}); update LightCraft to open it. The library was left unchanged.")]
Newer(String),
}
pub type Result<T> = std::result::Result<T, CatalogError>;
@@ -644,8 +655,12 @@ mod tests;
#[cfg(test)]
mod tests_background;
#[cfg(test)]
mod tests_format_version;
#[cfg(test)]
mod tests_journal;
#[cfg(test)]
mod tests_local;
#[cfg(test)]
mod tests_lock;
#[cfg(test)]
mod tests_torn_append;
+140
View File
@@ -0,0 +1,140 @@
//! One process per library: an exclusive OS lock on `<library>/catalog.lock`, held for as long as
//! the library is open (issue #99).
//!
//! Two processes on the same library (the app and `lightcraft-cli --library`, two app instances,
//! two computers on a shared folder) would each keep their own catalog and `seq` counter, and
//! whichever wrote the last snapshot would silently drop the other's edits. So opening a library
//! takes the lock first, and a second opener gets [`LockError::InUse`] instead.
//!
//! - The lock is the OS's (`flock` on Unix, `LockFileEx` on Windows, via [`std::fs::File::try_lock`]):
//! it is released when the [`LibraryLock`] is dropped and, after a crash or a kill, by the OS when
//! the process ends. A `catalog.lock` file left behind is therefore never "stale": only a lock
//! that's actually held refuses the open.
//! - Who holds it (process id, computer, program, version) is written to `catalog.lock.owner`, for
//! the error message. It's a separate file because a Windows lock also blocks reading the
//! locked file. It's only a hint: it may be missing or left over from a crash.
//! - File systems without lock support (some network shares): the library opens unlocked, with a
//! warning — refusing it would lock the user out of their own library. Locks over SMB/NFS are
//! advisory and depend on the server; the owner file is then the only hint.
//! - Browser storage (OPFS) has no file locks: see the web host for its own guard.
use std::fs::{File, OpenOptions, TryLockError};
use std::io;
use std::path::{Path, PathBuf};
/// The lock file in the library directory.
pub const LOCK: &str = "catalog.lock";
/// Who holds the lock (JSON, see [`LockOwner`]).
pub const OWNER: &str = "catalog.lock.owner";
/// Who has the library open (as written in [`OWNER`]).
#[derive(Clone, Debug, Default, PartialEq, serde::Serialize, serde::Deserialize)]
#[serde(default)]
pub struct LockOwner {
pub pid: u32,
/// Computer name, if known.
pub host: String,
/// The program (`LightCraft`, `lightcraft-cli`…) and its version.
pub program: String,
pub version: String,
/// When it opened the library (seconds since 1970).
pub since: u64,
}
impl LockOwner {
/// This process, as `program`.
pub fn this_process(program: &str) -> LockOwner {
LockOwner {
pid: std::process::id(),
host: host_name(),
program: program.to_string(),
version: env!("CARGO_PKG_VERSION").to_string(),
since: web_time::SystemTime::now().duration_since(web_time::UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0),
}
}
/// "LightCraft 0.3.0 (process 123 on studio-mac)".
pub fn describe(&self) -> String {
let program = if self.program.is_empty() { "another LightCraft" } else { self.program.as_str() };
let version = if self.version.is_empty() { String::new() } else { format!(" {}", self.version) };
let host = if self.host.is_empty() {
String::new()
} else if self.host == host_name() {
" on this computer".to_string()
} else {
format!(" on {}", self.host)
};
format!("{program}{version} (process {}{host})", self.pid)
}
}
/// The computer's name, best effort (empty if unknown).
fn host_name() -> String {
let env = ["COMPUTERNAME", "HOSTNAME"].iter().find_map(|k| std::env::var(k).ok().filter(|h| !h.trim().is_empty()));
env.or_else(|| std::fs::read_to_string("/etc/hostname").ok()).map(|h| h.trim().to_string()).filter(|h| !h.is_empty()).unwrap_or_default()
}
#[derive(Debug, thiserror::Error)]
pub enum LockError {
/// Another process has the library open.
#[error("This library is already open in {}. Close it there first.", .0.as_ref().map(LockOwner::describe).unwrap_or_else(|| "another LightCraft program".into()))]
InUse(Option<LockOwner>),
}
/// The exclusive lock on a library directory; released on drop (and by the OS if the process
/// dies).
#[derive(Debug)]
pub struct LibraryLock {
dir: PathBuf,
/// `None` when the file system can't lock (the library is open unlocked).
_file: Option<File>,
held: bool,
}
impl LibraryLock {
/// Lock `dir` (which must exist) for this process. Fails only when another process holds it.
pub fn acquire(dir: &Path, program: &str) -> Result<LibraryLock, LockError> {
let unlocked = |why: io::Error| {
log::warn!("library {}: can't lock {LOCK} ({why}); opening it without protection against a second program", dir.display());
LibraryLock { dir: dir.to_path_buf(), _file: None, held: false }
};
let file = match OpenOptions::new().read(true).write(true).create(true).truncate(false).open(dir.join(LOCK)) {
Ok(f) => f,
Err(e) => return Ok(unlocked(e)),
};
match file.try_lock() {
Ok(()) => {}
Err(TryLockError::WouldBlock) => {
let owner = std::fs::read(dir.join(OWNER)).ok().and_then(|b| serde_json::from_slice::<LockOwner>(&b).ok());
return Err(LockError::InUse(owner));
}
Err(TryLockError::Error(e)) => return Ok(unlocked(e)),
}
let owner = LockOwner::this_process(program);
if let Ok(json) = serde_json::to_vec_pretty(&owner)
&& let Err(e) = std::fs::write(dir.join(OWNER), json)
{
log::warn!("library {}: can't write {OWNER}: {e}", dir.display());
}
Ok(LibraryLock { dir: dir.to_path_buf(), _file: Some(file), held: true })
}
/// The OS lock is held (`false`: the file system can't lock; opened unprotected).
pub fn held(&self) -> bool {
self.held
}
pub fn dir(&self) -> &Path {
&self.dir
}
}
impl Drop for LibraryLock {
/// Remove the owner note while still holding the lock (so it can't delete the next holder's),
/// then the file handle closes and the OS releases the lock.
fn drop(&mut self) {
if self.held {
let _ = std::fs::remove_file(self.dir.join(OWNER));
}
}
}
+7 -1
View File
@@ -331,6 +331,11 @@ impl Photo {
pub fn develops_raw(&self) -> bool {
self.kind == MediaKind::Raw && self.preview_only.is_none()
}
/// The current ARW reader has vendor WB multipliers but no measured camera illuminant.
/// Use adjustments relative to the camera's as-shot look, as for rendered photographs.
pub fn relative_wb(&self) -> bool {
self.develops_raw() && self.format.eq_ignore_ascii_case("ARW")
}
/// The develop settings import gave this photo: [`Photo::camera_defaults`], or the user's
/// default preset applied on top of them ([`Photo::import_look`]).
pub fn import_defaults(&self) -> DevelopSettings {
@@ -342,7 +347,8 @@ impl Photo {
/// The built-in defaults for this photo, before any user default preset: raws start from
/// their as-shot white balance; embedded lens corrections on when the file has them.
pub fn camera_defaults(&self) -> DevelopSettings {
let mut d = match self.as_shot_wb {
let wb = if self.relative_wb() { Some((6500.0, 0.0)) } else { self.as_shot_wb };
let mut d = match wb {
Some((t, tint)) => DevelopSettings::for_raw(t, tint),
None => DevelopSettings::default(),
};
+79 -8
View File
@@ -1,11 +1,23 @@
//! Crash-safe writes of user-visible files: exports, renders, DNG conversions, smart previews and
//! XMP sidecars all go through here (the catalog's own files use [`crate::store::FsStore`]).
//!
//! **Write policy** (issue #134): what can't be recreated — originals, XMP sidecars, the catalog,
//! settings, DNGs that replace raws, smart previews (the only stand-in while an original is
//! offline), an edit copy that becomes a library photo — is written **durably**: synced to disk
//! before it counts as written. What can always be made again from the originals — exports,
//! renders, screenshots, merge previews — is written **atomically** only: a half-written file
//! never replaces a good one, but there is no per-file sync (on a USB drive or a NAS that sync
//! dominates a large export).
//!
//! - [`write_atomic`] replaces a file so that after a crash, a full disk or an unplugged drive
//! either the old or the complete new content is there — never a truncated mix. The content
//! goes to a new temp file in the same folder (unique name, `create_new`), is synced to disk and
//! renamed over the target; then the folder is synced (POSIX). Any failure removes the temp
//! file and leaves the target as it was.
//! - [`write_atomic_nosync`] is the same temp file + rename without the two syncs, for
//! recreatable outputs only: a failed write still removes the temp file and leaves the target as
//! it was, but after a power cut the new file may be lost (on some file systems left empty) —
//! it is then simply exported again.
//! - [`write_new`] / [`write_new_unique`] write a file that must not exist yet: the same temp file
//! and sync, plus a read-back check, and the file then appears under its final name **without
//! replacing anything** (a hard link, which fails on a taken name; a check-then-rename where the
@@ -21,6 +33,19 @@ use std::sync::atomic::{AtomicU64, Ordering};
thread_local! {
static FAIL_AFTER: Cell<Option<u64>> = const { Cell::new(None) };
static SYNCS: Cell<u64> = const { Cell::new(0) };
}
/// How many file and folder syncs this module made **on the calling thread** (for tests: a
/// durable write path syncs, a recreatable output doesn't).
#[doc(hidden)]
pub fn syncs_on_this_thread() -> u64 {
SYNCS.with(Cell::get)
}
fn sync_file(f: &File) -> io::Result<()> {
SYNCS.with(|c| c.set(c.get() + 1));
f.sync_all()
}
/// Fault injection for tests (in any crate): while the returned guard lives, every write made by
@@ -78,7 +103,7 @@ fn parent_of(path: &Path) -> &Path {
/// where NTFS journals the rename itself, so errors are ignored.
pub fn sync_dir(dir: &Path) {
if let Ok(d) = File::open(dir) {
let _ = d.sync_all();
let _ = sync_file(&d);
}
}
@@ -100,9 +125,9 @@ fn create_temp(path: &Path) -> io::Result<(PathBuf, File)> {
Err(io::Error::new(io::ErrorKind::AlreadyExists, format!("no free temp file name in {}", dir.display())))
}
/// Write a temp file next to `path` with what `fill` writes, synced to disk. On failure the temp
/// file is removed. Returns its path and length.
fn write_temp(path: &Path, fill: &mut dyn FnMut(&mut dyn Write) -> io::Result<()>) -> io::Result<(PathBuf, u64)> {
/// Write a temp file next to `path` with what `fill` writes, synced to disk when `sync`. On
/// failure the temp file is removed. Returns its path and length.
fn write_temp(path: &Path, sync: bool, fill: &mut dyn FnMut(&mut dyn Write) -> io::Result<()>) -> io::Result<(PathBuf, u64)> {
let (tmp, file) = create_temp(path)?;
let r = (|| {
let limit = FAIL_AFTER.with(Cell::get);
@@ -111,7 +136,9 @@ fn write_temp(path: &Path, fill: &mut dyn FnMut(&mut dyn Write) -> io::Result<()
w.flush()?;
let n = w.n;
let f = w.inner.into_inner().map_err(|e| e.into_error())?;
f.sync_all()?;
if sync {
sync_file(&f)?;
}
Ok(n)
})();
match r {
@@ -132,12 +159,26 @@ pub fn write_atomic(path: &Path, data: &[u8]) -> io::Result<()> {
/// [`write_atomic`] with streamed content: `fill` writes it (an error aborts and leaves the old
/// file). Returns the length written.
pub fn write_atomic_with(path: &Path, fill: &mut dyn FnMut(&mut dyn Write) -> io::Result<()>) -> io::Result<u64> {
let (tmp, n) = write_temp(path, fill)?;
replace_with(path, true, fill)
}
/// Replace (or create) `path` with `data` atomically but **without syncing** it to disk — only for
/// outputs that can always be recreated (exports, renders, screenshots); see the module docs.
/// Anything that can't be made again goes through [`write_atomic`].
pub fn write_atomic_nosync(path: &Path, data: &[u8]) -> io::Result<()> {
replace_with(path, false, &mut |w| w.write_all(data)).map(|_| ())
}
/// Temp file (synced when `sync`) renamed over `path`, then the folder synced when `sync`.
fn replace_with(path: &Path, sync: bool, fill: &mut dyn FnMut(&mut dyn Write) -> io::Result<()>) -> io::Result<u64> {
let (tmp, n) = write_temp(path, sync, fill)?;
if let Err(e) = fs::rename(&tmp, path) {
let _ = fs::remove_file(&tmp);
return Err(e);
}
sync_dir(parent_of(path));
if sync {
sync_dir(parent_of(path));
}
Ok(n)
}
@@ -155,8 +196,11 @@ pub fn write_new_unique(candidates: &mut dyn Iterator<Item = PathBuf>, data: &[u
let Some(first) = candidates.peek().cloned() else {
return Err(io::Error::new(io::ErrorKind::InvalidInput, "no file name to write"));
};
let (tmp, _) = write_temp(&first, &mut |w| w.write_all(data))?;
let (tmp, _) = write_temp(&first, true, &mut |w| w.write_all(data))?;
let r = (|| {
// Kept (issue #134 point 4): these files replace raws (Convert / Copy as DNG) or become
// library photos (merges). On a local disk the read usually comes from the cache, but on a
// network volume it can go back to the server, and next to encoding a DNG it costs little.
if !same_content(&tmp, data)? {
return Err(io::Error::other("the written file reads back different (failing drive or connection?)"));
}
@@ -287,6 +331,33 @@ mod tests {
let _ = fs::remove_dir_all(&d);
}
#[test]
fn recreatable_outputs_are_atomic_without_a_sync() {
let d = temp("nosync");
let p = d.join("export.jpg");
let before = syncs_on_this_thread();
write_atomic_nosync(&p, b"one").unwrap();
write_atomic_nosync(&p, b"two two").unwrap();
assert_eq!(syncs_on_this_thread(), before, "no file or folder sync");
assert_eq!(fs::read(&p).unwrap(), b"two two");
assert_eq!(names(&d), vec!["export.jpg"], "no temp file left");
// still atomic: a failure part-way leaves the previous file and no temp file
{
let _f = fail_writes_after(3);
assert!(write_atomic_nosync(&p, &[9u8; 10_000]).is_err());
}
assert_eq!(fs::read(&p).unwrap(), b"two two");
assert_eq!(names(&d), vec!["export.jpg"]);
// the durable writers do sync (the file; the folder too where it can be opened)
let before = syncs_on_this_thread();
write_atomic(&p, b"three").unwrap();
let mid = syncs_on_this_thread();
assert!(mid > before);
write_new(&d.join("new.dng"), b"dng").unwrap();
assert!(syncs_on_this_thread() > mid);
let _ = fs::remove_dir_all(&d);
}
#[test]
fn new_files_never_replace_anything() {
let d = temp("new");
+2 -1
View File
@@ -180,7 +180,8 @@ fn failed_background_snapshot_loses_nothing_and_retries_later() {
wait_written(&j);
assert!(j.poll().is_err());
assert_eq!((j.snapshot_seq(), j.stats().failed_snapshots), (0, 1));
assert!(g.files.get(SNAPSHOT).is_none());
// still the (empty) snapshot written when the library was created
assert_eq!(g.snapshot_seq_on_disk(), Some(0));
assert_recovers(&g, &live);
// no retry storm: the next attempt waits for another `max_records` records
assert!(!j.wants_snapshot());
+186
View File
@@ -0,0 +1,186 @@
//! Issue #102: catalog format versions. Older formats load and are upgraded; a newer format is
//! refused with a clear error and left untouched (never read in part, truncated or set aside).
use crate::journal::{LOG, SNAPSHOT, VERSION, encode_record};
use crate::*;
fn photo(c: &mut Catalog) -> Op {
let id = c.alloc_photo_id();
Op::AddPhoto { photo: Box::new(Photo::new(id, Source::File { path: format!("/p/{}.jpg", id.0) }, "p.jpg", "JPEG", 4, 3, "2026-01-01")) }
}
/// A catalog with two photos and the log line for a third op on top of it.
fn legacy_parts() -> (Catalog, String, Catalog) {
let mut c = Catalog::new();
for _ in 0..2 {
let op = photo(&mut c);
c.apply(op).unwrap();
}
let base = c.clone();
let id = c.photos().next().unwrap().id;
let op = Op::SetRating { id, rating: 4 };
c.apply(op.clone()).unwrap();
(base, format!("{}\n", encode_record(3, &op)), c)
}
fn snapshot_version(m: &MemStore) -> u64 {
let v: serde_json::Value = serde_json::from_slice(&m.get(SNAPSHOT).unwrap()).unwrap();
v["version"].as_u64().unwrap()
}
#[test]
fn v1_library_loads_and_is_upgraded() {
let (base, log, full) = legacy_parts();
let m = MemStore::new();
m.set(SNAPSHOT, format!("{{\"format\":\"lightcraft-catalog\",\"version\":1,\"seq\":2,\"catalog\":{}}}\n", base.to_snapshot()).into_bytes());
m.set(LOG, log.into_bytes());
let (mut j, c, r) = Journal::open(Box::new(m.clone())).unwrap();
assert_eq!(c.to_snapshot(), full.to_snapshot());
assert_eq!((r.replayed, r.upgraded_from), (1, Some(1)));
// rewritten in this format before anything is appended
assert_eq!(snapshot_version(&m), u64::from(VERSION));
assert!(m.get(LOG).unwrap().is_empty());
assert_eq!(j.seq(), 3);
j.append(&[Op::SetLabelName { label: ColorLabel::ALL[0], name: Some("x".into()) }]).unwrap();
drop(j);
let (_, _, r) = Journal::open(Box::new(m)).unwrap();
assert_eq!((r.replayed, r.upgraded_from), (1, None));
}
#[test]
fn versionless_log_only_library_is_upgraded() {
let mut c = Catalog::new();
let op = photo(&mut c);
c.apply(op.clone()).unwrap();
let m = MemStore::new();
m.set(LOG, format!("{}\n", encode_record(1, &op)).into_bytes());
let (_, loaded, r) = Journal::open(Box::new(m.clone())).unwrap();
assert_eq!(loaded.to_snapshot(), c.to_snapshot());
assert_eq!(r.upgraded_from, Some(0));
assert_eq!(snapshot_version(&m), u64::from(VERSION));
}
/// A new library gets a snapshot in this format at once, so an older build never sees a bare log
/// it would read in part.
#[test]
fn new_library_is_written_in_this_format() {
let m = MemStore::new();
let (_, _, r) = Journal::open(Box::new(m.clone())).unwrap();
assert!(r.created);
assert_eq!(r.upgraded_from, None);
assert_eq!(snapshot_version(&m), u64::from(VERSION));
}
fn assert_refused_untouched(m: &MemStore) {
let before = m.files.lock().unwrap().clone();
let e = Journal::open(Box::new(m.clone())).err().expect("refused");
assert!(matches!(e, CatalogError::Newer(_)), "{e:?}");
let msg = e.to_string();
assert!(msg.contains("newer version of LightCraft") && msg.contains("left unchanged"), "{msg}");
assert_eq!(*m.files.lock().unwrap(), before, "nothing modified, nothing renamed");
}
#[test]
fn newer_snapshot_is_refused_untouched() {
let (base, log, _) = legacy_parts();
let m = MemStore::new();
m.set(
SNAPSHOT,
format!("{{\"format\":\"lightcraft-catalog\",\"version\":{},\"seq\":2,\"catalog\":{}}}\n", VERSION + 1, base.to_snapshot()).into_bytes(),
);
m.set(LOG, log.into_bytes());
assert_refused_untouched(&m);
// even when its catalog doesn't parse as this version's
let m = MemStore::new();
m.set(
SNAPSHOT,
format!("{{\"format\":\"lightcraft-catalog\",\"version\":{},\"seq\":2,\"catalog\":{{\"photos\":7}}}}\n", VERSION + 1).into_bytes(),
);
assert_refused_untouched(&m);
let e = Journal::open(Box::new(m)).err().unwrap().to_string();
assert!(e.contains(&format!("v{}", VERSION + 1)), "{e}");
}
/// A log record with a valid CRC whose op this version doesn't know: refused, not taken for a
/// torn tail (truncated) or damage (renamed) — wherever it is in the log.
#[test]
fn unknown_op_in_the_log_is_refused_untouched() {
let (base, log, _) = legacy_parts();
let future = |seq: u64| {
let body = r#"{"op":"fromTheFuture","id":1,"what":"something new"}"#;
format!("{{\"seq\":{seq},\"crc\":{},\"op\":{body}}}\n", crc32fast::hash(body.as_bytes()))
};
let snap = format!("{{\"format\":\"lightcraft-catalog\",\"version\":{VERSION},\"seq\":2,\"catalog\":{}}}\n", base.to_snapshot());
let id = base.photos().next().unwrap().id;
let after = format!("{}\n", encode_record(5, &Op::SetRating { id, rating: 1 }));
for log in [format!("{log}{}", future(4)), format!("{log}{}{after}", future(4)), format!("{}{log}", future(3))] {
let m = MemStore::new();
m.set(SNAPSHOT, snap.clone().into_bytes());
m.set(LOG, log.into_bytes());
assert_refused_untouched(&m);
}
// a known op with a field of an unknown shape is "newer" too
let body = r#"{"op":"setRating","id":1,"rating":"five stars"}"#;
let m = MemStore::new();
m.set(SNAPSHOT, snap.into_bytes());
m.set(LOG, format!("{log}{{\"seq\":4,\"crc\":{},\"op\":{body}}}\n", crc32fast::hash(body.as_bytes())).into_bytes());
assert_refused_untouched(&m);
}
/// A CRC mismatch is still damage / a torn tail (not "newer").
#[test]
fn bad_crc_is_still_a_torn_tail() {
let (base, log, _) = legacy_parts();
let m = MemStore::new();
m.set(
SNAPSHOT,
format!("{{\"format\":\"lightcraft-catalog\",\"version\":{VERSION},\"seq\":2,\"catalog\":{}}}\n", base.to_snapshot()).into_bytes(),
);
let body = r#"{"op":"fromTheFuture"}"#;
m.set(LOG, format!("{log}{{\"seq\":4,\"crc\":1,\"op\":{body}}}\n").into_bytes());
let (_, _, r) = Journal::open(Box::new(m)).unwrap();
assert!(r.torn_bytes > 0 && r.damaged.is_none(), "{r:?}");
}
/// Guard for the version rule (see `journal` → *Format versions*): adding an [`Op`] variant breaks
/// this match. When it does, bump `journal::VERSION`, add a row to the format table, and add the
/// variant here under the new version.
#[test]
fn op_variants_are_versioned() {
fn since(op: &Op) -> u32 {
match op {
Op::AddPhoto { .. }
| Op::RemovePhoto { .. }
| Op::SetRating { .. }
| Op::SetFlag { .. }
| Op::SetLabel { .. }
| Op::SetDevelop { .. }
| Op::SetMeta { .. }
| Op::SetDeleted { .. }
| Op::SetLocal { .. }
| Op::SetVersions { .. }
| Op::SetHistory { .. }
| Op::PushHistory { .. }
| Op::AddAlbum { .. }
| Op::RemoveAlbum { .. }
| Op::RenameAlbum { .. }
| Op::MoveAlbum { .. }
| Op::SetAlbumPhotos { .. }
| Op::SetAlbumCover { .. }
| Op::SetAlbumRules { .. }
| Op::AddStack { .. }
| Op::RemoveStack { .. }
| Op::SetStack { .. }
| Op::SetCaptured { .. }
| Op::SetAnalysis { .. }
| Op::SetFile { .. }
| Op::Relink { .. }
| Op::SetContent { .. }
| Op::SetLabelName { .. }
| Op::Batch { .. } => 1,
Op::SetBrowsed { .. } => 2,
}
}
let newest = since(&Op::SetBrowsed { folder: String::new(), at: None });
assert_eq!(newest, VERSION, "the newest op's version must be the current format version");
}
+6 -1
View File
@@ -283,7 +283,12 @@ fn streamed_snapshot_keeps_the_on_disk_format() {
let (mut j, c, _) = open(&m);
assert_eq!(c.to_snapshot(), full.to_snapshot());
j.snapshot(&c).unwrap();
let legacy = format!("{{\"format\":\"lightcraft-catalog\",\"version\":1,\"seq\":{},\"catalog\":{}}}\n", j.seq(), c.to_snapshot());
let legacy = format!(
"{{\"format\":\"lightcraft-catalog\",\"version\":{},\"seq\":{},\"catalog\":{}}}\n",
crate::journal::VERSION,
j.seq(),
c.to_snapshot()
);
assert_eq!(String::from_utf8(m.get(SNAPSHOT).unwrap()).unwrap(), legacy);
assert_eq!(j.stats().last_snapshot.bytes, legacy.len() as u64);
+93
View File
@@ -0,0 +1,93 @@
//! Issue #99: one process per library.
use std::path::PathBuf;
use crate::lock::{LOCK, OWNER};
use crate::*;
fn temp_dir(tag: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("lc-lock-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
#[test]
fn second_opener_is_refused_until_the_first_lets_go() {
let dir = temp_dir("twice");
let first = LibraryLock::acquire(&dir, "LightCraft").unwrap();
assert!(first.held());
let owner: LockOwner = serde_json::from_slice(&std::fs::read(dir.join(OWNER)).unwrap()).unwrap();
assert_eq!((owner.pid, owner.program.as_str()), (std::process::id(), "LightCraft"));
let e = LibraryLock::acquire(&dir, "lightcraft-cli").unwrap_err();
let LockError::InUse(Some(who)) = &e else { panic!("{e:?}") };
assert_eq!(who.pid, std::process::id());
let msg = e.to_string();
assert!(msg.contains("already open in LightCraft") && msg.contains(&format!("process {}", std::process::id())), "{msg}");
drop(first);
assert!(!dir.join(OWNER).exists(), "the owner note goes with the lock");
assert!(dir.join(LOCK).exists(), "the lock file itself may stay: only a held lock refuses");
let again = LibraryLock::acquire(&dir, "lightcraft-cli").unwrap();
assert!(again.held());
drop(again);
let _ = std::fs::remove_dir_all(&dir);
}
/// Files left behind by a crash (lock file + owner note) don't lock anyone out.
#[test]
fn leftover_lock_files_are_not_a_lock() {
let dir = temp_dir("stale");
std::fs::write(dir.join(LOCK), b"").unwrap();
std::fs::write(dir.join(OWNER), br#"{"pid":999999,"host":"elsewhere","program":"LightCraft","version":"0.2.0","since":1}"#).unwrap();
let l = LibraryLock::acquire(&dir, "LightCraft").unwrap();
assert!(l.held());
let owner: LockOwner = serde_json::from_slice(&std::fs::read(dir.join(OWNER)).unwrap()).unwrap();
assert_eq!(owner.pid, std::process::id(), "the note now names this process");
drop(l);
let _ = std::fs::remove_dir_all(&dir);
}
/// Another process holds the lock; when it is killed (a crash), the OS releases it.
#[test]
fn lock_held_by_another_process_is_released_when_it_dies() {
let dir = temp_dir("child");
let exe = std::env::current_exe().unwrap();
let mut child = std::process::Command::new(exe)
.args(["--exact", "tests_lock::lock_holder_child", "--ignored", "--test-threads=1"])
.env("LC_LOCK_CHILD_DIR", &dir)
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.spawn()
.unwrap();
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(60);
while !dir.join("child-locked").exists() {
assert!(child.try_wait().unwrap().is_none(), "child exited before locking");
assert!(std::time::Instant::now() < deadline, "child never locked");
std::thread::sleep(std::time::Duration::from_millis(20));
}
let e = LibraryLock::acquire(&dir, "LightCraft").unwrap_err();
let LockError::InUse(Some(who)) = &e else { panic!("{e:?}") };
assert_eq!(who.pid, child.id());
assert!(e.to_string().contains("on this computer") || who.host.is_empty(), "{e}");
child.kill().unwrap(); // SIGKILL: no clean-up runs, like a crash
child.wait().unwrap();
assert!(dir.join(OWNER).exists(), "the crashed holder's note is left behind");
let l = LibraryLock::acquire(&dir, "LightCraft").expect("the OS released the dead process's lock");
assert!(l.held());
drop(l);
let _ = std::fs::remove_dir_all(&dir);
}
/// Run by [`lock_held_by_another_process_is_released_when_it_dies`] in a child process: take the
/// lock, say so, and wait to be killed.
#[test]
#[ignore = "helper process for lock_held_by_another_process_is_released_when_it_dies"]
fn lock_holder_child() {
let Some(dir) = std::env::var_os("LC_LOCK_CHILD_DIR") else { return };
let _l = LibraryLock::acquire(std::path::Path::new(&dir), "lightcraft-cli").unwrap();
std::fs::write(std::path::Path::new(&dir).join("child-locked"), b"").unwrap();
std::thread::sleep(std::time::Duration::from_secs(120));
}
+59
View File
@@ -0,0 +1,59 @@
//! Optional build input: with `CRAFT_FONTS_DIR=<storytold/craft-fonts checkout>`, embed every font
//! in its `fonts/manifest.txt` as `CRAFT_FONTS` (see src/fonts.rs). Unset, `CRAFT_FONTS` is empty.
//! Recipe from craft-fonts' docs/integration.md; craft-fonts is never a Cargo dependency, and this
//! script only reads a local directory (no network).
//!
//! wasm32 (the web build) embeds only the UI font, BIZ UDPGothic Regular (~4.5 MB), to keep the
//! `.wasm` within the web size budget (Cloudflare serves at most 25 MiB per file); the export
//! watermark falls back to it there. Native builds embed every font.
use std::fmt::Write as _;
use std::path::PathBuf;
fn main() {
println!("cargo::rerun-if-changed=build.rs");
println!("cargo::rerun-if-env-changed=CRAFT_FONTS_DIR");
println!("cargo::rerun-if-env-changed=CRAFT_FONTS_REQUIRED");
let mut src = String::from("pub static CRAFT_FONTS: &[CraftFont] = &[\n");
if let Some(dir) = std::env::var_os("CRAFT_FONTS_DIR").map(PathBuf::from) {
match craft_fonts(&dir) {
Ok(entries) => src.push_str(&entries),
Err(e) if std::env::var_os("CRAFT_FONTS_REQUIRED").is_some() => {
println!("cargo::error=CRAFT_FONTS_DIR={}: {e}", dir.display());
}
Err(e) => println!("cargo::warning=building without craft-fonts: CRAFT_FONTS_DIR={}: {e}", dir.display()),
}
}
src.push_str("];\n");
let out = PathBuf::from(std::env::var_os("OUT_DIR").unwrap_or_default()).join("craft_fonts.rs");
if let Err(e) = std::fs::write(&out, src) {
println!("cargo::error=writing {}: {e}", out.display());
}
}
/// One `CraftFont { .. }` initialiser per manifest line.
fn craft_fonts(dir: &std::path::Path) -> Result<String, String> {
let manifest = dir.join("fonts/manifest.txt");
println!("cargo::rerun-if-changed={}", manifest.display());
let text = std::fs::read_to_string(&manifest).map_err(|e| format!("{}: {e}", manifest.display()))?;
let wasm = std::env::var("CARGO_CFG_TARGET_ARCH").is_ok_and(|a| a == "wasm32");
let mut out = String::new();
for line in text.lines().map(str::trim).filter(|l| !l.is_empty() && !l.starts_with('#')) {
let f: Vec<&str> = line.split(" | ").map(str::trim).collect();
let [family, style, file, scripts, ..] = f.as_slice() else {
return Err(format!("malformed manifest line: {line}"));
};
if wasm && (*family, *style) != ("BIZ UDPGothic", "Regular") {
continue;
}
let path = dir.join(file).canonicalize().map_err(|e| format!("{file}: {e}"))?;
println!("cargo::rerun-if-changed={}", path.display());
let scripts: Vec<String> = scripts.split(',').map(|s| format!("{:?}", s.trim())).collect();
let _ = writeln!(
out,
" CraftFont {{ family: {family:?}, style: {style:?}, scripts: &[{}], bytes: include_bytes!({:?}) }},",
scripts.join(", "),
path.display().to_string(),
);
}
Ok(out)
}
+215
View File
@@ -0,0 +1,215 @@
//! Is a file there? — answered without blocking the caller.
//!
//! On a sleeping NAS, a dropped share or a USB drive spinning up, one `stat` can take as long as
//! the OS or SMB timeout. The UI asks about files every frame (grid cells, the Info panel, the
//! Missing Photos view), so in the app the answers come from a cache that a worker thread fills
//! ([`Availability::run_in_background`]): an unknown file is reported as `None` ("assume it is
//! there"; the render worker finds out if it isn't), known answers are rechecked after
//! [`RECHECK`]. Everywhere else (CLI, MCP, tests) the default is synchronous: every question is a
//! fresh check, as before.
use std::collections::{HashMap, HashSet, VecDeque};
use std::sync::{Arc, Mutex, MutexGuard, PoisonError};
use std::time::Duration;
/// Checks whether a path exists (injectable: tests use a slow one).
pub type ExistsProbe = Arc<dyn Fn(&str) -> bool + Send + Sync>;
/// Called (from the worker) when an answer changed, e.g. to repaint the UI.
pub type Waker = Arc<dyn Fn() + Send + Sync>;
/// How long a cached answer is used before it is checked again (it is still returned meanwhile).
pub const RECHECK: Duration = Duration::from_secs(15);
fn fs_exists(path: &str) -> bool {
// the browser build has no file system: its originals are always "there"
cfg!(target_arch = "wasm32") || std::path::Path::new(path).exists()
}
#[derive(Default)]
struct State {
/// path → (exists, when checked)
known: HashMap<String, (bool, std::time::Instant)>,
queue: VecDeque<String>,
queued: HashSet<String>,
worker: bool,
/// Bumped whenever an answer changes (callers recompute what depends on it).
generation: u64,
}
fn lock(m: &Mutex<State>) -> MutexGuard<'_, State> {
m.lock().unwrap_or_else(PoisonError::into_inner)
}
/// File availability, synchronous or cached + background-checked. Cloning shares the cache.
#[derive(Clone)]
pub struct Availability {
probe: ExistsProbe,
background: Option<Waker>,
state: Arc<Mutex<State>>,
}
impl Default for Availability {
fn default() -> Self {
Self { probe: Arc::new(fs_exists), background: None, state: Arc::default() }
}
}
impl std::fmt::Debug for Availability {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("Availability").field("background", &self.background.is_some()).finish()
}
}
impl Availability {
/// Replace the existence check (tests: a slow or counting probe). Forgets cached answers.
pub fn set_probe(&mut self, probe: ExistsProbe) {
self.probe = probe;
let mut st = lock(&self.state);
st.known.clear();
st.generation += 1;
}
/// Answer from the cache and check on a worker thread from now on; `wake` runs when an
/// answer changed. No effect in the browser build (no threads, no file system).
pub fn run_in_background(&mut self, wake: Waker) {
if cfg!(target_arch = "wasm32") {
return;
}
self.background = Some(wake);
}
/// The existence check itself (blocking: for worker threads, e.g. the Rename preview's).
pub fn probe(&self) -> ExistsProbe {
self.probe.clone()
}
pub fn is_background(&self) -> bool {
self.background.is_some()
}
/// Does `path` exist? Synchronous mode: checked now. Background mode: the cached answer
/// (`None` while unknown); unknown or stale answers are queued for the worker.
pub fn exists(&self, path: &str) -> Option<bool> {
let Some(wake) = &self.background else {
return Some((self.probe)(path));
};
let mut st = lock(&self.state);
let now = std::time::Instant::now();
let cached = st.known.get(path).copied();
if cached.is_none_or(|(_, at)| now.saturating_duration_since(at) >= RECHECK) && !st.queued.contains(path) {
st.queued.insert(path.to_string());
st.queue.push_back(path.to_string());
if !st.worker {
st.worker = true;
drop(st);
self.spawn_worker(wake.clone());
return cached.map(|c| c.0);
}
}
cached.map(|c| c.0)
}
/// Known to be missing (unknown counts as present).
pub fn is_offline(&self, path: &str) -> bool {
self.exists(path) == Some(false)
}
/// Changes whenever a cached answer changes.
pub fn generation(&self) -> u64 {
lock(&self.state).generation
}
/// Paths waiting for the worker.
pub fn pending(&self) -> usize {
lock(&self.state).queue.len()
}
fn spawn_worker(&self, wake: Waker) {
let (state, probe) = (self.state.clone(), self.probe.clone());
let started = std::thread::Builder::new().name("lc-availability".into()).spawn(move || {
loop {
let path = {
let mut st = lock(&state);
let Some(p) = st.queue.pop_front() else {
st.worker = false;
return;
};
p
};
let ok = probe(&path);
let changed = {
let mut st = lock(&state);
st.queued.remove(&path);
let prev = st.known.insert(path, (ok, std::time::Instant::now()));
let changed = prev.map(|p| p.0) != Some(ok);
if changed {
st.generation += 1;
}
changed
};
if changed {
wake();
}
}
});
if started.is_err() {
// try again on the next question
lock(&self.state).worker = false;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::atomic::{AtomicUsize, Ordering};
#[test]
fn synchronous_by_default() {
let mut a = Availability::default();
let n = Arc::new(AtomicUsize::new(0));
let c = n.clone();
a.set_probe(Arc::new(move |p| {
c.fetch_add(1, Ordering::SeqCst);
p == "here"
}));
assert_eq!(a.exists("here"), Some(true));
assert_eq!(a.exists("gone"), Some(false));
assert_eq!(a.exists("gone"), Some(false));
assert_eq!(n.load(Ordering::SeqCst), 3, "every question is a fresh check");
}
#[test]
fn background_answers_from_the_cache_without_blocking() {
let mut a = Availability::default();
let caller = std::thread::current().id();
let on_caller = Arc::new(AtomicUsize::new(0));
let c = on_caller.clone();
a.set_probe(Arc::new(move |p| {
if std::thread::current().id() == caller {
c.fetch_add(1, Ordering::SeqCst);
}
std::thread::sleep(Duration::from_millis(200)); // a sleeping NAS
p == "here"
}));
let woke = Arc::new(AtomicUsize::new(0));
let w = woke.clone();
a.run_in_background(Arc::new(move || {
w.fetch_add(1, Ordering::SeqCst);
}));
let t0 = std::time::Instant::now();
assert_eq!(a.exists("here"), None, "unknown at first");
assert_eq!(a.exists("gone"), None);
assert!(t0.elapsed() < Duration::from_millis(150), "the caller never waits for the probe");
let deadline = std::time::Instant::now() + Duration::from_secs(20);
while (a.exists("here").is_none() || a.exists("gone").is_none()) && std::time::Instant::now() < deadline {
std::thread::sleep(Duration::from_millis(10));
}
assert_eq!(a.exists("here"), Some(true));
assert_eq!(a.exists("gone"), Some(false));
assert!(a.is_offline("gone") && !a.is_offline("here") && !a.is_offline("unknown"));
assert_eq!(on_caller.load(Ordering::SeqCst), 0, "never probed on the calling thread");
assert!(woke.load(Ordering::SeqCst) >= 2);
assert!(a.generation() >= 2);
}
}
+277
View File
@@ -0,0 +1,277 @@
//! Estimate an ARW starting look from its own JPEG. Colour and luminance are fitted separately;
//! the JPEG supplies correspondences only, never output pixels or a replacement for RAW editing.
use lightcraft_color::{Mat3, luminance_2020};
use lightcraft_pipeline::tone::{CameraTone, ToneMap};
use lightcraft_raster::{
Rgb32f,
resample::{Filter, fit},
};
use lightcraft_raw::{RawImage, color::CameraTransform};
#[derive(Clone, Copy, Debug)]
pub(crate) struct CameraLook {
pub matrix: Mat3,
pub tone: CameraTone,
}
pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransform) -> Option<CameraLook> {
if !transform.matrix_is_fallback || raw.format != lightcraft_raw::RawFormat::Arw {
return None;
}
let jpeg = lightcraft_raw::embedded_preview(bytes)?;
let decoded = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions { max_size: Some((384, 384)), max_pixels: 64_000_000 }).ok()?;
let reference = decoded.to_working();
let crop = raw.crop.clipped(raw.active_area.width, raw.active_area.height);
if crop.width == 0 || crop.height == 0 || reference.width == 0 || reference.height == 0 {
return None;
}
let aspect = crop.width as f64 / crop.height as f64;
if (reference.width as f64 / reference.height as f64 / aspect - 1.0).abs() > 0.02 {
return None;
}
// Fixed, bounded proxy: the selected look cannot depend on thumbnail/export resolution.
let k = (crop.width.max(crop.height).div_ceil(384).max(2)).div_ceil(2) * 2;
let sensor = match raw.develop_binned(k, 0.99).ok()? {
Some(s) => s,
// reduced-size (M/S) ARWs are already RGB (issue #147): no demosaic, the box fit below does the binning
None if raw.cpp == 3 && raw.cfa.is_none() => raw.develop(lightcraft_raw::Method::Bilinear).ok()?,
None => return None,
};
let mut sensor = fit(&sensor, 96, 96, Filter::Box);
let reference = fit(&reference, sensor.width, sensor.height, Filter::Box);
let gain = 2f32.powf(transform.baseline_exposure as f32);
sensor.map_in_place(|p| transform.matrix.apply_f32(std::array::from_fn(|i| p[i] * transform.wb[i] * gain)));
let look = fit_pairs(&sensor, &reference)?;
if lightcraft_pipeline::profiling() {
eprintln!("[profile] ARW camera look: {:?}, {:?}", look.matrix.0, look.tone);
}
Some(look)
}
/// A fit must cut the held-out squared error to below this share of the fallback's.
const MIN_IMPROVEMENT: f64 = 0.7;
/// ...and stay within this per-channel RMS of the camera JPEG (linear display values). On public
/// raw.pixls.us samples (eight Sony bodies) good fits that still beat the fallback 1.5–3× landed
/// at 0.065–0.093 (camera local tone, vignetting and lens processing that a global matrix + curve
/// cannot follow): visibly better renders that 0.055 rejected.
const MAX_HOLDOUT_RMS: f64 = 0.10;
fn luma(p: [f64; 3]) -> f64 {
p[0] * 0.2627 + p[1] * 0.6780 + p[2] * 0.0593
}
fn displayed(scene: [f64; 3], tone: &ToneMap) -> [f64; 3] {
let scene = scene.map(|v| v.max(0.0));
let y = luma(scene);
if y <= 0.0 {
return [0.0; 3];
}
scene.map(|v| v * f64::from(tone.apply(y as f32)) / y)
}
fn fit_pairs(sensor: &Rgb32f, reference: &Rgb32f) -> Option<CameraLook> {
if (sensor.width, sensor.height) != (reference.width, reference.height) || sensor.data.len() != reference.data.len() {
return None;
}
let mut pairs = Vec::new();
let mut colour = 0;
for (input, output) in sensor.data.iter().zip(&reference.data) {
let y = luminance_2020(*output);
if !input.iter().all(|v| v.is_finite() && *v > 0.001 && *v < 1.5)
|| !output.iter().all(|v| v.is_finite() && *v > 0.004 && *v < 0.98)
|| !(0.015..0.85).contains(&y)
{
continue;
}
let min = output.iter().copied().fold(f32::INFINITY, f32::min);
let max = output.iter().copied().fold(0.0, f32::max);
colour += usize::from(max - min > 0.05);
pairs.push((input.map(f64::from), output.map(f64::from)));
}
if pairs.len() < 256 || colour < pairs.len() / 20 {
return None;
}
let mut gram = [[0.0; 3]; 3];
let mut cross = [[0.0; 3]; 3];
for (i, (x, y)) in pairs.iter().enumerate() {
if i % 3 == 0 {
continue;
}
let (lx, ly) = (luma(*x), luma(*y));
for row in 0..3 {
for col in 0..3 {
// Normalising by luminance prevents a camera S-curve from corrupting colour.
gram[row][col] += x[row] * x[col] / (lx * lx);
cross[row][col] += y[row] * x[col] / (ly * lx);
}
}
}
let trace: f64 = (0..3).map(|i| gram[i][i]).sum();
let inverse = Mat3(gram).inverse()?;
let condition = trace * (0..3).map(|i| inverse.0[i][i].abs()).sum::<f64>();
if trace <= 0.0 || !condition.is_finite() || condition > 1e6 {
return None;
}
let regularization = trace * 1e-4;
for i in 0..3 {
gram[i][i] += regularization;
cross[i][i] += regularization;
}
let matrix = Mat3(cross).mul(&Mat3(gram).inverse()?);
if !matrix.0.iter().flatten().all(|v| v.is_finite() && v.abs() < 8.0) {
return None;
}
let tone_pairs: Vec<_> =
pairs.iter().enumerate().filter(|(i, _)| i % 3 != 0).map(|(_, (x, y))| (luma(matrix.apply(*x).map(|v| v.max(0.0))), luma(*y))).collect();
let curve = fit_tone(tone_pairs)?;
let tone = ToneMap::camera(&curve, 0.0, 0.0, 0.0);
let original_tone = ToneMap::new(0.0, 0.0, 0.0);
let mut before = 0.0;
let mut after = 0.0;
let mut samples = 0;
for (i, (x, target)) in pairs.iter().enumerate() {
if i % 3 != 0 {
continue;
}
let corrected = displayed(matrix.apply(*x), &tone);
let original = displayed(*x, &original_tone);
for c in 0..3 {
before += (original[c] - target[c]).powi(2);
after += (corrected[c] - target[c]).powi(2);
samples += 1;
}
}
if lightcraft_pipeline::profiling() {
eprintln!("[profile] ARW holdout RMS {:.5} -> {:.5} ({samples} channels)", (before / samples as f64).sqrt(), (after / samples as f64).sqrt());
}
if !after.is_finite() || samples == 0 || after >= before * MIN_IMPROVEMENT || after / samples as f64 > MAX_HOLDOUT_RMS.powi(2) {
return None;
}
Some(CameraLook { matrix, tone: curve })
}
fn median(values: &mut [f64]) -> Option<f64> {
values.sort_by(f64::total_cmp);
values.get(values.len() / 2).copied()
}
fn fit_tone(mut pairs: Vec<(f64, f64)>) -> Option<CameraTone> {
if pairs.len() < 128 || !pairs.iter().all(|(x, y)| x.is_finite() && *x > 0.0 && y.is_finite()) {
return None;
}
pairs.sort_by(|a, b| a.0.total_cmp(&b.0));
let mut knots = [[0.0; 2]; 32];
for (i, knot) in knots.iter_mut().enumerate() {
let bin = pairs.get(i * pairs.len() / 32..(i + 1) * pairs.len() / 32)?;
let mut xs: Vec<_> = bin.iter().map(|p| p.0).collect();
let mut ys: Vec<_> = bin.iter().map(|p| p.1).collect();
*knot = [median(&mut xs)? as f32, median(&mut ys)? as f32];
}
if knots[31][0] < knots[0][0] * 1.5 {
return None;
}
// Pool adjacent violating bins (isotonic regression): no reversals or arbitrary polynomial.
let mut blocks: Vec<(f32, usize)> = Vec::new();
for knot in knots {
blocks.push((knot[1], 1));
while blocks.len() >= 2 {
let (a, an) = *blocks.get(blocks.len() - 2)?;
let (b, bn) = *blocks.last()?;
if a <= b {
break;
}
blocks.truncate(blocks.len() - 2);
blocks.push(((a * an as f32 + b * bn as f32) / (an + bn) as f32, an + bn));
}
}
let mut i = 0;
for (y, n) in blocks {
for knot in knots.get_mut(i..i + n)? {
knot[1] = y;
}
i += n;
}
CameraTone::new(knots)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn separates_nonlinear_tone_from_colour_and_keeps_sensor_headroom() {
let known = Mat3([[1.8, -0.4, -0.1], [-0.2, 1.5, -0.1], [-0.05, -0.3, 1.7]]);
let mut sensor = Rgb32f::new(64, 64);
let mut reference = sensor.clone();
for (i, (src, dst)) in sensor.data.iter_mut().zip(&mut reference.data).enumerate() {
let ev = 0.05 + (i % 31) as f32 * 0.017;
*src = [ev * (0.8 + (i % 11) as f32 * 0.025), ev, ev * (0.8 + (i % 17) as f32 * 0.014)];
let p = known.apply_f32(*src);
let y = luminance_2020(p);
*dst = p.map(|v| v * (1.0 - (-2.5 * y).exp()) / y);
}
let original = sensor.clone();
let fit = fit_pairs(&sensor, &reference).unwrap();
assert_eq!(sensor.data, original.data);
let tone = ToneMap::camera(&fit.tone, 0.0, 0.0, 0.0);
let error: f64 = sensor
.data
.iter()
.zip(&reference.data)
.map(|(x, y)| {
let p = displayed(fit.matrix.apply(x.map(f64::from)), &tone);
(0..3).map(|c| (p[c] - f64::from(y[c])).powi(2)).sum::<f64>() / 3.0
})
.sum::<f64>()
/ sensor.data.len() as f64;
assert!(error.sqrt() < 0.025, "{error}");
// The colour transform is homogeneous; tone mapping happens only after exposure.
let p = fit.matrix.apply([2.0, 2.0, 2.0]);
assert!(luma(p) > 1.0);
assert!(tone.apply(0.2) < tone.apply(0.4));
}
#[test]
fn accepts_a_much_better_fit_despite_local_camera_processing() {
// The camera JPEG departs from any global matrix + curve (local tone, vignetting): ±0.12
// per-pixel deviations, ~0.07 RMS. The fit is still far closer than the fallback.
let known = Mat3([[1.8, -0.4, -0.1], [-0.2, 1.5, -0.1], [-0.05, -0.3, 1.7]]);
let mut sensor = Rgb32f::new(64, 64);
let mut reference = sensor.clone();
let mut seed = 0x2545_f491_u32;
for (i, (src, dst)) in sensor.data.iter_mut().zip(&mut reference.data).enumerate() {
let ev = 0.05 + (i % 31) as f32 * 0.017;
*src = [ev * (0.8 + (i % 11) as f32 * 0.025), ev, ev * (0.8 + (i % 17) as f32 * 0.014)];
let p = known.apply_f32(*src);
let y = luminance_2020(p);
*dst = p.map(|v| {
seed ^= seed << 13;
seed ^= seed >> 17;
seed ^= seed << 5;
let noise = (seed % 2001) as f32 / 1000.0 - 1.0;
(v * (1.0 - (-2.5 * y).exp()) / y + 0.12 * noise).clamp(0.005, 0.97)
});
}
assert!(fit_pairs(&sensor, &reference).is_some());
}
#[test]
fn rejects_monochrome_invalid_and_unrelated_previews() {
let mut sensor = Rgb32f::new(32, 32);
let mut reference = sensor.clone();
for (i, p) in sensor.data.iter_mut().enumerate() {
*p = [0.1 + (i % 13) as f32 * 0.02, 0.15, 0.1];
}
reference.data.fill([0.2; 3]);
assert!(fit_pairs(&sensor, &reference).is_none());
reference.data.fill([f32::NAN; 3]);
assert!(fit_pairs(&sensor, &reference).is_none());
for (i, (src, dst)) in sensor.data.iter_mut().zip(&mut reference.data).enumerate() {
*src = [0.04 + (i % 11) as f32 * 0.02, 0.05 + (i % 17) as f32 * 0.01, 0.03 + (i % 23) as f32 * 0.01];
*dst = [0.05 + (i % 7) as f32 * 0.07, 0.05 + (i % 19) as f32 * 0.02, 0.05 + (i % 29) as f32 * 0.01];
}
assert!(fit_pairs(&sensor, &reference).is_none());
sensor.data.fill([0.1, 0.15, 0.12]);
assert!(fit_pairs(&sensor, &reference).is_none());
reference.data.truncate(8);
assert!(fit_pairs(&sensor, &reference).is_none());
}
}
+62 -35
View File
@@ -166,7 +166,7 @@ pub fn specs() -> Vec<CommandSpec> {
"Rename Folder",
[],
None,
"{path, name} — rename a folder on disk (its files and sidecars go along) and relink the photos in it; not an undo step → {path, relinked}",
"{path, name} — rename a folder on disk (its files and sidecars go along) and relink the photos in it; one undo step (undo renames it back, refused if the old name is taken) → {path, relinked}",
always,
|s, p| {
let from = str_param(p, "path").ok_or_else(|| bad("folder.rename", "missing path"))?.trim_end_matches(['/', '\\']).to_string();
@@ -184,7 +184,7 @@ pub fn specs() -> Vec<CommandSpec> {
"Move Folder",
[],
None,
"{path, into: destination folder} — move a folder on disk into another and relink the photos in it; not an undo step → {path, relinked}",
"{path, into: destination folder} — move a folder on disk into another and relink the photos in it; one undo step (undo moves it back, refused if the old place is taken) → {path, relinked}",
always,
|s, p| {
let from = str_param(p, "path").ok_or_else(|| bad("folder.move", "missing path"))?.trim_end_matches(['/', '\\']).to_string();
@@ -217,58 +217,85 @@ pub fn specs() -> Vec<CommandSpec> {
]
}
/// Rename or move a folder on disk (everything in it goes along, sidecars included) and relink
/// the photos inside. Like a rename in the file manager it is not an undo step: undoing would
/// point the photos back at a folder that no longer exists.
pub(crate) fn move_folder(s: &mut crate::Session, from: &str, to: &str) -> std::result::Result<usize, String> {
use std::path::Path;
/// Rename `from` to `to` on disk (a folder; everything in it goes along). Never overwrites:
/// refused when `to` exists (a case-only rename on a case-insensitive file system goes through
/// a temporary name). Missing parents of `to` are created.
pub(crate) fn rename_folder_on_disk(from: &str, to: &str) -> std::result::Result<(), String> {
let (src, dst) = (Path::new(from), Path::new(to));
if !src.is_dir() {
return Err(format!("{from} is not a folder"));
return Err(format!("{from} is not a folder (moved or deleted?)"));
}
if dst.exists() {
let case_only = from != to && from.to_lowercase() == to.to_lowercase();
if dst.exists() && !case_only {
return Err(format!("{to} already exists"));
}
if dst.starts_with(src) {
return Err("a folder can't move into itself".into());
}
if s.library.as_ref().is_some_and(|l| l.dir.starts_with(src) || src.starts_with(&l.dir)) {
return Err("the library's own folders can't be moved here".into());
}
if let Some(parent) = dst.parent() {
std::fs::create_dir_all(parent).map_err(|e| format!("{}: {e}", parent.display()))?;
}
std::fs::rename(src, dst).map_err(|e| format!("{from} → {to}: {e}"))?;
let mut n = 0;
let ops: Vec<lightcraft_catalog::Op> = s
if case_only {
let tmp = src.with_file_name(format!(".lc-rename-{}", std::process::id()));
if tmp.exists() {
return Err(format!("{} already exists", tmp.display()));
}
std::fs::rename(src, &tmp).map_err(|e| format!("{from} → {to}: {e}"))?;
return std::fs::rename(&tmp, dst).map_err(|e| {
let _ = std::fs::rename(&tmp, src);
format!("{from} → {to}: {e}")
});
}
std::fs::rename(src, dst).map_err(|e| format!("{from} → {to}: {e}"))
}
/// After a folder moved from `from` to `to`: a browsed folder at or below it follows.
pub(crate) fn follow_folder(s: &mut Session, from: &str, to: &str) {
if let Some(b) = &mut s.browse
&& let Ok(rel) = Path::new(&b.path).strip_prefix(from)
{
b.path = Path::new(to).join(rel).to_string_lossy().to_string();
let f = b.path.clone();
s.filter.folder = Some(f);
}
}
/// Rename or move a folder on disk (everything in it goes along, sidecars included) and relink
/// the photos inside, as one undo step: undo renames the folder back and restores the photos'
/// paths, redo repeats the move — each refused (and reported) when the destination is taken.
pub(crate) fn move_folder(s: &mut Session, from: &str, to: &str) -> std::result::Result<usize, String> {
let (src, dst) = (Path::new(from), Path::new(to));
if !src.is_dir() {
return Err(format!("{from} is not a folder"));
}
if s.library.as_ref().is_some_and(|l| l.dir.starts_with(src) || src.starts_with(&l.dir)) {
return Err("the library's own folders can't be moved here".into());
}
rename_folder_on_disk(from, to)?;
let ops: Vec<Op> = s
.catalog
.photos()
.filter_map(|p| match &p.source {
lightcraft_catalog::Source::File { path } => Path::new(path).strip_prefix(src).ok().map(|rel| (p.id, p.file_name.clone(), dst.join(rel))),
_ => None,
})
.map(|(id, file_name, np)| {
n += 1;
lightcraft_catalog::Op::Relink {
id,
file_name,
source: lightcraft_catalog::Source::File { path: np.to_string_lossy().to_string() },
format: None,
}
.map(|(id, file_name, np)| Op::Relink {
id,
file_name,
source: lightcraft_catalog::Source::File { path: np.to_string_lossy().to_string() },
format: None,
})
.collect();
for op in ops {
if s.catalog.apply(op.clone()).is_ok() {
s.pending_log.push(op);
}
}
// a browsed folder at or below it follows
if let Some(b) = &mut s.browse
&& let Ok(rel) = Path::new(&b.path).strip_prefix(src)
{
b.path = dst.join(rel).to_string_lossy().to_string();
let f = b.path.clone();
s.filter.folder = Some(f);
let n = ops.len();
let name = Path::new(to).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_default();
let renamed = src.parent() == dst.parent();
let label = if renamed { format!("Rename Folder to “{name}”") } else { format!("Move Folder “{name}”") };
let folder = crate::FolderMove { from: to.to_string(), to: from.to_string() };
if let Err(e) = s.commit_with_folder(&label, Op::Batch { ops }, folder) {
// the catalog didn't take it: put the folder back where the photos still point
let back = rename_folder_on_disk(to, from).err().map(|b| format!("; moving it back failed too: {b}")).unwrap_or_default();
return Err(format!("{e}{back}"));
}
follow_folder(s, from, to);
Ok(n)
}
+1 -1
View File
@@ -26,7 +26,7 @@ const PROBE_EDGE: usize = 384;
pub(crate) fn probe(s: &mut Session, c: &str, x: f64, y: f64, neutral: impl FnOnce(&mut DevelopSettings)) -> Result<[f32; 3]> {
let id = s.active().ok_or_else(|| bad(c, "no active photo"))?;
let src = s.source_now(id, SourceLevel::Thumb).map_err(|e| bad(c, e))?;
let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default();
let info = s.source_info(id);
let mut d = (*s.develop_of(id).unwrap_or_default()).clone();
neutral(&mut d);
let req = lightcraft_pipeline::RenderRequest::fit(PROBE_EDGE, PROBE_EDGE);
+8 -2
View File
@@ -26,7 +26,12 @@ fn dng_names(path: &str) -> impl Iterator<Item = std::path::PathBuf> {
/// written to a temp file, synced, read back identical and only then given its name — never
/// replacing an existing file. On any failure no DNG is left behind and the raw stays in use.
pub(crate) fn write_dng_for(s: &Session, path: &str, packet: String) -> std::result::Result<String, String> {
let bytes = match &s.media.file_bytes {
write_dng_with(s.media.file_bytes.as_ref(), path, packet)
}
/// [`write_dng_for`] without the session (imports convert on a worker thread).
pub(crate) fn write_dng_with(file_bytes: Option<&crate::merge::ByteReader>, path: &str, packet: String) -> std::result::Result<String, String> {
let bytes = match file_bytes {
Some(r) => r(path)?,
None => std::fs::read(path).map_err(|e| format!("{path}: {e}"))?,
};
@@ -144,7 +149,8 @@ fn edit_external(s: &mut Session, p: &Value) -> Result<Value> {
};
let mut written = Vec::new();
let mut write = |path: &str, bytes: &[u8]| -> std::result::Result<(), String> {
crate::export::write_file(path, bytes)?;
// durable: the TIFF becomes a library photo the external editor then changes
crate::export::write_file_durable(path, bytes)?;
written.push(path.to_string());
Ok(())
};
+38 -12
View File
@@ -98,7 +98,11 @@ pub fn specs() -> Vec<CommandSpec> {
}
}
let label = if vals.len() == 1 { controls::find(&vals[0].0).map(|c| c.label).unwrap_or("Edit").to_string() } else { "Edit".into() };
let apply = |d: &mut DevelopSettings| {
let apply = |d: &mut DevelopSettings, info: &lightcraft_pipeline::SourceInfo| {
if d.wb.mode == WbMode::AsShot && vals.iter().any(|(k, _)| k == "wb.temp" || k == "wb.tint") {
d.wb.temp = info.as_shot_temp;
d.wb.tint = info.as_shot_tint;
}
for (k, v) in &vals {
controls::set(d, k, *v);
if k == "wb.temp" || k == "wb.tint" {
@@ -113,15 +117,18 @@ pub fn specs() -> Vec<CommandSpec> {
.filter_map(|id| s.develop_of(*id).map(|d| (*id, d)))
.filter_map(|(id, d)| {
let mut d = (*d).clone();
apply(&mut d);
let info = s.source_info(id);
apply(&mut d, &info);
s.develop_op(id, d, &label)
})
.collect();
s.commit(&label, Op::Batch { ops })?;
return ok();
}
let id = active(s, "develop.set")?;
let info = s.source_info(id);
edit(s, "develop.set", &label, |d| {
apply(d);
apply(d, &info);
Ok(())
})
}
@@ -130,7 +137,15 @@ pub fn specs() -> Vec<CommandSpec> {
let c = str_param(p, "control").ok_or_else(|| bad("develop.adjust", "missing control"))?.to_string();
let spec = controls::find(&c).ok_or_else(|| bad("develop.adjust", "unknown control"))?;
let delta = f64_req(p, "delta", "develop.adjust")?;
let info = s.source_info(active(s, "develop.adjust")?);
edit(s, "develop.adjust", spec.label, |d| {
if c == "wb.temp" || c == "wb.tint" {
if d.wb.mode == WbMode::AsShot {
d.wb.temp = info.as_shot_temp;
d.wb.tint = info.as_shot_tint;
}
d.wb.mode = WbMode::Custom;
}
let v = controls::get(d, &c).unwrap_or(spec.default);
controls::set(d, &c, v + delta);
Ok(())
@@ -149,12 +164,18 @@ pub fn specs() -> Vec<CommandSpec> {
let ops = ids
.iter()
.filter_map(|id| s.develop_of(*id).map(|d| (*id, d)))
.filter_map(|(id, d)| {
.filter_map(|(id, _d)| {
// back to the photo's import defaults when a default preset gave it its look
let look = s.catalog.photo(id).and_then(|p| p.import_look.clone());
let fresh = match look {
Some(l) => (*l).clone(),
None => DevelopSettings { wb: lightcraft_develop::WhiteBalance { mode: WbMode::AsShot, ..d.wb }, ..Default::default() },
None => {
let info = s.source_info(id);
DevelopSettings {
wb: lightcraft_develop::WhiteBalance { mode: WbMode::AsShot, temp: info.as_shot_temp, tint: info.as_shot_tint },
..Default::default()
}
}
};
s.develop_op(id, fresh, "Reset")
})
@@ -203,7 +224,7 @@ pub fn specs() -> Vec<CommandSpec> {
|s, _| {
let id = active(s, "develop.autoBwMix")?;
let src = s.source_now(id, SourceLevel::Thumb).map_err(|e| bad("develop.autoBwMix", e))?;
let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default();
let info = s.source_info(id);
let d = s.develop_of(id).unwrap_or_default();
let m = lightcraft_pipeline::auto::auto_bw_mix(&src, &info, &d);
edit(s, "develop.autoBwMix", "Auto B&W Mix", |d| {
@@ -218,7 +239,7 @@ pub fn specs() -> Vec<CommandSpec> {
cmd!("develop.auto", "Auto Settings", ["Photo"], Some("Shift+A"), "{}", has_active, |s, _| {
let id = active(s, "develop.auto")?;
let src = s.source_now(id, SourceLevel::Thumb).map_err(|e| bad("develop.auto", e))?;
let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default();
let info = s.source_info(id);
let d = s.develop_of(id).unwrap_or_default();
let a = lightcraft_pipeline::auto::auto_tone(&src, &info, &d);
edit(s, "develop.auto", "Auto", |d| {
@@ -245,7 +266,7 @@ pub fn specs() -> Vec<CommandSpec> {
let mode: WbMode =
serde_json::from_value(p.get("mode").cloned().unwrap_or(json!("custom"))).map_err(|e| bad("develop.wb", e.to_string()))?;
let id = active(s, "develop.wb")?;
let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default();
let info = s.source_info(id);
let (mut t, mut tint) = match mode {
WbMode::AsShot => (info.as_shot_temp, info.as_shot_tint),
WbMode::Auto => {
@@ -256,7 +277,7 @@ pub fn specs() -> Vec<CommandSpec> {
.preset()
.map(|(t, ti)| {
// presets are relative to daylight for rendered files
if info.raw { (t, ti) } else { (6500.0 * t / 5500.0, ti) }
if info.raw && !info.relative_wb { (t, ti) } else { (6500.0 * t / 5500.0, ti) }
})
.unwrap_or((info.as_shot_temp, info.as_shot_tint)),
};
@@ -277,7 +298,7 @@ pub fn specs() -> Vec<CommandSpec> {
let (x, y) = (f64_req(p, "x", "develop.wbPick")?, f64_req(p, "y", "develop.wbPick")?);
let id = active(s, "develop.wbPick")?;
let src = s.source_now(id, SourceLevel::Thumb).map_err(|e| bad("develop.wbPick", e))?;
let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default();
let info = s.source_info(id);
let d = s.develop_of(id).unwrap_or_default();
// Sample a small neighbourhood in the oriented source.
let o = src.oriented(d.orientation);
@@ -405,7 +426,7 @@ pub fn specs() -> Vec<CommandSpec> {
|s, _| {
let id = active(s, "crop.autoStraighten")?;
let src = s.source_now(id, SourceLevel::Preview).map_err(|e| bad("crop.autoStraighten", e))?;
let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default();
let info = s.source_info(id);
let d = s.develop_of(id).unwrap_or_default();
let Some(level) = lightcraft_pipeline::upright::level_degrees(&src, &info, &d) else {
return Ok(json!({"changed": false, "reason": "no dominant horizontal or vertical lines"}));
@@ -501,7 +522,7 @@ pub fn specs() -> Vec<CommandSpec> {
Upright::Off | Upright::Guided => None,
m => {
let src = s.source_now(id, SourceLevel::Preview).map_err(|e| bad("geometry.upright", e))?;
let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default();
let info = s.source_info(id);
Some(lightcraft_pipeline::upright::auto_transform(&src, &info, &d, m))
}
};
@@ -657,6 +678,11 @@ pub fn specs() -> Vec<CommandSpec> {
.filter_map(|id| s.develop_of(id).map(|d| (id, d)))
.filter_map(|(id, d)| {
let mut nd = (*d).clone();
if nd.wb.mode == WbMode::AsShot && (ctl == "wb.temp" || ctl == "wb.tint") {
let info = s.source_info(id);
nd.wb.temp = info.as_shot_temp;
nd.wb.tint = info.as_shot_tint;
}
let v = controls::get(&nd, &ctl).unwrap_or(spec.default);
controls::set(&mut nd, &ctl, v + delta);
if ctl == "wb.temp" || ctl == "wb.tint" {
+119 -76
View File
@@ -7,6 +7,105 @@ use serde_json::{Value, json};
use super::{CommandSpec, always, bad, bool_or, cmd, has_active, has_selection, ok, str_param};
use crate::{LibrarySource, Result, Selection, Session};
/// The auto-import folder's visible files with their sizes (the file-system half of
/// `library.autoImportScan`; the app lists on a worker thread and passes `listing`).
pub fn list_auto_import_folder(folder: &str) -> std::result::Result<Vec<(String, u64)>, String> {
let rd = std::fs::read_dir(folder).map_err(|e| format!("{folder}: {e}"))?;
Ok(rd
.flatten()
.filter(|e| !e.file_name().to_string_lossy().starts_with('.'))
.filter_map(|e| {
let m = std::fs::metadata(e.path()).ok()?;
m.is_file().then(|| (e.path().to_string_lossy().to_string(), m.len()))
})
.collect())
}
/// `library.import`'s params, checked (no file-system calls).
pub struct ImportRequest {
pub paths: Vec<String>,
pub opts: crate::import::ImportOptions,
pub album: Option<u64>,
pub album_name: Option<String>,
}
/// Parse and check `library.import`'s params (the app's import task runs the import itself, on a
/// worker thread, with the same options).
pub fn import_params(s: &Session, p: &Value) -> Result<ImportRequest> {
let paths = strs(p, "paths");
let mode = match str_param(p, "mode").unwrap_or("add") {
"add" => crate::import::ImportMode::Add,
"copy" => crate::import::ImportMode::Copy,
"move" => crate::import::ImportMode::Move,
other => return Err(bad("library.import", format!("unknown mode `{other}` (add|copy|move)"))),
};
let preset = match str_param(p, "preset").filter(|x| !x.is_empty()) {
Some(id) => Some(s.presets.iter().find(|x| x.id == id).cloned().ok_or_else(|| bad("library.import", format!("unknown preset `{id}`")))?),
None => None,
};
let album = p.get("album").and_then(Value::as_u64);
if let Some(a) = album
&& s.catalog.album(AlbumId(a)).is_none_or(|al| al.folder || al.is_smart())
{
return Err(bad("library.import", "album must be a regular album"));
}
let organize = match str_param(p, "organize") {
Some(o) => crate::import::Organize::parse(o).ok_or_else(|| {
bad("library.import", format!("unknown organize `{o}` (date|month|flat, or a folder template like {{date:%Y}}/{{date:%Y%m%d}})"))
})?,
None => Default::default(),
};
if let crate::import::Organize::Template(t) = &organize
&& let Some(e) = crate::rename::folder_template_error(t)
{
return Err(bad("library.import", e));
}
let metadata_preset = str_param(p, "metadataPreset").map(str::trim).filter(|n| !n.is_empty()).map(str::to_string);
if let Some(n) = &metadata_preset
&& !s.metadata_presets.iter().any(|m| m.name.eq_ignore_ascii_case(n))
{
return Err(bad("library.import", format!("unknown metadata preset `{n}`")));
}
let opts = crate::import::ImportOptions {
mode,
preset,
keywords: strs(p, "keywords"),
destination: str_param(p, "destination").map(str::to_string),
organize,
rename: str_param(p, "rename").map(str::to_string),
rename_start: p.get("renameStart").and_then(Value::as_u64).unwrap_or(1) as usize,
metadata_preset,
convert_dng: bool_or(p, "dng", false),
local: bool_or(p, "local", false),
};
let album_name = str_param(p, "albumName").map(str::trim).filter(|n| !n.is_empty()).map(str::to_string);
Ok(ImportRequest { paths, opts, album, album_name })
}
/// After a batch of an import was committed: the album (a new one named `album_name` when there
/// is no `album` and something was imported) gets the photos. Returns the report as JSON (with
/// `album`).
pub fn import_batch_done(s: &mut Session, report: crate::import::ImportReport, mut album: Option<u64>, album_name: Option<&str>) -> Result<Value> {
let mut report = serde_json::to_value(report).unwrap_or_default();
let imported: Vec<u64> = report["imported"].as_array().map(|a| a.iter().filter_map(Value::as_u64).collect()).unwrap_or_default();
if album.is_none()
&& let Some(name) = album_name
&& !imported.is_empty()
{
let r = s.execute("album.create", &json!({"name": name}))?;
album = r["id"].as_u64();
}
if let Some(a) = album
&& !imported.is_empty()
{
// (album.addPhotos wants a selection: the new photos are about to be it)
s.selection = Selection::single(PhotoId(imported[0]));
s.execute("album.addPhotos", &json!({"id": a, "ids": imported}))?;
report["album"] = json!(a);
}
Ok(report)
}
fn album_param(p: &Value, key: &str, c: &str) -> Result<AlbumId> {
p.get(key).and_then(Value::as_u64).map(AlbumId).ok_or_else(|| bad(c, format!("missing album `{key}`")))
}
@@ -657,9 +756,9 @@ pub fn specs() -> Vec<CommandSpec> {
"Auto Import Now",
[],
None,
"{} — add the watched folder's new photos (files the library doesn't have yet; partial / still-copying files wait for the next scan) → {imported, folder}",
"{listing?: [[path, size]] (the folder as listed by the caller, e.g. on a worker thread), start?: bool (false: return the `library.import` params as `import` instead of importing)} — add the watched folder's new photos (files the library doesn't have yet; partial / still-copying files wait for the next scan) → {imported, folder, import?}",
always,
|s, _| {
|s, p| {
let Some(folder) = s.import_defaults.auto_folder.clone() else { return Ok(json!({"imported": [], "folder": null})) };
// only files that stopped growing: a file still being written is left for later
let known: std::collections::HashSet<String> = s
@@ -670,18 +769,20 @@ pub fn specs() -> Vec<CommandSpec> {
_ => None,
})
.collect();
// the folder's files and sizes: listed here, or already listed on a worker thread (the app)
let listing: Vec<(String, u64)> = match p.get("listing").and_then(Value::as_array) {
Some(l) => l.iter().filter_map(|e| Some((e.get(0)?.as_str()?.to_string(), e.get(1)?.as_u64()?))).collect(),
None => list_auto_import_folder(&folder).map_err(|e| bad("library.autoImportScan", e))?,
};
let mut fresh = Vec::new();
for e in std::fs::read_dir(&folder).map_err(|e| bad("library.autoImportScan", format!("{folder}: {e}")))?.flatten() {
let path = e.path();
let ps = path.to_string_lossy().to_string();
if !path.is_file() || known.contains(&ps) || path.file_name().is_some_and(|n| n.to_string_lossy().starts_with('.')) {
for (ps, size) in listing {
if known.contains(&ps) {
continue;
}
// each file is tried once (a non-photo isn't retried every scan)
if s.auto_import_seen.get(&ps) == Some(&u64::MAX) {
continue;
}
let size = e.metadata().map(|m| m.len()).unwrap_or(0);
let seen = s.auto_import_seen.insert(ps.clone(), size);
if size > 0 && seen == Some(size) {
fresh.push(ps);
@@ -701,6 +802,10 @@ pub fn specs() -> Vec<CommandSpec> {
for f in &fresh {
s.auto_import_seen.insert(f.clone(), u64::MAX);
}
if p.get("start").and_then(Value::as_bool) == Some(false) {
// the caller imports them (the app: on a worker thread)
return Ok(json!({"imported": [], "folder": folder, "import": params}));
}
let sel = s.selection.clone();
let r = s.execute("library.import", &params)?;
// arriving photos don't take over the selection
@@ -751,79 +856,15 @@ pub fn specs() -> Vec<CommandSpec> {
"{paths: [file or folder (recursive)], mode?: add|copy|move (add = reference the files in place; copy = into the library's Originals/YYYY/YYYY-MM-DD/; move = as copy, then each original and its XMP sidecars are removed from the source — only after the copy is verified (a hard link on the same volume, else copied, synced and compared byte for byte) and its catalog record is saved; failed, duplicate and unchecked files keep their sources; a taken name gets -1, -2…; undo removes the photos from the library but leaves the files at the destination), destination?: folder for copies / moves, organize?: date (YYYY/YYYY-MM-DD) | month (YYYY/YYYY-MM) | flat | a folder template, e.g. `{date:%Y}/{date:%Y%m%d}` → 2026/20260114 (the template's `/` make the folders, each level expanded with the rename tokens and made a safe folder name: never outside the destination; must be relative, no `..`; a level with missing metadata is `unknown`) — dated by capture time, else the import time, rename?: file-name template for copies, original extension added (tokens: {name} {num} {seq} {seq:N} {date} {date:%Y%m%d} {folder} {camera} {lens} {iso} {rating} {title} {creator} {ext}; photo.renameTokens explains each; blank = keep names), renameStart?: 1, metadataPreset?: name, dng?: bool (copy raws as DNG; copy only), local?: bool (browsing: the photos stay out of the library, like library.browse; not with move), album?: albumId, albumName?: new album, preset?: presetId, keywords?: [..]} → {imported, duplicates, failed, moved?: [{from, to, sidecars?}], kept?: [{path, reason}] (move: sources left in place and why), album?}",
always,
|s, p| {
let paths = strs(p, "paths");
if paths.is_empty() {
let req = import_params(s, p)?;
if req.paths.is_empty() {
return Err(bad("library.import", "no paths"));
}
let mode = match str_param(p, "mode").unwrap_or("add") {
"add" => crate::import::ImportMode::Add,
"copy" => crate::import::ImportMode::Copy,
"move" => crate::import::ImportMode::Move,
other => return Err(bad("library.import", format!("unknown mode `{other}` (add|copy|move)"))),
};
let preset = match str_param(p, "preset").filter(|x| !x.is_empty()) {
Some(id) => {
Some(s.presets.iter().find(|x| x.id == id).cloned().ok_or_else(|| bad("library.import", format!("unknown preset `{id}`")))?)
}
None => None,
};
let mut album = p.get("album").and_then(Value::as_u64);
if let Some(a) = album
&& s.catalog.album(AlbumId(a)).is_none_or(|al| al.folder || al.is_smart())
{
return Err(bad("library.import", "album must be a regular album"));
}
let organize = match str_param(p, "organize") {
Some(o) => crate::import::Organize::parse(o).ok_or_else(|| {
bad(
"library.import",
format!("unknown organize `{o}` (date|month|flat, or a folder template like {{date:%Y}}/{{date:%Y%m%d}})"),
)
})?,
None => Default::default(),
};
if let crate::import::Organize::Template(t) = &organize
&& let Some(e) = crate::rename::folder_template_error(t)
{
return Err(bad("library.import", e));
}
let metadata_preset = str_param(p, "metadataPreset").map(str::trim).filter(|n| !n.is_empty()).map(str::to_string);
if let Some(n) = &metadata_preset
&& !s.metadata_presets.iter().any(|m| m.name.eq_ignore_ascii_case(n))
{
return Err(bad("library.import", format!("unknown metadata preset `{n}`")));
}
let opts = crate::import::ImportOptions {
mode,
preset,
keywords: strs(p, "keywords"),
destination: str_param(p, "destination").map(str::to_string),
organize,
rename: str_param(p, "rename").map(str::to_string),
rename_start: p.get("renameStart").and_then(Value::as_u64).unwrap_or(1) as usize,
metadata_preset,
convert_dng: super::bool_or(p, "dng", false),
local: super::bool_or(p, "local", false),
};
let undo0 = s.undo.len();
let mut report = serde_json::to_value(crate::import::import_with(s, &paths, &opts)?).unwrap_or_default();
let imported: Vec<u64> = report["imported"].as_array().map(|a| a.iter().filter_map(Value::as_u64).collect()).unwrap_or_default();
if album.is_none()
&& let Some(name) = str_param(p, "albumName").map(str::trim).filter(|n| !n.is_empty())
&& !imported.is_empty()
{
let r = s.execute("album.create", &json!({"name": name}))?;
album = r["id"].as_u64();
}
if let Some(a) = album
&& !imported.is_empty()
{
// (album.addPhotos wants a selection: the new photos are about to be it)
s.selection = Selection::single(PhotoId(imported[0]));
s.execute("album.addPhotos", &json!({"id": a, "ids": imported}))?;
report["album"] = json!(a);
}
let report = crate::import::import_with(s, &req.paths, &req.opts)?;
let report = import_batch_done(s, report, req.album, req.album_name.as_deref())?;
// one undo step for the whole import
let imported: Vec<u64> = report["imported"].as_array().map(|a| a.iter().filter_map(Value::as_u64).collect()).unwrap_or_default();
let n = s.undo.len().saturating_sub(undo0);
s.merge_undo(n, &format!("Add {} Photo{}", imported.len(), if imported.len() == 1 { "" } else { "s" }));
if let Some(f) = imported.first() {
@@ -867,6 +908,8 @@ pub fn specs() -> Vec<CommandSpec> {
"logRecords": j.log_records(),
"logBytes": j.log_bytes(),
"lastError": lib.last_error,
// settings files that were unreadable or damaged at open (kept, defaults used)
"settingsWarnings": lib.settings_warnings,
// changes applied in memory whose write failed (retried by every save)
"unsavedOps": s.unsaved().map_or(0, |u| u.0),
"unsavedError": s.unsaved().map(|u| u.1),
+2 -2
View File
@@ -185,7 +185,7 @@ pub fn specs() -> Vec<CommandSpec> {
let add = p.get("add").and_then(Value::as_bool).unwrap_or(false);
let id = s.active().ok_or_else(|| bad(c, "no active photo"))?;
let src = s.source_now(id, crate::media::SourceLevel::Thumb).map_err(|e| bad(c, e))?;
let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default();
let info = s.source_info(id);
let d = s.develop_of(id).unwrap_or_default();
let req = lightcraft_pipeline::RenderRequest::fit(384, 384);
let lab = lightcraft_pipeline::color_range_sample(&src, &info, &d, &req, lightcraft_geom::Point::new(x, y))
@@ -646,7 +646,7 @@ fn spots_edit(s: &mut Session, c: &str, label: &str, f: impl FnOnce(&mut Vec<Spo
/// An automatic source for `spot` on photo `id` (a small proxy of the photo, framed by `d`).
fn pick_source(s: &mut Session, id: crate::PhotoId, d: &lightcraft_develop::DevelopSettings, spot: &Spot, avoid: Option<Point>) -> Option<Point> {
let src = s.source_now(id, crate::media::SourceLevel::Thumb).ok()?;
let info = crate::media::source_info(s.catalog.photo(id)?);
let info = s.source_info(id);
lightcraft_pipeline::spots::pick_source(&src, &info, d, spot, avoid)
}
+184 -42
View File
@@ -45,9 +45,10 @@ pub fn missing(s: &Session) -> Vec<(PhotoId, String)> {
}
/// Whether photo `id` is in scope and its file is gone (the Missing Photos view checks only
/// the photos its query already narrowed to).
pub fn is_missing(cat: &Catalog, id: PhotoId) -> bool {
!cfg!(target_arch = "wasm32") && cat.photo(id).and_then(|p| checked_path(p)).is_some_and(|f| !Path::new(f).exists())
/// the photos its query already narrowed to). In the app `avail` answers from its cache (files
/// not checked yet count as present until the background check finds them gone).
pub fn is_missing(cat: &Catalog, avail: &crate::availability::Availability, id: PhotoId) -> bool {
!cfg!(target_arch = "wasm32") && cat.photo(id).and_then(|p| checked_path(p)).is_some_and(|f| avail.is_offline(f))
}
fn relink_op(id: PhotoId, path: &str) -> Op {
@@ -94,8 +95,112 @@ fn file_hash(path: &str) -> Option<String> {
std::fs::read(path).ok().map(|b| lightcraft_preview::hash_bytes(&b).to_string())
}
/// Search `folder` (recursively) for each missing photo's file and relink every match in one
/// undoable step.
/// An in-scope photo ([`checked_path`]) as Find Missing Photos sees it: its file, the size and
/// content hash the library knows (0 / `None` = unknown).
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct FindCandidate {
pub id: PhotoId,
pub path: String,
pub size: u64,
pub hash: Option<String>,
}
/// Every in-scope photo, for [`plan_find_missing`]: the missing ones are looked for, and no
/// photo's file is used for another. No file-system calls, so the app takes this snapshot on the
/// UI thread and plans on a worker without holding the session.
pub fn find_candidates(cat: &Catalog) -> Vec<FindCandidate> {
cat.photos()
.filter_map(|p| {
checked_path(p).map(|f| FindCandidate { id: p.id, path: f.to_string(), size: p.file_size, hash: stored_hash(p).map(str::to_string) })
})
.collect()
}
/// How a missing photo's file was recognised.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum MatchBy {
/// Same name and size (and content, when the library knows its hash).
Name,
/// Renamed: same size and content hash.
Content,
}
impl MatchBy {
pub fn as_str(self) -> &'static str {
match self {
MatchBy::Name => "name",
MatchBy::Content => "content",
}
}
}
/// A missing photo and the file found for it.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct FoundFile {
pub id: PhotoId,
pub from: String,
pub to: String,
pub by: MatchBy,
}
/// A missing photo with several look-alike files (same name and size, no hash to tell them
/// apart): skipped, never guessed.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct AmbiguousFile {
pub id: PhotoId,
pub from: String,
pub candidates: Vec<String>,
}
/// What Find Missing Photos found ([`plan_find_missing`]), before relinking.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct FindPlan {
pub found: Vec<FoundFile>,
/// In-scope photos that stay missing (the ambiguous ones included).
pub missing: usize,
pub ambiguous: Vec<AmbiguousFile>,
}
impl FindPlan {
/// `{found: [{id, from, to, by}], missing, ambiguous: [{id, from, candidates}]}` — the
/// command's result, and what `library.findMissing {found, …}` takes back.
pub fn to_json(&self) -> Value {
let found: Vec<Value> = self.found.iter().map(|f| json!({"id": f.id.0, "from": f.from, "to": f.to, "by": f.by.as_str()})).collect();
let ambiguous: Vec<Value> = self.ambiguous.iter().map(|a| json!({"id": a.id.0, "from": a.from, "candidates": a.candidates})).collect();
json!({"found": found, "missing": self.missing, "ambiguous": ambiguous})
}
/// The inverse of [`FindPlan::to_json`] (entries it can't read are dropped).
pub fn from_json(v: &Value) -> FindPlan {
let list = |k: &str| v.get(k).and_then(Value::as_array).cloned().unwrap_or_default();
let found = list("found")
.iter()
.filter_map(|f| {
let by = if f.get("by").and_then(Value::as_str) == Some("content") { MatchBy::Content } else { MatchBy::Name };
Some(FoundFile {
id: PhotoId(f.get("id")?.as_u64()?),
from: str_param(f, "from")?.to_string(),
to: str_param(f, "to")?.to_string(),
by,
})
})
.collect();
let ambiguous = list("ambiguous")
.iter()
.filter_map(|a| {
let candidates = a.get("candidates")?.as_array()?.iter().filter_map(|c| c.as_str().map(str::to_string)).collect();
Some(AmbiguousFile { id: PhotoId(a.get("id")?.as_u64()?), from: str_param(a, "from")?.to_string(), candidates })
})
.collect();
let missing = v.get("missing").and_then(Value::as_u64).map_or(0, |n| usize::try_from(n).unwrap_or(usize::MAX));
FindPlan { found, missing, ambiguous }
}
}
/// The file-system half of Find Missing Photos: which `candidates` are gone, and which file in
/// `folder` (recursively) is each one's. Touches only the disk — never the session — so the app
/// runs it on a worker thread (walking a big folder on a network share and hashing look-alikes
/// takes a while) and relinks the result with `library.findMissing {found, missing, ambiguous}`.
///
/// Matching, per photo:
/// 1. a file with the same name (any letter case), the same size and the same content hash;
@@ -107,20 +212,21 @@ fn file_hash(path: &str) -> Option<String> {
///
/// Files that are already some photo's original, or that another photo was just matched with,
/// are not used.
fn find_missing(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "library.findMissing";
let folder = str_param(p, "folder").ok_or_else(|| bad(C, "missing `folder`"))?;
pub fn plan_find_missing(candidates: &[FindCandidate], folder: &str) -> std::result::Result<FindPlan, String> {
if !Path::new(folder).is_dir() {
return Err(bad(C, format!("{folder}: not a folder")));
return Err(format!("{folder}: not a folder"));
}
let lost = missing(s);
if cfg!(target_arch = "wasm32") {
return Ok(FindPlan::default());
}
let lost: Vec<&FindCandidate> = candidates.iter().filter(|c| !Path::new(&c.path).exists()).collect();
if lost.is_empty() {
return Ok(json!({"found": [], "missing": 0, "ambiguous": []}));
return Ok(FindPlan::default());
}
let in_use: HashSet<String> = s.catalog.photos().filter_map(|p| checked_path(p).map(str::to_string)).collect();
let in_use: HashSet<&str> = candidates.iter().map(|c| c.path.as_str()).collect();
// file name (lower case) → candidate paths; size → paths (filled lazily)
let mut by_name: HashMap<String, Vec<String>> = HashMap::new();
let files: Vec<String> = crate::import::expand(&[folder.to_string()], None).into_iter().filter(|f| !in_use.contains(f)).collect();
let files: Vec<String> = crate::import::expand(&[folder.to_string()], None).into_iter().filter(|f| !in_use.contains(f.as_str())).collect();
for f in &files {
if let Some(n) = Path::new(f).file_name() {
by_name.entry(n.to_string_lossy().to_lowercase()).or_default().push(f.clone());
@@ -131,24 +237,21 @@ fn find_missing(s: &mut Session, p: &Value) -> Result<Value> {
let mut hash_of = |f: &str| hashes.entry(f.to_string()).or_insert_with(|| file_hash(f)).clone();
let size_of = |f: &str| std::fs::metadata(f).map(|m| m.len()).ok();
let mut used: HashSet<String> = HashSet::new();
let mut ops = Vec::new();
let mut found = Vec::new();
let mut ambiguous = Vec::new();
for (id, old) in &lost {
let Some(ph) = s.catalog.photo(*id) else { continue };
let name = Path::new(old).file_name().map(|n| n.to_string_lossy().to_lowercase()).unwrap_or_default();
let size = ph.file_size;
let hash = stored_hash(ph).map(str::to_string);
let mut plan = FindPlan::default();
for c in &lost {
let name = Path::new(&c.path).file_name().map(|n| n.to_string_lossy().to_lowercase()).unwrap_or_default();
let size = c.size;
let hash = c.hash.as_deref();
let named: Vec<&String> = by_name
.get(&name)
.map(|c| c.iter().filter(|f| !used.contains(*f) && (size == 0 || size_of(f) == Some(size))).collect())
.map(|v| v.iter().filter(|f| !used.contains(*f) && (size == 0 || size_of(f) == Some(size))).collect())
.unwrap_or_default();
// 1. same name and size, confirmed by the content hash
let mut hit: Option<(String, &str)> =
hash.as_deref().and_then(|h| named.iter().find(|f| hash_of(f).as_deref() == Some(h))).map(|f| (f.to_string(), "name"));
let mut hit: Option<(String, MatchBy)> =
hash.and_then(|h| named.iter().find(|f| hash_of(f).as_deref() == Some(h))).map(|f| (f.to_string(), MatchBy::Name));
// 2. renamed: same size and content
if hit.is_none()
&& let Some(h) = &hash
&& let Some(h) = hash
&& size > 0
{
let sizes = by_size.get_or_insert_with(|| {
@@ -162,36 +265,75 @@ fn find_missing(s: &mut Session, p: &Value) -> Result<Value> {
});
hit = sizes
.get(&size)
.and_then(|c| c.iter().find(|f| !used.contains(*f) && hash_of(f).as_deref() == Some(h.as_str())))
.map(|f| (f.clone(), "content"));
.and_then(|v| v.iter().find(|f| !used.contains(*f) && hash_of(f).as_deref() == Some(h)))
.map(|f| (f.clone(), MatchBy::Content));
}
// 3. same name and size only (no hash, or the file was edited in place): one candidate
// is taken, several are ambiguous and skipped rather than guessed
if hit.is_none() {
match named.len() {
1 => hit = Some((named[0].clone(), "name")),
0 => {}
match named.as_slice() {
[one] => hit = Some(((*one).clone(), MatchBy::Name)),
[] => {}
_ => {
ambiguous.push(json!({"id": id.0, "from": old, "candidates": named}));
plan.ambiguous.push(AmbiguousFile { id: c.id, from: c.path.clone(), candidates: named.iter().map(|f| (*f).clone()).collect() });
continue;
}
}
}
let Some((path, by)) = hit else { continue };
ops.push(relink_op(*id, &path));
found.push(json!({"id": id.0, "from": old, "to": path, "by": by}));
used.insert(path);
used.insert(path.clone());
plan.found.push(FoundFile { id: c.id, from: c.path.clone(), to: path, by });
}
let still = lost.len() - found.len();
if !ops.is_empty() {
s.commit("Find Missing Photos", Op::Batch { ops })?;
plan.missing = lost.len() - plan.found.len();
Ok(plan)
}
/// Relink what [`plan_find_missing`] found, in one undoable step, re-checked against the
/// catalog as it is now (the search may have run on a worker while the library changed): a
/// photo that no longer points at the path it was missing from (relinked, removed) is left
/// alone, and a file that is now some photo's original — or that the plan names twice — is not
/// used (that photo stays missing). `missing` is the plan's count, corrected for what changed
/// among the photos the plan names.
fn commit_find(s: &mut Session, plan: FindPlan) -> Result<Value> {
let still_at = |cat: &Catalog, id: PhotoId, from: &str| cat.photo(id).and_then(|p| checked_path(p)) == Some(from);
let mut in_use: HashSet<String> = candidates(&s.catalog).into_iter().collect();
let mut found = Vec::new();
let mut missing = plan.missing;
for f in plan.found {
if !still_at(&s.catalog, f.id, &f.from) {
continue;
}
if in_use.contains(&f.to) {
missing += 1;
continue;
}
in_use.insert(f.to.clone());
found.push(f);
}
let planned = plan.ambiguous.len();
let ambiguous: Vec<AmbiguousFile> = plan.ambiguous.into_iter().filter(|a| still_at(&s.catalog, a.id, &a.from)).collect();
// ambiguous photos relinked meanwhile aren't missing any more
missing = missing.saturating_sub(planned - ambiguous.len());
if !found.is_empty() {
s.commit("Find Missing Photos", Op::Batch { ops: found.iter().map(|f| relink_op(f.id, &f.to)).collect() })?;
for f in &found {
if let Some(id) = f["id"].as_u64() {
s.media.forget(PhotoId(id));
}
s.media.forget(f.id);
}
}
Ok(json!({"found": found, "missing": still, "ambiguous": ambiguous}))
Ok(FindPlan { found, missing, ambiguous }.to_json())
}
/// Search `folder` (recursively) for each missing photo's file ([`plan_find_missing`]) and
/// relink every match in one undoable step. With `found` instead (what a search on a worker
/// thread returned, as `{found, missing, ambiguous}`), only the relinking is done.
fn find_missing(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "library.findMissing";
if p.get("found").is_some_and(Value::is_array) {
return commit_find(s, FindPlan::from_json(p));
}
let folder = str_param(p, "folder").ok_or_else(|| bad(C, "missing `folder`"))?;
let plan = plan_find_missing(&find_candidates(&s.catalog), folder).map_err(|e| bad(C, e))?;
commit_find(s, plan)
}
pub fn specs() -> Vec<CommandSpec> {
@@ -213,7 +355,7 @@ pub fn specs() -> Vec<CommandSpec> {
"Find Missing Photos",
[],
None,
"{folder} — relink every missing photo whose file is somewhere in the folder: same name and size (and content, when the library knows its hash), or — renamed — same size and content → {found: [{id, from, to, by: name|content}], missing, ambiguous: [{id, from, candidates}] (several same-name candidates and no hash to tell them apart: skipped)}",
"{folder} — relink every missing photo whose file is somewhere in the folder: same name and size (and content, when the library knows its hash), or — renamed — same size and content; or {found, missing, ambiguous} — relink what a search already found (the app searches on a worker thread; photos relinked meanwhile and files now in use are skipped) → {found: [{id, from, to, by: name|content}], missing, ambiguous: [{id, from, candidates}] (several same-name candidates and no hash to tell them apart: skipped)}",
always,
find_missing
),
+2 -2
View File
@@ -5,7 +5,7 @@
//! `edit.*` (undo/redo), and queries (`catalog.query`, `photo.inspect`, `develop.get`…).
mod before;
mod browse;
pub(crate) mod browse;
mod color;
pub(crate) mod convert;
mod cull;
@@ -15,7 +15,7 @@ mod edit;
mod export;
pub mod filters;
pub mod keywords;
mod library;
pub mod library;
pub mod lut_profiles;
pub mod manage;
mod masks;
+131 -38
View File
@@ -23,6 +23,12 @@ pub struct PreviewBuild {
pub failed: AtomicUsize,
pub cancel: AtomicBool,
pub finished: AtomicBool,
/// Damaged smart previews rebuilt (Build Smart Previews only).
pub repaired: AtomicUsize,
/// What is built ("" = previews; "smart previews" — Build Smart Previews runs the same way).
pub what: &'static str,
/// Why the whole build failed (e.g. the smart previews folder isn't there).
pub error: std::sync::Mutex<Option<String>>,
}
impl PreviewBuild {
@@ -33,8 +39,15 @@ impl PreviewBuild {
"failed": self.failed.load(Ordering::Relaxed),
"running": !self.finished.load(Ordering::Relaxed),
"cancelled": self.cancel.load(Ordering::Relaxed),
"what": if self.what.is_empty() { "previews" } else { self.what },
"repaired": self.repaired.load(Ordering::Relaxed),
"error": self.error(),
})
}
pub fn error(&self) -> Option<String> {
self.error.lock().unwrap_or_else(std::sync::PoisonError::into_inner).clone()
}
}
/// The jobs that build `id`'s previews: the largest grid thumbnail, then the view render at
@@ -94,56 +107,136 @@ fn build(s: &mut Session, p: &Value) -> Result<Value> {
Ok(state.json())
}
/// One photo's smart preview to build or discard: (photo, its file in the smart previews folder,
/// how to load the original at preview size).
#[cfg(not(target_arch = "wasm32"))]
type SmartJob = (lightcraft_catalog::PhotoId, std::path::PathBuf, Option<crate::media::SourceRef>);
/// What Build / Discard Smart Previews did.
#[cfg(not(target_arch = "wasm32"))]
#[derive(Default)]
struct SmartCounts {
/// Proxies there now (complete ones kept, missing or damaged ones written).
built: usize,
/// Of those, damaged ones (cut short by a crash or a full drive) that were rebuilt.
repaired: usize,
removed: usize,
/// `[[id, why]]`
failed: Vec<Value>,
}
/// The file-system half of Build / Discard Smart Previews (runs on a worker thread in the app).
/// `custom`: the folder was chosen (on another drive, perhaps): it must be there, it is never
/// recreated. With `state`, progress and cancel.
#[cfg(not(target_arch = "wasm32"))]
fn smart_run(
dir: &std::path::Path,
custom: bool,
discard: bool,
jobs: Vec<SmartJob>,
state: Option<&PreviewBuild>,
) -> std::result::Result<SmartCounts, String> {
let mut n = SmartCounts::default();
if !discard {
// a chosen folder on another drive must be there: never recreate it on this one
if custom {
crate::smart::check_writable(dir)?;
} else {
std::fs::create_dir_all(dir).map_err(|e| format!("{}: {e}", dir.display()))?;
}
}
for (id, path, source) in jobs {
if state.is_some_and(|st| st.cancel.load(Ordering::Relaxed)) {
break;
}
if discard {
if std::fs::remove_file(&path).is_ok() {
n.removed += 1;
}
} else {
// a complete proxy is kept; a missing or damaged one (cut short by a crash or a full
// drive) is (re)built
let damaged = path.exists();
if damaged && crate::smart::is_valid(&path) {
n.built += 1;
} else {
// atomic: a failed write leaves no partial proxy that would pass for a built one;
// and synced (issue #134): built so the photo can be edited while its original
// is offline, when it is the only copy — the sync is small next to the decode
let r = source
.ok_or_else(|| "nothing to build from".to_string())
.and_then(|s| s.load_source())
.and_then(|src| crate::smart::encode(&src.image, src.info_or(Default::default()).camera_tone.as_ref()))
.and_then(|b| lightcraft_catalog::safe_file::write_atomic(&path, &b).map_err(|e| format!("{}: {e}", path.display())));
match r {
Ok(()) => {
n.built += 1;
if damaged {
n.repaired += 1;
if let Some(st) = state {
st.repaired.fetch_add(1, Ordering::Relaxed);
}
}
}
Err(e) => {
n.failed.push(json!([id.0, e]));
if let Some(st) = state {
st.failed.fetch_add(1, Ordering::Relaxed);
}
}
}
}
}
if let Some(st) = state {
st.done.fetch_add(1, Ordering::Relaxed);
}
}
Ok(n)
}
/// Smart previews: build (from the original, at preview size) or discard the proxies that keep
/// photos editable while their originals are offline.
/// photos editable while their originals are offline. With `background` (the app) the files are
/// read and written on a worker thread; progress like Build Previews (`library.previewProgress`).
#[cfg(not(target_arch = "wasm32"))]
fn smart(s: &mut Session, p: &Value) -> Result<Value> {
const C: &str = "library.smartPreviews";
let dir = s.media.smart_dir.clone().ok_or_else(|| bad(C, "smart previews need a library on disk"))?;
let discard = p.get("discard").and_then(Value::as_bool).unwrap_or(false);
let ids = if p.get("ids").is_some() || !s.selection.ids.is_empty() { s.targets(p) } else { s.visible_cloned() };
let (mut built, mut repaired, mut removed, mut failed) = (0usize, 0usize, 0usize, Vec::new());
if !discard {
// a chosen folder on another drive must be there: never recreate it on this one
if s.smart_previews_dir.is_some() {
crate::smart::check_writable(&dir).map_err(|e| bad(C, e))?;
} else {
std::fs::create_dir_all(&dir).map_err(|e| bad(C, format!("{}: {e}", dir.display())))?;
}
let background = p.get("background").and_then(Value::as_bool).unwrap_or(false);
if background && s.preview_build.as_ref().is_some_and(|b| !b.finished.load(Ordering::Relaxed)) {
return Err(bad(C, "a preview build is already running"));
}
let ids = if p.get("ids").is_some() || !s.selection.ids.is_empty() { s.targets(p) } else { s.visible_cloned() };
let mut jobs: Vec<SmartJob> = Vec::new();
for id in ids {
let Some(ph) = s.catalog.photo(id).cloned() else { continue };
if !matches!(ph.source, lightcraft_catalog::Source::File { .. }) {
continue;
}
let path = dir.join(crate::smart::file_name(&ph));
if discard {
if std::fs::remove_file(&path).is_ok() {
removed += 1;
}
continue;
}
// a complete proxy is kept; a missing or damaged one (cut short by a crash or a full
// drive) is (re)built
let damaged = path.exists();
if damaged && crate::smart::is_valid(&path) {
built += 1;
continue;
}
// atomic: a failed write leaves no partial proxy that would pass for a built one
let r = s
.source_now(id, crate::media::SourceLevel::Preview)
.and_then(|src| crate::smart::encode(&src))
.and_then(|b| lightcraft_catalog::safe_file::write_atomic(&path, &b).map_err(|e| format!("{}: {e}", path.display())));
match r {
Ok(()) => {
built += 1;
repaired += usize::from(damaged);
}
Err(e) => failed.push(json!([id.0, e])),
}
let source = (!discard).then(|| s.media.source_ref(&ph, crate::media::SourceLevel::Preview));
jobs.push((id, path, source));
}
Ok(json!({"built": built, "repaired": repaired, "removed": removed, "failed": failed}))
let custom = s.smart_previews_dir.is_some();
if background {
let what = if discard { "discarding smart previews" } else { "smart previews" };
let state = Arc::new(PreviewBuild { total: jobs.len(), what, ..Default::default() });
s.preview_build = Some(state.clone());
let st = state.clone();
std::thread::Builder::new()
.name("lc-smart-previews".into())
.spawn(move || {
if let Err(e) = smart_run(&dir, custom, discard, jobs, Some(&st)) {
log::warn!("{C}: {e}");
*st.error.lock().unwrap_or_else(std::sync::PoisonError::into_inner) = Some(e);
}
st.finished.store(true, Ordering::Relaxed);
})
.map_err(|e| bad(C, format!("could not start: {e}")))?;
return Ok(state.json());
}
let r = smart_run(&dir, custom, discard, jobs, None).map_err(|e| bad(C, e))?;
Ok(json!({"built": r.built, "repaired": r.repaired, "removed": r.removed, "failed": r.failed}))
}
/// The smart previews folder: where it is, whether it is custom, what it holds, whether it can be
@@ -225,7 +318,7 @@ pub fn specs() -> Vec<CommandSpec> {
"Build Smart Previews",
[],
None,
"{ids?, discard?: bool} — build (or discard) the smart previews of the selected photos (else all in view): compact proxies in the library that keep photos editable and exportable (at proxy size) while their originals are offline; a damaged proxy (cut short) is rebuilt → {built, repaired, removed, failed}",
"{ids?, discard?: bool, background?: bool} — build (or discard) the smart previews of the selected photos (else all in view): compact proxies in the library that keep photos editable and exportable (at proxy size) while their originals are offline; a damaged proxy (cut short) is rebuilt → {built, repaired, removed, failed}; with background (the app's menu) the files are read and written on a worker thread → {total, done, failed, repaired, running, what} like library.buildPreviews (progress: library.previewProgress, stop: library.cancelPreviews)",
always,
smart
),
@@ -256,7 +349,7 @@ pub fn specs() -> Vec<CommandSpec> {
always,
build
),
cmd!(query "library.previewProgress", "Preview Build Progress", [], None, "{} → {total, done, failed, running, cancelled} | null", always, |s, _| {
cmd!(query "library.previewProgress", "Preview Build Progress", [], None, "{} → {total, done, failed, repaired, running, cancelled, what, error} | null", always, |s, _| {
Ok(s.preview_build.as_ref().map_or(Value::Null, |b| b.json()))
}),
cmd!("library.cancelPreviews", "Cancel Preview Build", [], None, "{}", always, |s, _| {
+83 -29
View File
@@ -244,14 +244,23 @@ impl Default for Watermark {
/// Inter SemiBold (OFL, see assets/ATTRIBUTION.md).
static WATERMARK_FONT: &[u8] = include_bytes!("../../../assets/fonts/Inter-SemiBold.ttf");
/// BIZ UDMincho (OFL), Japanese fallback on both native and web.
/// Bundled OFL Japanese font bytes, shared by watermarks and UI glyph fallback.
pub static WATERMARK_JAPANESE_FONT: &[u8] = include_bytes!("../../../assets/fonts/BIZUDMincho-Regular.ttf");
/// The watermark faces: Inter first, then the craft-fonts Japanese faces (Mincho first, the
/// watermark's serif look; then any other Japanese face). Without craft-fonts that is Inter alone,
/// and Japanese characters draw as Inter's missing-glyph box.
fn watermark_fonts(craft: &'static [crate::fonts::CraftFont]) -> Vec<ab_glyph::FontRef<'static>> {
let mut japanese: Vec<_> = crate::fonts::japanese(craft).collect();
japanese.sort_by_key(|f| !f.is_mincho());
std::iter::once(WATERMARK_FONT)
.chain(japanese.into_iter().map(|f| f.bytes))
.filter_map(|bytes| ab_glyph::FontRef::try_from_slice(bytes).ok())
.collect()
}
/// Draw `wm` onto `img` (straight alpha blending of the encoded values).
pub fn draw_watermark(img: &mut Rgba8, wm: &Watermark) {
let width = img.width;
watermark_coverage(img.width, img.height, wm, |x, y, k, col| {
watermark_coverage(img.width, img.height, wm, crate::fonts::CRAFT_FONTS, |x, y, k, col| {
let p = &mut img.data[y * width + x];
for c in 0..3 {
p[c] = (p[c] as f32 + (col[c] as f32 - p[c] as f32) * k).round() as u8;
@@ -264,14 +273,14 @@ pub fn draw_watermark(img: &mut Rgba8, wm: &Watermark) {
pub fn draw_watermark_deep(img: &mut DeepImage, wm: &Watermark) {
let (width, trc) = (img.width, img.space.trc());
match &mut img.samples {
DeepSamples::U16(v) => watermark_coverage(img.width, img.height, wm, |x, y, k, col| {
DeepSamples::U16(v) => watermark_coverage(img.width, img.height, wm, crate::fonts::CRAFT_FONTS, |x, y, k, col| {
let i = (y * width + x) * 3;
for c in 0..3 {
let p = v[i + c] as f32;
v[i + c] = (p + (col[c] as f32 * 257.0 - p) * k).round().clamp(0.0, 65535.0) as u16;
}
}),
DeepSamples::F32(v) => watermark_coverage(img.width, img.height, wm, |x, y, k, col| {
DeepSamples::F32(v) => watermark_coverage(img.width, img.height, wm, crate::fonts::CRAFT_FONTS, |x, y, k, col| {
let i = (y * width + x) * 3;
for c in 0..3 {
let target = trc.decode(col[c] as f32 / 255.0);
@@ -282,9 +291,16 @@ pub fn draw_watermark_deep(img: &mut DeepImage, wm: &Watermark) {
}
/// Lay out `wm` on a `width × height` image and call `blend(x, y, coverage × opacity, colour)` for
/// every covered pixel (shadow pass first).
fn watermark_coverage(width: usize, height: usize, wm: &Watermark, mut blend_px: impl FnMut(usize, usize, f32, [u8; 3])) {
use ab_glyph::{Font, FontRef, PxScale, ScaleFont, point};
/// every covered pixel (shadow pass first). `craft` is [`crate::fonts::CRAFT_FONTS`] (a parameter
/// so tests can render without it).
fn watermark_coverage(
width: usize,
height: usize,
wm: &Watermark,
craft: &'static [crate::fonts::CraftFont],
mut blend_px: impl FnMut(usize, usize, f32, [u8; 3]),
) {
use ab_glyph::{Font, PxScale, ScaleFont, point};
if !wm.image.trim().is_empty() {
logo_coverage(width, height, wm, blend_px);
return;
@@ -293,9 +309,8 @@ fn watermark_coverage(width: usize, height: usize, wm: &Watermark, mut blend_px:
if text.is_empty() || width == 0 || height == 0 {
return;
}
let Ok(font) = FontRef::try_from_slice(WATERMARK_FONT) else { return };
let Ok(japanese) = FontRef::try_from_slice(WATERMARK_JAPANESE_FONT) else { return };
let fonts = [font, japanese];
let fonts = watermark_fonts(craft);
let Some(latin) = fonts.first() else { return };
let short = width.min(height) as f32;
let px = (wm.size.clamp(0.005, 0.5) * short).max(6.0);
let mut glyphs = Vec::new();
@@ -333,9 +348,11 @@ fn watermark_coverage(width: usize, height: usize, wm: &Watermark, mut blend_px:
'…' => '︙',
other => other,
};
let ch = if wm.vertical && fonts[1].glyph_id(vertical_form).0 != 0 { vertical_form } else { original };
let face = usize::from(fonts[0].glyph_id(ch).0 == 0);
let sf = fonts[face].as_scaled(PxScale::from(px));
let has = |font: &ab_glyph::FontRef<'_>, c: char| font.glyph_id(c).0 != 0;
let ch = if wm.vertical && fonts.iter().skip(1).any(|f| has(f, vertical_form)) { vertical_form } else { original };
// The first face with the glyph; Inter (its missing-glyph box) when none has it.
let face = fonts.iter().position(|f| has(f, ch)).unwrap_or(0);
let sf = fonts.get(face).unwrap_or(latin).as_scaled(PxScale::from(px));
let id = sf.glyph_id(ch);
if wm.vertical {
let left = columns.saturating_sub(column.saturating_add(1)) as f32 * px;
@@ -1131,15 +1148,27 @@ pub fn export_batch(
}
/// Write an exported or rendered file on disk: its folder is created if needed, and the file is
/// replaced atomically ([`lightcraft_catalog::safe_file::write_atomic`]: a temp file, synced, then
/// renamed), so a failure part-way leaves any previous file intact and no truncated one. The
/// native writer behind exports, renders and screenshots (app, CLI, MCP).
/// replaced atomically ([`lightcraft_catalog::safe_file::write_atomic_nosync`]: a temp file
/// renamed into place), so a failure part-way leaves any previous file intact and no truncated
/// one. Not synced to disk (issue #134): an export can always be made again from the original.
/// The native writer behind exports, renders and screenshots (app, CLI, MCP).
pub fn write_file(path: &str, bytes: &[u8]) -> Result<(), String> {
write_with(path, bytes, false)
}
/// [`write_file`], synced to disk: for a written file that becomes a library photo (Edit a Copy)
/// and so can't simply be exported again once the catalog points at it.
pub fn write_file_durable(path: &str, bytes: &[u8]) -> Result<(), String> {
write_with(path, bytes, true)
}
fn write_with(path: &str, bytes: &[u8], durable: bool) -> Result<(), String> {
use lightcraft_catalog::safe_file::{write_atomic, write_atomic_nosync};
let p = std::path::Path::new(path);
if let Some(dir) = p.parent().filter(|d| !d.as_os_str().is_empty()) {
std::fs::create_dir_all(dir).map_err(|e| format!("{}: {e}", dir.display()))?;
}
lightcraft_catalog::safe_file::write_atomic(p, bytes).map_err(|e| format!("{path}: {e}"))
if durable { write_atomic(p, bytes) } else { write_atomic_nosync(p, bytes) }.map_err(|e| format!("{path}: {e}"))
}
/// The path of the sidecar with extension `ext` of the exported file `main`.
@@ -1334,21 +1363,46 @@ mod tests {
assert!(ExportOptions::from_json(&serde_json::json!({"watermark": ""})).watermark.is_none());
}
#[test]
fn japanese_watermarks_support_vertical_columns_and_legacy_defaults() {
use ab_glyph::Font;
let font = ab_glyph::FontRef::try_from_slice(WATERMARK_JAPANESE_FONT).unwrap();
for c in "日本語".chars() {
assert_ne!(font.glyph_id(c).0, 0);
}
let old = ExportOptions::from_json(&json!({"watermark": {"text": "日本語"}}));
assert!(!old.watermark.unwrap().vertical);
fn vertical_japanese() -> Watermark {
let options =
ExportOptions::from_json(&json!({"watermark": {"text": "日本語", "vertical": true, "size": 0.1, "shadow": false, "opacity": 1.0}}));
let wm = options.watermark.unwrap();
options.watermark.unwrap()
}
#[test]
fn japanese_watermark_options_and_legacy_defaults() {
let old = ExportOptions::from_json(&json!({"watermark": {"text": "日本語"}}));
assert!(!old.watermark.unwrap().vertical);
let wm = vertical_japanese();
assert!(wm.vertical);
let round: Watermark = serde_json::from_value(serde_json::to_value(&wm).unwrap()).unwrap();
assert!(round.vertical);
}
/// Built without craft-fonts, a Japanese watermark still lays out and draws (Inter's
/// missing-glyph boxes), and Latin text is unaffected.
#[test]
fn watermarks_work_without_craft_fonts() {
assert_eq!(watermark_fonts(&[]).len(), 1, "Inter only");
for wm in [vertical_japanese(), Watermark { text: "LightCraft 日本語".into(), ..Watermark::default() }] {
let mut covered = 0usize;
watermark_coverage(400, 300, &wm, &[], |_, _, k, _| covered += usize::from(k > 0.0));
assert!(covered > 0, "{:?} draws something", wm.text);
}
}
#[test]
fn japanese_watermarks_support_vertical_columns() {
use ab_glyph::Font;
let fonts = watermark_fonts(crate::fonts::CRAFT_FONTS);
if fonts.len() < 2 {
eprintln!("skipped: built without CRAFT_FONTS_DIR, so there is no Japanese watermark face");
return;
}
for c in "日本語の文字".chars() {
assert!(fonts[1..].iter().any(|f| f.glyph_id(c).0 != 0), "a craft-fonts face has {c}");
}
let wm = vertical_japanese();
let mut img = Rgba8::new(400, 300);
img.data.fill([0, 0, 0, 255]);
draw_watermark(&mut img, &wm);
+68 -10
View File
@@ -131,7 +131,9 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result<ProbeInfo, String> {
std::mem::swap(&mut w, &mut h);
}
let (t, tint) = xy_to_temp_tint(lightcraft_raw::color::as_shot_white_xy_of(&raw));
let as_shot_wb = Some((t.round(), tint.round()));
// Vendor RGB multipliers do not identify an absolute illuminant without camera calibration.
let relative = raw.format == lightcraft_raw::RawFormat::Arw && !lightcraft_raw::color::has_matrix(&raw.color);
let as_shot_wb = Some(if relative { (6500.0, 0.0) } else { (t.round(), tint.round()) });
let embedded_lens = embedded_lens(&raw);
return Ok(ProbeInfo {
embedded_lens,
@@ -221,6 +223,9 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result<
}
Err(e) => return Err(e.to_string()),
};
let xy = lightcraft_raw::color::as_shot_white_xy(&raw);
let t = lightcraft_raw::color::camera_transform(&raw, xy);
let camera_look = crate::camera_preview::fit_preview(&raw, &bytes, &t);
drop(bytes);
// Embedded lens corrections are applied by the pipeline ("Enable Profile Corrections"), not baked in.
let lens = embedded_lens(&raw.info());
@@ -242,21 +247,25 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result<
// the samples aren't needed any more (the colour model below reads only the tags)
raw.data = lightcraft_raw::RawData::U16(Vec::new());
let mut stages = vec![("develop", t0.elapsed())];
let xy = lightcraft_raw::color::as_shot_white_xy(&raw);
let t = lightcraft_raw::color::camera_transform(&raw, xy);
stages.push(("transform", t0.elapsed()));
lightcraft_raw::highlight::reconstruct(&mut img, t.wb, HIGHLIGHT_CLIP);
stages.push(("highlights", t0.elapsed()));
let m = t.matrix.to_f32();
let m = camera_look.map(|p| p.matrix.mul(&t.matrix)).unwrap_or(t.matrix).to_f32();
let gain = 2f32.powf(t.baseline_exposure as f32);
let wb = t.wb;
// A DNG's own profile look (hue/saturation map, look table), DNG spec chapter 6.
let tables = lightcraft_raw::profile::ProfileTables::new(&raw.color.profile, lightcraft_raw::color::illuminant_weight(&raw.color, xy));
img.map_in_place(|p| {
let c = [p[0] * wb[0], p[1] * wb[1], p[2] * wb[2]];
[
((m[0][0] * c[0] + m[0][1] * c[1] + m[0][2] * c[2]) * gain).max(0.0),
((m[1][0] * c[0] + m[1][1] * c[1] + m[1][2] * c[2]) * gain).max(0.0),
((m[2][0] * c[0] + m[2][1] * c[1] + m[2][2] * c[2]) * gain).max(0.0),
]
let rgb = [
m[0][0] * c[0] + m[0][1] * c[1] + m[0][2] * c[2],
m[1][0] * c[0] + m[1][1] * c[1] + m[1][2] * c[2],
m[2][0] * c[0] + m[2][1] * c[1] + m[2][2] * c[2],
];
match &tables {
Some(tables) => tables.apply(rgb, gain).map(|v| v.max(0.0)),
None => rgb.map(|v| (v * gain).max(0.0)),
}
});
stages.push(("colour", t0.elapsed()));
let img = fit(&img, max_edge, max_edge, Filter::Box);
@@ -276,7 +285,10 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result<
eprintln!("[profile] raw source {}×{} (max {max_edge}, ms after decode): {}", img.width, img.height, parts.join(", "));
}
let (temp, tint) = xy_to_temp_tint(xy);
return Ok((img, SourceInfo { raw: true, as_shot_temp: temp.round(), as_shot_tint: tint.round(), lens }));
let relative = raw.format == lightcraft_raw::RawFormat::Arw && t.matrix_is_fallback;
let camera_tone = camera_look.map(|p| p.tone).or_else(|| raw.color.profile.tone_curve.as_ref().and_then(dng_tone_curve));
let (temp, tint) = if relative { (6500.0, 0.0) } else { (temp.round(), tint.round()) };
return Ok((img, SourceInfo { raw: true, as_shot_temp: temp, as_shot_tint: tint, lens, relative_wb: relative, camera_tone }));
}
let d = lightcraft_codecs::decode(&bytes, lightcraft_codecs::DecodeOptions::fit(max_edge as u32, max_edge as u32)).map_err(|e| e.to_string())?;
drop(bytes);
@@ -285,6 +297,17 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result<
Ok((img.into_oriented(Orientation::from_exif(d.orientation)), SourceInfo::default()))
}
/// A DNG `ProfileToneCurve` (linear in, linear out, 1.0 = white after exposure compensation) as the
/// finish stage's camera tone curve: 32 knots, log-spaced over 12 stops below white; above white
/// the camera tone's shoulder continues it.
pub(crate) fn dng_tone_curve(curve: &lightcraft_raw::profile::ToneCurve) -> Option<lightcraft_pipeline::tone::CameraTone> {
let knots: [[f32; 2]; 32] = std::array::from_fn(|i| {
let x = 2f32.powf(-12.0 + 12.0 * i as f32 / 31.0);
[x, curve.eval(x).min(0.9995)]
});
lightcraft_pipeline::tone::CameraTone::new(knots)
}
/// Orientation for an embedded preview: its own EXIF orientation when it has one, else the raw file's.
fn preview_orientation(raw_bytes: &[u8], jpeg_orientation: u16) -> Orientation {
if jpeg_orientation > 1 {
@@ -390,6 +413,41 @@ mod tests {
b
}
/// Issue #138: a DNG's own profile look (hue/saturation map, look table, tone curve) is
/// applied when it's developed — Lightroom-converted DNGs rendered flat and muted without it.
#[test]
fn dng_profile_look_is_applied() {
use lightcraft_raw::profile::{HsvTable, ProfileLook, ToneCurve};
let plain = crate::tests_xmp::synthetic_dng_with(None, Default::default());
let mut raw = lightcraft_raw::decode(&plain).unwrap();
let sat = |img: &Rgb32f| {
img.data.iter().map(|p| (p[0].max(p[1]).max(p[2]) - p[0].min(p[1]).min(p[2])) / p[0].max(p[1]).max(p[2]).max(1e-6)).sum::<f32>()
/ img.data.len() as f32
};
let (before, info) = load_bytes(&plain, 64).unwrap();
assert!(sat(&before) > 0.05, "the synthetic scene is coloured");
assert!(info.camera_tone.is_none());
// a map that removes all saturation, and a tone curve
let grey = HsvTable { hue_divisions: 4, sat_divisions: 2, val_divisions: 1, data: vec![[0.0, 0.0, 1.0]; 8], srgb_value: false };
raw.color.profile =
ProfileLook { hue_sat_map: [Some(grey), None], look_table: None, tone_curve: ToneCurve::from_tag(&[0.0, 0.0, 0.18, 0.3, 1.0, 1.0]) };
let with = lightcraft_raw::write_dng(&raw, &Default::default()).unwrap();
let (after, info) = load_bytes(&with, 64).unwrap();
assert!(sat(&after) < 1e-3, "saturation {} → {}", sat(&before), sat(&after));
// a saturation-only map keeps brightness roughly (HSV value is kept in ProPhoto RGB)
let mean = |img: &Rgb32f| img.data.iter().map(|p| p[0].max(p[1]).max(p[2])).sum::<f32>() / img.data.len() as f32;
assert!((0.8..1.1).contains(&(mean(&after) / mean(&before))), "{} vs {}", mean(&after), mean(&before));
let tone = info.camera_tone.expect("the DNG tone curve becomes the camera tone");
assert!((tone.apply(0.18) - 0.3).abs() < 0.01, "{}", tone.apply(0.18));
// a look table alone changes the render too (applied after exposure)
raw.color.profile = ProfileLook {
look_table: Some(HsvTable { hue_divisions: 1, sat_divisions: 2, val_divisions: 1, data: vec![[0.0, 1.0, 0.5]; 2], srgb_value: false }),
..Default::default()
};
let (dim, _) = load_bytes(&lightcraft_raw::write_dng(&raw, &Default::default()).unwrap(), 64).unwrap();
assert!((mean(&dim) / mean(&before) - 0.5).abs() < 0.02, "{} vs {}", mean(&dim), mean(&before));
}
#[test]
fn unsupported_raw_falls_back_to_embedded_preview() {
let b = cr3_with_preview(48, 32);
+52
View File
@@ -0,0 +1,52 @@
//! Fonts from the optional craft-fonts build input (storytold/craft-fonts).
//!
//! Built with `CRAFT_FONTS_DIR=<craft-fonts checkout>`, [`CRAFT_FONTS`] holds every font in its
//! manifest (today: the Japanese UI and document fonts); otherwise it is empty and LightCraft uses
//! only its own bundled fonts (Inter). Both the egui UI and the export watermark renderer read it
//! from here. See craftrules `standards/fonts.md`.
/// A font from the optional craft-fonts build input (empty unless built with `CRAFT_FONTS_DIR`).
pub struct CraftFont {
pub family: &'static str,
pub style: &'static str,
/// ISO 15924 scripts the font is for, e.g. `"Jpan"`.
pub scripts: &'static [&'static str],
pub bytes: &'static [u8],
}
include!(concat!(env!("OUT_DIR"), "/craft_fonts.rs"));
impl CraftFont {
/// Whether the font is meant for `script` (ISO 15924, e.g. `"Jpan"`).
pub fn covers(&self, script: &str) -> bool {
self.scripts.contains(&script)
}
/// Mincho (serif) faces, the preferred Japanese faces for document-like text.
pub fn is_mincho(&self) -> bool {
self.family.contains("Mincho")
}
}
/// The craft-fonts faces for Japanese, in `fonts`' order (empty without craft-fonts).
pub fn japanese(fonts: &'static [CraftFont]) -> impl Iterator<Item = &'static CraftFont> {
fonts.iter().filter(|f| f.covers("Jpan"))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn craft_fonts_are_well_formed_or_absent() {
if CRAFT_FONTS.is_empty() {
eprintln!("built without CRAFT_FONTS_DIR: no craft-fonts to check");
return;
}
for f in CRAFT_FONTS {
assert!(!f.family.is_empty() && !f.style.is_empty() && !f.scripts.is_empty(), "{}", f.family);
assert!(ab_glyph::FontRef::try_from_slice(f.bytes).is_ok(), "{} {} parses", f.family, f.style);
}
assert!(japanese(CRAFT_FONTS).next().is_some(), "craft-fonts carries Japanese faces");
}
}
+376 -186
View File
@@ -375,13 +375,16 @@ fn probe_all(probe: Option<&crate::media::FileProbe>, paths: &[String], progress
/// Copy `src` into `root`/`folders` as `name` (default: its own name), made unique with -1, -2…;
/// returns the new path. The copy is verified ([`crate::import_move::copy_new`]: a new file,
/// synced and compared byte for byte); a bad one is removed and the photo reported as failed.
fn copy_into(root: &Path, src: &str, folders: &[String], name: Option<&str>) -> Result<String, String> {
/// synced and checked against `probe_hash`, the content hash the probe computed — or compared
/// byte for byte with the source when there is none); a bad one is removed and the photo
/// reported as failed.
fn copy_into(root: &Path, src: &str, folders: &[String], name: Option<&str>, probe_hash: Option<&str>) -> Result<String, String> {
let dir = folders.iter().fold(root.to_path_buf(), |d, f| d.join(f));
let name = name
.map(str::to_string)
.unwrap_or_else(|| Path::new(src).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| "photo".into()));
crate::import_move::copy_new(Path::new(src), &dir, &name).map(|p| p.to_string_lossy().to_string())
let expect = probe_hash.and_then(lightcraft_preview::Hash128::parse);
crate::import_move::copy_new(Path::new(src), &dir, &name, expect).map(|p| p.to_string_lossy().to_string())
}
/// What a [`scan`] needs from the session, so it can run on another thread (a folder on a network
@@ -517,205 +520,392 @@ pub fn import(s: &mut Session, paths: &[String], mode: ImportMode) -> crate::Res
}
/// Import files/folders with options (preset, keywords). See the module docs.
///
/// The file-system half ([`ImportJob::prepare`]: expanding folders, probing, reading sidecars,
/// copying / placing files) and the catalog half ([`commit_prepared`]) are separate so the app can
/// run the first on a worker thread; here both run in turn.
pub fn import_with(s: &mut Session, paths: &[String], opts: &ImportOptions) -> crate::Result<ImportReport> {
let mode = opts.mode;
// Only a library on disk has an `Originals/` folder. The browser build keeps the bytes of every
// added file in its own storage already, so "copy" there means "add".
let transfer = matches!(mode, ImportMode::Copy | ImportMode::Move);
let mode = if transfer && s.library.as_ref().is_some_and(|l| !l.on_disk) { ImportMode::Add } else { mode };
if mode == ImportMode::Move && opts.local {
return Err(crate::EngineError::Other("browsing a folder never moves its files".into()));
}
let lib_dir = s.library.as_ref().filter(|l| l.on_disk).map(|l| l.dir.clone());
// where copies go: the chosen folder, else the library's Originals/
let copy_root = opts
.destination
.as_deref()
.filter(|d| !d.trim().is_empty())
.map(std::path::PathBuf::from)
.or_else(|| lib_dir.as_ref().map(|l| l.join("Originals")));
if matches!(mode, ImportMode::Copy | ImportMode::Move) && copy_root.is_none() {
return Err(crate::EngineError::Other("copying or moving needs a destination folder or an open library".into()));
}
let mut seq = opts.rename_start.max(1);
let files = expand(paths, lib_dir.as_deref());
let mut report = ImportReport { scanned: files.len(), ..Default::default() };
let mut job = ImportJob::new(s, opts.clone())?;
let prepared = job.prepare(paths, &std::sync::atomic::AtomicBool::new(false));
// probes a scan left for files not imported now stay for a later import
s.import_probes.extend(std::mem::take(&mut job.cache));
commit_prepared(s, &job.opts, &job.now, prepared)
}
// existing paths and content hashes
let mut by_path: HashMap<&str, PhotoId> = HashMap::new();
let mut by_hash: HashMap<String, PhotoId> = HashMap::new();
for p in s.catalog.photos() {
if let Source::File { path } = &p.source {
by_path.insert(path.as_str(), p.id);
}
if let Some(h) = &p.content_hash {
by_hash.insert(h.clone(), p.id);
}
}
let mut todo = Vec::new();
// a file that was only browsed (Local) joins the library when it is imported for real
let mut promote = Vec::new();
for f in files {
match by_path.get(f.as_str()) {
Some(id) if !opts.local && s.catalog.photo(*id).is_some_and(|p| p.local) => {
promote.push(Op::SetLocal { id: *id, local: false });
report.imported.push(id.0);
/// Everything the file-system half of an import needs, detached from the session so it can run
/// on another thread (a network share or a card can take minutes). Built by [`ImportJob::new`];
/// [`ImportJob::prepare`] may be called for several batches in turn (renamed copies keep counting,
/// duplicates are found across batches); each batch is then [`commit_prepared`] on the session.
pub struct ImportJob {
pub opts: ImportOptions,
/// The mode in effect (Copy / Move become Add in a library without its own folder).
mode: ImportMode,
lib_dir: Option<PathBuf>,
copy_root: Option<PathBuf>,
/// path → (photo, a Local browse record?)
by_path: HashMap<String, (PhotoId, bool)>,
/// content hash → the photo that has it (`None`: a file earlier in this import)
by_hash: HashMap<String, Option<PhotoId>>,
probe: Option<crate::media::FileProbe>,
file_bytes: Option<crate::merge::ByteReader>,
/// Probes from a preceding scan (not read again).
cache: HashMap<String, ProbeInfo>,
naming: crate::sidecar::SidecarNaming,
now: String,
seq: usize,
/// Files found so far (folders expanded) and handled so far, for progress.
pub total: std::sync::Arc<std::sync::atomic::AtomicUsize>,
pub done: std::sync::Arc<std::sync::atomic::AtomicUsize>,
}
/// The outcome of [`ImportJob::prepare`] for one batch, in file order.
#[derive(Debug, Default)]
pub struct Prepared {
pub scanned: usize,
items: Vec<PreparedItem>,
}
#[derive(Debug)]
enum PreparedItem {
/// A Local (browsed) record imported for real.
Promote(PhotoId),
Duplicate {
path: String,
existing: Option<PhotoId>,
reason: &'static str,
hash: Option<String>,
},
Failed(String, String),
Kept(Kept),
Ready(Box<ReadyFile>),
}
#[derive(Debug)]
struct ReadyFile {
path: String,
stored: String,
info: ProbeInfo,
sidecar: Option<crate::sidecar::SidecarData>,
placed: Option<crate::import_move::Placed>,
}
impl Prepared {
/// Undo what the batch did on disk that only a commit would make permanent: files placed by a
/// Move (their sources were never touched). Copies stay, like copies of a failed commit.
pub fn rollback(&self) {
for it in &self.items {
if let PreparedItem::Ready(r) = it
&& let Some(pl) = &r.placed
{
crate::import_move::rollback(pl);
}
Some(id) => report.duplicates.push(Duplicate { path: f, existing: Some(id.0), reason: "path" }),
None => todo.push(f),
}
}
// files a preceding scan already probed are not read again (a network share is slow to read)
let cached: Vec<Option<ProbeInfo>> = todo.iter().map(|f| s.import_probes.remove(f)).collect();
let missing: Vec<String> = todo.iter().zip(&cached).filter(|(_, c)| c.is_none()).map(|(f, _)| f.clone()).collect();
let mut fresh = probe_all(s.media.file_probe.as_ref(), &missing, &ScanProgress::default()).into_iter();
let probed: Vec<Result<ProbeInfo, String>> =
cached.into_iter().map(|c| c.map(Ok).unwrap_or_else(|| fresh.next().unwrap_or_else(|| Err("not probed".into())))).collect();
crate::memory::release();
let now = (s.clock)();
let mut ops = promote;
// Move: placed at the destination, sources untouched until the records are committed
let mut placed: Vec<crate::import_move::Placed> = Vec::new();
for (path, info) in todo.into_iter().zip(probed) {
let info = match info {
Ok(i) => i,
Err(e) => {
log::warn!("import {path}: {e}");
report.failed.push((path, e));
/// Entries in the batch (after expanding folders).
pub fn len(&self) -> usize {
self.items.len()
}
pub fn is_empty(&self) -> bool {
self.items.is_empty()
}
}
impl ImportJob {
/// Snapshot what an import with `opts` needs from the session (no file-system calls). Takes
/// the probes a preceding scan left ([`Session::import_probes`]).
pub fn new(s: &mut Session, opts: ImportOptions) -> crate::Result<Self> {
let mode = opts.mode;
// Only a library on disk has an `Originals/` folder. The browser build keeps the bytes of
// every added file in its own storage already, so "copy" there means "add".
let transfer = matches!(mode, ImportMode::Copy | ImportMode::Move);
let mode = if transfer && s.library.as_ref().is_some_and(|l| !l.on_disk) { ImportMode::Add } else { mode };
if mode == ImportMode::Move && opts.local {
return Err(crate::EngineError::Other("browsing a folder never moves its files".into()));
}
let lib_dir = s.library.as_ref().filter(|l| l.on_disk).map(|l| l.dir.clone());
// where copies go: the chosen folder, else the library's Originals/
let copy_root = opts
.destination
.as_deref()
.filter(|d| !d.trim().is_empty())
.map(std::path::PathBuf::from)
.or_else(|| lib_dir.as_ref().map(|l| l.join("Originals")));
if matches!(mode, ImportMode::Copy | ImportMode::Move) && copy_root.is_none() {
return Err(crate::EngineError::Other("copying or moving needs a destination folder or an open library".into()));
}
let mut by_path = HashMap::new();
let mut by_hash = HashMap::new();
for p in s.catalog.photos() {
if let Source::File { path } = &p.source {
by_path.insert(path.clone(), (p.id, p.local));
}
if let Some(h) = &p.content_hash {
by_hash.insert(h.clone(), Some(p.id));
}
}
Ok(ImportJob {
seq: opts.rename_start.max(1),
opts,
mode,
lib_dir,
copy_root,
by_path,
by_hash,
probe: s.media.file_probe.clone(),
file_bytes: s.media.file_bytes.clone(),
cache: std::mem::take(&mut s.import_probes),
naming: s.xmp.naming,
now: (s.clock)(),
total: Default::default(),
done: Default::default(),
})
}
/// The file-system half of importing `paths`: expand folders, skip known paths, probe, skip
/// duplicates by content, read sidecars, copy or place (Move) the files. Nothing in the
/// catalog changes. Stops before the next file once `cancel` is set.
pub fn prepare(&mut self, paths: &[String], cancel: &std::sync::atomic::AtomicBool) -> Prepared {
let files = self.expand(paths);
self.prepare_files(files, cancel)
}
/// Expand `paths` (folders recursively, the library's own folder skipped) into the files an
/// import would handle; counted in [`ImportJob::total`].
pub fn expand(&self, paths: &[String]) -> Vec<String> {
let files = expand(paths, self.lib_dir.as_deref());
self.total.fetch_add(files.len(), std::sync::atomic::Ordering::Relaxed);
files
}
/// When the import started ([`Session::clock`]), for [`commit_prepared`].
pub fn now(&self) -> &str {
&self.now
}
/// [`ImportJob::prepare`] for files [`ImportJob::expand`] returned.
pub fn prepare_files(&mut self, files: Vec<String>, cancel: &std::sync::atomic::AtomicBool) -> Prepared {
use std::sync::atomic::Ordering::Relaxed;
let opts = self.opts.clone();
let mode = self.mode;
let mut out = Prepared { scanned: files.len(), items: Vec::new() };
let mut todo = Vec::new();
for f in files {
match self.by_path.get(f.as_str()).copied() {
Some((id, true)) if !opts.local => {
// a file that was only browsed (Local) joins the library when it is imported for real
self.by_path.insert(f, (id, false));
out.items.push(PreparedItem::Promote(id));
self.done.fetch_add(1, Relaxed);
}
Some((id, _)) => {
out.items.push(PreparedItem::Duplicate { path: f, existing: Some(id), reason: "path", hash: None });
self.done.fetch_add(1, Relaxed);
}
None => todo.push(f),
}
}
// files a preceding scan already probed are not read again (a network share is slow to read)
let cached: Vec<Option<ProbeInfo>> = todo.iter().map(|f| self.cache.remove(f)).collect();
let missing: Vec<String> = todo.iter().zip(&cached).filter(|(_, c)| c.is_none()).map(|(f, _)| f.clone()).collect();
let progress = ScanProgress::default();
let mut fresh = probe_all(self.probe.as_ref(), &missing, &progress).into_iter();
let probed: Vec<Result<ProbeInfo, String>> =
cached.into_iter().map(|c| c.map(Ok).unwrap_or_else(|| fresh.next().unwrap_or_else(|| Err("not probed".into())))).collect();
crate::memory::release();
for (path, info) in todo.into_iter().zip(probed) {
if cancel.load(Relaxed) {
break;
}
self.done.fetch_add(1, Relaxed);
let mut info = match info {
Ok(i) => i,
Err(e) => {
log::warn!("import {path}: {e}");
out.items.push(PreparedItem::Failed(path, e));
continue;
}
};
if let Some(h) = &info.content_hash
&& !opts.local
&& let Some(existing) = self.by_hash.get(h)
{
out.items.push(PreparedItem::Duplicate { path, existing: *existing, reason: "content", hash: Some(h.clone()) });
continue;
}
};
if let Some(h) = &info.content_hash
&& !opts.local
&& let Some(id) = by_hash.get(h)
{
report.duplicates.push(Duplicate { path, existing: Some(id.0), reason: "content" });
continue;
}
// the XMP sidecar (or a raw's embedded XMP), read before copying: its capture time files
// and names the copy when the file itself has none
let raw = info.kind == MediaKind::Raw;
let packet = crate::sidecar::find_sidecar(&path, s.xmp.naming)
.and_then(|f| std::fs::read_to_string(f).ok())
.or_else(|| info.xmp.clone().filter(|_| raw));
let sidecar = packet.and_then(|x| match crate::sidecar::parse_sidecar(&x, raw).map(|sc| sc.resolve_label(&s.catalog)) {
Ok(sc) => (sc != crate::sidecar::SidecarData::default()).then_some(sc),
Err(e) => {
log::warn!("import {path}: XMP: {e}");
None
// the XMP sidecar (or a raw's embedded XMP), read before copying: its capture time files
// and names the copy when the file itself has none
let raw = info.kind == MediaKind::Raw;
let packet = crate::sidecar::find_sidecar(&path, self.naming)
.and_then(|f| std::fs::read_to_string(f).ok())
.or_else(|| info.xmp.clone().filter(|_| raw));
let sidecar = packet.and_then(|x| match crate::sidecar::parse_sidecar(&x, raw) {
Ok(sc) => Some(sc),
Err(e) => {
log::warn!("import {path}: XMP: {e}");
None
}
});
if info.captured.is_none() {
info.captured = sidecar.as_ref().and_then(|sc| sc.captured.clone());
}
});
let mut info = info;
if info.captured.is_none() {
info.captured = sidecar.as_ref().and_then(|sc| sc.captured.clone());
}
let stored = match (mode, &copy_root) {
(ImportMode::Copy | ImportMode::Move, Some(root))
if !crate::import_move::inside(Path::new(&path), root)
&& !lib_dir.as_ref().is_some_and(|l| crate::import_move::inside(Path::new(&path), l)) =>
let mut placed = None;
let stored = match (mode, &self.copy_root) {
(ImportMode::Copy | ImportMode::Move, Some(root))
if !crate::import_move::inside(Path::new(&path), root)
&& !self.lib_dir.as_ref().is_some_and(|l| crate::import_move::inside(Path::new(&path), l)) =>
{
// the templates see the photo as it will be catalogued (undated: the import time)
let own = Path::new(&path).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_default();
let mut q = Photo::new(PhotoId(0), Source::File { path: path.clone() }, &own, &info.format, 0, 0, &self.now);
q.captured = info.captured.clone();
q.meta = info.meta.clone();
let name = opts.rename.as_deref().filter(|t| !t.trim().is_empty()).map(|t| {
let n = crate::rename::expand(t, &q, self.seq);
self.seq += 1;
n
});
let folders = opts.organize.folders(&q);
if mode == ImportMode::Move && !crate::import_move::is_symlink(Path::new(&path)) {
let dir = folders.iter().fold(root.to_path_buf(), |d, f| d.join(f));
let name = name.unwrap_or(own);
match crate::import_move::place(Path::new(&path), &dir, &name) {
Ok(pl) => {
let dst = pl.dst.to_string_lossy().to_string();
placed = Some(pl);
dst
}
Err(e) => {
out.items.push(PreparedItem::Failed(path, e));
continue;
}
}
} else {
if mode == ImportMode::Move {
let reason = "a symbolic link: the file it points to was copied, the link and its target stay";
out.items.push(PreparedItem::Kept(Kept { path: path.clone(), reason: reason.into() }));
}
match copy_into(root, &path, &folders, name.as_deref(), info.content_hash.as_deref()) {
Ok(p) => p,
Err(e) => {
out.items.push(PreparedItem::Failed(path, e));
continue;
}
}
}
}
(ImportMode::Move, _) => {
let reason = "already inside the destination or the library: added where it is";
out.items.push(PreparedItem::Kept(Kept { path: path.clone(), reason: reason.into() }));
path.clone()
}
_ => path.clone(),
};
if let Some(h) = &info.content_hash {
self.by_hash.insert(h.clone(), None);
}
// Copy as DNG: the copied raw becomes a DNG (the copy is ours to replace)
let mut stored = stored;
if opts.convert_dng
&& mode == ImportMode::Copy
&& stored != path
&& info.kind == MediaKind::Raw
&& !info.format.eq_ignore_ascii_case("DNG")
{
// the templates see the photo as it will be catalogued (undated: the import time)
let own = Path::new(&path).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_default();
let mut q = Photo::new(PhotoId(0), Source::File { path: path.clone() }, &own, &info.format, 0, 0, &now);
q.captured = info.captured.clone();
q.meta = info.meta.clone();
let name = opts.rename.as_deref().filter(|t| !t.trim().is_empty()).map(|t| {
let n = crate::rename::expand(t, &q, seq);
seq += 1;
n
match crate::cmd::convert::write_dng_with(self.file_bytes.as_ref(), &stored, String::new()) {
Ok(dng) => {
let _ = std::fs::remove_file(&stored);
stored = dng;
info.format = "DNG".into();
}
Err(e) => log::warn!("import {path}: copy as DNG: {e}"),
}
}
out.items.push(PreparedItem::Ready(Box::new(ReadyFile { path, stored, info, sidecar, placed })));
}
out
}
}
/// The catalog half of an import: add the photos [`ImportJob::prepare`] readied (one undoable op;
/// a Move's sources are removed once it is saved). On a failed commit a Move's placed files are
/// taken back.
pub fn commit_prepared(s: &mut Session, opts: &ImportOptions, now: &str, prepared: Prepared) -> crate::Result<ImportReport> {
let now = now.to_string();
let mut report = ImportReport { scanned: prepared.scanned, ..Default::default() };
let mut ops = Vec::new();
// Move: placed at the destination, sources untouched until the records are committed
let mut placed: Vec<crate::import_move::Placed> = Vec::new();
// photos added by this batch, by content (a duplicate of a file earlier in the import names it)
let mut new_hash: HashMap<String, PhotoId> = HashMap::new();
for it in prepared.items {
match it {
PreparedItem::Promote(id) => {
if s.catalog.photo(id).is_some_and(|p| p.local) {
ops.push(Op::SetLocal { id, local: false });
}
report.imported.push(id.0);
}
PreparedItem::Duplicate { path, existing, reason, hash } => {
let existing = existing.or_else(|| {
let h = hash?;
new_hash.get(&h).copied().or_else(|| s.catalog.photos().find(|p| p.content_hash.as_deref() == Some(h.as_str())).map(|p| p.id))
});
let folders = opts.organize.folders(&q);
if mode == ImportMode::Move && !crate::import_move::is_symlink(Path::new(&path)) {
let dir = folders.iter().fold(root.to_path_buf(), |d, f| d.join(f));
let name = name.unwrap_or(own);
match crate::import_move::place(Path::new(&path), &dir, &name) {
Ok(pl) => {
let dst = pl.dst.to_string_lossy().to_string();
placed.push(pl);
dst
}
Err(e) => {
report.failed.push((path, e));
continue;
}
}
} else {
if mode == ImportMode::Move {
let reason = "a symbolic link: the file it points to was copied, the link and its target stay";
report.kept.push(Kept { path: path.clone(), reason: reason.into() });
}
match copy_into(root, &path, &folders, name.as_deref()) {
Ok(p) => p,
Err(e) => {
report.failed.push((path, e));
continue;
}
report.duplicates.push(Duplicate { path, existing: existing.map(|i| i.0), reason });
}
PreparedItem::Failed(path, e) => report.failed.push((path, e)),
PreparedItem::Kept(k) => report.kept.push(k),
PreparedItem::Ready(r) => {
let ReadyFile { path, stored, info, sidecar, placed: pl } = *r;
placed.extend(pl);
let id = s.catalog.alloc_photo_id();
if let Some(h) = &info.content_hash {
new_hash.insert(h.clone(), id);
}
let sidecar = sidecar.map(|sc| sc.resolve_label(&s.catalog)).filter(|sc| *sc != crate::sidecar::SidecarData::default());
// a copy is catalogued under its new name
let name = Path::new(&stored).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| path.clone());
let mut p = Photo::new(id, Source::File { path: stored }, &name, &info.format, info.width, info.height, &now);
p.kind = info.kind;
p.file_size = info.file_size;
p.captured = info.captured;
p.meta = info.meta;
p.as_shot_wb = info.as_shot_wb;
p.content_hash = info.content_hash;
p.embedded_lens = info.embedded_lens;
p.preview_only = info.preview_only.clone();
apply_import_defaults(s, &mut p);
if let Some(sc) = &sidecar {
crate::sidecar::merge_into(&mut p, sc, &now);
report.sidecars += 1;
}
if let Some(mp) = opts.metadata_preset.as_ref().and_then(|n| s.metadata_presets.iter().find(|m| m.name.eq_ignore_ascii_case(n))) {
crate::cmd::metadata::apply_to(&mut p.meta, &mp.fields);
}
for k in &opts.keywords {
let k = lightcraft_catalog::keywords::clean(k);
if !k.is_empty() && !p.meta.keywords.iter().any(|x| x.eq_ignore_ascii_case(&k)) {
p.meta.keywords.push(k);
}
}
}
(ImportMode::Move, _) => {
let reason = "already inside the destination or the library: added where it is";
report.kept.push(Kept { path: path.clone(), reason: reason.into() });
path.clone()
}
_ => path.clone(),
};
let id = s.catalog.alloc_photo_id();
if let Some(h) = &info.content_hash {
by_hash.insert(h.clone(), id);
}
// Copy as DNG: the copied raw becomes a DNG (the copy is ours to replace)
let mut stored = stored;
if opts.convert_dng && mode == ImportMode::Copy && stored != path && info.kind == MediaKind::Raw && !info.format.eq_ignore_ascii_case("DNG") {
match crate::cmd::convert::write_dng_for(s, &stored, String::new()) {
Ok(dng) => {
let _ = std::fs::remove_file(&stored);
stored = dng;
info.format = "DNG".into();
if let Some(preset) = &opts.preset {
let d = std::sync::Arc::new(preset.apply(&p.develop, 1.0));
let label = format!("Preset: {}", preset.name);
p.develop = d.clone();
p.edited = Some(now.clone());
p.history.push(lightcraft_catalog::HistoryStep { label, settings: d });
}
Err(e) => log::warn!("import {path}: copy as DNG: {e}"),
report.imported.push(id.0);
p.local = opts.local;
if opts.local {
// the state as browsed: a later change keeps the record from being forgotten
p.set_local_baseline();
}
ops.push(Op::AddPhoto { photo: Box::new(p) });
}
}
// a copy is catalogued under its new name
let name = Path::new(&stored).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| path.clone());
let mut p = Photo::new(id, Source::File { path: stored }, &name, &info.format, info.width, info.height, &now);
p.kind = info.kind;
p.file_size = info.file_size;
p.captured = info.captured;
p.meta = info.meta;
p.as_shot_wb = info.as_shot_wb;
p.content_hash = info.content_hash;
p.embedded_lens = info.embedded_lens;
p.preview_only = info.preview_only.clone();
apply_import_defaults(s, &mut p);
if let Some(sc) = &sidecar {
crate::sidecar::merge_into(&mut p, sc, &now);
report.sidecars += 1;
}
if let Some(mp) = opts.metadata_preset.as_ref().and_then(|n| s.metadata_presets.iter().find(|m| m.name.eq_ignore_ascii_case(n))) {
crate::cmd::metadata::apply_to(&mut p.meta, &mp.fields);
}
for k in &opts.keywords {
let k = lightcraft_catalog::keywords::clean(k);
if !k.is_empty() && !p.meta.keywords.iter().any(|x| x.eq_ignore_ascii_case(&k)) {
p.meta.keywords.push(k);
}
}
if let Some(preset) = &opts.preset {
let d = std::sync::Arc::new(preset.apply(&p.develop, 1.0));
let label = format!("Preset: {}", preset.name);
p.develop = d.clone();
p.edited = Some(now.clone());
p.history.push(lightcraft_catalog::HistoryStep { label, settings: d });
}
report.imported.push(id.0);
p.local = opts.local;
if opts.local {
// the state as browsed: a later change keeps the record from being forgotten
p.set_local_baseline();
}
ops.push(Op::AddPhoto { photo: Box::new(p) });
}
let log0 = s.pending_log.len();
if !ops.is_empty()
+37 -11
View File
@@ -171,9 +171,15 @@ pub(crate) fn place(src: &Path, dir: &Path, name: &str) -> Result<Placed, String
/// Import → **Copy**: copy `src` into `dir` as `name` (-1, -2… when taken) with the same checks as
/// a move's copy — a new file (never overwriting one, even one that appears meanwhile), synced to
/// disk and compared byte for byte with the source; a bad copy is removed and reported, so a card
/// is never wiped on the strength of it. Returns the new path.
pub(crate) fn copy_new(src: &Path, dir: &Path, name: &str) -> Result<PathBuf, String> {
/// disk and verified; a bad copy is removed and reported, so a card is never wiped on the strength
/// of it. Returns the new path.
///
/// `expect` is the content hash the import's probe computed from its full read of the source
/// ([`lightcraft_preview::hash_bytes`]): the copy is read back and checked against it, which spares
/// a third read of the card (issue #134). It also catches a source that changed since the probe
/// (whose recorded metadata and hash would be stale). Without one the copy is compared byte for
/// byte with the source.
pub(crate) fn copy_new(src: &Path, dir: &Path, name: &str, expect: Option<lightcraft_preview::Hash128>) -> Result<PathBuf, String> {
fs::create_dir_all(dir).map_err(|e| format!("{}: {e}", dir.display()))?;
let (stem, ext) = match name.rsplit_once('.') {
Some((s, e)) if !s.is_empty() => (s.to_string(), format!(".{e}")),
@@ -181,7 +187,7 @@ pub(crate) fn copy_new(src: &Path, dir: &Path, name: &str) -> Result<PathBuf, St
};
for i in 0..100_000u32 {
let d = if i == 0 { dir.join(name) } else { dir.join(format!("{stem}-{i}{ext}")) };
match copy_verified(src, &d) {
match copy_verified(src, &d, expect) {
Ok(()) => return Ok(d),
Err(e) if e.kind() == io::ErrorKind::AlreadyExists => continue,
Err(e) => return Err(format!("copy {}: {e} (nothing was imported from it; the original is untouched)", src.display())),
@@ -262,12 +268,14 @@ fn link_or_copy(src: &Path, dst: &Path) -> io::Result<()> {
Err(_) => {}
}
}
copy_verified(src, dst)
// Move keeps the byte-for-byte comparison: the source is deleted on the strength of it
copy_verified(src, dst, None)
}
/// Copy into a new file, sync it to disk and compare it with the original; on any failure the
/// partial copy is removed.
fn copy_verified(src: &Path, dst: &Path) -> io::Result<()> {
/// Copy into a new file, sync it to disk and verify it: against `expect` (the probe's hash of the
/// source) when given, else byte for byte against the source. On any failure the partial copy is
/// removed.
pub(crate) fn copy_verified(src: &Path, dst: &Path, expect: Option<lightcraft_preview::Hash128>) -> io::Result<()> {
let mut out = OpenOptions::new().write(true).create_new(true).open(dst)?;
let r = (|| {
let mut inp = File::open(src)?;
@@ -283,10 +291,14 @@ fn copy_verified(src: &Path, dst: &Path) -> io::Result<()> {
if let Ok(m) = fs::metadata(src).and_then(|m| m.modified()) {
let _ = out.set_modified(m);
}
if !same_bytes(src, dst)? {
return Err(io::Error::other("the copy differs from the original"));
match expect {
Some(h) if hash_file(dst)? != h => {
Err(io::Error::other("the copy differs from the original as it was read for the import (a bad copy, or the file changed meanwhile)"))
}
Some(_) => Ok(()),
None if !same_bytes(src, dst)? => Err(io::Error::other("the copy differs from the original")),
None => Ok(()),
}
Ok(())
})();
drop(out);
if r.is_err() {
@@ -295,6 +307,20 @@ fn copy_verified(src: &Path, dst: &Path) -> io::Result<()> {
r
}
/// [`lightcraft_preview::hash_bytes`] of a file's content, read in chunks.
fn hash_file(path: &Path) -> io::Result<lightcraft_preview::Hash128> {
let mut f = File::open(path)?;
let mut h = lightcraft_preview::Hasher128::new();
let mut buf = vec![0u8; 1 << 20];
loop {
let n = fill(&mut f, &mut buf)?;
if n == 0 {
return Ok(h.finish());
}
h.update(buf.get(..n).unwrap_or(&[]));
}
}
/// Do two files hold the same bytes?
pub(crate) fn same_bytes(a: &Path, b: &Path) -> io::Result<bool> {
if fs::metadata(a)?.len() != fs::metadata(b)?.len() {
+91 -16
View File
@@ -10,6 +10,8 @@
//! run them off the UI thread.
#![forbid(unsafe_code)]
pub mod availability;
mod camera_preview;
pub mod cmd;
pub mod crs;
pub mod crs_masks;
@@ -17,6 +19,7 @@ pub mod demo;
pub mod devices;
pub mod export;
pub mod files;
pub mod fonts;
pub mod guard;
pub mod import;
mod import_move;
@@ -36,6 +39,7 @@ mod view;
use std::sync::Arc;
pub use cmd::{CommandInfo, CommandSpec, command_specs, find_command};
pub use fonts::{CRAFT_FONTS, CraftFont};
use lightcraft_catalog::{Catalog, Filter, Op, PhotoId, Sort};
use lightcraft_develop::DevelopSettings;
pub use media::{RenderJob, SourceLevel};
@@ -57,6 +61,9 @@ pub enum EngineError {
/// be written. They stay queued and are written by the next successful save.
#[error("saved in memory but not written to disk: {0}; LightCraft will retry")]
NotSaved(String),
/// Another process (the app, `lightcraft-cli`, another computer) has the library open.
#[error("{0}")]
LibraryInUse(String),
#[error("{0}")]
Other(String),
}
@@ -68,6 +75,22 @@ pub type Result<T> = std::result::Result<T, EngineError>;
pub struct UndoEntry {
pub label: String,
pub op: Op,
/// A folder to rename on disk (`from` → `to`) before `op` is applied: Rename / Move Folder
/// (whose `op` relinks the photos inside). Never overwrites; refused when `to` exists.
pub folder: Option<FolderMove>,
}
/// A folder renamed or moved on disk as part of an undo step.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct FolderMove {
pub from: String,
pub to: String,
}
impl FolderMove {
fn reversed(&self) -> Self {
Self { from: self.to.clone(), to: self.from.clone() }
}
}
/// An in-progress slider drag / brush stroke: one undo step when it ends.
@@ -253,11 +276,22 @@ impl Session {
(spec.enabled)(self).map_err(|why| EngineError::Disabled(id.to_string(), why))?;
let empty = Value::Object(Default::default());
let params = if params.is_null() { &empty } else { params };
self.run_command(id, spec.journal.then_some(params), |s| (spec.run)(s, params))
}
/// Run `f` as command `id` with what [`Session::execute`] does around every command: the
/// panic guard, auto versions, XMP sidecar auto-write and the durable save (a failed save is
/// `NotSaved`). Not journaled. The app's import task commits its batches this way.
pub fn execute_fn(&mut self, id: &str, f: impl FnOnce(&mut Session) -> Result<Value>) -> Result<Value> {
self.run_command(id, None, f)
}
fn run_command(&mut self, id: &str, journal: Option<&Value>, f: impl FnOnce(&mut Session) -> Result<Value>) -> Result<Value> {
let log_start = self.pending_log.len();
let was_active = self.active();
self.depth += 1;
// last-resort guard: a panic in a command is that command's error, not a crash
let r = guard::catch(&format!("`{id}`"), || (spec.run)(self, params)).unwrap_or_else(|e| Err(EngineError::Other(e)));
let r = guard::catch(&format!("`{id}`"), || f(self)).unwrap_or_else(|e| Err(EngineError::Other(e)));
self.depth -= 1;
if self.depth == 0 && was_active.is_some() && self.active() != was_active {
self.previous_active = was_active;
@@ -265,7 +299,10 @@ impl Session {
self.auto_version(left);
}
}
if r.is_ok() && spec.journal && self.depth == 0 {
if r.is_ok()
&& let Some(params) = journal
&& self.depth == 0
{
self.journal.push((id.to_string(), params.clone()));
if self.journal.len() > 10_000 {
self.journal.drain(..1000);
@@ -302,7 +339,7 @@ impl Session {
let fwd = op.clone();
let inv = self.catalog.apply(op)?;
self.pending_log.push(fwd);
self.undo.push(UndoEntry { label: label.to_string(), op: inv });
self.undo.push(UndoEntry { label: label.to_string(), op: inv, folder: None });
if self.undo.len() > 1000 {
self.undo.remove(0);
}
@@ -310,6 +347,16 @@ impl Session {
Ok(())
}
/// [`Session::commit`] for an op that goes with a folder already renamed on disk; the undo
/// step renames it back (`folder` is the undo direction: current place → old place).
pub(crate) fn commit_with_folder(&mut self, label: &str, op: Op, folder: FolderMove) -> Result<()> {
self.commit(label, op)?;
if let Some(e) = self.undo.last_mut() {
e.folder = Some(folder);
}
Ok(())
}
/// Leaving photo `id` after editing it: keep its settings as an automatic version (when they
/// differ from its latest version; at most [`AUTO_VERSIONS`] auto versions, oldest dropped).
/// Saved with the library but not an undo step.
@@ -340,9 +387,13 @@ impl Session {
if n < 2 || n > self.undo.len() {
return;
}
// a step that moves a folder on disk stays on its own
if self.undo[self.undo.len() - n..].iter().any(|e| e.folder.is_some()) {
return;
}
let tail = self.undo.split_off(self.undo.len() - n);
let ops = tail.into_iter().rev().map(|e| e.op).collect();
self.undo.push(UndoEntry { label: label.to_string(), op: Op::Batch { ops } });
self.undo.push(UndoEntry { label: label.to_string(), op: Op::Batch { ops }, folder: None });
}
/// Apply without recording undo (interactive previews).
@@ -353,7 +404,7 @@ impl Session {
pub fn undo_step(&mut self) -> Result<String> {
let e = self.undo.pop().ok_or_else(|| EngineError::Other("nothing to undo".into()))?;
let redo = match self.apply_with_files(&e.op) {
let redo = match self.apply_with_files(&e.op, e.folder.as_ref()) {
Ok(r) => r,
Err(err) => {
self.undo.push(e);
@@ -361,13 +412,13 @@ impl Session {
}
};
self.pending_log.push(e.op);
self.redo.push(UndoEntry { label: e.label.clone(), op: redo });
self.redo.push(UndoEntry { label: e.label.clone(), op: redo, folder: e.folder.as_ref().map(FolderMove::reversed) });
Ok(e.label)
}
pub fn redo_step(&mut self) -> Result<String> {
let e = self.redo.pop().ok_or_else(|| EngineError::Other("nothing to redo".into()))?;
let undo = match self.apply_with_files(&e.op) {
let undo = match self.apply_with_files(&e.op, e.folder.as_ref()) {
Ok(r) => r,
Err(err) => {
self.redo.push(e);
@@ -375,26 +426,44 @@ impl Session {
}
};
self.pending_log.push(e.op);
self.undo.push(UndoEntry { label: e.label.clone(), op: undo });
self.undo.push(UndoEntry { label: e.label.clone(), op: undo, folder: e.folder.as_ref().map(FolderMove::reversed) });
Ok(e.label)
}
/// Apply an undo/redo op, first moving the files its renames imply (all or nothing). Files
/// that can't be moved back after a failure are reported and the library follows them (a
/// logged change outside the undo history), so none goes missing.
fn apply_with_files(&mut self, op: &Op) -> Result<Op> {
/// Apply an undo/redo op, first renaming the step's folder and moving the files its renames
/// imply (all or nothing). Files that can't be moved back after a failure are reported and the
/// library follows them (a logged change outside the undo history), so none goes missing; the
/// folder is only moved back when no file was left behind at its new path.
fn apply_with_files(&mut self, op: &Op, folder: Option<&FolderMove>) -> Result<Op> {
let fs = rename::RealFs;
if let Some(f) = folder {
cmd::browse::rename_folder_on_disk(&f.from, &f.to).map_err(|e| EngineError::Other(format!("can't move the folder back: {e}")))?;
}
let undo_folder = || {
if let Some(f) = folder {
let _ = cmd::browse::rename_folder_on_disk(&f.to, &f.from);
}
};
let moves = self.file_moves(op);
if let Err(e) = rename::move_all(&fs, &moves) {
if e.stuck.is_empty() {
undo_folder();
}
return Err(EngineError::Other(format!("can't move the files back: {}", self.follow_stuck(op, e, false))));
}
match self.catalog.apply(op.clone()) {
Ok(inv) => Ok(inv),
Ok(inv) => {
if let Some(f) = folder {
cmd::browse::follow_folder(self, &f.from, &f.to);
}
Ok(inv)
}
Err(e) => {
let back: Vec<(String, String)> = moves.iter().rev().map(|(a, b)| (b.clone(), a.clone())).collect();
if let Err(be) = rename::move_all(&fs, &back) {
return Err(EngineError::Other(format!("{e}; the files could not all be moved back: {}", be.message)));
}
undo_folder();
Err(e.into())
}
}
@@ -509,7 +578,11 @@ impl Session {
/// Photos shown in the grid/filmstrip for the current source, filter and sort.
pub fn visible(&mut self) -> &[PhotoId] {
let key = (self.catalog.revision, format!("{:?}|{:?}|{:?}|{:?}", self.source, self.filter, self.sort, self.browse));
let mut key = (self.catalog.revision, format!("{:?}|{:?}|{:?}|{:?}", self.source, self.filter, self.sort, self.browse));
if self.source == LibrarySource::Missing && self.media.availability.is_background() {
// the view fills in as the background checks find files gone
key.1.push_str(&format!("|{}", self.media.availability.generation()));
}
if self.visible_key.as_ref() != Some(&key) {
// "in the last N days" rules count back from the session's clock
lightcraft_catalog::rules::set_now(Some((self.clock)()));
@@ -544,8 +617,8 @@ impl Session {
}
if self.source == LibrarySource::Missing {
// only the photos the query kept (library photos, not Local browse records) are checked
let cat = &self.catalog;
visible.retain(|id| cmd::missing::is_missing(cat, *id));
let (cat, avail) = (&self.catalog, &self.media.availability);
visible.retain(|id| cmd::missing::is_missing(cat, avail, *id));
}
if self.source != LibrarySource::RecentlyDeleted {
visible = self.catalog.arrange_stacks(&visible);
@@ -642,6 +715,8 @@ mod tests_persist;
#[cfg(test)]
mod tests_prefs;
#[cfg(test)]
mod tests_settings_files;
#[cfg(test)]
mod tests_spots;
#[cfg(test)]
mod tests_xmp;
+127 -14
View File
@@ -23,7 +23,7 @@
use std::path::{Path, PathBuf};
use lightcraft_catalog::{FsStore, Journal, LoadReport, Store};
use lightcraft_catalog::{FsStore, Journal, LibraryLock, LoadReport, Store};
use lightcraft_develop::Preset;
use serde::{Deserialize, Serialize};
@@ -64,6 +64,15 @@ pub struct Library {
pub forgot_local: Option<lightcraft_catalog::ForgetPlan>,
presets_written: String,
view_written: Vec<u8>,
/// Settings files that were unreadable or damaged when the library opened (`library.info` →
/// `settingsWarnings`; shown by the UI once, see [`Session::take_library_warnings`]).
pub settings_warnings: Vec<String>,
warnings_reported: usize,
/// Settings files that couldn't be read or set aside: never overwritten this session.
blocked: Vec<&'static str>,
/// Keeps other processes out of this library while it's open (last: released after the
/// journal's background snapshot has landed).
lock: Option<LibraryLock>,
}
/// Where a library's files live (see [`Session::open_library_in`]).
@@ -171,23 +180,112 @@ fn poll_compaction(lib: &mut Library) {
/// How long the frame loop waits before retrying a failed append (commands retry at once).
const RETRY_BACKOFF: std::time::Duration = std::time::Duration::from_secs(2);
fn read_json<T: serde::de::DeserializeOwned>(store: &mut dyn Store, name: &str) -> Option<T> {
store.read(name).ok().flatten().and_then(|b| serde_json::from_slice(&b).ok())
/// `a` and `b` name the same directory.
fn same_dir(a: &Path, b: &Path) -> bool {
match (std::fs::canonicalize(a), std::fs::canonicalize(b)) {
(Ok(a), Ok(b)) => a == b,
_ => a == b,
}
}
/// This program, for the lock owner note ("LightCraft", "lightcraft-cli").
fn program_name() -> String {
let exe = std::env::current_exe().ok().and_then(|p| p.file_stem().map(|s| s.to_string_lossy().to_string()));
match exe.as_deref() {
Some("lightcraft") | None => "LightCraft".into(),
Some(other) => other.to_string(),
}
}
/// Problems reading the settings files (`presets.json`, `prefs.json`, `view.json`) when the library
/// opened (issue #103). A missing file is not a problem (defaults). A file that doesn't parse is
/// kept as `<name>.corrupt-<unix time>` before the defaults are used (so the next save can't
/// lose it); a file that can't be read at all (locked by another program, I/O error) is never
/// written this session, so its content survives until the library is opened again.
#[derive(Default)]
struct SettingsLoad {
warnings: Vec<String>,
/// Files not to overwrite this session.
blocked: Vec<&'static str>,
}
impl SettingsLoad {
fn read<T: serde::de::DeserializeOwned>(&mut self, store: &mut dyn Store, name: &'static str) -> Option<T> {
let bytes = match store.read(name) {
Ok(b) => b?,
Err(e) => {
log::error!("library: {name}: {e}");
self.blocked.push(name);
self.warnings.push(format!(
"{name} couldn't be read ({e}). LightCraft uses the defaults for now and won't overwrite the file; reopen the library to try again."
));
return None;
}
};
let err = match serde_json::from_slice(&bytes) {
Ok(v) => return Some(v),
Err(e) => e,
};
let secs = web_time::SystemTime::now().duration_since(web_time::UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0);
let keep = format!("{name}.corrupt-{secs}");
log::error!("library: {name} is damaged: {err}");
match store.write_atomic(&keep, &bytes) {
Ok(()) => self.warnings.push(format!("{name} is damaged ({err}). It was kept as {keep}, and the defaults are used.")),
Err(w) => {
self.blocked.push(name);
self.warnings.push(format!(
"{name} is damaged ({err}) and couldn't be set aside ({w}). LightCraft uses the defaults and won't overwrite the file."
));
}
}
None
}
}
impl Session {
/// Open (or create) the library at `dir` into this session, replacing its catalog. With
/// `seed_demo`, a newly created library starts with the procedural demo photos.
///
/// The library is locked for this session ([`lightcraft_catalog::lock`]): if another process
/// has it open, this fails with [`EngineError::LibraryInUse`] and nothing is read or changed.
/// Reopening the library this session already has open keeps its lock.
pub fn open_library(&mut self, dir: impl AsRef<Path>, seed_demo: bool) -> Result<&LoadReport> {
let dir = dir.as_ref().to_path_buf();
let open = || FsStore::open(&dir).map_err(|e| EngineError::Other(format!("can't open library {}: {e}", dir.display())));
let stores = LibraryStores { catalog: Box::new(open()?), files: Box::new(open()?), on_disk: true, dir: dir.clone() };
self.open_library_in(stores, seed_demo)
// the library open in this session (same directory): hand its lock over
let reused = self.library.as_mut().filter(|l| same_dir(&l.dir, &dir)).and_then(|l| l.lock.take());
let reusing = reused.is_some();
let mut lock = match reused {
Some(l) => Some(l),
None => Some(LibraryLock::acquire(&dir, &program_name()).map_err(|e| EngineError::LibraryInUse(e.to_string()))?),
};
if let Err(e) = self.open_stores(stores, seed_demo, &mut lock) {
if reusing && let Some(lib) = self.library.as_mut() {
lib.lock = lock.take();
}
return Err(e);
}
self.loaded_report()
}
/// Open (or create) a library whose files live in `stores` (e.g. browser storage), replacing
/// this session's catalog.
/// this session's catalog. Not locked: the host guards against a second opener (see
/// [`Session::open_library`] for directories).
pub fn open_library_in(&mut self, stores: LibraryStores, seed_demo: bool) -> Result<&LoadReport> {
self.open_stores(stores, seed_demo, &mut None)?;
self.loaded_report()
}
fn loaded_report(&self) -> Result<&LoadReport> {
match &self.library {
Some(lib) => Ok(&lib.report),
None => Err(EngineError::Other("library closed while opening".into())),
}
}
/// Open `stores`; the new library takes `lock` once nothing can fail any more.
fn open_stores(&mut self, stores: LibraryStores, seed_demo: bool, lock: &mut Option<LibraryLock>) -> Result<()> {
let LibraryStores { dir, catalog, mut files, on_disk } = stores;
self.media.smart_dir = on_disk.then(|| crate::smart::dir(&dir));
// the current library may be these same files: let its background snapshot land first
@@ -208,8 +306,9 @@ impl Session {
crate::demo::load(self);
journal.snapshot(&self.catalog)?;
}
let mut settings = SettingsLoad::default();
// presets
if let Some(f) = read_json::<PresetsFile>(files.as_mut(), "presets.json") {
if let Some(f) = settings.read::<PresetsFile>(files.as_mut(), "presets.json") {
for p in &mut self.presets {
p.favorite = p.builtin && f.favorites.contains(&p.id) || (!p.builtin && p.favorite);
}
@@ -223,7 +322,7 @@ impl Session {
self.profile_recent = f.profile_recent.into_iter().filter(known).take(crate::presets::RECENT_PROFILES).collect();
}
// preferences
let prefs = read_json::<PrefsFile>(files.as_mut(), "prefs.json").unwrap_or_default();
let prefs = settings.read::<PrefsFile>(files.as_mut(), "prefs.json").unwrap_or_default();
self.xmp = prefs.xmp;
self.last_export = prefs.last_export;
self.export_presets = prefs.export_presets;
@@ -244,7 +343,7 @@ impl Session {
self.media.smart_dir = Some(d.clone());
}
// view state
if let Some(v) = read_json::<ViewFile>(files.as_mut(), "view.json") {
if let Some(v) = settings.read::<ViewFile>(files.as_mut(), "view.json") {
self.source = v.source;
self.browse = v.browse;
self.filter = v.filter;
@@ -277,6 +376,10 @@ impl Session {
forgot_local: None,
presets_written,
view_written,
settings_warnings: settings.warnings,
warnings_reported: 0,
blocked: settings.blocked,
lock: lock.take(),
});
// forget untouched Local records of folders not browsed for a while (journaled at once)
if self.forget_local_days > 0 {
@@ -290,10 +393,7 @@ impl Session {
lib.forgot_local = Some(plan);
}
}
match &self.library {
Some(lib) => Ok(&lib.report),
None => Err(EngineError::Other("library closed while opening".into())),
}
Ok(())
}
/// Write pending ops to the log (fsynced), compact when due, and save changed presets.
@@ -332,7 +432,7 @@ impl Session {
}
let presets = presets_json(self);
let Some(lib) = self.library.as_mut() else { return Ok(()) };
if presets != lib.presets_written {
if presets != lib.presets_written && !lib.blocked.contains(&"presets.json") {
if let Err(e) = lib.files.write_atomic("presets.json", presets.as_bytes()) {
log::error!("library: presets: {e}");
} else {
@@ -413,7 +513,7 @@ impl Session {
}
let v = self.view_json();
let Some(lib) = self.library.as_mut() else { return };
if v != lib.view_written {
if v != lib.view_written && !lib.blocked.contains(&"view.json") {
match lib.files.write_atomic("view.json", &v) {
Ok(()) => lib.view_written = v,
Err(e) => log::error!("library: view: {e}"),
@@ -442,9 +542,22 @@ impl Session {
})
.unwrap_or_default();
let Some(lib) = self.library.as_mut() else { return Ok(()) };
if lib.blocked.contains(&"prefs.json") {
return Err(EngineError::Other(
"prefs: prefs.json couldn't be read when the library opened, so it isn't overwritten; reopen the library to save preferences".into(),
));
}
lib.files.write_atomic("prefs.json", &v).map_err(|e| EngineError::Other(format!("prefs: {e}")))
}
/// Settings-file warnings of the open library not handed out yet (the UI shows each once).
pub fn take_library_warnings(&mut self) -> Vec<String> {
let Some(lib) = self.library.as_mut() else { return vec![] };
let new = lib.settings_warnings[lib.warnings_reported..].to_vec();
lib.warnings_reported = lib.settings_warnings.len();
new
}
/// The thumbnail disk cache budget in bytes ([`Session::cache_mb`], else the default).
pub fn cache_bytes(&self) -> u64 {
if self.cache_mb == 0 { crate::media::DISK_CACHE_BYTES } else { u64::from(self.cache_mb) << 20 }
+167 -42
View File
@@ -26,7 +26,7 @@ use lightcraft_raster::{Histogram, Rgb32f, Rgba8};
use serde::{Deserialize, Serialize};
/// Bump when the pipeline's output changes, to invalidate cached thumbnails.
pub const RENDER_CACHE_VERSION: u64 = 4;
pub const RENDER_CACHE_VERSION: u64 = 8;
/// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache).
pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512];
@@ -71,9 +71,31 @@ pub type FileLoader = Arc<dyn Fn(&str, usize) -> Result<(Rgb32f, SourceInfo), St
/// `max_edge` (set by the app). `None`: no usable preview.
pub type PreviewLoader = Arc<dyn Fn(&str, usize) -> Option<Rgba8> + Send + Sync>;
/// Pixels and the source interpretation learned while decoding; they must be cached together.
#[derive(Clone)]
pub struct DecodedSource {
pub image: Arc<Rgb32f>,
pub info: Option<SourceInfo>,
/// A smart preview's stored camera tone curve: the one decoder fact its pixels need that the
/// catalog's header facts lack (used when `info` is `None`).
pub camera_tone: Option<lightcraft_pipeline::tone::CameraTone>,
}
impl DecodedSource {
pub fn new(image: Arc<Rgb32f>, info: Option<SourceInfo>) -> Self {
DecodedSource { image, info, camera_tone: None }
}
/// What to render these pixels against: the decoder's facts, else `header` (the catalog's)
/// with any stored camera tone curve.
pub fn info_or(&self, header: SourceInfo) -> SourceInfo {
self.info.unwrap_or(SourceInfo { camera_tone: self.camera_tone.or(header.camera_tone), ..header })
}
}
#[derive(Clone)]
pub enum SourceRef {
Loaded(Arc<Rgb32f>),
Loaded(Box<DecodedSource>),
Demo {
scene: Box<lightcraft_scenes::Scene>,
max_edge: usize,
@@ -82,6 +104,9 @@ pub enum SourceRef {
path: String,
max_edge: usize,
loader: Option<FileLoader>,
/// The photo's smart preview, used when the original can't be read (offline drive). The
/// UI thread never checks whether the original is there; the render worker finds out.
fallback: Option<std::path::PathBuf>,
},
/// A smart preview standing in for a missing original.
Smart {
@@ -91,28 +116,41 @@ pub enum SourceRef {
impl SourceRef {
pub fn load(&self) -> Result<Arc<Rgb32f>, String> {
match self {
SourceRef::Loaded(a) => Ok(a.clone()),
SourceRef::Demo { scene, max_edge } => Ok(Arc::new(scene.render_fit(*max_edge))),
SourceRef::File { path, max_edge, loader } => match loader {
Some(l) => l(path, *max_edge).map(|(img, _)| Arc::new(img)),
None => Err(format!("no decoder available for {path}")),
},
self.load_source().map(|s| s.image)
}
pub fn load_source(&self) -> Result<DecodedSource, String> {
let image = match self {
SourceRef::Loaded(a) => return Ok((**a).clone()),
SourceRef::Demo { scene, max_edge } => Arc::new(scene.render_fit(*max_edge)),
SourceRef::File { path, max_edge, loader, fallback } => {
let r = match loader {
Some(l) => l(path, *max_edge).map(|(image, info)| DecodedSource::new(Arc::new(image), Some(info))),
None => Err(format!("no decoder available for {path}")),
};
// An offline original renders from its smart preview (no decoder facts: header ones).
return match (r, fallback) {
#[cfg(not(target_arch = "wasm32"))]
(Err(e), Some(sp)) if crate::smart::is_valid(sp) => crate::smart::load(sp).map_err(|e2| format!("{e}; smart preview: {e2}")),
(r, _) => r,
};
}
#[cfg(not(target_arch = "wasm32"))]
SourceRef::Smart { path } => crate::smart::load(path),
SourceRef::Smart { path } => return crate::smart::load(path),
#[cfg(target_arch = "wasm32")]
SourceRef::Smart { .. } => Err("smart previews are not available here".into()),
}
SourceRef::Smart { .. } => return Err("smart previews are not available here".into()),
};
Ok(DecodedSource::new(image, None))
}
}
pub struct MediaCache {
/// Decoded thumbnail-level sources (LRU by bytes).
thumbs: Lru<PhotoId, Arc<Rgb32f>>,
thumbs: Lru<PhotoId, DecodedSource>,
/// Decoded preview-level sources with their last use ([`lightcraft_preview::next_tick`]).
previews: Vec<(PhotoId, Arc<Rgb32f>, u64)>,
previews: Vec<(PhotoId, DecodedSource, u64)>,
/// The last full-resolution original (exports; one at a time: ~300 MB at 24 MP).
full: Option<(PhotoId, Arc<Rgb32f>, u64)>,
full: Option<(PhotoId, DecodedSource, u64)>,
/// How many previews to keep (LRU).
pub preview_capacity: usize,
/// Bytes all decoded sources and rendered previews in memory may take together (the cache
@@ -125,6 +163,9 @@ pub struct MediaCache {
pub file_bytes: Option<crate::merge::ByteReader>,
/// The library's smart previews folder (originals offline: render from the proxy).
pub smart_dir: Option<std::path::PathBuf>,
/// Whether originals (and smart previews) are on disk: cached and checked off the UI thread
/// in the app (see [`crate::availability`]).
pub availability: crate::availability::Availability,
scenes: Vec<lightcraft_scenes::Scene>,
/// Rendered thumbnails (memory, plus disk once a library is attached).
pub rendered: Arc<PreviewCache>,
@@ -144,6 +185,7 @@ impl Default for MediaCache {
preview_loader: None,
file_bytes: None,
smart_dir: None,
availability: Default::default(),
scenes: Vec::new(),
rendered: Arc::new(PreviewCache::memory(rendered_budget(budget))),
}
@@ -156,7 +198,7 @@ fn rendered_budget(share: usize) -> usize {
}
fn source_bytes(img: &Rgb32f) -> usize {
img.data.len() * 12 + 64
img.data.len() * 12 + 64 + std::mem::size_of::<SourceInfo>()
}
impl MediaCache {
@@ -207,8 +249,8 @@ impl MediaCache {
let Some((tick, which)) = candidates.into_iter().flatten().min() else { break };
let freed = match which {
0 => self.thumbs.pop_oldest(),
1 => self.previews.iter().position(|e| e.2 == tick).map(|i| source_bytes(&self.previews.remove(i).1)),
2 => self.full.take().map(|e| source_bytes(&e.1)),
1 => self.previews.iter().position(|e| e.2 == tick).map(|i| source_bytes(&self.previews.remove(i).1.image)),
2 => self.full.take().map(|e| source_bytes(&e.1.image)),
_ => self.rendered.evict_oldest(),
};
match freed {
@@ -219,6 +261,10 @@ impl MediaCache {
}
pub fn get(&mut self, id: PhotoId, level: SourceLevel) -> Option<Arc<Rgb32f>> {
self.get_source(id, level).map(|s| s.image)
}
fn get_source(&mut self, id: PhotoId, level: SourceLevel) -> Option<DecodedSource> {
match level {
SourceLevel::Thumb => self.thumbs.get(&id).cloned(),
SourceLevel::Preview => self.previews.iter_mut().find(|e| e.0 == id).map(|e| {
@@ -232,11 +278,31 @@ impl MediaCache {
}
}
/// The facts of photo `id`'s cached sources: the decoder's when one has them, else `header`
/// (with a smart preview's stored camera tone curve).
fn source_facts(&self, id: PhotoId, header: SourceInfo) -> SourceInfo {
let cached = || {
self.thumbs
.peek(&id)
.into_iter()
.chain(self.previews.iter().filter(|e| e.0 == id).map(|e| &e.1))
.chain(self.full.iter().filter(|e| e.0 == id).map(|e| &e.1))
};
match cached().find(|s| s.info.is_some()).or_else(|| cached().next()) {
Some(s) => s.info_or(header),
None => header,
}
}
pub fn insert(&mut self, id: PhotoId, level: SourceLevel, img: Arc<Rgb32f>) {
self.insert_source(id, level, DecodedSource::new(img, None));
}
fn insert_source(&mut self, id: PhotoId, level: SourceLevel, img: DecodedSource) {
let tick = lightcraft_preview::next_tick();
match level {
SourceLevel::Thumb => {
let cost = source_bytes(&img);
let cost = source_bytes(&img.image);
self.thumbs.insert(id, img, cost);
}
SourceLevel::Preview => {
@@ -270,29 +336,36 @@ impl MediaCache {
/// Decoded sources held: (thumbnail level, preview level, full size).
pub fn usage(&self) -> (crate::memory::Usage, crate::memory::Usage, crate::memory::Usage) {
use crate::memory::Usage;
let previews = Usage::new(self.previews.len(), self.previews.iter().map(|e| source_bytes(&e.1)).sum());
let full = self.full.as_ref().map(|e| Usage::new(1, source_bytes(&e.1))).unwrap_or_default();
let previews = Usage::new(self.previews.len(), self.previews.iter().map(|e| source_bytes(&e.1.image)).sum());
let full = self.full.as_ref().map(|e| Usage::new(1, source_bytes(&e.1.image))).unwrap_or_default();
(Usage::new(self.thumbs.len(), self.thumbs.cost()), previews, full)
}
pub fn source_ref(&mut self, p: &Photo, level: SourceLevel) -> SourceRef {
if let Some(a) = self.get(p.id, level) {
return SourceRef::Loaded(a);
if let Some(a) = self.get_source(p.id, level) {
return SourceRef::Loaded(Box::new(a));
}
// Procedural scenes have a nominal size: "full" is that size, not unbounded.
let max_edge = level.max_edge().min(match (&p.source, level) {
(Source::Demo { .. }, SourceLevel::Full) => p.width.max(p.height).max(1) as usize,
_ => usize::MAX,
});
// the original is offline: its smart preview, when there is one
// the original is offline: its smart preview, when there is one. Only cached answers are
// used here (no file-system call on the UI thread in the app); an original not known to
// be offline is tried first and its smart preview is the render worker's fallback.
#[cfg(not(target_arch = "wasm32"))]
if let (Source::File { path }, Some(dir)) = (&p.source, &self.smart_dir)
&& !std::path::Path::new(path).exists()
{
if let (Source::File { path }, Some(dir)) = (&p.source, &self.smart_dir) {
let sp = dir.join(crate::smart::file_name(p));
if crate::smart::is_valid(&sp) {
// (a proxy cut short fails to load, like the offline original; the worker's fallback
// in `SourceRef::load` only uses a proxy that passes `smart::is_valid`)
if self.availability.is_offline(path) && self.availability.exists(&sp.to_string_lossy()) == Some(true) {
return SourceRef::Smart { path: sp };
}
let mut r = self.origin_ref(&p.source, max_edge);
if let SourceRef::File { fallback, .. } = &mut r {
*fallback = Some(sp);
}
return r;
}
self.origin_ref(&p.source, max_edge)
}
@@ -307,10 +380,10 @@ impl MediaCache {
}
match self.scenes.iter().find(|s| s.id == *scene) {
Some(s) => SourceRef::Demo { scene: Box::new(s.clone()), max_edge },
None => SourceRef::File { path: format!("demo:{scene}"), max_edge, loader: None },
None => SourceRef::File { path: format!("demo:{scene}"), max_edge, loader: None, fallback: None },
}
}
Source::File { path } => SourceRef::File { path: path.clone(), max_edge, loader: self.file_loader.clone() },
Source::File { path } => SourceRef::File { path: path.clone(), max_edge, loader: self.file_loader.clone(), fallback: None },
}
}
}
@@ -344,7 +417,7 @@ pub struct RenderResult {
pub key: u64,
pub rendered: Result<Rendered, String>,
/// A source that was loaded by this job (to be inserted into the cache).
pub loaded: Option<Arc<Rgb32f>>,
pub loaded: Option<DecodedSource>,
/// Set for a [`QuickJob`]'s stand-in: where the image came from.
pub quick: Option<QuickSource>,
}
@@ -410,12 +483,14 @@ impl RenderJob {
};
}
let was_loaded = matches!(self.source, SourceRef::Loaded(_));
match self.source.load() {
Ok(src) => {
match self.source.load_source() {
Ok(source) => {
let src = &source.image;
let info = source.info_or(self.info);
// Thumbnails (many small jobs side by side) stay on the CPU; views and exports use
// the GPU when there is one.
let gpu = self.cache.is_none();
let rendered = develop(&src, &self.info, &self.settings, &self.request, self.stages.as_deref(), gpu);
let rendered = develop(src, &info, &self.settings, &self.request, self.stages.as_deref(), gpu);
if let Some((cache, key)) = &self.cache {
cache.put(*key, Arc::new(rendered.image.clone()));
}
@@ -429,7 +504,7 @@ impl RenderJob {
level: self.level,
key: self.key,
rendered: Ok(rendered),
loaded: (!was_loaded).then_some(src),
loaded: (!was_loaded).then_some(source),
quick: None,
}
}
@@ -521,11 +596,16 @@ pub fn content_key(p: &Photo) -> String {
pub fn source_info(p: &Photo) -> SourceInfo {
// Procedural demo scenes are scene-referred HDR (like raw files): use the filmic tone map.
if matches!(p.source, Source::Demo { .. }) {
return SourceInfo { raw: true, as_shot_temp: 6500.0, as_shot_tint: 0.0, lens: None };
return SourceInfo { raw: true, ..Default::default() };
}
// A raw shown from its embedded preview is a rendered (display-referred) JPEG: relative white
// balance and the display tone curve, like any other rendered file.
if p.develops_raw() { SourceInfo { raw: true, as_shot_temp: 5500.0, as_shot_tint: 0.0, lens: p.embedded_lens } } else { SourceInfo::default() }
if p.develops_raw() {
let (temp, tint) = if p.relative_wb() { (6500.0, 0.0) } else { p.as_shot_wb.unwrap_or((5500.0, 0.0)) };
SourceInfo { raw: true, as_shot_temp: temp, as_shot_tint: tint, lens: p.embedded_lens, relative_wb: p.relative_wb(), ..Default::default() }
} else {
SourceInfo::default()
}
}
impl crate::Session {
@@ -680,7 +760,7 @@ impl crate::Session {
/// Accept a finished job's loaded source into the cache.
pub fn accept(&mut self, r: &RenderResult) {
if let Some(src) = &r.loaded {
self.media.insert(r.photo, r.level, src.clone());
self.media.insert_source(r.photo, r.level, src.clone());
}
}
@@ -723,12 +803,19 @@ impl crate::Session {
Ok(job)
}
/// Prefer decoder facts to header-only metadata for pixel-statistics commands.
pub fn source_info(&self, id: PhotoId) -> SourceInfo {
let header = self.catalog.photo(id).map(|p| source_info(p)).unwrap_or_default();
self.media.source_facts(id, header)
}
/// The source proxy for pixel-statistics commands (auto tone/WB), loading synchronously.
pub fn source_now(&mut self, id: PhotoId, level: SourceLevel) -> Result<Arc<Rgb32f>, String> {
let p = self.catalog.photo(id).ok_or("no such photo")?.clone();
let r = self.media.source_ref(&p, level).load()?;
self.media.insert(id, level, r.clone());
Ok(r)
let r = self.media.source_ref(&p, level).load_source()?;
let image = r.image.clone();
self.media.insert_source(id, level, r);
Ok(image)
}
}
@@ -743,7 +830,9 @@ pub struct ProbeInfo {
pub captured: Option<String>,
pub meta: lightcraft_catalog::Meta,
pub as_shot_wb: Option<(f64, f64)>,
/// Hash of the file's bytes (hex), for duplicate detection.
/// Hash of the file's bytes (hex), for duplicate detection. When it is a 32-digit
/// [`lightcraft_preview::hash_bytes`] of the whole file (as the native probe computes it),
/// Import → Copy verifies each copy against it instead of reading the source again.
pub content_hash: Option<String>,
/// Lens corrections embedded in the file (DNG opcodes).
pub embedded_lens: Option<lightcraft_develop::EmbeddedLens>,
@@ -760,6 +849,42 @@ pub type FileProbe = Arc<dyn Fn(&str) -> Result<ProbeInfo, String> + Send + Sync
mod tests {
use super::*;
#[test]
fn decoder_info_survives_render_jobs_cache_and_eviction() {
let tone = lightcraft_pipeline::tone::CameraTone::new(std::array::from_fn(|i| {
let x = 0.01 * (i + 1) as f32;
[x, (x * 2.0).min(0.9)]
}))
.unwrap();
let info = SourceInfo { raw: true, relative_wb: true, camera_tone: Some(tone), ..Default::default() };
let mut s = crate::Session::with_demo();
let id = s.active().unwrap();
let mut job = s.render_job(id, 64, 64, false, true).unwrap();
job.source = SourceRef::File {
path: "synthetic.arw".into(),
max_edge: 64,
loader: Some(Arc::new(move |_, _| {
let mut image = Rgb32f::new(64, 64);
image.data.fill([0.1; 3]);
Ok((image, info))
})),
fallback: None,
};
job.info = SourceInfo::default(); // Header facts cannot override decoder facts.
job.settings = Arc::new(DevelopSettings::default());
let r = job.clone().run();
assert_eq!(r.loaded.as_ref().unwrap().info, Some(info));
let expected = lightcraft_pipeline::render(&r.loaded.as_ref().unwrap().image, &info, &job.settings, &job.request);
assert_eq!(r.rendered.as_ref().unwrap().image.data, expected.image.data);
s.accept(&r);
job.source = s.media.source_ref(s.catalog.photo(id).unwrap(), job.level);
let again = job.run();
assert!(again.loaded.is_none());
assert_eq!(r.rendered.unwrap().image.data, again.rendered.unwrap().image.data);
s.media.forget(id);
assert!(s.media.get_source(id, r.level).is_none());
}
#[test]
fn budget_evicts_least_recently_used_across_caches() {
let img = |w: usize| Arc::new(Rgb32f::new(w, w));
+7 -2
View File
@@ -68,9 +68,14 @@ impl crate::Session {
OriginalGuard::new(&self.catalog)
}
/// `Err` when writing `path` would replace a catalogued photo's original or sidecar.
/// `Err` when writing `path` would replace a catalogued photo's original or sidecar. A free
/// path (the usual case) is answered without building the guard over the whole library.
pub fn check_write_target(&self, path: &str) -> Result<(), String> {
self.original_guard().check(Path::new(path))
let path = Path::new(path);
if std::fs::symlink_metadata(path).is_err() {
return Ok(()); // nothing there to replace
}
self.original_guard().check(path)
}
}
+87 -64
View File
@@ -538,6 +538,92 @@ fn set_file_ops_to(op: &Op, to: &HashSet<&str>, out: &mut Vec<Op>) {
}
}
/// The photos among `ids` a rename handles, in order: photos sharing one file (virtual copies)
/// rename once. No file-system calls.
pub fn rename_photos(cat: &lightcraft_catalog::Catalog, ids: &[PhotoId]) -> Vec<std::sync::Arc<Photo>> {
let mut seen_paths: HashSet<String> = HashSet::new();
ids.iter()
.filter_map(|id| cat.photo(*id))
.filter(|p| match &p.source {
Source::File { path } => seen_paths.insert(path.clone()),
_ => true,
})
.cloned()
.collect()
}
/// [`Session::plan_rename`] for [`rename_photos`], with the "is this name taken on disk?" check
/// supplied: the Rename dialog plans its preview on a worker thread (a check can block on a slow
/// drive). Whether a case-only change names this very file is decided on the disk.
pub fn plan_rename_photos(photos: &[std::sync::Arc<Photo>], template: &str, start: usize, exists: &dyn Fn(&str) -> bool) -> Vec<RenamePlan> {
plan_rename_core(photos, template, start, &|p| exists(&p.to_string_lossy()), &|a, b| RealFs.same_file(a, b))
}
/// Plan renaming `photos`: `exists` says whether something answers to a path, `same_file` whether
/// two spellings name one file (see [`MoveFs`]).
fn plan_rename_core(
photos: &[std::sync::Arc<Photo>],
template: &str,
start: usize,
exists: &dyn Fn(&Path) -> bool,
same_file: &dyn Fn(&Path, &Path) -> bool,
) -> Vec<RenamePlan> {
let mut taken: HashSet<String> = HashSet::new(); // lower-case target paths/names claimed in this batch
let mut out = Vec::new();
for (seq, p) in (start..).zip(photos) {
let want = expand(template, p, seq);
let (stem, ext) = split_ext(&want);
let (stem, ext) = (stem.to_string(), ext.to_string());
let candidate = |k: usize| {
if k == 0 {
want.clone()
} else if ext.is_empty() {
format!("{stem}-{k}")
} else {
format!("{stem}-{k}.{ext}")
}
};
let (to, to_path) = match &p.source {
Source::File { path } => {
let dir = Path::new(path).parent().map(Path::to_path_buf).unwrap_or_default();
let mut k = 0;
loop {
let name = candidate(k);
let tp = dir.join(&name).to_string_lossy().to_string();
let key = tp.to_lowercase();
// this very file, maybe spelled differently (case-insensitive volume)? By
// identity: on a case-sensitive volume `img_1.jpg` may be another photo
let (from, to) = (Path::new(path), Path::new(&tp));
let same = tp == *path || (from != to && key == path.to_lowercase() && exists(to) && same_file(from, to));
// free: not claimed in this batch and not on disk (unless it is this very file).
// A file this batch moves away still counts as taken: simple and safe.
if !taken.contains(&key) && (same || !exists(to)) {
taken.insert(key);
break (name, Some(tp));
}
k += 1;
}
}
Source::Demo { .. } => {
let mut k = 0;
loop {
let name = candidate(k);
if taken.insert(format!("demo:{}", name.to_lowercase())) {
break (name, None);
}
k += 1;
}
}
};
let from_path = match &p.source {
Source::File { path } => Some(path.clone()),
_ => None,
};
out.push(RenamePlan { id: p.id.0, from: p.file_name.clone(), to, from_path, to_path });
}
out
}
impl Session {
/// Plan renaming `ids` with `template` (sequence numbers from `start`), resolving collisions.
pub fn plan_rename(&self, ids: &[PhotoId], template: &str, start: usize) -> Vec<RenamePlan> {
@@ -546,70 +632,7 @@ impl Session {
/// [`Session::plan_rename`] on the file system `fs`.
pub(crate) fn plan_rename_with(&self, fs: &dyn MoveFs, ids: &[PhotoId], template: &str, start: usize) -> Vec<RenamePlan> {
// photos sharing one file (virtual copies) rename once
let mut seen_paths: HashSet<String> = HashSet::new();
let mut taken: HashSet<String> = HashSet::new(); // lower-case target paths/names claimed in this batch
let mut out = Vec::new();
let mut seq = start;
for id in ids {
let Some(p) = self.catalog.photo(*id) else { continue };
if let Source::File { path } = &p.source
&& !seen_paths.insert(path.clone())
{
continue;
}
let want = expand(template, p, seq);
seq += 1;
let (stem, ext) = split_ext(&want);
let (stem, ext) = (stem.to_string(), ext.to_string());
let candidate = |k: usize| {
if k == 0 {
want.clone()
} else if ext.is_empty() {
format!("{stem}-{k}")
} else {
format!("{stem}-{k}.{ext}")
}
};
let (to, to_path) = match &p.source {
Source::File { path } => {
let dir = Path::new(path).parent().map(Path::to_path_buf).unwrap_or_default();
let mut k = 0;
loop {
let name = candidate(k);
let tp = dir.join(&name).to_string_lossy().to_string();
let key = tp.to_lowercase();
// this very file, maybe spelled differently (case-insensitive volume)? By
// identity: on a case-sensitive volume `img_1.jpg` may be another photo
let same = tp == *path || is_respelling(fs, Path::new(path), Path::new(&tp));
// free: not claimed in this batch and not on disk (unless it is this very file).
// A file this batch moves away still counts as taken: simple and safe.
let on_disk = fs.exists(Path::new(&tp)) && !same;
if !taken.contains(&key) && !on_disk {
taken.insert(key);
break (name, Some(tp));
}
k += 1;
}
}
Source::Demo { .. } => {
let mut k = 0;
loop {
let name = candidate(k);
if taken.insert(format!("demo:{}", name.to_lowercase())) {
break (name, None);
}
k += 1;
}
}
};
let from_path = match &p.source {
Source::File { path } => Some(path.clone()),
_ => None,
};
out.push(RenamePlan { id: id.0, from: p.file_name.clone(), to, from_path, to_path });
}
out
plan_rename_core(&rename_photos(&self.catalog, ids), template, start, &|p| fs.exists(p), &|a, b| fs.same_file(a, b))
}
/// Carry out `plans`: move the files (rolled back on failure), then commit one undoable op
+60 -13
View File
@@ -1,13 +1,16 @@
//! Smart previews: a compact proxy of each photo (its decoded preview-size source, about 1 MB)
//! kept in the library, so photos stay editable — and exportable at proxy size — while their
//! originals are offline (an unplugged drive). The proxy is the scene-linear source scaled into
//! 0..1 by a stored factor, sRGB-encoded and saved as a JPEG after a one-line header.
//! 0..1 by a stored factor, sRGB-encoded and saved as a JPEG after a one-line header. The header
//! also keeps the decoder's file-local camera tone curve (Sony ARW camera look), which is applied
//! at render time and is not baked into the pixels.
use std::path::{Path, PathBuf};
use std::sync::Arc;
use lightcraft_catalog::Photo;
use lightcraft_color::transfer::linear_to_srgb;
use lightcraft_pipeline::tone::CameraTone;
use lightcraft_raster::{Rgb32f, Rgba8};
const MAGIC: &[u8] = b"LCSP1\n";
@@ -18,8 +21,11 @@ pub fn file_name(p: &Photo) -> String {
format!("{:032x}.lcsp", h.0)
}
/// Encode a source image as a smart preview.
pub fn encode(img: &Rgb32f) -> Result<Vec<u8>, String> {
/// The most header [`is_valid`] reads (the JSON line before the JPEG).
const HEADER_MAX: u64 = 4096;
/// Encode a source image (and the decoder's camera tone curve, if any) as a smart preview.
pub fn encode(img: &Rgb32f, tone: Option<&CameraTone>) -> Result<Vec<u8>, String> {
// scale so all but the brightest 0.05 % fit into 0..1
let mut lum: Vec<f32> = img.data.iter().map(|c| c[0].max(c[1]).max(c[2])).filter(|v| v.is_finite()).collect();
let scale = if lum.is_empty() {
@@ -47,22 +53,29 @@ pub fn encode(img: &Rgb32f) -> Result<Vec<u8>, String> {
)
.map_err(|e| e.to_string())?;
let mut out = MAGIC.to_vec();
out.extend_from_slice(format!("{{\"w\":{},\"h\":{},\"scale\":{scale}}}\n", img.width, img.height).as_bytes());
let mut head = serde_json::json!({"w": img.width, "h": img.height, "scale": scale});
if let Some(t) = tone {
head["tone"] = serde_json::to_value(t).map_err(|e| e.to_string())?;
}
out.extend_from_slice(head.to_string().as_bytes());
out.push(b'\n');
out.extend_from_slice(&jpg);
Ok(out)
}
/// Decode a smart preview back into a source image.
pub fn decode(bytes: &[u8]) -> Result<Rgb32f, String> {
/// Decode a smart preview back into a source image and its stored camera tone curve (an invalid
/// curve is ignored, like a missing one).
pub fn decode(bytes: &[u8]) -> Result<(Rgb32f, Option<CameraTone>), String> {
let rest = bytes.strip_prefix(MAGIC).ok_or("not a smart preview")?;
let nl = rest.iter().position(|b| *b == b'\n').ok_or("bad smart preview")?;
let head: serde_json::Value = serde_json::from_slice(&rest[..nl]).map_err(|e| e.to_string())?;
let scale = head["scale"].as_f64().unwrap_or(1.0) as f32;
let tone = head.get("tone").and_then(|t| serde_json::from_value::<CameraTone>(t.clone()).ok());
let d = lightcraft_codecs::decode(&rest[nl + 1..], Default::default()).map_err(|e| e.to_string())?;
// the decoder undoes the sRGB encoding; the values are the source's own primaries
let mut img = d.image;
img.data.iter_mut().for_each(|c| *c = c.map(|v| v * scale));
Ok(img)
Ok((img, tone))
}
/// Where a library keeps its smart previews.
@@ -157,8 +170,8 @@ pub fn migrate(from: &Path, to: &Path, what: Existing) -> (usize, Vec<String>) {
pub fn is_valid(path: &Path) -> bool {
use std::io::{Read, Seek, SeekFrom};
let Ok(mut f) = std::fs::File::open(path) else { return false };
let mut head = Vec::with_capacity(256);
if (&mut f).take(256).read_to_end(&mut head).is_err() {
let mut head = Vec::with_capacity(1024);
if (&mut f).take(HEADER_MAX).read_to_end(&mut head).is_err() {
return false;
}
let Some(rest) = head.strip_prefix(MAGIC) else { return false };
@@ -175,9 +188,9 @@ pub fn is_valid(path: &Path) -> bool {
}
/// Load the proxy at `path`.
pub fn load(path: &Path) -> Result<Arc<Rgb32f>, String> {
pub fn load(path: &Path) -> Result<crate::media::DecodedSource, String> {
let b = std::fs::read(path).map_err(|e| format!("{}: {e}", path.display()))?;
decode(&b).map(Arc::new)
decode(&b).map(|(image, camera_tone)| crate::media::DecodedSource { image: Arc::new(image), info: None, camera_tone })
}
#[cfg(test)]
@@ -228,7 +241,7 @@ mod tests {
std::fs::create_dir_all(&a).unwrap();
std::fs::create_dir_all(&b).unwrap();
let img = lightcraft_scenes::demo_library()[0].render(64, 40);
let good = encode(&img).unwrap();
let good = encode(&img, None).unwrap();
std::fs::write(a.join("1.lcsp"), &good).unwrap();
assert!(is_valid(&a.join("1.lcsp")));
for cut in [good.len() - 1, good.len() / 2, 10, 0] {
@@ -242,11 +255,45 @@ mod tests {
let _ = std::fs::remove_dir_all(&base);
}
#[test]
fn camera_tone_travels_with_the_proxy() {
let tone = CameraTone::new(std::array::from_fn(|i| {
let x = 0.004 * 1.17f32.powi(i as i32);
[x, 0.95 * (1.0 - (-2.7 * x).exp())]
}))
.unwrap();
let img = lightcraft_scenes::demo_library()[0].render(64, 40);
let bytes = encode(&img, Some(&tone)).unwrap();
assert_eq!(decode(&bytes).unwrap().1, Some(tone));
let dir = temp("tone");
let path = dir.join("t.lcsp");
std::fs::write(&path, &bytes).unwrap();
// the longer header is still read by the validity check, and the curve by `load`
assert!(is_valid(&path));
let loaded = load(&path).unwrap();
assert_eq!(loaded.camera_tone, Some(tone));
let header = lightcraft_pipeline::SourceInfo { raw: true, ..Default::default() };
assert_eq!(loaded.info_or(header).camera_tone, Some(tone));
// a hostile curve (reversing knots) is ignored, not trusted
let nl = MAGIC.len() + bytes[MAGIC.len()..].iter().position(|b| *b == b'\n').unwrap();
let mut head: serde_json::Value = serde_json::from_slice(&bytes[MAGIC.len()..nl]).unwrap();
let mut knots: Vec<[f32; 2]> = serde_json::from_value(head["tone"]["knots"].clone()).unwrap();
knots.swap(3, 4);
head["tone"]["knots"] = serde_json::json!(knots);
let mut hostile = MAGIC.to_vec();
hostile.extend_from_slice(head.to_string().as_bytes());
hostile.extend_from_slice(&bytes[nl..]);
let (pixels, tone) = decode(&hostile).unwrap();
assert_eq!((pixels.width, tone), (img.width, None));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn roundtrip_keeps_the_picture() {
let img = lightcraft_scenes::demo_library()[0].render(160, 100);
let back = decode(&encode(&img).unwrap()).unwrap();
let (back, tone) = decode(&encode(&img, None).unwrap()).unwrap();
assert_eq!((back.width, back.height), (img.width, img.height));
assert!(tone.is_none());
let mut err = 0.0f64;
for (a, b) in img.data.iter().zip(&back.data) {
for k in 0..3 {
+24
View File
@@ -10,6 +10,30 @@ fn active_dev(s: &Session) -> lightcraft_develop::DevelopSettings {
(*s.develop_of(s.active().unwrap()).unwrap()).clone()
}
#[test]
fn changing_one_wb_control_resolves_as_shot_without_stale_tint() {
use lightcraft_catalog::{Photo, PhotoId, Source};
use lightcraft_develop::{DevelopSettings, WbMode};
let mut s = demo();
let id = PhotoId(100);
let mut p = Photo::new(id, Source::File { path: "synthetic.arw".into() }, "synthetic.arw", "ARW", 16, 16, "");
// A catalog made by the old generic-matrix Kelvin inference.
p.as_shot_wb = Some((6829.0, -127.0));
p.develop = std::sync::Arc::new(DevelopSettings::for_raw(6829.0, -127.0));
s.catalog.apply(lightcraft_catalog::Op::AddPhoto { photo: Box::new(p) }).unwrap();
s.execute("library.select", &json!({"ids": [100], "active": 100})).unwrap();
s.execute("develop.set", &json!({"control": "wb.temp", "value": 8000})).unwrap();
let d = active_dev(&s);
assert_eq!(d.wb.mode, WbMode::Custom);
assert_eq!(d.wb.temp, 8000.0);
assert_eq!(d.wb.tint, 0.0, "untouched tint comes from the current as-shot reference");
s.execute("develop.wb", &json!({"mode": "asShot"})).unwrap();
s.execute("develop.set", &json!({"control": "wb.tint", "value": 10})).unwrap();
assert_eq!(active_dev(&s).wb.temp, 6500.0);
s.execute("develop.reset", &json!({})).unwrap();
assert_eq!((active_dev(&s).wb.temp, active_dev(&s).wb.tint), (6500.0, 0.0));
}
#[test]
fn demo_library_loads() {
let mut s = demo();
+21
View File
@@ -323,3 +323,24 @@ fn a_failed_export_write_keeps_the_previous_file() {
assert_ne!(std::fs::read(&prev).unwrap(), b"last week's export");
let _ = std::fs::remove_dir_all(&dir);
}
/// Issue #134: exports can always be made again, so they are written atomically but not synced to
/// disk file by file; an edit copy that becomes a library photo still is.
#[test]
fn exports_are_atomic_without_a_sync() {
use crate::export::{Conflict, Destination};
use lightcraft_catalog::safe_file::syncs_on_this_thread;
let (mut s, id, dir, _src, _) = library_with_jpeg("export-nosync");
let out = dir.join("out");
let to = Destination { dir: out.to_string_lossy().to_string(), exact: None };
let o = ExportOptions { conflict: Conflict::Unique, ..Default::default() };
let before = syncs_on_this_thread();
for _ in 0..3 {
disk_batch(&mut s, id, &o, &to).unwrap();
}
assert_eq!(syncs_on_this_thread(), before, "no per-file sync for exports");
assert_eq!(std::fs::read_dir(&out).unwrap().count(), 3, "three exports, no temp files");
crate::export::write_file_durable(&out.join("edit.tif").to_string_lossy(), b"tiff").unwrap();
assert!(syncs_on_this_thread() > before, "the durable writer syncs");
let _ = std::fs::remove_dir_all(&dir);
}
+140 -7
View File
@@ -144,11 +144,10 @@ fn browsing_a_folder_lists_its_photos_without_adding_them() {
let _ = std::fs::remove_dir_all(&dir);
}
/// Issue #105: Find Missing finds renamed files by content, prefers the content match over a
/// same-name same-size impostor, and skips (reports) photos it can't tell apart.
#[test]
fn find_missing_matches_renamed_files_by_content() {
let dir = temp_dir("missing-content");
/// Photos a.png, b.png, c.png imported from `old`, then: a renamed into `moved`; b renamed, with
/// an impostor (same name and size, other bytes) under its old name; c gone, with two impostors.
fn renamed_and_ambiguous_scene(tag: &str) -> (std::path::PathBuf, Session) {
let dir = temp_dir(tag);
std::fs::create_dir_all(dir.join("old")).unwrap();
std::fs::create_dir_all(dir.join("moved/real")).unwrap();
for (n, seed) in [("a.png", 1), ("b.png", 2), ("c.png", 3)] {
@@ -156,20 +155,26 @@ fn find_missing_matches_renamed_files_by_content() {
}
let mut s = Session::new().with_fs();
s.execute("library.import", &json!({"paths": [dir.join("old").to_string_lossy()]})).unwrap();
// an impostor: same name and size, other bytes
let impostor = |src: &Path, dst: &Path| {
let mut b = std::fs::read(src).unwrap();
let k = b.len() - 20;
b[k] ^= 0xff;
std::fs::write(dst, b).unwrap();
};
// a: renamed; b: renamed, with an impostor under its old name; c: gone, two impostors
std::fs::rename(dir.join("old/a.png"), dir.join("moved/Trip-001.png")).unwrap();
impostor(&dir.join("old/b.png"), &dir.join("moved/b.png"));
std::fs::rename(dir.join("old/b.png"), dir.join("moved/real/Trip-002.png")).unwrap();
impostor(&dir.join("old/c.png"), &dir.join("moved/c.png"));
impostor(&dir.join("old/c.png"), &dir.join("moved/real/c.png"));
std::fs::remove_file(dir.join("old/c.png")).unwrap();
(dir, s)
}
/// Issue #105: Find Missing finds renamed files by content, prefers the content match over a
/// same-name same-size impostor, and skips (reports) photos it can't tell apart.
#[test]
fn find_missing_matches_renamed_files_by_content() {
let (dir, mut s) = renamed_and_ambiguous_scene("missing-content");
let r = s.execute("library.findMissing", &json!({"folder": dir.join("moved").to_string_lossy()})).unwrap();
let found = r["found"].as_array().unwrap();
assert_eq!(found.len(), 2, "{r}");
@@ -183,6 +188,55 @@ fn find_missing_matches_renamed_files_by_content() {
let _ = std::fs::remove_dir_all(&dir);
}
/// Issue #104: the app plans Find Missing on a worker thread (no session) and relinks the plan
/// afterwards; that gives exactly what the synchronous command gives — renamed files by content,
/// impostors and look-alikes skipped (`ambiguous`).
#[test]
fn find_missing_planned_on_a_worker_matches_the_command() {
use crate::cmd::missing::{find_candidates, plan_find_missing};
let (dir, mut s) = renamed_and_ambiguous_scene("missing-worker");
let folder = dir.join("moved").to_string_lossy().to_string();
let candidates = find_candidates(&s.catalog);
let f = folder.clone();
let plan = std::thread::spawn(move || plan_find_missing(&candidates, &f)).join().unwrap().unwrap();
assert_eq!((plan.found.len(), plan.missing, plan.ambiguous.len()), (2, 1, 1), "{plan:?}");
let applied = s.execute("library.findMissing", &plan.to_json()).unwrap();
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.execute("library.missing", &json!({})).unwrap().as_array().map(Vec::len), Some(3), "undo restores all three");
let direct = s.execute("library.findMissing", &json!({"folder": folder})).unwrap();
assert_eq!(applied, direct);
assert_eq!(direct["found"].as_array().map(Vec::len), Some(2), "{direct}");
assert_eq!(direct["ambiguous"].as_array().map(Vec::len), Some(1), "{direct}");
let _ = std::fs::remove_dir_all(&dir);
}
/// Issue #104: a plan made on a worker is re-checked when it is applied: a photo relinked
/// meanwhile is left alone, and a file that is now another photo's original isn't used.
#[test]
fn find_missing_plan_is_rechecked_when_applied() {
use crate::cmd::missing::{find_candidates, plan_find_missing};
let (dir, mut s) = renamed_and_ambiguous_scene("missing-recheck");
let plan = plan_find_missing(&find_candidates(&s.catalog), &dir.join("moved").to_string_lossy()).unwrap();
let to = |name: &str| plan.found.iter().find(|f| f.from.ends_with(name)).unwrap().clone();
let (a, b) = (to("a.png"), to("b.png"));
// meanwhile: a is relinked by hand (elsewhere), and the ambiguous c is pointed at b's file
std::fs::copy(&a.to, dir.join("a-elsewhere.png")).unwrap();
s.execute("photo.relink", &json!({"id": a.id.0, "path": dir.join("a-elsewhere.png").to_string_lossy()})).unwrap();
let c = plan.ambiguous[0].id;
s.execute("photo.relink", &json!({"id": c.0, "path": b.to})).unwrap();
let r = s.execute("library.findMissing", &plan.to_json()).unwrap();
assert_eq!(r["found"].as_array().map(Vec::len), Some(0), "{r}");
assert_eq!(r["ambiguous"].as_array().map(Vec::len), Some(0), "c isn't missing any more: {r}");
assert_eq!(r["missing"], 1, "b stays missing: its file is c's now: {r}");
let path_of = |id: lightcraft_catalog::PhotoId| match &s.catalog.photo(id).unwrap().source {
lightcraft_catalog::Source::File { path } => path.clone(),
_ => String::new(),
};
assert_eq!(path_of(a.id), dir.join("a-elsewhere.png").to_string_lossy());
assert_eq!(path_of(b.id), b.from);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn missing_files_are_found_and_relinked() {
let dir = std::env::temp_dir().join(format!("lc-missing-{}", std::process::id()));
@@ -425,6 +479,20 @@ fn damaged_smart_previews_are_rebuilt_and_failed_writes_leave_none() {
let r = s.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!((r["built"].as_u64(), r["repaired"].as_u64()), (Some(1), Some(1)), "{r}");
assert_eq!(std::fs::read(&file).unwrap(), full, "rebuilt whole");
// the app's background build (issue #104) checks and repairs the same way, on a worker thread
std::fs::write(&file, &full[..full.len() / 2]).unwrap();
let r = s.execute("library.smartPreviews", &json!({"background": true})).unwrap();
assert_eq!(r["what"], "smart previews", "{r}");
let t0 = std::time::Instant::now();
let r = loop {
let r = s.execute("library.previewProgress", &json!({})).unwrap();
if r["running"] == false || t0.elapsed() > std::time::Duration::from_secs(60) {
break r;
}
std::thread::sleep(std::time::Duration::from_millis(5));
};
assert_eq!((r["done"].as_u64(), r["failed"].as_u64(), r["repaired"].as_u64()), (Some(1), Some(0), Some(1)), "{r}");
assert_eq!(std::fs::read(&file).unwrap(), full, "rebuilt whole in the background");
let _ = std::fs::remove_dir_all(&src);
let _ = std::fs::remove_dir_all(&lib);
}
@@ -518,6 +586,70 @@ fn folders_rename_and_move_with_their_photos() {
let _ = std::fs::remove_dir_all(&root);
}
/// Rename / Move Folder are undo steps (issue #97): undo renames the folder back on disk (files and
/// sidecars along) and restores the photos' paths, redo repeats it; a destination taken in the
/// meantime is never overwritten — the step is refused, reported and stays on the stack.
#[test]
fn folder_rename_and_move_undo_and_redo_on_disk() {
let root = temp_dir("folders-undo");
let a = root.join("Trip");
write_png(&a.join("one.png"), 3);
std::fs::write(a.join("one.xmp"), "<x:xmpmeta xmlns:x=\"adobe:ns:meta/\"/>").unwrap();
let mut s = Session::new().with_fs();
s.execute("library.import", &json!({"paths": [a.to_string_lossy()]})).unwrap();
let id = s.active().unwrap();
let path = |s: &Session| match &s.catalog.photo(id).unwrap().source {
lightcraft_catalog::Source::File { path } => path.clone(),
_ => panic!(),
};
let undo0 = s.undo.len();
let b = root.join("Italy");
s.execute("folder.rename", &json!({"path": a.to_string_lossy(), "name": "Italy"})).unwrap();
assert_eq!(s.undo.len(), undo0 + 1, "one undo step");
assert!(s.undo.last().unwrap().label.contains("Rename Folder"));
assert!(b.join("one.png").exists() && !a.exists());
s.execute("edit.undo", &json!({})).unwrap();
assert!(a.join("one.png").exists() && a.join("one.xmp").exists() && !b.exists(), "renamed back on disk");
assert_eq!(path(&s), a.join("one.png").to_string_lossy());
s.media.forget(id);
assert!(s.render_now(id, 16, 16).is_ok());
s.execute("edit.redo", &json!({})).unwrap();
assert!(b.join("one.png").exists() && b.join("one.xmp").exists() && !a.exists());
assert_eq!(path(&s), b.join("one.png").to_string_lossy());
// the old name is taken now: undo is refused, nothing is overwritten, the step stays
std::fs::create_dir_all(&a).unwrap();
std::fs::write(a.join("keep.txt"), "mine").unwrap();
let undo_n = s.undo.len();
let e = s.execute("edit.undo", &json!({})).unwrap_err().to_string();
assert!(e.contains("already exists"), "{e}");
assert_eq!(s.undo.len(), undo_n, "the step stays on the stack");
assert_eq!(std::fs::read_to_string(a.join("keep.txt")).unwrap(), "mine");
assert!(b.join("one.png").exists());
assert_eq!(path(&s), b.join("one.png").to_string_lossy());
std::fs::remove_dir_all(&a).unwrap();
// move into another folder, undo, redo
let into = root.join("Archive");
s.execute("folder.move", &json!({"path": b.to_string_lossy(), "into": into.to_string_lossy()})).unwrap();
let moved = into.join("Italy");
assert_eq!(path(&s), moved.join("one.png").to_string_lossy());
s.execute("edit.undo", &json!({})).unwrap();
assert!(b.join("one.png").exists() && !moved.exists());
assert_eq!(path(&s), b.join("one.png").to_string_lossy());
// the redo target is taken (even by an empty folder): refused, nothing replaced
std::fs::create_dir_all(&moved).unwrap();
assert!(s.execute("edit.redo", &json!({})).is_err());
assert!(b.join("one.png").exists());
std::fs::remove_dir(&moved).unwrap();
s.execute("edit.redo", &json!({})).unwrap();
assert!(moved.join("one.png").exists() && moved.join("one.xmp").exists() && !b.exists());
assert_eq!(path(&s), moved.join("one.png").to_string_lossy());
let _ = std::fs::remove_dir_all(&root);
}
/// Assisted culling: a burst (same scene, seconds apart) of a sharp and two soft shots, plus an
/// unrelated photo: the burst is grouped, the sharp one is its best, blurry shots can be rejected.
#[test]
@@ -667,6 +799,7 @@ fn smart_previews_folder_is_chosen_per_library() {
assert_eq!(s.execute("photo.smartPreview", &json!({})).unwrap()["smartPreview"], true);
// the setting is saved with the library
drop(s); // one session per library (issue #99)
let mut again = Session::new().with_fs();
again.open_library(&lib, false).unwrap();
assert_eq!(again.execute("library.smartPreviewsLocation", &json!({})).unwrap()["path"], to.to_string_lossy().as_ref());
+113
View File
@@ -207,6 +207,67 @@ fn copy_import_verifies_each_copy() {
let _ = std::fs::remove_dir_all(&base);
}
/// Copy verifies against the probe's content hash (issue #134): the hash, not a re-read of the
/// source, is the reference — a copy that matches the source but not the hash is refused, a
/// corrupted copy is caught, and the file hash agrees with `hash_bytes` across read chunks.
#[test]
fn copy_is_verified_against_the_probe_hash() {
use crate::import_move::copy_verified;
let base = temp_dir("copyhash");
let src = base.join("src.bin");
// larger than one 1 MB read, not a multiple of it
let bytes: Vec<u8> = (0..(3 << 20) + 12_345u32).map(|i| (i.wrapping_mul(2_654_435_761) >> 13) as u8).collect();
std::fs::write(&src, &bytes).unwrap();
let good = lightcraft_preview::hash_bytes(&bytes);
copy_verified(&src, &base.join("ok.bin"), Some(good)).unwrap();
assert_eq!(std::fs::read(base.join("ok.bin")).unwrap(), bytes);
// a hash that doesn't match: refused (a byte compare with the source would have passed)
let stale = lightcraft_preview::hash_bytes(b"what the probe read earlier");
let e = copy_verified(&src, &base.join("stale.bin"), Some(stale)).unwrap_err();
assert!(e.to_string().contains("differs"), "{e}");
assert!(!base.join("stale.bin").exists(), "the refused copy is removed");
// a copy that comes out different: caught by the hash
inject(Fault::Corrupt);
let r = copy_verified(&src, &base.join("bad.bin"), Some(good));
inject(Fault::None);
assert!(r.is_err());
assert!(!base.join("bad.bin").exists());
assert_eq!(std::fs::read(&src).unwrap(), bytes, "the source is untouched");
let _ = std::fs::remove_dir_all(&base);
}
/// A file replaced after Review Import probed it is not imported on the strength of the old probe:
/// its copy no longer matches the probe's hash, so it's reported as failed and nothing is left.
#[test]
fn copy_import_refuses_a_file_changed_since_the_review() {
let base = temp_dir("copychanged");
let (card, dest) = (base.join("card"), base.join("out"));
write_png(&card.join("a.png"), 3);
let mut s = session();
let r = s.execute("library.importPreview", &json!({"paths": [card.to_string_lossy()]})).unwrap();
assert_eq!(r["scanned"], 1, "{r}");
write_png(&card.join("a.png"), 9); // other pixels
let r = s
.execute(
"library.import",
&json!({"paths": [card.to_string_lossy()], "mode": "copy", "destination": dest.to_string_lossy(), "organize": "flat"}),
)
.unwrap();
assert_eq!(len(&r, "imported"), 0, "{r}");
assert_eq!(len(&r, "failed"), 1, "{r}");
assert!(r.to_string().contains("changed meanwhile"), "{r}");
assert_eq!(files_under(&dest), Vec::<String>::new());
// importing again probes afresh and succeeds
let r = s
.execute(
"library.import",
&json!({"paths": [card.to_string_lossy()], "mode": "copy", "destination": dest.to_string_lossy(), "organize": "flat"}),
)
.unwrap();
assert_eq!(len(&r, "imported"), 1, "{r}");
let _ = std::fs::remove_dir_all(&base);
}
/// A source that can't be removed (a read-only card) stays and is reported; the photo is
/// imported from its copy.
#[test]
@@ -320,3 +381,55 @@ fn move_into_the_library_is_saved_before_sources_go() {
assert_eq!(photo_paths(&again), [want.to_string_lossy()]);
let _ = std::fs::remove_dir_all(&base);
}
/// Timing for issue #134 (not a pass/fail test): `cargo test -p lightcraft-engine --release
/// write_policy_timing -- --ignored --nocapture`. Writes N export-sized files durably (temp +
/// sync + rename) and atomically only, and copies N files verified byte for byte and against the
/// probe hash, alternating so a loaded machine affects both alike. `LIGHTCRAFT_BENCH_DIR` puts
/// the files on another volume (a USB drive or a NAS is where the difference shows).
#[test]
#[ignore]
fn write_policy_timing() {
use crate::import_move::copy_verified;
use lightcraft_catalog::safe_file::{write_atomic, write_atomic_nosync};
use std::time::{Duration, Instant};
let root = std::env::var_os("LIGHTCRAFT_BENCH_DIR").map(PathBuf::from).unwrap_or_else(std::env::temp_dir);
let base = root.join(format!("lc-write-policy-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&base);
std::fs::create_dir_all(&base).unwrap();
let data = |n: usize, seed: u32| -> Vec<u8> { (0..n as u32).map(|i| (i.wrapping_add(seed).wrapping_mul(2_654_435_761) >> 11) as u8).collect() };
let (n, size) = (200usize, 300 << 10);
let (mut synced, mut unsynced) = (Duration::ZERO, Duration::ZERO);
for i in 0..n {
let b = data(size, i as u32);
let t = Instant::now();
write_atomic(&base.join(format!("sync-{i}.jpg")), &b).unwrap();
synced += t.elapsed();
let t = Instant::now();
write_atomic_nosync(&base.join(format!("nosync-{i}.jpg")), &b).unwrap();
unsynced += t.elapsed();
}
println!("export {n} × {} KB: write_atomic (synced) {synced:?}, write_atomic_nosync {unsynced:?}", size >> 10);
let (n, size) = (40usize, 8 << 20);
let card = base.join("card");
std::fs::create_dir_all(&card).unwrap();
let hashes: Vec<_> = (0..n)
.map(|i| {
let b = data(size, 7 * i as u32);
std::fs::write(card.join(format!("{i}.raw")), &b).unwrap();
lightcraft_preview::hash_bytes(&b)
})
.collect();
let (mut bytewise, mut hashed) = (Duration::ZERO, Duration::ZERO);
for (i, h) in hashes.iter().enumerate() {
let src = card.join(format!("{i}.raw"));
let t = Instant::now();
copy_verified(&src, &base.join(format!("bytes-{i}.raw")), None).unwrap();
bytewise += t.elapsed();
let t = Instant::now();
copy_verified(&src, &base.join(format!("hash-{i}.raw")), Some(*h)).unwrap();
hashed += t.elapsed();
}
println!("copy {n} × {} MB: byte compare with the source {bytewise:?}, probe hash {hashed:?}", size >> 20);
let _ = std::fs::remove_dir_all(&base);
}
+31
View File
@@ -113,6 +113,7 @@ fn close_writes_snapshot_and_view_state() {
let expect = s.catalog.to_snapshot();
s.close_library().unwrap();
assert_eq!(std::fs::metadata(dir.join("catalog.log")).unwrap().len(), 0);
drop(s); // one session per library (issue #99)
let s2 = open(&dir, true);
let r = &s2.library.as_ref().unwrap().report;
@@ -178,6 +179,7 @@ fn new_library_without_seed_is_empty_and_compacts() {
assert!(p["snapshots"].as_u64() >= Some(1), "{p}");
assert!(p["lastSnapshot"]["bytes"].as_u64() > Some(0), "{p}");
assert!(p["lastSnapshot"]["totalMs"].as_f64() >= p["lastSnapshot"]["serializeMs"].as_f64(), "{p}");
drop(s); // one session per library (issue #99)
let s2 = open(&dir, true);
assert_eq!(s2.catalog.len(), 0, "an existing library is never seeded");
assert_eq!(s2.catalog.albums().count(), 1);
@@ -376,3 +378,32 @@ fn scale_100k_library_queries() {
worst.sort_by(|a, b| b.1.total_cmp(&a.1));
eprintln!("slowest: {:?}", &worst[..3]);
}
/// Issue #99: a library is open in one session at a time. A second opener is refused (nothing
/// read or written), the owning session can reopen it, and it is free again once closed.
#[test]
fn a_library_open_elsewhere_is_refused() {
let dir = temp_dir("locked");
let mut s = open(&dir, true);
s.execute("photo.rate", &json!({"rating": 3})).unwrap();
let log = std::fs::read(dir.join("catalog.log")).unwrap();
let mut other = Session::new();
let e = other.open_library(&dir, true).unwrap_err();
assert!(matches!(e, crate::EngineError::LibraryInUse(_)), "{e:?}");
assert!(e.to_string().contains("already open in"), "{e}");
assert!(other.library.is_none());
assert_eq!(std::fs::read(dir.join("catalog.log")).unwrap(), log, "the refused opener wrote nothing");
// the session that has it open may reopen it (Settings → Open Library on the same folder)
s.close_library().unwrap();
s.open_library(&dir, true).unwrap();
assert!(other.open_library(&dir, true).is_err(), "still locked after reopening");
s.execute("photo.rate", &json!({"rating": 4})).unwrap();
let expect = s.catalog.to_snapshot();
drop(s);
other.open_library(&dir, true).unwrap();
assert_eq!(other.catalog.to_snapshot(), expect);
let _ = std::fs::remove_dir_all(&dir);
}
+2
View File
@@ -126,11 +126,13 @@ fn preferences_persist_with_the_library() {
assert_eq!(r["persistent"], true);
// export presets are library preferences too
s.execute("export.savePreset", &json!({"name": "Proof", "params": {"format": "jpeg", "percent": 25}})).unwrap();
drop(s); // one session per library (issue #99)
let mut s = Session::new().with_fs();
s.open_library(&lib, false).unwrap();
assert_eq!(s.export_params(&json!({"preset": "Proof"})).unwrap(), json!({"format": "jpeg", "percent": 25}));
// older prefs.json files (snake_case `last_export`, no import section) still load
std::fs::write(lib.join("prefs.json"), r#"{"xmp": {"autoWrite": false}, "last_export": {"format": "png"}}"#).unwrap();
drop(s);
let mut s = Session::new().with_fs();
s.open_library(&lib, false).unwrap();
assert_eq!(s.last_export, Some(json!({"format": "png"})));
+97
View File
@@ -0,0 +1,97 @@
//! Issue #103: a damaged or unreadable settings file (prefs.json, presets.json, view.json) is
//! never silently replaced by the defaults: a damaged one is kept aside, an unreadable one is not
//! written this session, and the user is told.
use std::sync::{Arc, Mutex};
use lightcraft_catalog::{MemStore, Store};
use serde_json::json;
use crate::Session;
use crate::library::LibraryStores;
fn temp_dir(tag: &str) -> std::path::PathBuf {
let d = std::env::temp_dir().join(format!("lc-settings-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
#[test]
fn damaged_prefs_are_kept_aside_and_reported() {
let lib = temp_dir("damaged");
let bad = br#"{"exportPresets": [{"name": "Proof", "par"#;
std::fs::write(lib.join("prefs.json"), bad).unwrap();
let mut s = Session::new().with_fs();
s.open_library(&lib, false).unwrap();
let kept: Vec<_> = std::fs::read_dir(&lib)
.unwrap()
.map(|e| e.unwrap().file_name().to_string_lossy().to_string())
.filter(|n| n.starts_with("prefs.json.corrupt-"))
.collect();
assert_eq!(kept.len(), 1, "{kept:?}");
assert_eq!(std::fs::read(lib.join(&kept[0])).unwrap(), bad);
let warnings = s.take_library_warnings();
assert_eq!(warnings.len(), 1);
assert!(warnings[0].contains("prefs.json is damaged") && warnings[0].contains(&kept[0]), "{warnings:?}");
assert!(s.take_library_warnings().is_empty(), "each warning is handed out once");
let info = s.execute("library.info", &json!({})).unwrap();
assert_eq!(info["settingsWarnings"].as_array().map(Vec::len), Some(1), "{info}");
// saving goes on (the damaged original is safe in its copy)
s.execute("library.preferences", &json!({"cacheMb": 300})).unwrap();
assert!(String::from_utf8(std::fs::read(lib.join("prefs.json")).unwrap()).unwrap().contains("300"));
assert_eq!(std::fs::read(lib.join(&kept[0])).unwrap(), bad);
drop(s);
let _ = std::fs::remove_dir_all(&lib);
}
/// `presets.json` and `prefs.json` can't be read (locked by another program, an I/O error).
#[derive(Clone, Default)]
struct Unreadable {
files: MemStore,
unreadable: Arc<Mutex<Vec<String>>>,
}
impl Store for Unreadable {
fn read(&mut self, name: &str) -> std::io::Result<Option<Vec<u8>>> {
if self.unreadable.lock().unwrap().iter().any(|n| n == name) {
return Err(std::io::Error::other("locked by another process"));
}
self.files.read(name)
}
fn write_atomic(&mut self, name: &str, data: &[u8]) -> std::io::Result<()> {
self.files.write_atomic(name, data)
}
fn append(&mut self, name: &str, data: &[u8]) -> std::io::Result<()> {
self.files.append(name, data)
}
fn truncate(&mut self, name: &str, len: u64) -> std::io::Result<()> {
self.files.truncate(name, len)
}
fn describe(&self) -> String {
"unreadable".into()
}
}
#[test]
fn unreadable_settings_are_not_overwritten() {
let files = Unreadable::default();
let presets = br#"{"user": [{"id": "user.mine", "name": "Mine", "group": "User Presets", "settings": {}}]}"#.to_vec();
let prefs = br#"{"cacheMb": 777}"#.to_vec();
files.files.set("presets.json", presets.clone());
files.files.set("prefs.json", prefs.clone());
*files.unreadable.lock().unwrap() = vec!["presets.json".into(), "prefs.json".into()];
let mut s = Session::new();
let stores = LibraryStores { dir: "unreadable".into(), catalog: Box::new(MemStore::new()), files: Box::new(files.clone()), on_disk: false };
s.open_library_in(stores, true).unwrap();
let warnings = s.take_library_warnings();
assert_eq!(warnings.len(), 2, "{warnings:?}");
assert!(warnings.iter().all(|w| w.contains("couldn't be read") && w.contains("won't overwrite")), "{warnings:?}");
// a preset change and a preference change: neither file is replaced by defaults
s.execute("preset.create", &json!({"name": "New"})).unwrap();
let e = s.execute("library.preferences", &json!({"cacheMb": 300}));
assert!(e.is_err(), "saving preferences is refused, not done over the unread file");
assert_eq!(files.files.get("presets.json").unwrap(), presets);
assert_eq!(files.files.get("prefs.json").unwrap(), prefs);
}
+234
View File
@@ -0,0 +1,234 @@
//! Which wgpu backends LightCraft lets wgpu use — for its compute device and for the desktop
//! window (eframe) — and the crash sentinel around device creation (issue #136).
//!
//! wgpu loads the system driver of *every* backend in the instance's set while it enumerates
//! adapters, even when it then picks another one: with Vulkan in the set, a broken Vulkan driver
//! can crash the process (an access violation inside the driver DLL, which nothing can catch)
//! although DX12 would have been used. So Windows defaults to DX12 alone. macOS uses Metal, Linux
//! Vulkan (plus GL for the window when the build has it).
//!
//! Overrides, read once per process:
//! - `LIGHTCRAFT_GPU_BACKEND` = `dx12` | `vulkan` | `metal` | `gl` (comma lists allowed), `auto`
//! (the platform default), or `off` (no GPU compute; the window still needs one backend and
//! keeps the platform default);
//! - else `WGPU_BACKEND` (wgpu's own variable, same names) for both the window and compute.
//!
//! Backends this build doesn't contain (e.g. `gl`: the GL backend isn't compiled in) are ignored
//! with a warning and the platform default is used.
use std::path::PathBuf;
use std::sync::Mutex;
pub use wgpu::Backends;
/// What the environment asks for.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum BackendChoice {
/// The platform default.
Auto,
/// No GPU compute.
Off,
/// These backends.
Use(Backends),
}
/// Parse a backend list (`"dx12"`, `"vulkan,gl"`, `"auto"`, `"off"`…). `None`: nothing usable in
/// it (empty or unknown names).
pub fn parse_backends(s: &str) -> Option<BackendChoice> {
let s = s.trim().to_ascii_lowercase();
match s.as_str() {
"" => None,
"auto" | "default" => Some(BackendChoice::Auto),
"off" | "none" | "cpu" | "0" | "false" | "no" => Some(BackendChoice::Off),
_ => {
let b = Backends::from_comma_list(&s);
(!b.is_empty()).then_some(BackendChoice::Use(b))
}
}
}
/// The choice from `LIGHTCRAFT_GPU_BACKEND` (`lightcraft`) or else `WGPU_BACKEND` (`wgpu`): the
/// first one that parses wins; an unknown value is skipped (with a warning).
pub fn choose(lightcraft: Option<&str>, wgpu: Option<&str>) -> BackendChoice {
for (name, v) in [("LIGHTCRAFT_GPU_BACKEND", lightcraft), ("WGPU_BACKEND", wgpu)] {
let Some(v) = v else { continue };
match parse_backends(v) {
// WGPU_BACKEND is wgpu's variable: it selects backends, it never turns the GPU off
Some(BackendChoice::Off) if name == "WGPU_BACKEND" => log::warn!("gpu: WGPU_BACKEND={v} ignored (use LIGHTCRAFT_GPU_BACKEND=off)"),
Some(c) => return c,
None => log::warn!("gpu: {name}={v} names no known backend; ignored"),
}
}
BackendChoice::Auto
}
/// The backends compute devices use when nothing is asked for on `os` (`std::env::consts::OS`).
pub fn default_compute_backends(os: &str) -> Backends {
match os {
// never touch the Vulkan driver unless asked: see the module docs (issue #136)
"windows" => Backends::DX12,
"macos" | "ios" => Backends::METAL,
_ => Backends::PRIMARY,
}
}
/// The backends the window uses when nothing is asked for on `os`.
pub fn default_window_backends(os: &str) -> Backends {
match os {
"windows" => Backends::DX12,
"macos" | "ios" => Backends::METAL,
// eframe's default (GL only when the build has it)
_ => Backends::PRIMARY | Backends::GL,
}
}
/// Resolve `choice` against the backends this build contains (`compiled`): `None` = GPU off.
/// Asked-for backends that aren't compiled in fall back to `default`.
pub fn resolve(choice: BackendChoice, default: Backends, compiled: Backends) -> Option<Backends> {
match choice {
BackendChoice::Off => None,
BackendChoice::Auto => Some(default),
BackendChoice::Use(b) if b.intersects(compiled) => Some(b & compiled),
BackendChoice::Use(b) => {
log::warn!("gpu: backend {b:?} is not in this build ({compiled:?}); using {default:?}");
Some(default)
}
}
}
fn env_choice() -> BackendChoice {
static C: std::sync::OnceLock<BackendChoice> = std::sync::OnceLock::new();
*C.get_or_init(|| {
let lc = std::env::var("LIGHTCRAFT_GPU_BACKEND").ok();
let wg = std::env::var("WGPU_BACKEND").ok();
choose(lc.as_deref(), wg.as_deref())
})
}
/// The backends of the compute device (`None`: `LIGHTCRAFT_GPU_BACKEND=off`).
pub fn compute_backends() -> Option<Backends> {
resolve(env_choice(), default_compute_backends(std::env::consts::OS), wgpu::Instance::enabled_backend_features())
}
/// The backends for the desktop window's renderer (eframe/egui-wgpu). The window always needs
/// one: `off` keeps the platform default.
pub fn window_backends() -> Backends {
let default = default_window_backends(std::env::consts::OS);
let choice = match env_choice() {
BackendChoice::Off => BackendChoice::Auto,
c => c,
};
resolve(choice, default, wgpu::Instance::enabled_backend_features()).unwrap_or(default)
}
/// `LIGHTCRAFT_GPU_BACKEND=off`.
pub(crate) fn env_off() -> bool {
env_choice() == BackendChoice::Off
}
static MARKER: Mutex<Option<PathBuf>> = Mutex::new(None);
/// Write this file before the compute device is created and remove it once creation returned
/// (successfully or not). A file still there at the next launch means the process died inside
/// the driver: see [`take_init_marker`]. `None` (the default) writes nothing.
pub fn set_init_marker(path: Option<PathBuf>) {
*MARKER.lock().unwrap_or_else(|e| e.into_inner()) = path;
}
/// Run device creation `f` between writing and removing the init marker (if one is set).
pub(crate) fn with_init_marker<T>(backends: Backends, f: impl FnOnce() -> T) -> T {
let marker = MARKER.lock().unwrap_or_else(|e| e.into_inner()).clone();
if let Some(m) = &marker {
if let Some(d) = m.parent() {
let _ = std::fs::create_dir_all(d);
}
let secs = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0);
let _ = std::fs::write(m, format!("GPU device creation started (backends {backends:?}, unix time {secs})\n"));
}
let r = f();
if let Some(m) = &marker {
let _ = std::fs::remove_file(m);
}
r
}
/// If the init marker `path` is present — the last process died while creating the GPU device —
/// remove it and return what it recorded.
pub fn take_init_marker(path: &std::path::Path) -> Option<String> {
let text = std::fs::read_to_string(path).ok()?;
let _ = std::fs::remove_file(path);
Some(text.trim().to_string())
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::*;
#[test]
fn parses_backend_names() {
assert_eq!(parse_backends("dx12"), Some(BackendChoice::Use(Backends::DX12)));
assert_eq!(parse_backends(" D3D12 "), Some(BackendChoice::Use(Backends::DX12)));
assert_eq!(parse_backends("vulkan"), Some(BackendChoice::Use(Backends::VULKAN)));
assert_eq!(parse_backends("vk,gl"), Some(BackendChoice::Use(Backends::VULKAN | Backends::GL)));
assert_eq!(parse_backends("metal"), Some(BackendChoice::Use(Backends::METAL)));
assert_eq!(parse_backends("auto"), Some(BackendChoice::Auto));
for off in ["off", "OFF", "none", "cpu", "0"] {
assert_eq!(parse_backends(off), Some(BackendChoice::Off), "{off}");
}
assert_eq!(parse_backends(""), None);
assert_eq!(parse_backends("directx9"), None);
}
#[test]
fn lightcraft_variable_wins_over_wgpu_backend() {
assert_eq!(choose(None, None), BackendChoice::Auto);
assert_eq!(choose(None, Some("dx12")), BackendChoice::Use(Backends::DX12));
assert_eq!(choose(Some("vulkan"), Some("dx12")), BackendChoice::Use(Backends::VULKAN));
assert_eq!(choose(Some("off"), Some("dx12")), BackendChoice::Off);
// unknown LightCraft value: WGPU_BACKEND still applies
assert_eq!(choose(Some("bogus"), Some("gl")), BackendChoice::Use(Backends::GL));
// WGPU_BACKEND never turns the GPU off
assert_eq!(choose(None, Some("off")), BackendChoice::Auto);
}
/// Issue #136: Windows must not load the Vulkan driver unless asked to.
#[test]
fn windows_defaults_to_dx12_only() {
assert_eq!(default_compute_backends("windows"), Backends::DX12);
assert_eq!(default_window_backends("windows"), Backends::DX12);
assert!(!default_window_backends("windows").contains(Backends::VULKAN));
assert_eq!(default_compute_backends("macos"), Backends::METAL);
assert!(default_compute_backends("linux").contains(Backends::VULKAN));
assert!(default_window_backends("linux").contains(Backends::VULKAN | Backends::GL));
}
#[test]
fn resolves_against_the_build() {
let compiled = Backends::DX12 | Backends::VULKAN;
let def = Backends::DX12;
assert_eq!(resolve(BackendChoice::Auto, def, compiled), Some(def));
assert_eq!(resolve(BackendChoice::Off, def, compiled), None);
assert_eq!(resolve(BackendChoice::Use(Backends::VULKAN), def, compiled), Some(Backends::VULKAN));
assert_eq!(resolve(BackendChoice::Use(Backends::VULKAN | Backends::GL), def, compiled), Some(Backends::VULKAN));
// GL isn't compiled in: the default, not an empty set (no adapter at all)
assert_eq!(resolve(BackendChoice::Use(Backends::GL), def, compiled), Some(def));
}
#[test]
fn init_marker_is_written_during_creation_and_removed_after() {
let dir = std::env::temp_dir().join(format!("lc-gpu-marker-{}", std::process::id()));
let m = dir.join("gpu-init.marker");
let _ = std::fs::remove_dir_all(&dir);
set_init_marker(Some(m.clone()));
let seen = with_init_marker(Backends::DX12, || m.exists());
set_init_marker(None);
assert!(seen, "the marker exists while the device is created");
assert!(!m.exists(), "and is removed afterwards");
// a marker left behind by a crashed process is reported once
std::fs::write(&m, "GPU device creation started (backends DX12)").unwrap();
assert!(take_init_marker(&m).unwrap().contains("DX12"));
assert_eq!(take_init_marker(&m), None);
let _ = std::fs::remove_dir_all(&dir);
}
}
+4 -3
View File
@@ -276,14 +276,15 @@ fn module_source(m: &Module, consts: &str) -> String {
impl Gpu {
/// Create a device on the best available adapter (no software fallback), or why there is none.
pub fn new() -> Result<Gpu, String> {
pub fn new(backends: wgpu::Backends) -> Result<Gpu, String> {
let mut desc = wgpu::InstanceDescriptor::new_without_display_handle();
desc.backends = wgpu::Backends::PRIMARY;
// only these drivers are loaded (`crate::backend`: never Vulkan on Windows unless asked)
desc.backends = backends;
let instance = wgpu::Instance::new(desc);
let adapter = pollster::block_on(
instance.request_adapter(&wgpu::RequestAdapterOptions { power_preference: wgpu::PowerPreference::HighPerformance, ..Default::default() }),
)
.map_err(|e| format!("no GPU adapter ({e})"))?;
.map_err(|e| format!("no GPU adapter among {backends:?} ({e})"))?;
let info = adapter.get_info();
if info.device_type == wgpu::DeviceType::Cpu {
log::info!("gpu: only a software adapter ({}), using the CPU pipeline", info.name);
+33 -9
View File
@@ -6,8 +6,11 @@
//! same settings on both and bound the difference in 8-bit sRGB. Stages without a kernel run on the
//! CPU inside the same render (per-stage hybrid); see `docs/gpu-pipeline.md`.
//!
//! Use [`render`]: it returns `None` when the GPU is unavailable, disabled (`LIGHTCRAFT_GPU=0` or
//! [`set_enabled`]), the render does not fit the device, or the device reported an error, ran out
//! Which backends wgpu may load (DX12 only on Windows; `LIGHTCRAFT_GPU_BACKEND`, `WGPU_BACKEND`) and
//! the crash sentinel around device creation: [`backend`] (issue #136).
//!
//! Use [`render`]: it returns `None` when the GPU is unavailable, disabled (`LIGHTCRAFT_GPU=0`,
//! `LIGHTCRAFT_GPU_BACKEND=off` or [`set_enabled`]), the render does not fit the device, or the device reported an error, ran out
//! of memory or returned an incomplete image — callers then render on the CPU. [`unavailable_reason`]
//! and [`last_fallback`] say why (`ui.inspect` → `perf.gpuReason` / `perf.gpuFallback`).
//! The browser build has no GPU path yet (WebGPU device creation is asynchronous): everything here
@@ -22,6 +25,8 @@ use lightcraft_develop::DevelopSettings;
use lightcraft_pipeline::{RenderRequest, Rendered, SourceInfo, StageCache};
use lightcraft_raster::Rgb32f;
#[cfg(not(target_arch = "wasm32"))]
pub mod backend;
#[cfg(not(target_arch = "wasm32"))]
mod ctx;
#[cfg(not(target_arch = "wasm32"))]
@@ -61,6 +66,10 @@ fn record_fallback(reason: String) {
/// Never blocks.
pub fn unavailable_reason() -> Option<String> {
if env_disabled() {
#[cfg(not(target_arch = "wasm32"))]
if backend::env_off() {
return Some("disabled by LIGHTCRAFT_GPU_BACKEND=off".into());
}
return Some("disabled by LIGHTCRAFT_GPU=0".into());
}
if !ENABLED.load(Ordering::Relaxed) {
@@ -136,9 +145,15 @@ pub fn enabled() -> bool {
ENABLED.load(Ordering::Relaxed) && !BROKEN.load(Ordering::Relaxed) && !env_disabled()
}
/// `LIGHTCRAFT_GPU=0` (or `LIGHTCRAFT_GPU_BACKEND=off`): no GPU for the whole process.
fn env_disabled() -> bool {
static OFF: std::sync::OnceLock<bool> = std::sync::OnceLock::new();
*OFF.get_or_init(|| std::env::var("LIGHTCRAFT_GPU").is_ok_and(|v| matches!(v.trim(), "0" | "off" | "false" | "no")))
*OFF.get_or_init(|| {
let off = std::env::var("LIGHTCRAFT_GPU").is_ok_and(|v| matches!(v.trim(), "0" | "off" | "false" | "no"));
#[cfg(not(target_arch = "wasm32"))]
let off = off || backend::env_off();
off
})
}
#[cfg(not(target_arch = "wasm32"))]
@@ -146,10 +161,18 @@ static GPU: std::sync::OnceLock<Result<ctx::Gpu, String>> = std::sync::OnceLock:
#[cfg(not(target_arch = "wasm32"))]
pub(crate) fn device() -> Option<&'static ctx::Gpu> {
if env_disabled() {
return None;
// turned off (preference, environment): don't even load the driver (issue #136)
if env_disabled() || !ENABLED.load(Ordering::Relaxed) {
return existing_device();
}
GPU.get_or_init(|| std::panic::catch_unwind(ctx::Gpu::new).unwrap_or_else(|_| Err("device creation panicked".into()))).as_ref().ok()
GPU.get_or_init(|| {
let Some(backends) = backend::compute_backends() else { return Err("disabled by LIGHTCRAFT_GPU_BACKEND=off".into()) };
backend::with_init_marker(backends, || {
std::panic::catch_unwind(|| ctx::Gpu::new(backends)).unwrap_or_else(|_| Err("device creation panicked".into()))
})
})
.as_ref()
.ok()
}
/// The device if it has been created (never creates it).
@@ -158,11 +181,12 @@ pub(crate) fn existing_device() -> Option<&'static ctx::Gpu> {
GPU.get().and_then(|g| g.as_ref().ok())
}
/// Create the device and compile the kernels on a background thread now (app start), so the
/// first render doesn't wait ~0.3–0.4 s for it and the UI thread never does.
/// Create the device and compile the kernels on a background thread now (once the window is up),
/// so the first render doesn't wait ~0.3–0.4 s for it and the UI thread never does. Does nothing
/// while GPU rendering is turned off.
pub fn warm_up() {
#[cfg(not(target_arch = "wasm32"))]
if !env_disabled() && GPU.get().is_none() {
if !env_disabled() && ENABLED.load(Ordering::Relaxed) && GPU.get().is_none() {
let _ = std::thread::Builder::new().name("lc-gpu-init".into()).spawn(|| {
let _ = device();
});
+21
View File
@@ -22,6 +22,27 @@ fn gpu() -> bool {
ok
}
#[test]
fn camera_tone_and_relative_wb() {
if !gpu() {
return;
}
let src = scene(2, 320, 240);
let curve = lightcraft_pipeline::tone::CameraTone::new(std::array::from_fn(|i| {
let x = 0.004 * 1.18f32.powi(i as i32);
[x, 1.0 - (-2.0 * x).exp()]
}))
.unwrap();
let info = SourceInfo { raw: true, relative_wb: true, camera_tone: Some(curve), ..Default::default() };
let mut s = DevelopSettings::default();
check("camera tone neutral", &src, &info, &s, &RenderRequest::fit(320, 240));
s.light.exposure = 1.0;
s.light.contrast = 40.0;
s.wb.mode = WbMode::Custom;
s.wb.temp = 8000.0;
check("camera tone edited", &src, &info, &s, &RenderRequest::fit(320, 240));
}
fn scene(i: usize, w: usize, h: usize) -> Arc<Rgb32f> {
Arc::new(lightcraft_scenes::demo_library()[i].render(w, h))
}
+1
View File
@@ -45,6 +45,7 @@ pub fn color_data(color: &FrameColor) -> ColorData {
as_shot_neutral: Some([1.0, 1.0, 1.0]),
as_shot_white_xy: None,
baseline_exposure: 0.0,
profile: Default::default(),
}
}
}
+3 -1
View File
@@ -221,7 +221,9 @@ impl FinishParams {
FinishParams {
calib: if calibration { crate::colorops::calibration_matrix(&s.calibration) } else { None },
shadow_tint: if calibration { (s.calibration.shadows_tint / 100.0) as f32 } else { 0.0 },
tone: if info.raw {
tone: if let Some(curve) = info.camera_tone.as_ref().filter(|_| info.raw) {
ToneMap::camera(curve, s.light.contrast, s.light.whites, s.light.blacks)
} else if info.raw {
ToneMap::new(s.light.contrast, s.light.whites, s.light.blacks)
} else {
ToneMap::display(s.light.contrast, s.light.whites, s.light.blacks)
+4 -1
View File
@@ -64,11 +64,14 @@ pub struct SourceInfo {
pub raw: bool,
pub as_shot_temp: f64,
pub as_shot_tint: f64,
/// No measured camera illuminant: WB adjustments are relative to the camera's rendered look.
pub relative_wb: bool,
pub camera_tone: Option<tone::CameraTone>,
}
impl Default for SourceInfo {
fn default() -> Self {
Self { raw: false, as_shot_temp: 6500.0, as_shot_tint: 0.0, lens: None }
Self { raw: false, as_shot_temp: 6500.0, as_shot_tint: 0.0, lens: None, relative_wb: false, camera_tone: None }
}
}
+1 -1
View File
@@ -61,7 +61,7 @@ pub fn effective_wb(info: &SourceInfo, s: &DevelopSettings) -> (f64, f64) {
use lightcraft_develop::WbMode;
match s.wb.mode {
WbMode::AsShot => (info.as_shot_temp, info.as_shot_tint),
m if info.raw => m.preset().unwrap_or((s.wb.temp, s.wb.tint)),
m if info.raw && !info.relative_wb => m.preset().unwrap_or((s.wb.temp, s.wb.tint)),
_ => (s.wb.temp, s.wb.tint),
}
}
+89
View File
@@ -14,12 +14,75 @@ pub const LUT_MIN_EV: f32 = -14.0;
pub const LUT_MAX_EV: f32 = 10.0;
pub const LUT_N: usize = 4096;
/// A file-local camera look, fitted independently of the scene-linear colour transform.
/// Knots are scene/display-linear luminance pairs. Keeping this in the finish stage preserves
/// RAW exposure and highlight headroom; it is never baked into the decoded sensor pixels.
#[derive(Clone, Copy, Debug, PartialEq, serde::Serialize)]
pub struct CameraTone {
knots: [[f32; 2]; 32],
}
impl<'de> serde::Deserialize<'de> for CameraTone {
fn deserialize<D: serde::Deserializer<'de>>(d: D) -> Result<Self, D::Error> {
#[derive(serde::Deserialize)]
struct Wire {
knots: [[f32; 2]; 32],
}
let w = Wire::deserialize(d)?;
Self::new(w.knots).ok_or_else(|| serde::de::Error::custom("invalid camera tone curve"))
}
}
impl CameraTone {
pub fn new(knots: [[f32; 2]; 32]) -> Option<Self> {
let mut previous = [0.0, 0.0];
for p in knots {
if !p.iter().all(|v| v.is_finite()) || p[0] <= previous[0] || p[1] < previous[1] || p[1] >= 1.0 {
return None;
}
previous = p;
}
Some(Self { knots })
}
pub fn apply(&self, y: f32) -> f32 {
if !y.is_finite() || y <= 0.0 {
return 0.0;
}
let mut previous = [0.0, 0.0];
for p in self.knots {
if y <= p[0] {
let t = (y - previous[0]) / (p[0] - previous[0]);
return previous[1] + t * (p[1] - previous[1]);
}
previous = p;
}
let a = self.knots[30];
let b = self.knots[31];
// Extend beyond observed (unclipped) highlights with a continuous, bounded shoulder.
let slope = ((b[1] - a[1]) / (b[0] - a[0])).clamp(0.1, 16.0);
1.0 - (1.0 - b[1]) * (-(y - b[0]) * slope / (1.0 - b[1]).max(0.01)).exp()
}
}
#[derive(Clone, Debug)]
pub struct ToneMap {
lut: Vec<f32>,
}
impl ToneMap {
pub fn camera(curve: &CameraTone, contrast: f64, whites: f64, blacks: f64) -> ToneMap {
let adjustment = Self::display(contrast, whites, blacks);
let neutral = contrast == 0.0 && whites == 0.0 && blacks == 0.0;
let lut = (0..LUT_N)
.map(|i| {
let ev = LUT_MIN_EV + (LUT_MAX_EV - LUT_MIN_EV) * i as f32 / (LUT_N - 1) as f32;
let y = curve.apply(GREY * 2f32.powf(ev));
if neutral { y } else { adjustment.apply(y) }
})
.collect();
ToneMap { lut }
}
/// `contrast`, `whites`, `blacks` in −100..100 (Lightroom slider units).
pub fn new(contrast: f64, whites: f64, blacks: f64) -> ToneMap {
let c = (contrast / 100.0) as f32;
@@ -116,6 +179,32 @@ fn smooth(e0: f32, e1: f32, x: f32) -> f32 {
mod tests {
use super::*;
#[test]
fn camera_curve_preserves_black_and_extends_headroom() {
let knots = std::array::from_fn(|i| {
let x = 0.005 * 1.15f32.powi(i as i32);
[x, 1.0 - (-3.0 * x).exp()]
});
let curve = CameraTone::new(knots).unwrap();
let base = ToneMap::camera(&curve, 0.0, 0.0, 0.0);
assert_eq!(base.apply(0.0), 0.0);
assert!(base.apply(0.1) < base.apply(0.2));
let mut previous = 0.0;
for i in 0..2000 {
let y = 1e-6 * 1.01f32.powi(i);
let v = base.apply(y);
assert!((0.0..=1.0).contains(&v));
assert!(v >= previous - 1e-6);
previous = v;
}
assert!((base.apply(0.1) - curve.apply(0.1)).abs() < 0.001);
assert!(ToneMap::camera(&curve, 50.0, 0.0, 0.0).apply(0.3) > base.apply(0.3));
let mut invalid = knots;
invalid[1][0] = invalid[0][0];
assert!(CameraTone::new(invalid).is_none());
assert!(serde_json::from_value::<CameraTone>(serde_json::json!({"knots": invalid})).is_err());
}
#[test]
fn monotone_and_bounded() {
for (c, w, b) in [(0.0, 0.0, 0.0), (100.0, 100.0, -100.0), (-100.0, -100.0, 100.0), (50.0, -30.0, -40.0)] {
+46
View File
@@ -0,0 +1,46 @@
//! TEMPORARY analysis helper (not committed): decode every tile of a subsampled lossless ARW into full planes.
//! Output: per plane `u32 w, u32 h` + `u16` samples, planes concatenated.
use lightcraft_tiff::Tiff;
use lightcraft_tiff::image::chunk_bytes;
fn main() {
let a: Vec<String> = std::env::args().skip(1).collect();
let bytes = std::fs::read(&a[0]).unwrap();
let tiff = Tiff::parse(&bytes).unwrap();
let raw = tiff.all_ifds().into_iter().max_by_key(|i| i.u64(256).unwrap_or(0) * i.u64(257).unwrap_or(0)).unwrap();
let info = raw.image().unwrap();
let (w, h) = (info.width as usize, info.height as usize);
let chunks = info.chunks(bytes.len() as u64);
let mut planes: Vec<(usize, usize, Vec<u16>)> = Vec::new();
for c in &chunks {
let src = chunk_bytes(&bytes, c).unwrap();
let f = lightcraft_raw::ljpeg::decode_planar(src, 1 << 24).unwrap();
if planes.is_empty() {
for p in &f.planes {
let (sx, sy) = (f.width / p.width, f.height / p.height);
planes.push((w / sx, h / sy, vec![0; (w / sx) * (h / sy)]));
}
}
for (k, p) in f.planes.iter().enumerate() {
let (sx, sy) = (f.width / p.width, f.height / p.height);
let (pw, ph, ref mut d) = planes[k];
let (x0, y0) = (c.x as usize / sx, c.y as usize / sy);
for y in 0..p.height {
for x in 0..p.width {
if x0 + x < pw && y0 + y < ph {
d[(y0 + y) * pw + x0 + x] = p.data[y * p.width + x];
}
}
}
}
}
let mut out = Vec::new();
for (pw, ph, d) in &planes {
out.extend_from_slice(&(*pw as u32).to_le_bytes());
out.extend_from_slice(&(*ph as u32).to_le_bytes());
for v in d {
out.extend_from_slice(&v.to_le_bytes());
}
}
std::fs::write(&a[1], out).unwrap();
}
+22
View File
@@ -1,6 +1,7 @@
//! DNG (Adobe Digital Negative Specification 1.7): raw IFD selection, pixel data (via [`crate::tiffraw`]),
//! linearization, black/white levels, active area, default crop, CFA description, colour tags, opcode lists.
use crate::profile::{HsvTable, ProfileLook, ToneCurve};
use crate::tiffraw::{Packing, read_image_in};
use crate::{BlackLevel, Cfa, ColorData, Mat3, Mode, RawData, RawError, RawFormat, RawImage, Rect, Result, opcodes};
use lightcraft_color::Xy;
@@ -45,6 +46,27 @@ pub(crate) fn color_data(ifd0: &Ifd, raw: &Ifd) -> ColorData {
as_shot_white_xy: get(t::AS_SHOT_WHITE_XY).filter(|v| v.len() == 2 && v[0] > 0.0 && v[1] > 0.0).map(|v| Xy::new(v[0], v[1])),
baseline_exposure: get(t::BASELINE_EXPOSURE).and_then(|v| v.first().copied()).filter(|v| v.is_finite()).unwrap_or(0.0)
+ get(t::BASELINE_EXPOSURE_OFFSET).and_then(|v| v.first().copied()).filter(|v| v.is_finite()).unwrap_or(0.0),
profile: profile_look(ifd0, raw),
}
}
/// The profile look tags (`ProfileHueSatMap*`, `ProfileLookTable*`, `ProfileToneCurve`); malformed
/// ones are ignored. Like the colour tags, read from the raw IFD first, else IFD 0.
pub(crate) fn profile_look(ifd0: &Ifd, raw: &Ifd) -> ProfileLook {
let pick = |tag: u16| if raw.contains(tag) { raw } else { ifd0 };
let table = |dims: u16, data: u16, enc: u16| {
let ifd = pick(dims);
let dims = ifd.u64s(dims)?;
let data = ifd.f64s(data)?;
HsvTable::from_tags(&dims, &data, ifd.u64s(enc).and_then(|v| v.first().copied()).unwrap_or(0))
};
ProfileLook {
hue_sat_map: [
table(t::PROFILE_HUE_SAT_MAP_DIMS, t::PROFILE_HUE_SAT_MAP_DATA_1, t::PROFILE_HUE_SAT_MAP_ENCODING),
table(t::PROFILE_HUE_SAT_MAP_DIMS, t::PROFILE_HUE_SAT_MAP_DATA_2, t::PROFILE_HUE_SAT_MAP_ENCODING),
],
look_table: table(t::PROFILE_LOOK_TABLE_DIMS, t::PROFILE_LOOK_TABLE_DATA, t::PROFILE_LOOK_TABLE_ENCODING),
tone_curve: pick(t::PROFILE_TONE_CURVE).f64s(t::PROFILE_TONE_CURVE).and_then(|v| ToneCurve::from_tag(&v)),
}
}
+32
View File
@@ -179,6 +179,38 @@ pub fn write_dng(raw: &RawImage, opts: &DngWriteOptions) -> Result<Vec<u8>> {
ifd.set(t::AS_SHOT_WHITE_XY, rat_vec(&[xy.x, xy.y]));
}
ifd.set(t::BASELINE_EXPOSURE, Value::SRational(vec![srational(c.baseline_exposure)]));
// the source's own profile look travels with its data (conversions, smart previews, merges)
let p = &c.profile;
let table_tags = |ifd: &mut IfdBuilder, table: &crate::profile::HsvTable, dims: u16, enc: u16| {
ifd.set(dims, Value::Long(vec![table.hue_divisions as u32, table.sat_divisions as u32, table.val_divisions as u32]));
if table.srgb_value {
ifd.set(enc, Value::Long(vec![1]));
}
};
let flat = |table: &crate::profile::HsvTable| Value::Float(table.data.iter().flatten().copied().collect());
match &p.hue_sat_map {
[Some(a), b] => {
table_tags(&mut ifd, a, t::PROFILE_HUE_SAT_MAP_DIMS, t::PROFILE_HUE_SAT_MAP_ENCODING);
ifd.set(t::PROFILE_HUE_SAT_MAP_DATA_1, flat(a));
if let Some(b) =
b.as_ref().filter(|b| (b.hue_divisions, b.sat_divisions, b.val_divisions) == (a.hue_divisions, a.sat_divisions, a.val_divisions))
{
ifd.set(t::PROFILE_HUE_SAT_MAP_DATA_2, flat(b));
}
}
[None, Some(b)] => {
table_tags(&mut ifd, b, t::PROFILE_HUE_SAT_MAP_DIMS, t::PROFILE_HUE_SAT_MAP_ENCODING);
ifd.set(t::PROFILE_HUE_SAT_MAP_DATA_1, flat(b));
}
[None, None] => {}
}
if let Some(l) = &p.look_table {
table_tags(&mut ifd, l, t::PROFILE_LOOK_TABLE_DIMS, t::PROFILE_LOOK_TABLE_ENCODING);
ifd.set(t::PROFILE_LOOK_TABLE_DATA, flat(l));
}
if let Some(curve) = &p.tone_curve {
ifd.set(t::PROFILE_TONE_CURVE, Value::Float(curve.points.iter().flatten().copied().collect()));
}
for (list, tag) in [(&raw.opcodes.list1, t::OPCODE_LIST_1), (&raw.opcodes.list2, t::OPCODE_LIST_2), (&raw.opcodes.list3, t::OPCODE_LIST_3)] {
if !list.is_empty() {
ifd.set(tag, Value::Undefined(opcodes::write_list(list)));
+7 -1
View File
@@ -8,7 +8,8 @@
//! [`RawImage::develop`] does all of it plus `OpcodeList3` and the default crop; [`RawImage::develop_binned`]
//! produces the same at 1/k of the size straight from the mosaic (previews, thumbnails).
//! - [`color`] implements the DNG colour model (dual-illuminant interpolation, forward matrices, white balance)
//! and produces camera → linear Rec.2020 D65 matrices.
//! and produces camera → linear Rec.2020 D65 matrices; [`profile`] reads and applies a DNG's own profile
//! look tables and tone curve.
//!
//! Formats: DNG (uncompressed, lossless JPEG, lossy JPEG (Smart Previews), Deflate incl. floating point, tiled/stripped, CFA and LinearRaw),
//! Canon CR2, Nikon NEF/NRW (uncompressed, Huffman lossless / lossy compressed), Sony ARW (uncompressed, ARW2, lossless), Fujifilm RAF (uncompressed Bayer
@@ -29,6 +30,7 @@ pub mod highlight;
pub mod ljpeg;
pub mod opcodes;
mod preview;
pub mod profile;
mod tiffraw;
mod unpack;
mod vendor;
@@ -393,6 +395,10 @@ pub struct ColorData {
pub as_shot_white_xy: Option<Xy>,
/// EV to add for a "normal" rendering (`BaselineExposure` + `BaselineExposureOffset`).
pub baseline_exposure: f64,
/// The file's own camera-profile look (`ProfileHueSatMap*`, `ProfileLookTable*`,
/// `ProfileToneCurve`), applied by [`color`]'s users at render time.
#[serde(default)]
pub profile: profile::ProfileLook,
}
/// A decoded raw image.
+406 -6
View File
@@ -1,9 +1,11 @@
//! Lossless JPEG ("LJ92"): ITU-T T.81 / ISO 10918-1 process 14 (Annex H), Huffman-coded, as used by DNG,
//! Canon CR2 and Sony lossless ARW.
//!
//! - All seven predictors, 2–16 bit precision, 1–4 interleaved components (1×1 sampling), point transform,
//! restart intervals, `SSSS = 16` differences.
//! - [`decode`] returns the frame as interleaved samples in raster order (`width × height × components`),
//! - All seven predictors, 2–16 bit precision, 1–4 interleaved components, point transform, restart intervals,
//! `SSSS = 16` differences.
//! - Subsampled components (sampling factors other than 1×1, e.g. Sony's M/S-size lossless ARW with 4:2:0 /
//! 4:2:2 YCbCr): [`decode_planar`] returns one plane per component.
//! - [`decode`] returns a 1×1-sampled frame as interleaved samples in raster order (`width × height × components`),
//! which callers re-tile (DNG tiles, CR2 slices).
//! - [`encode`] writes a conforming stream with per-component optimal Huffman tables built with the
//! T.81 Annex K.2 procedure — used by tests (bit-exact round trips) and later by DNG export.
@@ -207,6 +209,8 @@ struct Header {
height: usize,
width: usize,
comps: Vec<u8>,
/// Horizontal and vertical sampling factors per component (T.81 `Hi`, `Vi`).
sampling: Vec<(usize, usize)>,
tables: [Option<Huffman>; 4],
table_for: Vec<usize>,
predictor: u8,
@@ -228,6 +232,7 @@ fn parse_header(d: &[u8]) -> Result<Header, RawError> {
height: 0,
width: 0,
comps: vec![],
sampling: vec![],
tables: [None, None, None, None],
table_for: vec![],
predictor: 1,
@@ -272,10 +277,16 @@ fn parse_header(d: &[u8]) -> Result<Header, RawError> {
}
for c in 0..n {
let (id, samp) = (seg[6 + 3 * c], seg[7 + 3 * c]);
if samp != 0x11 {
return Err(RawError::Unsupported("lossless JPEG with subsampled components".into()));
let (hs, vs) = ((samp >> 4) as usize, (samp & 15) as usize);
if !(1..=4).contains(&hs) || !(1..=4).contains(&vs) {
return Err(err("bad sampling factor"));
}
h.comps.push(id);
h.sampling.push((hs, vs));
}
// T.81 B.2.2: at most 10 data units per MCU
if h.sampling.iter().map(|(a, b)| a * b).sum::<usize>() > 10 {
return Err(err("too many samples per MCU"));
}
if !(2..=16).contains(&h.precision) {
return Err(err("bad precision"));
@@ -343,9 +354,18 @@ pub fn frame_info(d: &[u8]) -> Result<(usize, usize, usize, u8), RawError> {
Ok((h.width, h.height, h.comps.len(), h.precision))
}
/// Decode a lossless JPEG stream. `max_samples` bounds the allocation (hostile headers).
/// The components' sampling factors `(H, V)` from the frame header (all `(1, 1)` unless subsampled).
pub fn sampling_factors(d: &[u8]) -> Result<Vec<(usize, usize)>, RawError> {
Ok(parse_header(d)?.sampling)
}
/// Decode a lossless JPEG stream. `max_samples` bounds the allocation (hostile headers). Subsampled frames
/// (sampling factors other than 1×1) are decoded by [`decode_planar`].
pub fn decode(d: &[u8], max_samples: usize) -> Result<Frame, RawError> {
let h = parse_header(d)?;
if h.sampling.iter().any(|&s| s != (1, 1)) {
return Err(RawError::Unsupported("lossless JPEG with subsampled components (planar decode only)".into()));
}
let (w, ht, nc) = (h.width, h.height, h.comps.len());
if w == 0 || ht == 0 {
return Err(err("zero frame size"));
@@ -430,6 +450,163 @@ pub fn decode(d: &[u8], max_samples: usize) -> Result<Frame, RawError> {
Ok(Frame { width: w, height: ht, components: nc, precision: h.precision, predictor: pred, point_transform: h.pt, data: out })
}
/// One component of a [`PlanarFrame`].
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Plane {
/// Sampling factors `(H, V)`.
pub sampling: (usize, usize),
/// Size in samples: `⌈X · H / Hmax⌉ × ⌈Y · V / Vmax⌉` (T.81 A.1.1).
pub width: usize,
pub height: usize,
/// `width × height` samples in raster order.
pub data: Vec<u16>,
}
/// A decoded lossless-JPEG frame with one plane per component, for frames with subsampled components.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct PlanarFrame {
/// Frame size `X × Y` (the size of a component with the largest sampling factors).
pub width: usize,
pub height: usize,
pub precision: u8,
pub predictor: u8,
pub point_transform: u8,
pub planes: Vec<Plane>,
}
/// Decode a lossless JPEG stream into one plane per component, honouring sampling factors (ITU-T T.81 A.1.1,
/// A.2.3 and Annex H): the scan is a sequence of MCUs, each holding `Hi × Vi` samples of component `i` (in raster
/// order within the MCU) for every component in turn. Each sample is predicted from its neighbours in its own
/// component (H.1.2.1): the first sample of the scan or of a restart interval from `2^(P − Pt − 1)`, the rest of
/// that component line from the left neighbour, all samples not in the first column with the scan's predictor.
///
/// First column, with vertical sampling `V > 1`: the first sample of each *MCU row* is predicted from the first
/// sample of the previous MCU row (`V` lines up), the first samples of the other lines of the MCU row from the
/// line above. T.81 reads as "the line above" for every line; the rule used here was established black-box on
/// Sony's 4:2:0 lossless ARWs (ILCE-7M4 M-size), where it is the only candidate that decodes without row-wide
/// steps (mean absolute step between adjacent row means 10.6 vs 45.6 for "the line above" on a 512-line tile,
/// and no wrapped samples, against 17 % wrapped over the whole image). With `V = 1` both readings agree.
///
/// Restart intervals must span whole MCU rows. Works for 1×1 sampling too (then the planes are the
/// de-interleaved [`decode`] output). `max_samples` bounds the allocation.
pub fn decode_planar(d: &[u8], max_samples: usize) -> Result<PlanarFrame, RawError> {
let h = parse_header(d)?;
let (w, ht, nc) = (h.width, h.height, h.comps.len());
if w == 0 || ht == 0 {
return Err(err("zero frame size"));
}
let hmax = h.sampling.iter().map(|s| s.0).max().unwrap_or(1);
let vmax = h.sampling.iter().map(|s| s.1).max().unwrap_or(1);
let (mcux, mcuy) = (w.div_ceil(hmax), ht.div_ceil(vmax));
// planes are decoded at whole-MCU size and trimmed afterwards
let padded: Vec<(usize, usize)> = h.sampling.iter().map(|&(hs, vs)| (mcux * hs, mcuy * vs)).collect();
let total = padded
.iter()
.try_fold(0usize, |acc, &(pw, ph)| pw.checked_mul(ph).and_then(|n| acc.checked_add(n)))
.ok_or_else(|| err("frame too large"))?;
if total > max_samples {
return Err(RawError::Limit("lossless JPEG frame larger than expected"));
}
let entropy = &d[h.scan_start..];
if (entropy.len() as u64 + 64) * 8 < total as u64 {
return Err(err("entropy data too short for frame"));
}
if h.restart > 0 && h.restart % mcux != 0 {
return Err(RawError::Unsupported("lossless JPEG restart interval inside an MCU row of a subsampled frame".into()));
}
let tables: Vec<&Huffman> = h
.table_for
.iter()
.map(|&t| h.tables.get(t).and_then(Option::as_ref).ok_or_else(|| err("scan uses an undefined Huffman table")))
.collect::<Result<_, RawError>>()?;
let mut planes: Vec<Vec<u16>> = padded.iter().map(|&(pw, ph)| vec![0u16; pw * ph]).collect();
let mut br = BitReader::new(entropy);
let init = 1i32 << (h.precision - h.pt - 1);
let pred = h.predictor;
let mut restart_left = h.restart;
// MCU row where the scan / current restart interval started: its first line per component is the "first line"
let mut interval_row = 0usize;
let mut reset_pending = vec![true; nc];
for my in 0..mcuy {
for mx in 0..mcux {
if h.restart > 0 {
if restart_left == 0 {
if !br.restart() {
return Err(err("missing restart marker"));
}
restart_left = h.restart;
interval_row = my;
reset_pending.iter_mut().for_each(|r| *r = true);
}
restart_left -= 1;
}
for c in 0..nc {
let (hs, vs) = h.sampling[c];
let pw = padded[c].0;
let plane = &mut planes[c];
let first_line = interval_row * vs;
for by in 0..vs {
let y = my * vs + by;
for bx in 0..hs {
let x = mx * hs + bx;
let i = y * pw + x;
let p = if reset_pending[c] {
reset_pending[c] = false;
init
} else if y == first_line {
// x > 0 here: the line's first sample was the reset sample
plane[i - 1] as i32
} else if pred == 0 {
0
} else if x == 0 {
// the first sample of an MCU row: from the first sample of the previous MCU row
// (see the function docs); the first sample of any other line: from the one above
if by == 0 { plane[i - vs * pw] as i32 } else { plane[i - pw] as i32 }
} else {
let ra = plane[i - 1] as i32;
let rb = plane[i - pw] as i32;
let rc = plane[i - pw - 1] as i32;
match pred {
1 => ra,
2 => rb,
3 => rc,
4 => ra + rb - rc,
5 => ra + ((rb - rc) >> 1),
6 => rb + ((ra - rc) >> 1),
_ => (ra + rb) >> 1,
}
};
let ssss = tables[c].decode(&mut br)?;
let diff = diff_value(&mut br, ssss);
plane[i] = ((p + diff) & 0xffff) as u16;
}
}
}
}
if br.overrun > 64 * 8 {
return Err(err("entropy data exhausted"));
}
}
let planes = planes
.into_iter()
.zip(&h.sampling)
.zip(&padded)
.map(|((mut data, &(hs, vs)), &(pw, _))| {
let (cw, ch) = ((w * hs).div_ceil(hmax), (ht * vs).div_ceil(vmax));
if cw != pw {
data = data.chunks_exact(pw).take(ch).flat_map(|row| row[..cw].iter().copied()).collect();
} else {
data.truncate(cw * ch);
}
if h.pt > 0 {
data.iter_mut().for_each(|v| *v <<= h.pt);
}
Plane { sampling: (hs, vs), width: cw, height: ch, data }
})
.collect();
Ok(PlanarFrame { width: w, height: ht, precision: h.precision, predictor: pred, point_transform: h.pt, planes })
}
// ------------------------------------------------------------------------------------------------ encoder
/// Build Huffman code lengths from frequencies (T.81 Annex K.2, Figures K.1–K.3), limited to 16 bits.
@@ -700,6 +877,143 @@ pub fn encode(data: &[u16], width: usize, height: usize, components: usize, prec
out
}
/// Test encoder for subsampled frames: `planes[c]` is component `c` at `⌈w · H / Hmax⌉ × ⌈h · V / Vmax⌉` samples,
/// coded in interleaved MCUs with the prediction rules of [`decode_planar`] (one Huffman table per component).
#[cfg(test)]
pub(crate) fn encode_planar(
planes: &[Vec<u16>],
w: usize,
h: usize,
sampling: &[(usize, usize)],
precision: u8,
predictor: u8,
restart: usize,
) -> Vec<u8> {
let nc = planes.len();
let (hmax, vmax) = (sampling.iter().map(|s| s.0).max().unwrap_or(1), sampling.iter().map(|s| s.1).max().unwrap_or(1));
let (mcux, mcuy) = (w.div_ceil(hmax), h.div_ceil(vmax));
// pad every plane to whole MCUs by repeating its last column / row
let padded: Vec<(usize, Vec<u16>)> = planes
.iter()
.zip(sampling)
.map(|(p, &(hs, vs))| {
let (cw, ch) = ((w * hs).div_ceil(hmax), (h * vs).div_ceil(vmax));
let (pw, ph) = (mcux * hs, mcuy * vs);
(pw, (0..pw * ph).map(|i| p[(i / pw).min(ch - 1) * cw + (i % pw).min(cw - 1)]).collect())
})
.collect();
let init = 1i32 << (precision - 1);
// differences in coding order, with the component of each
let mut diffs: Vec<(usize, i32)> = Vec::new();
let mut marks = Vec::new();
let mut restart_left = restart;
let mut interval_row = 0;
let mut reset = vec![true; nc];
for my in 0..mcuy {
for mx in 0..mcux {
if restart > 0 {
if restart_left == 0 {
marks.push(diffs.len());
restart_left = restart;
interval_row = my;
reset.iter_mut().for_each(|r| *r = true);
}
restart_left -= 1;
}
for c in 0..nc {
let (hs, vs) = sampling[c];
let (pw, ref p) = padded[c];
for by in 0..vs {
for bx in 0..hs {
let (x, y) = (mx * hs + bx, my * vs + by);
let i = y * pw + x;
let pred = if reset[c] {
reset[c] = false;
init
} else if y == interval_row * vs {
p[i - 1] as i32
} else if x == 0 {
if by == 0 { p[i - vs * pw] as i32 } else { p[i - pw] as i32 }
} else {
let (ra, rb, rc) = (p[i - 1] as i32, p[i - pw] as i32, p[i - pw - 1] as i32);
match predictor {
1 => ra,
2 => rb,
3 => rc,
4 => ra + rb - rc,
5 => ra + ((rb - rc) >> 1),
6 => rb + ((ra - rc) >> 1),
_ => (ra + rb) >> 1,
}
};
let mut d = (p[i] as i32 - pred) & 0xffff;
if d > 32768 {
d -= 65536;
}
diffs.push((c, d));
}
}
}
}
}
let tables: Vec<([u8; 16], Vec<u8>)> = (0..nc)
.map(|c| {
let mut freq = [0u32; 17];
diffs.iter().filter(|(k, _)| *k == c).for_each(|(_, d)| freq[ssss_of(*d) as usize] += 1);
code_lengths(&freq)
})
.collect();
let mut out = vec![0xff, 0xd8, 0xff, 0xc3];
out.extend_from_slice(&((8 + 3 * nc) as u16).to_be_bytes());
out.push(precision);
out.extend_from_slice(&(h as u16).to_be_bytes());
out.extend_from_slice(&(w as u16).to_be_bytes());
out.push(nc as u8);
for (c, &(hs, vs)) in sampling.iter().enumerate() {
out.extend_from_slice(&[c as u8 + 1, (hs << 4 | vs) as u8, 0]);
}
for (c, (counts, values)) in tables.iter().enumerate() {
out.extend_from_slice(&[0xff, 0xc4]);
out.extend_from_slice(&((2 + 17 + values.len()) as u16).to_be_bytes());
out.push(c as u8);
out.extend_from_slice(counts);
out.extend_from_slice(values);
}
if restart > 0 {
out.extend_from_slice(&[0xff, 0xdd, 0, 4]);
out.extend_from_slice(&(restart as u16).to_be_bytes());
}
out.extend_from_slice(&[0xff, 0xda]);
out.extend_from_slice(&((6 + 2 * nc) as u16).to_be_bytes());
out.push(nc as u8);
for c in 0..nc {
out.extend_from_slice(&[c as u8 + 1, (c as u8) << 4]);
}
out.extend_from_slice(&[predictor, 0, 0]);
let codes: Vec<[(u32, u8); 17]> = tables.iter().map(|(c, v)| canonical_codes(c, v)).collect();
let mut bw = BitWriter { out, acc: 0, n: 0 };
let mut rst = 0u8;
for (i, &(c, d)) in diffs.iter().enumerate() {
if marks.contains(&i) {
bw.flush();
bw.out.extend_from_slice(&[0xff, 0xd0 + rst]);
rst = (rst + 1) & 7;
bw.acc = 0;
}
let s = ssss_of(d);
let (code, len) = codes[c][s as usize];
bw.put(code, len as u32);
if s > 0 && s < 16 {
let v = if d < 0 { d - 1 } else { d };
bw.put(v as u32 & ((1 << s) - 1), s as u32);
}
}
bw.flush();
let mut out = bw.out;
out.extend_from_slice(&[0xff, 0xd9]);
out
}
#[cfg(test)]
mod tests {
use super::*;
@@ -771,6 +1085,92 @@ mod tests {
let _ = decode(&bad, usize::MAX);
}
/// Smooth test planes for a `w × h` frame with the given sampling factors.
fn planar_input(w: usize, h: usize, sampling: &[(usize, usize)], seed: u64) -> Vec<Vec<u16>> {
let (hmax, vmax) = (sampling.iter().map(|s| s.0).max().unwrap(), sampling.iter().map(|s| s.1).max().unwrap());
sampling
.iter()
.enumerate()
.map(|(c, &(hs, vs))| {
let (pw, ph) = ((w * hs).div_ceil(hmax), (h * vs).div_ceil(vmax));
let n = noise(pw * ph, 12, seed + c as u64);
(0..pw * ph).map(|i| 1000 + (i / pw * 7 + i % pw * 3) as u16 % 2000 + n[i] % 300).collect()
})
.collect()
}
#[test]
fn planar_roundtrip_subsampled() {
for (w, h, sampling) in [
(16usize, 12usize, vec![(2usize, 2usize), (1, 1), (1, 1)]), // 4:2:0 (Sony M-size)
(16, 12, vec![(2, 1), (1, 1), (1, 1)]), // 4:2:2 (Sony S-size)
(15, 11, vec![(2, 2), (1, 1), (1, 1)]), // partial MCUs at the right and bottom
(9, 7, vec![(1, 1), (1, 1)]), // no subsampling
(8, 8, vec![(1, 2), (2, 1), (1, 1)]),
] {
let planes = planar_input(w, h, &sampling, 3);
for pred in 1..=7 {
let enc = encode_planar(&planes, w, h, &sampling, 15, pred, 0);
let f = decode_planar(&enc, usize::MAX).unwrap_or_else(|e| panic!("{w}×{h} {sampling:?} p{pred}: {e}"));
assert_eq!((f.width, f.height, f.predictor), (w, h, pred));
for (k, p) in f.planes.iter().enumerate() {
assert_eq!(p.sampling, sampling[k]);
assert_eq!(p.data, planes[k], "{w}×{h} {sampling:?} predictor {pred} component {k}");
}
}
}
// restart intervals of whole MCU rows
let s = [(2, 2), (1, 1), (1, 1)];
let planes = planar_input(16, 12, &s, 9);
let enc = encode_planar(&planes, 16, 12, &s, 15, 1, 16);
assert_eq!(decode_planar(&enc, usize::MAX).unwrap().planes.into_iter().map(|p| p.data).collect::<Vec<_>>(), planes);
// ... others are reported, not mis-decoded
let enc = encode_planar(&planes, 16, 12, &s, 15, 1, 5);
assert!(matches!(decode_planar(&enc, usize::MAX), Err(RawError::Unsupported(_))));
// the interleaved decoder hands subsampled frames to the planar one
let enc = encode_planar(&planes, 16, 12, &s, 15, 1, 0);
assert!(matches!(decode(&enc, usize::MAX), Err(RawError::Unsupported(_))));
assert_eq!(sampling_factors(&enc).unwrap(), s.to_vec());
}
#[test]
fn planar_first_column_follows_the_mcu_row() {
// a frame whose first column steps between MCU rows: decoding with "the line above" for the first sample
// of an MCU row would shift every later row; our encoder mirrors the rule of `decode_planar`
let s = [(2, 2), (1, 1), (1, 1)];
let mut planes = planar_input(8, 8, &s, 5);
for (y, row) in planes[0].chunks_mut(8).enumerate() {
row[0] = 4000 + 500 * y as u16;
}
let enc = encode_planar(&planes, 8, 8, &s, 15, 1, 0);
assert_eq!(decode_planar(&enc, usize::MAX).unwrap().planes[0].data, planes[0]);
}
#[test]
fn planar_rejects_bad_streams() {
let s = [(2, 2), (1, 1), (1, 1)];
let planes = planar_input(16, 12, &s, 1);
let enc = encode_planar(&planes, 16, 12, &s, 15, 1, 0);
assert!(decode_planar(&[], 100).is_err());
assert!(matches!(decode_planar(&enc, 100), Err(RawError::Limit(_))));
for n in 0..enc.len() {
let _ = decode_planar(&enc[..n], usize::MAX);
}
let mut bad = enc.clone();
for i in (20..bad.len()).step_by(3) {
bad[i] ^= 0x5a;
}
let _ = decode_planar(&bad, usize::MAX);
// sampling factors 0 or > 4, and more than 10 samples per MCU
let sof = enc.windows(2).position(|w| w == [0xff, 0xc3]).unwrap();
for samp in [0x00u8, 0x50, 0x05, 0x44] {
let mut b = enc.clone();
b[sof + 11] = samp; // first component's sampling byte
b[sof + 14] = samp;
assert!(decode_planar(&b, usize::MAX).is_err(), "sampling {samp:#x}");
}
}
#[test]
fn code_lengths_are_limited_to_16() {
// Fibonacci-like frequencies produce very long codes before adjustment
+387
View File
@@ -0,0 +1,387 @@
//! DNG camera-profile look data carried by the file itself (Adobe DNG Specification 1.6/1.7,
//! chapter 6 and the `ProfileHueSatMap*`, `ProfileLookTable*` and `ProfileToneCurve` tags).
//!
//! - A hue/saturation/value table holds, for a grid of (value, hue, saturation) inputs, a hue shift
//! in degrees, a saturation scale and a value scale (value-major, hue-middle, saturation-minor).
//! Hue samples are spread evenly around the circle with wrap-around, saturation and value samples
//! evenly over 0..=1; a dimension with one division is constant. Lookups are tri-linear.
//! - Tables are applied in HSV of linear ProPhoto RGB (D50). With encoding 1 ("sRGB") the value
//! axis is indexed through the sRGB transfer curve (3D tables only).
//! - `ProfileHueSatMapData1/2` follow the colour matrices (interpolated by the same illuminant
//! weight); `ProfileLookTableData` comes after exposure compensation and before any tone curve.
//! - `ProfileToneCurve`: (input, output) pairs in linear gamma from (0, 0) to (1, 1).
//!
//! The data is read from the user's own file at run time; nothing here ships profile data.
//! Values above 1.0 (highlight headroom) keep their headroom: the table's value scale only ever
//! lowers them, so a lookup can't clip what the scene-referred pipeline still needs.
use lightcraft_color::transfer::{linear_to_srgb, srgb_to_linear};
use lightcraft_color::{D50, D65, Mat3, PROPHOTO, REC2020, bradford};
use serde::{Deserialize, Serialize};
/// Upper bound on table entries (Adobe profiles use ≤ 90 × 30 × 16; guards header-driven sizes).
const MAX_ENTRIES: usize = 1 << 20;
/// A DNG hue/saturation/value mapping table (`ProfileHueSatMapData*` / `ProfileLookTableData`).
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct HsvTable {
pub hue_divisions: usize,
pub sat_divisions: usize,
pub val_divisions: usize,
/// `(hue shift °, saturation scale, value scale)`, value-major, hue-middle, saturation-minor.
pub data: Vec<[f32; 3]>,
/// The value axis is indexed through the sRGB encoding curve (`…Encoding` = 1).
pub srgb_value: bool,
}
impl HsvTable {
/// A table from its `…Dims` (hue, saturation, value divisions), data floats and encoding.
/// `None` when the dimensions are out of range, the data length doesn't match or a value is
/// not finite.
pub fn from_tags(dims: &[u64], data: &[f64], encoding: u64) -> Option<HsvTable> {
let [h, s, v] = <[u64; 3]>::try_from(dims).ok()?;
let (h, s, v) = (usize::try_from(h).ok()?, usize::try_from(s).ok()?, usize::try_from(v).ok()?);
if h < 1 || s < 2 || v < 1 || h > 4096 || s > 4096 || v > 4096 {
return None;
}
let n = h.checked_mul(s)?.checked_mul(v)?;
if n > MAX_ENTRIES || data.len() != n * 3 || data.iter().any(|x| !x.is_finite()) {
return None;
}
let data = data.as_chunks::<3>().0.iter().map(|c| [c[0] as f32, (c[1] as f32).max(0.0), (c[2] as f32).max(0.0)]).collect();
Some(HsvTable { hue_divisions: h, sat_divisions: s, val_divisions: v, data, srgb_value: encoding == 1 && v > 1 })
}
/// The entry-wise blend `g · a + (1 − g) · b` of two tables of the same shape (`None` otherwise).
pub fn blend(a: &HsvTable, b: &HsvTable, g: f64) -> Option<HsvTable> {
if (a.hue_divisions, a.sat_divisions, a.val_divisions) != (b.hue_divisions, b.sat_divisions, b.val_divisions) {
return None;
}
let g = g.clamp(0.0, 1.0) as f32;
let data = a.data.iter().zip(&b.data).map(|(x, y)| std::array::from_fn(|i| g * x[i] + (1.0 - g) * y[i])).collect();
Some(HsvTable { data, ..a.clone() })
}
fn entry(&self, v: usize, h: usize, s: usize) -> [f32; 3] {
self.data.get((v * self.hue_divisions + h) * self.sat_divisions + s).copied().unwrap_or([0.0, 1.0, 1.0])
}
/// Tri-linear lookup at hue `h` (degrees), saturation `s` and value `v` (table coordinates).
pub fn lookup(&self, h: f32, s: f32, v: f32) -> [f32; 3] {
let hf = (h.rem_euclid(360.0) / 360.0 * self.hue_divisions as f32).clamp(0.0, self.hue_divisions as f32);
let h0 = (hf as usize).min(self.hue_divisions - 1);
let h1 = (h0 + 1) % self.hue_divisions;
let th = (hf - h0 as f32).clamp(0.0, 1.0);
let axis = |x: f32, n: usize| {
let f = x.clamp(0.0, 1.0) * (n - 1) as f32;
let i0 = (f as usize).min(n.saturating_sub(2));
let i1 = (i0 + 1).min(n - 1);
(i0, i1, (f - i0 as f32).clamp(0.0, 1.0))
};
let (s0, s1, ts) = axis(s, self.sat_divisions);
let (v0, v1, tv) = if self.val_divisions > 1 { axis(v, self.val_divisions) } else { (0, 0, 0.0) };
let lerp = |a: [f32; 3], b: [f32; 3], t: f32| -> [f32; 3] { std::array::from_fn(|i| a[i] + (b[i] - a[i]) * t) };
let plane = |vi: usize| {
let lo = lerp(self.entry(vi, h0, s0), self.entry(vi, h0, s1), ts);
let hi = lerp(self.entry(vi, h1, s0), self.entry(vi, h1, s1), ts);
lerp(lo, hi, th)
};
if v0 == v1 { plane(v0) } else { lerp(plane(v0), plane(v1), tv) }
}
/// Apply the table to one linear ProPhoto RGB colour.
pub fn apply(&self, rgb: [f32; 3]) -> [f32; 3] {
let max = rgb[0].max(rgb[1]).max(rgb[2]);
if max.is_nan() || max <= 0.0 || !max.is_finite() {
return rgb;
}
let min = rgb[0].min(rgb[1]).min(rgb[2]);
let d = max - min;
let hue = hue_of(rgb, max, d);
let sat = d / max;
let v_index = if self.srgb_value { linear_to_srgb(max.min(1.0)) } else { max };
let [shift, ss, vs] = self.lookup(hue, sat, v_index);
// saturation and value: scaled and clipped to 1.0 (DNG chapter 6), except that inputs
// already above 1 (out-of-ProPhoto colours, highlight headroom) are only ever lowered.
let s2 = limited(sat, ss);
let v2 = if self.srgb_value && max <= 1.0 {
srgb_to_linear((linear_to_srgb(max) * vs).min(1.0))
} else if self.srgb_value {
max * srgb_to_linear(vs.min(1.0)).min(1.0)
} else {
limited(max, vs)
};
hsv_to_rgb(hue + shift, s2, v2)
}
}
/// `x · k` clipped to 1, for `x ≤ 1`; above 1, only scale factors below 1 apply (continuous at 1).
fn limited(x: f32, k: f32) -> f32 {
if x <= 1.0 { (x * k).min(1.0) } else { x * k.min(1.0) }
}
fn hue_of(c: [f32; 3], max: f32, d: f32) -> f32 {
if d <= 0.0 {
return 0.0;
}
let h = if max == c[0] {
((c[1] - c[2]) / d).rem_euclid(6.0)
} else if max == c[1] {
(c[2] - c[0]) / d + 2.0
} else {
(c[0] - c[1]) / d + 4.0
};
h * 60.0
}
/// HSV → RGB for any saturation ≥ 0 (above 1 yields negative components, the inverse of
/// [`hue_of`] for out-of-gamut inputs).
fn hsv_to_rgb(h: f32, s: f32, v: f32) -> [f32; 3] {
let hp = h.rem_euclid(360.0) / 60.0;
let sector = (hp as u32).min(5);
let f = hp - sector as f32;
let (p, q, t) = (v * (1.0 - s), v * (1.0 - s * f), v * (1.0 - s * (1.0 - f)));
match sector {
0 => [v, t, p],
1 => [q, v, p],
2 => [p, v, t],
3 => [p, q, v],
4 => [t, p, v],
_ => [v, p, q],
}
}
/// A DNG tone curve (`ProfileToneCurve`): linear-gamma (input, output) pairs, increasing input.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct ToneCurve {
pub points: Vec<[f32; 2]>,
}
impl ToneCurve {
/// From the tag's flat float pairs. `None` unless there are 2..=65536 finite pairs within
/// 0..=1 with strictly increasing inputs.
pub fn from_tag(v: &[f64]) -> Option<ToneCurve> {
if !v.len().is_multiple_of(2) || v.len() < 4 || v.len() > 131_072 {
return None;
}
let points: Vec<[f32; 2]> = v.as_chunks::<2>().0.iter().map(|p| [p[0] as f32, p[1] as f32]).collect();
let ok = points.iter().all(|p| p.iter().all(|x| x.is_finite() && (0.0..=1.0).contains(x))) && points.windows(2).all(|w| w[1][0] > w[0][0]);
ok.then_some(ToneCurve { points })
}
/// The curve at `x` (0..=1): monotone cubic (Fritsch–Carlson) through the samples.
pub fn eval(&self, x: f32) -> f32 {
let p = &self.points;
let n = p.len();
let x = x.clamp(p[0][0], p[n - 1][0]);
let i = p.partition_point(|q| q[0] <= x).clamp(1, n - 1) - 1;
let slope = |k: usize| (p[k + 1][1] - p[k][1]) / (p[k + 1][0] - p[k][0]);
let tangent = |k: usize| -> f32 {
if k == 0 {
return slope(0);
}
if k == n - 1 {
return slope(n - 2);
}
let (a, b) = (slope(k - 1), slope(k));
if a * b <= 0.0 { 0.0 } else { 2.0 / (1.0 / a + 1.0 / b) }
};
let (x0, x1) = (p[i][0], p[i + 1][0]);
let h = x1 - x0;
let t = (x - x0) / h;
let (t2, t3) = (t * t, t * t * t);
let y = (2.0 * t3 - 3.0 * t2 + 1.0) * p[i][1]
+ (t3 - 2.0 * t2 + t) * h * tangent(i)
+ (-2.0 * t3 + 3.0 * t2) * p[i + 1][1]
+ (t3 - t2) * h * tangent(i + 1);
y.clamp(0.0, 1.0)
}
}
/// The DNG profile look tags of a file (all optional).
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
pub struct ProfileLook {
/// `ProfileHueSatMapData1/2` (same shape, `ProfileHueSatMapDims`).
pub hue_sat_map: [Option<HsvTable>; 2],
/// `ProfileLookTableData`.
pub look_table: Option<HsvTable>,
/// `ProfileToneCurve`.
pub tone_curve: Option<ToneCurve>,
}
impl ProfileLook {
pub fn is_empty(&self) -> bool {
self.hue_sat_map.iter().all(Option::is_none) && self.look_table.is_none() && self.tone_curve.is_none()
}
/// The hue/saturation map for calibration-1 weight `g` (one table: used for any white).
pub fn hue_sat_for(&self, g: f64) -> Option<HsvTable> {
match &self.hue_sat_map {
[Some(a), Some(b)] => HsvTable::blend(a, b, g).or_else(|| Some(a.clone())),
[Some(a), None] | [None, Some(a)] => Some(a.clone()),
[None, None] => None,
}
}
}
/// The tables resolved for one white balance, applied to linear Rec.2020 D65 pixels.
#[derive(Clone, Debug)]
pub struct ProfileTables {
hue_sat: Option<HsvTable>,
look: Option<HsvTable>,
to_prophoto: [[f32; 3]; 3],
from_prophoto: [[f32; 3]; 3],
}
impl ProfileTables {
/// `None` when the profile has no hue/saturation map and no look table.
pub fn new(look: &ProfileLook, illuminant_weight: f64) -> Option<ProfileTables> {
let hue_sat = look.hue_sat_for(illuminant_weight);
if hue_sat.is_none() && look.look_table.is_none() {
return None;
}
let to: Mat3 = PROPHOTO.from_xyz().mul(&bradford(D65, D50)).mul(&REC2020.to_xyz());
let from = to.inverse()?;
Some(ProfileTables { hue_sat, look: look.look_table.clone(), to_prophoto: to.to_f32(), from_prophoto: from.to_f32() })
}
/// Hue/saturation map, then `gain` (exposure compensation), then the look table:
/// linear Rec.2020 D65 in, linear Rec.2020 D65 out.
#[inline]
pub fn apply(&self, rgb: [f32; 3], gain: f32) -> [f32; 3] {
let m = &self.to_prophoto;
let mut p: [f32; 3] = std::array::from_fn(|i| m[i][0] * rgb[0] + m[i][1] * rgb[1] + m[i][2] * rgb[2]);
if let Some(t) = &self.hue_sat {
p = t.apply(p);
}
p = p.map(|v| v * gain);
if let Some(t) = &self.look {
p = t.apply(p);
}
let m = &self.from_prophoto;
std::array::from_fn(|i| m[i][0] * p[0] + m[i][1] * p[1] + m[i][2] * p[2])
}
}
#[cfg(test)]
mod tests {
use super::*;
fn identity(h: usize, s: usize, v: usize) -> HsvTable {
HsvTable { hue_divisions: h, sat_divisions: s, val_divisions: v, data: vec![[0.0, 1.0, 1.0]; h * s * v], srgb_value: false }
}
fn close(a: [f32; 3], b: [f32; 3], tol: f32) -> bool {
a.iter().zip(&b).all(|(x, y)| (x - y).abs() <= tol)
}
#[test]
fn identity_table_is_identity() {
let t = identity(6, 3, 1);
for c in [[0.2, 0.5, 0.1], [0.9, 0.9, 0.9], [0.0, 0.0, 0.4], [3.0, 1.0, 0.5], [0.3, -0.05, 0.2]] {
assert!(close(t.apply(c), c, 1e-5), "{c:?} → {:?}", t.apply(c));
}
assert_eq!(t.apply([0.0; 3]), [0.0; 3]);
assert_eq!(t.apply([-1.0, -2.0, -0.5]), [-1.0, -2.0, -0.5]);
}
#[test]
fn parses_tags_and_rejects_bad_shapes() {
let data: Vec<f64> = (0..4).flat_map(|_| [0.0, 1.0, 1.0]).collect();
assert!(HsvTable::from_tags(&[2, 2, 1], &data, 0).is_some());
assert!(HsvTable::from_tags(&[2, 2], &data, 0).is_none());
assert!(HsvTable::from_tags(&[2, 1, 2], &data, 0).is_none(), "saturation needs ≥ 2 divisions");
assert!(HsvTable::from_tags(&[2, 2, 2], &data, 0).is_none(), "length mismatch");
assert!(HsvTable::from_tags(&[u64::MAX, 2, 2], &data, 0).is_none());
let mut nan = data.clone();
nan[1] = f64::NAN;
assert!(HsvTable::from_tags(&[2, 2, 1], &nan, 0).is_none());
// the sRGB encoding only applies to 3D tables
assert!(!HsvTable::from_tags(&[2, 2, 1], &data, 1).unwrap().srgb_value);
}
#[test]
fn saturation_scale_follows_the_table_layout() {
// 4 hues (0°, 90°, 180°, 270°), 2 saturations; saturation-minor: entry (h, s) at h·2 + s.
let mut t = identity(4, 2, 1);
t.data[1] = [0.0, 1.5, 1.0]; // hue 0°, saturation 1 → ×1.5
let red = [0.5, 0.4, 0.4]; // hue 0°, saturation 0.2
let out = t.apply(red);
// half-way along the saturation axis is ×1.1 (0.2 → 0.22 sat), value kept
let sat = (out[0] - out[1].min(out[2])) / out[0];
assert!((sat - 0.2 * 1.1).abs() < 1e-4, "{out:?}");
assert!((out[0] - 0.5).abs() < 1e-6);
// other hues are untouched (hue 180°)
let cyan = [0.3, 0.4, 0.4];
assert!(close(t.apply(cyan), cyan, 1e-6));
// saturation clips to 1.0
assert!(t.apply([1.0, 0.1, 0.1]).iter().all(|v| *v >= -1e-6));
}
#[test]
fn hue_wraps_and_value_scales() {
// 2 hues: entries at 0° and 180°, hue shift +20° at 0°, value ×0.5 at 180°.
let mut t = identity(2, 2, 1);
t.data = vec![[20.0, 1.0, 1.0], [20.0, 1.0, 1.0], [0.0, 1.0, 0.5], [0.0, 1.0, 0.5]];
// 350° sits between the last sample (180°) and the first (0° = 360°), 10/180 from 360
assert!((t.lookup(350.0, 0.5, 0.5)[0] - 20.0 * (170.0 / 180.0)).abs() < 1e-3);
assert!((t.lookup(180.0, 0.5, 0.5)[2] - 0.5).abs() < 1e-6);
// headroom above 1 only goes down
let bright = [2.0, 2.4, 2.4];
let out = t.apply(bright);
assert!((out[1] - 1.2).abs() < 1e-5, "{out:?}");
let mut up = identity(1, 2, 1);
up.data = vec![[0.0, 1.0, 2.0]; 2];
assert!((up.apply([0.4, 0.4, 0.4])[0] - 0.8).abs() < 1e-6);
assert!((up.apply([0.8, 0.8, 0.8])[0] - 1.0).abs() < 1e-6, "clipped to 1");
assert!((up.apply([3.0, 3.0, 3.0])[0] - 3.0).abs() < 1e-6, "headroom kept");
}
#[test]
fn value_axis_and_srgb_encoding() {
// 1 hue, 2 sats, 2 values: saturation ×2 at value 1 only
let mut t = identity(1, 2, 2);
t.data[3] = [0.0, 2.0, 1.0];
let sat = |c: [f32; 3]| (c[0] - c[2]) / c[0];
let dark = [0.25, 0.2, 0.2]; // V 0.25, S 0.2
// linear: 1/4 of the way → S scale at s=0.2: lerp(1, lerp(1,2,0.2)=1.2, 0.25) = 1.05
assert!((sat(t.apply(dark)) - 0.2 * 1.05).abs() < 1e-4);
t.srgb_value = true;
let k = linear_to_srgb(0.25); // ≈ 0.537
let want = 0.2 * (1.0 + 0.2 * k);
assert!((sat(t.apply(dark)) - want).abs() < 1e-4);
}
#[test]
fn blend_and_tables_round_trip_through_prophoto() {
let a = identity(2, 2, 1);
let mut b = identity(2, 2, 1);
b.data[1] = [10.0, 2.0, 1.0];
let m = HsvTable::blend(&a, &b, 0.25).unwrap();
assert_eq!(m.data[1], [7.5, 1.75, 1.0]);
assert!(HsvTable::blend(&a, &identity(3, 2, 1), 0.5).is_none());
let look = ProfileLook { hue_sat_map: [Some(a.clone()), None], look_table: Some(a), tone_curve: None };
let t = ProfileTables::new(&look, 0.5).unwrap();
let c = [0.3, 0.2, 0.1];
assert!(close(t.apply(c, 2.0), [0.6, 0.4, 0.2], 1e-4), "{:?}", t.apply(c, 2.0));
assert!(ProfileTables::new(&ProfileLook::default(), 0.5).is_none());
}
#[test]
fn tone_curve_interpolates_monotonically() {
assert!(ToneCurve::from_tag(&[0.0, 0.0]).is_none());
assert!(ToneCurve::from_tag(&[0.0, 0.0, 0.5, 0.6, 0.4, 0.7, 1.0, 1.0]).is_none(), "inputs must increase");
assert!(ToneCurve::from_tag(&[0.0, 0.0, 1.0, 1.5]).is_none());
let c = ToneCurve::from_tag(&[0.0, 0.0, 0.25, 0.4, 0.5, 0.7, 1.0, 1.0]).unwrap();
assert_eq!(c.eval(0.0), 0.0);
assert!((c.eval(0.25) - 0.4).abs() < 1e-6);
assert!((c.eval(1.0) - 1.0).abs() < 1e-6);
let mut last = 0.0;
for i in 0..=100 {
let y = c.eval(i as f32 / 100.0);
assert!(y >= last - 1e-6, "monotone at {i}");
last = y;
}
assert!(c.eval(0.1) > 0.1 && c.eval(0.1) < 0.4);
}
}
+556 -11
View File
@@ -14,16 +14,23 @@
//! a 128-bit little-endian block: 11-bit max, 11-bit min, 4-bit index of the max, 4-bit index of the min and 14
//! seven-bit deltas above the min, scaled by the smallest shift that fits `max − min` into 7 bits.
//!
//! White balance: the raw IFD's `0x7313` levels when present (bodies from about 2017 on); otherwise the
//! `WB_RGBLevels` of the maker note's enciphered `Tag2010` block (see [`DECIPHER`] and [`TAG2010_WB`]). Without
//! either, an ARW opened with unit multipliers, i.e. a strong green cast (issue #148). Black level: the raw IFD's
//! `0x7310`, else the level stored in the encrypted `SR2SubIFD` (see [`SR2_BLACK_AT`]; 800 rather than the
//! default 512 on 1″-sensor bodies such as the RX100 series), else 512 (14-bit) / 128 (12-bit).
//!
//! Also: uncompressed 16-bit ARW, and lossless-compressed ARW (Compression 7, ILCE-7M4 and later): LJ92 tiles whose
//! frames hold one 2×2 CFA cell per four-component sample ([`read_quad_tiles`]). Other lossless-JPEG layouts go
//! through the generic TIFF path.
//! frames hold one 2×2 CFA cell per four-component sample ([`read_quad_tiles`]); reduced-size (M/S) lossless ARWs
//! store subsampled YCbCr instead ([`decode_ycbcr`], issue #147). Other lossless-JPEG layouts go through the
//! generic TIFF path.
use crate::tiffraw::{Packing, check_image, read_image_in};
use crate::{BlackLevel, Cfa, ColorData, Mode, OpcodeLists, RawData, RawError, RawFormat, RawImage, Rect, Result, ljpeg};
use lightcraft_geom::Orientation;
use lightcraft_tiff::image::{Chunk, ImageInfo, Layout, chunk_bytes};
use lightcraft_tiff::tags::{self as t, photometric};
use lightcraft_tiff::{Ifd, Tiff};
use lightcraft_tiff::{Ifd, Tiff, makernote};
use rayon::prelude::*;
const TONE_CURVE: u16 = 0x7010;
@@ -31,6 +38,137 @@ const BLACK_LEVEL: u16 = 0x7310;
const WB_RGGB: u16 = 0x7313;
const CROP_TOP_LEFT: u16 = 0x74c7;
const CROP_SIZE: u16 = 0x74c8;
/// Maker-note tags (ExifTool Sony tag names): the enciphered `Tag2010` block and `FullImageSize` (height, width).
const MN_TAG2010: u16 = 0x2010;
const MN_FULL_IMAGE_SIZE: u16 = 0xb02b;
/// Inverse of Sony's maker-note byte substitution. ExifTool's Sony tag documentation states that the data of
/// tags `0x2010`, `0x9050` and `0x94xx` "is encrypted by a simple substitution cipher" (no decoder source was
/// consulted). The substitution, checked black-box on CC0 raw.pixls.us samples: every byte `p < 249` is stored as
/// `p³ mod 249` (a bijection on 0..249, as 3 is coprime to φ(249) = 164) and bytes 249–255 are stored unchanged.
/// Verified on 15 files from 13 bodies covering seven `Tag2010` layouts: the deciphered `SonyISO` field encodes
/// the Exif ISO (`100 · 2^(16 − v/256)`), the green level is 255–256 throughout, and on the ILCE-7M3 the deciphered
/// `WB_RGBLevels` equal the plain raw-IFD `0x7313` levels.
const DECIPHER: [u8; 256] = {
let mut inv = [0u8; 256];
let mut p = 0usize;
while p < 256 {
let c = if p < 249 { p * p % 249 * p % 249 } else { p };
inv[c] = p as u8;
p += 1;
}
inv
};
/// `Tag2010` layouts (ExifTool "Sony Tag2010a" … "Tag2010i" tables): the models each is documented for and the
/// byte offset of `WB_RGBLevels` (three `u16`, the as-shot white balance as R, G, B gains; Tag2010g and h share
/// the offset). Newer bodies (Tag2010i) also write the levels in plain form in the raw IFD (`0x7313`), which takes
/// precedence.
const TAG2010_WB: &[(&[&str], usize)] = &[
(&["NEX-5N"], 4476),
(&["SLT-A65", "SLT-A77", "NEX-7", "NEX-VG20E"], 4480),
(&["SLT-A37", "SLT-A57", "NEX-F3"], 4444),
(&["DSC-HX10V", "DSC-HX20V", "DSC-HX200V", "DSC-TX66", "DSC-TX200V", "DSC-TX300V", "DSC-WX50", "DSC-WX100", "DSC-WX150"], 4568),
(
&[
"SLT-A58",
"SLT-A99",
"ILCE-3000",
"ILCE-3500",
"NEX-3N",
"NEX-5R",
"NEX-5T",
"NEX-6",
"NEX-VG30E",
"NEX-VG900",
"DSC-RX100",
"DSC-RX1",
"DSC-RX1R",
"DSC-HX300",
"DSC-HX50V",
"DSC-TX30",
"DSC-WX60",
"DSC-WX200",
"DSC-WX300",
],
4532,
),
(&["DSC-RX100M2", "DSC-QX10", "DSC-QX100"], 4204),
(
&[
"DSC-HX60V",
"DSC-HX350",
"DSC-HX400V",
"DSC-QX30",
"DSC-RX10",
"DSC-RX100M3",
"DSC-WX220",
"DSC-WX350",
"ILCE-7",
"ILCE-7R",
"ILCE-7S",
"ILCE-7M2",
"ILCE-5000",
"ILCE-5100",
"ILCE-6000",
"ILCE-QX1",
"ILCA-68",
"ILCA-77M2",
"DSC-HX80",
"DSC-HX90V",
"DSC-RX0",
"DSC-RX1RM2",
"DSC-RX10M2",
"DSC-RX10M3",
"DSC-RX100M4",
"DSC-RX100M5",
"DSC-WX500",
"ILCE-6300",
"ILCE-6500",
"ILCE-7RM2",
"ILCE-7SM2",
"ILCA-99M2",
],
612,
),
(
&[
"ILCE-6100",
"ILCE-6400",
"ILCE-6600",
"ILCE-7C",
"ILCE-7M3",
"ILCE-7RM3",
"ILCE-7RM4",
"ILCE-9",
"ILCE-9M2",
"DSC-RX0M2",
"DSC-RX10M4",
"DSC-RX100M6",
"DSC-RX100M5A",
"DSC-RX100M7",
"DSC-HX99",
],
594,
),
];
/// As-shot white balance from the maker note's enciphered `Tag2010` block (bodies that do not write `0x7313` in
/// the raw IFD: most ARWs before 2017). `model` is the Exif model; Sony appends a regional "V" to some names
/// (SLT-A77V), which the documented lists omit.
fn tag2010_wb(model: &str, block: &[u8], order: lightcraft_tiff::ByteOrder) -> Option<[f32; 3]> {
let model = model.trim();
let &(_, offset) = TAG2010_WB.iter().find(|(models, _)| models.iter().any(|m| model == *m || model.strip_suffix('V') == Some(*m)))?;
let bytes: Vec<u8> = block.get(offset..offset + 6)?.iter().map(|&b| DECIPHER[b as usize]).collect();
let level = |i: usize| order.read_u16(&bytes, 2 * i).map(f32::from);
let (r, g, b) = (level(0)?, level(1)?, level(2)?);
// levels are fixed-point gains (green 255–256 on every sample seen); reject implausible values from an unknown layout
if !(16.0..=16384.0).contains(&g) {
return None;
}
let (r, b) = (r / g, b / g);
((0.2..=8.0).contains(&r) && (0.2..=8.0).contains(&b)).then_some([r, 1.0, b])
}
/// The inverse tone curve: 11-bit code → 14-bit sensor value.
pub(crate) fn code_curve(thresholds: &[u64]) -> Vec<u16> {
@@ -163,6 +301,266 @@ fn read_quad_tiles(bytes: &[u8], info: &ImageInfo) -> Result<Vec<u16>> {
Ok(out)
}
/// Black offset of the luma in Sony's YCbCr ARWs, in the samples' own units. The files' `ReferenceBlackWhite`
/// says 0, but on the CC0 samples the darkest pixels sit at ≈1060 and, registered against the same scene shot as
/// a full-size (CFA) ARW (ILCE-7RM5), luma = 0.997 × (white-balanced, black-subtracted CFA luma) + 1034.
const YCC_BLACK: f32 = 1024.0;
/// Sony's clip level of the white-balanced RGB behind the YCbCr samples: clipped highlights are stored as a
/// neutral luma plateau at 18 876 (ILCE-7M4 M-size sample), i.e. 17 852 above [`YCC_BLACK`].
const YCC_WHITE: f32 = 18876.0;
/// The values written for the camera RGB we reconstruct: `YCC_OUT_BLACK` + black-subtracted camera RGB (a raw-like
/// scale), white level `YCC_OUT_BLACK + YCC_WHITE − YCC_BLACK` for every channel.
const YCC_OUT_BLACK: f32 = 512.0;
/// TIFF 6.0 `YCbCrCoefficients` (`LumaRed`, `LumaGreen`, `LumaBlue`).
const YCBCR_COEFFICIENTS: u16 = 0x0211;
/// Sony's reduced-size lossless ARWs ("M" and "S" RAW size on the ILCE-7M4, -7RM5, -1 and later): TIFF
/// compression 7 with `PhotometricInterpretation` 6 (YCbCr), three samples per pixel, `YCbCrSubSampling` 2×2 (M)
/// or 2×1 (S). Each tile is one lossless-JPEG frame of the tile's size whose luma has sampling factors 2×2 / 2×1
/// and whose two chroma components are 1×1 ([`ljpeg::decode_planar`]). The image is already demosaiced and white
/// balanced: luma and chroma follow the TIFF 6.0 YCbCr definition (section 21) with the file's
/// `YCbCrCoefficients` (BT.601: 0.299, 0.587, 0.114), chroma centred on 16 384 and on the luma's scale — checked
/// by registering an M- and an S-size ILCE-7RM5 file against the L-size (CFA) one of the same scene: luma
/// = 0.997·Y + 1034 (M) and chroma = 1.01 × the TIFF formula (S: Cb 1.014, Cr 1.008; M: Cb 1.006, Cr 0.89, the
/// least reliable of the four with the M frame registered at a non-integer scale). We invert that (with chroma
/// co-sited, `YCbCrPositioning` 2, and interpolated bilinearly), remove [`YCC_BLACK`] and divide by the file's white balance (`0x7313`), so
/// the result is linear camera RGB like a `LinearRaw` DNG, with the as-shot multipliers reported as usual.
fn decode_ycbcr(
bytes: &[u8],
mode: Mode,
tiff: &Tiff,
raw: &Ifd,
info: &ImageInfo,
wb: Option<[f32; 3]>,
mn: Option<&makernote::MakerNote>,
) -> Result<RawImage> {
let (w, h) = (info.width as usize, info.height as usize);
if info.samples_per_pixel != 3 || !matches!(info.layout, Layout::Tiles { .. }) {
return Err(RawError::Unsupported("Sony YCbCr ARW that is not tiled with three samples per pixel".into()));
}
check_image(bytes, info)?;
let coeffs = match raw.f64s(YCBCR_COEFFICIENTS).as_deref() {
Some([r, g, b]) if [r, g, b].iter().all(|v| (0.01..1.0).contains(*v)) && (r + g + b - 1.0).abs() < 0.01 => [*r as f32, *g as f32, *b as f32],
_ => [0.299, 0.587, 0.114],
};
let gains = wb.unwrap_or([1.0; 3]);
let data = match mode {
Mode::Header => Vec::new(),
Mode::Full => {
let (luma, chroma, (hs, vs)) = read_ycbcr_planes(bytes, info)?;
ycbcr_to_camera_rgb(&luma, &chroma, w, h, (hs, vs), coeffs, gains)
}
};
// one white level for all channels, as for a mosaic: Sony clipped after white balance, so a clipped highlight
// (R, G, B = clip / gains in camera RGB) becomes exactly white once the pipeline applies the multipliers
let white = vec![YCC_OUT_BLACK + (YCC_WHITE - YCC_BLACK)];
let crop = match (raw.u64s(CROP_TOP_LEFT).as_deref(), raw.u64s(CROP_SIZE).as_deref()) {
(Some([x, y]), Some([cw, ch])) if *cw > 0 && *ch > 0 => Rect::new(*x as usize, *y as usize, *cw as usize, *ch as usize).clipped(w, h),
_ => default_crop(raw, mn, w, h),
};
let mut metadata = lightcraft_meta::from_tiff(tiff);
metadata.width = Some(crop.width as u32);
metadata.height = Some(crop.height as u32);
let img = RawImage {
format: RawFormat::Arw,
width: w,
height: h,
cpp: 3,
data: RawData::U16(data),
cfa: None,
bits: info.bits() as u32,
black: BlackLevel::uniform(YCC_OUT_BLACK),
white,
active_area: Rect::new(0, 0, w, h),
crop,
orientation: Orientation::from_exif(tiff.ifds[0].u16(t::ORIENTATION).unwrap_or(1)),
color: ColorData::default(),
wb_multipliers: wb,
linearized: false,
opcodes: OpcodeLists::default(),
metadata,
};
img.validate_for(mode)?;
Ok(img)
}
/// Decode every tile of a subsampled YCbCr ARW into a full-size luma plane and two chroma planes (Cb, Cr) of
/// `⌈w / H⌉ × ⌈h / V⌉` samples; returns them with the luma sampling factors `(H, V)`.
type YccPlanes = (Vec<u16>, [Vec<u16>; 2], (usize, usize));
fn read_ycbcr_planes(bytes: &[u8], info: &ImageInfo) -> Result<YccPlanes> {
let (w, h) = (info.width as usize, info.height as usize);
let chunks = info.chunks(bytes.len() as u64);
let decoded: Vec<Result<(Chunk, ljpeg::PlanarFrame)>> = chunks
.par_iter()
.map(|c| {
let src = chunk_bytes(bytes, c).ok_or_else(|| RawError::Corrupt("tile offset past end of file".into()))?;
let f = ljpeg::decode_planar(src, (c.width as usize * c.height as usize * 3).max(1 << 16))?;
let ok = f.planes.len() == 3
&& (f.width, f.height) == (c.width as usize, c.height as usize)
&& f.planes[1..].iter().all(|p| p.sampling == (1, 1))
&& matches!(f.planes[0].sampling, (1 | 2, 1 | 2));
if !ok {
return Err(RawError::Unsupported(format!(
"Sony YCbCr ARW tile: {}×{} frame with sampling {:?}",
f.width,
f.height,
f.planes.iter().map(|p| p.sampling).collect::<Vec<_>>()
)));
}
Ok((*c, f))
})
.collect();
let mut sampling = None;
let mut luma = vec![0u16; w * h];
let mut chroma: [Vec<u16>; 2] = [Vec::new(), Vec::new()];
let (mut cw, mut ch) = (0, 0);
let mut ok = 0usize;
let mut first_err = None;
for r in decoded {
let (c, f) = match r {
Ok(v) => v,
Err(e) => {
first_err.get_or_insert(e);
continue;
}
};
let (hs, vs) = f.planes[0].sampling;
match sampling {
None => {
sampling = Some((hs, vs));
(cw, ch) = (w.div_ceil(hs), h.div_ceil(vs));
chroma = [vec![16384u16; cw * ch], vec![16384u16; cw * ch]];
}
Some(s) if s != (hs, vs) => {
first_err.get_or_insert(RawError::Corrupt("Sony YCbCr ARW: tiles with different sampling".into()));
continue;
}
Some(_) => {}
}
ok += 1;
let (x0, y0) = (c.x as usize, c.y as usize);
let p = &f.planes[0];
for y in 0..p.height.min(h.saturating_sub(y0)) {
let n = p.width.min(w.saturating_sub(x0));
luma[(y0 + y) * w + x0..][..n].copy_from_slice(&p.data[y * p.width..][..n]);
}
let (cx0, cy0) = (x0 / hs, y0 / vs);
for (k, plane) in f.planes[1..].iter().enumerate() {
for y in 0..plane.height.min(ch.saturating_sub(cy0)) {
let n = plane.width.min(cw.saturating_sub(cx0));
chroma[k][(cy0 + y) * cw + cx0..][..n].copy_from_slice(&plane.data[y * plane.width..][..n]);
}
}
}
match sampling {
Some(s) if ok > 0 => Ok((luma, chroma, s)),
_ => Err(first_err.unwrap_or_else(|| RawError::Corrupt("no decodable tiles".into()))),
}
}
/// YCbCr → camera RGB (see [`decode_ycbcr`]): interleaved `w × h × 3` samples on a raw-like scale.
fn ycbcr_to_camera_rgb(luma: &[u16], chroma: &[Vec<u16>; 2], w: usize, h: usize, (hs, vs): (usize, usize), coeffs: [f32; 3], gains: [f32; 3]) -> Vec<u16> {
let (cw, ch) = (w.div_ceil(hs), h.div_ceil(vs));
let [kr, kg, kb] = coeffs;
let inv = gains.map(|g| if g > 0.0 && g.is_finite() { 1.0 / g } else { 1.0 });
let mut out = vec![0u16; w * h * 3];
out.par_chunks_mut(w * 3).enumerate().for_each(|(y, row)| {
// co-sited chroma: sample (i, j) sits on luma (i·H, j·V)
let fy = y as f32 / vs as f32;
let j0 = (fy as usize).min(ch - 1);
let j1 = (j0 + 1).min(ch - 1);
let ty = fy - j0 as f32;
for x in 0..w {
let fx = x as f32 / hs as f32;
let i0 = (fx as usize).min(cw - 1);
let i1 = (i0 + 1).min(cw - 1);
let tx = fx - i0 as f32;
let at = |p: &Vec<u16>| {
let v = |i: usize, j: usize| f32::from(p.get(j * cw + i).copied().unwrap_or(16384));
let top = v(i0, j0) + (v(i1, j0) - v(i0, j0)) * tx;
let bottom = v(i0, j1) + (v(i1, j1) - v(i0, j1)) * tx;
top + (bottom - top) * ty - 16384.0
};
let (cb, cr) = (at(&chroma[0]), at(&chroma[1]));
let yv = f32::from(luma.get(y * w + x).copied().unwrap_or(0)) - YCC_BLACK;
let r = cr * (2.0 - 2.0 * kr) + yv;
let b = cb * (2.0 - 2.0 * kb) + yv;
let g = (yv - kb * b - kr * r) / kg;
for (c, v) in [r, g, b].into_iter().enumerate() {
row[x * 3 + c] = (YCC_OUT_BLACK + v * inv[c]).round().clamp(0.0, 65535.0) as u16;
}
}
});
out
}
/// `SR2Private` tags (ExifTool "Sony SR2Private" table; the IFD is referenced by IFD0's `DNGPrivateData`).
const SR2_SUBIFD_OFFSET: u16 = 0x7200;
const SR2_SUBIFD_LENGTH: u16 = 0x7201;
const SR2_SUBIFD_KEY: u16 = 0x7221;
/// Black level inside the encrypted `SR2SubIFD` (bodies that do not write `0x7310` in the raw IFD), recovered by
/// black-box known-plaintext analysis of CC0 raw.pixls.us samples; no description of Sony's cipher was used.
///
/// Every Sony ARW we examined (16 files, 15 bodies, 2012–2021) stores the same key (`11 22 33 44`), so its
/// keystream is the same in every file. The DSC-RX100M6 and ILCE-7M3 also write their black levels in plain form
/// (`0x7310`: 800 and 512); XOR-ing their `SR2SubIFD`s against each other matched `pack(512×4) ^ pack(800×4)` at
/// exactly one position, byte 2510, which gave the eight keystream bytes there. Applied to the other files at that
/// position they decode to four equal levels that agree with each sensor's dark-pixel floor: 800 on every 1″-sensor
/// body (RX10, RX100 … RX100M5), 512 on the APS-C/full-frame ones (NEX-5N/6, SLT-A77, ILCE-6000/7/7M2/7RM2), in
/// both `SR2SubIFD` sizes seen (29252 and 56958 bytes). Newer layouts (ILCE-7M4, 35398 bytes) put other data there
/// and are rejected by the plausibility checks; those bodies write `0x7310` anyway.
const SR2_BLACK_AT: usize = 2510;
const SR2_BLACK_KEYSTREAM: [u8; 8] = [0x43, 0xcb, 0x86, 0xb6, 0x11, 0xd2, 0x1a, 0x73];
const SR2_KEY: [u8; 4] = [0x11, 0x22, 0x33, 0x44];
/// The per-channel black level from the encrypted `SR2SubIFD` (see [`SR2_BLACK_AT`]), in 14-bit units; `None`
/// unless the file uses the known key and the four decoded levels are equal and plausible.
fn sr2_black(bytes: &[u8], ifd0: &Ifd, order: lightcraft_tiff::ByteOrder) -> Option<f32> {
let private = ifd0.bytes(t::DNG_PRIVATE_DATA)?;
let at = order.read_u32(private, 0)? as u64;
let opts = lightcraft_tiff::ParseOptions { max_ifds: 1, max_depth: 0, follow_children: false, ..Default::default() };
let (sr2, _) = lightcraft_tiff::parse_ifd_at(bytes, at, order, 0, false, &opts).ok()?;
if sr2.bytes(SR2_SUBIFD_KEY)? != SR2_KEY {
return None;
}
let (start, len) = (usize::try_from(sr2.u64(SR2_SUBIFD_OFFSET)?).ok()?, usize::try_from(sr2.u64(SR2_SUBIFD_LENGTH)?).ok()?);
if len < SR2_BLACK_AT + 8 {
return None;
}
sr2_black_levels(bytes.get(start.checked_add(SR2_BLACK_AT)?..start.checked_add(SR2_BLACK_AT + 8)?)?, order)
}
/// Decipher the eight `SR2SubIFD` bytes at [`SR2_BLACK_AT`] into one black level (the mean of four near-equal,
/// plausible per-channel levels).
fn sr2_black_levels(cipher: &[u8], order: lightcraft_tiff::ByteOrder) -> Option<f32> {
let plain: Vec<u8> = cipher.iter().zip(SR2_BLACK_KEYSTREAM).map(|(c, k)| c ^ k).collect();
let levels: Vec<u16> = (0..4).filter_map(|i| order.read_u16(&plain, 2 * i)).collect();
let (&lo, &hi) = (levels.iter().min()?, levels.iter().max()?);
(levels.len() == 4 && (64..=4096).contains(&lo) && hi - lo <= 64).then(|| levels.iter().map(|&v| f32::from(v)).sum::<f32>() / 4.0)
}
/// The image area for files without Sony's crop tags (`0x74c7/0x74c8`, written since about 2017): the DNG-style
/// default crop when the raw IFD has one, else the maker note's `FullImageSize` (the camera JPEG's size) anchored
/// at the top-left. Older bodies store a few columns of padding at the right edge of the raw frame (constant
/// values, 8–32 columns on the samples we checked) inside a frame 16–48 pixels wider than `FullImageSize`, so
/// the anchored crop removes them while keeping the CFA phase.
fn default_crop(raw: &Ifd, mn: Option<&makernote::MakerNote>, w: usize, h: usize) -> Rect {
let full = Rect::new(0, 0, w, h);
if let (Some([x, y]), Some([cw, ch])) = (raw.u64s(t::DEFAULT_CROP_ORIGIN).as_deref(), raw.u64s(t::DEFAULT_CROP_SIZE).as_deref())
&& *cw > 0
&& *ch > 0
{
return Rect::new(*x as usize, *y as usize, *cw as usize, *ch as usize).clipped(w, h);
}
match mn.and_then(|m| m.ifd.u64s(MN_FULL_IMAGE_SIZE)).as_deref() {
// only a plausible trim: never more than 64 pixels per side, never an enlargement
Some([fh, fw]) if *fw as usize <= w && *fh as usize <= h && *fw as usize + 64 >= w && *fh as usize + 64 >= h => {
Rect::new(0, 0, *fw as usize, *fh as usize)
}
_ => full,
}
}
fn raw_ifd(tiff: &Tiff) -> Option<&Ifd> {
tiff.all_ifds()
.into_iter()
@@ -179,7 +577,23 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
if w * h > crate::MAX_SAMPLES {
return Err(RawError::Limit("image too large"));
}
let model = ifd0.string(t::MODEL).unwrap_or_default();
let mn = tiff
.exif()
.and_then(|e| e.get(t::MAKER_NOTE))
.and_then(|e| makernote::parse_makernote(bytes, e.offset, e.count() as u64, tiff.order, &ifd0.string(t::MAKE).unwrap_or_default()));
let wb = raw
.f64s(WB_RGGB)
.filter(|v| v.len() == 4 && v[1] > 0.0 && v[0] > 0.0 && v[3] > 0.0)
.map(|v| {
let g = (v[1] + v[2]) / 2.0;
[(v[0] / g) as f32, 1.0, (v[3] / g) as f32]
})
.or_else(|| mn.as_ref().and_then(|m| tag2010_wb(&model, m.ifd.bytes(MN_TAG2010)?, m.order)));
let bits = info.bits() as u32;
if info.compression == 7 && info.photometric == photometric::YCBCR {
return decode_ycbcr(bytes, mode, &tiff, raw, &info, wb, mn.as_ref());
}
let chunks = info.chunks(bytes.len() as u64);
let strip_len: u64 = chunks.iter().map(|c| c.len).sum();
let (data, out_bits) = match info.compression {
@@ -215,26 +629,30 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
_ => (read_image_in(mode, bytes, &info, tiff.order, Packing::Msb)?, bits),
};
let RawData::U16(ref samples) = data else { return Err(RawError::Unsupported("float ARW".into())) };
// "12-bit uncompressed" files (e.g. ILCE-7RM2) say BitsPerSample 12 but store 16-bit words on the 14-bit scale
// of the other modes (black 512, peaks near 16383): black and white follow the data, `bits` keeps the tag
let scale_bits = if info.compression == 1 && out_bits < 14 && strip_len >= (w * h * 2) as u64 && samples.iter().any(|&v| v >> (out_bits + 1) != 0)
{
14
} else {
out_bits
};
let cfa = match (raw.u64s(t::CFA_REPEAT_PATTERN_DIM).as_deref(), raw.bytes(t::CFA_PATTERN_EP)) {
(Some([2, 2]), Some(p)) if p.len() == 4 && p.iter().all(|&c| c <= 2) => Cfa { width: 2, height: 2, pattern: p.to_vec() },
_ => Cfa::bayer_static("RGGB"),
};
let default_black = if out_bits >= 14 { 512.0 } else { 128.0 };
let default_black = if scale_bits >= 14 { 512.0 } else { 128.0 };
let black = match raw.f64s(BLACK_LEVEL).as_deref() {
Some([a, b, c, d]) => {
BlackLevel { repeat_rows: 2, repeat_cols: 2, values: vec![*a as f32, *b as f32, *c as f32, *d as f32], ..Default::default() }
}
_ => BlackLevel::uniform(default_black),
_ => BlackLevel::uniform(sr2_black(bytes, ifd0, tiff.order).filter(|_| scale_bits >= 14).unwrap_or(default_black)),
};
let white = raw.f64(t::WHITE_LEVEL).map(|v| v as f32).filter(|v| *v > 0.0).unwrap_or_else(|| super::white_from_data(samples, out_bits));
let wb = raw.f64s(WB_RGGB).filter(|v| v.len() == 4 && v[1] > 0.0 && v[0] > 0.0 && v[3] > 0.0).map(|v| {
let g = (v[1] + v[2]) / 2.0;
[(v[0] / g) as f32, 1.0, (v[3] / g) as f32]
});
let white = raw.f64(t::WHITE_LEVEL).map(|v| v as f32).filter(|v| *v > 0.0).unwrap_or_else(|| super::white_from_data(samples, scale_bits));
let crop = match (raw.u64s(CROP_TOP_LEFT).as_deref(), raw.u64s(CROP_SIZE).as_deref()) {
(Some([x, y]), Some([cw, ch])) if *cw > 0 && *ch > 0 => Rect::new(*x as usize, *y as usize, *cw as usize, *ch as usize).clipped(w, h),
_ => Rect::new(0, 0, w, h),
_ => default_crop(raw, mn.as_ref(), w, h),
};
let mut metadata = lightcraft_meta::from_tiff(&tiff);
metadata.width = Some(crop.width as u32);
@@ -329,6 +747,133 @@ mod tests {
let _ = code_curve(&[60000, 1, 0, 0]);
}
#[test]
fn decipher_inverts_the_cube_substitution() {
let mut seen = [false; 256];
for p in 0..=255u8 {
let c = if p < 249 { (p as u32).pow(3) % 249 } else { p as u32 } as u8;
assert_eq!(DECIPHER[c as usize], p);
seen[DECIPHER[p as usize] as usize] = true;
}
assert!(seen.iter().all(|&s| s), "not a bijection");
assert_eq!((DECIPHER[8], DECIPHER[27], DECIPHER[94]), (2, 3, 7)); // 7³ = 343 ≡ 94
}
fn encipher(plain: &[u8]) -> Vec<u8> {
plain.iter().map(|&p| if p < 249 { ((p as u32).pow(3) % 249) as u8 } else { p }).collect()
}
#[test]
fn tag2010_white_balance_by_model() {
use lightcraft_tiff::ByteOrder::Little;
let mut plain = vec![0u8; 700];
for (i, v) in [669u16, 256, 441].iter().enumerate() {
plain[612 + 2 * i..614 + 2 * i].copy_from_slice(&v.to_le_bytes());
}
let block = encipher(&plain);
let wb = tag2010_wb("DSC-RX100M3", &block, Little).unwrap();
assert!((wb[0] - 669.0 / 256.0).abs() < 1e-6 && wb[1] == 1.0 && (wb[2] - 441.0 / 256.0).abs() < 1e-6, "{wb:?}");
// same layout (Tag2010h), regional "V" suffix, other layouts, unknown models, truncated or implausible data
assert!(tag2010_wb("ILCE-7RM2", &block, Little).is_some());
assert!(tag2010_wb("DSC-RX100M3V", &block, Little).is_some());
assert!(tag2010_wb("SLT-A77V", &block, Little).is_none()); // Tag2010b: offset 4480 is past the block
assert!(tag2010_wb("DSC-RX100M3X", &block, Little).is_none());
assert!(tag2010_wb("ILCE-1", &block, Little).is_none());
assert!(tag2010_wb("DSC-RX100M3", &block[..615], Little).is_none());
assert!(tag2010_wb("DSC-RX100M3", &encipher(&[0u8; 700]), Little).is_none());
let mut odd = plain.clone();
odd[612..614].copy_from_slice(&9000u16.to_le_bytes()); // R/G ≈ 35
assert!(tag2010_wb("DSC-RX100M3", &encipher(&odd), Little).is_none());
}
#[test]
fn sr2_black_level_decodes_and_rejects_garbage() {
use lightcraft_tiff::ByteOrder::Little;
let cipher =
|levels: [u16; 4]| -> Vec<u8> { levels.iter().flat_map(|v| v.to_le_bytes()).zip(SR2_BLACK_KEYSTREAM).map(|(p, k)| p ^ k).collect() };
assert_eq!(sr2_black_levels(&cipher([800; 4]), Little), Some(800.0));
assert_eq!(sr2_black_levels(&cipher([512, 512, 514, 512]), Little), Some(512.5));
assert_eq!(sr2_black_levels(&cipher([65320, 64928, 65260, 488]), Little), None); // ILCE-7M4's layout
assert_eq!(sr2_black_levels(&cipher([0; 4]), Little), None);
assert_eq!(sr2_black_levels(&cipher([800; 4])[..5], Little), None);
assert_eq!(sr2_black_levels(&[], Little), None);
}
/// Camera RGB → Sony's YCbCr samples (the model `decode_ycbcr` inverts).
fn to_ycc(rgb: [f32; 3], gains: [f32; 3]) -> [f32; 3] {
let [r, g, b] = [rgb[0] * gains[0], rgb[1] * gains[1], rgb[2] * gains[2]];
let y = 0.299 * r + 0.587 * g + 0.114 * b;
[y + YCC_BLACK, (b - y) / 1.772 + 16384.0, (r - y) / 1.402 + 16384.0]
}
#[test]
fn ycbcr_tiles_decode_to_camera_rgb() {
// 4:2:0 (M) and 4:2:2 (S) images of 20×12 in 8×8 tiles, from camera RGB that is linear in x and y (so the
// bilinear interpolation of co-sited chroma is exact) and a white balance of 2.4 / 1.7
let gains = [2.4f32, 1.0, 1.7];
for (hs, vs) in [(2usize, 2usize), (2, 1)] {
let (w, h, tw) = (20usize, 12usize, 8usize);
let rgb_at = |x: usize, y: usize| [300.0 + 20.0 * x as f32, 900.0 + 12.0 * y as f32, 500.0 + 5.0 * (x + y) as f32];
let (tx_n, ty_n) = (w.div_ceil(tw), h.div_ceil(tw));
let mut file = vec![0u8; 16];
let (mut offsets, mut counts) = (vec![0u64; tx_n * ty_n], vec![0u64; tx_n * ty_n]);
for ty in 0..ty_n {
for tx in 0..tx_n {
// a tile overhanging the image repeats its edge
let px = |x: usize, y: usize| rgb_at((tx * tw + x).min(w - 1), (ty * tw + y).min(h - 1));
let luma: Vec<u16> =
(0..tw * tw).map(|i| to_ycc(px(i % tw, i / tw), gains)[0].round() as u16).collect();
let (cw, ch) = (tw / hs, tw / vs);
let chroma = |k: usize| -> Vec<u16> { (0..cw * ch).map(|i| to_ycc(px(i % cw * hs, i / cw * vs), gains)[k].round() as u16).collect() };
let enc = ljpeg::encode_planar(&[luma, chroma(1), chroma(2)], tw, tw, &[(hs, vs), (1, 1), (1, 1)], 16, 1, 0);
offsets[ty * tx_n + tx] = file.len() as u64;
counts[ty * tx_n + tx] = enc.len() as u64;
file.extend(enc);
}
}
let info = ImageInfo {
width: w as u32,
height: h as u32,
bits_per_sample: vec![15, 15, 15],
samples_per_pixel: 3,
compression: 7,
photometric: photometric::YCBCR,
planar: 1,
predictor: 1,
sample_format: 1,
new_subfile_type: 0,
layout: Layout::Tiles { tile_width: tw as u32, tile_height: tw as u32 },
offsets,
byte_counts: counts,
};
let (luma, chroma, sampling) = read_ycbcr_planes(&file, &info).unwrap();
assert_eq!(sampling, (hs, vs));
let out = ycbcr_to_camera_rgb(&luma, &chroma, w, h, sampling, [0.299, 0.587, 0.114], gains);
for y in 0..h {
for x in 0..w {
let want = rgb_at(x, y);
for c in 0..3 {
let got = out[(y * w + x) * 3 + c] as f32 - YCC_OUT_BLACK;
// rounding of the samples, amplified by the inverse transform; the last odd column / row
// has no chroma sample to its right / below and repeats the previous one
let edge = (hs == 2 && x == w - 1 && x % 2 == 1) || (vs == 2 && y == h - 1 && y % 2 == 1);
let tol = if edge { 40.0 } else { 3.0 };
assert!((got - want[c]).abs() <= tol, "{hs}×{vs} ({x}, {y}) channel {c}: {got} vs {}", want[c]);
}
}
}
// a corrupt tile is skipped (left at zero luma) as long as one decodes; none decodable is an error
let mut bad = info.clone();
bad.offsets[0] = file.len() as u64 + 10;
assert!(read_ycbcr_planes(&file, &bad).is_ok());
bad.offsets.iter_mut().for_each(|o| *o = 3);
assert!(read_ycbcr_planes(&file, &bad).is_err());
for n in [0, 20, file.len() / 2] {
let _ = read_ycbcr_planes(&file[..n], &info);
}
}
}
#[test]
fn quad_tiles_place_cfa_cells() {
// 14×12 mosaic in 8×8 tiles (the right and bottom tiles overhang), each tile a 4×4 frame of 2×2 cells,
+67
View File
@@ -193,3 +193,70 @@ fn corpus_nef_compressed_matches_uncompressed() {
assert!(hi <= 4095.0 && hi - lo > 500.0, "D7000 block means {lo}..{hi}");
}
}
/// Issue #138: DNGs converted by Adobe software carry their camera profile's hue/saturation map and
/// look table; we read them (and render with them). Camera-written DNGs here carry none.
#[test]
fn corpus_adobe_dngs_carry_profile_looks() {
let dir = corpus_root().join("raw");
let Ok(rd) = std::fs::read_dir(&dir) else {
eprintln!("skip: {} absent", dir.display());
return;
};
let mut seen = 0;
for p in rd.flatten().map(|e| e.path()) {
let name = p.file_name().unwrap().to_string_lossy().to_lowercase();
if !name.starts_with("dng-") || !name.ends_with(".dng") {
continue;
}
let info = probe_info(&std::fs::read(&p).unwrap()).unwrap_or_else(|e| panic!("{name}: {e}"));
let look = &info.color.profile;
if name.starts_with("dng-adobe-") {
seen += 1;
let hsm = look.hue_sat_map[0].as_ref().unwrap_or_else(|| panic!("{name}: no hue/sat map"));
assert!(hsm.hue_divisions > 1 && hsm.sat_divisions > 1, "{name}");
assert!(look.look_table.is_some(), "{name}: no look table");
// a profile applied to a mid grey keeps it (close to) neutral
let t = lightcraft_raw::profile::ProfileTables::new(look, 0.5).unwrap();
let g = t.apply([0.18; 3], 1.0);
assert!(g.iter().all(|v| (v - g[0]).abs() < 0.01 * g[0].max(0.01)), "{name}: grey → {g:?}");
}
eprintln!(
"{name:44} profile look: hsm {} look {} tone {}",
look.hue_sat_map[0].is_some(),
look.look_table.is_some(),
look.tone_curve.is_some()
);
}
eprintln!("{seen} Adobe-converted DNGs checked");
}
/// Issue #148: Sony ARWs from before ~2017 carry no plain white-balance, black-level or crop tags in the raw IFD.
/// White balance comes from the maker note's enciphered `Tag2010`, the black level from the encrypted `SR2SubIFD`
/// and the crop from `FullImageSize`; without them the RX100 III opened bright green. Expected values: the black
/// levels agree with each sensor's dark-pixel floor, the gains with the neutral sky of the camera JPEG.
#[test]
fn corpus_sony_pre2017_colour_metadata() {
let dir = corpus_root().join("raw");
// (file, black, approximate R and B gains, crop width × height)
let cases = [
("arw-sony-rx100m3.arw", 800.0, [2.61, 1.72], (5472, 3648)),
("arw-sony-rx100.arw", 800.0, [2.23, 2.00], (5472, 3648)),
("arw-sony-a7rm2-12bit-uncompressed.arw", 512.0, [2.58, 1.46], (7952, 5304)),
];
let mut seen = 0;
for (name, black, [r, b], (cw, ch)) in cases {
let Ok(bytes) = std::fs::read(dir.join(name)) else {
eprintln!("skip: {name} absent");
continue;
};
let img = decode(&bytes).unwrap_or_else(|e| panic!("{name}: {e}"));
assert_eq!(img.black.mean(), black, "{name}: black level");
let wb = img.wb_multipliers.unwrap_or_else(|| panic!("{name}: no as-shot white balance"));
assert!((wb[0] - r).abs() < 0.01 && wb[1] == 1.0 && (wb[2] - b).abs() < 0.01, "{name}: white balance {wb:?}");
assert_eq!((img.crop.width, img.crop.height), (cw, ch), "{name}: crop");
assert!(img.white_at(0) > 16000.0, "{name}: white {} (14-bit scale)", img.white_at(0));
seen += 1;
}
eprintln!("pre-2017 Sony ARW colour metadata checked on {seen} files");
}
+71
View File
@@ -182,6 +182,11 @@ fn base_ifd(w: usize, h: usize, bits: u16, cpp: usize, cfa: bool) -> IfdBuilder
}
fn dng(raw: IfdBuilder, order: ByteOrder) -> Vec<u8> {
dng_with(raw, order, |_| {})
}
/// [`dng`] with extra IFD 0 tags (where DNG writers put camera-profile tags).
fn dng_with(raw: IfdBuilder, order: ByteOrder, extra: impl FnOnce(&mut IfdBuilder)) -> Vec<u8> {
let mut ifd0 = IfdBuilder::new();
ifd0.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![1]));
ifd0.set(t::DNG_VERSION, Value::Byte(vec![1, 6, 0, 0]));
@@ -200,6 +205,7 @@ fn dng(raw: IfdBuilder, order: ByteOrder) -> Vec<u8> {
exif.set(t::ISO_SPEED, Value::Short(vec![200]));
ifd0.set_child(t::EXIF_IFD, exif);
ifd0.add_sub_ifd(raw);
extra(&mut ifd0);
TiffWriter::new(order, false).write(&[ifd0]).unwrap()
}
@@ -461,3 +467,68 @@ fn rejects_unsupported_and_broken() {
raw.set_image(ImageData::Strips { rows_per_strip: 8, strips: vec![vec![0; 128]] });
assert!(decode(&dng(raw, ByteOrder::Little)).is_err());
}
/// A Lightroom-style DNG: profile look tags in IFD 0, LinearRaw data in a SubIFD (issue #138).
fn profile_dng(order: ByteOrder, extra: impl FnOnce(&mut IfdBuilder)) -> Vec<u8> {
let (w, h) = (8, 6);
let px = pattern(w, h, 3, 16);
let mut raw = base_ifd(w, h, 16, 3, false);
raw.set(t::COMPRESSION, Value::Short(vec![1]));
raw.set_image(ImageData::Strips { rows_per_strip: h as u32, strips: vec![u16_bytes(&px, order)] });
dng_with(raw, order, extra)
}
fn hsv_floats(h: usize, s: usize, v: usize, f: impl Fn(usize, usize, usize) -> [f32; 3]) -> Vec<f32> {
(0..v).flat_map(|vi| (0..h).flat_map(move |hi| (0..s).map(move |si| (vi, hi, si)))).flat_map(|(vi, hi, si)| f(vi, hi, si)).collect()
}
#[test]
fn profile_look_tags_are_read_from_ifd0() {
for order in [ByteOrder::Little, ByteOrder::Big] {
let bytes = profile_dng(order, |ifd0| {
ifd0.set(t::PROFILE_HUE_SAT_MAP_DIMS, Value::Long(vec![6, 3, 1]));
ifd0.set(t::PROFILE_HUE_SAT_MAP_DATA_1, Value::Float(hsv_floats(6, 3, 1, |_, h, s| [h as f32, 1.0 + s as f32 * 0.1, 1.0])));
ifd0.set(t::PROFILE_HUE_SAT_MAP_DATA_2, Value::Float(hsv_floats(6, 3, 1, |_, _, _| [0.0, 1.0, 1.0])));
ifd0.set(t::PROFILE_LOOK_TABLE_DIMS, Value::Long(vec![4, 2, 3]));
ifd0.set(t::PROFILE_LOOK_TABLE_DATA, Value::Float(hsv_floats(4, 2, 3, |v, _, _| [0.0, 1.0, 1.0 - v as f32 * 0.05])));
ifd0.set(t::PROFILE_LOOK_TABLE_ENCODING, Value::Long(vec![1]));
ifd0.set(t::PROFILE_TONE_CURVE, Value::Float(vec![0.0, 0.0, 0.25, 0.15, 0.5, 0.55, 1.0, 1.0]));
});
let img = decode(&bytes).unwrap();
assert_eq!(img.cpp, 3);
let p = &img.color.profile;
let hsm = p.hue_sat_map[0].as_ref().unwrap();
assert_eq!((hsm.hue_divisions, hsm.sat_divisions, hsm.val_divisions), (6, 3, 1));
// value-major, hue-middle, saturation-minor: entry (h = 2, s = 1) is at 2·3 + 1
assert_eq!(hsm.data[7], [2.0, 1.1, 1.0]);
assert!(p.hue_sat_map[1].is_some());
let look = p.look_table.as_ref().unwrap();
assert_eq!((look.hue_divisions, look.sat_divisions, look.val_divisions, look.srgb_value), (4, 2, 3, true));
assert_eq!(look.data[2 * 4 * 2], [0.0, 1.0, 0.9]);
assert_eq!(p.tone_curve.as_ref().unwrap().points.len(), 4);
// the header-only probe sees the same profile
assert_eq!(lightcraft_raw::probe_info(&bytes).unwrap().color.profile, *p);
// our DNG writer keeps the look with the data (conversions, smart previews)
let again = decode(&write_dng(&img, &DngWriteOptions::default()).unwrap()).unwrap();
assert_eq!(again.color.profile, *p);
}
}
#[test]
fn malformed_profile_look_tags_are_ignored() {
let bytes = profile_dng(ByteOrder::Little, |ifd0| {
// data count doesn't match the dimensions
ifd0.set(t::PROFILE_HUE_SAT_MAP_DIMS, Value::Long(vec![6, 3, 1]));
ifd0.set(t::PROFILE_HUE_SAT_MAP_DATA_1, Value::Float(vec![0.0, 1.0, 1.0]));
// one saturation division is not allowed
ifd0.set(t::PROFILE_LOOK_TABLE_DIMS, Value::Long(vec![2, 1, 1]));
ifd0.set(t::PROFILE_LOOK_TABLE_DATA, Value::Float(vec![0.0, 1.0, 1.0, 0.0, 1.0, 1.0]));
// decreasing inputs and a NaN
ifd0.set(t::PROFILE_TONE_CURVE, Value::Float(vec![0.0, 0.0, 0.6, 0.5, 0.4, f32::NAN, 1.0, 1.0]));
});
let img = decode(&bytes).unwrap();
assert!(img.color.profile.is_empty(), "{:?}", img.color.profile);
// and without any of the tags there is nothing to apply
let plain = decode(&profile_dng(ByteOrder::Little, |_| {})).unwrap();
assert!(plain.color.profile.is_empty());
}
+9
View File
@@ -152,6 +152,15 @@ pub const CAMERA_CALIBRATION_SIGNATURE: u16 = 50931;
pub const PROFILE_CALIBRATION_SIGNATURE: u16 = 50932;
pub const AS_SHOT_PROFILE_NAME: u16 = 50934;
pub const PROFILE_NAME: u16 = 50936;
pub const PROFILE_HUE_SAT_MAP_DIMS: u16 = 50937;
pub const PROFILE_HUE_SAT_MAP_DATA_1: u16 = 50938;
pub const PROFILE_HUE_SAT_MAP_DATA_2: u16 = 50939;
pub const PROFILE_TONE_CURVE: u16 = 50940;
pub const PROFILE_EMBED_POLICY: u16 = 50941;
pub const PROFILE_LOOK_TABLE_DIMS: u16 = 50981;
pub const PROFILE_LOOK_TABLE_DATA: u16 = 50982;
pub const PROFILE_HUE_SAT_MAP_ENCODING: u16 = 51107;
pub const PROFILE_LOOK_TABLE_ENCODING: u16 = 51108;
pub const BASELINE_EXPOSURE_OFFSET: u16 = 51109;
pub const DEFAULT_BLACK_RENDER: u16 = 51110;
pub const FORWARD_MATRIX_1: u16 = 50964;
+3
View File
@@ -25,3 +25,6 @@ log = { workspace = true }
[dev-dependencies]
lightcraft-codecs = { workspace = true }
[build-dependencies]
serde_json = { workspace = true }
+18
View File
@@ -0,0 +1,18 @@
//! Compile translated format strings so Rust checks both languages' placeholders.
use std::{collections::BTreeMap, env, fs, path::PathBuf};
fn main() -> Result<(), Box<dyn std::error::Error>> {
println!("cargo:rerun-if-changed=locales/ja-formats.json");
let messages: BTreeMap<String, String> = serde_json::from_str(&fs::read_to_string("locales/ja-formats.json")?)?;
let mut source = String::from("macro_rules! tr_format {\n");
for (english, japanese) in messages {
let en = serde_json::to_string(&english)?;
let ja = serde_json::to_string(&japanese)?;
source.push_str(&format!(
"({en} $(, $($args:tt)*)?) => {{ if $crate::i18n::is_japanese() {{ format!({ja} $(, $($args)*)?) }} else {{ format!({en} $(, $($args)*)?) }} }};\n"
));
}
source.push_str("}\npub(crate) use tr_format;\n");
fs::write(PathBuf::from(env::var("OUT_DIR")?).join("ja-formats.rs"), source)?;
Ok(())
}
+104
View File
@@ -0,0 +1,104 @@
{
" · {skipped} skipped (file exists)": "・{skipped}件は同名ファイルのためスキップ",
" · {} duplicate{} skipped": "・重複{}枚をスキップ{:.0}",
" · {} failed: {}": "・{}件失敗:{}",
" · {} moved": "・{}件移動",
" · {} not readable": "・{}件は読み込み不可",
", {failed} file{} not readable": "、{failed}件を読み込めません{:.0}",
"Add {local_n} to My Photos": "{local_n}枚をマイフォトに追加",
"Add {} Photo{}": "{}枚の写真を追加{:.0}",
"Add “{}” to Favorites": "「{}」をお気に入りに追加",
"Added {n} photo{} to My Photos": "マイフォトに{n}枚を追加しました{:.0}",
"Added {n} photo{} to “{}”": "「{1}」に{n}枚を追加しました{0:.0}",
"All {n} selected photos move by the same amount.": "選択中の{n}枚を同じ時間だけずらします",
"Auto Sync: edits apply to {n} photos": "自動同期:{n}枚に編集を適用",
"Automatic ({} MB)": "自動({}MB)",
"Click {}:1": "クリック時 {}:1",
"Create {n} Virtual Copies": "{n}枚の仮想コピーを作成",
"Decoder: {reason}": "デコーダー:{reason}",
"Delete \"{label}\"": "「{label}」を削除",
"Delete \"{label}\" (and the mask)": "「{label}」とマスクを削除",
"Delete {n} Photos": "{n}枚の写真を削除",
"Delete “{current}”": "「{current}」を削除",
"Duplicate \"{label}\"": "「{label}」を複製",
"Example: {example}": "例:{example}",
"Exported {n} photo{}": "{n}枚を書き出しました{:.0}",
"Exported {ok} of {total} photo{}": "{total}枚中{ok}枚を書き出しました{:.0}",
"Exported {} point curve preset{}": "{}件のポイントカーブを書き出しました{:.0}",
"Exported {} preset{}": "{}件のプリセットを書き出しました{:.0}",
"Exporting {} of {total}": "書き出し中:{total}枚中{}枚",
"Filter bar — {active} active filter{}": "フィルターバー:{active}件適用中{:.0}",
"Healed {n} dust spot{}": "{n}か所のほこりを修復しました{:.0}",
"Imported {n} point curve preset{}": "{n}件のポイントカーブを読み込みました{:.0}",
"Imported {n} preset{}": "{n}件のプリセットを読み込みました{:.0}",
"Imported {n} preset{}, {f} file{} not readable": "{n}件のプリセットを読み込みました{:.0}。{f}件は読み込めません{:.0}",
"Imported {} photo{}": "{}枚を読み込みました{:.0}",
"Matches: {}": "一致条件:{}",
"Merging… {stage} {:.0}%": "合成中… {stage} {:.0}%",
"Move {what} to Recently Deleted?": "{what}を「最近削除した写真」に移動しますか?",
"Named ({named_n})": "名前付き({named_n})",
"Paste into {n} photo{} — only settings that were copied are pasted": "{n}枚に貼り付け{:.0}(コピーされた設定だけを適用)",
"Preset: {name}": "プリセット:{name}",
"Preview only — editing the camera's embedded JPEG ({})": "プレビューのみ:カメラ内蔵のJPEGを編集中({})",
"Preview only — {}": "プレビューのみ:{}",
"Profile: {}": "プロファイル:{}",
"Proof Copy ({})": "校正用コピー({})",
"Proof Preview · {}": "校正プレビュー・{}",
"Quick Develop · {n} photos": "クイック現像・{n}枚",
"Rated {}": "評価:{}",
"Reading photos… {done} of {total}": "写真を読み込み中:{total}枚中{done}枚",
"Redo {}": "やり直す:{}",
"Remove “{}” from Favorites": "「{}」をお気に入りから削除",
"Rename {n} Photos…": "{n}枚の写真の名前を変更…",
"Rename “{name}”": "「{name}」の名前を変更",
"Rendering on the CPU: {why}": "CPUで描画:{why}",
"Reset {} Channel": "{}チャンネルをリセット",
"Saved export preset “{}”": "書き出しプリセット「{}」を保存しました",
"Show {hidden} hidden location{}": "非表示の{hidden}か所を表示{:.0}",
"Showing {n} of {}": "{}枚中{n}枚を表示",
"Smart album: {}": "スマートアルバム:{}",
"Undo {}": "取り消す:{}",
"Version {} — a clean-room, pure-Rust photo library and raw developer.": "バージョン{} — Rust製の写真管理・RAW現像アプリ",
"Version: {}": "バージョン:{}",
"none (CPU rendering): {why}": "なし(CPU描画):{why}",
"the {n} selected photos": "選択中の{n}枚",
"the {} photos in view": "表示中の{}枚",
"these {count} photos": "この{count}枚の写真",
"{count} smart preview{} · {:.1} MB": "スマートプレビュー{count}枚{:.0}・{:.1}MB",
"{f} folders (and their subfolders)": "{f}個のフォルダとそのサブフォルダ",
"{label} Grading": "{label}のカラーグレーディング",
"{label} {d:+} on every selected photo": "選択したすべての写真の{label}を{d:+}調整",
"{label}, {n} photos": "{label}、{n}枚",
"{matching} of {t} photos": "{t}枚中{matching}枚",
"{matching} photos": "{matching}枚",
"{mode} spot {} of {}": "{mode}:{1}か所中{0}か所",
"{msg} ({failed} failed)": "{msg}({failed}件失敗)",
"{name} Label": "{name}のラベル",
"{name} Off": "{name}を無効にする",
"{name} On": "{name}を有効にする",
"{name} — double-click to reset it": "{name}:ダブルクリックでリセット",
"{n} change{} saved in memory but not written to disk: {e}\nLightCraft retries automatically; quitting now would lose {}.": "{n}件の変更{:.0}をディスクに保存できません:{e}\n自動で再試行します。終了すると変更が失われます{:.0}",
"{n} found · {sel} selected": "{n}枚検出・{sel}枚選択",
"{n} photos": "{n}枚",
"{n} photos in {} are hidden by the filters above": "{}の{n}枚は上のフィルターで非表示になっています",
"{n} photo{}": "{n}枚{:.0}",
"{n} photo{} match · updates automatically as photos change": "{n}枚が一致{:.0}・写真の変更に応じて自動更新",
"{n} photo{} now · updates automatically as photos change": "現在{n}枚{:.0}・写真の変更に応じて自動更新",
"{sel_n} selected · {counted}": "{sel_n}枚選択・{counted}",
"{stage}… {:.0}%": "{stage}… {:.0}%",
"{what} merge added": "{what}合成を追加しました",
"{what} merge failed: {e}": "{what}合成に失敗:{e}",
"{} and higher": "{}以上",
"{} files": "{}個のファイル",
"{} files and {f} folder{}": "{}個のファイルと{f}個のフォルダ{:.0}",
"{} of 4 guides — drag along lines that should be vertical or horizontal.": "ガイド4本中{}本:垂直または水平にする線に沿ってドラッグ",
"{} photos": "{}枚",
"{} photo{} not readable": "{}枚を読み込めません{:.0}",
"{} spot(s) on this photo": "この写真に{}か所のスポット",
"· {dups} already in the library": "・{dups}枚は読み込み済み",
"· {n} photo{}": "・{n}枚{:.0}",
"… and {more} more": "…ほか{more}件",
"Import cancelled · {} photo{} added": "読み込みを中止しました・{}枚を追加{:.0}",
"Reading photos… {} of {}": "写真を確認中:{1}枚中{0}枚",
"Adding photos… {} of {}": "写真を追加中:{1}枚中{0}枚"
}
File diff suppressed because it is too large. Load diff
+5
View File
@@ -82,9 +82,14 @@ pub fn inspect(app: &LightcraftApp, ctx: &egui::Context) -> Value {
"loupe": app.loupe_shown.map(|(p, src)| json!({"photo": p.0, "source": src, "pending": app.renderer.is_pending(crate::render::Slot::Main)})),
"hoverPreview": app.hover_preview.as_ref().map(|h| h.label.clone()),
"status": app.ui.status,
"notices": app.notices,
"quitPrompt": app.quit_prompt,
"unsaved": app.session.unsaved().map(|(n, e)| json!({"ops": n, "error": e})),
"libraryProblem": app.library_problem.as_ref().map(crate::panels::library_problem::LibraryProblem::to_json),
"scan": app.scan.as_ref().map(crate::import::ScanTask::status),
"export": {"running": app.export.as_ref().map(crate::export_task::ExportTask::status), "last": app.last_export_result},
"import": app.import.as_ref().map(crate::import::ImportTask::status),
"tasks": app.tasks.labels(),
"memory": memory(app),
})
}
+9 -5
View File
@@ -75,12 +75,13 @@ pub fn poll(app: &mut LightcraftApp, ctx: &egui::Context) {
let ok = files.iter().filter(|f| f.get("path").is_some()).count();
let failed: Vec<&Value> = files.iter().filter(|f| f.get("error").is_some()).collect();
let skipped = files.iter().filter(|f| f.get("skipped").is_some()).count();
let mut m = format!("Exported {ok} of {total} photo{}", if total == 1 { "" } else { "s" });
let mut m =
crate::i18n::tr_format!("Exported {ok} of {total} photo{}", if total == 1 { "" } else { "s" }, ok = ok, total = total);
if skipped > 0 {
m += &format!(" · {skipped} skipped (file exists)");
m += &crate::i18n::tr_format!(" · {skipped} skipped (file exists)", skipped = skipped);
}
if let Some(first) = failed.first() {
m += &format!(" · {} failed: {}", failed.len(), first["error"].as_str().unwrap_or("error"));
m += &crate::i18n::tr_format!(" · {} failed: {}", failed.len(), first["error"].as_str().unwrap_or("error"));
}
if cancelled {
m += " · cancelled";
@@ -96,14 +97,17 @@ pub fn poll(app: &mut LightcraftApp, ctx: &egui::Context) {
let (done, current) = task.progress.lock().map(|g| g.clone()).unwrap_or_default();
let (total, cancel) = (task.total, task.cancel.clone());
let t = Tokens::get(ctx);
egui::Window::new("Exporting")
egui::Window::new(crate::i18n::tr("Exporting"))
.title_bar(false)
.resizable(false)
.anchor(Align2::LEFT_BOTTOM, [16.0, -56.0])
.fixed_size([300.0, 64.0])
.show(ctx, |ui| {
ui.horizontal(|ui| {
ui.label(egui::RichText::new(format!("Exporting {} of {total}", (done + 1).min(total))).color(t.text));
ui.label(
egui::RichText::new(crate::i18n::tr_format!("Exporting {} of {total}", (done + 1).min(total), total = total))
.color(t.text),
);
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
let stopping = cancel.load(Ordering::Relaxed);
if crate::widgets::text_button(ui, "exportCancel", if stopping { "Stopping…" } else { "Cancel" }, false).clicked() {
+14 -1
View File
@@ -176,6 +176,7 @@ impl Headless {
|| self.app.scan.is_some()
|| self.app.import.is_some()
|| self.app.export.is_some()
|| !self.app.tasks.is_empty()
|| !self.app.synthetic.is_empty()
|| !self.events.is_empty()
}
@@ -285,7 +286,10 @@ mod tests {
let mut h = demo([900.0, 600.0]);
let r = h.request("ui.set", json!({"view": "detail"}), Duration::from_secs(10));
assert_eq!(r["ok"], true, "{r}");
h.snapshot(SETTLE)
// Compare only fully settled frames: a timed-out settle would compare half-rendered
// pictures and fail as a misleading pixel diff (seen on loaded CI machines).
assert!(h.settle(Duration::from_secs(300)), "the demo detail view did not settle within 300 s");
h.paint()
};
let (a, b) = (shot(), shot());
assert_eq!(a.size, b.size);
@@ -412,6 +416,15 @@ mod tests {
assert_eq!(after.develop, before.develop);
assert_eq!(after.history.len(), before.history.len(), "no history entry while hovering");
assert!(h.app.session.undo.is_empty());
// hovering down across group headers within the presets list retains the preview (no flicker)
h.request("ui.hoverWidget", json!({"id": "presetGroup:Creative"}), t);
h.step();
assert_eq!(h.app.hover_preview.as_ref().map(|p| p.label.as_str()), Some("Preset: High Contrast B&W"));
// hovering another preset switches smoothly to it
h.request("ui.hoverWidget", json!({"id": "preset:lc.warm-glow"}), t);
h.settle(SETTLE);
h.step();
assert_eq!(h.app.hover_preview.as_ref().map(|p| p.label.as_str()), Some("Preset: Warm Glow"));
// thumbnails: variant textures for the visible presets
h.app.ui.preset_thumbs = true;
h.request("ui.move", json!({"x": 5, "y": 500}), t);
+187 -117
View File
@@ -1,24 +1,27 @@
//! Interface translations. Command ids, document text and file names remain stable.
//! Untranslated labels fall back to English so coverage can grow incrementally.
//! Presentation-only localization. Command ids, user names and catalog data remain unchanged.
use std::{cell::Cell, collections::BTreeMap, sync::OnceLock};
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "lowercase")]
use serde::{Deserialize, Serialize};
include!(concat!(env!("OUT_DIR"), "/ja-formats.rs"));
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
pub enum Language {
#[default]
#[serde(rename = "en")]
En,
#[serde(rename = "ja")]
Ja,
}
impl Language {
pub const ALL: [Self; 2] = [Self::En, Self::Ja];
pub fn name(self) -> &'static str {
match self {
Self::En => "English",
Self::Ja => "日本語",
}
}
pub fn parse(code: &str) -> Option<Self> {
match code {
"en" => Some(Self::En),
@@ -26,129 +29,196 @@ impl Language {
_ => None,
}
}
pub fn tr(self, text: &str) -> &str {
if self == Self::Ja
&& let Some((_, japanese)) = JAPANESE.iter().find(|(english, _)| *english == text)
{
return japanese;
}
text
pub fn tr(self, source: &str) -> &str {
if self == Self::Ja { japanese().get(source).map(String::as_str).unwrap_or(source) } else { source }
}
}
const JAPANESE: &[(&str, &str)] = &[
("Photo", "写真"),
("Import Photos…", "写真を読み込み…"),
("Detail", "詳細"),
("Photo Grid", "写真グリッド"),
("Square Grid", "正方形グリッド"),
("Object", "オブジェクト"),
("Effect", "効果"),
("Settings…", "環境設定…"),
("Image", "画像"),
("Layer", "レイヤー"),
("Type", "書式"),
("Select", "選択"),
("Filter", "フィルター"),
("Window", "ウィンドウ"),
("Language", "表示言語"),
("Save As…", "別名で保存…"),
("Exit", "終了"),
("New…", "新規…"),
("New", "新規"),
("Horizontal", "横書き"),
("Vertical", "縦書き"),
("Orientation", "組み方向"),
("Layers", "レイヤー"),
("History", "履歴"),
("Properties", "プロパティ"),
("Color", "カラー"),
("Brush Settings", "ブラシ設定"),
("Tools", "ツール"),
("Options", "オプション"),
("Zoom In", "ズームイン"),
("Zoom Out", "ズームアウト"),
("Fit on Screen", "画面に合わせる"),
("Copy", "コピー"),
("Cut", "切り取り"),
("Paste", "貼り付け"),
("Select All", "すべて選択"),
("Deselect", "選択を解除"),
("Export", "書き出し"),
("Export As…", "形式を指定して書き出し…"),
("Search…", "検索…"),
("Theme", "テーマ"),
("Menu", "メニュー"),
("File", "ファイル"),
("Edit", "編集"),
("Pages", "ページ"),
("View", "表示"),
("Help", "ヘルプ"),
("Preferences", "環境設定"),
("Preferences…", "環境設定…"),
("Interface language", "表示言語"),
("Open…", "開く…"),
("New blank PDF", "空白の PDF を作成"),
("Create PDF from file…", "ファイルから PDF を作成…"),
("Create PDF from images…", "画像から PDF を作成…"),
("Create PDF from clipboard", "クリップボードから PDF を作成"),
("Combine files…", "ファイルを結合…"),
("Save", "保存"),
("Save as…", "別名で保存…"),
("Close file", "ファイルを閉じる"),
("Close all", "すべて閉じる"),
("Revert", "保存済みの状態に戻す"),
("Print…", "印刷…"),
("Document properties…", "文書のプロパティ…"),
("Undo", "取り消し"),
("Redo", "やり直し"),
("Find…", "検索…"),
("Advanced search…", "高度な検索…"),
("Copy pages", "ページをコピー"),
("Cut pages", "ページを切り取り"),
("Paste pages", "ページを貼り付け"),
("Fit visible", "表示範囲に合わせる"),
("Marquee zoom", "範囲指定ズーム"),
("Take a snapshot", "スナップショットを作成"),
("Full screen mode", "全画面表示"),
("Read mode", "閲覧モード"),
("Switch light / dark theme", "明るい/暗いテーマを切り替え"),
("Comments panel", "コメントパネル"),
("Form fields panel", "フォームフィールドパネル"),
("Clear form", "フォームをクリア"),
("Find tools and commands…", "ツールとコマンドを検索…"),
("Zoom", "ズーム"),
("Actual size", "実際のサイズ"),
("Zoom to page level", "ページ全体を表示"),
("Fit to width", "幅に合わせる"),
("Display theme", "表示テーマ"),
("Side panels", "サイドパネル"),
("OK", "OK"),
];
thread_local! {
static LANGUAGE: Cell<Language> = const { Cell::new(Language::En) };
}
pub fn default_language() -> Language {
if std::env::var("LIGHTCRAFT_LANGUAGE").as_deref() == Ok("ja") { Language::Ja } else { Language::En }
}
pub fn set_language(language: Language) {
LANGUAGE.with(|value| value.set(language));
}
pub fn is_japanese() -> bool {
LANGUAGE.with(|value| value.get() == Language::Ja)
}
fn japanese() -> &'static BTreeMap<String, String> {
static MESSAGES: OnceLock<BTreeMap<String, String>> = OnceLock::new();
MESSAGES.get_or_init(|| {
serde_json::from_str(include_str!("../locales/ja.json")).unwrap_or_else(|error| {
log::error!("Invalid Japanese message catalog: {error}");
BTreeMap::new()
})
})
}
/// Translate a built-in display label, preserving unknown labels verbatim.
/// Never call this on editable user text, filenames or command identifiers.
pub fn tr(source: &str) -> &str {
if is_japanese() { japanese().get(source).map(String::as_str).unwrap_or(source) } else { source }
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn language_setting_persists() {
let state = crate::state::UiState { language: Language::Ja, ..Default::default() };
let saved = serde_json::to_string(&state).unwrap();
let restored: crate::state::UiState = serde_json::from_str(&saved).unwrap();
assert_eq!(restored.language, Language::Ja);
assert_eq!(crate::state::UiState::default().language, Language::En);
fn catalog_is_valid_and_contains_core_workflows() {
let messages: BTreeMap<String, String> = serde_json::from_str(include_str!("../locales/ja.json")).unwrap();
for key in ["Import Photos…", "Export…", "Exposure", "White Balance", "Settings", "Language"] {
assert!(messages.get(key).is_some_and(|value| !value.is_empty() && value != key), "{key}");
}
}
#[test]
fn translations_are_unique_and_preserve_unknown_text() {
for (i, (en, ja)) in JAPANESE.iter().enumerate() {
assert!(!ja.is_empty());
assert!(JAPANESE.iter().take(i).all(|(other, _)| en != other));
assert_eq!(Language::En.tr(en), *en);
fn language_switches_and_unknown_text_survives() {
set_language(Language::Ja);
assert_eq!(tr("Exposure"), "露出");
assert_eq!(tr("my-photo.jpg"), "my-photo.jpg");
assert_eq!(tr("develop.set"), "develop.set");
assert_eq!(crate::menubar::display_item_label("album.addPhotos", &serde_json::json!({"id": 1}), "Color"), "Color");
assert_eq!(crate::menubar::display_item_label("app.export", &serde_json::json!({"preset": "Color"}), "Color"), "Color");
assert_eq!(crate::menubar::display_item_label("view.photoGrid", &serde_json::Value::Null, "Color"), "カラー");
set_language(Language::En);
assert_eq!(tr("Exposure"), "Exposure");
}
#[test]
fn translated_formats_preserve_counts_and_remove_english_plural_suffixes() {
set_language(Language::Ja);
assert_eq!(tr_format!("{n} photo{}", "s", n = 12), "12枚");
assert_eq!(tr_format!("Exported {ok} of {total} photo{}", "s", ok = 4, total = 12), "12枚中4枚を書き出しました");
set_language(Language::En);
assert_eq!(tr_format!("{n} photo{}", "s", n = 12), "12 photos");
}
#[test]
fn preferences_round_trip_and_old_settings_remain_readable() {
let old: crate::state::UiState = serde_json::from_str("{}").unwrap();
assert_eq!(old.language, Language::En);
let settings = crate::state::UiState { language: Language::Ja, ..old };
let saved = serde_json::to_string(&settings).unwrap();
let restored: crate::state::UiState = serde_json::from_str(&saved).unwrap();
assert_eq!(restored.language, Language::Ja);
}
#[test]
fn japanese_is_painted_and_both_font_weights_cover_the_catalog() {
let ctx = egui::Context::default();
crate::theme::install_fonts(&ctx);
let mut app = crate::LightcraftApp::new(lightcraft_engine::Session::with_demo(), Default::default());
app.ui.language = Language::Ja;
app.ui.left_panel = true;
let mut text = String::new();
fn collect(shape: &egui::epaint::Shape, text: &mut String) {
match shape {
egui::epaint::Shape::Text(shape) => {
text.push_str(&shape.galley.job.text);
text.push('\n');
}
egui::epaint::Shape::Vec(shapes) => shapes.iter().for_each(|shape| collect(shape, text)),
_ => {}
}
}
assert_eq!(Language::Ja.tr("File"), "ファイル");
assert_eq!(Language::Ja.tr("日本語の文書.pdf"), "日本語の文書.pdf");
assert_eq!(Language::parse("xx"), None);
for frame in 0..4 {
let input = crate::headless::HeadlessView::raw_input(egui::vec2(1600.0, 1000.0), 1.0, frame as f64 / 60.0, vec![]);
let mut out = ctx.run_ui(input, |ui| {
app.logic(ui.ctx());
app.ui(ui);
});
// This assertion inspects shapes without a renderer; discard texture uploads explicitly.
out.textures_delta.clear();
text.clear();
for shape in out.shapes {
collect(&shape.shape, &mut text);
}
}
assert!(text.contains("マイフォト"), "{text}");
assert!(text.contains("すべての写真"), "{text}");
if lightcraft_engine::fonts::japanese(lightcraft_engine::CRAFT_FONTS).next().is_none() {
eprintln!("skipped glyph coverage: built without CRAFT_FONTS_DIR, so there is no Japanese UI font");
} else {
ctx.fonts_mut(|fonts| {
for family in [egui::FontFamily::Proportional, egui::FontFamily::Name(crate::theme::FONT_SEMIBOLD.into())] {
let font = egui::FontId::new(13.0, family);
for message in japanese().values() {
for ch in message.chars().filter(|ch| !ch.is_whitespace()) {
assert!(fonts.has_glyph(&font, ch), "Missing glyph {ch} in {message}");
}
}
}
});
}
// Locale affects presentation only: command ids remain the same.
let ids = |app: &crate::LightcraftApp| crate::menus::menu_entries(app).into_iter().map(|entry| entry.id).collect::<Vec<_>>();
let japanese_ids = ids(&app);
set_language(Language::En);
assert_eq!(japanese_ids, ids(&app));
}
/// The UI families, after one frame so the font definitions are loaded.
fn fonts_ctx(craft: &'static [lightcraft_engine::CraftFont]) -> egui::Context {
let ctx = egui::Context::default();
ctx.set_fonts(crate::theme::font_definitions(craft));
let mut out = ctx.run_ui(egui::RawInput::default(), |_| {});
out.textures_delta.clear();
ctx
}
fn ui_families() -> [egui::FontFamily; 3] {
[egui::FontFamily::Proportional, egui::FontFamily::Name(crate::theme::FONT_SEMIBOLD.into()), egui::FontFamily::Monospace]
}
/// Built with craft-fonts, Japanese renders with real glyphs (no tofu) in every UI family.
#[test]
fn craft_fonts_render_japanese_in_the_ui() {
if lightcraft_engine::fonts::japanese(lightcraft_engine::CRAFT_FONTS).next().is_none() {
eprintln!("skipped: built without CRAFT_FONTS_DIR, so there is no Japanese UI font");
return;
}
let ctx = fonts_ctx(lightcraft_engine::CRAFT_FONTS);
ctx.fonts_mut(|fonts| {
for family in ui_families() {
let font = egui::FontId::new(13.0, family);
for ch in "日本語の文字".chars() {
assert!(fonts.has_glyph(&font, ch), "{ch} in {font:?}");
}
let galley = fonts.layout_no_wrap("日本語の文字".into(), font.clone(), egui::Color32::WHITE);
assert!(galley.size().x > 13.0 * 5.0, "{font:?}: six full-width glyphs, {:?}", galley.size());
}
});
}
/// Built without craft-fonts, the UI (in Japanese, too) still installs its fonts and runs;
/// Latin text keeps Inter.
#[test]
fn the_ui_works_without_craft_fonts() {
let ctx = fonts_ctx(&[]);
ctx.fonts_mut(|fonts| {
// (Not Monospace: egui's `has_glyph` reports false for glyphs of the family's
// replacement-glyph face, which there is Hack, the face that draws Latin.)
for family in [egui::FontFamily::Proportional, egui::FontFamily::Name(crate::theme::FONT_SEMIBOLD.into())] {
let font = egui::FontId::new(13.0, family);
assert!("LightCraft".chars().all(|ch| fonts.has_glyph(&font, ch)), "{font:?}");
}
});
let mut app = crate::LightcraftApp::new(lightcraft_engine::Session::with_demo(), Default::default());
app.ui.language = Language::Ja;
for frame in 0..3 {
let input = crate::headless::HeadlessView::raw_input(egui::vec2(1200.0, 800.0), 1.0, frame as f64 / 60.0, vec![]);
let mut out = ctx.run_ui(input, |ui| {
app.logic(ui.ctx());
app.ui(ui);
});
out.textures_delta.clear();
}
set_language(Language::En);
}
}
+334 -74
View File
@@ -3,8 +3,9 @@
//! a grid of thumbnails with checkboxes (duplicates marked and unchecked), the destination (add in
//! place / copy or move into the library's `Originals/` or a chosen folder, filed by day, by month,
//! into one folder or by a custom folder template, optionally renamed), an album
//! (existing or new), a preset and keywords to apply. Importing runs in small batches, one per
//! frame, with a progress window; the whole import is one undo step.
//! (existing or new), a preset and keywords to apply. Importing runs on a worker thread (files are
//! probed, copied or moved there) and its batches join the catalog between frames, with a progress
//! window and Cancel; the whole import is one undo step.
use egui::{Align2, Color32, Rect, Sense, Stroke, StrokeKind, pos2, vec2};
use lightcraft_engine::import::{ImportCandidate, ScanInput, ScanOutput, ScanProgress, scan_with};
@@ -17,7 +18,7 @@ use crate::render::Slot;
use crate::theme::Tokens;
use crate::widgets::register;
/// Files per batch (one batch per frame, so the progress window updates).
/// Files per batch (each batch joins the catalog as it is ready, so the progress window updates).
pub(crate) const BATCH: usize = 8;
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
@@ -91,9 +92,13 @@ impl ImportDialog {
}
}
/// A running import (see the module docs).
/// A running import (see the module docs). The file-system work — expanding folders, probing,
/// reading sidecars, copying or placing files — runs on a worker thread
/// ([`lightcraft_engine::import::ImportJob`]); each batch it readies is added to the catalog on the
/// UI thread between frames, so a slow drive never stalls the window.
#[derive(Debug, Default)]
pub struct ImportTask {
/// Files and folders to import (handed to the worker when the task starts).
queue: Vec<String>,
pub total: usize,
pub done: usize,
@@ -109,6 +114,156 @@ pub struct ImportTask {
/// Reading a folder for the Local view: the photos stay out of the library, and nothing is
/// selected or announced as added.
browse: bool,
/// Cancel was pressed: no further files are started; batches already readied are added.
pub cancelled: bool,
/// Auto Import (the watched folder): the selection stays as it is, a short toast when done.
auto: bool,
/// Auto Import: the selection to keep.
keep_selection: Option<lightcraft_engine::Selection>,
run: Option<ImportRun>,
}
impl ImportTask {
/// An import of `paths` (files or folders) with `library.import` params (without `paths`).
pub fn new(paths: Vec<String>, params: Value, undo0: usize, browse: bool) -> Self {
ImportTask { total: paths.len(), queue: paths, params, undo0, browse, ..Default::default() }
}
/// An Auto Import (see [`ImportTask::auto`]).
pub fn auto(mut self) -> Self {
self.auto = true;
self
}
/// `{done, total, imported, cancelled}` for `ui.inspect`.
pub fn status(&self) -> Value {
json!({"done": self.done, "total": self.total, "imported": self.imported, "cancelled": self.cancelled})
}
}
/// Where an import's batches are readied: a worker thread, or (browser build: no threads, and no
/// slow drives) inline, one batch per frame.
enum Runner {
#[cfg(not(target_arch = "wasm32"))]
Thread(std::sync::mpsc::Receiver<lightcraft_engine::import::Prepared>),
#[cfg(target_arch = "wasm32")]
Inline { job: Box<lightcraft_engine::import::ImportJob>, queue: Vec<String>, files: Option<std::collections::VecDeque<String>> },
}
struct ImportRun {
runner: Runner,
cancel: std::sync::Arc<std::sync::atomic::AtomicBool>,
total: std::sync::Arc<std::sync::atomic::AtomicUsize>,
done: std::sync::Arc<std::sync::atomic::AtomicUsize>,
opts: lightcraft_engine::import::ImportOptions,
now: String,
album: Option<u64>,
album_name: Option<String>,
}
impl std::fmt::Debug for ImportRun {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("ImportRun").field("cancel", &self.cancel.load(Ordering::Relaxed)).finish()
}
}
impl Drop for ImportRun {
/// A dropped import (replaced, or the app closing) stops its worker at the next file.
fn drop(&mut self) {
self.cancel.store(true, Ordering::Relaxed);
}
}
impl ImportRun {
fn start(
job: lightcraft_engine::import::ImportJob,
queue: Vec<String>,
album: Option<u64>,
album_name: Option<String>,
ctx: &egui::Context,
) -> Result<Self, String> {
let cancel = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false));
let (total, done) = (job.total.clone(), job.done.clone());
let (opts, now) = (job.opts.clone(), job.now().to_string());
let runner = Self::spawn(job, queue, cancel.clone(), ctx)?;
Ok(ImportRun { runner, cancel, total, done, opts, now, album, album_name })
}
#[cfg(not(target_arch = "wasm32"))]
fn spawn(
mut job: lightcraft_engine::import::ImportJob,
queue: Vec<String>,
cancel: std::sync::Arc<std::sync::atomic::AtomicBool>,
ctx: &egui::Context,
) -> Result<Runner, String> {
let (tx, rx) = std::sync::mpsc::channel();
let ctx = ctx.clone();
let work = move || {
let files = job.expand(&queue);
for chunk in files.chunks(BATCH) {
if cancel.load(Ordering::Relaxed) {
break;
}
let prepared = match lightcraft_engine::guard::catch("import", || job.prepare_files(chunk.to_vec(), &cancel)) {
Ok(p) => p,
Err(_) => break, // logged; the files readied so far are added
};
if let Err(unsent) = tx.send(prepared) {
// the import was dropped: nothing will add these, so take back what a Move placed
unsent.0.rollback();
break;
}
ctx.request_repaint();
}
ctx.request_repaint();
};
std::thread::Builder::new().name("lc-import".into()).spawn(work).map(|_| Runner::Thread(rx)).map_err(|e| format!("could not start: {e}"))
}
#[cfg(target_arch = "wasm32")]
fn spawn(
job: lightcraft_engine::import::ImportJob,
queue: Vec<String>,
_cancel: std::sync::Arc<std::sync::atomic::AtomicBool>,
_ctx: &egui::Context,
) -> Result<Runner, String> {
Ok(Runner::Inline { job: Box::new(job), queue, files: None })
}
/// Readied batches (at most a few per frame). `None` once the worker is done.
fn next_batches(&mut self) -> Option<Vec<lightcraft_engine::import::Prepared>> {
match &mut self.runner {
#[cfg(not(target_arch = "wasm32"))]
Runner::Thread(rx) => {
let mut out = Vec::new();
while out.len() < 4 {
match rx.try_recv() {
Ok(p) => out.push(p),
Err(std::sync::mpsc::TryRecvError::Empty) => return Some(out),
Err(std::sync::mpsc::TryRecvError::Disconnected) => return if out.is_empty() { None } else { Some(out) },
}
}
Some(out)
}
#[cfg(target_arch = "wasm32")]
Runner::Inline { job, queue, files } => {
if self.cancel.load(Ordering::Relaxed) {
return None;
}
if files.is_none() {
*files = Some(job.expand(queue).into());
return Some(Vec::new());
}
let Some(files) = files.as_mut() else { return None };
if files.is_empty() {
return None;
}
let n = files.len().min(BATCH);
let batch: Vec<String> = files.drain(..n).collect();
Some(vec![job.prepare_files(batch, &self.cancel)])
}
}
}
}
/// A folder scan running on a worker thread (a network share can take minutes to read; the
@@ -232,7 +387,8 @@ pub fn poll_scan(app: &mut LightcraftApp, ctx: &egui::Context) {
if !queue.is_empty() {
let undo0 = app.session.undo.len();
let total = queue.len();
app.import = Some(ImportTask { queue, total, params: json!({"mode": "add", "local": true}), undo0, browse: true, ..Default::default() });
let _ = total;
app.import = Some(ImportTask::new(queue, json!({"mode": "add", "local": true}), undo0, true));
}
return;
}
@@ -256,19 +412,21 @@ pub fn scan_progress(app: &mut LightcraftApp, ctx: &egui::Context) {
let text = if total == 0 {
if task.browse { "Reading folder…" } else { "Looking for photos…" }.to_string()
} else {
format!("Reading photos… {done} of {total}")
crate::i18n::tr_format!("Reading photos… {done} of {total}", done = done, total = total)
};
let mut cancel = false;
egui::Window::new("Scanning").title_bar(false).resizable(false).anchor(Align2::CENTER_BOTTOM, [0.0, -80.0]).fixed_size([340.0, 80.0]).show(
ctx,
|ui| {
egui::Window::new(crate::i18n::tr("Scanning"))
.title_bar(false)
.resizable(false)
.anchor(Align2::CENTER_BOTTOM, [0.0, -80.0])
.fixed_size([340.0, 80.0])
.show(ctx, |ui| {
ui.label(egui::RichText::new(text).color(t.text));
ui.add(egui::ProgressBar::new(done as f32 / total.max(1) as f32).desired_width(320.0));
let r = ui.button("Cancel");
let r = ui.button(crate::i18n::tr("Cancel"));
register(ui.ctx(), "button:scanCancel", r.rect);
cancel = r.clicked();
},
);
});
if cancel {
// the worker stops at its next file; don't wait for it (a NAS read can take a while)
task.progress.cancel.store(true, Ordering::Relaxed);
@@ -326,28 +484,82 @@ pub fn start(app: &mut LightcraftApp, d: &ImportDialog) -> Result<Value, String>
}
}
let total = queue.len();
app.import = Some(ImportTask { queue, total, params, undo0, ..Default::default() });
app.import = Some(ImportTask::new(queue, params, undo0, false));
app.renderer.forget_imports();
Ok(json!({"importing": total}))
}
/// Run one batch of the import in progress (called every frame).
pub fn tick(app: &mut LightcraftApp, ctx: &egui::Context) {
let Some(task) = app.import.as_mut() else { return };
// (browsing reads a sidecar per file, slow on a network share: keep each frame short)
let n = task.queue.len().min(if task.browse { 2 } else { BATCH });
let batch: Vec<String> = task.queue.drain(..n).collect();
let mut p = task.params.clone();
p["paths"] = json!(batch);
// renamed copies keep counting across batches
if p.get("rename").is_some() {
p["renameStart"] = json!(1 + task.imported);
/// Start importing `paths` (files or folders) in the background, e.g. dropped on the window.
pub fn start_paths(app: &mut LightcraftApp, paths: Vec<String>) -> Result<Value, String> {
if app.import.is_some() || app.scan.as_ref().is_some_and(|t| !t.browse) {
return Err("an import is running".into());
}
let r = app.session.execute("library.import", &p);
let Some(task) = app.import.as_mut() else { return };
task.done += n;
let undo0 = app.session.undo.len();
let total = paths.len();
app.import = Some(ImportTask::new(paths, json!({"mode": "add"}), undo0, false));
Ok(json!({"importing": total}))
}
/// Advance the import in progress (called every frame): start its worker, then add the batches it
/// has readied to the catalog.
pub fn tick(app: &mut LightcraftApp, ctx: &egui::Context) {
let Some(mut task) = app.import.take() else { return };
if task.run.is_none() {
let mut p = task.params.clone();
p["paths"] = json!(task.queue);
let started = lightcraft_engine::cmd::library::import_params(&app.session, &p)
.and_then(|req| Ok((lightcraft_engine::import::ImportJob::new(&mut app.session, req.opts)?, req.album, req.album_name)))
.map_err(|e| e.to_string())
.and_then(|(job, album, name)| ImportRun::start(job, std::mem::take(&mut task.queue), album, name, ctx));
if task.auto {
task.keep_selection = Some(app.session.selection.clone());
}
match started {
Ok(run) => task.run = Some(run),
Err(e) => {
log::warn!("import: {e}");
app.toast(ctx, format!("Import failed: {e}"));
return;
}
}
}
ctx.request_repaint_after(std::time::Duration::from_millis(100));
let Some(run) = task.run.as_mut() else { return };
let batches = run.next_batches();
let finished = batches.is_none();
for prepared in batches.into_iter().flatten() {
commit_batch(app, &mut task, prepared);
// arriving photos don't take over the selection
if let Some(sel) = &task.keep_selection {
app.session.selection = sel.clone();
}
}
if let Some(run) = &task.run {
task.total = run.total.load(Ordering::Relaxed).max(task.total);
task.done = run.done.load(Ordering::Relaxed);
}
if !finished {
app.import = Some(task);
return;
}
finish(app, ctx, task);
}
/// Add one readied batch to the catalog (an undo step merged into the import's at the end).
fn commit_batch(app: &mut LightcraftApp, task: &mut ImportTask, prepared: lightcraft_engine::import::Prepared) {
let Some(run) = task.run.as_ref() else { return };
let n = prepared.len();
let (opts, now, album, album_name) = (&run.opts, run.now.as_str(), run.album, run.album_name.clone());
let r = app.session.execute_fn("library.import", |s| {
let report = lightcraft_engine::import::commit_prepared(s, opts, now, prepared)?;
lightcraft_engine::cmd::library::import_batch_done(s, report, album, album_name.as_deref())
});
match r {
Ok(v) => {
// a new album (named in the params) is created with the first photos; later batches join it
if let (Some(a), Some(run)) = (v["album"].as_u64(), task.run.as_mut()) {
run.album = Some(a);
}
let len = |k: &str| v[k].as_array().map_or(0, Vec::len);
task.imported += len("imported");
task.duplicates += len("duplicates");
@@ -366,17 +578,22 @@ pub fn tick(app: &mut LightcraftApp, ctx: &egui::Context) {
task.failed += n;
}
}
ctx.request_repaint();
if !task.queue.is_empty() {
}
/// The import is done (or cancelled): one undo step, select the first photo, say what happened.
fn finish(app: &mut LightcraftApp, ctx: &egui::Context, task: ImportTask) {
let steps = app.session.undo.len().saturating_sub(task.undo0);
let label = crate::i18n::tr_format!("Add {} Photo{}", task.imported, if task.imported == 1 { "" } else { "s" });
app.session.merge_undo(steps, &label);
if task.auto {
if task.imported > 0 {
app.toast(ctx, format!("Auto Import: added {} photo{}", task.imported, if task.imported == 1 { "" } else { "s" }));
}
return;
}
let Some(task) = app.import.take() else { return };
let steps = app.session.undo.len().saturating_sub(task.undo0);
let label = format!("Add {} Photo{}", task.imported, if task.imported == 1 { "" } else { "s" });
app.session.merge_undo(steps, &label);
if task.browse {
if task.failed > 0 {
app.toast(ctx, format!("{} photo{} not readable", task.failed, if task.failed == 1 { "" } else { "s" }));
app.toast(ctx, crate::i18n::tr_format!("{} photo{} not readable", task.failed, if task.failed == 1 { "" } else { "s" }));
}
return;
}
@@ -384,35 +601,58 @@ pub fn tick(app: &mut LightcraftApp, ctx: &egui::Context) {
let _ = app.run("library.select", json!({"ids": [f]}));
}
let plural = |n: usize| if n == 1 { "" } else { "s" };
let mut msg = format!("Imported {} photo{}", task.imported, plural(task.imported));
let mut msg = if task.cancelled {
crate::i18n::tr_format!("Import cancelled · {} photo{} added", task.imported, plural(task.imported))
} else {
crate::i18n::tr_format!("Imported {} photo{}", task.imported, plural(task.imported))
};
if task.params["mode"] == "move" {
msg.push_str(&format!(" · {} moved", task.moved));
msg.push_str(&crate::i18n::tr_format!(" · {} moved", task.moved));
if task.kept > 0 {
msg.push_str(&format!(" · {} original{} left at the source", task.kept, plural(task.kept)));
}
}
if task.duplicates > 0 {
msg.push_str(&format!(" · {} duplicate{} skipped", task.duplicates, if task.duplicates == 1 { "" } else { "s" }));
msg.push_str(&crate::i18n::tr_format!(" · {} duplicate{} skipped", task.duplicates, if task.duplicates == 1 { "" } else { "s" }));
}
if task.failed > 0 {
msg.push_str(&format!(" · {} not readable", task.failed));
msg.push_str(&crate::i18n::tr_format!(" · {} not readable", task.failed));
}
app.toast(ctx, msg);
}
/// The progress window while an import runs.
/// The progress window while an import runs, with Cancel (files already copied or added stay;
/// nothing new is started).
pub fn progress(app: &mut LightcraftApp, ctx: &egui::Context) {
let Some(task) = &app.import else { return };
let t = Tokens::get(ctx);
let frac = task.done as f32 / task.total.max(1) as f32;
let text = format!("{} photos… {} of {}", if task.browse { "Reading" } else { "Adding" }, task.done, task.total);
egui::Window::new("Importing").title_bar(false).resizable(false).anchor(Align2::CENTER_BOTTOM, [0.0, -80.0]).fixed_size([340.0, 60.0]).show(
ctx,
|ui| {
let text = if task.cancelled {
crate::i18n::tr("Stopping…").to_string()
} else if task.browse {
crate::i18n::tr_format!("Reading photos… {} of {}", task.done, task.total)
} else {
crate::i18n::tr_format!("Adding photos… {} of {}", task.done, task.total)
};
let mut cancel = false;
egui::Window::new(crate::i18n::tr("Importing"))
.title_bar(false)
.resizable(false)
.anchor(Align2::CENTER_BOTTOM, [0.0, -80.0])
.fixed_size([340.0, 80.0])
.show(ctx, |ui| {
ui.label(egui::RichText::new(text).color(t.text));
ui.add(egui::ProgressBar::new(frac).desired_width(320.0));
},
);
let r = ui.add_enabled(!task.cancelled, egui::Button::new(crate::i18n::tr("Cancel")));
register(ui.ctx(), "button:importCancel", r.rect);
cancel = r.clicked();
});
if cancel && let Some(task) = app.import.as_mut() {
task.cancelled = true;
if let Some(run) = &task.run {
run.cancel.store(true, Ordering::Relaxed);
}
}
}
/// The dialog body: options, then the candidate grid.
@@ -422,15 +662,15 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, d: &mut ImportDialog) {
let dups = d.candidates.iter().filter(|c| c.duplicate.is_some()).count();
let sel = d.selected_paths().len();
ui.horizontal(|ui| {
ui.label(egui::RichText::new(format!("{n} found · {sel} selected")).color(t.text));
ui.label(egui::RichText::new(crate::i18n::tr_format!("{n} found · {sel} selected", n = n, sel = sel)).color(t.text));
if dups > 0 {
ui.label(egui::RichText::new(format!("· {dups} already in the library")).color(t.text_dim));
ui.label(egui::RichText::new(crate::i18n::tr_format!("· {dups} already in the library", dups = dups)).color(t.text_dim));
}
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
if crate::widgets::text_button(ui, "importNone", "Uncheck All", false).clicked() {
if crate::widgets::text_button(ui, "importNone", crate::i18n::tr("Uncheck All"), false).clicked() {
d.checked.iter_mut().for_each(|c| *c = false);
}
if crate::widgets::text_button(ui, "importAll", "Check All", false).clicked() {
if crate::widgets::text_button(ui, "importAll", crate::i18n::tr("Check All"), false).clicked() {
for i in 0..n {
d.checked[i] = d.importable(i);
}
@@ -458,29 +698,42 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, d: &mut ImportDialog) {
// options
if !d.sources.is_empty() {
field(ui, "Source", |ui| {
let r = ui.label(egui::RichText::new(source_summary(&d.sources)).color(t.text_label)).on_hover_text(d.sources.join("\n"));
// (whether each source is a folder is checked off the UI thread, once)
let joined = d.sources.join("\n");
let summary = crate::panels::left::fs_cached(ui, "import-source", &joined, f64::INFINITY, |s| {
source_summary(&s.split('\n').map(str::to_string).collect::<Vec<_>>())
})
.unwrap_or_else(|| format!("{} item{}", d.sources.len(), if d.sources.len() == 1 { "" } else { "s" }));
let r = ui.label(egui::RichText::new(summary).color(t.text_label)).on_hover_text(joined);
register(ui.ctx(), "label:importSource", r.rect);
});
ui.label(
egui::RichText::new("Importing scans the source for photos; it doesn't add the folder to Local (use Local → Browse Folder… to work in a folder without importing).")
egui::RichText::new(crate::i18n::tr("Importing scans the source for photos; it doesn't add the folder to Local (use Local → Browse Folder… to work in a folder without importing)."))
.color(t.text_dim)
.small(),
);
}
field(ui, "Transfer", |ui| {
ui.spacing_mut().item_spacing.x = 4.0;
if crate::widgets::text_button(ui, "importAdd", "Add in place", !d.copy).on_hover_text("Reference the files where they are").clicked() {
if crate::widgets::text_button(ui, "importAdd", crate::i18n::tr("Add in place"), !d.copy)
.on_hover_text(crate::i18n::tr("Reference the files where they are"))
.clicked()
{
d.copy = false;
d.move_files = false;
}
let can_copy = app.session.library.as_ref().is_some_and(|l| l.on_disk) || app.services.pick_folder.is_some();
let r = ui.add_enabled_ui(can_copy, |ui| crate::widgets::text_button(ui, "importCopy", "Copy", d.copy && !d.move_files)).inner;
if r.on_hover_text("Copy the files (into the library's Originals/, or a folder you choose)").clicked() {
let r =
ui.add_enabled_ui(can_copy, |ui| crate::widgets::text_button(ui, "importCopy", crate::i18n::tr("Copy"), d.copy && !d.move_files)).inner;
if r.on_hover_text(crate::i18n::tr("Copy the files (into the library's Originals/, or a folder you choose)")).clicked() {
d.copy = true;
d.move_files = false;
}
let r = ui.add_enabled_ui(can_copy, |ui| crate::widgets::text_button(ui, "importMove", "Move", d.copy && d.move_files)).inner;
if r.on_hover_text("Move the files (into the library's Originals/, or a folder you choose), removing them from the source").clicked() {
let r =
ui.add_enabled_ui(can_copy, |ui| crate::widgets::text_button(ui, "importMove", crate::i18n::tr("Move"), d.copy && d.move_files)).inner;
if r.on_hover_text(crate::i18n::tr("Move the files (into the library's Originals/, or a folder you choose), removing them from the source"))
.clicked()
{
d.copy = true;
d.move_files = true;
}
@@ -505,12 +758,12 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, d: &mut ImportDialog) {
let shown = if d.destination.trim().is_empty() { "Library Originals".to_string() } else { d.destination.clone() };
ui.label(egui::RichText::new(shown).color(Tokens::get(ui.ctx()).text_label));
if app.services.pick_folder.is_some()
&& crate::widgets::text_button(ui, "importDest", "Choose…", false).clicked()
&& crate::widgets::text_button(ui, "importDest", crate::i18n::tr("Choose…"), false).clicked()
&& let Some(f) = app.services.pick_folder.as_mut().and_then(|f| f())
{
d.destination = f;
}
if !d.destination.is_empty() && crate::widgets::text_button(ui, "importDestReset", "Library", false).clicked() {
if !d.destination.is_empty() && crate::widgets::text_button(ui, "importDestReset", crate::i18n::tr("Library"), false).clicked() {
d.destination.clear();
}
});
@@ -557,14 +810,16 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, d: &mut ImportDialog) {
None => unknown_tags_warning(ui, &d.folder_template),
}
ui.label(
egui::RichText::new("Each / starts a folder level; tags are filled in per photo (a level with missing metadata is \"unknown\").")
.color(t.text_dim)
.small(),
egui::RichText::new(crate::i18n::tr(
"Each / starts a folder level; tags are filled in per photo (a level with missing metadata is \"unknown\").",
))
.color(t.text_dim)
.small(),
);
}
if !d.move_files {
field(ui, "Raw files", |ui| {
let r = ui.checkbox(&mut d.dng, "Copy as DNG");
let r = ui.checkbox(&mut d.dng, crate::i18n::tr("Copy as DNG"));
register(ui.ctx(), "check:importDng", r.rect);
});
}
@@ -582,10 +837,12 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, d: &mut ImportDialog) {
unknown_tags_warning(ui, &d.rename);
if let Some(example) = example_destination(app, d) {
let t = Tokens::get(ui.ctx());
let r = ui.label(egui::RichText::new(format!("Example: {example}")).color(t.text_dim));
let r = ui.label(egui::RichText::new(crate::i18n::tr_format!("Example: {example}", example = example)).color(t.text_dim));
register(ui.ctx(), "label:importExample", r.rect);
ui.label(
egui::RichText::new("Folders and {date} use the capture time; a photo without one uses today's date.").color(t.text_dim).small(),
egui::RichText::new(crate::i18n::tr("Folders and {date} use the capture time; a photo without one uses today's date."))
.color(t.text_dim)
.small(),
);
}
}
@@ -602,11 +859,11 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, d: &mut ImportDialog) {
None => "None".into(),
};
egui::ComboBox::from_id_salt("import-album").selected_text(cur).show_ui(ui, |ui| {
if ui.selectable_label(d.album.is_none() && d.new_album.is_empty(), "None").clicked() {
if ui.selectable_label(d.album.is_none() && d.new_album.is_empty(), crate::i18n::tr("None")).clicked() {
d.album = None;
d.new_album.clear();
}
if ui.selectable_label(d.album.is_none() && !d.new_album.is_empty(), "New album…").clicked() {
if ui.selectable_label(d.album.is_none() && !d.new_album.is_empty(), crate::i18n::tr("New album…")).clicked() {
d.album = None;
if d.new_album.is_empty() {
d.new_album = "Imported Photos".into();
@@ -627,7 +884,7 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, d: &mut ImportDialog) {
field(ui, "Preset", |ui| {
let cur = app.session.presets.iter().find(|p| p.id == d.preset).map(|p| p.name.clone()).unwrap_or_else(|| "None".into());
egui::ComboBox::from_id_salt("import-preset").selected_text(cur).height(300.0).show_ui(ui, |ui| {
if ui.selectable_label(d.preset.is_empty(), "None").clicked() {
if ui.selectable_label(d.preset.is_empty(), crate::i18n::tr("None")).clicked() {
d.preset.clear();
}
for p in &app.session.presets {
@@ -641,7 +898,7 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, d: &mut ImportDialog) {
field(ui, "Metadata", |ui| {
let cur = if d.metadata_preset.is_empty() { "None".to_string() } else { d.metadata_preset.clone() };
egui::ComboBox::from_id_salt("import-metadata").selected_text(cur).show_ui(ui, |ui| {
if ui.selectable_label(d.metadata_preset.is_empty(), "None").clicked() {
if ui.selectable_label(d.metadata_preset.is_empty(), crate::i18n::tr("None")).clicked() {
d.metadata_preset.clear();
}
for m in &app.session.metadata_presets {
@@ -653,7 +910,7 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, d: &mut ImportDialog) {
});
}
field(ui, "Keywords", |ui| {
let r = ui.add(egui::TextEdit::singleline(&mut d.keywords).hint_text("comma, separated").desired_width(f32::INFINITY));
let r = ui.add(egui::TextEdit::singleline(&mut d.keywords).hint_text(crate::i18n::tr("comma, separated")).desired_width(f32::INFINITY));
register(ui.ctx(), "field:importKeywords", r.rect);
});
}
@@ -726,7 +983,8 @@ fn candidate_cell(app: &mut LightcraftApp, ui: &mut egui::Ui, d: &mut ImportDial
pub(crate) fn tag_toggle(ui: &mut egui::Ui, key: &str) -> bool {
let id = egui::Id::new((key, "tags-open"));
let mut open = ui.ctx().data(|d| d.get_temp::<bool>(id)).unwrap_or(false);
let r = crate::widgets::text_button(ui, &format!("{key}Tags"), "Tags", open).on_hover_text("Show the template tags; click one to insert it");
let r = crate::widgets::text_button(ui, &format!("{key}Tags"), crate::i18n::tr("Tags"), open)
.on_hover_text(crate::i18n::tr("Show the template tags; click one to insert it"));
if r.clicked() {
open = !open;
ui.ctx().data_mut(|d| d.insert_temp(id, open));
@@ -770,7 +1028,9 @@ pub(crate) fn tag_help(ui: &mut egui::Ui, key: &str, text: &mut String, edit_id:
let t = Tokens::get(ui.ctx());
egui::Frame::new().fill(t.inset).corner_radius(4.0).inner_margin(6.0).show(ui, |ui| {
ui.label(
egui::RichText::new("Click a tag to insert it. Examples are for IMG_0042.CR3 taken 14 Jan 2026, 05:58:48.").color(t.text_dim).small(),
egui::RichText::new(crate::i18n::tr("Click a tag to insert it. Examples are for IMG_0042.CR3 taken 14 Jan 2026, 05:58:48."))
.color(t.text_dim)
.small(),
);
egui::ScrollArea::vertical().id_salt((key, "tags")).max_height(170.0).show(ui, |ui| {
egui::Grid::new((key, "tag-grid")).num_columns(3).spacing([10.0, 2.0]).show(ui, |ui| {
@@ -848,9 +1108,9 @@ pub fn source_summary(sources: &[String]) -> String {
many => {
let folders = many.iter().filter(|p| std::path::Path::new(p.as_str()).is_dir()).count();
match folders {
0 => format!("{} files", many.len()),
f if f == many.len() => format!("{f} folders (and their subfolders)"),
f => format!("{} files and {f} folder{}", many.len() - f, if f == 1 { "" } else { "s" }),
0 => crate::i18n::tr_format!("{} files", many.len()),
f if f == many.len() => crate::i18n::tr_format!("{f} folders (and their subfolders)", f = f),
f => crate::i18n::tr_format!("{} files and {f} folder{}", many.len() - f, if f == 1 { "" } else { "s" }, f = f),
}
}
}
+94 -13
View File
@@ -21,6 +21,7 @@ pub mod render;
pub mod shortcuts;
pub mod softpaint;
pub mod state;
pub mod tasks;
pub mod theme;
pub mod widgets;
@@ -29,10 +30,16 @@ mod tests_curve;
#[cfg(test)]
mod tests_grid;
#[cfg(test)]
mod tests_library_problem;
#[cfg(test)]
mod tests_masking;
#[cfg(test)]
mod tests_offline;
#[cfg(test)]
mod tests_panels;
#[cfg(test)]
mod tests_quit_unsaved;
#[cfg(test)]
mod tests_scroll;
#[cfg(test)]
mod tests_unsaved;
@@ -60,6 +67,9 @@ pub type RevealFn = Box<dyn FnMut(&str) -> Result<(), String>>;
pub type OpenUrlFn = Box<dyn FnMut(&str) -> Result<(), String>>;
/// Open a file in an application (`app` = "" for the system's default one).
pub type OpenWithFn = Box<dyn FnMut(&str, &str) -> Result<(), String>>;
/// Something the host does with the session (web: back up or restore the library in browser
/// storage); the work may finish asynchronously.
pub type HostAction = Box<dyn FnMut(&mut Session) -> Result<Value, String>>;
/// Platform services injected by the host app (desktop or web).
#[derive(Default)]
@@ -89,6 +99,12 @@ pub struct Services {
pub open_url: Option<OpenUrlFn>,
/// Open a file in an external editor (Edit in External Editor; desktop only).
pub open_with: Option<OpenWithFn>,
/// File ▸ Back Up Library…: save the whole library (catalog and originals) as one file the
/// user keeps (web only: there the library lives in browser storage, which the browser may
/// clear; on the desktop it is a folder backed up like any other).
pub backup_library: Option<HostAction>,
/// File ▸ Restore Library from Backup… (web only; keeps the current library).
pub restore_library: Option<HostAction>,
}
#[derive(Clone, Copy, Debug, Default)]
@@ -123,6 +139,12 @@ pub struct LightcraftApp {
pub native_shortcuts: std::collections::HashSet<String>,
/// The host is [`headless::Headless`] (it answers viewport screenshot commands itself).
pub headless_host: bool,
/// Warnings to show one at a time (damaged settings files…, issue #103).
pub notices: Vec<String>,
/// Quitting was stopped because changes couldn't be saved: the prompt's text.
pub quit_prompt: Option<String>,
/// Quit Anyway was chosen: the window may close with unsaved changes.
pub quit_confirmed: bool,
control_rx: Option<Receiver<ControlRequest>>,
pending_screenshots: Vec<PendingShot>,
screenshot_token: u64,
@@ -159,6 +181,8 @@ pub struct LightcraftApp {
pub scan: Option<import::ScanTask>,
/// A background export in progress.
pub export: Option<export_task::ExportTask>,
/// Background file-system work of other commands (Find Missing Photos, auto import…).
pub tasks: tasks::Tasks,
/// The files of the last finished background export (`ui.inspect` → `export.last`).
pub last_export_result: Option<Value>,
/// The look the loupe shows while the pointer rests on a preset or profile (set by the
@@ -170,12 +194,13 @@ pub struct LightcraftApp {
gpu_applied: Option<bool>,
/// The memory budget setting last applied (MB, 0 = automatic).
memory_applied: Option<u32>,
/// The library failed to open at launch: the blocking window, then the temporary-session
/// banner (issue #100). Cleared once a library opens.
pub library_problem: Option<panels::library_problem::LibraryProblem>,
}
impl LightcraftApp {
pub fn new(session: Session, services: Services) -> Self {
// GPU device + kernels off the UI thread, before the first photo is opened
lightcraft_engine::gpu::warm_up();
Self {
session,
ui: UiState::default(),
@@ -187,6 +212,9 @@ impl LightcraftApp {
native_menu: false,
native_shortcuts: Default::default(),
headless_host: false,
notices: vec![],
quit_prompt: None,
quit_confirmed: false,
control_rx: None,
pending_screenshots: vec![],
screenshot_token: 0,
@@ -208,11 +236,13 @@ impl LightcraftApp {
import: None,
scan: None,
export: None,
tasks: Default::default(),
last_export_result: None,
hover_preview: None,
window_is_fullscreen: false,
gpu_applied: None,
memory_applied: None,
library_problem: None,
}
}
@@ -269,7 +299,12 @@ impl LightcraftApp {
if self.ui.preview_build_seen != Some((key, true)) {
self.ui.preview_build_seen = Some((key, true));
let (done, failed) = (b.done.load(Ordering::Relaxed), b.failed.load(Ordering::Relaxed));
let mut msg = format!("Previews ready for {done} photo{}", if done == 1 { "" } else { "s" });
let plural = if done == 1 { "" } else { "s" };
let mut msg = match (b.error(), b.what) {
(Some(e), what) => format!("{}: {e}", if what.is_empty() { "previews" } else { what }),
(None, "") => format!("Previews ready for {done} photo{plural}"),
(None, what) => format!("Done ({what}): {done} photo{plural}"),
};
if failed > 0 {
msg.push_str(&format!(" · {failed} couldn't be rendered"));
}
@@ -278,7 +313,11 @@ impl LightcraftApp {
} else {
if self.ui.preview_build_seen != Some((key, false)) {
self.ui.preview_build_seen = Some((key, false));
self.toast(ctx, format!("Building previews for {} photos…", b.total));
let msg = match b.what {
"" => format!("Building previews for {} photos…", b.total),
what => format!("Working on {what} for {} photos…", b.total),
};
self.toast(ctx, msg);
}
ctx.request_repaint_after(std::time::Duration::from_millis(250));
}
@@ -439,7 +478,9 @@ impl LightcraftApp {
/// Per-frame logic before layout (control channel, renders, shortcuts, drops).
pub fn logic(&mut self, ctx: &egui::Context) {
i18n::set_language(self.ui.language);
let t0 = now_ms();
panels::library_problem::logic(self);
self.logic_inner(ctx);
self.perf.logic_ms = now_ms() - t0;
}
@@ -451,10 +492,19 @@ impl LightcraftApp {
// File → Add from Device lists cards scanned in the background: show hot-plugs
let repaint = ctx.clone();
lightcraft_engine::devices::on_change(move || repaint.request_repaint());
// "is the original there?" (grid, Info panel, Missing Photos) answers from a cache a
// worker fills: a sleeping NAS or a dropped share never stalls a frame
let repaint = ctx.clone();
self.session.media.availability.run_in_background(std::sync::Arc::new(move || repaint.request_repaint()));
self.styled = true;
} else {
self.fonts_ready = true;
}
panels::notices::logic(self);
// closing the window (or Quit) with changes only in memory: retry, else ask first
if ctx.input(|i| i.viewport().close_requested()) && !panels::notices::may_close(self) {
ctx.send_viewport_cmd(egui::ViewportCommand::CancelClose);
}
let now = ctx.input(|i| i.time);
let dt = now - self.last_time;
if dt > 0.0 {
@@ -470,6 +520,7 @@ impl LightcraftApp {
merge::poll(self, ctx);
import::poll_scan(self, ctx);
import::tick(self, ctx);
tasks::poll(self, ctx);
self.preview_build_status(ctx);
self.save_status(ctx);
self.slideshow_tick(ctx);
@@ -484,17 +535,33 @@ impl LightcraftApp {
}
}
self.ui.was_focused = focused;
// auto import: scan the watched folder every few seconds
// auto import: list the watched folder every few seconds (on a worker thread: it may be on
// a network share), then import what's new like any import (also on a worker thread)
#[cfg(not(target_arch = "wasm32"))]
if self.session.import_defaults.auto_folder.is_some() {
if let Some(folder) = self.session.import_defaults.auto_folder.clone() {
const LABEL: &str = "Auto Import";
let now = ctx.input(|i| i.time);
if now - self.ui.auto_import_at >= 3.0 {
if now - self.ui.auto_import_at >= 3.0 && self.import.is_none() && !self.tasks.is_running(LABEL) {
self.ui.auto_import_at = now;
if let Ok(r) = self.session.execute("library.autoImportScan", &serde_json::json!({})) {
let n = r["imported"].as_array().map_or(0, Vec::len);
if n > 0 {
self.toast(ctx, format!("Auto Import: added {n} photo{}", if n == 1 { "" } else { "s" }));
let work = move || lightcraft_engine::cmd::library::list_auto_import_folder(&folder);
let done = |app: &mut LightcraftApp, _ctx: &egui::Context, listing: Result<Vec<(String, u64)>, String>| {
let listing = match listing {
Ok(l) => l,
Err(e) => return log::debug!("auto import: {e}"),
};
let p = serde_json::json!({"listing": listing, "start": false});
let Ok(r) = app.session.execute("library.autoImportScan", &p) else { return };
let Some(mut params) = r.get("import").cloned().filter(|_| app.import.is_none()) else { return };
let paths: Vec<String> =
params["paths"].as_array().into_iter().flatten().filter_map(|v| v.as_str().map(str::to_string)).collect();
if let Some(o) = params.as_object_mut() {
o.remove("paths");
}
let undo0 = app.session.undo.len();
app.import = Some(import::ImportTask::new(paths, params, undo0, false).auto());
};
if let Err(e) = tasks::spawn(self, LABEL, work, done) {
log::warn!("{e}");
}
}
ctx.request_repaint_after(std::time::Duration::from_secs(3));
@@ -514,8 +581,11 @@ impl LightcraftApp {
if !presets.is_empty() {
let _ = self.run("file.importPresets", serde_json::json!({"paths": presets}));
}
if !photos.is_empty() {
let _ = self.run("library.import", serde_json::json!({"paths": photos}));
// read and added on a worker thread (dropped folders can be large, or on a slow drive)
if !photos.is_empty()
&& let Err(e) = import::start_paths(self, photos)
{
self.toast(ctx, e);
}
}
}
@@ -525,6 +595,11 @@ impl LightcraftApp {
if self.gpu_applied != Some(self.ui.settings.gpu) {
self.gpu_applied = Some(self.ui.settings.gpu);
let _ = self.session.execute("app.gpu", &serde_json::json!({"enabled": self.ui.settings.gpu}));
// GPU device + kernels off the UI thread, once the window is up and only when GPU
// rendering is on: a broken driver must not keep the window from appearing (issue #136)
if self.ui.settings.gpu {
lightcraft_engine::gpu::warm_up();
}
}
let mb = self.ui.settings.memory_mb;
// automatic at startup: leave the engine's default alone
@@ -586,6 +661,7 @@ impl LightcraftApp {
/// Lay out the whole window.
pub fn ui(&mut self, ui: &mut egui::Ui) {
i18n::set_language(self.ui.language);
let ctx = ui.ctx().clone();
if !self.fonts_ready {
ctx.request_repaint();
@@ -601,7 +677,9 @@ impl LightcraftApp {
// full-screen preview: the photo alone on black
egui::CentralPanel::default().frame(egui::Frame::NONE.fill(egui::Color32::BLACK)).show(ui, |ui| panels::detail::show(self, ui));
panels::second::show(self, &ctx);
panels::notices::show(self, &ctx);
panels::dialogs::show(self, &ctx);
panels::library_problem::show(self, &ctx);
panels::toast(self, &ctx);
self.widgets = widgets::take_registry(&ctx);
self.end_frame(t0);
@@ -610,6 +688,7 @@ impl LightcraftApp {
// Order matters: earlier panels take the full edge (top bar spans the window; the tool strip,
// right panels and left panel run to the bottom; the bottom bar sits between them).
panels::topbar::show(self, ui);
panels::library_problem::banner(self, ui);
panels::strip::show(self, ui);
if self.ui.right != state::RightPanel::None {
panels::right::show(self, ui);
@@ -636,7 +715,9 @@ impl LightcraftApp {
state::ViewMode::Reference => panels::compare::show_reference(self, ui),
});
panels::second::show(self, &ctx);
panels::notices::show(self, &ctx);
panels::dialogs::show(self, &ctx);
panels::library_problem::show(self, &ctx);
import::progress(self, &ctx);
import::scan_progress(self, &ctx);
export_task::poll(self, &ctx);
+89 -42
View File
@@ -43,6 +43,15 @@ impl MenuNode {
}
}
/// User-defined names in parameterized menus are data, not message keys.
pub fn display_item_label<'a>(id: &str, params: &Value, label: &'a str) -> &'a str {
if matches!(id, "metadata.applyPreset" | "album.addPhotos" | "label.applySet") || (id == "app.export" && params.get("preset").is_some()) {
label
} else {
crate::i18n::tr(label)
}
}
/// Top-level menus in order (the macOS app menu is added by the host).
pub const MENUS: &[&str] = &["File", "Edit", "View", "Photo", "Window", "Help"];
@@ -58,6 +67,8 @@ const LAYOUT: &[(&str, &[&str])] = &[
"@Import from Device",
"---",
"app.openLibrary",
"file.backupLibrary",
"file.restoreLibrary",
"---",
"dialog.newAlbum",
"dialog.newFolder",
@@ -241,6 +252,15 @@ const HIDDEN: &[&str] = &[
"preset.create",
];
/// Items only some hosts have are left out of the others' menus (the web build's library backup).
fn host_supports(app: &LightcraftApp, id: &str) -> bool {
match id {
"file.backupLibrary" => app.services.backup_library.is_some(),
"file.restoreLibrary" => app.services.restore_library.is_some(),
_ => true,
}
}
fn item(id: &str, params: Value, label: impl Into<String>, shortcut: Option<&str>, enabled: bool, checked: Option<bool>) -> MenuNode {
MenuNode::Item { id: id.into(), params, label: label.into(), shortcut: shortcut.map(str::to_string), enabled, checked }
}
@@ -296,11 +316,15 @@ pub fn checked(app: &LightcraftApp, id: &str) -> Option<bool> {
fn live_label(app: &LightcraftApp, id: &str, label: &str) -> String {
let n = app.session.selection.ids.len();
match id {
"edit.undo" => app.session.undo.last().map(|e| format!("Undo {}", e.label)).unwrap_or_else(|| "Undo".into()),
"edit.redo" => app.session.redo.last().map(|e| format!("Redo {}", e.label)).unwrap_or_else(|| "Redo".into()),
"photo.delete" if n > 1 => format!("Delete {n} Photos"),
"photo.virtualCopy" if n > 1 => format!("Create {n} Virtual Copies"),
"dialog.rename" if n > 1 => format!("Rename {n} Photos…"),
"edit.undo" => {
app.session.undo.last().map(|e| crate::i18n::tr_format!("Undo {}", crate::i18n::tr(&e.label))).unwrap_or_else(|| "Undo".into())
}
"edit.redo" => {
app.session.redo.last().map(|e| crate::i18n::tr_format!("Redo {}", crate::i18n::tr(&e.label))).unwrap_or_else(|| "Redo".into())
}
"photo.delete" if n > 1 => crate::i18n::tr_format!("Delete {n} Photos", n = n),
"photo.virtualCopy" if n > 1 => crate::i18n::tr_format!("Create {n} Virtual Copies", n = n),
"dialog.rename" if n > 1 => crate::i18n::tr_format!("Rename {n} Photos…", n = n),
_ => label.to_string(),
}
}
@@ -325,20 +349,21 @@ fn expanded(app: &LightcraftApp, name: &str) -> Option<Vec<MenuNode>> {
item("library.buildPreviews", json!({"size": "full"}), format!("Build 1:1 Previews ({scope})"), None, !running, None),
item("library.cancelPreviews", Value::Null, "Stop Building Previews", None, running, None),
MenuNode::Separator,
// (read and written on a worker thread: the originals may be on a slow drive)
item(
"library.smartPreviews",
Value::Null,
json!({"background": true}),
format!("Build Smart Previews ({scope})"),
None,
app.session.media.smart_dir.is_some(),
app.session.media.smart_dir.is_some() && !running,
None,
),
item(
"library.smartPreviews",
json!({"discard": true}),
json!({"discard": true, "background": true}),
format!("Discard Smart Previews ({scope})"),
None,
app.session.media.smart_dir.is_some(),
app.session.media.smart_dir.is_some() && !running,
None,
),
MenuNode::Separator,
@@ -464,14 +489,21 @@ fn expanded(app: &LightcraftApp, name: &str) -> Option<Vec<MenuNode>> {
.map(|(f, label)| item("library.selectBy", json!({"flag": f}), label, None, true, None))
.collect();
v.push(MenuNode::Separator);
v.extend(
(1..=5u8).map(|r| item("library.selectBy", json!({"rating": r}), format!("{} and higher", "★".repeat(r as usize)), None, true, None)),
);
v.extend((1..=5u8).map(|r| {
item("library.selectBy", json!({"rating": r}), crate::i18n::tr_format!("{} and higher", "★".repeat(r as usize)), None, true, None)
}));
v.push(item("library.selectBy", json!({"rating": 0, "ratingOp": "eq"}), "Unrated", None, true, None));
v.push(MenuNode::Separator);
v.extend(lightcraft_catalog::ColorLabel::ALL.iter().map(|l| {
let name = format!("{l:?}");
item("library.selectBy", json!({"label": name.to_lowercase()}), format!("{name} Label"), None, true, None)
item(
"library.selectBy",
json!({"label": name.to_lowercase()}),
crate::i18n::tr_format!("{name} Label", name = name),
None,
true,
None,
)
}));
v
}
@@ -537,7 +569,8 @@ fn tidy(v: Vec<MenuNode>) -> Vec<MenuNode> {
/// The whole menu bar: (title, items) per menu in [`MENUS`] order.
pub fn menu_bar(app: &LightcraftApp) -> Vec<(String, Vec<MenuNode>)> {
let entries: Vec<MenuEntry> = crate::menus::menu_entries(app).into_iter().filter(|e| !HIDDEN.contains(&e.id.as_str())).collect();
let entries: Vec<MenuEntry> =
crate::menus::menu_entries(app).into_iter().filter(|e| !HIDDEN.contains(&e.id.as_str()) && host_supports(app, &e.id)).collect();
let mut used = vec![false; entries.len()];
let mut bar = Vec::new();
for title in MENUS {
@@ -595,22 +628,6 @@ pub fn menu_bar(app: &LightcraftApp) -> Vec<(String, Vec<MenuNode>)> {
}
bar.push((title.to_string(), tidy(items)));
}
fn translate(nodes: &mut [MenuNode], language: crate::i18n::Language) {
for node in nodes {
match node {
MenuNode::Submenu { label, children } => {
*label = language.tr(label).to_string();
translate(children, language);
}
MenuNode::Item { label, .. } => *label = language.tr(label).to_string(),
MenuNode::Separator => {}
}
}
}
for (title, nodes) in &mut bar {
*title = app.ui.language.tr(title).to_string();
translate(nodes, app.ui.language);
}
bar
}
@@ -664,9 +681,9 @@ pub fn shortcut_text(sc: &str, mac: bool) -> String {
const TITLE_GAP: f32 = 24.0;
/// Width of the in-window menu bar's titles.
pub fn bar_width(ui: &egui::Ui, language: crate::i18n::Language) -> f32 {
pub fn bar_width(ui: &egui::Ui) -> f32 {
let t = crate::theme::Tokens::get(ui.ctx());
MENUS.iter().map(|m| ui.painter().layout_no_wrap(language.tr(m).to_string(), t.font(13.0), t.text).size().x + TITLE_GAP).sum::<f32>()
MENUS.iter().map(|m| ui.painter().layout_no_wrap(crate::i18n::tr(m).to_string(), t.font(13.0), t.text).size().x + TITLE_GAP).sum::<f32>()
}
/// The in-window menu bar (hosts without a native one): one dropdown per menu, or a single
@@ -675,8 +692,10 @@ pub fn show_in_window(app: &mut LightcraftApp, ui: &mut egui::Ui, max_width: f32
let t = crate::theme::Tokens::get(ui.ctx());
let bar = menu_bar(app);
let font = t.font(13.0);
let widths: Vec<f32> =
bar.iter().map(|(title, _)| ui.painter().layout_no_wrap(title.clone(), font.clone(), t.text).size().x + TITLE_GAP).collect();
let widths: Vec<f32> = bar
.iter()
.map(|(title, _)| ui.painter().layout_no_wrap(crate::i18n::tr(title).to_string(), font.clone(), t.text).size().x + TITLE_GAP)
.collect();
let total: f32 = widths.iter().sum();
let mut clicked: Option<(String, Value)> = None;
let start = ui.cursor().left();
@@ -684,18 +703,18 @@ pub fn show_in_window(app: &mut LightcraftApp, ui: &mut egui::Ui, max_width: f32
if total <= max_width {
let saved = ui.spacing().item_spacing.x;
ui.spacing_mut().item_spacing.x = TITLE_GAP;
for (index, (title, items)) in bar.iter().enumerate() {
let r = ui.add(egui::Button::new(egui::RichText::new(title).font(font.clone()).color(t.text_label)).frame(false));
crate::widgets::register(ui.ctx(), format!("menu:{}", MENUS.get(index).copied().unwrap_or(title.as_str())), r.rect);
for (title, items) in &bar {
let r = ui.add(egui::Button::new(egui::RichText::new(crate::i18n::tr(title)).font(font.clone()).color(t.text_label)).frame(false));
crate::widgets::register(ui.ctx(), format!("menu:{title}"), r.rect);
egui::Popup::menu(&r).show(|ui| nodes_ui(ui, items, mac, &mut clicked));
}
ui.spacing_mut().item_spacing.x = saved;
} else {
let r = ui.add(egui::Button::new(egui::RichText::new(app.ui.language.tr("Menu")).font(font.clone()).color(t.text_label)).frame(false));
let r = ui.add(egui::Button::new(egui::RichText::new(crate::i18n::tr("Menu")).font(font.clone()).color(t.text_label)).frame(false));
crate::widgets::register(ui.ctx(), "menu:all", r.rect);
egui::Popup::menu(&r).show(|ui| {
for (title, items) in &bar {
ui.menu_button(title, |ui| nodes_ui(ui, items, mac, &mut clicked));
ui.menu_button(crate::i18n::tr(title), |ui| nodes_ui(ui, items, mac, &mut clicked));
}
});
}
@@ -713,11 +732,11 @@ fn nodes_ui(ui: &mut egui::Ui, nodes: &[MenuNode], mac: bool, clicked: &mut Opti
ui.separator();
}
MenuNode::Submenu { label, children } => {
ui.menu_button(format!(" {label}"), |ui| nodes_ui(ui, children, mac, clicked));
ui.menu_button(format!(" {}", crate::i18n::tr(label)), |ui| nodes_ui(ui, children, mac, clicked));
}
MenuNode::Item { id, params, label, shortcut, enabled, checked } => {
// a gutter for check marks, like native menus
let mut b = egui::Button::new(format!(" {label}"));
let mut b = egui::Button::new(format!(" {}", display_item_label(id, params, label)));
if let Some(sc) = shortcut {
b = b.shortcut_text(shortcut_text(sc, mac));
}
@@ -774,6 +793,34 @@ mod tests {
assert!(!text.contains("Add Folder") && !text.contains("\"Add Photos"), "no add-folder wording left in the menus");
}
/// Back Up / Restore Library are the browser build's (its library lives in browser storage):
/// absent from the desktop's menus, present and wired to the host where it provides them.
#[test]
fn library_backup_items_follow_the_host() {
let all = |app: &LightcraftApp| -> Vec<MenuNode> { menu_bar(app).into_iter().flat_map(|(_, v)| v).collect() };
let mut desktop = app();
assert!(find(&all(&desktop), "file.backupLibrary").is_none() && find(&all(&desktop), "file.restoreLibrary").is_none());
assert!(run_item(&mut desktop, "file.backupLibrary", Value::Null).is_err(), "not available without the host");
let calls = std::rc::Rc::new(std::cell::Cell::new(0));
let c = calls.clone();
let services = crate::Services {
backup_library: Some(Box::new(move |s: &mut lightcraft_engine::Session| {
c.set(c.get() + 1);
Ok(json!({"photos": s.catalog.len()}))
})),
restore_library: Some(Box::new(|_: &mut lightcraft_engine::Session| Ok(json!({"started": true})))),
..Default::default()
};
let mut web = LightcraftApp::new(lightcraft_engine::Session::with_demo(), services);
let bar = all(&web);
for id in ["file.backupLibrary", "file.restoreLibrary"] {
assert!(matches!(find(&bar, id), Some(MenuNode::Item { enabled: true, .. })), "{id}");
}
let r = run_item(&mut web, "file.backupLibrary", Value::Null).unwrap();
assert!(r["photos"].as_u64().unwrap() > 0);
assert_eq!(calls.get(), 1);
}
#[test]
fn bar_follows_the_registry_and_state() {
let mut app = app();
@@ -936,7 +983,7 @@ mod tests {
let mut app = LightcraftApp::new(lightcraft_engine::Session::new().with_fs(), Default::default());
app.session.execute("library.import", &json!({"paths": [dir.join("a").to_string_lossy()]})).unwrap();
std::fs::rename(dir.join("a/one.png"), dir.join("b/one.png")).unwrap();
let r = run_item(&mut app, "file.findMissing", json!({"folder": dir.join("b").to_string_lossy()})).unwrap();
let r = run_item(&mut app, "file.findMissing", json!({"folder": dir.join("b").to_string_lossy(), "wait": true})).unwrap();
assert_eq!(r["found"].as_array().map(Vec::len), Some(1), "{r}");
// Locate: an explicit file
std::fs::rename(dir.join("b/one.png"), dir.join("one-renamed.png")).unwrap();
+59 -20
View File
@@ -116,6 +116,8 @@ pub const UI_COMMANDS: &[UiCommand] = &[
("file.addFolder", "Import from Folder…", None, "File"),
("file.addFromDevice", "Import from Device", None, ""),
("file.findMissing", "Find Missing Photos…", None, "File"),
("file.backupLibrary", "Back Up Library…", None, "File"),
("file.restoreLibrary", "Restore Library from Backup…", None, "File"),
("photo.locate", "Locate Missing File…", None, ""),
("dialog.saveMetadataPreset", "Save Metadata Preset…", None, ""),
("app.quit", "Quit LightCraft", Some("Cmd+Q"), "File"),
@@ -149,10 +151,10 @@ pub const UI_COMMANDS: &[UiCommand] = &[
fn panel(app: &mut LightcraftApp, ctx: &egui::Context, p: RightPanel, name: &str) {
if app.ui.right == p {
app.ui.right = RightPanel::None;
app.toast(ctx, format!("{name} Off"));
app.toast(ctx, crate::i18n::tr_format!("{name} Off", name = name));
} else {
app.ui.right = p;
app.toast(ctx, format!("{name} On"));
app.toast(ctx, crate::i18n::tr_format!("{name} On", name = name));
if p.is_edit_tool() && !matches!(app.ui.view, ViewMode::Detail) {
app.ui.view = ViewMode::Detail;
}
@@ -381,6 +383,13 @@ pub fn run_ui_command(app: &mut LightcraftApp, id: &str, p: &Value) -> Option<Re
Ok(Value::Null)
}
"app.openLibrary" => crate::panels::settings::open_library(app, p),
"file.backupLibrary" | "file.restoreLibrary" => {
let action = if id == "file.backupLibrary" { app.services.backup_library.as_mut() } else { app.services.restore_library.as_mut() };
match action {
Some(f) => f(&mut app.session),
None => Err("not available here: on the desktop the library is a folder; back it up with your other files".into()),
}
}
"view.filmstrip" => {
app.ui.filmstrip = !app.ui.filmstrip;
Ok(Value::Null)
@@ -865,7 +874,7 @@ pub fn run_ui_command(app: &mut LightcraftApp, id: &str, p: &Value) -> Option<Re
(
"GPU".to_string(),
gpu.unwrap_or_else(|| match lightcraft_engine::gpu::unavailable_reason() {
Some(why) => format!("none (CPU rendering): {why}"),
Some(why) => crate::i18n::tr_format!("none (CPU rendering): {why}", why = why),
None => "none (CPU rendering)".into(),
}),
),
@@ -928,18 +937,40 @@ pub fn run_ui_command(app: &mut LightcraftApp, id: &str, p: &Value) -> Option<Re
None => app.services.pick_folder.as_mut().and_then(|f| f()),
};
let Some(folder) = folder else { return Some(Ok(Value::Null)) };
let r = app.run("library.findMissing", json!({"folder": folder}));
if let Ok(v) = &r {
let n = v["found"].as_array().map_or(0, Vec::len);
let left = v["missing"].as_u64().unwrap_or(0);
let unsure = v["ambiguous"].as_array().map_or(0, Vec::len);
let unsure = if unsure > 0 { format!(" ({unsure} with several look-alike files: use Locate)") } else { String::new() };
app.toast(
&egui::Context::default(),
format!("Found {n} missing photo{}; {left} still missing{unsure}", if n == 1 { "" } else { "s" }),
);
// the search (checking every photo's file, walking the folder) runs on a worker thread;
// the relinking happens back here, as one undo step
const LABEL: &str = "Find Missing Photos";
if app.tasks.is_running(LABEL) {
return Some(Err("Find Missing Photos is already searching".into()));
}
r
let candidates = lightcraft_engine::cmd::missing::find_candidates(&app.session.catalog);
let work = move || lightcraft_engine::cmd::missing::plan_find_missing(&candidates, &folder);
let done = |app: &mut LightcraftApp, ctx: &egui::Context, plan: Result<lightcraft_engine::cmd::missing::FindPlan, String>| {
// relinked under the session as it is now: photos relinked meanwhile and files
// now in use are skipped
let r = plan.and_then(|plan| app.run("library.findMissing", plan.to_json()));
match r {
Ok(v) => {
let n = v["found"].as_array().map_or(0, Vec::len);
let left = v["missing"].as_u64().unwrap_or(0);
let unsure = v["ambiguous"].as_array().map_or(0, Vec::len);
let unsure = if unsure > 0 { format!(" ({unsure} with several look-alike files: use Locate)") } else { String::new() };
app.toast(ctx, format!("Found {n} missing photo{}; {left} still missing{unsure}", if n == 1 { "" } else { "s" }));
app.ui.last_find_missing = Some(v);
}
Err(e) => app.toast(ctx, e),
}
};
app.ui.last_find_missing = None;
if let Err(e) = crate::tasks::spawn(app, LABEL, work, done) {
return Some(Err(e));
}
if p.get("wait").and_then(Value::as_bool).unwrap_or(false) {
let ctx = app.tasks.repaint.clone().unwrap_or_default();
crate::tasks::wait(app, &ctx, std::time::Duration::from_secs(600));
return Some(Ok(app.ui.last_find_missing.clone().unwrap_or(Value::Null)));
}
Ok(json!({"background": true}))
}
"photo.tagFromTracklog" => {
// a GPX file → GPS for the selected photos by capture time (one undo step)
@@ -1064,8 +1095,14 @@ pub fn run_ui_command(app: &mut LightcraftApp, id: &str, p: &Value) -> Option<Re
let failed = v["failed"].as_array().map_or(0, Vec::len);
let mut msg = match (n, failed) {
(0, 0) => "No new presets".to_string(),
(n, 0) => format!("Imported {n} preset{}", if n == 1 { "" } else { "s" }),
(n, f) => format!("Imported {n} preset{}, {f} file{} not readable", if n == 1 { "" } else { "s" }, if f == 1 { "" } else { "s" }),
(n, 0) => crate::i18n::tr_format!("Imported {n} preset{}", if n == 1 { "" } else { "s" }, n = n),
(n, f) => crate::i18n::tr_format!(
"Imported {n} preset{}, {f} file{} not readable",
if n == 1 { "" } else { "s" },
if f == 1 { "" } else { "s" },
f = f,
n = n
),
};
// settings with no counterpart here (the other editor's profiles, masks…)
let mut skipped: Vec<&str> = v["imported"]
@@ -1110,7 +1147,7 @@ pub fn run_ui_command(app: &mut LightcraftApp, id: &str, p: &Value) -> Option<Re
}
let r = app.session.execute("preset.export", &params).map_err(|e| e.to_string());
if let Ok(v) = &r {
app.toast(&ctx, format!("Exported {} preset{}", v["count"], if v["count"] == 1 { "" } else { "s" }));
app.toast(&ctx, crate::i18n::tr_format!("Exported {} preset{}", v["count"], if v["count"] == 1 { "" } else { "s" }));
}
return Some(r);
}
@@ -1129,9 +1166,9 @@ pub fn run_ui_command(app: &mut LightcraftApp, id: &str, p: &Value) -> Option<Re
if let Ok(v) = &r {
let n = v["imported"].as_array().map_or(0, Vec::len);
let failed = v["failed"].as_array().map_or(0, Vec::len);
let mut msg = format!("Imported {n} point curve preset{}", if n == 1 { "" } else { "s" });
let mut msg = crate::i18n::tr_format!("Imported {n} point curve preset{}", if n == 1 { "" } else { "s" }, n = n);
if failed > 0 {
msg += &format!(", {failed} file{} not readable", if failed == 1 { "" } else { "s" });
msg += &crate::i18n::tr_format!(", {failed} file{} not readable", if failed == 1 { "" } else { "s" }, failed = failed);
}
app.toast(&ctx, msg);
}
@@ -1152,7 +1189,7 @@ pub fn run_ui_command(app: &mut LightcraftApp, id: &str, p: &Value) -> Option<Re
}
let r = app.session.execute("curve.exportPresets", &params).map_err(|e| e.to_string());
if let Ok(v) = &r {
app.toast(&ctx, format!("Exported {} point curve preset{}", v["count"], if v["count"] == 1 { "" } else { "s" }));
app.toast(&ctx, crate::i18n::tr_format!("Exported {} point curve preset{}", v["count"], if v["count"] == 1 { "" } else { "s" }));
}
return Some(r);
}
@@ -1193,6 +1230,8 @@ pub fn ui_enabled(app: &LightcraftApp, id: &str) -> bool {
"view.compare" => app.session.catalog.len() > 1,
"view.fullScreenPreview" | "view.infoOverlay" | "view.navigator" => app.session.active().is_some() || app.ui.fullscreen,
"app.openLibrary" | "file.addFolder" => app.services.pick_folder.is_some(),
"file.backupLibrary" => app.services.backup_library.is_some(),
"file.restoreLibrary" => app.services.restore_library.is_some(),
"compare.swap" | "compare.makeSelect" => app.ui.view == ViewMode::Compare,
s if s.starts_with("dialog.merge") || (s.starts_with("merge.") && s.ends_with("Last")) => {
app.session.targets(&serde_json::json!({})).len() >= 2 && app.merge.final_task.is_none()
+13 -13
View File
@@ -257,18 +257,18 @@ pub fn poll(app: &mut LightcraftApp, ctx: &egui::Context) {
match r.map_err(lightcraft_engine::EngineError::Other).and_then(|out| app.session.finish_merge(&t.job, out)) {
Ok(v) => {
app.merge.last_result = Some(v);
app.toast(ctx, format!("{what} merge added"));
app.toast(ctx, crate::i18n::tr_format!("{what} merge added", what = what));
}
Err(e) => {
app.ui.status = e.to_string();
app.toast(ctx, format!("{what} merge failed: {e}"));
app.toast(ctx, crate::i18n::tr_format!("{what} merge failed: {e}", e = e, what = what));
}
}
}
Err(_) => {
let (f, stage) = t.progress.lock().map(|g| g.clone()).unwrap_or_default();
let now = ctx.input(|i| i.time);
app.ui.toast = Some((format!("Merging… {stage} {:.0}%", f * 100.0), now + 0.5));
app.ui.toast = Some((crate::i18n::tr_format!("Merging… {stage} {:.0}%", f * 100.0, stage = stage), now + 0.5));
ctx.request_repaint_after(std::time::Duration::from_millis(100));
}
}
@@ -299,7 +299,7 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, opts: &mut MergeDialog)
ui.painter().text(
rect.center_bottom() - egui::vec2(0.0, 14.0),
egui::Align2::CENTER_CENTER,
format!("{stage}… {:.0}%", f * 100.0),
crate::i18n::tr_format!("{stage}… {:.0}%", f * 100.0, stage = stage),
t.font(12.0),
t.text,
);
@@ -330,7 +330,7 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, opts: &mut MergeDialog)
ui.vertical(|ui| {
ui.set_width(220.0);
if opts.is_pano() {
ui.label(egui::RichText::new("Projection").color(t.text_label));
ui.label(egui::RichText::new(crate::i18n::tr("Projection")).color(t.text_label));
for (v, l) in [("auto", "Auto Select"), ("spherical", "Spherical"), ("cylindrical", "Cylindrical"), ("perspective", "Perspective")] {
if ui.radio(opts.projection == v, l).clicked() {
opts.projection = v.to_string();
@@ -341,13 +341,13 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, opts: &mut MergeDialog)
if num(ui, &BOUNDARY_WARP, &mut bw) {
opts.boundary_warp = bw.round();
}
ui.checkbox(&mut opts.fill_edges, "Fill Edges");
ui.checkbox(&mut opts.auto_crop, "Auto Crop");
ui.checkbox(&mut opts.fill_edges, crate::i18n::tr("Fill Edges"));
ui.checkbox(&mut opts.auto_crop, crate::i18n::tr("Auto Crop"));
ui.add_space(4.0);
}
if opts.is_hdr() {
ui.checkbox(&mut opts.align, "Auto Align");
ui.label(egui::RichText::new("Deghost Amount").color(t.text_label));
ui.checkbox(&mut opts.align, crate::i18n::tr("Auto Align"));
ui.label(egui::RichText::new(crate::i18n::tr("Deghost Amount")).color(t.text_label));
ui.horizontal(|ui| {
for (i, (v, l)) in [("none", "None"), ("low", "Low"), ("medium", "Med"), ("high", "High")].iter().enumerate() {
if crate::widgets::text_button(ui, &format!("mergeDeghost-{i}"), l, opts.deghost == *v).clicked() {
@@ -356,7 +356,7 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, opts: &mut MergeDialog)
}
});
if opts.command == "merge.hdr" {
ui.add_enabled_ui(opts.deghost != "none", |ui| ui.checkbox(&mut opts.show_overlay, "Show Deghost Overlay"));
ui.add_enabled_ui(opts.deghost != "none", |ui| ui.checkbox(&mut opts.show_overlay, crate::i18n::tr("Show Deghost Overlay")));
}
if opts.command == "merge.hdrPanorama" {
let mut b = opts.bracket as f64;
@@ -366,10 +366,10 @@ pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, opts: &mut MergeDialog)
}
ui.add_space(4.0);
}
ui.checkbox(&mut opts.auto_settings, "Auto Settings");
ui.checkbox(&mut opts.stack, "Create Stack");
ui.checkbox(&mut opts.auto_settings, crate::i18n::tr("Auto Settings"));
ui.checkbox(&mut opts.stack, crate::i18n::tr("Create Stack"));
ui.add_space(8.0);
ui.label(egui::RichText::new(format!("{} photos", app.merge.ids.len())).color(t.text_dim));
ui.label(egui::RichText::new(crate::i18n::tr_format!("{} photos", app.merge.ids.len())).color(t.text_dim));
});
});
}
+21 -9
View File
@@ -104,7 +104,13 @@ fn centre(app: &mut LightcraftApp, ui: &mut egui::Ui, full: Rect, from: f32, to:
let cresp = ui.interact(copy_r, egui::Id::new("copy-settings"), Sense::click());
register(ui.ctx(), "button:copySettings", copy_r);
ui.painter().rect_filled(copy_r, 15.0, if cresp.hovered() { t.hover } else { t.canvas });
ui.painter().text(copy_r.center(), Align2::CENTER_CENTER, label, t.font(13.0), if active.is_some() { t.text_label } else { t.text_disabled });
ui.painter().text(
copy_r.center(),
Align2::CENTER_CENTER,
crate::i18n::tr(label),
t.font(13.0),
if active.is_some() { t.text_label } else { t.text_disabled },
);
if cresp.clicked() && active.is_some() {
let _ = if has_clip { app.run("develop.paste", json!({})) } else { app.run("develop.copy", json!({})) };
let msg = if has_clip { "Settings pasted" } else { "Edit settings copied" };
@@ -168,7 +174,7 @@ fn right_side(app: &mut LightcraftApp, ui: &mut egui::Ui, full: Rect) -> f32 {
Zoom::Fill => "Fill".to_string(),
Zoom::Percent(p) => format!("{p}%"),
};
let zr = crate::widgets::dropdown(&mut child, "zoom", &zoom_label, t.font(13.0), t.text_label);
let zr = crate::widgets::dropdown(&mut child, "zoom", crate::i18n::tr(&zoom_label), t.font(13.0), t.text_label);
egui::Popup::menu(&zr).show(|ui| {
for (label, z) in [
("Fit", Zoom::Fit),
@@ -185,7 +191,13 @@ fn right_side(app: &mut LightcraftApp, ui: &mut egui::Ui, full: Rect) -> f32 {
}
});
child.add_space(6.0);
let cz = crate::widgets::dropdown(&mut child, "clickZoom", &format!("Click {}:1", app.ui.click_zoom / 100), t.font(13.0), t.text_label);
let cz = crate::widgets::dropdown(
&mut child,
"clickZoom",
&crate::i18n::tr_format!("Click {}:1", app.ui.click_zoom / 100),
t.font(13.0),
t.text_label,
);
egui::Popup::menu(&cz).show(|ui| {
for pct in crate::state::CLICK_ZOOMS {
if ui.selectable_label(app.ui.click_zoom == pct, format!("{}:1", pct / 100)).clicked() {
@@ -216,7 +228,7 @@ fn sort_menu(app: &mut LightcraftApp, ui: &mut egui::Ui) {
use lightcraft_catalog::GroupBy;
use lightcraft_catalog::SortKey::*;
let cur = app.session.sort;
ui.label(egui::RichText::new("Sort by").weak());
ui.label(egui::RichText::new(crate::i18n::tr("Sort by")).weak());
for (label, key, k) in [
("Capture Date", CaptureDate, "captureDate"),
("Import Date", ImportDate, "importDate"),
@@ -225,19 +237,19 @@ fn sort_menu(app: &mut LightcraftApp, ui: &mut egui::Ui) {
("Rating", Rating, "rating"),
("File Size", FileSize, "fileSize"),
] {
if ui.selectable_label(cur.key == key, label).clicked() {
if ui.selectable_label(cur.key == key, crate::i18n::tr(label)).clicked() {
let _ = app.run("library.sort", json!({"key": k}));
}
}
ui.separator();
if ui.selectable_label(cur.ascending, "Ascending").clicked() {
if ui.selectable_label(cur.ascending, crate::i18n::tr("Ascending")).clicked() {
let _ = app.run("library.sort", json!({"ascending": true}));
}
if ui.selectable_label(!cur.ascending, "Descending").clicked() {
if ui.selectable_label(!cur.ascending, crate::i18n::tr("Descending")).clicked() {
let _ = app.run("library.sort", json!({"ascending": false}));
}
ui.separator();
ui.label(egui::RichText::new("Group by date").weak());
ui.label(egui::RichText::new(crate::i18n::tr("Group by date")).weak());
for (label, g, k) in [
("Automatic", GroupBy::Auto, "auto"),
("Day", GroupBy::Day, "day"),
@@ -245,7 +257,7 @@ fn sort_menu(app: &mut LightcraftApp, ui: &mut egui::Ui) {
("Year", GroupBy::Year, "year"),
("None", GroupBy::None, "none"),
] {
if ui.selectable_label(cur.group == g, label).clicked() {
if ui.selectable_label(cur.group == g, crate::i18n::tr(label)).clicked() {
let _ = app.run("library.sort", json!({"group": k}));
}
}
Loaded 100 of 143 files, more files were not shown because too many files have changed in this diff. Show more