Compare commits

..
Author SHA1 Message Date
Brandon ThomasandClaude Opus 5.5 2472021091 Bump version to 0.4.0 (from 0.3.0) for release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:01:45 -04:00
60828c3eaf README Downloads: where to get it first, then builds with Windows before macOS (#306)
Co-authored-by: Brandon Thomas <shared2@storyteller.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 09:50:53 -04:00
ebe54f934e Release: FreeBSD, Flatpak x86_64/aarch64, AppImage zsync updates; README Downloads (#302)
* Release: FreeBSD x86_64, Flatpak x86_64/aarch64, AppImage zsync updates; README Downloads

- linux: appimagetool -u embeds gh-releases-zsync update information; the .AppImage.zsync is
  published beside each AppImage (zsync installed; checked in the job).
- flatpak: new job per arch repackages the Linux tarball into a single-file .flatpak
  (packaging/linux/flatpak-bundle.sh + ai.storyteller.lightcraft.bundle.yml); the Flathub
  manifest gets the CLI and the same commented finish-args; packaging-lint checks they agree.
- freebsd: new job builds lightcraft-<ver>-freebsd-x86_64.tar.gz in a FreeBSD 14.3 VM with the
  freebsd.yml packages (packaging/freebsd/package.sh; layout dry run in packaging-lint).
- version, linux and web no longer use the release environment (they sign nothing), so a
  branch dispatch dry-runs every unsigned job; the release job still needs macOS and Windows.
- README: Downloads section listing every build; docs/releasing.md updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Linux packages: describe LightCraft, not PhotoCraft, in the deb/rpm description

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Brandon Thomas <shared2@storyteller.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 09:45:26 -04:00
Brandon ThomasandClaude Opus 5.5 b8df5f472c Bump version to 0.3.0 (from 0.2.1) for release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 08:12:53 -04:00
Brandon ThomasandClaude Opus 5.5 629e39380e Release CI: macOS job on macos-15-xlarge
Custom macOS larger runners in a runner group do not pick up jobs; the
built-in M2 Pro label does. Only release.yml uses it, and the
actions_macos_xl budget caps the monthly spend.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 22:07:59 -04:00
Brandon ThomasandClaude Opus 5.5 8734e95dc8 Release CI: macOS job on the org macos-release runner
The standard macOS runners are shared with all CI and queue for an hour or
more. The macos-release runner (GitHub-hosted M2 Pro) is in a runner group
usable only by release.yml on the release branch, with a monthly spend cap
on the actions_macos_xl SKU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:54:56 -04:00
Brandon ThomasandClaude Opus 5.5 4159ceb1b0 About: contributor and model credits compiled in (grab bag + table, self-submitted names) (#252)
* About: contributors compiled in from contributors.json (Contributors + Models tabs, grab bag + table, sortable, name toggle)

Adds the craftrules standard (standards/contributors.md) to LightCraft: contributors/contributors.json
is generated by craftrules/scripts/contributors.py and baked into the binary by
crates/ui-egui/build.rs (merged next to the tr_format! generation) as static tables. The About
dialog gets tabs About · Contributors · Models. Each contributor line: @username, opt-in
display/real name, PRs, commits, lines +/−/Δ, binary assets +/−, first/last commit. Sortable by
every column; alphabetical ignores case and the leading @. Tabs and the list view are
control-channel widgets (button:aboutTab-*, button:creditsTable / button:creditsGrabBag).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors docs: craftrules is two levels up (../../craftrules)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors: self-submitted names in contributors/people.toml; AGENTS.md credit-yourself section

build.rs reads contributors/people.toml next to the generated contributors.json and shows a name only
for entries the generator verified as written by that person (craftrules standards/contributors.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors: regenerate (verified people.toml entries)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors: regenerate after rebase onto main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Contributors: regenerate (verified people.toml entries)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 21:22:08 -04:00
Brandon Thomas 2694d7089f Merge pull request #249 from radialmonster/ci-available-ram 2026-10-07 21:06:13 -04:00
Brandon Thomas 88f38c7e2a Merge pull request #248 from andrewyao (web_time in raw preview decode timing) 2026-10-07 21:06:13 -04:00
Brandon Thomas df574a537a Merge PR #213 from Orthic-Labs/orthic/arw-lossless-ycbcr
# Conflicts:
#	crates/ui-egui/src/headless.rs
2026-10-07 20:53:51 -04:00
Brandon Thomas 5bc923b4f9 Merge PR #246 2026-10-07 20:53:29 -04:00
Brandon ThomasandClaude Opus 5.5 71c8382b10 Merge main into #213 (keep main's Windows test fixes from #207; parity row combines RW2 and the downsized ARW)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:53:25 -04:00
Andrew YaoandClaude Opus 5.5 0a952b3536 Web: time raw preview decodes with web_time so wasm doesn't panic
load_bytes_now timed the binned demosaic with std::time::Instant, which
panics on wasm32-unknown-unknown ("time not implemented on this
platform"). Opening a raw in the web build aborted the decode. Use
web_time::Instant, as library.rs already does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 17:53:19 -07:00
radialmonsterandClaude Opus 5.5 f9cfe15db6 CI: pick build jobs from available RAM, not installed RAM
Installed RAM overcommits a busy machine: on a 32 GB desktop with browsers,
a VM and editors open, about 7 GB was free and the installed-RAM rule chose
16 jobs. A fresh build then pushed free RAM below 1 GB.

`cargo xtask ci` now uses one build job per 1.5 GB of RAM available when it
starts (Linux MemAvailable, Windows AvailableMBytes, macOS free + inactive +
speculative pages), capped at the CPU count. With line-table debuginfo,
4 jobs measured about 3 GB in use, so this leaves about half the available
RAM spare. If available RAM can't be read, half the installed RAM is
assumed; if neither can, 4 jobs. Test threads follow, capped at 4.
Explicit CARGO_BUILD_JOBS / RUST_TEST_THREADS still win.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:46:10 -04:00
radialmonsterandClaude Opus 5.5 ed0a2a4f07 GPU tests: drop the edge-pixel tolerance; the coverage mask makes edges match
#207 let up to two edge pixels differ because the GPU's f32 framing could
fall on the other side of the lens-warp boundary. 'GPU: decide warped-geometry
edges with a CPU coverage mask' (this PR) makes the GPU use the CPU's f64
decision, so every case is back within the plain MAX_LSB check (embedded
lens + perspective: max 1 LSB, was 91 before the mask).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:40:47 -04:00
Brandon ThomasandClaude Opus 5.5 0801219b17 Merge #243 (with main) into #246
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:38:46 -04:00
Brandon Thomas e716ede8e4 Merge remote-tracking branch 'origin/main' into HEAD 2026-10-07 20:33:23 -04:00
Brandon ThomasandClaude Opus 5.5 2e48f25997 Merge main into #243 (combine with #207's Windows test fixes)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:33:23 -04:00
Brandon ThomasandClaude Opus 5.5 d716d74af5 Merge main into #243 (combine with #207's Windows test fixes: hide_home_above plus local.addRoot, FakeFs path normalising plus case folding)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:32:47 -04:00
Brandon ThomasandClaude Opus 5.5 a1c5e2dad1 Tests: three headless UI tests wait for the textures and dialog they check
settle() returns after a quiet spell, which on a loaded machine can come before the first grid
thumbnails, the profile browser's variant thumbnails or the async import review (and its
thumbnails) land. demo_grid_snapshot_has_ui_pixels, profile_browser_previews_on_hover_and_applies_on_click
and import_review_dialog failed that way when two test runs shared the machine; they (and the
two other import-review tests) now step until what they assert is there. 2 concurrent runs: 138/138 each.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:32:34 -04:00
radialmonsterandClaude Opus 5.5 c816ca8e35 UI tests: wait for sidebar rows with main's step_until
main gained Headless::step_until for the same job as this branch's
test-only wait_for_widget (run frames until a condition holds). Use the one
helper and drop the duplicate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:23:25 -04:00
radialmonsterandClaude Opus 5.5 62325bd45e Thumbnails: retire the replaced preview cache so its old jobs can't write
Reopening a library (or opening another one, or set_disk_cache_bytes
falling back to attaching) replaced MediaCache::rendered with a new
PreviewCache, but render jobs still in flight kept the old object.
Each PreviewCache checked only its own generation, so after a clear on
the new cache the old one's generation was still current: a finishing
thumbnail job wrote its pixels through it into the directory the clear
had just emptied. The UI rejected the result, but the JPEG stayed.

attach_disk_cache now retires the cache it replaces before swapping it:
PreviewCache::retire takes the generation write lock (so it waits for a
write in progress, like clear) and marks the object retired; every
get/put/put_deferred through it is then refused and its clear() is a
no-op, as its directory may now belong to the replacement. Files on
disk are left alone, so a plain reopen still finds its thumbnails, and
resizing the budget in place keeps the same cache object as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:23:17 -04:00
radialmonsterandClaude Opus 5.5 4d5504cf16 Web: watch the thumbnail cache from worker start so an early clear invalidates
The main thread's Workers tracked no rendered-thumbnail cache until the
first render request carried one. If Clear Preview Cache ran before that
(e.g. starting on an empty filtered view), the per-frame observation in
finished() had nothing to compare with, and the first request only
recorded the already-cleared generation as its baseline. The persisted
ThumbIndex was never invalidated, so once the filter was removed the
request touched the old entry and the worker read the pre-clear JPEG
back from browser storage. Replacing the cache (another library) before
the first request had the same gap.

Workers::start now takes the session's cache and records it, with its
current generation, as the baseline the persisted index belongs to. A
normal start with no clear observes the same cache and generation and
keeps the index. The comparison moves into wire::CacheWatch so it is
tested natively (cargo test --workspace) next to ThumbIndex.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:23:17 -04:00
Brandon Thomas aec820033b Merge pull request #206 from anarion80/add_nix_build
Packaging: add a Nix flake (nix build / nix develop)
2026-10-07 20:15:50 -04:00
Brandon Thomas b90c06b347 Merge pull request #210 from voshart/pr/fetch-crate
Fetch: the model downloader becomes its own crate (no behaviour change)
2026-10-07 20:15:46 -04:00
Brandon Thomas 52f64ce0b9 Merge pull request #215 from XRTce/rw2-panasonic-lumix
LR-IMP-FORMATS: Panasonic RW2 / Leica RWL / Panasonic RAW in every raw format, with the RW2 camera look
2026-10-07 20:15:41 -04:00
Brandon Thomas 767ed15adc Merge pull request #235 from Gremiger/fix/issue-212-nef-dng-white-balance
Issue #212: Raw: keep vendor white balance when writing DNG
2026-10-07 20:15:37 -04:00
Brandon Thomas 1c025fbb97 Merge pull request #207 from voshart/pr/tests-windows
Tests: pass on Windows (path separators, temp folder inside home, one GPU edge pixel)
2026-10-07 20:15:33 -04:00
Brandon ThomasandClaude Opus 5.5 9b41e33a24 Merge main into #213 (keep main's waits in the two flaky UI tests)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:14:41 -04:00
Brandon ThomasandClaude Opus 5.5 0143704c7d Merge #243 (with main) into #246
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:14:28 -04:00
Brandon ThomasandClaude Opus 5.5 9da8163ae2 Merge main into #243 (keep main's settle and step_until in the two flaky UI tests)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:12:10 -04:00
Brandon ThomasandClaude Opus 5.5 46f5d212e7 Merge main into #207 (keep main's step_until wait for the folder row)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:11:00 -04:00
Brandon Thomas eba0b9723b Merge pull request #223 from Ndsanes/l10n/zh-hans-catalog-gap
LR-BEHAV-LOCALIZE: fill the zh-hans catalog to match zh-hant
2026-10-07 20:09:36 -04:00
Brandon Thomas 5372309ad8 Merge pull request #228 from RicSchonfelder/i18n-pt-br
i18n: add Brazilian Portuguese (pt-br) UI translation
2026-10-07 20:09:32 -04:00
Brandon Thomas b63ea8f5c2 Merge pull request #222 from Ndsanes/fix/craft-fonts-relative-path
Build: resolve a relative CRAFT_FONTS_DIR against the workspace root
2026-10-07 20:09:28 -04:00
Brandon Thomas 2dda7cf842 Merge pull request #241 from agorevski/copilot/issue-cluster-docs-consistency
Docs: fix release, sibling, and command references
2026-10-07 20:09:24 -04:00
Brandon Thomas be87e1b6e7 Merge pull request #238 from DWAA1660/fix/export-missing-folder
Export: clear error when there's no export folder (instead of "Access is denied")
2026-10-07 20:09:19 -04:00
Brandon Thomas 97e34434d0 Merge pull request #217 from nledford/feat/random-sort
Library: Random sort with a stable seed and Reshuffle
2026-10-07 20:09:14 -04:00
Brandon Thomas 30192541ea Merge pull request #242 from nledford/feat/collapsible-sidebar-sections
Sidebar: collapsible Albums, Local, By Date and Keywords sections
2026-10-07 20:09:10 -04:00
Brandon Thomas c4d7c00e74 Merge pull request #225 from nledford/fix/stale-panels-after-library-switch
Library: switching libraries no longer shows the old library's sidebar
2026-10-07 20:09:06 -04:00
Brandon Thomas 3fe0bef364 Merge pull request #239 from nledford/feat/smart-album-file-path
Smart albums: File Path rule field
2026-10-07 20:09:03 -04:00
Brandon Thomas 590b387a53 Merge pull request #221 from MusseJusse/codex/navigation-arrows-pr
M16.1: complete navigation arrowheads
2026-10-07 20:08:59 -04:00
Brandon Thomas 51c878ddd8 Merge pull request #199 from storytold/fix/flaky-profile-hover-test
Tests: fix three FreeBSD-flaky headless UI tests
2026-10-07 20:08:55 -04:00
Brandon ThomasandClaude Opus 5.5 034bb6e324 Rename: drop a redundant from != to (clippy nonminimal_bool)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:08:52 -04:00
Brandon ThomasandClaude Opus 5.5 41f9ae0c2b Loupe: a draft that finishes after a newer request still replaces older pixels, so a long drag keeps updating
#243 dropped every result whose request id wasn't the newest for its slot. The loupe requests a new
draft every frame while a slider moves, so when a draft took longer than a frame nothing landed
until the drag paused. Main, Before and Hover now take a superseded draft that is newer than what
they show (never one from before a library or preview-cache reset, never an error or its source).
Also saturate the xtask RAM rounding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:06:58 -04:00
Brandon ThomasandClaude Opus 5.5 e2a941887e Nix: mark the package community-maintained in the agent guide; craft-fonts holds CJK fonts, not only Japanese
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:03:05 -04:00
Brandon ThomasandClaude Opus 5.5 e574839724 Merge main into #213 (camera_preview: main's proxy edge/size with the sensor_proxy fallback)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:01:23 -04:00
Brandon ThomasandClaude Opus 5.5 f0632e7022 Fetch: crate root forbids unsafe and denies panicking lints, like every other crate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:56:42 -04:00
Brandon ThomasandClaude Opus 5.5 d3c6dc0aa8 i18n zh-hans: "Add Remove Spot" means adding a spot removal
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:56:03 -04:00
Brandon ThomasandClaude Opus 5.5 ce165dfa4a Export: calmer wording for the missing-folder error
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:56:00 -04:00
radialmonsterandClaude Opus 5.5 1237e198d5 GPU: build the warp coverage mask per block with interval bounds
The coverage mask from 94fe300 evaluated the f64 warp for every output
pixel. coverage_mask now asks Warp::block_coverage for each 32 x 16 block
(the shared warp formulas evaluated with outward-rounded intervals, no
edge margin) and evaluates only undecided blocks near the edge per pixel
with Warp::covers; the bits are unchanged
(coverage_mask_matches_per_pixel_decision, incl. partial words and
padding bits; coverage_mask_with_the_horizon_in_view, a perspective
denominator exactly 0 inside the image, so the clamp and
undecided-division paths go through block_coverage and the mask).

Measured (release, synthetic source, embedded lens warp + CA +
perspective as in embedded_lens_perspective_edges_match):
6000x4000 ~30 ms -> 2.7 ms (32 threads, 11x), ~580 ms -> 24.5 ms
(1 thread, 24x); 2048x1365 3.6 ms -> 1.0 ms (3.5x), 67.5 ms -> ~5 ms
(12-14x). Rotate270 the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:29:29 -04:00
radialmonsterandClaude Opus 5.5 2b2ebfc7ca Geom: write the warp formulas once over a Real scalar (f64 or Interval)
Homography::apply, Affine::apply, Warp::to_corrected,
Warp::corrected_to_source and embedded_warp are now generic over
lightcraft_geom::Real (private *_real variants in pipeline, public
apply_real in geom); the f64 API is a thin wrapper, so callers are
unchanged. Real is implemented for f64 (plain IEEE operators) and for
Interval, which rounds every bound one ulp outward and turns division by
a range containing zero, the homography's 1e-300 denominator clamp and
non-finite bounds into "undecided" (NaN). Because the interval runs the
same operations as the f64 path, it bounds the rounded f64 results, not
only the exact ones.

The f64 path keeps the same operations in the same order (only commuted
operands, which IEEE + and x leave bit-identical; no fma). Proof:
warp_f64_results_are_bit_identical_to_the_pre_generic_formulas hashes the
bits of 2000 random warps x 40 points through every formula (plus the
clamp branch, including +-1e-300 exactly) and matches the digest computed
by the same test on thumb-split (419261c) before the change; reassociating
one expression, or `<=` for `<` in the clamp, changes it.

The framing chain (output px -> transformed -> lens-corrected -> green
source, then inside the edges) is written once: Warp::framing, generic.
Warp::frame (f64, returns the corrected and green source points) serves
both the CPU resample (sample_warped) and Warp::covers; block_coverage
evaluates the same chain with intervals over a whole block of output
pixels and compares the bounds against the same edges() inside uses, with
no margin: the bounds enclose every pixel's f64 result, so the decision
equals the per-pixel one; else None.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:29:29 -04:00
radialmonsterandClaude Opus 5.5 6b6e756830 CI: pick build jobs from installed RAM; cap test threads at 4
The fixed four jobs were safe on 8 GB but too many for 4 GB, and slower than
needed on large machines. `cargo xtask ci` now uses one build job per 2 GB of
installed RAM, capped at the CPU count, and 4 if the RAM can't be read
(/proc/meminfo, Win32_ComputerSystem, hw.memsize). With line-table debuginfo,
4 jobs measured about 3 GB of RAM in use and 2 jobs about 1.5 GB.

Test threads follow but stop at 4: more barely shortens CI, and at 16 threads
on a loaded machine the wall-clock frame-budget test
offline_originals_never_block_a_frame failed (one 303 ms frame). Explicit
CARGO_BUILD_JOBS / RUST_TEST_THREADS still win.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:27:20 -04:00
radialmonsterandClaude Opus 5.5 e9931d6871 UI tests: wait for the sidebar's file-system checks before clicking
keyword_list_filters_renames_and_suggests failed intermittently under
full-suite load (1 photo instead of 2 after clicking travel|italy, or no
travel|italy row at all). No metadata was lost: at the failure every
photo still had its keywords, but the filter was `travel|france`.

Local's rows (Downloads, Home, ...) are added only once their is_dir
checks come back from fs_cached worker threads. Those start when the left
panel is first drawn, which in this test is right before the clicks.
Headless::busy() did not count them. Under load a row could land between
the frame that read a widget's rect and the frame that delivered the
press. The keyword rows then moved down 29 px and the click hit the row
above (travel|france), or the toggle click missed travel. Diagnostics
showed the click aimed at y=1307 and travel|italy drawn at 1365 after
the two Local rows landed. The same failure reproduces on 5cee0c0 (2 of
20 runs), so the thumbnail work did not cause it.

fs_cached now keeps a per-context count of checks in flight
(fs_cached_running), and busy() includes it, so settle() waits for them.
The keyword test settles after opening the left panel. The new test
settle_waits_for_file_system_checks holds a check open and fails without
the busy() change.

Full ui-egui lib runs with 4 test threads: 2 failures in 25 before the
change, none in the runs after it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:04:03 -04:00
radialmonsterandClaude Opus 5.5 b4dc8c10ee Thumbnails: don't rebuild raw stand-in jobs every frame
An unedited raw first shows its embedded preview (ThumbQuick), then the
grid queues the real thumbnail at BACKGROUND_THUMB_PRIORITY. The grid asks
at its own priority (10 on screen), so thumb_current found no match while
the stand-in ran (nothing pending for the real slot) and while the real job
was queued (pending at 3, asked at 10): request_thumb rebuilt the job every
frame without changing anything. thumb_inputs now records whether the
request went through a stand-in; such a request is current while its
stand-in is pending or its real job is pending/rendered at the background
priority. Direct requests keep their priority check, so a higher priority
still promotes a queued job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:04:03 -04:00
radialmonsterandClaude Opus 5.5 73cf0d50bb Thumbnails: keep textures when the disk cache is resized
Changing the disk cache size (library.preferences cacheMb) re-attached a
new PreviewCache object, and the renderer treats a new cache object as a
library reset: every texture, the loupe included, was dropped and
re-rendered. Thumbnails are keyed by content, so a new budget leaves them
valid. The budget now changes in place on the attached cache (same object,
same generation); the next write trims to it, as before. A preview clear,
a generation bump and a new library still invalidate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:04:03 -04:00
radialmonsterandClaude Opus 5.5 c7f5664ce2 UI: don't reveal a hidden Local folder inside Home's tree
Hiding a kept Local folder beneath Home while browsing it (Remove from
Local, local.hide) expanded Home down to that folder and pushed the
"Show hidden locations" row out of view. local_places picked the
innermost listed folder containing the browsed one as the tree to
reveal; once the kept row was hidden that became Home, so folder_tree
opened Home, AppData, Local, Temp, ... to show the hidden folder again.

A tree no longer opens on the way down through a hidden folder: the
owner is chosen only among places whose path to the browsed folder
passes no hidden one. Hiding Home itself still lets Pictures reveal a
folder browsed inside it.

local_location_can_be_hidden_and_restored again hides a browsed
temporary folder (beneath Home on Windows), kept as Browse Folder...
does, while it is still browsed, and checks the restore row is inside
the viewport before clicking it. It failed before this fix. New unit
test: hidden_folder_is_not_revealed_in_an_outer_tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:04:03 -04:00
dbc05bedc7 Bench: thumbnail request example
thumb_request_bench measures UI-thread time and allocations for grid
thumbnail requests (with and without pending work), using the same counting
allocator as the other bench examples. REAL_PHOTOS=1 samples up to four
photos from PHOTO_ROOT, which must now be set explicitly; the original
defaulted to a local photo library path.

Split from a327b98 (T0 thumbnail work).

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:04:03 -04:00
f2f7a1cb26 T0: optimize thumbnail requests and reject obsolete preview work
Skip unchanged thumbnail jobs and reuse bounded settings hashes (both maps
capped at 1024 entries). Reject stale render results and cache writes after
edits, library changes and preview clears: render requests carry a request
id and cache generation, PreviewCache writes and reads hold the generation
lock that clear() takes, and library.clearPreviews also drops cached
sources. The web worker index follows the same generation rule.

The decoder guards, Windows path fixes and test fixture changes that were
bundled in a327b98 are separate commits; the benchmark follows this one.

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 19:04:03 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 7f3d9ce25f Sidebar: collapsible Albums, Local, By Date and Keywords sections
Each header folds its section on click, with a chevron after the title
showing the state. The choice is kept in the UI state across restarts.
The Albums "+" button keeps working inside its header.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 18:56:42 -04:00
f73aedcc94 UI tests: wait for folder listings and keep sidebar fixtures off Home
Folder listings arrive from background threads that Headless::busy() does
not track, so a fixed number of frames was not always enough under
full-suite load. Two tests failed intermittently and passed alone:
kept_local_root_stays_while_browsing_below_and_elsewhere and
local_folder_tree_expands_and_browses. They now wait for the expected
widget (wait_for_widget, test-only) and use their temporary folders as kept
roots. The kept-root test hides Home (in the test's in-memory session) so
the tree does not depend on the user's real folders.

local_location_can_be_hidden_and_restored now hides a kept root, reached
via "All Photographs". It no longer covers hiding a browsed temporary
folder: hiding a folder beneath Home expands the whole parent tree and
pushes the restore button out of view. That UI bug is not fixed here.

Fixture changes split from a327b98 (T0 thumbnail work), plus follow-ups.

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:50:00 -04:00
928c2c974f Windows: treat separator-only path differences as the same file
Catalog paths may mix `/` and `\` on Windows, so raw string comparison saw
`N:/photos/IMG.jpg` and `N:/photos\IMG.jpg` as different files:

- Rename: planning, the move list, move_file_with and file_moves now compare
  as paths. A separator-only "rename" is a no-op: no file move, no catalog
  change and no undo entry, alone or batched with a genuine rename.
  Previously it attempted a plain rename onto the existing file, failed, and
  rolled back the rest of the batch. Case-only renames still work (#95).
- Import: the already-in-library index is keyed by PathBuf.
- Tests build expected paths with Path::join, and the CLI test escapes the
  device path with serde_json instead of string formatting.

Split from a327b98 (T0 thumbnail work), with the follow-up rename fix that
the committed version needed.

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:50:00 -04:00
ee9a989834 Codecs: bound thumbnail fallback decodes and WebP metadata allocations
Non-JPEG thumbnail fallbacks decode at source resolution before resizing, so
the output size does not bound memory. ThumbnailOptions gains max_pixels
(default 64 MP, previously about 1 GP via DecodeOptions::fit); larger sources
return Error::TooLarge. Nothing in the workspace calls decode_thumbnail yet.

WebP RIFF chunk lengths are untrusted: set_memory_limit(bytes.len()) stops a
tiny truncated file requesting gigabytes for metadata. In image-webp 0.2.4
the limit applies only to ICC, EXIF and XMP chunks. Fuzz tests cover both;
the roundtrip test only switches to struct update syntax.

Split from a327b98 (T0 thumbnail work), where it was bundled.

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:49:59 -04:00
radialmonsterandClaude Opus 5.5 94fe30015c GPU: decide warped-geometry edges with a CPU coverage mask
The lens + perspective warp classified source-edge pixels in f32 on the GPU.
A position 0.000035 px outside the edge rounded inside, so the GPU sampled
photo pixels where the CPU reference draws blank canvas (geometry_variants,
"embedded lens + perspective", max 91 LSB). The CPU now evaluates the f64
framing decision into a one-bit-per-pixel mask; sampling and colour stay on
the GPU. The embedded-lens cases move to their own test,
embedded_lens_perspective_edges_match, with the same tolerances.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:49:59 -04:00
radialmonsterandClaude Opus 5.5 e99ce5a578 CI: default to line-table debuginfo, four jobs and four test threads
A full-debuginfo workspace test build with one rustc/link.exe per CPU peaked
at 14-20 GB commit on a 32-thread Windows machine before any test ran; the
test binaries themselves peak at about 3.3 GB each. `cargo xtask ci` now sets
CARGO_PROFILE_DEV_DEBUG=line-tables-only, CARGO_BUILD_JOBS=4 and
RUST_TEST_THREADS=4 unless they are already set. Ordinary cargo commands are
unchanged; GPU coverage is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 18:49:59 -04:00
Alex GorevskiandCopilot 6aebd48eec Docs: fix release and command references
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 73897efc-ca9e-48b7-8b5a-50be0a930b09
2026-10-07 15:18:44 -07:00
Nathaniel LedfordandClaude Sonnet 5.5 030ccf9782 Smart albums: File Path rule field
Adds a "File Path" text rule so a smart album can match on where a photo
lives, e.g. contains "/Aliah Ira Polanco-Grylls/". "contains" /
"doesn't contain" match the whole string (spaces included) rather than
word by word; the other text operators are the shared ones. `\` and `/`
both separate and matching ignores case, so catalogs moved between
platforms still match. Demo photos have no path.

The rule editor, CLI and MCP pick the field up from FIELDS. parity.md's
smart-collection rows now list it (and the stale field count is fixed).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 18:11:13 -04:00
DWAA1660andClaude Opus 5.5 6b0274120b Export: say so when there's no export folder instead of failing with Access denied
On Windows HOME is usually unset, so the default export folder came out
empty and photos were written by bare filename into the working directory,
failing with 'Access is denied. (os error 5)'.

- Default export folder uses %USERPROFILE%\Pictures\LightCraft Exports on
  Windows (matching the default library location).
- A blank Folder in the Export dialog, or no folder resolvable at all, now
  fails with 'You need to specify an export location!' before any write.
- The Export dialog stays open on an error (like Import) and shows it;
  the menu and keyboard export paths show it too instead of doing nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PNY4WojPYorWa6rQLxSYHa
2026-10-07 16:08:01 -06:00
Gremiger 855d3a7959 Issue #212: Raw: keep vendor white balance when writing DNG
NEF (and other vendor raws without a colour matrix) carry their as-shot
white balance as vendor multipliers, not as a DNG AsShotNeutral. The DNG
writer only copied AsShotNeutral/AsShotWhiteXY, so Convert to DNG and
Copy as DNG produced files with no white balance at all, which develop
dark and green. Write the reciprocal of the vendor multipliers as
AsShotNeutral when the source has no DNG white of its own.
2026-10-07 21:43:23 +00:00
RicSchonfelderandDeepSeek V4 Flash fb01966a00 i18n: add Brazilian Portuguese (pt-br) UI translation
Add pt-br as a fifth interface language beside English, Simplified
Chinese, Traditional Chinese (Taiwan) and Japanese.

`crates/ui-egui/locales/pt-br.json` and `pt-br-formats.json` carry the
messages and the format strings; the language table gains the entry
(`PtBr, "pt-br", "Português (Brasil)", "Latn", ...`), and Edit > Language
plus `app.language.portuguese` select it. Locale tags pt-BR, pt_BR.UTF-8
and pt all normalize to the same language.

Translation follows pt-BR UI conventions (Adobe terms: revelação,
balanço de branco, máscara, cópia virtual…); proper nouns and file
formats stay as written (LightCraft, ArtCraft, GitHub, DNG, TIFF, XMP).
Placeholders, format specs and punctuation match the English source; the
English plural suffix is consumed with `{:.0}` where Portuguese does not
need it.

Tests: every_language_has_a_catalog_covering_the_source,
catalogs_agree_on_placeholders_and_report_gaps, locale_tags_normalize
and the painted-window test now cover pt-br. Docs: localization.md, a
new localization-pt-br.md and the parity rows.

Co-Authored-By: DeepSeek V4 Flash (opencode) <noreply@opencode.ai>
2026-10-07 17:03:20 -03:00
Nathaniel LedfordandClaude Sonnet 5.5 520d7b8616 Library: switching libraries no longer shows the old library's sidebar
A catalog loaded from disk counts revisions from 0, like a new empty one, and the sidebar
counts, keyword tree, date groups and grid list are cached by revision alone. Opening an empty
library after one that was never edited this session therefore kept the old library's numbers.
Each catalog loaded in a process now gets its own revision range.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 15:40:18 -04:00
NDsans a15bd574f9 LR-BEHAV-LOCALIZE: fill the zh-hans catalog to match zh-hant
zh-hant carries 1246 messages; zh-hans had 1029, so 217 recently added
strings — mask component editing, point colour samples, spot removal,
colour-label sets, smart-preview status, memory budget, the CLI/MCP command
list and the eighteen stock presets — fell back to English in the
Simplified Chinese UI.

Translate them and order the file like zh-hant so the two catalogs can be
compared line by line.
2026-10-08 03:08:33 +08:00
NDsans b040f9bb8f Build: resolve a relative CRAFT_FONTS_DIR against the workspace root
A build script's cwd is its package root, so CRAFT_FONTS_DIR=../craft-fonts —
the spelling the README and craft-fonts' integration docs both use, written
relative to the workspace root — resolved to crates/craft-fonts and failed.

The failure is silent from the user's side: the app runs, the language menu
lists every language, and selecting Chinese or Japanese paints boxes. Warn
when no font ends up embedded, not only when the variable is set but wrong.
2026-10-08 03:08:28 +08:00
MusseJusse 036993f194 M16.1: complete navigation arrowheads
Made with gpt-6.1-sol using the Codex harness in T3 Code.
2026-10-07 19:55:15 +01:00
Brandon ThomasandClaude Opus 5.5 7aab3fd97f Rename PrintCraft to PdfCraft (storytold/pdfcraft) (#219)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:45:42 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 6de619ca9b Random sort: choosing Random starts a fresh shuffle
Switching to Random from another key (without an explicit seed) derives a new seed
instead of reusing the one left behind, which was the fixed seed-0 order on a new library.
Re-choosing Random or changing only the direction keeps the shuffle on screen.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 14:08:11 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 89446df6a8 Random sort: review fixes (null seed, 2^53 seed bound, real tests), README
- library.shuffle treats a null seed as none given
- seeds are limited to 0..2^53-1 so JSON-double clients read them back exactly
- album test now fails if manual order overrides Random; persistence test goes through
  close/reopen of the library instead of a serde round trip
- README lists random among the sort options

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 14:03:52 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 b310de1bd6 Menus: no direction check while sorting at random
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:46:24 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 17daca8a87 Menus: list Reshuffle once in the Sort submenu
library.shuffle's registry menu path made the menu builder append it a second time after
the hand-built entry. Hide it from the auto-listing; test that it appears once and is
enabled only while sorting at random.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:42:42 -04:00
Nathaniel LedfordandClaude Sonnet 5.5 a72118ec6e Library: Random sort with a stable seed and Reshuffle
SortKey::Random orders photos by a splitmix64 rank of (Sort.seed, id): same seed gives
the same order, and added/edited photos never reshuffle the others. library.sort takes
an optional seed; library.shuffle picks a new one (below 2^53 for JSON clients). Menus
gain Random and Reshuffle; Ascending/Descending are disabled for Random. No effect in
Recently Added (always newest import first).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:33:38 -04:00
𝐀𝖓𝖆𝖗𝖎𝖔𝐍 c58128c36b Merge branch 'main' into add_nix_build 2026-10-07 19:32:10 +02:00
Orthic LabsandCodex 94780ed299 LR-IMP-FORMATS: decode downsized Sony lossless ARWs
Co-authored-by: Codex <noreply@openai.com>
2026-10-07 22:36:00 +05:30
Claude cabe5b7399 LR-PROF-CAMERACOLOR: Panasonic RW2 file-local camera look
RW2 / RWL / RAW join ARW and NEF: the starting look (matrix, hue/saturation table, tone and chroma curves) is
fitted to the file's own embedded JPEG behind the same gates, with white balance relative to the as-shot look.
Panasonic previews always show the whole active area while the default crop is the in-camera aspect ratio, so when
the preview matches the active area instead of the crop, the reference is cropped to the default crop before
fitting (corpus_rw2_with_an_in_camera_crop_gets_a_camera_look).

On 174 CC0 samples with a preview 140 fits are accepted (93 of the 114 bodies with a preview), median ΔE against
the camera JPEG 5.0 with matching lightness; most of the others are compacts and kit zooms whose camera JPEG is
distortion-corrected (tag 0x0119 isn't used yet). `lightcraft-cli calibrate` keeps collecting ARW and NEF only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QtQmZ6peZXGn3wiqeowV2v
2026-10-07 16:36:06 +00:00
Daniel VoshartandClaude Sonnet 5.5 7fa7506db4 Fetch: the model downloader is its own crate, lightcraft-fetch
The downloader in lightcraft-segment (mirrors, resume, pinned SHA-256, pure-Rust
HTTPS) was already generic apart from four SAM 3 constants. Move it to
crates/fetch (L0, native only) so other model features can use it without
depending on the SAM 3 inference code. lightcraft-segment keeps SAM3_FILES,
DEFAULT_MIRRORS, MIRRORS_ENV and wraps mirrors(); no behaviour change. The
downloader's 13 tests moved with it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 12:07:24 -04:00
Claude 6bd64e4ad4 LR-IMP-FORMATS: Panasonic RW2 / Leica RWL / Panasonic RAW in every raw format
Decode every raw encoding Panasonic and Leica bodies write, from the DMC-LX1 (2005) to the DC-S1RM2 (2025):
compressed formats 4 (most bodies, 2008-2022) and 6 (S1 / S1R / S1H / S5, GH5M2), the prefix-coded strips of
format 8 (GH6 and later), packed formats 2, 5 and 7, and the 16-bit words of the 2005-2007 .RAW files. Until now
only format 5 decoded; every other Panasonic file opened as preview only.

Clean-room: no raw-decoder source was read. Tag names come from tag documentation; every encoding was recovered
by black-box analysis of 178 CC0 files from raw.pixls.us (118 bodies), and the module docs note the measurement
that settled each rule. Format-8 tags of unknown meaning must carry the values every sample has, otherwise the file
is reported as unsupported rather than guessed at.

Levels and geometry: formats 4 and 6 store samples 15 above the recorded black level (files without one use 15);
white is the active area's saturation plateau (the linearity limits are wrong on extended low-ISO shots, the GF1
and the Digilux 3); the in-camera aspect ratio is the default crop; mapped-out defect pixels are filled from
same-colour neighbours; the CFA pattern applies at sample (0, 0).

.rwl and .raw files are imported (engine, MCP folder expansion, file dialog, Linux MIME types). One file per
encoding joins the never-panic proptest; 11 CC0 files join `cargo xtask corpus`, and corpus_panasonic_encodings
checks each encoding's bit depth, black level, crop and CFA layout against the camera's own JPEG.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QtQmZ6peZXGn3wiqeowV2v
2026-10-07 16:00:39 +00:00
Daniel VoshartandClaude Sonnet 5.5 9089092969 Tests: pass on Windows, and under load
- paths are built by component, not with a hard-coded '/' (cli, import, rename, find-missing)
- the rename tests' model volume treats both separators as one
- the Local-folder UI tests hide Home when the temp folder lies inside it, and wait for
  the worker-thread listing instead of three frames
- the GPU equivalence test tolerates up to two edge pixels where the CPU and the card
  fall on different sides of the lens-warp sampling boundary

cargo xtask ci: all 7 steps pass.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 11:28:33 -04:00
Anarion Dunedain a5966f9b2e ci: add nix build package 2026-10-07 13:54:51 +02:00
119 changed files with 9250 additions and 960 deletions

No files matched your search

+12 -1
View File
@@ -54,4 +54,15 @@ jobs:
desktop-file-validate packaging/linux/ai.storyteller.lightcraft.desktop
sed -e 's/@VERSION@/0.0.0/' -e 's/@DATE@/2026-01-01/' packaging/linux/ai.storyteller.lightcraft.metainfo.xml.in >"$RUNNER_TEMP/ai.storyteller.lightcraft.metainfo.xml"
appstreamcli validate --no-net --explain "$RUNNER_TEMP/ai.storyteller.lightcraft.metainfo.xml"
python3 -c "import yaml; m = yaml.safe_load(open('packaging/linux/flatpak/ai.storyteller.lightcraft.yml')); assert m['id'] == 'ai.storyteller.lightcraft' and m['command'] == 'lightcraft'"
python3 - <<'EOF'
import yaml
src = yaml.safe_load(open("packaging/linux/flatpak/ai.storyteller.lightcraft.yml"))
bundle = yaml.safe_load(open("packaging/linux/flatpak/ai.storyteller.lightcraft.bundle.yml"))
for m in (src, bundle):
assert m["id"] == "ai.storyteller.lightcraft" and m["command"] == "lightcraft", m["id"]
for key in ("runtime", "runtime-version", "sdk", "finish-args"):
assert src[key] == bundle[key], f"Flatpak manifests disagree on {key}: {src[key]} vs {bundle[key]}"
print("Flatpak manifests ok")
EOF
- name: FreeBSD tarball layout (stub binaries)
run: bash packaging/freebsd/package.sh --dry-run
+103 -11
View File
@@ -1,9 +1,12 @@
# Release pipeline: every push to `release` builds signed installers for every platform and
# creates or updates the draft GitHub Release "LightCraft v<version>". See docs/releasing.md.
# Release pipeline: every push to `release` builds platform artifacts (signed where credentials
# are configured) and creates or updates the draft GitHub Release "LightCraft v<version>".
# See docs/releasing.md.
#
# All jobs run in the `release` environment (restricted to the `release` branch), which holds
# the signing secrets. Every secret is optional: missing ones produce unsigned artifacts and a
# warning, never a failed build.
# The signing jobs (macOS, Windows) and the draft-release job run in the `release` environment
# (restricted to the `release` branch), which holds the signing secrets. Every secret is optional:
# missing ones produce unsigned artifacts and a warning, never a failed build. The jobs that sign
# nothing (version, Linux, Flatpak, FreeBSD, web) run without it, so a dispatch on any branch is
# a dry run of them; the release job needs the signed jobs, so a branch run never drafts a release.
name: Release
on:
@@ -35,7 +38,7 @@ jobs:
version:
name: version
runs-on: ubuntu-latest
environment: release
# No `environment: release` here or on the other jobs that sign nothing: no release secrets.
outputs:
version: ${{ steps.v.outputs.version }}
date: ${{ steps.v.outputs.date }}
@@ -63,7 +66,7 @@ jobs:
macos:
name: macOS universal
needs: version
runs-on: macos-15
runs-on: macos-15-xlarge # M2 Pro larger runner; use only in release.yml (actions_macos_xl budget caps spend)
environment: release
env:
LIGHTCRAFT_VERSION: ${{ needs.version.outputs.version }}
@@ -173,7 +176,6 @@ jobs:
# The oldest GitHub-hosted image, so the binaries only need glibc >= 2.35 (Ubuntu 22.04+,
# Debian 12+, Fedora 36+, RHEL 10). See docs/releasing.md.
runs-on: ${{ matrix.runner }}
environment: release
strategy:
fail-fast: false
matrix:
@@ -197,7 +199,7 @@ jobs:
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
libxkbcommon-dev libwayland-dev libx11-dev libxrandr-dev libxi-dev libgl1-mesa-dev \
desktop-file-utils appstream python3-yaml
desktop-file-utils appstream python3-yaml zsync
curl -fsSL -o "$RUNNER_TEMP/nfpm.deb" \
"https://github.com/goreleaser/nfpm/releases/download/v${NFPM_VERSION}/nfpm_${NFPM_VERSION}_${{ matrix.deb }}.deb"
sudo dpkg -i "$RUNNER_TEMP/nfpm.deb"
@@ -222,6 +224,9 @@ jobs:
for appimage in dist/release/*.AppImage; do
chmod +x "$appimage"
APPIMAGE_EXTRACT_AND_RUN=1 "$appimage" --version
# Update information is embedded, and the .zsync for delta updates sits beside it.
"$appimage" --appimage-updateinformation | grep -F 'gh-releases-zsync|'
test -s "$appimage.zsync"
done
- uses: actions/upload-artifact@v4
with:
@@ -229,11 +234,98 @@ jobs:
path: dist/release/*
if-no-files-found: error
flatpak:
name: Flatpak ${{ matrix.arch }}
needs: [version, linux]
# Repackages the Linux job's tarball (the same binaries as the AppImage/deb/rpm); no Rust
# build here. 24.04 for a newer flatpak-builder (1.4) than 22.04 ships.
runs-on: ${{ matrix.runner }}
strategy:
fail-fast: false
matrix:
include:
- { arch: x86_64, runner: ubuntu-24.04 }
- { arch: aarch64, runner: ubuntu-24.04-arm }
env:
LIGHTCRAFT_VERSION: ${{ needs.version.outputs.version }}
LIGHTCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
name: release-linux-${{ matrix.arch }}
path: dist/linux
- name: Install flatpak-builder
run: |
sudo apt-get update
# librsvg2-common: the SVG pixbuf loader. flatpak-builder runs the host's
# `appstreamcli compose`, which renders the scalable icon and fails without it.
sudo apt-get install -y --no-install-recommends flatpak flatpak-builder librsvg2-common
# Ubuntu 24.04's AppArmor blocks the unprivileged user namespaces bubblewrap needs.
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
- name: Build the Flatpak bundle
run: packaging/linux/flatpak-bundle.sh "dist/linux/lightcraft-$LIGHTCRAFT_VERSION-linux-${{ matrix.arch }}.tar.gz"
- uses: actions/upload-artifact@v4
with:
name: release-flatpak-${{ matrix.arch }}
path: dist/release/*.flatpak
if-no-files-found: error
freebsd:
name: FreeBSD x86_64
needs: version
# GitHub has no FreeBSD runners: build in a FreeBSD 14.3 VM, the same image and packages as
# .github/workflows/freebsd.yml.
runs-on: ubuntu-latest
timeout-minutes: 180
env:
LIGHTCRAFT_VERSION: ${{ needs.version.outputs.version }}
LIGHTCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
steps:
- uses: actions/checkout@v4
# Fonts (optional build input; required for releases): see craftrules standards/fonts.md.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
repository: storytold/craft-fonts
ref: abb83316d96aa59c1cf64784289e378fe9fa5695 # bump deliberately
path: craft-fonts
- name: Build and package in a FreeBSD VM (tar.gz)
# vmactions/freebsd-vm v1.5.9, pinned by commit (it runs our build).
uses: vmactions/freebsd-vm@c46abacb49f09938ca4e1702d15d836285d694cc
with:
release: "14.3"
usesh: true
cpu: 4
mem: 12288
# Passed into the VM by name; never templated into the script below.
envs: "LIGHTCRAFT_VERSION LIGHTCRAFT_BUILD_DATE LIGHTCRAFT_BUILD_SHA INPUT_VERSION"
# The FreeBSD CI packages, plus bash for the packaging script.
prepare: |
pkg install -y bash git curl pkgconf cmake python3 libxkbcommon wayland libX11 libXcursor libXrandr libXi libxcb mesa-libs vulkan-loader gtk3 fontconfig freetype2 alsa-lib
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain stable
run: |
set -e
. "$HOME/.cargo/env"
git config --global --add safe.directory "$PWD"
# Build outside the checkout so only dist/ is copied back out of the VM (copying the
# target dir back fails on large builds). 4 jobs: more runs the VM out of memory.
export CARGO_TARGET_DIR=/var/tmp/lightcraft-target CARGO_BUILD_JOBS=4
if [ -n "$INPUT_VERSION" ]; then cargo xtask version set "$LIGHTCRAFT_VERSION"; fi
# craft-fonts was checked out next to the sources and synced into the VM with them.
export CRAFT_FONTS_DIR="$PWD/craft-fonts" CRAFT_FONTS_REQUIRED=1
bash packaging/freebsd/package.sh
tar -tzf dist/release/lightcraft-*-freebsd-x86_64.tar.gz \
| grep -E '/bin/lightcraft$|/bin/lightcraft-cli$|/share/applications/ai\.storyteller\.lightcraft\.desktop$|/share/doc/lightcraft/LICENSE-MIT$'
- uses: actions/upload-artifact@v4
with:
name: release-freebsd-x86_64
path: dist/release/*
if-no-files-found: error
web:
name: Web (wasm)
needs: version
runs-on: ubuntu-latest
environment: release
env:
LIGHTCRAFT_VERSION: ${{ needs.version.outputs.version }}
LIGHTCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}
@@ -275,7 +367,7 @@ jobs:
release:
name: Draft GitHub Release
needs: [version, macos, windows, linux, web]
needs: [version, macos, windows, linux, flatpak, freebsd, web]
runs-on: ubuntu-latest
environment: release
permissions:
+1 -1
View File
@@ -1,7 +1,7 @@
# Windows on ARM: package the ARM64 release the way release.yml does (cross-compiled on an x64
# runner, unsigned here), then install that MSI on a Windows 11 ARM64 runner, check the installed
# binaries are ARM64, run the CLI natively, and uninstall. (A native ARM64 `cargo test` job is left
# out for now: PrintCraft saw an intermittent access violation in egui UI tests there.)
# out for now: PdfCraft saw an intermittent access violation in egui UI tests there.)
name: Windows ARM64
on:
+3
View File
@@ -8,6 +8,9 @@ plan/
.DS_Store
.claude/
/craft-fonts
# nix build / nix develop
/result
/result-*
# model weights are never committed (SAM 3: docs/ai-masks.md)
*.safetensors
*.safetensors.part
+23 -2
View File
@@ -1,6 +1,6 @@
# LightCraft — instructions for agents
LightCraft is a clean-room, open-source, pure-Rust photo library + non-destructive raw developer targeting Adobe Lightroom parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../printcraft` (Acrobat), `../photocraft` (Photoshop), `../drawcraft` (Illustrator) and `../filmcraft` (Premiere), with the same conventions.
LightCraft is a clean-room, open-source, pure-Rust photo library + non-destructive raw developer targeting Adobe Lightroom parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../pdfcraft` (Acrobat), `../photocraft` (Photoshop), `../vectorcraft` (Illustrator) and `../filmcraft` (Premiere), with the same conventions.
## Start every session here
1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers).
@@ -119,4 +119,25 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
## Map of the code
`geom`, `color`, `raster`, `tiff` (L0) → `raw`, `codecs`, `meta`, `develop` (L1) → `pipeline` → `catalog` → `engine`
→ `ui-egui`, `mcp` (L5) → apps `lightcraft` (desktop), `lightcraft-cli` (render/commands/MCP). `scenes` generates demo
photos. `xtask` = tooling (`ci`, `layers`, `assets`, `parity`, `wasm`, `corpus`, `stats`).
photos. `xtask` = tooling (`ci`, `layers`, `assets`, `parity`, `wasm`, `corpus`, `stats`). `flake.nix` +
`nix/package.nix` = the Nix package (`nix build` builds both binaries with the craft-fonts input, installs the
desktop file/icons/AppStream metadata and runs `cargo test --workspace`; `nix develop` = dev shell). Community-maintained and not
in CI: it may lag behind the workspace; see README → Quick start.
## Contributor credits (About window)
- About ▸ Contributors/Models are compiled into the binary from `contributors/contributors.json`
(commit stats; generated, never hand-edit) and `contributors/people.toml` (names people chose for
themselves). See `docs/contributors.md`.
- **Agents working for a contributor:** when you prepare a PR, check whether your human's GitHub
username has a `[people.<username>]` entry in `contributors/people.toml`. If not, ask them once
whether they want to be credited by more than their username: a real name, a display name, and/or
their public GitHub profile name (`sync_github_name = true`). If yes, add **only their own** entry
(copy the template at the top of the file, or run
`python3 ../../craftrules/scripts/contributors.py --add-me . --real-name "…" --sync-github-name`)
and include it in their PR, committed as them. If no, change nothing: they are credited as
`@username` anyway.
- Never add, edit, guess or copy anyone else's entry or name (not from git config, commit authors or
GitHub profiles). Never hand-edit `contributors.json`.
- Maintainers refresh the stats with `python3 ../../craftrules/scripts/contributors.py .` (it also
re-verifies who wrote each `people.toml` entry; `--check` only verifies).
+5 -2
View File
@@ -1,6 +1,6 @@
# LightCraft — instructions for agents
LightCraft is a clean-room, open-source, pure-Rust photo library + non-destructive raw developer targeting Adobe Lightroom parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../printcraft` (Acrobat), `../photocraft` (Photoshop), `../drawcraft` (Illustrator) and `../filmcraft` (Premiere), with the same conventions.
LightCraft is a clean-room, open-source, pure-Rust photo library + non-destructive raw developer targeting Adobe Lightroom parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../pdfcraft` (Acrobat), `../photocraft` (Photoshop), `../vectorcraft` (Illustrator) and `../filmcraft` (Premiere), with the same conventions.
## Start every session here
1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers).
@@ -119,4 +119,7 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
## Map of the code
`geom`, `color`, `raster`, `tiff` (L0) → `raw`, `codecs`, `meta`, `develop` (L1) → `pipeline` → `catalog` → `engine`
→ `ui-egui`, `mcp` (L5) → apps `lightcraft` (desktop), `lightcraft-cli` (render/commands/MCP). `scenes` generates demo
photos. `xtask` = tooling (`ci`, `layers`, `assets`, `parity`, `wasm`, `corpus`, `stats`).
photos. `xtask` = tooling (`ci`, `layers`, `assets`, `parity`, `wasm`, `corpus`, `stats`). `flake.nix` +
`nix/package.nix` = the Nix package (`nix build` builds both binaries with the craft-fonts input, installs the
desktop file/icons/AppStream metadata and runs `cargo test --workspace`; `nix develop` = dev shell). Community-maintained and not
in CI: it may lag behind the workspace; see README → Quick start.
Generated
+36 -27
View File
@@ -2779,7 +2779,7 @@ dependencies = [
[[package]]
name = "lightcraft"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"eframe",
"egui",
@@ -2797,7 +2797,7 @@ dependencies = [
[[package]]
name = "lightcraft-catalog"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"crc32fast",
"lightcraft-develop",
@@ -2812,7 +2812,7 @@ dependencies = [
[[package]]
name = "lightcraft-cli"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"dhat",
"lightcraft-codecs",
@@ -2827,7 +2827,7 @@ dependencies = [
[[package]]
name = "lightcraft-codecs"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"bytemuck",
"image",
@@ -2852,7 +2852,7 @@ dependencies = [
[[package]]
name = "lightcraft-color"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"lightcraft-geom",
"proptest",
@@ -2861,7 +2861,7 @@ dependencies = [
[[package]]
name = "lightcraft-develop"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"lightcraft-color",
"lightcraft-geom",
@@ -2873,7 +2873,7 @@ dependencies = [
[[package]]
name = "lightcraft-engine"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"ab_glyph",
"libc",
@@ -2900,9 +2900,20 @@ dependencies = [
"web-time",
]
[[package]]
name = "lightcraft-fetch"
version = "0.4.0"
dependencies = [
"log",
"rustls",
"rustls-rustcrypto",
"sha2",
"webpki-roots",
]
[[package]]
name = "lightcraft-geom"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"proptest",
"serde",
@@ -2910,7 +2921,7 @@ dependencies = [
[[package]]
name = "lightcraft-gpu"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"bytemuck",
"lightcraft-color",
@@ -2926,7 +2937,7 @@ dependencies = [
[[package]]
name = "lightcraft-mcp"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"lightcraft-codecs",
"lightcraft-engine",
@@ -2937,7 +2948,7 @@ dependencies = [
[[package]]
name = "lightcraft-merge"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"lightcraft-codecs",
"lightcraft-color",
@@ -2953,7 +2964,7 @@ dependencies = [
[[package]]
name = "lightcraft-meta"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"lightcraft-geom",
"lightcraft-tiff",
@@ -2965,7 +2976,7 @@ dependencies = [
[[package]]
name = "lightcraft-pipeline"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"image",
"lightcraft-color",
@@ -2981,7 +2992,7 @@ dependencies = [
[[package]]
name = "lightcraft-preview"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"jpeg-encoder",
"lightcraft-raster",
@@ -2993,7 +3004,7 @@ dependencies = [
[[package]]
name = "lightcraft-raster"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"libc",
"lightcraft-color",
@@ -3005,7 +3016,7 @@ dependencies = [
[[package]]
name = "lightcraft-raw"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"jpeg-encoder",
"lightcraft-color",
@@ -3024,7 +3035,7 @@ dependencies = [
[[package]]
name = "lightcraft-scenes"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"image",
"lightcraft-color",
@@ -3035,27 +3046,24 @@ dependencies = [
[[package]]
name = "lightcraft-segment"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"candle-core",
"candle-nn",
"lightcraft-fetch",
"log",
"rustls",
"rustls-rustcrypto",
"serde",
"serde_json",
"sha2",
"thiserror 2.0.21",
"webpki-roots",
]
[[package]]
name = "lightcraft-sysmem"
version = "0.2.1"
version = "0.4.0"
[[package]]
name = "lightcraft-tiff"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"proptest",
"serde",
@@ -3064,7 +3072,7 @@ dependencies = [
[[package]]
name = "lightcraft-ui-egui"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"egui",
"lightcraft-catalog",
@@ -3080,12 +3088,13 @@ dependencies = [
"log",
"serde",
"serde_json",
"toml",
"web-time",
]
[[package]]
name = "lightcraft-web"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"crc32fast",
"eframe",
@@ -6473,7 +6482,7 @@ checksum = "e450f9b2ed1dff33c94c12589a87338689467b9c4f5d8a5710bd09a847d2c8a7"
[[package]]
name = "xtask"
version = "0.2.1"
version = "0.4.0"
dependencies = [
"brotli",
"flate2",
+2 -1
View File
@@ -3,7 +3,7 @@ resolver = "3"
members = ["crates/*", "apps/*", "xtask"]
[workspace.package]
version = "0.2.1"
version = "0.4.0"
edition = "2024"
license = "MIT OR Apache-2.0"
rust-version = "1.90"
@@ -29,6 +29,7 @@ lightcraft-merge = { path = "crates/merge" }
lightcraft-engine = { path = "crates/engine" }
lightcraft-ui-egui = { path = "crates/ui-egui" }
lightcraft-mcp = { path = "crates/mcp" }
lightcraft-fetch = { path = "crates/fetch" }
lightcraft-segment = { path = "crates/segment" }
# foundation
+92 -11
View File
@@ -59,6 +59,7 @@
<a href="#feature-status">Status</a> ·
<a href="#quick-start">Quick start</a> ·
<a href="ROADMAP.md">Roadmap</a> ·
<a href="#downloads">Downloads</a> ·
<a href="#the-crafting-apps">Crafting Apps</a>
</p>
@@ -200,7 +201,7 @@ A library that stays out of your way: **All Photos**, **Recently Added**, **Pick
**Folders**, and **Recently Deleted**. Rate with <kbd>0</kbd>–<kbd>5</kbd>, flag with <kbd>P</kbd> / <kbd>X</kbd> /
<kbd>U</kbd>, colour-label with <kbd>6</kbd>–<kbd>9</kbd>. Search understands fields:
`rating:>3 flag:pick iso:>800 camera:x2 date:2026-04 keyword:mountains`. Every view sorts by capture date,
import date, edit date, name, rating or size. The justified **Photo Grid** and **Square Grid** views are virtualized,
import date, edit date, name, rating, size or at random (a stable shuffle; View → Sort → Reshuffle for a new one). The justified **Photo Grid** and **Square Grid** views are virtualized,
so they stay smooth whether you have forty photos or forty thousand.
<table>
@@ -233,11 +234,11 @@ lightcraft --control 7980 ~/Pictures/trip
{"method": "ui.screenshot", "params": {"path": "after.png"}}
```
- **91 engine commands** and **48 UI commands.** List them all with `engine.commands`; read every slider's range,
default and current value with `develop.controls`.
- **MCP server.** `lightcraft-cli mcp` gives Claude (or any MCP client) ~100 tools: import, query, develop, mask,
render (returned as an image), export. It runs headless, or attached to the running app with screenshots, clicks
and gestures. See [docs/mcp.md](docs/mcp.md).
- **Command registry.** `engine.commands` lists the available commands; `develop.controls` lists every slider's
range, default and current value.
- **MCP server.** `lightcraft-cli mcp` exposes the command registry to Claude (or any MCP client), alongside
helpers for import, query, develop, mask, render (returned as an image) and export. It runs headless, or attached
to the running app with screenshots, clicks and gestures. See [docs/mcp.md](docs/mcp.md).
```sh
cargo build --release -p lightcraft-cli
@@ -258,8 +259,8 @@ lightcraft --control 7980 ~/Pictures/trip
## Fast, native, private
- **Pure Rust, no C.** Our own RAW decoders (DNG, Canon CR2, Sony ARW, Nikon NEF, Fujifilm RAF incl. X-Trans,
Panasonic RW2, Pentax PEF, Olympus ORF), our own colour science, our own pipeline. JPEG, PNG, TIFF, WebP, PSD
composites and JPEG XL open today.
Panasonic RW2 / Leica RWL, Pentax PEF, Olympus ORF), our own colour science, our own pipeline. JPEG, PNG, TIFF, WebP,
PSD composites and JPEG XL open today.
- **Scene-referred & wide-gamut.** Linear Rec.2020 float internally, Bradford-adapted white balance, gamut mapping
instead of clipping, a filmic shoulder for raw and pixel-exact pass-through for JPEGs you haven't touched.
- **Resolution-independent edits.** Radii and brush sizes are relative to the image, so a 400 px preview, your
@@ -301,9 +302,9 @@ LightCraft is young and moving fast. **Where we honestly stand** (details in the
| Masking: brush, linear, radial, luminance/colour range, add/subtract/intersect | ✅ (AI subject/sky use classical heuristics for now) |
| Crop, straighten tool + auto straighten, flip, rotate, aspect ratios, overlays | ✅ |
| Profiles (Color, Neutral, Vivid, Landscape, Portrait, Monochrome: our own looks), presets, versions, history, copy/paste/sync settings | ✅ |
| Camera colour: DNG files use their own matrices | ✅ · our own calibration for other raws ⬜ (top priority; neutral fallback today) |
| Camera colour: DNG files use their own matrices | ✅ · our own calibration for other raws ⬜ (top priority; ARW, NEF and RW2 start from a look fitted to their own JPEG, other raws from a neutral fallback) |
| Native macOS menu bar (generated from the command registry), control channel + every widget addressable, headless UI snapshots | ✅ |
| RAW: DNG, CR2, ARW, NEF (uncompressed + lossless/lossy compressed), Fujifilm RAF (uncompressed, Bayer + X-Trans), Panasonic RW2, Pentax PEF, Olympus ORF (uncompressed); embedded previews for every format incl. CR3 | ✅ · CR3, compressed RAF/ORF decode ⬜ |
| RAW: DNG, CR2, ARW, NEF (uncompressed + lossless/lossy compressed), Fujifilm RAF (uncompressed, Bayer + X-Trans), Panasonic RW2 / Leica RWL / Panasonic RAW (every raw format, DMC-LX1 to DC-S1RM2), Pentax PEF, Olympus ORF (uncompressed); embedded previews for every format incl. CR3 | ✅ · CR3, compressed RAF/ORF decode ⬜ |
| Detail: sharpening, luminance + colour noise reduction | ✅ · AI Denoise, Super Resolution ⬜ |
| Remove / Heal / Clone spots (auto source), Visualize Spots, Red Eye and Pet Eye (auto pupil detection, catchlight) | ✅ · content-aware fill, spot pin editing 🚧 |
| Export: JPEG / PNG / TIFF / WebP / AVIF / DNG / original, sizing, file-size limit, output sharpening, naming templates, batch, metadata policy, text or image watermark | ✅ · HDR export ⬜ |
@@ -334,6 +335,14 @@ cargo run --release -p lightcraft-cli -- render photo.jpg -o out.jpg --set light
cargo xtask ci # fmt, clippy, tests, layering, wasm checks
```
CI defaults to line-table debug information and one build job per 1.5 GB of RAM
available when it starts (at most one per CPU; 4 if memory can't be read), so an
8 GB machine with 6 GB free builds with 4, and a busy machine with little free
builds with fewer. Test threads follow, capped at 4. Full-debuginfo linkers, one
per CPU, otherwise exhaust RAM before any test runs. GPU coverage is unchanged. Explicit
`CARGO_PROFILE_DEV_DEBUG`, `CARGO_BUILD_JOBS` and `RUST_TEST_THREADS` settings
override these defaults. Ordinary development commands keep their own settings.
**Chinese and Japanese text** need the shared font repo, an optional build input (official releases always include it):
```sh
@@ -350,6 +359,31 @@ Language**; the choice applies immediately and persists. See [docs/localization.
The web build needs the `wasm32-unknown-unknown` target and the matching `wasm-bindgen` CLI
(`cargo xtask web` prints the exact install command); see [docs/web.md](docs/web.md).
**Nix** builds the desktop app and `lightcraft-cli` (the Nix build always includes the craft-fonts input, so Japanese
text has glyphs):
```sh
nix run github:storytold/lightcraft # the desktop app
nix build github:storytold/lightcraft # → ./result/bin/{lightcraft,lightcraft-cli}
nix develop github:storytold/lightcraft # rust toolchain + native deps + fonts
```
In a flake configuration (NixOS, home-manager, nix-darwin):
```nix
# flake.nix
inputs.lightcraft.url = "github:storytold/lightcraft";
# optional: build against your own nixpkgs instead of the one LightCraft pins
# inputs.lightcraft.inputs.nixpkgs.follows = "nixpkgs";
# then, in a NixOS or home-manager module (where `inputs` is in scope):
nixpkgs.overlays = [ inputs.lightcraft.overlays.default ]; # makes `pkgs.lightcraft` available
environment.systemPackages = [ pkgs.lightcraft ]; # home-manager: home.packages = [ pkgs.lightcraft ];
```
`nix build` installs the same desktop file, hicolor icons and AppStream metadata as the .deb/.rpm, and runs
`cargo test --workspace` as its check phase (skip it with `pkgs.lightcraft.overrideAttrs { doCheck = false; }`).
**Keyboard:** <kbd>G</kbd> grid · <kbd>D</kbd> detail · <kbd>E</kbd> edit · <kbd>C</kbd> crop · <kbd>M</kbd> masking ·
<kbd>Shift</kbd>+<kbd>P</kbd> presets · <kbd>\\</kbd> original · <kbd>Y</kbd> before/after · <kbd>Z</kbd> zoom ·
<kbd>J</kbd> clipping · <kbd>⌘Z</kbd> undo · <kbd>⌘/</kbd> all shortcuts.
@@ -381,6 +415,53 @@ Questions, ideas or a bug you'd like to talk through first? Bring them to [Disco
<br>
## Downloads
**New to LightCraft?** Download it from the [LightCraft page on getartcraft.com](https://getartcraft.com/apps/lightcraft). That's the easiest way to install it.
**Want a specific build or format?** On GitHub, the [latest release](https://github.com/storytold/lightcraft/releases/latest) has every build listed below, and [all releases](https://github.com/storytold/lightcraft/releases) has earlier versions and their notes. `<ver>` in the file names is the version number, and `SHA256SUMS.txt` lists a checksum for every file.
### Windows
| Build | Installer | Portable |
|---|---|---|
| x64 (64-bit Intel/AMD) | `lightcraft-<ver>-windows-x64.msi` | `lightcraft-<ver>-windows-x64-portable.zip` |
| arm64 (Snapdragon and other ARM PCs) | `lightcraft-<ver>-windows-arm64.msi` | `lightcraft-<ver>-windows-arm64-portable.zip` |
| x86 (32-bit) | `lightcraft-<ver>-windows-x86.msi` | `lightcraft-<ver>-windows-x86-portable.zip` |
Installers and executables are code-signed.
### macOS
| Build | File | Notes |
|---|---|---|
| App, universal (Apple silicon + Intel) | `lightcraft-<ver>-macos-universal.dmg` | Signed and notarized |
| Command-line tool, universal | `lightcraft-cli-<ver>-macos-universal.zip` | Signed and notarized |
### Linux
| Format | x86_64 | aarch64 (ARM64) | Notes |
|---|---|---|---|
| AppImage | `lightcraft-<ver>-linux-x86_64.AppImage` | `lightcraft-<ver>-linux-aarch64.AppImage` | Runs anywhere; updates itself with [AppImageUpdate](https://github.com/AppImageCommunity/AppImageUpdate) (`.zsync` files) |
| Flatpak | `lightcraft-<ver>-linux-x86_64.flatpak` | `lightcraft-<ver>-linux-aarch64.flatpak` | Sandboxed; `flatpak install --user <file>` |
| Debian/Ubuntu | `lightcraft-<ver>-linux-x86_64.deb` | `lightcraft-<ver>-linux-aarch64.deb` | |
| Fedora/RHEL/openSUSE | `lightcraft-<ver>-linux-x86_64.rpm` | `lightcraft-<ver>-linux-aarch64.rpm` | |
| Tarball | `lightcraft-<ver>-linux-x86_64.tar.gz` | `lightcraft-<ver>-linux-aarch64.tar.gz` | Unpack anywhere |
### FreeBSD
| Build | File |
|---|---|
| x86_64 | `lightcraft-<ver>-freebsd-x86_64.tar.gz` |
### Web (WebAssembly)
| Build | File | Notes |
|---|---|---|
| Static site | `lightcraft-web-<ver>.zip` | Runs in a modern browser; host it on any static server |
<br>
## The Crafting Apps
LightCraft is one of the **Crafting Apps**: free, open-source creative tools from the
@@ -393,7 +474,7 @@ stand on its own.
| <img src="https://raw.githubusercontent.com/storytold/vectorcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.vectorcraft.png" alt="" width="32" height="32"> | **VectorCraft** | Vector illustration | [GitHub](https://github.com/storytold/vectorcraft) | [Website](https://getartcraft.com/apps/vectorcraft) |
| <img src="https://raw.githubusercontent.com/storytold/filmcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.filmcraft.png" alt="" width="32" height="32"> | **FilmCraft** | Video editing, color and sound | [GitHub](https://github.com/storytold/filmcraft) | [Website](https://getartcraft.com/apps/filmcraft) |
| <img src="https://raw.githubusercontent.com/storytold/lightcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.lightcraft.png" alt="" width="32" height="32"> | **LightCraft** | **Photo library and raw development · you are here** | [GitHub](https://github.com/storytold/lightcraft) | [Website](https://getartcraft.com/apps/lightcraft) |
| <img src="https://raw.githubusercontent.com/storytold/printcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.printcraft.png" alt="" width="32" height="32"> | **PrintCraft** | Reading, organizing and protecting PDFs | [GitHub](https://github.com/storytold/printcraft) | [Website](https://getartcraft.com/apps/printcraft) |
| <img src="https://raw.githubusercontent.com/storytold/pdfcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.pdfcraft.png" alt="" width="32" height="32"> | **PdfCraft** | Reading, organizing and protecting PDFs | [GitHub](https://github.com/storytold/pdfcraft) | [Website](https://getartcraft.com/apps/pdfcraft) |
| <img src="https://raw.githubusercontent.com/storytold/effectcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.effectcraft.png" alt="" width="32" height="32"> | **EffectCraft** | Motion graphics and visual effects | [GitHub](https://github.com/storytold/effectcraft) | [Website](https://getartcraft.com/apps/effectcraft) |
| <img src="https://raw.githubusercontent.com/storytold/designcraft/main/assets/app-icon/hicolor/64x64/apps/ai.storyteller.designcraft.png" alt="" width="32" height="32"> | **DesignCraft** | Page layout and publishing | [GitHub](https://github.com/storytold/designcraft) | [Website](https://getartcraft.com/apps/designcraft) |
+19 -14
View File
@@ -23,8 +23,8 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
| Dimension | Estimate | What's true today | Biggest gaps |
|---|---:|---|---|
| **Feature checklist** | 79% | P0 core and P1 nearly complete: import (Add / Copy / Move, templates, devices), library, grid/loupe/compare/survey, every Edit slider, curves, colour grading, masking tools, crop/Upright, heal/clone, presets/profiles, versions/history, sync, export, menus, shortcuts | P1: lens-profile database, content-aware fill (patch synthesis), video playback/trim |
| **RAW coverage** (formats people shoot) | ~50% | DNG (all kinds), CR2, ARW, NEF (uncompressed + Huffman lossless/lossy), uncompressed RAF/ORF, packed RW2, PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed RAF/ORF, RW2 v4, Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. NEFs labelled compressed but stored uncompressed (Z 6 packed 14-bit, D850 12-bit uncompressed). Per-model verification is thin (~40 corpus files vs >1,000 models) |
| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW and NEF have a guarded per-file embedded-JPEG colour estimate (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
| **RAW coverage** (formats people shoot) | ~55% | DNG (all kinds), CR2, ARW, NEF (uncompressed + Huffman lossless/lossy), uncompressed RAF/ORF, RW2 / RWL / Panasonic RAW (every raw format, checked on 178 files from 118 Panasonic and Leica bodies), PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed RAF/ORF, Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. NEFs labelled compressed but stored uncompressed (Z 6 packed 14-bit, D850 12-bit uncompressed). Per-model verification is thin (~55 corpus files vs >1,000 models) |
| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW, NEF and RW2 have a guarded per-file embedded-JPEG colour estimate (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
| **AI & computational** | ~15–20% | Assisted culling (focus, bursts), auto tone, HDR/panorama merge; subject/sky/background masks as classical heuristics | Real segmentation masks (subject, sky, people, objects, landscape, depth), AI denoise, super resolution, lens blur, generative remove, faces/people, natural-language search. **Blocked on a model strategy** (licensable weights or our own training; pure-Rust inference is feasible) |
| **Workflow & library** | ~85% (single machine) | Robust catalog (journal + snapshots, background compaction, crash-tested), 85k-photo libraries stay responsive, Local browsing with automatic cleanup, XMP interop, keywords, smart albums, Move import | Opening an 85k library takes 1.7–4.7 s; no cloud sync (out of scope), no tablet companion (#74, roadmap), shared albums, publish services, tethering |
| **Classic modules** | ~30% | Geotagging from GPX track logs, soft proofing (partial), slideshow (basic) | **Map view, Book, Print, Slideshow module, Web, publish services**: ~40 tracker rows ⬜ |
@@ -36,7 +36,7 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
| User | Readiness | What blocks them |
|---|---:|---|
| JPEG / DNG shooter, single machine | ~85% | Fidelity polish, AI masks |
| Nikon / Sony / older-Canon raw shooter | ~65% | Camera colour fidelity and coverage (ARW and NEF preview estimates are only a starting point) |
| Nikon / Sony / Panasonic / older-Canon raw shooter | ~65% | Camera colour fidelity and coverage (ARW, NEF and RW2 preview estimates are only a starting point); Panasonic compacts and kit zooms also miss their embedded distortion correction |
| Canon CR3 / Fujifilm / Olympus shooter | ~35% | Their raws open as embedded previews only (CR3: the full-size JPEG with full metadata) |
| Lightroom Classic power user | ~45% | Print, Book, Map, publish, tethering |
| Relies on AI (masks, denoise) | ~25% | Object / Describe masks via optional SAM 3 (download not yet hosted); no AI denoise |
@@ -46,9 +46,9 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
Priorities, in order. Each points at tracker rows in [`docs/parity.md`](docs/parity.md) → *Top gaps*.
1. **Camera colour calibration of our own** (LR-PROF-CAMERACOLOR, P0): fit each camera to its own embedded JPEG, use
matrices the files carry themselves, then chart shots. Sony ARW and Nikon NEF have the file-local fit (matrix + tone
curve from their own JPEG); generalise it to the other makes' raws (RW2, PEF, ORF…), then validate fidelity.
2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): **CR3** first, then compressed RAF / ORF, RW2 v4, NEF
matrices the files carry themselves, then chart shots. Sony ARW, Nikon NEF and Panasonic RW2 have the file-local fit
(matrix + tone curve from their own JPEG); generalise it to the other makes' raws (PEF, ORF…), then validate fidelity.
2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): **CR3** first, then compressed RAF / ORF, NEF
lossy-after-split, sRAW. Decided 2026-10-05: write our own decoders from prose descriptions (never decoder source,
no LGPL dependency); compressed NEF (#86) is the template.
3. **Verified camera coverage** (LR-IMP-CAMERA-COVERAGE, P0): a CC0 sample per model in the corpus, each decoded and
@@ -84,7 +84,7 @@ hardening (#78), copyright metadata (#51), GPX geotagging (#60), import tag help
| M8 | Heal / Remove | content-aware remove (PatchMatch), heal, clone, brush spots, visualize spots, red/pet eye | 6–10 | 🚧 (heal, clone, auto source, visualize spots, red/pet eye ✅; PatchMatch remove ⬜) |
| M9 | Presets, profiles, versions, sync | preset browser + amount, create/import presets, profile browser, versions, history, copy/paste/sync settings | 5–8 | ✅ |
| M10 | Export & share | export dialog (JPEG/PNG/TIFF/DNG/AVIF/JXL/original), sizing, sharpening, metadata, watermark, naming, batch jobs, XMP sidecars, HDR export | 6–10 | 🚧 (all formats incl. DNG/original, sizing, presets, background jobs ✅; JXL encode, HDR export ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 packed, PEF, ORF uncompressed ✅; **camera colour calibration** 🚧 (guarded ARW and NEF preview fitting; measured database still missing), CR3, compressed ORF/RAF ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 (every raw format), PEF, ORF uncompressed ✅; **camera colour calibration** 🚧 (guarded ARW, NEF and RW2 preview fitting; measured database still missing), CR3, compressed ORF/RAF ⬜) |
| M12 | AI & smart features | subject/sky/background/people/object masks, semantic search, faces/People (permissively licensed models, pure-Rust inference) | 20–40 | ⬜ |
| M13 | Merge | HDR merge (deghost), panorama (projections, boundary warp, fill edges), HDR panorama | 10–15 | ✅ |
| M14 | Video | import/playback/trim via FilmCraft crates, global edits + presets on video, video export | 6–10 | ⬜ |
@@ -118,7 +118,7 @@ trip, slideshow, auto import…) in ≈ 12 h on 2026-10-02; four to six parallel
| Work package | Tracker rows | Agent-hours | Risk |
|---|---|---:|---|
| Remaining P0/P1 UI and library features (folder rename/move, keyword painter, people view…) | ≈ 8 | 5–10 | low |
| Raw codecs: CR3 (CRX), compressed ORF / RAF, NEF lossy-after-split, RW2 v4, HEIC/AVIF decode, JPEG XL DNG | LR-IMP-FORMATS | 40–80 | **high** — clean-room black-box analysis, no permissive specs |
| Raw codecs: CR3 (CRX), compressed ORF / RAF, NEF lossy-after-split, HEIC/AVIF decode, JPEG XL DNG | LR-IMP-FORMATS | 40–80 | **high** — clean-room black-box analysis, no permissive specs |
| Lens-profile database of our own (calibration targets, fitting, data) | LR-EDIT-OPTICS-PROFILE | 15–30 | data collection |
| Video: playback, trim, edits, export (pure-Rust decode, ideally shared with FilmCraft) | R. Video | 20–40 | medium |
| AI: subject / sky / background / people / object masks, object-aware remove, AI denoise, super resolution, lens blur, people & faces, natural-language search, culling | ≈ 30 | 80–150 | **high** — permissively licensed weights, pure-Rust inference, maybe training |
@@ -162,10 +162,15 @@ The milestone estimates in the table above were made before work started and are
## Raw format coverage and known gaps
Sony ILCE-7M4 downsized lossless ARW (subsampled YCbCr 4:2:0 / 4:2:2 tiles) now decodes to linear RGB;
private 3:2 & 4:3 examples verified through native rendering & full-resolution 16-bit export.
Decoded (CC0 corpus from raw.pixls.us, `cargo xtask corpus --download`, `crates/raw/tests/corpus.rs`): DNG (uncompressed,
LJ92, lossy JPEG / Smart Previews, Deflate, float, linear), CR2, ARW (uncompressed, ARW2, LJ92; as-shot white balance and black level of pre-2017 bodies from the enciphered
maker-note `Tag2010` and the encrypted `SR2SubIFD`, both recovered by black-box analysis, `crates/raw/src/vendor/arw.rs`), NEF/NRW uncompressed and Huffman-compressed (lossless, lossy type 1/2, 12/14-bit), RAF uncompressed (Bayer and
X-Trans), RW2 packed 12/14-bit, PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Every
X-Trans), RW2 / Leica RWL / Panasonic RAW in every raw format (compressed formats 4 and 6, the prefix-coded strips of format 8,
packed formats 2, 5 and 7, the 16-bit words of the 2005–2007 bodies; all recovered by black-box analysis of 178 CC0 files from
118 bodies, `crates/raw/src/vendor/rw2.rs`), PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Every
supported container also yields its embedded JPEG preview (CR3 too), and the engine shows that preview for raw variants
it can't decode yet.
@@ -174,14 +179,11 @@ Not decoded yet — preview only (no permissively licensed description; black-bo
files): rows above the split decode with the regular lossy table; from the split row on a different code is used
that our black-box analysis has not recovered yet (none of the four regular tables fits, also not with byte
alignment or reset predictors at the split row). The other Nikon Huffman variants are decoded (`crates/raw/src/vendor/nefc.rs` documents the analysis).
- **Panasonic RW2 raw format 4** (quantised): the block layout is known (0x4000-byte chunks rotated by 0x1ff8; 128-bit
blocks of two 12-bit seeds plus four groups of a 2-bit scale and three 8-bit codes; scales 0/1 are ×1/×2 differences
from the same-colour pixel two to the left), the reconstruction rule for scales 2/3 is not established.
- **Olympus compressed ORF**, **Fujifilm compressed RAF**, **Canon CR3/CRX** (M11.1), **Canon sRAW/mRAW**, lossy DNG.
**Camera colour matrices:** ARW files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
**Camera colour matrices:** ARW, NEF and RW2 files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
`matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer
matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax/Panasonic equivalents) and our
matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax equivalents; Panasonic RW2 files carry none) and our
own chart-based calibration (M11.4). Adobe matrices are never used.
## Log
@@ -193,6 +195,9 @@ own chart-based calibration (M11.4). Adobe matrices are never used.
Local roots and cleanup, 85k-photo grid and catalog performance, background compaction, failed-save errors, GPU
export hardening, clippy 1.99. Added the honest *Where we stand* assessment and *Where we're going* priorities; added
tracker rows for camera colour, camera coverage and render fidelity.
- 2026-10-07: Panasonic RW2 / Leica RWL / Panasonic RAW decode in every raw format (compressed 4/6/8, packed 2/5/7,
16-bit words), black levels, in-camera aspect crops and mapped-out defects established on 178 CC0 files from 118
bodies; RW2 gets the file-local camera look (accepted on 140 of 174 files with a preview).
## Chinese and Japanese interfaces, and text watermarks
+3 -1
View File
@@ -292,7 +292,9 @@ fn devices_are_listed_and_imported_from() {
assert_eq!(dev["name"], "CARD", "{line}");
// the import review of its DCIM folder finds the photo
let dcim = dev["path"].as_str().unwrap().to_string();
let o = Command::new(BIN).args(["run", "library.importPreview", &format!("paths=[\"{dcim}\"]")]).output().unwrap();
// JSON-escaped: a Windows path has backslashes
let paths = serde_json::to_string(&[&dcim]).unwrap();
let o = Command::new(BIN).args(["run", "library.importPreview", &format!("paths={paths}")]).output().unwrap();
let line: Value = serde_json::from_slice(o.stdout.split(|b| *b == b'\n').next().unwrap()).unwrap();
assert_eq!(line["result"]["candidates"].as_array().map(Vec::len), Some(1), "{line}");
}
+2 -1
View File
@@ -428,8 +428,9 @@ impl WebApp {
let cores = window().map_or(1, |w| w.navigator().hardware_concurrency() as usize);
cores.saturating_sub(1).clamp(1, 4)
});
let cache = app.session.media.rendered.clone();
let workers =
(n > 0).then(|| Workers::start(n, backend.as_ref().map_or("memory", |b| b.kind()), backend.clone(), index, cc.egui_ctx.clone()));
(n > 0).then(|| Workers::start(n, backend.as_ref().map_or("memory", |b| b.kind()), backend.clone(), index, &cache, cc.egui_ctx.clone()));
if let Some(w) = &workers {
app.renderer.set_offload(Box::new(w.clone()));
}
+138 -4
View File
@@ -11,13 +11,13 @@
//! ([`ThumbIndex`]).
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::{Arc, Weak};
use lightcraft_engine::catalog::Source;
use lightcraft_engine::develop::{DevelopSettings, EmbeddedLens};
use lightcraft_engine::media::{DecodedSource, MediaCache, RenderJob, SourceLevel, SourceRef};
use lightcraft_engine::pipeline::{Quality, RenderRequest, Rendered, SourceInfo, StageCache};
use lightcraft_preview::Lru;
use lightcraft_preview::{Hash128, Lru, PreviewCache};
use serde::{Deserialize, Serialize};
use crate::store::hash_of_path;
@@ -25,6 +25,10 @@ use crate::store::hash_of_path;
/// One render request for a worker.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct WireJob {
#[serde(default)]
pub cache_generation: u64,
#[serde(default)]
pub source_identity: Option<String>,
pub level: SourceLevel,
pub origin: Source,
/// Decode the source at most this long.
@@ -60,6 +64,8 @@ impl WireJob {
SourceRef::Smart { .. } => 2560,
};
WireJob {
cache_generation: job.cache_generation,
source_identity: job.source_key.map(|key| key.to_string()),
level: job.level,
origin: job.origin.clone(),
max_edge,
@@ -108,8 +114,8 @@ impl WireJob {
/// Identifies the decoded source this job needs.
fn source_key(&self) -> String {
match &self.origin {
Source::File { path } => format!("{}:{path}", self.max_edge),
Source::Demo { scene } => format!("{}:demo:{scene}", self.max_edge),
Source::File { path } => format!("{}:{:?}:{}:{path}", self.cache_generation, self.source_identity, self.max_edge),
Source::Demo { scene } => format!("{}:{:?}:{}:demo:{scene}", self.cache_generation, self.source_identity, self.max_edge),
}
}
@@ -186,6 +192,8 @@ impl WorkerCore {
/// pruned like the native disk cache (least recently used first, down to 80 % of the budget).
#[derive(Default, Serialize, Deserialize)]
pub struct ThumbIndex {
#[serde(default)]
namespace: u64,
/// hex key → (bytes, last use).
entries: HashMap<String, (u64, u64)>,
clock: u64,
@@ -196,6 +204,24 @@ pub struct ThumbIndex {
}
impl ThumbIndex {
pub fn namespace(&self) -> u64 {
self.namespace
}
/// A persistent new namespace prevents late old workers restoring an invalidated disk hit.
pub fn cache_key(&self, key: Hash128) -> String {
if self.namespace == 0 {
return key.to_string();
}
lightcraft_preview::Hasher128::new().str(&key.to_string()).str("cleared").u64(self.namespace).finish().to_string()
}
pub fn invalidate(&mut self) -> Vec<String> {
self.namespace = self.namespace.wrapping_add(1);
self.total = 0;
self.dirty = true;
self.entries.drain().map(|(key, _)| key).collect()
}
pub fn from_json(bytes: &[u8]) -> ThumbIndex {
let mut i: ThumbIndex = serde_json::from_slice(bytes).unwrap_or_default();
i.total = i.entries.values().map(|e| e.0).sum();
@@ -273,11 +299,55 @@ impl ThumbIndex {
}
}
/// Which rendered-thumbnail cache, at which generation, the stored thumbnails ([`ThumbIndex`])
/// belong to (main thread). Clearing the previews bumps the cache's generation, and opening
/// another library replaces the cache: either makes every stored thumbnail obsolete.
pub struct CacheWatch(Option<(Weak<PreviewCache>, u64)>);
impl CacheWatch {
/// Watch `active`, the session's cache when the workers start: the persisted index belongs
/// to it at its current generation. (Without this baseline, the first observation after a
/// clear would only record the cleared generation and keep the obsolete thumbnails.)
pub fn new(active: &Arc<PreviewCache>) -> CacheWatch {
CacheWatch(Some((Arc::downgrade(active), active.generation())))
}
/// The watched cache, while it's alive.
pub fn cache(&self) -> Option<Arc<PreviewCache>> {
self.0.as_ref().and_then(|(cache, _)| cache.upgrade())
}
/// `cache` at `generation` is the active one now. If it was cleared or replaced since the
/// last observation, invalidate `index` and return the keys whose files should be deleted.
pub fn observe(&mut self, cache: &Arc<PreviewCache>, generation: u64, index: &mut ThumbIndex) -> Vec<String> {
let changed = self.0.as_ref().is_some_and(|(old, g)| old.as_ptr() != Arc::as_ptr(cache) || *g != generation);
self.0 = Some((Arc::downgrade(cache), generation));
if changed { index.invalidate() } else { Vec::new() }
}
}
#[cfg(test)]
mod tests {
use super::*;
use lightcraft_engine::Session;
#[test]
fn clearing_index_changes_disk_namespace_and_survives_restart() {
let key = lightcraft_preview::hash_bytes(b"thumbnail");
let mut index = ThumbIndex::default();
let old = index.cache_key(key);
assert_eq!(old, key.to_string());
index.insert(&old, 100);
assert_eq!(index.invalidate(), vec![old.clone()]);
let new = index.cache_key(key);
assert_ne!(old, new);
index.insert(&new, 100);
let reopened = ThumbIndex::from_json(&index.to_json());
assert_eq!(reopened.cache_key(key), new);
assert!(!reopened.contains(&old));
assert!(reopened.contains(&new));
}
#[test]
fn wire_job_renders_like_the_engine() {
let mut s = Session::with_demo();
@@ -304,6 +374,8 @@ mod tests {
let hash = crate::store::content_hash(&bytes);
let origin = Source::File { path: crate::store::original_path(&hash, "a.png") };
let job = WireJob {
cache_generation: 0,
source_identity: None,
level: SourceLevel::Thumb,
origin,
max_edge: 512,
@@ -328,6 +400,68 @@ mod tests {
let r = core.render(&job, Some(&bytes)).unwrap();
assert_eq!(r.image.width, 32);
assert_eq!(core.needs_original(&job), None, "decoded source is kept");
let mut reloaded = job.clone();
reloaded.source_identity = Some("changed content".into());
assert_eq!(core.needs_original(&reloaded).as_deref(), Some(hash.as_str()), "content reload must not reuse an old decoded source");
let mut refreshed = job.clone();
refreshed.cache_generation = 1;
assert_eq!(core.needs_original(&refreshed).as_deref(), Some(hash.as_str()), "explicit refresh must not reuse the old decoded source");
}
/// The persisted index as a restarted page loads it: one stored thumbnail.
fn persisted_index(key: Hash128) -> (ThumbIndex, String) {
let mut index = ThumbIndex::default();
let hex = index.cache_key(key);
index.insert(&hex, 100);
(ThumbIndex::from_json(&index.to_json()), hex)
}
#[test]
fn clearing_previews_before_the_first_request_invalidates_stored_thumbnails() {
let key = lightcraft_preview::hash_bytes(b"thumbnail");
let (mut index, old) = persisted_index(key);
let cache = Arc::new(PreviewCache::memory(1 << 20));
let mut watch = CacheWatch::new(&cache);
let mut gone = Vec::new();
// each frame (`Workers::finished`) observes the watched cache, even with no requests
let mut frame = |watch: &mut CacheWatch, index: &mut ThumbIndex| {
if let Some(c) = watch.cache() {
gone.extend(watch.observe(&c, c.generation(), index));
}
};
frame(&mut watch, &mut index); // an empty filtered view: no render request yet
cache.clear(); // File ▸ Clear Preview Cache
frame(&mut watch, &mut index);
// the filter is removed: the first thumbnail request (`Workers::try_start`)
gone.extend(watch.observe(&cache, cache.generation(), &mut index));
let disk_key = index.cache_key(key);
assert!(!index.touch(&disk_key), "the pre-clear thumbnail must not be read back");
assert_eq!(gone, vec![old.clone()], "the stored file is deleted");
assert_ne!(disk_key, old, "a fresh namespace: a late old write can't come back");
assert!(index.dirty, "the invalidated index is saved");
}
#[test]
fn replacing_the_cache_before_the_first_request_invalidates_stored_thumbnails() {
let key = lightcraft_preview::hash_bytes(b"thumbnail");
let (mut index, old) = persisted_index(key);
let first = Arc::new(PreviewCache::memory(1 << 20));
let mut watch = CacheWatch::new(&first);
let other = Arc::new(PreviewCache::memory(1 << 20)); // another library, generation 0 again
assert_eq!(watch.observe(&other, other.generation(), &mut index), vec![old]);
}
#[test]
fn a_normal_start_keeps_stored_thumbnails() {
let key = lightcraft_preview::hash_bytes(b"thumbnail");
let (mut index, old) = persisted_index(key);
let cache = Arc::new(PreviewCache::memory(1 << 20));
let mut watch = CacheWatch::new(&cache);
for _ in 0..3 {
assert!(watch.observe(&cache, cache.generation(), &mut index).is_empty());
}
assert_eq!(index.cache_key(key), old);
assert!(index.touch(&old), "persisted thumbnails survive a restart");
}
#[test]
+93 -10
View File
@@ -27,7 +27,7 @@ use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
use crate::backend::Backend;
use crate::wire::{ThumbIndex, WireJob, WorkerCore, thumb_storage_key};
use crate::wire::{CacheWatch, ThumbIndex, WireJob, WorkerCore, thumb_storage_key};
/// Storage key of the thumbnail index.
pub const THUMB_INDEX: &str = "thumbs/index.json";
@@ -155,6 +155,10 @@ async fn handle(scope: &web_sys::DedicatedWorkerGlobalScope, backend: Option<Bac
// main side
struct Busy {
disk_key: Option<String>,
namespace: u64,
request_id: u64,
cache_generation: u64,
id: u32,
slot: Slot,
photo: PhotoId,
@@ -171,6 +175,8 @@ struct W {
}
struct Inner {
/// The rendered-thumbnail cache the stored thumbnails belong to.
preview_generation: CacheWatch,
workers: Vec<W>,
done: Vec<(Slot, RenderResult, f64)>,
next_id: u32,
@@ -185,14 +191,32 @@ struct Inner {
inline_this_frame: bool,
}
impl Inner {
fn observe_cache(&mut self, cache: &Arc<PreviewCache>, generation: u64) {
let gone = self.preview_generation.observe(cache, generation, &mut self.index);
if !gone.is_empty()
&& let Some(backend) = self.backend.clone()
{
wasm_bindgen_futures::spawn_local(async move {
for key in gone {
let _ = backend.remove(&thumb_storage_key(&key)).await;
}
});
}
}
}
/// The main thread's handle on the render workers.
#[derive(Clone)]
pub struct Workers(Rc<RefCell<Inner>>);
impl Workers {
/// Start `n` workers (none: every job runs inline). `store` is the backend kind for workers.
pub fn start(n: usize, store: &str, backend: Option<Backend>, index: ThumbIndex, ctx: egui::Context) -> Workers {
/// `index` lists the stored thumbnails of `cache`, the session's rendered-thumbnail cache
/// now: watching it from the start, a clear before the first render request still counts.
pub fn start(n: usize, store: &str, backend: Option<Backend>, index: ThumbIndex, cache: &Arc<PreviewCache>, ctx: egui::Context) -> Workers {
let w = Workers(Rc::new(RefCell::new(Inner {
preview_generation: CacheWatch::new(cache),
workers: Vec::new(),
done: Vec::new(),
next_id: 0,
@@ -248,7 +272,16 @@ impl Workers {
w.dead = true;
w.worker.terminate();
if let Some(b) = w.busy.take() {
let r = RenderResult { photo: b.photo, level: b.level, key: b.key, rendered: Err(msg), loaded: None, quick: None };
let r = RenderResult {
request_id: b.request_id,
source_key: None,
photo: b.photo,
level: b.level,
key: b.key,
rendered: Err(msg),
loaded: None,
quick: None,
};
g.done.push((b.slot, r, 0.0));
}
}
@@ -284,9 +317,11 @@ impl Workers {
};
if let Ok(r) = &rendered
&& let Some((cache, key)) = &b.cache
&& cache.generation() == b.cache_generation
&& b.namespace == g.index.namespace()
{
cache.put(*key, Arc::new(r.image.clone()));
let hex = key.to_string();
cache.put_at(b.cache_generation, *key, Arc::new(r.image.clone()));
let hex = b.disk_key.clone().unwrap_or_else(|| key.to_string());
let stored = get(&data, "stored").as_f64().unwrap_or(0.0) as u64;
if get(&data, "cacheMiss").is_truthy() {
g.index.remove(&hex);
@@ -307,8 +342,31 @@ impl Workers {
}
}
g.affinity.insert((b.photo, b.level), i);
// A cleared worker may finish writing after the initial deletion. Its old namespace
// is never reused, so removing this file cannot delete a replacement's cache entry.
if let Some((cache, _)) = &b.cache
&& (cache.generation() != b.cache_generation || b.namespace != g.index.namespace())
&& let (Some(backend), Some(key)) = (g.backend.clone(), b.disk_key.clone())
{
wasm_bindgen_futures::spawn_local(async move {
let _ = backend.remove(&thumb_storage_key(&key)).await;
});
}
g.remote_done += 1;
g.done.push((b.slot, RenderResult { photo: b.photo, level: b.level, key: b.key, rendered, loaded: None, quick: None }, ms));
g.done.push((
b.slot,
RenderResult {
request_id: b.request_id,
source_key: None,
photo: b.photo,
level: b.level,
key: b.key,
rendered,
loaded: None,
quick: None,
},
ms,
));
}
/// The thumbnail index as JSON if it changed since the last call (the host saves it).
@@ -334,13 +392,18 @@ impl RenderOffload for Workers {
fn try_start(&mut self, slot: Slot, job: RenderJob) -> Option<RenderJob> {
let mut g = self.0.borrow_mut();
let g = &mut *g;
if let Some((cache, _)) = job.cache.as_ref().or(job.view_cache.as_ref()) {
g.observe_cache(cache, job.cache_generation);
}
// rendered thumbnail in memory: no worker needed
if let Some((cache, key)) = &job.cache
&& let Some(img) = cache.get(*key)
&& let Some(img) = cache.get_at(job.cache_generation, *key)
{
let image = Arc::unwrap_or_clone(img);
let histogram = Histogram::of_srgb8(&image);
let r = RenderResult {
request_id: job.request_id,
source_key: job.source_key,
photo: job.photo,
level: job.level,
key: job.key,
@@ -374,25 +437,45 @@ impl RenderOffload for Workers {
let Some(i) = pick else { return Some(job) };
// decided here, so the next job for this photo follows even before this one finishes
g.affinity.insert((job.photo, job.level), i);
let thumb_cached = job.cache.as_ref().is_some_and(|(_, k)| g.index.touch(&k.to_string()));
let wire = WireJob::from_job(&job, thumb_cached, &format!("{slot:?}"));
let disk_key = job.cache.as_ref().map(|(_, key)| g.index.cache_key(*key));
let thumb_cached = disk_key.as_ref().is_some_and(|key| g.index.touch(key));
let mut wire = WireJob::from_job(&job, thumb_cached, &format!("{slot:?}"));
// The namespace also changes when a new library's cache starts again at generation zero.
wire.cache_generation = g.index.namespace();
wire.thumb = disk_key.clone();
let Ok(text) = serde_json::to_string(&wire) else { return Some(job) };
g.next_id = g.next_id.wrapping_add(1);
let id = g.next_id;
let msg = Object::new();
set(&msg, "id", &id.into());
set(&msg, "job", &text.into());
let namespace = g.index.namespace();
let w = &mut g.workers[i];
if let Err(e) = w.worker.post_message(&msg) {
log::warn!("render worker {i}: {e:?}");
return Some(job);
}
w.busy = Some(Busy { id, slot, photo: job.photo, level: job.level, key: job.key, cache: job.cache });
w.busy = Some(Busy {
disk_key,
namespace,
request_id: job.request_id,
cache_generation: job.cache_generation,
id,
slot,
photo: job.photo,
level: job.level,
key: job.key,
cache: job.cache,
});
None
}
fn finished(&mut self) -> Vec<(Slot, RenderResult, f64)> {
let mut g = self.0.borrow_mut();
// Persist invalidation even when the cleared library has no visible render requests.
if let Some(cache) = g.preview_generation.cache() {
g.observe_cache(&cache, cache.generation());
}
g.inline_this_frame = false;
std::mem::take(&mut g.done)
}
+2 -2
View File
@@ -286,8 +286,8 @@ fn services() -> Services {
.add_filter(
lightcraft_ui_egui::i18n::tr("Photos"),
&[
"jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "nrw", "arw", "raf", "orf", "rw2", "pef", "psd",
"jxl", "gif", "bmp",
"jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "nrw", "arw", "raf", "orf", "rw2", "rwl", "raw",
"pef", "psd", "jxl", "gif", "bmp",
],
)
.pick_files()
+466
View File
@@ -0,0 +1,466 @@
{
"schema": 1,
"repo": "storytold/lightcraft",
"branch": "main",
"head": "2694d7089fbf4ada62c6738a7c62fecdfd5711fb",
"head_date": "2026-10-08T00:53:19Z",
"commits": 424,
"unattributed_commits": 0,
"contributors": [
{
"login": "echelon",
"people_entry": true,
"github_name": null,
"prs": 71,
"commits": 330,
"lines_added": 125623,
"lines_deleted": 8643,
"binary_added": 38,
"binary_deleted": 8,
"first_commit": "2026-09-30T14:04:12Z",
"last_commit": "2026-10-08T00:32:34Z"
},
{
"login": "leipsfur",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 131,
"lines_deleted": 5,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-02T15:51:58Z",
"last_commit": "2026-10-02T15:51:58Z"
},
{
"login": "GlemNL",
"people_entry": false,
"github_name": null,
"prs": 9,
"commits": 13,
"lines_added": 1233,
"lines_deleted": 96,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-02T21:16:29Z",
"last_commit": "2026-10-04T09:42:39Z"
},
{
"login": "storybored379",
"people_entry": false,
"github_name": null,
"prs": 7,
"commits": 7,
"lines_added": 1194,
"lines_deleted": 82,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-04T00:28:49Z",
"last_commit": "2026-10-06T06:51:43Z"
},
{
"login": "midasdf",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 424,
"lines_deleted": 25,
"binary_added": 1,
"binary_deleted": 0,
"first_commit": "2026-10-05T16:07:26Z",
"last_commit": "2026-10-05T16:07:26Z"
},
{
"login": "voshart",
"people_entry": false,
"github_name": null,
"prs": 3,
"commits": 12,
"lines_added": 2481,
"lines_deleted": 1389,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-05T16:23:27Z",
"last_commit": "2026-10-07T16:07:24Z"
},
{
"login": "arata-1972",
"people_entry": false,
"github_name": null,
"prs": 2,
"commits": 4,
"lines_added": 2892,
"lines_deleted": 726,
"binary_added": 2,
"binary_deleted": 0,
"first_commit": "2026-10-05T20:14:35Z",
"last_commit": "2026-10-05T22:16:15Z"
},
{
"login": "benlukka",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 49,
"lines_deleted": 4,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-06T11:18:03Z",
"last_commit": "2026-10-06T11:18:03Z"
},
{
"login": "kywk",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 72,
"lines_deleted": 8,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-06T20:21:14Z",
"last_commit": "2026-10-06T20:21:14Z"
},
{
"login": "treyreynolds",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 159,
"lines_deleted": 27,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-06T22:21:46Z",
"last_commit": "2026-10-06T22:21:46Z"
},
{
"login": "kovalskiguinther",
"people_entry": false,
"github_name": null,
"prs": 0,
"commits": 1,
"lines_added": 4328,
"lines_deleted": 17,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-06T23:15:04Z",
"last_commit": "2026-10-06T23:15:04Z"
},
{
"login": "cruzer45",
"people_entry": false,
"github_name": null,
"prs": 2,
"commits": 3,
"lines_added": 658,
"lines_deleted": 26,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T01:30:23Z",
"last_commit": "2026-10-07T03:53:09Z"
},
{
"login": "coygeek",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 4,
"lines_deleted": 0,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T02:27:15Z",
"last_commit": "2026-10-07T02:27:15Z"
},
{
"login": "lzjtugjc",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 1895,
"lines_deleted": 294,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T03:34:33Z",
"last_commit": "2026-10-07T03:34:33Z"
},
{
"login": "itsmunzir",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 46,
"lines_deleted": 30,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T07:40:55Z",
"last_commit": "2026-10-07T07:40:55Z"
},
{
"login": "haraldh",
"people_entry": false,
"github_name": null,
"prs": 2,
"commits": 5,
"lines_added": 1163,
"lines_deleted": 141,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T07:54:27Z",
"last_commit": "2026-10-07T13:16:08Z"
},
{
"login": "sakamoto824",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 2,
"lines_added": 3649,
"lines_deleted": 167,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T09:32:12Z",
"last_commit": "2026-10-07T10:54:19Z"
},
{
"login": "anarion80",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 402,
"lines_deleted": 2,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T11:54:51Z",
"last_commit": "2026-10-07T11:54:51Z"
},
{
"login": "radialmonster",
"people_entry": false,
"github_name": null,
"prs": 3,
"commits": 19,
"lines_added": 2630,
"lines_deleted": 259,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T12:33:51Z",
"last_commit": "2026-10-08T00:46:10Z"
},
{
"login": "bogusyogi",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 434,
"lines_deleted": 29,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T17:06:00Z",
"last_commit": "2026-10-07T17:06:00Z"
},
{
"login": "nledford",
"people_entry": false,
"github_name": null,
"prs": 4,
"commits": 8,
"lines_added": 525,
"lines_deleted": 60,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T17:33:38Z",
"last_commit": "2026-10-07T22:49:00Z"
},
{
"login": "MusseJusse",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 34,
"lines_deleted": 3,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T18:38:23Z",
"last_commit": "2026-10-07T18:38:23Z"
},
{
"login": "Ndsanes",
"people_entry": false,
"github_name": null,
"prs": 2,
"commits": 2,
"lines_added": 332,
"lines_deleted": 86,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T19:08:28Z",
"last_commit": "2026-10-07T19:08:33Z"
},
{
"login": "RicSchonfelder",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 1439,
"lines_deleted": 6,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T20:03:20Z",
"last_commit": "2026-10-07T20:03:20Z"
},
{
"login": "Gremiger",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 19,
"lines_deleted": 0,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T21:43:23Z",
"last_commit": "2026-10-07T21:43:23Z"
},
{
"login": "DWAA1660",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 54,
"lines_deleted": 12,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T22:08:01Z",
"last_commit": "2026-10-07T22:08:01Z"
},
{
"login": "agorevski",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 68,
"lines_deleted": 11,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-07T22:18:44Z",
"last_commit": "2026-10-07T22:18:44Z"
},
{
"login": "andrewyao",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 1,
"lines_added": 1,
"lines_deleted": 1,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": "2026-10-08T00:53:19Z",
"last_commit": "2026-10-08T00:53:19Z"
},
{
"login": "GuintherKovalski",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 0,
"lines_added": 0,
"lines_deleted": 0,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": null,
"last_commit": null
},
{
"login": "XRTce",
"people_entry": false,
"github_name": null,
"prs": 1,
"commits": 0,
"lines_added": 0,
"lines_deleted": 0,
"binary_added": 0,
"binary_deleted": 0,
"first_commit": null,
"last_commit": null
}
],
"models": [
{
"id": "anthropic/claude-opus/5.5",
"company": "Anthropic",
"model": "Claude Opus",
"version": "5.5",
"commits": 364,
"lines_added": 131810,
"lines_deleted": 9367
},
{
"id": "anthropic/claude-sonnet/5.5",
"company": "Anthropic",
"model": "Claude Sonnet",
"version": "5.5",
"commits": 31,
"lines_added": 3941,
"lines_deleted": 1543
},
{
"id": "openai/codex",
"company": "OpenAI",
"model": "Codex",
"version": "",
"commits": 14,
"lines_added": 8341,
"lines_deleted": 995
},
{
"id": "anthropic/claude",
"company": "Anthropic",
"model": "Claude",
"version": "",
"commits": 2,
"lines_added": 1304,
"lines_deleted": 192
},
{
"id": "anthropic/claude-sonnet/5",
"company": "Anthropic",
"model": "Claude Sonnet",
"version": "5",
"commits": 2,
"lines_added": 298,
"lines_deleted": 2
},
{
"id": "anthropic/claude-opus/4.8",
"company": "Anthropic",
"model": "Claude Opus",
"version": "4.8",
"commits": 1,
"lines_added": 1633,
"lines_deleted": 3
},
{
"id": "deepseek/deepseek/v3.2",
"company": "DeepSeek",
"model": "DeepSeek",
"version": "V3.2",
"commits": 1,
"lines_added": 1895,
"lines_deleted": 294
}
]
}
+23
View File
@@ -0,0 +1,23 @@
# Contributor names for the About window (craftrules standards/contributors.md).
#
# Everyone who contributes is credited by GitHub username automatically. Anything more is up to
# you: add YOUR OWN entry below, in your own PR (commit it yourself). Never add someone else; a
# maintainer may add a person only when that person asked, recording where in consent_source.
#
# Easiest: python3 ../../craftrules/scripts/contributors.py --add-me . \
# [--real-name "Ada Lovelace"] [--display-name "Ada"] [--sync-github-name]
#
# Or copy this block, replacing your-github-username (every field is optional):
#
# [people.your-github-username]
# real_name = "Your Name" # shown when the viewer picks "Real name"
# display_name = "Nickname" # shown when the viewer picks "Display name"
# sync_github_name = true # or: use the name on your public GitHub profile as display name
#
# To remove your names, delete your entry (or set fields empty) in a PR. The next
# contributors.py run, and the next build, drop them.
[people.echelon]
real_name = "Brandon Thomas"
display_name = "Brandon Thomas"
sync_github_name = false
+1 -1
View File
@@ -35,7 +35,7 @@ use lightcraft_develop::DevelopSettings;
pub use local::{DEFAULT_FORGET_DAYS, ForgetPlan, folder_of};
pub use lock::{LibraryLock, LockError, LockOwner};
pub use model::*;
pub use query::{DateGroup, Filter, Person, RatingOp, Sort, SortKey};
pub use query::{DateGroup, Filter, Person, RatingOp, Sort, SortKey, mix64};
pub use rules::{Match, Rule, RuleSet};
use serde::{Deserialize, Serialize};
pub use store::{FsStore, MemStore, Store};
+13 -4
View File
@@ -336,11 +336,12 @@ impl Photo {
pub fn develops_raw(&self) -> bool {
self.kind == MediaKind::Raw && self.preview_only.is_none()
}
/// The current ARW and NEF readers have vendor WB multipliers but no measured camera
/// The current ARW, NEF and RW2 readers have vendor WB multipliers but no measured camera
/// illuminant. Use adjustments relative to the camera's as-shot look, as for rendered
/// photographs (the engine's `camera_preview::file_local_look` covers the same formats).
/// photographs (the engine's `camera_preview::file_local_look` covers the same formats; RWL and
/// RAW are Leica's and the oldest Panasonic bodies' names for RW2 files).
pub fn relative_wb(&self) -> bool {
self.develops_raw() && ["ARW", "NEF", "NRW"].iter().any(|f| self.format.eq_ignore_ascii_case(f))
self.develops_raw() && ["ARW", "NEF", "NRW", "RW2", "RWL", "RAW"].iter().any(|f| self.format.eq_ignore_ascii_case(f))
}
/// The develop settings import gave this photo: [`Photo::camera_defaults`], or the user's
/// default preset applied on top of them ([`Photo::import_look`]).
@@ -430,7 +431,15 @@ mod edited_tests {
#[test]
fn sony_and_nikon_raws_use_relative_white_balance() {
for (name, format, relative) in [("a.arw", "ARW", true), ("a.nef", "NEF", true), ("a.nrw", "nrw", true), ("a.dng", "DNG", false)] {
for (name, format, relative) in [
("a.arw", "ARW", true),
("a.nef", "NEF", true),
("a.nrw", "nrw", true),
("a.rw2", "RW2", true),
("a.rwl", "RWL", true),
("a.raw", "RAW", true),
("a.dng", "DNG", false),
] {
let mut p = Photo::new(PhotoId(1), Source::Demo { scene: 0 }, name, format, 10, 10, "2026-10-01T00:00:00");
p.kind = MediaKind::Raw;
p.as_shot_wb = Some((5200.0, 4.0));
+21 -1
View File
@@ -72,6 +72,9 @@ pub enum SortKey {
FileName,
Rating,
FileSize,
/// A shuffle fixed by [`Sort::seed`]: the same seed always gives the same order, and photos
/// added or edited later never reshuffle the others. Pick a new seed to reshuffle.
Random,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
@@ -81,14 +84,30 @@ pub struct Sort {
pub ascending: bool,
/// Date headers in the grid (date sort keys only).
pub group: crate::GroupBy,
/// Which shuffle [`SortKey::Random`] gives (ignored by the other keys).
pub seed: u64,
}
impl Default for Sort {
fn default() -> Self {
Sort { key: SortKey::CaptureDate, ascending: false, group: crate::GroupBy::Auto }
Sort { key: SortKey::CaptureDate, ascending: false, group: crate::GroupBy::Auto, seed: 0 }
}
}
/// splitmix64 finaliser: a stateless, platform-independent mix (no RNG state, no `rand` version
/// to drift), so a seed names the same shuffle on every machine.
pub fn mix64(x: u64) -> u64 {
let mut z = x.wrapping_add(0x9E37_79B9_7F4A_7C15);
z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9);
z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB);
z ^ (z >> 31)
}
/// A photo's place in the shuffle for `seed`.
fn shuffle_rank(seed: u64, id: PhotoId) -> u64 {
mix64(mix64(seed) ^ id.0)
}
fn token_matches(p: &Photo, tok: &str) -> bool {
let t = tok.to_lowercase();
if let Some((field, val)) = t.split_once(':') {
@@ -307,6 +326,7 @@ impl Catalog {
SortKey::FileName => a.file_name.to_lowercase().cmp(&b.file_name.to_lowercase()),
SortKey::Rating => a.rating.cmp(&b.rating),
SortKey::FileSize => a.file_size.cmp(&b.file_size),
SortKey::Random => shuffle_rank(sort.seed, a.id).cmp(&shuffle_rank(sort.seed, b.id)),
}
.then(a.id.cmp(&b.id));
if sort.ascending { o } else { o.reverse() }
+27 -1
View File
@@ -7,7 +7,7 @@
use serde::{Deserialize, Serialize};
use serde_json::Value;
use crate::{Catalog, Photo};
use crate::{Catalog, Photo, Source};
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
@@ -62,6 +62,7 @@ pub const FIELDS: &[(&str, &str, Kind)] = &[
("keywords", "Keywords", Kind::Keywords),
("text", "Any Searchable Text", Kind::Text),
("fileName", "Filename", Kind::Text),
("filePath", "File Path", Kind::Text),
("format", "File Format", Kind::Text),
("title", "Title", Kind::Text),
("caption", "Caption", Kind::Text),
@@ -304,6 +305,20 @@ impl Rule {
text(&all)
}
"fileName" => text(&p.file_name),
"filePath" => {
// `/` and `\` both separate, whatever platform the catalog came from; demo photos have no path
let path = match &p.source {
Source::File { path } => path.replace('\\', "/"),
Source::Demo { .. } => String::new(),
};
let want = want.replace('\\', "/");
match op {
// the whole string, spaces included (not word by word like the other text fields)
"contains" => !want.trim().is_empty() && path.to_lowercase().contains(want.trim()),
"notContains" => want.trim().is_empty() || !path.to_lowercase().contains(want.trim()),
_ => text_op(op, &path, &want),
}
}
"format" => text(&p.format),
"title" => text(&m.title),
"caption" => text(&m.caption),
@@ -445,6 +460,17 @@ mod tests {
);
no(json!({"rules": [{"field": "keywords", "op": "isEmpty"}]}));
yes(json!({"rules": [{"field": "fileName", "op": "startsWith", "value": "img_"}, {"field": "fileName", "op": "endsWith", "value": ".cr2"}]}));
// file path: the whole string (spaces included), either separator, case-insensitive
let mut fp = photo(3);
fp.source = Source::File { path: "D:\\Photos\\Aliah Ira Polanco-Grylls\\2026\\IMG_0042.CR2".into() };
let fpm = |v: serde_json::Value| rs(v).matches(&fp, &cat);
assert!(fpm(json!({"rules": [{"field": "filePath", "op": "contains", "value": "/Aliah Ira Polanco-Grylls/"}]})));
assert!(!fpm(json!({"rules": [{"field": "filePath", "op": "contains", "value": "/Ira Aliah/"}]})));
assert!(!fpm(json!({"rules": [{"field": "filePath", "op": "contains", "value": "Polanco Grylls"}]})));
assert!(fpm(json!({"rules": [{"field": "filePath", "op": "notContains", "value": "/Other/"}]})));
assert!(fpm(json!({"rules": [{"field": "filePath", "op": "startsWith", "value": "d:/photos/"}]})));
no(json!({"rules": [{"field": "filePath", "op": "contains", "value": "/Aliah/"}]})); // demo photo has no path
yes(json!({"rules": [{"field": "filePath", "op": "isEmpty"}]}));
yes(json!({"rules": [{"field": "camera", "op": "contains", "value": "x2"}, {"field": "title", "op": "isEmpty"}]}));
yes(json!({"rules": [{"field": "iso", "op": "between", "value": [800, 3200]}, {"field": "aperture", "op": "lte", "value": "f/4"}]}));
yes(json!({"rules": [{"field": "megapixels", "op": "gte", "value": 24}]}));
+66
View File
@@ -380,3 +380,69 @@ fn empty_regions_are_not_serialized_and_default_when_missing() {
let back: Meta = serde_json::from_str(&serde_json::to_string(&with).unwrap()).unwrap();
assert_eq!(back, with);
}
// ---- random sort: Given a seed, the order is a stable, reproducible shuffle of the matching photos
fn many(c: &mut Catalog, n: usize) -> Vec<PhotoId> {
(0..n).map(|i| photo(c, &format!("p{i:03}.jpg"), &format!("2026-04-{:02}T10:00:00", i % 28 + 1))).collect()
}
fn random(seed: u64) -> Sort {
Sort { key: SortKey::Random, seed, ..Default::default() }
}
#[test]
fn random_sort_is_a_reproducible_permutation() {
let mut c = Catalog::new();
let ids = many(&mut c, 50);
let a = c.query(&Filter::default(), &random(7));
assert_eq!(a, c.query(&Filter::default(), &random(7)), "same seed, same order");
let mut sorted = a.clone();
sorted.sort();
let mut all = ids.clone();
all.sort();
assert_eq!(sorted, all, "every photo exactly once");
assert_ne!(a, c.query(&Filter::default(), &random(8)), "another seed, another order");
assert_ne!(a, c.query(&Filter::default(), &Sort::default()), "not just the date order");
}
#[test]
fn random_sort_keeps_the_relative_order_when_the_set_changes() {
let mut c = Catalog::new();
let ids = many(&mut c, 30);
let before = c.query(&Filter::default(), &random(3));
let added = photo(&mut c, "new.jpg", "2026-05-01T10:00:00");
c.apply(Op::SetRating { id: ids[0], rating: 5 }).unwrap();
let after: Vec<PhotoId> = c.query(&Filter::default(), &random(3)).into_iter().filter(|id| *id != added).collect();
assert_eq!(after, before, "adding a photo or editing metadata must not reshuffle the others");
}
#[test]
fn random_sort_respects_the_filter_and_handles_tiny_sets() {
let mut c = Catalog::new();
assert!(c.query(&Filter::default(), &random(1)).is_empty());
let ids = many(&mut c, 10);
c.apply(Op::SetRating { id: ids[4], rating: 5 }).unwrap();
assert_eq!(c.query(&Filter { rating: 5, ..Default::default() }, &random(1)), vec![ids[4]]);
}
#[test]
fn random_sort_direction_reverses_and_serde_defaults_hold() {
let mut c = Catalog::new();
many(&mut c, 20);
let desc = c.query(&Filter::default(), &Sort { ascending: false, ..random(5) });
let mut asc = c.query(&Filter::default(), &Sort { ascending: true, ..random(5) });
asc.reverse();
assert_eq!(desc, asc);
// saved sorts from before the seed existed still load
let old: Sort = serde_json::from_str(r#"{"key":"fileName","ascending":true}"#).unwrap();
assert_eq!(old.seed, 0);
assert_eq!(serde_json::from_str::<Sort>(r#"{"key":"random","seed":9}"#).unwrap(), Sort { key: SortKey::Random, seed: 9, ..Default::default() });
}
#[test]
fn random_sort_has_no_date_headers() {
let mut c = Catalog::new();
let ids = many(&mut c, 5);
assert!(c.date_runs(&ids, SortKey::Random, GroupBy::Day).is_empty());
}
+6
View File
@@ -24,6 +24,12 @@ encode_webp_lossless(&img, &meta)?;
encode_avif(&img, 70, 6, &meta)?; // native + feature `avif`
```
Thumbnail fallback decoding limits the source to 64 million pixels by default.
Non-JPEG decoders allocate at source resolution before resizing, even for a tiny
thumbnail. Use `decode_thumbnail_with` and `ThumbnailOptions::max_pixels` to set
a different source budget. Oversized sources return `Error::TooLarge`; embedded
JPEG previews can still be used without decoding the full source image.
`Format::RawTiffLike` (DNG, CR2, NEF, ARW, PEF, ORF, RW2, SRW, …) and `Format::RawOther` (CR3, RAF,
CRW, MRW, X3F) are detected so the engine can route them to `lightcraft-raw`; `decode` returns
`Error::Unsupported` for them.
+10 -2
View File
@@ -217,11 +217,19 @@ pub struct ThumbnailOptions {
/// smaller size). Defaults to `max_edge`, i.e. only previews that need no upscaling. Lower it
/// (e.g. 160 for typical EXIF thumbnails) for an instant placeholder.
pub min_embedded_edge: u32,
/// Maximum source pixels for fallback decoding (default 64 MP). Non-JPEG formats
/// decode at source resolution before resizing, so output size does not bound memory.
#[serde(default = "thumbnail_max_pixels")]
pub max_pixels: u64,
}
fn thumbnail_max_pixels() -> u64 {
64_000_000
}
impl ThumbnailOptions {
pub fn new(max_edge: u32) -> Self {
ThumbnailOptions { max_edge, min_embedded_edge: max_edge }
ThumbnailOptions { max_edge, min_embedded_edge: max_edge, max_pixels: thumbnail_max_pixels() }
}
}
@@ -240,7 +248,7 @@ pub fn decode_thumbnail_with(bytes: &[u8], opts: &ThumbnailOptions) -> Result<Th
{
return Ok(t);
}
let d = decode(bytes, DecodeOptions::fit(max_edge, max_edge))?;
let d = decode(bytes, DecodeOptions { max_size: Some((max_edge, max_edge)), max_pixels: opts.max_pixels })?;
Ok(Thumbnail {
image: d.to_srgb8(),
orientation: d.orientation,
+43
View File
@@ -163,6 +163,9 @@ fn tiff_is_raw(b: &[u8]) -> bool {
}
let Some(t) = Tiff::new(b) else { return false };
let Some(ifd0) = t.first_ifd() else { return false };
let sony = t.ifd(ifd0).is_some_and(|(entries, _)| {
entries.iter().any(|e| e.tag == 0x010f && t.bytes(e).and_then(|v| v.get(..4)).is_some_and(|v| v.eq_ignore_ascii_case(b"SONY")))
});
let mut stack = vec![ifd0];
let mut seen = 0;
while let Some(pos) = stack.pop() {
@@ -171,6 +174,12 @@ fn tiff_is_raw(b: &[u8]) -> bool {
break;
}
let Some((entries, _)) = t.ifd(pos) else { continue };
// Sony M/S lossless ARWs retain a dummy CFA pattern, but store linear YCbCr.
// Requiring the raw pattern plus this layout keeps ordinary Sony TIFFs as TIFFs.
let uint = |tag| entries.iter().find(|e| e.tag == tag).and_then(|e| t.uint(e));
if sony && entries.iter().any(|e| e.tag == 0x828e) && uint(0x0103) == Some(7) && uint(0x0106) == Some(6) && uint(0x0115) == Some(3) {
return true;
}
for e in &entries {
match e.tag {
0xC612 => return true, // DNGVersion
@@ -233,4 +242,38 @@ mod tests {
b.extend_from_slice(&0u32.to_le_bytes());
assert_eq!(sniff(&b), Some(Format::RawTiffLike));
}
#[test]
fn sony_linear_ycbcr_subifd_is_raw_but_ordinary_tiff_is_not() {
fn fixture(compression: u32, pattern: bool) -> Vec<u8> {
let mut b = b"II*\0\x08\0\0\0".to_vec();
let entry = |b: &mut Vec<u8>, tag: u16, typ: u16, count: u32, value: u32| {
b.extend_from_slice(&tag.to_le_bytes());
b.extend_from_slice(&typ.to_le_bytes());
b.extend_from_slice(&count.to_le_bytes());
b.extend_from_slice(&value.to_le_bytes());
};
b.extend_from_slice(&2u16.to_le_bytes());
entry(&mut b, 0x010f, 2, 5, 38);
entry(&mut b, 0x014a, 4, 1, 44);
b.extend_from_slice(&0u32.to_le_bytes());
b.extend_from_slice(b"SONY\0\0");
b.extend_from_slice(&(if pattern { 4u16 } else { 3 }).to_le_bytes());
entry(&mut b, 0x0103, 3, 1, compression);
entry(&mut b, 0x0106, 3, 1, 6);
entry(&mut b, 0x0115, 3, 1, 3);
if pattern {
entry(&mut b, 0x828e, 1, 4, u32::MAX);
}
b.extend_from_slice(&0u32.to_le_bytes());
b
}
let mut raw = fixture(7, true);
assert_eq!(sniff(&raw), Some(Format::RawTiffLike));
assert_eq!(sniff(&fixture(7, false)), Some(Format::Tiff));
assert_eq!(sniff(&fixture(1, true)), Some(Format::Tiff));
assert_eq!(sniff(&raw[..38]), Some(Format::Tiff));
raw[38..42].copy_from_slice(b"TEST");
assert_eq!(sniff(&raw), Some(Format::Tiff));
}
}
+3
View File
@@ -12,6 +12,9 @@ fn err(e: impl std::fmt::Display) -> Error {
pub(crate) fn decode(bytes: &[u8], opts: &DecodeOptions) -> Result<Decoded> {
let mut d = image_webp::WebPDecoder::new(std::io::Cursor::new(bytes)).map_err(err)?;
// RIFF chunk lengths are untrusted. Metadata cannot exceed the input;
// otherwise a tiny truncated file can request gigabytes before read_exact fails.
d.set_memory_limit(bytes.len());
let (w, h) = d.dimensions();
check_size(F, w as u64, h as u64, opts)?;
let alpha = d.has_alpha();
+37 -1
View File
@@ -32,10 +32,46 @@ fn try_all(bytes: &[u8]) {
let _ = decode_unguarded(bytes, f, &DecodeOptions { max_size: Some((8, 8)), max_pixels: 1 << 22 });
}
let _ = decode(bytes, small_opts());
let _ = decode_thumbnail(bytes, 16);
let _ = decode_thumbnail_with(bytes, &ThumbnailOptions { max_pixels: small_opts().max_pixels, ..ThumbnailOptions::new(16) });
let _ = lightcraft_codecs::icc::parse(bytes);
}
#[test]
fn webp_truncated_metadata_cannot_allocate_declared_chunk_size() {
let mut bytes = seeds()[5].clone();
let tag = bytes.windows(4).rposition(|w| w == b"XMP ").unwrap();
bytes[tag + 4..tag + 8].copy_from_slice(&1_946_157_060u32.to_le_bytes());
let decoded = decode(&bytes, small_opts()).unwrap();
assert_eq!((decoded.image.width, decoded.image.height), (24, 16));
assert!(decoded.xmp.is_none(), "invalid optional metadata is ignored");
}
#[test]
fn thumbnail_rejects_huge_gif_before_allocating_source_pixels() {
// A logical screen header without a global palette; previously this reached a
// multi-gigabyte output allocation before discovering that no frame follows.
let bytes = [
0x47, 0x49, 0x46, 0x38, 0x39, 0x61, 0x8f, 0xa4, 0xb0, 0x3f, 0x57, 0x14, 0xe8, 0x8a, 0xf8, 0xe0, 0xc7, 0x41, 0x9d, 0x0c, 0x10, 0xde, 0x54,
0x24, 0xcb, 0xed, 0xff, 0x4a, 0xe7, 0xa0, 0x82, 0x10, 0xa0, 0xbc, 0xac, 0xe5, 0x33, 0x3f, 0xc9, 0x8d, 0x69, 0xf4, 0x24, 0xce, 0x99, 0x4f,
0x0d, 0x47, 0xa0, 0xb0, 0xf4, 0xa8, 0x59, 0xad, 0x29, 0xbd, 0x09, 0x48, 0x63, 0xea, 0x59, 0xb7, 0xb7, 0xf2, 0xa1, 0x90, 0xeb, 0x92, 0xc5,
0x9c, 0xde, 0xcc, 0x96, 0x67, 0xc4, 0xe5, 0x8c, 0x03, 0x50, 0xb2, 0x30, 0xf8, 0xf4, 0x69, 0x1d, 0xeb, 0xb9, 0xbd, 0x4e, 0xfb, 0xbd, 0xb6,
0x02,
];
let result = decode_thumbnail(&bytes, 16);
assert!(matches!(result, Err(Error::TooLarge(42_127, 16_304))), "{result:?}");
}
#[test]
fn thumbnail_respects_configured_source_pixel_limit() {
let image = Rgba8::from_fn(3, 2, |_, _| [40, 80, 120, 255]);
let bytes = encode_png(&EncodeImage::rgba8(&image), &EncodeMeta::default()).unwrap();
let mut opts = ThumbnailOptions::new(1);
opts.max_pixels = 5;
assert!(matches!(decode_thumbnail_with(&bytes, &opts), Err(Error::TooLarge(3, 2))));
opts.max_pixels = 6;
assert_eq!(decode_thumbnail_with(&bytes, &opts).unwrap().image.width, 1);
}
fn seeds() -> &'static Vec<Vec<u8>> {
static S: OnceLock<Vec<Vec<u8>>> = OnceLock::new();
S.get_or_init(|| {
+1 -1
View File
@@ -461,7 +461,7 @@ fn thumbnail_accepts_smaller_preview_when_asked() {
encode_jpeg(&EncodeImage::rgba8(&main), 80, ChromaSubsampling::S420, &EncodeMeta { exif: Some(&exif), ..Default::default() }).unwrap();
let t = decode_thumbnail(&bytes, 256).unwrap();
assert_eq!(t.source, ThumbnailSource::Scaled);
let t = decode_thumbnail_with(&bytes, &ThumbnailOptions { max_edge: 256, min_embedded_edge: 160 }).unwrap();
let t = decode_thumbnail_with(&bytes, &ThumbnailOptions { min_embedded_edge: 160, ..ThumbnailOptions::new(256) }).unwrap();
assert_eq!(t.source, ThumbnailSource::ExifThumbnail);
assert_eq!((t.image.width, t.image.height), (160, 120));
}
+31 -4
View File
@@ -7,16 +7,20 @@
//! `.wasm` within the web size budget (Cloudflare serves at most 25 MiB per file); the export
//! watermark falls back to it there. Native builds embed every font.
use std::fmt::Write as _;
use std::path::PathBuf;
use std::path::{Path, PathBuf};
fn main() {
println!("cargo::rerun-if-changed=build.rs");
println!("cargo::rerun-if-env-changed=CRAFT_FONTS_DIR");
println!("cargo::rerun-if-env-changed=CRAFT_FONTS_REQUIRED");
let mut src = String::from("pub static CRAFT_FONTS: &[CraftFont] = &[\n");
if let Some(dir) = std::env::var_os("CRAFT_FONTS_DIR").map(PathBuf::from) {
let mut embedded = 0;
if let Some(dir) = std::env::var_os("CRAFT_FONTS_DIR").map(PathBuf::from).map(resolve) {
match craft_fonts(&dir) {
Ok(entries) => src.push_str(&entries),
Ok(entries) => {
embedded = entries.lines().count();
src.push_str(&entries);
}
Err(e) if std::env::var_os("CRAFT_FONTS_REQUIRED").is_some() => {
println!("cargo::error=CRAFT_FONTS_DIR={}: {e}", dir.display());
}
@@ -24,14 +28,37 @@ fn main() {
}
}
src.push_str("];\n");
// Without them the app still runs and every language still selectable, so Chinese and Japanese
// just paint boxes — the one failure a user cannot diagnose from what they see.
if embedded == 0 {
println!(
"cargo::warning=no craft-fonts embedded: Chinese and Japanese text will render as empty boxes. \
Clone https://github.com/storytold/craft-fonts next to this repo and rebuild with \
CRAFT_FONTS_DIR=../craft-fonts"
);
}
let out = PathBuf::from(std::env::var_os("OUT_DIR").unwrap_or_default()).join("craft_fonts.rs");
if let Err(e) = std::fs::write(&out, src) {
println!("cargo::error=writing {}: {e}", out.display());
}
}
/// Resolve a relative `CRAFT_FONTS_DIR` against both roots a caller might mean: a build script's
/// cwd is the *package* root (`crates/engine`), while the `../craft-fonts` the README and
/// craft-fonts' docs use is written relative to the *workspace* root and would otherwise resolve
/// to `crates/craft-fonts`.
fn resolve(dir: PathBuf) -> PathBuf {
if dir.is_absolute() {
return dir;
}
let package = PathBuf::from(std::env::var_os("CARGO_MANIFEST_DIR").unwrap_or_default());
let workspace = package.parent().and_then(Path::parent).unwrap_or(package.as_path());
let candidates = [workspace.join(&dir), package.join(&dir)];
candidates.iter().find(|candidate| candidate.is_dir()).cloned().unwrap_or_else(|| candidates[0].clone())
}
/// One `CraftFont { .. }` initialiser per manifest line.
fn craft_fonts(dir: &std::path::Path) -> Result<String, String> {
fn craft_fonts(dir: &Path) -> Result<String, String> {
let manifest = dir.join("fonts/manifest.txt");
println!("cargo::rerun-if-changed={}", manifest.display());
let text = std::fs::read_to_string(&manifest).map_err(|e| format!("{}: {e}", manifest.display()))?;
+76 -11
View File
@@ -1,5 +1,5 @@
//! Estimate the starting look of a raw without a camera colour matrix (Sony ARW, Nikon NEF) from its
//! own JPEG. Colour and luminance are fitted separately; the JPEG supplies correspondences only,
//! Estimate the starting look of a raw without a camera colour matrix (Sony ARW, Nikon NEF, Panasonic
//! RW2) from its own JPEG. Colour and luminance are fitted separately; the JPEG supplies correspondences only,
//! never output pixels or a replacement for RAW editing.
//! A global matrix can't follow the camera's hue-dependent rendering (the best matrix rendered a
//! lime shirt olive that the camera kept lime): a hue/saturation table fitted to the residuals
@@ -31,7 +31,7 @@ pub(crate) const PROFILE_PROXY: usize = 192;
/// as-shot look (`docs/camera-preview-colour.md`). The catalog's `Photo::relative_wb` matches the
/// same formats by file extension.
pub(crate) fn file_local_look(format: RawFormat) -> bool {
matches!(format, RawFormat::Arw | RawFormat::Nef | RawFormat::Nrw)
matches!(format, RawFormat::Arw | RawFormat::Nef | RawFormat::Nrw | RawFormat::Rw2)
}
pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransform) -> Option<CameraLook> {
@@ -63,18 +63,28 @@ fn proxies(raw: &RawImage, bytes: &[u8], transform: &CameraTransform, size: usiz
let jpeg = lightcraft_raw::embedded_preview(bytes)?;
let edge = (2 * size).max(384) as u32;
let decoded = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions { max_size: Some((edge, edge)), max_pixels: 64_000_000 }).ok()?;
let reference = decoded.to_working();
let crop = raw.crop.clipped(raw.active_area.width, raw.active_area.height);
let mut reference = decoded.to_working();
let (a, crop) = (raw.active_area, raw.crop.clipped(raw.active_area.width, raw.active_area.height));
if crop.width == 0 || crop.height == 0 || reference.width == 0 || reference.height == 0 {
return None;
}
let aspect = crop.width as f64 / crop.height as f64;
if (reference.width as f64 / reference.height as f64 / aspect - 1.0).abs() > 0.02 {
return None;
let matches = |w: usize, h: usize, rw: usize, rh: usize| (rw as f64 / rh as f64 / (w as f64 / h as f64) - 1.0).abs() <= 0.02;
if !matches(crop.width, crop.height, reference.width, reference.height) {
// Panasonic previews show the whole active area while the default crop is the in-camera aspect ratio
if !matches(a.width, a.height, reference.width, reference.height) {
return None;
}
let (sx, sy) = (reference.width as f64 / a.width as f64, reference.height as f64 / a.height as f64);
let (x, y) = ((crop.x as f64 * sx).round() as usize, (crop.y as f64 * sy).round() as usize);
let (w, h) = ((crop.width as f64 * sx).round() as usize, (crop.height as f64 * sy).round() as usize);
if w < 16 || h < 16 || x + w > reference.width || y + h > reference.height {
return None;
}
reference = reference.into_crop(x, y, w, h);
}
// Fixed, bounded proxy: the selected look cannot depend on thumbnail/export resolution.
let k = (crop.width.max(crop.height).div_ceil(edge as usize).max(2)).div_ceil(2) * 2;
let sensor = raw.develop_binned(k, 0.99).ok()??;
let sensor = sensor_proxy(raw, k, edge as usize)?;
let mut sensor = fit(&sensor, size, size, Filter::Box);
let reference = fit(&reference, sensor.width, sensor.height, Filter::Box);
let gain = 2f32.powf(transform.baseline_exposure as f32);
@@ -103,6 +113,14 @@ pub(crate) fn fit_profile(pairs: &[([f64; 3], [f64; 3])]) -> Option<(Mat3, Optio
Some((matrix, fit_hue_sat(pairs, &matrix)))
}
fn sensor_proxy(raw: &RawImage, k: usize, edge: usize) -> Option<Rgb32f> {
Some(match raw.develop_binned(k, 0.99).ok()? {
Some(sensor) => sensor,
None if raw.cpp == 3 && raw.cfa.is_none() => fit(&raw.develop(lightcraft_raw::Method::Bilinear).ok()?, edge, edge, Filter::Box),
None => return None,
})
}
/// A fit must cut the held-out squared error to below this share of the fallback's.
const MIN_IMPROVEMENT: f64 = 0.7;
/// ...and stay within this per-channel RMS of the camera JPEG (linear display values). On public
@@ -511,6 +529,35 @@ fn fit_tone(mut pairs: Vec<(f64, f64)>) -> Option<CameraTone> {
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn linear_arw_gets_a_sensor_proxy_without_demosaicing() {
use lightcraft_raw::{BlackLevel, ColorData, OpcodeLists, Orientation, RawData, RawFormat, Rect};
let mut raw = RawImage {
format: RawFormat::Arw,
width: 32,
height: 32,
cpp: 3,
data: RawData::F32([0.2, 0.3, 0.4].repeat(32 * 32)),
cfa: None,
bits: 16,
black: BlackLevel::uniform(0.0),
white: vec![1.0],
active_area: Rect::new(0, 0, 32, 32),
crop: Rect::new(0, 0, 32, 32),
orientation: Orientation::from_exif(1),
color: ColorData::default(),
wb_multipliers: Some([1.0; 3]),
linearized: true,
opcodes: OpcodeLists::default(),
metadata: lightcraft_meta::Metadata::default(),
};
let proxy = sensor_proxy(&raw, 2, 384).unwrap();
assert_eq!((proxy.width, proxy.height), (32, 32));
assert_eq!(proxy.data[0], [0.2, 0.3, 0.4]);
raw.data = RawData::F32(Vec::new());
assert!(sensor_proxy(&raw, 2, 384).is_none());
}
#[test]
fn separates_nonlinear_tone_from_colour_and_keeps_sensor_headroom() {
let known = Mat3([[1.8, -0.4, -0.1], [-0.2, 1.5, -0.1], [-0.05, -0.3, 1.7]]);
@@ -731,8 +778,8 @@ mod tests {
}
#[test]
fn sony_and_nikon_raws_get_a_file_local_look() {
assert!([RawFormat::Arw, RawFormat::Nef, RawFormat::Nrw].into_iter().all(file_local_look));
fn sony_nikon_and_panasonic_raws_get_a_file_local_look() {
assert!([RawFormat::Arw, RawFormat::Nef, RawFormat::Nrw, RawFormat::Rw2].into_iter().all(file_local_look));
assert!(![RawFormat::Dng, RawFormat::Cr2, RawFormat::Raf].into_iter().any(file_local_look));
}
@@ -753,6 +800,24 @@ mod tests {
assert!(info.relative_wb && info.as_shot_temp == 6500.0 && info.as_shot_tint == 0.0);
}
/// A public DC-FZ1000 II RW2 shot at 4:3 on its 3:2 sensor (skipped without the corpus): the default crop is 4:3
/// while the embedded JPEG shows the whole sensor; the look is still fitted, against the matching part of it.
#[test]
fn corpus_rw2_with_an_in_camera_crop_gets_a_camera_look() {
let path = std::env::var_os("LIGHTCRAFT_CORPUS")
.map(std::path::PathBuf::from)
.unwrap_or_else(|| std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus"))
.join("raw/rw2-panasonic-fz1000m2-4x3.rw2");
let Ok(bytes) = std::fs::read(&path) else {
eprintln!("skip: {} absent", path.display());
return;
};
let (img, info) = crate::files::load_bytes(&bytes, 400).unwrap();
assert_eq!((img.width, img.height), (400, 300), "framed in the in-camera aspect ratio");
assert!(info.camera_tone.is_some(), "no camera look fitted");
assert!(info.relative_wb && info.as_shot_temp == 6500.0 && info.as_shot_tint == 0.0);
}
#[test]
fn rejects_monochrome_invalid_and_unrelated_previews() {
let mut sensor = Rgb32f::new(32, 32);
+46 -2
View File
@@ -224,6 +224,25 @@ fn select_by(s: &mut Session, p: &Value) -> Result<Value> {
Ok(json!({"selected": n}))
}
/// Seeds stay exactly representable as a JSON double, so web and agent clients read back what they wrote.
const MAX_SEED: u64 = 1 << 53;
/// `seed` as an integer in `0..2^53`, or `default` when absent or null; anything else is an error.
fn seed_param(p: &Value, cmd: &str, default: u64) -> Result<u64> {
match p.get("seed") {
None | Some(Value::Null) => Ok(default),
Some(v) => v.as_u64().filter(|n| *n < MAX_SEED).ok_or_else(|| bad(cmd, "seed must be an integer from 0 to 2^53 - 1")),
}
}
/// The seed after `prev` at time `now`: a mix of `prev` and the clock text, so repeated reshuffles
/// within one clock tick still differ (barring a 2^-53 collision). Kept below 2^53 so JSON
/// clients that read numbers as doubles (web, MCP agents) get the same seed back.
fn next_seed(prev: u64, now: &str) -> u64 {
let t = now.bytes().fold(0u64, |h, b| lightcraft_catalog::mix64(h ^ u64::from(b)));
(lightcraft_catalog::mix64(prev.wrapping_add(1)) ^ t) & (MAX_SEED - 1)
}
pub fn specs() -> Vec<CommandSpec> {
vec![
// ---- view source / filter / sort
@@ -281,7 +300,7 @@ pub fn specs() -> Vec<CommandSpec> {
"Sort",
["View", "Sort"],
None,
"{key?: captureDate|importDate|editDate|fileName|rating|fileSize, ascending?: bool, group?: auto|none|day|month|year}",
"{key?: captureDate|importDate|editDate|fileName|rating|fileSize|random, ascending?: bool, group?: auto|none|day|month|year, seed?: u64 (the shuffle `random` gives)}",
always,
|s, p| {
let key: SortKey = match p.get("key") {
@@ -292,7 +311,31 @@ pub fn specs() -> Vec<CommandSpec> {
Some(g) => GroupBy::parse(g).ok_or_else(|| bad("library.sort", "group must be auto|none|day|month|year"))?,
None => s.sort.group,
};
s.sort = Sort { key, ascending: bool_or(p, "ascending", s.sort.ascending), group };
let mut seed = seed_param(p, "library.sort", s.sort.seed)?;
let explicit = p.get("seed").is_some_and(|v| !v.is_null());
if key == SortKey::Random && s.sort.key != SortKey::Random && !explicit {
// switching to Random is a fresh shuffle, not whatever seed was left behind
seed = next_seed(seed, &(s.clock)());
}
s.sort = Sort { key, ascending: bool_or(p, "ascending", s.sort.ascending), group, seed };
ok()
}
),
cmd!(
"library.shuffle",
"Reshuffle",
["View", "Sort"],
None,
"{seed?: 0..2^53-1} — sort at random; without `seed` a new shuffle each time",
always,
|s, p| {
let seed = match seed_param(p, "library.shuffle", s.sort.seed)? {
given if p.get("seed").is_some_and(|v| !v.is_null()) => given,
// derived from the previous seed and the session clock: reproducible under a test
// clock, and no RNG needed
prev => next_seed(prev, &(s.clock)()),
};
s.sort = Sort { key: SortKey::Random, seed, ..s.sort };
ok()
}
),
@@ -991,6 +1034,7 @@ pub fn specs() -> Vec<CommandSpec> {
}),
cmd!("library.clearPreviews", "Clear Preview Cache", ["File"], None, "{}", always, |s, _| {
s.media.rendered.clear();
s.media.clear_sources();
ok()
}),
]
+1 -1
View File
@@ -233,7 +233,7 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result<
raw.opcodes.list3.retain(|op| !is_lens_opcode(op));
// Previews and thumbnails bin the mosaic straight to (about) the size they need; only
// larger levels (exports, 1:1) demosaic the whole sensor.
let t0 = std::time::Instant::now();
let t0 = web_time::Instant::now();
let binned = match bin_factor(&raw, max_edge) {
Some(k) => raw.develop_binned(k, HIGHLIGHT_CLIP).map_err(|e| e.to_string())?,
None => None,
+10 -7
View File
@@ -32,8 +32,8 @@ use crate::media::ProbeInfo;
/// File extensions LightCraft imports (lower case).
pub const EXTENSIONS: &[&str] = &[
"jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "nrw", "arw", "raf", "orf", "rw2", "pef", "psd", "jxl", "gif", "bmp",
"heic", "avif",
"jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "nrw", "arw", "raf", "orf", "rw2", "rwl", "raw", "pef", "psd", "jxl",
"gif", "bmp", "heic", "avif",
];
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
@@ -393,7 +393,7 @@ pub struct ScanInput {
probe: Option<crate::media::FileProbe>,
skip: Option<PathBuf>,
/// path → (photo, its summary) for files already in the library.
by_path: HashMap<String, (PhotoId, ImportCandidate)>,
by_path: HashMap<PathBuf, (PhotoId, ImportCandidate)>,
by_hash: HashMap<String, PhotoId>,
cache: HashMap<String, ProbeInfo>,
}
@@ -432,7 +432,7 @@ impl ScanInput {
existing: Some(p.id.0),
..Default::default()
};
by_path.insert(path.clone(), (p.id, c));
by_path.insert(PathBuf::from(path), (p.id, c));
}
if let Some(h) = &p.content_hash {
by_hash.insert(h.clone(), p.id);
@@ -458,7 +458,7 @@ pub fn scan(s: &mut Session, paths: &[String]) -> Vec<ImportCandidate> {
pub fn scan_with(mut input: ScanInput, paths: &[String], progress: &ScanProgress) -> ScanOutput {
use std::sync::atomic::Ordering::Relaxed;
let files = expand(paths, input.skip.as_deref());
let todo: Vec<String> = files.iter().filter(|f| !input.by_path.contains_key(f.as_str())).cloned().collect();
let todo: Vec<String> = files.iter().filter(|f| !input.by_path.contains_key(Path::new(f))).cloned().collect();
progress.total.store(todo.len(), Relaxed);
// probes from a preceding `scan` are reused when the file is unchanged (same size)
let cached: Vec<Option<ProbeInfo>> = todo
@@ -480,7 +480,7 @@ pub fn scan_with(mut input: ScanInput, paths: &[String], progress: &ScanProgress
let mut out = Vec::with_capacity(files.len());
let mut kept = HashMap::new();
for f in files {
if let Some((_, c)) = input.by_path.get(&f) {
if let Some((_, c)) = input.by_path.get(Path::new(&f)) {
let mut c = c.clone();
c.name = Path::new(&f).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| f.clone());
out.push(c);
@@ -993,6 +993,9 @@ impl Session {
let source = self.media.origin_ref(&p.source, level.max_edge());
let key = lightcraft_preview::Hasher128::new().str(&c.path).u64(c.file_size).u64(edge as u64).finish().0 as u64;
let small = crate::media::RenderJob {
request_id: 0,
cache_generation: self.media.rendered.generation(),
source_key: None,
photo: id,
level,
source,
@@ -1009,7 +1012,7 @@ impl Session {
(Some(l), MediaKind::Raw) => Some((c.path.clone(), l.clone(), edge)),
_ => None,
};
Some(crate::media::QuickJob { photo: id, key, cached: Vec::new(), embedded, small: Some(Box::new(small)) })
Some(crate::media::QuickJob { request_id: 0, photo: id, key, cached: Vec::new(), embedded, small: Some(Box::new(small)) })
}
/// Use the system clock for import/edit times (native hosts).
+11 -4
View File
@@ -295,7 +295,13 @@ impl Session {
{
log::error!("library: {e}");
}
let (mut journal, catalog, report) = Journal::open(catalog)?;
let (mut journal, mut catalog, report) = Journal::open(catalog)?;
// A loaded catalog counts revisions from 0, like every other one. Caches (sidebar counts,
// keyword tree, the grid's list…) are keyed on the revision, so give each library loaded
// in this process its own range: otherwise an empty library opened after one that was
// never edited this session would look unchanged and show the old library's numbers.
static LOADS: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
catalog.revision = LOADS.fetch_add(1, std::sync::atomic::Ordering::Relaxed).wrapping_add(1) << 32;
self.catalog = catalog;
self.undo.clear();
self.redo.clear();
@@ -557,13 +563,14 @@ impl Session {
if self.cache_mb == 0 { crate::media::DISK_CACHE_BYTES } else { u64::from(self.cache_mb) << 20 }
}
/// Change the thumbnail disk cache budget (MB, 0 = default): re-attaches the cache, which
/// trims it to the new size. Saved with the library preferences.
/// Change the thumbnail disk cache budget (MB, 0 = default) in place: the cached thumbnails
/// stay (the next write trims to the new size), and so do the textures shown from them.
/// Saved with the library preferences.
pub fn set_cache_mb(&mut self, mb: u32) -> Result<()> {
self.cache_mb = mb;
if let Some(lib) = self.library.as_ref().filter(|l| l.on_disk) {
let dir = lib.thumbs_dir();
self.media.attach_disk_cache(&dir, self.cache_bytes());
self.media.set_disk_cache_bytes(&dir, self.cache_bytes());
}
self.save_prefs()
}
+213 -15
View File
@@ -16,7 +16,8 @@
//! embedded JPEG for an unedited raw, else a thumbnail-level render — while the real render is
//! prepared. Grid thumbnails of unedited raws likewise start from the embedded preview.
use std::sync::Arc;
use std::collections::{HashMap, VecDeque};
use std::sync::{Arc, Weak};
use lightcraft_catalog::{MediaKind, Photo, PhotoId, Source};
use lightcraft_develop::DevelopSettings;
@@ -25,12 +26,51 @@ use lightcraft_preview::{Hash128, Hasher128, Lru, PreviewCache};
use lightcraft_raster::{Histogram, Rgb32f, Rgba8};
use serde::{Deserialize, Serialize};
const SETTINGS_HASH_ENTRIES: usize = 1024;
#[derive(Default)]
struct SettingsHashes {
entries: HashMap<PhotoId, (Weak<DevelopSettings>, u64)>,
order: VecDeque<PhotoId>,
#[cfg(test)]
computations: usize,
}
impl SettingsHashes {
fn get(&mut self, id: PhotoId, settings: &Arc<DevelopSettings>) -> u64 {
if let Some((old, hash)) = self.entries.get(&id)
&& old.as_ptr() == Arc::as_ptr(settings)
{
return *hash;
}
let hash = settings.hash64();
#[cfg(test)]
{
self.computations += 1;
}
if !self.entries.contains_key(&id) {
if self.entries.len() >= SETTINGS_HASH_ENTRIES
&& let Some(oldest) = self.order.pop_front()
{
self.entries.remove(&oldest);
}
self.order.push_back(id);
}
self.entries.insert(id, (Arc::downgrade(settings), hash));
hash
}
}
/// Bump when the pipeline's output changes, to invalidate cached thumbnails.
pub const RENDER_CACHE_VERSION: u64 = 11;
/// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache).
pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512];
pub fn thumb_bucket(long_edge: usize) -> usize {
THUMB_SIZES.iter().copied().find(|s| *s >= long_edge).unwrap_or(512)
}
/// Memory budgets.
const THUMB_SOURCE_BYTES: usize = 384 << 20;
const RENDERED_MEM_BYTES: usize = 128 << 20;
@@ -145,6 +185,7 @@ impl SourceRef {
}
pub struct MediaCache {
settings_hashes: SettingsHashes,
/// Decoded thumbnail-level sources (LRU by bytes).
thumbs: Lru<PhotoId, DecodedSource>,
/// Decoded preview-level sources with their last use ([`lightcraft_preview::next_tick`]).
@@ -175,6 +216,7 @@ impl Default for MediaCache {
fn default() -> Self {
let budget = crate::memory::cache_share(crate::memory::budget());
MediaCache {
settings_hashes: SettingsHashes::default(),
thumbs: Lru::new(THUMB_SOURCE_BYTES.min(budget)),
previews: Vec::new(),
full: None,
@@ -202,13 +244,27 @@ fn source_bytes(img: &Rgb32f) -> usize {
}
impl MediaCache {
/// Keep rendered thumbnails on disk in `dir` as well.
/// Keep rendered thumbnails on disk in `dir` as well. The cache this replaces is retired
/// (its files stay): render jobs still holding it can't write through it any more, even
/// into a directory a later clear emptied (`dir` may be the same one).
pub fn attach_disk_cache(&mut self, dir: &std::path::Path, disk_bytes: u64) {
self.rendered.retire();
self.rendered = Arc::new(PreviewCache::with_disk(rendered_budget(self.budget), dir, disk_bytes));
}
/// Change the disk budget of the cache in `dir`: in place when that cache is attached (its
/// thumbnails are keyed by content, so they and the textures shown from them stay valid,
/// and the same cache object keeps its generation), else attach it.
pub fn set_disk_cache_bytes(&mut self, dir: &std::path::Path, disk_bytes: u64) {
match self.rendered.disk().filter(|d| d.dir() == dir) {
Some(d) => d.set_budget(disk_bytes),
None => self.attach_disk_cache(dir, disk_bytes),
}
}
/// Forget every decoded source (photo ids changed meaning, e.g. another library was opened).
pub fn clear_sources(&mut self) {
self.settings_hashes = SettingsHashes::default();
self.thumbs = Lru::new(THUMB_SOURCE_BYTES.min(self.budget));
self.previews.clear();
self.full = None;
@@ -391,6 +447,11 @@ impl MediaCache {
/// Everything needed to render one photo, detached from the session.
#[derive(Clone)]
pub struct RenderJob {
/// Assigned by a frontend to distinguish late completions, even for an identical render key.
pub request_id: u64,
pub cache_generation: u64,
/// Content identity for validating a decoded source returned after a reload.
pub source_key: Option<Hash128>,
pub photo: PhotoId,
pub level: SourceLevel,
pub source: SourceRef,
@@ -412,6 +473,8 @@ pub struct RenderJob {
}
pub struct RenderResult {
pub request_id: u64,
pub source_key: Option<Hash128>,
pub photo: PhotoId,
pub level: SourceLevel,
pub key: u64,
@@ -469,11 +532,13 @@ impl RenderJob {
pub fn run(self) -> RenderResult {
if let Some((cache, key)) = &self.cache
&& let Some(img) = cache.get(*key)
&& let Some(img) = cache.get_at(self.cache_generation, *key)
{
let image = Arc::unwrap_or_clone(img);
let histogram = Histogram::of_srgb8(&image);
return RenderResult {
request_id: self.request_id,
source_key: self.source_key,
photo: self.photo,
level: self.level,
key: self.key,
@@ -492,14 +557,16 @@ impl RenderJob {
let gpu = self.cache.is_none();
let rendered = develop(src, &info, &self.settings, &self.request, self.stages.as_deref(), gpu);
if let Some((cache, key)) = &self.cache {
cache.put(*key, Arc::new(rendered.image.clone()));
cache.put_at(self.cache_generation, *key, Arc::new(rendered.image.clone()));
}
if let Some((cache, key)) = &self.view_cache
&& self.request.quality == Quality::Full
{
cache.put_deferred(*key, Arc::new(rendered.image.clone()));
cache.put_deferred_at(self.cache_generation, *key, Arc::new(rendered.image.clone()));
}
RenderResult {
request_id: self.request_id,
source_key: self.source_key,
photo: self.photo,
level: self.level,
key: self.key,
@@ -508,7 +575,16 @@ impl RenderJob {
quick: None,
}
}
Err(e) => RenderResult { photo: self.photo, level: self.level, key: self.key, rendered: Err(e), loaded: None, quick: None },
Err(e) => RenderResult {
request_id: self.request_id,
source_key: self.source_key,
photo: self.photo,
level: self.level,
key: self.key,
rendered: Err(e),
loaded: None,
quick: None,
},
}
}
}
@@ -548,6 +624,7 @@ pub enum QuickSource {
/// renders, the embedded preview of an unedited raw, a thumbnail-level render.
#[derive(Clone)]
pub struct QuickJob {
pub request_id: u64,
pub photo: PhotoId,
pub key: u64,
/// Cached renders to try first, in order: (cache, key).
@@ -561,7 +638,17 @@ pub struct QuickJob {
impl QuickJob {
pub fn run(self) -> RenderResult {
let (photo, key) = (self.photo, self.key);
let done = |rendered: Result<Rendered, String>, quick| RenderResult { photo, level: SourceLevel::Thumb, key, rendered, loaded: None, quick };
let request_id = self.request_id;
let done = |rendered: Result<Rendered, String>, quick| RenderResult {
request_id,
source_key: None,
photo,
level: SourceLevel::Thumb,
key,
rendered,
loaded: None,
quick,
};
let rendered = |image: Rgba8| {
let histogram = Histogram::of_srgb8(&image);
Ok(Rendered { image, histogram, deep: None })
@@ -578,7 +665,7 @@ impl QuickJob {
}
if let Some(job) = self.small {
let r = job.run();
return RenderResult { key, quick: Some(QuickSource::Small), ..r };
return RenderResult { request_id, key, quick: Some(QuickSource::Small), ..r };
}
done(Err("no quick preview".into()), None)
}
@@ -617,7 +704,7 @@ impl crate::Session {
/// A grid/filmstrip thumbnail job: the long edge is rounded up to one of [`THUMB_SIZES`]
/// (≥ `long_edge`) and the result comes from / goes to the thumbnail cache (memory + disk).
pub fn thumb_job(&mut self, id: PhotoId, long_edge: usize) -> Option<RenderJob> {
let b = THUMB_SIZES.iter().copied().find(|s| *s >= long_edge).unwrap_or(THUMB_SIZES[THUMB_SIZES.len() - 1]);
let b = thumb_bucket(long_edge);
self.build_job(id, b, b, false, true, Some(b))
}
@@ -638,8 +725,12 @@ impl crate::Session {
// the photo id is part of the key: two photos with the same settings and size must not
// share a result (a view slot showing photo A would otherwise look current for photo B)
// …and so is the file's content: a file changed on disk (Reload) renders afresh
let content = Hasher128::new().str(&content_key(&p)).finish().0 as u64;
let key = settings.hash64()
let content_key = content_key(&p);
let source_key = Hasher128::new().str(&content_key).finish();
let content = source_key.0 as u64;
// Temporary "before" settings must not replace the photo's cached develop hash.
let settings_hash = if before { settings.hash64() } else { self.media.settings_hashes.get(id, &settings) };
let key = settings_hash
^ ((max_w as u64) << 40)
^ ((max_h as u64) << 20)
^ (apply_crop as u64)
@@ -648,8 +739,8 @@ impl crate::Session {
^ content.rotate_left(17);
let cache = thumb_bucket.map(|b| {
let k = Hasher128::new()
.str(&content_key(&p))
.u64(settings.hash64())
.str(&content_key)
.u64(settings_hash)
.u64(b as u64)
.u64(RENDER_CACHE_VERSION)
.u64(crate::camera_profiles::cache_key())
@@ -657,6 +748,9 @@ impl crate::Session {
(self.media.rendered.clone(), k)
});
Some(RenderJob {
request_id: 0,
cache_generation: self.media.rendered.generation(),
source_key: Some(source_key),
photo: id,
level,
source,
@@ -704,6 +798,9 @@ impl crate::Session {
let source = self.media.source_ref(&p, level);
let ck = Self::variant_key(&p, settings, edge);
Some(RenderJob {
request_id: 0,
cache_generation: self.media.rendered.generation(),
source_key: Some(Hasher128::new().str(&content_key(&p)).finish()),
photo: id,
level,
source,
@@ -746,6 +843,9 @@ impl crate::Session {
let source = self.media.source_ref(&p, level);
let ck = Self::variant_key(&p, &settings, edge);
Some(RenderJob {
request_id: 0,
cache_generation: self.media.rendered.generation(),
source_key: Some(Hasher128::new().str(&content_key(&p)).finish()),
photo: id,
level,
source,
@@ -799,7 +899,7 @@ impl crate::Session {
let h = Hasher128::new().str(&content_key(&p)).str("quick").u64(p.develop.hash64()).u64(apply_crop as u64).finish();
let key = h.0 as u64;
let embedded = self.embedded_of(&p).map(|(path, l)| (path, l, max_edge.clamp(1, SourceLevel::Preview.max_edge())));
Some(QuickJob { photo: id, key, cached, embedded, small: apply_crop.then(|| Box::new(small)) })
Some(QuickJob { request_id: 0, photo: id, key, cached, embedded, small: apply_crop.then(|| Box::new(small)) })
}
/// A first grid thumbnail for an unedited raw: the cached render for `job` (the real
@@ -811,6 +911,7 @@ impl crate::Session {
let (path, l) = self.embedded_of(&p)?;
let edge = job.request.max_w.max(job.request.max_h);
Some(QuickJob {
request_id: 0,
photo: job.photo,
key: job.key,
cached: job.cache.clone().into_iter().collect(),
@@ -821,7 +922,9 @@ impl crate::Session {
/// Accept a finished job's loaded source into the cache.
pub fn accept(&mut self, r: &RenderResult) {
if let Some(src) = &r.loaded {
if let Some(src) = &r.loaded
&& r.source_key.is_none_or(|key| self.catalog.photo(r.photo).is_some_and(|p| Hasher128::new().str(&content_key(p)).finish() == key))
{
self.media.insert_source(r.photo, r.level, src.clone());
}
}
@@ -907,6 +1010,101 @@ pub struct ProbeInfo {
pub type FileProbe = Arc<dyn Fn(&str) -> Result<ProbeInfo, String> + Send + Sync>;
#[cfg(test)]
mod thumbnail_hash_tests {
use super::*;
use lightcraft_catalog::Op;
fn session() -> crate::Session {
let mut s = crate::Session::new();
let p = Photo::new(PhotoId(1), Source::File { path: "hash-fixture.jpg".into() }, "fixture.jpg", "JPEG", 6000, 4000, "2026-01-01");
s.catalog.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
s
}
#[test]
fn hashes_match_uncached_keys_and_are_not_recomputed_for_sizes_metadata_or_before() {
let mut s = session();
for edge in [256, 384, 512, 256] {
let p = s.catalog.photo(PhotoId(1)).unwrap().clone();
let b = thumb_bucket(edge);
// Independent original formula: never asks the new hash cache for expected values.
let settings = p.develop.hash64();
let content = Hasher128::new().str(&content_key(&p)).finish().0 as u64;
let expected =
settings ^ ((b as u64) << 40) ^ ((b as u64) << 20) ^ 1 ^ p.id.0.wrapping_mul(0x9e37_79b9_7f4a_7c15) ^ content.rotate_left(17);
let disk = Hasher128::new()
.str(&content_key(&p))
.u64(settings)
.u64(b as u64)
.u64(RENDER_CACHE_VERSION)
.u64(crate::camera_profiles::cache_key())
.finish();
let job = s.thumb_job(p.id, edge).unwrap();
assert_eq!(job.key, expected);
assert_eq!(job.cache.unwrap().1, disk);
}
s.catalog.apply(Op::SetRating { id: PhotoId(1), rating: 4 }).unwrap();
s.thumb_job(PhotoId(1), 256).unwrap();
s.render_job(PhotoId(1), 256, 256, true, true).unwrap();
s.thumb_job(PhotoId(1), 256).unwrap();
assert_eq!(s.media.settings_hashes.computations, 1);
let mut edit = DevelopSettings::default();
edit.light.exposure = 1.0;
let undo = s.catalog.apply(Op::SetDevelop { id: PhotoId(1), settings: Arc::new(edit), label: "edit".into(), edited: None }).unwrap();
s.thumb_job(PhotoId(1), 256).unwrap();
s.thumb_job(PhotoId(1), 384).unwrap();
assert_eq!(s.media.settings_hashes.computations, 2);
s.catalog.apply(undo).unwrap();
s.thumb_job(PhotoId(1), 256).unwrap();
assert_eq!(s.media.settings_hashes.computations, 3);
}
#[test]
fn hash_cache_is_bounded_weak_and_shared_settings_remain_independent() {
let mut hashes = SettingsHashes::default();
let shared = Arc::new(DevelopSettings::default());
for id in 1..=SETTINGS_HASH_ENTRIES as u64 * 3 {
assert_eq!(hashes.get(PhotoId(id), &shared), shared.hash64());
}
assert_eq!(hashes.entries.len(), SETTINGS_HASH_ENTRIES);
assert_eq!(hashes.order.len(), SETTINGS_HASH_ENTRIES);
assert_eq!(Arc::strong_count(&shared), 1);
let mut edit = (*shared).clone();
edit.light.exposure = 1.0;
let edit = Arc::new(edit);
let a = PhotoId(10000);
let b = PhotoId(10001);
let initial = hashes.get(a, &shared);
assert_eq!(hashes.get(b, &shared), initial);
assert_ne!(hashes.get(a, &edit), initial);
assert_eq!(hashes.get(b, &shared), initial);
let mut s = session();
s.thumb_job(PhotoId(1), 256).unwrap();
s.media.clear_sources();
assert!(s.media.settings_hashes.entries.is_empty());
assert!(s.media.settings_hashes.order.is_empty());
}
#[test]
fn source_from_before_content_reload_is_not_accepted() {
let mut s = session();
s.media.file_loader = Some(Arc::new(|_, _| Ok((Rgb32f::new(8, 8), SourceInfo::default()))));
let old = s.thumb_job(PhotoId(1), 256).unwrap().run();
assert!(old.loaded.is_some());
s.catalog
.apply(Op::SetContent {
id: PhotoId(1),
width: 6000,
height: 4000,
file_size: 99,
content_hash: Some("replacement".into()),
preview_only: None,
})
.unwrap();
s.media.forget(PhotoId(1));
s.accept(&old);
assert_eq!(s.media.source_usage().0, 0);
}
}
#[cfg(test)]
mod tests {
use super::*;
+75 -18
View File
@@ -395,7 +395,12 @@ impl MoveFs for RealFs {
/// `from` → `to` differs only in letter case and `to` is that very file (a case-insensitive
/// volume): the rename must go through a temporary name.
fn is_respelling(fs: &dyn MoveFs, from: &Path, to: &Path) -> bool {
from != to && from.to_string_lossy().to_lowercase() == to.to_string_lossy().to_lowercase() && fs.exists(to) && fs.same_file(from, to)
from != to && folded_path(&from.to_string_lossy()) == folded_path(&to.to_string_lossy()) && fs.exists(to) && fs.same_file(from, to)
}
/// Windows accepts both separators; spelling differences must not hide file identity.
fn folded_path(path: &str) -> String {
if cfg!(windows) { path.replace('\\', "/").to_lowercase() } else { path.to_lowercase() }
}
/// Rename `a` to `b` without ever replacing a file; a change of letter case only goes through a
@@ -439,7 +444,7 @@ pub fn move_file(from: &str, to: &str) -> std::result::Result<(), MoveError> {
/// [`move_file`] on the file system `fs`.
pub(crate) fn move_file_with(fs: &dyn MoveFs, from: &str, to: &str) -> std::result::Result<(), MoveError> {
if from == to {
if Path::new(from) == Path::new(to) {
return Ok(());
}
let fail = |message: String| MoveError { message, moved: false };
@@ -590,11 +595,11 @@ fn plan_rename_core(
loop {
let name = candidate(k);
let tp = dir.join(&name).to_string_lossy().to_string();
let key = tp.to_lowercase();
let key = folded_path(&tp);
// this very file, maybe spelled differently (case-insensitive volume)? By
// identity: on a case-sensitive volume `img_1.jpg` may be another photo
let (from, to) = (Path::new(path), Path::new(&tp));
let same = tp == *path || (from != to && key == path.to_lowercase() && exists(to) && same_file(from, to));
let same = from == to || (key == folded_path(path) && exists(to) && same_file(from, to));
// free: not claimed in this batch and not on disk (unless it is this very file).
// A file this batch moves away still counts as taken: simple and safe.
if !taken.contains(&key) && (same || !exists(to)) {
@@ -656,6 +661,7 @@ impl Session {
};
let Some(plan) = plan else { continue };
let source = match &plan.to_path {
Some(t) if matches!(&p.source, Source::File { path } if Path::new(path) == Path::new(t)) => p.source.clone(),
Some(t) => Source::File { path: t.clone() },
None => p.source.clone(),
};
@@ -668,7 +674,7 @@ impl Session {
let moves: Vec<(String, String)> = plans
.iter()
.filter_map(|pl| match (&pl.from_path, &pl.to_path) {
(Some(a), Some(b)) if a != b => Some((a.clone(), b.clone())),
(Some(a), Some(b)) if Path::new(a) != Path::new(b) => Some((a.clone(), b.clone())),
_ => None,
})
.collect();
@@ -732,7 +738,7 @@ impl Session {
Op::Batch { ops } => ops.iter().for_each(|o| rec(s, o, v)),
Op::SetFile { id, source: Source::File { path: to }, .. } => {
if let Some(Source::File { path: from }) = s.catalog.photo(*id).map(|p| &p.source)
&& from != to
&& Path::new(from) != Path::new(to)
&& !v.iter().any(|(f, _)| f == from)
{
v.push((from.clone(), to.clone()));
@@ -768,13 +774,17 @@ mod tests {
fail: RefCell::new(Vec::new()),
}
}
/// The model volume spells paths with `/`; the code under test joins them with the host's separator.
fn norm(p: &Path) -> String {
p.to_string_lossy().replace('\\', "/")
}
fn idx(&self, p: &Path) -> Option<usize> {
let p = p.to_string_lossy();
self.files.borrow().iter().position(|(f, _)| if self.ci { f.to_lowercase() == p.to_lowercase() } else { *f == p })
let p = Self::norm(p);
self.files.borrow().iter().position(|(f, _)| if self.ci { folded_path(f) == folded_path(&p) } else { *f == p })
}
/// The listing: (exact path, contents), sorted.
pub fn listing(&self) -> Vec<(String, String)> {
let mut v: Vec<_> = self.files.borrow().iter().map(|(p, c)| (p.clone(), String::from_utf8_lossy(c).to_string())).collect();
let mut v: Vec<_> = self.files.borrow().iter().map(|(p, c)| (p.replace('\\', "/"), String::from_utf8_lossy(c).to_string())).collect();
v.sort();
v
}
@@ -791,14 +801,14 @@ mod tests {
self.idx(a).is_some() && self.idx(a) == self.idx(b)
}
fn rename_no_replace(&self, a: &Path, b: &Path) -> std::io::Result<()> {
if self.fail.borrow().iter().any(|f| a.to_string_lossy().contains(f.as_str())) {
if self.fail.borrow().iter().any(|f| Self::norm(a).contains(f.as_str())) {
return Err(std::io::Error::other("injected failure"));
}
if self.exists(b) {
return Err(std::io::Error::new(std::io::ErrorKind::AlreadyExists, "exists"));
}
let i = self.idx(a).ok_or_else(|| std::io::Error::new(std::io::ErrorKind::NotFound, "no such file"))?;
self.files.borrow_mut()[i].0 = b.to_string_lossy().to_string();
self.files.borrow_mut()[i].0 = Self::norm(b);
Ok(())
}
fn copy_no_replace(&self, a: &Path, b: &Path) -> std::io::Result<()> {
@@ -807,7 +817,7 @@ mod tests {
}
let i = self.idx(a).ok_or_else(|| std::io::Error::new(std::io::ErrorKind::NotFound, "no such file"))?;
let c = self.files.borrow()[i].1.clone();
self.files.borrow_mut().push((b.to_string_lossy().to_string(), c));
self.files.borrow_mut().push((Self::norm(b), c));
Ok(())
}
fn remove_file(&self, p: &Path) -> std::io::Result<()> {
@@ -816,17 +826,64 @@ mod tests {
Ok(())
}
fn stem_sibling(&self, sidecar: &Path) -> Option<String> {
let stem = sidecar.with_extension("").to_string_lossy().to_string();
let stem = Self::norm(&sidecar.with_extension(""));
self.files.borrow().iter().map(|(f, _)| f.clone()).find(|f| {
let p = Path::new(f);
p.with_extension("").to_string_lossy() == stem && !p.extension().is_some_and(|e| e.eq_ignore_ascii_case("xmp"))
Self::norm(&p.with_extension("")) == stem && !p.extension().is_some_and(|e| e.eq_ignore_ascii_case("xmp"))
})
}
}
fn file_photo(id: u64, path: &str) -> Photo {
let name = Path::new(path).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_default();
Photo::new(PhotoId(id), Source::File { path: path.into() }, &name, "JPG", 1, 1, "2026-01-01T00:00:00")
// the host's separator, so `dir.join(name)` in the planner spells the same path the same way
let path = path.replace('/', std::path::MAIN_SEPARATOR_STR);
Photo::new(PhotoId(id), Source::File { path }, &name, "JPG", 1, 1, "2026-01-01T00:00:00")
}
#[cfg(windows)]
#[test]
fn mixed_separator_noop_rename_preserves_files_catalog_and_undo() {
let fs = FakeFs::new(true, &[("N:/photos/IMG.jpg", "image"), ("N:/photos/IMG.xmp", "edits"), ("N:/photos/other.jpg", "other")]);
// Any attempt to move the unchanged photo (including rollback) must fail.
fs.fail.borrow_mut().push("IMG".into());
let mut s = Session::new();
// spelled with `/` on purpose (not `file_photo`, which uses the host's separator)
for (id, path) in [(1, "N:/photos/IMG.jpg"), (2, "N:/photos/other.jpg")] {
let name = Path::new(path).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_default();
let photo = Photo::new(PhotoId(id), Source::File { path: path.into() }, &name, "JPG", 1, 1, "2026-01-01T00:00:00");
s.catalog.apply(Op::AddPhoto { photo: Box::new(photo) }).unwrap();
}
let before = fs.listing();
let plans = s.plan_rename_with(&fs, &[PhotoId(1)], "{name}", 1);
assert_ne!(plans[0].from_path, plans[0].to_path, "fixture exercises different separator spelling");
assert_eq!(s.apply_rename_with(&fs, &plans).unwrap(), 0);
assert_eq!(fs.listing(), before);
assert_eq!(s.catalog.photo(PhotoId(1)).unwrap().source, Source::File { path: "N:/photos/IMG.jpg".into() });
assert!(s.undo.is_empty());
move_file_with(&fs, "N:/photos/IMG.jpg", "N:/photos\\IMG.jpg").unwrap();
let separator_only = Op::SetFile { id: PhotoId(1), file_name: "IMG.jpg".into(), source: Source::File { path: "N:/photos\\IMG.jpg".into() } };
assert!(s.file_moves(&separator_only).is_empty());
// A batch may combine that no-op with a genuine rename, without touching IMG or its sidecar.
let mut batch = plans;
batch.extend(s.plan_rename_with(&fs, &[PhotoId(2)], "renamed", 1));
assert_eq!(s.apply_rename_with(&fs, &batch).unwrap(), 1);
assert_eq!(
fs.listing(),
vec![
("N:/photos/IMG.jpg".into(), "image".into()),
("N:/photos/IMG.xmp".into(), "edits".into()),
("N:/photos/renamed.jpg".into(), "other".into())
]
);
let undo = &s.undo.last().unwrap().op;
let moves = s.file_moves(undo);
assert_eq!(moves.len(), 1);
assert_eq!(Path::new(&moves[0].1), Path::new("N:/photos/other.jpg"));
move_all(&fs, &moves).unwrap();
s.catalog.apply(undo.clone()).unwrap();
assert_eq!(fs.listing(), before);
}
/// Issue #95: a case-only rename is a real change on a case-insensitive volume and never
@@ -884,7 +941,7 @@ mod tests {
let plans = s.plan_rename_with(&fs, &ids, "Trip-{seq}", 1);
// c can't be renamed (the share went away), and neither can Trip-1 be moved back
fs.fail.borrow_mut().extend(["c.jpg".to_string(), "Trip-1".to_string()]);
let err = s.apply_rename_with(&fs, &plans).unwrap_err().to_string();
let err = s.apply_rename_with(&fs, &plans).unwrap_err().to_string().replace('\\', "/");
assert!(err.contains("rename /p/c.jpg"), "{err}");
assert!(err.contains("/p/Trip-1.jpg could not be moved back"), "{err}");
assert!(err.contains("/p/a.jpg → /p/Trip-1.jpg"), "lists what stayed renamed: {err}");
@@ -899,7 +956,7 @@ mod tests {
);
// the catalog matches the disk
let path = |s: &Session, id: u64| match &s.catalog.photo(PhotoId(id)).unwrap().source {
Source::File { path } => path.clone(),
Source::File { path } => path.replace('\\', "/"),
Source::Demo { .. } => String::new(),
};
assert_eq!(path(&s, 1), "/p/Trip-1.jpg");
@@ -913,7 +970,7 @@ mod tests {
fs.fail.borrow_mut().extend(["d.jpg.xmp".to_string(), "/p/e.jpg".to_string()]);
let e = move_file_with(&fs, "/p/d.jpg", "/p/e.jpg").unwrap_err();
assert!(e.moved, "{e}");
assert!(e.message.contains("it is now /p/e.jpg with its sidecar /p/e.xmp"), "{e}");
assert!(e.message.replace('\\', "/").contains("it is now /p/e.jpg with its sidecar /p/e.xmp"), "{e}");
assert_eq!(fs.listing(), vec![("/p/d.jpg.xmp".into(), "Df".into()), ("/p/e.jpg".into(), "D".into()), ("/p/e.xmp".into(), "Ds".into())]);
// … and when the file does go back, its sidecars follow it
let fs = FakeFs::new(false, &[("/p/d.jpg", "D"), ("/p/d.xmp", "Ds"), ("/p/d.jpg.xmp", "Df")]);
+1 -1
View File
@@ -260,7 +260,7 @@ impl Segmenter {
{
let mirrors = self.mirrors();
if mirrors.is_empty() {
return Err(lightcraft_segment::fetch::DownloadError::NoMirrors.to_string());
return Err(lightcraft_segment::fetch::no_mirrors_message());
}
self.download.start(lightcraft_segment::fetch::SAM3_FILES, mirrors, dir, lightcraft_segment::fetch::Options::default())
}
+7 -3
View File
@@ -181,8 +181,12 @@ fn find_missing_matches_renamed_files_by_content() {
let to = |name: &str| {
found.iter().find(|f| f["from"].as_str().unwrap().ends_with(name)).map(|f| (f["to"].as_str().unwrap().to_string(), f["by"].clone()))
};
assert_eq!(to("a.png"), Some((dir.join("moved/Trip-001.png").to_string_lossy().to_string(), json!("content"))));
assert_eq!(to("b.png"), Some((dir.join("moved/real/Trip-002.png").to_string_lossy().to_string(), json!("content"))), "not the impostor");
assert_eq!(to("a.png"), Some((dir.join("moved").join("Trip-001.png").to_string_lossy().to_string(), json!("content"))));
assert_eq!(
to("b.png"),
Some((dir.join("moved").join("real").join("Trip-002.png").to_string_lossy().to_string(), json!("content"))),
"not the impostor"
);
assert_eq!(r["missing"], 1);
assert_eq!(r["ambiguous"][0]["candidates"].as_array().map(Vec::len), Some(2), "{r}");
let _ = std::fs::remove_dir_all(&dir);
@@ -582,7 +586,7 @@ fn folders_rename_and_move_with_their_photos() {
assert!(s.execute("folder.move", &json!({"path": b.to_string_lossy(), "into": b.join("deeper").to_string_lossy()})).is_err(), "not into itself");
let r = s.execute("folder.move", &json!({"path": b.to_string_lossy(), "into": root.join("Archive").to_string_lossy()})).unwrap();
assert_eq!(r["relinked"], 1);
assert_eq!(path(&s), root.join("Archive/Italy 2026/one.png").to_string_lossy());
assert_eq!(path(&s), root.join("Archive").join("Italy 2026").join("one.png").to_string_lossy());
let _ = std::fs::remove_dir_all(&root);
}
+86 -1
View File
@@ -273,7 +273,7 @@ fn import_review_preview_and_options() {
let mut s = Session::new().with_fs();
s.open_library(&lib, false).unwrap();
// c.png is already in the library
s.execute("library.import", &json!({"paths": [src.join("sub/c.png").to_string_lossy()]})).unwrap();
s.execute("library.import", &json!({"paths": [src.join("sub").join("c.png").to_string_lossy()]})).unwrap();
let n0 = s.catalog.len();
let undo0 = s.undo.len();
let pv = s.execute("library.importPreview", &json!({"paths": [src.to_string_lossy()]})).unwrap();
@@ -466,3 +466,88 @@ fn tracklog_auto_tag() {
assert!(s.execute("photo.autoTagTracklog", &json!({"gpx": GPX, "ids": [ids[0]], "offset": "noon"})).is_err());
let _ = std::fs::remove_dir_all(&dir);
}
// ---- random sort: Given the library, When the user picks Random, Then the grid is a stable shuffle
#[test]
fn random_sort_is_stable_until_reshuffled() {
use lightcraft_catalog::SortKey;
let mut s = Session::with_demo();
let dated = s.visible_cloned();
s.execute("library.sort", &json!({"key": "random", "seed": 11})).unwrap();
assert_eq!((s.sort.key, s.sort.seed), (SortKey::Random, 11));
let a = s.visible_cloned();
assert_ne!(a, dated);
let mut sorted = a.clone();
sorted.sort();
let mut all = dated.clone();
all.sort();
assert_eq!(sorted, all, "a permutation of the same photos");
// an unrelated catalog change must not move anything
let first = a[0];
s.execute("photo.rate", &json!({"rating": 4, "ids": [first.0]})).unwrap();
assert_eq!(s.visible_cloned(), a, "same seed, same grid after an edit");
assert_eq!(s.execute("library.groups", &json!({})).unwrap(), json!([]), "no date headers for a shuffle");
// Reshuffle picks a new seed (and works even with a frozen clock)
s.execute("library.shuffle", &json!({})).unwrap();
let seed1 = s.sort.seed;
assert_eq!(s.sort.key, SortKey::Random);
assert_ne!(seed1, 11);
assert_ne!(s.visible_cloned(), a);
s.execute("library.shuffle", &json!({"seed": null})).unwrap();
assert_ne!(s.sort.seed, seed1, "every reshuffle is new, and a null seed means none given");
assert!(s.sort.seed < 1 << 53, "seeds survive a round trip through a JSON double");
// an explicit seed is reproducible
s.execute("library.shuffle", &json!({"seed": 11})).unwrap();
assert_eq!(s.visible_cloned(), a);
}
#[test]
fn random_sort_rejects_bad_params() {
let mut s = Session::with_demo();
let before = s.sort;
assert!(s.execute("library.sort", &json!({"seed": "x"})).is_err());
assert!(s.execute("library.sort", &json!({"seed": -1})).is_err());
assert!(s.execute("library.shuffle", &json!({"seed": 1.5})).is_err());
assert!(s.execute("library.sort", &json!({"seed": 1u64 << 53})).is_err(), "beyond what a JSON double holds exactly");
assert!(s.execute("library.sort", &json!({"seed": (1u64 << 53) - 1})).is_ok());
assert_eq!(s.sort.key, before.key, "a rejected command leaves the sort alone");
}
#[test]
fn random_sort_applies_to_albums_too() {
let mut s = Session::with_demo();
s.execute("library.sort", &json!({"key": "random", "seed": 5})).unwrap();
let all = s.visible_cloned();
// manual order shows newest-first (descending), i.e. the reverse of this insertion order, so it
// differs from the shuffle unless Random really overrides it
let ids: Vec<u64> = all.iter().take(12).map(|i| i.0).collect();
let album = s.execute("album.create", &json!({"name": "Shuffled"})).unwrap()["id"].as_u64().unwrap();
s.execute("album.addPhotos", &json!({"id": album, "ids": ids})).unwrap();
s.execute("library.source", &json!({"kind": "album", "id": album})).unwrap();
let in_album = s.visible_cloned();
assert_eq!(in_album.len(), 12);
let expect: Vec<_> = all.iter().filter(|i| in_album.contains(i)).copied().collect();
assert_eq!(in_album, expect, "the album shows its photos in the shuffle's order, not manual order");
}
/// Choosing Random from another sort is a fresh shuffle (not the fixed seed-0 order); picking it
/// again, or changing only the direction, keeps the shuffle on screen; an explicit seed wins.
#[test]
fn choosing_random_starts_a_fresh_shuffle() {
let mut s = Session::with_demo();
assert_eq!(s.sort.seed, 0);
s.execute("library.sort", &json!({"key": "random"})).unwrap();
let first = s.sort.seed;
assert_ne!(first, 0, "the first Random is not the fixed seed-0 order");
s.execute("library.sort", &json!({"key": "random"})).unwrap();
s.execute("library.sort", &json!({"ascending": true})).unwrap();
assert_eq!(s.sort.seed, first, "already random: the shuffle stays");
s.execute("library.sort", &json!({"key": "fileName"})).unwrap();
s.execute("library.sort", &json!({"key": "random"})).unwrap();
assert_ne!(s.sort.seed, first, "coming back to Random reshuffles");
s.execute("library.sort", &json!({"key": "rating"})).unwrap();
s.execute("library.sort", &json!({"key": "random", "seed": 42})).unwrap();
assert_eq!(s.sort.seed, 42, "an explicit seed is honoured");
assert!(s.sort.seed < 1 << 53);
}
+14
View File
@@ -242,3 +242,17 @@ fn filter_chips_clear_one_filter_and_total_ignores_the_filter() {
}
assert_eq!(s.filter, Default::default());
}
/// A shuffle is part of the saved view: the same seed (and so the same grid) is back on the next launch.
#[test]
fn random_sort_persists_with_the_view() {
let dir = temp_dir("random-view");
let mut s = open(&dir);
s.execute("library.sort", &json!({"key": "random", "seed": 123_456_789})).unwrap();
s.close_library().unwrap();
drop(s);
let s2 = open(&dir);
assert_eq!(s2.sort.key, lightcraft_catalog::SortKey::Random);
assert_eq!(s2.sort.seed, 123_456_789);
let _ = std::fs::remove_dir_all(&dir);
}
+54
View File
@@ -2,6 +2,7 @@
//! persistence in prefs.json.
use std::path::{Path, PathBuf};
use std::sync::Arc;
use lightcraft_catalog::MediaKind;
use lightcraft_develop::Preset;
@@ -263,3 +264,56 @@ fn curve_presets_save_apply_export_import_and_persist() {
assert_eq!(names, ["Faded Red", "Linear (imported)"]);
let _ = std::fs::remove_dir_all(&lib);
}
#[test]
fn resizing_the_disk_cache_keeps_the_cache_and_its_thumbnails() {
let lib = temp_dir("resize");
let mut s = Session::new().with_fs();
s.open_library(&lib, false).unwrap();
let cache = s.media.rendered.clone();
let key = lightcraft_preview::hash_bytes(b"thumbnail");
let img = lightcraft_raster::Rgba8 { width: 4, height: 4, data: vec![[10, 20, 30, 255]; 16] };
cache.put(key, Arc::new(img));
let generation = cache.generation();
s.execute("library.preferences", &json!({"cacheMb": 300})).unwrap();
// the same cache object (the UI keeps its textures), the same generation, the new budget
assert!(Arc::ptr_eq(&cache, &s.media.rendered));
assert_eq!(s.media.rendered.generation(), generation);
assert_eq!(s.media.rendered.disk().unwrap().budget(), 300 << 20);
assert!(s.media.rendered.get(key).is_some());
// an explicit clear still invalidates
s.execute("library.clearPreviews", &json!({})).unwrap();
assert_ne!(s.media.rendered.generation(), generation);
assert!(s.media.rendered.get(key).is_none());
drop(s);
let _ = std::fs::remove_dir_all(&lib);
}
#[test]
fn reopening_a_library_retires_the_old_cache_so_its_jobs_cannot_write_after_a_clear() {
let lib = temp_dir("reopen-writer");
let mut s = Session::new().with_fs();
s.open_library(&lib, true).unwrap();
let id = s.catalog.photos().next().unwrap().id;
// a thumbnail job still holding the first open's cache object
let job = s.thumb_job(id, 128).unwrap();
let (old, key) = job.cache.clone().unwrap();
// a thumbnail written before the reopen is still valid afterwards
let kept = lightcraft_preview::hash_bytes(b"kept thumbnail");
let img = lightcraft_raster::Rgba8 { width: 4, height: 4, data: vec![[10, 20, 30, 255]; 16] };
old.put(kept, Arc::new(img.clone()));
s.open_library(&lib, false).unwrap();
assert!(!Arc::ptr_eq(&old, &s.media.rendered));
assert!(s.media.rendered.get(kept).is_some(), "a plain reopen keeps the disk cache");
s.execute("library.clearPreviews", &json!({})).unwrap();
// the old job finishes: its render must not land in the cleared directory
let r = job.run();
assert!(r.rendered.is_ok());
old.put(kept, Arc::new(img));
let fresh = lightcraft_preview::PreviewCache::with_disk(1 << 20, &lib.join("thumbs"), 1 << 30);
assert!(fresh.get(key).is_none(), "stale thumbnail written through the replaced cache");
assert!(fresh.get(kept).is_none(), "stale put through the replaced cache");
assert!(s.media.rendered.get(key).is_none());
drop(s);
let _ = std::fs::remove_dir_all(&lib);
}
+17
View File
@@ -0,0 +1,17 @@
[package]
name = "lightcraft-fetch"
version.workspace = true
edition.workspace = true
license = "Apache-2.0"
description = "LightCraft model downloads: mirrors, resume, pinned SHA-256, over HTTPS in pure Rust (L0)"
[lints]
workspace = true
# downloads run natively only (the web build fetches no models)
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
log = { workspace = true }
rustls = { version = "0.23", default-features = false, features = ["std", "tls12"] }
rustls-rustcrypto = "0.0.2-alpha"
webpki-roots = "1"
sha2 = { version = "0.10", default-features = false, features = ["std"] }
File renamed without changes.
+394
View File
@@ -0,0 +1,394 @@
//! Downloading a model's files, only when the user asked for it: from an ordered list of mirrors
//! (the next one when a mirror fails), resuming partial files, with connect and stall timeouts,
//! a size cap, progress and cancellation, and checked (exact size, SHA-256 where pinned) before
//! each file is moved into the model folder. Runs on whatever thread calls [`download`] (the
//! engine uses a background thread; the UI thread never waits for the network). Pure Rust: the
//! HTTPS client is rustls with the RustCrypto provider (see [`http`]).
//!
//! Files are written to `<name>.part` next to their final place and renamed when verified, so a
//! model folder never holds a half-written file under a real name, and an interrupted download
//! resumes from where it stopped. A file that fails its check is deleted, never kept.
//!
//! What to download and where from belongs to the caller: it passes the [`FileSpec`]s and the
//! default mirrors (see [`mirrors`]). `lightcraft-segment` does so for the SAM 3 weights, which
//! are never part of LightCraft (SAM License, see docs/ai-masks.md).
//!
//! Native only: on wasm32 this crate is empty (the web build downloads no models).
#![cfg(not(target_arch = "wasm32"))]
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
pub mod http;
use std::fs::{File, OpenOptions};
use std::io::{Read, Write};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::Duration;
use sha2::{Digest, Sha256};
use http::{HttpError, Limits, Url};
/// One file of the model.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct FileSpec {
pub name: &'static str,
/// Exact size in bytes, when pinned (then the download resumes and is checked against it).
pub size: Option<u64>,
/// SHA-256 (lower-case hex), when pinned.
pub sha256: Option<&'static str>,
/// Largest size accepted when `size` is not pinned.
pub max: u64,
}
/// Most mirrors used (a hostile list can't make a download loop forever).
const MAX_MIRRORS: usize = 16;
/// Redirects followed per request.
const MAX_REDIRECTS: usize = 8;
/// The mirrors to try, in order: `env`'s (base URLs separated by commas, spaces or newlines), then
/// the mirrors file's (one base URL per line, `#` comments), then `defaults`; duplicates and
/// unusable URLs dropped, at most 16.
pub fn mirrors(env: Option<&str>, file: Option<&Path>, defaults: &[&str]) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
let mut add = |s: &str| {
let s = s.trim().trim_end_matches('/');
if !s.is_empty() && Url::parse(s).is_ok() && !out.iter().any(|m| m == s) && out.len() < MAX_MIRRORS {
out.push(s.to_string());
}
};
if let Some(env) = env {
env.split([',', ' ', '\n', '\t']).for_each(&mut add);
}
// a small text file; anything larger isn't a mirror list
if let Some(text) = file.and_then(|f| std::fs::metadata(f).ok().filter(|m| m.len() < 64 * 1024).and_then(|_| std::fs::read_to_string(f).ok())) {
text.lines().map(|l| l.split('#').next().unwrap_or_default()).for_each(&mut add);
}
defaults.iter().for_each(|m| add(m));
out
}
/// Download settings.
#[derive(Clone, Debug)]
pub struct Options {
/// Longest wait to connect to a mirror.
pub connect_timeout: Duration,
/// Longest wait for the next bytes once connected.
pub stall_timeout: Duration,
/// Tries per mirror and file (each resumes where the last one stopped).
pub attempts: u32,
}
impl Default for Options {
fn default() -> Self {
Options { connect_timeout: Duration::from_secs(15), stall_timeout: Duration::from_secs(30), attempts: 3 }
}
}
/// Where a download is.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct Progress {
pub file: String,
/// Bytes of all files on disk so far, and the total expected (pinned sizes + announced ones).
pub done: u64,
pub total: u64,
/// The mirror in use (without query).
pub mirror: String,
}
/// Why a download failed.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum DownloadError {
/// No mirror is configured.
NoMirrors,
Cancelled,
/// A local file problem (folder not writable, disk full…).
Disk(String),
/// Every mirror failed for `file`; the reason for each.
AllMirrorsFailed {
file: String,
errors: Vec<String>,
},
}
impl std::fmt::Display for DownloadError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
DownloadError::NoMirrors => write!(f, "no download location is configured for this model in this build"),
DownloadError::Cancelled => write!(f, "the download was cancelled"),
DownloadError::Disk(e) => write!(f, "{e}"),
DownloadError::AllMirrorsFailed { file, errors } => write!(f, "could not download {file}: {}", errors.join("; ")),
}
}
}
/// One mirror's failure for one file; `Retry` resumes on the same mirror.
enum Fail {
Retry(String),
NextMirror(String),
Cancelled,
Disk(String),
}
/// Download `files` into `dir` from `mirrors` (in order; see [`mirrors`]). Files already there
/// and verified are kept. `progress` is called as bytes arrive (often: throttle in it).
pub fn download(
files: &[FileSpec],
mirrors: &[String],
dir: &Path,
opts: &Options,
cancel: &AtomicBool,
progress: &mut dyn FnMut(&Progress),
) -> Result<(), DownloadError> {
std::fs::create_dir_all(dir).map_err(|e| DownloadError::Disk(format!("can't create {}: {e}", dir.display())))?;
// what's left to fetch
let mut todo = Vec::new();
for f in files {
if !verified(f, &dir.join(f.name))? {
todo.push(f);
}
}
if todo.is_empty() {
return Ok(());
}
if mirrors.is_empty() {
return Err(DownloadError::NoMirrors);
}
let total: u64 = files.iter().map(|f| f.size.unwrap_or(0)).sum();
let mut before: u64 = files.iter().filter(|f| !todo.contains(f)).filter_map(|f| f.size).sum();
for f in todo {
let mut errors = Vec::new();
let mut ok = false;
'mirrors: for m in mirrors {
let url = format!("{m}/{}", f.name);
for _ in 0..opts.attempts.max(1) {
if cancel.load(Ordering::Relaxed) {
return Err(DownloadError::Cancelled);
}
let mut report = |n: u64, announced: u64| {
let mirror = Url::parse(m).map(|u| u.display()).unwrap_or_default();
progress(&Progress {
file: f.name.to_string(),
done: before.saturating_add(n),
total: total.max(before.saturating_add(announced)),
mirror,
})
};
match fetch_file(f, &url, dir, opts, cancel, &mut report) {
Ok(()) => {
ok = true;
break 'mirrors;
}
Err(Fail::Cancelled) => return Err(DownloadError::Cancelled),
Err(Fail::Disk(e)) => return Err(DownloadError::Disk(e)),
Err(Fail::Retry(e)) => {
log::warn!("download of {url}: {e} (retrying)");
errors.push(format!("{}: {e}", short(m)));
}
Err(Fail::NextMirror(e)) => {
log::warn!("download of {url}: {e}");
errors.push(format!("{}: {e}", short(m)));
continue 'mirrors;
}
}
}
}
if !ok {
return Err(DownloadError::AllMirrorsFailed { file: f.name.to_string(), errors });
}
before = before.saturating_add(std::fs::metadata(dir.join(f.name)).map(|m| m.len()).unwrap_or(0));
}
Ok(())
}
/// A mirror for messages (no query string: it may hold a token).
fn short(m: &str) -> String {
Url::parse(m).map(|u| u.display()).unwrap_or_else(|_| "mirror".into())
}
/// Whether `path` is already the right file (exact size and hash when pinned).
fn verified(f: &FileSpec, path: &Path) -> Result<bool, DownloadError> {
let Ok(meta) = std::fs::metadata(path) else { return Ok(false) };
if !meta.is_file() || meta.len() == 0 {
return Ok(false);
}
if f.size.is_some_and(|s| s != meta.len()) || meta.len() > f.max {
return Ok(false);
}
match f.sha256 {
Some(want) => Ok(sha256_file(path).map_err(DownloadError::Disk)? == want),
None => Ok(true),
}
}
fn sha256_file(path: &Path) -> Result<String, String> {
let mut file = File::open(path).map_err(|e| format!("{}: {e}", path.display()))?;
let mut h = Sha256::new();
let mut buf = vec![0u8; 1 << 20];
loop {
let n = file.read(&mut buf).map_err(|e| format!("{}: {e}", path.display()))?;
if n == 0 {
break;
}
h.update(buf.get(..n).unwrap_or_default());
}
Ok(h.finalize().iter().map(|b| format!("{b:02x}")).collect())
}
fn part_path(dir: &Path, name: &str) -> PathBuf {
dir.join(format!("{name}.part"))
}
/// Fetch one file from one mirror into `<name>.part` (resuming it when its size is pinned),
/// check it and move it into place.
fn fetch_file(f: &FileSpec, url: &str, dir: &Path, opts: &Options, cancel: &AtomicBool, report: &mut dyn FnMut(u64, u64)) -> Result<(), Fail> {
let part = part_path(dir, f.name);
let limit = f.size.unwrap_or(f.max);
let disk = |e: std::io::Error| Fail::Disk(format!("{}: {e}", part.display()));
// resume only a pinned file (an unpinned one can't be checked after resuming)
let mut have = match std::fs::metadata(&part) {
Ok(m) if f.size.is_some() && m.len() <= limit => m.len(),
Ok(_) => {
std::fs::remove_file(&part).map_err(disk)?;
0
}
Err(_) => 0,
};
let fetched = f.size != Some(have) || have == 0;
if fetched {
let limits = Limits { connect: opts.connect_timeout, stall: opts.stall_timeout, cancel };
let mut url = Url::parse(url).map_err(|e| Fail::NextMirror(e.to_string()))?;
let origin = url.host.clone();
let mut redirects = 0;
let mut resp = loop {
let mut headers = vec![];
if have > 0 {
headers.push(("Range", format!("bytes={have}-")));
}
// a token for the configured host only (never sent on to a redirect's host)
if let Ok(t) = std::env::var("LIGHTCRAFT_SAM3_TOKEN")
&& !t.trim().is_empty()
&& url.tls
&& url.host == origin
{
headers.push(("Authorization", format!("Bearer {}", t.trim())));
}
let r = http::get(&url, &headers, &limits).map_err(http_fail)?;
match r.status {
301 | 302 | 303 | 307 | 308 => {
redirects += 1;
if redirects > MAX_REDIRECTS {
return Err(Fail::NextMirror("too many redirects".into()));
}
let loc = r.header("location").ok_or_else(|| Fail::NextMirror("redirect without a location".into()))?;
url = url.join(loc).map_err(|e| Fail::NextMirror(e.to_string()))?;
}
_ => break r,
}
};
let mut file = match resp.status {
200 => {
// the whole file (from the start, also when a range was asked for)
if resp.content_length().is_some_and(|n| n > limit || f.size.is_some_and(|s| s != n)) {
return Err(Fail::NextMirror(format!("the server's file has the wrong size ({:?} bytes)", resp.content_length())));
}
have = 0;
File::create(&part).map_err(disk)?
}
206 if have > 0 => {
let range = resp.header("content-range").unwrap_or_default();
let (start, total) = parse_content_range(range).ok_or_else(|| Fail::NextMirror(format!("bad Content-Range `{range}`")))?;
if start != have || total.is_some_and(|t| Some(t) != f.size) {
return Err(Fail::NextMirror(format!("the server resumed at the wrong place ({range})")));
}
OpenOptions::new().append(true).open(&part).map_err(disk)?
}
416 => {
// the partial file doesn't fit what the server has: start over
std::fs::remove_file(&part).map_err(disk)?;
return Err(Fail::Retry("the partial download didn't match; starting over".into()));
}
401 | 403 => return Err(Fail::NextMirror(format!("access denied (HTTP {})", resp.status))),
404 | 410 => return Err(Fail::NextMirror(format!("not found (HTTP {})", resp.status))),
s if s >= 500 => return Err(Fail::Retry(format!("server error (HTTP {s})"))),
s => return Err(Fail::NextMirror(format!("HTTP {s}"))),
};
let announced = f.size.unwrap_or_else(|| resp.content_length().unwrap_or(0));
report(have, announced);
let mut buf = vec![0u8; 256 * 1024];
loop {
let n = match resp.read(&mut buf, &limits) {
Ok(n) => n,
Err(e) => {
let _ = file.flush();
if f.size.is_none() {
drop(file);
let _ = std::fs::remove_file(&part);
}
return Err(http_fail(e));
}
};
if n == 0 {
break;
}
if have.saturating_add(n as u64) > limit {
drop(file);
let _ = std::fs::remove_file(&part);
return Err(Fail::NextMirror(format!("the server sent more than the expected {limit} bytes")));
}
file.write_all(buf.get(..n).unwrap_or_default()).map_err(disk)?;
have = have.saturating_add(n as u64);
report(have, announced);
}
file.sync_all().map_err(disk)?;
}
// checks: exact size, hash
let len = std::fs::metadata(&part).map_err(disk)?.len();
if f.size.is_some_and(|s| s != len) {
return Err(Fail::Retry(format!("incomplete ({len} of {} bytes)", f.size.unwrap_or(0))));
}
if len == 0 {
let _ = std::fs::remove_file(&part);
return Err(Fail::NextMirror("the server sent an empty file".into()));
}
if let Some(want) = f.sha256 {
let got = sha256_file(&part).map_err(Fail::Disk)?;
if got != want {
std::fs::remove_file(&part).map_err(disk)?;
let why = format!("the file is damaged or not the official one (SHA-256 {got})");
// a damaged partial file from an earlier run isn't this mirror's fault
return Err(if fetched { Fail::NextMirror(why) } else { Fail::Retry(why) });
}
}
std::fs::rename(&part, dir.join(f.name)).map_err(disk)?;
Ok(())
}
fn http_fail(e: HttpError) -> Fail {
match e {
HttpError::Cancelled => Fail::Cancelled,
HttpError::Stalled | HttpError::Io(_) | HttpError::Protocol(_) => Fail::Retry(e.to_string()),
HttpError::BadUrl(_) | HttpError::Connect(_) | HttpError::Tls(_) => Fail::NextMirror(e.to_string()),
}
}
/// `bytes a-b/total` → (a, total) (`*` total → None).
fn parse_content_range(v: &str) -> Option<(u64, Option<u64>)> {
let v = v.trim().strip_prefix("bytes")?.trim();
let (range, total) = v.split_once('/')?;
let (a, b) = range.split_once('-')?;
let (a, b) = (a.trim().parse::<u64>().ok()?, b.trim().parse::<u64>().ok()?);
if b < a {
return None;
}
let total = match total.trim() {
"*" => None,
t => Some(t.parse::<u64>().ok()?),
};
Some((a, total))
}
#[cfg(test)]
mod tests;
@@ -294,13 +294,15 @@ fn no_mirrors_is_a_clear_error_and_mirror_lists_are_parsed() {
let dir = tmp("nomirror");
let e = run(&[pinned()], &[], &dir, &AtomicBool::new(false)).0.unwrap_err();
assert_eq!(e, DownloadError::NoMirrors);
assert!(e.to_string().contains(MIRRORS_ENV));
assert!(e.to_string().contains("no download location"));
let file = std::env::temp_dir().join(format!("lc-sam3-mirrors-{}.txt", std::process::id()));
std::fs::write(&file, "# mine\nhttps://b.example/m/ # second\n\nnot a url\nhttps://a.example/x\n").unwrap();
let m = mirrors(Some("https://a.example/x/, ftp://no"), Some(&file));
assert_eq!(m[..2], ["https://a.example/x".to_string(), "https://b.example/m".to_string()]);
assert_eq!(m.len(), 2 + DEFAULT_MIRRORS.len());
assert_eq!(mirrors(None, Some(Path::new("/nonexistent/mirrors.txt"))).len(), DEFAULT_MIRRORS.len());
let defaults = ["https://c.example/d", "https://a.example/x"];
let m = mirrors(Some("https://a.example/x/, ftp://no"), Some(&file), &defaults);
// the environment's, then the file's, then the defaults; a repeated one is kept once
assert_eq!(m, ["https://a.example/x", "https://b.example/m", "https://c.example/d"]);
assert_eq!(mirrors(None, Some(Path::new("/nonexistent/mirrors.txt")), &defaults).len(), 2);
assert!(mirrors(None, None, &[]).is_empty());
let _ = std::fs::remove_file(&file);
let _ = std::fs::remove_dir_all(&dir);
}
+9 -4
View File
@@ -2,7 +2,7 @@
use serde::{Deserialize, Serialize};
use crate::{Affine, Point};
use crate::{Affine, Point, Real};
/// Row-major 3×3 matrix mapping homogeneous points: p' = H·p.
#[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize)]
@@ -23,10 +23,15 @@ impl Homography {
}
pub fn apply(&self, p: Point) -> Point {
let (x, y) = self.apply_real(p.x, p.y);
Point::new(x, y)
}
/// [`Homography::apply`] for any [`Real`] (e.g. [`Interval`](crate::Interval) bounds over many points).
pub fn apply_real<T: Real>(&self, x: T, y: T) -> (T, T) {
let m = &self.0;
let w = m[6] * p.x + m[7] * p.y + m[8];
let w = if w.abs() < 1e-300 { 1e-300 } else { w };
Point::new((m[0] * p.x + m[1] * p.y + m[2]) / w, (m[3] * p.x + m[4] * p.y + m[5]) / w)
let w = (x * m[6] + y * m[7] + m[8]).clamp_tiny(1e-300);
((x * m[0] + y * m[1] + m[2]) / w, (x * m[3] + y * m[4] + m[5]) / w)
}
pub fn mul(&self, o: &Homography) -> Homography {
+8 -1
View File
@@ -7,9 +7,11 @@
mod crop;
mod homography;
mod real;
pub use crop::{CropGeometry, crop_fit_angle, max_inscribed_scale};
pub use homography::Homography;
pub use real::{Interval, Real};
use serde::{Deserialize, Serialize};
@@ -213,8 +215,13 @@ impl Affine {
Affine::translate(p.to_vec()) * Affine::rotate(a) * Affine::translate(-p.to_vec())
}
pub fn apply(&self, p: Point) -> Point {
let (x, y) = self.apply_real(p.x, p.y);
Point::new(x, y)
}
/// [`Affine::apply`] for any [`Real`] (e.g. [`Interval`] bounds over many points).
pub fn apply_real<T: Real>(&self, x: T, y: T) -> (T, T) {
let [a, b, c, d, e, f] = self.0;
Point::new(a * p.x + c * p.y + e, b * p.x + d * p.y + f)
(x * a + y * c + e, x * b + y * d + f)
}
pub fn apply_vec(&self, v: Vec2) -> Vec2 {
let [a, b, c, d, ..] = self.0;
+229
View File
@@ -0,0 +1,229 @@
//! Numbers the coordinate formulas are generic over: `f64` (the reference evaluation) and [`Interval`]
//! (bounds on the `f64` results over a whole range of inputs, e.g. a block of output pixels).
//!
//! Formulas written once over [`Real`] serve both. For `f64` every operator is the plain IEEE operation, so a
//! generic formula evaluated with `f64` is bit-identical to the same formula written with `f64` directly, as long
//! as it keeps the operation order (`a * k` instead of `k * a` is fine: IEEE `+` and `×` are commutative).
use std::ops::{Add, Div, Mul, Sub};
/// A scalar the geometry formulas can be evaluated with. Mixed operations take a parameter (`f64`) on the right.
pub trait Real:
Copy
+ Add<Output = Self>
+ Sub<Output = Self>
+ Mul<Output = Self>
+ Div<Output = Self>
+ Add<f64, Output = Self>
+ Sub<f64, Output = Self>
+ Mul<f64, Output = Self>
+ Div<f64, Output = Self>
{
/// `if |v| < tiny { tiny } else { v }`: the guard of a denominator against zero ([`crate::Homography::apply`]).
fn clamp_tiny(self, tiny: f64) -> Self;
}
impl Real for f64 {
fn clamp_tiny(self, tiny: f64) -> f64 {
if self.abs() < tiny { tiny } else { self }
}
}
/// A closed interval `[lo, hi]` with outward rounding, or *undecided* (both NaN).
///
/// Invariant: if the `f64` operands of an operation lie in the operand intervals, the `f64` result (rounded to
/// nearest) lies in the result interval. Each bound is computed in `f64` and then moved one ulp outward
/// (`next_down` / `next_up`); since rounding to nearest is monotonic and errs by less than one ulp, the moved
/// bound is on the far side of every rounded result. By induction a formula evaluated with intervals bounds the
/// *`f64` evaluation* of the same formula (not only the exact real value) for every input in the input intervals.
///
/// A result that can't be bounded becomes undecided and stays so: division by an interval that contains zero,
/// [`Real::clamp_tiny`] when some value could be clamped, and any infinite or NaN bound. Undecided intervals fail
/// every comparison of their bounds, so callers must test the bounds with conditions that are false for NaN.
#[derive(Clone, Copy, Debug)]
pub struct Interval {
lo: f64,
hi: f64,
}
impl Interval {
const UNDECIDED: Interval = Interval { lo: f64::NAN, hi: f64::NAN };
/// `[lo, hi]` exactly (the bounds are `f64` values already); undecided unless finite with `lo <= hi`.
pub fn new(lo: f64, hi: f64) -> Interval {
if lo.is_finite() && hi.is_finite() && lo <= hi { Interval { lo, hi } } else { Interval::UNDECIDED }
}
/// The lower bound (NaN if undecided).
pub fn lo(self) -> f64 {
self.lo
}
/// The upper bound (NaN if undecided).
pub fn hi(self) -> f64 {
self.hi
}
/// Whether the bounds are usable (not undecided).
pub fn is_decided(self) -> bool {
self.lo <= self.hi
}
/// Bounds computed in round-to-nearest, moved one ulp outward.
fn outward(lo: f64, hi: f64) -> Interval {
Interval::new(lo.next_down(), hi.next_up())
}
/// The hull of four products or quotients, moved outward. NaN in any of them makes it undecided.
fn hull(v: [f64; 4]) -> Interval {
if v.iter().any(|v| !v.is_finite()) {
return Interval::UNDECIDED;
}
let lo = v.iter().copied().fold(f64::INFINITY, f64::min);
let hi = v.iter().copied().fold(f64::NEG_INFINITY, f64::max);
Interval::outward(lo, hi)
}
}
impl From<f64> for Interval {
fn from(v: f64) -> Interval {
Interval::new(v, v)
}
}
impl Add for Interval {
type Output = Interval;
fn add(self, o: Interval) -> Interval {
Interval::outward(self.lo + o.lo, self.hi + o.hi)
}
}
impl Sub for Interval {
type Output = Interval;
fn sub(self, o: Interval) -> Interval {
Interval::outward(self.lo - o.hi, self.hi - o.lo)
}
}
impl Mul for Interval {
type Output = Interval;
fn mul(self, o: Interval) -> Interval {
Interval::hull([self.lo * o.lo, self.lo * o.hi, self.hi * o.lo, self.hi * o.hi])
}
}
impl Div for Interval {
type Output = Interval;
/// Undecided when the divisor may be zero (or is undecided).
fn div(self, o: Interval) -> Interval {
if !(o.lo > 0.0 || o.hi < 0.0) {
return Interval::UNDECIDED;
}
Interval::hull([self.lo / o.lo, self.lo / o.hi, self.hi / o.lo, self.hi / o.hi])
}
}
impl Add<f64> for Interval {
type Output = Interval;
fn add(self, k: f64) -> Interval {
Interval::outward(self.lo + k, self.hi + k)
}
}
impl Sub<f64> for Interval {
type Output = Interval;
fn sub(self, k: f64) -> Interval {
Interval::outward(self.lo - k, self.hi - k)
}
}
impl Mul<f64> for Interval {
type Output = Interval;
fn mul(self, k: f64) -> Interval {
let (a, b) = (self.lo * k, self.hi * k);
if k >= 0.0 { Interval::outward(a, b) } else { Interval::outward(b, a) }
}
}
impl Div<f64> for Interval {
type Output = Interval;
/// Undecided when `k` is zero (or NaN).
fn div(self, k: f64) -> Interval {
let (a, b) = (self.lo / k, self.hi / k);
if k > 0.0 {
Interval::outward(a, b)
} else if k < 0.0 {
Interval::outward(b, a)
} else {
Interval::UNDECIDED
}
}
}
impl Real for Interval {
/// Unchanged when no value in the interval would be clamped, else undecided.
fn clamp_tiny(self, tiny: f64) -> Interval {
if self.lo >= tiny || self.hi <= -tiny { self } else { Interval::UNDECIDED }
}
}
#[cfg(test)]
mod tests {
use super::*;
use proptest::prelude::*;
fn contains(i: Interval, v: f64) -> bool {
i.lo <= v && v <= i.hi
}
/// A value inside `[lo, hi]` at fraction `t`.
fn pick(i: Interval, t: f64) -> f64 {
(i.lo + (i.hi - i.lo) * t).clamp(i.lo, i.hi)
}
proptest! {
/// Every `f64` result of `a ∘ b` with `a ∈ A`, `b ∈ B` lies in `A ∘ B` (or that is undecided).
#[test]
fn operations_enclose_the_f64_results(
a0 in -1e6f64..1e6, aw in 0.0f64..1e3, b0 in -1e3f64..1e3, bw in 0.0f64..10.0,
ta in 0.0f64..=1.0, tb in 0.0f64..=1.0, k in -1e3f64..1e3,
) {
let (ia, ib) = (Interval::new(a0, a0 + aw), Interval::new(b0, b0 + bw));
let (a, b) = (pick(ia, ta), pick(ib, tb));
prop_assert!(contains(ia + ib, a + b));
prop_assert!(contains(ia - ib, a - b));
prop_assert!(contains(ia * ib, a * b));
let q = ia / ib;
prop_assert!(!q.is_decided() || contains(q, a / b));
prop_assert!(contains(ia + k, a + k));
prop_assert!(contains(ia - k, a - k));
prop_assert!(contains(ia * k, a * k));
prop_assert!(k == 0.0 || contains(ia / k, a / k));
let c = ib.clamp_tiny(1.0);
prop_assert!(!c.is_decided() || contains(c, b.clamp_tiny(1.0)));
}
}
#[test]
fn unbounded_results_are_undecided() {
let i = Interval::new(-1.0, 2.0);
assert!(!(i / Interval::new(-1e-9, 1.0)).is_decided());
assert!(!(i / Interval::new(0.0, 1.0)).is_decided());
assert!((i / Interval::new(1e-9, 1.0)).is_decided());
assert!(!(i / 0.0).is_decided());
assert!(!(Interval::new(f64::MAX, f64::MAX) * 2.0).is_decided());
assert!(!Interval::new(2.0, 1.0).is_decided());
assert!(!Interval::new(f64::NAN, 1.0).is_decided());
// undecided propagates through everything
let u = Interval::UNDECIDED;
for r in [u + i, i - u, u * i, i / u, u + 1.0, u * 0.0, u.clamp_tiny(1e-300)] {
assert!(!r.is_decided());
assert!(r.lo.is_nan() && r.hi.is_nan(), "undecided bounds are NaN, which fails every comparison");
}
// the denominator guard: unchanged only when no value is clamped
assert!(Interval::new(1e-300, 1.0).clamp_tiny(1e-300).is_decided());
assert!(Interval::new(-1.0, -1e-300).clamp_tiny(1e-300).is_decided());
assert!(!Interval::new(-1.0, 1.0).clamp_tiny(1e-300).is_decided());
assert!(!Interval::new(5e-301, 1.0).clamp_tiny(1e-300).is_decided());
}
}
+1 -1
View File
@@ -62,7 +62,7 @@ const MODULES: &[Module] = &[
},
Module {
src: include_str!("wgsl/geom.wgsl"),
bindings: &[("src", false, "f32"), ("dst", true, "f32")],
bindings: &[("src", false, "f32"), ("dst", true, "f32"), ("coverage", false, "u32")],
entries: &["orient", "sample_affine", "sample_warp"],
},
];
+109 -3
View File
@@ -410,6 +410,42 @@ fn warp_params(
p
}
/// Output rows per block of [`coverage_mask`]; a block is one mask word (32 px) wide. 16 rows measured
/// ~20 % faster than 8 at 6000 × 4000 (fewer interval evaluations, few more edge pixels).
const COVER_ROWS: usize = 16;
/// The reference framing decision ([`Warp::covers`](lightcraft_pipeline::optics::Warp::covers)) for every
/// output pixel, one bit per pixel, rows padded to 32-bit words. Blocks of 32 × [`COVER_ROWS`] pixels whose
/// interval bounds place them clearly inside or outside the image are filled at once
/// ([`Warp::block_coverage`](lightcraft_pipeline::optics::Warp::block_coverage), the same formulas evaluated
/// with outward-rounded intervals); only blocks near the image edge evaluate each pixel. Same bits as
/// evaluating every pixel.
fn coverage_mask(wp: &lightcraft_pipeline::optics::Warp, o2t: &lightcraft_geom::Affine, w: usize, h: usize) -> Vec<u32> {
let words = w.div_ceil(32);
let mut coverage = vec![0u32; words * h];
if words == 0 {
return coverage;
}
lightcraft_raster::par_rows(&mut coverage, words * COVER_ROWS, |band, rows| {
let y0 = band * COVER_ROWS;
let y1 = y0 + rows.len() / words;
for word in 0..words {
let (x0, x1) = (word * 32, (word * 32 + 32).min(w));
let full = if x1 - x0 == 32 { u32::MAX } else { (1u32 << (x1 - x0)) - 1 };
let block = wp.block_coverage(o2t, x0, x1, y0, y1);
for (y, row) in (y0..y1).zip(rows.chunks_exact_mut(words)) {
let Some(bits) = row.get_mut(word) else { continue };
*bits = match block {
Some(true) => full,
Some(false) => 0,
None => (x0..x1).filter(|&x| wp.covers(o2t, x, y)).fold(0, |b, x| b | 1 << (x - x0)),
};
}
}
});
coverage
}
/// Resample the source into the output frame (`Frame::sample`). May return the source buffer
/// itself when the frame is the identity.
fn sample(cx: &mut Cx<'_>, src: &Arc<Rgb32f>, src_buf: Arc<Buf>, plan: &Plan<'_>) -> Arc<Buf> {
@@ -422,7 +458,7 @@ fn sample(cx: &mut Cx<'_>, src: &Arc<Rgb32f>, src_buf: Arc<Buf>, plan: &Plan<'_>
let out = cx.gpu.buffer(sp.ow * sp.oh * 3);
let mut p = vec![src.width as u32, src.height as u32, sp.ow as u32, sp.oh as u32];
p.extend(orient_map(fr.orient, src.width, src.height).map(|v| v as u32));
cx.run("orient", &p, &[Some(&src_buf), Some(&out)], groups2(sp.ow, sp.oh, [16, 16]));
cx.run("orient", &p, &[Some(&src_buf), Some(&out), None], groups2(sp.ow, sp.oh, [16, 16]));
Arc::new(out)
};
let (base, (bw, bh)) = match sp.prefilter {
@@ -433,7 +469,7 @@ fn sample(cx: &mut Cx<'_>, src: &Arc<Rgb32f>, src_buf: Arc<Buf>, plan: &Plan<'_>
let out = cx.gpu.buffer(w * h * 3);
let mut p = vec![bw as u32, bh as u32, w as u32, h as u32];
p.extend(affine_bits(xf));
cx.run("sample_affine", &p, &[Some(&base), Some(&out)], groups2(w, h, [16, 16]));
cx.run("sample_affine", &p, &[Some(&base), Some(&out), None], groups2(w, h, [16, 16]));
Arc::new(out)
};
match (&sp.mode, fr.warp.as_ref()) {
@@ -444,7 +480,10 @@ fn sample(cx: &mut Cx<'_>, src: &Arc<Rgb32f>, src_buf: Arc<Buf>, plan: &Plan<'_>
(SampleMode::Warp(o2t), Some(wp)) => {
let out = cx.gpu.buffer(w * h * 3);
let p = warp_params(wp, (bw, bh), (w, h), o2t, sp.sx, sp.sy);
cx.run("sample_warp", &p, &[Some(&base), Some(&out)], groups2(w, h, [16, 16]));
// f32 rounding can cross a source edge. Keep the reference f64
// framing decision in a bit mask; color sampling stays on the GPU.
let coverage = cx.gpu.upload(&coverage_mask(wp, o2t, w, h));
cx.run("sample_warp", &p, &[Some(&base), Some(&out), Some(&coverage)], groups2(w, h, [16, 16]));
Arc::new(out)
}
}
@@ -950,6 +989,73 @@ fn masks(cx: &mut Cx<'_>, lin: &Buf, prep: &Prep, plan: &Plan<'_>, host: &mut Ho
mod tests {
use super::*;
#[test]
fn coverage_mask_matches_per_pixel_decision() {
use lightcraft_develop::{EmbeddedLens, EmbeddedWarp};
let lens = EmbeddedLens {
warp: Some(EmbeddedWarp {
planes: [[1.0, -0.03, 0.01, 0.0, 0.001, -0.002]; 3],
center: lightcraft_geom::Point::new(0.52, 0.48),
radius: 0.6,
}),
vignette: None,
};
let mut s = DevelopSettings::default();
s.optics.lens_profile = true;
s.optics.distortion = -40.0;
s.geometry.horizontal = -15.0;
s.geometry.rotate = 3.0;
for orientation in [Orientation::Normal, Orientation::Rotate270] {
s.orientation = orientation;
let f = lightcraft_pipeline::geometry::Frame::with_lens(900, 600, &s, true, Some(&lens));
let wp = f.warp.as_ref().expect("warp");
// widths that do and don't fill the last word; heights that do and don't fill the last band
for (w, h) in [f.fit(700, 700), (333, 221), (64, 16), (1, 1)] {
let o2t = f.out_to_oriented(w, h);
let mask = coverage_mask(wp, &o2t, w, h);
let words = w.div_ceil(32);
assert_eq!(mask.len(), words * h);
for y in 0..h {
for x in 0..w {
let bit = mask[y * words + x / 32] >> (x % 32) & 1 == 1;
assert_eq!(bit, wp.covers(&o2t, x, y), "{orientation:?} {w}x{h}: pixel {x},{y}");
}
let pad = mask[y * words + words - 1] >> (w % 32);
assert!(w.is_multiple_of(32) || pad == 0, "padding bits set");
}
}
}
}
/// The perspective horizon in view: its denominator is exactly 0 on output row 75 (clamped to 1e-300) and
/// changes sign there, so the blocks around it are undecided (clamp, and division by a range that may hold
/// zero) and are decided per pixel, while blocks clear of it are filled at once.
#[test]
fn coverage_mask_with_the_horizon_in_view() {
use lightcraft_geom::{Affine, Homography};
let (w, h) = (900, 600);
let mut wp = lightcraft_pipeline::optics::Warp::identity(w as f64, h as f64);
// centred coordinates: v = (y − 300) / 450, denominator 2·v + m8, exactly 0 at y = 75.5
let m8 = -((75.5 - 300.0) / 450.0 * 2.0);
wp.persp_inv = Homography([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 2.0, m8]);
wp.persp = wp.persp_inv.inverse().expect("invertible");
let o2t = Affine::IDENTITY;
assert_eq!(wp.block_coverage(&o2t, 0, 32, 64, 80), None, "the block holding the horizon");
assert_eq!(wp.block_coverage(&o2t, 0, 32, 0, 16), Some(false), "beyond the horizon");
assert_eq!(wp.block_coverage(&o2t, 448, 480, 288, 304), Some(true), "the centre");
let mask = coverage_mask(&wp, &o2t, w, h);
let words = w.div_ceil(32);
let mut covered = 0;
for y in 0..h {
for x in 0..w {
let bit = mask[y * words + x / 32] >> (x % 32) & 1 == 1;
assert_eq!(bit, wp.covers(&o2t, x, y), "pixel {x},{y}");
covered += bit as usize;
}
}
assert!(covered > 0 && covered < w * h, "{covered} px covered");
}
/// Kernel timings on a 24 MP plane: `cargo test --release -p lightcraft-gpu -- --ignored --nocapture`.
#[test]
#[ignore]
+3 -1
View File
@@ -177,7 +177,9 @@ fn sample_warp(@builtin(global_invocation_id) g: vec3<u32>) {
let s = corrected_to_source(c, 1u);
let sx = pf(10u);
let sy = pf(11u);
if (!(s.x >= -0.5 && s.y >= -0.5 && s.x <= pf(12u) + 0.5 && s.y <= pf(13u) + 0.5)) {
// CPU reference coverage avoids an f32 rounding discontinuity at the edge.
let word = g.y * ((w + 31u) / 32u) + g.x / 32u;
if ((coverage[word] & (1u << (g.x % 32u))) == 0u) {
put(g.y * w + g.x, vec3<f32>(BLANK_R, BLANK_G, BLANK_B));
return;
}
+14 -3
View File
@@ -430,7 +430,6 @@ fn geometry_variants() {
if !gpu() {
return;
}
use lightcraft_develop::{EmbeddedLens, EmbeddedVignette, EmbeddedWarp};
let src = scene(2, 900, 600);
let raw = SourceInfo { raw: true, ..Default::default() };
use Orientation::*;
@@ -445,7 +444,19 @@ fn geometry_variants() {
check(&format!("{o:?} crop={crop}"), &src, &raw, &s, &req);
}
}
// embedded DNG lens corrections (per-plane warp + vignette) with manual CA
}
#[test]
fn embedded_lens_perspective_edges_match() {
if !gpu() {
return;
}
use lightcraft_develop::{EmbeddedLens, EmbeddedVignette, EmbeddedWarp};
let src = scene(2, 900, 600);
let raw = SourceInfo { raw: true, ..Default::default() };
// The unrotated case includes a position only 0.000035 px outside the edge.
// f32 classification previously sampled the photo instead of blank canvas.
// Embedded DNG lens corrections (per-plane warp + vignette) with manual CA.
let lens = EmbeddedLens {
warp: Some(EmbeddedWarp {
planes: [[1.0, -0.03, 0.01, 0.0, 0.001, -0.002], [1.0, -0.028, 0.01, 0.0, 0.001, -0.002], [1.0, -0.026, 0.01, 0.0, 0.001, -0.002]],
@@ -460,7 +471,7 @@ fn geometry_variants() {
s.optics.ca_red = 30.0;
s.geometry.horizontal = -15.0;
check("embedded lens + perspective", &src, &info, &s, &RenderRequest::fit(700, 700));
s.orientation = Rotate270;
s.orientation = Orientation::Rotate270;
check("embedded lens rotated", &src, &info, &s, &RenderRequest::fit(700, 700));
}
+4 -2
View File
@@ -11,8 +11,10 @@ use serde_json::{Value, json};
use crate::backend::Backend;
/// File extensions recognised as photos when expanding folders.
pub const PHOTO_EXTENSIONS: &[&str] =
&["jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "arw", "raf", "orf", "rw2", "pef", "psd", "jxl", "gif", "bmp", "avif"];
pub const PHOTO_EXTENSIONS: &[&str] = &[
"jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "arw", "raf", "orf", "rw2", "rwl", "raw", "pef", "psd", "jxl", "gif",
"bmp", "avif",
];
/// Headless backend: a [`Session`] with filesystem hooks.
pub struct Headless {
+2 -5
View File
@@ -282,13 +282,10 @@ fn sample_warped(base: &Rgb32f, sx: f64, sy: f64, wp: &Warp, o2t: Affine, w: usi
let mut out = Rgb32f::new(w, h);
par_rows(&mut out.data, w, |y, row| {
for (x, px) in row.iter_mut().enumerate() {
let t = o2t.apply(Point::new(x as f64 + 0.5, y as f64 + 0.5));
let c = wp.to_corrected(t);
let g = wp.corrected_to_source(c, 1);
if !wp.inside(g) {
let Some((c, g)) = wp.frame(&o2t, x, y) else {
*px = BLANK;
continue;
}
};
let mut v = base.sample_bilinear((g.x * sx) as f32, (g.y * sy) as f32);
if per_channel {
for ch in [0usize, 2] {
+313 -21
View File
@@ -17,7 +17,7 @@ use std::sync::Mutex;
use lightcraft_color::perceptual::{oklab_from_2020, oklab_to_2020};
use lightcraft_develop::{DevelopSettings, EmbeddedLens, EmbeddedVignette, EmbeddedWarp};
use lightcraft_geom::{Homography, Orientation, Point};
use lightcraft_geom::{Affine, Homography, Interval, Orientation, Point, Real};
use lightcraft_raster::{Plane, Rgb32f};
use crate::for_rows;
@@ -122,12 +122,18 @@ impl Warp {
/// Transformed px → lens-corrected px (inverse perspective).
pub fn to_corrected(&self, p: Point) -> Point {
let (x, y) = self.to_corrected_real(p.x, p.y);
Point::new(x, y)
}
/// [`Warp::to_corrected`] for any [`Real`].
fn to_corrected_real<T: Real>(&self, x: T, y: T) -> (T, T) {
if self.persp == Homography::IDENTITY {
return p;
return (x, y);
}
let (c, l) = (self.centre(), self.half_long());
let q = self.persp_inv.apply(Point::new((p.x - c.x) / l, (p.y - c.y) / l));
Point::new(c.x + q.x * l, c.y + q.y * l)
let (qx, qy) = self.persp_inv.apply_real((x - c.x) / l, (y - c.y) / l);
(qx * l + c.x, qy * l + c.y)
}
/// Lens-corrected px → transformed px (forward perspective).
@@ -142,26 +148,32 @@ impl Warp {
/// Lens-corrected px → source px for colour plane `ch` (0 = R, 1 = G, 2 = B).
pub fn corrected_to_source(&self, p: Point, ch: usize) -> Point {
let (x, y) = self.corrected_to_source_real(p.x, p.y, ch);
Point::new(x, y)
}
/// [`Warp::corrected_to_source`] for any [`Real`].
fn corrected_to_source_real<T: Real>(&self, x: T, y: T, ch: usize) -> (T, T) {
let c = self.centre();
let hd = (self.w.hypot(self.h) / 2.0).max(1e-9);
let mut q = p;
let (mut qx, mut qy) = (x, y);
if self.k1 != 0.0 {
let (dx, dy) = (q.x - c.x, q.y - c.y);
let (dx, dy) = (qx - c.x, qy - c.y);
let r2 = (dx * dx + dy * dy) / (hd * hd);
let f = 1.0 + self.k1 * r2;
q = Point::new(c.x + dx * f, c.y + dy * f);
let f = r2 * self.k1 + 1.0;
(qx, qy) = (dx * f + c.x, dy * f + c.y);
}
if self.lens_dist != 0.0
&& let Some(wp) = self.lens.and_then(|l| l.warp)
{
let s = embedded_warp(&wp, q, ch, self.w, self.h);
q = Point::new(q.x + (s.x - q.x) * self.lens_dist, q.y + (s.y - q.y) * self.lens_dist);
let (sx, sy) = embedded_warp_real(&wp, qx, qy, ch, self.w, self.h);
(qx, qy) = (qx + (sx - qx) * self.lens_dist, qy + (sy - qy) * self.lens_dist);
}
let k = self.ca[ch];
if k != 0.0 {
q = Point::new(c.x + (q.x - c.x) * (1.0 + k), c.y + (q.y - c.y) * (1.0 + k));
(qx, qy) = ((qx - c.x) * (1.0 + k) + c.x, (qy - c.y) * (1.0 + k) + c.y);
}
q
(qx, qy)
}
/// Transformed px → source px for plane `ch`.
@@ -185,24 +197,86 @@ impl Warp {
g as f32
}
/// Whether a source position lies on the image (with a half-pixel margin).
pub fn inside(&self, s: Point) -> bool {
s.x >= -0.5 && s.y >= -0.5 && s.x <= self.w + 0.5 && s.y <= self.h + 0.5
/// Whether a source position lies on the image (within [`edges`](Self::edges)).
fn inside(&self, s: Point) -> bool {
let (lo, hx, hy) = self.edges();
s.x >= lo && s.y >= lo && s.x <= hx && s.y <= hy
}
/// The source area that frames output pixels, `lo..=hx × lo..=hy`: the image with a half-pixel margin.
fn edges(&self) -> (f64, f64, f64) {
(-0.5, self.w + 0.5, self.h + 0.5)
}
/// The framing chain of output position `(x, y)` (`o2t`: output px → transformed px): its lens-corrected
/// position and its green-plane source position.
fn framing<T: Real>(&self, o2t: &Affine, x: T, y: T) -> ((T, T), (T, T)) {
let (tx, ty) = o2t.apply_real(x, y);
let (cx, cy) = self.to_corrected_real(tx, ty);
((cx, cy), self.corrected_to_source_real(cx, cy, 1))
}
/// The reference framing of output pixel `(x, y)`: its lens-corrected and green-plane source positions if
/// the latter is [`inside`](Self::inside) the image, else `None` (the pixel is blank canvas).
pub fn frame(&self, o2t: &Affine, x: usize, y: usize) -> Option<(Point, Point)> {
let ((cx, cy), (gx, gy)) = self.framing(o2t, x as f64 + 0.5, y as f64 + 0.5);
let g = Point::new(gx, gy);
self.inside(g).then(|| (Point::new(cx, cy), g))
}
/// Whether output pixel `(x, y)` shows the image ([`frame`](Self::frame) is `Some`).
pub fn covers(&self, o2t: &Affine, x: usize, y: usize) -> bool {
self.frame(o2t, x, y).is_some()
}
/// [`covers`](Self::covers) for every pixel of the output block `x0..x1 × y0..y1` at once, when that is
/// certain: `Some(true)` if all are covered, `Some(false)` if none is, `None` if the block may straddle the
/// image edge (or can't be bounded) and each pixel must be decided alone.
///
/// Why the answer equals the per-pixel decision: the block's pixel centres `x + 0.5` are exact `f64` values in
/// `[x0 + 0.5, x1 − 0.5]` (likewise y). [`framing`](Self::framing) evaluated with [`Interval`]s runs the very
/// same operations as its `f64` evaluation in [`frame`](Self::frame), each rounded outward, so its result
/// bounds the `f64` source position of every pixel of the block — the rounded values, not only the exact
/// ones (see [`Interval`]). Divisions by a range that may contain zero, the homography's denominator clamp
/// and overflow make the bounds undecided (NaN), and NaN fails every test below. So when the bounds lie
/// within [`edges`](Self::edges) every pixel's position does, and when they lie beyond one edge every
/// pixel's position does: the same `f64` comparisons [`inside`](Self::inside) makes, no margin needed.
pub fn block_coverage(&self, o2t: &Affine, x0: usize, x1: usize, y0: usize, y1: usize) -> Option<bool> {
if x1 <= x0 || y1 <= y0 {
return None;
}
let px = Interval::new(x0 as f64 + 0.5, (x1 - 1) as f64 + 0.5);
let py = Interval::new(y0 as f64 + 0.5, (y1 - 1) as f64 + 0.5);
let (_, (sx, sy)) = self.framing(o2t, px, py);
let (lo, hx, hy) = self.edges();
if sx.lo() >= lo && sy.lo() >= lo && sx.hi() <= hx && sy.hi() <= hy {
Some(true)
} else if sx.hi() < lo || sy.hi() < lo || sx.lo() > hx || sy.lo() > hy {
Some(false)
} else {
None
}
}
}
/// DNG `WarpRectilinear` source position for corrected position `p` (px) and plane `ch`.
pub fn embedded_warp(wp: &EmbeddedWarp, p: Point, ch: usize, w: f64, h: f64) -> Point {
let (x, y) = embedded_warp_real(wp, p.x, p.y, ch, w, h);
Point::new(x, y)
}
/// [`embedded_warp`] for any [`Real`].
fn embedded_warp_real<T: Real>(wp: &EmbeddedWarp, x: T, y: T, ch: usize, w: f64, h: f64) -> (T, T) {
let (cx, cy) = (wp.center.x * w, wp.center.y * h);
let m = (wp.radius * w.max(h)).max(1e-9);
let [kr0, kr1, kr2, kr3, kt0, kt1] = wp.planes[ch.min(2)];
let dx = (p.x - cx) / m;
let dy = (p.y - cy) / m;
let dx = (x - cx) / m;
let dy = (y - cy) / m;
let r2 = dx * dx + dy * dy;
let f = kr0 + r2 * (kr1 + r2 * (kr2 + r2 * kr3));
let sx = dx * f + kt0 * 2.0 * dx * dy + kt1 * (r2 + 2.0 * dx * dx);
let sy = dy * f + kt1 * 2.0 * dx * dy + kt0 * (r2 + 2.0 * dy * dy);
Point::new(cx + m * sx, cy + m * sy)
let f = (((r2 * kr3 + kr2) * r2 + kr1) * r2) + kr0;
let sx = dx * f + dx * (kt0 * 2.0) * dy + (r2 + dx * 2.0 * dx) * kt1;
let sy = dy * f + dx * (kt1 * 2.0) * dy + (r2 + dy * 2.0 * dy) * kt0;
(sx * m + cx, sy * m + cy)
}
/// DNG `FixVignetteRadial` gain at source position `p` (px).
@@ -514,6 +588,224 @@ fn min_filter(p: &Plane, r: usize, max: bool) -> Plane {
mod tests {
use super::*;
/// splitmix64: deterministic pseudo-random numbers for the warp tests.
struct Rng(u64);
impl Rng {
fn next(&mut self) -> u64 {
self.0 = self.0.wrapping_add(0x9E37_79B9_7F4A_7C15);
let mut z = self.0;
z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9);
z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB);
z ^ (z >> 31)
}
/// Uniform in `lo..hi`.
fn range(&mut self, lo: f64, hi: f64) -> f64 {
lo + (hi - lo) * ((self.next() >> 11) as f64 / (1u64 << 53) as f64)
}
/// `range(lo, hi)` or, one time in four, exactly 0 (to take the "no correction" branches).
fn maybe(&mut self, lo: f64, hi: f64) -> f64 {
if self.next().is_multiple_of(4) { 0.0 } else { self.range(lo, hi) }
}
}
/// A pseudo-random warp: perspective, manual distortion, CA and an embedded lens warp, each sometimes off.
/// Sizes are Pythagorean (exact `hypot`), so results don't depend on the platform's libm.
fn random_warp(r: &mut Rng) -> Warp {
const SIZES: [(f64, f64); 6] = [(4000.0, 3000.0), (3000.0, 4000.0), (1200.0, 900.0), (1200.0, 500.0), (56.0, 33.0), (6000.0, 4500.0)];
let (w, h) = SIZES[(r.next() % SIZES.len() as u64) as usize];
let mut wp = Warp::identity(w, h);
if !r.next().is_multiple_of(4) {
let m = [
1.0 + r.range(-0.2, 0.2),
r.range(-0.2, 0.2),
r.range(-0.1, 0.1),
r.range(-0.2, 0.2),
1.0 + r.range(-0.2, 0.2),
r.range(-0.1, 0.1),
r.range(-0.4, 0.4),
r.range(-0.4, 0.4),
1.0 + r.range(-0.1, 0.1),
];
wp.persp_inv = Homography(m);
wp.persp = wp.persp_inv.inverse().unwrap_or(Homography::IDENTITY);
}
wp.k1 = r.maybe(-0.2, 0.2);
wp.ca = [r.maybe(-0.003, 0.003), r.maybe(-0.003, 0.003), r.maybe(-0.003, 0.003)];
if !r.next().is_multiple_of(4) {
let mut plane = || {
[
1.0 + r.range(-0.05, 0.05),
r.range(-0.08, 0.08),
r.range(-0.05, 0.05),
r.range(-0.02, 0.02),
r.range(-0.003, 0.003),
r.range(-0.003, 0.003),
]
};
let planes = [plane(), plane(), plane()];
let warp = EmbeddedWarp { planes, center: Point::new(r.range(0.4, 0.6), r.range(0.4, 0.6)), radius: r.range(0.5, 0.8) };
wp.lens = Some(EmbeddedLens { warp: Some(warp), vignette: None });
wp.lens_dist = r.maybe(0.0, 2.0);
}
wp
}
/// FNV-1a over the bits of the f64 warp results (`Affine::apply`, `Homography::apply`, `to_corrected`,
/// `from_corrected`, `corrected_to_source`, `to_source`, `embedded_warp`) for pseudo-random warps and points.
/// The expected value was captured before these formulas became generic over `Real`: the f64 path must keep
/// the same operations in the same order, bit for bit. To recompute it after changing the inputs, run this
/// test (printing `hash`) on a scratch copy of 419261c (thumb-split, pre-generic formulas) and paste it here.
#[test]
fn warp_f64_results_are_bit_identical_to_the_pre_generic_formulas() {
let mut hash = 0xCBF2_9CE4_8422_2325u64;
let mut eat = |p: Point| {
for v in [p.x, p.y] {
for b in v.to_bits().to_le_bytes() {
hash = (hash ^ b as u64).wrapping_mul(0x0000_0100_0000_01B3);
}
}
};
let mut r = Rng(42);
for _ in 0..2000 {
let wp = random_warp(&mut r);
let a = lightcraft_geom::Affine([
r.range(-2.0, 2.0),
r.range(-2.0, 2.0),
r.range(-2.0, 2.0),
r.range(-2.0, 2.0),
r.range(-500.0, 500.0),
r.range(-500.0, 500.0),
]);
for _ in 0..40 {
let p = Point::new(r.range(-0.2, 1.2) * wp.w, r.range(-0.2, 1.2) * wp.h);
eat(a.apply(p));
eat(wp.persp_inv.apply(Point::new(p.x / wp.w - 0.5, p.y / wp.h - 0.5)));
eat(wp.to_corrected(p));
eat(wp.from_corrected(p));
for ch in 0..3 {
eat(wp.corrected_to_source(p, ch));
eat(wp.to_source(p, ch));
}
if let Some(ew) = wp.lens.and_then(|l| l.warp) {
for ch in 0..4 {
eat(embedded_warp(&ew, p, ch, wp.w, wp.h));
}
}
}
}
// `Homography::apply`'s near-zero denominator clamp, including subnormal and signed-zero denominators and
// ±1e-300 exactly (not clamped: `<`, not `<=`)
let hz = Homography([1.0, 0.5, 0.25, -0.5, 1.0, 0.125, 1.0, 0.0, 0.0]);
for x in [0.0, -0.0, 1e-310, -1e-310, 1e-300, -1e-300, 1e-299, -1e-299, 0.5] {
eat(hz.apply(Point::new(x, 0.75)));
}
assert_eq!(hash, 0x3FA5_7473_D795_9E32, "f64 warp results changed");
}
/// The interval evaluation of `framing` bounds the f64 evaluation of every point of the box.
#[test]
fn interval_framing_encloses_every_f64_result() {
let mut r = Rng(7);
let (mut decided, mut total) = (0, 0);
for _ in 0..500 {
let wp = random_warp(&mut r);
let a = lightcraft_geom::Affine([
r.range(0.5, 2.0),
r.range(-0.2, 0.2),
r.range(-0.2, 0.2),
r.range(0.5, 2.0),
r.range(-300.0, 300.0),
r.range(-300.0, 300.0),
]);
for _ in 0..20 {
let (x0, y0) = (r.range(-0.2, 1.1) * wp.w, r.range(-0.2, 1.1) * wp.h);
let (bw, bh) = (r.range(0.0, 40.0), r.range(0.0, 40.0));
let (ix, iy) = (Interval::new(x0, x0 + bw), Interval::new(y0, y0 + bh));
let (_, (sx, sy)) = wp.framing(&a, ix, iy);
total += 1;
if !(sx.is_decided() && sy.is_decided()) {
continue;
}
decided += 1;
for i in 0..=6 {
for j in 0..=6 {
let (x, y) = if i == 6 && j == 6 {
(r.range(x0, x0 + bw), r.range(y0, y0 + bh))
} else {
(x0 + bw * i as f64 / 5.0, y0 + bh * j as f64 / 5.0)
};
let (x, y) = (x.clamp(ix.lo(), ix.hi()), y.clamp(iy.lo(), iy.hi()));
let (_, (fx, fy)) = wp.framing(&a, x, y);
assert!(
sx.lo() <= fx && fx <= sx.hi() && sy.lo() <= fy && fy <= sy.hi(),
"({x}, {y}) → ({fx}, {fy}) outside {sx:?} × {sy:?}"
);
}
}
}
}
assert!(decided * 10 > total * 9, "{decided}/{total} boxes bounded");
}
#[test]
fn block_coverage_agrees_with_every_pixel() {
let lens = EmbeddedLens {
warp: Some(EmbeddedWarp {
planes: [[1.0, -0.03, 0.01, 0.0, 0.001, -0.002], [1.0, -0.028, 0.01, 0.0, 0.001, -0.002], [1.0, -0.026, 0.01, 0.0, 0.001, -0.002]],
center: Point::new(0.52, 0.48),
radius: 0.6,
}),
vignette: None,
};
let mut cases: Vec<DevelopSettings> = Vec::new();
let mut s = DevelopSettings::default();
s.optics.lens_profile = true;
s.optics.ca_red = 30.0;
s.geometry.horizontal = -15.0;
cases.push(s.clone());
s.orientation = Orientation::Rotate270;
cases.push(s.clone());
let mut s = DevelopSettings::default();
s.optics.distortion = -60.0;
s.optics.ca_blue = 50.0;
s.geometry.vertical = 25.0;
s.geometry.rotate = 7.0;
cases.push(s.clone());
s.optics.distortion = 80.0;
s.geometry.horizontal = 30.0;
s.orientation = Orientation::Rotate90;
cases.push(s);
for (i, s) in cases.iter().enumerate() {
let f = crate::geometry::Frame::with_lens(900, 600, s, true, Some(&lens));
let wp = f.warp.as_ref().expect("warp");
let (w, h) = f.fit(700, 700);
let o2t = f.out_to_oriented(w, h);
let (mut decided, mut tiles, mut blank) = (0, 0, 0);
for (tw, th) in [(32, 8), (7, 5), (64, 64)] {
for y0 in (0..h).step_by(th) {
for x0 in (0..w).step_by(tw) {
let (x1, y1) = ((x0 + tw).min(w), (y0 + th).min(h));
tiles += 1;
let Some(all) = wp.block_coverage(&o2t, x0, x1, y0, y1) else { continue };
decided += 1;
for y in y0..y1 {
for x in x0..x1 {
assert_eq!(wp.covers(&o2t, x, y), all, "case {i}: pixel {x},{y} in block {x0}..{x1} × {y0}..{y1}");
blank += !all as usize;
}
}
}
}
}
// most blocks are decided wholesale, including blank ones away from the edge
assert!(decided * 10 > tiles * 8, "case {i}: {decided}/{tiles} blocks decided");
// strong distortion leaves blank margins wide enough for whole blocks
assert!(blank > 0 || i < 2, "case {i}: no blank block decided");
assert_eq!(wp.block_coverage(&o2t, 5, 5, 0, 1), None, "empty block");
}
}
/// A grid of dark lines on white with lateral CA: red magnified by `ar`, blue by `ab` (about the centre).
fn ca_grid(w: usize, h: usize, ar: f64, ab: f64) -> Rgb32f {
let (cx, cy) = (w as f64 / 2.0, h as f64 / 2.0);
+16 -4
View File
@@ -22,7 +22,8 @@ const QUALITY: u8 = 90;
pub struct DiskCache {
dir: PathBuf,
budget: u64,
/// Bytes the cache may hold (changed in place by [`DiskCache::set_budget`]).
budget: AtomicU64,
/// Bytes on disk (`None` until first scanned).
total: Mutex<Option<u64>>,
pub hits: AtomicU64,
@@ -34,7 +35,7 @@ impl DiskCache {
pub fn new(dir: &Path, budget: u64) -> DiskCache {
DiskCache {
dir: dir.to_path_buf(),
budget,
budget: AtomicU64::new(budget),
total: Mutex::new(None),
hits: AtomicU64::new(0),
misses: AtomicU64::new(0),
@@ -46,6 +47,17 @@ impl DiskCache {
&self.dir
}
/// Bytes the cache may hold.
pub fn budget(&self) -> u64 {
self.budget.load(Ordering::Relaxed)
}
/// Change the budget in place: the files stay valid (they are keyed by content), so a resize
/// keeps them; a smaller budget is enforced by the next write, like any other overflow.
pub fn set_budget(&self, bytes: u64) {
self.budget.store(bytes, Ordering::Relaxed);
}
fn path(&self, key: Hash128) -> PathBuf {
let hex = key.to_string();
self.dir.join(&hex[..2]).join(format!("{hex}.jpg"))
@@ -94,7 +106,7 @@ impl DiskCache {
let mut t = self.total.lock().unwrap_or_else(|e| e.into_inner());
let total = t.get_or_insert(0);
*total += bytes.len() as u64;
*total > self.budget
*total > self.budget()
};
if over {
self.prune();
@@ -134,7 +146,7 @@ impl DiskCache {
pub fn prune(&self) {
let mut files = self.scan();
let mut total: u64 = files.iter().map(|f| f.1).sum();
let target = self.budget / 5 * 4;
let target = self.budget() / 5 * 4;
files.sort_by_key(|f| f.2);
for (p, len, _) in files {
if total <= target {
+57 -4
View File
@@ -18,7 +18,7 @@ pub mod lru;
pub mod pool;
use std::path::Path;
use std::sync::{Arc, Mutex};
use std::sync::{Arc, Mutex, RwLock};
pub use disk::{DiskCache, decode_jpeg, encode_jpeg};
pub use hash::{Hash128, Hasher128, hash_bytes};
@@ -28,6 +28,8 @@ pub use pool::JobPool;
/// Rendered-thumbnail cache: memory LRU over an optional disk cache.
pub struct PreviewCache {
/// Current generation; `None` once [`Self::retire`]d (nothing is read or written any more).
generation: RwLock<Option<u64>>,
mem: Mutex<Lru<Hash128, Arc<Rgba8>>>,
disk: Option<DiskCache>,
}
@@ -35,12 +37,12 @@ pub struct PreviewCache {
impl PreviewCache {
/// Memory-only cache of at most `mem_bytes`.
pub fn memory(mem_bytes: usize) -> PreviewCache {
PreviewCache { mem: Mutex::new(Lru::new(mem_bytes)), disk: None }
PreviewCache { generation: RwLock::new(Some(0)), mem: Mutex::new(Lru::new(mem_bytes)), disk: None }
}
/// Memory LRU + disk cache in `dir` (at most `disk_bytes`).
pub fn with_disk(mem_bytes: usize, dir: &Path, disk_bytes: u64) -> PreviewCache {
PreviewCache { mem: Mutex::new(Lru::new(mem_bytes)), disk: Some(DiskCache::new(dir, disk_bytes)) }
PreviewCache { generation: RwLock::new(Some(0)), mem: Mutex::new(Lru::new(mem_bytes)), disk: Some(DiskCache::new(dir, disk_bytes)) }
}
pub fn disk(&self) -> Option<&DiskCache> {
@@ -48,6 +50,20 @@ impl PreviewCache {
}
pub fn get(&self, key: Hash128) -> Option<Arc<Rgba8>> {
self.get_at(self.generation(), key)
}
/// Changes when explicitly cleared; jobs captured before that clear cannot repopulate it.
/// A retired cache reports `u64::MAX`, which no job's generation matches.
pub fn generation(&self) -> u64 {
self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner).unwrap_or(u64::MAX)
}
pub fn get_at(&self, generation: u64, key: Hash128) -> Option<Arc<Rgba8>> {
let current = self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner);
if *current != Some(generation) {
return None;
}
if let Some(v) = self.mem.lock().unwrap_or_else(|e| e.into_inner()).get(&key) {
return Some(v.clone());
}
@@ -57,6 +73,14 @@ impl PreviewCache {
}
pub fn put(&self, key: Hash128, img: Arc<Rgba8>) {
self.put_at(self.generation(), key, img);
}
pub fn put_at(&self, generation: u64, key: Hash128, img: Arc<Rgba8>) {
let current = self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner);
if *current != Some(generation) {
return;
}
if let Some(d) = &self.disk {
d.put(key, &img);
}
@@ -67,13 +91,26 @@ impl PreviewCache {
/// Like [`Self::put`], but the disk write (a JPEG encode) happens on a background thread, so
/// the caller (a render about to hand its result to the screen) isn't held up.
pub fn put_deferred(self: &Arc<Self>, key: Hash128, img: Arc<Rgba8>) {
self.put_deferred_at(self.generation(), key, img);
}
pub fn put_deferred_at(self: &Arc<Self>, generation: u64, key: Hash128, img: Arc<Rgba8>) {
let current = self.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner);
if *current != Some(generation) {
return;
}
let c = cost(&img);
self.mem.lock().unwrap_or_else(|e| e.into_inner()).insert(key, img.clone(), c);
drop(current);
if self.disk.is_none() {
return;
}
let me = self.clone();
let write = move || {
let current = me.generation.read().unwrap_or_else(std::sync::PoisonError::into_inner);
if *current != Some(generation) {
return;
}
if let Some(d) = &me.disk {
d.put(key, &img);
}
@@ -86,14 +123,30 @@ impl PreviewCache {
write();
}
/// Drop everything (memory and disk).
/// Drop everything (memory and disk). Does nothing once [`Self::retire`]d.
pub fn clear(&self) {
let mut generation = self.generation.write().unwrap_or_else(std::sync::PoisonError::into_inner);
// a retired cache no longer owns its directory (its replacement may)
let Some(g) = generation.as_mut() else { return };
// (never reaches u64::MAX, which stands for "retired")
*g = g.wrapping_add(1) % u64::MAX;
self.mem.lock().unwrap_or_else(|e| e.into_inner()).clear();
if let Some(d) = &self.disk {
d.clear();
}
}
/// Stop this cache object for good, leaving its disk files alone: called when another cache
/// object replaces it (a library reopened or another one opened). Jobs still holding it then
/// can neither read nor write through it, so a later clear of the replacement's directory
/// (possibly the same one) can't be repopulated by them. Waits for writes in progress, like
/// [`Self::clear`].
pub fn retire(&self) {
let mut generation = self.generation.write().unwrap_or_else(std::sync::PoisonError::into_inner);
*generation = None;
self.mem.lock().unwrap_or_else(|e| e.into_inner()).clear();
}
/// Use tick ([`next_tick`]) of the least recently used image in memory.
pub fn oldest_tick(&self) -> Option<u64> {
self.mem.lock().unwrap_or_else(|e| e.into_inner()).oldest_tick()
+46
View File
@@ -4,6 +4,52 @@ use lightcraft_raster::Rgba8;
use super::*;
#[test]
fn cleared_generation_cannot_read_or_repopulate_memory_or_disk() {
let dir = std::env::temp_dir().join(format!("lc-thumbnail-generation-{}-{}", std::process::id(), next_tick()));
let cache = Arc::new(PreviewCache::with_disk(1 << 20, &dir, 1 << 20));
let key = hash_bytes(b"same-key");
let old = cache.generation();
let wrong = Arc::new(gradient(8, 8, 255));
let correct = Arc::new(gradient(8, 8, 0));
cache.put_at(old, key, wrong.clone());
assert!(cache.get_at(old, key).is_some());
cache.clear();
assert!(cache.get(key).is_none());
assert!(cache.get_at(old, key).is_none());
cache.put_at(old, key, wrong.clone());
cache.put_deferred_at(old, key, wrong);
assert!(cache.get(key).is_none());
assert!(cache.disk().unwrap().get(key).is_none());
cache.put_at(cache.generation(), key, correct.clone());
assert_eq!(cache.get(key).unwrap().as_bytes(), correct.as_bytes());
cache.clear();
}
#[test]
fn retired_cache_keeps_its_files_but_refuses_every_read_and_write() {
let dir = temp_dir("retire");
let old = Arc::new(PreviewCache::with_disk(1 << 20, &dir, 1 << 20));
let (kept, late) = (hash_bytes(b"kept"), hash_bytes(b"late"));
let img = Arc::new(gradient(8, 8, 7));
let generation = old.generation();
old.put_at(generation, kept, img.clone());
old.retire();
assert_ne!(old.generation(), generation);
assert!(old.get_at(generation, kept).is_none());
assert!(old.get(kept).is_none());
old.put_at(generation, late, img.clone());
old.put_deferred_at(generation, late, img.clone());
old.put(late, img.clone());
old.put_deferred(late, img.clone());
old.clear();
// a replacement for the same directory still finds the valid file, and only that
let new = PreviewCache::with_disk(1 << 20, &dir, 1 << 20);
assert_eq!(new.get(kept).map(|i| (i.width, i.height)), Some((8, 8)), "the disk file stays (JPEG: lossy)");
assert!(new.get(late).is_none());
new.clear();
}
fn temp_dir(tag: &str) -> std::path::PathBuf {
let d = std::env::temp_dir().join(format!("lc-preview-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
+3
View File
@@ -177,6 +177,9 @@ pub fn write_dng(raw: &RawImage, opts: &DngWriteOptions) -> Result<Vec<u8>> {
ifd.set(t::AS_SHOT_NEUTRAL, rat_vec(&v));
} else if let Some(xy) = c.as_shot_white_xy {
ifd.set(t::AS_SHOT_WHITE_XY, rat_vec(&[xy.x, xy.y]));
} else if let Some(m) = raw.wb_multipliers.filter(|m| m.iter().all(|v| *v > 0.0 && v.is_finite())) {
// vendor raws (NEF, ARW…) carry white balance as multipliers: the DNG neutral is their reciprocal
ifd.set(t::AS_SHOT_NEUTRAL, rat_vec(&m.map(|v| 1.0 / v as f64)));
}
ifd.set(t::BASELINE_EXPOSURE, Value::SRational(vec![srational(c.baseline_exposure)]));
// the source's own profile look travels with its data (conversions, smart previews, merges)
+3 -2
View File
@@ -13,9 +13,10 @@
//!
//! Formats: DNG (uncompressed, lossless JPEG, lossy JPEG (Smart Previews), Deflate incl. floating point, tiled/stripped, CFA and LinearRaw),
//! Canon CR2, Nikon NEF/NRW (uncompressed, Huffman lossless / lossy compressed), Sony ARW (uncompressed, ARW2, lossless), Fujifilm RAF (uncompressed Bayer
//! and X-Trans), Panasonic RW2 (packed 12/14-bit), Pentax PEF (uncompressed, Huffman), Olympus ORF (uncompressed).
//! and X-Trans), Panasonic RW2 / Leica RWL / Panasonic RAW (every raw format: compressed 4 and 6, the prefix-coded strips of 8,
//! packed 2/5/7, the 16-bit words of the oldest bodies), Pentax PEF (uncompressed, Huffman), Olympus ORF (uncompressed).
//! [`embedded_preview`] covers all of them plus CR3. Variants we can't decode yet (Nikon "lossy after split" NEF,
//! Panasonic quantised RW2, compressed ORF/RAF, CR3) return [`RawError::Unsupported`]; each vendor module documents its sources
//! compressed ORF/RAF, CR3) return [`RawError::Unsupported`]; each vendor module documents its sources
//! (public specifications, tag-name documentation, black-box analysis of CC0 samples) and gaps. Non-DNG files carry no
//! colour matrix: [`color`] falls back to a documented neutral model. The decoders never panic on malformed input.
#![forbid(unsafe_code)]
+174 -3
View File
@@ -207,6 +207,7 @@ struct Header {
height: usize,
width: usize,
comps: Vec<u8>,
sampling: Vec<u8>,
tables: [Option<Huffman>; 4],
table_for: Vec<usize>,
predictor: u8,
@@ -228,6 +229,7 @@ fn parse_header(d: &[u8]) -> Result<Header, RawError> {
height: 0,
width: 0,
comps: vec![],
sampling: vec![],
tables: [None, None, None, None],
table_for: vec![],
predictor: 1,
@@ -272,10 +274,11 @@ fn parse_header(d: &[u8]) -> Result<Header, RawError> {
}
for c in 0..n {
let (id, samp) = (seg[6 + 3 * c], seg[7 + 3 * c]);
if samp != 0x11 {
return Err(RawError::Unsupported("lossless JPEG with subsampled components".into()));
if !(1..=4).contains(&(samp >> 4)) || !(1..=4).contains(&(samp & 15)) {
return Err(err("invalid sampling factors"));
}
h.comps.push(id);
h.sampling.push(samp);
}
if !(2..=16).contains(&h.precision) {
return Err(err("bad precision"));
@@ -340,12 +343,119 @@ fn parse_header(d: &[u8]) -> Result<Header, RawError> {
/// Parse only the frame header: (width, height, components, precision).
pub fn frame_info(d: &[u8]) -> Result<(usize, usize, usize, u8), RawError> {
let h = parse_header(d)?;
check_full_resolution(&h)?;
Ok((h.width, h.height, h.comps.len(), h.precision))
}
fn check_full_resolution(h: &Header) -> Result<(), RawError> {
if h.sampling.iter().any(|&s| s != 0x11) {
return Err(RawError::Unsupported("lossless JPEG with subsampled components".into()));
}
Ok(())
}
fn check_subsampled(h: &Header) -> Result<usize, RawError> {
let vertical = match h.sampling.as_slice() {
[0x22, 0x11, 0x11] => 2,
[0x21, 0x11, 0x11] => 1,
_ => return Err(RawError::Unsupported("lossless JPEG subsampling layout".into())),
};
if h.predictor != 1 || h.restart != 0 {
return Err(RawError::Unsupported("lossless JPEG subsampled predictor / restarts".into()));
}
if h.width == 0 || h.height == 0 || !h.width.is_multiple_of(2) || !h.height.is_multiple_of(vertical) {
return Err(err("subsampled frame dimensions do not match MCU size"));
}
Ok(vertical)
}
/// Sony M/S tiles: horizontal prediction, no restart markers. Keep subsampled planes separate;
/// expanding them into a mosaic would silently corrupt the existing CFA callers.
pub(crate) fn frame_info_subsampled(d: &[u8]) -> Result<(usize, usize), RawError> {
let h = parse_header(d)?;
check_subsampled(&h)?;
Ok((h.width, h.height))
}
pub(crate) struct FrameSubsampled {
pub width: usize,
pub height: usize,
pub vertical_subsampling: usize,
/// Y at full resolution; Cb and Cr at half width, with 1× or 2× vertical subsampling.
pub planes: [Vec<u16>; 3],
}
/// Sony's 4:2:0 / 4:2:2 LJ92 variants: T.81 MCU ordering and differences, modulo 2^16.
/// At the first column, the top luma row predicts from the previous MCU row's top
/// sample (two image rows above), and the bottom row from the current top sample.
/// Observed black-box: using the immediately preceding image row introduces tile seams.
pub(crate) fn decode_subsampled(d: &[u8], max_samples: usize) -> Result<FrameSubsampled, RawError> {
let h = parse_header(d)?;
let vertical = check_subsampled(&h)?;
let pixels = h.width.checked_mul(h.height).ok_or_else(|| err("frame too large"))?;
let chroma = pixels / (2 * vertical);
let total = pixels.checked_add(chroma.checked_mul(2).ok_or_else(|| err("frame too large"))?).ok_or_else(|| err("frame too large"))?;
if total > max_samples {
return Err(RawError::Limit("lossless JPEG frame larger than expected"));
}
let entropy = &d[h.scan_start..];
if (entropy.len() as u64 + 64) * 8 < total as u64 {
return Err(err("entropy data too short for frame"));
}
let tables: Vec<&Huffman> = h
.table_for
.iter()
.map(|&t| h.tables.get(t).and_then(Option::as_ref).ok_or_else(|| err("missing Huffman table")))
.collect::<Result<_, _>>()?;
let mut planes = [vec![0u16; pixels], vec![0u16; chroma], vec![0u16; chroma]];
let mut br = BitReader::new(entropy);
let init = 1i32 << (h.precision - h.pt - 1);
for my in 0..h.height / vertical {
for mx in 0..h.width / 2 {
for (c, plane) in planes.iter_mut().enumerate() {
let horizontal_factor = if c == 0 { 2 } else { 1 };
let vertical_factor = if c == 0 { vertical } else { 1 };
let width = h.width * horizontal_factor / 2;
for dy in 0..vertical_factor {
for dx in 0..horizontal_factor {
let (x, y) = (mx * horizontal_factor + dx, my * vertical_factor + dy);
let i = y * width + x;
let pred = if x > 0 {
plane[i - 1] as i32
} else if c == 0 && dy == 0 && y >= vertical {
plane[i - vertical * width] as i32
} else if y > 0 {
plane[i - width] as i32
} else {
init
};
let ssss = tables[c].decode(&mut br)?;
if ssss > 16 {
return Err(err("invalid difference category"));
}
plane[i] = ((pred + diff_value(&mut br, ssss)) & 0xffff) as u16;
}
}
}
}
if br.overrun > br.bits {
return Err(err("entropy data exhausted"));
}
}
if h.pt > 0 {
for plane in &mut planes {
for v in plane {
*v <<= h.pt;
}
}
}
Ok(FrameSubsampled { width: h.width, height: h.height, vertical_subsampling: vertical, planes })
}
/// Decode a lossless JPEG stream. `max_samples` bounds the allocation (hostile headers).
pub fn decode(d: &[u8], max_samples: usize) -> Result<Frame, RawError> {
let h = parse_header(d)?;
check_full_resolution(&h)?;
let (w, ht, nc) = (h.width, h.height, h.comps.len());
if w == 0 || ht == 0 {
return Err(err("zero frame size"));
@@ -701,9 +811,70 @@ pub fn encode(data: &[u16], width: usize, height: usize, components: usize, prec
}
#[cfg(test)]
mod tests {
pub(crate) mod tests {
use super::*;
/// Independent 4×4, 16-bit Sony 4:2:0 stream. Hand-specified differences in MCU order.
pub(crate) fn fixture_420() -> Vec<u8> {
fixture_subsampled(4, 0x22, &[-31768, 1, 1000, 1, -16384, -16284, 1, 1, 1, 1, 1, 2, 100, 1, 1000, 1, 100, 100, 1, 1, 1, 1, 1, 2])
}
fn fixture_subsampled(height: u8, sampling: u8, differences: &[i32]) -> Vec<u8> {
let mut out = vec![0xff, 0xd8, 0xff, 0xc3, 0, 17, 16, 0, height, 0, 4, 3, 1, sampling, 0, 2, 0x11, 0, 3, 0x11, 0];
out.extend_from_slice(&[0xff, 0xc4, 0, 36, 0]);
let mut counts = [0; 16];
counts[4] = 17;
out.extend_from_slice(&counts);
out.extend(0..=16);
out.extend_from_slice(&[0xff, 0xda, 0, 12, 3, 1, 0, 2, 0, 3, 0, 1, 0, 0]);
let mut bw = BitWriter { out, acc: 0, n: 0 };
for &d in differences {
let s = ssss_of(d);
bw.put(s as u32, 5);
if s > 0 && s < 16 {
let v = if d < 0 { d - 1 } else { d };
bw.put(v as u32 & ((1 << s) - 1), s as u32);
}
}
bw.flush();
bw.out.extend_from_slice(&[0xff, 0xd9]);
bw.out
}
#[test]
fn horizontal_only_subsampling_preserves_rows_and_chroma() {
// Independent 4×2, 16-bit 4:2:2 stream: two luma samples, Cb, Cr per MCU.
let enc = fixture_subsampled(2, 0x21, &[-31768, 1, -16384, -16284, 1, 1, 1, 2, 100, 1, 100, 100, 1, 1, 1, 2]);
let frame = decode_subsampled(&enc, 16).unwrap();
assert_eq!((frame.width, frame.height, frame.vertical_subsampling), (4, 2, 1));
assert_eq!(frame.planes[0], [1000, 1001, 1002, 1003, 1100, 1101, 1102, 1103]);
assert_eq!(frame.planes[1], [16384, 16385, 16484, 16485]);
assert_eq!(frame.planes[2], [16484, 16486, 16584, 16586]);
assert!(decode(&enc, 24).is_err());
assert!(matches!(decode_subsampled(&enc, 15), Err(RawError::Limit(_))));
assert!(decode_subsampled(&enc[..enc.len() - 6], 16).is_err());
}
#[test]
fn subsampled_mcu_order_preserves_each_plane() {
let enc = fixture_420();
let frame = decode_subsampled(&enc, 24).unwrap();
assert_eq!((frame.width, frame.height), (4, 4));
assert_eq!(frame.planes[0], [1000, 1001, 1002, 1003, 2000, 2001, 2002, 2003, 1100, 1101, 1102, 1103, 2100, 2101, 2102, 2103]);
assert_eq!(frame.planes[1], [16384, 16385, 16484, 16485]);
assert_eq!(frame.planes[2], [16484, 16486, 16584, 16586]);
assert!(decode(&enc, 48).is_err()); // CFA API must keep rejecting subsampling.
assert!(matches!(decode_subsampled(&enc, 23), Err(RawError::Limit(_))));
assert!(decode_subsampled(&enc[..enc.len() - 6], 24).is_err());
let mut bad = enc.clone();
bad[10] = 3; // odd frame width
assert!(frame_info_subsampled(&bad).is_err());
let sos = bad.windows(2).position(|w| w == [0xff, 0xda]).unwrap();
bad = enc.clone();
bad[sos + 11] = 2; // unsupported predictor
assert!(frame_info_subsampled(&bad).is_err());
}
fn noise(n: usize, bits: u32, seed: u64) -> Vec<u16> {
let mut s = seed;
(0..n)
+112 -4
View File
@@ -37,6 +37,8 @@ const BLACK_LEVEL: u16 = 0x7310;
const WB_RGGB: u16 = 0x7313;
const CROP_TOP_LEFT: u16 = 0x74c7;
const CROP_SIZE: u16 = 0x74c8;
const YCBCR_COEFFICIENTS: u16 = 529;
const REFERENCE_BLACK_WHITE: u16 = 532;
/// Maker-note tags (ExifTool Sony tag names): the enciphered `Tag2010` block and `FullImageSize` (height, width).
const MN_TAG2010: u16 = 0x2010;
const MN_FULL_IMAGE_SIZE: u16 = 0xb02b;
@@ -300,6 +302,71 @@ fn read_quad_tiles(bytes: &[u8], info: &ImageInfo) -> Result<Vec<u16>> {
Ok(out)
}
/// Downsized M/S ARWs are linear YCbCr 4:2:0 / 4:2:2, not CFA, despite retaining dummy CFA tags.
/// T.81 supplies the lossless coding; TIFF 6.0 supplies the YCbCr coefficients and reference
/// levels. Sony's reference chroma black and white are identical (the neutral offset).
fn read_ycbcr_tiles(bytes: &[u8], info: &ImageInfo, raw: &Ifd, mode: Mode) -> Result<Vec<u16>> {
let total = check_image(bytes, info)?;
if info.samples_per_pixel != 3 || info.planar != 1 || !matches!(info.layout, Layout::Tiles { .. }) {
return Err(RawError::Unsupported("Sony linear YCbCr tile layout".into()));
}
let chunks = info.chunks(bytes.len() as u64);
let first = chunks.first().and_then(|c| chunk_bytes(bytes, c)).ok_or_else(|| RawError::Corrupt("missing YCbCr tile".into()))?;
ljpeg::frame_info_subsampled(first)?;
let coefficients = raw.f64s(YCBCR_COEFFICIENTS).unwrap_or_else(|| vec![0.299, 0.587, 0.114]);
let [kr, kg, kb] = coefficients.as_slice() else { return Err(RawError::Corrupt("YCbCr coefficients".into())) };
if !coefficients.iter().all(|v| v.is_finite() && *v > 0.0 && *v < 1.0) || (kr + kg + kb - 1.0).abs() > 0.001 {
return Err(RawError::Corrupt("invalid YCbCr coefficients".into()));
}
let references = raw.f64s(REFERENCE_BLACK_WHITE).unwrap_or_else(|| vec![0.0, 16383.0, 16384.0, 16384.0, 16384.0, 16384.0]);
let [yblack, ywhite, cbzero, cbwhite, crzero, crwhite] = references.as_slice() else {
return Err(RawError::Corrupt("YCbCr reference levels".into()));
};
if !references.iter().all(|v| v.is_finite() && (0.0..=65535.0).contains(v))
|| *yblack != 0.0
|| *ywhite != 16383.0
|| cbzero != cbwhite
|| crzero != crwhite
{
return Err(RawError::Unsupported("Sony YCbCr reference levels".into()));
}
if mode == Mode::Header {
return Ok(Vec::new());
}
let decoded: Vec<(Chunk, ljpeg::FrameSubsampled)> = chunks
.par_iter()
.map(|c| {
let src = chunk_bytes(bytes, c).ok_or_else(|| RawError::Corrupt("YCbCr tile outside file".into()))?;
let limit = (c.width as usize).checked_mul(c.height as usize).and_then(|n| n.checked_mul(3)).ok_or(RawError::Limit("tile too large"))?;
let f = ljpeg::decode_subsampled(src, limit)?;
if f.width != c.width as usize || f.height != c.height as usize {
return Err(RawError::Corrupt("Sony YCbCr tile dimensions".into()));
}
Ok((*c, f))
})
.collect::<Result<_>>()?;
let (w, h) = (info.width as usize, info.height as usize);
let mut out = vec![0u16; total];
for (c, f) in decoded {
for y in 0..f.height.min(h.saturating_sub(c.y as usize)) {
for x in 0..f.width.min(w.saturating_sub(c.x as usize)) {
let luma = f.planes[0][y * f.width + x] as f64;
let ci = (y / f.vertical_subsampling) * (f.width / 2) + x / 2;
let cb = f.planes[1][ci] as f64 - cbzero;
let cr = f.planes[2][ci] as f64 - crzero;
let r = luma + (2.0 - 2.0 * kr) * cr;
let b = luma + (2.0 - 2.0 * kb) * cb;
let g = (luma - kr * r - kb * b) / kg;
let dst = (((c.y as usize + y) * w) + c.x as usize + x) * 3;
for (s, value) in [r, g, b].into_iter().enumerate() {
out[dst + s] = value.round().clamp(0.0, 65535.0) as u16;
}
}
}
}
Ok(out)
}
/// `SR2Private` tags (ExifTool "Sony SR2Private" table; the IFD is referenced by IFD0's `DNGPrivateData`).
const SR2_SUBIFD_OFFSET: u16 = 0x7200;
const SR2_SUBIFD_LENGTH: u16 = 0x7201;
@@ -384,9 +451,11 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
return Err(RawError::Limit("image too large"));
}
let bits = info.bits() as u32;
let linear_rgb = info.compression == 7 && info.photometric == photometric::YCBCR;
let chunks = info.chunks(bytes.len() as u64);
let strip_len: u64 = chunks.iter().map(|c| c.len).sum();
let (data, out_bits) = match info.compression {
7 if linear_rgb => (RawData::U16(read_ycbcr_tiles(bytes, &info, raw, mode)?), 14),
32767 if chunks.len() == 1 && strip_len >= (w * h) as u64 && strip_len < (w * h) as u64 * 5 / 4 && mode == Mode::Header => {
chunk_bytes(bytes, &chunks[0]).ok_or_else(|| RawError::Corrupt("raw strip outside file".into()))?;
(RawData::U16(Vec::new()), 14)
@@ -434,12 +503,17 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
};
let default_black = if scale_bits >= 14 { 512.0 } else { 128.0 };
let black = match raw.f64s(BLACK_LEVEL).as_deref() {
Some(v) if linear_rgb && !v.is_empty() => BlackLevel::uniform((v.iter().sum::<f64>() / v.len() as f64) as f32),
Some([a, b, c, d]) => {
BlackLevel { repeat_rows: 2, repeat_cols: 2, values: vec![*a as f32, *b as f32, *c as f32, *d as f32], ..Default::default() }
}
_ => BlackLevel::uniform(sr2_black(bytes, ifd0, tiff.order).filter(|_| scale_bits >= 14).unwrap_or(default_black)),
};
let white = raw.f64(t::WHITE_LEVEL).map(|v| v as f32).filter(|v| *v > 0.0).unwrap_or_else(|| super::white_from_data(samples, scale_bits));
let white = if linear_rgb {
16383.0
} else {
raw.f64(t::WHITE_LEVEL).map(|v| v as f32).filter(|v| *v > 0.0).unwrap_or_else(|| super::white_from_data(samples, scale_bits))
};
let model = ifd0.string(t::MODEL).unwrap_or_default();
let mn = tiff
.exif()
@@ -464,9 +538,9 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
format: RawFormat::Arw,
width: w,
height: h,
cpp: 1,
cpp: if linear_rgb { 3 } else { 1 },
data,
cfa: Some(cfa),
cfa: if linear_rgb { None } else { Some(cfa) },
bits: out_bits,
black,
white: vec![white],
@@ -474,7 +548,9 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
crop,
orientation: Orientation::from_exif(ifd0.u16(t::ORIENTATION).unwrap_or(1)),
color: ColorData::default(),
wb_multipliers: wb,
// Sony's linear YCbCr already carries as-shot WB. Applying the CFA gains again
// makes a neutral surface magenta. WB edits remain relative to this as-shot RGB.
wb_multipliers: if linear_rgb { Some([1.0; 3]) } else { wb },
linearized: false,
opcodes: OpcodeLists::default(),
metadata,
@@ -487,6 +563,38 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
mod tests {
use super::*;
#[test]
fn downsized_lossless_is_linear_rgb_not_cfa() {
use lightcraft_tiff::{IfdBuilder, ImageData, TiffWriter, Value};
let mut raw = IfdBuilder::new();
raw.set(t::MAKE, Value::Ascii("SONY".into()));
raw.set(t::MODEL, Value::Ascii("ILCE-7M4".into()));
raw.set(t::IMAGE_WIDTH, Value::Long(vec![4]));
raw.set(t::IMAGE_LENGTH, Value::Long(vec![4]));
raw.set(t::BITS_PER_SAMPLE, Value::Short(vec![15, 15, 15]));
raw.set(t::SAMPLES_PER_PIXEL, Value::Short(vec![3]));
raw.set(t::PHOTOMETRIC, Value::Short(vec![photometric::YCBCR]));
raw.set(t::COMPRESSION, Value::Short(vec![7]));
raw.set(TONE_CURVE, Value::Short(vec![0; 4]));
raw.set(BLACK_LEVEL, Value::Short(vec![512; 4]));
raw.set(WB_RGGB, Value::Short(vec![2048, 1024, 1024, 2048]));
raw.set_image(ImageData::Tiles { tile_width: 4, tile_height: 4, tiles: vec![ljpeg::tests::fixture_420()] });
let file = TiffWriter::default().write(&[raw]).unwrap();
let header = decode(&file, Mode::Header).unwrap();
let full = decode(&file, Mode::Full).unwrap();
assert_eq!(header.info(), full.info());
assert_eq!((full.width, full.height, full.cpp), (4, 4, 3));
assert!(full.cfa.is_none());
assert_eq!(full.data.len(), 48);
assert_eq!(full.wb_multipliers, Some([1.0; 3]));
let RawData::U16(ref pixels) = full.data else {
panic!("integer ARW");
};
assert_eq!(&pixels[..3], &[1140, 929, 1000]);
let developed = full.develop(crate::Method::Bilinear).unwrap();
assert_eq!((developed.width, developed.height), (4, 4));
}
/// Encode one 16-value set as an ARW2 block (the paper's scheme, used here to test the decoder).
pub(crate) fn encode_block(v: &[u16; 16]) -> [u8; 16] {
let (imax, &max) = v.iter().enumerate().max_by_key(|(i, x)| (**x, usize::MAX - *i)).unwrap();
+956 -133
View File
File diff suppressed because it is too large. Load diff
+145 -7
View File
@@ -16,12 +16,7 @@ fn corpus_root() -> PathBuf {
/// Variants known not to decode yet (see the crate docs): matched against the lower-case file name.
const KNOWN_UNSUPPORTED: &[&str] = &[
"cr3-", // CR3 / CRX (M11.1)
"arw-sony-a7m4-lossless-m", // Sony lossless compressed M/S: subsampled (YCbCr) lossless JPEG
"arw-sony-a7m4-lossless-s", // "
"raf-fuji-xt20-compressed", // Fujifilm compressed RAF
"rw2-panasonic-gh5.", // Panasonic raw format 4 (quantised)
"rw2-panasonic-gx80", // "
"rw2-panasonic-g9-b", // "
"orf-olympus-em", // Olympus compressed ORF
"sraw", // Canon sRAW / mRAW
];
@@ -46,11 +41,12 @@ fn corpus_raw_decodes() {
let t0 = Instant::now();
let preview = embedded_preview(&bytes);
let tp = t0.elapsed().as_secs_f64() * 1e3;
// DNG previews are optional (and some carry only an uncompressed RGB thumbnail); vendor raws always embed a JPEG
// DNG previews are optional (and some carry only an uncompressed RGB thumbnail); vendor raws embed a JPEG,
// except the Panasonic `.RAW` files of 2005–2007
if let Some(p) = &preview {
assert!(p.starts_with(&[0xff, 0xd8]) && p.ends_with(&[0xff, 0xd9]), "{name}: preview is not a JPEG");
} else {
assert_eq!(fmt, RawFormat::Dng, "{name}: no embedded preview");
assert!(fmt == RawFormat::Dng || name.starts_with("raw-panasonic-"), "{name}: no embedded preview");
}
let preview_kb = preview.as_ref().map_or(0, |p| p.len() / 1024);
let t1 = Instant::now();
@@ -289,3 +285,145 @@ fn corpus_sony_pre2017_colour_metadata() {
}
eprintln!("pre-2017 Sony ARW colour metadata checked on {seen} files");
}
/// The embedded JPEG as linear RGB, reduced to `gw × gh` cells.
fn jpeg_cells(jpeg: &[u8], gw: usize, gh: usize) -> Vec<[f64; 3]> {
use zune_core::{bytestream::ZCursor, colorspace::ColorSpace, options::DecoderOptions};
let opts = DecoderOptions::default().jpeg_set_out_colorspace(ColorSpace::RGB);
let mut d = zune_jpeg::JpegDecoder::new_with_options(ZCursor::new(jpeg), opts);
let px = d.decode().unwrap();
let info = d.info().unwrap();
let (w, h) = (info.width as usize, info.height as usize);
let (mut sum, mut n) = (vec![[0f64; 3]; gw * gh], vec![0f64; gw * gh]);
for y in 0..h {
for x in 0..w {
let i = (y * gh / h) * gw + x * gw / w;
for c in 0..3 {
let v = px[(y * w + x) * 3 + c] as f64 / 255.0;
sum[i][c] += if v <= 0.04045 { v / 12.92 } else { ((v + 0.055) / 1.055).powf(2.4) };
}
n[i] += 1.0;
}
}
sum.iter().zip(&n).map(|(s, n)| s.map(|v| v / n)).collect()
}
/// Black-subtracted means of the four sites of the 2×2 cells (anchored at sample (0, 0)) of the active area,
/// reduced to `gw × gh` cells.
fn raw_sites(img: &lightcraft_raw::RawImage, gw: usize, gh: usize) -> Vec<[f64; 4]> {
let lightcraft_raw::RawData::U16(d) = &img.data else { panic!("float data") };
let (w, a, black) = (img.width, img.active_area, img.black.mean() as f64);
let (x0, y0) = ((a.x + 1) & !1, (a.y + 1) & !1);
let (cw, ch) = ((a.x + a.width - x0) / 2, (a.y + a.height - y0) / 2);
let (mut sum, mut n) = (vec![[0f64; 4]; gw * gh], vec![0f64; gw * gh]);
for cy in 0..ch {
for cx in 0..cw {
let i = (cy * gh / ch) * gw + cx * gw / cw;
for site in 0..4 {
sum[i][site] += d[(y0 + 2 * cy + site / 2) * w + x0 + 2 * cx + site % 2] as f64 - black;
}
n[i] += 1.0;
}
}
sum.iter().zip(&n).map(|(s, n)| s.map(|v| v / n)).collect()
}
fn ranks(v: &[f64]) -> Vec<f64> {
let mut idx: Vec<usize> = (0..v.len()).collect();
idx.sort_by(|&a, &b| v[a].total_cmp(&v[b]));
let mut r = vec![0f64; v.len()];
for (rank, &i) in idx.iter().enumerate() {
r[i] = rank as f64;
}
r
}
/// How well the raw's chromaticities (log r/g, log b/g per cell) follow the JPEG's when the mosaic is read with
/// `layout` (colour of each 2×2 site at sample (0, 0)).
fn chroma_agreement(sites: &[[f64; 4]], jpeg: &[[f64; 3]], layout: &[u8]) -> f64 {
let (mut x, mut y) = ([vec![], vec![]], [vec![], vec![]]);
for (s, j) in sites.iter().zip(jpeg) {
let (mut rgb, mut n) = ([0f64; 3], [0f64; 3]);
for (site, &c) in layout.iter().enumerate() {
rgb[c as usize] += s[site];
n[c as usize] += 1.0;
}
let rgb = [rgb[0] / n[0], rgb[1] / n[1], rgb[2] / n[2]];
if rgb.iter().all(|v| *v > 1.0) && j.iter().all(|v| (0.002..0.9).contains(v)) {
for (k, c) in [0, 2].into_iter().enumerate() {
x[k].push((rgb[c] / rgb[1]).ln());
y[k].push((j[c] / j[1]).ln());
}
}
}
(correlation(&x[0], &y[0]) + correlation(&x[1], &y[1])) / 2.0
}
/// Panasonic RW2 / RWL / RAW (`crates/raw/src/vendor/rw2.rs`): one file per raw encoding (formats 2, 4, 5, 6, 7 and
/// 8 — at 12, 14 and 16 bits — and the 16-bit words of the oldest bodies) decodes to the image of the camera's own
/// JPEG, with the black levels, crops and colour-filter layouts the module docs derive and no defect markers left.
#[test]
fn corpus_panasonic_encodings() {
let dir = corpus_root().join("raw");
// (file, bits, mean black level, crop width × height, CFA layout)
let cases = [
("raw-panasonic-fz50.raw", 12, 15.0, (3648, 2736), "BGGR"),
("raw-panasonic-fz8.raw", 12, 15.0, (3072, 2304), "RGGB"),
("rw2-panasonic-fz1000m2-4x3.rw2", 12, 143.0, (4864, 3648), "GBRG"),
("rw2-panasonic-g9-b.rw2", 12, 143.0, (5184, 3888), "RGGB"),
("rw2-panasonic-g9.rw2", 12, 127.5, (5184, 3888), "RGGB"),
("rw2-panasonic-gh1.rw2", 12, 15.0, (4000, 3000), "GBRG"),
("rw2-panasonic-gh5.rw2", 12, 143.5, (5184, 3888), "RGGB"),
("rw2-panasonic-gh5m2.rw2", 12, 144.5, (5184, 3888), "RGGB"),
("rw2-panasonic-gh5s.rw2", 14, 509.5, (3680, 2760), "RGGB"),
("rw2-panasonic-gh6.rw2", 16, 2048.0, (5776, 4336), "RGGB"),
("rw2-panasonic-gx80.rw2", 12, 143.0, (4592, 3448), "BGGR"),
("rw2-panasonic-s1.rw2", 14, 526.0, (6000, 4000), "RGGB"),
("rw2-panasonic-s5-format7.rw2", 14, 512.0, (6000, 4000), "RGGB"),
("rw2-panasonic-s5m2.rw2", 14, 512.0, (6000, 4000), "RGGB"),
("rw2-panasonic-s9.rw2", 12, 128.0, (6000, 4000), "RGGB"),
("rwl-leica-dlux7.rwl", 12, 143.0, (4736, 3552), "BGGR"),
];
let mut seen = 0;
for (name, bits, black, (cw, ch), cfa) in cases {
let Ok(bytes) = std::fs::read(dir.join(name)) else {
eprintln!("skip: {name} absent");
continue;
};
let img = decode(&bytes).unwrap_or_else(|e| panic!("{name}: {e}"));
assert_eq!(img.bits, bits, "{name}: bits");
assert_eq!(img.black.mean(), black, "{name}: black level");
assert_eq!((img.crop.width, img.crop.height), (cw, ch), "{name}: crop");
let layout = img.cfa.clone().unwrap_or_else(|| panic!("{name}: no CFA"));
assert_eq!(layout.name(), cfa, "{name}: CFA layout");
let lightcraft_raw::RawData::U16(d) = &img.data else { panic!("float data") };
let a = img.active_area;
let zeros =
(a.y..a.y + a.height).map(|y| d[y * img.width + a.x..y * img.width + a.x + a.width].iter().filter(|&&v| v == 0).count()).sum::<usize>();
assert_eq!(zeros, 0, "{name}: defect markers left in the active area");
let Some(jpeg) = embedded_preview(&bytes) else {
// the 2005–2007 `.RAW` files have no preview; their flat scenes make the green diagonal unambiguous
assert_eq!(green_on_main_diagonal(&img), cfa.starts_with('G'), "{name}: mosaic statistics disagree with {cfa}");
seen += 1;
continue;
};
// the camera's JPEG shows the whole active area (whatever the crop): the raw's green must rank like it, and
// its colours must follow the JPEG's best with the decoded colour-filter layout
let (gw, gh) = (24, 18);
let (sites, jpeg) = (raw_sites(&img, gw, gh), jpeg_cells(&jpeg, gw, gh));
let green: Vec<f64> = sites.iter().map(|s| (0..4).filter(|&i| layout.pattern[i] == 1).map(|i| s[i]).sum::<f64>()).collect();
let jg: Vec<f64> = jpeg.iter().map(|j| j[1]).collect();
let rho = correlation(&ranks(&green), &ranks(&jg));
let agree = |l: &str| chroma_agreement(&sites, &jpeg, &lightcraft_raw::Cfa::bayer(l).unwrap().pattern);
let best_other = ["RGGB", "GRBG", "GBRG", "BGGR"].into_iter().filter(|l| *l != cfa).map(agree).fold(f64::MIN, f64::max);
let own = agree(cfa);
eprintln!(
"{name:32} {}x{} {bits}-bit: green rank correlation {rho:.3}, colour agreement {own:.3} (others ≤ {best_other:.3})",
img.width, img.height
);
assert!(rho > 0.9, "{name}: rank correlation with the camera JPEG {rho}");
assert!(own > 0.5 && own > best_other, "{name}: colours follow the JPEG better with another layout ({own} vs {best_other})");
seen += 1;
}
eprintln!("Panasonic encodings checked on {seen} files");
}
+16
View File
@@ -453,6 +453,22 @@ fn colour_transform_from_decoded_dng() {
assert!(o.iter().all(|v| (v - 1.0).abs() < 1e-9));
}
/// Issue #212: a NEF converted to DNG came out dark and green. NEF white balance is vendor multipliers
/// (no colour matrix, no `AsShotNeutral`); the DNG must carry them as its as-shot neutral.
#[test]
fn vendor_white_balance_survives_dng_conversion() {
let mut raw = synthetic(16, 16, Some(Cfa::bayer("RGGB").unwrap()), 1);
raw.color = ColorData::default();
raw.wb_multipliers = Some([2.25, 1.0, 1.7421875]);
let before = lightcraft_raw::color::camera_transform(&raw, lightcraft_raw::color::as_shot_white_xy(&raw));
let back = decode(&write_dng(&raw, &DngWriteOptions::default()).unwrap()).unwrap();
let after = lightcraft_raw::color::camera_transform(&back, lightcraft_raw::color::as_shot_white_xy(&back));
for c in 0..3 {
assert!((before.wb[c] - after.wb[c]).abs() < 1e-3, "{:?} vs {:?}", before.wb, after.wb);
}
assert!((after.wb[0] - 2.25).abs() < 1e-3 && (after.wb[2] - 1.7421875).abs() < 1e-3, "{:?}", after.wb);
}
#[test]
fn rejects_unsupported_and_broken() {
let (w, h) = (8, 8);
+85 -2
View File
@@ -1,4 +1,4 @@
//! Malformed vendor raw files (NEF, ARW, PEF, ORF, RW2, CR2, RAF) must decode to an error, never panic.
//! Malformed vendor raw files (NEF, ARW, PEF, ORF, RW2 of every raw format, CR2, RAF) must decode to an error, never panic.
use lightcraft_tiff::tags as t;
use lightcraft_tiff::{ByteOrder, IfdBuilder, ImageData, TiffWriter, Value};
@@ -68,6 +68,66 @@ fn rw2() -> Vec<u8> {
b
}
/// A Panasonic RW2 of raw format `format` (or, with `format` 0, the oldest 16-bit-word kind): sensor size, borders,
/// bit depth, black levels, an in-camera crop and `n` bytes of payload at the end, at `RawDataOffset`. Format 8 also
/// gets two strips covering the payload and a code table.
fn rw2_of(format: u16, bits: u16, n: usize) -> Vec<u8> {
let u16s = |v: &[u16]| Value::Undefined(v.iter().flat_map(|x| x.to_le_bytes()).collect());
let build = |off: u32| {
let mut ifd = IfdBuilder::new();
for (tag, v) in [(2, W as u16), (3, H as u16), (4, 2), (5, 4), (6, H as u16), (7, W as u16), (9, 2), (0x0a, bits), (0x1c, 128), (0x1d, 128)] {
ifd.set(tag, Value::Short(vec![v]));
}
for (tag, v) in [(0x2f, 2u16), (0x30, 6), (0x31, H as u16 - 2), (0x32, W as u16 - 2)] {
ifd.set(tag, Value::Short(vec![v]));
}
if format == 0 {
ifd.set(0x000b, Value::Short(vec![34828]));
ifd.set(t::STRIP_OFFSETS, Value::Long(vec![off]));
} else {
ifd.set(0x002d, Value::Short(vec![format]));
ifd.set(0x0118, Value::Long(vec![off]));
}
if format == 8 {
let table = [
(6, 62),
(7, 126),
(6, 61),
(5, 28),
(4, 12),
(3, 4),
(3, 2),
(2, 0),
(3, 3),
(3, 5),
(4, 13),
(5, 29),
(6, 60),
(8, 254),
(8, 255),
(12, 4094),
(12, 4095),
];
ifd.set(0x0040, u16s(&std::iter::once(17).chain(table.iter().flat_map(|&(l, c)| [l, c])).collect::<Vec<u16>>()));
let half = (n / 2) as u32;
let (a, b) = (off, off + half);
ifd.set(0x0044, u16s(&[2, a as u16, (a >> 16) as u16, b as u16, (b >> 16) as u16]));
ifd.set(0x0045, u16s(&[2, 0, 0, W as u16 / 2, 0]));
ifd.set(0x0046, u16s(&[2, (half * 8) as u16, ((half * 8) >> 16) as u16, (half * 8) as u16, ((half * 8) >> 16) as u16]));
ifd.set(0x0047, u16s(&[2, W as u16 / 2, W as u16 / 2]));
ifd.set(0x0048, u16s(&[2, H as u16, H as u16]));
}
ifd.set(t::MAKE, Value::Ascii("Panasonic".into()));
let mut b = TiffWriter::new(ByteOrder::Little, false).write(&[ifd]).unwrap();
b[..4].copy_from_slice(b"IIU\0");
b
};
let off = build(0).len();
let mut b = build(off as u32);
b.extend((0..n).map(|i| (i * 89 % 253) as u8));
b
}
/// A Pentax PEF: the raw image in IFD0, wide enough for the dark-border trim.
fn pef(compression: u16) -> Vec<u8> {
let (w, h) = (264u32, 4u32);
@@ -181,6 +241,20 @@ fn samples() -> Vec<Vec<u8>> {
cr2(),
raf(),
]
.into_iter()
.chain(rw2_formats())
.collect()
}
/// One RW2 per encoding with a sample decoder of its own (formats 4, 6 at 14 and 12 bits, 8, and 16-bit words).
fn rw2_formats() -> Vec<Vec<u8>> {
vec![
rw2_of(4, 12, (W * H).div_ceil(14) as usize * 16),
rw2_of(6, 14, (W * H).div_ceil(11) as usize * 16),
rw2_of(6, 12, (W * H).div_ceil(14) as usize * 16),
rw2_of(8, 14, (W * H) as usize * 2),
rw2_of(0, 12, (W * H) as usize * 2),
]
}
fn exercise(bytes: &[u8]) {
@@ -208,7 +282,7 @@ proptest! {
#![proptest_config(ProptestConfig { cases: 2000, .. ProptestConfig::default() })]
#[test]
fn mutated_vendor_files_never_panic(kind in 0usize..12, flips in proptest::collection::vec((any::<usize>(), any::<u8>()), 1..16), cut in any::<usize>()) {
fn mutated_vendor_files_never_panic(kind in 0usize..17, flips in proptest::collection::vec((any::<usize>(), any::<u8>()), 1..16), cut in any::<usize>()) {
let mut data = samples().swap_remove(kind);
let n = data.len();
for (i, v) in flips {
@@ -218,3 +292,12 @@ proptest! {
exercise(&data[..keep]);
}
}
/// The RW2 samples decode as they are, so the mutations above reach every format's sample decoder.
#[test]
fn rw2_samples_decode_unmutated() {
for s in &rw2_formats() {
let r = lightcraft_raw::decode(s).unwrap();
assert_eq!((r.width, r.height), (W as usize, H as usize));
}
}
+1 -4
View File
@@ -18,10 +18,7 @@ log = { workspace = true }
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
candle-core = { workspace = true }
candle-nn = { workspace = true }
rustls = { version = "0.23", default-features = false, features = ["std", "tls12"] }
rustls-rustcrypto = "0.0.2-alpha"
webpki-roots = "1"
sha2 = { version = "0.10", default-features = false, features = ["std"] }
lightcraft-fetch = { workspace = true }
[target.'cfg(target_os = "macos")'.dependencies]
candle-core = { workspace = true, features = ["metal"] }
+23 -376
View File
@@ -1,39 +1,13 @@
//! Downloading the SAM 3 model, only when the user asked for it: from an ordered list of mirrors
//! (the next one when a mirror fails), resuming partial files, with connect and stall timeouts,
//! a size cap, progress and cancellation, and checked (exact size, SHA-256 where pinned) before
//! each file is moved into the model folder. Runs on whatever thread calls [`download`] (the
//! engine uses a background thread; the UI thread never waits for the network).
//!
//! Files are written to `<name>.part` next to their final place and renamed when verified, so a
//! model folder never holds a half-written file under a real name, and an interrupted download
//! resumes from where it stopped. A file that fails its check is deleted, never kept.
//! Downloading the SAM 3 model, only when the user asked for it. The downloader itself (mirrors,
//! resuming, checks, the pure-Rust HTTPS client) is the `lightcraft-fetch` crate; this module
//! says what SAM 3 needs and where it may come from.
//!
//! The weights are never part of LightCraft (SAM License, see docs/ai-masks.md); where they are
//! downloaded from is configured by [`DEFAULT_MIRRORS`] and the user's own list.
pub mod http;
pub use lightcraft_fetch::*;
use std::fs::{File, OpenOptions};
use std::io::{Read, Write};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::Duration;
use sha2::{Digest, Sha256};
use http::{HttpError, Limits, Url};
/// One file of the model.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct FileSpec {
pub name: &'static str,
/// Exact size in bytes, when pinned (then the download resumes and is checked against it).
pub size: Option<u64>,
/// SHA-256 (lower-case hex), when pinned.
pub sha256: Option<&'static str>,
/// Largest size accepted when `size` is not pinned.
pub max: u64,
}
use std::path::Path;
/// The `facebook/sam3` checkpoint files [`crate::Sam3::load`] needs.
///
@@ -71,354 +45,27 @@ pub const DEFAULT_MIRRORS: &[&str] = &[];
/// tried before the defaults.
pub const MIRRORS_ENV: &str = "LIGHTCRAFT_SAM3_MIRRORS";
/// Most mirrors used (a hostile list can't make a download loop forever).
const MAX_MIRRORS: usize = 16;
/// Redirects followed per request.
const MAX_REDIRECTS: usize = 8;
/// The mirrors to try, in order: the environment variable's, then the mirrors file's (one base
/// URL per line, `#` comments), then [`DEFAULT_MIRRORS`]; duplicates and unusable URLs dropped.
/// The mirrors to try for SAM 3, in order: the environment variable's, then the mirrors file's
/// (one base URL per line, `#` comments), then [`DEFAULT_MIRRORS`].
pub fn mirrors(env: Option<&str>, file: Option<&Path>) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
let mut add = |s: &str| {
let s = s.trim().trim_end_matches('/');
if !s.is_empty() && Url::parse(s).is_ok() && !out.iter().any(|m| m == s) && out.len() < MAX_MIRRORS {
out.push(s.to_string());
}
};
if let Some(env) = env {
env.split([',', ' ', '\n', '\t']).for_each(&mut add);
}
// a small text file; anything larger isn't a mirror list
if let Some(text) = file.and_then(|f| std::fs::metadata(f).ok().filter(|m| m.len() < 64 * 1024).and_then(|_| std::fs::read_to_string(f).ok())) {
text.lines().map(|l| l.split('#').next().unwrap_or_default()).for_each(&mut add);
}
DEFAULT_MIRRORS.iter().for_each(|m| add(m));
out
lightcraft_fetch::mirrors(env, file, DEFAULT_MIRRORS)
}
/// Download settings.
#[derive(Clone, Debug)]
pub struct Options {
/// Longest wait to connect to a mirror.
pub connect_timeout: Duration,
/// Longest wait for the next bytes once connected.
pub stall_timeout: Duration,
/// Tries per mirror and file (each resumes where the last one stopped).
pub attempts: u32,
}
impl Default for Options {
fn default() -> Self {
Options { connect_timeout: Duration::from_secs(15), stall_timeout: Duration::from_secs(30), attempts: 3 }
}
}
/// Where a download is.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct Progress {
pub file: String,
/// Bytes of all files on disk so far, and the total expected (pinned sizes + announced ones).
pub done: u64,
pub total: u64,
/// The mirror in use (without query).
pub mirror: String,
}
/// Why a download failed.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum DownloadError {
/// No mirror is configured.
NoMirrors,
Cancelled,
/// A local file problem (folder not writable, disk full…).
Disk(String),
/// Every mirror failed for `file`; the reason for each.
AllMirrorsFailed {
file: String,
errors: Vec<String>,
},
}
impl std::fmt::Display for DownloadError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
DownloadError::NoMirrors => write!(
f,
"no download location is configured for the SAM 3 model in this build (set {MIRRORS_ENV}, or put the files in the model folder yourself; see docs/ai-masks.md)"
),
DownloadError::Cancelled => write!(f, "the download was cancelled"),
DownloadError::Disk(e) => write!(f, "{e}"),
DownloadError::AllMirrorsFailed { file, errors } => write!(f, "could not download {file}: {}", errors.join("; ")),
}
}
}
/// One mirror's failure for one file; `Retry` resumes on the same mirror.
enum Fail {
Retry(String),
NextMirror(String),
Cancelled,
Disk(String),
}
/// Download `files` into `dir` from `mirrors` (in order; see [`mirrors`]). Files already there
/// and verified are kept. `progress` is called as bytes arrive (often: throttle in it).
pub fn download(
files: &[FileSpec],
mirrors: &[String],
dir: &Path,
opts: &Options,
cancel: &AtomicBool,
progress: &mut dyn FnMut(&Progress),
) -> Result<(), DownloadError> {
std::fs::create_dir_all(dir).map_err(|e| DownloadError::Disk(format!("can't create {}: {e}", dir.display())))?;
// what's left to fetch
let mut todo = Vec::new();
for f in files {
if !verified(f, &dir.join(f.name))? {
todo.push(f);
}
}
if todo.is_empty() {
return Ok(());
}
if mirrors.is_empty() {
return Err(DownloadError::NoMirrors);
}
let total: u64 = files.iter().map(|f| f.size.unwrap_or(0)).sum();
let mut before: u64 = files.iter().filter(|f| !todo.contains(f)).filter_map(|f| f.size).sum();
for f in todo {
let mut errors = Vec::new();
let mut ok = false;
'mirrors: for m in mirrors {
let url = format!("{m}/{}", f.name);
for _ in 0..opts.attempts.max(1) {
if cancel.load(Ordering::Relaxed) {
return Err(DownloadError::Cancelled);
}
let mut report = |n: u64, announced: u64| {
let mirror = Url::parse(m).map(|u| u.display()).unwrap_or_default();
progress(&Progress {
file: f.name.to_string(),
done: before.saturating_add(n),
total: total.max(before.saturating_add(announced)),
mirror,
})
};
match fetch_file(f, &url, dir, opts, cancel, &mut report) {
Ok(()) => {
ok = true;
break 'mirrors;
}
Err(Fail::Cancelled) => return Err(DownloadError::Cancelled),
Err(Fail::Disk(e)) => return Err(DownloadError::Disk(e)),
Err(Fail::Retry(e)) => {
log::warn!("SAM 3 download of {url}: {e} (retrying)");
errors.push(format!("{}: {e}", short(m)));
}
Err(Fail::NextMirror(e)) => {
log::warn!("SAM 3 download of {url}: {e}");
errors.push(format!("{}: {e}", short(m)));
continue 'mirrors;
}
}
}
}
if !ok {
return Err(DownloadError::AllMirrorsFailed { file: f.name.to_string(), errors });
}
before = before.saturating_add(std::fs::metadata(dir.join(f.name)).map(|m| m.len()).unwrap_or(0));
}
Ok(())
}
/// A mirror for messages (no query string: it may hold a token).
fn short(m: &str) -> String {
Url::parse(m).map(|u| u.display()).unwrap_or_else(|_| "mirror".into())
}
/// Whether `path` is already the right file (exact size and hash when pinned).
fn verified(f: &FileSpec, path: &Path) -> Result<bool, DownloadError> {
let Ok(meta) = std::fs::metadata(path) else { return Ok(false) };
if !meta.is_file() || meta.len() == 0 {
return Ok(false);
}
if f.size.is_some_and(|s| s != meta.len()) || meta.len() > f.max {
return Ok(false);
}
match f.sha256 {
Some(want) => Ok(sha256_file(path).map_err(DownloadError::Disk)? == want),
None => Ok(true),
}
}
fn sha256_file(path: &Path) -> Result<String, String> {
let mut file = File::open(path).map_err(|e| format!("{}: {e}", path.display()))?;
let mut h = Sha256::new();
let mut buf = vec![0u8; 1 << 20];
loop {
let n = file.read(&mut buf).map_err(|e| format!("{}: {e}", path.display()))?;
if n == 0 {
break;
}
h.update(buf.get(..n).unwrap_or_default());
}
Ok(h.finalize().iter().map(|b| format!("{b:02x}")).collect())
}
fn part_path(dir: &Path, name: &str) -> PathBuf {
dir.join(format!("{name}.part"))
}
/// Fetch one file from one mirror into `<name>.part` (resuming it when its size is pinned),
/// check it and move it into place.
fn fetch_file(f: &FileSpec, url: &str, dir: &Path, opts: &Options, cancel: &AtomicBool, report: &mut dyn FnMut(u64, u64)) -> Result<(), Fail> {
let part = part_path(dir, f.name);
let limit = f.size.unwrap_or(f.max);
let disk = |e: std::io::Error| Fail::Disk(format!("{}: {e}", part.display()));
// resume only a pinned file (an unpinned one can't be checked after resuming)
let mut have = match std::fs::metadata(&part) {
Ok(m) if f.size.is_some() && m.len() <= limit => m.len(),
Ok(_) => {
std::fs::remove_file(&part).map_err(disk)?;
0
}
Err(_) => 0,
};
let fetched = f.size != Some(have) || have == 0;
if fetched {
let limits = Limits { connect: opts.connect_timeout, stall: opts.stall_timeout, cancel };
let mut url = Url::parse(url).map_err(|e| Fail::NextMirror(e.to_string()))?;
let origin = url.host.clone();
let mut redirects = 0;
let mut resp = loop {
let mut headers = vec![];
if have > 0 {
headers.push(("Range", format!("bytes={have}-")));
}
// a token for the configured host only (never sent on to a redirect's host)
if let Ok(t) = std::env::var("LIGHTCRAFT_SAM3_TOKEN")
&& !t.trim().is_empty()
&& url.tls
&& url.host == origin
{
headers.push(("Authorization", format!("Bearer {}", t.trim())));
}
let r = http::get(&url, &headers, &limits).map_err(http_fail)?;
match r.status {
301 | 302 | 303 | 307 | 308 => {
redirects += 1;
if redirects > MAX_REDIRECTS {
return Err(Fail::NextMirror("too many redirects".into()));
}
let loc = r.header("location").ok_or_else(|| Fail::NextMirror("redirect without a location".into()))?;
url = url.join(loc).map_err(|e| Fail::NextMirror(e.to_string()))?;
}
_ => break r,
}
};
let mut file = match resp.status {
200 => {
// the whole file (from the start, also when a range was asked for)
if resp.content_length().is_some_and(|n| n > limit || f.size.is_some_and(|s| s != n)) {
return Err(Fail::NextMirror(format!("the server's file has the wrong size ({:?} bytes)", resp.content_length())));
}
have = 0;
File::create(&part).map_err(disk)?
}
206 if have > 0 => {
let range = resp.header("content-range").unwrap_or_default();
let (start, total) = parse_content_range(range).ok_or_else(|| Fail::NextMirror(format!("bad Content-Range `{range}`")))?;
if start != have || total.is_some_and(|t| Some(t) != f.size) {
return Err(Fail::NextMirror(format!("the server resumed at the wrong place ({range})")));
}
OpenOptions::new().append(true).open(&part).map_err(disk)?
}
416 => {
// the partial file doesn't fit what the server has: start over
std::fs::remove_file(&part).map_err(disk)?;
return Err(Fail::Retry("the partial download didn't match; starting over".into()));
}
401 | 403 => return Err(Fail::NextMirror(format!("access denied (HTTP {})", resp.status))),
404 | 410 => return Err(Fail::NextMirror(format!("not found (HTTP {})", resp.status))),
s if s >= 500 => return Err(Fail::Retry(format!("server error (HTTP {s})"))),
s => return Err(Fail::NextMirror(format!("HTTP {s}"))),
};
let announced = f.size.unwrap_or_else(|| resp.content_length().unwrap_or(0));
report(have, announced);
let mut buf = vec![0u8; 256 * 1024];
loop {
let n = match resp.read(&mut buf, &limits) {
Ok(n) => n,
Err(e) => {
let _ = file.flush();
if f.size.is_none() {
drop(file);
let _ = std::fs::remove_file(&part);
}
return Err(http_fail(e));
}
};
if n == 0 {
break;
}
if have.saturating_add(n as u64) > limit {
drop(file);
let _ = std::fs::remove_file(&part);
return Err(Fail::NextMirror(format!("the server sent more than the expected {limit} bytes")));
}
file.write_all(buf.get(..n).unwrap_or_default()).map_err(disk)?;
have = have.saturating_add(n as u64);
report(have, announced);
}
file.sync_all().map_err(disk)?;
}
// checks: exact size, hash
let len = std::fs::metadata(&part).map_err(disk)?.len();
if f.size.is_some_and(|s| s != len) {
return Err(Fail::Retry(format!("incomplete ({len} of {} bytes)", f.size.unwrap_or(0))));
}
if len == 0 {
let _ = std::fs::remove_file(&part);
return Err(Fail::NextMirror("the server sent an empty file".into()));
}
if let Some(want) = f.sha256 {
let got = sha256_file(&part).map_err(Fail::Disk)?;
if got != want {
std::fs::remove_file(&part).map_err(disk)?;
let why = format!("the file is damaged or not the official one (SHA-256 {got})");
// a damaged partial file from an earlier run isn't this mirror's fault
return Err(if fetched { Fail::NextMirror(why) } else { Fail::Retry(why) });
}
}
std::fs::rename(&part, dir.join(f.name)).map_err(disk)?;
Ok(())
}
fn http_fail(e: HttpError) -> Fail {
match e {
HttpError::Cancelled => Fail::Cancelled,
HttpError::Stalled | HttpError::Io(_) | HttpError::Protocol(_) => Fail::Retry(e.to_string()),
HttpError::BadUrl(_) | HttpError::Connect(_) | HttpError::Tls(_) => Fail::NextMirror(e.to_string()),
}
}
/// `bytes a-b/total` → (a, total) (`*` total → None).
fn parse_content_range(v: &str) -> Option<(u64, Option<u64>)> {
let v = v.trim().strip_prefix("bytes")?.trim();
let (range, total) = v.split_once('/')?;
let (a, b) = range.split_once('-')?;
let (a, b) = (a.trim().parse::<u64>().ok()?, b.trim().parse::<u64>().ok()?);
if b < a {
return None;
}
let total = match total.trim() {
"*" => None,
t => Some(t.parse::<u64>().ok()?),
};
Some((a, total))
/// What to tell the user when there is no mirror for the SAM 3 model.
pub fn no_mirrors_message() -> String {
format!(
"no download location is configured for the SAM 3 model in this build (set {MIRRORS_ENV}, or put the files in the model folder yourself; see docs/ai-masks.md)"
)
}
#[cfg(test)]
mod tests;
mod tests {
use super::*;
#[test]
fn sam3_mirrors_come_from_the_environment_the_file_and_the_defaults() {
let m = mirrors(Some("https://a.example/x/"), Some(Path::new("/nonexistent/mirrors.txt")));
assert_eq!(m.len(), 1 + DEFAULT_MIRRORS.len());
assert!(no_mirrors_message().contains(MIRRORS_ENV));
}
}
+1
View File
@@ -29,3 +29,4 @@ lightcraft-codecs = { workspace = true }
[build-dependencies]
serde_json = { workspace = true }
toml = "1"
+108
View File
@@ -10,6 +10,12 @@
//! Every catalog must carry the same messages: a `tr_format!` call whose message is in no catalog
//! does not compile, so a message missing from one language would otherwise go unnoticed until it
//! shows English.
//!
//! It also bakes the contributor credits into the binary as static Rust tables (`$OUT_DIR/credits.rs`,
//! included by `src/credits.rs`; craftrules `standards/contributors.md`): commit stats from
//! `contributors/contributors.json` (generated by craftrules `scripts/contributors.py`) and the names
//! people chose for themselves from `contributors/people.toml`. Nothing is read at run time. A
//! missing or malformed file gives empty tables (or usernames only) and a warning, never a failed build.
use std::{
collections::{BTreeMap, BTreeSet},
env, fs,
@@ -22,6 +28,8 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
println!("cargo:rerun-if-changed=locales");
let out = PathBuf::from(env::var("OUT_DIR")?);
fs::write(out.join("tr-formats.rs"), generate()?)?;
let (json, people) = credits::load();
fs::write(out.join("credits.rs"), credits::generate(&json, &people))?;
Ok(())
}
@@ -98,3 +106,103 @@ fn generate() -> Result<String, Box<dyn std::error::Error>> {
));
Ok(source)
}
/// The About window's credits tables.
mod credits {
use std::fmt::Write as _;
use serde_json::Value;
/// `contributors/contributors.json` and `contributors/people.toml`; `Null`/empty (with a build
/// warning) when missing or malformed. Names come only from people.toml entries the JSON marks
/// verified (`people_entry`).
pub fn load() -> (Value, toml::Table) {
let dir = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../contributors");
let json = read(&dir.join("contributors.json"), "the About window shows no contributors", |t| {
serde_json::from_str::<Value>(t).map_err(|e| e.to_string())
})
.unwrap_or(Value::Null);
let people =
read(&dir.join("people.toml"), "contributors are shown by username only", |t| t.parse::<toml::Table>().map_err(|e| e.to_string()))
.unwrap_or_default();
(json, people)
}
fn read<T>(path: &std::path::Path, fallback: &str, parse: impl Fn(&str) -> Result<T, String>) -> Option<T> {
println!("cargo:rerun-if-changed={}", path.display());
let r = std::fs::read_to_string(path).map_err(|e| e.to_string()).and_then(|t| parse(&t));
if let Err(e) = &r {
println!("cargo:warning={}: {e}; {fallback}", path.display());
}
r.ok()
}
/// A non-empty string field of someone's people.toml entry, as a Rust `Option<&str>` literal.
fn name(v: Option<&str>) -> String {
match v.map(str::trim).filter(|s| !s.is_empty()) {
Some(x) => format!("Some({x:?})"),
None => "None".into(),
}
}
fn s(v: &Value, k: &str) -> String {
format!("{:?}", v.get(k).and_then(Value::as_str).unwrap_or(""))
}
fn n(v: &Value, k: &str) -> u64 {
v.get(k).and_then(Value::as_u64).unwrap_or(0)
}
fn rows(json: &Value, k: &str) -> Vec<Value> {
json.get(k).and_then(Value::as_array).cloned().unwrap_or_default()
}
pub fn generate(json: &Value, people: &toml::Table) -> String {
let mut o = String::new();
let _ = writeln!(o, "pub static TOTAL_COMMITS: u64 = {};", n(json, "commits"));
let _ = writeln!(o, "pub static CONTRIBUTORS: &[Contributor] = &[");
for c in rows(json, "contributors") {
let login = c.get("login").and_then(Value::as_str).unwrap_or("");
if login.is_empty() {
continue;
}
// Names only from a verified, self-chosen people.toml entry.
let entry = people.get("people").and_then(|p| p.get(login)).filter(|_| c.get("people_entry").and_then(Value::as_bool) == Some(true));
let field = |k: &str| entry.and_then(|e| e.get(k)).and_then(toml::Value::as_str);
let synced = entry.and_then(|e| e.get("sync_github_name")).and_then(toml::Value::as_bool) == Some(true);
let display = field("display_name")
.filter(|s| !s.trim().is_empty())
.or_else(|| if synced { c.get("github_name").and_then(Value::as_str) } else { None });
let _ = writeln!(
o,
" Contributor {{ login: {login:?}, display_name: {}, real_name: {}, prs: {}, commits: {}, lines_added: {}, lines_deleted: {}, binary_added: {}, binary_deleted: {}, first_commit: {}, last_commit: {} }},",
name(display),
name(field("real_name")),
n(&c, "prs"),
n(&c, "commits"),
n(&c, "lines_added"),
n(&c, "lines_deleted"),
n(&c, "binary_added"),
n(&c, "binary_deleted"),
s(&c, "first_commit"),
s(&c, "last_commit"),
);
}
let _ = writeln!(o, "];");
let _ = writeln!(o, "pub static MODELS: &[Model] = &[");
for m in rows(json, "models") {
let _ = writeln!(
o,
" Model {{ company: {}, model: {}, version: {}, commits: {}, lines_added: {}, lines_deleted: {} }},",
s(&m, "company"),
s(&m, "model"),
s(&m, "version"),
n(&m, "commits"),
n(&m, "lines_added"),
n(&m, "lines_deleted"),
);
}
let _ = writeln!(o, "];");
o
}
}
@@ -0,0 +1,341 @@
use lightcraft_catalog::{Op, Photo, PhotoId, Source};
use lightcraft_develop::{BrushStroke, DevelopSettings, Mask, MaskComponent, MaskShape};
use lightcraft_engine::Session;
use lightcraft_engine::media::{RenderJob, RenderResult};
use lightcraft_ui_egui::{
LightcraftApp, Services,
headless::Headless,
panels::grid::request_thumb,
render::{RenderOffload, Slot, Tex},
};
use std::alloc::{GlobalAlloc, Layout, System};
use std::cell::Cell;
use std::{
hint::black_box,
sync::Arc,
time::{Duration, Instant},
};
/// Counts allocations made by the current thread (the UI thread is the one measured).
struct Counting;
thread_local! {
static ALLOCS: Cell<u64> = const { Cell::new(0) };
static BYTES: Cell<u64> = const { Cell::new(0) };
}
fn count(size: usize) {
let _ = ALLOCS.try_with(|n| n.set(n.get() + 1));
let _ = BYTES.try_with(|n| n.set(n.get() + size as u64));
}
// SAFETY: every call is forwarded unchanged to the system allocator; the counters are plain
// thread-locals that never allocate. Dev-only example.
#[allow(unsafe_code)]
unsafe impl GlobalAlloc for Counting {
unsafe fn alloc(&self, l: Layout) -> *mut u8 {
count(l.size());
// SAFETY: same contract as ours.
unsafe { System.alloc(l) }
}
unsafe fn alloc_zeroed(&self, l: Layout) -> *mut u8 {
count(l.size());
// SAFETY: same contract as ours.
unsafe { System.alloc_zeroed(l) }
}
unsafe fn realloc(&self, p: *mut u8, l: Layout, new: usize) -> *mut u8 {
count(new);
// SAFETY: same contract as ours.
unsafe { System.realloc(p, l, new) }
}
unsafe fn dealloc(&self, p: *mut u8, l: Layout) {
// SAFETY: same contract as ours.
unsafe { System.dealloc(p, l) }
}
}
#[global_allocator]
static A: Counting = Counting;
fn allocs() -> (u64, u64) {
(ALLOCS.with(Cell::get), BYTES.with(Cell::get))
}
struct Blocked;
impl RenderOffload for Blocked {
fn try_start(&mut self, _: Slot, job: RenderJob) -> Option<RenderJob> {
Some(job)
}
fn finished(&mut self) -> Vec<(Slot, RenderResult, f64)> {
vec![]
}
}
fn settings(points: usize) -> DevelopSettings {
let mut s = DevelopSettings::default();
if points > 0 {
let stroke = BrushStroke {
points: (0..points).map(|i| lightcraft_geom::Point::new((i % 100) as f64 / 100.0, (i / 100) as f64 / 100.0)).collect(),
..Default::default()
};
s.masks.push(Mask {
components: vec![MaskComponent { name: None, op: Default::default(), invert: false, shape: MaskShape::Brush { strokes: vec![stroke] } }],
..Default::default()
});
}
s
}
fn summary(name: &str, mut times: Vec<f64>) {
times.sort_by(f64::total_cmp);
println!("{name},median_ms={:.6},p95_ms={:.6}", times[times.len() / 2], times[times.len() * 95 / 100]);
}
fn frames() -> usize {
std::env::var("FRAMES").ok().and_then(|s| s.parse().ok()).unwrap_or(300).max(1)
}
fn micro(points: usize, pending: bool) {
let frames = frames();
let n = 200;
let mut session = Session::new();
for i in 1..=n {
let mut p =
Photo::new(PhotoId(i), Source::File { path: format!("N:/audit-only/IMG_{i}.jpg") }, "sample.jpg", "JPEG", 6000, 4000, "2026-01-01");
p.develop = Arc::new(settings(points));
p.content_hash = Some(format!("{i:032x}"));
session.catalog.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
}
let mut app = LightcraftApp::new(session, Services::default());
app.renderer.set_offload(Box::new(Blocked));
let ctx = egui::Context::default();
let texture = ctx.load_texture("audit", egui::ColorImage::new([1, 1], vec![egui::Color32::GRAY]), Default::default());
for i in 1..=n {
let id = PhotoId(i);
if !pending {
let key = app.session.thumb_job(id, 384).unwrap().key;
app.renderer.textures.insert(
Slot::Thumb(id),
Tex { key, photo: id, tex: texture.clone(), size: [1, 1], histogram: None, ms: 0.0, quick: None, pixels: None },
);
}
request_thumb(&mut app, id, 384, 10);
}
let mut times = Vec::with_capacity(frames);
let (a0, b0) = allocs();
for _ in 0..frames {
let t = Instant::now();
for i in 1..=n {
request_thumb(black_box(&mut app), PhotoId(i), 384, 10);
}
times.push(t.elapsed().as_secs_f64() * 1000.0);
}
let (a1, b1) = allocs();
println!("micro,points={points},pending={pending},allocs_per_frame={},bytes_per_frame={}", (a1 - a0) / frames as u64, (b1 - b0) / frames as u64);
summary(&format!("micro,points={points},pending={pending},cells={n}"), times);
// The draw fast path must notice a develop edit and a size-bucket change.
let mut edit = settings(points);
edit.light.exposure = 1.0;
app.session.catalog.apply(Op::SetDevelop { id: PhotoId(1), settings: Arc::new(edit), label: "audit edit".into(), edited: None }).unwrap();
request_thumb(&mut app, PhotoId(1), 384, 10);
assert_eq!(app.renderer.wanted(Slot::Thumb(PhotoId(1))), Some(app.session.thumb_job(PhotoId(1), 384).unwrap().key));
request_thumb(&mut app, PhotoId(1), 512, 10);
assert_eq!(app.renderer.wanted(Slot::Thumb(PhotoId(1))), Some(app.session.thumb_job(PhotoId(1), 512).unwrap().key));
}
fn collect(root: &std::path::Path, ext: &str, paths: &mut Vec<std::path::PathBuf>) {
for e in std::fs::read_dir(root).unwrap() {
let p = e.unwrap().path();
if p.is_dir() {
collect(&p, ext, paths);
} else if p.extension().is_some_and(|x| x.eq_ignore_ascii_case(ext)) {
paths.push(p);
}
}
}
fn verify_texture(h: &Headless, id: PhotoId) {
let p = h.app.session.catalog.photo(id).unwrap();
let tex = h.app.renderer.thumb(id).unwrap();
// Justified rows can choose different buckets even with the same slider value.
// Verify the actual displayed resolution independently of candidate request/hash helpers.
let edge = tex.size[0].max(tex.size[1]);
assert!([128, 256, 384, 512].contains(&edge), "thumbnail must use a supported size bucket");
let Source::File { path } = &p.source else { unreachable!() };
let (source, info) = h.app.session.media.file_loader.as_ref().unwrap()(path, 512).unwrap();
println!("decode,path={path},raw={},preview_only={:?}", info.raw, p.preview_only);
if std::env::var_os("REQUIRE_RAW").is_some() {
assert!(p.preview_only.is_none(), "embedded-preview fallback does not validate RAW decoding");
assert!(info.raw, "fresh loader must return decoded RAW data");
}
let req = lightcraft_engine::pipeline::RenderRequest { apply_crop: true, ..lightcraft_engine::pipeline::RenderRequest::fit(edge, edge) };
// Fresh direct CPU render: no candidate job/hash/source/preview cache helpers.
let expected = lightcraft_engine::pipeline::render(&source, &info, &p.develop, &req).image;
assert_eq!(tex.size, [expected.width, expected.height]);
assert_eq!(tex.pixels.as_ref().unwrap().pixels.iter().flat_map(|c| c.to_array()).collect::<Vec<_>>(), expected.as_bytes());
}
fn real() {
let frames = frames();
let mut paths = vec![];
let ext = std::env::var("PHOTO_EXT").unwrap_or_else(|_| "jpg".into());
let Ok(root) = std::env::var("PHOTO_ROOT") else {
eprintln!("REAL_PHOTOS needs PHOTO_ROOT, a folder of photos to sample");
return;
};
collect(std::path::Path::new(&root), &ext, &mut paths);
paths.sort();
let offset = std::env::var("PHOTO_OFFSET").ok().and_then(|s| s.parse::<usize>().ok()).unwrap_or(0);
let limit = std::env::var("PHOTO_LIMIT").ok().and_then(|s| s.parse::<usize>().ok()).unwrap_or(4).clamp(1, 4);
paths = paths.into_iter().skip(offset).take(limit).collect();
assert!(!paths.is_empty(), "selected photo batch is empty");
println!("batch,extension={ext},offset={offset},limit={limit},photos={}", paths.len());
let mut session = Session::new().with_fs();
for (i, path) in paths.iter().enumerate() {
let path = path.to_string_lossy().into_owned();
let probe = session.media.file_probe.as_ref().unwrap()(&path).unwrap();
let mut p =
Photo::new(PhotoId(i as u64 + 1), Source::File { path: path.clone() }, "photo", &probe.format, probe.width, probe.height, "2026-01-01");
p.file_size = probe.file_size;
p.kind = probe.kind;
p.meta = probe.meta;
p.as_shot_wb = probe.as_shot_wb;
p.embedded_lens = probe.embedded_lens;
p.content_hash = probe.content_hash;
p.preview_only = probe.preview_only;
lightcraft_engine::import::apply_import_defaults(&session, &mut p);
session.catalog.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
}
let mut h = Headless::new(LightcraftApp::new(session, Services { png: None, ..Default::default() }), [1400.0, 900.0], 1.0);
h.app.renderer = lightcraft_ui_egui::render::Renderer::with_worker_threads(1);
h.app.renderer.keep_pixels = true;
h.app.ui.view = lightcraft_ui_egui::state::ViewMode::PhotoGrid;
h.app.ui.thumb_size = if ext == "rw2" { 160.0 } else { 100.0 };
let mut loading_frame = vec![];
let mut loading_grid = vec![];
let mut busy = 0;
for _ in 0..120 {
let t = Instant::now();
h.step();
loading_frame.push(t.elapsed().as_secs_f64() * 1000.0);
loading_grid.push(h.app.caches.grid_stats.last_show.as_secs_f64() * 1000.0);
busy += usize::from(h.app.renderer.in_flight() > 0);
}
summary("loading,frame", loading_frame);
summary("loading,grid", loading_grid);
println!("loading,frames_with_pending={busy}");
assert!(h.settle(Duration::from_secs(120)), "real library did not settle");
for i in 1..=paths.len() {
request_thumb(&mut h.app, PhotoId(i as u64), 256, 20);
}
assert!(h.settle(Duration::from_secs(120)));
println!("real,photos={},textures={},in_flight={}", paths.len(), h.app.renderer.thumb_textures(), h.app.renderer.in_flight());
let stats = h.app.caches.grid_stats.clone();
let mut frame = Vec::with_capacity(frames);
let mut grid = Vec::with_capacity(frames);
let (a0, b0) = allocs();
for _ in 0..frames {
let t = Instant::now();
h.step();
frame.push(t.elapsed().as_secs_f64() * 1000.0);
grid.push(h.app.caches.grid_stats.last_show.as_secs_f64() * 1000.0);
}
let (a1, b1) = allocs();
println!("real,allocs_per_frame={},bytes_per_frame={}", (a1 - a0) / frames as u64, (b1 - b0) / frames as u64);
summary("real,frame", frame);
summary("real,grid", grid);
println!("real,cells_per_frame={}", (h.app.caches.grid_stats.cells_visited - stats.cells_visited) / frames as u64);
assert!(h.settle(Duration::from_secs(120)), "grid's displayed thumbnail bucket must settle before verification");
let mut checks: Vec<_> = h
.app
.renderer
.textures
.iter()
.filter_map(|(slot, t)| {
if let Slot::Thumb(id) = slot {
let mut hash = 0xcbf29ce484222325u64;
for color in &t.pixels.as_ref().unwrap().pixels {
for b in color.to_array() {
hash = (hash ^ b as u64).wrapping_mul(0x100000001b3);
}
}
Some((id.0, t.key, t.size, hash))
} else {
None
}
})
.collect();
checks.sort_by_key(|x| x.0);
let mut hash = 0xcbf29ce484222325u64;
for b in format!("{checks:?}").bytes() {
hash = (hash ^ b as u64).wrapping_mul(0x100000001b3);
}
println!("real,verified_textures={},keys_sizes_pixels_checksum={hash:016x}", checks.len());
assert_eq!(checks.len(), paths.len(), "every original must render successfully");
if std::env::var_os("VERIFY_PIXELS").is_some() {
for i in 1..=paths.len() {
verify_texture(&h, PhotoId(i as u64));
}
println!("oracle,exact_fresh_pixel_matches={}", paths.len());
}
let mut scroll_frame = vec![];
let mut scroll_grid = vec![];
let start = h.app.grid_scroll;
let mut max_scroll = 0.0f32;
for i in 0..frames {
h.app.synthetic.push(egui::Event::PointerMoved(egui::pos2(800.0, 400.0)));
h.app.synthetic.push(egui::Event::MouseWheel {
unit: egui::MouseWheelUnit::Point,
delta: egui::vec2(0.0, if (i / 45) % 2 == 0 { -12.0 } else { 12.0 }),
phase: egui::TouchPhase::Move,
modifiers: Default::default(),
});
let t = Instant::now();
h.step();
scroll_frame.push(t.elapsed().as_secs_f64() * 1000.0);
scroll_grid.push(h.app.caches.grid_stats.last_show.as_secs_f64() * 1000.0);
max_scroll = max_scroll.max(h.app.grid_scroll.unwrap_or(0.0));
}
summary("scroll,frame", scroll_frame);
summary("scroll,grid", scroll_grid);
println!("scroll,start={start:?},end={:?},max={max_scroll}", h.app.grid_scroll);
if max_scroll == 0.0 {
println!("scroll,not_applicable=bounded_batch_fits_view");
}
let id = PhotoId(1);
let old = h.app.renderer.thumb(id).unwrap().pixels.as_ref().unwrap().clone();
let mut edit = (*h.app.session.catalog.photo(id).unwrap().develop).clone();
edit.light.exposure += 0.5;
let undo = h.app.session.catalog.apply(Op::SetDevelop { id, settings: Arc::new(edit), label: "audit".into(), edited: None }).unwrap();
request_thumb(&mut h.app, id, 256, 20);
assert!(h.settle(Duration::from_secs(120)));
let edited = h.app.renderer.thumb(id).unwrap().pixels.as_ref().unwrap().clone();
assert_ne!(old.pixels, edited.pixels, "edit must change actual rendered pixels");
if std::env::var_os("VERIFY_PIXELS").is_some() {
verify_texture(&h, id);
}
let mut hash = 0xcbf29ce484222325u64;
for c in &edited.pixels {
for b in c.to_array() {
hash = (hash ^ b as u64).wrapping_mul(0x100000001b3);
}
}
println!("edit,exposure_plus_half_stop,pixels_checksum={hash:016x}");
h.app.session.catalog.apply(undo).unwrap();
request_thumb(&mut h.app, id, 256, 20);
assert!(h.settle(Duration::from_secs(120)));
assert_eq!(old.pixels, h.app.renderer.thumb(id).unwrap().pixels.as_ref().unwrap().pixels, "undo restores pixels");
if std::env::var_os("VERIFY_PIXELS").is_some() {
verify_texture(&h, id);
println!("oracle,edit_and_undo_match_fresh_pixels=true");
}
println!("undo,pixels_restored=true");
}
fn main() {
println!("mode={}", std::env::var("THUMB_AUDIT_MODE").unwrap_or_else(|_| "baseline".into()));
if std::env::var_os("SKIP_MICRO").is_none() {
for points in [0, 200, 1000] {
for pending in [false, true] {
micro(points, pending);
}
}
}
if std::env::var_os("REAL_PHOTOS").is_some() {
real();
}
}
+2
View File
@@ -287,6 +287,8 @@
"File Name": "ファイル名",
"File Path": "ファイルの場所",
"File Size": "ファイルサイズ",
"Random": "ランダム",
"Reshuffle": "シャッフルし直す",
"File name": "ファイル名",
"Fill Edges": "境界を補完",
"Filmstrip": "フィルムストリップ",
+122
View File
@@ -0,0 +1,122 @@
{
" ({} in Recently Deleted)": " ({} em Excluídos recentemente)",
" · {skipped} skipped (file exists)": " · {skipped} ignorada(s) (arquivo existente)",
" · {} duplicate{} skipped": " · {} duplicada{:.0} ignorada",
" · {} failed: {}": " · {} falhou: {}",
" · {} moved": " · {} movida(s)",
" · {} not readable": " · {} não legível",
", {failed} file{} not readable": ", {failed} arquivo{:.0} não legível",
"Add {local_n} to My Photos": "Adicionar {local_n} a Minhas fotos",
"Add {} Photo{}": "Adicionar {} foto{:.0}",
"Add “{}” to Favorites": "Adicionar “{}” aos favoritos",
"Added {n} photo{} to My Photos": "Adicionada {n} foto{:.0} a Minhas fotos",
"Added {n} photo{} to “{}”": "Adicionada {n} foto{:.0} a “{}”",
"Adding photos… {} of {}": "Adicionando fotos… {} de {}",
"All {n} selected photos move by the same amount.": "Todas as {n} fotos selecionadas se movem pela mesma quantidade.",
"Already in the library, in Recently Deleted: {} photo{} selected · right-click ▸ Restore, or Delete Permanently to import afresh": "Já na biblioteca, em Excluídos recentemente: {} foto{:.0} selecionada · clique com o botão direito ▸ Restaurar, ou Excluir permanentemente para importar de novo",
"Already in the library: {} photo{} selected in All Photos": "Já na biblioteca: {} foto{:.0} selecionada em Todas as fotos",
"Auto ({})": "Automático ({})",
"Auto Sync: edits apply to {n} photos": "Sincronização automática: edições aplicadas a {n} fotos",
"Automatic ({} MB)": "Automático ({} MB)",
"Build 1:1 Previews ({scope})": "Gerar pré-visualizações 1:1 ({scope})",
"Build Smart Previews ({scope})": "Gerar pré-visualizações inteligentes ({scope})",
"Build Standard-Sized Previews ({scope})": "Gerar pré-visualizações em tamanho padrão ({scope})",
"Click {}:1": "Clique em {}:1",
"Create {n} Virtual Copies": "Criar {n} cópias virtuais",
"Decoder: {reason}": "Decodificador: {reason}",
"Delete \"{label}\"": "Excluir \"{label}\"",
"Delete \"{label}\" (and the mask)": "Excluir \"{label}\" (e a máscara)",
"Delete {n} Photos": "Excluir {n} fotos",
"Delete “{current}”": "Excluir “{current}”",
"Discard Smart Previews ({scope})": "Descartar pré-visualizações inteligentes ({scope})",
"Duplicate \"{label}\"": "Duplicar \"{label}\"",
"Example: {example}": "Exemplo: {example}",
"Exported {n} photo{}": "Exportada {n} foto{:.0}",
"Exported {ok} of {total} photo{}": "Exportadas {ok} de {total} foto{:.0}",
"Exported {} point curve preset{}": "Exportadas {} predefinição{:.0} de curva de pontos",
"Exported {} preset{}": "Exportadas {} predefinição{:.0}",
"Exporting {} of {total}": "Exportando {} de {total}",
"Filter bar — {active} active filter{}": "Barra de filtros — {active} filtro{:.0} ativo",
"Healed {n} dust spot{}": "Recuperadas {n} mancha{:.0} de poeira",
"Import cancelled · {} photo{} added": "Importação cancelada · {} foto{:.0} adicionada",
"Imported {n} point curve preset{}": "Importadas {n} predefinição{:.0} de curva de pontos",
"Imported {n} preset{}": "Importadas {n} predefinição{:.0}",
"Imported {n} preset{}, {f} file{} not readable": "Importadas {n} predefinição{:.0}, {f} arquivo{:.0} não legível",
"Imported {} photo{}": "Importadas {} foto{:.0}",
"Label Set: {name}": "Conjunto de rótulos: {name}",
"Matches: {}": "Correspondências: {}",
"Merging… {stage} {:.0}%": "Mesclando… {stage} {:.0}%",
"Move {what} to Recently Deleted?": "Mover {what} para Excluídos recentemente?",
"Named ({named_n})": "Nomeadas ({named_n})",
"Paste into {n} photo{} — only settings that were copied are pasted": "Colar em {n} foto{:.0} — somente os ajustes copiados são colados",
"Preset: {name}": "Predefinição: {name}",
"Preview only — editing the camera's embedded JPEG ({})": "Somente pré-visualização — editando o JPEG incorporado da câmera ({})",
"Preview only — {}": "Somente pré-visualização — {}",
"Profile: {}": "Perfil: {}",
"Proof Copy ({})": "Cópia de prova ({})",
"Proof Preview · {}": "Pré-visualização de prova · {}",
"Quick Develop · {n} photos": "Revelação rápida · {n} fotos",
"Quick Develop: {name}": "Revelação rápida: {name}",
"Rated {}": "Avaliada {}",
"Reading photos… {done} of {total}": "Lendo fotos… {done} de {total}",
"Reading photos… {} of {}": "Lendo fotos… {} de {}",
"Redo {}": "Refazer {}",
"Remove “{}” from Favorites": "Remover “{}” dos favoritos",
"Rename {n} Photos…": "Renomear {n} fotos…",
"Rename “{name}”": "Renomear “{name}”",
"Rendering on the CPU: {why}": "Renderizando na CPU: {why}",
"Reset {name}": "Redefinir {name}",
"Reset {} Channel": "Redefinir canal {}",
"Saved export preset “{}”": "Predefinição de exportação “{}” salva",
"Show {hidden} hidden location{}": "Mostrar {hidden} local{:.0} oculto",
"Showing {n} of {}": "Mostrando {n} de {}",
"Smart album: {}": "Álbum inteligente: {}",
"Undo {}": "Desfazer {}",
"Updated “{name}”": "“{name}” atualizado",
"Version {} — a clean-room, pure-Rust photo library and raw developer.": "Versão {} — uma biblioteca de fotos e revelador RAW em Rust puro, desenvolvida de forma independente.",
"Version: {}": "Versão: {}",
"none (CPU rendering): {why}": "nenhum (renderização na CPU): {why}",
"the {n} selected photos": "as {n} fotos selecionadas",
"the {} photos in view": "as {} fotos em exibição",
"these {count} photos": "estas {count} fotos",
"{count} smart preview{} · {:.1} MB": "{count} pré-visualizações{:.0} · {:.1} MB",
"{f} folders (and their subfolders)": "{f} pastas (e suas subpastas)",
"{label} Grading": "Graduação de {label}",
"{label} {d:+} on every selected photo": "{label} {d:+} em cada foto selecionada",
"{label}, {n} photos": "{label}, {n} fotos",
"{matching} of {t} photos": "{matching} de {t} fotos",
"{matching} photos": "{matching} fotos",
"{mode} spot {} of {}": "{mode}: ponto {} de {}",
"{month}": "{month}",
"{month} {year}": "{month}/{year}",
"{msg} ({failed} failed)": "{msg} ({failed} com falha)",
"{name} Label": "Rótulo {name}",
"{name} Off": "{name} desativado",
"{name} On": "{name} ativado",
"{name} — double-click to reset it": "{name} — clique duas vezes para redefinir",
"{n} change{} saved in memory but not written to disk: {e}\nLightCraft retries automatically; quitting now would lose {}.": "{n} alteração{:.0} salva na memória mas não gravada no disco: {e}\nO LightCraft tenta novamente automaticamente; encerrar agora perderia {}.",
"{n} found · {sel} selected": "{n} encontradas · {sel} selecionadas",
"{n} photos": "{n} fotos",
"{n} photos in {} are hidden by the filters above": "{n} fotos em {} estão ocultas pelos filtros acima",
"{n} photo{}": "{n} foto{:.0}",
"{n} photo{} match · updates automatically as photos change": "{n} foto{:.0} correspondente(s) · atualiza automaticamente conforme as fotos mudam",
"{n} photo{} now · updates automatically as photos change": "{n} foto{:.0} agora · atualiza automaticamente conforme as fotos mudam",
"{sel_n} selected · {counted}": "{sel_n} selecionadas · {counted}",
"{stage}… {:.0}%": "{stage}… {:.0}%",
"{verb} {n} Photo{}": "{verb} {n} foto{:.0}",
"{weekday}, {day}": "{weekday}, {day}",
"{weekday}, {day} {month} {year}": "{weekday}, {day}/{month}/{year}",
"{what} merge added": "{what} mesclagem adicionada",
"{what} merge failed: {e}": "{what} falha na mesclagem: {e}",
"{year}": "{year}",
"{} and higher": "{} ou superior",
"{} files": "{} arquivos",
"{} files and {f} folder{}": "{} arquivos e {f} pasta{:.0}",
"{} of 4 guides — drag along lines that should be vertical or horizontal.": "{} de 4 guias — arraste ao longo das linhas que devem ficar verticais ou horizontais.",
"{} photos": "{} fotos",
"{} photo{} not readable": "{} foto{:.0} não legível",
"{} spot(s) on this photo": "{} ponto(s) nesta foto",
"· {dups} already in the library": "· {dups} já na biblioteca",
"· {n} photo{}": "· {n} foto{:.0}",
"… and {more} more": "… e mais {more}"
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+2
View File
@@ -389,6 +389,8 @@
"File Name": "檔案名稱",
"File Path": "檔案路徑",
"File Size": "檔案大小",
"Random": "隨機",
"Reshuffle": "重新隨機排序",
"File name": "檔案名稱",
"Fill Edges": "填補邊緣",
"Film": "底片",
+26 -6
View File
@@ -334,9 +334,16 @@ pub fn handle(app: &mut LightcraftApp, ctx: &egui::Context, req: &ControlRequest
}
}
/// Default export folder: `~/Pictures/LightCraft Exports` (falls back to the working directory).
/// Shown when an export has nowhere to go (no folder typed or chosen, and no home folder to default to).
pub const NO_EXPORT_FOLDER: &str = "Choose an export folder first.";
/// Default export folder: `~/Pictures/LightCraft Exports` (`%USERPROFILE%\Pictures\LightCraft Exports`
/// on Windows, where `HOME` usually isn't set). Empty when no home folder is known.
pub fn default_export_dir() -> String {
std::env::var("HOME").map(|h| format!("{h}/Pictures/LightCraft Exports")).unwrap_or_default()
let home = if cfg!(windows) { std::env::var_os("USERPROFILE").or_else(|| std::env::var_os("HOME")) } else { std::env::var_os("HOME") };
home.filter(|h| !h.is_empty())
.map(|h| std::path::PathBuf::from(h).join("Pictures").join("LightCraft Exports").to_string_lossy().into_owned())
.unwrap_or_default()
}
/// Export the selected photos (UI command `app.export`). Params: see
@@ -365,10 +372,23 @@ pub fn export_active(app: &mut LightcraftApp, p: &Value) -> Result<Value, String
if ids.is_empty() {
return Err("no photo selected".into());
}
// no folder given (e.g. File → Export with Preset): the last export's, else the default
let last_dir = app.session.last_export.as_ref().and_then(|l| l.get("dir")).and_then(Value::as_str).map(str::to_string);
let dir = p.get("dir").and_then(Value::as_str).map(str::to_string).or(last_dir).filter(|d| !d.is_empty()).unwrap_or_else(default_export_dir);
let to = Destination { dir: dir.clone(), exact: p.get("path").and_then(Value::as_str).map(str::to_string) };
// An exact output file needs no folder. A folder given but left blank (the Export dialog's Folder
// field cleared) is an error, like Lightroom refusing to export to an unspecified folder, rather
// than a silent write into the working directory. No folder at all (e.g. File → Export with
// Preset): the last export's, else the default.
let exact = p.get("path").and_then(Value::as_str).filter(|s| !s.trim().is_empty()).map(str::to_string);
let dir = match p.get("dir").and_then(Value::as_str) {
Some(d) => d.trim().to_string(),
None => {
let last_dir =
app.session.last_export.as_ref().and_then(|l| l.get("dir")).and_then(Value::as_str).map(str::trim).filter(|d| !d.is_empty());
last_dir.map(str::to_string).unwrap_or_else(default_export_dir)
}
};
if dir.is_empty() && exact.is_none() {
return Err(NO_EXPORT_FOLDER.into());
}
let to = Destination { dir: dir.clone(), exact };
let background = p.get("background").and_then(Value::as_bool).unwrap_or(false) && app.services.write_shared.is_some();
let out = if background {
let items = lightcraft_engine::export::prepare_batch(&mut app.session, &ids, &opts)?;
+429
View File
@@ -0,0 +1,429 @@
//! The About window's credits: every contributor (GitHub login always; display and real names only
//! with recorded consent) and the AI models that helped. The tables are baked into the binary by
//! build.rs from contributors/contributors.json (craftrules standards/contributors.md); nothing is
//! read at run time.
use std::cmp::Ordering;
use egui::RichText;
use crate::{LightcraftApp, theme::Tokens};
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Contributor {
pub login: &'static str,
pub display_name: Option<&'static str>,
pub real_name: Option<&'static str>,
pub prs: u64,
pub commits: u64,
pub lines_added: u64,
pub lines_deleted: u64,
pub binary_added: u64,
pub binary_deleted: u64,
/// ISO 8601 UTC; empty for someone credited only through a merged PR.
pub first_commit: &'static str,
pub last_commit: &'static str,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Model {
pub company: &'static str,
pub model: &'static str,
pub version: &'static str,
pub commits: u64,
pub lines_added: u64,
pub lines_deleted: u64,
}
include!(concat!(env!("OUT_DIR"), "/credits.rs"));
/// Which name to show. Display and real names fall back to the `@username` when not given.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub enum NameMode {
#[default]
Username,
DisplayName,
RealName,
}
impl NameMode {
pub const ALL: [NameMode; 3] = [NameMode::Username, NameMode::DisplayName, NameMode::RealName];
pub fn label(self) -> &'static str {
match self {
NameMode::Username => "Username",
NameMode::DisplayName => "Display name",
NameMode::RealName => "Real name",
}
}
}
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub enum SortKey {
Name,
Prs,
Commits,
LinesAdded,
LinesDeleted,
LinesDelta,
BinaryAdded,
BinaryDeleted,
#[default]
FirstCommit,
LastCommit,
}
impl SortKey {
pub const ALL: [SortKey; 10] = [
SortKey::Name,
SortKey::Prs,
SortKey::Commits,
SortKey::LinesAdded,
SortKey::LinesDeleted,
SortKey::LinesDelta,
SortKey::BinaryAdded,
SortKey::BinaryDeleted,
SortKey::FirstCommit,
SortKey::LastCommit,
];
/// Menu label and table header.
pub fn label(self) -> (&'static str, &'static str) {
match self {
SortKey::Name => ("Name (A–Z)", "Name"),
SortKey::Prs => ("Merged PRs", "PRs"),
SortKey::Commits => ("Commits", "Commits"),
SortKey::LinesAdded => ("Lines added", "+LOC"),
SortKey::LinesDeleted => ("Lines deleted", "−LOC"),
SortKey::LinesDelta => ("Line delta", "ΔLOC"),
SortKey::BinaryAdded => ("Binary assets added", "+Bin"),
SortKey::BinaryDeleted => ("Binary assets removed", "−Bin"),
SortKey::FirstCommit => ("First commit", "First"),
SortKey::LastCommit => ("Last commit", "Last"),
}
}
/// Names and dates read naturally oldest/A first; counts read biggest first.
pub fn default_ascending(self) -> bool {
matches!(self, SortKey::Name | SortKey::FirstCommit | SortKey::LastCommit)
}
}
impl Contributor {
pub fn name(&self, mode: NameMode) -> String {
let chosen = match mode {
NameMode::Username => None,
NameMode::DisplayName => self.display_name,
NameMode::RealName => self.real_name,
};
chosen.map_or_else(|| format!("@{}", self.login), str::to_string)
}
pub fn lines_delta(&self) -> i128 {
i128::from(self.lines_added) - i128::from(self.lines_deleted)
}
/// One line with everything we know, for tooltips.
pub fn summary(&self) -> String {
format!(
"@{}: {} PRs, {} commits, +{} / −{} lines (Δ {}), +{} / −{} binary assets, {} – {}",
self.login,
self.prs,
self.commits,
group(self.lines_added),
group(self.lines_deleted),
signed(self.lines_delta()),
self.binary_added,
self.binary_deleted,
day(self.first_commit),
day(self.last_commit),
)
}
}
/// Alphabetical key: case-insensitive, ignoring the `@` of usernames.
fn name_key(name: &str) -> String {
name.trim_start_matches('@').to_lowercase()
}
fn compare(a: &Contributor, b: &Contributor, mode: NameMode, key: SortKey) -> Ordering {
match key {
SortKey::Name => name_key(&a.name(mode)).cmp(&name_key(&b.name(mode))),
SortKey::Prs => a.prs.cmp(&b.prs),
SortKey::Commits => a.commits.cmp(&b.commits),
SortKey::LinesAdded => a.lines_added.cmp(&b.lines_added),
SortKey::LinesDeleted => a.lines_deleted.cmp(&b.lines_deleted),
SortKey::LinesDelta => a.lines_delta().cmp(&b.lines_delta()),
SortKey::BinaryAdded => a.binary_added.cmp(&b.binary_added),
SortKey::BinaryDeleted => a.binary_deleted.cmp(&b.binary_deleted),
// A PR-only contributor (no commit date) sorts after everyone with a date.
SortKey::FirstCommit => dated(a.first_commit).cmp(&dated(b.first_commit)),
SortKey::LastCommit => dated(a.last_commit).cmp(&dated(b.last_commit)),
}
}
fn dated(d: &str) -> (bool, &str) {
(d.is_empty(), d)
}
/// Contributors in display order. Ties fall back to the name (A–Z) so the order is stable.
pub fn sorted(list: &[Contributor], mode: NameMode, key: SortKey, ascending: bool) -> Vec<&Contributor> {
let mut v: Vec<&Contributor> = list.iter().collect();
v.sort_by(|a, b| {
let o = compare(a, b, mode, key);
let o = if ascending { o } else { o.reverse() };
o.then_with(|| compare(a, b, mode, SortKey::Name)).then_with(|| a.login.cmp(b.login))
});
v
}
fn group(n: u64) -> String {
let s = n.to_string();
let mut out = String::with_capacity(s.len() + s.len() / 3);
for (i, ch) in s.chars().enumerate() {
if i > 0 && (s.len() - i).is_multiple_of(3) {
out.push(',');
}
out.push(ch);
}
out
}
fn signed(n: i128) -> String {
let g = group(u64::try_from(n.unsigned_abs()).unwrap_or(u64::MAX));
if n < 0 { format!("−{g}") } else { format!("+{g}") }
}
fn day(iso: &str) -> &str {
if iso.is_empty() { "—" } else { iso.get(..10).unwrap_or(iso) }
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
struct View {
names: NameMode,
key: SortKey,
ascending: bool,
table: bool,
}
impl Default for View {
fn default() -> Self {
View { names: NameMode::Username, key: SortKey::FirstCommit, ascending: true, table: false }
}
}
/// The scrolling list's height: the About dialog is a fixed-size, centred window.
const LIST_HEIGHT: f32 = 360.0;
fn profile_url(c: &Contributor) -> String {
format!("https://github.com/{}", c.login)
}
/// A contributor's name as a link to their GitHub profile, with the whole line as its tooltip.
fn name_link(app: &mut LightcraftApp, ui: &mut egui::Ui, c: &Contributor, names: NameMode) {
if ui.link(c.name(names)).on_hover_text(c.summary()).clicked() {
let _ = crate::links::open(app, &profile_url(c));
}
}
/// About ▸ Contributors: a name toggle, a sort, and the list as a grab bag or a table.
pub fn contributors_ui(app: &mut LightcraftApp, ui: &mut egui::Ui) {
let t = Tokens::get(ui.ctx());
let id = egui::Id::new("credits_view");
let mut v = ui.data_mut(|d| d.get_temp::<View>(id)).unwrap_or_default();
ui.horizontal_wrapped(|ui| {
ui.label("Show");
for m in NameMode::ALL {
if ui.selectable_label(v.names == m, m.label()).clicked() {
v.names = m;
}
}
ui.separator();
ui.label("Sort");
egui::ComboBox::from_id_salt("credits_sort").selected_text(v.key.label().0).show_ui(ui, |ui| {
for k in SortKey::ALL {
if ui.selectable_label(v.key == k, k.label().0).clicked() {
v.key = k;
v.ascending = k.default_ascending();
}
}
});
if ui.button(if v.ascending { "▲" } else { "▼" }).on_hover_text("Reverse the order").clicked() {
v.ascending = !v.ascending;
}
ui.separator();
let r = ui.selectable_label(!v.table, "Grab bag");
crate::widgets::register(ui.ctx(), "button:creditsGrabBag", r.rect);
if r.clicked() {
v.table = false;
}
let r = ui.selectable_label(v.table, "Table");
crate::widgets::register(ui.ctx(), "button:creditsTable", r.rect);
if r.clicked() {
v.table = true;
}
});
let list = sorted(CONTRIBUTORS, v.names, v.key, v.ascending);
ui.label(RichText::new(format!("{} contributors · {} commits", list.len(), group(TOTAL_COMMITS))).small().color(t.text_dim));
ui.separator();
egui::ScrollArea::both().id_salt("credits_scroll").max_height(LIST_HEIGHT).auto_shrink([false, true]).show(ui, |ui| {
if list.is_empty() {
ui.label("No contributor data was built into this copy.");
} else if v.table {
table(app, ui, &list, &mut v);
} else {
ui.horizontal_wrapped(|ui| {
for (i, c) in list.iter().enumerate() {
if i > 0 {
ui.label(RichText::new("·").color(t.text_dim));
}
name_link(app, ui, c, v.names);
}
});
}
});
ui.data_mut(|d| d.insert_temp(id, v));
}
fn table(app: &mut LightcraftApp, ui: &mut egui::Ui, list: &[&Contributor], v: &mut View) {
egui::Grid::new("credits_table").striped(true).num_columns(SortKey::ALL.len()).show(ui, |ui| {
for k in SortKey::ALL {
let arrow = if v.key == k { if v.ascending { " ▲" } else { " ▼" } } else { "" };
if ui.button(RichText::new(format!("{}{arrow}", k.label().1)).strong()).on_hover_text(k.label().0).clicked() {
if v.key == k {
v.ascending = !v.ascending;
} else {
v.key = k;
v.ascending = k.default_ascending();
}
}
}
ui.end_row();
for c in list {
name_link(app, ui, c, v.names);
ui.label(group(c.prs));
ui.label(group(c.commits));
ui.label(group(c.lines_added));
ui.label(group(c.lines_deleted));
ui.label(signed(c.lines_delta()));
ui.label(group(c.binary_added));
ui.label(group(c.binary_deleted));
ui.label(day(c.first_commit));
ui.label(day(c.last_commit));
ui.end_row();
}
});
}
/// About ▸ Models: AI models credited in Co-Authored-By trailers.
pub fn models_ui(ui: &mut egui::Ui) {
if MODELS.is_empty() {
ui.label("No model credits were built into this copy.");
return;
}
let assisted: u64 = MODELS.iter().map(|m| m.commits).max().unwrap_or(0).max(1);
egui::ScrollArea::both().id_salt("credits_models_scroll").max_height(LIST_HEIGHT).auto_shrink([false, true]).show(ui, |ui| {
egui::Grid::new("credits_models").striped(true).num_columns(6).show(ui, |ui| {
for h in ["Company", "Model", "Version", "Commits", "% of all commits", "Lines +/−"] {
ui.label(RichText::new(h).strong());
}
ui.end_row();
for m in MODELS {
ui.label(m.company);
ui.label(m.model);
ui.label(m.version);
ui.label(group(m.commits));
ui.label(format!("{:.0}%", 100.0 * m.commits as f64 / TOTAL_COMMITS.max(assisted) as f64));
ui.label(format!("+{} / −{}", group(m.lines_added), group(m.lines_deleted)));
ui.end_row();
}
});
});
}
#[cfg(test)]
mod tests {
use super::*;
fn c(
login: &'static str,
display: Option<&'static str>,
real: Option<&'static str>,
added: u64,
deleted: u64,
first: &'static str,
) -> Contributor {
Contributor {
login,
display_name: display,
real_name: real,
prs: 0,
commits: 1,
lines_added: added,
lines_deleted: deleted,
binary_added: 0,
binary_deleted: 0,
first_commit: first,
last_commit: first,
}
}
fn people() -> [Contributor; 4] {
[
c("zed", None, None, 5, 50, "2026-01-02T00:00:00Z"),
c("Bob", Some("Ann"), None, 10, 0, "2026-01-01T00:00:00Z"),
c("alice", None, None, 7, 1, ""),
c("carol", None, Some("Dana"), 1, 0, "2026-01-03T00:00:00Z"),
]
}
fn logins(v: &[&Contributor]) -> Vec<&'static str> {
v.iter().map(|c| c.login).collect()
}
#[test]
fn names_fall_back_to_the_username() {
let p = c("bob", Some("Bobby"), None, 0, 0, "");
assert_eq!(p.name(NameMode::Username), "@bob");
assert_eq!(p.name(NameMode::DisplayName), "Bobby");
assert_eq!(p.name(NameMode::RealName), "@bob");
}
#[test]
fn alphabetical_ignores_at_and_case() {
let all = people();
let v = sorted(&all, NameMode::Username, SortKey::Name, true);
assert_eq!(logins(&v), ["alice", "Bob", "carol", "zed"]);
let v = sorted(&all, NameMode::DisplayName, SortKey::Name, true);
assert_eq!(logins(&v), ["alice", "Bob", "carol", "zed"]); // "Ann" sorts with the @usernames
let v = sorted(&all, NameMode::RealName, SortKey::Name, false);
assert_eq!(logins(&v), ["zed", "carol", "Bob", "alice"]); // "Dana" between bob and zed
}
#[test]
fn delta_and_dates() {
let all = people();
let v = sorted(&all, NameMode::Username, SortKey::LinesDelta, false);
assert_eq!(logins(&v), ["Bob", "alice", "carol", "zed"]);
let v = sorted(&all, NameMode::Username, SortKey::FirstCommit, true);
assert_eq!(logins(&v), ["Bob", "zed", "carol", "alice"]); // undated last
}
#[test]
fn the_committed_json_is_baked_in() {
// contributors/contributors.json is committed, so a build always has someone to credit
assert!(!CONTRIBUTORS.is_empty(), "build.rs produced no contributors");
assert!(TOTAL_COMMITS > 0);
assert!(CONTRIBUTORS.iter().all(|c| !c.login.is_empty() && !c.login.starts_with('@')));
assert!(MODELS.iter().all(|m| !m.model.is_empty()));
}
#[test]
fn formatting() {
assert_eq!(group(1234567), "1,234,567");
assert_eq!(group(12), "12");
assert_eq!(signed(-1000), "−1,000");
assert_eq!(day("2026-10-07T18:45:23Z"), "2026-10-07");
assert_eq!(day(""), "—");
}
}
+146 -15
View File
@@ -169,9 +169,11 @@ impl Headless {
}
}
/// Is anything still in progress (renders, queued input)?
/// Is anything still in progress (renders, file-system checks the sidebar waits for, queued
/// input)?
pub fn busy(&self) -> bool {
self.app.renderer.in_flight() > 0
|| crate::panels::left::fs_cached_running(&self.view.ctx) > 0
|| self.app.merge.busy()
|| self.app.scan.is_some()
|| self.app.import.is_some()
@@ -229,6 +231,18 @@ impl Headless {
self.paint()
}
/// Tests that browse a folder in the temp directory expect it to be a Local location of its
/// own. Where the temp directory lies inside the home folder (Windows), the sidebar lists it
/// inside Home's tree instead, so hide Home for the test; elsewhere this does nothing.
#[cfg(test)]
pub(crate) fn hide_home_above(&mut self, path: &std::path::Path) {
let home = std::env::var("HOME").or_else(|_| std::env::var("USERPROFILE")).unwrap_or_default();
if !home.is_empty() && lightcraft_catalog::query::folder_within(&path.to_string_lossy(), &home) {
let r = self.request("engine.execute", json!({"command": "local.hide", "params": {"path": home}}), Duration::from_secs(10));
assert_eq!(r["ok"], true, "{r}");
}
}
/// Send a control-protocol request (see [`crate::control`]) and run frames until it is
/// answered, then until its input has been consumed. Returns `{"ok": …, "result"|"error": …}`.
pub fn request(&mut self, method: &str, params: Value, timeout: Duration) -> Value {
@@ -278,6 +292,8 @@ mod tests {
fn demo_grid_snapshot_has_ui_pixels() {
let t0 = Instant::now();
let mut h = demo([1200.0, 760.0]);
// settle() can see a quiet spell before the first thumbnails land on a loaded machine
h.step_until(SETTLE, |h| h.app.renderer.thumb_textures() > 0);
let img = h.snapshot(SETTLE);
eprintln!("headless snapshot: {:?} in {:?} ({} frames)", img.size, t0.elapsed(), h.frames());
assert_eq!(img.size, [1200, 760]);
@@ -382,6 +398,7 @@ mod tests {
let id = h.app.session.active().unwrap();
let photo = |h: &Headless| h.app.session.catalog.photo(id).unwrap().clone();
let before = photo(&h);
h.step_until(SETTLE, |h| h.app.renderer.variant_textures() >= 6);
assert!(h.app.renderer.variant_textures() >= 6, "variant thumbnails rendered: {}", h.app.renderer.variant_textures());
// hover: the loupe shows the look, nothing is committed
let r = h.request("ui.hoverWidget", json!({"id": "profileCell:lc.vivid"}), t);
@@ -529,6 +546,14 @@ mod tests {
fn keyword_list_filters_renames_and_suggests() {
// tall: the demo library's own keywords come first in the list
let mut h = demo([1300.0, 1800.0]);
// Host folders arrive asynchronously above Keywords. Keep this keyword fixture's
// geometry independent of their existence and the background stat timing.
if let Ok(home) = std::env::var("HOME").or_else(|_| std::env::var("USERPROFILE")) {
h.app.ui.hidden_locations.extend(
["Pictures", "Desktop", "Downloads", ""]
.map(|sub| if sub.is_empty() { home.clone() } else { std::path::Path::new(&home).join(sub).to_string_lossy().to_string() }),
);
}
let t = Duration::from_secs(10);
let vis: Vec<u64> = h.app.session.visible_cloned().iter().map(|p| p.0).collect();
let ex = |h: &mut Headless, c: &str, p: Value| h.request("engine.execute", json!({"command": c, "params": p}), Duration::from_secs(10));
@@ -559,12 +584,43 @@ mod tests {
h.request("engine.execute", json!({"command": "library.select", "params": {"ids": [vis[1]]}}), t);
ex(&mut h, "photo.setMeta", json!({"ids": [vis[0]], "addKeywords": ["gelato"]}));
h.request("ui.set", json!({"right": "keywords"}), t);
assert!(h.settle(SETTLE), "keyword suggestions did not settle");
let r = h.request("ui.clickWidget", json!({"id": "kwSuggest:gelato"}), t);
assert_eq!(r["ok"], true, "{r}");
assert!(h.app.session.catalog.photo(lightcraft_catalog::PhotoId(vis[1])).unwrap().meta.keywords.contains(&"gelato".to_string()));
h.settle(SETTLE);
}
/// The sidebar's file-system checks run on worker threads and add rows when they land, which
/// moves every row below them: `busy()` counts them, so `settle` waits for them before a test
/// reads widget positions (a click aimed at a stale rect hits the neighbouring row).
#[test]
fn settle_waits_for_file_system_checks() {
use std::sync::atomic::{AtomicBool, Ordering};
static RELEASE: AtomicBool = AtomicBool::new(false);
fn slow_check(_: &str) -> bool {
let t0 = Instant::now();
while !RELEASE.load(Ordering::Acquire) && t0.elapsed() < Duration::from_secs(60) {
std::thread::sleep(Duration::from_millis(1));
}
true
}
let mut h = demo([800.0, 600.0]);
h.settle(SETTLE);
let check = |h: &mut Headless| {
let raw = HeadlessView::raw_input(h.size, 1.0, 0.0, vec![]);
let mut got = None;
h.view.run(raw, |ui| got = crate::panels::left::fs_cached(ui, "test-slow", "x", f64::INFINITY, slow_check));
got
};
assert_eq!(check(&mut h), None, "the answer is worked out off the UI thread");
assert!(h.busy(), "a pending file-system check is pending work");
RELEASE.store(true, Ordering::Release);
assert!(h.settle(SETTLE));
assert!(!h.busy());
assert_eq!(check(&mut h), Some(true));
}
/// Photo > Rename Photos…: the dialog previews and renames the selection.
#[test]
fn rename_dialog_renames_the_selection() {
@@ -831,35 +887,68 @@ mod tests {
}
/// Local: Remove from Local hides a sidebar location (nothing on disk changes) and the
/// "Show hidden locations" row puts it back.
/// "Show hidden locations" row puts it back. The folder is browsed and kept the way Browse
/// Folder… does it, in the temporary folder — on Windows beneath Home, whose tree must not
/// open down to the hidden folder and push the restore row out of view.
#[test]
fn local_location_can_be_hidden_and_restored() {
let mut h = demo([1300.0, 900.0]);
let size = [1300.0, 900.0];
let mut h = demo(size);
let t = Duration::from_secs(10);
let dir = std::env::temp_dir().join(format!("lc-ui-hide-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let path = dir.to_string_lossy().to_string();
let ids = |h: &mut Headless| -> Vec<String> {
let rects = |h: &mut Headless| -> Vec<(String, Vec<f64>)> {
let w = h.request("ui.widgets", json!({"filter": "source:local:"}), t);
w["result"].as_array().unwrap().iter().filter_map(|x| x["id"].as_str().map(String::from)).collect()
w["result"]
.as_array()
.unwrap()
.iter()
.map(|x| (x["id"].as_str().unwrap().to_string(), x["rect"].as_array().unwrap().iter().filter_map(|v| v.as_f64()).collect()))
.collect()
};
let ids = |h: &mut Headless| -> Vec<String> { rects(h).into_iter().map(|(id, _)| id).collect() };
h.request("ui.set", json!({"leftPanel": true}), t);
h.request("engine.execute", json!({"command": "library.browse", "params": {"path": path}}), t);
h.hide_home_above(&dir);
// Browse Folder… browses the picked folder and keeps it in Local within one frame; a
// request runs frames, so keep it first (no frame sees it browsed but not yet kept)
let r = h.request("engine.execute", json!({"command": "local.addRoot", "params": {"path": path}}), t);
assert_eq!(r["ok"], true, "{r}");
let r = h.request("engine.execute", json!({"command": "library.browse", "params": {"path": path}}), t);
assert_eq!(r["ok"], true, "{r}");
h.settle(SETTLE);
assert!(ids(&mut h).contains(&format!("source:local:{path}")));
assert!(!ids(&mut h).iter().any(|i| i == "source:local:restoreHidden"));
let row = format!("source:local:{path}");
assert!(h.step_until(t, |h| h.app.widgets.iter().any(|(w, _)| *w == row)), "the browsed folder is listed: {:?}", ids(&mut h));
// the "Show hidden locations" row appears only while something is hidden
let hidden_before = !h.app.ui.hidden_locations.is_empty();
assert_eq!(ids(&mut h).iter().any(|i| i == "source:local:restoreHidden"), hidden_before);
// hide it while it is still being browsed
let r = h.request("engine.execute", json!({"command": "local.hide", "params": {"path": path}}), t);
assert_eq!(r["ok"], true, "{r}");
h.settle(SETTLE);
let after = ids(&mut h);
assert!(!after.contains(&format!("source:local:{path}")), "{after:?}");
assert!(after.iter().any(|i| i == "source:local:restoreHidden"), "{after:?}");
// give any listing a reveal would wait for time to arrive
for _ in 0..20 {
h.step();
std::thread::sleep(Duration::from_millis(5));
}
h.settle(SETTLE);
let after = rects(&mut h);
let after_ids: Vec<&str> = after.iter().map(|(id, _)| id.as_str()).collect();
assert!(!after_ids.contains(&format!("source:local:{path}").as_str()), "{after_ids:?}");
let restore = after.iter().find(|(id, _)| id == "source:local:restoreHidden").map(|(_, r)| r.clone());
let restore = restore.unwrap_or_else(|| panic!("no restore row: {after_ids:?}"));
assert!(restore[1] >= 0.0 && restore[1] + restore[3] <= f64::from(size[1]), "the restore row is in view: {restore:?} {after_ids:?}");
assert!(h.app.session.browse.is_some(), "hiding does not stop browsing");
assert!(dir.is_dir(), "the folder itself is untouched");
assert_eq!(h.request("ui.clickWidget", json!({"id": "source:local:restoreHidden"}), t)["ok"], true);
h.settle(SETTLE);
assert!(h.app.ui.hidden_locations.is_empty());
assert!(ids(&mut h).contains(&format!("source:local:{path}")));
assert!(!ids(&mut h).iter().any(|i| i == "source:local:restoreHidden"), "nothing left to restore");
if !hidden_before {
// (with Home hidden for this test, the folder is back inside Home's tree instead)
assert!(ids(&mut h).contains(&format!("source:local:{path}")));
}
let _ = std::fs::remove_dir_all(&dir);
}
@@ -876,19 +965,28 @@ mod tests {
std::fs::create_dir_all(base.join(d)).unwrap();
}
let s = |p: std::path::PathBuf| p.to_string_lossy().to_string();
let (photos, day1, day2, other) =
(s(base.join("Photos")), s(base.join("Photos/2026/20260101")), s(base.join("Photos/2026/20260114")), s(base.join("Other")));
// joined by component: the sidebar's ids spell paths with the platform's separator
let year = base.join("Photos").join("2026");
let (photos, day1, day2, other) = (s(base.join("Photos")), s(year.join("20260101")), s(year.join("20260114")), s(base.join("Other")));
h.hide_home_above(&base);
let exec = |h: &mut Headless, c: &str, p: Value| h.request("engine.execute", json!({"command": c, "params": p}), t);
let rects = |h: &mut Headless| -> std::collections::HashMap<String, f64> {
let w = h.request("ui.widgets", json!({"filter": "lc-ui-roots-"}), t);
w["result"].as_array().unwrap().iter().map(|x| (x["id"].as_str().unwrap().to_string(), x["rect"][0].as_f64().unwrap_or(0.0))).collect()
};
h.request("ui.set", json!({"leftPanel": true}), t);
// Home may also own this temporary directory. Keep the fixture tree
// independent of the user's real folders and their async listings.
if let Ok(home) = std::env::var("HOME").or_else(|_| std::env::var("USERPROFILE")) {
assert_eq!(exec(&mut h, "local.hide", json!({"path": home}))["ok"], true);
}
assert_eq!(exec(&mut h, "local.addRoot", json!({"path": photos}))["ok"], true);
assert_eq!(exec(&mut h, "local.addRoot", json!({"path": format!("{photos}/")}))["result"]["roots"].as_array().map(Vec::len), Some(1));
exec(&mut h, "library.browse", json!({"path": day1}));
h.settle(SETTLE);
h.step();
let row = format!("source:local:{day2}");
assert!(h.step_until(t, |h| h.app.widgets.iter().any(|(w, _)| *w == row)), "the sibling listing did not arrive");
let r = rects(&mut h);
assert!(r.contains_key(&format!("source:local:{photos}")), "the kept root stays: {r:?}");
assert!(r.contains_key(&format!("source:local:{day2}")), "the sibling stays reachable: {r:?}");
@@ -898,9 +996,11 @@ mod tests {
h.settle(SETTLE);
assert_eq!(h.app.session.browse.as_ref().map(|b| b.path.clone()), Some(day2.clone()), "the sibling is browsed");
// another location: the kept root stays listed
exec(&mut h, "library.browse", json!({"path": other}));
assert_eq!(exec(&mut h, "library.browse", json!({"path": other}))["ok"], true);
h.settle(SETTLE);
h.step();
let row = format!("source:local:{other}");
assert!(h.step_until(t, |h| h.app.widgets.iter().any(|(w, _)| *w == row)), "the other location did not arrive");
let r = rects(&mut h);
assert!(r.contains_key(&format!("source:local:{photos}")) && r.contains_key(&format!("source:local:{other}")), "{r:?}");
// restart: the kept roots come back with the saved UI state
@@ -1012,6 +1112,7 @@ mod tests {
let r = h.request("engine.execute", json!({"command": "file.addFromDevice", "params": {"path": sub.to_string_lossy()}}), t);
assert_eq!(r["ok"], true, "{r}");
h.settle(SETTLE);
h.step_until(SETTLE, |h| matches!(h.app.ui.dialog, Some(crate::state::Dialog::Import { .. })));
let Some(crate::state::Dialog::Import { opts }) = &h.app.ui.dialog else { panic!("no import review") };
assert!(opts.copy);
let _ = std::fs::remove_dir_all(&dir);
@@ -1226,6 +1327,8 @@ mod tests {
let r = h.request("engine.execute", json!({"command": "file.addPhotos", "params": {"paths": [dir.to_string_lossy()]}}), t);
assert_eq!(r["result"]["scanning"], true, "{r}");
h.settle(SETTLE);
h.step_until(SETTLE, |h| matches!(h.app.ui.dialog, Some(crate::state::Dialog::Import { .. })));
h.step_until(SETTLE, |h| h.app.renderer.textures.keys().filter(|s| matches!(s, crate::render::Slot::Import(_))).count() >= 5);
let Some(crate::state::Dialog::Import { opts }) = &h.app.ui.dialog else { panic!("no import review") };
assert_eq!(opts.candidates.len(), 6);
assert_eq!(opts.candidates.iter().filter(|c| c.duplicate.is_some()).count(), 1);
@@ -1358,6 +1461,7 @@ mod tests {
let r = h.request("engine.execute", json!({"command": "file.addPhotos", "params": {"paths": [src.to_string_lossy()]}}), t);
assert_eq!(r["result"]["scanning"], true, "{r}");
h.settle(SETTLE);
h.step_until(SETTLE, |h| matches!(h.app.ui.dialog, Some(crate::state::Dialog::Import { .. })));
let Some(crate::state::Dialog::Import { opts }) = &h.app.ui.dialog else { panic!("no import review") };
assert_eq!(opts.candidates.len(), 10);
for id in ["button:importCopy", "button:importDest"] {
@@ -1533,6 +1637,33 @@ mod tests {
let _ = std::fs::remove_dir_all(&dir);
}
/// Help ▸ About: the About, Contributors and Models tabs switch and paint (credits are
/// compiled in).
#[test]
fn about_dialog_tabs_show_the_credits() {
// an empty library: the dialog needs no photos, and no decodes compete with other tests
let services = crate::Services { png: None, ..Default::default() };
let mut h = Headless::new(LightcraftApp::new(lightcraft_engine::Session::new(), services), [1300.0, 820.0], 1.0);
let t = Duration::from_secs(10);
let r = h.request("ui.menu.invoke", json!({"id": "app.about"}), t);
assert_eq!(r["ok"], true, "{r}");
assert_eq!(h.app.ui.dialog, Some(crate::state::Dialog::About));
// a new window sizes itself on its first frame: let it settle before clicking its tabs
h.step();
h.step();
for (i, (tab, _)) in crate::panels::dialogs::ABOUT_TABS.iter().enumerate().rev() {
let r = h.request("ui.clickWidget", json!({"id": format!("button:aboutTab-{tab}")}), t);
assert_eq!(r["ok"], true, "{tab}: {r}");
h.step();
let shown = h.view.ctx.data_mut(|d| d.get_temp::<u8>(egui::Id::new("about_tab")));
assert_eq!(shown.map(usize::from), Some(i), "{tab}");
}
h.request("ui.clickWidget", json!({"id": "button:aboutTab-contributors"}), t);
let img = h.snapshot(SETTLE);
assert_eq!(img.size, [1300, 820]);
assert_eq!(h.app.ui.dialog, Some(crate::state::Dialog::About), "switching tabs keeps the dialog open");
}
/// Settings (⌘,): tabs switch, app settings change the UI state, library settings go through
/// the engine; the delete confirmation guards ⌫.
#[test]
+11
View File
@@ -120,6 +120,7 @@ language_table! {
ZhHans, "zh-hans", "简体中文", "Hans", include_str!("../locales/zh-hans.json");
ZhHant, "zh-hant", "繁體中文(台灣)", "Hant", include_str!("../locales/zh-hant.json");
Ja, "ja", "日本語", "Jpan", include_str!("../locales/ja.json");
PtBr, "pt-br", "Português (Brasil)", "Latn", include_str!("../locales/pt-br.json");
}
// The settings file stores the BCP-47 code (`"zh-hans"`), never the Rust variant name, so a
@@ -339,6 +340,10 @@ mod tests {
assert_eq!(Locale::parse_tag("zh-Hant-TW"), Some(Locale::ZhHant));
assert_eq!(Locale::parse_tag("zh-CN"), Some(Locale::ZhHans));
assert_eq!(Locale::parse_tag("ja_JP.UTF-8"), Some(Locale::Ja));
assert_eq!(Locale::parse_tag("pt-br"), Some(Locale::PtBr));
assert_eq!(Locale::parse_tag("pt-BR"), Some(Locale::PtBr));
assert_eq!(Locale::parse_tag("pt_BR.UTF-8"), Some(Locale::PtBr));
assert_eq!(Locale::parse_tag("pt"), Some(Locale::PtBr));
assert_eq!(Locale::parse_tag("de"), None);
}
@@ -353,6 +358,9 @@ mod tests {
assert_eq!(tr("A string no catalog has"), "A string no catalog has");
set_language(Locale::Ja);
assert_eq!(tr("Exposure"), "露出");
set_language(Locale::PtBr);
assert_eq!(tr("Exposure"), "Exposição");
assert_eq!(tr("Settings"), "Configurações");
set_language(Locale::En);
assert_eq!(tr("Exposure"), "Exposure");
}
@@ -457,6 +465,7 @@ mod tests {
("app.language.english", Locale::En),
("app.language.simplifiedChinese", Locale::ZhHans),
("app.language.japanese", Locale::Ja),
("app.language.portuguese", Locale::PtBr),
];
// One command per language, and every command reachable from the menu table.
assert_eq!(commands.len(), Locale::ALL.len());
@@ -559,6 +568,8 @@ mod tests {
assert!(text.contains("マイフォト") && text.contains("すべての写真"), "{text}");
let text = painted_text(&ctx, &mut app, Locale::ZhHans);
assert!(text.contains("我的照片") && text.contains("所有照片"), "{text}");
let text = painted_text(&ctx, &mut app, Locale::PtBr);
assert!(text.contains("Minhas fotos") && text.contains("Todas as fotos"), "{text}");
set_language(Locale::En);
}
+33 -2
View File
@@ -151,8 +151,14 @@ pub fn paint(p: &Painter, rect: Rect, icon: Icon, color: Color32) {
pen.rect(3.0, 4.0, 17.0, 16.0, 1.5);
pen.line(&[(8.0, 4.0), (8.0, 16.0)]);
}
Back => pen.line(&[(16.0, 10.0), (4.0, 10.0), (9.0, 5.0)]),
Forward => pen.line(&[(4.0, 10.0), (16.0, 10.0), (11.0, 5.0)]),
Back => {
pen.line(&[(16.0, 10.0), (4.0, 10.0)]);
pen.line(&[(9.0, 5.0), (4.0, 10.0), (9.0, 15.0)]);
}
Forward => {
pen.line(&[(4.0, 10.0), (16.0, 10.0)]);
pen.line(&[(11.0, 5.0), (16.0, 10.0), (11.0, 15.0)]);
}
Search => {
pen.circle(8.5, 8.5, 5.0);
pen.line(&[(12.2, 12.2), (16.5, 16.5)]);
@@ -479,3 +485,28 @@ pub fn paint(p: &Painter, rect: Rect, icon: Icon, color: Color32) {
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn navigation_arrowheads_have_both_halves() {
for icon in [Icon::Back, Icon::Forward] {
for scale in [1.0, 2.0] {
let mut view = crate::headless::HeadlessView::new();
let raw = crate::headless::HeadlessView::raw_input(vec2(64.0, 64.0), scale, 0.0, vec![]);
view.run(raw, |ui| {
ui.painter().rect_filled(ui.max_rect(), 0.0, Color32::BLACK);
paint(ui.painter(), Rect::from_center_size(pos2(32.0, 32.0), vec2(18.6, 18.6)), icon, Color32::WHITE);
});
let image = view.paint(&Default::default());
let middle = image.pixels.len() / 2;
let upper: u64 = image.pixels[..middle].iter().map(|p| u64::from(p.r())).sum();
let lower: u64 = image.pixels[middle..].iter().map(|p| u64::from(p.r())).sum();
assert!(upper > 0 && lower > 0, "the arrow must render");
assert!(upper.abs_diff(lower) < upper / 20, "arrowhead must be symmetric at scale {scale}: {upper} above, {lower} below");
}
}
}
}
+3
View File
@@ -7,6 +7,7 @@
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
pub mod control;
pub mod credits;
pub mod export_task;
pub mod headless;
pub mod i18n;
@@ -42,6 +43,8 @@ mod tests_quit_unsaved;
#[cfg(test)]
mod tests_scroll;
#[cfg(test)]
mod tests_switch_library;
#[cfg(test)]
mod tests_unsaved;
use std::sync::mpsc::{Receiver, Sender};
+70 -3
View File
@@ -251,6 +251,7 @@ const HIDDEN: &[&str] = &[
"photo.unflag",
"photo.label",
"library.sort",
"library.shuffle",
"album.addPhotos",
"album.create",
"library.import",
@@ -458,13 +459,30 @@ fn expanded(app: &LightcraftApp, name: &str) -> Option<Vec<MenuNode>> {
("File Name", FileName, "fileName"),
("Rating", Rating, "rating"),
("File Size", FileSize, "fileSize"),
("Random", Random, "random"),
]
.into_iter()
.map(|(label, key, k)| item("library.sort", json!({"key": k}), label, None, true, Some(cur.key == key)))
.collect();
v.push(item("library.shuffle", json!({}), "Reshuffle", None, cur.key == Random, None));
v.push(MenuNode::Separator);
v.push(item("library.sort", json!({"ascending": true}), "Ascending", None, true, Some(cur.ascending)));
v.push(item("library.sort", json!({"ascending": false}), "Descending", None, true, Some(!cur.ascending)));
// a shuffle has no direction worth choosing
v.push(item(
"library.sort",
json!({"ascending": true}),
"Ascending",
None,
cur.key != Random,
(cur.key != Random).then_some(cur.ascending),
));
v.push(item(
"library.sort",
json!({"ascending": false}),
"Descending",
None,
cur.key != Random,
(cur.key != Random).then_some(!cur.ascending),
));
v.push(MenuNode::Separator);
use lightcraft_catalog::GroupBy;
let groups = [
@@ -735,7 +753,13 @@ pub fn show_in_window(app: &mut LightcraftApp, ui: &mut egui::Ui, max_width: f32
});
}
if let Some((id, params)) = clicked {
let _ = run_item(app, &id, params);
let r = run_item(app, &id, params);
// an export that can't start (e.g. no folder) says why instead of doing nothing
if let Err(e) = r
&& matches!(id.as_str(), "app.export" | "app.exportPrevious")
{
app.toast(ui.ctx(), e);
}
}
ui.cursor().left() - start
}
@@ -787,6 +811,42 @@ mod tests {
})
}
/// The Sort submenu is expanded by hand: Reshuffle appears once (not again from the registry),
/// only while sorting at random, and the direction items are off for a shuffle.
#[test]
fn sort_menu_lists_reshuffle_once_and_only_enables_it_for_random() {
fn sort_children(bar: &[(String, Vec<MenuNode>)]) -> Vec<MenuNode> {
let view = &bar.iter().find(|(t, _)| t == "View").expect("View menu").1;
view.iter()
.find_map(|n| match n {
MenuNode::Submenu { label, children } if label == "Sort" => Some(children.clone()),
_ => None,
})
.expect("Sort submenu")
}
let count = |nodes: &[MenuNode]| nodes.iter().filter(|n| matches!(n, MenuNode::Item { id, .. } if id == "library.shuffle")).count();
let mut a = app();
let kids = sort_children(&menu_bar(&a));
assert_eq!(count(&kids), 1);
assert!(matches!(find(&kids, "library.shuffle"), Some(MenuNode::Item { enabled: false, .. })), "off until Random is chosen");
a.session.execute("library.sort", &json!({"key": "random"})).expect("sort at random");
let kids = sort_children(&menu_bar(&a));
assert_eq!(count(&kids), 1);
assert!(matches!(find(&kids, "library.shuffle"), Some(MenuNode::Item { enabled: true, .. })));
// no direction is shown as chosen while shuffling
let dir_checked = |kids: &[MenuNode]| {
kids.iter()
.filter_map(|n| match n {
MenuNode::Item { label, checked, .. } if label == "Ascending" || label == "Descending" => Some(*checked),
_ => None,
})
.collect::<Vec<_>>()
};
assert_eq!(dir_checked(&kids), vec![None, None]);
a.session.execute("library.sort", &json!({"key": "fileName"})).expect("sort by name");
assert!(dir_checked(&sort_children(&menu_bar(&a))).iter().all(Option::is_some), "checks come back for other keys");
}
/// File opens with the import entry points, worded as importing (not as adding a sidebar
/// folder): Import Photos… (⇧⌘I), Import from Folder…, Import from Device ▸.
#[test]
@@ -945,6 +1005,13 @@ mod tests {
// remembered (expanded) for Export with Previous, folder included
let last = app.session.last_export.clone().unwrap();
assert_eq!((last["format"].as_str(), last["width"].as_u64(), last.get("preset")), (Some("png"), Some(40), None));
// a blank folder (the Export dialog's Folder field cleared) is refused with a clear message
// instead of writing into the working directory
let n = w.len();
drop(w);
let r = run_item(&mut app, "app.export", json!({"preset": "Tiny PNG", "dir": " "}));
assert_eq!(r.unwrap_err(), crate::control::NO_EXPORT_FOLDER);
assert_eq!(written.lock().unwrap().len(), n, "nothing written without a folder");
}
#[test]
+2
View File
@@ -17,6 +17,7 @@ pub const LANGUAGE_COMMANDS: &[UiCommand] = &[
("app.language.simplifiedChinese", crate::i18n::Locale::ZhHans.name(), None, "Edit>Language"),
("app.language.traditionalChinese", crate::i18n::Locale::ZhHant.name(), None, "Edit>Language"),
("app.language.japanese", crate::i18n::Locale::Ja.name(), None, "Edit>Language"),
("app.language.portuguese", crate::i18n::Locale::PtBr.name(), None, "Edit>Language"),
];
/// Every UI command: the languages, then everything else. `xtask parity` reads both tables from
@@ -33,6 +34,7 @@ pub fn language_from_command(id: &str) -> Option<crate::i18n::Locale> {
"app.language.simplifiedChinese" => Some(crate::i18n::Locale::ZhHans),
"app.language.traditionalChinese" => Some(crate::i18n::Locale::ZhHant),
"app.language.japanese" => Some(crate::i18n::Locale::Ja),
"app.language.portuguese" => Some(crate::i18n::Locale::PtBr),
_ => None,
}
}
+13 -6
View File
@@ -242,18 +242,25 @@ fn sort_menu(app: &mut LightcraftApp, ui: &mut egui::Ui) {
("File Name", FileName, "fileName"),
("Rating", Rating, "rating"),
("File Size", FileSize, "fileSize"),
("Random", Random, "random"),
] {
if ui.selectable_label(cur.key == key, crate::i18n::tr(label)).clicked() {
let _ = app.run("library.sort", json!({"key": k}));
}
}
if cur.key == Random && ui.button(crate::i18n::tr("Reshuffle")).clicked() {
let _ = app.run("library.shuffle", json!({}));
}
ui.separator();
if ui.selectable_label(cur.ascending, crate::i18n::tr("Ascending")).clicked() {
let _ = app.run("library.sort", json!({"ascending": true}));
}
if ui.selectable_label(!cur.ascending, crate::i18n::tr("Descending")).clicked() {
let _ = app.run("library.sort", json!({"ascending": false}));
}
// a shuffle has no direction worth choosing
ui.add_enabled_ui(cur.key != Random, |ui| {
if ui.selectable_label(cur.key != Random && cur.ascending, crate::i18n::tr("Ascending")).clicked() {
let _ = app.run("library.sort", json!({"ascending": true}));
}
if ui.selectable_label(cur.key != Random && !cur.ascending, crate::i18n::tr("Descending")).clicked() {
let _ = app.run("library.sort", json!({"ascending": false}));
}
});
ui.separator();
ui.label(egui::RichText::new(crate::i18n::tr("Group by date")).weak());
for (label, g, k) in [
+46 -23
View File
@@ -54,6 +54,9 @@ pub(crate) fn rename_preview(
(read(&rows), total)
}
/// About dialog tabs: (widget id suffix, label). The credits come from `crate::credits`.
pub const ABOUT_TABS: &[(&str, &str)] = &[("about", "About"), ("contributors", "Contributors"), ("models", "Models")];
/// Help ▸ What's New (docs/whats-new.md).
pub const WHATS_NEW: &str = include_str!("../../../../docs/whats-new.md");
@@ -708,27 +711,46 @@ pub fn show(app: &mut LightcraftApp, ctx: &egui::Context) {
ui.label(egui::RichText::new(crate::i18n::tr("They can be restored from Recently Deleted until it is emptied.")).color(t.text_dim));
}
Dialog::About => {
ui.label(egui::RichText::new("LightCraft").font(t.semibold(20.0)).color(t.text));
ui.label(crate::i18n::tr_format!("Version {} — a clean-room, pure-Rust photo library and raw developer.", env!("CARGO_PKG_VERSION")));
ui.label(format!("MIT OR Apache-2.0. Fonts: {} (OFL). Icons: original.", crate::theme::font_credits()));
ui.add_space(10.0);
let discord = egui::Button::new(egui::RichText::new(crate::i18n::tr("Join the ArtCraft Discord")).font(t.semibold(15.0)).color(egui::Color32::WHITE))
.fill(t.accent)
.min_size(egui::vec2(260.0, 34.0));
let r = ui.add(discord).on_hover_text(crate::links::DISCORD);
crate::widgets::register(ui.ctx(), "button:aboutDiscord", r.rect);
if r.clicked() {
let _ = crate::links::open(app, crate::links::DISCORD);
}
ui.add_space(6.0);
for (label, url) in [
("LightCraft website", crate::links::APP_PAGE),
("Source code on GitHub", crate::links::GITHUB),
("ArtCraft — more creative apps", crate::links::WEBSITE),
] {
let r = ui.link(crate::i18n::tr(label)).on_hover_text(url);
if r.clicked() {
let _ = crate::links::open(app, url);
ui.set_min_width(680.0);
let tab_id = egui::Id::new("about_tab");
let mut tab = ui.data_mut(|d| d.get_temp::<u8>(tab_id)).unwrap_or(0);
ui.horizontal(|ui| {
ui.spacing_mut().item_spacing.x = 4.0;
for (i, (id, label)) in ABOUT_TABS.iter().enumerate() {
if crate::widgets::text_button(ui, &format!("aboutTab-{id}"), label, usize::from(tab) == i).clicked() {
tab = u8::try_from(i).unwrap_or(0);
}
}
});
ui.data_mut(|d| d.insert_temp(tab_id, tab));
ui.separator();
match tab {
1 => crate::credits::contributors_ui(app, ui),
2 => crate::credits::models_ui(ui),
_ => {
ui.label(egui::RichText::new("LightCraft").font(t.semibold(20.0)).color(t.text));
ui.label(crate::i18n::tr_format!("Version {} — a clean-room, pure-Rust photo library and raw developer.", env!("CARGO_PKG_VERSION")));
ui.label(format!("MIT OR Apache-2.0. Fonts: {} (OFL). Icons: original.", crate::theme::font_credits()));
ui.add_space(10.0);
let discord = egui::Button::new(egui::RichText::new(crate::i18n::tr("Join the ArtCraft Discord")).font(t.semibold(15.0)).color(egui::Color32::WHITE))
.fill(t.accent)
.min_size(egui::vec2(260.0, 34.0));
let r = ui.add(discord).on_hover_text(crate::links::DISCORD);
crate::widgets::register(ui.ctx(), "button:aboutDiscord", r.rect);
if r.clicked() {
let _ = crate::links::open(app, crate::links::DISCORD);
}
ui.add_space(6.0);
for (label, url) in [
("LightCraft website", crate::links::APP_PAGE),
("Source code on GitHub", crate::links::GITHUB),
("ArtCraft — more creative apps", crate::links::WEBSITE),
] {
let r = ui.link(crate::i18n::tr(label)).on_hover_text(url);
if r.clicked() {
let _ = crate::links::open(app, url);
}
}
}
}
}
@@ -824,8 +846,9 @@ pub fn show(app: &mut LightcraftApp, ctx: &egui::Context) {
}
if confirm {
match confirm_dialog(app, &dlg) {
// the import review stays open on an error (e.g. an unusable folder template)
Err(e) if matches!(dlg, Dialog::Import { .. }) => app.toast(ctx, e),
// the import review stays open on an error (e.g. an unusable folder template), and so
// does Export (e.g. no folder chosen) so the choices aren't lost
Err(e) if matches!(dlg, Dialog::Import { .. } | Dialog::Export { .. }) => app.toast(ctx, e),
// the SAM 3 dialog stays open to show the download (or why it can't start)
Err(e) if matches!(dlg, Dialog::SamModel { .. }) => {
if let Dialog::SamModel { error, .. } = &mut dlg {
+13
View File
@@ -437,8 +437,21 @@ fn group_header(app: &mut LightcraftApp, ui: &mut egui::Ui, run: &DateRun, ids:
/// first gets a stand-in (its cached thumbnail, else its embedded camera preview), and its real
/// render then follows in the background.
pub fn request_thumb(app: &mut LightcraftApp, id: PhotoId, size: usize, priority: u32) {
let bucket = lightcraft_engine::media::thumb_bucket(size);
if let Some(photo) = app.session.catalog.photo(id)
&& app.renderer.thumb_current(photo, bucket, priority)
{
return;
}
let Some(job) = app.session.thumb_job(id, size) else { return };
#[cfg(test)]
{
app.renderer.thumb_jobs_built += 1;
}
let quick = if app.renderer.textures.contains_key(&Slot::Thumb(id)) { None } else { app.session.quick_thumb_job(&job) };
if let Some(photo) = app.session.catalog.photo(id) {
app.renderer.remember_thumb(photo, bucket, job.key, quick.is_some());
}
match quick {
Some(q) => {
app.renderer.request_quick(Slot::ThumbQuick(id), q, priority + 1);
+87 -17
View File
@@ -55,6 +55,31 @@ fn row(
resp
}
/// A collapsible section header (Albums, Local, By Date, Keywords): the bold title with a
/// disclosure chevron after it; a click folds or unfolds the section (kept in the UI state, so it
/// survives restarts). Returns the header's rect and whether the section is now open.
fn sidebar_section_header(app: &mut LightcraftApp, ui: &mut egui::Ui, id: &str, title: &str) -> (Rect, bool) {
let t = Tokens::get(ui.ctx());
let title = crate::i18n::tr(title);
let (r, resp) = ui.allocate_exact_size(vec2(ui.available_width(), 34.0), Sense::click());
register(ui.ctx(), format!("sidebarSection:{id}"), r);
if resp.clicked() {
app.ui.toggle_sidebar_section(id);
}
let open = !app.ui.sidebar_section_collapsed(id);
resp.widget_info(|| egui::WidgetInfo::selected(egui::WidgetType::CollapsingHeader, true, open, title));
let text = ui.painter().text(pos2(r.left() + 18.0, r.center().y), Align2::LEFT_CENTER, title, t.semibold(13.5), t.text_label);
let c = pos2(text.right() + 10.0, r.center().y);
let col = if resp.hovered() { t.text } else { t.text_dim };
let pts = if open {
vec![c + vec2(-4.0, -2.0), c + vec2(4.0, -2.0), c + vec2(0.0, 3.0)]
} else {
vec![c + vec2(-2.0, -4.0), c + vec2(3.0, 0.0), c + vec2(-2.0, 4.0)]
};
ui.painter().add(egui::Shape::convex_polygon(pts, col, egui::Stroke::NONE));
(r, open)
}
pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui) {
let t = Tokens::get(ui.ctx());
let frame = egui::Frame::NONE.fill(t.chrome).stroke(egui::Stroke::new(1.0, t.divider));
@@ -85,8 +110,7 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui) {
}
ui.add_space(10.0);
// Albums header
let (ar, _) = ui.allocate_exact_size(vec2(ui.available_width(), 34.0), Sense::hover());
ui.painter().text(pos2(ar.left() + 18.0, ar.center().y), Align2::LEFT_CENTER, crate::i18n::tr("Albums"), t.semibold(13.5), t.text_label);
let (ar, albums_open) = sidebar_section_header(app, ui, "albums", "Albums");
let mut hdr = ui.new_child(
egui::UiBuilder::new()
.max_rect(Rect::from_min_max(pos2(ar.right() - 50.0, ar.top()), ar.right_bottom()))
@@ -111,14 +135,16 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui) {
app.ui.dialog = Some(crate::state::Dialog::NewAlbum { name: String::new(), folder: true });
}
});
let albums: Vec<Album> = app.session.catalog.albums().cloned().collect();
albums_tree(app, ui, &albums, None, 0.0);
if albums_open {
let albums: Vec<Album> = app.session.catalog.albums().cloned().collect();
albums_tree(app, ui, &albums, None, 0.0);
}
ui.add_space(10.0);
local_section(app, ui);
// By date
let (dr, _) = ui.allocate_exact_size(vec2(ui.available_width(), 34.0), Sense::hover());
ui.painter().text(pos2(dr.left() + 18.0, dr.center().y), Align2::LEFT_CENTER, crate::i18n::tr("By Date"), t.semibold(13.5), t.text_label);
for g in app.caches.date_groups(&app.session.catalog).iter() {
let (_, dates_open) = sidebar_section_header(app, ui, "byDate", "By Date");
let groups = if dates_open { app.caches.date_groups(&app.session.catalog) } else { Default::default() };
for g in groups.iter() {
// year → month → day; a click filters by that prefix, the triangle opens a level
if date_row(app, ui, &g.year, &crate::i18n::date_group_label(&g.year, true), g.count, 0.0) {
for (m, n) in &g.months {
@@ -190,9 +216,7 @@ fn local_section(app: &mut LightcraftApp, ui: &mut egui::Ui) {
if cfg!(target_arch = "wasm32") {
return;
}
let t = Tokens::get(ui.ctx());
let (lr, _) = ui.allocate_exact_size(vec2(ui.available_width(), 34.0), Sense::hover());
ui.painter().text(pos2(lr.left() + 18.0, lr.center().y), Align2::LEFT_CENTER, crate::i18n::tr("Local"), t.semibold(13.5), t.text_label);
let (_, open) = sidebar_section_header(app, ui, "local", "Local");
let home = std::env::var("HOME").or_else(|_| std::env::var("USERPROFILE")).unwrap_or_default();
let mut builtin: Vec<(String, String)> = Vec::new();
if !home.is_empty() {
@@ -211,6 +235,10 @@ fn local_section(app: &mut LightcraftApp, ui: &mut egui::Ui) {
if current.is_some() {
app.ui.local_browse_root = local.browse_root.clone();
}
if !open {
ui.add_space(10.0);
return;
}
for (i, (name, path)) in local.places.iter().enumerate() {
let transient = local.browse_root.as_deref() == Some(path.as_str());
let reveal = local.owner == Some(i);
@@ -254,7 +282,7 @@ pub(crate) struct LocalPlaces {
/// (label, path) of each top-level folder, in order.
pub places: Vec<(String, String)>,
/// The top-level folder the browsed folder lies in (the innermost one): its tree opens on
/// the way down to it.
/// the way down to it. None when that way passes through a hidden folder.
pub owner: Option<usize>,
/// A folder listed only for this session because the browsed folder is in no saved
/// location (browsed from a breadcrumb, the CLI…); it stays while browsing below it.
@@ -299,7 +327,15 @@ pub(crate) fn local_places(
}
}
if let Some(c) = browsing {
out.owner = places.iter().enumerate().filter(|(_, (_, p))| folder_within(c, p)).max_by_key(|(_, (_, p))| folder_key(p).len()).map(|(i, _)| i);
// A tree never opens on the way down through a hidden folder: hiding a kept folder
// beneath Home would otherwise reveal it again inside Home's (possibly huge) tree.
let through_hidden = |p: &str| hidden.iter().any(|h| folder_within(c, h) && folder_within(h, p));
out.owner = places
.iter()
.enumerate()
.filter(|(_, (_, p))| folder_within(c, p) && !through_hidden(p))
.max_by_key(|(_, (_, p))| folder_key(p).len())
.map(|(i, _)| i);
}
out.places = places;
out
@@ -327,6 +363,17 @@ fn list_subfolders(path: &str) -> Vec<(String, String)> {
v
}
/// How many [`fs_cached`] answers for `ctx` are being worked out right now. Rows appear (and
/// the sidebar below them moves) when they land, so the headless driver counts them as pending
/// work and waits for them before acting on widget positions.
pub(crate) fn fs_cached_running(ctx: &egui::Context) -> usize {
fs_running_counter(ctx).load(std::sync::atomic::Ordering::Acquire)
}
fn fs_running_counter(ctx: &egui::Context) -> std::sync::Arc<std::sync::atomic::AtomicUsize> {
ctx.data_mut(|d| d.get_temp_mut_or_default::<std::sync::Arc<std::sync::atomic::AtomicUsize>>(egui::Id::new("fs-cached-running")).clone())
}
/// A file-system answer for `path` (`f(path)`), kept per `kind` and path and refreshed on a worker
/// thread at most every `every` seconds: a folder on a sleeping NAS, a dropped share or a
/// spinning-up drive never blocks a frame. `None` until the first answer arrives.
@@ -358,18 +405,24 @@ pub(crate) fn fs_cached<T: Clone + Send + 'static>(ui: &egui::Ui, kind: &'static
due
};
if start {
let (out, path, repaint) = (cell.clone(), path.to_string(), ui.ctx().clone());
use std::sync::atomic::Ordering;
let running = fs_running_counter(ui.ctx());
running.fetch_add(1, Ordering::AcqRel);
let (out, path, repaint, done) = (cell.clone(), path.to_string(), ui.ctx().clone(), running.clone());
let work = move || {
let v = f(&path);
let mut e = out.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
e.value = Some(v);
e.running = false;
drop(e);
// after the answer is stored: a frame that sees the count drop also sees the answer
done.fetch_sub(1, Ordering::AcqRel);
repaint.request_repaint();
};
#[cfg(not(target_arch = "wasm32"))]
if std::thread::Builder::new().name("lc-fs-list".into()).spawn(work).is_err() {
cell.lock().unwrap_or_else(std::sync::PoisonError::into_inner).running = false;
running.fetch_sub(1, Ordering::AcqRel);
}
#[cfg(target_arch = "wasm32")]
work();
@@ -666,15 +719,14 @@ fn is_within(app: &LightcraftApp, id: lightcraft_catalog::AlbumId, ancestor: lig
/// filters the grid by the keyword (children included), the triangle opens a level, and the
/// context menu renames, merges or deletes the keyword across the library.
fn keywords_section(app: &mut LightcraftApp, ui: &mut egui::Ui) {
let t = Tokens::get(ui.ctx());
let tree = app.caches.keyword_tree(&app.session.catalog);
if tree.is_empty() {
return;
}
ui.add_space(10.0);
let (kr, _) = ui.allocate_exact_size(vec2(ui.available_width(), 34.0), Sense::hover());
ui.painter().text(pos2(kr.left() + 18.0, kr.center().y), Align2::LEFT_CENTER, crate::i18n::tr("Keywords"), t.semibold(13.5), t.text_label);
keyword_rows(app, ui, &tree, 0.0);
if sidebar_section_header(app, ui, "keywords", "Keywords").1 {
keyword_rows(app, ui, &tree, 0.0);
}
}
fn keyword_rows(app: &mut LightcraftApp, ui: &mut egui::Ui, nodes: &[KeywordNode], indent: f32) {
@@ -803,4 +855,22 @@ mod tests {
let l = local_places(Vec::new(), &[], Some("/t/base"), None, &["/t/base/".into()]);
assert!(l.places.is_empty() && l.browse_root.is_none());
}
/// A hidden folder inside a listed one (a kept folder beneath Home) is not revealed in that
/// one's tree while it is browsed, nor are the folders below it; hiding Home itself still
/// lets Pictures open down to a folder browsed inside it.
#[test]
fn hidden_folder_is_not_revealed_in_an_outer_tree() {
let builtin = || vec![("Pictures".to_string(), "/home/example/Pictures".to_string()), ("Home".to_string(), "/home/example".to_string())];
let kept = ["/home/example/AppData/Temp/lc".to_string()];
let l = local_places(builtin(), &kept, Some("/home/example/AppData/Temp/lc"), None, &[]);
assert_eq!(l.owner, Some(2), "shown as its own kept row");
for browsing in ["/home/example/AppData/Temp/lc", "/home/example/AppData/Temp/lc/Day 1"] {
let l = local_places(builtin(), &kept, Some(browsing), None, &["/home/example/AppData/Temp/lc/".into()]);
assert_eq!(names(&l.places), ["Pictures", "Home"], "{browsing}");
assert_eq!((l.owner, l.browse_root.as_deref()), (None, None), "Home does not open down to {browsing}");
}
let l = local_places(builtin(), &[], Some("/home/example/Pictures/Trip"), None, &["/home/example".into()]);
assert_eq!((names(&l.places), l.owner), (vec!["Pictures"], Some(0)));
}
}
+520 -14
View File
@@ -85,6 +85,19 @@ pub const VARIANT_TEXTURES: usize = 96;
const VARIANT_PRIORITY: u32 = 6;
pub struct Renderer {
#[cfg(test)]
pub(crate) thumb_jobs_built: usize,
request_ids: HashMap<Slot, u64>,
/// Request id of the pixels each interactive view (loupe, before, hover) shows: a draft that
/// finishes after a newer request was made still replaces older pixels, so a long slider drag
/// keeps updating the loupe instead of waiting for the drag to pause.
shown_ids: HashMap<Slot, u64>,
/// Requests made before this id belong to a library or preview cache that is gone.
epoch: u64,
preview_generation: Option<(std::sync::Weak<lightcraft_preview::PreviewCache>, u64)>,
/// Last thumbnail inputs; weak identities do not retain photo histories or force deep clones.
/// (photo, size bucket, job key, whether the request went through an embedded stand-in)
thumb_inputs: HashMap<PhotoId, (std::sync::Weak<lightcraft_catalog::Photo>, usize, u64, bool)>,
pool: JobPool<Slot, RenderResult>,
/// Jobs run elsewhere (see [`RenderOffload`]); `queue` holds the ones not started yet.
offload: Option<Box<dyn RenderOffload>>,
@@ -127,8 +140,24 @@ const IDLE_TRIM: std::time::Duration = std::time::Duration::from_secs(3);
impl Default for Renderer {
fn default() -> Self {
let threads = if cfg!(target_arch = "wasm32") { 0 } else { JobPool::<Slot, RenderResult>::default_threads().min(6) };
Self::with_worker_threads(threads)
}
}
impl Renderer {
/// Explicit worker count for hosts and bounded validation runs (one to six on native).
/// Browser hosts continue to use inline jobs or their installed worker offload.
pub fn with_worker_threads(threads: usize) -> Self {
let threads = if cfg!(target_arch = "wasm32") { 0 } else { threads.clamp(1, 6) };
Renderer {
#[cfg(test)]
thumb_jobs_built: 0,
request_ids: HashMap::new(),
shown_ids: HashMap::new(),
epoch: 0,
preview_generation: None,
pool: JobPool::new(threads),
thumb_inputs: HashMap::new(),
offload: None,
queue: Vec::new(),
seq: 0,
@@ -151,6 +180,45 @@ impl Default for Renderer {
}
impl Renderer {
/// Would asking for this thumbnail again (at `priority`) change nothing?
pub fn thumb_current(&self, photo: &Arc<lightcraft_catalog::Photo>, bucket: usize, priority: u32) -> bool {
let Some(&(ref old, size, key, quick)) = self.thumb_inputs.get(&photo.id) else { return false };
if old.as_ptr() != Arc::as_ptr(photo) || size != bucket {
return false;
}
let slot = Slot::Thumb(photo.id);
if quick && !self.textures.contains_key(&slot) {
// Stand-in path (`grid::request_thumb`): the embedded preview is still on its way, or
// the real thumbnail is requested at the background priority whatever the caller's.
return self.pending.get(&Slot::ThumbQuick(photo.id)).is_some_and(|p| p.0 == key)
|| !self.request_needed(slot, key, crate::panels::grid::BACKGROUND_THUMB_PRIORITY);
}
!self.request_needed(slot, key, priority)
}
/// Remember the inputs of the thumbnail just requested (`quick`: through a stand-in).
pub fn remember_thumb(&mut self, photo: &Arc<lightcraft_catalog::Photo>, bucket: usize, key: u64, quick: bool) {
if self.thumb_inputs.len() >= 1024 && !self.thumb_inputs.contains_key(&photo.id) {
self.thumb_inputs.retain(|_, (p, _, _, _)| p.strong_count() > 0);
if self.thumb_inputs.len() >= 1024
&& let Some(id) = self.thumb_inputs.keys().next().copied()
{
self.thumb_inputs.remove(&id);
}
}
self.thumb_inputs.insert(photo.id, (Arc::downgrade(photo), bucket, key, quick));
}
fn request_needed(&self, slot: Slot, key: u64, priority: u32) -> bool {
// A matching old texture must not mask a different newer request (e.g. undo).
if let Some(&(k, p)) = self.pending.get(&slot) {
return k != key || (p != priority && self.is_queued(slot));
}
if self.textures.get(&slot).is_some_and(|t| t.key == key) || self.failed.get(&slot).is_some_and(|f| f.0 == key && f.2 == self.catalog_rev) {
return false;
}
true
}
/// Run jobs through `offload` from now on (instead of the job pool / inline).
pub fn set_offload(&mut self, offload: Box<dyn RenderOffload>) {
self.offload = Some(offload);
@@ -167,23 +235,17 @@ impl Renderer {
}
pub fn is_current(&self, slot: Slot, key: u64) -> bool {
self.textures.get(&slot).is_some_and(|t| t.key == key) || self.pending.get(&slot).is_some_and(|p| p.0 == key)
self.wanted(slot) == Some(key)
}
/// Request a render for `slot` (no-op if already current or pending at the same priority).
pub fn request(&mut self, slot: Slot, job: RenderJob, priority: u32) {
if self.textures.get(&slot).is_some_and(|t| t.key == job.key)
|| self.failed.get(&slot).is_some_and(|f| f.0 == job.key && f.2 == self.catalog_rev)
{
return;
}
if let Some(&(key, prio)) = self.pending.get(&slot)
&& key == job.key
&& (prio == priority || !self.is_queued(slot))
{
pub fn request(&mut self, slot: Slot, mut job: RenderJob, priority: u32) {
if !self.request_needed(slot, job.key, priority) {
return;
}
self.pending.insert(slot, (job.key, priority));
job.request_id = lightcraft_preview::next_tick();
self.request_ids.insert(slot, job.request_id);
// (an offload keeps its own per-view stage caches; the flag tells it to)
let job =
if matches!(slot, Slot::Main | Slot::Before | Slot::Hover) { job.with_stages(self.stages.entry(slot).or_default().clone()) } else { job };
@@ -205,24 +267,28 @@ impl Renderer {
}
/// Request a stand-in for `slot` (once per job key).
pub fn request_quick(&mut self, slot: Slot, job: QuickJob, priority: u32) {
pub fn request_quick(&mut self, slot: Slot, mut job: QuickJob, priority: u32) {
if self.quick_tried.get(&slot) == Some(&job.key) {
return;
}
self.quick_tried.insert(slot, job.key);
self.pending.insert(slot, (job.key, priority));
job.request_id = lightcraft_preview::next_tick();
self.request_ids.insert(slot, job.request_id);
let key = job.key;
self.pool.submit(slot, key, priority, Box::new(move || job.run()));
}
/// Prepare a photo in the background (a [`Slot::Prefetch`] job runs once per key; a newer one
/// replaces it while queued). Memory stays bounded by the engine's source caches.
pub fn prefetch(&mut self, slot: Slot, job: RenderJob, priority: u32) {
pub fn prefetch(&mut self, slot: Slot, mut job: RenderJob, priority: u32) {
if self.prefetched.get(&slot) == Some(&job.key) {
return;
}
self.prefetched.insert(slot, job.key);
self.pending.insert(slot, (job.key, priority));
job.request_id = lightcraft_preview::next_tick();
self.request_ids.insert(slot, job.request_id);
let key = job.key;
self.pool.submit(slot, key, priority, Box::new(move || lightcraft_engine::memory::in_background(|| job.run())));
}
@@ -230,6 +296,12 @@ impl Renderer {
/// Is a request for `slot` queued or running?
/// Drop every texture and cached stage (another library was opened: photo ids changed meaning).
pub fn forget_all(&mut self) {
self.request_ids.clear();
self.shown_ids.clear();
self.epoch = lightcraft_preview::next_tick();
self.preview_generation = None;
self.pool.reprioritize(|_, _| None);
self.thumb_inputs.clear();
self.textures.clear();
self.stages.clear();
self.quick_tried.clear();
@@ -311,11 +383,14 @@ impl Renderer {
});
for s in dropped {
self.pending.remove(&s);
self.request_ids.remove(&s);
}
let request_ids = &mut self.request_ids;
let pending = &mut self.pending;
self.queue.retain(|q| match q.slot {
Slot::Variant(k) if stale(used.get(&k)) => {
pending.remove(&q.slot);
request_ids.remove(&q.slot);
false
}
_ => true,
@@ -346,6 +421,12 @@ impl Renderer {
/// Collect finished jobs into textures. Returns true if anything changed.
pub fn poll(&mut self, ctx: &egui::Context, session: &mut Session) -> bool {
let cache = &session.media.rendered;
let generation = cache.generation();
if self.preview_generation.as_ref().is_some_and(|(old, g)| old.as_ptr() != Arc::as_ptr(cache) || *g != generation) {
self.forget_all();
}
self.preview_generation = Some((Arc::downgrade(cache), generation));
self.catalog_rev = session.catalog.revision;
self.frame += 1;
// wasm: run one job per frame on this thread, timed with the host clock
@@ -367,8 +448,25 @@ impl Renderer {
self.dispatch();
let mut changed = false;
for (mut slot, r, ms) in finished {
session.accept(&r);
self.completed += 1;
// Ignore obsolete pixels, failures and decoded sources, even for the same render key;
// an interactive view still takes a superseded draft that is newer than what it shows.
if self.request_ids.get(&slot) != Some(&r.request_id) {
let newer_draft = matches!(slot, Slot::Main | Slot::Before | Slot::Hover)
&& self.request_ids.contains_key(&slot)
&& r.request_id >= self.epoch
&& self.shown_ids.get(&slot).is_none_or(|shown| r.request_id > *shown)
&& r.rendered.is_ok();
if !newer_draft {
continue;
}
} else {
self.request_ids.remove(&slot);
session.accept(&r);
}
if matches!(slot, Slot::Main | Slot::Before | Slot::Hover) {
self.shown_ids.insert(slot, r.request_id);
}
if self.pending.get(&slot).is_some_and(|p| p.0 == r.key) {
self.pending.remove(&slot);
}
@@ -488,6 +586,10 @@ impl Renderer {
/// Drop the import review thumbnails (a new review, or the dialog closed).
pub fn forget_imports(&mut self) {
self.pool.reprioritize(|s, p| if matches!(s, Slot::Import(_)) { None } else { Some(p) });
self.queue.retain(|q| !matches!(q.slot, Slot::Import(_)));
self.pending.retain(|s, _| !matches!(s, Slot::Import(_)));
self.request_ids.retain(|s, _| !matches!(s, Slot::Import(_)));
self.textures.retain(|s, _| !matches!(s, Slot::Import(_)));
self.quick_tried.retain(|s, _| !matches!(s, Slot::Import(_)));
}
@@ -495,18 +597,22 @@ impl Renderer {
/// Drop thumbnails that aren't in `keep` (bounded memory for huge libraries), and queued
/// thumbnail jobs for photos that scrolled out of `keep`.
pub fn evict_thumbs(&mut self, keep: &std::collections::HashSet<PhotoId>, max: usize) {
self.thumb_inputs.retain(|id, _| keep.contains(id));
let dropped = self.pool.reprioritize(|s, p| match s {
Slot::Thumb(id) | Slot::ThumbQuick(id) if !keep.contains(id) && p <= 11 => None,
_ => Some(p),
});
for s in dropped {
self.pending.remove(&s);
self.request_ids.remove(&s);
self.quick_tried.remove(&s);
}
let request_ids = &mut self.request_ids;
let pending = &mut self.pending;
self.queue.retain(|q| match q.slot {
Slot::Thumb(id) if !keep.contains(&id) && q.priority <= 10 => {
pending.remove(&q.slot);
request_ids.remove(&q.slot);
false
}
_ => true,
@@ -519,3 +625,403 @@ impl Renderer {
self.quick_tried.retain(|s, _| !matches!(s, Slot::ThumbQuick(id) if !keep.contains(id)));
}
}
#[cfg(test)]
mod thumbnail_tests {
use super::*;
use lightcraft_catalog::{Catalog, Op, Photo, Source};
struct Blocked;
impl RenderOffload for Blocked {
fn try_start(&mut self, _: Slot, job: RenderJob) -> Option<RenderJob> {
Some(job)
}
fn finished(&mut self) -> Vec<(Slot, RenderResult, f64)> {
vec![]
}
}
fn photo() -> Arc<Photo> {
Arc::new(Photo::new(PhotoId(1), Source::File { path: "audit.jpg".into() }, "audit.jpg", "JPEG", 6000, 4000, "2026-01-01"))
}
#[test]
fn snapshot_tracks_cow_edits_undo_reload_and_replacement() {
let mut catalog = Catalog::default();
catalog.apply(Op::AddPhoto { photo: Box::new((*photo()).clone()) }).unwrap();
let old = catalog.photo(PhotoId(1)).unwrap().clone();
let mut r = Renderer::default();
r.remember_thumb(&old, 384, 77, false);
r.pending.insert(Slot::Thumb(PhotoId(1)), (77, 10));
assert!(r.thumb_current(&old, 384, 10));
assert!(!r.thumb_current(&old, 512, 10));
let mut settings = lightcraft_develop::DevelopSettings::default();
settings.light.exposure = 1.0;
let undo = catalog.apply(Op::SetDevelop { id: PhotoId(1), settings: Arc::new(settings), label: "edit".into(), edited: None }).unwrap();
assert!(!r.thumb_current(catalog.photo(PhotoId(1)).unwrap(), 384, 10));
catalog.apply(undo).unwrap();
assert!(!r.thumb_current(catalog.photo(PhotoId(1)).unwrap(), 384, 10));
catalog
.apply(Op::SetContent {
id: PhotoId(1),
width: 6000,
height: 4000,
file_size: 99,
content_hash: Some("changed".into()),
preview_only: None,
})
.unwrap();
assert!(!r.thumb_current(catalog.photo(PhotoId(1)).unwrap(), 384, 10));
assert!(!r.thumb_current(&photo(), 384, 10));
r.forget_all();
assert!(r.thumb_inputs.is_empty());
}
#[test]
fn queue_promotions_running_work_and_cancellation() {
let p = photo();
let slot = Slot::Thumb(p.id);
let mut session = Session::new();
session.catalog.apply(Op::AddPhoto { photo: Box::new((*p).clone()) }).unwrap();
let job = session.thumb_job(p.id, 384).unwrap();
let key = job.key;
let mut r = Renderer::default();
r.set_offload(Box::new(Blocked));
r.remember_thumb(&p, 384, key, false);
r.request(slot, job, 5);
assert!(r.thumb_current(&p, 384, 5));
assert!(!r.thumb_current(&p, 384, 10));
r.request(slot, session.thumb_job(p.id, 384).unwrap(), 10);
assert_eq!(r.pending.get(&slot), Some(&(key, 10)));
r.queue.clear(); // running work: cannot reprioritize, just as Renderer::request
assert!(r.thumb_current(&p, 384, 5));
r.pending.clear();
assert!(!r.thumb_current(&p, 384, 10));
r.evict_thumbs(&Default::default(), 600);
assert!(r.thumb_inputs.is_empty());
}
#[test]
fn failures_retry_after_revision_and_quick_only_is_not_final() {
let p = photo();
let slot = Slot::Thumb(p.id);
let mut r = Renderer::default();
r.remember_thumb(&p, 384, 77, false);
r.pending.insert(Slot::ThumbQuick(p.id), (77, 11));
assert!(!r.thumb_current(&p, 384, 10));
r.failed.insert(slot, (77, "no decoder".into(), 0));
assert!(r.thumb_current(&p, 384, 10));
r.catalog_rev = 1;
assert!(!r.thumb_current(&p, 384, 10));
}
#[derive(Default)]
struct Work {
jobs: Vec<(Slot, RenderJob)>,
finished: Vec<(Slot, RenderResult, f64)>,
}
struct Manual(std::rc::Rc<std::cell::RefCell<Work>>);
impl RenderOffload for Manual {
fn try_start(&mut self, slot: Slot, job: RenderJob) -> Option<RenderJob> {
self.0.borrow_mut().jobs.push((slot, job));
None
}
fn finished(&mut self) -> Vec<(Slot, RenderResult, f64)> {
std::mem::take(&mut self.0.borrow_mut().finished)
}
}
fn fixture() -> (crate::LightcraftApp, std::rc::Rc<std::cell::RefCell<Work>>, egui::Context) {
fixture_with(Session::new(), (*photo()).clone())
}
fn fixture_with(mut s: Session, photo: Photo) -> (crate::LightcraftApp, std::rc::Rc<std::cell::RefCell<Work>>, egui::Context) {
s.media.file_loader = Some(Arc::new(|_, _| {
let mut image = lightcraft_raster::Rgb32f::new(8, 8);
image.data.fill([0.3, 0.2, 0.1]);
Ok((image, Default::default()))
}));
s.catalog.apply(Op::AddPhoto { photo: Box::new(photo) }).unwrap();
let mut app = crate::LightcraftApp::new(s, crate::Services::default());
app.renderer.keep_pixels = true;
let work = std::rc::Rc::new(std::cell::RefCell::new(Work::default()));
app.renderer.set_offload(Box::new(Manual(work.clone())));
let ctx = egui::Context::default();
app.renderer.poll(&ctx, &mut app.session);
(app, work, ctx)
}
fn take_job(work: &std::rc::Rc<std::cell::RefCell<Work>>) -> (Slot, RenderJob) {
work.borrow_mut().jobs.pop().unwrap()
}
fn finish(app: &mut crate::LightcraftApp, work: &std::rc::Rc<std::cell::RefCell<Work>>, ctx: &egui::Context, slot: Slot, result: RenderResult) {
work.borrow_mut().finished.push((slot, result, 0.0));
app.renderer.poll(ctx, &mut app.session);
}
fn request(app: &mut crate::LightcraftApp) {
crate::panels::grid::request_thumb(app, PhotoId(1), 256, 10);
}
#[test]
fn unchanged_ready_and_running_frames_build_no_jobs() {
let (mut app, work, ctx) = fixture();
request(&mut app);
assert_eq!(app.renderer.thumb_jobs_built, 1);
for _ in 0..180 {
request(&mut app);
}
assert_eq!(app.renderer.thumb_jobs_built, 1);
let (slot, job) = take_job(&work);
finish(&mut app, &work, &ctx, slot, job.run());
for _ in 0..180 {
request(&mut app);
}
assert_eq!(app.renderer.thumb_jobs_built, 1);
assert!(work.borrow().jobs.is_empty());
}
#[test]
fn unchanged_queued_frames_build_no_jobs() {
let (mut app, _, _) = fixture();
app.renderer.set_offload(Box::new(Blocked));
request(&mut app);
for _ in 0..180 {
request(&mut app);
}
assert_eq!(app.renderer.thumb_jobs_built, 1);
assert_eq!(app.renderer.queue.len(), 1);
}
#[test]
fn resizing_the_disk_cache_keeps_textures_and_skips_rebuilds() {
let lib = std::env::temp_dir().join(format!("lc-ui-thumb-resize-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&lib);
let mut s = Session::new().with_fs();
s.open_library(&lib, false).unwrap();
let (mut app, work, ctx) = fixture_with(s, (*photo()).clone());
request(&mut app);
let (slot, job) = take_job(&work);
finish(&mut app, &work, &ctx, slot, job.run());
assert!(app.renderer.thumb(PhotoId(1)).is_some());
let cache = app.session.media.rendered.clone();
app.session.execute("library.preferences", &serde_json::json!({"cacheMb": 300})).unwrap();
app.renderer.poll(&ctx, &mut app.session);
// thumbnails are keyed by content: a new budget leaves them (and the cache) valid
assert!(app.renderer.thumb(PhotoId(1)).is_some());
assert!(Arc::ptr_eq(&cache, &app.session.media.rendered));
for _ in 0..30 {
request(&mut app);
}
assert_eq!(app.renderer.thumb_jobs_built, 1);
assert!(work.borrow().jobs.is_empty());
// an explicit clear still drops them
app.session.execute("library.clearPreviews", &serde_json::json!({})).unwrap();
app.renderer.poll(&ctx, &mut app.session);
assert!(app.renderer.thumb(PhotoId(1)).is_none());
drop(app);
let _ = std::fs::remove_dir_all(&lib);
}
#[test]
fn raw_quick_path_frames_build_no_jobs() {
// an unedited raw: the grid shows its embedded preview first (held here until released),
// then queues the real thumbnail behind everything on screen
let gate = Arc::new(std::sync::Mutex::new(()));
let held = gate.lock().unwrap();
let mut s = Session::new();
let g = gate.clone();
s.media.preview_loader = Some(Arc::new(move |_: &str, _| {
drop(g.lock().unwrap_or_else(std::sync::PoisonError::into_inner));
Some(lightcraft_raster::Rgba8 { width: 4, height: 4, data: vec![[200, 100, 50, 255]; 16] })
}));
let mut raw = (*photo()).clone();
raw.kind = lightcraft_catalog::MediaKind::Raw;
raw.develop = Arc::new(raw.camera_defaults());
let (mut app, _, ctx) = fixture_with(s, raw);
app.renderer.set_offload(Box::new(Blocked));
let quick = Slot::ThumbQuick(PhotoId(1));
// the stand-in is running
for _ in 0..180 {
request(&mut app);
}
assert!(app.renderer.is_pending(quick));
assert_eq!(app.renderer.thumb_jobs_built, 1);
drop(held);
let t0 = std::time::Instant::now();
while !app.renderer.textures.contains_key(&quick) {
assert!(t0.elapsed() < std::time::Duration::from_secs(20), "the embedded preview never arrived");
std::thread::sleep(std::time::Duration::from_millis(5));
app.renderer.poll(&ctx, &mut app.session);
}
// the stand-in is shown: one more job queues the real thumbnail at background priority,
// and the grid asking at its own (higher) priority every frame does not rebuild it
for _ in 0..180 {
request(&mut app);
}
assert_eq!(app.renderer.thumb_jobs_built, 2);
assert_eq!(app.renderer.queue.len(), 1);
assert_eq!(app.renderer.pending.get(&Slot::Thumb(PhotoId(1))).map(|p| p.1), Some(crate::panels::grid::BACKGROUND_THUMB_PRIORITY));
}
#[test]
fn closing_import_review_cancels_queued_and_running_identities() {
let (mut app, work, ctx) = fixture();
let job = app.session.thumb_job(PhotoId(1), 256).unwrap();
app.renderer.request(Slot::Import(0), job.clone(), 10);
let (slot, running) = take_job(&work);
app.renderer.set_offload(Box::new(Blocked));
app.renderer.request(Slot::Import(1), job, 10);
app.renderer.forget_imports();
assert!(!app.renderer.is_pending(Slot::Import(0)));
assert!(!app.renderer.is_pending(Slot::Import(1)));
assert!(app.renderer.queue.is_empty());
work.borrow_mut().finished.push((slot, running.run(), 0.0));
app.renderer.set_offload(Box::new(Manual(work.clone())));
app.renderer.poll(&ctx, &mut app.session);
assert!(!app.renderer.textures.contains_key(&slot));
assert_eq!(app.session.media.source_usage().0, 0);
}
#[test]
fn loupe_shows_superseded_drafts_newer_than_its_pixels_during_a_drag() {
let (mut app, work, ctx) = fixture();
let mut jobs = Vec::new();
for exposure in [0.0, 1.0, 2.0] {
let mut edit = lightcraft_develop::DevelopSettings::default();
edit.light.exposure = exposure;
app.session.catalog.apply(Op::SetDevelop { id: PhotoId(1), settings: Arc::new(edit), label: "drag".into(), edited: None }).unwrap();
let job = app.session.loupe_job(PhotoId(1), 8, 8, true).unwrap().draft();
app.renderer.request(Slot::Main, job, 100);
jobs.push(take_job(&work).1);
}
let keys: Vec<u64> = jobs.iter().map(|j| j.key).collect();
let mut jobs = jobs.into_iter();
let (first, second, last) = (jobs.next().unwrap(), jobs.next().unwrap(), jobs.next().unwrap());
// the first draft lands while the newest is still rendering: the loupe moves on
finish(&mut app, &work, &ctx, Slot::Main, first.run());
assert_eq!(app.renderer.textures.get(&Slot::Main).map(|t| t.key), Some(keys[0]));
assert!(app.renderer.is_pending(Slot::Main));
finish(&mut app, &work, &ctx, Slot::Main, last.run());
assert_eq!(app.renderer.textures.get(&Slot::Main).map(|t| t.key), Some(keys[2]));
// a draft older than the pixels shown never replaces them
finish(&mut app, &work, &ctx, Slot::Main, second.run());
assert_eq!(app.renderer.textures.get(&Slot::Main).map(|t| t.key), Some(keys[2]));
assert!(!app.renderer.is_pending(Slot::Main));
}
#[test]
fn library_reset_rejects_old_job_even_when_photo_id_and_key_are_reused() {
let (mut app, work, ctx) = fixture();
request(&mut app);
let (slot, old) = take_job(&work);
let old_result = old.run();
app.renderer.forget_all();
let photo = app.session.catalog.photo(PhotoId(1)).unwrap().as_ref().clone();
app.session = lightcraft_engine::Session::new();
app.session.catalog.apply(Op::AddPhoto { photo: Box::new(photo) }).unwrap();
app.session.media.file_loader = Some(Arc::new(|_, _| {
let mut image = lightcraft_raster::Rgb32f::new(8, 8);
image.data.fill([0.1, 0.2, 0.9]);
Ok((image, Default::default()))
}));
app.renderer.poll(&ctx, &mut app.session);
request(&mut app);
let (_, new) = take_job(&work);
assert_eq!(old_result.key, new.key);
assert_ne!(old_result.request_id, new.request_id);
let expected = new.clone().run().rendered.unwrap().image.as_bytes();
finish(&mut app, &work, &ctx, slot, new.run());
finish(&mut app, &work, &ctx, slot, old_result);
assert_eq!(
app.renderer.thumb(PhotoId(1)).unwrap().pixels.as_ref().unwrap().pixels.iter().flat_map(|c| c.to_array()).collect::<Vec<_>>(),
expected
);
}
#[test]
fn late_results_and_errors_cannot_replace_newest_pixels_or_sources() {
for order in [0, 1, 2] {
let (mut app, work, ctx) = fixture();
request(&mut app);
let (slot, old) = take_job(&work);
let mut edit = lightcraft_develop::DevelopSettings::default();
edit.light.exposure = 1.0;
app.session.catalog.apply(Op::SetDevelop { id: PhotoId(1), settings: Arc::new(edit), label: "edit".into(), edited: None }).unwrap();
request(&mut app);
let (_, new) = take_job(&work);
let expected = new.clone().run().rendered.unwrap().image.as_bytes();
if order != 0 {
finish(&mut app, &work, &ctx, slot, new.run());
// An obsolete source and error must not poison the current successful request.
let mut stale = old.run();
if order == 2 {
stale.rendered = Err("old failure".into());
}
finish(&mut app, &work, &ctx, slot, stale);
} else {
finish(&mut app, &work, &ctx, slot, old.run());
assert!(app.renderer.thumb(PhotoId(1)).is_none());
assert_eq!(app.session.media.source_usage().0, 0);
finish(&mut app, &work, &ctx, slot, new.run());
}
assert_eq!(
app.renderer.thumb(PhotoId(1)).unwrap().pixels.as_ref().unwrap().pixels.iter().flat_map(|c| c.to_array()).collect::<Vec<_>>(),
expected
);
assert!(app.renderer.failure(slot).is_none());
}
}
#[test]
fn undo_to_existing_texture_supersedes_pending_edit() {
let (mut app, work, ctx) = fixture();
request(&mut app);
let (slot, first) = take_job(&work);
let key = first.key;
finish(&mut app, &work, &ctx, slot, first.run());
let original = app.renderer.thumb(PhotoId(1)).unwrap().pixels.clone().unwrap();
let mut edit = lightcraft_develop::DevelopSettings::default();
edit.light.exposure = 1.0;
let undo =
app.session.catalog.apply(Op::SetDevelop { id: PhotoId(1), settings: Arc::new(edit), label: "edit".into(), edited: None }).unwrap();
request(&mut app);
let (_, edited) = take_job(&work);
app.session.catalog.apply(undo).unwrap();
request(&mut app);
let (_, restored) = take_job(&work);
assert_eq!(restored.key, key);
finish(&mut app, &work, &ctx, slot, restored.run());
finish(&mut app, &work, &ctx, slot, edited.run());
assert_eq!(app.renderer.thumb(PhotoId(1)).unwrap().pixels.as_ref().unwrap().pixels, original.pixels);
}
#[test]
fn explicit_clear_repairs_same_key_bad_pixels_and_rejects_old_same_key_job() {
let (mut app, work, ctx) = fixture();
let job = app.session.thumb_job(PhotoId(1), 256).unwrap();
let expected = job.clone().run().rendered.unwrap().image;
let (cache, key) = job.cache.clone().unwrap();
let mut bad = expected.clone();
bad.data.fill([255, 0, 255, 255]);
cache.put(key, Arc::new(bad.clone()));
request(&mut app);
let (slot, corrupt) = take_job(&work);
finish(&mut app, &work, &ctx, slot, corrupt.run());
assert_eq!(app.renderer.thumb(PhotoId(1)).unwrap().pixels.as_ref().unwrap().pixels[0].to_array(), [255, 0, 255, 255]);
// An old job can have the SAME render key and stale decoded pixels.
let mut old = job.clone();
old.source = lightcraft_engine::media::SourceRef::Loaded(Box::new(lightcraft_engine::media::DecodedSource::new(
Arc::new(lightcraft_raster::Rgb32f::new(8, 8)),
None,
)));
app.renderer.textures.clear();
app.renderer.request(slot, old, 10);
let (_, old) = take_job(&work);
app.session.execute("library.clearPreviews", &serde_json::json!({})).unwrap();
app.renderer.poll(&ctx, &mut app.session);
assert!(app.renderer.thumb(PhotoId(1)).is_none());
request(&mut app);
let (_, repaired) = take_job(&work);
assert_eq!(repaired.key, job.key);
finish(&mut app, &work, &ctx, slot, repaired.run());
finish(&mut app, &work, &ctx, slot, old.run());
assert_eq!(cache.get(key).unwrap().as_bytes(), expected.as_bytes());
assert_eq!(
app.renderer.thumb(PhotoId(1)).unwrap().pixels.as_ref().unwrap().pixels.iter().flat_map(|c| c.to_array()).collect::<Vec<_>>(),
expected.as_bytes()
);
}
}
+11 -3
View File
@@ -158,8 +158,11 @@ pub fn handle(app: &mut LightcraftApp, ctx: &egui::Context) {
o.remove("advance");
}
cull(app, id, params, advance);
} else {
let _ = app.run(id, params);
} else if let Err(e) = app.run(id, params)
&& matches!(id, "app.export" | "app.exportPrevious")
{
// an export that can't start (e.g. no folder) says why instead of doing nothing
app.toast(ctx, e);
}
}
for f in fire {
@@ -255,7 +258,12 @@ pub fn handle(app: &mut LightcraftApp, ctx: &egui::Context) {
continue;
}
}
let _ = app.run(&f, json!({}));
// an export that can't start (e.g. no folder) says why instead of doing nothing
if let Err(e) = app.run(&f, json!({}))
&& matches!(f.as_str(), "app.export" | "app.exportPrevious")
{
app.toast(ctx, e);
}
match f.as_str() {
"photo.pick" => app.toast(ctx, "Flagged as Pick"),
"photo.reject" => app.toast(ctx, "Flagged as Reject"),
+14
View File
@@ -266,6 +266,9 @@ pub struct UiState {
pub show_counts: bool,
/// Face / pet boxes (read from XMP) over the photo in the loupe.
pub face_boxes: bool,
/// Left-sidebar sections folded shut by their header (`albums`, `local`, `byDate`,
/// `keywords`); the rest are open.
pub collapsed_sidebar: Vec<String>,
/// Local sidebar locations hidden with “Remove from Local” (folders on disk are untouched).
pub hidden_locations: Vec<String>,
/// Copies opened in an external editor this session (reloaded when the window is focused
@@ -573,6 +576,7 @@ impl Default for UiState {
grid_info: "filename".into(),
show_counts: true,
face_boxes: true,
collapsed_sidebar: Vec::new(),
hidden_locations: Vec::new(),
dragging_control: None,
external_edits: Vec::new(),
@@ -641,6 +645,16 @@ impl UiState {
self.open_sections.push(id.to_string());
}
}
pub fn sidebar_section_collapsed(&self, id: &str) -> bool {
self.collapsed_sidebar.iter().any(|s| s == id)
}
pub fn toggle_sidebar_section(&mut self, id: &str) {
if self.sidebar_section_collapsed(id) {
self.collapsed_sidebar.retain(|s| s != id);
} else {
self.collapsed_sidebar.push(id.to_string());
}
}
pub fn flyout_open(&self, id: &str) -> bool {
self.open_flyouts.iter().any(|s| s == id)
}
+2
View File
@@ -366,6 +366,8 @@ fn local_folder_tree_expands_and_browses() {
std::fs::create_dir_all(base.join(".hidden")).unwrap();
let mut h = detail("panel.edit");
exec(&mut h, "view.leftPanel", json!({"show": true}));
h.hide_home_above(&base);
exec(&mut h, "local.addRoot", json!({"path": base.to_string_lossy()}));
exec(&mut h, "library.browse", json!({"path": base.to_string_lossy()}));
h.settle(SETTLE);
let base_s = base.to_string_lossy().to_string();
+51
View File
@@ -148,3 +148,54 @@ fn a_narrow_window_shrinks_the_panels_without_forgetting_their_width() {
h.settle(SETTLE);
assert_eq!(widget(&h, "panel:right_panel").width(), 480.0);
}
/// The left sidebar's Albums, Local, By Date and Keywords headers fold their sections; the choice
/// is part of the saved UI state.
#[test]
fn sidebar_sections_collapse_and_remember_it() {
let mut h = demo([1400.0, 900.0], json!({"view": "photoGrid", "leftPanel": true}));
let has = |h: &Headless, id: &str| h.app.widgets.iter().any(|(w, _)| w == id);
let click = |h: &mut Headless, id: &str| {
let r = h.request("ui.clickWidget", json!({"id": id}), T);
assert_eq!(r["ok"], true, "{r}");
h.step();
h.step();
};
// By Date lists years while open; its header folds them away and back
assert!(has(&h, "sidebarSection:byDate"));
let year_rows = |h: &Headless| h.app.widgets.iter().filter(|(w, _)| w.starts_with("source:date:")).count();
assert!(year_rows(&h) > 0, "the demo library has dated photos");
click(&mut h, "sidebarSection:byDate");
assert_eq!(year_rows(&h), 0, "folded");
assert!(h.app.ui.sidebar_section_collapsed("byDate") && !h.app.ui.sidebar_section_collapsed("albums"));
assert!(has(&h, "sidebarSection:byDate"), "the header stays so it can be reopened");
// the choice survives a save/load of the UI state
let saved = serde_json::to_value(&h.app.ui).unwrap();
let back: crate::state::UiState = serde_json::from_value(saved).unwrap();
assert!(back.sidebar_section_collapsed("byDate"));
click(&mut h, "sidebarSection:byDate");
assert!(year_rows(&h) > 0, "unfolded again");
// Albums folds too, and the plus button inside its header still works on its own
click(&mut h, "sidebarSection:albums");
assert!(h.app.ui.sidebar_section_collapsed("albums"));
assert!(has(&h, "icon:albumNew"), "the Create Album button stays in the header");
click(&mut h, "sidebarSection:albums");
assert!(!h.app.ui.sidebar_section_collapsed("albums"));
// Keywords and Local fold their rows too
let rows = |h: &Headless, prefix: &str| h.app.widgets.iter().filter(|(w, _)| w.starts_with(prefix)).count();
assert!(rows(&h, "source:keyword:") > 0, "the demo library has keywords");
click(&mut h, "sidebarSection:keywords");
assert_eq!(rows(&h, "source:keyword:"), 0, "keywords folded");
click(&mut h, "sidebarSection:keywords");
assert!(rows(&h, "source:keyword:") > 0);
if has(&h, "sidebarSection:local") {
click(&mut h, "sidebarSection:local");
assert_eq!(rows(&h, "source:local:"), 0, "local folded");
assert!(!has(&h, "source:local:browse"), "Browse Folder… folds with it");
click(&mut h, "sidebarSection:local");
assert!(!h.app.ui.sidebar_section_collapsed("local"));
}
// a click on the plus is the button's, not the header's
click(&mut h, "icon:albumNew");
assert!(!h.app.ui.sidebar_section_collapsed("albums"), "the plus does not fold Albums");
}
@@ -0,0 +1,43 @@
//! Switching libraries must not leave the old library's sidebar numbers, keyword tree or date
//! groups on screen. A library loaded from disk starts at catalog revision 0, and so does a brand
//! new empty one, so caches keyed on the revision alone would see "nothing changed".
use serde_json::json;
use crate::{LightcraftApp, Services};
#[test]
fn opening_another_library_drops_the_old_librarys_cached_panels() {
let dir = std::env::temp_dir().join(format!("lc-ui-switchlib-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
let (a, b) = (dir.join("a"), dir.join("b"));
std::fs::create_dir_all(&a).unwrap();
std::fs::create_dir_all(&b).unwrap();
// library A with photos, reopened from disk (as at launch)
let mut session = lightcraft_engine::Session::new().with_fs();
session.open_library(&a, true).unwrap();
session.close_library().unwrap();
session.open_library(&a, false).unwrap();
assert!(!session.catalog.is_empty());
let mut app = LightcraftApp::new(session, Services::default());
let (cat, caches) = (&app.session.catalog, &mut app.caches);
assert!(caches.counts(cat).total > 0);
assert!(!caches.keyword_tree(cat).is_empty());
assert!(!caches.date_groups(cat).is_empty());
// the grid's own list is cached too
assert!(!app.session.visible().is_empty());
app.run("app.openLibrary", json!({"path": b.to_string_lossy()})).unwrap();
assert!(app.session.catalog.is_empty(), "library B is empty");
let (cat, caches) = (&app.session.catalog, &mut app.caches);
assert_eq!(caches.counts(cat).total, 0, "All Photos count");
assert_eq!(caches.counts(cat).picks, 0);
assert!(caches.keyword_tree(cat).is_empty(), "keywords");
assert!(caches.date_groups(cat).is_empty(), "By Date");
assert!(app.session.visible().is_empty(), "grid");
let _ = std::fs::remove_dir_all(&dir);
}
+1 -1
View File
@@ -170,7 +170,7 @@ wait and return the result (when built with the `sam` feature; the default CLI h
with 2-D RoPE and windowed attention, the feature pyramids, the SAM 2-style prompt encoder and
two-way mask decoder for clicks, and the CLIP text encoder, DETR encoder/decoder (box
relative-position bias, presence token) and pixel decoder for text. The CLIP tokenizer is a
small BPE in `tokenizer.rs`. `fetch/` is the downloader (original code).
small BPE in `tokenizer.rs`. The downloader (original code) is the `lightcraft-fetch` crate, shared by any model that is downloaded; `fetch/` here says what SAM 3 needs.
- candle is pinned at 0.9.2: later releases make `candle-core` depend on `tokenizers` with the
Oniguruma C library, and the product is pure Rust. Weights are read with positional reads (no
memory map: `unsafe` stays in `lightcraft-sysmem`), only the tensors a path needs, and every
+7 -1
View File
@@ -1,4 +1,4 @@
# Sony ARW and Nikon NEF starting look
# Sony ARW, Nikon NEF and Panasonic RW2 starting look
ARW and NEF decoding supply a Bayer mosaic and camera white-balance multipliers, but currently no measured camera colour matrix. Treating camera RGB as linear sRGB produces a dark, muted default rendering. The loader now estimates a file-local starting look from that file's embedded camera JPEG, only when a usable camera matrix is absent. All output pixels still come from the RAW mosaic; there is no JPEG replacement or uniform saturation boost.
@@ -40,6 +40,12 @@ Measuring this found a decoder bug: 12-bit NEFs store maker note `0x003d` BlackL
On one sample, Exposure +1 EV, Saturation +100, Contrast +50, warm/cool WB, tint, highlights and shadows all changed the output. The same matrix and curve were selected for 400-pixel and full-resolution renders. Headless UI checks wait until the loupe reports `source: render` and no jobs remain; the camera JPEG stand-in does not count as verification.
### Panasonic RW2
The same fit, gates and relative WB apply to Panasonic RW2, Leica RWL and the older Panasonic RAW files (all decode as `RawFormat::Rw2`; white balance from tags `0x0024`–`0x0026`). Their embedded JPEG (`JpgFromRaw`, 1920 px wide) always shows the whole active area, while the default crop is the aspect ratio set in the camera (4:3 on the 3:2 sensors, 1:1, 16:9, 2.71 on the S1R II…); when the preview matches the active area rather than the crop, the reference is cropped to the default crop before fitting.
Public CC0 samples from raw.pixls.us (178 files from 118 Panasonic and Leica bodies; the four from 2005–2007 carry no preview), rendered at 1200 px and compared with each file's embedded JPEG (mean CIE76 ΔE at 200 px): 140 of 174 fits accepted, from 93 of the 114 bodies with a preview, median ΔE 5.0 (1.2–21.3) with matching mean lightness (median |ΔL*| 0.4); 124 of them use a hue/saturation table. (Fitted with the matrix and tone curve alone, 137 were accepted at median ΔE 6.1; every one of those is as close or closer with the table and chroma curve, by 0.75 on average.) On the 14 corpus files with a preview the neutral fallback scored ΔE 9.6–27 and rendered up to 16 L* too dark; 13 fits were accepted, ΔE 2.8–8.9, lightness within 2 L*. Of the 34 files left on the neutral fallback, 21 fail the gates, almost all compacts and kit zooms (TZ/ZS, FZ2500, D-Lux, V-Lux, GF2, GX80 with its 12–32 mm): their camera JPEG is distortion-corrected, the pixel pairs no longer line up and the held-out RMS stays above 0.10 (GX80: 0.138 → 0.107); two S1 II shots just miss it (0.103, 0.107). The other 13 never reach the gates, mostly nearly colourless scenes (grey walls, concrete: under 5% coloured pixels) or dark ones (a night sky, a near-black frame). Panasonic's embedded distortion data (tag `0x0119`) is not used yet, and `lightcraft-cli calibrate` doesn't collect RW2 files: the misaligned pairs of distortion-corrected JPEGs would distort a pooled profile.
This is a per-file camera-look estimate, **not measured spectral calibration or Lightroom parity**. Camera local tone, picture styles, noise reduction and lens warps cannot all be reproduced by a global matrix/curve; same-aspect crops may evade the aspect check. Skin, mixed-light scenes and saturated highlights can still differ from a supplied JPEG. Broader Sony and Nikon model/lighting coverage, chart-based calibration, absolute WB and other manufacturers remain unverified. Rejected fits and other RAW formats retain their existing behaviour.
## Camera profiles (pooled per model)
+58
View File
@@ -0,0 +1,58 @@
# Contributors in the About window
**About ▸ Contributors** credits everyone who contributed to LightCraft, and **About ▸ Models** credits
the AI models named in `Co-Authored-By` trailers. This follows the shared craftrules standard
[`standards/contributors.md`](https://github.com/storytold/craftrules/blob/main/standards/contributors.md);
this page is the local copy of the decision.
## Decision
- Two files, both **compiled into the binary** by the UI crate's `build.rs` (nothing is read from
disk or the network at run time; the web build has them too):
- [`contributors/contributors.json`](../contributors/contributors.json): commit stats per GitHub
username. **Generated** by `craftrules/scripts/contributors.py`; never hand-edit.
- [`contributors/people.toml`](../contributors/people.toml): the names people chose for
themselves. **Each contributor adds only their own entry**, in their own PR.
- Each contributor gets one line: GitHub username (always), display name (opt-in, or their synced
GitHub profile name), real name (only if they told us), merged PRs, commits, lines added, lines
deleted, binary assets added, binary assets removed, first and last commit date.
- The list is shown as a **grab bag** (names flowing as text) or a **table**, in the same order. It
sorts by name, PRs, commits, lines added, lines deleted, line delta, binary assets added/removed,
first or last commit date. The name toggle cycles **Username → Display name → Real name**; a
missing name falls back to `@username`. Alphabetical sorting is case-insensitive and ignores the `@`.
## In LightCraft
- The About dialog (`crates/ui-egui/src/panels/dialogs.rs`, Help ▸ About LightCraft) has the tabs
**About · Contributors · Models**; the control channel can switch them with
`ui.clickWidget {"id": "button:aboutTab-contributors"}` (`-about`, `-models`), and the list view
with `button:creditsGrabBag` / `button:creditsTable`.
## Credit yourself
Add your entry to `contributors/people.toml` in a PR you commit yourself (every field optional):
```toml
[people.your-github-username]
real_name = "Your Name"
display_name = "Nickname"
sync_github_name = true # use the name on your public GitHub profile as display name
```
or, with craftrules checked out next to `craft-apps/`:
```sh
python3 ../../craftrules/scripts/contributors.py --add-me . --real-name "Your Name" --sync-github-name
```
Names show after the next refresh: the script checks that each entry was committed by its own
user (or by a maintainer with a `consent_source`). Delete your entry to remove your names.
## Refreshing (maintainers)
```sh
python3 ../../craftrules/scripts/contributors.py . # regenerate + verify; needs git, Python 3.11+, gh
python3 ../../craftrules/scripts/contributors.py --check . # verify people.toml only
```
Commit the updated `contributors/contributors.json`. Git author names and emails are never recorded.
+17
View File
@@ -4,6 +4,23 @@ The CPU pipeline (`crates/pipeline`) is the reference ("oracle"). `lightcraft-gp
evaluates the same stages with wgpu compute shaders (WGSL) on Metal / Vulkan / DX12. Everything is
pure Rust (wgpu, naga); the drivers are the system's. No GL backend is compiled in.
Warped geometry uses a CPU-computed coverage bit mask for the source boundary.
The CPU reference evaluates that boundary in double precision; GPU float
rounding otherwise can turn a blank edge pixel into a photo pixel. The mask
uses one bit per output pixel (rows padded to 32 bits). Sampling and color
corrections remain on the GPU, and existing geometry stage caching reuses the
result. The mask is built per 32 × 16 block (`Warp::block_coverage`): the warp
formulas are written once, generic over `lightcraft_geom::Real`, and evaluated
with outward-rounded `Interval`s over the block. Those bounds enclose the f64
result of every pixel in the block, so a block whose bounds lie inside the
image edges (the same f64 comparisons, no margin), or beyond one of them, is
filled at once; only blocks across the edge, or whose bounds are undecided (a
perspective denominator that may vanish), evaluate each pixel (`Warp::covers`,
the framing chain `Warp::frame` the CPU resample uses too). The bits equal the
per-pixel decision. At 6000 × 4000 with lens warp + perspective the mask takes
~2.7 ms (32 threads; ~24.5 ms on one) instead of ~30 ms (~580 ms), and only
when warped geometry rebuilds.
## Backends, environment variables and troubleshooting (issue #136)
wgpu loads the driver of **every** backend in an instance's set while it enumerates adapters — even
when it then picks another one. A Vulkan driver that crashes there (issue #136: an access violation
+36
View File
@@ -0,0 +1,36 @@
# LightCraft em português do Brasil
O LightCraft pode ser exibido em inglês, chinês simplificado, chinês tradicional (Taiwan), japonês e
português do Brasil. A preferência de idioma é gravada em `language` no `ui.json`.
- Troque em **Editar → Idioma** ou em **Configurações → Geral → Idioma**. A escolha é mantida nas
próximas inicializações.
- Traduzimos menus, edição de fotos, máscaras, recorte, configurações, importação, exportação e as
principais mensagens de progresso.
- Menus, telas e o texto com valores variáveis usam as fontes Inter (regular e semibold). Texto em
chinês e japonês exibido nos nomes de idioma (por exemplo, 日本語) continua dependendo do repositório
de fontes [storytold/craft-fonts](https://github.com/storytold/craft-fonts).
- A tradução fica na camada de apresentação. IDs de comandos, nomes de arquivos de fotos e metadados
digitados pelo usuário nunca são alterados.
- Erros técnicos, informações complementares e notas de versão não traduzidos aparecem em inglês.
- A tradução para chinês está em [`localization-zh-hans.md`](localization-zh-hans.md); para japonês,
em [`localization-ja.md`](localization-ja.md). Como adicionar outro idioma está em
[`localization.md`](localization.md).
## Manutenção da tradução
Os textos fixos ficam em `crates/ui-egui/locales/pt-br.json` e os textos com valores variáveis em
`crates/ui-egui/locales/pt-br-formats.json`. O inglês é usado como chave.
Toda tradução de formato é verificada por `format!` do Rust no build. Referencie os valores por nome
(`{n}`) ou por posição (`{0}`, `{1}`) e use as mesmas especificações de formato do inglês
(`{:.1}`, `{d:+}`, `{:.0}%`). Quando o plural inglês (`"s"` / `""`) não for necessário em português,
consuma o argumento correspondente com `{:.0}` — um texto com precisão 0 não imprime nada.
Aparência, fontes, troca de idioma, persistência da preferência e estabilidade dos IDs de comando são
verificadas por `cargo test -p lightcraft-ui-egui i18n::tests` (a checagem de glifos só roda com
`CRAFT_FONTS_DIR`). Para visualizar o idioma, renderize sem interface gráfica:
```sh
LIGHTCRAFT_LANGUAGE=pt-br lightcraft-cli snapshot --demo --script tour.jsonl -o out.png --size 1600x1000
```
+5 -3
View File
@@ -1,13 +1,15 @@
# LightCraft 简体中文界面
LightCraft 的界面支持简体中文、日语和英语。语言设置保存在 `ui.json` 的 `language` 字段中。
LightCraft 的界面支持英语、简体中文、繁体中文(台湾)和日语。语言设置保存在 `ui.json` 的 `language` 字段中。
- 通过「编辑 → 语言」或「设置 → 一般 → 语言」切换,选择会保留到下次启动。
- 已翻译的范围:菜单、照片编辑、蒙版、裁剪、设置、导入、导出,以及主要的进度提示。
- 已翻译的范围与 `zh-hant.json` 键集完全一致(1246 条),两份目录的键与键序一一对应。
- 正文字重与半粗体都使用 SIL OFL 的 **Noto Sans CJK SC**(简体中文)与 BIZ UDPGothic(日语)。
这些字体不在本仓库,而在 [storytold/craft-fonts](https://github.com/storytold/craft-fonts);
使用可选的构建输入 `CRAFT_FONTS_DIR=../craft-fonts` 时(所有正式发布版)会内嵌进来。
不指定该输入时程序照常构建运行,但中文与日文文字没有字形,会显示为方框。
不指定该输入时程序照常构建运行,但中文与日文文字没有字形,会显示为方框 ——
语言菜单里四种语言仍然都在,选中「简体中文」也只会得到一屏方框。未内嵌字体时构建会打印一条
`cargo::warning` 提示这一点。
- 翻译只发生在显示层:命令 ID、照片文件名、用户输入的元数据都不做改动。
- 尚未翻译的技术性错误、补充说明和发行说明仍以英文显示。
- 已知限制:**网页版(wasm32)目前只内嵌日文字体**,因此浏览器里的中文仍显示为方框
Loaded 100 of 119 files, more files were not shown because too many files have changed in this diff. Show more