Cryptomatte eyedropper: the viewer pick (#473)

panels::crypto_pick is the eyedropper's viewer side; it is wired in by the next commit. Armed,
the hover names the ID under the cursor and highlights its coverage, a click replaces the
Selection with it, Shift adds, Alt or Ctrl (Option or Cmd on a Mac) removes. Each pick is one
undo step, "Pick Cryptomatte".

The probe the hover needs is built off the UI thread, one at a time for the app
(EffectcraftApp::crypto_probe_busy, re-arming included), not while playing and not until a
scrub has settled for 100 ms; meanwhile the chip says so. Session::crypto_probe_ready asks for
readiness without changing anything; a job is asked for only when it starts.

Clicks made meanwhile are queued, own the build and are made in order at the end of the pass,
after its keys and edits. A click is the frame, point, resolution and project revision its
press saw. If any of them changed before it is made (an edit, Ctrl+Z, a nudge, a frame step),
it and every click waiting with it are dropped and the status says "Pick not made". A pick's
own change of the revision doesn't count as an edit for a click still held. Losing the window
focus drops a held press. A click that takes the focus from the Selection field waits for the
field to commit and is made on top of it.

The overlay takes drags too: a click that drifts picks at its press, and one dragged further
than 24 points picks nothing ("Drag cancelled the pick"). Presses and releases are read from
the pass's events in order rather than from egui's end-of-pass click state, since a slow pass
can hold a whole click. Space, the Hand and the Zoom tool keep their clicks, the middle button
pans, and a right-click is passed on under viewer::FORWARDED_SECONDARY for the viewer to open
its context menu (the viewer side is in the next commit). Errors show in the chip in red,
without the command's prefix, wrapped to the viewer's width.

The hovered ID's coverage (Session::crypto_coverage) is drawn as a texture through the layer's
effect-space transform, rebuilt only when the hovered ID, frame or revision changes, and
exposed to the test harness as viewer.cryptoPick.highlight.

Supporting changes:
- EffectcraftApp::say sets the status line and restarts its 5 s; each status message now gets
  its own 5 s, also when the same text is shown again.
- Points snap to the centre of the shown frame's pixel at the exact 1/n scale the viewer
  renders (frames::render_scale; the frame key keeps 0.333 for Third, which named the next
  row's ID). Cache Frames When Idle renders at that exact scale too, so idle-cached frames are
  on the grid the pick reads.

crates/ui-egui/src/app.rs

@@ -77,6 +77,7 @@ impl EffectcraftApp {
             passive_tex: std::collections::HashMap::new(),
             viewer_native: None,
             viewer_shown: None,
+            crypto_probe_busy: Default::default(),
             viewer_image: None,
             viewer_readback: Default::default(),
             wgpu: None,

crates/ui-egui/src/app/frame.rs

@@ -454,10 +454,22 @@ impl EffectcraftApp {
         }
     }
 
+    /// Sets the status line and restarts its 5 s, also when the same text is still showing (a
+    /// message set again would otherwise keep the time it was first shown).
+    pub(crate) fn say(&mut self, ctx: &egui::Context, msg: String) {
+        ctx.data_mut(|d| d.remove::<(f64, String)>(egui::Id::unique("status-since")));
+        self.ui.status = msg;
+    }
+
     /// The status message or the latest toast, bottom left (`lift` points higher while the
     /// Importing card is there).
     fn draw_toast(&mut self, ui: &mut egui::Ui, full: egui::Rect, lift: f32) {
         let now = self.now;
+        // A message cleared before its time (a click on it, Esc, another pick) forgets its time, so
+        // the same message shown again later gets its own 5 s.
+        if self.ui.status.is_empty() {
+            ui.data_mut(|d| d.remove::<(f64, String)>(egui::Id::unique("status-since")));
+        }
         let msg = if !self.ui.status.is_empty() {
             Some(self.ui.status.clone())
         } else {
@@ -485,11 +497,14 @@ impl EffectcraftApp {
             }
             ui.ctx().request_repaint_after(std::time::Duration::from_millis(500));
             if !self.ui.status.is_empty() {
-                let since: f64 = ui.data(|d| d.get_temp(egui::Id::unique("status-since")).unwrap_or(now));
-                ui.data_mut(|d| d.insert_temp(egui::Id::unique("status-since"), since));
+                // Each message gets its own 5 s (a new one doesn't inherit the old one's time).
+                let key = egui::Id::unique("status-since");
+                let shown = ui.data(|d| d.get_temp::<(f64, String)>(key)).filter(|(_, m)| *m == self.ui.status);
+                let since = shown.map_or(now, |(t, _)| t);
+                ui.data_mut(|d| d.insert_temp(key, (since, self.ui.status.clone())));
                 if now - since > 5.0 {
                     self.ui.status.clear();
-                    ui.data_mut(|d| d.remove::<f64>(egui::Id::unique("status-since")));
+                    ui.data_mut(|d| d.remove::<(f64, String)>(key));
                 }
             }
         }

crates/ui-egui/src/app/prefetch.rs

@@ -405,11 +405,12 @@ impl EffectcraftApp {
             return;
         }
         let cur = comp.frame_rate.frame_at(self.session.time()).clamp(wa, wb);
-        let fit = self.frames_that_fit(comp, series.scale as f64 / 1000.0).min(n);
+        // The viewer's exact factor (1/3, not the key's 0.333): it shows these frames.
+        let scale = frames::render_scale(series.scale, comp.resolution.factor());
+        let fit = self.frames_that_fit(comp, scale).min(n);
         // One thread each (`Frames::request_background`), so every frame thread takes one.
         let slots = self.frames_parallelism().max(1);
         let mut queued = self.frames.inflight();
-        let scale = series.scale as f64 / 1000.0;
         let mut pending = false;
         for k in 0..fit {
             let f = wa + (cur - wa + k).rem_euclid(n);

crates/ui-egui/src/frames.rs

@@ -1580,5 +1580,13 @@ impl Queue {
     }
 }
 
+/// The render scale a viewer frame's key stands for: it keeps thousandths, but the viewer
+/// renders every n-th pixel, and 0.333 is not 1/3 (2 pixels off by row 1000 at Third). The
+/// comp's own factor decides when two factors share the thousandths (from 1/34).
+pub(crate) fn render_scale(thousandths: u32, factor: Option<u8>) -> f64 {
+    let n = factor.map(f64::from).filter(|n| (1000.0 / n).round() == f64::from(thousandths));
+    1.0 / n.unwrap_or_else(|| (1000.0 / f64::from(thousandths.max(1))).round()).max(1.0)
+}
+
 #[cfg(test)]
 mod tests;

crates/ui-egui/src/frames/tests.rs

@@ -1,5 +1,16 @@
 use super::*;
 
+#[test]
+fn render_scales_are_the_factors_the_thousandths_stand_for() {
+    assert_eq!(render_scale(333, Some(3)), 1.0 / 3.0);
+    assert_eq!(render_scale(333, None), 1.0 / 3.0);
+    assert_eq!(render_scale(167, Some(6)), 1.0 / 6.0);
+    assert_eq!((render_scale(1000, None), render_scale(500, None), render_scale(250, Some(4))), (1.0, 0.5, 0.25));
+    // 1/34 and 1/35 are both 29 thousandths: the comp's factor decides.
+    assert_eq!((render_scale(29, Some(35)), render_scale(29, Some(34))), (1.0 / 35.0, 1.0 / 34.0));
+    assert_eq!(render_scale(29, Some(2)), 1.0 / 34.0);
+}
+
 #[test]
 fn converting_a_frame_with_nan_alpha_keeps_the_viewer_alive() {
     let image = effectcraft_engine::render::Image::filled(1, 1, [0.5, 0.25, 0.1, f32::NAN]);

crates/ui-egui/src/lib.rs

@@ -312,6 +312,8 @@ pub struct EffectcraftApp {
     pub(crate) viewer_native: Option<(egui::TextureId, FrameKey, Arc<effectcraft_gpu::DisplayFrame>)>,
     /// The texture the viewer draws and its frame key (CPU or GPU frame).
     pub(crate) viewer_shown: Option<(egui::TextureId, FrameKey)>,
+    /// A Cryptomatte eyedropper probe is being built (one at a time, re-arming included).
+    pub(crate) crypto_probe_busy: Arc<std::sync::atomic::AtomicBool>,
     /// Last displayed image (Info panel colour, eyedroppers, histograms). GPU frames fill it on
     /// demand ([`EffectcraftApp::viewer_pixels`]).
     pub(crate) viewer_image: Option<Arc<egui::ColorImage>>,

crates/ui-egui/src/panels.rs

@@ -4,6 +4,7 @@ pub mod anim_tools;
 pub mod camera_tracker_ui;
 pub mod comp_settings;
 pub mod content_fill_panel;
+pub mod crypto_pick;
 pub mod delete_items;
 pub mod dialogs;
 pub mod dialogs_3d;

crates/ui-egui/src/panels/crypto_pick.rs

@@ -0,0 +1,600 @@
+//! The Cryptomatte eyedropper in the Composition viewer (#473). The button next to the effect's
+//! Selection arms it ([`super::effect_controls::toggle_pick`] with [`KIND`]); while it is armed
+//! the viewer names the ID under the cursor, and a click picks it into the Selection: a plain
+//! click replaces it, Shift adds, Alt or Ctrl (Option or Cmd on a Mac) removes. It stays armed
+//! for the next click until Esc or the button again, and highlights the hovered ID's coverage.
+//! Arming it whenever the effect is selected (After Effects' behaviour, #1811) would only need
+//! to set the same pick.
+//!
+//! The hover needs the frame's Cryptomatte probe ([`effectcraft_engine::Session::crypto_probe_job`]):
+//! it is built off the UI thread, one at a time, not while playing or scrubbing; the chip says so
+//! meanwhile, and clicks wait for it (the hover then builds nothing), except while playing, where
+//! a click without its probe is not made. Once built, a hover is a few array reads.
+//!
+//! A click is the frame and the project shown at its press (and the resolution it read). It is
+//! made at the end of a pass ([`upkeep`]), checked against the frame and the project right
+//! before: if either changed since its press, it and every click waiting with it are dropped,
+//! and the status says so. A press dragged further than [`SLOP`] picks nothing, and says so.
+
+use std::sync::Arc;
+use std::sync::atomic::{AtomicBool, Ordering};
+
+use effectcraft_engine::commands::CryptoProbeJob;
+use effectcraft_engine::geom::Mat3;
+use effectcraft_engine::project::Layer;
+use effectcraft_engine::render::EvalCtx;
+use egui::{Color32, Rect, Sense, vec2};
+use serde_json::{Value, json};
+
+use super::fx_widgets as fw;
+use crate::EffectcraftApp;
+use crate::i18n::{tr, tr_args};
+use crate::state::{FxPick, Tool};
+use crate::theme::Tokens;
+
+/// [`FxPick::kind`] of the Cryptomatte eyedropper.
+pub const KIND: &str = "crypto";
+
+/// How long the time must stay put before a probe is built for it (scrubbing).
+const SETTLE: f64 = 0.1;
+
+/// A press that moves further than this (points) before its release picks nothing.
+const SLOP: f32 = 24.0;
+
+/// The status when waiting clicks are dropped.
+const NOT_MADE: &str = "Pick not made: the frame or the project changed before its Cryptomatte IDs were read";
+
+/// The status when a click is not made because the preview plays.
+const PLAYING: &str = "Pick not made: stop playback to pick";
+
+/// egui temp-data id for the eyedropper's [`State`] between frames.
+const VIEWER_CRYPTO_PICK: &str = "viewer-crypto-pick";
+
+/// egui temp-data id for the project revision and frame at the start of a pass.
+const VIEWER_CRYPTO_PICK_START: &str = "viewer-crypto-pick-start";
+
+fn pick_id() -> egui::Id {
+    egui::Id::unique(VIEWER_CRYPTO_PICK)
+}
+
+fn pick_start_id() -> egui::Id {
+    egui::Id::unique(VIEWER_CRYPTO_PICK_START)
+}
+
+/// The press of a click in progress.
+#[derive(Clone)]
+struct Press {
+    pos: egui::Pos2,
+    /// Its modifiers: letting go of Shift a moment early still adds.
+    keys: egui::Modifiers,
+    /// A menu or popup list was open: the click only closes it.
+    menu: bool,
+    /// A text field had the focus: the click commits it (the project may change by that).
+    typing: bool,
+    /// The pick query at the press (frame, resolution, layer point) and the project revision.
+    q: Option<Value>,
+    rev: u64,
+}
+
+/// The hovered ID's coverage highlight drawn over the viewer.
+#[derive(Clone)]
+struct HighlightMask {
+    key: (u32, Value),
+    texture: egui::TextureHandle,
+    size: [f64; 2],
+    uv: [f32; 2],
+}
+
+/// Hover and click state between frames.
+#[derive(Clone, Default)]
+struct State {
+    /// The last hover query (with the project revision) and its answer (or why there is none).
+    asked: Option<(Value, u64)>,
+    reply: Option<Result<Value, String>>,
+    /// Clicks waiting for their probe, in order, and the project revision they were made at.
+    pending: Vec<Value>,
+    pending_rev: u64,
+    /// The waiting clicks took the focus from a text field (the Selection being typed): they
+    /// are made once it has committed, on top of what it committed.
+    after_typing: bool,
+    /// A text field or a popup had the keyboard the pass before: its Esc isn't the pick's
+    /// (egui lets go of the focus and closes popups before the viewer runs).
+    esc_taken: bool,
+    /// The press of the click in progress.
+    press: Option<Press>,
+    /// Probes may be built for the waiting clicks (not while playing or scrubbing).
+    start: bool,
+    /// The comp time shown and since when (to wait for a scrub to settle).
+    shown: Option<(f64, f64)>,
+    /// The pick (layer, Selection) and the egui pass this state was last used for.
+    seen: Option<((u64, u64), u64)>,
+    /// The hovered ID's highlight: its hash and probe query, coverage texture, size and uv.
+    mask: Option<HighlightMask>,
+}
+
+/// Clears the busy flag and repaints when a build ends, a panic included.
+struct Done(Arc<AtomicBool>, egui::Context);
+impl Drop for Done {
+    fn drop(&mut self) {
+        self.0.store(false, Ordering::Release);
+        self.1.request_repaint();
+    }
+}
+
+fn run(job: CryptoProbeJob, done: Done) {
+    let _done = done;
+    job();
+}
+
+/// Whether the probe `q` needs is built; if not and `start`, start building it unless one is
+/// being built (one at a time for the app). Asking changes nothing: only a build started here
+/// supersedes another.
+fn ready(app: &EffectcraftApp, ctx: &egui::Context, q: &Value, start: bool) -> Result<bool, String> {
+    let busy = &app.crypto_probe_busy;
+    if app.session.crypto_probe_ready(q).map_err(|e| shown_error(&e.to_string()))? {
+        return Ok(true);
+    }
+    // (Only this thread sets the flag, so it stays clear until the store.)
+    if !start || busy.load(Ordering::Acquire) {
+        return Ok(false);
+    }
+    let Some(job) = app.session.crypto_probe_job(q).map_err(|e| shown_error(&e.to_string()))? else { return Ok(true) };
+    busy.store(true, Ordering::Release);
+    let done = Done(busy.clone(), ctx.clone());
+    if cfg!(target_arch = "wasm32") {
+        run(job, done);
+    } else if std::thread::Builder::new().name("crypto-probe".into()).spawn(move || run(job, done)).is_err() {
+        return Err("could not start reading the Cryptomatte IDs".into());
+    }
+    Ok(false)
+}
+
+/// An engine error as the chip and the status line show it: without the command's "invalid
+/// parameters for `…`:" prefix, capitalised.
+fn shown_error(e: &str) -> String {
+    let e = e.strip_prefix("invalid parameters for `").and_then(|r| r.split_once("`: ")).map_or(e, |(_, m)| m);
+    let mut c = e.chars();
+    c.next().map_or_else(String::new, |f| f.to_uppercase().chain(c).collect())
+}
+
+/// The keys that remove an ID (Ctrl removes on a Mac too, but means right-click there).
+pub fn remove_keys(os: egui::os::OperatingSystem) -> &'static str {
+    if os == egui::os::OperatingSystem::Mac { tr("Option or Cmd") } else { tr("Alt or Ctrl") }
+}
+
+/// At the start of every pass, before its keys: the project revision and frame on screen when
+/// the pass's input happened. A press in the pass is taken against them, not against what the
+/// pass's own keys made of them (a slow pass can hold a press, a PageDown and the release).
+pub fn pass_start(app: &EffectcraftApp, ctx: &egui::Context) {
+    ctx.data_mut(|d| d.insert_temp(pick_start_id(), (app.session.revision, app.session.time().seconds())));
+}
+
+/// Every frame: a disarmed eyedropper lets go of its state and the session's probe.
+pub fn upkeep(app: &mut EffectcraftApp, ctx: &egui::Context) {
+    let sid = pick_id();
+    // Another project (New, Open, Revert) ends a pick: it is kept by layer id and property uid,
+    // which the next project reuses.
+    let eid = egui::Id::unique("viewer-fx-pick-project");
+    let epoch = app.session.script_ui.project_epoch;
+    let seen = ctx.data(|d| d.get_temp::<u64>(eid));
+    if seen != Some(epoch) {
+        if seen.is_some() && app.ui.fx_pick.is_some() {
+            app.ui.fx_pick = None;
+            app.ui.status.clear();
+        }
+        ctx.data_mut(|d| d.insert_temp(eid, epoch));
+    }
+    if app.ui.fx_pick.as_ref().is_none_or(|p| p.kind != KIND) && ctx.data(|d| d.get_temp::<State>(sid)).is_some() {
+        ctx.data_mut(|d| d.remove::<State>(sid));
+        app.session.drop_crypto_probe();
+    }
+    // The clicks waiting in the viewer, after everything else this pass did (its keys, edits and
+    // frame steps included).
+    if let Some(mut st) = ctx.data(|d| d.get_temp::<State>(sid)).filter(|s| s.seen.is_some_and(|(_, n)| n == ctx.cumulative_pass_nr())) {
+        make_waiting(app, ctx, &mut st);
+        ctx.data_mut(|d| d.insert_temp(sid, st));
+    }
+}
+
+/// Drop every waiting click, and say so.
+fn drop_waiting(app: &mut EffectcraftApp, ctx: &egui::Context, st: &mut State) {
+    (st.pending, st.after_typing) = (vec![], false);
+    app.say(ctx, tr(if app.playback.playing { PLAYING } else { NOT_MADE }).to_string());
+}
+
+/// Make one click. Our own pick is not "the project changed" for a click still held.
+fn make(app: &mut EffectcraftApp, ctx: &egui::Context, st: &mut State, q: Value) -> Result<Value, String> {
+    let before = app.session.revision;
+    let r = crate::menus::invoke(app, ctx, "effect.pickCryptomatte", q);
+    if let Some(p) = st.press.as_mut()
+        && p.rev == before
+    {
+        p.rev = app.session.revision;
+    }
+    r
+}
+
+/// Make the waiting clicks whose probe is built, in order. Each is checked against the frame and
+/// the project right before it is made, with nothing in between: if either changed since its
+/// press, it and the clicks waiting with it are not made.
+fn make_waiting(app: &mut EffectcraftApp, ctx: &egui::Context, st: &mut State) {
+    // A click that took the focus from a text field waits for the field to commit (it may do so
+    // later in this pass or in the next), then is made on what it committed.
+    let typing = st.after_typing && ctx.text_edit_focused();
+    if st.after_typing && !typing {
+        (st.after_typing, st.pending_rev) = (false, app.session.revision);
+    }
+    while let Some(q) = st.pending.first().cloned().filter(|_| !typing) {
+        let comp = app.session.active_comp_id().map(|c| c.0);
+        if st.pending_rev != app.session.revision || q["comp"].as_u64() != comp || q["time"].as_f64() != Some(app.session.time().seconds()) {
+            return drop_waiting(app, ctx, st);
+        }
+        match ready(app, ctx, &q, st.start) {
+            Ok(true) => match make(app, ctx, st, q) {
+                Ok(r) if r["pending"] == true => break,
+                r => {
+                    st.pending.remove(0);
+                    (st.asked, st.pending_rev) = (None, app.session.revision);
+                    let msg = match r {
+                        Ok(r) => tr_args("Cryptomatte Selection: {}", &[&r["selection"].as_str().unwrap_or_default()]),
+                        Err(e) => shown_error(&e),
+                    };
+                    app.say(ctx, msg);
+                }
+            },
+            Ok(false) => break,
+            Err(e) => {
+                (st.pending, st.after_typing) = (vec![], false);
+                app.say(ctx, e);
+            }
+        }
+    }
+}
+
+/// Middle-button pan and forwarded right-click for the armed eyedropper overlay.
+fn viewer_pan_and_secondary(app: &mut EffectcraftApp, ctx: &egui::Context, resp: &egui::Response) {
+    // The middle button pans the viewer, as the Hand tool (and as without the eyedropper).
+    if resp.dragged_by(egui::PointerButton::Middle) {
+        let pid = egui::Id::unique("viewer-fx-pick-pan");
+        if resp.drag_started() {
+            let start = app.ui.viewer.pan;
+            ctx.data_mut(|d| d.insert_temp(pid, start));
+        }
+        if let (Some(start), Some(d)) = (ctx.data(|m| m.get_temp::<[f32; 2]>(pid)), resp.total_drag_delta()) {
+            app.ui.viewer.pan = [start[0] + d.x, start[1] + d.y];
+        }
+    }
+    // The right button opens the viewer's context menu (next pass).
+    if resp.secondary_clicked()
+        && let Some(p) = resp.interact_pointer_pos()
+    {
+        ctx.data_mut(|d| d.insert_temp(egui::Id::unique(super::viewer::FORWARDED_SECONDARY), p));
+    }
+}
+
+/// Whether Space / Hand / Zoom (`tools`) or middle/right buttons (`nav`) own the pointer.
+struct PointerNav {
+    tools: bool,
+    nav: bool,
+}
+
+/// Queues Cryptomatte picks from primary presses and releases on the armed eyedropper overlay.
+///
+/// A press is taken against the pass-start revision and frame, then queued for [`make_waiting`]
+/// (or dropped if the drag, playback, frame or project make it invalid). `nav` suppresses a new
+/// press when another pointer button or tool owns the click.
+fn viewer_click_to_pick(
+    app: &mut EffectcraftApp,
+    ui: &mut egui::Ui,
+    map: &super::viewer::ViewerMap,
+    ectx: &EvalCtx,
+    st: &mut State,
+    pointer: PointerNav,
+    at: &dyn Fn(egui::Pos2) -> Option<Value>,
+) {
+    let ctx = ui.ctx().clone();
+    // The click is what the press saw: its frame, resolution, point and project. The pass's
+    // primary presses and releases are taken in order: a slow pass can hold a whole click, or one
+    // click's release and the next one's press (egui then reports no press on the overlay).
+    // `None`: the window lost the focus, so a held press may never see its release.
+    let events: Vec<Option<(egui::Pos2, bool, egui::Modifiers)>> = ui.input(|i| {
+        i.events
+            .iter()
+            .filter_map(|e| match e {
+                egui::Event::PointerButton { pos, button: egui::PointerButton::Primary, pressed, modifiers } => Some(Some((*pos, *pressed, *modifiers))),
+                egui::Event::WindowFocused(false) => Some(None),
+                _ => None,
+            })
+            .collect()
+    });
+    // Space (or the Hand or Zoom tool) while the button is held cancels the press (a pan needs a
+    // new press).
+    if pointer.tools {
+        st.press = None;
+    }
+    let (rev0, t0) = ctx.data(|d| d.get_temp::<(u64, f64)>(pick_start_id())).unwrap_or((app.session.revision, ectx.time.seconds()));
+    for e in events {
+        let Some((pos, pressed, keys)) = e else {
+            st.press = None;
+            continue;
+        };
+        if pressed {
+            let ours = map.area.contains(pos) && ctx.layer_id_at(pos) == Some(ui.layer_id());
+            st.press = (!pointer.nav && ours).then(|| Press {
+                pos,
+                keys,
+                menu: crate::widgets::any_menu_open(&ctx),
+                typing: ctx.text_edit_focused(),
+                // The frame on screen at the press: a frame step earlier in this pass drops it.
+                q: at(pos).map(|mut q| {
+                    if t0 != ectx.time.seconds() {
+                        q["time"] = json!(t0);
+                    }
+                    q
+                }),
+                rev: rev0,
+            });
+            continue;
+        }
+        // The release ends the press, whatever egui made of it.
+        let Some(Press { pos: from, keys: k, menu: false, typing, q: Some(mut q), rev }) = st.press.take() else { continue };
+        // Ctrl too on a Mac, where `command` is Cmd.
+        q["mode"] = json!(if k.alt || k.command || k.ctrl {
+            "remove"
+        } else if k.shift {
+            "add"
+        } else {
+            "replace"
+        });
+        let same_frame = q["time"].as_f64() == Some(ectx.time.seconds());
+        if pos.distance(from) > SLOP {
+            app.say(&ctx, tr("Drag cancelled the pick").to_string());
+        } else if app.playback.playing && !(same_frame && app.session.crypto_probe_ready(&q).unwrap_or(false)) {
+            // Playing, a frame without its IDs read is gone before they could be: not queued.
+            app.say(&ctx, tr(PLAYING).to_string());
+        } else if !same_frame || !typing && rev != app.session.revision {
+            // A frame step or an edit came between the press and the release.
+            drop_waiting(app, &ctx, st);
+        } else {
+            if st.pending.is_empty() {
+                st.pending_rev = rev;
+            }
+            st.after_typing |= typing || ctx.text_edit_focused();
+            st.pending.push(q);
+        }
+    }
+}
+
+/// Drop stale hover/click state when the pick or pass gap means it no longer applies.
+fn reset_stale_state(app: &mut EffectcraftApp, ctx: &egui::Context, st: &mut State, pick: &FxPick) {
+    let pass = ctx.cumulative_pass_nr();
+    // Disarmed (or the viewer hidden) since, or another pick: start afresh, so an old click
+    // waiting for its probe is never made. (A probe still being built keeps the app's flag.)
+    if st.seen.is_none_or(|(k, n)| k != (pick.layer, pick.prop) || n + 1 < pass) {
+        if !st.pending.is_empty() {
+            app.say(ctx, tr(NOT_MADE).to_string());
+        }
+        *st = State::default();
+    }
+    st.seen = Some(((pick.layer, pick.prop), pass));
+}
+
+/// Track the shown time for scrub settle; returns whether probe builds may start.
+fn track_scrub_settle(app: &EffectcraftApp, ctx: &egui::Context, st: &mut State, time_secs: f64) -> bool {
+    // No builds while playing, nor until a scrub has settled.
+    let now = ctx.input(|i| i.time);
+    if st.shown.is_none_or(|s| s.0 != time_secs) {
+        st.shown = Some((time_secs, if st.shown.is_some() { now } else { f64::MIN }));
+    }
+    let settled = st.shown.is_some_and(|s| now - s.1 >= SETTLE);
+    if !settled {
+        ctx.request_repaint_after(std::time::Duration::from_secs_f64(SETTLE));
+    }
+    let start = settled && !app.playback.playing;
+    st.start = start;
+    start
+}
+
+/// Read the Cryptomatte ID under the cursor; returns whether a probe is still being read.
+fn hover_read(
+    app: &mut EffectcraftApp,
+    ctx: &egui::Context,
+    map: &super::viewer::ViewerMap,
+    st: &mut State,
+    resp: &egui::Response,
+    start: bool,
+    at: &dyn Fn(egui::Pos2) -> Option<Value>,
+) -> bool {
+    let mut reading = false;
+    match resp.hover_pos() {
+        Some(hp) => {
+            ctx.set_cursor_icon(egui::CursorIcon::Crosshair);
+            let c = map.to_comp(hp);
+            app.pointer_comp = Some([c[0] as f32, c[1] as f32]);
+            let q = at(hp);
+            let asked = q.clone().map(|q| (q, app.session.revision));
+            if asked != st.asked {
+                st.reply = None;
+                // While clicks wait, they own the build: the hover only reads.
+                match q.as_ref().map(|q| ready(app, ctx, q, start && st.pending.is_empty())) {
+                    Some(Ok(true)) => {
+                        let r = q.map(|q| app.session.execute("effect.cryptomatteAt", q).map_err(|e| shown_error(&e.to_string())));
+                        if r.as_ref().is_some_and(|r| r.as_ref().is_ok_and(|v| v["pending"] == true)) {
+                            reading = true;
+                        } else {
+                            st.reply = r.filter(|r| !matches!(r, Ok(Value::Null)));
+                            st.asked = asked;
+                        }
+                    }
+                    Some(Ok(false)) => reading = true,
+                    Some(Err(e)) => (st.reply, st.asked) = (Some(Err(e)), asked),
+                    None => st.asked = None,
+                }
+            }
+        }
+        None => (st.asked, st.reply) = (None, None),
+    }
+    reading
+}
+
+/// Rebuilds the texture when the hovered ID, or the layer, effect, scale, frame or project
+/// revision it was read at, changes; then draws it.
+fn draw_coverage_highlight(app: &mut EffectcraftApp, ctx: &egui::Context, painter: &egui::Painter, map: &super::viewer::ViewerMap, m: &Mat3, st: &mut State) {
+    let hovered = st.reply.as_ref().and_then(|r| u32::from_str_radix(r.as_ref().ok()?["hash"].as_str()?, 16).ok());
+    let hovered = hovered.zip(st.asked.as_ref()).map(|(h, (q, rev))| (h, json!([q["layer"], q["effect"], q["scale"], q["time"], rev])));
+    if let (Some(h), Some(q)) = (&hovered, st.asked.as_ref().map(|a| &a.0))
+        && st.mask.as_ref().is_none_or(|mask| mask.key != *h)
+    {
+        st.mask = app.session.crypto_coverage(q, h.0, 1024).map(|c| {
+            let a = app.tokens.accent;
+            let px = c.coverage.iter().map(|v| Color32::from_rgba_unmultiplied(a.r(), a.g(), a.b(), (v * 140.0) as u8)).collect();
+            let img = egui::ColorImage::new([c.width, c.height], px);
+            let size = [c.size[0], c.size[1]];
+            HighlightMask {
+                key: h.clone(),
+                texture: ctx.load_texture("crypto-pick-highlight", img, egui::TextureOptions::NEAREST),
+                size,
+                uv: [c.uv[0] as f32, c.uv[1] as f32],
+            }
+        });
+    }
+    if let (Some(h), Some(mask)) = (&hovered, &st.mask)
+        && h == &mask.key
+    {
+        let (tex, size, uv) = (&mask.texture, mask.size, mask.uv);
+        let corners = [[0.0, 0.0], [size[0], 0.0], size, [0.0, size[1]]].map(|p| map.to_screen(fw::layer_to_comp(m, p)));
+        let mut mesh = egui::Mesh::with_texture(tex.id());
+        for (pos, t) in corners.iter().zip([[0.0, 0.0], [uv[0], 0.0], uv, [0.0, uv[1]]]) {
+            mesh.vertices.push(egui::epaint::Vertex { pos: *pos, uv: egui::pos2(t[0], t[1]), color: Color32::WHITE });
+        }
+        mesh.indices = vec![0, 1, 2, 0, 2, 3];
+        painter.add(mesh);
+        // Its corners (screen points), for tests and agents.
+        let label = corners.iter().map(|p| format!("{:.2},{:.2}", p.x, p.y)).collect::<Vec<_>>().join(" ");
+        app.auto.add("viewer.cryptoPick.highlight", Rect::from_points(&corners), &label);
+    }
+}
+
+/// Draw the status chip (hovered name, reading, or usage hint).
+fn draw_status_chip(app: &mut EffectcraftApp, ctx: &egui::Context, painter: &egui::Painter, map: &super::viewer::ViewerMap, st: &State, reading: bool) {
+    let err = !reading && matches!(st.reply, Some(Err(_)));
+    let text = match &st.reply {
+        _ if reading && app.playback.playing => tr("Stop playback to pick").to_string(),
+        _ if reading => tr("Reading Cryptomatte IDs…").to_string(),
+        Some(Ok(r)) => {
+            let name = r["name"].as_str().map_or_else(|| format!("<{}>", r["hash"].as_str().unwrap_or_default()), str::to_string);
+            if r["selected"] == true { tr_args("{} (selected)", &[&name]) } else { name }
+        }
+        Some(Err(e)) => e.clone(),
+        None => tr_args("Click: replace · Shift: add · {}: remove", &[&remove_keys(ctx.os())]),
+    };
+    // One line, at most the viewer's width (long names end in an ellipsis); an error in red, on
+    // as many lines as it needs (at least three, more in a narrow viewer), so its hint shows.
+    let max = (map.area.width() - 44.0).clamp(40.0, 480.0);
+    let rows = ((map.area.height() - 28.0) / 14.0).max(3.0) as usize;
+    let colour = if err { app.tokens.danger } else { app.tokens.text };
+    let mut job = egui::text::LayoutJob::simple_singleline(text.clone(), Tokens::ui(11.5), colour);
+    job.wrap = egui::text::TextWrapping { max_width: max, max_rows: if err { rows } else { 1 }, break_anywhere: !err, overflow_character: Some('…') };
+    let galley = painter.layout_job(job);
+    let chip = Rect::from_min_size(map.area.min + vec2(10.0, 10.0), vec2(galley.size().x + 24.0, (galley.size().y + 8.0).max(22.0)));
+    app.auto.add("viewer.cryptoPick", chip, &text);
+    if err {
+        app.auto.add("viewer.cryptoPick.error", chip, if galley.elided { "cut" } else { "whole" });
+        painter.rect(chip, 6.0, Color32::from_rgba_unmultiplied(48, 10, 8, 220), egui::Stroke::new(1.0, colour), egui::StrokeKind::Inside);
+    } else {
+        painter.rect_filled(chip, 11.0, Color32::from_black_alpha(180));
+    }
+    painter.galley(chip.left_center() + vec2(12.0, -galley.size().y / 2.0), galley, colour);
+}
+
+/// Esc disarms the eyedropper only when no text field or popup had the keyboard this pass or
+/// the pass before (egui lets go of the focus and closes popups before the viewer runs).
+fn handle_escape(app: &mut EffectcraftApp, ui: &mut egui::Ui, st: &mut State) {
+    let ctx = ui.ctx();
+    // (Esc in a text field, the Selection being typed, only ends the typing; in a menu, it.)
+    let esc_taken = std::mem::replace(&mut st.esc_taken, ctx.text_edit_focused() || egui::Popup::is_any_open(ctx));
+    if ui.input(|i| i.key_pressed(egui::Key::Escape)) && !esc_taken && !st.esc_taken {
+        app.ui.fx_pick = None;
+        app.ui.status.clear();
+        st.pending.clear();
+    }
+}
+
+/// The armed eyedropper's part of the viewer. `m` maps the layer's effect space to the comp.
+#[allow(clippy::too_many_arguments)]
+pub fn viewer_hook(
+    app: &mut EffectcraftApp,
+    ui: &mut egui::Ui,
+    painter: &egui::Painter,
+    map: &super::viewer::ViewerMap,
+    ectx: &EvalCtx,
+    layer: &Layer,
+    m: &Mat3,
+    pick: &FxPick,
+) {
+    let ctx = ui.ctx().clone();
+    let sid = pick_id();
+    let mut st: State = ctx.data(|d| d.get_temp(sid)).unwrap_or_default();
+    reset_stale_state(app, &ctx, &mut st, pick);
+    // The Cryptomatte effect that owns the Selection (gone: the pick ends).
+    let Some(fx) = layer.effects().and_then(|f| f.groups().find(|g| g.find(pick.prop).is_some())).map(|g| g.uid) else {
+        app.ui.fx_pick = None;
+        return;
+    };
+    let start = track_scrub_settle(app, &ctx, &mut st, ectx.time.seconds());
+    // The resolution of the frame on screen: the pick reads the pixel it shows.
+    let scale = app
+        .viewer_shown
+        .as_ref()
+        .filter(|(_, k)| k.comp == ectx.comp_id.0)
+        .map_or(1.0, |(_, k)| crate::frames::render_scale(k.scale, ectx.comp.resolution.factor()));
+    // The centre of the shown frame's pixel under the cursor: the layer may be resampled into
+    // the frame (a proxy, a scaled layer), and that pixel is what the user sees.
+    let region = super::viewer_tools::shown_region(&ctx, map.comp);
+    let at = |pos: egui::Pos2| {
+        let c = map.to_comp(pos);
+        let snap = |v: f64, o: f64| o + (((v - o) * scale).floor() + 0.5) / scale;
+        let p = fw::comp_to_layer(m, [snap(c[0], region[0]), snap(c[1], region[1])])?;
+        let (x, y, time) = (p[0], p[1], ectx.time.seconds());
+        Some(
+            json!({"comp": ectx.comp_id.0, "layer": layer.id.0, "effect": fx, "x": x, "y": y, "scale": scale.clamp(1e-3, 1.0), "time": time, "probeOnly": true}),
+        )
+    };
+    // The Hand and Zoom tools (Space held too) and other buttons keep their own clicks and drags.
+    let tools = super::space_hand(&ctx) || matches!(app.ui.tool, Tool::Hand | Tool::Zoom);
+    let nav = tools || ctx.input(|i| i.pointer.middle_down() || i.pointer.secondary_down());
+    // Drags too: a click that drifts must not move the layer under it. (egui hands a press to
+    // what sensed it the pass before, so middle and right presses land here: passed on below.)
+    let resp = ui.interact(map.area, egui::Id::unique("viewer-fx-pick"), if tools { Sense::hover() } else { Sense::click_and_drag() });
+    viewer_pan_and_secondary(app, &ctx, &resp);
+    app.auto.add("viewer.fxPick", map.area, &pick.name);
+    let mut reading = hover_read(app, &ctx, map, &mut st, &resp, start, &at);
+    viewer_click_to_pick(app, ui, map, ectx, &mut st, PointerNav { tools, nav }, &at);
+    // Clicks waiting for their probe ([`make_waiting`], at the end of the pass).
+    reading |= !st.pending.is_empty();
+    draw_coverage_highlight(app, &ctx, painter, map, m, &mut st);
+    draw_status_chip(app, &ctx, painter, map, &st, reading);
+    handle_escape(app, ui, &mut st);
+    ctx.data_mut(|d| d.insert_temp(sid, st));
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn errors_show_without_the_command_prefix() {
+        let e = "invalid parameters for `effect.cryptomatteAt`: this layer has no pixels";
+        assert_eq!(shown_error(e), "This layer has no pixels");
+        assert_eq!(shown_error("Reading the Cryptomatte IDs failed: x"), "Reading the Cryptomatte IDs failed: x");
+    }
+
+    /// A build that panics still clears the busy flag, so the next hover can build again.
+    #[test]
+    fn a_panicking_build_clears_busy() {
+        let busy = Arc::new(AtomicBool::new(true));
+        let done = Done(busy.clone(), egui::Context::default());
+        let r = std::thread::spawn(move || run(Box::new(|| panic!("a broken build")), done)).join();
+        assert!(r.is_err() && !busy.load(Ordering::Acquire));
+    }
+}

crates/ui-egui/src/panels/viewer.rs

@@ -63,6 +63,10 @@ fn map_id() -> egui::Id {
     egui::Id::unique("viewer-map")
 }
 
+/// The egui data key (an `egui::Pos2`) of a right-click on the viewer that an overlay sensed:
+/// the viewer opens its context menu there on the next pass.
+pub(crate) const FORWARDED_SECONDARY: &str = "viewer-forwarded-secondary";
+
 pub fn comp_to_screen(ctx: &egui::Context, p: [f32; 2]) -> Option<Pos2> {
     let m: ViewerMap = ctx.data(|d| d.get_temp(map_id()))?;
     Some(m.to_screen([p[0] as f64, p[1] as f64]))