Files
photocraft/crates/automation/tests/it/bridge.rs
T
274f6d3e26 MCP bridge: waited commands poll the app's job, with progress and cancel (#8130) (#8134)
* MCP headless: waited commands run as jobs, with progress and cancel (#8129)

Headless command_run and command_batch steps that wait (the default) ran
job-capable commands inline while holding the session lock. Cancelling a
long Gaussian Blur did not stop it: it still added its history step, and
doc_inspect and jobs_list queued behind it. No progress was reported.

A waited job-capable command now starts as a background job
(Headless::command_begin) and the server polls it with the lock released
(Headless::job_poll), so other calls are served meanwhile. With a progress
token it sends notifications/progress (strictly increasing, at most ten a
second; for a batch, finished steps plus the current step's fraction).
notifications/cancelled cancels the job and leaves the document unchanged;
a drop guard does the same when rmcp drops the request future. Headless
keeps the outcome of each waited job as sync_jobs sees it end, since
jobs.list keeps only the last few. Both backends share one batch loop,
which runs no further step once the request is cancelled.

Tool hints: jobs_list is read-only, doc_select is non-destructive and
idempotent, and the scoped exports (which refuse to overwrite) are
non-destructive. doc_save's description lost a run of spaces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MCP headless: cancel a waited job whose request was dropped before it started

The Abandon guard was created only after command_begin returned. A request
dropped while the start still waited for the session lock left the job it
then started running, and its `waited` entry behind. The guard is now armed
inside the locked closure that starts the job, so dropping the unread output
cancels it. Review feedback on #8132.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MCP headless: job_poll applies finished jobs before checking the job

A caller polling Headless::job_poll directly, without sync_jobs in between,
kept seeing Running after the worker finished, because the session only
drops a finished job from its running list when its jobs are polled.
job_poll now syncs first. The MCP path already synced in headless_op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MCP headless: a cancel that loses the race returns the job's result

job_abandon ignored whether Session::cancel_job cancelled anything and
always dropped the stored outcome, so headless_wait answered `Cancelled`
for a job another request had already applied to the document. It now
returns how the job really ended: Cancelled only when it cancelled the
job, the job's result when it had already ended, and an error when the
job cannot be cancelled (printing) and is still running.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MCP headless: waited commands run as jobs, with progress and cancel (#8129)

Headless command_run and command_batch steps that wait (the default) ran
job-capable commands inline while holding the session lock. Cancelling a
long Gaussian Blur did not stop it: it still added its history step, and
doc_inspect and jobs_list queued behind it. No progress was reported.

A waited job-capable command now starts as a background job
(Headless::command_begin) and the server polls it with the lock released
(Headless::job_poll), so other calls are served meanwhile. With a progress
token it sends notifications/progress (strictly increasing, at most ten a
second; for a batch, finished steps plus the current step's fraction).
notifications/cancelled cancels the job and leaves the document unchanged;
a drop guard does the same when rmcp drops the request future. Headless
keeps the outcome of each waited job as sync_jobs sees it end, since
jobs.list keeps only the last few. Both backends share one batch loop,
which runs no further step once the request is cancelled.

Tool hints: jobs_list is read-only, doc_select is non-destructive and
idempotent, and the scoped exports (which refuse to overwrite) are
non-destructive. doc_save's description lost a run of spaces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MCP headless: cancel a waited job whose request was dropped before it started

The Abandon guard was created only after command_begin returned. A request
dropped while the start still waited for the session lock left the job it
then started running, and its `waited` entry behind. The guard is now armed
inside the locked closure that starts the job, so dropping the unread output
cancels it. Review feedback on #8132.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MCP bridge: waited commands poll the app's job, with progress and cancel (#8130)

A waited command_run or command_batch step of an engine command now starts it
with engine.execute {wait: false} (sent once) and, when the app runs it as a
background job, polls jobs.list until it ends. A job can run past the bridge's
55s reply timeout, its progress is forwarded, and cancelling or dropping the
request sends jobs.cancel. UI command ids keep wait: true. A failed poll or a
job no longer listed is reported with its id rather than guessed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MCP bridge: keep the cancel guard armed until jobs.cancel returns; clarify the docs

The cancel branch disarmed the Abandon guard before awaiting jobs.cancel, so a
request future dropped during that await sent no cancel. The control-protocol
table now says polling applies only to a waited command_run. Review feedback
on #8134.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MCP headless: job_poll applies finished jobs before checking the job

A caller polling Headless::job_poll directly, without sync_jobs in between,
kept seeing Running after the worker finished, because the session only
drops a finished job from its running list when its jobs are polled.
job_poll now syncs first. The MCP path already synced in headless_op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MCP headless: a cancel that loses the race returns the job's result

job_abandon ignored whether Session::cancel_job cancelled anything and
always dropped the stored outcome, so headless_wait answered `Cancelled`
for a job another request had already applied to the document. It now
returns how the job really ended: Cancelled only when it cancelled the
job, the job's result when it had already ended, and an error when the
job cannot be cancelled (printing) and is still running.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MCP bridge: a refused jobs.cancel is not reported as a cancellation

bridge_wait discarded the app's reply to jobs.cancel and always answered
`Cancelled`. The app refuses to cancel a job that is no longer running,
so a job that finished between the jobs.list poll and the cancel was
reported as cancelled although its edit may be applied. A refused cancel
now answers that the job was not cancelled and points to jobs_list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MCP bridge: a cancel during a failed poll still cancels the job

When notifications/cancelled arrived while a jobs.list poll was in
flight, and the poll and its retry then failed, bridge_wait disarmed the
Abandon guard and returned the poll error without checking the waiter,
so no jobs.cancel reached the app and the edit ran on after the caller
cancelled it. That error path now sends jobs.cancel first when the
request was cancelled. A failed poll without a cancel still leaves the
job running and names it.

Reported in review of #8134.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MCP bridge: a cancel doesn't wait for a slow jobs.list poll

bridge_wait checked for a cancel only between polls, so a cancel that
arrived while the app was slow to answer jobs.list waited for the poll,
up to the 55s call timeout, before jobs.cancel was sent. The poll now
races the cancel.

That needs BridgeClient to drop the connection of a call abandoned
before its reply: it kept the one cached connection, so jobs.cancel
queued behind the unanswered poll on the same socket (the app serves a
connection's requests in order), and a half-sent request could share it.
A call now takes the connection out of the cache and returns it only
once answered, as a timeout or lost reply already dropped it.

The poll-error check from the previous commit stays for a cancel that
races a failing poll; its test is replaced by one for the slow poll,
which covers the reviewed case. A BridgeClient test checks that a
dropped call's connection is not reused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(automation): retain cleanup after failed bridge cancellation

* chore: restart CI for PR #8134

* test(automation): classify the smart object replacement MCP tool

* test(automation): synchronize dropped bridge waits and accept reconnections

* Fix CI after merging upstream

* Fix CI after merging upstream

* Deduplicate Arabic closure messages after upstream merge

* fix(engine): bound active and cancelling background workers

* Fix cancellation and draining in bridge merge

* Test release-aware draining cleanup

* Fix draining regression warning

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Khawar Zamman Wani <wanikhawar@Khawars-MacBook-Air.local>
Co-authored-by: FredApps <fredapps@users.noreply.github.com>
2026-11-05 13:35:13 +01:00

203 lines
9.9 KiB
Rust

//! Bridge client behavior against a scripted fake control server: reconnect before
//! idle sockets die, retry only caller-vouched reads, never resend edits (#1007),
//! never retry timeouts.
use std::time::Duration;
use photocraft_automation::security::authentication_reply;
use photocraft_automation::{AutomationError, BridgeClient};
use serde_json::{Value, json};
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::net::{TcpListener, TcpStream};
const CONTROL_TOKEN: &str = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
/// What the fake server does after serving one request on a connection.
#[derive(Clone, Copy, PartialEq)]
enum After {
KeepOpen,
Close,
}
/// Serve connections until `max_connections` were seen. Each connection authenticates,
/// then serves requests until it closes; every served request is recorded, and `accepted`
/// counts connections as they arrive.
async fn fake_server(
max_connections: usize,
after: After,
seen: tokio::sync::mpsc::UnboundedSender<Vec<Value>>,
accepted: std::sync::Arc<std::sync::atomic::AtomicUsize>,
) -> String {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap().to_string();
tokio::spawn(async move {
let mut connections = 0;
while connections < max_connections {
let Ok((socket, _)) = listener.accept().await else { break };
connections += 1;
accepted.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
let seen = seen.clone();
let after = after;
tokio::spawn(async move {
let mut served = Vec::new();
let mut socket = socket;
authenticate(&mut socket).await;
let (read, mut write) = socket.into_split();
let mut reader = BufReader::new(read);
let mut line = String::new();
loop {
line.clear();
match reader.read_line(&mut line).await {
Ok(0) | Err(_) => break,
Ok(_) => {}
}
let Ok(request) = serde_json::from_str::<Value>(&line) else { continue };
served.push(request.clone());
let reply = json!({"id": request["id"], "ok": true, "result": {"served": served.len()}});
if write.write_all(format!("{reply}\n").as_bytes()).await.is_err() {
break;
}
if after == After::Close {
break; // the reply is flushed, then both halves drop: a dead connection
}
}
let _ = seen.send(served);
});
}
});
addr
}
/// Authenticate one connection: read the auth line, reply to it.
async fn authenticate(socket: &mut TcpStream) {
let (read, mut write) = socket.split();
let mut reader = BufReader::new(read);
let mut line = String::new();
reader.read_line(&mut line).await.unwrap();
let (auth, ok) = authentication_reply(&line, CONTROL_TOKEN);
assert!(ok, "each connection must authenticate");
write.write_all(format!("{auth}\n").as_bytes()).await.unwrap();
}
#[tokio::test(flavor = "multi_thread")]
async fn read_retries_on_a_dropped_connection_and_edits_do_not() {
// Connection 1 serves one read then dies; the next vouched read retries on
// connection 2. The edit, sent after connection 2 also died, must fail with the
// #1007 wording and never open connection 3.
let (seen_tx, mut seen_rx) = tokio::sync::mpsc::unbounded_channel();
let accepted = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let addr = fake_server(2, After::Close, seen_tx, accepted).await;
let bridge = BridgeClient::new(&addr, CONTROL_TOKEN).unwrap();
let first = bridge.call_read("document.inspect", json!({})).await.expect("read served before the connection dies");
assert_eq!(first["served"], 1);
let retried = bridge.call_read("document.inspect", json!({})).await.expect("a vouched read retried on a fresh connection");
assert_eq!(retried["served"], 1, "connection 2 sees it as its own first request");
let edit = bridge.call("engine.execute", json!({"command": "layer.new.layer", "params": {}})).await;
let error = edit.unwrap_err().to_string();
assert!(error.contains("operation may have completed"), "{error}");
assert!(error.contains("inspect state before retrying"), "{error}");
assert!(!error.contains("retried"), "{error}");
let mut all = Vec::new();
while let Ok(served) = seen_rx.try_recv() {
all.extend(served);
}
assert_eq!(all.len(), 2, "one request per connection, no edit resend: {all:?}");
}
#[tokio::test(flavor = "multi_thread")]
async fn an_idle_connection_is_replaced_before_the_next_call() {
// The production idle threshold sits under the app's 30s idle cutoff; the test
// shrinks it so the reconnect path runs quickly. The next call after idling must
// authenticate a new connection before sending.
let (seen_tx, mut seen_rx) = tokio::sync::mpsc::unbounded_channel();
let accepted = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let addr = fake_server(2, After::KeepOpen, seen_tx, std::sync::Arc::clone(&accepted)).await;
let bridge = BridgeClient::new(&addr, CONTROL_TOKEN).unwrap().with_idle_reconnect(Duration::from_millis(100));
bridge.call_read("document.inspect", json!({})).await.expect("first call connects");
tokio::time::sleep(Duration::from_millis(250)).await;
bridge.call_read("document.inspect", json!({})).await.expect("second call reconnects after idling");
assert_eq!(accepted.load(std::sync::atomic::Ordering::SeqCst), 2, "the idle connection was replaced, not reused");
let first = seen_rx.try_recv().expect("the replaced connection closed, reporting what it served");
assert_eq!(first.len(), 1, "each connection served only the request that followed its auth: {first:?}");
}
#[tokio::test(flavor = "multi_thread")]
async fn a_timed_out_call_is_not_retried() {
// The server authenticates but never answers requests, keeping the connection open:
// the call hits the client's own timeout, which must name the likely causes and must
// not double the wait by retrying (the fake server counts every request it sees).
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap().to_string();
let requests = std::sync::Arc::new(std::sync::Mutex::new(Vec::<Value>::new()));
let seen = std::sync::Arc::clone(&requests);
tokio::spawn(async move {
let (mut socket, _) = listener.accept().await.unwrap();
authenticate(&mut socket).await;
let (read, _write) = socket.into_split();
let mut reader = BufReader::new(read);
let mut line = String::new();
loop {
line.clear();
if reader.read_line(&mut line).await.unwrap_or(0) == 0 {
return;
}
if let Ok(request) = serde_json::from_str::<Value>(&line) {
seen.lock().unwrap().push(request);
}
}
});
let bridge = BridgeClient::new(&addr, CONTROL_TOKEN).unwrap().with_timeout(Duration::from_millis(300));
let error = match bridge.call_read("document.inspect", json!({})).await {
Err(e @ AutomationError::Bridge(_)) => e.to_string(),
other => panic!("expected the bridge timeout, got {other:?}"),
};
assert!(error.contains("timed out after"), "{error}");
assert!(error.contains("not drawing frames"), "{error}");
assert!(error.contains("a read may be retried safely"), "{error}");
assert!(!error.contains("may have completed"), "a read timeout never claims an applied edit: {error}");
assert_eq!(requests.lock().unwrap().len(), 1, "a timed-out call is not resent");
}
#[tokio::test(flavor = "multi_thread")]
async fn a_call_dropped_before_its_reply_does_not_reuse_its_connection() {
// Connection 1 never answers. A call dropped while waiting on it (a cancelled MCP wait) must
// close it: on the same socket the next request would queue behind the unanswered one.
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap().to_string();
let accepted = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let count = std::sync::Arc::clone(&accepted);
tokio::spawn(async move {
while let Ok((mut socket, _)) = listener.accept().await {
let silent = count.fetch_add(1, std::sync::atomic::Ordering::SeqCst) == 0;
tokio::spawn(async move {
authenticate(&mut socket).await;
let (read, mut write) = socket.into_split();
let mut lines = BufReader::new(read).lines();
while let Ok(Some(line)) = lines.next_line().await {
let Ok(request) = serde_json::from_str::<Value>(&line) else { continue };
if !silent {
let reply = json!({"id": request["id"], "ok": true, "result": {"served": true}});
if write.write_all(format!("{reply}\n").as_bytes()).await.is_err() {
return;
}
}
}
});
}
});
let bridge = BridgeClient::new(&addr, CONTROL_TOKEN).unwrap().with_timeout(Duration::from_secs(30));
let dropped = tokio::time::timeout(Duration::from_millis(300), bridge.call_read("jobs.list", json!({}))).await;
assert!(dropped.is_err(), "the silent connection answered: {dropped:?}");
let next = tokio::time::timeout(Duration::from_secs(10), bridge.call("jobs.cancel", json!({"job": 7}))).await;
assert_eq!(next.expect("the next call waited on the dropped call's connection").unwrap()["served"], true);
assert_eq!(accepted.load(std::sync::atomic::Ordering::SeqCst), 2, "the next call opened a fresh connection");
}