mirror of
https://github.com/storytold/craft-libs.git
synced 2026-11-18 00:05:08 +00:00
* craft-ipc: local-socket server, client and discovery for Craft apps Adds craft-ipc, the transport for a Craft app that talks to other Craft apps on one machine. It is native only: interprocess has no wasm32 build. - server and client: one JSON object per line, push events, a bulk connection for frames, token auth on every connection, connection and time limits. - discovery: one record per running app in a per-user runtime directory, with a lock file that shows whether the owner is alive. - launch, trust, cache: find an app binary, start it only after an executable check, keep caches in a private per-user directory. - activate: focus hand-off between apps. - moniker, record, payload, asset, refresh, fetch, watch: asset names, stored link records, open payloads, linked-file polling and a frame cache. - PROTOCOL.md describes the wire protocol. The record schemas and fixtures are in the crate. - The crate compiles and passes clippy -D warnings on x86_64-pc-windows-gnu and aarch64-apple-darwin. This is a compile and lint fix only. No one ran the Windows or macOS code paths, so this does not claim they work there. Part of #414. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: app.activate no longer claims a request it cannot make ActivationBackend gains can_apply_token (default false). On Linux a token only counts when the backend can apply it to a live window. Stock winit cannot, so the answer is false plus a request for user attention. Wire values are unchanged. PROTOCOL.md says what each value means. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: keep a partial reply line across client read timeouts pull_line dropped the bytes it had read when a read timed out, so next_event returned Ok(None) and the next read saw only the tail of the line. The client now keeps the partial line and resumes it on the next read. A test splits an event line across a timeout. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: scope the macOS fallback runtime directory to the user With TMPDIR unset or relative, the fallback was the shared path /tmp/craft. The first user to create it owned it, and every other user then failed the owner check. The fallback is now /tmp/craft-<euid>, as on Linux. The path choice is a pure function with a test. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: fetch rejects non-finite time, bounds its queue, keeps waited-for asks Three changes to FrameFetcher::fetch. A non-finite time_s was queued and failed in request_reply before any socket I/O. The worker read that as a dead connection and exited, so enough bad asks took every worker down. fetch now returns Fetched::Failed and queues nothing. The queue had no length limit, so stalled workers let it grow with the request rate. It now holds at most QUEUE_MAX asks (256, a new public constant). A miss past the cap drops the oldest queued ask and counts it in stats.skipped. next_job dropped any job older than max_age, even when the caller had waited longer than max_age and was still there. The caller then re-queued at the tail. Each job now records the latest deadline of its waiting callers and is skipped for age only when no caller is left. Tests cover each case. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: reject file:// media URLs that do not percent-decode local_path returned None for a file:// URL with a bad percent escape, and missing_media then skipped it, so the server accepted a local target without the media probe. Such a URL now gets payload_invalid. URLs that name no local file (remote host, other scheme, relative path) are still left to the app. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: open watched files without waiting on a FIFO Tracker::poll, Tracker::new and fingerprint opened the path with File::open before stat checked the file type. A FIFO with no writer made that open wait, and poll_entry held the entry lock for the whole call, so one such path stalled every link on the watcher and the watcher drop. The open now sets O_NONBLOCK (new crate fsopen), and stat rejects the FIFO on the open handle with LinkError::Invalid. A check before the open would not close the window between check and open. craft-ipc gains a unix-only dependency on libc for the open flags. libc 0.2.190 is already in Cargo.lock. The lock file gains one dependency line for craft-ipc. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: read_private checks and reads one open handle read_private checked the path with symlink_metadata and then read it with fs::read, so the entry could change between the two calls. On Unix it now opens once with O_NOFOLLOW and O_NONBLOCK, checks type, owner and mode on that handle, and reads from the same handle. A symlink is still Refused (found by looking at the path after a failed open, because the errno differs between Unix systems), a FIFO is Refused and cannot stall the open, and a missing file is still an I/O error. Other platforms keep the old checks. Tests cover the symlink and FIFO cases. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: Tracker reports an epoch it cannot raise instead of a false bump apply saturated the epoch at u64::MAX while poll and relink still returned bumped: true, so Change::bumped (documented as one added to the epoch) was false and a cached frame kept passing for changed content. poll and relink now return LinkError::Invalid and leave the tracker as it was when the bump would pass u64::MAX. A test covers both calls. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: document write_atomic errors that follow the rename The directory flush and the fingerprint run after the rename. An error from either can be returned while the path already holds the new bytes. The doc comment now says so. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: say that check_executable accepts group-writable paths The module and function docs described the check as protection from other local users. A group-writable file or directory passes, and the other members of its group can replace it. The docs now state that and keep the private-group reason. Behavior is unchanged. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: refuse an auth line that completes after the deadline authenticate returned a completed line before it looked at the deadline, so a line that finished after auth_deadline still authenticated. The overshoot was up to one 250 ms read poll. The deadline is now checked first, with the same auth failure reply as the other expiry paths. A test sends the line between the deadline and the next poll. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: do not append an error record after a partial frame write send_frame wrote an error record whenever write_frame failed. When the failure came from the stream (for example a send timeout in the middle of a record), the first record was cut short, and a peer that drained the buffer then read the error record as the rest of it. send_frame now checks that the record can be built before it writes. A record that cannot be built gets the error record as before. A write failure is returned alone and the bulk loop closes the connection. wire gains check_cfrm, the pre-write part of write_cfrm. write_cfrm behaves as before. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: LineReader applies the line limit to a last line without a newline At end of stream LineReader::read returned a buffered line with no length check, so a peer that sent one byte over the limit and closed its write side got that line dispatched, while the same bytes with a newline were refused. The end of stream path now returns TooLarge for a line over the limit. A test covers both paths. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: answer a control reply over the line limit with payload_too_large write_value passed every reply to write_line, which has no size check. A sink result larger than MAX_LINE went out as one line that the client refuses, and the rest of the line stayed unread on the connection. Replies now go through wire::write_reply_line. A reply over the limit is replaced by a payload_too_large error that keeps the request id. When that does not fit either, or the value has no id, nothing is written and the request loop ends. Event lines are unchanged. wire gains write_reply_line. write_line behaves as before. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: accept a shallow or deep poll in the native ticker test The test expected the first notice to come from a shallow poll. With a 50 ms deep interval the poll that first sees the edit can be a deep one, so the assertion failed on a slow or loaded machine. It now takes the deep flag from the notice and checks every other field. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: refuse a frame stride that cannot hold one row FrameLayout::check only tested len == stride * h, so w = 4, stride = 1, h = 3, len = 3 was a valid rgba8 layout. A reader that splits the payload by stride cannot decode it. frames.open, a rendered frame and the client reply parser now require stride >= w * bytes per pixel (4 for rgba8_premul, 8 for rgba16f_premul). open checks before it mints a ticket, so a refused layout leaves no ticket. The bulk test source used stride 16 for rgba16f with w = 4, which needs 32. It now uses a stride that fits the format. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: reject calendar days that do not exist in a record's created field valid_rfc3339 checked the day only against 1 to 31, so 2026-02-31T09:30:00Z passed LinkRecord::validate while the schema's date-time format refused it. The year, month and day now go through chrono::NaiveDate::from_ymd_opt, as the workspace rules require for calendar logic. The accepted syntax is unchanged. The schema agreement table gets rows for impossible days and for leap days. craft-ipc gains the workspace chrono dependency. chrono is already in Cargo.lock; the lock file gains one dependency line for craft-ipc. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: make the link record moniker pattern match Moniker::parse The asset.moniker pattern accepted any id without / ? #, so a record the schema passed could fail LinkRecord::validate (bad or truncated escape, raw space, escape that decodes to invalid UTF-8, id over 512 bytes). The pattern is now two patterns under allOf. One checks the shape: unreserved characters and two-digit escapes only, 1 to 512 of them, which is 1 to 512 decoded bytes. The other checks that the decoded bytes are valid UTF-8, written as the well-formed UTF-8 byte sequences. Neither uses lookahead, so any ECMA 262 engine can run them. The schema agreement test gets rows for each case in the list above and for the length limits, and a sweep of lead, second and tail bytes that compares the schema verdict with the Rust verdict. This tightens the published schema: documents it accepted that Rust refused are now refused. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: document that Assets methods must return in bounded time Server::shutdown and drop join every connection thread, and an Assets call runs on one with no deadline and no view of the stop flag. The trait docs now say so and ask implementations to bound their own I/O. Behavior is unchanged. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: name the writer type in the listener tests Clippy's type_complexity lint failed on the test helper's return type. Behavior is unchanged. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: run Assets and Activate calls on detached helper threads with a deadline Server::shutdown and drop join every connection thread, and an Assets or Activate call ran on one with no deadline and no view of the stop flag, so a provider that never returned made shutdown hang. These calls now run on a helper thread (server/provider.rs). The connection thread waits for the answer in 100 ms slices, for at most Limits::provider_deadline (default REPLY_DEADLINE_SECS) and not past the stop flag. On timeout the caller gets a timeout reply and the helper is detached, so shutdown does not join it. At most 32 helper threads are alive at once, stuck ones included, and a call past that gets a busy error without starting a thread. A panic in a provider becomes an io_error reply. Frames stays on the connection thread because it is one call per frame. Its trait docs now say that the stop flag is checked before each request and that a running call is waited for. The module docs describe the shutdown guarantee for each provider kind. Public API: new Limits::provider_deadline field. Tests use a provider that blocks until released, for Assets and for Activate. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: give a caller whose queued ask was dropped a Failed outcome When the fetch queue was full, the oldest ask was dropped but its waiting callers got no answer. They woke, found the key neither cached nor in flight, and queued it again, which dropped another ask and repeated the cycle. A dropped ask now records a failure for its key before the waiters are woken, so each waiter returns Fetched::Failed and does not queue the key again. The drop-oldest policy and stats.skipped are unchanged. Failure records are shared with the worker path and kept to a fixed number, oldest first. No type or variant changes. The Fetched::Failed doc now covers this case. A test overflows the queue while a caller waits on the oldest key, and checks that the caller returns at once and that no further asks are dropped. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: require two hex digits in a file URL percent escape percent_decode used u8::from_str_radix, which accepts a leading plus, so file:///tmp/a%+fb decoded to the byte 0x0f and went on to the media probe instead of getting payload_invalid. The two bytes after a percent sign must now both be ASCII hex digits. The decode test gets a plus, a minus, a space and a short escape. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: make the euid probe create only its own file, and gate the macOS path test On Unix systems other than Linux, current_euid read the owner of a file named .craft-uid-<pid> in the temp directory, made with create and truncate. A file or symlink planted at that name was opened and truncated. The probe now lives in probe_euid: random name, create_new (which refuses an existing path), mode 0600, and removal before it returns. The std library has no safe euid call and this crate forbids unsafe code, so the probe stays. A test plants a file and a symlink at the old and a similar name and checks that both are untouched and that the probe file is gone. macos_fallback_is_per_user compares Unix paths, so it and the macos_runtime_path helper it needs are now compiled for tests on Unix only. That removes the unused function warning in Windows test builds. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * craft-ipc: an event larger than one line becomes an events.overflow marker Server::emit wrote the serialized event unchecked. A line over MAX_LINE made Client::next_event return TooLarge and left the stream out of sync. enqueue now checks the serialized size and queues the existing overflow marker instead. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 * ipc: validate executable path ancestry * ipc: preserve queued replies after macOS peer shutdown * ipc: validate native transports and use portable platform fixtures * Make IPC integration fixtures portable and collision-free * Use native paths and unique directories across IPC fixtures * Bound Windows pipe I/O and stabilize IPC tests * Retry empty nonblocking Windows pipes * #433: Serialize hub tests sharing a Windows pipe * #433: Use native endpoint in schema capability tests * ipc: report a non-regular watch target as Invalid on every platform * ipc: format the new error classification --------- Co-authored-by: User <bigwolfeman@users.noreply.github.com> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
562 lines
21 KiB
Rust
562 lines
21 KiB
Rust
//! JSON lines and bulk frame records. No sockets live here.
|
|
|
|
use std::io::{BufRead, Read, Write};
|
|
use std::time::{Duration, Instant};
|
|
|
|
use serde_json::{Map, Value, json};
|
|
|
|
use crate::error::LinkError;
|
|
|
|
/// Longest accepted control line, without the newline. Matches the app control servers.
|
|
pub const MAX_LINE: usize = 4 * 1024 * 1024;
|
|
/// Open connections per server, authenticated or not. Bulk connections count.
|
|
pub const MAX_CONNECTIONS: usize = 16;
|
|
/// Connections that have not authenticated yet. The other `MAX_CONNECTIONS - this` slots are
|
|
/// reserved for authenticated peers.
|
|
pub const MAX_UNAUTHENTICATED: usize = 4;
|
|
pub const EVENT_QUEUE: usize = 256;
|
|
pub const REPLY_DEADLINE_SECS: u64 = 60;
|
|
/// `link.open` payload ceiling. The control line cap is the real limit: a longer request line
|
|
/// closes the connection. This number is the same, so `OpenPayload` gives `payload_too_large`
|
|
/// to a caller that does not come through the line reader.
|
|
pub const MAX_LINK_PAYLOAD: u64 = MAX_LINE as u64;
|
|
pub const BULK_HEADER_MAX: u32 = 1024 * 1024;
|
|
pub const FRAME_PAYLOAD_MAX: u64 = 256 * 1024 * 1024;
|
|
pub const PROTOCOL_MIN: u32 = 1;
|
|
pub const PROTOCOL_MAX: u32 = 1;
|
|
pub const AUTH_ERROR: &str = "authentication required";
|
|
|
|
const CFRQ: &[u8; 4] = b"CFRQ";
|
|
const CFRM: &[u8; 4] = b"CFRM";
|
|
|
|
/// One control request. `id` is JSON null when the caller omitted it.
|
|
#[derive(Debug, Clone, PartialEq)]
|
|
pub struct Request {
|
|
pub id: Value,
|
|
pub method: String,
|
|
pub params: Value,
|
|
}
|
|
|
|
/// Read one line of at most `max` bytes. `Ok(None)` is a clean end of stream.
|
|
///
|
|
/// This keeps no state between calls. A read timeout drops the bytes read so far, so use it only
|
|
/// on a stream that cannot time out. A server loop that polls with a receive timeout uses
|
|
/// [`LineReader`].
|
|
pub fn read_line(reader: &mut impl BufRead, max: usize) -> Result<Option<Vec<u8>>, LinkError> {
|
|
LineReader::new(max).read(reader)
|
|
}
|
|
|
|
/// Reads lines from a stream whose reads can time out. Bytes read before a timeout stay in the
|
|
/// reader, so a line that arrives in pieces more than one poll apart is not lost.
|
|
#[derive(Debug)]
|
|
pub struct LineReader {
|
|
buf: Vec<u8>,
|
|
max: usize,
|
|
started: Option<Instant>,
|
|
patience: Option<Duration>,
|
|
}
|
|
|
|
impl LineReader {
|
|
pub fn new(max: usize) -> Self {
|
|
Self { buf: Vec::new(), max, started: None, patience: None }
|
|
}
|
|
|
|
/// How long the current unfinished line has been open. `None` when no line is in progress.
|
|
pub fn partial_age(&self) -> Option<Duration> {
|
|
self.started.map(|start| start.elapsed())
|
|
}
|
|
|
|
/// Return `Err(Timeout)` once a line has been open this long, even when bytes keep arriving.
|
|
/// Without it a peer that sends one byte per poll holds the connection for ever.
|
|
pub fn with_patience(mut self, patience: Duration) -> Self {
|
|
self.patience = Some(patience);
|
|
self
|
|
}
|
|
|
|
/// Change the limit that [`LineReader::with_patience`] sets.
|
|
pub fn set_patience(&mut self, patience: Duration) {
|
|
self.patience = Some(patience);
|
|
}
|
|
|
|
/// Read the next line. On `Err(Timeout)` the partial line stays here and the next call
|
|
/// continues it. `Ok(None)` is a clean end of stream with no partial line.
|
|
pub fn read(&mut self, reader: &mut impl BufRead) -> Result<Option<Vec<u8>>, LinkError> {
|
|
loop {
|
|
let available = match reader.fill_buf() {
|
|
Ok(bytes) => bytes,
|
|
Err(err) if err.kind() == std::io::ErrorKind::Interrupted => continue,
|
|
Err(err) => return Err(err.into()),
|
|
};
|
|
if available.is_empty() {
|
|
// End of stream. A final line without a newline is still a line.
|
|
if self.buf.is_empty() {
|
|
return Ok(None);
|
|
}
|
|
let line = self.finish();
|
|
if line.len() > self.max {
|
|
return Err(LinkError::TooLarge);
|
|
}
|
|
return Ok(Some(line));
|
|
}
|
|
let newline = available.iter().position(|byte| *byte == b'\n');
|
|
let take = newline.map_or(available.len(), |at| at + 1);
|
|
if self.buf.len().saturating_add(take) > self.max.saturating_add(1) {
|
|
return Err(LinkError::TooLarge);
|
|
}
|
|
self.buf.extend_from_slice(&available[..take]);
|
|
reader.consume(take);
|
|
if self.started.is_none() {
|
|
self.started = Some(Instant::now());
|
|
}
|
|
if newline.is_some() {
|
|
let mut line = self.finish();
|
|
line.pop();
|
|
if line.last() == Some(&b'\r') {
|
|
line.pop();
|
|
}
|
|
if line.len() > self.max {
|
|
return Err(LinkError::TooLarge);
|
|
}
|
|
return Ok(Some(line));
|
|
}
|
|
if self.patience.zip(self.partial_age()).is_some_and(|(limit, age)| age >= limit) {
|
|
return Err(LinkError::Timeout);
|
|
}
|
|
}
|
|
}
|
|
|
|
fn finish(&mut self) -> Vec<u8> {
|
|
self.started = None;
|
|
std::mem::take(&mut self.buf)
|
|
}
|
|
}
|
|
|
|
/// Parse one control line.
|
|
///
|
|
/// `Ok(None)` is a blank line (the caller skips it). `Err` is the existing
|
|
/// control-server reject text: one reply, then close.
|
|
pub fn parse_request(bytes: &[u8]) -> Result<Option<Request>, String> {
|
|
let text = std::str::from_utf8(bytes).map_err(|_| "not a control request (invalid UTF-8); closing the connection".to_string())?;
|
|
let text = text.trim();
|
|
if text.is_empty() {
|
|
return Ok(None);
|
|
}
|
|
let msg: Value =
|
|
serde_json::from_str(text).map_err(|err| format!("bad JSON ({err}); expected one JSON request object per line; closing the connection"))?;
|
|
let Some(obj) = msg.as_object() else {
|
|
return Err("not a control request (expected a JSON object); closing the connection".into());
|
|
};
|
|
let Some(method) = obj.get("method").and_then(Value::as_str) else {
|
|
return Err("not a control request (missing string `method`); closing the connection".into());
|
|
};
|
|
if method.is_empty() {
|
|
return Err("not a control request (missing string `method`); closing the connection".into());
|
|
}
|
|
let id = obj.get("id").cloned().unwrap_or(Value::Null);
|
|
if !(id.is_null() || id.is_number() || id.is_string()) {
|
|
return Err("not a control request (id must be a number, string, or null); closing the connection".into());
|
|
}
|
|
let params = obj.get("params").cloned().unwrap_or_else(|| json!({}));
|
|
Ok(Some(Request { id, method: method.to_string(), params }))
|
|
}
|
|
|
|
pub fn params_object(params: &Value) -> Result<&Map<String, Value>, LinkError> {
|
|
params.as_object().ok_or_else(|| LinkError::BadParams("params must be an object".into()))
|
|
}
|
|
|
|
pub fn auth_failure() -> Value {
|
|
json!({"id": 0, "ok": false, "error": AUTH_ERROR})
|
|
}
|
|
|
|
pub fn auth_success(id: &Value) -> Value {
|
|
let id = if id.is_null() { json!(0) } else { id.clone() };
|
|
json!({"id": id, "ok": true, "result": {"authenticated": true}})
|
|
}
|
|
|
|
pub fn reply_ok(id: &Value, result: Value) -> Value {
|
|
json!({"id": id, "ok": true, "result": result})
|
|
}
|
|
|
|
pub fn reply_err_text(id: &Value, error: &str) -> Value {
|
|
json!({"id": id, "ok": false, "error": error})
|
|
}
|
|
|
|
pub fn reply_err(id: &Value, err: &LinkError) -> Value {
|
|
json!({"id": id, "ok": false, "error": err.to_object()})
|
|
}
|
|
|
|
pub fn push_event(event: &str, seq: u64, data: Value) -> Value {
|
|
json!({"event": event, "seq": seq, "data": data})
|
|
}
|
|
|
|
/// Write a reply line that a peer will read with the [`MAX_LINE`] limit. A value longer than the
|
|
/// limit is replaced by a `payload_too_large` reply that keeps the request `id`. When even that
|
|
/// does not fit, or the value has no `id` to answer, nothing is written and the result is
|
|
/// `Err(TooLarge)`. The limit counts the content without the newline, as the readers do.
|
|
pub fn write_reply_line(writer: &mut impl Write, value: &Value) -> Result<(), LinkError> {
|
|
let mut bytes = serde_json::to_vec(value)?;
|
|
if bytes.len() > MAX_LINE {
|
|
let id = value.get("id").ok_or(LinkError::TooLarge)?;
|
|
bytes = serde_json::to_vec(&reply_err(id, &LinkError::TooLarge))?;
|
|
if bytes.len() > MAX_LINE {
|
|
return Err(LinkError::TooLarge);
|
|
}
|
|
}
|
|
bytes.push(b'\n');
|
|
writer.write_all(&bytes)?;
|
|
writer.flush()?;
|
|
Ok(())
|
|
}
|
|
|
|
pub fn write_line(writer: &mut impl Write, value: &Value) -> Result<(), LinkError> {
|
|
let mut bytes = serde_json::to_vec(value)?;
|
|
bytes.push(b'\n');
|
|
writer.write_all(&bytes)?;
|
|
writer.flush()?;
|
|
Ok(())
|
|
}
|
|
|
|
pub fn protocols_overlap(min: u32, max: u32) -> bool {
|
|
min <= max && min <= PROTOCOL_MAX && max >= PROTOCOL_MIN
|
|
}
|
|
|
|
pub fn write_cfrq(writer: &mut impl Write, header: &Value) -> Result<(), LinkError> {
|
|
write_magic(writer, CFRQ, header)?;
|
|
Ok(())
|
|
}
|
|
|
|
pub fn read_cfrq(reader: &mut impl Read) -> Result<Value, LinkError> {
|
|
read_magic(reader, CFRQ)
|
|
}
|
|
|
|
/// The checks that [`write_cfrm`] makes before it writes a byte: the header is an object that fits
|
|
/// `BULK_HEADER_MAX`, and the payload fits `FRAME_PAYLOAD_MAX`. A record that fails them has put
|
|
/// nothing on the wire, so a caller may still send a different record. An error from `write_cfrm`
|
|
/// after this check passed came from the stream and may have left part of a record on it.
|
|
pub fn check_cfrm(header: &Value, payload: &[u8]) -> Result<(), LinkError> {
|
|
cfrm_parts(header, payload).map(|_| ())
|
|
}
|
|
|
|
fn cfrm_parts(header: &Value, payload: &[u8]) -> Result<(Vec<u8>, u64), LinkError> {
|
|
let header_bytes = header_bytes(header)?;
|
|
let payload_len = u64::try_from(payload.len()).map_err(|_| LinkError::TooLarge)?;
|
|
if payload_len > FRAME_PAYLOAD_MAX {
|
|
return Err(LinkError::TooLarge);
|
|
}
|
|
Ok((header_bytes, payload_len))
|
|
}
|
|
|
|
pub fn write_cfrm(writer: &mut impl Write, header: &Value, payload: &[u8]) -> Result<(), LinkError> {
|
|
let (header_bytes, payload_len) = cfrm_parts(header, payload)?;
|
|
writer.write_all(CFRM)?;
|
|
writer.write_all(&u32_len(&header_bytes)?.to_le_bytes())?;
|
|
writer.write_all(&payload_len.to_le_bytes())?;
|
|
writer.write_all(&header_bytes)?;
|
|
writer.write_all(payload)?;
|
|
writer.flush()?;
|
|
Ok(())
|
|
}
|
|
|
|
pub fn read_cfrm(reader: &mut impl Read) -> Result<(Value, Vec<u8>), LinkError> {
|
|
let mut magic = [0u8; 4];
|
|
reader.read_exact(&mut magic)?;
|
|
if &magic != CFRM {
|
|
return Err(LinkError::Invalid("bulk reply magic".into()));
|
|
}
|
|
let header_len = read_u32(reader)?;
|
|
if header_len > BULK_HEADER_MAX {
|
|
return Err(LinkError::TooLarge);
|
|
}
|
|
let payload_len = read_u64(reader)?;
|
|
if payload_len > FRAME_PAYLOAD_MAX {
|
|
return Err(LinkError::TooLarge);
|
|
}
|
|
let header = read_header(reader, header_len)?;
|
|
let mut payload = vec![0u8; usize::try_from(payload_len).map_err(|_| LinkError::TooLarge)?];
|
|
reader.read_exact(&mut payload)?;
|
|
Ok((header, payload))
|
|
}
|
|
|
|
fn write_magic(writer: &mut impl Write, magic: &[u8; 4], header: &Value) -> Result<(), LinkError> {
|
|
let bytes = header_bytes(header)?;
|
|
writer.write_all(magic)?;
|
|
writer.write_all(&u32_len(&bytes)?.to_le_bytes())?;
|
|
writer.write_all(&bytes)?;
|
|
writer.flush()?;
|
|
Ok(())
|
|
}
|
|
|
|
fn read_magic(reader: &mut impl Read, expect: &[u8; 4]) -> Result<Value, LinkError> {
|
|
let mut magic = [0u8; 4];
|
|
reader.read_exact(&mut magic)?;
|
|
if &magic != expect {
|
|
return Err(LinkError::Invalid("bulk request magic".into()));
|
|
}
|
|
let len = read_u32(reader)?;
|
|
if len > BULK_HEADER_MAX {
|
|
return Err(LinkError::TooLarge);
|
|
}
|
|
read_header(reader, len)
|
|
}
|
|
|
|
fn header_bytes(header: &Value) -> Result<Vec<u8>, LinkError> {
|
|
if !header.is_object() {
|
|
return Err(LinkError::BadParams("bulk header must be an object".into()));
|
|
}
|
|
let bytes = serde_json::to_vec(header)?;
|
|
if bytes.len() > usize::try_from(BULK_HEADER_MAX).unwrap_or(usize::MAX) {
|
|
return Err(LinkError::TooLarge);
|
|
}
|
|
Ok(bytes)
|
|
}
|
|
|
|
fn u32_len(bytes: &[u8]) -> Result<u32, LinkError> {
|
|
u32::try_from(bytes.len()).map_err(|_| LinkError::TooLarge)
|
|
}
|
|
|
|
fn read_u32(reader: &mut impl Read) -> Result<u32, LinkError> {
|
|
let mut buf = [0u8; 4];
|
|
reader.read_exact(&mut buf)?;
|
|
Ok(u32::from_le_bytes(buf))
|
|
}
|
|
|
|
fn read_u64(reader: &mut impl Read) -> Result<u64, LinkError> {
|
|
let mut buf = [0u8; 8];
|
|
reader.read_exact(&mut buf)?;
|
|
Ok(u64::from_le_bytes(buf))
|
|
}
|
|
|
|
fn read_header(reader: &mut impl Read, len: u32) -> Result<Value, LinkError> {
|
|
let mut buf = vec![0u8; len as usize];
|
|
reader.read_exact(&mut buf)?;
|
|
let value: Value = serde_json::from_slice(&buf)?;
|
|
if !value.is_object() {
|
|
return Err(LinkError::BadParams("bulk header must be an object".into()));
|
|
}
|
|
Ok(value)
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use std::io::Cursor;
|
|
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn blank_and_request_round_trip() {
|
|
assert!(parse_request(b" \n").unwrap().is_none());
|
|
let req = parse_request(br#"{"id":1,"method":"engine.execute","params":{"command":"x"}}"#).unwrap().unwrap();
|
|
assert_eq!(req.method, "engine.execute");
|
|
assert_eq!(req.id, json!(1));
|
|
assert!(parse_request(b"[]").is_err());
|
|
assert!(parse_request(b"{").is_err());
|
|
}
|
|
|
|
#[test]
|
|
fn a_reply_over_the_line_limit_becomes_a_too_large_error_with_the_same_id() {
|
|
let mut out = Vec::new();
|
|
let small = reply_ok(&json!(3), json!({"a": 1}));
|
|
write_reply_line(&mut out, &small).unwrap();
|
|
assert_eq!(out, [serde_json::to_vec(&small).unwrap(), b"\n".to_vec()].concat(), "a reply inside the limit is written as it is");
|
|
|
|
// The content is exactly MAX_LINE bytes: still inside.
|
|
let overhead = serde_json::to_vec(&reply_ok(&json!(3), json!(""))).unwrap().len();
|
|
let exact = reply_ok(&json!(3), json!("x".repeat(MAX_LINE - overhead)));
|
|
assert_eq!(serde_json::to_vec(&exact).unwrap().len(), MAX_LINE);
|
|
let mut out = Vec::new();
|
|
write_reply_line(&mut out, &exact).unwrap();
|
|
assert_eq!(out.len(), MAX_LINE + 1);
|
|
|
|
let big = reply_ok(&json!(3), json!("x".repeat(MAX_LINE - overhead + 1)));
|
|
let mut out = Vec::new();
|
|
write_reply_line(&mut out, &big).unwrap();
|
|
let line = read_line(&mut Cursor::new(out), MAX_LINE).unwrap().unwrap();
|
|
let reply: Value = serde_json::from_slice(&line).unwrap();
|
|
assert_eq!(reply["id"], json!(3));
|
|
assert_eq!(reply["ok"], json!(false));
|
|
assert_eq!(reply["error"]["code"], json!("payload_too_large"));
|
|
|
|
// No id to answer: nothing is written.
|
|
let mut out = Vec::new();
|
|
let anonymous = json!({"ok": true, "result": "x".repeat(MAX_LINE)});
|
|
assert!(matches!(write_reply_line(&mut out, &anonymous), Err(LinkError::TooLarge)));
|
|
assert!(out.is_empty());
|
|
// An id so long that the error reply does not fit either.
|
|
let huge_id = json!({"id": "i".repeat(MAX_LINE + 1), "ok": true});
|
|
assert!(matches!(write_reply_line(&mut out, &huge_id), Err(LinkError::TooLarge)));
|
|
assert!(out.is_empty());
|
|
}
|
|
|
|
#[test]
|
|
fn a_last_line_without_a_newline_obeys_the_limit() {
|
|
let at_limit = vec![b'a'; 8];
|
|
let line = read_line(&mut Cursor::new(at_limit.clone()), 8).unwrap().unwrap();
|
|
assert_eq!(line, at_limit);
|
|
let over = vec![b'a'; 9];
|
|
assert!(matches!(read_line(&mut Cursor::new(over), 8), Err(LinkError::TooLarge)));
|
|
// The same bytes with a newline are refused too, so the two paths agree.
|
|
let mut with_newline = vec![b'a'; 9];
|
|
with_newline.push(b'\n');
|
|
assert!(matches!(read_line(&mut Cursor::new(with_newline), 8), Err(LinkError::TooLarge)));
|
|
}
|
|
|
|
#[test]
|
|
fn line_limit_and_crlf() {
|
|
let mut cur = Cursor::new(b"abc\r\n".to_vec());
|
|
let line = read_line(&mut cur, 8).unwrap().unwrap();
|
|
assert_eq!(line, b"abc");
|
|
let mut huge = vec![b'a'; 20];
|
|
huge.push(b'\n');
|
|
let mut cur = Cursor::new(huge);
|
|
assert!(matches!(read_line(&mut cur, 8), Err(LinkError::TooLarge)));
|
|
}
|
|
|
|
/// A reader that hands out `chunks` and fails with a timeout after each one, like a socket
|
|
/// with a receive timeout and a slow peer.
|
|
struct Stuttering {
|
|
chunks: std::collections::VecDeque<Vec<u8>>,
|
|
pending_timeout: bool,
|
|
current: Vec<u8>,
|
|
pos: usize,
|
|
}
|
|
|
|
impl Stuttering {
|
|
fn new(chunks: Vec<Vec<u8>>) -> Self {
|
|
Self { chunks: chunks.into(), pending_timeout: false, current: Vec::new(), pos: 0 }
|
|
}
|
|
}
|
|
|
|
impl Read for Stuttering {
|
|
fn read(&mut self, out: &mut [u8]) -> std::io::Result<usize> {
|
|
if self.pos >= self.current.len() {
|
|
if self.pending_timeout {
|
|
self.pending_timeout = false;
|
|
return Err(std::io::ErrorKind::WouldBlock.into());
|
|
}
|
|
let Some(next) = self.chunks.pop_front() else { return Ok(0) };
|
|
self.current = next;
|
|
self.pos = 0;
|
|
if self.current.is_empty() {
|
|
return Err(std::io::ErrorKind::WouldBlock.into());
|
|
}
|
|
self.pending_timeout = true;
|
|
}
|
|
let n = out.len().min(self.current.len() - self.pos);
|
|
out[..n].copy_from_slice(&self.current[self.pos..self.pos + n]);
|
|
self.pos += n;
|
|
Ok(n)
|
|
}
|
|
}
|
|
|
|
fn collect_lines(chunks: Vec<Vec<u8>>, max: usize) -> Vec<Vec<u8>> {
|
|
let mut reader = std::io::BufReader::with_capacity(4, Stuttering::new(chunks));
|
|
let mut lines = LineReader::new(max);
|
|
let mut got = Vec::new();
|
|
loop {
|
|
match lines.read(&mut reader) {
|
|
Ok(Some(line)) => got.push(line),
|
|
Ok(None) => return got,
|
|
Err(LinkError::Timeout) => {}
|
|
Err(other) => panic!("{other:?}"),
|
|
}
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn a_line_split_at_every_offset_survives_a_timeout() {
|
|
let text = b"{\"id\":1,\"method\":\"link.hello\"}\r\n{\"id\":2}\n";
|
|
let want = vec![b"{\"id\":1,\"method\":\"link.hello\"}".to_vec(), b"{\"id\":2}".to_vec()];
|
|
for cut in 0..=text.len() {
|
|
let chunks = vec![text[..cut].to_vec(), text[cut..].to_vec()];
|
|
assert_eq!(collect_lines(chunks, 1024), want, "cut {cut}");
|
|
}
|
|
// Two cuts, and one byte at a time.
|
|
for a in 0..text.len() {
|
|
for b in a..text.len() {
|
|
let chunks = vec![text[..a].to_vec(), text[a..b].to_vec(), text[b..].to_vec()];
|
|
assert_eq!(collect_lines(chunks, 1024), want, "cuts {a} {b}");
|
|
}
|
|
}
|
|
let bytes: Vec<Vec<u8>> = text.iter().map(|byte| vec![*byte]).collect();
|
|
assert_eq!(collect_lines(bytes, 1024), want);
|
|
}
|
|
|
|
#[test]
|
|
fn the_line_cap_holds_across_timeouts() {
|
|
let mut reader = std::io::BufReader::new(Stuttering::new(vec![vec![b'a'; 6], vec![b'b'; 6], b"\n".to_vec()]));
|
|
let mut lines = LineReader::new(8);
|
|
let mut result = None;
|
|
for _ in 0..8 {
|
|
match lines.read(&mut reader) {
|
|
Err(LinkError::Timeout) => {}
|
|
other => {
|
|
result = Some(other);
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
assert!(matches!(result, Some(Err(LinkError::TooLarge))), "{result:?}");
|
|
}
|
|
|
|
/// One byte per read, a short pause each, and never a newline: a slow-loris peer whose reads
|
|
/// never time out.
|
|
struct Trickle;
|
|
|
|
impl Read for Trickle {
|
|
fn read(&mut self, out: &mut [u8]) -> std::io::Result<usize> {
|
|
std::thread::sleep(Duration::from_millis(10));
|
|
out[0] = b'a';
|
|
Ok(1)
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn a_trickling_line_is_cut_by_patience() {
|
|
let mut reader = std::io::BufReader::with_capacity(1, Trickle);
|
|
let mut lines = LineReader::new(MAX_LINE).with_patience(Duration::from_millis(100));
|
|
let start = Instant::now();
|
|
assert!(matches!(lines.read(&mut reader), Err(LinkError::Timeout)));
|
|
assert!(start.elapsed() < Duration::from_secs(2), "{:?}", start.elapsed());
|
|
assert!(lines.partial_age().is_some_and(|age| age >= Duration::from_millis(100)));
|
|
}
|
|
|
|
#[test]
|
|
fn partial_age_tracks_an_open_line() {
|
|
let mut reader = std::io::BufReader::new(Stuttering::new(vec![b"abc".to_vec(), b"d\n".to_vec()]));
|
|
let mut lines = LineReader::new(64);
|
|
assert!(lines.partial_age().is_none());
|
|
assert!(matches!(lines.read(&mut reader), Err(LinkError::Timeout)));
|
|
assert!(lines.partial_age().is_some());
|
|
assert_eq!(lines.read(&mut reader).unwrap().unwrap(), b"abcd");
|
|
assert!(lines.partial_age().is_none());
|
|
}
|
|
|
|
#[test]
|
|
fn bulk_round_trip() {
|
|
let header = json!({"req": 7, "time_s": 1.5, "epoch": 3, "region": null});
|
|
let mut buf = Vec::new();
|
|
write_cfrq(&mut buf, &header).unwrap();
|
|
let got = read_cfrq(&mut Cursor::new(&buf)).unwrap();
|
|
assert_eq!(got["req"], json!(7));
|
|
|
|
let mut buf = Vec::new();
|
|
let reply = json!({"req": 7, "error": "no"});
|
|
write_cfrm(&mut buf, &reply, b"").unwrap();
|
|
let (h, payload) = read_cfrm(&mut Cursor::new(&buf)).unwrap();
|
|
assert_eq!(h["error"], json!("no"));
|
|
assert!(payload.is_empty());
|
|
|
|
let mut buf = Vec::new();
|
|
write_cfrm(&mut buf, &json!({"req": 1, "w": 2}), &[9, 8, 7]).unwrap();
|
|
let (_h, payload) = read_cfrm(&mut Cursor::new(buf)).unwrap();
|
|
assert_eq!(payload, vec![9, 8, 7]);
|
|
}
|
|
|
|
#[test]
|
|
fn protocol_window() {
|
|
assert!(protocols_overlap(1, 1));
|
|
assert!(!protocols_overlap(2, 2));
|
|
assert!(!protocols_overlap(2, 1));
|
|
}
|
|
}
|