diff --git a/ROADMAP.md b/ROADMAP.md index 3d65350f0..4219bad02 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,7 @@ **Stage: alpha** · next: beta, ~25 points of "ready for real work" and ~160–240 h away -> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-02 · **Change:** minor (RefEdit reference boxes in dialogs: Goal Seek, Solver and Data Analysis take ranges by selection on the grid; earlier: Python mode in the formula bar; earlier: Python cells stored and kept in files; earlier: equations: Insert › Equation, editor, gridcraft-math; earlier: What-if and analysis: Solver Sensitivity report for Simplex LP and GRG Nonlinear; earlier: floating objects: one selection, eight resize handles, keyboard delete; earlier: Chart data tables: live source values beneath column, bar, line, area and compatible combo charts; legend keys and borders; XLSX round-trip and print/PDF; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and Insert Function categories; XLSX workbook-view selection, bounded tab ratios and sheet-edit regressions; readiness estimates unchanged; earlier: CSV and text open detects each column's date order, so day-first dates are not silently swapped) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-02 · **Change:** minor (form controls: buttons, sliders and spin buttons; earlier: RefEdit reference boxes in dialogs: Goal Seek, Solver and Data Analysis take ranges by selection on the grid; earlier: Python mode in the formula bar; earlier: Python cells stored and kept in files; earlier: equations: Insert › Equation, editor, gridcraft-math; earlier: What-if and analysis: Solver Sensitivity report for Simplex LP and GRG Nonlinear; earlier: floating objects: one selection, eight resize handles, keyboard delete; earlier: Chart data tables: live source values beneath column, bar, line, area and compatible combo charts; legend keys and borders; XLSX round-trip and print/PDF; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and Insert Function categories; XLSX workbook-view selection, bounded tab ratios and sheet-edit regressions; readiness estimates unchanged; earlier: CSV and text open detects each column's date order, so day-first dates are not silently swapped) · **Target:** Microsoft Excel (Microsoft 365) GridCraft aims at full Microsoft Excel parity, and at being better in places: faster, open (XLSX is the native format), fully scriptable by agents (MCP, control channel, CLI), and on the web. @@ -22,7 +22,7 @@ This page is the summary. The detail is in: | Number | Value | Kind | |---|---|---| | 2026-10-14 | Editing a cell with text that would read back as a number, boolean, date or formula (entered with a leading apostrophe) shows the apostrophe in the editor and formula bar, so Enter unchanged keeps it text (#1362) | -| **Feature breadth** (weighted) | **~75%** | estimated, anchored on two measurements: ribbon catalog 277 / 292 (94.9%), worksheet functions 513 / 523 (98.1%; 504 working, 96.4%) | +| **Feature breadth** (weighted) | **~75%** | estimated, anchored on two measurements: ribbon catalog 281 / 296 (94.9%), worksheet functions 513 / 523 (98.1%; 504 working, 96.4%) | | **Ready for real work** | **~50%** | estimated from written weights (51.6); [target-app-parity.md](docs/target-app-parity.md) | | **Mainstream practitioner** | **~50%** | estimated: typical analyst/finance user, weekly features, × 0.90 interaction × 0.92 stability × 0.88 file exchange; [method](docs/target-app-parity.md#mainstream-practitioner) | | **Essentials user** | **~63%** | estimated: core features only, × 0.92 launch/stability × 0.95 clarity × 0.90 opening sent files; [method](docs/target-app-parity.md#essentials-user) | @@ -135,6 +135,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | Date | What landed | |---|---| +| 2026-11-02 | Form controls: Insert › Controls places buttons, sliders (scroll bars) and spin buttons on the sheet. A slider or spin button holds a whole number from 0 to 30000 in a linked cell (a drag or a held arrow is one undo step, and editing the cell moves the control); a button runs a macro or one command through the macro policy, asking first in a workbook opened from a file; Assign Action and Format Control edit them, and XLSX files read and write them (control properties, `` and VML), alongside the form controls GridCraft keeps unchanged | | 2026-11-02 | Dialogs take ranges by selection on the grid: reference boxes (RefEdit) with a collapse button that shrinks the dialog while the person selects, in Goal Seek, Solver (including the Model Range of Load/Save) and Data Analysis; while collapsed, the arrow keys move the selection and Shift+arrow extends it, and F4 cycles a reference's anchors | | 2026-11-02 | Python mode in the formula bar: Ctrl+Alt+Shift+P, `=PY` then Tab, or editing a Python cell edits its code in the formula bar, with Enter for a new line, Tab to indent and Ctrl+Enter to commit through `python.set`. The code still doesn't run | | 2026-11-02 | Python cells as data: `python.set` stores code in a cell, Python cells copy, move, fill and undo like formulas, and `.xlsx` reads and writes Python cells as [MS-XLSX] specifies (`_xlfn._xlws.PY(index,0 or 1)`, the code in the workbook's Python part) with cached values and spills, so such workbooks open with their results; nothing runs Python (new code shows `#BLOCKED!`), and opening a file limits code size, result size and spill ranges | @@ -257,6 +258,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | Date | Change | Summary | |---|---|---| +| 2026-11-02 | minor | Form controls; catalog 277 / 292 → 281 / 296 (Developer › Controls); progress log row | | 2026-11-02 | minor | Progress log: RefEdit reference boxes in dialogs | | 2026-11-02 | minor | Progress log: Python mode in the formula bar | | 2026-11-02 | minor | Progress log: Python cells as data | diff --git a/crates/engine/src/catalog.rs b/crates/engine/src/catalog.rs index 45455e00b..234e92e58 100644 --- a/crates/engine/src/catalog.rs +++ b/crates/engine/src/catalog.rs @@ -275,6 +275,11 @@ pub const CATALOG: &[(&str, &str, &str, &str)] = &[ ("automate.allScripts", "Automate", "Scripting Tools", "All Scripts"), ("automate.recordActions", "Automate", "Scripting Tools", "Record Actions"), ("automate.trustScripts", "Automate", "Scripting Tools", "Trust Scripts"), + // Developer (Insert › Controls in GridCraft) + ("insert.button", "Developer", "Controls", "Button (Form Control)"), + ("insert.spinButton", "Developer", "Controls", "Spin Button (Form Control)"), + ("insert.slider", "Developer", "Controls", "Scroll Bar (Form Control)"), + ("control.format", "Developer", "Controls", "Properties"), // Table Design ("table.rename", "Table Design", "Properties", "Table Name"), ("table.resize", "Table Design", "Properties", "Resize Table"), diff --git a/crates/engine/src/cmd.rs b/crates/engine/src/cmd.rs index 02f6dabca..bd18996bd 100644 --- a/crates/engine/src/cmd.rs +++ b/crates/engine/src/cmd.rs @@ -2,6 +2,7 @@ //! `data.sortAscending`) plus primitive editing commands (`cell.set`, `selection.set`). pub mod analysis; +pub mod controls; pub mod data; pub mod draw; pub mod edit; @@ -165,6 +166,7 @@ pub fn command_specs() -> &'static [CommandSpec] { v.extend(format::specs()); v.extend(sheet::specs()); v.extend(insert::specs()); + v.extend(controls::specs()); v.extend(equation::specs()); v.extend(data::specs()); v.extend(formulas::specs()); @@ -435,6 +437,7 @@ pub(crate) fn commit(d: &mut DocState, f: impl FnOnce(&mut Ctx) -> Result) } else if !changed.is_empty() { d.calc.cells_changed(&mut wb, &changed); } + controls::sync_linked_values(&mut wb); if let Some(t0) = t0 { eprintln!("commit: + recalc {:?}", since(t0)); } diff --git a/crates/engine/src/cmd/controls.rs b/crates/engine/src/cmd/controls.rs new file mode 100644 index 000000000..f1064ce33 --- /dev/null +++ b/crates/engine/src/cmd/controls.rs @@ -0,0 +1,711 @@ +//! Form controls (Insert › Controls): buttons that run a macro or a command, and sliders and spin +//! buttons that hold a whole number from 0 to 30000 and write it to a linked cell. +//! +//! A slider or spin button's value lives on the control and in its linked cell. Setting it goes +//! through `cell.set`, so the cell's dependents recalculate and Undo takes both back as one step; +//! editing the linked cell moves the control (see [`sync_linked_values`]). Pressing a button runs +//! its action through the macro policy with the document's trust, so a button from a file someone +//! else made asks before it runs anything. + +use gridcraft_core::CellRef; +use gridcraft_model::{ + Anchor, AnchorMode, CONTROL_MAX, ControlAction, ControlKind, FormControl, MAX_MACRO_NAME, MacroInput, ObjKind, Sheet, Workbook, +}; +use serde_json::{Value as Json, json}; + +use super::{ + CommandSpec, EngineError, PROTECTED, Result, Session, anchor_mode_param, bad, bool_param, cell_param, cmd, edit, f64_param, find_command, + has_doc, split_sheet, stash, str_param, u32_param, +}; +use crate::macros::policy::{Policy, macro_policy}; + +pub fn specs() -> Vec { + vec![ + cmd!( + "insert.button", + "Button", + ["Insert", "Controls"], + None, + "{at?: \"B2\" (or cell), dx?, dy?, width?, height?, text?, macro?: name, input?: \"selection\" | \"B2:D9\", output?: \"F2\", command?: id, params?: {..}} places a button; pressing it runs the macro or the command → {control: id}", + has_doc, + insert_button + ), + cmd!( + "insert.slider", + "Slider", + ["Insert", "Controls"], + None, + "{at?: \"B2\" (or cell), dx?, dy?, width?, height?, link?: \"$B$1\" | \"Sheet2!$B$1\", min?: 0, max?: 100, step?: 1, page?: 10, value?, horizontal?: true} whole numbers, 0 ≤ min < max ≤ 30000, 1 ≤ step ≤ max − min, page ≥ step → {control: id}", + has_doc, + |s, p| insert_number_control(s, p, ControlKind::Slider) + ), + cmd!( + "insert.spinButton", + "Spin Button", + ["Insert", "Controls"], + None, + "{at?: \"B2\" (or cell), dx?, dy?, width?, height?, link?: \"$B$1\", min?: 0, max?: 100, step?: 1, value?} whole numbers, 0 ≤ min < max ≤ 30000, 1 ≤ step ≤ max − min → {control: id}", + has_doc, + |s, p| insert_number_control(s, p, ControlKind::Spin) + ), + cmd!( + "control.format", + "Format Control", + [], + None, + "{id, text?, name?, link?: \"$B$1\" | \"\" (no link), min?, max?, step?, page?, value?, horizontal?, print?, locked?} → the control (as control.list)", + has_doc, + format + ), + cmd!( + "control.assign", + "Assign Action", + [], + None, + "{id, macro?: name, input?: \"selection\" | \"B2:D9\", output?: \"F2\" | command?: id, params?: {..} | clear?: true} sets what a button runs → the control", + has_doc, + assign + ), + cmd!( + "control.setValue", + "Change Control Value", + [], + None, + "{id, value} clamps the value to min..max, snaps it to the step, and writes it to the linked cell → {value}", + has_doc, + set_value + ), + cmd!( + "control.press", + "Press Button", + [], + None, + "{id, approve?: bool, onConflict?, overwrite?} runs a button's macro or command (through the macro policy, with the workbook's trust; a workbook from a file asks first) → what the macro or command returned", + has_doc, + press + ), + cmd!(query "control.list", "Form Controls", [], None, "{} → the active sheet's controls: [{id, kind, name, text, at, link?, value, min, max, step, page, horizontal, action?}]", has_doc, list), + ] +} + +/// The longest caption or name a control takes. +const MAX_TEXT: usize = 1_024; + +fn control_json(c: &FormControl) -> Json { + let mut o = json!({ + "id": c.id, + "kind": match c.kind { ControlKind::Button => "button", ControlKind::Slider => "slider", ControlKind::Spin => "spinButton" }, + "name": c.display_name(), + "at": c.anchor.cell.a1(), + "width": c.anchor.width, + "height": c.anchor.height, + "print": c.print, + "locked": c.locked, + }); + if c.kind == ControlKind::Button { + o["text"] = json!(c.text); + if let Some(a) = &c.action { + o["action"] = serde_json::to_value(a).unwrap_or(Json::Null); + } + } else { + o["value"] = json!(c.value); + o["min"] = json!(c.min); + o["max"] = json!(c.max); + o["step"] = json!(c.step); + if c.kind == ControlKind::Slider { + o["page"] = json!(c.page); + o["horizontal"] = json!(c.horizontal); + } + } + if let Some(l) = &c.link { + o["link"] = json!(l); + } + o +} + +fn list(s: &mut Session, _: &Json) -> Result { + let sh = s.doc()?.wb.active().ok_or(EngineError::NoDocument)?; + Ok(Json::Array(sh.controls.iter().map(control_json).collect())) +} + +/// A whole number from 0 to 30000 named `key`, if given. Numbers in text (`"12"`) count; anything +/// else is refused with a message that names the parameter. +fn whole(cmd: &str, p: &Json, key: &str) -> Result> { + let v = match p.get(key) { + None | Some(Json::Null) => return Ok(None), + Some(Json::Number(n)) => n.as_f64(), + Some(Json::String(t)) => t.trim().parse::().ok(), + Some(_) => None, + }; + match v { + Some(x) if x.is_finite() && x.fract() == 0.0 && (0.0..=CONTROL_MAX as f64).contains(&x) => Ok(Some(x as u32)), + _ => Err(bad(cmd, format!("`{key}` must be a whole number from 0 to {CONTROL_MAX}"))), + } +} + +fn text_param(cmd: &str, p: &Json, key: &str) -> Result> { + match p.get(key) { + None | Some(Json::Null) => Ok(None), + Some(Json::String(t)) if t.chars().count() <= MAX_TEXT => Ok(Some(t.clone())), + Some(Json::String(_)) => Err(bad(cmd, format!("`{key}` is longer than {MAX_TEXT} characters"))), + Some(_) => Err(bad(cmd, format!("`{key}` must be text"))), + } +} + +/// The cell a link names, from the control's sheet `host`: `(sheet index, cell)`. `None` when it +/// doesn't name one cell of an existing sheet (a link into a deleted sheet reads `#REF!`). +pub fn link_target(wb: &Workbook, host: usize, link: &str) -> Option<(usize, CellRef)> { + let text = link.trim().trim_start_matches('='); + let (sheet, body) = split_sheet(text); + let si = match sheet { + Some(n) => wb.sheet_index(&n)?, + None => host, + }; + wb.sheet(si)?; + Some((si, CellRef::parse(body.trim())?)) +} + +/// A link parameter in its stored form: `$B$2` on the host sheet, `'My Sheet'!$B$2` elsewhere. +/// An empty text removes the link. +fn link_param(cmd: &str, wb: &Workbook, host: usize, p: &Json) -> Result>> { + let text = match p.get("link") { + None => return Ok(None), + Some(Json::Null) => return Ok(Some(None)), + Some(Json::String(t)) => t.trim(), + Some(_) => return Err(bad(cmd, "`link` must be a cell reference such as $B$1")), + }; + if text.is_empty() { + return Ok(Some(None)); + } + let (si, c) = link_target(wb, host, text) + .ok_or_else(|| bad(cmd, format!("`{text}` isn't a cell of this workbook; the link must be one cell, such as $B$1")))?; + let abs = format!("${}${}", gridcraft_core::col_to_letters(c.col), c.row as u64 + 1); + Ok(Some(Some(if si == host { + abs + } else { + let name = wb.sheet(si).map(|sh| sh.name.clone()).unwrap_or_default(); + format!("{}!{abs}", gridcraft_formula::quote_sheet(&name)) + }))) +} + +/// The anchor of a new control: `at` or `cell` (default the active cell), with its size. +fn new_anchor(s: &Session, p: &Json, size: (f32, f32)) -> Result { + let cell = match cell_param(s, p, "at")? { + Some(c) => c, + None => cell_param(s, p, "cell")?.unwrap_or(s.doc()?.selection.active), + }; + let dim = |k: &str, v: f32| f64_param(p, k).filter(|x| x.is_finite()).map_or(v, |x| x.clamp(4.0, 10_000.0) as f32); + let off = |k: &str| f64_param(p, k).filter(|x| x.is_finite()).map_or(0.0, |x| x.clamp(-10_000.0, 10_000.0) as f32); + Ok(Anchor { + cell, + dx: off("dx"), + dy: off("dy"), + width: dim("width", size.0), + height: dim("height", size.1), + mode: anchor_mode_param(p).unwrap_or(AnchorMode::MoveOnly), + }) +} + +fn insert_button(s: &mut Session, p: &Json) -> Result { + const CMD: &str = "insert.button"; + let anchor = new_anchor(s, p, (96.0, 28.0))?; + let action = action_param(s, CMD, p)?; + let d = s.doc()?; + let id = d.wb.next_object_id(); + let text = text_param(CMD, p, "text")?.unwrap_or_else(|| format!("Button {id}")); + let sheet = d.wb.active_sheet(); + let control = FormControl { id, kind: ControlKind::Button, anchor, text, action, ..FormControl::default() }; + edit(s, |cx| { + cx.sheet_mut(sheet)?.controls.push(control); + Ok(json!({"control": id})) + }) +} + +fn insert_number_control(s: &mut Session, p: &Json, kind: ControlKind) -> Result { + let cmd = if kind == ControlKind::Slider { "insert.slider" } else { "insert.spinButton" }; + let horizontal = bool_param(p, "horizontal").unwrap_or(true); + let size = match kind { + ControlKind::Slider if horizontal => (144.0, 18.0), + ControlKind::Slider => (18.0, 144.0), + _ => (18.0, 36.0), + }; + let anchor = new_anchor(s, p, size)?; + let d = s.doc()?; + let sheet = d.wb.active_sheet(); + let id = d.wb.next_object_id(); + let mut c = FormControl { id, kind, anchor, horizontal, ..FormControl::default() }; + numbers(cmd, p, &mut c)?; + if let Some(link) = link_param(cmd, &d.wb, sheet, p)? { + c.link = link; + } + let given = whole(cmd, p, "value")?; + // Without a value, a control starts at the number its linked cell holds (or its minimum). + let start = given.map(f64::from).or_else(|| linked_number(&d.wb, sheet, &c)).unwrap_or(c.min as f64); + c.value = c.snap(start); + let value = c.value; + let link = c.link.clone(); + s.grouped(&format!("Insert {}", kind.label()), |s| { + edit(s, |cx| { + cx.sheet_mut(sheet)?.controls.push(c); + Ok(()) + })?; + if given.is_some() { + write_link(s, sheet, link.as_deref(), value)?; + } + Ok(json!({"control": id})) + }) +} + +/// Applies `min`, `max`, `step`, `page` and `value` from `p` to `c` and checks the result. +fn numbers(cmd: &str, p: &Json, c: &mut FormControl) -> Result<()> { + for (key, slot) in [("min", &mut c.min), ("max", &mut c.max), ("step", &mut c.step), ("page", &mut c.page)] { + if let Some(v) = whole(cmd, p, key)? { + *slot = v; + } + } + // A new slider's page follows a larger step unless it was given. + if p.get("page").is_none() && c.page < c.step { + c.page = c.step; + } + c.check_numbers().map_err(|e| bad(cmd, e)) +} + +/// The number the control's linked cell holds, if it holds one. +fn linked_number(wb: &Workbook, host: usize, c: &FormControl) -> Option { + let (si, cell) = link_target(wb, host, c.link.as_deref()?)?; + wb.sheet(si)?.cell(cell)?.value.as_f64() +} + +/// Writes `value` to the linked cell through `cell.set` (nothing without a link, or when the cell +/// already shows `value`, formula or not: Format Control's OK sends the value back every time, and +/// a write that changes nothing must not turn a formula into a constant or fail on a locked cell +/// of a protected sheet; a real change replaces a formula with the constant, as in Excel). +fn write_link(s: &mut Session, host: usize, link: Option<&str>, value: u32) -> Result<()> { + let Some(link) = link else { return Ok(()) }; + let Some((si, cell)) = link_target(&s.doc()?.wb, host, link) else { return Ok(()) }; + let holds = s.doc()?.wb.sheet(si).and_then(|sh| sh.cell(cell)).and_then(|c| c.value.as_f64()); + if holds == Some(value as f64) { + return Ok(()); + } + s.execute("cell.set", json!({"sheet": si, "cell": cell.a1(), "input": value.to_string()}))?; + Ok(()) +} + +/// The control `id` names on the active sheet: (sheet, its index in `controls`, a copy). +fn find(s: &Session, cmd: &str, p: &Json) -> Result<(usize, usize, FormControl)> { + let id = u32_param(p, "id").ok_or_else(|| bad(cmd, "missing `id` (a control id from control.list)"))?; + let d = s.doc()?; + let sheet = d.wb.active_sheet(); + let sh = d.wb.sheet(sheet).ok_or(EngineError::NoDocument)?; + let i = sh.controls.iter().position(|c| c.id == id).ok_or_else(|| bad(cmd, format!("there is no control {id} on this sheet")))?; + let c = sh.controls.get(i).cloned().ok_or_else(|| bad(cmd, "no such control"))?; + Ok((sheet, i, c)) +} + +/// Drawing objects on a protected sheet can't be changed while they are locked. +/// For the object commands (`object.move`, `object.delete`, `object.setAnchorMode`, `arrange.*`, +/// `arrange.align`): refuses to change form controls `ids` on `sheet` when one is locked and the +/// sheet is protected, as `control.format` and `control.assign` do. Other kinds pass. +pub(crate) fn check_objects_unlocked(s: &Session, sheet: usize, kind: ObjKind, ids: &[u32]) -> Result<()> { + if kind != ObjKind::Control { + return Ok(()); + } + let Some(sh) = s.doc()?.wb.sheet(sheet) else { return Ok(()) }; + if sh.is_protected() && sh.controls.iter().any(|c| c.locked && ids.contains(&c.id)) { + return Err(EngineError::Other(PROTECTED.into())); + } + Ok(()) +} + +fn check_unlocked(s: &Session, sheet: usize, c: &FormControl) -> Result<()> { + if c.locked && s.doc()?.wb.sheet(sheet).is_some_and(Sheet::is_protected) { + return Err(EngineError::Other(PROTECTED.into())); + } + Ok(()) +} + +fn format(s: &mut Session, p: &Json) -> Result { + const CMD: &str = "control.format"; + let (sheet, i, mut c) = find(s, CMD, p)?; + check_unlocked(s, sheet, &c)?; + if let Some(t) = text_param(CMD, p, "text")? { + c.text = t; + } + if let Some(n) = text_param(CMD, p, "name")? { + c.name = n.trim().to_string(); + } + for (key, slot) in [("print", &mut c.print), ("locked", &mut c.locked), ("horizontal", &mut c.horizontal)] { + if let Some(v) = bool_param(p, key) { + *slot = v; + } + } + let mut given = None; + if c.kind != ControlKind::Button { + for (key, slot) in [("min", &mut c.min), ("max", &mut c.max), ("step", &mut c.step), ("page", &mut c.page)] { + if let Some(v) = whole(CMD, p, key)? { + *slot = v; + } + } + c.check_numbers().map_err(|e| bad(CMD, e))?; + given = whole(CMD, p, "value")?; + } + if let Some(link) = link_param(CMD, &s.doc()?.wb, sheet, p)? { + c.link = link; + } + // New bounds pull the value in; a given value is snapped to them. + c.value = c.snap(given.map_or(c.value as f64, f64::from)); + let number = c.kind != ControlKind::Button; + let write = number && (given.is_some() || p.get("min").is_some() || p.get("max").is_some() || p.get("step").is_some()); + let (value, link) = (c.value, c.link.clone()); + s.grouped("Format Control", |s| { + edit(s, |cx| { + if let Some(slot) = cx.sheet_mut(sheet)?.controls.get_mut(i) { + *slot = c; + } + Ok(()) + })?; + if write { + write_link(s, sheet, link.as_deref(), value)?; + } + Ok(()) + })?; + // The reply is the control as committed: a new link to a cell holding another number has + // moved its value there. + let (_, _, c) = find(s, CMD, p)?; + Ok(control_json(&c)) +} + +/// Commands a button never runs, beyond the macro policy's lists: a button has no way to show +/// what its command would open, so a web page is refused when the action is assigned or comes in +/// from a file. A command that turns out to open a page or a dialog when it runs is refused +/// then, by [`press`]. +const BUTTON_DENIED: &[(&str, &str)] = &[("link.open", "buttons don't open web pages")]; + +/// The macro policy for a button's command, with the button's own refusals first. +fn button_policy(id: &str, params: &Json, trust: crate::Trust) -> Policy { + match BUTTON_DENIED.iter().find(|(d, _)| *d == id) { + Some((_, why)) => Policy::Deny((*why).into()), + None => macro_policy(id, params, trust), + } +} + +/// Drops the UI requests a button's command queued from `queued` on, keeping messages for the +/// person, and says why the press is refused when one would open a dialog or a web page, which +/// buttons don't do (`name` is the button's). A cell edit the command asked for is dropped too. +pub(crate) fn take_requests(s: &mut Session, queued: usize, name: &str) -> Option { + use crate::UiRequest; + let mut blocked = None; + let asked: Vec = s.ui_requests.drain(queued.min(s.ui_requests.len())..).collect(); + for q in asked { + match q { + UiRequest::Dialog(d, _) => blocked = Some(format!("'{name}' would open the {d} dialog; give its command the parameters it needs.")), + UiRequest::OpenUrl(_) => blocked = Some(format!("'{name}' would open a web page, which buttons don't do.")), + UiRequest::EditCell(_) | UiRequest::InsertName(_) => {} + m @ UiRequest::Message(_) => s.ui_requests.push(m), + } + } + blocked +} + +/// The stored name of the workbook's macro `name` when it exists and declares no inputs: a fixed +/// input range would make every press fail (`macro.run` refuses inputs it doesn't take), so such +/// an assignment is refused when it is made and dropped to the selection when a file brings one +/// in. A macro the workbook doesn't have (yet) is not checked: the assignment is kept as given. +fn takes_no_inputs(wb: &Workbook, name: &str) -> Option { + crate::macros::find(wb, name).and_then(|i| wb.macros.get(i)).filter(|m| m.inputs.is_empty()).map(|m| m.name.clone()) +} + +/// A button action from `macro` (with `input` and `output`) or `command` (with `params`). +fn action_param(s: &Session, cmd: &str, p: &Json) -> Result> { + if let Some(name) = str_param(p, "macro").map(str::trim).filter(|n| !n.is_empty()) { + if name.chars().count() > MAX_MACRO_NAME { + return Err(bad(cmd, "`macro` is not a macro name")); + } + let input = match str_param(p, "input").map(str::trim) { + None | Some("") => MacroInput::Selection, + Some(t) if t.eq_ignore_ascii_case("selection") => MacroInput::Selection, + Some(t) => { + let (_, body) = split_sheet(t); + if gridcraft_core::RangeRef::parse(body).is_none() { + return Err(bad(cmd, format!("`input` must be \"selection\" or a range such as B2:D9, not `{t}`"))); + } + if let Some(stored) = takes_no_inputs(&s.doc()?.wb, name) { + return Err(bad(cmd, format!("macro '{stored}' takes no inputs, so `input` must be \"selection\""))); + } + MacroInput::Fixed(t.to_string()) + } + }; + let output = str_param(p, "output").map(str::trim).filter(|o| !o.is_empty()).map(str::to_string); + if let Some(o) = &output + && gridcraft_core::RangeRef::parse(split_sheet(o).1).is_none() + { + return Err(bad(cmd, format!("`output` must be a cell such as F2, not `{o}`"))); + } + return Ok(Some(ControlAction::Macro { name: name.to_string(), input, output })); + } + if let Some(id) = str_param(p, "command").map(str::trim).filter(|n| !n.is_empty()) { + if find_command(id).is_none() { + return Err(bad(cmd, format!("there is no command `{id}`"))); + } + let params = p.get("params").cloned().unwrap_or_else(|| json!({})); + if !params.is_object() { + return Err(bad(cmd, "`params` must be an object")); + } + let trust = s.doc()?.trust; + if let Policy::Deny(why) = button_policy(id, ¶ms, trust) { + return Err(EngineError::Other(format!("A button can't run {id}: {why}."))); + } + return Ok(Some(ControlAction::Command { id: id.to_string(), params })); + } + Ok(None) +} + +fn assign(s: &mut Session, p: &Json) -> Result { + const CMD: &str = "control.assign"; + let (sheet, i, mut c) = find(s, CMD, p)?; + check_unlocked(s, sheet, &c)?; + if c.kind != ControlKind::Button { + return Err(bad(CMD, "only buttons run actions; a slider or spin button writes its value to its linked cell")); + } + let action = action_param(s, CMD, p)?; + if action.is_none() && bool_param(p, "clear") != Some(true) { + return Err(bad(CMD, "pass `macro` or `command` (or `clear: true` to remove the action)")); + } + c.action = action; + let out = control_json(&c); + edit(s, |cx| { + if let Some(slot) = cx.sheet_mut(sheet)?.controls.get_mut(i) { + *slot = c; + } + Ok(out) + }) +} + +fn set_value(s: &mut Session, p: &Json) -> Result { + const CMD: &str = "control.setValue"; + let (sheet, i, c) = find(s, CMD, p)?; + if c.kind == ControlKind::Button { + return Err(bad(CMD, "a button has no value; use control.press")); + } + let v = match p.get("value") { + Some(Json::Number(n)) => n.as_f64(), + Some(Json::String(t)) => t.trim().parse::().ok(), + _ => None, + } + .filter(|v| v.is_finite()) + .ok_or_else(|| bad(CMD, "`value` must be a number"))?; + let value = c.snap(v); + let label = format!("Change {}", c.kind.label()); + s.grouped(&label, |s| { + if c.value != value { + edit(s, |cx| { + if let Some(slot) = cx.sheet_mut(sheet)?.controls.get_mut(i) { + slot.value = value; + } + Ok(()) + })?; + } + write_link(s, sheet, c.link.as_deref(), value)?; + Ok(json!({"value": value})) + }) +} + +fn press(s: &mut Session, p: &Json) -> Result { + const CMD: &str = "control.press"; + let (_, _, c) = find(s, CMD, p)?; + if c.kind != ControlKind::Button { + return Err(bad(CMD, "only buttons can be pressed; use control.setValue for a slider or spin button")); + } + let name = c.display_name(); + match c.action { + None => Err(EngineError::Other(format!("'{name}' has no action. Right-click it and choose Assign Action."))), + Some(ControlAction::Macro { name: mac, input, output }) => { + let mut params = json!({"name": mac}); + if let MacroInput::Fixed(r) = input { + params["inputs"] = json!(r); + } + if let Some(o) = output { + params["output"] = json!(o); + } + for key in ["approve", "onConflict", "overwrite", "timeLimit"] { + if let Some(v) = p.get(key) { + params[key] = v.clone(); + } + } + // The macro runner gates every step with the document's trust. + crate::macros::run::run(s, &mac, ¶ms) + } + Some(ControlAction::Command { id, params }) => { + let d = s.doc()?; + let (doc, trust) = (d.uid, d.trust); + let policy = button_policy(&id, ¶ms, trust); + let file_command = matches!(policy, Policy::NeedsPermission(_)); + let why = match policy { + Policy::Allow => None, + Policy::Ask(why) | Policy::NeedsPermission(why) => Some(why), + Policy::Deny(why) => return Err(EngineError::Other(format!("'{name}' can't run {id}: {why}."))), + }; + if let Some(why) = why { + let key = (doc, approval_key(&id, ¶ms)); + if !s.macro_approvals.contains(&key) { + if bool_param(p, "approve") != Some(true) { + return Err(EngineError::Approve(format!("Run {id} from '{name}'? It {why}{}.", path_note(¶ms)))); + } + s.macro_approvals.insert(key); + } + } + // As a macro step: only an approved file command reaches the file system, so a + // command that reads or writes a file under another parameter name can't. + let _sandbox = (!file_command).then(crate::io::ScriptSandbox::enter); + s.grouped(&name, |s| { + let queued = s.ui_requests.len(); + let r = s.execute(&id, params.clone()); + // As a macro step: a command that would open a dialog or a web page is refused + // and its group rolled back; messages for the person stay queued. + let blocked = take_requests(s, queued, &name); + let r = r?; + match blocked { + Some(why) => Err(EngineError::Other(why)), + None => Ok(r), + } + }) + } + } +} + +/// The file a button's command names in its `path`, as ` (path)` for its approval question, so +/// the person sees what it reads or writes before saying yes. Empty without a `path`; a long one +/// is shortened in the middle. +fn path_note(params: &Json) -> String { + let text = match params.get("path") { + None | Some(Json::Null) => return String::new(), + Some(Json::String(t)) => t.clone(), + Some(other) => other.to_string(), + }; + let text: String = text.chars().map(|c| if c.is_control() { ' ' } else { c }).collect(); + let n = text.chars().count(); + let shown = if n > 200 { + let head: String = text.chars().take(100).collect(); + let tail: String = text.chars().skip(n - 99).collect(); + format!("{head}…{tail}") + } else { + text + }; + format!(" ({shown})") +} + +/// The approval key of a button's command: its id and parameters, with a prefix that keeps it +/// apart from a macro's fingerprint (canonical JSON, which starts with `{`). +fn approval_key(id: &str, params: &Json) -> String { + format!("control-command\u{0}{id}\u{0}{params}") +} + +/// After an edit: each slider and spin button whose linked cell holds a number takes that number +/// (clamped and snapped), so editing the cell moves the control. Sheets without such a control +/// are left untouched. +pub(crate) fn sync_linked_values(wb: &mut Workbook) { + for si in 0..wb.sheets.len() { + let updates: Vec<(usize, u32)> = match wb.sheet(si) { + Some(sh) if sh.controls.iter().any(|c| c.link.is_some()) => sh + .controls + .iter() + .enumerate() + .filter(|(_, c)| c.kind != ControlKind::Button) + .filter_map(|(i, c)| linked_number(wb, si, c).map(|n| (i, c.snap(n))).filter(|(_, v)| *v != c.value)) + .collect(), + _ => continue, + }; + if updates.is_empty() { + continue; + } + if let Some(sh) = wb.sheet_mut(si) { + for (i, v) in updates { + if let Some(c) = sh.controls.get_mut(i) { + c.value = v; + } + } + } + } +} + +// ---------------------------------------------------------------- xlsx + +/// The hidden-data key of the button actions xlsx has no place for. +pub(crate) const ACTIONS_KEY: &str = "control_actions"; + +/// A button's action that xlsx carries itself: a macro on the selection (`macro="[0]!Name"`). +fn plain(a: &ControlAction) -> bool { + matches!(a, ControlAction::Macro { input: MacroInput::Selection, output: None, .. }) +} + +/// A copy of `wb` for xlsx with the button actions xlsx can't express (commands, fixed inputs, +/// output cells) in a hidden name, keyed by sheet name and the control's place on the sheet, with +/// the button's name and anchor cell to check it is still the same button when the file comes +/// back. `None` when there is nothing to store. +pub(crate) fn with_action_names(wb: &Workbook) -> Result> { + let mut items = Vec::new(); + for sh in &wb.sheets { + for (i, c) in sh.controls.iter().enumerate() { + if let Some(a) = c.action.as_ref().filter(|a| !plain(a)) { + items.push(json!({"sheet": sh.name, "index": i, "name": c.display_name(), "at": c.anchor.cell.a1(), "action": a})); + } + } + } + let stale = stash::get(wb, ACTIONS_KEY).is_some(); + if items.is_empty() && !stale { + return Ok(None); + } + let mut out = wb.clone(); + stash::remove(&mut out, ACTIONS_KEY); + if !items.is_empty() { + stash::set_json(&mut out, ACTIONS_KEY, &Json::Array(items))?; + } + Ok(Some(out)) +} + +/// After a file is opened: gives buttons back the actions kept in the hidden name, and removes +/// it. Entries that don't parse, or don't match a button, are dropped: an entry must carry the +/// button's name and anchor cell, the button at the stored place must have that name and anchor +/// cell (another application may have added, removed or reordered controls), and it must not +/// have been given another macro since (a stored macro action comes back only on a button that +/// still runs that macro, a stored command only on one the file gives no macro). A stored fixed +/// input for a macro that takes no inputs becomes the selection (see [`takes_no_inputs`]). +pub(crate) fn load_action_names(wb: &mut Workbook) { + let Some(items) = stash::get_json(wb, ACTIONS_KEY) else { return }; + stash::remove(wb, ACTIONS_KEY); + for item in items.as_array().into_iter().flatten().take(100_000) { + let (Some(sheet), Some(index)) = (item.get("sheet").and_then(Json::as_str), item.get("index").and_then(Json::as_u64)) else { continue }; + let Ok(action) = serde_json::from_value::(item.get("action").cloned().unwrap_or(Json::Null)) else { continue }; + let valid = match &action { + ControlAction::Macro { name, .. } => !name.trim().is_empty() && name.chars().count() <= MAX_MACRO_NAME, + ControlAction::Command { id, params } => !id.is_empty() && id.len() <= 200 && params.is_object(), + }; + let Some(si) = wb.sheet_index(sheet) else { continue }; + let Ok(index) = usize::try_from(index) else { continue }; + let text = |k: &str| item.get(k).and_then(Json::as_str); + let same = |c: &FormControl| { + c.kind == ControlKind::Button + && text("name").is_some_and(|n| n == c.display_name()) + && text("at").is_some_and(|a| a == c.anchor.cell.a1()) + && match (&c.action, &action) { + (None, ControlAction::Command { .. }) => true, + (Some(ControlAction::Macro { name: now, .. }), ControlAction::Macro { name: kept, .. }) => now.eq_ignore_ascii_case(kept), + _ => false, + } + }; + if !valid || !wb.sheet(si).is_some_and(|sh| sh.controls.get(index).is_some_and(same)) { + continue; + } + let action = match action { + // The macro takes no inputs now: the stored range would fail every press, so the + // button runs the macro on the selection, as the file itself assigns it. + ControlAction::Macro { name, input: MacroInput::Fixed(_), output } if takes_no_inputs(wb, &name).is_some() => { + ControlAction::Macro { name, input: MacroInput::Selection, output } + } + other => other, + }; + if let Some(c) = wb.sheet_mut(si).and_then(|sh| sh.controls.get_mut(index)) { + c.action = Some(action); + } + } +} diff --git a/crates/engine/src/cmd/edit/paste_special.rs b/crates/engine/src/cmd/edit/paste_special.rs index a19bceb67..352c8e201 100644 --- a/crates/engine/src/cmd/edit/paste_special.rs +++ b/crates/engine/src/cmd/edit/paste_special.rs @@ -396,14 +396,15 @@ pub(crate) fn rewrite_all_formulas(wb: &mut gridcraft_model::Workbook, target: & } /// Runs `f` over every reference a floating object holds (a chart's series values, categories, -/// name and bubble sizes, and the source block it was made from). Every edit that moves +/// name and bubble sizes, and the source block it was made from, and a form control's linked +/// cell). Every edit that moves /// references calls it: inserting and deleting lines and cells and cut-moves (through /// [`rewrite_all_formulas`]), and renaming and deleting sheets. `f` maps a reference held by an /// object on the sheet named `host` to its new form; text that does not parse is kept. A source /// block that no longer exists is dropped. pub(crate) fn rewrite_cell_links(wb: &mut gridcraft_model::Workbook, f: impl Fn(&str, gridcraft_formula::Expr) -> gridcraft_formula::Expr) { for si in 0..wb.sheets.len() { - let Some(host) = wb.sheets.get(si).filter(|s| !s.charts.is_empty()).map(|s| s.name.clone()) else { continue }; + let Some(host) = wb.sheets.get(si).filter(|s| !s.charts.is_empty() || !s.controls.is_empty()).map(|s| s.name.clone()) else { continue }; let Some(sh) = wb.sheet_mut(si) else { continue }; let fix = |text: &str| -> String { match gridcraft_formula::parse(text) { @@ -420,6 +421,9 @@ pub(crate) fn rewrite_cell_links(wb: &mut gridcraft_model::Workbook, f: impl Fn( se.bubble_sizes = se.bubble_sizes.as_deref().map(fix); } } + for c in sh.controls.iter_mut() { + c.link = c.link.as_deref().map(fix); + } } } diff --git a/crates/engine/src/cmd/extra.rs b/crates/engine/src/cmd/extra.rs index a6f66b41b..6325de45a 100644 --- a/crates/engine/src/cmd/extra.rs +++ b/crates/engine/src/cmd/extra.rs @@ -97,7 +97,7 @@ pub fn specs() -> Vec { "Bring Forward", ["Page Layout", "Arrange"], None, - "{kind: chart|image|shape|slicer|equation, id, toFront?: bool}", + "{kind: chart|image|shape|slicer|equation|control, id, toFront?: bool}", has_doc, |s, p| arrange(s, p, true) ), @@ -106,7 +106,7 @@ pub fn specs() -> Vec { "Send Backward", ["Page Layout", "Arrange"], None, - "{kind: chart|image|shape|slicer|equation, id, toBack?: bool}", + "{kind: chart|image|shape|slicer|equation|control, id, toBack?: bool}", has_doc, |s, p| arrange(s, p, false) ), @@ -116,7 +116,7 @@ pub fn specs() -> Vec { "Align", ["Page Layout", "Arrange"], None, - "{kind: chart|image|shape|slicer|equation, ids: [..], align: left|center|right|top|middle|bottom}", + "{kind: chart|image|shape|slicer|equation|control, ids: [..], align: left|center|right|top|middle|bottom}", has_doc, align ), diff --git a/crates/engine/src/cmd/extra/objects.rs b/crates/engine/src/cmd/extra/objects.rs index fa5a5f438..112b530ac 100644 --- a/crates/engine/src/cmd/extra/objects.rs +++ b/crates/engine/src/cmd/extra/objects.rs @@ -57,6 +57,7 @@ pub(super) fn arrange(s: &mut Session, p: &Json, forward: bool) -> Result let kind = kind_param(p, "arrange")?; let extreme = bool_param(p, "toFront").or(bool_param(p, "toBack")).unwrap_or(false); let si = s.doc()?.wb.active_sheet(); + crate::cmd::controls::check_objects_unlocked(s, si, kind, &[id])?; fn reorder(v: &mut Vec, i: usize, forward: bool, extreme: bool) { if i >= v.len() { return; @@ -81,6 +82,7 @@ pub(super) fn arrange(s: &mut Session, p: &Json, forward: bool) -> Result ObjKind::Shape | ObjKind::Equation => { sh.shapes.iter().position(|c| c.id == id && shape_is(kind, c)).map(|i| reorder(&mut sh.shapes, i, forward, extreme)) } + ObjKind::Control => sh.controls.iter().position(|c| c.id == id).map(|i| reorder(&mut sh.controls, i, forward, extreme)), }; found.ok_or_else(|| bad("arrange", "no such object"))?; Ok(Json::Null) @@ -106,6 +108,12 @@ pub(super) fn selection_pane(s: &mut Session, _: &Json) -> Result { let name = if c.caption.is_empty() { &c.column } else { &c.caption }; v.push(json!({"kind": "slicer", "id": c.id, "name": name, "at": c.anchor.cell.a1(), "anchorMode": anchor_mode_name(c.anchor.mode)})); } + // Form controls paint above everything else. + for c in &sh.controls { + v.push( + json!({"kind": "control", "id": c.id, "name": c.display_name(), "at": c.anchor.cell.a1(), "anchorMode": anchor_mode_name(c.anchor.mode)}), + ); + } Ok(Json::Array(v)) } @@ -115,6 +123,7 @@ fn anchors_of(sh: &mut Sheet, kind: ObjKind) -> Vec<(u32, &mut Anchor)> { ObjKind::Chart => sh.charts.iter_mut().map(|c| (c.id, &mut c.anchor)).collect(), ObjKind::Image => sh.images.iter_mut().map(|c| (c.id, &mut c.anchor)).collect(), ObjKind::Slicer => sh.slicers.iter_mut().map(|c| (c.id, &mut c.anchor)).collect(), + ObjKind::Control => sh.controls.iter_mut().map(|c| (c.id, &mut c.anchor)).collect(), ObjKind::Shape | ObjKind::Equation => sh.shapes.iter_mut().filter(|c| shape_is(kind, c)).map(|c| (c.id, &mut c.anchor)).collect(), } } @@ -133,6 +142,7 @@ pub(super) fn align(s: &mut Session, p: &Json) -> Result { }; let how = str_param(p, "align").unwrap_or("left").to_string(); let si = s.doc()?.wb.active_sheet(); + crate::cmd::controls::check_objects_unlocked(s, si, kind, &ids)?; edit(s, |cx| { let sheet = cx.sheet_mut(si)?; let geo: Vec<(u32, f64, f64, f32, f32)> = { diff --git a/crates/engine/src/cmd/insert.rs b/crates/engine/src/cmd/insert.rs index 648350d81..68fb46505 100644 --- a/crates/engine/src/cmd/insert.rs +++ b/crates/engine/src/cmd/insert.rs @@ -146,13 +146,13 @@ pub fn specs() -> Vec { &with(p, "kind", json!("textBox")) )), cmd!("shape.setText", "Edit Text Box", [], None, "{id: textBox id, text: string}", has_doc, shape_set_text), - cmd!("object.delete", "Delete Object", [], None, "{kind: chart|image|shape|slicer|equation, id}", has_doc, delete_object), + cmd!("object.delete", "Delete Object", [], None, "{kind: chart|image|shape|slicer|equation|control, id}", has_doc, delete_object), cmd!( "object.move", "Move Object", [], None, - "{kind: chart|image|shape|slicer|equation, id, at?: \"C3\", dx?, dy?, width?, height? (each at most 10,000 points, or the object's current size when a file made it larger)}", + "{kind: chart|image|shape|slicer|equation|control, id, at?: \"C3\", dx?, dy?, width?, height? (each at most 10,000 points, or the object's current size when a file made it larger)}", has_doc, move_object ), @@ -161,7 +161,7 @@ pub fn specs() -> Vec { "Object Properties", [], None, - "{kind: chart|image|shape|slicer|equation, id, mode: moveAndSize|moveOnly|absolute}", + "{kind: chart|image|shape|slicer|equation|control, id, mode: moveAndSize|moveOnly|absolute}", has_doc, set_anchor_mode ), @@ -1235,13 +1235,16 @@ fn shape_set_text(s: &mut Session, p: &Json) -> Result { /// objects are errors, so a stale id never succeeds silently. fn object_param(s: &Session, p: &Json, cmd: &str) -> Result<(usize, ObjRef)> { let id = u32_param(p, "id").ok_or_else(|| bad(cmd, "missing `id`"))?; - let kind = str_param(p, "kind").and_then(ObjKind::parse).ok_or_else(|| bad(cmd, "kind must be chart, image, shape, slicer or equation"))?; + let kind = + str_param(p, "kind").and_then(ObjKind::parse).ok_or_else(|| bad(cmd, "kind must be chart, image, shape, slicer, equation or control"))?; let d = s.doc()?; let sheet = d.wb.active_sheet(); let o = ObjRef::new(kind, id); if !d.wb.sheet(sheet).is_some_and(|sh| sh.has_object(o)) { return Err(bad(cmd, "no such object")); } + // A locked form control on a protected sheet stays as it is. + super::controls::check_objects_unlocked(s, sheet, kind, &[id])?; Ok((sheet, o)) } diff --git a/crates/engine/src/cmd/macros.rs b/crates/engine/src/cmd/macros.rs index 396e0e28f..37f400f05 100644 --- a/crates/engine/src/cmd/macros.rs +++ b/crates/engine/src/cmd/macros.rs @@ -28,7 +28,7 @@ pub fn specs() -> Vec { "Run Macro", [], None, - "{name, inputs?: {Input: \"B2:D9\"} | \"B2:D9\", onConflict?: ask|fail|overwrite|shiftDown|shiftRight, overwrite?: bool, approve?: bool, timeLimit?: seconds} runs as one undo step; any error rolls it all back → {macro, ran, inputs, stopped?, shifted?}", + "{name, inputs?: {Input: \"B2:D9\"} | \"B2:D9\", output?: \"F2\" (instead of the macro's output anchor), onConflict?: ask|fail|overwrite|shiftDown|shiftRight, overwrite?: bool, approve?: bool, timeLimit?: seconds} runs as one undo step; any error rolls it all back → {macro, ran, inputs, stopped?, shifted?}", has_doc, run ), @@ -37,7 +37,7 @@ pub fn specs() -> Vec { "Check Macro", [], None, - "{name, inputs?, onConflict?, overwrite?, timeLimit?: seconds} validates the inputs and dry-runs the macro on the workbook (then puts everything back) → {ok (macro.run with these parameters would go ahead without a question), errors?, needsApproval?, onConflict, inputs, output?, conflicts, footprint}", + "{name, inputs?, output?: \"F2\" (as macro.run), onConflict?, overwrite?, timeLimit?: seconds} validates the inputs and dry-runs the macro on the workbook (then puts everything back) → {ok (macro.run with these parameters would go ahead without a question), errors?, needsApproval?, onConflict, inputs, output?, conflicts, footprint}", has_doc, check ), @@ -55,7 +55,7 @@ pub fn specs() -> Vec { "Step Into", [], None, - "{action: start|step|continue|runTo|stop|state, name, inputs?, onConflict?, overwrite?, approve?, timeLimit?: seconds per action (start, as macro.run), step: path (runTo), keep?: bool (stop), breakpoints?: [\"3\", \"2.1\"] (steps of the macro), breakpointsOnly?: bool (the list replaces the steps' own breakpoints)} debugs a macro one step at a time; stop keeps the changes as one undo step or rolls them back. Runs on its own, never as a step of a script or macro. Until it finishes or stops, its workbook runs only macro.debug, queries, selection and sheet changes, Go To, Find, Copy, zoom, Switch Window, Close and Stop Action (other workbooks stay editable) → {state: paused|finished|stopped|failed, current: {step, kind, command?, params?, iteration?}, lastParams, executed, changed (at most 500), changedCells, atBreakpoint?, shifted?, wroteFiles?, message?, error?}", + "{action: start|step|continue|runTo|stop|state, name, inputs?, output?, overwrite?, approve?, timeLimit?: seconds per action (start, as macro.run), step: path (runTo), keep?: bool (stop), breakpoints?: [\"3\", \"2.1\"] (steps of the macro), breakpointsOnly?: bool (the list replaces the steps' own breakpoints)} debugs a macro one step at a time; stop keeps the changes as one undo step or rolls them back. Runs on its own, never as a step of a script or macro. Until it finishes or stops, its workbook runs only macro.debug, queries, selection and sheet changes, Go To, Find, Copy, zoom, Switch Window, Close and Stop Action (other workbooks stay editable) → {state: paused|finished|stopped|failed, current: {step, kind, command?, params?, iteration?}, lastParams, executed, changed (at most 500), changedCells, atBreakpoint?, shifted?, wroteFiles?, message?, error?}", has_doc, |s, p| crate::macros::debug::debug(s, p) ), @@ -170,7 +170,21 @@ fn rename(s: &mut Session, p: &Json) -> Result { renamed.name = to.clone(); // The whole check, size included: a longer name can take a macro past what a file holds. crate::macros::check(&renamed).map_err(EngineError::Other)?; - *m = renamed; + let from = std::mem::replace(m, renamed).name; + // Buttons that run the macro follow the new name. + let runs = |c: &gridcraft_model::FormControl| matches!(&c.action, Some(gridcraft_model::ControlAction::Macro { name, .. }) if name.eq_ignore_ascii_case(&from)); + for si in 0..cx.wb.sheets.len() { + if !cx.wb.sheet(si).is_some_and(|sh| sh.controls.iter().any(runs)) { + continue; + } + for c in cx.sheet_mut(si)?.controls.iter_mut() { + if let Some(gridcraft_model::ControlAction::Macro { name, .. }) = &mut c.action + && name.eq_ignore_ascii_case(&from) + { + *name = to.clone(); + } + } + } Ok(()) })?; Ok(json!({"name": to})) diff --git a/crates/engine/src/cmd/sheet/tabs.rs b/crates/engine/src/cmd/sheet/tabs.rs index 0c3925d22..d21851553 100644 --- a/crates/engine/src/cmd/sheet/tabs.rs +++ b/crates/engine/src/cmd/sheet/tabs.rs @@ -170,7 +170,7 @@ pub(super) fn move_sheet(s: &mut Session, p: &Json) -> Result { // Object ids are unique across the workbook (selection and the `{kind, id}` commands // rely on it), so the copied objects get fresh ones (free ones, even when a file used // the largest id). - let count = sh.charts.len() + sh.images.len() + sh.shapes.len() + sh.slicers.len(); + let count = sh.charts.len() + sh.images.len() + sh.shapes.len() + sh.slicers.len() + sh.controls.len(); let ids = cx.wb.free_object_ids(count); if ids.len() < count { return Err(bad("sheet.move", "no free object ids are left for the copied objects")); @@ -181,6 +181,7 @@ pub(super) fn move_sheet(s: &mut Session, p: &Json) -> Result { sh.images.iter_mut().for_each(|o| o.id = fresh()); sh.shapes.iter_mut().for_each(|o| o.id = fresh()); sh.slicers.iter_mut().for_each(|o| o.id = fresh()); + sh.controls.iter_mut().for_each(|o| o.id = fresh()); let at = to.min(n); // The copy gets its own copies of the sheet's names, referring to the copy. let copies: Vec = cx diff --git a/crates/engine/src/io.rs b/crates/engine/src/io.rs index 602c17ccd..6cd39f839 100644 --- a/crates/engine/src/io.rs +++ b/crates/engine/src/io.rs @@ -123,12 +123,13 @@ fn decrypt_message(name: &str, e: gridcraft_xlsx::DecryptError) -> String { } } -/// Parses bytes that aren't encrypted, then takes the workbook's macros out of their hidden names -/// and adopts the Python cells they hold (see [`adopt_python`]) within the limits of -/// [`enforce_python_limits`]. +/// Parses bytes that aren't encrypted, then takes the workbook's macros, and the button actions +/// xlsx has no place for, out of their hidden names, and adopts the Python cells they hold (see +/// [`adopt_python`]) within the limits of [`enforce_python_limits`]. fn open_plain(name: &str, bytes: &[u8], csv_as_text: bool) -> Result<(Workbook, Vec)> { let (mut wb, mut warnings) = read_plain(name, bytes, csv_as_text)?; warnings.extend(crate::macros::load(&mut wb)); + crate::cmd::controls::load_action_names(&mut wb); warnings.extend(adopt_python(&mut wb)); warnings.extend(enforce_python_limits(&mut wb)); Ok((wb, warnings)) @@ -493,9 +494,13 @@ pub fn save_bytes_as(wb: &Workbook, path: &str, kind: FileKind) -> Result", + if move_cells { " moveWithCells=\"1\"" } else { "" }, + if size_cells { " sizeWithCells=\"1\"" } else { "" }, + marker("from", from, fdx, fdy), + marker("to", to, tdx, tdy), + ); + vml.push_str(&vml_shape(sheet, c, id, i)); + } + Written { entries: xml, vml } +} + +/// The control properties part. +fn ctrl_prop(c: &FormControl) -> String { + let mut s = format!("{} { + s.push_str(" lockText=\"1\""); + } + ControlKind::Slider | ControlKind::Spin => { + s.push_str(" dx=\"22\""); + if let Some(l) = &c.link { + let _ = write!(s, " fmlaLink=\"{}\"", esc_attr(l)); + } + if c.kind == ControlKind::Slider && c.horizontal { + s.push_str(" horiz=\"1\""); + } + let _ = write!(s, " inc=\"{}\" max=\"{}\" min=\"{}\"", c.step, c.max, c.min); + if c.kind == ControlKind::Slider { + let _ = write!(s, " page=\"{}\"", c.page); + } + let _ = write!(s, " val=\"{}\"", c.value); + } + } + s.push_str("/>"); + s +} + +/// A control's legacy VML shape. +fn vml_shape(sheet: &Sheet, c: &FormControl, id: u64, z: usize) -> String { + let ((from, fdx, fdy), (to, tdx, tdy)) = corners(sheet, &c.anchor); + let pt = |px: f64| px * 0.75; + let x = sheet.col_left(from.col) + fdx; + let y = sheet.row_top(from.row) + fdy; + let mut data = format!( + "{}{}{}, {}, {}, {}, {}, {}, {}, {}", + object_type(c.kind), + // These two elements mean the object does *not* move (size) with its cells. + if c.anchor.mode == AnchorMode::Absolute { "" } else { "" }, + if c.anchor.mode == AnchorMode::MoveAndSize { "" } else { "" }, + from.col, + fdx.round() as i64, + from.row, + fdy.round() as i64, + to.col, + tdx.round() as i64, + to.row, + tdy.round() as i64 + ); + if !c.print { + data.push_str("False"); + } + if !c.locked { + data.push_str("False"); + } + data.push_str("False"); + match c.kind { + ControlKind::Button => { + if let Some(m) = macro_ref(c) { + let _ = write!(data, "{}", esc(&m)); + } + data.push_str("CenterCenter"); + } + ControlKind::Slider | ControlKind::Spin => { + if let Some(l) = &c.link { + let _ = write!(data, "{}", esc(l)); + } + let _ = write!(data, "{}{}{}{}", c.value, c.min, c.max, c.step); + if c.kind == ControlKind::Slider { + let _ = write!(data, "{}", c.page); + if c.horizontal { + data.push_str(""); + } + } + data.push_str("22"); + } + } + data.push_str(""); + let text = if c.kind == ControlKind::Button { + format!("
{}
", esc(&c.text)) + } else { + String::new() + }; + format!( + "{text}{data}", + pt(x), + pt(y), + pt(c.anchor.width.max(0.0) as f64), + pt(c.anchor.height.max(0.0) as f64), + z + 1, + if c.kind == ControlKind::Button { " o:button=\"t\"" } else { " stroked=\"f\" filled=\"f\"" }, + ) +} + +/// The VML shape type form controls use. +pub const SHAPETYPE: &str = ""; + +// ---------------------------------------------------------------- reading + +/// What a control's properties say, from `formControlPr` or VML `x:ClientData`. +#[derive(Clone, Default)] +struct Props { + kind: Option, + link: Option, + min: Option, + max: Option, + inc: Option, + page: Option, + val: Option, + horiz: Option, + macro_: Option, + text: String, + print: Option, + locked: Option, +} + +fn props_from_ctrl_prop(e: &El) -> Props { + let int = |k: &str| e.attr_f64(k).map(|v| v.round() as i64); + Props { + kind: e.attr("objectType").and_then(kind_of), + link: e.attr("fmlaLink").map(str::to_string), + min: int("min"), + max: int("max"), + inc: int("inc"), + page: int("page"), + val: int("val"), + horiz: e.attr_bool("horiz"), + macro_: None, + text: String::new(), + print: None, + locked: None, + } +} + +fn props_from_vml(shape: &El) -> Option { + let data = shape.child("ClientData")?; + let kind = data.attr("ObjectType").and_then(kind_of)?; + let text_of = |n: &str| data.child(n).map(|c| c.text.trim().to_string()); + let int = |n: &str| text_of(n).and_then(|t| t.parse::().ok()).filter(|v| v.is_finite()).map(|v| v.round() as i64); + let truth = |n: &str| text_of(n).map(|t| !t.eq_ignore_ascii_case("false")); + let text = shape.child("textbox").map(|t| all_text(t, 0)).unwrap_or_default().trim().to_string(); + Some(Props { + kind: Some(kind), + link: text_of("FmlaLink").filter(|t| !t.is_empty()), + min: int("Min"), + max: int("Max"), + inc: int("Inc"), + page: int("Page"), + val: int("Val"), + horiz: data.child("Horiz").map(|_| true), + macro_: text_of("FmlaMacro"), + text, + print: truth("PrintObject"), + locked: truth("Locked"), + }) +} + +/// The text inside `e` and its descendants (a VML text box holds `
` and ``), up to a +/// caption's length; `
` is a line break. +fn all_text(e: &El, depth: usize) -> String { + let mut s = e.text.clone(); + if depth < 16 { + for k in &e.children { + if k.name == "br" { + s.push('\n'); + } + s.push_str(&all_text(k, depth + 1)); + if s.len() > 4_096 { + break; + } + } + } + s +} + +/// A control from what the file says, its numbers clamped to whole numbers 0..30000 that pass +/// the model's rules (a hostile or odd file can't give a control it couldn't have). +fn build(p: Props, kind: ControlKind, anchor: Anchor, name: Option<&str>, id: u32) -> FormControl { + let clamp = |v: Option, d: u32| v.map_or(d, |v| v.clamp(0, CONTROL_MAX as i64) as u32); + let mut c = FormControl { id, kind, anchor, ..FormControl::default() }; + c.min = clamp(p.min, 0); + c.max = clamp(p.max, 100); + if c.min >= c.max { + if c.min >= CONTROL_MAX { + c.min = CONTROL_MAX - 1; + } + c.max = c.min + 1; + } + c.step = clamp(p.inc, 1).clamp(1, c.max - c.min); + c.page = clamp(p.page, 10).max(c.step); + c.value = c.snap(clamp(p.val, c.min) as f64); + // Only sliders have an orientation; `horiz` is left out for a vertical one. + c.horizontal = kind != ControlKind::Slider || p.horiz.unwrap_or(false); + c.link = p.link.map(|l| l.trim().trim_start_matches('=').to_string()).filter(|l| !l.is_empty() && l.chars().count() <= 300); + c.text = p.text.chars().take(1_024).collect(); + if let Some(n) = name { + c.name = n.chars().take(255).collect(); + } + c.print = p.print.unwrap_or(true); + c.locked = p.locked.unwrap_or(true); + if kind == ControlKind::Button + && let Some(m) = p.macro_.as_deref().and_then(macro_name) + { + c.action = Some(ControlAction::Macro { name: m, input: MacroInput::Selection, output: None }); + } + c +} + +/// The `` elements one child of a `` element holds: the child itself, or those +/// inside its `mc:AlternateContent` (the Choice, else the Fallback). +fn controls_of_child(k: &El) -> Vec<&El> { + match k.name.as_str() { + "control" => vec![k], + "AlternateContent" => k + .child("Choice") + .filter(|c| c.child("control").is_some()) + .or_else(|| k.child("Fallback")) + .map(|inner| inner.kids("control").collect()) + .unwrap_or_default(), + _ => Vec::new(), + } +} + +/// The `` elements under a `` element, looking through `mc:AlternateContent` +/// (the Choice, else the Fallback). +fn controls_in(e: &El) -> Vec<&El> { + e.children.iter().flat_map(controls_of_child).collect() +} + +/// The `` element of a worksheet child (`controls`, or one inside AlternateContent). +pub fn controls_of(e: &El) -> Option<&El> { + match e.name.as_str() { + "controls" => Some(e), + "AlternateContent" => e.child("Choice").and_then(|c| c.child("controls")).or_else(|| e.child("Fallback").and_then(|c| c.child("controls"))), + _ => None, + } +} + +/// The shape number of a VML shape id (`_x0000_s1025` → 1025) or a `shapeId` attribute. +pub(crate) fn shape_number(id: &str) -> Option { + id.trim().trim_start_matches("_x0000_s").parse().ok() +} + +/// A VML shape's number: from its `id` (`_x0000_s1025`), else from `o:spid`, where producers that +/// give the shape a named id (`id="Button 1"`) keep it. +pub(crate) fn number_of(shape: &El) -> Option { + shape.attr("id").and_then(shape_number).or_else(|| shape.attr("spid").and_then(shape_number)) +} + +/// What [`read`] took from a sheet's form controls, for pass-through. +#[derive(Debug, Default)] +pub struct ReadControls { + /// The VML shape numbers of the controls read into the model: GridCraft writes those, so they + /// are left out of the kept content. + pub modelled: HashSet, + /// The VML drawing also holds a control that the sheet doesn't list and the model doesn't + /// hold (a check box only the drawing has): the drawing is kept for it even when no + /// `` element is left to keep. + pub vml_only_unmodelled: bool, +} + +/// Whether a VML shape the sheet doesn't list is a control that only pass-through can keep: it +/// has client data that is neither a note (GridCraft writes notes) nor an embedded object's +/// picture (`Pict`, which means nothing without its sheet entry). +fn unlisted_kept(shape: &El) -> bool { + object_type_of(shape).is_some_and(|t| !t.eq_ignore_ascii_case("note") && !t.eq_ignore_ascii_case("pict")) +} + +/// A VML shape's client data `ObjectType`, trimmed (`Note`, `Button`, `Checkbox`). +fn object_type_of(shape: &El) -> Option<&str> { + shape.child("ClientData").and_then(|d| d.attr("ObjectType")).map(str::trim) +} + +/// Whether a VML shape's client data is a note's: its `ObjectType` is `Note` in any case and with +/// spaces around it. Only that attribute counts, not text elsewhere in the shape (a caption). +pub(crate) fn is_note_data(data: &El) -> bool { + data.attr("ObjectType").is_some_and(|t| t.trim().eq_ignore_ascii_case("note")) +} + +/// Reads a sheet's buttons, scroll bars and spin buttons: each `` with its properties +/// part, else its VML shape, and VML shapes the sheet doesn't list (files that only have VML). +/// The shapes come from the VML drawing with relationship id `vml_rid` (``). +/// Returns the shape numbers of the controls it read, so pass-through keeps only the others. +pub fn read(cx: &mut Ctx<'_>, controls: Option<&El>, vml_rid: Option<&str>, rels: &[Rel], sheet: &mut Sheet) -> ReadControls { + let mut read = HashSet::new(); + let vml = match vml_rid.and_then(|id| rels.iter().find(|r| r.id == id && r.kind == "vmlDrawing")) { + Some(r) => cx.pkg.read_xml(&r.target).ok().flatten(), + None => None, + }; + let shapes: Vec<&El> = vml.as_ref().map(|v| v.kids("shape").collect()).unwrap_or_default(); + // Looked up once per listed control: indexes keep a sheet of many controls linear. + let mut by_number: HashMap = HashMap::with_capacity(shapes.len()); + for s in &shapes { + if let Some(n) = number_of(s) { + by_number.entry(n).or_insert(s); + } + } + let shape_of = |n: u64| by_number.get(&n).copied(); + // The relationships likewise: one per control, looked up by id (the first of a repeated id wins, + // as a scan would find it). + let mut rel_by_id: HashMap<&str, &Rel> = HashMap::with_capacity(rels.len()); + for r in rels { + rel_by_id.entry(r.id.as_str()).or_insert(r); + } + // Each properties part is read and parsed once, however many controls name it: every read is + // charged to the package's budget, so a file whose controls all point at one part would + // otherwise spend it on that part alone. A part that fails to read, or isn't a + // `formControlPr`, is remembered as nothing and not read again. + let mut props_by_part: HashMap<&str, Option> = HashMap::new(); + // Every shape the sheet lists, read here or not. + let mut listed: HashSet = HashSet::new(); + for ctl in controls.map(controls_in).unwrap_or_default() { + let Some(n) = ctl.attr("shapeId").and_then(shape_number) else { continue }; + listed.insert(n); + if sheet.controls.len() >= MAX_CONTROLS { + continue; + } + let shape = shape_of(n); + let vml_props = shape.and_then(props_from_vml); + let prop_part = ctl.attr("id").and_then(|id| rel_by_id.get(id)).map(|r| r.target.as_str()); + let file_props = match prop_part { + Some(t) => props_by_part + .entry(t) + .or_insert_with(|| cx.pkg.read_xml(t).ok().flatten().filter(|e| e.name == "formControlPr").map(|e| props_from_ctrl_prop(&e))) + .clone(), + None => None, + }; + let pr = ctl.child("controlPr"); + let mut p = match (file_props, vml_props) { + (Some(mut f), v) => { + // The properties part has no caption; the VML text box does. + if let Some(v) = v { + f.text = v.text; + f.print = f.print.or(v.print); + f.locked = f.locked.or(v.locked); + f.macro_ = v.macro_; + } + f + } + (None, Some(v)) => v, + (None, None) => continue, + }; + if let Some(pr) = pr { + if let Some(m) = pr.attr("macro").filter(|m| !m.trim().is_empty()) { + p.macro_ = Some(m.to_string()); + } + p.print = pr.attr_bool("print").or(p.print); + p.locked = pr.attr_bool("locked").or(p.locked); + } + // Other kinds (a check box, a list box) stay with pass-through. + let Some(kind) = p.kind else { continue }; + let anchor = + pr.and_then(|pr| pr.child("anchor")).and_then(|a| anchor_from_markers(a, sheet)).or_else(|| shape.and_then(|s| vml_anchor(s, sheet))); + let Some(anchor) = anchor else { continue }; + let id = cx.next_object_id(); + sheet.controls.push(build(p, kind, anchor, ctl.attr("name"), id)); + read.insert(n); + } + // Controls known only from the VML. The ones not read (a check box, or past the limit) stay + // in the kept drawing. + let mut vml_only_unmodelled = false; + for s in &shapes { + let n = number_of(s); + if n.is_some_and(|n| listed.contains(&n)) { + continue; + } + let modelled = n.filter(|_| sheet.controls.len() < MAX_CONTROLS).and_then(|n| { + let p = props_from_vml(s)?; + let (kind, anchor) = (p.kind?, vml_anchor(s, sheet)?); + let id = cx.next_object_id(); + sheet.controls.push(build(p, kind, anchor, None, id)); + read.insert(n); + Some(()) + }); + if modelled.is_none() && unlisted_kept(s) { + vml_only_unmodelled = true; + } + } + ReadControls { modelled: read, vml_only_unmodelled } +} + +fn anchor_from_markers(a: &El, sheet: &Sheet) -> Option { + let (cell, dx, dy) = crate::drawing::marker(a.child("from")?); + let (to, tdx, tdy) = a.child("to").map(crate::drawing::marker).unwrap_or((cell, dx + 64.0, dy + 20.0)); + let x1 = sheet.col_left(cell.col) + dx as f64; + let y1 = sheet.row_top(cell.row) + dy as f64; + let x2 = sheet.col_left(to.col) + tdx as f64; + let y2 = sheet.row_top(to.row) + tdy as f64; + let mode = match (a.flag("moveWithCells", false), a.flag("sizeWithCells", false)) { + (_, true) => AnchorMode::MoveAndSize, + (true, false) => AnchorMode::MoveOnly, + _ => AnchorMode::Absolute, + }; + Some(Anchor { cell, dx, dy, width: (x2 - x1).clamp(1.0, 10_000.0) as f32, height: (y2 - y1).clamp(1.0, 10_000.0) as f32, mode }) +} + +/// A VML `x:Anchor` (`LeftColumn, LeftOffset, TopRow, TopOffset, RightColumn, RightOffset, +/// BottomRow, BottomOffset`, offsets in pixels). +fn vml_anchor(shape: &El, sheet: &Sheet) -> Option { + let data = shape.child("ClientData")?; + let nums: Vec = data.child("Anchor")?.text.split(',').filter_map(|t| t.trim().parse::().ok()).collect(); + let [c1, dx1, r1, dy1, c2, dx2, r2, dy2] = nums.as_slice() else { return None }; + let col = |c: i64| c.clamp(0, gridcraft_core::MAX_COLS as i64 - 1) as u32; + let row = |r: i64| r.clamp(0, gridcraft_core::MAX_ROWS as i64 - 1) as u32; + let off = |v: i64| v.clamp(0, 10_000) as f64; + let cell = CellRef::new(row(*r1), col(*c1)); + let x1 = sheet.col_left(cell.col) + off(*dx1); + let y1 = sheet.row_top(cell.row) + off(*dy1); + let x2 = sheet.col_left(col(*c2)) + off(*dx2); + let y2 = sheet.row_top(row(*r2)) + off(*dy2); + // The elements are there when the object does *not* move (size) with its cells. + let mode = match (data.child("MoveWithCells").is_none(), data.child("SizeWithCells").is_none()) { + (_, true) => AnchorMode::MoveAndSize, + (true, false) => AnchorMode::MoveOnly, + _ => AnchorMode::Absolute, + }; + Some(Anchor { + cell, + dx: off(*dx1) as f32, + dy: off(*dy1) as f32, + width: (x2 - x1).clamp(1.0, 10_000.0) as f32, + height: (y2 - y1).clamp(1.0, 10_000.0) as f32, + mode, + }) +} diff --git a/crates/xlsx/src/drawing.rs b/crates/xlsx/src/drawing.rs index 288927283..b702331c8 100644 --- a/crates/xlsx/src/drawing.rs +++ b/crates/xlsx/src/drawing.rs @@ -24,7 +24,7 @@ fn emu_px(v: Option) -> f32 { v.map(|e| (e as f64 / EMU_PER_PX).clamp(-1e7, 1e7) as f32).unwrap_or(0.0) } -fn marker(e: &El) -> (CellRef, f32, f32) { +pub(crate) fn marker(e: &El) -> (CellRef, f32, f32) { let num = |n: &str| e.child(n).and_then(|c| c.text.trim().parse::().ok()); let col = num("col").unwrap_or(0).clamp(0, MAX_COLS as i64 - 1) as u32; let row = num("row").unwrap_or(0).clamp(0, MAX_ROWS as i64 - 1) as u32; diff --git a/crates/xlsx/src/lib.rs b/crates/xlsx/src/lib.rs index a6ede6af5..4d5038012 100644 --- a/crates/xlsx/src/lib.rs +++ b/crates/xlsx/src/lib.rs @@ -15,6 +15,7 @@ mod array_formula; mod cfb; mod chart; mod chartex; +mod controls; mod crypto; mod csv; mod drawing; diff --git a/crates/xlsx/src/package.rs b/crates/xlsx/src/package.rs index 9b4160dfc..002a2bced 100644 --- a/crates/xlsx/src/package.rs +++ b/crates/xlsx/src/package.rs @@ -52,6 +52,14 @@ impl<'a> Package<'a> { Ok(Package { zip, names, total: 0 }) } + /// Every part's (normalized) name, sorted. + #[cfg(test)] + pub fn names(&self) -> Vec { + let mut v: Vec = self.names.keys().cloned().collect(); + v.sort(); + v + } + pub fn has(&self, part: &str) -> bool { self.names.contains_key(&normalize(part)) } diff --git a/crates/xlsx/src/passthrough.rs b/crates/xlsx/src/passthrough.rs index 3e2fea105..9d85e03b2 100644 --- a/crates/xlsx/src/passthrough.rs +++ b/crates/xlsx/src/passthrough.rs @@ -4,8 +4,8 @@ //! relationship graph, per owner: the package root (custom document properties, ribbon //! customisation, add-in task panes), the workbook (custom XML, links to external workbooks with //! their ``, data connections, XML maps) and each sheet (query tables, cells -//! mapped to XML elements, form controls, ActiveX controls, OLE objects, the sheet background and -//! printer settings, with the sheet's ``, `` and `` XML, the `r:id` +//! mapped to XML elements, form controls other than the buttons, scroll bars and spin buttons +//! `controls.rs` models, ActiveX controls, OLE objects, the sheet background and printer settings, with the sheet's ``, `` and `` XML, the `r:id` //! of ``, the legacy VML shapes those controls need and the pictures of the headers and //! footers, `&G`, with their ``). The workbook also keeps its //! `` and the calcFeatures extension, which Excel needs to trust the saved values. diff --git a/crates/xlsx/src/passthrough/read.rs b/crates/xlsx/src/passthrough/read.rs index 6be20d76d..dde7aac5f 100644 --- a/crates/xlsx/src/passthrough/read.rs +++ b/crates/xlsx/src/passthrough/read.rs @@ -283,8 +283,10 @@ impl KeptReader { (!p.is_empty()).then(|| Arc::new(p)) } - /// Sheet-level content of the sheet part `part`. - pub fn sheet(&mut self, cx: &mut Ctx<'_>, part: &str, name: &str) -> Option> { + /// Sheet-level content of the sheet part `part`. `controls` says which form controls the + /// sheet reader modelled (their VML shapes are left out of the kept content) and whether the + /// VML drawing holds others that only it lists. + pub fn sheet(&mut self, cx: &mut Ctx<'_>, part: &str, name: &str, controls: &crate::controls::ReadControls) -> Option> { if !self.on { return None; } @@ -305,13 +307,24 @@ impl KeptReader { // Notes have one VML drawing per sheet; another one holds the header and footer pictures. let vml_drawings = rels.iter().filter(|r| is_kind(r, &["vmlDrawing"])).count(); let notes = rels.iter().filter(|r| is_kind(r, &["comments"])).count(); - if rels.iter().any(|r| is_kind(r, SHEET_ELEMENT_KINDS)) || vml_drawings > notes { - self.sheet_elements(cx, part, name, &rels, &mut p, &mut seen); + if rels.iter().any(|r| is_kind(r, SHEET_ELEMENT_KINDS)) || vml_drawings > notes || controls.vml_only_unmodelled { + self.sheet_elements(cx, part, name, &rels, &mut p, &mut seen, controls); } (!p.is_empty()).then(|| Arc::new(p)) } - fn sheet_elements(&mut self, cx: &mut Ctx<'_>, part: &str, name: &str, rels: &[PreservedRel], p: &mut Preserved, seen: &mut HashSet) { + #[allow(clippy::too_many_arguments)] + fn sheet_elements( + &mut self, + cx: &mut Ctx<'_>, + part: &str, + name: &str, + rels: &[PreservedRel], + p: &mut Preserved, + seen: &mut HashSet, + controls: &crate::controls::ReadControls, + ) { + let read_controls = &controls.modelled; let scan = match cx.pkg.read(part) { Ok(Some(bytes)) => scan_sheet(&bytes), _ => None, @@ -344,10 +357,16 @@ impl KeptReader { Err(why) => cx.warn(format!("sheet \"{name}\": header and footer pictures were dropped: {why}")), } } - let mut frags = scan.fragments; - // Controls and OLE objects are drawn from shapes in the legacy VML drawing. - if frags.iter().any(|(slot, _)| slot != "picture") { - match self.vml(cx, rels, scan.legacy_drawing.as_deref(), seen) { + // Form controls the sheet reader modelled are written from the model; the others stay. + let mut frags: Vec<(String, String)> = scan + .fragments + .into_iter() + .filter_map(|(slot, x)| if slot == "controls" { without_controls(&x, read_controls).map(|x| (slot, x)) } else { Some((slot, x)) }) + .collect(); + // Controls and OLE objects are drawn from shapes in the legacy VML drawing. Controls only + // the drawing lists need it too, whether or not a `` element is left. + if frags.iter().any(|(slot, _)| slot != "picture") || controls.vml_only_unmodelled { + match self.vml(cx, rels, scan.legacy_drawing.as_deref(), seen, read_controls) { Ok((vml, parts)) => { p.parts.extend(parts); p.vml = Some(vml); @@ -376,21 +395,22 @@ impl KeptReader { Err(why) => cx.warn(format!("sheet \"{name}\": {slot} were dropped: {why}")), } } - if p.elements.iter().any(|e| e.slot == "oleObjects" || e.slot == "controls") { + if p.elements.iter().any(|e| e.slot == "oleObjects" || e.slot == "controls") || (controls.vml_only_unmodelled && p.vml.is_some()) { cx.warn(format!("sheet \"{name}\": embedded objects and form controls are not shown; they are kept for saving")); } else { p.vml = None; } } - /// The sheet's VML drawing without its notes (GridCraft writes those), and the parts its - /// shapes refer to (preview images). + /// The sheet's VML drawing without its notes and the form controls read into the model + /// (GridCraft writes those), and the parts its shapes refer to (preview images). fn vml( &mut self, cx: &mut Ctx<'_>, rels: &[PreservedRel], rid: Option<&str>, seen: &mut HashSet, + read_controls: &HashSet, ) -> Result<(PreservedPart, Vec), String> { let rel = rid.and_then(|id| rels.iter().find(|r| r.id == id && !r.external)).ok_or("the sheet has no VML drawing")?; // Charged to the kept total like any other part: many sheets can point at one highly @@ -407,7 +427,7 @@ impl KeptReader { }; self.budget = self.budget.saturating_sub(data.len()); let text = String::from_utf8(data).map_err(|_| "the VML drawing is not UTF-8".to_string())?; - let text = strip_notes(&text); + let text = strip_shapes(&text, read_controls); let vrels = raw_rels(cx, &rel.target)?; let mut q = Preserved::default(); let roots: Vec = vrels.iter().filter(|r| !r.external).cloned().collect(); @@ -557,13 +577,20 @@ fn shape_spans(vml: &str) -> Vec<(usize, usize)> { spans } -/// VML without its note shapes (`ObjectType="Note"`). -fn strip_notes(vml: &str) -> String { +/// VML without its note shapes and the shapes numbered in `drop`, each judged as the control +/// reader judges it, from the shape's own tags and never from text elsewhere in it: a note by its +/// client data's `ObjectType` ([`crate::controls::is_note_data`]), its number from its `id`, else +/// its `o:spid` ([`crate::controls::number_of`]). Each tag is read on its own ([`xml::first_start`]), +/// so a caption's unclosed `
` doesn't hide the client data after it. A shape with no number +/// that isn't a note is kept. +fn strip_shapes(vml: &str, drop: &HashSet) -> String { let mut out = String::with_capacity(vml.len()); let mut last = 0; for (a, b) in shape_spans(vml) { - let shape = vml.get(a..b).unwrap_or(""); - if shape.contains("ObjectType=\"Note\"") { + let shape = vml.get(a..b).unwrap_or("").as_bytes(); + let note = xml::first_start(shape, "ClientData").is_some_and(|d| crate::controls::is_note_data(&d)); + let numbered = xml::first_start(shape, "shape").and_then(|s| crate::controls::number_of(&s)).is_some_and(|n| drop.contains(&n)); + if note || numbered { out.push_str(vml.get(last..a).unwrap_or("")); last = b; } @@ -595,19 +622,32 @@ pub(super) fn vml_block_max(vml: &str) -> u64 { used.max(claimed) } -/// Kept VML (controls, OLE objects) plus GridCraft's note shapes: the notes get ids in a fresh -/// shape block so they can't collide with the kept shapes the sheet XML refers to. +/// The first shape block free in kept VML: GridCraft's own shapes (notes, then form controls) are +/// numbered from `block * 1024 + 1` when they join it ([`merge_vml`]). +pub(super) fn first_free_block(kept: &str) -> u64 { + shape_numbers(kept).into_iter().max().unwrap_or(0) / 1024 + 1 +} + +/// Kept VML (controls, OLE objects) plus GridCraft's note and form control shapes: ours get ids +/// in fresh shape blocks so they can't collide with the kept shapes the sheet XML refers to. pub(super) fn merge_vml(kept: &str, ours: &str) -> String { let Some(close) = kept.rfind("") else { return ours.to_string() }; - let block = shape_numbers(kept).into_iter().max().unwrap_or(0) / 1024 + 1; + let block = first_free_block(kept); let mut add = String::new(); - if !kept.contains("id=\"_x0000_t202\"") - && let Some(i) = ours.find("")) - { - add.push_str(ours.get(i..i + j + "".len()).unwrap_or("")); + // Our shape types (notes, form controls) that the kept drawing doesn't define. + let mut from = 0; + while let Some(i) = ours.get(from..).and_then(|s| s.find("")).map(|j| i + j + "".len()) else { break }; + let ty = ours.get(i..j).unwrap_or(""); + let id = attr_value(ty, "id").unwrap_or(""); + if !id.is_empty() && !kept.contains(&format!("id=\"{id}\"")) && !kept.contains(&format!("id='{id}'")) { + add.push_str(ty); + } + from = j; } - for (k, (a, b)) in shape_spans(ours).into_iter().enumerate() { + let spans = shape_spans(ours); + let blocks = (spans.len() as u64 + 1).div_ceil(1024).max(1); + for (k, (a, b)) in spans.into_iter().enumerate() { let shape = ours.get(a..b).unwrap_or(""); let id = block.saturating_mul(1024).saturating_add(1 + k as u64); match (shape.find("id=\"_x0000_s"), shape.find("id=\"_x0000_s").and_then(|i| shape.get(i + 4..)?.find('"').map(|j| i + 4 + j))) { @@ -618,16 +658,134 @@ pub(super) fn merge_vml(kept: &str, ours: &str) -> String { } } let mut head = kept.get(..close).unwrap_or("").to_string(); - // Claim the block in the drawing's id map. + // Claim the blocks in the drawing's id map. if let Some(i) = head.find("(x: &'a str, name: &str) -> Option<&'a str> { + let mut from = 0; + while let Some(i) = x.get(from..)?.find(name).map(|i| i + from) { + from = i + name.len(); + let starts_token = x.get(..i).is_some_and(|before| before.ends_with(|c: char| c.is_whitespace() || c == '<')); + let Some(rest) = x.get(from..).map(str::trim_start).and_then(|r| r.strip_prefix('=')).map(str::trim_start) else { continue }; + if !starts_token { + continue; + } + if let Some(q) = rest.chars().next().filter(|q| *q == '"' || *q == '\'') { + let body = rest.get(q.len_utf8()..)?; + return body.find(q).and_then(|e| body.get(..e)); + } + } + None +} + +/// A kept `` fragment without the controls whose `shapeId` is in `drop` (they are +/// written from the model): a bare `` child goes when it is one of them, and within an +/// `mc:AlternateContent` child only those controls go, in each of its branches, the wrapper +/// staying while a control remains. `None` when nothing is left. A fragment that doesn't parse +/// is returned unchanged. +fn without_controls(frag: &str, drop: &HashSet) -> Option { + if drop.is_empty() { + return Some(frag.to_string()); + } + let mut reader = Reader::from_str(frag); + let mut out = String::with_capacity(frag.len()); + let (mut last, mut left, mut inside) = (0usize, 0usize, false); + loop { + let Ok(start) = usize::try_from(reader.buffer_position()) else { return Some(frag.to_string()) }; + let ev = match reader.read_event() { + Ok(ev) => ev, + Err(_) => return Some(frag.to_string()), + }; + let child = match ev { + Event::Eof => break, + Event::Start(e) if e.local_name().as_ref() == b"controls" => { + inside = true; + continue; + } + Event::End(e) if e.local_name().as_ref() == b"controls" => { + inside = false; + continue; + } + Event::Start(e) if inside => { + if reader.read_to_end(e.name()).is_err() { + return Some(frag.to_string()); + } + true + } + Event::Empty(_) if inside => true, + _ => false, + }; + if !child { + continue; + } + let Ok(end) = usize::try_from(reader.buffer_position()) else { return Some(frag.to_string()) }; + let text = frag.get(start..end).unwrap_or(""); + let Some((kept, controls, dropped)) = controls_kept(text, drop) else { return Some(frag.to_string()) }; + if dropped == 0 { + // Nothing of ours in it: the child stays as it is. + left += 1; + } else { + out.push_str(frag.get(last..start).unwrap_or("")); + if controls > 0 { + out.push_str(&kept); + left += 1; + } + last = end; + } + } + out.push_str(frag.get(last..).unwrap_or("")); + (left > 0).then_some(out) +} + +/// One child of a `` element with the `` elements numbered in `drop` taken +/// out, wherever they are in it (a bare control, or the Choice and Fallback branches of an +/// `mc:AlternateContent`), as (the text, the controls that remain, the controls taken out). The +/// `shapeId` is read as the control reader reads it (an attribute, in either quote style). `None` +/// when the child doesn't parse. +fn controls_kept(child: &str, drop: &HashSet) -> Option<(String, usize, usize)> { + let mut reader = Reader::from_str(child); + let mut out = String::with_capacity(child.len()); + let (mut last, mut kept, mut dropped) = (0usize, 0usize, 0usize); + loop { + let start = usize::try_from(reader.buffer_position()).ok()?; + let (e, has_end) = match reader.read_event().ok()? { + Event::Eof => break, + Event::Start(e) => (e, true), + Event::Empty(e) => (e, false), + _ => continue, + }; + if e.local_name().as_ref() != b"control" { + // A wrapper or a branch: its controls are the events that follow. + continue; + } + let number = + e.try_get_attribute("shapeId").ok().flatten().and_then(|a| std::str::from_utf8(&a.value).ok().and_then(crate::controls::shape_number)); + if has_end { + reader.read_to_end(e.name()).ok()?; + } + let end = usize::try_from(reader.buffer_position()).ok()?; + if number.is_some_and(|n| drop.contains(&n)) { + out.push_str(child.get(last..start)?); + last = end; + dropped += 1; + } else { + kept += 1; + } + } + out.push_str(child.get(last..)?); + Some((out, kept, dropped)) +} + /// Rewrites relationship-id attributes (`r:id`, `r:embed`…: any attribute in the relationships /// namespace) of an XML fragment through `map`. `None` when the fragment is damaged or `map` has /// no id for one of them. diff --git a/crates/xlsx/src/passthrough/rebase.rs b/crates/xlsx/src/passthrough/rebase.rs index d758f86d6..e9232658d 100644 --- a/crates/xlsx/src/passthrough/rebase.rs +++ b/crates/xlsx/src/passthrough/rebase.rs @@ -1,4 +1,4 @@ -use super::read::merge_vml; +use super::read::{first_free_block, merge_vml}; use super::write::{NameMap, attach, part_rels_xml, remap, write_parts}; use super::*; @@ -28,7 +28,15 @@ impl<'a> SheetKept<'a> { } } - /// GridCraft's notes VML for part `name`, merged with the kept control and object shapes. + /// The number GridCraft's own VML shapes start after when they join the kept drawing + /// ([`SheetKept::notes_vml`] numbers them `base + 1`, `base + 2`…); `None` without one. + pub fn shape_base(&self) -> Option { + let vml = self.vml()?; + Some(first_free_block(&String::from_utf8_lossy(&vml.data)).saturating_mul(1024)) + } + + /// GridCraft's notes and form controls VML for part `name`, merged with the kept control and + /// object shapes. pub fn notes_vml(&self, ours: String, name: &str, out: &mut Out) -> String { let Some(vml) = self.vml() else { return ours }; self.write_vml_rels(vml, name, out); @@ -50,21 +58,71 @@ impl<'a> SheetKept<'a> { self.ids.get(old).map(String::as_str) } - /// The kept sheet elements, in schema order. - pub fn elements(&self) -> String { - let Some(p) = self.p else { return String::new() }; + /// The kept sheet elements, in schema order, with GridCraft's own form control entries + /// (`controls`, see `controls::write`) in the sheet's one `` element. + pub fn elements(&self, controls: &str) -> String { let mut s = String::new(); - for slot in SHEET_SLOTS { - for e in p.elements.iter().filter(|e| e.slot == *slot) { - if let Some(x) = remap(&e.xml, &self.ids) { - s.push_str(&x); + let mut placed = controls.is_empty(); + if let Some(p) = self.p { + for slot in SHEET_SLOTS { + for e in p.elements.iter().filter(|e| e.slot == *slot) { + if let Some(mut x) = remap(&e.xml, &self.ids) { + if *slot == "controls" + && !placed + && let Some(i) = controls_insert_at(&x) + { + x.insert_str(i, controls); + placed = true; + } + s.push_str(&x); + } } } } + // `` comes last of the kept elements. + if !placed { + s.push_str(&crate::controls::controls_element(controls)); + } s } } +/// Where GridCraft's control entries go in a kept `` fragment: just before the close +/// tag of the `` element the reader reads (`controls::controls_of`), whatever its +/// namespace prefix. In an `mc:AlternateContent` with separate `` elements in its +/// Choice and Fallback branches that is the Choice's, the one an application that knows the +/// x14 extensions reads; the Fallback's is left as the file had it. With no Choice, or a Choice +/// without a `` element, it is the first `` element (a plain fragment, or a +/// Fallback alone). +fn controls_insert_at(x: &str) -> Option { + let from = open_tag(x, "Choice").unwrap_or(0); + close_tag_from(x, from, "controls") +} + +/// The local name of the tag whose text starts at `name` (`x:controls` → `controls`). +fn local_name(name: &str) -> &str { + name.rsplit(':').next().unwrap_or(name) +} + +/// The byte offset of the first open tag of `local` in `x`, with any namespace prefix. +fn open_tag(x: &str, local: &str) -> Option { + x.match_indices('<').find_map(|(i, _)| { + let rest = x.get(i + 1..)?; + let end = rest.find(|c: char| c.is_whitespace() || c == '>' || c == '/')?; + (local_name(rest.get(..end)?) == local).then_some(i) + }) +} + +/// The byte offset of the first close tag of `local` at or after `from` in `x`, with any +/// namespace prefix (``, ``). +fn close_tag_from(x: &str, from: usize, local: &str) -> Option { + x.get(from..)?.match_indices("')?)?.trim_end(); + (local_name(name) == local).then_some(from + i) + }) +} + // ---------------------------------------------------------------- linked workbooks /// `wb` as it is saved into the folder `to` (absolute): the relative paths of its linked diff --git a/crates/xlsx/src/passthrough/write.rs b/crates/xlsx/src/passthrough/write.rs index 8b8d24591..cf11d0216 100644 --- a/crates/xlsx/src/passthrough/write.rs +++ b/crates/xlsx/src/passthrough/write.rs @@ -15,6 +15,12 @@ pub struct KeptOut { } impl KeptOut { + /// Reserves a part name GridCraft writes itself, so kept parts don't take it. False when it is + /// taken. + pub fn reserve(&mut self, name: &str) -> bool { + self.used.insert(name.to_ascii_lowercase()) + } + /// Reserves the VML shape blocks the kept header and footer drawings of `wb` use, so that the /// notes drawings GridCraft writes don't reuse their shape ids. pub fn reserve_vml_blocks(&mut self, wb: &Workbook) { diff --git a/crates/xlsx/src/read.rs b/crates/xlsx/src/read.rs index 32b87183f..93efb7f02 100644 --- a/crates/xlsx/src/read.rs +++ b/crates/xlsx/src/read.rs @@ -327,12 +327,12 @@ pub fn read_xlsx<'a>(bytes: &'a [u8], known_function: Option<&'a dyn Fn(&str) -> continue; } let name = names.unique(&name, i); - let mut sheet = match crate::sheet_read::read_sheet(&mut cx, &rel.target, &name) { + let (mut sheet, read_controls) = match crate::sheet_read::read_sheet(&mut cx, &rel.target, &name) { Ok(s) => s, Err(IoError::TooLarge(m)) => return Err(IoError::TooLarge(m)), Err(e) => { cx.warn(format!("sheet \"{name}\" could not be read: {e}")); - Sheet::new(name.clone()) + (Sheet::new(name.clone()), Default::default()) } }; match crate::pivot::read_pivots(&mut cx, &mut pivot_caches, &rel.target, &name) { @@ -340,7 +340,7 @@ pub fn read_xlsx<'a>(bytes: &'a [u8], known_function: Option<&'a dyn Fn(&str) -> Err(IoError::TooLarge(m)) => return Err(IoError::TooLarge(m)), Err(e) => cx.warn(format!("PivotTables on sheet \"{name}\" could not be read: {e}")), } - sheet.preserved = kept.sheet(&mut cx, &rel.target, &name); + sheet.preserved = kept.sheet(&mut cx, &rel.target, &name, &read_controls); sheet.file_id = se.attr_u32("sheetId").unwrap_or(0); sheet.visibility = match se.attr("state") { Some("hidden") => Visibility::Hidden, diff --git a/crates/xlsx/src/sheet_read.rs b/crates/xlsx/src/sheet_read.rs index 73c98386a..f2047b6ee 100644 --- a/crates/xlsx/src/sheet_read.rs +++ b/crates/xlsx/src/sheet_read.rs @@ -157,7 +157,9 @@ fn read_sheet_format(e: &El, sheet: &mut Sheet) { } } -pub fn read_sheet(cx: &mut Ctx<'_>, part: &str, name: &str) -> Result { +/// Reads the sheet part `part`, with the VML shape numbers of the form controls read into the +/// model (pass-through keeps the other controls). +pub fn read_sheet(cx: &mut Ctx<'_>, part: &str, name: &str) -> Result<(Sheet, crate::controls::ReadControls), IoError> { let bytes = cx.pkg.read(part)?.ok_or_else(|| IoError::Format(format!("missing part {part}")))?; let rels = cx.pkg.rels(part)?; let mut sheet = Sheet::new(name); @@ -309,6 +311,7 @@ pub fn read_sheet(cx: &mut Ctx<'_>, part: &str, name: &str) -> Result {} } } + let read_controls = read_form_controls(cx, &root, &rels, &mut sheet); // Comments are related to the sheet without an element in it. for rel in rels.iter().filter(|r| r.kind == "comments") { if let Some(x) = cx.optional_xml(&rel.target)? { @@ -330,7 +333,17 @@ pub fn read_sheet(cx: &mut Ctx<'_>, part: &str, name: &str) -> Result` list (also inside +/// `mc:AlternateContent`) with its properties parts, else the shapes of the VML drawing that +/// `` names (not the header and footer drawing of ``). +/// Returns the VML shape numbers read, so pass-through keeps only the others. +fn read_form_controls(cx: &mut Ctx<'_>, root: &El, rels: &[Rel], sheet: &mut Sheet) -> crate::controls::ReadControls { + let controls = root.children.iter().filter_map(crate::controls::controls_of).next_back(); + let vml = root.child("legacyDrawing").and_then(|e| e.attr("id")); + if controls.is_some() || vml.is_some() { crate::controls::read(cx, controls, vml, rels, sheet) } else { Default::default() } } /// A protection password hash: the iterated attributes (`algorithmName`, `hashValue`, diff --git a/crates/xlsx/src/sheet_write.rs b/crates/xlsx/src/sheet_write.rs index ed3cb89ea..4a0f4fd03 100644 --- a/crates/xlsx/src/sheet_write.rs +++ b/crates/xlsx/src/sheet_write.rs @@ -281,29 +281,51 @@ fn write_sheet_extensions(s: &mut String, sheet: &Sheet, prio: &[u32]) { } fn write_sheet_notes(s: &mut String, sheet: &Sheet, si: usize, kept: &crate::passthrough::SheetKept<'_>, rels: &mut Rels, out: &mut Out) { - // Notes: comments part + VML so Excel shows them. Threaded comments also get a legacy note - // (its text with the replies as lines, for readers without threaded comments) linked by id. - if !sheet.comments.is_empty() { - out.comments += 1; - let n = out.comments; - let ids = thread_ids(sheet, si); - out.part(&format!("xl/comments{n}.xml"), Some(CT_COMMENTS), comments_xml(sheet, &ids).into_bytes()); - if !ids.is_empty() { - let x = threaded_xml(sheet, &ids, &mut out.persons); - out.part(&format!("xl/threadedComments/threadedComment{n}.xml"), Some(CT_THREADED_COMMENTS), x.into_bytes()); - rels.add_uri(REL_THREADED_COMMENT, &format!("../threadedComments/threadedComment{n}.xml")); + // Notes and form controls share one legacy VML drawing: notes need it for Excel to show them, + // and controls carry their `x:ClientData` there. Shape ids come from 1024-id blocks listed in + // the drawing's idmap (after the blocks kept header and footer drawings use), notes first, so + // a control's `shapeId` names its VML shape. Threaded comments also get a legacy note (its + // text with the replies as lines, for readers without threaded comments) linked by id. + let mut controls = String::new(); + if !sheet.comments.is_empty() || !sheet.controls.is_empty() { + let shapes = (sheet.comments.len() as u64).saturating_add(sheet.controls.len() as u64); + let blocks = u32::try_from(shapes.saturating_add(1).div_ceil(1024)).unwrap_or(u32::MAX).max(1); + let first = out.kept.notes_block(out.vml_block.saturating_add(1)); + out.vml_block = out.vml_block.saturating_add(blocks); + let idmap = (first..first.saturating_add(blocks)).map(|b| b.to_string()).collect::>().join(","); + // With a kept drawing, our shapes join it in a block of their own. + let base = kept.shape_base().unwrap_or(u64::from(first) * 1024); + let notes = sheet.comments.len() as u64; + let mut control_shapes = String::new(); + if !sheet.controls.is_empty() { + let w = crate::controls::write(out, rels, sheet, |i| base + 1 + notes + i as u64); + controls = w.entries; + control_shapes = format!("{}{}", crate::controls::SHAPETYPE, w.vml); } + if !sheet.comments.is_empty() { + out.comments += 1; + let n = out.comments; + let ids = thread_ids(sheet, si); + out.part(&format!("xl/comments{n}.xml"), Some(CT_COMMENTS), comments_xml(sheet, &ids).into_bytes()); + if !ids.is_empty() { + let x = threaded_xml(sheet, &ids, &mut out.persons); + out.part(&format!("xl/threadedComments/threadedComment{n}.xml"), Some(CT_THREADED_COMMENTS), x.into_bytes()); + rels.add_uri(REL_THREADED_COMMENT, &format!("../threadedComments/threadedComment{n}.xml")); + } + rels.add("comments", &format!("../comments{n}.xml")); + } + out.vmls += 1; + let v = out.vmls; out.default_type("vml", "application/vnd.openxmlformats-officedocument.vmlDrawing"); - let vml_name = format!("xl/drawings/vmlDrawing{n}.vml"); - let vml = kept.notes_vml(vml_xml(sheet, out.kept.notes_block(n)), &vml_name, out); + let vml_name = format!("xl/drawings/vmlDrawing{v}.vml"); + let vml = kept.notes_vml(vml_xml(sheet, &idmap, base, &control_shapes), &vml_name, out); out.part(&vml_name, None, vml.into_bytes()); - rels.add("comments", &format!("../comments{n}.xml")); - let rid = rels.add("vmlDrawing", &format!("../drawings/vmlDrawing{n}.vml")); + let rid = rels.add("vmlDrawing", &format!("../drawings/vmlDrawing{v}.vml")); let _ = write!(s, ""); } else if let Some(rid) = kept.lone_vml(rels, out) { let _ = write!(s, ""); } - s.push_str(&kept.elements()); + s.push_str(&kept.elements(&controls)); } fn write_sheet_drawing(s: &mut String, wb: &Workbook, si: usize, sheet: &Sheet, rels: &mut Rels, out: &mut Out) { diff --git a/crates/xlsx/src/sheet_write/parts.rs b/crates/xlsx/src/sheet_write/parts.rs index 040e2b9bd..eab48d097 100644 --- a/crates/xlsx/src/sheet_write/parts.rs +++ b/crates/xlsx/src/sheet_write/parts.rs @@ -358,13 +358,14 @@ fn note_anchor(cell: CellRef, b: Option<&NoteBox>) -> (String, (f64, f64)) { (format!("{}, {}", corner(&b.from), corner(&b.to)), b.size_pt.unwrap_or(DEFAULT_NOTE_SIZE_PT)) } -/// Legacy VML shapes Excel uses to display notes. -pub(super) fn vml_xml(sheet: &Sheet, n: u32) -> String { +/// Legacy VML shapes Excel uses to display notes (numbered from `base + 1`), followed by the +/// form control shapes `controls`. `idmap` lists the shape blocks the drawing uses. +pub(super) fn vml_xml(sheet: &Sheet, idmap: &str, base: u64, controls: &str) -> String { let mut s = format!( - "" + "" ); for (i, (cell, c)) in sheet.comments.iter().enumerate() { - let id = n as u64 * 1024 + 1 + i as u64; + let id = base + 1 + i as u64; let vis = if c.visible { "visible" } else { "hidden" }; let (anchor, (w, h)) = note_anchor(*cell, c.note_box.as_ref()); let _ = write!( @@ -378,6 +379,7 @@ pub(super) fn vml_xml(sheet: &Sheet, n: u32) -> String { if c.visible { "" } else { "" } ); } + s.push_str(controls); s.push_str(""); s } diff --git a/crates/xlsx/src/tests.rs b/crates/xlsx/src/tests.rs index 91d04f281..94b78ff65 100644 --- a/crates/xlsx/src/tests.rs +++ b/crates/xlsx/src/tests.rs @@ -7,6 +7,7 @@ mod chart_bool_default; mod chart_combo_kinds; mod chart_data_table; mod chart_ref_offsets; +mod controls_tests; mod csv_tests; mod encrypted_tests; mod equation_tests; diff --git a/crates/xlsx/src/tests/controls_tests.rs b/crates/xlsx/src/tests/controls_tests.rs new file mode 100644 index 000000000..ee56c9421 --- /dev/null +++ b/crates/xlsx/src/tests/controls_tests.rs @@ -0,0 +1,832 @@ +//! Form controls: the three parts written for each control agree, they read back, and files with +//! only VML or with odd values give valid controls. + +use gridcraft_core::CellRef; +use gridcraft_model::{Anchor, AnchorMode, Comment, ControlAction, ControlKind, FormControl, MacroInput, Sheet, Workbook}; + +use super::minimal; +use crate::package::Package; +use crate::{read_xlsx, write_xlsx}; + +fn at(a: &str) -> CellRef { + CellRef::parse(a).unwrap() +} + +fn anchor(cell: &str, width: f32, height: f32) -> Anchor { + Anchor { cell: at(cell), dx: 4.0, dy: 3.0, width, height, mode: AnchorMode::MoveOnly } +} + +fn note(author: &str, text: &str) -> Comment { + Comment { + author: author.into(), + text: text.into(), + replies: vec![], + threaded: false, + resolved: false, + visible: false, + dates: vec![], + note_box: None, + } +} + +fn book() -> Workbook { + let mut wb = Workbook::default(); + wb.sheets.clear(); + let mut sh = Sheet::new("Data"); + sh.comments.insert(at("A1"), note("Me", "a note")); + sh.controls.push(FormControl { + id: 1, + kind: ControlKind::Slider, + anchor: anchor("C3", 150.0, 20.0), + link: Some("$B$1".into()), + min: 5, + max: 500, + step: 5, + page: 50, + value: 35, + name: "Rate".into(), + ..Default::default() + }); + sh.controls.push(FormControl { + id: 2, + kind: ControlKind::Spin, + anchor: Anchor { mode: AnchorMode::Absolute, ..anchor("F3", 20.0, 40.0) }, + link: Some("'Other Sheet'!$A$2".into()), + max: 30_000, + step: 100, + value: 30_000, + print: false, + ..Default::default() + }); + sh.controls.push(FormControl { + id: 3, + kind: ControlKind::Button, + anchor: Anchor { mode: AnchorMode::MoveAndSize, ..anchor("C8", 100.0, 28.0) }, + text: "Run & go".into(), + action: Some(ControlAction::Macro { name: "Fill Down".into(), input: MacroInput::Selection, output: None }), + locked: false, + ..Default::default() + }); + sh.controls.push(FormControl { id: 4, kind: ControlKind::Slider, anchor: anchor("H3", 20.0, 120.0), horizontal: false, ..Default::default() }); + wb.sheets.push(sh.into()); + wb.sheets.push(Sheet::new("Other Sheet").into()); + wb +} + +fn text(pkg: &mut Package<'_>, part: &str) -> String { + String::from_utf8(pkg.read(part).unwrap().unwrap_or_else(|| panic!("missing {part}"))).unwrap() +} + +#[test] +fn the_three_parts_agree_and_read_back() { + let wb = book(); + let bytes = write_xlsx(&wb).unwrap(); + let mut pkg = Package::open(&bytes).unwrap(); + let sheet = text(&mut pkg, "xl/worksheets/sheet1.xml"); + let vml = text(&mut pkg, "xl/drawings/vmlDrawing1.vml"); + let types = text(&mut pkg, "[Content_Types].xml"); + assert!(sheet.contains(""), "{sheet}"); + // Each control's shape id is a VML shape of the same kind; the note keeps its own id. + let ids: Vec = sheet.split("").next().unwrap().contains(&format!("ObjectType=\"{kind}\"")), "{id}"); + } + assert!(vml.contains("id=\"_x0000_s1025\"") && vml.contains("ObjectType=\"Note\""), "the note is shape 1025"); + assert!(!ids.contains(&"1025".to_string())); + for n in 1..=4 { + assert!(types.contains(&format!("/xl/ctrlProps/ctrlProp{n}.xml")), "content type {n}"); + } + let prop = text(&mut pkg, "xl/ctrlProps/ctrlProp1.xml"); + assert!( + prop.contains("objectType=\"Scroll\"") && prop.contains("fmlaLink=\"$B$1\"") && prop.contains("page=\"50\"") && prop.contains("horiz=\"1\""), + "{prop}" + ); + assert!(sheet.contains("macro=\"[0]!Fill Down\"")); + + let (back, report) = read_xlsx(&bytes).unwrap(); + assert!(report.warnings.is_empty(), "{:?}", report.warnings); + let got = &back.sheet(0).unwrap().controls; + assert_eq!(got.len(), 4); + for (a, b) in wb.sheet(0).unwrap().controls.iter().zip(got) { + let same = |x: &FormControl| { + (x.kind, x.link.clone(), x.min, x.max, x.step, x.value, x.horizontal, x.print, x.locked, x.action.clone(), x.anchor.cell, x.anchor.mode) + }; + assert_eq!(same(a), same(b)); + assert!( + (a.anchor.width - b.anchor.width).abs() < 1.0 + && (a.anchor.height - b.anchor.height).abs() < 1.0 + && (a.anchor.dx - b.anchor.dx).abs() < 1.0 + ); + if a.kind == ControlKind::Slider { + assert_eq!(a.page, b.page); + } + } + assert_eq!(got[0].name, "Rate"); + assert_eq!(got[2].text, "Run & go"); + assert_eq!(back.sheet(0).unwrap().comments.len(), 1, "notes still read"); +} + +#[test] +fn many_shapes_take_several_id_blocks() { + let mut wb = Workbook::default(); + wb.sheets.clear(); + let mut sh = Sheet::new("Data"); + for i in 0..1100u32 { + sh.controls.push(FormControl { id: i + 1, kind: ControlKind::Spin, anchor: anchor("C3", 20.0, 30.0), ..Default::default() }); + } + wb.sheets.push(sh.into()); + wb.sheets.push(Sheet::new("Next").into()); + if let Some(s) = wb.sheet_mut(1) { + s.comments.insert(at("B2"), note("x", "y")); + } + let bytes = write_xlsx(&wb).unwrap(); + let mut pkg = Package::open(&bytes).unwrap(); + assert!(text(&mut pkg, "xl/drawings/vmlDrawing1.vml").contains("data=\"1,2\"")); + let second = text(&mut pkg, "xl/drawings/vmlDrawing2.vml"); + assert!(second.contains("data=\"3\"") && second.contains("_x0000_s3073"), "the next sheet's ids don't collide"); + let (back, _) = read_xlsx(&bytes).unwrap(); + assert_eq!(back.sheet(0).unwrap().controls.len(), 1100); +} + +const VML_ONLY: &str = r##" +2, 0, 3, 0, 5, 0, 4, 0$B$150000-5999990-3 +1, 0, 1, 0, 1, 20, 3, 0799 +
Go
1, 0, 6, 0, 3, 0, 7, 0[0]!Tidy +1, 0, 1, 0, 2, 0, 2, 0 +junk +"##; + +#[test] +fn vml_only_controls_are_read_and_clamped() { + let rels = r#""#; + let bytes = minimal( + r#""#, + &[("xl/worksheets/_rels/sheet1.xml.rels", rels), ("xl/drawings/vmlDrawing1.vml", VML_ONLY)], + "", + "", + ); + let (wb, report) = read_xlsx(&bytes).unwrap(); + assert!(report.warnings.iter().all(|w| !w.contains("form controls are not supported")), "{:?}", report.warnings); + let c = &wb.sheet(0).unwrap().controls; + assert_eq!(c.len(), 3, "the drop-down and the shape without an anchor are skipped"); + assert_eq!((c[0].kind, c[0].min, c[0].max, c[0].step, c[0].value, c[0].horizontal), (ControlKind::Slider, 0, 30_000, 1, 30_000, true)); + assert!(c[0].page >= c[0].step && c[0].check_numbers().is_ok()); + assert_eq!((c[1].min, c[1].max), (9, 10), "a range with no width gets one"); + assert!(c[1].check_numbers().is_ok()); + assert_eq!(c[2].text, "Go"); + assert_eq!(c[2].action, Some(ControlAction::Macro { name: "Tidy".into(), input: MacroInput::Selection, output: None })); +} + +/// A producer that quotes `shapeId` with single quotes: the control is read, and the kept +/// `` fragment drops it as it drops a double-quoted one, so a save holds the control +/// once and no kept copy points at a VML shape that was taken out. +#[test] +fn a_single_quoted_shape_id_is_dropped_from_the_kept_controls_too() { + let rels = r#""#; + let vml = r##"2, 0, 3, 0, 5, 0, 4, 0$B$1400100110"##; + let bytes = minimal( + r#""#, + &[("xl/worksheets/_rels/sheet1.xml.rels", rels), ("xl/drawings/vmlDrawing1.vml", vml)], + "", + "", + ); + let (wb, report) = read_xlsx(&bytes).unwrap(); + // The minimal package has no styles part; nothing about the control is reported. + assert!(report.warnings.iter().all(|w| w == "no styles part"), "{:?}", report.warnings); + let sh = wb.sheet(0).unwrap(); + assert_eq!(sh.controls.len(), 1); + assert_eq!((sh.controls[0].kind, sh.controls[0].value, sh.controls[0].name.as_str()), (ControlKind::Slider, 40, "Scroll 1")); + let kept = sh.preserved.as_ref().and_then(|k| k.elements.iter().find(|e| e.slot == "controls")).map(|e| e.xml.clone()); + assert_eq!(kept, None, "nothing of the modelled control is kept for pass-through"); + // The saved file holds the control once, with its one VML shape, and reads back once. + let out = write_xlsx(&wb).unwrap(); + let mut pkg = Package::open(&out).unwrap(); + let sheet = text(&mut pkg, "xl/worksheets/sheet1.xml"); + // One element with one control (each control is written as a Choice with a + // Fallback inside the element's own AlternateContent), and no kept copy. + assert_eq!(sheet.matches("").count(), 1, "{sheet}"); + assert!(!sheet.contains("shapeId='1025'"), "the kept copy would point at a shape that is gone: {sheet}"); + assert_eq!(text(&mut pkg, "xl/drawings/vmlDrawing1.vml").matches(""#; + let prop_a = r#""#; + let prop_b = r#""#; + let vml = r##"2, 0, 3, 0, 5, 0, 4, 0$B$14001001102, 0, 6, 0, 5, 0, 9, 0$D$1:$D$32"##; + let anchor = |from: u32, to: u32| { + format!( + r#"20{from}050{to}0"# + ) + }; + let controls = format!( + r#"{}{}"#, + anchor(3, 4), + anchor(6, 9) + ); + let bytes = minimal( + &format!(r#"{controls}"#), + &[ + ("xl/worksheets/_rels/sheet1.xml.rels", rels), + ("xl/ctrlProps/a.xml", prop_a), + ("xl/ctrlProps/b.xml", prop_b), + ("xl/drawings/vmlDrawing1.vml", vml), + ], + "", + "", + ); + let (wb, _) = read_xlsx(&bytes).unwrap(); + let sh = wb.sheet(0).unwrap(); + assert_eq!(sh.controls.len(), 1, "the scroll bar is modelled, the list box is not"); + assert_eq!((sh.controls[0].kind, sh.controls[0].name.as_str()), (ControlKind::Slider, "Scroll 1")); + let kept = sh.preserved.as_ref().and_then(|k| k.elements.iter().find(|e| e.slot == "controls")).map(|e| e.xml.clone()).unwrap_or_default(); + assert!(kept.contains("List 1") && kept.contains(""), "the list box stays, in its wrapper: {kept}"); + assert!(!kept.contains("Scroll 1") && !kept.contains("shapeId=\"1025\""), "the scroll bar leaves every branch: {kept}"); + // Saved: the scroll bar once (one Fallback of ours), the list box in the kept wrapper with its + // shape and its properties part. + let out = write_xlsx(&wb).unwrap(); + let mut pkg = Package::open(&out).unwrap(); + let sheet = text(&mut pkg, "xl/worksheets/sheet1.xml"); + assert_eq!(sheet.matches(" = pkg.names().into_iter().filter(|n| n.starts_with("xl/ctrlprops/")).collect(); + assert_eq!(props.len(), 2, "ours and the kept one: {props:?}"); + assert!(props.iter().any(|n| text(&mut pkg, n).contains("objectType=\"List\"")), "the list box's properties part survives: {props:?}"); + // Read back: the one modelled control and the kept one. + let (back, _) = read_xlsx(&out).unwrap(); + let sh = back.sheet(0).unwrap(); + assert_eq!(sh.controls.len(), 1); + let kept = sh.preserved.as_ref().and_then(|k| k.elements.iter().find(|e| e.slot == "controls")).map(|e| e.xml.clone()).unwrap_or_default(); + assert!(kept.contains("List 1") && !kept.contains("Scroll 1"), "{kept}"); +} + +/// A producer that prefixes the main namespace (``): the kept element is still the +/// sheet's one `` element, so a modelled control is written into it rather than into a +/// second one, which the worksheet schema does not allow. +#[test] +fn a_modelled_control_joins_a_kept_controls_element_with_a_namespace_prefix() { + use gridcraft_model::{Anchor, ControlKind, FormControl}; + let rels = r#""#; + let prop_b = r#""#; + let vml = r##"2, 0, 6, 0, 5, 0, 9, 0$D$1:$D$32"##; + let controls = r#"20605090"#; + let bytes = minimal( + &format!(r#"{controls}"#), + &[("xl/worksheets/_rels/sheet1.xml.rels", rels), ("xl/ctrlProps/b.xml", prop_b), ("xl/drawings/vmlDrawing1.vml", vml)], + "", + "", + ); + let (mut wb, _) = read_xlsx(&bytes).unwrap(); + assert!(wb.sheet(0).unwrap().controls.is_empty(), "the list box stays with pass-through"); + let kept = wb + .sheet(0) + .unwrap() + .preserved + .as_ref() + .and_then(|k| k.elements.iter().find(|e| e.slot == "controls")) + .map(|e| e.xml.clone()) + .unwrap_or_default(); + assert!(kept.contains(""), "kept with its prefix: {kept}"); + let anchor = Anchor { cell: gridcraft_core::CellRef::new(2, 1), width: 120.0, height: 20.0, ..Default::default() }; + wb.sheet_mut(0).unwrap().controls.push(FormControl { + id: 900, + kind: ControlKind::Slider, + anchor, + link: Some("$B$1".into()), + value: 40, + ..Default::default() + }); + let out = write_xlsx(&wb).unwrap(); + let mut pkg = Package::open(&out).unwrap(); + let sheet = text(&mut pkg, "xl/worksheets/sheet1.xml"); + let opened = sheet.matches("").unwrap(); + let inner = &sheet[start..end]; + assert!(inner.contains("name=\"List 1\"") && inner.contains(""), "both controls inside the one element: {sheet}"); + let (back, _) = read_xlsx(&out).unwrap(); + assert_eq!(back.sheet(0).unwrap().controls.len(), 1, "the slider reads back; the list box stays kept"); +} + +/// A kept `mc:AlternateContent` with separate `` elements in its Choice and Fallback: +/// a modelled control is written into the Choice's element, the one the reader (and an +/// application that knows x14) reads, so it comes back with its name and its properties part; +/// the Fallback's element is left as the file had it. +#[test] +fn a_modelled_control_joins_the_choice_controls_element_of_a_kept_alternate_content() { + use gridcraft_model::{Anchor, ControlKind, FormControl}; + let rels = r#""#; + let prop_b = r#""#; + let vml = r##"2, 0, 6, 0, 5, 0, 9, 0$D$1:$D$32"##; + let list = r#"20605090"#; + let controls = format!( + r#"{list}"# + ); + let bytes = minimal( + &format!(r#"{controls}"#), + &[("xl/worksheets/_rels/sheet1.xml.rels", rels), ("xl/ctrlProps/b.xml", prop_b), ("xl/drawings/vmlDrawing1.vml", vml)], + "", + "", + ); + let (mut wb, _) = read_xlsx(&bytes).unwrap(); + assert!(wb.sheet(0).unwrap().controls.is_empty(), "the list box stays with pass-through"); + let anchor = Anchor { cell: gridcraft_core::CellRef::new(2, 1), width: 120.0, height: 20.0, ..Default::default() }; + wb.sheet_mut(0).unwrap().controls.push(FormControl { + id: 900, + kind: ControlKind::Slider, + anchor, + name: "Speed".into(), + link: Some("$B$1".into()), + value: 40, + ..Default::default() + }); + let out = write_xlsx(&wb).unwrap(); + let mut pkg = Package::open(&out).unwrap(); + let sheet = text(&mut pkg, "xl/worksheets/sheet1.xml"); + // The Choice's comes first in the sheet, the Fallback's last. + let (first, last) = (sheet.find(" +
Go
1, 0, 6, 0, 3, 0, 7, 0[0]!Tidy +1, 0, 1, 0, 1, 20, 3, 079 +"##; + +#[test] +fn vml_shapes_are_also_found_by_their_spid() { + let rels = r#""#; + // The button is listed in without a properties part; the spin button is only in + // the VML. + let bytes = minimal( + r#""#, + &[("xl/worksheets/_rels/sheet1.xml.rels", rels), ("xl/drawings/vmlDrawing1.vml", VML_SPID)], + "", + "", + ); + let (wb, _) = read_xlsx(&bytes).unwrap(); + let c = &wb.sheet(0).unwrap().controls; + assert_eq!(c.len(), 2, "{c:?}"); + assert_eq!((c[0].kind, c[0].name.as_str(), c[0].text.as_str()), (ControlKind::Button, "Go Button", "Go")); + assert_eq!(c[0].action, Some(ControlAction::Macro { name: "Tidy".into(), input: MacroInput::Selection, output: None })); + assert_eq!((c[1].kind, c[1].value, c[1].max), (ControlKind::Spin, 7, 9)); + // Saved again, each is written once and nothing of them is kept twice. + let out = write_xlsx(&wb).unwrap(); + let (back, _) = read_xlsx(&out).unwrap(); + assert_eq!(back.sheet(0).unwrap().controls.len(), 2); +} + +#[test] +fn the_properties_part_wins_over_the_vml() { + let mut wb = book(); + if let Some(s) = wb.sheet_mut(0) { + s.comments.clear(); + s.controls.truncate(1); + } + let bytes = write_xlsx(&wb).unwrap(); + // Rewrite the VML's numbers: the reader still takes the properties part's. + let mut pkg = Package::open(&bytes).unwrap(); + let mut parts = Vec::new(); + for name in pkg.names() { + let mut data = pkg.read(&name).unwrap().unwrap(); + if name.ends_with(".vml") { + data = String::from_utf8(data).unwrap().replace("500", "9").into_bytes(); + } + parts.push((name, data)); + } + let zipped = super::make_zip(&parts.iter().map(|(n, d)| (n.as_str(), d.as_slice())).collect::>()); + let (back, _) = read_xlsx(&zipped).unwrap(); + assert_eq!(back.sheet(0).unwrap().controls[0].max, 500); +} + +/// A hostile sheet: more controls than a sheet keeps, properties parts that are not XML or hold +/// numbers no control can take, shape ids that are not numbers, and oversized names, captions and +/// links. Reading must not panic, must keep at most the cap, and every control it keeps must pass +/// the model's rules. +#[test] +fn hostile_form_controls_are_capped_and_repaired() { + let n = 13_000; + let mut controls = String::from(""); + let anchor = r#"1-99999999999999999999990-4020"#; + for i in 0..n { + let rid = match i % 10 { + 0 => "rIdB", + 1 => "rIdC", + _ => "rIdA", + }; + let _ = std::fmt::Write::write_fmt( + &mut controls, + format_args!(r#"{anchor}"#, 2000 + i, "n".repeat(400)), + ); + } + controls + .push_str(r#""#); + let rels = r#""#; + let link = format!("$B${}", "9".repeat(500)); + let prop_a = format!( + r#""# + ); + let body = format!(r#"{controls}"#) + .replace("", r#""#); + let bytes = minimal( + &body, + &[("xl/worksheets/_rels/sheet1.xml.rels", rels), ("xl/ctrlProps/a.xml", &prop_a), ("xl/ctrlProps/b.xml", "\u{0}\u{1}not xml <<<")], + "", + "", + ); + let (wb, _) = read_xlsx(&bytes).unwrap(); + let c = &wb.sheet(0).unwrap().controls; + assert_eq!(c.len(), 10_000, "the cap of one sheet"); + for x in c { + assert_eq!(x.kind, ControlKind::Slider); + assert!(x.check_numbers().is_ok(), "{x:?}"); + assert!(x.value >= x.min && x.value <= x.max, "{x:?}"); + assert!(x.name.chars().count() <= 255 && x.link.is_none(), "{x:?}"); + assert!(x.anchor.width >= 1.0 && x.anchor.height >= 1.0 && x.anchor.cell.row < gridcraft_core::MAX_ROWS, "{x:?}"); + } + // Written back and read again, the kept controls stay valid. + let (back, _) = read_xlsx(&write_xlsx(&wb).unwrap()).unwrap(); + assert_eq!(back.sheet(0).unwrap().controls.len(), c.len()); +} + +/// Each listed control finds its VML shape through an index, so a sheet of many controls reads in +/// linear time (it used to scan every shape for every control). +#[test] +#[allow(clippy::disallowed_methods)] // a native-only timing test: the clock is read off wasm +fn many_listed_controls_read_in_linear_time() { + let mut wb = Workbook::default(); + if let Some(sh) = wb.sheet_mut(0) { + for i in 0..10_000u32 { + let a = anchor(&format!("B{}", i + 1), 60.0, 18.0); + sh.controls.push(FormControl { id: i + 1, kind: ControlKind::Spin, anchor: a, ..Default::default() }); + } + } + let bytes = write_xlsx(&wb).unwrap(); + let t0 = std::time::Instant::now(); + let (back, _) = read_xlsx(&bytes).unwrap(); + let took = t0.elapsed(); + assert_eq!(back.sheet(0).unwrap().controls.len(), 10_000); + // About 0.4 s here; scanning every shape for every control took over 8 s. + assert!(took < std::time::Duration::from_secs(3), "{took:?}"); +} + +/// The sheet relates each control properties part with the relationship type MS-XLSX defines and +/// Excel writes (in the officeDocument namespace); Excel finds the part by that type. +#[test] +fn the_properties_parts_use_the_ctrl_prop_relationship_type() { + let bytes = write_xlsx(&book()).unwrap(); + let mut pkg = Package::open(&bytes).unwrap(); + let rels = text(&mut pkg, "xl/worksheets/_rels/sheet1.xml.rels"); + let types: Vec<&str> = rels.split("Type=\"").skip(1).filter_map(|r| r.split('"').next()).filter(|t| t.ends_with("/ctrlProp")).collect(); + assert_eq!(types.len(), 4, "{rels}"); + assert!(types.iter().all(|t| *t == "http://schemas.openxmlformats.org/officeDocument/2006/relationships/ctrlProp"), "{types:?}"); +} + +/// A sheet with a header and footer drawing and a drawing for its controls: the controls come +/// from the drawing `` names, whichever relationship is listed first. +#[test] +fn controls_come_from_the_legacy_drawing_not_the_header_and_footer_drawing() { + let hf = r##""##; + let rels = r#""#; + let bytes = minimal( + r#""#, + &[("xl/worksheets/_rels/sheet1.xml.rels", rels), ("xl/drawings/vmlDrawingHF1.vml", hf), ("xl/drawings/vmlDrawing1.vml", VML_ONLY)], + "", + "", + ); + let (wb, _) = read_xlsx(&bytes).unwrap(); + let c = &wb.sheet(0).unwrap().controls; + assert_eq!(c.len(), 3, "{c:?}"); + assert_eq!(c[2].text, "Go"); +} + +/// Every listed control that names the same properties part shares one read of it. Each read is +/// charged to the package's 1 GB budget, so 10,000 reads of a 200 KB part would spend it, drop the +/// controls after that, and fail the parts read after the controls (here the sheet's notes). A +/// hostile file can list that many controls against one part. +#[test] +fn controls_naming_one_properties_part_read_it_once() { + let n = 13_000; + let mut controls = String::from(r#""#); + for i in 0..n { + let _ = std::fmt::Write::write_fmt( + &mut controls, + format_args!( + r#"10{i}020{}0"#, + 2000 + i, + i + 1 + ), + ); + } + controls.push_str(""); + let rels = r#""#; + // 200 KB of attribute whitespace: read once per control, 10,000 reads would be 2 GB. + let prop_a = format!( + r#""#, + " ".repeat(200 * 1024) + ); + let notes = r#"Annread after the controls"#; + let bytes = minimal( + &format!("{controls}"), + &[("xl/worksheets/_rels/sheet1.xml.rels", rels), ("xl/ctrlProps/a.xml", &prop_a), ("xl/comments1.xml", notes)], + "", + "", + ); + let (wb, report) = read_xlsx(&bytes).unwrap(); + let sh = wb.sheet(0).unwrap(); + assert_eq!(sh.controls.len(), 10_000, "the cap of one sheet, every control read from the one part"); + assert!(sh.controls.iter().all(|c| { + c.kind == ControlKind::Slider && c.link.as_deref() == Some("$B$1") && (c.min, c.max, c.step, c.page, c.value) == (0, 200, 5, 20, 45) + })); + assert_eq!(sh.comments.len(), 1, "a part read after the controls still reads: {:?}", report.warnings); +} + +/// A legacy VML drawing with a button (`1025`, modelled), a check box (`1026`) and an option +/// button (`1027`, with a picture through the drawing's own relationship) that only the drawing +/// lists, plus `note` (a note shape for A1 whose client data carries that `ObjectType` attribute, +/// with its comments part) when given. `listed`: the sheet's +/// `` lists the button. +fn vml_only_others(listed: bool, button: bool, note: Option<&str>) -> Vec { + const REL: &str = "http://schemas.openxmlformats.org/officeDocument/2006/relationships"; + let mut rels = format!(r#""#); + if listed { + rels.push_str(&format!(r#""#)); + } + if note.is_some() { + rels.push_str(&format!(r#""#)); + } + let rels = + format!(r#"{rels}"#); + let vml_rels = format!( + r#""# + ); + let prop_a = r#""#; + let mut shapes = String::new(); + if let Some(attr) = note { + shapes.push_str(&format!(r##"1, 15, 0, 2, 3, 15, 4, 400"##)); + } + if button { + shapes.push_str(r##"
Go
2, 0, 3, 0, 4, 0, 5, 0[0]!Go"##); + } + shapes.push_str(r##"
Paid
2, 0, 7, 0, 4, 0, 8, 0$B$71"##); + shapes.push_str(r##"2, 0, 9, 0, 4, 0, 10, 0$B$9"##); + let vml = format!( + r#"{shapes}"# + ); + let controls = if listed { + r#"20304050"# + } else { + "" + }; + let notes = r#"Anncheck the totals"#; + let mut parts: Vec<(&str, &str)> = vec![ + ("xl/worksheets/_rels/sheet1.xml.rels", &rels), + ("xl/drawings/vmlDrawing1.vml", &vml), + ("xl/drawings/_rels/vmlDrawing1.vml.rels", &vml_rels), + ("xl/media/image1.png", "not really a png"), + ]; + if listed { + parts.push(("xl/ctrlProps/a.xml", prop_a)); + } + if note.is_some() { + parts.push(("xl/comments1.xml", notes)); + } + // As `minimal` builds it, with content types for the picture and the drawing. + let ct = super::CT.replace( + "", + r#""#, + ); + let base = minimal(&format!(r#"{controls}"#), &parts, "", ""); + let mut zip = Package::open(&base).unwrap(); + let mut files: Vec<(String, Vec)> = vec![("[Content_Types].xml".into(), ct.into_bytes())]; + for n in zip.names() { + if n != "[Content_Types].xml" { + let data = zip.read(&n).unwrap().unwrap(); + files.push((n, data)); + } + } + super::make_zip(&files.iter().map(|(n, d)| (n.as_str(), d.as_slice())).collect::>()) +} + +/// Checks a saved package keeps the check box and option button shapes once each, in the sheet's +/// one VML drawing, with that drawing's one relationship to its one picture; returns the drawing. +fn kept_vml_only_others(out: &[u8]) -> String { + let mut pkg = Package::open(out).unwrap(); + let sheet = text(&mut pkg, "xl/worksheets/sheet1.xml"); + assert_eq!(sheet.matches(" = names.iter().filter(|n| n.ends_with(".vml")).collect(); + assert_eq!(vmls.len(), 1, "{names:?}"); + let vml_name = vmls[0].clone(); + let vml = text(&mut pkg, &vml_name); + assert_eq!(vml.matches("ObjectType=\"Checkbox\"").count(), 1, "the check box is kept once: {vml}"); + assert_eq!(vml.matches("ObjectType=\"Radio\"").count(), 1, "the option button is kept once: {vml}"); + let (dir, file) = vml_name.rsplit_once('/').unwrap(); + let vml_rels = text(&mut pkg, &format!("{dir}/_rels/{file}.rels")); + assert_eq!(vml_rels.matches(" = names.iter().filter(|n| n.starts_with("xl/media/")).collect(); + assert_eq!(media.len(), 1, "the option button's picture once: {names:?}"); + vml +} + +/// The sheet's `` lists only a button, which is modelled; a check box and an option +/// button in the same VML drawing have no worksheet entry. With the button's entry gone from the +/// kept content no `` element is left, but the drawing still holds the two, so it is +/// kept: the save writes them beside the regenerated button, and they survive a second round trip. +#[test] +fn vml_only_controls_survive_when_the_last_listed_control_is_modelled() { + let bytes = vml_only_others(true, true, None); + let (wb, report) = read_xlsx(&bytes).unwrap(); + let sh = wb.sheet(0).unwrap(); + assert_eq!(sh.controls.len(), 1, "the button is modelled"); + assert_eq!(sh.controls[0].kind, ControlKind::Button); + assert!(sh.preserved.as_ref().is_some_and(|p| p.vml.is_some()), "the drawing is kept: {:?}", report.warnings); + assert!(report.warnings.iter().any(|w| w.contains("form controls are not shown")), "{:?}", report.warnings); + let out = write_xlsx(&wb).unwrap(); + let vml = kept_vml_only_others(&out); + assert_eq!(vml.matches("ObjectType=\"Button\"").count(), 1, "the button once, written from the model: {vml}"); + let (back, _) = read_xlsx(&out).unwrap(); + let sh = back.sheet(0).unwrap(); + assert_eq!(sh.controls.len(), 1); + assert_eq!(sh.controls[0].text, "Go"); + // Saved again: still once each. + let again = write_xlsx(&back).unwrap(); + let vml = kept_vml_only_others(&again); + assert_eq!(vml.matches("ObjectType=\"Button\"").count(), 1, "{vml}"); +} + +/// A sheet whose only controls are a check box and an option button that only the VML drawing +/// lists (no ``, no notes, nothing modelled): the drawing is kept on its own. +#[test] +fn a_sheet_whose_only_controls_are_vml_only_and_unmodelled_keeps_them() { + let bytes = vml_only_others(false, false, None); + let (wb, report) = read_xlsx(&bytes).unwrap(); + assert!(wb.sheet(0).unwrap().controls.is_empty()); + let out = write_xlsx(&wb).unwrap(); + let vml = kept_vml_only_others(&out); + assert!(!vml.contains("ObjectType=\"Button\""), "{vml}"); + let (back, _) = read_xlsx(&out).unwrap(); + let again = write_xlsx(&back).unwrap(); + kept_vml_only_others(&again); + assert!(report.warnings.iter().any(|w| w.contains("form controls are not shown")), "{:?}", report.warnings); +} + +/// The same with a note in the drawing (one VML drawing for one comments part) and a VML-only +/// button that is modelled: the note and the button are written from the model, the check box and +/// option button stay, all in the one drawing. +#[test] +fn vml_only_controls_beside_a_note_are_kept() { + let bytes = vml_only_others(false, true, Some(r#"ObjectType="Note""#)); + let (wb, report) = read_xlsx(&bytes).unwrap(); + let sh = wb.sheet(0).unwrap(); + assert_eq!((sh.controls.len(), sh.comments.len()), (1, 1), "{:?}", report.warnings); + let out = write_xlsx(&wb).unwrap(); + let vml = kept_vml_only_others(&out); + assert_eq!(vml.matches("ObjectType=\"Note\"").count(), 1, "{vml}"); + assert_eq!(vml.matches("ObjectType=\"Button\"").count(), 1, "{vml}"); + let (back, _) = read_xlsx(&out).unwrap(); + let sh = back.sheet(0).unwrap(); + assert_eq!((sh.controls.len(), sh.comments.len()), (1, 1)); + let vml = kept_vml_only_others(&write_xlsx(&back).unwrap()); + assert_eq!(vml.matches("ObjectType=\"Note\"").count(), 1, "{vml}"); +} + +#[test] +fn a_note_spelled_in_another_case_or_quote_is_not_kept_beside_the_written_one() { + // The note's client data reads ObjectType='note': still a note, so the kept drawing drops it + // and the note GridCraft writes for the comment is the only one. + for attr in ["ObjectType='note'", "ObjectType = \"NOTE\"", "ObjectType=\" Note \""] { + let (wb, report) = read_xlsx(&vml_only_others(false, true, Some(attr))).unwrap(); + assert_eq!(wb.sheet(0).unwrap().comments.len(), 1, "{attr}: {:?}", report.warnings); + let out = write_xlsx(&wb).unwrap(); + let vml = kept_vml_only_others(&out); + assert_eq!(note_shapes(&vml), 1, "{attr}: {vml}"); + let (back, _) = read_xlsx(&out).unwrap(); + assert_eq!(back.sheet(0).unwrap().comments.len(), 1, "{attr}"); + } +} + +/// The note shapes in VML text, counted as the reader tells a note: by each shape's client data +/// `ObjectType`, whatever its spelling. +fn note_shapes(vml: &str) -> usize { + crate::xml::parse(vml.as_bytes()).unwrap().kids("shape").filter(|s| s.child("ClientData").is_some_and(crate::controls::is_note_data)).count() +} + +/// The saved package's one VML drawing. +fn only_vml(out: &[u8]) -> String { + let mut pkg = Package::open(out).unwrap(); + let names: Vec = pkg.names().into_iter().filter(|n| n.ends_with(".vml")).collect(); + assert_eq!(names.len(), 1, "{names:?}"); + text(&mut pkg, &names[0]) +} + +/// A sheet's VML drawing, as the only part beside the sheet's relationships. +fn with_vml(shapes: &str) -> Vec { + let rels = r#""#; + let vml = format!( + r#"{shapes}"# + ); + minimal( + r#""#, + &[("xl/worksheets/_rels/sheet1.xml.rels", rels), ("xl/drawings/vmlDrawing1.vml", &vml)], + "", + "", + ) +} + +/// Text that reads like a note's `ObjectType` in a check box's caption or in another attribute of +/// an option button doesn't make either a note: both are kept, and a modelled button with that +/// caption keeps it. +#[test] +fn a_control_whose_text_reads_like_a_note_object_type_is_kept() { + let bytes = with_vml(concat!( + r##"
ObjectType="Note"
2, 0, 3, 0, 4, 0, 5, 0"##, + r##"
ObjectType="Note"
2, 0, 7, 0, 4, 0, 8, 0$B$7"##, + r##"2, 0, 9, 0, 4, 0, 10, 0"##, + )); + let (wb, report) = read_xlsx(&bytes).unwrap(); + let sh = wb.sheet(0).unwrap(); + assert_eq!(sh.controls.len(), 1, "{:?}", report.warnings); + assert_eq!(sh.controls[0].text, "ObjectType=\"Note\""); + let mut out = write_xlsx(&wb).unwrap(); + for _ in 0..2 { + let vml = only_vml(&out); + assert_eq!(vml.matches("ObjectType=\"Checkbox\"").count(), 1, "the check box is kept: {vml}"); + assert_eq!(vml.matches("ObjectType=\"Radio\"").count(), 1, "the option button is kept: {vml}"); + assert_eq!(vml.matches("ObjectType=\"Button\"").count(), 1, "the button once: {vml}"); + assert_eq!(note_shapes(&vml), 0, "{vml}"); + let (back, _) = read_xlsx(&out).unwrap(); + let sh = back.sheet(0).unwrap(); + assert_eq!(sh.controls.len(), 1); + assert_eq!(sh.controls[0].text, "ObjectType=\"Note\""); + out = write_xlsx(&back).unwrap(); + } +} + +/// Kept VML leaves out the modelled controls by the number the reader gave them: the `id`, else +/// the `o:spid`, wherever either comes in the shape's tag. A button whose `o:spid` comes before a +/// different `id`, and one with a plain numeric `id`, are written once, from the model; a check +/// box whose `o:spid` is a modelled button's number but whose `id` is its own is kept; a button +/// with no number isn't modelled and is kept; notes are dropped whatever their numbers. +#[test] +fn kept_vml_leaves_out_modelled_controls_by_the_readers_shape_number() { + let bytes = with_vml(concat!( + r##"
A
2, 0, 1, 0, 4, 0, 2, 0"##, + r##"
B
2, 0, 3, 0, 4, 0, 4, 0"##, + r##"
Paid
2, 0, 5, 0, 4, 0, 6, 0"##, + r##"
C
2, 0, 7, 0, 4, 0, 8, 0"##, + r##"00"##, + r##"10"##, + )); + let (wb, report) = read_xlsx(&bytes).unwrap(); + let texts: Vec<&str> = wb.sheet(0).unwrap().controls.iter().map(|c| c.text.as_str()).collect(); + assert_eq!(texts, ["A", "B"], "{:?}", report.warnings); + let mut out = write_xlsx(&wb).unwrap(); + for _ in 0..2 { + let vml = only_vml(&out); + for caption in [">A
", ">B", ">C", ">Paid"] { + assert_eq!(vml.matches(caption).count(), 1, "each control once, kept or written: {caption} {vml}"); + } + assert_eq!(vml.matches("ObjectType=\"Checkbox\"").count(), 1, "the check box is kept: {vml}"); + assert_eq!(note_shapes(&vml), 0, "{vml}"); + let (back, _) = read_xlsx(&out).unwrap(); + assert_eq!(back.sheet(0).unwrap().controls.len(), 2); + out = write_xlsx(&back).unwrap(); + } +} diff --git a/crates/xlsx/src/tests/passthrough_tests.rs b/crates/xlsx/src/tests/passthrough_tests.rs index 28113be0d..a71b34422 100644 --- a/crates/xlsx/src/tests/passthrough_tests.rs +++ b/crates/xlsx/src/tests/passthrough_tests.rs @@ -849,6 +849,10 @@ const LOGO: &[u8] = b"\x89PNG\r\n\x1a\n letterhead logo"; const SIGNATURE: &[u8] = b"\x89PNG\r\n\x1a\n footer signature"; const HF_VML: &str = r##""##; +/// The VML drawing of one note and nothing else. [`VML`] also holds a check box that only the +/// drawing lists, which is kept with the drawing (and its source shape block). +const NOTE_VML: &str = r##"
old note
00"##; + /// #1257: a letterhead in the header and a picture in the footer (`&G`), drawn from a VML part /// that `` names, next to a note whose VML drawing GridCraft rewrites. fn header_picture_package() -> Vec { @@ -875,7 +879,7 @@ fn header_picture_package() -> Vec { ("xl/worksheets/sheet1.xml", sheet.as_bytes()), ("xl/worksheets/_rels/sheet1.xml.rels", sheet_rels.as_bytes()), ("xl/comments1.xml", COMMENTS.as_bytes()), - ("xl/drawings/vmlDrawing1.vml", VML.as_bytes()), + ("xl/drawings/vmlDrawing1.vml", NOTE_VML.as_bytes()), ("xl/drawings/vmlDrawing2.vml", HF_VML.as_bytes()), ("xl/drawings/_rels/vmlDrawing2.vml.rels", hf_rels.as_bytes()), ("xl/media/image1.png", LOGO), @@ -938,3 +942,64 @@ fn notes_take_shape_blocks_the_header_pictures_dont_use() { } assert!(notes_ids.iter().all(|b| notes_map.contains(b)), "note ids outside their drawing's blocks"); } + +#[test] +fn modelled_form_controls_join_kept_ones_in_one_controls_element() { + use gridcraft_model::{Anchor, ControlKind, FormControl}; + let (mut wb, _) = read_xlsx(&rich_package()).unwrap(); + assert!(wb.sheet(0).unwrap().controls.is_empty(), "a check box stays with pass-through"); + let anchor = Anchor { cell: gridcraft_core::CellRef::new(8, 1), width: 120.0, height: 20.0, ..Default::default() }; + let slider = FormControl { id: 900, kind: ControlKind::Slider, anchor, link: Some("$B$9".into()), value: 40, ..Default::default() }; + if let Some(sh) = wb.sheet_mut(0) { + sh.controls.push(slider); + } + let out = write_xlsx(&wb).unwrap(); + let u = Unpacked::new(&out); + u.assert_valid(); + let sheet = sheet_part(&u, "Data"); + let sx = u.text(&sheet); + assert_eq!(sx.matches(" = sx.split(", pub table_id: u32, @@ -255,6 +259,9 @@ pub fn write_xlsx_named(wb: &Workbook, kind: XlsxKind, book_name: Option<&str>) drawings: 0, tables: 0, comments: 0, + vmls: 0, + vml_block: 0, + ctrl_props: 0, persons: vec![], table_id: 0, dynamic_arrays: false, diff --git a/crates/xlsx/src/xml.rs b/crates/xlsx/src/xml.rs index 5417b9122..d9ff04f51 100644 --- a/crates/xlsx/src/xml.rs +++ b/crates/xlsx/src/xml.rs @@ -213,8 +213,33 @@ pub fn parse(xml: &[u8]) -> Result { parse_streaming(xml, &[], &mut |_| Ok(())) } -/// Parses a part; complete elements named in `split` (below the root) are handed to `cb` and not -/// kept in the returned tree. +/// The first element named `local_name` (namespace prefix dropped) in `xml`, read from its start tag +/// alone: no children, and nothing after the tag is read, so unbalanced markup around it (a VML +/// caption's unclosed `
`) neither hides nor moves it. `None` when there is none, or when the +/// markup before it is malformed. +pub fn first_start(xml: &[u8], local_name: &str) -> Option { + let mut reader = Reader::from_reader(xml); + { + let cfg = reader.config_mut(); + cfg.check_end_names = false; + cfg.allow_unmatched_ends = true; + cfg.allow_dangling_amp = true; + } + let mut buf = Vec::new(); + loop { + match reader.read_event_into(&mut buf).ok()? { + Event::Start(e) | Event::Empty(e) if local(e.name().as_ref()) == local_name => return Some(start_el(&e)), + Event::Eof => return None, + _ => {} + } + buf.clear(); + } +} + +/// Parses a part; complete elements named in `split` that are children or grandchildren of the +/// root (`sst/si`, `worksheet/sheetData/row`) are handed to `cb` and not kept in the returned +/// tree. Deeper elements of the same name (the `xdr:row` of a form control's anchor) stay in the +/// tree. pub fn parse_streaming(xml: &[u8], split: &[&str], cb: &mut dyn FnMut(El) -> Result<(), IoError>) -> Result { let data = to_utf8(xml); let mut reader = Reader::from_reader(data.as_ref()); @@ -245,9 +270,10 @@ pub fn parse_streaming(xml: &[u8], split: &[&str], cb: &mut dyn FnMut(El) -> Res kept: &mut usize, kept_at_start: Option, ) -> Result<(), IoError> { + let shallow = stack.len() <= 2; match stack.last_mut() { Some(parent) => { - if split.contains(&el.name.as_str()) { + if shallow && split.contains(&el.name.as_str()) { cb(el)?; // The element and everything parsed inside it have left the tree. if let Some(k) = kept_at_start { diff --git a/docs/control-protocol.md b/docs/control-protocol.md index 62b9f7354..c4ca5072f 100644 --- a/docs/control-protocol.md +++ b/docs/control-protocol.md @@ -93,12 +93,14 @@ include `window` or `doc` to run it in that window, as with `engine.execute`; th them as their own parameters. `ui.inspect` reports the selected floating object as `selectedObject: {kind, id}` (or `null`), -where `kind` is `chart`, `image`, `shape`, `equation` or `slicer`; the same names go in the `kind` parameter +where `kind` is `chart`, `image`, `shape`, `equation`, `slicer` or `control` (a form button, slider or spin +button); the same names go in the `kind` parameter of `object.delete`, `object.move`, `object.setAnchorMode` and the `arrange.*` commands. An equation is a shape too, so its id works under `shape`; a plain shape's id given as `equation`, a `kind` that is present but not one of these names (`null` included), or an id of no object of the kind, is an error. `arrange.align` and `arrange.group` check every id in `ids` the same way; with fewer than two ids they change nothing. +Form controls are worked with `control.setValue` and `control.press`, and listed by `control.list`. `object.move` and `chart.set` cap `width` and `height` at 10,000 points, or at the object's current size when a file made it larger: such an object keeps its size when it is moved and shrinks by what a resize asks, but does not grow further. diff --git a/docs/file-format-parity.md b/docs/file-format-parity.md index d2660dde0..9e1fa6e30 100644 --- a/docs/file-format-parity.md +++ b/docs/file-format-parity.md @@ -1,6 +1,6 @@ # File-format parity -> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-03 · **Change:** minor (a web page's first table opens as a sheet, also when the page is named `.xls`; earlier: `workbookPr filterPrivacy` kept, and honoured on save; earlier: a workbook's R1C1 reference style, `calcPr refMode`, is kept through open and save; earlier: XLSX page margins are written as their decimals, `0.3` rather than `0.30000001192092896`; earlier: Python cells in `.xlsx` as [MS-XLSX] specifies, `_xlfn._xlws.PY(…)` naming code in the Python part, with cached values, and in GridCraft JSON, with limits on open; earlier: equations in xlsx as Office Math with a marker in `descr` so an edit made in Excel reads back, read back only up to the equation source cap, and a brace with a label and a script on its other side keeps both; Symbol font in PDF; earlier: XLSX reads and writes Page Setup's page order, `pageSetup pageOrder`, and PDF export follows it; earlier: chart data tables: native c:dTable import/export; earlier: XLSX workbook-view selection, normalized tab ratios and sheet-edit regressions; fidelity estimates unchanged; earlier: formulas from Google Sheets' XLSX export open as the original formula instead of `IFERROR(__xludf.DUMMYFUNCTION(…),…)` when GridCraft calculates every function in it, and save as it; `file.exportCsv {values: true}` writes raw values for automation; .xls dialog sheets are skipped instead of read as worksheets; `.xls` import reads each sheet's display options from WINDOW2: right-to-left, gridlines, headings, zeros, formulas; XLSX text boxes and shapes keep their paragraphs' right-to-left direction on save; CSV/TSV/TXT can open with every field kept as text via `file.open {csvAsText}`; formulas over Excel's 8,192 characters, and `%` or call chains past the nesting limit, are parse errors instead of stack overflows when a file is opened; XLSX PivotTables read and write `rowGrandTotals`/`colGrandTotals` the way Excel does, so a table saved with only one grand total opens with the right one; HTML export keeps right-to-left; CSV and text import detects likely Windows-1256 Arabic when invalid UTF-8 has Arabic byte runs; other invalid UTF-8 remains Windows-1252; HTML export places pictures and charts, as inline SVG, at their anchors; earlier: `file.exportCsv {values: true}` writes raw values for automation; .xls dialog sheets are skipped instead of read as worksheets; `.xls` import reads each sheet's display options from WINDOW2: right-to-left, gridlines, headings, zeros, formulas; XLSX text boxes and shapes keep their paragraphs' right-to-left direction on save; CSV/TSV/TXT can open with every field kept as text via `file.open {csvAsText}`; formulas over Excel's 8,192 characters, and `%` or call chains past the nesting limit, are parse errors instead of stack overflows when a file is opened; XLSX PivotTables read and write `rowGrandTotals`/`colGrandTotals` the way Excel does, so a table saved with only one grand total opens with the right one; HTML export keeps right-to-left; CSV and text import detects likely Windows-1256 Arabic when invalid UTF-8 has Arabic byte runs; other invalid UTF-8 remains Windows-1252; HTML export places pictures and charts, as inline SVG, at their anchors; CSV and text open detects each column's date order from values that read only one way, warns when every date is ambiguous, and keeps a column with both kinds as text (#2765)) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 +> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-03 · **Change:** minor (form controls: buttons, scroll bars and spin buttons read and written; earlier: a web page's first table opens as a sheet, also when the page is named `.xls`; earlier: `workbookPr filterPrivacy` kept, and honoured on save; earlier: a workbook's R1C1 reference style, `calcPr refMode`, is kept through open and save; earlier: XLSX page margins are written as their decimals, `0.3` rather than `0.30000001192092896`; earlier: Python cells in `.xlsx` as [MS-XLSX] specifies, `_xlfn._xlws.PY(…)` naming code in the Python part, with cached values, and in GridCraft JSON, with limits on open; earlier: equations in xlsx as Office Math with a marker in `descr` so an edit made in Excel reads back, read back only up to the equation source cap, and a brace with a label and a script on its other side keeps both; Symbol font in PDF; earlier: XLSX reads and writes Page Setup's page order, `pageSetup pageOrder`, and PDF export follows it; earlier: chart data tables: native c:dTable import/export; earlier: XLSX workbook-view selection, normalized tab ratios and sheet-edit regressions; fidelity estimates unchanged; earlier: formulas from Google Sheets' XLSX export open as the original formula instead of `IFERROR(__xludf.DUMMYFUNCTION(…),…)` when GridCraft calculates every function in it, and save as it; `file.exportCsv {values: true}` writes raw values for automation; .xls dialog sheets are skipped instead of read as worksheets; `.xls` import reads each sheet's display options from WINDOW2: right-to-left, gridlines, headings, zeros, formulas; XLSX text boxes and shapes keep their paragraphs' right-to-left direction on save; CSV/TSV/TXT can open with every field kept as text via `file.open {csvAsText}`; formulas over Excel's 8,192 characters, and `%` or call chains past the nesting limit, are parse errors instead of stack overflows when a file is opened; XLSX PivotTables read and write `rowGrandTotals`/`colGrandTotals` the way Excel does, so a table saved with only one grand total opens with the right one; HTML export keeps right-to-left; CSV and text import detects likely Windows-1256 Arabic when invalid UTF-8 has Arabic byte runs; other invalid UTF-8 remains Windows-1252; HTML export places pictures and charts, as inline SVG, at their anchors; earlier: `file.exportCsv {values: true}` writes raw values for automation; .xls dialog sheets are skipped instead of read as worksheets; `.xls` import reads each sheet's display options from WINDOW2: right-to-left, gridlines, headings, zeros, formulas; XLSX text boxes and shapes keep their paragraphs' right-to-left direction on save; CSV/TSV/TXT can open with every field kept as text via `file.open {csvAsText}`; formulas over Excel's 8,192 characters, and `%` or call chains past the nesting limit, are parse errors instead of stack overflows when a file is opened; XLSX PivotTables read and write `rowGrandTotals`/`colGrandTotals` the way Excel does, so a table saved with only one grand total opens with the right one; HTML export keeps right-to-left; CSV and text import detects likely Windows-1256 Arabic when invalid UTF-8 has Arabic byte runs; other invalid UTF-8 remains Windows-1252; HTML export places pictures and charts, as inline SVG, at their anchors; CSV and text open detects each column's date order from values that read only one way, warns when every date is ambiguous, and keeps a column with both kinds as text (#2765)) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 Every format Excel opens or saves, with GridCraft's read and write support, fidelity and tests. Part of [target-app-parity.md](target-app-parity.md). **File formats: ~50% ready, 80–120 h** @@ -76,7 +76,7 @@ past the parser's limit, and a wrapper inside the text. **Saving writes the real | Format | Excel | GridCraft read | GridCraft write | Fidelity / notes | Tests | |---|---|---|---|---|---| -| `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, shared strings (with their Japanese phonetic runs — ふりがな — kept through open + save: `` offsets and readings and `` are written back next to the same string; the table holds each text once, so duplicate `` items whose text is the same but whose readings differ keep only the first reading, with a warning on open), styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, equations (Office Math with a plain-text fallback and the LaTeX source; Office Math from other writers read for a common subset), charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, calc settings, show objects (`workbookPr showObjects`). **Kept unchanged on save (pass-through, `passthrough.rs`):** custom document properties, ribbon customisation and add-in task panes, custom XML, links to external workbooks (never followed or refreshed), data connections, sheet query tables, printer settings (#324), form controls, ActiveX controls, OLE objects and sheet backgrounds (with their VML shapes), and chart sheets (#727: the sheet, drawing, chart, style and colour parts are written back unchanged in their place, and left out when a sheet they plot was deleted or renamed or a worksheet took their name; they can't be edited or created); dropped when their sheet is deleted or their parts point at content GridCraft rewrites. **Not preserved:** slicers and slicer caches, timelines, data model, query tables attached to tables, sheet and workbook `extLst` entries we don't model, chart details; Strict Open XML files keep nothing. Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) A picture used on several sheets is written once (#985). | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | +| `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, shared strings (with their Japanese phonetic runs — ふりがな — kept through open + save: `` offsets and readings and `` are written back next to the same string; the table holds each text once, so duplicate `` items whose text is the same but whose readings differ keep only the first reading, with a warning on open), styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, equations (Office Math with a plain-text fallback and the LaTeX source; Office Math from other writers read for a common subset), charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), form controls (buttons, scroll bars and spin buttons: control properties parts, `` and VML `x:ClientData`; a VML-only control is read too; a button's command, fixed input range or output cell travels in a hidden defined name and comes back only onto the same button while the file still assigns its macro, or none for a command), protection, calc settings, show objects (`workbookPr showObjects`). **Kept unchanged on save (pass-through, `passthrough.rs`):** custom document properties, ribbon customisation and add-in task panes, custom XML, links to external workbooks (never followed or refreshed), data connections, sheet query tables, printer settings (#324), other form controls (check boxes, list boxes, drop-downs; written in the same `` element and VML drawing as the modelled ones, also those only the VML drawing lists, with no `` entry), ActiveX controls, OLE objects and sheet backgrounds (with their VML shapes), and chart sheets (#727: the sheet, drawing, chart, style and colour parts are written back unchanged in their place, and left out when a sheet they plot was deleted or renamed or a worksheet took their name; they can't be edited or created); dropped when their sheet is deleted or their parts point at content GridCraft rewrites. **Not preserved:** slicers and slicer caches, timelines, data model, query tables attached to tables, sheet and workbook `extLst` entries we don't model, chart details; Strict Open XML files keep nothing. Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) A picture used on several sheets is written once (#985). | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | | `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, shared strings (with their Japanese phonetic runs — ふりがな — kept through open + save: `` offsets and readings and `` are written back next to the same string; the table holds each text once, so duplicate `` items whose text is the same but whose readings differ keep only the first reading, with a warning on open), styles, themes, merges, CF, validation, tables, notes (with their shown/hidden state) and threaded comments with replies, resolved state and timestamps (`threadedComments` and `persons` parts, #300), hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, equations (Office Math with a plain-text fallback and the LaTeX source; Office Math from other writers read for a common subset), charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, calc settings, show objects (`workbookPr showObjects`). **Not preserved:** unmodelled parts and extensions (form controls, slicer caches, timelines, external links, OLE objects, custom XML, data model, query tables, chart details). Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) A picture used on several sheets is written once (#985). | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | | `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, shared strings (with their Japanese phonetic runs — ふりがな — kept through open + save: `` offsets and readings and `` are written back next to the same string; the table holds each text once, so duplicate `` items whose text is the same but whose readings differ keep only the first reading, with a warning on open), styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, equations (Office Math with a plain-text fallback and the LaTeX source; Office Math from other writers read for a common subset), charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, Allow Edit Ranges (`protectedRanges` with their password hashes, #307), calc settings, show objects (`workbookPr showObjects`). **Not preserved:** unmodelled parts and extensions (form controls, slicer caches, timelines, external links, OLE objects, custom XML, data model, query tables, chart details). Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) A picture used on several sheets is written once (#985). | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | | `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes (the major/minor `latin`, East Asian `ea` and complex-script `cs` typefaces and the per-script `` lists are kept through a save), merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, equations (Office Math with a plain-text fallback and the LaTeX source; Office Math from other writers read for a common subset), charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, calc settings. **Kept unchanged on save (pass-through, `passthrough.rs`):** custom document properties, ribbon customisation and add-in task panes, custom XML, links to external workbooks (never followed or refreshed), data connections, sheet query tables, printer settings (#324), form controls, ActiveX controls, OLE objects and sheet backgrounds (with their VML shapes), pictures in headers and footers (`&G`, #1257: their VML drawing and images are written back unchanged; they are not shown, and can't be inserted), and chart sheets (#727: the sheet, drawing, chart, style and colour parts are written back unchanged in their place, and left out when a sheet they plot was deleted or renamed or a worksheet took their name; they can't be edited or created); dropped when their sheet is deleted or their parts point at content GridCraft rewrites. **Not preserved:** slicers and slicer caches, timelines, data model, query tables attached to tables, sheet and workbook `extLst` entries we don't model, chart details; Strict Open XML files keep nothing. Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | @@ -158,6 +158,7 @@ refresh. Counted under features (Get & Transform, ~10% ready), not here. | Date | Change | Summary | |---|---|---| +| 2026-11-02 | minor | XLSX: buttons, scroll bars and spin buttons read and written (control properties parts, ``, VML `x:ClientData`); other form controls stay with pass-through; a button's command, fixed input or output cell travels in a hidden defined name and comes back only while the file still assigns the button's macro (or none, for a command) | | 2026-11-03 | minor | SYLK (`.slk`) import: values, R1C1 formulas as A1 (shared formulas too) with cached results, number formats and column widths; import only (#1481) | | 2026-11-03 | minor | Opening a web page (HTML) reads its first table as a sheet, whatever the file is called: a web app's "Export to Excel" saved as `.xls` no longer opens as lines of markup. Values read as in a CSV (or as text with `csvAsText`), `` cells are bold and centred, `` names the sheet, and a page not named `.htm`/`.html` opens with a warning (#3166) | | 2026-11-03 | minor | XLSX `` (Remove personal information from file properties on save) is read and written back, and while it is on GridCraft saves no author (`dc:creator`) and no company, as Excel leaves them empty (#3089, file part) | diff --git a/docs/macros.md b/docs/macros.md index 1cd2cfbb5..586800045 100644 --- a/docs/macros.md +++ b/docs/macros.md @@ -1,6 +1,6 @@ # Macros -> **Last updated:** 2026-11-01 · **Change:** minor (a close that waits on the step editor's question runs in the editor's window, and a further close asked meanwhile waits for the answer or is refused; earlier: closing a workbook or quitting asks about the step editor's unsaved draft first; recording macros and the management commands; Record Macro takes a name, shortcut and description; the macro interface: Automate › Macros, the Macros and Record Macro dialogs, the step editor, shortcuts) +> **Last updated:** 2026-11-02 · **Change:** minor (form buttons run macros, `macro.run` takes `output`, and a button action kept in a hidden name comes back on reopen only while the file still agrees with it; earlier: a close that waits on the step editor's question runs in the editor's window, and a further close asked meanwhile waits for the answer or is refused; earlier: closing a workbook or quitting asks about the step editor's unsaved draft first; recording macros and the management commands; Record Macro takes a name, shortcut and description; the macro interface: Automate › Macros, the Macros and Record Macro dialogs, the step editor, shortcuts) GridCraft macros are typed, declarative lists of commands, not VBA. Every step runs a GridCraft command (the same ids the ribbon, CLI and MCP use) or one of three bounded control steps. A macro @@ -98,12 +98,18 @@ refused rather than read as the header. ## Running -`macro.run {name, inputs?, onConflict?, overwrite?, approve?, timeLimit?}`: +`macro.run {name, inputs?, output?, onConflict?, overwrite?, approve?, timeLimit?}` (`output` +replaces the macro's output anchor for this run, as a form button that sends the output elsewhere +does; `macro.check` and the debugger take it too). The policy and the approval go by the macro as +it is stored; the footprint, the conflict rule and the check of each step work on the cells +`output` names, so data there is protected as at the macro's own anchor. An `output` that isn't +reference text of at most 1,024 characters, or that names no cell (a bad reference, a sheet that +doesn't exist, a cell outside the grid), is refused before anything runs: 1. **Policy.** File commands other than Properties, exports and printing (such as Open, Save and Close), app, window (including New Window, Hide and Unhide), automate and macro commands, - and Undo and Redo, never run in a macro. File › Properties runs, and so do exports and - printing (`file.exportPdf`, `exportCsv`, `exportHtml`, `print`, `printPreview`, `saveBytes`) + Undo and Redo, and pressing a form control (`control.press`), never run in a macro. File › + Properties runs, and so do exports and printing (`file.exportPdf`, `exportCsv`, `exportHtml`, `print`, `printPreview`, `saveBytes`) that only produce bytes. Reading or writing a file (`data.getData`, `data.fromTextCsv`, `insert.picture` or `pageLayout.background` with a `path`, or an export or print with a `path`) needs the command in the macro's `permissions`. Any `path` other than `null` counts, @@ -138,9 +144,9 @@ refused rather than read as the header. do; a step that deletes one of them, or changes the size of a range a Repeat is going over, stops the run. -`macro.check {name, inputs?, onConflict?, overwrite?, timeLimit?}` reports the bound inputs, the -output anchor, any approval needed and the footprint, and puts back everything the dry run -changed. `ok` says whether `macro.run` with the same parameters would go ahead without a question; +`macro.check {name, inputs?, output?, onConflict?, overwrite?, timeLimit?}` reports the bound +inputs, the output anchor (`output` when given), any approval needed and the footprint, and puts +back everything the dry run changed. `ok` says whether `macro.run` with the same parameters would go ahead without a question; when it wouldn't, `errors` says why: the run would fail at a step, a Stop step rolls it back, it would write over data under `ask` or `fail`, shifting can't make room (the check shifts and checks again, then puts everything back), or the run would go past @@ -246,6 +252,35 @@ kept but does not run the macro. {name, to}`. Saving under a name a macro has is refused unless `replace: true` is passed, and renaming to one is refused; names compare ignoring the case of the letters A to Z. A renamed macro is checked against every limit again, its size included. Saving, renaming and deleting are undoable. +Renaming a macro also renames it in the form buttons that run it. + +## Form buttons + +A form button (Insert › Controls › Button, `insert.button`) runs a macro or one command when it is +pressed (`control.press {id, approve?, onConflict?}`). `control.assign {id, macro, input?, +output?}` makes it run a macro, on the selection when it is pressed (`input: "selection"`, the +default) or always on one range (`input: "B2:D9"`), with an optional output cell; `control.assign +{id, command, params}` makes it run a command with fixed parameters. A macro runs through +`macro.run` with its checks; a command goes through the same policy as a macro step and runs, as +a step does, inside the scripts' file sandbox unless it is an approved file command, so a button +in a workbook opened from a file asks before it runs anything. Like a step, a button's command +never opens a web page or a dialog: `link.open` is refused when it is assigned or comes in from +a file, and a command that would open a dialog or a page once it runs (a command given no +parameters opens its dialog) is refused then, with the request dropped and nothing done. +In the window, a press that asks (approve the action, or overwrite data in the macro's way) opens +a prompt whose answer applies only to what it asked about: if the workbook, the sheet, the +button, its action, its macro or the selection changes while the prompt is open (an agent +reassigns the button), the answer runs nothing and the next press asks again. + +In xlsx a button carries its macro as Excel writes it (`macro="[0]!Name"`, a macro run on the +selection). A command, a fixed input range or an output cell has no place there and is kept in a +hidden defined name (`_gc_control_actions_`), keyed by the sheet, the button's place, name and +anchor cell. When the file is opened the hidden name is removed and each kept action goes back +onto its button only when the file still agrees with it: a kept macro action needs the file to +assign that macro still, so an assignment removed in another application stays removed and a +button given another macro keeps that one; a kept command needs the file to assign no macro. A +button whose name, anchor cell or place changed, or which another application removed, gets no +kept action. Macros are saved in the workbook: as a field in GridCraft's JSON format, and in xlsx as hidden defined names (`_gc_macro__`, the macro's JSON split into string literals within Excel's diff --git a/docs/parity-checklist.md b/docs/parity-checklist.md index 5ca7ada2f..4100ec630 100644 --- a/docs/parity-checklist.md +++ b/docs/parity-checklist.md @@ -4,7 +4,7 @@ Each entry is an Excel ribbon or menu command (feature names only). It counts as implemented when GridCraft registers a command with the same id. This measures breadth only; the overall assessment (depth, fidelity, effort) is in [target-app-parity.md](target-app-parity.md). -**Overall: 277 / 292 commands implemented (94.9%).** +**Overall: 281 / 296 commands implemented (94.9%).** | Tab | Implemented | Total | Parity | |---|---|---|---| @@ -18,6 +18,7 @@ Each entry is an Excel ribbon or menu command (feature names only). It counts as | Review | 15 | 18 | 83% | | View | 21 | 21 | 100% | | Automate | 4 | 4 | 100% | +| Developer | 4 | 4 | 100% | | Table Design | 13 | 13 | 100% | | Chart Design | 8 | 8 | 100% | | Format | 3 | 3 | 100% | @@ -334,6 +335,15 @@ Implemented: - `automate.recordActions` Record Actions (Scripting Tools) - `automate.trustScripts` Trust Scripts (Scripting Tools) +## Developer (4 / 4, 100%) + +Implemented: + +- `insert.button` Button (Form Control) (Controls) +- `insert.spinButton` Spin Button (Form Control) (Controls) +- `insert.slider` Scroll Bar (Form Control) (Controls) +- `control.format` Properties (Controls) + ## Table Design (13 / 13, 100%) Implemented: diff --git a/docs/target-app-parity.md b/docs/target-app-parity.md index 3295e6948..c18df8d40 100644 --- a/docs/target-app-parity.md +++ b/docs/target-app-parity.md @@ -1,6 +1,6 @@ # Target-app parity: GridCraft vs Microsoft Excel -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-11-02 · **Change:** minor (insert objects: equations; earlier: what-if and analysis: Solver Sensitivity report; no score change; earlier: chart resizing no longer listed as missing; no score change; earlier: what-if and analysis: Analysis ToolPak has all 19 tools; earlier: page layout and print: Page Setup's page order; no score change; earlier: charts: data tables with XLSX round-trip and print/PDF; no score change; earlier: what-if and analysis: Solver Options; the Insert Function dialog is named as in Excel; no score change; earlier: Solver Load/Save Model and Reset All; automation: macro interface; catalog count 276 / 292, as `cargo xtask parity` measures; automation: typed macros in the engine; several windows: a native window per workbook window, New Window on the same workbook, Arrange All; no score change) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-11-02 · **Change:** minor (form controls; catalog 281 / 296; earlier: insert objects: equations; earlier: what-if and analysis: Solver Sensitivity report; no score change; earlier: chart resizing no longer listed as missing; no score change; earlier: what-if and analysis: Analysis ToolPak has all 19 tools; earlier: page layout and print: Page Setup's page order; no score change; earlier: charts: data tables with XLSX round-trip and print/PDF; no score change; earlier: what-if and analysis: Solver Options; the Insert Function dialog is named as in Excel; no score change; earlier: Solver Load/Save Model and Reset All; automation: macro interface; catalog count 276 / 292, as `cargo xtask parity` measures; automation: typed macros in the engine; several windows: a native window per workbook window, New Window on the same workbook, Arrange All; no score change) · **Target:** Microsoft Excel (Microsoft 365) This is the authoritative parity assessment. [`ROADMAP.md`](../ROADMAP.md) summarizes it, [`gaps.md`](gaps.md) lists every shortfall, and the area checklists hold the detail: @@ -13,7 +13,7 @@ generated ribbon checklist is [`parity-checklist.md`](parity-checklist.md). | Number | Value | Kind | |---|---|---| | Feature breadth (weighted by use) | **~75%** | estimated from the area table below; anchored on two measurements | -| Ribbon and menu catalog | **277 / 292 (94.9%)** | measured: `cargo xtask parity` (`crates/engine/src/catalog.rs` ∩ registered command ids) | +| Ribbon and menu catalog | **281 / 296 (94.9%)** | measured: `cargo xtask parity` (`crates/engine/src/catalog.rs` ∩ registered command ids) | | Worksheet functions, by name | **513 / 523 (98.1%)** | measured: registered functions vs Microsoft's published function list (method below) | | Worksheet functions that do real work | **504 / 523 (96.4%)** | measured: excludes 9 stubs (7 CUBE functions, WEBSERVICE, FILTERXML) | | **Ready for real work** | **~50%** | estimated; written weights below (51.6 computed) | @@ -54,11 +54,12 @@ long `.xls` track, so about half parallelizes (≈30–50 wall-clock hours with COPILOT and PY: **523 functions**. Ours: every `f!(…)` spec in `crates/functions/src/*.rs` plus `SPECIAL_FUNCTIONS` in `crates/calc/src/eval.rs` (evaluator special forms), minus the test-only `ADD2`: **505**, all of them Excel names. -- **Ribbon catalog:** `crates/engine/src/catalog.rs` (292 unique ids across 15 tabs) against the - 313 command ids registered in `crates/engine/src/cmd/*.rs`, recomputed from source by script and +- **Ribbon catalog:** `crates/engine/src/catalog.rs` (296 unique ids across 16 tabs) against the + 321 command ids registered in `crates/engine/src/cmd/*.rs`, recomputed from source by script and identical to the committed `parity-checklist.md`. The catalog is our own transcription of Excel's main ribbon tabs; it leaves out most contextual tabs (Picture Format, Shape Format, - Header & Footer, Sparkline, Slicer, Timeline), the Developer and Help tabs and sub-menu items, so + Header & Footer, Sparkline, Slicer, Timeline), most of the Developer tab (its form controls are + in), the Help tab and sub-menu items, so 89% of it is roughly **60–65% of Excel's full command surface** (estimated). - **Depth and readiness:** source reading (the model types in `crates/model/src/features.rs`, readers and writers in `crates/xlsx/src`, the evaluator), 478 `#[test]` functions, and the @@ -91,9 +92,9 @@ performance 8, stability 8, localization 4, hardware 2, platforms 2, ecosystem 2 | Page layout, print, PDF | 5 | ~80% | ~55% | 12–20 | Page setup, print titles, breaks, page order, Page Break Preview, headers/footers, PDF export and printing. No true Page Layout view; the PDF writer uses only the 14 standard fonts without embedding (no CJK or other non-Latin text in PDF); web print only exports (#141, #143) | | What-if and analysis | 4 | ~50% | ~40% | 22–35 | Goal Seek, Scenario Manager, data tables, Analysis ToolPak (all 19 tools, no charts), Solver (Simplex LP, a local nonlinear method and Evolutionary; Answer, Sensitivity and Limits reports; Load/Save Model, Reset All and Solver Options; no Population report), Forecast Sheet. Missing: Analyze Data | | Get & Transform, external data, data model | 5 | ~15% | ~10% | 60–100 | Import of CSV/TSV, JSON, HTML. No Power Query editor, no From Web, database or folder sources, no connections or refresh, no relationships or data model, no external workbook links (dropped with a warning) | -| Insert objects | 3 | ~55% | ~45% | 25–40 | Pictures, 8 shape kinds (Excel has ~170), text boxes, own icon set, ink with Ink to Shape, hyperlinks, checkboxes, equations (LaTeX with a live preview, Office Math in xlsx, Symbol font in PDF). Missing: SmartArt, WordArt, signature line, objects, 3-D models, screenshot, pictures in cells (PR #105) | +| Insert objects | 3 | ~55% | ~45% | 25–40 | Pictures, 8 shape kinds (Excel has ~170), text boxes, own icon set, ink with Ink to Shape, hyperlinks, checkboxes, form buttons, scroll bars and spin buttons, equations (LaTeX with a live preview, Office Math in xlsx, Symbol font in PDF). Missing: SmartArt, WordArt, signature line, objects, 3-D models, screenshot, pictures in cells (PR #105) | | Review, comments, protection, collaboration | 4 | ~55% | ~40% | 35–55 | Threaded comments, notes, spelling (system dictionary), thesaurus, sheet and workbook protection, Allow Edit Ranges with Excel-compatible password hashes (#307). Missing: comment navigation, Show Changes, notes-to-comments conversion, accessibility checker, co-authoring, opening password-encrypted files (refused with a message since #17) | -| Automation | 4 | ~30% | ~20% | 12–20 | Recorded, editable GridCraft scripts (Automate tab). Typed macros in the engine (`macro.*`: recorder, runner with input checks and a footprint dry run, debugger; [macros.md](macros.md)), recorded from Automate › Macros, edited and debugged in a step editor, and run from the Macros dialog or a Ctrl+Shift shortcut. VBA is out of scope by owner decision: an `.xlsm` keeps its VBA project unchanged on save and never runs it. No Office Scripts compatibility | +| Automation | 4 | ~30% | ~20% | 12–20 | Recorded, editable GridCraft scripts (Automate tab). Typed macros in the engine (`macro.*`: recorder, runner with input checks and a footprint dry run, debugger; [macros.md](macros.md)), recorded from Automate › Macros, edited and debugged in a step editor, and run from the Macros dialog, a Ctrl+Shift shortcut or a form button (which can also run one command). VBA is out of scope by owner decision: an `.xlsm` keeps its VBA project unchanged on save and never runs it. No Office Scripts compatibility | | **Weighted** | 100 | **~75%** | **~59%** | **300–485** | | ## Dimensions @@ -334,6 +335,7 @@ The inventory carried over from the 2026-10-07 ROADMAP.md, updated for what land | Date | Change | Summary | |---|---|---| +| 2026-11-02 | minor | Form controls (buttons, scroll bars, spin buttons) in Insert objects and Automation; catalog 277 / 292 → 281 / 296 (Developer › Controls) | | 2026-11-02 | minor | Insert objects: equations; catalog count 277 / 292 (no score change) | | 2026-11-02 | minor | What-if and analysis: Solver Sensitivity report (no score change) | | 2026-11-02 | minor | Charts move and resize from eight handles like every floating object (#190): the Essentials chart work is polish only (5–8 h), and the chart rows and the interaction discount no longer list resizing as missing; the Mainstream chart row no longer lists trendlines as missing (no score change) | diff --git a/docs/ui-parity.md b/docs/ui-parity.md index 4f56e81d7..e68946466 100644 --- a/docs/ui-parity.md +++ b/docs/ui-parity.md @@ -1,10 +1,10 @@ # UI parity -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-03 · **Change:** minor (Highlight the active cell's row, a global preference in File › Options › General; earlier: a linked cell's menu edits, opens or removes the link; earlier: Display options for this workbook: sheet tabs and scroll bars; earlier: dragging over row or column headers in point mode extends the reference; earlier: a picture on the system clipboard pastes as a floating picture at the active cell with Cmd/Ctrl+V or Paste (#699); earlier: frozen columns keep their text, gridlines and merged borders whole while the sheet scrolls (#2544); earlier: every Paste Special option has its access key; earlier: the Draw tab's live stroke preview uses the pen's colour and thickness; earlier: copy quotes only cells with a tab or line break, as Excel does; earlier: Ctrl+. Next Corner of Selection; earlier: dragging the selection's border onto data asks before replacing it; earlier: RefEdit reference boxes in dialogs: Goal Seek, Solver and Data Analysis use them, the arrow keys select while a dialog is collapsed and F4 cycles a reference's anchors; earlier: Python cells: Python mode in the formula bar; earlier: Insert › Equation and the equation editor; earlier: every floating object selects, resizes from eight handles and deletes with the keyboard; earlier: Formulas › Error Checking › Trace Error; trace arrows stay through a zoom (#3097); earlier: Page Setup sets the page order, Down, then over or Over, then down, which printing, the print preview, Page Break Preview and Page Layout view follow; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and the Insert Function categories; Esc leaves a File-page text field before closing the view; earlier: checkbox cells print as boxes; Ctrl+arrow and Ctrl+Shift+arrow while pointing a formula reference; editing a hyperlink loads its existing fields; AutoSum over a selected row puts its total to the right; ribbon catalog count 276 / 292) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-03 · **Change:** minor (form controls: buttons, sliders and spin buttons on the sheet; earlier: Highlight the active cell's row, a global preference in File › Options › General; earlier: a linked cell's menu edits, opens or removes the link; earlier: Display options for this workbook: sheet tabs and scroll bars; earlier: dragging over row or column headers in point mode extends the reference; earlier: a picture on the system clipboard pastes as a floating picture at the active cell with Cmd/Ctrl+V or Paste (#699); earlier: frozen columns keep their text, gridlines and merged borders whole while the sheet scrolls (#2544); earlier: every Paste Special option has its access key; earlier: the Draw tab's live stroke preview uses the pen's colour and thickness; earlier: copy quotes only cells with a tab or line break, as Excel does; earlier: Ctrl+. Next Corner of Selection; earlier: dragging the selection's border onto data asks before replacing it; earlier: RefEdit reference boxes in dialogs: Goal Seek, Solver and Data Analysis use them, the arrow keys select while a dialog is collapsed and F4 cycles a reference's anchors; earlier: Python cells: Python mode in the formula bar; earlier: Insert › Equation and the equation editor; earlier: every floating object selects, resizes from eight handles and deletes with the keyboard; earlier: Formulas › Error Checking › Trace Error; trace arrows stay through a zoom (#3097); earlier: Page Setup sets the page order, Down, then over or Over, then down, which printing, the print preview, Page Break Preview and Page Layout view follow; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and the Insert Function categories; Esc leaves a File-page text field before closing the view; earlier: checkbox cells print as boxes; Ctrl+arrow and Ctrl+Shift+arrow while pointing a formula reference; editing a hyperlink loads its existing fields; AutoSum over a selected row puts its total to the right; ribbon catalog count 276 / 292) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 Interaction, controls and feel, against Excel. Part of [target-app-parity.md](target-app-parity.md). **UI/UX fidelity: ~55% ready, 50–80 h** (estimated). Ribbon command coverage is measured -separately in [parity-checklist.md](parity-checklist.md) (277 / 292). +separately in [parity-checklist.md](parity-checklist.md) (281 / 296). Sources: `crates/ui-egui/src` (ribbon, grid, editor, formula bar, dialogs, panes), `crates/engine/src/cmd` (shortcuts in command specs), the owner's black-box notes @@ -73,6 +73,7 @@ this logic, with one undo step. | Copy as HTML table for other apps | done (#36, 10-10) | | Context menu on cells | 16 items (Cut, Copy, Paste, Paste Special, Insert, Delete, Clear Contents, Filter by value, Sort, New Comment, New Note, Format Cells, Pick From Drop-down List, Define Name, Link). Insert Copied Cells (Insert Cut Cells for whole rows/columns) while the clipboard holds a copy (#21). Show Details on a PivotTable's value, subtotal or grand total cell (`pivot.showDetails`). Missing: Insert Cut Cells for a cell block, paste-option icons, Quick Analysis, Get Data from Table, Show Changes, Smart Lookup | | Move and resize charts, pictures and shapes | done (#190, #213): charts, pictures, shapes, equations and slicers select with a frame of eight handles; each handle resizes only its own edges (they never cross), the body moves the object, and the pointer shows a resize or move cursor; Delete or Backspace deletes the selected object and Escape drops the selection; double-click edits a text box, opens Format Chart for a chart. Rotation handle, Shift for proportional resizing, Alt to snap to cells and multi-object selection missing | +| Form controls: buttons, scroll bars (sliders), spin buttons | done (10-31): Insert › Controls (Button, Slider, Spin Button; Excel has them under Developer › Insert) places one by a drag, or at its default size by a click; a click works a control: drag the thumb, page on the track, click or hold an arrow (a drag or a hold is one undo step), press a button, which runs a macro or a command and asks first in a workbook opened from a file (an answer runs only the action it asked about: if an agent changes the button, its macro or the selection while the prompt is open, nothing runs); the control last worked takes the arrows, Page Up and Page Down, Home, End, and Space or Enter (a button gives the keyboard back as soon as the grid takes a key or a right-click; Space on a slider or spin button does what it does with no control focused: it toggles a checkbox cell or starts a cell edit like any other character); Ctrl or Cmd+click or a right-click selects it to move or resize; its menu has Edit Text, Assign Action, Format Control and Delete. Check boxes, option buttons, list boxes and drop-downs (form controls) are kept from files but not shown | | Context menus on row/column headers, sheet tabs, charts, shapes | headers and tabs done; objects partial | | Quick Analysis (Ctrl+Q) | missing | | Smooth scrolling, scrollbars | done; vertical scrollbar added 10-09 (#73); horizontal scrollbar in the tab bar (60/40 split, drag, track click pages) 10-13 (#176, #840); dragging the handle at the scrollbar's left edge gives the tabs more or less of the bar; each workbook keeps its share while it is open (not saved in the file) | @@ -241,6 +242,7 @@ scrollbar bugs 2–4 · pixel/feel pass against observed Excel 7–12. | Date | Change | Summary | |---|---|---| +| 2026-11-02 | minor | Form controls: buttons, sliders (scroll bars) and spin buttons placed, worked, selected and edited on the sheet (Insert › Controls, Assign Action, Format Control); catalog 281 / 296 | | 2026-11-03 | minor | Highlight the active cell's row: a global preference that tints the whole row, off by default | | 2026-11-03 | minor | The formula bar's bottom edge is a resize handle: dragging it sets the bar's height in whole lines, up to half the window, and dragging it to one line collapses it; expanding (the chevron, Ctrl+Shift+U) returns to the dragged height, which is kept with the UI preferences. `view.expandFormulaBar {lines}` sets it (#1246) | | 2026-11-03 | minor | Cell context menu on a cell with a link: Edit Hyperlink… (the link dialog, titled Edit Hyperlink and filled with the link), Open Hyperlink and Remove Hyperlink, as in Excel, instead of Link… (#2120) |